mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-16 18:05:31 +02:00
v1.84.1.0 fix: default Codex and Claude to frontier models (#2835)
* fix: default cross-model workflows to frontier models * chore: bump version and changelog (v1.82.1.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: repair frontier eval budgets and workflow instructions Preserve frontier models and quality thresholds while fixing truncated judge output, ordered section expansion, consent checks, QA scoring, and ship audit gates. Add regression coverage and refresh generated docs. Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: resolve workflow gaps exposed by frontier evals Clarify plan-review ordering and fallback modes, preserve deploy readiness gates, honor configured merge methods, correct benchmark and canary contracts, and restore vendored installs on setup failure. Cover recovery with real-shell regressions. Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: use agent capture budgets for deploy evals Multi-turn deploy and benchmark sessions were incorrectly limited to the single-call judge timeout. Use the existing capture tier and leave outer-test cleanup headroom, with a free policy regression test. Keep all behavioral assertions and frontier models unchanged. Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: clarify retro workflow and evaluate compare instructions Include compare mode in the frontier judge excerpt, define metric sources and snapshot ordering, and preserve the existing prompt-size budget. Co-authored-by: OpenAI Codex <noreply@openai.com> * fix: make documentation release review and publication consistent Review before commit, clarify changelog safeguards and unavailable reviewer modes, and preserve raw PR bodies across separate shell calls. Keep title sync in one shell and add regression coverage. Co-authored-by: OpenAI Codex <noreply@openai.com> --------- Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Garry Tan
co-authored by
OpenAI Codex
OpenAI Codex
parent
c8f0c4e368
commit
71f6048e8a
+28
-27
@@ -927,6 +927,34 @@ Present these approach options via AskUserQuestion using the preamble's AskUserQ
|
||||
**STOP.** AskUserQuestion once per issue. Do NOT batch. Recommend + WHY. Do NOT proceed to Step 0D or 0F until the user responds to 0C-bis. A "clearly winning approach" is still an approach decision and still needs explicit user approval before it lands in the plan.
|
||||
**Reminder: Do NOT make any code changes. Review only.**
|
||||
|
||||
### 0F. Mode Selection
|
||||
Run after 0C-bis and before 0D; labels remain stable for cross-references.
|
||||
In every mode, you are 100% in control. No scope is added without your explicit approval.
|
||||
|
||||
Present four options:
|
||||
1. **SCOPE EXPANSION:** The plan is good but could be great. Dream big — propose the ambitious version. Every expansion is presented individually for your approval. You opt in to each one.
|
||||
2. **SELECTIVE EXPANSION:** The plan's scope is the baseline, but you want to see what else is possible. Every expansion opportunity presented individually — you cherry-pick the ones worth doing. Neutral recommendations.
|
||||
3. **HOLD SCOPE:** The plan's scope is right. Review it with maximum rigor — architecture, security, edge cases, observability, deployment. Make it bulletproof. No expansions surfaced.
|
||||
4. **SCOPE REDUCTION:** The plan is overbuilt or wrong-headed. Propose a minimal version that achieves the core goal, then review that.
|
||||
|
||||
Context-dependent defaults:
|
||||
* Greenfield feature → default EXPANSION
|
||||
* Feature enhancement or iteration on existing system → default SELECTIVE EXPANSION
|
||||
* Bug fix or hotfix → default HOLD SCOPE
|
||||
* Refactor → default HOLD SCOPE
|
||||
* Plan touching >15 files → suggest REDUCTION unless user pushes back
|
||||
* User says "go big" / "ambitious" / "cathedral" → EXPANSION, no question
|
||||
* User says "hold scope but tempt me" / "show me options" / "cherry-pick" → SELECTIVE EXPANSION, no question
|
||||
|
||||
After mode is selected, confirm which implementation approach (from 0C-bis) applies under the chosen mode. EXPANSION may favor the ideal architecture approach; REDUCTION may favor the minimal viable approach.
|
||||
|
||||
Once selected, commit fully. Do not silently drift.
|
||||
|
||||
Present these mode options via AskUserQuestion using the preamble's AskUserQuestion Format section: include RECOMMENDATION. These options differ in kind (review posture), not coverage — do NOT emit `Completeness: N/10` per option. Include the one-line note from step 4 of the preamble format rule instead: `Note: options differ in kind, not coverage — no completeness score.`
|
||||
|
||||
**STOP.** Unless the user already explicitly selected a mode, ask via AskUserQuestion and wait for their choice. Then continue to 0D-prelude, 0D, 0D-POST, and 0E as applicable.
|
||||
**Reminder: Do NOT make any code changes. Review only.**
|
||||
|
||||
### 0D-prelude. Expansion Framing (shared by EXPANSION and SELECTIVE EXPANSION)
|
||||
|
||||
Every expansion proposal you generate in SCOPE EXPANSION or SELECTIVE EXPANSION mode follows this framing pattern:
|
||||
@@ -1093,33 +1121,6 @@ both scales when discussing effort.
|
||||
|
||||
Surface these as questions for the user NOW, not as "figure it out later."
|
||||
|
||||
### 0F. Mode Selection
|
||||
In every mode, you are 100% in control. No scope is added without your explicit approval.
|
||||
|
||||
Present four options:
|
||||
1. **SCOPE EXPANSION:** The plan is good but could be great. Dream big — propose the ambitious version. Every expansion is presented individually for your approval. You opt in to each one.
|
||||
2. **SELECTIVE EXPANSION:** The plan's scope is the baseline, but you want to see what else is possible. Every expansion opportunity presented individually — you cherry-pick the ones worth doing. Neutral recommendations.
|
||||
3. **HOLD SCOPE:** The plan's scope is right. Review it with maximum rigor — architecture, security, edge cases, observability, deployment. Make it bulletproof. No expansions surfaced.
|
||||
4. **SCOPE REDUCTION:** The plan is overbuilt or wrong-headed. Propose a minimal version that achieves the core goal, then review that.
|
||||
|
||||
Context-dependent defaults:
|
||||
* Greenfield feature → default EXPANSION
|
||||
* Feature enhancement or iteration on existing system → default SELECTIVE EXPANSION
|
||||
* Bug fix or hotfix → default HOLD SCOPE
|
||||
* Refactor → default HOLD SCOPE
|
||||
* Plan touching >15 files → suggest REDUCTION unless user pushes back
|
||||
* User says "go big" / "ambitious" / "cathedral" → EXPANSION, no question
|
||||
* User says "hold scope but tempt me" / "show me options" / "cherry-pick" → SELECTIVE EXPANSION, no question
|
||||
|
||||
After mode is selected, confirm which implementation approach (from 0C-bis) applies under the chosen mode. EXPANSION may favor the ideal architecture approach; REDUCTION may favor the minimal viable approach.
|
||||
|
||||
Once selected, commit fully. Do not silently drift.
|
||||
|
||||
Present these mode options via AskUserQuestion using the preamble's AskUserQuestion Format section: include RECOMMENDATION. These options differ in kind (review posture), not coverage — do NOT emit `Completeness: N/10` per option. Include the one-line note from step 4 of the preamble format rule instead: `Note: options differ in kind, not coverage — no completeness score.`
|
||||
|
||||
**STOP.** AskUserQuestion once per issue. Do NOT batch. Recommend + WHY. If this section turned up zero findings, state "No issues, moving on" and proceed. If the section has findings, you MUST call AskUserQuestion as a tool_use — a finding with an "obvious fix" is still a finding and still needs user approval before any change lands in the plan. Do NOT proceed until the user responds.
|
||||
**Reminder: Do NOT make any code changes. Review only.**
|
||||
|
||||
> **STOP.** Before running the 11-section deep review, required outputs, and review report (only after Step 0 scope and mode are agreed), Read `~/.claude/skills/gstack/plan-ceo-review/sections/review-sections.md` and execute it
|
||||
> in full. Do not work from memory — that section is the source of truth for this step.
|
||||
|
||||
|
||||
@@ -285,6 +285,34 @@ Present these approach options via AskUserQuestion using the preamble's AskUserQ
|
||||
**STOP.** AskUserQuestion once per issue. Do NOT batch. Recommend + WHY. Do NOT proceed to Step 0D or 0F until the user responds to 0C-bis. A "clearly winning approach" is still an approach decision and still needs explicit user approval before it lands in the plan.
|
||||
**Reminder: Do NOT make any code changes. Review only.**
|
||||
|
||||
### 0F. Mode Selection
|
||||
Run after 0C-bis and before 0D; labels remain stable for cross-references.
|
||||
In every mode, you are 100% in control. No scope is added without your explicit approval.
|
||||
|
||||
Present four options:
|
||||
1. **SCOPE EXPANSION:** The plan is good but could be great. Dream big — propose the ambitious version. Every expansion is presented individually for your approval. You opt in to each one.
|
||||
2. **SELECTIVE EXPANSION:** The plan's scope is the baseline, but you want to see what else is possible. Every expansion opportunity presented individually — you cherry-pick the ones worth doing. Neutral recommendations.
|
||||
3. **HOLD SCOPE:** The plan's scope is right. Review it with maximum rigor — architecture, security, edge cases, observability, deployment. Make it bulletproof. No expansions surfaced.
|
||||
4. **SCOPE REDUCTION:** The plan is overbuilt or wrong-headed. Propose a minimal version that achieves the core goal, then review that.
|
||||
|
||||
Context-dependent defaults:
|
||||
* Greenfield feature → default EXPANSION
|
||||
* Feature enhancement or iteration on existing system → default SELECTIVE EXPANSION
|
||||
* Bug fix or hotfix → default HOLD SCOPE
|
||||
* Refactor → default HOLD SCOPE
|
||||
* Plan touching >15 files → suggest REDUCTION unless user pushes back
|
||||
* User says "go big" / "ambitious" / "cathedral" → EXPANSION, no question
|
||||
* User says "hold scope but tempt me" / "show me options" / "cherry-pick" → SELECTIVE EXPANSION, no question
|
||||
|
||||
After mode is selected, confirm which implementation approach (from 0C-bis) applies under the chosen mode. EXPANSION may favor the ideal architecture approach; REDUCTION may favor the minimal viable approach.
|
||||
|
||||
Once selected, commit fully. Do not silently drift.
|
||||
|
||||
Present these mode options via AskUserQuestion using the preamble's AskUserQuestion Format section: include RECOMMENDATION. These options differ in kind (review posture), not coverage — do NOT emit `Completeness: N/10` per option. Include the one-line note from step 4 of the preamble format rule instead: `Note: options differ in kind, not coverage — no completeness score.`
|
||||
|
||||
**STOP.** Unless the user already explicitly selected a mode, ask via AskUserQuestion and wait for their choice. Then continue to 0D-prelude, 0D, 0D-POST, and 0E as applicable.
|
||||
**Reminder: Do NOT make any code changes. Review only.**
|
||||
|
||||
### 0D-prelude. Expansion Framing (shared by EXPANSION and SELECTIVE EXPANSION)
|
||||
|
||||
Every expansion proposal you generate in SCOPE EXPANSION or SELECTIVE EXPANSION mode follows this framing pattern:
|
||||
@@ -389,33 +417,6 @@ both scales when discussing effort.
|
||||
|
||||
Surface these as questions for the user NOW, not as "figure it out later."
|
||||
|
||||
### 0F. Mode Selection
|
||||
In every mode, you are 100% in control. No scope is added without your explicit approval.
|
||||
|
||||
Present four options:
|
||||
1. **SCOPE EXPANSION:** The plan is good but could be great. Dream big — propose the ambitious version. Every expansion is presented individually for your approval. You opt in to each one.
|
||||
2. **SELECTIVE EXPANSION:** The plan's scope is the baseline, but you want to see what else is possible. Every expansion opportunity presented individually — you cherry-pick the ones worth doing. Neutral recommendations.
|
||||
3. **HOLD SCOPE:** The plan's scope is right. Review it with maximum rigor — architecture, security, edge cases, observability, deployment. Make it bulletproof. No expansions surfaced.
|
||||
4. **SCOPE REDUCTION:** The plan is overbuilt or wrong-headed. Propose a minimal version that achieves the core goal, then review that.
|
||||
|
||||
Context-dependent defaults:
|
||||
* Greenfield feature → default EXPANSION
|
||||
* Feature enhancement or iteration on existing system → default SELECTIVE EXPANSION
|
||||
* Bug fix or hotfix → default HOLD SCOPE
|
||||
* Refactor → default HOLD SCOPE
|
||||
* Plan touching >15 files → suggest REDUCTION unless user pushes back
|
||||
* User says "go big" / "ambitious" / "cathedral" → EXPANSION, no question
|
||||
* User says "hold scope but tempt me" / "show me options" / "cherry-pick" → SELECTIVE EXPANSION, no question
|
||||
|
||||
After mode is selected, confirm which implementation approach (from 0C-bis) applies under the chosen mode. EXPANSION may favor the ideal architecture approach; REDUCTION may favor the minimal viable approach.
|
||||
|
||||
Once selected, commit fully. Do not silently drift.
|
||||
|
||||
Present these mode options via AskUserQuestion using the preamble's AskUserQuestion Format section: include RECOMMENDATION. These options differ in kind (review posture), not coverage — do NOT emit `Completeness: N/10` per option. Include the one-line note from step 4 of the preamble format rule instead: `Note: options differ in kind, not coverage — no completeness score.`
|
||||
|
||||
**STOP.** AskUserQuestion once per issue. Do NOT batch. Recommend + WHY. If this section turned up zero findings, state "No issues, moving on" and proceed. If the section has findings, you MUST call AskUserQuestion as a tool_use — a finding with an "obvious fix" is still a finding and still needs user approval before any change lands in the plan. Do NOT proceed until the user responds.
|
||||
**Reminder: Do NOT make any code changes. Review only.**
|
||||
|
||||
{{SECTION:review-sections}}
|
||||
|
||||
## Section self-check (before you finish)
|
||||
|
||||
@@ -303,15 +303,17 @@ Branch on the echoed `CODEX_MODE`:
|
||||
- **`under_codex`** — this session is already running INSIDE a Codex host, so spawning codex again is the same model reviewing itself at multiplied token cost (#2519). Print exactly one line: "[running under Codex — nested codex passes skipped; set GSTACK_FORCE_CODEX_REVIEW=1 to force]" and skip the codex invocations below; run the section's free in-host pass instead if it defines one.
|
||||
- **`not_authed`** — installed but no credentials. Print: "Codex installed but not authenticated — falling back to a Claude subagent (same model family, not an outside model). Run `codex login` or set `$CODEX_API_KEY`." Fall back to the Claude subagent path.
|
||||
- **`broken_install`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: `npm install -g @openai/codex`." Relay the probe's HINT lines and fall back to the Claude subagent path. This state exists because a missing binary used to land in the model probe's fail-open bucket and report `ready`, so every Codex pass was skipped silently (#2742).
|
||||
- **`model_unusable`** — authed but the account cannot use its configured model (#2477: HTTP 400 on every call, usually a stale `model =` pin in `~/.codex/config.toml`). Relay the probe's HINT lines, tell the user the one-line fix (update the pin; `[notice.model_migrations]` names the replacement), and fall back to the Claude subagent path. The ~10s round trip is cached for 1h; timeouts fail open to `ready`.
|
||||
- **`model_unusable`** — authed but the account cannot use gstack's selected Codex model (#2477: HTTP 400 on every call). Relay the probe's HINT lines, tell the user the one-line fix (set `GSTACK_CODEX_MODEL=<supported-model>` or pass an explicit `-c model=...` override), and fall back to the Claude subagent path. The ~10s round trip is cached for 1h; timeouts fail open to `ready`.
|
||||
- **`ready`** — run the Codex pass below.
|
||||
|
||||
When the mode is `ready`, `not_installed`, or `not_authed`, print one line so the off-switch
|
||||
On `under_codex`, no in-host substitute is defined here: skip this outside-voice section and continue to the required outputs. Do not invoke Codex again or label a self-review as independent.
|
||||
|
||||
For all other non-disabled modes (`ready`, `not_installed`, `not_authed`, `broken_install`, `model_unusable`), print one line so the off-switch
|
||||
stays discoverable: "Running the outside voice automatically (standard step). Disable: `gstack-config set codex_reviews disabled`."
|
||||
|
||||
**Construct the plan review prompt** (for `ready`, `not_installed`, and `not_authed` — skip only on `disabled`).
|
||||
**Construct the plan review prompt** for every remaining mode, including all Claude fallback modes (skip on `disabled` or `under_codex`).
|
||||
Read the plan file being reviewed (the file the user pointed this review at, or the branch
|
||||
diff scope). If a CEO plan document was written in Step 0D-POST, read that too — it contains
|
||||
diff scope). If a CEO plan document from an earlier `/plan-ceo-review` Step 0D-POST is available, read that too — it contains
|
||||
the scope decisions and vision.
|
||||
|
||||
Construct this prompt (substitute the actual plan content — if plan content exceeds 30KB,
|
||||
@@ -335,7 +337,7 @@ THE PLAN:
|
||||
```bash
|
||||
TMPERR_PV=$(mktemp /tmp/codex-planreview-XXXXXXXX)
|
||||
_REPO_ROOT=$(git rev-parse --show-toplevel) || { echo "ERROR: not in a git repo" >&2; exit 1; }
|
||||
codex exec "<prompt>" -C "$_REPO_ROOT" -s read-only -c 'model_reasoning_effort="high"' -c 'web_search="cached"' < /dev/null 2>"$TMPERR_PV"
|
||||
codex exec "<prompt>" -C "$_REPO_ROOT" -s read-only -c "model=\"${GSTACK_CODEX_MODEL:-gpt-6-astra}\"" -c 'model_reasoning_effort="high"' -c 'web_search="cached"' < /dev/null 2>"$TMPERR_PV"
|
||||
```
|
||||
|
||||
Use a 5-minute timeout (`timeout: 300000`). After the command completes, read stderr:
|
||||
@@ -357,7 +359,7 @@ CODEX SAYS (plan review — outside voice):
|
||||
- Timeout: "Codex timed out after 5 minutes." Fall back to the Claude subagent below.
|
||||
- Empty response: "Codex returned no response." Fall back to the Claude subagent below.
|
||||
|
||||
**If `CODEX_MODE: not_installed` or `not_authed` (or Codex errored at runtime):**
|
||||
**If `CODEX_MODE: not_installed`, `not_authed`, `broken_install`, or `model_unusable` (or Codex errored at runtime):**
|
||||
|
||||
Dispatch via the Agent tool with `run_in_background: false` (subagents default to background since Claude Code v2.1.198; the findings must land before the workflow continues). The subagent has fresh context and no conversation bias — but it is the SAME model family, not an outside model; weigh its agreement accordingly.
|
||||
Bound it the same way as Codex: cap the dispatch at a 5-minute timeout so "never blocking"
|
||||
|
||||
Reference in New Issue
Block a user