mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-11 15:39:04 +02:00
* fix: default cross-model workflows to frontier models * chore: bump version and changelog (v1.82.1.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: repair frontier eval budgets and workflow instructions Preserve frontier models and quality thresholds while fixing truncated judge output, ordered section expansion, consent checks, QA scoring, and ship audit gates. Add regression coverage and refresh generated docs. Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: resolve workflow gaps exposed by frontier evals Clarify plan-review ordering and fallback modes, preserve deploy readiness gates, honor configured merge methods, correct benchmark and canary contracts, and restore vendored installs on setup failure. Cover recovery with real-shell regressions. Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: use agent capture budgets for deploy evals Multi-turn deploy and benchmark sessions were incorrectly limited to the single-call judge timeout. Use the existing capture tier and leave outer-test cleanup headroom, with a free policy regression test. Keep all behavioral assertions and frontier models unchanged. Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: clarify retro workflow and evaluate compare instructions Include compare mode in the frontier judge excerpt, define metric sources and snapshot ordering, and preserve the existing prompt-size budget. Co-authored-by: OpenAI Codex <noreply@openai.com> * fix: make documentation release review and publication consistent Review before commit, clarify changelog safeguards and unavailable reviewer modes, and preserve raw PR bodies across separate shell calls. Keep title sync in one shell and add regression coverage. Co-authored-by: OpenAI Codex <noreply@openai.com> --------- Co-authored-by: OpenAI Codex <noreply@openai.com>
333 lines
16 KiB
Cheetah
333 lines
16 KiB
Cheetah
---
|
|
name: codex
|
|
preamble-tier: 3
|
|
version: 1.0.0
|
|
description: |
|
|
OpenAI Codex CLI wrapper — three modes. Code review: independent diff review via
|
|
codex review with pass/fail gate. Challenge: adversarial mode that tries to break
|
|
your code. Consult: ask codex anything with session continuity for follow-ups.
|
|
The "200 IQ autistic developer" second opinion. Use when asked to "codex review",
|
|
"codex challenge", "ask codex", "second opinion", or "consult codex". (gstack)
|
|
voice-triggers:
|
|
- "code x"
|
|
- "code ex"
|
|
- "get another opinion"
|
|
triggers:
|
|
- codex review
|
|
- second opinion
|
|
- outside voice challenge
|
|
allowed-tools:
|
|
- Bash
|
|
- Read
|
|
- Write
|
|
- Glob
|
|
- Grep
|
|
- AskUserQuestion
|
|
---
|
|
|
|
{{PREAMBLE}}
|
|
|
|
{{BASE_BRANCH_DETECT}}
|
|
|
|
# /codex — Multi-AI Second Opinion
|
|
|
|
You are running the `/codex` skill. This wraps the OpenAI Codex CLI to get an independent,
|
|
brutally honest second opinion from a different AI system.
|
|
|
|
Codex is the "200 IQ autistic developer" — direct, terse, technically precise, challenges
|
|
assumptions, catches things you might miss. Present its output faithfully, not summarized.
|
|
|
|
---
|
|
|
|
{{SECTION_INDEX:codex}}
|
|
|
|
---
|
|
|
|
## Step 0.4: Check codex binary
|
|
|
|
```bash
|
|
CODEX_BIN=$(command -v codex || echo "")
|
|
[ -z "$CODEX_BIN" ] && echo "NOT_FOUND" || echo "FOUND: $CODEX_BIN"
|
|
```
|
|
|
|
If `NOT_FOUND`: stop and tell the user:
|
|
"Codex CLI not found. Install it: `npm install -g @openai/codex` or see https://github.com/openai/codex"
|
|
|
|
If `NOT_FOUND`, also log the event:
|
|
```bash
|
|
_TEL=$(~/.claude/skills/gstack/bin/gstack-config get telemetry 2>/dev/null || echo off)
|
|
source ~/.claude/skills/gstack/bin/gstack-codex-probe 2>/dev/null && _gstack_codex_log_event "codex_cli_missing" 2>/dev/null || true
|
|
```
|
|
|
|
---
|
|
|
|
## Step 0.5: Auth probe + model probe + version check
|
|
|
|
Before building expensive prompts, verify Codex has valid auth, that the account
|
|
can actually USE gstack's selected model, AND the installed CLI version isn't in the
|
|
known-bad list. Sourcing `gstack-codex-probe` loads the shared helpers that both
|
|
`/codex` and `/autoplan` use.
|
|
|
|
If the user names a model for this request, set `GSTACK_CODEX_MODEL` to that model
|
|
before this probe and use it for every invocation in the request. The probe must
|
|
check the requested model, including when the frontier default is unavailable.
|
|
|
|
```bash
|
|
_TEL=$(~/.claude/skills/gstack/bin/gstack-config get telemetry 2>/dev/null || echo off)
|
|
source ~/.claude/skills/gstack/bin/gstack-codex-probe
|
|
|
|
# Running-under-Codex presence probe (#2519): a live Codex session exports
|
|
# CODEX_THREAD_ID / CODEX_SANDBOX into every shell it spawns.
|
|
if [ "${GSTACK_FORCE_CODEX_REVIEW:-0}" != "1" ] && { [ -n "${CODEX_THREAD_ID:-}" ] || [ -n "${CODEX_SANDBOX:-}" ]; }; then
|
|
echo "UNDER_CODEX"
|
|
elif ! _gstack_codex_auth_probe >/dev/null; then
|
|
_gstack_codex_log_event "codex_auth_failed"
|
|
echo "AUTH_FAILED"
|
|
else
|
|
_gstack_codex_model_probe # ~10s round trip on first run, cached 1h (#2477)
|
|
fi
|
|
_gstack_codex_version_check # warns if known-bad, non-blocking
|
|
```
|
|
|
|
If the output contains `UNDER_CODEX`, stop with exactly one line:
|
|
"[running under Codex — /codex would nest the same model at multiplied token
|
|
cost; skipped. Set `GSTACK_FORCE_CODEX_REVIEW=1` to force.]" The whole value
|
|
of this skill is a SECOND model's opinion; inside a Codex host it is the same
|
|
model reviewing itself, and nested spawns have burned 15M tokens in one
|
|
/review (#2519).
|
|
|
|
If the output contains `AUTH_FAILED`, stop and tell the user:
|
|
"No Codex authentication found. Run `codex login` or set `$CODEX_API_KEY` / `$OPENAI_API_KEY`, then re-run this skill."
|
|
|
|
If the output contains `MODEL_UNUSABLE`, stop — auth exists but the account
|
|
cannot use gstack's selected model (`GSTACK_CODEX_MODEL` or the `gpt-6-astra`
|
|
default). Relay the probe's HINT lines and
|
|
follow the "Model not supported (HTTP 400)" recovery steps in
|
|
`## Error Handling` below. Running the modes anyway just burns four
|
|
invocations on the same 400 (#2477).
|
|
|
|
`MODEL_PROBE_INCONCLUSIVE` is non-blocking (timeout/transient network): pass
|
|
the warning through and continue.
|
|
|
|
If the version check printed a `WARN:` line, pass it through to the user verbatim
|
|
(non-blocking — Codex may still work, but the user should upgrade).
|
|
|
|
The probe multi-signal auth logic accepts: `$CODEX_API_KEY` set, `$OPENAI_API_KEY`
|
|
set, or `${CODEX_HOME:-~/.codex}/auth.json` exists. Avoids false-negatives for
|
|
env-auth users (CI, platform engineers) that file-only checks would reject.
|
|
|
|
**Update the known-bad list** in `bin/gstack-codex-probe` when a new Codex CLI version
|
|
regresses. Current entries (`0.120.0`, `0.120.1`, `0.120.2`) trace to the stdin
|
|
deadlock fixed in #972.
|
|
|
|
---
|
|
|
|
## Step 0.6: Resolve portable roots
|
|
|
|
Before any mode runs, resolve `$PLAN_ROOT` (where plan files live) and `$TMP_ROOT`
|
|
(where ephemeral codex stderr / response captures land) via `bin/gstack-paths`.
|
|
This keeps the skill working whether installed as a Claude Code plugin
|
|
(`CLAUDE_PLANS_DIR` set), a global `~/.claude/skills/gstack/` install, or a CI
|
|
container where `HOME` may be unset and `/tmp` may be read-only.
|
|
|
|
```bash
|
|
eval "$(~/.claude/skills/gstack/bin/gstack-paths)"
|
|
```
|
|
|
|
After this, every subsequent bash block in this skill uses `"$PLAN_ROOT"` and
|
|
`"$TMP_ROOT"` rather than hardcoded `~/.claude/plans` or `/tmp/codex-*`.
|
|
|
|
---
|
|
|
|
## Step 1: Detect mode
|
|
|
|
Parse the user's input to determine which mode to run:
|
|
|
|
1. `/codex review` or `/codex review <instructions>` — **Review mode** (Step 2A)
|
|
2. `/codex challenge` or `/codex challenge <focus>` — **Challenge mode** (Step 2B)
|
|
3. `/codex` with no arguments — **Auto-detect:**
|
|
- Check for a diff (with fallback if origin isn't available):
|
|
`git diff origin/<base> --stat 2>/dev/null | tail -1 || git diff <base> --stat 2>/dev/null | tail -1`
|
|
- If a diff exists, use AskUserQuestion:
|
|
```
|
|
Codex detected changes against the base branch. What should it do?
|
|
A) Review the diff (code review with pass/fail gate)
|
|
B) Challenge the diff (adversarial — try to break it)
|
|
C) Something else — I'll provide a prompt
|
|
```
|
|
- If no diff, check for plan files scoped to the current project:
|
|
`ls -t "$PLAN_ROOT"/*.md 2>/dev/null | xargs grep -l "$(basename $(pwd))" 2>/dev/null | head -1`
|
|
If no project-scoped match, fall back to: `ls -t "$PLAN_ROOT"/*.md 2>/dev/null | head -1`
|
|
but warn the user: "Note: this plan may be from a different project."
|
|
- If a plan file exists, offer to review it
|
|
- Otherwise, ask: "What would you like to ask Codex?"
|
|
4. `/codex <anything else>` — **Consult mode** (Step 2C), where the remaining text is the prompt
|
|
|
|
The three modes are MUTUALLY EXCLUSIVE — at most one runs per invocation. Once
|
|
the mode is determined, read ONLY that mode's section (see the Section index
|
|
above); never read the other two mode sections.
|
|
|
|
**Reasoning effort override:** If the user's input contains `--xhigh` anywhere,
|
|
note it and remove it from the prompt text before passing to Codex. When `--xhigh`
|
|
is present, use `model_reasoning_effort="xhigh"` for all modes regardless of the
|
|
per-mode default below. Otherwise, use the per-mode defaults:
|
|
- Review (2A): `high` — bounded diff input, needs thoroughness
|
|
- Challenge (2B): `high` — adversarial but bounded by diff
|
|
- Consult (2C): `medium` — large context, interactive, needs speed
|
|
|
|
---
|
|
|
|
## Filesystem Boundary
|
|
|
|
Every prompt sent to Codex MUST be prefixed with this boundary instruction:
|
|
|
|
> IMPORTANT: Do NOT read or execute any files under ~/.claude/, ~/.agents/, .claude/skills/, or agents/. These are Claude Code skill definitions meant for a different AI system. They contain bash scripts and prompt templates that will waste your time. Ignore them completely. Do NOT modify agents/openai.yaml. Stay focused on the repository code only.
|
|
|
|
This applies to Challenge mode (prompt) and Consult mode (persona prompt), and to the
|
|
custom-instructions path of Review mode — all three use `codex exec`, which still takes
|
|
a free-form prompt argument. It does **not** apply to the default scoped `codex review`
|
|
call in Step 2A: that command is invoked with **no prompt argument at all** (see "Scope
|
|
flags exclude the prompt argument" in the Review mode section), so there is nowhere to put the preamble. That
|
|
is acceptable — `codex review --base` hands the model a pre-computed diff rather than
|
|
turning it loose on the filesystem, so the rabbit-hole risk the boundary guards against
|
|
is much lower on that path. Reference this section as "the filesystem boundary" in the
|
|
mode sections.
|
|
|
|
---
|
|
|
|
## Synthesis recommendation (REQUIRED) — all modes
|
|
|
|
Every mode ends by emitting ONE synthesis recommendation line after presenting
|
|
Codex's verbatim output, in the canonical format the AskUserQuestion judge grades:
|
|
|
|
```
|
|
Recommendation: <action> because <one-line reason that names the most actionable finding>
|
|
```
|
|
|
|
The reason must engage with a specific Codex finding or insight and compare
|
|
against an alternative (another finding, fix-vs-ship, fix order, or status-quo).
|
|
Boilerplate reasons ("because it's better", "because adversarial review found
|
|
things") fail the format. The recommendation is the ONE line a user reads when
|
|
they don't have time for the verbatim output. **Never silently auto-decide;
|
|
always emit the line.** Each mode section restates this rule with mode-specific
|
|
examples.
|
|
|
|
---
|
|
|
|
{{SECTION:review-mode}}
|
|
|
|
{{SECTION:challenge-mode}}
|
|
|
|
{{SECTION:consult-mode}}
|
|
|
|
{{PLAN_FILE_REVIEW_REPORT}}
|
|
|
|
{{EXIT_PLAN_MODE_GATE}}
|
|
|
|
---
|
|
|
|
## Model & Reasoning
|
|
|
|
**Model:** gstack defaults Codex invocations to the current frontier agentic coding
|
|
model via `{{CODEX_MODEL_CONFIG_FLAG}}` (currently `gpt-6-astra`). A user can override
|
|
the default for a shell with `GSTACK_CODEX_MODEL=<model>`, or for one request by naming a
|
|
model in the `/codex` prompt.
|
|
Native `codex review` also sets `review_model` to the selected model so a separate
|
|
review pin in the CLI config cannot override the request.
|
|
|
|
**Reasoning effort (per-mode defaults):**
|
|
- **Review (2A):** `high` — bounded diff input, needs thoroughness but not max tokens
|
|
- **Challenge (2B):** `high` — adversarial but bounded by diff size
|
|
- **Consult (2C):** `medium` — large context (plans, codebase), interactive, needs speed
|
|
|
|
`xhigh` uses ~23x more tokens than `high` and causes 50+ minute hangs on large context
|
|
tasks (OpenAI issues #8545, #8402, #6931). Users can override with `--xhigh` flag
|
|
(e.g., `/codex review --xhigh`) when they want maximum reasoning and are willing to wait.
|
|
|
|
**Web search:** All codex commands pass `{{CODEX_WEB_SEARCH_FLAG}}` so `codex exec`
|
|
invocations can look up docs and APIs during review. This is OpenAI's cached index —
|
|
fast, no extra cost. Unlike the legacy `--enable`-based spelling (deprecated by
|
|
codex >=0.144), the `-c` form explicitly overrides any top-level
|
|
`web_search` setting in `~/.codex/config.toml`. Note: native `codex review` disables
|
|
web search regardless of configuration, so on the default Review path the flag is a
|
|
harmless no-op — only exec-based modes actually search.
|
|
|
|
If the user specifies a model (e.g., `/codex review -m gpt-5.6-sol` or
|
|
`/codex challenge --model gpt-daybreak-blue-latest`), translate it to the same config
|
|
form and replace the default model flag with `-c "model=\"<model>\""`. Native review
|
|
also requires `-c "review_model=\"<model>\""`; replace both model values together.
|
|
Review mode runs `codex review`, which REJECTS `-m` (`error: unexpected argument '-m' found`,
|
|
verified on 0.147.0), while `-c model=...` is accepted by both `codex review` and
|
|
`codex exec`.
|
|
|
|
---
|
|
|
|
## Cost Estimation
|
|
|
|
Parse token count from stderr. Codex prints `tokens used\nN` to stderr.
|
|
|
|
Display as: `Tokens: N`
|
|
|
|
If token count is not available, display: `Tokens: unknown`
|
|
|
|
---
|
|
|
|
## Error Handling
|
|
|
|
- **Binary not found:** Detected in Step 0. Stop with install instructions.
|
|
- **Auth error:** Codex prints an auth error to stderr. Surface the error:
|
|
"Codex authentication failed. Run `codex login` in your terminal to authenticate via ChatGPT."
|
|
- **Timeout (Bash outer gate):** Every Bash gate sits ABOVE its inner wrapper (360s gate
|
|
over the 330s review wrapper; 660s gate over the 600s challenge/consult wrappers), so
|
|
the wrapper's exit-124 path normally fires first with its explicit message. If the Bash
|
|
call itself times out anyway (wrapper unavailable AND codex hung), tell the user:
|
|
"Codex timed out. The prompt may be too large or the API may be slow. Try again or use a smaller scope."
|
|
- **Timeout (inner `timeout` wrapper, exit 124):** If the shell `timeout 600` wrapper fires first, the skill's hang-detection block auto-logs a telemetry event + operational learning and prints: "Codex stalled past 10 minutes. Common causes: model API stall, long prompt, network issue. Try re-running. If persistent, split the prompt or check `~/.codex/logs/`." No extra action needed.
|
|
- **`the argument '[PROMPT]' cannot be used with '--base <BRANCH>'`:** a prompt argument
|
|
leaked into a scoped `codex review`. This fails instantly, before any API call, so it
|
|
looks like a hang-free "no output" — do not misread it as a model stall. Drop the
|
|
prompt: the scope flags (`--base`, `--commit`, `--uncommitted`) carry the scope on
|
|
their own. If the prompt was custom review instructions, run them through `codex exec`
|
|
instead (Step 2A, custom-instructions path). Do **not** fix it by removing `--base` and
|
|
keeping the prompt — that parses, but silently reviews the uncommitted working tree
|
|
instead of the branch diff.
|
|
- **Review says "no changes" on a branch that clearly has changes:** the scope flag is
|
|
missing or wrong. A prompt-only `codex review` defaults to uncommitted changes, so a
|
|
clean working tree reads as an empty review even when `<base>...HEAD` is large. Confirm
|
|
`--base <base>` is actually on the command line.
|
|
- **Model not supported (HTTP 400):** stderr shows
|
|
`The '<model>' model is not supported when using Codex with a ChatGPT account`
|
|
(a `status: 400` / `invalid_request_error` naming a model). This is a
|
|
model-entitlement problem, not an auth or network failure, and the auth probe
|
|
cannot catch it. Recovery, in order:
|
|
1. Check whether `GSTACK_CODEX_MODEL` is set. If so, update it to a model the
|
|
account can use.
|
|
2. If no override is set, gstack defaults to `gpt-6-astra`. If the account cannot
|
|
use it yet, set `GSTACK_CODEX_MODEL=<supported-model>` or replace the default
|
|
flag with `-c "model=\"<supported-model>\""`.
|
|
3. If Codex printed `[notice.model_migrations]`, use that replacement model.
|
|
Never present this as a model stall or a PASS — it is a fail-closed gate result.
|
|
- **Empty response:** If `$TMPRESP` is empty or doesn't exist, tell the user:
|
|
"Codex returned no response. Check stderr for errors."
|
|
- **Session resume failure:** If resume fails, delete the session file and start fresh.
|
|
|
|
---
|
|
|
|
## Important Rules
|
|
|
|
- **Never modify files.** This skill is read-only. Codex runs in read-only sandbox mode.
|
|
- **Present output verbatim.** Do not truncate, summarize, or editorialize Codex's output
|
|
before showing it. Show it in full inside the CODEX SAYS block.
|
|
- **Add synthesis after, not instead of.** Any Claude commentary comes after the full output.
|
|
- **Bash gate above the wrapper.** Every Bash call to codex sets its `timeout`
|
|
parameter ABOVE the inner `_gstack_codex_timeout_wrapper` budget (Review:
|
|
`timeout: 360000` over the 330s wrapper; Challenge/Consult: `timeout: 660000`
|
|
over the 600s wrappers) so the wrapper fires first with a diagnosable exit 124.
|
|
- **No double-reviewing.** If the user already ran `/review`, Codex provides a second
|
|
independent opinion. Do not re-run Claude Code's own review.
|
|
- **Detect skill-file rabbit holes.** After receiving Codex output, scan for signs
|
|
that Codex got distracted by skill files: `gstack-config`, `gstack-update-check`,
|
|
`SKILL.md`, or `skills/gstack`. If any of these appear in the output, append a
|
|
warning: "Codex appears to have read gstack skill files instead of reviewing your
|
|
code. Consider retrying."
|