mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-10 02:30:39 +01:00
* Add failing tests: outside reviews read clean only from a last-line verdict
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Count an outside review as clean only from its last VERDICT line
Every outside-review prompt now asks the reviewer to end with exactly
VERDICT: CLEAN or VERDICT: FINDINGS, and the parser reads a pass only from
that line. Prose no-findings sentences are no longer evidence for clean.
Codex's structured review and /codex review run through codex exec, since
codex review --base takes no prompt and could never give the line.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Repin the outside reviewer briefs and fix the tests the verdict line broke (GST-1821)
The outside brief, both adversarial sections and the review-army prompt-file line now carry the last-line VERDICT request; their pins move under the preregistered no-loss replay GST-1821. Test fakes that wrote a clean reply now end it with VERDICT: CLEAN, the review carve marker follows the codex exec sentence, size pins drop where skills shrank, and the plan-review prompt-file line is shortened so /plan-eng-review stays at its pin.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Give the plan-review invocation test's fake reviewer the verdict line it now needs
CI's free-suite shard 5 failed at 1de52b0 with 18 cases in test/model-role-invocation.test.ts: the fake Codex reply had no VERDICT line, so it read unverified (exit 4), and the review-mode template now has one codex exec dispatch block instead of two. The fake reply now ends with VERDICT: FINDINGS and the block count expects one.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Stop the changelog correction from claiming the verdict line closes every false-clean case
Review-Finding: p1/red-team#2 reader=red-team
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Split the changelog correction's last sentence for readability
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Close the review fail-opens the fix-check found (GST-1821)
A /codex review whose git diff fails now stops before Codex runs instead of sending an empty diff. The verdict is read from the raw last line, so inline code after VERDICT: CLEAN no longer reads clean, and VERDICT: CLEAN beside numbered items with no severity reads unverified. Codex structured review gets the base-branch diff appended to its prompt, the way the Claude path does, so the scope no longer depends on the prompt the host writes.
Fix round granted by the Captain (a-aec3882c9d48). Review-section pins move under replay GST-1821: the free 681-input replay and the 54 scored probe replies are unchanged through the fixed parser.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Say in the changelog which numbered-list replies now read unverified, and that /codex review stops without its diff
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Say in the skills guide that /codex review now sends the diff through codex exec
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* Check that the structured outside review appends the base-branch diff to its prompt
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
339 lines
16 KiB
Cheetah
339 lines
16 KiB
Cheetah
---
|
|
name: codex
|
|
preamble-tier: 3
|
|
version: 1.0.0
|
|
description: |
|
|
OpenAI Codex CLI wrapper — three modes. Code review: independent diff review via
|
|
codex review with pass/fail gate. Challenge: adversarial mode that tries to break
|
|
your code. Consult: ask codex anything with session continuity for follow-ups.
|
|
The "200 IQ autistic developer" second opinion. Use when asked to "codex review",
|
|
"codex challenge", "ask codex", "second opinion", or "consult codex". (gstack)
|
|
voice-triggers:
|
|
- "code x"
|
|
- "code ex"
|
|
- "get another opinion"
|
|
triggers:
|
|
- codex review
|
|
- second opinion
|
|
- outside voice challenge
|
|
allowed-tools:
|
|
- Bash
|
|
- Read
|
|
- Write
|
|
- Glob
|
|
- Grep
|
|
- AskUserQuestion
|
|
---
|
|
|
|
{{PREAMBLE}}
|
|
|
|
{{BASE_BRANCH_DETECT}}
|
|
|
|
# /codex — Multi-AI Second Opinion
|
|
|
|
You are running the `/codex` skill. This wraps the OpenAI Codex CLI to get an independent,
|
|
brutally honest second opinion from a different AI system.
|
|
|
|
Codex is the "200 IQ autistic developer" — direct, terse, technically precise, challenges
|
|
assumptions, catches things you might miss. Present its output faithfully, not summarized.
|
|
|
|
---
|
|
|
|
{{SECTION_INDEX:codex}}
|
|
|
|
---
|
|
|
|
## Step 0.4: Check codex binary
|
|
|
|
```bash
|
|
CODEX_BIN=$(command -v codex || echo "")
|
|
[ -z "$CODEX_BIN" ] && echo "NOT_FOUND" || echo "FOUND: $CODEX_BIN"
|
|
```
|
|
|
|
If `NOT_FOUND`: stop and tell the user:
|
|
"Codex CLI not found. Install it: `npm install -g @openai/codex` or see https://github.com/openai/codex"
|
|
|
|
If `NOT_FOUND`, also log the event:
|
|
```bash
|
|
~/.claude/skills/gstack/bin/gstack-codex-probe log-event codex_cli_missing 2>/dev/null || true
|
|
```
|
|
|
|
---
|
|
|
|
## Step 0.5: Auth probe + model probe + version check
|
|
|
|
Before building expensive prompts, verify Codex has valid auth, that the account
|
|
can actually USE gstack's selected model, AND the installed CLI version isn't in the
|
|
known-bad list. `gstack-codex-probe` is the shared helper that `/codex`, `/autoplan`
|
|
and every outside review run as a command, one subcommand per check.
|
|
|
|
Model order: a model the user names for this request, `GSTACK_CODEX_MODEL`, Codex
|
|
`config.toml` `model` (`review_model` first for `codex review`; honors `$CODEX_HOME`),
|
|
then `gpt-6-astra`. Each call prints `CODEX_MODEL: <model> (<kind>; source: ...)` first.
|
|
For a named model, add `--model '<model>'` to every `select-model`, `probe-model` and
|
|
`role-ready` call, including the probe below. An invalid or unavailable choice stops
|
|
with a repair message, never the default; a named model also wins over a role.
|
|
Only for an explicit `--role plan-review` (e.g. `/codex challenge --role plan-review`),
|
|
set `_CODEX_ROLE='plan-review'` below and in the mode block; the mode block then
|
|
probes and dispatches the policy model within its own timeout. [Policy setup](https://github.com/garrytan/gstack/blob/main/docs/model-policy.md).
|
|
|
|
**Effort flags, read before the mode:** `--effort <level>`, `--effort=<level>` or `--xhigh`
|
|
(= `--effort xhigh`) directly after `/codex` or its mode word, among its leading flags
|
|
(`-m`, `--model`, `--role`) and before any prompt text, set `<effort-level>` (else `''`);
|
|
remove them before Step 1. A later `--effort` is prompt text; if a level follows, say so. Missing
|
|
value, not one lowercase word, or differing levels: stop, name the levels; nothing is sent.
|
|
|
|
```bash
|
|
_CODEX_PROBE=~/.claude/skills/gstack/bin/gstack-codex-probe
|
|
_CODEX_ROLE=''
|
|
[ -x "$_CODEX_PROBE" ] || { echo "HELPER_UNAVAILABLE"; exit 1; }
|
|
"$_CODEX_PROBE" check-effort high --request '<effort-level>' >/dev/null || exit 1 # free
|
|
|
|
{{OUTSIDE_SELF_GUARD:codex}}
|
|
if ! "$_CODEX_PROBE" check-auth >/dev/null; then
|
|
"$_CODEX_PROBE" log-event codex_auth_failed
|
|
echo "AUTH_FAILED"
|
|
elif "$_CODEX_PROBE" check-sandbox && [ -z "$_CODEX_ROLE" ]; then # free; Linux only
|
|
"$_CODEX_PROBE" probe-model exec # ~10s round trip on first run, cached 1h
|
|
fi
|
|
"$_CODEX_PROBE" check-version # warns if known-bad, non-blocking
|
|
```
|
|
|
|
If `check-effort` failed, relay its lines and stop; no Codex call was made.
|
|
|
|
If the runtime guard reports a harness mismatch, stop. Outside coverage is unavailable. Repair with `./setup --host codex`; do not silently substitute another provider or force a same-harness invocation.
|
|
|
|
If the output contains `HELPER_UNAVAILABLE`, stop: `gstack-codex-probe` is missing or
|
|
not executable in this install. Tell the user to re-run `./setup` (or `/gstack-upgrade`).
|
|
|
|
If the output contains `AUTH_FAILED`, stop and tell the user:
|
|
"No Codex authentication found. Run `codex login` or set `$CODEX_API_KEY` / `$OPENAI_API_KEY`, then re-run this skill."
|
|
|
|
If the output contains `MODEL_UNUSABLE`, stop — the selected model (named with its
|
|
source on the `CODEX_MODEL:` line) is invalid or the account cannot use it. Relay the
|
|
probe's HINT lines and
|
|
follow the "Model not supported (HTTP 400 or 404)" recovery steps in
|
|
`## Error Handling` below. Running the modes anyway just burns four
|
|
invocations on the same 400.
|
|
|
|
If the output contains `MODEL_QUOTA_EXHAUSTED`, stop: the account hit its Codex
|
|
usage limit. Relay Codex's own line under the marker verbatim (it names the reset
|
|
time) and the HINT line (how long gstack skips Codex, and how to retry now); the model
|
|
is fine, so do not change it. Running the modes anyway fails the same way.
|
|
|
|
`MODEL_PROBE_RATE_LIMITED` is non-blocking: Codex answered 429. Report
|
|
`CODEX_MODE: unverified (rate_limited)`, relay Codex's line and continue; a
|
|
rate-limited mode run is missing coverage, never a pass.
|
|
|
|
If the output contains `CODEX_SANDBOX: unavailable`, stop: Codex's sandbox cannot
|
|
start here, so every command it runs would fail and its review would read nothing.
|
|
Relay the `Codex outside review unavailable: ...` line verbatim, including its fix.
|
|
No paid call was made.
|
|
|
|
`MODEL_PROBE_INCONCLUSIVE` is non-blocking (timeout/transient network): report
|
|
`CODEX_MODE: unverified`, pass the warning through and continue; the mode's own
|
|
validator still decides the result.
|
|
|
|
If the version check printed a `WARN:` line, pass it through to the user verbatim
|
|
(non-blocking — Codex may still work, but the user should upgrade).
|
|
|
|
The probe multi-signal auth logic accepts: `$CODEX_API_KEY` set, `$OPENAI_API_KEY`
|
|
set, or `${CODEX_HOME:-~/.codex}/auth.json` exists. Avoids false-negatives for
|
|
env-auth users (CI, platform engineers) that file-only checks would reject.
|
|
|
|
---
|
|
|
|
## Step 0.6: Resolve portable roots
|
|
|
|
Before any mode runs, resolve `$PLAN_ROOT` (where plan files live) and `$TMP_ROOT`
|
|
(where ephemeral codex stderr / response captures land) via `bin/gstack-paths`.
|
|
This keeps the skill working whether installed as a Claude Code plugin
|
|
(`CLAUDE_PLANS_DIR` set), a global `~/.claude/skills/gstack/` install, or a CI
|
|
container where `HOME` may be unset and `/tmp` may be read-only.
|
|
|
|
```bash
|
|
GSTACK_STATE_ROOT=$(~/.claude/skills/gstack/bin/gstack-paths --get GSTACK_STATE_ROOT); : "${GSTACK_STATE_ROOT:?gstack-paths failed; reinstall with ./setup or /gstack-upgrade}"
|
|
PLAN_ROOT=$(~/.claude/skills/gstack/bin/gstack-paths --get PLAN_ROOT)
|
|
TMP_ROOT=$(~/.claude/skills/gstack/bin/gstack-paths --get TMP_ROOT)
|
|
```
|
|
|
|
After this, every subsequent bash block in this skill uses `"$PLAN_ROOT"` and
|
|
`"$TMP_ROOT"` rather than hardcoded `~/.claude/plans` or `/tmp/codex-*`.
|
|
|
|
---
|
|
|
|
## Step 1: Detect mode
|
|
|
|
Parse the input, effort flags removed, to determine which mode to run:
|
|
|
|
1. `/codex review` or `/codex review <instructions>` — **Review mode** (Step 2A)
|
|
2. `/codex challenge` or `/codex challenge <focus>` — **Challenge mode** (Step 2B)
|
|
3. `/codex` with no arguments — **Auto-detect:**
|
|
- Check for a diff (with fallback if origin isn't available):
|
|
`git diff origin/<base> --stat 2>/dev/null | tail -1 || git diff <base> --stat 2>/dev/null | tail -1`
|
|
- If a diff exists, use AskUserQuestion:
|
|
```
|
|
Codex detected changes against the base branch. What should it do?
|
|
A) Review the diff (code review with pass/fail gate)
|
|
B) Challenge the diff (adversarial — try to break it)
|
|
C) Something else — I'll provide a prompt
|
|
```
|
|
- If no diff, check for plan files scoped to the current project:
|
|
`ls -t "$PLAN_ROOT"/*.md 2>/dev/null | xargs grep -l "$(basename $(pwd))" 2>/dev/null | head -1`
|
|
If no project-scoped match, fall back to: `ls -t "$PLAN_ROOT"/*.md 2>/dev/null | head -1`
|
|
but warn the user: "Note: this plan may be from a different project."
|
|
- If a plan file exists, offer to review it
|
|
- Otherwise, ask: "What would you like to ask Codex?"
|
|
4. `/codex <anything else>` — **Consult mode** (Step 2C), where the remaining text is the prompt
|
|
|
|
The three modes are MUTUALLY EXCLUSIVE — at most one runs per invocation. Once
|
|
the mode is determined, read ONLY that mode's section (see the Section index
|
|
above); never read the other two mode sections.
|
|
|
|
**Reasoning effort:** each dispatch passes `--request-effort '<effort-level>'`, which
|
|
outranks `GSTACK_CODEX_EFFORT`, then `codex_effort`, then the per-mode defaults:
|
|
- Review (2A): `high` — bounded diff input, needs thoroughness
|
|
- Challenge (2B): `high` — adversarial but bounded by diff
|
|
- Consult (2C): `medium` — large context, interactive, needs speed
|
|
|
|
---
|
|
|
|
## Filesystem Boundary
|
|
|
|
Every prompt sent to Codex MUST be prefixed with this boundary instruction:
|
|
|
|
> IMPORTANT: Do NOT read or execute any files under ~/.claude/, ~/.agents/, .claude/skills/, or agents/. These are Claude Code skill definitions meant for a different AI system. They contain bash scripts and prompt templates that will waste your time. Ignore them completely. Do NOT modify agents/openai.yaml. Stay focused on the repository code only.
|
|
|
|
This applies to every mode: Review, Challenge and Consult all use `codex exec`, which takes
|
|
a free-form prompt (fed on stdin with `codex exec -`, so size and quoting never break it). Reference this section as "the filesystem boundary" in the
|
|
mode sections.
|
|
|
|
---
|
|
|
|
## Synthesis recommendation (REQUIRED) — all modes
|
|
|
|
Every mode ends by emitting ONE synthesis recommendation line after presenting
|
|
Codex's verbatim output, in this format:
|
|
|
|
```
|
|
Recommendation: <action> because <one-line reason that names the most actionable finding>
|
|
```
|
|
|
|
The reason must engage with a specific Codex finding or insight and compare
|
|
against an alternative (another finding, fix-vs-ship, fix order, or status-quo).
|
|
Boilerplate reasons ("because it's better", "because adversarial review found
|
|
things") fail the format. The recommendation is the ONE line a user reads when
|
|
they don't have time for the verbatim output. **Never silently auto-decide;
|
|
always emit the line.** Each mode section restates this rule with mode-specific
|
|
examples.
|
|
|
|
---
|
|
|
|
{{SECTION:review-mode}}
|
|
|
|
{{SECTION:challenge-mode}}
|
|
|
|
{{SECTION:consult-mode}}
|
|
|
|
{{PLAN_FILE_REVIEW_REPORT}}
|
|
|
|
{{EXIT_PLAN_MODE_GATE}}
|
|
|
|
---
|
|
|
|
## Model & Reasoning
|
|
|
|
**Model:** every Codex call passes the selection above via `{{CODEX_MODEL_CONFIG_FLAG}}`.
|
|
The flag also keeps installed skills out of Codex's context, so a review cannot become a
|
|
nested skill run.
|
|
|
|
**Reasoning effort** (defaults in Step 1): `run-with-timeout --effort <default>` resolves
|
|
`--request-effort`, `GSTACK_CODEX_EFFORT`, `codex_effort`, then the default; an invalid
|
|
value stops before any Codex call.
|
|
`--user-invoked` runs even with `codex_reviews` disabled: the user asked by name.
|
|
|
|
Override one run with `--effort <level>` (`/codex review --effort medium`). `xhigh`
|
|
uses ~23x more tokens than `high` and causes 50+ minute hangs on large context tasks
|
|
(OpenAI issues #8545, #8402, #6931).
|
|
|
|
**Web search:** All codex commands pass `{{CODEX_WEB_SEARCH_FLAG}}` so `codex exec`
|
|
invocations can look up docs and APIs during review. This is OpenAI's cached index —
|
|
fast, no extra cost. Unlike the legacy `--enable`-based spelling (deprecated by
|
|
codex >=0.144), the `-c` form explicitly overrides any top-level
|
|
`web_search` setting in `~/.codex/config.toml`.
|
|
|
|
If the user specifies a model (e.g., `/codex review -m gpt-6.1-sol` or
|
|
`/codex challenge --model gpt-daybreak-blue-latest`), translate it to the same config
|
|
form and replace the default model flag with `-c "model=\"<model>\""`.
|
|
|
|
---
|
|
|
|
## Cost Estimation
|
|
|
|
Parse token count from stderr. Codex prints `tokens used\nN` to stderr.
|
|
|
|
Display as: `Tokens: N`
|
|
|
|
If token count is not available, display: `Tokens: unknown`
|
|
|
|
---
|
|
|
|
## Error Handling
|
|
|
|
- **Binary not found:** Detected in Step 0. Stop with install instructions.
|
|
- **Auth error:** Codex prints an auth error to stderr. Surface the error:
|
|
"Codex authentication failed. Run `codex login` in your terminal to authenticate via ChatGPT."
|
|
- **Timeout (Bash outer gate):** Every Bash gate sits ABOVE its inner wrapper (360s gate
|
|
over the 330s review wrapper; 600s gate, the tool maximum, over the 540s challenge/consult
|
|
wrappers), so
|
|
the wrapper's exit-124 path normally fires first with its explicit message. If the Bash
|
|
call itself times out anyway (wrapper unavailable AND codex hung), tell the user:
|
|
"Codex timed out. The prompt may be too large or the API may be slow. Try again or use a smaller scope."
|
|
- **Timeout (inner `timeout` wrapper, exit 124):** If the wrapper fires first (it TERMs Codex, then KILLs it after 10s), the skill's hang-detection block auto-logs a telemetry event + operational learning and prints: "Codex stalled past 9 minutes. Common causes: model API stall, long prompt, network issue. Try re-running. If persistent, split the prompt or check `~/.codex/logs/`." No extra action needed.
|
|
- **Review says "no changes" on a branch that clearly has changes:** Step 2A writes
|
|
`git diff $(git merge-base <base> HEAD)` into the prompt. Confirm `<base>` is the
|
|
branch this one forked from; the wrong base reads an empty or unrelated diff.
|
|
- **Model not supported (HTTP 400 or 404):** stderr shows
|
|
`The '<model>' model is not supported when using Codex with a ChatGPT account`
|
|
(a `status: 400` / `invalid_request_error` naming a model), or
|
|
`404 Not Found: The model '<model>' does not exist or you do not have access to it`
|
|
(a retired model; a bare 404 usually means a custom provider's `base_url` is wrong).
|
|
`requires a newer version of Codex` means the CLI is too old: upgrade it instead.
|
|
None of these is an auth or network failure, and the auth probe cannot catch them.
|
|
Recovery, in order:
|
|
1. Read the `CODEX_MODEL:` line: it names the model and where it came from.
|
|
2. Fix that source and relay its HINT/Repair lines. With `--role plan-review`,
|
|
tier pins outrank native settings; native `model`/`review_model` applies
|
|
only in `host` mode. Without a role, request, `GSTACK_CODEX_MODEL`, then
|
|
native settings choose the model, falling back to `gpt-6-astra`.
|
|
3. If Codex printed `[notice.model_migrations]`, use that replacement for
|
|
no-role calls. For a role, offer an explicit override; do not silently substitute.
|
|
Never present this as a model stall or a PASS — it is a fail-closed gate result.
|
|
- **`VERDICT: unavailable`:** the shared validator found the run did not execute (for
|
|
example `Codex's sandbox could not start here`). Relay its line and fix verbatim; it is
|
|
missing coverage, never a PASS. Details: `docs/troubleshooting.md` in the gstack checkout.
|
|
- **Empty response:** If `$TMPRESP` is empty or doesn't exist, tell the user:
|
|
"Codex returned no response. Check stderr for errors."
|
|
- **Session resume failure:** If resume fails, delete the session file and start fresh.
|
|
|
|
---
|
|
|
|
## Important Rules
|
|
|
|
- **Never modify files.** This skill is read-only. Codex runs in read-only sandbox mode
|
|
(full access only when the user exported `GSTACK_CODEX_NO_SANDBOX=1`; it warns on every use).
|
|
- **Present output verbatim.** Do not truncate, summarize, or editorialize Codex's output
|
|
before showing it. Show it in full inside the CODEX SAYS block.
|
|
- **Add synthesis after, not instead of.** Any Claude commentary comes after the full output.
|
|
- **Bash gate above the wrapper.** Every Bash call to codex sets its `timeout`
|
|
parameter ABOVE the inner `run-with-timeout` budget (Review:
|
|
`timeout: 360000` over the 330s wrapper; Challenge/Consult: `timeout: 600000`
|
|
over the 540s wrappers) so the wrapper fires first with a diagnosable exit 124.
|
|
- **No double-reviewing.** If the user already ran `/review`, Codex provides a second
|
|
independent opinion. Do not re-run Claude Code's own review.
|
|
- **Detect skill-file rabbit holes.** After receiving Codex output, scan for signs
|
|
that Codex got distracted by skill files: `gstack-config`, `gstack-update-check`,
|
|
`SKILL.md`, or `skills/gstack`. If any of these appear in the output, append a
|
|
warning: "Codex appears to have read gstack skill files instead of reviewing your
|
|
code. Consider retrying."
|