Files
gstack/codex/SKILL.md.tmpl
Garry Tan dc5e0538e5 feat: worktree isolation for E2E tests + infrastructure elegance (v0.11.12.0) (#425)
* refactor: extract gen-skill-docs into modular resolver architecture

Break the 3000-line monolith into 10 domain modules under scripts/resolvers/:
types, constants, preamble, utility, browse, design, testing, review,
codex-helpers, and index. Each module owns one domain of template generation.

The preamble module introduces a 4-tier composition system (T1-T4) so skills
only pay for the preamble sections they actually need, reducing token usage
for lightweight skills by ~40%.

Adds a token budget dashboard that prints after every generation run showing
per-skill and total token counts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: tiered preamble — skills only pay for what they use

Tag all 23 templates with preamble-tier (T1-T4). Lightweight skills
like /browse and /benchmark get a minimal preamble (~40% fewer tokens),
while review skills get the full stack. Regenerate all SKILL.md files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: migrate eval storage to project-scoped paths

Move eval results and E2E run artifacts from ~/.gstack-dev/evals/ to
~/.gstack/projects/$SLUG/evals/ so each project's eval history lives
alongside its other gstack data. Falls back to legacy path if slug
detection fails.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: sync package.json version with VERSION after merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add WorktreeManager for isolated test environments

Reusable platform module (lib/worktree.ts) that creates git worktrees
for test isolation and harvests useful changes as patches. Includes
SHA-256 dedup, original SHA tracking for committed change detection,
and automatic gitignored artifact copying (.agents/, browse/dist/).

12 unit tests covering lifecycle, harvest, dedup, and error handling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: integrate worktree isolation into E2E test infrastructure

Add createTestWorktree(), harvestAndCleanup(), and describeWithWorktree()
helpers to e2e-helpers.ts. Add harvest field to EvalTestEntry for
eval-store integration. Register lib/worktree.ts as a global touchfile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: run Gemini and Codex E2E tests in worktrees

Switch both test suites from cwd: ROOT to worktree isolation.
Gemini (--yolo) no longer pollutes the working tree. Codex
(read-only) gets worktree for consistency. Useful changes are
harvested as patches for cherry-picking.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: skip symlinks in copyDirSync to prevent infinite recursion

Adversarial review caught that .claude/skills/gstack may be a symlink
back to the repo root, causing copyDirSync to recurse infinitely
when copying gitignored artifacts into worktrees.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: bump version and changelog (v0.11.12.0)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: relax session-awareness assertion to accept structured options

The LLM consistently presents well-formatted A/B choices with pros/cons
but doesn't always use the exact string "RECOMMENDATION". Accept
case-insensitive "recommend", "option a", "which do you want", or
"which approach" as equivalent signals of a structured recommendation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 23:05:22 -07:00

358 lines
14 KiB
Cheetah

---
name: codex
preamble-tier: 3
version: 1.0.0
description: |
OpenAI Codex CLI wrapper — three modes. Code review: independent diff review via
codex review with pass/fail gate. Challenge: adversarial mode that tries to break
your code. Consult: ask codex anything with session continuity for follow-ups.
The "200 IQ autistic developer" second opinion. Use when asked to "codex review",
"codex challenge", "ask codex", "second opinion", or "consult codex".
allowed-tools:
- Bash
- Read
- Write
- Glob
- Grep
- AskUserQuestion
---
{{PREAMBLE}}
{{BASE_BRANCH_DETECT}}
# /codex — Multi-AI Second Opinion
You are running the `/codex` skill. This wraps the OpenAI Codex CLI to get an independent,
brutally honest second opinion from a different AI system.
Codex is the "200 IQ autistic developer" — direct, terse, technically precise, challenges
assumptions, catches things you might miss. Present its output faithfully, not summarized.
---
## Step 0: Check codex binary
```bash
CODEX_BIN=$(which codex 2>/dev/null || echo "")
[ -z "$CODEX_BIN" ] && echo "NOT_FOUND" || echo "FOUND: $CODEX_BIN"
```
If `NOT_FOUND`: stop and tell the user:
"Codex CLI not found. Install it: `npm install -g @openai/codex` or see https://github.com/openai/codex"
---
## Step 1: Detect mode
Parse the user's input to determine which mode to run:
1. `/codex review` or `/codex review <instructions>` — **Review mode** (Step 2A)
2. `/codex challenge` or `/codex challenge <focus>` — **Challenge mode** (Step 2B)
3. `/codex` with no arguments — **Auto-detect:**
- Check for a diff (with fallback if origin isn't available):
`git diff origin/<base> --stat 2>/dev/null | tail -1 || git diff <base> --stat 2>/dev/null | tail -1`
- If a diff exists, use AskUserQuestion:
```
Codex detected changes against the base branch. What should it do?
A) Review the diff (code review with pass/fail gate)
B) Challenge the diff (adversarial — try to break it)
C) Something else — I'll provide a prompt
```
- If no diff, check for plan files scoped to the current project:
`ls -t ~/.claude/plans/*.md 2>/dev/null | xargs grep -l "$(basename $(pwd))" 2>/dev/null | head -1`
If no project-scoped match, fall back to: `ls -t ~/.claude/plans/*.md 2>/dev/null | head -1`
but warn the user: "Note: this plan may be from a different project."
- If a plan file exists, offer to review it
- Otherwise, ask: "What would you like to ask Codex?"
4. `/codex <anything else>` — **Consult mode** (Step 2C), where the remaining text is the prompt
---
## Step 2A: Review Mode
Run Codex code review against the current branch diff.
1. Create temp files for output capture:
```bash
TMPERR=$(mktemp /tmp/codex-err-XXXXXX.txt)
```
2. Run the review (5-minute timeout):
```bash
codex review --base <base> -c 'model_reasoning_effort="xhigh"' --enable web_search_cached 2>"$TMPERR"
```
Use `timeout: 300000` on the Bash call. If the user provided custom instructions
(e.g., `/codex review focus on security`), pass them as the prompt argument:
```bash
codex review "focus on security" --base <base> -c 'model_reasoning_effort="xhigh"' --enable web_search_cached 2>"$TMPERR"
```
3. Capture the output. Then parse cost from stderr:
```bash
grep "tokens used" "$TMPERR" 2>/dev/null || echo "tokens: unknown"
```
4. Determine gate verdict by checking the review output for critical findings.
If the output contains `[P1]` — the gate is **FAIL**.
If no `[P1]` markers are found (only `[P2]` or no findings) — the gate is **PASS**.
5. Present the output:
```
CODEX SAYS (code review):
════════════════════════════════════════════════════════════
<full codex output, verbatim — do not truncate or summarize>
════════════════════════════════════════════════════════════
GATE: PASS Tokens: 14,331 | Est. cost: ~$0.12
```
or
```
GATE: FAIL (N critical findings)
```
6. **Cross-model comparison:** If `/review` (Claude's own review) was already run
earlier in this conversation, compare the two sets of findings:
```
CROSS-MODEL ANALYSIS:
Both found: [findings that overlap between Claude and Codex]
Only Codex found: [findings unique to Codex]
Only Claude found: [findings unique to Claude's /review]
Agreement rate: X% (N/M total unique findings overlap)
```
7. Persist the review result:
```bash
~/.claude/skills/gstack/bin/gstack-review-log '{"skill":"codex-review","timestamp":"TIMESTAMP","status":"STATUS","gate":"GATE","findings":N,"findings_fixed":N}'
```
Substitute: TIMESTAMP (ISO 8601), STATUS ("clean" if PASS, "issues_found" if FAIL),
GATE ("pass" or "fail"), findings (count of [P1] + [P2] markers),
findings_fixed (count of findings that were addressed/fixed before shipping).
8. Clean up temp files:
```bash
rm -f "$TMPERR"
```
{{PLAN_FILE_REVIEW_REPORT}}
---
## Step 2B: Challenge (Adversarial) Mode
Codex tries to break your code — finding edge cases, race conditions, security holes,
and failure modes that a normal review would miss.
1. Construct the adversarial prompt. If the user provided a focus area
(e.g., `/codex challenge security`), include it:
Default prompt (no focus):
"Review the changes on this branch against the base branch. Run `git diff origin/<base>` to see the diff. Your job is to find ways this code will fail in production. Think like an attacker and a chaos engineer. Find edge cases, race conditions, security holes, resource leaks, failure modes, and silent data corruption paths. Be adversarial. Be thorough. No compliments — just the problems."
With focus (e.g., "security"):
"Review the changes on this branch against the base branch. Run `git diff origin/<base>` to see the diff. Focus specifically on SECURITY. Your job is to find every way an attacker could exploit this code. Think about injection vectors, auth bypasses, privilege escalation, data exposure, and timing attacks. Be adversarial."
2. Run codex exec with **JSONL output** to capture reasoning traces and tool calls (5-minute timeout):
```bash
codex exec "<prompt>" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>/dev/null | python3 -c "
import sys, json
for line in sys.stdin:
line = line.strip()
if not line: continue
try:
obj = json.loads(line)
t = obj.get('type','')
if t == 'item.completed' and 'item' in obj:
item = obj['item']
itype = item.get('type','')
text = item.get('text','')
if itype == 'reasoning' and text:
print(f'[codex thinking] {text}')
print()
elif itype == 'agent_message' and text:
print(text)
elif itype == 'command_execution':
cmd = item.get('command','')
if cmd: print(f'[codex ran] {cmd}')
elif t == 'turn.completed':
usage = obj.get('usage',{})
tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0)
if tokens: print(f'\ntokens used: {tokens}')
except: pass
"
```
This parses codex's JSONL events to extract reasoning traces, tool calls, and the final
response. The `[codex thinking]` lines show what codex reasoned through before its answer.
3. Present the full streamed output:
```
CODEX SAYS (adversarial challenge):
════════════════════════════════════════════════════════════
<full output from above, verbatim>
════════════════════════════════════════════════════════════
Tokens: N | Est. cost: ~$X.XX
```
---
## Step 2C: Consult Mode
Ask Codex anything about the codebase. Supports session continuity for follow-ups.
1. **Check for existing session:**
```bash
cat .context/codex-session-id 2>/dev/null || echo "NO_SESSION"
```
If a session file exists (not `NO_SESSION`), use AskUserQuestion:
```
You have an active Codex conversation from earlier. Continue it or start fresh?
A) Continue the conversation (Codex remembers the prior context)
B) Start a new conversation
```
2. Create temp files:
```bash
TMPRESP=$(mktemp /tmp/codex-resp-XXXXXX.txt)
TMPERR=$(mktemp /tmp/codex-err-XXXXXX.txt)
```
3. **Plan review auto-detection:** If the user's prompt is about reviewing a plan,
or if plan files exist and the user said `/codex` with no arguments:
```bash
ls -t ~/.claude/plans/*.md 2>/dev/null | xargs grep -l "$(basename $(pwd))" 2>/dev/null | head -1
```
If no project-scoped match, fall back to `ls -t ~/.claude/plans/*.md 2>/dev/null | head -1`
but warn: "Note: this plan may be from a different project — verify before sending to Codex."
Read the plan file and prepend the persona to the user's prompt:
"You are a brutally honest technical reviewer. Review this plan for: logical gaps and
unstated assumptions, missing error handling or edge cases, overcomplexity (is there a
simpler approach?), feasibility risks (what could go wrong?), and missing dependencies
or sequencing issues. Be direct. Be terse. No compliments. Just the problems.
THE PLAN:
<plan content>"
4. Run codex exec with **JSONL output** to capture reasoning traces (5-minute timeout):
For a **new session:**
```bash
codex exec "<prompt>" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | python3 -c "
import sys, json
for line in sys.stdin:
line = line.strip()
if not line: continue
try:
obj = json.loads(line)
t = obj.get('type','')
if t == 'thread.started':
tid = obj.get('thread_id','')
if tid: print(f'SESSION_ID:{tid}')
elif t == 'item.completed' and 'item' in obj:
item = obj['item']
itype = item.get('type','')
text = item.get('text','')
if itype == 'reasoning' and text:
print(f'[codex thinking] {text}')
print()
elif itype == 'agent_message' and text:
print(text)
elif itype == 'command_execution':
cmd = item.get('command','')
if cmd: print(f'[codex ran] {cmd}')
elif t == 'turn.completed':
usage = obj.get('usage',{})
tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0)
if tokens: print(f'\ntokens used: {tokens}')
except: pass
"
```
For a **resumed session** (user chose "Continue"):
```bash
codex exec resume <session-id> "<prompt>" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | python3 -c "
<same python streaming parser as above>
"
```
5. Capture session ID from the streamed output. The parser prints `SESSION_ID:<id>`
from the `thread.started` event. Save it for follow-ups:
```bash
mkdir -p .context
```
Save the session ID printed by the parser (the line starting with `SESSION_ID:`)
to `.context/codex-session-id`.
6. Present the full streamed output:
```
CODEX SAYS (consult):
════════════════════════════════════════════════════════════
<full output, verbatim — includes [codex thinking] traces>
════════════════════════════════════════════════════════════
Tokens: N | Est. cost: ~$X.XX
Session saved — run /codex again to continue this conversation.
```
7. After presenting, note any points where Codex's analysis differs from your own
understanding. If there is a disagreement, flag it:
"Note: Claude Code disagrees on X because Y."
---
## Model & Reasoning
**Model:** No model is hardcoded — codex uses whatever its current default is (the frontier
agentic coding model). This means as OpenAI ships newer models, /codex automatically
uses them. If the user wants a specific model, pass `-m` through to codex.
**Reasoning effort:** All modes use `xhigh` — maximum reasoning power. When reviewing code, breaking code, or consulting on architecture, you want the model thinking as hard as possible.
**Web search:** All codex commands use `--enable web_search_cached` so Codex can look up
docs and APIs during review. This is OpenAI's cached index — fast, no extra cost.
If the user specifies a model (e.g., `/codex review -m gpt-5.1-codex-max`
or `/codex challenge -m gpt-5.2`), pass the `-m` flag through to codex.
---
## Cost Estimation
Parse token count from stderr. Codex prints `tokens used\nN` to stderr.
Display as: `Tokens: N`
If token count is not available, display: `Tokens: unknown`
---
## Error Handling
- **Binary not found:** Detected in Step 0. Stop with install instructions.
- **Auth error:** Codex prints an auth error to stderr. Surface the error:
"Codex authentication failed. Run `codex login` in your terminal to authenticate via ChatGPT."
- **Timeout:** If the Bash call times out (5 min), tell the user:
"Codex timed out after 5 minutes. The diff may be too large or the API may be slow. Try again or use a smaller scope."
- **Empty response:** If `$TMPRESP` is empty or doesn't exist, tell the user:
"Codex returned no response. Check stderr for errors."
- **Session resume failure:** If resume fails, delete the session file and start fresh.
---
## Important Rules
- **Never modify files.** This skill is read-only. Codex runs in read-only sandbox mode.
- **Present output verbatim.** Do not truncate, summarize, or editorialize Codex's output
before showing it. Show it in full inside the CODEX SAYS block.
- **Add synthesis after, not instead of.** Any Claude commentary comes after the full output.
- **5-minute timeout** on all Bash calls to codex (`timeout: 300000`).
- **No double-reviewing.** If the user already ran `/review`, Codex provides a second
independent opinion. Do not re-run Claude Code's own review.