test(codex): binding tripwire sees the request-effort dispatch; valid and unfilled levels through Step 0.5 and every fence; doc accuracy

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-12 01:15:44 +00:00
1 parent 19ccec4030
commit ed434bec08
8 files changed
+20 -7

No files matched your search

+1 -1
View File
@@ -15,7 +15,7 @@
### For contributors
- `gstack-codex-probe check-effort <default> --request <level>` and `run-with-timeout --effort <default> --request-effort <level>` take the request level (empty means none) and name `/codex --effort` as its source. The five `/codex` dispatch fences pass it as the allowlisted `'<effort-level>'` placeholder (grammar `^[a-z]*$`), so the level reaches the shell only through the placeholder lint, never through prose. Test parsers of the `run-with-timeout` line accept the new argument.
- `gstack-codex-probe check-effort <default> --request <level>` and `run-with-timeout --effort <default> --request-effort <level>` take the request level (empty means none) and name `/codex --effort` as its source. The five `/codex` dispatch fences pass it as the allowlisted `'<effort-level>'` placeholder (grammar `^[a-z]*$`), so the level reaches the shell only as that quoted, allowlisted placeholder (never through prose), and the probe accepts only the seven levels. Test parsers of the `run-with-timeout` line accept the new argument.
- `test/codex-effort.test.ts` runs all five rendered dispatches against a fake `codex`, and the bogus-level case asserts the rejection and zero calls. `test/model-role-invocation.test.ts` adds a zero-call case for an invalid level in Step 0.5.
Contributed by @jnunemaker
+1 -1
View File
@@ -4,7 +4,7 @@
### P3: `/codex --effort` follow-ups (filed 2026-10-11, from the #3146 autoplan review)
- **Show the effort that ran.** The probe's `CODEX_EFFORT: <level> (source: ...)` line stays in `$TMPERR`, so a user cannot see that `--effort` took effect. Add an `Effort:` line to the review, challenge and consult result footers. Blocked on skeleton bytes in `codex/SKILL.md` (cap 60,200). **Effort:** S. **Priority:** P3.
- **Show the effort that ran.** Step 0.5 prints `CODEX_EFFORT: <level> (source: /codex --effort)` once, but the result footers do not repeat the effort, so it scrolls away. Add an `Effort:` line to the review, challenge and consult result footers. Blocked on skeleton bytes in `codex/SKILL.md` (cap 60,200). **Effort:** S. **Priority:** P3.
- **Timeouts at high effort.** `xhigh`, `max` and `ultra` can exceed the fixed 330 s / 540 s wrappers, and the timeout copy blames prompt size or API speed. Name the effort in the timeout message and suggest a lower `--effort`; consider effort-scaled budgets. **Effort:** S. **Priority:** P3.
- **Per-run effort for gate voices (#2934).** `/review`, `/ship` and plan-review outside voices take `GSTACK_CODEX_EFFORT` and `codex_effort` but have no per-request flag; the Claude Code runner takes no effort at all. Needs its own precedence and consent review. **Effort:** M. **Priority:** P3.
- **`minimal` with web search.** Every `/codex` dispatch passes `web_search="cached"`, and Codex reportedly rejects that with `reasoning.effort: minimal` (openai/codex#3559, #5002). Unverified here (no Codex CLI on the release machine); the same path already ships through `codex_effort minimal`. Verify on the pinned CLI and drop web search for `minimal` if it fails. **Effort:** S. **Priority:** P3.
+1 -1
View File
@@ -486,7 +486,7 @@ probes and dispatches the policy model within its own timeout. [Policy setup](ht
**Effort flags, read before the mode:** `--effort <level>`, `--effort=<level>` or `--xhigh`
(= `--effort xhigh`) directly after `/codex` or its mode word, among its leading flags
(`-m`, `--model`, `--role`) and before any prompt text, set `<effort-level>` (else `''`);
remove them before Step 1. A later `--effort` is prompt text; say so in one line. Missing
remove them before Step 1. A later `--effort` is prompt text; if a level follows, say so. Missing
value, not one lowercase word, or differing levels: stop, name the levels; nothing is sent.
```bash
+1 -1
View File
@@ -80,7 +80,7 @@ probes and dispatches the policy model within its own timeout. [Policy setup](ht
**Effort flags, read before the mode:** `--effort <level>`, `--effort=<level>` or `--xhigh`
(= `--effort xhigh`) directly after `/codex` or its mode word, among its leading flags
(`-m`, `--model`, `--role`) and before any prompt text, set `<effort-level>` (else `''`);
remove them before Step 1. A later `--effort` is prompt text; say so in one line. Missing
remove them before Step 1. A later `--effort` is prompt text; if a level follows, say so. Missing
value, not one lowercase word, or differing levels: stop, name the levels; nothing is sent.
```bash
+1 -1
View File
@@ -1195,7 +1195,7 @@ A severity-gate PASS is separate from [review freshness](#review-readiness-dashb
**Challenge** — adversarial mode. Codex actively tries to break your code. It looks for edge cases, race conditions, security holes, and assumptions that would fail under load. Think of it as a penetration test for your logic.
**Reasoning effort** — Review and Challenge run at `high`, Consult at `medium`. `gstack-config set codex_effort <level>` changes that for every run; `--effort <level>` right after the mode changes it for one (`/codex review --effort medium` for a faster, cheaper review). `--xhigh` is shorthand for `--effort xhigh`. An unknown level stops before Codex runs. See [model policy](model-policy.md#reasoning-effort) for levels and precedence.
**Reasoning effort** — Review and Challenge run at `high`, Consult at `medium`. `gstack-config set codex_effort <level>` changes that for every run; `--effort <level>` right after `/codex` or its mode word, among the leading flags (`-m`, `--model`, `--role`), changes it for one (`/codex review --effort medium` for a faster, cheaper review). `--xhigh` is shorthand for `--effort xhigh`. An unknown level stops before Codex runs. See [model policy](model-policy.md#reasoning-effort) for levels and precedence.
**Consult** — open conversation with session continuity. Ask Codex anything about the codebase. Follow-up questions reuse the same session, so context carries over. Great for "am I thinking about this correctly?" moments.
+3 -1
View File
@@ -120,7 +120,9 @@ describe('content-binding template drift', () => {
const codex = rendered('codex/sections/review-mode.md');
const starts = [...codex.matchAll(/gstack-review-log --start codex-review/g)];
expect(starts).toHaveLength(2);
expect(starts[0].index).toBeLessThan(codex.indexOf('run-with-timeout --user-invoked --effort high 330 codex review'));
const nativeReview = codex.indexOf(`run-with-timeout --user-invoked --effort high --request-effort '<effort-level>' 330 codex review`);
expect(nativeReview).toBeGreaterThan(-1);
expect(starts[0].index).toBeLessThan(nativeReview);
expect(starts[1].index).toBeLessThan(codex.indexOf('git diff "<base>...HEAD"'));
expect(codex).toContain('--finish CODEX_REVIEW_START');
expect(codex).toContain('"completed":COMPLETED,"converged":CONVERGED');
+4 -1
View File
@@ -43,7 +43,7 @@ describe('manual /codex modes (#3098)', () => {
const flags = between(body, '**Effort flags, read before the mode:**', '```bash');
expectTokens(flags, ['--effort <level>', '--effort=<level>', '--xhigh', '--effort xhigh', '<effort-level>'], 'codex effort flags');
expectMentions(flags, [['directly after', 'mode word', 'leading flags', '--model', '--role', 'before any prompt text'], ['later', 'prompt text'], ['remove', 'before Step 1'],
['missing', 'lowercase word', 'differing levels', 'stop'], ['later', 'say so']], 'codex effort flags');
['missing', 'lowercase word', 'differing levels', 'stop'], ['later', 'level', 'say so']], 'codex effort flags');
const step1 = between(body, '## Step 1: Detect mode', '- Review (2A)');
expectMentions(step1, [['effort flags removed'], ['--request-effort', 'outranks']], 'codex Step 1');
});
@@ -149,6 +149,9 @@ echo '{"type":"item.completed","item":{"type":"agent_message","text":"SCORE: 8"}
expect(run(line, 'medium').args, mode).toContain('model_reasoning_effort="medium"');
expect(run(line, 'xhigh').args, mode).toContain('model_reasoning_effort="xhigh"');
expect(run(line, 'medium', 'max').args, mode).toContain('model_reasoning_effort="medium"');
const unfilled = run(line, '<effort-level>');
expect(unfilled.args, mode).toEqual([]);
expect(unfilled.stderr, mode).toContain('placeholder was left unfilled');
const bogus = run(line, 'bogus');
expect(bogus.args, mode).toEqual([]);
expect(bogus.status, mode).toBe(1);
+8
View File
@@ -586,6 +586,14 @@ describe('manual /codex entry: explicit role only, paid probe only for the dispa
expect(role.stderr).not.toContain('NOTICE: gstack plan reviews');
});
test('a valid /codex --effort level passes Step 0.5, shows its source once and probes once', () => {
const r = shell(home(), withEffort(preflight, 'medium'));
expect(r.status).toBe(0);
expect(r.stdout).not.toContain('CODEX_EFFORT');
expect(r.stderr).toContain('CODEX_EFFORT: medium (source: /codex --effort)');
expect(r.calls.map(codexModel)).toEqual(['model="gpt-5.6-terra"']);
});
test('an invalid /codex --effort level stops Step 0.5 before the paid model probe', () => {
const r = shell(home(), withEffort(preflight, 'turbo'));
expect(r.status).toBe(1);