feat(codex): model round-trip probe — an unusable configured model fails fast with guidance (#2477)

The auth probe accepts 'auth exists' as readiness, but a ChatGPT account
with a stale model pin in ~/.codex/config.toml passes it and then EVERY
mode dies with an HTTP 400 ('The <model> model is not supported when using
Codex with a ChatGPT account') and no pointer to where the model came from
— one report burned ~40 minutes and four invocations plus a strings dump
of the binary before finding the one-line config fix.

bin/gstack-codex-probe gains _gstack_codex_model_probe: a short
codex exec 'reply OK' round trip with the configured model, gated behind
the cheap auth probe at all three preflight sites (codex Step 0.5, the
shared codexPreflight in scripts/resolvers/constants.ts — which grows a
model_unusable CODEX_MODE branch — and autoplan's availability chain).
Verdicts: MODEL_OK (cached 1h, keyed on config.toml + auth.json mtimes so
a pin edit or re-login re-probes immediately), MODEL_UNUSABLE (exit 1,
prints the rejection plus HINTs at the model= pin and the
[notice.model_migrations] table), MODEL_PROBE_INCONCLUSIVE (timeout or
transient: FAIL-OPEN so network luck never wedges codex mode).

The 'Model not supported (HTTP 400)' Error Handling entry already shipped
in v1.64.0.0; Step 0.5's prose now routes MODEL_UNUSABLE to it.

test/codex-model-probe.test.ts drives all four behaviors against a stubbed
codex binary (invocation-counted cache hit, hint content, fail-open
polarity, mtime invalidation).

Fixes #2477

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-16 10:59:35 -07:00
co-authored by Claude Fable 5
parent 63e2b7ac2c
commit 73c96f9c12
14 changed files with 289 additions and 9 deletions
@@ -381,6 +381,8 @@ elif ! command -v codex >/dev/null 2>&1; then
_CODEX_MODE="not_installed"; _gstack_codex_log_event "codex_cli_missing" 2>/dev/null || true
elif ! _gstack_codex_auth_probe >/dev/null 2>&1; then
_CODEX_MODE="not_authed"; _gstack_codex_log_event "codex_auth_failed" 2>/dev/null || true
elif ! _gstack_codex_model_probe; then
_CODEX_MODE="model_unusable"
else
_CODEX_MODE="ready"; _gstack_codex_version_check 2>/dev/null || true
fi
@@ -391,6 +393,7 @@ Branch on the echoed `CODEX_MODE`:
- **`disabled`** — the user turned Codex reviews off (`codex_reviews=disabled`). Skip this section entirely; do NOT fall back to a Claude subagent — disabled means no extra review step. Print: "Codex review skipped (codex_reviews disabled). Re-enable: `gstack-config set codex_reviews enabled`."
- **`not_installed`** — Codex CLI absent. Print: "Codex not installed — using Claude subagent. Install for cross-model coverage: `npm install -g @openai/codex`." Fall back to the Claude subagent path.
- **`not_authed`** — installed but no credentials. Print: "Codex installed but not authenticated — using Claude subagent. Run `codex login` or set `$CODEX_API_KEY`." Fall back to the Claude subagent path.
- **`model_unusable`** — authed but the account cannot use its configured model (#2477: HTTP 400 on every call, usually a stale `model =` pin in `~/.codex/config.toml`). Relay the probe's HINT lines, tell the user the one-line fix (update the pin; `[notice.model_migrations]` names the replacement), and fall back to the Claude subagent path. The ~10s round trip is cached for 1h; timeouts fail open to `ready`.
- **`ready`** — run the Codex pass below.
When the mode is `ready`, `not_installed`, or `not_authed`, print one line so the off-switch