fix: a Codex usage limit is its own state, not 'inconclusive; proceeding' (#3051)

The model probe matched only model and broken-install errors, so a usage or
rate limit fell through to MODEL_PROBE_INCONCLUSIVE: Codex's own message
(with the reset time) was dropped, the hint pointed at model config, and
every later skill re-probed for 30 s. The probe now returns
MODEL_QUOTA_EXHAUSTED (exit 4) on a failed, non-timeout call that names a
usage limit, rate limit, insufficient_quota or 429, relays Codex's line
verbatim and caches it for 15 minutes. Preflights map it to
CODEX_MODE: quota_exhausted (missing coverage, no further Codex calls), and
classifyOutsideReview reports a mid-run limit as unavailable (quota) with
its own troubleshooting anchor.

Co-authored-by: venkatareddyg <venkatareddyg@users.noreply.github.com>
This commit is contained in:
garrytanandvenkatareddyg committed 2026-10-06 18:09:47 +00:00
1 parent a853e413ad
commit 597e483dd1
16 files changed
+172 -2

No files matched your search

+3
View File
@@ -773,6 +773,8 @@ else
_CODEX_MODE="sandbox_unavailable"
elif [ "$_CODEX_MP" -eq 2 ]; then
_CODEX_MODE="broken_install"
elif [ "$_CODEX_MP" -eq 4 ]; then
_CODEX_MODE="quota_exhausted"
elif [ "$_CODEX_MP" -ne 0 ]; then
_CODEX_MODE="model_unusable"
elif [ "${_GSTACK_CODEX_PROBE_STATE:-}" = inconclusive ]; then
@@ -791,6 +793,7 @@ Branch on the echoed `CODEX_MODE`:
- **`not_authed`** — installed but no credentials. Print: "Codex not authenticated; outside coverage unavailable. Run `codex login` or set `$CODEX_API_KEY`." Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`broken_install`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: `npm install -g @openai/codex`." Relay the probe's HINT lines. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`model_unusable`** — the selected model (see `CODEX_MODEL:`) is invalid or unavailable to the account (HTTP 400 on every call). Relay the probe's HINT lines and the fix (`GSTACK_CODEX_MODEL=<supported-model>` or config.toml `model`); never substitute a model. Keep the required Claude adversarial pass; do not dispatch a duplicate. The ~10s round trip is cached for 1h.
- **`quota_exhausted`** — Codex usage/rate limit: relay its line verbatim (reset time), no more Codex calls. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`sandbox_unavailable`** — Codex's sandbox cannot start here (containers without user namespaces); the probe printed the reason and fix. No paid call ran; outside coverage is unavailable. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`ready`** or **`unverified`** — run the Codex pass below. `unverified` means the model check timed out; say so, and let the pass's own verdict decide.
+25 -1
View File
@@ -8,7 +8,8 @@
# _gstack_codex_sandbox_mode — read-only, or full access only for GSTACK_CODEX_NO_SANDBOX=1
# _gstack_codex_sandbox_preflight — free `codex sandbox` check before paid calls (Linux)
# _gstack_codex_first_use_notice — once per machine: provider, account, how to turn off
# _gstack_codex_model_probe — round-trip probe of the selected model (#2477)
# _gstack_codex_model_probe — round-trip probe of the selected model (#2477);
# exit 4 = the account's Codex usage limit (#3051)
# _gstack_codex_version_check — warn on known-bad Codex CLI versions
# _gstack_codex_timeout_wrapper — gtimeout -> timeout -> bash-native watchdog
# _gstack_codex_log_event — telemetry emission to ~/.gstack/analytics/
@@ -256,6 +257,12 @@ _gstack_codex_model_probe() {
echo "MODEL_OK (cached)"
return 0
fi
if [ "$_c_status" = "MODEL_QUOTA_EXHAUSTED" ] && [ "$_c_sig" = "$_sig" ] && [ $((_now - _c_ts)) -lt 900 ]; then
echo "MODEL_QUOTA_EXHAUSTED (cached)"
sed -n '2,4p' "$_cache" 2>/dev/null
echo "HINT: Codex refused this account's calls for its usage limit within the last 15 minutes; outside coverage is unavailable until the reset time above. No Codex call was made."
return 4
fi
if [ "$_c_status" = "MODEL_UNUSABLE" ] && [ "$_c_sig" = "$_sig" ] && [ $((_now - _c_ts)) -lt 900 ]; then
echo "MODEL_UNUSABLE (cached)"
echo "HINT: gstack requested model '$_model' (source: $_GSTACK_CODEX_SEL_SRC)."
@@ -271,6 +278,23 @@ _gstack_codex_model_probe() {
echo "MODEL_OK"
return 0
fi
# A usage limit or rate limit is the account, not the model or the network
# (#3051): Codex says so plainly ("You've hit your usage limit ... try again
# at <time>"). Falling through to the inconclusive branch dropped that line,
# re-probed for 30 s in every skill, and pointed users at model config.
# Only a failed, non-timeout call counts; Codex's own line (it carries the
# reset time and the purchase link) is relayed verbatim and cached briefly.
_QUOTA_SIG="usage limit|rate limit|rate_limit_exceeded|insufficient_quota|exceeded your current quota|too many requests|\"status\":[[:space:]]*429|status:?[[:space:]]*429"
if [ "$_code" -ne 124 ] && printf '%s' "$_out" | grep -qiE "$_QUOTA_SIG"; then
local _q_lines
_q_lines=$(printf '%s\n' "$_out" | grep -iE "$_QUOTA_SIG" | head -3)
[ -n "$_cache" ] && { mkdir -p "$_gstack_home" 2>/dev/null; printf 'MODEL_QUOTA_EXHAUSTED %s %s\n%s\n' "$_now" "$_sig" "$_q_lines" > "$_cache" 2>/dev/null; }
echo "MODEL_QUOTA_EXHAUSTED"
printf '%s\n' "$_q_lines"
echo "HINT: Codex refused the call for this account's usage or rate limit (the line above is Codex's own message, with its reset time). Outside coverage is unavailable until then; the model choice is fine."
_gstack_codex_log_event "codex_quota_exhausted" 2>/dev/null || true
return 4
fi
# A model rejection is deterministic: a 400 ("is not supported", "requires a
# newer version of Codex") or a 404. A retired model answers 404 "does not
# exist or you do not have access to it", and a custom provider with a wrong
+5
View File
@@ -509,6 +509,11 @@ follow the "Model not supported (HTTP 400 or 404)" recovery steps in
`## Error Handling` below. Running the modes anyway just burns four
invocations on the same 400.
If the output contains `MODEL_QUOTA_EXHAUSTED`, stop: the account hit its Codex
usage or rate limit. Relay Codex's own line under the marker verbatim (it names the
reset time); the model is fine, so do not change it. Running the modes anyway fails
the same way.
If the output contains `CODEX_SANDBOX: unavailable`, stop: Codex's sandbox cannot
start here, so every command it runs would fail and its review would read nothing.
Relay the `Codex outside review unavailable: ...` line verbatim, including its fix.
+5
View File
@@ -101,6 +101,11 @@ follow the "Model not supported (HTTP 400 or 404)" recovery steps in
`## Error Handling` below. Running the modes anyway just burns four
invocations on the same 400.
If the output contains `MODEL_QUOTA_EXHAUSTED`, stop: the account hit its Codex
usage or rate limit. Relay Codex's own line under the marker verbatim (it names the
reset time); the model is fine, so do not change it. Running the modes anyway fails
the same way.
If the output contains `CODEX_SANDBOX: unavailable`, stop: Codex's sandbox cannot
start here, so every command it runs would fail and its review would read nothing.
Relay the `Codex outside review unavailable: ...` line verbatim, including its fix.
+20
View File
@@ -152,6 +152,26 @@ export GSTACK_CODEX_MODEL=<supported-model>
**Expected result.** `CODEX_MODE: ready`.
<a id="codex-quota-exhausted"></a>
### `CODEX_MODE: quota_exhausted` / `MODEL_QUOTA_EXHAUSTED`
**Meaning.** Codex refused the call because the account behind it hit its
usage limit or rate limit. The line under the marker is Codex's own message,
with its reset time and where to buy more. The model choice is fine. gstack
skips the remaining Codex calls in that run and reports outside coverage as
unavailable; it never counts as a pass. The result is cached for 15 minutes,
so later skills do not spend another 30 seconds learning the same thing.
**Fix.** Wait for the reset time in Codex's message, or add credits or a
higher plan for that account. To use a different account, sign in again:
```bash
codex login
```
**Expected result.** After the reset (or after `codex login`, which changes
the auth signature and re-probes at once), `CODEX_MODE: ready`.
<a id="codex-mode-unverified"></a>
### `CODEX_MODE: unverified` / `MODEL_PROBE_INCONCLUSIVE`
@@ -243,6 +243,8 @@ else
_CODEX_MODE="sandbox_unavailable"
elif [ "$_CODEX_MP" -eq 2 ]; then
_CODEX_MODE="broken_install"
elif [ "$_CODEX_MP" -eq 4 ]; then
_CODEX_MODE="quota_exhausted"
elif [ "$_CODEX_MP" -ne 0 ]; then
_CODEX_MODE="model_unusable"
elif [ "${_GSTACK_CODEX_PROBE_STATE:-}" = inconclusive ]; then
@@ -261,6 +263,7 @@ Branch on the echoed `CODEX_MODE`:
- **`not_authed`** — installed but no credentials. Print: "Codex not authenticated; outside coverage unavailable. Run `codex login` or set `$CODEX_API_KEY`." Fall back to the Claude subagent path.
- **`broken_install`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: `npm install -g @openai/codex`." Relay the probe's HINT lines. Fall back to the Claude subagent path.
- **`model_unusable`** — the selected model (see `CODEX_MODEL:`) is invalid or unavailable to the account (HTTP 400 on every call). Relay the probe's HINT lines and the fix (`GSTACK_CODEX_MODEL=<supported-model>` or config.toml `model`); never substitute a model. Fall back to the Claude subagent path. The ~10s round trip is cached for 1h.
- **`quota_exhausted`** — Codex usage/rate limit: relay its line verbatim (reset time), no more Codex calls. Fall back to the Claude subagent path.
- **`sandbox_unavailable`** — Codex's sandbox cannot start here (containers without user namespaces); the probe printed the reason and fix. No paid call ran; outside coverage is unavailable. Fall back to the Claude subagent path.
- **`ready`** or **`unverified`** — run the Codex pass below. `unverified` means the model check timed out; say so, and let the pass's own verdict decide.
+5
View File
@@ -63,6 +63,11 @@ export const GATE_OUTCOMES = {
summary: 'Codex could not use the selected model',
fix: 'set GSTACK_CODEX_MODEL=<supported-model> or fix model (and base_url for a custom provider) in your Codex config.toml',
},
quota_exhausted: {
state: 'unavailable', anchor: 'codex-quota-exhausted',
summary: "Codex refused the call for this account's usage or rate limit",
fix: "wait for the reset time in Codex's message above, or add credits or a higher plan for that account; the model choice is fine",
},
probe_inconclusive: {
state: 'ran', anchor: 'codex-mode-unverified',
summary: 'the Codex model check timed out or hit a network error, so readiness is unverified',
+8
View File
@@ -43,6 +43,12 @@ const REFUSAL = /\b(?:(?:I (?:cannot|can't|won't|will not|am unable to)|I'm unab
const SANDBOX_FAILURE = /^\s*bwrap: \S.*$|^.*\bbubblewrap is unavailable\b.*$|^.*\blandlock\b.{0,60}\b(?:fail\w*|error|not supported|unsupported)\b.*$|^.*\bseccomp\b.{0,60}\b(?:fail\w*|error)\b.*$|^.*\buser namespaces?\b.{0,80}\b(?:not (?:allowed|permitted|supported)|denied|disabled)\b.*$/im;
/** Exact phrases a reviewer uses when it could not execute; the last fallback, used only without positive evidence. */
const EXECUTION_FAILURE_PHRASE = /\b(?:commands? (?:could not|couldn't|cannot|can't) (?:be )?run|(?:could not|couldn't|was unable to|am unable to|unable to) (?:run (?:any )?(?:shell )?commands|execute (?:any )?commands|inspect the diff|read the diff|access the diff)|the diff could not be (?:read|inspected|accessed)|every (?:shell )?(?:command|invocation) failed)\b/i;
/**
* An account usage or rate limit (#3051). Codex states it plainly ("You've hit
* your usage limit ... try again at <time>"); the line carries the reset time,
* so it is relayed verbatim. Mirrors _QUOTA_SIG in bin/gstack-codex-probe.
*/
export const QUOTA_FAILURE = /^.*(?:usage limit|rate limit|rate_limit_exceeded|insufficient_quota|exceeded your current quota|too many requests|"status":\s*429|status:?\s*429).*$/im;
/** `codex review` transcript on stderr: a command that ran prints " succeeded in Nms:". */
const TRANSCRIPT_SUCCESS = /^\s*succeeded in \d+(?:\.\d+)?m?s:?\s*$/m;
@@ -72,6 +78,8 @@ function execution(input: OutsideReviewInput): OutsideReviewClassification['exec
if (exit !== 0) {
const detail = sandbox(stderr);
if (detail) return { state: 'unavailable', reason: 'sandbox_unavailable', detail };
const quota = exit !== 124 ? stderr.match(QUOTA_FAILURE)?.[0].trim().slice(0, 240) : undefined;
if (quota) return { state: 'unavailable', reason: 'quota_exhausted', detail: quota };
const head = stderrHead(stderr);
return { state: 'unavailable', reason: exit === 124 ? 'timeout' : 'execution_failed', detail: head ? `exit ${exit}: ${head}` : `exit ${exit}` };
}
@@ -378,6 +378,8 @@ else
_CODEX_MODE="sandbox_unavailable"
elif [ "$_CODEX_MP" -eq 2 ]; then
_CODEX_MODE="broken_install"
elif [ "$_CODEX_MP" -eq 4 ]; then
_CODEX_MODE="quota_exhausted"
elif [ "$_CODEX_MP" -ne 0 ]; then
_CODEX_MODE="model_unusable"
elif [ "${_GSTACK_CODEX_PROBE_STATE:-}" = inconclusive ]; then
@@ -396,6 +398,7 @@ Branch on the echoed `CODEX_MODE`:
- **`not_authed`** — installed but no credentials. Print: "Codex not authenticated; outside coverage unavailable. Run `codex login` or set `$CODEX_API_KEY`." Fall back to the Claude subagent path.
- **`broken_install`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: `npm install -g @openai/codex`." Relay the probe's HINT lines. Fall back to the Claude subagent path.
- **`model_unusable`** — the selected model (see `CODEX_MODEL:`) is invalid or unavailable to the account (HTTP 400 on every call). Relay the probe's HINT lines and the fix (`GSTACK_CODEX_MODEL=<supported-model>` or config.toml `model`); never substitute a model. Fall back to the Claude subagent path. The ~10s round trip is cached for 1h.
- **`quota_exhausted`** — Codex usage/rate limit: relay its line verbatim (reset time), no more Codex calls. Fall back to the Claude subagent path.
- **`sandbox_unavailable`** — Codex's sandbox cannot start here (containers without user namespaces); the probe printed the reason and fix. No paid call ran; outside coverage is unavailable. Fall back to the Claude subagent path.
- **`ready`** or **`unverified`** — run the Codex pass below. `unverified` means the model check timed out; say so, and let the pass's own verdict decide.
@@ -317,6 +317,8 @@ else
_CODEX_MODE="sandbox_unavailable"
elif [ "$_CODEX_MP" -eq 2 ]; then
_CODEX_MODE="broken_install"
elif [ "$_CODEX_MP" -eq 4 ]; then
_CODEX_MODE="quota_exhausted"
elif [ "$_CODEX_MP" -ne 0 ]; then
_CODEX_MODE="model_unusable"
elif [ "${_GSTACK_CODEX_PROBE_STATE:-}" = inconclusive ]; then
@@ -335,6 +337,7 @@ Branch on the echoed `CODEX_MODE`:
- **`not_authed`** — installed but no credentials. Print: "Codex not authenticated; outside coverage unavailable. Run `codex login` or set `$CODEX_API_KEY`." Fall back to the Claude subagent path.
- **`broken_install`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: `npm install -g @openai/codex`." Relay the probe's HINT lines. Fall back to the Claude subagent path.
- **`model_unusable`** — the selected model (see `CODEX_MODEL:`) is invalid or unavailable to the account (HTTP 400 on every call). Relay the probe's HINT lines and the fix (`GSTACK_CODEX_MODEL=<supported-model>` or config.toml `model`); never substitute a model. Fall back to the Claude subagent path. The ~10s round trip is cached for 1h.
- **`quota_exhausted`** — Codex usage/rate limit: relay its line verbatim (reset time), no more Codex calls. Fall back to the Claude subagent path.
- **`sandbox_unavailable`** — Codex's sandbox cannot start here (containers without user namespaces); the probe printed the reason and fix. No paid call ran; outside coverage is unavailable. Fall back to the Claude subagent path.
- **`ready`** or **`unverified`** — run the Codex pass below. `unverified` means the model check timed out; say so, and let the pass's own verdict decide.
@@ -823,6 +823,8 @@ else
_CODEX_MODE="sandbox_unavailable"
elif [ "$_CODEX_MP" -eq 2 ]; then
_CODEX_MODE="broken_install"
elif [ "$_CODEX_MP" -eq 4 ]; then
_CODEX_MODE="quota_exhausted"
elif [ "$_CODEX_MP" -ne 0 ]; then
_CODEX_MODE="model_unusable"
elif [ "${_GSTACK_CODEX_PROBE_STATE:-}" = inconclusive ]; then
@@ -841,6 +843,7 @@ Branch on the echoed `CODEX_MODE`:
- **`not_authed`** — installed but no credentials. Print: "Codex not authenticated; outside coverage unavailable. Run `codex login` or set `$CODEX_API_KEY`." Fall back to the Claude subagent path.
- **`broken_install`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: `npm install -g @openai/codex`." Relay the probe's HINT lines. Fall back to the Claude subagent path.
- **`model_unusable`** — the selected model (see `CODEX_MODEL:`) is invalid or unavailable to the account (HTTP 400 on every call). Relay the probe's HINT lines and the fix (`GSTACK_CODEX_MODEL=<supported-model>` or config.toml `model`); never substitute a model. Fall back to the Claude subagent path. The ~10s round trip is cached for 1h.
- **`quota_exhausted`** — Codex usage/rate limit: relay its line verbatim (reset time), no more Codex calls. Fall back to the Claude subagent path.
- **`sandbox_unavailable`** — Codex's sandbox cannot start here (containers without user namespaces); the probe printed the reason and fix. No paid call ran; outside coverage is unavailable. Fall back to the Claude subagent path.
- **`ready`** or **`unverified`** — run the Codex pass below. `unverified` means the model check timed out; say so, and let the pass's own verdict decide.
+3
View File
@@ -40,6 +40,8 @@ else
_CODEX_MODE="sandbox_unavailable"
elif [ "$_CODEX_MP" -eq 2 ]; then
_CODEX_MODE="broken_install"
elif [ "$_CODEX_MP" -eq 4 ]; then
_CODEX_MODE="quota_exhausted"
elif [ "$_CODEX_MP" -ne 0 ]; then
_CODEX_MODE="model_unusable"
elif [ "${_GSTACK_CODEX_PROBE_STATE:-}" = inconclusive ]; then
@@ -58,6 +60,7 @@ Branch on the echoed `CODEX_MODE`:
- **`not_authed`** — installed but no credentials. Print: "Codex not authenticated; outside coverage unavailable. Run `codex login` or set `$CODEX_API_KEY`." Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`broken_install`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: `npm install -g @openai/codex`." Relay the probe's HINT lines. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`model_unusable`** — the selected model (see `CODEX_MODEL:`) is invalid or unavailable to the account (HTTP 400 on every call). Relay the probe's HINT lines and the fix (`GSTACK_CODEX_MODEL=<supported-model>` or config.toml `model`); never substitute a model. Keep the required Claude adversarial pass; do not dispatch a duplicate. The ~10s round trip is cached for 1h.
- **`quota_exhausted`** — Codex usage/rate limit: relay its line verbatim (reset time), no more Codex calls. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`sandbox_unavailable`** — Codex's sandbox cannot start here (containers without user namespaces); the probe printed the reason and fix. No paid call ran; outside coverage is unavailable. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`ready`** or **`unverified`** — run the Codex pass below. `unverified` means the model check timed out; say so, and let the pass's own verdict decide.
+4 -1
View File
@@ -111,7 +111,7 @@ On any error: continue — ${feature} is informational, not a gate.`;
* gates later blocks on the echoed value.
*
* Mode values: `disabled` (config off) | `not_installed` | `not_authed` |
* `broken_install` | `sandbox_unavailable` | `model_unusable` | `unverified` | `ready`.
* `broken_install` | `sandbox_unavailable` | `model_unusable` | `quota_exhausted` | `unverified` | `ready`.
* The path is host-rewritten at gen-skill-docs time (pathRewrites), so the
* literal `~/.claude/skills/gstack` is correct here and becomes `$GSTACK_ROOT`
* etc. for non-Claude hosts.
@@ -154,6 +154,8 @@ else
${m}="sandbox_unavailable"
elif [ "$_CODEX_MP" -eq 2 ]; then
${m}="broken_install"
elif [ "$_CODEX_MP" -eq 4 ]; then
${m}="quota_exhausted"
elif [ "$_CODEX_MP" -ne 0 ]; then
${m}="model_unusable"
elif [ "\${_GSTACK_CODEX_PROBE_STATE:-}" = inconclusive ]; then
@@ -172,6 +174,7 @@ Branch on the echoed \`CODEX_MODE\`:
- **\`not_authed\`** — installed but no credentials. Print: "Codex not authenticated; outside coverage unavailable. Run \`codex login\` or set \`$CODEX_API_KEY\`." ${nativeRoute}
- **\`broken_install\`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: \`npm install -g @openai/codex\`." Relay the probe's HINT lines. ${nativeRoute}
- **\`model_unusable\`** — the selected model (see \`CODEX_MODEL:\`) is invalid or unavailable to the account (HTTP 400 on every call). Relay the probe's HINT lines and the fix (\`GSTACK_CODEX_MODEL=<supported-model>\` or config.toml \`model\`); never substitute a model. ${nativeRoute} The ~10s round trip is cached for 1h.
- **\`quota_exhausted\`** — Codex usage/rate limit: relay its line verbatim (reset time), no more Codex calls. ${nativeRoute}
- **\`sandbox_unavailable\`** — Codex's sandbox cannot start here (containers without user namespaces); the probe printed the reason and fix. No paid call ran; outside coverage is unavailable. ${nativeRoute}
- **\`ready\`** or **\`unverified\`** — run the Codex pass below. \`unverified\` means the model check timed out; say so, and let the pass's own verdict decide.`;
}
+3
View File
@@ -40,6 +40,8 @@ else
_CODEX_MODE="sandbox_unavailable"
elif [ "$_CODEX_MP" -eq 2 ]; then
_CODEX_MODE="broken_install"
elif [ "$_CODEX_MP" -eq 4 ]; then
_CODEX_MODE="quota_exhausted"
elif [ "$_CODEX_MP" -ne 0 ]; then
_CODEX_MODE="model_unusable"
elif [ "${_GSTACK_CODEX_PROBE_STATE:-}" = inconclusive ]; then
@@ -58,6 +60,7 @@ Branch on the echoed `CODEX_MODE`:
- **`not_authed`** — installed but no credentials. Print: "Codex not authenticated; outside coverage unavailable. Run `codex login` or set `$CODEX_API_KEY`." Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`broken_install`** — the CLI is on PATH but cannot execute (spawn ENOENT, non-executable binary, missing vendor payload). Print: "Codex is installed but its binary cannot run — Codex passes skipped. Reinstall: `npm install -g @openai/codex`." Relay the probe's HINT lines. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`model_unusable`** — the selected model (see `CODEX_MODEL:`) is invalid or unavailable to the account (HTTP 400 on every call). Relay the probe's HINT lines and the fix (`GSTACK_CODEX_MODEL=<supported-model>` or config.toml `model`); never substitute a model. Keep the required Claude adversarial pass; do not dispatch a duplicate. The ~10s round trip is cached for 1h.
- **`quota_exhausted`** — Codex usage/rate limit: relay its line verbatim (reset time), no more Codex calls. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`sandbox_unavailable`** — Codex's sandbox cannot start here (containers without user namespaces); the probe printed the reason and fix. No paid call ran; outside coverage is unavailable. Keep the required Claude adversarial pass; do not dispatch a duplicate.
- **`ready`** or **`unverified`** — run the Codex pass below. `unverified` means the model check timed out; say so, and let the pass's own verdict decide.
+49
View File
@@ -53,6 +53,11 @@ case "\${STUB_MODE:-ok}" in
echo 'ERROR: {"type":"error","status":400,"error":{"type":"invalid_request_error","message":"The '"'"'gpt-6-astra'"'"' model is not supported when using Codex with a ChatGPT account."}}' >&2
exit 1 ;;
transient) echo "stream error: network unreachable" >&2; exit 7 ;;
quota)
echo "ERROR: You've hit your usage limit. Upgrade to Pro (https://chatgpt.com/explore/pro), visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Oct 10th, 2026 2:55 AM." >&2
exit 1 ;;
ratelimit429) echo 'ERROR: {"type":"error","status":429,"error":{"type":"rate_limit_exceeded","message":"Rate limit reached"}}' >&2; exit 1 ;;
quotaok) echo "OK (note: you are close to your usage limit)"; exit 0 ;;
retired404)
echo 'ERROR: unexpected status 404 Not Found: The model \`gpt-5.2-codex\` does not exist or you do not have access to it., url: https://chatgpt.com/backend-api/codex/responses' >&2
exit 1 ;;
@@ -438,6 +443,7 @@ describe('B1: codex sandbox preflight and unverified readiness', () => {
['ok', 'transient', 'unverified', 1],
['ok', 'ok', 'ready', 1],
['ok', 'model400', 'model_unusable', 1],
['ok', 'quota', 'quota_exhausted', 1],
] as const) {
const f = makeFixture();
try {
@@ -454,6 +460,49 @@ describe('B1: codex sandbox preflight and unverified readiness', () => {
});
});
describe('#3051: a Codex usage or rate limit is its own state, never "inconclusive; proceeding"', () => {
for (const mode of ['quota', 'ratelimit429'] as const) {
test(`${mode} -> MODEL_QUOTA_EXHAUSTED, exit 4, Codex's own line relayed, cached 15 min, event logged`, () => {
const f = makeFixture();
try {
const r = runProbe(f, mode, { _TEL: 'community' }, '_gstack_codex_model_probe; echo "rc=$?"');
expect(r.stdout).toContain('MODEL_QUOTA_EXHAUSTED');
expect(r.stdout).toContain('rc=4');
expect(r.stdout).not.toContain('MODEL_PROBE_INCONCLUSIVE');
expect(r.stdout).not.toContain('MODEL_UNUSABLE');
if (mode === 'quota') expect(r.stdout).toContain('try again at Oct 10th, 2026 2:55 AM');
expect(fs.readFileSync(path.join(f.gstackHome, 'analytics', 'skill-usage.jsonl'), 'utf-8')).toContain('"event":"codex_quota_exhausted"');
const again = runProbe(f, 'ok', {}, '_gstack_codex_model_probe; echo "rc=$?"');
expect(again.stdout).toContain('MODEL_QUOTA_EXHAUSTED (cached)');
expect(again.stdout).toContain('rc=4');
if (mode === 'quota') expect(again.stdout).toContain('try again at Oct 10th, 2026 2:55 AM');
expect(invocations(f)).toBe(1);
} finally { fs.rmSync(f.home, { recursive: true, force: true }); }
});
}
test('a successful call that mentions a usage limit stays MODEL_OK', () => {
const f = makeFixture();
try {
const r = runProbe(f, 'quotaok', {}, '_gstack_codex_model_probe; echo "rc=$?"');
expect(r.stdout).toContain('MODEL_OK');
expect(r.stdout).toContain('rc=0');
} finally { fs.rmSync(f.home, { recursive: true, force: true }); }
});
test('re-login (auth.json change) re-probes past a cached quota result', () => {
const f = makeFixture();
try {
runProbe(f, 'quota');
const future = new Date(Date.now() + 5000);
fs.utimesSync(path.join(f.codexHome, 'auth.json'), future, future);
const r = runProbe(f, 'ok', {}, '_gstack_codex_model_probe; echo "rc=$?"');
expect(r.stdout).toContain('MODEL_OK');
expect(invocations(f)).toBe(2);
} finally { fs.rmSync(f.home, { recursive: true, force: true }); }
});
});
describe('B2: a 404 model rejection is unusable, never ready (#2843)', () => {
for (const [mode, hint, event] of [
['retired404', "Codex answered 404: either the model is retired or not visible to this account, or a custom provider's base_url", 'codex_model_retired'],
+30
View File
@@ -196,3 +196,33 @@ describe('B1: a review whose sandbox could not start is unavailable, not clean',
expect(r.stderr).toStartWith(`Codex outside review unavailable: Codex's sandbox could not start here (${BWRAP}`);
});
});
describe('#3051: a mid-run Codex usage limit reads as unavailable (quota), with Codex\'s own line', () => {
const LINE = "ERROR: You've hit your usage limit. Upgrade to Pro (https://chatgpt.com/explore/pro), visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Oct 10th, 2026 2:55 AM.";
test('failed call with the usage-limit line -> unavailable, reason quota_exhausted, detail is the line', () => {
const result = classifyOutsideReview({ text: '', stderr: `Reading prompt from stdin...\n${LINE}\n`, exit: 1, gate: 'review' });
expect([result.verdict, result.reason]).toEqual(['unavailable', 'quota_exhausted']);
expect(result.detail).toBe(LINE);
});
test('a timeout whose partial stderr mentions a rate limit keeps the timeout reason', () => {
const result = classifyOutsideReview({ text: '', stderr: 'retrying after rate limit\n', exit: 124, gate: 'review' });
expect(result.reason).toBe('timeout');
});
test('a completed review that discusses rate limits is not unavailable', () => {
const text = '[P2] the client ignores the API rate limit header\nRecommendation: fix the retry loop because it ignores Retry-After';
const result = classifyOutsideReview({ text, stderr: '', exit: 0, gate: 'review' });
expect(result.execution.state).toBe('ran');
});
test('the probe and the classifier recognise the same Codex lines', () => {
const probe = fs.readFileSync(path.join(ROOT, 'bin', 'gstack-codex-probe'), 'utf8').match(/_QUOTA_SIG="([^\n]+)"\n/)![1]!;
for (const line of [LINE, 'ERROR: {"type":"error","status":429,"error":{"type":"rate_limit_exceeded"}}', 'insufficient_quota: You exceeded your current quota']) {
const bash = spawnSync('bash', ['-c', `printf '%s' "$1" | grep -qiE "$2"`, '_', line, probe.replace(/\\"/g, '"')], { timeout: 5000 });
expect({ line, bash: bash.status }).toEqual({ line, bash: 0 });
expect(classifyOutsideReview({ text: '', stderr: line, exit: 1, gate: 'review' }).reason).toBe('quota_exhausted');
}
});
});