Second-model review keeps the reviewer's result and reports budget stops (GST-3181) (#4515)

When Codex reviews a change, gstack now keeps the review file the
reviewer wrote instead of replacing it with Codex's one-line closing
message, and a provider that refuses with HTTP 402 is reported as a
used-up budget instead of "exit 1" or a login failure.

Why: gstack-evals lost 22 review attempts to the overwrite and 5 of 45
sessions to a 402 from a metering proxy whose cause gstack threw away.

Nothing to do: a 401 or 403 is still a login problem. Open notes, one of
them a known gap, are listed below.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Review-Finding: p1/specialist-batched#1 reader=testing
Review-Finding: p1/red-team#1 reader=red-team
Review-Finding: p1/red-team#2 reader=red-team
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-11-19 10:27:37 +00:00
1 parent f13dbb0d4a
commit 957fab79c5
14 files changed
+311 -10

No files matched your search

+7 -1
View File
@@ -560,7 +560,13 @@ If token count is not available, display: `Tokens: unknown`
## Error Handling
- **Binary not found:** Detected in Step 0. Stop with install instructions.
- **Auth error:** Codex prints an auth error to stderr. Surface the error:
- **Budget or quota stop (HTTP 402, payment required, quota):** the validator prints
`unavailable: the provider refused the review with HTTP 402` or names the quota. This is
not an auth error, even when the provider's message mentions the API key. Relay Codex's line
and tell the user: "Codex refused the call because its budget or quota is used up. Check the
allowance on the proxy or gateway in front of the provider, or the account's quota and
credits; logging in again will not fix it."
- **Auth error:** Codex prints an auth error to stderr (401 or 403). Surface the error:
"Codex authentication failed. Run `codex login` in your terminal to authenticate via ChatGPT."
- **Timeout (Bash outer gate):** Every Bash gate sits ABOVE its inner wrapper (360s gate
over the 330s review wrapper; 600s gate, the tool maximum, over the 540s challenge/consult
+7 -1
View File
@@ -281,7 +281,13 @@ If token count is not available, display: `Tokens: unknown`
## Error Handling
- **Binary not found:** Detected in Step 0. Stop with install instructions.
- **Auth error:** Codex prints an auth error to stderr. Surface the error:
- **Budget or quota stop (HTTP 402, payment required, quota):** the validator prints
`unavailable: the provider refused the review with HTTP 402` or names the quota. This is
not an auth error, even when the provider's message mentions the API key. Relay Codex's line
and tell the user: "Codex refused the call because its budget or quota is used up. Check the
allowance on the proxy or gateway in front of the provider, or the account's quota and
credits; logging in again will not fix it."
- **Auth error:** Codex prints an auth error to stderr (401 or 403). Surface the error:
"Codex authentication failed. Run `codex login` in your terminal to authenticate via ChatGPT."
- **Timeout (Bash outer gate):** Every Bash gate sits ABOVE its inner wrapper (360s gate
over the 330s review wrapper; 600s gate, the tool maximum, over the 540s challenge/consult