Files
gstack/test/fixtures/credential-rejection-39696775546.json
Garry Tanandcapy-ai[bot] 9c7b4c81ca v1.91.183.0 fix(evals): a rejected API key is INFRA, never 131 assertion reds and reruns (#3361)
* fix(evals): a rejected API key is INFRA, never 131 assertion reds and reruns

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: register the credential-rejection fixture as free

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: keep the credential-rejection CLI replay off the Windows subset, like paid-rerun

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: name the Windows run in the curation reason

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* v1.91.183.0 fix(evals): a rejected API key is INFRA, never 131 assertion reds and reruns (#3361)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-23 18:05:39 +00:00

62 lines
3.1 KiB
JSON
Raw Permalink Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"source": "GitHub Actions run 39696775546 (train gate, 2026-10-22): every gated case failed on a rejected ANTHROPIC_API_KEY; trimmed captures",
"claude_p_events": [
{
"type": "system",
"subtype": "status",
"status": "requesting",
"session_id": "fc99f0a9-26d1-4c20-b835-7e2c7bd5a629"
},
{
"type": "assistant",
"message": {
"id": "c885e094-9dfe-45d9-ba48-1d7955f9ba48",
"model": "<synthetic>",
"role": "assistant",
"stop_reason": "stop_sequence",
"type": "message",
"content": [
{
"type": "text",
"text": "Invalid API key · Fix external API key"
}
]
},
"parent_tool_use_id": null,
"session_id": "fc99f0a9-26d1-4c20-b835-7e2c7bd5a629",
"error": "authentication_failed",
"is_api_error_message": true
},
{
"type": "result",
"subtype": "success",
"is_error": true,
"api_error_status": 401,
"terminal_reason": "api_error",
"num_turns": 1,
"total_cost_usd": 0,
"result": "Invalid API key · Fix external API key",
"session_id": "fc99f0a9-26d1-4c20-b835-7e2c7bd5a629"
}
],
"claude_p_record": {
"name": "ship-docsync-store",
"suite": "Ship doc-sync lifecycle",
"tier": "e2e",
"passed": false,
"duration_ms": 1540,
"cost_usd": 0,
"output": "Invalid API key · Fix external API key",
"turns_used": 1,
"tokens_used": 0,
"browse_errors": [],
"exit_reason": "exit_code_1",
"model": "claude-opus-5-5"
},
"claude_p_junit_message": "expect(received).toBeGreaterThanOrEqual(expected)\n\nExpected: >= 0\nReceived: -1\n",
"pty_junit_message": "plan-ceo-review smoke FAILED: outcome=timeout\nTimeout means the agent neither asked nor completed within the budget — likely hung mid-question or stuck on a permission dialog.\nsummary: no terminal outcome within 420000ms (last LLM judge: state=unknown — claude exited 1 ())\nelapsed: 420002ms\n--- evidence (last 2KB visible) ---\n",
"pty_screen": "\r❯ /plan-ceo-review\r\r✢ Ebbing… \r\r\r\n────────────────────────────────────────\r\r\n❯  \r\r\n────────────────────────────────────────\r\r\n⏸planmodeon (shift+tabtocycle)·esctointerrupt·←foragents◐medium·/effort\r\r\n\r·n\r\r\n\r ⎿  Invalid API key · Fix external API key\r\r✻Baked for 0s · done 12:00 AM\r\r────────────────────────────────────────\r❯  \r────────────────────────────────────────\r\r\n⏸planmodeon (shift+tabtocycle)·←foragents◐medium·/effort\r\r\n\r\r\r\n",
"judge_error": "401 {\"type\":\"error\",\"error\":{\"type\":\"authentication_error\",\"message\":\"invalid x-api-key\"},\"request_id\":\"req_011CgGHuinAhnU3wapQEbUe1\"}",
"agent_sdk_error": "Error: Claude Code returned an error result: Invalid API key · Fix external API key"
}