evals: kill the 3,372-line zombie monolith; revive 4 never-run tests

test/skill-e2e.test.ts survived the v1.56 split as a zombie: the paid
glob needs the skill-e2e-* hyphen, so with EVALS=1 NOTHING has executed
it for ~8 releases — and it held the ONLY implementations of four
map-registered tests: review-coverage-audit (gate), plan-eng-coverage-
audit (gate), ship-triage (gate), ship-idempotency (periodic). Three
gate tests silently never ran — the exact 0%-execution class this
branch exists to kill.

Rehomed into test/skill-e2e-coverage-audit.test.ts, -triage.test.ts, and
-ship-idempotency-sdk.test.ts with bodies byte-identical modulo collector
wiring and fixture extraction (drift observed in the skills since v1.56
is DOCUMENTED in each header, not fixed — their first paid run in 8
releases must attribute failures to drift, not to this move). All 24
other monolith names were true duplicates of the split files — dropped
with the monolith. Matrix rows added to both eval workflows; the paid
glob's zombie-exclusion is now a commented regression pin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-15 09:03:16 -07:00
co-authored by Claude Fable 5
parent 9b8f8d7767
commit 1e0ff1e1e2
11 changed files with 599 additions and 3381 deletions
+1 -1
View File
@@ -3,7 +3,7 @@
*
* Spawns `codex exec` with skills installed in a temp HOME, parses JSONL
* output, and validates structured results. Follows the same pattern as
* skill-e2e.test.ts but adapted for Codex CLI.
* the skill-e2e-*.test.ts suites but adapted for Codex CLI.
*
* Prerequisites:
* - `codex` binary installed (npm install -g @openai/codex)