fix(ci): kill the three zero-test eval jobs (hollow green)

- delete the vestigial e2e-codex / e2e-gemini matrix rows: both files
  are whole-file periodic-tier, so with no row tier: they ran ZERO
  tests and reported green on every PR (~2 min of runner each, pure
  false confidence; the periodic lane owns those suites)
- e2e-pty-plan-smoke gains tier: gate — its two files are whole-file
  describeE2ETier('gate'), so the job burned ~7 min of container setup
  then skipped every describe
- KNOWN_TIER_UNSET burned down to empty; the ratchet stays armed so a
  future row/file tier mismatch fails the suite instead of shipping
  hollow green

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-29 04:37:37 +00:00
co-authored by Claude Fable 5
parent 29d94a505d
commit 9fbd0700ff
2 changed files with 13 additions and 21 deletions
+8 -4
View File
@@ -148,10 +148,10 @@ jobs:
tier: gate tier: gate
- name: e2e-routing - name: e2e-routing
file: test/skill-routing-e2e.test.ts file: test/skill-routing-e2e.test.ts
- name: e2e-codex # (e2e-codex / e2e-gemini rows deleted: both files are whole-file
file: test/codex-e2e.test.ts # periodic-tier, so with no row tier: they ran ZERO tests and
- name: e2e-gemini # reported green on every PR — ~2 min of runner per PR of pure
file: test/gemini-e2e.test.ts # false confidence. The periodic lane owns these suites.)
# Real-PTY plan-mode smokes. Only the deterministically-reliable ones # Real-PTY plan-mode smokes. Only the deterministically-reliable ones
# are CI-gated: office-hours (asks its mode question first, caught by # are CI-gated: office-hours (asks its mode question first, caught by
# the collapsed/bullet prose-AUQ detector) and plan-mode-no-op (no # the collapsed/bullet prose-AUQ detector) and plan-mode-no-op (no
@@ -161,6 +161,10 @@ jobs:
# wedge on the fresh-container onboarding/API-key dialog. # wedge on the fresh-container onboarding/API-key dialog.
- name: e2e-pty-plan-smoke - name: e2e-pty-plan-smoke
file: test/skill-e2e-office-hours-auto-mode.test.ts test/skill-e2e-plan-mode-no-op.test.ts file: test/skill-e2e-office-hours-auto-mode.test.ts test/skill-e2e-plan-mode-no-op.test.ts
# Both files are whole-file describeE2ETier('gate') — without this
# row tier: the job burned ~7 min of setup then skipped every
# describe (hollow-green since the files adopted the self-gate).
tier: gate
timeout: 35 timeout: 35
# The documented contention-heavy PTY family: ROTATING members # The documented contention-heavy PTY family: ROTATING members
# failed attempt 2 in consecutive PR #2593 rounds # failed attempt 2 in consecutive PR #2593 rounds
+5 -17
View File
@@ -52,24 +52,12 @@ const KNOWN_MATRIX_GAPS = new Set([
/** /**
* Matrix files whose whole-file tier guard has no matching row `tier:` * Matrix files whose whole-file tier guard has no matching row `tier:`
* property (pre-existing, found 2026-08-26). Consequences today: * property. Burned down to empty 2026-08-29: the vestigial codex/gemini rows
* - codex-e2e / gemini-e2e declare 'periodic' → both jobs run ZERO tests and * were deleted (periodic-tier files, zero tests per PR) and
* report green on every PR (vestigial rows; the periodic cron lane owns * e2e-pty-plan-smoke gained its `tier: gate`. The ratchet stays so a future
* these suites). * row/file tier mismatch fails the suite instead of shipping hollow green.
* - the two PTY plan-mode smokes declare 'gate' → the e2e-pty-plan-smoke job
* spends ~7 min on container setup and skill registration, then bun test
* skips every describe — hollow-green since the files adopted
* describeE2ETier.
* Fixing either means deliberately (re)activating paid suites on every PR —
* tracked in the same TODOS burn-down. Fix = add `tier:` to the row (or
* delete the vestigial row), then DELETE the entry here.
*/ */
const KNOWN_TIER_UNSET = new Map([ const KNOWN_TIER_UNSET = new Map<string, string>([]);
['test/codex-e2e.test.ts', 'periodic'],
['test/gemini-e2e.test.ts', 'periodic'],
['test/skill-e2e-office-hours-auto-mode.test.ts', 'gate'],
['test/skill-e2e-plan-mode-no-op.test.ts', 'gate'],
]);
interface MatrixRow { interface MatrixRow {
name: string; name: string;