feat: delete the legacy 17-row eval matrix — the sliced lane is the only paid lane

Every PR paid twice: the hand-enumerated matrix (18 test files, 22.6 min,
~$21 API measured on run 33263204465) ran serialized AHEAD of the strictly
superior sliced lane via 'needs: evals' — 35.5 min wall and ~2x paid spend
for the same diff. 14 of 17 rows carried no tier:, so periodic Opus
benchmarks leaked into every PR (the e2e-plan row alone: 12/12 tests,
21.7 min, $7.28 — the wall-clock bound of ALL of CI).

Parity receipt (static, pre-deletion): the sliced lane's gate census (49
files, derived from the runner itself) strictly contains all 18 matrix test
files, plus 31 files the matrix never ran. Pure deletion — one revert
restores it. The PR comment moved into slices-report (same '## E2E Evals'
upsert marker, now sourced from slice artifacts + carrying the fail-closed
reconciliation verdict). plan-slices loses the needs edge; the dead
workflow-level EVALS_TIER env goes with it.

test/evals-workflow-matrix.test.ts (and its KNOWN_MATRIX_GAPS /
KNOWN_TIER_UNSET burn-down ratchets — retired: the sliced census makes
'every gate file runs' true by construction) is rewritten as
test/evals-workflow-wiring.test.ts: matrix stays deleted, planner/executor/
report tier + slice-count agreement, both surviving lanes on the shared
register-skills composite with its fail-fast verification loop, PR comment
survival. Expected: PR eval wall 35.5 -> ~13 min, per-PR paid spend ~halved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

test/e2e-tier-alignment.test.ts

@@ -42,8 +42,8 @@ const SELF_GATE_RE = /EVALS_TIER\s*===\s*['"](gate|periodic)['"]/g;
 const HELPER_GATE_RE = /\b(?:describeE2ETier|e2eTierEnabled)\(\s*['"](gate|periodic)['"]/g;
 
 /**
- * Ratchet, not amnesty (same contract as KNOWN_MATRIX_GAPS in
- * test/evals-workflow-matrix.test.ts): skill-e2e files that are named in NO
+ * Ratchet, not amnesty (the contract KNOWN_MATRIX_GAPS pioneered before the
+ * legacy eval matrix and its test retired): skill-e2e files that are named in NO
  * touchfiles dep list because no E2E map key exists for them. Every entry
  * carries a one-line reason. Do NOT add new files here — give the test an
  * E2E map key (touchfiles + tier) and register the file in its dep list.
@@ -141,8 +141,8 @@ describe('E2E tier alignment (touchfiles declaration vs test self-gate)', () =>
     expect(misaligned).toEqual([]);
   });
 
-  // Ratchet cleanup enforcement (same contract as evals-workflow-matrix's
-  // burn-down test): a KNOWN_UNREGISTERED entry whose file was deleted, or
+  // Ratchet cleanup enforcement (same shrink-only contract as the retired
+  // matrix test's burn-down): a KNOWN_UNREGISTERED entry whose file was deleted, or
   // whose file is now named in a dep list, is stale — delete the entry so
   // the set can only shrink.
   test('KNOWN_UNREGISTERED holds only live, still-unregistered files', () => {

test/evals-workflow-matrix.test.ts

@@ -1,197 +0,0 @@
-/**
- * CI eval-matrix completeness tripwire — kills the silent-skip class where a
- * gate-tier test exists in the repo but the hand-enumerated matrix in
- * .github/workflows/evals.yml never runs it, so "gate tier blocks merge" is
- * quietly false in CI. This has happened before (see the "rehomed from the
- * deleted pre-split monolith" comment in evals.yml) and was found again on
- * PR #2700: nine gate-hosting files absent from the matrix, plus matrix rows
- * whose whole-file tier guards can never fire because the Run step exported
- * no EVALS_TIER.
- *
- * Ratchet, not amnesty: the KNOWN_* lists below enumerate the PRE-EXISTING
- * gaps with reasons, so no NEW gap can land while the backlog burns down
- * (same pattern as SCANNER_EXEMPT in egress-receipt-wiring). If you fix a
- * listed gap (add its matrix row / tier property), this test FAILS until you
- * remove the entry — stale exemptions are enforced, not decorative.
- *
- * Wiring pinned:
- *  - every matrix `file:` path exists on disk (no stale rows),
- *  - every gate-hosting paid file (whole-file gate self-gate, or named in the
- *    dep list of a gate-tier E2E_TOUCHFILES key) appears in the matrix or in
- *    KNOWN_MATRIX_GAPS,
- *  - every matrix file with a whole-file tier guard has a matching row-level
- *    `tier:` property (else the suite self-skips and the job is hollow-green)
- *    or sits in KNOWN_TIER_UNSET.
- */
-import { describe, test, expect } from 'bun:test';
-import * as fs from 'fs';
-import * as path from 'path';
-import { E2E_TOUCHFILES, E2E_TIERS } from './helpers/touchfiles-data';
-import { isPaidTestFile } from './helpers/paid-test-set';
-
-const ROOT = path.join(import.meta.dir, '..');
-const WORKFLOW = path.join(ROOT, '.github', 'workflows', 'evals.yml');
-
-/**
- * Pre-existing gate-hosting files with no matrix row (found 2026-08-26,
- * PR #2700). Adding a row activates real paid runs on every PR — a cost and
- * flake-surface decision per file, tracked in TODOS.md ("CI gate-lane
- * hollow-coverage burn-down"). Fix = add a matrix row (plus `tier: gate` when
- * the file is whole-file gated), then DELETE the entry here.
- */
-const KNOWN_MATRIX_GAPS = new Set([
-  'test/skill-e2e-ask-user-question-format-compliance.test.ts',
-  'test/skill-e2e-hermetic-canary.test.ts',
-  'test/skill-e2e-ios.test.ts',
-  'test/skill-e2e-plan-ceo-finding-floor.test.ts',
-  'test/skill-e2e-plan-ceo-plan-mode.test.ts',
-  'test/skill-e2e-plan-design-with-ui.test.ts',
-  'test/skill-e2e-plan-devex-finding-floor.test.ts',
-  'test/skill-e2e-plan-devex-plan-mode.test.ts',
-  // Exposed by the 2026-08 dep-list self-registration sweep: these eight had
-  // zero gate-key dep-list membership before it, so the census never saw
-  // them as gate-hosting. Their gate tests run in NO CI lane today. The
-  // paid-lane re-platform (test-paid-shards.ts as the CI engine) runs every
-  // gate-tier file by construction and retires this whole ratchet.
-  'test/skill-e2e-cso.test.ts',
-  'test/skill-e2e-diagram.test.ts',
-  'test/skill-e2e-learnings.test.ts',
-  'test/skill-e2e-plan-tune.test.ts',
-  'test/skill-e2e-plan-tune-cathedral.test.ts',
-  'test/skill-e2e-review-army.test.ts',
-  'test/skill-e2e-session-intelligence.test.ts',
-  'test/skill-e2e-skillify.test.ts',
-]);
-
-/**
- * Matrix files whose whole-file tier guard has no matching row `tier:`
- * property. Burned down to empty 2026-08-29: the vestigial codex/gemini rows
- * were deleted (periodic-tier files, zero tests per PR) and
- * e2e-pty-plan-smoke gained its `tier: gate`. The ratchet stays so a future
- * row/file tier mismatch fails the suite instead of shipping hollow green.
- */
-const KNOWN_TIER_UNSET = new Map<string, string>([]);
-
-interface MatrixRow {
-  name: string;
-  files: string[];
-  tier?: string;
-}
-
-/** Parse the `matrix: suite:` rows (name / file / optional tier) from evals.yml. */
-function parseMatrixRows(source: string): MatrixRow[] {
-  const rows: MatrixRow[] = [];
-  let current: MatrixRow | null = null;
-  for (const line of source.split('\n')) {
-    const name = line.match(/^\s+- name: (\S+)\s*$/);
-    if (name) {
-      if (current) rows.push(current);
-      current = { name: name[1], files: [] };
-      continue;
-    }
-    if (!current) continue;
-    const file = line.match(/^\s+file: (.+?)\s*$/);
-    if (file) current.files.push(...file[1].trim().split(/\s+/));
-    const tier = line.match(/^\s+tier: (\S+)\s*$/);
-    if (tier) current.tier = tier[1];
-    // `steps:` ends the strategy block — stop before step-level keys leak in.
-    if (/^\s{4}steps:\s*$/.test(line)) break;
-  }
-  if (current) rows.push(current);
-  return rows.filter((r) => r.files.length > 0);
-}
-
-const wholeFileTier = (source: string): string | null => {
-  const m =
-    /\b(?:describeE2ETier|e2eTierEnabled)\(\s*['"`](gate|periodic)['"`]/.exec(source) ||
-    /EVALS_TIER\s*===\s*['"`](gate|periodic)['"`]/.exec(source);
-  return m ? m[1] : null;
-};
-
-const workflowSource = fs.readFileSync(WORKFLOW, 'utf-8');
-const rows = parseMatrixRows(workflowSource);
-const matrixFiles = new Map<string, MatrixRow>();
-for (const row of rows) for (const f of row.files) matrixFiles.set(f, row);
-
-const paidFiles = fs
-  .readdirSync(path.join(ROOT, 'test'))
-  .filter((f) => f.endsWith('.test.ts'))
-  .map((f) => `test/${f}`)
-  .filter(isPaidTestFile);
-
-describe('evals.yml matrix completeness (gate-lane silent-skip tripwire)', () => {
-  test('matrix parse sanity: rows and known suites present', () => {
-    expect(rows.length).toBeGreaterThanOrEqual(15);
-    expect(matrixFiles.has('test/skill-e2e-workflow.test.ts')).toBe(true);
-    expect(matrixFiles.has('test/skill-e2e-ship-docsync.test.ts')).toBe(true);
-  });
-
-  test('every matrix file exists on disk', () => {
-    const missing = [...matrixFiles.keys()].filter(
-      (f) => !fs.existsSync(path.join(ROOT, f))
-    );
-    expect(missing).toEqual([]);
-  });
-
-  test('every gate-hosting paid file is in the matrix (or the documented backlog)', () => {
-    const gaps: string[] = [];
-    for (const file of paidFiles) {
-      const source = fs.readFileSync(path.join(ROOT, file), 'utf-8');
-      const declaresGate = wholeFileTier(source) === 'gate';
-      const inGateDeps = Object.entries(E2E_TOUCHFILES).some(
-        ([key, deps]) =>
-          (E2E_TIERS as Record<string, string>)[key] === 'gate' &&
-          (deps as string[]).includes(file)
-      );
-      if (!declaresGate && !inGateDeps) continue;
-      if (matrixFiles.has(file) || KNOWN_MATRIX_GAPS.has(file)) continue;
-      gaps.push(file);
-    }
-    expect(
-      gaps,
-      `Gate-hosting test file(s) missing from the evals.yml matrix — CI will ` +
-        `never run them and "gate tier blocks merge" becomes silently false. ` +
-        `Add a matrix row (with tier: gate when the file is whole-file gated). ` +
-        `Do NOT extend KNOWN_MATRIX_GAPS for new files.`
-    ).toEqual([]);
-  });
-
-  test('matrix rows for whole-file-gated files carry a matching tier property', () => {
-    const mismatches: string[] = [];
-    for (const [file, row] of matrixFiles) {
-      if (!fs.existsSync(path.join(ROOT, file))) continue;
-      const declared = wholeFileTier(fs.readFileSync(path.join(ROOT, file), 'utf-8'));
-      if (!declared) continue;
-      if (row.tier === declared) continue;
-      if (KNOWN_TIER_UNSET.get(file) === declared && row.tier === undefined) continue;
-      mismatches.push(`${file} declares '${declared}' but row '${row.name}' has tier: ${row.tier ?? 'unset'}`);
-    }
-    expect(
-      mismatches,
-      `A whole-file tier guard with no matching row tier means the suite ` +
-        `self-skips and the CI job reports a hollow green. Set tier: <declared> ` +
-        `on the row (the Run step exports it as EVALS_TIER).`
-    ).toEqual([]);
-  });
-
-  test('burn-down lists hold only live gaps (ratchet cleanup enforcement)', () => {
-    const staleGaps = [...KNOWN_MATRIX_GAPS].filter(
-      (f) => matrixFiles.has(f) || !fs.existsSync(path.join(ROOT, f))
-    );
-    expect(
-      staleGaps,
-      'Entry fixed or file removed — delete it from KNOWN_MATRIX_GAPS.'
-    ).toEqual([]);
-    const staleTiers = [...KNOWN_TIER_UNSET.entries()].filter(([f, declared]) => {
-      const row = matrixFiles.get(f);
-      if (!row) return true; // row deleted — entry no longer applies
-      if (row.tier === declared) return true; // fixed — entry must go
-      if (!fs.existsSync(path.join(ROOT, f))) return true;
-      return wholeFileTier(fs.readFileSync(path.join(ROOT, f), 'utf-8')) !== declared;
-    });
-    expect(
-      staleTiers.map(([f]) => f),
-      'Entry fixed, row removed, or guard changed — delete it from KNOWN_TIER_UNSET.'
-    ).toEqual([]);
-  });
-});

test/evals-workflow-wiring.test.ts

@@ -0,0 +1,131 @@
+/**
+ * Sliced-lane wiring pins for the paid CI workflows — the successor to
+ * evals-workflow-matrix.test.ts, which enforced completeness of a
+ * hand-enumerated 17-row matrix (and carried KNOWN_MATRIX_GAPS /
+ * KNOWN_TIER_UNSET burn-down ratchets for the files that matrix missed).
+ * The matrix is deleted: the sliced lane's planner derives the gate census
+ * from the runner itself (collectPaidTestFiles + tier selection), so "every
+ * gate-hosting file is in the census" is true BY CONSTRUCTION and the
+ * burn-down ratchets retired with the rows.
+ *
+ * What still needs pinning is the WIRING — the yml plumbing that free tests
+ * are the only guard for:
+ *   - the legacy matrix (and its `needs: evals` serialization) stays deleted,
+ *   - planner/executor/report all run tier=gate and agree on the slice count,
+ *   - both surviving lanes register skills through the SHARED composite that
+ *     carries the fail-fast dangling-symlink/frontmatter verification loop
+ *     (the sliced + periodic copies had silently dropped it — the loop was
+ *     written after a silent "Unknown command" + 35-min-timeout incident),
+ *   - the PR comment survives the matrix-report deletion (it moved into
+ *     slices-report, keyed on the same "## E2E Evals" upsert marker).
+ */
+import { describe, test, expect } from 'bun:test';
+import * as fs from 'fs';
+import * as path from 'path';
+
+const ROOT = path.join(import.meta.dir, '..');
+const read = (rel: string) => fs.readFileSync(path.join(ROOT, rel), 'utf-8');
+
+const evalsYml = read('.github/workflows/evals.yml');
+const periodicYml = read('.github/workflows/evals-periodic.yml');
+const registerAction = read('.github/actions/register-gstack-skills/action.yml');
+
+/** Slice count the planner emits (`--slices N`) in a workflow source. */
+function plannedSlices(source: string): number[] {
+  return [...source.matchAll(/--emit-plan\s+\S+\s+--slices\s+(\d+)/g)].map((m) => Number(m[1]));
+}
+
+/** The executor matrix's slice list (`slice: [1, 2, ...]`). */
+function matrixSlices(source: string): number[][] {
+  return [...source.matchAll(/^\s+slice: \[([\d,\s]+)\]\s*$/gm)].map((m) =>
+    m[1].split(',').map((n) => Number(n.trim())),
+  );
+}
+
+describe('evals.yml sliced-lane wiring (post-matrix)', () => {
+  test('the legacy matrix job stays deleted', () => {
+    // Row-enumeration shapes from the deleted matrix. Any reappearance means
+    // someone is re-growing a hand-maintained enumeration next to a lane
+    // whose census is derived — the drift class the deletion killed.
+    expect(evalsYml).not.toMatch(/^\s+suite:\s*$/m);
+    expect(evalsYml).not.toMatch(/^\s+file: test\//m);
+    expect(evalsYml).not.toContain('needs: [build-image, evals]');
+    expect(evalsYml).not.toMatch(/^\s+needs: evals\s*$/m);
+  });
+
+  test('no workflow-level EVALS_TIER env (each command sets its own)', () => {
+    // The workflow-level `EVALS_TIER: gate` was dead config once every
+    // consumer set its own; a resurrected copy would silently leak gate
+    // semantics into steps that must choose explicitly.
+    expect(evalsYml).not.toMatch(/^env:[\s\S]{0,120}^\s+EVALS_TIER:/m);
+  });
+
+  test('planner, executors, and report all run tier=gate on the shared runner', () => {
+    expect(evalsYml).toMatch(/EVALS_TIER=gate bun run scripts\/test-paid-shards\.ts --tier gate --emit-plan/);
+    expect(evalsYml).toMatch(/EVALS_TIER=gate bun run scripts\/test-paid-shards\.ts --tier gate --plan .* --slice /);
+    expect(evalsYml).toMatch(/EVALS_TIER=gate bun run scripts\/test-paid-shards\.ts --tier gate --report /);
+  });
+
+  test('executor matrix slice list matches the planner --slices count', () => {
+    const planned = plannedSlices(evalsYml);
+    const matrices = matrixSlices(evalsYml);
+    expect(planned, 'expected exactly one --emit-plan site in evals.yml').toHaveLength(1);
+    expect(matrices, 'expected exactly one slice matrix in evals.yml').toHaveLength(1);
+    const n = planned[0];
+    expect(matrices[0]).toEqual(Array.from({ length: n }, (_, i) => i + 1));
+  });
+
+  test('the PR comment survived the matrix-report deletion (moved to slices-report)', () => {
+    // Keyed on the upsert marker so the migration keeps updating the SAME
+    // comment; and the job holding it needs the issues permission (#1802).
+    expect(evalsYml).toContain('## E2E Evals');
+    expect(evalsYml).toMatch(/pull-requests: write/);
+    expect(evalsYml).toMatch(/issues: write/);
+  });
+});
+
+describe('evals-periodic.yml sliced-lane wiring', () => {
+  test('planner/executor/report tier=periodic and slice counts agree', () => {
+    expect(periodicYml).toMatch(/EVALS_TIER=periodic bun run scripts\/test-paid-shards\.ts --tier periodic --emit-plan/);
+    expect(periodicYml).toMatch(/EVALS_TIER=periodic bun run scripts\/test-paid-shards\.ts --tier periodic --plan .* --slice /);
+    expect(periodicYml).toMatch(/EVALS_TIER=periodic bun run scripts\/test-paid-shards\.ts --tier periodic --report /);
+    const planned = plannedSlices(periodicYml);
+    const matrices = matrixSlices(periodicYml);
+    expect(planned).toHaveLength(1);
+    expect(matrices).toHaveLength(1);
+    expect(matrices[0]).toEqual(Array.from({ length: planned[0] }, (_, i) => i + 1));
+  });
+});
+
+describe('shared setup composites (both surviving lanes)', () => {
+  test('both lanes register skills through the shared composite', () => {
+    for (const [name, source] of [['evals.yml', evalsYml], ['evals-periodic.yml', periodicYml]] as const) {
+      expect(source, `${name} must use the register-gstack-skills composite`)
+        .toContain('uses: ./.github/actions/register-gstack-skills');
+      // No inline re-implementation creeping back beside the composite.
+      expect(source, `${name} re-inlines the skill registry instead of using the composite`)
+        .not.toContain('ln -snf "$REPO" "$SKILLS_DIR/gstack"');
+    }
+  });
+
+  test('the register composite carries the fail-fast verification loop', () => {
+    // The loop is the POINT of the composite: a dangling symlink or renamed
+    // committed target fails in seconds with a named path, never as a wedged
+    // PTY session at the shard wall. Pin its load-bearing markers.
+    expect(registerAction).toContain('skill registry OK');
+    expect(registerAction).toContain('skill-registry target missing');
+    expect(registerAction).toContain('gstack root symlink dangles');
+    expect(registerAction).toMatch(/grep -m1 "\^name: \$s\\\$"/);
+  });
+
+  test('seed/deps/temp composites exist and both lanes use them', () => {
+    for (const action of ['seed-claude-config', 'restore-deps', 'fix-bun-temp']) {
+      expect(fs.existsSync(path.join(ROOT, '.github', 'actions', action, 'action.yml')), `missing composite: ${action}`).toBe(true);
+    }
+    for (const [name, source] of [['evals.yml', evalsYml], ['evals-periodic.yml', periodicYml]] as const) {
+      expect(source, `${name} must use seed-claude-config`).toContain('uses: ./.github/actions/seed-claude-config');
+      expect(source, `${name} must use restore-deps`).toContain('uses: ./.github/actions/restore-deps');
+      expect(source, `${name} must use fix-bun-temp`).toContain('uses: ./.github/actions/fix-bun-temp');
+    }
+  });
+});

test/paid-orphan-tripwire.test.ts

@@ -7,8 +7,9 @@
  * execute anywhere — forever, silently. Four files were in that state
  * (codex-e2e-plan-format, codex-e2e-recommendation-substance,
  * llm-judge-recommendation, carve-section-loading), and the tripwire built
- * for the adjacent class (test/evals-workflow-matrix.test.ts) couldn't see
- * them because it filters on isPaidTestFile() FIRST.
+ * for the adjacent class (the since-retired evals-workflow-matrix test;
+ * successor: test/evals-workflow-wiring.test.ts) couldn't see them because
+ * it filtered on isPaidTestFile() FIRST.
  *
  * Detection is over source text, so meta-tests and helpers that mention the
  * gate patterns need reasoned exemptions (same convention as
@@ -30,7 +31,7 @@ const SCANNER_EXEMPT = new Map<string, string>([
   // Meta-tests that quote gate-pattern strings to test classification:
   ['test/helpers/e2e-gate.unit.test.ts', 'free unit test OF the gate predicates (env stubbed)'],
   ['test/paid-shards.test.ts', 'quotes tier-guard strings as classification fixtures'],
-  ['test/evals-workflow-matrix.test.ts', 'parses tier guards out of matrix files'],
+  ['test/evals-workflow-wiring.test.ts', 'pins the sliced-lane yml wiring (successor to the matrix test)'],
   ['test/e2e-tier-alignment.test.ts', 'parses tier guards to enforce alignment'],
   ['test/paid-orphan-tripwire.test.ts', 'this scanner'],
 ]);