mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-12 07:59:02 +02:00
feat: delete the legacy 17-row eval matrix — the sliced lane is the only paid lane
Every PR paid twice: the hand-enumerated matrix (18 test files, 22.6 min, ~$21 API measured on run 33263204465) ran serialized AHEAD of the strictly superior sliced lane via 'needs: evals' — 35.5 min wall and ~2x paid spend for the same diff. 14 of 17 rows carried no tier:, so periodic Opus benchmarks leaked into every PR (the e2e-plan row alone: 12/12 tests, 21.7 min, $7.28 — the wall-clock bound of ALL of CI). Parity receipt (static, pre-deletion): the sliced lane's gate census (49 files, derived from the runner itself) strictly contains all 18 matrix test files, plus 31 files the matrix never ran. Pure deletion — one revert restores it. The PR comment moved into slices-report (same '## E2E Evals' upsert marker, now sourced from slice artifacts + carrying the fail-closed reconciliation verdict). plan-slices loses the needs edge; the dead workflow-level EVALS_TIER env goes with it. test/evals-workflow-matrix.test.ts (and its KNOWN_MATRIX_GAPS / KNOWN_TIER_UNSET burn-down ratchets — retired: the sliced census makes 'every gate file runs' true by construction) is rewritten as test/evals-workflow-wiring.test.ts: matrix stays deleted, planner/executor/ report tier + slice-count agreement, both surviving lanes on the shared register-skills composite with its fail-fast verification loop, PR comment survival. Expected: PR eval wall 35.5 -> ~13 min, per-PR paid spend ~halved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
0f26859bd1
commit
07c2452e6d
@@ -42,8 +42,8 @@ const SELF_GATE_RE = /EVALS_TIER\s*===\s*['"](gate|periodic)['"]/g;
|
||||
const HELPER_GATE_RE = /\b(?:describeE2ETier|e2eTierEnabled)\(\s*['"](gate|periodic)['"]/g;
|
||||
|
||||
/**
|
||||
* Ratchet, not amnesty (same contract as KNOWN_MATRIX_GAPS in
|
||||
* test/evals-workflow-matrix.test.ts): skill-e2e files that are named in NO
|
||||
* Ratchet, not amnesty (the contract KNOWN_MATRIX_GAPS pioneered before the
|
||||
* legacy eval matrix and its test retired): skill-e2e files that are named in NO
|
||||
* touchfiles dep list because no E2E map key exists for them. Every entry
|
||||
* carries a one-line reason. Do NOT add new files here — give the test an
|
||||
* E2E map key (touchfiles + tier) and register the file in its dep list.
|
||||
@@ -141,8 +141,8 @@ describe('E2E tier alignment (touchfiles declaration vs test self-gate)', () =>
|
||||
expect(misaligned).toEqual([]);
|
||||
});
|
||||
|
||||
// Ratchet cleanup enforcement (same contract as evals-workflow-matrix's
|
||||
// burn-down test): a KNOWN_UNREGISTERED entry whose file was deleted, or
|
||||
// Ratchet cleanup enforcement (same shrink-only contract as the retired
|
||||
// matrix test's burn-down): a KNOWN_UNREGISTERED entry whose file was deleted, or
|
||||
// whose file is now named in a dep list, is stale — delete the entry so
|
||||
// the set can only shrink.
|
||||
test('KNOWN_UNREGISTERED holds only live, still-unregistered files', () => {
|
||||
|
||||
Reference in New Issue
Block a user