CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-20 03:42:24 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat(test): e2e-gate — one tier-gate implementation, side-effect-free, with the trap pinned

Browse Source 
The EVALS/EVALS_TIER gate was copy-pasted into ~40 test files and had drifted
into six different predicates — the drift that made 'eval:bg:all runs
everything' silently false. test/helpers/e2e-gate.ts owns the semantics now:
describeE2ETier(tier) + e2eTierEnabled(tier), env read at call time, zero
side effects (the existing e2e-helpers module runs a ~30s claude ping at
import under EVALS=1, so the gate lives in its own module; purity is pinned
by tests that scan imports and comment-stripped source).

The unit matrix pins all four env combos — including EVALS=1 with EVALS_TIER
unset -> SKIP, the exact trap that made eval:bg:all a non-run. The
tier-alignment tripwire gains a second regex for the helper shape (old shape
still detected — stragglers can't hide), and the sharded paid runner's
PRE-SPAWN tier classifier learns the helper shape too: without that, every
gate-sharded run would have spawned all 28 periodic shards just to skip them,
each paying the e2e-helpers import ping (~15 min of dead wall clock in the
CI-blocking lane). Verified: gate runs exclude the 29 periodic files,
periodic excludes the 8 gate files — identical to pre-migration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-14 21:12:23 -07:00
co-authored by Claude Fable 5
parent bcfdd7da4a
commit e23a3ae3b1
5 changed files with 184 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/e2e-tier-alignment.test.ts
+7
View File
@@ -26,6 +26,12 @@ const TEST_DIR = import.meta.dir;
// silently drop a file from the invariant (fail-open is the defect class
// this test exists to kill).
const SELF_GATE_RE = /EVALS_TIER\s*===\s*['"](gate|periodic)['"]/g;
// Consolidated gate helper (test/helpers/e2e-gate.ts). Both regexes stay
// active: migrated files self-gate via `describeE2ETier('<tier>')` (or the
// boolean form `e2eTierEnabled('<tier>')`), while stragglers still using the
// raw predicate are caught by SELF_GATE_RE above. The tier argument maps to
// the declared tier exactly like the raw predicate's tier literal did.
const HELPER_GATE_RE = /\b(?:describeE2ETier|e2eTierEnabled)\(\s*['"](gate|periodic)['"]/g;
describe('E2E tier alignment (touchfiles declaration vs test self-gate)', () => {
const testFiles = readdirSync(TEST_DIR)
@@ -42,6 +48,7 @@ describe('E2E tier alignment (touchfiles declaration vs test self-gate)', () =>
const content = readFileSync(path.join(TEST_DIR, file), 'utf-8');
const tiers = new Set<string>();
for (const m of content.matchAll(SELF_GATE_RE)) tiers.add(m[1]);
for (const m of content.matchAll(HELPER_GATE_RE)) tiers.add(m[1]);
const repoPath = `test/${file}`;
if (tiers.size === 0) {
// Every skill-e2e file is expected to self-gate; zero matches means