mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-22 21:00:51 +02:00
feat(test): e2e-gate — one tier-gate implementation, side-effect-free, with the trap pinned
The EVALS/EVALS_TIER gate was copy-pasted into ~40 test files and had drifted into six different predicates — the drift that made 'eval:bg:all runs everything' silently false. test/helpers/e2e-gate.ts owns the semantics now: describeE2ETier(tier) + e2eTierEnabled(tier), env read at call time, zero side effects (the existing e2e-helpers module runs a ~30s claude ping at import under EVALS=1, so the gate lives in its own module; purity is pinned by tests that scan imports and comment-stripped source). The unit matrix pins all four env combos — including EVALS=1 with EVALS_TIER unset -> SKIP, the exact trap that made eval:bg:all a non-run. The tier-alignment tripwire gains a second regex for the helper shape (old shape still detected — stragglers can't hide), and the sharded paid runner's PRE-SPAWN tier classifier learns the helper shape too: without that, every gate-sharded run would have spawned all 28 periodic shards just to skip them, each paying the e2e-helpers import ping (~15 min of dead wall clock in the CI-blocking lane). Verified: gate runs exclude the 29 periodic files, periodic excludes the 8 gate files — identical to pre-migration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
bcfdd7da4a
commit
e23a3ae3b1
@@ -0,0 +1,51 @@
|
||||
/**
|
||||
* Whole-file E2E tier gate — the single definition of the
|
||||
* `EVALS=1 && EVALS_TIER === '<tier>'` predicate that tier-gated paid test
|
||||
* files used to copy-paste (~36 local copies before consolidation).
|
||||
*
|
||||
* This module MUST stay side-effect-free. It is imported at module scope by
|
||||
* every tier-gated test file, including files the sharded paid runner
|
||||
* (scripts/test-paid-shards.ts) spawns one-process-each — unlike
|
||||
* test/helpers/e2e-helpers.ts, whose EVALS=1 module-scope work includes a
|
||||
* ~30s `claude -p` connectivity ping, diff-based selection, and ~/.gstack
|
||||
* pre-seeding. The only import allowed here is `bun:test`.
|
||||
* test/helpers/e2e-gate.unit.test.ts enforces this with a source scan.
|
||||
*
|
||||
* Env is read at CALL time (the importing test file's module top-level), not
|
||||
* captured at this module's load time, so the gate behaves identically under
|
||||
* single-process `bun test` globs and the sharded runner's per-shard env.
|
||||
*
|
||||
* Static-grep consumers that must recognize the call shape
|
||||
* `describeE2ETier('<tier>')` / `e2eTierEnabled('<tier>')` alongside the raw
|
||||
* `EVALS_TIER === '<tier>'` predicate:
|
||||
* - test/e2e-tier-alignment.test.ts (HELPER_GATE_RE) — tier-alignment invariant
|
||||
* - scripts/test-paid-shards.ts classifyPaidTestFile — pre-spawn tier exclusion
|
||||
*/
|
||||
|
||||
import { describe } from 'bun:test';
|
||||
|
||||
export type E2ETier = 'gate' | 'periodic';
|
||||
|
||||
/**
|
||||
* True when this process should run whole-file-gated paid tests of `tier`:
|
||||
* EVALS=1 AND EVALS_TIER exactly equals the tier.
|
||||
*
|
||||
* Deliberate consequence: EVALS=1 with EVALS_TIER unset is false for BOTH
|
||||
* tiers. Tierless runs (`test:evals` / `eval:bg` / `eval:bg:all`) skip every
|
||||
* tier-gated file and rely on diff-based per-test selection instead — that is
|
||||
* the long-standing behavior of the copy-pasted predicates, pinned by
|
||||
* test/helpers/e2e-gate.unit.test.ts.
|
||||
*/
|
||||
export function e2eTierEnabled(tier: E2ETier): boolean {
|
||||
return !!process.env.EVALS && process.env.EVALS_TIER === tier;
|
||||
}
|
||||
|
||||
/**
|
||||
* `describe` when `e2eTierEnabled(tier)`, else `describe.skip`.
|
||||
*
|
||||
* Usage (module top-level of a tier-gated test file):
|
||||
* const describeE2E = describeE2ETier('periodic');
|
||||
*/
|
||||
export function describeE2ETier(tier: E2ETier): typeof describe | typeof describe.skip {
|
||||
return e2eTierEnabled(tier) ? describe : describe.skip;
|
||||
}
|
||||
Reference in New Issue
Block a user