test: free runner — strict output, parallel execution, stable shard indices

Three coupled changes to scripts/test-free-shards.ts:

1. STRICT OUTPUT: runFreeShard streams through the paid runner's
   BunTestOutputClassifier — exit 0 without bun's 'Ran N tests across M
   files' summary, with (fail) lines, or with a wrong file count is a
   FAILURE (anti-truncation backstop at the runner layer), plus an
   external wall-clock timeout that SIGKILLs the process group
   (timed-out distinct from failed; exit 124 vs 1). Also fixes a latent
   shard-bleed: file selectors now use exactTestFileSelectors (relative
   paths were substring filters that matched sibling roots).

2. PARALLEL: full-suite mode is one 'bun test --parallel' invocation
   (Bun 1.3.13). Measured semantics recorded in the header: per-file
   worker isolation, standard summary, and mid-suite process.exit
   surfaces as a crashed-worker FAIL with exit 1 — strictly safer than
   serial, where the same exit truncates silently. No static weight
   lists; --shards M --shard i keeps deterministic hash partitioning for
   CI matrices (native --shard rejected: round-robin renumbers when
   files land). Spawned shards get throwaway GSTACK_HOME/TMPDIR so
   parallel shards can't contend on real state. Per-shard epilogue
   prints files/seconds/status every run.

3. Stable indices: assignFilesToShards no longer drops empty shards, so
   a shard's index depends only on the file hash and requested count —
   an empty CI matrix slot is a fast no-op success, not a renumbering.

package.json 'test' now delegates to the runner (TEST_ROOTS becomes the
single source of truth for roots; slop:diff tail preserved; the runner
inherits the 30s per-test timeout the old glob passed inline).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-15 08:22:44 -07:00
co-authored by Claude Fable 5
parent 587217e2ec
commit 41160a14ed
4 changed files with 501 additions and 58 deletions
+29
View File
@@ -103,6 +103,35 @@ export function installChildSignalForwarding(
};
}
/**
* SIGKILL the shard's whole process group. Orphaned grandchildren (browsers,
* claude sessions) are how a stalled run once burned a core for 15.7 hours.
*/
export function killProcessGroup(child: ChildProcess, signal: NodeJS.Signals): void {
if (process.platform === 'win32' || typeof child.pid !== 'number') {
child.kill(signal);
return;
}
try {
process.kill(-child.pid, signal);
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
if (code === 'ESRCH') return; // group already gone
if (code !== 'EPERM') throw err;
// Observed on macOS after a SIGKILLed group is reaped: signalling the
// now-empty group id returns EPERM, not ESRCH. Throwing here loses the
// shard's real outcome (a timeout gets recorded as a failure) and, from
// the timeout timer, leaves the shard promise unsettled — a hang, which
// is the exact failure class this runner exists to kill. Fall back to the
// direct pid so a genuinely-live child is still signalled.
try {
child.kill(signal);
} catch {
// Best-effort reap: nothing actionable is left if this fails too.
}
}
}
export function classifyBunTestOutputLine(rawLine: string): BunTestOutputFinding | null {
const line = rawLine.replace(ANSI_ESCAPE, '').replace(/\r$/, '');
if (BUN_FAIL_RESULT.test(line)) return 'failed-test';