fix: adversarial-review runtime fixes across the telemetry + kill paths

- session-runner: exit-labeling keys off 'exit', not 'close' — an orphan
  holding the pipes could relabel a REAL exit (auth failure) as
  'timeout_startup' availability noise; the kill path still always
  group-kills and cancels the reader (labeling and unblocking are separate
  concerns). Work phase arms on a flag, not firstResponseMs===0 (a same-ms
  first byte left the startup timer live all run). The CI startup grace is
  now a real FLOOR (Math.max), matching its name and pinning test.
- gstack-detach: pgid captured AT SPAWN (== child pid under
  start_new_session) — resolving it after the grace raised ESRCH once the
  leader died on SIGTERM, orphaning TERM-immune grandchildren forever.
- test-free-shards: ledger entries carry branch + git_sha (rev-parse split:
  '--abbrev-ref HEAD HEAD' printed the branch twice and recorded it as the
  sha); local ledger default is per-PROJECT, not the machine-global tmpdir.
- eval-flake-rank: per-LINE ledger parse (one torn JSONL line vanished the
  whole series), 60-day recency bound (transcript-bearing files are MBs),
  shared isFinalizedEvalResultFile predicate (the artifact-taxonomy rule
  lived in three places); eval-store exports the predicate and finalize
  stops computing flakyRetries twice; paid-shards cleanup uses async rm
  (a SIGKILLed shard's git-workspace teardown blocked every sibling's
  stream classification on the parent event loop).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-31 05:38:26 +00:00
co-authored by Claude Fable 5
parent 5b04f05ba4
commit 513111a166
6 changed files with 116 additions and 21 deletions
+17 -1
View File
@@ -192,6 +192,21 @@ export function isPartialEval(data: unknown, filename: string): boolean {
return Boolean((data as { _partial?: unknown } | null)?._partial);
}
/**
* Is this path a FINALIZED eval-store result file? Single owner of the
* filename taxonomy (manifest.json / slice-N.json are runner artifacts,
* _partial* are in-progress accumulators) — the paid runner's report mode
* and eval-flake-rank both consume this instead of re-encoding the rule
* (review finding: the rule lived in three places).
*/
export function isFinalizedEvalResultFile(relPath: string): boolean {
const base = path.basename(relPath);
if (!base.endsWith('.json')) return false;
if (base === 'manifest.json' || /^slice-\d+\.json$/.test(base)) return false;
if (base.startsWith('_partial')) return false;
return true;
}
/**
* List eval JSON files in `evalDir` plus one level of `<evalDir>/shards/<slug>/`
* subdirectories (where the sharded paid runner points each shard's collector).
@@ -900,6 +915,7 @@ export class EvalCollector {
const totalDuration = this.tests.reduce((s, t) => s + t.duration_ms, 0);
const passed = this.tests.filter(t => t.passed).length;
const flaky = this.flakyRetries();
const result: EvalResult = {
schema_version: SCHEMA_VERSION,
version,
@@ -917,7 +933,7 @@ export class EvalCollector {
wall_clock_ms: Date.now() - this.createdAt,
tests: this.tests,
...(this.shard ? { shard: this.shard } : {}),
...(this.flakyRetries().length > 0 ? { flaky_retries: this.flakyRetries() } : {}),
...(flaky.length > 0 ? { flaky_retries: flaky } : {}),
};
// Write eval file