feat: eval:flake-rank — the flake-telemetry dial

Aggregates per-test series across every finalized eval-store run (shard
dirs included) plus the free flake ledger: runs, fails, RETRIED PASSES
(the flake signature), avg duration — ranked retries-first. This is the
readable dial behind two policies: a flaky pass never blocks a merge but
is always ranked here, and the WS16 required-check promotion needs weeks
of clean flake-rank, not vibes. --json for machines, --dir for downloaded
CI artifacts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-31 04:50:31 +00:00
co-authored by Claude Fable 5
parent 7f36eacbd2
commit 0c0077100a
2 changed files with 128 additions and 0 deletions
+1
View File
@@ -44,6 +44,7 @@
"eval:list": "bun run scripts/eval-list.ts",
"eval:compare": "bun run scripts/eval-compare.ts",
"eval:summary": "bun run scripts/eval-summary.ts",
"eval:flake-rank": "bun run scripts/eval-flake-rank.ts",
"eval:watch": "bun run scripts/eval-watch.ts",
"eval:select": "bun run scripts/eval-select.ts",
"analytics": "bun run scripts/analytics.ts",