scripts/judge-calibration.ts draws two old 3-sample panels and one
schema-transport panel per corpus item, each sample a direct callJudge call
(never judgePanel or the workflow passing-panel cache, so old-vs-old flips
are real). Every sample records a value or an error kind (refusal,
truncation, parse, schema, transport), latency and priced usage via a new
callJudge onResponse observer. The manifest prices every call with the free
count_tokens API in priority order; dispatch stops at the budget or after two
consecutive transport failures. Analysis applies the eng landing rule (flips
<= old + 1, agreement does not drop, zero new false passes, p95 <= 80% of
JUDGE_MS), reports the held-out third separately, and flags the UC3 prose
trigger. Free stubbed tests cover error accounting, cache bypass, latency,
landing, stops and pricing.
garrytan
6185205bab