garrytan and capy-ai[bot]
30bcb700c1
feat(ship-measure): the measurement bar and the weekly off-ship sweep CLI
...
Decide each diagnostic measurement with one bar (scripts/lib/measure-bar.ts):
MEETS at 9/10; MEETS-qualified at 8/10 when every red is a mechanical
provider error, judge noise or a cited model miss; EXTEND at 7/10 (or an
unqualified 8/10) for one more batch on identical inputs, decided once on
the pooled 20; BELOW otherwise and for any timeout, hang, regression or
fixable cause. Behavior uses 12 trials (11/10/9; 22/20 of 24); judge counts
median-panel outputs. A batch with more than 30% provider-evidence failures
is void and redispatched once. Below-strict reds nobody classified return
needs-classify.
ship-measure gains classify (append-only records; provider refused without
evidence), decide and extend (refuses a changed identity, names the field,
never a third batch), exit codes 5 and 6, and a pooling identity (frozen
tree, runner, CLI, model, CI image, Bun, policy and harness versions).
ship-measure sweep ranks main's gate cases by their cost to the all-green
probability from eval:pass-rates history, measures the top K with the same
runner and bar under a weekly cap shared with the last 7 days of sweep
reports (ship_measure_sweep_cases, ship_measure_sweep_budget_usd), and
writes a report with the predicted all-green probability before and after.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-07 01:18:09 +00:00
..
2026-09-29 08:58:49 -07:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-08-15 22:20:30 -07:00
2026-08-29 10:10:35 -07:00
2026-09-06 08:54:25 -07:00
2026-10-03 07:54:09 -07:00
2026-09-25 13:27:07 -04:00
2026-10-04 07:58:41 -07:00
2026-10-06 18:07:50 +00:00
2026-09-22 14:57:52 -04:00
2026-09-29 08:58:49 -07:00
2026-09-29 08:58:49 -07:00
2026-09-22 14:57:52 -04:00
2026-10-04 07:58:41 -07:00
2026-09-22 14:57:52 -04:00
2026-09-22 14:57:52 -04:00
2026-10-06 23:42:19 +00:00
2026-09-05 14:45:28 -07:00
2026-09-09 08:57:21 -07:00
2026-09-29 06:07:35 -07:00
2026-10-03 18:49:34 -07:00
2026-10-05 01:29:54 +00:00
2026-08-27 09:50:31 -07:00
2026-08-31 08:55:30 -07:00
2026-10-06 23:42:19 +00:00
2026-09-22 14:57:52 -04:00
2026-10-05 01:29:54 +00:00
2026-10-04 07:58:41 -07:00
2026-10-01 13:55:16 -07:00
2026-10-04 07:58:41 -07:00
2026-10-01 13:55:16 -07:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-10-05 01:29:54 +00:00
2026-10-01 11:57:48 -07:00
2026-10-01 13:55:16 -07:00
2026-10-01 11:57:48 -07:00
2026-10-01 11:57:48 -07:00
2026-10-01 11:57:48 -07:00
2026-10-04 07:58:41 -07:00
2026-09-22 14:57:52 -04:00
2026-10-03 07:54:09 -07:00
2026-10-04 07:58:41 -07:00
2026-10-03 18:49:34 -07:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-10-04 07:58:41 -07:00
2026-09-26 18:57:53 -04:00
2026-09-25 12:06:45 -04:00
2026-10-03 07:54:09 -07:00
2026-10-03 07:54:09 -07:00
2026-09-29 08:58:49 -07:00
2026-09-25 12:06:45 -04:00
2026-10-06 10:32:19 -07:00
2026-10-06 23:42:19 +00:00
2026-10-03 01:19:03 -07:00
2026-09-29 06:07:35 -07:00
2026-10-01 11:57:48 -07:00
2026-10-04 07:58:41 -07:00
2026-10-04 07:58:41 -07:00
2026-10-06 18:07:50 +00:00
2026-09-14 14:32:45 -07:00
2026-10-05 01:29:54 +00:00
2026-10-05 01:29:54 +00:00
2026-10-03 07:54:09 -07:00
2026-08-24 08:55:58 -07:00
2026-09-14 14:32:45 -07:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-09-14 14:32:45 -07:00
2026-10-04 07:58:41 -07:00
2026-10-01 13:55:16 -07:00
2026-09-22 14:57:52 -04:00
2026-10-06 18:07:50 +00:00
2026-10-05 01:29:54 +00:00
2026-08-28 11:43:13 -07:00
2026-10-06 23:42:19 +00:00
2026-10-02 11:11:00 -07:00
2026-10-04 07:58:41 -07:00
2026-10-04 07:58:41 -07:00
2026-09-22 14:57:52 -04:00
2026-10-05 01:29:54 +00:00
2026-10-03 18:49:34 -07:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-10-01 11:57:48 -07:00
2026-10-06 18:07:50 +00:00
2026-10-05 01:29:54 +00:00
2026-10-05 01:29:54 +00:00
2026-10-06 18:07:50 +00:00
2026-10-04 07:58:41 -07:00
2026-09-29 06:07:35 -07:00
2026-09-14 14:32:45 -07:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-10-06 18:07:50 +00:00
2026-09-22 14:57:52 -04:00
2026-09-22 14:57:52 -04:00
2026-09-22 14:57:52 -04:00
2026-09-22 14:57:52 -04:00
2026-10-05 01:29:54 +00:00
2026-09-14 15:14:58 -07:00
2026-10-06 18:07:50 +00:00
2026-10-06 23:42:19 +00:00
2026-09-24 01:53:58 -04:00
2026-10-06 18:07:50 +00:00
2026-10-03 07:54:09 -07:00
2026-09-14 14:32:45 -07:00
2026-09-24 01:53:58 -04:00
2026-09-22 14:57:52 -04:00
2026-10-04 07:58:41 -07:00
2026-10-04 07:58:41 -07:00
2026-10-03 18:49:34 -07:00
2026-09-14 14:32:45 -07:00
2026-10-03 01:19:03 -07:00
2026-10-03 01:19:03 -07:00
2026-09-29 08:58:49 -07:00
2026-09-29 08:58:49 -07:00
2026-10-01 13:55:16 -07:00
2026-09-14 14:32:45 -07:00
2026-10-01 13:55:16 -07:00
2026-09-22 14:57:52 -04:00
2026-10-06 23:42:19 +00:00
2026-09-22 14:57:52 -04:00
2026-09-22 14:57:52 -04:00
2026-10-03 07:54:09 -07:00
2026-10-05 01:29:54 +00:00
2026-10-06 18:07:50 +00:00
2026-10-01 11:57:48 -07:00
2026-09-14 14:32:45 -07:00
2026-10-04 21:47:21 -07:00
2026-10-04 07:58:41 -07:00
2026-10-03 18:49:34 -07:00
2026-10-01 13:55:16 -07:00
2026-10-05 01:29:54 +00:00
2026-10-04 07:58:41 -07:00
2026-09-29 06:07:35 -07:00
2026-10-03 01:19:03 -07:00
2026-10-01 13:55:16 -07:00
2026-10-03 18:49:34 -07:00
2026-09-29 08:58:49 -07:00
2026-09-29 08:58:49 -07:00
2026-08-29 09:06:54 -07:00
2026-08-29 09:06:54 -07:00
2026-10-03 18:49:34 -07:00
2026-09-29 08:58:49 -07:00
2026-09-29 06:07:35 -07:00
2026-10-06 18:07:50 +00:00
2026-09-23 08:54:53 -04:00
2026-10-06 18:07:50 +00:00
2026-09-22 14:57:52 -04:00
2026-09-22 14:57:52 -04:00
2026-10-01 13:55:16 -07:00
2026-10-06 18:07:50 +00:00
2026-09-29 06:07:35 -07:00
2026-10-04 07:58:41 -07:00
2026-09-24 01:53:58 -04:00
2026-09-29 06:07:35 -07:00
2026-10-03 07:54:09 -07:00
2026-10-06 23:42:19 +00:00
2026-09-29 06:07:35 -07:00
2026-10-03 01:19:03 -07:00
2026-10-03 01:19:03 -07:00
2026-10-03 01:19:03 -07:00
2026-09-29 06:07:35 -07:00
2026-09-22 14:57:52 -04:00
2026-09-22 14:57:52 -04:00
2026-10-05 01:29:54 +00:00
2026-09-22 14:57:52 -04:00
2026-09-22 14:57:52 -04:00
2026-10-01 13:55:16 -07:00
2026-10-01 13:55:16 -07:00
2026-10-01 13:55:16 -07:00
2026-10-03 01:19:03 -07:00
2026-09-25 13:27:07 -04:00
2026-09-06 08:54:25 -07:00
2026-09-29 08:58:49 -07:00
2026-10-07 01:18:09 +00:00
2026-08-15 22:20:30 -07:00
2026-10-03 07:54:09 -07:00
2026-10-03 01:19:03 -07:00
2026-10-03 01:19:03 -07:00
2026-10-06 18:07:50 +00:00
2026-10-03 01:19:03 -07:00