Files
gstack/scripts
Garry TanandClaude Fable 5 ac9291b146 evals: retry 1, not 2, on every paid path
Measured on the llm-judge shard: --retry 2 amplified 25 tests into 46
executions (+84%), with retried runs at 138s vs a 10-12s baseline (429
backoff), and a permanently-failing test paying 3x. One retry still
absorbs one-off flakes; chronic flakes become visible fix-work instead
of silent wall-clock.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 07:53:39 -07:00
..