fix(review): pass Review Army checklists by path, run research alongside dispatch, always probe the design detector; state review-log invocation and statuses in the caller fixture

- review-army-perf-n-plus-one: the parent copied full checklists into agent
  prompts and ran web research before dispatch (290 s on a 12-line diff); 212 s now.
- review-design-lite: 5 of 6 captured trials reported the detector absent
  without probing; the probe is mandatory and its first line is reported, and
  the contract credits only fake-engine rule ids the checklist never names.
- review-exploratory-small-cli: the fixture never gave review-log's direct
  invocation or status vocabulary; the model ran it through bun and wrote
  status "blocked". The prompt states both and the validator rejects
  out-of-vocabulary review statuses.
Each case passed a focused paid run after repair.
This commit is contained in:
garrytan committed 2026-09-29 22:49:12 +00:00
1 parent a18e6cf655
commit a06d22e52a
15 files changed
+155 -62

No files matched your search

+6 -2
View File
@@ -10,8 +10,7 @@
budgets. The CI image stays on 2.1.251 until those cases get faster. Effort M.
- **Recurring reds to repair, not rerun** — `plan-design-review-plan-mode`
(one ~250 s thinking block before its single write; times out at 300 s on
2.1.251 in every recent run), `review-army-perf-n-plus-one` (290-300 s on a
12-line diff; web search plus a conditional red-team pass), the HOLD SCOPE
2.1.251 in every recent run) and the HOLD SCOPE
routing case when its next brief happens not to name the mode (see the
handoff item below). Effort M each.
- **`/plan-ceo-review` skips its Step 0E mode handoff** — in 4 of 4 asked-mode
@@ -20,6 +19,11 @@
decisions: …` chat. The routing case still passes on other posture text; a
wording change moving the handoff ahead of the question log did not change
the behavior in two paid runs, so it was not shipped. Effort M.
- **`/ship` design-lite probe wording** — `scripts/resolvers/design.ts` still
says "Probe for a design detector the user installed", the wording that let
`/review` skip its probe in 5 of 6 captured trials before this release made it
mandatory. The ship union ratio is at 1.3966 of 1.397, so the same sentence
needs a trim elsewhere first. Effort S.
- **Let pass-rate history decide the rest** — every census on this branch had
a different handful of single-trial reds. Once `eval:pass-rates` has 10 weekly
trials per case, apply the CASE_QUARANTINE entry rule instead of chasing one