docs(inventory): add the development-suite limit to the /review caveats (GST-1370)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-11-04 20:44:55 +00:00
1 parent e23dee4aa5
commit 56c1fb045e
2 files changed
+3 -2

No files matched your search

@@ -1 +1 @@
- **The evidence ledger records /review as helps, with its two caveats beside the status.** R2 at the after pin (gstack-evals #207, superseded by #211) measured /review with the contract Stop hook on Opus 5.5 at gstack 699a41cb: blocking recall 26.2% against 3.6% for plain Opus 5.5 with no gstack, +22.6 points (95% CI +11.0 to +33.7). The `review` row in `docs/inventory/evidence.json` now says helps, with its sources, scope and R48 KEEP. A new `caveats` field, allowed only on helps, no-effect and hurts rows, holds the result's limits, and `docs/inventory.md` and `gstack-skill-bill` print them beside the status: /review and the hook count as one unit (42 of 66 sessions ran the readers only after the hook blocked), and 41 of the suite's 42 bugs came from gstack-family reviews. The earlier review results stay under `prior`.
- **The evidence ledger records /review as helps, with its three caveats beside the status.** R2 at the after pin (gstack-evals #207, superseded by #211) measured /review with the contract Stop hook on Opus 5.5 at gstack 699a41cb: blocking recall 26.2% against 3.6% for plain Opus 5.5 with no gstack, +22.6 points (95% CI +11.0 to +33.7). The `review` row in `docs/inventory/evidence.json` now says helps, with its sources, scope and R48 KEEP. A new `caveats` field, allowed only on helps, no-effect and hurts rows, holds the result's limits, and `docs/inventory.md` and `gstack-skill-bill` print them beside the status: /review and the hook count as one unit (42 of 66 sessions ran the readers only after the hook blocked), 41 of the suite's 42 bugs came from gstack-family reviews, and stacks-r1 is a development suite, so the result makes no release claim. The earlier review results stay under `prior`.
+2 -1
View File
@@ -438,7 +438,8 @@
},
"caveats": [
"one unit: /review and the contract Stop hook were measured together, and neither gets credit alone; 42 of 66 gstack sessions ran the review readers only after the hook blocked them (24 unprompted), and that split is not randomized",
"home-field bias in gstack's favor: 41 of the 42 blocking bugs in stacks-r1 came from gstack-family reviews, so +22.6 may run high; a suite seeded from other reviewers' bugs would remove the bias (GEVA-2 lists it as future work)"
"home-field bias in gstack's favor: 41 of the 42 blocking bugs in stacks-r1 came from gstack-family reviews, so +22.6 may run high; a suite seeded from other reviewers' bugs would remove the bias (GEVA-2 lists it as future work)",
"development suite: stacks-r1 is a development suite, so the result makes no release claim and holds only for these 33 units at these settings"
],
"prior": [
{