Files
gstack/docs
Garry Tanandcapy-ai[bot] b0f1e65e0d v1.91.108.0 fix(evals): census repair — the late-input and simplification fixtures stop inviting their own failures (#3153) (#3200)
* fix(qa-callers): the caller scheduling contract no longer calls a probe read-only discovery (#3153)

Census 38212265208 (main @ e23f7b807, 2026-10-12) failed ship-exploratory-late-input:
"probe preceded resource read: exploratory / system-functional / system-functional".
The native events show capture 001 issued in the same provider response as the
Reads of ship/sections/plan-completion.md and ship/sections/review-army.md, before
the Step 9 section (which holds the "never batch a probe with a Read" STOP) returned
and before any QA section was read. The Oct 7 capture (mtb1007a) is the same class.

Diagnosis: fixture defect, not product prose. /ship's prose never names the QA
capture helper (only qa/sections/exploratory.md does); the fixture prompt supplies
it up front, and the fixture's appended scheduling contract (7697fb2, 2026-10-06)
told the session to batch "Reads and read-only discovery" while its never-group list
omitted Read-result dependencies. The case passed on Oct 3 and Oct 5, before that
contract, and failed on Oct 7 and Oct 12 after it.

Repair: the contract defines read-only discovery (captures, guard starts/runs and
diagnostic-client runs are probes, never discovery), which removes the fixture's own
invitation to batch them. It deliberately does not restate the product's Step 9 STOP:
the case exists to detect that STOP being lost. The detector (callerProbeReadErrors)
is unchanged.

Test: the Oct 12 native tool order is replayed inline (no new fixture file, so the
paid-runner global inputs stay untouched) with a reordered negative control, and the
actual runner callback's contract is checked. Fails on 47d639a at
the discovery-definition assertion; passes after.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(review-army): the simplification case drops its false premise and stages only its section (#3153)

Census 38212265208's pass-rate gate flagged review-army-simplification at
159 s of its 180 s session budget (88.1%, above the 85% cap; main-only panel
142/91/146 s). The three Oct 12 transcripts show where the time went:

- the prompt claimed "the diff is over 100 lines", but the fixture diff is 68;
  every trial reconciled that (scope tool, specialist stats, force-include
  reasoning, coverage-gap prose in the report and final message);
- each session read the whole extracted review-SKILL.md and the 18.7 KB
  checklist, and grepped for a "## Step 4.5" heading the carved fixture no
  longer has;
- the report was unbounded (30-45 s of output composition); trial 3 also
  dispatched a subagent and re-verified its findings.

Repair (fixture work only, budget unchanged at 180 s / 20 turns): the case uses
the same stageReviewArmySession/reviewArmyScope staging as perf-n-plus-one and
consensus (recorded scope, stats, learnings and diff; only review-army.md and
simplification.md staged), selects the specialist with the real
/review --simplification force flag, applies the checklist in-session as the
original prompt asked, and bounds the report to 30 lines with a brief final
acknowledgement. The planted-structure and disavowed-frame assertions are
unchanged.

Test: review-army-budget.test.ts drives the actual registered callback with a
mocked runner (pass control plus two negative controls); it fails on 47d639a
("Expected to contain: DIFF_LINES: 68") and passes after.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(evals): record the #3153 census diagnoses (review-army-simplification headroom, ship-exploratory-late-input) (#3153)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore(release): tentative v1.91.108.0 stamp and CHANGELOG for the #3153 census repair (#3153)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-16 23:24:55 +00:00
..