From d6d82fb593cd2ccfaae711ba4d735cdc349f9744 Mon Sep 17 00:00:00 2001 From: garrytan Date: Sat, 17 Oct 2026 22:36:33 +0000 Subject: [PATCH] fix(qa): state the input-rule principle instead of a worked example (W3-9) The signed-amount example in 0fe616e was misread in the focused rerun as 'a signed amount breaks only the digits-only rule', and the run again missed the seeded non-digit defect. The stop rule now states the principle: each field's documented input rule needs a probe that breaks only it and satisfies every other rule. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 2 +- qa-only/sections/exploratory.md | 2 +- qa/sections/exploratory.md | 2 +- scripts/resolvers/qa.ts | 2 +- test/qa-functional-prompt.test.ts | 6 ++++-- 5 files changed, 8 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ae1772c2..e2057130d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ - **The State Machines specialist runs when its detector fails.** The /review and /ship scope block now runs `gstack-state-matrix detect` itself and uses its output only on exit 0. A failure prints `SCOPE_STATE_MACHINE=unknown` with the detector's stderr, and unknown dispatches the specialist. Its output reaches `gstack-state-matrix check` as a file the agent writes, never as text in a shell command. - **Consent options are lettered B, C and D when Aside is absent**, matching the question's answer keys. - **Plan-reality tier 1 rows are never n/a.** A run no longer loops repairing `deferred-asks n/a` past its deadline. -- **/qa's exploratory stop rule needs one probe per documented input rule of each field.** A value that breaks two rules, such as a signed amount under a digits-only rule, covers neither. Rules left uncovered at the probe or time cap go to limits. After a fix, every earlier capture is superseded: rerun each one the verdict relies on, or report it as open. +- **/qa's exploratory stop rule needs one probe per documented input rule of each field.** Each probe breaks only its rule and satisfies every other, so a value that breaks two rules covers neither. Rules left uncovered at the probe or time cap go to limits. After a fix, every earlier capture is superseded: rerun each one the verdict relies on, or report it as open. - **`gstack-docs-candidate snapshot --help` and `compare --help` exit 0** before any side effect, and a compare reject names the path, each bad field and a fix that never re-snapshots over the pre-audit baseline. - **The docsync fault actor uses the production candidate protocol.** It writes candidates through the installed snapshot, and a failed snapshot reports a stable reason with one retry under a fresh audit id. - **The required-probe reminder counts only captures after the last source edit** when the caller declares `GSTACK_QA_REQUIRED_AFTER_EDIT`. Pre-repair captures no longer look like post-repair coverage. Without the declaration nothing changes. diff --git a/qa-only/sections/exploratory.md b/qa-only/sections/exploratory.md index bb3927e74..e9540fe2c 100644 --- a/qa-only/sections/exploratory.md +++ b/qa-only/sections/exploratory.md @@ -36,7 +36,7 @@ For /qa and /qa-only: - Functional Full, Quick and Regression have no default total timer. Set SECONDS to the shorter mode/caller limit; an unlimited mode uses the caller's bound. Without a total time limit, do not create DEADLINE_FILE; announce finite command timeouts. -Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe breaking only it (a signed amount breaks sign and digits-only, covering neither); rules uncovered at a probe/time cap go in limits. +Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe that breaks only it and satisfies every other rule; rules uncovered at a probe/time cap go in limits. Clocks/checkpoints use REPORT_DIR; mixed standalone runs use REPORT_DIR/browser and REPORT_DIR/functional, with one final report at REPORT_DIR. Caller paths win. Names used below (quote every path): - PROBE_DIR: this surface's owned probe directory per the line above. DEADLINE_FILE: `PROBE_DIR/deadline.json`, which exists only for a bounded run. diff --git a/qa/sections/exploratory.md b/qa/sections/exploratory.md index 1f1ee6d7b..0c62850b0 100644 --- a/qa/sections/exploratory.md +++ b/qa/sections/exploratory.md @@ -30,7 +30,7 @@ For /qa and /qa-only: - Functional Full, Quick and Regression have no default total timer. Set SECONDS to the shorter mode/caller limit; an unlimited mode uses the caller's bound. Without a total time limit, do not create DEADLINE_FILE; announce finite command timeouts. -Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe breaking only it (a signed amount breaks sign and digits-only, covering neither); rules uncovered at a probe/time cap go in limits. +Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe that breaks only it and satisfies every other rule; rules uncovered at a probe/time cap go in limits. Clocks/checkpoints use REPORT_DIR; mixed standalone runs use REPORT_DIR/browser and REPORT_DIR/functional, with one final report at REPORT_DIR. Caller paths win. Names used below (quote every path): - PROBE_DIR: this surface's owned probe directory per the line above. DEADLINE_FILE: `PROBE_DIR/deadline.json`, which exists only for a bounded run. diff --git a/scripts/resolvers/qa.ts b/scripts/resolvers/qa.ts index babc12329..330b8b43f 100644 --- a/scripts/resolvers/qa.ts +++ b/scripts/resolvers/qa.ts @@ -77,7 +77,7 @@ For /qa and /qa-only: - Functional Full, Quick and Regression have no default total timer. Set SECONDS to the shorter mode/caller limit; an unlimited mode uses the caller's bound. Without a total time limit, do not create DEADLINE_FILE; announce finite command timeouts. -Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe breaking only it (a signed amount breaks sign and digits-only, covering neither); rules uncovered at a probe/time cap go in limits. +Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe that breaks only it and satisfies every other rule; rules uncovered at a probe/time cap go in limits. Clocks/checkpoints use REPORT_DIR; mixed standalone runs use REPORT_DIR/browser and REPORT_DIR/functional, with one final report at REPORT_DIR. Caller paths win. Names used below (quote every path): - PROBE_DIR: this surface's owned probe directory per the line above. DEADLINE_FILE: \`PROBE_DIR/deadline.json\`, which exists only for a bounded run. diff --git a/test/qa-functional-prompt.test.ts b/test/qa-functional-prompt.test.ts index 202f62228..a6f285a92 100644 --- a/test/qa-functional-prompt.test.ts +++ b/test/qa-functional-prompt.test.ts @@ -303,8 +303,10 @@ test('fix runs rerun every pre-repair capture their verdict relies on; supersede test('exploratory sections state one stop rule tied to per-rule input probes, and fix runs close superseded captures', () => { // PR gates 38983306684, 39001107574 and 39073934326 (#3260 at 93449bc): cli-report counted `-5` as its sign probe and never - // probed the amount's digits-only rule with an otherwise valid value, so it missed the seeded non-digit defect. - const stop = "Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe breaking only it (a signed amount breaks sign and digits-only, covering neither); rules uncovered at a probe/time cap go in limits."; + // probed the amount's digits-only rule with an otherwise valid value, so it missed the seeded non-digit defect. A worked + // example naming the signed amount (0fe616e) was misread as 'a signed amount breaks only the digits-only rule' (local + // run fw3b-cli1), so the rule states the principle instead: break one rule, satisfy every other. + const stop = "Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe that breaks only it and satisfies every other rule; rules uncovered at a probe/time cap go in limits."; const fix = 'After a fix, every earlier capture is superseded, even from a client that prints no snapshot: rerun each one your verdict relies on, or report it as an open remainder.'; for (const rel of ['qa/sections/exploratory.md', 'qa-only/sections/exploratory.md']) { const section = readFileSync(join(import.meta.dir, '..', rel), 'utf8');