From 933fdb9eef0467aca84f1eb850c1357a85a26f1a Mon Sep 17 00:00:00 2001 From: garrytan Date: Sun, 4 Oct 2026 02:50:10 +0000 Subject: [PATCH] =?UTF-8?q?qa:=20materialize=20runs=20once=20per=20PROBE?= =?UTF-8?q?=5FDIR,=20and=20qa-only=20says=20=C2=A71=20reuses=20the=20REPOR?= =?UTF-8?q?T=5FDIR=20it=20resolved?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Run 37170610789 slice 3 (feaa28d): the qa-only workflow judge scored clarity 3.33 and named two ambiguities: §4 said materialize runs once per report root while mixed runs materialize each surface's PROBE_DIR, and SKILL.md deferred exploratory §1 without saying §1 reuses the REPORT_DIR resolved under Prepare Report Artifacts. A local run of the same judge on the repaired text: clarity 3.67, completeness 4.33, actionability 4 (pass); all three samples call the workflow internally consistent. --- qa-only/SKILL.md.tmpl | 2 +- scripts/resolvers/qa.ts | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/qa-only/SKILL.md.tmpl b/qa-only/SKILL.md.tmpl index f4ed2ce31..aacc16cc9 100644 --- a/qa-only/SKILL.md.tmpl +++ b/qa-only/SKILL.md.tmpl @@ -73,7 +73,7 @@ If neither exists, use git diff analysis. Load the shared preparation gate now (the exploratory STOP just below): complete its scope and selected-method Reads, await their results, and select the surfaces. Defer charters, clocks and probes to -Run the Selected Checks, after report ownership and conditional browser setup below. +Run the Selected Checks; its §1 then reuses the REPORT_DIR resolved under Prepare Report Artifacts. {{SECTION:exploratory}} diff --git a/scripts/resolvers/qa.ts b/scripts/resolvers/qa.ts index 7fe0fd0a3..28a6f7685 100644 --- a/scripts/resolvers/qa.ts +++ b/scripts/resolvers/qa.ts @@ -150,7 +150,7 @@ Never freeze buggy output, weaken tests or delete valid red tests.`} ## 4. Final report Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests. -Write PROBE_DIR/annotations.json {evidence: [{capture, command, contract, expected, classification}], limits} (browser-only: evidence [], checkpoints in limits); before Markdown \`bun EVIDENCE_TOOL materialize PROBE_DIR annotations.json\` (fills observed/metadata; prints reportLinks; runs once per report root); you classify. Annotate every safe capture, including failures/replays: an omitted capture is withheld and keeps the verdict inconclusive. Classify a capture taken before an input change \`superseded\`; it closes when the same command reran on current inputs. Disclose withheld/incomplete evidence. +Write PROBE_DIR/annotations.json {evidence: [{capture, command, contract, expected, classification}], limits} (browser-only: evidence [], checkpoints in limits); before Markdown \`bun EVIDENCE_TOOL materialize PROBE_DIR annotations.json\` (fills observed/metadata; prints reportLinks; runs once per PROBE_DIR); you classify. Annotate every safe capture, including failures/replays: an omitted capture is withheld and keeps the verdict inconclusive. Classify a capture taken before an input change \`superseded\`; it closes when the same command reran on current inputs. Disclose withheld/incomplete evidence. Evidence is invocation-local${reportOnly ? '.' : '; /ship reruns once per invocation.'} Missing prerequisites/expectations/observations, timeouts and refusal never pass. Pass requires all required current-input contracts to pass with no required remainder.