From f4fa38ac68607e1beee34523648f1ae72791a074 Mon Sep 17 00:00:00 2001 From: garrytan Date: Wed, 30 Sep 2026 13:59:06 +0000 Subject: [PATCH] fix(qa): define evidence.json where it is built, point the preparation gate at the next section, name measured command durations in the report template Recurring qa/qa-only workflow-judge complaints in CI (clarity/actionability 3.33). --- qa-only/SKILL.md | 2 +- qa-only/SKILL.md.tmpl | 2 +- qa-only/sections/exploratory.md | 4 ++-- qa/sections/exploratory.md | 4 ++-- qa/templates/functional-report-template.md | 2 +- scripts/resolvers/qa.ts | 4 ++-- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/qa-only/SKILL.md b/qa-only/SKILL.md index f3519075f..d82eb5b68 100644 --- a/qa-only/SKILL.md +++ b/qa-only/SKILL.md @@ -464,7 +464,7 @@ the current behavior. Reading old notes never requires writing new ones. ## Select Surfaces and Isolation -Load the shared preparation gate now: complete its scope and selected-method Reads, +Load the shared preparation gate now (next section): complete its scope and selected-method Reads, await their results, and select the surfaces. Defer charters, clocks and probes to Run the Selected Checks, after report ownership and conditional browser setup below. diff --git a/qa-only/SKILL.md.tmpl b/qa-only/SKILL.md.tmpl index 55169e631..8ad77b792 100644 --- a/qa-only/SKILL.md.tmpl +++ b/qa-only/SKILL.md.tmpl @@ -71,7 +71,7 @@ If neither exists, use git diff analysis. ## Select Surfaces and Isolation -Load the shared preparation gate now: complete its scope and selected-method Reads, +Load the shared preparation gate now (next section): complete its scope and selected-method Reads, await their results, and select the surfaces. Defer charters, clocks and probes to Run the Selected Checks, after report ownership and conditional browser setup below. diff --git a/qa-only/sections/exploratory.md b/qa-only/sections/exploratory.md index 55e1c32a3..3a0a56619 100644 --- a/qa-only/sections/exploratory.md +++ b/qa-only/sections/exploratory.md @@ -96,8 +96,8 @@ with their failing contract and expected assertion; never create tests or freeze ## 4. Final report Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests. -For evidence.json, Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. -Run `bun Q materialize R annotations.json` before Markdown; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. +Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. +Before Markdown, `bun Q materialize R annotations.json` builds evidence.json from it; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. Evidence is invocation-local. Missing prerequisites/expectations/observations, timeouts and refusal never pass. Pass requires all required current-input contracts to pass with no required remainder. diff --git a/qa/sections/exploratory.md b/qa/sections/exploratory.md index c769f8ebc..779eb593a 100644 --- a/qa/sections/exploratory.md +++ b/qa/sections/exploratory.md @@ -81,8 +81,8 @@ Never freeze buggy output, weaken tests or delete valid red tests. ## 4. Final report Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests. -For evidence.json, Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. -Run `bun Q materialize R annotations.json` before Markdown; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. +Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. +Before Markdown, `bun Q materialize R annotations.json` builds evidence.json from it; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. Evidence is invocation-local; /ship reruns once per invocation. Missing prerequisites/expectations/observations, timeouts and refusal never pass. Pass requires all required current-input contracts to pass with no required remainder. diff --git a/qa/templates/functional-report-template.md b/qa/templates/functional-report-template.md index 96a3189c2..fcae95955 100644 --- a/qa/templates/functional-report-template.md +++ b/qa/templates/functional-report-template.md @@ -7,7 +7,7 @@ | Surfaces / scope | {API, CLI, job, worker, webhook; changed and adjacent contracts} | | Runtime / native tools | {VERSIONS AND REPOSITORY-SUPPORTED COMMANDS} | | Fixture ownership / destinations | {ISOLATED ROOT, STORES, DOWNSTREAM TARGETS} | -| Duration / stop reason | {MEASURED DURATION, COMPLETE OR BOUND/BLOCKER} | +| Duration / stop reason | {MEASURED COMMAND DURATIONS, COMPLETE OR BOUND/BLOCKER} | ## Contract outcomes diff --git a/scripts/resolvers/qa.ts b/scripts/resolvers/qa.ts index b16accf36..6a56bc113 100644 --- a/scripts/resolvers/qa.ts +++ b/scripts/resolvers/qa.ts @@ -143,8 +143,8 @@ Never freeze buggy output, weaken tests or delete valid red tests.`} ## 4. Final report Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests. -For evidence.json, Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. -Run \`bun Q materialize R annotations.json\` before Markdown; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. +Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. +Before Markdown, \`bun Q materialize R annotations.json\` builds evidence.json from it; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. Evidence is invocation-local${reportOnly ? '.' : '; /ship reruns once per invocation.'} Missing prerequisites/expectations/observations, timeouts and refusal never pass. Pass requires all required current-input contracts to pass with no required remainder.