fix(qa): define evidence.json where it is built, point the preparation gate at the next section, name measured command durations in the report template

Recurring qa/qa-only workflow-judge complaints in CI (clarity/actionability 3.33).
This commit is contained in:
garrytan committed 2026-09-30 13:59:06 +00:00
1 parent 53c5b7505d
commit f4fa38ac68
6 files changed
+9 -9

No files matched your search

+1 -1
View File
@@ -464,7 +464,7 @@ the current behavior. Reading old notes never requires writing new ones.
## Select Surfaces and Isolation ## Select Surfaces and Isolation
Load the shared preparation gate now: complete its scope and selected-method Reads, Load the shared preparation gate now (next section): complete its scope and selected-method Reads,
await their results, and select the surfaces. Defer charters, clocks and probes to await their results, and select the surfaces. Defer charters, clocks and probes to
Run the Selected Checks, after report ownership and conditional browser setup below. Run the Selected Checks, after report ownership and conditional browser setup below.
+1 -1
View File
@@ -71,7 +71,7 @@ If neither exists, use git diff analysis.
## Select Surfaces and Isolation ## Select Surfaces and Isolation
Load the shared preparation gate now: complete its scope and selected-method Reads, Load the shared preparation gate now (next section): complete its scope and selected-method Reads,
await their results, and select the surfaces. Defer charters, clocks and probes to await their results, and select the surfaces. Defer charters, clocks and probes to
Run the Selected Checks, after report ownership and conditional browser setup below. Run the Selected Checks, after report ownership and conditional browser setup below.
+2 -2
View File
@@ -96,8 +96,8 @@ with their failing contract and expected assertion; never create tests or freeze
## 4. Final report ## 4. Final report
Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests. Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests.
For evidence.json, Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}.
Run `bun Q materialize R annotations.json` before Markdown; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. Before Markdown, `bun Q materialize R annotations.json` builds evidence.json from it; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence.
Evidence is invocation-local. Evidence is invocation-local.
Missing prerequisites/expectations/observations, timeouts and refusal never pass. Missing prerequisites/expectations/observations, timeouts and refusal never pass.
Pass requires all required current-input contracts to pass with no required remainder. Pass requires all required current-input contracts to pass with no required remainder.
+2 -2
View File
@@ -81,8 +81,8 @@ Never freeze buggy output, weaken tests or delete valid red tests.
## 4. Final report ## 4. Final report
Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests. Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests.
For evidence.json, Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}.
Run `bun Q materialize R annotations.json` before Markdown; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. Before Markdown, `bun Q materialize R annotations.json` builds evidence.json from it; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence.
Evidence is invocation-local; /ship reruns once per invocation. Evidence is invocation-local; /ship reruns once per invocation.
Missing prerequisites/expectations/observations, timeouts and refusal never pass. Missing prerequisites/expectations/observations, timeouts and refusal never pass.
Pass requires all required current-input contracts to pass with no required remainder. Pass requires all required current-input contracts to pass with no required remainder.
+1 -1
View File
@@ -7,7 +7,7 @@
| Surfaces / scope | {API, CLI, job, worker, webhook; changed and adjacent contracts} | | Surfaces / scope | {API, CLI, job, worker, webhook; changed and adjacent contracts} |
| Runtime / native tools | {VERSIONS AND REPOSITORY-SUPPORTED COMMANDS} | | Runtime / native tools | {VERSIONS AND REPOSITORY-SUPPORTED COMMANDS} |
| Fixture ownership / destinations | {ISOLATED ROOT, STORES, DOWNSTREAM TARGETS} | | Fixture ownership / destinations | {ISOLATED ROOT, STORES, DOWNSTREAM TARGETS} |
| Duration / stop reason | {MEASURED DURATION, COMPLETE OR BOUND/BLOCKER} | | Duration / stop reason | {MEASURED COMMAND DURATIONS, COMPLETE OR BOUND/BLOCKER} |
## Contract outcomes ## Contract outcomes
+2 -2
View File
@@ -143,8 +143,8 @@ Never freeze buggy output, weaken tests or delete valid red tests.`}
## 4. Final report ## 4. Final report
Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests. Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests.
For evidence.json, Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}. Write R/annotations.json: {revision, runtime, cwd, evidence: [{capture, command, contract, expected, classification}], learning: [checkpoint IDs], limits}.
Run \`bun Q materialize R annotations.json\` before Markdown; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence. Before Markdown, \`bun Q materialize R annotations.json\` builds evidence.json from it; Q fills observed/learning, not classifications. Retain all safe probes, including failures/replays; disclose withheld/incomplete evidence.
Evidence is invocation-local${reportOnly ? '.' : '; /ship reruns once per invocation.'} Evidence is invocation-local${reportOnly ? '.' : '; /ship reruns once per invocation.'}
Missing prerequisites/expectations/observations, timeouts and refusal never pass. Missing prerequisites/expectations/observations, timeouts and refusal never pass.
Pass requires all required current-input contracts to pass with no required remainder. Pass requires all required current-input contracts to pass with no required remainder.