C2: /qa defines its outcome filename fields; QA report templates name the measured timing fields

Low judge samples in failing panels named both: qa workflow (37186854666)
'an undefined {user} in the outcome filename', and qa-only workflow
(37198445662) 'template asks for Duration / durationMs totals while
reporting.md mandates Probe budget and Guarded command time'. The qa
union stays under its 1.11 size ratio (93292 of 93294 bytes).
This commit is contained in:
garrytan committed 2026-10-04 17:41:29 +00:00
1 parent 880a7a472b
commit e14c0c1de1
4 files changed
+37 -8

No files matched your search

+4 -6
View File
@@ -261,12 +261,12 @@ about a worse score or regressed contract; blocked/inconclusive rechecks never v
## Phase 10: Report
Write the Output Structure report locally and copy the same content to project context:
**Project-scoped:** Write test outcome artifact for cross-session context:
```bash
{{SLUG_SETUP}}
```
Write to `<PROJECT_DIR>/{user}-{branch}-test-outcome-{datetime}.md` (`PROJECT_DIR` printed above)
from `git config user.name`, `git branch --show-current` (else `unknown-user`, `detached`;
non-alphanumerics → `-`) and UTC `YYYYMMDDTHHMMSSZ`.
**Per-issue additions:**
- Fix Status: verified / best-effort / reverted / deferred
@@ -279,10 +279,8 @@ For browser coverage include the score delta. For functional coverage include
passing/failing/blocked/not-run contracts, permanent regressions and remaining risks,
never a score. Keep mixed results separate.
**PR Summary:** Include one line:
> "QA found N issues, fixed M, health score X → Y."
For functional targets, use those contract outcomes instead of a score in the PR summary.
**PR Summary:** one line, "QA found N issues, fixed M, health score X → Y."
Functional targets: contract outcomes, not a score.
---
+1 -1
View File
@@ -7,7 +7,7 @@
| Surfaces / scope | {API, CLI, job, worker, webhook; changed and adjacent contracts} |
| Runtime / native tools | {VERSIONS AND REPOSITORY-SUPPORTED COMMANDS} |
| Fixture ownership / destinations | {ISOLATED ROOT, STORES, DOWNSTREAM TARGETS} |
| Duration / stop reason | {CAPTURE durationMs TOTALS, COMPLETE OR BOUND/BLOCKER} |
| Probe budget / guarded command time / stop reason | {CONFIGURED LIMIT / SUM OF CAPTURE durationMs / COMPLETE OR BOUND/BLOCKER} |
## Contract outcomes
+1 -1
View File
@@ -9,7 +9,7 @@
| **PR** | {PR_NUMBER} ({PR_URL}) or "—" |
| **Tier** | Quick / Standard / Exhaustive |
| **Scope** | {SCOPE or "Full app"} |
| **Duration** | {DURATION} |
| **Probe budget / guarded command time** | {CONFIGURED LIMIT / SUM OF MEASURED COMMAND SPANS} |
| **Pages visited** | {COUNT} |
| **Screenshots** | {COUNT} |
| **Framework** | {DETECTED or "Unknown"} |
+31
View File
@@ -0,0 +1,31 @@
import { describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
const ROOT = path.resolve(import.meta.dir, '..');
const read = (p: string) => fs.readFileSync(path.join(ROOT, p), 'utf8');
// C2: judge rationales from failing panels named these as execution gaps.
describe('QA report fields are defined where they are used', () => {
test('/qa defines every placeholder in its project outcome filename', () => {
// 37186854666 qa/SKILL.md workflow sample 2: "an undefined `{user}` in the outcome filename".
const body = read('qa/SKILL.md.tmpl');
const start = body.indexOf('-test-outcome-{datetime}.md');
const sentence = body.slice(start, body.indexOf('**Per-issue additions:**', start)).replace(/\s+/g, ' ');
for (const source of ['`git config user.name`', '`git branch --show-current`', '`unknown-user`', '`detached`', 'UTC `YYYYMMDDTHHMMSSZ`'])
expect(sentence).toContain(source);
});
test('report templates name the timing fields the reporting rules define', () => {
// 37198445662 qa-only/SKILL.md workflow sample 1: template asks for "Duration / durationMs totals"
// while reporting.md mandates Probe budget and Guarded command time.
const rules = read('qa-only/sections/reporting.md.tmpl');
expect(rules).toContain('**Probe budget**');
expect(rules).toContain('**Guarded command time**');
for (const template of ['qa/templates/functional-report-template.md', 'qa/templates/qa-report-template.md']) {
const text = read(template);
expect(text, template).toMatch(/Probe budget \/ guarded command time/i);
expect(text, template).not.toMatch(/\|\s*\**Duration\b/);
}
});
});