From a615303213ba0ebed0cf42551e965a2e0097fd9b Mon Sep 17 00:00:00 2001 From: garrytan Date: Wed, 30 Sep 2026 20:34:37 +0000 Subject: [PATCH] test(qa-functional): point the fixture at the helper's --help instead of its source A CI webhook-fix run spent three turns reading lib/qa-evidence.ts to learn the interface and timed out just before materialize (agreed with #3002's owner). --- test/helpers/qa-functional-eval.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/helpers/qa-functional-eval.ts b/test/helpers/qa-functional-eval.ts index 84e75c512..319d3b898 100644 --- a/test/helpers/qa-functional-eval.ts +++ b/test/helpers/qa-functional-eval.ts @@ -54,7 +54,7 @@ Fixture execution boundary: - Reports belong only in existing qa-reports. Retain fixture state; the owner cleans it after preserving evidence. Authorized source/test edits use Write/Edit. Read/Glob/Grep support arbitrary read-only discovery, including directory/path inventory. - Bash accepts separate literal commands only: no shell composition, scripts or added path operands. Read-only forms are pwd, ls, ls -la, git status --short, git status --porcelain, git branch --show-current, git diff, git diff --stat, git rev-parse HEAD, bun --version, and exactly date -u +%Y-%m-%dT%H:%M:%SZ. Native tests use bun test with optional named test/*.test.ts selectors. - These are complete command forms, not general shell examples. For inventory inside a named directory, use Read/Glob/Grep; the listed ls forms inspect only the working directory. Do not add operands or flags beyond the declared forms, even for read-only discovery. -- The installed production helper is bin/gstack-qa-evidence (absolute owned path also accepted). Probe outputs are declared public/synthetic, so capture with: bun bin/gstack-qa-evidence capture qa-reports NNN --public --timeout-ms 10000 -- NATIVE_PROBE. The child must be one of the observation forms below. Each execution/replay gets a fresh three-digit ID. The helper does not authorize another command, interpreter, path, pipeline or redirect. +- The installed production helper is bin/gstack-qa-evidence (absolute owned path also accepted); \`bun bin/gstack-qa-evidence --help\` prints its usage, and the helper source is not part of the task. Probe outputs are declared public/synthetic, so capture with: bun bin/gstack-qa-evidence capture qa-reports NNN --public --timeout-ms 10000 -- NATIVE_PROBE. The child must be one of the observation forms below. Each execution/replay gets a fresh three-digit ID. The helper does not authorize another command, interpreter, path, pipeline or redirect. - Publish causal intent from the most recent completed native probe with bun bin/gstack-qa-evidence checkpoint qa-reports NNN CAPTURE_ID 'full prior capture command' 'causal hypothesis' 'full next capture command'; quote each argument literally. For complex quoting, Write only capture, observationCommand, hypothesis and nextCommand to qa-reports/intent.json, then use bun bin/gstack-qa-evidence checkpoint qa-reports NNN intent.json. Wait for successful publication before dispatching the exact next command. Only the helper writes observed fields. - Write annotations.json inside qa-reports, then run bun bin/gstack-qa-evidence materialize qa-reports annotations.json to produce evidence.json before writing Markdown. Annotations have revision, runtime, cwd, evidence rows {capture,command,contract,expected,classification}, learning (selected checkpoint IDs) and limits; omit observed, which the helper supplies from captures. In each evidence row, capture is the three-digit capture ID and command is the exact full outer capture invocation, including that ID and all wrapper options, not just the native child command after --. This same full-command definition applies to observationCommand and nextCommand. Select a checkpoint whose next native command differs, not a same-command replay with a new capture ID. Retain all required safe observations and every executed probe. - ${entry.family === 'cli' ? 'CLI observation forms: bun run probe -- balance; bun run probe -- export; bun run probe -- apply with zero to three literal arguments; bun cancel.ts. The equivalent bun run cli commands may be diagnostic but do not emit probe JSON. Arguments use ASCII letters/digits/._+- or quoted forms including spaces. The generic wrapper does NOT support wait: the only bounded wait/cancellation interface is bun cancel.ts.' : `Webhook observation form: bun run probe -- followed by one of happy, reject, duplicate, partial, concurrent-ab, concurrent-ba, cancel, dependency. ${entry.mode === 'qa-only' ? 'All eight scenarios are required coverage; a replay does not replace another scenario. ' : ''}Choose their order from observations after the happy path. bun cancel.ts is a CLI-only entrypoint, not part of this fixture.`}