test(qa-callers): bind QA section names in the caller prompt instead of teaching helper forms (W3-4)

The caller session prompt taught full capture/checkpoint syntax but only
mentioned materialize, so two ship-exploratory-small-cli sessions (paid
gate runs 38973329985, 38993076767) skipped qa/sections/exploratory.md and
system-functional.md and guessed a bare materialize. The prompt now
declares only fixture bindings (DEADLINE_TOOL, EVIDENCE_TOOL, PROBE_DIR,
DEADLINE_FILE, --public approval, --timeout-ms 10000 for plan probes) and
points to the section for every form. The read requirement is unchanged;
both captured transcripts are kept as negative controls.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

test/helpers/qa-callers-fixture.ts

@@ -697,7 +697,7 @@ export function callerReviewRecordTemplate(fixture: Pick<QaCallerFixture, 'calle
 
 export function qaCallerSessionOptions(fixture: QaCallerFixture, runId: string): Parameters<typeof runSkillTest>[0] {
   return {
-    prompt: `Load gstack's /${fixture.caller} supplied parent phase from caller-${fixture.caller}.md and resume it on the selected working-tree diff against origin/main. This excerpt comes from ${fixture.runtime}/${fixture.caller}/SKILL.md; resolve installed-relative references there, not from the excerpt file or product directory. That path identifies the asset base, not another entrypoint: do not read or invoke the full parent SKILL.md or rerun its preamble. Earlier preamble/branch/base setup is complete; use the existing local origin/main ref without fetch. Earlier-phase asset locators are ${fixture.runtime}/review/checklist.md and ${fixture.runtime}/qa/templates/functional-report-template.md. Read those files directly when referenced; recursive Glob does not follow the installed asset symlinks. Cross-project learnings are configured off in this owned fixture. ${fixture.reviewStart ? `The actual review-start helper already captured REVIEW_START=${fixture.reviewStart} for this unchanged core pass; retain that token. ` : ''} This fixture evaluates only the supplied parent phase, not later publication stages. The fixture owner recorded these read-only observations when this phase began; they are current until an input changes, so use them instead of re-running those commands, and recheck freshness before completion outputs:\n${fixture.startObservations}\n Read README.md for the project contract and commands. Use diagnostic-client commands such as \`bun scripts/probe.ts <literal>\` for exploratory discoveries and their checkpoint evidence. A required \`bun run test\` is separate suite verification: report it as verification, never as a diagnostic observation or checkpoint anchor/target. Use the production evidence helper to publish each diagnostic checkpoint as \`reports/exploration-NNN.json\`, not inside a nested directory; do not transcribe its observed payload. ${fixture.caseId === 'ship-exploratory-plan-checks' ? 'The previously discovered plan is PLAN.md.' : 'No plan file was found.'} There is no remote service and no release publication is authorized. There is no interactive approver; do not invent answers or permission. Keep normal parent decision gates. Before the completion report and each completed:true review record, read HANDOFF.md and reports/HANDOFF.md if present for any concurrent collaborator update, await the results, and compare evidence with current inputs. If a probe's snapshot differs from an earlier probe's, read reports/HANDOFF.md first; it names any collaborator change. A gstack-review-log completed:true record is a completion, not preliminary bookkeeping; a later handoff read cannot validate an earlier completion.\n\nDeadline bookkeeping additionally permits \`bun ${fixture.runtime}/bin/gstack-qa-deadline start ${fixture.cwd}/reports/deadline.json SECONDS [EARLIER_UTC]\`, \`bun ${fixture.runtime}/bin/gstack-qa-deadline status ${fixture.cwd}/reports/deadline.json\`, and \`bun ${fixture.runtime}/bin/gstack-qa-deadline run ${fixture.cwd}/reports/deadline.json -- bun scripts/probe.ts [literal]\`. These are closed literal forms: SECONDS must be positive and at most 300, EARLIER_UTC is the optional caller absolute deadline: use the section clock's Hard deadline UTC, never its Runner entry UTC, reserve-start time or a clock-read time. The child is only the existing diagnostic client with zero or one literal argument. Resolve these exact helper and state paths; do not use variables, another helper, another state file, nested wrappers, scripts, operators or substitutions. Only this helper may create or change reports/deadline.json and its .qa-deadline- temporary files; never use Write/Edit/MultiEdit on those paths. Record the full outer run command in checkpoints and evidence; keep the unchanged child JSON as observed, separate from prefixed guard diagnostics. A completed expired guard-run is not a probe or a pass: retain its unused checkpoint, report not-run coverage and do not restart the deadline. Keep the 12-probe smoke limit. Required suites and explicit plan checks are outside the bounded smoke budget, not permission to reset it.\n\nFunctional evidence uses the same production helper and existing diagnostic client: \`bun ${fixture.runtime}/bin/gstack-qa-evidence capture ${fixture.cwd}/reports NNN --public --deadline ${fixture.cwd}/reports/deadline.json -- bun scripts/probe.ts [literal]\`. These diagnostic receipts are declared public/synthetic, so --public is approved; a fresh three-digit ID is required each time. Explicit plan probes outside the smoke budget may replace --deadline with --timeout-ms 10000; this does not reset or bypass the smoke deadline. After the first capture, put causal intent in the next capture itself by adding \`--after PREV --hypothesis 'causal hypothesis'\` before \`--\` (PREV = the most recent complete capture ID); the helper publishes reports/exploration-NNN.json linking capture PREV's observation to that probe before running it, so one call is both checkpoi
Line truncated
+    prompt: `Load gstack's /${fixture.caller} supplied parent phase from caller-${fixture.caller}.md and resume it on the selected working-tree diff against origin/main. This excerpt comes from ${fixture.runtime}/${fixture.caller}/SKILL.md; resolve installed-relative references there, not from the excerpt file or product directory. That path identifies the asset base, not another entrypoint: do not read or invoke the full parent SKILL.md or rerun its preamble. Earlier preamble/branch/base setup is complete; use the existing local origin/main ref without fetch. Earlier-phase asset locators are ${fixture.runtime}/review/checklist.md and ${fixture.runtime}/qa/templates/functional-report-template.md. Read those files directly when referenced; recursive Glob does not follow the installed asset symlinks. Cross-project learnings are configured off in this owned fixture. ${fixture.reviewStart ? `The actual review-start helper already captured REVIEW_START=${fixture.reviewStart} for this unchanged core pass; retain that token. ` : ''} This fixture evaluates only the supplied parent phase, not later publication stages. The fixture owner recorded these read-only observations when this phase began; they are current until an input changes, so use them instead of re-running those commands, and recheck freshness before completion outputs:\n${fixture.startObservations}\n Read README.md for the project contract and commands. Use diagnostic-client commands such as \`bun scripts/probe.ts <literal>\` for exploratory discoveries and their checkpoint evidence. A required \`bun run test\` is separate suite verification: report it as verification, never as a diagnostic observation or checkpoint anchor/target. Use the production evidence helper to publish each diagnostic checkpoint as \`reports/exploration-NNN.json\`, not inside a nested directory; do not transcribe its observed payload. ${fixture.caseId === 'ship-exploratory-plan-checks' ? 'The previously discovered plan is PLAN.md.' : 'No plan file was found.'} There is no remote service and no release publication is authorized. There is no interactive approver; do not invent answers or permission. Keep normal parent decision gates. Before the completion report and each completed:true review record, read HANDOFF.md and reports/HANDOFF.md if present for any concurrent collaborator update, await the results, and compare evidence with current inputs. If a probe's snapshot differs from an earlier probe's, read reports/HANDOFF.md first; it names any collaborator change. A gstack-review-log completed:true record is a completion, not preliminary bookkeeping; a later handoff read cannot validate an earlier completion.\n\nQA fixture bindings: the deadline guard, capture, checkpoint and materialize command forms are in the installed qa/sections/exploratory.md this phase requires you to Read; use those forms verbatim with these bindings substituted literally. DEADLINE_TOOL = ${fixture.runtime}/bin/gstack-qa-deadline. EVIDENCE_TOOL = ${fixture.runtime}/bin/gstack-qa-evidence. PROBE_DIR = ${fixture.cwd}/reports. DEADLINE_FILE = ${fixture.cwd}/reports/deadline.json. SECONDS must be positive and at most 300. EARLIER_UTC is the optional caller absolute deadline: use the section clock's Hard deadline UTC, never its Runner entry UTC, reserve-start time or a clock-read time. The only native COMMAND is the existing diagnostic client, bun scripts/probe.ts, with zero or one literal argument. Resolve these exact helper and state paths; do not use variables, another helper, another state file, nested wrappers, scripts, operators or substitutions. Its diagnostic receipts are declared public/synthetic, so --public is approved. Explicit plan probes outside the smoke budget may use --timeout-ms 10000 in place of the deadline; this does not reset or bypass the smoke deadline. Only the deadline helper may create or change reports/deadline.json and its .qa-deadline- temporary files; never use Write/Edit/MultiEdit on those paths. Record the full outer command in checkpoints and evidence; keep the unchanged child JSON as observed, separate from prefixed guard diagnostics. A completed expired guard-run is not a probe or a pass: retain its unused checkpoint, report not-run coverage and do not restart the deadline. Keep the 12-probe smoke limit. Required suites and explicit plan checks are outside the bounded smoke budget, not permission to reset it. Quote arguments literally. Sources stay inside reports. Decide to execute the next probe before publishing its checkpoint, then await successful publication and dispatch that exact probe. If you defer an optional idea or stop exploration, do not publish a checkpoint for it; describe it as not run in Markdown. An unused checkpoint requires an actual authenticated expired-capture result; nearing the deadline or choosing to stop is not enough. A complete capture can truthfully retain a nonzero domain exit (including expected rejection or unavailable dependency); it is no
Line truncated
     appendSystemPrompt: `Caller execution scheduling (fixture contract):
 This session has at most 25 assistant turns, including required verification and final artifacts. The command boundary applies to each Bash call, not to the number of independent tool calls in an assistant turn.
 After required clock and approval prerequisites settle, issue independent source Reads and read-only discovery together as separate native tool calls once their paths and inputs are known. Read-only discovery is Read, Glob, Grep and the declared inventory, Git and review-read forms; a capture, a deadline guard start or run, or a diagnostic-client run is a probe, never discovery. For example, after the first clock read, one response can Read the caller excerpt, README.md, every product file and the referenced checklist and templates. Wait for their results before decisions that depend on them.

test/qa-exploratory-callers.test.ts

@@ -155,6 +155,47 @@ describe('caller native-event observer controls', () => {
     } finally { await fixture.close(); fs.rmSync(fixture.root, { recursive: true, force: true }); }
   });
 
+  test('captured section skips 38973329985 and 38993076767: the reads stay required, and the prompt binds names instead of teaching helper forms', async () => {
+    // Paid gate runs 38973329985 and 38993076767, ship-exploratory-small-cli: both actors read ship/sections/review-army.md, never
+    // read qa/sections/exploratory.md or system-functional.md, ran captures with the syntax the session prompt taught, then a bare
+    // materialize the prompt had not taught. Every passing session in the window read both sections.
+    const captured = JSON.parse(fs.readFileSync(path.join(import.meta.dir, 'fixtures/qa-caller-section-skip-ship-small-cli.json'), 'utf8'));
+    const reportRoot = fs.mkdtempSync(path.join(fs.realpathSync(os.tmpdir()), 'qc-skip-'));
+    checkpointRoots.push(reportRoot);
+    const validate = (transcript: unknown[]) => validateCallerEvidence({
+      caller: 'ship', result: { exitReason: 'success', transcript }, probes: [], receipt: { status: 'pass', probes: [], remaining: [] },
+      currentSnapshot: snapshot, requiredCharters: [], mutations: [], observerComplete: true,
+      reportRoot, checkpointFiles: {}, reportMarkdown: '',
+    });
+    const missing = ['missing executed resource read: /qa/sections/exploratory.md', 'missing executed resource read: /qa/sections/system-functional.md'];
+    for (const run of ['38973329985', '38993076767']) {
+      const events = captured.runs[run] as unknown[];
+      expect(callerTools(events).filter(tool => / materialize \/qc\/product\/reports$/.test(String(tool.input.command)))).toHaveLength(1);
+      const errors = validate(events);
+      for (const error of missing) expect(errors).toContain(error);
+      const control = [...events.slice(0, 4), ...nativeCall(`${run}-exploratory`, 'Read', { file_path: '/qc/host/runtime/qa/sections/exploratory.md' }, '# Shared exploratory QA'),
+        ...nativeCall(`${run}-functional`, 'Read', { file_path: '/qc/host/runtime/qa/sections/system-functional.md' }, '# Functional QA'), ...events.slice(4)];
+      for (const error of missing) expect(validate(control)).not.toContain(error);
+    }
+    for (const caseId of QA_CALLER_CASES) {
+      const fixture = createQaCallerFixture(caseId, { installRuntime: false });
+      try {
+        const prompt = qaCallerSessionOptions(fixture, 'free-bindings').prompt;
+        expect(prompt).not.toMatch(/gstack-qa-(?:evidence|deadline) (?:capture|checkpoint|materialize|start|status|run)\b|--after PREV|--hypothesis/);
+        expect(prompt).toContain('forms are in the installed qa/sections/exploratory.md this phase requires you to Read');
+        for (const binding of [`DEADLINE_TOOL = ${fixture.runtime}/bin/gstack-qa-deadline`, `EVIDENCE_TOOL = ${fixture.runtime}/bin/gstack-qa-evidence`,
+          `PROBE_DIR = ${fixture.cwd}/reports`, `DEADLINE_FILE = ${fixture.cwd}/reports/deadline.json`, 'so --public is approved', '--timeout-ms 10000']) {
+          expect(prompt).toContain(binding);
+        }
+      } finally { await fixture.close(); fs.rmSync(fixture.root, { recursive: true, force: true }); }
+    }
+    const section = fs.readFileSync(path.join(import.meta.dir, '../qa/sections/exploratory.md'), 'utf8');
+    for (const form of ['bun DEADLINE_TOOL start DEADLINE_FILE SECONDS [EARLIER_UTC]', 'bun EVIDENCE_TOOL capture PROBE_DIR NNN [--public] --deadline DEADLINE_FILE -- COMMAND ARGS',
+      "--after PREV --hypothesis 'why'", 'bun EVIDENCE_TOOL materialize PROBE_DIR annotations.json']) {
+      expect(section).toContain(form);
+    }
+  });
+
   test.each(['pending-result', 'same-event', 'same-message-id'])('method Read %s cannot authorize a dependent probe', ordering => {
     const observed = evidence();
     const events = observed.result.transcript as any[];