fix(evals): docs-sync observer ignores an unexecuted candidate call that exited 127 at a missing path

PR #3295's gate (run 39223031058, slice 2) failed ship-docsync-current after Opus
called gstack-docs-candidate compare at a path missing the fixture-home segment,
got exit 127 (no such file or directory), and reran it at the declared path. The
observer now ignores such a call only when the absolute binary path does not
exist, the recorded result is exactly the shell's exit-127 not-found message for
that path, and a later call runs the same command at the declared path without
failing. A free replay of all 43 captured tool calls passes and fails on the
previous observer; negative controls cover an existing wrong-path binary, other
exits and messages, a missing/earlier/failing/changed retry, and non-candidate or
relative binaries. GStack Minder-approved.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-19 09:07:50 +00:00
1 parent 3918043a27
commit 5fd97c6b4f
5 files changed
+416 -5

No files matched your search

+2 -1
View File
@@ -2,13 +2,14 @@
## [Unreleased]
**The qa-only deadline fixture accepts pure local reads, so `qa-only-no-fix` no longer fails when the agent checks a skill file with `wc -l …; ls …`.** Opus 5.5's first Bash call in TM1's focused `qa-only-no-fix` run at e03bfeb was `wc -l qa-only/SKILL.md; ls qa-only qa-only/sections qa/sections qa/templates`. The fixture's narrow list rejected it with `QA deadline: unsupported outer shell composition`, though it only read inside the fixture. The fixture now opts into the shared read-only grammar from 1.91.143.0, as the docs-sync fixture did. A focused Opus run with the opt-in then failed on one more read, `ls; ls qa/templates; cat TODOS.md 2>/dev/null | head -20`, so the grammar now also lets a read discard or merge its stderr. PR #3272's Opus gate (run 39217387697) then failed the case after the agent stopped the browser inside the guard with `"$B" stop`, which the checkpoint rule counted as a third probe.
**The qa-only deadline fixture accepts pure local reads, so `qa-only-no-fix` no longer fails when the agent checks a skill file with `wc -l …; ls …`.** Opus 5.5's first Bash call in TM1's focused `qa-only-no-fix` run at e03bfeb was `wc -l qa-only/SKILL.md; ls qa-only qa-only/sections qa/sections qa/templates`. The fixture's narrow list rejected it with `QA deadline: unsupported outer shell composition`, though it only read inside the fixture. The fixture now opts into the shared read-only grammar from 1.91.143.0, as the docs-sync fixture did. A focused Opus run with the opt-in then failed on one more read, `ls; ls qa/templates; cat TODOS.md 2>/dev/null | head -20`, so the grammar now also lets a read discard or merge its stderr. PR #3272's Opus gate (run 39217387697) then failed the case after the agent stopped the browser inside the guard with `"$B" stop`, which the checkpoint rule counted as a third probe. This PR's own gate (run 39223031058) failed `ship-docsync-current` after Opus called the docs candidate helper at a path that does not exist, got exit 127, and reran the same command at the declared path.
### Changed
- **`test/helpers/qa-browser-deadline-evidence.ts`:** the deadline check also accepts any command the shared grammar allows inside the fixture directory, and the fixture prompt declares the grammar instead of "no other ls, wc, cat or find Bash calls". Receipts are still trusted only from the guard: a grammar read whose output has a line starting with `QA_DEADLINE ` fails as `receipt outside trusted guard invocation`. A grammar read of skill prose that only mentions QA_DEADLINE receipts mid-line passes, just as the same file read with Read does; the second focused Opus run tripped on `cat qa-only/sections/exploratory.md`. The fixture's existing allowed commands keep the stricter any-occurrence check.
- **Gate integrity, widening:** pure reads inside the fixture root, such as listing `qa/sections` or `ls -laR` of the screenshots directory, now pass where they used to fail. A free replay runs the captured slip inside a real guarded attempt. Reads that leave the root (`…/../../..`, `/etc`), joins with a non-read tool, redirects, `| tee`, `awk` and `sed -i` still fail. No threshold, kind, trial or behavior assertion changes. The `qa-quick`, `qa-only-no-fix`, `qa-fix-loop` and `qa-bootstrap` touchfiles add `test/helpers/read-only-commands.ts`.
- **Gate integrity, widening (shared grammar, GStack Minder-approved):** a read may end with exactly `2>/dev/null` or `2>&1`. Both fixtures that use the grammar accept it: qa-only and docs-sync. Every other redirect still fails, each with its own negative control: `2>` to a file, `2> /dev/null` with a space, `>` and `>>` on stdout, `&>`, `1>/dev/null`, `3>/dev/null`, `>&` and `2>&` to a file, `2>&2`, `2>/dev/tcp/…`, a leading `2>/dev/null`, a quoted `'2'>`, and `| tee`. The main control `ls docs 2>/dev/null` became `ls docs 2>docs/err.txt`. Two existing docs-sync controls only failed because of their stderr redirect, and each got a replacement that still fails: `ls 2>&1 | head` became `ls 2>&1 | sh`. The stored miss `cat {other}/bash-placeholder 2>/dev/null; cat {home}/state/config.yaml` moved to known_good_posix, and its `2>` to a file and `1>/dev/null` variants stay known_bad. The docs-sync prompt carries the updated grammar text, so the next PR gate selects the docs-sync cases.
- **Gate integrity, widening (checkpoint rule):** a guarded child that is exactly `"$B" stop`, `$B stop` or `<browse> stop` now counts as browser cleanup, the same as `closetab`. Cleanup needs no checkpoint and may come only after the last probe. The fixture prompt names both cleanup commands. A free replay of the three guarded calls captured in run 39217387697 (two probes with one checkpoint, then `stop`) passes, and fails on the previous rule with `follow-up lacks one acknowledged preceding checkpoint`. These still fail: `stop` before a probe (`browser cleanup preceded a probe`), a probe disguised as cleanup (`"$B" goto …; "$B" stop`), `stop --all`, and `stop` outside the guard (direct, or through a shell variable). The replay fixture `test/fixtures/qa-only-stop-cleanup-public.json` is registered in the `qa-only-no-fix` touchfiles, next to the existing observation fixture.
- **Gate integrity, widening (docs-sync observer, GStack Minder-approved):** only executed candidate calls count. The observer ignores a `gstack-docs-candidate` call outside the declared path only when all of these hold: the binary is an absolute literal path that does not exist, the recorded result is exactly `Exit code 127` with the shell's `no such file or directory: <that path>`, and a later call runs the same command at the declared path without failing. A free replay of all 43 tool calls captured in run 39223031058 passes, and fails on the previous observer. These still fail: a wrong-path binary that exists (whether it runs or claims 127), any other exit code or shell message, a 127 with no later declared-path run (or one that comes before, fails, or changes the arguments), and a non-candidate or relative binary. No threshold, kind, trial or behavior assertion changes. The replay fixture `test/fixtures/docsync-candidate-not-found-public.json` is registered in the `ship-docsync-current` touchfiles.
## [1.91.144.0] - 2026-10-18
+84 -1
View File
@@ -3,7 +3,7 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { fixtureDocs } from './helpers/docsync-fixture';
import { docsCommandAllowed, docsNativeInterface, docsPreambleCommands, docsToolFailures } from './helpers/docsync-observer';
import type { SkillTestResult } from './helpers/session-runner';
import { parseNDJSON, type SkillTestResult } from './helpers/session-runner';
let fixture: ReturnType<typeof fixtureDocs>;
beforeAll(() => { fixture = fixtureDocs('updated'); });
@@ -140,3 +140,86 @@ describe('stored docsync observer commands', () => {
}
});
});
describe.skipIf(process.platform === 'win32')('candidate call at a missing binary path (exit 127)', () => {
const captured = JSON.parse(fs.readFileSync(path.join(import.meta.dir, 'fixtures/docsync-candidate-not-found-public.json'), 'utf8')) as
{ calls: SkillTestResult['toolCalls'] };
let current: ReturnType<typeof fixtureDocs>;
let publish: string;
let missing: string;
beforeAll(() => {
current = fixtureDocs('current');
publish = path.join(current.home, 'fixture-publish.ts');
missing = path.join(current.home, 'missing-shard');
});
afterAll(() => current?.clean());
const fill = (text: string) => text.replaceAll('{shard}', missing).replaceAll('{home}', current.home);
const replay = (calls = captured.calls) => ({ toolCalls: calls.map(call => ({ tool: call.tool, output: fill(call.output),
input: Object.fromEntries(Object.entries(call.input).map(([key, value]) => [key, typeof value === 'string' ? fill(value) : value])) })) }) as SkillTestResult;
const wrong = () => captured.calls.findIndex(call => call.output.startsWith('Exit code 127'));
const retry = () => wrong() + 1;
const failures = (result: SkillTestResult) => docsToolFailures(result, current, [publish], true);
const pair = (wrongOutput: string, retryOutput = '{}', wrongCommand = captured.calls[wrong()].input.command) => replay([
{ tool: 'Bash', input: { command: wrongCommand }, output: wrongOutput },
{ tool: 'Bash', input: { command: captured.calls[retry()].input.command }, output: retryOutput },
]);
const notFound = (binary: string) => `Exit code 127\n(eval):1: no such file or directory: ${binary}`;
const wrongBinary = () => fill(captured.calls[wrong()].input.command.split(' ')[0]);
test('captured run 39223031058 ship-docsync-current passes; its wrong-path call alone is outside the interface', () => {
const result = replay();
expect(result.toolCalls[retry()].input.command).toBe(result.toolCalls[wrong()].input.command.replace(wrongBinary(),
`${current.skills}/bin/gstack-docs-candidate`));
expect(docsCommandAllowed(result.toolCalls[wrong()].input.command, current, [publish])).toBe(false);
expect(failures(result)).toEqual([]);
});
test('the native exit-127 tool result reaches the observer as the recorded output', () => {
const command = fill(captured.calls[wrong()].input.command);
const parsed = parseNDJSON([
{ type: 'assistant', parent_tool_use_id: 'child', message: { content: [{ type: 'tool_use', id: 'wrong', name: 'Bash', input: { command } }] } },
{ type: 'user', parent_tool_use_id: 'child', message: { content: [{ type: 'tool_result', tool_use_id: 'wrong', content: notFound(wrongBinary()), is_error: true }] },
tool_use_result: `Error: ${notFound(wrongBinary())}` },
].map(event => JSON.stringify(event)));
expect(parsed.toolCalls[0].output).toBe(notFound(wrongBinary()));
});
test('a 127 with no later run of the same command at the declared path still fails', () => {
const calls = captured.calls.filter((_, index) => index !== retry());
expect(failures(replay(calls))).toContain('command outside declared docs observation interface');
const before = [captured.calls[retry()], ...calls];
expect(failures(replay(before))).toContain('command outside declared docs observation interface');
expect(failures(pair(notFound(wrongBinary()), 'Exit code 1\nboom'))).toContain('command outside declared docs observation interface');
const otherArgs = captured.calls[wrong()].input.command.replace('ship2-docs-a1.json', 'other.json');
expect(failures(pair(notFound(wrongBinary()), '{}', otherArgs))).toContain('command outside declared docs observation interface');
});
test('any other exit or shell message outside the declared path still fails', () => {
expect(failures(pair(notFound(wrongBinary())))).toEqual([]);
for (const output of [`Exit code 126\n(eval):1: permission denied: ${wrongBinary()}`, 'Exit code 1\nerror', '',
`Exit code 127\n(eval):1: command not found: ${wrongBinary()}`, notFound(wrongBinary()) + '\nextra',
notFound(path.join(missing, 'other/gstack-docs-candidate'))]) {
expect(failures(pair(output)), output).toContain('command outside declared docs observation interface');
}
});
test('a wrong-path binary that exists still fails, whether it ran or claims 127', () => {
fs.mkdirSync(path.dirname(wrongBinary()), { recursive: true });
fs.writeFileSync(wrongBinary(), 'candidate stand-in\n');
try {
for (const output of ['{}', notFound(wrongBinary())]) {
expect(failures(pair(output)), output).toContain('command outside declared docs observation interface');
}
} finally { fs.rmSync(missing, { recursive: true, force: true }); }
});
test('only a candidate binary at an absolute literal path qualifies', () => {
const declared = current.skills.split(path.sep).join('/');
const skillEnd = `${missing}/.claude/skills/gstack/bin/gstack-skill-start --help`;
expect(failures(replay([{ tool: 'Bash', input: { command: skillEnd }, output: notFound(skillEnd.split(' ')[0]) },
{ tool: 'Bash', input: { command: `${declared}/bin/gstack-skill-start --help` }, output: 'usage' }]))).toContain('command outside declared docs observation interface');
const relative = 'missing/gstack-docs-candidate --help';
expect(failures(replay([{ tool: 'Bash', input: { command: relative }, output: notFound('missing/gstack-docs-candidate') },
{ tool: 'Bash', input: { command: `${declared}/bin/gstack-docs-candidate --help` }, output: 'usage' }]))).toContain('command outside declared docs observation interface');
});
});
+308
View File
@@ -0,0 +1,308 @@
{
"sourceRun": "39223031058 slice 2 ship-docsync-current (PR #3295 head 3918043)",
"sourceFileSha256": "7e4a144fcb47cfff0d54f51a839dd51451054cd8af9e9d36c73516b8b0ae6aef",
"note": "Every tool call of the captured parent and child in order. {home} is the fixture home and {shard} its shard root. Inputs keep only command or file_path; outputs are kept only for gstack-docs-candidate calls and failed calls.",
"calls": [
{
"tool": "Read",
"input": {
"file_path": "{home}/phase.md"
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/fixture-interface.md"
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/.claude/skills/gstack/ship/sections/documentation.md"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "find {home}/.claude/skills/gstack/document-release {home}/.claude/skills/gstack/bin -maxdepth 3"
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/.claude/skills/gstack/document-release/SKILL.md"
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/.claude/skills/gstack/document-release/sections/audit-scope.md"
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/.claude/skills/gstack/document-release/sections/release-body.md"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git status"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git merge-base main HEAD"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git ls-files --cached --others --exclude-standard"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "{home}/.claude/skills/gstack/bin/gstack-docs-candidate --help"
},
"output": "usage: gstack-docs-candidate snapshot --out <candidate.json> --audit-id <id> --mode edit|read-only --base <rev> [--select <path>]... [--docs <path>]...\n gstack-docs-candidate compare <candidate.json>"
},
{
"tool": "Bash",
"input": {
"command": "git diff c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 HEAD --stat"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git diff c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 HEAD"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "ls -la {home} {home}/repo"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "{home}/.claude/skills/gstack/bin/gstack-docs-candidate snapshot --out {home}/ship2-docs-a1.json --audit-id ship2-docs-20261019-a1 --mode edit --base c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 --docs handbook --docs README.md --docs AGENTS.md"
},
"output": "CANDIDATE: {home}/ship2-docs-a1.json\nbase=c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 head=940ecb8e1008f52768d1f98ffed974d039a32251 branch=feature/docs mode=edit\ncommitted=1 staged=0 unstaged=0 untracked=1 selected=2 docs=3 hashed=4\ncommitted: \"handbook/reference/commands/widget.md.tmpl\"\nuntracked: \"personal-note.txt\"\ndocs: \"AGENTS.md\" \"README.md\" \"handbook/reference/commands/widget.md.tmpl\""
},
{
"tool": "Bash",
"input": {
"command": "{home}/.claude/skills/gstack/bin/gstack-docs-candidate snapshot --out {home}/ship2-docs-a1.json --audit-id ship2-docs-20261019-a1 --mode edit --base c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 --select handbook/reference/commands/widget.md.tmpl --docs handbook --docs README.md --docs AGENTS.md"
},
"output": "CANDIDATE: {home}/ship2-docs-a1.json\nbase=c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 head=940ecb8e1008f52768d1f98ffed974d039a32251 branch=feature/docs mode=edit\ncommitted=1 staged=0 unstaged=0 untracked=1 selected=1 docs=3 hashed=4\ncommitted: \"handbook/reference/commands/widget.md.tmpl\"\nuntracked: \"personal-note.txt\"\ndocs: \"AGENTS.md\" \"README.md\" \"handbook/reference/commands/widget.md.tmpl\""
},
{
"tool": "Agent",
"input": {
"run_in_background": false
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/fixture-interface.md"
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/.claude/skills/gstack/document-release/SKILL.md"
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/.claude/skills/gstack/document-release/sections/audit-scope.md"
},
"output": ""
},
{
"tool": "Read",
"input": {
"file_path": "{home}/.claude/skills/gstack/document-release/sections/release-body.md"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "GSTACK_SESSION_KIND=spawned {home}/.claude/skills/gstack/bin/gstack-skill-start --skill document-release --model claude"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "cat {home}/ship2-docs-a1.json"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git status"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git ls-files -z --cached --others --exclude-standard"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git diff c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 HEAD"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git diff --cached"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git diff"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "cat AGENTS.md README.md handbook/reference/commands/widget.md.tmpl app.ts package.json"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git log c37ee3a1a1c98972abc7cd26cb49a6f388c7c608..HEAD --oneline"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "cat CHANGELOG.md VERSION TODOS.md"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "{home}/.claude/skills/gstack/bin/gstack-skill-end --skill document-release --outcome success --session-id 842-1792399327-88f7a744 --tel-start 1792399327 --used-browse no"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "{shard}/.claude/skills/gstack/bin/gstack-docs-candidate compare {home}/ship2-docs-a1.json"
},
"output": "Exit code 127\n(eval):1: no such file or directory: {shard}/.claude/skills/gstack/bin/gstack-docs-candidate"
},
{
"tool": "Bash",
"input": {
"command": "{home}/.claude/skills/gstack/bin/gstack-docs-candidate compare {home}/ship2-docs-a1.json"
},
"output": "{\"audit_id\":\"ship2-docs-20261019-a1\",\"head_changed\":false,\"index_changed\":false,\"content_changed\":[],\"newly_dirty\":[]}"
},
{
"tool": "Bash",
"input": {
"command": "git status"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "git rev-parse HEAD"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "cat app.ts README.md AGENTS.md handbook/reference/commands/widget.md.tmpl CHANGELOG.md VERSION package.json TODOS.md"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "{home}/.claude/skills/gstack/bin/gstack-docs-candidate snapshot --out {home}/ship2-docs-20261019-a1-post.json --audit-id ship2-docs-20261019-a1 --mode edit --base c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 --select handbook/reference/commands/widget.md.tmpl --docs handbook --docs README.md --docs AGENTS.md"
},
"output": "CANDIDATE: {home}/ship2-docs-20261019-a1-post.json\nbase=c37ee3a1a1c98972abc7cd26cb49a6f388c7c608 head=940ecb8e1008f52768d1f98ffed974d039a32251 branch=feature/docs mode=edit\ncommitted=1 staged=0 unstaged=0 untracked=1 selected=1 docs=3 hashed=4\ncommitted: \"handbook/reference/commands/widget.md.tmpl\"\nuntracked: \"personal-note.txt\"\ndocs: \"AGENTS.md\" \"README.md\" \"handbook/reference/commands/widget.md.tmpl\""
},
{
"tool": "Write",
"input": {
"file_path": "{home}/ship2-docs-20261019-a1-documentation.md"
},
"output": ""
},
{
"tool": "Write",
"input": {
"file_path": "{home}/ship-report.md"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "cat {home}/ship2-docs-20261019-a1-documentation.md >> {home}/ship-report.md"
},
"output": ""
},
{
"tool": "Bash",
"input": {
"command": "bun {home}/fixture-publish.ts"
},
"output": ""
},
{
"tool": "Edit",
"input": {
"file_path": "{home}/ship-report.md"
},
"output": ""
}
]
}
+21 -2
View File
@@ -367,12 +367,31 @@ function docsPrivateArtifact(file: string, fixture: ReturnType<typeof fixtureDoc
path.basename(file) !== 'actor-state.json' && path.basename(actual) !== 'actor-state.json';
}
/** A candidate call at a wrong binary path that does not exist exited 127 before anything ran; it is ignored only when the same command at the declared path follows and runs. */
function candidateNotFound(calls: SkillTestResult['toolCalls'], index: number, fixture: ReturnType<typeof fixtureDocs>, scripts: string[]): boolean {
const command = String(calls[index].input?.command ?? '').trim();
const binary = command.split(' ')[0]!;
const candidate = path.join(fixture.skills, 'bin/gstack-docs-candidate').split(path.sep).join('/');
if (!/^\/[\w.\/-]+\/gstack-docs-candidate$/.test(binary) || binary === candidate) return false;
if (calls[index].output.trimEnd() !== `Exit code 127\n(eval):1: no such file or directory: ${binary}`) return false;
try {
fs.lstatSync(binary);
return false;
} catch (error) {
if (!['ENOENT', 'ENOTDIR'].includes((error as NodeJS.ErrnoException).code ?? '')) return false;
}
const retry = candidate + command.slice(binary.length);
return docsCommandAllowed(retry, fixture, scripts) && calls.slice(index + 1).some(call =>
call.tool === 'Bash' && String(call.input?.command ?? '').trim() === retry && !call.output.startsWith('Exit code'));
}
export function docsToolFailures(result: SkillTestResult, fixture: ReturnType<typeof fixtureDocs>, scripts: string[] = [], readOnly = false): string[] {
const failures: string[] = [];
const repo = fs.realpathSync(fixture.repo);
const authoredDoc = path.join(repo, DOC_PATH);
for (const call of result.toolCalls) {
if (call.tool === 'Bash' && !docsCommandAllowed(String(call.input?.command ?? ''), fixture, scripts)) failures.push('command outside declared docs observation interface');
for (const [index, call] of result.toolCalls.entries()) {
if (call.tool === 'Bash' && !docsCommandAllowed(String(call.input?.command ?? ''), fixture, scripts) &&
!candidateNotFound(result.toolCalls, index, fixture, scripts)) failures.push('command outside declared docs observation interface');
if (['Write', 'Edit'].includes(call.tool)) {
const file = path.resolve(fixture.repo, call.input?.file_path ?? '');
const actual = actualWritePath(file);
+1 -1
View File
@@ -815,7 +815,7 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
'scripts/resolvers/testing.ts'
],
'ship-docsync-completion': ['bin/gstack-docs-candidate', 'bin/gstack-state-root.sh', 'lib/state-root.ts', 'ship/**', 'document-release/**', 'test/skill-e2e-ship-docsync.test.ts', 'test/helpers/docsync-*.ts', 'test/helpers/read-only-commands.ts', 'test/helpers/session-runner.ts', 'test/helpers/hermetic-env.ts', 'bin/gstack-skill-start', 'bin/gstack-session-kind', 'scripts/resolvers/sections.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/gen-skill-docs.ts', 'scripts/resolvers/testing.ts', 'test/helpers/qa-checkpoint-evidence.ts', 'test/helpers/qa-functional-observer.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/qa-evidence-producer.ts', 'test/helpers/qa-functional-fixture.ts', 'bin/gstack-context-recovery'],
'ship-docsync-current': ['bin/gstack-docs-candidate', 'bin/gstack-state-root.sh', 'lib/state-root.ts', 'ship/**', 'document-release/**', 'test/skill-e2e-ship-docsync.test.ts', 'test/helpers/docsync-*.ts', 'test/helpers/read-only-commands.ts', 'test/helpers/session-runner.ts', 'test/helpers/hermetic-env.ts', 'bin/gstack-skill-start', 'bin/gstack-session-kind', 'scripts/resolvers/sections.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/gen-skill-docs.ts', 'scripts/resolvers/testing.ts', 'test/helpers/qa-checkpoint-evidence.ts', 'test/helpers/qa-functional-observer.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/qa-evidence-producer.ts', 'test/helpers/qa-functional-fixture.ts', 'bin/gstack-context-recovery'],
'ship-docsync-current': ['bin/gstack-docs-candidate', 'bin/gstack-state-root.sh', 'lib/state-root.ts', 'ship/**', 'document-release/**', 'test/skill-e2e-ship-docsync.test.ts', 'test/helpers/docsync-*.ts', 'test/helpers/read-only-commands.ts', 'test/helpers/session-runner.ts', 'test/helpers/hermetic-env.ts', 'bin/gstack-skill-start', 'bin/gstack-session-kind', 'scripts/resolvers/sections.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/gen-skill-docs.ts', 'scripts/resolvers/testing.ts', 'test/helpers/qa-checkpoint-evidence.ts', 'test/helpers/qa-functional-observer.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/qa-evidence-producer.ts', 'test/helpers/qa-functional-fixture.ts', 'bin/gstack-context-recovery', 'test/fixtures/docsync-candidate-not-found-public.json'],
'ship-docsync-failure': ['bin/gstack-docs-candidate', 'bin/gstack-state-root.sh', 'lib/state-root.ts', 'ship/**', 'document-release/**', 'test/skill-e2e-ship-docsync.test.ts', 'test/helpers/docsync-*.ts', 'test/helpers/read-only-commands.ts', 'test/helpers/session-runner.ts', 'test/helpers/hermetic-env.ts', 'bin/gstack-skill-start', 'bin/gstack-session-kind', 'scripts/resolvers/sections.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/gen-skill-docs.ts', 'scripts/resolvers/testing.ts', 'test/helpers/qa-checkpoint-evidence.ts', 'test/helpers/qa-functional-observer.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/qa-evidence-producer.ts', 'test/helpers/qa-functional-fixture.ts', 'bin/gstack-context-recovery'],
'ship-docsync-store': ['bin/gstack-docs-candidate', 'bin/gstack-state-root.sh', 'lib/state-root.ts', 'ship/**', 'document-release/**', 'test/skill-e2e-ship-docsync.test.ts', 'test/helpers/docsync-*.ts', 'test/helpers/read-only-commands.ts', 'test/helpers/session-runner.ts', 'test/helpers/hermetic-env.ts', 'bin/gstack-skill-start', 'bin/gstack-session-kind', 'scripts/resolvers/sections.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/gen-skill-docs.ts', 'scripts/resolvers/testing.ts', 'test/helpers/qa-checkpoint-evidence.ts', 'test/helpers/qa-functional-observer.ts', 'test/helpers/e2e-gate.ts', 'test/helpers/qa-evidence-producer.ts', 'test/helpers/qa-functional-fixture.ts', 'bin/gstack-context-recovery'],
// #2733 behavioral proof: the JSON contract survives a firing gate inside a