mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-31 22:14:25 +01:00
* fix(preamble): completeness means complete coverage with minimal code (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(preamble): tier 2+ skills load the shared preamble once per session (GSTA-96) Every tier 2+ SKILL.md inlined the same ~23 KB preamble. It now renders once per host into sections/preamble.md beside each tier 2+ SKILL.md; the SKILL.md keeps a pointer that asks for the Read only when the file's id is not already in context. Only sections byte-identical to the shared render move, so every skill keeps exactly its pre-hoist paragraphs (verified across all 41 skills). Parity caps for ten skills raised to their measured unions (owner-approved, Garry 2026-10-25: "do it"); 63 size pins lowered and 26 deleted. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(paid-select): a generated sections/preamble.md takes its skill template's identity in the PR planner (GSTA-96) screen-summary failed on #3434 with 'PR profile requires full validation' for nine skills whose only paid coverage is periodic: their new sections/preamble.md had no dependency identity, while the SKILL.md beside it inherits its template's. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(section-manifest): the shared sections/preamble.md renders from the hoist resolver, not a section template (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (CHANGELOG says 3 characters shorter; model-overlays guards the coverage half) The completeness section shrank by 3 characters (5 bytes), matching the 89 pins lowered by 3. The default-copy test pinned only the minimal-code half; an expectMentions check now fails if the full-coverage or never-excuses-a-gap clause is dropped. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (model-overlays binds smaller diff to equal coverage) The default-copy check now also fails if 'on equal coverage' is dropped from the smaller-diff rule (which would tell models to prefer the smaller diff unconditionally) or if the no-impossible-state-checks clause is removed. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG lists the model-overlays coverage checks The [Unreleased] Tests bullet now says the default-copy test checks both halves of the rule (full coverage of the requirement; smaller diff only at equal coverage; no impossible-state checks; never excuses a gap), matching88e3cc0and44eacaf. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: plan-ceo-review and ETHOS say completeness means coverage, not more code Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (CEO bullet bounds code by accepted scope; tests bind each completeness rule) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: cover the shared-preamble hoist (ship Step 7) Adds test/preamble-hoist.test.ts (tier 1 unchanged; tier 2+ SKILL.md plus sections/preamble.md reproduces the pre-hoist preamble; pointer position and id) and two gen-skill-docs-checks cases (one shared preamble per host, beside every pointer and named by its id; preamble artifact kind). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (ship Step 9.4) - setup: follow a symlinked sections/ dir in the carved-pointer status check - touchfiles: select preamble-hoist.ts alongside every preamble.ts entry - section-manifest: exempt sections/preamble.md only for skills whose SKILL.md carries the shared-preamble pointer - skill-size-budget: growth ceiling measures SKILL.md plus its shared preamble - host-conformance: tolerate a missing sections/ dir, as setup does - skill-validation: read bin/gstack-skill-start directly - skill-fixture: error messages and docblocks name the actual boundary (harness-version non-behavioral note updated) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: describe the size-budget growth check's startup measure (ship Step 9.4, pass 2) The growth ceiling now measures SKILL.md plus the shared preamble, the same measure as the shrink floor; the docblock said SKILL.md only. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG says simplification.md quotes the old rule and CLAUDE.md restates the new one Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG lists the hoist tests and the growth-ceiling measure (ship Step 9.4) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: cover the symlinked sections/ status check and the ETHOS house-rule quote (ship Step 7) host-conformance: --status still finds a broken bare sections/ pointer when a carved Claude skill's installed sections/ is a directory symlink (fails at the pre-repair0e5d11fd9). gen-skill-docs: the house rule review/specialists/ simplification.md quotes from ETHOS.md still appears in ETHOS.md. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: describe the shared preamble file (ship Step 14.5 docs audit) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: the shared-preamble skip needs the skill's own Read; digest and first-run intro say complete coverage, minimal code (GSTA-96) The pointer skipped the shared file when any text carrying its id was in context, so a page or file with that heading could stand in for the anti-spoofing rules (security, red team and adversarial review, ship run 2). It now requires the skill's own earlier Read. The digest's Boil the Ocean bullet and the first-run intro still said 'AI makes completeness cheap, so do the complete thing' (docs audit blocker); both now carry Garry's 'complete coverage, minimal code' (2026-10-25: '1/ yes fix this'). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: pin the shared-preamble provenance skip and the new Boil the Ocean wording (ship Step 7) preamble-hoist: on every host, tier and render option the pointer allows the skip only when the skill's own Read is in context. agents-digest and gstack-skill-start: the digest bullet and the lake intro say complete coverage, minimal code, never the retired phrasing. Each fails at9af36d010. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: describe the provenance skip where the docs still described the id-only one (ship Step 9.4) The preamble-hoist.ts docblock and ARCHITECTURE.md still said the shared preamble is re-read only when its id is absent; since2acf1a44athe pointer requires the skill's own earlier Read of the file. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG lists the provenance-pointer and Boil the Ocean wording tests (ship Step 13) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * revert(digest): keep the agents digest at main's bytes; an unstamped head cannot change a release output (GSTA-96) CI 'Check VERSION is not stale vs queue' classified #3434 as stamped because it touched agents-digest/gstack-AGENTS.md (a release_outputs path), while regenerating only the generator fails the digest freshness test. The first-run intro rewording stays. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: voice judge reads the hoisted preamble; register ETHOS.md on its reader cases Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: refresh the claude ship golden the main merge took from main (ship Step 5 preflight) The1a4dcaf07merge resolved test/fixtures/golden/claude-ship-SKILL.md to main's pre-hoist bytes. The golden is a copy of ship/SKILL.md; regenerated with bun scripts/gen-ship-goldens.ts (pregate regen: REGEN_STALE). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: pr-dependencies' unplaceable-file example uses SECURITY.md now that ETHOS.md is mapped6183ca8aeregistered ETHOS.md on its reader cases in touchfiles-data.ts, so a PR touching only ETHOS.md plans in pr mode and no longer restores the full gate. The fail-closed test and the real-unknown label check now use SECURITY.md, which is tracked and unmapped; the assertions are unchanged. CHANGELOG names the third test file. (CI free shard 7 at 7c21720ac; remote pregate shard 6.) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: host path-leak checks read the moved shared preamble too; name SHARED_PREAMBLE_FILE (ship review pass 2) The external-host path-leak assertions in gen-skill-docs.test.ts (Codex, Factory) and host-config.test.ts (Copilot, Antigravity) read SKILL.md alone, so a Claude path that moved into sections/preamble.md with the hoist went unchecked. They now read the skill's startup text through readSkillStartup. workflow-judge-input and host-conformance use the SHARED_PREAMBLE_FILE constant instead of a bare 'preamble.md' literal, and preamble.ts's header describes the hoisted composition. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG token savings name the per-skill skip and the autoplan follow-up (ship review pass 2) The pointer's skip is bound to the skill's own copy of sections/preamble.md, so "every later one saves about 5,800" holds per skill, not per session, and /autoplan's skip list and the inline /office-hours loader do not yet skip the pointer section. The entry now says so instead of claiming the full /autoplan saving. Adding the skip entry grows size-pinned skills, so it is a follow-up. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: corpus budget measures startup bytes; carve-safety counts shared preamble as uncarved (ship review pass 2) skill-size-budget's growth ceiling and shrink floor already measured SKILL.md plus sections/preamble.md, but the total-corpus check still summed hoisted SKILL.md bytes against the pre-hoist baseline, hiding about 0.9 MB of headroom. It now sums the same startup bytes (3,114,885 -> 2,590,484, x0.832); the unreachable SKILL.md-missing fallback is gone. auq-format-always-loaded's carve-safety guard treated any sections/ dir as carved, so every skill with only the shared preamble ran it and its sanity check passed on preamble.md. It now uses setup --status's rule: a section other than the shared preamble. Those skills keep the per-skill checks. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: remaining SKILL.md-only negatives read the shared preamble too (ship review pass 3) The parameterized external-host path-leak smoke test, the two Codex .claude path checks and both vestigial-handshake negatives read SKILL.md alone, so text that moved into sections/preamble.md went unchecked. They now read readSkillStartup. preamble.ts's header no longer says tier 1 renders through generatePreamble(). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: ratchet skeleton caps and eager context ceilings down after the hoist (ship review pass 3) The hoist moved about 22 KB out of every tier 2+ SKILL.md, which left each hoisted maxSkeletonBytes cap and each eagerPerInvocation ceiling roughly 22 KB / 5-6K tokens slack. The 20 hoisted skeleton caps now sit at the post-hoist skeleton plus their pre-hoist headroom, and 42 context-budget ceilings take the recaptured (lower) values. Nothing was raised: the four ceilings the recapture would have raised keep their old values. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: preamble-hoist's section splitter closes a fence only on a bare delimiter (ship Step 11) Inside an open fence, a line such as "```bash" is content in CommonMark, but splitSections treated it as the closing fence, so a later "## " line in the same block could split a section mid-code-block and hoist half of it. A closing delimiter now needs nothing but whitespace after it. Every host's render is unchanged (gen-skill-docs --dry-run FRESH on all twelve hosts). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: skill-startup.ts is a global, pinned harness file like skill-fixture.ts (ship review pass 3) skill-fixture.ts is a global touchfile pinned in scripts/harness-version.json because it reshapes the skill content most E2E suites read, and it now builds that content through test/helpers/skill-startup.ts. That helper was in neither list, so a later edit to it would change fixture inputs without the harness-version guard and select only 27 keys. It is now global and pinned, recorded non-behavioral (its fixtures are byte-identical to the pre-hoist output). ARCHITECTURE.md and the CHANGELOG stop calling the pointer id a content id: it hashes the shared text before host rewrites. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG matches the merged skip lists and the baseline diff (ship review, run 4) The headline still called the autoplan and inline /office-hours skip entries a follow-up, but the main merge added them; it now says the loaded skills skip their shared Read. The #3451 sentence is dropped, because that fix shipped in v1.91.237.0 and is recorded there. The typecheck baseline line says the TS2769 entry was re-keyed for the 'preamble' artifact kind with its count unchanged, not fixed and lowered. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: pin the pointer entry in the loader skip lists; drop a carve-safety tautology (ship review, run 4) A new INVOKE_SKILL test checks that plan-ceo-review's inline /office-hours skip list and autoplan's section skip list both name the shared-preamble pointer heading, and that office-hours and the four plan reviews carry it. The carve-safety "sanity" assertion repeated the predicate discoverSkills() now uses to set sectionsDir, so it could never fail; it is removed. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: gstack-doctor finds the Codex model patch in the hoisted shared preamble (ship Step 11, run 4) The doctor's patch row grepped only the Codex review SKILL.md for the "Model-Specific Behavioral Patch (" heading. The hoist moves that section into sections/preamble.md, so the required row (for ship, review, the plan reviews, office-hours, investigate and more) would fail on every correct post-hoist Codex install and block install.sh's "gstack: ok". It now checks the SKILL.md and then its shared preamble and names the file it found. A new doctor test covers a hoisted render. preamble-hoist's variantSubsections also returns null when no subsection differs, instead of emitting a replacing heading with no subsections. Every host's render is unchanged. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(docsync): fixtureDocs installs the generated sections/preamble.md (hunk taken from #3472) A generated skill that loads the shared preamble names sections/preamble.md as a required asset before skill start, and a spawned document-release child blocks on its absence, so docsync-spawned stayed red on every #3434 screen. fixtureDocs now copies that file when the generated skill has it. This single hunk is taken unchanged from gstack#3472 (head2e3f7ba1, GSTA-107), which trains ahead of #3434, so the train merge sees identical lines. Nothing else from #3472 is included. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG notes the docsync fixture hunk taken from #3472 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG names the post-merge context-budget ratchet (captain hoisted) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(auq): planted skill fixtures resolve the shared preamble like an install (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(plan-format): mode menu delivers its question without output-harvesting wording (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): record the v1.91.270.0 merge values (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(auq): parallel-capture contract expects the planted skill with its sections path mapped (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): record the v1.91.273.0 merge re-measurements (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): restore the released entries the previous commit truncated (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): end two sentences the prose gate flags (GSTA-96) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
584 lines
34 KiB
TypeScript
584 lines
34 KiB
TypeScript
import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
|
|
import * as fs from 'node:fs';
|
|
import * as os from 'node:os';
|
|
import * as path from 'node:path';
|
|
import { spawnSync } from 'node:child_process';
|
|
import { runGeneration } from '../scripts/gen-skill-docs';
|
|
import { docsDispatchIndex, extractDocsDispatch, parseDocsCompletion, vetDocsCompletion } from './helpers/docsync-contract';
|
|
import { fixtureDocs, repoSnapshot, changedFiles, gitAt, DOC_PATH, preserveDocsEvidence } from './helpers/docsync-fixture';
|
|
import type { SkillTestResult } from './helpers/session-runner';
|
|
import { observeDocsWrites, docsWriteFailures, docsCommandAllowed, docsPreambleCommands, docsCompletedRead } from './helpers/docsync-observer';
|
|
import { docsActorCommand, docsActorHook, installDocsActor, type DocsActorState, type DocsFault } from './helpers/docsync-fault-actor';
|
|
import { docsActorVerdict } from './helpers/docsync-fault-eval';
|
|
import { expectMentions, expectTokens } from './helpers/prompt-structure';
|
|
|
|
const ROOT = path.join(import.meta.dir, '..');
|
|
const read = (p: string) => fs.readFileSync(path.join(ROOT, p), 'utf8');
|
|
let generated: string;
|
|
|
|
beforeAll(async () => {
|
|
generated = fs.mkdtempSync(path.join(os.tmpdir(), 'docsync-render-'));
|
|
for (const host of ['claude', 'codex', 'factory'] as const) {
|
|
const result = await runGeneration({ host, outputRoot: generated, contentLinkRoot: null, log: () => {} });
|
|
expect(result.exitCode).toBe(0);
|
|
}
|
|
});
|
|
|
|
afterAll(() => fs.rmSync(generated, { recursive: true, force: true }));
|
|
|
|
describe('pre-publication documentation lifecycle', () => {
|
|
test('final verification orders build, bounded documentation refresh, freeze and evidence', () => {
|
|
const body = read('ship/SKILL.md.tmpl');
|
|
const gate = body.slice(body.indexOf('## Step 16:'), body.indexOf('## Step 17:'));
|
|
const steps = ['### 1. Finish writers and prepare outputs', '### 2. Choose the change route',
|
|
'### 3. Resolve documentation freshness', '### 4. Verify the frozen candidate', '### 5. Report, then push'];
|
|
const positions = steps.map(step => gate.indexOf(step));
|
|
expect(positions.every(position => position >= 0)).toBe(true);
|
|
expect(positions).toEqual([...positions].sort((a, b) => a - b));
|
|
const recovery = body.slice(body.indexOf('### 3. Resolve documentation freshness'), body.indexOf('### 4. Verify the frozen candidate')).replace(/\s+/g, ' ');
|
|
expect(recovery).toContain('Validate the outcome before Step 15');
|
|
expect(recovery).toContain('Never run a third audit');
|
|
expect(body.replace(/\s+/g, ' ')).toContain('its initial-plus-ONE limit never resets');
|
|
expect(gate.replace(/\s+/g, ' ')).toContain('Docs, TODO edits, new/generated tests and fixes make evidence STALE');
|
|
const docs = read('ship/sections/documentation.md.tmpl');
|
|
expectMentions(docs.replace(/\s+/g, ' '), [['never', 'attempt', 'changes']], 'docs.replace(/\s+/g, )');
|
|
expect(docs.replace(/\s+/g, ' ')).toContain('Otherwise STOP before commit/publication and do not launch another child');
|
|
});
|
|
|
|
test('dispatch is carved before commit and final verification on every host', () => {
|
|
const claude = fs.readFileSync(path.join(generated, 'ship/SKILL.md'), 'utf8');
|
|
const marker = '## Step 14.5: Documentation audit (every ship)';
|
|
expect(claude.indexOf(marker)).toBeGreaterThan(0);
|
|
expect(claude.indexOf('ship/sections/documentation.md', claude.indexOf(marker))).toBeLessThan(claude.indexOf('## Step 15:'));
|
|
expect(claude).not.toContain('Dispatch /document-release as a subagent');
|
|
// C4: ship is carved on external hosts too; the pointer is relative to the installed skill.
|
|
for (const dir of ['.agents/skills/gstack-ship', '.factory/skills/gstack-ship']) {
|
|
const body = fs.readFileSync(path.join(generated, dir, 'SKILL.md'), 'utf8');
|
|
const pointer = body.indexOf('`sections/documentation.md` relative to the installed `gstack-ship` SKILL.md directory', body.indexOf(marker));
|
|
expect(pointer).toBeGreaterThan(body.indexOf(marker));
|
|
expect(pointer).toBeLessThan(body.indexOf('## Step 15:'));
|
|
expect(body).not.toContain('Dispatch /document-release as a subagent');
|
|
expect(fs.readFileSync(path.join(generated, dir, 'sections/documentation.md'), 'utf8')).toContain('Dispatch /document-release as a subagent');
|
|
expect(body.indexOf('## Step 16:')).toBeLessThan(body.indexOf('## Step 17:'));
|
|
}
|
|
});
|
|
|
|
test('the actual dispatch has foreground, marker, typed result and ownership guards', () => {
|
|
const body = fs.readFileSync(path.join(generated, 'ship/sections/documentation.md'), 'utf8');
|
|
expect(body).toContain('run_in_background: false');
|
|
expect(body).toContain('subagent_type: "general-purpose"');
|
|
const prompt = extractDocsDispatch(body);
|
|
for (const marker of ['GSTACK_SESSION_KIND=spawned', 'LAST nonempty line', 'schema_version', 'read-only',
|
|
'no Git mutation', 'CHANGELOG or TODOS mutation', 'never auto-approve', 'staged, unstaged and selected new']) expect(prompt).toContain(marker);
|
|
expect(docsDispatchIndex([{ tool: 'Agent', input: { prompt } }])).toBe(0);
|
|
expect(docsDispatchIndex([{ tool: 'Agent', input: { prompt: body } }])).toBe(-1);
|
|
expect(() => extractDocsDispatch('no markers')).toThrow();
|
|
});
|
|
|
|
test('documentation preflight follows the installed host layout', () => {
|
|
const claude = fs.readFileSync(path.join(generated, 'ship/sections/documentation.md'), 'utf8');
|
|
for (const section of ['audit-scope', 'release-body']) {
|
|
expect(fs.existsSync(path.join(generated, `document-release/sections/${section}.md`))).toBe(true);
|
|
}
|
|
for (const host of ['.agents', '.factory']) {
|
|
const ship = fs.readFileSync(path.join(generated, host, 'skills/gstack-ship/sections/documentation.md'), 'utf8');
|
|
const directory = path.join(generated, host, 'skills/gstack-document-release');
|
|
const document = fs.readFileSync(path.join(directory, 'SKILL.md'), 'utf8');
|
|
expect(document).toContain('# Documentation scope and discovery');
|
|
expect(document).toContain('## Step 2: Per-File Documentation Audit');
|
|
expect(document).toContain('## Ship-owned documentation mode');
|
|
expect(fs.existsSync(path.join(directory, 'sections/audit-scope.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(directory, 'sections/release-body.md'))).toBe(false);
|
|
}
|
|
});
|
|
|
|
test('failure is visible, bounded and settled before another writer', () => {
|
|
const body = read('ship/sections/documentation.md.tmpl').replace(/\s+/g, ' ');
|
|
for (const text of ['Terminal completion or confirmed termination is sufficient', 'the request alone is insufficient',
|
|
'an initial audit plus ONE repair/re-audit', 'specific named documentation risk', 'Preserve partial',
|
|
'Compare actual changes against the candidate',
|
|
'enforcing prompt/audit-scope permissions and protected-file exclusions',
|
|
'HEAD and index must be unchanged, existing dirty/untracked user content preserved',
|
|
'changed paths exactly `files_updated`. Reject any read-only write',
|
|
'Only verified permitted child edits may differ. Other edits or base changes make the audit stale',
|
|
'Later changes require the remaining re-audit or a risk decision',
|
|
'never silently refreshed hashes']) expect(body).toContain(text);
|
|
expect(body).not.toContain('block /ship on subagent failure');
|
|
});
|
|
|
|
test('stale detection does not spend the remaining audit, but repair and inline takeover do', () => {
|
|
const body = read('ship/sections/documentation.md.tmpl').replace(/\s+/g, ' ');
|
|
for (const text of ['Increment before each launch or inline takeover',
|
|
'including failed launches',
|
|
'A stale snapshot is neither a new attempt nor a current audit',
|
|
'inline work follows the same validation gates',
|
|
'never a third attempt, even after Step 16 changes',
|
|
'Confirm the child stopped before any repair, retry, inline takeover or other writer',
|
|
'request stop and inspect its status; the request alone is insufficient',
|
|
'If an attempt remains and either the audited inputs changed or a concrete launch/input/permission correction or reviewed patch repair is available',
|
|
'using current inputs and a fresh id/snapshot, run the remaining attempt, then validate it through Parent processing',
|
|
'Otherwise STOP before commit/publication',
|
|
'do not launch another child']) expect(body).toContain(text);
|
|
});
|
|
|
|
test('PR creation and reruns keep current and blocked audits visible', () => {
|
|
const body = read('ship/sections/pr-body.md.tmpl');
|
|
// Oct 7 wave B4: the title travels in Step 18's agent-written file and both
|
|
// PR paths publish through gstack-post, which scans the exact bytes it sends.
|
|
expect(body).toContain('`grep -Eq \'^v<new-version>( |$)\' "$TITLE_FILE"` and never publish an unprefixed title');
|
|
expect(body).toContain('gstack-pr-title-rewrite.sh <new-version> --stdin > "$TITLE_FILE"');
|
|
expect(body).toContain('gstack-post pr-create --base <base> --title-file "$TITLE_FILE"');
|
|
expect(body).toContain('gstack-post pr-title <pr-number> --title-file "$TITLE_FILE"');
|
|
expect(body).toContain('`gstack-post` detects GitHub or GitLab from the remote');
|
|
expect(body).not.toContain('Dispatch /document-release');
|
|
expectMentions(body, [['never', 'section', 'another']], 'body');
|
|
expect(body).toContain('gstack-post pr-body <pr-number> --body-file "${PR_BODY_FILE:?restore the composed body path}"');
|
|
expect(body).toContain('gstack-redact --from-file "$PR_BODY_FILE" --auto-redact');
|
|
expect(read('ship/SKILL.md.tmpl')).toContain('existing PRs and docs-only changes');
|
|
expect(read('ship/sections/apple-release.md.tmpl')).toContain('read-only');
|
|
expect(read('ship/sections/apple-release.md.tmpl')).toContain('ship/sections/documentation.md');
|
|
});
|
|
|
|
test('ship-owned documentation_section carries its status so /ship embeds it unchanged', () => {
|
|
// ci-36641820398-1-gate-census-3 ship-docsync-completion: the section had scope, health and debt but no result,
|
|
// so the parent spliced a Status line into it and the report no longer contained the returned section.
|
|
const scope = read('document-release/sections/audit-scope.md.tmpl').replace(/\s+/g, ' ');
|
|
expectTokens(scope, ['`**Status:**`', '`status`'], 'scope');
|
|
expect(read('ship/sections/pr-body.md.tmpl')).toContain("Embed Step 14.5's vetted nonempty `documentation_section`");
|
|
});
|
|
|
|
test('a missing installed document-release section blocks before launch', () => {
|
|
// ci-36709485593-1-eval-slices-2 ship-docsync-missing-asset: audit-scope.md was absent, but the parent saw the
|
|
// SKILL.md "Ship-owned documentation mode" heading, read the gate as satisfied and dispatched.
|
|
const gate = read('ship/sections/documentation.md.tmpl').replace(/\s+/g, ' ');
|
|
expectTokens(gate, ['`Ship-owned documentation mode`'], 'gate');
|
|
expectMentions(gate, [['never', 'substitute', 'missing']], 'gate');
|
|
});
|
|
|
|
test('nested authored discovery and standalone protections survive', () => {
|
|
const skill = read('document-release/SKILL.md.tmpl') + read('document-release/sections/audit-scope.md.tmpl');
|
|
expect(skill).not.toContain('find . -maxdepth 2');
|
|
for (const text of ['declared documentation roots', '.tmpl', 'generated output', 'Read before editing',
|
|
'Ship-owned documentation mode', 'standalone branch gate']) expect(skill).toContain(text);
|
|
const body = read('document-release/sections/release-body.md.tmpl');
|
|
for (const text of ['never `git add -A`', 'Never regenerate a CHANGELOG entry',
|
|
'body-original.md', 'UNTRUSTED TRACKER CONTENT', 'gstack-redact --from-file']) expect(body).toContain(text);
|
|
const versionStep = body.slice(body.indexOf('## Step 8'), body.indexOf('\n## ', body.indexOf('## Step 8') + 1));
|
|
expect(versionStep).toMatch(/ask[^\n]*before[^\n]*VERSION/i);
|
|
expect(body).toMatch(/does not rewrite, replace, or regenerate CHANGELOG/i);
|
|
});
|
|
});
|
|
|
|
const current = {
|
|
schema_version: 1, audit_id: 'audit-1', status: 'current', files_updated: [],
|
|
files_reviewed: ['docs/reference/cli.md.tmpl'], documentation_section: 'Current — CLI reference reviewed.',
|
|
blockers: [], decisions: [],
|
|
};
|
|
const evidence = {
|
|
settled: true, markerSeen: true, headUnchanged: true, indexUnchanged: true,
|
|
candidateUnchanged: true, readOnly: false, changedPaths: [] as string[],
|
|
allowedDocs: ['docs/reference/cli.md.tmpl'],
|
|
};
|
|
|
|
describe('completion observer negative controls', () => {
|
|
test('updated, current, and partial blocked results retain their actual meaning', () => {
|
|
for (const value of [current,
|
|
{ ...current, status: 'updated', files_updated: evidence.allowedDocs },
|
|
{ ...current, status: 'blocked', files_updated: evidence.allowedDocs, blockers: ['Security narrative requires a decision'] }]) {
|
|
const parsed = parseDocsCompletion(`Audit complete\n${JSON.stringify(value)}\n`, 'audit-1');
|
|
vetDocsCompletion(parsed, { ...evidence, changedPaths: value.files_updated });
|
|
expect(parsed.status).toBe(value.status);
|
|
}
|
|
});
|
|
|
|
test.each([
|
|
['old shape', { files_updated: [], commit_sha: null, pushed: false, documentation_section: null }],
|
|
['old version', { ...current, schema_version: 0 }],
|
|
['late callback', { ...current, audit_id: 'abandoned' }],
|
|
['missing key', { ...current, decisions: undefined }],
|
|
['null summary', { ...current, documentation_section: null }],
|
|
['empty summary', { ...current, documentation_section: '' }],
|
|
['error field', { ...current, error: 'preamble failed' }],
|
|
['false current', { ...current, files_updated: evidence.allowedDocs }],
|
|
['empty updated', { ...current, status: 'updated' }],
|
|
['false blocked', { ...current, status: 'blocked' }],
|
|
['invalid array', { ...current, files_reviewed: [1] }],
|
|
['path escape', { ...current, files_reviewed: ['../README.md'] }],
|
|
['duplicate path', { ...current, files_reviewed: ['README.md', 'README.md'] }],
|
|
])('rejects %s', (_name, value) => {
|
|
expect(() => parseDocsCompletion(JSON.stringify(value), 'audit-1')).toThrow();
|
|
});
|
|
|
|
test.each(['{broken', 'agent_id: 123', `${JSON.stringify(current)}\nDone`, `${JSON.stringify(current)}\n\`\`\``])('rejects malformed or non-last-line completion %s', output => {
|
|
expect(() => parseDocsCompletion(output, 'audit-1')).toThrow();
|
|
});
|
|
|
|
test.each([
|
|
['running after timeout', { settled: false }], ['missing marker', { markerSeen: false }],
|
|
['unexpected commit', { headUnchanged: false }], ['staged edits', { indexUnchanged: false }],
|
|
['edit before callback', { candidateUnchanged: false }], ['edit after callback', { candidateUnchanged: false }],
|
|
['unreported partial edit', { changedPaths: evidence.allowedDocs }],
|
|
])('blocks %s', (_name, overrides) => {
|
|
expect(() => vetDocsCompletion(parseDocsCompletion(JSON.stringify(current), 'audit-1'), { ...evidence, ...overrides })).toThrow();
|
|
});
|
|
|
|
test.each(['VERSION', 'CHANGELOG.md', 'TODOS.md', 'package.json', 'nested/manifest.json', 'docs/generated.md', 'app.ts'])('rejects actual protected or unapproved change %s', changed => {
|
|
const result = parseDocsCompletion(JSON.stringify({ ...current, status: 'updated', files_updated: [changed] }), 'audit-1');
|
|
expect(() => vetDocsCompletion(result, { ...evidence, changedPaths: [changed] })).toThrow();
|
|
});
|
|
|
|
test('read-only store mode cannot accept an updated document', () => {
|
|
const result = parseDocsCompletion(JSON.stringify({ ...current, status: 'updated', files_updated: evidence.allowedDocs }), 'audit-1');
|
|
expect(() => vetDocsCompletion(result, { ...evidence, readOnly: true, changedPaths: evidence.allowedDocs })).toThrow();
|
|
});
|
|
});
|
|
|
|
describe('native docs fixture preflight', () => {
|
|
// PR #3434 screen 40269992077 slice 13 (docsync-spawned): the generated skill named
|
|
// sections/preamble.md as a required asset, the fixture had not installed it, and the
|
|
// spawned child blocked on the missing asset instead of auditing the docs.
|
|
test('installs a generated shared preamble beside the required sections, and nothing when the skill has none', () => {
|
|
const withPreamble = fs.mkdtempSync(path.join(os.tmpdir(), 'docsync-preamble-'));
|
|
try {
|
|
fs.cpSync(generated, withPreamble, { recursive: true });
|
|
fs.rmSync(path.join(withPreamble, 'ship/sections/preamble.md'), { force: true });
|
|
const preamble = path.join(withPreamble, 'document-release/sections/preamble.md');
|
|
fs.writeFileSync(preamble, '# Shared preamble\n\nFixture copy marker.\n');
|
|
const fixture = fixtureDocs('risky', withPreamble);
|
|
try {
|
|
expect(fs.readFileSync(path.join(fixture.skills, 'document-release/sections/preamble.md'), 'utf8')).toBe(fs.readFileSync(preamble, 'utf8'));
|
|
expect(fs.existsSync(path.join(fixture.skills, 'ship/sections/preamble.md'))).toBe(false);
|
|
expect(fs.readdirSync(path.join(fixture.skills, 'document-release/sections')).sort()).toEqual(['audit-scope.md', 'preamble.md', 'release-body.md']);
|
|
} finally { fixture.clean(); }
|
|
} finally { fs.rmSync(withPreamble, { recursive: true, force: true }); }
|
|
const withoutPreamble = fs.mkdtempSync(path.join(os.tmpdir(), 'docsync-no-preamble-'));
|
|
try {
|
|
fs.cpSync(generated, withoutPreamble, { recursive: true });
|
|
fs.rmSync(path.join(withoutPreamble, 'document-release/sections/preamble.md'), { force: true });
|
|
const plain = fixtureDocs('risky', withoutPreamble);
|
|
try {
|
|
expect(fs.readdirSync(path.join(plain.skills, 'document-release/sections')).sort()).toEqual(['audit-scope.md', 'release-body.md']);
|
|
} finally { plain.clean(); }
|
|
} finally { fs.rmSync(withoutPreamble, { recursive: true, force: true }); }
|
|
});
|
|
|
|
test('staged, unstaged and new content are distinct from HEAD and user content is excluded', () => {
|
|
const fixture = fixtureDocs('updated', generated);
|
|
try {
|
|
expect(gitAt(fixture.repo, 'diff', '--cached', '--name-only')).toBe('app.ts');
|
|
expect(gitAt(fixture.repo, 'diff', '--name-only')).toBe('README.md');
|
|
const candidate = JSON.parse(fs.readFileSync(fixture.candidate, 'utf8'));
|
|
expect(candidate.selected_paths).toContain('options.ts');
|
|
expect(candidate.selected_paths).toContain(DOC_PATH);
|
|
expect(candidate.selected_paths).not.toContain('personal-note.txt');
|
|
expect(gitAt(fixture.repo, 'show', 'HEAD:app.ts')).toContain('text');
|
|
expect(fs.readFileSync(path.join(fixture.repo, 'app.ts'), 'utf8')).toContain('json');
|
|
fs.writeFileSync(path.join(fixture.repo, 'VERSION'), '9.9.9.9\n');
|
|
expect(changedFiles(fixture.before, repoSnapshot(fixture.repo))).toEqual(['VERSION']);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('docs-only repeat fixture is actually committed and already pushed', () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
expect(gitAt(fixture.repo, 'rev-parse', 'HEAD')).toBe(gitAt(fixture.repo, 'rev-parse', '@{u}'));
|
|
expect(gitAt(fixture.repo, 'diff', 'main...HEAD', '--name-only')).toBe(DOC_PATH);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('store fixture stays on base and supplies a read-only candidate', () => {
|
|
const fixture = fixtureDocs('store', generated);
|
|
try {
|
|
expect(gitAt(fixture.repo, 'branch', '--show-current')).toBe('main');
|
|
expect(JSON.parse(fs.readFileSync(fixture.candidate, 'utf8')).mode).toBe('read-only');
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test.each(['before callback', 'after callback'])('a real late content edit invalidates evidence %s', order => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
const completion = () => parseDocsCompletion(JSON.stringify(current), 'audit-1');
|
|
let result = order === 'after callback' ? completion() : undefined;
|
|
fs.appendFileSync(path.join(fixture.repo, 'app.ts'), 'export const lateChange = true;\n');
|
|
result ??= completion();
|
|
const after = repoSnapshot(fixture.repo);
|
|
expect(() => vetDocsCompletion(result!, {
|
|
...evidence, candidateUnchanged: changedFiles(fixture.before, after).length === 0,
|
|
})).toThrow('stale candidate');
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('snapshotting rejects a documentation symlink outside the product root', () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
fs.writeFileSync(path.join(fixture.home, 'outside.md'), 'outside');
|
|
fs.symlinkSync(path.join(fixture.home, 'outside.md'), path.join(fixture.repo, 'escape.md'));
|
|
expect(() => repoSnapshot(fixture.repo)).toThrow('fixture path escaped');
|
|
expect(fs.readFileSync(path.join(fixture.home, 'outside.md'), 'utf8')).toBe('outside');
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('the real preamble marker and private fixture evidence survive cleanup', () => {
|
|
const fixture = fixtureDocs('risky', generated);
|
|
const evidenceRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'docs-evidence-'));
|
|
let retained: string | undefined;
|
|
try {
|
|
const result = spawnSync('bash', [path.join(fixture.skills, 'bin/gstack-skill-start'), '--skill', 'document-release'], {
|
|
cwd: fixture.repo, encoding: 'utf8', timeout: 30000,
|
|
env: { ...process.env, ...fixture.env, GSTACK_SESSION_KIND: 'spawned' },
|
|
});
|
|
expect(result.status).toBe(0);
|
|
expect(result.stdout).toContain('SESSION_KIND: spawned');
|
|
retained = preserveDocsEvidence(fixture, { output: 'preflight', toolCalls: [] } as unknown as SkillTestResult,
|
|
`docs-free-${path.basename(fixture.home)}`, 'fixture', {}, evidenceRoot);
|
|
fixture.clean();
|
|
expect(fs.existsSync(retained)).toBe(true);
|
|
expect(fs.statSync(retained).mode & 0o777).toBe(0o600);
|
|
expect(JSON.parse(fs.readFileSync(retained, 'utf8')).before.contents['SECURITY.md']).toBeDefined();
|
|
} finally {
|
|
fixture.clean();
|
|
fs.rmSync(evidenceRoot, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
(process.platform === 'linux' ? describe : describe.skip)('independent docs write observer', () => {
|
|
test('read-only native Git commands generate no forbidden writes', async () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const observer = await observeDocsWrites(fixture);
|
|
try {
|
|
gitAt(fixture.repo, 'status', '--porcelain');
|
|
gitAt(fixture.repo, 'diff', 'main...HEAD');
|
|
expect(docsWriteFailures(observer.stop(), [])).toEqual([]);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test.each(['write-revert', 'rename-revert', 'delete-recreate', 'git-config-revert', 'shell-write-revert'])('rejects transient %s despite restored bytes', async operation => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const file = path.join(fixture.repo, operation === 'git-config-revert' ? '.git/config' : 'app.ts');
|
|
const before = fs.readFileSync(file);
|
|
const observer = await observeDocsWrites(fixture);
|
|
try {
|
|
if (operation === 'rename-revert') {
|
|
fs.renameSync(file, file + '.moved');
|
|
fs.renameSync(file + '.moved', file);
|
|
} else if (operation === 'delete-recreate') {
|
|
fs.unlinkSync(file);
|
|
fs.writeFileSync(file, before);
|
|
} else if (operation === 'git-config-revert') {
|
|
gitAt(fixture.repo, 'config', 'docs.fixture', 'transient');
|
|
fs.writeFileSync(file, before);
|
|
} else if (operation === 'shell-write-revert') {
|
|
const child = spawnSync(process.execPath, ['-e', 'const fs=require("fs");const p=process.argv[1];const b=fs.readFileSync(p);fs.writeFileSync(p,"transient");fs.writeFileSync(p,b)', file], { timeout: 10000, encoding: 'utf8' });
|
|
expect(child.status).toBe(0);
|
|
} else {
|
|
fs.writeFileSync(file, 'transient');
|
|
fs.writeFileSync(file, before);
|
|
}
|
|
const observation = observer.stop();
|
|
expect(fs.readFileSync(file).equals(before)).toBe(true);
|
|
expect(observation.events.some(e => e.path === path.relative(fixture.repo, file))).toBe(true);
|
|
expect(docsWriteFailures(observation, [])).not.toEqual([]);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test.each(['overflow', 'truncated', 'unknown-watch'])('fails closed on %s observations', async fault => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const observer = await observeDocsWrites(fixture);
|
|
try {
|
|
const record = Buffer.alloc(fault === 'truncated' ? 1 : 16);
|
|
if (record.length === 16) {
|
|
record.writeInt32LE(-1, 0);
|
|
record.writeUInt32LE(fault === 'overflow' ? 0x4000 : 0x2, 4);
|
|
}
|
|
observer.injectKernelRecordsForTest(record);
|
|
const observation = observer.stop();
|
|
expect(observation.complete).toBe(false);
|
|
expect(docsWriteFailures(observation, [DOC_PATH])).toContain('incomplete docs write observation');
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('allows named authored edits while retaining their full mutation trace', async () => {
|
|
const fixture = fixtureDocs('updated', generated);
|
|
const observer = await observeDocsWrites(fixture);
|
|
try {
|
|
const file = path.join(fixture.repo, DOC_PATH);
|
|
fs.writeFileSync(file, fs.readFileSync(file, 'utf8').replace('text.', 'JSON.'));
|
|
const observation = observer.stop();
|
|
expect(docsWriteFailures(observation, [DOC_PATH])).toEqual([]);
|
|
expect(docsWriteFailures(observation, [])).not.toEqual([]);
|
|
expect(observation.changed).toContain(DOC_PATH);
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('kernel evidence remains private and readable after fixture cleanup', async () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const observer = await observeDocsWrites(fixture);
|
|
const evidenceRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'docs-evidence-'));
|
|
let evidence: string | undefined;
|
|
try {
|
|
const file = path.join(fixture.repo, 'app.ts');
|
|
const original = fs.readFileSync(file);
|
|
fs.writeFileSync(file, 'transient');
|
|
fs.writeFileSync(file, original);
|
|
const observation = observer.stop();
|
|
evidence = preserveDocsEvidence(fixture, { output: 'observer preflight', toolCalls: [] },
|
|
`docs-observer-${path.basename(fixture.home)}`, 'transient', { observation }, evidenceRoot);
|
|
fixture.clean();
|
|
expect(fs.statSync(evidence).mode & 0o777).toBe(0o600);
|
|
const retained = JSON.parse(fs.readFileSync(evidence, 'utf8')).observation;
|
|
expect(retained.complete).toBe(true);
|
|
expect(docsWriteFailures(retained, [])).toContain('forbidden docs write: app.ts');
|
|
} finally {
|
|
fixture.clean();
|
|
fs.rmSync(evidenceRoot, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
test('declared native interface rejects interpreters, shell composition and Git writes', () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
for (const command of ['bun -e "new Uint8Array(1)"', 'python3 -c "pass"', 'git status && node attack.js',
|
|
'git hash-object -w app.ts', 'git diff --output=app.ts', 'git show --textconv HEAD:app.ts', 'git config x.y z']) {
|
|
expect(docsCommandAllowed(command, fixture)).toBe(false);
|
|
}
|
|
for (const command of ['git status --porcelain', 'git diff --cached', 'git hash-object app.ts', ...docsPreambleCommands(fixture)]) {
|
|
expect(docsCommandAllowed(command, fixture)).toBe(true);
|
|
}
|
|
} finally { fixture.clean(); }
|
|
});
|
|
|
|
test('source-read evidence resolves installed paths but rejects a partial excerpt', () => {
|
|
const fixture = fixtureDocs('current', generated);
|
|
try {
|
|
const file = path.join(fixture.skills, 'ship/sections/documentation.md');
|
|
const output = fs.readFileSync(file, 'utf8').split('\n').map((line, i) => `${i + 1}→${line}`).join('\n');
|
|
for (const file_path of [file, '~/.claude/skills/gstack/ship/sections/documentation.md']) {
|
|
const result = { toolCalls: [{ tool: 'Read', input: { file_path }, output }] } as SkillTestResult;
|
|
expect(docsCompletedRead(result, file, fixture)).toBe(true);
|
|
result.toolCalls[0].output = '# Documentation audit gate';
|
|
expect(docsCompletedRead(result, file, fixture)).toBe(false);
|
|
}
|
|
} finally { fixture.clean(); }
|
|
});
|
|
});
|
|
|
|
describe('deterministic native-parent fault actor preflight', () => {
|
|
function setup(scenario: DocsFault) {
|
|
const fixture = fixtureDocs('current', generated);
|
|
const stateFile = installDocsActor(fixture, scenario);
|
|
const prompt = path.join(fixture.home, 'prompt.md');
|
|
const dispatch = (id: string) => {
|
|
fs.writeFileSync(prompt, `document-release files_updated audit ${id}`);
|
|
return docsActorCommand(stateFile, 'dispatch', { audit_id: id, candidate: fixture.candidate, prompt, run_in_background: 'false' });
|
|
};
|
|
const state = () => JSON.parse(fs.readFileSync(stateFile, 'utf8')) as DocsActorState;
|
|
return { fixture, stateFile, dispatch, state };
|
|
}
|
|
|
|
test.each(['missing-marker', 'launch-failure'] as const)('%s cannot publish', fault => {
|
|
const x = setup(fault);
|
|
try {
|
|
const response = x.dispatch('a');
|
|
expect(response.exit).toBe(fault === 'launch-failure' ? 23 : 0);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).toEqual([]);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: current', true)).not.toEqual([]);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('missing asset is actually absent and requires zero dispatches', () => {
|
|
const x = setup('missing-asset');
|
|
try {
|
|
expect(fs.existsSync(path.join(x.fixture.skills, 'document-release/sections/audit-scope.md'))).toBe(false);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).toEqual([]);
|
|
x.dispatch('a');
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).not.toEqual([]);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('unsettled cancellation is distinct from a stop acknowledgment', () => {
|
|
const x = setup('timeout-unsettled');
|
|
try {
|
|
const task = JSON.parse(x.dispatch('a').text).task_id;
|
|
expect(JSON.parse(docsActorCommand(x.stateFile, 'status', { task_id: task }).text).elapsed_ms).toBeGreaterThan(600000);
|
|
expect(JSON.parse(docsActorCommand(x.stateFile, 'stop', { task_id: task }).text).settled).toBe(false);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).not.toEqual([]);
|
|
docsActorCommand(x.stateFile, 'status', { task_id: task });
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).toEqual([]);
|
|
expect(x.dispatch('b').exit).not.toBe(0);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('late callback after settlement and transport repair remains tied to the old audit', () => {
|
|
const x = setup('late-result');
|
|
try {
|
|
const task = JSON.parse(x.dispatch('a').text).task_id;
|
|
docsActorCommand(x.stateFile, 'status', { task_id: task });
|
|
docsActorCommand(x.stateFile, 'stop', { task_id: task });
|
|
docsActorCommand(x.stateFile, 'repair');
|
|
const late = x.dispatch('b').text;
|
|
expect(() => parseDocsCompletion(late, 'b')).toThrow('completion identity');
|
|
expect(docsActorVerdict(x.state(), 'Documentation: blocked', false)).toEqual([]);
|
|
expect(docsActorVerdict(x.state(), 'Documentation: current', true)).not.toEqual([]);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test.each(['stale-before', 'stale-after', 'recovery'] as const)('%s requires a fresh successful audit before publication', fault => {
|
|
const x = setup(fault);
|
|
try {
|
|
x.dispatch('a');
|
|
if (fault === 'recovery') docsActorCommand(x.stateFile, 'repair');
|
|
else if (fault === 'stale-after') {
|
|
expect(fs.readFileSync(path.join(x.fixture.repo, 'app.ts'), 'utf8')).toContain('text');
|
|
docsActorHook(x.stateFile, JSON.stringify({ hook_event_name: 'PreToolUse', cwd: x.fixture.repo,
|
|
tool_name: 'Bash', tool_input: { command: 'git diff' } }));
|
|
}
|
|
if (fault.startsWith('stale-')) expect(fs.readFileSync(path.join(x.fixture.repo, 'app.ts'), 'utf8')).toContain('json');
|
|
const result = parseDocsCompletion(x.dispatch('b').text, 'b');
|
|
const report = path.join(x.fixture.home, 'report.md');
|
|
fs.writeFileSync(report, result.documentation_section);
|
|
expect(docsActorCommand(x.stateFile, 'publish', { audit_id: 'b', report }).exit).toBe(0);
|
|
expect(docsActorVerdict(x.state(), result.documentation_section, true)).toEqual([]);
|
|
const bad = x.state();
|
|
bad.events = bad.events.filter(e => e.action !== (fault === 'recovery' ? 'repair' : 'scheduled-input-edit'));
|
|
expect(docsActorVerdict(bad, result.documentation_section, true)).not.toEqual([]);
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('native command entrypoint returns the injected result and records real tool state', () => {
|
|
const x = setup('missing-marker');
|
|
try {
|
|
const prompt = path.join(x.fixture.home, 'native-prompt.md');
|
|
fs.writeFileSync(prompt, 'document-release files_updated audit native');
|
|
const result = spawnSync(process.execPath, [path.join(ROOT, 'test/helpers/docsync-fault-actor.ts'), 'dispatch', x.stateFile,
|
|
'audit_id=native', `candidate=${x.fixture.candidate}`, `prompt=${prompt}`, 'run_in_background=false'], {
|
|
cwd: x.fixture.repo, timeout: 10000, encoding: 'utf8',
|
|
});
|
|
expect(result.status).toBe(0);
|
|
expect(parseDocsCompletion(result.stdout, 'native').status).toBe('blocked');
|
|
expect(x.state().events[0].action).toBe('dispatch');
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
|
|
test('the registered native callback delivers the after-result edit, not a guessed delay', () => {
|
|
const x = setup('stale-after');
|
|
try {
|
|
x.dispatch('a');
|
|
const config = JSON.parse(fs.readFileSync(path.join(x.fixture.env.CLAUDE_CONFIG_DIR, 'settings.json'), 'utf8'));
|
|
const hook = config.hooks.PreToolUse.at(-1).hooks[0];
|
|
const event = { hook_event_name: 'PreToolUse', cwd: x.fixture.repo, tool_name: 'Bash',
|
|
tool_input: { command: 'git diff' }, session_id: 'fixture-session', tool_use_id: 'fixture-call' };
|
|
const invoke = (cwd: string) => spawnSync('bash', ['-c', hook.command], {
|
|
cwd: x.fixture.repo, input: JSON.stringify({ ...event, cwd }), encoding: 'utf8', timeout: 10000,
|
|
});
|
|
expect(invoke(x.fixture.home).status).toBe(0);
|
|
expect(x.state().lateChanged).toBe(false);
|
|
expect(invoke(x.fixture.repo).status).toBe(0);
|
|
expect(x.state().lateChanged).toBe(true);
|
|
expect(fs.readFileSync(path.join(x.fixture.repo, 'app.ts'), 'utf8')).toContain('json');
|
|
const actions = x.state().events.map(e => e.action);
|
|
expect(actions.indexOf('scheduled-input-edit')).toBeGreaterThan(actions.indexOf('completion'));
|
|
} finally { x.fixture.clean(); }
|
|
});
|
|
});
|