Files
gstack/test/hermetic-skill-runtime.test.ts
T
cadbd0af15 v1.91.116.0 feat: field list 2 wave B: gstack-ship-queue, approval classes, state-machine review, pregate timing check, update-check main moves (#3235)
* fix(capy): the install trailer says behind=<n> and its fix, so a stale install shows in every relayed trailer (field list 2 item 13, #2378)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(capy): a lease records its holder's start identity, so a pid reused after a wake is reclaimed; a skipped session-start upgrade ends fail behind, never ok (GBRA-105, GBRA-106)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(lease): stale needs proof (unreadable or malformed identities keep the lease), ps identity is timezone-independent, and reclaim never deletes a concurrent renewal

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(autoplan): enforced round cap and spend budget, deferred close, public export without machine paths, opt-in durable checkpoints (field list 2 W5-W7)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(lease): one directory lock for publish, prune and reclaim; reclaim rereads the record judged stale, so a renewal is never deleted or hidden

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(lease): the directory lock names its holder; broken only when the holder is dead or its pid reused, unlock removes only its own lock, touch exits 3 under a live holder

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(lease): breakers serialize on .lock.break and reread the lock before removing it; gstack-skill-start reports a lease the held lock kept out

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(lease): name the double-crash limit of breaking a dead lock

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(capy): --for review-voice install profile, no-Drive bootstrap command, reviewer prompt trailer (field list 2 W11)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 89bf556c11)

* test: pin review-voice row, helpers row and the no-Drive fallback in existing install tests

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 7f6c70459a)

* test: the DRIVER template names the reviewer trailer relay (pregate lane)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 2545d9a6d2)

* test(context-budget): capy skill eager ceiling for the review-voice subagent paragraph (W11; ratchet protocol, capture value)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(eval-plan): prereg diagnostics dry run (W8) and .gstack/eval-models.json lint (W9), WIP

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit b509096218)

* feat(eval-plan): W8/W9 tests, docs/eval-models.md, troubleshooting anchors, /eval-plan steps

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 9335e45de2)

* test(eval-plan): eval-plan/SKILL.md names only defined codes and accepted flags

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit a7a974d268)

* test(review-voice): the launch line points at the verdict's bun-only directory (#3233's launch contract)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(third-party-actions): the Aside drive-offer detector matches the product name, not the English idiom (paid run 38988927989 tpa-absent-linux false positive)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): export refuses artifact paths outside the run before writing; a deferred close keeps earlier undecided findings; reopening CEO grants Eng its rerun (review of #3234)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(restamp): gate-ahead keeps the lane CHANGELOG entry when both PRs took the same tentative stamp (GBRA-102 diagnosis fixture)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 43c75a6bb6)

* test(tree-receipt): a real gbrain stamp pair is same-modulo-stamps under gbrain committed policy (W1 acceptance)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 6cf53f9fed)

* feat(ship-policy): gate_command, full_e2e and min_gstack_version; unknown keys stay fail-closed; validate prints introducing versions

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 878a7123d7)

* feat(ship-queue): gstack-ship-queue ready, stamp, verify and gate-ahead (WIP, tests next)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 409a5957ad)

* test(ship-queue): stamp journal and resume, verify from a separate clone, gate-ahead attempts, ready

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit ade554087b)

* docs(ship-policy): Run a shared merge queue; new keys and compatibility rule; troubleshooting anchors; /ship pointer

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 7fc73e8892)

* fix(ship): shared-queue pointer rides the show-effective ship: line (ship skeleton has 2 bytes of parity headroom)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 5b33d4a015)

* test(windows): curate the shared-queue tests out of the Windows subset (shebang spawns)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 8deee14728)

* test(fixtures): register the gbrain stamp-pair fixture as free-only

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 563a07aa03)

* test(ship-queue): receipt digest contract names lib/ship-queue.ts for its pregate lane

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 501637c258)

* feat(pregate): timing check flags elapsed-time bounds and sleep-then-assert in added test lines (W4)

lib/pregate/checks/timing.ts prints TIMING_ASSERTION: <file>:<line> <pattern>
and a TIMING_ASSERTIONS summary; gstack-timing-ok: <reason> in any comment
syntax on the line or the line above escapes one. It warns
(PREGATE_TIMING_ASSERTION): measured on gstack (9 of 47 test-touching
commits, 263 hits in the tree) and gbrain (37 of 95 commits; 31 of 38 sites
of the five timing-flake fixes), blocking would stop a fifth to two fifths
of test changes. Adds the /review checklist rule and the /investigate
timing-flake playbook.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit a3b86b10b2)

* feat(update-check): an equal VERSION with a moved main is an upgrade, via the compare API (W12)

When VERSION matches but ls-remote's main SHA differs from this checkout's
HEAD, bin/gstack-update-check asks the GitHub compare API (status, ahead_by)
through the existing receipted curl and prints
UPGRADE_AVAILABLE <ver> <ver>+<n> <sha>; a diverged or unreadable compare is
CHECK_FAILED (unknown), and behind/identical is up to date. Never fetches.

The cache gains an IDENTITY line (head, compared remote SHA, upstream): an
outcome replays only for the same HEAD and upstream, a same-remote refresh
reuses it without another compare, and a git checkout ignores a legacy
one-line cache (non-git installs keep the one-line format). The snooze is
keyed by the SHA. gstack-upgrade's question, snooze note and What's New step
handle the four-field line; gstack-skill-end reports it.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 421e66d2a4)

* fix(pregate): treat planning and evidence markdown as an empty lane

A docs-only diff with no .gstack/pregate.json failed lanes as
PREGATE_NO_LANE. Planning and evidence Markdown is now a docs/no-tests
empty lane. Unmapped source files still fail closed.

Co-authored-by: Nick <surlymo@apache.org>
(cherry picked from commit e8793e59e8)

* fix(pregate): keep test-read markdown on its test lane

Unmarked Markdown that no test depends on is a docs/no-tests empty lane.
A file a test reads, including a fixture, still selects that test through
the import graph. Unmapped source and unreferenced test-tree files still
fail closed.

Co-authored-by: Nick <surlymo@apache.org>
(cherry picked from commit 7aabbddf3d)

* test(pregate): reviewer edges for the docs/no-tests empty lane (#3215)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 2fb3fa8613)

* test(pregate): requires-remote on a gbrain-shaped repo (ci:ubicloud, UBI_OWNER) (W10)

A base pregate.json with gbrain's dependencies and remote_gate
"bun run ci:ubicloud" (a package script whose stub needs UBICLOUD_API_KEY,
names VMs from UBI_OWNER and prints PREGATE_REMOTE). Under 8 cores the test
tier is one obligation naming the command; --remote passes the caller's
UBI_OWNER and key through and clears it on the report; without the key the
gate's own error shows and the obligation stays uncleared. #3190's path
needed no change.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 4d69ff02df)

* test(pregate): the ship receipt counts the timing check (7 checks)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit cb03ef67e7)

* test(context-budget): gstack-upgrade eager ceiling for the W12 equal-version upgrade question (ratchet protocol, capture value)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(approval-classes): gstack-approval-classes and ship-receipt citation enforcement (W2, WIP)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 847c7dae3d)

* feat(approval-classes): tests, docs, /review and /ship wiring (W2)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 269bd5b8b5)

* feat(review): state-machines specialist and gstack-state-matrix round-trip check with a captured #6792 regression (W3)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 1a634979c6)

* fix(review-army): keep the --simplification force flag pinned; approval test timeouts

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 789d392bce)

* test(free-fixtures): register the state-matrix-6792 fixture

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 1bacef296d)

* test(state-matrix): cover state-machines NEVER_GATE in gstack-specialist-stats

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
(cherry picked from commit 859063d56f)

* test(fixtures): register the pregate-timing fixtures as free-only (W4)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(ship-queue): full_e2e needs bound gate evidence in ready and verify, CI reads every check-run page and commit status, next: quotes its paths (review #3235 P1-1, P1-5, P2-1)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): timing reads a native diff and never passes an unread one

The timing check's git diff honored diff.external, textconv and
diff.noprefix, so a configured driver hid every added line and the check
passed. Run it with --no-ext-diff --no-textconv and explicit a/ b/
prefixes, and report a git diff that fails or does not finish as
incomplete (PREGATE_INCOMPLETE), which a warn downgrade does not soften.

Review of #3235, sibling of P1-4 (lib/pregate/checks/timing.ts:189).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(approval-classes): only affirmative APPROVED evidence counts, explicit tokens are authoritative, native diffs fail closed (review #3235 P1-2, P1-3, P1-4)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(state-matrix): detect reads the native diff, ignoring diff.external and textconv (review #3235 P1-4)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(e2e): review-army-state-matrix behavior case on the #6792 fixture (field list 2, W3)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(review-army): tighten the State Machines dispatch line so /ship stays under its parity size ratio with main's #3231 review-army text

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(approval-classes): only top-level approval lines count; quoted, fenced, indented-code and commented text never does (review #3235 r2)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(approval-classes): approval evidence is the first non-blank line only, at column 0 (review #3235 r3)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(autoplan-budget): bind the reservation expectation to lib/autoplan-budget's default estimate (test-of-test ratchet)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: keep gstack-upgrade under its size budget, exempt state-matrix's fetch hint, fit the periodic plan's 37 slices

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(evals): the docs observer accepts the spawned-prefixed gstack-skill-start --help read

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(windows): curate capy-review-voice out of the Windows subset (POSIX launcher contract)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: changelog entry for field list 2 wave A

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: stamp v1.91.115.0

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: the scope-signal clarity check follows helper-printed signals; the hermetic update-check fixture writes the identity cache

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* refactor: move the release-path result codes to lib/result-codes-release.ts (module-size ratchet); fit the periodic plan's 38 slices

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: changelog entry for field list 2 wave B

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: stamp v1.91.116.0

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: copy lib/result-codes-release.ts wherever a fixture copies lib/result-codes.ts; the scope-signal check requires the helper's computed emission

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: state-matrix stats test resolves its slug under its own state root (home containment)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(windows): curate pregate-docs-lane-review out of the Windows subset (runBin shebang, like the other pregate fixtures)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(windows): curate pregate-remote-gbrain out of the Windows subset (runBin shebang, like pregate-remote)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Nick <surlymo@apache.org>
2026-10-17 03:32:16 +00:00

469 lines
30 KiB
TypeScript

/** Free installed-runtime probe through the real PTY launcher; no provider calls. */
import { describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { execFileSync } from 'node:child_process';
import { getHermeticDirs, hermeticSkillsConfigDir } from './helpers/hermetic-env';
import { refreshHermeticSkillRuntime, questionCompanionReadSettings, hermeticSkillRuntime } from './helpers/hermetic-skill-runtime';
import { pathToFileURL } from 'node:url';
import { createHash } from 'node:crypto';
const ROOT = path.resolve(import.meta.dir, '..');
const digest = (file: string) => createHash('sha256').update(fs.readFileSync(file)).digest('hex');
describe('hermetic seeded PTY runtime', () => {
test.skipIf(process.platform === 'win32')('uses current lazy files and tools with scoped access, preserving auth, caches, and explicit overrides', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-runtime-probe-'));
const operatorHome = path.join(dir, 'operator-home');
const oldRoot = path.join(operatorHome, '.claude', 'skills', 'gstack');
const section = 'autoplan/sections/ceo-phase.md';
fs.mkdirSync(path.join(oldRoot, 'autoplan', 'sections'), { recursive:true });
fs.mkdirSync(path.join(oldRoot, 'bin'));
fs.writeFileSync(path.join(oldRoot, section), 'STALE OPERATOR PHASE\n');
fs.writeFileSync(path.join(oldRoot, 'bin/gstack-config'), '#!/bin/sh\necho stale-runtime\n', { mode:0o755 });
const state = path.join(dir, 'state');
fs.mkdirSync(state);
fs.writeFileSync(path.join(state, 'config.yaml'), 'codex_reviews: disabled\n');
const codexDefault = path.join(operatorHome, '.codex');
const codexConfigured = path.join(dir, 'configured-codex');
const codexOverride = path.join(dir, 'override-codex');
for (const home of [codexDefault,codexConfigured,codexOverride]) {
fs.mkdirSync(home);
fs.writeFileSync(path.join(home,'config.toml'), 'model = "unchanged-fixture-model"\n');
fs.writeFileSync(path.join(home,'auth.json'), '{"fixture_auth":true}\n');
}
const customConfig = path.join(dir,'custom-config');
fs.mkdirSync(customConfig);
const fake = path.join(dir,'fake-claude');
fs.writeFileSync(fake, `#!${process.execPath}\n` + String.raw`
import * as fs from 'node:fs';
import * as path from 'node:path';
import { createHash } from 'node:crypto';
import { spawnSync } from 'node:child_process';
const root = path.join(process.env.HOME,'.claude','skills','gstack');
const args = process.argv.slice(2);
const addDirs = args.flatMap((arg,index) => arg === '--add-dir' ? [args[index+1]] : []);
const addDir = addDirs[0] || null;
const registryDirectory = path.join(process.env.CLAUDE_CONFIG_DIR || path.join(process.env.HOME,'.claude'),'skills');
const registeredSection = path.join(registryDirectory,'autoplan','sections','design-phase.md');
const stateDirectory = path.join(process.env.HOME,'.gstack');
const methodologyPath = path.join(stateDirectory,'projects','probe','autoplan-design-methodology-owned','methodology.md');
const inside = (file,directory) => { const relative=path.relative(directory,file); return relative==='' || (!relative.startsWith('..'+path.sep) && relative!=='..' && !path.isAbsolute(relative)); };
const phase = spawnSync('bash',['-c','cat ~/.claude/skills/gstack/autoplan/sections/ceo-phase.md'],{timeout:5000});
const config = spawnSync('bash',['-c','"$HOME/.claude/skills/gstack/bin/gstack-config" get codex_reviews'],{timeout:5000});
const canonicalConfig = process.env.CLAUDE_CONFIG_DIR && spawnSync('bash',['-c','"$CLAUDE_CONFIG_DIR/skills/gstack/bin/gstack-config" get codex_reviews'],{timeout:5000});
const codexHome = process.env.CODEX_HOME || path.join(process.env.HOME,'.codex');
const record = {
pid:process.pid, home:process.env.HOME, root:fs.realpathSync(root), args, addDirs,
stateDirectory, stateDirectoryExists:fs.existsSync(stateDirectory), methodologyPath,
methodologyAllowed:addDirs.some(directory => inside(methodologyPath,directory)),
registryDirectory, registeredSection,
registeredSectionHash:fs.existsSync(registeredSection) ? createHash('sha256').update(fs.readFileSync(registeredSection)).digest('hex') : null,
registeredSectionAllowed:addDirs.some(directory => inside(registeredSection,directory)),
registryNeighborsAllowed:['../settings.json','../plans/unrelated.md','../skills-other/unrelated.md'].map(relative => addDirs.some(directory => inside(path.resolve(registryDirectory,relative),directory))),
phaseExit:phase.status, phaseHash:createHash('sha256').update(phase.stdout).digest('hex'),
configExit:config.status, configValue:config.stdout.toString().trim(),
canonicalConfigExit:canonicalConfig?.status, canonicalConfigValue:canonicalConfig?.stdout.toString().trim(),
configDir:process.env.CLAUDE_CONFIG_DIR, state:process.env.GSTACK_HOME,
runtimeAllowed:addDir !== null && fs.realpathSync(addDir) === fs.realpathSync(root),
claudeAuthPreserved:process.env.ANTHROPIC_API_KEY === 'fixture-api-key',
codexHome, codexConfig:fs.readFileSync(path.join(codexHome,'config.toml'),'utf8'),
codexAuthPreserved:JSON.parse(fs.readFileSync(path.join(codexHome,'auth.json'),'utf8')).fixture_auth === true,
browserCache:process.env.PLAYWRIGHT_BROWSERS_PATH,
discovery: ['plan-design-review/sections/review-sections.md', 'plan-devex-review/dx-hall-of-fame.md', 'plan-devex-review/sections/review-sections.md', 'review/checklist.md'].map(relative => {
const homePath = path.join(process.env.HOME, '.claude', 'skills', relative);
const configPath = path.join(process.env.CLAUDE_CONFIG_DIR || path.join(process.env.HOME, '.claude'), 'skills', relative);
const configRuntimePath = path.join(process.env.CLAUDE_CONFIG_DIR || path.join(process.env.HOME, '.claude'), 'skills', 'gstack', relative);
return {relative, home:fs.existsSync(homePath) ? fs.realpathSync(homePath) : null,
config:fs.existsSync(configPath) ? fs.realpathSync(configPath) : null,
configRuntime:fs.existsSync(configRuntimePath) ? fs.realpathSync(configRuntimePath) : null};
}),
};
fs.writeFileSync(process.env.RUNTIME_RECORD,JSON.stringify(record));
if (process.env.RUNTIME_FAIL === '1') throw new Error('fixture startup failure');
process.stdout.write('RUNTIME_PROBE_READY\n');
process.stdin.resume();
`);
fs.chmodSync(fake,0o755);
const runner = pathToFileURL(path.join(ROOT,'test/helpers/claude-pty-runner.ts')).href;
const cases = [
{name:'seeded-default',seed:true,owned:true},
{name:'seeded-startup-failure',seed:true,owned:true,failure:true},
{name:'seeded-configured-codex',seed:true,owned:true,configuredCodex:codexConfigured},
{name:'seeded-reset-codex',seed:true,owned:true,configuredCodex:codexConfigured,overrides:{CODEX_HOME:''}},
{name:'seeded-overrides',seed:true,owned:true,configuredCodex:codexConfigured,overrides:{CODEX_HOME:codexOverride,PLAYWRIGHT_BROWSERS_PATH:'0'}},
{name:'unseeded',seed:false,owned:false},
{name:'explicit-home',seed:true,owned:false,overrides:{HOME:operatorHome}},
{name:'explicit-config',seed:true,owned:false,overrides:{CLAUDE_CONFIG_DIR:customConfig}},
{name:'legacy',seed:true,owned:false,legacy:true},
];
const children: Array<ReturnType<typeof Bun.spawn>> = [];
const records: string[] = [];
const originalSection = digest(path.join(oldRoot,section));
try {
for (const item of cases) {
const record = path.join(dir,item.name+'.json'); records.push(record);
const worker = path.join(dir,item.name+'.ts');
fs.writeFileSync(worker, `
import { launchClaudePty } from ${JSON.stringify(runner)};
const session = await launchClaudePty({seedSkills:${item.seed},timeoutMs:10000,
model:'unchanged-claude-model',env:${JSON.stringify({GSTACK_HOME:state,RUNTIME_RECORD:record,RUNTIME_FAIL:item.failure?'1':'0',...item.overrides})}});
try {
let failed = false;
try { await session.waitFor('RUNTIME_PROBE_READY',{timeoutMs:7000,pollMs:20}); }
catch (error) { failed = true; if (!${Boolean(item.failure)}) throw error; }
if (failed !== ${Boolean(item.failure)}) throw new Error('unexpected startup outcome');
const recordPath = ${JSON.stringify(record)};
const record = JSON.parse(await Bun.file(recordPath).text());
record.ownedStateRoot = session.hermeticSkillStateRoot ?? null;
await Bun.write(recordPath, JSON.stringify(record));
} finally { await session.close(); }
`);
const child = Bun.spawn([process.execPath,worker], {
env:{...process.env,HOME:operatorHome,ANTHROPIC_API_KEY:'fixture-api-key',CODEX_HOME:item.configuredCodex,
PLAYWRIGHT_BROWSERS_PATH:undefined,EVALS_HERMETIC:item.legacy?'0':'1',BROWSE_TERMINAL_BINARY:fake},
stdout:'pipe',stderr:'pipe',
});
children.push(child);
const timer = setTimeout(() => child.kill('SIGKILL'),15000);
try {
const [code,stdout,stderr] = await Promise.all([child.exited,new Response(child.stdout).text(),new Response(child.stderr).text()]);
expect(code,item.name+'\n'+stdout+stderr).toBe(0);
} finally { clearTimeout(timer); }
const result = JSON.parse(fs.readFileSync(record,'utf8'));
expect(result.claudeAuthPreserved).toBe(true);
expect(result.codexAuthPreserved).toBe(true);
expect(result.codexConfig).toBe('model = "unchanged-fixture-model"\n');
expect(result.args.slice(0,4)).toEqual(['--model','unchanged-claude-model','--permission-mode','plan']);
expect(result.state).toBe(state);
expect(result.phaseExit).toBe(0); expect(result.configExit).toBe(0);
if (item.owned) {
expect(result.phaseHash).toBe(digest(path.join(ROOT,section)));
expect(result.home).not.toBe(operatorHome);
expect(result.root).toBe(fs.realpathSync(ROOT));
expect(result.configValue).toBe('disabled'); expect(result.runtimeAllowed).toBe(true);
// Actual Autoplan snapshots live under generated HOME/.gstack, outside
// the runtime checkout. Approve only that owned state tree, not HOME,
// an inherited/explicit GSTACK_HOME, or a broad policy-setting answer.
expect(result.methodologyAllowed, item.name + ': owned methodology Read is covered').toBe(true);
expect(result.registeredSectionAllowed, item.name + ': installed registry section Read is covered').toBe(true);
expect(result.registeredSectionHash).toBe(digest(path.join(ROOT,'autoplan/sections/design-phase.md')));
expect(result.registryNeighborsAllowed).toEqual([false,false,false]);
expect(result.stateDirectoryExists).toBe(true);
expect(result.ownedStateRoot).toBe(result.stateDirectory);
expect(result.addDirs).toEqual([path.join(result.home,'.claude','skills','gstack'),path.join(result.home,'.gstack'),path.join(result.configDir,'skills')]);
for (const external of [operatorHome,state,path.dirname(result.home),result.codexHome,result.browserCache])
expect(result.addDirs).not.toContain(external);
expect(result.args).not.toContain('--dangerously-skip-permissions');
expect(result.args).not.toContain('--settings');
expect(result.canonicalConfigExit).toBe(0); expect(result.canonicalConfigValue).toBe('disabled');
for (const asset of result.discovery) {
expect(asset.home, item.name + ': HOME discovery ' + asset.relative).toBe(fs.realpathSync(path.join(ROOT,asset.relative)));
expect(asset.config, item.name + ': config discovery ' + asset.relative).toBe(asset.home);
expect(asset.configRuntime, item.name + ': config runtime discovery ' + asset.relative).toBe(asset.home);
}
const expectedCodex = item.overrides?.CODEX_HOME !== undefined
? item.overrides.CODEX_HOME || codexDefault : item.configuredCodex || codexDefault;
expect(result.codexHome).toBe(expectedCodex);
const cacheBase = process.platform === 'darwin' ? path.join(operatorHome,'Library','Caches') : path.join(operatorHome,'.cache');
expect(result.browserCache).toBe(item.overrides?.PLAYWRIGHT_BROWSERS_PATH || path.join(cacheBase,'ms-playwright'));
expect(fs.existsSync(result.home)).toBe(false);
} else {
expect(result.home).toBe(operatorHome); expect(result.phaseHash).toBe(originalSection);
expect(result.configValue).toBe('stale-runtime'); expect(result.runtimeAllowed).toBe(false);
expect(result.addDirs).toEqual([]); expect(result.methodologyAllowed).toBe(false);
expect(result.registeredSectionAllowed).toBe(false);
expect(result.registryNeighborsAllowed).toEqual([false,false,false]);
expect(result.ownedStateRoot).toBeNull();
}
if (item.name === 'explicit-config') expect(result.configDir).toBe(customConfig);
expect(() => process.kill(result.pid,0)).toThrow();
}
expect(digest(path.join(oldRoot,section))).toBe(originalSection);
expect(fs.readFileSync(path.join(state,'config.yaml'),'utf8')).toBe('codex_reviews: disabled\n');
expect(fs.readdirSync(path.join(operatorHome,'.claude'))).toEqual(['skills']);
} finally {
for (const child of children) try { child.kill('SIGKILL'); } catch { /* already reaped */ }
for (const file of records) if (fs.existsSync(file)) {
try { process.kill(JSON.parse(fs.readFileSync(file,'utf8')).pid,'SIGKILL'); } catch { /* already reaped */ }
}
fs.rmSync(dir,{recursive:true,force:true});
}
},120000);
});
describe("explicit bound-runtime compatibility", () => {
const ROOT = path.resolve(import.meta.dir, '..');
const read = (file: string) => fs.readFileSync(file, 'utf8');
function write(file: string, content: string): void {
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, content);
}
function fixture(check: (source: string, privateDir: string, home: string) => void): void {
const base = fs.mkdtempSync(path.join(os.tmpdir(), 'hermetic-runtime-'));
try {
const source = path.join(base, 'source');
const home = path.join(base, 'home');
fs.mkdirSync(home);
write(path.join(source, 'SKILL.md'), '# Router\n');
write(path.join(source, 'alpha', 'SKILL.md'), '---\nname: alpha\n---\nCURRENT alpha\n');
write(path.join(source, 'alpha', 'sections', 'phase.md'), 'CURRENT phase\n');
write(path.join(source, 'VERSION'), 'CURRENT_VERSION\n');
write(path.join(source, '.git', 'HEAD'), 'ref: refs/heads/main\n');
check(source, path.join(base, 'private'), home);
} finally {
fs.rmSync(base, { recursive: true, force: true });
}
}
describe('hermetic skill runtime', () => {
test('all global forms and variable-based reads use current bins/assets; project fallbacks stay literal', () => {
fixture((source, privateDir, home) => {
const script = [
'ROOT=~/.claude/skills/gstack',
'"$HOME/.claude/skills/gstack/bin/version"',
'"${HOME}/.claude/skills/gstack/bin/version"',
'~/.claude/skills/gstack/bin/version',
'cat "$ROOT/alpha/sections/phase.md"',
].join('\n');
write(path.join(source, 'alpha', 'SKILL.md'), script + '\n');
write(path.join(source, 'alpha', 'sections', 'phase.md'), 'Read ~/.claude/skills/gstack/alpha/SKILL.md\n');
write(path.join(source, 'alpha', 'sections', 'fallbacks.md'), '.claude/skills/gstack\n$_ROOT/.claude/skills/gstack\n~/.claude/skills/gstack-upgrade\n');
write(path.join(source, 'bin', 'version'), '#!/usr/bin/env bash\nROOT=$(cd "$(dirname "$0")/.." && pwd)\ncat "$ROOT/VERSION"\n');
fs.chmodSync(path.join(source, 'bin', 'version'), 0o755);
const old = path.join(home, '.claude', 'skills', 'gstack', 'bin', 'version');
write(old, '#!/usr/bin/env bash\nprintf STALE_OPERATOR\n');
fs.chmodSync(old, 0o755);
const config = refreshHermeticSkillRuntime(source, privateDir);
const runtime = path.join(privateDir, 'runtime');
const bound = read(path.join(config, 'skills', 'alpha', 'SKILL.md'));
const output = execFileSync('bash', ['-c', bound], { env: { PATH: process.env.PATH!, HOME: home }, encoding: 'utf8', timeout: 5000 });
expect(output).toBe(`CURRENT_VERSION\nCURRENT_VERSION\nCURRENT_VERSION\nRead ${runtime}/alpha/SKILL.md\n`);
expect(read(path.join(runtime, 'alpha', 'sections', 'fallbacks.md')))
.toBe(read(path.join(source, 'alpha', 'sections', 'fallbacks.md')));
expect(read(old)).toContain('STALE_OPERATOR');
expect(fs.realpathSync(path.join(runtime, '.git'))).toBe(path.join(source, '.git'));
fs.rmSync(privateDir, { recursive: true, force: true });
expect(read(path.join(source, '.git', 'HEAD'))).toBe('ref: refs/heads/main\n');
expect(read(path.join(source, 'alpha', 'SKILL.md'))).toBe(script + '\n');
});
});
test('later refreshes preserve paths and config while updating, adding, and retiring live documents', () => {
fixture((source, privateDir) => {
const config = refreshHermeticSkillRuntime(source, privateDir);
const skill = path.join(config, 'skills', 'alpha', 'SKILL.md');
const runtime = path.join(privateDir, 'runtime');
const originalTarget = fs.realpathSync(skill);
const originalInode = fs.statSync(skill).ino;
write(path.join(config, '.claude.json'), '{"session":"keep"}');
write(path.join(config, 'plans', 'keep.md'), 'plan evidence');
expect(refreshHermeticSkillRuntime(source, privateDir)).toBe(config);
expect(fs.statSync(skill).ino).toBe(originalInode); // unchanged docs are not rewritten
write(path.join(source, 'alpha', 'SKILL.md'), '---\nname: alpha\n---\nUPDATED alpha\n');
write(path.join(source, 'alpha', 'sections', 'phase.md'), 'UPDATED phase\n');
write(path.join(source, 'alpha', 'sections', 'new.md'), 'NEW phase\n');
write(path.join(source, 'beta', 'SKILL.md'), '---\nname: beta\n---\nNEW beta\n');
write(path.join(source, 'VERSION'), 'LIVE_ASSET\n');
expect(read(path.join(runtime, 'VERSION'))).toBe('LIVE_ASSET\n');
expect(refreshHermeticSkillRuntime(source, privateDir)).toBe(config);
expect(fs.realpathSync(skill)).toBe(originalTarget);
expect(fs.statSync(skill).ino).not.toBe(originalInode); // changed docs are replaced atomically
expect(read(skill)).toContain('UPDATED alpha');
expect(fs.readdirSync(path.join(config, 'skills')).sort()).toEqual(['_gstack-command', 'alpha', 'beta']);
expect(fs.readdirSync(path.join(runtime, 'alpha', 'sections')).sort()).toEqual(['new.md', 'phase.md']);
expect(read(path.join(runtime, 'alpha', 'sections', 'phase.md'))).toBe('UPDATED phase\n');
expect(read(path.join(config, '.claude.json'))).toBe('{"session":"keep"}');
expect(read(path.join(config, 'plans', 'keep.md'))).toBe('plan evidence');
fs.rmSync(path.join(source, 'beta'), { recursive: true });
fs.unlinkSync(path.join(source, 'alpha', 'sections', 'phase.md'));
refreshHermeticSkillRuntime(source, privateDir);
expect(fs.existsSync(path.join(config, 'skills', 'beta'))).toBe(false);
expect(fs.existsSync(path.join(runtime, 'beta'))).toBe(false);
expect(fs.readdirSync(path.join(runtime, 'alpha', 'sections'))).toEqual(['new.md']);
fs.rmSync(path.join(source, 'alpha', 'sections'), { recursive: true });
refreshHermeticSkillRuntime(source, privateDir);
expect(fs.existsSync(path.join(config, 'skills', 'alpha', 'sections'))).toBe(false);
});
});
test('refresh never writes or cleans through substituted destination symlinks', () => {
fixture((source, privateDir, home) => {
const config = refreshHermeticSkillRuntime(source, privateDir);
const runtime = path.join(privateDir, 'runtime');
write(path.join(home, 'sentinel'), 'operator stays untouched');
const original = read(path.join(source, 'alpha', 'SKILL.md'));
for (const target of [path.join(runtime, 'alpha', 'sections'), path.join(config, 'skills', 'alpha')]) {
fs.rmSync(target, { recursive: true });
fs.symlinkSync(home, target, 'dir');
}
fs.unlinkSync(path.join(runtime, 'alpha', 'SKILL.md'));
fs.symlinkSync(path.join(source, 'alpha', 'SKILL.md'), path.join(runtime, 'alpha', 'SKILL.md'));
fs.symlinkSync(home, path.join(runtime, 'retired'), 'dir');
refreshHermeticSkillRuntime(source, privateDir);
expect(read(path.join(source, 'alpha', 'SKILL.md'))).toBe(original);
expect(fs.readdirSync(home)).toEqual(['sentinel']);
expect(fs.lstatSync(path.join(runtime, 'alpha', 'sections')).isDirectory()).toBe(true);
expect(fs.lstatSync(path.join(config, 'skills', 'alpha')).isDirectory()).toBe(true);
expect(fs.lstatSync(path.join(runtime, 'alpha', 'SKILL.md')).isSymbolicLink()).toBe(false);
expect(fs.existsSync(path.join(runtime, 'retired'))).toBe(false);
});
});
test('unsafe paths and a failed first refresh do not publish a partial tree', () => {
fixture((source, privateDir, home) => {
for (const suffix of [' space', '$dollar', ';command', '\nnewline']) {
expect(() => refreshHermeticSkillRuntime(source, privateDir + suffix)).toThrow('Set TMPDIR');
expect(fs.existsSync(privateDir + suffix)).toBe(false);
}
fs.symlinkSync(home, privateDir, 'dir');
expect(() => refreshHermeticSkillRuntime(source, privateDir)).toThrow('real directory');
expect(fs.readdirSync(home)).toEqual([]);
fs.unlinkSync(privateDir);
fs.symlinkSync(path.join(source, 'missing'), path.join(source, 'alpha', 'sections', 'missing.md'));
expect(() => refreshHermeticSkillRuntime(source, privateDir)).toThrow('ENOENT');
expect(fs.existsSync(privateDir)).toBe(false);
expect(read(path.join(source, 'VERSION'))).toBe('CURRENT_VERSION\n');
});
});
test('cached registration retains live source links without resetting Claude config', () => {
const config = hermeticSkillsConfigDir();
const skill = path.join(config, 'skills', 'autoplan', 'SKILL.md');
const source = path.join(ROOT, 'autoplan', 'SKILL.md');
const before = digest(source);
const seed = read(path.join(config, '.claude.json'));
// Main uses live links. Never write through realpath here: that would
// corrupt the source being read by other test processes.
expect(fs.lstatSync(skill).isSymbolicLink()).toBe(true);
expect(fs.realpathSync(skill)).toBe(fs.realpathSync(source));
expect(hermeticSkillsConfigDir()).toBe(config);
expect(read(skill)).toBe(read(source));
expect(digest(source)).toBe(before);
expect(read(path.join(config, '.claude.json'))).toBe(seed);
});
test('cached seeding repairs a substituted config directory without writing through it', () => {
fixture((_source, _privateDir, home) => {
const config = hermeticSkillsConfigDir();
write(path.join(home, '.claude.json'), '{"operator":"untouched"}');
fs.rmSync(config, { recursive: true });
fs.symlinkSync(home, config, 'dir');
expect(hermeticSkillsConfigDir()).toBe(config);
expect(fs.lstatSync(config).isDirectory()).toBe(true);
expect(JSON.parse(read(path.join(config, '.claude.json'))).hasCompletedOnboarding).toBe(true);
expect(read(path.join(home, '.claude.json'))).toBe('{"operator":"untouched"}');
expect(fs.readdirSync(home)).toEqual(['.claude.json']);
});
});
test('real runtime preamble and update helper resolve current bins, VERSION, and git metadata', () => {
fixture((_source, _privateDir, home) => {
const config = hermeticSkillsConfigDir();
const { root: runtime, home: runtimeHome } = hermeticSkillRuntime();
const state = path.join(home, '.gstack');
const commands = path.join(home, 'commands');
const old = path.join(home, '.claude', 'skills', 'gstack', 'bin', 'gstack-skill-start');
write(old, '#!/usr/bin/env bash\nprintf STALE_OPERATOR\n');
fs.chmodSync(old, 0o755);
write(path.join(commands, 'curl'), '#!/usr/bin/env bash\nprintf blocked > "$HOME/network-attempt"\nexit 1\n');
fs.chmodSync(path.join(commands, 'curl'), 0o755);
write(path.join(state, 'config.yaml'), 'update_check: false\nartifacts_sync_mode_prompted: true\n');
const env = { PATH: `${commands}${path.delimiter}${process.env.PATH!}`, HOME: runtimeHome, GSTACK_HOME: state };
const preamble = read(path.join(config, 'skills', 'autoplan', 'SKILL.md'))
.match(/## Preamble \(run first\)\n\n```bash\n([\s\S]*?)\n```/)![1];
const output = execFileSync('bash', ['-c', preamble], { cwd: home, env, encoding: 'utf8', timeout: 20_000 });
expect(output).toContain('SKILL_START_PROTO: 1');
expect(output).not.toContain('STALE_OPERATOR');
const version = read(path.join(ROOT, 'VERSION')).trim();
write(path.join(state, 'config.yaml'), 'update_check: true\n');
write(path.join(state, '.codex-desc-healed'), '');
const runtimeHead = execFileSync('git', ['rev-parse', 'HEAD'], { cwd: runtime, encoding: 'utf8', timeout: 5000 }).trim();
write(path.join(state, 'last-update-check'), `UP_TO_DATE ${version}\nIDENTITY head=${runtimeHead} remote=none upstream=https://github.com/garrytan/gstack.git\n`);
write(path.join(state, 'just-upgraded-from'), 'TEST_OLD\n');
expect(execFileSync(path.join(runtime, 'bin', 'gstack-update-check'), [], { cwd: home, env: { ...env, GSTACK_STATE_DIR: state }, encoding: 'utf8', timeout: 10_000 }))
.toBe(`JUST_UPGRADED TEST_OLD ${version}\n`);
expect(fs.existsSync(path.join(runtimeHome, 'network-attempt'))).toBe(false);
const gitDir = (cwd: string) => fs.realpathSync(path.resolve(cwd,
execFileSync('git', ['rev-parse', '--git-common-dir'], { cwd, encoding: 'utf8', timeout: 5000 }).trim()));
expect(gitDir(runtime)).toBe(gitDir(ROOT));
expect(read(old)).toContain('STALE_OPERATOR');
});
});
test('a first .claude.json write failure cleans the runtime and preserves the original error', () => {
fixture((_source, privateDir, home) => {
const script = `
import { spyOn } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { getHermeticDirs, hermeticSkillsConfigDir } from ${JSON.stringify(path.join(ROOT, 'test/helpers/hermetic-env.ts'))};
const dirs = getHermeticDirs();
const expected = new Error('seed-write-failed');
const original = fs.writeFileSync;
const spy = spyOn(fs, 'writeFileSync').mockImplementation((file, ...args) => {
if (String(file).endsWith('/with-skills/.claude/.claude.json')) throw expected;
return original(file, ...args);
});
let sameError = false;
try { hermeticSkillsConfigDir(); } catch (error) { sameError = error === expected; }
spy.mockRestore();
const removed = !fs.existsSync(path.join(dirs.runRoot, 'with-skills'));
const config = hermeticSkillsConfigDir();
console.log(JSON.stringify({ sameError, removed, retrySeeded: fs.existsSync(path.join(config, '.claude.json')) }));
`;
fs.mkdirSync(privateDir);
const result = execFileSync(process.execPath, ['-e', script], {
cwd: ROOT, env: { PATH: process.env.PATH!, HOME: home, TMPDIR: privateDir, EVALS_HERMETIC: '1' }, encoding: 'utf8', timeout: 20_000,
});
expect(JSON.parse(result)).toEqual({ sameError: true, removed: true, retrySeeded: true });
expect(fs.readdirSync(privateDir)).toEqual([]); // process-exit cleanup owns the completed tree
});
});
});
for (const suffix of ['', ' space é', '[*?](literal)']) test(`question companion Read rules keep literal path boundaries (${suffix})`, () => {
fixture((initial, privateDir) => {
const source = initial + suffix;
if (suffix) fs.renameSync(initial, source);
for (const name of ['askuserquestion-split.md', 'askuserquestion-cjk.md']) write(path.join(source, 'docs', name), name);
refreshHermeticSkillRuntime(source, privateDir);
if (suffix.includes('[')) {
expect(() => questionCompanionReadSettings(source, path.join(privateDir, 'runtime'))).toThrow('unsupported permission-pattern syntax');
return;
}
const settings = questionCompanionReadSettings(source, path.join(privateDir, 'runtime'));
expect(Object.keys(settings)).toEqual(['permissions']);
expect(Object.keys(settings.permissions)).toEqual(['allow']);
expect(settings.permissions.allow).toHaveLength(4);
for (const rule of settings.permissions.allow) {
expect(rule.startsWith('Read(//')).toBe(true);
expect(rule.endsWith('.md)')).toBe(true);
expect(rule).not.toContain('/**');
}
const sourceRules = settings.permissions.allow.filter(rule => rule.includes('/source'));
expect(sourceRules).toHaveLength(2);
expect(sourceRules).toEqual(['askuserquestion-split.md', 'askuserquestion-cjk.md'].map(name => `Read(/${source}/docs/${name})`));
expect(settings.permissions.allow.filter(rule => rule.includes('/private/runtime/')))
.toEqual(['askuserquestion-split.md', 'askuserquestion-cjk.md'].map(name => `Read(/${privateDir}/runtime/docs/${name})`));
});
});
test('question companion allowances refuse substituted or missing source documents', () => {
fixture((source, privateDir, home) => {
for (const name of ['askuserquestion-split.md', 'askuserquestion-cjk.md']) write(path.join(source, 'docs', name), name);
refreshHermeticSkillRuntime(source, privateDir);
const runtime = path.join(privateDir, 'runtime');
fs.unlinkSync(path.join(runtime, 'docs')); fs.symlinkSync(home, path.join(runtime, 'docs'), 'dir');
for (const name of ['askuserquestion-split.md', 'askuserquestion-cjk.md']) write(path.join(home, name), name);
expect(() => questionCompanionReadSettings(source, runtime)).toThrow('exact source document');
refreshHermeticSkillRuntime(source, privateDir);
fs.unlinkSync(path.join(source, 'docs', 'askuserquestion-cjk.md'));
expect(() => questionCompanionReadSettings(source, runtime)).toThrow('ENOENT');
});
});
});