feat(evals): seedSkills opt-in for PTY slash-command tests + tripwire

Wire ClaudePtyOptions.seedSkills through launchClaudePty: when set (and
hermetic, and no per-test CLAUDE_CONFIG_DIR override), the child gets
hermeticSkillsConfigDir() so typed /skill slash commands resolve instead
of dying as Unknown command before any model turn. Opted in at the three
runPlanSkill* helpers and the four direct-launch slash-command tests
(plan-design-with-ui, plan-ceo-mode-routing, autoplan-chain,
ship-idempotency).

New static tripwire (test/pty-skill-seeding-wiring.test.ts): any test
file that sends a slash command over the PTY must route through a
runPlanSkill* helper or pass seedSkills: true — an unseeded slash-command
test spends money and measures nothing. hermetic-wiring.test.ts now
blesses the repo-tree seeding path explicitly (config dir under runRoot,
symlinks into the repo checkout, never operator ~/.claude).

The CI "Register gstack skills for PTY smoke" step keeps a keep-me note:
container cross-mount symlinks defeat the TUI scanner and HOME is not
hermeticized, so the real-file copies there must survive this change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 63c52269daaffb833b3105ea9b4b99be6df8fec7)
This commit is contained in:
Garry Tan
2026-08-12 15:31:47 -07:00
parent 5819203905
commit 281227262d
8 changed files with 122 additions and 1 deletions
+21
View File
@@ -16,6 +16,8 @@
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { getHermeticDirs, hermeticSkillsConfigDir } from './helpers/hermetic-env';
const ROOT = path.resolve(new URL(import.meta.url).pathname, '..', '..');
@@ -110,4 +112,23 @@ describe('hermetic wiring tripwire', () => {
offenders.join(', '),
).toEqual([]);
});
test('skill seeding stays under runRoot and reads the live repo tree, never operator ~/.claude', () => {
// hermeticSkillsConfigDir() is a BLESSED non-hermetic edge: it registers
// the LIVE repo tree's skills (the skills are the subject under test).
// What it must never do is hand children the operator's ~/.claude — the
// seeded CLAUDE_CONFIG_DIR lives under the hermetic runRoot, and every
// registered symlink resolves into the repo checkout.
const configDir = hermeticSkillsConfigDir();
const { runRoot } = getHermeticDirs();
const operatorClaude = path.join(os.homedir(), '.claude') + path.sep;
expect(configDir.startsWith(runRoot + path.sep)).toBe(true);
expect(configDir.startsWith(operatorClaude)).toBe(false);
const skillsDir = path.join(configDir, 'skills');
for (const entry of fs.readdirSync(skillsDir)) {
const target = fs.readlinkSync(path.join(skillsDir, entry, 'SKILL.md'));
expect(target.startsWith(operatorClaude), `${entry}: symlink escapes to ${target}`).toBe(false);
expect(fs.realpathSync(target).startsWith(fs.realpathSync(ROOT) + path.sep), `${entry}: symlink outside repo: ${target}`).toBe(true);
}
});
});