mirror of
https://github.com/garrytan/gstack.git
synced 2026-08-29 09:20:39 +02:00
* test(wireup): make gbrain-missing PATH fixture hermetic The gbrain-missing test appended the host PATH (and a hardcoded /opt/homebrew/bin) to the fixture PATH, so on any machine with a real gbrain installed the 'missing' case saw it, exited 0 instead of 2, and could never fail where the bug exists — a false green for a whole machine class. The fixture now keeps only root-owned OS dirs on the child PATH, and a new determinism check plants a host-like gbrain to prove it is unreachable. Absorbed from PR #2615 with authorship preserved; the PR-thread liveness screenshot (docs/images/gstack-pr-liveness-2255.png) is dropped — referenced by nothing in the tree. Fixes #2255 Co-authored-by: CommandCodeBot <noreply@commandcode.ai> * fix(evidence): stop bun's dotenv autoload from reaching the spawned command `bin/gstack-evidence` has a `#!/usr/bin/env bun` shebang, and bun AUTO-LOADS `.env`, `.env.<NODE_ENV>` and `.env.local` from the cwd into `process.env`. The wrapper then spawned the command with no `env` override, so every command run through it inherited those variables — and a repo `.env.local` routinely holds production credentials. Two things go wrong, and the second is worse than the leak: 1. Secrets reach a child that would not otherwise have them. `npm test` run by hand in the same shell sees none of them; the same command through the wrapper sees all of them. 2. THE COMMAND UNDER TEST BEHAVES DIFFERENTLY, so the ledger certifies a run that is not the run CI performs. Observed in a Next.js repo on 2026-08-20: four tests failed 4/4 through the wrapper and passed 5/5 without it, because app code branched on env vars only the wrapper supplied. Nearly an hour went into chasing a "flake" that was the measuring instrument. The wrapper exists to record trustworthy evidence, so silently altering the environment defeats its purpose. The fix builds the child env from `process.env` minus the keys bun injected, and detection is exact rather than heuristic: verified on bun 1.3.11, a dotenv file does NOT override a variable the shell already exported (the shell's value wins). So a key whose live value equals the dotenv file's value was injected by bun, and dropping it restores the environment the user's own shell would have given the command. A key whose live value differs is genuinely the caller's and survives. `BUN_DOTENV_FILES()` mirrors bun's precedence, including that `.env.local` is skipped when NODE_ENV is "test" — scrubbing a key bun never loaded would strip a variable the caller legitimately provided. Escape hatch: GSTACK_EVIDENCE_KEEP_DOTENV=1 keeps the old behaviour. When keys are scrubbed the wrapper warns with the KEY NAMES ONLY, so the diagnostic cannot become the leak it prevents. Tests: 6 cases, mutation-verified — removing `env: spawnEnv` reddens exactly the two leak tests and restoring it gives 30/30. Every leak test asserts the scrub warning fired, because `bun test` sets NODE_ENV=test and the first version of these tests passed vacuously against a `.env.local` bun had never loaded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Absorbed from PR #2652 with authorship preserved. Wave additions: a doc-comment on the ${VAR}-expansion limitation (bun expands refs, the reader compares raw text — those keys are left in the child env, failing open) and a regression pin for the unreadable-.env fail-open path with a functional DAC-override skip guard. Fixes #2624 * fix(setup): reap dangling skill dirs when the payload is gone cleanup_old_claude_symlinks derived its work list from the payload directory, so when the payload was gone — precisely when orphans exist — the glob matched nothing and the loop never ran; the -f guard also followed symlinks, hiding dangling SKILL.md links even with a payload present. The cleanup now scans the DESTINATION skills dir (-e/-L, so dangling symlinks are visible) and anchors SKILL.md provenance to path segments (gstack/*, */gstack/*, */.gstack/render/claude/*) instead of a bare *gstack* substring that would eat a user skill under ~/tools/gstack-fork/. The Windows real-file arm stays payload-gated: a real file has no provable owner. Absorbed from PR #2634 (2 commits squashed) with authorship preserved. The symmetric cleanup_prefixed_claude_symlinks hole is filed as a TODOS.md residual in this wave. Fixes #2204 * fix(redact): tolerate EEXIST from recursive mkdir in install-prepush-hook on bun/Windows (#2635) fs.mkdirSync(dir, { recursive: true }) is a no-op on an existing directory in Node, but bun on Windows throws EEXIST - crashing hook install on any repo whose .git/hooks already existed, leaving the repo unprotected. Add lib/fs-utils.ts mkdirpSync: swallow EEXIST only when statSync confirms the path is an existing directory; a regular file occupying the path, a stat failure, or any other errno still rethrows. Use it in installPrepushHook(). The regression test emulates the Windows bun fs semantics via a bun --preload fixture, so the exact crash path runs (and fails on the old code) on any platform, including CI Linux. Absorbed from PR #2641 with authorship preserved. Fixes #2635 * fix(bin): route remaining Windows-reachable mkdirSync sites through mkdirpSync Sweep follow-up to #2641's lib/fs-utils.ts helper: bun on Windows throws EEXIST from a recursive mkdir on an existing dir, so every unguarded recursive mkdirSync on a Windows-reachable path is a latent crash. Converted: bin/gstack-decision-log (unguarded, runs on every decision log — the second call on any machine hits the pre-existing projects dir), bin/gstack-evidence logsDir + ledger dir sites, and bin/gstack-redact-prepush's skip-log site (already try-wrapped, so its failure mode was a silent skip-log loss rather than a crash — the fix makes the log survive). The ~15 remaining gbrain/mac-lane sites are deliberately left alone. Regression: fs-utils.test.ts drives gstack-decision-log twice, the second run under the bun-Windows EEXIST preload fixture — the pre-sweep code exits 1 with EEXIST there; verified red against v1.68.3.0. * fix(setup-gbrain): warn about the ZeroEntropy sunset before Sept 4 ZeroEntropy was acquired by Notion and sunsets its hosted API on September 4, 2026. A gbrain configured with the zeroentropyai embedding recipe keeps importing pages after that date but embedding silently fails — pages land structurally with no semantic search, this repo's tracker P1 (TODOS.md NEXT PRIORITY). Nothing in gstack ever recommended ZeroEntropy (the dependency is gbrain-internal), so the gstack side is detection + advisory: the wireup helper warns when ~/.gbrain/config.json names the recipe (fail-open grep — a missing, unreadable, or other-provider config stays silent and never blocks a working setup), the setup-gbrain provider-default comments say never to select the legacy recipe for a new brain, and USING_GBRAIN_WITH_GSTACK.md gains a troubleshooting entry. The gbrain-side provider migration stays open upstream. Refs #2365 * fix(gbrain-source-wireup): first sync targets the registered source, not --repo The wireup registered a federated source by id, then ran 'gbrain sync --repo $WORKTREE' — which resolves against the brain's DEFAULT source and (on gbrain 0.46.x) rewrites that source's local_path anchor to our worktree. Net effect: the user's primary knowledge source silently repointed at the gstack brain worktree while the just-registered source got zero pages, and pages_synced still reported success. The sync now targets the registered id ('gbrain sync --source $id', the same form the repo's own troubleshooting documents). Because the script's stated floor is gbrain >= 0.18.0 and nothing proves --source exists there, support is probed via 'gbrain sync --help' first: an older gbrain keeps the wrong-but-working --repo call with an upgrade warning instead of converting it into a hard failure. The probe sits after the GSTACK_BRAIN_NO_SYNC early-exit and is unreachable in --probe mode. Regression tests (fail on v1.68.3.0): a no-skip sync case asserting the call log shows 'sync --source gstack-brain-<id>' and never 'sync --repo', and an old-gbrain fallback case (fake sync --help without --source) asserting --repo plus the upgrade warning. Fixes #2662 * fix(setup): --host slate exits informatively instead of silently installing nothing slate passed --host validation (added to the accept-list in v1.64.1.0) but never got a dispatch arm, and the all-INSTALL_*-zero fallback lives inside the auto branch — so './setup --host slate' configured nothing and exited 0, a silent no-op strictly worse than the original hard rejection. slate is now an informational arm (per docs/designs/SLATE_HOST.md it is blocked on the host-config refactor; Slate reads .claude/skills as a compatibility fallback, so the arm points at './setup --host claude'), and a defensive guard after the dispatch chain errors loudly (naming the host, the missing arm, and the valid targets, exit 1) if a future host is ever accepted without being wired. Regression tests (fail on v1.68.3.0): a dispatch-arm ratchet asserting every accept-listed install target has a matching dispatch branch — the exact drift class; a registry cross-check deriving both sides from hosts/index.ts and setup's case arms; a behavioral slate probe (exit 0, points at --host claude, never reaches the installer — on unfixed code it fell through into the installer); and a static pin on the guard's shape. Fixes #2361 * fix(make-pdf): resolve the sibling browse binary from execPath, not argv[0] In a bun-compiled binary process.argv[0] is the raw invocation string — often relative ('./pdf', 'pdf') — so dirname(argv[0]) yielded '.' and the sibling candidates (../browse/dist/browse etc.) resolved against the CWD instead of the install dir. Resolution was cwd-dependent: correct-by-luck when the fallbacks rescued it, wrong when a cwd-relative path matched. process.execPath is always the absolute binary path. The resolution step takes an injectable selfPath (defaulted) because under bun test the process path is the bun runtime and the compiled-binary shapes are otherwise unreachable. The issue's other half — pdf setup failing on newtab('about:blank') — was already fixed on main in v1.64.0.0 (browse/src/url-validation.ts exact-match allows about:blank; its comment names this exact smoke). This commit closes what remains. Regression tests (the sibling-via-selfPath case fails on v1.68.3.0 — pre-fix code ignores the seam and either resolves the global install or throws): sibling resolution from an install-shaped tree, and a decoy-browse-DIRECTORY case pinning that a directory never wins resolution. Fixes #2156 * fix(memory-ingest): store the normalized git_remote so unattributed pages hit the policy filter buildTranscriptPage wrote the normalized '_unattributed' sentinel into the page FRONTMATTER but stored the raw resolved remote ('' when unresolvable) on the page object. The policy filter fast-paths !p.git_remote, so under --include-unattributed an explicit '_unattributed → deny' (or read-only) policy never applied to exactly the pages it names — they ingested unpoliced. The stored value now matches the frontmatter. Regression test (fails on v1.68.3.0): seeds the REAL bin/gstack-gbrain-repo-policy store with '_unattributed → deny' through its own set verb, ingests an unresolvable-remote session with --include-unattributed, and asserts nothing reaches gbrain — pre-fix the '' remote bypassed the filter and the import ran. A fake echoing tiers would pass on both sides of the fix; the real helper prints 'none' for unknown keys, so only a genuinely applied deny distinguishes the two. Fixes #2353 * fix(land-and-deploy): MERGED recovery reconciles and reports remote-branch cleanup Step 4's merge commands carry --delete-branch, and the success path tells the user 'The branch has been cleaned up.' When gh exits non-zero AFTER GitHub already merged (routine in worktree layouts: gh's local cleanup runs git checkout <base> and fails), the §4a-postfail MERGED recovery re-established everything EXCEPT the branch deletion — and said nothing about it, so the discrepancy was invisible. The MERGED path now reconciles: git ls-remote --heads distinguishes branch-already-gone (exit 0, empty → 'already cleaned up', idempotent on re-runs) from branch-survived (offer confirm-first deletion, matching the section's worktree posture; -d not -D for any local branch) from check-itself-failed (non-zero exit → 'couldn't verify', skip the offer — never read a failed check as a clean branch). Template + regenerated SKILL.md + test extensions land in one commit (the md-sync assertion goes red otherwise). Regression assertions (fail on v1.68.3.0: no delete-branch reconciliation existed in test/ at all) pin the ls-remote check, the confirm-first delete, and the absent-vs-failed distinction. Fixes #2656 * fix(scripts): stop heredoc bodies deadlocking under Homebrew bash `./setup --help` can hang forever on macOS, printing nothing, with no way to tell it apart from a slow install. Eleven scripts carry the same latent hang, `setup` itself being the one every user hits first. bash 5.2+ delivers a heredoc body of 64KiB or less through a pipe: the forked child writes the entire body before exec, and nothing reads the other end until the command starts. Under macOS pipe-KVA pressure the kernel hands a fresh pipe a 512-byte buffer instead of the usual 16-64KiB, so any body of 512 bytes or more blocks write() permanently. The capacity check bash would need to notice (F_GETPIPE_SZ) is Linux-only, so it never fires here. It is pressure-dependent, which is why it reads as "worked on my machine" — the same script runs fine all day and then wedges. Homebrew bash is what `#!/usr/bin/env bash` resolves to on a Mac with brew on PATH, which is most of them. Apple's /bin/bash 3.2 predates the pipe path and is unaffected, so the bug is invisible to anyone testing with the system shell. The fix is `BASH_COMPAT=50` in each affected script, which restores the pre-5.2 tempfile path: $ bash -c 'probe() { [ -p /dev/stdin ] && echo PIPE || echo TEMPFILE; } probe <<EOF $(printf "x%.0s" $(seq 1 1000)) EOF' PIPE $ BASH_COMPAT=50 bash -c '...same...' TEMPFILE - Not a `#!/bin/bash` shebang swap: that pins the script to whatever bash lives at /bin (3.2 on macOS, absent on some Linux distributions) and is bypassed entirely by `bash script.sh` call sites. The variable survives both. - Not exported, so child processes keep their own compat level. - Placed below any `--help` sed range that reads $0, so usage output is unchanged (verified on all eleven). - Every guarded script is bash-3.2-clean — no associative arrays, case conversion, or mapfile — so compat level 50 costs them nothing. test/heredoc-pipe-deadlock.test.ts scans every tracked shell script for a heredoc body in the 512B-64KiB window and fails without the guard, and proves the mechanism at runtime on bash 5.2+ by asserting the body moves from PIPE to TEMPFILE. On older bash the runtime half is skipped, since the pipe path does not exist there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Absorbed from PR #2640 with authorship preserved. Wave adaptations: the pipe-probe test skips on minimal-/dev environments without /dev/stdin (it would report OTHER for an unobservable fd), and one caveat verified during review: on bash 4.3/4.4 (e.g. Git Bash), assigning BASH_COMPAT=50 prints a non-fatal 'invalid value' warning to stderr — those bashes are already on tempfiles, so the guard is a no-op there; windows-setup-e2e exercises this empirically. * docs: TODOS.md v1.69 wave close-out Move the slate P4 entry and the ZeroEntropy P1's gstack-side half to Completed (v1.69.0.0); reframe the ZeroEntropy NEXT PRIORITY entry around the remaining gbrain-side work; file the wave's four residuals with rationale — the prefixed-cleanup symmetric conversion, the #2163 legacy-slug checkpoint heal, the invited #2657 --reconcile contribution, and the table-driven setup host dispatch behind the new cross-check ratchet. * chore: bump version and changelog (v1.69.0.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Som Samantray <som.samantray@gmail.com> Co-authored-by: CommandCodeBot <noreply@commandcode.ai> Co-authored-by: Connex Client Access <paul@paulkortman.com> Co-authored-by: y$un_ <forrest.sun527@gmail.com> Co-authored-by: Lockyer <135391289+Lockyer228@users.noreply.github.com> Co-authored-by: Benjamin D. Smith <benjamin.smith@binarysword.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
3752 lines
164 KiB
TypeScript
3752 lines
164 KiB
TypeScript
import { describe, test, expect, beforeAll } from 'bun:test';
|
|
import { assertSinglePreamble } from '../scripts/gen-skill-docs';
|
|
import { COMMAND_DESCRIPTIONS } from '../browse/src/commands';
|
|
import { SNAPSHOT_FLAGS } from '../browse/src/snapshot';
|
|
import * as fs from 'fs';
|
|
import * as path from 'path';
|
|
import * as os from 'os';
|
|
import { spawnSync } from 'child_process';
|
|
|
|
const ROOT = path.resolve(import.meta.dir, '..');
|
|
const MAX_SKILL_DESCRIPTION_LENGTH = 1024;
|
|
|
|
// Carved-skill aware (v2 plan T9): ship is now a skeleton SKILL.md + sections/*.md.
|
|
// Read the union so assertions about content that MOVED into a section still pass.
|
|
// The skeleton is a subset of the union, so skeleton-only assertions also hold,
|
|
// and negative assertions stay safe (the absent phrases live in neither file).
|
|
function readSkillUnion(skill: string): string {
|
|
let t = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const secDir = path.join(ROOT, skill, 'sections');
|
|
if (fs.existsSync(secDir)) {
|
|
for (const f of fs.readdirSync(secDir).sort()) {
|
|
if (f.endsWith('.md')) t += '\n' + fs.readFileSync(path.join(secDir, f), 'utf-8');
|
|
}
|
|
}
|
|
return t;
|
|
}
|
|
function readShipUnion(): string {
|
|
return readSkillUnion('ship');
|
|
}
|
|
|
|
function extractDescription(content: string): string {
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
expect(fmEnd).toBeGreaterThan(0);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
const lines = frontmatter.split('\n');
|
|
let description = '';
|
|
let inDescription = false;
|
|
const descLines: string[] = [];
|
|
|
|
for (const line of lines) {
|
|
if (line.match(/^description:\s*\|?\s*$/)) {
|
|
inDescription = true;
|
|
continue;
|
|
}
|
|
if (line.match(/^description:\s*\S/)) {
|
|
return line.replace(/^description:\s*/, '').trim();
|
|
}
|
|
if (inDescription) {
|
|
if (line === '' || line.match(/^\s/)) {
|
|
descLines.push(line.replace(/^ /, ''));
|
|
} else {
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
|
|
if (descLines.length > 0) {
|
|
description = descLines.join('\n').trim();
|
|
}
|
|
return description;
|
|
}
|
|
|
|
function extractMarkdownSection(content: string, heading: string): string {
|
|
const escaped = heading.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
const startMatch = content.match(new RegExp(`^${escaped}.*$`, 'm'));
|
|
expect(startMatch?.index).toBeDefined();
|
|
const start = startMatch!.index!;
|
|
const afterHeading = start + startMatch![0].length;
|
|
const nextSection = content.slice(afterHeading).match(/\n## /);
|
|
const end = nextSection?.index === undefined
|
|
? content.length
|
|
: afterHeading + nextSection.index;
|
|
return content.slice(start, end).trim();
|
|
}
|
|
|
|
function extractPreambleBeforeWorkflow(content: string, workflowMarkers: string[]): string {
|
|
const markerIndexes = workflowMarkers
|
|
.map(marker => content.indexOf(marker))
|
|
.filter(index => index >= 0);
|
|
expect(markerIndexes.length).toBeGreaterThan(0);
|
|
return content.slice(0, Math.min(...markerIndexes));
|
|
}
|
|
|
|
function isRepoRootSymlink(candidateDir: string): boolean {
|
|
try {
|
|
return fs.realpathSync(candidateDir) === fs.realpathSync(ROOT);
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
|
|
// Dynamic template discovery — matches the generator's findTemplates() behavior.
|
|
// New skills automatically get test coverage without updating a static list.
|
|
const ALL_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; name: string }> = [];
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
skills.push({ dir: '.', name: 'root gstack' });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) {
|
|
skills.push({ dir: entry.name, name: entry.name });
|
|
}
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
// hosts/claude.ts generation.skipSkills entries would filter here; the set is
|
|
// currently empty (the /claude outside-voice template was removed).
|
|
// The claude host deliberately skips some skills (skipSkills — e.g. the
|
|
// /claude outside-voice skill exists only for non-Claude hosts), so those
|
|
// dirs have a SKILL.md.tmpl but no generated claude-host SKILL.md on a fresh
|
|
// checkout. Every generated-file assertion must exclude them or it is red on
|
|
// every clean clone (it was, invisibly, until the free suite ran in CI).
|
|
import { getHostConfig as __getHostConfig } from '../hosts/index';
|
|
const CLAUDE_SKIPPED = new Set(__getHostConfig('claude').generation.skipSkills ?? []);
|
|
const CLAUDE_GENERATED_SKILLS = ALL_SKILLS.filter(s => !CLAUDE_SKIPPED.has(s.dir));
|
|
|
|
describe('gen-skill-docs', () => {
|
|
test('generated SKILL.md contains all command categories', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
const categories = new Set(Object.values(COMMAND_DESCRIPTIONS).map(d => d.category));
|
|
for (const cat of categories) {
|
|
expect(content).toContain(`### ${cat}`);
|
|
}
|
|
});
|
|
|
|
test('generated SKILL.md contains all commands', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
const display = meta.usage || cmd;
|
|
expect(content).toContain(display);
|
|
}
|
|
});
|
|
|
|
test('command table is sorted alphabetically within categories', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
// Extract command names from the Navigation section as a test
|
|
const navSection = content.match(/### Navigation\n\|.*\n\|.*\n([\s\S]*?)(?=\n###|\n## )/);
|
|
expect(navSection).not.toBeNull();
|
|
const rows = navSection![1].trim().split('\n');
|
|
const commands = rows.map(r => {
|
|
const match = r.match(/\| `(\w+)/);
|
|
return match ? match[1] : '';
|
|
}).filter(Boolean);
|
|
const sorted = [...commands].sort();
|
|
expect(commands).toEqual(sorted);
|
|
});
|
|
|
|
test('generated header is present in SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
});
|
|
|
|
test('generated header is present in browse/SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
});
|
|
|
|
test('snapshot flags section contains all flags', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
expect(content).toContain(flag.short);
|
|
expect(content).toContain(flag.description);
|
|
}
|
|
});
|
|
|
|
test('every skill has a SKILL.md.tmpl template', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
expect(fs.existsSync(tmplPath)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('every skill has a generated SKILL.md with auto-generated header', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const mdPath = path.join(ROOT, skill.dir, 'SKILL.md');
|
|
expect(fs.existsSync(mdPath)).toBe(true);
|
|
const content = fs.readFileSync(mdPath, 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
}
|
|
});
|
|
|
|
// #1778: strict YAML parsers (Codex/OpenAI skill loading) reject frontmatter
|
|
// whose plain `description:` scalar contains an interior ": " (read as a nested
|
|
// mapping). Parse EVERY generated frontmatter block with a strict YAML parser,
|
|
// not just string-check that name:/description: exist.
|
|
function frontmatterBlock(content: string): string {
|
|
expect(content.startsWith('---\n')).toBe(true);
|
|
const end = content.indexOf('\n---', 4);
|
|
expect(end).toBeGreaterThan(0);
|
|
return content.slice(4, end);
|
|
}
|
|
|
|
test('every generated SKILL.md frontmatter parses as strict YAML', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const fm = frontmatterBlock(content);
|
|
let parsed: any;
|
|
expect(() => { parsed = Bun.YAML.parse(fm); },
|
|
`frontmatter for ${skill.dir} must be valid YAML`).not.toThrow();
|
|
expect(typeof parsed?.name).toBe('string');
|
|
expect(typeof parsed?.description).toBe('string');
|
|
}
|
|
});
|
|
|
|
test('every generated Codex (.agents/skills) frontmatter parses as strict YAML', () => {
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return; // skip if external hosts not generated
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const mdPath = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(mdPath)) continue;
|
|
const fm = frontmatterBlock(fs.readFileSync(mdPath, 'utf-8'));
|
|
expect(() => Bun.YAML.parse(fm),
|
|
`Codex frontmatter for ${entry.name} must be valid YAML`).not.toThrow();
|
|
}
|
|
});
|
|
|
|
test(`every generated SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const description = extractDescription(content);
|
|
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
|
|
}
|
|
});
|
|
|
|
test(`every Codex SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return; // skip if not generated
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
const description = extractDescription(content);
|
|
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
|
|
}
|
|
});
|
|
|
|
test('every Codex SKILL.md description stays under 900-char warning threshold', () => {
|
|
const WARN_THRESHOLD = 900;
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return;
|
|
const violations: string[] = [];
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
const description = extractDescription(content);
|
|
if (description.length > WARN_THRESHOLD) {
|
|
violations.push(`${entry.name}: ${description.length} chars (limit ${MAX_SKILL_DESCRIPTION_LENGTH}, ${MAX_SKILL_DESCRIPTION_LENGTH - description.length} remaining)`);
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('package.json version matches VERSION file (npm-valid translation)', () => {
|
|
// Decision 11 (v1.67 wave): VERSION stays the 4-digit source of truth;
|
|
// package.json carries the npm-valid 3-digit translation (npm rejects a
|
|
// fourth component). The pre-v1.67 1:1 four-digit mirror is also accepted
|
|
// (grandfathered until the next write), matching gstack-version-bump's
|
|
// own drift contract.
|
|
const pkg = JSON.parse(fs.readFileSync(path.join(ROOT, 'package.json'), 'utf-8'));
|
|
const version = fs.readFileSync(path.join(ROOT, 'VERSION'), 'utf-8').trim();
|
|
const npmTranslation = version.split('.').slice(0, 3).join('.');
|
|
expect([npmTranslation, version]).toContain(pkg.version);
|
|
});
|
|
|
|
test('generated files are fresh (match --dry-run)', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// Every skill should be FRESH
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const file = skill.dir === '.' ? 'SKILL.md' : `${skill.dir}/SKILL.md`;
|
|
expect(output).toContain(`FRESH: ${file}`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('no generated SKILL.md contains unresolved placeholders', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const unresolved = content.match(/\{\{[A-Z_]+\}\}/g);
|
|
expect(unresolved).toBeNull();
|
|
}
|
|
});
|
|
|
|
test('templates contain placeholders', () => {
|
|
// P2 (v1.2.0): the root template is a pure router — only {{PREAMBLE}}.
|
|
// The browse command/snapshot placeholders live in browse/SKILL.md.tmpl now.
|
|
const rootTmpl = fs.readFileSync(path.join(ROOT, 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(rootTmpl).toContain('{{PREAMBLE}}');
|
|
expect(rootTmpl).not.toContain('{{COMMAND_REFERENCE}}');
|
|
expect(rootTmpl).not.toContain('{{SNAPSHOT_FLAGS}}');
|
|
|
|
const browseTmpl = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(browseTmpl).toContain('{{COMMAND_REFERENCE}}');
|
|
expect(browseTmpl).toContain('{{SNAPSHOT_FLAGS}}');
|
|
expect(browseTmpl).toContain('{{PREAMBLE}}');
|
|
});
|
|
|
|
test('generated SKILL.md contains operational self-improvement (replaced contributor mode)', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('Contributor Mode');
|
|
expect(content).not.toContain('gstack_contributor');
|
|
expect(content).not.toContain('contributor-logs');
|
|
expect(content).toContain('Operational Self-Improvement');
|
|
expect(content).toContain('gstack-learnings-log');
|
|
expect(content).toContain('gstack-learnings-search --limit 3');
|
|
});
|
|
|
|
test('generated SKILL.md with LEARNINGS_LOG contains operational type', () => {
|
|
// Check a skill that has LEARNINGS_LOG (e.g., review)
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('operational');
|
|
});
|
|
|
|
test('generated SKILL.md contains session awareness', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_SESSIONS');
|
|
expect(content).toContain('RECOMMENDATION');
|
|
});
|
|
|
|
test('generated SKILL.md contains branch detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_BRANCH');
|
|
expect(content).toContain('git branch --show-current');
|
|
});
|
|
|
|
// #2001: update_check: false silences the binary but the upgrade-handling
|
|
// instruction prose used to ship unconditionally. Every skill that carries
|
|
// the runtime config-echo cluster must (a) echo UPDATE_CHECK so the
|
|
// instruction layer can read it, and (b) gate the UPGRADE_AVAILABLE /
|
|
// JUST_UPGRADED prose on it — the same echo-then-gate convention every other
|
|
// flag (PROACTIVE, SKILL_PREFIX, EXPLAIN_LEVEL, QUESTION_TUNING) follows.
|
|
test('update_check opt-out gates preamble echo and upgrade-handling prose (issue #2001)', () => {
|
|
let checked = 0;
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
// Scope: only skills that render the runtime config-echo cluster.
|
|
if (!content.includes('echo "QUESTION_TUNING: $_QUESTION_TUNING"')) continue;
|
|
checked++;
|
|
expect(content, `${skill.dir} must echo UPDATE_CHECK`).toContain('echo "UPDATE_CHECK: $_UPDATE_CHECK"');
|
|
expect(content, `${skill.dir} must read update_check config`).toContain('_UPDATE_CHECK=$(');
|
|
// Whenever the upgrade-handling prose ships, it must gate on the flag.
|
|
if (content.includes('UPGRADE_AVAILABLE <old> <new>')) {
|
|
expect(content, `${skill.dir} upgrade prose must gate on UPDATE_CHECK`)
|
|
.toContain('If `UPDATE_CHECK` is `"false"`');
|
|
}
|
|
}
|
|
// Guard against the scope filter silently matching nothing.
|
|
expect(checked).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('tier 2+ skills contain ELI10 simplification rules (AskUserQuestion format)', () => {
|
|
// Root SKILL.md is tier 1 (no AskUserQuestion format). Check a tier 2+ skill instead.
|
|
// v1.7.0.0 Pros/Cons format uses "ELI10 (ALWAYS)" rather than "Simplify (ELI10".
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('ELI10');
|
|
expect(content).toContain('plain English');
|
|
expect(content).toContain('not function names');
|
|
});
|
|
|
|
test('tier 1 skills do NOT contain AskUserQuestion format', () => {
|
|
// Use benchmark (tier 1) instead of root — root SKILL.md gets overwritten by Codex test setup
|
|
const content = fs.readFileSync(path.join(ROOT, 'benchmark', 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('## AskUserQuestion Format');
|
|
expect(content).not.toContain('## Completeness Principle');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry line', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('skill-usage.jsonl');
|
|
expect(content).toContain('~/.gstack/analytics');
|
|
});
|
|
|
|
test('plan-review generated preambles stay under the Option A budget', () => {
|
|
const reviewSkills = [
|
|
{
|
|
path: path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
|
|
markers: ['# Mega Plan Review Mode', '## Step 0: Detect platform and base branch'],
|
|
},
|
|
{
|
|
path: path.join(ROOT, 'plan-eng-review', 'SKILL.md'),
|
|
markers: ['# Plan Review Mode'],
|
|
},
|
|
];
|
|
|
|
// Plan skills carry the same preamble surface as other tier-≥2 skills
|
|
// (Artifacts Sync, Context Recovery, Routing Injection are load-bearing
|
|
// functionality, not optional). Budget is set to current size + small
|
|
// headroom; ratchet down if a future slim trims real bytes.
|
|
// Ratcheted from 33000 → 35000 when the gbrain context-load block was
|
|
// added (per /sync-gbrain plan §4). Ratcheted 35000 → 36500 in v1.27.0.0
|
|
// when generate-brain-sync-block.ts gained the gbrain_mcp_mode probe +
|
|
// remote-mode ARTIFACTS_SYNC status line (Path 4 of /setup-gbrain).
|
|
// Ratcheted 36500 → 39000 in the contributor wave when #1205 added the
|
|
// \\u-escape CJK rule (rule 12 + self-check item) to the AskUserQuestion
|
|
// preamble.
|
|
// Ratcheted 39000 → 40000 in plan-tune cathedral T14: question-tuning
|
|
// resolver gained the <gstack-qid:...> marker convention + the
|
|
// (recommended) label requirement (D2 + D18 — both load-bearing for
|
|
// hook enforcement). Adds ~700 bytes.
|
|
// Ratcheted 40000 → 60000 in v1.52.0.0 cap audit: ~20K headroom so
|
|
// future preamble adds don't trip the gate on each PR. Real runaway
|
|
// (preamble doubling) still trips; normal scope growth doesn't.
|
|
for (const skill of reviewSkills) {
|
|
const content = fs.readFileSync(skill.path, 'utf-8');
|
|
const preamble = extractPreambleBeforeWorkflow(content, skill.markers);
|
|
expect(Buffer.byteLength(preamble, 'utf-8')).toBeLessThan(60_000);
|
|
}
|
|
});
|
|
|
|
test('voice and writing-style preamble sections stay compact', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
const voice = extractMarkdownSection(content, '## Voice');
|
|
const writingStyle = extractMarkdownSection(content, '## Writing Style');
|
|
|
|
expect(Buffer.byteLength(voice, 'utf-8')).toBeLessThan(3_000);
|
|
expect(Buffer.byteLength(writingStyle, 'utf-8')).toBeLessThan(2_000);
|
|
});
|
|
|
|
test('slim voice section preserves the gstack voice contract', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
const voice = extractMarkdownSection(content, '## Voice');
|
|
|
|
expect(voice).toMatch(/lead with the point|direct/i);
|
|
expect(voice).toMatch(/file|function|line|command|real numbers/i);
|
|
expect(voice).toMatch(/user.*outcome|user.*experience|real user/i);
|
|
expect(voice).toMatch(/corporate|academic|PR|hype/i);
|
|
expect(voice).toMatch(/AI vocabulary|delve|crucial|robust/i);
|
|
expect(voice).toMatch(/user decides|user.*context|sovereignty|recommendation, not a decision/i);
|
|
});
|
|
|
|
test('preamble .pending-* glob is zsh-safe (uses find, not shell glob)', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
if (!content.includes('.pending-')) continue;
|
|
// Must NOT have a bare shell glob ".pending-*" outside of find's -name argument
|
|
expect(content).not.toMatch(/for _PF in [^\n]*\/\.pending-\*/);
|
|
// Must use find to avoid zsh NOMATCH error on glob expansion
|
|
expect(content).toContain("find ~/.gstack/analytics -maxdepth 1 -name '.pending-*'");
|
|
}
|
|
});
|
|
|
|
test('bash blocks with shell globs are zsh-safe (setopt guard or find)', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const bashBlocks = [...content.matchAll(/```bash\n([\s\S]*?)```/g)].map(m => m[1]);
|
|
|
|
for (const block of bashBlocks) {
|
|
const lines = block.split('\n');
|
|
|
|
for (const line of lines) {
|
|
const trimmed = line.trimStart();
|
|
if (trimmed.startsWith('#')) continue;
|
|
if (!trimmed.includes('*')) continue;
|
|
// Skip lines where * is inside find -name, git pathspecs, or $(find)
|
|
if (/\bfind\b/.test(trimmed)) continue;
|
|
if (/\bgit\b/.test(trimmed)) continue;
|
|
if (/\$\(find\b/.test(trimmed)) continue;
|
|
|
|
// Check 1: "for VAR in <glob>" must use $(find ...) — caught above by the
|
|
// $(find check, so any surviving for-in with a glob pattern is a violation
|
|
if (/\bfor\s+\w+\s+in\b/.test(trimmed) && /\*\./.test(trimmed)) {
|
|
throw new Error(
|
|
`Unsafe for-in glob in ${skill.dir}/SKILL.md: "${trimmed}". ` +
|
|
`Use \`for f in $(find ... -name '*.ext')\` for zsh compatibility.`
|
|
);
|
|
}
|
|
|
|
// Check 2: ls/cat/rm/grep with glob file args must have setopt guard
|
|
const isGlobCmd = /\b(?:ls|cat|rm|grep)\b/.test(trimmed) &&
|
|
/(?:\/\*[a-z.*]|\*\.[a-z])/.test(trimmed);
|
|
if (isGlobCmd) {
|
|
expect(block).toContain('setopt +o nomatch');
|
|
}
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
test('preamble-using skills have correct skill name in telemetry', () => {
|
|
const PREAMBLE_SKILLS = [
|
|
{ dir: '.', name: 'gstack' },
|
|
{ dir: 'ship', name: 'ship' },
|
|
{ dir: 'review', name: 'review' },
|
|
{ dir: 'qa', name: 'qa' },
|
|
{ dir: 'retro', name: 'retro' },
|
|
];
|
|
for (const skill of PREAMBLE_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain(`"skill":"${skill.name}"`);
|
|
}
|
|
});
|
|
|
|
test('qa and qa-only templates use QA_METHODOLOGY placeholder', () => {
|
|
const qaTmpl = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(qaTmpl).toContain('{{QA_METHODOLOGY}}');
|
|
|
|
const qaOnlyTmpl = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(qaOnlyTmpl).toContain('{{QA_METHODOLOGY}}');
|
|
});
|
|
|
|
test('QA_METHODOLOGY appears expanded in both qa and qa-only generated files', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
|
|
|
|
// Both should contain the health score rubric
|
|
expect(qaContent).toContain('Health Score Rubric');
|
|
expect(qaOnlyContent).toContain('Health Score Rubric');
|
|
|
|
// Both should contain framework guidance
|
|
expect(qaContent).toContain('Framework-Specific Guidance');
|
|
expect(qaOnlyContent).toContain('Framework-Specific Guidance');
|
|
|
|
// Both should contain the important rules
|
|
expect(qaContent).toContain('Important Rules');
|
|
expect(qaOnlyContent).toContain('Important Rules');
|
|
|
|
// Both should contain the 6 phases
|
|
expect(qaContent).toContain('Phase 1');
|
|
expect(qaOnlyContent).toContain('Phase 1');
|
|
expect(qaContent).toContain('Phase 6');
|
|
expect(qaOnlyContent).toContain('Phase 6');
|
|
});
|
|
|
|
test('qa-only has no-fix guardrails', () => {
|
|
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
|
|
expect(qaOnlyContent).toContain('Never fix bugs');
|
|
expect(qaOnlyContent).toContain('NEVER fix anything');
|
|
// Should not have Edit, Glob, or Grep in allowed-tools.
|
|
// Scope to frontmatter (between the first two --- lines) — the body can
|
|
// legitimately mention these tool names in prose (e.g., Claude model
|
|
// overlay says "prefer Read, Edit, Write, Glob, Grep over Bash").
|
|
const fmMatch = qaOnlyContent.match(/^---\n([\s\S]*?)\n---/);
|
|
expect(fmMatch).not.toBeNull();
|
|
const frontmatter = fmMatch![1];
|
|
expect(frontmatter).toMatch(/allowed-tools:/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Edit/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Glob/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Grep/);
|
|
});
|
|
|
|
test('qa has fix-loop tools and phases', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
// Should have Edit, Glob, Grep in allowed-tools
|
|
expect(qaContent).toContain('Edit');
|
|
expect(qaContent).toContain('Glob');
|
|
expect(qaContent).toContain('Grep');
|
|
// Should have fix-loop phases
|
|
expect(qaContent).toContain('Phase 7');
|
|
expect(qaContent).toContain('Phase 8');
|
|
expect(qaContent).toContain('Fix Loop');
|
|
expect(qaContent).toContain('Triage');
|
|
expect(qaContent).toContain('WTF');
|
|
});
|
|
});
|
|
|
|
describe('BASE_BRANCH_DETECT resolver', () => {
|
|
// Find a generated SKILL.md that uses the placeholder (ship is guaranteed to)
|
|
const shipContent = readShipUnion();
|
|
|
|
test('resolver output contains PR base detection command', () => {
|
|
expect(shipContent).toContain('gh pr view --json baseRefName');
|
|
});
|
|
|
|
test('resolver output contains repo default branch detection command', () => {
|
|
expect(shipContent).toContain('gh repo view --json defaultBranchRef');
|
|
});
|
|
|
|
test('resolver output contains fallback to main', () => {
|
|
expect(shipContent).toMatch(/fall\s*back\s+to\s+`main`/i);
|
|
});
|
|
|
|
test('resolver output uses "the base branch" phrasing', () => {
|
|
expect(shipContent).toContain('the base branch');
|
|
});
|
|
|
|
test('resolver output contains GitLab CLI commands', () => {
|
|
expect(shipContent).toContain('glab');
|
|
});
|
|
|
|
test('resolver output contains git-native fallback', () => {
|
|
expect(shipContent).toContain('git symbolic-ref');
|
|
});
|
|
|
|
test('resolver output mentions GitLab platform', () => {
|
|
expect(shipContent).toMatch(/gitlab/i);
|
|
});
|
|
});
|
|
|
|
describe('GitLab support in generated skills', () => {
|
|
const retroContent = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
|
const shipSkillContent = readShipUnion();
|
|
|
|
test('retro contains GitLab MR number extraction', () => {
|
|
expect(retroContent).toContain('[#!]');
|
|
});
|
|
|
|
test('retro uses BASE_BRANCH_DETECT (contains glab)', () => {
|
|
expect(retroContent).toContain('glab');
|
|
});
|
|
|
|
test('ship contains glab mr create', () => {
|
|
expect(shipSkillContent).toContain('glab mr create');
|
|
});
|
|
|
|
test('ship checks .gitlab-ci.yml', () => {
|
|
expect(shipSkillContent).toContain('.gitlab-ci.yml');
|
|
});
|
|
});
|
|
|
|
/**
|
|
* Quality evals — catch description regressions.
|
|
*
|
|
* These test that generated output is *useful for an AI agent*,
|
|
* not just structurally valid. Each test targets a specific
|
|
* regression we actually shipped and caught in review.
|
|
*/
|
|
describe('description quality evals', () => {
|
|
// Regression: snapshot flags lost value hints (-d <N>, -s <sel>, -o <path>)
|
|
test('snapshot flags with values include value hints in output', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
if (flag.takesValue) {
|
|
expect(flag.valueHint).toBeDefined();
|
|
expect(content).toContain(`${flag.short} ${flag.valueHint}`);
|
|
}
|
|
}
|
|
});
|
|
|
|
// Regression: "is" lost the valid states enum
|
|
test('is command lists valid state values', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['is'].description;
|
|
for (const state of ['visible', 'hidden', 'enabled', 'disabled', 'checked', 'editable', 'focused']) {
|
|
expect(desc).toContain(state);
|
|
}
|
|
});
|
|
|
|
// Regression: "press" lost common key examples
|
|
test('press command lists example keys', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['press'].description;
|
|
expect(desc).toContain('Enter');
|
|
expect(desc).toContain('Tab');
|
|
expect(desc).toContain('Escape');
|
|
});
|
|
|
|
// Regression: "console" lost --errors filter note
|
|
test('console command describes --errors behavior', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['console'].description;
|
|
expect(desc).toContain('--errors');
|
|
});
|
|
|
|
// Regression: snapshot -i lost "@e refs" context
|
|
test('snapshot -i mentions @e refs', () => {
|
|
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-i')!;
|
|
expect(flag.description).toContain('@e');
|
|
});
|
|
|
|
// Regression: snapshot -C lost "@c refs" context
|
|
test('snapshot -C mentions @c refs', () => {
|
|
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-C')!;
|
|
expect(flag.description).toContain('@c');
|
|
});
|
|
|
|
// Guard: every description must be at least 8 chars (catches empty or stub descriptions)
|
|
test('all command descriptions have meaningful length', () => {
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
expect(meta.description.length).toBeGreaterThanOrEqual(8);
|
|
}
|
|
});
|
|
|
|
// Guard: snapshot flag descriptions must be at least 10 chars
|
|
test('all snapshot flag descriptions have meaningful length', () => {
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
expect(flag.description.length).toBeGreaterThanOrEqual(10);
|
|
}
|
|
});
|
|
|
|
// Guard: descriptions must not contain pipe (breaks markdown table cells)
|
|
// Usage strings are backtick-wrapped in the table so pipes there are safe.
|
|
test('no command description contains pipe character', () => {
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
expect(meta.description).not.toContain('|');
|
|
}
|
|
});
|
|
|
|
// Guard: generated output uses → not ->
|
|
test('generated SKILL.md uses unicode arrows', () => {
|
|
// P2 (v1.2.0): the browse body moved out of the top-level router into
|
|
// browse/SKILL.md. Guard arrow style on the browse body (sliced from its
|
|
// H1 so the auto-generated `-->` header comments are excluded).
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
const body = content.slice(content.indexOf('# browse: QA Testing'));
|
|
expect(body).toContain('→');
|
|
expect(body).not.toContain('->');
|
|
});
|
|
});
|
|
|
|
describe('REVIEW_DASHBOARD resolver', () => {
|
|
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review'];
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`review dashboard appears in ${skill} generated file`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('gstack-review');
|
|
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
|
});
|
|
}
|
|
|
|
test('review dashboard appears in ship generated file', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('reviews.jsonl');
|
|
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
|
});
|
|
|
|
test('dashboard treats review as a valid Eng Review source', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
|
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
|
expect(content).toContain('`plan-eng-review` (plan-stage architecture review)');
|
|
expect(content).toContain('from either \\`review\\` or \\`plan-eng-review\\`');
|
|
});
|
|
|
|
test('shared dashboard propagates review source to plan-eng-review', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
|
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
|
});
|
|
|
|
test('resolver output contains key dashboard elements', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('VERDICT');
|
|
expect(content).toContain('CLEARED');
|
|
expect(content).toContain('Eng Review');
|
|
expect(content).toContain('7 days');
|
|
expect(content).toContain('Design Review');
|
|
expect(content).toContain('skip_eng_review');
|
|
});
|
|
|
|
test('dashboard bash block includes git HEAD for staleness detection', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('git rev-parse --short HEAD');
|
|
expect(content).toContain('---HEAD---');
|
|
});
|
|
|
|
test('dashboard includes staleness detection prose', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('Staleness detection');
|
|
expect(content).toContain('commit');
|
|
});
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`${skill} contains review chaining section`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('Review Chaining');
|
|
});
|
|
|
|
test(`${skill} Review Log includes commit field`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('"commit"');
|
|
});
|
|
}
|
|
|
|
test('plan-ceo-review chaining mentions eng and design reviews', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-design-review');
|
|
});
|
|
|
|
test('plan-eng-review chaining mentions design and ceo reviews', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('/plan-design-review');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
});
|
|
|
|
test('plan-design-review chaining mentions eng, ceo, and design skills', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
expect(content).toContain('/design-shotgun');
|
|
expect(content).toContain('/design-html');
|
|
});
|
|
|
|
test('ship does NOT contain review chaining', () => {
|
|
const content = readShipUnion();
|
|
expect(content).not.toContain('Review Chaining');
|
|
});
|
|
});
|
|
|
|
// ─── Test Coverage Audit Resolver Tests ─────────────────────
|
|
|
|
describe('TEST_COVERAGE_AUDIT placeholders', () => {
|
|
const planSkill = readSkillUnion('plan-eng-review'); // carved
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan and ship modes share codepath tracing methodology', () => {
|
|
// Review mode delegates test coverage to the Testing specialist subagent (Review Army)
|
|
const sharedPhrases = [
|
|
'Trace data flow',
|
|
'Diagram the execution',
|
|
'Quality scoring rubric',
|
|
'★★★',
|
|
'★★',
|
|
'GAP',
|
|
];
|
|
for (const phrase of sharedPhrases) {
|
|
expect(planSkill).toContain(phrase);
|
|
expect(shipSkill).toContain(phrase);
|
|
}
|
|
// Plan mode traces the plan, not a git diff
|
|
expect(planSkill).toContain('Trace every codepath in the plan');
|
|
expect(planSkill).not.toContain('git diff origin');
|
|
// Ship mode traces the diff
|
|
expect(shipSkill).toContain('Trace every codepath changed');
|
|
});
|
|
|
|
test('review mode uses Review Army for specialist dispatch', () => {
|
|
expect(reviewSkill).toContain('Review Army');
|
|
expect(reviewSkill).toContain('Specialist Dispatch');
|
|
expect(reviewSkill).toContain('testing.md');
|
|
});
|
|
|
|
test('plan and ship modes include E2E decision matrix', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('E2E Test Decision Matrix');
|
|
expect(skill).toContain('→E2E');
|
|
expect(skill).toContain('→EVAL');
|
|
}
|
|
});
|
|
|
|
test('plan and ship modes include regression rule', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('REGRESSION RULE');
|
|
expect(skill).toContain('IRON RULE');
|
|
}
|
|
});
|
|
|
|
test('plan and ship modes include test framework detection', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('Test Framework Detection');
|
|
expect(skill).toContain('CLAUDE.md');
|
|
}
|
|
});
|
|
|
|
test('plan mode adds tests to plan + includes test plan artifact', () => {
|
|
expect(planSkill).toContain('Add missing tests to the plan');
|
|
expect(planSkill).toContain('eng-review-test-plan');
|
|
expect(planSkill).toContain('Test Plan Artifact');
|
|
});
|
|
|
|
test('ship mode auto-generates tests + includes before/after count', () => {
|
|
expect(shipSkill).toContain('Generate tests for uncovered paths');
|
|
expect(shipSkill).toContain('Before/after test count');
|
|
expect(shipSkill).toContain('30 code paths max');
|
|
expect(shipSkill).toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review mode uses Fix-First + Review Army for specialist coverage', () => {
|
|
expect(reviewSkill).toContain('Fix-First');
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
// Review Army handles test coverage via Testing specialist subagent
|
|
expect(reviewSkill).toContain('Review Army');
|
|
expect(reviewSkill).toContain('Testing');
|
|
});
|
|
|
|
test('plan mode does NOT include ship-specific content', () => {
|
|
expect(planSkill).not.toContain('Before/after test count');
|
|
expect(planSkill).not.toContain('30 code paths max');
|
|
expect(planSkill).not.toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review mode does NOT include test plan artifact', () => {
|
|
expect(reviewSkill).not.toContain('Test Plan Artifact');
|
|
expect(reviewSkill).not.toContain('eng-review-test-plan');
|
|
expect(reviewSkill).not.toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review/specialists/ directory has all expected checklist files', () => {
|
|
const specDir = path.join(ROOT, 'review', 'specialists');
|
|
const expected = [
|
|
'testing.md',
|
|
'maintainability.md',
|
|
'security.md',
|
|
'performance.md',
|
|
'data-migration.md',
|
|
'api-contract.md',
|
|
'red-team.md',
|
|
];
|
|
for (const f of expected) {
|
|
expect(fs.existsSync(path.join(specDir, f))).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('each specialist file has standard header with scope and output format', () => {
|
|
const specDir = path.join(ROOT, 'review', 'specialists');
|
|
const files = fs.readdirSync(specDir).filter(f => f.endsWith('.md'));
|
|
for (const f of files) {
|
|
const content = fs.readFileSync(path.join(specDir, f), 'utf-8');
|
|
// All specialist files must have Scope and Output/JSON in header
|
|
expect(content).toContain('Scope:');
|
|
expect(content.toLowerCase()).toMatch(/output|json/);
|
|
// Must define NO FINDINGS behavior
|
|
expect(content).toContain('NO FINDINGS');
|
|
}
|
|
});
|
|
|
|
// Regression guard: ship output contains key phrases from before the refactor
|
|
test('ship SKILL.md regression guard — key phrases preserved', () => {
|
|
const regressionPhrases = [
|
|
'100% coverage is the goal',
|
|
'ASCII coverage diagram',
|
|
'processPayment',
|
|
'refundPayment',
|
|
'billing.test.ts',
|
|
'checkout.e2e.ts',
|
|
'COVERAGE:',
|
|
'QUALITY:',
|
|
'GAPS:',
|
|
'Code paths:',
|
|
'User flows:',
|
|
];
|
|
for (const phrase of regressionPhrases) {
|
|
expect(shipSkill).toContain(phrase);
|
|
}
|
|
});
|
|
|
|
test('ship SKILL.md contains review army specialist dispatch', () => {
|
|
expect(shipSkill).toContain('Specialist Dispatch');
|
|
expect(shipSkill).toContain('Step 9.1');
|
|
expect(shipSkill).toContain('Step 9.2');
|
|
});
|
|
|
|
test('ship SKILL.md contains cross-review finding dedup', () => {
|
|
expect(shipSkill).toContain('Cross-review finding dedup');
|
|
expect(shipSkill).toContain('Step 9.3');
|
|
});
|
|
|
|
test('ship SKILL.md contains re-run idempotency behavior', () => {
|
|
expect(shipSkill).toContain('Re-run behavior (idempotency)');
|
|
expect(shipSkill).toContain('Never skip a verification step');
|
|
});
|
|
});
|
|
|
|
// --- {{TEST_FAILURE_TRIAGE}} resolver tests ---
|
|
|
|
describe('TEST_FAILURE_TRIAGE resolver', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('contains all 4 triage steps', () => {
|
|
expect(shipSkill).toContain('Step T1: Classify each failure');
|
|
expect(shipSkill).toContain('Step T2: Handle in-branch failures');
|
|
expect(shipSkill).toContain('Step T3: Handle pre-existing failures');
|
|
expect(shipSkill).toContain('Step T4: Execute the chosen action');
|
|
});
|
|
|
|
test('T1 includes classification criteria (in-branch vs pre-existing)', () => {
|
|
expect(shipSkill).toContain('In-branch');
|
|
expect(shipSkill).toContain('Likely pre-existing');
|
|
expect(shipSkill).toContain('git diff origin/');
|
|
});
|
|
|
|
test('T3 branches on REPO_MODE (solo vs collaborative)', () => {
|
|
expect(shipSkill).toContain('REPO_MODE');
|
|
expect(shipSkill).toContain('solo');
|
|
expect(shipSkill).toContain('collaborative');
|
|
});
|
|
|
|
test('solo mode offers fix-now, TODO, and skip options', () => {
|
|
expect(shipSkill).toContain('Investigate and fix now');
|
|
expect(shipSkill).toContain('Add as P0 TODO');
|
|
expect(shipSkill).toContain('Skip');
|
|
});
|
|
|
|
test('collaborative mode offers blame + assign option', () => {
|
|
expect(shipSkill).toContain('Blame + assign GitHub issue');
|
|
expect(shipSkill).toContain('gh issue create');
|
|
});
|
|
|
|
test('defaults ambiguous failures to in-branch (safety)', () => {
|
|
expect(shipSkill).toContain('When ambiguous, default to in-branch');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_FILE_REVIEW_REPORT}} resolver tests ---
|
|
|
|
describe('PLAN_FILE_REVIEW_REPORT resolver', () => {
|
|
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review', 'codex'];
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`plan file review report appears in ${skill} generated file`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK REVIEW REPORT');
|
|
});
|
|
}
|
|
|
|
test('resolver output contains key report elements', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: report writer moved to section
|
|
expect(content).toContain('Trigger');
|
|
expect(content).toContain('Findings');
|
|
expect(content).toContain('VERDICT');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-design-review');
|
|
expect(content).toContain('/codex review');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_COMPLETION_AUDIT}} resolver tests ---
|
|
|
|
describe('PLAN_COMPLETION_AUDIT placeholders', () => {
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains plan completion audit step', () => {
|
|
expect(shipSkill).toContain('Plan Completion Audit');
|
|
expect(shipSkill).toContain('Step 8');
|
|
});
|
|
|
|
test('review SKILL.md contains plan completion in scope drift', () => {
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Actionable Item Extraction');
|
|
expect(reviewSkill).toContain('Integration with Scope Drift Detection');
|
|
});
|
|
|
|
test('both modes share plan file discovery methodology', () => {
|
|
expect(shipSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
// Both should have conversation context first
|
|
expect(shipSkill).toContain('Conversation context (primary)');
|
|
expect(reviewSkill).toContain('Conversation context (primary)');
|
|
// Both should have grep fallback
|
|
expect(shipSkill).toContain('Content-based search (fallback)');
|
|
expect(reviewSkill).toContain('Content-based search (fallback)');
|
|
});
|
|
|
|
test('ship mode has gate logic for NOT DONE items', () => {
|
|
expect(shipSkill).toContain('NOT DONE');
|
|
expect(shipSkill).toContain('Stop — implement the missing items');
|
|
expect(shipSkill).toContain('Ship anyway — defer');
|
|
expect(shipSkill).toContain('intentionally dropped');
|
|
});
|
|
|
|
test('review mode is INFORMATIONAL only', () => {
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
expect(reviewSkill).toContain('MISSING REQUIREMENTS');
|
|
expect(reviewSkill).toContain('SCOPE CREEP');
|
|
});
|
|
|
|
test('item extraction has 50-item cap', () => {
|
|
expect(shipSkill).toContain('at most 50 items');
|
|
});
|
|
|
|
test('uses file-level traceability (not commit-level)', () => {
|
|
expect(shipSkill).toContain('Cite the specific file');
|
|
expect(shipSkill).not.toContain('commit-level traceability');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_VERIFICATION_EXEC}} resolver tests ---
|
|
|
|
describe('PLAN_VERIFICATION_EXEC placeholder', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('ship SKILL.md contains plan verification step', () => {
|
|
expect(shipSkill).toContain('Step 8.1');
|
|
expect(shipSkill).toContain('Plan Verification');
|
|
});
|
|
|
|
test('references /qa-only invocation', () => {
|
|
expect(shipSkill).toContain('qa-only/SKILL.md');
|
|
expect(shipSkill).toContain('qa-only');
|
|
});
|
|
|
|
test('contains dev-server discovery (CLAUDE.md first, then a port probe)', () => {
|
|
// Fork port wave 2: the hardcoded 4-port list became read-CLAUDE.md-or-
|
|
// probe; the probe loops common ports instead of naming each once.
|
|
expect(shipSkill).toContain('CLAUDE.md first');
|
|
expect(shipSkill).toContain('http://localhost:$_p');
|
|
expect(shipSkill).toContain('NO_SERVER');
|
|
});
|
|
|
|
test('skips gracefully when no verification section', () => {
|
|
expect(shipSkill).toContain('No verification steps found in plan');
|
|
});
|
|
|
|
test('skips gracefully when no dev server', () => {
|
|
expect(shipSkill).toContain('No dev server detected');
|
|
});
|
|
});
|
|
|
|
// --- Coverage gate tests ---
|
|
|
|
describe('Coverage gate in ship', () => {
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains coverage gate with thresholds', () => {
|
|
expect(shipSkill).toContain('Coverage gate');
|
|
expect(shipSkill).toContain('>= target');
|
|
expect(shipSkill).toContain('< minimum');
|
|
});
|
|
|
|
test('ship SKILL.md supports configurable thresholds via CLAUDE.md', () => {
|
|
expect(shipSkill).toContain('## Test Coverage');
|
|
expect(shipSkill).toContain('Minimum:');
|
|
expect(shipSkill).toContain('Target:');
|
|
});
|
|
|
|
test('coverage gate skips on parse failure (not block)', () => {
|
|
expect(shipSkill).toContain('could not determine percentage — skipping');
|
|
});
|
|
|
|
test('review SKILL.md delegates coverage to Testing specialist', () => {
|
|
// Coverage audit moved to Testing specialist subagent in Review Army
|
|
expect(reviewSkill).toContain('testing.md');
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
});
|
|
});
|
|
|
|
// --- Ship metrics logging ---
|
|
|
|
describe('Ship metrics logging', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('ship SKILL.md contains metrics persistence step', () => {
|
|
expect(shipSkill).toContain('Step 20');
|
|
expect(shipSkill).toContain('coverage_pct');
|
|
expect(shipSkill).toContain('plan_items_total');
|
|
expect(shipSkill).toContain('plan_items_done');
|
|
expect(shipSkill).toContain('verification_result');
|
|
});
|
|
});
|
|
|
|
// --- Plan file discovery shared helper ---
|
|
|
|
describe('Plan file discovery shared helper', () => {
|
|
// The shared helper should appear in ship (via PLAN_COMPLETION_AUDIT_SHIP)
|
|
// and in review (via PLAN_COMPLETION_AUDIT_REVIEW)
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan file discovery appears in both ship and review', () => {
|
|
expect(shipSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
});
|
|
|
|
test('both include conversation context first', () => {
|
|
expect(shipSkill).toContain('Conversation context (primary)');
|
|
expect(reviewSkill).toContain('Conversation context (primary)');
|
|
});
|
|
|
|
test('both include content-based fallback', () => {
|
|
expect(shipSkill).toContain('Content-based search (fallback)');
|
|
expect(reviewSkill).toContain('Content-based search (fallback)');
|
|
});
|
|
});
|
|
|
|
// --- Retro plan completion ---
|
|
|
|
describe('Retro plan completion section', () => {
|
|
const retroSkill = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
|
|
|
test('retro SKILL.md contains plan completion section', () => {
|
|
expect(retroSkill).toContain('### Plan Completion');
|
|
expect(retroSkill).toContain('plan_items_total');
|
|
expect(retroSkill).toContain('Plan Completion This Period');
|
|
});
|
|
});
|
|
|
|
// --- Plan status footer in preamble ---
|
|
|
|
describe('Plan status footer in preamble', () => {
|
|
test('preamble contains plan status footer as neutral forward reference to EXIT PLAN MODE GATE', () => {
|
|
// Read any skill that uses PREAMBLE
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
expect(content).toContain('Plan Status Footer');
|
|
expect(content).toContain('GSTACK REVIEW REPORT');
|
|
expect(content).toContain('ExitPlanMode');
|
|
expect(content).toContain('EXIT PLAN MODE GATE');
|
|
// The preamble must NOT impose review-report rules on operational skills
|
|
// that have no review report. It's a forward reference, not enforcement.
|
|
expect(content).not.toContain('NO REVIEWS YET');
|
|
});
|
|
});
|
|
|
|
// --- make-pdf setup ordering ---
|
|
|
|
describe('make-pdf setup ordering', () => {
|
|
test('MAKE-PDF SETUP appears before generic preamble footer sections', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'make-pdf', 'SKILL.md'), 'utf-8');
|
|
const preambleIdx = content.indexOf('## Preamble (run first)');
|
|
const setupIdx = content.indexOf('## MAKE-PDF SETUP');
|
|
const planModeIdx = content.indexOf('## Plan Mode Safe Operations');
|
|
const telemetryIdx = content.indexOf('## Telemetry (run last)');
|
|
const workflowIdx = content.indexOf('# make-pdf: publication-quality PDFs from markdown');
|
|
|
|
expect(preambleIdx).toBeGreaterThanOrEqual(0);
|
|
expect(setupIdx).toBeGreaterThan(preambleIdx);
|
|
expect(setupIdx).toBeLessThan(planModeIdx);
|
|
expect(setupIdx).toBeLessThan(telemetryIdx);
|
|
expect(setupIdx).toBeLessThan(workflowIdx);
|
|
expect(content.match(/^## MAKE-PDF SETUP/gm)?.length ?? 0).toBe(1);
|
|
});
|
|
});
|
|
|
|
// --- Skill invocation during plan mode in preamble ---
|
|
|
|
describe('Skill invocation during plan mode in preamble', () => {
|
|
test('preamble contains skill invocation plan mode section', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
expect(content).toContain('Skill Invocation During Plan Mode');
|
|
expect(content).toContain('precedence over generic plan mode behavior');
|
|
expect(content).toContain('Do not continue the workflow');
|
|
expect(content).toContain('cancel the skill or leave plan mode');
|
|
});
|
|
});
|
|
|
|
// --- {{SPEC_REVIEW_LOOP}} resolver tests ---
|
|
|
|
describe('SPEC_REVIEW_LOOP resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('contains all 5 review dimensions', () => {
|
|
for (const dim of ['Completeness', 'Consistency', 'Clarity', 'Scope', 'Feasibility']) {
|
|
expect(content).toContain(dim);
|
|
}
|
|
});
|
|
|
|
test('references Agent tool for subagent dispatch', () => {
|
|
expect(content).toMatch(/Agent.*tool/i);
|
|
});
|
|
|
|
test('specifies max 3 iterations', () => {
|
|
expect(content).toMatch(/3.*iteration|maximum.*3/i);
|
|
});
|
|
|
|
test('includes quality score', () => {
|
|
expect(content).toContain('quality score');
|
|
});
|
|
|
|
test('includes metrics path', () => {
|
|
expect(content).toContain('spec-review.jsonl');
|
|
});
|
|
|
|
test('includes convergence guard', () => {
|
|
expect(content).toMatch(/[Cc]onvergence/);
|
|
});
|
|
|
|
test('includes graceful failure handling', () => {
|
|
expect(content).toMatch(/skip.*review|unavailable/i);
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_SKETCH}} resolver tests ---
|
|
|
|
describe('DESIGN_SKETCH resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('references DESIGN.md for design system constraints', () => {
|
|
expect(content).toContain('DESIGN.md');
|
|
});
|
|
|
|
test('contains wireframe or sketch terminology', () => {
|
|
expect(content).toMatch(/wireframe|sketch/i);
|
|
});
|
|
|
|
test('references browse binary for rendering', () => {
|
|
expect(content).toContain('$B goto');
|
|
});
|
|
|
|
test('references screenshot capture', () => {
|
|
expect(content).toContain('$B screenshot');
|
|
});
|
|
|
|
test('specifies rough aesthetic', () => {
|
|
expect(content).toMatch(/[Rr]ough|hand-drawn/);
|
|
});
|
|
|
|
test('includes skip conditions', () => {
|
|
expect(content).toMatch(/no UI component|skip/i);
|
|
});
|
|
});
|
|
|
|
// --- {{CODEX_SECOND_OPINION}} resolver tests ---
|
|
|
|
describe('CODEX_SECOND_OPINION resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
const codexContent = fs.readFileSync(path.join(ROOT, '.agents', 'skills', 'gstack-office-hours', 'SKILL.md'), 'utf-8');
|
|
|
|
test('Phase 3.5 section appears in office-hours SKILL.md', () => {
|
|
expect(content).toContain('Phase 3.5: Cross-Model Second Opinion');
|
|
});
|
|
|
|
test('contains codex exec invocation', () => {
|
|
expect(content).toContain('codex exec');
|
|
});
|
|
|
|
test('contains opt-in AskUserQuestion text', () => {
|
|
expect(content).toContain('second opinion from an independent AI perspective');
|
|
});
|
|
|
|
test('contains cross-model synthesis instructions', () => {
|
|
expect(content).toMatch(/[Ss]ynthesis/);
|
|
expect(content).toContain('Where Claude agrees with the second opinion');
|
|
});
|
|
|
|
test('contains Claude subagent fallback', () => {
|
|
expect(content).toContain('CODEX_NOT_AVAILABLE');
|
|
expect(content).toContain('Agent tool');
|
|
expect(content).toContain('SECOND OPINION (Claude subagent)');
|
|
});
|
|
|
|
test('contains premise revision check', () => {
|
|
expect(content).toContain('Codex challenged premise');
|
|
});
|
|
|
|
test('contains error handling for auth, timeout, and empty', () => {
|
|
expect(content).toMatch(/[Aa]uth.*fail/);
|
|
expect(content).toMatch(/[Tt]imeout/);
|
|
expect(content).toMatch(/[Ee]mpty response/);
|
|
});
|
|
|
|
test('Codex host variant does NOT contain the Phase 3.5 resolver output', () => {
|
|
// The resolver returns '' for codex host, so the interactive section is stripped.
|
|
// Static template references to "Phase 3.5" in prose/conditionals are fine.
|
|
// Other resolvers (design review lite) may contain CODEX_NOT_AVAILABLE, so we
|
|
// check for Phase 3.5-specific markers only.
|
|
expect(codexContent).not.toContain('Phase 3.5: Cross-Model Second Opinion');
|
|
expect(codexContent).not.toContain('TMPERR_OH');
|
|
expect(codexContent).not.toContain('gstack-codex-oh-');
|
|
});
|
|
});
|
|
|
|
// --- Codex filesystem boundary tests ---
|
|
|
|
describe('Codex filesystem boundary', () => {
|
|
// Skills that call codex exec/review and should contain boundary text
|
|
const CODEX_CALLING_SKILLS = [
|
|
'codex', // /codex skill — 3 modes
|
|
'autoplan', // /autoplan — CEO/design/eng voices
|
|
'review', // /review — adversarial step resolver
|
|
'ship', // /ship — adversarial step resolver
|
|
'plan-eng-review', // outside voice resolver
|
|
'plan-ceo-review', // outside voice resolver
|
|
'office-hours', // second opinion resolver
|
|
];
|
|
|
|
const BOUNDARY_MARKER = 'Do NOT read or execute any';
|
|
|
|
test('boundary instruction appears in all skills that call codex', () => {
|
|
for (const skill of CODEX_CALLING_SKILLS) {
|
|
// Union: ship's codex call lives in sections/adversarial.md after the carve.
|
|
const content = readSkillUnion(skill);
|
|
expect(content).toContain(BOUNDARY_MARKER);
|
|
}
|
|
});
|
|
|
|
test('codex skill has Filesystem Boundary section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('## Filesystem Boundary');
|
|
expect(content).toContain('skill definitions meant for a different AI system');
|
|
});
|
|
|
|
test('codex skill has rabbit-hole detection rule', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Detect skill-file rabbit holes');
|
|
expect(content).toContain('gstack-update-check');
|
|
expect(content).toContain('Consider retrying');
|
|
});
|
|
|
|
test('review.ts CODEX_BOUNDARY constant is interpolated into resolver output', () => {
|
|
// The adversarial step resolver should include boundary text in codex exec prompts
|
|
const reviewContent = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
// Boundary should appear near codex exec invocations
|
|
const boundaryIdx = reviewContent.indexOf(BOUNDARY_MARKER);
|
|
const codexExecIdx = reviewContent.indexOf('codex exec');
|
|
// Both must exist and boundary must come before a codex exec call
|
|
expect(boundaryIdx).toBeGreaterThan(-1);
|
|
expect(codexExecIdx).toBeGreaterThan(-1);
|
|
});
|
|
|
|
test('autoplan boundary text avoids host-specific paths for cross-host compatibility', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'autoplan', 'SKILL.md.tmpl'), 'utf-8');
|
|
// autoplan template uses generic 'skills/gstack' pattern instead of host-specific
|
|
// paths like ~/.claude/ or .agents/skills (which break Codex/Claude output tests)
|
|
const boundaryStart = content.indexOf('Filesystem Boundary');
|
|
const boundaryEnd = content.indexOf('---', boundaryStart + 1);
|
|
const boundarySection = content.slice(boundaryStart, boundaryEnd);
|
|
expect(boundarySection).not.toContain('~/.claude/');
|
|
expect(boundarySection).not.toContain('.agents/skills');
|
|
expect(boundarySection).toContain('skills/gstack');
|
|
expect(boundarySection).toContain(BOUNDARY_MARKER);
|
|
});
|
|
});
|
|
|
|
// --- {{BENEFITS_FROM}} resolver tests ---
|
|
|
|
describe('BENEFITS_FROM resolver', () => {
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
const engContent = readSkillUnion('plan-eng-review'); // carved
|
|
|
|
test('plan-ceo-review contains prerequisite skill offer', () => {
|
|
expect(ceoContent).toContain('Prerequisite Skill Offer');
|
|
expect(ceoContent).toContain('/office-hours');
|
|
});
|
|
|
|
test('plan-eng-review contains prerequisite skill offer', () => {
|
|
expect(engContent).toContain('Prerequisite Skill Offer');
|
|
expect(engContent).toContain('/office-hours');
|
|
});
|
|
|
|
test('offer includes graceful decline', () => {
|
|
expect(ceoContent).toContain('No worries');
|
|
});
|
|
|
|
test('skills without benefits-from do NOT have prerequisite offer', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
expect(qaContent).not.toContain('Prerequisite Skill Offer');
|
|
});
|
|
|
|
test('inline invocation — no "another window" language', () => {
|
|
expect(ceoContent).not.toContain('another window');
|
|
expect(engContent).not.toContain('another window');
|
|
});
|
|
|
|
test('inline invocation — read-and-follow path present', () => {
|
|
expect(ceoContent).toContain('office-hours/SKILL.md');
|
|
expect(engContent).toContain('office-hours/SKILL.md');
|
|
});
|
|
|
|
test('BENEFITS_FROM delegates to INVOKE_SKILL pattern', () => {
|
|
// Should contain the INVOKE_SKILL-style loading prose (not the old manual skip list)
|
|
expect(engContent).toContain('Follow its instructions from top to bottom');
|
|
expect(engContent).toContain('skipping these sections');
|
|
expect(ceoContent).toContain('Follow its instructions from top to bottom');
|
|
});
|
|
});
|
|
|
|
// --- {{INVOKE_SKILL}} resolver tests ---
|
|
|
|
describe('INVOKE_SKILL resolver', () => {
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan-ceo-review uses INVOKE_SKILL for mid-session office-hours fallback', () => {
|
|
// The mid-session detection path should use INVOKE_SKILL-generated prose
|
|
expect(ceoContent).toContain('office-hours/SKILL.md');
|
|
expect(ceoContent).toContain('Follow its instructions from top to bottom');
|
|
});
|
|
|
|
test('INVOKE_SKILL output includes default skip list', () => {
|
|
expect(ceoContent).toContain('Preamble (run first)');
|
|
expect(ceoContent).toContain('Telemetry (run last)');
|
|
expect(ceoContent).toContain('AskUserQuestion Format');
|
|
});
|
|
|
|
test('INVOKE_SKILL output includes error handling', () => {
|
|
expect(ceoContent).toContain('If unreadable');
|
|
expect(ceoContent).toContain('Could not load');
|
|
});
|
|
|
|
test('template uses {{INVOKE_SKILL:office-hours}} placeholder', () => {
|
|
const tmpl = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(tmpl).toContain('{{INVOKE_SKILL:office-hours}}');
|
|
});
|
|
});
|
|
|
|
// --- {{CHANGELOG_WORKFLOW}} resolver tests ---
|
|
|
|
describe('CHANGELOG_WORKFLOW resolver', () => {
|
|
const shipContent = readShipUnion();
|
|
|
|
test('ship SKILL.md contains changelog workflow', () => {
|
|
expect(shipContent).toContain('CHANGELOG (auto-generate)');
|
|
expect(shipContent).toContain('git log <base>..HEAD --oneline');
|
|
});
|
|
|
|
test('changelog workflow includes cross-check step', () => {
|
|
expect(shipContent).toContain('Cross-check');
|
|
expect(shipContent).toContain('Every commit must map to at least one bullet point');
|
|
});
|
|
|
|
test('changelog workflow includes voice guidance', () => {
|
|
expect(shipContent).toContain('Lead with what the user can now **do**');
|
|
});
|
|
|
|
test('template uses {{CHANGELOG_WORKFLOW}} placeholder', () => {
|
|
// Post-carve (T9): the skeleton points to the changelog section, which carries
|
|
// the resolver. Neither should inline the old changelog content.
|
|
const skel = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');
|
|
const changelogSection = fs.readFileSync(path.join(ROOT, 'ship', 'sections', 'changelog.md.tmpl'), 'utf-8');
|
|
expect(skel).toContain('{{SECTION:changelog}}');
|
|
expect(changelogSection).toContain('{{CHANGELOG_WORKFLOW}}');
|
|
expect(skel + changelogSection).not.toContain('Group commits by theme');
|
|
});
|
|
|
|
test('changelog workflow includes keep-changelog format', () => {
|
|
expect(shipContent).toContain('### Added');
|
|
expect(shipContent).toContain('### Fixed');
|
|
});
|
|
});
|
|
|
|
// --- Duplicate {{PREAMBLE}} guard (#2508/#2362) ---
|
|
|
|
describe('assertSinglePreamble', () => {
|
|
test('one {{PREAMBLE}} passes', () => {
|
|
expect(() => assertSinglePreamble('a\n{{PREAMBLE}}\nb', 'x/SKILL.md.tmpl')).not.toThrow();
|
|
});
|
|
|
|
test('zero {{PREAMBLE}} passes (sections have none)', () => {
|
|
expect(() => assertSinglePreamble('no macro here', 'x/sections/y.md.tmpl')).not.toThrow();
|
|
});
|
|
|
|
test('a second occurrence throws with the template path — even in prose', () => {
|
|
// The original #2508 bug WAS a prose mention: "emitted by {{PREAMBLE}}'s
|
|
// preamble bash". Resolution is context-blind, so the guard must be too.
|
|
const tmpl = '{{PREAMBLE}}\n\n...later: emitted by {{PREAMBLE}}\'s preamble bash';
|
|
expect(() => assertSinglePreamble(tmpl, 'spec/SKILL.md.tmpl')).toThrow(/spec\/SKILL\.md\.tmpl.*2 times/);
|
|
});
|
|
});
|
|
|
|
// --- Parameterized resolver infrastructure tests ---
|
|
|
|
describe('parameterized resolver support', () => {
|
|
test('gen-skill-docs regex handles colon-separated args', () => {
|
|
// Verify the template containing {{INVOKE_SKILL:office-hours}} was processed
|
|
// without leaving unresolved placeholders
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(ceoContent).not.toMatch(/\{\{INVOKE_SKILL:[^}]+\}\}/);
|
|
});
|
|
|
|
test('templates with parameterized resolvers pass unresolved check', () => {
|
|
// All generated SKILL.md files should have no unresolved {{...}} placeholders
|
|
const skillDirs = fs.readdirSync(ROOT).filter(d =>
|
|
fs.existsSync(path.join(ROOT, d, 'SKILL.md'))
|
|
);
|
|
for (const dir of skillDirs) {
|
|
const content = fs.readFileSync(path.join(ROOT, dir, 'SKILL.md'), 'utf-8');
|
|
const unresolved = content.match(/\{\{[A-Z_]+(?::[^}]*)?\}\}/g);
|
|
if (unresolved) {
|
|
throw new Error(`${dir}/SKILL.md has unresolved placeholders: ${unresolved.join(', ')}`);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
// --- Preamble routing injection tests ---
|
|
|
|
describe('preamble routing injection', () => {
|
|
const shipContent = readShipUnion();
|
|
|
|
test('preamble bash checks for routing section in CLAUDE.md and AGENTS.md', () => {
|
|
// #2500: the probe iterates CLAUDE.md AND AGENTS.md — non-Claude hosts
|
|
// route skills via AGENTS.md, the cross-harness convention file.
|
|
expect(shipContent).toContain('for _RF in CLAUDE.md AGENTS.md');
|
|
expect(shipContent).toContain('grep -q "## Skill routing" "$_RF"');
|
|
expect(shipContent).toContain('HAS_ROUTING');
|
|
});
|
|
|
|
test('preamble bash reads routing_declined config', () => {
|
|
expect(shipContent).toContain('routing_declined');
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
});
|
|
|
|
test('preamble includes routing injection AskUserQuestion', () => {
|
|
expect(shipContent).toContain('Add routing rules to CLAUDE.md');
|
|
expect(shipContent).toContain("I'll invoke skills manually");
|
|
});
|
|
|
|
test('routing injection respects prior decline', () => {
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
expect(shipContent).toMatch(/routing_declined.*true/);
|
|
});
|
|
|
|
test('routing injection only fires when all conditions met', () => {
|
|
// Must be: HAS_ROUTING=no AND ROUTING_DECLINED=false AND PROACTIVE_PROMPTED=yes
|
|
expect(shipContent).toContain('HAS_ROUTING');
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
expect(shipContent).toContain('PROACTIVE_PROMPTED');
|
|
});
|
|
|
|
test('routing section content includes key routing rules', () => {
|
|
expect(shipContent).toContain('invoke /office-hours');
|
|
expect(shipContent).toContain('invoke /investigate');
|
|
expect(shipContent).toContain('invoke /ship');
|
|
expect(shipContent).toContain('invoke /qa');
|
|
});
|
|
|
|
test('routing section uses renamed checkpoint skills (not stale /checkpoint)', () => {
|
|
expect(shipContent).toContain('invoke /context-save');
|
|
expect(shipContent).toContain('invoke /context-restore');
|
|
expect(shipContent).not.toContain('invoke checkpoint');
|
|
});
|
|
|
|
test('routing section uses soft "when in doubt" policy, not hard "ALWAYS invoke"', () => {
|
|
expect(shipContent).toContain('When in doubt, invoke the skill');
|
|
expect(shipContent).not.toContain('Do NOT answer directly');
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_OUTSIDE_VOICES}} resolver tests ---
|
|
|
|
describe('DESIGN_OUTSIDE_VOICES resolver', () => {
|
|
test('plan-design-review contains outside voices section', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('CODEX_AVAILABLE');
|
|
expect(content).toContain('LITMUS SCORECARD');
|
|
});
|
|
|
|
test('design-review contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('source audit');
|
|
});
|
|
|
|
test('design-consultation contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('design direction');
|
|
});
|
|
|
|
test('branches correctly per skillName — different prompts', () => {
|
|
const planContent = readSkillUnion('plan-design-review');
|
|
const consultContent = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
|
|
// plan-design-review uses analytical prompt (high reasoning)
|
|
expect(planContent).toContain('model_reasoning_effort="high"');
|
|
// design-consultation uses creative prompt (medium reasoning)
|
|
expect(consultContent).toContain('model_reasoning_effort="medium"');
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_HARD_RULES}} resolver tests ---
|
|
|
|
describe('DESIGN_HARD_RULES resolver', () => {
|
|
test('plan-design-review Pass 4 contains hard rules', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Design Hard Rules');
|
|
expect(content).toContain('Classifier');
|
|
expect(content).toContain('MARKETING/LANDING PAGE');
|
|
expect(content).toContain('APP UI');
|
|
});
|
|
|
|
test('design-review contains hard rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Hard Rules');
|
|
});
|
|
|
|
test('includes all 3 rule sets', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Landing page rules');
|
|
expect(content).toContain('App UI rules');
|
|
expect(content).toContain('Universal rules');
|
|
});
|
|
|
|
test('references shared AI slop blacklist items', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('3-column feature grid');
|
|
expect(content).toContain('Purple/violet/indigo');
|
|
});
|
|
|
|
test('includes OpenAI hard rejection criteria', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Generic SaaS card grid');
|
|
expect(content).toContain('Carousel with no narrative purpose');
|
|
});
|
|
|
|
test('includes OpenAI litmus checks', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Brand/product unmistakable');
|
|
expect(content).toContain('premium with all decorative shadows removed');
|
|
});
|
|
});
|
|
|
|
// --- Extended DESIGN_SKETCH resolver tests ---
|
|
|
|
describe('DESIGN_SKETCH extended with outside voices', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('contains outside design voices step', () => {
|
|
expect(content).toContain('Outside design voices');
|
|
});
|
|
|
|
test('offers opt-in via AskUserQuestion', () => {
|
|
expect(content).toContain('outside design perspectives');
|
|
});
|
|
|
|
test('still contains original wireframe steps', () => {
|
|
expect(content).toContain('wireframe');
|
|
expect(content).toContain('$B goto');
|
|
});
|
|
});
|
|
|
|
// --- Extended DESIGN_REVIEW_LITE resolver tests ---
|
|
|
|
describe('DESIGN_REVIEW_LITE extended with Codex', () => {
|
|
const content = readShipUnion();
|
|
|
|
test('contains Codex design voice block', () => {
|
|
expect(content).toContain('Codex design voice');
|
|
expect(content).toContain('CODEX (design)');
|
|
});
|
|
|
|
test('still contains original checklist steps', () => {
|
|
expect(content).toContain('design-checklist.md');
|
|
expect(content).toContain('SCOPE_FRONTEND');
|
|
});
|
|
|
|
});
|
|
|
|
// ─── Codex Generation Tests ─────────────────────────────────
|
|
|
|
describe('Codex generation (--host codex)', () => {
|
|
const AGENTS_DIR = path.join(ROOT, '.agents', 'skills');
|
|
|
|
// .agents/ is gitignored (v0.11.2.0) — generate on demand for tests
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
|
|
// Dynamic discovery of expected Codex skills: all templates except /codex
|
|
// Also excludes skills where .agents/skills/{name} is a symlink back to the repo root
|
|
// (vendored dev mode — gen-skill-docs skips these to avoid overwriting Claude SKILL.md)
|
|
const CODEX_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; codexName: string }> = [];
|
|
const isSymlinkLoop = (codexName: string): boolean => {
|
|
const agentSkillDir = path.join(ROOT, '.agents', 'skills', codexName);
|
|
try {
|
|
return fs.realpathSync(agentSkillDir) === fs.realpathSync(ROOT);
|
|
} catch { return false; }
|
|
};
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
if (!isSymlinkLoop('gstack')) {
|
|
skills.push({ dir: '.', codexName: 'gstack' });
|
|
}
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (entry.name === 'codex') continue; // /codex is excluded from Codex output
|
|
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
|
const codexName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
|
if (isSymlinkLoop(codexName)) continue;
|
|
skills.push({ dir: entry.name, codexName });
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
test('--host codex generates correct output paths', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const skillMd = path.join(AGENTS_DIR, skill.codexName, 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('root gstack bundle has OpenAI metadata for Codex skill browsing', () => {
|
|
const rootMetadata = path.join(ROOT, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(rootMetadata)).toBe(true);
|
|
const content = fs.readFileSync(rootMetadata, 'utf-8');
|
|
expect(content).toContain('display_name: "gstack"');
|
|
expect(content).toContain('Use $gstack to locate the bundled gstack skills.');
|
|
expect(content).toContain('allow_implicit_invocation: true');
|
|
});
|
|
|
|
test('externalSkillName mapping: root is gstack, others are gstack-{dir}', () => {
|
|
// Root → gstack
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack', 'SKILL.md'))).toBe(true);
|
|
// Subdirectories → gstack-{dir}
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'))).toBe(true);
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'))).toBe(true);
|
|
// gstack-upgrade doesn't double-prefix
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-upgrade', 'SKILL.md'))).toBe(true);
|
|
// No double-prefix: gstack-gstack-upgrade must NOT exist
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-gstack-upgrade', 'SKILL.md'))).toBe(false);
|
|
});
|
|
|
|
test('Codex frontmatter has ONLY name + description', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content.startsWith('---\n')).toBe(true);
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
expect(fmEnd).toBeGreaterThan(0);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
// Must have name and description
|
|
expect(frontmatter).toContain('name:');
|
|
expect(frontmatter).toContain('description:');
|
|
// Must NOT have allowed-tools, version, or hooks
|
|
expect(frontmatter).not.toContain('allowed-tools:');
|
|
expect(frontmatter).not.toContain('version:');
|
|
expect(frontmatter).not.toContain('hooks:');
|
|
}
|
|
});
|
|
|
|
test('all Codex skills have agents/openai.yaml metadata', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const metadata = path.join(AGENTS_DIR, skill.codexName, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(metadata)).toBe(true);
|
|
const content = fs.readFileSync(metadata, 'utf-8');
|
|
expect(content).toContain(`display_name: "${skill.codexName}"`);
|
|
expect(content).toContain('short_description:');
|
|
expect(content).toContain('allow_implicit_invocation: true');
|
|
}
|
|
});
|
|
|
|
test('no .claude/skills/ in Codex output', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('no ~/.claude/ paths in Codex output', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('~/.claude/');
|
|
}
|
|
});
|
|
|
|
test('/codex skill excluded from Codex output', () => {
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex'))).toBe(false);
|
|
});
|
|
|
|
test('Codex output includes Claude outside-voice skill with read-only boundary', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-claude', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('claude -p');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-prompt-');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-response-XXXXXX');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-error-XXXXXX');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-diff-');
|
|
expect(content).not.toMatch(/gstack-claude-(?:prompt|response|error|diff)-X{6,}\.\w+/);
|
|
expect(content).not.toContain('/tmp/gstack-claude-diff-$$');
|
|
expect(content).toContain('cat "$PROMPT_FILE" | "$CLAUDE_BIN" -p');
|
|
expect(content).toContain('Resolve the binary and invoke it in the same host execution context');
|
|
expect(content).toContain('--disable-slash-commands');
|
|
expect(content).toContain('--tools ""');
|
|
expect(content).toContain('--allowedTools Read,Grep,Glob');
|
|
expect(content).toContain('--disallowedTools Bash,Edit,Write');
|
|
expect(content).toContain('Do not infer authentication state from credential files');
|
|
expect(content).toContain('run the actual `claude -p`');
|
|
expect(content).not.toContain('AUTH_MISSING');
|
|
expect(content).not.toContain('$HOME/.claude/.credentials.json');
|
|
expect(content).toContain('is_error');
|
|
});
|
|
|
|
test('Claude temp file templates are accepted by host mktemp', () => {
|
|
for (const template of [
|
|
'/tmp/gstack-claude-prompt-XXXXXX',
|
|
'/tmp/gstack-claude-response-XXXXXX',
|
|
'/tmp/gstack-claude-error-XXXXXX',
|
|
'/tmp/gstack-claude-diff-XXXXXX',
|
|
]) {
|
|
const result = spawnSync('mktemp', [template], { encoding: 'utf-8' });
|
|
expect(result.status).toBe(0);
|
|
const created = result.stdout.trim();
|
|
expect(created.startsWith(template.replace('XXXXXX', ''))).toBe(true);
|
|
fs.unlinkSync(created);
|
|
}
|
|
});
|
|
|
|
test('Codex review step stripped from Codex-host ship and review', () => {
|
|
const shipContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(shipContent).not.toContain('codex review --base');
|
|
expect(shipContent).not.toContain('CODEX_REVIEWS');
|
|
|
|
const reviewContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(reviewContent).not.toContain('codex review --base');
|
|
expect(reviewContent).not.toContain('CODEX_REVIEWS');
|
|
});
|
|
|
|
test('--host codex --dry-run freshness', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// Every Codex skill should be FRESH
|
|
for (const skill of CODEX_SKILLS) {
|
|
expect(output).toContain(`FRESH: .agents/skills/${skill.codexName}/SKILL.md`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('--host agents alias produces same output as --host codex', () => {
|
|
const codexResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
const agentsResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'agents', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(codexResult.exitCode).toBe(0);
|
|
expect(agentsResult.exitCode).toBe(0);
|
|
// Both should produce the same output (same FRESH lines)
|
|
expect(codexResult.stdout.toString()).toBe(agentsResult.stdout.toString());
|
|
});
|
|
|
|
test('multiline descriptions preserved in Codex output', () => {
|
|
// office-hours has a multiline description — verify it survives the frontmatter transform
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-office-hours', 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
// Description should span multiple lines (block scalar)
|
|
const descLines = frontmatter.split('\n').filter(l => l.startsWith(' '));
|
|
expect(descLines.length).toBeGreaterThan(1);
|
|
// Verify key phrases survived
|
|
expect(frontmatter).toContain('YC Office Hours');
|
|
});
|
|
|
|
test('hook skills have safety prose and no hooks: in frontmatter', () => {
|
|
const HOOK_SKILLS = ['gstack-careful', 'gstack-freeze', 'gstack-guard'];
|
|
for (const skillName of HOOK_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skillName, 'SKILL.md'), 'utf-8');
|
|
// Must have safety advisory prose
|
|
expect(content).toContain('Safety Advisory');
|
|
// Must NOT have hooks: in frontmatter
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).not.toContain('hooks:');
|
|
}
|
|
});
|
|
|
|
test('all Codex SKILL.md files have auto-generated header', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
}
|
|
});
|
|
|
|
test('Codex preamble resolves runtime assets from repo-local or global gstack roots', () => {
|
|
// Check a skill that has a preamble (review is a good candidate)
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT');
|
|
expect(content).toContain('$_ROOT/.agents/skills/gstack');
|
|
expect(content).toContain('$GSTACK_BIN/gstack-config');
|
|
expect(content).toContain('$GSTACK_ROOT/gstack-upgrade/SKILL.md');
|
|
expect(content).not.toContain('~/.codex/skills/gstack/bin/gstack-config get telemetry');
|
|
});
|
|
|
|
// ─── Path rewriting regression tests ─────────────────────────
|
|
|
|
test('sidecar paths resolve through $GSTACK_ROOT (not gstack-review/)', () => {
|
|
// #2518: templates now anchor sidecars at the installed skill root
|
|
// (~/.claude/skills/gstack/review/...), which the codex path rewrite turns
|
|
// into $GSTACK_ROOT/review/... — resolved by the preamble against the
|
|
// repo-local .agents root or the global install. The old repo-relative
|
|
// form (.claude/skills/review/) only resolved inside gstack's own checkout.
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('$GSTACK_ROOT/review/checklist.md');
|
|
// design-checklist.md is now referenced via Review Army specialist (Claude only, stripped for Codex)
|
|
// Wrong: must NOT reference gstack-review/checklist.md (file doesn't exist there)
|
|
expect(content).not.toContain('.agents/skills/gstack-review/checklist.md');
|
|
});
|
|
|
|
test('sidecar paths in ship skill point to gstack/review/ for pre-landing review', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
// Ship references the review checklist in its pre-landing review step
|
|
if (content.includes('checklist.md')) {
|
|
expect(content).toContain('.agents/skills/gstack/review/');
|
|
expect(content).not.toContain('.agents/skills/gstack-review/checklist');
|
|
}
|
|
});
|
|
|
|
test('greptile-triage sidecar path is correct', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
if (content.includes('greptile-triage')) {
|
|
expect(content).toContain('$GSTACK_ROOT/review/greptile-triage.md');
|
|
expect(content).not.toContain('.agents/skills/gstack-review/greptile-triage');
|
|
}
|
|
});
|
|
|
|
test('all four path rewrite rules produce correct output', () => {
|
|
// Test each of the 4 path rewrite rules individually
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
|
|
// Rule 1: ~/.claude/skills/gstack → $GSTACK_ROOT
|
|
expect(content).not.toContain('~/.claude/skills/gstack');
|
|
expect(content).toContain('$GSTACK_ROOT');
|
|
|
|
// Rule 2: .claude/skills/gstack → .agents/skills/gstack
|
|
expect(content).not.toContain('.claude/skills/gstack');
|
|
|
|
// Rule 3: .claude/skills/review → .agents/skills/gstack/review
|
|
expect(content).not.toContain('.claude/skills/review');
|
|
|
|
// Rule 4: .claude/skills → .agents/skills (catch-all)
|
|
expect(content).not.toContain('.claude/skills');
|
|
});
|
|
|
|
test('path rewrite rules apply to all Codex skills with sidecar references', () => {
|
|
// Verify across ALL generated skills, not just review
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
// No skill should reference Claude paths
|
|
expect(content).not.toContain('~/.claude/skills');
|
|
expect(content).not.toContain('.claude/skills');
|
|
if (content.includes('gstack-config') || content.includes('gstack-update-check') || content.includes('gstack-telemetry-log')) {
|
|
expect(content).toContain('$GSTACK_ROOT');
|
|
}
|
|
// If a skill references checklist.md, it must use the correct sidecar path
|
|
if (content.includes('checklist.md') && !content.includes('design-checklist.md')) {
|
|
expect(content).not.toContain('gstack-review/checklist.md');
|
|
}
|
|
}
|
|
});
|
|
|
|
// ─── Claude output regression guard ─────────────────────────
|
|
|
|
test('Claude output uses installed-root review paths (#2518)', () => {
|
|
// Codex changes must NOT affect Claude output; the Claude form is the
|
|
// installed-root anchor, not the old repo-relative path that only
|
|
// resolved inside gstack's own checkout.
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('~/.claude/skills/gstack/review/checklist.md');
|
|
expect(content).toContain('~/.claude/skills/gstack');
|
|
// Must NOT contain Codex HOST paths. `~/.codex/sessions/` is exempt: the
|
|
// timeout-wrapper guidance documents the Codex CLI's own rollout-log
|
|
// location (a user-facing CLI path, same class as ~/.codex/logs/ in the
|
|
// codex skill), not the gstack Codex host install path.
|
|
// `~/.codex/config.toml` is the same user-facing class: the shared
|
|
// codexPreflight's model_unusable branch (#2477) points at the CLI's own
|
|
// config file, where the rejected `model =` pin lives.
|
|
expect(content).not.toContain('.agents/skills');
|
|
expect(
|
|
content
|
|
.replaceAll('~/.codex/sessions/', '')
|
|
.replaceAll('~/.codex/config.toml', ''),
|
|
).not.toContain('~/.codex/');
|
|
});
|
|
|
|
test('Claude output unchanged: ship skill still uses .claude/skills/ paths', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('~/.claude/skills/gstack');
|
|
expect(content).not.toContain('.agents/skills');
|
|
// ~/.codex/sessions/ is the Codex CLI's rollout-log path (user-facing),
|
|
// documented by the adversarial-pass timeout guidance; ~/.codex/config.toml
|
|
// is the CLI's own config file (model_unusable guidance, #2477) — see the
|
|
// review test above.
|
|
expect(
|
|
content
|
|
.replaceAll('~/.codex/sessions/', '')
|
|
.replaceAll('~/.codex/config.toml', ''),
|
|
).not.toContain('~/.codex/');
|
|
});
|
|
|
|
test('Claude output unchanged: all Claude skills have zero Codex paths', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
// pair-agent legitimately documents how Codex agents store credentials.
|
|
// codex + autoplan document the Codex CLI auth file (~/.codex/auth.json)
|
|
// and log path (~/.codex/logs/) — those are user-facing Codex CLI paths,
|
|
// not the gstack Codex host install path. ~/.codex/sessions/ (rollout
|
|
// logs, referenced by the review/ship timeout guidance) and
|
|
// ~/.codex/config.toml (the model_unusable guidance in the shared
|
|
// codexPreflight, #2477) are the same user-facing class, so they are
|
|
// scrubbed before the ban.
|
|
if (skill.dir !== 'pair-agent' && skill.dir !== 'codex' && skill.dir !== 'autoplan') {
|
|
expect(
|
|
content
|
|
.replaceAll('~/.codex/sessions/', '')
|
|
.replaceAll('~/.codex/config.toml', ''),
|
|
).not.toContain('~/.codex/');
|
|
}
|
|
// gstack-upgrade legitimately references .agents/skills for cross-platform detection
|
|
if (skill.dir !== 'gstack-upgrade') {
|
|
expect(content).not.toContain('.agents/skills');
|
|
}
|
|
}
|
|
});
|
|
|
|
// ─── Design outside voices: Codex host guard ─────────────────
|
|
|
|
test('codex host produces empty outside voices in design-review', () => {
|
|
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(codexContent).not.toContain('Design Outside Voices');
|
|
});
|
|
|
|
test('codex host does not include Codex design block in ship', () => {
|
|
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(codexContent).not.toContain('Codex design voice');
|
|
});
|
|
|
|
// ─── Explicit --model override wins over the host default ────
|
|
// Without --model the codex host renders its defaultModel (gpt) — pinned by
|
|
// the golden test. This pins the OTHER direction through the real CLI:
|
|
// `./setup --host codex --model <id>` depends on it. Runs last in this
|
|
// describe and restores the host-default render before finishing.
|
|
test('explicit --model overrides the codex host default', () => {
|
|
try {
|
|
const override = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--model', 'claude'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(override.exitCode).toBe(0);
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Model-Specific Behavioral Patch (claude)');
|
|
expect(content).toContain('MODEL_OVERLAY: claude');
|
|
} finally {
|
|
// Restore the host-default render — later tests and the host-config
|
|
// golden read this tree.
|
|
const restore = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(restore.exitCode).toBe(0);
|
|
}
|
|
const restored = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(restored).toContain('Model-Specific Behavioral Patch (gpt)');
|
|
});
|
|
});
|
|
|
|
// ─── Factory generation tests ────────────────────────────────
|
|
|
|
describe('Factory generation (--host factory)', () => {
|
|
const FACTORY_DIR = path.join(ROOT, '.factory', 'skills');
|
|
|
|
// Generate Factory output for tests
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
|
|
const FACTORY_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; factoryName: string }> = [];
|
|
const isSymlinkLoop = (name: string): boolean => {
|
|
const factorySkillDir = path.join(ROOT, '.factory', 'skills', name);
|
|
try { return fs.realpathSync(factorySkillDir) === fs.realpathSync(ROOT); }
|
|
catch { return false; }
|
|
};
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
if (!isSymlinkLoop('gstack')) skills.push({ dir: '.', factoryName: 'gstack' });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (entry.name === 'codex') continue;
|
|
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
|
const factoryName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
|
if (isSymlinkLoop(factoryName)) continue;
|
|
skills.push({ dir: entry.name, factoryName });
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
test('--host factory generates correct output paths', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const skillMd = path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('Factory frontmatter has name + description + user-invocable', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).toContain('name:');
|
|
expect(frontmatter).toContain('description:');
|
|
expect(frontmatter).toContain('user-invocable: true');
|
|
expect(frontmatter).not.toContain('allowed-tools:');
|
|
expect(frontmatter).not.toContain('preamble-tier:');
|
|
expect(frontmatter).not.toContain('sensitive:');
|
|
}
|
|
});
|
|
|
|
test('sensitive skills have disable-model-invocation', () => {
|
|
const SENSITIVE = ['gstack-ship', 'gstack-land-and-deploy', 'gstack-guard', 'gstack-careful', 'gstack-freeze', 'gstack-unfreeze'];
|
|
for (const name of SENSITIVE) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).toContain('disable-model-invocation: true');
|
|
}
|
|
});
|
|
|
|
test('non-sensitive skills lack disable-model-invocation', () => {
|
|
const NON_SENSITIVE = ['gstack-qa', 'gstack-review', 'gstack-investigate', 'gstack-browse'];
|
|
for (const name of NON_SENSITIVE) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).not.toContain('disable-model-invocation');
|
|
}
|
|
});
|
|
|
|
test('no .claude/skills/ in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('no ~/.claude/skills/ paths in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
// ~/.claude/skills should be rewritten, but ~/.claude/plans is legitimate
|
|
// (plan directory lookup) and ~/.claude/ in codex prompts is intentional
|
|
expect(content).not.toContain('~/.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('/codex skill excluded from Factory output', () => {
|
|
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex'))).toBe(false);
|
|
});
|
|
|
|
test('Factory keeps Codex integration blocks', () => {
|
|
// Factory users CAN use Codex second opinions (codex exec is a standalone binary)
|
|
const shipContent = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(shipContent).toContain('codex');
|
|
});
|
|
|
|
test('no agents/openai.yaml in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const yamlPath = path.join(FACTORY_DIR, skill.factoryName, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(yamlPath)).toBe(false);
|
|
}
|
|
});
|
|
|
|
test('--host droid alias works', () => {
|
|
const factoryResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
const droidResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'droid', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(factoryResult.exitCode).toBe(0);
|
|
expect(droidResult.exitCode).toBe(0);
|
|
expect(factoryResult.stdout.toString()).toBe(droidResult.stdout.toString());
|
|
});
|
|
|
|
test('--host factory --dry-run freshness', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
for (const skill of FACTORY_SKILLS) {
|
|
expect(output).toContain(`FRESH: .factory/skills/${skill.factoryName}/SKILL.md`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('Factory preamble uses .factory paths', () => {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT');
|
|
expect(content).toContain('$_ROOT/.factory/skills/gstack');
|
|
expect(content).toContain('$GSTACK_BIN/gstack-config');
|
|
});
|
|
});
|
|
|
|
// ─── Parameterized host smoke tests (config-driven) ─────────
|
|
|
|
import { ALL_HOST_CONFIGS, getExternalHosts } from '../hosts/index';
|
|
|
|
describe('Parameterized host smoke tests', () => {
|
|
// Regenerate every external host up front so the per-host `--dry-run` freshness
|
|
// checks are deterministic. These host dirs (.agents/.factory/.cursor/...) are
|
|
// gitignored regenerated artifacts, so the freshness check is really an
|
|
// idempotency/determinism check — it still catches non-deterministic gen, but no
|
|
// longer flakes on stale-on-disk state left by a missing `gen --host all` prestep
|
|
// (the canonical `bun test` does not run one). The tracked-claude freshness test
|
|
// (`generated files are fresh`) runs earlier and is unaffected.
|
|
beforeAll(() => {
|
|
for (const h of getExternalHosts()) {
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', h.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
});
|
|
|
|
for (const hostConfig of getExternalHosts()) {
|
|
describe(`${hostConfig.displayName} (--host ${hostConfig.name})`, () => {
|
|
const hostDir = path.join(ROOT, hostConfig.hostSubdir, 'skills');
|
|
|
|
test('generates output that exists on disk', () => {
|
|
// Generated dir should exist (created by earlier bun run gen:skill-docs --host all)
|
|
if (!fs.existsSync(hostDir)) {
|
|
// Generate if not already done
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
expect(fs.existsSync(hostDir)).toBe(true);
|
|
const skills = fs.readdirSync(hostDir).filter(d =>
|
|
fs.existsSync(path.join(hostDir, d, 'SKILL.md'))
|
|
);
|
|
expect(skills.length).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('no .claude/skills path leakage outside repo-root sidecar symlinks', () => {
|
|
if (!fs.existsSync(hostDir)) return; // skip if not generated
|
|
const skills = fs.readdirSync(hostDir);
|
|
for (const skill of skills) {
|
|
// Dev installs may mount the repo root at host/skills/gstack as a runtime
|
|
// sidecar. The generator skips that symlink loop, so leakage checks should too.
|
|
if (isRepoRootSymlink(path.join(hostDir, skill))) continue;
|
|
const skillMd = path.join(hostDir, skill, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
// Strip bash blocks (which have legitimate fallback paths)
|
|
const noBash = content.replace(/```bash\n[\s\S]*?```/g, '');
|
|
const leaks = noBash.split('\n').filter(l => l.includes('.claude/skills'));
|
|
if (leaks.length > 0) {
|
|
throw new Error(`${skill}: .claude/skills leakage:\n${leaks.slice(0, 3).join('\n')}`);
|
|
}
|
|
}
|
|
});
|
|
|
|
test('frontmatter has name and description', () => {
|
|
if (!fs.existsSync(hostDir)) return;
|
|
const skills = fs.readdirSync(hostDir);
|
|
for (const skill of skills) {
|
|
const skillMd = path.join(hostDir, skill, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content).toMatch(/^---\n/);
|
|
expect(content).toMatch(/^name:\s/m);
|
|
expect(content).toMatch(/^description:\s/m);
|
|
}
|
|
});
|
|
|
|
test('--dry-run freshness check passes', () => {
|
|
const result = Bun.spawnSync(
|
|
['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name, '--dry-run'],
|
|
{ cwd: ROOT, stdout: 'pipe', stderr: 'pipe' }
|
|
);
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
if (hostConfig.generation.skipSkills?.includes('codex')) {
|
|
test('/codex skill excluded', () => {
|
|
expect(fs.existsSync(path.join(hostDir, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
});
|
|
}
|
|
});
|
|
}
|
|
});
|
|
|
|
// ─── --host all tests ────────────────────────────────────────
|
|
|
|
describe('--host all', () => {
|
|
// Same determinism guard as the parameterized block: make external hosts fresh on
|
|
// disk so `--host all --dry-run` reports FRESH regardless of prior state.
|
|
beforeAll(() => {
|
|
for (const h of getExternalHosts()) {
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', h.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
});
|
|
|
|
test('--host all generates for all registered hosts', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'all', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// All hosts should appear in output
|
|
expect(output).toContain('FRESH: SKILL.md'); // claude
|
|
for (const hostConfig of getExternalHosts()) {
|
|
expect(output).toContain(`FRESH: ${hostConfig.hostSubdir}/skills/`);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── Setup script validation ─────────────────────────────────
|
|
// These tests verify the setup script's install layout matches
|
|
// what the generator produces — catching the bug where setup
|
|
// installed Claude-format source dirs for Codex users.
|
|
|
|
describe('setup script validation', () => {
|
|
const setupContent = fs.readFileSync(path.join(ROOT, 'setup'), 'utf-8');
|
|
|
|
test('setup has separate link functions for Claude and Codex', () => {
|
|
expect(setupContent).toContain('link_claude_skill_dirs');
|
|
expect(setupContent).toContain('link_codex_skill_dirs');
|
|
// Old unified function must not exist
|
|
expect(setupContent).not.toMatch(/^link_skill_dirs\(\)/m);
|
|
});
|
|
|
|
test('Claude install uses link_claude_skill_dirs', () => {
|
|
// The Claude install section (section 4) should use the Claude function
|
|
const claudeSection = setupContent.slice(
|
|
setupContent.indexOf('# 4. Install for Claude'),
|
|
setupContent.indexOf('# 5. Install for Codex')
|
|
);
|
|
expect(claudeSection).toContain('link_claude_skill_dirs');
|
|
expect(claudeSection).not.toContain('link_codex_skill_dirs');
|
|
});
|
|
|
|
test('Codex install uses link_codex_skill_dirs', () => {
|
|
// The Codex install section (section 5) should use the Codex function
|
|
// End marker: the next numbered section header (a marker that doesn't
|
|
// exist slices to EOF and the assertion reads unrelated sections).
|
|
const codexSection = setupContent.slice(
|
|
setupContent.indexOf('# 5. Install for Codex'),
|
|
setupContent.indexOf('# 6. Install for Kiro')
|
|
);
|
|
expect(setupContent.indexOf('# 6. Install for Kiro')).toBeGreaterThan(-1);
|
|
expect(codexSection).toContain('create_codex_runtime_root');
|
|
expect(codexSection).toContain('link_codex_skill_dirs');
|
|
expect(codexSection).not.toContain('link_claude_skill_dirs');
|
|
expect(codexSection).not.toContain('_link_or_copy "$GSTACK_DIR" "$CODEX_GSTACK"');
|
|
});
|
|
|
|
test('Codex install prefers repo-local .agents/skills when setup runs from there', () => {
|
|
expect(setupContent).toContain('SKILLS_PARENT_BASENAME');
|
|
expect(setupContent).toContain('CODEX_REPO_LOCAL=0');
|
|
expect(setupContent).toContain('[ "$SKILLS_PARENT_BASENAME" = ".agents" ]');
|
|
expect(setupContent).toContain('CODEX_REPO_LOCAL=1');
|
|
expect(setupContent).toContain('CODEX_SKILLS="$INSTALL_SKILLS_DIR"');
|
|
});
|
|
|
|
test('setup separates install path from source path for symlinked repo-local installs', () => {
|
|
expect(setupContent).toContain('INSTALL_GSTACK_DIR=');
|
|
expect(setupContent).toContain('SOURCE_GSTACK_DIR=');
|
|
expect(setupContent).toContain('INSTALL_SKILLS_DIR=');
|
|
expect(setupContent).toContain('CODEX_GSTACK="$INSTALL_GSTACK_DIR"');
|
|
expect(setupContent).toContain('link_codex_skill_dirs "$SOURCE_GSTACK_DIR" "$CODEX_SKILLS"');
|
|
});
|
|
|
|
test('Codex installs always create sidecar runtime assets for the real skill target', () => {
|
|
expect(setupContent).toContain('if [ "$INSTALL_CODEX" -eq 1 ]; then');
|
|
expect(setupContent).toContain('create_agents_sidecar "$SOURCE_GSTACK_DIR"');
|
|
});
|
|
|
|
test('link_codex_skill_dirs reads from .agents/skills/', () => {
|
|
// The Codex link function must reference .agents/skills for generated Codex skills
|
|
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('.agents/skills');
|
|
expect(fnBody).toContain('gstack*');
|
|
});
|
|
|
|
test('link_claude_skill_dirs creates real directories with absolute SKILL.md symlinks', () => {
|
|
// Claude links should be real directories with absolute SKILL.md symlinks
|
|
// to ensure Claude Code discovers them as top-level skills (not nested under gstack/)
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('mkdir -p "$target"');
|
|
// v1.36.0.0: routes through _link_or_copy helper for Windows fallback (cp on MSYS2/Git Bash).
|
|
// v1.67 (#2569): the source is render-aware — canonical SKILL.md, or the
|
|
// rendered :user variant from ${GSTACK_HOME}/render/claude when present.
|
|
expect(fnBody).toContain('_skill_md_src="$gstack_dir/$dir_name/SKILL.md"');
|
|
expect(fnBody).toContain('_link_or_copy "$_skill_md_src" "$target/SKILL.md"');
|
|
});
|
|
|
|
// REGRESSION: cleanup functions must handle both old symlinks AND new real-directory pattern
|
|
test('cleanup functions handle real directories with symlinked SKILL.md', () => {
|
|
// cleanup_old_claude_symlinks must detect and remove real dirs with SKILL.md symlinks
|
|
const cleanupOldStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
|
|
const cleanupOldEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up old', cleanupOldStart));
|
|
const cleanupOldBody = setupContent.slice(cleanupOldStart, cleanupOldEnd);
|
|
expect(cleanupOldBody).toContain('-d "$old_target"');
|
|
expect(cleanupOldBody).toContain('-L "$old_target/SKILL.md"');
|
|
expect(cleanupOldBody).toContain('rm -rf "$old_target"');
|
|
|
|
// cleanup_prefixed_claude_symlinks must also handle the new pattern
|
|
const cleanupPrefixedStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
|
|
const cleanupPrefixedEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up prefixed', cleanupPrefixedStart));
|
|
const cleanupPrefixedBody = setupContent.slice(cleanupPrefixedStart, cleanupPrefixedEnd);
|
|
expect(cleanupPrefixedBody).toContain('-d "$prefixed_target"');
|
|
expect(cleanupPrefixedBody).toContain('-L "$prefixed_target/SKILL.md"');
|
|
expect(cleanupPrefixedBody).toContain('rm -rf "$prefixed_target"');
|
|
});
|
|
|
|
// REGRESSION: link function must upgrade old directory symlinks
|
|
test('link_claude_skill_dirs removes old directory symlinks before creating real dirs', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// Must check for and remove old symlinks before mkdir
|
|
expect(fnBody).toContain('if [ -L "$target" ]');
|
|
expect(fnBody).toContain('rm -f "$target"');
|
|
});
|
|
|
|
test('setup links root gstack skill through a thin Claude wrapper alias', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_root_skill_alias()');
|
|
const fnEnd = setupContent.indexOf('# ─── Helper: remove old unprefixed Claude skill entries', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('_gstack-command');
|
|
// #2511: the alias must be a rewritten COPY (unique frontmatter name),
|
|
// never a verbatim symlink of the canonical SKILL.md.
|
|
expect(fnBody).toContain('_install_alias_skill_md "$gstack_dir/SKILL.md" "$target" "_gstack-command"');
|
|
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir/SKILL.md"');
|
|
|
|
const claudeSection = setupContent.slice(
|
|
setupContent.indexOf('# 4. Install for Claude'),
|
|
setupContent.indexOf('# 5. Install for Codex')
|
|
);
|
|
expect(claudeSection).toContain('link_claude_root_skill_alias "$SOURCE_GSTACK_DIR" "$INSTALL_SKILLS_DIR"');
|
|
});
|
|
|
|
test('setup supports --host auto|claude|codex|kiro|opencode|cursor; slate is informational', () => {
|
|
expect(setupContent).toContain('--host');
|
|
// #2361: slate moved OUT of the install accept-list (it was accepted but
|
|
// never dispatched — a silent exit-0 no-op) into an informational arm.
|
|
expect(setupContent).toContain('claude|codex|kiro|factory|opencode|cursor|auto');
|
|
expect(setupContent).toMatch(/^ {2}slate\)/m);
|
|
});
|
|
|
|
test('auto mode detects claude, codex, kiro, and opencode binaries', () => {
|
|
expect(setupContent).toContain('command -v claude');
|
|
expect(setupContent).toContain('command -v codex');
|
|
expect(setupContent).toContain('command -v kiro-cli');
|
|
expect(setupContent).toContain('command -v opencode');
|
|
});
|
|
|
|
// T1: Sidecar skip guard — prevents .agents/skills/gstack from being linked as a skill
|
|
test('link_codex_skill_dirs skips the gstack sidecar directory', () => {
|
|
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
|
|
});
|
|
|
|
// T2: Dynamic $GSTACK_ROOT paths in generated Codex preambles
|
|
test('generated Codex preambles use dynamic GSTACK_ROOT paths', () => {
|
|
const codexSkillDir = path.join(ROOT, '.agents', 'skills', 'gstack-ship');
|
|
if (!fs.existsSync(codexSkillDir)) return; // skip if .agents/ not generated
|
|
const content = fs.readFileSync(path.join(codexSkillDir, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT=');
|
|
expect(content).toContain('$GSTACK_BIN/');
|
|
});
|
|
|
|
test('setup supports --host kiro with install section and sed rewrites', () => {
|
|
expect(setupContent).toContain('INSTALL_KIRO=');
|
|
expect(setupContent).toContain('kiro-cli');
|
|
expect(setupContent).toContain('KIRO_SKILLS=');
|
|
expect(setupContent).toContain('~/.kiro/skills/gstack');
|
|
expect(setupContent).toContain('$KIRO_GSTACK/lib');
|
|
});
|
|
|
|
test('setup supports --host opencode with install section and OpenCode skill path vars', () => {
|
|
expect(setupContent).toContain('INSTALL_OPENCODE=');
|
|
expect(setupContent).toContain('OPENCODE_SKILLS="$HOME/.config/opencode/skills"');
|
|
expect(setupContent).toContain('OPENCODE_GSTACK="$OPENCODE_SKILLS/gstack"');
|
|
});
|
|
|
|
// --host cursor full install slice (#1358, PR #2547 by @szsunyuan re-derived)
|
|
test('auto mode detects Cursor via binary or ~/.cursor directory', () => {
|
|
expect(setupContent).toContain('command -v cursor');
|
|
expect(setupContent).toContain('[ -d "$HOME/.cursor" ] && INSTALL_CURSOR=1');
|
|
});
|
|
|
|
test('setup supports --host cursor with install section and Cursor skill path vars', () => {
|
|
expect(setupContent).toContain('INSTALL_CURSOR=');
|
|
expect(setupContent).toContain('CURSOR_SKILLS="$HOME/.cursor/skills"');
|
|
expect(setupContent).toContain('CURSOR_GSTACK="$CURSOR_SKILLS/gstack"');
|
|
expect(setupContent).toContain('create_cursor_runtime_root');
|
|
expect(setupContent).toContain('create_cursor_sidecar');
|
|
expect(setupContent).toContain('link_cursor_skill_dirs');
|
|
expect(setupContent).toContain('gstack ready (cursor).');
|
|
});
|
|
|
|
test('create_cursor_runtime_root exposes only Cursor runtime assets', () => {
|
|
const fnStart = setupContent.indexOf('create_cursor_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('create_cursor_sidecar()', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('gstack/SKILL.md');
|
|
expect(fnBody).toContain('browse/dist');
|
|
expect(fnBody).toContain('browse/bin');
|
|
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
|
|
expect(fnBody).toContain('checklist.md');
|
|
expect(fnBody).toContain('TODOS-format.md');
|
|
// bin scripts import ../lib — the two must travel together.
|
|
expect(fnBody).toContain('$cursor_gstack/lib');
|
|
expect(fnBody).not.toContain('design-checklist.md');
|
|
expect(fnBody).not.toContain('greptile-triage.md');
|
|
expect(fnBody).not.toContain('review/specialists');
|
|
expect(fnBody).not.toContain('qa/templates');
|
|
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$cursor_gstack"');
|
|
});
|
|
|
|
test('create_cursor_sidecar plants runtime assets without wiping generated SKILL.md', () => {
|
|
const fnStart = setupContent.indexOf('create_cursor_sidecar()');
|
|
const fnEnd = setupContent.indexOf('link_cursor_skill_dirs()', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('.cursor/skills/gstack');
|
|
expect(fnBody).toContain('bin');
|
|
expect(fnBody).toContain('browse/dist');
|
|
expect(fnBody).toContain('browse/bin');
|
|
expect(fnBody).toContain('ETHOS.md');
|
|
expect(fnBody).not.toContain('rm -rf');
|
|
});
|
|
|
|
test('link_cursor_skill_dirs skips the gstack runtime root directory', () => {
|
|
const fnStart = setupContent.indexOf('link_cursor_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
|
|
// #2444-aware guard: Windows bypass, else only replace symlink-or-missing.
|
|
expect(fnBody).toContain('[ "$IS_WINDOWS" -eq 1 ] || [ -L "$target" ] || [ ! -e "$target" ]');
|
|
});
|
|
|
|
// #2142 deleted existing ~/.cursor/skills/<name> dirs with `rm -rf "$target"`
|
|
// before relinking. That can wipe unowned Cursor skills. Only replace a
|
|
// symlink or a missing path; never the whole skills directory.
|
|
test('link_cursor_skill_dirs does not delete unowned Cursor skill directories', () => {
|
|
const fnStart = setupContent.indexOf('link_cursor_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).not.toContain('rm -rf "$target"');
|
|
expect(fnBody).not.toContain('rm -rf "$skills_dir"');
|
|
expect(setupContent).not.toContain('rm -rf "$CURSOR_SKILLS"');
|
|
});
|
|
|
|
test('Cursor install links generated skills before planting the sidecar', () => {
|
|
const cursorInstall = setupContent.slice(
|
|
setupContent.indexOf('# 6d. Install for Cursor'),
|
|
setupContent.indexOf('# 7. Create .agents/ sidecar'),
|
|
);
|
|
const linkCall = cursorInstall.indexOf('link_cursor_skill_dirs "$SOURCE_GSTACK_DIR"');
|
|
const sidecarCall = cursorInstall.indexOf('create_cursor_sidecar "$SOURCE_GSTACK_DIR"');
|
|
expect(linkCall).toBeGreaterThan(-1);
|
|
expect(sidecarCall).toBeGreaterThan(-1);
|
|
expect(linkCall).toBeLessThan(sidecarCall);
|
|
});
|
|
|
|
test('setup installs OpenCode skills into a nested gstack runtime root', () => {
|
|
expect(setupContent).toContain('create_opencode_runtime_root');
|
|
expect(setupContent).toContain('.opencode/skills');
|
|
expect(setupContent).toContain('review/specialists');
|
|
expect(setupContent).toContain('qa/templates');
|
|
expect(setupContent).toContain('qa/references');
|
|
expect(setupContent).toContain('dx-hall-of-fame.md');
|
|
expect(setupContent).toContain('$opencode_gstack/lib');
|
|
});
|
|
|
|
test('create_agents_sidecar links runtime assets', () => {
|
|
// Sidecar must link bin with its shared lib modules, plus browse, review, qa
|
|
const fnStart = setupContent.indexOf('create_agents_sidecar()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('bin');
|
|
expect(fnBody).toContain('lib');
|
|
expect(fnBody).toContain('browse');
|
|
expect(fnBody).toContain('review');
|
|
expect(fnBody).toContain('qa');
|
|
});
|
|
|
|
test('create_codex_runtime_root exposes only runtime assets', () => {
|
|
const fnStart = setupContent.indexOf('create_codex_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', setupContent.indexOf('review/', fnStart)));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('gstack/SKILL.md');
|
|
expect(fnBody).toContain('$codex_gstack/lib');
|
|
expect(fnBody).toContain('browse/dist');
|
|
expect(fnBody).toContain('browse/bin');
|
|
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
|
|
// Review runtime assets (individual files, not the whole dir)
|
|
expect(fnBody).toContain('checklist.md');
|
|
expect(fnBody).toContain('design-checklist.md');
|
|
expect(fnBody).toContain('greptile-triage.md');
|
|
expect(fnBody).toContain('TODOS-format.md');
|
|
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$codex_gstack"');
|
|
});
|
|
|
|
test('create_factory_runtime_root links shared lib modules beside bin', () => {
|
|
const fnStart = setupContent.indexOf('create_factory_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('create_opencode_runtime_root()', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('$factory_gstack/bin');
|
|
expect(fnBody).toContain('$factory_gstack/lib');
|
|
});
|
|
|
|
test('direct Codex installs are migrated out of ~/.codex/skills/gstack', () => {
|
|
expect(setupContent).toContain('migrate_direct_codex_install');
|
|
expect(setupContent).toContain('$HOME/.gstack/repos/gstack');
|
|
expect(setupContent).toContain('avoid duplicate skill discovery');
|
|
});
|
|
|
|
// --- Symlink prefix tests (PR #503) ---
|
|
|
|
test('link_claude_skill_dirs applies gstack- prefix by default', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('SKILL_PREFIX');
|
|
expect(fnBody).toContain('link_name="gstack-$skill_name"');
|
|
});
|
|
|
|
test('link_claude_skill_dirs preserves already-prefixed dirs', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// gstack-* dirs should keep their name (e.g., gstack-upgrade stays gstack-upgrade)
|
|
expect(fnBody).toContain('gstack-*) link_name="$skill_name"');
|
|
});
|
|
|
|
test('setup supports --no-prefix flag', () => {
|
|
expect(setupContent).toContain('--no-prefix');
|
|
expect(setupContent).toContain('SKILL_PREFIX=0');
|
|
});
|
|
|
|
test('cleanup_old_claude_symlinks removes only gstack-pointing symlinks', () => {
|
|
expect(setupContent).toContain('cleanup_old_claude_symlinks');
|
|
const fnStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// Should check readlink before removing
|
|
expect(fnBody).toContain('readlink');
|
|
expect(fnBody).toContain('gstack/*');
|
|
// Should skip already-prefixed dirs
|
|
expect(fnBody).toContain('gstack-*) continue');
|
|
});
|
|
|
|
test('cleanup runs before link when prefix is enabled', () => {
|
|
// In the Claude install section, cleanup should happen before linking
|
|
const claudeInstallSection = setupContent.slice(
|
|
setupContent.indexOf('INSTALL_CLAUDE'),
|
|
setupContent.lastIndexOf('link_claude_skill_dirs')
|
|
);
|
|
expect(claudeInstallSection).toContain('cleanup_old_claude_symlinks');
|
|
});
|
|
|
|
// --- Persistent config + interactive prompt tests ---
|
|
|
|
test('setup reads skill_prefix from config', () => {
|
|
expect(setupContent).toContain('get skill_prefix');
|
|
expect(setupContent).toContain('GSTACK_CONFIG');
|
|
});
|
|
|
|
test('setup supports --prefix flag', () => {
|
|
expect(setupContent).toContain('--prefix)');
|
|
expect(setupContent).toContain('SKILL_PREFIX=1; SKILL_PREFIX_FLAG=1');
|
|
});
|
|
|
|
test('--prefix and --no-prefix persist to config', () => {
|
|
expect(setupContent).toContain('set skill_prefix');
|
|
});
|
|
|
|
test('interactive prompt shows when no config', () => {
|
|
expect(setupContent).toContain('Short names');
|
|
expect(setupContent).toContain('Namespaced');
|
|
expect(setupContent).toContain('Choice [1/2]');
|
|
});
|
|
|
|
test('non-TTY defaults to flat names', () => {
|
|
// Should check if stdin is a TTY before prompting
|
|
expect(setupContent).toContain('-t 0');
|
|
});
|
|
|
|
test('cleanup_prefixed_claude_symlinks exists and uses readlink', () => {
|
|
expect(setupContent).toContain('cleanup_prefixed_claude_symlinks');
|
|
const fnStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('readlink');
|
|
expect(fnBody).toContain('gstack-$skill_name');
|
|
});
|
|
|
|
test('reverse cleanup runs before link when prefix is disabled', () => {
|
|
const claudeInstallSection = setupContent.slice(
|
|
setupContent.indexOf('INSTALL_CLAUDE'),
|
|
setupContent.lastIndexOf('link_claude_skill_dirs')
|
|
);
|
|
expect(claudeInstallSection).toContain('cleanup_prefixed_claude_symlinks');
|
|
});
|
|
|
|
test('welcome message references SKILL_PREFIX', () => {
|
|
// gstack-upgrade is always called gstack-upgrade (it's the actual dir name)
|
|
// but the welcome section should exist near the prefix logic
|
|
expect(setupContent).toContain('Run /gstack-upgrade anytime');
|
|
});
|
|
});
|
|
|
|
describe('discover-skills hidden directory filtering', () => {
|
|
test('discoverTemplates skips dot-prefixed directories', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-discover-'));
|
|
try {
|
|
// Create a hidden dir with a template (should be excluded)
|
|
fs.mkdirSync(path.join(tmpDir, '.hidden'), { recursive: true });
|
|
fs.writeFileSync(path.join(tmpDir, '.hidden', 'SKILL.md.tmpl'), '---\nname: evil\n---\ntest');
|
|
// Create a visible dir with a template (should be included)
|
|
fs.mkdirSync(path.join(tmpDir, 'visible'), { recursive: true });
|
|
fs.writeFileSync(path.join(tmpDir, 'visible', 'SKILL.md.tmpl'), '---\nname: good\n---\ntest');
|
|
|
|
const { discoverTemplates } = require('../scripts/discover-skills');
|
|
const results = discoverTemplates(tmpDir);
|
|
const dirs = results.map((r: { tmpl: string }) => r.tmpl);
|
|
|
|
expect(dirs).toContain('visible/SKILL.md.tmpl');
|
|
expect(dirs).not.toContain('.hidden/SKILL.md.tmpl');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('telemetry', () => {
|
|
test('generated SKILL.md contains telemetry start block', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_TEL_START');
|
|
expect(content).toContain('_SESSION_ID');
|
|
expect(content).toContain('TELEMETRY:');
|
|
expect(content).toContain('TEL_PROMPTED:');
|
|
expect(content).toContain('gstack-config get telemetry');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry opt-in prompt', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.telemetry-prompted');
|
|
expect(content).toContain('Help gstack get better');
|
|
expect(content).toContain('gstack-config set telemetry community');
|
|
expect(content).toContain('gstack-config set telemetry anonymous');
|
|
expect(content).toContain('gstack-config set telemetry off');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry epilogue', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Telemetry (run last)');
|
|
expect(content).toContain('gstack-telemetry-log');
|
|
expect(content).toContain('_TEL_END');
|
|
expect(content).toContain('_TEL_DUR');
|
|
expect(content).toContain('SKILL_NAME');
|
|
expect(content).toContain('OUTCOME');
|
|
expect(content).toContain('PLAN MODE EXCEPTION');
|
|
});
|
|
|
|
test('generated SKILL.md contains pending marker handling', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.pending');
|
|
expect(content).toContain('_pending_finalize');
|
|
});
|
|
|
|
test('telemetry blocks appear in all skill files that use PREAMBLE', () => {
|
|
const skills = ['qa', 'ship', 'review', 'plan-ceo-review', 'plan-eng-review', 'retro'];
|
|
for (const skill of skills) {
|
|
const skillPath = path.join(ROOT, skill, 'SKILL.md');
|
|
if (fs.existsSync(skillPath)) {
|
|
const content = fs.readFileSync(skillPath, 'utf-8');
|
|
expect(content).toContain('_TEL_START');
|
|
expect(content).toContain('Telemetry (run last)');
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('community fixes wave', () => {
|
|
// Helper to get all generated SKILL.md files
|
|
function getAllSkillMds(): Array<{ name: string; content: string }> {
|
|
const results: Array<{ name: string; content: string }> = [];
|
|
const rootPath = path.join(ROOT, 'SKILL.md');
|
|
if (fs.existsSync(rootPath)) {
|
|
results.push({ name: 'root', content: fs.readFileSync(rootPath, 'utf-8') });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
const skillPath = path.join(ROOT, entry.name, 'SKILL.md');
|
|
if (fs.existsSync(skillPath)) {
|
|
results.push({ name: entry.name, content: fs.readFileSync(skillPath, 'utf-8') });
|
|
}
|
|
}
|
|
return results;
|
|
}
|
|
|
|
// #594 — Discoverability: every SKILL.md.tmpl description contains "gstack"
|
|
test('every SKILL.md.tmpl description contains "gstack"', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
const content = fs.readFileSync(tmplPath, 'utf-8');
|
|
const desc = extractDescription(content);
|
|
expect(desc.toLowerCase()).toContain('gstack');
|
|
}
|
|
});
|
|
|
|
// #594 — Discoverability: first line of each description is under 120 chars
|
|
test('every SKILL.md.tmpl description first line is under 120 chars', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
const content = fs.readFileSync(tmplPath, 'utf-8');
|
|
const desc = extractDescription(content);
|
|
const firstLine = desc.split('\n')[0];
|
|
expect(firstLine.length).toBeLessThanOrEqual(120);
|
|
}
|
|
});
|
|
|
|
// #573 — Feature signals: ship/SKILL.md contains feature signal detection
|
|
test('ship/SKILL.md contains feature signal detection in Step 4', () => {
|
|
const content = readShipUnion();
|
|
expect(content.toLowerCase()).toContain('feature signal');
|
|
});
|
|
|
|
// #510 — Context warnings: no SKILL.md contains "running low on context"
|
|
test('no generated SKILL.md contains "running low on context"', () => {
|
|
const skills = getAllSkillMds();
|
|
for (const { name, content } of skills) {
|
|
expect(content).not.toContain('running low on context');
|
|
}
|
|
});
|
|
|
|
// #510 — Context warnings: plan-eng-review has explicit anti-warning
|
|
test('plan-eng-review/SKILL.md contains "Do not preemptively warn"', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('Do not preemptively warn');
|
|
});
|
|
|
|
// #474 — Safety Net: no SKILL.md uses find with -delete
|
|
test('no generated SKILL.md contains find with -delete flag', () => {
|
|
const skills = getAllSkillMds();
|
|
for (const { name, content } of skills) {
|
|
// Match find commands that use -delete (but not prose mentioning the word "delete")
|
|
const lines = content.split('\n');
|
|
for (const line of lines) {
|
|
if (line.includes('find ') && line.includes('-delete')) {
|
|
throw new Error(`${name}/SKILL.md contains find with -delete: ${line.trim()}`);
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
// #467 — Telemetry: preamble JSONL writes are gated by telemetry setting
|
|
test('preamble JSONL writes are inside telemetry conditional', () => {
|
|
const preamble = fs.readFileSync(path.join(ROOT, 'scripts/resolvers/preamble.ts'), 'utf-8');
|
|
// Find all skill-usage.jsonl write lines
|
|
const lines = preamble.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
if (lines[i].includes('skill-usage.jsonl') && lines[i].includes('>>')) {
|
|
// Look backwards for a telemetry conditional within 5 lines
|
|
let foundConditional = false;
|
|
for (let j = i - 1; j >= Math.max(0, i - 5); j--) {
|
|
if (lines[j].includes('_TEL') && lines[j].includes('off')) {
|
|
foundConditional = true;
|
|
break;
|
|
}
|
|
}
|
|
expect(foundConditional).toBe(true);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('codex commands must not use inline $(git rev-parse --show-toplevel) for cwd', () => {
|
|
// Regression test: inline $(git rev-parse --show-toplevel) in codex exec -C
|
|
// or codex review without cd evaluates in whatever cwd the background shell
|
|
// inherits, which may be a different project in Conductor workspaces.
|
|
// The fix is to resolve _REPO_ROOT eagerly at the top of each bash block.
|
|
|
|
// Scan all source files that could contain codex commands
|
|
// Use Bun.Glob to avoid ELOOP from .claude/skills/gstack symlink back to ROOT
|
|
const tmplGlob = new Bun.Glob('**/*.tmpl');
|
|
const sourceFiles = [
|
|
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
|
|
.filter(f => f.endsWith('.ts'))
|
|
.map(f => `scripts/resolvers/${f}`),
|
|
'scripts/gen-skill-docs.ts',
|
|
];
|
|
|
|
test('no codex exec command uses inline $(git rev-parse --show-toplevel) in -C flag', () => {
|
|
const violations: string[] = [];
|
|
for (const rel of sourceFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1}`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('no generated SKILL.md has codex exec with inline $(git rev-parse --show-toplevel) in -C flag', () => {
|
|
const violations: string[] = [];
|
|
const skillMdGlob = new Bun.Glob('**/SKILL.md');
|
|
const skillMdFiles = Array.from(skillMdGlob.scanSync({ cwd: ROOT, followSymlinks: false }));
|
|
for (const rel of skillMdFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1}`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('codex review commands must be preceded by cd "$_REPO_ROOT" (no -C support)', () => {
|
|
// codex review does not support -C, so the pattern must be:
|
|
// _REPO_ROOT=$(git rev-parse --show-toplevel) || { ... }
|
|
// cd "$_REPO_ROOT"
|
|
// codex review ...
|
|
// NOT: codex review ... with inline $(git rev-parse --show-toplevel)
|
|
const allFiles = [
|
|
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...Array.from(new Bun.Glob('**/SKILL.md').scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
|
|
.filter(f => f.endsWith('.ts'))
|
|
.map(f => `scripts/resolvers/${f}`),
|
|
'scripts/gen-skill-docs.ts',
|
|
];
|
|
const violations: string[] = [];
|
|
for (const rel of allFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
// Skip non-executable lines (markdown table cells, prose references)
|
|
if (line.includes('|') && line.includes('`/codex review`')) continue;
|
|
if (line.includes('`codex review`')) continue;
|
|
// Check for codex review with inline $(git rev-parse)
|
|
if (line.includes('codex review') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1} — inline git rev-parse in codex review`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('codex review commands take their scope from a flag, never from prompt text', () => {
|
|
// `codex review` scope comes ONLY from --base/--commit/--uncommitted. The
|
|
// positional [PROMPT] is mutually exclusive with all three (#1428, #1479),
|
|
// and a prompt-only `codex review` silently falls back to the *uncommitted
|
|
// working-tree* scope (`git status --short; git diff`) — so describing the
|
|
// diff range in prompt text produces a confident review of the wrong
|
|
// changes, with no error. Both halves are pinned here:
|
|
// (a) every `codex review` invocation carries a scope flag, and
|
|
// (b) no invocation puts a positional prompt in front of that flag.
|
|
//
|
|
// This does NOT apply to `codex exec`, which is agentic and really does run
|
|
// the git command it's told to — the adversarial pass legitimately scopes
|
|
// itself in prompt text.
|
|
const checkedFiles = [
|
|
'scripts/resolvers/review.ts',
|
|
'review/SKILL.md',
|
|
'ship/SKILL.md',
|
|
'codex/SKILL.md.tmpl',
|
|
'codex/SKILL.md',
|
|
];
|
|
|
|
const violations: string[] = [];
|
|
for (const rel of checkedFiles) {
|
|
// ship's codex/adversarial command moved into sections/adversarial.md (T9 carve).
|
|
const content = rel === 'ship/SKILL.md' ? readShipUnion() : fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
// Only inspect real shell invocations, not prose mentioning the command.
|
|
if (line.includes('`codex review`')) continue;
|
|
const match = line.match(/(?:^|[;&|]\s*|\s)codex\s+review\b(.*)$/);
|
|
if (!match) continue;
|
|
const rest = match[1];
|
|
const scopeFlag = /--base\b|--commit\b|--uncommitted\b/;
|
|
if (!scopeFlag.test(rest)) {
|
|
// A quoted prompt with no scope flag is the silent-wrong-scope bug.
|
|
if (/^\s*["'$]/.test(rest)) {
|
|
violations.push(`${rel}:${i + 1} — prompt-only codex review (falls back to working-tree scope)`);
|
|
}
|
|
continue;
|
|
}
|
|
const beforeFlag = rest.split(scopeFlag)[0].trim();
|
|
if (/^["'$]|^--\s*["']/.test(beforeFlag)) {
|
|
violations.push(`${rel}:${i + 1} — positional prompt passed alongside a scope flag`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
});
|
|
|
|
// ─── Learnings + Confidence Resolver Tests ─────────────────────
|
|
|
|
describe('LEARNINGS_SEARCH resolver', () => {
|
|
const SEARCH_SKILLS = ['review', 'ship', 'plan-eng-review', 'investigate', 'office-hours', 'plan-ceo-review'];
|
|
|
|
for (const skill of SEARCH_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains learnings search`, () => {
|
|
const content = readSkillUnion(skill); // ship: moved to sections/plan-completion.md
|
|
expect(content).toContain('Prior Learnings');
|
|
expect(content).toContain('gstack-learnings-search');
|
|
});
|
|
}
|
|
|
|
test('learnings search includes cross-project config check', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('cross_project_learnings');
|
|
expect(content).toContain('--cross-project');
|
|
});
|
|
|
|
test('learnings search includes AskUserQuestion for first-time cross-project opt-in', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Enable cross-project learnings');
|
|
expect(content).toContain('project-scoped only');
|
|
});
|
|
|
|
test('learnings search mentions prior learning applied display format', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Prior learning applied');
|
|
});
|
|
});
|
|
|
|
describe('LEARNINGS_LOG resolver', () => {
|
|
const LOG_SKILLS = ['review', 'retro', 'investigate'];
|
|
|
|
for (const skill of LOG_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains learnings log`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Capture Learnings');
|
|
expect(content).toContain('gstack-learnings-log');
|
|
});
|
|
}
|
|
|
|
test('learnings log documents all type values', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
for (const type of ['pattern', 'pitfall', 'preference', 'architecture', 'tool']) {
|
|
expect(content).toContain(type);
|
|
}
|
|
});
|
|
|
|
test('learnings log documents all source values', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
for (const source of ['observed', 'user-stated', 'inferred', 'cross-model']) {
|
|
expect(content).toContain(source);
|
|
}
|
|
});
|
|
|
|
test('learnings log includes files field for staleness detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"files"');
|
|
expect(content).toContain('staleness detection');
|
|
});
|
|
});
|
|
|
|
describe('CONFIDENCE_CALIBRATION resolver', () => {
|
|
const CONFIDENCE_SKILLS = ['review', 'ship', 'plan-eng-review', 'cso'];
|
|
|
|
for (const skill of CONFIDENCE_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains confidence calibration`, () => {
|
|
const content = readSkillUnion(skill); // ship: moved to sections/review-army.md
|
|
expect(content).toContain('Confidence Calibration');
|
|
expect(content).toContain('confidence score');
|
|
});
|
|
}
|
|
|
|
test('confidence calibration includes scoring rubric with all tiers', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('9-10');
|
|
expect(content).toContain('7-8');
|
|
expect(content).toContain('5-6');
|
|
expect(content).toContain('3-4');
|
|
expect(content).toContain('1-2');
|
|
});
|
|
|
|
test('confidence calibration includes display rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Show normally');
|
|
expect(content).toContain('Suppress from main report');
|
|
});
|
|
|
|
test('confidence calibration includes finding format example', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('[P1] (confidence:');
|
|
expect(content).toContain('SQL injection');
|
|
});
|
|
|
|
test('confidence calibration includes calibration learning feedback loop', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('calibration event');
|
|
expect(content).toContain('Log the corrected pattern');
|
|
});
|
|
|
|
test('skills without confidence calibration do NOT contain it', () => {
|
|
// office-hours and retro do NOT use confidence calibration
|
|
for (const skill of ['office-hours', 'retro']) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('## Confidence Calibration');
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('gen-skill-docs prefix warning (#620/#578)', () => {
|
|
const { execSync } = require('child_process');
|
|
|
|
test('warns about skill_prefix when config has prefix=true', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
|
|
try {
|
|
// Create a fake ~/.gstack/config.yaml with skill_prefix: true
|
|
const fakeHome = tmpDir;
|
|
const fakeGstack = path.join(fakeHome, '.gstack');
|
|
fs.mkdirSync(fakeGstack, { recursive: true });
|
|
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: true\n');
|
|
|
|
const output = execSync('bun run scripts/gen-skill-docs.ts', {
|
|
cwd: ROOT,
|
|
env: { ...process.env, HOME: fakeHome },
|
|
encoding: 'utf-8',
|
|
timeout: 30000,
|
|
});
|
|
expect(output).toContain('skill_prefix is true');
|
|
expect(output).toContain('gstack-relink');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
test('no warning when skill_prefix is false or absent', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
|
|
try {
|
|
const fakeHome = tmpDir;
|
|
const fakeGstack = path.join(fakeHome, '.gstack');
|
|
fs.mkdirSync(fakeGstack, { recursive: true });
|
|
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: false\n');
|
|
|
|
const output = execSync('bun run scripts/gen-skill-docs.ts', {
|
|
cwd: ROOT,
|
|
env: { ...process.env, HOME: fakeHome },
|
|
encoding: 'utf-8',
|
|
timeout: 30000,
|
|
});
|
|
expect(output).not.toContain('skill_prefix is true');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('voice-triggers processing', () => {
|
|
const { extractVoiceTriggers, processVoiceTriggers } = require('../scripts/gen-skill-docs') as {
|
|
extractVoiceTriggers: (content: string) => string[];
|
|
processVoiceTriggers: (content: string) => string;
|
|
};
|
|
|
|
test('extractVoiceTriggers parses valid YAML list', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit.\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
|
|
const triggers = extractVoiceTriggers(content);
|
|
expect(triggers).toEqual(['see-so', 'security review']);
|
|
});
|
|
|
|
test('extractVoiceTriggers returns [] when no field present', () => {
|
|
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
|
|
expect(extractVoiceTriggers(content)).toEqual([]);
|
|
});
|
|
|
|
test('processVoiceTriggers appends voice triggers to description', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
|
|
const result = processVoiceTriggers(content);
|
|
expect(result).toContain('Voice triggers (speech-to-text aliases): "see-so", "security review".');
|
|
});
|
|
|
|
test('processVoiceTriggers strips voice-triggers field from output', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n---\nBody`;
|
|
const result = processVoiceTriggers(content);
|
|
expect(result).not.toContain('voice-triggers:');
|
|
});
|
|
|
|
test('processVoiceTriggers returns content unchanged when no voice-triggers', () => {
|
|
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
|
|
expect(processVoiceTriggers(content)).toBe(content);
|
|
});
|
|
|
|
test('generated CSO SKILL.md contains voice triggers in description', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"see-so"');
|
|
expect(content).toContain('Voice triggers (speech-to-text aliases):');
|
|
});
|
|
|
|
test('generated CSO SKILL.md does NOT contain raw voice-triggers field', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(0, fmEnd);
|
|
expect(frontmatter).not.toContain('voice-triggers:');
|
|
});
|
|
});
|
|
|
|
describe('plan-mode-info resolver (handshake-replacement)', () => {
|
|
const REVIEW_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
];
|
|
|
|
// Header for the vestigial handshake that was removed. If it ever reappears,
|
|
// someone accidentally re-introduced the resolver.
|
|
const HANDSHAKE_MARKER = '## Plan Mode Handshake';
|
|
// Header for the new plan-mode-info section (previously lived at the tail
|
|
// of completion-status.ts; now hoisted to position 1 of the preamble).
|
|
const PLAN_MODE_INFO_MARKER = '## Skill Invocation During Plan Mode';
|
|
|
|
test('vestigial handshake is absent from all generated Claude SKILL.md files', () => {
|
|
// Scan every generated SKILL.md under ROOT (top-level directory per skill).
|
|
// Using fs.readdirSync + filter instead of a glob so we catch any skill
|
|
// that gets added later without updating this list.
|
|
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
|
|
let checked = 0;
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(ROOT, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content, `handshake marker in ${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
|
|
checked++;
|
|
}
|
|
expect(checked).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('vestigial handshake is absent from non-Claude host outputs when present on disk', () => {
|
|
// Non-Claude hosts render to hostSubdirs (.agents/, .openclaw/, etc). The
|
|
// plan-mode-info resolver has no host-scoping — all hosts get the new
|
|
// section, none get the old handshake. Scan all candidate host dirs.
|
|
const hostDirs = ['.agents', '.openclaw', '.opencode', '.factory', '.hermes', '.kiro', '.cursor', '.slate'];
|
|
let checked = 0;
|
|
for (const host of hostDirs) {
|
|
const skillsRoot = path.join(ROOT, host, 'skills');
|
|
if (!fs.existsSync(skillsRoot)) continue;
|
|
const entries = fs.readdirSync(skillsRoot, { withFileTypes: true });
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(skillsRoot, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content, `handshake marker in ${host}/skills/${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
|
|
checked++;
|
|
}
|
|
}
|
|
if (checked === 0) {
|
|
// eslint-disable-next-line no-console
|
|
console.warn(
|
|
'plan-mode-info: no non-Claude host outputs found for cross-host absence check — ' +
|
|
'run `bun run gen:skill-docs --host all` to populate',
|
|
);
|
|
}
|
|
});
|
|
|
|
test.each(REVIEW_SKILLS)(
|
|
'%s/SKILL.md contains the new plan-mode-info section near the top',
|
|
(skill) => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const idx = content.indexOf(PLAN_MODE_INFO_MARKER);
|
|
expect(idx).toBeGreaterThan(0);
|
|
// Position 1 in preamble composition = within the first ~300 lines.
|
|
// Roughly translates to first ~15KB of text.
|
|
expect(idx).toBeLessThan(15_000);
|
|
},
|
|
);
|
|
|
|
test('plan-mode-info is wired BEFORE generateUpgradeCheck in preamble', () => {
|
|
const content = fs.readFileSync(
|
|
path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
|
|
'utf-8',
|
|
);
|
|
const planModeIdx = content.indexOf(PLAN_MODE_INFO_MARKER);
|
|
const upgradeIdx = content.indexOf('UPGRADE_AVAILABLE');
|
|
expect(planModeIdx).toBeGreaterThan(0);
|
|
expect(upgradeIdx).toBeGreaterThan(0);
|
|
expect(planModeIdx).toBeLessThan(upgradeIdx);
|
|
});
|
|
|
|
test('0C-bis STOP block present in plan-ceo-review/SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
const presentIdx = content.indexOf('Present these approach options via AskUserQuestion');
|
|
const preludeIdx = content.indexOf('### 0D-prelude');
|
|
expect(presentIdx).toBeGreaterThan(0);
|
|
expect(preludeIdx).toBeGreaterThan(presentIdx);
|
|
const between = content.slice(presentIdx, preludeIdx);
|
|
expect(between).toContain('**STOP.**');
|
|
expect(between).toContain('Do NOT proceed to Step 0D or 0F until the user responds to 0C-bis');
|
|
});
|
|
});
|
|
|
|
// GSTACK REVIEW REPORT report-at-bottom contract — verifies the prompt-text
|
|
// fix in scripts/resolvers/review.ts (the load-bearing change for the
|
|
// "report not at bottom of plan in plan mode" bug). The bug is in the
|
|
// prompt's contradictory write-flow instructions, not in observable
|
|
// runtime behavior we can cheaply gate in CI. Verifying the prompt text
|
|
// directly is the deterministic equivalent of the regression test the
|
|
// PTY harness can't reliably drive (autoplan needs auto-progression of
|
|
// AskUserQuestions to reach the report-write step, which the harness
|
|
// doesn't support today).
|
|
describe('GSTACK REVIEW REPORT delete-then-append flow', () => {
|
|
const PLAN_REVIEW_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'plan-eng-review',
|
|
];
|
|
|
|
for (const skill of PLAN_REVIEW_SKILLS) {
|
|
test(`${skill}/SKILL.md prescribes delete-then-append, not in-place replace`, () => {
|
|
// Carved skills (v2 plan Phase B) relocate the review-report prose into
|
|
// sections/*.md; readSkillUnion follows the content wherever the carve put it.
|
|
const content = readSkillUnion(skill);
|
|
|
|
// The new (correct) instruction must be present.
|
|
expect(content).toContain('delete-then-append flow');
|
|
expect(content).toContain('never mid-file');
|
|
expect(content).toContain('Do NOT replace the section in place');
|
|
|
|
// The old contradictory bullets must be gone. The signature phrase
|
|
// from the buggy prompt was 'replace it entirely using the Edit tool'
|
|
// which is what allowed mid-file reports to stay mid-file.
|
|
expect(content).not.toContain('replace it** entirely using the Edit tool');
|
|
expect(content).not.toContain('If it was found mid-file, move it');
|
|
});
|
|
}
|
|
|
|
test('scripts/resolvers/review.ts source has the rewritten flow', () => {
|
|
const src = fs.readFileSync(path.join(ROOT, 'scripts', 'resolvers', 'review.ts'), 'utf-8');
|
|
expect(src).toContain('delete-then-append flow');
|
|
expect(src).toContain('never mid-file');
|
|
expect(src).toContain('Do NOT replace the section in place');
|
|
// Old contradictory bullets are gone from the source resolver.
|
|
expect(src).not.toContain('replace it** entirely using the Edit tool');
|
|
expect(src).not.toContain('If it was found mid-file, move it');
|
|
});
|
|
});
|
|
|
|
describe('LEARNINGS_SEARCH resolver: query parameter', () => {
|
|
// Lazy-load resolver and types after describe block to keep test file self-contained.
|
|
const { generateLearningsSearch } = require('../scripts/resolvers/learnings');
|
|
const { HOST_PATHS } = require('../scripts/resolvers/types');
|
|
|
|
const claudeCtx = {
|
|
skillName: 'test',
|
|
tmplPath: 'test/SKILL.md.tmpl',
|
|
host: 'claude',
|
|
paths: HOST_PATHS.claude,
|
|
};
|
|
const codexCtx = { ...claudeCtx, host: 'codex', paths: HOST_PATHS.codex };
|
|
|
|
test('no args → bash does not contain --query (backwards-compat)', () => {
|
|
const out = generateLearningsSearch(claudeCtx);
|
|
expect(out).not.toContain('--query');
|
|
});
|
|
|
|
test('claude host + query=foo bar → both cross-project and project-scoped branches contain --query', () => {
|
|
const out = generateLearningsSearch(claudeCtx, ['query=foo bar']);
|
|
// Both branches of the if/else must carry the flag.
|
|
const lines = out.split('\n').filter(l => l.includes('gstack-learnings-search'));
|
|
expect(lines.length).toBeGreaterThanOrEqual(2);
|
|
for (const line of lines) {
|
|
expect(line).toContain('--query "foo bar"');
|
|
}
|
|
});
|
|
|
|
test('codex host + query=foo bar → codex bash variant contains --query', () => {
|
|
const out = generateLearningsSearch(codexCtx, ['query=foo bar']);
|
|
expect(out).toContain('--query "foo bar"');
|
|
expect(out).toContain('$GSTACK_BIN/gstack-learnings-search');
|
|
});
|
|
|
|
test('empty value query= → bash does not contain --query (locked semantics: falls through)', () => {
|
|
const claudeOut = generateLearningsSearch(claudeCtx, ['query=']);
|
|
expect(claudeOut).not.toContain('--query');
|
|
const codexOut = generateLearningsSearch(codexCtx, ['query=']);
|
|
expect(codexOut).not.toContain('--query');
|
|
});
|
|
|
|
test('shell-injection chars in query= → throws at gen-time (defense in depth)', () => {
|
|
for (const bad of ['$(whoami)', '`cmd`', 'a;b', 'a&b', 'a"b', 'a\\b', 'foo$x']) {
|
|
expect(() => generateLearningsSearch(claudeCtx, [`query=${bad}`])).toThrow(/alphanumeric/);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('EXIT PLAN MODE GATE placement', () => {
|
|
// Fresh skill list — do NOT reuse REVIEW_SKILLS upstream (3 entries, missing plan-devex).
|
|
const planSkills = ['plan-eng-review', 'plan-ceo-review', 'plan-design-review', 'plan-devex-review'];
|
|
|
|
// Strip fenced code blocks before matching headings — PLAN_FILE_REVIEW_REPORT
|
|
// already contains `## GSTACK REVIEW REPORT` inside a markdown example fence,
|
|
// and the gate text itself shows `## GSTACK REVIEW REPORT` inside a fence too.
|
|
const stripFences = (md: string) => md.replace(/```[\s\S]*?```/g, '');
|
|
|
|
test('gate is the terminal ## heading in every plan-* review SKILL.md', () => {
|
|
for (const skill of planSkills) {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const stripped = stripFences(md);
|
|
const headings = [...stripped.matchAll(/^## .+$/gm)].map(m => m[0]);
|
|
const lastH2 = headings.at(-1);
|
|
expect(lastH2, `${skill}/SKILL.md last ## heading (fences stripped)`).toBe('## EXIT PLAN MODE GATE (BLOCKING)');
|
|
expect(md, `${skill}/SKILL.md gate body`).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
|
|
}
|
|
});
|
|
|
|
test('codex/SKILL.md contains gate (mid-file per D5; Step 2B/2C follow)', () => {
|
|
const codex = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(codex).toContain('## EXIT PLAN MODE GATE (BLOCKING)');
|
|
expect(codex).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
|
|
});
|
|
});
|
|
|
|
describe('scope-gate exceptions drift-guard', () => {
|
|
// The plan-mode auto-select-B exceptions block is hand-duplicated in the
|
|
// plan-eng-review and plan-design-review templates (matching the gate
|
|
// around it, which predates this block). The two copies must stay
|
|
// byte-identical modulo exactly two known variant slots:
|
|
// 1. the plan-mode bullet's action tail (Design Doc Check vs pre-review
|
|
// audit + mockups),
|
|
// 2. the named-target vocabulary ("a path, a doc" vs "a path, a page, a doc").
|
|
// A future edit to one copy that silently misses the other fails here
|
|
// instead of drifting. The real fix (shared {{SCOPE_GATE}} resolver) is a
|
|
// filed TODO — this guard is the stopgap that makes the duplication safe.
|
|
const START_MARKER = '**Exceptions — check in this order, BEFORE asking:**';
|
|
const END_MARKER = 'in any mode — it is a hard STOP.';
|
|
|
|
function extractExceptionsBlock(skill: string): string {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const start = md.indexOf(START_MARKER);
|
|
expect(start, `${skill}/SKILL.md: exceptions block start marker present`).toBeGreaterThan(-1);
|
|
const end = md.indexOf(END_MARKER, start);
|
|
expect(end, `${skill}/SKILL.md: exceptions block end marker present`).toBeGreaterThan(start);
|
|
return md.slice(start, end + END_MARKER.length);
|
|
}
|
|
|
|
const normalizeVariantSlots = (block: string) =>
|
|
block
|
|
.replace('Then run the Design Doc Check and Step 0 against that plan.', '<ACTION_TAIL>')
|
|
.replace('Then run the pre-review audit, mockups, and Step 0 against that plan.', '<ACTION_TAIL>')
|
|
.replace('a path, a page, a doc they pasted,', 'a path, a doc they pasted,');
|
|
|
|
test('eng and design exceptions blocks are identical modulo the two variant slots', () => {
|
|
const eng = normalizeVariantSlots(extractExceptionsBlock('plan-eng-review'));
|
|
const design = normalizeVariantSlots(extractExceptionsBlock('plan-design-review'));
|
|
expect(eng).toBe(design);
|
|
// The action tail must actually have been normalized in both (guards
|
|
// against a rewording that bypasses the normalizer and vacuously passes).
|
|
expect(eng).toContain('<ACTION_TAIL>');
|
|
});
|
|
|
|
test('exceptions block carries the announcement string the PTY detectors pin', () => {
|
|
for (const skill of ['plan-eng-review', 'plan-design-review']) {
|
|
const block = extractExceptionsBlock(skill);
|
|
expect(block, `${skill}: verbatim announcement`).toContain(
|
|
'Scope gate: plan mode — auto-selected B (reviewing <target>).',
|
|
);
|
|
}
|
|
});
|
|
|
|
test('gate menu carries the question strings the PTY question detector pins', () => {
|
|
// isScopeGateQuestionVisible (claude-pty-runner.ts) anchors on the
|
|
// question text + option A's body. If the menu is reworded without
|
|
// updating the detector, the paid smokes' must-stay-false assertions go
|
|
// vacuous — this free pin fails first.
|
|
for (const skill of ['plan-eng-review', 'plan-design-review']) {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(md, `${skill}: gate question text`).toContain('What should I review?');
|
|
expect(md, `${skill}: option A body text`).toContain('The current branch diff');
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('GSTACK REVIEW REPORT mandatory unresolved-decisions status', () => {
|
|
// Report text rides in PLAN_FILE_REVIEW_REPORT → every report consumer gets it.
|
|
// devex-review is a report consumer but NOT a gate consumer, so the two target
|
|
// sets differ (CP5/CX5). Regression guard: a future token-cut that drops the
|
|
// unresolved-status line again fails here. See plan-flag-unresolved-issues.
|
|
const REPORT_CONSUMERS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'codex',
|
|
'devex-review',
|
|
];
|
|
// Gate text rides in EXIT_PLAN_MODE_GATE (lives in SKILL.md, not sections).
|
|
const GATE_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'codex',
|
|
];
|
|
|
|
for (const skill of REPORT_CONSUMERS) {
|
|
test(`${skill}: report mandates the unresolved-decisions status as final content`, () => {
|
|
const content = readSkillUnion(skill);
|
|
expect(content).toContain('NO UNRESOLVED DECISIONS');
|
|
// The "never omit / always final" contract must be present, not just the phrase.
|
|
expect(content).toContain('Unresolved-decisions status (MANDATORY');
|
|
expect(content).toMatch(/never omitted/);
|
|
// \s+ tolerates prose line-wraps within "final non-whitespace line".
|
|
expect(content).toMatch(/final\s+non-whitespace\s+line/);
|
|
});
|
|
}
|
|
|
|
for (const skill of GATE_SKILLS) {
|
|
test(`${skill}: exit gate blocks unless the unresolved status is the final line`, () => {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
// Gate check #4 — present, sentinel named, and explicitly blocking (no escape).
|
|
expect(md).toContain('NO UNRESOLVED DECISIONS');
|
|
expect(md).toContain('FINAL non-whitespace line is the unresolved-decisions');
|
|
expect(md).toContain('FAILS the gate');
|
|
});
|
|
}
|
|
|
|
test('scripts/resolvers/review.ts source carries the mandatory block + blocking gate', () => {
|
|
const src = fs.readFileSync(path.join(ROOT, 'scripts', 'resolvers', 'review.ts'), 'utf-8');
|
|
// Report resolver: mandatory, never-omitted, exact sentinel, anti-double-count algorithm.
|
|
expect(src).toContain('Unresolved-decisions status (MANDATORY');
|
|
expect(src).toContain('NO UNRESOLVED DECISIONS');
|
|
expect(src).toContain('avoids double-counting');
|
|
expect(src).toContain('DROP the current skill');
|
|
// Gate resolver: the blocking final-line check with no "if applicable" escape.
|
|
expect(src).toContain('FINAL non-whitespace line is the unresolved-decisions');
|
|
expect(src).toContain('FAILS the gate');
|
|
// The old soft wording must be gone from the gate.
|
|
expect(src).not.toContain('absorbs CODEX / CROSS-MODEL / UNRESOLVED lines if applicable');
|
|
});
|
|
});
|
|
|
|
// ─── {{PREAMBLE}} requires an explicit preamble-tier ────────
|
|
|
|
describe('PREAMBLE resolution requires declared preamble-tier', () => {
|
|
test('resolving {{PREAMBLE}} without preamble-tier throws with the template path', async () => {
|
|
const { generatePreamble } = await import('../scripts/resolvers/preamble');
|
|
const { HOST_PATHS } = await import('../scripts/resolvers/types');
|
|
const ctx = {
|
|
skillName: 'tierless-skill',
|
|
tmplPath: 'tierless-skill/SKILL.md.tmpl',
|
|
host: 'claude' as const,
|
|
paths: HOST_PATHS.claude,
|
|
// preambleTier deliberately absent — the generator must refuse to default it.
|
|
};
|
|
expect(() => generatePreamble(ctx)).toThrow(/tierless-skill\/SKILL\.md\.tmpl/);
|
|
expect(() => generatePreamble(ctx)).toThrow(/preamble-tier/);
|
|
});
|
|
|
|
test('every template that resolves {{PREAMBLE}} declares preamble-tier in frontmatter', () => {
|
|
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
|
|
const offenders: string[] = [];
|
|
const checkTmpl = (tmplPath: string) => {
|
|
const tmpl = fs.readFileSync(tmplPath, 'utf-8');
|
|
if (tmpl.includes('{{PREAMBLE}}') && !/^preamble-tier:\s*\d+$/m.test(tmpl)) {
|
|
offenders.push(path.relative(ROOT, tmplPath));
|
|
}
|
|
};
|
|
checkTmpl(path.join(ROOT, 'SKILL.md.tmpl'));
|
|
for (const e of entries) {
|
|
if (!e.isDirectory() || e.name.startsWith('.') || e.name === 'node_modules') continue;
|
|
const tmplPath = path.join(ROOT, e.name, 'SKILL.md.tmpl');
|
|
if (fs.existsSync(tmplPath)) checkTmpl(tmplPath);
|
|
}
|
|
expect(offenders).toEqual([]);
|
|
});
|
|
});
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// #2499: gbrain MCP detection must read BOTH ~/.claude.json scopes.
|
|
// Claude Code registers MCP servers at user scope (.mcpServers) and project
|
|
// scope (.projects["/abs/path"].mcpServers — what `claude mcp add` without
|
|
// --scope user writes). The rendered brain-sync block previously read only
|
|
// user scope, so a correctly configured project-scoped brain was invisible.
|
|
// ---------------------------------------------------------------------------
|
|
describe('brain-sync block reads project-scoped MCP registrations (#2499)', () => {
|
|
const rendered = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
|
|
test('rendered _GBRAIN_MCP_ENTRY jq resolves project scope with nearest-ancestor cwd match', () => {
|
|
const line = rendered.split('\n').find((l) => l.includes('_GBRAIN_MCP_ENTRY=$('));
|
|
expect(line).toBeDefined();
|
|
// Project-scope read present, driven by $PWD.
|
|
expect(line!).toContain('--arg cwd "$PWD"');
|
|
expect(line!).toContain('.projects');
|
|
// User scope still resolved first.
|
|
expect(line!).toContain('.mcpServers.gbrain');
|
|
// The old user-scope-only filter is gone from the rendered output.
|
|
expect(rendered).not.toContain('.mcpServers.gbrain.type // .mcpServers.gbrain.transport');
|
|
expect(rendered).not.toContain(".mcpServers.gbrain.url // empty");
|
|
});
|
|
|
|
test('rendered _GBRAIN_MCP_TYPE and _GBRAIN_HOST extract from the resolved entry', () => {
|
|
const typeLine = rendered.split('\n').find((l) => l.includes('_GBRAIN_MCP_TYPE=$('));
|
|
const hostLine = rendered.split('\n').find((l) => l.includes('_GBRAIN_HOST=$('));
|
|
expect(typeLine).toBeDefined();
|
|
expect(hostLine).toBeDefined();
|
|
expect(typeLine!).toContain('_GBRAIN_MCP_ENTRY');
|
|
expect(hostLine!).toContain('_GBRAIN_MCP_ENTRY');
|
|
});
|
|
|
|
test('rendered jq lines FUNCTION: project-scoped registration resolves for a cwd inside the project', () => {
|
|
// Execute the exact rendered bytes, not a re-derivation: extract the
|
|
// _GBRAIN_MCP_ENTRY + _GBRAIN_MCP_TYPE lines from the generated SKILL.md
|
|
// and run them in bash against a fixture ~/.claude.json that carries ONLY
|
|
// a project-scoped gbrain registration.
|
|
const lines = rendered.split('\n');
|
|
const entryLine = lines.find((l) => l.includes('_GBRAIN_MCP_ENTRY=$('));
|
|
const typeLine = lines.find((l) => l.includes('_GBRAIN_MCP_TYPE=$('));
|
|
expect(entryLine).toBeDefined();
|
|
expect(typeLine).toBeDefined();
|
|
|
|
const tmpHome = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-2499-home-'));
|
|
const projectDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-2499-proj-'));
|
|
const nestedCwd = path.join(projectDir, 'src', 'deep');
|
|
fs.mkdirSync(nestedCwd, { recursive: true });
|
|
try {
|
|
fs.writeFileSync(
|
|
path.join(tmpHome, '.claude.json'),
|
|
JSON.stringify({
|
|
projects: {
|
|
[projectDir]: {
|
|
mcpServers: { gbrain: { type: 'http', url: 'https://brain.example.com/mcp' } },
|
|
},
|
|
},
|
|
}),
|
|
);
|
|
const script = `cd "$1" || exit 1\n${entryLine!.trim()}\n${typeLine!.trim()}\necho "RESOLVED:$_GBRAIN_MCP_TYPE"`;
|
|
const r = spawnSync('bash', ['-c', script, 'bash', nestedCwd], {
|
|
encoding: 'utf-8',
|
|
env: { ...process.env, HOME: tmpHome },
|
|
timeout: 10_000,
|
|
});
|
|
expect(r.stdout).toContain('RESOLVED:http');
|
|
|
|
// Discriminator: a cwd OUTSIDE the project must NOT resolve it.
|
|
const outside = spawnSync('bash', ['-c', script, 'bash', os.tmpdir()], {
|
|
encoding: 'utf-8',
|
|
env: { ...process.env, HOME: tmpHome },
|
|
timeout: 10_000,
|
|
});
|
|
expect(outside.stdout).toContain('RESOLVED:\n');
|
|
} finally {
|
|
fs.rmSync(tmpHome, { recursive: true, force: true });
|
|
fs.rmSync(projectDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|