Files
gstack/test/gen-skill-docs.test.ts
T
+8 28f1385eac v1.91.32.0 fix: Codex works from zsh, free text never enters shell commands, honest failures (#3057)
* fix: sourced helpers locate themselves under zsh (#3024)

Skill blocks source bin/gstack-codex-probe and bin/gstack-egress-lib.sh into
the host's shell. Under zsh (macOS default) BASH_SOURCE is empty, so Codex
model selection resolved /../scripts/... and every outside pass reported
CODEX_MODEL: invalid; the egress lib resolved to the caller's cwd. Resolve
the file once at source time (BASH_SOURCE under bash, %x under zsh), and
replace the bash-only ${!name} indirection in the auth probe. A free test
derives every sourced helper from the generated skills and sources each
under bash and zsh.

Co-authored-by: marcosdetry <marcosdetry@users.noreply.github.com>
Co-authored-by: akshaykalia <akshaykalia@users.noreply.github.com>
Co-authored-by: scottgigante-hubflow <scottgigante-hubflow@users.noreply.github.com>
Co-authored-by: woodsonl <woodsonl@users.noreply.github.com>
Co-authored-by: bezdekt <bezdekt@users.noreply.github.com>

* fix: reply and issue templates never pass untrusted text in double quotes (#3046)

greptile-triage's reply APIs posted -f body="<reply text>" while the
templates quote SHAs and diff code in backticks; in double quotes bash ran
each span (replies posted as 'Fixed in .', and diff or reviewer text ran as
a command). The pre-existing-failure issue template and /spec's issue title
had the same shape. All three now go through quoted heredocs. The
generated-bash lint gains untrusted-in-quotes and now also covers runtime
reference docs in skill directories (review/greptile-triage.md).

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix: a Codex usage limit is its own state, not 'inconclusive; proceeding' (#3051)

The model probe matched only model and broken-install errors, so a usage or
rate limit fell through to MODEL_PROBE_INCONCLUSIVE: Codex's own message
(with the reset time) was dropped, the hint pointed at model config, and
every later skill re-probed for 30 s. The probe now returns
MODEL_QUOTA_EXHAUSTED (exit 4) on a failed, non-timeout call that names a
usage limit, rate limit, insufficient_quota or 429, relays Codex's line
verbatim and caches it for 15 minutes. Preflights map it to
CODEX_MODE: quota_exhausted (missing coverage, no further Codex calls), and
classifyOutsideReview reports a mid-run limit as unavailable (quota) with
its own troubleshooting anchor.

Co-authored-by: venkatareddyg <venkatareddyg@users.noreply.github.com>

* test: record ship/review union sizes after the quota branch and issue template

* fix: Codex watchdog freezes the command before reaping its children

The bash-native watchdog (stock macOS, no timeout binary) ran pkill -P on
the command's children and then killed the command. Between the two, the
command continued past its killed child and printed output after its
deadline. Stop it first. The test's minimal PATH also lacked pkill, which
the watchdog uses to reap children (stock macOS ships it), so the orphaned
child held stdout open and the capture waited out its sleep.

* fix(autoplan): an oversize session journal is a hard too_large denial, not a retryable identity failure (#3050)

The guard capped journal reads at 32 MiB and reported a longer journal as
'identity', which the hook treats as deny-and-retry. A journal only grows,
so /autoplan could never enter Phase 1 in a long session, and the
troubleshooting row sent people chasing symlinks. Oversize journals now
report too_large as a hard cause that names the size and the supported
fallback (run the reviews by hand or start a new session).

Co-authored-by: venkatareddyg <venkatareddyg@users.noreply.github.com>

* fix(deps): clear the weekly OSV findings (#3040)

Raise overrides to fixed releases: ip-address 10.7.3 (our own pin was the
vulnerable 10.3.1; reachable through socks in browse's proxy path),
fast-uri 3.1.8, hono 4.13.13, proxy-addr 2.0.8, @modelcontextprotocol/sdk
1.31.0, sharp 0.35.5, and in lib/diagram-render dompurify 3.4.16 (bundled
by mermaid) and source-map-js 1.2.2; the render page is rebuilt. Three
advisories without a usable fix get justified, expiring ignores: braces
(build-time only, excalidraw -> sass -> chokidar; a test checks the page
carries no chokidar code), sprintf-js (roarr's constant-format logger) and
katex (needs pre-existing prototype pollution; fix outside mermaid's range).
osv-scanner with the repo config: no issues found.

* fix(autoplan): name a shell-built snapshot init as the cause of the missing-evidence denial (#3045)

The guard binds a phase entry only to a literal snapshot-init Bash call.
When the model resolved the tool path into a variable, chained or piped the
call, init ran and succeeded but the denial said 'Complete the existing
snapshot init step', which reads as already done, so the run gave up. When
no init binds but a Bash call looks like one, the denial now says the shape
cannot be bound and prints the literal re-run (answers reused:true). The
binding rule is unchanged, and the skill says to run init on its own.

Co-authored-by: CharlesBerg85 <CharlesBerg85@users.noreply.github.com>

* fix(browse): chain reads piped stdin synchronously, so Windows runs the flow (#3039)

On Windows, an awaited Bun.stdin.text() inside the CLI's un-awaited main()
let the process exit 0 before reading stdin: a replayed flow reported
success with nothing sent to the daemon. Read fd 0 synchronously (skipped
for a terminal); empty input now gets the server's usage error and a
non-zero exit.

Co-authored-by: VaibDeveloper <VaibDeveloper@users.noreply.github.com>

* fix(browse): clearer config-mismatch refusal and a WebGL note for the macOS GPU flags (#3030)

A plain call refused because another session started the daemon with
--headed/--proxy was told to 'apply --proxy/--headed', flags it never
passed. It now says who started the daemon and offers both remedies. The
BROWSER.md env table notes that the default macOS GPU flags disable WebGL
and that GSTACK_DISABLE_GPU=off restores it for WebGL QA.

Co-authored-by: Hagetino <Hagetino@users.noreply.github.com>

* fix(ship): skip Greptile triage when no PR exists yet (#3020)

/ship's first run on a branch dispatched a triage subagent only to learn
there was no PR. Step 10 now checks for a PR inline (gh's 'no pull requests
found') and records that Greptile runs on the PR once it exists; other gh
errors still dispatch so the child can report unavailable.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix(router): route only to loaded skills, and say when not to invoke (#3018)

The root router listed every skill and ended 'When in doubt, invoke the
skill'. Current Claude models follow that literally: they tried to invoke
skills a user had turned off in Claude Code's skillOverrides (which gstack's
own disabled_skills render cannot see), and borderline requests tipped into
full workflows. The router now routes only to skills in the session's
available-skills list and names when to answer directly; the CLAUDE.md
routing block skill-start offers says the same.

Co-authored-by: Rekin226 <Rekin226@users.noreply.github.com>

* fix: spawned sessions never auto-grant consent or publish off the machine (#3016)

Spawned sessions auto-choose the recommended option except for destructive
or irreversible choices. Main already withholds the artifacts-sync consent
prompt from spawned sessions; the remaining gap was any other prompt whose
recommended option grants a consent or syncs data off the machine. Every
spawned directive (skill-start's block, the escape sentence, the Conductor
deny reason and the AUQ-error fallback) now names that as irreversible. A
consent the user already gave interactively stays in force.

Co-authored-by: ekson73 <ekson73@users.noreply.github.com>

* test: fix the spawned-block slice and type the #3045 re-init events

* fix(redact): TypeScript types and JSX expressions are code, not env.kv secrets (#3048)

A credential-named parameter or prop with an unquoted code value
(`session: SessionState,`, `key={turn.requestId + turn.role}`,
`apiKey={settings.apiKey}`) cleared the entropy gate and blocked pushes as
a MEDIUM env.kv finding. One rule replaces the PR's two keyword heuristics:
a value that reads as a code name (no digits, word-like camelCase or dotted
parts) with a code terminator, or a brace expression of such names with no
quoted literal, is source syntax. Paired controls keep firing on quoted,
brace-wrapped, concatenated, YAML-bare and random-letter literals.

Co-authored-by: tianxchen <tianxchen@users.noreply.github.com>

* fix(setup): refuse to register a hook that does not parse (#3032)

Claude Code runs gstack's hook shims through /bin/sh; a shim that does not
parse exits 2, which for a PreToolUse hook blocks the tool call in every
session until someone finds the file. Before any hook registration is
healed or written, setup now runs bash -n on each registered shim and a Bun
parse of its TypeScript entry, and stops with the file, line and fix. A
lean rewrite of the PR's idea (one gate block plus a test that runs it).

Co-authored-by: BenjaminDSmithy <BenjaminDSmithy@users.noreply.github.com>

* fix(browse): Windows runtime roots start the server from the source checkout's bundle (#3026)

On Windows a host runtime root (~/.codex/skills/gstack and the other
env-var hosts) holds file copies of browse/dist and no node_modules, so
its server-node.mjs could not import playwright and every browser skill
died (the module-scope server.ts half was fixed in v1.91.19.0). setup now
records the source checkout in every runtime root's .source-path, and the
CLI runs that checkout's bundle when the adjacent one cannot reach
playwright. Verified by unit layout tests and the Windows CI lane, not
live on Windows.

Co-authored-by: ubay887 <ubay887@users.noreply.github.com>

* fix(copilot): on Windows, run bash blocks in Git for Windows Bash (#3047)

Copilot CLI on Windows ran gstack's extensionless helpers straight from
PowerShell, which opened an app picker, and bare bash can resolve to WSL.
The Copilot tool glossary (rendered into every Copilot skill) now says to
run each bash block in Git for Windows Bash by its full path and never to
translate a block into PowerShell. The PR's PowerShell repair skill and
README section are not taken.

Co-authored-by: nareshh74 <nareshh74@users.noreply.github.com>

* test: record ship's union size after the Greptile no-PR check

* fix: keep issue numbers out of model-read Copilot text; seed the new routing sentence in the autoplan fixture

* docs: Oct 6 fix-wave plan, reviewed through /autoplan

* fix(autoplan): every oversize-journal path reports too_large with a recovery that keeps the work

Review follow-up to the #3050 fix. The summed batch read and the narrowed
parent loop now report too_large with the real size instead of a plain
error naming a hard-coded 32 MiB. The denial names the size and the cap,
and gives the recovery that keeps work (/context-save, new session,
/context-restore, /autoplan) with a docs anchor. A test seam that can only
lower the cap replaces the 33 MiB fixture.

* test(autoplan): negative controls and more shapes for the shell-built init denial

* fix(redact): decide TS/JSX source syntax by file context, not by value shape

Review follow-up to the #3048 fix. The word-shape heuristic also exempted
real secrets made of words (API_KEY=VelvetRiverOrbitSunset; in a .env).
The scanner now takes the file path (ScanOptions.sourcePath): the pre-push
hook scans the pushed diff per file with its +++ b/ path, and gstack-redact
--from-file passes its path. Only in a TypeScript/JSX file is an unquoted
value after ':' or a JSX brace of names treated as source; the same line in
.env/YAML, or with no path, is scanned as a literal, and the jwt pattern
still fires in .ts files.

Co-authored-by: tianxchen <tianxchen@users.noreply.github.com>

* docs: gstack's own CLAUDE.md routing block matches the new router wording

Live check (Claude Code 2.1.284): a skill set to "off" in skillOverrides
is absent from the session's skill list while its sibling is listed, and
both are listed without the override, so 'route only to skills in your
available-skills list' holds.

* fix(sync-gbrain): don't offer a call-graph build for a source with no code pages (#3029)

Step 3.5 offered --dream for any source with pages, including one that
holds only markdown (synced with gbrain's default strategy), where a build
cannot help, and the injected CLAUDE.md guidance told future sessions to run
--dream for any count: 0. On gbrain >= 0.60, code-def for an impossible
symbol answers .status (probed live against gbrain 0.60.96: ready for a
code source, out_of_scope for a markdown-only one). Step 3.5 now probes the
pinned source first, reports the cause instead of asking on out_of_scope,
and keeps today's offer when the probe is unknown (older gbrain, an error).
gstack's own sync uses --strategy code, so the PR's soft-delete claim does
not apply to gstack's path.

Co-authored-by: syong <syong@users.noreply.github.com>

* fix(setup): register the hooks that parse, refuse the rest with file:line (#3032)

The landed gate stopped setup before any registration, so one unparseable
hook left every other hook unregistered. On the Claude install the canonical
root is the checkout Claude Code runs hooks from, so there is no staged swap
to abort: setup now registers the hooks that parse, refuses the ones that do
not with file and line, finishes the install and exits non-zero, saying it
is a gstack bug and that a skipped hook keeps running from the checkout.

bin/gstack-hook-check derives its list from setup's _hook_command_path calls
plus the autoplan phase-publication hook (a test keeps them equal), parses
each shim with its shebang's interpreter, and bundles each TypeScript entry
with its local imports (packages external) into a temp dir, so a broken or
missing import, or a missing entry, fails.

Co-authored-by: BenjaminDSmithy <BenjaminDSmithy@users.noreply.github.com>

* fix(auto-update): hold an update whose hooks do not parse, before the checkout moves

bin/gstack-session-update fast-forwards the live checkout before setup runs,
and Claude Code runs hooks straight from it, so setup's gate cannot protect
an auto-update. After the Bun floor check the updater extracts the incoming
revision with git archive and runs bin/gstack-hook-check on it; a failure
holds the update like bun-too-old, records hook-does-not-parse with file:line
in the update status, and the next session start says the current hooks keep
running. Fixture-remote tests cover a broken shim, a broken imported module
and a parseable control.

Co-authored-by: BenjaminDSmithy <BenjaminDSmithy@users.noreply.github.com>

* fix(browse): chain with no flow is a usage error before any daemon boots (#3039)

Empty piped input, a terminal with no arguments, and a readFileSync(0)
error (EAGAIN, EOF) now print the chain usage with the cause and its
troubleshooting link and exit 1 before ensureServer(), instead of sending
an empty argument to a daemon the CLI booted for it. Tests run the real CLI
with no daemon and no browser, plus the stub-daemon flow.

Co-authored-by: VaibDeveloper <VaibDeveloper@users.noreply.github.com>

* fix(browse): refuse a runtime-root CLI and checkout server from different builds (#3026)

A Windows runtime root runs the source checkout's Node server bundle through
.source-path. When the root's CLI and the checkout's bundle carry different
browse/dist/.version builds (the checkout was rebuilt without refreshing the
root), the CLI now refuses to start the server with both builds, the
./setup fix and its troubleshooting anchor. Unversioned dev builds are not
called skew.

Co-authored-by: ubay887 <ubay887@users.noreply.github.com>

* ci(windows): real isolated ./setup install, then a piped chain flow through the installed browse

windows-setup-e2e built the checkout but never ran ./setup. It now installs
the codex host into an isolated HOME (a runtime root copy with no
node_modules), then from an unrelated directory pipes a two-command chain
flow at a loopback page into the installed browse and asserts the page text,
and that empty piped input exits 1 with the usage error. Covers the
checkout-bundle path and the piped-stdin path end to end.

* test(lint): free text never enters a generated command (structural rule)

The generated-bash lint's untrusted-in-quotes rule (gh/glab/git/curl text
flags in double quotes only) becomes free-text-placeholder: any <placeholder>
anywhere in any command, in double, single or no quotes, and in heredoc
bodies (quoted or not), unless it is on the reviewed identifier allowlist
(test/helpers/placeholder-allowlist.ts) with the grammar a value must match.
Quoted-only identifiers (URLs, selectors, paths printed by gstack) must sit
inside quotes. Backslash-newline continuations now read as one command, so
a flag and its value on separate lines are still seen together.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix: reply, issue and spec text travel as agent-written files, never through a heredoc

Replaces the quoted-heredoc bodies with CEO-12's file mechanism: a bash
block creates a private mktemp file in the project's .gstack/tmp/ (kept out
of git through .git/info/exclude), the agent writes the text with its
file-write tool, and the next block posts with -F body=@file, --body-file
or "$(cat file)". A heredoc ends early at any body line equal to its
delimiter; a file has no such line. When the file was never written the
block refuses with the path and the command to send it by hand, and it
deletes the file after a successful post.

- review/greptile-triage.md: replies; the fetched JSON moves from fixed
  /tmp/greptile_*.json paths to a mktemp directory; REPO and PR are
  re-derived per block instead of read from a previous shell.
- Test-failure issues: title and body files, platform case on
  github/gitlab, assignee only when it is a valid login.
- /spec: one draft file feeds the audit hash, every redaction scan, the
  issue and the archive; the archive no longer expands <title>/<body>
  inside an unquoted heredoc; the decision-log JSON is built by jq from
  files.

Location verified against Claude Code 2.1.284 (Dockerfile.ci pin):
project-local .gstack/tmp is writable under acceptEdits with no prompt and
keeps mode 0600; $TMPDIR and the gstack state root prompt even under
acceptEdits.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix(ship): Greptile triage runs only when a PR exists; every other case says why

PR: unknown (gh missing, not logged in, a non-GitHub remote, or a failed
lookup) used to dispatch the triage child anyway. The Step 10 check now
tests gh, its login, the remote and the PR in that order, and every case
but 'PR: exists' skips the dispatch and records
'Greptile: not run (<reason>); runs on the PR once it exists'.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix: question-tuning commands carry no free text

The question log took the summary in single quotes and the preference write
took the user's own words; an apostrophe ended the quote and the rest ran as
shell. The summary is now a kebab-case slug (an identifier with a grammar),
and the optional free_text field is no longer passed from skills (the
binary still accepts it).

* fix(cso): Windows finds docker.exe only under known-folder roots and says transport is unsupported (#3028)

On Windows /cso looked for docker only in Git's directory and System32, so
Docker Desktop was never found and the gap read as 'not installed'. It now
resolves Program Files, Program Files (x86) and the Windows directory through
PowerShell's [Environment]::GetFolderPath (not the process environment),
checks each candidate's real path against those roots and rejects reparse
points. lib/cso/docker.ts admits only unix:/// endpoints, so a trusted
docker.exe reports 'native Windows Docker transport is not supported yet;
static assessment only'; one outside the roots is refused as outside the
trusted install locations, not overridable. Both carry an anchor. Native
named-pipe transport is not implemented.

* fix: sourced helpers share one self-locate block and fail loudly when they cannot locate themselves (#3024 rework)

The codex probe and the egress lib now carry one byte-identical block:
BASH_SOURCE under bash or %x under zsh, made absolute with $PWD, else a set
GSTACK_ROOT that holds the helper. Anything else (dash, sh, a shell that
cannot name the file) prints "gstack: cannot locate <helper> (shell: ...)"
with the docs/troubleshooting.md#sourced-helper-location URL and returns
before any bash- or zsh-only syntax. The Claude-only default path and the
$(pwd) fallback are gone.

The generated Codex preflight dot-sources the probe and, when it cannot load,
reports CODEX_MODE: helper_unavailable with the helper's own line instead of
falling through to not_authed. The class guard enumerates nested sources,
makes real resolver, auth, state-root and egress-home calls under bash and
zsh (absolute, relative then cd, bare name via PATH), covers dash and an
unknown shell, runs the full preflight, and lints BASH_SOURCE to the shared
block plus other bash-only constructs.

Co-authored-by: marcosdetry <marcosdetry@users.noreply.github.com>
Co-authored-by: akshaykalia <akshaykalia@users.noreply.github.com>
Co-authored-by: scottgigante-hubflow <scottgigante-hubflow@users.noreply.github.com>
Co-authored-by: woodsonl <woodsonl@users.noreply.github.com>
Co-authored-by: bezdekt <bezdekt@users.noreply.github.com>

* fix(cso): a Maven or Gradle repository is Java, not Python (#3028)

With no supported stack metadata, preparation defaulted to Python and asked
a Java repository for requirements.txt. A root pom.xml, build.gradle or
build.gradle.kts now names the stack as Java (unsupported for runtime
preparation, so the prerequisite says so), and Python helper scripts or a
root requirements.txt beside it no longer make it Python.

* fix(browse): BROWSER.md says plainly how to enable WebGL; the mismatch refusal links Daemon discipline (#3030)

The GSTACK_DISABLE_GPU row now states the effect and the switch: set
GSTACK_DISABLE_GPU=off to enable WebGL (off is the only recognized value).
The reworded config-mismatch refusal links BROWSER.md's Daemon discipline
section. Free tests pin both.

Co-authored-by: Hagetino <Hagetino@users.noreply.github.com>

* test(windows): name the B2, B4, B5 and B12 tests in the curated Windows lane

The runtime-root bundle tests lived in browse/test/config.test.ts and the
Copilot glossary test in test/host-config.test.ts, both excluded from the
Windows lane, so the two Windows-only fixes were never checked on Windows.
Move each into its own Windows-safe file (unchanged assertions) and list it,
the piped-stdin chain test and the /cso docker.exe discovery test as named
Windows coverage in scripts/lib/windows-curation.ts.

* fix: the free-text block exits loudly when mktemp fails; spec and redact tests follow the file mechanism

The redact resolver test, the #2679 mktemp guard test, the /spec invariant
and the spec secret-sink harness pinned the heredoc transport. They now pin
the agent-written file: the harness runs the draft-file block, writes the
draft the way the file-write tool would, then runs the real scan fence, with
every HIGH/MEDIUM/scanner-failure assertion unchanged.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* test: hostile payloads through the generated posting blocks, identifier grammars, every quoting shape

- The lint reads a whole command, so an unquoted multi-word placeholder is
  caught; negative controls cover double, single and no quotes, quoted and
  unquoted heredocs, continuation lines, and quoted-only identifiers used
  unquoted.
- Every allowlisted grammar rejects quote-breaking characters (and shell
  operators and whitespace when used unquoted); git accepts 'fix;id' and
  'fix`id`' as branch names, the ref grammar does not.
- Greptile replies, test-failure issues (GitHub and GitLab) and /spec filing
  and archive run from the real generated text with stub gh/glab: a body
  with backticks, $(...), an apostrophe and lines equal to EOF and the old
  heredoc delimiters arrives byte for byte, nothing in it runs, and the temp
  file is gone. An unwritten file is refused with its path and the manual
  command, and nothing is sent.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* test(ship): the PR check precedes the Greptile dispatch, and each skip names its reason

Runs Step 10's generated check against stub gh for gh missing, not logged
in, a non-GitHub remote, no PR, a failed lookup and an existing PR. Fails on
the previous text (which printed PR: unknown and dispatched).

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* feat(ship): ship-measure runner for the measure-then-fix loop (A6)

scripts/ship-measure.ts owns /ship's diagnostic measurement of a red eval
case: per-kind aggregation (rule trials; behavior panels of 3, each at its
own 2 of 3 plus 90% of trials; judge outputs scored by 3-sample panels,
standalone judge ids accepted), one artifact directory per baseline,
repair round and trial, a diagnostic label with verdict null that refuses
the eval history dir, an estimated admission budget that reserves every
concurrent trial and reconciles actual cost, the ask-once rule, the
repair-round limit, the unmeasured skip and the per-case PR table.

Free-suite flakes (CEO-1): 'free' reruns one shard's exact file list N
times, rerun 1 alone as the baseline and the rest in parallel at
max(1, floor(cores / shard concurrency)), each with its own HOME, state
root and flake ledger, GSTACK_FREE_RETRY_FLAKY=0, under a wall cap that
reports a partial count. test:ubicloud --diagnostic runs the same on one
VM with the retry off.

bin/gstack-config gains the DX-7 keys with their CEO-9 defaults and
rejecting validators. Free tests drive every kind, the budget stop, the
unmeasured skip and the free reruns with fake runners.

* fix: a Codex usage limit is quota_exhausted, a plain 429 is rate_limited, and only Codex's own error lines count (#3051 rework)

The probe's quota signature used to treat any rate limit, "too many
requests" or 429 as quota, and read the whole output, which echoes the
prompt. Now only Codex's trailing ERROR: / stream error lines are read.
A usage-limit or insufficient_quota line is MODEL_QUOTA_EXHAUSTED (exit 4,
cached 15 minutes; quota wins over a 429 that names insufficient_quota); a
plain 429 is MODEL_PROBE_RATE_LIMITED (exit 0, state rate_limited, never
cached), reported as CODEX_MODE: unverified (rate_limited) so the review
still runs and decides.

The quota HINT says how many minutes gstack skips Codex and how to retry
now (GSTACK_CODEX_PROBE_RETRY=1 skips the negative cache for that probe, or
delete <state root>/.codex-model-probe), with the full troubleshooting URL.
classifyOutsideReview reads the same trailing lines: a mid-review quota is
unavailable (quota_exhausted), a mid-review 429 unavailable (rate_limited).
New gate row rate_limited and docs anchor codex-rate-limited; the quota row
documents the retry. Tests cover each line shape, the echoed prompt, exit
0, timeout, retry recovery, probe/classifier parity, and two fresh-shell
preflight blocks where the second makes no paid Codex call.

Co-authored-by: venkatareddyg <venkatareddyg@users.noreply.github.com>

* ci: the eval image installs zsh, and the macOS lane sources the Codex probe from zsh

Dockerfile.ci adds zsh so the sourced-helper guard runs (and fails loudly
when zsh is missing) inside the eval container too. The macOS named
regressions job sources bin/gstack-codex-probe from zsh in an unrelated
directory and runs _gstack_codex_select_model exec (no paid call), then runs
the sourced-helper guard on the macOS runner. A free test pins both lanes and
runs the macOS step's script locally.

* test: deterministic regression for the Codex watchdog freezing the command before reaping its children

The existing stubborn-child test caught the reap-then-kill race only some of
the time (2 of 3 runs on the pre-fix watchdog). A pkill that sleeps after
reaping widens the window between reaping the children and killing the
command, so a watchdog that does not SIGSTOP the command first always lets
it print "late" after the deadline. Fails 3 of 3 on the pre-fix watchdog,
passes 3 of 3 on the current one.

* test: live codex smokes run codex in a private HOME and CODEX_HOME

codex-resume-flag-semantics and codex-e2e-sol-scope probe the codex CLI (the
latter at module load, in every free run), and every codex run writes
$CODEX_HOME/tmp/arg0, so with a real codex on PATH they wrote into the
developer's ~/.codex. Both now give codex a private HOME and CODEX_HOME and
remove it afterwards. A free test runs each file against a stub codex that
records HOME and CODEX_HOME and asserts neither is the caller's.

* feat(ship): a red eval runs classify, measure, fix, re-measure, gate once (A6)

Step 6 no longer stops at the first red paid eval: it reads the new lazily
loaded measure section and runs the loop for each red case, and Step 5
measures a failed free-suite shard (diagnostic reruns, retry off) before
the triage can call it pre-existing. The section states the diagnostic-
not-verdict rule, the per-kind table, the ask-once and named-red stops,
the unmeasured label and the honest-fix wording (observed k/n after fix
at <cause>). A free test pins the steps and their order in the generated
skill and checks every command and flag it names exists in the runner.

Ship's parity cap rises to 1.455 (measured 272,944 / 187,706 = 1.4541).

* docs(testing): diagnostic measurement is not a verdict; per-kind measure table (A6)

* test(ship): seeded-flake paid case for the measure loop (A6, periodic, behavior)

A fixture repo with no remote documents a single-case eval command and a
gate command, both free local stubs. queue-priorities claims trial slots
with atomic mkdir (reset per ship-measure round) and fails slots 2, 5 and
9 while src/priorities.js leaves its seed-shuffled list unsorted, printing
failure_cause and failure_detail that name the line. The paid case gives
/ship's generated Step 6 and measure section to a live session and asserts
it measured the case alone, fixed the named line without touching the
eval, re-measured at least 10 passing trials on the sorted file, and only
then ran the gate once; any question other than the spend question is a
contract violation.

A free test measures the fixture through the real ship-measure CLI: 7/10
unsorted, 7/10 after an unrelated edit with the gate still red (ENG-13's
negative control), 10/10 after the sort with the gate green.

* test(cso): the missing-scanner case points Docker at a dead socket

'an optional scanner prerequisite does not turn completed manual coverage
partial' assumed no usable Docker. On a machine with Docker and a pulled
gitleaks image the scan completed and the test failed (also on main). The
scan step now gets a DOCKER_HOST that cannot connect, which is the
prerequisite-missing condition the test describes.

* fix: Aside prompts travel as agent-written files, never inside the command

The open-ended reading recipe (browse, devex-review), scrape's fuzzy-intent
read and the shared web-research block put the model's question, query and
reply format inside a double-quoted _aside_exec argument, where backticks and
$(...) from page or repo text would run. The agent now writes the prompt into
a private .gstack/tmp file; the send block rebuilds the path from the printed
basename, refuses when the file is empty, and passes "$(cat "$PROMPT_FILE")".
The fixed read-only sentence stays in the shell source.

aside-driver pins move from the inline prompt to the file-based send block.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix: design briefs and approval feedback travel as agent-written files

Design briefs went into $D variants/generate/check as a double-quoted
argument, and approval feedback was spliced into a single-quoted echo of JSON,
so a quote, backtick or $(...) in either broke the command or ran. The agent
now writes each brief into a private .gstack/tmp file and the block passes
--brief "$(cat "$BRIEF_FILE")" (the CLI's --brief-file expects a JSON
brief, so plain text uses --brief). variants and the later check reuse the
same brief file. approved.json is built with jq -n from --arg values and
--rawfile feedback, so the feedback is data. The office-hours mockup flow
creates its brief and feedback files in one block.

design-approval-flow now writes the feedback file (with quotes, backticks and
$(...) in it) instead of substituting <FEEDBACK>, and checks the file is
removed after the save.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix: carve office-hours on every host to stay under the 160,000-byte ceiling

With its brief, feedback and research prompts moved into files, the inlined
office-hours render is 161,329 bytes on Codex and 160,302 on Factory, over
SKILL_BYTE_CEILING, so gen-skill-docs --host all fails. Per the C4 policy
(carve every skill whose external render exceeds the ceiling), office-hours
joins ship and plan-ceo-review in CARVED_ON_EVERY_HOST. The Claude render is
unchanged; external hosts read its sections relative to the installed skill.
external-host-carve and qa-lazy-sections now include office-hours in their
carved-skill lists.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* chore: regenerate after the free-text mktemp guard

* fix(codex): prompts and review focus travel as agent-written files

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

Challenge and consult prompts, and the /codex review focus text, are written
by the agent into private .gstack/tmp files and fed to Codex on stdin or read
back with cat; no heredoc carries them. Consult's new and resumed runs share
one block (and one parser) so no placeholder stands in for the parser; a
resumed run reads its session id from .context/codex-session-id and checks it.
Adds the A3 group 2 placeholder allowlist block.

* fix(benchmark-models): an inline prompt goes through a prompt file

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix(land-and-deploy): PR head repository and branch travel as data

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

The branch check reads headRepository/headRefName from gh's JSON into quoted
shell variables; ls-remote, the confirm-first delete and the local delete use
those variables instead of model-substituted names. The postfail test now pins
the variable form it replaces.

* fix(office-hours): the reviewer receipt must match the receipt grammar before the check runs

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix(setup-gbrain): cleanup-orphans collects the PAT with read_secret_to_env

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix(ship): PR body drafts and rejected test paths are agent-written files

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

The PR body is written into two private drafts (through ## Documentation, and
from ## Test plan on); the scan block splices Step 14.5's section between them
into the scanned temp file and deletes the drafts. Rejected test paths are read
from a written list, and paths outside the repository are skipped. Tests that
ran the old heredoc blocks now write the drafts and list first.

* fix(sync-gbrain): orchestrator flags must match an allowlisted grammar

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix: the gbrain page save and external-host placeholders pass the free-text rule

The gbrain save block passed the whole page through a quoted heredoc inside
--content; the agent now writes it to a mktemp file and gbrain put reads it
on stdin. The Claude Code outside-voice runtime root and session mode, and
the feature slug, join the allowlist with grammars; unused entries are
dropped, and a test keeps every allowlisted placeholder tied to a real use.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix: skill-specific Aside research queries travel as prompt files

investigate, office-hours, plan-ceo-review, review, plan-eng-review,
design-consultation Phase 2, the shared test bootstrap (B2) and QA's test
bootstrap each carried their own _aside_exec call with the model-filled query
inside a double-quoted argument. Skills that already carry the Web research
block now give only the prompt-file text and use that block to create, write
and send it; design-consultation, the shared bootstrap and QA create the file
and send it with the new {{ASIDE_RESEARCH_SEND}} resolver, which is the same
send block the research section uses (read-only sentence stays in the shell).

aside-driver's B2 pin and qa-browser-preservation's bootstrap contract move
from the inline call to the prompt file and the shared send resolver.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* fix: prose design commands pass briefs and feedback as data

The DESIGN_SETUP command list taught `iterate --feedback "..."`, and
design-consultation's late-change and plan-design-review's slop-regenerate
lines showed `--feedback "<feedback>"` / `"..."`; design-shotgun's retry
showed `generate --brief "<brief>"` and `evolve --brief "<brief>"`. Feedback
now reads "$(cat "$FEEDBACK_FILE")" from an agent-written mktemp file
(the setup note says how), and the shotgun retry reads the variant's own brief
out of briefs.json with jq instead of retyping it.

Co-authored-by: dgrant <dgrant@users.noreply.github.com>

* test: record parity sizes after the free-text file mechanism; refresh ship goldens

Measured union and skeleton sizes for ship, plan-eng-review,
plan-design-review, office-hours, design-consultation, review, qa, browse
and design-shotgun after every posting, prompt and brief site moved to
agent-written mktemp files; the ship goldens pick up the question-tuning
slug and the dropped free_text field.

* test: parity caps for the combined wave head (rounded up past the measured ratio)

* fix(plan-ceo-review): the mode handoff helper reminds the model to lead its chat with the line (#3038)

Periodic run 37272185151's auto-decide-preserved red replays as the Step 0E
handoff-line class: the model ran gstack-ceo-mode-handoff --auto, got the
exact line, then opened its chat with a heading and a paraphrase, so the
user never saw the auto-decide line (the terminal collapses tool output)
and the detector saw no outcome. The helper now prints a one-line reminder
on stderr beside the line; the detector accepts the line followed by that
reminder and nothing else. The ship Greptile check test reads its template
(test-of-test ratchet).

* test: every troubleshooting link in shipped code resolves, relative or full URL

* docs(todos): Oct 6 fix-wave follow-ups

* v1.91.32.0 release notes: Codex works from zsh, free text never enters shell commands, honest failures

* fix(plan-ceo-review): show the mode handoff line through a PostToolUse hook, not model compliance (#3038)

The stderr reminder beside the handoff line measured 1/10 on
auto-decide-preserved: the model still opened its chat with a paraphrase
('I've set review mode to HOLD SCOPE...'), so the user never saw the line and
the detector saw no outcome. /plan-ceo-review now declares a Claude-only
PostToolUse hook that shows the helper's exact line as a system message.
Diagnostic measurement on this head: 10/10, the line visible in every trial
while no trial's chat led with it. The reminder is reverted.

* fix: type the handoff hook's line filter

* fix: repairs from the integrated free suite

- The self-locate markers looked like heredocs to the pipe-deadlock guard;
  they are plain comment fences now.
- The spawned block keeps the canonical 'conservative non-destructive'
  phrase the cross-surface drift guard pins.
- The free-text block's mktemp refusal says 'mktemp failed', as every
  /codex temp file must.
- The batch transcript read reports sizes in MiB.
- docs/TESTING_INTERNALS.md gets a real 'Ship measure' heading for its
  anchor.
- The early-question harness admits /plan-ceo-review's handoff hook by its
  exact command (it only adds a system message); any other PostToolUse Bash
  hook is still refused.
- Paid file count and the unconsumed-file example follow the new seeded
  case; ship goldens refreshed.
- The CSO scanner test points Docker at a dead socket, so a machine with the
  pulled image does not scan.

* test(cso): a host with no Docker binary is also a no-scan host

The missing-image scan test failed on a Docker-less VM (docker is not
installed in a trusted directory). Every accepted message is a reason the
scan cannot run here; the not_assessed, helper-owned artifact assertions are
unchanged.

* ci: split the USERPROFILE assignment from export (SC2155)

* fix(browse): Windows runtime roots read .source-path as a native path (#3026)

The new windows-setup-e2e journey showed the installed browse could not
start its server. setup runs under Git Bash and wrote .source-path as an
MSYS path (/d/a/gstack); Node read it as \d\a\gstack on the current drive,
found nothing, and fell back to the root's own bundle, which cannot import
playwright. setup now writes cygpath -m, and the CLI maps an MSYS drive path
for installs written by an older setup. The journey prints the daemon and
startup logs when the chain fails.

* fix(ship-measure): defects from the parallel #3034 plan review

- A trial passes only with a JUnit testcase that executed and passed; a
  --case that selected nothing also exits 0.
- measure runs a no-cost --list preflight that must plan the case's own
  file before any paid trial.
- Diagnostic trials drop EVALS_CACHE_DIR/EVALS_CACHE_RUNTIME_ID, so they
  never reuse a stored judge pass or publish one the gate could reuse.
- Gate once: when CI runs the gate on push, the push is the gate run; the
  skill no longer also starts eval:bg:pr (a second paid gate).
- ubi-runner: down fails unless the VM is confirmed gone, and nothing
  sweeps other runners' VMs by default (UBI_GC_HOURS opt-in).

* test(windows): the seeded-flake fixture drives a POSIX shell stub, so it stays off the Windows lane

CreateProcess cannot exec the fixture's ./evals.sh shebang script (status
null on windows-free-shard 6). The measure runner is covered on Windows by
ship-measure.test.ts, which uses in-process fake runners.

* fix: the free-text block refuses an unset temp dir instead of writing at /

The rendered block ran mktemp "$_GT/..." with _GT unset in the portability
test; on Linux that fails at /, but Git Bash on Windows can write to its
install root, so the block went on (windows-free-shard 3). It now uses
${_GT:?}, so an unset directory always fails closed with the mktemp
message.

* test: pin the guarded ${_GT:?} temp-file form

* test: the Greptile check's gh stub uses a portable shebang

On Windows the absolute bash path has a space, so the stub never ran and
every case read 'gh not logged in' (windows-free-shard 1).

* test: the Greptile check runs with the native PATH and the gh stub first

The PATH-only stub bin could not run its stub on Windows (every case read
'gh not logged in'). Stub cases now keep the real PATH with the stub first;
only the no-gh case drops directories holding a real gh.

---------

Co-authored-by: marcosdetry <marcosdetry@users.noreply.github.com>
Co-authored-by: akshaykalia <akshaykalia@users.noreply.github.com>
Co-authored-by: scottgigante-hubflow <scottgigante-hubflow@users.noreply.github.com>
Co-authored-by: woodsonl <woodsonl@users.noreply.github.com>
Co-authored-by: bezdekt <bezdekt@users.noreply.github.com>
Co-authored-by: dgrant <dgrant@users.noreply.github.com>
Co-authored-by: venkatareddyg <venkatareddyg@users.noreply.github.com>
Co-authored-by: CharlesBerg85 <CharlesBerg85@users.noreply.github.com>
Co-authored-by: VaibDeveloper <VaibDeveloper@users.noreply.github.com>
Co-authored-by: Hagetino <Hagetino@users.noreply.github.com>
Co-authored-by: Rekin226 <Rekin226@users.noreply.github.com>
Co-authored-by: ekson73 <ekson73@users.noreply.github.com>
Co-authored-by: tianxchen <tianxchen@users.noreply.github.com>
Co-authored-by: BenjaminDSmithy <BenjaminDSmithy@users.noreply.github.com>
Co-authored-by: ubay887 <ubay887@users.noreply.github.com>
Co-authored-by: nareshh74 <nareshh74@users.noreply.github.com>
Co-authored-by: syong <syong@users.noreply.github.com>
2026-10-06 23:42:19 +00:00

4893 lines
240 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { describe, test, expect, afterAll } from 'bun:test';
import { assertSinglePreamble } from '../scripts/gen-skill-docs';
import { validateSkillFrontmatter } from '../scripts/skill-check';
import { externalHostPathLeaks } from './helpers/skill-parser';
import { COMMAND_DESCRIPTIONS } from '../browse/src/commands';
import { SNAPSHOT_FLAGS } from '../browse/src/snapshot';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { spawnSync } from 'child_process';
import { runCapturedCommand } from './helpers/sync-command-capture';
import { expectMentions, expectTokens } from './helpers/prompt-structure';
const ROOT = path.resolve(import.meta.dir, '..');
const MAX_SKILL_DESCRIPTION_LENGTH = 1024;
// Carved-skill aware (v2 plan T9): ship is now a skeleton SKILL.md + sections/*.md.
// Read the union so assertions about content that MOVED into a section still pass.
// The skeleton is a subset of the union, so skeleton-only assertions also hold,
// and negative assertions stay safe (the absent phrases live in neither file).
function readSkillUnion(skill: string): string {
let t = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
const secDir = path.join(ROOT, skill, 'sections');
if (fs.existsSync(secDir)) {
for (const f of fs.readdirSync(secDir).sort()) {
if (f.endsWith('.md')) t += '\n' + fs.readFileSync(path.join(secDir, f), 'utf-8');
}
}
return t;
}
function readShipUnion(): string {
return readSkillUnion('ship');
}
function ordered(text: string, anchors: string[]): void {
const positions = anchors.map(anchor => text.indexOf(anchor));
expect(positions.every(position => position >= 0), `missing one of ${JSON.stringify(anchors)}`).toBe(true);
expect(positions).toEqual([...positions].sort((a, b) => a - b));
}
function readExternalSkillUnion(output: string, hostSubdir: string, skill: string): string {
const dir = path.join(output, hostSubdir, 'skills', `gstack-${skill}`);
const sections = path.join(dir, 'sections');
return fs.readFileSync(path.join(dir, 'SKILL.md'), 'utf-8')
+ (fs.existsSync(sections) ? fs.readdirSync(sections).sort()
.filter(file => file.endsWith('.md'))
.map(file => '\n' + fs.readFileSync(path.join(sections, file), 'utf-8')).join('') : '');
}
// Token-reduction Phase 1: the preamble's inline bash (session bookkeeping,
// config echoes, telemetry producers, artifacts sync) moved into
// bin/gstack-skill-start / bin/gstack-skill-end. The render carries a one-line
// invocation fence + interpretation prose. Assertions that pinned inline-bash
// internals now pin the scripts (the new home); render-side assertions pin the
// fence + prose. Script behavior is pinned by test/gstack-skill-start.test.ts.
const SKILL_START_SCRIPT = fs.readFileSync(path.join(ROOT, 'bin', 'gstack-skill-start'), 'utf-8');
const SKILL_END_SCRIPT = fs.readFileSync(path.join(ROOT, 'bin', 'gstack-skill-end'), 'utf-8');
function extractDescription(content: string): string {
const fmEnd = content.indexOf('\n---', 4);
expect(fmEnd).toBeGreaterThan(0);
const frontmatter = content.slice(4, fmEnd);
const lines = frontmatter.split('\n');
let description = '';
let inDescription = false;
const descLines: string[] = [];
for (const line of lines) {
if (line.match(/^description:\s*\|?\s*$/)) {
inDescription = true;
continue;
}
if (line.match(/^description:\s*\S/)) {
return line.replace(/^description:\s*/, '').trim();
}
if (inDescription) {
if (line === '' || line.match(/^\s/)) {
descLines.push(line.replace(/^ /, ''));
} else {
break;
}
}
}
if (descLines.length > 0) {
description = descLines.join('\n').trim();
}
return description;
}
function extractMarkdownSection(content: string, heading: string): string {
const escaped = heading.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const startMatch = content.match(new RegExp(`^${escaped}.*$`, 'm'));
expect(startMatch?.index).toBeDefined();
const start = startMatch!.index!;
const afterHeading = start + startMatch![0].length;
const nextSection = content.slice(afterHeading).match(/\n## /);
const end = nextSection?.index === undefined
? content.length
: afterHeading + nextSection.index;
return content.slice(start, end).trim();
}
function extractPreambleBeforeWorkflow(content: string, workflowMarkers: string[]): string {
const markerIndexes = workflowMarkers
.map(marker => content.indexOf(marker))
.filter(index => index >= 0);
expect(markerIndexes.length).toBeGreaterThan(0);
return content.slice(0, Math.min(...markerIndexes));
}
function isRepoRootSymlink(candidateDir: string): boolean {
try {
return fs.realpathSync(candidateDir) === fs.realpathSync(ROOT);
} catch {
return false;
}
}
// Dynamic template discovery — matches the generator's findTemplates() behavior.
// New skills automatically get test coverage without updating a static list.
const ALL_SKILLS = (() => {
const skills: Array<{ dir: string; name: string }> = [];
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
skills.push({ dir: '.', name: 'root gstack' });
}
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
if (fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) {
skills.push({ dir: entry.name, name: entry.name });
}
}
return skills;
})();
// The claude host deliberately skips some skills (skipSkills — e.g. the
// /claude-code outside-voice skill exists only for non-Claude hosts), so those
// dirs have a SKILL.md.tmpl but no generated claude-host SKILL.md on a fresh
// checkout. Every generated-file assertion must exclude them or it is red on
// every clean clone (it was, invisibly, until the free suite ran in CI).
import { getHostConfig as __getHostConfig } from '../hosts/index';
const CLAUDE_SKIPPED = new Set(__getHostConfig('claude').generation.skipSkills ?? []);
const CLAUDE_GENERATED_SKILLS = ALL_SKILLS.filter(s => !CLAUDE_SKIPPED.has(s.dir));
// ─── Out-dir render isolation ────────────────────────────────
// Every generator invocation in this file that used to regenerate the live
// tree (the gitignored .agents/.factory/... host dirs included) now renders
// into this module-level out-dir: ONE `--host all` render covers the claude
// host plus every external host, and all golden-artifact reads plus the
// per-host `--dry-run` determinism checks point here. The tracked tree is
// only ever READ (the `generated files are fresh` dry-run deliberately
// compares against the committed files — that is a read, not a write).
// Out-dir renders of external hosts are byte-identical to in-place renders
// (pinned by test/gen-skill-docs-out-dir.test.ts).
const EXTERNAL_OUT = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-gen-docs-out-'));
{
const render = Bun.spawnSync(
['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'all', '--out-dir', EXTERNAL_OUT],
{ cwd: ROOT, stdout: 'pipe', stderr: 'pipe', timeout: 120_000 },
);
if (render.exitCode !== 0) {
throw new Error(
`gen-skill-docs --host all --out-dir failed (exit ${render.exitCode}):\n${render.stderr.toString()}`,
);
}
}
afterAll(() => {
fs.rmSync(EXTERNAL_OUT, { recursive: true, force: true });
});
describe('gen-skill-docs', () => {
// Browse carve (token-reduction Phase 4): the command reference + snapshot
// flags render into browse/sections/command-list.md now — read the
// skeleton+sections union so these pins hold across the carve.
test('generated SKILL.md contains all command categories', () => {
const content = readSkillUnion('browse');
const categories = new Set(Object.values(COMMAND_DESCRIPTIONS).map(d => d.category));
for (const cat of categories) {
expect(content).toContain(`### ${cat}`);
}
});
test('generated SKILL.md contains all commands', () => {
const content = readSkillUnion('browse');
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
const display = meta.usage || cmd;
expect(content).toContain(display);
}
});
test('command table is sorted alphabetically within categories', () => {
const content = readSkillUnion('browse');
// Extract command names from the Navigation section as a test
const navSection = content.match(/### Navigation\n\|.*\n\|.*\n([\s\S]*?)(?=\n###|\n## )/);
expect(navSection).not.toBeNull();
const rows = navSection![1].trim().split('\n');
const commands = rows.map(r => {
const match = r.match(/\| `(\w+)/);
return match ? match[1] : '';
}).filter(Boolean);
const sorted = [...commands].sort();
expect(commands).toEqual(sorted);
});
test('snapshot flags section contains all flags', () => {
const content = readSkillUnion('browse');
for (const flag of SNAPSHOT_FLAGS) {
expect(content).toContain(flag.short);
expect(content).toContain(flag.description);
}
});
test('every skill has a SKILL.md.tmpl template', () => {
for (const skill of ALL_SKILLS) {
const tmplPath = path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
expect(fs.existsSync(tmplPath)).toBe(true);
}
});
test('every skill has a generated SKILL.md with auto-generated header', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const mdPath = path.join(ROOT, skill.dir, 'SKILL.md');
expect(fs.existsSync(mdPath)).toBe(true);
const content = fs.readFileSync(mdPath, 'utf-8');
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
expect(content).toContain('Regenerate: bun run gen:skill-docs');
}
});
// #1778: strict YAML parsers (Codex/OpenAI skill loading) reject frontmatter
// whose plain `description:` scalar contains an interior ": " (read as a nested
// mapping). Parse EVERY generated frontmatter block with a strict YAML parser,
// not just string-check that name:/description: exist.
test('every generated SKILL.md frontmatter parses as strict YAML', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
expect(validateSkillFrontmatter(content),
`frontmatter for ${skill.dir} must be valid YAML with string name/description`).toEqual([]);
}
});
test('every generated Codex (.agents/skills) frontmatter parses as strict YAML', () => {
// Reads the module-level out-dir render (guaranteed present — the render
// throws at module load if it fails), never the live gitignored tree.
const agentsDir = path.join(EXTERNAL_OUT, '.agents', 'skills');
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
if (!entry.isDirectory()) continue;
const mdPath = path.join(agentsDir, entry.name, 'SKILL.md');
if (!fs.existsSync(mdPath)) continue;
expect(validateSkillFrontmatter(fs.readFileSync(mdPath, 'utf-8')),
`Codex frontmatter for ${entry.name} must be valid YAML with string name/description`).toEqual([]);
}
});
test(`every generated SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
const description = extractDescription(content);
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
}
});
test(`every Codex SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
const agentsDir = path.join(EXTERNAL_OUT, '.agents', 'skills');
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
if (!entry.isDirectory()) continue;
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
const description = extractDescription(content);
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
}
});
test('every Codex SKILL.md description stays under 900-char warning threshold', () => {
const WARN_THRESHOLD = 900;
const agentsDir = path.join(EXTERNAL_OUT, '.agents', 'skills');
const violations: string[] = [];
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
if (!entry.isDirectory()) continue;
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
const description = extractDescription(content);
if (description.length > WARN_THRESHOLD) {
violations.push(`${entry.name}: ${description.length} chars (limit ${MAX_SKILL_DESCRIPTION_LENGTH}, ${MAX_SKILL_DESCRIPTION_LENGTH - description.length} remaining)`);
}
}
expect(violations).toEqual([]);
});
test('package.json version matches VERSION file (npm-valid translation)', () => {
// Decision 11 (v1.67 wave): VERSION stays the 4-digit source of truth;
// package.json carries the npm-valid 3-digit translation (npm rejects a
// fourth component). The pre-v1.67 1:1 four-digit mirror is also accepted
// (grandfathered until the next write), matching gstack-version-bump's
// own drift contract.
const pkg = JSON.parse(fs.readFileSync(path.join(ROOT, 'package.json'), 'utf-8'));
const version = fs.readFileSync(path.join(ROOT, 'VERSION'), 'utf-8').trim();
const npmTranslation = version.split('.').slice(0, 3).join('.');
expect([npmTranslation, version]).toContain(pkg.version);
});
test('generated files are fresh (match --dry-run)', () => {
// Deliberately compares against the LIVE TRACKED SKILL.md files (no
// --out-dir): this is the freshness gate for the committed tree. Dry-run
// writes nothing — it is a read.
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--dry-run'], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
timeout: 120_000,
});
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
// Every skill should be FRESH
for (const skill of CLAUDE_GENERATED_SKILLS) {
const file = skill.dir === '.' ? 'SKILL.md' : `${skill.dir}/SKILL.md`;
expect(output).toContain(`FRESH: ${file}`);
}
expect(output).not.toContain('STALE');
});
test('no generated SKILL.md contains unresolved placeholders', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
const unresolved = content.match(/\{\{\w+\}\}/g);
expect(unresolved).toBeNull();
}
});
test('templates contain placeholders', () => {
// P2 (v1.2.0): the root template is a pure router — only {{PREAMBLE}}.
// The browse command/snapshot placeholders live in browse/SKILL.md.tmpl now.
const rootTmpl = fs.readFileSync(path.join(ROOT, 'SKILL.md.tmpl'), 'utf-8');
expect(rootTmpl).toContain('{{PREAMBLE}}');
expect(rootTmpl).not.toContain('{{COMMAND_REFERENCE}}');
expect(rootTmpl).not.toContain('{{SNAPSHOT_FLAGS}}');
// Browse carve: the reference resolvers moved into the on-demand section
// template (so gen-skill-docs keeps them fresh from browse/src); the
// skeleton points at the section instead of inlining the reference.
const browseTmpl = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md.tmpl'), 'utf-8');
expect(browseTmpl).not.toContain('{{COMMAND_REFERENCE}}');
expect(browseTmpl).not.toContain('{{SNAPSHOT_FLAGS}}');
expect(browseTmpl).toContain('{{SECTION:command-list}}');
expect(browseTmpl).toContain('{{PREAMBLE}}');
const browseSectionTmpl = fs.readFileSync(
path.join(ROOT, 'browse', 'sections', 'command-list.md.tmpl'), 'utf-8');
expect(browseSectionTmpl).toContain('{{COMMAND_REFERENCE}}');
expect(browseSectionTmpl).toContain('{{SNAPSHOT_FLAGS}}');
// Aside is the primary browser: every browsing skill renders the Aside
// contract ({{ASIDE_SETUP}}); the browse binary is its fallback.
const qaTmpl = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md.tmpl'), 'utf-8');
expect(qaTmpl).not.toContain('{{ASIDE_SETUP}}');
expect(qaTmpl).toContain('{{SECTION:browser-setup}}');
expect(fs.readFileSync(path.join(ROOT, 'qa/sections/browser-setup.md.tmpl'), 'utf8'))
.toContain('{{ASIDE_SETUP}}');
expect(browseTmpl).toContain('{{ASIDE_SETUP}}');
});
test('generated SKILL.md contains operational self-improvement (replaced contributor mode)', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('Contributor Mode');
expect(content).not.toContain('gstack_contributor');
expect(content).not.toContain('contributor-logs');
expect(content).toContain('Operational Self-Improvement');
expect(content).toContain('gstack-learnings-log');
// The learnings-resurface call moved from the inline preamble bash into
// the skill-start script (Phase 1) — same command, new home.
expect(SKILL_START_SCRIPT).toContain('gstack-learnings-search" --limit 3');
});
test('generated SKILL.md with LEARNINGS_LOG contains operational type', () => {
// Check a skill that has LEARNINGS_LOG (e.g., review)
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('operational');
});
test('session awareness lives in gstack-skill-start (registry touch + stale cleanup)', () => {
// The sessions registry moved from inline preamble bash into the script:
// it records the harness pid (--parent-pid identity) and expires entries
// older than 120 minutes.
expect(SKILL_START_SCRIPT).toContain('sessions/$PARENT_PID');
expect(SKILL_START_SCRIPT).toContain('-mmin +120');
// The render keeps the completion-status protocol the sessions feed into.
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('RECOMMENDATION');
});
test('branch detection lives in gstack-skill-start and is echoed as BRANCH', () => {
expect(SKILL_START_SCRIPT).toContain('_BRANCH=$(git branch --show-current');
expect(SKILL_START_SCRIPT).toContain('echo "BRANCH: $_BRANCH"');
});
// #2001: update_check: false silences the binary but the upgrade-handling
// instruction prose used to ship unconditionally. Token-reduction Phase 2
// made the gate STRUCTURAL: the prose left the renders entirely (absence is
// pinned by test/onboarding-moved-literals.test.ts) and now emits from
// gstack-skill-start's instruction layer ONLY when the update-check binary
// produced output — and that binary silences itself on update_check=false.
// Opted-out installs can never see the prose, by construction.
test('update_check opt-out gates the update binary and upgrade-flow emission (issue #2001)', () => {
// The config-echo cluster lives in gstack-skill-start: the flag is still
// read and echoed as a STATUS line for the model.
expect(SKILL_START_SCRIPT, 'script must read update_check config').toContain('_UPDATE_CHECK=$(');
expect(SKILL_START_SCRIPT, 'script must echo UPDATE_CHECK').toContain('echo "UPDATE_CHECK: $_UPDATE_CHECK"');
// Gate half 1: the update-check binary exits silently when opted out.
const updateCheck = fs.readFileSync(path.join(ROOT, 'bin', 'gstack-update-check'), 'utf-8');
expect(updateCheck, 'binary must read update_check config').toContain('get update_check');
expect(updateCheck, 'binary must exit silently on update_check=false')
.toMatch(/if \[ "\$_UC" = "false" \]; then\n\s*exit 0/);
// Gate half 2: the upgrade-flow instruction block emits only when the
// binary emitted something (empty when opted out, cached, or up to date).
expect(SKILL_START_SCRIPT, 'upgrade-flow must be gated on update-check output')
.toMatch(/if \[ -n "\$_UPD" \]; then\n\s*_emit_block upgrade-flow/);
});
test('tier 2+ skills contain ELI10 simplification rules (AskUserQuestion format)', () => {
// Root SKILL.md is tier 1 and CSO intentionally uses a private startup with
// no shared PREAMBLE. Check a regular tier 2+ PREAMBLE consumer instead.
// v1.7.0.0 Pros/Cons format uses "ELI10 (ALWAYS)" rather than "Simplify (ELI10".
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('ELI10');
expect(content).toContain('plain English');
expect(content).toContain('not function names');
});
test('tier 1 skills do NOT contain AskUserQuestion format', () => {
// Use benchmark (tier 1) instead of root — root SKILL.md gets overwritten by Codex test setup
const content = fs.readFileSync(path.join(ROOT, 'benchmark', 'SKILL.md'), 'utf-8');
expect(content).not.toContain('## AskUserQuestion Format');
expect(content).not.toContain('## Completeness Principle');
});
test('telemetry producer lives in the scripts; render documents the analytics sink', () => {
// The skill-usage.jsonl producers moved into the scripts (Phase 1).
expect(SKILL_START_SCRIPT).toContain('analytics/skill-usage.jsonl');
expect(SKILL_END_SCRIPT).toContain('analytics/skill-usage.jsonl');
// The render still tells the model where telemetry lands.
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('$GSTACK_STATE_ROOT/analytics/');
});
test('plan-review generated preambles stay under the Option A budget', () => {
const reviewSkills = [
{
path: path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
markers: ['# Mega Plan Review Mode', '## Step 0: Detect platform and base branch'],
},
{
path: path.join(ROOT, 'plan-eng-review', 'SKILL.md'),
markers: ['# Plan Review Mode'],
},
];
// Plan skills carry the same preamble surface as other tier-≥2 skills
// (Artifacts Sync, Context Recovery, Routing Injection are load-bearing
// functionality, not optional). Budget is set to current size + small
// headroom; ratchet down if a future slim trims real bytes.
// Ratcheted from 33000 → 35000 when the gbrain context-load block was
// added (per /sync-gbrain plan §4). Ratcheted 35000 → 36500 in v1.27.0.0
// when generate-brain-sync-block.ts gained the gbrain_mcp_mode probe +
// remote-mode ARTIFACTS_SYNC status line (Path 4 of /setup-gbrain).
// Ratcheted 36500 → 39000 in the contributor wave when #1205 added the
// \\u-escape CJK rule (rule 12 + self-check item) to the AskUserQuestion
// preamble.
// Ratcheted 39000 → 40000 in plan-tune cathedral T14: question-tuning
// resolver gained the <gstack-qid:...> marker convention + the
// (recommended) label requirement (D2 + D18 — both load-bearing for
// hook enforcement). Adds ~700 bytes.
// Ratcheted 40000 → 60000 in v1.52.0.0 cap audit: ~20K headroom so
// future preamble adds don't trip the gate on each PR. Real runaway
// (preamble doubling) still trips; normal scope growth doesn't.
for (const skill of reviewSkills) {
const content = fs.readFileSync(skill.path, 'utf-8');
const preamble = extractPreambleBeforeWorkflow(content, skill.markers);
expect(Buffer.byteLength(preamble, 'utf-8')).toBeLessThan(60_000);
}
});
test('voice and writing-style preamble sections stay compact', () => {
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
const voice = extractMarkdownSection(content, '## Voice');
const writingStyle = extractMarkdownSection(content, '## Writing Style');
expect(Buffer.byteLength(voice, 'utf-8')).toBeLessThan(3_000);
expect(Buffer.byteLength(writingStyle, 'utf-8')).toBeLessThan(2_000);
});
test('slim voice section preserves the gstack voice contract', () => {
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
const voice = extractMarkdownSection(content, '## Voice');
expect(voice).toMatch(/lead with the point|direct/i);
expect(voice).toMatch(/file|function|line|command|real numbers/i);
expect(voice).toMatch(/user.*outcome|user.*experience|real user/i);
expect(voice).toMatch(/corporate|academic|PR|hype/i);
expect(voice).toMatch(/AI vocabulary|delve|crucial|robust/i);
expect(voice).toMatch(/user decides|user.*context|sovereignty|recommendation, not a decision/i);
});
test('preamble .pending-* glob is zsh-safe (uses find, not shell glob)', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
if (!content.includes('.pending-')) continue;
// Must NOT have a bare shell glob ".pending-*" outside of find's -name argument
expect(content).not.toMatch(/for _PF in [^\n]*\/\.pending-\*/);
// Must use find to avoid zsh NOMATCH error on glob expansion
expect(content).toContain("find ~/.gstack/analytics -maxdepth 1 -name '.pending-*'");
}
});
test('bash blocks with shell globs are zsh-safe (setopt guard or find)', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
const bashBlocks = [...content.matchAll(/```bash\n([\s\S]*?)```/g)].map(m => m[1]);
for (const block of bashBlocks) {
const lines = block.split('\n');
for (const line of lines) {
const trimmed = line.trimStart();
if (trimmed.startsWith('#')) continue;
if (!trimmed.includes('*')) continue;
// Skip lines where * is inside find -name, git pathspecs, or $(find)
if (/\bfind\b/.test(trimmed)) continue;
if (/\bgit\b/.test(trimmed)) continue;
if (/\$\(find\b/.test(trimmed)) continue;
// Check 1: "for VAR in <glob>" must use $(find ...) — caught above by the
// $(find check, so any surviving for-in with a glob pattern is a violation
if (/\bfor\s+\w+\s+in\b/.test(trimmed) && /\*\./.test(trimmed)) {
throw new Error(
`Unsafe for-in glob in ${skill.dir}/SKILL.md: "${trimmed}". ` +
`Use \`for f in $(find ... -name '*.ext')\` for zsh compatibility.`
);
}
// Check 2: ls/cat/rm/grep with glob file args must have setopt guard
const isGlobCmd = /\b(?:ls|cat|rm|grep)\b/.test(trimmed) &&
/(?:\/\*[a-z.*]|\*\.[a-z])/.test(trimmed);
if (isGlobCmd) {
expect(block).toContain('setopt +o nomatch');
}
}
}
}
});
test('preamble-using skills have correct skill name in telemetry', () => {
const PREAMBLE_SKILLS = [
{ dir: '.', name: 'gstack' },
{ dir: 'ship', name: 'ship' },
{ dir: 'review', name: 'review' },
{ dir: 'qa', name: 'qa' },
{ dir: 'retro', name: 'retro' },
];
for (const skill of PREAMBLE_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
// The skill name now travels as --skill into gstack-skill-start (the
// preamble fence) and gstack-skill-end (the telemetry epilogue) — the
// scripts write it into the JSONL events.
expect(content, `${skill.dir} preamble fence must pass its own name`)
.toMatch(new RegExp(`--skill "${skill.name}" --model`));
expect(content, `${skill.dir} epilogue must pass its own name`)
.toContain(`gstack-skill-end --skill "${skill.name}"`);
}
});
test('qa and qa-only load the shared QA_METHODOLOGY through exploration', () => {
const qaSkeletonTmpl = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md.tmpl'), 'utf-8');
expect(qaSkeletonTmpl).toContain('{{SECTION:exploratory}}');
expect(qaSkeletonTmpl).not.toContain('{{QA_METHOD_READS}}');
expect(qaSkeletonTmpl).not.toContain('{{QA_METHODOLOGY}}');
const qaSectionTmpl = fs.readFileSync(path.join(ROOT, 'qa', 'sections', 'qa-patterns.md.tmpl'), 'utf-8');
expect(qaSectionTmpl).toContain('{{QA_METHODOLOGY}}');
const qaOnlyTmpl = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md.tmpl'), 'utf-8');
expect(qaOnlyTmpl).not.toContain('{{QA_METHODOLOGY}}');
expect(qaOnlyTmpl).toContain('{{SECTION:exploratory}}');
expect(qaOnlyTmpl).not.toContain('{{QA_METHOD_READS}}');
expect(qaOnlyTmpl).toContain('Load the shared preparation gate now');
for (const skill of ['qa', 'qa-only']) {
expect(fs.readFileSync(path.join(ROOT, skill, 'sections/exploratory.md.tmpl'), 'utf8'))
.toContain('{{QA_EXPLORATORY}}');
const entry = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf8');
const explorer = fs.readFileSync(path.join(ROOT, skill, 'sections/exploratory.md'), 'utf8');
const directoryRead = skill === 'qa'
? 'Read `sections/scope.md` relative to the installed `qa`/`gstack-qa` SKILL.md directory'
: "Use this host's installed `qa`/`gstack-qa` SKILL.md directory for these reads";
expect(entry).toContain('sections/exploratory.md');
expect(explorer).toContain(directoryRead);
for (const method of ['system-functional', 'qa-patterns']) {
const methodRead = `Read \`sections/${method}.md\` in full.`;
expect(entry).not.toContain(methodRead);
expect(explorer).toContain(methodRead);
expect(explorer.split(methodRead)).toHaveLength(2);
expect(explorer.indexOf(directoryRead)).toBeLessThan(explorer.indexOf(methodRead));
const directory = path.resolve(ROOT, skill, skill === 'qa-only' ? '../qa' : '.');
expect(fs.realpathSync(path.join(directory, 'sections', `${method}.md`)))
.toBe(path.join(ROOT, 'qa', 'sections', `${method}.md`));
}
expect(explorer).toContain('**Browser surfaces only:**\nRead `sections/qa-patterns.md` in full.');
expectMentions(explorer, [['before','complete','charters']], 'explorer');
expectMentions(explorer, [['do not','invocation','completed']], 'explorer');
expect(explorer.indexOf('Read `sections/qa-patterns.md`')).toBeLessThan(explorer.indexOf('Write a **charter**'));
}
});
test('QA_METHODOLOGY is expanded in the shared resource referenced by qa and qa-only', () => {
const qaContent = readSkillUnion('qa'); // carved: methodology lives in qa/sections/qa-patterns.md
const qaOnlyUnion = readSkillUnion('qa-only');
expect(qaOnlyUnion).toContain("Use this host's installed `qa`/`gstack-qa` SKILL.md directory for these reads");
expect(qaOnlyUnion).toContain('**Browser surfaces only:**\nRead `sections/qa-patterns.md` in full.');
expect(qaOnlyUnion).not.toContain('Health Score Rubric');
const qaOnlyContent = qaOnlyUnion + fs.readFileSync(path.join(ROOT, 'qa/sections/qa-patterns.md'), 'utf8');
// Both should contain the health score rubric
expect(qaContent).toContain('Health Score Rubric');
expect(qaOnlyContent).toContain('Health Score Rubric');
// Both should contain framework guidance
expect(qaContent).toContain('Framework-Specific Guidance');
expect(qaOnlyContent).toContain('Framework-Specific Guidance');
// Both should contain the important rules
expect(qaContent).toContain('Important Rules');
expect(qaOnlyContent).toContain('Important Rules');
// Both should contain the 6 phases
expect(qaContent).toContain('Phase 1');
expect(qaOnlyContent).toContain('Phase 1');
expect(qaContent).toContain('Phase 6');
expect(qaOnlyContent).toContain('Phase 6');
});
test('qa-only has no-fix guardrails', () => {
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
expect(extractMarkdownSection(qaOnlyContent, '## Additional Rules (qa-only specific)')).toMatch(/never fix/i);
// Should not have Edit, Glob, or Grep in allowed-tools.
// Scope to frontmatter (between the first two --- lines) — the body can
// legitimately mention these tool names in prose (e.g., Claude model
// overlay names the host's dedicated file and search tools).
const fmMatch = qaOnlyContent.match(/^---\n([\s\S]*?)\n---/);
expect(fmMatch).not.toBeNull();
const frontmatter = fmMatch![1];
expect(frontmatter).toMatch(/allowed-tools:/);
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Edit/);
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Glob/);
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Grep/);
});
test('qa has fix-loop tools and phases', () => {
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
// Should have Edit, Glob, Grep in allowed-tools
expect(qaContent).toContain('Edit');
expect(qaContent).toContain('Glob');
expect(qaContent).toContain('Grep');
// Should have fix-loop phases
expect(qaContent).toContain('Phase 7');
expect(qaContent).toContain('Phase 8');
expect(qaContent).toContain('Fix Loop');
expect(qaContent).toContain('Triage');
expect(qaContent).toMatch(/every 5 fixes/i);
});
});
describe('BASE_BRANCH_DETECT resolver', () => {
// Find a generated SKILL.md that uses the placeholder (ship is guaranteed to)
const shipContent = readShipUnion();
test('resolver output contains PR base detection command', () => {
expect(shipContent).toContain('gh pr view --json baseRefName');
});
test('resolver output contains repo default branch detection command', () => {
expect(shipContent).toContain('gh repo view --json defaultBranchRef');
});
test('resolver output contains fallback to main', () => {
expect(shipContent).toMatch(/fall\s*back\s+to\s+`main`/i);
});
test('resolver output uses "the base branch" phrasing', () => {
expect(shipContent).toContain('the base branch');
});
test('resolver output contains GitLab CLI commands', () => {
expect(shipContent).toContain('glab');
});
test('resolver output contains git-native fallback', () => {
expect(shipContent).toContain('git symbolic-ref');
});
test('resolver output mentions GitLab platform', () => {
expect(shipContent).toMatch(/gitlab/i);
});
});
describe('GitLab support in generated skills', () => {
const retroContent = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
const shipSkillContent = readShipUnion();
test('retro contains GitLab MR number extraction', () => {
expect(retroContent).toContain('[#!]');
});
test('retro uses BASE_BRANCH_DETECT (contains glab)', () => {
expect(retroContent).toContain('glab');
});
test('ship contains glab mr create', () => {
expect(shipSkillContent).toContain('glab mr create');
});
test('ship checks .gitlab-ci.yml', () => {
expect(shipSkillContent).toContain('.gitlab-ci.yml');
});
});
/**
* Quality evals — catch description regressions.
*
* These test that generated output is *useful for an AI agent*,
* not just structurally valid. Each test targets a specific
* regression we actually shipped and caught in review.
*/
describe('description quality evals', () => {
// Regression: snapshot flags lost value hints (-d <N>, -s <sel>, -o <path>)
// Browse carve: the flag reference renders into browse/sections/command-list.md.
test('snapshot flags with values include value hints in output', () => {
const content = readSkillUnion('browse');
for (const flag of SNAPSHOT_FLAGS) {
if (flag.takesValue) {
expect(flag.valueHint).toBeDefined();
expect(content).toContain(`${flag.short} ${flag.valueHint}`);
}
}
});
// Regression: "is" lost the valid states enum
test('is command lists valid state values', () => {
const desc = COMMAND_DESCRIPTIONS['is'].description;
for (const state of ['visible', 'hidden', 'enabled', 'disabled', 'checked', 'editable', 'focused']) {
expect(desc).toContain(state);
}
});
// Regression: "press" lost common key examples
test('press command lists example keys', () => {
const desc = COMMAND_DESCRIPTIONS['press'].description;
expect(desc).toContain('Enter');
expect(desc).toContain('Tab');
expect(desc).toContain('Escape');
});
// Regression: "console" lost --errors filter note
test('console command describes --errors behavior', () => {
const desc = COMMAND_DESCRIPTIONS['console'].description;
expect(desc).toContain('--errors');
});
// Regression: snapshot -i lost "@e refs" context
test('snapshot -i mentions @e refs', () => {
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-i')!;
expect(flag.description).toContain('@e');
});
// Regression: snapshot -C lost "@c refs" context
test('snapshot -C mentions @c refs', () => {
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-C')!;
expect(flag.description).toContain('@c');
});
// Guard: every description must be at least 8 chars (catches empty or stub descriptions)
test('all command descriptions have meaningful length', () => {
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
expect(meta.description.length).toBeGreaterThanOrEqual(8);
}
});
// Guard: snapshot flag descriptions must be at least 10 chars
test('all snapshot flag descriptions have meaningful length', () => {
for (const flag of SNAPSHOT_FLAGS) {
expect(flag.description.length).toBeGreaterThanOrEqual(10);
}
});
// Guard: descriptions must not contain pipe (breaks markdown table cells)
// Usage strings are backtick-wrapped in the table so pipes there are safe.
test('no command description contains pipe character', () => {
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
expect(meta.description).not.toContain('|');
}
});
// Guard: generated output uses → not ->
test('generated SKILL.md uses unicode arrows', () => {
// P2 (v1.2.0): the browse body moved out of the top-level router into
// browse/SKILL.md. Guard arrow style on the browse body (sliced from its
// H1 so the auto-generated `-->` header comments are excluded).
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
const h1 = content.search(/^# browse: /m);
expect(h1).toBeGreaterThan(-1);
const body = content.slice(h1);
expect(body).toContain('→');
expect(body).not.toContain('->');
});
});
describe('REVIEW_DASHBOARD resolver', () => {
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review'];
for (const skill of REVIEW_SKILLS) {
test(`review dashboard appears in ${skill} generated file`, () => {
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
expect(content).toContain('gstack-review');
expect(content).toContain('REVIEW READINESS DASHBOARD');
});
}
test('review dashboard appears in ship generated file', () => {
const content = readShipUnion();
// The dashboard reads the review log through its reader; the literal
// reviews.jsonl filename used to come only from the Context Recovery fence.
expect(extractMarkdownSection(content, '## Review Readiness Dashboard')).toContain('bin/gstack-review-read');
expect(content).toContain('REVIEW READINESS DASHBOARD');
});
test('dashboard treats review as a valid Eng Review source', () => {
const content = readShipUnion();
expect(content).toContain('| Eng Review | `review` or `plan-eng-review` | (DIFF) or (PLAN) |');
expect(content).toContain('**Content-first rule:** For `review`');
expect(content).toContain('**Plan records** (plan-ceo-review, plan-eng-review');
expect(content.replace(/\s+/g, ' ')).toContain('CLEARED requires the selected Eng Review to be `clean`, within 7 days and fresh under step 2');
});
test('shared dashboard propagates review source to plan-eng-review', () => {
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
expect(content).toContain('| Eng Review | `review` or `plan-eng-review` | (DIFF) or (PLAN) |');
expect(content).toContain('**Content-first rule:** For `review`');
});
test('resolver output contains key dashboard elements', () => {
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
expect(content).toContain('VERDICT');
expect(content).toContain('CLEARED');
expect(content).toContain('Eng Review');
expect(content).toContain('7 days');
expect(content).toContain('Design Review');
expect(content).toContain('skip_eng_review');
});
test('dashboard bash block includes git HEAD for staleness detection', () => {
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
expect(content).toContain('git rev-parse --short HEAD');
expect(content).toContain('---HEAD---');
});
test('dashboard includes staleness detection prose', () => {
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
expect(content).toContain('**2. Check freshness before choosing a verdict.**');
expect(content).toContain('commit');
});
for (const skill of REVIEW_SKILLS) {
test(`${skill} contains review chaining section`, () => {
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
expect(content).toContain('Review Chaining');
});
test(`${skill} Review Log includes commit field`, () => {
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
expect(content).toContain('"commit"');
});
}
test('plan-ceo-review chaining mentions eng and design reviews', () => {
// Carved skill: the chaining prose lives in sections/*.md. (It used to
// pass against the skeleton only because the preamble's routing-injection
// rules incidentally named these skills — that prose moved into
// bin/gstack-skill-start in token-reduction Phase 2.)
const content = readSkillUnion('plan-ceo-review');
expect(content).toContain('/plan-eng-review');
expect(content).toContain('/plan-design-review');
});
test('plan-eng-review chaining mentions design and ceo reviews', () => {
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
expect(content).toContain('/plan-design-review');
expect(content).toContain('/plan-ceo-review');
});
test('plan-design-review chaining mentions eng, ceo, and design skills', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('/plan-eng-review');
expect(content).toContain('/plan-ceo-review');
expect(content).toContain('/design-shotgun');
expect(content).toContain('/design-html');
});
test('ship does NOT contain review chaining', () => {
const content = readShipUnion();
expect(content).not.toContain('Review Chaining');
});
});
// ─── Test Coverage Audit Resolver Tests ─────────────────────
describe('TEST_COVERAGE_AUDIT placeholders', () => {
const planSkill = readSkillUnion('plan-eng-review'); // carved
const shipSkill = readShipUnion();
const reviewSkill = readSkillUnion('review'); // carved: Review Army moved to sections/review-army.md
test('plan and ship modes share codepath tracing methodology', () => {
// Review mode delegates test coverage to the Testing specialist subagent (Review Army)
const normalizedPlanSkill = planSkill.replace(/\s+/g, ' ');
const normalizedShipSkill = shipSkill.replace(/\s+/g, ' ');
for (const skill of [normalizedPlanSkill, normalizedShipSkill]) {
ordered(skill, ['concrete source and test files', '**Trace data flow.**', '**Diagram the execution.**', 'Quality scoring rubric']);
for (const marker of ['★★★', '★★', 'GAP']) expect(skill).toContain(marker);
}
// Plan mode traces the plan, not a git diff
expect(planSkill).toContain('Trace every codepath in the plan');
expect(planSkill).not.toContain('git diff origin');
// Ship mode traces the diff
expect(shipSkill).toContain('Trace every codepath changed');
});
test('review mode uses Review Army for specialist dispatch', () => {
expect(reviewSkill).toContain('Review Army');
expect(reviewSkill).toContain('Specialist Dispatch');
expect(reviewSkill).toContain('testing.md');
});
test('plan and ship modes include E2E decision matrix', () => {
// Review mode delegates to Testing specialist
for (const skill of [planSkill, shipSkill]) {
expect(skill).toContain('E2E Test Decision Matrix');
expect(skill).toContain('→E2E');
expect(skill).toContain('→EVAL');
}
});
test('plan and ship modes include regression rule', () => {
// Review mode delegates to Testing specialist
for (const skill of [planSkill, shipSkill]) {
expect(skill).toMatch(/^#{3,4} REGRESSION RULE/m);
}
expectMentions(planSkill, [['without','requirement','regression']], 'planSkill');
});
test('all five plan audit subheadings stay inside Test review while ship keeps its existing depth', () => {
const tests = planSkill.split('\n### 3. Test review\n')[1]!.split('\n### 4. Performance review\n')[0]!;
const instructions = tests.replace(/```[\s\S]*?```/g, '');
const headings = ['Test Framework Detection', 'E2E Test Decision Matrix', 'REGRESSION RULE (mandatory)',
'LLM/eval scope', 'Test Plan Artifact'];
expect([...instructions.matchAll(/^#### (.+)$/gm)].map(match => match[1])).toEqual(headings);
expect(instructions).not.toMatch(/^#{1,3} /m);
for (const heading of headings.filter(heading => heading !== 'LLM/eval scope')) {
expect(shipSkill).toContain(`\n### ${heading}\n`);
expect(shipSkill).not.toContain(`\n#### ${heading}\n`);
}
});
test('planned regression coverage gets its own approved contract without making coverage optional', () => {
const regression = planSkill.split('\n#### REGRESSION RULE (mandatory)\n')[1]!.split('**Step 4.')[0]!;
expect(regression).toContain('critical requirement');
expectMentions(regression, [['not','whether','cover']], 'regression');
expectMentions(regression, [['before','approved','contract']], 'regression');
expect(regression).not.toContain('No AskUserQuestion');
expect(regression).not.toContain('the diff broke');
expectMentions(regression, [['not','running','already']], 'regression');
});
test('plan gap additions wait for their test-contract decision', () => {
const action = planSkill.split('**Step 5. Add missing tests to the plan:**')[1]!.split('\n#### Test Plan Artifact\n')[0]!;
expectMentions(action, [['until','contracts','optional']], 'action');
expectMentions(action, [['wait','applying','answer']], 'action');
expect(action).not.toContain('explain what broke');
});
test('implemented regressions still require an immediate test without a question', () => {
const regression = shipSkill.split('### REGRESSION RULE (mandatory)')[1]!.split('**4.')[0]!;
expectMentions(regression, [['no','askuserquestion']], 'regression');
});
test('plan and ship modes include test framework detection', () => {
// Review mode delegates to Testing specialist
for (const skill of [planSkill, shipSkill]) {
expect(skill).toContain('Test Framework Detection');
expect(skill).toContain('CLAUDE.md');
}
});
test('plan framework absence keeps requirements without offering test generation', () => {
const detection = planSkill.split('\n#### Test Framework Detection\n')[1]!.split('**Step 1.')[0]!;
expect(detection).toContain('State that the framework is unknown');
expectMentions(detection, [['no','selection','framework']], 'detection');
expectMentions(detection, [['do not','framework','proposed']], 'detection');
expect(detection).not.toContain('skip test generation');
});
test('plan mode adds tests to plan + includes test plan artifact', () => {
expect(planSkill).toContain('Add missing tests to the plan');
expect(planSkill).toContain('eng-review-test-plan');
expect(planSkill).toContain('Test Plan Artifact');
const artifact = planSkill.split('\n#### Test Plan Artifact\n')[1]!;
// The artifact is written even when Test review choices are still unanswered (periodic run 37151477069).
const step5 = planSkill.split('**Step 5. Add missing tests to the plan:**')[1]!.split('\n#### Test Plan Artifact\n')[0]!;
expectMentions(artifact, [['not','implementation','unresolved']], 'artifact');
expect(artifact).toContain('TEST_PLAN_USER=$(whoami)');
expect(artifact).toContain('sanitized `BRANCH` from gstack-slug');
expect(artifact).toContain('without an origin, write `local-only`');
});
test('ship mode auto-generates tests + includes before/after count', () => {
expect(shipSkill).toContain('Generate tests for uncovered paths');
expect(shipSkill).toContain('Before/after test count');
expect(shipSkill).toContain('30 code paths max');
expect(shipSkill).toContain('ship-test-plan');
});
test('review mode uses Fix-First + Review Army for specialist coverage', () => {
expect(reviewSkill).toContain('Fix-First');
expect(reviewSkill).toContain('INFORMATIONAL');
// Review Army handles test coverage via Testing specialist subagent
expect(reviewSkill).toContain('Review Army');
expect(reviewSkill).toContain('Testing');
});
test('plan mode does NOT include ship-specific content', () => {
expect(planSkill).not.toContain('Before/after test count');
expect(planSkill).not.toContain('30 code paths max');
expect(planSkill).not.toContain('ship-test-plan');
});
test('review mode does NOT include test plan artifact', () => {
expect(reviewSkill).not.toContain('Test Plan Artifact');
expect(reviewSkill).not.toContain('eng-review-test-plan');
expect(reviewSkill).not.toContain('ship-test-plan');
});
test('review/specialists/ directory has all expected checklist files', () => {
const specDir = path.join(ROOT, 'review', 'specialists');
const expected = [
'testing.md',
'maintainability.md',
'security.md',
'performance.md',
'data-migration.md',
'api-contract.md',
'simplification.md',
'red-team.md',
];
for (const f of expected) {
expect(fs.existsSync(path.join(specDir, f))).toBe(true);
}
});
// Regression pins for the simplification specialist (advisory carve-out edits
// the pre-existing quality_score instruction, so the rendered contract is
// pinned statically — the carve-out and the early-out line must both survive
// regeneration verbatim).
test('simplification advisory carve-out and early-out render into review docs', () => {
const reviewArmySection = fs.readFileSync(
path.join(ROOT, 'review', 'sections', 'review-army.md'),
'utf-8',
);
expect(reviewArmySection).toContain('"advisory": true');
expectTokens(reviewArmySection, ['`quality_score`'], 'reviewArmySection');
expectMentions(reviewArmySection, [['do not', 'specialist', 'findings']], 'reviewArmySection');
expect(reviewArmySection).toContain('Use the merged NON-advisory specialist findings for both counts and score');
expect(reviewArmySection).toContain('net: -N lines possible');
expect(reviewArmySection).toContain('--simplification');
// The specialist itself must never carry a verdict-shaped zero-findings line.
const spec = fs.readFileSync(
path.join(ROOT, 'review', 'specialists', 'simplification.md'),
'utf-8',
);
expect(spec).toContain('NO FINDINGS');
expect(spec).not.toContain('Lean already. Ship.');
// Closed tag vocabulary: the disavowed yagni: frame must not appear.
expect(spec).toContain('speculative');
expect(spec.toLowerCase()).not.toContain('"yagni"');
});
test('each specialist file has standard header with scope and output format', () => {
const specDir = path.join(ROOT, 'review', 'specialists');
const files = fs.readdirSync(specDir).filter(f => f.endsWith('.md'));
for (const f of files) {
const content = fs.readFileSync(path.join(specDir, f), 'utf-8');
// All specialist files must have Scope and Output/JSON in header
expect(content).toContain('Scope:');
expect(content.toLowerCase()).toMatch(/output|json/);
// Must define NO FINDINGS behavior
expect(content).toContain('NO FINDINGS');
}
});
// Regression guard: ship output contains key phrases from before the refactor
test('ship SKILL.md regression guard — key phrases preserved', () => {
const regressionPhrases = [
'Coverage goal: every changed behavior is protected by a test that would catch a real regression.',
'ASCII coverage diagram',
'processPayment',
'refundPayment',
'billing.test.ts',
'checkout.e2e.ts',
'COVERAGE:',
'QUALITY:',
'GAPS:',
'Code paths:',
'User flows:',
];
for (const phrase of regressionPhrases) {
expect(shipSkill).toContain(phrase);
}
});
test('ship SKILL.md contains review army specialist dispatch', () => {
expect(shipSkill).toContain('Specialist Dispatch');
expect(shipSkill).toContain('Step 9.1');
expect(shipSkill).toContain('Step 9.2');
});
test('ship SKILL.md contains cross-review finding dedup', () => {
expect(shipSkill).toContain('Cross-review finding dedup');
expect(shipSkill).toContain('Step 9.3');
});
test('ship SKILL.md contains re-run idempotency behavior', () => {
expect(shipSkill).toContain('**Route:**');
expect(shipSkill.replace(/\s+/g, ' ')).toContain('integrate (1–3) → test and review (4–11.5) → prepare the release (12–15) → verify frozen content (16) → push and publish (17–21)');
expectMentions(shipSkill.replace(/\s+/g, ' '), [['never','verification','duplicate']], 'shipSkill.replace(/\s+/g, )');
});
});
// --- {{TEST_FAILURE_TRIAGE}} resolver tests ---
describe('TEST_FAILURE_TRIAGE resolver', () => {
const shipSkill = readShipUnion();
test('contains all 4 triage steps', () => {
expect(shipSkill).toContain('Step T1: Classify each failure');
expect(shipSkill).toContain('Step T2: Handle in-branch failures');
expect(shipSkill).toContain('Step T3: Handle pre-existing failures');
expect(shipSkill).toContain('Step T4: Execute the chosen action');
});
test('T1 includes classification criteria (in-branch vs pre-existing)', () => {
expect(shipSkill).toContain('In-branch');
expect(shipSkill).toContain('Likely pre-existing');
expect(shipSkill).toContain('git diff origin/');
});
test('T3 branches on REPO_MODE (solo vs collaborative)', () => {
expect(shipSkill).toContain('REPO_MODE');
expect(shipSkill).toContain('solo');
expect(shipSkill).toContain('collaborative');
});
test('solo mode offers fix-now, TODO, and skip options', () => {
expect(shipSkill).toContain('Investigate and fix now');
expect(shipSkill).toContain('Add as P0 TODO');
expect(shipSkill).toContain('Skip');
});
test('collaborative mode offers blame + assign option', () => {
expect(shipSkill).toContain('Blame + assign GitHub issue');
expect(shipSkill).toContain('gh issue create');
});
test('defaults ambiguous failures to in-branch (safety)', () => {
expect(shipSkill).toContain('When ambiguous, default to in-branch');
});
});
// --- {{PLAN_FILE_REVIEW_REPORT}} resolver tests ---
describe('PLAN_FILE_REVIEW_REPORT resolver', () => {
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review', 'codex'];
for (const [label, skill] of [['Eng', 'plan-eng-review'], ['CEO', 'plan-ceo-review']]) {
test(`${label} report example is fenced Markdown, and the original escaped fence is rejected`, async () => {
const { marked } = await import('marked');
const content = readSkillUnion(skill!);
const tokens = marked.lexer(content);
const headings = tokens.filter(token => token.type === 'heading' && token.depth === 2);
expect(headings.map(token => token.text)).not.toContain('GSTACK REVIEW REPORT');
const example = tokens.find(token => token.type === 'code'
&& token.lang === 'markdown' && token.text.startsWith('## GSTACK REVIEW REPORT\n'));
expect(example).toBeDefined();
if (example?.type !== 'code') throw new Error('Missing fenced report example');
expect(example.text).toContain('| Review | Trigger | Why | Runs | Status | Findings |');
expect(example.text).toContain('`/' + skill + '`');
// Reproduce the observed literal-backslash delimiters without changing
// the parser or preprocessing malformed Markdown into valid fences.
const broken = content.replace(example.raw, example.raw.replaceAll('`', '\\`'));
expect(broken).not.toBe(content);
const brokenTokens = marked.lexer(broken);
expect(brokenTokens.some(token => token.type === 'code' && token.lang === 'markdown'
&& token.text.startsWith('## GSTACK REVIEW REPORT\n'))).toBe(false);
expect(brokenTokens.filter(token => token.type === 'heading' && token.depth === 2)
.map(token => token.text)).toContain('GSTACK REVIEW REPORT');
});
}
test('Eng renderers use real code delimiters without changing confidence rules or report fields', async () => {
const {generateConfidenceCalibration} = await import('../scripts/resolvers/confidence');
const {generateReviewDashboard, generatePlanFileReviewReport} = await import('../scripts/resolvers/review-dashboard');
const {generateCodexPlanReview} = await import('../scripts/resolvers/outside-voice-steps');
const {HOST_PATHS} = await import('../scripts/resolvers/types');
for (const host of ALL_HOST_CONFIGS) {
const ctx = {skillName: 'plan-eng-review', tmplPath: 'plan-eng-review/SKILL.md.tmpl',
host: host.name, paths: HOST_PATHS[host.name]!};
const confidence = generateConfidenceCalibration(ctx);
const dashboard = generateReviewDashboard(ctx);
const report = generatePlanFileReviewReport(ctx);
const outside = generateCodexPlanReview(ctx);
// These four resolver fragments contain no intentional escaped-backtick
// example; the preamble does, and is deliberately outside this check.
for (const output of [confidence, dashboard, report, outside]) expect(output).not.toContain('\\`');
expect(confidence).toBe(generateConfidenceCalibration({...ctx, skillName: 'plan-ceo-review'}).replaceAll('\\`', '`'));
const ceoDashboard = generateReviewDashboard({...ctx, skillName: 'plan-ceo-review'}).replaceAll('\\`', '`');
const ceoVoiceSource = 'Below the dashboard, group `autoplan-voices` and `design-outside-voices` by workflow run and phase';
expect(dashboard.replace(/\s+/g, ' ')).toContain(ceoVoiceSource);
expect(ceoDashboard.replace(/\s+/g, ' ')).toContain(ceoVoiceSource);
expect(ceoDashboard).toBe(dashboard);
for (const field of ['status', 'unresolved', 'critical_gaps', 'issues_found', 'mode', 'commit']) {
expect(report).toContain('`' + field + '`');
}
expect(outside).toContain(host.name === 'codex' ? 'claude auth login' : 'codex login');
expectMentions(outside, [['never','supplies','coverage']], 'outside');
}
});
for (const skill of REVIEW_SKILLS) {
test(`plan file review report appears in ${skill} generated file`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('GSTACK REVIEW REPORT');
});
}
test('resolver output contains key report elements', () => {
const content = readSkillUnion('plan-ceo-review'); // carved: report writer moved to section
expect(content).toContain('Trigger');
expect(content).toContain('Findings');
expect(content).toContain('VERDICT');
expect(content).toContain('/plan-ceo-review');
expect(content).toContain('/plan-eng-review');
expect(content).toContain('/plan-design-review');
expect(content).toContain('/codex review');
});
test('Eng task output follows the same write policy as its report', () => {
const content = readSkillUnion('plan-eng-review');
// The task example itself has an H2; bound by the following real section.
const tasks = content.slice(content.indexOf('## Implementation Tasks'), content.indexOf('## Plan File Review Report'));
expectMentions(tasks, [['only','permits','review']], 'tasks');
expectMentions(tasks, [['not','persisted','complete']], 'tasks');
expect(tasks).not.toContain('always write');
});
});
// --- {{PLAN_COMPLETION_AUDIT}} resolver tests ---
describe('PLAN_COMPLETION_AUDIT placeholders', () => {
const shipSkill = readShipUnion();
const reviewSkill = readSkillUnion('review'); // carved: plan-completion audit moved to sections/plan-completion.md
test('ship SKILL.md contains plan completion audit step', () => {
expect(shipSkill).toContain('Plan Completion Audit');
expect(shipSkill).toContain('Step 8');
});
test('review SKILL.md contains plan completion in scope drift', () => {
expect(reviewSkill).toContain('Plan File Discovery');
expect(reviewSkill).toContain('Actionable Item Extraction');
expect(reviewSkill).toContain('Integration with Scope Drift Detection');
});
test('both modes share plan file discovery methodology', () => {
expect(shipSkill).toContain('Plan File Discovery');
expect(reviewSkill).toContain('Plan File Discovery');
// Both should have conversation context first
expect(shipSkill).toContain('Conversation context (primary)');
expect(reviewSkill).toContain('Conversation context (primary)');
// Both should have grep fallback
expect(shipSkill).toContain('Content-based search (fallback)');
expect(reviewSkill).toContain('Content-based search (fallback)');
});
test('ship mode has gate logic for NOT DONE items', () => {
expect(shipSkill).toContain('NOT DONE');
expect(shipSkill).toContain('Stop — implement the missing items');
expect(shipSkill).toContain('Ship anyway — defer');
expect(shipSkill).toContain('intentionally dropped');
});
test('review mode is INFORMATIONAL only', () => {
expect(reviewSkill).toContain('INFORMATIONAL');
expect(reviewSkill).toContain('MISSING REQUIREMENTS');
expect(reviewSkill).toContain('SCOPE CREEP');
});
test('item extraction has 50-item cap', () => {
expect(shipSkill).toContain('at most 50 items');
});
test('uses file-level traceability (not commit-level)', () => {
expect(shipSkill).toContain('Cite the specific file');
expect(shipSkill).not.toContain('commit-level traceability');
});
});
// --- {{PLAN_VERIFICATION_EXEC}} resolver tests ---
describe('PLAN_VERIFICATION_EXEC placeholder', () => {
const shipSkill = readShipUnion();
test('ship SKILL.md contains plan verification step', () => {
expect(shipSkill).toContain('Step 8.1');
expect(shipSkill).toContain('Plan Verification');
});
test('references the shared explorer without invoking an entire QA workflow', () => {
const resource = "From the installed /ship SKILL.md's directory, Read `../qa/sections/exploratory.md` in full";
expect(shipSkill).toContain(resource);
const load = shipSkill.indexOf(resource);
const preflight = shipSkill.indexOf('Run the shared preflight;');
const probes = shipSkill.indexOf('**3. Run smoke and plan checks.**');
expect(load).toBeGreaterThan(-1);
expect(preflight).toBeGreaterThan(load);
expect(probes).toBeGreaterThan(preflight);
const shared = fs.readFileSync(path.join(ROOT, 'qa/sections/exploratory.md'), 'utf8');
const selection = shared.indexOf('in full and select the surfaces');
const methods = shared.indexOf('Read `sections/system-functional.md`');
expect(selection).toBeGreaterThan(shared.indexOf('Read `sections/scope.md`'));
expect(methods).toBeGreaterThan(selection);
expect(shared.indexOf('Write a **charter**')).toBeGreaterThan(methods);
expect(shipSkill.slice(preflight, probes)).toContain("For browsers, Read QA's `sections/browser-setup.md`");
expectMentions(shipSkill, [['do not','invoke','entire']], 'shipSkill');
});
test('keeps declared browser URLs separate from native functional probes', () => {
expectMentions(shipSkill, [['without','discovering','functional']], 'shipSkill');
expectMentions(shipSkill.replace(/\s+/g, ' '), [['not', 'automatically', 'page']], 'shipSkill.replace(/\s+/g, )');
});
test('retains automatic exploration when there is no plan or verification section', () => {
expectMentions(shipSkill, [['no','verification','section']], 'shipSkill');
expect(shipSkill).toContain('Automatic diff-scoped QA still runs');
});
test('blocks unavailable required checks instead of silently skipping them', () => {
const flat = shipSkill.replace(/\s+/g, ' ');
expect(flat).toContain('Noninteractive runs return blocked');
expectMentions(flat, [['never','required-probe','silently']], 'flat');
expect(flat).toContain('Missing/unreadable assets block required QA');
expect(flat).toContain('Keep actual outcomes and incomplete flags; VERIFY_RESULT stays fail');
});
});
// --- Coverage gate tests ---
describe('Coverage gate in ship', () => {
const shipSkill = readShipUnion();
const reviewSkill = readSkillUnion('review'); // carved: testing.md specialist ref lives in sections/review-army.md
test('ship SKILL.md contains coverage gate with thresholds', () => {
expect(shipSkill).toContain('Coverage gate');
expect(shipSkill).toContain('>= target');
expect(shipSkill).toContain('< minimum');
});
test('ship SKILL.md supports configurable thresholds via CLAUDE.md', () => {
expect(shipSkill).toContain('## Test Coverage');
expect(shipSkill).toContain('Minimum:');
expect(shipSkill).toContain('Target:');
});
test('coverage gate skips on parse failure (not block)', () => {
expect(shipSkill).toContain('could not determine percentage — skipping');
});
test('review SKILL.md delegates coverage to Testing specialist', () => {
// Coverage audit moved to Testing specialist subagent in Review Army
expect(reviewSkill).toContain('testing.md');
expect(reviewSkill).toContain('INFORMATIONAL');
});
});
// --- Ship metrics logging ---
describe('Ship metrics logging', () => {
const shipSkill = readShipUnion();
test('ship SKILL.md contains metrics persistence step', () => {
expect(shipSkill).toContain('Step 20');
expect(shipSkill).toContain('coverage_pct');
expect(shipSkill).toContain('plan_items_total');
expect(shipSkill).toContain('plan_items_done');
expect(shipSkill).toContain('verification_result');
});
});
// --- Plan file discovery shared helper ---
describe('Plan file discovery shared helper', () => {
// The shared helper should appear in ship (via PLAN_COMPLETION_AUDIT_SHIP)
// and in review (via PLAN_COMPLETION_AUDIT_REVIEW)
const shipSkill = readShipUnion();
const reviewSkill = readSkillUnion('review'); // carved: plan-completion audit moved to sections/plan-completion.md
test('plan file discovery appears in both ship and review', () => {
expect(shipSkill).toContain('Plan File Discovery');
expect(reviewSkill).toContain('Plan File Discovery');
});
test('both include conversation context first', () => {
expect(shipSkill).toContain('Conversation context (primary)');
expect(reviewSkill).toContain('Conversation context (primary)');
});
test('both include content-based fallback', () => {
expect(shipSkill).toContain('Content-based search (fallback)');
expect(reviewSkill).toContain('Content-based search (fallback)');
});
});
// --- Retro plan completion ---
describe('Retro plan completion section', () => {
// Carved: the narrative report format (incl. Plan Completion) lives in
// retro/sections/report-format.md — read the skeleton+sections union.
const retroSkill = readSkillUnion('retro');
test('retro SKILL.md contains plan completion section', () => {
expect(retroSkill).toContain('### Plan Completion');
expect(retroSkill).toContain('plan_items_total');
expect(retroSkill).toContain('Plan Completion This Period');
});
});
// --- Plan status footer in preamble ---
describe('Plan status footer in preamble', () => {
test('preamble contains plan status footer as neutral forward reference to EXIT PLAN MODE GATE', () => {
// Read any skill that uses PREAMBLE
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
expect(content).toContain('Plan Status Footer');
expect(content).toContain('GSTACK REVIEW REPORT');
expect(content).toContain('ExitPlanMode');
expect(content).toContain('EXIT PLAN MODE GATE');
// The preamble must NOT impose review-report rules on operational skills
// that have no review report. It's a forward reference, not enforcement.
expect(content).not.toContain('NO REVIEWS YET');
});
});
// --- make-pdf setup ordering ---
describe('make-pdf setup ordering', () => {
test('MAKE-PDF SETUP appears before generic preamble footer sections', () => {
const content = fs.readFileSync(path.join(ROOT, 'make-pdf', 'SKILL.md'), 'utf-8');
const preambleIdx = content.indexOf('## Preamble (run first)');
const setupIdx = content.indexOf('## MAKE-PDF SETUP');
const planModeIdx = content.indexOf('## Plan Mode Safe Operations');
const telemetryIdx = content.indexOf('## Telemetry (run last)');
const workflowIdx = content.indexOf('# make-pdf: publication-quality PDFs from markdown');
expect(preambleIdx).toBeGreaterThanOrEqual(0);
expect(setupIdx).toBeGreaterThan(preambleIdx);
expect(setupIdx).toBeLessThan(planModeIdx);
expect(setupIdx).toBeLessThan(telemetryIdx);
expect(setupIdx).toBeLessThan(workflowIdx);
expect(content.match(/^## MAKE-PDF SETUP/gm)?.length ?? 0).toBe(1);
});
});
// --- Skill invocation during plan mode in preamble ---
describe('Skill invocation during plan mode in preamble', () => {
test('preamble contains skill invocation plan mode section', () => {
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
expect(content).toContain('Skill Invocation During Plan Mode');
expect(content).toContain('Do not continue the workflow');
expect(content).toContain('cancel the skill or leave plan mode');
});
// #2851: a skill cannot grant permissions or redefine the host's mode; the
// host's restrictions and the user's scope win, in every generated skill.
test('no generated skill claims precedence over host plan-mode restrictions (#2851)', () => {
let checked = 0;
for (const file of ['SKILL.md', ...fs.readdirSync(ROOT).map(d => path.join(d, 'SKILL.md'))]) {
if (!fs.existsSync(path.join(ROOT, file))) continue;
const content = fs.readFileSync(path.join(ROOT, file), 'utf-8');
if (!content.includes('## Skill Invocation During Plan Mode')) continue;
checked++;
expect(content, file).not.toContain('takes precedence over generic plan mode');
expect(content, file).not.toContain('overrides generic plan mode');
expect(content, file).not.toContain('allowed because they inform the plan');
expect(content, file).not.toContain('"PLAN MODE EXCEPTION — ALWAYS RUN" execute.');
expect(content, file).not.toContain('Execute "PLAN MODE EXCEPTION — ALWAYS RUN" commands');
expectMentions(content, [['only','restrictions','precedence']], 'content');
expect(content, file).toMatch(/"PLAN MODE EXCEPTION — ALWAYS RUN" (?:commands )?(?:run )?only where the host permits/);
}
expect(checked).toBeGreaterThan(40);
});
});
// --- {{SPEC_REVIEW_LOOP}} resolver tests ---
describe('SPEC_REVIEW_LOOP resolver', () => {
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
const { generateSpecReviewLoop } = require('../scripts/resolvers/spec-review');
const { HOST_PATHS } = require('../scripts/resolvers/types');
const render = (skillName: string, host = 'claude') => generateSpecReviewLoop({
skillName,
tmplPath: `${skillName}/SKILL.md.tmpl`,
host,
paths: HOST_PATHS[host],
});
test('office-hours prepares a complete reviewer prompt on every host', () => {
const { renderOfficeHoursReviewerPrompt } = require('../lib/office-hours-review');
for (const host of Object.keys(HOST_PATHS)) {
const output = render('office-hours', host);
expect(output).toContain('gstack-office-hours-review prepare --design');
expect(output).toContain('--out-dir "<review-directory>"');
ordered(output, ['mktemp -d "<design-path>.review.XXXXXX"', 'prepare --design', '`round-N.prompt.md`', '`dispatch`']);
}
const prompt = renderOfficeHoursReviewerPrompt({ document: '/tmp/design.md', verdictPath: '/tmp/round-1.json' });
expect(prompt).toContain('Document: /tmp/design.md');
expect(prompt).toContain('Verdict: /tmp/round-1.json');
expect(prompt).toContain('"document": "/tmp/design.md"');
expect(prompt).toMatch(/design and coaching document/i);
for (const section of ["'The Assignment'", "'What I noticed about how you think'"]) expect(prompt).toContain(section);
expect(prompt).toMatch(/never invent customer answers/i);
expect(prompt).toMatch(/including minor findings/i);
expect(prompt).toContain('`OFFICE_HOURS_VERDICT round=1 sha256=<hash> path=<verdict path>`');
expect(prompt).toMatch(/run the `Seal:` command/);
expect(prompt).not.toMatch(/identical JSON/i);
expect(prompt).toMatch(/classify every preceding finding/i);
ordered(prompt, ['1. **Completeness**', '2. **Consistency**', '3. **Clarity**', '4. **Scope**', '5. **Feasibility**']);
expect(prompt).toMatch(/distinguished from committed behavior/i);
expect(prompt).toContain('"version": 2');
expect(prompt).toContain('"prior": []');
});
test('office-hours checks stop conditions before edits and preserves concerns mechanically', () => {
const output = render('office-hours');
const step2 = output.slice(output.indexOf('**Step 2:'), output.indexOf('**Step 3:'));
expectMentions(step2, [['before','dispatching','findings']], 'step2');
ordered(step2, ['gstack-office-hours-review check --receipt "<receipt line>"', 'mismatched receipt fails the check', '- PASS:', '- CONVERGENCE:', '- MAX_ITERATIONS: round 3', '- CONTINUE:', 'gstack-office-hours-review finalize --design']);
expectMentions(step2, [['fix only the blocking findings'], ['do not edit for minor findings']], 'step2');
expectMentions(step2, [['do not','re-dispatch','again']], 'step2');
expectMentions(step2, [['do not','generated','section']], 'step2');
expect(step2.slice(step2.indexOf('finalize --design'))).toMatch(/existing user approval/i);
});
test('office-hours finalizes the report after writing it and labels derivable metrics', () => {
const output = render('office-hours');
expect(output).toContain('--report "<report-path>"');
expect(output).toMatch(/after the report exists/i);
expectMentions(output, [['do not','summarize','replace']], 'output');
expectMentions(output, [['approval','assignment','preserve']], 'output');
expect(output).toMatch(/explicit later reviewer evidence/i);
});
test('office-hours errors stay explicit and preserve preceding valid evidence', () => {
const output = render('office-hours');
expect(output).toMatch(/--unreviewed "<actual failure cause>"`\s+and only the\s+preceding valid round files/);
expect(output).toMatch(/do not fabricate JSON/i);
expect(output).toMatch(/not an approval gate/i);
});
test('CEO keeps its loop limits, scoring, failure handling, and reporting', () => {
const ceo = render('plan-ceo-review');
const report = ceo.split('**Step 3:')[1]!.replace(/\s+/g, ' ');
expect(report).toMatch(/use JSON `null`/i);
expect(ceo).toMatch(/implementable without follow-up questions/i);
expect(ceo).not.toContain('design and coaching document');
expect(ceo).not.toContain('gstack-office-hours-review');
const dispatch = ceo.slice(ceo.indexOf('**Step 1:'), ceo.indexOf('**Step 2:')).replace(/\s+/g, ' ');
expect(dispatch).toContain('`run_in_background: false`');
expectMentions(dispatch, [['wait','tool']], 'dispatch');
expectMentions(dispatch, [['do not','waiting','advance']], 'dispatch');
const fixes = ceo.slice(ceo.indexOf('**Step 2:'), ceo.indexOf('**Step 3:')).replace(/\s+/g, ' ');
ordered(fixes, ['use 0D', 'amend the working plan', 're-dispatch']);
expectMentions(fixes, [['stop','review','after']], 'fixes');
expect(fixes).toMatch(/stop the loop/i);
expectMentions(fixes, [['not','successful','reviewer']], 'fixes');
expectMentions(fixes, [['does not','missing','require']], 'fixes');
expect(report).toContain('"## Reviewer Concerns"');
expect(report).toMatch(/SCORE is the latest attempt's reported 1–10 grade/i);
expect(report).toContain('"prior review score"');
expect(report).toMatch(/as not persisted/i);
expect(report).toContain('**0H spec-review metrics**');
expectMentions(report, [['stop','completion','required']], 'report');
expect(report).toContain('GSTACK_STATE_ROOT=$(~/.claude/skills/gstack/bin/gstack-paths --get GSTACK_STATE_ROOT); : "${GSTACK_STATE_ROOT:?gstack-paths failed; reinstall with ./setup or /gstack-upgrade}"');
expect(report).toContain('mkdir -p "$GSTACK_STATE_ROOT/analytics" || exit 1');
expect(report).toContain('>> "$GSTACK_STATE_ROOT/analytics/spec-review.jsonl" || exit 1');
expect(report).not.toContain('Your doc survived');
});
test('CEO spec report separates findings, confirmed fixes, and unresolved concerns', () => {
const report = render('plan-ceo-review').split('**Step 3: Report and persist metrics**')[1]!.replace(/\s+/g, ' ');
expect(report).toMatch(/ITERATIONS counts actual reviewer launches/i);
ordered(report, ['FOUND, FIXED and REMAINING', 'reviewer-confirmed fixes']);
expect(report).toMatch(/actual counts, never estimates/i);
expect(report).not.toContain('M issues caught and fixed');
});
test('CEO spec review receives the full source plan as well as the scope artifact on every host', () => {
for (const host of Object.keys(HOST_PATHS)) {
const output = render('plan-ceo-review', host).replace(/\s+/g, ' ');
expect(output).toMatch(/all five dimensions/i);
expect(output).toMatch(/unsupported accepted expansions/i);
}
const source = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf8');
expect(source).toContain('## Plan under review\n{working plan path, or');
const template = source.replace(/\s+/g, ' ');
expectMentions(template, [['cannot','summary','serve']], 'template');
});
test('CEO shares both inputs after spec review and owns unresolved concerns in its scope document', () => {
const template = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf8');
const persist = (template.split('### 0H.')[1]?.split('### 0I.')[0] ?? '').replace(/\s+/g, ' ');
const review = persist.indexOf('{{SPEC_REVIEW_LOOP}}');
const sharing = persist.search(/present both inputs for final scope-document approval/i);
expect(review).toBeGreaterThanOrEqual(0);
expect(sharing).toBeGreaterThan(review);
const output = render('plan-ceo-review');
const processing = output.split('**Step 2:')[1]!.split('**Step 3:')[0]!.replace(/\s+/g, ' ');
const report = output.split('**Step 3:')[1] ?? '';
expect(report).toContain('CEO summary');
expect(report).toContain('Reviewer Concerns');
expect(report).toMatch(/owning\s+input/);
});
test('CEO reviewer receives one complete prompt without duplicate scoring instructions', () => {
const output = render('plan-ceo-review');
const dispatch = (output.split('**Step 1: Dispatch reviewer subagent**')[1]?.split('**Step 2:')[0] ?? '').replace(/\s+/g, ' ');
expect(dispatch.match(/Read both inputs in full/gi)).toHaveLength(1);
expect(dispatch.match(/quality score/gi)).toHaveLength(1);
expect(dispatch).toMatch(/quality score \(1-10\)/i);
expect(dispatch).toMatch(/overall PASS only if all dimensions pass/i);
expect(dispatch).toMatch(/PASS or numbered issues/i);
expect(dispatch).toMatch(/cite input and requirement/i);
});
test('contains all 5 review dimensions', () => {
for (const dim of ['Completeness', 'Consistency', 'Clarity', 'Scope', 'Feasibility']) {
expect(content).toContain(dim);
}
});
test('references Agent tool for subagent dispatch', () => {
expect(content).toMatch(/Agent.*tool/i);
});
test('specifies max 3 iterations', () => {
expect(content).toMatch(/3.*iteration|maximum.*3/i);
});
test('includes quality score', () => {
expect(content).toContain('quality score');
});
test('includes metrics path', () => {
expect(content).toContain('spec-review.jsonl');
});
test('includes convergence guard', () => {
expect(content).toMatch(/[Cc]onvergence/);
});
test('includes graceful failure handling', () => {
expect(content).toMatch(/skip.*review|unavailable/i);
});
});
// --- {{DESIGN_SKETCH}} resolver tests ---
describe('DESIGN_SKETCH resolver', () => {
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
test('references DESIGN.md for design system constraints', () => {
expect(content).toContain('DESIGN.md');
});
test('contains wireframe or sketch terminology', () => {
expect(content).toMatch(/wireframe|sketch/i);
});
test('wireframes render through gstack-render (Aside first)', () => {
expect(content).toContain('gstack-render.ts');
});
test('references screenshot capture', () => {
expect(content).toContain('--screenshot');
});
test('specifies rough aesthetic', () => {
expect(content).toMatch(/[Rr]ough|hand-drawn/);
});
test('includes skip conditions', () => {
expect(content).toMatch(/no UI component|skip/i);
});
});
// --- {{CODEX_SECOND_OPINION}} resolver tests ---
describe('CODEX_SECOND_OPINION resolver', () => {
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
const codexContent = readExternalSkillUnion(EXTERNAL_OUT, '.agents', 'office-hours');
test('Phase 3.5 section appears in office-hours SKILL.md', () => {
expect(content).toContain('Phase 3.5: Cross-Model Second Opinion');
});
test('contains codex exec invocation', () => {
expect(content).toContain('codex exec');
});
test('contains cross-model synthesis instructions', () => {
expect(content).toMatch(/[Ss]ynthesis/);
});
test('contains Claude subagent fallback', () => {
expect(content).toContain('CODEX_MODE: not_installed');
expect(content).toContain('If preflight is not ready');
expect(content).toContain('Agent tool');
expect(content).toContain('SECOND OPINION (Claude subagent)');
});
test('contains premise revision check', () => {
expect(content).toContain('Codex challenged premise');
});
test('contains error handling for auth, timeout, and empty', () => {
expect(content).toMatch(/[Aa]uth.*fail/);
expect(content).toMatch(/[Tt]imeout/);
expect(content).toMatch(/[Ee]mpty response/);
});
test('Codex host runs Phase 3.5 through Claude Code', () => {
expect(codexContent).toContain('Phase 3.5: Cross-Model Second Opinion');
expect(codexContent).toContain('gstack-claude-code');
expect(codexContent).toContain('Claude Code');
expect(codexContent).not.toMatch(/codex\s+(?:exec|review)\s/);
});
});
// --- Codex filesystem boundary tests ---
describe('Codex filesystem boundary', () => {
// Skills that call codex exec/review and should contain boundary text
const CODEX_CALLING_SKILLS = [
'codex', // /codex skill — 3 modes
'autoplan', // /autoplan — CEO/design/eng voices
'review', // /review — adversarial step resolver
'ship', // /ship — adversarial step resolver
'plan-eng-review', // outside voice resolver
'plan-ceo-review', // outside voice resolver
'office-hours', // second opinion resolver
];
const BOUNDARY_MARKER = /do not read or execute any/i;
test('boundary instruction appears in all skills that call codex', () => {
for (const skill of CODEX_CALLING_SKILLS) {
// Union: ship's codex call lives in sections/adversarial.md after the carve.
const content = readSkillUnion(skill);
expect(content).toMatch(BOUNDARY_MARKER);
}
});
test('codex skill has Filesystem Boundary section', () => {
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
expect(content).toContain('## Filesystem Boundary');
});
test('codex skill has rabbit-hole detection rule', () => {
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
expect(content).toContain('Detect skill-file rabbit holes');
expect(content).toContain('gstack-update-check');
expect(content).toContain('Consider retrying');
});
test('outside-voice-steps.ts CODEX_BOUNDARY constant is interpolated into resolver output', () => {
// The adversarial step resolver should include boundary text in codex exec
// prompts. Carved: the adversarial step lives in sections/adversarial.md.
const reviewContent = readSkillUnion('review');
// Boundary should appear near codex exec invocations
const boundaryIdx = reviewContent.search(BOUNDARY_MARKER);
const codexExecIdx = reviewContent.indexOf('codex exec');
// Both must exist and boundary must come before a codex exec call
expect(boundaryIdx).toBeGreaterThan(-1);
expect(codexExecIdx).toBeGreaterThan(-1);
});
test('autoplan boundary text avoids host-specific paths for cross-host compatibility', () => {
const content = fs.readFileSync(path.join(ROOT, 'autoplan', 'SKILL.md.tmpl'), 'utf-8');
// autoplan template uses generic 'skills/gstack' pattern instead of host-specific
// paths like ~/.claude/ or .agents/skills (which break Codex/Claude output tests)
const boundaryStart = content.indexOf('Filesystem Boundary');
const boundaryEnd = content.indexOf('---', boundaryStart + 1);
const boundarySection = content.slice(boundaryStart, boundaryEnd);
expect(boundarySection).not.toContain('~/.claude/');
expect(boundarySection).not.toContain('.agents/skills');
expect(boundarySection).toContain('skills/gstack');
expect(boundarySection).toMatch(BOUNDARY_MARKER);
});
});
// --- Codex outside-voice availability probe ---
describe('Codex outside voice is offered behind an executed availability probe', () => {
test.each(['office-hours', 'plan-ceo-review', 'plan-design-review', 'plan-eng-review'])('%s probes codex and has a not-installed fallback', skill => {
const content = readSkillUnion(skill);
expect(content).toContain('command -v codex');
expect(content).toContain('not_installed');
});
});
// --- {{BENEFITS_FROM}} resolver tests ---
describe('BENEFITS_FROM resolver', () => {
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
const engContent = readSkillUnion('plan-eng-review'); // carved
test('plan-ceo-review contains prerequisite skill offer', () => {
expect(ceoContent).toContain('Prerequisite Skill Offer');
expect(ceoContent).toContain('/office-hours');
});
test('plan-ceo-review offers /office-hours when no design doc exists and detects a lost user mid-session', () => {
const offer = extractMarkdownSection(ceoContent, '## Prerequisite Skill Offer');
expect(offer.replace(/\s+/g, ' ')).toMatch(/when the design doc check above prints "No design doc found,?"/i);
expect(offer).toContain('A) Run /office-hours now');
expect(offer).toContain('B) Skip — proceed with standard review');
expectMentions(offer, [['do not','re-offer','session']], 'offer');
expect(ceoContent).toContain('**Mid-session detection (0A):**');
const midSession = ceoContent.indexOf('**Mid-session detection (0A):**');
expect(ceoContent.slice(midSession, midSession + 300)).toMatch(/offer `\/office-hours`/);
});
test('plan-eng-review contains prerequisite skill offer', () => {
expect(engContent).toContain('Prerequisite Skill Offer');
expect(engContent).toContain('/office-hours');
});
test('the optional Eng prerequisite ends before mandatory engineering review', () => {
const offer = extractMarkdownSection(engContent, '## Prerequisite Skill Offer');
expect(offer).toContain('B) Skip — proceed with standard review');
expectMentions(offer, [['do not','re-offer','session']], 'offer');
expect(offer).not.toContain('### Step 0: Scope Challenge');
const review = extractMarkdownSection(engContent, '## Engineering review');
expect(review).toContain('### Step 0: Scope Challenge');
expectMentions(review, [['before','challenge','mandatory']], 'review');
expect(review).toContain('sections/review-sections.md');
});
test('Eng initial and prerequisite recheck both discover the canonical override and fall back on slug failure', () => {
const initial = extractMarkdownSection(engContent, '### Design Doc Check').match(/```bash\n([\s\S]*?)\n```/)![1]!;
const offer = extractMarkdownSection(engContent, '## Prerequisite Skill Offer');
expectTokens(offer, ['**Design Doc Check**'], 'offer');
expectMentions(offer, [['block', 'office-hours', 'completes']], 'offer');
expect(offer).toContain('This is a fresh execution');
expectMentions(offer, [['do not','prerequisite','preamble']], 'offer');
expect(offer).not.toContain('_REVIEW_SLUG=');
const recheck = initial; // Execute the one canonical block again after the prerequisite.
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'eng-design-slug-'));
try {
const home = path.join(dir, 'home'), cwd = path.join(dir, 'project');
const helper = path.join(home, '.claude/skills/gstack/bin/gstack-slug');
fs.mkdirSync(path.dirname(helper), {recursive: true});
fs.mkdirSync(cwd);
fs.copyFileSync(path.join(ROOT, 'bin/gstack-slug'), helper);
fs.chmodSync(helper, 0o755);
fs.copyFileSync(path.join(ROOT, 'bin/gstack-state-root.sh'), path.join(path.dirname(helper), 'gstack-state-root.sh'));
fs.copyFileSync(path.join(ROOT, 'bin/gstack-remote-identity.sh'), path.join(path.dirname(helper), 'gstack-remote-identity.sh'));
for (const bin of ['gstack-paths', 'gstack-design-doc-find']) {
fs.copyFileSync(path.join(ROOT, 'bin', bin), path.join(path.dirname(helper), bin));
fs.chmodSync(path.join(path.dirname(helper), bin), 0o755);
}
const expected = path.join(home, '.gstack/projects/canonical-override/session-unknown-design-current.md');
const wrong = path.join(home, '.gstack/projects/project/session-unknown-design-wrong.md');
for (const file of [expected, wrong]) {
fs.mkdirSync(path.dirname(file), {recursive: true});
fs.writeFileSync(file, '# Design fixture\n');
}
const env = {...process.env, HOME: home, GSTACK_HOME: path.join(home, '.gstack'),
GSTACK_PROJECT_SLUG: 'canonical-override', GIT_CEILING_DIRECTORIES: dir};
for (const command of [initial, recheck]) {
// No eval (#2763): worktree-isolated Claude Code sessions refuse it.
expect(command).toContain('if SLUG=$(~/.claude/skills/gstack/bin/gstack-slug --get SLUG); then');
expect(command).toContain('BRANCH=$(~/.claude/skills/gstack/bin/gstack-slug --get BRANCH)');
expect(command).not.toContain('eval ');
expect(command).toContain('echo "No design doc found"');
expect(command).not.toContain('remote-slug');
const result = runCapturedCommand('bash', ['-c', command], {cwd, env, timeout: 5000, captureStdout: true});
expect(result.status, result.stderr).toBe(0);
expect(result.stdout).toContain(`Design doc found: ${expected}`);
expect(result.stdout).not.toContain(wrong);
}
fs.writeFileSync(helper, '#!/usr/bin/env bash\necho "slug fixture failed" >&2\nexit 23\n', {mode: 0o755});
for (const command of [initial, recheck]) {
const result = runCapturedCommand('bash', ['-c', command], {cwd, env, timeout: 5000, captureStdout: true});
expect(result.status).toBe(0);
expect(result.stderr).toContain('slug fixture failed');
expect(result.stdout).not.toContain('Design doc found:');
expect(result.stdout).toContain('No design doc found');
}
} finally { fs.rmSync(dir, {recursive: true, force: true}); }
});
test('offer includes graceful decline', () => {
expect(ceoContent).toContain('No worries');
});
test('skills without benefits-from do NOT have prerequisite offer', () => {
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
expect(qaContent).not.toContain('Prerequisite Skill Offer');
});
test('inline invocation — no "another window" language', () => {
expect(ceoContent).not.toContain('another window');
expect(engContent).not.toContain('another window');
});
test('inline invocation — read-and-follow path present', () => {
expect(ceoContent).toContain('office-hours/SKILL.md');
expect(engContent).toContain('office-hours/SKILL.md');
});
test('BENEFITS_FROM delegates to INVOKE_SKILL pattern', () => {
// Should contain the INVOKE_SKILL-style loading prose (not the old manual skip list)
expect(engContent).toContain('skipping these sections');
});
});
// --- {{INVOKE_SKILL}} resolver tests ---
describe('INVOKE_SKILL resolver', () => {
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
test('plan-ceo-review uses INVOKE_SKILL for mid-session office-hours fallback', () => {
// The mid-session detection path should use INVOKE_SKILL-generated prose
expect(ceoContent).toContain('office-hours/SKILL.md');
});
test('INVOKE_SKILL output includes default skip list', () => {
expect(ceoContent).toContain('Preamble (run first)');
expect(ceoContent).toContain('Telemetry (run last)');
expect(ceoContent).toContain('AskUserQuestion Format');
});
test('INVOKE_SKILL output includes error handling', () => {
expect(ceoContent).toContain('If unreadable');
expect(ceoContent).toContain('Could not load');
});
test('template uses {{INVOKE_SKILL:office-hours}} placeholder', () => {
const tmpl = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf-8');
expect(tmpl).toContain('{{INVOKE_SKILL:office-hours}}');
});
});
// --- {{CHANGELOG_WORKFLOW}} resolver tests ---
describe('CHANGELOG_WORKFLOW resolver', () => {
const shipContent = readShipUnion();
test('ship SKILL.md contains changelog workflow', () => {
expect(shipContent).toContain('CHANGELOG (auto-generate)');
expect(shipContent).toContain('git log origin/<base>..HEAD --oneline');
});
test('changelog workflow includes cross-check step', () => {
expect(shipContent).toContain('Cross-check');
const crossCheck = shipContent.slice(shipContent.indexOf('**Cross-check:**'), shipContent.indexOf('**Do NOT ask the user to describe changes.**'));
expect(crossCheck).toContain('commit list from step 2');
expect(crossCheck).toMatch(/user-facing/i);
});
test('changelog workflow includes voice guidance', () => {
expect(shipContent).toContain('Lead with what the user can now **do**');
});
test('template uses {{CHANGELOG_WORKFLOW}} placeholder', () => {
// Post-carve (T9): the skeleton points to the changelog section, which carries
// the resolver. Neither should inline the old changelog content.
const skel = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');
const changelogSection = fs.readFileSync(path.join(ROOT, 'ship', 'sections', 'changelog.md.tmpl'), 'utf-8');
expect(skel).toContain('{{SECTION:changelog}}');
expect(changelogSection).toContain('{{CHANGELOG_WORKFLOW}}');
expect(skel + changelogSection).not.toContain('Group commits by theme');
});
test('changelog workflow includes keep-changelog format', () => {
expect(shipContent).toContain('### Added');
expect(shipContent).toContain('### Fixed');
});
});
// --- Duplicate {{PREAMBLE}} guard (#2508/#2362) ---
describe('assertSinglePreamble', () => {
test('one {{PREAMBLE}} passes', () => {
expect(() => assertSinglePreamble('a\n{{PREAMBLE}}\nb', 'x/SKILL.md.tmpl')).not.toThrow();
});
test('zero {{PREAMBLE}} passes (sections have none)', () => {
expect(() => assertSinglePreamble('no macro here', 'x/sections/y.md.tmpl')).not.toThrow();
});
test('a second occurrence throws with the template path — even in prose', () => {
// The original #2508 bug WAS a prose mention: "emitted by {{PREAMBLE}}'s
// preamble bash". Resolution is context-blind, so the guard must be too.
const tmpl = '{{PREAMBLE}}\n\n...later: emitted by {{PREAMBLE}}\'s preamble bash';
expect(() => assertSinglePreamble(tmpl, 'spec/SKILL.md.tmpl')).toThrow(/spec\/SKILL\.md\.tmpl.*2 times/);
});
});
// --- Parameterized resolver infrastructure tests ---
describe('parameterized resolver support', () => {
test('gen-skill-docs regex handles colon-separated args', () => {
// Verify the template containing {{INVOKE_SKILL:office-hours}} was processed
// without leaving unresolved placeholders
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
expect(ceoContent).not.toMatch(/\{\{INVOKE_SKILL:[^}]+\}\}/);
});
test('templates with parameterized resolvers pass unresolved check', () => {
// All generated SKILL.md files should have no unresolved {{...}} placeholders
const skillDirs = fs.readdirSync(ROOT).filter(d =>
fs.existsSync(path.join(ROOT, d, 'SKILL.md'))
);
for (const dir of skillDirs) {
const content = fs.readFileSync(path.join(ROOT, dir, 'SKILL.md'), 'utf-8');
const unresolved = content.match(/\{\{[A-Z_]+(?::[^}]*)?\}\}/g);
if (unresolved) {
throw new Error(`${dir}/SKILL.md has unresolved placeholders: ${unresolved.join(', ')}`);
}
}
});
});
// --- Preamble routing injection tests ---
describe('preamble routing injection (bin/gstack-skill-start emission layer)', () => {
// Token-reduction Phase 2: the routing-injection prose left the rendered
// preamble entirely — bin/gstack-skill-start probes, gates, and emits the
// whole flow as a GSTACK_INSTRUCTION block (with the AUQ, the routing rules
// to append, and the decline ack all INSIDE the block). Absence from the
// renders is pinned by test/onboarding-moved-literals.test.ts (tombstone);
// this suite pins the gate structure and the emitted block's content.
const routingBlock = (() => {
const start = SKILL_START_SCRIPT.indexOf('_emit_block routing-injection');
expect(start).toBeGreaterThan(0);
return SKILL_START_SCRIPT.slice(start, SKILL_START_SCRIPT.indexOf('\nEOI', start));
})();
test('routing probe checks CLAUDE.md and AGENTS.md (now in gstack-skill-start)', () => {
// #2500: the probe iterates CLAUDE.md AND AGENTS.md — non-Claude hosts
// route skills via AGENTS.md, the cross-harness convention file.
expect(SKILL_START_SCRIPT).toContain('for _RF in CLAUDE.md AGENTS.md');
expect(SKILL_START_SCRIPT).toContain('grep -q "## Skill routing" "$_RF"');
expect(SKILL_START_SCRIPT).toContain('echo "HAS_ROUTING: $_HAS_ROUTING"');
});
test('script reads and echoes routing_declined config', () => {
expect(SKILL_START_SCRIPT).toMatch(/_ROUTING_DECLINED=\$\("\$_BIN\/gstack-config" get routing_declined/);
expect(SKILL_START_SCRIPT).toContain('echo "ROUTING_DECLINED: $_ROUTING_DECLINED"');
});
test('emitted block carries the routing injection AskUserQuestion', () => {
expect(routingBlock).toContain('Add routing rules to CLAUDE.md');
expect(routingBlock).toContain("I'll invoke skills manually");
});
test('routing injection respects prior decline (gate + in-block ack)', () => {
expect(SKILL_START_SCRIPT).toContain('[ "$_ROUTING_DECLINED" = "false" ]');
expect(routingBlock).toMatch(/routing_declined.*true/);
expect(routingBlock).toContain('re-enable with `__BIN__/gstack-config set routing_declined false`');
});
test('routing injection only fires when all conditions met', () => {
// Must be: HAS_ROUTING=no AND ROUTING_DECLINED=false AND PROACTIVE_PROMPTED=yes
expect(SKILL_START_SCRIPT).toContain(
'if [ "$_HAS_ROUTING" = "no" ] && [ "$_ROUTING_DECLINED" = "false" ] && [ "$_PROACTIVE_PROMPTED" = "yes" ]; then',
);
});
test('routing section content includes key routing rules', () => {
expect(routingBlock).toContain('invoke /office-hours');
expect(routingBlock).toContain('invoke /investigate');
expect(routingBlock).toContain('invoke /ship');
expect(routingBlock).toContain('invoke /qa');
});
test('routing section uses renamed checkpoint skills (not stale /checkpoint)', () => {
expect(routingBlock).toContain('invoke /context-save');
expect(routingBlock).toContain('invoke /context-restore');
expect(routingBlock).not.toContain('invoke checkpoint');
});
test('routing section routes only to available skills, without "when in doubt" over-triggering (#3018)', () => {
expect(routingBlock).toContain("Route only to skills in the session's available-skills list");
expect(routingBlock).not.toContain('When in doubt');
expect(routingBlock).not.toContain('Do NOT answer directly');
});
test('root router skips routes to skills that are not loaded and says when not to invoke (#3018)', () => {
const router = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(router).toContain('Route only to skills in your available-skills list');
expect(router).toContain('skip it, never try to invoke it');
expect(router).toContain('Answer directly when\nno skill matches');
expect(router).not.toContain('When in doubt, invoke the skill');
});
});
// --- {{DESIGN_OUTSIDE_VOICES}} resolver tests ---
describe('DESIGN_OUTSIDE_VOICES resolver', () => {
test('plan-design-review contains outside voices section', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('Design Outside Voices');
expect(content).toContain('CODEX_MODE: ready');
expect(content).toContain('LITMUS SCORECARD');
});
test('design-review contains outside voices section', () => {
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Design Outside Voices');
expect(content).toContain('source audit');
});
test('design-consultation contains outside voices section', () => {
const content = readSkillUnion('design-consultation');
expect(content).toContain('Design Outside Voices');
expect(content).toContain('design direction');
});
test('consultation dispatch shares the complete brief without executing product text', () => {
const content = readSkillUnion('design-consultation');
const command = [...content.matchAll(/```bash\n([\s\S]*?)```/g)]
.map(match => match[1]).find(block => block.includes('codex exec'))!;
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'design-brief-contract-'));
const quote = (value: string) => "'" + value.replaceAll("'", "'\\''") + "'";
try {
const brief = path.join(dir, 'product.md');
const capture = path.join(dir, 'argv.json');
const marker = path.join(dir, 'must-not-execute');
const product = `A product with $(touch ${marker}), \`touch ${marker}\`, and "quotes".\nUsers: builders.\n`;
fs.writeFileSync(brief, product);
// Fake codex: the free sandbox preflight succeeds; exec writes its final message to -o.
fs.writeFileSync(path.join(dir, 'codex'), `#!${process.execPath}\nconst fs = require('fs'); const a = process.argv.slice(2); if (a[0] === 'sandbox') process.exit(0);\nfs.writeFileSync(process.env.CAPTURE, JSON.stringify({ args: a, stdin: a[1] === '-' ? fs.readFileSync(0, 'utf8') : '' }));\nconst msg = 'Recommendation: choose a clear hierarchy because builders need to find their work.';\nif (a.includes('-o')) fs.writeFileSync(a[a.indexOf('-o') + 1], msg);\nconsole.log(msg);\n`, { mode: 0o700 });
const prepare = (file: string) => command.replace("'<prepared-prompt-file>'", quote(file))
.replaceAll('$HOME/.claude/skills/gstack', ROOT);
const env = { ...process.env, PATH: dir + path.delimiter + process.env.PATH, CAPTURE: capture,
CODEX_THREAD_ID: '', CODEX_SANDBOX: '', CLAUDECODE: '1', GSTACK_ACTIVE_HOST: 'claude',
GSTACK_HOME: path.join(dir, 'state'), GSTACK_STATE_ROOT: '' };
const result = spawnSync('bash', ['-c', prepare(brief)], { cwd: ROOT, env, encoding: 'utf8', timeout: 5_000 });
expect(result.status, result.stderr).toBe(0);
const { args, stdin } = JSON.parse(fs.readFileSync(capture, 'utf8'));
expect(args[0]).toBe('exec');
expect(args[1]).toBe('-');
expect(stdin).toBe(product);
expect(args).toContain('read-only');
expect(fs.readFileSync(brief, 'utf8')).toBe(product);
expect(fs.existsSync(marker)).toBe(false);
fs.unlinkSync(capture);
const missing = spawnSync('bash', ['-c', prepare(path.join(dir, 'absent'))], { cwd: ROOT, env, encoding: 'utf8', timeout: 5_000 });
expect(missing.status).not.toBe(0);
expect(missing.stderr).toContain('No such file');
expect(fs.existsSync(capture)).toBe(false);
expect(content).toContain('await both before synthesis');
expect(content).toContain('use only the native voice');
expect(content).toContain('give the native Agent its absolute path');
expect(content).toContain('Read the complete product brief at [the absolute DESIGN_BRIEF path printed above]');
expect(content).toContain('Omit faces you cannot verify');
expect(content).toContain('a face may serve multiple roles');
expect(content).not.toContain('a single question that covers everything');
} finally { fs.rmSync(dir, { recursive: true, force: true }); }
});
test('branches correctly per skillName — different prompts', () => {
const planContent = readSkillUnion('plan-design-review');
const consultContent = readSkillUnion('design-consultation');
// plan-design-review uses analytical prompt (high reasoning)
expect(planContent).toContain('model_reasoning_effort="high"');
// design-consultation uses creative prompt (medium reasoning)
expect(consultContent).toContain('model_reasoning_effort="medium"');
});
});
// --- {{DESIGN_HARD_RULES}} resolver tests ---
describe('DESIGN_HARD_RULES resolver', () => {
test('plan-design-review Pass 4 contains hard rules', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('Design Hard Rules');
expect(content).toContain('Classifier');
expect(content).toContain('MARKETING/LANDING PAGE');
expect(content).toContain('APP UI');
});
test('design-review contains hard rules', () => {
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Design Hard Rules');
});
test('includes all 3 rule sets', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('Landing page rules');
expect(content).toContain('App UI rules');
expect(content).toContain('Universal rules');
});
test('classifier names the four visitor modes and keeps the legacy aliases', () => {
const content = readSkillUnion('plan-design-review');
for (const mode of ['PERSUADE', 'OPERATE', 'READ', 'EXPERIENCE', 'HYBRID']) expect(content).toContain(`**${mode}**`);
expect(content).toContain('Read rules');
expect(content).toContain('Experience rules');
expectMentions(content, [['not', 'classify', 'section']], 'content');
});
test('carries the craft-floor reflexes and the three-looks calibration', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('Reflexes no detector catches');
expect(content).toContain('Browser surfaces carry the design');
expect(content).toContain('One authored motion moment');
expect(content).toContain('Depth has an offset');
expect(content).toContain('Light or dark comes from the use scene');
expect(content).toContain('Calibration: the three looks');
});
test('slop section lists detector rule ids and judgment tells outside design-review', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('nested-cards: Nested cards');
expectMentions(content, [['no', 'judgment', 'detector']], 'content');
// Never a bracketed gstack-only id.
expect(content).not.toContain('[hero-metrics]');
});
test('design-consultation carries the font procedure, role-scoped lists, color strategies, and catalog bullets', () => {
const content = readSkillUnion('design-consultation');
expect(content).toContain('Choosing faces: a procedure, not a menu');
expect(content).toContain('**Overused as display**');
expect(content).toContain('**Banned in any role:** Papyrus');
expect(content).toContain('Restrained (1 accent + neutrals');
expectMentions(content, [['not', 'light', 'dials']], 'content');
expect(content).toContain('Calibration: the three looks');
// Bullets are prose only: never a bracketed rule id in the proposal skill.
expect(content).toContain('- A card inside a card is always wrong.');
expect(content).not.toMatch(/^- \[[a-z-]+\] /m);
// The old menu is gone.
expect(content).not.toContain('Font recommendations by purpose');
});
test('design-html blacklist lines carry catalog ids', () => {
const content = fs.readFileSync(path.join(ROOT, 'design-html', 'SKILL.md'), 'utf-8');
expect(content).toMatch(/AI slop blacklist/i);
expect(content).toContain('Purple/blue gradients as default <!-- ai-color-palette -->');
expect(content).toContain('lib/design-catalog.ts');
});
test('design-review renders the catalog once: Methodology category 9 carries it, Hard Rules points at it', () => {
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
expect(content.split('### Design Hard Rules').length - 1).toBe(1);
// Category 9 lists the rule once (ids only); Typography points at the same id from its overused-face item.
expect(content.split('[overused-font]').length - 1).toBe(2);
expect(content).toContain('are Methodology category 9');
expect(content).toContain('**9. AI Slop Detection**');
expect(content).toContain('Detector rules (ids only;');
expect(content).toContain('[nested-cards] nested cards');
expect(content).toContain('Judgment tells (no detector rule');
// The legacy blacklist is not repeated as a numbered list in design-review.
expect(content).not.toMatch(/^1\. Purple\/violet\/indigo/m);
});
test('references shared AI slop blacklist items', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('3-column feature grid');
expect(content).toContain('Purple/violet/indigo');
});
test('includes OpenAI hard rejection criteria', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('Generic SaaS card grid');
expect(content).toContain('Carousel with no narrative purpose');
});
test('includes OpenAI litmus checks', () => {
const content = readSkillUnion('plan-design-review');
expect(content).toContain('Brand/product unmistakable');
});
});
describe('Design approval reconciliation', () => {
test('token alignment reuses the approved outcome while preserving first decisions and new tradeoffs', () => {
const section = fs.readFileSync(path.join(ROOT, 'plan-design-review/sections/review-sections.md'), 'utf8');
const pass = extractMarkdownSection(section, '### Pass 5: Design System Alignment');
const ask = pass.search(/stop\W+AskUserQuestion once per issue/i);
expect(ask).toBeGreaterThan(0);
const beforeQuestion = pass.slice(0, ask);
expect(beforeQuestion).toMatch(/already approved that outcome/i);
expect(beforeQuestion).toMatch(/not a new issue/i);
expectMentions(beforeQuestion, [['only','evidence','again']], 'beforeQuestion');
});
test('loaded review section distinguishes individual issue decisions from navigation', () => {
const section = fs.readFileSync(path.join(ROOT, 'plan-design-review/sections/review-sections.md'), 'utf8').replace(/\s+/g, ' ');
expectMentions(section, [['no','next-step','remedies']], 'section');
expectMentions(section, [['never','askuserquestion','issue-approval']], 'section');
expectMentions(section, [['no','unresolved','findings']], 'section');
});
test('Design blocking Exit checklist reconciles approvals and refreshes stale output after a decision', () => {
const main = fs.readFileSync(path.join(ROOT, 'plan-design-review/SKILL.md'), 'utf8');
const check = extractMarkdownSection(main, '## Section self-check');
expect(check).toMatch(/run approval check 0/i);
const gate = extractMarkdownSection(main, '## EXIT PLAN MODE GATE (BLOCKING)').replace(/\s+/g, ' ');
ordered(gate, ['0. Approvals:', '1. Read the plan file']);
const approvals = gate.slice(gate.indexOf('0. Approvals:'), gate.indexOf('1. Read the plan file'));
expect(approvals).toMatch(/never group distinct issues/i);
expect(approvals).toMatch(/navigation are not approval/i);
expect(approvals).toContain('preamble-authorized');
expect(approvals).toMatch(/deferrals remain unresolved/i);
expectMentions(approvals, [['wait','pending','drafts']], 'approvals');
ordered(approvals, ['refresh the plan and report', 'Read-back gate', 'rerun this gate']);
});
test('CEO readiness binds actual approvals before its read-only exit verification', () => {
const main = fs.readFileSync(path.join(ROOT, 'plan-ceo-review/SKILL.md'), 'utf8');
const check = extractMarkdownSection(main, '## Section self-check').replace(/\s+/g, ' ');
const gate = extractMarkdownSection(main, '## EXIT PLAN MODE GATE (BLOCKING)').replace(/\s+/g, ' ');
const section = fs.readFileSync(path.join(ROOT, 'plan-ceo-review/sections/review-sections.md'), 'utf8');
const readiness = extractMarkdownSection(section, '## Approval readiness').replace(/\s+/g, ' ');
expect(section.indexOf('## Approval readiness')).toBeLessThan(section.indexOf('## Required Outputs'));
expectMentions(readiness, [['approval','preamble-authorized','exact']], 'readiness');
expectMentions(readiness, [['not','navigation','approvals']], 'readiness');
expectMentions(readiness, [['only','commitments','approves']], 'readiness');
expectMentions(readiness, [['only','applies','scope']], 'readiness');
expect(readiness).toMatch(/delivery-scope deferral is settled/i);
expect(readiness).toMatch(/leaves that choice unresolved/i);
expect(readiness).toContain('`Approval readiness: PASS`');
expect(readiness).toMatch(/navigation alone does not/i);
expect(main.replace(/\s+/g, ' ')).toMatch(/default to implementation-ready/i);
ordered(gate, ['Read-only verification', 'Verify `Approval readiness: PASS`', '1. Read the plan file']);
expectMentions(gate, [['only','readiness','return']], 'gate');
expect(gate).toContain('**Gate outcome: Blocked**');
expect(main).toContain('**Pass with log-only gaps:**');
expectMentions(gate, [['without','exitplanmode','telemetry']], 'gate');
expect(check).toContain('`sections/review-sections.md`');
expectMentions(check, [['stop','review','read']], 'check');
const decisions = main.slice(main.indexOf('### 0D.'), main.indexOf('### 0E.')).replace(/\s+/g, ' ');
ordered(decisions, ["**Choose the question's route first:**", '**Admin question:**', '**Plan decision:**']);
expect(decisions).toMatch(/reuse exact prior approvals/i);
expect(decisions).toContain('`D<N> — <ROW-ID>: <one-line question>`');
const row = decisions.slice(decisions.indexOf('**Pre-question checkpoint:**'), decisions.indexOf('Effort/risk must'));
for (const field of ['ID', 'owner', 'Current/Proposed', 'Status', 'Exact approval and scope']) expect(row).toContain(field);
expectMentions(decisions.slice(decisions.indexOf('**4. Ask')), [['stop','actual','answer']], 'section');
expect(decisions).toMatch(/amend only (?:what it authorizes|authorized work)/i);
expectMentions(decisions, [['only','contradictions','instructions']], 'decisions');
expect(decisions).toContain('"No issues, moving on."');
const outside = extractMarkdownSection(section, '**Cross-model tension:**').replace(/\s+/g, ' ');
expectMentions(outside, [['only','completed','external']], 'outside');
expect(outside).toContain('CROSS-MODEL');
const findings = extractMarkdownSection(section, '**Integrate reviewer findings:**').replace(/\s+/g, ' ');
expect(findings).toMatch(/bounded native fallback/i);
expect(findings).toContain('Outside Voice Integration Rule');
});
test('Eng cannot exit with unasked findings listed only in an unresolved-decisions report', () => {
const main = fs.readFileSync(path.join(ROOT, 'plan-eng-review/SKILL.md'), 'utf8');
const rawGate = extractMarkdownSection(main, '## EXIT PLAN MODE GATE (BLOCKING)');
const gate = rawGate.replace(/\s+/g, ' ');
const section = fs.readFileSync(path.join(ROOT, 'plan-eng-review/sections/review-sections.md'), 'utf8');
const readiness = extractMarkdownSection(section, '## Approval readiness');
const normalizedReadiness = readiness.replace(/\s+/g, ' ');
expect(section.indexOf('## Approval readiness')).toBeLessThan(section.indexOf('## Required outputs'));
expect(normalizedReadiness).toContain('`Approval readiness: PASS`');
expect(normalizedReadiness).toMatch(/navigation alone does not/i);
expect(readiness).not.toMatch(/^ {3}\S/m);
expect(gate).toMatch(/confirm approval readiness passed/i);
expectMentions(gate, [['not','verification','read-only']], 'gate');
const approvalParagraph = rawGate.slice(rawGate.indexOf('Confirm Approval readiness'), rawGate.indexOf('Verify all five checks'));
expect(approvalParagraph).not.toMatch(/^ {3}\S/m);
expect([...rawGate.matchAll(/^([1-5])\. /gm)].map(match => match[1])).toEqual(['1', '2', '3', '4', '5']);
ordered(gate, ['Confirm Approval readiness', '1. Read the report file']);
expect(normalizedReadiness).toMatch(/every accepted remedy/i);
expectMentions(normalizedReadiness, [['approval','auto-decision','authorized']], 'normalizedReadiness');
expect(normalizedReadiness).toMatch(/navigation do not count/i);
expect(normalizedReadiness).toMatch(/deferrals remain unresolved/i);
expect(normalizedReadiness).toMatch(/one dedicated decision/i);
expect(normalizedReadiness).toMatch(/mark that draft pending/i);
const decisionStart = section.indexOf('\n## Decision procedure\n');
const decisionEnd = section.indexOf('\n## Scope Challenge\n', decisionStart);
expect(decisionStart).toBeGreaterThan(0);
expect(decisionEnd).toBeGreaterThan(decisionStart);
const decisions = section.slice(decisionStart, decisionEnd).replace(/\s+/g, ' ');
expect(decisions).toContain('AskUserQuestion({ questions: [currentDecision] })');
const ask = decisions.indexOf('AskUserQuestion({ questions: [currentDecision] })');
const stop = decisions.search(/stop until the actual answer arrives/i);
expect(stop).toBeGreaterThan(ask);
expect(stop).toBeLessThan(decisions.indexOf('### Record the answer'));
expectMentions(gate, [['stop','verification','report']], 'gate');
expect(gate).toContain('**Recovery routing → Late change or missing work**');
const recovery = readSkillUnion('plan-eng-review').split('**Late change or missing work:**')[1]!.split('**Blocked outcome:**')[0]!.replace(/\s+/g, ' ');
ordered(recovery, ['Repeat Approval readiness', 'Required outputs', 'navigation']);
expect(gate).toContain('**Blocked outcome**');
const report = extractMarkdownSection(section, '### Write to the report file');
expect(report).toContain('**Blocked outcome**');
expect(report.replace(/\s+/g, ' ')).toMatch(/follow \*\*Blocked outcome\*\* before Review Log or decision logging/i);
});
test('approval entry does not alter other review Exit checklists', () => {
for (const skill of ['plan-devex-review']) {
const gate = extractMarkdownSection(readSkillUnion(skill), '## EXIT PLAN MODE GATE (BLOCKING)');
expect(gate).not.toContain('0. Approvals:');
expect(gate).toContain('1. Read the plan file');
expect(gate).toContain('5. If a plan file is in context');
}
});
test('task and decision reporting require approval while preserving unanswered findings', () => {
const section = fs.readFileSync(path.join(ROOT, 'plan-design-review/sections/review-sections.md'), 'utf8').replace(/\s+/g, ' ');
const reconcile = section.search(/before synthesizing tasks or the completion summary/i);
expect(reconcile).toBeGreaterThan(0);
expect(reconcile).toBeLessThan(section.indexOf('## Implementation Tasks'));
expectMentions(section, [['only','individually','decisions']], 'section');
expectMentions(section, [['not','including','finding']], 'section');
});
});
// --- {{DESIGN_DETECTOR}} resolver tests ---
describe('DESIGN_DETECTOR resolver', () => {
const designReview = () => fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
const designHtml = () => fs.readFileSync(path.join(ROOT, 'design-html', 'SKILL.md'), 'utf-8');
const bashBlocksOf = (content: string) => [...content.matchAll(/```bash\n([\s\S]*?)```/g)].map(m => m[1]);
test('design-review carries the probe, Phase 0, the DOM dump, and the run id', () => {
const c = designReview();
expect(c).toContain('gstack-design-detect.ts probe --host claude');
expect(c).toContain('IMPECCABLE_READY');
// the consent-gated install: offered once, only on the probe's say-so, never in spawned sessions, never via npx
expect(c).toContain('DESIGN_DETECTOR_INSTALL_OFFER');
expect(c).toContain('gstack-design-detect.ts install --host claude');
expect(c).toContain("Install impeccable's design detector engine?");
expect(c).toContain('gstack-config set design_detector_install_prompted true');
expect(c).toContain('`SESSION_KIND: spawned` or a headless run, never install and never ask');
expect(c).toContain('**Phase 0: mechanical scan**');
expect(c).toContain('scan --changed <base> --format gstack --host claude');
expect(c).toContain('### DOM dump (DOM mode only');
expect(c).toContain('data-gstack-dom-css');
expect(c).toContain(`$B js '('"$_DUMP"')()' --out "$_TMP/{page}.dom.html" --raw`);
expect(c).toContain('DOM_DUMP_OK');
expect(c).toContain('DOM_DUMP_REDACTION_BLOCKED');
expect(c).toContain('DOM_DUMP_TOO_LARGE');
expect(c).toContain('REPORT_DIR="$GSTACK_STATE_ROOT/projects/$SLUG/designs/design-audit-$(date +%Y%m%d)"');
expect(c).toContain('RUN_ID="$(date +%H%M%S)-$$"');
expect(c).toContain('"schemaVersion": 2');
expect(c).toContain('engine changed X → Y; rule set may differ');
expect(c).toContain('Detector: N → M');
expect(c).toContain('/impeccable typeset');
});
test('the DOM-dump script is loaded from lib/dom-dump.js, never inlined in the prose', () => {
const c = designReview();
expect(c).not.toMatch(/```js\n/);
expect(c).not.toContain('document.documentElement.cloneNode');
expect(c).toContain('_DUMP=$(cat "$HOME/.claude/skills/gstack/lib/dom-dump.js")');
expect(c).toContain(`const html = await pg.evaluate('"$_DUMP"');`);
expect(c).toContain('_TMP=$(mktemp -d "${TMPDIR:-/tmp}/gstack-dom.XXXXXX"); _DUMP=$(cat "$HOME/.claude/skills/gstack/lib/dom-dump.js")');
});
test('every rendered Aside script is single-quoted: a page-controlled <url> is never inside a double-quoted bash string', () => {
const files = [...fs.readdirSync(ROOT).filter(d => fs.existsSync(path.join(ROOT, d, 'SKILL.md'))).map(d => path.join(ROOT, d, 'SKILL.md')),
...fs.readdirSync(ROOT).flatMap(d => fs.existsSync(path.join(ROOT, d, 'sections')) ? fs.readdirSync(path.join(ROOT, d, 'sections')).filter(f => f.endsWith('.md')).map(f => path.join(ROOT, d, 'sections', f)) : [])];
expect(files.length).toBeGreaterThan(10);
for (const f of files) {
const c = fs.readFileSync(f, 'utf-8');
expect(c, path.relative(ROOT, f)).not.toMatch(/^aside repl "/m);
}
});
test('the E2E fixture slice markers exist in the rendered design skills (a template rename fails here, not in paid CI)', () => {
const dr = designReview();
const dh = fs.readFileSync(path.join(ROOT, 'design-html', 'SKILL.md'), 'utf-8');
for (const [a, b] of [['**Design detector (optional, deterministic):**', '**Create output directories:**'], ['**Phase 0: mechanical scan**', '## Phases 1-6'], ['### DOM dump (DOM mode only', '### Auth Detection']]) {
expect(sliceBetween(dr, a, b).length, `${a} .. ${b}`).toBeGreaterThan(100);
}
for (const [a, b] of [['**Design detector (optional, deterministic):**', '## Step 0: Input Detection'], ['### Slop Gate (bounded, never a loop)', '### Verification Screenshots']]) {
expect(sliceBetween(dh, a, b).length, `${a} .. ${b}`).toBeGreaterThan(100);
}
});
test('design-html carries the probe and the bounded slop gate', () => {
const c = designHtml();
expect(c).toContain('gstack-design-detect.ts probe --host claude');
expect(c).toContain('### Slop Gate (bounded, never a loop)');
expect(c).toContain('One pass, not a loop.');
expect(c).toContain('impeccable-disable <rule>: <reason>');
});
test('ship and review unions reach the detector through review-lite and the checklist', () => {
const ship = readSkillUnion('ship');
expect(ship).toContain('**Mechanical pass first.**');
expect(ship).toContain('scan --changed <base> --format gstack --host claude');
expect(ship).toContain('"detector":D');
expect(ship).toContain('Detector: "clean" | "N findings');
const review = readSkillUnion('review');
const checklist = fs.readFileSync(path.join(ROOT, 'review', 'design-checklist.md'), 'utf-8');
expect(checklist).toContain('**0. Mechanical pass first.**');
expect(checklist).toContain('IMPECCABLE_READY');
});
test('every rendered invocation uses bun --no-env-file and ends a scan with the exit echo; no bash block runs npx impeccable', () => {
for (const content of [designReview(), designHtml(), readSkillUnion('ship'), readSkillUnion('review'), fs.readFileSync(path.join(ROOT, 'review', 'design-checklist.md'), 'utf-8')]) {
for (const block of bashBlocksOf(content)) {
expect(block).not.toContain('npx impeccable');
for (const line of block.split('\n')) {
if (!line.includes('gstack-design-detect.ts')) continue;
expect(line).toContain('bun --no-env-file run ');
if (/gstack-design-detect\.ts scan /.test(line)) expect(line).toContain('echo "DETECT_EXIT_CODE=$?"');
}
}
}
});
test('--host is rendered per host', () => {
// Fresh codex render into a temp out-dir: the tracked tree is Claude-only and
// the gitignored .agents/ copy may be stale.
const out = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-detector-host-'));
try {
const r = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--out-dir', out], { cwd: ROOT, timeout: 120_000 });
expect(r.exitCode).toBe(0);
const codex = fs.readFileSync(path.join(out, '.agents', 'skills', 'gstack-design-review', 'SKILL.md'), 'utf-8');
expect(codex).toContain('gstack-design-detect.ts probe --host codex');
expect(codex).not.toContain('probe --host claude');
expect(codex).toContain('$GSTACK_ROOT/lib/dom-dump.js');
} finally {
fs.rmSync(out, { recursive: true, force: true });
}
});
});
// --- {{DESIGN_MD_CHECK}} resolver + open DESIGN.md adoption ---
describe('DESIGN_MD_CHECK resolver and open DESIGN.md adoption', () => {
test('design-consultation asks the conversion question once and writes the spec form', () => {
const c = readSkillUnion('design-consultation');
expect(c).toContain('gstack-design-md.ts check DESIGN.md');
expect(c).toContain('DESIGN_MD_FORMAT: spec');
expect(c).toContain('mark legacy-keep');
expect(c).toContain('convert --write');
expect(c).toContain('# gstack: design-md-format=spec');
expect(c).toContain("## Do's and Don'ts");
expect(c).toContain('## Elevation & Depth');
expect(c).toContain('fontFeature: tnum');
expect(c).toContain('"{colors.primary}"');
// the legacy template is gone
expect(c).not.toContain('## Product Context\n- **What this is:**');
});
test('design-review calibrates against tokens and never re-offers conversion; design-html writes the spec form', () => {
const dr = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
expect(dr).toContain('gstack-design-md.ts check DESIGN.md');
expect(dr).toContain('gstack-design-md.ts tokens DESIGN.md');
expect(dr).toMatch(/never offer a conversion/i);
expect(dr).not.toContain('mark legacy-keep');
const dh = fs.readFileSync(path.join(ROOT, 'design-html', 'SKILL.md'), 'utf-8');
expect(dh).toContain('# gstack: design-md-format=spec');
const pdr = readSkillUnion('plan-design-review');
expect(pdr).toContain('{colors.primary}');
const checklist = fs.readFileSync(path.join(ROOT, 'review', 'design-checklist.md'), 'utf-8');
expect(checklist).toContain('gstack-design-md.ts tokens DESIGN.md');
expect(readSkillUnion('ship')).toContain('gstack-design-md.ts tokens DESIGN.md');
});
test('every rendered gstack-design-md invocation uses bun --no-env-file', () => {
for (const content of [readSkillUnion('design-consultation'), fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8'), readSkillUnion('ship')]) {
for (const line of content.split('\n')) {
if (line.includes('gstack-design-md.ts')) expect(line).toContain('bun --no-env-file run ');
}
}
});
});
// --- PRODUCT.md prefill + /impeccable handoffs ---
describe('PRODUCT.md prefill and /impeccable handoffs', () => {
test('design-consultation and design-shotgun read PRODUCT.md and never open the impeccable skill', () => {
for (const skill of ['design-consultation', 'design-shotgun']) {
const c = readSkillUnion(skill);
expect(c).toContain('cat PRODUCT.md 2>/dev/null | head -120 || echo "NO_PRODUCT_MD"');
expect(c).toContain('do not re-ask');
expect(c).toContain('Never open `.claude/skills/impeccable/**`');
}
});
test('handoffs are gated on IMPECCABLE_SKILL: present in review-lite and design-review', () => {
expect(readSkillUnion('ship')).toContain('IMPECCABLE_SKILL: present`, end each NEEDS INPUT detector row with the `handoff=` command');
const dr = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
expectTokens(dr, ['`handoff=`', '`IMPECCABLE_SKILL: present`'], 'dr');
expectTokens(dr, ['`/impeccable`'], 'dr');
});
});
// --- Extended DESIGN_SKETCH resolver tests ---
describe('DESIGN_SKETCH extended with outside voices', () => {
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
test('contains outside design voices step', () => {
expect(content).toContain('Outside design voices');
});
test('offers opt-in via AskUserQuestion', () => {
expect(content).toContain('outside design perspectives');
});
test('still contains original wireframe steps', () => {
expect(content).toContain('wireframe');
expect(content).toContain('gstack-render.ts');
});
});
// --- Extended DESIGN_REVIEW_LITE resolver tests ---
describe('DESIGN_REVIEW_LITE extended with Codex', () => {
const content = readShipUnion();
test('contains Codex design voice block', () => {
expect(content).toContain('Codex design voice');
expect(content).toContain('CODEX (design)');
});
test('still contains original checklist steps', () => {
expect(content).toContain('design-checklist.md');
expect(content).toContain('SCOPE_FRONTEND');
});
test('design-checklist path uses installed gstack/review root (#2694)', () => {
// #2694: generateDesignReviewLite used to emit
// `.claude/skills/review/design-checklist.md` (missing the gstack/ segment).
// After install the file lives at ~/.claude/skills/gstack/review/design-checklist.md.
// The bad relative form must not appear — the good path does not contain it
// as a substring because `gstack/` sits between `skills/` and `review/`.
expect(content).toContain('~/.claude/skills/gstack/review/design-checklist.md');
expect(content).not.toContain('.claude/skills/review/design-checklist.md');
});
});
// ─── Codex Generation Tests ─────────────────────────────────
describe('Codex generation (--host codex)', () => {
// .agents/ is gitignored (v0.11.2.0) — read the module-level out-dir render
// (--host all covers codex) instead of regenerating the live tree in place.
const AGENTS_DIR = path.join(EXTERNAL_OUT, '.agents', 'skills');
// Dynamic discovery of expected Codex skills: all templates except /codex.
// The out-dir is a fresh mkdtemp, so the vendored-dev-mode symlink loop
// (.agents/skills/{name} → repo root) that made the generator skip skills
// in-place can never occur here — every template renders.
const CODEX_SKILLS = (() => {
const skills: Array<{ dir: string; codexName: string }> = [];
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
skills.push({ dir: '.', codexName: 'gstack' });
}
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
if (entry.name === 'codex') continue; // /codex is excluded from Codex output
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
const codexName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
skills.push({ dir: entry.name, codexName });
}
return skills;
})();
test('--host codex generates correct output paths', () => {
for (const skill of CODEX_SKILLS) {
const skillMd = path.join(AGENTS_DIR, skill.codexName, 'SKILL.md');
expect(fs.existsSync(skillMd)).toBe(true);
}
});
test('root gstack bundle has OpenAI metadata for Codex skill browsing', () => {
const rootMetadata = path.join(ROOT, 'agents', 'openai.yaml');
expect(fs.existsSync(rootMetadata)).toBe(true);
const content = fs.readFileSync(rootMetadata, 'utf-8');
expect(content).toContain('display_name: "gstack"');
expect(content).toContain('allow_implicit_invocation: true');
});
test('externalSkillName mapping: root is gstack, others are gstack-{dir}', () => {
// Root → gstack
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack', 'SKILL.md'))).toBe(true);
// Subdirectories → gstack-{dir}
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'))).toBe(true);
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'))).toBe(true);
// gstack-upgrade doesn't double-prefix
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-upgrade', 'SKILL.md'))).toBe(true);
// No double-prefix: gstack-gstack-upgrade must NOT exist
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-gstack-upgrade', 'SKILL.md'))).toBe(false);
});
test('Codex frontmatter has ONLY name + description', () => {
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
expect(content.startsWith('---\n')).toBe(true);
const fmEnd = content.indexOf('\n---', 4);
expect(fmEnd).toBeGreaterThan(0);
const frontmatter = content.slice(4, fmEnd);
// Must have name and description
expect(frontmatter).toContain('name:');
expect(frontmatter).toContain('description:');
// Must NOT have allowed-tools, version, or hooks
expect(frontmatter).not.toContain('allowed-tools:');
expect(frontmatter).not.toContain('version:');
expect(frontmatter).not.toContain('hooks:');
}
});
test('all Codex skills have agents/openai.yaml metadata', () => {
for (const skill of CODEX_SKILLS) {
const metadata = path.join(AGENTS_DIR, skill.codexName, 'agents', 'openai.yaml');
expect(fs.existsSync(metadata)).toBe(true);
const content = fs.readFileSync(metadata, 'utf-8');
expect(content).toContain(`display_name: "${skill.codexName}"`);
expect(content).toContain('short_description:');
expect(content).toContain('allow_implicit_invocation: true');
}
});
test('no .claude/skills/ in Codex output', () => {
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('.claude/skills');
}
});
test('no ~/.claude/ runtime paths in Codex output', () => {
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
// Outside prompts may explicitly forbid reading Claude's skill directory.
// Every executable/runtime path must still use the selected host root.
const withoutBoundary = content.replace(/^.*do not read or execute any files under[^\n]*$/gim, '');
expect(withoutBoundary).not.toContain('~/.claude/');
}
});
test('/codex skill excluded from Codex output', () => {
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex'))).toBe(false);
});
test('Codex output includes the renamed Claude Code skill and restricted runner', () => {
const content = readExternalSkillUnion(EXTERNAL_OUT, '.agents', 'claude-code');
expect(content).toContain('name: claude-code');
expect(content).toContain('gstack-claude-code');
expect(content).toContain('--access none');
expect(content).toContain('--access read-only');
expect(content).toContain('--timeout-ms');
expect(content).toContain('claude-session-id');
expect(content).not.toContain('$HOME/.claude/.credentials.json');
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-claude'))).toBe(false);
});
test('Codex ship and review preserve outside review through Claude Code', () => {
for (const skill of ['ship', 'review']) {
const content = readExternalSkillUnion(EXTERNAL_OUT, '.agents', skill);
expect(content).not.toMatch(/codex\s+(?:exec|review)\s/);
expect(content).toContain('gstack-claude-code');
expect(content).toContain('codex_reviews');
expect(content).toContain('adversarial-review');
}
});
test('--host codex --dry-run freshness', () => {
// Dry-run against the out-dir render: determinism/idempotency check
// (regenerating produces the same bytes the module-level render did).
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run', '--out-dir', EXTERNAL_OUT], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
timeout: 120_000,
});
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
// Every Codex skill should be FRESH
for (const skill of CODEX_SKILLS) {
expect(output).toContain(`FRESH: .agents/skills/${skill.codexName}/SKILL.md`);
}
expect(output).not.toContain('STALE');
});
test('--host agents alias produces same output as --host codex', () => {
const codexResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run', '--out-dir', EXTERNAL_OUT], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
timeout: 120_000,
});
const agentsResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'agents', '--dry-run', '--out-dir', EXTERNAL_OUT], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
timeout: 120_000,
});
expect(codexResult.exitCode).toBe(0);
expect(agentsResult.exitCode).toBe(0);
// Both should produce the same output (same FRESH lines)
expect(codexResult.stdout.toString()).toBe(agentsResult.stdout.toString());
});
test('multiline descriptions preserved in Codex output', () => {
// office-hours has a multiline description — verify it survives the frontmatter transform
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-office-hours', 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
// Description should span multiple lines (block scalar)
const descLines = frontmatter.split('\n').filter(l => l.startsWith(' '));
expect(descLines.length).toBeGreaterThan(1);
// Verify key phrases survived
expect(frontmatter).toContain('YC Office Hours');
});
test('hook skills have safety prose and no hooks: in frontmatter', () => {
const HOOK_SKILLS = ['gstack-careful', 'gstack-freeze', 'gstack-guard'];
for (const skillName of HOOK_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skillName, 'SKILL.md'), 'utf-8');
// Must have safety advisory prose
expect(content).toContain('Safety Advisory');
// Must NOT have hooks: in frontmatter
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
expect(frontmatter).not.toContain('hooks:');
}
});
test('all Codex SKILL.md files have auto-generated header', () => {
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
expect(content).toContain('Regenerate: bun run gen:skill-docs');
}
});
test('Codex preamble resolves runtime assets from repo-local or global gstack roots', () => {
// Check a skill that has a preamble (review is a good candidate)
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('GSTACK_ROOT');
// C1: one shared root resolution — repo-local first, then CODEX_HOME's global root.
expect(content).toContain('_r=$(git rev-parse --show-toplevel 2>/dev/null)/.agents/skills/gstack');
expect(content).toContain('_r=${CODEX_HOME:-~/.codex}/skills/gstack');
// Phase 1/2: config reads moved into gstack-skill-start — the fence itself
// is the bin asset the preamble must resolve through $GSTACK_BIN, and the
// question-preference runtime call still resolves the same way.
expect(content).toContain('$GSTACK_BIN/gstack-skill-start');
expect(content).toContain('$GSTACK_BIN/gstack-question-preference');
// The upgrade-skill doc reference moved into the script's upgrade-flow
// block, resolved $0-relative ($_ROOT_DIR) — host-neutral by construction,
// so the Codex render no longer needs its own copy.
expect(SKILL_START_SCRIPT).toContain('$_ROOT_DIR/gstack-upgrade/SKILL.md');
expect(SKILL_START_SCRIPT).toContain('_ROOT_DIR=$(dirname "$_BIN")');
expect(content).not.toContain('~/.codex/skills/gstack/bin/gstack-config get telemetry');
});
// ─── Path rewriting regression tests ─────────────────────────
test('sidecar paths resolve through $GSTACK_ROOT (not gstack-review/)', () => {
// #2518: templates now anchor sidecars at the installed skill root
// (~/.claude/skills/gstack/review/...), which the codex path rewrite turns
// into $GSTACK_ROOT/review/... — resolved by the preamble against the
// repo-local .agents root or the global install. The old repo-relative
// form (.claude/skills/review/) only resolved inside gstack's own checkout.
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('$GSTACK_ROOT/review/checklist.md');
// design-checklist.md is now referenced via Review Army specialist (Claude only, stripped for Codex)
// Wrong: must NOT reference gstack-review/checklist.md (file doesn't exist there)
expect(content).not.toContain('.agents/skills/gstack-review/checklist.md');
});
test('sidecar paths in ship skill point to gstack/review/ for pre-landing review', () => {
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
// Ship references the review checklist in its pre-landing review step
if (content.includes('checklist.md')) {
expect(content).toContain('.agents/skills/gstack/review/');
expect(content).not.toContain('.agents/skills/gstack-review/checklist');
}
});
test('greptile-triage sidecar path is correct', () => {
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
if (content.includes('greptile-triage')) {
expect(content).toContain('$GSTACK_ROOT/review/greptile-triage.md');
expect(content).not.toContain('.agents/skills/gstack-review/greptile-triage');
}
});
test('all four path rewrite rules produce correct output', () => {
// Test each of the 4 path rewrite rules individually
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
// Rule 1: ~/.claude/skills/gstack → $GSTACK_ROOT
expect(content).not.toContain('~/.claude/skills/gstack');
expect(content).toContain('$GSTACK_ROOT');
// Rule 2: .claude/skills/gstack → .agents/skills/gstack
expect(content).not.toContain('.claude/skills/gstack');
// Rule 3: .claude/skills/review → .agents/skills/gstack/review
expect(content).not.toContain('.claude/skills/review');
// Rule 4: .claude/skills → .agents/skills (catch-all)
expect(content).not.toContain('.claude/skills');
});
test('path rewrite rules apply to all Codex skills with sidecar references', () => {
// Verify across ALL generated skills, not just review
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
// No skill should reference Claude paths
expect(content).not.toContain('~/.claude/skills');
expect(content).not.toContain('.claude/skills');
if (content.includes('gstack-config') || content.includes('gstack-update-check') || content.includes('gstack-telemetry-log')) {
expect(content).toContain('$GSTACK_ROOT');
}
// If a skill references checklist.md, it must use the correct sidecar path
if (content.includes('checklist.md') && !content.includes('design-checklist.md')) {
expect(content).not.toContain('gstack-review/checklist.md');
}
}
});
// ─── Claude output regression guard ─────────────────────────
test('Claude output uses installed-root review paths (#2518)', () => {
// Codex changes must NOT affect Claude output; the Claude form is the
// installed-root anchor, not the old repo-relative path that only
// resolved inside gstack's own checkout.
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('~/.claude/skills/gstack/review/checklist.md');
expect(content).toContain('~/.claude/skills/gstack');
// Must NOT contain Codex HOST paths. `~/.codex/sessions/` is exempt: the
// timeout-wrapper guidance documents the Codex CLI's own rollout-log
// location (a user-facing CLI path, same class as ~/.codex/logs/ in the
// codex skill), not the gstack Codex host install path.
// `~/.codex/config.toml` is the same user-facing class: the shared
// codexPreflight's model_unusable branch (#2477) points at the CLI's own
// config file, where the rejected `model =` pin lives.
expect(content).not.toContain('.agents/skills');
expect(
content
.replaceAll('~/.codex/sessions/', '')
.replaceAll('~/.codex/config.toml', ''),
).not.toContain('~/.codex/');
});
test('Claude output unchanged: ship skill still uses .claude/skills/ paths', () => {
const content = readShipUnion();
expect(content).toContain('~/.claude/skills/gstack');
expect(content).not.toContain('.agents/skills');
// ~/.codex/sessions/ is the Codex CLI's rollout-log path (user-facing),
// documented by the adversarial-pass timeout guidance; ~/.codex/config.toml
// is the CLI's own config file (model_unusable guidance, #2477) — see the
// review test above.
expect(
content
.replaceAll('~/.codex/sessions/', '')
.replaceAll('~/.codex/config.toml', ''),
).not.toContain('~/.codex/');
});
test('Claude output unchanged: all Claude skills have zero Codex paths', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
// pair-agent legitimately documents how Codex agents store credentials.
// codex + autoplan document the Codex CLI auth file (~/.codex/auth.json)
// and log path (~/.codex/logs/) — those are user-facing Codex CLI paths,
// not the gstack Codex host install path. ~/.codex/sessions/ (rollout
// logs, referenced by the review/ship timeout guidance) and
// ~/.codex/config.toml (the model_unusable guidance in the shared
// codexPreflight, #2477) are the same user-facing class, so they are
// scrubbed before the ban.
if (skill.dir !== 'pair-agent' && skill.dir !== 'codex' && skill.dir !== 'autoplan') {
expect(
content
.replaceAll('~/.codex/sessions/', '')
.replaceAll('~/.codex/config.toml', ''),
).not.toContain('~/.codex/');
}
// gstack-upgrade legitimately references .agents/skills for cross-platform detection
if (skill.dir !== 'gstack-upgrade') {
expect(content).not.toContain('.agents/skills');
}
}
});
// ─── Design outside voices: Codex host guard ─────────────────
test('codex host retains design outside voices through Claude Code', () => {
const codexContent = readExternalSkillUnion(EXTERNAL_OUT, '.agents', 'design-review');
expect(codexContent).toContain('Design Outside Voices');
expect(codexContent).toContain('gstack-claude-code');
});
test('codex host does not include Codex design block in ship', () => {
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(codexContent).not.toContain('Codex design voice');
});
// ─── Explicit --model override wins over the host default ────
// Without --model the codex host renders its defaultModel (gpt) — pinned by
// the golden test. This pins the OTHER direction through the real CLI:
// `./setup --host codex --model <id>` depends on it. The override renders
// into its OWN out-dir, so no restore pass is needed — the host-default
// render (EXTERNAL_OUT) is untouched and asserted directly.
test('explicit --model overrides the codex host default', () => {
const overrideOut = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-model-override-'));
try {
const override = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--model', 'claude', '--out-dir', overrideOut], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
timeout: 120_000,
});
expect(override.exitCode).toBe(0);
const content = fs.readFileSync(path.join(overrideOut, '.agents', 'skills', 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(content).toContain('Model-Specific Behavioral Patch (claude)');
// The overlay now travels as --model into gstack-skill-start, which
// echoes MODEL_OVERLAY at runtime.
expect(content).toContain('--model "claude"');
} finally {
fs.rmSync(overrideOut, { recursive: true, force: true });
}
// Host-default direction: the untouched EXTERNAL_OUT render carries gpt.
const hostDefault = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(hostDefault).toContain('Model-Specific Behavioral Patch (gpt)');
expect(hostDefault).toContain('--model "gpt"');
});
});
// ─── Factory generation tests ────────────────────────────────
describe('Factory generation (--host factory)', () => {
// .factory/ is gitignored — read the module-level out-dir render
// (--host all covers factory) instead of regenerating in place.
const FACTORY_DIR = path.join(EXTERNAL_OUT, '.factory', 'skills');
// Fresh out-dir → the vendored-dev-mode symlink loop can never occur, so
// every template renders (see the Codex discovery note above).
const FACTORY_SKILLS = (() => {
const skills: Array<{ dir: string; factoryName: string }> = [];
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
skills.push({ dir: '.', factoryName: 'gstack' });
}
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
if (entry.name === 'codex') continue;
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
const factoryName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
skills.push({ dir: entry.name, factoryName });
}
return skills;
})();
test('--host factory generates correct output paths', () => {
for (const skill of FACTORY_SKILLS) {
const skillMd = path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md');
expect(fs.existsSync(skillMd)).toBe(true);
}
});
test('Factory frontmatter has name + description + user-invocable', () => {
for (const skill of FACTORY_SKILLS) {
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
expect(frontmatter).toContain('name:');
expect(frontmatter).toContain('description:');
expect(frontmatter).toContain('user-invocable: true');
expect(frontmatter).not.toContain('allowed-tools:');
expect(frontmatter).not.toContain('preamble-tier:');
expect(frontmatter).not.toContain('sensitive:');
}
});
test('sensitive skills have disable-model-invocation', () => {
const SENSITIVE = ['gstack-ship', 'gstack-land-and-deploy', 'gstack-guard', 'gstack-careful', 'gstack-freeze', 'gstack-unfreeze'];
for (const name of SENSITIVE) {
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
expect(frontmatter).toContain('disable-model-invocation: true');
}
});
test('non-sensitive skills lack disable-model-invocation', () => {
const NON_SENSITIVE = ['gstack-qa', 'gstack-review', 'gstack-investigate', 'gstack-browse'];
for (const name of NON_SENSITIVE) {
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
expect(frontmatter).not.toContain('disable-model-invocation');
}
});
test('no .claude/skills/ in Factory output', () => {
for (const skill of FACTORY_SKILLS) {
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('.claude/skills');
}
});
test('no ~/.claude/skills/ paths in Factory output', () => {
for (const skill of FACTORY_SKILLS) {
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
// ~/.claude/skills should be rewritten, but ~/.claude/plans is legitimate
// (plan directory lookup) and ~/.claude/ in codex prompts is intentional
expect(content).not.toContain('~/.claude/skills');
}
});
test('Factory installs both outside-review wrapper skills', () => {
for (const skill of ['gstack-codex', 'gstack-claude-code']) {
expect(fs.existsSync(path.join(FACTORY_DIR, skill, 'SKILL.md'))).toBe(true);
}
});
test('Factory keeps Codex integration blocks', () => {
// Factory users CAN use Codex second opinions (codex exec is a standalone binary)
const shipContent = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(shipContent).toContain('codex');
});
test('no agents/openai.yaml in Factory output', () => {
for (const skill of FACTORY_SKILLS) {
const yamlPath = path.join(FACTORY_DIR, skill.factoryName, 'agents', 'openai.yaml');
expect(fs.existsSync(yamlPath)).toBe(false);
}
});
test('--host droid alias works', () => {
const factoryResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run', '--out-dir', EXTERNAL_OUT], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe', timeout: 120_000,
});
const droidResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'droid', '--dry-run', '--out-dir', EXTERNAL_OUT], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe', timeout: 120_000,
});
expect(factoryResult.exitCode).toBe(0);
expect(droidResult.exitCode).toBe(0);
expect(factoryResult.stdout.toString()).toBe(droidResult.stdout.toString());
});
test('--host factory --dry-run freshness', () => {
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run', '--out-dir', EXTERNAL_OUT], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe', timeout: 120_000,
});
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
for (const skill of FACTORY_SKILLS) {
expect(output).toContain(`FRESH: .factory/skills/${skill.factoryName}/SKILL.md`);
}
expect(output).not.toContain('STALE');
});
test('Factory preamble uses .factory paths', () => {
const content = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('GSTACK_ROOT');
expect(content).toContain('_r=$(git rev-parse --show-toplevel 2>/dev/null)/.factory/skills/gstack');
expect(content).toContain('$GSTACK_BIN/gstack-config');
});
});
// ─── Parameterized host smoke tests (config-driven) ─────────
import { ALL_HOST_CONFIGS, getExternalHosts } from '../hosts/index';
import { sliceBetween } from './helpers/skill-fixture';
describe('Parameterized host smoke tests', () => {
// Every external host was rendered up front by the module-level
// `--host all --out-dir EXTERNAL_OUT` render, so the per-host `--dry-run`
// freshness checks are deterministic: they compare a regeneration against
// that render — an idempotency/determinism check that catches
// non-deterministic gen without ever writing (or depending on) the live
// gitignored host dirs. The tracked-claude freshness test
// (`generated files are fresh`) runs earlier and is unaffected.
for (const hostConfig of getExternalHosts()) {
describe(`${hostConfig.displayName} (--host ${hostConfig.name})`, () => {
const hostDir = path.join(EXTERNAL_OUT, hostConfig.hostSubdir, 'skills');
test('generates output that exists on disk', () => {
// The module-level --host all render must have produced this host's tree.
expect(fs.existsSync(hostDir)).toBe(true);
const skills = fs.readdirSync(hostDir).filter(d =>
fs.existsSync(path.join(hostDir, d, 'SKILL.md'))
);
expect(skills.length).toBeGreaterThan(0);
});
test('no .claude/skills path leakage outside repo-root sidecar symlinks', () => {
expect(fs.existsSync(hostDir)).toBe(true);
const skills = fs.readdirSync(hostDir);
for (const skill of skills) {
// Dev installs may mount the repo root at host/skills/gstack as a runtime
// sidecar. The generator skips that symlink loop, so leakage checks should too.
if (isRepoRootSymlink(path.join(hostDir, skill))) continue;
const skillMd = path.join(hostDir, skill, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
const leaks = externalHostPathLeaks(content);
if (leaks.length > 0) {
throw new Error(`${skill}: .claude/skills leakage:\n${leaks.slice(0, 3).join('\n')}`);
}
}
});
test('frontmatter has name and description', () => {
expect(fs.existsSync(hostDir)).toBe(true);
const skills = fs.readdirSync(hostDir);
for (const skill of skills) {
const skillMd = path.join(hostDir, skill, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
expect(content).toMatch(/^---\n/);
expect(content).toMatch(/^name:\s/m);
expect(content).toMatch(/^description:\s/m);
}
});
test('--dry-run freshness check passes', () => {
const result = Bun.spawnSync(
['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name, '--dry-run', '--out-dir', EXTERNAL_OUT],
{ cwd: ROOT, stdout: 'pipe', stderr: 'pipe', timeout: 120_000 }
);
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
expect(output).not.toContain('STALE');
});
if (hostConfig.generation.skipSkills?.includes('codex')) {
test('/codex skill excluded', () => {
expect(fs.existsSync(path.join(hostDir, 'gstack-codex', 'SKILL.md'))).toBe(false);
});
}
});
}
});
// ─── --host all tests ────────────────────────────────────────
describe('--host all', () => {
// Same determinism guard as the parameterized block: the module-level
// `--host all --out-dir EXTERNAL_OUT` render is the comparison baseline, so
// this dry-run reports FRESH regardless of live-tree state — and proves the
// claude host plus every external host regenerate deterministically.
test('--host all generates for all registered hosts', () => {
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'all', '--dry-run', '--out-dir', EXTERNAL_OUT], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe', timeout: 120_000,
});
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
// All hosts should appear in output
expect(output).toContain('FRESH: SKILL.md'); // claude
for (const hostConfig of getExternalHosts()) {
expect(output).toContain(`FRESH: ${hostConfig.hostSubdir}/skills/`);
}
});
});
// ─── Setup script validation ─────────────────────────────────
// These tests verify the setup script's install layout matches
// what the generator produces — catching the bug where setup
// installed Claude-format source dirs for Codex users.
describe('setup script validation', () => {
const setupContent = fs.readFileSync(path.join(ROOT, 'setup'), 'utf-8');
test('setup has separate link functions for Claude and Codex', () => {
expect(setupContent).toContain('link_claude_skill_dirs');
expect(setupContent).toContain('link_codex_skill_dirs');
// Old unified function must not exist
expect(setupContent).not.toMatch(/^link_skill_dirs\(\)/m);
});
test('Claude install uses link_claude_skill_dirs', () => {
// The Claude install section (section 4) should use the Claude function
const claudeSection = setupContent.slice(
setupContent.indexOf('# 4. Install for Claude'),
setupContent.indexOf('# 5. Install for Codex')
);
expect(claudeSection).toContain('link_claude_skill_dirs');
expect(claudeSection).not.toContain('link_codex_skill_dirs');
});
test('Codex install uses link_codex_skill_dirs', () => {
// The Codex install section (section 5) should use the Codex function
// End marker: the next numbered section header (a marker that doesn't
// exist slices to EOF and the assertion reads unrelated sections).
const codexSection = setupContent.slice(
setupContent.indexOf('# 5. Install for Codex'),
setupContent.indexOf('# 6. Install for Kiro')
);
expect(setupContent.indexOf('# 6. Install for Kiro')).toBeGreaterThan(-1);
expect(codexSection).toContain('create_codex_runtime_root');
expect(codexSection).toContain('link_codex_skill_dirs');
expect(codexSection).not.toContain('link_claude_skill_dirs');
expect(codexSection).not.toContain('_link_or_copy "$GSTACK_DIR" "$CODEX_GSTACK"');
});
test('Codex install prefers repo-local .agents/skills when setup runs from there', () => {
expect(setupContent).toContain('SKILLS_PARENT_BASENAME');
expect(setupContent).toContain('CODEX_REPO_LOCAL=0');
expect(setupContent).toContain('[ "$SKILLS_PARENT_BASENAME" = ".agents" ]');
expect(setupContent).toContain('CODEX_REPO_LOCAL=1');
expect(setupContent).toContain('CODEX_SKILLS="$INSTALL_SKILLS_DIR"');
});
test('setup separates install path from source path for symlinked repo-local installs', () => {
expect(setupContent).toContain('INSTALL_GSTACK_DIR=');
expect(setupContent).toContain('SOURCE_GSTACK_DIR=');
expect(setupContent).toContain('INSTALL_SKILLS_DIR=');
expect(setupContent).toContain('CODEX_GSTACK="$INSTALL_GSTACK_DIR"');
// Links come from the source checkout, or its per-install render (#1882).
expect(setupContent).toContain('link_codex_skill_dirs "${_CODEX_RENDER_ROOT:-$SOURCE_GSTACK_DIR}" "$CODEX_SKILLS"');
});
test('Codex installs always create sidecar runtime assets for the real skill target', () => {
expect(setupContent).toContain('if [ "$INSTALL_CODEX" -eq 1 ]; then');
expect(setupContent).toContain('create_agents_sidecar "$SOURCE_GSTACK_DIR"');
});
test('link_codex_skill_dirs reads from .agents/skills/', () => {
// The Codex link function must reference .agents/skills for generated Codex skills
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('.agents/skills');
expect(fnBody).toContain('gstack*');
});
test('link_claude_skill_dirs creates real directories with absolute SKILL.md symlinks', () => {
// Claude links should be real directories with absolute SKILL.md symlinks
// to ensure Claude Code discovers them as top-level skills (not nested under gstack/)
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('mkdir -p "$target"');
// v1.36.0.0: routes through _link_or_copy helper for Windows fallback (cp on MSYS2/Git Bash).
// v1.67 (#2569): the source is render-aware — canonical SKILL.md, or the
// rendered :user variant from ${GSTACK_HOME}/render/claude when present.
expect(fnBody).toContain('_skill_md_src="$gstack_dir/$dir_name/SKILL.md"');
expect(fnBody).toContain('_link_or_copy "$_skill_md_src" "$target/SKILL.md"');
});
// REGRESSION: cleanup functions must handle both old symlinks AND new real-directory pattern
test('cleanup functions handle real directories with symlinked SKILL.md', () => {
// cleanup_old_claude_symlinks must detect and remove real dirs with SKILL.md symlinks
const cleanupOldStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
const cleanupOldEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up old', cleanupOldStart));
const cleanupOldBody = setupContent.slice(cleanupOldStart, cleanupOldEnd);
expect(cleanupOldBody).toContain('-d "$old_target"');
expect(cleanupOldBody).toContain('-L "$old_target/SKILL.md"');
expect(cleanupOldBody).toContain('rm -rf "$old_target"');
// cleanup_prefixed_claude_symlinks must also handle the new pattern
const cleanupPrefixedStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
const cleanupPrefixedEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up prefixed', cleanupPrefixedStart));
const cleanupPrefixedBody = setupContent.slice(cleanupPrefixedStart, cleanupPrefixedEnd);
expect(cleanupPrefixedBody).toContain('-d "$prefixed_target"');
expect(cleanupPrefixedBody).toContain('-L "$prefixed_target/SKILL.md"');
expect(cleanupPrefixedBody).toContain('rm -rf "$prefixed_target"');
});
// REGRESSION: link function must upgrade old directory symlinks
test('link_claude_skill_dirs removes old directory symlinks before creating real dirs', () => {
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
// Must check for and remove old symlinks before mkdir
expect(fnBody).toContain('if [ -L "$target" ]');
expect(fnBody).toContain('rm -f "$target"');
});
test('setup links root gstack skill through a thin Claude wrapper alias', () => {
const fnStart = setupContent.indexOf('link_claude_root_skill_alias()');
const fnEnd = setupContent.indexOf('# ─── Helper: remove old unprefixed Claude skill entries', fnStart);
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('_gstack-command');
// #2511: the alias must be a rewritten COPY (unique frontmatter name),
// never a verbatim symlink of the canonical SKILL.md.
expect(fnBody).toContain('_install_alias_skill_md "$gstack_dir/SKILL.md" "$target" "_gstack-command"');
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir/SKILL.md"');
const claudeSection = setupContent.slice(
setupContent.indexOf('# 4. Install for Claude'),
setupContent.indexOf('# 5. Install for Codex')
);
expect(claudeSection).toContain('link_claude_root_skill_alias "$SOURCE_GSTACK_DIR" "$INSTALL_SKILLS_DIR"');
});
test('setup supports --host auto|claude|codex|kiro|opencode|cursor; slate is informational', () => {
expect(setupContent).toContain('--host');
// #2361: slate moved OUT of the install accept-list (it was accepted but
// never dispatched — a silent exit-0 no-op) into an informational arm.
expect(setupContent).toContain('claude|codex|kiro|factory|opencode|cursor|copilot|auto');
expect(setupContent).toMatch(/^ {2}slate\)/m);
});
test('auto mode detects claude, codex, kiro, and opencode binaries', () => {
expect(setupContent).toContain('command -v claude');
expect(setupContent).toContain('command -v codex');
expect(setupContent).toContain('command -v kiro-cli');
expect(setupContent).toContain('command -v opencode');
});
// T1: Sidecar skip guard — prevents .agents/skills/gstack from being linked as a skill
test('link_codex_skill_dirs skips the gstack sidecar directory', () => {
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
});
// T2: Dynamic $GSTACK_ROOT paths in generated Codex preambles
test('generated Codex preambles use dynamic GSTACK_ROOT paths', () => {
// Read the module-level out-dir render (always present).
const codexSkillDir = path.join(EXTERNAL_OUT, '.agents', 'skills', 'gstack-ship');
const content = fs.readFileSync(path.join(codexSkillDir, 'SKILL.md'), 'utf-8');
expect(content).toContain('GSTACK_ROOT=');
expect(content).toContain('$GSTACK_BIN/');
});
test('setup installs Kiro-native output and runtime assets', () => {
expect(setupContent).toContain('INSTALL_KIRO=');
expect(setupContent).toContain('kiro-cli');
expect(setupContent).toContain('KIRO_SKILLS=');
expect(setupContent).toContain('KIRO_SKILLS="$HOME/.kiro/skills"');
expect(setupContent).toContain('KIRO_DIR="$SOURCE_GSTACK_DIR/.kiro/skills"');
expect(setupContent).toContain('gen:skill-docs --host kiro');
expect(setupContent).toContain('$KIRO_GSTACK/lib');
});
test('setup supports --host opencode with install section and OpenCode skill path vars', () => {
expect(setupContent).toContain('INSTALL_OPENCODE=');
expect(setupContent).toContain('OPENCODE_SKILLS="$HOME/.config/opencode/skills"');
expect(setupContent).toContain('OPENCODE_GSTACK="$OPENCODE_SKILLS/gstack"');
});
// --host cursor full install slice (#1358, PR #2547 by @szsunyuan re-derived)
test('auto mode detects Cursor via binary or ~/.cursor directory', () => {
expect(setupContent).toContain('command -v cursor');
expect(setupContent).toContain('[ -d "$HOME/.cursor" ] && INSTALL_CURSOR=1');
});
test('setup supports --host cursor with install section and Cursor skill path vars', () => {
expect(setupContent).toContain('INSTALL_CURSOR=');
expect(setupContent).toContain('CURSOR_SKILLS="$HOME/.cursor/skills"');
expect(setupContent).toContain('CURSOR_GSTACK="$CURSOR_SKILLS/gstack"');
expect(setupContent).toContain('create_cursor_runtime_root');
expect(setupContent).toContain('create_cursor_sidecar');
expect(setupContent).toContain('link_cursor_skill_dirs');
expect(setupContent).toContain('gstack ready (cursor).');
});
test('create_cursor_runtime_root exposes only Cursor runtime assets', () => {
const fnStart = setupContent.indexOf('create_cursor_runtime_root()');
const fnEnd = setupContent.indexOf('create_cursor_sidecar()', fnStart);
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('gstack/SKILL.md');
expect(fnBody).toContain('browse/dist');
expect(fnBody).toContain('browse/bin');
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
expect(fnBody).toContain('checklist.md');
expect(fnBody).toContain('TODOS-format.md');
// bin scripts import ../lib — the two must travel together.
expect(fnBody).toContain('$cursor_gstack/lib');
expect(fnBody).not.toContain('design-checklist.md');
expect(fnBody).not.toContain('greptile-triage.md');
expect(fnBody).not.toContain('review/specialists');
expect(fnBody).not.toContain('qa/templates');
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$cursor_gstack"');
});
test('create_cursor_sidecar plants runtime assets without wiping generated SKILL.md', () => {
const fnStart = setupContent.indexOf('create_cursor_sidecar()');
const fnEnd = setupContent.indexOf('link_cursor_skill_dirs()', fnStart);
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('.cursor/skills/gstack');
expect(fnBody).toContain('bin');
expect(fnBody).toContain('browse/dist');
expect(fnBody).toContain('browse/bin');
expect(fnBody).toContain('ETHOS.md');
expect(fnBody).not.toContain('rm -rf');
});
test('link_cursor_skill_dirs skips the gstack runtime root directory', () => {
const fnStart = setupContent.indexOf('link_cursor_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
// #2444-aware guard: Windows bypass, else only replace symlink-or-missing.
expect(fnBody).toContain('[ "$IS_WINDOWS" -eq 1 ] || [ -L "$target" ] || [ ! -e "$target" ]');
});
// #2142 deleted existing ~/.cursor/skills/<name> dirs with `rm -rf "$target"`
// before relinking. That can wipe unowned Cursor skills. Only replace a
// symlink or a missing path; never the whole skills directory.
test('link_cursor_skill_dirs does not delete unowned Cursor skill directories', () => {
const fnStart = setupContent.indexOf('link_cursor_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).not.toContain('rm -rf "$target"');
expect(fnBody).not.toContain('rm -rf "$skills_dir"');
expect(setupContent).not.toContain('rm -rf "$CURSOR_SKILLS"');
});
test('Cursor install links generated skills before planting the sidecar', () => {
const cursorInstall = setupContent.slice(
setupContent.indexOf('# 6d. Install for Cursor'),
setupContent.indexOf('# 7. Create .agents/ sidecar'),
);
const linkCall = cursorInstall.indexOf('link_cursor_skill_dirs "$SOURCE_GSTACK_DIR"');
const sidecarCall = cursorInstall.indexOf('create_cursor_sidecar "$SOURCE_GSTACK_DIR"');
expect(linkCall).toBeGreaterThan(-1);
expect(sidecarCall).toBeGreaterThan(-1);
expect(linkCall).toBeLessThan(sidecarCall);
});
test('setup installs OpenCode skills into a nested gstack runtime root', () => {
expect(setupContent).toContain('create_opencode_runtime_root');
expect(setupContent).toContain('.opencode/skills');
expect(setupContent).toContain('review/specialists');
expect(setupContent).toContain('qa/templates');
expect(setupContent).toContain('qa/references');
expect(setupContent).toContain('dx-hall-of-fame.md');
expect(setupContent).toContain('$opencode_gstack/lib');
});
test('create_agents_sidecar links runtime assets', () => {
// Sidecar must link bin with its shared lib modules, plus browse, review, qa
const fnStart = setupContent.indexOf('create_agents_sidecar()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('bin');
expect(fnBody).toContain('lib');
expect(fnBody).toContain('browse');
expect(fnBody).toContain('review');
expect(fnBody).toContain('qa');
});
test('create_codex_runtime_root exposes only runtime assets', () => {
const fnStart = setupContent.indexOf('create_codex_runtime_root()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', setupContent.indexOf('review/', fnStart)));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('gstack/SKILL.md');
expect(fnBody).toContain('$codex_gstack/lib');
expect(fnBody).toContain('browse/dist');
expect(fnBody).toContain('browse/bin');
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
// Review runtime assets (individual files, not the whole dir)
expect(fnBody).toContain('checklist.md');
expect(fnBody).toContain('design-checklist.md');
expect(fnBody).toContain('greptile-triage.md');
expect(fnBody).toContain('TODOS-format.md');
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$codex_gstack"');
});
test('create_factory_runtime_root links shared lib modules beside bin', () => {
const fnStart = setupContent.indexOf('create_factory_runtime_root()');
const fnEnd = setupContent.indexOf('create_opencode_runtime_root()', fnStart);
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('$factory_gstack/bin');
expect(fnBody).toContain('$factory_gstack/lib');
});
test('direct Codex installs are migrated out of ~/.codex/skills/gstack', () => {
expect(setupContent).toContain('migrate_direct_codex_install');
expect(setupContent).toContain('$HOME/.gstack/repos/gstack');
expect(setupContent).toContain('avoid duplicate skill discovery');
});
// --- Symlink prefix tests (PR #503) ---
test('link_claude_skill_dirs applies gstack- prefix by default', () => {
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('SKILL_PREFIX');
expect(fnBody).toContain('link_name="gstack-$skill_name"');
});
test('link_claude_skill_dirs preserves already-prefixed dirs', () => {
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
// gstack-* dirs should keep their name (e.g., gstack-upgrade stays gstack-upgrade)
expect(fnBody).toContain('gstack-*) link_name="$skill_name"');
});
test('setup supports --no-prefix flag', () => {
expect(setupContent).toContain('--no-prefix');
expect(setupContent).toContain('SKILL_PREFIX=0');
});
test('cleanup_old_claude_symlinks removes only gstack-pointing symlinks', () => {
expect(setupContent).toContain('cleanup_old_claude_symlinks');
const fnStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
// Should check readlink before removing
expect(fnBody).toContain('readlink');
expect(fnBody).toContain('gstack/*');
// Should skip already-prefixed dirs
expect(fnBody).toContain('gstack-*) continue');
});
test('cleanup runs before link when prefix is enabled', () => {
// In the Claude install section, cleanup should happen before linking
const claudeInstallSection = setupContent.slice(
setupContent.indexOf('INSTALL_CLAUDE'),
setupContent.lastIndexOf('link_claude_skill_dirs')
);
expect(claudeInstallSection).toContain('cleanup_old_claude_symlinks');
});
// --- Persistent config + interactive prompt tests ---
test('setup reads skill_prefix from config', () => {
expect(setupContent).toContain('get skill_prefix');
expect(setupContent).toContain('GSTACK_CONFIG');
});
test('setup supports --prefix flag', () => {
expect(setupContent).toContain('--prefix)');
expect(setupContent).toContain('SKILL_PREFIX=1; SKILL_PREFIX_FLAG=1');
});
test('--prefix and --no-prefix persist to config', () => {
expect(setupContent).toContain('set skill_prefix');
});
test('interactive prompt shows when no config', () => {
expect(setupContent).toContain('Short names');
expect(setupContent).toContain('Namespaced');
expect(setupContent).toContain('Choice [1/2]');
});
test('non-TTY defaults to flat names', () => {
// Should check if stdin is a TTY before prompting
expect(setupContent).toContain('-t 0');
});
test('cleanup_prefixed_claude_symlinks exists and uses readlink', () => {
expect(setupContent).toContain('cleanup_prefixed_claude_symlinks');
const fnStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('readlink');
expect(fnBody).toContain('gstack-$skill_name');
});
test('reverse cleanup runs before link when prefix is disabled', () => {
const claudeInstallSection = setupContent.slice(
setupContent.indexOf('INSTALL_CLAUDE'),
setupContent.lastIndexOf('link_claude_skill_dirs')
);
expect(claudeInstallSection).toContain('cleanup_prefixed_claude_symlinks');
});
test('welcome message references SKILL_PREFIX', () => {
// gstack-upgrade is always called gstack-upgrade (it's the actual dir name)
// but the welcome section should exist near the prefix logic
expect(setupContent).toContain('Run /gstack-upgrade anytime');
});
});
describe('discover-skills hidden directory filtering', () => {
test('discoverTemplates skips dot-prefixed directories', () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-discover-'));
try {
// Create a hidden dir with a template (should be excluded)
fs.mkdirSync(path.join(tmpDir, '.hidden'), { recursive: true });
fs.writeFileSync(path.join(tmpDir, '.hidden', 'SKILL.md.tmpl'), '---\nname: evil\n---\ntest');
// Create a visible dir with a template (should be included)
fs.mkdirSync(path.join(tmpDir, 'visible'), { recursive: true });
fs.writeFileSync(path.join(tmpDir, 'visible', 'SKILL.md.tmpl'), '---\nname: good\n---\ntest');
const { discoverTemplates } = require('../scripts/discover-skills');
const results = discoverTemplates(tmpDir);
const dirs = results.map((r: { tmpl: string }) => r.tmpl);
expect(dirs).toContain('visible/SKILL.md.tmpl');
expect(dirs).not.toContain('.hidden/SKILL.md.tmpl');
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
});
describe('telemetry', () => {
test('telemetry start block lives in gstack-skill-start; render notes the handoff keys', () => {
// The start-block bash moved into the script (Phase 1): it reads the
// config, mints the session identity, and echoes the STATUS keys.
expect(SKILL_START_SCRIPT).toContain('_TEL_START=$(date +%s)');
expect(SKILL_START_SCRIPT).toContain('_SESSION_ID=');
expect(SKILL_START_SCRIPT).toContain('echo "TELEMETRY:');
expect(SKILL_START_SCRIPT).toContain('echo "TEL_PROMPTED:');
expect(SKILL_START_SCRIPT).toMatch(/gstack-config" get telemetry/);
// The render must tell the model to carry SESSION_ID/TEL_START to skill end.
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('SESSION_ID');
expect(content).toContain('TEL_START');
});
test('telemetry opt-in prompt lives in gstack-skill-start (marker-gated emit)', () => {
// Token-reduction Phase 2: the one-time consent prompt left the renders
// (absence pinned by test/onboarding-moved-literals.test.ts); the script
// gates it on the marker files and emits it as a GSTACK_INSTRUCTION block
// with all three config-set outcomes and the ack INSIDE the block.
expect(SKILL_START_SCRIPT).toContain(
'if [ "$_TEL_PROMPTED" = "no" ] && [ "$_LAKE_SEEN" = "yes" ]; then',
);
expect(SKILL_START_SCRIPT).toContain('_emit_block telemetry-prompt');
expect(SKILL_START_SCRIPT).toContain('gstack-config set telemetry community');
expect(SKILL_START_SCRIPT).toContain('gstack-config set telemetry anonymous');
expect(SKILL_START_SCRIPT).toContain('gstack-config set telemetry off');
expect(SKILL_START_SCRIPT).toContain('touch "$_GH/.telemetry-prompted"');
});
test('generated SKILL.md contains telemetry epilogue (one gstack-skill-end call)', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('Telemetry (run last)');
expect(content).toContain('gstack-skill-end --skill "gstack" --outcome OUTCOME');
expect(content).toContain('--tel-start "TEL_START"');
expect(content).toContain('PLAN MODE EXCEPTION');
// The duration math + remote-log dispatch moved into gstack-skill-end.
expect(SKILL_END_SCRIPT).toContain('_TEL_END');
expect(SKILL_END_SCRIPT).toContain('_TEL_DUR');
expect(SKILL_END_SCRIPT).toContain('SKILL_NAME');
expect(SKILL_END_SCRIPT).toContain('OUTCOME');
expect(SKILL_END_SCRIPT).toContain('gstack-telemetry-log');
});
test('pending marker handling lives in the scripts', () => {
// gstack-skill-start finalizes stale markers; gstack-skill-end clears the
// session's own marker.
expect(SKILL_START_SCRIPT).toContain("-name '.pending-*'");
expect(SKILL_START_SCRIPT).toContain('_pending_finalize');
expect(SKILL_END_SCRIPT).toContain('.pending-$SESSION_ID');
});
test('telemetry blocks appear in all skill files that use PREAMBLE', () => {
const skills = ['qa', 'ship', 'review', 'plan-ceo-review', 'plan-eng-review', 'retro'];
for (const skill of skills) {
const skillPath = path.join(ROOT, skill, 'SKILL.md');
if (fs.existsSync(skillPath)) {
const content = fs.readFileSync(skillPath, 'utf-8');
expect(content).toContain('Telemetry (run last)');
expect(content).toContain(`gstack-skill-end --skill "${skill}"`);
expect(content).toContain('--tel-start "TEL_START"');
}
}
});
});
describe('community fixes wave', () => {
// Helper to get all generated SKILL.md files
function getAllSkillMds(): Array<{ name: string; content: string }> {
const results: Array<{ name: string; content: string }> = [];
const rootPath = path.join(ROOT, 'SKILL.md');
if (fs.existsSync(rootPath)) {
results.push({ name: 'root', content: fs.readFileSync(rootPath, 'utf-8') });
}
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
const skillPath = path.join(ROOT, entry.name, 'SKILL.md');
if (fs.existsSync(skillPath)) {
results.push({ name: entry.name, content: fs.readFileSync(skillPath, 'utf-8') });
}
}
return results;
}
// #594 — Discoverability: every SKILL.md.tmpl description contains "gstack"
test('every SKILL.md.tmpl description contains "gstack"', () => {
for (const skill of ALL_SKILLS) {
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
const content = fs.readFileSync(tmplPath, 'utf-8');
const desc = extractDescription(content);
expect(desc.toLowerCase()).toContain('gstack');
}
});
// #594 — Discoverability: first line of each description is under 120 chars
test('every SKILL.md.tmpl description first line is under 120 chars', () => {
for (const skill of ALL_SKILLS) {
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
const content = fs.readFileSync(tmplPath, 'utf-8');
const desc = extractDescription(content);
const firstLine = desc.split('\n')[0];
expect(firstLine.length).toBeLessThanOrEqual(120);
}
});
// #573 — Feature signals: ship/SKILL.md contains feature signal detection
test('ship/SKILL.md contains feature signal detection in Step 4', () => {
const content = readShipUnion();
expect(content.toLowerCase()).toContain('feature signal');
});
// #510 — Context warnings: no SKILL.md contains "running low on context"
test('no generated SKILL.md contains "running low on context"', () => {
const skills = getAllSkillMds();
for (const { name, content } of skills) {
expect(content).not.toContain('running low on context');
}
});
// #510 — Context warnings: plan-eng-review has explicit anti-warning
test('plan-eng-review/SKILL.md explicitly forbids preemptive context warnings', () => {
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
expect(content.toLowerCase()).toContain('do not preemptively warn');
});
// #474 — Safety Net: no SKILL.md uses find with -delete
test('no generated SKILL.md contains find with -delete flag', () => {
const skills = getAllSkillMds();
for (const { name, content } of skills) {
// Match find commands that use -delete (but not prose mentioning the word "delete")
const lines = content.split('\n');
for (const line of lines) {
if (line.includes('find ') && line.includes('-delete')) {
throw new Error(`${name}/SKILL.md contains find with -delete: ${line.trim()}`);
}
}
}
});
// #467 — Telemetry: preamble JSONL writes are gated by telemetry setting
test('preamble JSONL writes are inside telemetry conditional', () => {
const preamble = fs.readFileSync(path.join(ROOT, 'scripts/resolvers/preamble.ts'), 'utf-8');
// Find all skill-usage.jsonl write lines
const lines = preamble.split('\n');
for (let i = 0; i < lines.length; i++) {
if (lines[i].includes('skill-usage.jsonl') && lines[i].includes('>>')) {
// Look backwards for a telemetry conditional within 5 lines
let foundConditional = false;
for (let j = i - 1; j >= Math.max(0, i - 5); j--) {
if (lines[j].includes('_TEL') && lines[j].includes('off')) {
foundConditional = true;
break;
}
}
expect(foundConditional).toBe(true);
}
}
});
});
describe('codex commands must not use inline $(git rev-parse --show-toplevel) for cwd', () => {
// Regression test: inline $(git rev-parse --show-toplevel) in codex exec -C
// or codex review without cd evaluates in whatever cwd the background shell
// inherits, which may be a different project in Conductor workspaces.
// The fix is to resolve _REPO_ROOT eagerly at the top of each bash block.
// Scan all source files that could contain codex commands
// Use Bun.Glob to avoid ELOOP from .claude/skills/gstack symlink back to ROOT
const tmplGlob = new Bun.Glob('**/*.tmpl');
const sourceFiles = [
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
.filter(f => f.endsWith('.ts'))
.map(f => `scripts/resolvers/${f}`),
'scripts/gen-skill-docs.ts',
];
test('no codex exec command uses inline $(git rev-parse --show-toplevel) in -C flag', () => {
const violations: string[] = [];
for (const rel of sourceFiles) {
const abs = path.join(ROOT, rel);
if (!fs.existsSync(abs)) continue;
const content = fs.readFileSync(abs, 'utf-8');
const lines = content.split('\n');
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
violations.push(`${rel}:${i + 1}`);
}
}
}
expect(violations).toEqual([]);
});
test('no generated SKILL.md has codex exec with inline $(git rev-parse --show-toplevel) in -C flag', () => {
const violations: string[] = [];
const skillMdGlob = new Bun.Glob('**/SKILL.md');
const skillMdFiles = Array.from(skillMdGlob.scanSync({ cwd: ROOT, followSymlinks: false }));
for (const rel of skillMdFiles) {
const abs = path.join(ROOT, rel);
if (!fs.existsSync(abs)) continue;
const content = fs.readFileSync(abs, 'utf-8');
const lines = content.split('\n');
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
violations.push(`${rel}:${i + 1}`);
}
}
}
expect(violations).toEqual([]);
});
test('codex review commands must be preceded by cd "$_REPO_ROOT" (no -C support)', () => {
// codex review does not support -C, so the pattern must be:
// _REPO_ROOT=$(git rev-parse --show-toplevel) || { ... }
// cd "$_REPO_ROOT"
// codex review ...
// NOT: codex review ... with inline $(git rev-parse --show-toplevel)
const allFiles = [
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
...Array.from(new Bun.Glob('**/SKILL.md').scanSync({ cwd: ROOT, followSymlinks: false })),
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
.filter(f => f.endsWith('.ts'))
.map(f => `scripts/resolvers/${f}`),
'scripts/gen-skill-docs.ts',
];
const violations: string[] = [];
for (const rel of allFiles) {
const abs = path.join(ROOT, rel);
if (!fs.existsSync(abs)) continue;
const content = fs.readFileSync(abs, 'utf-8');
const lines = content.split('\n');
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
// Skip non-executable lines (markdown table cells, prose references)
if (line.includes('|') && line.includes('`/codex review`')) continue;
if (line.includes('`codex review`')) continue;
// Check for codex review with inline $(git rev-parse)
if (line.includes('codex review') && line.includes('$(git rev-parse --show-toplevel)')) {
violations.push(`${rel}:${i + 1} — inline git rev-parse in codex review`);
}
}
}
expect(violations).toEqual([]);
});
test('codex review commands take their scope from a flag, never from prompt text', () => {
// `codex review` scope comes ONLY from --base/--commit/--uncommitted. The
// positional [PROMPT] is mutually exclusive with all three (#1428, #1479),
// and a prompt-only `codex review` silently falls back to the *uncommitted
// working-tree* scope (`git status --short; git diff`) — so describing the
// diff range in prompt text produces a confident review of the wrong
// changes, with no error. Both halves are pinned here:
// (a) every `codex review` invocation carries a scope flag, and
// (b) no invocation puts a positional prompt in front of that flag.
//
// This does NOT apply to `codex exec`, which is agentic and really does run
// the git command it's told to — the adversarial pass legitimately scopes
// itself in prompt text.
const checkedFiles = [
'scripts/resolvers/review-dashboard.ts',
'scripts/resolvers/plan-gates.ts',
'scripts/resolvers/spec-review.ts',
'scripts/resolvers/outside-voice-steps.ts',
'scripts/resolvers/review-scope.ts',
'review/SKILL.md',
'ship/SKILL.md',
'codex/SKILL.md.tmpl',
'codex/SKILL.md',
// codex's scoped invocations moved into the carved review-mode section
// (T9) — keep sweeping both the .tmpl source and the generated section.
'codex/sections/review-mode.md.tmpl',
'codex/sections/review-mode.md',
];
const violations: string[] = [];
for (const rel of checkedFiles) {
// ship's codex/adversarial command moved into sections/adversarial.md (T9 carve).
const content = rel === 'ship/SKILL.md' ? readShipUnion() : fs.readFileSync(path.join(ROOT, rel), 'utf-8');
const lines = content.split('\n');
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
// Only inspect real shell invocations, not prose mentioning the command.
if (line.includes('`codex review`')) continue;
const match = line.match(/(?:^|[;&|]\s*|\s)codex\s+review\b(.*)$/);
if (!match) continue;
const rest = match[1];
const scopeFlag = /--base\b|--commit\b|--uncommitted\b/;
if (!scopeFlag.test(rest)) {
// A quoted prompt with no scope flag is the silent-wrong-scope bug.
if (/^\s*["'$]/.test(rest)) {
violations.push(`${rel}:${i + 1} — prompt-only codex review (falls back to working-tree scope)`);
}
continue;
}
const beforeFlag = rest.split(scopeFlag)[0].trim();
if (/^["'$]|^--\s*["']/.test(beforeFlag)) {
violations.push(`${rel}:${i + 1} — positional prompt passed alongside a scope flag`);
}
}
}
expect(violations).toEqual([]);
});
});
// ─── Learnings + Confidence Resolver Tests ─────────────────────
describe('LEARNINGS_SEARCH resolver', () => {
const SEARCH_SKILLS = ['review', 'ship', 'plan-eng-review', 'investigate', 'office-hours', 'plan-ceo-review'];
for (const skill of SEARCH_SKILLS) {
test(`${skill} generated SKILL.md contains learnings search`, () => {
const content = readSkillUnion(skill); // ship: moved to sections/plan-completion.md
expect(content).toContain('Prior Learnings');
expect(content).toContain('gstack-learnings-search');
});
}
test('learnings search includes cross-project config check', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('cross_project_learnings');
expect(content).toContain('--cross-project');
});
test('learnings search includes AskUserQuestion for first-time cross-project opt-in', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Enable cross-project learnings');
expect(content).toContain('project-scoped only');
});
test('Eng clarifies its full brief without changing any host options or other skills', async () => {
const { generateLearningsSearch } = await import('../scripts/resolvers/learnings');
const { ALL_HOST_CONFIGS } = await import('../hosts');
const { HOST_PATHS } = await import('../scripts/resolvers/types');
const fullBrief = 'Build a full decision brief from these facts and options using the preamble format, then ask and wait:';
for (const host of ALL_HOST_CONFIGS) {
const ctx = { skillName: 'plan-eng-review', tmplPath: 'plan-eng-review/SKILL.md.tmpl', host: host.name, paths: HOST_PATHS[host.name]! };
const eng = generateLearningsSearch(ctx);
const standard = generateLearningsSearch({ ...ctx, skillName: 'review' });
expect(generateLearningsSearch({ ...ctx, skillName: 'plan-ceo-review' })).toBe(standard);
if (host.learningsMode === 'basic') {
expect(eng).toBe(standard);
expect(eng).not.toContain('CROSS_PROJECT');
} else {
expect(eng).toContain(fullBrief);
expect(eng.replace(fullBrief, 'Use AskUserQuestion:')).toBe(standard);
expect(eng).toContain('A) Enable cross-project learnings (recommended)');
expect(eng).toContain('B) Keep learnings project-scoped only');
expect(eng).toContain('gstack-config set cross_project_learnings true');
expect(eng).toContain('gstack-config set cross_project_learnings false');
}
}
});
test('learnings search mentions prior learning applied display format', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Prior learning applied');
});
});
describe('LEARNINGS_LOG resolver', () => {
const LOG_SKILLS = ['review', 'retro', 'investigate'];
for (const skill of LOG_SKILLS) {
test(`${skill} generated SKILL.md contains learnings log`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('Capture Learnings');
expect(content).toContain('gstack-learnings-log');
});
}
test('learnings log documents all type values', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
for (const type of ['pattern', 'pitfall', 'preference', 'architecture', 'tool']) {
expect(content).toContain(type);
}
});
test('learnings log documents all source values', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
for (const source of ['observed', 'user-stated', 'inferred', 'cross-model']) {
expect(content).toContain(source);
}
});
test('learnings log includes files field for staleness detection', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('"files"');
expect(content).toContain('staleness detection');
});
});
describe('CONFIDENCE_CALIBRATION resolver', () => {
// CSO owns a distinct evidence rubric; the shared numerical confidence
// resolver would conflict with that contract and its private startup.
const CONFIDENCE_SKILLS = ['review', 'ship', 'plan-eng-review'];
for (const skill of CONFIDENCE_SKILLS) {
test(`${skill} generated SKILL.md contains confidence calibration`, () => {
const content = readSkillUnion(skill); // ship: moved to sections/review-army.md
expect(content).toContain('Confidence Calibration');
expect(content).toContain(skill === 'review' ? 'score every finding (1-10)' : 'confidence score');
});
}
test('confidence calibration includes scoring rubric with all tiers', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('9-10');
expect(content).toContain('7-8');
expect(content).toContain('5-6');
expect(content).toContain('3-4');
expect(content).toContain('1-2');
});
test('confidence calibration includes display rules', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Show normally');
expect(content).toContain('Suppress from main report');
});
test('confidence calibration includes finding format example', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('[CRITICAL] (confidence:');
expect(content).toContain('SQL injection');
});
test('confidence calibration includes calibration learning feedback loop', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
const flat = content.replace(/\s+/g, ' ');
expect(flat).toContain('Calibration learning');
expect(flat).toContain('log the corrected pattern as a learning');
});
test('skills without confidence calibration do NOT contain it', () => {
// office-hours and retro do NOT use confidence calibration
for (const skill of ['office-hours', 'retro']) {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('## Confidence Calibration');
}
});
});
describe('gen-skill-docs prefix warning (#620/#578)', () => {
const { execSync } = require('child_process');
test('warns about skill_prefix when config has prefix=true', () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
try {
// Create a fake ~/.gstack/config.yaml with skill_prefix: true
const fakeHome = tmpDir;
const fakeGstack = path.join(fakeHome, '.gstack');
fs.mkdirSync(fakeGstack, { recursive: true });
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: true\n');
// Render into an out-dir under the fixture (the warning fires on any
// non-dry-run generation) so the live tree is never rewritten.
const outDir = path.join(tmpDir, 'out');
const output = execSync(`bun run scripts/gen-skill-docs.ts --out-dir "${outDir}"`, {
cwd: ROOT,
env: { ...process.env, HOME: fakeHome },
encoding: 'utf-8',
timeout: 30000,
});
expect(output).toContain('skill_prefix is true');
expect(output).toContain('gstack-relink');
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
test('no warning when skill_prefix is false or absent', () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
try {
const fakeHome = tmpDir;
const fakeGstack = path.join(fakeHome, '.gstack');
fs.mkdirSync(fakeGstack, { recursive: true });
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: false\n');
const outDir = path.join(tmpDir, 'out');
const output = execSync(`bun run scripts/gen-skill-docs.ts --out-dir "${outDir}"`, {
cwd: ROOT,
env: { ...process.env, HOME: fakeHome },
encoding: 'utf-8',
timeout: 30000,
});
expect(output).not.toContain('skill_prefix is true');
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
});
describe('voice-triggers processing', () => {
const { extractVoiceTriggers, processVoiceTriggers } = require('../scripts/gen-skill-docs') as {
extractVoiceTriggers: (content: string) => string[];
processVoiceTriggers: (content: string) => string;
};
test('extractVoiceTriggers parses valid YAML list', () => {
const content = `---\nname: cso\ndescription: |\n Security audit.\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
const triggers = extractVoiceTriggers(content);
expect(triggers).toEqual(['see-so', 'security review']);
});
test('extractVoiceTriggers returns [] when no field present', () => {
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
expect(extractVoiceTriggers(content)).toEqual([]);
});
test('processVoiceTriggers appends voice triggers to description', () => {
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
const result = processVoiceTriggers(content);
expect(result).toContain('Voice triggers (speech-to-text aliases): "see-so", "security review".');
});
test('processVoiceTriggers strips voice-triggers field from output', () => {
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n---\nBody`;
const result = processVoiceTriggers(content);
expect(result).not.toContain('voice-triggers:');
});
test('processVoiceTriggers returns content unchanged when no voice-triggers', () => {
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
expect(processVoiceTriggers(content)).toBe(content);
});
test('generated CSO SKILL.md contains voice triggers in description', () => {
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
expect(content).toContain('"see-so"');
expect(content).toContain('Voice triggers (speech-to-text aliases):');
});
test('generated CSO SKILL.md does NOT contain raw voice-triggers field', () => {
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(0, fmEnd);
expect(frontmatter).not.toContain('voice-triggers:');
});
test('generated Claude CSO skill preauthorizes only the trusted launcher', () => {
const expected = [
'Bash(~/.claude/skills/gstack/bin/gstack-cso-launcher *)',
'Bash(~/.claude/skills/gstack/bin/gstack-cso-launcher.exe *)',
];
for (const file of ['cso/SKILL.md.tmpl', 'cso/SKILL.md']) {
const content = fs.readFileSync(path.join(ROOT, file), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = Bun.YAML.parse(content.slice(4, fmEnd)) as Record<string, unknown>;
expect(frontmatter['allowed-tools'], file).toEqual(expected);
}
});
test('generated CSO host variants retain challenge fallback and host-containment disclosure', () => {
const claude = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
const codex = fs.readFileSync(path.join(EXTERNAL_OUT, '.agents', 'skills', 'gstack-cso', 'SKILL.md'), 'utf-8');
for (const content of [claude, codex]) {
expect(content).toContain('sequential challenge; independent agent unavailable');
expectMentions(content, [['does not','containment','sandbox']], 'content');
expectMentions(content, [['do not','independent','reviewer']], 'content');
}
});
// Gen-time-only keys: interactive + benefits-from are read from the .tmpl by
// buildContext; the generated copy has no reader (the host reads name/
// description/allowed-tools/hooks; gbrain: is runtime-read and NOT stripped).
// Pin the strip so a stripFields refactor can't silently re-add the always-on
// frontmatter weight — mirrors the voice-triggers pins above.
test('generated SKILL.md strips gen-time-only keys the .tmpl still declares', () => {
const tmpl = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf-8');
const tmplFm = tmpl.slice(0, tmpl.indexOf('\n---', 4));
expect(tmplFm).toContain('interactive:');
expect(tmplFm).toContain('benefits-from:');
const generated = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
const genFm = generated.slice(0, generated.indexOf('\n---', 4));
expect(genFm).not.toContain('interactive:');
expect(genFm).not.toContain('benefits-from:');
// The runtime-read and host-read keys survive the strip.
const investigate = fs.readFileSync(path.join(ROOT, 'investigate', 'SKILL.md'), 'utf-8');
const invFm = investigate.slice(0, investigate.indexOf('\n---', 4));
expect(invFm).toContain('hooks:');
expect(invFm).toContain('gbrain:');
});
});
describe('CEO accepted requirement preservation', () => {
test('HOLD SCOPE includes implementation work needed to meet accepted requirements', () => {
const main = fs.readFileSync(path.join(ROOT, 'plan-ceo-review/SKILL.md'), 'utf8');
const hold = main.slice(main.indexOf('**For HOLD SCOPE**'), main.indexOf('**For SCOPE REDUCTION**'));
expectMentions(hold, [['acceptance criteria', 'in scope']], 'HOLD SCOPE');
});
test('loaded sections keep requirement conflicts unresolved until an authorized decision', () => {
const section = fs.readFileSync(path.join(ROOT, 'plan-ceo-review/sections/review-sections.md'), 'utf8');
const policy = section.slice(section.indexOf('**Preserve accepted requirements.**'), section.indexOf('### Section 1:')).replace(/\s+/g, ' ');
expect(policy).toContain('Report gaps and propose remedies, including omitted mechanisms in HOLD SCOPE');
expectMentions(policy, [['never','guarantee','violation']], 'policy');
expectMentions(policy, [['do not','documentation','requirements']], 'policy');
expect(policy).toContain('Changing a requirement needs explicit authority');
expectMentions(policy, [['cannot','auto-decide','constraints']], 'policy');
});
});
describe('plan-mode-info resolver (handshake-replacement)', () => {
const REVIEW_SKILLS = [
'plan-ceo-review',
'plan-eng-review',
'plan-design-review',
'plan-devex-review',
];
// Header for the vestigial handshake that was removed. If it ever reappears,
// someone accidentally re-introduced the resolver.
const HANDSHAKE_MARKER = '## Plan Mode Handshake';
// Header for the new plan-mode-info section (previously lived at the tail
// of completion-status.ts; now hoisted to position 1 of the preamble).
const PLAN_MODE_INFO_MARKER = '## Skill Invocation During Plan Mode';
test('vestigial handshake is absent from all generated Claude SKILL.md files', () => {
// Scan every generated SKILL.md under ROOT (top-level directory per skill).
// Using fs.readdirSync + filter instead of a glob so we catch any skill
// that gets added later without updating this list.
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
let checked = 0;
for (const entry of entries) {
if (!entry.isDirectory()) continue;
const skillMd = path.join(ROOT, entry.name, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
expect(content, `handshake marker in ${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
checked++;
}
expect(checked).toBeGreaterThan(0);
});
test('vestigial handshake is absent from non-Claude host outputs', () => {
// Non-Claude hosts render to hostSubdirs (.agents/, .openclaw/, etc). The
// plan-mode-info resolver has no host-scoping — all hosts get the new
// section, none get the old handshake. Scan every candidate host tree in
// the module-level out-dir render (--host all), which is always present —
// so the check can no longer silently degrade to a console warning.
const hostDirs = ['.agents', '.openclaw', '.opencode', '.factory', '.hermes', '.kiro', '.cursor', '.slate'];
let checked = 0;
for (const host of hostDirs) {
const skillsRoot = path.join(EXTERNAL_OUT, host, 'skills');
if (!fs.existsSync(skillsRoot)) continue;
const entries = fs.readdirSync(skillsRoot, { withFileTypes: true });
for (const entry of entries) {
if (!entry.isDirectory()) continue;
const skillMd = path.join(skillsRoot, entry.name, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
expect(content, `handshake marker in ${host}/skills/${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
checked++;
}
}
expect(checked).toBeGreaterThan(0);
});
test.each(REVIEW_SKILLS)(
'%s/SKILL.md contains the new plan-mode-info section near the top',
(skill) => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
const idx = content.indexOf(PLAN_MODE_INFO_MARKER);
expect(idx).toBeGreaterThan(0);
// Position 1 in preamble composition = within the first ~300 lines.
// Roughly translates to first ~15KB of text.
expect(idx).toBeLessThan(15_000);
},
);
test('plan-mode-info is wired BEFORE generateUpgradeCheck in preamble', () => {
// Token-reduction Phase 2: generateUpgradeCheck's render output is now
// ONLY the steady-state PROACTIVE-false + SKILL_PREFIX rules (the
// UPGRADE_AVAILABLE prose emits from bin/gstack-skill-start at runtime),
// so those rules are the resolver's order marker.
const content = fs.readFileSync(
path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
'utf-8',
);
const planModeIdx = content.indexOf(PLAN_MODE_INFO_MARKER);
const upgradeIdx = content.search(/If `PROACTIVE` is `"?false"?`/i);
expect(planModeIdx).toBeGreaterThan(0);
expect(upgradeIdx).toBeGreaterThan(0);
expect(planModeIdx).toBeLessThan(upgradeIdx);
});
test('0D authority and fresh-approval paths precede mode selection', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8').replace(/\s+/g, ' ');
const approachIdx = content.indexOf('### 0D.');
const constructIdx = content.indexOf('Build one `currentDecision`', approachIdx);
const saveIdx = content.indexOf('**Pre-question checkpoint:**', constructIdx);
const presentIdx = content.indexOf('Ask one row per call', saveIdx);
const stopOffset = content.slice(presentIdx).search(/stop for the actual answer/i);
const stopIdx = stopOffset < 0 ? -1 : presentIdx + stopOffset;
const modeIdx = content.indexOf('### 0E. Mode Selection');
const preludeIdx = content.indexOf('### 0F');
const positions = [approachIdx, constructIdx, saveIdx, presentIdx, stopIdx, modeIdx, preludeIdx];
expect(positions.every(position => position > 0)).toBe(true);
expect(positions).toEqual([...positions].sort((a, b) => a - b));
const approach = content.slice(approachIdx, modeIdx);
expect(approach).toMatch(/Reuse (?:an )?exact approvals?/);
expect(content).toMatch(/(?:the actual instruction\/answer|Cite actual instructions\/answers) and exact scope/);
const reopenRule = approach.match(/Reuse exact approvals\. Reopen only for contradictions, changed assumptions or user instructions, never speculation or reviewer agreement/)?.[0];
expect(reopenRule).toBeDefined();
expect(reopenRule!).toMatch(/reopen only|requires reopening it/i);
expect(content.indexOf(reopenRule!)).toBeGreaterThan(approachIdx);
expect(content.indexOf(reopenRule!)).toBeLessThan(presentIdx);
const gate = content.slice(stopIdx, modeIdx);
const startup = content.slice(content.indexOf('### 0C.'), approachIdx);
expect(startup).toContain('Before 0E, call 0D for unresolved approaches');
expect(startup).toContain('A) current/requested plan, B) smallest scoped alternative');
expectMentions(startup, [['no','required','choices']], 'startup');
expectMentions(approach, [['not','selection','returns']], 'approach');
expectTokens(approach, ['**Mode change**'], 'approach');
const modeChange = content.slice(content.indexOf('**Mode change:**'), preludeIdx);
expectMentions(modeChange, [['ask','four-mode','answered']], 'modeChange');
expectMentions(gate, [['do not','calling','return']], 'gate');
expect(gate).toContain('even for a lone option');
expect(approach).toContain('A recommendation is not approval');
expectMentions(approach, [['without','recomposing','unchanged']], 'approach');
expect(gate).toMatch(/(?:Record its reference|Save the answer reference) and scope in Exact approval and scope/);
expect(gate).toMatch(/update Status,? and amend only (?:what it authorizes|authorized work)/);
expectMentions(gate, [['before','storage','another']], 'gate');
});
});
// Every skill that renders {{PLAN_FILE_REVIEW_REPORT}} / {{EXIT_PLAN_MODE_GATE}},
// on every host: the union covers each skill-specific branch of both resolvers.
function renderPlanReportAndGate(): string {
const { generatePlanFileReviewReport } = require('../scripts/resolvers/review-dashboard');
const { generateExitPlanModeGate } = require('../scripts/resolvers/plan-gates');
const { HOST_PATHS } = require('../scripts/resolvers/types');
const skills = ['codex', 'devex-review', 'plan-ceo-review', 'plan-design-review', 'plan-devex-review', 'plan-eng-review'];
return ALL_HOST_CONFIGS.flatMap((host) => skills.map((skillName) => {
const ctx = { skillName, tmplPath: `${skillName}/SKILL.md.tmpl`, host: host.name, paths: HOST_PATHS[host.name] };
return `${generatePlanFileReviewReport(ctx)}\n${generateExitPlanModeGate(ctx)}`;
})).join('\n');
}
// GSTACK REVIEW REPORT report-at-bottom contract — verifies the prompt-text
// fix in scripts/resolvers/review-dashboard.ts (the load-bearing change for the
// "report not at bottom of plan in plan mode" bug). The bug is in the
// prompt's contradictory write-flow instructions, not in observable
// runtime behavior we can cheaply gate in CI. Verifying the prompt text
// directly is the deterministic equivalent of the regression test the
// PTY harness can't reliably drive (autoplan needs auto-progression of
// AskUserQuestions to reach the report-write step, which the harness
// doesn't support today).
describe('GSTACK REVIEW REPORT delete-then-append flow', () => {
const PLAN_REVIEW_SKILLS = [
'plan-ceo-review',
'plan-design-review',
'plan-devex-review',
'plan-eng-review',
];
for (const skill of PLAN_REVIEW_SKILLS) {
test(`${skill}/SKILL.md prescribes delete-then-append, not in-place replace`, () => {
// Carved skills (v2 plan Phase B) relocate the review-report prose into
// sections/*.md; readSkillUnion follows the content wherever the carve put it.
const content = readSkillUnion(skill);
// The new (correct) instruction must be present.
expect(content).toContain('delete-then-append flow');
// The old contradictory bullets must be gone. The signature phrase
// from the buggy prompt was 'replace it entirely using the Edit tool'
// which is what allowed mid-file reports to stay mid-file.
expect(content).not.toContain('replace it** entirely using the Edit tool');
expect(content).not.toContain('If it was found mid-file, move it');
});
}
test('plan-file review report resolver renders the rewritten flow', () => {
const src = renderPlanReportAndGate();
expect(src).toContain('delete-then-append flow');
// Old contradictory bullets are gone from the source resolver.
expect(src).not.toContain('replace it** entirely using the Edit tool');
expect(src).not.toContain('If it was found mid-file, move it');
});
});
describe('LEARNINGS_SEARCH resolver: query parameter', () => {
// Lazy-load resolver and types after describe block to keep test file self-contained.
const { generateLearningsSearch } = require('../scripts/resolvers/learnings');
const { HOST_PATHS } = require('../scripts/resolvers/types');
const claudeCtx = {
skillName: 'test',
tmplPath: 'test/SKILL.md.tmpl',
host: 'claude',
paths: HOST_PATHS.claude,
};
const codexCtx = { ...claudeCtx, host: 'codex', paths: HOST_PATHS.codex };
test('no args → bash does not contain --query (backwards-compat)', () => {
const out = generateLearningsSearch(claudeCtx);
expect(out).not.toContain('--query');
});
test('claude host + query=foo bar → both cross-project and project-scoped branches contain --query', () => {
const out = generateLearningsSearch(claudeCtx, ['query=foo bar']);
// Both branches of the if/else must carry the flag.
const lines = out.split('\n').filter(l => l.includes('gstack-learnings-search'));
expect(lines.length).toBeGreaterThanOrEqual(2);
for (const line of lines) {
expect(line).toContain('--query "foo bar"');
}
});
test('codex host + query=foo bar → codex bash variant contains --query', () => {
const out = generateLearningsSearch(codexCtx, ['query=foo bar']);
expect(out).toContain('--query "foo bar"');
expect(out).toContain('$GSTACK_BIN/gstack-learnings-search');
});
test('empty value query= → bash does not contain --query (locked semantics: falls through)', () => {
const claudeOut = generateLearningsSearch(claudeCtx, ['query=']);
expect(claudeOut).not.toContain('--query');
const codexOut = generateLearningsSearch(codexCtx, ['query=']);
expect(codexOut).not.toContain('--query');
});
test('shell-injection chars in query= → throws at gen-time (defense in depth)', () => {
for (const bad of ['$(whoami)', '`cmd`', 'a;b', 'a&b', 'a"b', 'a\\b', 'foo$x']) {
expect(() => generateLearningsSearch(claudeCtx, [`query=${bad}`])).toThrow(/alphanumeric/);
}
});
});
describe('EXIT PLAN MODE GATE placement', () => {
// Fresh skill list — do NOT reuse REVIEW_SKILLS upstream (3 entries, missing plan-devex).
const planSkills = ['plan-eng-review', 'plan-ceo-review', 'plan-design-review', 'plan-devex-review'];
// Strip fenced code blocks before matching headings — PLAN_FILE_REVIEW_REPORT
// already contains `## GSTACK REVIEW REPORT` inside a markdown example fence,
// and the gate text itself shows `## GSTACK REVIEW REPORT` inside a fence too.
const stripFences = (md: string) => md.replace(/```[\s\S]*?```/g, '');
test('gate follows review work, with only the declared cache hook after CEO and Eng verification', () => {
for (const skill of planSkills) {
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
const stripped = stripFences(md);
const headings = [...stripped.matchAll(/^## .+$/gm)].map(m => m[0]);
const lastH2 = headings.at(-1);
if (['plan-ceo-review', 'plan-eng-review'].includes(skill)) {
const gate = headings.indexOf('## EXIT PLAN MODE GATE (BLOCKING)');
expect(gate).toBeGreaterThan(headings.indexOf('## Section self-check (before you finish)'));
expect(headings.slice(gate)).toEqual(['## EXIT PLAN MODE GATE (BLOCKING)', '## Brain Cache Background Refresh']);
const tail = md.slice(md.indexOf('## EXIT PLAN MODE GATE (BLOCKING)'));
const telemetry = tail.indexOf('**Telemetry (run last)**');
expect(telemetry).toBeGreaterThan(0);
const cache = tail.indexOf('## Brain Cache Background Refresh');
if (skill === 'plan-ceo-review') expect(telemetry).toBeGreaterThan(cache);
else expect(telemetry).toBeLessThan(cache);
expect(tail).toContain(skill === 'plan-ceo-review' ? 'Passed with a verified persisted report' : 'After the gate passes');
const finalHandoff = tail.slice(tail.indexOf('## Brain Cache Background Refresh'));
expect(finalHandoff).toContain(skill === 'plan-ceo-review' ? 'Call ExitPlanMode where required or return to the caller' : 'After success telemetry and cache dispatch, call ExitPlanMode');
if (skill === 'plan-ceo-review') {
expect(finalHandoff.indexOf('Call ExitPlanMode')).toBeGreaterThan(finalHandoff.indexOf('**Telemetry (run last)**'));
expect(finalHandoff).toContain('handoff starts a separate workflow');
}
if (skill === 'plan-eng-review') {
expectMentions(tail.replace(/\s+/g, ' '), [['cannot','persistence','unrecovered']], 'tail.replace(/\s+/g, )');
expectMentions(finalHandoff, [['only', 'host', 'plan']], 'finalHandoff');
expectMentions(finalHandoff, [['do not','conversation','exitplanmode']], 'finalHandoff');
} else {
expect(tail.replace(/\s+/g, ' ')).toContain('**Pass with log-only gaps:**');
expectTokens(tail.replace(/\s+/g, ' '), ['**not persisted**'], 'tail.replace(/\s+/g, )');
expectMentions(tail.replace(/\s+/g, ' '), [['do not', 'unwritten', 'artifacts']], 'tail.replace(/\s+/g, )');
expectMentions(tail.replace(/\s+/g, ' '), [['without','exitplanmode','telemetry']], 'tail.replace(/\s+/g, )');
}
} else {
expect(lastH2, `${skill}/SKILL.md last ## heading (fences stripped)`).toBe('## EXIT PLAN MODE GATE (BLOCKING)');
}
if (skill === 'plan-eng-review') {
const gate = extractMarkdownSection(md, '## EXIT PLAN MODE GATE (BLOCKING)').replace(/\s+/g, ' ');
expectTokens(gate, ['**Blocked outcome**'], 'gate');
expectMentions(gate, [['without', 'exitplanmode', 'telemetry']], 'gate');
} else if (skill === 'plan-ceo-review') {
expect(md, `${skill}/SKILL.md gate body`).toContain('Failed checks use **Gate outcome: Blocked**');
} else {
expect(extractMarkdownSection(md, '## EXIT PLAN MODE GATE (BLOCKING)'), `${skill}/SKILL.md gate body`).toMatch(/do not call ExitPlanMode/i);
}
}
});
test('codex/SKILL.md contains gate (mid-file per D5; Step 2B/2C follow)', () => {
const codex = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
expect(codex).toContain('## EXIT PLAN MODE GATE (BLOCKING)');
expect(extractMarkdownSection(codex, '## EXIT PLAN MODE GATE (BLOCKING)')).toMatch(/do not call ExitPlanMode/i);
});
});
describe('scope-gate exceptions drift-guard', () => {
// The plan-mode auto-select-B exceptions block is hand-duplicated in the
// plan-eng-review and plan-design-review templates (matching the gate
// around it, which predates this block). The two copies must stay
// identical in their target-selection policy. Eng resolves that policy
// before announcing its target and has a separate bootstrap transport;
// Design retains its own startup sequence and page vocabulary.
// A future edit to one copy that silently misses the other fails here
// instead of drifting. The real fix (shared {{SCOPE_GATE}} resolver) is a
// filed TODO — this guard is the stopgap that makes the duplication safe.
const START_MARKER = '**Exceptions — check in this order, BEFORE asking:**';
const END_MARKER = 'When no exception above applied:';
function extractExceptionsBlock(skill: string): string {
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
const start = md.indexOf(START_MARKER);
expect(start, `${skill}/SKILL.md: exceptions block start marker present`).toBeGreaterThan(-1);
const end = md.indexOf(END_MARKER, start);
expect(end, `${skill}/SKILL.md: exceptions block end marker present`).toBeGreaterThan(start);
return md.slice(start, end + END_MARKER.length);
}
const normalizeTargetPolicy = (block: string) => block
.split('\n').filter(line => /^[12]\. /.test(line)).join('\n')
.replace(/ Announce (?:it|an auto-selected plan) in one line so the user can interrupt: "Scope gate: plan mode — auto-selected B \(reviewing <target>\)\."/g, '')
.replace(/ Then run the pre-review audit, .*? against that plan\./, '')
.replace('a path, a page, a doc they pasted,', 'a path, a doc they pasted,');
test('eng and design retain the same target-selection policy across distinct startup sequences', () => {
const engRaw = extractExceptionsBlock('plan-eng-review');
const eng = normalizeTargetPolicy(engRaw);
const design = normalizeTargetPolicy(extractExceptionsBlock('plan-design-review'));
expect(eng).toBe(design);
expect(eng).toContain('their choice wins — use it instead');
expect(eng).toContain('still ambiguous — ask');
expectMentions(eng, [['ask','default','doubt']], 'eng');
expect(engRaw.indexOf('their choice wins')).toBeLessThan(engRaw.indexOf('Announce an auto-selected plan'));
const engMd = fs.readFileSync(path.join(ROOT, 'plan-eng-review/SKILL.md'), 'utf8');
const transport = engMd.indexOf('Choose listed, enabled MCP AskUserQuestion, otherwise listed native');
expect(transport).toBeGreaterThan(engMd.indexOf(END_MARKER));
expect(transport).toBeLessThan(engMd.indexOf('## Preamble (after scope gate)'));
});
test('exceptions block carries the announcement string the PTY detectors pin', () => {
for (const skill of ['plan-eng-review', 'plan-design-review']) {
const block = extractExceptionsBlock(skill);
expect(block, `${skill}: verbatim announcement`).toContain(
'Scope gate: plan mode — auto-selected B (reviewing <target>).',
);
}
});
test('gate menu carries the question strings the PTY question detector pins', () => {
// isScopeGateQuestionVisible (claude-pty-runner.ts) anchors on the
// question text + option A's body. If the menu is reworded without
// updating the detector, the paid smokes' must-stay-false assertions go
// vacuous — this free pin fails first.
for (const skill of ['plan-eng-review', 'plan-design-review']) {
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(md, `${skill}: gate question text`).toContain('What should I review?');
expect(md, `${skill}: option A body text`).toContain('The current branch diff');
}
});
});
describe('GSTACK REVIEW REPORT mandatory unresolved-decisions status', () => {
// Report text rides in PLAN_FILE_REVIEW_REPORT → every report consumer gets it.
// devex-review is a report consumer but NOT a gate consumer, so the two target
// sets differ (CP5/CX5). Regression guard: a future token-cut that drops the
// unresolved-status line again fails here. See plan-flag-unresolved-issues.
const REPORT_CONSUMERS = [
'plan-ceo-review',
'plan-eng-review',
'plan-design-review',
'plan-devex-review',
'codex',
'devex-review',
];
// Gate text rides in EXIT_PLAN_MODE_GATE (lives in SKILL.md, not sections).
const GATE_SKILLS = [
'plan-ceo-review',
'plan-eng-review',
'plan-design-review',
'plan-devex-review',
'codex',
];
for (const skill of REPORT_CONSUMERS) {
test(`${skill}: report mandates the unresolved-decisions status as final content`, () => {
const content = readSkillUnion(skill);
expect(content).toContain('NO UNRESOLVED DECISIONS');
// \s+ tolerates prose line-wraps within "final non-whitespace line".
expect(content).toMatch(/final\s+non-whitespace\s+line/i);
});
}
for (const skill of GATE_SKILLS) {
test(`${skill}: exit gate blocks unless the unresolved status is the final line`, () => {
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
// Gate check #4 — present, sentinel named, and explicitly blocking (no escape).
expect(md).toContain('NO UNRESOLVED DECISIONS');
if (['plan-ceo-review', 'plan-eng-review'].includes(skill)) {
const gate = md.split('## EXIT PLAN MODE GATE (BLOCKING)')[1]!.replace(/\s+/g, ' ');
expect(gate).toMatch(/final\s+non-whitespace\s+line is the exact unbolded `NO UNRESOLVED DECISIONS`/i);
expect(gate).toContain('`**UNRESOLVED DECISIONS:**`');
expect(gate).toMatch(/missing\s+status[^.]*fails/i);
expect(gate).toContain(skill === 'plan-eng-review'
? 'If any check fails, follow **Blocked outcome** without success telemetry or ExitPlanMode'
: 'Failed checks use **Gate outcome: Blocked**');
} else {
const gate = extractMarkdownSection(md, '## EXIT PLAN MODE GATE (BLOCKING)');
expect(gate).toMatch(/final\s+non-whitespace\s+line/i);
expect(gate).toContain('`**UNRESOLVED DECISIONS:**`');
expect(gate).toMatch(/missing\s+status[^.]*fails/i);
}
});
}
test('plan-file review report and exit gate resolvers render the mandatory block + blocking gate', () => {
const src = renderPlanReportAndGate();
expect(src).toContain('NO UNRESOLVED DECISIONS');
expect(src).toMatch(/avoids double-counting/i);
expect(src).toMatch(/final\s+non-whitespace\s+line/i);
expect(src).toMatch(/missing\s+status[^.]*fails/i);
// The old soft wording must be gone from the gate.
expect(src).not.toContain('absorbs CODEX / CROSS-MODEL / UNRESOLVED lines if applicable');
});
});
// ─── {{PREAMBLE}} requires an explicit preamble-tier ────────
describe('PREAMBLE resolution requires declared preamble-tier', () => {
test('resolving {{PREAMBLE}} without preamble-tier throws with the template path', async () => {
const { generatePreamble } = await import('../scripts/resolvers/preamble');
const { HOST_PATHS } = await import('../scripts/resolvers/types');
const ctx = {
skillName: 'tierless-skill',
tmplPath: 'tierless-skill/SKILL.md.tmpl',
host: 'claude' as const,
paths: HOST_PATHS.claude,
// preambleTier deliberately absent — the generator must refuse to default it.
};
expect(() => generatePreamble(ctx)).toThrow(/tierless-skill\/SKILL\.md\.tmpl/);
expect(() => generatePreamble(ctx)).toThrow(/preamble-tier/);
});
test('every template that resolves {{PREAMBLE}} declares preamble-tier in frontmatter', () => {
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
const offenders: string[] = [];
const checkTmpl = (tmplPath: string) => {
const tmpl = fs.readFileSync(tmplPath, 'utf-8');
if (tmpl.includes('{{PREAMBLE}}') && !/^preamble-tier:\s*\d+$/m.test(tmpl)) {
offenders.push(path.relative(ROOT, tmplPath));
}
};
checkTmpl(path.join(ROOT, 'SKILL.md.tmpl'));
for (const e of entries) {
if (!e.isDirectory() || e.name.startsWith('.') || e.name === 'node_modules') continue;
const tmplPath = path.join(ROOT, e.name, 'SKILL.md.tmpl');
if (fs.existsSync(tmplPath)) checkTmpl(tmplPath);
}
expect(offenders).toEqual([]);
});
});
// ---------------------------------------------------------------------------
// #2499: gbrain MCP detection must read BOTH ~/.claude.json scopes.
// Claude Code registers MCP servers at user scope (.mcpServers) and project
// scope (.projects["/abs/path"].mcpServers — what `claude mcp add` without
// --scope user writes). The rendered brain-sync block previously read only
// user scope, so a correctly configured project-scoped brain was invisible.
// ---------------------------------------------------------------------------
describe('brain-sync block reads project-scoped MCP registrations (#2499)', () => {
// Phase 1: the artifacts-sync bash (including the MCP-scope jq probe) moved
// from the rendered SKILL.md into bin/gstack-skill-start. Pin the LIVE
// script bytes — same assertions, new home. The render carries only the
// ARTIFACTS_SYNC interpretation prose.
const rendered = fs.readFileSync(path.join(ROOT, 'bin', 'gstack-skill-start'), 'utf-8');
test('rendered _GBRAIN_MCP_ENTRY jq resolves project scope with nearest-ancestor cwd match', () => {
const line = rendered.split('\n').find((l) => l.includes('_GBRAIN_MCP_ENTRY=$('));
expect(line).toBeDefined();
// Project-scope read present, driven by $PWD.
expect(line!).toContain('--arg cwd "$PWD"');
expect(line!).toContain('.projects');
// User scope still resolved first.
expect(line!).toContain('.mcpServers.gbrain');
// The old user-scope-only filter is gone from the rendered output.
expect(rendered).not.toContain('.mcpServers.gbrain.type // .mcpServers.gbrain.transport');
expect(rendered).not.toContain(".mcpServers.gbrain.url // empty");
});
test('rendered _GBRAIN_MCP_TYPE and _GBRAIN_HOST extract from the resolved entry', () => {
const typeLine = rendered.split('\n').find((l) => l.includes('_GBRAIN_MCP_TYPE=$('));
const hostLine = rendered.split('\n').find((l) => l.includes('_GBRAIN_HOST=$('));
expect(typeLine).toBeDefined();
expect(hostLine).toBeDefined();
expect(typeLine!).toContain('_GBRAIN_MCP_ENTRY');
expect(hostLine!).toContain('_GBRAIN_MCP_ENTRY');
});
test('rendered jq lines FUNCTION: project-scoped registration resolves for a cwd inside the project', () => {
// Execute the exact rendered bytes, not a re-derivation: extract the
// _GBRAIN_MCP_ENTRY + _GBRAIN_MCP_TYPE lines from the generated SKILL.md
// and run them in bash against a fixture ~/.claude.json that carries ONLY
// a project-scoped gbrain registration.
const lines = rendered.split('\n');
const entryLine = lines.find((l) => l.includes('_GBRAIN_MCP_ENTRY=$('));
const typeLine = lines.find((l) => l.includes('_GBRAIN_MCP_TYPE=$('));
expect(entryLine).toBeDefined();
expect(typeLine).toBeDefined();
const tmpHome = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-2499-home-'));
const projectDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-2499-proj-'));
const nestedCwd = path.join(projectDir, 'src', 'deep');
fs.mkdirSync(nestedCwd, { recursive: true });
try {
fs.writeFileSync(
path.join(tmpHome, '.claude.json'),
JSON.stringify({
projects: {
[projectDir]: {
mcpServers: { gbrain: { type: 'http', url: 'https://brain.example.com/mcp' } },
},
},
}),
);
const script = `cd "$1" || exit 1\n${entryLine!.trim()}\n${typeLine!.trim()}\necho "RESOLVED:$_GBRAIN_MCP_TYPE"`;
const r = spawnSync('bash', ['-c', script, 'bash', nestedCwd], {
encoding: 'utf-8',
env: { ...process.env, HOME: tmpHome },
timeout: 10_000,
});
expect(r.stdout).toContain('RESOLVED:http');
// Discriminator: a cwd OUTSIDE the project must NOT resolve it.
const outside = spawnSync('bash', ['-c', script, 'bash', os.tmpdir()], {
encoding: 'utf-8',
env: { ...process.env, HOME: tmpHome },
timeout: 10_000,
});
expect(outside.stdout).toContain('RESOLVED:\n');
} finally {
fs.rmSync(tmpHome, { recursive: true, force: true });
fs.rmSync(projectDir, { recursive: true, force: true });
}
});
});
// #2896: Claude Code replaces `$<digit>` tokens in a SKILL.md body with the
// invocation's argument words before the model reads it (2.1.271+:
// /\$(\d+)(?!\w)/g), so shell/awk fields arrive rewritten yet still parse.
// Write awk fields as $(N) and shell positionals as ${N}. Section files are
// read later with Read and are not substituted.
describe('generated Claude skill bodies carry no $N token Claude Code rewrites (#2896)', () => {
const CLAUDE_ARG_TOKEN = /\$\d+(?!\w)/g;
test('tripwire', () => {
const offenders: string[] = [];
let checked = 0;
for (const file of ['SKILL.md', ...fs.readdirSync(ROOT).map(d => path.join(d, 'SKILL.md'))]) {
const full = path.join(ROOT, file);
if (!fs.existsSync(full)) continue;
const lines = fs.readFileSync(full, 'utf-8').split('\n');
const bodyStart = lines[0] === '---' ? lines.indexOf('---', 1) + 1 : 0;
checked++;
lines.slice(bodyStart).forEach((line, i) => {
for (const m of line.matchAll(CLAUDE_ARG_TOKEN)) offenders.push(`${file}:${bodyStart + i + 1}: ${m[0]} in ${line.trim().slice(0, 120)}`);
});
}
expect(checked).toBeGreaterThan(40);
expect(offenders).toEqual([]);
});
test('the tripwire pattern matches what Claude Code substitutes', () => {
expect('awk \'{print $1}\' "$0" ~$0.05 $12'.match(CLAUDE_ARG_TOKEN)).toEqual(['$1', '$0', '$0', '$12']);
expect('awk \'{print $(1)}\' "${1}" $ARGUMENTS $1a $_x'.match(CLAUDE_ARG_TOKEN)).toBeNull();
});
});