mirror of
https://github.com/garrytan/gstack.git
synced 2026-08-23 22:42:31 +02:00
* fix(plan-tune): reject never-ask on one-way ids at --write --check already ignored those prefs; --write still stored them and --stats counted them as a working NEVER_ASK. Refuse the write and count leftover on-disk prefs as INERT_ONE_WAY. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix: gstack-config get returns "" with exit 0 for keys that have no default Skill preambles read configuration with VAR=$(gstack-config get <key> 2>/dev/null || echo "<default>") and that fallback only fires on a non-zero exit. lookup_default ended in a catch-all that echoed "" and returned 0, so for any key missing from the table VAR came back empty and the default written right there in the preamble was unreachable. The skill then branched on a value it never specified: "skip entirely if QUESTION_TUNING is false", reached with QUESTION_TUNING="". Four keys that skills actually read had no entry and took that path: question_tuning -> callers assume "false" repo_mode -> callers assume "unknown" team_mode -> callers assume "false" transcript_ingest_mode -> callers assume "off" Each default above is the value the call sites already substitute in their own `|| echo` fallback, so this only makes reachable what was already intended. The catch-all now returns non-zero. That is deliberately scoped to the unknown-key arm alone: keys whose default is intentionally empty still exit 0, because "" is their real answer and their callers depend on it -- cross_project_learnings ("unset triggers the first-time prompt"), redact_repo_visibility ("empty falls through to gh/glab detection"), salience_allowlist, user_slug_at_*. Making every empty answer an error would have broken those. test/gstack-config-defaults.test.ts pins the class rather than the four instances: it parses the case arms and asserts every `gstack-config get <key>` site in the tree is covered, so adding a read without a default fails CI. It also pins the exit-code contract in both directions. Verified failing against the pre-fix script, where it names exactly those four keys. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(redact): a typo'd subcommand no longer exits 0 having done nothing main() recognised exactly two subcommands and let everything else fall through to the stdin scan. On empty stdin that prints "(no findings)" and exits 0, so: $ gstack-redact install-prepush-hooks # plural typo gstack-redact scan — repo UNKNOWN (no findings) $ echo $? 0 No hook was installed, and the operator has every reason to believe the credential guard is armed. A guard that silently no-ops must never exit 0. Two smaller faults in the same dispatch, both of which lead people here: - There was no --help handler, so `gstack-redact --help` fell through to the scanner. Piping a credential to it scanned the secret and exited 3. - With no piped input and no --from-file, readInput() blocks on readSync(fd 0) until an EOF that an interactive terminal never sends. That prints nothing at all, so it reads as a hang rather than as "this is a filter, feed it". Now: --help/-h/help prints usage and exits 0; an unrecognised positional prints the offender and exits 1; a TTY with nothing piped in prints usage instead of blocking. "scan" stays accepted, because the human output header reads "gstack-redact scan — repo …" and that is what people type. Usage errors exit 1, deliberately not 2 or 3. Those mean MEDIUM and HIGH findings and callers gate dispatch on them, so a usage error exiting 2 would be read as "medium findings — prompt the user". A test pins that. Tests: 4 written failing first, then fixed. Full suite 7,722 pass / 0 fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(browse): one ambiguous ref no longer kills the whole annotated screenshot `snapshot -a` exits 1 with "Selector matched multiple elements" on most real pages, so /qa, /canary and /land-and-deploy silently produce reports whose screenshots do not exist. Plain `screenshot <path>` is unaffected. Refs are built as getByRole(role, {name}) and disambiguated with .nth() when role+name repeats. That disambiguation cannot fire for a node with NO accessible name: the locator degrades to getByRole(role) with no name filter, and the count driving .nth() is taken from the FILTERED aria snapshot while getByRole matches the unfiltered DOM. Measured on a live page: the tree surfaced 2 unnamed paragraphs, the DOM had 9. Landmarks (banner/main/contentinfo) and paragraphs are correctly unnamed per ARIA, so this is the common case rather than an edge case. boundingBox() then hits Playwright strict mode, and the catch allowlisted only timeout/closed/Target/Execution-context messages — so the strict-mode error was re-thrown and aborted every remaining annotation. Two changes: - `.first()` before boundingBox(), so an ambiguous ref draws a box on its first match instead of aborting. The heatmap path below has always tolerated this via a bare `catch {}`; annotate was the only path that could be killed outright. - the catch no longer re-throws on unrecognised messages. A box we cannot measure is a box we do not draw, never a reason to lose the rest of the page. Set BROWSE_DEBUG to see what was skipped. Also: `-o` passed without `-a`/`-H` was silently ignored (exit 0, no file), which reads as "screenshots are broken" rather than "you forgot a flag". It now warns and points at `browse screenshot <path>`. Verified by rebuilding both ways against the same page with 51 refs present: before — "Selector matched multiple elements", no file written after — exit 0, 229KB PNG Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(version-bump): missing or empty VERSION no longer repairs a fabricated 0.0.0.0 into package.json repair now fails with exit 2 when the VERSION file is absent or empty instead of folding to DEFAULT ("0.0.0.0") — which passed VERSION_RE and regressed package.json below where it started. classify gains an additive versionFileExists field so /ship can tell a real 0.0.0.0 from a fabricated one. Re-derived from PR #2612 under the generated-file screening rule. Fixes #2600 (repair half; the path-configurability half landed in v1.67 via #2531). Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(memory-ingest): --probe counts post-attribution, through the same gate --bulk uses probeMode previously stat'd every walked file, so setup-gbrain gated its silent bulk ingest on pre-filter counts that the write path would never ingest (#2394). The attribution decision now lives in ONE shared gate (sessionIsAttributable — cheap-parse: cwd extraction + memoized resolveGitRemote, never a full page build) used by BOTH probeMode and preparePages, so the two stages' post-attribution counts are structurally identical. ProbeReport gains skipped_unattributed; the probe prints what it excluded and --include-unattributed restores raw counts. The parity is pinned at the prepare stage (probe post-attribution == transcripts reaching import), deliberately NOT == final written. Re-derived from PR #2612 under the generated-file screening rule; the shared-gate design and the remote memo are additions from the plan review. Fixes #2394. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(browse): allow CPU and network throttling for performance measurement Adds Emulation.setCPUThrottlingRate and Network.emulateNetworkConditions to CDP_ALLOWLIST. Motivation: diagnosing a real "uploads take 1-2 minutes" report, the only machine available was a fast developer workstation. Client-side processing measured 1.4s where the user experienced minutes, so the conclusion had to be reached arithmetically rather than observed. Throttling would have let the measurement reproduce the reporter's conditions directly. Both fit the existing posture rather than widening it: - Emulation already allows setDeviceMetricsOverride, clearDeviceMetricsOverride and setUserAgentOverride, which are equally mutating and scoped to the tab. - Neither method reads page content. setCPUThrottlingRate affects only timing; emulateNetworkConditions constrains traffic rather than inspecting it, so no request bodies, headers or cookies are exposed. Both are output: 'trusted' because they return no page-derived data. scope 'tab' for both, matching the surrounding Emulation entries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(session-update): lock pidfile records the live holder; hard TTL bounds every wedge (#2613) echo $$ inside the backgrounded subshell recorded the PARENT hook's PID — which exits immediately — so every subsequent session judged the lock stale and rm -rf'd a LIVE holder's lock, letting concurrent updaters run over each other. The pidfile now records ${BASHPID:-$(sh -c 'echo $PPID')} (macOS bash 3.2 has no BASHPID; the sh child's PPID is exactly this subshell). Staleness is now two independent detectors: PID liveness (as before, but against the real holder), and a 30-minute hard TTL on the heartbeat mtime — reclaimed regardless of kill -0, so a recycled PID or hung holder can't wedge the lock forever. The holder touches the pidfile after the pull and after setup, so a legitimately-slow run keeps itself alive. Empty and missing pidfiles are respected inside the TTL window (the mkdir→echo race) and reclaimed past it. Fixes #2613. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(browse): explicit windowsHide on every Bun.spawn site + census tripwire (#2575 residual) Bun.spawn sites were structurally outside the windowsHide census (it swept child_process bindings only). The runtime was already safe — native Bun hides consoles by default and bun-polyfill.cjs defaults windowsHide !== false since #2523/#2539 — but implicit defaults are exactly what regress silently. Every Bun.spawn/spawnSync in browse/src now carries the explicit flag (harmless on unix-only sites like Xvfb/xattr/open), and a second SWEEP in windows-spawn-hide.test.ts fails CI on any new flagless Bun.spawn site. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gbrain): brain worktree advances on the daily sync — no more silently stale brains (#2516) The daily pull refreshed only ~/.gstack itself, never the detached worktree at ~/.gstack-brain-worktree that gbrain actually indexes — so after setup the brain served stale pages forever unless setup-gbrain/sync-gbrain happened to run. brain-sync --once now advances the worktree once per 24h behind an ATTEMPT stamp (.brain-worktree-last-advance — a persistently-failing advance warns once a day, not at every skill boundary), inside the existing run lock and before any ingest step touches the worktree. The new gstack-gbrain-source-wireup --advance-only is built for the unattended cadence: git-only (no gbrain prereqs), pins every operation to the managed worktree (refuses paths that are not worktrees of the artifacts repo), refuses dirty worktrees, and never runs the force-remove recovery — a cron path must not be able to delete local changes. A static pin keeps the force-remove out. docs/gbrain-sync.md stops overclaiming the old cadence. Fixes #2516. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(memory-ingest): honor the per-remote deny/read-only trust policy (#2392) Transcript ingest now respects the same trust store as code import — the gate existed only in gstack-gbrain-sync's runCodeImport, so memory-ingest happily ingested transcripts from deny-listed repos. preparePages filters prepared transcript pages through ONE batch policy lookup (new 'get --batch' verb on bin/gstack-gbrain-repo-policy — the script owns URL normalization; the client adds repoPolicyTierBatch, one spawn for all distinct remotes, so large corpora never pay a 10s-timeout subprocess per remote). Outcomes match code-import semantics: read-only → clean skip (skipped_policy_readonly), deny → counted refusal (skipped_policy_deny), corrupted/unreadable store → HARD ERROR before any write (state, staging, egress receipt, and import all untouched) with the recovery command named — policy corruption must never read as successful ingestion. Artifacts are never policy-filtered (their git_remote is a project slug, not a remote). Fixes #2392. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(config): repo_mode keeps its empty no-default semantics (#2611 follow-up) The ported defaults table synthesized repo_mode → "unknown", but EMPTY is load-bearing for that key: gstack-repo-mode treats any non-empty answer as a user override and skips its own repo classification — the synthesized default turned the classifier into dead code (REPO_MODE=unknown everywhere; caught by test/gstack-repo-mode.test.ts via the wave's cross-agent blame protocol). repo_mode joins the empty-is-real carve-outs (empty output, exit 0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pair-agent): consent before killing a healthy headless daemon The pair-agent headed switch spawned 'connect --force-restart' unconditionally — auto-killing a live headless daemon (open tabs, cookies, logins) in direct contradiction of the iron rule it sits beside ('only an explicit --force-restart may kill a live daemon'). The CLI now captures daemon liveness BEFORE ensureServer (which can itself boot a fresh daemon) and relaunches only when the user passed --force-restart to pair-agent; otherwise it prints the tab count and continues against the existing daemon. The /pair-agent skill gains a matching one-way-door consent question (template half rides the wave's template block). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gbrain-status): MCP scoping is per-project, and project-local beats user scope hasRemoteOnlyGbrainMcp scanned EVERY project's mcpServers in ~/.claude.json, so one project's remote gbrain registration reclassified broken local engines as thin-client machine-wide. It now reads user scope plus only the cwd's nearest-ancestor project key. The precedence itself was verified empirically and hermetically (fake HOME + CLAUDE_CONFIG_DIR fixtures, claude 2.1.233): with both scopes defining gbrain, 'claude mcp get gbrain' reports Scope: Local config — PROJECT-LOCAL WINS. Both in-repo consumers assumed the opposite; brain-cache's endpoint resolution flips to nearest-ancestor-project-first, and the stale user-first pin in brain-cache-roundtrip now pins the verified precedence. (The user-first jq in the brain-sync preamble resolver gets the same swap in the template block.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(slug): gstack-slug matches remote-slug's owner-repo canonical form (live misfile bug) Found live during this wave's CEO review: bin/gstack-slug emitted SLUG=garrytan for this garrytan/gstack worktree while remote-slug correctly gave garrytan-gstack — decisions, timeline, ceo-plans, and learnings were filing into the wrong project store (observed polluting Context Recovery with another repo's decisions). Root cause: a stray empty ~/.git directory made the walk-up crown $HOME as the outermost project root; the remote lookup ran only against that root, failed silently, and the basename fallback cached 'garrytan' sticky. NOT worktree-specific — any strong marker on a non-repo ancestor triggered it. Fix: the walk now finds the outermost ancestor whose .git actually resolves an origin remote and derives owner-repo with remote-slug's byte-identical parse; marker-only ancestors keep anchoring the basename fallback but can no longer shadow a real remote. A new cache self-heal recomputes the poisoned shape (cached == basename of a marker root while a remote-bearing repo exists below), preserving legit #2212 stickiness. Nested-repo walk-up, no-remote and non-git fallbacks, and the SLUG=/BRANCH= eval contract are unchanged, pinned by a 10-case parity suite. Store migration for pre-fix data is tracked in TODOS.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(brain-sync): per-record spool dir — the enqueue/drain race dies structurally Producers appended lines to .brain-queue.jsonl while the drain re-read and os.replace'd it; the in-code comment admitted a lockless append between the re-read and the replace was lost. Locks and rename-rotation designs were both reviewed and rejected (each retained a tail race); the shipped design is a maildir-style spool: one FILE per record in .brain-queue.d/ (tmp + atomic rename), the drain snapshots filenames, processes, and deletes exactly what it snapshotted. Writer and drainer never share an inode — nothing to race. Semantics: at-least-once (a crash between process and unlink re-drains; downstream content-hash dedup absorbs duplicates); retained (privacy-held) records keep their files; unparseable records are kept + warned, never destroyed. Legacy .brain-queue.jsonl migrates atomically on the next drain (crash-leftover .migrating files recovered too); status/drop-queue count both surfaces; discover-new writes spool records and advances its cursor per-record-written. The preamble's queue-depth line switches to spool count in this wave's template block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bin-context): native slug fallback walks up like bash gstack-slug slugFromEnvironment derived the slug from the INNERMOST repo's origin while bash gstack-slug walks to the outermost project root — nested/vendored repos split their stores across the bash/native boundary (win32 hits the native path constantly). The native fallback now ports _outermost_project_root faithfully (strong/weak markers, outermost-strong-wins, 64-depth cap, fixed-point termination) plus the full resolution order: env override → walk-up → sticky cache with the #1125 self-heal → remote get-url → basename. Twelve mirrored scenarios drive BOTH implementations against the same fixtures and pin identical slugs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(next-version): git fallback queries the live remote, never mutates, and keeps 3-digit width The degraded path counted every remote-tracking ref on every remote — stale experiment branches and second remotes inflated version allocation, and a failed base read flipped 3-digit repos to 4-digit slots. Now: ls-remote --heads origin first (GIT_TERMINAL_PROMPT=0, 5s timeout, zero local ref mutation); on failure, local refs/remotes/origin ONLY with an explicit stale-refs warning; a failed base read zeroes at the LOCAL version file's width so a 3-digit repo allocates 0.0.1, not 0.0.1.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(setup): hooks register the global-install path and re-point stale ones Registering hooks from a dev worktree baked that worktree's absolute path into settings.json — deleting the worktree left a dead hook erroring on every session stop, and the presence-only dedup (list-sources | grep) could never re-point it. setup's hook paths now route through _hook_install_path (global install preferred, source dir fallback), and the new ensure-event verb on gstack-settings-hook compares the registered command payload against canonical: identical → no write, different → single atomic replacement (never zero or two registrations). The plan-tune hooks had the same stale pattern and get the same fix without re-triggering their consent prompt. Also hardened: bun 1.3.13 turns an uncaught sync fs error in bun -e into a SILENT exit 0 — the registrar's write path now catches, prints, and exits 1, so a failed update can never report fake-green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(preamble): learnings capture is unconditional at completion (#2402) 43 of 44 learnings entries came from explicit /learn — the completion-status prose read 'if you discovered a durable project quirk... log it', which models treated as optional. The step now ALWAYS runs: review the session for durable learnings, log each one, and state 'No durable learnings this session' explicitly when the review comes up empty — an empty result, never a skipped step. Re-derived from PR #2612 under the generated-file screening rule. Fixes #2402. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scrape): untrusted-content warning on the page-fetching skills (#2441) /scrape and /skillify consumed page content with zero injection guidance — the CHANGELOG claimed coverage the skills didn't have. The warning now lives in ONE exported const (UNTRUSTED_CONTENT_WARNING in resolvers/browse.ts), embedded in the browse COMMAND_REFERENCE as before AND injected standalone into both skills via the new {{UNTRUSTED_CONTENT_WARNING}} token — single source, wording can never drift between surfaces. Re-derived from PR #2612 under the generated-file screening rule. (Structural isolation for skillify-generated code is tracked as its own TODO.) Fixes #2441. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): checklist paths resolve from the installed skill root (#2518) /review Step 2 read .claude/skills/review/checklist.md — a path relative to the TARGET repo, which only resolves in gstack's own checkout. Every checklist/greptile-triage/TODOS-format reference (six across five templates — two more than the issue named, same class) now uses the installed-root form ~/.claude/skills/gstack/review/... that the templates' other references already use. The install-root class itself (non-default install dirs) is #1882, deliberately its own PR. Fixes #2518. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pair-agent): one-way-door consent question before a daemon relaunch (template half) The skill flow now checks daemon liveness before Step 4 and asks an explicit one-way-door question (tabs/cookies/logins are lost) before passing --force-restart — never proceeding on a vague reply. Pairs with the CLI-half commit that stopped pair-agent auto-killing live daemons. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(codex): resume does not amortize the ~21K session prelude (#2387) Measured (#2387): every codex exec call pays Codex's session prelude, and a resumed call came in slightly ABOVE a fresh one — resume buys continuity, never token savings. The skill now says so where the resume flow lives: prefer one codex call per skill, batch questions into it. Fixes #2387. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(upgrade): fast-forward first; reset --hard only behind a proved-safe gate (#2517) /gstack-upgrade went straight to stash + reset --hard origin/main. Now it tries git pull --ff-only --autostash first (the same policy session-update's auto-upgrade uses). The destructive fallback runs unprompted ONLY when both git status --porcelain AND git rev-list origin/main..HEAD are empty — a clean tree with unpushed local commits is NOT safe, reset destroys them. Anything else requires an explicit one-way-door confirmation that lists every dirty file and unpushed commit being discarded. Fixes #2517. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(preamble): brain-sync block counts the spool queue and resolves MCP project-first Two resolver halves deferred from earlier wave commits: the queue-depth line counts .brain-queue.d/*.json spool records (plus legacy lines until the drain migrates them), and GBRAIN_MCP_ENTRY_JQ swaps its operands to nearest-ancestor-project-first — matching the empirically verified Claude Code precedence (project-local beats user scope) instead of the backwards user-first assumption. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate SKILL.md docs + golden fixtures (single regen for the template block) Pure generator output for the six template/resolver commits above (learnings capture, untrusted-content warning, review paths, pair-agent consent, codex resume note, upgrade ff-only, brain-sync block) — bun run gen:skill-docs + --host codex + --host factory, with the three ship golden fixtures refreshed per the documented procedure. The three sidecar-path pins in gen-skill-docs.test.ts move to the new installed-root/$GSTACK_ROOT contract (#2518). Restores template freshness; full suite green from here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: TODOS.md — strike the six wave-fixed residuals, add two follow-ups The v1.67 adversarial-review residuals section shrinks to the one item the wave couldn't reach (iOS tap routing — needs real-device verification). New entries: skillify structural isolation (a prose warning is not a boundary for page-derived generated code) and the slug store migration (pre-fix sessions on stray-marker machines filed data under the degraded slug; post-fix reads go to the correct store, so history needs a merge/alias). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: align cross-cutting pins with the wave's contracts Three suites pinned pre-wave behavior: browse's gstack-config test asserted the old unknown-key ''/exit-0 shape (#2611 made it exit 1); the Windows-paths suite pinned O_APPEND enqueue atomicity (the spool design satisfies the same invariant via tmp + os.replace, one file per record — pinned in its new form); and nine carve-guard skeleton ceilings absorbed the #2402 unconditional-learnings prose (~450B per skill), bumped with measured values per the guard's own protocol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: re-anchor the referenced-path scanner self-check to the gstack-rooted review refs The self-check pinned the review checklist as a class-1 alias-relative ref; #2518 moved those refs to the installed gstack root (class 2). The guard now proves the scanner sees them in their new class, so the class-2 assertion can't go vacuous. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: pin the wave's prose-tier behaviors (ship coverage-audit gap closure) The coverage audit found one regression-shaped gap: nothing pinned that the upgrade template's ff-only pull precedes the gated reset --hard (#2517) — a future template edit reverting to reset-first would fail nothing. Pinned: the ordering, the FF_OK gate, and the unpushed-commits check. Also pinned the two minor gaps: the {{UNTRUSTED_CONTENT_WARNING}} injection points in scrape/skillify (#2441) and brain-uninstall's spool-dir cleanup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: pre-landing review round — 8 auto-fixes + 8 accepted findings hardened The ship review army (4 specialists + red-team + checklist, 29 findings) produced 8 mechanical auto-fixes and 11 decisions; the accepted set: - win32 slug parity completed: lib/bin-context.ts gains the remote-first outermost walk + degraded-cache self-heal the bash side got this wave — the two implementations now agree on the stray-marker live-bug shape, pinned by shared fixtures (multi-specialist 9/10 finding). - probe honors the plan's bounded-read decision: 256KB prefix, extraction semantics mirrored from parseTranscriptJsonl so probe/prepare can never diverge on the same file (>1MB transcript test). - policy normalize parity: bash normalize() now matches canonicalizeRemote on .git/-trailing and uppercase-.GIT shapes (7-shape corpus pinned two ways) — a deny for those shapes could previously slip the transcript gate. - session-update reclaim is TOCTOU-safe (atomic mv-aside on both branches). - settings-hook: unparseable settings.json errors instead of being replaced with {}; ensure-event keys on (event, source) so matcher changes update in place — never zero or two registrations. - dot-only slug guard at both parse sites (hostile 'url = ..' can't escape projects/); enqueue tmp-file janitor (1h TTL, inside the drain lock); brain-sync .migrating never clobbered; drop-queue/status count .migrating; snapshot -o warning correct + surfaced in diff mode; version-bump test order-dependence removed; uninstall clears the advance stamp. Deferred with record: slug heal-probe cost sentinel (P3 TODO), FF_OK conflation (noted, misdiagnosis-only). 270 pass / 0 fail across the 10 touched suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: adversarial round — the P0 finalize fail-safe and 12 hardened findings Three adversarial passes (Claude fresh-context, Codex chaos, Codex structured with P1 gate) on the full wave diff. Multi-source findings, all fixed: - P0: finalize_queue is now explicit-delete-only — a record is unlinked ONLY when classification proves it staged or dropped; a classifier crash, a missing class file, or a malformed pulled .brain-privacy-map.json (which previously nuked the whole snapshotted queue, remotely triggerable) now retains everything, warns, and re-drains next run. load_privacy_map treats corrupt maps as retain-all, never as empty. - next-version cannot silently drop a live claim: unreadable advertised refs get a targeted --depth=1 fetch + retry; still-unreadable claims surface as UNKNOWN warnings instead of duplicate-version silence. - session-update lock: ownership-checked EXIT trap (a TTL-reclaimed holder can no longer delete the new holder's lock) + a 5-min background heartbeat so a legitimately-slow pull/setup is never reclaimed while alive. - ensure-event collapses ALL same-(event,source) duplicates to one canonical entry; unique per-process tmp path; setup call sites surface (not swallow) the hardened refusals. - memory-ingest: --limit counts only policy-permitted pages (denied records no longer starve permitted ones); --probe applies the same policy filter as --bulk (skipped_policy_* fields on the report). - version-bump repair accepts a genuine literal 0.0.0.0 VERSION file. - slug heal restricted to the stray-.git shape — package.json-anchored wrapper roots keep their legit sticky identity (#2212 preserved). - brain-sync: idle fast path sees leftover .migrating records; unparseable spool records quarantine instead of warning forever; migration comment stops overclaiming the transition-window race. - CDP throttling justifications document override persistence (callers own restoration), pinned in the allowlist test. Deferred with record: deny retroactivity for already-ingested pages (P2 TODO, same semantics as the code-import gate); legacy-migration tail race (transition-window, requires pre-spool writers). 288 pass / 0 fail across the 10 touched suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate SKILL.md docs + goldens (Windows-separator jq fix) Pure generator output for the brain-sync block's jq ancestor match now accepting backslash-formed Windows project keys — previously project-scoped brains were invisible on Windows while the TS scope resolvers saw them. Golden ship fixtures refreshed per the documented procedure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: codex verify-pass residuals — chunked cwd read, post-filter partial count, migrating depth The verify re-review passed the P1 gate (0 P1s) and left three residuals, all applied: transcriptCwdFromPrefix reads in chunks until one complete record (4MB cap) so a giant first prompt can't truncate mid-JSON and break probe/bulk parity; partial_pages derives from the FINAL prepared set instead of the whole scanned corpus; the preamble queue-depth line counts leftover .brain-queue.jsonl.migrating records like the status path does (regen + goldens included). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: bump version and changelog (v1.68.0.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: update project documentation for v1.68.0.0 BROWSER.md: fix the $B cdp example (positional JSON params, not --json; depth is the real CDP param) and add the new perf-throttling examples (Emulation.setCPUThrottlingRate, Network.emulateNetworkConditions) with their clear-override counterparts. USING_GBRAIN_WITH_GSTACK.md: the state-files table row for the sync queue now names the maildir-style spool dir .brain-queue.d/ that replaced .brain-queue.jsonl this release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: align memory-pipeline probe pins with the #2394 stage-count contract The paid-tier E2E pinned the pre-fix contract (probe headline = raw discovered). Probe now counts post-attribution — the same gate --bulk uses — with an explicit unattributed-skip line. Adds the --include-unattributed companion pin so all 9 fixtures stay accounted for. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(next-version): batch missing-tip fetches — one bounded round trip, never a per-branch crawl The targeted-fetch retry for branches whose advertised tip has no local object ran ONE git fetch per branch (10s cap each). On a shallow clone against a busy remote that crawls the network for minutes — CI's shard deadline killed the free suite mid-file. Missing tips now collect into a single batched shallow fetch (15s cap); refs still missing after the batch (one unservable ref fails the whole transfer) get a capped per-branch retry, and anything past the cap warns as an UNKNOWN claim instead of fetching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(next-version): pin the batched fetch + make the offline-contract tests hermetic Two new G2 pins: N unfetched claim branches resolve with exactly ONE fetch spawn (PATH-shimmed git counts invocations), and one unservable ref no longer poisons the batch — live claims resolve via the bounded retry while only the ghost warns UNKNOWN. The #2545 offline-contract tests now run the CLI in a local fixture repo instead of the repo's own checkout: the checkout path did a live ls-remote against the real origin (operator-network-dependent, and the CI shard-deadline hang). The online-contract test gains a succeeding gh stub, so fallback:null is asserted deterministically instead of only when the operator happens to be authed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(redact-cli): derive the synthetic AWS-key fixture — no contiguous credential literal in source The CI quality gate scans every ADDED diff line with the redact engine, so the #2610 port's raw fixture literals failed the very gate they exist to test. The fixture is now assembled at runtime; the scanner still receives the identical bytes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(next-version): pin the fixture's host via origin-URL sniff — kills the last environment dependence The hermetic offline-contract fixture had no origin remote, so detectHost() fell through to auth probes: a machine with glab authed passed via the gitlab path while a bare CI runner read host:unknown (offline stays false there) and failed. The fixture now pushes to a local bare origin at a path containing github.com — the URL sniff pins host:github identically everywhere, asserted explicitly in both tests, with every git call still local. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: y$un_ <forrest.sun527@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: benjamin beres <benjamin.beres@bienpreter.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Ricky <ricky@kinokostudio.com.hk> Co-authored-by: Connex Client Access <paul@paulkortman.com> Co-authored-by: henbima <henbima@gmail.com>
3749 lines
164 KiB
TypeScript
3749 lines
164 KiB
TypeScript
import { describe, test, expect, beforeAll } from 'bun:test';
|
|
import { assertSinglePreamble } from '../scripts/gen-skill-docs';
|
|
import { COMMAND_DESCRIPTIONS } from '../browse/src/commands';
|
|
import { SNAPSHOT_FLAGS } from '../browse/src/snapshot';
|
|
import * as fs from 'fs';
|
|
import * as path from 'path';
|
|
import * as os from 'os';
|
|
import { spawnSync } from 'child_process';
|
|
|
|
const ROOT = path.resolve(import.meta.dir, '..');
|
|
const MAX_SKILL_DESCRIPTION_LENGTH = 1024;
|
|
|
|
// Carved-skill aware (v2 plan T9): ship is now a skeleton SKILL.md + sections/*.md.
|
|
// Read the union so assertions about content that MOVED into a section still pass.
|
|
// The skeleton is a subset of the union, so skeleton-only assertions also hold,
|
|
// and negative assertions stay safe (the absent phrases live in neither file).
|
|
function readSkillUnion(skill: string): string {
|
|
let t = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const secDir = path.join(ROOT, skill, 'sections');
|
|
if (fs.existsSync(secDir)) {
|
|
for (const f of fs.readdirSync(secDir).sort()) {
|
|
if (f.endsWith('.md')) t += '\n' + fs.readFileSync(path.join(secDir, f), 'utf-8');
|
|
}
|
|
}
|
|
return t;
|
|
}
|
|
function readShipUnion(): string {
|
|
return readSkillUnion('ship');
|
|
}
|
|
|
|
function extractDescription(content: string): string {
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
expect(fmEnd).toBeGreaterThan(0);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
const lines = frontmatter.split('\n');
|
|
let description = '';
|
|
let inDescription = false;
|
|
const descLines: string[] = [];
|
|
|
|
for (const line of lines) {
|
|
if (line.match(/^description:\s*\|?\s*$/)) {
|
|
inDescription = true;
|
|
continue;
|
|
}
|
|
if (line.match(/^description:\s*\S/)) {
|
|
return line.replace(/^description:\s*/, '').trim();
|
|
}
|
|
if (inDescription) {
|
|
if (line === '' || line.match(/^\s/)) {
|
|
descLines.push(line.replace(/^ /, ''));
|
|
} else {
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
|
|
if (descLines.length > 0) {
|
|
description = descLines.join('\n').trim();
|
|
}
|
|
return description;
|
|
}
|
|
|
|
function extractMarkdownSection(content: string, heading: string): string {
|
|
const escaped = heading.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
const startMatch = content.match(new RegExp(`^${escaped}.*$`, 'm'));
|
|
expect(startMatch?.index).toBeDefined();
|
|
const start = startMatch!.index!;
|
|
const afterHeading = start + startMatch![0].length;
|
|
const nextSection = content.slice(afterHeading).match(/\n## /);
|
|
const end = nextSection?.index === undefined
|
|
? content.length
|
|
: afterHeading + nextSection.index;
|
|
return content.slice(start, end).trim();
|
|
}
|
|
|
|
function extractPreambleBeforeWorkflow(content: string, workflowMarkers: string[]): string {
|
|
const markerIndexes = workflowMarkers
|
|
.map(marker => content.indexOf(marker))
|
|
.filter(index => index >= 0);
|
|
expect(markerIndexes.length).toBeGreaterThan(0);
|
|
return content.slice(0, Math.min(...markerIndexes));
|
|
}
|
|
|
|
function isRepoRootSymlink(candidateDir: string): boolean {
|
|
try {
|
|
return fs.realpathSync(candidateDir) === fs.realpathSync(ROOT);
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
|
|
// Dynamic template discovery — matches the generator's findTemplates() behavior.
|
|
// New skills automatically get test coverage without updating a static list.
|
|
const ALL_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; name: string }> = [];
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
skills.push({ dir: '.', name: 'root gstack' });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) {
|
|
skills.push({ dir: entry.name, name: entry.name });
|
|
}
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
// hosts/claude.ts generation.skipSkills entries would filter here; the set is
|
|
// currently empty (the /claude outside-voice template was removed).
|
|
// The claude host deliberately skips some skills (skipSkills — e.g. the
|
|
// /claude outside-voice skill exists only for non-Claude hosts), so those
|
|
// dirs have a SKILL.md.tmpl but no generated claude-host SKILL.md on a fresh
|
|
// checkout. Every generated-file assertion must exclude them or it is red on
|
|
// every clean clone (it was, invisibly, until the free suite ran in CI).
|
|
import { getHostConfig as __getHostConfig } from '../hosts/index';
|
|
const CLAUDE_SKIPPED = new Set(__getHostConfig('claude').generation.skipSkills ?? []);
|
|
const CLAUDE_GENERATED_SKILLS = ALL_SKILLS.filter(s => !CLAUDE_SKIPPED.has(s.dir));
|
|
|
|
describe('gen-skill-docs', () => {
|
|
test('generated SKILL.md contains all command categories', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
const categories = new Set(Object.values(COMMAND_DESCRIPTIONS).map(d => d.category));
|
|
for (const cat of categories) {
|
|
expect(content).toContain(`### ${cat}`);
|
|
}
|
|
});
|
|
|
|
test('generated SKILL.md contains all commands', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
const display = meta.usage || cmd;
|
|
expect(content).toContain(display);
|
|
}
|
|
});
|
|
|
|
test('command table is sorted alphabetically within categories', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
// Extract command names from the Navigation section as a test
|
|
const navSection = content.match(/### Navigation\n\|.*\n\|.*\n([\s\S]*?)(?=\n###|\n## )/);
|
|
expect(navSection).not.toBeNull();
|
|
const rows = navSection![1].trim().split('\n');
|
|
const commands = rows.map(r => {
|
|
const match = r.match(/\| `(\w+)/);
|
|
return match ? match[1] : '';
|
|
}).filter(Boolean);
|
|
const sorted = [...commands].sort();
|
|
expect(commands).toEqual(sorted);
|
|
});
|
|
|
|
test('generated header is present in SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
});
|
|
|
|
test('generated header is present in browse/SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
});
|
|
|
|
test('snapshot flags section contains all flags', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
expect(content).toContain(flag.short);
|
|
expect(content).toContain(flag.description);
|
|
}
|
|
});
|
|
|
|
test('every skill has a SKILL.md.tmpl template', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
expect(fs.existsSync(tmplPath)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('every skill has a generated SKILL.md with auto-generated header', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const mdPath = path.join(ROOT, skill.dir, 'SKILL.md');
|
|
expect(fs.existsSync(mdPath)).toBe(true);
|
|
const content = fs.readFileSync(mdPath, 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
}
|
|
});
|
|
|
|
// #1778: strict YAML parsers (Codex/OpenAI skill loading) reject frontmatter
|
|
// whose plain `description:` scalar contains an interior ": " (read as a nested
|
|
// mapping). Parse EVERY generated frontmatter block with a strict YAML parser,
|
|
// not just string-check that name:/description: exist.
|
|
function frontmatterBlock(content: string): string {
|
|
expect(content.startsWith('---\n')).toBe(true);
|
|
const end = content.indexOf('\n---', 4);
|
|
expect(end).toBeGreaterThan(0);
|
|
return content.slice(4, end);
|
|
}
|
|
|
|
test('every generated SKILL.md frontmatter parses as strict YAML', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const fm = frontmatterBlock(content);
|
|
let parsed: any;
|
|
expect(() => { parsed = Bun.YAML.parse(fm); },
|
|
`frontmatter for ${skill.dir} must be valid YAML`).not.toThrow();
|
|
expect(typeof parsed?.name).toBe('string');
|
|
expect(typeof parsed?.description).toBe('string');
|
|
}
|
|
});
|
|
|
|
test('every generated Codex (.agents/skills) frontmatter parses as strict YAML', () => {
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return; // skip if external hosts not generated
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const mdPath = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(mdPath)) continue;
|
|
const fm = frontmatterBlock(fs.readFileSync(mdPath, 'utf-8'));
|
|
expect(() => Bun.YAML.parse(fm),
|
|
`Codex frontmatter for ${entry.name} must be valid YAML`).not.toThrow();
|
|
}
|
|
});
|
|
|
|
test(`every generated SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const description = extractDescription(content);
|
|
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
|
|
}
|
|
});
|
|
|
|
test(`every Codex SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return; // skip if not generated
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
const description = extractDescription(content);
|
|
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
|
|
}
|
|
});
|
|
|
|
test('every Codex SKILL.md description stays under 900-char warning threshold', () => {
|
|
const WARN_THRESHOLD = 900;
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return;
|
|
const violations: string[] = [];
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
const description = extractDescription(content);
|
|
if (description.length > WARN_THRESHOLD) {
|
|
violations.push(`${entry.name}: ${description.length} chars (limit ${MAX_SKILL_DESCRIPTION_LENGTH}, ${MAX_SKILL_DESCRIPTION_LENGTH - description.length} remaining)`);
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('package.json version matches VERSION file (npm-valid translation)', () => {
|
|
// Decision 11 (v1.67 wave): VERSION stays the 4-digit source of truth;
|
|
// package.json carries the npm-valid 3-digit translation (npm rejects a
|
|
// fourth component). The pre-v1.67 1:1 four-digit mirror is also accepted
|
|
// (grandfathered until the next write), matching gstack-version-bump's
|
|
// own drift contract.
|
|
const pkg = JSON.parse(fs.readFileSync(path.join(ROOT, 'package.json'), 'utf-8'));
|
|
const version = fs.readFileSync(path.join(ROOT, 'VERSION'), 'utf-8').trim();
|
|
const npmTranslation = version.split('.').slice(0, 3).join('.');
|
|
expect([npmTranslation, version]).toContain(pkg.version);
|
|
});
|
|
|
|
test('generated files are fresh (match --dry-run)', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// Every skill should be FRESH
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const file = skill.dir === '.' ? 'SKILL.md' : `${skill.dir}/SKILL.md`;
|
|
expect(output).toContain(`FRESH: ${file}`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('no generated SKILL.md contains unresolved placeholders', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const unresolved = content.match(/\{\{[A-Z_]+\}\}/g);
|
|
expect(unresolved).toBeNull();
|
|
}
|
|
});
|
|
|
|
test('templates contain placeholders', () => {
|
|
// P2 (v1.2.0): the root template is a pure router — only {{PREAMBLE}}.
|
|
// The browse command/snapshot placeholders live in browse/SKILL.md.tmpl now.
|
|
const rootTmpl = fs.readFileSync(path.join(ROOT, 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(rootTmpl).toContain('{{PREAMBLE}}');
|
|
expect(rootTmpl).not.toContain('{{COMMAND_REFERENCE}}');
|
|
expect(rootTmpl).not.toContain('{{SNAPSHOT_FLAGS}}');
|
|
|
|
const browseTmpl = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(browseTmpl).toContain('{{COMMAND_REFERENCE}}');
|
|
expect(browseTmpl).toContain('{{SNAPSHOT_FLAGS}}');
|
|
expect(browseTmpl).toContain('{{PREAMBLE}}');
|
|
});
|
|
|
|
test('generated SKILL.md contains operational self-improvement (replaced contributor mode)', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('Contributor Mode');
|
|
expect(content).not.toContain('gstack_contributor');
|
|
expect(content).not.toContain('contributor-logs');
|
|
expect(content).toContain('Operational Self-Improvement');
|
|
expect(content).toContain('gstack-learnings-log');
|
|
expect(content).toContain('gstack-learnings-search --limit 3');
|
|
});
|
|
|
|
test('generated SKILL.md with LEARNINGS_LOG contains operational type', () => {
|
|
// Check a skill that has LEARNINGS_LOG (e.g., review)
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('operational');
|
|
});
|
|
|
|
test('generated SKILL.md contains session awareness', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_SESSIONS');
|
|
expect(content).toContain('RECOMMENDATION');
|
|
});
|
|
|
|
test('generated SKILL.md contains branch detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_BRANCH');
|
|
expect(content).toContain('git branch --show-current');
|
|
});
|
|
|
|
// #2001: update_check: false silences the binary but the upgrade-handling
|
|
// instruction prose used to ship unconditionally. Every skill that carries
|
|
// the runtime config-echo cluster must (a) echo UPDATE_CHECK so the
|
|
// instruction layer can read it, and (b) gate the UPGRADE_AVAILABLE /
|
|
// JUST_UPGRADED prose on it — the same echo-then-gate convention every other
|
|
// flag (PROACTIVE, SKILL_PREFIX, EXPLAIN_LEVEL, QUESTION_TUNING) follows.
|
|
test('update_check opt-out gates preamble echo and upgrade-handling prose (issue #2001)', () => {
|
|
let checked = 0;
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
// Scope: only skills that render the runtime config-echo cluster.
|
|
if (!content.includes('echo "QUESTION_TUNING: $_QUESTION_TUNING"')) continue;
|
|
checked++;
|
|
expect(content, `${skill.dir} must echo UPDATE_CHECK`).toContain('echo "UPDATE_CHECK: $_UPDATE_CHECK"');
|
|
expect(content, `${skill.dir} must read update_check config`).toContain('_UPDATE_CHECK=$(');
|
|
// Whenever the upgrade-handling prose ships, it must gate on the flag.
|
|
if (content.includes('UPGRADE_AVAILABLE <old> <new>')) {
|
|
expect(content, `${skill.dir} upgrade prose must gate on UPDATE_CHECK`)
|
|
.toContain('If `UPDATE_CHECK` is `"false"`');
|
|
}
|
|
}
|
|
// Guard against the scope filter silently matching nothing.
|
|
expect(checked).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('tier 2+ skills contain ELI10 simplification rules (AskUserQuestion format)', () => {
|
|
// Root SKILL.md is tier 1 (no AskUserQuestion format). Check a tier 2+ skill instead.
|
|
// v1.7.0.0 Pros/Cons format uses "ELI10 (ALWAYS)" rather than "Simplify (ELI10".
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('ELI10');
|
|
expect(content).toContain('plain English');
|
|
expect(content).toContain('not function names');
|
|
});
|
|
|
|
test('tier 1 skills do NOT contain AskUserQuestion format', () => {
|
|
// Use benchmark (tier 1) instead of root — root SKILL.md gets overwritten by Codex test setup
|
|
const content = fs.readFileSync(path.join(ROOT, 'benchmark', 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('## AskUserQuestion Format');
|
|
expect(content).not.toContain('## Completeness Principle');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry line', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('skill-usage.jsonl');
|
|
expect(content).toContain('~/.gstack/analytics');
|
|
});
|
|
|
|
test('plan-review generated preambles stay under the Option A budget', () => {
|
|
const reviewSkills = [
|
|
{
|
|
path: path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
|
|
markers: ['# Mega Plan Review Mode', '## Step 0: Detect platform and base branch'],
|
|
},
|
|
{
|
|
path: path.join(ROOT, 'plan-eng-review', 'SKILL.md'),
|
|
markers: ['# Plan Review Mode'],
|
|
},
|
|
];
|
|
|
|
// Plan skills carry the same preamble surface as other tier-≥2 skills
|
|
// (Artifacts Sync, Context Recovery, Routing Injection are load-bearing
|
|
// functionality, not optional). Budget is set to current size + small
|
|
// headroom; ratchet down if a future slim trims real bytes.
|
|
// Ratcheted from 33000 → 35000 when the gbrain context-load block was
|
|
// added (per /sync-gbrain plan §4). Ratcheted 35000 → 36500 in v1.27.0.0
|
|
// when generate-brain-sync-block.ts gained the gbrain_mcp_mode probe +
|
|
// remote-mode ARTIFACTS_SYNC status line (Path 4 of /setup-gbrain).
|
|
// Ratcheted 36500 → 39000 in the contributor wave when #1205 added the
|
|
// \\u-escape CJK rule (rule 12 + self-check item) to the AskUserQuestion
|
|
// preamble.
|
|
// Ratcheted 39000 → 40000 in plan-tune cathedral T14: question-tuning
|
|
// resolver gained the <gstack-qid:...> marker convention + the
|
|
// (recommended) label requirement (D2 + D18 — both load-bearing for
|
|
// hook enforcement). Adds ~700 bytes.
|
|
// Ratcheted 40000 → 60000 in v1.52.0.0 cap audit: ~20K headroom so
|
|
// future preamble adds don't trip the gate on each PR. Real runaway
|
|
// (preamble doubling) still trips; normal scope growth doesn't.
|
|
for (const skill of reviewSkills) {
|
|
const content = fs.readFileSync(skill.path, 'utf-8');
|
|
const preamble = extractPreambleBeforeWorkflow(content, skill.markers);
|
|
expect(Buffer.byteLength(preamble, 'utf-8')).toBeLessThan(60_000);
|
|
}
|
|
});
|
|
|
|
test('voice and writing-style preamble sections stay compact', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
const voice = extractMarkdownSection(content, '## Voice');
|
|
const writingStyle = extractMarkdownSection(content, '## Writing Style');
|
|
|
|
expect(Buffer.byteLength(voice, 'utf-8')).toBeLessThan(3_000);
|
|
expect(Buffer.byteLength(writingStyle, 'utf-8')).toBeLessThan(2_000);
|
|
});
|
|
|
|
test('slim voice section preserves the gstack voice contract', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
const voice = extractMarkdownSection(content, '## Voice');
|
|
|
|
expect(voice).toMatch(/lead with the point|direct/i);
|
|
expect(voice).toMatch(/file|function|line|command|real numbers/i);
|
|
expect(voice).toMatch(/user.*outcome|user.*experience|real user/i);
|
|
expect(voice).toMatch(/corporate|academic|PR|hype/i);
|
|
expect(voice).toMatch(/AI vocabulary|delve|crucial|robust/i);
|
|
expect(voice).toMatch(/user decides|user.*context|sovereignty|recommendation, not a decision/i);
|
|
});
|
|
|
|
test('preamble .pending-* glob is zsh-safe (uses find, not shell glob)', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
if (!content.includes('.pending-')) continue;
|
|
// Must NOT have a bare shell glob ".pending-*" outside of find's -name argument
|
|
expect(content).not.toMatch(/for _PF in [^\n]*\/\.pending-\*/);
|
|
// Must use find to avoid zsh NOMATCH error on glob expansion
|
|
expect(content).toContain("find ~/.gstack/analytics -maxdepth 1 -name '.pending-*'");
|
|
}
|
|
});
|
|
|
|
test('bash blocks with shell globs are zsh-safe (setopt guard or find)', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const bashBlocks = [...content.matchAll(/```bash\n([\s\S]*?)```/g)].map(m => m[1]);
|
|
|
|
for (const block of bashBlocks) {
|
|
const lines = block.split('\n');
|
|
|
|
for (const line of lines) {
|
|
const trimmed = line.trimStart();
|
|
if (trimmed.startsWith('#')) continue;
|
|
if (!trimmed.includes('*')) continue;
|
|
// Skip lines where * is inside find -name, git pathspecs, or $(find)
|
|
if (/\bfind\b/.test(trimmed)) continue;
|
|
if (/\bgit\b/.test(trimmed)) continue;
|
|
if (/\$\(find\b/.test(trimmed)) continue;
|
|
|
|
// Check 1: "for VAR in <glob>" must use $(find ...) — caught above by the
|
|
// $(find check, so any surviving for-in with a glob pattern is a violation
|
|
if (/\bfor\s+\w+\s+in\b/.test(trimmed) && /\*\./.test(trimmed)) {
|
|
throw new Error(
|
|
`Unsafe for-in glob in ${skill.dir}/SKILL.md: "${trimmed}". ` +
|
|
`Use \`for f in $(find ... -name '*.ext')\` for zsh compatibility.`
|
|
);
|
|
}
|
|
|
|
// Check 2: ls/cat/rm/grep with glob file args must have setopt guard
|
|
const isGlobCmd = /\b(?:ls|cat|rm|grep)\b/.test(trimmed) &&
|
|
/(?:\/\*[a-z.*]|\*\.[a-z])/.test(trimmed);
|
|
if (isGlobCmd) {
|
|
expect(block).toContain('setopt +o nomatch');
|
|
}
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
test('preamble-using skills have correct skill name in telemetry', () => {
|
|
const PREAMBLE_SKILLS = [
|
|
{ dir: '.', name: 'gstack' },
|
|
{ dir: 'ship', name: 'ship' },
|
|
{ dir: 'review', name: 'review' },
|
|
{ dir: 'qa', name: 'qa' },
|
|
{ dir: 'retro', name: 'retro' },
|
|
];
|
|
for (const skill of PREAMBLE_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain(`"skill":"${skill.name}"`);
|
|
}
|
|
});
|
|
|
|
test('qa and qa-only templates use QA_METHODOLOGY placeholder', () => {
|
|
const qaTmpl = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(qaTmpl).toContain('{{QA_METHODOLOGY}}');
|
|
|
|
const qaOnlyTmpl = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(qaOnlyTmpl).toContain('{{QA_METHODOLOGY}}');
|
|
});
|
|
|
|
test('QA_METHODOLOGY appears expanded in both qa and qa-only generated files', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
|
|
|
|
// Both should contain the health score rubric
|
|
expect(qaContent).toContain('Health Score Rubric');
|
|
expect(qaOnlyContent).toContain('Health Score Rubric');
|
|
|
|
// Both should contain framework guidance
|
|
expect(qaContent).toContain('Framework-Specific Guidance');
|
|
expect(qaOnlyContent).toContain('Framework-Specific Guidance');
|
|
|
|
// Both should contain the important rules
|
|
expect(qaContent).toContain('Important Rules');
|
|
expect(qaOnlyContent).toContain('Important Rules');
|
|
|
|
// Both should contain the 6 phases
|
|
expect(qaContent).toContain('Phase 1');
|
|
expect(qaOnlyContent).toContain('Phase 1');
|
|
expect(qaContent).toContain('Phase 6');
|
|
expect(qaOnlyContent).toContain('Phase 6');
|
|
});
|
|
|
|
test('qa-only has no-fix guardrails', () => {
|
|
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
|
|
expect(qaOnlyContent).toContain('Never fix bugs');
|
|
expect(qaOnlyContent).toContain('NEVER fix anything');
|
|
// Should not have Edit, Glob, or Grep in allowed-tools.
|
|
// Scope to frontmatter (between the first two --- lines) — the body can
|
|
// legitimately mention these tool names in prose (e.g., Claude model
|
|
// overlay says "prefer Read, Edit, Write, Glob, Grep over Bash").
|
|
const fmMatch = qaOnlyContent.match(/^---\n([\s\S]*?)\n---/);
|
|
expect(fmMatch).not.toBeNull();
|
|
const frontmatter = fmMatch![1];
|
|
expect(frontmatter).toMatch(/allowed-tools:/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Edit/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Glob/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Grep/);
|
|
});
|
|
|
|
test('qa has fix-loop tools and phases', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
// Should have Edit, Glob, Grep in allowed-tools
|
|
expect(qaContent).toContain('Edit');
|
|
expect(qaContent).toContain('Glob');
|
|
expect(qaContent).toContain('Grep');
|
|
// Should have fix-loop phases
|
|
expect(qaContent).toContain('Phase 7');
|
|
expect(qaContent).toContain('Phase 8');
|
|
expect(qaContent).toContain('Fix Loop');
|
|
expect(qaContent).toContain('Triage');
|
|
expect(qaContent).toContain('WTF');
|
|
});
|
|
});
|
|
|
|
describe('BASE_BRANCH_DETECT resolver', () => {
|
|
// Find a generated SKILL.md that uses the placeholder (ship is guaranteed to)
|
|
const shipContent = readShipUnion();
|
|
|
|
test('resolver output contains PR base detection command', () => {
|
|
expect(shipContent).toContain('gh pr view --json baseRefName');
|
|
});
|
|
|
|
test('resolver output contains repo default branch detection command', () => {
|
|
expect(shipContent).toContain('gh repo view --json defaultBranchRef');
|
|
});
|
|
|
|
test('resolver output contains fallback to main', () => {
|
|
expect(shipContent).toMatch(/fall\s*back\s+to\s+`main`/i);
|
|
});
|
|
|
|
test('resolver output uses "the base branch" phrasing', () => {
|
|
expect(shipContent).toContain('the base branch');
|
|
});
|
|
|
|
test('resolver output contains GitLab CLI commands', () => {
|
|
expect(shipContent).toContain('glab');
|
|
});
|
|
|
|
test('resolver output contains git-native fallback', () => {
|
|
expect(shipContent).toContain('git symbolic-ref');
|
|
});
|
|
|
|
test('resolver output mentions GitLab platform', () => {
|
|
expect(shipContent).toMatch(/gitlab/i);
|
|
});
|
|
});
|
|
|
|
describe('GitLab support in generated skills', () => {
|
|
const retroContent = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
|
const shipSkillContent = readShipUnion();
|
|
|
|
test('retro contains GitLab MR number extraction', () => {
|
|
expect(retroContent).toContain('[#!]');
|
|
});
|
|
|
|
test('retro uses BASE_BRANCH_DETECT (contains glab)', () => {
|
|
expect(retroContent).toContain('glab');
|
|
});
|
|
|
|
test('ship contains glab mr create', () => {
|
|
expect(shipSkillContent).toContain('glab mr create');
|
|
});
|
|
|
|
test('ship checks .gitlab-ci.yml', () => {
|
|
expect(shipSkillContent).toContain('.gitlab-ci.yml');
|
|
});
|
|
});
|
|
|
|
/**
|
|
* Quality evals — catch description regressions.
|
|
*
|
|
* These test that generated output is *useful for an AI agent*,
|
|
* not just structurally valid. Each test targets a specific
|
|
* regression we actually shipped and caught in review.
|
|
*/
|
|
describe('description quality evals', () => {
|
|
// Regression: snapshot flags lost value hints (-d <N>, -s <sel>, -o <path>)
|
|
test('snapshot flags with values include value hints in output', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
if (flag.takesValue) {
|
|
expect(flag.valueHint).toBeDefined();
|
|
expect(content).toContain(`${flag.short} ${flag.valueHint}`);
|
|
}
|
|
}
|
|
});
|
|
|
|
// Regression: "is" lost the valid states enum
|
|
test('is command lists valid state values', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['is'].description;
|
|
for (const state of ['visible', 'hidden', 'enabled', 'disabled', 'checked', 'editable', 'focused']) {
|
|
expect(desc).toContain(state);
|
|
}
|
|
});
|
|
|
|
// Regression: "press" lost common key examples
|
|
test('press command lists example keys', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['press'].description;
|
|
expect(desc).toContain('Enter');
|
|
expect(desc).toContain('Tab');
|
|
expect(desc).toContain('Escape');
|
|
});
|
|
|
|
// Regression: "console" lost --errors filter note
|
|
test('console command describes --errors behavior', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['console'].description;
|
|
expect(desc).toContain('--errors');
|
|
});
|
|
|
|
// Regression: snapshot -i lost "@e refs" context
|
|
test('snapshot -i mentions @e refs', () => {
|
|
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-i')!;
|
|
expect(flag.description).toContain('@e');
|
|
});
|
|
|
|
// Regression: snapshot -C lost "@c refs" context
|
|
test('snapshot -C mentions @c refs', () => {
|
|
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-C')!;
|
|
expect(flag.description).toContain('@c');
|
|
});
|
|
|
|
// Guard: every description must be at least 8 chars (catches empty or stub descriptions)
|
|
test('all command descriptions have meaningful length', () => {
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
expect(meta.description.length).toBeGreaterThanOrEqual(8);
|
|
}
|
|
});
|
|
|
|
// Guard: snapshot flag descriptions must be at least 10 chars
|
|
test('all snapshot flag descriptions have meaningful length', () => {
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
expect(flag.description.length).toBeGreaterThanOrEqual(10);
|
|
}
|
|
});
|
|
|
|
// Guard: descriptions must not contain pipe (breaks markdown table cells)
|
|
// Usage strings are backtick-wrapped in the table so pipes there are safe.
|
|
test('no command description contains pipe character', () => {
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
expect(meta.description).not.toContain('|');
|
|
}
|
|
});
|
|
|
|
// Guard: generated output uses → not ->
|
|
test('generated SKILL.md uses unicode arrows', () => {
|
|
// P2 (v1.2.0): the browse body moved out of the top-level router into
|
|
// browse/SKILL.md. Guard arrow style on the browse body (sliced from its
|
|
// H1 so the auto-generated `-->` header comments are excluded).
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
const body = content.slice(content.indexOf('# browse: QA Testing'));
|
|
expect(body).toContain('→');
|
|
expect(body).not.toContain('->');
|
|
});
|
|
});
|
|
|
|
describe('REVIEW_DASHBOARD resolver', () => {
|
|
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review'];
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`review dashboard appears in ${skill} generated file`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('gstack-review');
|
|
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
|
});
|
|
}
|
|
|
|
test('review dashboard appears in ship generated file', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('reviews.jsonl');
|
|
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
|
});
|
|
|
|
test('dashboard treats review as a valid Eng Review source', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
|
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
|
expect(content).toContain('`plan-eng-review` (plan-stage architecture review)');
|
|
expect(content).toContain('from either \\`review\\` or \\`plan-eng-review\\`');
|
|
});
|
|
|
|
test('shared dashboard propagates review source to plan-eng-review', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
|
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
|
});
|
|
|
|
test('resolver output contains key dashboard elements', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('VERDICT');
|
|
expect(content).toContain('CLEARED');
|
|
expect(content).toContain('Eng Review');
|
|
expect(content).toContain('7 days');
|
|
expect(content).toContain('Design Review');
|
|
expect(content).toContain('skip_eng_review');
|
|
});
|
|
|
|
test('dashboard bash block includes git HEAD for staleness detection', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('git rev-parse --short HEAD');
|
|
expect(content).toContain('---HEAD---');
|
|
});
|
|
|
|
test('dashboard includes staleness detection prose', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('Staleness detection');
|
|
expect(content).toContain('commit');
|
|
});
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`${skill} contains review chaining section`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('Review Chaining');
|
|
});
|
|
|
|
test(`${skill} Review Log includes commit field`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('"commit"');
|
|
});
|
|
}
|
|
|
|
test('plan-ceo-review chaining mentions eng and design reviews', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-design-review');
|
|
});
|
|
|
|
test('plan-eng-review chaining mentions design and ceo reviews', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('/plan-design-review');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
});
|
|
|
|
test('plan-design-review chaining mentions eng, ceo, and design skills', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
expect(content).toContain('/design-shotgun');
|
|
expect(content).toContain('/design-html');
|
|
});
|
|
|
|
test('ship does NOT contain review chaining', () => {
|
|
const content = readShipUnion();
|
|
expect(content).not.toContain('Review Chaining');
|
|
});
|
|
});
|
|
|
|
// ─── Test Coverage Audit Resolver Tests ─────────────────────
|
|
|
|
describe('TEST_COVERAGE_AUDIT placeholders', () => {
|
|
const planSkill = readSkillUnion('plan-eng-review'); // carved
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan and ship modes share codepath tracing methodology', () => {
|
|
// Review mode delegates test coverage to the Testing specialist subagent (Review Army)
|
|
const sharedPhrases = [
|
|
'Trace data flow',
|
|
'Diagram the execution',
|
|
'Quality scoring rubric',
|
|
'★★★',
|
|
'★★',
|
|
'GAP',
|
|
];
|
|
for (const phrase of sharedPhrases) {
|
|
expect(planSkill).toContain(phrase);
|
|
expect(shipSkill).toContain(phrase);
|
|
}
|
|
// Plan mode traces the plan, not a git diff
|
|
expect(planSkill).toContain('Trace every codepath in the plan');
|
|
expect(planSkill).not.toContain('git diff origin');
|
|
// Ship mode traces the diff
|
|
expect(shipSkill).toContain('Trace every codepath changed');
|
|
});
|
|
|
|
test('review mode uses Review Army for specialist dispatch', () => {
|
|
expect(reviewSkill).toContain('Review Army');
|
|
expect(reviewSkill).toContain('Specialist Dispatch');
|
|
expect(reviewSkill).toContain('testing.md');
|
|
});
|
|
|
|
test('plan and ship modes include E2E decision matrix', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('E2E Test Decision Matrix');
|
|
expect(skill).toContain('→E2E');
|
|
expect(skill).toContain('→EVAL');
|
|
}
|
|
});
|
|
|
|
test('plan and ship modes include regression rule', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('REGRESSION RULE');
|
|
expect(skill).toContain('IRON RULE');
|
|
}
|
|
});
|
|
|
|
test('plan and ship modes include test framework detection', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('Test Framework Detection');
|
|
expect(skill).toContain('CLAUDE.md');
|
|
}
|
|
});
|
|
|
|
test('plan mode adds tests to plan + includes test plan artifact', () => {
|
|
expect(planSkill).toContain('Add missing tests to the plan');
|
|
expect(planSkill).toContain('eng-review-test-plan');
|
|
expect(planSkill).toContain('Test Plan Artifact');
|
|
});
|
|
|
|
test('ship mode auto-generates tests + includes before/after count', () => {
|
|
expect(shipSkill).toContain('Generate tests for uncovered paths');
|
|
expect(shipSkill).toContain('Before/after test count');
|
|
expect(shipSkill).toContain('30 code paths max');
|
|
expect(shipSkill).toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review mode uses Fix-First + Review Army for specialist coverage', () => {
|
|
expect(reviewSkill).toContain('Fix-First');
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
// Review Army handles test coverage via Testing specialist subagent
|
|
expect(reviewSkill).toContain('Review Army');
|
|
expect(reviewSkill).toContain('Testing');
|
|
});
|
|
|
|
test('plan mode does NOT include ship-specific content', () => {
|
|
expect(planSkill).not.toContain('Before/after test count');
|
|
expect(planSkill).not.toContain('30 code paths max');
|
|
expect(planSkill).not.toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review mode does NOT include test plan artifact', () => {
|
|
expect(reviewSkill).not.toContain('Test Plan Artifact');
|
|
expect(reviewSkill).not.toContain('eng-review-test-plan');
|
|
expect(reviewSkill).not.toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review/specialists/ directory has all expected checklist files', () => {
|
|
const specDir = path.join(ROOT, 'review', 'specialists');
|
|
const expected = [
|
|
'testing.md',
|
|
'maintainability.md',
|
|
'security.md',
|
|
'performance.md',
|
|
'data-migration.md',
|
|
'api-contract.md',
|
|
'red-team.md',
|
|
];
|
|
for (const f of expected) {
|
|
expect(fs.existsSync(path.join(specDir, f))).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('each specialist file has standard header with scope and output format', () => {
|
|
const specDir = path.join(ROOT, 'review', 'specialists');
|
|
const files = fs.readdirSync(specDir).filter(f => f.endsWith('.md'));
|
|
for (const f of files) {
|
|
const content = fs.readFileSync(path.join(specDir, f), 'utf-8');
|
|
// All specialist files must have Scope and Output/JSON in header
|
|
expect(content).toContain('Scope:');
|
|
expect(content.toLowerCase()).toMatch(/output|json/);
|
|
// Must define NO FINDINGS behavior
|
|
expect(content).toContain('NO FINDINGS');
|
|
}
|
|
});
|
|
|
|
// Regression guard: ship output contains key phrases from before the refactor
|
|
test('ship SKILL.md regression guard — key phrases preserved', () => {
|
|
const regressionPhrases = [
|
|
'100% coverage is the goal',
|
|
'ASCII coverage diagram',
|
|
'processPayment',
|
|
'refundPayment',
|
|
'billing.test.ts',
|
|
'checkout.e2e.ts',
|
|
'COVERAGE:',
|
|
'QUALITY:',
|
|
'GAPS:',
|
|
'Code paths:',
|
|
'User flows:',
|
|
];
|
|
for (const phrase of regressionPhrases) {
|
|
expect(shipSkill).toContain(phrase);
|
|
}
|
|
});
|
|
|
|
test('ship SKILL.md contains review army specialist dispatch', () => {
|
|
expect(shipSkill).toContain('Specialist Dispatch');
|
|
expect(shipSkill).toContain('Step 9.1');
|
|
expect(shipSkill).toContain('Step 9.2');
|
|
});
|
|
|
|
test('ship SKILL.md contains cross-review finding dedup', () => {
|
|
expect(shipSkill).toContain('Cross-review finding dedup');
|
|
expect(shipSkill).toContain('Step 9.3');
|
|
});
|
|
|
|
test('ship SKILL.md contains re-run idempotency behavior', () => {
|
|
expect(shipSkill).toContain('Re-run behavior (idempotency)');
|
|
expect(shipSkill).toContain('Never skip a verification step');
|
|
});
|
|
});
|
|
|
|
// --- {{TEST_FAILURE_TRIAGE}} resolver tests ---
|
|
|
|
describe('TEST_FAILURE_TRIAGE resolver', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('contains all 4 triage steps', () => {
|
|
expect(shipSkill).toContain('Step T1: Classify each failure');
|
|
expect(shipSkill).toContain('Step T2: Handle in-branch failures');
|
|
expect(shipSkill).toContain('Step T3: Handle pre-existing failures');
|
|
expect(shipSkill).toContain('Step T4: Execute the chosen action');
|
|
});
|
|
|
|
test('T1 includes classification criteria (in-branch vs pre-existing)', () => {
|
|
expect(shipSkill).toContain('In-branch');
|
|
expect(shipSkill).toContain('Likely pre-existing');
|
|
expect(shipSkill).toContain('git diff origin/');
|
|
});
|
|
|
|
test('T3 branches on REPO_MODE (solo vs collaborative)', () => {
|
|
expect(shipSkill).toContain('REPO_MODE');
|
|
expect(shipSkill).toContain('solo');
|
|
expect(shipSkill).toContain('collaborative');
|
|
});
|
|
|
|
test('solo mode offers fix-now, TODO, and skip options', () => {
|
|
expect(shipSkill).toContain('Investigate and fix now');
|
|
expect(shipSkill).toContain('Add as P0 TODO');
|
|
expect(shipSkill).toContain('Skip');
|
|
});
|
|
|
|
test('collaborative mode offers blame + assign option', () => {
|
|
expect(shipSkill).toContain('Blame + assign GitHub issue');
|
|
expect(shipSkill).toContain('gh issue create');
|
|
});
|
|
|
|
test('defaults ambiguous failures to in-branch (safety)', () => {
|
|
expect(shipSkill).toContain('When ambiguous, default to in-branch');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_FILE_REVIEW_REPORT}} resolver tests ---
|
|
|
|
describe('PLAN_FILE_REVIEW_REPORT resolver', () => {
|
|
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review', 'codex'];
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`plan file review report appears in ${skill} generated file`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK REVIEW REPORT');
|
|
});
|
|
}
|
|
|
|
test('resolver output contains key report elements', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: report writer moved to section
|
|
expect(content).toContain('Trigger');
|
|
expect(content).toContain('Findings');
|
|
expect(content).toContain('VERDICT');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-design-review');
|
|
expect(content).toContain('/codex review');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_COMPLETION_AUDIT}} resolver tests ---
|
|
|
|
describe('PLAN_COMPLETION_AUDIT placeholders', () => {
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains plan completion audit step', () => {
|
|
expect(shipSkill).toContain('Plan Completion Audit');
|
|
expect(shipSkill).toContain('Step 8');
|
|
});
|
|
|
|
test('review SKILL.md contains plan completion in scope drift', () => {
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Actionable Item Extraction');
|
|
expect(reviewSkill).toContain('Integration with Scope Drift Detection');
|
|
});
|
|
|
|
test('both modes share plan file discovery methodology', () => {
|
|
expect(shipSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
// Both should have conversation context first
|
|
expect(shipSkill).toContain('Conversation context (primary)');
|
|
expect(reviewSkill).toContain('Conversation context (primary)');
|
|
// Both should have grep fallback
|
|
expect(shipSkill).toContain('Content-based search (fallback)');
|
|
expect(reviewSkill).toContain('Content-based search (fallback)');
|
|
});
|
|
|
|
test('ship mode has gate logic for NOT DONE items', () => {
|
|
expect(shipSkill).toContain('NOT DONE');
|
|
expect(shipSkill).toContain('Stop — implement the missing items');
|
|
expect(shipSkill).toContain('Ship anyway — defer');
|
|
expect(shipSkill).toContain('intentionally dropped');
|
|
});
|
|
|
|
test('review mode is INFORMATIONAL only', () => {
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
expect(reviewSkill).toContain('MISSING REQUIREMENTS');
|
|
expect(reviewSkill).toContain('SCOPE CREEP');
|
|
});
|
|
|
|
test('item extraction has 50-item cap', () => {
|
|
expect(shipSkill).toContain('at most 50 items');
|
|
});
|
|
|
|
test('uses file-level traceability (not commit-level)', () => {
|
|
expect(shipSkill).toContain('Cite the specific file');
|
|
expect(shipSkill).not.toContain('commit-level traceability');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_VERIFICATION_EXEC}} resolver tests ---
|
|
|
|
describe('PLAN_VERIFICATION_EXEC placeholder', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('ship SKILL.md contains plan verification step', () => {
|
|
expect(shipSkill).toContain('Step 8.1');
|
|
expect(shipSkill).toContain('Plan Verification');
|
|
});
|
|
|
|
test('references /qa-only invocation', () => {
|
|
expect(shipSkill).toContain('qa-only/SKILL.md');
|
|
expect(shipSkill).toContain('qa-only');
|
|
});
|
|
|
|
test('contains dev-server discovery (CLAUDE.md first, then a port probe)', () => {
|
|
// Fork port wave 2: the hardcoded 4-port list became read-CLAUDE.md-or-
|
|
// probe; the probe loops common ports instead of naming each once.
|
|
expect(shipSkill).toContain('CLAUDE.md first');
|
|
expect(shipSkill).toContain('http://localhost:$_p');
|
|
expect(shipSkill).toContain('NO_SERVER');
|
|
});
|
|
|
|
test('skips gracefully when no verification section', () => {
|
|
expect(shipSkill).toContain('No verification steps found in plan');
|
|
});
|
|
|
|
test('skips gracefully when no dev server', () => {
|
|
expect(shipSkill).toContain('No dev server detected');
|
|
});
|
|
});
|
|
|
|
// --- Coverage gate tests ---
|
|
|
|
describe('Coverage gate in ship', () => {
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains coverage gate with thresholds', () => {
|
|
expect(shipSkill).toContain('Coverage gate');
|
|
expect(shipSkill).toContain('>= target');
|
|
expect(shipSkill).toContain('< minimum');
|
|
});
|
|
|
|
test('ship SKILL.md supports configurable thresholds via CLAUDE.md', () => {
|
|
expect(shipSkill).toContain('## Test Coverage');
|
|
expect(shipSkill).toContain('Minimum:');
|
|
expect(shipSkill).toContain('Target:');
|
|
});
|
|
|
|
test('coverage gate skips on parse failure (not block)', () => {
|
|
expect(shipSkill).toContain('could not determine percentage — skipping');
|
|
});
|
|
|
|
test('review SKILL.md delegates coverage to Testing specialist', () => {
|
|
// Coverage audit moved to Testing specialist subagent in Review Army
|
|
expect(reviewSkill).toContain('testing.md');
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
});
|
|
});
|
|
|
|
// --- Ship metrics logging ---
|
|
|
|
describe('Ship metrics logging', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('ship SKILL.md contains metrics persistence step', () => {
|
|
expect(shipSkill).toContain('Step 20');
|
|
expect(shipSkill).toContain('coverage_pct');
|
|
expect(shipSkill).toContain('plan_items_total');
|
|
expect(shipSkill).toContain('plan_items_done');
|
|
expect(shipSkill).toContain('verification_result');
|
|
});
|
|
});
|
|
|
|
// --- Plan file discovery shared helper ---
|
|
|
|
describe('Plan file discovery shared helper', () => {
|
|
// The shared helper should appear in ship (via PLAN_COMPLETION_AUDIT_SHIP)
|
|
// and in review (via PLAN_COMPLETION_AUDIT_REVIEW)
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan file discovery appears in both ship and review', () => {
|
|
expect(shipSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
});
|
|
|
|
test('both include conversation context first', () => {
|
|
expect(shipSkill).toContain('Conversation context (primary)');
|
|
expect(reviewSkill).toContain('Conversation context (primary)');
|
|
});
|
|
|
|
test('both include content-based fallback', () => {
|
|
expect(shipSkill).toContain('Content-based search (fallback)');
|
|
expect(reviewSkill).toContain('Content-based search (fallback)');
|
|
});
|
|
});
|
|
|
|
// --- Retro plan completion ---
|
|
|
|
describe('Retro plan completion section', () => {
|
|
const retroSkill = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
|
|
|
test('retro SKILL.md contains plan completion section', () => {
|
|
expect(retroSkill).toContain('### Plan Completion');
|
|
expect(retroSkill).toContain('plan_items_total');
|
|
expect(retroSkill).toContain('Plan Completion This Period');
|
|
});
|
|
});
|
|
|
|
// --- Plan status footer in preamble ---
|
|
|
|
describe('Plan status footer in preamble', () => {
|
|
test('preamble contains plan status footer as neutral forward reference to EXIT PLAN MODE GATE', () => {
|
|
// Read any skill that uses PREAMBLE
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
expect(content).toContain('Plan Status Footer');
|
|
expect(content).toContain('GSTACK REVIEW REPORT');
|
|
expect(content).toContain('ExitPlanMode');
|
|
expect(content).toContain('EXIT PLAN MODE GATE');
|
|
// The preamble must NOT impose review-report rules on operational skills
|
|
// that have no review report. It's a forward reference, not enforcement.
|
|
expect(content).not.toContain('NO REVIEWS YET');
|
|
});
|
|
});
|
|
|
|
// --- make-pdf setup ordering ---
|
|
|
|
describe('make-pdf setup ordering', () => {
|
|
test('MAKE-PDF SETUP appears before generic preamble footer sections', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'make-pdf', 'SKILL.md'), 'utf-8');
|
|
const preambleIdx = content.indexOf('## Preamble (run first)');
|
|
const setupIdx = content.indexOf('## MAKE-PDF SETUP');
|
|
const planModeIdx = content.indexOf('## Plan Mode Safe Operations');
|
|
const telemetryIdx = content.indexOf('## Telemetry (run last)');
|
|
const workflowIdx = content.indexOf('# make-pdf: publication-quality PDFs from markdown');
|
|
|
|
expect(preambleIdx).toBeGreaterThanOrEqual(0);
|
|
expect(setupIdx).toBeGreaterThan(preambleIdx);
|
|
expect(setupIdx).toBeLessThan(planModeIdx);
|
|
expect(setupIdx).toBeLessThan(telemetryIdx);
|
|
expect(setupIdx).toBeLessThan(workflowIdx);
|
|
expect(content.match(/^## MAKE-PDF SETUP/gm)?.length ?? 0).toBe(1);
|
|
});
|
|
});
|
|
|
|
// --- Skill invocation during plan mode in preamble ---
|
|
|
|
describe('Skill invocation during plan mode in preamble', () => {
|
|
test('preamble contains skill invocation plan mode section', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
expect(content).toContain('Skill Invocation During Plan Mode');
|
|
expect(content).toContain('precedence over generic plan mode behavior');
|
|
expect(content).toContain('Do not continue the workflow');
|
|
expect(content).toContain('cancel the skill or leave plan mode');
|
|
});
|
|
});
|
|
|
|
// --- {{SPEC_REVIEW_LOOP}} resolver tests ---
|
|
|
|
describe('SPEC_REVIEW_LOOP resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('contains all 5 review dimensions', () => {
|
|
for (const dim of ['Completeness', 'Consistency', 'Clarity', 'Scope', 'Feasibility']) {
|
|
expect(content).toContain(dim);
|
|
}
|
|
});
|
|
|
|
test('references Agent tool for subagent dispatch', () => {
|
|
expect(content).toMatch(/Agent.*tool/i);
|
|
});
|
|
|
|
test('specifies max 3 iterations', () => {
|
|
expect(content).toMatch(/3.*iteration|maximum.*3/i);
|
|
});
|
|
|
|
test('includes quality score', () => {
|
|
expect(content).toContain('quality score');
|
|
});
|
|
|
|
test('includes metrics path', () => {
|
|
expect(content).toContain('spec-review.jsonl');
|
|
});
|
|
|
|
test('includes convergence guard', () => {
|
|
expect(content).toMatch(/[Cc]onvergence/);
|
|
});
|
|
|
|
test('includes graceful failure handling', () => {
|
|
expect(content).toMatch(/skip.*review|unavailable/i);
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_SKETCH}} resolver tests ---
|
|
|
|
describe('DESIGN_SKETCH resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('references DESIGN.md for design system constraints', () => {
|
|
expect(content).toContain('DESIGN.md');
|
|
});
|
|
|
|
test('contains wireframe or sketch terminology', () => {
|
|
expect(content).toMatch(/wireframe|sketch/i);
|
|
});
|
|
|
|
test('references browse binary for rendering', () => {
|
|
expect(content).toContain('$B goto');
|
|
});
|
|
|
|
test('references screenshot capture', () => {
|
|
expect(content).toContain('$B screenshot');
|
|
});
|
|
|
|
test('specifies rough aesthetic', () => {
|
|
expect(content).toMatch(/[Rr]ough|hand-drawn/);
|
|
});
|
|
|
|
test('includes skip conditions', () => {
|
|
expect(content).toMatch(/no UI component|skip/i);
|
|
});
|
|
});
|
|
|
|
// --- {{CODEX_SECOND_OPINION}} resolver tests ---
|
|
|
|
describe('CODEX_SECOND_OPINION resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
const codexContent = fs.readFileSync(path.join(ROOT, '.agents', 'skills', 'gstack-office-hours', 'SKILL.md'), 'utf-8');
|
|
|
|
test('Phase 3.5 section appears in office-hours SKILL.md', () => {
|
|
expect(content).toContain('Phase 3.5: Cross-Model Second Opinion');
|
|
});
|
|
|
|
test('contains codex exec invocation', () => {
|
|
expect(content).toContain('codex exec');
|
|
});
|
|
|
|
test('contains opt-in AskUserQuestion text', () => {
|
|
expect(content).toContain('second opinion from an independent AI perspective');
|
|
});
|
|
|
|
test('contains cross-model synthesis instructions', () => {
|
|
expect(content).toMatch(/[Ss]ynthesis/);
|
|
expect(content).toContain('Where Claude agrees with the second opinion');
|
|
});
|
|
|
|
test('contains Claude subagent fallback', () => {
|
|
expect(content).toContain('CODEX_NOT_AVAILABLE');
|
|
expect(content).toContain('Agent tool');
|
|
expect(content).toContain('SECOND OPINION (Claude subagent)');
|
|
});
|
|
|
|
test('contains premise revision check', () => {
|
|
expect(content).toContain('Codex challenged premise');
|
|
});
|
|
|
|
test('contains error handling for auth, timeout, and empty', () => {
|
|
expect(content).toMatch(/[Aa]uth.*fail/);
|
|
expect(content).toMatch(/[Tt]imeout/);
|
|
expect(content).toMatch(/[Ee]mpty response/);
|
|
});
|
|
|
|
test('Codex host variant does NOT contain the Phase 3.5 resolver output', () => {
|
|
// The resolver returns '' for codex host, so the interactive section is stripped.
|
|
// Static template references to "Phase 3.5" in prose/conditionals are fine.
|
|
// Other resolvers (design review lite) may contain CODEX_NOT_AVAILABLE, so we
|
|
// check for Phase 3.5-specific markers only.
|
|
expect(codexContent).not.toContain('Phase 3.5: Cross-Model Second Opinion');
|
|
expect(codexContent).not.toContain('TMPERR_OH');
|
|
expect(codexContent).not.toContain('gstack-codex-oh-');
|
|
});
|
|
});
|
|
|
|
// --- Codex filesystem boundary tests ---
|
|
|
|
describe('Codex filesystem boundary', () => {
|
|
// Skills that call codex exec/review and should contain boundary text
|
|
const CODEX_CALLING_SKILLS = [
|
|
'codex', // /codex skill — 3 modes
|
|
'autoplan', // /autoplan — CEO/design/eng voices
|
|
'review', // /review — adversarial step resolver
|
|
'ship', // /ship — adversarial step resolver
|
|
'plan-eng-review', // outside voice resolver
|
|
'plan-ceo-review', // outside voice resolver
|
|
'office-hours', // second opinion resolver
|
|
];
|
|
|
|
const BOUNDARY_MARKER = 'Do NOT read or execute any';
|
|
|
|
test('boundary instruction appears in all skills that call codex', () => {
|
|
for (const skill of CODEX_CALLING_SKILLS) {
|
|
// Union: ship's codex call lives in sections/adversarial.md after the carve.
|
|
const content = readSkillUnion(skill);
|
|
expect(content).toContain(BOUNDARY_MARKER);
|
|
}
|
|
});
|
|
|
|
test('codex skill has Filesystem Boundary section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('## Filesystem Boundary');
|
|
expect(content).toContain('skill definitions meant for a different AI system');
|
|
});
|
|
|
|
test('codex skill has rabbit-hole detection rule', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Detect skill-file rabbit holes');
|
|
expect(content).toContain('gstack-update-check');
|
|
expect(content).toContain('Consider retrying');
|
|
});
|
|
|
|
test('review.ts CODEX_BOUNDARY constant is interpolated into resolver output', () => {
|
|
// The adversarial step resolver should include boundary text in codex exec prompts
|
|
const reviewContent = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
// Boundary should appear near codex exec invocations
|
|
const boundaryIdx = reviewContent.indexOf(BOUNDARY_MARKER);
|
|
const codexExecIdx = reviewContent.indexOf('codex exec');
|
|
// Both must exist and boundary must come before a codex exec call
|
|
expect(boundaryIdx).toBeGreaterThan(-1);
|
|
expect(codexExecIdx).toBeGreaterThan(-1);
|
|
});
|
|
|
|
test('autoplan boundary text avoids host-specific paths for cross-host compatibility', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'autoplan', 'SKILL.md.tmpl'), 'utf-8');
|
|
// autoplan template uses generic 'skills/gstack' pattern instead of host-specific
|
|
// paths like ~/.claude/ or .agents/skills (which break Codex/Claude output tests)
|
|
const boundaryStart = content.indexOf('Filesystem Boundary');
|
|
const boundaryEnd = content.indexOf('---', boundaryStart + 1);
|
|
const boundarySection = content.slice(boundaryStart, boundaryEnd);
|
|
expect(boundarySection).not.toContain('~/.claude/');
|
|
expect(boundarySection).not.toContain('.agents/skills');
|
|
expect(boundarySection).toContain('skills/gstack');
|
|
expect(boundarySection).toContain(BOUNDARY_MARKER);
|
|
});
|
|
});
|
|
|
|
// --- {{BENEFITS_FROM}} resolver tests ---
|
|
|
|
describe('BENEFITS_FROM resolver', () => {
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
const engContent = readSkillUnion('plan-eng-review'); // carved
|
|
|
|
test('plan-ceo-review contains prerequisite skill offer', () => {
|
|
expect(ceoContent).toContain('Prerequisite Skill Offer');
|
|
expect(ceoContent).toContain('/office-hours');
|
|
});
|
|
|
|
test('plan-eng-review contains prerequisite skill offer', () => {
|
|
expect(engContent).toContain('Prerequisite Skill Offer');
|
|
expect(engContent).toContain('/office-hours');
|
|
});
|
|
|
|
test('offer includes graceful decline', () => {
|
|
expect(ceoContent).toContain('No worries');
|
|
});
|
|
|
|
test('skills without benefits-from do NOT have prerequisite offer', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
expect(qaContent).not.toContain('Prerequisite Skill Offer');
|
|
});
|
|
|
|
test('inline invocation — no "another window" language', () => {
|
|
expect(ceoContent).not.toContain('another window');
|
|
expect(engContent).not.toContain('another window');
|
|
});
|
|
|
|
test('inline invocation — read-and-follow path present', () => {
|
|
expect(ceoContent).toContain('office-hours/SKILL.md');
|
|
expect(engContent).toContain('office-hours/SKILL.md');
|
|
});
|
|
|
|
test('BENEFITS_FROM delegates to INVOKE_SKILL pattern', () => {
|
|
// Should contain the INVOKE_SKILL-style loading prose (not the old manual skip list)
|
|
expect(engContent).toContain('Follow its instructions from top to bottom');
|
|
expect(engContent).toContain('skipping these sections');
|
|
expect(ceoContent).toContain('Follow its instructions from top to bottom');
|
|
});
|
|
});
|
|
|
|
// --- {{INVOKE_SKILL}} resolver tests ---
|
|
|
|
describe('INVOKE_SKILL resolver', () => {
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan-ceo-review uses INVOKE_SKILL for mid-session office-hours fallback', () => {
|
|
// The mid-session detection path should use INVOKE_SKILL-generated prose
|
|
expect(ceoContent).toContain('office-hours/SKILL.md');
|
|
expect(ceoContent).toContain('Follow its instructions from top to bottom');
|
|
});
|
|
|
|
test('INVOKE_SKILL output includes default skip list', () => {
|
|
expect(ceoContent).toContain('Preamble (run first)');
|
|
expect(ceoContent).toContain('Telemetry (run last)');
|
|
expect(ceoContent).toContain('AskUserQuestion Format');
|
|
});
|
|
|
|
test('INVOKE_SKILL output includes error handling', () => {
|
|
expect(ceoContent).toContain('If unreadable');
|
|
expect(ceoContent).toContain('Could not load');
|
|
});
|
|
|
|
test('template uses {{INVOKE_SKILL:office-hours}} placeholder', () => {
|
|
const tmpl = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(tmpl).toContain('{{INVOKE_SKILL:office-hours}}');
|
|
});
|
|
});
|
|
|
|
// --- {{CHANGELOG_WORKFLOW}} resolver tests ---
|
|
|
|
describe('CHANGELOG_WORKFLOW resolver', () => {
|
|
const shipContent = readShipUnion();
|
|
|
|
test('ship SKILL.md contains changelog workflow', () => {
|
|
expect(shipContent).toContain('CHANGELOG (auto-generate)');
|
|
expect(shipContent).toContain('git log <base>..HEAD --oneline');
|
|
});
|
|
|
|
test('changelog workflow includes cross-check step', () => {
|
|
expect(shipContent).toContain('Cross-check');
|
|
expect(shipContent).toContain('Every commit must map to at least one bullet point');
|
|
});
|
|
|
|
test('changelog workflow includes voice guidance', () => {
|
|
expect(shipContent).toContain('Lead with what the user can now **do**');
|
|
});
|
|
|
|
test('template uses {{CHANGELOG_WORKFLOW}} placeholder', () => {
|
|
// Post-carve (T9): the skeleton points to the changelog section, which carries
|
|
// the resolver. Neither should inline the old changelog content.
|
|
const skel = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');
|
|
const changelogSection = fs.readFileSync(path.join(ROOT, 'ship', 'sections', 'changelog.md.tmpl'), 'utf-8');
|
|
expect(skel).toContain('{{SECTION:changelog}}');
|
|
expect(changelogSection).toContain('{{CHANGELOG_WORKFLOW}}');
|
|
expect(skel + changelogSection).not.toContain('Group commits by theme');
|
|
});
|
|
|
|
test('changelog workflow includes keep-changelog format', () => {
|
|
expect(shipContent).toContain('### Added');
|
|
expect(shipContent).toContain('### Fixed');
|
|
});
|
|
});
|
|
|
|
// --- Duplicate {{PREAMBLE}} guard (#2508/#2362) ---
|
|
|
|
describe('assertSinglePreamble', () => {
|
|
test('one {{PREAMBLE}} passes', () => {
|
|
expect(() => assertSinglePreamble('a\n{{PREAMBLE}}\nb', 'x/SKILL.md.tmpl')).not.toThrow();
|
|
});
|
|
|
|
test('zero {{PREAMBLE}} passes (sections have none)', () => {
|
|
expect(() => assertSinglePreamble('no macro here', 'x/sections/y.md.tmpl')).not.toThrow();
|
|
});
|
|
|
|
test('a second occurrence throws with the template path — even in prose', () => {
|
|
// The original #2508 bug WAS a prose mention: "emitted by {{PREAMBLE}}'s
|
|
// preamble bash". Resolution is context-blind, so the guard must be too.
|
|
const tmpl = '{{PREAMBLE}}\n\n...later: emitted by {{PREAMBLE}}\'s preamble bash';
|
|
expect(() => assertSinglePreamble(tmpl, 'spec/SKILL.md.tmpl')).toThrow(/spec\/SKILL\.md\.tmpl.*2 times/);
|
|
});
|
|
});
|
|
|
|
// --- Parameterized resolver infrastructure tests ---
|
|
|
|
describe('parameterized resolver support', () => {
|
|
test('gen-skill-docs regex handles colon-separated args', () => {
|
|
// Verify the template containing {{INVOKE_SKILL:office-hours}} was processed
|
|
// without leaving unresolved placeholders
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(ceoContent).not.toMatch(/\{\{INVOKE_SKILL:[^}]+\}\}/);
|
|
});
|
|
|
|
test('templates with parameterized resolvers pass unresolved check', () => {
|
|
// All generated SKILL.md files should have no unresolved {{...}} placeholders
|
|
const skillDirs = fs.readdirSync(ROOT).filter(d =>
|
|
fs.existsSync(path.join(ROOT, d, 'SKILL.md'))
|
|
);
|
|
for (const dir of skillDirs) {
|
|
const content = fs.readFileSync(path.join(ROOT, dir, 'SKILL.md'), 'utf-8');
|
|
const unresolved = content.match(/\{\{[A-Z_]+(?::[^}]*)?\}\}/g);
|
|
if (unresolved) {
|
|
throw new Error(`${dir}/SKILL.md has unresolved placeholders: ${unresolved.join(', ')}`);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
// --- Preamble routing injection tests ---
|
|
|
|
describe('preamble routing injection', () => {
|
|
const shipContent = readShipUnion();
|
|
|
|
test('preamble bash checks for routing section in CLAUDE.md and AGENTS.md', () => {
|
|
// #2500: the probe iterates CLAUDE.md AND AGENTS.md — non-Claude hosts
|
|
// route skills via AGENTS.md, the cross-harness convention file.
|
|
expect(shipContent).toContain('for _RF in CLAUDE.md AGENTS.md');
|
|
expect(shipContent).toContain('grep -q "## Skill routing" "$_RF"');
|
|
expect(shipContent).toContain('HAS_ROUTING');
|
|
});
|
|
|
|
test('preamble bash reads routing_declined config', () => {
|
|
expect(shipContent).toContain('routing_declined');
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
});
|
|
|
|
test('preamble includes routing injection AskUserQuestion', () => {
|
|
expect(shipContent).toContain('Add routing rules to CLAUDE.md');
|
|
expect(shipContent).toContain("I'll invoke skills manually");
|
|
});
|
|
|
|
test('routing injection respects prior decline', () => {
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
expect(shipContent).toMatch(/routing_declined.*true/);
|
|
});
|
|
|
|
test('routing injection only fires when all conditions met', () => {
|
|
// Must be: HAS_ROUTING=no AND ROUTING_DECLINED=false AND PROACTIVE_PROMPTED=yes
|
|
expect(shipContent).toContain('HAS_ROUTING');
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
expect(shipContent).toContain('PROACTIVE_PROMPTED');
|
|
});
|
|
|
|
test('routing section content includes key routing rules', () => {
|
|
expect(shipContent).toContain('invoke /office-hours');
|
|
expect(shipContent).toContain('invoke /investigate');
|
|
expect(shipContent).toContain('invoke /ship');
|
|
expect(shipContent).toContain('invoke /qa');
|
|
});
|
|
|
|
test('routing section uses renamed checkpoint skills (not stale /checkpoint)', () => {
|
|
expect(shipContent).toContain('invoke /context-save');
|
|
expect(shipContent).toContain('invoke /context-restore');
|
|
expect(shipContent).not.toContain('invoke checkpoint');
|
|
});
|
|
|
|
test('routing section uses soft "when in doubt" policy, not hard "ALWAYS invoke"', () => {
|
|
expect(shipContent).toContain('When in doubt, invoke the skill');
|
|
expect(shipContent).not.toContain('Do NOT answer directly');
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_OUTSIDE_VOICES}} resolver tests ---
|
|
|
|
describe('DESIGN_OUTSIDE_VOICES resolver', () => {
|
|
test('plan-design-review contains outside voices section', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('CODEX_AVAILABLE');
|
|
expect(content).toContain('LITMUS SCORECARD');
|
|
});
|
|
|
|
test('design-review contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('source audit');
|
|
});
|
|
|
|
test('design-consultation contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('design direction');
|
|
});
|
|
|
|
test('branches correctly per skillName — different prompts', () => {
|
|
const planContent = readSkillUnion('plan-design-review');
|
|
const consultContent = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
|
|
// plan-design-review uses analytical prompt (high reasoning)
|
|
expect(planContent).toContain('model_reasoning_effort="high"');
|
|
// design-consultation uses creative prompt (medium reasoning)
|
|
expect(consultContent).toContain('model_reasoning_effort="medium"');
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_HARD_RULES}} resolver tests ---
|
|
|
|
describe('DESIGN_HARD_RULES resolver', () => {
|
|
test('plan-design-review Pass 4 contains hard rules', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Design Hard Rules');
|
|
expect(content).toContain('Classifier');
|
|
expect(content).toContain('MARKETING/LANDING PAGE');
|
|
expect(content).toContain('APP UI');
|
|
});
|
|
|
|
test('design-review contains hard rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Hard Rules');
|
|
});
|
|
|
|
test('includes all 3 rule sets', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Landing page rules');
|
|
expect(content).toContain('App UI rules');
|
|
expect(content).toContain('Universal rules');
|
|
});
|
|
|
|
test('references shared AI slop blacklist items', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('3-column feature grid');
|
|
expect(content).toContain('Purple/violet/indigo');
|
|
});
|
|
|
|
test('includes OpenAI hard rejection criteria', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Generic SaaS card grid');
|
|
expect(content).toContain('Carousel with no narrative purpose');
|
|
});
|
|
|
|
test('includes OpenAI litmus checks', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Brand/product unmistakable');
|
|
expect(content).toContain('premium with all decorative shadows removed');
|
|
});
|
|
});
|
|
|
|
// --- Extended DESIGN_SKETCH resolver tests ---
|
|
|
|
describe('DESIGN_SKETCH extended with outside voices', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('contains outside design voices step', () => {
|
|
expect(content).toContain('Outside design voices');
|
|
});
|
|
|
|
test('offers opt-in via AskUserQuestion', () => {
|
|
expect(content).toContain('outside design perspectives');
|
|
});
|
|
|
|
test('still contains original wireframe steps', () => {
|
|
expect(content).toContain('wireframe');
|
|
expect(content).toContain('$B goto');
|
|
});
|
|
});
|
|
|
|
// --- Extended DESIGN_REVIEW_LITE resolver tests ---
|
|
|
|
describe('DESIGN_REVIEW_LITE extended with Codex', () => {
|
|
const content = readShipUnion();
|
|
|
|
test('contains Codex design voice block', () => {
|
|
expect(content).toContain('Codex design voice');
|
|
expect(content).toContain('CODEX (design)');
|
|
});
|
|
|
|
test('still contains original checklist steps', () => {
|
|
expect(content).toContain('design-checklist.md');
|
|
expect(content).toContain('SCOPE_FRONTEND');
|
|
});
|
|
|
|
});
|
|
|
|
// ─── Codex Generation Tests ─────────────────────────────────
|
|
|
|
describe('Codex generation (--host codex)', () => {
|
|
const AGENTS_DIR = path.join(ROOT, '.agents', 'skills');
|
|
|
|
// .agents/ is gitignored (v0.11.2.0) — generate on demand for tests
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
|
|
// Dynamic discovery of expected Codex skills: all templates except /codex
|
|
// Also excludes skills where .agents/skills/{name} is a symlink back to the repo root
|
|
// (vendored dev mode — gen-skill-docs skips these to avoid overwriting Claude SKILL.md)
|
|
const CODEX_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; codexName: string }> = [];
|
|
const isSymlinkLoop = (codexName: string): boolean => {
|
|
const agentSkillDir = path.join(ROOT, '.agents', 'skills', codexName);
|
|
try {
|
|
return fs.realpathSync(agentSkillDir) === fs.realpathSync(ROOT);
|
|
} catch { return false; }
|
|
};
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
if (!isSymlinkLoop('gstack')) {
|
|
skills.push({ dir: '.', codexName: 'gstack' });
|
|
}
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (entry.name === 'codex') continue; // /codex is excluded from Codex output
|
|
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
|
const codexName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
|
if (isSymlinkLoop(codexName)) continue;
|
|
skills.push({ dir: entry.name, codexName });
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
test('--host codex generates correct output paths', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const skillMd = path.join(AGENTS_DIR, skill.codexName, 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('root gstack bundle has OpenAI metadata for Codex skill browsing', () => {
|
|
const rootMetadata = path.join(ROOT, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(rootMetadata)).toBe(true);
|
|
const content = fs.readFileSync(rootMetadata, 'utf-8');
|
|
expect(content).toContain('display_name: "gstack"');
|
|
expect(content).toContain('Use $gstack to locate the bundled gstack skills.');
|
|
expect(content).toContain('allow_implicit_invocation: true');
|
|
});
|
|
|
|
test('externalSkillName mapping: root is gstack, others are gstack-{dir}', () => {
|
|
// Root → gstack
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack', 'SKILL.md'))).toBe(true);
|
|
// Subdirectories → gstack-{dir}
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'))).toBe(true);
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'))).toBe(true);
|
|
// gstack-upgrade doesn't double-prefix
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-upgrade', 'SKILL.md'))).toBe(true);
|
|
// No double-prefix: gstack-gstack-upgrade must NOT exist
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-gstack-upgrade', 'SKILL.md'))).toBe(false);
|
|
});
|
|
|
|
test('Codex frontmatter has ONLY name + description', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content.startsWith('---\n')).toBe(true);
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
expect(fmEnd).toBeGreaterThan(0);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
// Must have name and description
|
|
expect(frontmatter).toContain('name:');
|
|
expect(frontmatter).toContain('description:');
|
|
// Must NOT have allowed-tools, version, or hooks
|
|
expect(frontmatter).not.toContain('allowed-tools:');
|
|
expect(frontmatter).not.toContain('version:');
|
|
expect(frontmatter).not.toContain('hooks:');
|
|
}
|
|
});
|
|
|
|
test('all Codex skills have agents/openai.yaml metadata', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const metadata = path.join(AGENTS_DIR, skill.codexName, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(metadata)).toBe(true);
|
|
const content = fs.readFileSync(metadata, 'utf-8');
|
|
expect(content).toContain(`display_name: "${skill.codexName}"`);
|
|
expect(content).toContain('short_description:');
|
|
expect(content).toContain('allow_implicit_invocation: true');
|
|
}
|
|
});
|
|
|
|
test('no .claude/skills/ in Codex output', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('no ~/.claude/ paths in Codex output', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('~/.claude/');
|
|
}
|
|
});
|
|
|
|
test('/codex skill excluded from Codex output', () => {
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex'))).toBe(false);
|
|
});
|
|
|
|
test('Codex output includes Claude outside-voice skill with read-only boundary', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-claude', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('claude -p');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-prompt-');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-response-XXXXXX');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-error-XXXXXX');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-diff-');
|
|
expect(content).not.toMatch(/gstack-claude-(?:prompt|response|error|diff)-X{6,}\.\w+/);
|
|
expect(content).not.toContain('/tmp/gstack-claude-diff-$$');
|
|
expect(content).toContain('cat "$PROMPT_FILE" | "$CLAUDE_BIN" -p');
|
|
expect(content).toContain('Resolve the binary and invoke it in the same host execution context');
|
|
expect(content).toContain('--disable-slash-commands');
|
|
expect(content).toContain('--tools ""');
|
|
expect(content).toContain('--allowedTools Read,Grep,Glob');
|
|
expect(content).toContain('--disallowedTools Bash,Edit,Write');
|
|
expect(content).toContain('Do not infer authentication state from credential files');
|
|
expect(content).toContain('run the actual `claude -p`');
|
|
expect(content).not.toContain('AUTH_MISSING');
|
|
expect(content).not.toContain('$HOME/.claude/.credentials.json');
|
|
expect(content).toContain('is_error');
|
|
});
|
|
|
|
test('Claude temp file templates are accepted by host mktemp', () => {
|
|
for (const template of [
|
|
'/tmp/gstack-claude-prompt-XXXXXX',
|
|
'/tmp/gstack-claude-response-XXXXXX',
|
|
'/tmp/gstack-claude-error-XXXXXX',
|
|
'/tmp/gstack-claude-diff-XXXXXX',
|
|
]) {
|
|
const result = spawnSync('mktemp', [template], { encoding: 'utf-8' });
|
|
expect(result.status).toBe(0);
|
|
const created = result.stdout.trim();
|
|
expect(created.startsWith(template.replace('XXXXXX', ''))).toBe(true);
|
|
fs.unlinkSync(created);
|
|
}
|
|
});
|
|
|
|
test('Codex review step stripped from Codex-host ship and review', () => {
|
|
const shipContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(shipContent).not.toContain('codex review --base');
|
|
expect(shipContent).not.toContain('CODEX_REVIEWS');
|
|
|
|
const reviewContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(reviewContent).not.toContain('codex review --base');
|
|
expect(reviewContent).not.toContain('CODEX_REVIEWS');
|
|
});
|
|
|
|
test('--host codex --dry-run freshness', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// Every Codex skill should be FRESH
|
|
for (const skill of CODEX_SKILLS) {
|
|
expect(output).toContain(`FRESH: .agents/skills/${skill.codexName}/SKILL.md`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('--host agents alias produces same output as --host codex', () => {
|
|
const codexResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
const agentsResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'agents', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(codexResult.exitCode).toBe(0);
|
|
expect(agentsResult.exitCode).toBe(0);
|
|
// Both should produce the same output (same FRESH lines)
|
|
expect(codexResult.stdout.toString()).toBe(agentsResult.stdout.toString());
|
|
});
|
|
|
|
test('multiline descriptions preserved in Codex output', () => {
|
|
// office-hours has a multiline description — verify it survives the frontmatter transform
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-office-hours', 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
// Description should span multiple lines (block scalar)
|
|
const descLines = frontmatter.split('\n').filter(l => l.startsWith(' '));
|
|
expect(descLines.length).toBeGreaterThan(1);
|
|
// Verify key phrases survived
|
|
expect(frontmatter).toContain('YC Office Hours');
|
|
});
|
|
|
|
test('hook skills have safety prose and no hooks: in frontmatter', () => {
|
|
const HOOK_SKILLS = ['gstack-careful', 'gstack-freeze', 'gstack-guard'];
|
|
for (const skillName of HOOK_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skillName, 'SKILL.md'), 'utf-8');
|
|
// Must have safety advisory prose
|
|
expect(content).toContain('Safety Advisory');
|
|
// Must NOT have hooks: in frontmatter
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).not.toContain('hooks:');
|
|
}
|
|
});
|
|
|
|
test('all Codex SKILL.md files have auto-generated header', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
}
|
|
});
|
|
|
|
test('Codex preamble resolves runtime assets from repo-local or global gstack roots', () => {
|
|
// Check a skill that has a preamble (review is a good candidate)
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT');
|
|
expect(content).toContain('$_ROOT/.agents/skills/gstack');
|
|
expect(content).toContain('$GSTACK_BIN/gstack-config');
|
|
expect(content).toContain('$GSTACK_ROOT/gstack-upgrade/SKILL.md');
|
|
expect(content).not.toContain('~/.codex/skills/gstack/bin/gstack-config get telemetry');
|
|
});
|
|
|
|
// ─── Path rewriting regression tests ─────────────────────────
|
|
|
|
test('sidecar paths resolve through $GSTACK_ROOT (not gstack-review/)', () => {
|
|
// #2518: templates now anchor sidecars at the installed skill root
|
|
// (~/.claude/skills/gstack/review/...), which the codex path rewrite turns
|
|
// into $GSTACK_ROOT/review/... — resolved by the preamble against the
|
|
// repo-local .agents root or the global install. The old repo-relative
|
|
// form (.claude/skills/review/) only resolved inside gstack's own checkout.
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('$GSTACK_ROOT/review/checklist.md');
|
|
// design-checklist.md is now referenced via Review Army specialist (Claude only, stripped for Codex)
|
|
// Wrong: must NOT reference gstack-review/checklist.md (file doesn't exist there)
|
|
expect(content).not.toContain('.agents/skills/gstack-review/checklist.md');
|
|
});
|
|
|
|
test('sidecar paths in ship skill point to gstack/review/ for pre-landing review', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
// Ship references the review checklist in its pre-landing review step
|
|
if (content.includes('checklist.md')) {
|
|
expect(content).toContain('.agents/skills/gstack/review/');
|
|
expect(content).not.toContain('.agents/skills/gstack-review/checklist');
|
|
}
|
|
});
|
|
|
|
test('greptile-triage sidecar path is correct', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
if (content.includes('greptile-triage')) {
|
|
expect(content).toContain('$GSTACK_ROOT/review/greptile-triage.md');
|
|
expect(content).not.toContain('.agents/skills/gstack-review/greptile-triage');
|
|
}
|
|
});
|
|
|
|
test('all four path rewrite rules produce correct output', () => {
|
|
// Test each of the 4 path rewrite rules individually
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
|
|
// Rule 1: ~/.claude/skills/gstack → $GSTACK_ROOT
|
|
expect(content).not.toContain('~/.claude/skills/gstack');
|
|
expect(content).toContain('$GSTACK_ROOT');
|
|
|
|
// Rule 2: .claude/skills/gstack → .agents/skills/gstack
|
|
expect(content).not.toContain('.claude/skills/gstack');
|
|
|
|
// Rule 3: .claude/skills/review → .agents/skills/gstack/review
|
|
expect(content).not.toContain('.claude/skills/review');
|
|
|
|
// Rule 4: .claude/skills → .agents/skills (catch-all)
|
|
expect(content).not.toContain('.claude/skills');
|
|
});
|
|
|
|
test('path rewrite rules apply to all Codex skills with sidecar references', () => {
|
|
// Verify across ALL generated skills, not just review
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
// No skill should reference Claude paths
|
|
expect(content).not.toContain('~/.claude/skills');
|
|
expect(content).not.toContain('.claude/skills');
|
|
if (content.includes('gstack-config') || content.includes('gstack-update-check') || content.includes('gstack-telemetry-log')) {
|
|
expect(content).toContain('$GSTACK_ROOT');
|
|
}
|
|
// If a skill references checklist.md, it must use the correct sidecar path
|
|
if (content.includes('checklist.md') && !content.includes('design-checklist.md')) {
|
|
expect(content).not.toContain('gstack-review/checklist.md');
|
|
}
|
|
}
|
|
});
|
|
|
|
// ─── Claude output regression guard ─────────────────────────
|
|
|
|
test('Claude output uses installed-root review paths (#2518)', () => {
|
|
// Codex changes must NOT affect Claude output; the Claude form is the
|
|
// installed-root anchor, not the old repo-relative path that only
|
|
// resolved inside gstack's own checkout.
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('~/.claude/skills/gstack/review/checklist.md');
|
|
expect(content).toContain('~/.claude/skills/gstack');
|
|
// Must NOT contain Codex HOST paths. `~/.codex/sessions/` is exempt: the
|
|
// timeout-wrapper guidance documents the Codex CLI's own rollout-log
|
|
// location (a user-facing CLI path, same class as ~/.codex/logs/ in the
|
|
// codex skill), not the gstack Codex host install path.
|
|
// `~/.codex/config.toml` is the same user-facing class: the shared
|
|
// codexPreflight's model_unusable branch (#2477) points at the CLI's own
|
|
// config file, where the rejected `model =` pin lives.
|
|
expect(content).not.toContain('.agents/skills');
|
|
expect(
|
|
content
|
|
.replaceAll('~/.codex/sessions/', '')
|
|
.replaceAll('~/.codex/config.toml', ''),
|
|
).not.toContain('~/.codex/');
|
|
});
|
|
|
|
test('Claude output unchanged: ship skill still uses .claude/skills/ paths', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('~/.claude/skills/gstack');
|
|
expect(content).not.toContain('.agents/skills');
|
|
// ~/.codex/sessions/ is the Codex CLI's rollout-log path (user-facing),
|
|
// documented by the adversarial-pass timeout guidance; ~/.codex/config.toml
|
|
// is the CLI's own config file (model_unusable guidance, #2477) — see the
|
|
// review test above.
|
|
expect(
|
|
content
|
|
.replaceAll('~/.codex/sessions/', '')
|
|
.replaceAll('~/.codex/config.toml', ''),
|
|
).not.toContain('~/.codex/');
|
|
});
|
|
|
|
test('Claude output unchanged: all Claude skills have zero Codex paths', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
// pair-agent legitimately documents how Codex agents store credentials.
|
|
// codex + autoplan document the Codex CLI auth file (~/.codex/auth.json)
|
|
// and log path (~/.codex/logs/) — those are user-facing Codex CLI paths,
|
|
// not the gstack Codex host install path. ~/.codex/sessions/ (rollout
|
|
// logs, referenced by the review/ship timeout guidance) and
|
|
// ~/.codex/config.toml (the model_unusable guidance in the shared
|
|
// codexPreflight, #2477) are the same user-facing class, so they are
|
|
// scrubbed before the ban.
|
|
if (skill.dir !== 'pair-agent' && skill.dir !== 'codex' && skill.dir !== 'autoplan') {
|
|
expect(
|
|
content
|
|
.replaceAll('~/.codex/sessions/', '')
|
|
.replaceAll('~/.codex/config.toml', ''),
|
|
).not.toContain('~/.codex/');
|
|
}
|
|
// gstack-upgrade legitimately references .agents/skills for cross-platform detection
|
|
if (skill.dir !== 'gstack-upgrade') {
|
|
expect(content).not.toContain('.agents/skills');
|
|
}
|
|
}
|
|
});
|
|
|
|
// ─── Design outside voices: Codex host guard ─────────────────
|
|
|
|
test('codex host produces empty outside voices in design-review', () => {
|
|
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(codexContent).not.toContain('Design Outside Voices');
|
|
});
|
|
|
|
test('codex host does not include Codex design block in ship', () => {
|
|
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(codexContent).not.toContain('Codex design voice');
|
|
});
|
|
|
|
// ─── Explicit --model override wins over the host default ────
|
|
// Without --model the codex host renders its defaultModel (gpt) — pinned by
|
|
// the golden test. This pins the OTHER direction through the real CLI:
|
|
// `./setup --host codex --model <id>` depends on it. Runs last in this
|
|
// describe and restores the host-default render before finishing.
|
|
test('explicit --model overrides the codex host default', () => {
|
|
try {
|
|
const override = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--model', 'claude'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(override.exitCode).toBe(0);
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Model-Specific Behavioral Patch (claude)');
|
|
expect(content).toContain('MODEL_OVERLAY: claude');
|
|
} finally {
|
|
// Restore the host-default render — later tests and the host-config
|
|
// golden read this tree.
|
|
const restore = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(restore.exitCode).toBe(0);
|
|
}
|
|
const restored = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(restored).toContain('Model-Specific Behavioral Patch (gpt)');
|
|
});
|
|
});
|
|
|
|
// ─── Factory generation tests ────────────────────────────────
|
|
|
|
describe('Factory generation (--host factory)', () => {
|
|
const FACTORY_DIR = path.join(ROOT, '.factory', 'skills');
|
|
|
|
// Generate Factory output for tests
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
|
|
const FACTORY_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; factoryName: string }> = [];
|
|
const isSymlinkLoop = (name: string): boolean => {
|
|
const factorySkillDir = path.join(ROOT, '.factory', 'skills', name);
|
|
try { return fs.realpathSync(factorySkillDir) === fs.realpathSync(ROOT); }
|
|
catch { return false; }
|
|
};
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
if (!isSymlinkLoop('gstack')) skills.push({ dir: '.', factoryName: 'gstack' });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (entry.name === 'codex') continue;
|
|
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
|
const factoryName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
|
if (isSymlinkLoop(factoryName)) continue;
|
|
skills.push({ dir: entry.name, factoryName });
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
test('--host factory generates correct output paths', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const skillMd = path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('Factory frontmatter has name + description + user-invocable', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).toContain('name:');
|
|
expect(frontmatter).toContain('description:');
|
|
expect(frontmatter).toContain('user-invocable: true');
|
|
expect(frontmatter).not.toContain('allowed-tools:');
|
|
expect(frontmatter).not.toContain('preamble-tier:');
|
|
expect(frontmatter).not.toContain('sensitive:');
|
|
}
|
|
});
|
|
|
|
test('sensitive skills have disable-model-invocation', () => {
|
|
const SENSITIVE = ['gstack-ship', 'gstack-land-and-deploy', 'gstack-guard', 'gstack-careful', 'gstack-freeze', 'gstack-unfreeze'];
|
|
for (const name of SENSITIVE) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).toContain('disable-model-invocation: true');
|
|
}
|
|
});
|
|
|
|
test('non-sensitive skills lack disable-model-invocation', () => {
|
|
const NON_SENSITIVE = ['gstack-qa', 'gstack-review', 'gstack-investigate', 'gstack-browse'];
|
|
for (const name of NON_SENSITIVE) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).not.toContain('disable-model-invocation');
|
|
}
|
|
});
|
|
|
|
test('no .claude/skills/ in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('no ~/.claude/skills/ paths in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
// ~/.claude/skills should be rewritten, but ~/.claude/plans is legitimate
|
|
// (plan directory lookup) and ~/.claude/ in codex prompts is intentional
|
|
expect(content).not.toContain('~/.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('/codex skill excluded from Factory output', () => {
|
|
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex'))).toBe(false);
|
|
});
|
|
|
|
test('Factory keeps Codex integration blocks', () => {
|
|
// Factory users CAN use Codex second opinions (codex exec is a standalone binary)
|
|
const shipContent = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(shipContent).toContain('codex');
|
|
});
|
|
|
|
test('no agents/openai.yaml in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const yamlPath = path.join(FACTORY_DIR, skill.factoryName, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(yamlPath)).toBe(false);
|
|
}
|
|
});
|
|
|
|
test('--host droid alias works', () => {
|
|
const factoryResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
const droidResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'droid', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(factoryResult.exitCode).toBe(0);
|
|
expect(droidResult.exitCode).toBe(0);
|
|
expect(factoryResult.stdout.toString()).toBe(droidResult.stdout.toString());
|
|
});
|
|
|
|
test('--host factory --dry-run freshness', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
for (const skill of FACTORY_SKILLS) {
|
|
expect(output).toContain(`FRESH: .factory/skills/${skill.factoryName}/SKILL.md`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('Factory preamble uses .factory paths', () => {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT');
|
|
expect(content).toContain('$_ROOT/.factory/skills/gstack');
|
|
expect(content).toContain('$GSTACK_BIN/gstack-config');
|
|
});
|
|
});
|
|
|
|
// ─── Parameterized host smoke tests (config-driven) ─────────
|
|
|
|
import { ALL_HOST_CONFIGS, getExternalHosts } from '../hosts/index';
|
|
|
|
describe('Parameterized host smoke tests', () => {
|
|
// Regenerate every external host up front so the per-host `--dry-run` freshness
|
|
// checks are deterministic. These host dirs (.agents/.factory/.cursor/...) are
|
|
// gitignored regenerated artifacts, so the freshness check is really an
|
|
// idempotency/determinism check — it still catches non-deterministic gen, but no
|
|
// longer flakes on stale-on-disk state left by a missing `gen --host all` prestep
|
|
// (the canonical `bun test` does not run one). The tracked-claude freshness test
|
|
// (`generated files are fresh`) runs earlier and is unaffected.
|
|
beforeAll(() => {
|
|
for (const h of getExternalHosts()) {
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', h.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
});
|
|
|
|
for (const hostConfig of getExternalHosts()) {
|
|
describe(`${hostConfig.displayName} (--host ${hostConfig.name})`, () => {
|
|
const hostDir = path.join(ROOT, hostConfig.hostSubdir, 'skills');
|
|
|
|
test('generates output that exists on disk', () => {
|
|
// Generated dir should exist (created by earlier bun run gen:skill-docs --host all)
|
|
if (!fs.existsSync(hostDir)) {
|
|
// Generate if not already done
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
expect(fs.existsSync(hostDir)).toBe(true);
|
|
const skills = fs.readdirSync(hostDir).filter(d =>
|
|
fs.existsSync(path.join(hostDir, d, 'SKILL.md'))
|
|
);
|
|
expect(skills.length).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('no .claude/skills path leakage outside repo-root sidecar symlinks', () => {
|
|
if (!fs.existsSync(hostDir)) return; // skip if not generated
|
|
const skills = fs.readdirSync(hostDir);
|
|
for (const skill of skills) {
|
|
// Dev installs may mount the repo root at host/skills/gstack as a runtime
|
|
// sidecar. The generator skips that symlink loop, so leakage checks should too.
|
|
if (isRepoRootSymlink(path.join(hostDir, skill))) continue;
|
|
const skillMd = path.join(hostDir, skill, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
// Strip bash blocks (which have legitimate fallback paths)
|
|
const noBash = content.replace(/```bash\n[\s\S]*?```/g, '');
|
|
const leaks = noBash.split('\n').filter(l => l.includes('.claude/skills'));
|
|
if (leaks.length > 0) {
|
|
throw new Error(`${skill}: .claude/skills leakage:\n${leaks.slice(0, 3).join('\n')}`);
|
|
}
|
|
}
|
|
});
|
|
|
|
test('frontmatter has name and description', () => {
|
|
if (!fs.existsSync(hostDir)) return;
|
|
const skills = fs.readdirSync(hostDir);
|
|
for (const skill of skills) {
|
|
const skillMd = path.join(hostDir, skill, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content).toMatch(/^---\n/);
|
|
expect(content).toMatch(/^name:\s/m);
|
|
expect(content).toMatch(/^description:\s/m);
|
|
}
|
|
});
|
|
|
|
test('--dry-run freshness check passes', () => {
|
|
const result = Bun.spawnSync(
|
|
['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name, '--dry-run'],
|
|
{ cwd: ROOT, stdout: 'pipe', stderr: 'pipe' }
|
|
);
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
if (hostConfig.generation.skipSkills?.includes('codex')) {
|
|
test('/codex skill excluded', () => {
|
|
expect(fs.existsSync(path.join(hostDir, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
});
|
|
}
|
|
});
|
|
}
|
|
});
|
|
|
|
// ─── --host all tests ────────────────────────────────────────
|
|
|
|
describe('--host all', () => {
|
|
// Same determinism guard as the parameterized block: make external hosts fresh on
|
|
// disk so `--host all --dry-run` reports FRESH regardless of prior state.
|
|
beforeAll(() => {
|
|
for (const h of getExternalHosts()) {
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', h.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
});
|
|
|
|
test('--host all generates for all registered hosts', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'all', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// All hosts should appear in output
|
|
expect(output).toContain('FRESH: SKILL.md'); // claude
|
|
for (const hostConfig of getExternalHosts()) {
|
|
expect(output).toContain(`FRESH: ${hostConfig.hostSubdir}/skills/`);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── Setup script validation ─────────────────────────────────
|
|
// These tests verify the setup script's install layout matches
|
|
// what the generator produces — catching the bug where setup
|
|
// installed Claude-format source dirs for Codex users.
|
|
|
|
describe('setup script validation', () => {
|
|
const setupContent = fs.readFileSync(path.join(ROOT, 'setup'), 'utf-8');
|
|
|
|
test('setup has separate link functions for Claude and Codex', () => {
|
|
expect(setupContent).toContain('link_claude_skill_dirs');
|
|
expect(setupContent).toContain('link_codex_skill_dirs');
|
|
// Old unified function must not exist
|
|
expect(setupContent).not.toMatch(/^link_skill_dirs\(\)/m);
|
|
});
|
|
|
|
test('Claude install uses link_claude_skill_dirs', () => {
|
|
// The Claude install section (section 4) should use the Claude function
|
|
const claudeSection = setupContent.slice(
|
|
setupContent.indexOf('# 4. Install for Claude'),
|
|
setupContent.indexOf('# 5. Install for Codex')
|
|
);
|
|
expect(claudeSection).toContain('link_claude_skill_dirs');
|
|
expect(claudeSection).not.toContain('link_codex_skill_dirs');
|
|
});
|
|
|
|
test('Codex install uses link_codex_skill_dirs', () => {
|
|
// The Codex install section (section 5) should use the Codex function
|
|
// End marker: the next numbered section header (a marker that doesn't
|
|
// exist slices to EOF and the assertion reads unrelated sections).
|
|
const codexSection = setupContent.slice(
|
|
setupContent.indexOf('# 5. Install for Codex'),
|
|
setupContent.indexOf('# 6. Install for Kiro')
|
|
);
|
|
expect(setupContent.indexOf('# 6. Install for Kiro')).toBeGreaterThan(-1);
|
|
expect(codexSection).toContain('create_codex_runtime_root');
|
|
expect(codexSection).toContain('link_codex_skill_dirs');
|
|
expect(codexSection).not.toContain('link_claude_skill_dirs');
|
|
expect(codexSection).not.toContain('_link_or_copy "$GSTACK_DIR" "$CODEX_GSTACK"');
|
|
});
|
|
|
|
test('Codex install prefers repo-local .agents/skills when setup runs from there', () => {
|
|
expect(setupContent).toContain('SKILLS_PARENT_BASENAME');
|
|
expect(setupContent).toContain('CODEX_REPO_LOCAL=0');
|
|
expect(setupContent).toContain('[ "$SKILLS_PARENT_BASENAME" = ".agents" ]');
|
|
expect(setupContent).toContain('CODEX_REPO_LOCAL=1');
|
|
expect(setupContent).toContain('CODEX_SKILLS="$INSTALL_SKILLS_DIR"');
|
|
});
|
|
|
|
test('setup separates install path from source path for symlinked repo-local installs', () => {
|
|
expect(setupContent).toContain('INSTALL_GSTACK_DIR=');
|
|
expect(setupContent).toContain('SOURCE_GSTACK_DIR=');
|
|
expect(setupContent).toContain('INSTALL_SKILLS_DIR=');
|
|
expect(setupContent).toContain('CODEX_GSTACK="$INSTALL_GSTACK_DIR"');
|
|
expect(setupContent).toContain('link_codex_skill_dirs "$SOURCE_GSTACK_DIR" "$CODEX_SKILLS"');
|
|
});
|
|
|
|
test('Codex installs always create sidecar runtime assets for the real skill target', () => {
|
|
expect(setupContent).toContain('if [ "$INSTALL_CODEX" -eq 1 ]; then');
|
|
expect(setupContent).toContain('create_agents_sidecar "$SOURCE_GSTACK_DIR"');
|
|
});
|
|
|
|
test('link_codex_skill_dirs reads from .agents/skills/', () => {
|
|
// The Codex link function must reference .agents/skills for generated Codex skills
|
|
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('.agents/skills');
|
|
expect(fnBody).toContain('gstack*');
|
|
});
|
|
|
|
test('link_claude_skill_dirs creates real directories with absolute SKILL.md symlinks', () => {
|
|
// Claude links should be real directories with absolute SKILL.md symlinks
|
|
// to ensure Claude Code discovers them as top-level skills (not nested under gstack/)
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('mkdir -p "$target"');
|
|
// v1.36.0.0: routes through _link_or_copy helper for Windows fallback (cp on MSYS2/Git Bash).
|
|
// v1.67 (#2569): the source is render-aware — canonical SKILL.md, or the
|
|
// rendered :user variant from ${GSTACK_HOME}/render/claude when present.
|
|
expect(fnBody).toContain('_skill_md_src="$gstack_dir/$dir_name/SKILL.md"');
|
|
expect(fnBody).toContain('_link_or_copy "$_skill_md_src" "$target/SKILL.md"');
|
|
});
|
|
|
|
// REGRESSION: cleanup functions must handle both old symlinks AND new real-directory pattern
|
|
test('cleanup functions handle real directories with symlinked SKILL.md', () => {
|
|
// cleanup_old_claude_symlinks must detect and remove real dirs with SKILL.md symlinks
|
|
const cleanupOldStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
|
|
const cleanupOldEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up old', cleanupOldStart));
|
|
const cleanupOldBody = setupContent.slice(cleanupOldStart, cleanupOldEnd);
|
|
expect(cleanupOldBody).toContain('-d "$old_target"');
|
|
expect(cleanupOldBody).toContain('-L "$old_target/SKILL.md"');
|
|
expect(cleanupOldBody).toContain('rm -rf "$old_target"');
|
|
|
|
// cleanup_prefixed_claude_symlinks must also handle the new pattern
|
|
const cleanupPrefixedStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
|
|
const cleanupPrefixedEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up prefixed', cleanupPrefixedStart));
|
|
const cleanupPrefixedBody = setupContent.slice(cleanupPrefixedStart, cleanupPrefixedEnd);
|
|
expect(cleanupPrefixedBody).toContain('-d "$prefixed_target"');
|
|
expect(cleanupPrefixedBody).toContain('-L "$prefixed_target/SKILL.md"');
|
|
expect(cleanupPrefixedBody).toContain('rm -rf "$prefixed_target"');
|
|
});
|
|
|
|
// REGRESSION: link function must upgrade old directory symlinks
|
|
test('link_claude_skill_dirs removes old directory symlinks before creating real dirs', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// Must check for and remove old symlinks before mkdir
|
|
expect(fnBody).toContain('if [ -L "$target" ]');
|
|
expect(fnBody).toContain('rm -f "$target"');
|
|
});
|
|
|
|
test('setup links root gstack skill through a thin Claude wrapper alias', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_root_skill_alias()');
|
|
const fnEnd = setupContent.indexOf('# ─── Helper: remove old unprefixed Claude skill entries', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('_gstack-command');
|
|
// #2511: the alias must be a rewritten COPY (unique frontmatter name),
|
|
// never a verbatim symlink of the canonical SKILL.md.
|
|
expect(fnBody).toContain('_install_alias_skill_md "$gstack_dir/SKILL.md" "$target" "_gstack-command"');
|
|
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir/SKILL.md"');
|
|
|
|
const claudeSection = setupContent.slice(
|
|
setupContent.indexOf('# 4. Install for Claude'),
|
|
setupContent.indexOf('# 5. Install for Codex')
|
|
);
|
|
expect(claudeSection).toContain('link_claude_root_skill_alias "$SOURCE_GSTACK_DIR" "$INSTALL_SKILLS_DIR"');
|
|
});
|
|
|
|
test('setup supports --host auto|claude|codex|kiro|opencode|cursor|slate', () => {
|
|
expect(setupContent).toContain('--host');
|
|
expect(setupContent).toContain('claude|codex|kiro|factory|opencode|cursor|slate|auto');
|
|
});
|
|
|
|
test('auto mode detects claude, codex, kiro, and opencode binaries', () => {
|
|
expect(setupContent).toContain('command -v claude');
|
|
expect(setupContent).toContain('command -v codex');
|
|
expect(setupContent).toContain('command -v kiro-cli');
|
|
expect(setupContent).toContain('command -v opencode');
|
|
});
|
|
|
|
// T1: Sidecar skip guard — prevents .agents/skills/gstack from being linked as a skill
|
|
test('link_codex_skill_dirs skips the gstack sidecar directory', () => {
|
|
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
|
|
});
|
|
|
|
// T2: Dynamic $GSTACK_ROOT paths in generated Codex preambles
|
|
test('generated Codex preambles use dynamic GSTACK_ROOT paths', () => {
|
|
const codexSkillDir = path.join(ROOT, '.agents', 'skills', 'gstack-ship');
|
|
if (!fs.existsSync(codexSkillDir)) return; // skip if .agents/ not generated
|
|
const content = fs.readFileSync(path.join(codexSkillDir, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT=');
|
|
expect(content).toContain('$GSTACK_BIN/');
|
|
});
|
|
|
|
test('setup supports --host kiro with install section and sed rewrites', () => {
|
|
expect(setupContent).toContain('INSTALL_KIRO=');
|
|
expect(setupContent).toContain('kiro-cli');
|
|
expect(setupContent).toContain('KIRO_SKILLS=');
|
|
expect(setupContent).toContain('~/.kiro/skills/gstack');
|
|
expect(setupContent).toContain('$KIRO_GSTACK/lib');
|
|
});
|
|
|
|
test('setup supports --host opencode with install section and OpenCode skill path vars', () => {
|
|
expect(setupContent).toContain('INSTALL_OPENCODE=');
|
|
expect(setupContent).toContain('OPENCODE_SKILLS="$HOME/.config/opencode/skills"');
|
|
expect(setupContent).toContain('OPENCODE_GSTACK="$OPENCODE_SKILLS/gstack"');
|
|
});
|
|
|
|
// --host cursor full install slice (#1358, PR #2547 by @szsunyuan re-derived)
|
|
test('auto mode detects Cursor via binary or ~/.cursor directory', () => {
|
|
expect(setupContent).toContain('command -v cursor');
|
|
expect(setupContent).toContain('[ -d "$HOME/.cursor" ] && INSTALL_CURSOR=1');
|
|
});
|
|
|
|
test('setup supports --host cursor with install section and Cursor skill path vars', () => {
|
|
expect(setupContent).toContain('INSTALL_CURSOR=');
|
|
expect(setupContent).toContain('CURSOR_SKILLS="$HOME/.cursor/skills"');
|
|
expect(setupContent).toContain('CURSOR_GSTACK="$CURSOR_SKILLS/gstack"');
|
|
expect(setupContent).toContain('create_cursor_runtime_root');
|
|
expect(setupContent).toContain('create_cursor_sidecar');
|
|
expect(setupContent).toContain('link_cursor_skill_dirs');
|
|
expect(setupContent).toContain('gstack ready (cursor).');
|
|
});
|
|
|
|
test('create_cursor_runtime_root exposes only Cursor runtime assets', () => {
|
|
const fnStart = setupContent.indexOf('create_cursor_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('create_cursor_sidecar()', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('gstack/SKILL.md');
|
|
expect(fnBody).toContain('browse/dist');
|
|
expect(fnBody).toContain('browse/bin');
|
|
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
|
|
expect(fnBody).toContain('checklist.md');
|
|
expect(fnBody).toContain('TODOS-format.md');
|
|
// bin scripts import ../lib — the two must travel together.
|
|
expect(fnBody).toContain('$cursor_gstack/lib');
|
|
expect(fnBody).not.toContain('design-checklist.md');
|
|
expect(fnBody).not.toContain('greptile-triage.md');
|
|
expect(fnBody).not.toContain('review/specialists');
|
|
expect(fnBody).not.toContain('qa/templates');
|
|
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$cursor_gstack"');
|
|
});
|
|
|
|
test('create_cursor_sidecar plants runtime assets without wiping generated SKILL.md', () => {
|
|
const fnStart = setupContent.indexOf('create_cursor_sidecar()');
|
|
const fnEnd = setupContent.indexOf('link_cursor_skill_dirs()', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('.cursor/skills/gstack');
|
|
expect(fnBody).toContain('bin');
|
|
expect(fnBody).toContain('browse/dist');
|
|
expect(fnBody).toContain('browse/bin');
|
|
expect(fnBody).toContain('ETHOS.md');
|
|
expect(fnBody).not.toContain('rm -rf');
|
|
});
|
|
|
|
test('link_cursor_skill_dirs skips the gstack runtime root directory', () => {
|
|
const fnStart = setupContent.indexOf('link_cursor_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
|
|
// #2444-aware guard: Windows bypass, else only replace symlink-or-missing.
|
|
expect(fnBody).toContain('[ "$IS_WINDOWS" -eq 1 ] || [ -L "$target" ] || [ ! -e "$target" ]');
|
|
});
|
|
|
|
// #2142 deleted existing ~/.cursor/skills/<name> dirs with `rm -rf "$target"`
|
|
// before relinking. That can wipe unowned Cursor skills. Only replace a
|
|
// symlink or a missing path; never the whole skills directory.
|
|
test('link_cursor_skill_dirs does not delete unowned Cursor skill directories', () => {
|
|
const fnStart = setupContent.indexOf('link_cursor_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).not.toContain('rm -rf "$target"');
|
|
expect(fnBody).not.toContain('rm -rf "$skills_dir"');
|
|
expect(setupContent).not.toContain('rm -rf "$CURSOR_SKILLS"');
|
|
});
|
|
|
|
test('Cursor install links generated skills before planting the sidecar', () => {
|
|
const cursorInstall = setupContent.slice(
|
|
setupContent.indexOf('# 6d. Install for Cursor'),
|
|
setupContent.indexOf('# 7. Create .agents/ sidecar'),
|
|
);
|
|
const linkCall = cursorInstall.indexOf('link_cursor_skill_dirs "$SOURCE_GSTACK_DIR"');
|
|
const sidecarCall = cursorInstall.indexOf('create_cursor_sidecar "$SOURCE_GSTACK_DIR"');
|
|
expect(linkCall).toBeGreaterThan(-1);
|
|
expect(sidecarCall).toBeGreaterThan(-1);
|
|
expect(linkCall).toBeLessThan(sidecarCall);
|
|
});
|
|
|
|
test('setup installs OpenCode skills into a nested gstack runtime root', () => {
|
|
expect(setupContent).toContain('create_opencode_runtime_root');
|
|
expect(setupContent).toContain('.opencode/skills');
|
|
expect(setupContent).toContain('review/specialists');
|
|
expect(setupContent).toContain('qa/templates');
|
|
expect(setupContent).toContain('qa/references');
|
|
expect(setupContent).toContain('dx-hall-of-fame.md');
|
|
expect(setupContent).toContain('$opencode_gstack/lib');
|
|
});
|
|
|
|
test('create_agents_sidecar links runtime assets', () => {
|
|
// Sidecar must link bin with its shared lib modules, plus browse, review, qa
|
|
const fnStart = setupContent.indexOf('create_agents_sidecar()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('bin');
|
|
expect(fnBody).toContain('lib');
|
|
expect(fnBody).toContain('browse');
|
|
expect(fnBody).toContain('review');
|
|
expect(fnBody).toContain('qa');
|
|
});
|
|
|
|
test('create_codex_runtime_root exposes only runtime assets', () => {
|
|
const fnStart = setupContent.indexOf('create_codex_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', setupContent.indexOf('review/', fnStart)));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('gstack/SKILL.md');
|
|
expect(fnBody).toContain('$codex_gstack/lib');
|
|
expect(fnBody).toContain('browse/dist');
|
|
expect(fnBody).toContain('browse/bin');
|
|
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
|
|
// Review runtime assets (individual files, not the whole dir)
|
|
expect(fnBody).toContain('checklist.md');
|
|
expect(fnBody).toContain('design-checklist.md');
|
|
expect(fnBody).toContain('greptile-triage.md');
|
|
expect(fnBody).toContain('TODOS-format.md');
|
|
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$codex_gstack"');
|
|
});
|
|
|
|
test('create_factory_runtime_root links shared lib modules beside bin', () => {
|
|
const fnStart = setupContent.indexOf('create_factory_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('create_opencode_runtime_root()', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('$factory_gstack/bin');
|
|
expect(fnBody).toContain('$factory_gstack/lib');
|
|
});
|
|
|
|
test('direct Codex installs are migrated out of ~/.codex/skills/gstack', () => {
|
|
expect(setupContent).toContain('migrate_direct_codex_install');
|
|
expect(setupContent).toContain('$HOME/.gstack/repos/gstack');
|
|
expect(setupContent).toContain('avoid duplicate skill discovery');
|
|
});
|
|
|
|
// --- Symlink prefix tests (PR #503) ---
|
|
|
|
test('link_claude_skill_dirs applies gstack- prefix by default', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('SKILL_PREFIX');
|
|
expect(fnBody).toContain('link_name="gstack-$skill_name"');
|
|
});
|
|
|
|
test('link_claude_skill_dirs preserves already-prefixed dirs', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// gstack-* dirs should keep their name (e.g., gstack-upgrade stays gstack-upgrade)
|
|
expect(fnBody).toContain('gstack-*) link_name="$skill_name"');
|
|
});
|
|
|
|
test('setup supports --no-prefix flag', () => {
|
|
expect(setupContent).toContain('--no-prefix');
|
|
expect(setupContent).toContain('SKILL_PREFIX=0');
|
|
});
|
|
|
|
test('cleanup_old_claude_symlinks removes only gstack-pointing symlinks', () => {
|
|
expect(setupContent).toContain('cleanup_old_claude_symlinks');
|
|
const fnStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// Should check readlink before removing
|
|
expect(fnBody).toContain('readlink');
|
|
expect(fnBody).toContain('gstack/*');
|
|
// Should skip already-prefixed dirs
|
|
expect(fnBody).toContain('gstack-*) continue');
|
|
});
|
|
|
|
test('cleanup runs before link when prefix is enabled', () => {
|
|
// In the Claude install section, cleanup should happen before linking
|
|
const claudeInstallSection = setupContent.slice(
|
|
setupContent.indexOf('INSTALL_CLAUDE'),
|
|
setupContent.lastIndexOf('link_claude_skill_dirs')
|
|
);
|
|
expect(claudeInstallSection).toContain('cleanup_old_claude_symlinks');
|
|
});
|
|
|
|
// --- Persistent config + interactive prompt tests ---
|
|
|
|
test('setup reads skill_prefix from config', () => {
|
|
expect(setupContent).toContain('get skill_prefix');
|
|
expect(setupContent).toContain('GSTACK_CONFIG');
|
|
});
|
|
|
|
test('setup supports --prefix flag', () => {
|
|
expect(setupContent).toContain('--prefix)');
|
|
expect(setupContent).toContain('SKILL_PREFIX=1; SKILL_PREFIX_FLAG=1');
|
|
});
|
|
|
|
test('--prefix and --no-prefix persist to config', () => {
|
|
expect(setupContent).toContain('set skill_prefix');
|
|
});
|
|
|
|
test('interactive prompt shows when no config', () => {
|
|
expect(setupContent).toContain('Short names');
|
|
expect(setupContent).toContain('Namespaced');
|
|
expect(setupContent).toContain('Choice [1/2]');
|
|
});
|
|
|
|
test('non-TTY defaults to flat names', () => {
|
|
// Should check if stdin is a TTY before prompting
|
|
expect(setupContent).toContain('-t 0');
|
|
});
|
|
|
|
test('cleanup_prefixed_claude_symlinks exists and uses readlink', () => {
|
|
expect(setupContent).toContain('cleanup_prefixed_claude_symlinks');
|
|
const fnStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('readlink');
|
|
expect(fnBody).toContain('gstack-$skill_name');
|
|
});
|
|
|
|
test('reverse cleanup runs before link when prefix is disabled', () => {
|
|
const claudeInstallSection = setupContent.slice(
|
|
setupContent.indexOf('INSTALL_CLAUDE'),
|
|
setupContent.lastIndexOf('link_claude_skill_dirs')
|
|
);
|
|
expect(claudeInstallSection).toContain('cleanup_prefixed_claude_symlinks');
|
|
});
|
|
|
|
test('welcome message references SKILL_PREFIX', () => {
|
|
// gstack-upgrade is always called gstack-upgrade (it's the actual dir name)
|
|
// but the welcome section should exist near the prefix logic
|
|
expect(setupContent).toContain('Run /gstack-upgrade anytime');
|
|
});
|
|
});
|
|
|
|
describe('discover-skills hidden directory filtering', () => {
|
|
test('discoverTemplates skips dot-prefixed directories', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-discover-'));
|
|
try {
|
|
// Create a hidden dir with a template (should be excluded)
|
|
fs.mkdirSync(path.join(tmpDir, '.hidden'), { recursive: true });
|
|
fs.writeFileSync(path.join(tmpDir, '.hidden', 'SKILL.md.tmpl'), '---\nname: evil\n---\ntest');
|
|
// Create a visible dir with a template (should be included)
|
|
fs.mkdirSync(path.join(tmpDir, 'visible'), { recursive: true });
|
|
fs.writeFileSync(path.join(tmpDir, 'visible', 'SKILL.md.tmpl'), '---\nname: good\n---\ntest');
|
|
|
|
const { discoverTemplates } = require('../scripts/discover-skills');
|
|
const results = discoverTemplates(tmpDir);
|
|
const dirs = results.map((r: { tmpl: string }) => r.tmpl);
|
|
|
|
expect(dirs).toContain('visible/SKILL.md.tmpl');
|
|
expect(dirs).not.toContain('.hidden/SKILL.md.tmpl');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('telemetry', () => {
|
|
test('generated SKILL.md contains telemetry start block', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_TEL_START');
|
|
expect(content).toContain('_SESSION_ID');
|
|
expect(content).toContain('TELEMETRY:');
|
|
expect(content).toContain('TEL_PROMPTED:');
|
|
expect(content).toContain('gstack-config get telemetry');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry opt-in prompt', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.telemetry-prompted');
|
|
expect(content).toContain('Help gstack get better');
|
|
expect(content).toContain('gstack-config set telemetry community');
|
|
expect(content).toContain('gstack-config set telemetry anonymous');
|
|
expect(content).toContain('gstack-config set telemetry off');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry epilogue', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Telemetry (run last)');
|
|
expect(content).toContain('gstack-telemetry-log');
|
|
expect(content).toContain('_TEL_END');
|
|
expect(content).toContain('_TEL_DUR');
|
|
expect(content).toContain('SKILL_NAME');
|
|
expect(content).toContain('OUTCOME');
|
|
expect(content).toContain('PLAN MODE EXCEPTION');
|
|
});
|
|
|
|
test('generated SKILL.md contains pending marker handling', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.pending');
|
|
expect(content).toContain('_pending_finalize');
|
|
});
|
|
|
|
test('telemetry blocks appear in all skill files that use PREAMBLE', () => {
|
|
const skills = ['qa', 'ship', 'review', 'plan-ceo-review', 'plan-eng-review', 'retro'];
|
|
for (const skill of skills) {
|
|
const skillPath = path.join(ROOT, skill, 'SKILL.md');
|
|
if (fs.existsSync(skillPath)) {
|
|
const content = fs.readFileSync(skillPath, 'utf-8');
|
|
expect(content).toContain('_TEL_START');
|
|
expect(content).toContain('Telemetry (run last)');
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('community fixes wave', () => {
|
|
// Helper to get all generated SKILL.md files
|
|
function getAllSkillMds(): Array<{ name: string; content: string }> {
|
|
const results: Array<{ name: string; content: string }> = [];
|
|
const rootPath = path.join(ROOT, 'SKILL.md');
|
|
if (fs.existsSync(rootPath)) {
|
|
results.push({ name: 'root', content: fs.readFileSync(rootPath, 'utf-8') });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
const skillPath = path.join(ROOT, entry.name, 'SKILL.md');
|
|
if (fs.existsSync(skillPath)) {
|
|
results.push({ name: entry.name, content: fs.readFileSync(skillPath, 'utf-8') });
|
|
}
|
|
}
|
|
return results;
|
|
}
|
|
|
|
// #594 — Discoverability: every SKILL.md.tmpl description contains "gstack"
|
|
test('every SKILL.md.tmpl description contains "gstack"', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
const content = fs.readFileSync(tmplPath, 'utf-8');
|
|
const desc = extractDescription(content);
|
|
expect(desc.toLowerCase()).toContain('gstack');
|
|
}
|
|
});
|
|
|
|
// #594 — Discoverability: first line of each description is under 120 chars
|
|
test('every SKILL.md.tmpl description first line is under 120 chars', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
const content = fs.readFileSync(tmplPath, 'utf-8');
|
|
const desc = extractDescription(content);
|
|
const firstLine = desc.split('\n')[0];
|
|
expect(firstLine.length).toBeLessThanOrEqual(120);
|
|
}
|
|
});
|
|
|
|
// #573 — Feature signals: ship/SKILL.md contains feature signal detection
|
|
test('ship/SKILL.md contains feature signal detection in Step 4', () => {
|
|
const content = readShipUnion();
|
|
expect(content.toLowerCase()).toContain('feature signal');
|
|
});
|
|
|
|
// #510 — Context warnings: no SKILL.md contains "running low on context"
|
|
test('no generated SKILL.md contains "running low on context"', () => {
|
|
const skills = getAllSkillMds();
|
|
for (const { name, content } of skills) {
|
|
expect(content).not.toContain('running low on context');
|
|
}
|
|
});
|
|
|
|
// #510 — Context warnings: plan-eng-review has explicit anti-warning
|
|
test('plan-eng-review/SKILL.md contains "Do not preemptively warn"', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('Do not preemptively warn');
|
|
});
|
|
|
|
// #474 — Safety Net: no SKILL.md uses find with -delete
|
|
test('no generated SKILL.md contains find with -delete flag', () => {
|
|
const skills = getAllSkillMds();
|
|
for (const { name, content } of skills) {
|
|
// Match find commands that use -delete (but not prose mentioning the word "delete")
|
|
const lines = content.split('\n');
|
|
for (const line of lines) {
|
|
if (line.includes('find ') && line.includes('-delete')) {
|
|
throw new Error(`${name}/SKILL.md contains find with -delete: ${line.trim()}`);
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
// #467 — Telemetry: preamble JSONL writes are gated by telemetry setting
|
|
test('preamble JSONL writes are inside telemetry conditional', () => {
|
|
const preamble = fs.readFileSync(path.join(ROOT, 'scripts/resolvers/preamble.ts'), 'utf-8');
|
|
// Find all skill-usage.jsonl write lines
|
|
const lines = preamble.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
if (lines[i].includes('skill-usage.jsonl') && lines[i].includes('>>')) {
|
|
// Look backwards for a telemetry conditional within 5 lines
|
|
let foundConditional = false;
|
|
for (let j = i - 1; j >= Math.max(0, i - 5); j--) {
|
|
if (lines[j].includes('_TEL') && lines[j].includes('off')) {
|
|
foundConditional = true;
|
|
break;
|
|
}
|
|
}
|
|
expect(foundConditional).toBe(true);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('codex commands must not use inline $(git rev-parse --show-toplevel) for cwd', () => {
|
|
// Regression test: inline $(git rev-parse --show-toplevel) in codex exec -C
|
|
// or codex review without cd evaluates in whatever cwd the background shell
|
|
// inherits, which may be a different project in Conductor workspaces.
|
|
// The fix is to resolve _REPO_ROOT eagerly at the top of each bash block.
|
|
|
|
// Scan all source files that could contain codex commands
|
|
// Use Bun.Glob to avoid ELOOP from .claude/skills/gstack symlink back to ROOT
|
|
const tmplGlob = new Bun.Glob('**/*.tmpl');
|
|
const sourceFiles = [
|
|
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
|
|
.filter(f => f.endsWith('.ts'))
|
|
.map(f => `scripts/resolvers/${f}`),
|
|
'scripts/gen-skill-docs.ts',
|
|
];
|
|
|
|
test('no codex exec command uses inline $(git rev-parse --show-toplevel) in -C flag', () => {
|
|
const violations: string[] = [];
|
|
for (const rel of sourceFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1}`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('no generated SKILL.md has codex exec with inline $(git rev-parse --show-toplevel) in -C flag', () => {
|
|
const violations: string[] = [];
|
|
const skillMdGlob = new Bun.Glob('**/SKILL.md');
|
|
const skillMdFiles = Array.from(skillMdGlob.scanSync({ cwd: ROOT, followSymlinks: false }));
|
|
for (const rel of skillMdFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1}`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('codex review commands must be preceded by cd "$_REPO_ROOT" (no -C support)', () => {
|
|
// codex review does not support -C, so the pattern must be:
|
|
// _REPO_ROOT=$(git rev-parse --show-toplevel) || { ... }
|
|
// cd "$_REPO_ROOT"
|
|
// codex review ...
|
|
// NOT: codex review ... with inline $(git rev-parse --show-toplevel)
|
|
const allFiles = [
|
|
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...Array.from(new Bun.Glob('**/SKILL.md').scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
|
|
.filter(f => f.endsWith('.ts'))
|
|
.map(f => `scripts/resolvers/${f}`),
|
|
'scripts/gen-skill-docs.ts',
|
|
];
|
|
const violations: string[] = [];
|
|
for (const rel of allFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
// Skip non-executable lines (markdown table cells, prose references)
|
|
if (line.includes('|') && line.includes('`/codex review`')) continue;
|
|
if (line.includes('`codex review`')) continue;
|
|
// Check for codex review with inline $(git rev-parse)
|
|
if (line.includes('codex review') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1} — inline git rev-parse in codex review`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('codex review commands take their scope from a flag, never from prompt text', () => {
|
|
// `codex review` scope comes ONLY from --base/--commit/--uncommitted. The
|
|
// positional [PROMPT] is mutually exclusive with all three (#1428, #1479),
|
|
// and a prompt-only `codex review` silently falls back to the *uncommitted
|
|
// working-tree* scope (`git status --short; git diff`) — so describing the
|
|
// diff range in prompt text produces a confident review of the wrong
|
|
// changes, with no error. Both halves are pinned here:
|
|
// (a) every `codex review` invocation carries a scope flag, and
|
|
// (b) no invocation puts a positional prompt in front of that flag.
|
|
//
|
|
// This does NOT apply to `codex exec`, which is agentic and really does run
|
|
// the git command it's told to — the adversarial pass legitimately scopes
|
|
// itself in prompt text.
|
|
const checkedFiles = [
|
|
'scripts/resolvers/review.ts',
|
|
'review/SKILL.md',
|
|
'ship/SKILL.md',
|
|
'codex/SKILL.md.tmpl',
|
|
'codex/SKILL.md',
|
|
];
|
|
|
|
const violations: string[] = [];
|
|
for (const rel of checkedFiles) {
|
|
// ship's codex/adversarial command moved into sections/adversarial.md (T9 carve).
|
|
const content = rel === 'ship/SKILL.md' ? readShipUnion() : fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
// Only inspect real shell invocations, not prose mentioning the command.
|
|
if (line.includes('`codex review`')) continue;
|
|
const match = line.match(/(?:^|[;&|]\s*|\s)codex\s+review\b(.*)$/);
|
|
if (!match) continue;
|
|
const rest = match[1];
|
|
const scopeFlag = /--base\b|--commit\b|--uncommitted\b/;
|
|
if (!scopeFlag.test(rest)) {
|
|
// A quoted prompt with no scope flag is the silent-wrong-scope bug.
|
|
if (/^\s*["'$]/.test(rest)) {
|
|
violations.push(`${rel}:${i + 1} — prompt-only codex review (falls back to working-tree scope)`);
|
|
}
|
|
continue;
|
|
}
|
|
const beforeFlag = rest.split(scopeFlag)[0].trim();
|
|
if (/^["'$]|^--\s*["']/.test(beforeFlag)) {
|
|
violations.push(`${rel}:${i + 1} — positional prompt passed alongside a scope flag`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
});
|
|
|
|
// ─── Learnings + Confidence Resolver Tests ─────────────────────
|
|
|
|
describe('LEARNINGS_SEARCH resolver', () => {
|
|
const SEARCH_SKILLS = ['review', 'ship', 'plan-eng-review', 'investigate', 'office-hours', 'plan-ceo-review'];
|
|
|
|
for (const skill of SEARCH_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains learnings search`, () => {
|
|
const content = readSkillUnion(skill); // ship: moved to sections/plan-completion.md
|
|
expect(content).toContain('Prior Learnings');
|
|
expect(content).toContain('gstack-learnings-search');
|
|
});
|
|
}
|
|
|
|
test('learnings search includes cross-project config check', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('cross_project_learnings');
|
|
expect(content).toContain('--cross-project');
|
|
});
|
|
|
|
test('learnings search includes AskUserQuestion for first-time cross-project opt-in', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Enable cross-project learnings');
|
|
expect(content).toContain('project-scoped only');
|
|
});
|
|
|
|
test('learnings search mentions prior learning applied display format', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Prior learning applied');
|
|
});
|
|
});
|
|
|
|
describe('LEARNINGS_LOG resolver', () => {
|
|
const LOG_SKILLS = ['review', 'retro', 'investigate'];
|
|
|
|
for (const skill of LOG_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains learnings log`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Capture Learnings');
|
|
expect(content).toContain('gstack-learnings-log');
|
|
});
|
|
}
|
|
|
|
test('learnings log documents all type values', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
for (const type of ['pattern', 'pitfall', 'preference', 'architecture', 'tool']) {
|
|
expect(content).toContain(type);
|
|
}
|
|
});
|
|
|
|
test('learnings log documents all source values', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
for (const source of ['observed', 'user-stated', 'inferred', 'cross-model']) {
|
|
expect(content).toContain(source);
|
|
}
|
|
});
|
|
|
|
test('learnings log includes files field for staleness detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"files"');
|
|
expect(content).toContain('staleness detection');
|
|
});
|
|
});
|
|
|
|
describe('CONFIDENCE_CALIBRATION resolver', () => {
|
|
const CONFIDENCE_SKILLS = ['review', 'ship', 'plan-eng-review', 'cso'];
|
|
|
|
for (const skill of CONFIDENCE_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains confidence calibration`, () => {
|
|
const content = readSkillUnion(skill); // ship: moved to sections/review-army.md
|
|
expect(content).toContain('Confidence Calibration');
|
|
expect(content).toContain('confidence score');
|
|
});
|
|
}
|
|
|
|
test('confidence calibration includes scoring rubric with all tiers', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('9-10');
|
|
expect(content).toContain('7-8');
|
|
expect(content).toContain('5-6');
|
|
expect(content).toContain('3-4');
|
|
expect(content).toContain('1-2');
|
|
});
|
|
|
|
test('confidence calibration includes display rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Show normally');
|
|
expect(content).toContain('Suppress from main report');
|
|
});
|
|
|
|
test('confidence calibration includes finding format example', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('[P1] (confidence:');
|
|
expect(content).toContain('SQL injection');
|
|
});
|
|
|
|
test('confidence calibration includes calibration learning feedback loop', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('calibration event');
|
|
expect(content).toContain('Log the corrected pattern');
|
|
});
|
|
|
|
test('skills without confidence calibration do NOT contain it', () => {
|
|
// office-hours and retro do NOT use confidence calibration
|
|
for (const skill of ['office-hours', 'retro']) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('## Confidence Calibration');
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('gen-skill-docs prefix warning (#620/#578)', () => {
|
|
const { execSync } = require('child_process');
|
|
|
|
test('warns about skill_prefix when config has prefix=true', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
|
|
try {
|
|
// Create a fake ~/.gstack/config.yaml with skill_prefix: true
|
|
const fakeHome = tmpDir;
|
|
const fakeGstack = path.join(fakeHome, '.gstack');
|
|
fs.mkdirSync(fakeGstack, { recursive: true });
|
|
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: true\n');
|
|
|
|
const output = execSync('bun run scripts/gen-skill-docs.ts', {
|
|
cwd: ROOT,
|
|
env: { ...process.env, HOME: fakeHome },
|
|
encoding: 'utf-8',
|
|
timeout: 30000,
|
|
});
|
|
expect(output).toContain('skill_prefix is true');
|
|
expect(output).toContain('gstack-relink');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
test('no warning when skill_prefix is false or absent', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
|
|
try {
|
|
const fakeHome = tmpDir;
|
|
const fakeGstack = path.join(fakeHome, '.gstack');
|
|
fs.mkdirSync(fakeGstack, { recursive: true });
|
|
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: false\n');
|
|
|
|
const output = execSync('bun run scripts/gen-skill-docs.ts', {
|
|
cwd: ROOT,
|
|
env: { ...process.env, HOME: fakeHome },
|
|
encoding: 'utf-8',
|
|
timeout: 30000,
|
|
});
|
|
expect(output).not.toContain('skill_prefix is true');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('voice-triggers processing', () => {
|
|
const { extractVoiceTriggers, processVoiceTriggers } = require('../scripts/gen-skill-docs') as {
|
|
extractVoiceTriggers: (content: string) => string[];
|
|
processVoiceTriggers: (content: string) => string;
|
|
};
|
|
|
|
test('extractVoiceTriggers parses valid YAML list', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit.\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
|
|
const triggers = extractVoiceTriggers(content);
|
|
expect(triggers).toEqual(['see-so', 'security review']);
|
|
});
|
|
|
|
test('extractVoiceTriggers returns [] when no field present', () => {
|
|
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
|
|
expect(extractVoiceTriggers(content)).toEqual([]);
|
|
});
|
|
|
|
test('processVoiceTriggers appends voice triggers to description', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
|
|
const result = processVoiceTriggers(content);
|
|
expect(result).toContain('Voice triggers (speech-to-text aliases): "see-so", "security review".');
|
|
});
|
|
|
|
test('processVoiceTriggers strips voice-triggers field from output', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n---\nBody`;
|
|
const result = processVoiceTriggers(content);
|
|
expect(result).not.toContain('voice-triggers:');
|
|
});
|
|
|
|
test('processVoiceTriggers returns content unchanged when no voice-triggers', () => {
|
|
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
|
|
expect(processVoiceTriggers(content)).toBe(content);
|
|
});
|
|
|
|
test('generated CSO SKILL.md contains voice triggers in description', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"see-so"');
|
|
expect(content).toContain('Voice triggers (speech-to-text aliases):');
|
|
});
|
|
|
|
test('generated CSO SKILL.md does NOT contain raw voice-triggers field', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(0, fmEnd);
|
|
expect(frontmatter).not.toContain('voice-triggers:');
|
|
});
|
|
});
|
|
|
|
describe('plan-mode-info resolver (handshake-replacement)', () => {
|
|
const REVIEW_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
];
|
|
|
|
// Header for the vestigial handshake that was removed. If it ever reappears,
|
|
// someone accidentally re-introduced the resolver.
|
|
const HANDSHAKE_MARKER = '## Plan Mode Handshake';
|
|
// Header for the new plan-mode-info section (previously lived at the tail
|
|
// of completion-status.ts; now hoisted to position 1 of the preamble).
|
|
const PLAN_MODE_INFO_MARKER = '## Skill Invocation During Plan Mode';
|
|
|
|
test('vestigial handshake is absent from all generated Claude SKILL.md files', () => {
|
|
// Scan every generated SKILL.md under ROOT (top-level directory per skill).
|
|
// Using fs.readdirSync + filter instead of a glob so we catch any skill
|
|
// that gets added later without updating this list.
|
|
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
|
|
let checked = 0;
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(ROOT, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content, `handshake marker in ${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
|
|
checked++;
|
|
}
|
|
expect(checked).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('vestigial handshake is absent from non-Claude host outputs when present on disk', () => {
|
|
// Non-Claude hosts render to hostSubdirs (.agents/, .openclaw/, etc). The
|
|
// plan-mode-info resolver has no host-scoping — all hosts get the new
|
|
// section, none get the old handshake. Scan all candidate host dirs.
|
|
const hostDirs = ['.agents', '.openclaw', '.opencode', '.factory', '.hermes', '.kiro', '.cursor', '.slate'];
|
|
let checked = 0;
|
|
for (const host of hostDirs) {
|
|
const skillsRoot = path.join(ROOT, host, 'skills');
|
|
if (!fs.existsSync(skillsRoot)) continue;
|
|
const entries = fs.readdirSync(skillsRoot, { withFileTypes: true });
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(skillsRoot, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content, `handshake marker in ${host}/skills/${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
|
|
checked++;
|
|
}
|
|
}
|
|
if (checked === 0) {
|
|
// eslint-disable-next-line no-console
|
|
console.warn(
|
|
'plan-mode-info: no non-Claude host outputs found for cross-host absence check — ' +
|
|
'run `bun run gen:skill-docs --host all` to populate',
|
|
);
|
|
}
|
|
});
|
|
|
|
test.each(REVIEW_SKILLS)(
|
|
'%s/SKILL.md contains the new plan-mode-info section near the top',
|
|
(skill) => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const idx = content.indexOf(PLAN_MODE_INFO_MARKER);
|
|
expect(idx).toBeGreaterThan(0);
|
|
// Position 1 in preamble composition = within the first ~300 lines.
|
|
// Roughly translates to first ~15KB of text.
|
|
expect(idx).toBeLessThan(15_000);
|
|
},
|
|
);
|
|
|
|
test('plan-mode-info is wired BEFORE generateUpgradeCheck in preamble', () => {
|
|
const content = fs.readFileSync(
|
|
path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
|
|
'utf-8',
|
|
);
|
|
const planModeIdx = content.indexOf(PLAN_MODE_INFO_MARKER);
|
|
const upgradeIdx = content.indexOf('UPGRADE_AVAILABLE');
|
|
expect(planModeIdx).toBeGreaterThan(0);
|
|
expect(upgradeIdx).toBeGreaterThan(0);
|
|
expect(planModeIdx).toBeLessThan(upgradeIdx);
|
|
});
|
|
|
|
test('0C-bis STOP block present in plan-ceo-review/SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
const presentIdx = content.indexOf('Present these approach options via AskUserQuestion');
|
|
const preludeIdx = content.indexOf('### 0D-prelude');
|
|
expect(presentIdx).toBeGreaterThan(0);
|
|
expect(preludeIdx).toBeGreaterThan(presentIdx);
|
|
const between = content.slice(presentIdx, preludeIdx);
|
|
expect(between).toContain('**STOP.**');
|
|
expect(between).toContain('Do NOT proceed to Step 0D or 0F until the user responds to 0C-bis');
|
|
});
|
|
});
|
|
|
|
// GSTACK REVIEW REPORT report-at-bottom contract — verifies the prompt-text
|
|
// fix in scripts/resolvers/review.ts (the load-bearing change for the
|
|
// "report not at bottom of plan in plan mode" bug). The bug is in the
|
|
// prompt's contradictory write-flow instructions, not in observable
|
|
// runtime behavior we can cheaply gate in CI. Verifying the prompt text
|
|
// directly is the deterministic equivalent of the regression test the
|
|
// PTY harness can't reliably drive (autoplan needs auto-progression of
|
|
// AskUserQuestions to reach the report-write step, which the harness
|
|
// doesn't support today).
|
|
describe('GSTACK REVIEW REPORT delete-then-append flow', () => {
|
|
const PLAN_REVIEW_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'plan-eng-review',
|
|
];
|
|
|
|
for (const skill of PLAN_REVIEW_SKILLS) {
|
|
test(`${skill}/SKILL.md prescribes delete-then-append, not in-place replace`, () => {
|
|
// Carved skills (v2 plan Phase B) relocate the review-report prose into
|
|
// sections/*.md; readSkillUnion follows the content wherever the carve put it.
|
|
const content = readSkillUnion(skill);
|
|
|
|
// The new (correct) instruction must be present.
|
|
expect(content).toContain('delete-then-append flow');
|
|
expect(content).toContain('never mid-file');
|
|
expect(content).toContain('Do NOT replace the section in place');
|
|
|
|
// The old contradictory bullets must be gone. The signature phrase
|
|
// from the buggy prompt was 'replace it entirely using the Edit tool'
|
|
// which is what allowed mid-file reports to stay mid-file.
|
|
expect(content).not.toContain('replace it** entirely using the Edit tool');
|
|
expect(content).not.toContain('If it was found mid-file, move it');
|
|
});
|
|
}
|
|
|
|
test('scripts/resolvers/review.ts source has the rewritten flow', () => {
|
|
const src = fs.readFileSync(path.join(ROOT, 'scripts', 'resolvers', 'review.ts'), 'utf-8');
|
|
expect(src).toContain('delete-then-append flow');
|
|
expect(src).toContain('never mid-file');
|
|
expect(src).toContain('Do NOT replace the section in place');
|
|
// Old contradictory bullets are gone from the source resolver.
|
|
expect(src).not.toContain('replace it** entirely using the Edit tool');
|
|
expect(src).not.toContain('If it was found mid-file, move it');
|
|
});
|
|
});
|
|
|
|
describe('LEARNINGS_SEARCH resolver: query parameter', () => {
|
|
// Lazy-load resolver and types after describe block to keep test file self-contained.
|
|
const { generateLearningsSearch } = require('../scripts/resolvers/learnings');
|
|
const { HOST_PATHS } = require('../scripts/resolvers/types');
|
|
|
|
const claudeCtx = {
|
|
skillName: 'test',
|
|
tmplPath: 'test/SKILL.md.tmpl',
|
|
host: 'claude',
|
|
paths: HOST_PATHS.claude,
|
|
};
|
|
const codexCtx = { ...claudeCtx, host: 'codex', paths: HOST_PATHS.codex };
|
|
|
|
test('no args → bash does not contain --query (backwards-compat)', () => {
|
|
const out = generateLearningsSearch(claudeCtx);
|
|
expect(out).not.toContain('--query');
|
|
});
|
|
|
|
test('claude host + query=foo bar → both cross-project and project-scoped branches contain --query', () => {
|
|
const out = generateLearningsSearch(claudeCtx, ['query=foo bar']);
|
|
// Both branches of the if/else must carry the flag.
|
|
const lines = out.split('\n').filter(l => l.includes('gstack-learnings-search'));
|
|
expect(lines.length).toBeGreaterThanOrEqual(2);
|
|
for (const line of lines) {
|
|
expect(line).toContain('--query "foo bar"');
|
|
}
|
|
});
|
|
|
|
test('codex host + query=foo bar → codex bash variant contains --query', () => {
|
|
const out = generateLearningsSearch(codexCtx, ['query=foo bar']);
|
|
expect(out).toContain('--query "foo bar"');
|
|
expect(out).toContain('$GSTACK_BIN/gstack-learnings-search');
|
|
});
|
|
|
|
test('empty value query= → bash does not contain --query (locked semantics: falls through)', () => {
|
|
const claudeOut = generateLearningsSearch(claudeCtx, ['query=']);
|
|
expect(claudeOut).not.toContain('--query');
|
|
const codexOut = generateLearningsSearch(codexCtx, ['query=']);
|
|
expect(codexOut).not.toContain('--query');
|
|
});
|
|
|
|
test('shell-injection chars in query= → throws at gen-time (defense in depth)', () => {
|
|
for (const bad of ['$(whoami)', '`cmd`', 'a;b', 'a&b', 'a"b', 'a\\b', 'foo$x']) {
|
|
expect(() => generateLearningsSearch(claudeCtx, [`query=${bad}`])).toThrow(/alphanumeric/);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('EXIT PLAN MODE GATE placement', () => {
|
|
// Fresh skill list — do NOT reuse REVIEW_SKILLS upstream (3 entries, missing plan-devex).
|
|
const planSkills = ['plan-eng-review', 'plan-ceo-review', 'plan-design-review', 'plan-devex-review'];
|
|
|
|
// Strip fenced code blocks before matching headings — PLAN_FILE_REVIEW_REPORT
|
|
// already contains `## GSTACK REVIEW REPORT` inside a markdown example fence,
|
|
// and the gate text itself shows `## GSTACK REVIEW REPORT` inside a fence too.
|
|
const stripFences = (md: string) => md.replace(/```[\s\S]*?```/g, '');
|
|
|
|
test('gate is the terminal ## heading in every plan-* review SKILL.md', () => {
|
|
for (const skill of planSkills) {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const stripped = stripFences(md);
|
|
const headings = [...stripped.matchAll(/^## .+$/gm)].map(m => m[0]);
|
|
const lastH2 = headings.at(-1);
|
|
expect(lastH2, `${skill}/SKILL.md last ## heading (fences stripped)`).toBe('## EXIT PLAN MODE GATE (BLOCKING)');
|
|
expect(md, `${skill}/SKILL.md gate body`).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
|
|
}
|
|
});
|
|
|
|
test('codex/SKILL.md contains gate (mid-file per D5; Step 2B/2C follow)', () => {
|
|
const codex = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(codex).toContain('## EXIT PLAN MODE GATE (BLOCKING)');
|
|
expect(codex).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
|
|
});
|
|
});
|
|
|
|
describe('scope-gate exceptions drift-guard', () => {
|
|
// The plan-mode auto-select-B exceptions block is hand-duplicated in the
|
|
// plan-eng-review and plan-design-review templates (matching the gate
|
|
// around it, which predates this block). The two copies must stay
|
|
// byte-identical modulo exactly two known variant slots:
|
|
// 1. the plan-mode bullet's action tail (Design Doc Check vs pre-review
|
|
// audit + mockups),
|
|
// 2. the named-target vocabulary ("a path, a doc" vs "a path, a page, a doc").
|
|
// A future edit to one copy that silently misses the other fails here
|
|
// instead of drifting. The real fix (shared {{SCOPE_GATE}} resolver) is a
|
|
// filed TODO — this guard is the stopgap that makes the duplication safe.
|
|
const START_MARKER = '**Exceptions — check in this order, BEFORE asking:**';
|
|
const END_MARKER = 'in any mode — it is a hard STOP.';
|
|
|
|
function extractExceptionsBlock(skill: string): string {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const start = md.indexOf(START_MARKER);
|
|
expect(start, `${skill}/SKILL.md: exceptions block start marker present`).toBeGreaterThan(-1);
|
|
const end = md.indexOf(END_MARKER, start);
|
|
expect(end, `${skill}/SKILL.md: exceptions block end marker present`).toBeGreaterThan(start);
|
|
return md.slice(start, end + END_MARKER.length);
|
|
}
|
|
|
|
const normalizeVariantSlots = (block: string) =>
|
|
block
|
|
.replace('Then run the Design Doc Check and Step 0 against that plan.', '<ACTION_TAIL>')
|
|
.replace('Then run the pre-review audit, mockups, and Step 0 against that plan.', '<ACTION_TAIL>')
|
|
.replace('a path, a page, a doc they pasted,', 'a path, a doc they pasted,');
|
|
|
|
test('eng and design exceptions blocks are identical modulo the two variant slots', () => {
|
|
const eng = normalizeVariantSlots(extractExceptionsBlock('plan-eng-review'));
|
|
const design = normalizeVariantSlots(extractExceptionsBlock('plan-design-review'));
|
|
expect(eng).toBe(design);
|
|
// The action tail must actually have been normalized in both (guards
|
|
// against a rewording that bypasses the normalizer and vacuously passes).
|
|
expect(eng).toContain('<ACTION_TAIL>');
|
|
});
|
|
|
|
test('exceptions block carries the announcement string the PTY detectors pin', () => {
|
|
for (const skill of ['plan-eng-review', 'plan-design-review']) {
|
|
const block = extractExceptionsBlock(skill);
|
|
expect(block, `${skill}: verbatim announcement`).toContain(
|
|
'Scope gate: plan mode — auto-selected B (reviewing <target>).',
|
|
);
|
|
}
|
|
});
|
|
|
|
test('gate menu carries the question strings the PTY question detector pins', () => {
|
|
// isScopeGateQuestionVisible (claude-pty-runner.ts) anchors on the
|
|
// question text + option A's body. If the menu is reworded without
|
|
// updating the detector, the paid smokes' must-stay-false assertions go
|
|
// vacuous — this free pin fails first.
|
|
for (const skill of ['plan-eng-review', 'plan-design-review']) {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(md, `${skill}: gate question text`).toContain('What should I review?');
|
|
expect(md, `${skill}: option A body text`).toContain('The current branch diff');
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('GSTACK REVIEW REPORT mandatory unresolved-decisions status', () => {
|
|
// Report text rides in PLAN_FILE_REVIEW_REPORT → every report consumer gets it.
|
|
// devex-review is a report consumer but NOT a gate consumer, so the two target
|
|
// sets differ (CP5/CX5). Regression guard: a future token-cut that drops the
|
|
// unresolved-status line again fails here. See plan-flag-unresolved-issues.
|
|
const REPORT_CONSUMERS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'codex',
|
|
'devex-review',
|
|
];
|
|
// Gate text rides in EXIT_PLAN_MODE_GATE (lives in SKILL.md, not sections).
|
|
const GATE_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'codex',
|
|
];
|
|
|
|
for (const skill of REPORT_CONSUMERS) {
|
|
test(`${skill}: report mandates the unresolved-decisions status as final content`, () => {
|
|
const content = readSkillUnion(skill);
|
|
expect(content).toContain('NO UNRESOLVED DECISIONS');
|
|
// The "never omit / always final" contract must be present, not just the phrase.
|
|
expect(content).toContain('Unresolved-decisions status (MANDATORY');
|
|
expect(content).toMatch(/never omitted/);
|
|
// \s+ tolerates prose line-wraps within "final non-whitespace line".
|
|
expect(content).toMatch(/final\s+non-whitespace\s+line/);
|
|
});
|
|
}
|
|
|
|
for (const skill of GATE_SKILLS) {
|
|
test(`${skill}: exit gate blocks unless the unresolved status is the final line`, () => {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
// Gate check #4 — present, sentinel named, and explicitly blocking (no escape).
|
|
expect(md).toContain('NO UNRESOLVED DECISIONS');
|
|
expect(md).toContain('FINAL non-whitespace line is the unresolved-decisions');
|
|
expect(md).toContain('FAILS the gate');
|
|
});
|
|
}
|
|
|
|
test('scripts/resolvers/review.ts source carries the mandatory block + blocking gate', () => {
|
|
const src = fs.readFileSync(path.join(ROOT, 'scripts', 'resolvers', 'review.ts'), 'utf-8');
|
|
// Report resolver: mandatory, never-omitted, exact sentinel, anti-double-count algorithm.
|
|
expect(src).toContain('Unresolved-decisions status (MANDATORY');
|
|
expect(src).toContain('NO UNRESOLVED DECISIONS');
|
|
expect(src).toContain('avoids double-counting');
|
|
expect(src).toContain('DROP the current skill');
|
|
// Gate resolver: the blocking final-line check with no "if applicable" escape.
|
|
expect(src).toContain('FINAL non-whitespace line is the unresolved-decisions');
|
|
expect(src).toContain('FAILS the gate');
|
|
// The old soft wording must be gone from the gate.
|
|
expect(src).not.toContain('absorbs CODEX / CROSS-MODEL / UNRESOLVED lines if applicable');
|
|
});
|
|
});
|
|
|
|
// ─── {{PREAMBLE}} requires an explicit preamble-tier ────────
|
|
|
|
describe('PREAMBLE resolution requires declared preamble-tier', () => {
|
|
test('resolving {{PREAMBLE}} without preamble-tier throws with the template path', async () => {
|
|
const { generatePreamble } = await import('../scripts/resolvers/preamble');
|
|
const { HOST_PATHS } = await import('../scripts/resolvers/types');
|
|
const ctx = {
|
|
skillName: 'tierless-skill',
|
|
tmplPath: 'tierless-skill/SKILL.md.tmpl',
|
|
host: 'claude' as const,
|
|
paths: HOST_PATHS.claude,
|
|
// preambleTier deliberately absent — the generator must refuse to default it.
|
|
};
|
|
expect(() => generatePreamble(ctx)).toThrow(/tierless-skill\/SKILL\.md\.tmpl/);
|
|
expect(() => generatePreamble(ctx)).toThrow(/preamble-tier/);
|
|
});
|
|
|
|
test('every template that resolves {{PREAMBLE}} declares preamble-tier in frontmatter', () => {
|
|
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
|
|
const offenders: string[] = [];
|
|
const checkTmpl = (tmplPath: string) => {
|
|
const tmpl = fs.readFileSync(tmplPath, 'utf-8');
|
|
if (tmpl.includes('{{PREAMBLE}}') && !/^preamble-tier:\s*\d+$/m.test(tmpl)) {
|
|
offenders.push(path.relative(ROOT, tmplPath));
|
|
}
|
|
};
|
|
checkTmpl(path.join(ROOT, 'SKILL.md.tmpl'));
|
|
for (const e of entries) {
|
|
if (!e.isDirectory() || e.name.startsWith('.') || e.name === 'node_modules') continue;
|
|
const tmplPath = path.join(ROOT, e.name, 'SKILL.md.tmpl');
|
|
if (fs.existsSync(tmplPath)) checkTmpl(tmplPath);
|
|
}
|
|
expect(offenders).toEqual([]);
|
|
});
|
|
});
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// #2499: gbrain MCP detection must read BOTH ~/.claude.json scopes.
|
|
// Claude Code registers MCP servers at user scope (.mcpServers) and project
|
|
// scope (.projects["/abs/path"].mcpServers — what `claude mcp add` without
|
|
// --scope user writes). The rendered brain-sync block previously read only
|
|
// user scope, so a correctly configured project-scoped brain was invisible.
|
|
// ---------------------------------------------------------------------------
|
|
describe('brain-sync block reads project-scoped MCP registrations (#2499)', () => {
|
|
const rendered = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
|
|
test('rendered _GBRAIN_MCP_ENTRY jq resolves project scope with nearest-ancestor cwd match', () => {
|
|
const line = rendered.split('\n').find((l) => l.includes('_GBRAIN_MCP_ENTRY=$('));
|
|
expect(line).toBeDefined();
|
|
// Project-scope read present, driven by $PWD.
|
|
expect(line!).toContain('--arg cwd "$PWD"');
|
|
expect(line!).toContain('.projects');
|
|
// User scope still resolved first.
|
|
expect(line!).toContain('.mcpServers.gbrain');
|
|
// The old user-scope-only filter is gone from the rendered output.
|
|
expect(rendered).not.toContain('.mcpServers.gbrain.type // .mcpServers.gbrain.transport');
|
|
expect(rendered).not.toContain(".mcpServers.gbrain.url // empty");
|
|
});
|
|
|
|
test('rendered _GBRAIN_MCP_TYPE and _GBRAIN_HOST extract from the resolved entry', () => {
|
|
const typeLine = rendered.split('\n').find((l) => l.includes('_GBRAIN_MCP_TYPE=$('));
|
|
const hostLine = rendered.split('\n').find((l) => l.includes('_GBRAIN_HOST=$('));
|
|
expect(typeLine).toBeDefined();
|
|
expect(hostLine).toBeDefined();
|
|
expect(typeLine!).toContain('_GBRAIN_MCP_ENTRY');
|
|
expect(hostLine!).toContain('_GBRAIN_MCP_ENTRY');
|
|
});
|
|
|
|
test('rendered jq lines FUNCTION: project-scoped registration resolves for a cwd inside the project', () => {
|
|
// Execute the exact rendered bytes, not a re-derivation: extract the
|
|
// _GBRAIN_MCP_ENTRY + _GBRAIN_MCP_TYPE lines from the generated SKILL.md
|
|
// and run them in bash against a fixture ~/.claude.json that carries ONLY
|
|
// a project-scoped gbrain registration.
|
|
const lines = rendered.split('\n');
|
|
const entryLine = lines.find((l) => l.includes('_GBRAIN_MCP_ENTRY=$('));
|
|
const typeLine = lines.find((l) => l.includes('_GBRAIN_MCP_TYPE=$('));
|
|
expect(entryLine).toBeDefined();
|
|
expect(typeLine).toBeDefined();
|
|
|
|
const tmpHome = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-2499-home-'));
|
|
const projectDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-2499-proj-'));
|
|
const nestedCwd = path.join(projectDir, 'src', 'deep');
|
|
fs.mkdirSync(nestedCwd, { recursive: true });
|
|
try {
|
|
fs.writeFileSync(
|
|
path.join(tmpHome, '.claude.json'),
|
|
JSON.stringify({
|
|
projects: {
|
|
[projectDir]: {
|
|
mcpServers: { gbrain: { type: 'http', url: 'https://brain.example.com/mcp' } },
|
|
},
|
|
},
|
|
}),
|
|
);
|
|
const script = `cd "$1" || exit 1\n${entryLine!.trim()}\n${typeLine!.trim()}\necho "RESOLVED:$_GBRAIN_MCP_TYPE"`;
|
|
const r = spawnSync('bash', ['-c', script, 'bash', nestedCwd], {
|
|
encoding: 'utf-8',
|
|
env: { ...process.env, HOME: tmpHome },
|
|
timeout: 10_000,
|
|
});
|
|
expect(r.stdout).toContain('RESOLVED:http');
|
|
|
|
// Discriminator: a cwd OUTSIDE the project must NOT resolve it.
|
|
const outside = spawnSync('bash', ['-c', script, 'bash', os.tmpdir()], {
|
|
encoding: 'utf-8',
|
|
env: { ...process.env, HOME: tmpHome },
|
|
timeout: 10_000,
|
|
});
|
|
expect(outside.stdout).toContain('RESOLVED:\n');
|
|
} finally {
|
|
fs.rmSync(tmpHome, { recursive: true, force: true });
|
|
fs.rmSync(projectDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|