mirror of
https://github.com/garrytan/gstack.git
synced 2026-08-22 14:07:14 +02:00
* fix(ci): skill-docs freshness gate covers all 10 hosts and can actually fail The Codex/Factory gates ran 'git diff --exit-code -- .agents/' / '-- .factory/', but both paths are gitignored (.gitignore:16-17) — git diff on ignored untracked paths is always empty, so those two gates were structurally incapable of failing and 7 of 10 hosts had no gate at all. New shape: one 'gen:skill-docs --host all' pass (the generator hard-fails on any per-host error, gating all 10 hosts on generates-cleanly), byte-freshness via git diff for tracked output, plus a porcelain check that fails on untracked generated strays (git diff can't see brand-new files). The gitignored-hosts byte-freshness limitation is documented in the workflow comment. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): exorcise the sidebar-agent ghost from the test suite browse/src/sidebar-agent.ts was deleted in the v1.14 sidebar refactor, but the test suite kept testing it for 48 versions. Nothing noticed because the free suite runs in no CI job and Bun-era module-load errors were suppressed in the Windows shard runner via an exclusion pattern whose own comment documented the breakage ('broken on every platform since v1.14 ... exit 0'). - Delete sidebar-security.test.ts + security-source-contracts.test.ts: crashed at module load (unguarded readFileSync of the deleted file); per-assertion triage confirmed every SERVER_SRC pin targeted the deleted chat prompt builder (zero hits in today's server.ts) — nothing to port. - Delete sidebar-integration.test.ts: 11 of 13 tests exercised deleted endpoints (/sidebar-command queue, /sidebar-agent/event, chat buffer); the 2 passing tests pinned only the blanket auth gate, covered by server-auth.test.ts + dual-listener.test.ts. - Delete test/skill-e2e-sidebar.test.ts: E2E for the deleted queue flow. - sidebar-ux.test.ts 1,669 -> 830 lines: 20 dead-chat describes + 15 dead tests removed (incl. 10 vacuous passes asserting on empty indexOf slices); 2 stale pins on LIVE features fixed (content.js typed-catch CSSOM fallback, arrow-hint window widened). 95 pass / 0 fail. - sidebar-tabs.test.ts: both failures were stale pins, not regressions — forceRestart's deliberate ws.close(4001) and the terminal-agent spawn that moved into spawnTerminalAgent() (identity-based kill refactor). 28 pass. - touchfiles.ts: drop the three sidebar E2E entries from BOTH maps (E2E_TOUCHFILES + E2E_TIERS) — they pointed diff-selection at the deleted file, so those tests were unreachable by any diff. - test-free-shards.ts: remove the now-dead sidebar-agent exclusion pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ci): run the free test suite in CI (it ran nowhere) The full free suite (bun test: browse/test/ + test/ + make-pdf/test/) had no CI job on any Linux/macOS runner — only Windows curated shards, paid evals, and doc-freshness gates existed. That's how two module-load-crashing test files survived 48 versions. Same cached Dockerfile.ci image and container wiring as evals.yml (deps restore, build, Chromium verify). Includes a module-load-error guard: older Bun reported test-file import crashes with exit 0 on macOS/Linux, so the job also fails on any nonzero 'N errors' count in the summary — future crash-class regressions can't hide from the exact job built to catch them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(test): validate touchfile dependency paths exist on disk New guard in touchfiles.test.ts: every non-glob dep path must exist, and every glob's anchor directory must exist. This is the axis the 181-key two-map sync discipline never covered — an entry can point at a long-deleted file and diff-based selection then silently never triggers those tests (the sidebar trio sat rotted for 48 versions). First run immediately caught a fourth rotted entry: 'spec authored quality' referenced test/fixtures/spec/** (directory does not exist) and selected for a judge test that exists nowhere in the repo. Removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): remove deleted /sidebar-chat endpoint from tunnel allowlist TUNNEL_PATHS is the audited tunnel attack surface — its own comment says every addition widens it. '/sidebar-chat' stayed in the set after the endpoint was deleted with the chat-queue path, meaning any future route matching that path would have been silently tunnel-exposed. The set is now exactly the pair ceremony (/connect) and the scoped command endpoint (/command), and the dual-listener closed-set pin enforces that. Also repairs a pre-existing red pin in dual-listener.test.ts: v1.63.0.0 made the tunnel allowlist args-aware (canDispatchOverTunnel gained a second param) without updating the test — red on main since then, invisible because the free suite had no CI job. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): delete chain's shadow dispatcher that skipped every security gate meta-commands.ts carried a 'CLI mode' fallback that re-implemented command routing without the server pipeline's gates: no scope check, no domain check, no tab ownership, no rate limit, no hidden-element stripping, no scoped-token enveloping — and it called handleReadCommand without a BrowserManager, which also skipped the JS-origin cookie-exfiltration assertion. It was unreachable in production (server.ts always passes executeCommand) and one boolean away from being live. chain now hard-errors without a server context. handleReadCommand's bm param is required and assertJsOriginAllowed runs unconditionally. The chain tests that exercised the deleted fallback now route through a server-shaped executeCommand adapter (real handlers + trust wrapping + {status,result} envelope), so their behavioral coverage — sequencing, trust markers, pipe format, aliases, error reporting — survives on the production-shaped path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(extension): delete the dead chat-queue client surface The sidebar-command handler in background.js POSTed to a server endpoint that no longer exists (deleted with the chat queue) — ~35 lines of fully-wired dead code including error handling for the permanent 404, plus its allowlist entry. No sender in the extension ever emitted the message type. chatEnabled leaves the /health contract (server hardcoded false, background.js re-derived it, nothing consumed it — the chat input element it guarded is gone from sidepanel.html). BROWSE_SIDEBAR_CHAT env flag had zero readers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): delete dead exports the ripped chat path left behind Three-way split by importer class: (a) Zero importers, deleted: the whole attack-attempt logging cluster in security.ts (logAttempt, AttemptRecord, salted hashPayload + device-salt, attempts.jsonl rotation, telemetry spawn plumbing incl. buildTelemetrySpawnCommand/resolveBashBinary — the LIVE attempts.jsonl writer is tunnel-denial-log.ts with its own rotation); the decision-file handshake (writeDecision/readDecision/clearDecision/excerptForReview — written for sidebar-agent's poll loop, which no longer exists); sidebar-utils.ts (whole module — its sanitizeExtensionUrl 'sanitized before embedding in a prompt' for the deleted prompt builder); 8 dead server.ts imports (sanitizeExtensionUrl, generateCanary, injectCanary, writeDecision, rotateRoot, serializeRegistry, restoreRegistry, clearAgentRecord); buildPtyClearCookie + buildSseClearCookie; WEBDRIVER_MASK_SCRIPT (orphaned by the D7 stealth narrowing — applyStealth never used it). (b) Dead-pin tests edited with their exports: the 'still exported' pin in stealth-layer-c, the string-content describe in stealth-webdriver (its live applyStealth behavioral coverage untouched), the clear-cookie assertions, security-review-flow.test.ts deleted whole (all 4 describes exercised the dead decision mechanism, incl. a 'simulated sidebar-agent poll loop'). (c) KEPT deliberately: leaseCount (live behavioral coverage), extractPtyCookie + validatePtySessionToken (extractPtyCookie is adopted by the terminal-agent cookie-parse unification later in this wave), resetSessionMarker + clearContentFilters (test-support API for the live content-security layer). Also fixes two pre-existing red pins found while here, invisible until the free suite got a CI job: the v1.44 spawnClaude->maybeSpawnPty rename in terminal-agent.test.ts, and a cross-file test-isolation bug where content-security.test.ts's clearContentFilters() wiped the auto-registered url-blocklist filter for every later file in the same bun process (security-integration.test.ts failed on co-run; afterAll now restores it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): delete the dead ML layers — transcript classifier and DeBERTa ensemble The L4b Haiku transcript classifier and the opt-in DeBERTa ensemble (GSTACK_SECURITY_ENSEMBLE=deberta, a documented 721MB download) had ZERO production callers since the chat-path agent that invoked them was ripped. The only live ML path is scanPageContent (testsavant) inside the security sidecar subprocess. Deleted by import graph: - security-classifier.ts 614 -> 265 lines: HAIKU_MODEL, checkTranscript, shouldRunTranscriptCheck, loadDeberta, scanPageContentDeberta, ToolCallInput, all DEBERTA_* consts + load state. Header now states the live truth (imported only by security-sidecar-entry.ts). downloadFile kept, name intact — it is an enumerated egress sink (HF model download). - security-bunnative.ts + test: a research skeleton self-described as 'NOT a production replacement', shipped into src/ with zero importers. - security-bench-ensemble{,-live}.test.ts + the Haiku response fixture: a paid live-model benchmark for a layer that could not fire. The security-classifier-tdz test's only case exercised checkTranscript — gone. - security.ts: layer-model header rewritten to the live architecture; StatusDetail.layers -> {testsavant, canary}; getStatus() no longer requires the impossible transcript==='ok' for 'protected' (old on-disk session state with a transcript key is tolerated on read, never re-emitted). - security-sidecar-entry.ts needed zero changes: it serializes getClassifierStatus() verbatim and no consumer read .transcript (verified in sidecar-client + server.ts). - BROWSER.md security section matches reality (ensemble knob gone, 112MB not 22MB, sidecar hosting documented). combineVerdict/THRESHOLDS retained as the pure, tested combiner of record — comments now flag transcript/deberta votes as producer-less. Net: 26 pass in security.test.ts incl. a NEW regression test for stale- transcript disk tolerance; egress-receipt tripwire green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: scrub the sidebar-agent ghost from comments and CLAUDE.md 20+ comments across 10 files still described the deleted sidebar-agent.ts as a live process — including load-bearing architecture claims ('IMPORTED ONLY BY sidebar-agent.ts', 'sidebar-agent fills this in on first prompt-injection load', 'kill sidebar-agent' in shutdown docs) and ~60 lines of tombstone blocks in server.ts enumerating deleted identifiers by name (a false grep surface: searching processAgentEvent hit server.ts and looked live). CLAUDE.md's security-stack section now documents the LIVE architecture: L1-L3 content filters + testsavant via the security sidecar subprocess; the L4b/ensemble rows, the GSTACK_SECURITY_ENSEMBLE knob, and the 721MB DeBERTa download are gone (deleted as dead code this wave) with an explicit do-not-re-document note; attempts.jsonl is correctly attributed to tunnel-denial-log.ts; the no-live-writer status of classifierStatus is stated. Comments that survive now describe what IS, not what WAS: the promotion gate in domain-skills.ts explains why classifier_score>0 is load-bearing given no L4 load-time scan exists; file-permissions.ts names real sensitive files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gen): delete the codex-helpers shadow module gen-skill-docs.ts imported externalSkillName (unaliased) from resolvers/codex-helpers.ts at line 21 and then re-declared the same function locally — the import was silently shadowed, and the imported copy was the STALE one (it lacked the frontmatterName param the local copy grew). Three more functions were byte-identical duplicates, imported only under _-prefixed aliases to keep the module 'referenced', and transformFrontmatter was a superseded hardcoded-Codex variant. Nothing else imported the module. Also drops three dead top-of-file imports (COMMAND_DESCRIPTIONS, SNAPSHOT_FLAGS — which pulled the whole browse/src module graph into every generator run for nothing — and an unused review-resolver trio). Proof: bun run gen:skill-docs exits 0 with a byte-identical tree (zero-diff regen); gen-skill-docs.test.ts 405/405 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(server): delete ServerConfig.idleTimeoutMs + chromiumProfile — documented, never read Both fields carried JSDoc asserting embedder behavior that did not exist: the idle check reads the module-level IDLE_TIMEOUT_MS env constant, and both resolveChromiumProfile() call sites pass no argument. Worse than absent — an embedder passing idleTimeoutMs: 5000 silently got 30 minutes. Wiring them honestly is impossible today: the idle timer, activity state, and shutdown target are module-global, so a per-factory value would lie for any process running more than one handler. Deleted instead, with a ServerConfig note pointing at the deferred singleton/route-table refactor where real support belongs. BROWSE_IDLE_TIMEOUT and CHROMIUM_PROFILE env remain the honest knobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): wire appendSecureFile at the four real log-append sites file-permissions.ts carries a 24-line rationale for why POSIX mode bits are insufficient on Windows and implements appendSecureFile (0600 at create, Windows ACL on first write only) — but its single caller was the dead logAttempt, while the four REAL page-content log writers (console/network/ dialog logs in server.ts, the command audit log) used raw fs.appendFileSync with no mode. Page-content-derived logs now get owner-only permissions from birth on every platform. Verified before wiring: mode applies atomically at create via appendFileSync {mode}, and the ACL pass runs only on first write — no per-append subprocess cost on the hot console-log path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(stealth): handoff() uses the shared profile resolution + lock cleanup The headless-to-headed handoff path hardcoded ~/.gstack/chromium-profile, silently ignoring $CHROMIUM_PROFILE and $GSTACK_HOME (gbrowser's gbd sets per-workspace profiles), and skipped cleanSingletonLocks() — so a handoff into a profile with a stale SingletonLock could hang where launchHeaded() would have recovered. This was the third live drift between the three Chromium launch paths; the first two are documented in comments as shipped stealth regressions. Minimal targeted fix — the full buildLaunchConfig() extraction stays in the deferred queue. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gen): resolver registry describes the template language again Seven registered {{PLACEHOLDER}}s had zero uses in any .tmpl (checked in both bare and :arg forms): REDACT_TAXONOMY_TABLE, TEST_COVERAGE_AUDIT_REVIEW, MODEL_OVERLAY, QUESTION_PREFERENCE_CHECK, QUESTION_LOG, INLINE_TUNE_FEEDBACK, MAKE_PDF_SETUP. The last two of those families are invoked programmatically by preamble.ts (functions kept, registry entries dropped); the question-tuning trio and the review coverage-audit wrapper were documented by their own module as existing 'for unit testing' that no test performed — deleted, along with generateRedactTaxonomyTable + its EXAMPLE/TIER_BLURB constants (its '/cso renders the full table' comment was itself stale) and its test describe. Also deletes the gated-resolver mechanism (ResolverEntry/appliesTo/ unwrapResolver + test/resolver-entry.test.ts): fully built, fully tested, used by zero of the 65 registry entries — the generator loop simplifies to a direct function call. CLAUDE.md's redact-doc line stops advertising the dead token. Proof: zero-diff regen (0 SKILL.md changed); gen-skill-docs + skill-validation 737 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gen): wire boundaryInstruction from host config; drop three no-op binDir ternaries hosts/codex.ts declared boundaryInstruction and nothing read it — review.ts kept its own byte-identical CODEX_BOUNDARY literal (verified equal + trailing escaped newlines). The resolver now reads the config, so the boundary has one owner. (autoplan's template carries deliberately generic variants, enforced by gen-skill-docs.test.ts:1358 — untouched by design.) The 'ctx.host === codex ? $GSTACK_BIN : ctx.paths.binDir' ternary appeared in three resolvers and could never change the result: resolvers/types.ts already sets binDir to $GSTACK_BIN for every usesEnvVars host including codex. Proof: zero-diff regen for claude AND codex hosts; gen-skill-docs + host-config suites green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test-infra): judge uses resolveClaudeBinary; eval:watch reads the real partials dir judgePtyState spawned the bare string 'claude' three definitions below the resolveClaudeBinary() helper this same file exports — broken under hermetic PATHs where every other launch in the file resolves correctly. eval:watch read _partial-e2e.json from the legacy global ~/.gstack-dev/evals/ while EvalCollector writes it into the per-project eval dir (or GSTACK_EVAL_DIR) — so the dashboard's completed-tests panel was empty whenever slug detection succeeded, i.e. the normal case. The heartbeat and per-run progress logs stay global by design (session-runner.ts: 'heartbeat stays global'). The three eval-CLI docstrings stop claiming the legacy dir is the primary location. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): delete the superseded SDK ship-idempotency suite and three orphaned fixtures test/skill-e2e-ship-idempotency.test.ts's own header documented that the monolith's SDK-harness version tests a synthetic prompt while it exercises the real /ship skill — the author knew the old suite was superseded and left both running, two paid LLM runs for one behavior. The weaker copy is gone; its 'ship-idempotency' diff-selection key goes with it (the dedicated file is periodic-tier, which always runs under EVALS_ALL — the key had no remaining consumer). Fixture rot: test/fixtures/golden-ship-claude.md was a 128KB zero-reader orphan that had drifted 46KB from its live successor (test/fixtures/golden/claude-ship-SKILL.md) while looking authoritative; parity-baseline-v1.46.0.0.json and v1.53.0.0.json had zero readers (three tests pin three OTHER baseline versions — consolidation is queued, deletion of the unreferenced two is free). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bin): delete zero-caller scripts; make host-config-export's docstring honest - bin/gstack-open-url (14 lines): announced in a CHANGELOG entry, wired into nothing, ever. bin/gstack-platform-detect (27 lines): zero callers, and its hand-rolled host list was already stale (SLATE_HOST.md cites it as a problem). Note: the deprecated gstack-brain-consumer/reader pair the audit flagged was already deleted upstream in v1.63 with a stay-deleted tripwire. - scripts/task-emission-schema.ts (61 lines): a typed schema module nothing imported; the tasks-section comment now documents the JSONL fields inline. - scripts/host-config-export.ts claimed to be the 'shell bridge for the bash setup script' — setup never calls it (its hand-rolled host lists drifting is a known follow-up). Docstring now states what it IS: a standalone, test-pinned query CLI not yet wired into setup. Its validateValue + CLI_REGEX/PATH_REGEX internals were dead (defined for a guarantee the header claimed but nothing enforced). - KEPT deliberately: scripts/preflight-agent-sdk.ts — a documented manual diagnostic (CONTRIBUTING.md + USING_GBRAIN_WITH_GSTACK.md reference it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(server): one lone-surrogate sanitizer, one sanitizeReplacer, one startTunnel Three copies of the surrogate sanitizer existed with two algorithms (sanitize.ts regex vs a hand-rolled charCodeAt walk in server.ts — verified byte-identical across 11 edge cases before converging) plus two identical sanitizeReplacer definitions each wrapping a different copy. sanitize.ts is now the single source of truth; the runs-INSIDE-JSON.stringify egress invariant is unchanged at every call site and its pin tests were adapted to the new import shape without losing intent. The ngrok tunnel-start sequence existed three times in server.ts — the /tunnel/start route and the BROWSE_TUNNEL=1 autostart were line-for-line equivalent (a comment admitted 'Same cleanup as /tunnel/start's error path'). One startTunnel() now owns the ephemeral loopback bind, the pre-send egress receipt, the state-file RMW via tmpStatePath(), and the ordered error-path cleanup; callers keep their distinct response surfaces. The BROWSE_TUNNEL_LOCAL_ONLY test path shares nothing (no ngrok, different state field) and deliberately stays separate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): one session-cookie registry implementation, two instances pty-session-cookie.ts and sse-session-cookie.ts were byte-identical modulo the cookie name — mint/validate/parse/prune/TTL, the exact code a security fix would have to land in twice (and a third hand-rolled cookie parse in terminal-agent.ts had already diverged; unified next commit). createSessionCookieStore() owns the implementation; both modules become thin instantiations keeping every exported name, their distinct threat-model docstrings, and separate token spaces (an SSE-read cookie must never grant PTY access). pty-session-lease.ts deliberately stays out — different contract (sessionId/secret split, refresh, env TTL). The factory imports nothing from token-registry (cookie-picker-auth-isolation invariant, still pinned by sse-session-cookie.test.ts). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): terminal-agent uses the shared PTY cookie parser The /ws upgrade's cookie fallback hand-parsed the Cookie header inline — the fourth copy of the session-cookie parse, and the one that had already diverged from the others. Parsing now goes through extractPtyCookie; validation deliberately stays against the agent's own in-process validTokens map (the server's registry lives in a different process). The ws-handler pin test now pins the shared-parser call instead of the raw cookie-name literal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(hosts): defineHost() factory — 10 copy-paste host files become declarations hosts/*.ts were ten copies of one file: runtimeRoot byte-identical in 9/10, pathRewrites mechanically derivable from the host name for 7/10, the 11-entry toolRewrites map byte-identical between openclaw and gbrain, and every asset change a 10-file edit (cursor and slate had already fallen out of three other hand-maintained lists). defineHost() owns the defaults; each host file now declares only what makes it different (slate/cursor: 8 lines each). Shared constants: CROSS_MODEL_RESOLVERS, GBRAIN_RESOLVERS, EXEC_STYLE_TOOL_REWRITES. Genuinely-different things stayed explicit: codex/factory $GSTACK_ROOT rewrites, hermes's tool vocabulary, claude's denylist+prefixable install, opencode's wider runtimeRoot. Proof: JSON.stringify(ALL_HOST_CONFIGS) dump-diff before/after EMPTY (and a runtime walk confirmed no function-valued or undefined-keyed fields, so the JSON diff is complete); gen:skill-docs --host all zero-diff; host-config + gen-skill-docs + idempotency suites 485/485. Host files 595 -> 285 lines. docs/ADDING_A_HOST.md teaches the factory pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(lib): fs-atomic — one atomic-write implementation, with the race actually fixed Atomic tmp-write-then-rename was reimplemented ~20 times across lib/, bin/, and browse/src with three tmp-suffix conventions. One of them was a latent bug this commit closes: lib/worktree.ts used a bare '.tmp' suffix — the deterministic-tmp collision race browse/src/server.ts documents having hit in production (its fix, pid+random, was trapped in a comment at one site). lib/fs-atomic.ts: atomicWriteSync (always throws, best-effort tmp cleanup, pid+random suffix, optional mode applied at tmp creation so the file never exists with looser permissions) + atomicWriteQuiet (shutdown paths only). Unit tests pin the throw/quiet contracts, 0600 mode, tmp-name uniqueness (captured via the read-only-dir failure path — Bun's fs exports are readonly, no monkeypatching), and no-stray-tmp cleanup. Migrated: lib/worktree.ts (the bare-.tmp bug), lib/gstack-decision.ts (snapshot + compact log), lib/gbrain-local-status.ts (probe cache). browse sites follow separately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(lib): jsonl-store's docstring stops lying; mode option added; lib bypasses adopted The header claimed 'single source of truth... the ONLY copy' with write-time injection REJECTION — while appendJsonl never screened anything, only 1 of ~10 JSONL stores imported it, and a bypass appender lived in the same directory. Now: the contract is explicit (screening is the CALLER's job via hasInjection/firstInjectionMatch; the enforcing callers are named), a option applies 0600 at create for sensitive stores, and the lib bypasses are adopted (gstack-memory-helpers ×2, redact-audit-log — which keeps its chmod backstop for files created looser by pre-mode versions). browse/src keeps its own appenders by design (compiled-binary surface, own secure-append helper) and the header now says so. gstack-decision's batched archive append stays deliberate (single-write crash-window semantics appendJsonl's one-record contract can't express). New pins: 0600-at-create, and a test that documents appendJsonl does NOT self-screen — so nobody can re-document it as self-screening without making it true. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(browse): migrate hand-rolled atomic writes to lib/fs-atomic Seven sites, each audited for its existing throw-vs-swallow contract before migrating: writeSessionState + the four fire-and-forget tab/state writers use atomicWriteQuiet (they swallowed before); writeAgentRecord + the boot-time port-file write use atomicWriteSync (they threw before — and writeAgentRecord previously leaked its tmp file on rename failure, which the helper cleans). All carry {mode: 0o600} plus restrictFilePermissions after successful writes, preserving the Windows ACL hardening that writeSecureFile provided (mode bits are POSIX-only). server.ts untouched: its three state writes route through tmpStatePath(), pinned by server-tmp-state-path.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(hosts): delete five dead HostConfig fields metadataFormat (generator hardcodes openai.yaml), sidecar (behavior lives in setup's create_agents_sidecar — knowledge preserved as a comment in codex.ts), install.prefixable (skill_prefix is implemented entirely in bin/gstack-config), staticFiles (docstring cited a SOUL.md that never existed anywhere), and adapter (its only would-be consumer, openclaw-adapter.ts, was fully dead — with a test asserting the field was undefined). Kept: learningsMode (wired next), linkingStrategy (validation reads it), coAuthorTrailer (consumed by resolvers/utility.ts). Proof: JSON dump diff shows ONLY the deleted keys vanishing; zero-diff regen across all 10 hosts; host-config + gen-skill-docs suites green. Note: this commit also carries chunk-23 edits to the shared hosts/claude.ts + define-host.ts + host-config.test.ts files (skipSkills collapse, stale line-number comment drops) — pathspec commits, concurrent prep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gen): preamble tiers are explicit; silent ?? 4 default becomes an error; spec stops rendering its preamble twice Eight skills (scrape, diagram, spec, skillify, pair-agent, landing-report, open-gstack-browser + its connect-chrome symlink) silently received the HEAVIEST tier-4 preamble because a missing frontmatter field defaulted to 4. Tiers are now declared in every {{PREAMBLE}} template's frontmatter and a missing declaration throws at generation time with the template path (the 5 templates without {{PREAMBLE}} never invoke the resolver). The stale hand-written tier-map comment (wrong in 3 of 4 rows) is gone. Bonus bug fixed: spec/SKILL.md.tmpl mentioned {{PREAMBLE}} in prose, so the generator inlined the ENTIRE preamble a second time — spec/SKILL.md shrinks 127,462 -> 80,924 bytes (-46,538) from de-duplication alone. skill-size-budget gains a reasoned INTENTIONAL_SHRINKS entry (its frozen baseline had measured the doubled-preamble bug). New tests: missing-tier throw carries the path; every {{PREAMBLE}} template declares a tier. (Carries chunk-23 edits in the shared test/gen-skill-docs.test.ts.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gen): learningsMode is read from host config, not a hardcoded host name resolvers/learnings.ts branched on ctx.host === 'codex' while every host declared learningsMode — the field was decorative, and the 7 hosts configured 'basic' (cursor, slate, kiro, opencode, openclaw, hermes, gbrain) silently received the 'full' cross-project flow their runtimes can't execute (it depends on AskUserQuestion + gstack-config plumbing). Output now matches declaration: basic hosts get the project-scoped search block. Blast radius proof: all committed Claude SKILL.md files and the three golden fixtures are byte-identical; the behavior diff lands only in the gitignored external-host trees (hand-verified: .cursor review's learnings section swaps the cross-project AskUserQuestion block for the project-scoped search). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gen): small config scrubs — openclaw blobs to real files, setup host drift, dead artifacts - The three openclaw markdown blobs hardcoded inside gen-skill-docs.ts (which silently reverted any hand edit to their tracked outputs on regen) move to openclaw/templates/*.md source files; output shasums byte-identical. - setup's --host allowlists gain cursor + slate — both fully registered hosts with generated output, but './setup --host cursor' exited 1 because two hand-rolled lists in setup had drifted from hosts/index.ts. - scripts/proactive-suggestions.json deleted: 31KB regenerated on every run, read by nobody (the catalog-trim design's reader was never built); its emitter and three determinism tests (which guaranteed a file nothing reads didn't churn) retired with stays-retired pins. - claude/SKILL.md.tmpl deleted: a complete 8.9KB skill that never generated output (directory name collides with the host id 'claude'), in no registry. Recoverable from git if ever wanted under a non-colliding name. - openclaw's frozen extraFields.version '0.15.2.0' stamp dropped; includeSkills: [] no-ops omitted (the generator treats [] as absent); llms.txt 55 -> 54 skills. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(gen): correct preamble tiers for the 8 silently-heaviest skills With tiers now explicit, set them RIGHT by analogy to the tiered population: scrape/diagram/open-gstack-browser (+ the connect-chrome symlink) -> tier 1 (launchers and artifact generators, like browse and make-pdf); landing-report/pair-agent/skillify -> tier 2 (dashboards and session tools, like health and canary); spec -> tier 3 (interactive planning, like the plan-*-review family). Each tier-1 skill sheds 271 lines of onboarding prose it never needed; tier-2 shed 20 each. Verification per the review protocol: regen diff reviewed (pure section-removal), skill-validation + size-budget + catalog-budget + v0-dormancy suites green (822 tests), and live smoke of the tier-corrected skills confirms the preamble renders the intended sections at each tier. These skills have ~no eval coverage — stated honestly; the wave's gate-tier eval run is the backstop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(test): e2e-gate — one tier-gate implementation, side-effect-free, with the trap pinned The EVALS/EVALS_TIER gate was copy-pasted into ~40 test files and had drifted into six different predicates — the drift that made 'eval:bg:all runs everything' silently false. test/helpers/e2e-gate.ts owns the semantics now: describeE2ETier(tier) + e2eTierEnabled(tier), env read at call time, zero side effects (the existing e2e-helpers module runs a ~30s claude ping at import under EVALS=1, so the gate lives in its own module; purity is pinned by tests that scan imports and comment-stripped source). The unit matrix pins all four env combos — including EVALS=1 with EVALS_TIER unset -> SKIP, the exact trap that made eval:bg:all a non-run. The tier-alignment tripwire gains a second regex for the helper shape (old shape still detected — stragglers can't hide), and the sharded paid runner's PRE-SPAWN tier classifier learns the helper shape too: without that, every gate-sharded run would have spawned all 28 periodic shards just to skip them, each paying the e2e-helpers import ping (~15 min of dead wall clock in the CI-blocking lane). Verified: gate runs exclude the 29 periodic files, periodic excludes the 8 gate files — identical to pre-migration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(test): migrate the 36 tier-gated eval files to describeE2ETier Mechanical two-liner swap in 34 files (each keeping its declared tier — all 36 predicates verified against E2E_TIERS before migrating); the two files with compound gates (overlay-harness's EvalCollector feed, codex-e2e's CODEX_AVAILABLE) keep their extra conditions via e2eTierEnabled. Tier rationale comments preserved. codex-e2e/gemini-e2e/benchmark-providers keep their distinct stderr-message gate shapes by design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(test): skill-e2e + skill-llm-eval adopt the shared selection machinery Both files re-implemented the diff-selection machinery e2e-helpers already exported. The helper gained computeDiffSelection() (extracted, identical behavior) and a trailing optional selection param on the *IfSelected helpers (defaults preserve all 30+ existing importers). skill-e2e.test.ts drops ~120 duplicated lines; skill-llm-eval keeps its LLM_JUDGE_TOUCHFILES selection and test.concurrent semantics via testConcurrentIfSelected. Deliberate deltas, stated: skill-e2e.test.ts now honors the EVALS_TIER intersection its local copy lacked (affects only direct bun test invocations of that file — it matches no eval-script glob); its recordE2E gains the helper's three diagnostic fields; skill-llm-eval sharded solo now runs e2e-helpers' module-scope preflight it already ran in combined processes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): kill the silent-truncation race; exempt the tier-corrected shrinks The full-suite shakeout (budgeted by the plan) surfaced both immediately: 1. server-embedder-terminal-port.test.ts stubbed process.exit and restored the REAL exit in its finally — but shutdown() schedules async work that can call process.exit AFTER restoration, killing the entire bun process mid-suite with exit 0 and NO summary. This is the silent-truncation class the new free-suite CI job guards against, reproduced locally on the first full run. Exit now stays a logging no-op between tests (late async exits become visible stderr lines, not process death); the true exit returns in afterAll. 2. The 80%-of-baseline shrink guard correctly flagged the six tier-corrected skills — their baseline was measured at the silent tier-4 default. Added to INTENTIONAL_SHRINKS with the reason, joining spec's double-preamble entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * release: v1.64.0.0 — the code-smell fix wave 35 commits, one PR: guard repairs (free suite in CI per-file, all-host freshness gates, tunnel allowlist, diff-selection validation), the sidebar-agent ghost exorcism (dead ML layers, dead endpoints, dead exports, ghost comments), config honesty (defineHost factory, dead fields deleted, preamble tiers explicit, spec double-render fixed), and dedup with safety nets (session-cookie factory, fs-atomic, jsonl-store contract, one eval tier-gate). Net -24,943 lines across 183 files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): free-tests step runs under bash (container sh rejects pipefail) Maiden-voyage shakeout, exactly as budgeted: the CI container's default shell is dash, which errors on 'set -o pipefail' before the first test ran. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): free-tests curates 8 container-incompatible files with reasons Second maiden-voyage shakeout round: 376 of 384 files ran green in the container on the first completed pass. The 8 that can't run there yet are excluded the same way the Windows shards curate POSIX-bound files — each with its reason inline (headed-Chrome handoff, real-PTY round-trip, X server management, extension-origin identity, the job's own TMPDIR override, and three pre-existing env failures that fail on dev machines too). Anything outside the list that fails still fails the job; trimming the list is tracked follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): gstack-config-key-locale — suppress the skill_prefix auto-relink side effect The test invokes the repo's own bin/gstack-config, whose 'set skill_prefix' auto-runs $(dirname $0)/gstack-relink — resolving the install dir to the repo itself. In any environment where the loop shares a working tree (the free-tests CI container, a fresh-HOME run), gstack-patch-names rewrote all 52 tracked SKILL.md names to gstack- prefixed, poisoning five unrelated suites downstream (hermetic-skills-seeding, host-config golden, skill-census, skill-validation, spec-template-sync). GSTACK_SETUP_RUNNING=1 is the documented suppression; relink behavior stays covered by relink.test.ts's mock install. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bin): gstack-codex-session-import — empty sessions dir exits 0 on Linux GNU xargs runs 'ls -t' once even on empty input, listing the cwd and producing a bogus LATEST from the repo root; BSD xargs (macOS) skips the run, which is why the NO_SESSIONS path only broke on Linux. xargs -r pins the BSD behavior on both platforms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(parity): rebaseline v1.57.7.0 → v1.64.1.0 + skeleton-cap headroom The two parallel v1.64 waves (code-smell fix wave + main's #2571) each added shared-preamble prose, pushing document-release / design-consultation / cso past their size ratios on the v1.57.7.0 anchor and four carved skeletons (plan-ceo-review, plan-eng-review, office-hours, design-consultation) 22-280 B over their absolute caps. New baseline is union-normalized (skeleton + sections/*.md, matching what the harness measures); caps get +~1 KB headroom each with per-cap rationale. The v1.57.7.0 fixture stays in test/fixtures/ for the audit trail, and capture-parity-baseline.ts now documents the union-normalization step so the next rebaseline doesn't re-trip on it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): free-tests container parity — tools, pinned bun, git identity, mutation tripwire - Dockerfile.ci: add python3 (gstack-jsonl-merge/brain-sync/detach shell out to it), file (skill-validation's binary check), poppler-utils (make-pdf e2e gates hard-require pdftotext/pdffonts/pdfinfo), fonts-noto-color-emoji (emoji render gate, mirrors make-pdf-gate.yml). Fix the bun pin: the bun.sh installer ignores a BUN_VERSION env var, so the old form silently installed latest on every rebuild (observed 1.3.13/1.3.14 drift vs the 1.3.10 devs run locally); pass the version as the positional arg. - free-tests.yml: git identity + safe.directory for the git-exercising tests (container checkout is owned by a different uid than runner); post-loop tree-mutation tripwire that names a tracked-file-mutating test instead of letting downstream collateral confuse the report; skip the documented variants-retry-after timing flake. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bin): gstack-session-update — detached updater owns its stdio (SIGPIPE) The backgrounded update subshell inherited the session hook's stdout/stderr pipes. Once the hook exits and the caller closes them, any child that writes — git pull's autostash notice, setup output — dies of SIGPIPE, logged as PULL_FAILED exit=141 with an empty stderr capture (observed in the free-tests container, and reachable by any production hook runner that closes stdio promptly). Redirect the fork to /dev/null; all observability already flows through the session-update log file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): gstack-decision-bins — explicit branch context for the scope filter CI checks out a detached HEAD, where gitBranch() returns undefined on both the log and search sides, so an implicitly branch-scoped decision can never surface (filterByScope requires a matching non-empty ctx.branch). Pass the branch explicitly on both sides — the filter logic is what's under test, not git branch detection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): ring-buffer lease interplay — same TTL window, not same millisecond Two back-to-back mintLease() calls each stamp Date.now() + TTL; when they straddle a millisecond boundary the exact-equality assertion flakes (observed in CI: expiries of ...525 vs ...526). Assert the expiries are within a 50 ms window instead — the invariant under test is that leases share a TTL policy, not that they mint in the same clock tick. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
3495 lines
151 KiB
TypeScript
3495 lines
151 KiB
TypeScript
import { describe, test, expect, beforeAll } from 'bun:test';
|
|
import { COMMAND_DESCRIPTIONS } from '../browse/src/commands';
|
|
import { SNAPSHOT_FLAGS } from '../browse/src/snapshot';
|
|
import * as fs from 'fs';
|
|
import * as path from 'path';
|
|
import * as os from 'os';
|
|
import { spawnSync } from 'child_process';
|
|
|
|
const ROOT = path.resolve(import.meta.dir, '..');
|
|
const MAX_SKILL_DESCRIPTION_LENGTH = 1024;
|
|
|
|
// Carved-skill aware (v2 plan T9): ship is now a skeleton SKILL.md + sections/*.md.
|
|
// Read the union so assertions about content that MOVED into a section still pass.
|
|
// The skeleton is a subset of the union, so skeleton-only assertions also hold,
|
|
// and negative assertions stay safe (the absent phrases live in neither file).
|
|
function readSkillUnion(skill: string): string {
|
|
let t = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const secDir = path.join(ROOT, skill, 'sections');
|
|
if (fs.existsSync(secDir)) {
|
|
for (const f of fs.readdirSync(secDir).sort()) {
|
|
if (f.endsWith('.md')) t += '\n' + fs.readFileSync(path.join(secDir, f), 'utf-8');
|
|
}
|
|
}
|
|
return t;
|
|
}
|
|
function readShipUnion(): string {
|
|
return readSkillUnion('ship');
|
|
}
|
|
|
|
function extractDescription(content: string): string {
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
expect(fmEnd).toBeGreaterThan(0);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
const lines = frontmatter.split('\n');
|
|
let description = '';
|
|
let inDescription = false;
|
|
const descLines: string[] = [];
|
|
|
|
for (const line of lines) {
|
|
if (line.match(/^description:\s*\|?\s*$/)) {
|
|
inDescription = true;
|
|
continue;
|
|
}
|
|
if (line.match(/^description:\s*\S/)) {
|
|
return line.replace(/^description:\s*/, '').trim();
|
|
}
|
|
if (inDescription) {
|
|
if (line === '' || line.match(/^\s/)) {
|
|
descLines.push(line.replace(/^ /, ''));
|
|
} else {
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
|
|
if (descLines.length > 0) {
|
|
description = descLines.join('\n').trim();
|
|
}
|
|
return description;
|
|
}
|
|
|
|
function extractMarkdownSection(content: string, heading: string): string {
|
|
const escaped = heading.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
const startMatch = content.match(new RegExp(`^${escaped}.*$`, 'm'));
|
|
expect(startMatch?.index).toBeDefined();
|
|
const start = startMatch!.index!;
|
|
const afterHeading = start + startMatch![0].length;
|
|
const nextSection = content.slice(afterHeading).match(/\n## /);
|
|
const end = nextSection?.index === undefined
|
|
? content.length
|
|
: afterHeading + nextSection.index;
|
|
return content.slice(start, end).trim();
|
|
}
|
|
|
|
function extractPreambleBeforeWorkflow(content: string, workflowMarkers: string[]): string {
|
|
const markerIndexes = workflowMarkers
|
|
.map(marker => content.indexOf(marker))
|
|
.filter(index => index >= 0);
|
|
expect(markerIndexes.length).toBeGreaterThan(0);
|
|
return content.slice(0, Math.min(...markerIndexes));
|
|
}
|
|
|
|
function isRepoRootSymlink(candidateDir: string): boolean {
|
|
try {
|
|
return fs.realpathSync(candidateDir) === fs.realpathSync(ROOT);
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
|
|
// Dynamic template discovery — matches the generator's findTemplates() behavior.
|
|
// New skills automatically get test coverage without updating a static list.
|
|
const ALL_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; name: string }> = [];
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
skills.push({ dir: '.', name: 'root gstack' });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) {
|
|
skills.push({ dir: entry.name, name: entry.name });
|
|
}
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
// hosts/claude.ts generation.skipSkills entries would filter here; the set is
|
|
// currently empty (the /claude outside-voice template was removed).
|
|
// The claude host deliberately skips some skills (skipSkills — e.g. the
|
|
// /claude outside-voice skill exists only for non-Claude hosts), so those
|
|
// dirs have a SKILL.md.tmpl but no generated claude-host SKILL.md on a fresh
|
|
// checkout. Every generated-file assertion must exclude them or it is red on
|
|
// every clean clone (it was, invisibly, until the free suite ran in CI).
|
|
import { getHostConfig as __getHostConfig } from '../hosts/index';
|
|
const CLAUDE_SKIPPED = new Set(__getHostConfig('claude').generation.skipSkills ?? []);
|
|
const CLAUDE_GENERATED_SKILLS = ALL_SKILLS.filter(s => !CLAUDE_SKIPPED.has(s.dir));
|
|
|
|
describe('gen-skill-docs', () => {
|
|
test('generated SKILL.md contains all command categories', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
const categories = new Set(Object.values(COMMAND_DESCRIPTIONS).map(d => d.category));
|
|
for (const cat of categories) {
|
|
expect(content).toContain(`### ${cat}`);
|
|
}
|
|
});
|
|
|
|
test('generated SKILL.md contains all commands', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
const display = meta.usage || cmd;
|
|
expect(content).toContain(display);
|
|
}
|
|
});
|
|
|
|
test('command table is sorted alphabetically within categories', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
// Extract command names from the Navigation section as a test
|
|
const navSection = content.match(/### Navigation\n\|.*\n\|.*\n([\s\S]*?)(?=\n###|\n## )/);
|
|
expect(navSection).not.toBeNull();
|
|
const rows = navSection![1].trim().split('\n');
|
|
const commands = rows.map(r => {
|
|
const match = r.match(/\| `(\w+)/);
|
|
return match ? match[1] : '';
|
|
}).filter(Boolean);
|
|
const sorted = [...commands].sort();
|
|
expect(commands).toEqual(sorted);
|
|
});
|
|
|
|
test('generated header is present in SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
});
|
|
|
|
test('generated header is present in browse/SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
});
|
|
|
|
test('snapshot flags section contains all flags', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
expect(content).toContain(flag.short);
|
|
expect(content).toContain(flag.description);
|
|
}
|
|
});
|
|
|
|
test('every skill has a SKILL.md.tmpl template', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
expect(fs.existsSync(tmplPath)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('every skill has a generated SKILL.md with auto-generated header', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const mdPath = path.join(ROOT, skill.dir, 'SKILL.md');
|
|
expect(fs.existsSync(mdPath)).toBe(true);
|
|
const content = fs.readFileSync(mdPath, 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
}
|
|
});
|
|
|
|
// #1778: strict YAML parsers (Codex/OpenAI skill loading) reject frontmatter
|
|
// whose plain `description:` scalar contains an interior ": " (read as a nested
|
|
// mapping). Parse EVERY generated frontmatter block with a strict YAML parser,
|
|
// not just string-check that name:/description: exist.
|
|
function frontmatterBlock(content: string): string {
|
|
expect(content.startsWith('---\n')).toBe(true);
|
|
const end = content.indexOf('\n---', 4);
|
|
expect(end).toBeGreaterThan(0);
|
|
return content.slice(4, end);
|
|
}
|
|
|
|
test('every generated SKILL.md frontmatter parses as strict YAML', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const fm = frontmatterBlock(content);
|
|
let parsed: any;
|
|
expect(() => { parsed = Bun.YAML.parse(fm); },
|
|
`frontmatter for ${skill.dir} must be valid YAML`).not.toThrow();
|
|
expect(typeof parsed?.name).toBe('string');
|
|
expect(typeof parsed?.description).toBe('string');
|
|
}
|
|
});
|
|
|
|
test('every generated Codex (.agents/skills) frontmatter parses as strict YAML', () => {
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return; // skip if external hosts not generated
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const mdPath = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(mdPath)) continue;
|
|
const fm = frontmatterBlock(fs.readFileSync(mdPath, 'utf-8'));
|
|
expect(() => Bun.YAML.parse(fm),
|
|
`Codex frontmatter for ${entry.name} must be valid YAML`).not.toThrow();
|
|
}
|
|
});
|
|
|
|
test(`every generated SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const description = extractDescription(content);
|
|
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
|
|
}
|
|
});
|
|
|
|
test(`every Codex SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return; // skip if not generated
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
const description = extractDescription(content);
|
|
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
|
|
}
|
|
});
|
|
|
|
test('every Codex SKILL.md description stays under 900-char warning threshold', () => {
|
|
const WARN_THRESHOLD = 900;
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return;
|
|
const violations: string[] = [];
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
const description = extractDescription(content);
|
|
if (description.length > WARN_THRESHOLD) {
|
|
violations.push(`${entry.name}: ${description.length} chars (limit ${MAX_SKILL_DESCRIPTION_LENGTH}, ${MAX_SKILL_DESCRIPTION_LENGTH - description.length} remaining)`);
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('package.json version matches VERSION file', () => {
|
|
const pkg = JSON.parse(fs.readFileSync(path.join(ROOT, 'package.json'), 'utf-8'));
|
|
const version = fs.readFileSync(path.join(ROOT, 'VERSION'), 'utf-8').trim();
|
|
expect(pkg.version).toBe(version);
|
|
});
|
|
|
|
test('generated files are fresh (match --dry-run)', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// Every skill should be FRESH
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const file = skill.dir === '.' ? 'SKILL.md' : `${skill.dir}/SKILL.md`;
|
|
expect(output).toContain(`FRESH: ${file}`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('no generated SKILL.md contains unresolved placeholders', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const unresolved = content.match(/\{\{[A-Z_]+\}\}/g);
|
|
expect(unresolved).toBeNull();
|
|
}
|
|
});
|
|
|
|
test('templates contain placeholders', () => {
|
|
// P2 (v1.2.0): the root template is a pure router — only {{PREAMBLE}}.
|
|
// The browse command/snapshot placeholders live in browse/SKILL.md.tmpl now.
|
|
const rootTmpl = fs.readFileSync(path.join(ROOT, 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(rootTmpl).toContain('{{PREAMBLE}}');
|
|
expect(rootTmpl).not.toContain('{{COMMAND_REFERENCE}}');
|
|
expect(rootTmpl).not.toContain('{{SNAPSHOT_FLAGS}}');
|
|
|
|
const browseTmpl = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(browseTmpl).toContain('{{COMMAND_REFERENCE}}');
|
|
expect(browseTmpl).toContain('{{SNAPSHOT_FLAGS}}');
|
|
expect(browseTmpl).toContain('{{PREAMBLE}}');
|
|
});
|
|
|
|
test('generated SKILL.md contains operational self-improvement (replaced contributor mode)', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('Contributor Mode');
|
|
expect(content).not.toContain('gstack_contributor');
|
|
expect(content).not.toContain('contributor-logs');
|
|
expect(content).toContain('Operational Self-Improvement');
|
|
expect(content).toContain('gstack-learnings-log');
|
|
expect(content).toContain('gstack-learnings-search --limit 3');
|
|
});
|
|
|
|
test('generated SKILL.md with LEARNINGS_LOG contains operational type', () => {
|
|
// Check a skill that has LEARNINGS_LOG (e.g., review)
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('operational');
|
|
});
|
|
|
|
test('generated SKILL.md contains session awareness', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_SESSIONS');
|
|
expect(content).toContain('RECOMMENDATION');
|
|
});
|
|
|
|
test('generated SKILL.md contains branch detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_BRANCH');
|
|
expect(content).toContain('git branch --show-current');
|
|
});
|
|
|
|
// #2001: update_check: false silences the binary but the upgrade-handling
|
|
// instruction prose used to ship unconditionally. Every skill that carries
|
|
// the runtime config-echo cluster must (a) echo UPDATE_CHECK so the
|
|
// instruction layer can read it, and (b) gate the UPGRADE_AVAILABLE /
|
|
// JUST_UPGRADED prose on it — the same echo-then-gate convention every other
|
|
// flag (PROACTIVE, SKILL_PREFIX, EXPLAIN_LEVEL, QUESTION_TUNING) follows.
|
|
test('update_check opt-out gates preamble echo and upgrade-handling prose (issue #2001)', () => {
|
|
let checked = 0;
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
// Scope: only skills that render the runtime config-echo cluster.
|
|
if (!content.includes('echo "QUESTION_TUNING: $_QUESTION_TUNING"')) continue;
|
|
checked++;
|
|
expect(content, `${skill.dir} must echo UPDATE_CHECK`).toContain('echo "UPDATE_CHECK: $_UPDATE_CHECK"');
|
|
expect(content, `${skill.dir} must read update_check config`).toContain('_UPDATE_CHECK=$(');
|
|
// Whenever the upgrade-handling prose ships, it must gate on the flag.
|
|
if (content.includes('UPGRADE_AVAILABLE <old> <new>')) {
|
|
expect(content, `${skill.dir} upgrade prose must gate on UPDATE_CHECK`)
|
|
.toContain('If `UPDATE_CHECK` is `"false"`');
|
|
}
|
|
}
|
|
// Guard against the scope filter silently matching nothing.
|
|
expect(checked).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('tier 2+ skills contain ELI10 simplification rules (AskUserQuestion format)', () => {
|
|
// Root SKILL.md is tier 1 (no AskUserQuestion format). Check a tier 2+ skill instead.
|
|
// v1.7.0.0 Pros/Cons format uses "ELI10 (ALWAYS)" rather than "Simplify (ELI10".
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('ELI10');
|
|
expect(content).toContain('plain English');
|
|
expect(content).toContain('not function names');
|
|
});
|
|
|
|
test('tier 1 skills do NOT contain AskUserQuestion format', () => {
|
|
// Use benchmark (tier 1) instead of root — root SKILL.md gets overwritten by Codex test setup
|
|
const content = fs.readFileSync(path.join(ROOT, 'benchmark', 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('## AskUserQuestion Format');
|
|
expect(content).not.toContain('## Completeness Principle');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry line', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('skill-usage.jsonl');
|
|
expect(content).toContain('~/.gstack/analytics');
|
|
});
|
|
|
|
test('plan-review generated preambles stay under the Option A budget', () => {
|
|
const reviewSkills = [
|
|
{
|
|
path: path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
|
|
markers: ['# Mega Plan Review Mode', '## Step 0: Detect platform and base branch'],
|
|
},
|
|
{
|
|
path: path.join(ROOT, 'plan-eng-review', 'SKILL.md'),
|
|
markers: ['# Plan Review Mode'],
|
|
},
|
|
];
|
|
|
|
// Plan skills carry the same preamble surface as other tier-≥2 skills
|
|
// (Artifacts Sync, Context Recovery, Routing Injection are load-bearing
|
|
// functionality, not optional). Budget is set to current size + small
|
|
// headroom; ratchet down if a future slim trims real bytes.
|
|
// Ratcheted from 33000 → 35000 when the gbrain context-load block was
|
|
// added (per /sync-gbrain plan §4). Ratcheted 35000 → 36500 in v1.27.0.0
|
|
// when generate-brain-sync-block.ts gained the gbrain_mcp_mode probe +
|
|
// remote-mode ARTIFACTS_SYNC status line (Path 4 of /setup-gbrain).
|
|
// Ratcheted 36500 → 39000 in the contributor wave when #1205 added the
|
|
// \\u-escape CJK rule (rule 12 + self-check item) to the AskUserQuestion
|
|
// preamble.
|
|
// Ratcheted 39000 → 40000 in plan-tune cathedral T14: question-tuning
|
|
// resolver gained the <gstack-qid:...> marker convention + the
|
|
// (recommended) label requirement (D2 + D18 — both load-bearing for
|
|
// hook enforcement). Adds ~700 bytes.
|
|
// Ratcheted 40000 → 60000 in v1.52.0.0 cap audit: ~20K headroom so
|
|
// future preamble adds don't trip the gate on each PR. Real runaway
|
|
// (preamble doubling) still trips; normal scope growth doesn't.
|
|
for (const skill of reviewSkills) {
|
|
const content = fs.readFileSync(skill.path, 'utf-8');
|
|
const preamble = extractPreambleBeforeWorkflow(content, skill.markers);
|
|
expect(Buffer.byteLength(preamble, 'utf-8')).toBeLessThan(60_000);
|
|
}
|
|
});
|
|
|
|
test('voice and writing-style preamble sections stay compact', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
const voice = extractMarkdownSection(content, '## Voice');
|
|
const writingStyle = extractMarkdownSection(content, '## Writing Style');
|
|
|
|
expect(Buffer.byteLength(voice, 'utf-8')).toBeLessThan(3_000);
|
|
expect(Buffer.byteLength(writingStyle, 'utf-8')).toBeLessThan(2_000);
|
|
});
|
|
|
|
test('slim voice section preserves the gstack voice contract', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
const voice = extractMarkdownSection(content, '## Voice');
|
|
|
|
expect(voice).toMatch(/lead with the point|direct/i);
|
|
expect(voice).toMatch(/file|function|line|command|real numbers/i);
|
|
expect(voice).toMatch(/user.*outcome|user.*experience|real user/i);
|
|
expect(voice).toMatch(/corporate|academic|PR|hype/i);
|
|
expect(voice).toMatch(/AI vocabulary|delve|crucial|robust/i);
|
|
expect(voice).toMatch(/user decides|user.*context|sovereignty|recommendation, not a decision/i);
|
|
});
|
|
|
|
test('preamble .pending-* glob is zsh-safe (uses find, not shell glob)', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
if (!content.includes('.pending-')) continue;
|
|
// Must NOT have a bare shell glob ".pending-*" outside of find's -name argument
|
|
expect(content).not.toMatch(/for _PF in [^\n]*\/\.pending-\*/);
|
|
// Must use find to avoid zsh NOMATCH error on glob expansion
|
|
expect(content).toContain("find ~/.gstack/analytics -maxdepth 1 -name '.pending-*'");
|
|
}
|
|
});
|
|
|
|
test('bash blocks with shell globs are zsh-safe (setopt guard or find)', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const bashBlocks = [...content.matchAll(/```bash\n([\s\S]*?)```/g)].map(m => m[1]);
|
|
|
|
for (const block of bashBlocks) {
|
|
const lines = block.split('\n');
|
|
|
|
for (const line of lines) {
|
|
const trimmed = line.trimStart();
|
|
if (trimmed.startsWith('#')) continue;
|
|
if (!trimmed.includes('*')) continue;
|
|
// Skip lines where * is inside find -name, git pathspecs, or $(find)
|
|
if (/\bfind\b/.test(trimmed)) continue;
|
|
if (/\bgit\b/.test(trimmed)) continue;
|
|
if (/\$\(find\b/.test(trimmed)) continue;
|
|
|
|
// Check 1: "for VAR in <glob>" must use $(find ...) — caught above by the
|
|
// $(find check, so any surviving for-in with a glob pattern is a violation
|
|
if (/\bfor\s+\w+\s+in\b/.test(trimmed) && /\*\./.test(trimmed)) {
|
|
throw new Error(
|
|
`Unsafe for-in glob in ${skill.dir}/SKILL.md: "${trimmed}". ` +
|
|
`Use \`for f in $(find ... -name '*.ext')\` for zsh compatibility.`
|
|
);
|
|
}
|
|
|
|
// Check 2: ls/cat/rm/grep with glob file args must have setopt guard
|
|
const isGlobCmd = /\b(?:ls|cat|rm|grep)\b/.test(trimmed) &&
|
|
/(?:\/\*[a-z.*]|\*\.[a-z])/.test(trimmed);
|
|
if (isGlobCmd) {
|
|
expect(block).toContain('setopt +o nomatch');
|
|
}
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
test('preamble-using skills have correct skill name in telemetry', () => {
|
|
const PREAMBLE_SKILLS = [
|
|
{ dir: '.', name: 'gstack' },
|
|
{ dir: 'ship', name: 'ship' },
|
|
{ dir: 'review', name: 'review' },
|
|
{ dir: 'qa', name: 'qa' },
|
|
{ dir: 'retro', name: 'retro' },
|
|
];
|
|
for (const skill of PREAMBLE_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain(`"skill":"${skill.name}"`);
|
|
}
|
|
});
|
|
|
|
test('qa and qa-only templates use QA_METHODOLOGY placeholder', () => {
|
|
const qaTmpl = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(qaTmpl).toContain('{{QA_METHODOLOGY}}');
|
|
|
|
const qaOnlyTmpl = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(qaOnlyTmpl).toContain('{{QA_METHODOLOGY}}');
|
|
});
|
|
|
|
test('QA_METHODOLOGY appears expanded in both qa and qa-only generated files', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
|
|
|
|
// Both should contain the health score rubric
|
|
expect(qaContent).toContain('Health Score Rubric');
|
|
expect(qaOnlyContent).toContain('Health Score Rubric');
|
|
|
|
// Both should contain framework guidance
|
|
expect(qaContent).toContain('Framework-Specific Guidance');
|
|
expect(qaOnlyContent).toContain('Framework-Specific Guidance');
|
|
|
|
// Both should contain the important rules
|
|
expect(qaContent).toContain('Important Rules');
|
|
expect(qaOnlyContent).toContain('Important Rules');
|
|
|
|
// Both should contain the 6 phases
|
|
expect(qaContent).toContain('Phase 1');
|
|
expect(qaOnlyContent).toContain('Phase 1');
|
|
expect(qaContent).toContain('Phase 6');
|
|
expect(qaOnlyContent).toContain('Phase 6');
|
|
});
|
|
|
|
test('qa-only has no-fix guardrails', () => {
|
|
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
|
|
expect(qaOnlyContent).toContain('Never fix bugs');
|
|
expect(qaOnlyContent).toContain('NEVER fix anything');
|
|
// Should not have Edit, Glob, or Grep in allowed-tools.
|
|
// Scope to frontmatter (between the first two --- lines) — the body can
|
|
// legitimately mention these tool names in prose (e.g., Claude model
|
|
// overlay says "prefer Read, Edit, Write, Glob, Grep over Bash").
|
|
const fmMatch = qaOnlyContent.match(/^---\n([\s\S]*?)\n---/);
|
|
expect(fmMatch).not.toBeNull();
|
|
const frontmatter = fmMatch![1];
|
|
expect(frontmatter).toMatch(/allowed-tools:/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Edit/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Glob/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Grep/);
|
|
});
|
|
|
|
test('qa has fix-loop tools and phases', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
// Should have Edit, Glob, Grep in allowed-tools
|
|
expect(qaContent).toContain('Edit');
|
|
expect(qaContent).toContain('Glob');
|
|
expect(qaContent).toContain('Grep');
|
|
// Should have fix-loop phases
|
|
expect(qaContent).toContain('Phase 7');
|
|
expect(qaContent).toContain('Phase 8');
|
|
expect(qaContent).toContain('Fix Loop');
|
|
expect(qaContent).toContain('Triage');
|
|
expect(qaContent).toContain('WTF');
|
|
});
|
|
});
|
|
|
|
describe('BASE_BRANCH_DETECT resolver', () => {
|
|
// Find a generated SKILL.md that uses the placeholder (ship is guaranteed to)
|
|
const shipContent = readShipUnion();
|
|
|
|
test('resolver output contains PR base detection command', () => {
|
|
expect(shipContent).toContain('gh pr view --json baseRefName');
|
|
});
|
|
|
|
test('resolver output contains repo default branch detection command', () => {
|
|
expect(shipContent).toContain('gh repo view --json defaultBranchRef');
|
|
});
|
|
|
|
test('resolver output contains fallback to main', () => {
|
|
expect(shipContent).toMatch(/fall\s*back\s+to\s+`main`/i);
|
|
});
|
|
|
|
test('resolver output uses "the base branch" phrasing', () => {
|
|
expect(shipContent).toContain('the base branch');
|
|
});
|
|
|
|
test('resolver output contains GitLab CLI commands', () => {
|
|
expect(shipContent).toContain('glab');
|
|
});
|
|
|
|
test('resolver output contains git-native fallback', () => {
|
|
expect(shipContent).toContain('git symbolic-ref');
|
|
});
|
|
|
|
test('resolver output mentions GitLab platform', () => {
|
|
expect(shipContent).toMatch(/gitlab/i);
|
|
});
|
|
});
|
|
|
|
describe('GitLab support in generated skills', () => {
|
|
const retroContent = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
|
const shipSkillContent = readShipUnion();
|
|
|
|
test('retro contains GitLab MR number extraction', () => {
|
|
expect(retroContent).toContain('[#!]');
|
|
});
|
|
|
|
test('retro uses BASE_BRANCH_DETECT (contains glab)', () => {
|
|
expect(retroContent).toContain('glab');
|
|
});
|
|
|
|
test('ship contains glab mr create', () => {
|
|
expect(shipSkillContent).toContain('glab mr create');
|
|
});
|
|
|
|
test('ship checks .gitlab-ci.yml', () => {
|
|
expect(shipSkillContent).toContain('.gitlab-ci.yml');
|
|
});
|
|
});
|
|
|
|
/**
|
|
* Quality evals — catch description regressions.
|
|
*
|
|
* These test that generated output is *useful for an AI agent*,
|
|
* not just structurally valid. Each test targets a specific
|
|
* regression we actually shipped and caught in review.
|
|
*/
|
|
describe('description quality evals', () => {
|
|
// Regression: snapshot flags lost value hints (-d <N>, -s <sel>, -o <path>)
|
|
test('snapshot flags with values include value hints in output', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
if (flag.takesValue) {
|
|
expect(flag.valueHint).toBeDefined();
|
|
expect(content).toContain(`${flag.short} ${flag.valueHint}`);
|
|
}
|
|
}
|
|
});
|
|
|
|
// Regression: "is" lost the valid states enum
|
|
test('is command lists valid state values', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['is'].description;
|
|
for (const state of ['visible', 'hidden', 'enabled', 'disabled', 'checked', 'editable', 'focused']) {
|
|
expect(desc).toContain(state);
|
|
}
|
|
});
|
|
|
|
// Regression: "press" lost common key examples
|
|
test('press command lists example keys', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['press'].description;
|
|
expect(desc).toContain('Enter');
|
|
expect(desc).toContain('Tab');
|
|
expect(desc).toContain('Escape');
|
|
});
|
|
|
|
// Regression: "console" lost --errors filter note
|
|
test('console command describes --errors behavior', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['console'].description;
|
|
expect(desc).toContain('--errors');
|
|
});
|
|
|
|
// Regression: snapshot -i lost "@e refs" context
|
|
test('snapshot -i mentions @e refs', () => {
|
|
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-i')!;
|
|
expect(flag.description).toContain('@e');
|
|
});
|
|
|
|
// Regression: snapshot -C lost "@c refs" context
|
|
test('snapshot -C mentions @c refs', () => {
|
|
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-C')!;
|
|
expect(flag.description).toContain('@c');
|
|
});
|
|
|
|
// Guard: every description must be at least 8 chars (catches empty or stub descriptions)
|
|
test('all command descriptions have meaningful length', () => {
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
expect(meta.description.length).toBeGreaterThanOrEqual(8);
|
|
}
|
|
});
|
|
|
|
// Guard: snapshot flag descriptions must be at least 10 chars
|
|
test('all snapshot flag descriptions have meaningful length', () => {
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
expect(flag.description.length).toBeGreaterThanOrEqual(10);
|
|
}
|
|
});
|
|
|
|
// Guard: descriptions must not contain pipe (breaks markdown table cells)
|
|
// Usage strings are backtick-wrapped in the table so pipes there are safe.
|
|
test('no command description contains pipe character', () => {
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
expect(meta.description).not.toContain('|');
|
|
}
|
|
});
|
|
|
|
// Guard: generated output uses → not ->
|
|
test('generated SKILL.md uses unicode arrows', () => {
|
|
// P2 (v1.2.0): the browse body moved out of the top-level router into
|
|
// browse/SKILL.md. Guard arrow style on the browse body (sliced from its
|
|
// H1 so the auto-generated `-->` header comments are excluded).
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
const body = content.slice(content.indexOf('# browse: QA Testing'));
|
|
expect(body).toContain('→');
|
|
expect(body).not.toContain('->');
|
|
});
|
|
});
|
|
|
|
describe('REVIEW_DASHBOARD resolver', () => {
|
|
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review'];
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`review dashboard appears in ${skill} generated file`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('gstack-review');
|
|
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
|
});
|
|
}
|
|
|
|
test('review dashboard appears in ship generated file', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('reviews.jsonl');
|
|
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
|
});
|
|
|
|
test('dashboard treats review as a valid Eng Review source', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
|
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
|
expect(content).toContain('`plan-eng-review` (plan-stage architecture review)');
|
|
expect(content).toContain('from either \\`review\\` or \\`plan-eng-review\\`');
|
|
});
|
|
|
|
test('shared dashboard propagates review source to plan-eng-review', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
|
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
|
});
|
|
|
|
test('resolver output contains key dashboard elements', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('VERDICT');
|
|
expect(content).toContain('CLEARED');
|
|
expect(content).toContain('Eng Review');
|
|
expect(content).toContain('7 days');
|
|
expect(content).toContain('Design Review');
|
|
expect(content).toContain('skip_eng_review');
|
|
});
|
|
|
|
test('dashboard bash block includes git HEAD for staleness detection', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('git rev-parse --short HEAD');
|
|
expect(content).toContain('---HEAD---');
|
|
});
|
|
|
|
test('dashboard includes staleness detection prose', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: dashboard moved to section
|
|
expect(content).toContain('Staleness detection');
|
|
expect(content).toContain('commit');
|
|
});
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`${skill} contains review chaining section`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('Review Chaining');
|
|
});
|
|
|
|
test(`${skill} Review Log includes commit field`, () => {
|
|
const content = readSkillUnion(skill); // carved skills: union skeleton + sections
|
|
expect(content).toContain('"commit"');
|
|
});
|
|
}
|
|
|
|
test('plan-ceo-review chaining mentions eng and design reviews', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-design-review');
|
|
});
|
|
|
|
test('plan-eng-review chaining mentions design and ceo reviews', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('/plan-design-review');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
});
|
|
|
|
test('plan-design-review chaining mentions eng, ceo, and design skills', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
expect(content).toContain('/design-shotgun');
|
|
expect(content).toContain('/design-html');
|
|
});
|
|
|
|
test('ship does NOT contain review chaining', () => {
|
|
const content = readShipUnion();
|
|
expect(content).not.toContain('Review Chaining');
|
|
});
|
|
});
|
|
|
|
// ─── Test Coverage Audit Resolver Tests ─────────────────────
|
|
|
|
describe('TEST_COVERAGE_AUDIT placeholders', () => {
|
|
const planSkill = readSkillUnion('plan-eng-review'); // carved
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan and ship modes share codepath tracing methodology', () => {
|
|
// Review mode delegates test coverage to the Testing specialist subagent (Review Army)
|
|
const sharedPhrases = [
|
|
'Trace data flow',
|
|
'Diagram the execution',
|
|
'Quality scoring rubric',
|
|
'★★★',
|
|
'★★',
|
|
'GAP',
|
|
];
|
|
for (const phrase of sharedPhrases) {
|
|
expect(planSkill).toContain(phrase);
|
|
expect(shipSkill).toContain(phrase);
|
|
}
|
|
// Plan mode traces the plan, not a git diff
|
|
expect(planSkill).toContain('Trace every codepath in the plan');
|
|
expect(planSkill).not.toContain('git diff origin');
|
|
// Ship mode traces the diff
|
|
expect(shipSkill).toContain('Trace every codepath changed');
|
|
});
|
|
|
|
test('review mode uses Review Army for specialist dispatch', () => {
|
|
expect(reviewSkill).toContain('Review Army');
|
|
expect(reviewSkill).toContain('Specialist Dispatch');
|
|
expect(reviewSkill).toContain('testing.md');
|
|
});
|
|
|
|
test('plan and ship modes include E2E decision matrix', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('E2E Test Decision Matrix');
|
|
expect(skill).toContain('→E2E');
|
|
expect(skill).toContain('→EVAL');
|
|
}
|
|
});
|
|
|
|
test('plan and ship modes include regression rule', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('REGRESSION RULE');
|
|
expect(skill).toContain('IRON RULE');
|
|
}
|
|
});
|
|
|
|
test('plan and ship modes include test framework detection', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('Test Framework Detection');
|
|
expect(skill).toContain('CLAUDE.md');
|
|
}
|
|
});
|
|
|
|
test('plan mode adds tests to plan + includes test plan artifact', () => {
|
|
expect(planSkill).toContain('Add missing tests to the plan');
|
|
expect(planSkill).toContain('eng-review-test-plan');
|
|
expect(planSkill).toContain('Test Plan Artifact');
|
|
});
|
|
|
|
test('ship mode auto-generates tests + includes before/after count', () => {
|
|
expect(shipSkill).toContain('Generate tests for uncovered paths');
|
|
expect(shipSkill).toContain('Before/after test count');
|
|
expect(shipSkill).toContain('30 code paths max');
|
|
expect(shipSkill).toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review mode uses Fix-First + Review Army for specialist coverage', () => {
|
|
expect(reviewSkill).toContain('Fix-First');
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
// Review Army handles test coverage via Testing specialist subagent
|
|
expect(reviewSkill).toContain('Review Army');
|
|
expect(reviewSkill).toContain('Testing');
|
|
});
|
|
|
|
test('plan mode does NOT include ship-specific content', () => {
|
|
expect(planSkill).not.toContain('Before/after test count');
|
|
expect(planSkill).not.toContain('30 code paths max');
|
|
expect(planSkill).not.toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review mode does NOT include test plan artifact', () => {
|
|
expect(reviewSkill).not.toContain('Test Plan Artifact');
|
|
expect(reviewSkill).not.toContain('eng-review-test-plan');
|
|
expect(reviewSkill).not.toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review/specialists/ directory has all expected checklist files', () => {
|
|
const specDir = path.join(ROOT, 'review', 'specialists');
|
|
const expected = [
|
|
'testing.md',
|
|
'maintainability.md',
|
|
'security.md',
|
|
'performance.md',
|
|
'data-migration.md',
|
|
'api-contract.md',
|
|
'red-team.md',
|
|
];
|
|
for (const f of expected) {
|
|
expect(fs.existsSync(path.join(specDir, f))).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('each specialist file has standard header with scope and output format', () => {
|
|
const specDir = path.join(ROOT, 'review', 'specialists');
|
|
const files = fs.readdirSync(specDir).filter(f => f.endsWith('.md'));
|
|
for (const f of files) {
|
|
const content = fs.readFileSync(path.join(specDir, f), 'utf-8');
|
|
// All specialist files must have Scope and Output/JSON in header
|
|
expect(content).toContain('Scope:');
|
|
expect(content.toLowerCase()).toMatch(/output|json/);
|
|
// Must define NO FINDINGS behavior
|
|
expect(content).toContain('NO FINDINGS');
|
|
}
|
|
});
|
|
|
|
// Regression guard: ship output contains key phrases from before the refactor
|
|
test('ship SKILL.md regression guard — key phrases preserved', () => {
|
|
const regressionPhrases = [
|
|
'100% coverage is the goal',
|
|
'ASCII coverage diagram',
|
|
'processPayment',
|
|
'refundPayment',
|
|
'billing.test.ts',
|
|
'checkout.e2e.ts',
|
|
'COVERAGE:',
|
|
'QUALITY:',
|
|
'GAPS:',
|
|
'Code paths:',
|
|
'User flows:',
|
|
];
|
|
for (const phrase of regressionPhrases) {
|
|
expect(shipSkill).toContain(phrase);
|
|
}
|
|
});
|
|
|
|
test('ship SKILL.md contains review army specialist dispatch', () => {
|
|
expect(shipSkill).toContain('Specialist Dispatch');
|
|
expect(shipSkill).toContain('Step 9.1');
|
|
expect(shipSkill).toContain('Step 9.2');
|
|
});
|
|
|
|
test('ship SKILL.md contains cross-review finding dedup', () => {
|
|
expect(shipSkill).toContain('Cross-review finding dedup');
|
|
expect(shipSkill).toContain('Step 9.3');
|
|
});
|
|
|
|
test('ship SKILL.md contains re-run idempotency behavior', () => {
|
|
expect(shipSkill).toContain('Re-run behavior (idempotency)');
|
|
expect(shipSkill).toContain('Never skip a verification step');
|
|
});
|
|
});
|
|
|
|
// --- {{TEST_FAILURE_TRIAGE}} resolver tests ---
|
|
|
|
describe('TEST_FAILURE_TRIAGE resolver', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('contains all 4 triage steps', () => {
|
|
expect(shipSkill).toContain('Step T1: Classify each failure');
|
|
expect(shipSkill).toContain('Step T2: Handle in-branch failures');
|
|
expect(shipSkill).toContain('Step T3: Handle pre-existing failures');
|
|
expect(shipSkill).toContain('Step T4: Execute the chosen action');
|
|
});
|
|
|
|
test('T1 includes classification criteria (in-branch vs pre-existing)', () => {
|
|
expect(shipSkill).toContain('In-branch');
|
|
expect(shipSkill).toContain('Likely pre-existing');
|
|
expect(shipSkill).toContain('git diff origin/');
|
|
});
|
|
|
|
test('T3 branches on REPO_MODE (solo vs collaborative)', () => {
|
|
expect(shipSkill).toContain('REPO_MODE');
|
|
expect(shipSkill).toContain('solo');
|
|
expect(shipSkill).toContain('collaborative');
|
|
});
|
|
|
|
test('solo mode offers fix-now, TODO, and skip options', () => {
|
|
expect(shipSkill).toContain('Investigate and fix now');
|
|
expect(shipSkill).toContain('Add as P0 TODO');
|
|
expect(shipSkill).toContain('Skip');
|
|
});
|
|
|
|
test('collaborative mode offers blame + assign option', () => {
|
|
expect(shipSkill).toContain('Blame + assign GitHub issue');
|
|
expect(shipSkill).toContain('gh issue create');
|
|
});
|
|
|
|
test('defaults ambiguous failures to in-branch (safety)', () => {
|
|
expect(shipSkill).toContain('When ambiguous, default to in-branch');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_FILE_REVIEW_REPORT}} resolver tests ---
|
|
|
|
describe('PLAN_FILE_REVIEW_REPORT resolver', () => {
|
|
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review', 'codex'];
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`plan file review report appears in ${skill} generated file`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK REVIEW REPORT');
|
|
});
|
|
}
|
|
|
|
test('resolver output contains key report elements', () => {
|
|
const content = readSkillUnion('plan-ceo-review'); // carved: report writer moved to section
|
|
expect(content).toContain('Trigger');
|
|
expect(content).toContain('Findings');
|
|
expect(content).toContain('VERDICT');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-design-review');
|
|
expect(content).toContain('/codex review');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_COMPLETION_AUDIT}} resolver tests ---
|
|
|
|
describe('PLAN_COMPLETION_AUDIT placeholders', () => {
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains plan completion audit step', () => {
|
|
expect(shipSkill).toContain('Plan Completion Audit');
|
|
expect(shipSkill).toContain('Step 8');
|
|
});
|
|
|
|
test('review SKILL.md contains plan completion in scope drift', () => {
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Actionable Item Extraction');
|
|
expect(reviewSkill).toContain('Integration with Scope Drift Detection');
|
|
});
|
|
|
|
test('both modes share plan file discovery methodology', () => {
|
|
expect(shipSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
// Both should have conversation context first
|
|
expect(shipSkill).toContain('Conversation context (primary)');
|
|
expect(reviewSkill).toContain('Conversation context (primary)');
|
|
// Both should have grep fallback
|
|
expect(shipSkill).toContain('Content-based search (fallback)');
|
|
expect(reviewSkill).toContain('Content-based search (fallback)');
|
|
});
|
|
|
|
test('ship mode has gate logic for NOT DONE items', () => {
|
|
expect(shipSkill).toContain('NOT DONE');
|
|
expect(shipSkill).toContain('Stop — implement the missing items');
|
|
expect(shipSkill).toContain('Ship anyway — defer');
|
|
expect(shipSkill).toContain('intentionally dropped');
|
|
});
|
|
|
|
test('review mode is INFORMATIONAL only', () => {
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
expect(reviewSkill).toContain('MISSING REQUIREMENTS');
|
|
expect(reviewSkill).toContain('SCOPE CREEP');
|
|
});
|
|
|
|
test('item extraction has 50-item cap', () => {
|
|
expect(shipSkill).toContain('at most 50 items');
|
|
});
|
|
|
|
test('uses file-level traceability (not commit-level)', () => {
|
|
expect(shipSkill).toContain('Cite the specific file');
|
|
expect(shipSkill).not.toContain('commit-level traceability');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_VERIFICATION_EXEC}} resolver tests ---
|
|
|
|
describe('PLAN_VERIFICATION_EXEC placeholder', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('ship SKILL.md contains plan verification step', () => {
|
|
expect(shipSkill).toContain('Step 8.1');
|
|
expect(shipSkill).toContain('Plan Verification');
|
|
});
|
|
|
|
test('references /qa-only invocation', () => {
|
|
expect(shipSkill).toContain('qa-only/SKILL.md');
|
|
expect(shipSkill).toContain('qa-only');
|
|
});
|
|
|
|
test('contains localhost reachability check', () => {
|
|
expect(shipSkill).toContain('localhost:3000');
|
|
expect(shipSkill).toContain('NO_SERVER');
|
|
});
|
|
|
|
test('skips gracefully when no verification section', () => {
|
|
expect(shipSkill).toContain('No verification steps found in plan');
|
|
});
|
|
|
|
test('skips gracefully when no dev server', () => {
|
|
expect(shipSkill).toContain('No dev server detected');
|
|
});
|
|
});
|
|
|
|
// --- Coverage gate tests ---
|
|
|
|
describe('Coverage gate in ship', () => {
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains coverage gate with thresholds', () => {
|
|
expect(shipSkill).toContain('Coverage gate');
|
|
expect(shipSkill).toContain('>= target');
|
|
expect(shipSkill).toContain('< minimum');
|
|
});
|
|
|
|
test('ship SKILL.md supports configurable thresholds via CLAUDE.md', () => {
|
|
expect(shipSkill).toContain('## Test Coverage');
|
|
expect(shipSkill).toContain('Minimum:');
|
|
expect(shipSkill).toContain('Target:');
|
|
});
|
|
|
|
test('coverage gate skips on parse failure (not block)', () => {
|
|
expect(shipSkill).toContain('could not determine percentage — skipping');
|
|
});
|
|
|
|
test('review SKILL.md delegates coverage to Testing specialist', () => {
|
|
// Coverage audit moved to Testing specialist subagent in Review Army
|
|
expect(reviewSkill).toContain('testing.md');
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
});
|
|
});
|
|
|
|
// --- Ship metrics logging ---
|
|
|
|
describe('Ship metrics logging', () => {
|
|
const shipSkill = readShipUnion();
|
|
|
|
test('ship SKILL.md contains metrics persistence step', () => {
|
|
expect(shipSkill).toContain('Step 20');
|
|
expect(shipSkill).toContain('coverage_pct');
|
|
expect(shipSkill).toContain('plan_items_total');
|
|
expect(shipSkill).toContain('plan_items_done');
|
|
expect(shipSkill).toContain('verification_result');
|
|
});
|
|
});
|
|
|
|
// --- Plan file discovery shared helper ---
|
|
|
|
describe('Plan file discovery shared helper', () => {
|
|
// The shared helper should appear in ship (via PLAN_COMPLETION_AUDIT_SHIP)
|
|
// and in review (via PLAN_COMPLETION_AUDIT_REVIEW)
|
|
const shipSkill = readShipUnion();
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan file discovery appears in both ship and review', () => {
|
|
expect(shipSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
});
|
|
|
|
test('both include conversation context first', () => {
|
|
expect(shipSkill).toContain('Conversation context (primary)');
|
|
expect(reviewSkill).toContain('Conversation context (primary)');
|
|
});
|
|
|
|
test('both include content-based fallback', () => {
|
|
expect(shipSkill).toContain('Content-based search (fallback)');
|
|
expect(reviewSkill).toContain('Content-based search (fallback)');
|
|
});
|
|
});
|
|
|
|
// --- Retro plan completion ---
|
|
|
|
describe('Retro plan completion section', () => {
|
|
const retroSkill = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
|
|
|
test('retro SKILL.md contains plan completion section', () => {
|
|
expect(retroSkill).toContain('### Plan Completion');
|
|
expect(retroSkill).toContain('plan_items_total');
|
|
expect(retroSkill).toContain('Plan Completion This Period');
|
|
});
|
|
});
|
|
|
|
// --- Plan status footer in preamble ---
|
|
|
|
describe('Plan status footer in preamble', () => {
|
|
test('preamble contains plan status footer as neutral forward reference to EXIT PLAN MODE GATE', () => {
|
|
// Read any skill that uses PREAMBLE
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
expect(content).toContain('Plan Status Footer');
|
|
expect(content).toContain('GSTACK REVIEW REPORT');
|
|
expect(content).toContain('ExitPlanMode');
|
|
expect(content).toContain('EXIT PLAN MODE GATE');
|
|
// The preamble must NOT impose review-report rules on operational skills
|
|
// that have no review report. It's a forward reference, not enforcement.
|
|
expect(content).not.toContain('NO REVIEWS YET');
|
|
});
|
|
});
|
|
|
|
// --- make-pdf setup ordering ---
|
|
|
|
describe('make-pdf setup ordering', () => {
|
|
test('MAKE-PDF SETUP appears before generic preamble footer sections', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'make-pdf', 'SKILL.md'), 'utf-8');
|
|
const preambleIdx = content.indexOf('## Preamble (run first)');
|
|
const setupIdx = content.indexOf('## MAKE-PDF SETUP');
|
|
const planModeIdx = content.indexOf('## Plan Mode Safe Operations');
|
|
const telemetryIdx = content.indexOf('## Telemetry (run last)');
|
|
const workflowIdx = content.indexOf('# make-pdf: publication-quality PDFs from markdown');
|
|
|
|
expect(preambleIdx).toBeGreaterThanOrEqual(0);
|
|
expect(setupIdx).toBeGreaterThan(preambleIdx);
|
|
expect(setupIdx).toBeLessThan(planModeIdx);
|
|
expect(setupIdx).toBeLessThan(telemetryIdx);
|
|
expect(setupIdx).toBeLessThan(workflowIdx);
|
|
expect(content.match(/^## MAKE-PDF SETUP/gm)?.length ?? 0).toBe(1);
|
|
});
|
|
});
|
|
|
|
// --- Skill invocation during plan mode in preamble ---
|
|
|
|
describe('Skill invocation during plan mode in preamble', () => {
|
|
test('preamble contains skill invocation plan mode section', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
expect(content).toContain('Skill Invocation During Plan Mode');
|
|
expect(content).toContain('precedence over generic plan mode behavior');
|
|
expect(content).toContain('Do not continue the workflow');
|
|
expect(content).toContain('cancel the skill or leave plan mode');
|
|
});
|
|
});
|
|
|
|
// --- {{SPEC_REVIEW_LOOP}} resolver tests ---
|
|
|
|
describe('SPEC_REVIEW_LOOP resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('contains all 5 review dimensions', () => {
|
|
for (const dim of ['Completeness', 'Consistency', 'Clarity', 'Scope', 'Feasibility']) {
|
|
expect(content).toContain(dim);
|
|
}
|
|
});
|
|
|
|
test('references Agent tool for subagent dispatch', () => {
|
|
expect(content).toMatch(/Agent.*tool/i);
|
|
});
|
|
|
|
test('specifies max 3 iterations', () => {
|
|
expect(content).toMatch(/3.*iteration|maximum.*3/i);
|
|
});
|
|
|
|
test('includes quality score', () => {
|
|
expect(content).toContain('quality score');
|
|
});
|
|
|
|
test('includes metrics path', () => {
|
|
expect(content).toContain('spec-review.jsonl');
|
|
});
|
|
|
|
test('includes convergence guard', () => {
|
|
expect(content).toMatch(/[Cc]onvergence/);
|
|
});
|
|
|
|
test('includes graceful failure handling', () => {
|
|
expect(content).toMatch(/skip.*review|unavailable/i);
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_SKETCH}} resolver tests ---
|
|
|
|
describe('DESIGN_SKETCH resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('references DESIGN.md for design system constraints', () => {
|
|
expect(content).toContain('DESIGN.md');
|
|
});
|
|
|
|
test('contains wireframe or sketch terminology', () => {
|
|
expect(content).toMatch(/wireframe|sketch/i);
|
|
});
|
|
|
|
test('references browse binary for rendering', () => {
|
|
expect(content).toContain('$B goto');
|
|
});
|
|
|
|
test('references screenshot capture', () => {
|
|
expect(content).toContain('$B screenshot');
|
|
});
|
|
|
|
test('specifies rough aesthetic', () => {
|
|
expect(content).toMatch(/[Rr]ough|hand-drawn/);
|
|
});
|
|
|
|
test('includes skip conditions', () => {
|
|
expect(content).toMatch(/no UI component|skip/i);
|
|
});
|
|
});
|
|
|
|
// --- {{CODEX_SECOND_OPINION}} resolver tests ---
|
|
|
|
describe('CODEX_SECOND_OPINION resolver', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
const codexContent = fs.readFileSync(path.join(ROOT, '.agents', 'skills', 'gstack-office-hours', 'SKILL.md'), 'utf-8');
|
|
|
|
test('Phase 3.5 section appears in office-hours SKILL.md', () => {
|
|
expect(content).toContain('Phase 3.5: Cross-Model Second Opinion');
|
|
});
|
|
|
|
test('contains codex exec invocation', () => {
|
|
expect(content).toContain('codex exec');
|
|
});
|
|
|
|
test('contains opt-in AskUserQuestion text', () => {
|
|
expect(content).toContain('second opinion from an independent AI perspective');
|
|
});
|
|
|
|
test('contains cross-model synthesis instructions', () => {
|
|
expect(content).toMatch(/[Ss]ynthesis/);
|
|
expect(content).toContain('Where Claude agrees with the second opinion');
|
|
});
|
|
|
|
test('contains Claude subagent fallback', () => {
|
|
expect(content).toContain('CODEX_NOT_AVAILABLE');
|
|
expect(content).toContain('Agent tool');
|
|
expect(content).toContain('SECOND OPINION (Claude subagent)');
|
|
});
|
|
|
|
test('contains premise revision check', () => {
|
|
expect(content).toContain('Codex challenged premise');
|
|
});
|
|
|
|
test('contains error handling for auth, timeout, and empty', () => {
|
|
expect(content).toMatch(/[Aa]uth.*fail/);
|
|
expect(content).toMatch(/[Tt]imeout/);
|
|
expect(content).toMatch(/[Ee]mpty response/);
|
|
});
|
|
|
|
test('Codex host variant does NOT contain the Phase 3.5 resolver output', () => {
|
|
// The resolver returns '' for codex host, so the interactive section is stripped.
|
|
// Static template references to "Phase 3.5" in prose/conditionals are fine.
|
|
// Other resolvers (design review lite) may contain CODEX_NOT_AVAILABLE, so we
|
|
// check for Phase 3.5-specific markers only.
|
|
expect(codexContent).not.toContain('Phase 3.5: Cross-Model Second Opinion');
|
|
expect(codexContent).not.toContain('TMPERR_OH');
|
|
expect(codexContent).not.toContain('gstack-codex-oh-');
|
|
});
|
|
});
|
|
|
|
// --- Codex filesystem boundary tests ---
|
|
|
|
describe('Codex filesystem boundary', () => {
|
|
// Skills that call codex exec/review and should contain boundary text
|
|
const CODEX_CALLING_SKILLS = [
|
|
'codex', // /codex skill — 3 modes
|
|
'autoplan', // /autoplan — CEO/design/eng voices
|
|
'review', // /review — adversarial step resolver
|
|
'ship', // /ship — adversarial step resolver
|
|
'plan-eng-review', // outside voice resolver
|
|
'plan-ceo-review', // outside voice resolver
|
|
'office-hours', // second opinion resolver
|
|
];
|
|
|
|
const BOUNDARY_MARKER = 'Do NOT read or execute any';
|
|
|
|
test('boundary instruction appears in all skills that call codex', () => {
|
|
for (const skill of CODEX_CALLING_SKILLS) {
|
|
// Union: ship's codex call lives in sections/adversarial.md after the carve.
|
|
const content = readSkillUnion(skill);
|
|
expect(content).toContain(BOUNDARY_MARKER);
|
|
}
|
|
});
|
|
|
|
test('codex skill has Filesystem Boundary section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('## Filesystem Boundary');
|
|
expect(content).toContain('skill definitions meant for a different AI system');
|
|
});
|
|
|
|
test('codex skill has rabbit-hole detection rule', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Detect skill-file rabbit holes');
|
|
expect(content).toContain('gstack-update-check');
|
|
expect(content).toContain('Consider retrying');
|
|
});
|
|
|
|
test('review.ts CODEX_BOUNDARY constant is interpolated into resolver output', () => {
|
|
// The adversarial step resolver should include boundary text in codex exec prompts
|
|
const reviewContent = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
// Boundary should appear near codex exec invocations
|
|
const boundaryIdx = reviewContent.indexOf(BOUNDARY_MARKER);
|
|
const codexExecIdx = reviewContent.indexOf('codex exec');
|
|
// Both must exist and boundary must come before a codex exec call
|
|
expect(boundaryIdx).toBeGreaterThan(-1);
|
|
expect(codexExecIdx).toBeGreaterThan(-1);
|
|
});
|
|
|
|
test('autoplan boundary text avoids host-specific paths for cross-host compatibility', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'autoplan', 'SKILL.md.tmpl'), 'utf-8');
|
|
// autoplan template uses generic 'skills/gstack' pattern instead of host-specific
|
|
// paths like ~/.claude/ or .agents/skills (which break Codex/Claude output tests)
|
|
const boundaryStart = content.indexOf('Filesystem Boundary');
|
|
const boundaryEnd = content.indexOf('---', boundaryStart + 1);
|
|
const boundarySection = content.slice(boundaryStart, boundaryEnd);
|
|
expect(boundarySection).not.toContain('~/.claude/');
|
|
expect(boundarySection).not.toContain('.agents/skills');
|
|
expect(boundarySection).toContain('skills/gstack');
|
|
expect(boundarySection).toContain(BOUNDARY_MARKER);
|
|
});
|
|
});
|
|
|
|
// --- {{BENEFITS_FROM}} resolver tests ---
|
|
|
|
describe('BENEFITS_FROM resolver', () => {
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
const engContent = readSkillUnion('plan-eng-review'); // carved
|
|
|
|
test('plan-ceo-review contains prerequisite skill offer', () => {
|
|
expect(ceoContent).toContain('Prerequisite Skill Offer');
|
|
expect(ceoContent).toContain('/office-hours');
|
|
});
|
|
|
|
test('plan-eng-review contains prerequisite skill offer', () => {
|
|
expect(engContent).toContain('Prerequisite Skill Offer');
|
|
expect(engContent).toContain('/office-hours');
|
|
});
|
|
|
|
test('offer includes graceful decline', () => {
|
|
expect(ceoContent).toContain('No worries');
|
|
});
|
|
|
|
test('skills without benefits-from do NOT have prerequisite offer', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
expect(qaContent).not.toContain('Prerequisite Skill Offer');
|
|
});
|
|
|
|
test('inline invocation — no "another window" language', () => {
|
|
expect(ceoContent).not.toContain('another window');
|
|
expect(engContent).not.toContain('another window');
|
|
});
|
|
|
|
test('inline invocation — read-and-follow path present', () => {
|
|
expect(ceoContent).toContain('office-hours/SKILL.md');
|
|
expect(engContent).toContain('office-hours/SKILL.md');
|
|
});
|
|
|
|
test('BENEFITS_FROM delegates to INVOKE_SKILL pattern', () => {
|
|
// Should contain the INVOKE_SKILL-style loading prose (not the old manual skip list)
|
|
expect(engContent).toContain('Follow its instructions from top to bottom');
|
|
expect(engContent).toContain('skipping these sections');
|
|
expect(ceoContent).toContain('Follow its instructions from top to bottom');
|
|
});
|
|
});
|
|
|
|
// --- {{INVOKE_SKILL}} resolver tests ---
|
|
|
|
describe('INVOKE_SKILL resolver', () => {
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan-ceo-review uses INVOKE_SKILL for mid-session office-hours fallback', () => {
|
|
// The mid-session detection path should use INVOKE_SKILL-generated prose
|
|
expect(ceoContent).toContain('office-hours/SKILL.md');
|
|
expect(ceoContent).toContain('Follow its instructions from top to bottom');
|
|
});
|
|
|
|
test('INVOKE_SKILL output includes default skip list', () => {
|
|
expect(ceoContent).toContain('Preamble (run first)');
|
|
expect(ceoContent).toContain('Telemetry (run last)');
|
|
expect(ceoContent).toContain('AskUserQuestion Format');
|
|
});
|
|
|
|
test('INVOKE_SKILL output includes error handling', () => {
|
|
expect(ceoContent).toContain('If unreadable');
|
|
expect(ceoContent).toContain('Could not load');
|
|
});
|
|
|
|
test('template uses {{INVOKE_SKILL:office-hours}} placeholder', () => {
|
|
const tmpl = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(tmpl).toContain('{{INVOKE_SKILL:office-hours}}');
|
|
});
|
|
});
|
|
|
|
// --- {{CHANGELOG_WORKFLOW}} resolver tests ---
|
|
|
|
describe('CHANGELOG_WORKFLOW resolver', () => {
|
|
const shipContent = readShipUnion();
|
|
|
|
test('ship SKILL.md contains changelog workflow', () => {
|
|
expect(shipContent).toContain('CHANGELOG (auto-generate)');
|
|
expect(shipContent).toContain('git log <base>..HEAD --oneline');
|
|
});
|
|
|
|
test('changelog workflow includes cross-check step', () => {
|
|
expect(shipContent).toContain('Cross-check');
|
|
expect(shipContent).toContain('Every commit must map to at least one bullet point');
|
|
});
|
|
|
|
test('changelog workflow includes voice guidance', () => {
|
|
expect(shipContent).toContain('Lead with what the user can now **do**');
|
|
});
|
|
|
|
test('template uses {{CHANGELOG_WORKFLOW}} placeholder', () => {
|
|
// Post-carve (T9): the skeleton points to the changelog section, which carries
|
|
// the resolver. Neither should inline the old changelog content.
|
|
const skel = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');
|
|
const changelogSection = fs.readFileSync(path.join(ROOT, 'ship', 'sections', 'changelog.md.tmpl'), 'utf-8');
|
|
expect(skel).toContain('{{SECTION:changelog}}');
|
|
expect(changelogSection).toContain('{{CHANGELOG_WORKFLOW}}');
|
|
expect(skel + changelogSection).not.toContain('Group commits by theme');
|
|
});
|
|
|
|
test('changelog workflow includes keep-changelog format', () => {
|
|
expect(shipContent).toContain('### Added');
|
|
expect(shipContent).toContain('### Fixed');
|
|
});
|
|
});
|
|
|
|
// --- Parameterized resolver infrastructure tests ---
|
|
|
|
describe('parameterized resolver support', () => {
|
|
test('gen-skill-docs regex handles colon-separated args', () => {
|
|
// Verify the template containing {{INVOKE_SKILL:office-hours}} was processed
|
|
// without leaving unresolved placeholders
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(ceoContent).not.toMatch(/\{\{INVOKE_SKILL:[^}]+\}\}/);
|
|
});
|
|
|
|
test('templates with parameterized resolvers pass unresolved check', () => {
|
|
// All generated SKILL.md files should have no unresolved {{...}} placeholders
|
|
const skillDirs = fs.readdirSync(ROOT).filter(d =>
|
|
fs.existsSync(path.join(ROOT, d, 'SKILL.md'))
|
|
);
|
|
for (const dir of skillDirs) {
|
|
const content = fs.readFileSync(path.join(ROOT, dir, 'SKILL.md'), 'utf-8');
|
|
const unresolved = content.match(/\{\{[A-Z_]+(?::[^}]*)?\}\}/g);
|
|
if (unresolved) {
|
|
throw new Error(`${dir}/SKILL.md has unresolved placeholders: ${unresolved.join(', ')}`);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
// --- Preamble routing injection tests ---
|
|
|
|
describe('preamble routing injection', () => {
|
|
const shipContent = readShipUnion();
|
|
|
|
test('preamble bash checks for routing section in CLAUDE.md', () => {
|
|
expect(shipContent).toContain('grep -q "## Skill routing" CLAUDE.md');
|
|
expect(shipContent).toContain('HAS_ROUTING');
|
|
});
|
|
|
|
test('preamble bash reads routing_declined config', () => {
|
|
expect(shipContent).toContain('routing_declined');
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
});
|
|
|
|
test('preamble includes routing injection AskUserQuestion', () => {
|
|
expect(shipContent).toContain('Add routing rules to CLAUDE.md');
|
|
expect(shipContent).toContain("I'll invoke skills manually");
|
|
});
|
|
|
|
test('routing injection respects prior decline', () => {
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
expect(shipContent).toMatch(/routing_declined.*true/);
|
|
});
|
|
|
|
test('routing injection only fires when all conditions met', () => {
|
|
// Must be: HAS_ROUTING=no AND ROUTING_DECLINED=false AND PROACTIVE_PROMPTED=yes
|
|
expect(shipContent).toContain('HAS_ROUTING');
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
expect(shipContent).toContain('PROACTIVE_PROMPTED');
|
|
});
|
|
|
|
test('routing section content includes key routing rules', () => {
|
|
expect(shipContent).toContain('invoke /office-hours');
|
|
expect(shipContent).toContain('invoke /investigate');
|
|
expect(shipContent).toContain('invoke /ship');
|
|
expect(shipContent).toContain('invoke /qa');
|
|
});
|
|
|
|
test('routing section uses renamed checkpoint skills (not stale /checkpoint)', () => {
|
|
expect(shipContent).toContain('invoke /context-save');
|
|
expect(shipContent).toContain('invoke /context-restore');
|
|
expect(shipContent).not.toContain('invoke checkpoint');
|
|
});
|
|
|
|
test('routing section uses soft "when in doubt" policy, not hard "ALWAYS invoke"', () => {
|
|
expect(shipContent).toContain('When in doubt, invoke the skill');
|
|
expect(shipContent).not.toContain('Do NOT answer directly');
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_OUTSIDE_VOICES}} resolver tests ---
|
|
|
|
describe('DESIGN_OUTSIDE_VOICES resolver', () => {
|
|
test('plan-design-review contains outside voices section', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('CODEX_AVAILABLE');
|
|
expect(content).toContain('LITMUS SCORECARD');
|
|
});
|
|
|
|
test('design-review contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('source audit');
|
|
});
|
|
|
|
test('design-consultation contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('design direction');
|
|
});
|
|
|
|
test('branches correctly per skillName — different prompts', () => {
|
|
const planContent = readSkillUnion('plan-design-review');
|
|
const consultContent = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
|
|
// plan-design-review uses analytical prompt (high reasoning)
|
|
expect(planContent).toContain('model_reasoning_effort="high"');
|
|
// design-consultation uses creative prompt (medium reasoning)
|
|
expect(consultContent).toContain('model_reasoning_effort="medium"');
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_HARD_RULES}} resolver tests ---
|
|
|
|
describe('DESIGN_HARD_RULES resolver', () => {
|
|
test('plan-design-review Pass 4 contains hard rules', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Design Hard Rules');
|
|
expect(content).toContain('Classifier');
|
|
expect(content).toContain('MARKETING/LANDING PAGE');
|
|
expect(content).toContain('APP UI');
|
|
});
|
|
|
|
test('design-review contains hard rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Hard Rules');
|
|
});
|
|
|
|
test('includes all 3 rule sets', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Landing page rules');
|
|
expect(content).toContain('App UI rules');
|
|
expect(content).toContain('Universal rules');
|
|
});
|
|
|
|
test('references shared AI slop blacklist items', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('3-column feature grid');
|
|
expect(content).toContain('Purple/violet/indigo');
|
|
});
|
|
|
|
test('includes OpenAI hard rejection criteria', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Generic SaaS card grid');
|
|
expect(content).toContain('Carousel with no narrative purpose');
|
|
});
|
|
|
|
test('includes OpenAI litmus checks', () => {
|
|
const content = readSkillUnion('plan-design-review');
|
|
expect(content).toContain('Brand/product unmistakable');
|
|
expect(content).toContain('premium with all decorative shadows removed');
|
|
});
|
|
});
|
|
|
|
// --- Extended DESIGN_SKETCH resolver tests ---
|
|
|
|
describe('DESIGN_SKETCH extended with outside voices', () => {
|
|
const content = readSkillUnion('office-hours'); // carved: Phase 5/6 prose moved to section
|
|
|
|
test('contains outside design voices step', () => {
|
|
expect(content).toContain('Outside design voices');
|
|
});
|
|
|
|
test('offers opt-in via AskUserQuestion', () => {
|
|
expect(content).toContain('outside design perspectives');
|
|
});
|
|
|
|
test('still contains original wireframe steps', () => {
|
|
expect(content).toContain('wireframe');
|
|
expect(content).toContain('$B goto');
|
|
});
|
|
});
|
|
|
|
// --- Extended DESIGN_REVIEW_LITE resolver tests ---
|
|
|
|
describe('DESIGN_REVIEW_LITE extended with Codex', () => {
|
|
const content = readShipUnion();
|
|
|
|
test('contains Codex design voice block', () => {
|
|
expect(content).toContain('Codex design voice');
|
|
expect(content).toContain('CODEX (design)');
|
|
});
|
|
|
|
test('still contains original checklist steps', () => {
|
|
expect(content).toContain('design-checklist.md');
|
|
expect(content).toContain('SCOPE_FRONTEND');
|
|
});
|
|
|
|
});
|
|
|
|
// ─── Codex Generation Tests ─────────────────────────────────
|
|
|
|
describe('Codex generation (--host codex)', () => {
|
|
const AGENTS_DIR = path.join(ROOT, '.agents', 'skills');
|
|
|
|
// .agents/ is gitignored (v0.11.2.0) — generate on demand for tests
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
|
|
// Dynamic discovery of expected Codex skills: all templates except /codex
|
|
// Also excludes skills where .agents/skills/{name} is a symlink back to the repo root
|
|
// (vendored dev mode — gen-skill-docs skips these to avoid overwriting Claude SKILL.md)
|
|
const CODEX_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; codexName: string }> = [];
|
|
const isSymlinkLoop = (codexName: string): boolean => {
|
|
const agentSkillDir = path.join(ROOT, '.agents', 'skills', codexName);
|
|
try {
|
|
return fs.realpathSync(agentSkillDir) === fs.realpathSync(ROOT);
|
|
} catch { return false; }
|
|
};
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
if (!isSymlinkLoop('gstack')) {
|
|
skills.push({ dir: '.', codexName: 'gstack' });
|
|
}
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (entry.name === 'codex') continue; // /codex is excluded from Codex output
|
|
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
|
const codexName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
|
if (isSymlinkLoop(codexName)) continue;
|
|
skills.push({ dir: entry.name, codexName });
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
test('--host codex generates correct output paths', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const skillMd = path.join(AGENTS_DIR, skill.codexName, 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('root gstack bundle has OpenAI metadata for Codex skill browsing', () => {
|
|
const rootMetadata = path.join(ROOT, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(rootMetadata)).toBe(true);
|
|
const content = fs.readFileSync(rootMetadata, 'utf-8');
|
|
expect(content).toContain('display_name: "gstack"');
|
|
expect(content).toContain('Use $gstack to locate the bundled gstack skills.');
|
|
expect(content).toContain('allow_implicit_invocation: true');
|
|
});
|
|
|
|
test('externalSkillName mapping: root is gstack, others are gstack-{dir}', () => {
|
|
// Root → gstack
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack', 'SKILL.md'))).toBe(true);
|
|
// Subdirectories → gstack-{dir}
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'))).toBe(true);
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'))).toBe(true);
|
|
// gstack-upgrade doesn't double-prefix
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-upgrade', 'SKILL.md'))).toBe(true);
|
|
// No double-prefix: gstack-gstack-upgrade must NOT exist
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-gstack-upgrade', 'SKILL.md'))).toBe(false);
|
|
});
|
|
|
|
test('Codex frontmatter has ONLY name + description', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content.startsWith('---\n')).toBe(true);
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
expect(fmEnd).toBeGreaterThan(0);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
// Must have name and description
|
|
expect(frontmatter).toContain('name:');
|
|
expect(frontmatter).toContain('description:');
|
|
// Must NOT have allowed-tools, version, or hooks
|
|
expect(frontmatter).not.toContain('allowed-tools:');
|
|
expect(frontmatter).not.toContain('version:');
|
|
expect(frontmatter).not.toContain('hooks:');
|
|
}
|
|
});
|
|
|
|
test('all Codex skills have agents/openai.yaml metadata', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const metadata = path.join(AGENTS_DIR, skill.codexName, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(metadata)).toBe(true);
|
|
const content = fs.readFileSync(metadata, 'utf-8');
|
|
expect(content).toContain(`display_name: "${skill.codexName}"`);
|
|
expect(content).toContain('short_description:');
|
|
expect(content).toContain('allow_implicit_invocation: true');
|
|
}
|
|
});
|
|
|
|
test('no .claude/skills/ in Codex output', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('no ~/.claude/ paths in Codex output', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('~/.claude/');
|
|
}
|
|
});
|
|
|
|
test('/codex skill excluded from Codex output', () => {
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex'))).toBe(false);
|
|
});
|
|
|
|
test('Codex output includes Claude outside-voice skill with read-only boundary', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-claude', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('claude -p');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-prompt-');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-response-XXXXXX');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-error-XXXXXX');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-diff-');
|
|
expect(content).not.toMatch(/gstack-claude-(?:prompt|response|error|diff)-X{6,}\.\w+/);
|
|
expect(content).not.toContain('/tmp/gstack-claude-diff-$$');
|
|
expect(content).toContain('cat "$PROMPT_FILE" | "$CLAUDE_BIN" -p');
|
|
expect(content).toContain('Resolve the binary and invoke it in the same host execution context');
|
|
expect(content).toContain('--disable-slash-commands');
|
|
expect(content).toContain('--tools ""');
|
|
expect(content).toContain('--allowedTools Read,Grep,Glob');
|
|
expect(content).toContain('--disallowedTools Bash,Edit,Write');
|
|
expect(content).toContain('Do not infer authentication state from credential files');
|
|
expect(content).toContain('run the actual `claude -p`');
|
|
expect(content).not.toContain('AUTH_MISSING');
|
|
expect(content).not.toContain('$HOME/.claude/.credentials.json');
|
|
expect(content).toContain('is_error');
|
|
});
|
|
|
|
test('Claude temp file templates are accepted by host mktemp', () => {
|
|
for (const template of [
|
|
'/tmp/gstack-claude-prompt-XXXXXX',
|
|
'/tmp/gstack-claude-response-XXXXXX',
|
|
'/tmp/gstack-claude-error-XXXXXX',
|
|
'/tmp/gstack-claude-diff-XXXXXX',
|
|
]) {
|
|
const result = spawnSync('mktemp', [template], { encoding: 'utf-8' });
|
|
expect(result.status).toBe(0);
|
|
const created = result.stdout.trim();
|
|
expect(created.startsWith(template.replace('XXXXXX', ''))).toBe(true);
|
|
fs.unlinkSync(created);
|
|
}
|
|
});
|
|
|
|
test('Codex review step stripped from Codex-host ship and review', () => {
|
|
const shipContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(shipContent).not.toContain('codex review --base');
|
|
expect(shipContent).not.toContain('CODEX_REVIEWS');
|
|
|
|
const reviewContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(reviewContent).not.toContain('codex review --base');
|
|
expect(reviewContent).not.toContain('CODEX_REVIEWS');
|
|
});
|
|
|
|
test('--host codex --dry-run freshness', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// Every Codex skill should be FRESH
|
|
for (const skill of CODEX_SKILLS) {
|
|
expect(output).toContain(`FRESH: .agents/skills/${skill.codexName}/SKILL.md`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('--host agents alias produces same output as --host codex', () => {
|
|
const codexResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
const agentsResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'agents', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(codexResult.exitCode).toBe(0);
|
|
expect(agentsResult.exitCode).toBe(0);
|
|
// Both should produce the same output (same FRESH lines)
|
|
expect(codexResult.stdout.toString()).toBe(agentsResult.stdout.toString());
|
|
});
|
|
|
|
test('multiline descriptions preserved in Codex output', () => {
|
|
// office-hours has a multiline description — verify it survives the frontmatter transform
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-office-hours', 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
// Description should span multiple lines (block scalar)
|
|
const descLines = frontmatter.split('\n').filter(l => l.startsWith(' '));
|
|
expect(descLines.length).toBeGreaterThan(1);
|
|
// Verify key phrases survived
|
|
expect(frontmatter).toContain('YC Office Hours');
|
|
});
|
|
|
|
test('hook skills have safety prose and no hooks: in frontmatter', () => {
|
|
const HOOK_SKILLS = ['gstack-careful', 'gstack-freeze', 'gstack-guard'];
|
|
for (const skillName of HOOK_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skillName, 'SKILL.md'), 'utf-8');
|
|
// Must have safety advisory prose
|
|
expect(content).toContain('Safety Advisory');
|
|
// Must NOT have hooks: in frontmatter
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).not.toContain('hooks:');
|
|
}
|
|
});
|
|
|
|
test('all Codex SKILL.md files have auto-generated header', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
}
|
|
});
|
|
|
|
test('Codex preamble resolves runtime assets from repo-local or global gstack roots', () => {
|
|
// Check a skill that has a preamble (review is a good candidate)
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT');
|
|
expect(content).toContain('$_ROOT/.agents/skills/gstack');
|
|
expect(content).toContain('$GSTACK_BIN/gstack-config');
|
|
expect(content).toContain('$GSTACK_ROOT/gstack-upgrade/SKILL.md');
|
|
expect(content).not.toContain('~/.codex/skills/gstack/bin/gstack-config get telemetry');
|
|
});
|
|
|
|
// ─── Path rewriting regression tests ─────────────────────────
|
|
|
|
test('sidecar paths point to .agents/skills/gstack/review/ (not gstack-review/)', () => {
|
|
// Regression: gen-skill-docs rewrote .claude/skills/review → .agents/skills/gstack-review
|
|
// but setup puts sidecars under .agents/skills/gstack/review/. Must match setup layout.
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
// Correct: references to sidecar files use gstack/review/ path
|
|
expect(content).toContain('.agents/skills/gstack/review/checklist.md');
|
|
// design-checklist.md is now referenced via Review Army specialist (Claude only, stripped for Codex)
|
|
// Wrong: must NOT reference gstack-review/checklist.md (file doesn't exist there)
|
|
expect(content).not.toContain('.agents/skills/gstack-review/checklist.md');
|
|
});
|
|
|
|
test('sidecar paths in ship skill point to gstack/review/ for pre-landing review', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
// Ship references the review checklist in its pre-landing review step
|
|
if (content.includes('checklist.md')) {
|
|
expect(content).toContain('.agents/skills/gstack/review/');
|
|
expect(content).not.toContain('.agents/skills/gstack-review/checklist');
|
|
}
|
|
});
|
|
|
|
test('greptile-triage sidecar path is correct', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
if (content.includes('greptile-triage')) {
|
|
expect(content).toContain('.agents/skills/gstack/review/greptile-triage.md');
|
|
expect(content).not.toContain('.agents/skills/gstack-review/greptile-triage');
|
|
}
|
|
});
|
|
|
|
test('all four path rewrite rules produce correct output', () => {
|
|
// Test each of the 4 path rewrite rules individually
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
|
|
// Rule 1: ~/.claude/skills/gstack → $GSTACK_ROOT
|
|
expect(content).not.toContain('~/.claude/skills/gstack');
|
|
expect(content).toContain('$GSTACK_ROOT');
|
|
|
|
// Rule 2: .claude/skills/gstack → .agents/skills/gstack
|
|
expect(content).not.toContain('.claude/skills/gstack');
|
|
|
|
// Rule 3: .claude/skills/review → .agents/skills/gstack/review
|
|
expect(content).not.toContain('.claude/skills/review');
|
|
|
|
// Rule 4: .claude/skills → .agents/skills (catch-all)
|
|
expect(content).not.toContain('.claude/skills');
|
|
});
|
|
|
|
test('path rewrite rules apply to all Codex skills with sidecar references', () => {
|
|
// Verify across ALL generated skills, not just review
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
// No skill should reference Claude paths
|
|
expect(content).not.toContain('~/.claude/skills');
|
|
expect(content).not.toContain('.claude/skills');
|
|
if (content.includes('gstack-config') || content.includes('gstack-update-check') || content.includes('gstack-telemetry-log')) {
|
|
expect(content).toContain('$GSTACK_ROOT');
|
|
}
|
|
// If a skill references checklist.md, it must use the correct sidecar path
|
|
if (content.includes('checklist.md') && !content.includes('design-checklist.md')) {
|
|
expect(content).not.toContain('gstack-review/checklist.md');
|
|
}
|
|
}
|
|
});
|
|
|
|
// ─── Claude output regression guard ─────────────────────────
|
|
|
|
test('Claude output unchanged: review skill still uses .claude/skills/ paths', () => {
|
|
// Codex changes must NOT affect Claude output
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.claude/skills/review/checklist.md');
|
|
expect(content).toContain('~/.claude/skills/gstack');
|
|
// Must NOT contain Codex HOST paths. `~/.codex/sessions/` is exempt: the
|
|
// timeout-wrapper guidance documents the Codex CLI's own rollout-log
|
|
// location (a user-facing CLI path, same class as ~/.codex/logs/ in the
|
|
// codex skill), not the gstack Codex host install path.
|
|
expect(content).not.toContain('.agents/skills');
|
|
expect(content.replaceAll('~/.codex/sessions/', '')).not.toContain('~/.codex/');
|
|
});
|
|
|
|
test('Claude output unchanged: ship skill still uses .claude/skills/ paths', () => {
|
|
const content = readShipUnion();
|
|
expect(content).toContain('~/.claude/skills/gstack');
|
|
expect(content).not.toContain('.agents/skills');
|
|
// ~/.codex/sessions/ is the Codex CLI's rollout-log path (user-facing),
|
|
// documented by the adversarial-pass timeout guidance — see review test above.
|
|
expect(content.replaceAll('~/.codex/sessions/', '')).not.toContain('~/.codex/');
|
|
});
|
|
|
|
test('Claude output unchanged: all Claude skills have zero Codex paths', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
// pair-agent legitimately documents how Codex agents store credentials.
|
|
// codex + autoplan document the Codex CLI auth file (~/.codex/auth.json)
|
|
// and log path (~/.codex/logs/) — those are user-facing Codex CLI paths,
|
|
// not the gstack Codex host install path. ~/.codex/sessions/ (rollout
|
|
// logs, referenced by the review/ship timeout guidance) is the same
|
|
// user-facing class, so it is scrubbed before the ban.
|
|
if (skill.dir !== 'pair-agent' && skill.dir !== 'codex' && skill.dir !== 'autoplan') {
|
|
expect(content.replaceAll('~/.codex/sessions/', '')).not.toContain('~/.codex/');
|
|
}
|
|
// gstack-upgrade legitimately references .agents/skills for cross-platform detection
|
|
if (skill.dir !== 'gstack-upgrade') {
|
|
expect(content).not.toContain('.agents/skills');
|
|
}
|
|
}
|
|
});
|
|
|
|
// ─── Design outside voices: Codex host guard ─────────────────
|
|
|
|
test('codex host produces empty outside voices in design-review', () => {
|
|
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(codexContent).not.toContain('Design Outside Voices');
|
|
});
|
|
|
|
test('codex host does not include Codex design block in ship', () => {
|
|
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(codexContent).not.toContain('Codex design voice');
|
|
});
|
|
});
|
|
|
|
// ─── Factory generation tests ────────────────────────────────
|
|
|
|
describe('Factory generation (--host factory)', () => {
|
|
const FACTORY_DIR = path.join(ROOT, '.factory', 'skills');
|
|
|
|
// Generate Factory output for tests
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
|
|
const FACTORY_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; factoryName: string }> = [];
|
|
const isSymlinkLoop = (name: string): boolean => {
|
|
const factorySkillDir = path.join(ROOT, '.factory', 'skills', name);
|
|
try { return fs.realpathSync(factorySkillDir) === fs.realpathSync(ROOT); }
|
|
catch { return false; }
|
|
};
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
if (!isSymlinkLoop('gstack')) skills.push({ dir: '.', factoryName: 'gstack' });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (entry.name === 'codex') continue;
|
|
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
|
const factoryName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
|
if (isSymlinkLoop(factoryName)) continue;
|
|
skills.push({ dir: entry.name, factoryName });
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
test('--host factory generates correct output paths', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const skillMd = path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('Factory frontmatter has name + description + user-invocable', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).toContain('name:');
|
|
expect(frontmatter).toContain('description:');
|
|
expect(frontmatter).toContain('user-invocable: true');
|
|
expect(frontmatter).not.toContain('allowed-tools:');
|
|
expect(frontmatter).not.toContain('preamble-tier:');
|
|
expect(frontmatter).not.toContain('sensitive:');
|
|
}
|
|
});
|
|
|
|
test('sensitive skills have disable-model-invocation', () => {
|
|
const SENSITIVE = ['gstack-ship', 'gstack-land-and-deploy', 'gstack-guard', 'gstack-careful', 'gstack-freeze', 'gstack-unfreeze'];
|
|
for (const name of SENSITIVE) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).toContain('disable-model-invocation: true');
|
|
}
|
|
});
|
|
|
|
test('non-sensitive skills lack disable-model-invocation', () => {
|
|
const NON_SENSITIVE = ['gstack-qa', 'gstack-review', 'gstack-investigate', 'gstack-browse'];
|
|
for (const name of NON_SENSITIVE) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).not.toContain('disable-model-invocation');
|
|
}
|
|
});
|
|
|
|
test('no .claude/skills/ in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('no ~/.claude/skills/ paths in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
// ~/.claude/skills should be rewritten, but ~/.claude/plans is legitimate
|
|
// (plan directory lookup) and ~/.claude/ in codex prompts is intentional
|
|
expect(content).not.toContain('~/.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('/codex skill excluded from Factory output', () => {
|
|
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex'))).toBe(false);
|
|
});
|
|
|
|
test('Factory keeps Codex integration blocks', () => {
|
|
// Factory users CAN use Codex second opinions (codex exec is a standalone binary)
|
|
const shipContent = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(shipContent).toContain('codex');
|
|
});
|
|
|
|
test('no agents/openai.yaml in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const yamlPath = path.join(FACTORY_DIR, skill.factoryName, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(yamlPath)).toBe(false);
|
|
}
|
|
});
|
|
|
|
test('--host droid alias works', () => {
|
|
const factoryResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
const droidResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'droid', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(factoryResult.exitCode).toBe(0);
|
|
expect(droidResult.exitCode).toBe(0);
|
|
expect(factoryResult.stdout.toString()).toBe(droidResult.stdout.toString());
|
|
});
|
|
|
|
test('--host factory --dry-run freshness', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
for (const skill of FACTORY_SKILLS) {
|
|
expect(output).toContain(`FRESH: .factory/skills/${skill.factoryName}/SKILL.md`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('Factory preamble uses .factory paths', () => {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT');
|
|
expect(content).toContain('$_ROOT/.factory/skills/gstack');
|
|
expect(content).toContain('$GSTACK_BIN/gstack-config');
|
|
});
|
|
});
|
|
|
|
// ─── Parameterized host smoke tests (config-driven) ─────────
|
|
|
|
import { ALL_HOST_CONFIGS, getExternalHosts } from '../hosts/index';
|
|
|
|
describe('Parameterized host smoke tests', () => {
|
|
// Regenerate every external host up front so the per-host `--dry-run` freshness
|
|
// checks are deterministic. These host dirs (.agents/.factory/.cursor/...) are
|
|
// gitignored regenerated artifacts, so the freshness check is really an
|
|
// idempotency/determinism check — it still catches non-deterministic gen, but no
|
|
// longer flakes on stale-on-disk state left by a missing `gen --host all` prestep
|
|
// (the canonical `bun test` does not run one). The tracked-claude freshness test
|
|
// (`generated files are fresh`) runs earlier and is unaffected.
|
|
beforeAll(() => {
|
|
for (const h of getExternalHosts()) {
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', h.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
});
|
|
|
|
for (const hostConfig of getExternalHosts()) {
|
|
describe(`${hostConfig.displayName} (--host ${hostConfig.name})`, () => {
|
|
const hostDir = path.join(ROOT, hostConfig.hostSubdir, 'skills');
|
|
|
|
test('generates output that exists on disk', () => {
|
|
// Generated dir should exist (created by earlier bun run gen:skill-docs --host all)
|
|
if (!fs.existsSync(hostDir)) {
|
|
// Generate if not already done
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
expect(fs.existsSync(hostDir)).toBe(true);
|
|
const skills = fs.readdirSync(hostDir).filter(d =>
|
|
fs.existsSync(path.join(hostDir, d, 'SKILL.md'))
|
|
);
|
|
expect(skills.length).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('no .claude/skills path leakage outside repo-root sidecar symlinks', () => {
|
|
if (!fs.existsSync(hostDir)) return; // skip if not generated
|
|
const skills = fs.readdirSync(hostDir);
|
|
for (const skill of skills) {
|
|
// Dev installs may mount the repo root at host/skills/gstack as a runtime
|
|
// sidecar. The generator skips that symlink loop, so leakage checks should too.
|
|
if (isRepoRootSymlink(path.join(hostDir, skill))) continue;
|
|
const skillMd = path.join(hostDir, skill, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
// Strip bash blocks (which have legitimate fallback paths)
|
|
const noBash = content.replace(/```bash\n[\s\S]*?```/g, '');
|
|
const leaks = noBash.split('\n').filter(l => l.includes('.claude/skills'));
|
|
if (leaks.length > 0) {
|
|
throw new Error(`${skill}: .claude/skills leakage:\n${leaks.slice(0, 3).join('\n')}`);
|
|
}
|
|
}
|
|
});
|
|
|
|
test('frontmatter has name and description', () => {
|
|
if (!fs.existsSync(hostDir)) return;
|
|
const skills = fs.readdirSync(hostDir);
|
|
for (const skill of skills) {
|
|
const skillMd = path.join(hostDir, skill, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content).toMatch(/^---\n/);
|
|
expect(content).toMatch(/^name:\s/m);
|
|
expect(content).toMatch(/^description:\s/m);
|
|
}
|
|
});
|
|
|
|
test('--dry-run freshness check passes', () => {
|
|
const result = Bun.spawnSync(
|
|
['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name, '--dry-run'],
|
|
{ cwd: ROOT, stdout: 'pipe', stderr: 'pipe' }
|
|
);
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
if (hostConfig.generation.skipSkills?.includes('codex')) {
|
|
test('/codex skill excluded', () => {
|
|
expect(fs.existsSync(path.join(hostDir, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
});
|
|
}
|
|
});
|
|
}
|
|
});
|
|
|
|
// ─── --host all tests ────────────────────────────────────────
|
|
|
|
describe('--host all', () => {
|
|
// Same determinism guard as the parameterized block: make external hosts fresh on
|
|
// disk so `--host all --dry-run` reports FRESH regardless of prior state.
|
|
beforeAll(() => {
|
|
for (const h of getExternalHosts()) {
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', h.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
});
|
|
|
|
test('--host all generates for all registered hosts', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'all', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// All hosts should appear in output
|
|
expect(output).toContain('FRESH: SKILL.md'); // claude
|
|
for (const hostConfig of getExternalHosts()) {
|
|
expect(output).toContain(`FRESH: ${hostConfig.hostSubdir}/skills/`);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── Setup script validation ─────────────────────────────────
|
|
// These tests verify the setup script's install layout matches
|
|
// what the generator produces — catching the bug where setup
|
|
// installed Claude-format source dirs for Codex users.
|
|
|
|
describe('setup script validation', () => {
|
|
const setupContent = fs.readFileSync(path.join(ROOT, 'setup'), 'utf-8');
|
|
|
|
test('setup has separate link functions for Claude and Codex', () => {
|
|
expect(setupContent).toContain('link_claude_skill_dirs');
|
|
expect(setupContent).toContain('link_codex_skill_dirs');
|
|
// Old unified function must not exist
|
|
expect(setupContent).not.toMatch(/^link_skill_dirs\(\)/m);
|
|
});
|
|
|
|
test('Claude install uses link_claude_skill_dirs', () => {
|
|
// The Claude install section (section 4) should use the Claude function
|
|
const claudeSection = setupContent.slice(
|
|
setupContent.indexOf('# 4. Install for Claude'),
|
|
setupContent.indexOf('# 5. Install for Codex')
|
|
);
|
|
expect(claudeSection).toContain('link_claude_skill_dirs');
|
|
expect(claudeSection).not.toContain('link_codex_skill_dirs');
|
|
});
|
|
|
|
test('Codex install uses link_codex_skill_dirs', () => {
|
|
// The Codex install section (section 5) should use the Codex function
|
|
const codexSection = setupContent.slice(
|
|
setupContent.indexOf('# 5. Install for Codex'),
|
|
setupContent.indexOf('# 6. Create')
|
|
);
|
|
expect(codexSection).toContain('create_codex_runtime_root');
|
|
expect(codexSection).toContain('link_codex_skill_dirs');
|
|
expect(codexSection).not.toContain('link_claude_skill_dirs');
|
|
expect(codexSection).not.toContain('_link_or_copy "$GSTACK_DIR" "$CODEX_GSTACK"');
|
|
});
|
|
|
|
test('Codex install prefers repo-local .agents/skills when setup runs from there', () => {
|
|
expect(setupContent).toContain('SKILLS_PARENT_BASENAME');
|
|
expect(setupContent).toContain('CODEX_REPO_LOCAL=0');
|
|
expect(setupContent).toContain('[ "$SKILLS_PARENT_BASENAME" = ".agents" ]');
|
|
expect(setupContent).toContain('CODEX_REPO_LOCAL=1');
|
|
expect(setupContent).toContain('CODEX_SKILLS="$INSTALL_SKILLS_DIR"');
|
|
});
|
|
|
|
test('setup separates install path from source path for symlinked repo-local installs', () => {
|
|
expect(setupContent).toContain('INSTALL_GSTACK_DIR=');
|
|
expect(setupContent).toContain('SOURCE_GSTACK_DIR=');
|
|
expect(setupContent).toContain('INSTALL_SKILLS_DIR=');
|
|
expect(setupContent).toContain('CODEX_GSTACK="$INSTALL_GSTACK_DIR"');
|
|
expect(setupContent).toContain('link_codex_skill_dirs "$SOURCE_GSTACK_DIR" "$CODEX_SKILLS"');
|
|
});
|
|
|
|
test('Codex installs always create sidecar runtime assets for the real skill target', () => {
|
|
expect(setupContent).toContain('if [ "$INSTALL_CODEX" -eq 1 ]; then');
|
|
expect(setupContent).toContain('create_agents_sidecar "$SOURCE_GSTACK_DIR"');
|
|
});
|
|
|
|
test('link_codex_skill_dirs reads from .agents/skills/', () => {
|
|
// The Codex link function must reference .agents/skills for generated Codex skills
|
|
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('.agents/skills');
|
|
expect(fnBody).toContain('gstack*');
|
|
});
|
|
|
|
test('link_claude_skill_dirs creates real directories with absolute SKILL.md symlinks', () => {
|
|
// Claude links should be real directories with absolute SKILL.md symlinks
|
|
// to ensure Claude Code discovers them as top-level skills (not nested under gstack/)
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('mkdir -p "$target"');
|
|
// v1.36.0.0: routes through _link_or_copy helper for Windows fallback (cp on MSYS2/Git Bash).
|
|
expect(fnBody).toContain('_link_or_copy "$gstack_dir/$dir_name/SKILL.md" "$target/SKILL.md"');
|
|
});
|
|
|
|
// REGRESSION: cleanup functions must handle both old symlinks AND new real-directory pattern
|
|
test('cleanup functions handle real directories with symlinked SKILL.md', () => {
|
|
// cleanup_old_claude_symlinks must detect and remove real dirs with SKILL.md symlinks
|
|
const cleanupOldStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
|
|
const cleanupOldEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up old', cleanupOldStart));
|
|
const cleanupOldBody = setupContent.slice(cleanupOldStart, cleanupOldEnd);
|
|
expect(cleanupOldBody).toContain('-d "$old_target"');
|
|
expect(cleanupOldBody).toContain('-L "$old_target/SKILL.md"');
|
|
expect(cleanupOldBody).toContain('rm -rf "$old_target"');
|
|
|
|
// cleanup_prefixed_claude_symlinks must also handle the new pattern
|
|
const cleanupPrefixedStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
|
|
const cleanupPrefixedEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up prefixed', cleanupPrefixedStart));
|
|
const cleanupPrefixedBody = setupContent.slice(cleanupPrefixedStart, cleanupPrefixedEnd);
|
|
expect(cleanupPrefixedBody).toContain('-d "$prefixed_target"');
|
|
expect(cleanupPrefixedBody).toContain('-L "$prefixed_target/SKILL.md"');
|
|
expect(cleanupPrefixedBody).toContain('rm -rf "$prefixed_target"');
|
|
});
|
|
|
|
// REGRESSION: link function must upgrade old directory symlinks
|
|
test('link_claude_skill_dirs removes old directory symlinks before creating real dirs', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// Must check for and remove old symlinks before mkdir
|
|
expect(fnBody).toContain('if [ -L "$target" ]');
|
|
expect(fnBody).toContain('rm -f "$target"');
|
|
});
|
|
|
|
test('setup links root gstack skill through a thin Claude wrapper alias', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_root_skill_alias()');
|
|
const fnEnd = setupContent.indexOf('# ─── Helper: remove old unprefixed Claude skill entries', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('_gstack-command');
|
|
expect(fnBody).toContain('_link_or_copy "$gstack_dir/SKILL.md" "$target/SKILL.md"');
|
|
|
|
const claudeSection = setupContent.slice(
|
|
setupContent.indexOf('# 4. Install for Claude'),
|
|
setupContent.indexOf('# 5. Install for Codex')
|
|
);
|
|
expect(claudeSection).toContain('link_claude_root_skill_alias "$SOURCE_GSTACK_DIR" "$INSTALL_SKILLS_DIR"');
|
|
});
|
|
|
|
test('setup supports --host auto|claude|codex|kiro|opencode|cursor|slate', () => {
|
|
expect(setupContent).toContain('--host');
|
|
expect(setupContent).toContain('claude|codex|kiro|factory|opencode|cursor|slate|auto');
|
|
});
|
|
|
|
test('auto mode detects claude, codex, kiro, and opencode binaries', () => {
|
|
expect(setupContent).toContain('command -v claude');
|
|
expect(setupContent).toContain('command -v codex');
|
|
expect(setupContent).toContain('command -v kiro-cli');
|
|
expect(setupContent).toContain('command -v opencode');
|
|
});
|
|
|
|
// T1: Sidecar skip guard — prevents .agents/skills/gstack from being linked as a skill
|
|
test('link_codex_skill_dirs skips the gstack sidecar directory', () => {
|
|
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
|
|
});
|
|
|
|
// T2: Dynamic $GSTACK_ROOT paths in generated Codex preambles
|
|
test('generated Codex preambles use dynamic GSTACK_ROOT paths', () => {
|
|
const codexSkillDir = path.join(ROOT, '.agents', 'skills', 'gstack-ship');
|
|
if (!fs.existsSync(codexSkillDir)) return; // skip if .agents/ not generated
|
|
const content = fs.readFileSync(path.join(codexSkillDir, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT=');
|
|
expect(content).toContain('$GSTACK_BIN/');
|
|
});
|
|
|
|
test('setup supports --host kiro with install section and sed rewrites', () => {
|
|
expect(setupContent).toContain('INSTALL_KIRO=');
|
|
expect(setupContent).toContain('kiro-cli');
|
|
expect(setupContent).toContain('KIRO_SKILLS=');
|
|
expect(setupContent).toContain('~/.kiro/skills/gstack');
|
|
expect(setupContent).toContain('$KIRO_GSTACK/lib');
|
|
});
|
|
|
|
test('setup supports --host opencode with install section and OpenCode skill path vars', () => {
|
|
expect(setupContent).toContain('INSTALL_OPENCODE=');
|
|
expect(setupContent).toContain('OPENCODE_SKILLS="$HOME/.config/opencode/skills"');
|
|
expect(setupContent).toContain('OPENCODE_GSTACK="$OPENCODE_SKILLS/gstack"');
|
|
});
|
|
|
|
test('setup installs OpenCode skills into a nested gstack runtime root', () => {
|
|
expect(setupContent).toContain('create_opencode_runtime_root');
|
|
expect(setupContent).toContain('.opencode/skills');
|
|
expect(setupContent).toContain('review/specialists');
|
|
expect(setupContent).toContain('qa/templates');
|
|
expect(setupContent).toContain('qa/references');
|
|
expect(setupContent).toContain('dx-hall-of-fame.md');
|
|
expect(setupContent).toContain('$opencode_gstack/lib');
|
|
});
|
|
|
|
test('create_agents_sidecar links runtime assets', () => {
|
|
// Sidecar must link bin with its shared lib modules, plus browse, review, qa
|
|
const fnStart = setupContent.indexOf('create_agents_sidecar()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('bin');
|
|
expect(fnBody).toContain('lib');
|
|
expect(fnBody).toContain('browse');
|
|
expect(fnBody).toContain('review');
|
|
expect(fnBody).toContain('qa');
|
|
});
|
|
|
|
test('create_codex_runtime_root exposes only runtime assets', () => {
|
|
const fnStart = setupContent.indexOf('create_codex_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', setupContent.indexOf('review/', fnStart)));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('gstack/SKILL.md');
|
|
expect(fnBody).toContain('$codex_gstack/lib');
|
|
expect(fnBody).toContain('browse/dist');
|
|
expect(fnBody).toContain('browse/bin');
|
|
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
|
|
// Review runtime assets (individual files, not the whole dir)
|
|
expect(fnBody).toContain('checklist.md');
|
|
expect(fnBody).toContain('design-checklist.md');
|
|
expect(fnBody).toContain('greptile-triage.md');
|
|
expect(fnBody).toContain('TODOS-format.md');
|
|
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$codex_gstack"');
|
|
});
|
|
|
|
test('create_factory_runtime_root links shared lib modules beside bin', () => {
|
|
const fnStart = setupContent.indexOf('create_factory_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('create_opencode_runtime_root()', fnStart);
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('$factory_gstack/bin');
|
|
expect(fnBody).toContain('$factory_gstack/lib');
|
|
});
|
|
|
|
test('direct Codex installs are migrated out of ~/.codex/skills/gstack', () => {
|
|
expect(setupContent).toContain('migrate_direct_codex_install');
|
|
expect(setupContent).toContain('$HOME/.gstack/repos/gstack');
|
|
expect(setupContent).toContain('avoid duplicate skill discovery');
|
|
});
|
|
|
|
// --- Symlink prefix tests (PR #503) ---
|
|
|
|
test('link_claude_skill_dirs applies gstack- prefix by default', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('SKILL_PREFIX');
|
|
expect(fnBody).toContain('link_name="gstack-$skill_name"');
|
|
});
|
|
|
|
test('link_claude_skill_dirs preserves already-prefixed dirs', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// gstack-* dirs should keep their name (e.g., gstack-upgrade stays gstack-upgrade)
|
|
expect(fnBody).toContain('gstack-*) link_name="$skill_name"');
|
|
});
|
|
|
|
test('setup supports --no-prefix flag', () => {
|
|
expect(setupContent).toContain('--no-prefix');
|
|
expect(setupContent).toContain('SKILL_PREFIX=0');
|
|
});
|
|
|
|
test('cleanup_old_claude_symlinks removes only gstack-pointing symlinks', () => {
|
|
expect(setupContent).toContain('cleanup_old_claude_symlinks');
|
|
const fnStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// Should check readlink before removing
|
|
expect(fnBody).toContain('readlink');
|
|
expect(fnBody).toContain('gstack/*');
|
|
// Should skip already-prefixed dirs
|
|
expect(fnBody).toContain('gstack-*) continue');
|
|
});
|
|
|
|
test('cleanup runs before link when prefix is enabled', () => {
|
|
// In the Claude install section, cleanup should happen before linking
|
|
const claudeInstallSection = setupContent.slice(
|
|
setupContent.indexOf('INSTALL_CLAUDE'),
|
|
setupContent.lastIndexOf('link_claude_skill_dirs')
|
|
);
|
|
expect(claudeInstallSection).toContain('cleanup_old_claude_symlinks');
|
|
});
|
|
|
|
// --- Persistent config + interactive prompt tests ---
|
|
|
|
test('setup reads skill_prefix from config', () => {
|
|
expect(setupContent).toContain('get skill_prefix');
|
|
expect(setupContent).toContain('GSTACK_CONFIG');
|
|
});
|
|
|
|
test('setup supports --prefix flag', () => {
|
|
expect(setupContent).toContain('--prefix)');
|
|
expect(setupContent).toContain('SKILL_PREFIX=1; SKILL_PREFIX_FLAG=1');
|
|
});
|
|
|
|
test('--prefix and --no-prefix persist to config', () => {
|
|
expect(setupContent).toContain('set skill_prefix');
|
|
});
|
|
|
|
test('interactive prompt shows when no config', () => {
|
|
expect(setupContent).toContain('Short names');
|
|
expect(setupContent).toContain('Namespaced');
|
|
expect(setupContent).toContain('Choice [1/2]');
|
|
});
|
|
|
|
test('non-TTY defaults to flat names', () => {
|
|
// Should check if stdin is a TTY before prompting
|
|
expect(setupContent).toContain('-t 0');
|
|
});
|
|
|
|
test('cleanup_prefixed_claude_symlinks exists and uses readlink', () => {
|
|
expect(setupContent).toContain('cleanup_prefixed_claude_symlinks');
|
|
const fnStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('readlink');
|
|
expect(fnBody).toContain('gstack-$skill_name');
|
|
});
|
|
|
|
test('reverse cleanup runs before link when prefix is disabled', () => {
|
|
const claudeInstallSection = setupContent.slice(
|
|
setupContent.indexOf('INSTALL_CLAUDE'),
|
|
setupContent.lastIndexOf('link_claude_skill_dirs')
|
|
);
|
|
expect(claudeInstallSection).toContain('cleanup_prefixed_claude_symlinks');
|
|
});
|
|
|
|
test('welcome message references SKILL_PREFIX', () => {
|
|
// gstack-upgrade is always called gstack-upgrade (it's the actual dir name)
|
|
// but the welcome section should exist near the prefix logic
|
|
expect(setupContent).toContain('Run /gstack-upgrade anytime');
|
|
});
|
|
});
|
|
|
|
describe('discover-skills hidden directory filtering', () => {
|
|
test('discoverTemplates skips dot-prefixed directories', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-discover-'));
|
|
try {
|
|
// Create a hidden dir with a template (should be excluded)
|
|
fs.mkdirSync(path.join(tmpDir, '.hidden'), { recursive: true });
|
|
fs.writeFileSync(path.join(tmpDir, '.hidden', 'SKILL.md.tmpl'), '---\nname: evil\n---\ntest');
|
|
// Create a visible dir with a template (should be included)
|
|
fs.mkdirSync(path.join(tmpDir, 'visible'), { recursive: true });
|
|
fs.writeFileSync(path.join(tmpDir, 'visible', 'SKILL.md.tmpl'), '---\nname: good\n---\ntest');
|
|
|
|
const { discoverTemplates } = require('../scripts/discover-skills');
|
|
const results = discoverTemplates(tmpDir);
|
|
const dirs = results.map((r: { tmpl: string }) => r.tmpl);
|
|
|
|
expect(dirs).toContain('visible/SKILL.md.tmpl');
|
|
expect(dirs).not.toContain('.hidden/SKILL.md.tmpl');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('telemetry', () => {
|
|
test('generated SKILL.md contains telemetry start block', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_TEL_START');
|
|
expect(content).toContain('_SESSION_ID');
|
|
expect(content).toContain('TELEMETRY:');
|
|
expect(content).toContain('TEL_PROMPTED:');
|
|
expect(content).toContain('gstack-config get telemetry');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry opt-in prompt', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.telemetry-prompted');
|
|
expect(content).toContain('Help gstack get better');
|
|
expect(content).toContain('gstack-config set telemetry community');
|
|
expect(content).toContain('gstack-config set telemetry anonymous');
|
|
expect(content).toContain('gstack-config set telemetry off');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry epilogue', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Telemetry (run last)');
|
|
expect(content).toContain('gstack-telemetry-log');
|
|
expect(content).toContain('_TEL_END');
|
|
expect(content).toContain('_TEL_DUR');
|
|
expect(content).toContain('SKILL_NAME');
|
|
expect(content).toContain('OUTCOME');
|
|
expect(content).toContain('PLAN MODE EXCEPTION');
|
|
});
|
|
|
|
test('generated SKILL.md contains pending marker handling', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.pending');
|
|
expect(content).toContain('_pending_finalize');
|
|
});
|
|
|
|
test('telemetry blocks appear in all skill files that use PREAMBLE', () => {
|
|
const skills = ['qa', 'ship', 'review', 'plan-ceo-review', 'plan-eng-review', 'retro'];
|
|
for (const skill of skills) {
|
|
const skillPath = path.join(ROOT, skill, 'SKILL.md');
|
|
if (fs.existsSync(skillPath)) {
|
|
const content = fs.readFileSync(skillPath, 'utf-8');
|
|
expect(content).toContain('_TEL_START');
|
|
expect(content).toContain('Telemetry (run last)');
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('community fixes wave', () => {
|
|
// Helper to get all generated SKILL.md files
|
|
function getAllSkillMds(): Array<{ name: string; content: string }> {
|
|
const results: Array<{ name: string; content: string }> = [];
|
|
const rootPath = path.join(ROOT, 'SKILL.md');
|
|
if (fs.existsSync(rootPath)) {
|
|
results.push({ name: 'root', content: fs.readFileSync(rootPath, 'utf-8') });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
const skillPath = path.join(ROOT, entry.name, 'SKILL.md');
|
|
if (fs.existsSync(skillPath)) {
|
|
results.push({ name: entry.name, content: fs.readFileSync(skillPath, 'utf-8') });
|
|
}
|
|
}
|
|
return results;
|
|
}
|
|
|
|
// #594 — Discoverability: every SKILL.md.tmpl description contains "gstack"
|
|
test('every SKILL.md.tmpl description contains "gstack"', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
const content = fs.readFileSync(tmplPath, 'utf-8');
|
|
const desc = extractDescription(content);
|
|
expect(desc.toLowerCase()).toContain('gstack');
|
|
}
|
|
});
|
|
|
|
// #594 — Discoverability: first line of each description is under 120 chars
|
|
test('every SKILL.md.tmpl description first line is under 120 chars', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
const content = fs.readFileSync(tmplPath, 'utf-8');
|
|
const desc = extractDescription(content);
|
|
const firstLine = desc.split('\n')[0];
|
|
expect(firstLine.length).toBeLessThanOrEqual(120);
|
|
}
|
|
});
|
|
|
|
// #573 — Feature signals: ship/SKILL.md contains feature signal detection
|
|
test('ship/SKILL.md contains feature signal detection in Step 4', () => {
|
|
const content = readShipUnion();
|
|
expect(content.toLowerCase()).toContain('feature signal');
|
|
});
|
|
|
|
// #510 — Context warnings: no SKILL.md contains "running low on context"
|
|
test('no generated SKILL.md contains "running low on context"', () => {
|
|
const skills = getAllSkillMds();
|
|
for (const { name, content } of skills) {
|
|
expect(content).not.toContain('running low on context');
|
|
}
|
|
});
|
|
|
|
// #510 — Context warnings: plan-eng-review has explicit anti-warning
|
|
test('plan-eng-review/SKILL.md contains "Do not preemptively warn"', () => {
|
|
const content = readSkillUnion('plan-eng-review'); // carved: review body moved to section
|
|
expect(content).toContain('Do not preemptively warn');
|
|
});
|
|
|
|
// #474 — Safety Net: no SKILL.md uses find with -delete
|
|
test('no generated SKILL.md contains find with -delete flag', () => {
|
|
const skills = getAllSkillMds();
|
|
for (const { name, content } of skills) {
|
|
// Match find commands that use -delete (but not prose mentioning the word "delete")
|
|
const lines = content.split('\n');
|
|
for (const line of lines) {
|
|
if (line.includes('find ') && line.includes('-delete')) {
|
|
throw new Error(`${name}/SKILL.md contains find with -delete: ${line.trim()}`);
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
// #467 — Telemetry: preamble JSONL writes are gated by telemetry setting
|
|
test('preamble JSONL writes are inside telemetry conditional', () => {
|
|
const preamble = fs.readFileSync(path.join(ROOT, 'scripts/resolvers/preamble.ts'), 'utf-8');
|
|
// Find all skill-usage.jsonl write lines
|
|
const lines = preamble.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
if (lines[i].includes('skill-usage.jsonl') && lines[i].includes('>>')) {
|
|
// Look backwards for a telemetry conditional within 5 lines
|
|
let foundConditional = false;
|
|
for (let j = i - 1; j >= Math.max(0, i - 5); j--) {
|
|
if (lines[j].includes('_TEL') && lines[j].includes('off')) {
|
|
foundConditional = true;
|
|
break;
|
|
}
|
|
}
|
|
expect(foundConditional).toBe(true);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('codex commands must not use inline $(git rev-parse --show-toplevel) for cwd', () => {
|
|
// Regression test: inline $(git rev-parse --show-toplevel) in codex exec -C
|
|
// or codex review without cd evaluates in whatever cwd the background shell
|
|
// inherits, which may be a different project in Conductor workspaces.
|
|
// The fix is to resolve _REPO_ROOT eagerly at the top of each bash block.
|
|
|
|
// Scan all source files that could contain codex commands
|
|
// Use Bun.Glob to avoid ELOOP from .claude/skills/gstack symlink back to ROOT
|
|
const tmplGlob = new Bun.Glob('**/*.tmpl');
|
|
const sourceFiles = [
|
|
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
|
|
.filter(f => f.endsWith('.ts'))
|
|
.map(f => `scripts/resolvers/${f}`),
|
|
'scripts/gen-skill-docs.ts',
|
|
];
|
|
|
|
test('no codex exec command uses inline $(git rev-parse --show-toplevel) in -C flag', () => {
|
|
const violations: string[] = [];
|
|
for (const rel of sourceFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1}`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('no generated SKILL.md has codex exec with inline $(git rev-parse --show-toplevel) in -C flag', () => {
|
|
const violations: string[] = [];
|
|
const skillMdGlob = new Bun.Glob('**/SKILL.md');
|
|
const skillMdFiles = Array.from(skillMdGlob.scanSync({ cwd: ROOT, followSymlinks: false }));
|
|
for (const rel of skillMdFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1}`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('codex review commands must be preceded by cd "$_REPO_ROOT" (no -C support)', () => {
|
|
// codex review does not support -C, so the pattern must be:
|
|
// _REPO_ROOT=$(git rev-parse --show-toplevel) || { ... }
|
|
// cd "$_REPO_ROOT"
|
|
// codex review ...
|
|
// NOT: codex review ... with inline $(git rev-parse --show-toplevel)
|
|
const allFiles = [
|
|
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...Array.from(new Bun.Glob('**/SKILL.md').scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
|
|
.filter(f => f.endsWith('.ts'))
|
|
.map(f => `scripts/resolvers/${f}`),
|
|
'scripts/gen-skill-docs.ts',
|
|
];
|
|
const violations: string[] = [];
|
|
for (const rel of allFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
// Skip non-executable lines (markdown table cells, prose references)
|
|
if (line.includes('|') && line.includes('`/codex review`')) continue;
|
|
if (line.includes('`codex review`')) continue;
|
|
// Check for codex review with inline $(git rev-parse)
|
|
if (line.includes('codex review') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1} — inline git rev-parse in codex review`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('codex review commands take their scope from a flag, never from prompt text', () => {
|
|
// `codex review` scope comes ONLY from --base/--commit/--uncommitted. The
|
|
// positional [PROMPT] is mutually exclusive with all three (#1428, #1479),
|
|
// and a prompt-only `codex review` silently falls back to the *uncommitted
|
|
// working-tree* scope (`git status --short; git diff`) — so describing the
|
|
// diff range in prompt text produces a confident review of the wrong
|
|
// changes, with no error. Both halves are pinned here:
|
|
// (a) every `codex review` invocation carries a scope flag, and
|
|
// (b) no invocation puts a positional prompt in front of that flag.
|
|
//
|
|
// This does NOT apply to `codex exec`, which is agentic and really does run
|
|
// the git command it's told to — the adversarial pass legitimately scopes
|
|
// itself in prompt text.
|
|
const checkedFiles = [
|
|
'scripts/resolvers/review.ts',
|
|
'review/SKILL.md',
|
|
'ship/SKILL.md',
|
|
'codex/SKILL.md.tmpl',
|
|
'codex/SKILL.md',
|
|
];
|
|
|
|
const violations: string[] = [];
|
|
for (const rel of checkedFiles) {
|
|
// ship's codex/adversarial command moved into sections/adversarial.md (T9 carve).
|
|
const content = rel === 'ship/SKILL.md' ? readShipUnion() : fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
// Only inspect real shell invocations, not prose mentioning the command.
|
|
if (line.includes('`codex review`')) continue;
|
|
const match = line.match(/(?:^|[;&|]\s*|\s)codex\s+review\b(.*)$/);
|
|
if (!match) continue;
|
|
const rest = match[1];
|
|
const scopeFlag = /--base\b|--commit\b|--uncommitted\b/;
|
|
if (!scopeFlag.test(rest)) {
|
|
// A quoted prompt with no scope flag is the silent-wrong-scope bug.
|
|
if (/^\s*["'$]/.test(rest)) {
|
|
violations.push(`${rel}:${i + 1} — prompt-only codex review (falls back to working-tree scope)`);
|
|
}
|
|
continue;
|
|
}
|
|
const beforeFlag = rest.split(scopeFlag)[0].trim();
|
|
if (/^["'$]|^--\s*["']/.test(beforeFlag)) {
|
|
violations.push(`${rel}:${i + 1} — positional prompt passed alongside a scope flag`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
});
|
|
|
|
// ─── Learnings + Confidence Resolver Tests ─────────────────────
|
|
|
|
describe('LEARNINGS_SEARCH resolver', () => {
|
|
const SEARCH_SKILLS = ['review', 'ship', 'plan-eng-review', 'investigate', 'office-hours', 'plan-ceo-review'];
|
|
|
|
for (const skill of SEARCH_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains learnings search`, () => {
|
|
const content = readSkillUnion(skill); // ship: moved to sections/plan-completion.md
|
|
expect(content).toContain('Prior Learnings');
|
|
expect(content).toContain('gstack-learnings-search');
|
|
});
|
|
}
|
|
|
|
test('learnings search includes cross-project config check', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('cross_project_learnings');
|
|
expect(content).toContain('--cross-project');
|
|
});
|
|
|
|
test('learnings search includes AskUserQuestion for first-time cross-project opt-in', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Enable cross-project learnings');
|
|
expect(content).toContain('project-scoped only');
|
|
});
|
|
|
|
test('learnings search mentions prior learning applied display format', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Prior learning applied');
|
|
});
|
|
});
|
|
|
|
describe('LEARNINGS_LOG resolver', () => {
|
|
const LOG_SKILLS = ['review', 'retro', 'investigate'];
|
|
|
|
for (const skill of LOG_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains learnings log`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Capture Learnings');
|
|
expect(content).toContain('gstack-learnings-log');
|
|
});
|
|
}
|
|
|
|
test('learnings log documents all type values', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
for (const type of ['pattern', 'pitfall', 'preference', 'architecture', 'tool']) {
|
|
expect(content).toContain(type);
|
|
}
|
|
});
|
|
|
|
test('learnings log documents all source values', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
for (const source of ['observed', 'user-stated', 'inferred', 'cross-model']) {
|
|
expect(content).toContain(source);
|
|
}
|
|
});
|
|
|
|
test('learnings log includes files field for staleness detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"files"');
|
|
expect(content).toContain('staleness detection');
|
|
});
|
|
});
|
|
|
|
describe('CONFIDENCE_CALIBRATION resolver', () => {
|
|
const CONFIDENCE_SKILLS = ['review', 'ship', 'plan-eng-review', 'cso'];
|
|
|
|
for (const skill of CONFIDENCE_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains confidence calibration`, () => {
|
|
const content = readSkillUnion(skill); // ship: moved to sections/review-army.md
|
|
expect(content).toContain('Confidence Calibration');
|
|
expect(content).toContain('confidence score');
|
|
});
|
|
}
|
|
|
|
test('confidence calibration includes scoring rubric with all tiers', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('9-10');
|
|
expect(content).toContain('7-8');
|
|
expect(content).toContain('5-6');
|
|
expect(content).toContain('3-4');
|
|
expect(content).toContain('1-2');
|
|
});
|
|
|
|
test('confidence calibration includes display rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Show normally');
|
|
expect(content).toContain('Suppress from main report');
|
|
});
|
|
|
|
test('confidence calibration includes finding format example', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('[P1] (confidence:');
|
|
expect(content).toContain('SQL injection');
|
|
});
|
|
|
|
test('confidence calibration includes calibration learning feedback loop', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('calibration event');
|
|
expect(content).toContain('Log the corrected pattern');
|
|
});
|
|
|
|
test('skills without confidence calibration do NOT contain it', () => {
|
|
// office-hours and retro do NOT use confidence calibration
|
|
for (const skill of ['office-hours', 'retro']) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('## Confidence Calibration');
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('gen-skill-docs prefix warning (#620/#578)', () => {
|
|
const { execSync } = require('child_process');
|
|
|
|
test('warns about skill_prefix when config has prefix=true', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
|
|
try {
|
|
// Create a fake ~/.gstack/config.yaml with skill_prefix: true
|
|
const fakeHome = tmpDir;
|
|
const fakeGstack = path.join(fakeHome, '.gstack');
|
|
fs.mkdirSync(fakeGstack, { recursive: true });
|
|
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: true\n');
|
|
|
|
const output = execSync('bun run scripts/gen-skill-docs.ts', {
|
|
cwd: ROOT,
|
|
env: { ...process.env, HOME: fakeHome },
|
|
encoding: 'utf-8',
|
|
timeout: 30000,
|
|
});
|
|
expect(output).toContain('skill_prefix is true');
|
|
expect(output).toContain('gstack-relink');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
test('no warning when skill_prefix is false or absent', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
|
|
try {
|
|
const fakeHome = tmpDir;
|
|
const fakeGstack = path.join(fakeHome, '.gstack');
|
|
fs.mkdirSync(fakeGstack, { recursive: true });
|
|
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: false\n');
|
|
|
|
const output = execSync('bun run scripts/gen-skill-docs.ts', {
|
|
cwd: ROOT,
|
|
env: { ...process.env, HOME: fakeHome },
|
|
encoding: 'utf-8',
|
|
timeout: 30000,
|
|
});
|
|
expect(output).not.toContain('skill_prefix is true');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('voice-triggers processing', () => {
|
|
const { extractVoiceTriggers, processVoiceTriggers } = require('../scripts/gen-skill-docs') as {
|
|
extractVoiceTriggers: (content: string) => string[];
|
|
processVoiceTriggers: (content: string) => string;
|
|
};
|
|
|
|
test('extractVoiceTriggers parses valid YAML list', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit.\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
|
|
const triggers = extractVoiceTriggers(content);
|
|
expect(triggers).toEqual(['see-so', 'security review']);
|
|
});
|
|
|
|
test('extractVoiceTriggers returns [] when no field present', () => {
|
|
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
|
|
expect(extractVoiceTriggers(content)).toEqual([]);
|
|
});
|
|
|
|
test('processVoiceTriggers appends voice triggers to description', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
|
|
const result = processVoiceTriggers(content);
|
|
expect(result).toContain('Voice triggers (speech-to-text aliases): "see-so", "security review".');
|
|
});
|
|
|
|
test('processVoiceTriggers strips voice-triggers field from output', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n---\nBody`;
|
|
const result = processVoiceTriggers(content);
|
|
expect(result).not.toContain('voice-triggers:');
|
|
});
|
|
|
|
test('processVoiceTriggers returns content unchanged when no voice-triggers', () => {
|
|
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
|
|
expect(processVoiceTriggers(content)).toBe(content);
|
|
});
|
|
|
|
test('generated CSO SKILL.md contains voice triggers in description', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"see-so"');
|
|
expect(content).toContain('Voice triggers (speech-to-text aliases):');
|
|
});
|
|
|
|
test('generated CSO SKILL.md does NOT contain raw voice-triggers field', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(0, fmEnd);
|
|
expect(frontmatter).not.toContain('voice-triggers:');
|
|
});
|
|
});
|
|
|
|
describe('plan-mode-info resolver (handshake-replacement)', () => {
|
|
const REVIEW_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
];
|
|
|
|
// Header for the vestigial handshake that was removed. If it ever reappears,
|
|
// someone accidentally re-introduced the resolver.
|
|
const HANDSHAKE_MARKER = '## Plan Mode Handshake';
|
|
// Header for the new plan-mode-info section (previously lived at the tail
|
|
// of completion-status.ts; now hoisted to position 1 of the preamble).
|
|
const PLAN_MODE_INFO_MARKER = '## Skill Invocation During Plan Mode';
|
|
|
|
test('vestigial handshake is absent from all generated Claude SKILL.md files', () => {
|
|
// Scan every generated SKILL.md under ROOT (top-level directory per skill).
|
|
// Using fs.readdirSync + filter instead of a glob so we catch any skill
|
|
// that gets added later without updating this list.
|
|
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
|
|
let checked = 0;
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(ROOT, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content, `handshake marker in ${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
|
|
checked++;
|
|
}
|
|
expect(checked).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('vestigial handshake is absent from non-Claude host outputs when present on disk', () => {
|
|
// Non-Claude hosts render to hostSubdirs (.agents/, .openclaw/, etc). The
|
|
// plan-mode-info resolver has no host-scoping — all hosts get the new
|
|
// section, none get the old handshake. Scan all candidate host dirs.
|
|
const hostDirs = ['.agents', '.openclaw', '.opencode', '.factory', '.hermes', '.kiro', '.cursor', '.slate'];
|
|
let checked = 0;
|
|
for (const host of hostDirs) {
|
|
const skillsRoot = path.join(ROOT, host, 'skills');
|
|
if (!fs.existsSync(skillsRoot)) continue;
|
|
const entries = fs.readdirSync(skillsRoot, { withFileTypes: true });
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(skillsRoot, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content, `handshake marker in ${host}/skills/${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
|
|
checked++;
|
|
}
|
|
}
|
|
if (checked === 0) {
|
|
// eslint-disable-next-line no-console
|
|
console.warn(
|
|
'plan-mode-info: no non-Claude host outputs found for cross-host absence check — ' +
|
|
'run `bun run gen:skill-docs --host all` to populate',
|
|
);
|
|
}
|
|
});
|
|
|
|
test.each(REVIEW_SKILLS)(
|
|
'%s/SKILL.md contains the new plan-mode-info section near the top',
|
|
(skill) => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const idx = content.indexOf(PLAN_MODE_INFO_MARKER);
|
|
expect(idx).toBeGreaterThan(0);
|
|
// Position 1 in preamble composition = within the first ~300 lines.
|
|
// Roughly translates to first ~15KB of text.
|
|
expect(idx).toBeLessThan(15_000);
|
|
},
|
|
);
|
|
|
|
test('plan-mode-info is wired BEFORE generateUpgradeCheck in preamble', () => {
|
|
const content = fs.readFileSync(
|
|
path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
|
|
'utf-8',
|
|
);
|
|
const planModeIdx = content.indexOf(PLAN_MODE_INFO_MARKER);
|
|
const upgradeIdx = content.indexOf('UPGRADE_AVAILABLE');
|
|
expect(planModeIdx).toBeGreaterThan(0);
|
|
expect(upgradeIdx).toBeGreaterThan(0);
|
|
expect(planModeIdx).toBeLessThan(upgradeIdx);
|
|
});
|
|
|
|
test('0C-bis STOP block present in plan-ceo-review/SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
const presentIdx = content.indexOf('Present these approach options via AskUserQuestion');
|
|
const preludeIdx = content.indexOf('### 0D-prelude');
|
|
expect(presentIdx).toBeGreaterThan(0);
|
|
expect(preludeIdx).toBeGreaterThan(presentIdx);
|
|
const between = content.slice(presentIdx, preludeIdx);
|
|
expect(between).toContain('**STOP.**');
|
|
expect(between).toContain('Do NOT proceed to Step 0D or 0F until the user responds to 0C-bis');
|
|
});
|
|
});
|
|
|
|
// GSTACK REVIEW REPORT report-at-bottom contract — verifies the prompt-text
|
|
// fix in scripts/resolvers/review.ts (the load-bearing change for the
|
|
// "report not at bottom of plan in plan mode" bug). The bug is in the
|
|
// prompt's contradictory write-flow instructions, not in observable
|
|
// runtime behavior we can cheaply gate in CI. Verifying the prompt text
|
|
// directly is the deterministic equivalent of the regression test the
|
|
// PTY harness can't reliably drive (autoplan needs auto-progression of
|
|
// AskUserQuestions to reach the report-write step, which the harness
|
|
// doesn't support today).
|
|
describe('GSTACK REVIEW REPORT delete-then-append flow', () => {
|
|
const PLAN_REVIEW_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'plan-eng-review',
|
|
];
|
|
|
|
for (const skill of PLAN_REVIEW_SKILLS) {
|
|
test(`${skill}/SKILL.md prescribes delete-then-append, not in-place replace`, () => {
|
|
// Carved skills (v2 plan Phase B) relocate the review-report prose into
|
|
// sections/*.md; readSkillUnion follows the content wherever the carve put it.
|
|
const content = readSkillUnion(skill);
|
|
|
|
// The new (correct) instruction must be present.
|
|
expect(content).toContain('delete-then-append flow');
|
|
expect(content).toContain('never mid-file');
|
|
expect(content).toContain('Do NOT replace the section in place');
|
|
|
|
// The old contradictory bullets must be gone. The signature phrase
|
|
// from the buggy prompt was 'replace it entirely using the Edit tool'
|
|
// which is what allowed mid-file reports to stay mid-file.
|
|
expect(content).not.toContain('replace it** entirely using the Edit tool');
|
|
expect(content).not.toContain('If it was found mid-file, move it');
|
|
});
|
|
}
|
|
|
|
test('scripts/resolvers/review.ts source has the rewritten flow', () => {
|
|
const src = fs.readFileSync(path.join(ROOT, 'scripts', 'resolvers', 'review.ts'), 'utf-8');
|
|
expect(src).toContain('delete-then-append flow');
|
|
expect(src).toContain('never mid-file');
|
|
expect(src).toContain('Do NOT replace the section in place');
|
|
// Old contradictory bullets are gone from the source resolver.
|
|
expect(src).not.toContain('replace it** entirely using the Edit tool');
|
|
expect(src).not.toContain('If it was found mid-file, move it');
|
|
});
|
|
});
|
|
|
|
describe('LEARNINGS_SEARCH resolver: query parameter', () => {
|
|
// Lazy-load resolver and types after describe block to keep test file self-contained.
|
|
const { generateLearningsSearch } = require('../scripts/resolvers/learnings');
|
|
const { HOST_PATHS } = require('../scripts/resolvers/types');
|
|
|
|
const claudeCtx = {
|
|
skillName: 'test',
|
|
tmplPath: 'test/SKILL.md.tmpl',
|
|
host: 'claude',
|
|
paths: HOST_PATHS.claude,
|
|
};
|
|
const codexCtx = { ...claudeCtx, host: 'codex', paths: HOST_PATHS.codex };
|
|
|
|
test('no args → bash does not contain --query (backwards-compat)', () => {
|
|
const out = generateLearningsSearch(claudeCtx);
|
|
expect(out).not.toContain('--query');
|
|
});
|
|
|
|
test('claude host + query=foo bar → both cross-project and project-scoped branches contain --query', () => {
|
|
const out = generateLearningsSearch(claudeCtx, ['query=foo bar']);
|
|
// Both branches of the if/else must carry the flag.
|
|
const lines = out.split('\n').filter(l => l.includes('gstack-learnings-search'));
|
|
expect(lines.length).toBeGreaterThanOrEqual(2);
|
|
for (const line of lines) {
|
|
expect(line).toContain('--query "foo bar"');
|
|
}
|
|
});
|
|
|
|
test('codex host + query=foo bar → codex bash variant contains --query', () => {
|
|
const out = generateLearningsSearch(codexCtx, ['query=foo bar']);
|
|
expect(out).toContain('--query "foo bar"');
|
|
expect(out).toContain('$GSTACK_BIN/gstack-learnings-search');
|
|
});
|
|
|
|
test('empty value query= → bash does not contain --query (locked semantics: falls through)', () => {
|
|
const claudeOut = generateLearningsSearch(claudeCtx, ['query=']);
|
|
expect(claudeOut).not.toContain('--query');
|
|
const codexOut = generateLearningsSearch(codexCtx, ['query=']);
|
|
expect(codexOut).not.toContain('--query');
|
|
});
|
|
|
|
test('shell-injection chars in query= → throws at gen-time (defense in depth)', () => {
|
|
for (const bad of ['$(whoami)', '`cmd`', 'a;b', 'a&b', 'a"b', 'a\\b', 'foo$x']) {
|
|
expect(() => generateLearningsSearch(claudeCtx, [`query=${bad}`])).toThrow(/alphanumeric/);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('EXIT PLAN MODE GATE placement', () => {
|
|
// Fresh skill list — do NOT reuse REVIEW_SKILLS upstream (3 entries, missing plan-devex).
|
|
const planSkills = ['plan-eng-review', 'plan-ceo-review', 'plan-design-review', 'plan-devex-review'];
|
|
|
|
// Strip fenced code blocks before matching headings — PLAN_FILE_REVIEW_REPORT
|
|
// already contains `## GSTACK REVIEW REPORT` inside a markdown example fence,
|
|
// and the gate text itself shows `## GSTACK REVIEW REPORT` inside a fence too.
|
|
const stripFences = (md: string) => md.replace(/```[\s\S]*?```/g, '');
|
|
|
|
test('gate is the terminal ## heading in every plan-* review SKILL.md', () => {
|
|
for (const skill of planSkills) {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const stripped = stripFences(md);
|
|
const headings = [...stripped.matchAll(/^## .+$/gm)].map(m => m[0]);
|
|
const lastH2 = headings.at(-1);
|
|
expect(lastH2, `${skill}/SKILL.md last ## heading (fences stripped)`).toBe('## EXIT PLAN MODE GATE (BLOCKING)');
|
|
expect(md, `${skill}/SKILL.md gate body`).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
|
|
}
|
|
});
|
|
|
|
test('codex/SKILL.md contains gate (mid-file per D5; Step 2B/2C follow)', () => {
|
|
const codex = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(codex).toContain('## EXIT PLAN MODE GATE (BLOCKING)');
|
|
expect(codex).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
|
|
});
|
|
});
|
|
|
|
describe('scope-gate exceptions drift-guard', () => {
|
|
// The plan-mode auto-select-B exceptions block is hand-duplicated in the
|
|
// plan-eng-review and plan-design-review templates (matching the gate
|
|
// around it, which predates this block). The two copies must stay
|
|
// byte-identical modulo exactly two known variant slots:
|
|
// 1. the plan-mode bullet's action tail (Design Doc Check vs pre-review
|
|
// audit + mockups),
|
|
// 2. the named-target vocabulary ("a path, a doc" vs "a path, a page, a doc").
|
|
// A future edit to one copy that silently misses the other fails here
|
|
// instead of drifting. The real fix (shared {{SCOPE_GATE}} resolver) is a
|
|
// filed TODO — this guard is the stopgap that makes the duplication safe.
|
|
const START_MARKER = '**Exceptions — check in this order, BEFORE asking:**';
|
|
const END_MARKER = 'in any mode — it is a hard STOP.';
|
|
|
|
function extractExceptionsBlock(skill: string): string {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const start = md.indexOf(START_MARKER);
|
|
expect(start, `${skill}/SKILL.md: exceptions block start marker present`).toBeGreaterThan(-1);
|
|
const end = md.indexOf(END_MARKER, start);
|
|
expect(end, `${skill}/SKILL.md: exceptions block end marker present`).toBeGreaterThan(start);
|
|
return md.slice(start, end + END_MARKER.length);
|
|
}
|
|
|
|
const normalizeVariantSlots = (block: string) =>
|
|
block
|
|
.replace('Then run the Design Doc Check and Step 0 against that plan.', '<ACTION_TAIL>')
|
|
.replace('Then run the pre-review audit, mockups, and Step 0 against that plan.', '<ACTION_TAIL>')
|
|
.replace('a path, a page, a doc they pasted,', 'a path, a doc they pasted,');
|
|
|
|
test('eng and design exceptions blocks are identical modulo the two variant slots', () => {
|
|
const eng = normalizeVariantSlots(extractExceptionsBlock('plan-eng-review'));
|
|
const design = normalizeVariantSlots(extractExceptionsBlock('plan-design-review'));
|
|
expect(eng).toBe(design);
|
|
// The action tail must actually have been normalized in both (guards
|
|
// against a rewording that bypasses the normalizer and vacuously passes).
|
|
expect(eng).toContain('<ACTION_TAIL>');
|
|
});
|
|
|
|
test('exceptions block carries the announcement string the PTY detectors pin', () => {
|
|
for (const skill of ['plan-eng-review', 'plan-design-review']) {
|
|
const block = extractExceptionsBlock(skill);
|
|
expect(block, `${skill}: verbatim announcement`).toContain(
|
|
'Scope gate: plan mode — auto-selected B (reviewing <target>).',
|
|
);
|
|
}
|
|
});
|
|
|
|
test('gate menu carries the question strings the PTY question detector pins', () => {
|
|
// isScopeGateQuestionVisible (claude-pty-runner.ts) anchors on the
|
|
// question text + option A's body. If the menu is reworded without
|
|
// updating the detector, the paid smokes' must-stay-false assertions go
|
|
// vacuous — this free pin fails first.
|
|
for (const skill of ['plan-eng-review', 'plan-design-review']) {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(md, `${skill}: gate question text`).toContain('What should I review?');
|
|
expect(md, `${skill}: option A body text`).toContain('The current branch diff');
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('GSTACK REVIEW REPORT mandatory unresolved-decisions status', () => {
|
|
// Report text rides in PLAN_FILE_REVIEW_REPORT → every report consumer gets it.
|
|
// devex-review is a report consumer but NOT a gate consumer, so the two target
|
|
// sets differ (CP5/CX5). Regression guard: a future token-cut that drops the
|
|
// unresolved-status line again fails here. See plan-flag-unresolved-issues.
|
|
const REPORT_CONSUMERS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'codex',
|
|
'devex-review',
|
|
];
|
|
// Gate text rides in EXIT_PLAN_MODE_GATE (lives in SKILL.md, not sections).
|
|
const GATE_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'codex',
|
|
];
|
|
|
|
for (const skill of REPORT_CONSUMERS) {
|
|
test(`${skill}: report mandates the unresolved-decisions status as final content`, () => {
|
|
const content = readSkillUnion(skill);
|
|
expect(content).toContain('NO UNRESOLVED DECISIONS');
|
|
// The "never omit / always final" contract must be present, not just the phrase.
|
|
expect(content).toContain('Unresolved-decisions status (MANDATORY');
|
|
expect(content).toMatch(/never omitted/);
|
|
// \s+ tolerates prose line-wraps within "final non-whitespace line".
|
|
expect(content).toMatch(/final\s+non-whitespace\s+line/);
|
|
});
|
|
}
|
|
|
|
for (const skill of GATE_SKILLS) {
|
|
test(`${skill}: exit gate blocks unless the unresolved status is the final line`, () => {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
// Gate check #4 — present, sentinel named, and explicitly blocking (no escape).
|
|
expect(md).toContain('NO UNRESOLVED DECISIONS');
|
|
expect(md).toContain('FINAL non-whitespace line is the unresolved-decisions');
|
|
expect(md).toContain('FAILS the gate');
|
|
});
|
|
}
|
|
|
|
test('scripts/resolvers/review.ts source carries the mandatory block + blocking gate', () => {
|
|
const src = fs.readFileSync(path.join(ROOT, 'scripts', 'resolvers', 'review.ts'), 'utf-8');
|
|
// Report resolver: mandatory, never-omitted, exact sentinel, anti-double-count algorithm.
|
|
expect(src).toContain('Unresolved-decisions status (MANDATORY');
|
|
expect(src).toContain('NO UNRESOLVED DECISIONS');
|
|
expect(src).toContain('avoids double-counting');
|
|
expect(src).toContain('DROP the current skill');
|
|
// Gate resolver: the blocking final-line check with no "if applicable" escape.
|
|
expect(src).toContain('FINAL non-whitespace line is the unresolved-decisions');
|
|
expect(src).toContain('FAILS the gate');
|
|
// The old soft wording must be gone from the gate.
|
|
expect(src).not.toContain('absorbs CODEX / CROSS-MODEL / UNRESOLVED lines if applicable');
|
|
});
|
|
});
|
|
|
|
// ─── {{PREAMBLE}} requires an explicit preamble-tier ────────
|
|
|
|
describe('PREAMBLE resolution requires declared preamble-tier', () => {
|
|
test('resolving {{PREAMBLE}} without preamble-tier throws with the template path', async () => {
|
|
const { generatePreamble } = await import('../scripts/resolvers/preamble');
|
|
const { HOST_PATHS } = await import('../scripts/resolvers/types');
|
|
const ctx = {
|
|
skillName: 'tierless-skill',
|
|
tmplPath: 'tierless-skill/SKILL.md.tmpl',
|
|
host: 'claude' as const,
|
|
paths: HOST_PATHS.claude,
|
|
// preambleTier deliberately absent — the generator must refuse to default it.
|
|
};
|
|
expect(() => generatePreamble(ctx)).toThrow(/tierless-skill\/SKILL\.md\.tmpl/);
|
|
expect(() => generatePreamble(ctx)).toThrow(/preamble-tier/);
|
|
});
|
|
|
|
test('every template that resolves {{PREAMBLE}} declares preamble-tier in frontmatter', () => {
|
|
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
|
|
const offenders: string[] = [];
|
|
const checkTmpl = (tmplPath: string) => {
|
|
const tmpl = fs.readFileSync(tmplPath, 'utf-8');
|
|
if (tmpl.includes('{{PREAMBLE}}') && !/^preamble-tier:\s*\d+$/m.test(tmpl)) {
|
|
offenders.push(path.relative(ROOT, tmplPath));
|
|
}
|
|
};
|
|
checkTmpl(path.join(ROOT, 'SKILL.md.tmpl'));
|
|
for (const e of entries) {
|
|
if (!e.isDirectory() || e.name.startsWith('.') || e.name === 'node_modules') continue;
|
|
const tmplPath = path.join(ROOT, e.name, 'SKILL.md.tmpl');
|
|
if (fs.existsSync(tmplPath)) checkTmpl(tmplPath);
|
|
}
|
|
expect(offenders).toEqual([]);
|
|
});
|
|
});
|