mirror of
https://github.com/garrytan/gstack.git
synced 2026-08-29 09:20:39 +02:00
* feat(gen): strip gen-time-only frontmatter keys from Claude renders
interactive + benefits-from are read from the .tmpl by buildContext at
generation time; no runtime, host, or test reader consumes them from the
generated SKILL.md (e2e-harness-audit reads .tmpl; benefits-from tests
assert rendered prose). gbrain: stays (bin/gstack-brain-context-load reads
it from the installed render); hooks: stays (Claude Code host wires
PreToolUse from it).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(gen): regenerate SKILL.md — dead frontmatter keys removed
Mechanical regen after hosts/claude.ts stripFields change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(test): context-budget ratchet — CI ceilings on always-on + eager token ledgers
New free test grades the two ledgers nothing else guards: the full-frontmatter
always-on catalog (aggregate) and per-skill eager tokens (SKILL.md +
forced-read refs), via checkBudget from lib/context-bill.ts. Ceilings live in
test/fixtures/context-budget.json with x1.05/x1.10 headroom; regenerate with
bun test/helpers/capture-context-budget.ts. New skills fail until consciously
budgeted; removed skills fail until the fixture is refreshed; reductions
ratchet the ceilings down so wins lock in.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(todos): file output-template carve wave + plan-ceo doctrine revisit; mark preamble-carve P3 in flight
Two follow-ups deferred from the approved token-reduction program (CEO review
'NOT in scope' list), filed with full context per TODOS format. The existing
P3 preamble-carve entry gets a status update pointing at the program that
supersedes it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): review findings — Windows path normalization, full totals rebuild, ratchet coverage
Pre-landing review (5 specialists) found one critical: the ratchet test runs
in the curated Windows lane, where path.relative yields backslash skill names
that miss the test/ filter and mismatch every POSIX fixture key. Names are now
normalized once in buildRatchetBill (toPosixName) and the fixture filter is
tightened to test/fixtures/. All eight Bill.totals fields are rebuilt from the
filtered list (no fixture-polluted perInvocation/totalMd numbers for future
consumers). New coverage: Windows-separator normalization pins, a
captureContextBudget round-trip against tree-a (headroom math exact), a
stripFields regression pin (interactive/benefits-from absent from renders,
hooks/gbrain preserved), and the ceilings test no longer double-reports
stale-fixture entries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): adversarial findings — stable root key, symlink-alias dedupe, fixture-shape guard
Adversarial review (Claude subagent) verified the fixture's root-skill key was
the capture machine's checkout dirname: any non-gstack-named clone (every
Conductor worktree) failed the free suite, and the documented re-run-the-capture
recovery baked the local dirname into the committed fixture — silent corruption
through the tool's own protocol. The root skill is now pinned to ROOT_SKILL_KEY
('gstack', its frontmatter name). Symlink aliases are realpath-deduped (census
precedent): connect-chrome no longer gets its own ceiling, so Windows checkouts
that materialize the symlink as a plain file can't fail the stale-ceiling
set-equality test. New guards: fixture-shape validation (a string alwaysOnTotal
can no longer silently disable the ceiling), a mutation pin that the filter
shrinks the always-on ledger vs the raw bill, an alwaysOnTotal violation test
(the branch was load-bearing with only under-budget coverage), and an atomic
temp+rename fixture write. Fixture regenerated: 59 ceilings, alwaysOnTotal 6344.
Deferred with a TODO: anchoring transformFrontmatter's denylist strip to the
frontmatter block (latent, zero live collisions, pre-existing path).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v1.69.1.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v1.69.1.0
CLAUDE.md: Token ceiling section documents the context-budget ratchet as
the third guard (test file, fixture, new-skill budgeting, capture command).
CONTRIBUTING.md: Tier 1 guard list gains a Context-budget ratchet bullet;
the Adding-a-new-skill checklist gains the budget-capture step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: pin exact guard semantics for the context-budget ratchet in CLAUDE.md
Doc-review finding: "a third enforced ceiling" undercounted the guard
family (skill-size-budget floors and parity ratios also watch these
ledgers, relatively). Rephrased to match the ratchet test's own header:
absolute ceilings vs relative floors/ratios.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): heaviest-skill claim matches the fixture (land-and-deploy edges review by 0.2%)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(bin): gstack-skill-start + gstack-skill-end — the preamble runtime, consolidated
Absorbs the ~13KB of bash every tier-2+ SKILL.md inlined twice over (bootstrap
fence + artifacts-sync fence) and the skill-end telemetry/sync fences. Same
KEY: value STATUS-line contract the prose interprets, plus SKILL_START_PROTO
handshake (OV5), SESSION_ID/TEL_START echoes, GSTACK_HOME-normalized state
paths (EOV7), --parent-pid session identity (EOV5: $PPID inside the script is
the ephemeral tool-call shell), OV4 sanitization of passthrough output, and a
receipted daily artifacts pull (_receipted_git, brain-sync class, fail-closed).
Per-line || true error style throughout (F3) — a mid-script failure never drops
later STATUS lines.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(gen): preamble resolvers emit a script invocation fence instead of inline bash
generate-preamble-bash: ~6.3KB fence -> 4-line gstack-skill-start invocation
(quoted-tilde pitfall handled: leading ~ interpolates through $HOME; env-var
hosts keep $GSTACK_BIN) + degraded-mode prose (F1/EOV8: safe defaults, consent
gates deferred-never-lost; OV5: proto rule). generate-brain-sync-block: ~6.8KB
bash -> interpretation prose + the privacy stop-gate (stays inline until
Phase 2's gated emission). generate-completion-status: telemetry fence -> one
gstack-skill-end call with SESSION_ID/TEL_START handoff.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(gen): regenerate all skills + golden fixtures — inline preamble bash removed
Mechanical regen after the resolver change: −12,628 lines across 52 renders
(corpus 952K -> 806K render tokens; tier-2 skills −11-13KB each). Golden
per-host ship fixtures refreshed from the fresh claude/codex/factory renders.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: skill-start contract suite + preamble A/B eval + touchfiles registration
test/gstack-skill-start.test.ts (11 free tests): STATUS-key contract vs the
prose (F2), per-host fence resolution shapes (E1), proto-first, OV4 marker
sanitization, --parent-pid identity, headless suppression, skill-end duration
math + pending cleanup. test/skill-e2e-preamble-script-ab.test.ts (gate tier,
OV7): inline-bash render (pinned from 29785978) vs script render with the
fence redirected at the worktree bin (EOV2 — hermetic evals otherwise resolve
the operator install and silently exercise degraded mode). 21 touchfiles dep
lists gain the two bin scripts (EOV9) so future script edits select the
preamble evals; selection-count pin updated 23->24.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: repin ~70 assertions to the script contract — every literal gets a successor
Assertions that pinned inline-bash internals (update-check guard, _SESSIONS
reaping, telemetry start/end blocks, routing probe, repo-strip producer,
first-task gating, EXPLAIN_LEVEL/QUESTION_TUNING echoes, #2499 jq scope
resolution, Issue-8 CONDUCTOR gate) now pin the same invariants in their new
home: bin/gstack-skill-start / bin/gstack-skill-end file content for script
internals, the invocation fence + interpretation prose for render-side
behavior. No assertion deleted without a successor; live-execution tests
(routing probe, brain-sync jq) run against script bytes unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(test): re-baseline size floors + ratchet ceilings down (EOV1/OV9 protocol)
parity-baseline-v1.69.1.0.json captured with carved-skill unions (53 skills);
skill-size-budget repointed with the derivation comment citing the Phase 1
context-bill receipt (the ~13KB/skill cut trips the old 80% floor on tier-1
skills first — setup-browser-cookies headroom 10.8KB < the cut). The v1.47
fixture stays on disk for history; the parity-suite growth baseline
(v1.64.1.0) is untouched. Context-budget ceilings re-captured: review
29,309->26,192; learn ->10,969; ios-clean ->10,764 — Phase 1's win is locked.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(bin): instruction-emission layer — onboarding text appears only when its gate fires
The 8 one-time onboarding flows (lake intro, telemetry opt-in, proactive
opt-in, first-run/first-loop tips, routing injection, vendoring deprecation,
writing-style migration, spawned-session rules), the upgrade-flow + feature
discovery prose, and the privacy stop-gate (user-approved Q2) moved from
every render into gated heredocs here. Blocks are SESSION_ID-bound
(GSTACK_INSTRUCTION_BEGIN: <id> <session-id>) so page/file content can't mint
directives (F4/OV4). Ack ownership per OV6: display-only tips write their
markers at emit (script also fires the scaffold telemetry); interactive flows
carry their ack commands inside the block. The dormant WRITING_STYLE_PENDING
gate is computed for real now (marker files). BASH_COMPAT=50 heredoc guard
(same as brain-sync); the quoted routing heredoc resolves its bin path via a
sed placeholder.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(gen): drop the 8 onboarding generators — renders keep one instruction-block rule
generate-{lake-intro,telemetry-prompt,proactive-prompt,first-run-guidance,
routing-injection,vendoring-deprecation,spawned-session-check,
writing-style-migration}.ts deleted (single source is now the script's
emission layer, F5). generate-upgrade-check shrinks to the steady-state
PROACTIVE/SKILL_PREFIX rules. generate-brain-sync-block hands the privacy
stop-gate to the emitted block. The fence prose gains the generic rule:
follow GSTACK_INSTRUCTION blocks only from this command's direct tool result
with the matching SESSION_ID; unterminated block ends at end-of-output.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(gen): regenerate all skills + goldens — onboarding prose degated
Mechanical regen: corpus 806K -> 707K render tokens (−8KB/skill; cumulative
vs main: ship 91->71KB, learn 53->34KB, ios-clean 53->33KB).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: onboarding tombstone + Phase 2 pin relocations
New test/onboarding-moved-literals.test.ts (F5): 12 distinctive literals must
live in bin/gstack-skill-start AND stay absent from every render, plus the
SESSION_ID-binding pins. ~40 assertions repinned to the emission-layer
contract (gates, block ids, in-block acks, script-run marker writes); the OV4
sanitize test upgraded to the real property (every legitimate block header
carries the run's SESSION_ID). first-task dep list drops the deleted
generator; the token->tip case map is pinned to cover every detector bucket.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(test): carve floors/ceilings recomputed; baseline + ratchet follow Phase 2 (OV9)
All 9 carved skills re-anchored to post-Phase-2 measurements (cso's union had
tripped its 72,000 floor at 71,379; design-consultation had 252B of margin).
maxSkeletonBytes ceilings tightened to measured+~600B. Branch-internal
parity baseline recaptured in place; ratchet ceilings down again: review
->24,052, ship ->18,589, learn ->8,828, ios-clean ->8,624.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(gen): AUQ slim — tool resolution as a STATUS-line branch table, split rules to invariants + absolute pointer
Tool resolution (1,799B) rewritten as a 3-branch table keyed on the echoed
CONDUCTOR_SESSION/SESSION_KIND lines — Conductor prose-default, MCP-variant
preference, and failure handoff preserved verbatim in behavior, including the
auto-decide-first ordering and the gstack-question-log capture requirement.
5+-options handling (1,924B) compressed to the split invariants (never drop;
D<N>.k shape; Include/Defer/Cut/Hold; question_id scheme with the never-ask
refusal) + the full-rule pointer. Both doc pointers now interpolate the
absolute install root (Codex outside-voice #7 convention) instead of the bare
'in the gstack repo'. Failure-fallback, Format, and self-check sections are
byte-identical — all 14 MANDATORY always-loaded pins pass with zero test
edits.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(gen): regenerate all skills + goldens — AUQ slim
Mechanical regen: −1.3KB per tier-2+ skill (ship 69.9KB, learn 32.5KB).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(test): baseline + ratchet follow Phase 3 (OV9); OV8 evaluated — shrink floor stays
Branch-internal baseline recaptured; ratchet ceilings down again. OV8's
floor-retirement question, evaluated as planned after Phase 3: the 80% shrink
floor stays — it uniquely catches accidental body deletion in non-carved
skills BETWEEN ratchet recaptures, and the capture command has amortized the
fixture-refresh cost that motivated retiring it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(review): carve adversarial, plan-completion, and review-army into sections
The three resolver macros ship already carves as siblings now load on demand
for /review too: skeleton 100.2KB -> 55.0KB (-45%), union 93.4KB. Resolvers
stay the single source of truth (sections wrap the macros). Step 0/1, scope
drift, critical pass, confidence calibration, and fix-first stay always-loaded.
Fixtures and pins follow the moved content (codex-hardening wrapped-sites,
review-army E2E fixture builds skeleton+sections with an empty-fixture guard).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(codex): carve the three mutually exclusive modes into sections
Review/Challenge/Consult mode bodies (34.7KB where at most one ever runs)
load on demand: skeleton 81.0KB -> 55.2KB, union 1.04x the monolith. The mode
dispatch, filesystem boundary, and a new always-loaded 'Synthesis
recommendation (REQUIRED) — all modes' block stay skeleton-side (the AUQ
per-skill pins pass unchanged); the plan-file report + exit gate render after
the last section pointer per the gateAfterStop pattern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(land-and-deploy): carve first-run validation, readiness gate, and merge/deploy into sections
The once-per-repo dry-run validation, the pre-merge readiness gate, and the
merge + deploy-strategy steps (37.8KB) load on demand: skeleton 91.1KB ->
55.7KB. Step 1.5 keeps its detection bash as the dispatch; the first-run
section's fingerprint-save block gained {{SLUG_EVAL}} so it is self-contained.
Zero content lost (line-coverage checked against HEAD).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ios): demote the four ios skills to preamble-tier 2 (Phase 5)
They never consume the tier-3 sections (repo-mode ownership, search-before-
building) but do fire AskUserQuestion, which tier >=2 provides — verified by
grep before the plan review. -2.2KB per skill. Render assertions pin the
demotion (tier-3 sections absent, AUQ format present).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(guards): register wave-1 carves; monolith invariants retire; baselines + ratchet follow
CARVE_GUARDS gains review/codex/land-and-deploy (12 carved skills total);
their MONOLITH_INVARIANTS entries retire (invariants now generate from the
registry, cso precedent). Touchfiles: carve-section-loading covers the three
new carves; the codex + land-and-deploy LLM-judge dep lists widen to their
sections. Regen + goldens + branch-internal baseline + ratchet ceilings
recaptured (review 24,052 -> skeleton-based ceiling; union floors hold).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(gen-skill-docs): review render pins read the carved union
The review carve's readSkillUnion conversions (same pattern its neighbor
carved-skill pins already use).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(autoplan): carve the four review phases + tasks aggregator into sections
Phase bodies (CEO/Design/Eng/DX consensus flows) and the Implementation Tasks
aggregator load on demand; Design and DX stay separate sections because each
is independently conditional on scope. Skeleton 83.7KB -> 58.7KB (-30%
always-loaded); the 6 decision principles, classification, sequencing, and
explicit skip-condition dispatch stay always-loaded. The chain E2E's
phase-complete markers now live only in sections, so its assertions double as
section-read proof (behavioral: external).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(spec): carve the post-confirmation gate-and-file tail into one section
Phases 1-4 are the turn-1 conversational spine — carving them would force the
Read on the first user message for zero real savings. The mechanical tail
(4.5/4.5a/4.5b redaction gates + Phase 5 filing + TTHW telemetry) fires only
after draft confirmation: a genuine lazy boundary, kept as ONE section so the
gh-issue-create bash can never load without the fail-closed redaction gate
that precedes it. Skeleton 65.4KB -> 50.7KB; all ~85 phase-structure
invariants migrated location-aware plus a new carve-shape suite (56 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(setup-gbrain): carve the branch-exclusive install paths into sections
Brain-init (Paths 1/2/3/4 bodies), engine remediation, transcript gate, and
CLAUDE.md persist load on demand — at most one install route ever runs.
Skeleton 75.3KB -> 57.0KB; the Step 1 detect and Step 2 path dispatch stay
always-loaded. New buildSetupGbrainFixture helper gives the periodic E2Es
extract-don't-copy fixtures with a non-empty guard; the voyage-code-3 gate
counts scan the tmpl union (the third init site lives in engine-remediation).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(guards): register wave-2 carves (15 carved skills); autoplan monolith retires; baselines follow
CARVE_GUARDS gains autoplan (behavioral: external via the chain eval), spec,
and setup-gbrain; autoplan's MONOLITH_INVARIANTS entry retires. Touchfiles:
setup-gbrain periodic dep lists gain the section tmpls + fixture helper; the
stale-brain-refs scan covers setup-gbrain/sections. Regen + goldens + branch
baseline + ratchet recaptured.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(qa): carve QA patterns + health rubric into on-demand sections (68→48KB skeleton)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(browse): carve full command list + snapshot flags into sections/command-list.md (39→27KB skeleton)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(retro): absorb inline git/awk metrics into bin/gstack-retro-metrics + carve report format
RETRO_METRICS_PROTO: 1 contract, local git reads only (fetch stays in the
skill prose), degraded path documented in the skeleton.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: register wave-3 carves (qa, browse, retro) — guards, touchfiles, pins, baselines
CARVE_GUARDS gains the three entries; qa's monolith invariant retires.
auq-format carve-safety now keys on the skeleton+sections union shipping
the AUQ block (first tier-1 carve: browse never renders it by design).
Baselines: parity v1.69.1.0 at 18 sectioned skills; ratchet recaptured.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): drop stale generate-lake-intro import (generator deleted in the emission-layer move)
Sol scope discipline stays pinned via the model overlay + completeness
section; the lake intro is now a single script-emitted blurb.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(office-hours): carve Phase 2A/2B into mode-exclusive sections (81→67KB skeleton)
A session runs exactly one mode, so a builder session never loads the
13KB startup diagnostic. Mode mapping and the vibe-shift upgrade rule
stay in the skeleton.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(design): carve UX doctrine + Pretext patterns into read-on-demand sections
design-html 57→49KB, design-shotgun 53→50KB. Sections wrap
{{UX_PRINCIPLES}} so scripts/resolvers/design.ts stays the source of
truth; the pretext-patterns STOP sits at the top of Step 3 so the read
provably precedes the Write.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: register wave-4 carves (office-hours ext, design-html, design-shotgun) — 20 carved skills
Both design entries carry requiredReads + loading-eval scenarios (D3A
condition). office-hours phase sections are mode-exclusive, so only the
always-reached design/handoff section is a deterministic requiredRead.
Baselines and ratchet recaptured.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: trim CLAUDE.md 66.4→44.9KB — verbatim moves to docs/, pointers stay inline
Moved: browser/sidebar/server internals, CHANGELOG release-summary format
spec, project tree, hermetic-E2E detail, slop-scan reference, OpenClaw
publishing. Kept inline: every hard behavioral rule (dist/ ban, redaction
scan-at-sink, egress receipts, bisect commits, eval detach, CHANGELOG
entry rules), the machine-managed GBrain block (byte-identical), and the
'## Deploying to the active skill' header with gbrain-refresh in range
(pinned by test/gbrain-refresh-install-render.test.ts). No voice rewrites.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): seed onboarding markers into the hermetic child GSTACK_HOME
EOV7 made bin/gstack-skill-start honor GSTACK_HOME, so the operator-HOME
seeding in e2e-helpers.ts no longer reaches hermetic children — the
emission layer fired lake-intro/telemetry prompts that burned turns and
stalled PTY tests waiting on an answer (observed: plan-mode-no-op derailed
by the telemetry question). Onboarding-specific tests pin their own
GSTACK_HOME per-test, which merges over this seed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: raise carve-section-loading wall clock to 480s SDK / 540s bun
The heavy full-workflow scenarios satisfy their required section reads
inside 60s but need 300-450s to finish the report on slower sandboxes;
the 300s default read as a loading failure when the carve invariant held
(traces: plan-eng-review read its section at 8s, office-hours all three
at 24s, design-html both at 50s — all timed out mid-report).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(security): harden the skill-start trust boundary — review-army findings
Session ID gains a urandom suffix (block binding unforgeable by reflected
content); _sanitize also neutralizes spoofed SESSION_ID: lines; branch
names are charset-clamped before JSON embedding (skill-start + skill-end);
.brain-last-push reads first line only with a charset clamp; the artifacts
URL echo routes through _sanitize; the privacy consent gate fires in
interactive sessions only (spawned auto-choose could accept consent no
human gave — emission order is not a safety property); the daily pull gets
non-interactive + slow-network git guards and stamps only when the
receipted path ran; ~/.claude.json gets a grep pre-filter before the jq
parse.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(resolvers): question-log session_id becomes a substitution placeholder + stale-comment sweep
The question-log block bound $_SESSION_ID, a shell variable the
consolidated fence never sets — hook-less hosts logged empty session_id,
breaking /plan-tune per-session grouping. It now uses the same
substitute-from-the-skill-start-echoes contract as the telemetry block.
Also: retired the pre-Phase-2 stop-gate docstring, repointed the
gbrain-local-status cross-reference at the script's inline jq, dropped an
orphaned section comment, documented retro-metrics' suffix-only census.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: regenerate renders for the question-log placeholder; goldens + baselines follow
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: hermetic update-check, onboarding gate sequencing, seeding parity
The contract test's child did a live git ls-remote + curl to github.com on
every bun run test (update_check config now gates it off); the headless
test gets a fresh GSTACK_HOME so the suppression is actually exercised; a
new OV6 test drives the script three times to pin ack-at-emit and gate
sequencing; hermetic seeding covers the config-keyed privacy gate; the
EVALS_HERMETIC=0 debug seeding reaches marker parity.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(ci): demote the preamble A/B to periodic (OV7) and add it to the periodic matrix
Post-Phase-3 demotion per the plan; the eval needs fetch-depth 0 (it git
shows a pre-Phase-1 sha), which only the periodic workflow provides — and
a static matrix entry so it can't silently never run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v1.70.0.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v1.70.0.0
ARCHITECTURE.md: the preamble section now describes the v1.70 runtime —
the rendered {{PREAMBLE}} block invokes bin/gstack-skill-start and reads
STATUS lines, gstack-skill-end logs telemetry, and one-time onboarding
text arrives as gated GSTACK_INSTRUCTION blocks instead of riding in
every render.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: doc-review fixes — repair moved-file links, drop unbacked session-count claim
docs/BROWSER_INTERNALS.md: the two ARCHITECTURE.md anchor links broke when
the section moved from repo-root CLAUDE.md into docs/ — now ../ARCHITECTURE.md.
ARCHITECTURE.md: the preamble's session-tracking item claimed an active-session
count and an "ELI16 mode" that no shipped code implements (the count
computation was deleted with the inline preamble); describe the real
touch-and-prune behavior instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): correct numeric claims against measured counts
50 of 62 installed skills dropped (fixture/alias entries have no preamble);
11 new carves + a deeper office-hours carve = 9→20; test counts match the
files (13 / 11 / 3 / 7).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: repoint the preamble-runtime version reference after the queue rebump (v1.71.0.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e-design): widen the Aesthetic synonym set — vocabulary variance, not a regression
Both attempts in run 33090283032 produced judge-praised DESIGN.md files
phrased as 'design principles'/'design language' without any of the four
original literals; inputs were identical to the prior passing run
32899975845 (design-consultation untouched by the intervening merge).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): stage design-consultation's sections/ into the E2E fixture
The skill has been carved since v1.57.0.0 — the DESIGN.md structure
prescription (the AESTHETIC proposal template) lives in
sections/proposal-and-preview.md behind a STOP-read. The fixture only
copied SKILL.md, so the agent improvised structure from the skeleton and
the section-synonym check has been a coin flip since the carve (CI run
33090283032 trace shows 'no sections dir'; the local eval store has the
same failure on 2026-08-25 while that day's CI run passed on lucky
vocabulary).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
674 lines
25 KiB
TypeScript
674 lines
25 KiB
TypeScript
/**
|
|
* /plan-tune tests (gate tier)
|
|
*
|
|
* Covers the foundation of /plan-tune v1:
|
|
* - Question registry schema validation
|
|
* - Registry completeness (every AskUserQuestion pattern has an id)
|
|
* - Id uniqueness (no duplicates)
|
|
* - One-way door safety declarations
|
|
* - Signal map references valid registry ids
|
|
*
|
|
* Binary-level tests (question-log, question-preference, developer-profile)
|
|
* and migration tests live in sibling files created as those binaries ship.
|
|
*/
|
|
|
|
import { describe, test, expect } from 'bun:test';
|
|
import {
|
|
QUESTIONS,
|
|
getQuestion,
|
|
getOneWayDoorIds,
|
|
getAllRegisteredIds,
|
|
getRegistryStats,
|
|
type QuestionDef,
|
|
} from '../scripts/question-registry';
|
|
import {
|
|
classifyQuestion,
|
|
isOneWayDoor,
|
|
DESTRUCTIVE_PATTERN_LIST,
|
|
ONE_WAY_SKILL_CATEGORY_SET,
|
|
} from '../scripts/one-way-doors';
|
|
import {
|
|
SIGNAL_MAP,
|
|
applySignal,
|
|
validateRegistrySignalKeys,
|
|
newDimensionTotals,
|
|
normalizeToDimensionValue,
|
|
ALL_DIMENSIONS,
|
|
} from '../scripts/psychographic-signals';
|
|
import {
|
|
ARCHETYPES,
|
|
FALLBACK_ARCHETYPE,
|
|
matchArchetype,
|
|
getAllArchetypeNames,
|
|
} from '../scripts/archetypes';
|
|
import * as fs from 'fs';
|
|
import * as path from 'path';
|
|
|
|
const ROOT = path.resolve(import.meta.dir, '..');
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Schema validation
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('question-registry schema', () => {
|
|
test('every entry has required fields', () => {
|
|
for (const [key, q] of Object.entries(QUESTIONS as Record<string, QuestionDef>)) {
|
|
expect(q.id).toBeDefined();
|
|
expect(q.skill).toBeDefined();
|
|
expect(q.category).toBeDefined();
|
|
expect(q.door_type).toBeDefined();
|
|
expect(q.description).toBeDefined();
|
|
expect(q.description.length).toBeGreaterThan(0);
|
|
expect(q.id).toBe(key); // key and id must match
|
|
}
|
|
});
|
|
|
|
test('all ids are kebab-case and start with skill name', () => {
|
|
for (const q of Object.values(QUESTIONS as Record<string, QuestionDef>)) {
|
|
expect(q.id).toMatch(/^[a-z0-9-]+$/);
|
|
expect(q.id.startsWith(q.skill + '-')).toBe(true);
|
|
expect(q.id.length).toBeLessThanOrEqual(64);
|
|
}
|
|
});
|
|
|
|
test('no duplicate ids (keys and id fields are 1:1 by construction)', () => {
|
|
const ids = Object.values(QUESTIONS as Record<string, QuestionDef>).map((q) => q.id);
|
|
const unique = new Set(ids);
|
|
expect(unique.size).toBe(ids.length);
|
|
});
|
|
|
|
test('category is one of the allowed values', () => {
|
|
const ALLOWED = new Set(['approval', 'clarification', 'routing', 'cherry-pick', 'feedback-loop']);
|
|
for (const q of Object.values(QUESTIONS as Record<string, QuestionDef>)) {
|
|
expect(ALLOWED.has(q.category)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('door_type is one-way or two-way', () => {
|
|
for (const q of Object.values(QUESTIONS as Record<string, QuestionDef>)) {
|
|
expect(q.door_type === 'one-way' || q.door_type === 'two-way').toBe(true);
|
|
}
|
|
});
|
|
|
|
test('options (if present) are non-empty arrays of strings', () => {
|
|
for (const q of Object.values(QUESTIONS as Record<string, QuestionDef>)) {
|
|
if (q.options) {
|
|
expect(Array.isArray(q.options)).toBe(true);
|
|
expect(q.options.length).toBeGreaterThan(0);
|
|
for (const opt of q.options) {
|
|
expect(typeof opt).toBe('string');
|
|
expect(opt.length).toBeGreaterThan(0);
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
test('descriptions are short and informative (<= 200 chars, no newlines)', () => {
|
|
for (const q of Object.values(QUESTIONS as Record<string, QuestionDef>)) {
|
|
expect(q.description.length).toBeLessThanOrEqual(200);
|
|
expect(q.description.includes('\n')).toBe(false);
|
|
}
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Runtime helpers
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('question-registry helpers', () => {
|
|
test('getQuestion returns entry for known id', () => {
|
|
const q = getQuestion('ship-test-failure-triage');
|
|
expect(q).toBeDefined();
|
|
expect(q?.skill).toBe('ship');
|
|
expect(q?.door_type).toBe('one-way');
|
|
});
|
|
|
|
test('getQuestion returns undefined for unknown id', () => {
|
|
expect(getQuestion('this-is-not-registered')).toBeUndefined();
|
|
});
|
|
|
|
test('getOneWayDoorIds returns Set of one-way ids', () => {
|
|
const ids = getOneWayDoorIds();
|
|
expect(ids.has('ship-test-failure-triage')).toBe(true);
|
|
expect(ids.has('review-sql-safety')).toBe(true);
|
|
expect(ids.has('land-and-deploy-merge-confirm')).toBe(true);
|
|
// And does NOT include a known two-way door:
|
|
expect(ids.has('ship-changelog-voice-polish')).toBe(false);
|
|
});
|
|
|
|
test('getAllRegisteredIds count matches QUESTIONS keys', () => {
|
|
expect(getAllRegisteredIds().size).toBe(Object.keys(QUESTIONS).length);
|
|
});
|
|
|
|
test('getRegistryStats totals are consistent', () => {
|
|
const stats = getRegistryStats();
|
|
expect(stats.total).toBe(Object.keys(QUESTIONS).length);
|
|
expect(stats.one_way + stats.two_way).toBe(stats.total);
|
|
const bySkillSum = Object.values(stats.by_skill).reduce((a, b) => a + b, 0);
|
|
expect(bySkillSum).toBe(stats.total);
|
|
const byCategorySum = Object.values(stats.by_category).reduce((a, b) => a + b, 0);
|
|
expect(byCategorySum).toBe(stats.total);
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Safety contract — one-way doors
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('one-way door safety', () => {
|
|
test('every destructive/security question is declared one-way', () => {
|
|
// Safety-critical question ids must exist and be one-way.
|
|
const mustBeOneWay = [
|
|
'ship-test-failure-triage', // shipping broken tests
|
|
'review-sql-safety', // SQL injection path
|
|
'review-llm-trust-boundary', // LLM trust boundary
|
|
'cso-global-scan-approval', // scans outside branch
|
|
'cso-finding-fix', // security finding
|
|
'land-and-deploy-merge-confirm', // actual merge
|
|
'land-and-deploy-rollback', // rollback decision
|
|
'investigate-fix-apply', // applying a fix
|
|
'plan-ceo-review-premise-revise', // changing agreed premise
|
|
'plan-eng-review-arch-finding', // architecture change
|
|
'office-hours-landscape-privacy-gate',// sending data to search provider
|
|
'autoplan-user-challenge', // scope direction change
|
|
];
|
|
const oneWayIds = getOneWayDoorIds();
|
|
for (const id of mustBeOneWay) {
|
|
expect(getQuestion(id)).toBeDefined();
|
|
expect(oneWayIds.has(id)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('at least 10 one-way doors are declared', () => {
|
|
// Sanity check — if we lose one-way classification on critical questions,
|
|
// this fails before safety bugs ship.
|
|
expect(getOneWayDoorIds().size).toBeGreaterThanOrEqual(10);
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Coverage breadth — make sure we span the high-volume skills
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('registry breadth', () => {
|
|
test('high-volume skills have at least one registered question', () => {
|
|
const stats = getRegistryStats();
|
|
const highVolume = [
|
|
'ship',
|
|
'review',
|
|
'office-hours',
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'qa',
|
|
'investigate',
|
|
'land-and-deploy',
|
|
'cso',
|
|
];
|
|
for (const skill of highVolume) {
|
|
expect(stats.by_skill[skill] ?? 0).toBeGreaterThan(0);
|
|
}
|
|
});
|
|
|
|
test('preamble one-time prompts are registered (telemetry, proactive, routing)', () => {
|
|
expect(getQuestion('preamble-telemetry-consent')).toBeDefined();
|
|
expect(getQuestion('preamble-proactive-behavior')).toBeDefined();
|
|
expect(getQuestion('preamble-routing-injection')).toBeDefined();
|
|
});
|
|
|
|
test('/plan-tune itself registers its enable + setup + mutation-confirm', () => {
|
|
expect(getQuestion('plan-tune-enable-setup')).toBeDefined();
|
|
expect(getQuestion('plan-tune-declared-dimension')).toBeDefined();
|
|
expect(getQuestion('plan-tune-confirm-mutation')).toBeDefined();
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Signal map consistency
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('psychographic signal map', () => {
|
|
test('signal_keys in registry are typed strings', () => {
|
|
for (const q of Object.values(QUESTIONS as Record<string, QuestionDef>)) {
|
|
if (q.signal_key !== undefined) {
|
|
expect(typeof q.signal_key).toBe('string');
|
|
expect(q.signal_key.length).toBeGreaterThan(0);
|
|
expect(q.signal_key).toMatch(/^[a-z0-9-]+$/);
|
|
}
|
|
}
|
|
});
|
|
|
|
test('every signal_key in registry has a SIGNAL_MAP entry', () => {
|
|
const { missing } = validateRegistrySignalKeys();
|
|
expect(missing).toEqual([]);
|
|
});
|
|
|
|
test('applySignal mutates dimension totals per mapping', () => {
|
|
const dims = newDimensionTotals();
|
|
const applied = applySignal(dims, 'scope-appetite', 'expand');
|
|
expect(applied.length).toBeGreaterThan(0);
|
|
expect(dims.scope_appetite).toBeCloseTo(0.06, 5);
|
|
});
|
|
|
|
test('applySignal returns [] for unknown signal_key', () => {
|
|
const dims = newDimensionTotals();
|
|
const applied = applySignal(dims, 'no-such-signal', 'anything');
|
|
expect(applied).toEqual([]);
|
|
expect(dims.scope_appetite).toBe(0);
|
|
});
|
|
|
|
test('applySignal returns [] for unknown user_choice', () => {
|
|
const dims = newDimensionTotals();
|
|
const applied = applySignal(dims, 'scope-appetite', 'definitely-not-a-real-choice');
|
|
expect(applied).toEqual([]);
|
|
});
|
|
|
|
test('normalizeToDimensionValue maps 0 → 0.5 (neutral)', () => {
|
|
expect(normalizeToDimensionValue(0)).toBeCloseTo(0.5, 5);
|
|
});
|
|
|
|
test('normalizeToDimensionValue returns values in [0, 1]', () => {
|
|
for (const total of [-10, -1, -0.5, 0, 0.5, 1, 10]) {
|
|
const v = normalizeToDimensionValue(total);
|
|
expect(v).toBeGreaterThanOrEqual(0);
|
|
expect(v).toBeLessThanOrEqual(1);
|
|
}
|
|
});
|
|
|
|
test('ALL_DIMENSIONS has 5 entries', () => {
|
|
expect(ALL_DIMENSIONS.length).toBe(5);
|
|
});
|
|
|
|
test('no extra SIGNAL_MAP keys without registry reference (informational)', () => {
|
|
// Extra keys are allowed (a signal might be reserved for upcoming registry
|
|
// entries). But list them so drift is visible.
|
|
const { extra } = validateRegistrySignalKeys();
|
|
// Allow up to 3 "reserved" extras before flagging. Tighten later.
|
|
expect(extra.length).toBeLessThanOrEqual(3);
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Archetypes
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('archetypes', () => {
|
|
test('each archetype has name, description, center, tightness', () => {
|
|
for (const arch of ARCHETYPES) {
|
|
expect(arch.name).toBeDefined();
|
|
expect(arch.description).toBeDefined();
|
|
expect(arch.center).toBeDefined();
|
|
expect(arch.tightness).toBeGreaterThan(0);
|
|
for (const d of ALL_DIMENSIONS) {
|
|
expect(typeof arch.center[d]).toBe('number');
|
|
expect(arch.center[d]).toBeGreaterThanOrEqual(0);
|
|
expect(arch.center[d]).toBeLessThanOrEqual(1);
|
|
}
|
|
}
|
|
});
|
|
|
|
test('archetype names are unique', () => {
|
|
const names = ARCHETYPES.map((a) => a.name);
|
|
expect(new Set(names).size).toBe(names.length);
|
|
});
|
|
|
|
test('matchArchetype returns Cathedral Builder for boil-the-ocean profile', () => {
|
|
const dims = {
|
|
scope_appetite: 0.88,
|
|
risk_tolerance: 0.55,
|
|
detail_preference: 0.5,
|
|
autonomy: 0.5,
|
|
architecture_care: 0.85,
|
|
};
|
|
const match = matchArchetype(dims);
|
|
expect(match.name).toBe('Cathedral Builder');
|
|
});
|
|
|
|
test('matchArchetype returns Ship-It Pragmatist for small-scope/fast profile', () => {
|
|
const dims = {
|
|
scope_appetite: 0.22,
|
|
risk_tolerance: 0.78,
|
|
detail_preference: 0.25,
|
|
autonomy: 0.7,
|
|
architecture_care: 0.38,
|
|
};
|
|
const match = matchArchetype(dims);
|
|
expect(match.name).toBe('Ship-It Pragmatist');
|
|
});
|
|
|
|
test('matchArchetype returns Polymath for extreme-outlier profile', () => {
|
|
const dims = {
|
|
scope_appetite: 0.05,
|
|
risk_tolerance: 0.95,
|
|
detail_preference: 0.95,
|
|
autonomy: 0.05,
|
|
architecture_care: 0.05,
|
|
};
|
|
const match = matchArchetype(dims);
|
|
expect(match.name).toBe(FALLBACK_ARCHETYPE.name);
|
|
});
|
|
|
|
test('getAllArchetypeNames includes Polymath fallback', () => {
|
|
const names = getAllArchetypeNames();
|
|
expect(names).toContain('Polymath');
|
|
expect(names.length).toBe(ARCHETYPES.length + 1);
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Registry completeness — warn about SKILL.md.tmpl AskUserQuestion calls
|
|
// that don't appear to map to any registry entry.
|
|
//
|
|
// This is NOT a strict CI failure. Many AskUserQuestion invocations are
|
|
// dynamic (agent generates question text at runtime), which is fine — the
|
|
// agent picks the best-fitting registry id or generates an ad-hoc id.
|
|
//
|
|
// The test reports a count for visibility. A future enhancement will scan
|
|
// for specific question_id references in template prose and require those
|
|
// referenced ids to exist in the registry.
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('AskUserQuestion template coverage (informational)', () => {
|
|
test('count of templates using AskUserQuestion is non-trivial', () => {
|
|
const templates = findAllTemplates();
|
|
const usingAsk = templates.filter((p) =>
|
|
fs.readFileSync(p, 'utf-8').includes('AskUserQuestion'),
|
|
);
|
|
// At the time of writing, ~35 templates reference AskUserQuestion.
|
|
// This sanity check catches an accidental global removal.
|
|
expect(usingAsk.length).toBeGreaterThan(20);
|
|
});
|
|
|
|
test('registry covers >= 10 skills from template files', () => {
|
|
const stats = getRegistryStats();
|
|
expect(Object.keys(stats.by_skill).length).toBeGreaterThanOrEqual(10);
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// One-way door classifier (belt-and-suspenders keyword fallback)
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('one-way-doors classifier', () => {
|
|
test('registry lookup wins when question_id is known', () => {
|
|
const result = classifyQuestion({ question_id: 'ship-test-failure-triage' });
|
|
expect(result.oneWay).toBe(true);
|
|
expect(result.reason).toBe('registry');
|
|
|
|
const safeResult = classifyQuestion({ question_id: 'ship-changelog-voice-polish' });
|
|
expect(safeResult.oneWay).toBe(false);
|
|
expect(safeResult.reason).toBe('registry');
|
|
});
|
|
|
|
test('unknown question_id falls through to other checks', () => {
|
|
const result = classifyQuestion({ question_id: 'some-ad-hoc-question-id' });
|
|
expect(result.reason).not.toBe('registry');
|
|
});
|
|
|
|
test('keyword fallback catches destructive summaries', () => {
|
|
const cases = [
|
|
'Delete this directory and all its contents?',
|
|
'Run rm -rf /tmp/scratch — proceed?',
|
|
'Force-push main?',
|
|
'git reset --hard origin/main — ok?',
|
|
'DROP TABLE users — confirm?',
|
|
'kubectl delete namespace prod',
|
|
'terraform destroy the staging cluster',
|
|
'rotate the API key',
|
|
'breaking change to the public API — ship anyway?',
|
|
];
|
|
for (const summary of cases) {
|
|
const result = classifyQuestion({ summary });
|
|
expect(result.oneWay).toBe(true);
|
|
expect(result.reason).toBe('keyword');
|
|
expect(result.matched).toBeDefined();
|
|
}
|
|
});
|
|
|
|
test('skill-category fallback fires for cso:approval and land-and-deploy:approval', () => {
|
|
expect(isOneWayDoor({ skill: 'cso', category: 'approval' })).toBe(true);
|
|
expect(isOneWayDoor({ skill: 'land-and-deploy', category: 'approval' })).toBe(true);
|
|
});
|
|
|
|
test('benign questions default to two-way', () => {
|
|
const benign = [
|
|
'Want to update the changelog voice?',
|
|
'Which mode should plan review use?',
|
|
'Open the essay in your browser?',
|
|
];
|
|
for (const summary of benign) {
|
|
const result = classifyQuestion({ summary });
|
|
expect(result.oneWay).toBe(false);
|
|
expect(result.reason).toBe('default-two-way');
|
|
}
|
|
});
|
|
|
|
test('keyword patterns are non-empty', () => {
|
|
expect(DESTRUCTIVE_PATTERN_LIST.length).toBeGreaterThan(15);
|
|
});
|
|
|
|
test('skill-category set covers security + deploy', () => {
|
|
expect(ONE_WAY_SKILL_CATEGORY_SET.has('cso:approval')).toBe(true);
|
|
expect(ONE_WAY_SKILL_CATEGORY_SET.has('land-and-deploy:approval')).toBe(true);
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Preamble injection — the QUESTION_TUNING section must appear for tier >=2
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('preamble — QUESTION_TUNING injection', () => {
|
|
test('tier 2+ skills include the Question Tuning section', async () => {
|
|
const { generatePreamble } = await import('../scripts/resolvers/preamble');
|
|
const ctx = {
|
|
skillName: 'test-skill',
|
|
tmplPath: 'test.tmpl',
|
|
host: 'claude' as const,
|
|
paths: {
|
|
skillRoot: '~/.claude/skills/gstack',
|
|
localSkillRoot: '.claude/skills/gstack',
|
|
binDir: '~/.claude/skills/gstack/bin',
|
|
browseDir: '~/.claude/skills/gstack/browse/dist',
|
|
designDir: '~/.claude/skills/gstack/design/dist',
|
|
},
|
|
preambleTier: 2,
|
|
};
|
|
const out = generatePreamble(ctx);
|
|
// Phase 1: the config echo moved into bin/gstack-skill-start; the render's
|
|
// section gates itself on the echoed key.
|
|
const script = fs.readFileSync(
|
|
path.join(import.meta.dir, '..', 'bin', 'gstack-skill-start'),
|
|
'utf-8',
|
|
);
|
|
expect(script).toContain('echo "QUESTION_TUNING: $_QUESTION_TUNING"');
|
|
expect(out).toContain('QUESTION_TUNING: false');
|
|
expect(out).toContain('## Question Tuning');
|
|
expect(out).toContain('gstack-question-preference --check');
|
|
expect(out).toContain('gstack-question-log');
|
|
expect(out).toContain('profile-poisoning defense');
|
|
expect(out).toContain('inline-user');
|
|
});
|
|
|
|
test('tier 1 skills do NOT include Question Tuning section', async () => {
|
|
const { generatePreamble } = await import('../scripts/resolvers/preamble');
|
|
const ctx = {
|
|
skillName: 'test-skill',
|
|
tmplPath: 'test.tmpl',
|
|
host: 'claude' as const,
|
|
paths: {
|
|
skillRoot: '~/.claude/skills/gstack',
|
|
localSkillRoot: '.claude/skills/gstack',
|
|
binDir: '~/.claude/skills/gstack/bin',
|
|
browseDir: '~/.claude/skills/gstack/browse/dist',
|
|
designDir: '~/.claude/skills/gstack/design/dist',
|
|
},
|
|
preambleTier: 1,
|
|
};
|
|
const out = generatePreamble(ctx);
|
|
// QUESTION_TUNING config echo still fires (it's in the bash block which all tiers get),
|
|
// but the prose section should NOT be present for tier 1.
|
|
expect(out).not.toContain('## Question Tuning');
|
|
});
|
|
|
|
test('codex host produces different paths', async () => {
|
|
const { generateQuestionTuning } = await import('../scripts/resolvers/question-tuning');
|
|
const codexCtx = {
|
|
skillName: 'test',
|
|
tmplPath: 'x',
|
|
host: 'codex' as const,
|
|
paths: {
|
|
skillRoot: '$GSTACK_ROOT',
|
|
localSkillRoot: '.agents/skills/gstack',
|
|
binDir: '$GSTACK_BIN',
|
|
browseDir: '$GSTACK_BROWSE',
|
|
designDir: '$GSTACK_DESIGN',
|
|
},
|
|
};
|
|
const out = generateQuestionTuning(codexCtx);
|
|
expect(out).toContain('$GSTACK_BIN/gstack-question-preference');
|
|
expect(out).toContain('$GSTACK_BIN/gstack-question-log');
|
|
});
|
|
});
|
|
|
|
// -----------------------------------------------------------------------
|
|
// End-to-end: log → preference → derive pipeline
|
|
//
|
|
// Exercises the real binaries (not mocks) to make sure the schema contract
|
|
// between them actually holds.
|
|
// -----------------------------------------------------------------------
|
|
|
|
describe('end-to-end pipeline (binaries working together)', () => {
|
|
test('log many expand choices → derive pushes scope_appetite up', () => {
|
|
const tmpHome = fs.mkdtempSync(path.join(require('os').tmpdir(), 'gstack-e2e-'));
|
|
try {
|
|
// GSTACK_QUESTION_LOG_NO_DERIVE=1 suppresses gstack-question-log's
|
|
// fire-and-forget background `--derive` (it nohups one per write). Without
|
|
// it, the 5 rapid log writes spawn 5 racing background derives that collide
|
|
// with this test's explicit --derive below — a late background derive that
|
|
// only saw 3 entries can clobber developer-profile.json after the explicit
|
|
// one wrote sample_size=5, making the test flaky (~25-50% fail). The binary
|
|
// documents this flag for exactly this case. The explicit --derive still
|
|
// runs (it ignores the flag), so real derive behavior is still asserted.
|
|
const env = { ...process.env, GSTACK_HOME: tmpHome, GSTACK_QUESTION_LOG_NO_DERIVE: '1' };
|
|
const { spawnSync } = require('child_process');
|
|
const logBin = path.join(ROOT, 'bin', 'gstack-question-log');
|
|
const devBin = path.join(ROOT, 'bin', 'gstack-developer-profile');
|
|
|
|
for (let i = 0; i < 5; i++) {
|
|
const r = spawnSync(
|
|
logBin,
|
|
[
|
|
JSON.stringify({
|
|
skill: 'plan-ceo-review',
|
|
question_id: 'plan-ceo-review-mode',
|
|
question_summary: 'mode?',
|
|
user_choice: 'expand',
|
|
session_id: `s${i}`,
|
|
ts: `2026-04-0${i + 1}T10:00:00Z`,
|
|
}),
|
|
],
|
|
{ env, cwd: ROOT, encoding: 'utf-8' },
|
|
);
|
|
expect(r.status).toBe(0);
|
|
}
|
|
|
|
const derive = spawnSync(devBin, ['--derive'], { env, cwd: ROOT, encoding: 'utf-8' });
|
|
expect(derive.status).toBe(0);
|
|
|
|
const profileOut = spawnSync(devBin, ['--profile'], { env, cwd: ROOT, encoding: 'utf-8' });
|
|
const p = JSON.parse(profileOut.stdout);
|
|
expect(p.inferred.sample_size).toBe(5);
|
|
expect(p.inferred.values.scope_appetite).toBeGreaterThan(0.5);
|
|
} finally {
|
|
fs.rmSync(tmpHome, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
test('preference blocks tune: write from inline-tool-output in full pipeline', () => {
|
|
const tmpHome = fs.mkdtempSync(path.join(require('os').tmpdir(), 'gstack-e2e-'));
|
|
try {
|
|
const env = { ...process.env, GSTACK_HOME: tmpHome };
|
|
const { spawnSync } = require('child_process');
|
|
const prefBin = path.join(ROOT, 'bin', 'gstack-question-preference');
|
|
|
|
const r = spawnSync(
|
|
prefBin,
|
|
[
|
|
'--write',
|
|
JSON.stringify({ question_id: 'fake-id', preference: 'never-ask', source: 'inline-tool-output' }),
|
|
],
|
|
{ env, cwd: ROOT, encoding: 'utf-8' },
|
|
);
|
|
expect(r.status).toBe(2);
|
|
expect(r.stderr).toContain('poisoning');
|
|
|
|
// Verify no preference was written
|
|
const read = spawnSync(prefBin, ['--read'], { env, cwd: ROOT, encoding: 'utf-8' });
|
|
const prefs = JSON.parse(read.stdout);
|
|
expect(prefs['fake-id']).toBeUndefined();
|
|
} finally {
|
|
fs.rmSync(tmpHome, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
test('migration preserves sessions, builder-profile shim still works', () => {
|
|
const tmpHome = fs.mkdtempSync(path.join(require('os').tmpdir(), 'gstack-e2e-'));
|
|
try {
|
|
const env = { ...process.env, GSTACK_HOME: tmpHome };
|
|
const { spawnSync } = require('child_process');
|
|
const devBin = path.join(ROOT, 'bin', 'gstack-developer-profile');
|
|
const shimBin = path.join(ROOT, 'bin', 'gstack-builder-profile');
|
|
|
|
// Seed a legacy file
|
|
fs.writeFileSync(
|
|
path.join(tmpHome, 'builder-profile.jsonl'),
|
|
[
|
|
{ date: '2026-01-01', mode: 'builder', project_slug: 'x', signals: ['taste'] },
|
|
{ date: '2026-02-01', mode: 'startup', project_slug: 'x', signals: ['named_users'] },
|
|
{ date: '2026-03-01', mode: 'builder', project_slug: 'y', signals: ['agency'] },
|
|
]
|
|
.map((e) => JSON.stringify(e))
|
|
.join('\n') + '\n',
|
|
);
|
|
|
|
// Migrate
|
|
const m = spawnSync(devBin, ['--migrate'], { env, cwd: ROOT, encoding: 'utf-8' });
|
|
expect(m.status).toBe(0);
|
|
|
|
// Legacy shim should still return the same KEY: VALUE shape
|
|
const shimOut = spawnSync(shimBin, [], { env, cwd: ROOT, encoding: 'utf-8' });
|
|
expect(shimOut.status).toBe(0);
|
|
expect(shimOut.stdout).toContain('SESSION_COUNT: 3');
|
|
expect(shimOut.stdout).toContain('TIER: welcome_back');
|
|
expect(shimOut.stdout).toContain('CROSS_PROJECT: true');
|
|
} finally {
|
|
fs.rmSync(tmpHome, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
function findAllTemplates(): string[] {
|
|
const results: string[] = [];
|
|
function walk(dir: string) {
|
|
let entries: fs.Dirent[];
|
|
try {
|
|
entries = fs.readdirSync(dir, { withFileTypes: true });
|
|
} catch {
|
|
return;
|
|
}
|
|
for (const entry of entries) {
|
|
const full = path.join(dir, entry.name);
|
|
if (entry.isDirectory()) {
|
|
// Skip node_modules and dotfiles
|
|
if (entry.name === 'node_modules' || entry.name.startsWith('.')) continue;
|
|
walk(full);
|
|
} else if (entry.isFile() && entry.name === 'SKILL.md.tmpl') {
|
|
results.push(full);
|
|
}
|
|
}
|
|
}
|
|
walk(ROOT);
|
|
return results;
|
|
}
|