Commit Graph

4 Commits

Author SHA1 Message Date
Garry Tan 0427c957f2 test: end-to-end pipeline + preamble injection coverage
Added 6 tests to test/plan-tune.test.ts:

Preamble injection (3 tests):
- tier 2+ includes Question Tuning section with preference check, log,
  and user-origin gate language ('profile-poisoning defense', 'inline-user')
- tier 1 does NOT include the prose section (QUESTION_TUNING bash echo
  still fires since it's in the bash block all tiers share)
- codex host swaps binDir references to $GSTACK_BIN

End-to-end pipeline (3 tests) — real binaries working together, not mocks:
- Log 5 expand choices → --derive → profile shows scope_appetite > 0.5
  (full log → registry lookup → signal map → normalization round-trip)
- --write source: inline-tool-output rejected; --read confirms no pref
  was persisted (the profile-poisoning defense actually works end-to-end)
- Migrate a 3-session legacy file; confirm legacy gstack-builder-profile
  shim still returns SESSION_COUNT: 3, TIER: welcome_back, CROSS_PROJECT: true

test/plan-tune.test.ts now has 47 tests total.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 06:45:09 +08:00
Garry Tan 5e20a3b718 feat: psychographic signal map + builder archetypes
scripts/psychographic-signals.ts — hand-crafted {signal_key, user_choice} →
{dimension, delta} map. Version 0.1.0. Conservative deltas (±0.03 to ±0.06
per event). Covers 9 signal keys: scope-appetite, architecture-care,
code-quality-care, test-discipline, detail-preference, design-care,
devex-care, distribution-care, session-mode.

Helpers: applySignal() mutates running totals, newDimensionTotals() creates
empty starting state, normalizeToDimensionValue() sigmoid-clamps accumulated
delta to [0,1] (0 → 0.5 neutral), validateRegistrySignalKeys() checks that
every signal_key in the registry has a SIGNAL_MAP entry.

In v1 the signal map is used ONLY to compute inferred dimension values for
/plan-tune inspection output. No skill behavior adapts to these signals
until v2.

scripts/archetypes.ts — 8 named archetypes + Polymath fallback:
- Cathedral Builder (boil-the-ocean + architecture-first)
- Ship-It Pragmatist (small scope + fast)
- Deep Craft (detail-verbose + principled)
- Taste Maker (intuitive, overrides recommendations)
- Solo Operator (high-autonomy, delegates)
- Consultant (hands-on, consulted on everything)
- Wedge Hunter (narrow scope aggressively)
- Builder-Coach (balanced steering)
- Polymath (fallback when no archetype matches)

matchArchetype() uses L2 distance scaled by tightness, with a 0.55 threshold
below which we return Polymath. v1 ships the model stable; v2 narrative/vibe
commands wire it into user-facing output.

14 new tests: signal map consistency vs registry, applySignal behavior for
known/unknown keys, normalization bounds, archetype schema validity, name
uniqueness, matchArchetype correctness for each reference profile, Polymath
fallback for outliers.

41 pass, 0 fail total in test/plan-tune.test.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 06:14:55 +08:00
Garry Tan db3b6412b9 feat: one-way door classifier (belt-and-suspenders safety fallback)
scripts/one-way-doors.ts — secondary keyword-pattern classifier that catches
destructive questions even when the registry doesn't have an entry for them.

The registry's door_type field (from scripts/question-registry.ts) is the
PRIMARY safety gate. This classifier is the fallback for ad-hoc question_ids
that agents generate at runtime.

Classification priority:
  1. Registry lookup by question_id → use declared door_type
  2. Skill:category fallback (cso:approval, land-and-deploy:approval)
  3. Keyword pattern match against question_summary
  4. Default: treat as two-way (safer to log the miss than auto-decide unsafely)

Covers 21 destructive patterns across:
  - File system (rm -rf, delete, wipe, purge, truncate)
  - Database (drop table/database/schema, delete from)
  - Git/VCS (force-push, reset --hard, checkout --, branch -D)
  - Deploy/infra (kubectl delete, terraform destroy, rollback)
  - Credentials (revoke/reset/rotate API key|token|secret|password)
  - Architecture (breaking change, schema migration, data model change)

7 new tests in test/plan-tune.test.ts covering: registry-first lookup,
unknown-id fallthrough, keyword matching on destructive phrasings including
embedded filler words ("rotate the API key"), skill-category fallback,
benign questions defaulting to two-way, pattern-list non-empty.

27 pass, 0 fail. 1270 expect() calls.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 06:12:39 +08:00
Garry Tan 2c0c95099f test: registry schema + safety + coverage tests (gate tier)
20 tests validating the question registry:

Schema (7 tests):
- Every entry has required fields
- All ids are kebab-case and start with their skill name
- No duplicate ids
- Categories are from the allowed set
- door_type is one-way | two-way
- Options arrays are well-formed
- Descriptions are short and single-line

Helpers (5 tests):
- getQuestion returns entry for known id, undefined for unknown
- getOneWayDoorIds includes destructive questions, excludes two-way
- getAllRegisteredIds count matches QUESTIONS keys
- getRegistryStats totals are internally consistent

One-way door safety (2 tests):
- Every critical question (test failure, SQL safety, LLM trust boundary,
  security scan, merge confirm, rollback, fix apply, premise revise,
  arch finding, privacy gate, user challenge) is declared one-way
- At least 10 one-way doors exist (catches regression if declarations
  are accidentally dropped)

Registry breadth (3 tests):
- 11 high-volume skills each have >= 1 registered question
- Preamble one-time prompts are registered
- /plan-tune's own questions are registered

Signal map references (1 test):
- signal_key values are typed kebab-case strings

Template coverage (2 tests, informational):
- AskUserQuestion usage across templates is non-trivial (>20)
- Registry spans >= 10 skills

20 pass, 0 fail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 06:10:57 +08:00