mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-09 22:48:57 +02:00
* chore(design): pin impeccable rule ids and detector JSON shape as fixtures Real captures from a human-initiated `npx impeccable install` in a scratch directory (engine 0.1.3, linux-x64), never a runtime download: - test/fixtures/impeccable-antipatterns.json: upstream crates/live/assets/antipatterns.json at 87d8f6d6 (the state engine-v0.1.3 shipped), 61 rules, source commit recorded in `_source`. - test/fixtures/impeccable-detect-sample.json: `detect --json` over gstack's planted-slop fixture (source mode), paths normalized. - test/fixtures/review-eval-design-slop.dom.html + impeccable-detect-dom-sample.json: the same page served locally, dumped through the browse engine with the shared DOM-dump script, then scanned. Pins the load-bearing assumption that the static engine reads inline <style> in a .html file: the DOM scan yields the same id set as the source scan. - lib/dom-dump-script.ts: the one dump script both browser engines evaluate (IIFE, no single quotes). Folds CSSOM rgb() back to author hex so palette rules still fire, and removes inlined <link> nodes so the engine does not warn about an unresolvable stylesheet. Both verified against the engine. - test/fixtures/impeccable-detect-help.txt + impeccable-captures.meta.json: the flags, exit codes, finding fields, and re-capture protocol. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(design): typed slop catalog in lib/; AI_SLOP_BLACKLIST derived lib/design-catalog.ts is the single source of truth for gstack's design anti-pattern vocabulary: the 11 legacy blacklist lines (verbatim, flagged `legacyBlacklist`), every one of impeccable's 61 registry ids with gstack prose, tier, impact, confidence, grep heuristic, and /impeccable handoff, plus the gstack-only tells the LLM pass judges (hero metrics, identical cards, glassmorphism, missing states, unthemed browser surfaces, ...). `impeccableId` is set only when the id exists in the registry fixture, and `renderCatalog({style:'ids'})` brackets an id only then, so rendered prose never shows an id the detector cannot emit. Role-scoped font lists (OVERUSED_FONTS_DISPLAY, BANNED_FONTS, FONTS_BODY_UI_OK, FONTS_MONO_OK, FONTS_VERIFIED_FREE) live beside the entries. scripts/resolvers/constants.ts now derives AI_SLOP_BLACKLIST from the catalog. Generated output is byte-identical (bun run gen:skill-docs is a zero diff). Pure module: no I/O, no scripts/ imports, loading prints nothing, so bin/ can import it at runtime on every host. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(review): generate review/design-checklist.md from the catalog review/design-checklist.md was hand-written and its own header admitted it drifted from DESIGN_METHODOLOGY category 9. It is now rendered by scripts/resolvers/design-checklist.ts from lib/design-catalog.ts: category 1 lists every grep-detectable slop entry plus the legacy blacklist lines, sorted HIGH/MEDIUM/LOW, each with its heuristic and, where the detector knows the rule, its bracketed id (27 items, up from 6). The font blacklist renders from BANNED_FONTS. Categories 2-5, Instructions, Classification, Output Format, and Suppressions keep their prose. Title and slop heading are unchanged (test/skill-e2e-review.test.ts and hosts/opencode.ts key on them). gen-skill-docs writes the file for the Claude host only (a Claude-side runtime asset; other hosts copy or inline the render), honors --out-dir, and reports STALE/FRESH under --dry-run like sections do. test/design-checklist-sync.test.ts pins committed == generated, the host/out-dir scoping, and the dry-run freshness line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): modes, craft-floor reflexes, calibration, catalog in doctrine DESIGN_HARD_RULES: the classifier names four visitor modes (Persuade, Operate, Read, Experience, plus Hybrid per section) and keeps the MARKETING/LANDING PAGE and APP UI aliases; Read and Experience get three rules each; a "Reflexes no detector catches" block (browser surfaces, one authored motion moment, depth has an offset, tinted secondary text, space above headings, light-or-dark from the use scene) and the three-looks calibration follow the universal rules. The slop section renders the 11 legacy lines plus the detector rule ids and judgment tells from the catalog; in design-review, which also renders DESIGN_METHODOLOGY, it becomes a one-line pointer so the catalog is paid for once. Header counts are computed, not hardcoded. DESIGN_METHODOLOGY: category 9 renders the catalog in three registers (legacy lines verbatim, detector rules that need judgment with bracketed ids, gstack-only judgment tells as prose, polish-level ids on one line); categories 5 and 7 carry the browser-surface and one-motion-moment reflexes; the typography overused-face item points at [overused-font] with the role-scoped exception. The consultation Codex prompt's anti-slop line reads from the catalog. Budget: design-review eager 25.6K -> 27.0K (ceiling 27,984), plan-design-review unchanged at 17.4K; no carve-guard or context-budget re-baseline needed; ship goldens unchanged (ship never renders the hard rules). Derived from pbakaus/impeccable reference/craft-floor.md + new-work.md (Apache-2.0), rewritten. See NOTICE.md (commit 12). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design-consultation): font procedure, role-scoped overused list, color strategies The proposal section stops handing out a font menu. "Choosing faces: a procedure, not a menu" names the subject's world, shortlists per role, strikes the overused list for that role, verifies availability in-session, and states the loading strategy. {{OVERUSED_FONTS}} renders the role-scoped lists from lib/design-catalog.ts: overused as display (the detector's overused-font set plus the training-data defaults), fine as body/UI on an Operate or Read surface, mono for data and code, banned in any role, and a short verified-free list with its verification date. Color approaches become Restrained / Committed / Full palette / Drenched. The anti-convergence directive drops light-vs-dark as a dial (it comes from the use scene) and the three-looks calibration sits under Your Design Knowledge. The slop list is {{DESIGN_SLOP_BULLETS}}: prose from the catalog, no rule ids, polish-level tells omitted. design-html's "Never include (AI slop blacklist)" list keeps its literal (carve guard) and each line now carries a trailing <!-- id --> naming a catalog entry, pinned by test/design-catalog.test.ts so the last surviving duplicate is derived-by-test. Both resolvers are registered and listed in ARCHITECTURE.md. No carve-guard or budget re-baseline needed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(bin): gstack-design-detect wrapper + design_detector config key bin/gstack-design-detect.ts finds and runs an impeccable engine the user installed; it never installs, downloads, or executes anything that could download. `probe` reads only: config (design_detector off → DISABLED), IMPECCABLE_BIN (absolute, realpath outside the repo and cwd), a PATH walk (absolute entries outside the repo; a #! shim counts as launcher-present, never READY), the ~/.impeccable/bin/<newest semver>/ cache, and the engine installed beside a skill launcher (scripts/bin/<os>-<arch>/impeccable, the layout a real install produced). It reports IMPECCABLE_SKILL, host-aware IMPECCABLE_HOOK (+ HOOK_OTHER), the ignore lists from .impeccable/config*.json, IMPECCABLE_ENGINE_UNTESTED for versions outside the fixture set, and a hint only when a launcher exists without its engine. `scan` re-probes, refuses URLs and anything outside the repo root or the design-report allow-list (realpath, so symlinks cannot escape), derives `--changed <base>` targets NUL-safely through git and lib/frontend-scope.ts, batches 100 absolute paths per engine call with stdin ignored, a SIGKILL timeout, a 50 MB stdout cap, and sanitized length-capped fields, then prints one normalized JSON document (--format gstack) or the engine's bytes (--format raw); DETECT_TOP (fenced as untrusted content), DETECT_SUMMARY, and DETECT_EXIT go to stderr; exit code passes through with 1 over 2 over 0; exit 3 is a gstack bug. `rules` prints the mapped set. Every run appends a content-free line to the local analytics file. lib/design-detect-contract.ts owns every sentinel string, the limits, and the normalized-finding shape (pure module); test/design-detect-contract.test.ts asserts every sentinel-shaped token the agent can read exists there. lib/frontend-scope.ts mirrors gstack-diff-scope's frontend arm, pinned by a parity test that runs the bash script. bin/gstack-config gains design_detector (auto | off, default auto, invalid values rejected with the file unchanged). test/fixtures/fake-impeccable.ts is the env-driven engine stand-in; test/gstack-design-detect.test.ts covers READY/NOT_CACHED/ NOT_AVAILABLE/DISABLED, env trust (.env never loaded, in-repo IMPECCABLE_BIN ignored), newest-semver cache, hook and ignore detection, refusals, exit passthrough, raw byte-identity, normalization, the display cap, timeout, parse errors, diagnostics, --changed, and analytics. The egress scanner test records the wrapper as a documented non-sink. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): {{DESIGN_DETECTOR}} wired into design-review, ship review-lite, review army, design-html The user-installed impeccable engine becomes a deterministic pre-pass in four skills, through one resolver with three renders: {{DESIGN_DETECTOR}} (the probe block and how to read every sentinel), {{DESIGN_DETECTOR:phase0}} (design- review's mechanical scan), {{DESIGN_DETECTOR:gate}} (design-html's bounded slop gate). Every rendered invocation is `bun --no-env-file run <bin>/gstack-design- detect.ts ... --host <host>` and every scan ends with the DETECT_EXIT_CODE echo so exit 2 (findings) never aborts a block. design-review: probe in Setup; Phase 0 picks DOM mode (URL target) or source mode (diff-aware, no URL) once; source mode scans the changed frontend files in Setup, DOM mode never reads source (Rule 4). Phase 3 gains a DOM-dump step per page: both browser engines load the shared script from lib/dom-dump.js (Aside splices it into a double-quoted repl script; the fallback engine copies it into a temp dir for `$B eval --out --raw`), the dump is size-capped, run through gstack-redact (a HIGH finding skips the page), and persisted under $REPORT_DIR/dom/$RUN_ID/; one scan runs after the last page, labeled "static scan of the rendered DOM; cross-origin CSS not resolved". REPORT_DIR honors GSTACK_HOME so the wrapper's allow-list and the report dir agree; RUN_ID is set once in Setup. design-baseline.json is schemaVersion 2 with runId, targetSet, base, and a detector block (mode, engine, byRule, byPage), written temp+rename with a per-run copy; Regression Output diffs ids only when mode and target set match, caveats an engine change, and calls live-page count deltas advisory. Phase 7 hands deferred detector findings to the `handoff=` command the scan printed; Phase 9 recomputes the same way and deletes the dumps unless --keep-dom; Phase 10 reports `Detector: N → M`. ship review-lite gains step 0 (probe, `scan --changed <base>`, tier buckets, detector + checklist dedupe, advisory and ignored never count) and a `detector` count in its log payload; the PR body gets a Detector line (rule ids and counts only). The Review Army Design specialist runs the mechanical pass at the top of review/design-checklist.md, which now carries it. design- html probes after DESIGN_SETUP and runs the one-pass gate before screenshots. lib/dom-dump.js is generated by gen-skill-docs from lib/dom-dump-script.ts (Claude host, --out-dir aware, dry-run freshness) and pinned byte-equal, so the prose never carries the script. The contract gains DETECT_JSON, DOM_DUMP_OK, and the self-describing set; its test now checks both directions. Budget: design-review eager 25.6K → 28.5K. The plan's target was +2.5K; after the levers it named (ids-only detector rules, no inline script, trimmed prose) it lands at +2.87K, and the remainder is doctrine and detector wiring, so the ceiling moves to the captured 31,319 for design-review only (the full capture would also have loosened 21 ceilings this branch never touched; those stay). design-html skeleton re-baselined to 54,000 (measured 53,592). Codex and Factory ship goldens refreshed (review-lite step 0 and the PR-body line render inline there). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): catalog never-lines in the mockup prompt Ten catalog ids carry `mockupNever` (kicker-above-heading, icon-tile-stack, gradient-text, ai-color-palette, cream-palette, nested-cards, dark-glow, pulsing-dot, identical-cards, hero-metrics) and lib/design-catalog.ts exports their deduped plain-English names as MOCKUP_NEVER_NAMES. briefToPrompt() in the design binary appends "Never: <names>." before its fixed tail, so `$D generate | variants | evolve` stop reaching for purple gradients, icon tiles, and cream defaults before the comparison board opens. The binary still bundles (`bun build --compile design/src/cli.ts`); ./setup rebuilds it. design-html's Never-include list now covers every mockupNever id (kicker / icon tile, hero metric rows, gradient text, cream palette, nested and identical cards, glow and pulsing dots), each line tagged with its catalog ids; test/design-catalog.test.ts pins the exact ten flags, the deduped names, and that the template list is a superset. New design/test/brief.test.ts pins the prompt shape. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(lib): open DESIGN.md reader/writer + gstack-design-md bin lib/design-md.ts implements the open DESIGN.md format (google-labs-code/ design.md, Apache-2.0): YAML front matter with the five token groups (colors, typography, rounded, spacing, components) and eight canonical `##` sections in spec order (Overview, Colors, Typography, Layout, Elevation & Depth, Shapes, Components, Do's and Don'ts), aliases mapped, extras preserved after them in their original order. parseDesignMd never throws (unparsable front matter → `unknown` with a reason); renderDesignMd re-emits the preserved front matter bytes and only `convert` writes fresh YAML through a small block-style emitter (Bun.YAML.stringify is flow style); upsertSection splices the body only; tokensFlat resolves `{path}` references to primitives and reports group, self, dangling, and cyclic refs as DESIGN_MD_TOKEN_REF_INVALID. convertLegacy turns gstack's pre-spec DESIGN.md into the open format: Product Context and Aesthetic Direction fold into Overview, Typography roles become display/body/label/mono tokens (mono carries fontFeature: tnum), Color hexes become colors (mode-qualified labels keep their qualifier; strategy lines are not colors), the Spacing scale and Layout radii become spacing and rounded, Motion / Grain Texture / Decisions Log survive as extras. The format marker lives inside the file: a YAML comment on line 2 of a spec file, an HTML comment on line 1 of a legacy file. bin/gstack-design-md.ts: `check` (DESIGN_MD_FORMAT + marker), `convert [--write]` (backup to DESIGN.md.legacy.bak, temp+rename, refuses ambiguous input with DESIGN_MD_CONVERT_REFUSED), `tokens` (flat JSON), `mark <spec|legacy-keep>`. Exit 3 + DESIGN_MD_INTERNAL_ERROR is a gstack bug. design/src/memory.ts: updateDesignMd upserts "Extracted Design Language" through the lib (front matter bytes untouched, canonical order kept, section replaced on rerun) and creates a spec skeleton with tokens from the extraction when no file exists; readDesignConstraints leads with the flat tokens and the Overview for spec files. The design binary still bundles. test/design-md.test.ts pins all of it against gstack's own DESIGN.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): write/read DESIGN.md in the open spec; persisted format choice gstack's design skills now write DESIGN.md in the open DESIGN.md format and read tokens from it. {{DESIGN_MD_CHECK}} renders the format check through bin/gstack-design-md.ts: design-consultation's Phase 0 settles the format once (spec → update tokens in the front matter; legacy without a marker → one AskUserQuestion: convert with a .legacy.bak, keep the legacy file, or start fresh; the answer is written into the file as the format marker so no skill asks again; a marker already present is obeyed silently; unknown → prose; missing → Phase 6 writes one). Phase 6's template is the spec form: YAML front matter with name, description, and exactly the five token groups (colors, typography.display/body/label/mono with fontFeature: tnum on mono, rounded, spacing, components with {path} references), then Overview (Creative North Star, product context, mode per surface, references, key characteristics), Colors (opening with the Restrained / Committed / Full palette / Drenched strategy), Typography, Layout, Elevation & Depth, Shapes, Components, Do's and Don'ts, plus gstack's Motion and Decisions Log as extras; the template ends with a check that the file parses as `spec`. design-review runs the `:calibrate` form in Setup: a spec file's flat tokens are the calibration source (a value present in the tokens is never a finding), the marker is respected, and conversion is never offered there; its DESIGN.md export writes the spec form. design-html's token extraction writes the spec form and respects an existing choice. review/design-checklist.md category 5 and ship's review-lite step 1 name `gstack-design-md tokens` as the calibration source; plan-design-review Pass 5 cites tokens by path when front matter exists. The contract owns the bin's DESIGN_MD_MARKER / REASON / WRITTEN / BACKUP lines; the contract test's pending list closes. Carve guard: design- consultation skeleton 66,500 → 67,500 (measured 67,014; +1,508 B against the 1.5 KB cap). Codex and Factory ship goldens refreshed (review-lite step 1). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): PRODUCT.md prefill + /impeccable handoffs design-consultation's context gathering and design-shotgun's auto-gather read PRODUCT.md (impeccable's product-context file) when it exists: it counts as the user's prior answers, gets confirmed in one line, and is never re-asked. Neither skill opens `.claude/skills/impeccable/**`; PRODUCT.md and DESIGN.md are the shared surface, and impeccable's prose never loads inside a gstack skill. Handoffs: ship's review-lite ends each NEEDS INPUT detector row with the `handoff=` command the scan printed (`/impeccable <cmd>`) when the probe reported IMPECCABLE_SKILL: present, recommending the command and never opening its files; design-review's Phase 7 does the same for deferred findings, and `design_detector: off` silences handoff lines with the rest. Codex and Factory ship goldens refreshed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(design): convert gstack's own DESIGN.md to the open spec `gstack-design-md convert --write` on the repo's DESIGN.md: tokens in YAML front matter (typography.display/body/label/mono, colors with their light/dark qualifiers, spacing scale, rounded scale), Overview from Product Context and Aesthetic Direction, Colors / Typography / Layout as canonical sections, Motion, Grain Texture, and Decisions Log preserved as extras, format marker on line 2. Hand-checked; `check` reports spec with no token-reference errors. A Decisions Log row records the conversion and that DM Sans stays the body face: it is on the overused-as-display list, and body/UI use on an Operate surface is the allowed exception under the role-scoped rule. The pre-conversion file lives on as test/fixtures/design-md-legacy.md, which test/design-md.test.ts now uses for its legacy cases; the converted root file is asserted to be spec. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: NOTICE, Apache license text, README interop, project structure NOTICE.md names what gstack derived from impeccable (rule ids and names in the catalog and the registry fixture; the visitor modes, craft-floor reflexes, and calibration in the design resolvers; the font procedure in the consultation template) and from Google's DESIGN.md specification (the format lib/design-md.ts implements), states that gstack does not distribute or audit the impeccable engine, and points at licenses/Apache-2.0.txt (verbatim). README: the design-consultation, design-review, and design-html rows say what changes when impeccable or the open DESIGN.md format is in play, and a "Works with impeccable" paragraph explains the pre-pass, the shared ids, PRODUCT.md and DESIGN.md as the shared surface, the handoffs, the no-nag posture without impeccable, and the off switch. docs/skills.md gets the detector paragraph under /design-review. docs/PROJECT_STRUCTURE.md lists the new lib and bin files, NOTICE.md, and licenses/. docs/designs/IMPECCABLE_INTEROP.md promotes the CEO plan (its ~/.gstack copy is flipped to PROMOTED) with a "what shipped" summary. TODOS.md files the seven deferrals from the reviews: the design-review Phases 7-11 carve (the budget lever, with the +2.87K vs 2.5K landing recorded), the Bun .env audit across bin/*.ts, the Kiro bin/lib gap, the $D check slop rubric, taste-profile interplay, the CEO Section 11 bullets, and the scan cache. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: touchfiles, tiers, shim E2E, real-engine fixture Touchfiles: the catalog, contract, detector bin, checklist resolver, review- army resolver, and DESIGN.md lib join the dep lists of review-design-lite, design-review-fix, the design-consultation cases, and plan-design-review-no- ui-scope, so editing any of them re-selects the tests that read their output. Three new E2E keys: design-review-detector-shim (gate; source mode on a feature-branch diff), design-review-detector-shim-dom (gate; DOM mode: the slop fixture served on loopback, dumped through the browse binary with lib/dom-dump.js, persisted under a GSTACK_HOME-scoped REPORT_DIR, scanned once; self-skips when browse/dist/browse is absent), and design-html-slop-gate (periodic; one fix pass, at most two scans, remaining findings accepted with reason). Every case reaches the engine through test/fixtures/fake-impeccable.ts via IMPECCABLE_BIN from outside the temp repo, reads extracted skill sections (never a whole SKILL.md) with the installed bin path rewritten to this checkout, and asserts the probe ran, the right scan verb ran, `npx impeccable` never did, and the output carries FINDING rows tagged [ai-color-palette] and [low-contrast]. review-design-lite gets the fake engine and an eighth tally signal for a detector row; its 4-hit threshold is unchanged. test/gstack-design-detect.test.ts evaluates design-review's REPORT_DIR expression with GSTACK_HOME set and proves a dump under it is accepted by the wrapper's allow-list. The sample fixtures were real captures from commit 1 (engine 0.1.3), so there is nothing hand-written left to swap. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-detect): never execute a repository-controlled engine; allow-list --changed targets; sanitize engine text Pre-landing review findings (security + checklist), all reproduced before the fix: - A checked-out branch could commit `.claude/skills/impeccable/scripts/bin/<os>-<arch>/impeccable` and the probe would report READY and `scan` would run it, with the agent's full environment. Launchers and sibling engines under the repo or cwd now count as "skill present" only (IMPECCABLE_NOT_CACHED: repository-local install, and the hint never names a repository-local launcher to run); only HOME-rooted installs, IMPECCABLE_BIN, the cache, and PATH entries outside the repo qualify, all by realpath. The engine now sees a minimal environment (PATH, HOME, TMPDIR, locale, IMPECCABLE_*), never the agent's tokens. - `scan --changed <base>` pushed git-derived paths without the allow-list, so a committed symlink with a frontend extension handed a file outside the repo to the engine. Derived targets now go through the same allow-list as explicit ones and symlinks named by git are refused outright. - A repo-controlled `scripts/VERSION` with embedded newlines forged probe lines; the version is trusted only when it is semver, and every printed version is sanitized. Engine text containing the untrusted-content fence or a `SENTINEL:` prefix is neutralized with a zero-width space (neutralizeSentinels in the contract), so page text cannot close the envelope or forge a probe line. - A failing `git diff <base>...HEAD` (unknown or unfetched base) was swallowed and read as "no frontend changes"; it is now DETECT_REFUSED with exit 1. - The scan allow-list root follows `${GSTACK_HOME:-$HOME/.gstack}` like the templates and gstack-slug (config.yaml keeps gstack-config's STATE_ROOT precedence); a quoted or commented design_detector value reads correctly. Smaller: raw engine chunks are kept only in --format raw; diagnostics are capped (200 kept, 20 echoed); the engine identity hash reads size + 4 MB, not the whole binary; PROBE_STEP and ENGINE_STDERR are contract sentinels; the --verbose gate covers every probe step; analytics use one sentinel vocabulary; bare limits live in DETECT_LIMITS. The fake engine's knobs are IMPECCABLE_FAKE_* (so they pass the minimal env) and a shared test helper installs it. New tests cover each item above plus clean runs, `{}` parse errors, missing paths, and the 50 MB stdout cap. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-md): mark and updateDesignMd never rewrite the user's file; refuse a contradictory mark renderDesignMd re-sorted canonical section names into spec order on every render, so `gstack-design-md mark legacy-keep` (the "leave it alone" answer) and the design binary's mockup extraction reordered a legacy DESIGN.md (Typography and Layout jumped to the top) and normalized its whitespace, while the bin promised "body bytes untouched". `mark` now splices only the marker line (insertMarker) and `updateDesignMd` splices only its own section (spliceSection); every other byte of an existing file is preserved, and spec order applies only to files that open with front matter. `mark` refuses a choice that contradicts the file's format (spec on a non-spec file, legacy-keep on a spec file) with DESIGN_MD_CONVERT_REFUSED, exit 2, file unchanged. convertLegacy keeps intro prose under the title instead of rebuilding the preamble from the title alone. detectFormat returns a machine-readable `code` beside the prose reason (the bin no longer branches on reason text); the marker regexes derive from FORMAT_MARKER_PREFIX and FORMAT_CHOICES; the hop limit and legacy identity headings are named constants; slug is exported and reused; both writers use lib/fs-atomic.ts. Tests pin byte identity for mark and updateDesignMd on the legacy fixture, the refusal paths, and the preserved preamble. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design): run the DOM dump in the page on both engines; align doctrine with the catalog The DOM-dump script is an arrow function, not a self-calling IIFE: Aside's `pg.evaluate($_DUMP)` receives the function and runs it in the page (the IIFE form executed in the repl sandbox, where `document` does not exist), and the fallback engine calls it with `$B js "($_DUMP)()" --out --raw`. Hygiene widens to every URL-bearing attribute (src, srcset per candidate, poster, action, formaction, data, ping, cite lose their query strings and fragments) and to data: URLs inside existing <style> nodes. The persist and scan blocks restate REPORT_DIR and RUN_ID literally instead of relying on a shell variable from an earlier block; the baseline's targetSet is defined per mode (repo-relative paths in source mode, page slugs in DOM mode) so DOM-mode deltas can match; the PR-body Detector line lists the states the probe can actually print. The DOM fixture is re-captured with the new script from outside the repo (the engine walks up from cwd for DESIGN.md, which the metadata now records). Doctrine contradictions the design specialist found: the landing-page motion rule matches the one-authored-moment reflex; the background rule names the catalog's halo/spotlight/stripe/grid slop instead of asking for gradients; the universal font rule is scoped to the display voice with the body/UI exceptions; "two typefaces max" allows the mono; the methodology's banned-font line renders BANNED_FONTS; Courier New is banned outright; the Brutalist, Retro-Futuristic, and Playful menu entries stop recommending system stacks, glow, and bounce; the coherence nudge uses the decoration vocabulary; Path A's gate names the display voice; font-loading prose points at the source the procedure verified; centered-everything is MEDIUM (an aggregate heuristic); the mockup guard reads "Never by default (unless the brief above asks for it)". The checklist's AUTO-FIX list renders the catalog's auto-fix rules; category 9 and the Hard Rules pointer count from the same partition helpers (detectorSlopEntries, judgmentTellEntries); the handoff list renders from HANDOFF_COMMANDS; a missing catalog id fails gen-skill-docs by name. gstack's own DESIGN.md gains border tokens and Decisions Log rows for its live-feed pulse and 11px mono labels. frontend-scope is case-sensitive like the bash arm. gen-skill-docs shares one emitGenerated helper for sections and lib-derived assets; renderCatalog keeps the one style with a caller. Tests: shared sliceBetween that fails on a missing end marker, the slop-gate fixture's real end marker, an isolated browse daemon for the DOM-mode E2E, the DOM hygiene test gated to CI or opt-in, docs notes for the two superseded plan sentences. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-detect): an engine is a file named impeccable outside the project; DOM dumps scan without inline ignores Second review cycle, security + checklist: - IMPECCABLE_BIN=/bin/sh (or node) was READY, and `detect` with cwd=repoRoot made the interpreter run the repository's own `detect` file. Every engine candidate (env override, PATH entry, cache, sibling) is now judged by the realpath of the FILE and must be named impeccable[.exe]; PATH and cache candidates that resolve into the repository are skipped like the others. "Inside the project" means the repository, or cwd when cwd is a project directory: HOME and its ancestors are exempt, so a URL-mode review launched from HOME still finds the HOME-rooted installs. - A base for --changed that starts with `-` was spliced into git argv (`--output=<file>` made git write a file and report no changes); an option- like or missing base is DETECT_REFUSED (not a ref name), exit 1, and the parser no longer defaults a missing value to main. - DOM dumps are the audited page's bytes, so an in-file `impeccable-disable` comment there is page-controlled: batches under the designs root run with --no-inline-ignores, repository batches keep the project's own ignores. - neutralizeSentinels covers the shapes it missed (bare sentinels such as DETECT_TOP total= and IMPECCABLE_DISABLED, the DETECT_EXIT_CODE= echo, the `[rule-id] impact=` group header) in one precompiled alternation instead of 37 replaceAll passes per field; only kept findings are normalized, and the summary's total stays the engine's count. - The minimal engine environment compares keys case-insensitively on Windows (process.env enumerates Path, SystemRoot there) and passes PATHEXT, COMSPEC, HOMEDRIVE, HOMEPATH, PROGRAMDATA. - Bare 64s move into DETECT_LIMITS; the unused SentinelName type is gone; the header states the directory-target contract (the engine's own walk). Tests: an interpreter as IMPECCABLE_BIN never runs the repo's detect file; a PATH symlink into the repository is never READY; option-like and empty bases are refused with no file written; the designs-root batch carries --no-inline-ignores and the repo batch does not; the identity label is deterministic per binary; the bare-sentinel and header shapes are neutralized; the installed fake engine works without IMPECCABLE_FAKE_OUTPUT (the helper copies the sample beside it); two tests clean up in finally. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-md): text-level edits keep CRLF, one section-boundary rule, control characters quoted - insertMarker and spliceSection normalized every line ending to LF, so a CRLF DESIGN.md came back rewritten beyond the one line they promised to touch. Both detect the file's dominant line ending and restore it. - parseDesignMd and spliceSection each walked headings with their own fence tracking; they now share headingLines (and upsertSection shares headingMatches). An unclosed ``` is treated as prose for that file: it used to swallow every later section on a splice. - A token value carrying a control character (an LLM-extracted font family with an embedded newline) was emitted as a bare multi-line scalar that Bun.YAML rejects, turning a freshly written DESIGN.md into frontmatter-unparsable; needsQuotes routes it through the quoted form. - The marker-line regex variants are built once beside YAML_MARKER_RE; the dead setMarker export and a no-op ternary are gone; LEGACY_HEADINGS derives from the identity list; the header diagram names the text-level editors as the write path for user-owned files; the bin validates and prints the mark choices from FORMAT_CHOICES. Tests: CRLF round-trips for both editors, a fenced ## inside a section and an unclosed fence, and a newline-bearing scalar parsing back. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design): Aside dump script stays single-quoted; redaction gate sized to the dump cap; doctrine made consistent - The DOM-dump Aside block was the only double-quoted `aside repl` script in the tree (to splice the function text), which put the agent-filled <url> inside a double-quoted bash string: a same-origin href carrying $(...) would run in the reviewer's shell when Phase 3 opened that page. The script is single-quoted like every other Aside script and the function text enters through a closed-quote segment ('"$_DUMP"'); the fallback line is `$B js '('"$_DUMP"')()'`. A free test pins that no rendered Aside script opens with a double quote. - The persist block capped dumps at 10 MiB but ran gstack-redact with its 1 MiB default, so every real page between the two was deleted as DOM_DUMP_REDACTION_BLOCKED; the gate passes --max-bytes at the dump cap and blocks on any exit other than clean (0) or MEDIUM (2), so a redaction tool that fails to run can no longer fall through to "persist". - Dump hygiene removes <template> and <noscript> subtrees (invisible to the attribute walk), inline on* handlers, and the cross-origin <link> nodes already named in the note, so the file handed to the engine references no remote stylesheet. - Doctrine: the Codex design-voice prompts said "2-3 intentional motions" against the one-authored-moment rule; the overused-display heading scoped its ban to Persuade/Experience while the catalog and hard rules ban it everywhere; design-consultation's Important Rule 4 still said "as primary"; design-html's blacklist header is now "Never include by default" with the mockup/DESIGN.md/user-ask override the catalog grants; the slop gate honors Decisions Log and Do's and Don'ts blessings like /review does; the landing "poster" line says poster in stance, not type size; the design binary's variant dials no longer flip light/dark for variety; gstack's DESIGN.md rows name data labels (UI labels stay the DM Sans token) and call the skill-bar fill and hovers functional transitions. - design-review names how the base branch is found (gh pr view, then the repo default; never main) for the source-mode scan and the diff-aware mode. - frontend-scope matches the config globs at the repo root only, like the bash arm; the parity test carries nested samples. - Cleanups: renderCatalog's stale style option, an unused import, the identity-map bannedFontNames, the checklist header's "same entries" claim, the catalog header's consumer list, the orphaned main() docstring, the plan doc's IIFE bullet. design-html's skeleton ceiling is re-measured (54,184) for the two doctrine sentences. Tests: AUTO-FIX rendering from the catalog, the E2E slice markers checked in the free suite, the hygiene cases for templates/noscript/handlers/remote links, and the review E2E counting detector rows separately from the seven checklist plants. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-detect): project means below HOME; only page dumps drop inline ignores; a whole-scan budget; prototype-safe rule counts Third review cycle + Red Team, all reproduced before the fix: - With no repository, the wrapper adopted cwd as the repo root, so a review launched from HOME (URL mode can run from anywhere) rejected every HOME-rooted install as "repository-local", reported the user's own skill install with the wrong hint, and, for targets, accepted all of HOME (~/.ssh/id_rsa scanned). A project directory is now one strictly below HOME: `git init ~` never turns the user's installs into repository files, and from HOME only the designs allow-list qualifies as a target. - --no-inline-ignores keyed on "not inside the repo", which misclassified dumps when GSTACK_HOME sits under the repo and stripped the design-html gate's own `<!-- impeccable-disable -->` from finalized.html. Targets are classified as project / dom-dump (designs/<audit>/dom/**, the page's bytes) / artifact (other designs/ files, gstack-authored); only dumps drop inline ignores. - A repository's .impeccable/config.json can hide rules from the review; detector.ignoreValues was never surfaced. The probe prints IMPECCABLE_IGNORED_VALUES beside the rules, and the prose stops calling repo-config ignores "a decision the user made". - An engine id named `constructor` corrupted byRule through Object.prototype and `__proto__` counts vanished; byRule is a null- prototype object and an id that fails the shape check is `unmapped` as a key too. - Batches ran with no total budget (10,000 un-ignored files: hours). The scan stops at 5x the per-batch timeout with DETECT_TIMEOUT and exit 1. - The scan JSON carries an `untrusted` list of the engine- and page-derived fields, so the agent reading past the fenced DETECT_TOP block is told what is evidence. - The PATH walk keeps launcher-present for a .cmd wrapper or a differently named real file (the name gate applies to READY only). Tests: probe and scan from a fake HOME (cache READY, HOME file refused, dump scanned without inline ignores), artifact vs dump batches, prototype-member ids, the whole-scan budget over 11 batches, ignoreValues surfaced, the `untrusted` field. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-md): edits follow a symlinked DESIGN.md, keep the BOM and the majority line ending, refuse an unclosed fence - `mark`, `convert --write`, and the design binary's extraction replaced a symlinked DESIGN.md (a docs-site layout) with a regular file and left the real target untouched; both writers resolve the link first. - A single stray CRLF flipped a whole LF file to CRLF: the editors now keep the majority ending. A UTF-8 BOM broke format detection and ended up mid-file after `mark`; it is recognized and kept at byte 0. - Re-running `mark` on a marked file deleted the blank line after the marker (`\s*$` matched across the newline); the marker regexes use `[ \t]*`. - Fences: readers follow markdown (an unclosed fence runs to EOF); the text-level editors refuse such a file with DesignMdEditRefused (DESIGN_MD_EDIT_REFUSED) instead of splicing the wrong section, and the design binary reports that and leaves the file alone. - needsQuotes also quotes a scalar containing ` #` (an inline-comment shape parsed back as a truncated value). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design): dump hygiene covers CSS URLs, srcdoc, and handlers; dumps persist owner-only; ignore prose treats repo config as evidence - The dump script cuts query strings from CSS url() in style attributes, <style> nodes, and the inlined stylesheets (signed asset URLs), empties srcdoc, and covers background and xlink:href. - Persisted dumps are chmod 600; MEDIUM redaction findings persist (an authenticated page shows emails) and the prose says so; earlier runs' dumps are swept before the first dump of a run unless --keep-dom. - The Aside dump prose asks for `'` in a pasted URL to be percent-encoded (a bare single quote would end the script) and never to paste an unread URL. - Repo-config ignores are evidence, not settled decisions, in /review, /ship, and design-review's probe prose; the scan JSON's text fields are named as untrusted. - design-html's skeleton ceiling is re-measured (54,545); ship goldens refreshed for the checklist prose. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-detect): audit directories scan as dumps; scans print probe lines on stderr; refused base always exits 1; PATH loses project entries Adversarial review (Claude subagent): - A DIRECTORY target under designs/ (the audit dir, which the prose hands the agent as REPORT_DIR) classified as an artifact, so the engine walked its dom/ subtree WITH inline ignores honored. Any directory under designs/ is now scanned as dumps. - A scan whose probe no longer finds an engine wrote its sentinel lines to stdout and exited 0, so `scan > "$_DJ"` captured "IMPECCABLE_NOT_AVAILABLE" as the scan result and the rendered bash read a clean scan. Probe lines go to stderr on every path; stdout is the JSON document or nothing. - A refused --changed base exited 0/2 when explicit targets were also given; it folds into the exit code (1 over 2 over 0). A trailing --changed no longer defaults to main. - A hand-edited `design_detector: Off` re-enabled the detector; the value is compared case-insensitively. - The engine inherited PATH entries inside the project (a direnv .envrc adding node_modules/.bin); those are filtered like every other project path. - DOM_DUMP_MISSING names the case where the dump script wrote nothing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-md): markdown edge cases: rule-opened legacy files, spaced fences, ~~~ blocks, duplicate headings, YAML 1.2 numerics - insertMarker keyed on "starts with ---", so a legacy file opening with a horizontal rule got a `# gstack:` line rendered as a heading that the parser then never read back (the conversion question re-asked every run). It keys on parsed front matter. - A closing front-matter fence with trailing spaces (`--- `) made a valid spec file `unknown`; the closer is any whole `---` line. - `~~~` fences hid nothing, so a `## ` inside one was a section boundary and a splice corrupted the fence; both fence kinds are tracked and only the same kind closes an opener. - convertLegacy silently kept the first of two `## Layout` bodies (and one of `## Color` / `## Colors`); it refuses with DESIGN_MD_CONVERT_REFUSED and the bin leaves the file and writes no backup. - needsQuotes covers 0x / 0o / .inf / .nan (YAML 1.2 numerics that changed type on round-trip); emitYamlBlock throws on an object inside an array instead of writing "[object Object]". - The design binary coerces the model's extraction JSON at the parse boundary (null names, missing arrays) so the paid call's result survives. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design): print and alternate stylesheets are not scanned as page CSS; no cross-run dump sweep; probe-state and design-system caveats in prose - The dump inlined every linked sheet's rules as active CSS, so a print sheet's 12pt black text or an alternate theme produced tiny-text and palette findings the user never sees; disabled and alternate sheets are skipped and a media-scoped sheet is wrapped in its @media block. - The cross-run dump sweep is gone: two same-day reviews shared REPORT_DIR and one run's sweep deleted the other's dumps mid-audit. Dumps stay per run, owner-only, deleted after Phase 9 unless --keep-dom (now defined in the prose), and an interrupted run's dumps wait for the user. - Prose: design-system-* rows in DOM mode compare the page to THIS repo's DESIGN.md and apply only to the repo's own app; an empty scan JSON with exit 0 means the probe state changed since Setup (read stderr); the persist block names a missing dump instead of mislabeling it as a redaction block. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * v1.82.0.0: impeccable interop, detector pre-pass, open DESIGN.md format Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: update project documentation for v1.82.0.0 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * changelog: name the measure behind the test-count row Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(test): drive the DOM hygiene test through Playwright Chromium directly Under the six-shard CI free suite the test's private browse daemon never answered its health probe (two minutes of retries), failed the shard, and starved two unrelated test files into failing before the runner's timeout. The test now launches the same Chromium through playwright-core and calls the dump function with page.evaluate, the way Aside's pg.evaluate does: no state file, no daemon, no health window. It self-skips when the Playwright Chromium bundle is absent. Two more hygiene rules are pinned along the way (print sheets keep their @media, alternate sheets are dropped). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(test): compare gen-skill-docs paths with forward slashes on Windows gen-skill-docs prints repo-relative paths with the OS separator, so the checklist render pins (`GENERATED: review/design-checklist.md`) failed on the Windows lane against `review\design-checklist.md`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(test): assemble the planted PEM block at runtime The quality gate scans every added line of the PR diff through gstack-redact; the redaction test's literal PEM header was a HIGH finding on our own test file. The block is now built from fragments, so the scanned file never carries a key-shaped line while the test still plants a HIGH finding. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design-detect): consent-gated engine install, checksum-pinned and receipted `gstack-design-detect.ts install` is the one download gstack makes, and only after a design skill's one-time question got a yes. It fetches the engine version gstack has tested (0.1.3) for this platform from impeccable's own GitHub release, verifies it against the checksum pinned in lib/design-detect-contract.ts (all five platforms, captured from the release's .sha256 sidecars; linux-x64 equals the fixture engine), writes an egress receipt before the fetch and refuses to download when the receipt cannot be written (fail-closed; the sink is registered in the wiring test's polarity table), caps the download at 32 MB, streams with the cap enforced, writes the file only after the hash matches, and places it under ~/.impeccable/bin/<version>/ (a trusted IMPECCABLE_HOME is honored; never inside a project). No skill, no hook, no launcher, no npx. --sha256 accepts a sidecar checksum for a version gstack has not pinned; --base allows a mirror (https, or http on loopback for tests). After a successful install the probe runs and its lines follow, so the skill sees READY at once. The probe ends with DESIGN_DETECTOR_INSTALL_OFFER (version, platform, bytes, destination) whenever it found no engine and the user has not answered the question; once design_detector_install_prompted is true it prints neither the offer nor the NOT_CACHED hint, which used to repeat on every run. The hint's npx wording is corrected: `npx impeccable detect --help` caches the engine for npx only, not where the probe looks. gstack-config gains design_detector_install_prompted (true|false, typo rejected, enumerated in list and defaults). Tests: a loopback mirror (async spawn, so the in-process server can answer) covers install, re-install as a verified no-op, checksum mismatch, 404, unpinned version, non-https base, design_detector off, and IMPECCABLE_HOME inside the repo; the offer and the silenced hint; pin completeness per platform. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): ask once before downloading impeccable's engine When the probe prints DESIGN_DETECTOR_INSTALL_OFFER the design skills ask the user one AskUserQuestion, in interactive sessions only (spawned or headless runs never install and never ask; Conductor gets the prose brief), before any other step: install the engine now, not now, never ask again (design_detector_install_prompted), or turn the detector off. A yes runs the receipted, checksum-pinned install and the skill continues with a READY probe. The brief says what impeccable is, what the one file is, where it goes, how it is verified and logged, and that no skill or hook comes with it; users who want the /impeccable skill run npx impeccable install themselves. design-review carries the brief inline (it is not carved). design-html keeps its skeleton small: the probe block points at a new read-on-demand section, sections/detector-install-offer.md, registered in its manifest and carve guard; its skeleton ceiling is re-measured (55,262) and its eager ceiling set to the measured 13,767. The review and ship passes state that they never offer an install. NOTICE.md, README, docs/skills.md, the interop design doc, and the CHANGELOG describe the new posture: gstack still never runs impeccable's installer or launcher; the one download is consented, pinned, and receipted. Ship goldens refreshed for the review-pass wording. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
893 lines
70 KiB
TypeScript
893 lines
70 KiB
TypeScript
/**
|
|
* Touchfile maps — the DATA half of diff-based test selection.
|
|
*
|
|
* LITERALS ONLY. This file must contain zero import statements and zero
|
|
* executable logic: no function calls, no spreads, no template literals —
|
|
* just string / array / Record literals. That property is load-bearing:
|
|
* map-diff selection evaluates OLD git versions of this file standalone to
|
|
* diff the maps across commits, which only works while the file stays pure,
|
|
* importable data. test/touchfiles-facade.test.ts enforces this with a
|
|
* comment-and-string-stripping tripwire.
|
|
*
|
|
* The selection logic (matchGlob, detectBaseBranch, getChangedFiles,
|
|
* selectTests) lives in ./test-selection.ts. Import sites should keep using
|
|
* the ./touchfiles facade, which re-exports both halves.
|
|
*/
|
|
|
|
// --- Touchfile maps ---
|
|
|
|
/**
|
|
* E2E test touchfiles — keyed by testName (the string passed to runSkillTest).
|
|
* Each test lists the file patterns that, if changed, require the test to run.
|
|
*/
|
|
export const E2E_TOUCHFILES: Record<string, string[]> = {
|
|
// Browse core (+ test-server dependency)
|
|
'browse-basic': ['browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-bws.test.ts'],
|
|
'browse-snapshot': ['browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-bws.test.ts'],
|
|
|
|
// Aside-driven browsing skills — live E2E against the Aside AI browser, the
|
|
// primary browser (test/skill-e2e-aside.test.ts self-skips without a running Aside)
|
|
'aside-browse-basic': ['browse/**', 'scripts/resolvers/browse.ts', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/basic.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
|
'aside-browse-flow': ['browse/**', 'scripts/resolvers/browse.ts', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/forms.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
|
'aside-qa-quick': ['qa/**', 'scripts/resolvers/browse.ts', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/basic.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
|
'aside-scrape-json': ['scrape/**', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/basic.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
|
'aside-canary-quick': ['canary/**', 'scripts/resolvers/aside.ts', 'browse/test/test-server.ts', 'browse/test/fixtures/basic.html', 'test/helpers/aside-available.ts', 'test/skill-e2e-aside.test.ts'],
|
|
|
|
// Hermetic isolation canaries (hermetic-env.ts is also a GLOBAL touchfile;
|
|
// these entries exist so the canaries themselves stay tier-classified)
|
|
'hermetic-canary': ['test/helpers/hermetic-env.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-hermetic-canary.test.ts', 'lib/conductor-env-shim.ts'],
|
|
'hermetic-sentinel': ['test/helpers/hermetic-env.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-hermetic-canary.test.ts', 'lib/conductor-env-shim.ts'],
|
|
|
|
// P4 first-run scaffold (activation lift) — the detection binary end-to-end
|
|
// through the real runner, plus the script wiring that gates + maps it
|
|
// (token-reduction Phase 2: generate-first-run-guidance.ts was deleted; the
|
|
// gate + token→tip map live in bin/gstack-skill-start's emission layer).
|
|
'first-task-scaffold': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'bin/gstack-first-task-detect', 'scripts/resolvers/preamble/generate-preamble-bash.ts', 'test/skill-e2e-first-task-scaffold.test.ts', 'test/helpers/session-runner.ts'],
|
|
|
|
// SKILL.md setup + preamble (depend on ROOT SKILL.md + gen-skill-docs)
|
|
'skillmd-setup-discovery': ['SKILL.md', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-bws.test.ts'],
|
|
'skillmd-no-local-binary': ['SKILL.md', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-bws.test.ts'],
|
|
'skillmd-outside-git': ['SKILL.md', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-bws.test.ts'],
|
|
|
|
'session-awareness': ['SKILL.md', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-bws.test.ts'],
|
|
'operational-learning': ['scripts/resolvers/preamble.ts', 'bin/gstack-learnings-log', 'test/skill-e2e-bws.test.ts'],
|
|
|
|
// QA (+ test-server dependency). /qa drives Aside first (the resolver) and
|
|
// the browse binary as fallback (browse/src), so both are deps.
|
|
'qa-quick': ['qa/**', 'scripts/resolvers/browse.ts', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-qa-workflow.test.ts'],
|
|
'qa-b6-static': ['qa/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval.html', 'test/fixtures/qa-eval-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
|
'qa-b7-spa': ['qa/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval-spa.html', 'test/fixtures/qa-eval-spa-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
|
'qa-b8-checkout': ['qa/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/helpers/llm-judge.ts', 'browse/test/fixtures/qa-eval-checkout.html', 'test/fixtures/qa-eval-checkout-ground-truth.json', 'test/skill-e2e-qa-bugs.test.ts'],
|
|
'qa-only-no-fix': ['qa-only/**', 'qa/templates/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-qa-workflow.test.ts'],
|
|
'qa-fix-loop': ['qa/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'browse/test/test-server.ts', 'test/skill-e2e-qa-workflow.test.ts'],
|
|
'qa-bootstrap': ['qa/**', 'ship/**', 'test/skill-e2e-qa-workflow.test.ts'],
|
|
|
|
// Review
|
|
'review-sql-injection': ['review/**', 'test/fixtures/review-eval-vuln.rb', 'test/skill-e2e-review.test.ts'],
|
|
'review-enum-completeness': ['review/**', 'test/fixtures/review-eval-enum*.rb', 'test/skill-e2e-review.test.ts'],
|
|
'review-base-branch': ['review/**', 'test/skill-e2e-review-attribution.test.ts'],
|
|
'review-design-lite': ['review/**', 'test/fixtures/review-eval-design-slop.*', 'test/fixtures/fake-impeccable.ts', 'test/fixtures/impeccable-detect-sample.json', 'lib/design-catalog.ts', 'lib/design-detect-contract.ts', 'bin/gstack-design-detect.ts', 'scripts/resolvers/design-checklist.ts', 'scripts/resolvers/review-army.ts', 'test/skill-e2e-review.test.ts'],
|
|
|
|
// Review Army (specialist dispatch)
|
|
'review-army-migration-safety': ['review/**', 'scripts/resolvers/review-army.ts', 'bin/gstack-diff-scope', 'test/skill-e2e-review-army.test.ts'],
|
|
'review-army-perf-n-plus-one': ['review/**', 'scripts/resolvers/review-army.ts', 'bin/gstack-diff-scope', 'test/skill-e2e-review-army.test.ts'],
|
|
'review-army-delivery-audit': ['review/**', 'scripts/resolvers/review.ts', 'scripts/resolvers/review-army.ts', 'test/skill-e2e-review-army.test.ts'],
|
|
'review-army-quality-score': ['review/**', 'scripts/resolvers/review-army.ts', 'test/skill-e2e-review-army.test.ts'],
|
|
'review-army-json-findings': ['review/**', 'scripts/resolvers/review-army.ts', 'test/skill-e2e-review-army.test.ts'],
|
|
'review-army-red-team': ['review/**', 'scripts/resolvers/review-army.ts', 'test/skill-e2e-review-army.test.ts'],
|
|
'review-army-simplification': ['review/**', 'scripts/resolvers/review-army.ts', 'test/fixtures/review-army-overbuild.js', 'test/fixtures/review-army-lean-complete.js', 'test/skill-e2e-review-army.test.ts'],
|
|
'review-army-simplification-precision': ['review/**', 'scripts/resolvers/review-army.ts', 'test/fixtures/review-army-overbuild.js', 'test/fixtures/review-army-lean-complete.js', 'test/skill-e2e-review-army.test.ts'],
|
|
'review-army-consensus': ['review/**', 'scripts/resolvers/review-army.ts', 'test/skill-e2e-review-army.test.ts'],
|
|
|
|
// Office Hours
|
|
'office-hours-spec-review': ['office-hours/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-plan.test.ts'],
|
|
'office-hours-forcing-energy': ['office-hours/**', 'scripts/resolvers/preamble.ts', 'test/fixtures/mode-posture/**', 'test/helpers/llm-judge.ts', 'test/skill-e2e-office-hours.test.ts'],
|
|
'office-hours-builder-wildness': ['office-hours/**', 'scripts/resolvers/preamble.ts', 'test/fixtures/mode-posture/**', 'test/helpers/llm-judge.ts', 'test/skill-e2e-office-hours.test.ts'],
|
|
|
|
// Plan reviews
|
|
'plan-ceo-review': ['plan-ceo-review/**', 'test/skill-e2e-plan.test.ts'],
|
|
'plan-ceo-review-selective': ['plan-ceo-review/**', 'test/skill-e2e-plan.test.ts'],
|
|
'plan-ceo-review-benefits': ['plan-ceo-review/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-plan.test.ts'],
|
|
'plan-ceo-review-expansion-energy': ['plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'test/fixtures/mode-posture/**', 'test/helpers/llm-judge.ts', 'test/skill-e2e-plan.test.ts'],
|
|
'plan-eng-review': ['plan-eng-review/**', 'test/skill-e2e-plan.test.ts'],
|
|
'plan-eng-review-artifact': ['plan-eng-review/**', 'test/skill-e2e-plan.test.ts'],
|
|
'plan-review-report': ['plan-eng-review/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-plan.test.ts'],
|
|
|
|
// Plan-mode smoke tests — gate-tier safety regression tests. Each test file
|
|
// contains TWO test cases as of v1.21: the baseline plan-mode case and the
|
|
// AskUserQuestion-blocked regression case (--disallowedTools AskUserQuestion
|
|
// parameterized — the flag set Conductor uses by default). Touchfiles
|
|
// include question-tuning.ts and generate-ask-user-format.ts because the
|
|
// AUTO_DECIDE preamble injection lives there and changes can flip the
|
|
// regression test outcome between 'asked' and 'auto_decided'.
|
|
'plan-ceo-review-plan-mode': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-ceo-review/**', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/question-tuning.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-ceo-plan-mode.test.ts'],
|
|
'plan-eng-review-plan-mode': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-eng-review/**', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/question-tuning.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-eng-plan-mode.test.ts'],
|
|
'plan-design-review-plan-mode': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-design-review/**', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/question-tuning.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-design-plan-mode.test.ts', 'test/skill-e2e-design.test.ts'],
|
|
'plan-devex-review-plan-mode': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-devex-review/**', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/question-tuning.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-devex-plan-mode.test.ts'],
|
|
// Covers ceo (preamble misfire) + eng/design (scope-gate bypass must not
|
|
// fire outside plan mode) + the named-target exception case. 4 PTY runs;
|
|
// in CI these run CONCURRENT with the rest of the pty-plan-smoke suite
|
|
// (--max-concurrency + --retry 1), so worst-case cost is ~2x a single
|
|
// pass of each, sharing the API budget with sibling tests — not the
|
|
// sequential ~+10min a local read suggests.
|
|
'plan-mode-no-op': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-ceo-review/**', 'plan-eng-review/**', 'plan-design-review/**', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/preamble.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-mode-no-op.test.ts'],
|
|
|
|
// v1.21+ AskUserQuestion-blocked regression tests — Conductor launches
|
|
// claude with `--disallowedTools AskUserQuestion --permission-mode default`
|
|
// (verified via `ps`); skills must still surface user-decisions through a
|
|
// fallback path (mcp__conductor__AskUserQuestion or plan-file flow) rather
|
|
// than silently auto-deciding. Parameterized regression test cases live
|
|
// INSIDE the existing 4 plan-X-review-plan-mode test files (covered
|
|
// transitively by the entries above). Two new standalone files exist for
|
|
// skills with no prior plan-mode test:
|
|
'office-hours-auto-mode': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'office-hours/**', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/question-tuning.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-office-hours-auto-mode.test.ts'],
|
|
'office-hours-phase4-fork': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'office-hours/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/question-tuning.ts', 'test/helpers/llm-judge.ts', 'test/skill-e2e-office-hours-phase4.test.ts'],
|
|
'llm-judge-recommendation': ['codex/**', 'test/helpers/llm-judge.ts', 'test/llm-judge-recommendation.test.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'codex/SKILL.md.tmpl', 'scripts/resolvers/review.ts'],
|
|
// v1.21+ AUTO_DECIDE preserve eval (periodic). Verifies the Tool resolution
|
|
// fix doesn't trip the legitimate /plan-tune opt-in path: when the user has
|
|
// written a never-ask preference, AUQ should still auto-decide rather than
|
|
// surfacing the question. Touches the question-tuning + preference
|
|
// infrastructure plus the resolvers that own the AUTO_DECIDE preamble.
|
|
'auto-decide-preserved': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'bin/gstack-session-kind', 'scripts/resolvers/question-tuning.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-preamble-bash.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'plan-ceo-review/**', 'bin/gstack-question-preference', 'bin/gstack-config', 'bin/gstack-slug', 'hosts/claude/hooks/question-preference-hook.ts', 'hosts/claude/hooks/spawned-directive.ts', 'lib/is-conductor.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-auto-decide-preserved.test.ts'],
|
|
|
|
// Conductor → prose decision brief (Conductor signal makes prose the default;
|
|
// the PreToolUse hook denies the flaky tool). Touches the resolver that owns
|
|
// the Conductor rule, the preamble signal, the hook, and the detection helper.
|
|
'conductor-prose': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'bin/gstack-session-kind', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-preamble-bash.ts', 'scripts/resolvers/preamble.ts', 'plan-eng-review/**', 'hosts/claude/hooks/question-preference-hook.ts', 'hosts/claude/hooks/spawned-directive.ts', 'lib/is-conductor.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-conductor-prose.test.ts'],
|
|
|
|
// Real-PTY E2E batch (#6 new tests on the harness).
|
|
// Each one tests behavior the SDK harness can't observe (rendered TTY,
|
|
// numbered-option lists, multi-phase ordering, idempotency state echo).
|
|
'auq-format-gate': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completeness-section.ts', 'scripts/resolvers/preamble.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/session-runner.ts', 'test/helpers/llm-judge.ts', 'test/skill-e2e-ask-user-question-format-compliance.test.ts'],
|
|
'plan-ceo-mode-routing': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-ceo-mode-routing.test.ts'],
|
|
'plan-design-with-ui-scope': ['plan-design-review/**', 'test/fixtures/plans/ui-heavy-feature.md', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-design-with-ui.test.ts'],
|
|
'ship-idempotency-pty': ['ship/**', 'bin/gstack-next-version', 'bin/gstack-version-bump', 'scripts/resolvers/sections.ts', 'lib/worktree.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-ship-idempotency.test.ts'],
|
|
'tpa-present': ['scripts/resolvers/third-party-actions.ts', 'ship/SKILL.md.tmpl', 'ship/sections/apple-release.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-third-party-actions.test.ts'],
|
|
'tpa-absent-linux': ['scripts/resolvers/third-party-actions.ts', 'ship/SKILL.md.tmpl', 'ship/sections/apple-release.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-third-party-actions.test.ts'],
|
|
'tpa-broken': ['scripts/resolvers/third-party-actions.ts', 'ship/SKILL.md.tmpl', 'ship/sections/apple-release.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-third-party-actions.test.ts'],
|
|
'tpa-absent-darwin': ['scripts/resolvers/third-party-actions.ts', 'ship/SKILL.md.tmpl', 'ship/sections/apple-release.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-third-party-actions.test.ts'],
|
|
'tpa-apple-ban': ['scripts/resolvers/third-party-actions.ts', 'ship/SKILL.md.tmpl', 'ship/sections/apple-release.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-third-party-actions.test.ts'],
|
|
'ship-section-loading': ['ship/**', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-ship-section-loading.test.ts'],
|
|
'plan-ceo-section-loading': ['plan-ceo-review/**', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-plan-ceo-review-section-loading.test.ts'],
|
|
// Data-driven behavioral guard for the 'plan'/'prompt' carves (eng, design,
|
|
// devex, office-hours + future PR2 carves). One file iterating CARVE_GUARDS;
|
|
// the selector sets GSTACK_CARVE_SKILL=<name> to scope cost to the changed
|
|
// skill (D-CODEX A). Touching the registry/helper or sections.ts runs all.
|
|
'carve-section-loading': ['design-html/**', 'design-shotgun/**', 'qa/**', 'browse/**', 'retro/**', 'autoplan/**', 'spec/**', 'setup-gbrain/**', 'review/**', 'codex/**', 'land-and-deploy/**', 'plan-eng-review/**', 'plan-design-review/**', 'plan-devex-review/**', 'office-hours/**', 'document-release/**', 'design-consultation/**', 'cso/**', 'test/helpers/carve-guards.ts', 'scripts/resolvers/sections.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/session-runner.ts', 'test/carve-section-loading.test.ts'],
|
|
'autoplan-chain-pty': ['autoplan/**', 'plan-ceo-review/**', 'plan-design-review/**', 'plan-eng-review/**', 'plan-devex-review/**', 'test/fixtures/plans/ui-heavy-feature.md', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-autoplan-chain.test.ts'],
|
|
|
|
// Per-finding AskUserQuestion count + review-report-at-bottom assertion.
|
|
// Each test drives its skill end-to-end; touchfiles include preamble +
|
|
// completion-status resolvers because they affect question cadence and
|
|
// terminal output (the regression surface this test catches).
|
|
'plan-ceo-finding-count': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-ceo-finding-count.test.ts'],
|
|
'plan-eng-finding-count': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-eng-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-eng-finding-count.test.ts'],
|
|
'plan-design-finding-count': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-design-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-design-finding-count.test.ts'],
|
|
'plan-devex-finding-count': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-devex-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/skill-e2e-plan-devex-finding-count.test.ts'],
|
|
|
|
// Gate-tier reviewCount-floor counterparts. Catch the May 2026 transcript
|
|
// bug (model wrote a plan-mode plan and ExitPlanMode'd without firing any
|
|
// review-phase AskUserQuestion). Uses runPlanSkillFloorCheck — minimal
|
|
// "did agent fire ANY AUQ?" observer that exits early on first non-permission
|
|
// numbered-option render. ~1-3 min typical wall time per test, ~$2-6 total.
|
|
'plan-eng-finding-floor': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-eng-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-eng-finding-floor.test.ts'],
|
|
'plan-ceo-finding-floor': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-ceo-finding-floor.test.ts'],
|
|
'plan-design-finding-floor': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-design-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-design-finding-floor.test.ts'],
|
|
'plan-devex-finding-floor': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-devex-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-devex-finding-floor.test.ts'],
|
|
|
|
// Multi-finding batching regression — periodic tier complement to the
|
|
// gate-tier finding-floor. Catches the May 2026 transcript shape where
|
|
// a model fires one AUQ then batches the rest into a "## Decisions to
|
|
// confirm" plan write. runPlanSkillFloorCheck cannot detect that shape
|
|
// (it exits on first AUQ); runPlanSkillCounting can.
|
|
'plan-eng-multi-finding-batching': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-eng-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-eng-multi-finding-batching.test.ts'],
|
|
'plan-ceo-split-overflow': ['plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'bin/gstack-question-preference', 'test/helpers/claude-pty-runner.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-ceo-split-overflow.test.ts'],
|
|
'brain-privacy-gate': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'scripts/resolvers/preamble/generate-brain-sync-block.ts', 'scripts/resolvers/preamble.ts', 'bin/gstack-brain-sync', 'bin/gstack-artifacts-init', 'bin/gstack-config', 'test/helpers/agent-sdk-runner.ts', 'test/skill-e2e-brain-privacy-gate.test.ts'],
|
|
|
|
// /setup-gbrain Path 4 (Remote MCP) — happy + bad-token end-to-end via
|
|
// Agent SDK. Gate-tier (deterministic stub server, fixed inputs); fires
|
|
// when the skill template, the verify helper, the artifacts-init helper,
|
|
// or the detect script changes.
|
|
'setup-gbrain-remote': ['setup-gbrain/sections/brain-init.md.tmpl', 'setup-gbrain/sections/claude-md-persist.md.tmpl', 'setup-gbrain/sections/manifest.json', 'test/helpers/setup-gbrain-fixture.ts', 'setup-gbrain/SKILL.md.tmpl', 'bin/gstack-gbrain-mcp-verify', 'bin/gstack-artifacts-init', 'bin/gstack-gbrain-detect', 'test/helpers/agent-sdk-runner.ts', 'test/skill-e2e-setup-gbrain-remote.test.ts'],
|
|
'setup-gbrain-bad-token': ['setup-gbrain/sections/brain-init.md.tmpl', 'setup-gbrain/sections/manifest.json', 'test/helpers/setup-gbrain-fixture.ts', 'setup-gbrain/SKILL.md.tmpl', 'bin/gstack-gbrain-mcp-verify', 'test/helpers/agent-sdk-runner.ts', 'test/skill-e2e-setup-gbrain-bad-token.test.ts'],
|
|
// v1.34.0.0 split-engine Path 4 + Step 4.5 Yes (local PGLite for code).
|
|
// Periodic-tier per codex #12 (AgentSDK harness is non-deterministic).
|
|
// Fires when the setup-gbrain template, install/verify/init helpers, or
|
|
// the agent-sdk-runner harness changes.
|
|
'setup-gbrain-path4-local-pglite': ['setup-gbrain/sections/brain-init.md.tmpl', 'setup-gbrain/sections/claude-md-persist.md.tmpl', 'setup-gbrain/sections/manifest.json', 'test/helpers/setup-gbrain-fixture.ts', 'setup-gbrain/SKILL.md.tmpl', 'bin/gstack-gbrain-mcp-verify', 'bin/gstack-gbrain-install', 'bin/gstack-gbrain-detect', 'lib/gbrain-local-status.ts', 'test/helpers/agent-sdk-runner.ts', 'test/skill-e2e-setup-gbrain-path4-local-pglite.test.ts'],
|
|
|
|
// AskUserQuestion format regression (RECOMMENDATION + Completeness: N/10)
|
|
// Fires when either template OR the two preamble resolvers change.
|
|
'plan-ceo-review-format-mode': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completeness-section.ts', 'scripts/resolvers/preamble.ts', 'model-overlays/opus-4-7.md', 'test/helpers/llm-judge.ts', 'test/skill-e2e-plan-format.test.ts'],
|
|
'plan-ceo-review-format-approach': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completeness-section.ts', 'scripts/resolvers/preamble.ts', 'model-overlays/opus-4-7.md', 'test/helpers/llm-judge.ts', 'test/skill-e2e-plan-format.test.ts'],
|
|
'plan-eng-review-format-coverage': ['plan-eng-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completeness-section.ts', 'scripts/resolvers/preamble.ts', 'model-overlays/opus-4-7.md', 'test/helpers/llm-judge.ts', 'test/skill-e2e-plan-format.test.ts'],
|
|
'plan-eng-review-format-kind': ['plan-eng-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completeness-section.ts', 'scripts/resolvers/preamble.ts', 'model-overlays/opus-4-7.md', 'test/helpers/llm-judge.ts', 'test/skill-e2e-plan-format.test.ts'],
|
|
|
|
// v1.7.0.0 Pros/Cons format cadence + format + negative-escape evals.
|
|
// Dependencies: same as format-mode + the 4 plan-review templates + overlay.
|
|
// All periodic-tier (non-deterministic Opus 4.7 behavior).
|
|
'plan-ceo-review-prosons-cadence': ['plan-ceo-review/**', 'plan-eng-review/**', 'plan-design-review/**', 'plan-devex-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'model-overlays/opus-4-7.md', 'test/skill-e2e-plan-prosons.test.ts'],
|
|
'plan-review-prosons-format': ['plan-ceo-review/**', 'plan-eng-review/**', 'plan-design-review/**', 'plan-devex-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'model-overlays/opus-4-7.md', 'test/skill-e2e-plan-prosons.test.ts'],
|
|
'plan-review-prosons-hardstop-neg': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'model-overlays/opus-4-7.md', 'test/skill-e2e-plan-prosons.test.ts'],
|
|
'plan-review-prosons-neutral-neg': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'model-overlays/opus-4-7.md', 'test/skill-e2e-plan-prosons.test.ts'],
|
|
|
|
// Expanded coverage (CT3) — 6 non-plan-review skills inherit Pros/Cons via preamble
|
|
|
|
// /plan-tune (v1 observational)
|
|
'plan-tune-inspect': ['plan-tune/**', 'scripts/question-registry.ts', 'scripts/psychographic-signals.ts', 'scripts/one-way-doors.ts', 'bin/gstack-question-log', 'bin/gstack-question-preference', 'bin/gstack-developer-profile', 'test/skill-e2e-plan-tune.test.ts'],
|
|
|
|
// /plan-tune cathedral (T16 — 5 E2E scenarios, all gate per D12)
|
|
'plan-tune-hook-capture': ['hosts/claude/hooks/**', 'bin/gstack-question-log', 'bin/gstack-developer-profile', 'plan-tune/**', 'test/skill-e2e-plan-tune-cathedral.test.ts'],
|
|
'plan-tune-enforcement': ['hosts/claude/hooks/**', 'bin/gstack-question-preference', 'scripts/question-registry.ts', 'test/skill-e2e-plan-tune-cathedral.test.ts'],
|
|
'plan-tune-annotation': ['hosts/claude/hooks/**', 'scripts/declared-annotation.ts', 'scripts/psychographic-signals.ts', 'scripts/question-registry.ts', 'test/skill-e2e-plan-tune-cathedral.test.ts'],
|
|
'plan-tune-codex-import': ['bin/gstack-codex-session-import', 'bin/gstack-question-log', 'docs/spikes/codex-session-format.md', 'test/skill-e2e-plan-tune-cathedral.test.ts'],
|
|
'plan-tune-dream-cycle': ['bin/gstack-distill-free-text', 'bin/gstack-distill-apply', 'hosts/claude/hooks/**', 'plan-tune/**', 'test/skill-e2e-plan-tune-cathedral.test.ts'],
|
|
|
|
// Codex offering verification
|
|
'codex-offered-office-hours': ['office-hours/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-plan.test.ts'],
|
|
'codex-offered-ceo-review': ['plan-ceo-review/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-plan.test.ts'],
|
|
'codex-offered-design-review': ['plan-design-review/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-plan.test.ts'],
|
|
'codex-offered-eng-review': ['plan-eng-review/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-plan.test.ts'],
|
|
|
|
// Ship
|
|
'ship-base-branch': ['ship/**', 'bin/gstack-repo-mode', 'test/skill-e2e-review-attribution.test.ts'],
|
|
'ship-local-workflow': ['ship/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-workflow.test.ts'],
|
|
'review-dashboard-via': ['ship/**', 'scripts/resolvers/review.ts', 'codex/**', 'autoplan/**', 'land-and-deploy/**', 'test/skill-e2e-review-attribution.test.ts'],
|
|
|
|
// Retro
|
|
'retro': ['bin/gstack-retro-metrics', 'retro/**', 'test/skill-e2e-retro.test.ts'],
|
|
'retro-base-branch': ['bin/gstack-retro-metrics', 'retro/**', 'test/skill-e2e-retro.test.ts'],
|
|
|
|
// CSO
|
|
'cso-full-audit': ['cso/**', 'test/skill-e2e-cso.test.ts'],
|
|
'cso-diff-mode': ['cso/**', 'test/skill-e2e-cso.test.ts'],
|
|
'cso-infra-scope': ['cso/**', 'test/skill-e2e-cso.test.ts'],
|
|
|
|
// Learnings
|
|
'learnings-show': ['learn/**', 'bin/gstack-learnings-search', 'bin/gstack-learnings-log', 'scripts/resolvers/learnings.ts', 'test/skill-e2e-learnings.test.ts'],
|
|
|
|
// Session Intelligence (timeline, context recovery, /context-save + /context-restore)
|
|
'timeline-event-flow': ['bin/gstack-timeline-log', 'bin/gstack-timeline-read', 'test/skill-e2e-session-intelligence.test.ts'],
|
|
'context-recovery-artifacts': ['scripts/resolvers/preamble.ts', 'bin/gstack-timeline-log', 'bin/gstack-slug', 'learn/**', 'test/skill-e2e-session-intelligence.test.ts'],
|
|
'context-save-writes-file': ['context-save/**', 'bin/gstack-slug', 'test/skill-e2e-session-intelligence.test.ts'],
|
|
'context-restore-loads-latest': ['context-restore/**', 'bin/gstack-slug', 'test/skill-e2e-session-intelligence.test.ts'],
|
|
|
|
// Context skills E2E (live-fire, Skill-tool routing path) — see
|
|
// test/skill-e2e-context-skills.test.ts. These are periodic-tier because
|
|
// each one spawns claude -p and costs ~$0.20-$0.40. Collectively they
|
|
// verify the thing the /checkpoint → /context-save rename was for.
|
|
'context-save-routing': ['context-save/**', 'scripts/resolvers/preamble.ts', 'test/skill-e2e-context-skills.test.ts'],
|
|
'context-save-then-restore-roundtrip': ['context-save/**', 'context-restore/**', 'bin/gstack-slug', 'test/skill-e2e-context-skills.test.ts'],
|
|
'context-restore-fragment-match': ['context-restore/**', 'test/skill-e2e-context-skills.test.ts'],
|
|
'context-restore-empty-state': ['context-restore/**', 'test/skill-e2e-context-skills.test.ts'],
|
|
'context-restore-list-delegates': ['context-restore/**', 'test/skill-e2e-context-skills.test.ts'],
|
|
'context-restore-legacy-compat': ['context-restore/**', 'test/skill-e2e-context-skills.test.ts'],
|
|
'context-save-list-current-branch': ['context-save/**', 'test/skill-e2e-context-skills.test.ts'],
|
|
'context-save-list-all-branches': ['context-save/**', 'test/skill-e2e-context-skills.test.ts'],
|
|
|
|
// Document-release
|
|
'document-release': ['document-release/**', 'test/skill-e2e-workflow.test.ts'],
|
|
|
|
// Codex (Claude E2E — tests /codex skill via Claude)
|
|
'codex-review': ['codex/**', 'test/skill-e2e-workflow.test.ts'],
|
|
|
|
// Codex E2E (tests skills via Codex CLI + worktree)
|
|
'codex-discover-skill': ['codex/**', 'scripts/gen-skill-docs.ts', 'test/helpers/codex-session-runner.ts', 'lib/worktree.ts', 'test/codex-e2e.test.ts'],
|
|
'codex-review-findings': ['review/**', 'scripts/gen-skill-docs.ts', 'codex/**', 'test/helpers/codex-session-runner.ts', 'lib/worktree.ts', 'test/codex-e2e.test.ts'],
|
|
|
|
// GPT-5.6 Sol scope-termination E2E (Codex CLI, full generated investigate skill)
|
|
'codex-sol-scope-termination': ['model-overlays/gpt-5.6-sol.md', 'scripts/models.ts', 'scripts/resolvers/model-overlay.ts', 'scripts/resolvers/preamble/**', 'investigate/**', 'test/helpers/codex-session-runner.ts', 'test/codex-e2e-sol-scope.test.ts'],
|
|
|
|
// Gemini E2E — smoke test only (Gemini gets lost in worktrees on complex tasks)
|
|
'gemini-smoke': ['scripts/gen-skill-docs.ts', 'test/helpers/gemini-session-runner.ts', 'lib/worktree.ts', 'test/gemini-e2e.test.ts'],
|
|
|
|
|
|
// Coverage audit (shared fixture) + triage + gates
|
|
'ship-coverage-audit': ['ship/**', 'test/fixtures/coverage-audit-fixture.ts', 'bin/gstack-repo-mode', 'test/skill-e2e-workflow.test.ts'],
|
|
'review-coverage-audit': ['review/**', 'test/fixtures/coverage-audit-fixture.ts', 'test/skill-e2e-coverage-audit.test.ts'],
|
|
'plan-eng-coverage-audit': ['plan-eng-review/**', 'test/fixtures/coverage-audit-fixture.ts', 'test/skill-e2e-coverage-audit.test.ts'],
|
|
'ship-triage': ['ship/**', 'bin/gstack-repo-mode', 'test/skill-e2e-triage.test.ts'],
|
|
'ship-docsync': ['ship/**', 'document-release/**', 'scripts/gen-skill-docs.ts', 'scripts/resolvers/sections.ts', 'test/skill-e2e-ship-docsync.test.ts'],
|
|
// #2733 behavioral proof: the JSON contract survives a firing gate inside a
|
|
// spawned-marked subagent. Deps name every behavior under test — the
|
|
// session-kind override, the skill-start gates, both hooks + the shared
|
|
// directive, and the AUQ prose rule — so changing any of them selects it.
|
|
'docsync-spawned': [
|
|
'document-release/**',
|
|
'ship/sections/pr-body.md',
|
|
'bin/gstack-session-kind',
|
|
'bin/gstack-skill-start',
|
|
'hosts/claude/hooks/question-preference-hook.ts',
|
|
'hosts/claude/hooks/auq-error-fallback-hook.ts',
|
|
'hosts/claude/hooks/spawned-directive.ts',
|
|
'scripts/resolvers/preamble/generate-ask-user-format.ts',
|
|
'test/skill-e2e-docsync-spawned.test.ts',
|
|
],
|
|
|
|
// Design
|
|
'design-consultation-core': ['design-consultation/**', 'lib/design-catalog.ts', 'lib/design-md.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/llm-judge.ts', 'test/skill-e2e-design.test.ts'],
|
|
'design-consultation-existing': ['design-consultation/**', 'lib/design-md.ts', 'bin/gstack-design-md.ts', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
|
'design-consultation-research': ['design-consultation/**', 'scripts/resolvers/aside.ts', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
|
'design-consultation-preview': ['design-consultation/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
|
'plan-design-review-no-ui-scope': ['plan-design-review/**', 'lib/design-catalog.ts', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
|
'design-review-fix': ['design-review/**', 'scripts/resolvers/aside.ts', 'scripts/resolvers/design.ts', 'lib/design-catalog.ts', 'browse/src/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-design.test.ts'],
|
|
// Design detector (user-installed impeccable engine) through the fake engine shim: source mode on a diff and DOM mode on a served page.
|
|
'design-review-detector-shim': ['design-review/**', 'scripts/resolvers/design.ts', 'lib/design-catalog.ts', 'lib/design-detect-contract.ts', 'lib/dom-dump-script.ts', 'lib/dom-dump.js', 'bin/gstack-design-detect.ts', 'test/fixtures/fake-impeccable.ts', 'test/fixtures/impeccable-detect-sample.json', 'test/fixtures/review-eval-design-slop.*', 'test/skill-e2e-design.test.ts'],
|
|
'design-review-detector-shim-dom': ['design-review/**', 'scripts/resolvers/design.ts', 'lib/design-detect-contract.ts', 'lib/dom-dump-script.ts', 'lib/dom-dump.js', 'bin/gstack-design-detect.ts', 'browse/src/**', 'test/fixtures/fake-impeccable.ts', 'test/fixtures/impeccable-detect-sample.json', 'test/fixtures/review-eval-design-slop.*', 'test/skill-e2e-design.test.ts'],
|
|
'design-html-slop-gate': ['design-html/**', 'scripts/resolvers/design.ts', 'lib/design-detect-contract.ts', 'bin/gstack-design-detect.ts', 'test/fixtures/fake-impeccable.ts', 'test/fixtures/impeccable-detect-sample.json', 'test/skill-e2e-design.test.ts'],
|
|
|
|
// /diagram (diagram-render bundle consumers). Triplet = deterministic
|
|
// functional (gate); authoring quality = LLM-judged benchmark (periodic).
|
|
// Both render the triplet through gstack-render (lib/aside-render.ts +
|
|
// bin/gstack-render.ts): Aside when it is running, the browse daemon
|
|
// otherwise — so both engines are deps. Triplet = deterministic functional
|
|
// (gate); authoring quality = LLM-judged benchmark (periodic).
|
|
'diagram-triplet': ['diagram/**', 'lib/diagram-render/**', 'lib/aside-render.ts', 'bin/gstack-render.ts', 'test/helpers/aside-available.ts', 'browse/src/**', 'test/skill-e2e-diagram.test.ts'],
|
|
'diagram-authoring-quality': ['diagram/**', 'lib/diagram-render/**', 'lib/aside-render.ts', 'bin/gstack-render.ts', 'test/helpers/aside-available.ts', 'browse/src/**', 'test/helpers/llm-judge.ts', 'test/skill-e2e-diagram.test.ts'],
|
|
|
|
// gstack-upgrade
|
|
'gstack-upgrade-happy-path': ['gstack-upgrade/**', 'test/skill-e2e-workflow.test.ts'],
|
|
|
|
// Deploy skills
|
|
'land-and-deploy-workflow': ['land-and-deploy/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-deploy.test.ts'],
|
|
'land-and-deploy-first-run': ['land-and-deploy/**', 'scripts/gen-skill-docs.ts', 'bin/gstack-slug', 'test/skill-e2e-deploy.test.ts'],
|
|
'land-and-deploy-review-gate': ['land-and-deploy/**', 'bin/gstack-review-read', 'test/skill-e2e-deploy.test.ts'],
|
|
'canary-workflow': ['canary/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'test/skill-e2e-deploy.test.ts'],
|
|
'benchmark-workflow': ['benchmark/**', 'scripts/resolvers/aside.ts', 'browse/src/**', 'test/skill-e2e-deploy.test.ts'],
|
|
'setup-deploy-workflow': ['setup-deploy/**', 'scripts/gen-skill-docs.ts', 'test/skill-e2e-deploy.test.ts'],
|
|
|
|
|
|
// Autoplan
|
|
'autoplan-dual-voice': ['autoplan/**', 'codex/**', 'bin/gstack-codex-probe', 'scripts/resolvers/review.ts', 'scripts/resolvers/design.ts', 'test/skill-e2e-autoplan-dual-voice.test.ts'],
|
|
|
|
// Multi-provider benchmark adapters — live API smoke against real claude/codex/gemini CLIs
|
|
'benchmark-providers-live': ['bin/gstack-model-benchmark', 'test/helpers/providers/**', 'test/helpers/benchmark-runner.ts', 'test/helpers/pricing.ts', 'test/skill-e2e-benchmark-providers.test.ts'],
|
|
|
|
// Browser-skills Phase 2a — /scrape + /skillify (v1.19.0.0). Gate-tier
|
|
// E2E covers the D1 (provenance guard), D3 (atomic write) contracts plus
|
|
// the basic loop. Shared deps: both skill templates, the D3 helper, the
|
|
// Phase 1 runtime, and the bundled hackernews-frontpage reference (the
|
|
// match-path test relies on it).
|
|
'scrape-match-path': [
|
|
'scrape/**', 'browse/src/browser-skills.ts', 'browse/src/browser-skill-commands.ts',
|
|
'browser-skills/hackernews-frontpage/**',
|
|
'test/skill-e2e-skillify.test.ts',
|
|
],
|
|
'scrape-prototype-path': [
|
|
'scrape/**', 'browse/src/browser-skills.ts', 'browse/src/browser-skill-commands.ts',
|
|
'test/skill-e2e-skillify.test.ts',
|
|
],
|
|
'skillify-happy-path': [
|
|
'skillify/**', 'scrape/**', 'browse/src/browser-skill-write.ts',
|
|
'browse/src/browser-skills.ts', 'browse/src/browser-skill-commands.ts',
|
|
'test/skill-e2e-skillify.test.ts',
|
|
],
|
|
'skillify-provenance-refusal': [
|
|
'skillify/**', 'browse/src/browser-skill-write.ts',
|
|
'test/skill-e2e-skillify.test.ts',
|
|
],
|
|
'skillify-approval-reject': [
|
|
'skillify/**', 'scrape/**', 'browse/src/browser-skill-write.ts',
|
|
'test/skill-e2e-skillify.test.ts',
|
|
],
|
|
|
|
// Skill routing — journey-stage tests (depend on ALL skill descriptions)
|
|
'journey-ideation': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-plan-eng': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-debug': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-qa': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-code-review': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-ship': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-docs': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-retro': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-design-system': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
'journey-visual-qa': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-routing-e2e.test.ts'],
|
|
|
|
// Opus 4.7 behavior evals — keys match testName: values in the test file.
|
|
// Routing sub-tests use template literal `routing-${c.name}` testNames,
|
|
// which the touchfile completeness scanner skips; they inherit selection
|
|
// from the file-level touchfile entry via GLOBAL_TOUCHFILES.
|
|
'fanout-arm-overlay-on':
|
|
['model-overlays/claude.md', 'model-overlays/opus-4-7.md', 'scripts/models.ts', 'scripts/resolvers/model-overlay.ts', 'test/skill-e2e-opus-47.test.ts'],
|
|
'fanout-arm-overlay-off':
|
|
['model-overlays/claude.md', 'model-overlays/opus-4-7.md', 'scripts/models.ts', 'scripts/resolvers/model-overlay.ts', 'test/skill-e2e-opus-47.test.ts'],
|
|
|
|
// Overlay efficacy harness (SDK) — measures whether overlay nudges change
|
|
// behavior under @anthropic-ai/claude-agent-sdk (closer to real Claude Code
|
|
// than `claude -p`). testNames in the file are template literals so the
|
|
// completeness scanner doesn't require them; these entries exist for
|
|
// diff-based selection accuracy.
|
|
'overlay-harness-opus-4-7-fanout-toy': [
|
|
'model-overlays/**',
|
|
'test/fixtures/overlay-nudges.ts',
|
|
'test/helpers/agent-sdk-runner.ts',
|
|
'scripts/resolvers/model-overlay.ts',
|
|
'test/skill-e2e-overlay-harness.test.ts',
|
|
],
|
|
'overlay-harness-opus-4-7-fanout-realistic': [
|
|
'model-overlays/**',
|
|
'test/fixtures/overlay-nudges.ts',
|
|
'test/helpers/agent-sdk-runner.ts',
|
|
'scripts/resolvers/model-overlay.ts',
|
|
'test/skill-e2e-overlay-harness.test.ts',
|
|
],
|
|
|
|
// /ios-qa — agent flow E2E. Daemon + stub StateServer + codegen
|
|
// exercised end-to-end. The no-device path is gate-tier; the with-device
|
|
// path requires GSTACK_HAS_IOS_DEVICE=1 and is periodic-tier.
|
|
'ios-qa-e2e': ['ios-qa/**', 'ios-fix/**', 'ios-design-review/**', 'ios-clean/**', 'ios-sync/**', 'test/skill-e2e-ios.test.ts'],
|
|
// Swift-build invariant test — requires the Swift toolchain. Compiles the
|
|
// fixture SPM package + runs the XCTest suite that validates the real
|
|
// Swift StateServer implementation (loopback bind, boot token rotation,
|
|
// session lock). Periodic-tier — Swift build is heavier than TS unit tests.
|
|
'ios-qa-swift-build': ['ios-qa/templates/**', 'test/fixtures/ios-qa/FixtureApp/**', 'test/skill-e2e-ios-swift-build.test.ts'],
|
|
// Real-device path — only runs with GSTACK_HAS_IOS_DEVICE=1 + a paired
|
|
// iPhone. Validates the CoreDevice agent + iOS SDK toolchain. Periodic-tier.
|
|
'ios-qa-device': ['ios-qa/templates/**', 'test/fixtures/ios-qa/FixtureApp/**', 'test/skill-e2e-ios-device.test.ts'],
|
|
|
|
// /spec end-to-end via PTY — exercises the full Phase 1→5 pipeline
|
|
// including --execute spawn. Periodic-tier — paid + non-deterministic.
|
|
'spec-execute': ['spec/**', 'test/skill-e2e-spec-execute.test.ts'],
|
|
|
|
// /office-hours brain-writeback path under fake gbrain CLI (v1.50.0.0
|
|
// T7). Drives /office-hours with a regenerated SKILL.md that has the
|
|
// compressed GBRAIN_SAVE_RESULTS block + a fake gbrain on PATH; asserts
|
|
// the agent calls `gbrain put office-hours/<slug>` with valid YAML
|
|
// frontmatter. Touched by anything that changes resolver output, gen
|
|
// pipeline, detection helper, refresh subcommand, or the on-demand
|
|
// docs the resolver points to.
|
|
'office-hours-brain-writeback': ['office-hours/sections/**',
|
|
'scripts/resolvers/gbrain.ts',
|
|
'scripts/gen-skill-docs.ts',
|
|
'bin/gstack-gbrain-detect',
|
|
'bin/gstack-config',
|
|
'office-hours/SKILL.md.tmpl',
|
|
'docs/gbrain-write-surfaces.md',
|
|
'test/fixtures/office-hours-brain-writeback/**',
|
|
'test/skill-e2e-office-hours-brain-writeback.test.ts',
|
|
],
|
|
|
|
// gbrain CLI real round-trip against a local PGLite store (v1.50.0.0
|
|
// T11). Proves the gbrain CLI persistence contract gstack relies on —
|
|
// a `gbrain put` followed by `gbrain get` returns the body. Skips if
|
|
// VOYAGE_API_KEY is unset OR gbrain CLI not on PATH. Touched by the
|
|
// resolver (which emits the CLI shape) and the test itself.
|
|
'gbrain-roundtrip-local': [
|
|
'scripts/resolvers/gbrain.ts',
|
|
'test/skill-e2e-gbrain-roundtrip-local.test.ts',
|
|
],
|
|
|
|
// WS2 arm benchmark — with-skill vs without-skill agentic arms scored on
|
|
// the git diff left behind (research instrument, never a release gate).
|
|
// Fires when the behavioral layer under test (reuse ladder + bounded
|
|
// closer resolvers), the judge, the fixtures, or the harness change.
|
|
'arm-benchmark-native-overbuild': [
|
|
'scripts/resolvers/preamble/generate-search-before-building.ts',
|
|
'scripts/resolvers/preamble/generate-voice-directive.ts',
|
|
'test/fixtures/arm-benchmark/**',
|
|
'test/helpers/llm-judge.ts',
|
|
'test/helpers/arm-benchmark-harness.ts',
|
|
'test/skill-e2e-arm-benchmark.test.ts',
|
|
'ship/SKILL.md',
|
|
],
|
|
'arm-benchmark-crud-endpoint': [
|
|
'scripts/resolvers/preamble/generate-search-before-building.ts',
|
|
'scripts/resolvers/preamble/generate-voice-directive.ts',
|
|
'test/fixtures/arm-benchmark/**',
|
|
'test/helpers/llm-judge.ts',
|
|
'test/helpers/arm-benchmark-harness.ts',
|
|
'test/skill-e2e-arm-benchmark.test.ts',
|
|
'ship/SKILL.md',
|
|
],
|
|
'arm-benchmark-bugfix-decoys': [
|
|
'scripts/resolvers/preamble/generate-search-before-building.ts',
|
|
'scripts/resolvers/preamble/generate-voice-directive.ts',
|
|
'test/fixtures/arm-benchmark/**',
|
|
'test/helpers/llm-judge.ts',
|
|
'test/helpers/arm-benchmark-harness.ts',
|
|
'test/skill-e2e-arm-benchmark.test.ts',
|
|
'ship/SKILL.md',
|
|
],
|
|
|
|
};
|
|
|
|
/**
|
|
* E2E test tiers — 'gate' blocks PRs, 'periodic' runs weekly/on-demand.
|
|
* Must have exactly the same keys as E2E_TOUCHFILES.
|
|
*/
|
|
export const E2E_TIERS: Record<string, 'gate' | 'periodic'> = {
|
|
// Browse core — gate (if browse breaks, everything breaks)
|
|
'browse-basic': 'gate',
|
|
'browse-snapshot': 'gate',
|
|
|
|
// Aside-driven browsing — periodic (external app: the Aside browser; the
|
|
// file self-gates on 'periodic' and skips without a live Aside)
|
|
'aside-browse-basic': 'periodic',
|
|
'aside-browse-flow': 'periodic',
|
|
'aside-qa-quick': 'periodic',
|
|
'aside-scrape-json': 'periodic',
|
|
'aside-canary-quick': 'periodic',
|
|
|
|
// Hermetic isolation — gate (deterministic env/config assertions; if the
|
|
// clean room breaks, every other eval's signal is contaminated)
|
|
'hermetic-canary': 'gate',
|
|
'hermetic-sentinel': 'gate',
|
|
|
|
// SKILL.md setup — gate (if setup breaks, no skill works)
|
|
'skillmd-setup-discovery': 'gate',
|
|
'skillmd-no-local-binary': 'gate',
|
|
'skillmd-outside-git': 'gate',
|
|
'session-awareness': 'gate',
|
|
'operational-learning': 'gate',
|
|
|
|
// P4 first-run scaffold — periodic (onboarding, non-safety, model-touched marker)
|
|
'first-task-scaffold': 'periodic',
|
|
|
|
// QA — gate for functional, periodic for quality/benchmarks
|
|
'qa-quick': 'gate',
|
|
'qa-b6-static': 'periodic',
|
|
'qa-b7-spa': 'periodic',
|
|
'qa-b8-checkout': 'periodic',
|
|
'qa-only-no-fix': 'gate', // CRITICAL guardrail: Edit tool forbidden
|
|
'qa-fix-loop': 'periodic',
|
|
'qa-bootstrap': 'gate',
|
|
|
|
// Review — gate for functional/guardrails, periodic for quality
|
|
'review-sql-injection': 'gate', // Security guardrail
|
|
'review-enum-completeness': 'gate',
|
|
'review-base-branch': 'gate',
|
|
'review-design-lite': 'periodic', // 4/7 threshold is subjective
|
|
'review-coverage-audit': 'gate',
|
|
'review-dashboard-via': 'gate',
|
|
|
|
// Review Army — gate for core functionality, periodic for multi-specialist
|
|
'review-army-migration-safety': 'gate', // Specialist activation guardrail
|
|
'review-army-perf-n-plus-one': 'gate', // Specialist activation guardrail
|
|
'review-army-delivery-audit': 'gate', // Delivery integrity guardrail
|
|
'review-army-quality-score': 'gate', // Score computation
|
|
'review-army-json-findings': 'gate', // JSON schema compliance
|
|
'review-army-red-team': 'periodic', // Multi-agent coordination
|
|
'review-army-consensus': 'periodic', // Multi-specialist agreement
|
|
'review-army-simplification': 'periodic', // Advisory lens quality benchmark
|
|
'review-army-simplification-precision': 'periodic', // False-flag noise benchmark
|
|
|
|
// Office Hours
|
|
'office-hours-spec-review': 'gate',
|
|
// Brain-writeback E2E — periodic per cost (claude -p) + non-deterministic
|
|
// (model interprets the gbrain instruction). Matches nearby
|
|
// setup-gbrain-path4-* tier classification.
|
|
'office-hours-brain-writeback': 'periodic',
|
|
// GBrain CLI round-trip — periodic per Voyage embedding cost (~$0.001/run)
|
|
// and external-API-dependency (skips cleanly if VOYAGE_API_KEY unset).
|
|
'gbrain-roundtrip-local': 'periodic',
|
|
'office-hours-forcing-energy': 'periodic', // D2a demotion 2026-08: posture score, periodic-grade signal (sibling precedent at office-hours-tone)
|
|
// 'office-hours-builder-wildness' retiered to periodic in v1.32 contributor
|
|
// wave: this is an LLM-judge creativity score (axis_a ≥4 on a "wildness"
|
|
// posture). Per CLAUDE.md tier-classification rules, non-deterministic
|
|
// quality benchmarks belong in periodic, not gate. The wave's +21-line
|
|
// CJK preamble cascade (#1205) pushed the score from 5/5 → 3/3 on the
|
|
// same /office-hours BUILDER prompt — same model, same fixture — proving
|
|
// the bar is sensitive to preamble-byte changes that have nothing to do
|
|
// with the test's intent (creativity, not preamble compliance).
|
|
'office-hours-builder-wildness': 'periodic',
|
|
|
|
// Plan reviews — gate for cheap functional, periodic for Opus quality
|
|
'plan-ceo-review': 'periodic',
|
|
'plan-ceo-review-selective': 'periodic',
|
|
'plan-ceo-review-benefits': 'gate',
|
|
'plan-ceo-review-expansion-energy': 'periodic', // Demoted from gate (2026-08 audit): Opus generator + subjective 2-axis >=4/5 LLM-judge threshold in the merge lane — the exact class siblings were demoted for (a +21-line preamble change once flipped the score). CLAUDE.md's own rule: Opus model test -> periodic.
|
|
'plan-eng-review': 'periodic',
|
|
'plan-eng-review-artifact': 'periodic',
|
|
'plan-eng-coverage-audit': 'gate',
|
|
'plan-review-report': 'gate',
|
|
|
|
// Plan-mode handshake. plan-ceo/plan-devex ask-first reliably (gate-tier);
|
|
// plan-eng/plan-design run a long explore/audit before their first
|
|
// AskUserQuestion, so whether they reach a terminal outcome within the 300s
|
|
// budget hinges on stochastic ask-first compliance (~50-67%/run measured).
|
|
// Per the "non-deterministic -> periodic" tiering rule they are periodic:
|
|
// the hardened ask-first gate + the collapsed-form detector lifted them from
|
|
// always-failing to mostly-passing, but they are not deterministic gates.
|
|
'plan-ceo-review-plan-mode': 'gate',
|
|
'plan-eng-review-plan-mode': 'periodic',
|
|
'plan-design-review-plan-mode': 'periodic',
|
|
'plan-devex-review-plan-mode': 'gate',
|
|
'plan-mode-no-op': 'gate',
|
|
// v1.21+ auto-mode regression tests
|
|
'office-hours-auto-mode': 'gate',
|
|
'auto-decide-preserved': 'periodic',
|
|
'conductor-prose': 'periodic',
|
|
|
|
// Real-PTY E2E batch — tier classification:
|
|
// gate: cheap, deterministic, run on every PR
|
|
// periodic: long-running or expensive (>$3/run), run weekly
|
|
'auq-format-gate': 'gate', // ~$0.50/run, SDK capture, single skill probe
|
|
'plan-ceo-mode-routing': 'periodic', // ~$3/run, deep navigation through 8-12 prior AskUserQuestions
|
|
'plan-design-with-ui-scope': 'gate', // ~$0.80/run
|
|
'ship-idempotency-pty': 'periodic', // ~$3/run, real /ship in plan mode
|
|
'tpa-present': 'gate', // consent/credential safety guardrail; deterministic shims + grep asserts
|
|
'tpa-absent-linux': 'gate', // consent/credential safety guardrail; deterministic shims + grep asserts
|
|
'tpa-broken': 'gate', // consent/credential safety guardrail; deterministic shims + grep asserts
|
|
'tpa-absent-darwin': 'gate', // consent/credential safety guardrail; deterministic shims + grep asserts
|
|
'tpa-apple-ban': 'gate', // consent/credential safety guardrail; deterministic shims + grep asserts
|
|
|
|
'ship-section-loading': 'periodic', // ~$3/run, real /ship; asserts section reads
|
|
'plan-ceo-section-loading': 'periodic', // ~$3-5/run, real /plan-ceo-review; asserts section read
|
|
'carve-section-loading': 'periodic', // ~$1-2/skill, data-driven; GSTACK_CARVE_SKILL scopes to one
|
|
'autoplan-chain-pty': 'periodic', // ~$8/run, all 3 phases sequential
|
|
|
|
// Per-finding count + review-report-at-bottom — periodic because each
|
|
// run drives a full skill end-to-end (~25 min, ~$5/run). Sequential
|
|
// execution during calibration; concurrent opt-in only after measured
|
|
// comparison agrees (plan §D15).
|
|
'plan-ceo-finding-count': 'periodic',
|
|
'plan-eng-finding-count': 'periodic',
|
|
'plan-design-finding-count': 'periodic',
|
|
'plan-devex-finding-count': 'periodic',
|
|
'plan-eng-finding-floor': 'periodic', // stochastic ask-first (see plan-mode-handshake note); periodic
|
|
'plan-ceo-finding-floor': 'gate',
|
|
'plan-design-finding-floor': 'periodic', // stochastic ask-first (see plan-mode-handshake note); periodic
|
|
'plan-devex-finding-floor': 'gate',
|
|
'plan-eng-multi-finding-batching': 'periodic',
|
|
'plan-ceo-split-overflow': 'periodic',
|
|
|
|
// Privacy gate for gstack-brain-sync — periodic (non-deterministic LLM call,
|
|
// costs ~$0.30-$0.50 per run, not needed on every commit)
|
|
'brain-privacy-gate': 'periodic',
|
|
|
|
// /setup-gbrain Path 4 (Remote MCP) — periodic-tier. The stub HTTP
|
|
// server is deterministic but the model's interpretation of "follow
|
|
// Path 4 only" is not — assertions on which steps the model ran are
|
|
// flaky. The deterministic gate-tier coverage for Path 4 lives in
|
|
// test/setup-gbrain-path4-structure.test.ts (free, <200ms). These
|
|
// E2E tests stay available for on-demand verification of the live
|
|
// model's behavior against a stub MCP server.
|
|
'setup-gbrain-remote': 'periodic',
|
|
'setup-gbrain-bad-token': 'periodic',
|
|
'setup-gbrain-path4-local-pglite': 'periodic',
|
|
|
|
// AskUserQuestion format regression — periodic (Opus 4.7 non-deterministic benchmark)
|
|
'plan-ceo-review-format-mode': 'periodic',
|
|
'plan-ceo-review-format-approach': 'periodic',
|
|
'plan-eng-review-format-coverage': 'periodic',
|
|
'plan-eng-review-format-kind': 'periodic',
|
|
|
|
// Office-hours Phase 4 silent-auto-decide regression — periodic (Phase 4
|
|
// requires the agent to invent 2-3 architectures, more open-ended than the
|
|
// 4 plan-format cases above). Reclassify to gate if it turns out stable.
|
|
'office-hours-phase4-fork': 'periodic',
|
|
// judgeRecommendation rubric sanity (fixture-based, ~$0.04/run via Haiku)
|
|
'llm-judge-recommendation': 'periodic',
|
|
|
|
// v1.7.0.0 Pros/Cons format — cadence + negative-escape evals (all periodic)
|
|
'plan-ceo-review-prosons-cadence': 'periodic',
|
|
'plan-review-prosons-format': 'periodic',
|
|
'plan-review-prosons-hardstop-neg': 'periodic',
|
|
'plan-review-prosons-neutral-neg': 'periodic',
|
|
|
|
// CT3 expanded coverage — non-plan-review skills inheriting Pros/Cons (all periodic)
|
|
|
|
// /plan-tune — gate (core v1 DX promise: plain-English intent routing)
|
|
'plan-tune-inspect': 'gate',
|
|
|
|
// /plan-tune cathedral (T16 per D12 — all gate)
|
|
'plan-tune-hook-capture': 'gate',
|
|
'plan-tune-enforcement': 'gate',
|
|
'plan-tune-annotation': 'gate',
|
|
'plan-tune-codex-import': 'gate',
|
|
'plan-tune-dream-cycle': 'gate',
|
|
|
|
// Codex offering verification
|
|
'codex-offered-office-hours': 'gate',
|
|
'codex-offered-ceo-review': 'gate',
|
|
'codex-offered-design-review': 'gate',
|
|
'codex-offered-eng-review': 'gate',
|
|
|
|
// Session Intelligence — gate for data flow, periodic for agent integration
|
|
'timeline-event-flow': 'gate', // Binary data flow (no LLM needed)
|
|
'context-recovery-artifacts': 'gate', // Preamble reads seeded artifacts
|
|
'context-save-writes-file': 'gate', // /context-save writes a file
|
|
'context-restore-loads-latest': 'gate', // Cross-branch newest-by-filename restore
|
|
|
|
// Context skills live-fire — periodic (each test spawns claude -p, ~$0.20-$0.40)
|
|
'context-save-routing': 'periodic', // Proves /context-save routes via Skill tool
|
|
'context-save-then-restore-roundtrip': 'periodic', // Full cycle in one session
|
|
'context-restore-fragment-match': 'periodic', // /context-restore <fragment>
|
|
'context-restore-empty-state': 'periodic', // Graceful zero-saves message
|
|
'context-restore-list-delegates': 'periodic', // /context-restore list redirect
|
|
'context-restore-legacy-compat': 'periodic', // Pre-rename files still load
|
|
'context-save-list-current-branch': 'periodic', // Default branch filter
|
|
'context-save-list-all-branches': 'periodic', // --all flag
|
|
|
|
// Ship — gate (end-to-end ship path)
|
|
'ship-base-branch': 'gate',
|
|
'ship-local-workflow': 'gate',
|
|
'ship-coverage-audit': 'gate',
|
|
'ship-triage': 'gate',
|
|
'ship-docsync': 'gate',
|
|
'docsync-spawned': 'gate', // #2733 JSON-contract-through-a-firing-gate proof (deterministic safety)
|
|
// (merge note: main's side also re-added ship-plan-completion /
|
|
// ship-plan-verification here — phantom keys with no declaring test,
|
|
// deleted by the census-integrity commit; the reverse invariant in
|
|
// test/touchfiles.test.ts now fails the suite if they come back.)
|
|
|
|
// Retro — gate for cheap branch detection, periodic for full Opus retro
|
|
'retro': 'periodic',
|
|
'retro-base-branch': 'gate',
|
|
|
|
// CSO — gate for security guardrails, periodic for quality
|
|
'cso-full-audit': 'periodic', // D2a demotion 2026-08: 250s/$0.57 full audit; cso targeted tests stay gate
|
|
'cso-diff-mode': 'gate',
|
|
'cso-infra-scope': 'periodic',
|
|
|
|
// Learnings — gate (functional guardrail: seeded learnings must appear)
|
|
'learnings-show': 'gate',
|
|
|
|
// Document-release — gate (CHANGELOG guardrail)
|
|
'document-release': 'gate',
|
|
|
|
// Codex — periodic (Opus, requires codex CLI)
|
|
'codex-review': 'periodic',
|
|
|
|
// Multi-AI — periodic (require external CLIs)
|
|
'codex-discover-skill': 'periodic',
|
|
'codex-review-findings': 'periodic',
|
|
'codex-sol-scope-termination': 'periodic',
|
|
'gemini-smoke': 'periodic',
|
|
|
|
// Design — gate for cheap functional, periodic for Opus/quality
|
|
'design-consultation-core': 'periodic',
|
|
'design-consultation-existing': 'periodic',
|
|
'design-consultation-research': 'periodic', // D2a demotion 2026-08: the two most expensive gate tests ($0.91/304s)
|
|
'design-consultation-preview': 'periodic', // D2a demotion 2026-08 ($0.89/481s)
|
|
'plan-design-review-no-ui-scope': 'gate',
|
|
'design-review-fix': 'periodic',
|
|
'design-review-detector-shim': 'gate', // deterministic sentinels from the fake engine (source mode on a diff)
|
|
'design-review-detector-shim-dom': 'gate', // same shim, DOM mode through the browse binary's dump; self-skips when the binary is absent
|
|
'design-html-slop-gate': 'periodic', // one-pass gate behavior is a judgment call on a fake engine's fixed output
|
|
|
|
// /diagram — triplet is deterministic functional (gstack-render falls back
|
|
// to the browse daemon, so CI runs it); judge is a quality benchmark
|
|
'diagram-triplet': 'gate',
|
|
'diagram-authoring-quality': 'periodic',
|
|
|
|
// gstack-upgrade
|
|
'gstack-upgrade-happy-path': 'gate',
|
|
|
|
// Deploy skills
|
|
'land-and-deploy-workflow': 'gate',
|
|
'land-and-deploy-first-run': 'gate',
|
|
'land-and-deploy-review-gate': 'gate',
|
|
'canary-workflow': 'gate',
|
|
'benchmark-workflow': 'gate',
|
|
'setup-deploy-workflow': 'gate',
|
|
|
|
|
|
// Autoplan — periodic (not yet implemented)
|
|
'autoplan-dual-voice': 'periodic',
|
|
|
|
// Multi-provider benchmark — periodic (requires external CLIs + auth, paid)
|
|
'benchmark-providers-live': 'periodic',
|
|
|
|
// Browser-skills Phase 2a — skillify keys gate (D1/D3 contracts must not
|
|
// silently break). The two scrape keys are periodic: /scrape is Aside-first
|
|
// and its fallback no longer prescribes the `$B skill list` / `skill run`
|
|
// match + prototype flow the tests assert, so a pass rides on prompt
|
|
// compliance (non-deterministic). Flip back to gate when scrape's fallback
|
|
// carries the browser-skills flow again.
|
|
'scrape-match-path': 'periodic',
|
|
'scrape-prototype-path': 'periodic',
|
|
'skillify-happy-path': 'gate',
|
|
'skillify-provenance-refusal': 'gate',
|
|
'skillify-approval-reject': 'gate',
|
|
|
|
// Skill routing — periodic (LLM routing is non-deterministic)
|
|
'journey-ideation': 'periodic',
|
|
'journey-plan-eng': 'periodic',
|
|
'journey-debug': 'periodic',
|
|
'journey-qa': 'periodic',
|
|
'journey-code-review': 'periodic',
|
|
'journey-ship': 'periodic',
|
|
'journey-docs': 'periodic',
|
|
'journey-retro': 'periodic',
|
|
'journey-design-system': 'periodic',
|
|
'journey-visual-qa': 'periodic',
|
|
|
|
// Opus 4.7 overlay evals — periodic (non-deterministic LLM behavior + Opus cost)
|
|
'fanout-arm-overlay-on': 'periodic',
|
|
'fanout-arm-overlay-off': 'periodic',
|
|
|
|
// Overlay efficacy harness (SDK, paid) — periodic only
|
|
'overlay-harness-opus-4-7-fanout-toy': 'periodic',
|
|
'overlay-harness-opus-4-7-fanout-realistic': 'periodic',
|
|
|
|
// /ios-qa daemon + codegen. Demoted gate -> periodic (2026-08 audit): the
|
|
// gate declaration was never executable in CI — the file sits in
|
|
// PERIODIC_CI_EXCLUDE ("not a CI runner capability"), but that exclusion
|
|
// only applies at tier=periodic, so the gate lane planned a HOLLOW shard
|
|
// on every Linux PR. Periodic keeps it in the weekly census on capable
|
|
// hosts; re-promote if a macOS runner lands (flagged decision in the
|
|
// test-infra overhaul plan).
|
|
'ios-qa-e2e': 'periodic',
|
|
// Swift toolchain only, no device required, but heavier than TS unit tests.
|
|
'ios-qa-swift-build': 'periodic',
|
|
// Requires a real connected + paired iPhone. Manual-trigger only.
|
|
'ios-qa-device': 'periodic',
|
|
// /spec end-to-end PTY pipeline (paid, non-deterministic — periodic-tier).
|
|
'spec-execute': 'periodic',
|
|
|
|
// WS2 arm benchmark — periodic: full build-shaped agentic workflows, paid,
|
|
// non-deterministic by construction (research instrument, not a gate).
|
|
'arm-benchmark-native-overbuild': 'periodic',
|
|
'arm-benchmark-crud-endpoint': 'periodic',
|
|
'arm-benchmark-bugfix-decoys': 'periodic',
|
|
};
|
|
|
|
/**
|
|
* LLM-judge test touchfiles — keyed by test description string.
|
|
*/
|
|
export const LLM_JUDGE_TOUCHFILES: Record<string, string[]> = {
|
|
'command reference table': ['browse/sections/**', 'SKILL.md', 'SKILL.md.tmpl', 'browse/src/commands.ts', 'test/skill-llm-eval.test.ts'],
|
|
'snapshot flags reference': ['browse/sections/**', 'SKILL.md', 'SKILL.md.tmpl', 'browse/src/snapshot.ts', 'test/skill-llm-eval.test.ts'],
|
|
'browse/SKILL.md reference': ['browse/sections/**', 'browse/SKILL.md', 'browse/SKILL.md.tmpl', 'browse/src/**', 'test/skill-llm-eval.test.ts'],
|
|
'setup block': ['browse/SKILL.md', 'browse/SKILL.md.tmpl', 'scripts/resolvers/aside.ts', 'scripts/resolvers/browse.ts', 'test/skill-llm-eval.test.ts'],
|
|
'regression vs baseline': ['browse/sections/**', 'SKILL.md', 'SKILL.md.tmpl', 'browse/src/commands.ts', 'test/fixtures/eval-baselines.json', 'test/skill-llm-eval.test.ts'],
|
|
'qa/SKILL.md workflow': ['qa/sections/**', 'qa/SKILL.md', 'qa/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'qa/SKILL.md health rubric': ['qa/sections/**', 'qa/SKILL.md', 'qa/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'qa/SKILL.md anti-refusal': ['qa/sections/**', 'qa/SKILL.md', 'qa/SKILL.md.tmpl', 'qa-only/SKILL.md', 'qa-only/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'cross-skill greptile consistency': ['review/SKILL.md', 'review/SKILL.md.tmpl', 'ship/SKILL.md', 'ship/SKILL.md.tmpl', 'review/greptile-triage.md', 'retro/SKILL.md', 'retro/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'baseline score pinning': ['browse/sections/**', 'SKILL.md', 'SKILL.md.tmpl', 'test/fixtures/eval-baselines.json', 'test/skill-llm-eval.test.ts'],
|
|
|
|
// Ship & Release
|
|
'ship/SKILL.md workflow': ['ship/SKILL.md', 'ship/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'document-release/SKILL.md workflow': ['document-release/SKILL.md', 'document-release/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
|
|
// Plan Reviews
|
|
'plan-ceo-review/SKILL.md modes': ['plan-ceo-review/SKILL.md', 'plan-ceo-review/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'plan-eng-review/SKILL.md sections': ['plan-eng-review/SKILL.md', 'plan-eng-review/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
|
|
// /spec authored-spec quality (paid LLM-judge — periodic-tier).
|
|
'plan-design-review/SKILL.md passes': ['plan-design-review/SKILL.md', 'plan-design-review/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
|
|
// Design skills
|
|
'design-review/SKILL.md fix loop': ['design-review/SKILL.md', 'design-review/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'design-consultation/SKILL.md research': ['design-consultation/SKILL.md', 'design-consultation/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
|
|
// Deploy skills
|
|
'land-and-deploy/SKILL.md workflow': ['land-and-deploy/SKILL.md', 'land-and-deploy/SKILL.md.tmpl', 'land-and-deploy/sections/**', 'test/skill-llm-eval.test.ts'],
|
|
'canary/SKILL.md monitoring loop': ['canary/SKILL.md', 'canary/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'benchmark/SKILL.md perf collection': ['benchmark/SKILL.md', 'benchmark/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'setup-deploy/SKILL.md platform setup': ['setup-deploy/SKILL.md', 'setup-deploy/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
|
|
// Other skills
|
|
'retro/SKILL.md instructions': ['retro/sections/**', 'retro/SKILL.md', 'retro/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'qa-only/SKILL.md workflow': ['qa-only/SKILL.md', 'qa-only/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
'gstack-upgrade/SKILL.md upgrade flow': ['gstack-upgrade/SKILL.md', 'gstack-upgrade/SKILL.md.tmpl', 'test/skill-llm-eval.test.ts'],
|
|
|
|
// Voice directive
|
|
'voice directive tone': ['scripts/resolvers/preamble.ts', 'review/SKILL.md', 'review/SKILL.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/skill-llm-eval.test.ts'],
|
|
};
|
|
|
|
/**
|
|
* Changes to any of these files trigger ALL tests (both E2E and LLM-judge).
|
|
*
|
|
* Keep this list minimal — only files that genuinely affect every test.
|
|
* Scoped dependencies (gen-skill-docs, llm-judge, test-server, worktree,
|
|
* codex/gemini session runners) belong in individual test entries instead.
|
|
*/
|
|
export const GLOBAL_TOUCHFILES = [
|
|
'test/helpers/session-runner.ts', // All E2E tests use this runner
|
|
'test/helpers/hermetic-env.ts', // Changes every E2E child's environment
|
|
'test/helpers/eval-store.ts', // All E2E tests store results here
|
|
'test/helpers/test-selection.ts', // Selection logic itself — a bug here mis-selects every test
|
|
'test/helpers/touchfiles.ts', // The facade is executable selection-path code; an edit must run everything (it should never change, so the cost is ~zero)
|
|
'test/helpers/e2e-helpers.ts', // Shared harness every paid test imports (selection wiring, preflight, describeIfSelected) — an edit here changes every test's behavior
|
|
'test/helpers/paid-test-set.ts', // Paid-vs-free classification — an edit moves files between suites
|
|
'test/helpers/skill-fixture.ts', // SKILL.md fixture extraction — reshapes the skill content most E2E suites read
|
|
// NOTE: this file (touchfiles-data.ts) is deliberately NOT a global
|
|
// touchfile. Changes to it route through map-diff selection in
|
|
// test-selection.ts: the old git version is evaluated and the maps are
|
|
// diffed per key, so a data-only edit runs just the affected tests.
|
|
// Map-diff fails CLOSED — any error on that path still runs everything.
|
|
];
|