mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-08 04:11:18 +02:00
7fca42ad8b6c707b8a38f579f72bf3c4f7de6d85
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Garry Tan
|
7fca42ad8b |
v1.91.12.0 v1.91.12.0: audit fix wave, ~11-minute paid eval lanes, eval reliability policy (#2999)
* test: delete test-infrastructure dead code (G) - exit-propagation drives the runner's real strict verdict (BunTestOutputClassifier + strictTestExitCode); delete the unused shardRunLooksTruncated predicate. - delete skill-coverage-matrix registry + its gate (nothing reads it; the floor already iterates skillCensus()). - delete touchfiles-facade export-parity tests (Bun fails missing imports at link time) and the duplicated E2E_TIERS tier-value test. - delete brain-cache-spec TRANSPORT_DEFAULT_POLICY, SKILL_RUN_RETENTION_DAYS and the now-unused BrainTrustPolicy type with their literal tests. AUTOPLAN_PREFLIGHT_BUDGET_BYTES stays: skill-preflight-budget enforces it against real resolver output. - delete audit-compliance's JSDoc-comment grep. * test: replace product tests that fake the product with real-boundary tests (F) - design: serve.test.ts drove an inline mirror server; now two tests run the real serve() on an ephemeral port (reload confinement, submit exit 0). - setup-gbrain: rollback + voyage tests execute the template-extracted init blocks (3 sites) instead of drifted local bash copies. - terminal-agent: internalHandler source greps replaced by a behavioral /internal/grant + /internal/revoke auth matrix (no/wrong/valid token). - /health: server-security-surface and the server-auth / security-audit-r2 / sidebar-tabs source greps fold into one liveness-only check on the real body; the L4 sidecar wiring gets a behavioral /pty-inject-scan test. - delete tautologies (browser-manager onDisconnect, memory-command #12), ios swiftui tap fixture self-check, memory-ingest put_page grep, detach source greps, sidebar-agent absence pins, dead-CSS pins + the dead CSS, security-audit-r2 Task 1 + the test-only meta-commands re-export, duplicate generated-SKILL.md checks. - make-pdf coverage-gaps cases move into their owner test files. * test: delete tests of dead eval code (A) - A1: the retired Eng lexical oracle (evaluateEngSeedCoverage, isEngSeedDecisionAUQ), the completion-handoff detector and the retained corpus had no paid caller since v1.87.6; delete their 26 replay files, ~2.6k helper LOC and fixtures, and the dead blocks in 8 mixed files (live hasNativePlanTerminal / batching assertions stay). - A2: dead viewport approvers in autoplan-artifact-permission and their 11 replay files + fixtures; recorder/launcher cases stay. - A3: never-wired oracles and seeders (autoplan-phase-order, eng-finding-fixture, ceo-paired-fixture, design-ui-scope, plan-skill-completion, pty-current-screen, required-reads, transcript-section-logger); plan-seed-submission now decodes through the production createPtyScreen; section manifests name their actual guard. - A4: zero-reference helper exports, plus execGit and invokeAndObserve found by the reachability pass. - 52 fixtures orphaned by the deletions; touchfile and selection-table entries for every deleted path. * test: clean up the paid eval lane (B1-B4, B6, B7) - B1: delete paid files that assert nothing or cannot pass meaningfully: skill-llm-eval-spec and skill-e2e-spec-execute (test.todo), gemini-e2e (+ gemini-session-runner; no gemini CLI in CI), ship-idempotency (red since v1.63), the two opus-4-7 *-sonnet overlay wrappers, conductor-prose (+ its source-evaluation replay), codex-e2e-plan-format; drop their keys, scripts and census rows. - B2: skill-llm-eval grades browse/sections/command-list.md with one union judge that also carries the baseline score pin; regression-vs-baseline deleted (paid run: pass, c4/c4/a4). - B3: memory-pipeline, ios-qa, ios-qa-swift-build and plan-tune-cathedral make no model calls; renamed out of the paid glob so they run on every PR. Swift builds need GSTACK_TEST_SWIFT=1; device stub deleted. - B4: codex-e2e*, outside-voice, aside and ios-device cannot run in the CI image; excluded from the weekly lane with a tracked re-entry condition. - B6: fold opus-47's negative routing controls into skill-routing-e2e journey-negatives (paid run: 3/3 unrouted) and delete the file. - B7: delete the never-green brain-privacy-gate eval; a free gstack-skill-start test now proves consent precedes artifacts egress. * test: retire the finding-count cluster and trim its helpers (C) - C0/C1: the five never-green evals (skill-e2e-autoplan-chain and skill-e2e-plan-{ceo,eng,design,devex}-finding-count) failed on harness and budget, never on skill behavior; delete them, their touchfile/tier ids, AUTOPLAN_CHAIN_BUDGET and the dedicated eighth periodic slice (--slices 7). - C2: delete the helper groups whose only paid consumers were those files (11 modules), trim claude-pty-runner and eng-seeded-coverage to the paid closure, and delete the free replay tests whose assertions exercised only that dead code (89 files, 135 orphaned fixtures). Blocks that used dead code only as input for a live subject keep their assertions: the multiSelect default moved to plan-review-decisions, runner PTY tests use inline caller policies, and the timer-safe budget checks moved to eng-finding-retry-budget. - The eight production-touching files stay except ceo-current-decision-record (its template read only feeds the retired counter). - CARVE_GUARDS.autoplan is behavioral 'none'; TODOS records the lost chain and per-finding cadence coverage with their re-entry tests. * test: fold per-incident replay series into their detector owners (D) Twelve detector families move into one owner test each: 73 incident files become describe blocks in ceo-section-loading-fixture (stale-fill race), model-overlays, coverage-audit-evidence, autoplan-phase-observer, native-auto-decide, outside-voice-evidence, eng-first-review, plan-count-completion, plan-count-file-permission, ceo-mode-option, plan-scope-selection and plan-count-prerequisite. Each block keeps its original code and fixture, so every case still runs; only tests asserting the incident file's own touchfile registration are dropped (41). Touchfile lists that named an incident now name its owner. * test: start the plan-count history PTY on its readiness marker (H) The fake CLI prints a startup marker and the runner waits for it instead of the fixed 8 s startup sleep (8.6 s -> 0.9 s locally). eng-semantic-terminal's sleeping registration cases went with C; plan-count-timeout keeps the fixed wait because it asserts deadline behavior. * test: derive paid touchfiles from each eval's static closure (E) touchfiles.test.ts now checks, per key, that the paid file's static test/helpers and test/fixtures closure (plus fixture paths it names in string literals) is covered, and names the file, path, chain and key to fix when it is not. Free *.test.ts files are no longer touchfiles, so editing a free replay test stops selecting paid evals: 950 entries removed, 653 real closure paths added. The hand-copied inventories go: periodic-fixture-selection, fake-impeccable-touchfiles and 45 per-file selection examples. Selection for the sample edits (plan-eng-review template, claude-pty-runner, plan-count-fixture, gstack-config) loses no case under either profile. CONTRIBUTING documents the rule and its lower bound. * test: skip hollow tier shards and census judges in the paid planner (B5) A paid file is now skipped for a tier lane only when every E2E id it registers is known statically and none has that tier; ids come from the touchfile registrations and literal testName/*IfSelected arguments, so a comment or skill path that quotes another id cannot unschedule it, and computed names keep today's scheduling. --list and the manifest show each skip as "skipped: no E2E_TIERS id has tier <tier>". The weekly gate census drops the LLM judges (--skip-judges); they still run in the periodic census and PR gate lanes. Gate lane 52 -> 42 files, census 41; periodic 77 -> 69. * test: run seven paid evals on the current default capture model (B8) skill-e2e-{auq-matrix,plan-format,qa-bugs,retro,workflow} pinned claude-opus-4-7 and skill-e2e-office-hours plus -brain-writeback pinned claude-sonnet-4-6; none tests a historical model, so they now capture with resolveEvalModel('capture'), and the free harness tests that execute these registrations receive the same resolver. The paid re-pin run passed all of them. skill-e2e-{design,office-hours-phase4,plan-prosons,plan} keep claude-opus-4-7: six of their cases failed on the default model (three timeouts, a missing report file, a format miss and a posture score of 3), so per the plan's fallback they keep their pins with a TODOS entry. The pre-spend estimate and drop threshold are in docs/test-audit-2026-09.md. * test: guard the reduced suite against new test-of-test files - test/test-of-test-ratchet.test.ts records the 228 free tests that import only test/ code and fails on a new one, naming the owner test to extend instead; a stale baseline entry fails with the remove instruction. - test/helpers/resolve-repo-path.ts is the one specifier/literal resolver for the ratchet and the touchfile closure invariant, with its own unit tests. - CONTRIBUTING "Test tiers" describes the paid-failure workflow (fix, then one row in the detector's owner test) and the ratchet; TEST_PORTFOLIO gains the detector -> owner-test table and no longer claims an Autoplan chain eval. - TODOS: automatic exclusion policy for chronically red periodic files (P3), the deferred native-completion table collapse, the unused CEO payment seeder; the PTY readiness item is narrowed to the paid runner. - docs/test-audit-2026-09.md collects the triage, security mapping, inventories, selection proof, behavior-commit decisions and retained false positives. * v1.91.8.0 test: smaller suite, derived paid selection, retired never-green evals Release metadata for the test-reduction branch: VERSION 1.91.8.0 (1.91.7.0 is claimed by #2983), CHANGELOG with the measured before/after table and a contributor section, durations re-recorded on Ubicloud standard-16 (857 files, 0 failures), the agents digest, CONTRIBUTING's after-measurement row, the B8 fallback TODOS entry, and the after metrics, kept-vs-plan notes, B8 run and census estimate in docs/test-audit-2026-09.md. * fix(ubicloud): skip retrieval globs that match nothing instead of reporting a failed pull * test: pin DISABLE_AUTOUPDATER in hermetic env and capture corrupt-seed warning Both EVALS_HERMETIC branches of buildHermeticEnv now carry DISABLE_AUTOUPDATER=1 (the allowlist scrubbed the workflow's copy, so every PTY screen showed the updater's npm-prefix failure). Per-test overrides still win. The corrupt durations-seed test now captures its expected warning and restores the console spy. * style(cso): format lib/cso TypeScript with pinned Prettier Mechanical reformat only. Minified transpile output is byte-identical for 21 of 22 files; witness.ts differs only in three regex flag orders (/mi -> /im), which JavaScript canonicalizes. Source-text assertions over lib/cso now compare whitespace-insensitively with the same tokens. * fix(cso): import join for compiled-launcher assertion witnesses Compiled installs always take the non-Bun branch, which called an unimported join and threw before any runtime-tested assertion could be witnessed. The child command selection is now a pure, platform-aware function; a missing sibling launcher fails with its expected path. * fix(browse): make connect --supervise actually respawn a crashed server The supervisor respawned with a block-scoped env that no longer existed, so every attempt threw and the loop gave up after five tries. The headed env is now one pure helper used by connect and respawn, the loop is an injectable runHeadedSupervisor with behavioral tests, failures name the daemon log and relaunch command, and connect's usage advertises --supervise. * test: one finite PR world for the shared-libs fixture; name dual-voice probe evidence The shared-libs shim served 2 PRs for pulls?state=all and endless full pages for state=open. gh pr list, pulls?state=open|all|closed (per_page/page, short last page, direction) and search/issues now page one deterministic table: PR 7, 600 older open PRs, PR 42 and 3 closed PRs, so five 100-item open-metadata pages still leave older open PRs unchecked. The Contents API lists pinned directories (the captured attempt got 404 for contents/ and contents/src while files resolved, then fell back to a raw host), unknown endpoints return 404 instead of repo metadata, and the read-only detector is unchanged. Free tests cover view agreement, the budget bound, gh/curl agreement and the empty world. Dual-voice outside-voice failures now report probeToolUseId, probeMode and the canonical-match result with the reason the probe output was rejected. * feat: require a zero-error product typecheck and a test type-debt ratchet Adds tsconfig.json (strict) over product code, fixes its remaining 90 diagnostics (type-only, interface corrections, and explicit narrowing), and adds a typecheck job to the required free-tests aggregate running bun run typecheck, the test-code ratchet (identity -> count baseline, fails on new, repeated, or unlocked fixed diagnostics), and the lib/cso format check. Reuses fixes from #2447 where they still applied. * test: follow the headed env helper and the typecheck gate in source-shape checks * fix(test): pin the package.json change kind in shared-input selection tests computePaidCaseSelection read the version-only exemption from git even when changed files were injected, so the shared-input test failed on main and on version-only branches. The exemption is now an optional input; the test pins a real package.json change and covers the version-only case. * test: judge plan-count completion on structured evidence, not wording Replaying run 36385945043's two Design attempts showed the existing routes rejected correct endings: attempt 1 at the typed-completion path field ('- Reviewed plan written to …' is not a 'Plan written to' line), attempt 2 at the leading-fence veto (its final message opens with the dashboard). nativePlanTerminalPreconditions is the structural prefix of hasNativePlanTerminal (behavior unchanged). structuredPlanCompletion adds, inside the existing nativeSummary branch: a complete report (Design binding for Design), a completed review-log row for the expected skill appended during this attempt under the child's GSTACK_HOME/project slug (resolved with bin/gstack-slug) and stamped with the fixture commit, timed between the report/last answer (second resolution) and the final native message, a final message with stop_reason end_turn (now carried on public transcript messages), and no visible question or permission prompt. Timeout summaries add idleFor and lastTerminalCandidate. Terminal and throw captures copy the plan file and review-log rows into the artifact directory; copies are best-effort and recorded in evidence-copy.json. Free regressions: both captured Design endings (trimmed fixture with provenance; report, row and end_turn reconstructed and labelled), the negative controls, and real-PTY completion/timeout runs through the real review logger. * test: structural Design count boundary; TODO proposals are not findings Replaying run 36385945043 through the Design count predicates: routing, focus and learnings setup was not recognized as setup, Issue 1 was counted pre-review in both attempts (the boundary fired on it), and attempt 2 counted the Font TODO proposal as a finding (review=4 and review=5 for five issues). The paid caller now starts review at the first answered native decision that is not setup (recognized packet, or setup header/question ID), a completion handoff, artifact rendering or a TODO proposal (the review's Add to TODOS.md / Skip / Build it now menu). TODO proposals are recorded as administrative extra decisions. The replay asserts each counted call: both attempts review=5 (Issues 1-5). isDesignCountFirstReview and its controls are unchanged. * test: CEO classifier throws name the question and matched predicates Replaying run 36385945043's FAN-1 and ERR-1 throws (ledger rows reconstructed from rendered diffs) through ceoPaymentFinding: the email obligation's row, subject, option and proposal predicates pass and the ELI10 explanation-defect predicate fails first ('lets that exception fly out', 'the error bubbles up'). Binding the defect to the named ledger row instead (the planned fix) was tried and reverted: scoped to the email seed it flips 30+ existing cf74 still-rejects replays, which require a vocabulary-free, ledger-bound email question to earn credit only through a complete saved comparison. With FAN-1's rendered currentDecision payload reconstructed, the recorded- decision path counts it, so the real saved plan (not uploaded) must have differed; failure artifacts now retain it. The classifier stays fail-closed and unchanged. Its throw now prints the header, the first 200 question characters and each obligation's predicate results. Free regressions with provenance and negative controls: an unrelated question, an email question whose row says it is already rescued, and a ledger ID whose row belongs to another seed. * chore: regenerate the test type-debt baseline on top of #2994 * fix(typecheck): strip the checkout root from ratchet diagnostic identities * fix(test): recognize ledger row-ID split candidates so collection stops at the last ACK Run 36385945043's split-overflow case asked all five candidate decisions by 8m55s, but the live candidate check required the question to open with "E1:" and every option to be a known disposition. The skill cited ledger row IDs ("D2.1 — R-E1: …") and offered "Hold, discuss first", so no candidate was recognized and the attempt ran the whole review (1302s). Identity now comes from the native header; the question must open with that candidate's ledger reference, name only that candidate, and offer exactly one include, defer and cut disposition. The selected answer must still be one of those three. The semantic evaluator and every existing negative control are unchanged; a trimmed capture from the run adds the positive case and four row-ID negative controls. * fix(test): stop the eng batching eval once its floor is proven The case's only verdict is reviewCount >= FLOOR (3). Run 36385945043 had three distinct acknowledged review decisions at 6m41s but kept answering until the ceiling (7) at 12m13s. The registration now passes the runner's existing isCollectionComplete stop once FLOOR non-setup, non-administrative review decisions are acknowledged; the floor check, ceiling, budget and counter are unchanged. A child-process registration test proves the stop predicate and that below-floor and timeout outcomes still fail. * test: add the non-blocking 'marathon' E2E tier Full start-to-finish flows move out of the blocking lanes. E2E_TIERS and E2ETier gain 'marathon'; describeE2ETier('marathon') is enabled only when EVALS_TIER=marathon, so the gate/PR and periodic lanes (and the gate census) never run those cases. The PR profile accepts marathon ids as scheduled elsewhere and defers them with their own reason, even on full fallback. * test: move the full office-hours workflow to marathon; add a periodic design-draft checkpoint The full startup workflow runs 1–3 real spec-review rounds (~280s each) and hit its 1200s capture in run 36385945043 at finalize. Review depth is the product's loop, so the case cannot fit a blocking lane without cutting rounds. It is now marathon tier with every assertion unchanged. skill-e2e-office-hours-design-draft.test.ts (periodic) runs the same fixed interview only through the Write that creates the design (269s in that run) and applies the full validator's design-draft checks, the required section reads and the launch/foreign-skill-read guards. validateOfficeHoursDesignDraft is extracted from validateOfficeHoursCompletion, which still applies it. Selection: office-hours-design-draft is registered periodic; the marathon-only file is already excluded from the gate and periodic plans by the B5 planner rule. Tier-alignment regexes and the valid-tier check accept 'marathon'. A type-only cast in plan-scope-selection.test.ts removes a diagnostic whose union print order made the ratchet identity unstable; baseline tightened. * test: supply the split-overflow fixture's HOLD SCOPE mode as a prerequisite The split actor always answered 0E's mode question with HOLD SCOPE. The skill skips that question on an explicit choice, so the fixture now states it and the attempt starts at the five candidate decisions (about 1.5 min earlier in run 36385945043). Candidates, actor policy, floor and semantic evaluation are unchanged; the fixture test pins the supplied choice. * test: start the eng batching eval with its setup prerequisites supplied Routing setup and cross-project learnings (D1/D2 in run 36385945043) are never counted and are not what the case measures. The registration now uses the runner's existing preconfiguredReviewActor so the attempt starts at the review; engSetupAUQ still vetoes any late setup question. The registration test pins the option. * test: count the design-draft paid file and defer marathon ids in PR selection pins The discovered paid-file census grows by one (skill-e2e-office-hours-design-draft). Full-fallback PR selection defers every non-gate id; the shared-input pins now expect periodic and marathon ids there. * fix(review): resolve the judged revalidation, setup-authority, plan-gate and findings-record ambiguities The census review workflow judge scored clarity/actionability 3 on both attempts: smoke-clock limits appeared to forbid post-repair revalidation, the caller deadline was undefined, 'ask for setup' conflicted with the report-only browser rule, fallback-sourced HIGH discrepancies had no gate decision, and the Step 5.8 record omitted adversarial findings. * fix(office-hours): load the builder section for every builder-mode reply Both census builder-wildness attempts answered a direct request for adjacent unlocks without reading phase-2b-builder-brainstorm.md, whose trigger read as applying only to the generative questions. * fix(sync-gbrain): define Step 4 helper args and one atomic write path Both census read-ready attempts spent turns reading the helper source to resolve <user-args>, inspecting fixture internals kept inside the repo, and reconciling 'Read + Edit' with the tmp+mv atomic write, then hit max turns before the verdict. * refactor(evals): share the import-closure walker and add the E2E shard reuse identity sourceDependencyClosure moves from the workflow-judge adapter into scripts/eval-input-cache.ts unchanged, so judge keys stay byte-identical. scripts/e2e-shard-reuse.ts builds the consumed-input identity of one PR-lane E2E shard (test import closure, every registered case's touchfiles, globals, runner/workflow/setup actions, child env pins, CI image, Claude CLI) and fails closed on anything unknown. Marathon joins the always-fresh purposes. * feat(evals): ~12-minute blocking paid lanes and a non-blocking marathon lane - Planner budget mode (--slice-budget S --jobs J): recorded per-tier wall times pack into as many ~9-minute executors as the work needs; the plan records per-slice estimates and the CI job timeout (supervised worst case + 20 min). evals.yml and evals-periodic.yml derive matrix size and timeout-minutes from it; max-parallel covers every slice at once. - Case shards: plan/design/review-army/shared-libs(-paths) run one registered case per process (<file>#<case id>, exact name pattern, exactly one case). - Retry rule: a timed-out attempt is a verdict. Only files whose every case budget is CAPTURE tier or shorter keep one retry; walls shrink to match. - Marathon tier: positive selection, excluded from gate/periodic planners, run by the new evals-marathon.yml (weekly + dispatch, fresh, own report). - PR-lane E2E reuse of verified first-attempt passes on identical inputs; the report rejects reuse outside the fast PR profile. - Duration seed from census run 36385945043, per tier and per case shard. * docs: blocking lane budget, marathon lane, retry policy and E2E reuse * chore(typecheck): lock in two fixed test diagnostics * fix(ci): drop a duplicated env/jobs block in evals-marathon.yml * test(ship-docsync): shard the doc-sync lifecycle by case and drop the duplicate dispatch-only case ship-docsync ran the same fixture and prompt as ship-docsync-completion and asserted a subset of it. The file now runs one case per process, so its lane wall is its longest case instead of half the sum of thirteen. * fix(evals): plan CI-unrunnable cases as excluded entries, not empty case shards design-review-fix drives the Aside browser and registers test.skip on Linux runners, so its case shard executed zero cases and failed the exact-one-case check in proof census 36597762183 (eval-slices 6). CASE_CI_EXCLUDE (reason + tracking, beside PERIODIC_CI_EXCLUDE) now turns such cases into excluded manifest entries that --list and the manifest surface; every planned case shard still must execute exactly its case. * docs(todos): list the case-level Aside exclusion with the CI-unrunnable evals * fix(plan-ceo-review): restore experience-first expansion framing, require the mode handoff, skip pacing menus Census 36597762183: both mode-routing runs logged provenance and moved on without the mandated handoff chat; the EXPANSION run asked an unauthorized batch/narrow pacing menu instead of the first per-addition question; the expansion-energy proposals led with the spec because v1.87.6.0 dropped 'lead with the felt experience'. The HOLD review detector also rejected a decision whose grounding line named no plan file although the owned source Read binds it. * test(outside-plan-disabled): bind quoted prior-record values by their sentence, not phrase order The parent obeyed the off switch and twice named the seeded completed record as pre-existing, once with the quotation after its owner and once with slash separators; the order-specific stripper counted both as current completion. Timestamp, location, current-claim and value-match controls still reject. * test(outside-plan-disabled): compare named record timestamps as instants; negated authorship is not a current claim The repair rerun named the seeded record by its ISO second (2026-09-29T16:58:52Z vs .727Z) and said 'I did not write'; both were misread as a foreign timestamp and a current write. * test(ceo-section-loading): recognize an arrow-ordered stale-fill execution by event roles The census review traced the seeded race as 'R1 miss -> R1 store read (v1) -> W commit v2 -> W cache.delete -> W fulfills -> R1 cache.set(v1) -> R2 (begun after W) hits v1', but the in-flight gate only accepted race vocabulary or fixed sentence shapes. Order, actor, version and dismissal mutations still fail. * test(design-floor): answer the seed-declared all-seven 0D focus menu while it is pending The actor declares 'Design: review all seven dimensions', but its picker reused designReviewSetupAUQ, which only matches already-answered calls (and a narrower header/label set), so the pending D1 focus menu was never answered and the case waited out its 609 s deadline. The skill's Step 0D requires asking; the fixture now answers it. * test(ceo-mode-routing): accept the skill-mandated Note form and Recommendation reason as HOLD posture HOLD Defer/Keep briefs must use 'Note: options differ in kind' (preamble), but the answered-HOLD path demanded a Completeness score, rejected a one-line Net with a semicolon, and read posture only from ELI10. The rerun's brief applied HOLD SCOPE in its Recommendation reason. Revert the ineffective 'always'/'handoff chat' wording: two runs still skipped the mode handoff. * test(qa-bugs): keep claude-opus-4-7 after qa-b6-static stalled on the default model qa-b6-static timed out on claude-fable-5-1 in census 36597762183 and in one of two targeted reruns. Both times the stream stopped mid-message with no pending tool, right after the model found the disabled submit button, and stayed silent until the 300 s deadline. Per the B8 fallback, re-pin with a TODOS entry; budgets and retries are unchanged. A rerun on opus-4-7 passed (125 s, 5/5 detected). * test(evals): add E2E_KINDS, BEHAVIOR_WHY, EVAL_POLICY and CASE_QUARANTINE skeletons Every E2E_TIERS and LLM_JUDGE_TOUCHFILES key starts as 'rule'; BEHAVIOR_WHY and CASE_QUARANTINE start empty. EVAL_POLICY pre-registers the approved panel (3, majority 2), quarantine entry 0.95/10 and exit 0.97/10, 10% cap, 8-weekly-run expiry, Fisher drift alarm and one INFRA re-dispatch. * test(evals): add trial records, panelVerdict, expectContract and trial-outcomes JSONL EvalTestEntry gains case_id, kind, trial, panel, failure_class and policy_version, stamped from the runner's TRIAL_ENV on isolated trial shards. panelVerdict() is the single verdict function (INCOMPLETE on missing or duplicate trials, contract veto at any count, quarantine hard-break rule, INFRA/INCOMPLETE machine classification). expectContract() records failure_class 'contract' on the collector entry and a sidecar before throwing. trial-outcomes JSONL has a fail-closed writer and a data-only reader. * test(evals): pin the fail-closed rule-shard gate through the real --report path Synthetic slice artifacts for rule fail, timeout, missing slice, unreported entry, hollow, never-started, collector failure and wrong-slice reports all exit red before the panel-verdict gate change lands. * test(evals): retire every paid automatic retry Paid evals never retry (approved 2026-09-29): delete SHORT_CASE_RETRY_FILES and retriesWithinCaseCap, drop the retry fields from the registered wall rows (walls now cover one run plus reserve), make retriesForFiles return 0, pass --retry 0 explicitly, and drop --retry 1 from the package.json paid scripts. Add the eval:pass-rates alias. Tests that pinned the old retry allowance are updated as a policy change; review-finalization-budget now proves late-result recording under the production zero-retry arguments. * test(llm-judge): sample every judge as a pre-registered 3-sample panel Each of the 24 skill-llm-eval judges now draws EVAL_POLICY.judge.samples independent samples of the same prompt concurrently inside the unchanged JUDGE_MS budget. Numeric dimensions gate on the per-dimension panel mean against the unchanged threshold; booleans (would_browse, consistent) on a strict majority. An erroring sample fails the whole panel and is never resampled; a refusal is an unscored panel only when every sample refused. callJudge's 429 backoff stays: it is transport before any model output. The workflow-judge cache stores and validates only complete panels, and its identity now records the panel and zero file retries. Harness tests that pinned one provider call per case now pin the panel size. * test(evals): classify every live case and re-select a case when its kind changes E2E_KINDS: rule by default (191 E2E ids), 22 behavior cases whose verdict is a live model choice with an acceptable sub-100% per-trial rate, each with a BEHAVIOR_WHY tolerance, and 25 judge entries (the 24 workflow judges plus the fixed-fixture llm-judge-recommendation rubric check). Contract-shaped cases (ask-before-decide, plan-mode no-writes, mandated steps, secrets, the batching floor) stay rule. Behavior requires a known literal registration and an exact Bun test name so the case runs as its own trial shard. Map-diff selection now diffs E2E_KINDS and BEHAVIOR_WHY per key, and a base revision without them selects every key, so a kind flip runs the panel it introduces. test/eval-kinds.test.ts enforces coverage, tolerances, isolatability and the reviewed counts, printing the literal to add. * feat(evals): per-case pass rates with Wilson intervals, identity series and quarantine policy scripts/eval-flake-rank.ts becomes eval:pass-rates (eval:flake-rank stays an alias, and the legacy aggregate stays exported). It reads eval-store's trial-outcomes JSONL from the last N completed evals-periodic runs on this branch and main (gh, downloading only the trial-outcomes artifact, cached and size-capped, parsed as data), plus local eval dirs, and prints per-case per-trial pass rates with 95% Wilson intervals. A series is a case's own touchfiles minus GLOBAL_TOUCHFILES (caseSeriesIdentities, for the report job to stamp), per model, CLI version and policy version. Labels: INCONCLUSIVE, BROKEN, FLAKY, FAILING, PASSING. --backfill imports legacy slice artifacts as pre-policy trials (first attempt only, attributed by registry id, never guessed) for display only. --gate fails with ACTION REQUIRED on post-policy evidence only: drift below the quarantine entry rule, a rule case behaving like behavior, a one-sided Fisher drop against the previous identity (Holm-controlled), and quarantine entries that met their exit rule, expired after 8 weekly runs, broke the 10% tier cap or are invalid. CASE_QUARANTINE entries now carry a failureClass (detector, harness or model-latency); a product defect has no class and is never quarantined. The policy test pins EVAL_POLICY's approved constants. * feat(eval-pass-rates): attribute legacy records by the exact slug of their display name * ci(image): pin Claude Code 2.1.284 so the eval model is recognized 2.1.251 logs [claude-code:unrecognized_model] for claude-fable-5-1, the eval capture/judge default. 2.1.284 does not. The gate PTY smoke subset (plan-ceo/plan-devex plan-mode, plan-mode-no-op) parses on the new TUI; plan-design-review-plan-mode passed at 293 s on 2.1.284 and timed out at 300 s on 2.1.251 on the same tree. * test(eng-batching): grade the floor once the review report is complete A completed GSTACK REVIEW REPORT ends the review, so the review-question count is final there. Run 36606688266 wrote its report at 1,248 s and closed the session at 1,318 s; the case now stops collection and applies the unchanged floor at the report instead of waiting out the session. No budget changes. * test(eng-batching): bind unsourced native briefs through the report's target Run 36606688266 asked ten separate native review questions (D1-D9 bound to ledger records R1-R9) and failed reviewCount=0 < FLOOR=3: its briefs named the plan by title instead of citing PLAN.md, its report declared 'Review target (fixed): PLAN.md' under '# Engineering review: <plan>', and it kept an unfenced copy of the plan's own H1. The named-source route now accepts those spellings and non-inline ledger briefs. The same replay rejects a foreign, mixed, duplicate or missing target, another plan's title or copied H1, a brief naming another plan or file, a mismatched saved brief, and re-asks. The run-36597762183 capture still counts 3. * fix(plan-design-review): treat a designer with no API key as unavailable Both proof runs (36597762183, 36606688266) printed DESIGN_READY, hit 'No OpenAI API key found' on the first $D variants call, then hand-built HTML/CSS wireframes, screenshots and a comparison board for ~195-245 s before the first review question; the second run timed out at 600 s. A failed first generation now takes the existing text-only path, and the skill forbids substituting hand-built mockups. * fix(deslop-shared-libs): read related sources together within the turn limit Run 36606688266's opportunity audit read sixteen sources one per turn and stopped at error_max_turns; the passing run 36597762183 read the same files in three batched commands. The skill now says turns are bounded and asks for parallel reads or one read-only command per step. * test(ceo-mode-routing): submit a mode review that scrolled past the viewport Run 36606688266 bundled routing, learnings and the mode choice into one native call. Its review panel was taller than the terminal, so the tab bar scrolled off, ceoModeSubmissionInput returned null for 240 s and HOLD SCOPE was never submitted ('no posture match'). With no bar on screen the viewport must still end at the focused Submit prompt, and the accumulated screen text supplies the one complete panel, authenticated exactly as before. Replay controls reject another mode, an unoffered answer, an altered question, a quoted panel, trailing output, a moved cursor and an answered or changed call. * docs(evals): document the pre-registered verdict policy, quarantine, pass-rate history and arithmetic AGENTS.md replaces the retry rule with the approved policy text (no retries; kind fixes trials; no added trials, samples or dispatches after a result; quarantine by CASE_QUARANTINE only; one INFRA/INCOMPLETE re-dispatch) and notes that a pre-registered fixed panel is not rejudging. CONTRIBUTING gains the kind rules, the judge panel, eval:pass-rates and an 'Add a paid eval' checklist. TESTING_INTERNALS describes verdicts, quarantine, history and the arithmetic, including the rule term: 1 trial vs 2-of-3 red rates at p = 0.99/0.95/0.90/0.70/0.30 and lane all-green probabilities for the current 191 rule / 22 behavior / 25 judge registry. * feat(evals): trial planner, slice exit split and panel-verdict report Planner: behavior and quarantined cases become panels of isolated trial shards (<file>#<id>~t<N>) bound by EVALS_SELECTION_JSON=[id] and the exact test name; the file shard excludes them by name. Trials of one case never share a slice, result slugs are unique, panels are validated whole, unknown registrations throw, and the planner prints a capacity preflight. Executor: each trial shard gets its TRIAL_ENV identity and a trial record (outcome, failure class, cause, cost); every shard writes a JUnit report. The slice exit now means execution completeness: a failed rule shard or a trial without a record reds the runner, a failed trial does not. Report: panelVerdict() decides every panel of the first run attempt (later attempts are reported, never replacing it); rule shards keep the unchanged fail-closed checks; collector records all count (no last-attempt wins); census runs enforce the quarantine cap and expiry. It writes collector-outcomes v2, trial-outcomes.jsonl (trials plus JUnit rule/judge cases), report-summary.md, and one headline + failure block with rerun commands, and flags INFRA/INCOMPLETE-only reds for the one re-dispatch. The fail-open suite gains the panel cases: behavior 1/3 red, 2/3 green with its failed trial shown, missing trial INCOMPLETE, contract at 2/3 red, quarantined 1/3 green, 0/3 and contract red, missing slice red, and a later attempt never replacing the first. * chore(evals): refresh paid duration seeds from proof runs 36597762183 and 36606688266 Both tiers, merged in run order (the later run wins). Notable: split-overflow 1332s -> 504s, section-loading 604s -> 342s, mode-routing 575s -> 444s; multi-finding-batching 734s -> 1318s (its red path in run 36606688266). * feat(evals): stamp trial series identities and fit panels to the live registry - scripts/eval-trial-series.ts stamps series_identity (eval-flake-rank's caseSeriesIdentities) on a report's trial-outcomes JSONL as its own step, keeping the history tool out of the paid runner's closure; TrialOutcomeRecord gains the optional series_identity field. - Slice-count plans let a registered trial spill into an ordinary lane when its siblings hold every long lane, so panels never share a runner. - Re-audited test-selection.ts (Stream B added the E2E_KINDS/BEHAVIOR_WHY map-diff; no new module loading) and repinned its hash. - Detach and release floors now count trial shards (66 periodic trials in 22 panels): periodic floor 33,821s, still under eval:bg:periodic's 67,380s. - Coordination fixtures supply the executor's trial records. * ci(evals): attempt-scoped artifacts, verdict-v2 PR comment, weekly pass-rate gate and one INFRA re-dispatch - Slice, census and marathon artifacts carry -a<run_attempt>; reports download them per artifact (no merge), so records never overwrite and a re-run never replaces the first attempt's verdict. - Planners pass --max-parallel for the capacity preflight (24/16 unchanged: the refreshed periodic plan needs 24 slices, the gate census 12). - PR comment: jq-only job reads collector-outcomes v2 (headline, sanitized failure block); the group_by(.name)|last recomputation is gone. - Reports stamp series identities, upload trial-outcomes-* for history, and shard logs upload always (a failed trial no longer reds its runner). - Weekly report: headline + failure block of both lanes in the issue body, the eval:pass-rates --gate step (fails closed without history), close the issue on a green run, and UC-E1: when every red is machine-classified INFRA/INCOMPLETE, one re-dispatch as a new run in its own concurrency group (redispatch_of), both runs reported. * feat(evals): planner-side whole-panel reuse and negative receipts The planner job restores this PR's receipt store once and ships a single filtered set with the plan: a pass or panel receipt with a same-or-newer FAIL for its input identity is dropped, and a panel receipt ships only as a whole PASS panel (re-verified with panelVerdict) from one run. Executors read only that set (no per-slice cache restore or save), so every trial of a panel sees the same receipts; a trial reuses its own record from the panel receipt, keeping a split PASS's failed trial. Trial identities drop the trial index (run-scoped) and bind the panel policy. Executed shards carry their input identity; the report turns a whole fresh PASS panel into a panel receipt and a FAIL panel or failed rule shard into a negative receipt, and marks a panel that mixes reused and fresh trials INCOMPLETE. The report job merges plan, slice and report receipts (newest per file) and saves one store per run. Also fixes two TS2352 casts in browse/test/dia-macos-qualification.test.ts whose diagnostic text drifted with program order (baseline locked, fix only). * feat(evals): --case/--trials local diagnosis and panels in local sharded runs bun run scripts/test-paid-shards.ts --case <id> [--trials N] runs N independent trials of one case through the CI panel runner (trial shards, TRIAL_ENV identity, name-pattern isolation) and prints its panelVerdict(); N defaults to the case's policy panel and CI never reads it. The local sharded path (test:gate:sharded, test:periodic:sharded) now plans the same trial shards and exclusions as CI and exits on execution completeness plus panel verdicts. * test(pty): grant an owned Create pane whose title row is cropped The targeted batching rerun on Claude Code 2.1.284 left its first report Write unanswered for 1,372 s and timed out: the viewport began at the pane's relative file row and rule, with the 'Create file' title cropped above, so the preview parser rejected the file row as foreign. That row must now resolve to the owned path and is skipped before the unchanged line-by-line preview match. Replay controls reject another file, another directory and an edited preview row. * fix(evals): tsx-safe generics in eval-flake-rank, legacy artifact names, no-retry wall docs * test(evals): record the read-only and detector-row invariants as contracts shared-libs-opportunity-judgment and review-design-lite are behavior cases: their recommendation and checklist judgments may vary, but the read-only invariant (commands, provider requests, fixture bytes, hooks, state) and the deterministic fake-engine detector rows are contracts. Both now go through expectContract, so any failure vetoes the panel. * test(judges): sample the recommendation rubric as a panel; never re-ask armJudge llm-judge-recommendation is a judge case: each fixture now draws a 3-sample judgePanel, gates reason_substance on the panel mean and the present/commits/has_because checks on a 2-of-3 majority, thresholds unchanged. armJudge no longer re-asks on a malformed verdict; it is a failed sample, as the judge policy requires. * test(evals): record a pre-turn API or CLI failure as infra recordE2E sets failure_class 'infra' on a failed session whose runner reports error_api, timeout_startup, error_output_stream or a non-zero CLI exit with zero turns and no assistant event. A model refusal, a timeout after model work, max turns, or an explicit caller pass/class keeps its ordinary classification. * test: pin every-record outcome counts and the twelve doc-sync callbacks * test(eng-batching): read the report target as a field, not a spelling The next targeted rerun (Claude Code 2.1.284) again asked eleven separate native questions and again counted zero: its briefs named no plan and its report declared '- **Review target (fixed):** `/abs/PLAN.md`' under '# Eng Review — PLAN.md: <plan>'. An unsourced brief now inherits the one current target field that names a PLAN.md file, whatever its list or emphasis markup; its ledger record still supplies the cited finding and must reproduce the brief exactly. A brief that names its plan must still match the report title. Replays of all three captures count 9, 9 and 3; controls reject a foreign, duplicate or missing target and an archived title. * fix(evals): --case list mode and name precheck; case-shard qa-callers; refresh batching and design-with-ui seeds * chore(release): v1.91.9.0 * test: settle the post-response composer before seeding; give the TPA recorder adapter its infra helper submitPlanSeed accepted a stale empty composer when the transcript recorded end_turn before the CLI repainted (late-repaint-typed-current fails 5/5 on the old helper, passes 5/5 now). The TPA recording fixture extracted recordE2E without isPreTurnInfraFailure, so every failed case threw before recording. * test(autoplan-dual-voice): unwrap Claude Code 2.1.284 subagent hand-back frames; accept read-only probe diagnostics; record before asserting Census run 36626737820: the native CEO report arrived framed and indented, so its INPUT line never matched, and the model's exact probe plus two variable echoes was not canonical. A column-zero line inside a frame, command substitution, backticks, redirects, assignments, CODEX_MODE echoes and output line-count mismatches stay rejected. The failure now records before asserting. * ci(image): keep Claude Code 2.1.251; test(ceo-mode-routing): keep HOLD's own deferrals in scope before assessing its rigor decision 2.1.284 enables per-turn effort for claude-fable-5-1: in gate census 36626737820, 66 of 84 sessions ran longer than on 2.1.251 (+20% session time, +32% thinking tokens) and 11 cases timed out on unchanged budgets. HOLD SCOPE's 0G step asks its own defer/keep menu; the actor answered it Defer and the assessment then judged that scope question as the rigor decision. The actor now answers that menu Keep and assesses the next one. * test: attribute quoted prior-record field lists, state the judge reason bound in its schema, move split-overflow to marathon Census 36629958451 reds: - outside-plan-disabled-no-fallback: the model quoted the pre-existing record as a parenthesized field list with its exact timestamp; attribution now requires that exact timestamp and the record's own field values. - plan-devex-peer-comparison-classification: the judge correctly returned missing but wrote a 1069-character reason, voiding the judgment; structured outputs cannot enforce maxLength, so the bound is stated on the field. - plan-ceo-split-overflow ran 504-1188 s as one PTY flow and set the periodic lane's wall clock; it now runs weekly in the marathon lane. * test: supply holdDeferKeepIndex to the CEO routing mocks and follow split-overflow into the marathon lane The registered-callback fixtures mock ceo-mode-option and lacked the new export; the split fixtures asserted the periodic tier; the registered-budget check looked for split-overflow only in the periodic manifest. * fix(qa): checkpoint receipts print the report link for their exploration file qa-functional-webhook-report failed in two of three censuses because the report linked .qa-evidence/NNN capture folders as "checkpoints" and never linked exploration-NNN.json. The checkpoint receipt now prints link: [checkpoint NNN](exploration-NNN.json), and the functional report template says capture folders are not checkpoints. * docs: final census numbers in the v1.91.9.0 entry; file the paid-eval follow-ups * ci(evals): name the PR-comment loop's unused fields so shellcheck passes (SC2034) * fix(plan-ceo-review): tighten expansion pacing wording to fit the skeleton cap after the main merge The merged skeleton measured 80,166 bytes against its unchanged 80,150 cap. Same instructions: ask separately for each addition, in turn, with no pacing menu; lead each proposal with the felt experience, then shape, effort and impact. * fix(eval-pass-rates): match trial-outcome files by basename so Windows backslash paths are read * fix(evals): repair proof-run reds in design-consultation, document-release, design and QA fixtures - design-consultation Phase 1 asks one brief that confirms context and decides research; the confirm-only first question scored substance 2. - document-release defines ship-owned inputs, exact steps and the JSON result, and drops stale spawned-from-/ship text (judge actionability 3.67 -> 4/4/4). - plan-design-with-ui accepts the Step 0D focus menu the same way the shared picker does ("focus on specific ones?"). - plan-design-review plan-mode saves in three Edits instead of one final Write. - QA functional annotations ask for the full 40-character revision. - Outside-disabled attribution judges quoted prior-record data by its exact timestamp or a dated, pre-existing-record sentence; four captured phrasings replay clean and current claims still fail. - --case can select autoplan-dual-voice by its literal test name. * test(design): revert the three-Edit plan-mode flow A focused paid run still timed out at 300 s: the first three passes alone took 150 s of thinking. The case stays a named timeout red rather than cutting review depth. * test: accept 'review mode = X' auto-decide declarations and parenthetical scope exclusions in the shared-libs actor auto-decide-preserved: the product auto-decided HOLD SCOPE and said "Decision: review mode = HOLD SCOPE"; the grammar knew only "is" and ":". shared-libs-plan-callers: the recommended option said "(no hardening)" and the actor read "hardening" as an expansion. Both replay the captured text, keep negative controls, and passed focused paid runs. * fix(review): pass Review Army checklists by path, run research alongside dispatch, always probe the design detector; state review-log invocation and statuses in the caller fixture - review-army-perf-n-plus-one: the parent copied full checklists into agent prompts and ran web research before dispatch (290 s on a 12-line diff); 212 s now. - review-design-lite: 5 of 6 captured trials reported the detector absent without probing; the probe is mandatory and its first line is reported, and the contract credits only fake-engine rule ids the checklist never names. - review-exploratory-small-cli: the fixture never gave review-log's direct invocation or status vocabulary; the model ran it through bun and wrote status "blocked". The prompt states both and the validator rejects out-of-vocabulary review statuses. Each case passed a focused paid run after repair. * docs(changelog): proof-run product fixes * fix(ship): always run the design-lite detector probe; test(shared-libs): credit a failed first file view and deferred-reuse Skip wording - /ship design-lite: the probe is mandatory and any non-ready first line is stated, matching /review (5 of 6 captured /review trials had skipped it). - shared-libs-pr-coverage: the first PR 42 page-1 read printed only a jq error, so the one refetch is a legitimate recovery, charged to the same budget. - shared-libs-review-prior-coverage: the Skip option said a future review can "reuse it once snapshot coverage holds"; a conditional tail on the recorded decision is not product work. Captured-text regressions and negative controls. * fix(ship,qa,document-release): repair proof-run regressions and fixture gaps - ship-docsync-completion: yesterday's audit-scope result dropped the section's status, so /ship spliced one in; the section now opens with **Status:**. - ship-docsync-missing-asset: a missing section or old Ship-owned mode blocks before launch. - ship-docsync-late-result: the invocation record says prepare already saves the candidate selection (no extra Read; budget unchanged). - qa exploratory: await the method Reads before the first probe. - qa-callers fixture: quote the real review-log record template; allow the git log command plan-completion prescribes. - qa functional observer: a receipt caught mid-link(2) is checked at stop instead of failing with ENOENT (reproduced from CI). Each repaired case passed a focused paid run. * ci(image): pin Claude Code 2.1.284, the version users run Request-body capture shows both 2.1.251 and 2.1.284 send effort "high" to claude-fable-5-1; 2.1.284 adds the model's own profile. The slower 2.1.284 census was mostly API latency: its SDK-only judges were 25% slower too. Nine previously slow cases pass on 2.1.284 within unchanged budgets. * test: one owner per case id, a structural devex 0B setup rule, and correct design/gbrain actors - plan-design-review-plan-mode was registered by two files; the PTY smoke is now plan-design-review-plan-mode-smoke, and a registry test requires one owner per case in case-sharded files. - plan-devex-finding-floor: the template's 0B narrative-confirmation question is classified as setup structurally instead of timing out a Haiku assessor. - setup-gbrain-remote: the actor accepted 'skip' on the MCP-registration question the test asserts; it now accepts that question and declines others. - design-review-plugin-handoff: the fake engine cited a file absent from the fixture repo and index.html linked a missing styles.css. Captured-question regressions with negative controls; each case passed a focused paid run. * test: PTY harness handles clipped reviews and bundled setup tabs; AUQ judge uses structured output; design-consultation carve declines optional outside voices - ceo mode routing: a Submit review taller than the viewport, a setup tab bundled after the mode tab, and a clip through the mode question each hung or misread the run; the native answer is still verified after Submit. - judgeRecommendation requests a 1-5 enum schema; a malformed Haiku reply had scored substance 0 for a 4/5 brief. Judge failures now propagate. - carve section-loading for design-consultation declines the optional outside voices (a supported path) and treats DESIGN.md as the report; timeout unchanged. The Step 0E handoff defect is not fixed (0/15 samples across four wordings, none shipped) and is filed in TODOS. * test: fold the design-consultation completion replay into carve-section-sharding (test-of-test ratchet) * docs(todos): record the pre-push hook shard-order hang * test(qa-callers): disable git auto maintenance in the fixture repo (same guard as shared-libs; from #3002) * test(office-hours-attempt): the fake judge SDK response carries stop_reason like the real API (structured judge requires end_turn) * fix(qa): the caller STOP line says to await the method Reads before any probe ship-exploratory-plan-checks: the model read exploratory.md and sent a capture in the same response, before seeing the section's own await rule. * fix(qa): number the qa value-bar questions from 1 and say reproduced bugs already answer the first two * fix(qa): define evidence.json where it is built, point the preparation gate at the next section, name measured command durations in the report template Recurring qa/qa-only workflow-judge complaints in CI (clarity/actionability 3.33). * fix(plan-eng-review,review): a disallowed question tool is not headless; report kept tests only when some were skipped * fix(plan-eng-review): keep the headless-rule contract phrases adjacent * fix(evals): cut path variance at its measured sources - gstack-qa-evidence capture prints startedAt/completedAt/durationMs and, for --deadline captures, remainingMs; the functional report takes durations from them. The section clock notice asks for one clock read up front instead of one after every checkpoint (QA runs spent 7-14% of tool calls on date -u). - ship plan-completion: skip the audit dispatch when discovery already found no plan (the dispatch-vs-skip conflict produced an optional 60-100 s subagent). - materialize/checkpoint validation errors state the expected schema, so a rejected annotations file is fixable in one call instead of blocking the phase. - session-runner counts turns from the transcript when a run times out, so timeouts stop reporting 'turn 0'. * fix(evals): count timeout turns only from object transcript events * test(qa-callers): deterministic child transport, completion-time handoff reads, compact phase report The exploratory caller cases exist to prove the caller starts and bounds exploratory QA. Their native adversarial reviewer (review) and plan audit (ship plan-checks) now come from recorded child outputs instead of a live subagent, handoff freshness reads are required before completion records rather than every bookkeeping log, and the phase report is compact. Measured: 194-257 s per case against 208-284 s before, no subagent calls. * test(ship-docsync): seed fault cases at their gate instead of replaying attempt 1 The post-dispatch fault cases (missing-marker, launch-failure, timeout-unsettled, late-result, stale-before, stale-after, recovery) now start from a fixture-owned attempt 1: the real actor prepares and dispatches it, its verbatim output is saved once, and the invocation journal carries its pre-dispatch entry with the child asset hashes. The model resumes at Parent processing with a trimmed read list, inspect named as the authoritative repository observation, and recovery's intermediate checkpoint folded into the next attempt's pre-dispatch entry. Assertions count only parent-issued transport events and require a read of the saved attempt-1 output; missing-asset and the legacy failure case keep the full model-driven first attempt, and their prompts are byte-identical. * test(ship-docsync): name the seeded read list and cap journal/report length The first seeded stale-before run spent calls locating documentation.md (two ls sweeps), reading through cat and re-Reading the record before Edit, and ~40 s composing 1.5-2.2 KB entries and report. Name every seeded read path, ask for native Read, and bound entry/report length. * test(ship-docsync): trim the seeded parent's measured model time Measured on the seeded runs: one read the 78 KB ship/SKILL.md, the post-child freshness comparison spent 18-32 s of thinking over full inspect contents, and the final response restated the report (~1.1 KB). Say the phase excerpt stands in for ship/SKILL.md, compare hashes first and read content only for changed paths, and end with one status line. * feat(qa-evidence): enforce the checkpoint sequence and fill report bookkeeping in code - capture refuses to run another probe until a checkpoint anchored on the latest complete capture names this capture as its next command, and every complete capture prints that requirement. - materialize fills revision, runtime, cwd and learning (checkpoints whose next native command differs) when omitted and prints the reportLinks the report must include; the QA section shrinks accordingly. * test(qa-callers): hand the caller phase its invocation-start observations and review token; fix(next-version): fetch without auto maintenance - Every caller case receives the diff, status, log, untracked list, HEAD and an already-captured review start token, so the phase spends its budget on the contract under test instead of re-running setup reads. - gstack-next-version's fetches pass --no-auto-maintenance. On git 2.55 a completed fetch forks detached maintenance in the caller's repository; the free suite's live smoke test ran it inside the CI checkout, and every shard-12 pre-push hook hang so far followed a completed smoke fetch. * feat(deslop-shared-libs): route every Git read through bin/gstack-safe-git The skill made the model retype a long safe-Git prefix on each call and a dropped flag failed shared-libs-read-only. bin/gstack-safe-git applies the fixed env + flag prefix, adds --no-ext-diff --no-textconv to log/show/diff, allows diff only between two explicit object IDs and ls-files only in the NUL-delimited overlay form, and refuses every other shape with one line naming the allowed forms. The template now points at the installed helper (host global runtime via {{SAFE_GIT}}) and drops the prose it enforces. Fixtures resolve the helper to this checkout, the git shim records the safety environment, and isGuardedGitRequest requires the complete prefix (env included) for every repository read. * test(shared-libs): tee to a discard device is not a file write Paid shared-libs-opportunity-judgment t1 on 1213b01 failed read-only on '... | tee /dev/null | sha256sum'. The detector flagged any tee operand while the same devices are allowed for redirection. tee now fails only when an operand is a real file; tee to a file, -a file and -- -a stay violations. * fix(qa-evidence,observer): reject placeholder metadata and replay-only learning; declare the docs atomic-write target - materialize measures revision, runtime and cwd itself and rejects supplied values that differ (CI run wrote revision "HEAD" and runtime "bun"), and refuses learning checkpoints that replay the same probe, naming the fix. - The docs write observer treats Claude Code's atomic temp for the authorized doc target as transient, so a temp renamed before its per-file watch no longer marks the observation incomplete (ship-docsync-completion flake). Per-file monitoring outside declared targets stays fail-closed. * test(qa-functional): fix mode requires only the happy scenario from the model (carried byte-identical from #3002 183b01f4..3e6074b4) verifyQANativeRegression already reruns all eight webhook scenarios on the repaired source, so the model-side eight-scenario requirement in fix mode duplicated harness coverage and pushed qa-functional-webhook-fix past its budget. qa-only still requires every scenario. * fix(deslop-shared-libs): probe the audited repository with -C <repo> A CI run probed safe-git from the session directory above the target repo, so the capability probe never touched the repository and the run fell back to the API without a local attempt. The probe (and any call from elsewhere) now names the audited repository. * test(qa-deadline): never attach a reader to the full-pipe fixture's stdout The full-pipe receipt test attached a 'data' listener (flowing mode) and then paused; on CI the reader could drain the 2 MB write before the pause, so the receipt write never blocked and the helper exited 0 in ~126 ms. The stdout pipe now stays unread until the assertion, which is what the test means to model. * feat(qa): helpers answer --help, and the QA eval interfaces declare it Approved by Garry: asking gstack-qa-evidence or gstack-qa-deadline for usage is read-only, so both helpers print usage and exit 0 on --help (the evidence usage now names the annotation shape), and the functional and caller command allowlists accept exactly 'bun <path>/bin/gstack-qa-{evidence,deadline} --help'. Two CI runs failed only on that call. * fix(qa): after an input change, a probe is affected unless shown otherwise CI late-input run finished in time but revalidated only the happy probe after the locale input changed and reported the stale adverse probe green. The revalidation step now treats any probe not shown to be unaffected as affected. * test(shared-libs): seed the lifecycle replay's first Step 3 pass instead of replaying it shared-libs-review-lifecycle ran ~88% of its 300 s session budget (12-run census median 265 s, 4/24 sessions timed out). The fixture now executes pass 1's Step 3 once with the real logger and Git: a real unused REVIEW_START, then the diff, inventories, attributes/config/index flags, gstack-review-read output and every file's bytes and sha256, saved to one observation. The model resumes at Step 4 with an exact four-file first read, the observation named as the authoritative pass-1 repository read, one post-fix verification, an explicit pass-2 read list and a twelve-line summary. Pass 2 still runs its own --start, diff, reads, fingerprint and stage actor before --finish. The actor scope now states that a current settled final-pass actor result supplies the replaced QA/adversarial prerequisites and that the no-credit disclosure is a reporting label: one r1 session persisted completed:false from that ambiguity. New assertions: the final binding never uses the seeded token's start or tree, and the observation was read; free controls finish the seeded token (binding changed) and omit the observation read, and both fail. * test(shared-libs): trim the resumed review replays' setup and report Every sibling review session (revalidation, path-eligibility, index-flags, prior-coverage) loaded qa/sections/exploratory.md and often scope.md although its QA and native adversarial results are supplied synthetic inputs, then spent a second request on shared-code-reuse.md and base metadata. The resumed scope now states that the supplied results replace Step 4's QA method loading; the revalidation contract names one first response (workflow, checklist, finding, prerequisites, shared-code-reuse.md, base metadata) and caps the summary at twelve lines. Receipt order, direct source reads, the checker, the question and final persistence are unchanged. * fix(review): define what a Step 5c Skip option says Step 5c named "B) Skip" without saying what its description may claim. Two CI captures (path-eligibility on |
||
|
Garry Tan
|
dcaea52800 |
v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983)
* feat: add surface-aware exploratory QA and ship documentation gates * test: preserve delegated QA setup authority after main integration * fix(qa): clarify exploration order and preserve report artifacts * test(qa): follow the shared setup reference directly * refactor(ship): make verification and recovery routes explicit * test(ship): align evidence and review guards with explicit routes * fix(workflows): clarify ship recovery and functional QA evidence * fix(workflows): clarify approval recovery and full QA coverage * refactor(workflows): order review transactions and clarify ship state * fix(ship): clarify final verification and fail closed at publication * fix(evals): attribute native atomic documentation writes * fix(ship): clarify recovery and documentation lifecycle guidance * fix(test): preserve observed native placeholder styling in CI * fix(codex): report watchdog timeouts without a process-exit race * Checkpoint functional QA implementation and workflow validation repairs * Fix documentation and shared-review fixture contracts * docs: clarify judge reuse and evaluation supervision * test: align review evidence and selected case contracts * test: verify append-only documentation checkpoints and recovery * fix: qualify QA workflows and CI validation repairs * fix: launch shared-libs fixture scripts on Windows * fix: qualify QA deadlines, fixture isolation, and shard cleanup * fix: preserve qualified QA and cancellation repairs * fix: enforce functional fixture authority and share strict event decoding * fix: retain free-test evidence and explain recovery * fix: reject malformed native evidence after decoder consolidation * test: use reliable capture for telemetry privacy filters * test: refresh measured quick coverage and document validation costs * Fix native fixture receipts and preserve VM validation evidence * Align negative judge controls with upstream clarity policy * Fix report-only QA preparation and public evidence handling * Clarify QA-only preparation and current-report preservation * Stream Ship quality judgments with an explicit 64k response contract * Validate compact judge reasoning locally with supported wire schema * Align functional QA fixture instructions with evidence acceptance * Bind native browser diagnostics to execution evidence and align review verdicts * Preserve native diagnostic line boundaries * Serialize functional QA evidence from native captures * Keep large QA evidence fixture payload out of Windows argv |
||
|
|
01593aa67c |
v1.91.2.0 fix: consolidate gstack reliability wave (#2959)
* fix(memory-ingest): --scan-secrets scans the rendered page and fails closed --scan-secrets ran gitleaks on the raw transcript .jsonl, then imported a page rendered from it. gitleaks' assignment rules don't match across a JSON-escaped quote (KEY=\"v\" on disk), so a secret the rendered page shows as KEY="v" was imported unflagged. And the gate skipped a file only on scanner "gitleaks" with findings, so a scan that errored (non-zero exit, 16MB maxBuffer overflow on a file with many findings, unparseable report) or could not run (gitleaks missing, slow-probe cooldown) imported the file unscanned. Scan the rendered page body, the exact bytes writeStaged() writes, via a new secretScanText() helper, and skip the file whenever the scan did not complete. Skipped files stay out of the state file, so the next run retries them. Reword the helper warnings and setup-gbrain/memory.md, which described the fail-open as intended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(test): reconcile Bun failure markers and footer counts * fix(sync-gbrain): verify source-scoped reads without mutation * fix(test): recognize grounded TTHW target choices structurally * fix(aside): make the readiness probe work under zsh and report why it failed The probe built its deadline into `_T` and expanded it unquoted, so `$_T aside repl …` only worked in a shell that word-splits. zsh does not: it looked for a command literally named "gtimeout 30", the probe answered ASIDE_NOT_RUNNING with Aside installed and ready, and every browsing skill fell back to the bundled Chromium in silence. zsh is the macOS default and Aside is macOS-only, so on a stock Mac the probe could never report READY. The deadline becomes a function, `_gs_d`. It receives the command as "$@", already split, so sh, bash and zsh all behave the same, and the gtimeout → timeout → perl alarm chain is unchanged. A 4th arm runs the call unbounded when none of the three is present, which is what the empty `_T` did before. Not `eval`: it re-parses the string, so the parens and `;` of the perl arm become syntax and that arm dies in bash *and* zsh — on a stock Mac, the arm that actually runs. On failure the probe now prints the CLI's reason after ASIDE_NOT_RUNNING:, the shape gstack-render already uses: the first line that starts with a capital letter, i.e. the CLI's own sentence or Node's `Error:` line below its loader frame. "Not running" covers states with different fixes — no window open for the profile, a NODE_OPTIONS preload that kills the CLI — and a bare verdict sent all of them to "open the Aside app". The BROWSER SETUP prose quotes that reason before asking the user to open the app. The text pin asserted the broken invocation verbatim, so it now pins the function and asserts neither `$_T aside repl` nor an eval form comes back. A second test executes the rendered probe in sh, bash and zsh on each of the four deadline arms with stubbed binaries on a narrowed PATH, plus two failing CLIs: one that prints its own sentence, one that crashes like Node with the useful line below the frame. The deadline function costs zero bytes against the lines it replaces; the reason costs 53 per copy of the probe (44 where the reworded BROWSER SETUP line gives 9 back). That moves four guards by the measured amount: plan-devex-review's skeleton cap to 68,550 (measured 68,544), plan-ceo-review's skeleton cap to 80,150 (measured 80,111) and union ratio to 1.081 (measured 1.0803), and plan-eng-review's union ratio to 1.151 (measured 1.1504). Fixes #2842, #2941. * Clarify engineering review startup and decision flow * Fix Windows readiness fixture PATH and command shim * fix(test): recognize grounded TTHW target choices structurally * Clarify engineering review startup and decision flow * fix(test): restrict QA-only fixture tools to its no-Edit contract * v1.90.0.0 fix(sync-gbrain): guard readiness verdicts and refresh metadata * fix(browse): validate canonical upload targets * fix(gbrain): classify structured PGLite busy response * fix(browse): preserve native extension runtime APIs * Fix displayless browser handoff ownership * Accept unique installed autoplan methodology aliases * fix(skills): preserve positional literals during installation * fix(browse): checksum installer contents through stdin * fix(test): normalize Windows checksum fixture paths * test: emulate unavailable shasum in Windows checksum fixture * fix(investigate): preserve owned freeze lifecycle * fix(review): preserve N+1 retry and Red Team completion * fix: bound Aside readiness and preserve safe fallback * test: exercise setup and Chromium on native ARM * fix: preserve install ownership and ARM browser selection * Fix gbrain ingest scan boundaries and seed observation * Refresh managed ship hooks and supervise expanded paid census * Reject resumed gbrain pages excluded by current policy * Recover zombie agent locks safely and enable CI Python venv * Repair paid actor declarations and Aside pitch assertions * Bump consolidated wave to next free minor release * Clarify CEO review admin choices and option tradeoffs * Preserve CEO mode handoff anchors in clarified workflow * Make Windows portability fixtures use shell-native paths * Restore ARM Bun alias and clarify ship review gates * Refresh ship workflow golden snapshots * Fix Windows DX documentation controls without piped stdin * Decode Codex child pipes without Bun's encoded-stream stall * Bound DX pre-review audit before product questions * Clarify trusted review-start read in paid revalidation * Bump consolidated wave to next free minor release * Clarify CEO review admin choices and option tradeoffs * Preserve CEO mode handoff anchors in clarified workflow * Make Windows portability fixtures use shell-native paths * Restore ARM Bun alias and clarify ship review gates * Refresh ship workflow golden snapshots * Fix Windows DX documentation controls without piped stdin * Decode Codex child pipes without Bun's encoded-stream stall * Bound DX pre-review audit before product questions * Clarify trusted review-start read in paid revalidation * Reconcile new main planning flow and paid judge census * fix: reconcile rebased planning and source-bound validation * test: pin cookie workflow judge to scored Sonnet model * fix: keep terminal agent boot out of module imports * fix: preserve pending-question uncertainty in engineering review * fix: stabilize Windows reliability-wave fixtures * fix: clarify design consultation research workflow * fix: preserve independent design consultation inputs * fix: resolve design taste scope and browser research guidance * fix: make consultation opt-in preflight unambiguous * test: await native Edge owner readiness or terminal result --------- Co-authored-by: Bruce Krysiak <brucek@alum.mit.edu> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Antonio Vitalic <antoninte99@gmail.com> |
||
|
Garry TanandOpenAI Codex
|
06ed920a97 |
v1.89.0.0 feat: add shared-code extraction audit (#2925)
* feat: bind shared-code review advice to source and branch * feat: add shared-code extraction audit and scoped review checks * test: recognize complete source reads and explicit coverage legends * chore: bump version and changelog (v1.88.0.0) Co-Authored-By: OpenAI Codex <noreply@openai.com> * test: capture native review questions and retain public evidence Capture the actual first public native question with strict ownership and display matching. Preserve terminal failures and raw evidence, and retain SDK completion checks. * test: recognize verified review evidence and complete fixtures Recognize complete source and diagram evidence, concrete design and developer-experience decisions, and the complete planted scenario contracts. Preserve negative controls and grading thresholds. * fix: preserve decision brief structure in native questions Keep the required pros-and-cons heading and final Net field in native question text. Regenerate host outputs and document the release and evaluation repairs. Co-Authored-By: OpenAI Codex <noreply@openai.com> * docs: update project documentation for v1.88.0.0 Co-Authored-By: OpenAI Codex <noreply@openai.com> * fix: correct eval retry accounting and ship workflow gates * fix: capture native eval evidence and stabilize CI fixtures * fix: keep shared-code eval skips read-only Choose explicit no-change answers instead of mixed fix/preservation options. Reuse the bounded revalidation prompt for path fixtures so required review metadata is available without repeated discovery. Preserve source checks, retry limits, and failed native terminal outcomes. Add captured-question and callback regressions, plus evaluation selection coverage for the affected fixtures. --------- Co-authored-by: OpenAI Codex <noreply@openai.com> |
||
|
Garry TanandClaude Fable 5.1
|
c8f0c4e368 |
v1.84.0.0 feat: impeccable interop: detector pre-pass in four design skills, DOM-mode scans, open DESIGN.md format, one typed slop catalog (#2832)
* chore(design): pin impeccable rule ids and detector JSON shape as fixtures Real captures from a human-initiated `npx impeccable install` in a scratch directory (engine 0.1.3, linux-x64), never a runtime download: - test/fixtures/impeccable-antipatterns.json: upstream crates/live/assets/antipatterns.json at 87d8f6d6 (the state engine-v0.1.3 shipped), 61 rules, source commit recorded in `_source`. - test/fixtures/impeccable-detect-sample.json: `detect --json` over gstack's planted-slop fixture (source mode), paths normalized. - test/fixtures/review-eval-design-slop.dom.html + impeccable-detect-dom-sample.json: the same page served locally, dumped through the browse engine with the shared DOM-dump script, then scanned. Pins the load-bearing assumption that the static engine reads inline <style> in a .html file: the DOM scan yields the same id set as the source scan. - lib/dom-dump-script.ts: the one dump script both browser engines evaluate (IIFE, no single quotes). Folds CSSOM rgb() back to author hex so palette rules still fire, and removes inlined <link> nodes so the engine does not warn about an unresolvable stylesheet. Both verified against the engine. - test/fixtures/impeccable-detect-help.txt + impeccable-captures.meta.json: the flags, exit codes, finding fields, and re-capture protocol. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(design): typed slop catalog in lib/; AI_SLOP_BLACKLIST derived lib/design-catalog.ts is the single source of truth for gstack's design anti-pattern vocabulary: the 11 legacy blacklist lines (verbatim, flagged `legacyBlacklist`), every one of impeccable's 61 registry ids with gstack prose, tier, impact, confidence, grep heuristic, and /impeccable handoff, plus the gstack-only tells the LLM pass judges (hero metrics, identical cards, glassmorphism, missing states, unthemed browser surfaces, ...). `impeccableId` is set only when the id exists in the registry fixture, and `renderCatalog({style:'ids'})` brackets an id only then, so rendered prose never shows an id the detector cannot emit. Role-scoped font lists (OVERUSED_FONTS_DISPLAY, BANNED_FONTS, FONTS_BODY_UI_OK, FONTS_MONO_OK, FONTS_VERIFIED_FREE) live beside the entries. scripts/resolvers/constants.ts now derives AI_SLOP_BLACKLIST from the catalog. Generated output is byte-identical (bun run gen:skill-docs is a zero diff). Pure module: no I/O, no scripts/ imports, loading prints nothing, so bin/ can import it at runtime on every host. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(review): generate review/design-checklist.md from the catalog review/design-checklist.md was hand-written and its own header admitted it drifted from DESIGN_METHODOLOGY category 9. It is now rendered by scripts/resolvers/design-checklist.ts from lib/design-catalog.ts: category 1 lists every grep-detectable slop entry plus the legacy blacklist lines, sorted HIGH/MEDIUM/LOW, each with its heuristic and, where the detector knows the rule, its bracketed id (27 items, up from 6). The font blacklist renders from BANNED_FONTS. Categories 2-5, Instructions, Classification, Output Format, and Suppressions keep their prose. Title and slop heading are unchanged (test/skill-e2e-review.test.ts and hosts/opencode.ts key on them). gen-skill-docs writes the file for the Claude host only (a Claude-side runtime asset; other hosts copy or inline the render), honors --out-dir, and reports STALE/FRESH under --dry-run like sections do. test/design-checklist-sync.test.ts pins committed == generated, the host/out-dir scoping, and the dry-run freshness line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): modes, craft-floor reflexes, calibration, catalog in doctrine DESIGN_HARD_RULES: the classifier names four visitor modes (Persuade, Operate, Read, Experience, plus Hybrid per section) and keeps the MARKETING/LANDING PAGE and APP UI aliases; Read and Experience get three rules each; a "Reflexes no detector catches" block (browser surfaces, one authored motion moment, depth has an offset, tinted secondary text, space above headings, light-or-dark from the use scene) and the three-looks calibration follow the universal rules. The slop section renders the 11 legacy lines plus the detector rule ids and judgment tells from the catalog; in design-review, which also renders DESIGN_METHODOLOGY, it becomes a one-line pointer so the catalog is paid for once. Header counts are computed, not hardcoded. DESIGN_METHODOLOGY: category 9 renders the catalog in three registers (legacy lines verbatim, detector rules that need judgment with bracketed ids, gstack-only judgment tells as prose, polish-level ids on one line); categories 5 and 7 carry the browser-surface and one-motion-moment reflexes; the typography overused-face item points at [overused-font] with the role-scoped exception. The consultation Codex prompt's anti-slop line reads from the catalog. Budget: design-review eager 25.6K -> 27.0K (ceiling 27,984), plan-design-review unchanged at 17.4K; no carve-guard or context-budget re-baseline needed; ship goldens unchanged (ship never renders the hard rules). Derived from pbakaus/impeccable reference/craft-floor.md + new-work.md (Apache-2.0), rewritten. See NOTICE.md (commit 12). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design-consultation): font procedure, role-scoped overused list, color strategies The proposal section stops handing out a font menu. "Choosing faces: a procedure, not a menu" names the subject's world, shortlists per role, strikes the overused list for that role, verifies availability in-session, and states the loading strategy. {{OVERUSED_FONTS}} renders the role-scoped lists from lib/design-catalog.ts: overused as display (the detector's overused-font set plus the training-data defaults), fine as body/UI on an Operate or Read surface, mono for data and code, banned in any role, and a short verified-free list with its verification date. Color approaches become Restrained / Committed / Full palette / Drenched. The anti-convergence directive drops light-vs-dark as a dial (it comes from the use scene) and the three-looks calibration sits under Your Design Knowledge. The slop list is {{DESIGN_SLOP_BULLETS}}: prose from the catalog, no rule ids, polish-level tells omitted. design-html's "Never include (AI slop blacklist)" list keeps its literal (carve guard) and each line now carries a trailing <!-- id --> naming a catalog entry, pinned by test/design-catalog.test.ts so the last surviving duplicate is derived-by-test. Both resolvers are registered and listed in ARCHITECTURE.md. No carve-guard or budget re-baseline needed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(bin): gstack-design-detect wrapper + design_detector config key bin/gstack-design-detect.ts finds and runs an impeccable engine the user installed; it never installs, downloads, or executes anything that could download. `probe` reads only: config (design_detector off → DISABLED), IMPECCABLE_BIN (absolute, realpath outside the repo and cwd), a PATH walk (absolute entries outside the repo; a #! shim counts as launcher-present, never READY), the ~/.impeccable/bin/<newest semver>/ cache, and the engine installed beside a skill launcher (scripts/bin/<os>-<arch>/impeccable, the layout a real install produced). It reports IMPECCABLE_SKILL, host-aware IMPECCABLE_HOOK (+ HOOK_OTHER), the ignore lists from .impeccable/config*.json, IMPECCABLE_ENGINE_UNTESTED for versions outside the fixture set, and a hint only when a launcher exists without its engine. `scan` re-probes, refuses URLs and anything outside the repo root or the design-report allow-list (realpath, so symlinks cannot escape), derives `--changed <base>` targets NUL-safely through git and lib/frontend-scope.ts, batches 100 absolute paths per engine call with stdin ignored, a SIGKILL timeout, a 50 MB stdout cap, and sanitized length-capped fields, then prints one normalized JSON document (--format gstack) or the engine's bytes (--format raw); DETECT_TOP (fenced as untrusted content), DETECT_SUMMARY, and DETECT_EXIT go to stderr; exit code passes through with 1 over 2 over 0; exit 3 is a gstack bug. `rules` prints the mapped set. Every run appends a content-free line to the local analytics file. lib/design-detect-contract.ts owns every sentinel string, the limits, and the normalized-finding shape (pure module); test/design-detect-contract.test.ts asserts every sentinel-shaped token the agent can read exists there. lib/frontend-scope.ts mirrors gstack-diff-scope's frontend arm, pinned by a parity test that runs the bash script. bin/gstack-config gains design_detector (auto | off, default auto, invalid values rejected with the file unchanged). test/fixtures/fake-impeccable.ts is the env-driven engine stand-in; test/gstack-design-detect.test.ts covers READY/NOT_CACHED/ NOT_AVAILABLE/DISABLED, env trust (.env never loaded, in-repo IMPECCABLE_BIN ignored), newest-semver cache, hook and ignore detection, refusals, exit passthrough, raw byte-identity, normalization, the display cap, timeout, parse errors, diagnostics, --changed, and analytics. The egress scanner test records the wrapper as a documented non-sink. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): {{DESIGN_DETECTOR}} wired into design-review, ship review-lite, review army, design-html The user-installed impeccable engine becomes a deterministic pre-pass in four skills, through one resolver with three renders: {{DESIGN_DETECTOR}} (the probe block and how to read every sentinel), {{DESIGN_DETECTOR:phase0}} (design- review's mechanical scan), {{DESIGN_DETECTOR:gate}} (design-html's bounded slop gate). Every rendered invocation is `bun --no-env-file run <bin>/gstack-design- detect.ts ... --host <host>` and every scan ends with the DETECT_EXIT_CODE echo so exit 2 (findings) never aborts a block. design-review: probe in Setup; Phase 0 picks DOM mode (URL target) or source mode (diff-aware, no URL) once; source mode scans the changed frontend files in Setup, DOM mode never reads source (Rule 4). Phase 3 gains a DOM-dump step per page: both browser engines load the shared script from lib/dom-dump.js (Aside splices it into a double-quoted repl script; the fallback engine copies it into a temp dir for `$B eval --out --raw`), the dump is size-capped, run through gstack-redact (a HIGH finding skips the page), and persisted under $REPORT_DIR/dom/$RUN_ID/; one scan runs after the last page, labeled "static scan of the rendered DOM; cross-origin CSS not resolved". REPORT_DIR honors GSTACK_HOME so the wrapper's allow-list and the report dir agree; RUN_ID is set once in Setup. design-baseline.json is schemaVersion 2 with runId, targetSet, base, and a detector block (mode, engine, byRule, byPage), written temp+rename with a per-run copy; Regression Output diffs ids only when mode and target set match, caveats an engine change, and calls live-page count deltas advisory. Phase 7 hands deferred detector findings to the `handoff=` command the scan printed; Phase 9 recomputes the same way and deletes the dumps unless --keep-dom; Phase 10 reports `Detector: N → M`. ship review-lite gains step 0 (probe, `scan --changed <base>`, tier buckets, detector + checklist dedupe, advisory and ignored never count) and a `detector` count in its log payload; the PR body gets a Detector line (rule ids and counts only). The Review Army Design specialist runs the mechanical pass at the top of review/design-checklist.md, which now carries it. design- html probes after DESIGN_SETUP and runs the one-pass gate before screenshots. lib/dom-dump.js is generated by gen-skill-docs from lib/dom-dump-script.ts (Claude host, --out-dir aware, dry-run freshness) and pinned byte-equal, so the prose never carries the script. The contract gains DETECT_JSON, DOM_DUMP_OK, and the self-describing set; its test now checks both directions. Budget: design-review eager 25.6K → 28.5K. The plan's target was +2.5K; after the levers it named (ids-only detector rules, no inline script, trimmed prose) it lands at +2.87K, and the remainder is doctrine and detector wiring, so the ceiling moves to the captured 31,319 for design-review only (the full capture would also have loosened 21 ceilings this branch never touched; those stay). design-html skeleton re-baselined to 54,000 (measured 53,592). Codex and Factory ship goldens refreshed (review-lite step 0 and the PR-body line render inline there). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): catalog never-lines in the mockup prompt Ten catalog ids carry `mockupNever` (kicker-above-heading, icon-tile-stack, gradient-text, ai-color-palette, cream-palette, nested-cards, dark-glow, pulsing-dot, identical-cards, hero-metrics) and lib/design-catalog.ts exports their deduped plain-English names as MOCKUP_NEVER_NAMES. briefToPrompt() in the design binary appends "Never: <names>." before its fixed tail, so `$D generate | variants | evolve` stop reaching for purple gradients, icon tiles, and cream defaults before the comparison board opens. The binary still bundles (`bun build --compile design/src/cli.ts`); ./setup rebuilds it. design-html's Never-include list now covers every mockupNever id (kicker / icon tile, hero metric rows, gradient text, cream palette, nested and identical cards, glow and pulsing dots), each line tagged with its catalog ids; test/design-catalog.test.ts pins the exact ten flags, the deduped names, and that the template list is a superset. New design/test/brief.test.ts pins the prompt shape. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(lib): open DESIGN.md reader/writer + gstack-design-md bin lib/design-md.ts implements the open DESIGN.md format (google-labs-code/ design.md, Apache-2.0): YAML front matter with the five token groups (colors, typography, rounded, spacing, components) and eight canonical `##` sections in spec order (Overview, Colors, Typography, Layout, Elevation & Depth, Shapes, Components, Do's and Don'ts), aliases mapped, extras preserved after them in their original order. parseDesignMd never throws (unparsable front matter → `unknown` with a reason); renderDesignMd re-emits the preserved front matter bytes and only `convert` writes fresh YAML through a small block-style emitter (Bun.YAML.stringify is flow style); upsertSection splices the body only; tokensFlat resolves `{path}` references to primitives and reports group, self, dangling, and cyclic refs as DESIGN_MD_TOKEN_REF_INVALID. convertLegacy turns gstack's pre-spec DESIGN.md into the open format: Product Context and Aesthetic Direction fold into Overview, Typography roles become display/body/label/mono tokens (mono carries fontFeature: tnum), Color hexes become colors (mode-qualified labels keep their qualifier; strategy lines are not colors), the Spacing scale and Layout radii become spacing and rounded, Motion / Grain Texture / Decisions Log survive as extras. The format marker lives inside the file: a YAML comment on line 2 of a spec file, an HTML comment on line 1 of a legacy file. bin/gstack-design-md.ts: `check` (DESIGN_MD_FORMAT + marker), `convert [--write]` (backup to DESIGN.md.legacy.bak, temp+rename, refuses ambiguous input with DESIGN_MD_CONVERT_REFUSED), `tokens` (flat JSON), `mark <spec|legacy-keep>`. Exit 3 + DESIGN_MD_INTERNAL_ERROR is a gstack bug. design/src/memory.ts: updateDesignMd upserts "Extracted Design Language" through the lib (front matter bytes untouched, canonical order kept, section replaced on rerun) and creates a spec skeleton with tokens from the extraction when no file exists; readDesignConstraints leads with the flat tokens and the Overview for spec files. The design binary still bundles. test/design-md.test.ts pins all of it against gstack's own DESIGN.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): write/read DESIGN.md in the open spec; persisted format choice gstack's design skills now write DESIGN.md in the open DESIGN.md format and read tokens from it. {{DESIGN_MD_CHECK}} renders the format check through bin/gstack-design-md.ts: design-consultation's Phase 0 settles the format once (spec → update tokens in the front matter; legacy without a marker → one AskUserQuestion: convert with a .legacy.bak, keep the legacy file, or start fresh; the answer is written into the file as the format marker so no skill asks again; a marker already present is obeyed silently; unknown → prose; missing → Phase 6 writes one). Phase 6's template is the spec form: YAML front matter with name, description, and exactly the five token groups (colors, typography.display/body/label/mono with fontFeature: tnum on mono, rounded, spacing, components with {path} references), then Overview (Creative North Star, product context, mode per surface, references, key characteristics), Colors (opening with the Restrained / Committed / Full palette / Drenched strategy), Typography, Layout, Elevation & Depth, Shapes, Components, Do's and Don'ts, plus gstack's Motion and Decisions Log as extras; the template ends with a check that the file parses as `spec`. design-review runs the `:calibrate` form in Setup: a spec file's flat tokens are the calibration source (a value present in the tokens is never a finding), the marker is respected, and conversion is never offered there; its DESIGN.md export writes the spec form. design-html's token extraction writes the spec form and respects an existing choice. review/design-checklist.md category 5 and ship's review-lite step 1 name `gstack-design-md tokens` as the calibration source; plan-design-review Pass 5 cites tokens by path when front matter exists. The contract owns the bin's DESIGN_MD_MARKER / REASON / WRITTEN / BACKUP lines; the contract test's pending list closes. Carve guard: design- consultation skeleton 66,500 → 67,500 (measured 67,014; +1,508 B against the 1.5 KB cap). Codex and Factory ship goldens refreshed (review-lite step 1). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): PRODUCT.md prefill + /impeccable handoffs design-consultation's context gathering and design-shotgun's auto-gather read PRODUCT.md (impeccable's product-context file) when it exists: it counts as the user's prior answers, gets confirmed in one line, and is never re-asked. Neither skill opens `.claude/skills/impeccable/**`; PRODUCT.md and DESIGN.md are the shared surface, and impeccable's prose never loads inside a gstack skill. Handoffs: ship's review-lite ends each NEEDS INPUT detector row with the `handoff=` command the scan printed (`/impeccable <cmd>`) when the probe reported IMPECCABLE_SKILL: present, recommending the command and never opening its files; design-review's Phase 7 does the same for deferred findings, and `design_detector: off` silences handoff lines with the rest. Codex and Factory ship goldens refreshed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(design): convert gstack's own DESIGN.md to the open spec `gstack-design-md convert --write` on the repo's DESIGN.md: tokens in YAML front matter (typography.display/body/label/mono, colors with their light/dark qualifiers, spacing scale, rounded scale), Overview from Product Context and Aesthetic Direction, Colors / Typography / Layout as canonical sections, Motion, Grain Texture, and Decisions Log preserved as extras, format marker on line 2. Hand-checked; `check` reports spec with no token-reference errors. A Decisions Log row records the conversion and that DM Sans stays the body face: it is on the overused-as-display list, and body/UI use on an Operate surface is the allowed exception under the role-scoped rule. The pre-conversion file lives on as test/fixtures/design-md-legacy.md, which test/design-md.test.ts now uses for its legacy cases; the converted root file is asserted to be spec. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: NOTICE, Apache license text, README interop, project structure NOTICE.md names what gstack derived from impeccable (rule ids and names in the catalog and the registry fixture; the visitor modes, craft-floor reflexes, and calibration in the design resolvers; the font procedure in the consultation template) and from Google's DESIGN.md specification (the format lib/design-md.ts implements), states that gstack does not distribute or audit the impeccable engine, and points at licenses/Apache-2.0.txt (verbatim). README: the design-consultation, design-review, and design-html rows say what changes when impeccable or the open DESIGN.md format is in play, and a "Works with impeccable" paragraph explains the pre-pass, the shared ids, PRODUCT.md and DESIGN.md as the shared surface, the handoffs, the no-nag posture without impeccable, and the off switch. docs/skills.md gets the detector paragraph under /design-review. docs/PROJECT_STRUCTURE.md lists the new lib and bin files, NOTICE.md, and licenses/. docs/designs/IMPECCABLE_INTEROP.md promotes the CEO plan (its ~/.gstack copy is flipped to PROMOTED) with a "what shipped" summary. TODOS.md files the seven deferrals from the reviews: the design-review Phases 7-11 carve (the budget lever, with the +2.87K vs 2.5K landing recorded), the Bun .env audit across bin/*.ts, the Kiro bin/lib gap, the $D check slop rubric, taste-profile interplay, the CEO Section 11 bullets, and the scan cache. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: touchfiles, tiers, shim E2E, real-engine fixture Touchfiles: the catalog, contract, detector bin, checklist resolver, review- army resolver, and DESIGN.md lib join the dep lists of review-design-lite, design-review-fix, the design-consultation cases, and plan-design-review-no- ui-scope, so editing any of them re-selects the tests that read their output. Three new E2E keys: design-review-detector-shim (gate; source mode on a feature-branch diff), design-review-detector-shim-dom (gate; DOM mode: the slop fixture served on loopback, dumped through the browse binary with lib/dom-dump.js, persisted under a GSTACK_HOME-scoped REPORT_DIR, scanned once; self-skips when browse/dist/browse is absent), and design-html-slop-gate (periodic; one fix pass, at most two scans, remaining findings accepted with reason). Every case reaches the engine through test/fixtures/fake-impeccable.ts via IMPECCABLE_BIN from outside the temp repo, reads extracted skill sections (never a whole SKILL.md) with the installed bin path rewritten to this checkout, and asserts the probe ran, the right scan verb ran, `npx impeccable` never did, and the output carries FINDING rows tagged [ai-color-palette] and [low-contrast]. review-design-lite gets the fake engine and an eighth tally signal for a detector row; its 4-hit threshold is unchanged. test/gstack-design-detect.test.ts evaluates design-review's REPORT_DIR expression with GSTACK_HOME set and proves a dump under it is accepted by the wrapper's allow-list. The sample fixtures were real captures from commit 1 (engine 0.1.3), so there is nothing hand-written left to swap. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-detect): never execute a repository-controlled engine; allow-list --changed targets; sanitize engine text Pre-landing review findings (security + checklist), all reproduced before the fix: - A checked-out branch could commit `.claude/skills/impeccable/scripts/bin/<os>-<arch>/impeccable` and the probe would report READY and `scan` would run it, with the agent's full environment. Launchers and sibling engines under the repo or cwd now count as "skill present" only (IMPECCABLE_NOT_CACHED: repository-local install, and the hint never names a repository-local launcher to run); only HOME-rooted installs, IMPECCABLE_BIN, the cache, and PATH entries outside the repo qualify, all by realpath. The engine now sees a minimal environment (PATH, HOME, TMPDIR, locale, IMPECCABLE_*), never the agent's tokens. - `scan --changed <base>` pushed git-derived paths without the allow-list, so a committed symlink with a frontend extension handed a file outside the repo to the engine. Derived targets now go through the same allow-list as explicit ones and symlinks named by git are refused outright. - A repo-controlled `scripts/VERSION` with embedded newlines forged probe lines; the version is trusted only when it is semver, and every printed version is sanitized. Engine text containing the untrusted-content fence or a `SENTINEL:` prefix is neutralized with a zero-width space (neutralizeSentinels in the contract), so page text cannot close the envelope or forge a probe line. - A failing `git diff <base>...HEAD` (unknown or unfetched base) was swallowed and read as "no frontend changes"; it is now DETECT_REFUSED with exit 1. - The scan allow-list root follows `${GSTACK_HOME:-$HOME/.gstack}` like the templates and gstack-slug (config.yaml keeps gstack-config's STATE_ROOT precedence); a quoted or commented design_detector value reads correctly. Smaller: raw engine chunks are kept only in --format raw; diagnostics are capped (200 kept, 20 echoed); the engine identity hash reads size + 4 MB, not the whole binary; PROBE_STEP and ENGINE_STDERR are contract sentinels; the --verbose gate covers every probe step; analytics use one sentinel vocabulary; bare limits live in DETECT_LIMITS. The fake engine's knobs are IMPECCABLE_FAKE_* (so they pass the minimal env) and a shared test helper installs it. New tests cover each item above plus clean runs, `{}` parse errors, missing paths, and the 50 MB stdout cap. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-md): mark and updateDesignMd never rewrite the user's file; refuse a contradictory mark renderDesignMd re-sorted canonical section names into spec order on every render, so `gstack-design-md mark legacy-keep` (the "leave it alone" answer) and the design binary's mockup extraction reordered a legacy DESIGN.md (Typography and Layout jumped to the top) and normalized its whitespace, while the bin promised "body bytes untouched". `mark` now splices only the marker line (insertMarker) and `updateDesignMd` splices only its own section (spliceSection); every other byte of an existing file is preserved, and spec order applies only to files that open with front matter. `mark` refuses a choice that contradicts the file's format (spec on a non-spec file, legacy-keep on a spec file) with DESIGN_MD_CONVERT_REFUSED, exit 2, file unchanged. convertLegacy keeps intro prose under the title instead of rebuilding the preamble from the title alone. detectFormat returns a machine-readable `code` beside the prose reason (the bin no longer branches on reason text); the marker regexes derive from FORMAT_MARKER_PREFIX and FORMAT_CHOICES; the hop limit and legacy identity headings are named constants; slug is exported and reused; both writers use lib/fs-atomic.ts. Tests pin byte identity for mark and updateDesignMd on the legacy fixture, the refusal paths, and the preserved preamble. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design): run the DOM dump in the page on both engines; align doctrine with the catalog The DOM-dump script is an arrow function, not a self-calling IIFE: Aside's `pg.evaluate($_DUMP)` receives the function and runs it in the page (the IIFE form executed in the repl sandbox, where `document` does not exist), and the fallback engine calls it with `$B js "($_DUMP)()" --out --raw`. Hygiene widens to every URL-bearing attribute (src, srcset per candidate, poster, action, formaction, data, ping, cite lose their query strings and fragments) and to data: URLs inside existing <style> nodes. The persist and scan blocks restate REPORT_DIR and RUN_ID literally instead of relying on a shell variable from an earlier block; the baseline's targetSet is defined per mode (repo-relative paths in source mode, page slugs in DOM mode) so DOM-mode deltas can match; the PR-body Detector line lists the states the probe can actually print. The DOM fixture is re-captured with the new script from outside the repo (the engine walks up from cwd for DESIGN.md, which the metadata now records). Doctrine contradictions the design specialist found: the landing-page motion rule matches the one-authored-moment reflex; the background rule names the catalog's halo/spotlight/stripe/grid slop instead of asking for gradients; the universal font rule is scoped to the display voice with the body/UI exceptions; "two typefaces max" allows the mono; the methodology's banned-font line renders BANNED_FONTS; Courier New is banned outright; the Brutalist, Retro-Futuristic, and Playful menu entries stop recommending system stacks, glow, and bounce; the coherence nudge uses the decoration vocabulary; Path A's gate names the display voice; font-loading prose points at the source the procedure verified; centered-everything is MEDIUM (an aggregate heuristic); the mockup guard reads "Never by default (unless the brief above asks for it)". The checklist's AUTO-FIX list renders the catalog's auto-fix rules; category 9 and the Hard Rules pointer count from the same partition helpers (detectorSlopEntries, judgmentTellEntries); the handoff list renders from HANDOFF_COMMANDS; a missing catalog id fails gen-skill-docs by name. gstack's own DESIGN.md gains border tokens and Decisions Log rows for its live-feed pulse and 11px mono labels. frontend-scope is case-sensitive like the bash arm. gen-skill-docs shares one emitGenerated helper for sections and lib-derived assets; renderCatalog keeps the one style with a caller. Tests: shared sliceBetween that fails on a missing end marker, the slop-gate fixture's real end marker, an isolated browse daemon for the DOM-mode E2E, the DOM hygiene test gated to CI or opt-in, docs notes for the two superseded plan sentences. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-detect): an engine is a file named impeccable outside the project; DOM dumps scan without inline ignores Second review cycle, security + checklist: - IMPECCABLE_BIN=/bin/sh (or node) was READY, and `detect` with cwd=repoRoot made the interpreter run the repository's own `detect` file. Every engine candidate (env override, PATH entry, cache, sibling) is now judged by the realpath of the FILE and must be named impeccable[.exe]; PATH and cache candidates that resolve into the repository are skipped like the others. "Inside the project" means the repository, or cwd when cwd is a project directory: HOME and its ancestors are exempt, so a URL-mode review launched from HOME still finds the HOME-rooted installs. - A base for --changed that starts with `-` was spliced into git argv (`--output=<file>` made git write a file and report no changes); an option- like or missing base is DETECT_REFUSED (not a ref name), exit 1, and the parser no longer defaults a missing value to main. - DOM dumps are the audited page's bytes, so an in-file `impeccable-disable` comment there is page-controlled: batches under the designs root run with --no-inline-ignores, repository batches keep the project's own ignores. - neutralizeSentinels covers the shapes it missed (bare sentinels such as DETECT_TOP total= and IMPECCABLE_DISABLED, the DETECT_EXIT_CODE= echo, the `[rule-id] impact=` group header) in one precompiled alternation instead of 37 replaceAll passes per field; only kept findings are normalized, and the summary's total stays the engine's count. - The minimal engine environment compares keys case-insensitively on Windows (process.env enumerates Path, SystemRoot there) and passes PATHEXT, COMSPEC, HOMEDRIVE, HOMEPATH, PROGRAMDATA. - Bare 64s move into DETECT_LIMITS; the unused SentinelName type is gone; the header states the directory-target contract (the engine's own walk). Tests: an interpreter as IMPECCABLE_BIN never runs the repo's detect file; a PATH symlink into the repository is never READY; option-like and empty bases are refused with no file written; the designs-root batch carries --no-inline-ignores and the repo batch does not; the identity label is deterministic per binary; the bare-sentinel and header shapes are neutralized; the installed fake engine works without IMPECCABLE_FAKE_OUTPUT (the helper copies the sample beside it); two tests clean up in finally. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-md): text-level edits keep CRLF, one section-boundary rule, control characters quoted - insertMarker and spliceSection normalized every line ending to LF, so a CRLF DESIGN.md came back rewritten beyond the one line they promised to touch. Both detect the file's dominant line ending and restore it. - parseDesignMd and spliceSection each walked headings with their own fence tracking; they now share headingLines (and upsertSection shares headingMatches). An unclosed ``` is treated as prose for that file: it used to swallow every later section on a splice. - A token value carrying a control character (an LLM-extracted font family with an embedded newline) was emitted as a bare multi-line scalar that Bun.YAML rejects, turning a freshly written DESIGN.md into frontmatter-unparsable; needsQuotes routes it through the quoted form. - The marker-line regex variants are built once beside YAML_MARKER_RE; the dead setMarker export and a no-op ternary are gone; LEGACY_HEADINGS derives from the identity list; the header diagram names the text-level editors as the write path for user-owned files; the bin validates and prints the mark choices from FORMAT_CHOICES. Tests: CRLF round-trips for both editors, a fenced ## inside a section and an unclosed fence, and a newline-bearing scalar parsing back. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design): Aside dump script stays single-quoted; redaction gate sized to the dump cap; doctrine made consistent - The DOM-dump Aside block was the only double-quoted `aside repl` script in the tree (to splice the function text), which put the agent-filled <url> inside a double-quoted bash string: a same-origin href carrying $(...) would run in the reviewer's shell when Phase 3 opened that page. The script is single-quoted like every other Aside script and the function text enters through a closed-quote segment ('"$_DUMP"'); the fallback line is `$B js '('"$_DUMP"')()'`. A free test pins that no rendered Aside script opens with a double quote. - The persist block capped dumps at 10 MiB but ran gstack-redact with its 1 MiB default, so every real page between the two was deleted as DOM_DUMP_REDACTION_BLOCKED; the gate passes --max-bytes at the dump cap and blocks on any exit other than clean (0) or MEDIUM (2), so a redaction tool that fails to run can no longer fall through to "persist". - Dump hygiene removes <template> and <noscript> subtrees (invisible to the attribute walk), inline on* handlers, and the cross-origin <link> nodes already named in the note, so the file handed to the engine references no remote stylesheet. - Doctrine: the Codex design-voice prompts said "2-3 intentional motions" against the one-authored-moment rule; the overused-display heading scoped its ban to Persuade/Experience while the catalog and hard rules ban it everywhere; design-consultation's Important Rule 4 still said "as primary"; design-html's blacklist header is now "Never include by default" with the mockup/DESIGN.md/user-ask override the catalog grants; the slop gate honors Decisions Log and Do's and Don'ts blessings like /review does; the landing "poster" line says poster in stance, not type size; the design binary's variant dials no longer flip light/dark for variety; gstack's DESIGN.md rows name data labels (UI labels stay the DM Sans token) and call the skill-bar fill and hovers functional transitions. - design-review names how the base branch is found (gh pr view, then the repo default; never main) for the source-mode scan and the diff-aware mode. - frontend-scope matches the config globs at the repo root only, like the bash arm; the parity test carries nested samples. - Cleanups: renderCatalog's stale style option, an unused import, the identity-map bannedFontNames, the checklist header's "same entries" claim, the catalog header's consumer list, the orphaned main() docstring, the plan doc's IIFE bullet. design-html's skeleton ceiling is re-measured (54,184) for the two doctrine sentences. Tests: AUTO-FIX rendering from the catalog, the E2E slice markers checked in the free suite, the hygiene cases for templates/noscript/handlers/remote links, and the review E2E counting detector rows separately from the seven checklist plants. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-detect): project means below HOME; only page dumps drop inline ignores; a whole-scan budget; prototype-safe rule counts Third review cycle + Red Team, all reproduced before the fix: - With no repository, the wrapper adopted cwd as the repo root, so a review launched from HOME (URL mode can run from anywhere) rejected every HOME-rooted install as "repository-local", reported the user's own skill install with the wrong hint, and, for targets, accepted all of HOME (~/.ssh/id_rsa scanned). A project directory is now one strictly below HOME: `git init ~` never turns the user's installs into repository files, and from HOME only the designs allow-list qualifies as a target. - --no-inline-ignores keyed on "not inside the repo", which misclassified dumps when GSTACK_HOME sits under the repo and stripped the design-html gate's own `<!-- impeccable-disable -->` from finalized.html. Targets are classified as project / dom-dump (designs/<audit>/dom/**, the page's bytes) / artifact (other designs/ files, gstack-authored); only dumps drop inline ignores. - A repository's .impeccable/config.json can hide rules from the review; detector.ignoreValues was never surfaced. The probe prints IMPECCABLE_IGNORED_VALUES beside the rules, and the prose stops calling repo-config ignores "a decision the user made". - An engine id named `constructor` corrupted byRule through Object.prototype and `__proto__` counts vanished; byRule is a null- prototype object and an id that fails the shape check is `unmapped` as a key too. - Batches ran with no total budget (10,000 un-ignored files: hours). The scan stops at 5x the per-batch timeout with DETECT_TIMEOUT and exit 1. - The scan JSON carries an `untrusted` list of the engine- and page-derived fields, so the agent reading past the fenced DETECT_TOP block is told what is evidence. - The PATH walk keeps launcher-present for a .cmd wrapper or a differently named real file (the name gate applies to READY only). Tests: probe and scan from a fake HOME (cache READY, HOME file refused, dump scanned without inline ignores), artifact vs dump batches, prototype-member ids, the whole-scan budget over 11 batches, ignoreValues surfaced, the `untrusted` field. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-md): edits follow a symlinked DESIGN.md, keep the BOM and the majority line ending, refuse an unclosed fence - `mark`, `convert --write`, and the design binary's extraction replaced a symlinked DESIGN.md (a docs-site layout) with a regular file and left the real target untouched; both writers resolve the link first. - A single stray CRLF flipped a whole LF file to CRLF: the editors now keep the majority ending. A UTF-8 BOM broke format detection and ended up mid-file after `mark`; it is recognized and kept at byte 0. - Re-running `mark` on a marked file deleted the blank line after the marker (`\s*$` matched across the newline); the marker regexes use `[ \t]*`. - Fences: readers follow markdown (an unclosed fence runs to EOF); the text-level editors refuse such a file with DesignMdEditRefused (DESIGN_MD_EDIT_REFUSED) instead of splicing the wrong section, and the design binary reports that and leaves the file alone. - needsQuotes also quotes a scalar containing ` #` (an inline-comment shape parsed back as a truncated value). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design): dump hygiene covers CSS URLs, srcdoc, and handlers; dumps persist owner-only; ignore prose treats repo config as evidence - The dump script cuts query strings from CSS url() in style attributes, <style> nodes, and the inlined stylesheets (signed asset URLs), empties srcdoc, and covers background and xlink:href. - Persisted dumps are chmod 600; MEDIUM redaction findings persist (an authenticated page shows emails) and the prose says so; earlier runs' dumps are swept before the first dump of a run unless --keep-dom. - The Aside dump prose asks for `'` in a pasted URL to be percent-encoded (a bare single quote would end the script) and never to paste an unread URL. - Repo-config ignores are evidence, not settled decisions, in /review, /ship, and design-review's probe prose; the scan JSON's text fields are named as untrusted. - design-html's skeleton ceiling is re-measured (54,545); ship goldens refreshed for the checklist prose. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-detect): audit directories scan as dumps; scans print probe lines on stderr; refused base always exits 1; PATH loses project entries Adversarial review (Claude subagent): - A DIRECTORY target under designs/ (the audit dir, which the prose hands the agent as REPORT_DIR) classified as an artifact, so the engine walked its dom/ subtree WITH inline ignores honored. Any directory under designs/ is now scanned as dumps. - A scan whose probe no longer finds an engine wrote its sentinel lines to stdout and exited 0, so `scan > "$_DJ"` captured "IMPECCABLE_NOT_AVAILABLE" as the scan result and the rendered bash read a clean scan. Probe lines go to stderr on every path; stdout is the JSON document or nothing. - A refused --changed base exited 0/2 when explicit targets were also given; it folds into the exit code (1 over 2 over 0). A trailing --changed no longer defaults to main. - A hand-edited `design_detector: Off` re-enabled the detector; the value is compared case-insensitively. - The engine inherited PATH entries inside the project (a direnv .envrc adding node_modules/.bin); those are filtered like every other project path. - DOM_DUMP_MISSING names the case where the dump script wrote nothing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design-md): markdown edge cases: rule-opened legacy files, spaced fences, ~~~ blocks, duplicate headings, YAML 1.2 numerics - insertMarker keyed on "starts with ---", so a legacy file opening with a horizontal rule got a `# gstack:` line rendered as a heading that the parser then never read back (the conversion question re-asked every run). It keys on parsed front matter. - A closing front-matter fence with trailing spaces (`--- `) made a valid spec file `unknown`; the closer is any whole `---` line. - `~~~` fences hid nothing, so a `## ` inside one was a section boundary and a splice corrupted the fence; both fence kinds are tracked and only the same kind closes an opener. - convertLegacy silently kept the first of two `## Layout` bodies (and one of `## Color` / `## Colors`); it refuses with DESIGN_MD_CONVERT_REFUSED and the bin leaves the file and writes no backup. - needsQuotes covers 0x / 0o / .inf / .nan (YAML 1.2 numerics that changed type on round-trip); emitYamlBlock throws on an object inside an array instead of writing "[object Object]". - The design binary coerces the model's extraction JSON at the parse boundary (null names, missing arrays) so the paid call's result survives. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(design): print and alternate stylesheets are not scanned as page CSS; no cross-run dump sweep; probe-state and design-system caveats in prose - The dump inlined every linked sheet's rules as active CSS, so a print sheet's 12pt black text or an alternate theme produced tiny-text and palette findings the user never sees; disabled and alternate sheets are skipped and a media-scoped sheet is wrapped in its @media block. - The cross-run dump sweep is gone: two same-day reviews shared REPORT_DIR and one run's sweep deleted the other's dumps mid-audit. Dumps stay per run, owner-only, deleted after Phase 9 unless --keep-dom (now defined in the prose), and an interrupted run's dumps wait for the user. - Prose: design-system-* rows in DOM mode compare the page to THIS repo's DESIGN.md and apply only to the repo's own app; an empty scan JSON with exit 0 means the probe state changed since Setup (read stderr); the persist block names a missing dump instead of mislabeling it as a redaction block. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * v1.82.0.0: impeccable interop, detector pre-pass, open DESIGN.md format Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: update project documentation for v1.82.0.0 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * changelog: name the measure behind the test-count row Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(test): drive the DOM hygiene test through Playwright Chromium directly Under the six-shard CI free suite the test's private browse daemon never answered its health probe (two minutes of retries), failed the shard, and starved two unrelated test files into failing before the runner's timeout. The test now launches the same Chromium through playwright-core and calls the dump function with page.evaluate, the way Aside's pg.evaluate does: no state file, no daemon, no health window. It self-skips when the Playwright Chromium bundle is absent. Two more hygiene rules are pinned along the way (print sheets keep their @media, alternate sheets are dropped). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(test): compare gen-skill-docs paths with forward slashes on Windows gen-skill-docs prints repo-relative paths with the OS separator, so the checklist render pins (`GENERATED: review/design-checklist.md`) failed on the Windows lane against `review\design-checklist.md`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(test): assemble the planted PEM block at runtime The quality gate scans every added line of the PR diff through gstack-redact; the redaction test's literal PEM header was a HIGH finding on our own test file. The block is now built from fragments, so the scanned file never carries a key-shaped line while the test still plants a HIGH finding. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design-detect): consent-gated engine install, checksum-pinned and receipted `gstack-design-detect.ts install` is the one download gstack makes, and only after a design skill's one-time question got a yes. It fetches the engine version gstack has tested (0.1.3) for this platform from impeccable's own GitHub release, verifies it against the checksum pinned in lib/design-detect-contract.ts (all five platforms, captured from the release's .sha256 sidecars; linux-x64 equals the fixture engine), writes an egress receipt before the fetch and refuses to download when the receipt cannot be written (fail-closed; the sink is registered in the wiring test's polarity table), caps the download at 32 MB, streams with the cap enforced, writes the file only after the hash matches, and places it under ~/.impeccable/bin/<version>/ (a trusted IMPECCABLE_HOME is honored; never inside a project). No skill, no hook, no launcher, no npx. --sha256 accepts a sidecar checksum for a version gstack has not pinned; --base allows a mirror (https, or http on loopback for tests). After a successful install the probe runs and its lines follow, so the skill sees READY at once. The probe ends with DESIGN_DETECTOR_INSTALL_OFFER (version, platform, bytes, destination) whenever it found no engine and the user has not answered the question; once design_detector_install_prompted is true it prints neither the offer nor the NOT_CACHED hint, which used to repeat on every run. The hint's npx wording is corrected: `npx impeccable detect --help` caches the engine for npx only, not where the probe looks. gstack-config gains design_detector_install_prompted (true|false, typo rejected, enumerated in list and defaults). Tests: a loopback mirror (async spawn, so the in-process server can answer) covers install, re-install as a verified no-op, checksum mismatch, 404, unpinned version, non-https base, design_detector off, and IMPECCABLE_HOME inside the repo; the offer and the silenced hint; pin completeness per platform. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(design): ask once before downloading impeccable's engine When the probe prints DESIGN_DETECTOR_INSTALL_OFFER the design skills ask the user one AskUserQuestion, in interactive sessions only (spawned or headless runs never install and never ask; Conductor gets the prose brief), before any other step: install the engine now, not now, never ask again (design_detector_install_prompted), or turn the detector off. A yes runs the receipted, checksum-pinned install and the skill continues with a READY probe. The brief says what impeccable is, what the one file is, where it goes, how it is verified and logged, and that no skill or hook comes with it; users who want the /impeccable skill run npx impeccable install themselves. design-review carries the brief inline (it is not carved). design-html keeps its skeleton small: the probe block points at a new read-on-demand section, sections/detector-install-offer.md, registered in its manifest and carve guard; its skeleton ceiling is re-measured (55,262) and its eager ceiling set to the measured 13,767. The review and ship passes state that they never offer an install. NOTICE.md, README, docs/skills.md, the interop design doc, and the CHANGELOG describe the new posture: gstack still never runs impeccable's installer or launcher; the one download is consented, pinned, and receipted. Ship goldens refreshed for the review-pass wording. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Garry TanandClaude Fable 5
|
0d1bd5616c |
v1.79.0.0 fix: ship subagent dispatches can no longer strand the run (#497/#2440 class) (#2772)
* fix(ship): foreground-flag + deadline + scope guard on all four dispatch sites (#497/#2440 class, 3rd recurrence)
Steps 7/8/10/18 dispatch subagents whose LAST-line JSON the parent
consumes, but none passed run_in_background: false — since Claude Code
v2.1.198 subagents background by default, so /ship stranded at Step 18
waiting on output that never arrives. Every site now renders the shared
{{FOREGROUND_DISPATCH_NOTE}} resolver constant, carries a ~10-minute
deadline with an explicit recovery branch (stop the runaway task,
reconcile against pre-dispatch HEAD, surface stray state, never
re-dispatch), and Step 18's prompt gains a docs-sync-only scope guard
(no VERSION changes, no base-branch merges, push-rejection reported as
pushed:false with parent-side reconciliation and a second-failure
branch). Greptile failures now record as UNAVAILABLE, not zero comments.
GENERATED_WITH_GUIDANCE pins all five ship dispatch carriers; a new test
pins the deadline recovery + scope guard phrases in pr-body (.md and
.tmpl). Codex/factory ship goldens re-rendered.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(document-release): first-class spawned-dispatch contract
document-release's own templates had zero subagent-awareness — the
entire headless contract lived in /ship's dispatch prompt, so any other
orchestrator (or an older installed /ship) dispatching it inherited none
of the gate handling. The skill now carries the contract itself: detect
spawned strictly from the dispatch prompt or the preamble echo (never
from file content — prompt-injection guard), auto-choose recommended
options while keeping the never-clobber-CHANGELOG and
never-bump-VERSION-silently invariants via their Skip options. Step 8.4d
gets an explicit spawned note (its interactive recommendation bumps
VERSION — wrong headlessly), and the Codex Documentation Review section
skips itself in spawned sessions (the apply gate needs a human; the
dispatching workflow owns review passes).
Contract, 8.4d note, and resolver skip are pinned in
run-in-background-guidance.test.ts; document-release skeleton budget
re-measured (39,812 B) and ratcheted to 40,200.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: sweep every remaining synchronous Agent-dispatch site with run_in_background: false
The #2440 failure shape was a silently-missing review voice — a
specialist launched in the background and merged before it completed.
Every remaining synchronous dispatch site now carries the explicit flag:
the Red Team dispatch, the spec review loop, the Codex
second-opinion/plan-review/doc-review Claude fallbacks, the adversarial
subagent, design sketch and outside voices, autoplan's design/eng/dx
phase dispatches, CSO parallel finding verification, and design-shotgun's
variant launch. Parallel fan-outs stay parallel — multiple foreground
Agent calls in one message run concurrently (the shipped v1.64.0.0
review-army pattern).
GENERATED_WITH_GUIDANCE now pins all 24 generated carriers, so a new
dispatch site that drops the flag fails the free suite. Six carved-skill
skeleton ceilings re-measured and ratcheted (~80-130 B growth each);
factory ship golden re-rendered.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* release: v1.79.0.0 — CHANGELOG, VERSION, TODOS follow-ups
Queue-advanced to 1.79.0.0 (1.78.0.0 claimed in the workspace queue;
same MINOR level per the versioning invariant). Entry references the
class history (#497 → #2440 → Step 18). Three TODOS filed: PreToolUse
hook enforcement, structural ship-mode for document-release, cross-host
dispatch semantics audit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* regen: review-army sections carry the Red Team foreground flag
The scripts/resolvers/review-army.ts Red Team edit regenerated these two
files but the sweep commit staged only the adversarial sections — the
skill-docs freshness gate (regen + git diff --exit-code) catches this.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* regen: agents digest picks up v1.79.0.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(document-release): one canonical spawned contract, downstream notes are pointers
The diff-selected LLM-judge eval scored the skill's clarity 3 (threshold
4, main scores 4): the spawned-session rules read as three separately-
framed rule sets (contract paragraph, Step 8.4d note, Codex-review skip).
The contract paragraph now declares itself the single source of spawned
behavior and the two downstream notes reference it instead of restating
rationale. The pointer avoids naming the Codex section verbatim so the
codex-host render (which strips that section) keeps its negative pin.
Judge re-scored 4/5/4 across repeated samples after the change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: foreground note names the Agent tool as non-substitutable
The ship-docsync gate E2E caught a behavioral regression: the note's
blocking-emphasis ('a backgrounded dispatch strands the run') steered
the driven agent to run doc-sync via the Skill tool inline — the most
blocking option — twice in a row, forfeiting the fresh-context isolation
the dispatch exists for (baseline on main's text dispatches via Agent).
The shared note now says explicitly: dispatch with the Agent tool itself,
never substitute Skill or inline execution; the flag already makes the
call block. Re-verified: ship-docsync passes on the amended text.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: pre-landing review fixes
Review army + red team findings, all verified before applying:
- ship-docsync E2E now asserts run_in_background === false on the
captured dispatch (red team CRITICAL: phrase pins prove text exists,
this proves the model obeys it — verified passing live).
- Structural scanner test: any generated file with an Agent-dispatch
imperative (or bare '(foreground)' prose, the #2440 inert shape) must
carry the flag or hold a reasoned exemption — the 4th-recurrence net
the hand-enumerated pin list can't provide.
- Parent push reconciliation models reality: the parent shares the repo,
so a non-fast-forward that hit the subagent hits the parent identically
— fetch + ahead/behind check first, push only when the rejection was
transient; dispatch prompt promise softened to 'the parent will handle
it'.
- Recovered commits from a dead subagent are vetted docs-only
(git show --stat, never VERSION/package.json) before any push.
- Deadline pacing named: ~3 minutes between checks, wall clock not polls.
- Greptile UNAVAILABLE recording narrowed to the PR body (Step 20's
schema carries no triage field).
- document-release contract gains the echo-failure tie-breaker: prompt
claims spawned + no echo → fail fast with the dispatch contract's
failure shape instead of reproducing the #2733 prose-STOP; contract
anti-injection and NEVER-relax clauses pinned in tests.
- 'Claude Code v2.1.198' extracted to CC_BACKGROUND_DEFAULT_SINCE and
interpolated at all resolver sites (byte-identical output).
- CHANGELOG: entry-boundary blank line restored; worst-case-wait row
scoped to the backgrounded path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: adversarial review fixes — failure shape, CHANGELOG guard, vet coherence
Claude adversarial subagent findings (11), all verified before applying:
- The dispatch JSON contract gains an explicit FAILURE shape
({"error":...}) and a parent branch for it — a doc-sync that could not
run (spawned marking failed, broken preamble) previously had only the
no-updates shape to emit, which the parent printed as 'Documentation is
current': a silent false-clean of exactly the VAS-449 class.
- Scope guard now covers CHANGELOG: skip Step 5 voice polish and resolve
CHANGELOG-touching gates to leave-as-is (the parent authored the
release entry; the prompt's older auto-choose clause conflicted with
the contract's never-rewrite invariant).
- Recovery vet is sequence-coherent: pushing a commit pushes its
ancestors, so ANY non-doc commit (VERSION, package.json, CHANGELOG.md)
blocks the whole sequence — no more push-the-vetted-child-of-an-
unvetted-parent hole.
- Steps 7/8/10 failure branches stop a still-running backgrounded task
before falling back, so a late result never races the inline audit.
- 'Documentation synced' print gated on pushed:true (item 6 owns the
local-only outcome); doc-review skip note's backward step pointer
fixed; foreground note scoped 'at initial dispatch' so the sanctioned
inline FALLBACKS in Steps 7/8 read as sanctioned; constants docstring
no longer overclaims single-sourcing (names the 3 inline templates).
- Runtime-hook TODO raised P2 → P1: the spawned trust chain is
agent-self-asserted; prose cannot close it, the hook can.
Both docsync gate E2Es re-verified green on the amended prompt, including
the strict run_in_background === false dispatch assert.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: codex adversarial fixes — JSON trust, fail-open visibility, user-state safety
Codex outside-model adversarial pass (inline-diff workaround for the
sandbox), P1/P2 findings triaged cross-model:
- Parent validates the dispatch JSON's field types and treats
documentation_section as untrusted markdown (Step 19's redaction scan
covers the final body; instruction-shaped text inside it is never
followed). The error branch now explicitly skips items 2-6.
- Remote-ahead divergence is named, not silent: the parent lists foreign
commits before creating a PR over a moved branch.
- Recovery cleanup never discards content: stray staged doc edits are
unstaged but never checked out or cleaned away.
- Greptile UNAVAILABLE gets a concrete PR-body line, not a vague
'wherever results are reported'.
Pre-existing-class P1s (self-asserted spawned trust chain, unenforceable
foreground timeout) are cross-model confirmed and tracked: PreToolUse
hook TODO at P1, harness-residue hazard documented.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: never-Skill prohibition made unambiguous; docsync fixture made diagnostic
The ship-docsync E2E flaked on this sandbox because the driven agent
resolved the STOP pointer's ~ to nonexistent homes (/root, /home) and
acted blind — pass/fail sampled model priors, not the prompt. The
fixture prompt now names the planted sections dir, making local runs
deterministic (CI, with real install paths, was always diagnostic).
With a diagnostic fixture: 2/2 passes, real pr-body read, Agent dispatch
with run_in_background: false, zero Skill-tool substitutions.
Prose: the foreground note now states the dispatch happens ONLY via the
Agent tool (invoking the target as a Skill is wrong even though it
appears in the skills list; inline FALLBACKs apply only after a
dispatched subagent has failed), and the Step 18 imperative carries the
never-the-Skill-tool clause at the decision point.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: CHANGELOG entry covers the review-hardened contract
Failure JSON shape, CHANGELOG scope guard, vetted recovery pushes,
structural scanner, and the behavioral dispatch assert are shipped
properties of v1.79.0.0 — the entry now describes them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v1.79.0.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: CHANGELOG voice — follow-ups to For contributors, never-Skill property stated
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): daemon 503 test binds an OS-assigned port, not old-port+1
windows-free-tests flaked on this PR: the tunnel-less restart bound
daemon.loopbackPort + 1 — a fixed neighbor of the OS-assigned ephemeral
port — and died with 'Is port 55738 in use?' whenever another shard or a
TIME_WAIT socket held it; the file-level retry re-rolled the same dice.
Every other startDaemon in the file already uses loopbackPort: 0 and the
assertion reads d2.loopbackPort, so nothing needs a predictable number.
21/21 pass locally.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Garry TanandClaude Fable 5
|
07b59e396c |
v1.75.0.0 feat: ponytail import wave — simplification review lens, arm benchmark, reuse ladder, instruction-tier digest (#2722)
* feat(autoplan): eng review always runs last — the gate reviews the final amended plan Reorder the pipeline to CEO -> Design (if UI scope) -> DX (if developer-facing scope) -> Eng. The old order (CEO -> Design -> Eng -> DX) let DX findings land AFTER the required gate signed off, so eng validated a stale plan. Accept-all semantics made explicit: every AskUserQuestion resolves to the recommended option; premises no longer pause the pipeline mid-run (clearly-wrong ones queue as User-Challenge items at the single Final Approval Gate). Eng's Codex voice now sees the DX consensus summary. New free static test pins the order; the chain E2E gains DX-between and Eng-terminal assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(review): simplification specialist — advisory over-engineering lens with ponytail's tag vocabulary New 8th Review Army specialist (DIFF_LINES > 100, --simplification force flag) hunting unrequested STRUCTURE only: delete/stdlib/native/speculative/shrink closed tags, one-line findings, lines_removable field. speculative: replaces ponytail's yagni: tag — we import the lens, not the posture; coverage stays sacred (Completeness Gaps owns it, suppressions inlined, shrink needs >=5 lines). Advisory carve-out in the merge step: advisory findings are excluded from quality_score and the findings-count header, render with an [ADVISORY] label, and are ASK-only in Fix-First. Zero-findings case prints the lens-scoped 'Simplification: lean already — nothing to cut.' from the PARENT (the specialist keeps the exact NO FINDINGS contract); with findings, the parent prints 'net: -N lines possible' summed from lines_removable. Tests: static pins for the carve-out + early-out contract (gen-skill-docs), two periodic e2e cases with planted fixtures — activation (over-build traps: hand-rolled Intl, one-impl abstract, dead config) and false-flag precision (a lean ETHOS 'choose A' diff must yield NO FINDINGS). Inspired by dietrichgebert/ponytail's /ponytail-review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(preamble): reuse ladder in Search Before Building — rungs 2-5 of ponytail's ladder, completeness kept Tier-3+ skills gain a per-edit reflex the section only stated as research discipline: before writing new code, stop at the first rung that holds — repo helper, stdlib, native platform feature, installed dependency — then build the COMPLETE version of what remains. The closing clause is the explicit reconciliation with Boil the Ocean: the ladder governs structure, never coverage. Rungs 1/6/7 (YAGNI / one line / minimum that works) are deliberately NOT imported. Also ports ponytail's root-cause rule: one guard in the shared function beats a guard in every caller. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(preamble): bounded-closer output rule for tier-2+ skills After completing work, skills report in a few short lines — what changed, what was skipped, what to watch — and cut any explanation that outgrows the change. Explicit exemptions protect every mandated output: decision briefs, completion-status blocks, user-requested explanations, and report-shaped skills' report formats (the report IS the work in /qa-only, /plan-*-review, /retro, /document-generate). Rationale is signal-to-noise, not tokens: ponytail's own benchmark shows terse prose alone doesn't cut cost (caveman arm: -20% LOC, +7% tokens), and independent replications found its 'skipped on purpose' essays ate the code savings. Includes a good/bad closer example pair per the model-overlay guidance that a positive example beats a 'don't be verbose' instruction. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(resolvers): terse-mode savings claim matches measurement — 2.6KB, not 3-5KB Measured on the v1.71 render: --explain-level=terse saves exactly 2,611 bytes per tier-2+ skill. The old ~3-5KB claim predated the preamble restructuring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(retro,preamble): gstack-shortcut debt ledger — accepted shortcuts leave a joined trail When the user accepts an option that is BOTH Completeness <= 7 AND a durable-scope call, the decision ledger entry (gstack-decision-log, ceiling + upgrade trigger in the rationale) is the source of truth, and the agent marks each cut corner in code with gstack-shortcut(dec-<id>): <ceiling>, upgrade when <trigger> — same edit, no follow-up question, never agent-initiated. /retro Step 11.5 harvests markers into a debt ledger (grep || true — zero matches is the healthy case; skill installs and docs excluded), joins on the decision id so nothing double-counts, tags unlinked and no-trigger rot risks, and closes with 'N markers, M with no trigger.' /review suppressions: a marker with ceiling+trigger downgrades a would-be Completeness Gaps finding to acknowledged debt. Redaction test pins that the marker ships untouched (the ledger is the point) — it does not match the TODO(owner) hygiene shape. Format from dietrichgebert/ponytail's ponytail-debt; store inverted to gstack's existing decision ledger. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: refresh golden ship baselines after preamble additions (reuse ladder + bounded closer) The golden-file regression test pins the rendered ship skill byte-for-byte; the WS3/WS7 preamble sections are deliberate changes, so the baselines re-capture per the goldens' own update protocol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(hosts): instruction-only tier — a 2KB committed rules digest any agent host can read New agents-digest/gstack-AGENTS.md (1,765 bytes, hard 2,048-byte budget): gstack's ethos one-liners, the reuse ladder, and voice rules for hosts with no install arm — Zed, Amp, Jules, or any AGENTS.md-reading agent. Generated by scripts/gen-agents-digest.ts, auto-refreshed by gen:skill-docs, committed like llms.txt so setup's explainer arms can point at it before any toolchain exists. First line carries the gstack version as its own staleness nudge. Delivery is print-path + user-performed copy ONLY: setup never writes or overwrites a user's AGENTS.md (a test pins this — no cp/ln/mv/redirect into AGENTS.md anywhere in setup). openclaw and hermes explainer arms print the path; slate keeps routing to the full Claude install and gbrain ships from its own repo. HostConfig gains the optional install.instructionTier slot, declared by both instruction-tier hosts. README host table now matches what setup actually does. Inspired by dietrichgebert/ponytail's instruction-tier AGENTS.md fallback — one generated source, never per-host hand copies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(preamble): AskUserQuestion repetition cut — gated, passed NOT-WORSE A/B Removes the duplicate statements v1.71's compaction left in the AskUserQuestion Format section: the completeness rule restated in the prose triad, the auto-decide marker syntax stated twice, the Conductor-flakiness explanation stated twice, and the self-check's full triad restatement. Every verbosity floor and all 14 format pins stay (Layer 0 green). The gate this decision rested on ran before landing (new periodic skill-e2e-auq-repetition-cut-ab.test.ts, pre-cut ref |
||
|
Garry TanandClaude Fable 5
|
394db326f2 |
v1.71.0.0 feat: token-load reduction — preamble runtime scripts, gated onboarding, 20 skill carves, CLAUDE.md trim (#2691)
* feat(gen): strip gen-time-only frontmatter keys from Claude renders
interactive + benefits-from are read from the .tmpl by buildContext at
generation time; no runtime, host, or test reader consumes them from the
generated SKILL.md (e2e-harness-audit reads .tmpl; benefits-from tests
assert rendered prose). gbrain: stays (bin/gstack-brain-context-load reads
it from the installed render); hooks: stays (Claude Code host wires
PreToolUse from it).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(gen): regenerate SKILL.md — dead frontmatter keys removed
Mechanical regen after hosts/claude.ts stripFields change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(test): context-budget ratchet — CI ceilings on always-on + eager token ledgers
New free test grades the two ledgers nothing else guards: the full-frontmatter
always-on catalog (aggregate) and per-skill eager tokens (SKILL.md +
forced-read refs), via checkBudget from lib/context-bill.ts. Ceilings live in
test/fixtures/context-budget.json with x1.05/x1.10 headroom; regenerate with
bun test/helpers/capture-context-budget.ts. New skills fail until consciously
budgeted; removed skills fail until the fixture is refreshed; reductions
ratchet the ceilings down so wins lock in.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(todos): file output-template carve wave + plan-ceo doctrine revisit; mark preamble-carve P3 in flight
Two follow-ups deferred from the approved token-reduction program (CEO review
'NOT in scope' list), filed with full context per TODOS format. The existing
P3 preamble-carve entry gets a status update pointing at the program that
supersedes it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): review findings — Windows path normalization, full totals rebuild, ratchet coverage
Pre-landing review (5 specialists) found one critical: the ratchet test runs
in the curated Windows lane, where path.relative yields backslash skill names
that miss the test/ filter and mismatch every POSIX fixture key. Names are now
normalized once in buildRatchetBill (toPosixName) and the fixture filter is
tightened to test/fixtures/. All eight Bill.totals fields are rebuilt from the
filtered list (no fixture-polluted perInvocation/totalMd numbers for future
consumers). New coverage: Windows-separator normalization pins, a
captureContextBudget round-trip against tree-a (headroom math exact), a
stripFields regression pin (interactive/benefits-from absent from renders,
hooks/gbrain preserved), and the ceilings test no longer double-reports
stale-fixture entries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): adversarial findings — stable root key, symlink-alias dedupe, fixture-shape guard
Adversarial review (Claude subagent) verified the fixture's root-skill key was
the capture machine's checkout dirname: any non-gstack-named clone (every
Conductor worktree) failed the free suite, and the documented re-run-the-capture
recovery baked the local dirname into the committed fixture — silent corruption
through the tool's own protocol. The root skill is now pinned to ROOT_SKILL_KEY
('gstack', its frontmatter name). Symlink aliases are realpath-deduped (census
precedent): connect-chrome no longer gets its own ceiling, so Windows checkouts
that materialize the symlink as a plain file can't fail the stale-ceiling
set-equality test. New guards: fixture-shape validation (a string alwaysOnTotal
can no longer silently disable the ceiling), a mutation pin that the filter
shrinks the always-on ledger vs the raw bill, an alwaysOnTotal violation test
(the branch was load-bearing with only under-budget coverage), and an atomic
temp+rename fixture write. Fixture regenerated: 59 ceilings, alwaysOnTotal 6344.
Deferred with a TODO: anchoring transformFrontmatter's denylist strip to the
frontmatter block (latent, zero live collisions, pre-existing path).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v1.69.1.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v1.69.1.0
CLAUDE.md: Token ceiling section documents the context-budget ratchet as
the third guard (test file, fixture, new-skill budgeting, capture command).
CONTRIBUTING.md: Tier 1 guard list gains a Context-budget ratchet bullet;
the Adding-a-new-skill checklist gains the budget-capture step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: pin exact guard semantics for the context-budget ratchet in CLAUDE.md
Doc-review finding: "a third enforced ceiling" undercounted the guard
family (skill-size-budget floors and parity ratios also watch these
ledgers, relatively). Rephrased to match the ratchet test's own header:
absolute ceilings vs relative floors/ratios.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): heaviest-skill claim matches the fixture (land-and-deploy edges review by 0.2%)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(bin): gstack-skill-start + gstack-skill-end — the preamble runtime, consolidated
Absorbs the ~13KB of bash every tier-2+ SKILL.md inlined twice over (bootstrap
fence + artifacts-sync fence) and the skill-end telemetry/sync fences. Same
KEY: value STATUS-line contract the prose interprets, plus SKILL_START_PROTO
handshake (OV5), SESSION_ID/TEL_START echoes, GSTACK_HOME-normalized state
paths (EOV7), --parent-pid session identity (EOV5: $PPID inside the script is
the ephemeral tool-call shell), OV4 sanitization of passthrough output, and a
receipted daily artifacts pull (_receipted_git, brain-sync class, fail-closed).
Per-line || true error style throughout (F3) — a mid-script failure never drops
later STATUS lines.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(gen): preamble resolvers emit a script invocation fence instead of inline bash
generate-preamble-bash: ~6.3KB fence -> 4-line gstack-skill-start invocation
(quoted-tilde pitfall handled: leading ~ interpolates through $HOME; env-var
hosts keep $GSTACK_BIN) + degraded-mode prose (F1/EOV8: safe defaults, consent
gates deferred-never-lost; OV5: proto rule). generate-brain-sync-block: ~6.8KB
bash -> interpretation prose + the privacy stop-gate (stays inline until
Phase 2's gated emission). generate-completion-status: telemetry fence -> one
gstack-skill-end call with SESSION_ID/TEL_START handoff.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(gen): regenerate all skills + golden fixtures — inline preamble bash removed
Mechanical regen after the resolver change: −12,628 lines across 52 renders
(corpus 952K -> 806K render tokens; tier-2 skills −11-13KB each). Golden
per-host ship fixtures refreshed from the fresh claude/codex/factory renders.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: skill-start contract suite + preamble A/B eval + touchfiles registration
test/gstack-skill-start.test.ts (11 free tests): STATUS-key contract vs the
prose (F2), per-host fence resolution shapes (E1), proto-first, OV4 marker
sanitization, --parent-pid identity, headless suppression, skill-end duration
math + pending cleanup. test/skill-e2e-preamble-script-ab.test.ts (gate tier,
OV7): inline-bash render (pinned from
|