CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-08-20 13:07:17 +02:00
Code Issues Packages Projects Releases Wiki Activity
369 Commits 326 Branches 0 Tags
1cab5e11083a37ea0bc62117e9a9c5d05d68785e
Commit Graph
4 Commits
Author SHA1 Message Date
Garry TanandClaude Fable 5 1cab5e1108 v1.66.1.0 feat: content binding — evidence ledger, wtree staleness, tracker trust envelope, fail-closed hooks (#2603) 
* fix(hooks): fail-closed freeze + shared extractor + careful HIGH tier

Freeze boundary hook had four verified bugs: the grep-first JSON extractor
truncated at escaped quotes and failed OPEN on unparseable payloads; the deny
JSON was printf-interpolated so a quote- or newline-bearing path silently
no-oped the block; the freeze path read stripped INTERNAL spaces (a boundary
like ~/My Project could never match); and the path resolver skipped the final
component, letting an in-boundary symlink write through to an out-of-boundary
target.

Fixes, structurally: one shared sourced helper (careful/bin/hook-extract.sh)
now owns JSON extraction and JSON-encoded decision envelopes for BOTH hooks --
the two-copy drift is how freeze kept a broken extractor after careful's was
fixed. Freeze is now deny-tier fail-closed (unparseable payload denies,
parsed-but-no-file_path still allows), trims only leading/trailing whitespace,
and resolves symlinks through the final path component.

Careful gains a HIGH tier (hard deny, simple commands only): recursive delete
of /, ~, or $HOME, and force-push to the repo's default branch. Compound
commands always fall through to the MEDIUM ask; --force-with-lease is never
HIGH. Documented as a best-effort advisory hard-stop, not a policy boundary.
Plus additive-only project patterns (~/.gstack/careful-patterns.txt +
per-project file): config can only ADD warn rules, never suppress a baseline
family.

test/hook-scripts.test.ts: 89 tests incl. malformed-payload deny, parseable
deny JSON for hostile paths, space-bearing boundaries, symlink escape, HIGH
tier splits, additive invariant, invalid-regex resilience.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(review): content-addressed staleness via working-tree fingerprint

Review records now bind to the content they were made on. bin/gstack-review-log
stamps every appended record with commit_full, tree, dirty (informational) and
wtree — a working-tree fingerprint from the new bin/gstack-wtree (temp index
seeded from HEAD + git add -A + write-tree). The binding fields are computed
authoritatively; caller-supplied values for those keys are ignored, so a stale
rendered template or a forged field can't bind a record to content it wasn't
made on.

Why a working-tree fingerprint instead of HEAD^{tree}: committing identical
content doesn't change it (a record made on a dirty tree stays valid after the
same content is committed), untracked new source files DO change it (new code
can't hide from freshness), and gitignored scratch stays out. Rebase, amend
and squash with identical content grade CURRENT instead of stale.

Grading: the dashboard (scripts/resolvers/review.ts) and /land-and-deploy Step
3.5a apply a content-first rule to diff-scoped review rows — wtree match with
both sides clean is CURRENT, full stop. Plan-tier reviews grade a plan file,
not the repo tree, so they keep the 7-day logic (optional plan_sha256 caller
field noted). The rev-list fallback no longer errors when the stored commit
was rebased away: it grades UNKNOWN and treats it as stale.
bin/gstack-review-read emits ---WTREE---/---TREE---/---DIRTY--- so graders
consume one tool output. Old records without wtree fall back to the existing
heuristics; no migration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(evidence): verification-evidence ledger mechanizes /ship's IRON LAW

New bin/gstack-evidence: a transparent wrapper that records every verification
run as {ts, label, command, cmd_sha256, exit, duration_s, commit, tree, dirty,
wtree, log_path} in ~/.gstack/projects/<slug>/<branch>-evidence.jsonl, plus a
read-only `check` that grades FRESH/STALE/MISSING per label. "Tests passed"
now binds to the exact working-tree content it ran on (bin/gstack-wtree
fingerprint), so evidence recorded on uncommitted code stays FRESH after the
exact tested content is committed — the /ship Step 5 -> Step 16 case — while
an untracked new source file or any content change invalidates it.

Check semantics: every named label's latest record must be green, within
--max-age, matching --expect-cmd's hash when given, and fingerprint-identical
(or diff confined to --allow-paths — mechanizing Step 16's existing "CHANGELOG
edits don't count" carve-out). No --any mode: a green lane can never mask a
red sibling. Any git failure inside check (gc'd tree object, not a repo)
degrades to STALE/MISSING, never an error into the calling skill flow.

Transparency invariant (load-bearing, test-pinned): the child's exit code is
ALWAYS the wrapper's exit code; ledger/log/redact failures are stderr
warnings. Logs are per-run (0600, exclusive-open, 2MB truncation marker,
30-day opportunistic prune) — no more shared /tmp collisions between
concurrent ships. Command strings are redact-scanned before recording (HIGH
credential -> stored redacted). Machine-local by design: neither ledger nor
logs brain-sync.

Wired: ship Step 5 lanes run wrapped (per-lane labels), ship Step 16 and
land-and-deploy 3.5b check the ledger first and cite FRESH evidence instead of
re-running; a failed CHECK never blocks (run live), a failed RUN does.
test/evidence.test.ts: 21 tests incl. the keystone dirty-record -> commit ->
FRESH case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(security): trust envelope for tracker text at every model-context ingress

Web page content has had a trust envelope since v1.38; tracker text did not —
PR bodies, PR/issue comment bodies, and model-judged issue titles entered
agent context raw. Anyone who can comment on a PR could put instructions in
front of the agent.

New lib/tracker-guard.ts + bin/gstack-issue-guard: every tracker-text read now
emits inside a "BEGIN UNTRUSTED TRACKER CONTENT" envelope. Content is enveloped
even when clean (a pattern scan is not proof of safety); injection-shaped lines
get a visible [INJECTION-PATTERN] label; NFKC + zero-width normalization runs
for DETECTION only (fullwidth/invisible evasion caught, content bytes never
rewritten); forged END banners are zero-width-spliced so they can't close the
envelope early. Fetch failure exits non-zero with NO envelope — never a
fake-trusted empty one. Issue numbers are validated and gh is spawned via argv
arrays. Patterns reuse lib/jsonl-store's INJECTION_PATTERNS single copy plus a
separate TRACKER_EXTRA list (kept separate so decision/learning store
write-rejection semantics don't change).

8 sites wired: greptile findings + replies fetches (metadata/body split — ids
and paths stay machine-raw for reply POSTs), review.ts PR-body reads x2,
land-and-deploy 3.5c, document-release PR/MR body (two-artifact flow: the
enveloped rendering is what the agent READS, the raw tempfile is what the
pipeline mutates, and a write-side banner tripwire aborts any edit that leaked
envelope markup), and spec's issue-title dedupe (titles are model-judged for
similarity, so they're ingress). Title-prefix rewrites and state-routing
fetches are mechanical, not ingress — deliberately not enveloped.

test/tracker-guard-wiring.test.ts is the CI tripwire: raw tracker-text reads
outside the guard fail the suite unless carried by a reasoned SCANNER_EXEMPT
entry; exemptions are liveness-checked so a moved site forces a re-audit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(binding-wave): drift tripwire, golden fixtures, TODOS follow-ups

test/binding-template-drift.test.ts pins the load-bearing prose rules in the
GENERATED templates (ship Step 16 evidence check, per-lane wrapped test lanes,
land-and-deploy wtree-first grading + UNKNOWN fallback, dashboard content-first
rule, release-body banner tripwire, greptile guard pipes) so a template
refactor can't silently drop a rule while the bins keep passing their unit
tests.

Golden ship fixtures re-pinned to the new intentional output (claude/codex/
factory variants). TODOS.md gains the five deferred follow-ups from the review
wave: eval-run evidence records, spec-spawn outcome ledger, merge-SHA custody,
default-if-silent escalations, and the paid eval case proving agents apply the
staleness grading rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(careful): trim HIGH-tier + project-pattern docs under the size budget

The new sections pushed careful/SKILL.md to 2551 -> 3879 bytes (x1.52, gate
caps growth at x1.5 of the v1.47 baseline). Same content, tighter prose:
3516 bytes (x1.38).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(tests): scratch-repo fixtures never invoke the operator's gpg

The evidence/review-log/hook fixtures inherited global commit.gpgsign, so
fixture commits called the operator's gpg-agent — which fails with "Cannot
allocate memory" under parallel shard load, breaking test SETUP (not the code
under test). All fixture git invocations now pass -c commit.gpgsign=false
-c tag.gpgsign=false. Hermetic repos, no pinentry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: pre-landing review fixes (27 specialist findings, 3 critical)

Specialist army findings, all quote-verified before fixing:

Security: careful force-push guard now catches git's plus-refspec force
syntax (git push origin +main carried force with no flag — silently allowed
before) and refspec-form targets (HEAD:main); default-branch matching is
tokenized FIXED-STRING comparison on the full branch path (slashed defaults
like release/2.0 work; no ERE interpolation), glob-safe via noglob. HIGH rm
tier is tokenized too: trailing long options (--no-preserve-root) and /* are
root-class. Stored evidence fingerprints are 40-hex re-validated before
reaching git argv. normalizeForDetection sweeps ALL Unicode format chars
(\p{Cf}: soft hyphens, bidi marks, tag chars) instead of five enumerated
zero-widths. The wiring scanner gains flagless gh pr/issue view patterns. The
release-body banner tripwire diffs against the fetched original so a hostile
pre-existing banner string can't permanently DoS doc updates. Ship/land
evidence checks now pass --expect-cmd (a green `echo ok` recorded under the
label can never mint FRESH); package.json stays allow-listed with the
residual documented.

Performance: gstack-wtree seeds its temp index by COPYING the real index
(stat cache preserved — measured 40x faster than read-tree seeding, identical
hash) with read-tree fallback; evidence uses findLast and one gstack-slug
spawn; the stream pump honors backpressure via drain; careful's pattern block
short-circuits before slug resolution when no pattern file exists.

Testing: the gh-failure envelope test was VACUOUS (killing PATH killed the
bun shebang before the code under test ran) — replaced with a PATH gh shim
that exercises the real branch, plus shimmed happy paths (issue/pr-body/
unparseable JSON); evidence check --all + empty ledger + non-numeric
--max-age (now a usage error, was silent fail-open) covered; HIGH-tier
variants pinned; hook analytics respect GSTACK_HOME so tests stop writing the
operator's real skill-usage.jsonl.

Maintainability: dead exit ternary removed; flagValue deduped into
bin-context; sentinel defusal derived from the banner constants (no invisible
literals — \u escapes only); scratch-repo git fixture extracted to
test/helpers/scratch-repo.ts (one hermetic incantation, three consumers);
shared gstack_hook_log_fire in hook-extract.sh; the dashboard/land diff-scoped
row lists are aligned (codex-review) and drift-pinned.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: red-team review fixes (9 findings, 2 critical)

Red team reviewed what four specialists missed — cross-cutting and
self-contradiction class:

CRITICAL: the release-body banner tripwire failed OPEN on the exact leak it
guards (grep -c prints 0 AND exits 1 on no-match, so a fallback echo
double-emitted "0" twice and the -gt comparison fell into the clean branch) —
counts now default via parameter expansion, and a functional drift test
executes the rendered tripwire block against a 0->1 banner delta to prove the
ABORT branch fires. CRITICAL: evidence fingerprints were captured AFTER the
child exited, so a working-tree edit made DURING a long suite was certified as
tested content — wtree is now captured before spawn and re-checked after;
mid-run drift omits the fingerprint (grades STALE) with a warning.

Also: the review-grading rule dropped its dirty-gates (they nullified the
keystone dirty-record->commit->CURRENT property that evidence checks already
honor — wtree equality alone proves identical content); careful's HIGH
force-push tier falls back to probing origin/main|master when the origin/HEAD
symbolic ref is absent (Conductor worktrees — the tier was silently inert in
the primary deploy environment); quoted tokens (rm -rf "/", push "main") no
longer dodge the deny; freeze fails CLOSED when its own helper file is missing
(bash makes a missing source target fatal non-interactively, so an existence
pre-check guards it); spec dedupe distinguishes pipeline failure from zero
matches instead of silently skipping dedupe on gh/jq breakage; land 3.5b sets
the cross-session --expect-cmd mismatch expectation; hook analytics JSON
fields are encoder-built per this wave's own rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: re-pin codex/factory golden fixtures post-regeneration

The suite regenerates .agents/.factory in place mid-run; the prior pin
snapshotted them before the dashboard-rule regen landed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: bump version and changelog (v1.66.1.0)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: adversarial review fixes (Claude pass, 14 findings, 1 verified-live critical)

The fresh-context adversarial pass caught a live bug in this branch's own
performance fix: gstack-wtree exported GIT_INDEX_FILE BEFORE resolving the
real index path, so `git rev-parse --git-path index` returned the temp index
itself, the stat-cache copy self-copied and failed, and every invocation fell
back to the full re-hash — the fast path was dead code (verified with bash -x).
Resolution now happens before the export; measured 0.08s per call on this repo.

Also fixed: careful fails to an ASK (not silence) when its own helper file is
missing (same partial-install state freeze already defends against); the
--source label is sanitized inside the envelope lib (newline-stripped,
sentinel-defused, length-capped — it sits in trusted framing); the HIGH rm
tokenizer skips redirections/backgrounding/`--` (rm -rf / 2>/dev/null now
denies) and knows ${HOME}; user pattern lines starting with a dash work
(grep --); greptile bodies carry per-comment id headers inside the envelope so
multi-comment PRs stay attributable (ids verified against raw metadata, never
trusted in-body); the release-body tripwire fails CLOSED when its input files
are missing (separate-shell $$ reality); land 3.5b gets the same allow-paths
as ship; the "either side dirty" fallback leftover is gone from both grading
surfaces; the evidence pump races drain against error (EPIPE consumers can't
hang the wrapper); an unset HOME skips bookkeeping instead of creating a
literal ~ dir inside the repo; a write-failure log ends with a visible marker;
freeze expands a literal leading ~ in the boundary; review-log documents its
log-time binding window.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: pin golden fixtures from --host all generation

`bun run gen:skill-docs` generates the claude host only; .agents/.factory
regenerate when the suite's --host codex/factory tests run in place. Fixture
pins must come from `gen-skill-docs --host all` output or they lag one
resolver edit behind and fail the next full-suite run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: assemble the fixture PAT by concatenation (no live-format literal)

The repo's own pre-push credential guard (correctly) blocked the push: the
redaction test's fabricated GitHub PAT was a live-format literal in the diff.
The token is now concatenated at runtime — the source carries nothing the
scanner can match, the engine still receives a live-format value.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: update project documentation for v1.66.1.0

CLAUDE.md: add gstack-wtree/gstack-evidence/gstack-issue-guard to the bin/
structure line and tracker-guard.ts to the lib/ line. README.md +
docs/skills.md: /careful descriptions no longer claim every warning is
overridable — the HIGH tier hard-denies root/home recursive deletes and
default-branch force-pushes; skills.md also documents the additive-only
careful-patterns.txt warn rules.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: doc-review fixes — new bins in README table, careful claims precise

README.md: add gstack-wtree, gstack-evidence, and gstack-issue-guard to the
Standalone binaries table (they shipped in v1.66.1.0 with no user-facing
reference outside CHANGELOG). docs/skills.md: the safety-skills intro said
"no configuration files" which the optional careful-patterns.txt now
contradicts, and the hard-deny description undersold the deny set (the hook
also denies /*, ~/, and $HOME/ forms, not just bare / and ~).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: guard reflects the hard-deny tier; changelog stats current

guard/SKILL.md claimed every destructive warning was overridable — the shared
careful hook now hard-denies the catastrophic shapes. CHANGELOG numbers
updated to the final measured state (0.09s fingerprint, 50 findings/6
critical across all review passes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-16 09:53:31 -07:00
Garry TanandClaude Opus 4.8 a5833c413f v1.57.10.0 feat: Codex review default-on across review/ship/plan/docs (#1966) 
* feat(config): make codex_reviews the master switch for all Codex review

Broaden the codex_reviews doc to describe it governing /review, /ship,
/document-release, plan reviews, and /autoplan. Reject invalid values on
set (preserving the existing value) so a typo can never silently flip
paid Codex calls on or off.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(review): Codex review default-on across review/ship/plan/docs

Add a shared codexPreflight() helper (constants.ts) that, in one bash
block, reads codex_reviews, sources gstack-codex-probe, checks install +
auth, and echoes a single canonical mode (ready/not_installed/not_authed/
disabled). All Codex resolvers route through it.

- generateCodexPlanReview: opt-in question removed; the outside voice now
  runs automatically (default-on), falling back to a Claude subagent when
  Codex is missing/unauthed. Cross-model tension still gates on user
  approval (sovereignty preserved).
- generateAdversarialStep: probe-based availability (install AND auth),
  distinct not-installed vs not-authed guidance; 200-line structured-review
  threshold unchanged.
- generateCodexDocReview (new, wired via CODEX_DOC_REVIEW): reviews the
  release's docs against the shipped diff range, informational + an explicit
  apply-fixes decision point, never auto-edits.
- autoplan Phase 0.5 now honors codex_reviews=disabled so the switch is
  truly global.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(docs): regenerate SKILL docs + refresh ship golden

Output of gen:skill-docs for the Codex-default-on resolver/template
changes. Refreshes the factory-ship golden fixture (codex-host output
unchanged — resolvers strip for the codex host).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(infra): widen size-budget guards for default-on Codex outside-voice

The codexPreflight() block + CODEX_MODE branch prose (replacing the
smaller opt-in question) grows plan-ceo/eng/devex-review and review by
5-7% over baseline. Each bump carries a comment justifying it as
intentional capability, not slop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: guard Codex default-on + config reject-on-set

skill-validation: assert plan reviews no longer carry the opt-in question
and render the default-on outside-voice, document-release carries the doc
review, and the codex host strips all of it.

gstack-config: codex_reviews defaults to enabled, accepts enabled/disabled,
and rejects an invalid value while preserving the existing one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(test): align gstack-config tests with defaults-fallback behavior

Three tests (last touched v0.13.7.0) asserted get/list print empty for
unset keys, but gstack-config falls back to the documented defaults table
(get returns the default, list shows the active-values block). Update the
assertions to the real behavior and split out an unknown-key case that does
still return empty. Pre-existing red, unrelated to codex review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* v1.57.10.0 feat: Codex review default-on across review/ship/plan/docs

Codex cross-model review now runs by default on /review, /ship, all four
plan reviews, /document-release, and /autoplan, governed by one master
switch (codex_reviews, default enabled). Plan-review outside voice is
default-on; /document-release gets a new Codex doc-vs-diff audit; every
call site detects install AND auth and falls back to a Claude subagent
with a clear reason. Disable everything with:
gstack-config set codex_reviews disabled

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 21:14:58 -07:00
Garry TanandClaude Opus 4.8 1626d4857b v1.57.7.0 feat: GSTACK REVIEW REPORT always declares unresolved decisions (#1916) 
* fix(plan-devex-review): add missing gstack-review-log step

plan-devex-review carried the EXIT PLAN MODE GATE but never wrote a
review-log entry, so the gate's 'review log was called' check was
structurally unsatisfiable and the Review Readiness Dashboard / GSTACK
REVIEW REPORT had no plan-devex-review data to read. Add a Review Log
section before the dashboard read, logging the devex fields the report
parser already expects (status, scores, product_type, tthw, persona,
competitive_tier, unresolved, commit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(review): make unresolved-decisions status mandatory in GSTACK REVIEW REPORT

The report's UNRESOLVED line was optional ('omit if empty') and the EXIT
PLAN MODE GATE only checked it 'if applicable', so a plan could ship with
no statement about open decisions at all — a missed ambiguity read
identically to a clean plan. Now every report ends with a mandatory
unresolved-decisions status as its final line: either the exact unbolded
sentinel 'NO UNRESOLVED DECISIONS', or a '**UNRESOLVED DECISIONS:**' block
of bullets. The gate blocks ExitPlanMode unless that final line is present.

generatePlanFileReviewReport: current-review items are listed from context;
prior reviews contribute an aggregate count computed as latest-fresh-row-
per-skill minus the current run (no double-count, dashboard 7-day window).
generateExitPlanModeGate: check #3 is now blocking with no 'if applicable'
escape; bolded sentinel does not satisfy it.

Tests: static guard in gen-skill-docs.test.ts asserts the mandatory status
across all six report consumers and the gate across gate-bearing skills;
skill-e2e-plan.test.ts asserts the written report's final line is the
status (and fixes a stale 'four review rows' -> five-row prompt).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(review): compress unresolved-status prose to fit parity budget

After merging origin/main (v1.57.3.0), plan-devex-review exceeded the 1.05x
parity ratio vs the v1.53.0.0 baseline. Rather than rebase the baseline,
compressed the new prose to stay under the cap honestly: the report's
unresolved-status block (~32 -> ~9 lines) and the EXIT PLAN MODE GATE's
final-line check (~7 -> ~5 lines), plus the plan-devex-review review-log
step. All load-bearing rules and the exact gate-checkable tokens are
preserved; the static guards in gen-skill-docs.test.ts still pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: regenerate stale ship golden fixtures (#1909 follow-up)

#1909 (v1.57.3.0) added the always-loaded PR-title-version rule to ship's
template and committed the regenerated ship/SKILL.md, but did not refresh the
three ship golden fixtures, leaving the golden-file regression test red on
main. Regenerate them from current output. The diff is purely #1909 content:
the PR-title invariant line plus a previously-unresolved ${ctx.paths.binDir}
placeholder that current generation correctly resolves. No feature content
from this branch leaks into ship (ship does not consume the review report
resolvers).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(plan-devex-review): restore TIMESTAMP fill instruction in review-log

Adversarial review caught that compressing the devex review-log block dropped
the TIMESTAMP substitution guidance the three sibling plan-review skills carry.
A literal "timestamp":"TIMESTAMP" parses as JSON but is an unparseable date,
so the Review Readiness Dashboard's 7-day freshness window silently drops the
plan-devex-review row (and the report's prior-review aggregation loses it).
Restore the one-line instruction. Also: the plan-review-report E2E now derives
its last-line check from the report slice, not the whole file, so a mis-placed
report surfaces the real trailing content in the failure message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(parity): rebase parity baseline v1.53.0.0 -> v1.57.7.0

The v1.53 anchor is four minor versions stale. v1.54-v1.57 (ship/plan carving,
carve-guards, AUQ prose fallback, the cross-session decision-log preamble) plus
this branch's mandatory unresolved-decisions status line pushed the three
plan-review skills past the 5% ratchet even after exhaustive compression. The
new baseline captures current UNION sizes (skeleton + sections/*.md, matching
what parity-harness measures) so the per-skill 1.05 ratio keeps catching future
bloat. The frozen v1.44.1 integrity anchor and the v1.47 size-budget baseline
are untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v1.57.7.0)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-08 21:17:18 -07:00
Garry TanandClaude Opus 4.8 cab774cced v1.56.0.0 Token-reduction Phase B + AUQ paranoid safety net (#1849) 
* refactor(plan-ceo-review): carve review body into on-demand section

Carve the largest skill (138,838 B) into a skeleton + one on-demand
section, the documented next Phase B target after /ship (v2_PLAN.md:216).

- sections/review-sections.md(.tmpl): the 11-section deep review, codex/
  outside-voice rules, how-to-ask, Required Outputs, registries, Completion
  Summary, Review Log, REVIEW_DASHBOARD, PLAN_FILE_REVIEW_REPORT, Next Steps,
  docs/designs promotion, Formatting Rules, and the Mode Quick Reference.
- sections/manifest.json: passive registry (CM2), one entry.
- SKILL.md.tmpl: {{SECTION_INDEX}} after the system audit, a single
  {{SECTION:review-sections}} STOP-Read after Step 0 mode selection, and a
  Section self-check. All of Step 0 (the scope/mode conversation) stays in
  the always-loaded skeleton; only EXIT_PLAN_MODE_GATE follows the section.

Measured: always-loaded skeleton 138,838 -> 80,731 B (-42%, ~14.4K tokens
off every invocation). Union (skeleton + section) 139,110 B, behavior held.

Boundary honors Codex P1: nothing review-governing (formatting rules, mode
reference, how-to-ask, required outputs) sits in the skeleton below the
STOP. Housekeeping resolvers ride in the section, matching the ship
precedent (adversarial.md carries LEARNINGS_LOG + GBRAIN_SAVE_RESULTS).

Tests (atomic with the carve — skill-docs.yml gates gen:skill-docs
freshness on every push, so source + regen + tests must land together):
- parity-harness: plan-ceo flipped to sectioned, maxSkeletonBytes 90_000
  (measured 80,731 + headroom); content/minBytes run against the union.
- skill-size-budget: plan-ceo-review added to SECTIONS_EXTRACTED.
- section-manifest-consistency: generalized to discover every carved skill,
  vars computed per-skill-case (Codex P2).
- skill-ceo-section-ordering (new, gate): per-PR static guard — STOP after
  Step 0, review body absent from skeleton, report writer in the section,
  nothing review-governing below the STOP.
- skill-e2e-plan-ceo-review-section-loading (new, periodic): refreshes the
  installed skill first (Codex P1), drives full Step 0, asserts the section
  is Read before the report.
- gen-skill-docs + skill-validation: read the skeleton+sections union for
  carved skills so relocated prose still counts.
- touchfiles: plan-ceo-section-loading registered (periodic).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump VERSION + CHANGELOG for plan-ceo-review carve (v1.56.0.0)

MINOR: carves the largest skill into skeleton + on-demand section,
dropping plan-ceo-review's always-loaded cost 42% (138,838 -> 80,731 B,
~14.4K tokens off every invocation). User-facing release notes lead with
the measured token win.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(todos): file P3 follow-up — carve the shared {{PREAMBLE}} reference blocks

Surfaced by /plan-eng-review on the plan-ceo-review carve: per-skill section
carves stay modest because the ~40-50KB shared preamble dominates the
always-loaded surface. A single preamble-reference carve would help every
tier->=2 skill at once. Records the why, the cold-vs-hot split to measure,
and the guards it needs. Not implemented this PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(auq): Layer 0 — guarantee AUQ format spec is always-loaded

Deterministic, free, per-PR keystone for the token-reduction era. For every
interactive (tier>=2) skill, asserts the full AskUserQuestion decision-brief
format (ELI10/Recommendation/Pros-cons/checks/Net/(recommended)/Stakes/
self-check) lives in the always-loaded SKILL.md skeleton, NOT only in an
on-demand section. Plus a roster guard (a carve can't silently drop the block)
and per-skill rule survival in the skeleton+sections union. 51 cases + a
negative control. Fails the instant a future carve strands AUQ-governing text
where it won't be loaded when a question fires.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(auq): SDK capture engine + verbose-vs-carved no-degradation A/B

Adds the reusable SDK $OUT_FILE capture engine (auq-sdk-capture.ts): drives a
skill to its AUQ and captures the verbatim text the model GENERATES, cleanly
(real-PTY mangles plan-mode AUQs via cursor escapes). Pins the skill to an
absolute path with Read/Write-only tools so the agent can't wander to the
global install. gradeAuqRecommendation normalizes a non-"because" connective
before grading so substantive reasons aren't false-flagged (without touching
the pinned shared judge).

The A/B drives the same prompt through the carved 80KB skeleton and the
pre-carve 137KB monolith and fails if carved scores worse. Result: both 7/7
format, substance 5 — proven no degradation, transcript-verified each side read
its own planted SKILL.md. Periodic tier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(auq): consistency — same trigger N runs, stable format + substance

Drives the carved /plan-ceo-review AUQ N=3 times and fails if any format
element appears in one run but not another, or substance craters. Targets the
"fine one run, broken the next" failure class a single snapshot can't see.
Result: 3/3 stable, 7/7 + substance 5 every run. Periodic tier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(auq): behavioral matrix across AUQ-heavy skills

Data-driven test that drives each AUQ-heavy skill (plan-eng/design/devex,
office-hours, cso, spec, design-consultation) to its first AskUserQuestion and
grades it to the plan-ceo bar: 7/7 decision-brief format + recommendation
substance >=4. One case per skill (isolated failures), env-subsettable via
AUQ_MATRIX_ONLY. Browser/design-binary skills are intentionally excluded
(comparison boards, not format-AUQs; Layer 0 covers their spec). All targeted
skills pass 7/7 with substance 4-5. Periodic tier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(codex): live recommendation-substance grade for /codex

Closes the gap where /codex's synthesis recommendation was only checked
statically (template grep) and via fixtures. Drives the real /codex skill over
a flawed diff and grades the emitted "Recommendation: ... because ..." line
with judgeRecommendation (present/commits/has_because/substance>=4). The named
weak spot holds up: substance 5. Periodic tier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(auq): deterministic trigger for format-compliance gate

A bare /plan-ceo-review against a repo whose work is already implemented makes
the model improvise an off-script "what should I review?" scope question that
skips the decision-brief format, which the gate test then times out waiting for.
Hand it a concrete plan to review (FORCING_FLOOR_CEO) so it reaches the real
Step 0 mode-selection AUQ that is the intended format check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(office-hours): carve Phase 5+6 into on-demand section

Third Phase B carve (v2_PLAN.md:216, after ship and plan-ceo-review). Moves
Phase 5 (Design Doc templates) + Phase 6 (tiered relationship handoff) — the
session's output + closing tail, only reached after the conversation and
alternatives are done — into sections/design-and-handoff.md, behind a single
STOP-Read after Phase 4.5. The live conversation (Phases 1-4.5) and the
always-run Important Rules stay in the always-loaded skeleton.

Measured: always-loaded skeleton 118,280 -> 88,975 B (-24.8%). Union preserved.
The carved AUQ is identical to pre-carve (matrix: 7/7 format, substance 5),
and Layer 0 confirms the AUQ format spec stays in the skeleton — the AUQ
paranoid suite de-risked this carve end to end.

Atomic with tests + regen (skill-docs.yml gates gen:skill-docs freshness on
every push, so source + regen + tests land together; --host all regenerates
the inlined non-Claude variants):
- sections/manifest.json: passive registry, one entry.
- parity-harness: office-hours flipped to sectioned, maxSkeletonBytes 96_000
  (measured 88,975 + headroom); content/minBytes run against the union.
- skill-size-budget: office-hours added to SECTIONS_EXTRACTED.
- gen-skill-docs + skill-validation: read the skeleton+sections union for
  office-hours so relocated Phase 5/6 prose still counts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump VERSION + CHANGELOG for office-hours carve + AUQ suite (v1.57.0.0)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(preamble): carve CJK-escaping manual to on-demand doc

The AskUserQuestion format block is inlined into every interactive skill (~33).
It carried the full multi-paragraph non-ASCII/CJK escaping manual inline, but
that rationale only matters when a question contains CJK text and the operative
rule already lives in the always-loaded self-check. Moved the justification to
docs/askuserquestion-cjk.md (read on demand); kept the rule + a pointer.

Corpus: Claude-host SKILL.md total 3,087,499 -> 3,057,975 B (-29,524 B, ~900 B
x ~33 skills). Layer 0 still passes — the core decision-brief format stays
always-loaded; only the rare CJK rationale moved. Atomic with the all-host
regen (skill-docs.yml freshness gate). VERSION + package.json -> 1.58.0.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(plan-eng-review): carve review body into on-demand section

Fourth Phase B carve (v2_PLAN.md:220). Moves the 4-section review (Architecture,
Code Quality, Tests, Performance), outside voice, required outputs, and review
report — everything after Step 0 scope — into sections/review-sections.md behind
a single STOP-Read. Step 0 (scope challenge) and EXIT_PLAN_MODE_GATE stay in the
always-loaded skeleton.

Measured: skeleton 106,984 -> 54,892 B (-48.7%). Union preserved. Atomic with
tests + all-host regen (freshness gate): parity flipped to sectioned
(maxSkeletonBytes 62K), plan-eng-review added to SECTIONS_EXTRACTED, gen-skill-docs
reads the union for relocated review/TEST_COVERAGE/dashboard prose. Layer 0 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(plan-design-review): carve review body into on-demand section

Fifth Phase B carve (v2_PLAN.md:220, bundled with plan-eng). Moves the 7 design
passes, required outputs, and review report — everything after Step 0 scope and
the mockup/rating phase — into sections/review-sections.md behind a STOP-Read.
Step 0, Step 0.5 mockups, the rating method, and EXIT_PLAN_MODE_GATE stay in the
always-loaded skeleton.

Measured: skeleton 112,057 -> 76,024 B (-32.2%). Union preserved. Atomic with
tests + all-host regen: parity sectioned (maxSkeletonBytes 82K), added to
SECTIONS_EXTRACTED, gen-skill-docs reads the union. Layer 0 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(plan-devex-review): carve review body into on-demand section

Sixth Phase B carve. Moves the 8 DX passes, required outputs, and review report
— everything after the Step 0 DX investigation — into sections/review-sections.md
behind a STOP-Read. All of Step 0 (persona, empathy, benchmark, journey trace,
roleplay) + the rating method + EXIT_PLAN_MODE_GATE stay always-loaded.

Measured: skeleton 110,621 -> 69,658 B (-37%). Union preserved. Atomic with
tests + all-host regen: added to SECTIONS_EXTRACTED, gen-skill-docs reads the
union. Layer 0 green. (No parity invariant entry for plan-devex-review.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump VERSION + CHANGELOG for plan-* family carves (v1.59.0.0)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: refresh ship golden baselines + gbrain-detection union after carves

Two follow-ups the carve commits should have carried (caught by the full suite,
missed by targeted subsets):
- ship golden baselines (claude/codex/factory) regenerated: the preamble CJK
  trim (v1.58) changed ship's always-loaded AskUserQuestion block.
- gbrain-detection-override probes the office-hours skeleton+section union:
  GBRAIN_SAVE_RESULTS moved into sections/design-and-handoff.md when office-hours
  was carved, so the detection assertions now check both files.

Full `bun test` green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(auq): grade format-compliance gate from SDK capture, not the TUI

The real-PTY version grepped the stripAnsi'd interactive AUQ picker. Verified
directly that this cannot work: plan-mode AUQs render as a cursor picker whose
cursor-positioning escapes stripAnsi can't flatten — the picker renders fine for
a human (cursorSeen=45) but the flattened text drops ELI10:/(recommended) and
parseNumberedOptions returns 0. The test was grading a lossy projection and
failed by construction.

Rewritten to drive /plan-ceo-review via the SDK $OUT_FILE capture (the agent
writes the verbatim question it would have shown — clean text, no rendering
loss) and grade 7/7 format + kind-note + recommendation substance >=4. Same
property, reliable, environment-independent; shares the engine with the periodic
A/B and matrix evals. Result: 7/7 format, substance 5. Touchfiles key renamed
ask-user-question-format-pty -> auq-format-gate (no longer a PTY test).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: fix carve-broken CI evals (union reads + section fixtures)

Two CI eval jobs failed on the carved plan-* skills because they read content
that moved into sections/:

- llm-judge (skill-llm-eval): runWorkflowJudge sliced SKILL.md between markers
  like "## Review Sections" / "## CRITICAL RULE" that now live in
  sections/review-sections.md. The markers vanished from the skeleton, so the
  judge scored empty/wrong content. Fix: read the skeleton+sections union.
  Verified: plan-ceo modes / plan-eng sections / plan-design passes all PASS
  (25/25).

- e2e-plan (skill-e2e-plan): setupPlanDir copied only <skill>/SKILL.md into the
  fixture, not sections/. The carved skill's STOP pointed at a section file that
  was absent, so the model improvised a compressed report table instead of the
  canonical "| Review | Trigger | Why | Runs | Status | Findings |". Fix: copy
  sections/ alongside SKILL.md in all 6 setup sites. Verified: report test PASS,
  canonical table emitted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: copy carved sections into all e2e fixtures (prevent more carve-blind CI fails)

Proactive sweep beyond the two CI logs: every e2e test that copies a carved
skill's SKILL.md into a temp fixture must also copy its sections/, or the
model hits a STOP pointing at a missing section file and improvises/degrades.

- skill-e2e.test.ts: plan-ceo/plan-eng/plan-design/office-hours copies across
  planDir/reviewDir/ohDir/benefitsDir dests now copy sections/.
- skill-e2e-plan.test.ts: the office-hours copy + the 4-skill codex-offering
  loop now copy sections/.
- skill-e2e-design.test.ts: plan-design-review copy now copies sections/.
- skill-e2e-office-hours.test.ts: both office-hours copies now copy sections/.
- skill-e2e-office-hours-brain-writeback.test.ts: GBRAIN_SAVE_RESULTS moved into
  the section, so check the regenerated skeleton+section UNION for the gbrain put
  block, ship both into the workdir, and restore both (the section regen was also
  leaking into the working tree — finally now restores it).

ship copies (single-file Step-0 slices) and review/retro (not carved) untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: migrate section-loading E2E to lossless SDK tool-stream detection

The /ship and /plan-ceo-review section-loading tests drove a real PTY and
scraped the ANSI screen buffer for sections/<file>.md paths. That silently
saw nothing in a Conductor PTY (cursor-positioned tool renders and an
unanswered Step 0 question loop both defeat the regex), so both reported
read: [] even when the agent did the work.

They now run the skill through claude -p (the same SDK path the AUQ matrix
uses) and detect section reads from the tool-use stream — Read calls whose
file_path contains sections/<file>.md — with no rendering layer to mangle.
The run is also hermetic: the freshly-generated worktree skeleton + sections
are copied into a throwaway fixture with the absolute path pinned, so the
test validates this branch's carve without mutating the user's ~/.claude
install.

Validated EVALS_TIER=periodic: both pass (plan-ceo Reads review-sections.md;
ship Reads review-army.md + changelog.md), ~6.5 min for both vs ~23 min
combined on the old PTY path where both were failing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: consolidate branch to v1.56.0.0 (single MINOR above main)

The branch bumped VERSION several times during development (1.56 → 1.57 →
1.58 → 1.59), but none of those landed on main (main is at 1.55.1.0). Per
the "never orphan branch-internal versions" discipline, collapse all four
into a single 1.56.0.0 entry — one MINOR release covering the whole branch:
five skills carved (plan-ceo, office-hours, plan-eng, plan-design,
plan-devex), the shared AskUserQuestion preamble CJK trim, and the paranoid
AUQ no-degradation test suite + lossless section-loading tests.

VERSION and package.json set to 1.56.0.0; main's 1.55.1.0 entry preserved
below the consolidated entry. No SKILL.md drift (VERSION is not embedded in
generated bodies).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-04 11:14:43 -07:00