* chore(scripts): retire hollow and legacy paid package scripts behind one-release stubs (W8a, DX-4, CEO-31)
test:codex*, test:evals*, test:e2e*, test:gate, test:periodic, eval:bg,
eval:bg:all, eval:flake-rank, eval:watch and test:audit now print their
replacement and exit 1 via scripts/retired-command.ts. CONTRIBUTING.md
carries the same table; test/retired-commands.test.ts pins both. Eval CLI
empty-state hints point at eval:bg:pr.
* chore: delete dead update-readme-throughput, eval-watch, orphan fixtures and QA_FUNCTIONAL_INPUTS (W8b, CEO-24)
Retired with named behavior:
- update-readme-throughput.ts + readme-throughput test: replaced a README
anchor that no longer exists from a JSON file that was never produced;
the script was a no-op. Guarantee withdrawn (nothing to protect).
- eval-watch.ts + its dashboard tests: rendered ~/.gstack-dev/e2e-live.json
and _partial-e2e.json, written only by unsharded runs, so it showed
nothing for every supported (sharded) entry point. Guarantee withdrawn;
live progress is the gstack-detach log or gh run watch. Manual-review
rendering stays covered by eval-list and eval-summary tests.
- Orphan fixtures (autoplan-pending-artifact-ae.json, autoplan-existing-app/,
autoplan-sign-in-ui/, parity baselines v1.47.0.0/v1.57.7.0/v1.69.1.0):
no test, touchfile or script reads them; superseded baselines stay in
git history.
- QA_FUNCTIONAL_INPUTS: exported, never imported.
* refactor(test-free): delete dead WORKER_HOSTILE list and buildShardArgs parallel mode
Neither had a production caller: full-suite mode runs shard processes, so the
worker quarantine was inert and --parallel was never passed (W8b).
* docs(test-value): narrow prompt-byte contract to machine-read tokens (D2)
Define prompt-byte contract as machine-read tokens (markers, field names,
command lines, enum values, file paths, security-boundary strings) and add a
sentence-pin glossary entry: convert to structural checks or delete when a
behavioral eval covers the behavior.
* test(ratchet): count free tests that slice a paid test's source
A free test that binds readFileSync(<paid test file>) and cuts it with
.slice() tests the paid harness's code out of context even when it imports a
lib/ symbol. Record the 15 existing files with reasons; new ones fail with fix
text. Self-test covers positive and negative shapes.
* test(browse): un-quarantine compare-board; all 16 board tests pass headless (W8d, CEO-24)
The GSTACK_COMPARE_BOARD_TESTS opt-in hid a headless mismatch that no
longer reproduces: all 16 tests pass on Bun 1.4.0 with the bundled
headless Chromium, solo, repeated, and beside other in-process browser
files. Protected behavior (the comparison-board DOM contract the
plan-design-review/office-hours/design-consultation feedback loop polls)
is checked on every free run again.
* test: replace silent existsSync early returns with explicit skips or assertions (W8d)
- browse/test/build.test.ts: server-node.mjs checks skip only when the
bundle is absent and GSTACK_EXPECT_BINARIES is unset; the free CI lane
and test:ubicloud build it and set the flag, so a dropped build step
fails with the build command instead of passing with no assertions.
- gstack-update-check: VERSION is tracked; the guard could only hide a bug.
- gen-skill-docs host leakage/frontmatter checks assert the host tree
exists (the module-level render always produces it).
- cso-docker staged-runtime test uses test.skipIf so reports show it as
skipped, not passed.
* test(ship): convert ship-workflow-clarity sentence pins to structural checks
Add test/helpers/prompt-structure.ts (between, expectTokens, expectAbsent,
expectOrdered, expectMentions) with fix text on failure. ship-workflow-clarity
now pins headings, step order and machine-read tokens (commands, fields, enum
and status strings); safety rules are meaning-level keyword checks. Wording is
judged by the 'ship/SKILL.md workflow' judge in skill-llm-eval.
Sentence pins: 90 -> 0.
* ci: add platform-qualification lane for opt-in ML, gitleaks and Swift tests (W8d, CEO-24)
security-bench + security-live-playwright L4 (SECURITY_BENCH), the real
gitleaks 8.30.1 memory-ingest cases (GSTACK_TEST_GITLEAKS) and the
DebugBridge Swift build invariants (GSTACK_TEST_SWIFT) self-skip in every
lane today. No scheduled lane has their prerequisites, so per CEO-24 they
move to an owned dispatch + quarterly workflow instead of being deleted.
Each job was run locally (ML 10 pass, gitleaks 130 pass; Swift needs
macOS). test/platform-qualification-workflow.test.ts pins gate variable,
job and file together.
* fix(scripts): capture-baseline and preflight-agent-sdk no longer act on --help or typos (W8e)
capture-baseline parses its flags strictly: --help prints usage, an unknown
or valueless flag exits 2, and only a valid invocation writes a fixture
(--help used to write test/fixtures/parity-baseline-current.json).
preflight-agent-sdk gains --help, rejects unknown flags, and runs the live
SDK query (API spend) only with --live.
* test(evals): register the three keyless periodic paid files with case ids (W1b)
skill-e2e-auq-consistency, skill-e2e-auq-verbose-vs-carved-ab and
codex-e2e-recommendation-substance now carry touchfiles, a periodic tier and
kind rule (one trial, unchanged), so editing one selects its own case instead
of restoring the full PR gate. The verbose A/B case also registers the vendored
pre-cut fixture it reads, and review-army-migration-safety registers the SQL
fixture it reads.
* feat(evals): narrow PR fallback for free-only inputs (W1a, CEO-13/20, ENG-13)
The PR profile now exempts the duration seeds, every free-lane workflow
(explicit list; inventory test fails on an unclassified workflow, .yaml
and workflow_call included), scripts/ubicloud/** and fixtures declared in
the explicit FREE_FIXTURES map. Each remaining unknown file carries a label
and fix (needs touchfile entry / add to FREE_ONLY_PR_FILES / real unknown
dependency). evals.yml itself still restores the full gate.
* ci: delete ci-image.yml; the eval workflows already build and push the same content-hash tag (W8f)
evals.yml, evals-periodic.yml and evals-marathon.yml each compute the
identical hashFiles() tag and build/push it (with :buildcache and :latest)
when missing, so ci-image's push trigger and weekly cron were no-ops
(30-day median 0.5 min, 36/36 success). ci-image-tag-binding now pins the
three eval workflows.
* ci(evals): key periodic/marathon concurrency on ref+event; branch periodic dispatch skips the gate census (W2a, ENG-9, CEO-04)
* refactor(paid): drop ~160 unused imports; leaf paid-types/paid-select modules break the paid-* import cycle (W8c)
* ci(quality-gate): per-PR concurrency, free runner, drop unscoped bun audit (W2a, W8f)
- Concurrency keys on pull_request.number || run_id, so back-to-back main
pushes no longer cancel each other's credential scan.
- ubuntu-24.04 instead of a billed 8-vCPU Ubicloud runner for a ~1 min job.
- bun audit --audit-level=critical ran on every PR regardless of diff, so a
newly published advisory turned unrelated PRs red, with its own ignore
policy. dependency-review (lockfile PRs, fail >= high) and the weekly OSV
sweep remain the dependency gates.
* chore(test-free): refresh the free duration seed from the audit's Ubicloud recording
Adds the 19 files merged after the last refresh, including the 115 s
claude-overlay-setup that packed as ~1 s and doubled a shard (W6a). The seed
is re-recorded on the final tree at the end of this wave.
* ci: move tiny ubicloud-standard-2 jobs to free ubuntu-24.04 runners (W8f)
actionlint, skill-docs, dependency-review, version-gate and pr-title-sync
are sub-minute jobs; GitHub-hosted runners are free for this public repo.
* feat(test-free): pessimistic unseeded packing, seed growth ratchet and CI health summary
- Unseeded files pack at the seed's p99 instead of p75 (W6a).
- --ci-plan warns in the job summary past 5 unseeded files; --ci-verify
lists shards that outran their prediction by 1.5x (and 30 s) plus every
flaky pass absorbed by the serial retry (W6a, W7a job summary).
- test/free-seed-ratchet.test.ts fails when a seeded file over 60 s is not
on scripts/free-test-seed-allowlist.json, when an entry is stale, and when
the allowlist grows versus its merge-base copy unless the new entry is an
annotated exception with a todo (CEO-10, DX-7). free-plan runs it with
full history so the shrink check is real in the required lane.
- Every message names the fix command and docs anchor (DX-6).
* feat(evals): state PR coverage mode, counts, reuse and fallback files in the summary and comment (CEO-13)
* test: make the tracked-tree skill-doc freshness check local-only; tidy skill-docs header (W8f)
Every free CI shard regenerates the tracked SKILL.md files before running
tests, so the --dry-run freshness assertion compared the regenerated tree
with itself and could not fail there. It now skips under GitHub Actions;
skill-docs.yml's git diff --exit-code is CI's freshness gate. Removes a
stale concurrency comment in skill-docs.yml.
* ci(free-tests): install zsh so the bash+zsh portability arms run (W6f)
~42 zsh-arm tests (aside-probe-shell, qa-browser-preservation,
generated-docs-fences) skipped in CI because zsh is not in the runner image.
All 141 tests in those files pass with zsh present.
* feat(evals): executors write negative receipts at the execution boundary (CEO-25, ENG-6)
A completed single-trial rule/judge shard that did not pass now writes its
negative receipt beside the pass receipts in the slice artifact, so a
cancelled run's FAIL still blocks an older PASS when its report never runs.
Receipts merge negatives first and order by (run attempt, completedAt); a
same-second FAIL/PASS tie resolves to FAIL. The runner call site is a
one-line hunk in scripts/test-paid-shards.ts.
* docs(cso): state that contained runtime/scanner execution is unavailable today
The scanner catalog is empty while its build inputs await review and the
runtime catalog holds no qualified profile until the private evaluator
exists, so /cso runs static assessment only. AGENTS.md, README and
docs/cso-release-qualification.md now say so; docs/cso-scanners.md
already did.
* test(helpers): expectMentions splits on sentence ends only
Clauses joined by ';' or ':' are one rule; splitting there made a reworded
rule fail for punctuation.
* test(plan-ceo): convert skill-ceo-section-ordering sentence pins
Safety and authority rules (never/do not/stop/only/without/wait/approval) are
meaning-level expectMentions checks; other sentence pins are dropped. Headings,
labels, option rows, placeholders and machine-read lines (auto-decide
annotation, Mode:/approved decisions format, STATUS/SOURCE fields,
CROSS-MODEL marker, gstack-ceo-mode-handoff) stay as tokens; ordering checks
unchanged. Behavior is covered by the periodic PTY case in
skill-e2e-plan-ceo-review-section-loading.test.ts.
Sentence pins: 210 -> 23.
* test(helpers): expectMentions treats wrapped lines as one sentence
Templates hard-wrap prose; only sentence ends and blank lines separate rules.
* test(gen-skill-docs): convert sentence pins to meaning checks or drop them
Safety/authority sentences become expectMentions keyword checks; other English
sentences are dropped (wording is judged by the per-skill workflow judges in
skill-llm-eval and the E2E cases in touchfiles-data). Commands, config keys,
markers (PLAN MODE EXCEPTION, NO UNRESOLVED DECISIONS, Gate outcome), field
names and alternation-based meaning regexes stay. Removed the now-empty
'contains opt-in AskUserQuestion text' test (only pinned a sentence; the
resolver's AskUserQuestion wiring is covered by the codex second-opinion
tests in the same describe). Lane E's existsSync hunks are untouched.
Sentence pins: 216 -> 65 (remaining lines carry machine tokens).
* ci(windows): shard the Windows lane via --ci-plan/--ci-run/--ci-verify; move native campaigns to native-qualification.yml
- windows-free-tests.yml: a Linux plan job curates and packs the Windows-safe
files into 6 shards, one strict serial shard per windows-latest job, and a
Linux aggregate verifies exact coverage and every result (8.1 min on one
runner today). --windows-only now combines with the CI modes; the plan binds
the curated file set (W6e).
- Every Windows plan and --list run prints the curated count with its top
exclusion reasons in the job summary; a free ratchet fails when the curated
subset drops below its floor, with the fix (W6e).
- The four Dia/cookie diagnostic dispatch campaigns move to the dispatch-only
native-qualification.yml with one exclusive `mode` choice input, so their
red dispatch runs no longer read as a red PR lane (W8f). Their workflow pins
move to test/windows-native-workflows.test.ts; the boolean-exclusivity
guard is replaced by the choice input.
* feat(paid): per-slice CI job ceilings and an in-process slice deadline with not_run/hung outcomes and checkpoints (W2c, ENG-2)
* fix(pass-rates): segment trial history strictly by EVAL_POLICY version
Reader-compatibility floor for EVAL_POLICY v2 (ENG-14/CEO-35/DX-12). Only
trials under the reader's own policy version feed labels and alarms; older
policy trials stay visible but are never scored or used as a drift baseline;
trials from a newer policy are ignored with a printed count and fix text.
Reverting the later v2 writer commits leaves a reader that cannot pool v2
records into v1 series (the pre-PR reader at 2db0b3a does: it reports a
mixed fixture as FAILING with drift alarms).
* docs: one contributor command table; retired commands, seed sources and current workflows (W8g, DX-5, DX-13, CEO-12)
- CONTRIBUTING.md gains 'Which command do I run?' (task -> command ->
cost/time -> needs, including branch validation via gh workflow run);
CLAUDE.md, AGENTS.md and TESTING_INTERNALS.md link to it instead of
repeating command lists. Tier tables drop retired commands and stale
per-run dollar estimates.
- Retired names (test:evals, test:e2e, eval:bg, eval:flake-rank, eval:watch,
...) are gone from CLAUDE.md, CONTRIBUTING.md, TESTING_INTERNALS.md,
ARCHITECTURE.md and USING_GBRAIN_WITH_GSTACK.md.
- The flake-ledger claim now says what happens: CI uploads flake-ledger-*
artifacts; eval:pass-rates reads only the local ledger.
- TESTING_INTERNALS states that only test:ubicloud --record-durations
produces an acceptable duration seed (DX-13).
- CI section: quality gate no longer runs bun audit; dependency-review's
real trigger; the platform-qualification lane.
- docs/test-audit-2026-09.md gets a corrections section for its stale claims.
- TODOS.md: audit follow-ups (post-merge metrics, stub removal, name reclaim,
diff-aware planner); eval-watch shard-awareness and the image freshness
tripwire closed as won't-do.
* feat(evals): recover receipts of cancelled PR runs (W1c, CEO-14/34, ENG-4/5/7)
A new recover-receipts job runs base-ref code with actions: read, restores
the PR's store, merges the receipts of completed cancelled evals.yml runs
for this PR (provenance-checked, at most 5, oldest first, negatives first),
reconstructs completed panel negatives with panelVerdict() (partial panels
get no credit) and keeps a recovery.json checkpoint that never advances past
an unresolved run. plan-slices keeps contents: read, takes the store as an
artifact, turns reuse off when recovery degraded or a run is unresolved, and
carries the whole store to slices-report so nothing is lost. Any gh problem
degrades to no recovered receipts with a named reason.
* feat(evals): classify this wave's free-only tools and workflows for PR selection
platform-qualification.yml, retired-command.ts, test-health-report.ts,
bump-harness-version.ts and eval-bg.ts never run inside a paid case.
* feat(evals): debounce PR push bursts before planning paid slices (W1d, CEO-15, ENG-11, DX-7)
A tiny ubuntu-24.04 job waits 90 s only on synchronize when another run for
the PR started in the last 15 minutes; cancel-in-progress supersedes, and a
run that outlives the wait reports "superseded by <sha>" and plans nothing.
Labels: evals-no-debounce skips the wait, evals-fresh disables receipt reuse.
* perf(test): move the real-process browser settlement cases out of test-free-shards.test.ts (W6b)
The 19 owned-browser settlement modes were 85 s of the file's 95 s and could
not be packed apart from the fast runner unit cases. They move to a shared
fixture with two files (lifecycle 47 s, ownership 39 s); the unit file drops
to 10 s. Same modes, same assertions.
* chore(evals): delete the finished validation_phase dispatch and its inline planner (W8f)
Retirement (CEO-24): the protected behavior was a one-off cookie-repair
validation subset (eight dependent cases) and a quality/behavior split of
the census; that validation is complete, so the guarantee is withdrawn.
Branch validation dispatches evals.yml with evals_all.
* perf(test): inject the office-hours record grace in non-cooperative deadline tests
* perf(test): shrink aside-render process slack via GSTACK_RENDER_SLACK_MS in timeout tests
* perf(test): make the memorable lock give-up configurable for lock-contention tests
* perf(test): check install-timeout knob normalization through its own setup lines
* perf(test): inject a 2 s storage-reset deadline in the late isolated dispatch test
* perf(test): shorten the slow gbrain fake to 1 s, still past the 300 ms probe timeout
* ci(free-tests): run only cso-docker-integration in the free CSO Docker job (F16)
cso-node-lifecycle-integration and cso-stack-cold-integration need
GSTACK_CSO_TEST_STACK images and always skipped there; they stay in the CSO
runtime workflows through test:cso:docker.
* test(gen-skill-docs,plan-ceo): split mixed sentence pins into tokens and rules
Sentences that wrap a backticked or bold token keep the token (expectTokens)
and, when they state a safety rule, a keyword check; short safety sentences
(25-39 chars) get the same treatment.
* test(plan-ceo): convert ceo-workflow-clarity.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: periodic PTY case skill-e2e-plan-ceo-review-section-loading and the plan-ceo workflow judges.
Sentence pins: 13 -> 1.
* test(plan-eng): convert eng-review-routing.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: plan-eng E2E cases (skill-e2e-plan-eng-*) and the plan-eng workflow judge.
Sentence pins: 28 -> 6.
* test(plan-eng): convert eng-scope-entry-ap.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: plan-eng E2E cases (skill-e2e-plan-eng-*) and the plan-eng workflow judge.
Sentence pins: 16 -> 5.
* test(plan): convert plan-scope-selection.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: plan-review E2E cases and workflow judges in skill-llm-eval.
Sentence pins: 19 -> 2.
* test(qa): convert qa-lazy-sections.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: qa/qa-only E2E cases (skill-e2e-qa-*) and the qa workflow judge.
Sentence pins: 71 -> 11.
* test(qa): convert qa-browser-preservation.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: qa/qa-only E2E cases (skill-e2e-qa-*) and the qa workflow judge.
Sentence pins: 25 -> 11.
* test(qa): convert qa-probe-gates.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: qa/qa-only E2E cases (skill-e2e-qa-*) and the qa workflow judge.
Sentence pins: 20 -> 5.
* test(qa): convert qa-checkpoint-evidence.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: qa/qa-only E2E cases (skill-e2e-qa-*) and the qa workflow judge.
Sentence pins: 7 -> 5.
* test(qa): convert qa-evidence.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: qa/qa-only E2E cases (skill-e2e-qa-*) and the qa workflow judge.
Sentence pins: 4 -> 3.
* test(design): convert design-consultation-contract.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: design-consultation E2E/judge cases in touchfiles-data.
Sentence pins: 44 -> 11.
* test(outside-voice): convert run-in-background-guidance.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: outside-voice E2E cases and plan-review workflow judges.
Sentence pins: 35 -> 5.
* test(spec): convert spec-template-invariants.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: spec E2E/judge cases in touchfiles-data.
Sentence pins: 36 -> 21.
* perf(evals): 420 s census slice budget; case-shard skill-e2e-qa-workflow; seed its case walls (W5c)
* feat(evals): evals.yml dispatch contract for eval:bg (CEO-17/29, DX-1, ENG-8/12)
workflow_dispatch gains base_ref, base_sha, expected_sha, nonce and
pr_receipts. The nonce appears in run-name so the dispatcher resolves its
own run; evals_all=false runs the PR profile against base_sha/base_ref; a
checkout that differs from expected_sha fails before any paid work with a
named fix; pr_receipts reads that PR's receipts read-only through the
recover-receipts artifact path (never the cache) and never saves.
* ci(windows): pack the Windows lane with Windows-measured durations (W6e)
The first sharded run (37221451711) finished in 6.0 min: the Linux seed has
browse/test/cookie-import-native-job.test.ts at ~0 s (its cases skip on
Linux) but it takes 122 s on Windows, so one shard ran 242 s against an 84 s
prediction. The plan now reads scripts/free-test-durations-windows.json; a
record_durations dispatch times every Windows-safe file alone and uploads it.
* test(free): settlement case files import the runner rule they exercise
Keeps the split files out of the test-of-test ratchet: the retry-eligibility
rule is a direct runner import passed to the shared fixture.
* fix(marathon): no hollow plan-decision-classification shard; office-hours marathon scenario scoped to the pilot (W2f)
* test(free): satisfy the typecheck:test ratchet in the seed ratchet test
* refactor(ci-history): one shared GitHub history reader with the CI flake-ledger reader
Extract pass-rates' gh fetch layer into scripts/lib/ci-history.ts (CEO-09) and
add the W7a reader: paged run listings that fail whole on any page error,
per-run artifact downloads (cached, size-capped, data only), flake-ledger
parsing, the pre-registered flaky-file rule (> 5% of main free-tests runs over
a window of at least 20 runs, widened to the newest 20 runs when the window is
thinner), a job-summary table, and named unavailable reasons with next steps.
* feat(test-health): test:health report of the audit success metrics
scripts/test-health-report.ts (CEO-09/06/22/32, DX-8/9/12/13) prints every
success metric with its counting rule from CI history: PR full-fallback rate,
reused PR trial records, PR-lane spend per merged PR and per validated
revision, cancelled slice-minutes per week, push-to-verdict p50/p95, census
expected failed trials per run with raw failure rate per kind/class and
P(lane red) under the recorded panels vs single-trial aggregation, INFRA
re-dispatches, quarantine evidence (insufficient evidence distinct from
healthy), free flaky-pass rate, free/Windows wall, free serial time and the
net-complexity line. An unreadable input prints 'unavailable: <reason>; next:
<step>', never 0. --enforce fails on a flaky file (W7a) or > 5 unseeded free
files (CEO-10); unavailable checks never fail and keep the issue open.
* test(review): convert review-workflow-clarity.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: review E2E cases (skill-e2e-review*) and the review workflow judge in skill-llm-eval.
Sentence pins: 85 -> 17.
* test(review): convert review-entry-and-design-clarity-au.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: review E2E cases (skill-e2e-review*) and the review workflow judge in skill-llm-eval.
Sentence pins: 15 -> 4.
* test(review): convert review-enum-lifecycle.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: review E2E cases (skill-e2e-review*) and the review workflow judge in skill-llm-eval.
Sentence pins: 8 -> 0.
* test(ship): convert ship-control-flow.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 74 -> 9.
* test(ship): convert ship-skip-requeue.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 28 -> 3.
* test(ship): convert ship-plan-completion-invariants.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 28 -> 12.
* test(ship): convert ship-reentry-gates.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 16 -> 4.
* test(ship): convert ship-document-release-dispatch.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 11 -> 4.
* test(ship): convert ship-apple-gate.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 7 -> 0.
* test(ship): convert ship-skip-actor.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 6 -> 1.
* test(ship): convert ship-template-redaction.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 5 -> 4.
* test(ship): convert ship-publication-gates.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 5 -> 2.
* test(ship): convert ship-review-loop.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 3 -> 1.
* test(ship): convert ship-version-sync.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 5 -> 2.
* test(ship): convert ship-pr-liveness-policy.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: ship E2E cases (skill-e2e-ship*, skill-e2e-workflow) and the 'ship/SKILL.md workflow' judge.
Sentence pins: 4 -> 0.
* test(plan-review): convert plan-review-cases.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: plan-review E2E cases and workflow judges in skill-llm-eval.
Sentence pins: 58 -> 15.
* test(autoplan): convert autoplan-phase-order.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: autoplan E2E cases (skill-e2e-autoplan-*) and the autoplan workflow judge.
Sentence pins: 42 -> 11.
* test(autoplan): convert autoplan-phase-handoff.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: autoplan E2E cases (skill-e2e-autoplan-*) and the autoplan workflow judge.
Sentence pins: 20 -> 4.
* test(autoplan): convert autoplan-snapshot.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: autoplan E2E cases (skill-e2e-autoplan-*) and the autoplan workflow judge.
Sentence pins: 21 -> 5.
* test(outside-voice): convert outside-voice-preflight.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: outside-voice E2E cases and plan-review workflow judges.
Sentence pins: 26 -> 9.
* test(ask-format): convert resolver-ask-user-format.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: AUQ E2E cases (skill-e2e-auq-*); prompt-injection rule kept as a meaning check.
Sentence pins: 32 -> 15.
* test(skills): convert binding-template-drift.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: workflow judges in skill-llm-eval.
Sentence pins: 19 -> 13.
* test(skills): convert carve-plan-fixture.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: workflow judges in skill-llm-eval.
Sentence pins: 20 -> 6.
* test(land-and-deploy): convert land-and-deploy-postfail.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: land-and-deploy E2E cases and its workflow judge.
Sentence pins: 21 -> 14.
* test(land-and-deploy): convert land-and-deploy-ci-gate.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: land-and-deploy E2E cases and its workflow judge.
Sentence pins: 5 -> 4.
* test(land-and-deploy): convert land-and-deploy-flow.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: land-and-deploy E2E cases and its workflow judge.
Sentence pins: 5 -> 3.
* test(office-hours): convert office-hours-completion.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: office-hours E2E cases (skill-e2e-office-hours*).
Sentence pins: 16 -> 1.
* test(plan-ceo): restore the stale closing-route guard
The negative check for 'Closing hooks' / 'return to section' guards a retired
route token, not a sentence; the first conversion pass dropped it.
* ci(test-health): weekly test-health workflow with an honest tracking issue
New test-health.yml (CEO-18/DX-9): Monday 10:00 UTC after the periodic census
plus dispatch, free ubuntu-24.04 runner, permissions contents:read +
actions:read + issues:write only. Runs bun run test:health --since-days 7
--enforce; an enforced failure upserts one tracking issue, a run with
unavailable inputs keeps it open and lists them, and it closes only when every
enforced check has evidence and passes. Only main runs touch the issue.
* feat(evals): an edited paid test file runs its own gate cases in the PR lane (DX-11)
A gate case outside the audited fast profile is selected when its single
owning paid test file changed and the runner can select it by name (its
Bun test is named by the case id). The manifest records directCases and
validation accepts only recorded, name-addressable ones; the runner and
planner read the file->cases map from test-pr-profile. Cases the runner
cannot address stay deferred with the fix. An inventory test covers every
owned gate case. Touches one hunk each in test-paid-shards.ts and
lib/paid-plan.ts, and pr-shared-input-selection.test.ts's ship-skip pin.
* ci(osv): tracking issue for scheduled reds and a 14-day suppression expiry test
F11/W8f: a failed main OSV scan now upserts one tracking issue and a clean
scan closes it (scheduled reds notified no one during the August backlog).
A free test fails 14 days before any .osv-scanner.toml ignoreUntil (all four
expire 2026-11-30), with the fix: upgrade and delete, or re-justify and move
the date.
* test(ship,review): keep guards against retired fail-open instructions
'skip silently' on reviewer failure and 'do not block /ship on subagent
failure' were harmful instructions, not prose style; keep them absent with the
shortest distinctive phrase.
* feat(eval-select): default to the pr profile
DX-10: eval:select previews the PR lane's selection by default, matching
test:pr and eval:bg:pr; --profile full (or EVALS_PROFILE=full) keeps the plain
touchfile selection.
* test(evals): type-clean the recovery gh fake (typecheck:test ratchet)
* chore(windows): record the Windows duration seed (run 37221963236)
573 Windows-safe files timed alone on windows-latest; cookie-import-native-job
is 102 s there. Six shards now predict 154 s each.
* feat(evals): eval:bg dispatches the pushed ref to CI or runs locally under a 1.5x-planned cap (W5d, CEO-17/29, DX-1/2/3/7, ENG-12/19)
* feat(runners): --help on both shard runners via the shared flag loop; paid runner header describes today's planner/executor/report (W8e)
* test(windows): drop the stale Playwright exclusion and force-include probe-proven files (W6e)
A temporary windows-latest probe (run 37221995077) ran each of the 97 files
excluded for 'launches Playwright browser' or 'raw /tmp/ path' alone: all 17
Playwright files pass (Chromium is installed and Bun drives it there), and 41
of the /tmp files pass with executed cases. The Playwright pattern is removed
and the 41 files are force-included with the probe as their reason; the 36
real failures stay excluded. The curated subset grows from 573 to 631 files
and the ratchet floor follows. Windows curation moves to
scripts/lib/windows-curation.ts (the runner drops to 2000 lines).
* test(free): one install fixture cloned from a read-only seed (W6c, ENG-1)
Merge test/helpers/setup-codex-scope-fixture.ts into install-fixture.ts.
The checkout is built once per test file into a read-only seed and cloned
per fixture with copy-on-write where the filesystem has it (full copy
otherwise), restoring each file's own mode; clones never hard-link the
seed because put() and tests write in place. Callers register
afterAll(cleanupSeed); Bun's runner fires no exit event, so seeds of
killed runs are swept by pid. test/install-fixture.test.ts is the ENG-1
tripwire: put/setVersion/chmod in one clone leave the seed and a sibling
unchanged, and clones share no inodes with the read-only seed.
* test(free): split claude-overlay-setup by describe (W6b)
The 115 s serial file becomes default/lifecycle/installs files (5/4/3
tests, unchanged bodies) sharing test/helpers/claude-overlay-fixture.ts,
so the shard packer can balance them.
* test(cso): convert cso-contracts.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: cso E2E/judge cases (skill-e2e-cso*, cso-eval).
Sentence pins: 16 -> 8.
* test(cso): convert cso-preserved.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: cso E2E/judge cases (skill-e2e-cso*, cso-eval).
Sentence pins: 10 -> 3.
* test(cso): convert cso-eval.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: cso E2E/judge cases (skill-e2e-cso*, cso-eval).
Sentence pins: 7 -> 4.
* test(browser): convert aside-driver.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: Aside/browse E2E cases; the installer ban stays a meaning check.
Sentence pins: 27 -> 9.
* test(browser): convert third-party-actions.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: Aside/browse E2E cases; the installer ban stays a meaning check.
Sentence pins: 17 -> 2.
* test(skills): convert skill-validation.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: workflow judges in skill-llm-eval.
Sentence pins: 19 -> 6.
* test(skills): convert workflow-excerpt.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: workflow judges in skill-llm-eval.
Sentence pins: 37 -> 10.
* test(skills): convert question-tuning-registry-path.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: workflow judges in skill-llm-eval.
Sentence pins: 4 -> 0.
* test(skills): convert host-conformance.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: workflow judges in skill-llm-eval.
Sentence pins: 4 -> 4.
* test(memory): convert memory-ingest-landing.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: memory-ingest free tests that run the binary.
Sentence pins: 11 -> 9.
* test(windows): exclude the skip-only claude-overlay setup files from the Windows subset
Their real-setup cases are describe.skipIf(win32); after the split their bin/
spawns live in the shared helper, so the source scan stopped seeing them.
* test(skills): convert agents-digest.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 1 -> 0.
* test(skills): convert brain-preflight.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 2 -> 1.
* test(skills): convert codex-skill-isolation.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 1 -> 0.
* test(skills): convert office-hours-review.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 2 -> 1.
* test(skills): convert onboarding-moved-literals.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 2 -> 2.
* test(skills): convert open-gstack-browser-liveness.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 3 -> 2.
* test(skills): convert plan-floor-review.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 3 -> 2.
* test(skills): convert plan-review-report-recording.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 4 -> 2.
* test(skills): convert regression-1539-review-self-verify.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 4 -> 0.
* test(skills): convert skill-fixture.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 6 -> 3.
* test(skills): convert template-context-parity.test.ts sentence pins
Safety and authority sentences become expectMentions keyword checks (reword
freely, keep the rule); other English sentence pins are dropped. Headings,
labels, placeholders, commands, field names, enum/status tokens and ordering
checks stay. Behavior and wording: the owning skills' E2E cases and workflow judges; CLI/hook output and fixture-data pins in the tail were left alone.
Sentence pins: 1 -> 0.
* test: restore pins on renderer output and paid-fixture inputs
cso-contracts (the report renderer's required empty phrase), memory-ingest
(binary output), carve-plan-fixture, ship-skip-actor and cso-eval (inputs the
paid harness builds) assert product or fixture output, not skill prose, so the
value-bar narrowing does not apply to them.
* feat(eval-policy): v2 series identity, HARNESS_VERSION guard and identity backtest
W4a (CEO-27/ENG-3/DX-6), not yet wired to the writer: caseSeriesIdentitiesV2
keys a case by the bytes it owns (paid test file, fixtures, its skills' prompt
files including the generated SKILL.md) plus HARNESS_VERSION, and records the
full consumed-input fingerprint as provenance. scripts/harness-version.json
pins the blob of every shared-harness file (GLOBAL_TOUCHFILES plus the session
runners, PTY harness, e2e gate and LLM judge), one line per file;
test/harness-version.test.ts fails on drift with both fixes, and
scripts/bump-harness-version.ts records the decision: --bump (new series for
every case) or --non-behavioral (compatibility entry, no reset).
eval-trial-series --backtest replays identities over weekly main revisions
and recorded trials (ENG-3/DX-12).
* feat(pass-rates): weekly history is main's scheduled runs plus main dispatches
W4b: pass-rates reads evals-periodic history from main only, keeping scheduled
runs and main dispatches and never a branch dispatch, so back-to-back branch
validation runs neither feed pass rates nor count as weeks for quarantine
expiry or drift. --branch <name> still reads one branch's runs for inspection.
* docs(test-audit): EVAL_POLICY v2 decision memo with the identity backtest
D1 memo (CEO-33, ENG-3, DX-12, CEO-27): P(green) arithmetic, false-blocking
targets, the v2 identity rule and harness guard, weekly history, W4c evidence
(the named 'stochastic' cases pass 26/27 to 34/34, so they stay rule), the
backtest over 29 weekly main revisions and 14,898 recorded trials, what it
means for quarantine reachability, alternatives (gating vs trended, adaptive
panels not adopted, buy vs build), and the rollback proof on the pre-PR reader.
* feat(eval-policy): EVAL_POLICY v2 writer: case-owned series identity
The v2 writer, last commit of the W4 block (DX-12/ENG-14/ENG-21): the report
job's stamp now writes caseSeriesIdentitiesV2 (case-owned bytes plus
HARNESS_VERSION) as series_identity, the full consumed-input fingerprint as
series_fingerprint and harness_version as provenance, and EVAL_POLICY.version
is 2 (approved 2026-10-04, D1). The v1 touchfile identity is removed.
Rollback = revert this commit (and the W4 commits above the reader-compat
floor); readers then ignore the v2 records with a printed count.
* test: restore pins on paid-harness prompt inputs
review-enum-lifecycle and plan-review-report-recording assert the prompt a
paid case's harness builds (opts.prompt / call.prompt), a fixture contract,
not skill prose.
* test(security): check injected safety lines on meaning, not capitals
content-security, the canary instruction and the arm judge prompt still must
carry their never-follow / never-output / exact-"none" rules; the checks are
case-insensitive keyword checks so emphasis can change. The UNTRUSTED envelope
markers and "none" value stay exact tokens. (Hidden element stripping fails
locally only because Playwright Chromium is not installed; same at base.)
* test(plan-ceo): keep the outside-voice scope-summary rule as a meaning check
The first codemod pass left an empty if body (TS1313).
* chore(windows): re-record the Windows duration seed for the 631-file subset (run 37222668939)
* ci(free-tests): summarize every shard's flake ledger in the aggregate job (W7a)
The aggregate downloads flake-ledger-* (one subdirectory per shard) and
appends lane C's `scripts/test-health-report.ts flake-summary` output to the
job summary before strict verification. The runner's own flaky table is
dropped so the ledger reader stays the single source; --ci-verify keeps the
shard-overrun warning.
* feat(evals): free test files anywhere and the stored-zip helper never restore the full PR gate (W1a)
The CEO-23 replay found design/test free tests forcing fallback (36485436963).
* feat(evals): replay recorded PR pushes through today's selector (CEO-23, ENG-18)
scripts/replay-pr-selection.ts reports per push the recorded vs replayed
mode and case count, dropped gate cases, retired failed cases and every
miss (a failed case that depends on the diff but is no longer selected).
* refactor(evals): one case-sharded carve-section-loading file with a case id and touchfiles per skill (W5a)
* test(free): hold qa-deadline's full-pipe case on a FIFO, not a paused Bun stream
Bun gives a child 'pipe' stdio a unix socketpair, so whether the 2 MB fill
actually blocks depends on the parent's reader and the host's socket buffer
sizes. CI (run 36761671811, both attempts) saw exit 0 after ~130 ms; with
net.core.{w,r}mem_default=4 MiB locally the old case fails 3/3 the same way.
A FIFO the test never reads is bounded by the kernel pipe buffer, so the
receipt write genuinely blocks and the 5 s settlement deadline must fire.
The 7 s bound now starts once the preload has queued the write.
* test(free): event-driven waits in pty-output-wake
close-releases-observers: the 500 ms wall race becomes an ordering check
(the waiter is released while only SIGINT has been sent, before close's
escalation finishes), and the Bun.spawn spy fakes only the PTY launch so a
shard neighbor's in-process spawn cannot add a stray SIGTERM (CI runs
36015568981, 36036582721, 36074636848).
continuous-redraws: the fixture spinner stops once the observer has made
five coalesced reads, so the run reaches its work deadline with no redraw
in flight. Streaming right up to the deadline raced the helper's screen
drain and failed as 'PTY screen parse failed; viewport is incomplete'
(5 CI runs 09-30..10-03; 4/40 locally under 8 busy loops, 0/40 after).
* test(free): session-runner long-capture handshake waits for the collected Read event
The fake CLI's init line arms the runner's work-phase timer, so the worker's
handshake (work-ready + a >200 s timer) could pass before the runner's read
loop had collected the Read event printed just before work-ready. Advancing
the clock then killed the run with the Read lost: readSection false (CI run
37194511124). The worker now also waits for the runner's own progress line
for that Read. Forcing the observed ordering (work-ready 200 ms before the
Read line) fails the old test deterministically and passes the new one.
* test(free): budget-override restores the CI variables it rewrites
Its provenance cases set and then deleted CI, GITHUB_ACTIONS, GITHUB_REF_NAME,
GITHUB_SHA and the GitLab CI_* names without restoring them, so in a CI shard
every later file ran with CI unset. qa-only-cleanup's real browse daemons
then launched Chromium without --no-sandbox, the sandbox failed on the
runner, and both real-Chromium cases timed out with 'QA cleanup fixture did
not become ready' (shard 12 attempt 1 of six runs on 10-04, retry alone
green). Retained startup log: 'Chromium sandboxing failed!', child env with
no CI. Reproduced 3/3 with CI=true, userns denied and budget-override ahead
of qa-only-cleanup in one process; 0/3 after.
* test(free): pr-shared-input-selection names its base and package history
Every selection now passes EVALS_BASE and packageVersionOnly explicitly, so
no case reads the checkout's git state. Before, 12 calls ran
detectBaseBranch and the cumulative case diffed package.json against
merge-base(origin/main, HEAD) — the ambient input that turned main red in
runs 36572856858 and 36594314027. A PATH git spy counts 14 git calls in the
old file and 0 now.
* feat(evals): derive PR-lane dependents for unmapped files instead of restoring the full gate
scripts/pr-dependencies.ts derives, from the checked-out tree, which paid
cases consume each repository file: test import closures, repo paths and
path.join segments in code/shell/skill text (whole-line comments and
selection data excluded), bin script names, compiled binaries by outfile
path (scripts/build.sh), package scripts, template placeholders through
RESOLVERS, and host configs by host id. The global touchfiles' import
closure is global.
PR selection: an unmapped file with derived consumers puts the run in the
new 'dependents' mode (every dependent gate case and judge, no profile cut);
a tracked file under a derivable directory that nothing reaches is exempt;
periodic/marathon workflows, .gitignore and tsconfig.test.json never run
the PR gate. PR_FULL_GATE_FILES is the fail-closed list (deps, CI image,
setup, build, evals.yml and its actions, test preload, tsconfig.json,
.gitattributes, the skill generator and preamble resolvers, the eval
policy). Untracked or underivable files restore the full gate with fix
text. The summary and PR comment list derived dependents.
* feat(evals): replay misses count derived dependents; report wave vs ordinary pushes (CEO-23)
* test: drop FREE_FIXTURES entries for fixtures deleted in W8b
* test: re-shape the census attribution fixture to qa-workflow and carve case shards
* test: census fixture case order
* ci: 420 s PR-lane slices with per-slice ceilings; periodic tracking issue only on main
evals.yml joins the census lanes at the 420 s slice budget (W5c/CEO-15) and
indexes each eval slice's job timeout from .plan.sliceCiTimeoutMinutes with
the slice-deadline clock as its first step (W2c/ENG-2). evals-periodic.yml's
tracking-issue upsert and close run only on refs/heads/main, so a branch
dispatch whose gate census is skipped (CEO-04) never opens or closes#2755.
* docs: integrate lane docs for the audit wave (eval-bg backends, PR fallback, policy v2, ceilings, evidence)
- CLAUDE.md: 'Running evals as an agent' describes scripts/eval-bg.ts's
dispatch and local backends, forcing flags, the sentinel and status;
eval:select's PR-profile default; prompt-structure helper and value-bar
terms. instruction-facts now requires scripts/eval-bg.ts in that section.
- CONTRIBUTING.md: decision table gains test:health, eval:select default,
eval:bg backends and the include_gate_census / evals_all dispatch lines;
420 s runners and per-runner ceilings; flake ledger, OSV tracking issue
and expiry test, weekly test health; prompt-structure guidance.
- AGENTS.md: build commands name the eval:bg backend choice and test:health.
- docs/TESTING_INTERNALS.md: anchors the fix texts point to
(#pr-paid-lane-fallback, #pass-rate-policy-versions, #harness-version,
#flake-ledger, #duration-seed), receipts/recovery/debounce, per-slice
ceilings, the slice deadline with not_run/hung and checkpoints, the eval-bg
cap formula in place of the detach floors, EVAL_POLICY v2 history, carve
case shards via --case.
- docs/test-audit-2026-10.md: CEO-23 replay (165/168 -> 164/168
full-fallback, 0 misses), ENG-17 slice table (re-measured on this tree),
D2 prose-pin metric.
- TODOS.md: progress note on 'Let pass-rate history decide the rest'.
- test/testing-internals-anchors.test.ts: every TESTING_INTERNALS.md#anchor
named in code must match a heading.
* test(free): hold qa-deadline's blocked-output-pipe receipts on a FIFO
The five 'guard %s receipts survive a blocked output pipe' cases filled a
paused Bun child pipe with 2 MiB. That pipe is a unix socketpair, so the
fill only blocks while the host's socket buffers are smaller than the write:
with net.core.{w,r}mem_default=4 MiB the old cases fail 13 of 15 case runs
(the guard finishes before the test resumes the stream). The held stream is
now a FIFO the test drains only after resume(), bounded by the kernel pipe
buffer; Windows keeps the paused pipe. Bounds and assertions are unchanged.
* test(free): qa-deadline waits for a written pid, not just the pid file
DRIVER and LEAF write their pid with writeFileSync, so under load ready()
can see the file before its digits and fail 'expected > 0, received 0'
(seen once in 150 loaded runs). After any early failure the finally
SIGKILLs the CLI, which orphans the detached command group; a leaf whose pid
file is not yet readable then keeps the output stream open and the case
hangs to its 15 s timeout (2/80 old-file runs and 3/310 FIFO-only runs
under 8 busy loops; the instrumented hang showed the CLI already SIGKILLed
by the finally). ready() now polls within the same 3 s bound until the file
holds a positive pid.
* fix(evals): a deleted file selects the dependents of the live files that still reference it
PR selection treated every path absent from the head tree as an unknown
dependency, so any PR that deleted a file forced the full gate. A deleted
path is now placed like a live one, against the head tree: a matching
touchfile selects its cases; live code, skill text or a full-gate input that
still references it (path, extensionless import target, relative import,
bin name, ancestor directory, compiled-binary outfile; whole-line comments
and data files excluded) contributes its dependents, a full-gate referencer
restores the full gate with a 'deleted but still referenced' fix, and an
unreferenced deleted path has no dependents. Deleted PR_FULL_GATE_FILES
entries still force the full gate. Tests that used absent paths as stand-ins
for new unmapped files now use tracked root files (or an explicit tracked
set); the CEO-23 replay counts misses through those references too.
* docs: integrate free-suite lane docs (seed anchor, Windows lane, native qualification, install fixture, flake rules)
- docs/TESTING_INTERNALS.md: '### Free suite duration seed' (the anchor the
seed messages link) with DX-13, the 99th-percentile packing, --ci-plan
warning, --ci-verify overruns and the CEO-10 seed ratchet; the Windows
6-job lane, its Windows-measured seed and refresh dispatch, curation and
WINDOWS_CURATED_FLOOR; native-qualification.yml modes; the merged
install fixture; test-only timing knobs; the flake-ledger job summary,
Windows ledger names and three free-suite flake rules; zsh in the
Ubicloud mirror.
- scripts/test-health-report.ts links #free-suite-duration-seed, so the
two seed messages share one heading (the anchor test caught the split).
- CONTRIBUTING.md: Windows seed refresh and native qualification rows;
seed-source pointer. AGENTS.md: Windows six-job lane.
- docs/cso-release-qualification.md: the free CSO Docker job runs only
cso-docker-integration.test.ts.
- docs/test-audit-2026-10.md: free-suite evidence lines.
- TODOS.md: CEO-11b progress note on the pre-push hook hang.
* test: final duration seeds on the integrated tree; retire the emptied cookie-validation-phases file
Linux seed re-recorded on Ubicloud standard-16 and the Windows seed from
dispatch run 37227140871, both at c903310. The seed allowlist drops the three
entries that are no longer over the ceiling. cookie-validation-phases lost its
validation-phase pins with W8f and its Windows/Dia pins moved to
windows-native-workflows with native-qualification.yml, so the file is
deleted and the curated Windows floor rises to 638. The Windows recorder writes
the seed at its default path so GSTACK_FREE_TEST_DURATIONS never leaks into
timed tests.
* test: re-express stale PR-lane pins against the new evals.yml contract
- ci-eval-cache: the profile/fresh expressions now include evals_all=false
dispatches and pr_receipts/evals-fresh; the PR store is restored only by
recover-receipts (still scoped to repository_id + PR, PR events only),
plan-slices takes it as an artifact and never touches the cache, saves
stay PR-only, EVALS_CACHE_PR binds pr_receipts dispatches, and the report
verdict artifact carries report receipts.
- ci-paid-coordination: plan-slices needs only the two ubuntu coordination
jobs (never the image); the fork/Dependabot clause is unchanged on
plan-slices and also gates debounce and recover-receipts; only recovery
holds actions: read.
- qa-supervision-selection: auq-consistency and auq-verbose-vs-carved-ab
(W1b) reach claude-pty-runner and pty-screen through auq-sdk-capture,
exactly as auq-format-gate does.
- paid-retry-supervision: the full-validation plan uses a tracked full-gate
input (setup); an absent path is now a deletion placed by its references.
* ci(windows): record the Windows seed at the default path
* chore: HARNESS_VERSION non-behavioral decision for the audit-wave integration
* v1.91.20.0 release metadata: test, eval and CI audit wave
* test(compare-board): wait for the post-submit state the feedback POST settles
The two post-submit assertions read the board once, but the board disables
the submit button and shows the message only after its feedback POST
resolves. On a loaded Ubicloud shard the read beat the round trip (FLAKY-PASS
in the final free-suite run). Delaying the POST by 400 ms reproduces both
failures at HEAD; polling for the settled state passes.
* test(plan-eng plan-mode): run the seeded STOP-gate smoke in its own fixture project
The seeded plan proposes a custom test runner, so the review's Step 0 reads
the working tree's test setup. In the gstack checkout that is gstack's own
sharded runner, and the case timed out at 300 s in census 37228573062 after
this wave reshaped the runner. It now runs in a four-file Bun project with
git history, with the product-notice suppression auto-decide-preserved's
standalone fixture already uses. Local pass on Bun 1.4.0 + Claude Code
2.1.284: asked in 121 s, seed tokens observed.
* test(plan-mode evidence): pin the seeded eng smoke's isolated fixture cwd and env
* ci(image): Git >= 2.44 so paid shared-libs cases run the supported safe-git path
bin/gstack-safe-git prefixes every /deslop-shared-libs Git read with
--no-lazy-fetch, which noble's git 2.43 rejects (exit 129). The eval image
therefore ran every paid shared-libs case on the unsupported-Git fallback:
local history unavailable, branch HEAD unknown. shared-libs-read-only failed
on exactly that in PR run 37230639146 ("Current branch HEAD: unknown. The
safe-git probe failed (unknown option: --no-lazy-fetch, git 2.43.0)") and in
2 of 74 earlier PR runs. The image installs current Git from the Ubuntu Git
Maintainers PPA and fails its build if --no-lazy-fetch is missing; the
deliberate unsupported-Git case keeps its fake Git.
* eval: pool matching branch census trials into main's pass-rate series (D1 option b)
Approved 2026-10-05. eval:pass-rates also reads completed branch census runs
in the weekly window. A branch trial counts only toward a series a main run
also has (same case-owned bytes, HARNESS_VERSION, model and CLI), so a branch
extends main's history but never starts or becomes a case's current series;
weeks for quarantine expiry still count main runs only. The D1 backtest put
this at 67 of 210 cases with enough trials for the entry rule, against 0 of
195 on main alone.
* C2: /qa defines its outcome filename fields; QA report templates name the measured timing fields
Low judge samples in failing panels named both: qa workflow (37186854666)
'an undefined {user} in the outcome filename', and qa-only workflow
(37198445662) 'template asks for Duration / durationMs totals while
reporting.md mandates Probe budget and Guarded command time'. The qa
union stays under its 1.11 size ratio (93292 of 93294 bytes).
(cherry picked from commit e14c0c1de1)
* C2: Browser Quick gets a feasible budget; QA loop names its directory and stop path
Judge rationales on the qa-only panels (2 of the last 3) cited the 30 s
Browser Quick budget: homepage + 5 targets, each a checkpoint write and a
guarded command, left ~5 s per probe. Quick is now 180 s (method text: 3
minutes); the qa-only-no-fix paid check expects the same native deadline.
PROBE_DIR points at its source line, a stopped loop goes to the final
report steps (annotations, materialize, Markdown), and /review and /ship
say that after smoke expiry only plan checks rerun in revalidation.
(cherry picked from commit 78d4a2a343)
* qa: fit the C2 judge fixes under the qa and review size caps
Brings GSTA-21's qa.ts trims (3bd625d, ce73af0) for the cherry-picked C2
wording: PROBE_DIR keeps 'owned', the stop line points at §4, and the
post-expiry rule is one sentence, so /review fits its cap. qa measures
93,303 / 84,049 = 1.11010 against the same 1.1102 cap GSTA-21 set. The
qa-only workflow judge passed 10 of 10 local trials on this wording (7 of 10
before the C2 fixes).
* test: qa-only fixture keeps its caller-set 30 s deadline
The qa-only-no-fix fixture tells the model to start the guard with 30 s
(a caller limit), so its deadline assertion's default stays 30 000 ms; the
product's Browser Quick default (180 s) does not apply to it. Reverts the
helper and its test to their base.
(cherry picked from commit e36e622add)
* test: qa-quick sets a caller probe limit and drains past its session budget
C2 raised Browser Quick's native budget to 180 s. qa-quick is the only gate case
on the native budget, and its wall went 194 s (main periodic, pre-C2) -> 254 s and
274 s (this PR's runs) -> 300 s session timeout (run 37242691913): the agent now
probes every navigation target with a checkpoint, then needs ~80 s to report.
Like qa-only-no-fix, the eval now sets a caller probe limit (30 s), scopes the
fixture, and gives bun's outer timeout the session drain margin instead of racing
the 300 s session. The product's 180 s budget and the session budget are unchanged.
Diagnostic rerun passed in 195 s.
(cherry picked from commit f8dc64031e)
* chore: restore the audit wave's package scripts after the main merge (version 1.91.23)
* qa: keep main's one-sentence post-expiry rule for /review and /ship
The C2 variants ("step 4c reruns plan checks only" and "even in 4c") read
to the review workflow judge as conflicting with Step 5.8's repeat-pass
reruns: 6 of 10 local trials passed with the longer sentence and PR run
37248650879 failed on the shorter one. Main's "Post-expiry smoke rechecks are
not-run." passed 10 of 10. qa (9/10), qa-only (10/10) and
ship-managed-hook-refresh (10/10) were measured on the same tree.
* v1.91.24.0: move off the slot #3036 claims
* fix(models): allowlist routing for model-pinned Claude families (W1)
resolveModel lowercases input and strips one trailing [...] context marker.
Opus 4.7, Opus 4.8 and Sonnet 5 keep their pinned overlay only for the bare
ID, the bare ID plus an 8-digit date, and the bare ID plus -latest; any other
suffix (point releases such as claude-sonnet-5-5, claude-opus-4-7-1) falls to
the generic claude overlay. claude-opus-5-5 routes to claude explicitly (no new
overlay, no new ALL_MODEL_NAMES entry). Fable keeps its prefix match while
CLAUDE_FRONTIER_EVAL_MODEL is claude-fable-5-1, guarded by a test.
Tests: one test/model-overlays.test.ts case per shape in the plan, plus the
lowercase / [1m] marker / family-name cases and the Fable guard.
* docs(CLAUDE.md): define when prompt bytes are a test contract (W7 prompt-byte rule)
Adds the gstack-repo definition to the Testing section at normal volume:
prompt bytes are a contract only when software reads them or a recorded eval
shows the wording matters. Amends the test-value-bar line so 'prompt bytes'
refers to that definition. scripts/resolvers/test-value.ts and
docs/test-value-bar.md are unchanged.
* fix(browse): sidebar tab hint names real tabs.json fields and how $B resolves (s11 H1, s11 M8, s11 M9)
* fix(router): route telemetry uses the echoed SESSION_ID (s01 H1)
$_SESSION_ID is never set in the router's bash block since
bin/gstack-skill-start took over session bookkeeping, so every /gstack route
event logged an empty session id. Use the literal SESSION_ID placeholder and
tell the model to substitute the echoed value, matching the completion-status
telemetry block. No W7a pins on the changed line.
* docs(CLAUDE.md): correct stale test-infra facts (s01 H2, H3, H4, H5)
H2: TREE_MUTATING is not empty; it still holds bootstrap-retention.
H3: catalog ceiling is CATALOG_BUDGET_TOKEN_EQUIVALENTS (1,194), not 1,171.
H4: detach timeouts point at package.json's --timeout values (gate is 49320s).
H5: Writing Style lives in preamble/generate-writing-style.ts and the jargon
list is Read at runtime, not baked at generation time.
No W7a pins on the changed lines.
* fix(preamble): drop the stale jargon-list count (s01 H6)
The list has 77 terms, not 80+, and the model Reads the file anyway. No W7a
pins on the changed text.
* fix(overlay): claude.md dedicated-tools nudge stops naming tools the host may lack (s01 M1)
The nudge named Glob and Grep, which some Claude hosts do not expose. It now
names the host's dedicated file tools and its search tools when it has them,
keeping the prefer-dedicated-tools behavior the claude-dedicated-tools-vs-bash
overlay eval measures. Updates the prose comment at
test/gen-skill-docs.test.ts:643 (comment only, no assertion change). The
matching comment in test/fixtures/overlay-nudges.ts is left to its owner.
* fix(preamble): progress notes on phase changes, not a [PROGRESS] cadence (s01 M2)
Context Health told the model to periodically write a tagged [PROGRESS]
summary. No code reads that tag; the line now says when to tell the user
(finishing a phase or changing direction). Loop detection stays.
W7a pin: test/terse-build.test.ts:103 pinned 'PROGRESS'; it now checks the
default build emits a non-empty Context Health body (terse still emits '').
* fix(preamble): learnings review without incident archaeology (s01 M3)
Drops the issue number, the 43-of-44 statistic and the ALWAYS caps from the
Operational Self-Improvement step; keeps the rule that the review runs every
time and that an empty result is stated explicitly. No W7a pins on the
removed text.
* docs(CLAUDE.md): fold 'Long-running tasks' into the detach section (s01 M4, CEO W4)
The 'don't give up' section told agents to poll a plain background task with
sleep 180 + TaskOutput, which the next section forbids for evals. Its one
durable rule (keep checking until the run finishes, report progress at each
check) now lives in the detach section's poll step, pointed at the
gstack-detach EXIT sentinel, at normal volume. No tests pin the removed text.
* fix(preamble): test-failure triage questions defer to the AskUserQuestion Format (s01 M5)
The two quoted pseudo-briefs pinned a superseded question shape (RECOMMENDATION:
Choose X, a completeness score on the recommendation, no D<N>/ELI10/Net). The
triage step now gives the facts, the options with their completeness values,
and the recommended option as data, and defers the shape to the Format section.
Option labels that Step T4 and test/skill-validation.test.ts:2004-2016 depend
on are verbatim. No W7a pins on the removed text.
* fix(proactive): PROACTIVE=false means no suggestions, including 'want me to run /X?' (W5.1)
The router forbade suggesting skills when PROACTIVE is false while the preamble
told the model to ask 'want me to run it?'. Decision: false means never
auto-invoke and never suggest, including by asking. The preamble line in
generate-upgrade-check.ts now matches the router, and both quote the value
as `false`, the way gstack-skill-start prints it. Restore with
gstack-config set proactive true.
W7a pin: test/gen-skill-docs.test.ts:4488 used the old quoted sentence as a
preamble-order marker; it now finds the PROACTIVE-false rule with a
quote-tolerant, case-insensitive pattern.
* docs(ETHOS.md): correct the claim that ETHOS is injected into every preamble (W5.7)
Nothing injects ETHOS.md. Preambles carry condensed principles (completeness,
voice, search before building) and the Search Before Building section links to
this file. Edit approved by Garry for this PR.
* office-hours: fix stale refs, state-root path, profile query, SESSION_ID (s07 H1, H2, H3, M7, M10; s01 H1 office-hours note)
- Phase 4.5 points at Founder Resources instead of the nonexistent Beat 3.5 (H1).
- builder-journey.md is written to $GSTACK_STATE_ROOT, where it is opened (H2).
- builder-profile context query reads developer-profile.json, the file the
migration leaves behind; drop meaningless tail (H3).
- Drop issue numbers from three headings (M7) and the MANDATORY/NOT optional
markers on Phase 4 (M10).
- Handoff telemetry uses the echoed SESSION_ID placeholder; $_SESSION_ID is
never set in a fresh shell.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* spec: plain Phase 3 gate, drop duplicates, dead telemetry and review IDs (s07 H4, M1, M2, M3, M4, M5, M6)
- H4: Standard template heading no longer names nonexistent --bug/--feature/--refactor flags.
- M1: Phase 3 read-code-first gate at normal volume, constraint and reason kept.
- M2: drop the in-phase repeat of the read-first rule (Rules recap stays).
- M3: drop the duplicate audit-sink paragraph.
- M4: drop TTHW telemetry block and its never-set frontmatter keys.
- M5/M6: drop plan-review finding IDs from Phase 5 headings and the /ship handoff.
W7a pins: spec-template-invariants:314 (Mandatory/MUST) becomes a Phase 3
section check for the read-evidence-before-asking constraint; :203 matches
'raw spec must not be persisted' across line wraps so no duplicate paragraph
is needed to satisfy it.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* investigate: drop issue numbers from freeze-script comment (s07 M8)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* qa: replace WTF point table with named signals and a plain stop rule (s07 M9)
The self-regulation check keeps its cadence (every 5 fixes or after a revert),
its signals (reverts, unrelated files, sprawl, long runs, low-severity tail),
STOP-and-ask, and the 50-fix hard cap. The percentage table goes; the stop
threshold (two reverts, or one unrelated-file edit) stops at least as early.
W7a pins: qa-browser-preservation:220 drops 'WTF > 20%' and checks the
signals case-insensitively; skill-validation:1250 checks the test-only
exclusion rule instead of the heading name; carve-guards qa anchor
'WTF-LIKELIHOOD' becomes the '### 8f. Self-Regulation' heading.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* design-html: match Pretext docs to the vendored bundle; fix cross-block server state (s08 H1, H2, H3, H4, H7, M6)
- H1-H3: walkLineRanges, layoutNextLine and layoutWithLines signatures and
return shapes now match design-html/vendor/pretext.js; patterns 2-4 rewritten.
- H4: inline the ES-module bundle in <script type="module"> with page code in
the same module; no window.Pretext global exists.
- H7: remember the printed server URL and PID; later blocks restate them.
- M6: drop the 2-3 line cap on the refinement summary.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* make-pdf: drop nonexistent --no-syntax flag, trigger list and relative phrasing (s08 H5, M7, M9)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* design-shotgun: DESIGN.md decides what variants vary; fix $SLUG; drop history (s08 H6, M1, M2, M3; W5.6)
- H6: the office-hours lookup block evals gstack-slug before using $SLUG.
- M1 + W5.6: with a DESIGN.md, variants keep its fonts and palette and vary
layout and composition; without one they vary font, palette and layout.
The swap-the-headline test and the 'Anti-convergence directive' heading
(carve-guards anchor) stay.
- M2: drop the Claude Code version pin; run_in_background: false and its
reason stay.
- M3: /tmp-then-cp rule and reason without the incident story.
G1 (replace executor subagents) deferred to TODOS.md per plan.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* design-review: plain identity line; scope 'never read source' to the audit (s08 M4, M10)
M5 (risk-percentage stop conditions) is a UC2 gate and keeps its wording.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* design-consultation: plain CLAUDE.md design-system snippet with its reason (s08 M8)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* plan-tune: describe the never-ask hook truthfully; drop plan codes (s09 H8, H9, H10, M10; W4 plan-tune)
W4: make the text true rather than change the hook. The consent copy and
scope now say never-ask questions are auto-decided with gstack's
recommendation by the question-preference hook (one-way doors still ask),
and the profile itself never changes defaults. H10 removes the nonexistent
--dismiss command. M10 strips cathedral/Layer/D-codes and keeps mechanisms.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* context-save/restore: drop rename history; fix command-list branch rule (s09 H11, M1, M2, M3, M4, M5, M6)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* retro: correct what the skill writes; drop word cap and archaeology (s09 H13, M7, M8, M9, M25)
H13 and M7 touch neighbouring lines and were merged by hand in that order.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* health: compute the composite in code; fix example weights; drop D6 codes (s09 H12, M21, M22)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* gstack-upgrade: drop issue numbers and version history, keep rationale (s09 M20)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* canary: replace the 30-second/don't-over-analyze rule with the goal (s09 M23)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* benchmark: self-contained page discovery instead of a /canary reference (s09 M24)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* codex: drop judge vocabulary and the maintainer known-bad-list note (s10 M1, M2)
The synthesis line keeps its Recommendation: <action> because <reason> shape.
M16 (consult-mode embed rule) is a UC2 gate and keeps its wording; the
outside-voice prompt bodies sent to Codex are unchanged.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* document-generate: true redaction-fence behavior; drop thoroughness booster (s10 H8, M3)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* document-release: changelog sell test as three questions, not a 0-3 score (s10 M6)
The threshold stays (fewer than two answered = needs attention, not
replacement). M4/M5 (NEVER CLOBBER / NEVER BUMP) keep their wording; the
integrator owns the eval-gated rewrite.
W7a pin: workflow-excerpt:170 'Entries scoring <2 need attention, not
replacement' becomes a case-insensitive check that weak entries get
attention, not replacement.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* ios-clean/fix/sync: match the generator's real header, package shape and regen command (s10 H2, H3, H4, H5, M9, M12)
- H2: inventory greps the header the generator actually writes.
- H4/H5: remove the local .package(path:) entry and DebugBridge* product
deps; delete the generated DebugBridge/ and DebugBridgeGenerated/ dirs so
the Phase 3 check can pass.
- H3: regenerate accessors with /ios-sync (gstack-ios-qa-regen).
- M12: ios-fix writes the regression with the project's own test runner.
- M9: drop migration-relative sentence.
ios-qa DEMO MODE (M8) is a UC2 gate and keeps its wording.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* setup-deploy: drop User-invocable boilerplate; say default branch, not main (s10 M10, M11)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* skillify: test staged skills directly; stop after Aside scrapes; drop roadmap notes (s10 H1, H6, H7, M14, M15)
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
* W4: transcript ingest needs affirmative consent (recent|all)
normalizeTranscriptConsent() in gstack-memory-ingest reads
transcript_ingest_mode once per process. Only recent (90 days) and all
(all history) are consent; an absent key, off, legacy gate letters,
incremental and unknown values skip transcripts while other memory still
syncs. The ingest's default source set drops transcript without consent;
an explicit --sources list naming transcript overrides.
gstack-gbrain-sync reads consent once at sync start and pins a non-consented
run to an explicit source list, so a mid-run config change cannot add
transcripts. It prints the pinned skip notice on every such run (even
--quiet), records the normalized mode and skip reason in the memory stage,
restages an interrupted import when the consent policy changed, passes
--all-history on --full for mode all, and documents the default in --help.
Tests: new gstack-gbrain-sync-transcript-mode.test.ts; memory-ingest
fixtures seed consent; regression-2922's every-type case seeds consent.
Findings: s09 H16 (enforce the consent choice in code).
* W4: gstack-brain-sync holds staged transcript pages without consent
Queued transcripts/ pages stay queued and on disk unless
transcript_ingest_mode is recent or all; other classes still push.
Test: brain-sync.test.ts push-filter case (absent, off, legacy A, then recent).
* W4: gstack-config validates transcript_ingest_mode; set rejects empty and multi-line values
transcript_ingest_mode accepts only recent|all|off and leaves the stored
value unchanged otherwise. set rejects an empty value or one containing a
newline for every key instead of truncating it. The config header documents
the key; lookup_default keeps off.
* setup-gbrain, sync-gbrain: fix stale facts and strip plan archaeology
Applies s09 H1a, H1c (Mac-only wording; the installer supports Linux and
Windows), H2 (smoke test has no delete step), H3 (five invocation modes),
H4 (no CI grep test backs the telemetry claim), H7 (Step 4.5 does not exist;
the prompt is Path 4 step 4d), M11, M12, M13, M15, M18, M19 (plan codes,
issue numbers, version pins, rename story), M14 (restore-from-sync never
shipped), M16 (draft self-dialogue in the Supabase steps), M17 (passed
sunset date, manual merge into both sections).
W7a: retires the setup-gbrain-bin-invocation-paths 'now persists staged
transcripts' change-detector (t06 #17); the no-bare-invocation regex tests
keep the contract.
* W4: transcript consent prose in /setup-gbrain, /sync-gbrain and memory.md
The Step 7.5 gate is rewritten as a consent question (manual merge of s09 H5,
with H1b and H6): it checks gstack-config has, asks once with A) recent,
B) all history, E) never, names Claude Code and Codex sessions from every
repo-policy-allowed project and the destination brain, stores the mode value
(never the letter), asks yes/no when no transcripts exist, never asks
spawned or headless sessions, and drops the silent no-question bulk paths
and the false skill-start-hook claim (s09 H5c). Path 4 no longer stores off
on the user's behalf. /sync-gbrain gains Step 1.6 (ask when the key is absent
or holds a legacy value). Step 10 and the sync verdict show the mode.
setup-gbrain/memory.md gains #transcripts (values, sources, destination,
override precedence, one-time restage, held staged pages, already-ingested
pages stay).
W7a: setup-gbrain-bin-invocation-paths drops the silent-bulk and
preamble-hook pins (that prose is gone) for negative checks (no pre-answer
--bulk, no skill-start claim), loosens the gstack-config pin (t06 #18) to
the command without <choice>, and adds the DX 'gated on has' order check for
both skills.
* W5.2: /sync-gbrain is the only writer of the CLAUDE.md search-guidance block
setup-gbrain's claude-md-persist section stops writing its own
'## GBrain Search Guidance' block, which also removes the false
'auto-syncs on every gstack skill start' text (s09 H15; s09 H14 decided).
Step 10 offers to run /sync-gbrain now and names it otherwise. Test sweep:
only skill-e2e-sync-gbrain-readiness references the heading, and it checks
the block /sync-gbrain writes, so no test changes.
* E4: config-key reader tripwire in gstack-config-defaults.test.ts
Every key in bin/gstack-config's DEFAULTS table must be read by code
(gstack-config get/has <key>, a config-binary variable, a spawn argv, a
config-reader call, or an anchored ^key: pattern in bin/, lib/, scripts/,
setup or any .tmpl; wildcard keys by prefix) or appear in PROSE_ONLY_KEYS
with a reason. Run on the current tree first: every key has a reader, so
the seeded prose-only list is empty. Failure messages name the two fixes.
A synthetic-tree case checks each reader form and that docs prose does not
count. setup is scanned because timeline_stop_hook is read only there.
* W4: /open-gstack-browser asks before replacing a live browse daemon
Step 0 probes with BROWSE_NO_AUTOSTART=1 $B status (no daemon is booted by
the probe). No live daemon: clear Chromium profile locks and run plain
$B connect, which reaps orphans and refuses to replace a busy daemon. Already
headed: connect reports it. Live headless daemon: AskUserQuestion, and only
an explicit yes runs $B connect --force-restart; spawned or headless sessions
leave it running and print the stop line with the resolved browse binary.
The hand-rolled kill / kill -9 of the recorded PID and the browse.json
deletion are gone. Supersedes s10 M13.
Tests: new open-gstack-browser-liveness.test.ts (no kill of the recorded PID,
status probe before connect, --force-restart only after the yes branch,
profile-lock cleanup kept, spawned/headless stop line). The state-extraction
tripwire now covers only the port snippet, since the pid snippet is gone.
* fix(browse): untrusted-content warnings name both marker formats (s02 H5, s02 L2 / W7f)
The /browse warning named only the --- UNTRUSTED EXTERNAL CONTENT --- envelope
and the browser fallback only the ═══ UNTRUSTED WEB CONTENT ═══ one; $B prints
both (commands.ts wrapUntrustedContent, content-security.ts ENVELOPE_BEGIN).
Both surfaces now name both formats.
W7f pin: test/aside-driver.test.ts:347-350 no longer bans the EXTERNAL marker;
it reads both marker labels from browse/src/commands.ts and
browse/src/content-security.ts and requires both in the fallback and warning.
Negative control: reverting browse.ts turns the test red.
Parity cap (test/helpers/carve-guards.ts): qa maxSizeRatio 1.103 -> 1.104
(measured 1.1035); the second marker adds bytes to every skill that renders
the browser fallback.
expected red until the regeneration commit (generated SKILL.md not committed)
* fix(design): mockup board loop matches the daemon CLI (s02 H1, s02 H2, s02 H3)
- H1: $D compare --serve publishes to the design daemon and exits; read
BOARD_URL from stderr, wait with AskUserQuestion, then read feedback.json /
feedback-pending.json. The fallback no longer names a $D serve command the
step never runs.
- H2: the reload POST uses the jq --arg form, so $_DESIGN_DIR expands (the
single-quoted JSON sent the literal variable name).
- H3: Content & Microcopy lists 11 items, not 8.
No test pins the removed text. expected red until the regeneration commit
* design: name the defaults to avoid in native design-direction prompts; plain path rule (s02 H4a, s02 H4b, s02 M9)
- H4a/H4b: the design-consultation and design-sketch native subagent prompts
now name the default looks to avoid (the three looks from the Hard Rules
calibration plus the catalog entries the sibling outside-voice prompt
already lists) and say to pick again if the first idea is one of them.
- M9: 'CRITICAL PATH RULE' -> 'Path rule'; the rule and its reason stay.
No test pins the changed text. Parity cap (test/helpers/carve-guards.ts):
design-consultation maxSizeRatio 1.085 -> 1.092 (measured 1.0911). expected red until the regeneration commit
* fix(outside-voice): preflight mode list no longer looks exhaustive (s02 M10)
The preflight also prints under_codex and under_current_harness; not_authed
exists only on the Codex path. 'every other mode skips them' covers all
non-ready modes. No test pins the old line. expected red until the regeneration commit
* outside-voice: drop booster sentences and plan-review decision IDs (s02 M1, s02 M7)
- M1: native adversarial subagent prompt drops 'Be adversarial. Be thorough.';
the attacker framing, defect list and 'No compliments' output rule stay.
- M7: doc-review headings drop the D3 / T3B plan-review decision IDs (D<N> also
collides with AskUserQuestion brief numbering).
Not changed: s02 M2 (outside adversarial prompt body sent to Codex/GPT: out of
scope per plan) and s02 M3 (CODEX_BOUNDARY: UC2 gate keeps its wording).
No test pins the removed text. expected red until the regeneration commit
* confidence: drop issue IDs and the contributor-only design-doc pointer (s02 M4a, s02 M4b)
- M4a: gate heading and FP-class intro lose '#1539' and the Django sprint
measurement note; the gate rule and its four FP classes stay.
- M4b: remove the pointer to ~/.gstack-dev/plans/1539-framework-aware-review.md,
a contributor-local file that never exists where /plan-eng-review and /ship run.
W7a pins (test/regression-1539-review-self-verify.test.ts):
- :34 toMatch(/#1539/) removed (cosmetic issue-number pin; the gate-header
check stays).
- :64-67 'references the deferred design doc' replaced by a structural check
that no .gstack-dev/plans/ path reaches the emitted prose.
expected red until the regeneration commit
* question-tuning: state why the summary is piped instead of citing #2024 (s02 M6)
No test pins the resolver text (#2024 appears only in test names/comments).
expected red until the regeneration commit
* fix(testing): shared TEST_BOOTSTRAP undoes only owned changes (s03 H1, s03 H2, s03 H3, s03 M1)
The shared {{TEST_BOOTSTRAP}} (/ship, /design-review) still told the agent to
'git checkout -- package.json package-lock.json', to 'revert all bootstrap
changes', to 'delete silently' a failing first test, and to stage 'all
bootstrap files'. The QA copy (qa/sections/test-bootstrap.md.tmpl) already
says undo only owned changes, never blanket-checkout, never silently delete a
valid red regression, and stop on unrelated staged edits. The shared copy now
matches it. This removes destructive behavior (UC2: lands regardless of eval).
W7g: test/qa-browser-preservation.test.ts gains a check that the shared copy
(ship, design-review) has no 'git checkout --', no silent deletion, no
blanket revert, and keeps the owned-change and staged-edits rules
(case-insensitive). Negative control: the old testing.ts turns it red.
Parity cap (test/helpers/carve-guards.ts): ship maxSizeRatio 1.404 -> 1.405
(measured 1.4044 at the branch tip, with the s03 M2/M3 wording).
expected red until the regeneration commit
* testing: coverage trace and test generation follow the value bar (s03 M2, s03 M3)
- M2: 'Every branch in this diagram needs a test' / 'Each step in the journey
needs a test' contradicted COVERAGE_GOAL ('Test count is not a goal') in the
same rendered audit. Branches and journey steps now need coverage that would
catch a regression; the value bar decides new test vs extension vs covered.
- M3: 'Mock all external dependencies' -> the smallest native test, mocking
only services unrelated to the behavior (matches the QA handoff in qa.ts and
the audit's own E2E routing).
No test pins the removed text. expected red until the regeneration commit
* plan-gates: per-item confirmation states its reason, not an incident ID (s03 M4)
'VAS-449' meant nothing to the model and reached the user in the >5-items
warning. The rule now gives the reason (a blanket confirmation lets the user
click through without opening any file / lets unchecked items pass as
verified). The per-item confirmation rule is unchanged.
W7a pin: test/ship-plan-completion-invariants.test.ts:122 toMatch(/VAS-449/)
removed (cosmetic incident-ID pin); the per-item rule and blanket-confirm ban
checks on the same test stay. expected red until the regeneration commit
* plan-gates, review-dashboard: terminal-report rules without trait claims or 'prior versions' narrative (s03 M5, s03 M7)
- M5: exit gate drops 'will (correctly) reject it' and the 'Self-deception
failure mode' aside; keeps the contract-violation sentence (pinned by
test/gen-skill-docs.test.ts) and the body-prose-is-not-the-report rule.
- M7: report-write step states the rule and its reason (delete and append at
EOF so the report stays last) instead of the history of how it broke.
No test pins the removed text. expected red until the regeneration commit
* spec-review: default anti-shortcut clause without incident narrative (s03 M6)
Drops 'the May 2026 transcript bug ... that's the bug, recognize it' and the
caps; keeps the rule: each non-trivial finding goes through AskUserQuestion
before it enters the plan, and only a zero-finding review skips asking. The
four skill-specific branches are unchanged.
W7a pin: test/skill-ceo-section-ordering.test.ts:868-873 pinned the SHA-256
of the default clause. It now checks, on every host, that the clause still
says the plan cannot replace the interactive review, routes findings through
AskUserQuestion, and lets only zero findings reach ExitPlanMode without
asking (case-insensitive), and that the clause is host-independent. The old
clause also passes; deleting the zero-findings rule turns it red. The DX
hash pin is unchanged.
expected red until the regeneration commit
* constants: foreground-dispatch note in current-state terms at normal volume (s03 M8)
Keeps every constraint (explicit run_in_background: false, Agent tool only,
never inline or as a Skill, fallback only after a dispatched subagent fails)
and the reason (fresh-context isolation); drops 'no longer', BACKGROUND/ONLY/
WRONG/FALLBACK caps. test/ship-control-flow.test.ts and
run-in-background-guidance.test.ts check the placeholder and the explicit
flag, not this prose. expected red until the regeneration commit
* tasks-section: inline the effort ratios for every phase (s03 M9)
The non-CEO/Eng branch said 'Effort uses the AI-compression table from
CLAUDE.md', but at runtime CLAUDE.md is the user's project file; the table
exists only in gstack's own repo. Every phase now renders the inline ratios
the CEO/Eng branch already used. No test pins the removed text.
expected red until the regeneration commit
* hosts, utility: model-neutral Claude co-author trailer (s03 M10, s11 M10)
The Claude host trailer and the resolver fallback (cursor, kiro, opencode,
slate) signed commits as 'Claude Opus 4.7' whatever model ran. Both now read
'Co-Authored-By: Claude <noreply@anthropic.com>' so the trailer cannot go
stale at the next model release (the s11 report offered this neutral form;
s03 M10's hunk uses it). document-release's release body renders the same
{{CO_AUTHOR_TRAILER}} and follows on regeneration.
test/host-config.test.ts:589 only requires 'Claude' and stays green. Goldens
move on regeneration (not committed). expected red until the regeneration commit
* changelog workflow: cross-check covers user-facing changes, not every commit (s03 M11)
'Every commit must map to at least one bullet' contradicted the same block's
voice rule (never mention contributor-facing details) and CLAUDE.md's entry
rule (the diff vs main, not how the branch got there). Every user-facing
change in the commit list must be represented; merges, version bumps and
mid-branch fixes need no bullet.
W7a pin: test/gen-skill-docs.test.ts:2038 no longer pins the old sentence; it
checks the Cross-check step still compares against the step-2 commit list
and scopes to user-facing changes (case-insensitive), without re-pinning the
new sentence. expected red until the regeneration commit
* plan-ceo-review, plan-eng-review: storage paths follow the state root (s04 F2, s04 F3)
The CEO storage policy and the eng write policy named ~/.gstack/projects/ and
~/.gstack/analytics/, while the writers (tasks-section.ts, testing.ts,
spec-review.ts) use $GSTACK_STATE_ROOT. When the state root is relocated the
prose pointed at the wrong directory.
W7a pin: test/plan-review-cases.test.ts:852 pinned the stale path; it now
checks the discovery path is under $GSTACK_STATE_ROOT and that the hardcoded
~/.gstack/projects/{slug}/ is gone. expected red until the regeneration commit
* plan-ceo-review: remove the handoff-note reader and cleanup for files nothing writes (s04 F6, s04 F7)
CEO handoff notes (*-ceo-handoff-*.md) have not been written since v0.11.5.1;
no template, resolver or bin script produces them. The reader cost a bash
call on every CEO run, depended on $SLUG/$BRANCH from another shell, and
looked under $GSTACK_STATE_ROOT where pre-state-root notes never lived.
W7a pin: test/skill-ceo-section-ordering.test.ts:320 sliced up to the
removed '## Handoff Note Cleanup' heading; it now slices to the next heading
('## Review Log') with the same assertions.
Note: the ceo-mode-option / ceo-mode-prerequisite PTY cases fail under the
image's Bun 1.3.14 with or without this change and pass under Bun 1.4.0.
expected red until the regeneration commit
* plan reviews: replace context-pressure triage with the full-completion rule (s04 F1; manual merge of s05 M2)
plan-ceo-review, plan-design-review and plan-devex-review ranked sections
'under context pressure' ending in 'Everything else', which licensed dropping
sections their own anti-skip rules require. plan-eng-review already replaced
the same block with 'Complete every required stage ... do not preemptively
warn'. All three now use that rule and name what may never be shortened
(design: Step 0, Step 0.5 mockups unless the user opted out, Passes 1-7;
devex: Step 0 with persona/empathy/benchmark/magical moment/TTHW, Passes 1-8).
s04 F1 noted the same block in the s05 files; the design copy is a manual
merge of s05 M2 (drops 'non-negotiable', which contradicted the user's
explicit 'skip mockups' opt-out). No test pins the removed text.
expected red until the regeneration commit
* plan-ceo-review: shared-code guidance matches the eng review (s04 F4, s04 F5)
'DRY: flag repetition aggressively' and 'DRY violations. Be aggressive.'
contradicted plan-eng-review's newer rule (require common behavior and
improved reliability or net savings; similar-looking code alone is
insufficient), so the two reviews gave the same plan opposite advice. Both
CEO lines now follow the eng rule; Section 5 keeps the file:line citation.
No test pins the removed text. expected red until the regeneration commit
* plan reviews: normal-volume rules; eng sections report every finding (s04 F8, s04 F10, s04 F11)
- F8: eng 'at most 8 findings' per section conflicted with 'never condense or
skip' and left a 9th high-confidence finding nowhere to go; report every
main-report finding, most severe first (confidence calibration still
suppresses low-confidence ones).
- F10: drop 'It is not optional.' (the anti-skip rule already covers Section 2).
- F11: 'ALWAYS a smell' -> 'a smell'.
No test pins the removed text. Two eng PTY cases fail under Bun 1.3.14 and
pass under Bun 1.4.0 (CI pin). expected red until the regeneration commit
* plan reviews: '## CRITICAL RULE — How to ask questions' -> '## How to ask questions' (s04 F9, s05 L3)
The bodies are plain procedure; the caps label added nothing and competed
with the CRITICAL GAP output label. Applied in plan-ceo-review,
plan-design-review and plan-devex-review sections (s05 L3 promoted by the
plan).
W7a: test/devex-finding-fixture.test.ts:83-84 now finds the section by a
heading regex ending in 'How to ask questions'; the skill-e2e-plan-ceo-plan-mode
failure diagnosis names the new heading.
Judge anchors NOT changed here (builder E moves them to structural anchors):
skill-ceo-section-ordering.test.ts (old :938), workflow-excerpt.test.ts:109,296,
skill-llm-eval.test.ts:638, workflow-judge-input.test.ts:366,373; the mirror
in touchfiles.test.ts:219 also uses '## CRITICAL RULE'. These stay red until
E's anchor commit lands. expected red until the regeneration commit
* autoplan: phase drivers match the loaded review skills (s05 H1, s05 H2, s05 H3, s05 H4, s05 H5, s05 M5)
- H1/H2: the skip list names the current headings ('Step 0: Detect platform
and base branch', 'Design Outside Voices (independent)'); the old names
never matched, risking a duplicate base-branch detection or outside-voice pass.
- H3: the DX journey map is the 6-stage table from Step 0F, not 9 stages.
- H4: the DX checklist names Step 0 'DX Investigation' and scores only after
the evidence exists, as plan-devex-review requires.
- H5 (manual merge): the eng test-plan artifact uses the eng skill's
{user}-{branch}-eng-review-test-plan-{datetime}.md name under
$GSTACK_STATE_ROOT/projects/$SLUG (consistent with s04 F3).
- M5: the Decision Audit Trail separator has 7 columns to match its header.
No test pins the removed text. Three autoplan PTY cases fail under Bun 1.3.14
and pass under Bun 1.4.0. expected red until the regeneration commit
* plan-design-review: scope gate runs Step 0 before mockups (W5.4, s05 H6)
The scope gate said 'audit → mockups → Step 0' while the body runs Step 0
(0A-0D) first and 0D announces 'I'll generate visual mockups next'. Per the
W5.4 decision, the three gate phrases now read audit → Step 0 → Step 0.5
mockups. 'Reply with A, B, or C' is unchanged.
W7a pins:
- test/plan-scope-selection.test.ts:616 pinned the old order sentence; it now
checks the order of the 'After resolution:' steps (preamble, base branch,
audit, Step 0, mockups). Negative control: the old order turns it red.
- test/gen-skill-docs.test.ts:4737 normalizer strips the design-only
'Then run the pre-review audit, ... against that plan.' sentence by pattern.
expected red until the regeneration commit
* plan-design-review: mockup default and retrospective at normal volume (s05 M1, s05 M3, s05 M4)
- M1: 'YOUR PRIMARY TOOL' / 'The rule is simple ... Don't ask permission ...
Mockups ARE the plan' -> generate mockups without asking when the plan has
UI and the designer is available, with the reason; skip only under the
Step 0.5 conditions (which include the user's opt-out).
- M3: 'be MORE aggressive' on previously flagged areas -> check whether the
earlier issue recurs or was only partly fixed and say so.
- M4: drop the unverifiable 'Tier 2+ (15+ RPM)' rate-tier aside; keep the
one-at-a-time rule and the /design-shotgun pointer.
(s05 M2 landed with s04 F1.) No test pins the removed text; the remaining
failing case is the plan-scope-selection PTY case that fails under Bun 1.3.14
and passes under Bun 1.4.0. expected red until the regeneration commit
* plan-devex-review: DX TRIAGE keeps every pass and reports blockers only (W5.5, s05 H7)
The Mode Quick Reference let TRIAGE skip the benchmark, the magical moment
and passes 2/4-8, contradicting the anti-skip rule, the 0C target gate and
the 'executed all 8 DX passes' self-check. Per the W5.5 decision the TRIAGE
cells now read 'Target only' / 'Verify exists' / 'All 8, blockers only',
matching the Rating Method's 'flag adoption blockers; skip nice-to-haves'.
No test pins the removed cells. expected red until the regeneration commit
* ship, review, land-and-deploy, landing-report: fix stale cross-references (s06 H1, s06 H2, s06 H3, s06 H4, s06 H5, s06 H8, s06 M6)
- H1: TODOS-format cites /ship Step 14, not 5.5.
- H2: the land readiness review is the checklist /ship applies in Step 9 (there
is no Step 3.5).
- H3: the Apple adapter names the no-credential-talk rule by location instead
of 'line 14' (it renders at line 16).
- H4: greptile triage diffs against origin/<base> from the caller's detected
base, not origin/main.
- H5: the land Important Rules recap matches the steps (4-minute CI rounds in
Step 3; 30 seconds for merge readback and deploy status).
- H8: landing-report calls the util once per bump level (four), and drops the
unsupported 'cached by bun' claim.
- M6: landing-report says what its base detection does instead of claiming
it matches other skills.
No test pins the removed text. expected red until the regeneration commit
* fix(land-and-deploy): readiness gate uses the project's test command and the live eval store (s06 H6, s06 H7)
- H6 (manual merge): TEST_COMMAND comes from CLAUDE.md or AGENTS.md; if
neither documents one, ask the user instead of defaulting to 'bun test'
(wrong for non-Bun projects and a bare suite run in gstack). The audit
hunk's 'persist the answer to CLAUDE.md' write is not adopted: it adds a
write the readiness gate never had.
- H7: E2E and judge results are searched in the eval store the collector
writes (~/.gstack/projects/<slug>/evals/ and its shards/*/ directories),
then the legacy ~/.gstack-dev/evals/ fallback; the gate no longer reports
'no E2E tests run today' when results exist.
test/state-root-ratchet.allowlist.json: allowlist the EVAL_DIR line with its
reason (test/helpers/eval-store.ts writes under os.homedir()/.gstack, so the
reader must use the same path, not $GSTACK_STATE_ROOT).
Parity cap (test/helpers/carve-guards.ts): land-and-deploy maxSizeRatio
1.10 -> 1.108 (measured 1.1076 at the branch tip, including the W7f marker).
expected red until the regeneration commit
* ship: Step 9 Pass 1 defers to the checklist's critical categories (W5.3, s06 H9)
/ship restated the checklist's Pass 1 as two of its five CRITICAL categories
(missing Race Conditions, Shell Injection, Enum Completeness). Per the W5.3
decision it now names 'the checklist's Pass 1 categories', matching /review.
No test pins the removed text. expected red until the regeneration commit
* land-and-deploy: host-neutral post-merge branch example (s06 M5)
The squash-merge cleanup example created 'codex/post-merge-pr-N' branches on
every host. The example is now 'post-merge/pr-N', naming Codex Desktop's
codex/ prefix as the host-specific case and keeping the detached-HEAD reason
and the no-force-push rule. No test pins the removed text.
expected red until the regeneration commit
* ship: drop issue/incident IDs and the 'continue silently' suppressor (s06 M2, s06 M3, s06 M4, s06 M7)
- M2/M3: '(#1946)' and '(#1079)' removed from the credential guard and REST
fallback headings; the rules are unchanged.
- M4: the audit-failure fallback states the reason (a skipped audit lets
unverified plan items ship as done) instead of 'the failure shape that
VAS-449 surfaced'.
- M7: 'Continue silently — just note the counts briefly' -> report the pass
counts in one line and continue to Step 6 (Step 16 cites these counts).
Not changed: s06 M1, the Step 16 IRON LAW (UC2 gate keeps its wording).
W7a pin: test/ship-plan-completion-invariants.test.ts:127 pinned the VAS-449
sentence; it now checks that the audit-failure fallback still forbids a
silent fail-open (case-insensitive). Negative control: deleting that
sentence turns it red. expected red until the regeneration commit
* test: anchor plan-design excerpts on the question-procedure heading, not '## CRITICAL RULE' (W7a judge-anchor move; workflow-excerpt.test.ts:109,296)
The end marker now accepts a RegExp. ASK_QUESTIONS_HEADING matches the heading
with or without its 'CRITICAL RULE — ' prefix, so the excerpt is byte-identical
on the current templates and survives the s04 F9 / s05 L3 rename.
* test: plan-design judge-input case anchors on the question-procedure heading (W7a judge-anchor move; workflow-judge-input.test.ts:366,373)
* test: plan-design workflow judge ends its slice at the question-procedure heading (W7a judge-anchor move; skill-llm-eval.test.ts:638)
Judge input is byte-identical on the current templates; judge prompts and
schemas are unchanged.
* test: CEO governing-stage order anchors on the question-procedure heading (W7a judge-anchor move; skill-ceo-section-ordering.test.ts:938)
* test: dedicated-tools overlay gates on correct output plus 20% fewer Bash calls (Eng W7 overlay predicate; replaces overlay-nudges.ts:213 taskCorrect)
Every ON trial must pass the fixture's verify (already a valid-measurement
requirement) and lowerIsBetter20Pct must hold; when the OFF median is 0, every
ON trial must also be 0. assessOverlayArms enforces both through a new
fixture.gate. Overlay contract bumps to v3 so v2 records are never
reinterpreted. overlay-measurement gains rows for correct+worse (fails),
20%+incorrect (fails) and the zero baseline (passes only at 0). The prose
comment follows the s01 M1 claude.md wording.
* test: document-release E2E stops coaching the rules it measures (W7b; UC2 gate eval for NEVER CLOBBER / NEVER BUMP)
Drops the IMPORTANT block at skill-e2e-workflow.test.ts:76-83, including
'auto-approve everything' and the CHANGELOG/VERSION instructions; keeps only
the environment facts (non-interactive, no remote). Asserts the outcome: the
original CHANGELOG entries remain, VERSION still reads 1.1.1, no commit after
the fixture head touches VERSION, and README mentions Feature C.
* test: routing E2E uses the routing instruction gstack ships (W7b routing fixture; skill-routing-e2e.test.ts:113)
readShippedSkillRouting() parses the '## Skill routing' section from the
routing-injection heredoc in bin/gstack-skill-start at test time. The journey
fixture now carries that instruction instead of 'ALWAYS invoke … FIRST action.
Do NOT answer directly', and still withholds the per-skill rule list so a
regressed description can fail. bin/gstack-skill-start and the helper join the
journey, context-skills and skillify touchfiles. A free check catches heredoc
drift. Runs on unchanged templates set the new baseline.
* test: context-skills E2E routes with the shipped ## Skill routing section (W7b routing fixture)
Replaces the shouted 'ALWAYS invoke it … FIRST action' fixture and its
hand-written context-save/restore rules with the section gstack appends to
CLAUDE.md, which already routes save/resume to those skills.
* test: skillify E2E stops scripting the skill and routes with the shipped section (W7b skillify prompt + routing fixture; W7d skillify narration ban)
- CLAUDE.md fixture: the shipped ## Skill routing section replaces 'ALWAYS
invoke it via the Skill tool as your FIRST action' and its hand-written rules.
- scrape-match-path: prompt is 'Run /scrape latest hacker news stories'; the
'You MUST follow … 1. Run skill list … Do NOT enter the prototype phase'
script is gone. Asserts the outcome: the bundled skill ran, no goto
prototyping, no new skill or skillify staging written. Listing is logged.
- scrape-prototype-path: drops the five-step script and the self-fulfilling
'Suggest /skillify'; keeps the user's JSON shape request.
- skillify-provenance-refusal: prompt no longer dictates the refusal.
- Narration grader moves to helpers/skill-body-narration.ts: line-start
'I '/'I'll '/'Let me ' plus mid-sentence 'let me <verb>', so 'let me know'
is legal. Stored known-good/known-bad bodies show it still rejects every
known-bad case the old /Let me /i ban caught.
* test: land-and-deploy E2Es state environment facts instead of scripting the skill (W7b deploy prompts)
The workflow, first-run and review-gate prompts drop the 'IMPORTANT … simulate:
1. Detect … 2. Infer … 3. Note …' scripts and the answers they supplied
(platform, app, URL, merge method, 'Eng Review should be NOT RUN'). They keep
the facts the fixture cannot provide (no remote, no PR, gh/fly unavailable,
gstack-review-read output, report directory). Assertions now check the outcome:
the artifacts name Fly and test-app, and the readiness report records the
missing review. Canary, benchmark and setup-deploy prompts are unchanged.
* test: ship triage asserts each failure's in-branch/pre-existing label (W7b ship triage)
The old check passed if the words 'in-branch', 'pre-existing', 'math' and
'string' appeared anywhere in the final message, in any pairing. The prompt
now asks for a closing JSON map; helpers/ship-triage-labels.ts reads it (or
unambiguous per-failure lines) and the test requires truncate/string ->
in-branch and divide/math -> pre-existing, plus a Bash call that ran the
suite. Stored outputs show swapped, words-only and one-label runs fail.
Drops the untyped recordE2E extras (typecheck baseline shrinks by one).
* test: design-review-fix asserts a fixed design issue, not only the Aside ritual (W7b design-review-fix)
Keeps the engine-routing checks (Aside driven, retired binary unused) and adds
the outcome: design-audit.md exists, names a seeded defect, and index.html or
style.css differs from the initial fixture commit. The style(design) commit
prefix is not required. This case still cannot gate the design-review
stop-condition rewrite (it self-skips without Aside and caps fixes in its
prompt), per the Eng item.
* test: CEO HOLD preservation grader checks the decision, not its phrasing (W7d CEO mode-routing regex judge, ceo-mode-option.ts:410-440)
Drops the exact 'Keep <x> in scope' / 'Defer <x> to TODOS' labels, the
'D<n> — Keep … in scope, or defer' title, the '<plan>.md lists' phrase and the
2-of-5 keyword quota. Keeps the evidence bindings (native answer, session,
timestamps, source read, mode context, later withdrawal) and now requires: one
keep option and one defer option, the answer selects keep, a word of the kept
label names a listed plan requirement, no expansion language, and an ELI10 +
Stakes rationale of at least 20 words (so a bare mode label still fails).
Stored captures: both retained known-good runs pass; every existing known-bad
mutation still fails; four rewordings the old grader rejected now pass; three
new known-bad cases fail. The Net/Note single-brief checks (t02 #3) are
unchanged: the 'second sentence after Net' row adds scope ('Also add shared
views'), so it stays a known-bad.
* test: docsync observer accepts harmless read wrappers (W7d docsync observer)
docsCommandAllowed now unwraps, on read-only commands only (pwd, ls, cat,
stat, sha256sum, Git reads): a leading 'cd <fixture repo> &&', '2>/dev/null',
a trailing '|| true', '--no-pager', '-C <fixture repo>' and '-c' with
core.pager=cat / core.quotepath / color.* settings. Lifecycle and script calls
stay literal so their failures cannot be suppressed. Writes, other repos,
--git-dir/--work-tree, config that can execute code, real composition and
redirects stay rejected.
Stored commands (fixtures/docsync-observer-commands.json): 11 known-good read
forms, which the old observer rejected, now pass; all 17 known-bad forms still
fail under the new and old observers. docsync-command-grammar moves its three
harmless rows to an accept test and keeps unsafe variants of them as rejects.
* test: AUQ matrix fails only on the format fields software reads (W7d AUQ matrix emoji check)
auqMachineFormatProblems() fails a first-question run when the
Recommendation: line is missing or empty, or when the brief does not label
exactly one option (recommended); recommendation substance >= 4 is unchanged.
ELI10, Pros / cons, ✅/❌ and Net: are still scored and logged but no longer
fail a paid case. scoreAuqFormat and the consistency suite are unchanged.
Stored briefs: three known-good (including one without emoji or Net) pass;
four known-bad (no or empty Recommendation, zero or two (recommended)) fail.
* test: coverage audit grades a closing tested/untested summary before the diagram grammar (W7d coverage-audit-evidence)
The diagram grammar grew through four waves of capture patches to accept
correct-but-different diagrams. The E2E prompt now asks for one closing JSON
line {"tested": [...], "untested": [...]}; coverageAuditVerdict passes the
outcome when processPayment is only in tested and refundPayment only in
untested (last summary wins; quoted or example summaries ignored). The legacy
grammar stays as a fallback, so every stored capture keeps its verdict
(all existing rows unchanged). Read evidence and capture success are still
required. Stored summaries: three known-good (which the old grader rejected)
pass; seven known-bad fail.
* test: disabled outside-plan grader reads the run's own coverage self-report (W7d disabled-plan-review-evidence)
The false-completion oracle kept being patched to tell historical review-log
mentions from current claims. The E2E prompt now asks for a closing JSON line
{"outside_review_this_run": ...}. When present, the coverage claim is that
report ('completed' fails) plus any undenied 'both reviewers agree';
'disabled' also satisfies attribution. Runs without it keep the existing
prose oracle, so all stored captures and mutation rows keep their verdicts.
Dispatch, CLI execution, preflight and persisted-disabled checks are
unchanged. Stored outputs: three known-good pass (the old grader rejected
the table form); four known-bad fail.
* test: investigate boundary grader checks the verified fix, not one Edit and one verify (W7d investigate exactly-one ritual)
Completion now needs at least one allowed Edit, the correction in place, at
least one verifier run whose final status is 0, and no edit after the final
verification. The ending-error case still requires a single verifier run
(status 69), because that fixture's error tells the actor to stop without
retrying. Ownership acquisition/release checks are unchanged. New free
protocol controls: a passing re-verification passes (the old grader failed
it); a failing final re-verification and a retry after the ending error both
fail, as they did before.
* test: CEO split-overflow call floor is two, the minimum for five options at four per call (W7d split-overflow 4-call floor)
The regression is a candidate silently dropped to fit the four-option cap.
The scope validator already requires an offered Include/Defer/Cut disposition
for all five candidates; the extra N-1 call floor failed agents that batch
compatible candidates, which the test header calls legitimate. FLOOR now comes
from CEO_SPLIT_CALL_FLOOR = 2. Stored shapes (ceo-split-floor.test.ts): five
sequential, 2-call and 3-call batches pass; a dropped candidate fails with
'missing target decisions'. The registration controls keep their verdicts;
the 3-call plan_ready scenario still fails, now on missing decisions.
* test: shared-libs PR coverage checks the stated page budget, not per-PR ceilings from one run (W7d shared-libs per-endpoint page caps)
prCoverageRequestViolations keeps the skill's stated older-open-PR budget
(five metadata pages, fifty file-list pages plus recent PR 7) and the coverage
outcome (PR 42's file list read from page 1 through page 2). It drops the PR 42
three/four-page and single-first-page ceilings, which were tuned to one
observed capture with a truncated first view. The judge criteria are
unchanged. Stored endpoint lists: three known-good (including a re-read first
page the old ceilings rejected) pass; four known-bad plus an over-budget
file-list run fail. incompleteFirstFileView is no longer used by the paid case.
* test: test-value review accepts any caller search, not only the literal 'git grep' (W7d literal git grep in test-value)
recordsNonTestCallerSearch keeps the value-card contract (non_test_callers
recorded) and accepts git grep, rg, grep or the Grep tool as the search.
Severity checks and the golden exclusion are unchanged. Stored evidence:
four known-good (three the old check rejected) pass; three known-bad (no
search, no caller count, prose only) fail.
* test: plan-format accepts any statement that kind-differentiated options are not comparable on coverage (W7d 'options differ in kind' phrase)
The outcome, no fabricated Completeness: N/10, stays asserted. The required
note now matches any brief wording of the same fact (KIND_NOTE_RE in
helpers/plan-format-kind-note.ts). The format-compliance suite that pins the
declared kind-note string is unchanged. Stored lines: four known-good pass,
three known-bad (a completeness score, a recommendation, 'kind to the user')
fail.
* test: plan-mode smokes check the targeted plan, not the scope-gate announcement, and reject plan-file ## Decisions (W7d exact scope-gate line; W7g plan-mode ## Decisions)
- Seeded eng/design plan-mode cases keep 'no scope menu' and replace the
exact 'Scope gate: plan mode — auto-selected B' requirement with
seededPlanTargeted(): the announcement or native seeded-plan selection, or
seed-only plan names appearing in the review output after the slash command
(trackTokens). The announcement is logged.
- The AskUserQuestion format forbids writing a decision to the plan file as a
substitute for asking. assertNoPlanFileDecisions() fails any plan_ready run
whose plan carries ## Decisions (eng, design, devex, office-hours
auto-mode). The AUQ-blocked cases (eng seeded, devex, office-hours) now
require outcome 'asked' (AskUserQuestion variant or prose fallback).
- plan-mode-evidence registration controls keep their verdicts; the
missing-decisions control now fails on 'without a question'.
- plan-mode-targeting.test.ts stores known-good (announcement; paraphrase plus
seed content) and known-bad (neither; an untracked token) observations and
plan files with and without ## Decisions.
* test: plan-design judge selection check anchors on the question-procedure heading (W7a judge-anchor move; touchfiles.test.ts:219)
Same structural anchor as the other judge-anchor moves; selects the same text
on the current templates and after the '## How to ask questions' rename.
* test: CEO plan-mode failure message names the current rule, not the '## CRITICAL RULE' heading (t13 F20)
* test: browse reference judge gates on its stated floors; the baseline comparison is recorded, not a ratchet (W7d browse judge baseline ratchet; t08 F7)
expect(regressions).toEqual([]) compared a noisy three-sample mean against
eval-baselines.json (clarity 4), silently raising the stated clarity floor of
3. The comparison stays in judge_reasoning; pass and assertions use
BROWSE_JUDGE_FLOORS (clarity 3, completeness 4, actionability 4). Judge
prompt and schema are unchanged. Stored panel means: three known-good (two the
ratchet rejected) pass; three below-floor panels fail.
* test: eval-kinds drops the exact 22 behavior / 25 judge census (W7d exact count in eval-kinds; t12 F6)
The kind contract is already asserted: every live case has one kind, kinds
are rule/behavior/judge, every LLM judge is judge-kind, every behavior case
has a literal registration and a BEHAVIOR_WHY, and the named contract-shaped
cases stay rule. The census count only broke on adding or retiring a case.
* test: plan-eng artifact case binds to the hermetic state root and requires the artifact; report case drops its shouted requirement (W7g plan-artifact fixture binding; W7g skill-e2e-plan.test.ts:479,720)
- plan-eng-review-artifact: actor, observer and cleanup use one isolated
GSTACK_HOME (the hermetic run's, or a seeded temporary one when hermetic
mode is off) and the slug gstack-slug resolves for the fixture, instead of
os.homedir()/.gstack/projects/test-project. A guard refuses a state root
under the real home, and cleanup removes only the fixture's project
directory. The 'IMPORTANT … you MUST write the test-plan artifact' line is
replaced by the environment fact (state root and slug). The soft warning
becomes an assertion: exactly one new *eng-review-test-plan-*.md about the
reviewed dashboard change.
- plan-review-report: removes 'CRITICAL REQUIREMENT … MOST IMPORTANT
deliverable' and the restated report contract; keeps 'plan.md is the plan
file'. The existing assertions (report present, last, rows, unresolved
status) now measure the skill's own report prose.
* test: session-runner-tools checks harness prompt properties, not a byte-for-byte copy (W7e session-runner-tools.test.ts:544; t09 F7, F8)
The ship capture prompt test asserts its purpose (no CEO-only writing
guidance) plus the functional rules: one allowed skill path, AskUserQuestion
unavailable, the section Read rule, no git/gh/mutation, and the REPORT.md
path. The two 'MUST actually Read' pins match the Read rule case-insensitively.
Negative control: dropping the no-mutation rule from the harness prompt turns
the test red.
* test: section-capture native tools checks harness prompt properties, not a copied golden (W7e section-capture-native-tools.test.ts:467; t04 F4)
Asserts the absolute skill path is the only SKILL.md named, the scenario,
AskUserQuestion unavailable, the section Read rule (case-insensitive, also at
:308), no git/gh/mutation, and the REPORT.md path. argv/tool-list checks are
unchanged. Negative control: dropping the no-mutation rule turns it red.
* test: office-hours review-preservation judge test keeps its schema and untrusted-data checks, not its wording (W7e office-hours-completion.test.ts:416-429; t09 F10)
Drops 13 phrase pins on the judge prompt. Kept: both inputs carried as JSON,
the response schema (wire format), and the separate untrusted-DATA test. The
validator behavior tests already use stub judges. Negative control: changing
the schema string in the prompt builder turns the test red.
* test: docsync lifecycle interface drops pins on its own helper prompt (W7e docsync-lifecycle-interface.test.ts; t02 #17)
docsNativeInterface is the test helper's own prompt, so its sentence pins
only checked that the helper says what it says. Kept: owned skill path, the
two extracted lifecycle commands (exact), the scripts-list interpolation, the
repo path and Git read examples, and every docsCommandAllowed /
docsToolFailures behavior check that enforces the interface.
* test: shared-libs revalidation prompt keeps its machine-shaped checks, not ~69 phrase pins (W7e shared-libs-revalidation-prompt; t11 #3, #8; t05 F17)
reviewRevalidationPrompt and reviewPrompt are test-owned harness prompts
(non-test callers: 0); the paid shared-libs E2Es measure whether they work.
Kept: the three exact receipt commands, base-prompt prefix and resumed suffix
identity, branch selection (actor / default / resumed coverage), the state
path and turn-budget interpolations, read-record-before-diff and
first-response-before-start ordering, no embedded token, no supplied Skip
answer, reusable:true before reusable:false, the captured R59 stage boundaries
in the exclusion list, asset roots and interfaces, and the stale step-6
reference ban. The review template's Step 4.7/4.8 completion rule is matched
by step number on one line, not by sentence. Negative control: reordering the
diff-read instruction turns the test red.
* test: plan-design SDK fixture pins its harness prompt once, by ordered properties (W7e plan-design-sdk-fixture.test.ts:91-120; t01 F13)
The prompt was pinned three ways (sha256, ~20 sentences, an order check).
Kept: the skill and lazy-section paths, the plan path, the ordered
read -> review -> edit -> Write -> read-back -> summary steps, and the
no-browsing rule (case-insensitive). Dropped the hash and sentence pins.
Negative control: removing the read-back step turns it red.
* docs: overlay benchmark contract v3 (dedicated-tools gate = correct output + 20% fewer Bash)
* test: dedicated-tools overlay case pinned to claude-opus-5-5 (E2)
* chore: regenerate SKILL.md, goldens and context-budget fixture for the Opus 5.5 prompt cleanup
* document-release: CHANGELOG-preservation and VERSION-ask rules at normal volume (s10 M4, M5; UC2, eval-gated)
* chore: regenerate document-release
* Revert "test: dedicated-tools overlay case pinned to claude-opus-5-5 (E2)"
This reverts commit 737bf478d9.
* test: memory-pipeline E2E seeds transcript consent (W4 consent-first default)
* test: plan-review-report free check follows the de-coached prompt (W7g)
* test(W7h): drop issue-number pins in land-and-deploy-postfail, match guards case-insensitively
W7h (t09 F12, F13). One test file.
Removed assertion groups:
- `cli/cli#3442` / `cli/cli#13380` pins (t09 F12). Invariant: none of its
own (upstream archaeology; the IDs stay in the template, only the pin goes).
The behavior they decorated, "after any non-zero `gh pr merge` exit, query
authoritative state before retrying", is now a kept free meaning check.
- `Universal invariant` label pin. Invariant: query-before-retry. Remaining
protection: the new case-insensitive check
`/non-zero exit from `gh pr merge`, query authoritative PR state before retrying/i`
(stronger: the mutation that inverts the rule now goes red; the old test
stayed green).
Loosened to case-insensitive meaning checks (kept, W7 keep list, data-loss
and destructive-op guards): STOP worktree cleanup without removing, no
`--force`, primary working tree never removed, CLOSED -> stop, never replay a
merge after MERGED, one direct fallback, no fallback from a direct attempt,
the OPEN/no-auto/no-queue fallback precondition, branch-absent vs check-failed
messages. Gate-tier E2E also covering the flow: land-and-deploy-workflow
(gate), judge `land-and-deploy/SKILL.md workflow` (gate lane).
Cosmetic pins dropped (no control needed): capitals on STOP / Do NOT.
Mutation log: .context/mutation-land-and-deploy-postfail.md (12 mutations,
0 unexpected: reorder, field delete and 7 safety deletes red; 3 rewordings green).
* test(W7c): ship-apple-gate matches its gates on meaning, not capitals
W7c (t06 #19). One test file.
Removed assertion groups:
- "A genuine blocker may require a safety or named documentation-risk
decision" and "routine interactions and blocking decisions above" sentence
pins. Invariant: routine release authorization never waives a blocking
safety/docs decision. Remaining protection: kept free meaning check
`/stop for that decision rather than treating release authorization as a waiver/i`
plus `/named documentation-risk exception before distribution/i`.
- Full-sentence "in `read-only` mode against the selected release source".
Invariant: the docs preflight is read-only. Remaining protection: kept free
check on "`read-only` mode".
Loosened (case-insensitive, kept): apply-only-to-repository-landing scope
line, CLASSIFY-before-credentials (credential boundary), never abort a store
release over branch topology (demonstrated failure), two routine interactions
(numeric limit), stop-for-decision (consent/stop line).
Kept unchanged: manifest trigger routing, Apple read before the branch gate,
byte-exact branch gate appearing once, docs preflight before the
authorization moment, fastlane/iris/appPriceSchedules command anchors,
stale-wording bans. No gate-tier E2E covers the Apple adapter, so every
invariant keeps a free check here.
Cosmetic pins dropped: capitals on ONLY / CLASSIFY / STOP.
Mutation log: .context/mutation-ship-apple-gate.md (14 mutations, 0
unexpected: 2 reorders, 1 marker and 6 safety mutations red; 5 rewordings green).
* test(W7c): ship-reentry-gates keeps stop/consent meaning, drops sentence pins
W7c (t08 F3). One test file.
Removed assertion groups (invariant -> remaining protection):
- "Retain queued fixes and restore coverage" -> a missing reviewer keeps
queued fixes: gate E2E ship-skipped-queued-finding (gate) and judge
`ship/SKILL.md workflow` (gate lane); the stop itself stays a free check.
- qa decline print line ("Test bootstrap previously declined — skipping.")
-> qa skips bootstrap after a saved decline: kept free meaning check
`/BOOTSTRAP_DECLINED ... skip the rest of bootstrap/i`.
- "with the user's free-text evidence" -> UNVERIFIABLE items are confirmed
one by one: kept free `/per-item confirmation/i` here plus the
blanket-confirm check in ship-plan-completion-invariants.
- Docs reentry sentences ("On reentry, reuse only ...", its duplicate, "base/
input hashes still match", "retain its actual status and scope", "Otherwise
use Blocked recovery ...") -> reentry reuses only this invocation's
validated evidence: kept free checks "never reuse an audit across
invocations" and "reentry never resets the count or authorizes a launch";
gate E2Es ship-docsync-stale-before, ship-docsync-stale-after,
ship-docsync-recovery, ship-docsync-late-result (all gate).
- "left uncommitted after Step 15" -> late commits take only remaining
changes: kept free "never create an empty commit" and "preserve unrelated
user files"; gate E2E ship-local-workflow (gate).
Loosened to case-insensitive meaning checks (stop/consent/approval lines,
always kept): STOP blocks advancement, answer each AskUserQuestion, routine
authorization never waives gates, dispatched-reviewer-missing -> stop,
CYCLES >= 3 -> stop, Any N -> stop as NOT DONE, resume only after verified,
no second deferral, per-item confirmation, explicit Add-tests choice overrides
the decline for this invocation only, framework approval, first entry
launches the audit, no third attempt, unwaivable redaction/security gates.
Kept unchanged: bootstrap branch order, no `rm` in the decline branch, the
Step 14 -> 14.5 routing line, stale-string bans, `NEW_TITLE` field and
`--title "$NEW_TITLE"`.
Cosmetic pins dropped: STOP capitals, "mandatory".
Mutation log: .context/mutation-ship-reentry-gates.md (17 mutations, 0
unexpected: 1 reorder, 2 marker and 9 safety mutations red; 5 rewordings green).
* test(W7c): shrink ceo-workflow-clarity to structural and meaning checks
W7c conversion (t05 F5-F8). Pin lines 48 -> 29 (toContain|toMatch|indexOf|includes|ordered); sentence pins
collapse to one case-insensitive meaning check per invariant. Per the Eng
"Retirements" item, CEO invariants count as periodic-only once
plan-ceo-review-benefits retires, so every invariant below keeps a free check.
Removed assertion groups (invariant -> remaining protection):
- Required/Pending/Settled status sentences (4 pins): pending work stays
unapproved; only exact authority settles a choice -> kept free: label order
plus /Pending: unapproved/i and /Settled: ... a recommendation does not/i.
- Admin-route sentences (Skip steps 1-4, 0E/0H ownership, admin answer that
requests a plan change, 0G proposals use this route, mode-change pointer):
admin menus approve nothing and return locally -> kept free: Admin < Plan
decision < steps 1-4 < checkpoints order, /this approves no plan changes/i,
/0D returns to its caller, not to mode selection/i.
- not.toContain('0D never restarts mode selection') and
not.toContain('using the Step 0E mode-handoff format') (t05 F8, bans of
retired phrasings): same invariants -> kept free: the positive route checks.
- '**STOP for the actual answer, even for a lone option.**' exact bold pin:
wait for the answer before recording -> kept free: /stop for the actual
answer/i, located between step 4 and the post-answer checkpoint.
- Mode-change sentences (repeat handoff record, complete newly applicable
Step 0 work, resume the paused step): a mode change waits for an answer ->
kept free: /pause and ask ... keep the mode until answered/i and
/selecting a mode does not approve changes/i; routing table rows kept exact.
- Scope-count sentences: reuse counts against limits, mode recommendation
excludes unchanged reuse, neither approves -> kept free: three short
case-insensitive meaning checks plus the >15 / 8-file / 2-class numbers and
the route order.
- Completion-fraction sentences (select answered questions, exclude unscored,
Y/X definitions): fraction definition -> kept free: 10/7/3 scale, reopened
choice counted once, 'Report X/Y, or `N/A` when Y is zero' (output field).
- Section-gate sentences ('0D's Plan decision route through its post-answer
save', 'Resolve critical risks now') and the literal Decision gate count 11
(t05 F7): one gate per section -> kept free: path order, /no path selects
the mode again/i, gate count derived from '### Section N:' headings.
- Section 1 opening sentences (retain mode/rationale, governing row ID): Section
1 publishes current dispositions, not a mode handoff -> kept free:
**Current scope** present, the four disposition names, /not another mode
handoff; do not ask or log the mode again/i.
Gate-tier judge that also grades this prose: 'plan-ceo-review/SKILL.md modes'.
Mutation pass (scratch worktree, generated plan-ceo-review skill): 16/16 as
expected; 11 kept-step/marker/safety/numeric mutations go red, 5 rewordings of
dropped sentences (and a lowered-register STOP line) stay green. Log:
.context/mutation-ceo-workflow-clarity.md.
* test(W7c): shrink eng-review-routing to structural and meaning checks
W7c conversion (t02 #4-#10, W7a s04 F15 pin). Pin lines 88 -> 74 (toContain|toMatch|indexOf|includes|ordered); ~50
full-sentence pins become order checks, field/marker checks and short
case-insensitive meaning checks. Remaining gate-tier judge for the plan-eng
prose: 'plan-eng-review/SKILL.md sections' (judge input is ENG_REVIEW_EXCERPT,
which the first test proves contains review-sections and Recovery routing).
Removed assertion groups (invariant -> remaining protection):
- Preparation sentences ('Follow the blocks below in order after startup',
'Confidence Calibration and Decision procedure are reference rules'): blocks
run in order -> kept free: heading order check now starts at
'## Review preparation'; judge 'plan-eng-review/SKILL.md sections'.
- Priority/compression sentences ('Shorten only optional commentary', 'never
Scope Challenge, Sections 1-4...', 'Never condense, abbreviate or skip a
section'): compression cannot drop required stages -> kept free:
/complete every required stage, decision gate and output/i and the
'Everything else' stale ban; judge.
- Exact seven-label toEqual on Scope Challenge assessments (t02 #8): all
seven checks present -> kept free: arrayContaining on the same labels plus
the A/B/C heading toEqual (excerpt anchors).
- 'Complete these checks before the complexity decision in B': A runs before
B -> kept free: heading order A < B < C.
- Below-threshold / findings sentences (numbered step full sentences): C
always runs, findings approve nothing -> kept free: step-number order,
/fewer than 8 files AND fewer than 2 new classes/i, /run C whether B was
completed or skipped/i, /approve no remedies/i, '"No issues found"'.
- '... STOP before Section 1' exact caps (t02 #4): high complexity halts ->
kept free: /at 8\+ files or 2\+ new classes\/services, stop before
section 1/i.
- High-complexity sentences ('With no proposed cuts...', 'Both retain the same
approved feature list...', 'Initial scope selectors need no grid...'): cuts
and structure asked separately -> kept free: step order with short anchors,
'Pending remedies not decided here: <ids>' field; judge.
- Smaller-arrangement sentences ('offer confirmation of the original
arrangement or a pause', 'then return to this structure selector'): a pause
authorizes no cut -> kept free: order + /do not continue to C until it is
settled/i + /investigate only the agreed question/i.
- Selector summary sentences ('post-answer scope summary, not a remedy's pending
ledger record', 'Do not invent a pre-answer record afterward', 'Scope
Challenge B also uses its own selectors'): answers saved before applying ->
kept free: field template order, /a failed save or read blocks
advancement/i, '**not persisted**', /these selections approve no
engineering remedy/i.
- Decision-procedure Read-choreography anchors ('use Read to fetch the entire
saved record', 'Read the entire resolution block, including State') and the
bold '**STOP until the actual answer arrives.**' anchor (t02 #5/#6, s04 F18):
ask once, wait, record before the next choice -> kept free: Compare <
Pending-record checkpoint < Send once < AskUserQuestion call < Record < next
choice, /stop until the actual answer arrives/i located between Send and
Record, /do not apply a remedy ... while the choice awaits an answer/i,
/approves no implementation/i; AUQ completeness and kind-note strings kept
exact (plan-tune reads them).
- 'An Investigate/Defer option must bound the investigation', 'Apply the
preamble's Completeness scores or kind-note accordingly': judge only (gate).
- Research/outside-voice sentences ('If Aside is unavailable, use host
WebSearch', 'Record the actual coverage...'): unavailable research is
recorded, not hidden -> kept free: exact 'Search unavailable' output string,
'Outside voice: recorded provider, completed / unavailable / disabled /
skipped (reason)' field, /only completed reviews enter cross-model tension/i,
TODO < Approval readiness < Required outputs order.
- Recovery sentences: kept as /i meaning checks (ExitPlanMode/telemetry wait,
do not resend, stop before dependent output, no chat-only success, skip
forbidden writes); 'If no recovery is specified or it fails' full sentence
-> kept free: 'follow **Blocked outcome**' routing; OUTCOME=error kept.
- Late-change sentences ('Refresh affected tests, tasks, dependencies and
parallelization', 'Unchanged saved outputs may reuse their successful Review
Log'): rerun affected outputs before navigation -> kept free: order of
return/Decision procedure/Approval readiness/Required outputs/navigation and
the stale-evidence Blocked outcome route; judge.
- Final sequence anchor 'Run Learning hooks, including gated Brain Calibration
Write-Back; then return to the entrypoint's Section self-check' -> kept
free: 'Section self-check' as the last ordered anchor.
- Coverage-audit sentences ('For each existing or proposed component in the
selected target', 'the selected target. For each existing or proposed
feature', 'dedicated tool call before drawing the diagram' (s06 L9
choreography)): plan diagrams cover proposed paths, ship covers changed code
-> kept free: /existing or proposed (component|function\/method)/,
/future paths remain proposals/i, the four 'Every ...' obligations,
/no skipping regression coverage/i, ship has no 'existing or proposed';
gate E2E plan-eng-coverage-audit.
- Brain write-back ordered prose list ('reserved default-off gate', 'this
runtime does not set it', 'not an instruction to write now', t02 #9 / W7a
s04 F15): no gbrain write without the default-off gate and trust policy ->
kept free: 'Skip unless `BRAIN_CALIBRATION_WRITEBACK` is set' precedes both
write tools, eng prefix says /skip this section/i and /do not enable it or
infer permission/i, trust-policy key, /if unknown, skip/i, no env
interpolation, other skills carry no eng-only skip prefix.
Mutation pass (scratch worktree; plan-eng-review templates and
scripts/resolvers/gbrain.ts): 21/21 as expected; 14 reorder/marker/stop/
approval/consent/numeric mutations go red, 7 rewordings of dropped sentences
and lowered-register STOP lines stay green. Log:
.context/mutation-eng-review-routing.md.
* test: fold the harness-freeze fixture tests into the tests that own their helpers (test-of-test ratchet)
Seven new files imported only test code; their stored known-good/known-bad
rows move unchanged into the owning tests:
- auq-matrix-format-gate -> auq-native-capture.test.ts
- browse-judge-floors -> workflow-judge-cache.test.ts
- ceo-split-floor -> plan-review-cases.test.ts (one block appended at the end)
- plan-mode-targeting -> plan-mode-evidence.test.ts
- shared-libs-pr-coverage-budget -> shared-libs-fixture.test.ts
- skill-body-narration -> skill-fixture.test.ts
- test-value-seam-evidence -> test-value-bar.test.ts
plan-format-kind-note and ship-triage-labels have no other owner, so each now
checks its grader against the production source it grades: the kind note
generateAskUserFormat prescribes, and the in-branch / pre-existing labels in
ship/sections/tests.md.tmpl. No BASELINE entries added.
* test(W7c): ship-plan-completion-invariants keeps tokens and gates, drops prose
W7c (t06 #9, #10, #11; #7/#8 already landed with W7a). One test file.
Removed assertion groups (invariant -> remaining protection):
- Execution-only extraction sentences ("implementation and test-creation
requirements", "retain its command, expected outcome and source verbatim",
"not EXTERNAL-STATE merely because ...", "Keep genuine external-state ...")
-> execution-only checks route to Step 8.1/9 outside counts and are never
DONE statically: kept free checks `Step 8.1/9`, outside implementation
counts, `never DONE from static inspection`, do-not-waive; gate E2E
ship-exploratory-plan-checks (gate).
- Plan-verification prep sentences (~17: "Clarify unknown outcomes", browser
vs functional items, "An API URL is not automatically a page", "Only browser
evidence needs screenshots", smoke-budget and command-limit sentences, risk
acceptance wording) -> plan checks are collected now, executed by the Step
9.2.1 owner before Fix-First, and never silently waived: kept free checks on
the heading list, `Collect now; execute in Step 9`, `Step 8.2`, `Handoff to
Step 9.2.1` order, `before Fix-First`, never silently waive, noninteractive
-> blocked, the `VERIFY_RESULT=` line and `## Verification Results`; gate
E2Es ship-exploratory-plan-checks, ship-exploratory-small-cli (gate).
- Parent-processing duplicates on the generated section (same pins as the
template test) -> kept on the template test; the generated copy keeps
section order + seven-field anchor; freshness via gen-skill-docs.
- "Check the task's terminal status", "recording the reason in the PR body and
Step 20 metrics" -> failed audits use the fallback: kept free LAST-line JSON
-> audit-failure fallback check and `(recommended/default)` marker.
- Step 16 evidence sentences ("Independently prove ...", "exact command, exit,
timestamp and log", "This recovery also applies ... stage 5") -> storage
failure is not stale input: kept free table-row structure, storage column
must not rerun `gstack-evidence run`, STALE/MISSING without proof, ledger
unavailable never FRESH; judge `ship/SKILL.md workflow` (gate lane).
- "Each commit must work independently" -> bisectable commits: kept
`bisectable commits`, `continue to Step 16`, no empty commit, history-
rewrite bans; judge `ship/SKILL.md workflow` (gate lane).
Loosened to case-insensitive meaning checks (kept): per-item confirmation,
blanket-confirm ban, classified DONE/NOT DONE by `[ -f` (MUST dropped), stop
publication on new/changed/unwaived test failure + `Run Steps 5–15`, waiver
reuse limited to verified pre-existing failures, fetch/push failure -> stop,
never rewrite history, only a successful push, stop live child before inline
audit, never race a late result, AskUserQuestion on second failure.
Kept unchanged: seven-field JSON contract parse, review-mode negatives,
CONTENT-SHAPE/validator tokens, Step 14 deferral routing, CHANGELOG negatives,
the executed push-idempotency shell test.
Cosmetic: MUST / mandatory / Do NOT capitals; "VAS-449" dropped from the
describe title.
Mutation log: .context/mutation-ship-plan-completion-invariants.md (19
mutations, 0 unexpected: 3 reorders, 4 markers, 7 safety red; 5 rewordings green).
* test(W7c): shrink eng-scope-entry-ap to structural and meaning checks
W7c conversion (t05 F1-F4; W7a s04 F15/F18 pins). Pin lines 97 -> 69
(toContain|toMatch|indexOf|includes|ordered). The AskUserQuestion-format
byte-equivalence test (t05 Kept) is unchanged. eng scope-gate E2Es are
periodic (plan-eng-review-plan-mode), so every scope-gate invariant keeps a
free check; duplicates of eng-review-routing cite that file.
Removed assertion groups (invariant -> remaining protection):
- 'Reply with A, B, or C. STOP and wait' order anchors (t05 F1): menu before
bootstrap -> kept free: the consumed prefix 'Reply with A, B, or C'
(pty/classify.ts, pty/judge.ts) before {{PREAMBLE}}, {{GBRAIN_CONTEXT_LOAD}}
and the first ```bash; new presence checks for the announcement and menu
after '## Scope gate' (an indexOf of -1 used to pass the order check).
- Preamble 'starting from the Scope gate...' exact pin: kept as /i meaning.
- Scope entry sentences ('Before discovery tools or preamble, check provided
messages...', 'Do not probe for session state', 'When no exception above
applied:', 'Announce an auto-selected plan in one line...', 'A fresh
announcement made before skill loading...', 'Step 0 below still verifies or
sends the public auto-selection line...') (t05 F4): target bound before
bootstrap and announced -> kept free: announcement/menu-before-bootstrap
order checks above, /first tool call = AskUserQuestion/i, /clarify
ambiguous, conflicting, quoted or stale targets/i; periodic E2E
plan-eng-review-plan-mode.
- Startup step sentences ('3. Check web-research readiness at ...', '4. Run
**Design Doc Check**, then ...', 'Keep the reviewed target fixed', 'Continue
at ...'): startup order -> kept free: ordered startup anchors (now including
Prerequisite Skill Offer), '**Engineering review → Step 0**' pointer, the
stale-path ban; 'Keep the reviewed target fixed' stays pinned (/i) in
eng-review-routing.test.ts.
- Entry anchors 'Before Step 0, require resolved scope' and '**Complexity
gate:** while a Scope Challenge complexity question' -> shortened anchors
'require resolved scope' and '**Complexity gate:**', same order check.
- Plan-selection exception sentences and 'If the tool is unavailable,
disallowed (`--disallowedTools`) ... send the menu as plain prose and STOP'
(t05 F4): pasted text is not a mode signal; ambiguity and named targets ask;
unavailable tool stops -> kept as /i meaning checks for each (safety and
consent lines); A/B/C menu body kept exact (consumed by the PTY classifier).
- 'Reply with A, B, or C. STOP and wait for the answer.' exact: kept as the
exact prefix plus /reply with A, B, or C\. stop and wait for the answer/i.
- Judge-bundle sentences ('format/transport rules apply throughout', 'Follow
the blocks below in order after startup', 'It may be the selected plan or a
separate file', 'Save permitted auxiliary artifacts under the write policy'):
ordering and write permission -> kept free: stage order, write-policy table
rows exact, /permission for one path authorizes no other/i, /defer ...
telemetry and plan status footer to finish/i, forbidden-log Blocked outcome
route (/i), section Read paths, log exit-code tails.
- Complexity-path duplicates of eng-review-routing (t05 F2: 'Complete these
checks before...', 'After verification, apply only accepted scope changes',
'Run C whether B...', 'Always ask the structure question...', post-answer
summary/Read-back sentences, 'A failed save or Read blocks advancement',
'Findings and scope answers approve no remedies', 'Continue to Section 1
only when...', 'Send one question object for one choice', 'Compare every
native field...', 'Repair any difference and repeat the complete Read
before asking', 'Read the selected saved label...', 'Check the save result,
then Read the entire resolution block', '**STOP until the actual answer
arrives.**', the Paused question / Blocked outcome lines): the same
invariants -> kept free in eng-review-routing.test.ts (order of
Send/AskUserQuestion/Record, /stop until the actual answer arrives/i, failed
save blocks, approve-no-remedies, Paused/Blocked recovery checks) and
auq-format-always-loaded.test.ts (one question object per choice); here:
A/B/C stage order and the 8-file / 2-class thresholds (/i, no caps STOP
literal), Recovery routing pointer for unreadable records. The Read-back
choreography is the dated text s04 F18 flags; judge
'plan-eng-review/SKILL.md sections' (gate) grades the procedure.
- Calibration note exact sentences and the one-line formatting pin (t05 F3,
W7a s04 F15): no shipped source enables BRAIN_CALIBRATION_WRITEBACK ->
kept free: {{BRAIN_WRITE_BACK}} once after {{LEARNINGS_LOG}}, /never set it
yourself/i, and a new code check that no file in bin/, lib/ or scripts/
assigns or exports BRAIN_CALIBRATION_WRITEBACK (only the gbrain resolver
names it).
Mutation pass (scratch worktree; plan-eng-review templates, generated
plan-eng-review skill, bin/gstack-config): 21/21 as expected; 14
reorder/marker/table/stop/safety/consent/code/numeric mutations go red, 7
rewordings and lowered-register lines stay green. Log:
.context/mutation-eng-scope-entry-ap.md.
* test: type the model-routing table; point skillify comment at skill-fixture.test.ts
* test(W7c): skill-validation matches three capital-letter pins on meaning
W7c (t08 F2, F4, F5). One test file; only these three pins change.
Removed assertion groups: none. Loosened, invariant kept as a free
case-insensitive meaning check:
- `MUTUALLY EXCLUSIVE` (codex skeleton) -> one codex mode per invocation:
`/mutually exclusive/i` (t08 F4).
- `skip the Codex passes ONLY` (review adversarial) -> `codex_reviews=disabled`
skips only the Codex passes, Claude adversarial still runs:
`/disabled[^\n]*skip the Codex passes only/i` (t08 F5).
- `If any in-branch failures remain unfixed, **STOP**. Do not proceed` (ship
tests section) -> unfixed in-branch failures block advancement:
`/in-branch failures remain unfixed ... stop ... do not proceed/i` (t08 F2).
Also covered by gate E2E ship-triage (gate).
Cosmetic pins dropped: capitals and bold only.
Mutation log: .context/mutation-skill-validation.md (6 mutations, 0
unexpected: 3 rule deletions red; 3 lower-case rewordings green).
* test(W7c): drop coverage-audit checklist tool-call prose pins
W7c conversion (t05 F11; unblocks s06 L9). Pin lines 17 -> 14.
Removed assertion group (invariant -> remaining protection):
- review/specialists/testing.md prose pins 'If the caller explicitly asks
for an ASCII coverage diagram', 'dedicated tool call' and /Read diffs,
package files, configs, or other context in separate tool calls\./ (tool-use
choreography): the agent reads the named source and test files before
drawing the diagram -> kept free: the same test still extracts the
checklist's example diagram and requires coverageAuditVerdict to accept it
(sourceRead/testsRead/diagram/passed), plus the kept [OK]/[GAP] row and
legend pins; gate-tier E2Es review-coverage-audit, plan-eng-coverage-audit
and ship-coverage-audit (E2E_TIERS: gate) grade the reads from transcript
evidence with the same oracle.
Mutation pass (scratch worktree, review/specialists/testing.md): 6/6 as
expected; reordered diagram rows, deleted legend, deleted [OK] marker and a
removed text fence go red; two rewordings of the dropped tool-call sentences
stay green. Log: .context/mutation-coverage-audit.md.
* test(W7c): loosen auq-format-always-loaded per-skill rule pins
W7c conversion (t01 F10-F12; W7c list item auq-format-always-loaded:61-72).
The MANDATORY format list (machine-read Recommendation:/(recommended) and the
AUQ format labels, W7 keep list) is unchanged.
Removed assertion groups (invariant -> remaining protection):
- plan-ceo-review /Do NOT batch/i (t01 F10): one decision per question ->
kept free: /One decision unit = one AskUserQuestion call/i in the same rule
list.
- plan-eng-review full-sentence regexes 'Send `AskUserQuestion({ questions:
[currentDecision] })` only after the pending-record checkpoint passes. Send
one question object for one choice; other IDs wait' and 'If you discover
another independent choice, separate it and rebuild this comparison before
saving or sending the question' (t01 F11): one question object per
independent choice survives the carve -> kept free: /one\s+question object
for one choice/i and /independently selectable changes separate IDs/i; the
checkpoint-before-send order stays pinned in eng-review-routing.test.ts.
- codex /Synthesis recommendation \(REQUIRED\)/i (t01 F12, caps marker):
codex keeps its synthesis rule always loaded -> kept free: /Synthesis
recommendation/i and the machine format /Recommendation\s*:\s*<action>\s*
because/i.
Mutation pass (scratch worktree, generated SKILL.md and sections): 8/8 as
expected; removing Recommendation: <action> because, the synthesis rule, or
the CEO/eng/design decision-unit rules goes red; dropping (REQUIRED) and
rewording the two dropped eng sentences stays green. Log:
.context/mutation-auq-format-always-loaded.md.
* test(W7c): qa-probe-gates keeps loop order, bounds and the CI-proven Read gate
W7c (t07 F3, F4, F5). One test file. This file stays the single owner of
shared QA loop ordering (qa-functional-prompt defers to it).
Removed assertion groups (invariant -> remaining protection):
- `> **STOP.** Before any probe ...` prefix pin (t07 F3, retire) -> parent
QA loads methods before any probe: kept free meaning check on "before any
probe, including plan checks, complete the ordered scope/method Reads" and
its order before the required-checks step.
- Preparation prose in the stage list ("in full and select the surfaces",
"2. Read the selected surface methods below in full", "Write a **charter**",
"Start once before baseline:") -> stage order scope -> methods -> charter ->
deadline start -> loop: kept as structural anchors (section Reads,
surface labels, `## 1. Charter and preflight`, `bun G start D SECONDS`,
step 1 title); added a reorder control moving a method Read after the charter.
- Bounds/layout sentences ("an unlimited mode uses the caller's bound",
"announce finite command timeouts", "EARLIER_UTC = caller's absolute
deadline, if set", "in the probe directory, beside its deadline if
bounded", duplicate SECONDS pins in the mode-bounds test, "charters as
Markdown in the report", the full "observed: its exact decoded child JSON
..." sentence) -> numeric bounds and field names are kept free checks;
the observed-field shape is enforced by the native checkpoint validator
(qa-functional-prompt validator tests, free) and gate E2Es
qa-functional-cli-report, qa-functional-webhook-report (gate).
- Publication prose ("No drafts/placeholders", "and replay limits in the
report", "An absolute state path is not itself a secret", "If capture is
incomplete, report that limit instead of reconstructing it") -> notes are
immutable and unsafe values withheld: kept free checks on "corrections
cannot repair published notes", never invent a substitute, withhold
secrets/unsafe values and stop the chain; validator-backed in
qa-functional-prompt; gate E2Es review-exploratory-small-cli,
ship-exploratory-small-cli (gate).
- Loop-step prose ("Guard if bounded; await completion", "Run that exact
probe; G enforces ..." duplicate, "then minimize via those gates", "Expiry
leaves confirmation/minimization incomplete", "for each affected
revalidation", "If bounded, run", "Reuse Step 4's ... completed Reads",
"Finish missing methods before charters") -> loop has 5 steps, guarded by
G, replays via steps 2–3, revalidation returns to step 2: kept free checks on
step count, `G enforces the deadline`, `bun G status D`, `via steps 2–3`,
`return to step 2`, refusals as not-run, do not repeat completed Reads.
- CI-fixture clause "description, section index or remembered method is not
a completed instruction Read" -> covered by the kept CI-proven clauses
(`## 0. Preparation gate`, await each Read, complete missing Reads before
Charter) and gate E2E ship-exploratory-small-cli (gate; the CI case).
Kept (meaning checks, case-insensitive where prose): CI-proven "await their
results before the first probe" rule (comment + mutation kept), Reads in
order, no repeated Reads, SECONDS=30/900, shorter mode/caller limit, no D
without a total limit, finite timeouts, REPORT_DIR layout and caller paths
win, plan checks after smoke with no smoke guard and no clock reset, caller
cap, not-run on expiry, await clock results, never reset D/bypass G,
invalid/missing D stops probes, expiry -> write the report, Step 4
read-only, isolation/permission preflight before setup, field names,
QA_DEADLINE receipts, all semantic-flip mutation controls, method/report
resource paths, helper commands.
Cosmetic pins dropped: STOP / AND capitals, bold STOP prefix.
Mutation log: .context/mutation-qa-probe-gates.md (22 mutations, 0
unexpected: 3 reorders, 4 markers, 8 safety red; 7 rewordings green).
* test(W7c): shrink autoplan-phase-order to structural and meaning checks
W7c conversion (t06 #13; W7c list item 'Eng runs LAST, always', 'ONLY/FINAL').
Pin lines 203 -> 166. autoplan has only periodic E2Es (autoplan-dual-voice,
autoplan-journal-drift: E2E_TIERS periodic), so every invariant keeps one
free check: sentence pins collapse to short /i meaning checks; commands, CLI
signatures, placeholders, tool parameters, report fields, packet outputs
(code-generated) and stale-string bans stay exact.
Removed assertion groups (invariant -> remaining protection):
- 'Eng runs LAST, always' (caps sentence): Eng is the terminal phase -> kept
free: /CEO → Design.*→ DX.*→ Eng/s, heading order 1 < 2 < 2.5 < 3 and the
'CEO → Design → Eng → DX' ban.
- Final-gate sentences ('...Use Decision Classification; ask once at Final
Approval Gate, never mid-run', 'Queue clearly-wrong/challenged premises',
'The user decides there; never stop mid-pipeline'): premises queue for the
single final gate -> kept free: /never auto-decide user challenges/i, /ask
once at final approval gate, never mid-run/i, 'as User Challenges for Phase
4', /never stop mid-pipeline/i, the two old-gate bans.
- Generated setup sentences ('Resolve this phase', 'Run all applicable skills
and lazy sections fully', 'log successful ranges/total to EOF'): phases load
at their own boundary -> kept free: /never prefetch future phases/i,
/missing skill: report phase and setup repair/i, per-phase methodology CLI
and Read-before-create order, no in-full-now Reads.
- Intake load sentence ("Resolve this phase's source to absolute
`<REVIEW_SKILL>`; load via its checkpoint") -> kept free: `<REVIEW_SKILL>`
token and /never prefetch future phases/i; section load keeps
`readRanges`/EOF/skip-listed checks.
- Native dispatch sentences ("ONLY/FINAL tool call", 'Keep native Reads
enabled', 'all criteria + plan', 'if its schema exposes it', 'Claude Code:
end response immediately', 'Other hosts await that ID', "Then outside →
this phase's review ONLY", 'No inline substitute; apply failure policy'):
the native review blocks and completes before outside review -> kept free:
native < outside order, run_in_background: false, isAsync: true, /final
tool call/i, /end response immediately/i, /no further tool calls\/review
until/i, `nativePromptPath` to EOF, INPUT-hash match, /retry invalid input
once/i, /no inline substitute/i, 720000ms timeout, provenance placeholder,
N/A and CONFIRMED status tokens, /primary cannot replace/i.
- Sequential Execution sentences (Keep ONE phase active, compaction reload,
'reconcile saved artifacts and sent conversation messages separately',
'Publication is a separate operation', 'an earlier Read is not this close',
"Then continue to the next phase's tool calls in the same turn", 'record
verbatim into fenced history', 'retaining its original source SHA',
'compaction resumes the existing invocation', the full LOG/amend-input
sentence): one phase at a time, publish before advancing -> kept free:
/keep one phase active/i, /never draft future-phase reviews/i, compaction
reload (/i), two ordered chains (load < snapshot < dispatch < consume <
remaining sections; phase-close < parent message < driver advance < after
Eng), skip/resume/step 6 (Publish) routes, open-gate, uptake, pending,
never-skip and raw-transcript lines (/i), rerun fields
`baselineEdits.record`/`sourceSha256`, `amend-input`.
- Amendment-binding sentences ('load the shared close steps afresh, even if
read earlier', 'Keep this checkpoint for this invocation; review exports do
not replace it') -> kept free: checkpoint field per phase, `methodologyPath`,
/afresh/i, /review exports do not replace it/i.
- Shared-close sentences (~20: completion prerequisites, readRanges Read
choreography, 'A Read of only the edited tail does not satisfy...', 'If a
result is truncated...', 'If a Read fails...', 'regenerate the packet...',
'Retention checks prove bytes; counts, hashes...', 'Review history stays in
Review record', 'Taste remains provisional', 'Apply the phase's failure
policy...'): read the whole packet, verify, then publish -> kept free: the
seven-step order, terminal-results/INPUT/pending/no-credit/User
Challenges/None/fixed-checkpoint lines (/i), prepare-close CLI,
`readRanges` with exact `offset` and `limit`, /through EOF/i, no advance
without a result, step-3 return, verification compare/recheck/keep-open,
stale ban, generated copy contains the template.
- Publication sentences ('After sending the actual parent report', 'using
actual findings and voice statuses', "The sent conversation message is step
6's output", 'This message is the next operation...', the duplicated driver
pins) -> kept free: publish < report < return order, /send the filled
report ... visible parent assistant text/i, /before any next-phase tool
call/i, both-voices rule, same-turn/driver-alone/no-continue-wait/skip
lines (/i), Bash-is-not-publication (/i), stale ban.
- Report-format sentences ("the actual host's reviewer names", 'Include the DX
metrics line only when ...', "Resolve `report.next` using the driver's
applicable scope/skip rules") -> kept free: every report field line,
`report.includeDxMetrics`, `report.next`, and the code-generated packet
checks.
- Intake/DX-scope/call-site sentences ('without losing requirements', 'Use
returned paths/`scope`', 'false and neither semantic trigger applies',
'checks exact retention', the 'for both voices' tail) -> kept free: ACTIVE_PLAN
field, save-to-ACTIVE_PLAN, conversation approval, backup-exactly, stop on
helper error, `## Review record` (all /i where prose), init/scope/create/
prepare-close CLIs, `dxRequired`, 2+ threshold, enabling flags, no-negation
rule, snapshot/verbatim dispatch/outside-inline lines, stale path bans.
Mutation pass (scratch worktree; autoplan template, sections and generated
phase-close): 23/23 as expected; 16 reorder/old-order/tool-param/placeholder/
numeric/consent/stop/safety mutations go red, 7 rewordings and lowered caps
(ONLY/FINAL, Keep ONE phase) stay green. Log:
.context/mutation-autoplan-phase-order.md.
* test(W7c): qa-functional-prompt keeps the verifier contract, drops driver prose
W7c (t07 F6, F7, F8). One test file. The eval prompt itself is unchanged.
Removed assertion groups (invariant -> remaining protection):
- ~40 driver-prompt sentence pins in the disclosure, artifact and fix-budget
tests ("All four reads are required before probing ...", "command is the
exact full outer capture invocation, including ...", "different later
command", "not the required same-command replay", "Only the helper writes
observed fields", "Link the evidence and checkpoint files ...", "Evidence
rows contain ONLY complete JSON ...", "aim under 400 words", "one causal
sentence per checkpoint hypothesis ...", "a reproduced in-tier defect
requires ...", duplicates of "retain its headings ..." etc.) -> the run
produces verifiable evidence and a linked report: free validator tests in
this file (qaFunctionalVerdict replay, validateQACheckpoints on CI, R29 and
R85 captures) and gate E2Es qa-functional-cli-report,
qa-functional-webhook-report, qa-functional-cli-fix,
qa-functional-webhook-fix (gate). Kept the machine-read contract: read
list, `"command":"<exact full outer capture invocation>"`,
`annotations.learning`, the three copied field names, the >20-character
limit, evidence.json before report.md.
- `maxTurns: 40` / `completionReserveMs: timeout / 4` exact source pins ->
replaced by a `maxTurns >= 40` floor and presence of `completionReserveMs`.
- Fix-stage provenance sentences ("most recent completed native probe",
"Tests, source edits and clock reads ...", "write no checkpoint when there
is no next probe") -> test output never becomes `observed`: kept one free
meaning check plus the R29 validator test (free).
- Duplicate loop-order checks on qa-only/sections/exploratory.md and grep of
`scripts/resolvers/qa.ts` source (t07 F6, F8) -> loop ordering has one
owner, qa-probe-gates (free, every host). Here the source grep becomes a
check on `generateQAExploratory` output (one probe per native command,
never batch probes), and the caller test keeps resource paths, field names
and structural stage anchors.
Loosened to case-insensitive meaning checks (kept): never synthesize JSON,
never shorten native JSON, existing tests read-only, freeze tests after red,
repair scope `src/<family>.ts`, green suite does not replace native probes,
report incomplete honestly, report-only proposes test stubs, completion
reserve does not end coverage, all eight scenarios required, never backfill
notes, owned probe directory, only complete emitted JSON, wrapper/entrypoint
boundary lines.
Kept unchanged: paid-case selection, CI/R29/R85 validator replays, actor
boundary and command allowlist, fix-stage order, verification probe list,
scenario lists.
Cosmetic pins dropped: ONLY / NOT / Never capitals.
Mutation log: .context/mutation-qa-functional-prompt.md (19 mutations, 0
unexpected: 3 reorders, 4 markers, 7 safety red; 5 rewordings green).
* test(W7c): match plan-scope-selection scope-gate stop lines case-insensitively
W7c conversion, scope-gate caps pins only (t06 #14). The scope classifiers
(nativeSeededPlanSelection, isScopeGateAutoSelectVisible,
isScopeGateQuestionVisible), the consumed announcement and A/B/C menu bytes,
and every non-caps assertion are unchanged.
Removed assertion groups (invariant -> remaining protection):
- 'Reply with A, B, or C. STOP and wait' order anchors (design, template and
every host expansion): menu precedes bootstrap -> kept free: the consumed
prefix 'Reply with A, B, or C' (pty/classify.ts, pty/judge.ts), now also
required to exist after '## Scope gate' so a missing menu cannot pass the
order check with indexOf -1.
- 'call AskUserQuestion FIRST and wait.', 'does NOT count as the mode signal',
'If the user explicitly named a DIFFERENT target' (design and eng), 'STOP
and wait for the answer — only after the user picks', 'STOP and wait for the
answer' (toContain and indexOf before the first ```bash on every host):
ask first, pasted text is not a mode signal, explicit targets win, stop
until answered -> kept free: the same rules as case-insensitive meaning
checks, including the per-host ordering (search /stop and wait for the
answer/i before the first ```bash, with a presence check).
Mutation pass (scratch worktree; plan-design-review and plan-eng-review
templates): 12/12 as expected; gate-after-preamble, removed design menu
prefix, removed stop lines (design, eng), removed mode-signal, ask-first and
explicit-target rules go red; lowering STOP/FIRST/NOT/DIFFERENT stays green.
The eng menu prefix is not pinned here (eng-scope-entry-ap.test.ts M1 covers
it). Log: .context/mutation-plan-scope-selection.md.
* test(W7c): match document-release spawned-dispatch contract on meaning
W7c conversion (t04 F3; W7a run-in-background-guidance.test.ts:472,474, named
by the Eng W7c gate as anti-injection and VERSION invariants that stay as
case-insensitive meaning checks). The run_in_background flag pins, structural
scanner and inverted-phrasing ban are unchanged.
Removed assertion groups (invariant -> remaining protection):
- 'NEVER trigger it on their own' (caps literal): prompt/file/tool claims do
not enable spawned mode (anti-injection) -> kept free: /claims never
trigger it on their own/i in the generated skill and template.
- 'The NEVER-do invariants below do' (short form of a wrapped sentence that
names the caps headings s10 rewrites): spawned auto-pick still never
rewrites CHANGELOG or changes VERSION -> kept free: /skip any
recommendation that rewrites CHANGELOG or changes VERSION/i
(whitespace-normalized) in both files; gate E2E document-release.
- 'A spawned run must never change VERSION': same rule, now /i.
Mutation pass (scratch worktree; document-release skill, template and
release-body section, ship/sections/documentation.md): 7/7 as expected;
deleting the anti-injection clause, the CHANGELOG/VERSION carve-out, the
spawned VERSION line, the run_in_background flag or the dispatch heading
goes red; lowering NEVER and rewording 'NEVER-do invariants' stays green.
Log: .context/mutation-run-in-background-guidance.md.
* test(W7c): match retro clock rule case-insensitively, drop proximity regex
W7c conversion (t02 #11, #12).
Removed assertion groups (invariant -> remaining protection):
- /NEVER from `date`/ (caps): today comes from the session reminder, not the
system clock -> kept free: /session reminder/ and /never from `date`/i.
- /(?:disclosure[\s\S]{0,200}narrative|narrative[\s\S]{0,200}disclosure)/
(200-char word-proximity window): offline/local-only runs disclose that the
window is not freshness-verified -> kept free: the exact disclosure string
'offline run, window not freshness-verified' (it reaches the retro output),
RETRO_REF disclosure and the GUARD_* script checks; gate judge
'retro/SKILL.md instructions'.
Mutation pass (scratch worktree, retro/SKILL.md.tmpl): 4/4 as expected;
allowing `date` or deleting the disclosure string goes red; lowering NEVER
and rewording the narrative clause stays green. Log:
.context/mutation-regression-1624-retro-stale-base.md.
* test(W7c): match redact scan-at-sink exact-bytes rule case-insensitively
W7c conversion (t02 #13). Credential-handling checks (mktemp, --from-file,
exit codes 3/2/0, visibility lookups, no HIGH skip flag) are unchanged.
Removed assertion group (invariant -> remaining protection):
- /EXACT bytes/ (caps): scan the exact bytes that will be sent -> kept free:
/exact bytes/i in the same test, next to the mktemp and --from-file checks.
Mutation pass (scratch worktree, scripts/resolvers/redact-doc.ts): 3/3 as
expected; scanning a summary instead of the exact bytes or dropping the
Exit 3 (HIGH) branch goes red; lowering EXACT stays green. Log:
.context/mutation-redact-doc-resolver.md.
* test(W7c): check the codex boundary on meaning and its path list
W7c conversion (t02 #14, W7a host-config.test.ts:565). The boundaryInstruction
bytes sent to Codex are unchanged; only the test loosens.
Removed assertion group (invariant -> remaining protection):
- toContain('Do NOT read') (caps fragment): Codex must not read or execute
Claude skill definitions -> kept free: /do not read or execute any files
under/i plus each excluded path ('~/.claude/', '~/.agents/',
'.claude/skills/', 'agents/'), which is the actual contract.
Mutation pass (scratch worktree, hosts/codex.ts): 3/3 as expected; replacing
the prohibition or dropping an excluded path goes red; lowering IMPORTANT/NOT
stays green. Log: .context/mutation-host-config.md.
* test(W7c): shrink skill-ceo-section-ordering to structural and meaning checks
W7c conversion of test/skill-ceo-section-ordering.test.ts: t06 #1-#6, t05 F5
(cross-reference: the remaining bold-STOP pins), t13 F3 / W7.md CRITICAL RULE
anchor. #1 (ASK_QUESTIONS_HEADING anchor) was already on the base branch; #5
now slices the blocked chat return to the next "\n## " heading, so removing or
renaming `## Review Log` cannot widen or break the window.
Pins (grep -cE 'toContain|toMatch|indexOf|includes'): 647 -> 388 lines;
toContain( 535 -> 136; expect() calls per run 1657 -> 1399. The carve block,
section-capture fixtures and validateCeoReviewCompletion tests are unchanged
(t06 keep list) except two exact-duplicate 0D pins.
CEO invariants count as periodic-only (plan-ceo-review-benefits retires), so
every invariant below keeps a free structural or case-insensitive check.
Gate-tier judge that also grades this prose: 'plan-ceo-review/SKILL.md modes'.
New helpers: ordered() (exact labels/headings/placeholders/table cells as
strings, meaning anchors as /i RegExp) and between() (fails loudly on a missing
end anchor instead of silently running to EOF).
Removed assertion groups (invariant -> remaining protection):
- t06 #2 / t05 F5: '**STOP for the actual answer, even for a lone option.**'
(x6 exact pins + 3 split/indexOf anchors) and '**STOP for the answer**'
(x3): wait for the actual answer before recording -> kept free: one
/stop for the actual answer/i per slice (template 0D in dispatch order
between the ask-one-row check and the answer save; every generated host
carrier between **4. Ask and **Post-answer checkpoint:**), /stop for the
answer/i in 0E step 3 (template and generated), plus the Pre-question <
4. Ask < Post-answer checkpoint order and the 2-checkpoint count. Dispatch
slices now end at **Post-answer checkpoint:**.
- t06 #4 'Recommendation is ALWAYS present': recommendation label contract ->
kept free: '(recommended)' and 'AUTO_DECIDE depends on it' in generated
SKILL.md; exactly-one-(recommended) rule in 0D's currentDecision table.
- t06 #6 and other bans of retired phrasings ('If the section has findings,
you MUST call AskUserQuestion', 'Otherwise, use AskUserQuestion for each
finding', 'After each section, pause and wait for feedback', 'Save pending
rows before comparisons', 'using the Step 0E mode-handoff format', 'Save a
chat-only plan', 'quality bonus, not a gate', 'If no file is in scope, skip
this section', 'always write', 'For every question', 'for architecture
choices', 'These approaches differ in coverage (...)', 'Finish with
ExitPlanMode', 'Before summaries, review logs or next-step menus...', "put the
complete comparison in the question's brief", 'Use this exact route:',
'citing its ID', /\blog (?:with|that ID)\b/): same invariants -> kept free:
the positive checks (one **Decision gate.** per section wired to Analyze →
Resolve → Apply; pending-rows-before-comparisons order; **Otherwise:** in
the 0H order; report not persisted + Read-back gate; write-when-permitted
incl. zero tasks; Outcome routing count 1; record-provenance-after-handoff
order; literal dispatch-object comparison). 'Do NOT proceed ... 0C-bis'
becomes not.toContain('0C-bis') on 0D. Short stale-token bans stay
(<approved 0D approach>, reference | commitment | current value, ___
methods, retry once, telemetry has logged, NUMBER + LETTER / "3A", the
Forbidden-storage regex, removed menu option, unbound placeholders).
- t06 #3, magic counts: literal 11 sections / toHaveLength(7) closing steps ->
kept free: section numbers 1..N derived from '## Review Sections (N
sections' and checked against the ### Section headings; closing-step count
derived from the 7 route labels.
- Decision cycle / storage / comparison sentences (Start at step 1, run steps
2–4 only when needed, mode-change pointer, storage-policy save sentence,
Copy the grid, Validate every field, Show unchanged/shared/pending repeated
x4, other rows stay fixed or pending x5, never prewrite x3, Build one
currentDecision x3, Ask one row per call x6, Save the answer reference x5,
Keep independent changes separate x4): full decision protocol in order ->
kept free: label order 1. Check < 2. Record < 3. Compare < grid <
Pre-question < Read-back < 4. Ask < Post-answer (template + every host),
one /i check each for pending-rows-before-comparisons, save-before-ask,
amend only authorized work, failed save stops, decline-all stop, show
unchanged/shared/pending, independent changes separate.
Admin/plan routes also kept in ceo-workflow-clarity.test.ts.
- Handoff/provenance sentences (After selection, send brief chat, Keep rows
separate, logging-row repeats, Section 1 current-scope sentences): mode
handoff applies before questions and logs provenance after -> kept free:
/before tools or further questions/i, handoff < Other selections <
provenance < selecting-a-mode order, the two exact output-format lines with
<rows or none> / <Application and rationale>, auto_decided true/false
fields, 'No new approach decision was needed' (quoted chat output). Section
1 dispositions -> ceo-workflow-clarity's Section 1 test.
- Scope/mode-selection sentences (REDUCTION, HOLD and SELECTIVE's HOLD
checks; For both expansion modes...; Skip/Defer destination repeats;
present both inputs for final approval; 0E counts changed files; Count all
deliverables; >15/greenfield repeats; SELECTIVE balances...): scope
decisions per item with exact destinations -> kept free: prescribed menu
label orders (A Add/B Defer/C Skip; A Defer/B Keep; TODO A/B/C; depth
A/B), Defer -> TODOS.md + NOT in scope, Skip -> no TODO, /ask separately/i,
/all four 0D steps for each unanswered addition or deferral/i, routing rows
and >15 / 8-file / 2-class numbers (once each; ceo-workflow-clarity keeps
the reuse-count checks), carve block's document-approval order.
- TODO menu (per plan-review-calibration t04 F11 retirement, which names this
file as the remaining free protection): kept free in the TODOS.md-updates
slice: /only unanswered TODO proposals reach this menu/i, /do not ask again
about an item already deferred, skipped or kept/i, /one per question/i.
- Resolver prose (spec review, plan report, tasks emit, exit gate, codex plan
review, brain cache refresh, approval readiness, design clause): each
sentence pin -> one short /i check per invariant on every host (five
dimensions, at most three launches, required save stops, destination-file
Read/Write order, zero tasks, unknown files JSONL format, ExitPlanMode only
after checks, forbidden history, outcome routing and skip routes, native
fallback never outside coverage); exact strings kept only for machine/output
tokens (NO UNRESOLVED DECISIONS, '- + N unresolved from prior reviews',
STATUS/SOURCE/OUTSIDE_STATUS assignments, CROSS-MODEL, OUTSIDE COVERAGE,
'N findings; R resolved; U unresolved', quoted report phrases, Gate outcome:
Blocked, Read-back gate). Dropped: 'If both counts are zero', 'For
**Outside Review**, use ...', 'Enter after either an external reviewer...',
'identify the causal mechanism', 'A credible material risk...', "check the
assembled set's capacity", 'including findings that needed only factual
correction', 'show unsaved fields and errors', Strategy-only tasks sentence,
verified-report-can-pass sentence -> neighbouring kept /i checks in the
same slice plus the gate-tier 'plan-ceo-review/SKILL.md modes' judge.
- Review-section prose (continuity requirement lists, test-review and outside
rule duplicates, strategy-only diagram sentences, Review facts repeats,
Stage 3 publishes it, Count a reopened choice once, Keep them before the
terminal report repeats, EXIT gate verifies..., no report or log needed
yet, then run telemetry as the last review operation): per-section gate and
ledger continuity -> kept free: Analyze < Resolve < Apply, Resolve paths
via check-scope < correct < record order, /resolve critical risks now/i,
/review only; do not change code/i, ledger header + status tokens, one /i
per approval/stop line; reopened-once lives in ceo-workflow-clarity;
terminal order kept as cache refresh < {{BRAIN_CACHE_REFRESH}} < Telemetry
< finished < Call ExitPlanMode < handoff.
- Duplicate pins removed outright (same string asserted 2-6 times across
tests, incl. carve-block 'Ask one row per call' and bold-STOP pins): each
keeps its single remaining check named above.
Cosmetic pins dropped without controls: bold/caps (**STOP**, ALWAYS, MUST),
double compactProse wrappers.
Mutation pass (scratch worktree; templates, generated SKILL.md and
review-dashboard resolver): 39/39 as expected; 28 kept-step/marker/
placeholder/safety/consent/numeric mutations go red (incl. the three TODO-menu
lines), 11 rewordings of dropped sentences, the CRITICAL RULE heading rename,
re-adding the retired #6 sentence, and lowered-register STOP lines stay green.
Log: .context/mutation-skill-ceo-section-ordering.md.
Runtime (Bun 1.4.0, 3 runs): before 2.05/1.84/1.77 s (median 1.84),
after 1.87/1.80/2.08 s (median 1.87); dominated by runGeneration and the
capture subprocess fixtures.
* test(W7c): shrink plan-review-cases to structural and meaning checks
W7c conversion (t03 F1-F4; t02 F4/F5 duplicate STOP/threshold pins). Pins
713 -> 208 lines by the brief's grep (toContain|toMatch|indexOf|includes);
prose assertions 592 -> 366 (toContain/toMatch calls plus expectAll regex
rules; expect() calls per run 3232 -> 2808). Tests 218 -> 215. t03 F1 (stale
~/.gstack/projects discovery path) was already fixed by W7a on the base; its
$GSTACK_STATE_ROOT check and the stale-path ban are kept.
Kept byte-exact or structural: marker-order arrays (### Prepare an unanswered
choice -> ### Send once and wait -> ### Record the answer, heading list
equality, Pending-record checkpoint, finish steps 1-6, entrypoint recovery
order, output placeholder order), the parsed write-policy table routes, the
generateAntiShortcutClause host matrix, every not.toContain(stale) ban,
numeric limits (8+ files / 2+ new classes, fewer than 8 / 2, under 5 words,
2-3 options, six-column / five-field / four-step), consumed tokens and field
names (`scope accepted as-is`, `scope reduced per recommendation`, FULL_REVIEW/
SCOPE_REDUCED mapping, **issues_found**, SOURCE=/OUTSIDE_STATUS=/STATUS=,
OUTCOME=, `BLOCKED`, `## GSTACK REVIEW REPORT`, `currentDecision`, D<N> and
ROW-ID formats, prescribed A-D menus), placeholders, commands/paths and all
picker behavior tests (unchanged). STOP/wait/approval/safety lines become
case-insensitive meaning checks located between their headings (e.g.
/stop until the actual answer arrives/i between Send once and Record the
answer; /do not replace the section in place/i plus /append the new report
at EOF/i). CEO invariants (periodic-only once plan-ceo-review-benefits
retires) each keep one free check. Eng/Design/DX invariants keep one free
check too; nothing below relies on a periodic-only E2E.
Removed assertion groups (invariant -> remaining protection):
- CEO route/narrow depth (:19) full sentences: admin answers approve nothing,
plan decisions run steps 1-4, settled choices go to 0E -> kept free:
Admin < Plan decision order, checkpoint order, five short /i checks, stale ban.
- Eng evidence paths/labels (:33) 'Prior Learnings configuration', path rule
sentences -> kept free: the three path field labels, /never invent paths/i,
`not available`, label tokens, stale ban.
- Eng first report save (:47) 9 requirement sentences -> 5 short /i
requirements with the same remove-one-must-throw self-control; parent-dir,
original-plan-copy and scope-record clauses dropped (initialization
invariant kept by target-header/preserve/no-findings/placement/terminal-
report checks; collision suffix kept in the write-policy test).
- Eng setup selectors (:79) and its duplicate in the big Eng ordering test:
selectors approve no remedy -> kept free: /these selections approve no
engineering remedy/i, B selectors, no pre-answer ledger, stale ban; the
'For the next choice' sentence -> kept free: transaction marker order.
- Eng performance (:94) exact bullet list -> kept free: four concern
keywords from the parsed bullets; Decision-procedure sentence -> kept free
in the four-section test.
- Eng complexity/MODE (:105) per-target counting, 'Count each once', 'Keep it
current', 'Continue to Section 1' sentences -> kept free: count-selected-
work, both thresholds, MODE tokens and mapping, scope-reduction meaning,
'approves no pending remedy', Continue-to-Section-1 in the C order array.
- Eng three-stage transaction (:125) sentence markers -> same 13-step order
array with short /i anchors and the remove-one-must-throw control.
- Eng Test Plan refresh, coverage trace (:150, :162) sentences -> kept free:
placeholder count/order, prepare step count, short /i checks; coverage audit
also graded by plan-eng-coverage-audit (gate).
- CEO fallback (:178) sentences -> kept free: host mode token matrix, status
field values, /final report is written later in Required Outputs/i, ban.
- Persistence per skill (:229) Eng duplicates ('If the required log is
forbidden', 'Neither supplies', FULL_REVIEW mapping, 'after successful
Read-back') -> kept free in the readiness, complexity and write-policy
tests; CEO 'retain its diagnostic' -> kept free: /do not claim that entry
was recorded/i and three other CEO log checks.
- Report resolver per host (:261) duplicate 'Do not choose another
destination', 'Review record and write policy', 'full review output',
explicit/named/active sentences -> kept free: target precedence order;
CEO writer's seven sentences -> kept free: six-anchor order ending at
'4. **Read-back gate:**' plus the blocked-return check.
- Generated carriers (:327) bold STOP pin -> /stop until the actual answer
arrives/i in the dispatch order array.
- Big Eng ordering test (:358) duplicates of the gate describe (one question
object, 8+/STOP, both-retain, setup, separate instrumentation, keep chosen
values, calibration<procedure<scope, Decision procedure < Architecture,
outside-voice consent lines) -> kept free once each (boundary list, queue
test, preparation order array now ending at ### 1. Architecture review);
'Use existing tests ... bounded probes', 'post-answer scope summary',
'Compare only the file/class arrangement', 'contracts and approved
security...', 'With no proposed cuts', 'Present numbered...' sentences ->
kept free: build-boundary order check, structure-question and
unapproved-fixes checks, C-step order array.
- Eng gate describe: exact-duplicate pins (same sentence 2-4x in one test or
repeated per section inside the 4-section loop) -> one copy kept; ledger
placeholder descriptions (Plan baseline/Runtime evidence/Actual answer/
Accepted scope text) -> kept free: parsed ledger field-name list, R1 header,
State values, A)/B) selectors, ledger position.
- Gate tests 'identifies commitments before comparing values', 'common new
defaults...', 'exact prior answers authorize...', 'finding evidence, stable
decision identity...' merged: every non-duplicate invariant (optional
depths, shared new value needs approval, unknown risks visible, drafts grant
nothing, reopened ID keeps D<N>, test scores not answer status) -> kept free
as one /i check in 'current contracts...' or 'all four section gates...';
their order checks -> kept free: prepare<identify<compare<save<send<record.
- Remove-old-occurrence, 'Otherwise leave the remedy pending', 'no question
or comparison grid is needed', 'never the saved-report gate', 'Compare the
label separately', 'When revising, replace the whole current payload',
'It may be the selected plan', 'Present each completely as not persisted',
'If none is permitted...', 'A forbidden write follows its row', 'Run
Learning hooks ...', 'Refresh affected tests...', 'If startup values...',
'instead of creating another approval row', 'Keep that remedy pending' ->
same invariant kept free by an adjacent check (State replace/once checks,
approves-no-implementation, factual-correction, recovery routing, payload
rebuild, write-policy routes, return-to-entrypoint count, late-change and
blocked checks, common-work check).
- Worked example anchor sentence -> anchored on the Compare section and
'Ask about jitter first:'; parsed R1/R2 table equality kept.
- Outside-voice queue: dead generic branch (all three skills `continue`
before it; never executed) removed -> none needed (unreachable). Eng
procedure re-checks inside the queue (5 sentences) -> kept free in the
gate describe. Shared CEO/Eng/DX scope-revision sentences hoisted to one
shared /i block per skill; CEO 'Keep candidate scope ...' etc. -> kept free
as short /i checks; prescribed A-D menus kept exact.
- Design register (:1771) sentences -> kept free: grounding-before-question
order and three /i checks; 'Surface real missing decisions' dropped
(positive restatement of the same rule).
Gate-tier coverage that also grades this prose: judges 'plan-eng-review/
SKILL.md sections', 'plan-design-review/SKILL.md passes', 'plan-ceo-review/
SKILL.md modes'; E2Es plan-eng-coverage-audit, plan-review-report,
plan-devex-review-plan-mode (gate). Cosmetic pins dropped without controls:
bold/caps STOP and NOT, issue-number comments unchanged.
Mutation pass (scratch worktree; eng/ceo/design templates and the
outside-voice, testing and review-dashboard resolvers): 33/33 as expected;
24 kept-step/marker/safety/numeric mutations go red, 6 rewordings of dropped
sentences and 3 lowered-register STOP/NOT lines stay green. Log:
.context/mutation-plan-review-cases.md.
Runtime (bun test, Bun 1.4.0, median of 3): 0.85 s -> 0.77 s.
* test(W7c): qa-caller-authority keeps authority and safety lines, drops report prose
W7c (t10 F12, F13). One test file; still renders every host x caller.
Removed assertion groups (invariant -> remaining protection):
- `STOP and wait` / `STOP for the answer` / `AskUserQuestion and WAIT`
capitals (t10 F12) -> approval before `./setup` or install: now an order
check (explicit approval < "only after approval" < `cd <SKILL_DIR> &&
./setup`; OK to proceed? < setup; AskUserQuestion-and-wait < install only
the actual choice), plus case-insensitive "stop for the answer". Stronger:
the mutation that moves setup ahead of the approval ask now goes red (the
old test stayed green). New negative control removes the approval ask.
- Report-layout prose for /review and /ship (keep metadata/outcome tables,
demote headings one/two levels, fields as subsections, keep scores
separate, save baseline normally) -> one final linked report section:
kept free checks on the `## Exploratory QA and Verification Results` /
`### Browser results` / `## Exploratory QA` headings, link every checkpoint
and no second report; gate E2Es review-exploratory-small-cli,
ship-exploratory-small-cli (gate).
- Smoke-pacing prose ("one success and the riskiest changed failure/edge",
"Required even for small diffs ...", "Required: plan commands/assertions,
listed separately", "Other ideas are optional, untested") and duplicate
report-status pins -> kept the numeric `5 minutes/12 probes` bound and the
report-clean-only-when-all-required-pass gate; gate E2Es
ship-exploratory-small-cli, ship-exploratory-plan-checks (gate).
- Dedup prose ("Combine saved `findings` ...", "If both history ... classify
normally", "Report the suppressed count once if nonzero", "If N > 0, print
once:", "Otherwise skip the summary") -> only user `skipped` findings are
suppressed: kept free order check (read decisions before matching) and the
`skipped` / never `fixed` / `auto-fixed` lines; gate E2E
ship-skipped-queued-finding (gate).
- Preparation sentence pins duplicated from qa-probe-gates (Reads in order,
no repeated Reads, "in full and select the surfaces", "Write a **charter**",
"charter** per behavior", "announce finite command timeouts") -> owned by
qa-probe-gates (free, every host); this file keeps scope < methods <
charter structural order for every caller.
- Learnings prose ("Otherwise skip this optional lookup", "Reading old notes
never requires writing new ones") -> report-only lookup never writes: kept
free checks (directory already known, caller permits, do not run the
search helper, no bash/gstack-config/AskUserQuestion, injection throw); the
query interpolation stays checked via `"webhook retries"`.
- Misc prose ("Re-run affected contracts and adjacent happy paths ...", "Run
only for selected browser surfaces", "Confirm each issue by retrying once",
"parent applies queued fixes", "No test framework detected", "Report QA
setup blockers", "required QA") -> judge `qa/SKILL.md workflow` and
`review/SKILL.md workflow` (gate lane); the adjacent kept gates remain free.
Kept as case-insensitive meaning checks (authority, consent, safety): probe-
local blocker rule, report-only never bootstraps/invokes skills, unknown
caller -> report-only, declined -> blocked, credentials/passwords/other tabs,
local-target ownership, no unit-test/curl substitute, functional-only reads
nothing browser-side, never-pass list, named-risk acceptance, noninteractive
-> blocked, test_stub ASK approval, ship waiver cannot complete review,
caller-required rechecks, blocked rechecks never verify repairs, no edits
during Step 4.8, no inner repair loop, REVIEW_START/token-once, native pass
required for 5.8, coverage-cap stop lines, report-only never commits or
dispatches, CI/test-weakening ban, never delete a valid red regression.
Kept unchanged: machine tokens (`bun G start D`, `$B snapshot -i`,
`DIFF_START`, `CONSOLE_ERRORS=`, `$B handoff/resume`), resource paths, the
`If A and allowance remains:` count, `Minimum = 60%, Target = 80%`, all
`.not.toContain` stale-workflow bans, existing negative controls.
Cosmetic pins dropped: STOP / WAIT / LOCAL / Never capitals.
Mutation log: .context/mutation-qa-caller-authority.md (23 mutations, 0
unexpected: 3 reorders, 3 markers, 11 safety red; 6 rewordings green).
* test(W7h): retire CEO TODO-menu prose pins from plan-review-calibration
W7h retirement (t04 F11; W7c list). The calibration corpus and
buildPlanReviewDecisionPrompt never read this CEO prose; the AUQ format
tokens (consumed by plan-tune and judges) and the calibration-harness
outcome tests are unchanged.
Retirement card:
- test: plan-review-calibration.test.ts, the six CEO TODO-menu pins ('Only
unanswered TODO proposals reach this menu', 'Do not ask again about an item
already deferred, skipped or kept', 'carry its actual answer and destination
forward', 'Resolve each remaining proposal through all four steps of 0D,
using the menu below', 'Never batch TODOs — one per question').
- detects: edits to plan-ceo-review TODO-menu prose.
- non_test_callers: none (test-only reads of the template).
- search_command: grep -rn "Never batch TODOs\|Only unanswered TODO" test/
- stronger_proof / replacement: skill-ceo-section-ordering.test.ts 'TODOS.md
updates' slice keeps /i meaning checks for unanswered-only, no re-asking
and one TODO per question (CEO invariants are periodic-only under the W7c
gate, so they keep a free check); 'route through 0D' stays covered by that
file's 0D route checks.
- history: 636175d v1.87.6.0.
- validation: bun test test/plan-review-calibration.test.ts.
Controls (scratch worktree): replacing 'Never batch TODOs — one per question'
or deleting 'Only unanswered TODO proposals reach this menu' turns
skill-ceo-section-ordering.test.ts red (2/2); in this file, removing an AUQ
format token goes red and rewording the retired TODO sentence stays green
(2/2). Log: .context/mutation-plan-review-calibration.md.
* test(W7h): retire spec-template-sync as a duplicate of the freshness check
W7h retirement (t05 F18; Eng factual correction: the file takes 145 ms, its
"2 minutes" was the 120 s timeout, so this is a duplicate-only retirement).
Removes its scripts/free-test-durations.json entry.
Retirement card:
- test: test/spec-template-sync.test.ts (regenerate /spec into an out-dir and
compare spec/SKILL.md + spec/sections/gate-and-file.md with the tracked
bytes; check the generated header).
- detects: /spec template edited without regeneration, or a hand-edited
generated file.
- non_test_callers: none.
- search_command: grep -rn spec-template-sync package.json scripts test .github
- stronger_proof / replacement: gen-skill-docs.test.ts 'generated files are
fresh (match --dry-run)' (FRESH for every skill, no STALE anywhere; the
dry-run covers carved sections such as spec/sections/gate-and-file.md); the
generated-banner checks in the same file cover the header.
- history: c118e24 / 410b492 / 394db32 / b1485d8 (codex T8).
- validation: bun test test/gen-skill-docs.test.ts.
Control (scratch worktree, run against the replacement): a stale
spec/sections/gate-and-file.md, a stale spec/SKILL.md, and a spec template
edit without regeneration each fail gen-skill-docs 'generated files are fresh'
(3/3). Log: .context/mutation-spec-template-sync.md.
* test(W7h): retire the literal 75 KB autoplan budget duplicate
W7h retirement (t04 F18). test/skill-preflight-budget.test.ts asserted the
same four-skill autoplan sum twice: once against
AUTOPLAN_PREFLIGHT_BUDGET_BYTES * 3 (25,600 x 3 = 76,800 = 75 KiB) and once
against a literal 75 * 1024, which would drift silently if the constant
changed.
Retirement card:
- test: skill-preflight-budget.test.ts 'autoplan total under 25 KB
instruction cap × 3 (75 KB instruction budget)'.
- detects: the four plan-* skills' brain preflight + refresh + write-back
instructions exceeding 75 KiB.
- non_test_callers: none.
- stronger_proof / replacement: 'autoplan: sum across 4 plan-* skills stays
under AUTOPLAN_PREFLIGHT_BUDGET_BYTES × 3 (instructions)' in the same file,
which reads the constant from scripts/brain-cache-spec.ts.
- history: c118e24 v1.64.1.0.
- validation: bun test test/skill-preflight-budget.test.ts.
Controls (scratch worktree): shrinking AUTOPLAN_PREFLIGHT_BUDGET_BYTES below
the current total, or growing the eng write-back instructions by 80 KB, turns
the kept constant-based check red (2/2). Log:
.context/mutation-skill-preflight-budget.md.
* test(W7c): workflow-excerpt keeps heading order and tokens, drops judged prose
W7c (t04 F5, F6, F7, F15; F1/F2 already landed with W7a). One test file.
This file guards judge-input fidelity; the judges that read these excerpts
grade the prose itself.
Removed assertion groups (invariant -> remaining protection):
- Ship rule sentences ("Use the documented selector and pre-merge command.",
"During pre-flight, read the existing review log", "Tests must pass or
retain approval ...", "For another repair, repeat rule 2 ...", "The
unchanged release steps follow", "Use this example only after confirming
...", "Retain queued fixes and restore coverage", "Each commit must work
independently", "Keep actual outcomes and incomplete flags", the long
"Continue to Step 9.2 with the core/design-lite findings ..." form, the
Y'd/D'd evidence sentence) -> judge `ship/SKILL.md workflow` (gate lane),
which reads this exact excerpt; kept free checks on step routing (`Step 4`,
`Step 8.1`, 9.2 -> 9.2.1 QA -> 9.3, `Insert Step 5, affected Steps 6–8 and
all of Step 9 before the pending Step 10`, `5 → 6 → … → 11.5`,
`continue to Step 16`), tokens (`BASE_VERSION`, `VERIFY_RESULT stays
fail`, `"partial":N,"not_done":N`, "intentionally dropped"), and the review
cycle order (9.1 < 9.2 < 9.2.1 < 9.3 < 9.4) as a structural check.
- Outside-review routing sentences ("Other preflight mode, including harness
mismatch", "Outside execution or output validation fails", "Retain its
output and diagnosis ...", "No prompt, outside process or native
replacement", "Other preflight failures retain their printed diagnosis",
the full fallback-list regex) -> disabled means no outside call: kept free
order of disabled/prompt/ready/fallback blocks, `outside_status` tokens,
`_DISABLED_REVIEW_MODE`, disabled recheck before dispatch, "disabled means
no replacement", fallback-not-completion; judges
`document-release/SKILL.md workflow`, `plan-eng-review/SKILL.md sections`
(gate lane).
- Eng/design prose ("quote the motivating plan requirement", "Use a scoped
Edit to save this record ... Leave other choices unchanged" full form,
"edit the plan and re-rate") -> judge `plan-eng-review/SKILL.md sections`
and `plan-design-review/SKILL.md passes` (gate lane); approval meaning
checks kept.
- Retro prose ("does not require saved history", "PRs referenced", "not
newly added test cases", "is the live **team** streak", "draft the
tweetable summary using the format in Step 14, then save") -> judge
`retro/SKILL.md instructions` (gate lane); kept headings, `prs_merged:
null`, `streak_days`, the one-second compare window, Capture Learnings
after Step 14, no `$(date`, two `today="<today>"`.
Loosened to case-insensitive meaning checks (stop/consent/approval and
safety lines, always kept): STOP for each pending decision (now a regex
anchor between Step 5 and the Test Plan Artifact), STOP until the actual
answer arrives, no remedy/next section/ExitPlanMode while a choice awaits an
answer, evidence-not-approval, reopened choices need answers, wait for
approval (design), AskUserQuestion once (t04 F6), per-finding approvals,
dispatched-reviewer-missing -> stop, CYCLES >= 3 -> stop with
`converged:false`, no fourth cycle, required-probe gate before Step 10,
failed probes block continuation, coverage-gate not bypassed, failed/unrun
probes not passed, three snapshots match, document-release VERSION-absent
rule, never create an empty commit, `read any other SKILL.md` (t04 F15),
AskUserQuestion-and-wait in the capture prompt.
Kept unchanged: heading/step order checks, `## Review Sections` single-copy
check, fail-closed marker tests, redact-before-edit ordering, approval gates
outside subagent prompts, plan-tune auto-decide output lines, CEO capture
harness contract, deploy formula execution, all stale-string bans.
Cosmetic pins dropped: STOP / ONCE / NOT / OTHER capitals.
Mutation log: .context/mutation-workflow-excerpt.md (21 mutations, 0
unexpected: 3 reorders, 3 markers, 8 safety red; 7 rewordings green).
* test(W7h): retire the paid first-task-scaffold E2E duplicate
W7h retirement (t12 F12). Removes the case from test/helpers/touchfiles-data.ts
(E2E_TOUCHFILES, E2E_TIERS periodic, E2E_KINDS rule) and both
scripts/paid-test-durations.json entries; test/helpers/periodic-exclude-data.ts
had no entry. e2e-tier-alignment, eval-kinds, eval-detach-timeout-floor,
periodic-exclude-policy, touchfiles and paid-shards tests stay green.
Retirement card:
- test: test/skill-e2e-first-task-scaffold.test.ts ('first-task-scaffold',
periodic, ~$0.02): Haiku is told to run exactly one detector command and the
test greps the tool result for code_node / greenfield.
- detects: detector bucket regressions in bin/gstack-first-task-detect.
- non_test_callers: none.
- search_command: grep -rn "first-task-scaffold" test scripts .github
- stronger_proof / replacement: test/preamble-first-task-scaffold.test.ts
(free; executes the real detector on fixture repos for every bucket,
including code_node and greenfield, plus first-run gating and telemetry in
bin/gstack-skill-start). The model added nothing: the prompt dictated the
exact command.
- history: c118e24 v1.64.1.0.
- validation: bun test test/preamble-first-task-scaffold.test.ts plus the
registration tests above.
Control (scratch worktree, against the replacement): making the detector emit
the wrong bucket for a Node project or a commit-less repo turns
preamble-first-task-scaffold.test.ts red (2/2). Log:
.context/mutation-first-task-scaffold.md.
* test(W7c): shrink gen-skill-docs to structural and meaning checks
W7c conversion of the sentence/caps pins in test/gen-skill-docs.test.ts
(t01 F2-F9, F18, F19; t08 F12; t10 F4; t02 #5 duplicate STOP literal).
t01 F1 ('Every commit must map to at least one bullet point') was already gone
on the base. Pins (grep -cE 'toContain|toMatch|indexOf|includes') 1446 -> 1360;
tests 457 -> 457. Untouched: the FRESH/STALE freshness check, the routing
soft-policy negative, skill-start/end script pins, the SESSION_KIND spawned
line, and the old-wording negative pins (delete-then-append, 0D/0F, gate
"if applicable"). Machine markers stay byte-exact: NO UNRESOLVED DECISIONS,
`**UNRESOLVED DECISIONS:**`, `Approval readiness: PASS`, `D<N> — <ROW-ID>: ...`,
`## EXIT PLAN MODE GATE (BLOCKING)` anchors, `**Gate outcome: Blocked**`, helper
commands/flags (gstack-office-hours-review prepare/check/finalize --design,
--out-dir, --report, --unreviewed), helper stop reasons, analytics commands.
Consent, stop, approval, decision-cycle and navigation lines (incl. the Eng
W7c-gate lines at the old 2362-2370) are kept as short case-insensitive checks.
Removed assertion groups (invariant -> remaining protection):
- qa-only 'Never fix bugs' + 'NEVER fix anything' (t01 F8): report-only skill
-> kept free: /never fix/i in Additional Rules + frontmatter no Edit/Glob/Grep.
- qa 'WTF' token (t08 F12): fix-loop self-check -> kept free: /every 5 fixes/i.
- TEST_COVERAGE_AUDIT sharedPhrases sentences and the three Test-step sentences
(t01 F4): read source before tracing/diagramming in plan and ship -> kept
free: order 'concrete source and test files' < Trace data flow < Diagram the
execution < Quality scoring rubric in both, star/GAP markers, the plan/diff
mode split, the Test review subheading structure test; gate judges
'plan-eng-review/SKILL.md sections', 'ship/SKILL.md workflow'.
- 'REGRESSION RULE' / 'IRON RULE' labels (t01 F4): regression coverage is
required -> kept free: REGRESSION RULE heading, /without regression coverage,
that coverage is a critical requirement/i (plan), /regression test is written
immediately/i (ship), plus the existing regression-contract tests.
- '**STOP for each pending decision.**' (t01 F7, retired) -> kept free: 'Wait
for its answer before applying that remedy' (control C1).
- F19 rule sentences: step numbers dropped; kept free /prevent duplicate bumps,
pushes and PRs, never verification/i, /native result never supplies outside
coverage/i, /required-probe gate, never silently waive/i, /never offer a
conversion/i, /actual counts, never estimates/i (deduped 2 -> 1).
- SPEC_REVIEW_LOOP (t01 F3, about 74 sentence pins):
- office-hours dispatch sentences (Before EACH dispatch, parent Read does not
deliver, string unchanged) -> kept free: prepare --design/--out-dir,
order mktemp < prepare < round-N.prompt.md < `dispatch`, /all preceding valid
round files in order/i, /read the entire prepared prompt/i.
- reviewer prompt sentences -> kept free: document/verdict path and schema
wiring, mode sections ('The Assignment', 'What I noticed...'), dimension
order, short checks for coaching protection, no invented customer answers,
minor findings included, identical JSON, classify every prior finding,
open questions vs committed behavior. Dropped: 'Could an engineer implement
this...' ban (positive design-and-coaching check carries it), absence-is-not-
confirmation, unsupported-claims and labeling sentences -> gate E2E
office-hours-spec-review; prior-status validation in office-hours-review.test.ts.
- convergence evidence sentences, 'Recording concerns does not mark them
fixed', Disposition, observation/attempt counting, 'unavailable score is
null', 'missing or invalid verdict is ... never PASS', 'Preserve the failed
output' -> office-hours-review.test.ts (convergence, finalize, invalid
artifact vs --unreviewed, null quality_score) + gate E2E
office-hours-spec-review; kept free: helper stop-reason order and round 3,
/before fixing any findings or dispatching again/i, /on a stop, do not fix
again or re-dispatch/i, /do not edit that generated section/i, finalize before
/existing user approval/i, --unreviewed wiring, /do not fabricate JSON/i,
/not an approval gate/i.
- CEO spec-review sentences (Read Agent's tool definition, omit field, consume
completed review, Keep both consistent, duplicated 'both complete labeled
texts'/'all five dimensions'/'Preserve the failure'/'citing the owning
input'/'## Reviewer Concerns', 'Read these documents' ban, strategy-only and
narrow-decision sentences) -> CEO is periodic-only, so kept free: one short
check per invariant (wait tool, no-wait resume, no advance while waiting,
three launches, stop after third, stop the loop, missing score, null grade,
SCORE 1-10, not persisted, mkdir/append stop, save-failure stop, ITERATIONS,
FOUND/FIXED/REMAINING order, both inputs in full, partial-input refusal,
summary-is-not-the-plan, 1-10 score, PASS rule, cite input) plus the
existing order/count/command checks.
- Design approval reconciliation (t01 F2, about 79 pins):
- Pass 5 sentences and 'AskUserQuestion once per issue. Do NOT batch.
Recommend + WHY.' -> kept free: /stop\W+AskUserQuestion once per issue/i
located after four short reuse/new-evidence/first-decision checks; batching ->
auq-format-always-loaded.test.ts PER_SKILL_RULES ('One issue = one
AskUserQuestion call').
- duplicate 'Never group distinct issues.' pins (2 -> 1) and the rest of the
Design gate 0 sentences -> kept free: 0 < 1 order, short checks per
approval line, refresh < Read-back gate < rerun order; gate judge
'plan-design-review/SKILL.md passes'.
- CEO readiness/gate/0D sentences -> kept free: one short check per approval
line, readiness order, `Approval readiness: PASS`, gate order Read-only <
Verify readiness < 1. Read, route order, /stop for the actual answer/i in
step 4, `D<N> — <ROW-ID>` header, row fields; admin-route sentences ->
ceo-workflow-clarity.test.ts kept checks. Dropped: 'No report or completion
log is needed' (order check carries it), 'Save or present the updated plan',
'checked row IDs...' (marker kept), strategy-only/narrow-decision
sentences (default kept), cross-model/integration sentences reduced to
external-only and native-fallback checks.
- Eng readiness/procedure sentences -> kept free: short approval checks,
gate numbering/order, AUQ call < /stop until the actual answer arrives/i <
Record the answer, recovery order; 'separate it and rebuild this
comparison' -> auq-format-always-loaded.test.ts PER_SKILL_RULES.
- Exit gate 'contract violation' sentence (t01 F9, design/devex/codex) ->
kept free: heading anchor + /do not call ExitPlanMode/i inside the gate.
- Unresolved-decisions gate (t01 F5): '(MANDATORY', 'Never omit'/'never
omitted', 'FINAL non-whitespace line is the unresolved-decisions', 'FAILS the
gate', 'DROP the current skill' -> kept free: NO UNRESOLVED DECISIONS,
/final\s+non-whitespace\s+line/i, `**UNRESOLVED DECISIONS:**`, /missing\s+status
[^.]*fails/i in every gate skill and the resolver, /avoids double-counting/i,
the old soft-wording negative pin.
- delete-then-append 'never mid-file' / 'Do NOT replace the section in place'
(t01 F6) -> kept free: 'delete-then-append flow' + /append the new report at
EOF/i; old-wording negatives kept.
- design-html '**Never include by default (AI slop blacklist):**' (t01 F18) ->
kept free: /AI slop blacklist/i, catalog id comment, lib/design-catalog.ts.
- CODEX_BOUNDARY 'Do NOT read or execute any' casing (t10 F4) -> kept free:
/do not read or execute any/i in every codex-calling skill.
- CEO 0D order anchor '**STOP for the actual answer...**' (one of the 8) ->
kept free: /stop for the actual answer/i at the same position.
Mutation pass (scratch worktree, generated SKILL.md/sections plus
spec-review.ts and lib/office-hours-review.ts; the freshness test is excluded
with -t because any mutation makes it STALE): 43/43 as expected. 25 kept-step,
marker, safety, numeric and retirement-control mutations go red; 18 rewordings
of dropped sentences and lowered-register STOP/NEVER/FAILS lines stay green.
Log: .context/mutation-gen-skill-docs.md.
Runtime (Bun 1.4.0, wall, 3 runs): before 4.42/4.49/4.70s (median 4.49s),
after 4.42/4.53/4.63s (median 4.53s).
* test(W7c): review-workflow-clarity keeps order, tokens and gates, drops judged prose
W7c (t06 #12). One test file. Each removed pin was the text fix for a past
`review/SKILL.md workflow` judge finding; that judge (gate lane) still
grades the prose, so the free test keeps only structure, tokens and gates.
Removed assertion groups (invariant -> remaining protection):
- Plan-audit sentences ("retain the exact command ...", "A mixed item
contributes to both lists", "Keep external-state ...", "If only behavioral
checks remain ...", "File-existence checks ... are static audit checks")
-> audited deliverables vs deferred behavioral checks: kept free checks on
the separation, never DONE from a diff, zero-deliverable no-waive,
UNVERIFIABLE -> Step 4.7, read-only audit (no apps/APIs/state) and inspect
-before-invoke order; gate E2E review-exploratory-small-cli (gate).
- QA loop/report duplicates of qa-probe-gates / qa-caller-authority (probe
loop sentences, shared contracts, report layout prose, "Only the parent
runs report-only discovery", "Read agent/user updates ...", "Prepare one
provisional QA section", smoke composition) -> owned there (free, every
host); here only `5 minutes/12 probes`, headings, link-every-checkpoint,
no second report, never overwrite another run, freshness vs current inputs.
- Persistence/finalization prose (~45: "final zero-edit pass, reconcile",
"by structural identity", "without requiring deleted pre-extraction
blocks", "raw bytes equal the bound snapshot blobs", "Keep both through
re-reviews", "Earlier core tokens remain unused", "Re-read its final-
snapshot supporting source ...", "Fixed advice needs no skip coverage",
"The required in-host adversarial result controls native completion",
"Optional outside attempts keep their own incomplete records ...", etc.)
-> kept free token list (REVIEW_START/PASS_START, COMPLETED, CONVERGED,
CYCLES=0 once, `review_binding`, `snapshot_covered_paths`,
sharedLibsFingerprint, `completed:false`, evidence_paths/helper_target),
cycle limit (at most 3, Below 3 repeat, At 3 persist `converged:false`),
section order, and the integrity gates below; judge `review/SKILL.md
workflow` (gate lane); gate E2Es review-army-delivery-audit,
review-army-json-findings, review-army-quality-score (gate).
- Confidence-calibration prose (data loss/outage examples, missing field /
nullable / race quoting, metaclass/ORM sentence, missing-literal sentence,
learning-log sentence) -> kept the score table as a routing table (each
band's action), `[CRITICAL]` / `[CRITICAL|INFORMATIONAL]`, no P0-2,
`file:line`, force 4-5, never invent 7+, unverified-without-quote, gate
before table; judge `review/SKILL.md workflow` (gate lane).
- Shared-code reuse safeguard enumeration (assume-unchanged/skip-worktree,
sparse index, symlinks, submodules, Git filters, encodings, fsmonitor,
textconv, provenance, binding wording) -> enforced in code and covered by
free tests test/shared-libs-snapshot-check.test.ts,
test/shared-libs-evidence.test.ts, test/shared-libs-checker-interface-
evidence.test.ts; kept the checker command/heredoc/JSON tokens and the
fail-closed lines.
- Misc definitions ("A probe is affected when ...", "An invocation is this
/review run ...", "Step 1.5's provisional notes ...", "Otherwise record
each considered specialist", "Core findings use the confidence gates
below", "highest confidence", "distinct specialists", "all source names",
"active or unknown reader/writer", Greptile "Before edits, inspect ...")
-> judge `review/SKILL.md workflow` (gate lane).
Kept as case-insensitive meaning checks (stop/consent/approval/safety):
no edits before Step 5, readers returned or confirmed stopped, no edits in
Step 4.8, stop without edits on unconfirmed readers (review and ship),
stop at 3 cycles with no clean summary or fourth pass, failed/blocked
probes mean incomplete, ship named-risk cannot complete review, native vs
outside cannot substitute, structured-review gate, never carry coverage
forward, never invent a binding, skipped advisory cannot suppress a defect,
user-skip-only suppression, Greptile reply != code approval and wait for
approval before editing, HIGH-impact gate options (A ends, read-only audit,
no shipping authorization), never downgrade severity, never trust a supplied
hash, missing metadata cannot dedupe, test creation needs approval,
report-only review never runs setup/installs/cookie import.
Kept unchanged: all marker-order arrays, manifest order, Pre-Landing summary
after the logger, finish-token count and negatives, stale-wording bans
(incl. `git diff origin/main`), ship/review routing negatives.
Cosmetic pins dropped: STOP / NON / COUNT capitals; the test titled "resolves
the judged ... ambiguities" is renamed to say what it keeps.
Mutation log: .context/mutation-review-workflow-clarity.md (23 mutations,
0 unexpected: 3 reorders, 4 markers, 11 safety red; 5 rewordings green).
* test(W7c): ship-control-flow keeps step routing, tokens and gates, drops rule prose
W7c (t09 F1, F2, F3, F4). One test file.
Removed assertion groups (invariant -> remaining protection):
- Work-list prose ("Expand a repair into individual steps ...", "This
replaces the current item ...", "Add its destination only if ...", "The
saved list takes precedence ...", "For another repair, repeat rule 2 ...",
"The unchanged release steps follow", "Follow the saved work list",
"Keep CYCLES and scoped approvals across this repeat", "Keep the findings
and their approvals", "any further repair inserts its checks ...") ->
repairs insert ordered steps ahead of pending work: kept free checks on the
example sequences (`5 → … → 11.5`, `9 → 10 → 11` before 11.5, `5–11.5 →
12–14 → 16`, `14.5 → 15 → 16`, `9 → 10 → 11 → 11.5`), insert-before-
pending routing lines, no unlisted steps, same attempt counts, no reset of
the one-repair limit; judge `ship/SKILL.md workflow` (gate lane); gate
E2Es ship-docsync-recovery, ship-docsync-stale-after (gate).
- Queue-qualification sentence pins duplicated in ship-version-sync (t09 F4:
"require successful utility output ...", "`offline:false` qualifies ...",
"Offline output without that fallback ...") -> kept in
test/ship-version-sync.test.ts next to its script-behavior tests (free);
here the Qualify < Usable < No-usable order, `NEW_VERSION=CANDIDATE_VERSION`,
`>= NEW_VERSION`, `BUMP_LEVEL`, `currentVersion` and the approval line stay.
- Distribution prose ("List candidate distribution paths", untracked files,
manifest sentences) -> kept the New-artifact branch label, AskUserQuestion
and the no-release-during-/ship safety line; gate E2E ship-local-workflow
(gate).
- Dispatch/dashboard/coverage prose ("Omitting the flag runs the subagent in
the background", "keeping a fresh context", "Use one row for each entry in
step 1", "including its undetermined-percentage and test-only rules") ->
kept the `run_in_background: false` contract at all three sites, the
dashboard header/verdict tokens, no inline Skill fallback, coverage gate
not bypassed; gate E2E ship-coverage-audit (gate).
- Recovery/docs/test sentences ("If an attempt remains and either ...", "a
concrete launch/input/permission correction ...", "restart Step 16 stage 1
to regenerate ...", "This recovery also applies ... stage 5", "Content-
preserving commits keep valid evidence", "Retain queued fixes and restore
coverage", "If this pass made edits, resume at the next decision ...",
"Undispatched host-unsupported/gated specialists do not block", "count it
in the invocation record before launch", "Keep Step 9.4's incomplete
flags", "saved handle, original token and source", "unknown or missing
status is unavailable", "Finish the saved replies ...", "fetch and inspect
the remote, then merge ...") -> judge `ship/SKILL.md workflow` (gate
lane); gate E2Es ship-docsync-* (gate) and ship-skipped-queued-finding
(gate); the adjacent stop/approval lines stay free.
Kept as case-insensitive meaning checks (t09 F1/F3; stop/consent/approval
and safety lines): STOP and AskUserQuestion gates apply during repairs, a
repair never resets or expands approvals, children return evidence not
permission, prior-invocation audits never qualify, one repair/re-audit and
no third attempt (initial-plus-one, once), Otherwise -> stop unless the user
accepts the named docs risk, never silently refresh hashes, never run a third
audit, dispatched-reviewer-missing -> stop, CYCLES >= 3 -> stop with
`converged:false`, no fourth cycle, missing coverage cannot be replaced,
failed probes never clean, required-probe gate before Step 10, cannot waive
reviewer/test/security gates, native recovery stops the task first and
remains blocked, outside output cannot replace native, never attach new
tokens to old work, failed build stops shipping, never edit the child's
counts, test failure stops publication, push failure stops with no Step 19,
never rewrite history, never bypass failed guards, both redaction scans
before publishing, no release publish during /ship, only approval changes the
version, declining bootstrap is not test approval.
Kept unchanged: the executed plan-tune nudge shell test (3 roots), section/
step ordering checks, Step 11.5 placement and `review_binding.*` fields,
branch-label order in the adversarial finish (every host), all stale-
wording bans, `(recommended)` marker on Add tests.
Cosmetic pins dropped: STOP / ONE capitals (t09 F1, F3).
Mutation log: .context/mutation-ship-control-flow.md (21 mutations, 0
unexpected: 3 reorders, 3 markers, 9 safety red; 6 rewordings green).
* test(W7h): replace workflow-judge-cache source-grep counts with an invariant
W7h retirement (t09 F16, t13 F6) of the source-grep counts in 'workflow
registration preserves model work and reserves only terminal-recording
grace'. The stage-order check over runWorkflowJudge (the cheapest guard) and
every behavioral callback test stay.
Retirement card:
- test: workflow-judge-cache.test.ts pins of skill-llm-eval.test.ts source:
"execution: reused ? 'reused' : 'executed'", 'const workDeadline = started
+ JUDGE_MS;', 'const WORKFLOW_JUDGE_RECORD_MS = 5_000;', 'const
WORKFLOW_JUDGE_TEST_MS = JUDGE_MS + 10_000;', "testName: 'review/SKILL.md
workflow'", and the magic counts /}, WORKFLOW_JUDGE_TEST_MS\);/ x17 and
/}, JUDGE_MS\);/ x7.
- detects: any workflow eval added or removed, any constant reformatted.
- non_test_callers: none.
- stronger_proof / replacement: (a) a new invariant in the same test: every
`await runWorkflowJudge(` registration closes with WORKFLOW_JUDGE_TEST_MS
(the property the 17-count stood in for, now independent of how many evals
exist); (b) the real-callback tests in this file exercise execute-once,
reuse with provenance (execution reused/executed), timeout, supersede,
expiry and the recording deadline.
- history: 636175d (#2898); last 7fca42a.
- validation: bun test test/workflow-judge-cache.test.ts.
Controls (scratch worktree, test/skill-llm-eval.test.ts): moving cache.publish
before the score assertions or registering the ship workflow judge with
JUDGE_MS goes red; adding a correctly registered eval and reformatting
5_000 as 5000 (both red under the old pins) stay green (4/4). Log:
.context/mutation-workflow-judge-cache.md.
* test(W7h): retire the bws session-awareness (ELI16) paid case
W7h retirement (t14 F5). Removes the case from test/skill-e2e-bws.test.ts and
its E2E_TOUCHFILES / E2E_TIERS (gate) / E2E_KINDS (rule) entries in
test/helpers/touchfiles-data.ts; scripts/paid-test-durations.json is
per-file and test/helpers/periodic-exclude-data.ts had no entry.
test/touchfiles.test.ts selection examples now name sibling cases with the
same touchfiles (skillmd-outside-git, skillmd-no-local-binary).
e2e-tier-alignment, eval-kinds and touchfiles stay green.
Retirement card:
- test: skill-e2e-bws.test.ts 'session-awareness' (gate, paid).
- detects: nothing gstack ships. The prompt coaches a mode that no longer
exists ("_SESSIONS=4, so ELI16 mode is active ... Re-ground them";
`git grep ELI16` outside test/ and CHANGELOG is empty) and says "DO NOT
actually call AskUserQuestion"; every asserted word (billing/acme,
payment/feature, stripe/checkout, recommend) comes from the prompt itself.
- non_test_callers: none.
- search_command: git grep -n "session-awareness\|ELI16" -- test scripts
- stronger_proof / replacement: the AUQ format's grounding line
'Project/branch/task: <1 short grounding sentence using _BRANCH>' is pinned
free by plan-review-calibration.test.ts (format tokens) and graded by gate
E2E auq-format-gate; session identity in the preamble stays pinned by
skill-validation.test.ts 'contains session awareness' and
gen-skill-docs.test.ts 'session awareness lives in gstack-skill-start'.
- history: c118e24 v1.64.1.0 (#2572).
- validation: bun test test/touchfiles.test.ts test/e2e-tier-alignment.test.ts
test/eval-kinds.test.ts.
Control (scratch worktree, against the replacement): removing the
'Project/branch/task:' grounding line from the AUQ format resolver turns
plan-review-calibration.test.ts red (1/1). Log:
.context/mutation-session-awareness.md.
* test(W7h): pin the Codex availability probe free for the four offering skills
Replacement check for the codex-offered-* paid retirement (t14 F8): each of
office-hours, plan-ceo-review, plan-design-review and plan-eng-review must
render the executed `command -v codex` probe and its `not_installed`
fallback mode in its generated SKILL.md + sections. These are bash lines the
agent runs, so a deterministic grep replaces a paid model summary of them.
skill-validation.test.ts already covers the default-on outside voice for
eng/CEO/devex; this adds office-hours and plan-design-review and the
fallback token.
Control (scratch worktree, generated files): removing the probe from
office-hours, plan-design-review or plan-eng-review, or the CEO
not_installed branch, turns the new test red (4/4). Log:
.context/mutation-codex-offering.md.
* test(W7c): ship-workflow-clarity becomes a structural check of ship-only invariants
W7c (t07 F1, F2, F15). One test file: 59 tests / ~438 pins -> 33 tests.
Round-tagged titles (r6/r12/r22) are gone. Every invariant this file shared
with the other converted ship files is now checked once, in its owner:
ship-control-flow (work list, repair ranges, docs/test/push recovery,
adversarial finish order, Step 11.5 binding), ship-plan-completion-
invariants (evidence ledger, push/commit, waivers), ship-reentry-gates (stop
lines, docs reentry, NEW_TITLE), workflow-excerpt (review cycle order,
coverage/probe gates), qa-probe-gates / qa-caller-authority (QA loop).
Removed assertion groups (invariant -> remaining protection):
- Duplicated literals (45 repeated 2-7x, e.g. "Keep the same attempt counts
throughout the invocation" x7, "Docs, TODO edits ... make evidence STALE"
x5, the CYCLES >= 3 / fixes-below-cap / no-edits sentences, push-failure
and native-recovery sentences) -> one free check each in the owners above.
- Smoke/plan-check loop sentences and "Required even for small diffs ..."
-> qa-probe-gates (free, every host); gate E2E ship-exploratory-small-cli
(gate). Kept here: smoke guard started once before step 3, every host.
- Invocation-record field lists (receipt/handle/attempt field prose, "reuse
this invocation's saved level", FRESH/ALREADY_BUMPED level derivation,
receipt definition, start-token definition) -> kept `BUMP_LEVEL`, record
outside the product tree, approval scope, never borrow/replace a token,
REVIEW_START/PASS_START/DESIGN_START, `gstack-review-log --start`,
`gstack-wtree`; version derivation also in test/ship-version-sync.test.ts
(free); judge `ship/SKILL.md workflow` (gate lane).
- Reuse/judge-evidence prose ("original command, result/counts ...",
"Compare hashes or complete bytes", "consumed files, fixtures ...",
"complete expanded request, rubric ...") -> kept never resample, changed/
unknown dependencies rerun, mandatory reviews still run, freeze inputs;
judge `ship/SKILL.md workflow` (gate lane).
- Build/route prose ("If none exists, record not applicable ...", "declared
generation/build commands ...", "Run them and save results", "Explain why
other changes cannot affect it", "Inspect writer handles, including the
docs child", "Keep Step 8's original child report and counts", "Recheck
affected plan items ...", "append current evidence ...") -> kept stage
order, route-label order, prompts/templates count as behavior, no-change ->
stage 3, writer termination before another writer, stop until confirmed,
build-failed label, no substitute command; gate E2Es ship-docsync-*
(gate).
- Docs-collector prose ("Terminal completion or confirmed termination is
sufficient", "Require every field/type ...", "Compare saved base and input
hashes ...", "Otherwise save post-child hashes ...", "A stale snapshot is
neither ...", "Reconcile those before proceeding") -> kept attempt counting
incl. failed launches, request alone insufficient, exited child blocked,
HEAD/index unchanged + `files_updated`, protected-file exclusions,
`documentation_section`, failed/blocked -> recovery, actual user exception,
blocked status retained, same approved scope and exact content, retry
restarts stage 1; gate E2Es ship-docsync-missing-marker,
ship-docsync-launch-failure, ship-docsync-late-result (gate).
- Release/commit prose ("Record the release decision after a version was
actually written", "Skip unchanged versions and manifest-only repairs",
"Best-effort, non-interactive, non-blocking", "Group VERSION + CHANGELOG
+ TODOS.md ...", "Only the final VERSION/CHANGELOG commit gets the release
version and co-author trailer", "use the first changed component ...
(major/minor/patch/micro") -> kept ALREADY_BUMPED, the stale "skip if
ALREADY_BUMPED" ban, `baseVersion`/`currentVersion`, absent fourth
component is zero, no automatic rebump, digest tokens and pre-push digest
check, no Git tag, commit-before-push order; gate E2E ship-local-workflow
(gate).
- Coverage/design prose ("Count each generation-authorized attempt ...",
"including the initial audit, failures and zero-test results", "Two passes
already used ...", "read-only reassessment uses no pass", design-lite
ownership, same path/line merge, specialist stats, distinct defects) ->
kept the 2-pass and 30/5/2-minute limits, re-entry never resets, missing
permission is not approval, audit-only header/override order/no commit/
return decisions, child JSON, one item with both sources and stricter ASK,
neither pass substitutes; gate E2Es ship-coverage-audit,
ship-coverage-value (gate). Design-review-lite E2E is periodic, so the
merge rule keeps its free meaning check here.
- Spec/title/comment prose ("existing open PR/MR", "one match supplies the
existing title/identity", "Save the result for Step 19's recheck", "In a
new shell, restore ...", "Partial, deferred, failed, dropped or unverified
scope uses ...", "newest", "Try to auto-resolve if they are simple",
"distinguish an app from a library/CLI", "If the ask is App Store/
TestFlight distribution", "a new package.json or Cargo.toml alone ...",
"After edits, the full re-review repeats ...") -> judge `ship/SKILL.md
workflow` (gate lane); ship-apple-gate (free) for the Apple route.
Kept as case-insensitive meaning checks (stop/consent/approval/safety,
always kept): never invent credentials, ask for unknown targets first,
include the workflow in tests/review, frontmatter read as data never shell
source, exact spec_branch match, positive-integer issue number, Closes #N
only for fully completed scope, lookup failure/ambiguous match -> stop never
"no PR", never publish an unprefixed title, ambiguous conflicts -> stop,
AskUserQuestion and wait, unclear Apple target -> AskUserQuestion and wait,
confirmed app -> stop and Read, stop for repair (recommended) or accept each
named probe risk, skipping a fix is not risk acceptance, accepted risk is not
passing verification, queue without editing in Steps 10/11, never credit
outside coverage, do not report a clean review, unavailable outside
challenge never resets the three-cycle limit, records mismatch blocks
release, missing snapshots block regardless of HEAD, uncertain edits cannot
be exempted, scripts/dependencies/runtime config need live tests, named
testing gap A/B/C (recommended), reuse prior B only for same scope, declared
but unavailable suite blocks, explicit untested-scope approval, report the
gap never FRESH, no documentation writer after push, do not run create
commands for an existing PR, Greptile status truthfulness.
Kept unchanged: all stale-string bans, machine tokens and JSON contracts
(`gh pr list ... --json number,title,url`, glab list, `v$NEW_VERSION `,
`${NEW_TITLE:?...}` guard, exit 3, `--allow-paths ...`, `| FRESH (exit 0) |`,
`--label <lane> --expect-cmd`, `gstack-review-read`, `git diff <reviewed-tree>
<current-tree>`, `"status":"complete|no_pr|unavailable"`, specialist and
coverage JSON, `status:"unavailable"`/`completed:false`/`converged:false`,
`Pre-Landing Review: INCOMPLETE`, `ALREADY_PUSHED`), the threshold/branch/
installed-asset reference test, the section-index row check.
Cosmetic pins dropped: STOP / ONE / ONLY capitals and bold (t07 F2).
Mutation log: .context/mutation-ship-workflow-clarity.md (23 mutations, 0
unexpected: 4 reorders, 4 markers, 9 safety red; 6 rewordings green, all of
which turned the old test red).
* test(W7c): qa-exploratory-callers keeps phase order and authority, drops duplicated prose
W7c (t04 F8, F16). One test file; only the generated-parent-path and
runner-prompt pins change. The validator-backed caller tests (observer,
checkpoint causality, read-only diff grammar, write watcher, deadline
authority) are unchanged.
Removed assertion groups (invariant -> remaining protection):
- Parent-path loop and preparation pins duplicated from qa-probe-gates
(Reads in order, "Reuse Step 4's surfaces ...", "Finish missing methods
before charters", isolation preflight, shared preparation order, the six
plan-execution sentences, the three shared loop contracts) -> owned by
qa-probe-gates (free, every host, with semantic mutation controls).
- Parent prose ("Templates cannot replace them", "Only the parent runs
report-only discovery", "Read agent/user updates and await results ...",
report-clean and not-run list duplicates of qa-caller-authority) -> caller
observer attribution tests in this file (free) and gate E2Es
review-exploratory-small-cli, ship-exploratory-small-cli (gate); report
gates stay in qa-caller-authority (free, every host).
- Shared-loop duplicates (preserve key/value, Q supplies observed, write the
report, return to step 2, pass requires) -> qa-probe-gates /
qa-caller-authority; kept the observationCommand/observed field definitions
and identity-hash rule here.
- Runner harness prompt sentences (t04 F16: receipt-interface prose,
appendSystemPrompt rules, scheduling prompt) -> reduced to one anchor per
rule; `at most ${maxTurns} assistant turns`, maxTurns/timeout/reserve
values, `(probe-...)`, `reports/review.md`, and every `not.toMatch`
scenario-answer ban stay. validateCallerEvidence remains the behavioral
oracle.
Kept as case-insensitive meaning checks: phase order 1-4 per caller,
review readiness follows report-only rules and never installs/imports
cookies/bootstraps, smoke stop at 5 minutes or 12 probes, G enforces the
deadline, never reset D/bypass G, plan checks remain required past the
smoke budget, /review incomplete and /ship blocked unless the user accepts
the named risk, repeat step 3 for affected checks, harness keeps parent
decision gates, turn limit never authorizes skipping work, never group
probes, Hard deadline UTC rule, HANDOFF.md read before completed records,
no checkpoint for deferred ideas.
Cosmetic pins dropped: AND / Never capitals.
Mutation log: .context/mutation-qa-exploratory-callers.md (15 mutations, 0
unexpected: 2 reorders, 2 markers, 6 safety red; 5 rewordings green).
* test(W7h): pin the CEO /office-hours prerequisite structure free
Replacement check for the plan-ceo-review-benefits paid retirement (t14 F7).
The paid case had a model summarize plan-ceo-review/SKILL.md and grep the
summary for "office hours" and "design doc". The rendered text is static, so
this free test checks it directly: the Prerequisite Skill Offer triggers on
the design-doc check's "No design doc found", offers
'A) Run /office-hours now' and 'B) Skip — proceed with standard review', does
not re-offer later in the session, and Mid-session detection (0A) offers
`/office-hours`. Per the Eng Retirements item, CEO invariants become
periodic-only once this gate case retires, so they keep this structural
free check.
Control (scratch worktree, generated plan-ceo-review/SKILL.md): removing the
/office-hours option, the no-design-doc trigger, the mid-session offer or the
no-re-offer rule goes red; rewording the pitch stays green (5/5). Log:
.context/mutation-ceo-benefits.md.
* test(W7h): retire the paid SKILL.md-summary cases in skill-e2e-plan
W7h retirement (t14 F6-F8): office-hours-spec-review, plan-ceo-review-benefits
and codex-offered-{office-hours,ceo-review,design-review,eng-review}. Each
asked a model to read a generated SKILL.md, write a summary, and grepped the
summary for facts the generated file states directly.
Registration cleanup in the same commit: E2E_TOUCHFILES, E2E_TIERS (all six
gate) and E2E_KINDS (rule) in test/helpers/touchfiles-data.ts; six
scripts/paid-test-durations.json case-shard entries; no
test/helpers/periodic-exclude-data.ts entries existed. The fast PR profile
(scripts/test-pr-profile.ts) drops plan-ceo-review-benefits and
office-hours-spec-review from PR_PROFILE_CASE_IDS and PR_PROFILE_FILES. Tests
that named the retired ids as live examples move to existing ones:
paid-free-boundary and paid-pr-profile (a CEO template change now plans
auq-format-gate plus the CEO judge; the case-shard fixture uses
plan-review-report), test-pr-profile, touchfiles (CEO selection count
21 -> 19), the touchfile-moved-code w5 golden (retired ids removed), the
behavior-lane case-shard census (78 -> 72) in cookie-validation-phases, and
the skill-e2e-plan timeout list and full-gate PR floor (78,425 -> 72,755 s)
in paid-retry-supervision. e2e-tier-alignment, eval-kinds,
eval-detach-timeout-floor and paid-shards stay green.
Coverage consequence: per the Eng Retirements item, plan-ceo-review-benefits
was the only gate-tier CEO E2E in the fast PR profile, so CEO invariants are
treated as periodic-only under the W7c gate (every CEO conversion in this
branch keeps one free check per invariant); the gate judge
'plan-ceo-review/SKILL.md modes' still runs for CEO prompt changes. An
office-hours prompt change now has no fast PR E2E and is reported as
deferred to broad coverage rather than run.
Retirement cards:
- office-hours-spec-review: detects the Spec Review Loop's stated dimensions,
dispatch tool, iteration cap and metrics. Replacement: gen-skill-docs.test.ts
'SPEC_REVIEW_LOOP resolver' (round-3 cap, prepare/check/finalize commands,
verdict and round-file wiring) and test/office-hours-review.test.ts (verdict
parsing). Control: changing the round limit in
scripts/resolvers/spec-review.ts turns gen-skill-docs red.
- plan-ceo-review-benefits: detects the CEO /office-hours prerequisite and
mid-session offer. Replacement: gen-skill-docs.test.ts 'plan-ceo-review
offers /office-hours when no design doc exists and detects a lost user
mid-session' (previous commit). Control: 4/4 red mutations there.
- codex-offered-* (x4): detects the `command -v codex` probe, a fallback and
optionality. Replacement: gen-skill-docs.test.ts 'Codex outside voice is
offered behind an executed availability probe' (probe + not_installed
fallback for all four skills) and skill-validation.test.ts 'plan reviews run
the Codex outside voice default-on'. Control: 4/4 red there.
- non_test_callers: none; history: c118e24 v1.64.1.0 (#2572).
- validation: the registration tests named above.
Controls log: .context/mutation-skill-e2e-plan-retirements.md.
* test(W7h): delete the codex-offering prompt builder left unused by the retirement
Dead-code follow-up to the codex-offered-* retirement: buildCodexOfferingPrompt
(test/helpers/codex-offering-fixture.ts) had no caller besides those cases.
Deletes the helper, its free test (test/codex-offering-fixture.test.ts), the
two captured public fixtures it replayed, its scripts/free-test-durations.json
entry, and the helper/fixture paths still listed as touchfiles of six plan
review E2Es in test/helpers/touchfiles-data.ts.
Retirement card:
- test: codex-offering-fixture.test.ts (pinned the builder's prompt text and
replayed captured timeout/CDD transcripts against it).
- detects: changes to a prompt that no paid case sends any more.
- non_test_callers: none (grep -rn codex-offering test scripts is empty).
- replacement: none needed; the behavior it supported is now the free probe
check in gen-skill-docs.test.ts.
- validation: bun test test/touchfiles.test.ts test/touchfiles-map-diff.test.ts
test/paid-shards.test.ts test/e2e-tier-alignment.test.ts; bun run typecheck.
* test(W7h): delete validateOfficeHoursSpecSummary left unused by the retirement
Dead-code follow-up to the office-hours-spec-review retirement:
validateOfficeHoursSpecSummary in test/helpers/office-hours-completion.ts graded
only that paid case's model-written summary. Deletes it and its free
describe block in test/office-hours-completion.test.ts ('office-hours
spec-review summary completion'); every other validator and test in both
files is unchanged.
Retirement card:
- detects: a summary missing five dimensions, the Agent dispatch, the 3
iteration cap or the four metrics.
- non_test_callers: none.
- replacement: the facts it graded are pinned free on the generated text by
gen-skill-docs.test.ts 'SPEC_REVIEW_LOOP resolver' (round-3 cap and loop
wiring); see the retirement commit's control.
- validation: bun test test/office-hours-completion.test.ts.
* test(W7h): drop the no-op review-report check from setup-gbrain-remote
W7h retirement (t12 F9) of one assertion in the paid setup-gbrain-remote
case. No registration changes (the case stays).
Retirement card:
- assertion: `expect(/## GSTACK REVIEW REPORT|critical_gaps/i
.test(result.output)).toBe(false)` with its comment.
- detects: /setup-gbrain writing a plan-review report or critical_gaps
findings, which it never produces; the removed comment itself said
"Setup-gbrain doesn't have a review report contract". Copied from the
plan-review tests (c118e24, #2572).
- non_test_callers: none.
- replacement: none needed; `grep -rln "PLAN_FILE_REVIEW_REPORT\|GSTACK REVIEW
REPORT\|critical_gaps" setup-gbrain/*.tmpl setup-gbrain/sections/*.tmpl`
is empty, so there is no behavior to control. The case's real contracts stay:
the secret token never reaches CLAUDE.md, `mcp add ... --transport http`,
and `Mode: remote-http` (read by bin/gstack-gbrain-detect).
- validation: bun test test/skill-e2e-setup-gbrain-remote.test.ts (skips
without EVALS; module loads).
* test(W7h): delete incompleteFirstFileView, which no paid case uses
W7h retirement (task scope: "incompleteFirstFileView + its free test in
test/shared-libs-fixture.test.ts if no paid case uses it"). `grep -rn
incompleteFirstFileView test scripts` showed only the export in
test/helpers/shared-libs-eval-fixture.ts and its own free describe block;
no paid case or grader calls it, so the helper and the two tests that pinned
it are dead code.
Retirement card:
- test: shared-libs-fixture.test.ts 'incompleteFirstFileView (census
36641820398 shared-libs-pr-coverage)' (2 tests).
- detects: regressions in a page-1 recovery classifier nothing consumes.
- non_test_callers: none.
- replacement: none needed; the shared-libs page budgets that a recovery read
is charged to stay enforced by the kept budget function and its tests.
- validation: bun test test/shared-libs-fixture.test.ts (241 pass);
bun run typecheck.
* test: update the periodic release floor for the opus-5-5 overlay case
paid-retry-supervision.test.ts 'detached PR fallback and release commands
cover their actual default worker budgets' was red on the integrated base
23a0be82d (green on f30b7b7): 737bf478d (E2, "dedicated-tools overlay case
pinned to claude-opus-5-5") added a periodic file, so the recomputed periodic
release floor is 33,852 s, not the pinned 33,821 s. Updates the pinned floor
and the two-tier sum (55,546 -> 55,577 s); the detached release wall
(116,700 s) still exceeds the recomputed census wall plus 120 s, which the
same test asserts, so package.json does not change. eval-detach-timeout-floor
stays green.
* test(E1): pin CLAUDE.md facts, /ship step references and state-root literals
New free test test/instruction-facts.test.ts (CEO/DX/Eng E1 items; DX-13 one
named test() per check, each failure prints both values, file:line and the
fix):
- 'CLAUDE.md names every TREE_MUTATING file and does not call the set
empty': imports TREE_MUTATING from scripts/test-free-shards.ts (today
test/bootstrap-retention.test.ts).
- 'CLAUDE.md states the current catalog ceiling from
test/catalog-budget.test.ts': the "`CATALOG_BUDGET_TOKEN_EQUIVALENTS`
(N today" value must equal the constant (1,194).
- 'CLAUDE.md points at package.json for detach timeouts instead of stating
numbers': no eval:bg* --timeout value appears in CLAUDE.md (plain or with
commas), and the detach section names package.json's eval:bg:gate /
eval:bg:periodic and states no timeout number.
- 'every explicit /ship Step reference names a defined /ship step with the
expected title': matches only `/ship Step N` and `/ship's Step N`
(backticked or not) across every .tmpl and scripts/resolvers/; each must
be a table row whose step is a heading in ship/SKILL.md.tmpl or
ship/sections/*.md.tmpl whose title contains the row's subject, and the
reference's line or enclosing heading must mention that subject; stale rows
fail. Today: document-release release-body Step 7 -> /ship Step 14
(TODOS.md), plus the incident-history comment in
scripts/resolvers/constants.ts ("/ship Step 18"), kept as a historical row
with a reason (handoff: that comment's step is stale; the dispatch is Step
14.5).
- 'negative control: moving a reference to another existing /ship step
fails': rewrites the TODOS reference to Step 12 in memory and asserts the
checker reports it.
- 'every ~/.gstack/projects/ and ~/.gstack/analytics/ hit is allowlisted with
a reason' and 'every state-root allowlist row still matches a hit': all 37
current hits in .tmpl and scripts/resolvers/ are covered by 36 rows, each with one of four
reasons (user-facing description; gbrain context_queries manifest glob read
literally by bin/gstack-brain-context-load.ts; source comment; pending
conversion tracked in TODOS.md). The readiness-gate EVAL_DIR rows reuse the
reason from test/state-root-ratchet.allowlist.json. New hits fail, and
converting a literal fails until its row is deleted.
Mutation pass (scratch worktree): 15/15 as expected; drifting the catalog
number in CLAUDE.md or the constant, dropping or emptying TREE_MUTATING text,
adding a TREE_MUTATING key, copying a detach timeout into CLAUDE.md,
moving/undefining/retitling a /ship reference, a new /ship reference, a new
state-root literal and a converted allowlisted literal all go red; unrelated
rewording stays green. Log: .context/mutation-instruction-facts.md.
* test: integration repairs after F2 (calibration test renders the AUQ format; census counts and release floors without the reverted overlay case)
* chore: constants comment names the /ship documentation dispatch, not a stale step number
* test: office-hours-auto-mode joins the fast PR profile (replaces the retired office-hours-spec-review probe)
* docs: TODOS for the Opus 5.5 prompt-cleanup deferrals
* fix(state-root): design-doc discovery reads $GSTACK_STATE_ROOT/projects
DESIGN_DOC_DISCOVERY and autoplan's Step 2 design-doc lookup read the literal
~/.gstack/projects path, so under a non-default state root they missed the
docs office-hours writes. The block now globs "$GSTACK_STATE_ROOT"/projects,
and each caller's bash block (autoplan, plan-ceo/devex/eng-review, the
spec-review prerequisite recheck) resolves the root with the guarded
gstack-paths eval first.
Allowlist: drops the autoplan ls row, both _LOCALDOC rows and the resolver
source-comment row from STATE_ROOT_ALLOWLIST.
Tests: the eng Design Doc Check fixture installs gstack-paths and seeds docs
under a non-default root; the CEO discovery fixture passes an explicit
GSTACK_STATE_ROOT.
* chore: regenerate SKILL.md and context budget after design-doc discovery state-root change
* fix(state-root): history, report and ledger paths name $GSTACK_STATE_ROOT
Agent-facing read/write paths that named the literal ~/.gstack default now
name the state root their writers use:
- canary-history.jsonl and health-history.jsonl (the blocks above already
resolve the root and print PROJECT_DIR)
- ios-design-review report: adds {{SLUG_SETUP}} so the report path is the
PROJECT_DIR it prints
- plan-tune dream-cycle gate's distillation-proposals.json
- retro's skill-usage and eureka reads: read the path gstack-retro-metrics
prints (it already checks $_gstack_sr_root/analytics)
- ship's apple-effects.log durable-effect ledger and the gstack-evidence
ledger description (bin/gstack-evidence writes under resolveStateRoot())
Allowlist: drops the nine matching PENDING rows from STATE_ROOT_ALLOWLIST.
* chore: regenerate SKILL.md, ship goldens and context budget after history/ledger state-root paths
* fix(state-root): taste-profile legacy fallback globs $GSTACK_STATE_ROOT designs
The legacy approved.json fallback in {{TASTE_PROFILE}} (design-consultation,
design-shotgun) named the literal ~/.gstack/projects path while the approval
writers use $GSTACK_STATE_ROOT/projects/$SLUG/designs.
Allowlist: drops the design.ts PENDING row.
Pin (W7a): design-consultation-contract's exact glob string becomes a
meaning-level check (legacy fallback names the state-root designs dir and
approved.json, and no ~/.gstack literal).
* chore: regenerate SKILL.md and context budget after taste-profile state-root glob
* fix(state-root): completion-status telemetry prose names $GSTACK_STATE_ROOT/analytics
gstack-skill-end writes analytics under the resolved state root ($_GH), so the
PLAN MODE EXCEPTION note now names $GSTACK_STATE_ROOT/analytics/ instead of
the ~/.gstack default.
Allowlist: drops the generate-completion-status.ts PENDING row.
Pin (W7a): gen-skill-docs 'render documents the analytics sink' now checks
that the render names the state-root analytics dir.
* chore: regenerate SKILL.md, ship goldens and context budget after telemetry prose state-root change
* fix(state-root): gbrain context_queries globs use {gstack_state_root}/
bin/gstack-brain-context-load.ts resolves the {gstack_state_root}/ prefix
through gstack-paths (configuredStateGlob); the legacy ~ form only matched the
default root. The nine projects/analytics manifest globs in design-consultation,
design-shotgun, investigate, office-hours, plan-ceo-review and retro now use the
prefix, as does office-hours' builder-profile developer-profile.json glob
(plan-tune writes it under $GSTACK_STATE_ROOT).
Allowlist: drops the nine MANIFEST_GLOB rows and the now-unused MANIFEST_GLOB
and PENDING reasons; the header lists the remaining row kinds.
Tests: new brain-context-load case loads the shipped investigate and
office-hours queries against a configured root and proves a file only under
~/.gstack is not read.
* chore: regenerate SKILL.md and context budget after manifest state-root globs
* plan-tune, office-hours, retro: read gate and profile files from the state root (E1 follow-up)
* chore: regenerate plan-tune, office-hours, retro
* release: v1.91.15.0 — Opus 5.5 prompt cleanup, behavior-level skill tests, transcript consent
* test: calibration-gate scan reads tracked files with POSIX paths (windows-free-tests)
* test: skillify E2E sweeps the state root (GSTACK_HOME) where the global tier actually writes
* qa-only: one rule for mixed-run clocks; annotations and materialize are per probe directory
* chore: regenerate qa-only
* test: shared-libs evals deliver the skill instructions in context, as a loaded skill does
* Revert design-doc discovery state-root conversion: it made plan-eng-review ask its scope gate in prose (plan-mode no-op eval)
* chore: regenerate after design-doc discovery revert
* test: plan-review-report prompt asks plainly to save the review to plan.md
* docs: CHANGELOG and TODOS reflect the held design-doc discovery conversion
* docs: exact eval counts for the held design-doc conversion
* test: coverage-audit grader accepts read-only git diff with a -- pathspec (CI run 37094035231 shape)
* qa: plain self-regulation stop rule; drop the report template's link to an index.md nothing writes
* chore: regenerate qa
* qa: --quick sets both tier and exploration; say how browser phases and functional contracts share the probe loop
* chore: regenerate qa
* test: a bare QA helper invocation is a usage probe, like --help (CI run ship-exploratory-unavailable)
* fix: acknowledge seeded plans before invoking review skills
* fix: distinguish current plan input from conversation history
* fix: keep hermetic plan reviews on manual permissions
* fix: distinguish tool discovery from file permission ownership
* fix: preserve initial plan mode in observation tests
* fix: wait for scope decisions before writing review findings
* fix: carry autoplan decisions consistently into review artifacts
* test: retain native failure context in periodic assertions
* fix: advance active file permissions before queued questions
* fix: finish red-team attempts before retry and cleanup
* fix: finalize plan format captures and judges before retry
* fix: cancel setup-gbrain SDK attempts before fixture cleanup
* test: select periodic consumers of the bounded attempt helper
* fix native Bash permission cards and queued questions
* fix: preserve independent decisions and review scope
Keep CEO approach, engineering scope and outside-review choices from approving independent remedies together. Carry declared contracts through DX polish and resolve new gaps before editing the plan. Regenerate every host and retain existing stop boundaries.
Validation: 654 focused tests passed across nine files; all-host generation passed. Full free and periodic validation pending.
Co-Authored-By: OpenAI Codex <noreply@openai.com>
* fix: require approval before design plan amendments
Align the Design review philosophy and rating recipe with its section protocol: resolve one proposed fix, then apply only that approved decision and retain honest scores for declined fixes.
Validation: 469 focused tests passed across four files; all-host generation passed.
Co-Authored-By: OpenAI Codex <noreply@openai.com>
* fix: observe native question completion before transcript persistence
Match owned completion hooks to submitted choices, reject conflicting or late answers, and retain bounded failure evidence.
* test: recognize review posture in acknowledged native questions
Require the selected mode acknowledgement, a completed follow-up question, and its current decoded display while preserving existing posture assertions.
* fix: preserve settled CEO choices and isolate pending remedies
Resolve established approach gates with cited authority and keep independent fixes out of unrelated option commitments and plan amendments.
* fix: carry approved DX choices through later review steps
Choose documentation approaches within the accepted scope and map resolved confusion points without reopening them through a bulk menu.
* test: handle native settings-file edit prompts
Keep one-time owned-file approvals and retain the actual sampled Autoplan permission frame with its matching barrier state.
* test: accept standard CEO reply directives with tuning footers
Recognize the exact trailing preference footer and letter-list directive while preserving current-display and exact acknowledgement checks.
* test: scope split reviewers to their generated plan artifacts
* test: observe native Bash permissions and invocation results
* test: handle owned Bash prompts during mode preference checks
* test: preserve synchronous subprocess rejection in Codex fixture
* Fix periodic review handoff navigation
Recognize review-first and explicit manual-next-step labels while preserving exact action families, manual preference, and ambiguous-menu rejection.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Bind pending file permissions to distinct current targets
Allow one captured file request to own the complete current dialog while unrelated file work is pending. Preserve same-path ambiguity, exact input ownership, and one-time grant checks.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Make paired CEO verification choices genuinely unresolved
Start the positive control with proposed manual checks so its unchanged oracle measures two new coverage decisions. Preserve runtime contracts, targets, count bounds, and all assertions.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Keep CEO review options and verification within approved scope
Audit every offered option for independent add-ons and keep new verification depth pending until accepted. Preserve already requested coverage and trace plan changes to the actual decision.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Assemble DX review artifacts before appending the final report
Keep early DX evidence above decisions, update artifact sections in place, and append the report using the actual current file suffix. Re-read after deleting an existing report before choosing the append anchor.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Keep outside plan reviews exclusive and invocation-owned
Follow one preflight-selected backend, terminate failed Codex work before fallback, and allocate extra prompt/output files uniquely. Consume only the current invocation’s completed output.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Select periodic completion evaluations for report writer changes
Register the shared review resolver for eight missing consumers and regress selection for all nine completion cases without changing their IDs or tiers.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Keep permission ambiguity fixtures on the same normalized target
Use distinct raw spellings of one target in the four negative fixtures so they exercise the normalized duplicate-owner guard after exact current-file disambiguation. Preserve the existing exception, no-input, diagnostic and cleanup assertions.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Clarify preserved contracts in engineering review fixture
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Recognize the offered DX follow-up handoff
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Check independent commitments before presenting review options
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Keep Codex review output and status in one shell invocation
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Distinguish seeded plans from reports written by a test attempt
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Recover clipped Autoplan file approvals with bounded viewport resizing
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Recover clipped Bash approvals before binding the complete command
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Isolate setup message tests from the shared checkout
Run the real installer in a temporary payload with private config, require successful completion, and guard source and binary contents and mtimes.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Fix periodic native permission and report completion handling
Match the pinned CLI's soft wraps and clipped headings without granting from incomplete frames. Retire completed file requests, retain mode annotations, and ask section captures for a short final acknowledgement after their full report is saved.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Preserve review approvals and validate DX comparison artifacts
Keep independent remedies and approved amendments explicit. Give the synthetic DX review its existing documentation and validate peer comparison as required analysis alongside four native decisions. Add positive and negative semantic calibrations while preserving review counts, model budgets and prompt size limits.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Make the five-finding CEO fixture's application boundary explicit
Materialize the request adapter and service composition used by the synthetic payment application. Explicitly declare the revised unregistered-event and mail-telemetry assumptions while preserving uncaught handler errors, the original invoice path and all five unresolved findings.
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Keep CEO state-path checks scoped to directory preparation
Co-authored-by: OpenAI Codex <noreply@openai.com>
* Use checked ports and bounded cleanup in pair-agent tests
Discover the daemon port from its owned state file, retain startup diagnostics, and await failed-start cleanup. Add occupied-port, early-exit, deadline, and foreign-state regressions while preserving the existing HTTP assertions and hook budgets.
Co-authored-by: Codex <noreply@openai.com>
* Preserve queued edit identity and recover clipped Bash permissions
Distinguish separately queued unfinished edits from mutation of one native tool ID. Keep grants bound to an exact owned request and reject reused IDs, ambiguous inputs, and competing owners.
Support the pinned renderer's literal em dash and request a repaint when only the Bash card's top rule is clipped. Grants still require the complete fresh card and an exact native acknowledgment.
Validation: 413 integrated parser/event tests passed; private repaint controls and joint source review passed. Full canonical suite and native periodic rerun remain pending.
Co-authored-by: Codex <noreply@openai.com>
* Keep periodic reviews within their approved contracts and deliverables
Carry exact approvals through engineering review, preserve declared contracts when amending CEO plans, and keep prioritization at the requested decision level. Materialize the revised synthetic SDK reference contract while retaining the five original documentation gaps.
Accept the observed semicolon in the finite DX handoff menu and register the direct source dependencies used by the engineering cases. Regenerate canonical review documents without changing model budgets, retries, count bands, or native completion assertions.
Validation: all-host generation and 275 review, fixture, selection and parity tests passed. Full free-suite and native periodic validation remain pending.
Co-authored-by: Codex <noreply@openai.com>
* Keep Eng approval cadence and independence guards explicit
* Accept ordinary punctuation in manual review handoffs
* Recover file permissions alongside queued Bash calls
* Carry approved DX work through later review findings
* Clarify the synthetic auth internal failure decision
* Bound the periodic DX fixture to onboarding changes
* Recognize native Design review handoff labels
* Hold scope in the integration-choice review fixture
* Carry approved Design decisions through review evidence
* Capture listener state when feedback reload fails
* Exclude workspace caches before checking deprecated flags
* Verify Design UI scope against a seeded review plan
* Clarify plan review decisions and outside-voice approval flow
* Reject setup menus in the Design UI gate
* docs: require focused repair validation before final acceptance
* fix: separate review commitments within existing prompt budgets
* docs: align generation and contributor validation guidance
* fix: advance native review prompts and count acknowledged findings
* chore: bump version and changelog (v1.87.1.0)
Co-Authored-By: OpenAI Codex <noreply@openai.com>
* chore: enforce cheap checks and side-effect-free validation previews
* fix: handle owned Fetch permissions and oversized native cards
* test: ground review fixtures in independent executable contracts
* fix: preserve review decisions and verify reports before completion
* test: construct the synthetic credential URL without a scanner false positive
* test: materialize DX examples and verify their actual local behavior
* fix: clarify CEO review decisions and execution order
* fix: clarify review workflow ordering and select Design quality checks
* Fix review decision gates and incomplete evaluation fixtures
Persist CEO and engineering commitment ledgers before menus, preserve exact
approvals, and distinguish implementation structure from feature scope.
Route Autoplan through the canonical CEO Step 0 ordering. Classify DX findings
before requesting approval and ground runtime claims in actual evidence.
Complete neutral non-target fixture contracts and accept the captured Design
handoff purpose without relaxing its ownership or acknowledgment checks.
Record runtime-capability verification in AGENTS.md validation discipline.
Validation: 1,335 focused tests passed across 21 files; build, all-host freshness,
skill validation (647 artifacts / 107 tracked), and credential checks passed.
Prior paid failures are preserved; behavioral acceptance remains pending.
* Fix review decision boundaries and owned Read prompts
Preserve exact approvals across review options, compare consistent DX milestones,
and keep proposed implementation separate from review evidence. Bind modern
Read prompts to one immutable native request and wait for its result.
Retain captured regression verdicts, correct fixture error names, improve import
probe diagnostics, and record focused-first validation discipline in AGENTS.md.
* Clarify CEO and engineering review decisions
Use explicit decision steps, one engineering ledger, and clear scope/write transitions. Preserve exact approvals and distinguish pending test requirements. Keep unrelated generated content unchanged.
* Fix review decision ordering and native evaluation interactions
* Clarify engineering decisions and test artifact order
* Clarify pending choices and approvals in CEO reviews
* Make CEO review phases sequential and clarify completion
* Fix Design board submission intent matching
* Seed an existing browser test baseline for Autoplan
* Document decision-log payloads before state initialization
* Preserve exact review scope and decide one change before drafting options
* Require input identity before repeating passing model judges
* Honor permitted storage throughout CEO review completion
* Match complete native permission text within the pinned renderer contract
* Align review approvals, independent choices, and bounded validation
* fix: preserve reopened approvals and declare fixture interfaces
* fix: isolate review artifacts and audit complete questions
* fix: match detector artifact permissions to configured storage
* fix: complete native permissions and review fixture workflows
* fix: order CEO review work and separate engineering guarantees
* fix: preserve native validation and separate review choices
* fix: clarify review decisions and judge complete report context
* fix: constrain review judgments and retain parse failures
* fix: compare each affected value before review decisions
* fix: make engineering review decisions and completion order explicit
* fix: give the complete Autoplan evaluation a bounded chain budget
* fix(cso): diagnose forbidden Docker endpoints before tool lookup
* fix(reviews): reconcile workflow contracts and generated artifacts after main integration
* fix(evals): migrate retained regressions to the native review harness
* fix(tests): close native harness and workflow integration regressions
* fix(evals): preserve complete permission context and native menu contracts
* fix(tests): capture synchronous command output without pipe drain stalls
* fix(reviews): clarify decision and completion ordering
* fix(reviews): separate decision readiness from final completion checks
* refactor(reviews): consolidate decision rules and completion branches
* fix(plan-eng-review): order preparation and clarify decision routing
* fix(plan-eng-review): restore size and question-format guard parity
* fix(plan-eng-review): clarify scope phases and blocked completion
* fix(plan-eng-review): unify review flow and report destination
* fix(plan-eng-review): define bootstrap and question stage ownership
* fix(plan-eng-review): clarify review structure and design lookup
* fix(plan-eng-review): render report examples and show saved decisions
* fix: consolidate Eng review decisions and select their evaluations
* test: cover overlapping terminal attachments and clean merged runner type
* fix: preserve Office Hours relationship closings during review updates
* fix: retain pasted review targets across slash invocations
* docs: preserve validation traces and correct release scope
* test: cover pasted targets in both review skills
* fix: validate report artifacts before recording success
* fix: redact source roots at CSO report boundaries
* fix: bind native Design questions before answering
* test: select report privacy and native recovery regressions
* test: bind rejection predicate in extracted observers
* fix: bind complete boxed native questions
* test: keep the Design UI fixture on native review
* fix: preserve review decisions and evaluation completion outcomes
* fix: clarify CEO approval and report completion order
* fix: align native review evaluation ownership and completion
* fix: bind review evaluators to native decisions and owned artifacts
* fix: validate review decisions against native outcomes
* fix: preserve review evidence and Autoplan phase handoffs
* test: bind review evidence to owned decisions and completion
* fix: retain owned native history across compaction
* fix(evals): validate current review decisions and setup choices
* fix: bind Autoplan reviews and phase completion to current amended input
* fix: reconcile native review evidence and close Autoplan phases
* test: recognize owned whole-candidate complexity decisions
* test: preserve report freshness for approved investigation handoffs
* fix: recognize scoped review findings and isolate dual voice fixtures
* fix: make review handoffs and question dispatch self-contained
* test: recognize complete CEO decisions and procedural pauses
* fix: bind current CEO comparison options and risk intervals
* test: bind engineering decisions and completion to owned evidence
* fix: publish Autoplan phase reports before continuing tools
* test: verify actual Autoplan dual-review dispatch evidence
* test: select dual review when shared evidence fixtures change
* fix: clarify plan review decisions and completion gates
* fix: make CEO review decisions and return paths explicit
* test: keep Autoplan prompt files inside attempt state
* test: preserve source whitespace across permission dialog wraps
* fix: publish Autoplan phase reports before continuing
* test: recognize current CEO comparisons and reject inactive records
* fix: reconcile engineering decision states before completion
* test: recognize complete Design decisions and reports
* test: verify current engineering decisions before navigation
* Recognize source-owned component reduction choices
* fix: recognize current CEO ledger and commitment grids
* test: supply RequestPolicy context to Eng count fixture
* fix: save complete engineering decisions before asking
* fix: bind Autoplan publication to the complete phase readback
* chore: prepare 1.87.5.0 reliability release
* fix: clarify engineering review completion and preserve log failures
* fix: bind CEO saved choices and current section ancestry
* fix(evals): bind review execution and completion evidence
* fix(plan-ceo-review): verify complete decisions before asking
* fix(evals): preserve complete engineering choice records
* fix(evals): preserve complete review outcomes and bounded fixtures
* fix(autoplan): publish phase reports before advancing
* fix(plan-ceo-review): validate option fields before asking
* fix(plan-eng-review): verify current decisions after answers
* fix(evals): bind review decisions and bound fixture scope
* fix(plan-ceo-review): verify decision rows and edit saved checkpoints
* fix(evals): bind review evidence and scope document lookup
* fix(plan-eng-review): update resolution state with its answer
* fix(reviews): preserve complete questions through dispatch
* fix(evals): recognize completed mode declarations
* fix(evals): define cache consistency at wrapper completion
* fix(evals): validate owned initial scope and completed review handoffs
* fix: assemble complete CEO decision fields before saving
* fix: authenticate automatic mode decisions without guessing selectors
* fix: bind engineering coverage to approved regression contracts
* fix(evals): supply review helpers to native Eng capture
* fix(plan-eng-review): preserve the full selected option scope
* fix(evals): recognize owned engineering seed and regression evidence
* fix(evals): bind engineering retry reports to native approvals
* docs: clarify release guarantees (v1.87.5.0)
Co-Authored-By: OpenAI Codex <noreply@openai.com>
* fix(evals): recognize owned engineering decisions and handoffs
* fix(evals): bind engineering decisions and completion evidence
* fix(tests): align review contracts and selection fixtures
* fix(skills): restore review prompt size limits
* fix(plan-eng-review): clarify review execution and completion
* fix(evals): preserve configured retries through all supervision layers
* Clarify Engineering decisions and report completion
* Keep native decision assertions within their source boundary
* fix: recognize owned engineering decisions and completed navigation
* fix: bind completed auto decisions to their current review
* fix: recognize explicit CEO source attribution
* fix: dispatch verified CEO decisions without recomposing fields
* test: expose existing execution deadlines to review actors
* fix: distinguish CEO decision records from incidental headings
* test: bind split-scope choices to the registered native actor
* test: connect reviewed regressions to required evaluation coverage
* Clarify CEO decision routing and completion stages
* test: expose existing section review deadlines to fixture actors
* test: recognize complete native CEO pacing inventories
* test: exclude answered history from current CEO payloads
* test: detect phase entry through owned skill HOME aliases
* test: validate native review completion and owned report permissions
* fix: make Autoplan close packets carry the parent handoff steps
* test: assess source-bound HOLD decisions within the existing deadline
* fix: keep CEO native decision fields under one formatting authority
* test: register integrated review and permission dependencies
* test: align native review adapters and finding coverage
Preserve explicit AUTO decisions, apply native single-select defaults, and bind complete cropped questions and report permissions to their owned requests. Require seeded review findings instead of crediting setup menus.
Keep captured failure controls and additive selection dependencies. The integrated candidate passed 3,099 focused tests across 65 files; affected paid validation remains required before publication.
* fix(autoplan): require phase reports before advancing
* fix(evals): bind setup and evidence to complete attempts
* fix(evals): bind native answers and pending writes to fixture scope
Preserve complete option rows when native descriptions wrap, retain current
owned Write arguments before journal publication, and keep engineering and
DX answers within their declared fixture interfaces. Add captured free
regressions without increasing model budgets or relaxing completion checks.
* fix(autoplan): verify phase reports across native tool paths
Guard owned methodology reads and reviewer dispatches, detect complete driver
loads through Bash, and distinguish report-only edits from implementation
changes. Follow authenticated native UUID ancestry when journal writes arrive
out of order and verify earlier native content for cached phase reads.
Keep current close acknowledgment and parent publication in order, require CEO
entry before later phases, and register captured failure regressions.
* fix(evals): honor native input and collection lifecycles
Match complete native Edit panes and truncated question borders, reject stderr close before EOF, and stop the CEO split fixture once its acknowledged scope decisions are collected. Keep semantic validation, process failures, report requirements, and absolute deadlines authoritative.
Add captured-event and real-process regressions with selection dependencies. Focused checks pass; final integrated paid and full-suite acceptance remain pending.
* fix(autoplan): retain native session ownership across directory changes
Recover missed native UUID ancestry through the existing strict graph while preserving ordinary event order and legacy scoping. Bind publication hooks to Claude's original project directory while retaining current cwd for requested file paths.
Captured public-event regressions, existing caller checks, and a pinned native CLI loopback verify both fixes. Preserve failed attempts and require fresh paid and final full-suite acceptance.
* docs: align evaluation limits and completion version
* fix(autoplan): allow authenticated phase reads during journal streaming
* fix(evals): bind clipped native questions and owned edit dialogs
* fix: preserve overlay retries and bounded cleanup
* fix: recognize owned planning preludes in native questions
* docs: explain overlay scheduling and cleanup guarantees
* fix: require fresh publication after Autoplan phase reruns
* Release gstack 1.87.6
* fix: preserve CI paths, process identity, and test deadlines
* fix: keep informational setup commands independent of install probes
* fix: clarify plan review decisions and bound source audit reports
* Fix remaining Windows identity and native path CI failures
* Clarify CEO review decision and reviewer-result routing
* test: accept no-install planner in retry supervision
* fix(ceo-review): make review decisions and report completion explicit
* perf(test): add fast PR gates, input-keyed judge reuse and isolated free shards
* fix(test): start isolated CEO smoke from its existing project plan
* fix(test): repair CI fixture races and preserve retry evidence
* fix(ceo-review): clarify approvals, depth and saved completion
---------
Co-authored-by: OpenAI Codex <noreply@openai.com>
Garry Tan