Commit Graph
610 Commits
Author SHA1 Message Date
garrytan 50ec7c57dd docs: CHANGELOG and TODOS reflect the held design-doc discovery conversion 2026-10-03 04:32:56 +00:00
garrytan 11c8a4eb2b test: plan-review-report prompt asks plainly to save the review to plan.md 2026-10-03 04:32:44 +00:00
garrytan 6b76fdd381 chore: regenerate after design-doc discovery revert 2026-10-03 04:32:32 +00:00
garrytan fbc0c15aaf Revert design-doc discovery state-root conversion: it made plan-eng-review ask its scope gate in prose (plan-mode no-op eval) 2026-10-03 04:32:31 +00:00
garrytan 35d4efdc01 test: shared-libs evals deliver the skill instructions in context, as a loaded skill does 2026-10-03 03:41:05 +00:00
garrytan a686a40ba0 chore: regenerate qa-only 2026-10-03 03:10:39 +00:00
garrytan 93b6e11fe9 qa-only: one rule for mixed-run clocks; annotations and materialize are per probe directory 2026-10-03 03:10:39 +00:00
garrytan 8f34e6eb99 test: skillify E2E sweeps the state root (GSTACK_HOME) where the global tier actually writes 2026-10-03 03:06:11 +00:00
garrytan f4cfae6661 test: calibration-gate scan reads tracked files with POSIX paths (windows-free-tests) 2026-10-03 02:48:45 +00:00
garrytan ff245b11a9 release: v1.91.15.0 — Opus 5.5 prompt cleanup, behavior-level skill tests, transcript consent 2026-10-03 02:34:59 +00:00
garrytan 6c2e9a70a0 chore: regenerate plan-tune, office-hours, retro 2026-10-03 02:34:00 +00:00
garrytan eb768a55dd plan-tune, office-hours, retro: read gate and profile files from the state root (E1 follow-up) 2026-10-03 02:34:00 +00:00
garrytan 7eb343904d Merge remote-tracking branch 'origin/garrytan/opus55-wip-G' into garrytan/opus-5-5-prompt-audit 2026-10-03 02:31:37 +00:00
garrytan 4d371a98ad chore: regenerate SKILL.md and context budget after manifest state-root globs 2026-10-03 02:21:06 +00:00
garrytan 44e9c1ed3a fix(state-root): gbrain context_queries globs use {gstack_state_root}/
bin/gstack-brain-context-load.ts resolves the {gstack_state_root}/ prefix
through gstack-paths (configuredStateGlob); the legacy ~ form only matched the
default root. The nine projects/analytics manifest globs in design-consultation,
design-shotgun, investigate, office-hours, plan-ceo-review and retro now use the
prefix, as does office-hours' builder-profile developer-profile.json glob
(plan-tune writes it under $GSTACK_STATE_ROOT).

Allowlist: drops the nine MANIFEST_GLOB rows and the now-unused MANIFEST_GLOB
and PENDING reasons; the header lists the remaining row kinds.
Tests: new brain-context-load case loads the shipped investigate and
office-hours queries against a configured root and proves a file only under
~/.gstack is not read.
2026-10-03 02:21:06 +00:00
garrytan 3215981a2e chore: regenerate SKILL.md, ship goldens and context budget after telemetry prose state-root change 2026-10-03 02:19:26 +00:00
garrytan d7e9f0b501 fix(state-root): completion-status telemetry prose names $GSTACK_STATE_ROOT/analytics
gstack-skill-end writes analytics under the resolved state root ($_GH), so the
PLAN MODE EXCEPTION note now names $GSTACK_STATE_ROOT/analytics/ instead of
the ~/.gstack default.

Allowlist: drops the generate-completion-status.ts PENDING row.
Pin (W7a): gen-skill-docs 'render documents the analytics sink' now checks
that the render names the state-root analytics dir.
2026-10-03 02:19:26 +00:00
garrytan fc0aacd080 chore: regenerate SKILL.md and context budget after taste-profile state-root glob 2026-10-03 02:18:36 +00:00
garrytan 10d8c1c027 fix(state-root): taste-profile legacy fallback globs $GSTACK_STATE_ROOT designs
The legacy approved.json fallback in {{TASTE_PROFILE}} (design-consultation,
design-shotgun) named the literal ~/.gstack/projects path while the approval
writers use $GSTACK_STATE_ROOT/projects/$SLUG/designs.

Allowlist: drops the design.ts PENDING row.
Pin (W7a): design-consultation-contract's exact glob string becomes a
meaning-level check (legacy fallback names the state-root designs dir and
approved.json, and no ~/.gstack literal).
2026-10-03 02:18:36 +00:00
garrytan b7a1d77050 chore: regenerate SKILL.md, ship goldens and context budget after history/ledger state-root paths 2026-10-03 02:17:50 +00:00
garrytan 1fdda72ab9 fix(state-root): history, report and ledger paths name $GSTACK_STATE_ROOT
Agent-facing read/write paths that named the literal ~/.gstack default now
name the state root their writers use:
- canary-history.jsonl and health-history.jsonl (the blocks above already
  resolve the root and print PROJECT_DIR)
- ios-design-review report: adds {{SLUG_SETUP}} so the report path is the
  PROJECT_DIR it prints
- plan-tune dream-cycle gate's distillation-proposals.json
- retro's skill-usage and eureka reads: read the path gstack-retro-metrics
  prints (it already checks $_gstack_sr_root/analytics)
- ship's apple-effects.log durable-effect ledger and the gstack-evidence
  ledger description (bin/gstack-evidence writes under resolveStateRoot())

Allowlist: drops the nine matching PENDING rows from STATE_ROOT_ALLOWLIST.
2026-10-03 02:17:50 +00:00
garrytan 3c3a013a3b chore: regenerate SKILL.md and context budget after design-doc discovery state-root change 2026-10-03 02:14:37 +00:00
garrytan 569cdae04f fix(state-root): design-doc discovery reads $GSTACK_STATE_ROOT/projects
DESIGN_DOC_DISCOVERY and autoplan's Step 2 design-doc lookup read the literal
~/.gstack/projects path, so under a non-default state root they missed the
docs office-hours writes. The block now globs "$GSTACK_STATE_ROOT"/projects,
and each caller's bash block (autoplan, plan-ceo/devex/eng-review, the
spec-review prerequisite recheck) resolves the root with the guarded
gstack-paths eval first.

Allowlist: drops the autoplan ls row, both _LOCALDOC rows and the resolver
source-comment row from STATE_ROOT_ALLOWLIST.
Tests: the eng Design Doc Check fixture installs gstack-paths and seeds docs
under a non-default root; the CEO discovery fixture passes an explicit
GSTACK_STATE_ROOT.
2026-10-03 02:14:37 +00:00
garrytan 84253fc5f4 docs: TODOS for the Opus 5.5 prompt-cleanup deferrals 2026-10-03 02:08:35 +00:00
garrytan f7958d8d5d test: office-hours-auto-mode joins the fast PR profile (replaces the retired office-hours-spec-review probe) 2026-10-03 02:06:49 +00:00
garrytan 5d44196e80 chore: constants comment names the /ship documentation dispatch, not a stale step number 2026-10-03 02:05:57 +00:00
garrytan 39e670035a test: integration repairs after F2 (calibration test renders the AUQ format; census counts and release floors without the reverted overlay case) 2026-10-03 02:05:39 +00:00
garrytan 7ec098f9a1 Merge remote-tracking branch 'origin/garrytan/opus55-wip-F2' into garrytan/opus-5-5-prompt-audit 2026-10-03 02:04:15 +00:00
garrytan 727481797e test(E1): pin CLAUDE.md facts, /ship step references and state-root literals
New free test test/instruction-facts.test.ts (CEO/DX/Eng E1 items; DX-13 one
named test() per check, each failure prints both values, file:line and the
fix):

- 'CLAUDE.md names every TREE_MUTATING file and does not call the set
  empty': imports TREE_MUTATING from scripts/test-free-shards.ts (today
  test/bootstrap-retention.test.ts).
- 'CLAUDE.md states the current catalog ceiling from
  test/catalog-budget.test.ts': the "`CATALOG_BUDGET_TOKEN_EQUIVALENTS`
  (N today" value must equal the constant (1,194).
- 'CLAUDE.md points at package.json for detach timeouts instead of stating
  numbers': no eval:bg* --timeout value appears in CLAUDE.md (plain or with
  commas), and the detach section names package.json's eval:bg:gate /
  eval:bg:periodic and states no timeout number.
- 'every explicit /ship Step reference names a defined /ship step with the
  expected title': matches only `/ship Step N` and `/ship's Step N`
  (backticked or not) across every .tmpl and scripts/resolvers/; each must
  be a table row whose step is a heading in ship/SKILL.md.tmpl or
  ship/sections/*.md.tmpl whose title contains the row's subject, and the
  reference's line or enclosing heading must mention that subject; stale rows
  fail. Today: document-release release-body Step 7 -> /ship Step 14
  (TODOS.md), plus the incident-history comment in
  scripts/resolvers/constants.ts ("/ship Step 18"), kept as a historical row
  with a reason (handoff: that comment's step is stale; the dispatch is Step
  14.5).
- 'negative control: moving a reference to another existing /ship step
  fails': rewrites the TODOS reference to Step 12 in memory and asserts the
  checker reports it.
- 'every ~/.gstack/projects/ and ~/.gstack/analytics/ hit is allowlisted with
  a reason' and 'every state-root allowlist row still matches a hit': all 37
  current hits in .tmpl and scripts/resolvers/ are covered by 36 rows, each with one of four
  reasons (user-facing description; gbrain context_queries manifest glob read
  literally by bin/gstack-brain-context-load.ts; source comment; pending
  conversion tracked in TODOS.md). The readiness-gate EVAL_DIR rows reuse the
  reason from test/state-root-ratchet.allowlist.json. New hits fail, and
  converting a literal fails until its row is deleted.

Mutation pass (scratch worktree): 15/15 as expected; drifting the catalog
number in CLAUDE.md or the constant, dropping or emptying TREE_MUTATING text,
adding a TREE_MUTATING key, copying a detach timeout into CLAUDE.md,
moving/undefining/retitling a /ship reference, a new /ship reference, a new
state-root literal and a converted allowlisted literal all go red; unrelated
rewording stays green. Log: .context/mutation-instruction-facts.md.
2026-10-03 01:54:28 +00:00
garrytan 4819d8e2b8 test: update the periodic release floor for the opus-5-5 overlay case
paid-retry-supervision.test.ts 'detached PR fallback and release commands
cover their actual default worker budgets' was red on the integrated base
23a0be82d (green on f30b7b7): 737bf478d (E2, "dedicated-tools overlay case
pinned to claude-opus-5-5") added a periodic file, so the recomputed periodic
release floor is 33,852 s, not the pinned 33,821 s. Updates the pinned floor
and the two-tier sum (55,546 -> 55,577 s); the detached release wall
(116,700 s) still exceeds the recomputed census wall plus 120 s, which the
same test asserts, so package.json does not change. eval-detach-timeout-floor
stays green.
2026-10-03 01:49:53 +00:00
garrytan b27c77c8bc test(W7h): delete incompleteFirstFileView, which no paid case uses
W7h retirement (task scope: "incompleteFirstFileView + its free test in
test/shared-libs-fixture.test.ts if no paid case uses it"). `grep -rn
incompleteFirstFileView test scripts` showed only the export in
test/helpers/shared-libs-eval-fixture.ts and its own free describe block;
no paid case or grader calls it, so the helper and the two tests that pinned
it are dead code.

Retirement card:
- test: shared-libs-fixture.test.ts 'incompleteFirstFileView (census
  36641820398 shared-libs-pr-coverage)' (2 tests).
- detects: regressions in a page-1 recovery classifier nothing consumes.
- non_test_callers: none.
- replacement: none needed; the shared-libs page budgets that a recovery read
  is charged to stay enforced by the kept budget function and its tests.
- validation: bun test test/shared-libs-fixture.test.ts (241 pass);
  bun run typecheck.
2026-10-03 01:49:29 +00:00
garrytan 1f3e192098 test(W7h): drop the no-op review-report check from setup-gbrain-remote
W7h retirement (t12 F9) of one assertion in the paid setup-gbrain-remote
case. No registration changes (the case stays).

Retirement card:
- assertion: `expect(/## GSTACK REVIEW REPORT|critical_gaps/i
  .test(result.output)).toBe(false)` with its comment.
- detects: /setup-gbrain writing a plan-review report or critical_gaps
  findings, which it never produces; the removed comment itself said
  "Setup-gbrain doesn't have a review report contract". Copied from the
  plan-review tests (c118e24, #2572).
- non_test_callers: none.
- replacement: none needed; `grep -rln "PLAN_FILE_REVIEW_REPORT\|GSTACK REVIEW
  REPORT\|critical_gaps" setup-gbrain/*.tmpl setup-gbrain/sections/*.tmpl`
  is empty, so there is no behavior to control. The case's real contracts stay:
  the secret token never reaches CLAUDE.md, `mcp add ... --transport http`,
  and `Mode: remote-http` (read by bin/gstack-gbrain-detect).
- validation: bun test test/skill-e2e-setup-gbrain-remote.test.ts (skips
  without EVALS; module loads).
2026-10-03 01:48:49 +00:00
garrytan 56e4fa201a test(W7h): delete validateOfficeHoursSpecSummary left unused by the retirement
Dead-code follow-up to the office-hours-spec-review retirement:
validateOfficeHoursSpecSummary in test/helpers/office-hours-completion.ts graded
only that paid case's model-written summary. Deletes it and its free
describe block in test/office-hours-completion.test.ts ('office-hours
spec-review summary completion'); every other validator and test in both
files is unchanged.

Retirement card:
- detects: a summary missing five dimensions, the Agent dispatch, the 3
  iteration cap or the four metrics.
- non_test_callers: none.
- replacement: the facts it graded are pinned free on the generated text by
  gen-skill-docs.test.ts 'SPEC_REVIEW_LOOP resolver' (round-3 cap and loop
  wiring); see the retirement commit's control.
- validation: bun test test/office-hours-completion.test.ts.
2026-10-03 01:48:29 +00:00
garrytan ca395095a1 test(W7h): delete the codex-offering prompt builder left unused by the retirement
Dead-code follow-up to the codex-offered-* retirement: buildCodexOfferingPrompt
(test/helpers/codex-offering-fixture.ts) had no caller besides those cases.
Deletes the helper, its free test (test/codex-offering-fixture.test.ts), the
two captured public fixtures it replayed, its scripts/free-test-durations.json
entry, and the helper/fixture paths still listed as touchfiles of six plan
review E2Es in test/helpers/touchfiles-data.ts.

Retirement card:
- test: codex-offering-fixture.test.ts (pinned the builder's prompt text and
  replayed captured timeout/CDD transcripts against it).
- detects: changes to a prompt that no paid case sends any more.
- non_test_callers: none (grep -rn codex-offering test scripts is empty).
- replacement: none needed; the behavior it supported is now the free probe
  check in gen-skill-docs.test.ts.
- validation: bun test test/touchfiles.test.ts test/touchfiles-map-diff.test.ts
  test/paid-shards.test.ts test/e2e-tier-alignment.test.ts; bun run typecheck.
2026-10-03 01:48:29 +00:00
garrytan fc9049a8b5 test(W7h): retire the paid SKILL.md-summary cases in skill-e2e-plan
W7h retirement (t14 F6-F8): office-hours-spec-review, plan-ceo-review-benefits
and codex-offered-{office-hours,ceo-review,design-review,eng-review}. Each
asked a model to read a generated SKILL.md, write a summary, and grepped the
summary for facts the generated file states directly.

Registration cleanup in the same commit: E2E_TOUCHFILES, E2E_TIERS (all six
gate) and E2E_KINDS (rule) in test/helpers/touchfiles-data.ts; six
scripts/paid-test-durations.json case-shard entries; no
test/helpers/periodic-exclude-data.ts entries existed. The fast PR profile
(scripts/test-pr-profile.ts) drops plan-ceo-review-benefits and
office-hours-spec-review from PR_PROFILE_CASE_IDS and PR_PROFILE_FILES. Tests
that named the retired ids as live examples move to existing ones:
paid-free-boundary and paid-pr-profile (a CEO template change now plans
auq-format-gate plus the CEO judge; the case-shard fixture uses
plan-review-report), test-pr-profile, touchfiles (CEO selection count
21 -> 19), the touchfile-moved-code w5 golden (retired ids removed), the
behavior-lane case-shard census (78 -> 72) in cookie-validation-phases, and
the skill-e2e-plan timeout list and full-gate PR floor (78,425 -> 72,755 s)
in paid-retry-supervision. e2e-tier-alignment, eval-kinds,
eval-detach-timeout-floor and paid-shards stay green.

Coverage consequence: per the Eng Retirements item, plan-ceo-review-benefits
was the only gate-tier CEO E2E in the fast PR profile, so CEO invariants are
treated as periodic-only under the W7c gate (every CEO conversion in this
branch keeps one free check per invariant); the gate judge
'plan-ceo-review/SKILL.md modes' still runs for CEO prompt changes. An
office-hours prompt change now has no fast PR E2E and is reported as
deferred to broad coverage rather than run.

Retirement cards:
- office-hours-spec-review: detects the Spec Review Loop's stated dimensions,
  dispatch tool, iteration cap and metrics. Replacement: gen-skill-docs.test.ts
  'SPEC_REVIEW_LOOP resolver' (round-3 cap, prepare/check/finalize commands,
  verdict and round-file wiring) and test/office-hours-review.test.ts (verdict
  parsing). Control: changing the round limit in
  scripts/resolvers/spec-review.ts turns gen-skill-docs red.
- plan-ceo-review-benefits: detects the CEO /office-hours prerequisite and
  mid-session offer. Replacement: gen-skill-docs.test.ts 'plan-ceo-review
  offers /office-hours when no design doc exists and detects a lost user
  mid-session' (previous commit). Control: 4/4 red mutations there.
- codex-offered-* (x4): detects the `command -v codex` probe, a fallback and
  optionality. Replacement: gen-skill-docs.test.ts 'Codex outside voice is
  offered behind an executed availability probe' (probe + not_installed
  fallback for all four skills) and skill-validation.test.ts 'plan reviews run
  the Codex outside voice default-on'. Control: 4/4 red there.
- non_test_callers: none; history: c118e24 v1.64.1.0 (#2572).
- validation: the registration tests named above.

Controls log: .context/mutation-skill-e2e-plan-retirements.md.
2026-10-03 01:47:21 +00:00
garrytan 51f1d947ee test(W7h): pin the CEO /office-hours prerequisite structure free
Replacement check for the plan-ceo-review-benefits paid retirement (t14 F7).
The paid case had a model summarize plan-ceo-review/SKILL.md and grep the
summary for "office hours" and "design doc". The rendered text is static, so
this free test checks it directly: the Prerequisite Skill Offer triggers on
the design-doc check's "No design doc found", offers
'A) Run /office-hours now' and 'B) Skip — proceed with standard review', does
not re-offer later in the session, and Mid-session detection (0A) offers
`/office-hours`. Per the Eng Retirements item, CEO invariants become
periodic-only once this gate case retires, so they keep this structural
free check.

Control (scratch worktree, generated plan-ceo-review/SKILL.md): removing the
/office-hours option, the no-design-doc trigger, the mid-session offer or the
no-re-offer rule goes red; rewording the pitch stays green (5/5). Log:
.context/mutation-ceo-benefits.md.
2026-10-03 01:47:06 +00:00
garrytan 6d7935eae8 Merge remote-tracking branch 'origin/garrytan/opus55-wip-F1' into garrytan/opus-5-5-prompt-audit 2026-10-03 01:43:14 +00:00
garrytan f8043bd8fd test(W7c): qa-exploratory-callers keeps phase order and authority, drops duplicated prose
W7c (t04 F8, F16). One test file; only the generated-parent-path and
runner-prompt pins change. The validator-backed caller tests (observer,
checkpoint causality, read-only diff grammar, write watcher, deadline
authority) are unchanged.

Removed assertion groups (invariant -> remaining protection):
- Parent-path loop and preparation pins duplicated from qa-probe-gates
  (Reads in order, "Reuse Step 4's surfaces ...", "Finish missing methods
  before charters", isolation preflight, shared preparation order, the six
  plan-execution sentences, the three shared loop contracts) -> owned by
  qa-probe-gates (free, every host, with semantic mutation controls).
- Parent prose ("Templates cannot replace them", "Only the parent runs
  report-only discovery", "Read agent/user updates and await results ...",
  report-clean and not-run list duplicates of qa-caller-authority) -> caller
  observer attribution tests in this file (free) and gate E2Es
  review-exploratory-small-cli, ship-exploratory-small-cli (gate); report
  gates stay in qa-caller-authority (free, every host).
- Shared-loop duplicates (preserve key/value, Q supplies observed, write the
  report, return to step 2, pass requires) -> qa-probe-gates /
  qa-caller-authority; kept the observationCommand/observed field definitions
  and identity-hash rule here.
- Runner harness prompt sentences (t04 F16: receipt-interface prose,
  appendSystemPrompt rules, scheduling prompt) -> reduced to one anchor per
  rule; `at most ${maxTurns} assistant turns`, maxTurns/timeout/reserve
  values, `(probe-...)`, `reports/review.md`, and every `not.toMatch`
  scenario-answer ban stay. validateCallerEvidence remains the behavioral
  oracle.

Kept as case-insensitive meaning checks: phase order 1-4 per caller,
review readiness follows report-only rules and never installs/imports
cookies/bootstraps, smoke stop at 5 minutes or 12 probes, G enforces the
deadline, never reset D/bypass G, plan checks remain required past the
smoke budget, /review incomplete and /ship blocked unless the user accepts
the named risk, repeat step 3 for affected checks, harness keeps parent
decision gates, turn limit never authorizes skipping work, never group
probes, Hard deadline UTC rule, HANDOFF.md read before completed records,
no checkpoint for deferred ideas.

Cosmetic pins dropped: AND / Never capitals.

Mutation log: .context/mutation-qa-exploratory-callers.md (15 mutations, 0
unexpected: 2 reorders, 2 markers, 6 safety red; 5 rewordings green).
2026-10-03 01:39:06 +00:00
garrytan 3a4c34f184 test(W7c): ship-workflow-clarity becomes a structural check of ship-only invariants
W7c (t07 F1, F2, F15). One test file: 59 tests / ~438 pins -> 33 tests.
Round-tagged titles (r6/r12/r22) are gone. Every invariant this file shared
with the other converted ship files is now checked once, in its owner:
ship-control-flow (work list, repair ranges, docs/test/push recovery,
adversarial finish order, Step 11.5 binding), ship-plan-completion-
invariants (evidence ledger, push/commit, waivers), ship-reentry-gates (stop
lines, docs reentry, NEW_TITLE), workflow-excerpt (review cycle order,
coverage/probe gates), qa-probe-gates / qa-caller-authority (QA loop).

Removed assertion groups (invariant -> remaining protection):
- Duplicated literals (45 repeated 2-7x, e.g. "Keep the same attempt counts
  throughout the invocation" x7, "Docs, TODO edits ... make evidence STALE"
  x5, the CYCLES >= 3 / fixes-below-cap / no-edits sentences, push-failure
  and native-recovery sentences) -> one free check each in the owners above.
- Smoke/plan-check loop sentences and "Required even for small diffs ..."
  -> qa-probe-gates (free, every host); gate E2E ship-exploratory-small-cli
  (gate). Kept here: smoke guard started once before step 3, every host.
- Invocation-record field lists (receipt/handle/attempt field prose, "reuse
  this invocation's saved level", FRESH/ALREADY_BUMPED level derivation,
  receipt definition, start-token definition) -> kept `BUMP_LEVEL`, record
  outside the product tree, approval scope, never borrow/replace a token,
  REVIEW_START/PASS_START/DESIGN_START, `gstack-review-log --start`,
  `gstack-wtree`; version derivation also in test/ship-version-sync.test.ts
  (free); judge `ship/SKILL.md workflow` (gate lane).
- Reuse/judge-evidence prose ("original command, result/counts ...",
  "Compare hashes or complete bytes", "consumed files, fixtures ...",
  "complete expanded request, rubric ...") -> kept never resample, changed/
  unknown dependencies rerun, mandatory reviews still run, freeze inputs;
  judge `ship/SKILL.md workflow` (gate lane).
- Build/route prose ("If none exists, record not applicable ...", "declared
  generation/build commands ...", "Run them and save results", "Explain why
  other changes cannot affect it", "Inspect writer handles, including the
  docs child", "Keep Step 8's original child report and counts", "Recheck
  affected plan items ...", "append current evidence ...") -> kept stage
  order, route-label order, prompts/templates count as behavior, no-change ->
  stage 3, writer termination before another writer, stop until confirmed,
  build-failed label, no substitute command; gate E2Es ship-docsync-*
  (gate).
- Docs-collector prose ("Terminal completion or confirmed termination is
  sufficient", "Require every field/type ...", "Compare saved base and input
  hashes ...", "Otherwise save post-child hashes ...", "A stale snapshot is
  neither ...", "Reconcile those before proceeding") -> kept attempt counting
  incl. failed launches, request alone insufficient, exited child blocked,
  HEAD/index unchanged + `files_updated`, protected-file exclusions,
  `documentation_section`, failed/blocked -> recovery, actual user exception,
  blocked status retained, same approved scope and exact content, retry
  restarts stage 1; gate E2Es ship-docsync-missing-marker,
  ship-docsync-launch-failure, ship-docsync-late-result (gate).
- Release/commit prose ("Record the release decision after a version was
  actually written", "Skip unchanged versions and manifest-only repairs",
  "Best-effort, non-interactive, non-blocking", "Group VERSION + CHANGELOG
  + TODOS.md ...", "Only the final VERSION/CHANGELOG commit gets the release
  version and co-author trailer", "use the first changed component ...
  (major/minor/patch/micro") -> kept ALREADY_BUMPED, the stale "skip if
  ALREADY_BUMPED" ban, `baseVersion`/`currentVersion`, absent fourth
  component is zero, no automatic rebump, digest tokens and pre-push digest
  check, no Git tag, commit-before-push order; gate E2E ship-local-workflow
  (gate).
- Coverage/design prose ("Count each generation-authorized attempt ...",
  "including the initial audit, failures and zero-test results", "Two passes
  already used ...", "read-only reassessment uses no pass", design-lite
  ownership, same path/line merge, specialist stats, distinct defects) ->
  kept the 2-pass and 30/5/2-minute limits, re-entry never resets, missing
  permission is not approval, audit-only header/override order/no commit/
  return decisions, child JSON, one item with both sources and stricter ASK,
  neither pass substitutes; gate E2Es ship-coverage-audit,
  ship-coverage-value (gate). Design-review-lite E2E is periodic, so the
  merge rule keeps its free meaning check here.
- Spec/title/comment prose ("existing open PR/MR", "one match supplies the
  existing title/identity", "Save the result for Step 19's recheck", "In a
  new shell, restore ...", "Partial, deferred, failed, dropped or unverified
  scope uses ...", "newest", "Try to auto-resolve if they are simple",
  "distinguish an app from a library/CLI", "If the ask is App Store/
  TestFlight distribution", "a new package.json or Cargo.toml alone ...",
  "After edits, the full re-review repeats ...") -> judge `ship/SKILL.md
  workflow` (gate lane); ship-apple-gate (free) for the Apple route.

Kept as case-insensitive meaning checks (stop/consent/approval/safety,
always kept): never invent credentials, ask for unknown targets first,
include the workflow in tests/review, frontmatter read as data never shell
source, exact spec_branch match, positive-integer issue number, Closes #N
only for fully completed scope, lookup failure/ambiguous match -> stop never
"no PR", never publish an unprefixed title, ambiguous conflicts -> stop,
AskUserQuestion and wait, unclear Apple target -> AskUserQuestion and wait,
confirmed app -> stop and Read, stop for repair (recommended) or accept each
named probe risk, skipping a fix is not risk acceptance, accepted risk is not
passing verification, queue without editing in Steps 10/11, never credit
outside coverage, do not report a clean review, unavailable outside
challenge never resets the three-cycle limit, records mismatch blocks
release, missing snapshots block regardless of HEAD, uncertain edits cannot
be exempted, scripts/dependencies/runtime config need live tests, named
testing gap A/B/C (recommended), reuse prior B only for same scope, declared
but unavailable suite blocks, explicit untested-scope approval, report the
gap never FRESH, no documentation writer after push, do not run create
commands for an existing PR, Greptile status truthfulness.

Kept unchanged: all stale-string bans, machine tokens and JSON contracts
(`gh pr list ... --json number,title,url`, glab list, `v$NEW_VERSION `,
`${NEW_TITLE:?...}` guard, exit 3, `--allow-paths ...`, `| FRESH (exit 0) |`,
`--label <lane> --expect-cmd`, `gstack-review-read`, `git diff <reviewed-tree>
<current-tree>`, `"status":"complete|no_pr|unavailable"`, specialist and
coverage JSON, `status:"unavailable"`/`completed:false`/`converged:false`,
`Pre-Landing Review: INCOMPLETE`, `ALREADY_PUSHED`), the threshold/branch/
installed-asset reference test, the section-index row check.

Cosmetic pins dropped: STOP / ONE / ONLY capitals and bold (t07 F2).

Mutation log: .context/mutation-ship-workflow-clarity.md (23 mutations, 0
unexpected: 4 reorders, 4 markers, 9 safety red; 6 rewordings green, all of
which turned the old test red).
2026-10-03 01:38:09 +00:00
garrytan d3491c1734 test(W7h): pin the Codex availability probe free for the four offering skills
Replacement check for the codex-offered-* paid retirement (t14 F8): each of
office-hours, plan-ceo-review, plan-design-review and plan-eng-review must
render the executed `command -v codex` probe and its `not_installed`
fallback mode in its generated SKILL.md + sections. These are bash lines the
agent runs, so a deterministic grep replaces a paid model summary of them.
skill-validation.test.ts already covers the default-on outside voice for
eng/CEO/devex; this adds office-hours and plan-design-review and the
fallback token.

Control (scratch worktree, generated files): removing the probe from
office-hours, plan-design-review or plan-eng-review, or the CEO
not_installed branch, turns the new test red (4/4). Log:
.context/mutation-codex-offering.md.
2026-10-03 01:36:35 +00:00
garrytan d636245c38 test(W7h): retire the bws session-awareness (ELI16) paid case
W7h retirement (t14 F5). Removes the case from test/skill-e2e-bws.test.ts and
its E2E_TOUCHFILES / E2E_TIERS (gate) / E2E_KINDS (rule) entries in
test/helpers/touchfiles-data.ts; scripts/paid-test-durations.json is
per-file and test/helpers/periodic-exclude-data.ts had no entry.
test/touchfiles.test.ts selection examples now name sibling cases with the
same touchfiles (skillmd-outside-git, skillmd-no-local-binary).
e2e-tier-alignment, eval-kinds and touchfiles stay green.

Retirement card:
- test: skill-e2e-bws.test.ts 'session-awareness' (gate, paid).
- detects: nothing gstack ships. The prompt coaches a mode that no longer
  exists ("_SESSIONS=4, so ELI16 mode is active ... Re-ground them";
  `git grep ELI16` outside test/ and CHANGELOG is empty) and says "DO NOT
  actually call AskUserQuestion"; every asserted word (billing/acme,
  payment/feature, stripe/checkout, recommend) comes from the prompt itself.
- non_test_callers: none.
- search_command: git grep -n "session-awareness\|ELI16" -- test scripts
- stronger_proof / replacement: the AUQ format's grounding line
  'Project/branch/task: <1 short grounding sentence using _BRANCH>' is pinned
  free by plan-review-calibration.test.ts (format tokens) and graded by gate
  E2E auq-format-gate; session identity in the preamble stays pinned by
  skill-validation.test.ts 'contains session awareness' and
  gen-skill-docs.test.ts 'session awareness lives in gstack-skill-start'.
- history: c118e24 v1.64.1.0 (#2572).
- validation: bun test test/touchfiles.test.ts test/e2e-tier-alignment.test.ts
  test/eval-kinds.test.ts.

Control (scratch worktree, against the replacement): removing the
'Project/branch/task:' grounding line from the AUQ format resolver turns
plan-review-calibration.test.ts red (1/1). Log:
.context/mutation-session-awareness.md.
2026-10-03 01:35:14 +00:00
garrytan 8ac565d615 test(W7h): replace workflow-judge-cache source-grep counts with an invariant
W7h retirement (t09 F16, t13 F6) of the source-grep counts in 'workflow
registration preserves model work and reserves only terminal-recording
grace'. The stage-order check over runWorkflowJudge (the cheapest guard) and
every behavioral callback test stay.

Retirement card:
- test: workflow-judge-cache.test.ts pins of skill-llm-eval.test.ts source:
  "execution: reused ? 'reused' : 'executed'", 'const workDeadline = started
  + JUDGE_MS;', 'const WORKFLOW_JUDGE_RECORD_MS = 5_000;', 'const
  WORKFLOW_JUDGE_TEST_MS = JUDGE_MS + 10_000;', "testName: 'review/SKILL.md
  workflow'", and the magic counts /}, WORKFLOW_JUDGE_TEST_MS\);/ x17 and
  /}, JUDGE_MS\);/ x7.
- detects: any workflow eval added or removed, any constant reformatted.
- non_test_callers: none.
- stronger_proof / replacement: (a) a new invariant in the same test: every
  `await runWorkflowJudge(` registration closes with WORKFLOW_JUDGE_TEST_MS
  (the property the 17-count stood in for, now independent of how many evals
  exist); (b) the real-callback tests in this file exercise execute-once,
  reuse with provenance (execution reused/executed), timeout, supersede,
  expiry and the recording deadline.
- history: 636175d (#2898); last 7fca42a.
- validation: bun test test/workflow-judge-cache.test.ts.

Controls (scratch worktree, test/skill-llm-eval.test.ts): moving cache.publish
before the score assertions or registering the ship workflow judge with
JUDGE_MS goes red; adding a correctly registered eval and reformatting
5_000 as 5000 (both red under the old pins) stay green (4/4). Log:
.context/mutation-workflow-judge-cache.md.
2026-10-03 01:34:23 +00:00
garrytan e149521414 test(W7c): ship-control-flow keeps step routing, tokens and gates, drops rule prose
W7c (t09 F1, F2, F3, F4). One test file.

Removed assertion groups (invariant -> remaining protection):
- Work-list prose ("Expand a repair into individual steps ...", "This
  replaces the current item ...", "Add its destination only if ...", "The
  saved list takes precedence ...", "For another repair, repeat rule 2 ...",
  "The unchanged release steps follow", "Follow the saved work list",
  "Keep CYCLES and scoped approvals across this repeat", "Keep the findings
  and their approvals", "any further repair inserts its checks ...") ->
  repairs insert ordered steps ahead of pending work: kept free checks on the
  example sequences (`5 → … → 11.5`, `9 → 10 → 11` before 11.5, `5–11.5 →
  12–14 → 16`, `14.5 → 15 → 16`, `9 → 10 → 11 → 11.5`), insert-before-
  pending routing lines, no unlisted steps, same attempt counts, no reset of
  the one-repair limit; judge `ship/SKILL.md workflow` (gate lane); gate
  E2Es ship-docsync-recovery, ship-docsync-stale-after (gate).
- Queue-qualification sentence pins duplicated in ship-version-sync (t09 F4:
  "require successful utility output ...", "`offline:false` qualifies ...",
  "Offline output without that fallback ...") -> kept in
  test/ship-version-sync.test.ts next to its script-behavior tests (free);
  here the Qualify < Usable < No-usable order, `NEW_VERSION=CANDIDATE_VERSION`,
  `>= NEW_VERSION`, `BUMP_LEVEL`, `currentVersion` and the approval line stay.
- Distribution prose ("List candidate distribution paths", untracked files,
  manifest sentences) -> kept the New-artifact branch label, AskUserQuestion
  and the no-release-during-/ship safety line; gate E2E ship-local-workflow
  (gate).
- Dispatch/dashboard/coverage prose ("Omitting the flag runs the subagent in
  the background", "keeping a fresh context", "Use one row for each entry in
  step 1", "including its undetermined-percentage and test-only rules") ->
  kept the `run_in_background: false` contract at all three sites, the
  dashboard header/verdict tokens, no inline Skill fallback, coverage gate
  not bypassed; gate E2E ship-coverage-audit (gate).
- Recovery/docs/test sentences ("If an attempt remains and either ...", "a
  concrete launch/input/permission correction ...", "restart Step 16 stage 1
  to regenerate ...", "This recovery also applies ... stage 5", "Content-
  preserving commits keep valid evidence", "Retain queued fixes and restore
  coverage", "If this pass made edits, resume at the next decision ...",
  "Undispatched host-unsupported/gated specialists do not block", "count it
  in the invocation record before launch", "Keep Step 9.4's incomplete
  flags", "saved handle, original token and source", "unknown or missing
  status is unavailable", "Finish the saved replies ...", "fetch and inspect
  the remote, then merge ...") -> judge `ship/SKILL.md workflow` (gate
  lane); gate E2Es ship-docsync-* (gate) and ship-skipped-queued-finding
  (gate); the adjacent stop/approval lines stay free.

Kept as case-insensitive meaning checks (t09 F1/F3; stop/consent/approval
and safety lines): STOP and AskUserQuestion gates apply during repairs, a
repair never resets or expands approvals, children return evidence not
permission, prior-invocation audits never qualify, one repair/re-audit and
no third attempt (initial-plus-one, once), Otherwise -> stop unless the user
accepts the named docs risk, never silently refresh hashes, never run a third
audit, dispatched-reviewer-missing -> stop, CYCLES >= 3 -> stop with
`converged:false`, no fourth cycle, missing coverage cannot be replaced,
failed probes never clean, required-probe gate before Step 10, cannot waive
reviewer/test/security gates, native recovery stops the task first and
remains blocked, outside output cannot replace native, never attach new
tokens to old work, failed build stops shipping, never edit the child's
counts, test failure stops publication, push failure stops with no Step 19,
never rewrite history, never bypass failed guards, both redaction scans
before publishing, no release publish during /ship, only approval changes the
version, declining bootstrap is not test approval.

Kept unchanged: the executed plan-tune nudge shell test (3 roots), section/
step ordering checks, Step 11.5 placement and `review_binding.*` fields,
branch-label order in the adversarial finish (every host), all stale-
wording bans, `(recommended)` marker on Add tests.

Cosmetic pins dropped: STOP / ONE capitals (t09 F1, F3).

Mutation log: .context/mutation-ship-control-flow.md (21 mutations, 0
unexpected: 3 reorders, 3 markers, 9 safety red; 6 rewordings green).
2026-10-03 01:32:48 +00:00
garrytan 9c5aedea12 test(W7c): review-workflow-clarity keeps order, tokens and gates, drops judged prose
W7c (t06 #12). One test file. Each removed pin was the text fix for a past
`review/SKILL.md workflow` judge finding; that judge (gate lane) still
grades the prose, so the free test keeps only structure, tokens and gates.

Removed assertion groups (invariant -> remaining protection):
- Plan-audit sentences ("retain the exact command ...", "A mixed item
  contributes to both lists", "Keep external-state ...", "If only behavioral
  checks remain ...", "File-existence checks ... are static audit checks")
  -> audited deliverables vs deferred behavioral checks: kept free checks on
  the separation, never DONE from a diff, zero-deliverable no-waive,
  UNVERIFIABLE -> Step 4.7, read-only audit (no apps/APIs/state) and inspect
  -before-invoke order; gate E2E review-exploratory-small-cli (gate).
- QA loop/report duplicates of qa-probe-gates / qa-caller-authority (probe
  loop sentences, shared contracts, report layout prose, "Only the parent
  runs report-only discovery", "Read agent/user updates ...", "Prepare one
  provisional QA section", smoke composition) -> owned there (free, every
  host); here only `5 minutes/12 probes`, headings, link-every-checkpoint,
  no second report, never overwrite another run, freshness vs current inputs.
- Persistence/finalization prose (~45: "final zero-edit pass, reconcile",
  "by structural identity", "without requiring deleted pre-extraction
  blocks", "raw bytes equal the bound snapshot blobs", "Keep both through
  re-reviews", "Earlier core tokens remain unused", "Re-read its final-
  snapshot supporting source ...", "Fixed advice needs no skip coverage",
  "The required in-host adversarial result controls native completion",
  "Optional outside attempts keep their own incomplete records ...", etc.)
  -> kept free token list (REVIEW_START/PASS_START, COMPLETED, CONVERGED,
  CYCLES=0 once, `review_binding`, `snapshot_covered_paths`,
  sharedLibsFingerprint, `completed:false`, evidence_paths/helper_target),
  cycle limit (at most 3, Below 3 repeat, At 3 persist `converged:false`),
  section order, and the integrity gates below; judge `review/SKILL.md
  workflow` (gate lane); gate E2Es review-army-delivery-audit,
  review-army-json-findings, review-army-quality-score (gate).
- Confidence-calibration prose (data loss/outage examples, missing field /
  nullable / race quoting, metaclass/ORM sentence, missing-literal sentence,
  learning-log sentence) -> kept the score table as a routing table (each
  band's action), `[CRITICAL]` / `[CRITICAL|INFORMATIONAL]`, no P0-2,
  `file:line`, force 4-5, never invent 7+, unverified-without-quote, gate
  before table; judge `review/SKILL.md workflow` (gate lane).
- Shared-code reuse safeguard enumeration (assume-unchanged/skip-worktree,
  sparse index, symlinks, submodules, Git filters, encodings, fsmonitor,
  textconv, provenance, binding wording) -> enforced in code and covered by
  free tests test/shared-libs-snapshot-check.test.ts,
  test/shared-libs-evidence.test.ts, test/shared-libs-checker-interface-
  evidence.test.ts; kept the checker command/heredoc/JSON tokens and the
  fail-closed lines.
- Misc definitions ("A probe is affected when ...", "An invocation is this
  /review run ...", "Step 1.5's provisional notes ...", "Otherwise record
  each considered specialist", "Core findings use the confidence gates
  below", "highest confidence", "distinct specialists", "all source names",
  "active or unknown reader/writer", Greptile "Before edits, inspect ...")
  -> judge `review/SKILL.md workflow` (gate lane).

Kept as case-insensitive meaning checks (stop/consent/approval/safety):
no edits before Step 5, readers returned or confirmed stopped, no edits in
Step 4.8, stop without edits on unconfirmed readers (review and ship),
stop at 3 cycles with no clean summary or fourth pass, failed/blocked
probes mean incomplete, ship named-risk cannot complete review, native vs
outside cannot substitute, structured-review gate, never carry coverage
forward, never invent a binding, skipped advisory cannot suppress a defect,
user-skip-only suppression, Greptile reply != code approval and wait for
approval before editing, HIGH-impact gate options (A ends, read-only audit,
no shipping authorization), never downgrade severity, never trust a supplied
hash, missing metadata cannot dedupe, test creation needs approval,
report-only review never runs setup/installs/cookie import.

Kept unchanged: all marker-order arrays, manifest order, Pre-Landing summary
after the logger, finish-token count and negatives, stale-wording bans
(incl. `git diff origin/main`), ship/review routing negatives.

Cosmetic pins dropped: STOP / NON / COUNT capitals; the test titled "resolves
the judged ... ambiguities" is renamed to say what it keeps.

Mutation log: .context/mutation-review-workflow-clarity.md (23 mutations,
0 unexpected: 3 reorders, 4 markers, 11 safety red; 5 rewordings green).
2026-10-03 01:29:35 +00:00
garrytan 93d2d43c2c test(W7c): shrink gen-skill-docs to structural and meaning checks
W7c conversion of the sentence/caps pins in test/gen-skill-docs.test.ts
(t01 F2-F9, F18, F19; t08 F12; t10 F4; t02 #5 duplicate STOP literal).
t01 F1 ('Every commit must map to at least one bullet point') was already gone
on the base. Pins (grep -cE 'toContain|toMatch|indexOf|includes') 1446 -> 1360;
tests 457 -> 457. Untouched: the FRESH/STALE freshness check, the routing
soft-policy negative, skill-start/end script pins, the SESSION_KIND spawned
line, and the old-wording negative pins (delete-then-append, 0D/0F, gate
"if applicable"). Machine markers stay byte-exact: NO UNRESOLVED DECISIONS,
`**UNRESOLVED DECISIONS:**`, `Approval readiness: PASS`, `D<N> — <ROW-ID>: ...`,
`## EXIT PLAN MODE GATE (BLOCKING)` anchors, `**Gate outcome: Blocked**`, helper
commands/flags (gstack-office-hours-review prepare/check/finalize --design,
--out-dir, --report, --unreviewed), helper stop reasons, analytics commands.
Consent, stop, approval, decision-cycle and navigation lines (incl. the Eng
W7c-gate lines at the old 2362-2370) are kept as short case-insensitive checks.

Removed assertion groups (invariant -> remaining protection):
- qa-only 'Never fix bugs' + 'NEVER fix anything' (t01 F8): report-only skill
  -> kept free: /never fix/i in Additional Rules + frontmatter no Edit/Glob/Grep.
- qa 'WTF' token (t08 F12): fix-loop self-check -> kept free: /every 5 fixes/i.
- TEST_COVERAGE_AUDIT sharedPhrases sentences and the three Test-step sentences
  (t01 F4): read source before tracing/diagramming in plan and ship -> kept
  free: order 'concrete source and test files' < Trace data flow < Diagram the
  execution < Quality scoring rubric in both, star/GAP markers, the plan/diff
  mode split, the Test review subheading structure test; gate judges
  'plan-eng-review/SKILL.md sections', 'ship/SKILL.md workflow'.
- 'REGRESSION RULE' / 'IRON RULE' labels (t01 F4): regression coverage is
  required -> kept free: REGRESSION RULE heading, /without regression coverage,
  that coverage is a critical requirement/i (plan), /regression test is written
  immediately/i (ship), plus the existing regression-contract tests.
- '**STOP for each pending decision.**' (t01 F7, retired) -> kept free: 'Wait
  for its answer before applying that remedy' (control C1).
- F19 rule sentences: step numbers dropped; kept free /prevent duplicate bumps,
  pushes and PRs, never verification/i, /native result never supplies outside
  coverage/i, /required-probe gate, never silently waive/i, /never offer a
  conversion/i, /actual counts, never estimates/i (deduped 2 -> 1).
- SPEC_REVIEW_LOOP (t01 F3, about 74 sentence pins):
  - office-hours dispatch sentences (Before EACH dispatch, parent Read does not
    deliver, string unchanged) -> kept free: prepare --design/--out-dir,
    order mktemp < prepare < round-N.prompt.md < `dispatch`, /all preceding valid
    round files in order/i, /read the entire prepared prompt/i.
  - reviewer prompt sentences -> kept free: document/verdict path and schema
    wiring, mode sections ('The Assignment', 'What I noticed...'), dimension
    order, short checks for coaching protection, no invented customer answers,
    minor findings included, identical JSON, classify every prior finding,
    open questions vs committed behavior. Dropped: 'Could an engineer implement
    this...' ban (positive design-and-coaching check carries it), absence-is-not-
    confirmation, unsupported-claims and labeling sentences -> gate E2E
    office-hours-spec-review; prior-status validation in office-hours-review.test.ts.
  - convergence evidence sentences, 'Recording concerns does not mark them
    fixed', Disposition, observation/attempt counting, 'unavailable score is
    null', 'missing or invalid verdict is ... never PASS', 'Preserve the failed
    output' -> office-hours-review.test.ts (convergence, finalize, invalid
    artifact vs --unreviewed, null quality_score) + gate E2E
    office-hours-spec-review; kept free: helper stop-reason order and round 3,
    /before fixing any findings or dispatching again/i, /on a stop, do not fix
    again or re-dispatch/i, /do not edit that generated section/i, finalize before
    /existing user approval/i, --unreviewed wiring, /do not fabricate JSON/i,
    /not an approval gate/i.
  - CEO spec-review sentences (Read Agent's tool definition, omit field, consume
    completed review, Keep both consistent, duplicated 'both complete labeled
    texts'/'all five dimensions'/'Preserve the failure'/'citing the owning
    input'/'## Reviewer Concerns', 'Read these documents' ban, strategy-only and
    narrow-decision sentences) -> CEO is periodic-only, so kept free: one short
    check per invariant (wait tool, no-wait resume, no advance while waiting,
    three launches, stop after third, stop the loop, missing score, null grade,
    SCORE 1-10, not persisted, mkdir/append stop, save-failure stop, ITERATIONS,
    FOUND/FIXED/REMAINING order, both inputs in full, partial-input refusal,
    summary-is-not-the-plan, 1-10 score, PASS rule, cite input) plus the
    existing order/count/command checks.
- Design approval reconciliation (t01 F2, about 79 pins):
  - Pass 5 sentences and 'AskUserQuestion once per issue. Do NOT batch.
    Recommend + WHY.' -> kept free: /stop\W+AskUserQuestion once per issue/i
    located after four short reuse/new-evidence/first-decision checks; batching ->
    auq-format-always-loaded.test.ts PER_SKILL_RULES ('One issue = one
    AskUserQuestion call').
  - duplicate 'Never group distinct issues.' pins (2 -> 1) and the rest of the
    Design gate 0 sentences -> kept free: 0 < 1 order, short checks per
    approval line, refresh < Read-back gate < rerun order; gate judge
    'plan-design-review/SKILL.md passes'.
  - CEO readiness/gate/0D sentences -> kept free: one short check per approval
    line, readiness order, `Approval readiness: PASS`, gate order Read-only <
    Verify readiness < 1. Read, route order, /stop for the actual answer/i in
    step 4, `D<N> — <ROW-ID>` header, row fields; admin-route sentences ->
    ceo-workflow-clarity.test.ts kept checks. Dropped: 'No report or completion
    log is needed' (order check carries it), 'Save or present the updated plan',
    'checked row IDs...' (marker kept), strategy-only/narrow-decision
    sentences (default kept), cross-model/integration sentences reduced to
    external-only and native-fallback checks.
  - Eng readiness/procedure sentences -> kept free: short approval checks,
    gate numbering/order, AUQ call < /stop until the actual answer arrives/i <
    Record the answer, recovery order; 'separate it and rebuild this
    comparison' -> auq-format-always-loaded.test.ts PER_SKILL_RULES.
- Exit gate 'contract violation' sentence (t01 F9, design/devex/codex) ->
  kept free: heading anchor + /do not call ExitPlanMode/i inside the gate.
- Unresolved-decisions gate (t01 F5): '(MANDATORY', 'Never omit'/'never
  omitted', 'FINAL non-whitespace line is the unresolved-decisions', 'FAILS the
  gate', 'DROP the current skill' -> kept free: NO UNRESOLVED DECISIONS,
  /final\s+non-whitespace\s+line/i, `**UNRESOLVED DECISIONS:**`, /missing\s+status
  [^.]*fails/i in every gate skill and the resolver, /avoids double-counting/i,
  the old soft-wording negative pin.
- delete-then-append 'never mid-file' / 'Do NOT replace the section in place'
  (t01 F6) -> kept free: 'delete-then-append flow' + /append the new report at
  EOF/i; old-wording negatives kept.
- design-html '**Never include by default (AI slop blacklist):**' (t01 F18) ->
  kept free: /AI slop blacklist/i, catalog id comment, lib/design-catalog.ts.
- CODEX_BOUNDARY 'Do NOT read or execute any' casing (t10 F4) -> kept free:
  /do not read or execute any/i in every codex-calling skill.
- CEO 0D order anchor '**STOP for the actual answer...**' (one of the 8) ->
  kept free: /stop for the actual answer/i at the same position.

Mutation pass (scratch worktree, generated SKILL.md/sections plus
spec-review.ts and lib/office-hours-review.ts; the freshness test is excluded
with -t because any mutation makes it STALE): 43/43 as expected. 25 kept-step,
marker, safety, numeric and retirement-control mutations go red; 18 rewordings
of dropped sentences and lowered-register STOP/NEVER/FAILS lines stay green.
Log: .context/mutation-gen-skill-docs.md.

Runtime (Bun 1.4.0, wall, 3 runs): before 4.42/4.49/4.70s (median 4.49s),
after 4.42/4.53/4.63s (median 4.53s).
2026-10-03 01:29:08 +00:00
garrytan af697dc6ec test(W7h): retire the paid first-task-scaffold E2E duplicate
W7h retirement (t12 F12). Removes the case from test/helpers/touchfiles-data.ts
(E2E_TOUCHFILES, E2E_TIERS periodic, E2E_KINDS rule) and both
scripts/paid-test-durations.json entries; test/helpers/periodic-exclude-data.ts
had no entry. e2e-tier-alignment, eval-kinds, eval-detach-timeout-floor,
periodic-exclude-policy, touchfiles and paid-shards tests stay green.

Retirement card:
- test: test/skill-e2e-first-task-scaffold.test.ts ('first-task-scaffold',
  periodic, ~$0.02): Haiku is told to run exactly one detector command and the
  test greps the tool result for code_node / greenfield.
- detects: detector bucket regressions in bin/gstack-first-task-detect.
- non_test_callers: none.
- search_command: grep -rn "first-task-scaffold" test scripts .github
- stronger_proof / replacement: test/preamble-first-task-scaffold.test.ts
  (free; executes the real detector on fixture repos for every bucket,
  including code_node and greenfield, plus first-run gating and telemetry in
  bin/gstack-skill-start). The model added nothing: the prompt dictated the
  exact command.
- history: c118e24 v1.64.1.0.
- validation: bun test test/preamble-first-task-scaffold.test.ts plus the
  registration tests above.

Control (scratch worktree, against the replacement): making the detector emit
the wrong bucket for a Node project or a commit-less repo turns
preamble-first-task-scaffold.test.ts red (2/2). Log:
.context/mutation-first-task-scaffold.md.
2026-10-03 01:29:02 +00:00
garrytan 851328b35f test(W7c): workflow-excerpt keeps heading order and tokens, drops judged prose
W7c (t04 F5, F6, F7, F15; F1/F2 already landed with W7a). One test file.
This file guards judge-input fidelity; the judges that read these excerpts
grade the prose itself.

Removed assertion groups (invariant -> remaining protection):
- Ship rule sentences ("Use the documented selector and pre-merge command.",
  "During pre-flight, read the existing review log", "Tests must pass or
  retain approval ...", "For another repair, repeat rule 2 ...", "The
  unchanged release steps follow", "Use this example only after confirming
  ...", "Retain queued fixes and restore coverage", "Each commit must work
  independently", "Keep actual outcomes and incomplete flags", the long
  "Continue to Step 9.2 with the core/design-lite findings ..." form, the
  Y'd/D'd evidence sentence) -> judge `ship/SKILL.md workflow` (gate lane),
  which reads this exact excerpt; kept free checks on step routing (`Step 4`,
  `Step 8.1`, 9.2 -> 9.2.1 QA -> 9.3, `Insert Step 5, affected Steps 6–8 and
  all of Step 9 before the pending Step 10`, `5 → 6 → … → 11.5`,
  `continue to Step 16`), tokens (`BASE_VERSION`, `VERIFY_RESULT stays
  fail`, `"partial":N,"not_done":N`, "intentionally dropped"), and the review
  cycle order (9.1 < 9.2 < 9.2.1 < 9.3 < 9.4) as a structural check.
- Outside-review routing sentences ("Other preflight mode, including harness
  mismatch", "Outside execution or output validation fails", "Retain its
  output and diagnosis ...", "No prompt, outside process or native
  replacement", "Other preflight failures retain their printed diagnosis",
  the full fallback-list regex) -> disabled means no outside call: kept free
  order of disabled/prompt/ready/fallback blocks, `outside_status` tokens,
  `_DISABLED_REVIEW_MODE`, disabled recheck before dispatch, "disabled means
  no replacement", fallback-not-completion; judges
  `document-release/SKILL.md workflow`, `plan-eng-review/SKILL.md sections`
  (gate lane).
- Eng/design prose ("quote the motivating plan requirement", "Use a scoped
  Edit to save this record ... Leave other choices unchanged" full form,
  "edit the plan and re-rate") -> judge `plan-eng-review/SKILL.md sections`
  and `plan-design-review/SKILL.md passes` (gate lane); approval meaning
  checks kept.
- Retro prose ("does not require saved history", "PRs referenced", "not
  newly added test cases", "is the live **team** streak", "draft the
  tweetable summary using the format in Step 14, then save") -> judge
  `retro/SKILL.md instructions` (gate lane); kept headings, `prs_merged:
  null`, `streak_days`, the one-second compare window, Capture Learnings
  after Step 14, no `$(date`, two `today="<today>"`.

Loosened to case-insensitive meaning checks (stop/consent/approval and
safety lines, always kept): STOP for each pending decision (now a regex
anchor between Step 5 and the Test Plan Artifact), STOP until the actual
answer arrives, no remedy/next section/ExitPlanMode while a choice awaits an
answer, evidence-not-approval, reopened choices need answers, wait for
approval (design), AskUserQuestion once (t04 F6), per-finding approvals,
dispatched-reviewer-missing -> stop, CYCLES >= 3 -> stop with
`converged:false`, no fourth cycle, required-probe gate before Step 10,
failed probes block continuation, coverage-gate not bypassed, failed/unrun
probes not passed, three snapshots match, document-release VERSION-absent
rule, never create an empty commit, `read any other SKILL.md` (t04 F15),
AskUserQuestion-and-wait in the capture prompt.

Kept unchanged: heading/step order checks, `## Review Sections` single-copy
check, fail-closed marker tests, redact-before-edit ordering, approval gates
outside subagent prompts, plan-tune auto-decide output lines, CEO capture
harness contract, deploy formula execution, all stale-string bans.

Cosmetic pins dropped: STOP / ONCE / NOT / OTHER capitals.

Mutation log: .context/mutation-workflow-excerpt.md (21 mutations, 0
unexpected: 3 reorders, 3 markers, 8 safety red; 7 rewordings green).
2026-10-03 01:24:11 +00:00
garrytan 958f27a2b8 test(W7h): retire the literal 75 KB autoplan budget duplicate
W7h retirement (t04 F18). test/skill-preflight-budget.test.ts asserted the
same four-skill autoplan sum twice: once against
AUTOPLAN_PREFLIGHT_BUDGET_BYTES * 3 (25,600 x 3 = 76,800 = 75 KiB) and once
against a literal 75 * 1024, which would drift silently if the constant
changed.

Retirement card:
- test: skill-preflight-budget.test.ts 'autoplan total under 25 KB
  instruction cap × 3 (75 KB instruction budget)'.
- detects: the four plan-* skills' brain preflight + refresh + write-back
  instructions exceeding 75 KiB.
- non_test_callers: none.
- stronger_proof / replacement: 'autoplan: sum across 4 plan-* skills stays
  under AUTOPLAN_PREFLIGHT_BUDGET_BYTES × 3 (instructions)' in the same file,
  which reads the constant from scripts/brain-cache-spec.ts.
- history: c118e24 v1.64.1.0.
- validation: bun test test/skill-preflight-budget.test.ts.

Controls (scratch worktree): shrinking AUTOPLAN_PREFLIGHT_BUDGET_BYTES below
the current total, or growing the eng write-back instructions by 80 KB, turns
the kept constant-based check red (2/2). Log:
.context/mutation-skill-preflight-budget.md.
2026-10-03 01:22:31 +00:00
garrytan 6423617fe6 test(W7h): retire spec-template-sync as a duplicate of the freshness check
W7h retirement (t05 F18; Eng factual correction: the file takes 145 ms, its
"2 minutes" was the 120 s timeout, so this is a duplicate-only retirement).
Removes its scripts/free-test-durations.json entry.

Retirement card:
- test: test/spec-template-sync.test.ts (regenerate /spec into an out-dir and
  compare spec/SKILL.md + spec/sections/gate-and-file.md with the tracked
  bytes; check the generated header).
- detects: /spec template edited without regeneration, or a hand-edited
  generated file.
- non_test_callers: none.
- search_command: grep -rn spec-template-sync package.json scripts test .github
- stronger_proof / replacement: gen-skill-docs.test.ts 'generated files are
  fresh (match --dry-run)' (FRESH for every skill, no STALE anywhere; the
  dry-run covers carved sections such as spec/sections/gate-and-file.md); the
  generated-banner checks in the same file cover the header.
- history: c118e24 / 410b492 / 394db32 / b1485d8 (codex T8).
- validation: bun test test/gen-skill-docs.test.ts.

Control (scratch worktree, run against the replacement): a stale
spec/sections/gate-and-file.md, a stale spec/SKILL.md, and a spec template
edit without regeneration each fail gen-skill-docs 'generated files are fresh'
(3/3). Log: .context/mutation-spec-template-sync.md.
2026-10-03 01:22:01 +00:00
garrytan d2f53fcb9c test(W7h): retire CEO TODO-menu prose pins from plan-review-calibration
W7h retirement (t04 F11; W7c list). The calibration corpus and
buildPlanReviewDecisionPrompt never read this CEO prose; the AUQ format
tokens (consumed by plan-tune and judges) and the calibration-harness
outcome tests are unchanged.

Retirement card:
- test: plan-review-calibration.test.ts, the six CEO TODO-menu pins ('Only
  unanswered TODO proposals reach this menu', 'Do not ask again about an item
  already deferred, skipped or kept', 'carry its actual answer and destination
  forward', 'Resolve each remaining proposal through all four steps of 0D,
  using the menu below', 'Never batch TODOs — one per question').
- detects: edits to plan-ceo-review TODO-menu prose.
- non_test_callers: none (test-only reads of the template).
- search_command: grep -rn "Never batch TODOs\|Only unanswered TODO" test/
- stronger_proof / replacement: skill-ceo-section-ordering.test.ts 'TODOS.md
  updates' slice keeps /i meaning checks for unanswered-only, no re-asking
  and one TODO per question (CEO invariants are periodic-only under the W7c
  gate, so they keep a free check); 'route through 0D' stays covered by that
  file's 0D route checks.
- history: 636175d v1.87.6.0.
- validation: bun test test/plan-review-calibration.test.ts.

Controls (scratch worktree): replacing 'Never batch TODOs — one per question'
or deleting 'Only unanswered TODO proposals reach this menu' turns
skill-ceo-section-ordering.test.ts red (2/2); in this file, removing an AUQ
format token goes red and rewording the retired TODO sentence stays green
(2/2). Log: .context/mutation-plan-review-calibration.md.
2026-10-03 01:21:02 +00:00