diff --git a/TODOS.md b/TODOS.md index 3270e2ece..535ccacf5 100644 --- a/TODOS.md +++ b/TODOS.md @@ -2329,7 +2329,7 @@ Shipped as v0.5.0 on main. Includes `/plan-design-review` (report-only design au ### Auto-invoke /document-release from /ship — SHIPPED -Shipped in v0.8.4; redesigned twice since. Current design (v0.18.1.0+, carved in +Shipped in v0.8.4; redesigned twice since. Current design (v0.18.2.0+, carved in v1.54.0.0): `/ship` Step 18 (`ship/sections/pr-body.md`) dispatches `/document-release` as a general-purpose subagent AFTER Step 17 (push) and BEFORE Step 19 (PR creation); the subagent's JSON contract (`files_updated`, @@ -2380,16 +2380,18 @@ pin) and `test/skill-e2e-ship-docsync.test.ts` (dispatch E2E, gate tier). ### Periodic paid-test shard census is one ungated file from the detach-timeout floor -**What:** The periodic tier's shard census sits at exactly the 17×4 ceiling -(68-file boundary). The next paid `skill-e2e-*` file WITHOUT a whole-file -`describeE2ETier` self-gate counts in BOTH tier censuses, trips periodic to 18 -shards → 34,020s floor > the 32,400s configured detach timeout, and +**What:** The periodic tier's shard census is 67 files — one ungated slot below +the 68-file (17×4) ceiling. The next paid `skill-e2e-*` file WITHOUT a +whole-file `describeE2ETier` self-gate lands at 68 (still 17 waves, floor +32,130s ≤ 32,400s — passes); the SECOND ungated file trips 18 waves → 34,020s +floor > the 32,400s configured detach timeout, and `test/eval-detach-timeout-floor.test.ts` fails with a confusing message. -**Why:** Whoever adds the next periodic E2E gets a floor failure unrelated to -their change. Fix options: raise the periodic detach timeout, or enforce -whole-file tier self-gates on all paid files (upgrades them from the -tier-alignment warn-only bucket to the hard invariant at the same time). +**Why:** Whoever adds the second ungated periodic E2E gets a floor failure +unrelated to their change. Fix options: raise the periodic detach timeout, or +enforce whole-file tier self-gates on all paid files (upgrades them from the +tier-alignment warn-only bucket to the hard invariant, and — bonus — restores +tierless `bun run test:evals` coverage decisions to diff selection alone). **Context:** `scripts/test-paid-shards.ts` `classifyPaidTestFile` counts ungated files in both tiers; `ship-docsync` composed `describeE2ETier('gate')` diff --git a/test/ship-document-release-dispatch.test.ts b/test/ship-document-release-dispatch.test.ts index de93df904..118124a00 100644 --- a/test/ship-document-release-dispatch.test.ts +++ b/test/ship-document-release-dispatch.test.ts @@ -59,6 +59,12 @@ describe('/ship Step 18 dispatches /document-release (carve visibility)', () => expect(content).toContain('"documentation_section"'); // Deliberate design: docs sync never holds a ship hostage. expect(content).toContain('Do not block /ship on subagent failure'); + // These two strings are the ship-docsync E2E's dispatch-matcher markers + // (test/skill-e2e-ship-docsync.test.ts) — they come verbatim from the + // dictated Step 18 subagent prompt. Rewording them in pr-body.md.tmpl + // silently decouples the paid matcher; update both in lockstep. + expect(content).toContain('You are executing the /document-release workflow'); + expect(content).toContain('.claude/skills/gstack/document-release/SKILL.md'); } }); diff --git a/test/skill-e2e-ship-docsync.test.ts b/test/skill-e2e-ship-docsync.test.ts index 853b8859f..048d626cb 100644 --- a/test/skill-e2e-ship-docsync.test.ts +++ b/test/skill-e2e-ship-docsync.test.ts @@ -8,8 +8,11 @@ * * Gating: whole-file gate-tier self-gate (describeE2ETier) COMPOSED with * diff-based selection (describeIfSelected). The self-gate keeps this file - * out of the periodic shard census (which sits at its ceiling) and under - * the hard tier-alignment invariant. Run locally with: + * out of the periodic shard census (near its ceiling) and under the hard + * tier-alignment invariant. DELIBERATE TRADEOFF: tierless runs (`bun run + * test:evals` / `test:e2e`) skip every tier-gated file, so this test does + * NOT run there even when ship/** changed — use the gate lane locally: + * EVALS_TIER=gate bun run test:evals # diff-selected gate lane * EVALS=1 EVALS_TIER=gate EVALS_ALL=1 bun test test/skill-e2e-ship-docsync.test.ts * * Fixture layout (non-obvious — fake HOME + planted skill tree): @@ -208,13 +211,19 @@ describeE2E('Ship doc-sync dispatch E2E (gate)', () => { // (e.g. a PR-body drafter) must NOT count — that false-pass would mask // the exact regression this test exists to catch. Verified against // recorded burn-in transcripts: real dispatch inputs carry both markers. - const dispatchIdx = calls.findIndex( - (tc) => - (tc.tool === 'Agent' || tc.tool === 'Task') && - /document-release\/SKILL\.md|executing the \/document-release workflow/i.test( - JSON.stringify(tc.input ?? {}) - ) - ); + // Section-paste exclusion: a subagent handed the WHOLE pr-body.md as + // context carries the markers too. The dictated Step 18 prompt never + // contains the section's scaffolding, so its presence disqualifies. + // Verified across all recorded runs: real dispatches match markers, + // zero contain scaffold strings. + const dispatchIdx = calls.findIndex((tc) => { + if (tc.tool !== 'Agent' && tc.tool !== 'Task') return false; + const input = JSON.stringify(tc.input ?? {}); + return ( + /document-release\/SKILL\.md|executing the \/document-release workflow/i.test(input) && + !/## Step 19: Create PR\/MR|Parent processing:/.test(input) + ); + }); const prCreateIdx = calls.findIndex( (tc) => tc.tool === 'Bash' &&