docs: CHANGELOG accuracy fixes from the doc-release review

Three factual corrections the Step 18 doc subagent caught in the fresh
v1.70.1.0 entry: 5 tripwire tests (not 6), cost floor $0.63 per the cited
eval store (not $0.59), and the visibility claim scoped to decision points
(the re-run checklist mention survived the carve). Plus the E2E header's
stale pending-burn-in note replaced with the observed numbers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-26 16:27:23 +00:00
co-authored by Claude Fable 5
parent a5b1396599
commit bb56dfef96
2 changed files with 5 additions and 4 deletions
+3 -3
View File
@@ -5,7 +5,7 @@
**Ship names its documentation subagent at every decision point.**
**The handoff is now pinned by tests that fail loud if it ever goes quiet.**
`/ship` has dispatched `/document-release` as Step 18 since v0.18.2.0, but the v1.54.0.0 carve moved that step into an on-demand section and the always-loaded skeleton stopped saying "document-release" anywhere in the workflow body. The wiring was intact. The visibility was gone, and nothing tested the handoff. This release restores the visibility and locks it in: the section index, the STOP pointer, the Step 17 handoff line, and a new hoisted doc-sync invariant all name "the /document-release subagent" (subagent-framed on purpose, so an agent dispatches the isolated worker instead of running a weaker inline copy). A free tripwire pins the wording, carve-guard anchors pin each touchpoint independently, and a new gate-tier E2E proves a live agent actually fires the dispatch before creating the PR.
`/ship` has dispatched `/document-release` as Step 18 since v0.18.2.0, but the v1.54.0.0 carve moved that step into an on-demand section and the always-loaded skeleton stopped saying "document-release" at any decision point (one mention survived, buried in the re-run checklist). The wiring was intact. The visibility was gone, and nothing tested the handoff. This release restores the visibility and locks it in: the section index, the STOP pointer, the Step 17 handoff line, and a new hoisted doc-sync invariant all name "the /document-release subagent" (subagent-framed on purpose, so an agent dispatches the isolated worker instead of running a weaker inline copy). A free tripwire pins the wording, carve-guard anchors pin each touchpoint independently, and a new gate-tier E2E proves a live agent actually fires the dispatch before creating the PR.
### The numbers that matter
@@ -14,8 +14,8 @@ Source: this branch's eval store (`~/.gstack/projects/<slug>/evals/`, runs of `t
| Property | Before | After |
|--------|--------|-------|
| Doc-sync subagent named in the Claude-host ship workflow body | 0 mentions at any decision point | 4 (section index, STOP pointer, Step 17 handoff, hoisted invariant) |
| Tests pinning the ship→document-release handoff | none | 6 free tripwire tests + 3 per-touchpoint carve anchors + 1 gate E2E |
| Live dispatch proof | never measured | 9/9 runs fire the dispatch before PR creation ($0.59-1.04, 234-319s each, sonnet-4-6) |
| Tests pinning the ship→document-release handoff | none | 5 free tripwire tests + 3 per-touchpoint carve anchors + 1 gate E2E |
| Live dispatch proof | never measured | 9/9 runs fire the dispatch before PR creation ($0.63-1.04, 234-319s each, sonnet-4-6) |
| Always-loaded skeleton cost | 91,267 B | 91,764 B (+497 B, cap raised to 92,300) |
Nine out of nine live runs is the line that matters. The E2E asserts on the actual tool-call stream, with a dispatch-specific matcher that a subagent merely quoting section text cannot satisfy, and a timeout-tolerant exit check that never softens the dispatch assert itself.
+2 -1
View File
@@ -45,7 +45,8 @@
* test would make the assert tautological (and the prompt echoes into the
* transcript, which is why asserts only ever read result.toolCalls).
*
* Cost: ~$0.30-1/run. Gate tier (pending burn-in; see plan).
* Cost: observed $0.63-1.04/run, 234-319s (9/9 burn-in + review runs passed;
* gate tier confirmed).
*/
import { expect, beforeAll, afterAll } from 'bun:test';
import * as fs from 'fs';