mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-09 06:28:59 +02:00
docs: CHANGELOG accuracy fixes from the doc-release review
Three factual corrections the Step 18 doc subagent caught in the fresh v1.70.1.0 entry: 5 tripwire tests (not 6), cost floor $0.63 per the cited eval store (not $0.59), and the visibility claim scoped to decision points (the re-run checklist mention survived the carve). Plus the E2E header's stale pending-burn-in note replaced with the observed numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
a5b1396599
commit
bb56dfef96
+3
-3
@@ -5,7 +5,7 @@
|
||||
**Ship names its documentation subagent at every decision point.**
|
||||
**The handoff is now pinned by tests that fail loud if it ever goes quiet.**
|
||||
|
||||
`/ship` has dispatched `/document-release` as Step 18 since v0.18.2.0, but the v1.54.0.0 carve moved that step into an on-demand section and the always-loaded skeleton stopped saying "document-release" anywhere in the workflow body. The wiring was intact. The visibility was gone, and nothing tested the handoff. This release restores the visibility and locks it in: the section index, the STOP pointer, the Step 17 handoff line, and a new hoisted doc-sync invariant all name "the /document-release subagent" (subagent-framed on purpose, so an agent dispatches the isolated worker instead of running a weaker inline copy). A free tripwire pins the wording, carve-guard anchors pin each touchpoint independently, and a new gate-tier E2E proves a live agent actually fires the dispatch before creating the PR.
|
||||
`/ship` has dispatched `/document-release` as Step 18 since v0.18.2.0, but the v1.54.0.0 carve moved that step into an on-demand section and the always-loaded skeleton stopped saying "document-release" at any decision point (one mention survived, buried in the re-run checklist). The wiring was intact. The visibility was gone, and nothing tested the handoff. This release restores the visibility and locks it in: the section index, the STOP pointer, the Step 17 handoff line, and a new hoisted doc-sync invariant all name "the /document-release subagent" (subagent-framed on purpose, so an agent dispatches the isolated worker instead of running a weaker inline copy). A free tripwire pins the wording, carve-guard anchors pin each touchpoint independently, and a new gate-tier E2E proves a live agent actually fires the dispatch before creating the PR.
|
||||
|
||||
### The numbers that matter
|
||||
|
||||
@@ -14,8 +14,8 @@ Source: this branch's eval store (`~/.gstack/projects/<slug>/evals/`, runs of `t
|
||||
| Property | Before | After |
|
||||
|--------|--------|-------|
|
||||
| Doc-sync subagent named in the Claude-host ship workflow body | 0 mentions at any decision point | 4 (section index, STOP pointer, Step 17 handoff, hoisted invariant) |
|
||||
| Tests pinning the ship→document-release handoff | none | 6 free tripwire tests + 3 per-touchpoint carve anchors + 1 gate E2E |
|
||||
| Live dispatch proof | never measured | 9/9 runs fire the dispatch before PR creation ($0.59-1.04, 234-319s each, sonnet-4-6) |
|
||||
| Tests pinning the ship→document-release handoff | none | 5 free tripwire tests + 3 per-touchpoint carve anchors + 1 gate E2E |
|
||||
| Live dispatch proof | never measured | 9/9 runs fire the dispatch before PR creation ($0.63-1.04, 234-319s each, sonnet-4-6) |
|
||||
| Always-loaded skeleton cost | 91,267 B | 91,764 B (+497 B, cap raised to 92,300) |
|
||||
|
||||
Nine out of nine live runs is the line that matters. The E2E asserts on the actual tool-call stream, with a dispatch-specific matcher that a subagent merely quoting section text cannot satisfy, and a timeout-tolerant exit check that never softens the dispatch assert itself.
|
||||
|
||||
@@ -45,7 +45,8 @@
|
||||
* test would make the assert tautological (and the prompt echoes into the
|
||||
* transcript, which is why asserts only ever read result.toolCalls).
|
||||
*
|
||||
* Cost: ~$0.30-1/run. Gate tier (pending burn-in; see plan).
|
||||
* Cost: observed $0.63-1.04/run, 234-319s (9/9 burn-in + review runs passed;
|
||||
* gate tier confirmed).
|
||||
*/
|
||||
import { expect, beforeAll, afterAll } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
|
||||
Reference in New Issue
Block a user