diff --git a/CHANGELOG.md b/CHANGELOG.md index df35fbf13..12dcfe8ab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,12 +8,15 @@ - **Thresholds sit above the host call floor.** `lib/ship-budget-calibration.json` gives each meter a `call_floor` (Capy 150k, Claude Code 40k, estimate 0); `start --call-floor ` overrides it and children inherit it. `BUDGET:` and `STOP:` lines print `call_floor=` and `above_floor=`. - **A child never stops on the context rule.** After a completed step it hands its remaining steps to a grandchild through the same handoff, verify, return and reconcile commands (chain depth 3 at most). Before its first completed step, at the maximum depth, or past its segment it continues with one `CONTEXT_HIGH:` line. The admission cap still stops it. - **A delegated child cannot return complete with work left over.** `return` reports `status=incomplete` with `UNRECONCILED:` or `DROPPED:` lines while one of its own grandchildren is unreconciled or a batch remainder was never re-checked. +- **Reviewer drift is measured to each reader's return, per reader.** A dispatch is measured only to the reading it was settled with when it returned (`check --returned`, `settle --reading`); `--finish` and a step change settle without one and print `UNMEASURED:`. A step is measured at the step change minus the dispatches reserved inside it, so the parent's own calls stay in the step. On #34 a Red Team interval that covered the child's own review and QA taught a 1.818 factor; on #36 a six-reader batch reserved as one role taught 2.041 and priced the Red Team at 15.17M, a false STOP. The budget section now asks for one role per `DISPATCH:` line. +- **A `--prior-reading` older than the run's last reading is ignored** (`CALL_UNKNOWN:`, `call=unknown`). On #34 a reading six calls old produced `call=1068386` and a false `CONTEXT_HIGH`. The meter read now happens inside the same program as the budget command, never as its own call (the #34 child made 11 meter-only calls at about 286k each). +- **A handoff brief names only its segment's sections** (Steps 7-11.5: test-coverage, plan-completion, review-army, adversarial; Steps 12-22: changelog, documentation, pr-body, capy-pr; budget-handoff.md always). It never sends the child to preamble.md in full or docs/ship-budget.md. ### Changed - **Parallel reviewer batches near the cap admit the largest prefix that fits** (`ADMIT: k of n (remaining=)`) instead of refusing the whole batch; each reader is priced at the larger of its projection and the run's measured mean for that role. `close` flags remaining roles nobody re-checked (`DROPPED:`, `status=incomplete` until the owner accepts). ### Tests -- `test/ship-budget-chain.test.ts` replays #36 (a child at its first boundary continues) and covers the Capy floor, grandchild handoff at depth 2, the depth-3 cap, charging a chained return once on all three meters, and the incomplete return. `test/ship-budget-batch.test.ts` replays #35's 7-reader batch near the cap. +- `test/ship-budget-chain.test.ts` replays #36 (a child at its first boundary continues) and covers the Capy floor, grandchild handoff at depth 2, the depth-3 cap, charging a chained return once on all three meters, and the incomplete return. `test/ship-budget-batch.test.ts` replays #35's 7-reader batch near the cap. `test/ship-budget-reader-drift.test.ts` replays #36's per-reader Red Team price and #34's Red Team interval and stale prior reading; `test/ship-handoff-brief.test.ts` checks the brief's section list. ## [1.91.324.0] - 2026-10-29 diff --git a/docs/ship-budget.md b/docs/ship-budget.md index 2180eb82b..b496c1a13 100644 --- a/docs/ship-budget.md +++ b/docs/ship-budget.md @@ -23,19 +23,24 @@ you set stays as given); the stop overhead never scales. ## Quickstart -**Capy.** Read the root counter in one execute program that returns only that number: +**Capy.** Read the root counter inside the execute program that runs the command, never as a +call of its own: ```js -return (await tools.thread.get({ thread_id: "this" })).cost.tokens; +const n = (await tools.thread.get({ thread_id: "this" })).cost.tokens; ``` -then +then pass it to ```bash gstack-ship-budget start --record --token-cap 25M --reading gstack-ship-budget check --run --step 6 --next 7 --reading [--prior-reading ] ``` +`--prior-reading` must come from the call just before this one; a prior older than the run's last +reading spans several calls, so it is ignored (`CALL_UNKNOWN:`, `call=unknown`). Two readings in +one program measure nothing: Capy books a call after its program ends. + The counter rolls up every subagent of the thread, outside voices included; Capy exposes no per-subagent counter, so other work in the same thread counts too (it only stops earlier). @@ -60,7 +65,7 @@ With a cap set and no readable meter, start refuses | `admission_cap` | The cap, or `none` | | `tail` | Tail reserve (scaled by the step factor once measured) plus stop overhead, held back from every admission | | `unit`, `meter`, `epoch` | The meter contract: `capy-tokens` (root counter), `claude-usage-tokens` (journal usage), `estimate-tokens`; a meter change opens a new epoch | -| `call` | The per-call charge: two Capy readings in consecutive calls, or the last Claude Code call's usage | +| `call` | The per-call charge: two Capy readings in consecutive calls (a prior older than the run's last reading is ignored), or the last Claude Code call's usage | | `call_floor`, `above_floor` | The host's per-call floor (see Delegation) and the charge above it, max(0, call − floor); the delegate rule compares `above_floor` | | `next`, `projected` | What the check was for and its projection: the prior times the run's factor for that kind | | `calibration` | The run's step and dispatch factors (see Calibration), or `prior` before the first measurement | @@ -80,8 +85,9 @@ gstack-ship-budget check --run --step 9 --next dispatch:specialist,red-team gstack-review-log --start review --reservation ``` -Each reader is priced at max(its projection, the run's measured mean for that role over settled -dispatches). When the whole batch does not fit, `check` admits the largest prefix that does and +Name one role per `DISPATCH:` line (six specialists are `specialist` six times): drift and the +measured means are per reader. Each reader is priced at max(its projection, the run's measured +mean per reader for that role over measured dispatches). When the whole batch does not fit, `check` admits the largest prefix that does and prints `ADMIT: k of n (remaining=)`: dispatch those k, then re-check the remaining roles once they settle. Only a batch whose first reader does not fit stops, and `close` flags remaining roles no later check requested (`DROPPED:`, status=incomplete until the owner accepts). @@ -91,7 +97,10 @@ every other dispatch reservation started in the same step, so reviewers admitted do not stay reserved. Name a reviewer that came back in your next check (`check ... --returned [,...]`), or, when other work comes first, `settle --reservation --reading ` as it returns so its cost is measured alone. A check that moves to the next step -settles every dispatch a reviewer started in the step before. +settles every dispatch a reviewer started in the step before. Only a dispatch settled with a +reading at its return is measured: `--finish` and a step change settle without one, so their +dispatches print `UNMEASURED:` and leave the factor alone (their interval also holds the parent's +own work). `close` audits after the fact: each review-log row of the run with no reservation gets one BUDGET_CHECK_SKIPPED line and the run ends status=incomplete for the owner to accept (`close --approved-by --words ""`); a reservation is never created after the fact. @@ -177,12 +186,17 @@ Claude-metered /ship completes. Each unit's `call_floor` is calibrated separatel evidence in `call_floor_why`; a floor is the largest 10,000 multiple at or below its smallest observed fresh call. -These constants are the prior. Each settled step or dispatch is measured on the next fresh -reading (a step from its check's reading to the next one; dispatches settled together as one -group, split by role), and prints +These constants are the prior. A dispatch is measured from its check's reading to the reading it +was settled with on return (dispatches settled at one reading split it per reader). A step is +measured at the next step change, from its check's reading minus the dispatches reserved inside +it, so the parent's interleaved calls stay in the step; a step holding an unmeasured dispatch is +not measured. Each measurement prints DRIFT: id= kind=step|dispatch next= projected= actual= ratio= +and each reservation that cannot be measured prints +`UNMEASURED: id= kind=step|dispatch next= reason=settled_without_reading|dispatch_unmeasured`. + From then on the run projects steps and dispatches with its own factor: the largest actual/prior ratio among its last three measurements of that kind, times 1.5. The step factor also scales the default tail reserve. Dispatches keep the prior until the first diff --git a/lib/ship-budget-cli.ts b/lib/ship-budget-cli.ts index e8bb1bd3f..a499d27d0 100644 --- a/lib/ship-budget-cli.ts +++ b/lib/ship-budget-cli.ts @@ -44,8 +44,13 @@ const meterFile = (l: Ledger) => path.join(l.dir, 'claude-meter.json'); /** Whether this command took a fresh reading (Claude Code reads its journals on every command). */ const freshReading = (l: Ledger, o: Opts) => l.meter.contract.meter === 'claude' || str(o, 'reading') !== undefined; -/** Take this command's reading under the run's meter contract; returns the per-call charge when known. */ -export function readMeter(l: Ledger, o: Opts, requireReading: boolean): number | null { +/** + * Take this command's reading under the run's meter contract; returns the + * per-call charge when known. On Capy that charge is --reading minus a + * --prior-reading taken in the previous call; a prior older than the run's last + * reading spans several calls, so it is ignored (call unknown) with a note. + */ +export function readMeter(l: Ledger, o: Opts, requireReading: boolean, notes: string[] = []): number | null { const reading = tokensOpt(o, 'reading'); const prior = tokensOpt(o, 'prior-reading'); const meter = l.meter.contract.meter; @@ -63,8 +68,14 @@ export function readMeter(l: Ledger, o: Opts, requireReading: boolean): number | return null; } if (prior !== undefined && prior > reading) throw budgetError('BUDGET_READING_REJECTED', `--prior-reading ${prior} is above --reading ${reading}`); + const last = l.meter.last; applyReading(l, reading, 'flag'); - return prior === undefined ? null : reading - prior; + if (prior === undefined) return null; + if (last !== null && prior < last) { + notes.push(`CALL_UNKNOWN: --prior-reading ${prior} is older than the run's last reading ${last}; it spans several calls, so the per-call charge is unknown (pass the reading your previous call returned)`); + return null; + } + return reading - prior; } /** The stop block: STOP, the resumable RECOVER line and GSTACK_RESULT incomplete; a child also writes its return. */ @@ -249,15 +260,17 @@ export function cmdCheck(argv: string[], ctx: Ctx): Outcome { const back = returned.map(id => l.reservations.find(r => r.id === id && r.kind === 'dispatch')); const bad = returned.filter((_, i) => !back[i]); if (bad.length) throw usageError(`--returned ${bad.join(',')} is not a dispatch reservation of run ${l.run}; fix: pass the RESERVATION ids check printed for the reviewers that came back`); - const call = readMeter(l, o, true); + const notes: string[] = []; + const call = readMeter(l, o, true, notes); + const fresh = freshReading(l, o); for (const r of l.reservations) { const done = r.kind === 'step' ? r.state === 'open' : r.kind === 'dispatch' && (back.includes(r) || (next.kind === 'step' && r.state === 'consumed')); - if (done) settleIn(l, r); + if (done) settleIn(l, r, { reading: fresh && back.includes(r) ? l.meter.last ?? undefined : undefined }); } - const drift = freshReading(l, o) ? measureSettled(l) : []; + const drift = fresh ? measureSettled(l, next.kind === 'step') : []; if (l.parent && next.kind === 'step' && SHIP_STEPS.indexOf(step as any) >= SHIP_STEPS.indexOf(l.parent.from as any)) l.steps_done = [...new Set([...(l.steps_done ?? []), step])]; const out = decide(l, step, next, call); - return { stdout: [...drift, ...out.stdout], exit: out.exit }; + return { stdout: [...notes, ...drift, ...out.stdout], exit: out.exit }; }); } @@ -270,8 +283,9 @@ export function cmdSettle(argv: string[], ctx: Ctx, release = false): Outcome { if (!r) throw usageError(`reservation ${id} is not in run ${l.run}`); if (release) { if (r.state === 'open') r.state = 'released'; return { stdout: [`RELEASED: id=${id} state=${r.state}`], exit: 0 }; } readMeter(l, o, false); - settleIn(l, r, { unknown: o.unknown === true, cal: loadCalibration(l.calibration) }); - return { stdout: [`SETTLED: id=${id} state=${r.state}`, ...(freshReading(l, o) ? measureSettled(l) : [])], exit: 0 }; + const fresh = freshReading(l, o); + settleIn(l, r, { unknown: o.unknown === true, cal: loadCalibration(l.calibration), reading: fresh ? l.meter.last ?? undefined : undefined }); + return { stdout: [`SETTLED: id=${id} state=${r.state}`, ...(fresh ? measureSettled(l, false) : [])], exit: 0 }; }); } diff --git a/lib/ship-budget.ts b/lib/ship-budget.ts index 0a8655b6d..a5bd9eb85 100644 --- a/lib/ship-budget.ts +++ b/lib/ship-budget.ts @@ -57,6 +57,7 @@ export interface Reservation { id: string; kind: 'step' | 'dispatch' | 'child'; step: string; next: string; roles?: string[]; tokens: number; at: string; state: 'open' | 'consumed' | 'settled' | 'released'; token?: string; started_at?: string; settled_at?: string; prior?: number; reading_at?: number | null; epoch?: number; actual?: number; unseen?: boolean; remaining?: string[]; + settled_reading?: number; unmeasured?: boolean; } export interface Sample { id: string; kind: 'step' | 'dispatch'; epoch: number; prior: number; projected: number; actual: number } export interface Handoff { @@ -189,26 +190,48 @@ export function projector(l: Ledger, cal: Calibration = loadCalibration(l.calibr } /** - * Measure settled reservations against the current (fresh) reading: a step - * from its reading to now; settled dispatches as one group from the earliest - * of their readings, split by role. A charge the meter cannot see (settled - * --unknown, or a Claude-metered outside voice) is never measured. Returns one - * DRIFT line per measurement. + * Measure settled reservations against the current (fresh) reading (GSTA-167). + * A dispatch is measured only to the reading it was settled with when it + * returned (settle --reading, check --returned): dispatches settled at the same + * reading form one group from the earliest of their readings, split per reader + * (one role per reader). One settled without a reading (review-log --finish, a + * step change) is UNMEASURED: its interval also holds the parent's own work. + * A step is measured on a step change, from its reading to now minus the + * dispatches reserved after it, so the parent's interleaved calls stay in the + * step; a window holding an unmeasured dispatch leaves the step unmeasured. A + * charge the meter cannot see (settled --unknown, or a Claude-metered outside + * voice) is never measured. Returns one DRIFT or UNMEASURED line each. */ -export function measureSettled(l: Ledger): string[] { +export function measureSettled(l: Ledger, stepChange = true): string[] { const last = l.meter.last; if (last === null) return []; - const pending = l.reservations.filter(r => r.state === 'settled' && r.kind !== 'child' && r.actual === undefined && !r.unseen + const pending = l.reservations.filter(r => r.state === 'settled' && r.kind !== 'child' && r.actual === undefined && !r.unseen && !r.unmeasured && typeof r.reading_at === 'number' && r.epoch === l.meter.epoch && last >= r.reading_at); - const dispatches = pending.filter(r => r.kind === 'dispatch'); - const from = Math.min(...dispatches.map(r => r.reading_at!)); - const roles = dispatches.reduce((n, r) => n + (r.roles?.length ?? 1), 0); - return pending.map(r => { - const actual = r.kind === 'dispatch' ? Math.round(((last - from) * (r.roles?.length ?? 1)) / roles) : last - r.reading_at!; + const lines: string[] = []; + const measure = (r: Reservation, actual: number) => { r.actual = actual; (l.samples ??= []).push({ id: r.id, kind: r.kind as Sample['kind'], epoch: l.meter.epoch, prior: r.prior ?? r.tokens, projected: r.tokens, actual }); - return `DRIFT: id=${r.id} kind=${r.kind} next=${r.next} projected=${r.tokens} actual=${actual} ratio=${(actual / Math.max(1, r.tokens)).toFixed(2)}`; - }); + lines.push(`DRIFT: id=${r.id} kind=${r.kind} next=${r.next} projected=${r.tokens} actual=${actual} ratio=${(actual / Math.max(1, r.tokens)).toFixed(2)}`); + }; + const unmeasured = (r: Reservation, reason: string) => { + r.unmeasured = true; + lines.push(`UNMEASURED: id=${r.id} kind=${r.kind} next=${r.next} reason=${reason}`); + }; + const dispatches = pending.filter(r => r.kind === 'dispatch'); + for (const r of dispatches) if (r.settled_reading === undefined) unmeasured(r, 'settled_without_reading'); + for (const end of new Set(dispatches.filter(r => !r.unmeasured).map(r => r.settled_reading!))) { + const group = dispatches.filter(r => r.settled_reading === end); + const from = Math.min(...group.map(r => r.reading_at!)); + const readers = group.reduce((n, r) => n + (r.roles?.length ?? 1), 0); + for (const r of group) measure(r, Math.round(((end - from) * (r.roles?.length ?? 1)) / readers)); + } + if (!stepChange) return lines; + for (const r of pending.filter(x => x.kind === 'step')) { + const inside = l.reservations.slice(l.reservations.indexOf(r) + 1).filter(d => d.kind === 'dispatch' && d.state !== 'released' && !d.unseen && d.epoch === l.meter.epoch); + if (inside.some(d => d.actual === undefined)) { unmeasured(r, 'dispatch_unmeasured'); continue; } + measure(r, Math.max(0, last - r.reading_at! - inside.reduce((n, d) => n + d.actual!, 0))); + } + return lines; } /** Steps from..to inclusive plus their usual reviewer batches at the prior: a child's segment estimate. */ @@ -297,10 +320,11 @@ export function reserveIn(l: Ledger, r: Omit } /** Settle at completion: the meter already holds the spend; estimate and unseen charges are booked here. */ -export function settleIn(l: Ledger, r: Reservation, opts: { unknown?: boolean; cal?: Calibration } = {}): void { +export function settleIn(l: Ledger, r: Reservation, opts: { unknown?: boolean; cal?: Calibration; reading?: number } = {}): void { if (r.state === 'settled' || r.state === 'released') return; r.state = 'settled'; r.settled_at = new Date().toISOString(); + if (opts.reading !== undefined) r.settled_reading = opts.reading; if (opts.unknown) { l.unknown += r.tokens; r.unseen = true; return; } if (l.meter.contract.meter === 'estimate') { l.estimate_used += r.tokens; return; } const unseen = l.meter.contract.meter === 'claude' ? (r.roles ?? []).filter(x => x === 'outside-voice').length : 0; diff --git a/lib/ship-handoff.ts b/lib/ship-handoff.ts index fba024659..6fa465a64 100644 --- a/lib/ship-handoff.ts +++ b/lib/ship-handoff.ts @@ -51,6 +51,17 @@ export function recordSection(text: string, name: string): string[] | undefined return lines.slice(at, end < 0 ? lines.length : end); } +/** The ship sections each step reads (ship/sections/manifest.json); conditional ones (apple-release, measure, greptile, shared-code-reuse, handoff) load only when their trigger fires. */ +const STEP_SECTIONS: Record = { + '4': ['tests'], '5': ['tests'], '6': ['tests'], '6.5': ['greptile-early'], '7': ['test-coverage'], '8': ['plan-completion'], '9': ['review-army'], + '11': ['adversarial'], '13': ['changelog'], '14.5': ['documentation'], '18': ['pr-body', 'capy-pr'], '19': ['pr-body', 'capy-pr'], +}; + +/** The sections a child running Steps from..to reads (GSTA-167): its own steps' only. */ +export function segmentSections(from: string, to: string): string[] { + return [...new Set(SHIP_STEPS.slice(SHIP_STEPS.indexOf(from as any), SHIP_STEPS.indexOf(to as any) + 1).flatMap(s => STEP_SECTIONS[s] ?? []))]; +} + function brief(l: Ledger, h: Omit, pins: Record, recordText: string, o: Record): string { const reviews = recordSection(recordText, 'Reviews') ?? []; const tokens = [...new Set(reviews.join('\n').match(/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/g) ?? [])]; @@ -65,6 +76,8 @@ function brief(l: Ledger, h: Omit, pins: Record block.trail ? `2. Run \`gstack-decision-trail start ${block.trail} \` before any trail write.` : '2. No decision trail was named.', `3. Run /ship --continue for Steps ${h.from}-${h.to} only, under the printed child run: \`check\` at every boundary and before every dispatch batch, obeying \`verdict=\`. The context rule never stops you: \`verdict=delegate\` hands your remaining steps to a fresh child the same way (ship/sections/budget-handoff.md), and a \`CONTEXT_HIGH:\` note means continue. This run is at chain depth ${h.depth} of ${MAX_CHAIN_DEPTH}.`, block.launch_file ? `4. Source the launch file \`${block.launch_file}\` in every shell that runs a gstack command (its path only; never print its values).` : '4. No GSTACK_LAUNCH_FILE was named.', '', + '## Read', '', + `Only ship/SKILL.md's step list and, under ship/sections/, ${[...segmentSections(h.from, h.to), 'budget-handoff'].map(s => s + '.md').join(', ')}; a conditional section only when its manifest.json trigger fires. Never read ship/sections/preamble.md in full or docs/ship-budget.md: budget-handoff.md is the whole budget procedure.`, '', '## Record', '', `You are the only writer of \`${l.record}\` during this segment. It keeps its six headings (${RECORD_HEADINGS.join(', ')}). Original start tokens: ${tokens.join(', ') || 'none'}.`, '', '## Decisions and approvals in scope (verbatim from the record)', '', ...(recordSection(recordText, 'Decisions') ?? ['(none recorded)']), '', '## Budget', '', `Sub-cap ${h.sub_cap} ${block.unit} (meter=${block.meter}); the parent keeps a ${l.tail} tail reserve. Ledger: ${l.file}.`, '', diff --git a/ship/sections/budget-handoff.md b/ship/sections/budget-handoff.md index b828deada..fd7a3d2e2 100644 --- a/ship/sections/budget-handoff.md +++ b/ship/sections/budget-handoff.md @@ -9,15 +9,15 @@ Field-by-field reference: docs/ship-budget.md. Exit 0 continue or delegate, 3 st 2 usage; any non-zero exit means the next step does not start. **Meter.** Read one per command: -- Capy: in one execute program return only `(await tools.thread.get({ thread_id: "this" })).cost.tokens` - and pass it as `--reading `. For the per-call charge, read it again in your next - call and pass the earlier value as `--prior-reading `. +- Capy: read `(await tools.thread.get({ thread_id: "this" })).cost.tokens` in the same program as the + `gstack-ship-budget` command, never its own call: `--reading `. `--prior-reading ` (per-call + charge) is your previous call's reading; one older than the run's last reading is ignored. - Claude Code: `start --session `, or echo the nonce the refusal prints once and rerun `start --nonce `; the helper reads the journals itself (never `--reading`). - Elsewhere: `--meter estimate` (printed `meter=estimate`), only when the owner accepts it. **Every boundary.** `gstack-ship-budget check --run --step --next --reading `. -Before each dispatch batch, reserve the batch as one sum, then pass the printed id once +Before each dispatch batch, reserve the batch as one sum, one role per `DISPATCH:` line, then pass the printed id once (`READING` is the current counter reading; `RESERVATION_ID` is the id the check prints): ```bash @@ -26,10 +26,10 @@ Before each dispatch batch, reserve the batch as one sum, then pass the printed ``` On `ADMIT: k of n (remaining=)`, dispatch only those k and re-check the remaining roles -after they settle; `close` flags roles never re-checked. `--finish` settles the reservation and the pass's other dispatch reservations. Admitting reviewers one at -a time, name each one that came back in the next check's `--returned `. A dispatch with no `review-log --start` settles with -`gstack-ship-budget settle --run --reservation ` when it returns -(`--unknown` when the meter cannot see it), or `release` if it never ran. +after they settle; `close` flags roles never re-checked. `--finish` settles the reservation and the pass's other dispatch reservations. +Settle each reader when it returns: the next check's `--returned `, or +`gstack-ship-budget settle --run --reservation --reading ` (`--unknown` when the meter +cannot see it); `release` one that never ran. Only these are measured: `--finish` or a step change settles without a reading. **verdict=stop.** Print the STOP, RECOVER and GSTACK_RESULT lines as your last output and end status=incomplete. Raising the cap is the owner's call: `/ship --resume diff --git a/ship/sections/budget-handoff.md.tmpl b/ship/sections/budget-handoff.md.tmpl index 9dfcee283..eba10e5d5 100644 --- a/ship/sections/budget-handoff.md.tmpl +++ b/ship/sections/budget-handoff.md.tmpl @@ -7,15 +7,15 @@ Field-by-field reference: docs/ship-budget.md. Exit 0 continue or delegate, 3 st 2 usage; any non-zero exit means the next step does not start. **Meter.** Read one per command: -- Capy: in one execute program return only `(await tools.thread.get({ thread_id: "this" })).cost.tokens` - and pass it as `--reading `. For the per-call charge, read it again in your next - call and pass the earlier value as `--prior-reading `. +- Capy: read `(await tools.thread.get({ thread_id: "this" })).cost.tokens` in the same program as the + `gstack-ship-budget` command, never its own call: `--reading `. `--prior-reading ` (per-call + charge) is your previous call's reading; one older than the run's last reading is ignored. - Claude Code: `start --session `, or echo the nonce the refusal prints once and rerun `start --nonce `; the helper reads the journals itself (never `--reading`). - Elsewhere: `--meter estimate` (printed `meter=estimate`), only when the owner accepts it. **Every boundary.** `gstack-ship-budget check --run --step --next --reading `. -Before each dispatch batch, reserve the batch as one sum, then pass the printed id once +Before each dispatch batch, reserve the batch as one sum, one role per `DISPATCH:` line, then pass the printed id once (`READING` is the current counter reading; `RESERVATION_ID` is the id the check prints): ```bash @@ -24,10 +24,10 @@ Before each dispatch batch, reserve the batch as one sum, then pass the printed ``` On `ADMIT: k of n (remaining=)`, dispatch only those k and re-check the remaining roles -after they settle; `close` flags roles never re-checked. `--finish` settles the reservation and the pass's other dispatch reservations. Admitting reviewers one at -a time, name each one that came back in the next check's `--returned `. A dispatch with no `review-log --start` settles with -`gstack-ship-budget settle --run --reservation ` when it returns -(`--unknown` when the meter cannot see it), or `release` if it never ran. +after they settle; `close` flags roles never re-checked. `--finish` settles the reservation and the pass's other dispatch reservations. +Settle each reader when it returns: the next check's `--returned `, or +`gstack-ship-budget settle --run --reservation --reading ` (`--unknown` when the meter +cannot see it); `release` one that never ran. Only these are measured: `--finish` or a step change settles without a reading. **verdict=stop.** Print the STOP, RECOVER and GSTACK_RESULT lines as your last output and end status=incomplete. Raising the cap is the owner's call: `/ship --resume diff --git a/test/ship-budget-projection.test.ts b/test/ship-budget-projection.test.ts index a0fd46ad8..8411a618b 100644 --- a/test/ship-budget-projection.test.ts +++ b/test/ship-budget-projection.test.ts @@ -158,7 +158,7 @@ describe('dispatch reservations settle when the reviewer returns (GST-101)', () expect(bad.err).toContain('--returned'); }); - test('--finish settles every consumed dispatch reservation of the pass, and the next reading measures them as one group', () => { + test('--finish settles every consumed dispatch reservation of the pass; settled without a reading, they stay unmeasured (GSTA-167)', () => { const h = fresh(); const s = started(h, '80M'); const tokens: string[] = []; @@ -175,7 +175,8 @@ describe('dispatch reservations settle when the reviewer returns (GST-101)', () expect(dispatches().map((r: any) => r.state)).toEqual(['settled', 'settled', 'settled']); const next = h.sb(['check', '--run', s.run, '--step', '9', '--next', '10', '--reading', String(reading)]); expect(h.field(next.out, 'BUDGET', 'reserved')).toBe('0'); - expect(dispatches().map((r: any) => r.actual)).toEqual([500_000, 500_000, 500_000]); + expect(dispatches().map((r: any) => [r.actual, r.unmeasured])).toEqual([[undefined, true], [undefined, true], [undefined, true]]); + expect(next.out.split('\n').filter(l => l.startsWith('UNMEASURED:') && l.includes('kind=dispatch'))).toHaveLength(3); }); test('moving to the next step settles dispatch reservations a reviewer consumed in the step before', () => { @@ -185,7 +186,8 @@ describe('dispatch reservations settle when the reviewer returns (GST-101)', () expect(h.reviewLog(['--start', 'adversarial-review', '--reservation', id]).code).toBe(0); const c = h.sb(['check', '--run', s.run, '--step', '11', '--next', '11.5', '--reading', '600000']); expect(c.code).toBe(0); - expect(h.ledger(s.ledger).reservations.find((r: any) => r.id === id)).toMatchObject({ state: 'settled', actual: 600_000 }); + expect(h.ledger(s.ledger).reservations.find((r: any) => r.id === id)).toMatchObject({ state: 'settled', unmeasured: true }); + expect(c.out).toContain(`UNMEASURED: id=${id} kind=dispatch next=dispatch:outside-voice reason=settled_without_reading`); }); test('close prints projected against actual for the run', () => { diff --git a/test/ship-budget-reader-drift.test.ts b/test/ship-budget-reader-drift.test.ts new file mode 100644 index 000000000..23c1e0dcd --- /dev/null +++ b/test/ship-budget-reader-drift.test.ts @@ -0,0 +1,173 @@ +/** + * GSTA-167 PR 3a: dispatch drift is measured per reader and only to the reading + * a dispatch was settled with when it returned, never across the parent's own + * work; the parent's interleaved calls stay in the step's sample. The per-call + * charge takes a prior reading from the previous call only. Replays + * gstack-evals #36 (a six-reader batch taught a 2.041 factor and priced the Red + * Team at 15.17M, a false STOP with 14.4M of sub-cap unused) and #34 (the Red + * Team's reservation interval covered the child's own review and QA, and a + * prior reading six calls old fired CONTEXT_HIGH at call=1068386); evidence in + * handoffs/GSTA-167/replay-36.md and replay-34.md in the GStack Minder drive. + */ +import { afterEach, describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { loadCalibration, unitConstants } from '../lib/ship-budget'; +import { budgetHarness, type Harness } from './helpers/ship-budget-harness'; + +const cal = loadCalibration(); +const K = unitConstants(cal, 'capy-tokens'); +const harnesses: Harness[] = []; +const fresh = () => { const h = budgetHarness(); harnesses.push(h); return h; }; +afterEach(() => { while (harnesses.length) harnesses.pop()!.cleanup(); }); + +/** Opus medium readers on #34/#36 cost about 1.7M each (replay-36: ~1.69M per specialist). */ +const READER = 1_690_000; +const SIX = Array(6).fill('specialist'); +const roundUp = (n: number) => Math.ceil(n / cal.round_to) * cal.round_to; +/** The per-reader price after a measured batch: the prior times actual/(prior x readers) times the headroom. */ +const priced = (actual: number, readers: number) => roundUp(K.per_dispatch * ((actual / (K.per_dispatch * readers)) * cal.headroom)); + +function start(h: Harness, cap: number, reading = 0) { + const r = h.sb(['start', '--record', h.record(), '--token-cap', String(cap), '--reading', String(reading), '--reserve', '0']); + expect(r.code, r.out + r.err).toBe(0); + return { run: h.field(r.out, 'BUDGET_RUN', 'id'), ledger: h.field(r.out, 'BUDGET_RUN', 'ledger') }; +} +function ok(h: Harness, args: string[]) { + const r = h.sb(args); + expect(r.code, r.out + r.err).toBe(0); + return r; +} +const check = (h: Harness, run: string, step: string, next: string, reading: number, extra: string[] = []) => + ok(h, ['check', '--run', run, '--step', step, '--next', next, '--reading', String(reading), ...extra]); + +describe('per-reader dispatch drift (#36)', () => { + test('#36 replay: six readers measured at their return price the Red Team per reader, so it is admitted instead of a 15.17M STOP', () => { + const h = fresh(); + const cap = 35_244_164; + const s = start(h, cap); + let reading = 10_710_000; + const first = check(h, s.run, '9', 'dispatch:' + SIX.join(','), reading); + expect(first.out).toMatch(/^ADMIT: 2 of 6 /m); + const r1 = h.field(first.out, 'RESERVATION', 'id'); + expect(h.reviewLog(['--start', 'review', '--reservation', r1]).code).toBe(0); + reading += 2 * READER; + const rest = check(h, s.run, '9', 'dispatch:' + SIX.slice(2).join(','), reading, ['--returned', r1]); + expect(rest.out).toMatch(new RegExp(`^DRIFT: id=${r1} kind=dispatch next=dispatch:specialist,specialist projected=${2 * K.per_dispatch} actual=${2 * READER} ratio=0\\.23$`, 'm')); + expect(rest.out).not.toContain('ADMIT:'); + expect(Number(h.field(rest.out, 'BUDGET', 'projected'))).toBe(4 * priced(2 * READER, 2)); + const r2 = h.field(rest.out, 'RESERVATION', 'id'); + reading += 4 * READER; + const red = check(h, s.run, '9', 'dispatch:red-team', reading, ['--returned', r2]); + expect(h.field(red.out, 'BUDGET', 'verdict')).toBe('continue'); + expect(Number(h.field(red.out, 'BUDGET', 'projected'))).toBe(priced(2 * READER, 2)); + expect(Number(h.field(red.out, 'BUDGET', 'projected'))).toBeLessThan(3_000_000); + expect(red.out).not.toContain('STOP:'); + }); + + test('a measured role prices its next reader at the larger of the per-reader projection and its per-reader mean', () => { + const h = fresh(); + const s = start(h, 200_000_000); + const a = h.field(check(h, s.run, '9', 'dispatch:' + SIX.join(','), 0).out, 'RESERVATION', 'id'); + const b = check(h, s.run, '9', 'dispatch:red-team', 6 * READER, ['--returned', a]); + const r = h.ledger(s.ledger).reservations.find((x: any) => x.id === a); + expect(r.actual).toBe(6 * READER); + expect(Number(h.field(b.out, 'BUDGET', 'projected'))).toBe(priced(6 * READER, 6)); + const red = h.field(b.out, 'RESERVATION', 'id'); + const heavy = 3_910_000; + const c = check(h, s.run, '9', 'dispatch:red-team,specialist', 6 * READER + heavy, ['--returned', red]); + const factor = Math.max(READER / K.per_dispatch, heavy / K.per_dispatch) * cal.headroom; + expect(Number(h.field(c.out, 'BUDGET', 'projected'))).toBe(Math.max(roundUp(K.per_dispatch * factor), heavy) + Math.max(roundUp(K.per_dispatch * factor), READER)); + }); +}); + +describe('a dispatch is measured to its return, never across the parent\'s own work (#34)', () => { + test('#34 replay: a Red Team settled by the step change after the child\'s review and QA is UNMEASURED, so the voices keep the per-reader factor', () => { + const h = fresh(); + const s = start(h, 200_000_000); + check(h, s.run, '8', '9', 3_500_000); + const specialists = h.field(check(h, s.run, '9', 'dispatch:' + SIX.join(','), 4_000_000).out, 'RESERVATION', 'id'); + expect(h.reviewLog(['--start', 'review', '--reservation', specialists]).code).toBe(0); + let reading = 4_000_000 + 6 * READER; + const redCheck = check(h, s.run, '9', 'dispatch:red-team', reading, ['--returned', specialists]); + const red = h.field(redCheck.out, 'RESERVATION', 'id'); + expect(h.reviewLog(['--start', 'review', '--reservation', red]).code).toBe(0); + reading += READER + 3_000_000; + const step = check(h, s.run, '9', '10', reading); + expect(step.out).toMatch(new RegExp(`^UNMEASURED: id=${red} kind=dispatch next=dispatch:red-team reason=settled_without_reading$`, 'm')); + expect(step.out).not.toMatch(new RegExp(`^DRIFT: id=${red} `, 'm')); + expect(step.out).toMatch(/^UNMEASURED: id=\S+ kind=step next=9 reason=dispatch_unmeasured$/m); + expect(h.ledger(s.ledger).reservations.find((x: any) => x.id === red)).toMatchObject({ state: 'settled', unmeasured: true }); + check(h, s.run, '10', '11', reading + 400_000); + const voices = check(h, s.run, '11', 'dispatch:adversarial-native,outside-voice', reading + 800_000); + expect(Number(h.field(voices.out, 'BUDGET', 'projected'))).toBe(2 * priced(6 * READER, 6)); + }); + + test('settled with a reading at its return, the Red Team measures only its own interval; the parent\'s QA after it lands in Step 9\'s sample', () => { + const h = fresh(); + const s = start(h, 200_000_000); + check(h, s.run, '8', '9', 3_500_000); + const red = h.field(check(h, s.run, '9', 'dispatch:red-team', 4_000_000).out, 'RESERVATION', 'id'); + expect(h.reviewLog(['--start', 'review', '--reservation', red]).code).toBe(0); + const settled = ok(h, ['settle', '--run', s.run, '--reservation', red, '--reading', String(4_000_000 + READER)]); + expect(settled.out).toMatch(new RegExp(`^DRIFT: id=${red} kind=dispatch next=dispatch:red-team projected=${K.per_dispatch} actual=${READER} ratio=0\\.23$`, 'm')); + const step = check(h, s.run, '9', '10', 4_000_000 + READER + 3_000_000); + expect(step.out).toMatch(/^DRIFT: id=\S+ kind=step next=9 projected=\d+ actual=3500000 ratio=\d+\.\d\d$/m); + expect(step.out).not.toContain('UNMEASURED:'); + }); + + test('--finish settles the pass without a reading, so its dispatches are UNMEASURED and the factor stays at the prior', () => { + const h = fresh(); + const s = start(h, 200_000_000); + const id = h.field(check(h, s.run, '9', 'dispatch:' + SIX.join(','), 0).out, 'RESERVATION', 'id'); + const token = h.reviewLog(['--start', 'review', '--reservation', id]).out.trim(); + const row = JSON.stringify({ skill: 'review', timestamp: new Date().toISOString(), status: 'clean', issues_found: 0, critical: 0, informational: 0 }); + expect(h.reviewLog([row, '--finish', token]).code).toBe(0); + const next = check(h, s.run, '9', 'dispatch:red-team', 6 * READER + 5_000_000); + expect(next.out).toMatch(new RegExp(`^UNMEASURED: id=${id} kind=dispatch `, 'm')); + expect(h.field(next.out, 'BUDGET', 'calibration')).toEndWith('dispatch:prior'); + expect(Number(h.field(next.out, 'BUDGET', 'projected'))).toBe(K.per_dispatch); + }); +}); + +describe('the per-call charge takes a prior reading from the previous call only (#34)', () => { + test('a prior reading older than the run\'s last reading is ignored: call=unknown, never a six-call span', () => { + const h = fresh(); + const s = start(h, 60_000_000, 1_000_000); + check(h, s.run, '4', '5', 8_900_000); + const stale = check(h, s.run, '5', '6', 9_162_515, ['--prior-reading', '8094129']); + expect(h.field(stale.out, 'BUDGET', 'call')).toBe('unknown'); + expect(stale.out).toMatch(/^CALL_UNKNOWN: --prior-reading 8094129 is older than the run's last reading 8900000; /m); + expect(stale.out).not.toContain('1068386'); + const fine = check(h, s.run, '6', '6.5', 9_450_000, ['--prior-reading', '9162515']); + expect(h.field(fine.out, 'BUDGET', 'call')).toBe('287485'); + expect(fine.out).not.toContain('CALL_UNKNOWN'); + }); + + test('#34 replay: a child passing the parent\'s handoff reading as its prior gets call=unknown, not 1068386', () => { + const h = fresh(); + const s = start(h, 400_000_000, 2_110_477); + const ho = ok(h, ['handoff', '--run', s.run, '--from', '7', '--to', '11.5', '--base', 'origin/main', '--reading', '8094129']); + const brief = ho.out.match(/^HANDOFF: (\S+) /m)![1]!; + const v = ok(h, ['verify-handoff', '--in', brief, '--reading', '8800000']); + const child = h.field(v.out, 'BUDGET_RUN', 'id'); + const c = check(h, child, '6.5', '7', 9_162_515, ['--prior-reading', '8094129']); + expect(h.field(c.out, 'BUDGET', 'call')).toBe('unknown'); + expect(c.out).not.toContain('call=1068386'); + }); +}); + +describe('the budget section states both contracts', () => { + const tmpl = fs.readFileSync(path.join(import.meta.dir, '..', 'ship', 'sections', 'budget-handoff.md.tmpl'), 'utf8'); + test('the meter read happens inside the same program as the gstack-ship-budget command, never as its own call', () => { + expect(tmpl).not.toContain('pass the earlier value'); + expect(tmpl).toMatch(/same (?:execute )?program as the\s+(?:`)?gstack-ship-budget(?:`)? command, never (?:as )?its own call/); + expect(tmpl).toMatch(/--prior-reading[\s\S]{0,200}previous call/); + expect(tmpl).toMatch(/older than the run's last reading is ignored/); + }); + test('a batch names one role per reader and each reader is settled with a reading when it returns', () => { + expect(tmpl).toMatch(/one role per (?:reader|`DISPATCH:` line)/); + expect(tmpl).toMatch(/Settle each reader when it returns[\s\S]{0,200}--reservation --reading /); + expect(tmpl).toMatch(/Only these are measured: `--finish` or a step change settles without a reading/); + }); +}); diff --git a/test/ship-budget.test.ts b/test/ship-budget.test.ts index 1f4aeb023..c26066937 100644 --- a/test/ship-budget.test.ts +++ b/test/ship-budget.test.ts @@ -92,8 +92,8 @@ describe('reservations', () => { const c = h.sb(['check', '--run', s.run, '--step', '9', '--next', 'dispatch:specialist', '--reading', '0']); const id = h.field(c.out, 'RESERVATION', 'id'); expect(Number(h.field(c.out, 'RESERVATION', 'tokens'))).toBe(K.per_dispatch); - h.sb(['settle', '--run', s.run, '--reservation', id]); const spent = 2 * K.per_dispatch; + h.sb(['settle', '--run', s.run, '--reservation', id, '--reading', String(spent)]); const next = h.sb(['check', '--run', s.run, '--step', '9', '--next', 'dispatch:red-team', '--reading', String(spent)]); expect(Number(h.field(next.out, 'BUDGET', 'used'))).toBe(spent); const projected = Number(h.field(next.out, 'BUDGET', 'projected')); diff --git a/test/ship-handoff-brief.test.ts b/test/ship-handoff-brief.test.ts new file mode 100644 index 000000000..33be61e5e --- /dev/null +++ b/test/ship-handoff-brief.test.ts @@ -0,0 +1,54 @@ +/** + * GSTA-167 PR 3a: a handoff child reads only its segment's sections. On #34 + * and #36 the Steps 7-11.5 child booted at 1.29M by reading the whole ship + * skill, preamble.md and docs/ship-budget.md before its first step; the brief + * now names ship/SKILL.md's step list, the segment's sections and + * budget-handoff.md, and nothing else. + */ +import { afterEach, describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import { segmentSections } from '../lib/ship-handoff'; +import { budgetHarness, type Harness } from './helpers/ship-budget-harness'; + +const harnesses: Harness[] = []; +const fresh = () => { const h = budgetHarness(); harnesses.push(h); return h; }; +afterEach(() => { while (harnesses.length) harnesses.pop()!.cleanup(); }); + +function briefFor(from: string, to: string): string { + const h = fresh(); + const s = h.sb(['start', '--record', h.record(), '--token-cap', '400M', '--reading', '98000']); + expect(s.code, s.out + s.err).toBe(0); + const ho = h.sb(['handoff', '--run', h.field(s.out, 'BUDGET_RUN', 'id'), '--from', from, '--to', to, '--base', 'origin/main', '--reading', '1000000']); + expect(ho.code, ho.out + ho.err).toBe(0); + return fs.readFileSync(ho.out.match(/^HANDOFF: (\S+) /m)![1]!, 'utf8'); +} +const readSection = (text: string) => text.slice(text.indexOf('## Read'), text.indexOf('\n## ', text.indexOf('## Read') + 1)); + +describe('the handoff brief names only the segment\'s sections', () => { + test('Steps 7-11.5: the step list plus test-coverage, plan-completion, review-army and adversarial', () => { + const read = readSection(briefFor('7', '11.5')); + expect(read).toContain("ship/SKILL.md's step list"); + for (const s of ['test-coverage.md', 'plan-completion.md', 'review-army.md', 'adversarial.md', 'budget-handoff.md']) expect(read).toContain(s); + for (const s of ['changelog.md', 'documentation.md', 'pr-body.md', 'capy-pr.md', 'tests.md']) expect(read).not.toContain(s); + expect(read).toMatch(/Never read ship\/sections\/preamble\.md in full or docs\/ship-budget\.md/); + }); + + test('Steps 12-22: the step list plus changelog, documentation, pr-body and capy-pr', () => { + const read = readSection(briefFor('12', '22')); + for (const s of ['changelog.md', 'documentation.md', 'pr-body.md', 'capy-pr.md', 'budget-handoff.md']) expect(read).toContain(s); + for (const s of ['test-coverage.md', 'plan-completion.md', 'review-army.md', 'adversarial.md']) expect(read).not.toContain(s); + expect(read).toMatch(/Never read ship\/sections\/preamble\.md in full or docs\/ship-budget\.md/); + }); + + test('a grandchild segment names only the sections of its own steps', () => { + expect(segmentSections('9', '11.5')).toEqual(['review-army', 'adversarial']); + expect(segmentSections('19', '22')).toEqual(['pr-body', 'capy-pr']); + expect(segmentSections('1', '22')).toEqual(['tests', 'greptile-early', 'test-coverage', 'plan-completion', 'review-army', 'adversarial', 'changelog', 'documentation', 'pr-body', 'capy-pr']); + }); + + test('every section the brief can name is a registered ship section', () => { + const manifest = JSON.parse(fs.readFileSync(`${import.meta.dir}/../ship/sections/manifest.json`, 'utf8')); + const ids = new Set(manifest.sections.map((s: any) => s.id)); + for (const id of segmentSections('1', '22')) expect(ids.has(id)).toBe(true); + }); +}); diff --git a/test/size-ledger/ship/gsta-167-handoff-floor.json b/test/size-ledger/ship/gsta-167-handoff-floor.json new file mode 100644 index 000000000..c9a798034 --- /dev/null +++ b/test/size-ledger/ship/gsta-167-handoff-floor.json @@ -0,0 +1,4 @@ +{ + "bytes": 73, + "why": "GSTA-167 PR 3a: budget-handoff meter read inside the budget command's program, one role per DISPATCH line, settle readers with a reading at return" +}