v1.92.50.0 wave(autoplan): #3781 #3743 #3751 #3762 (#3788)

* fix(eval-plan): per-arm pilot projection and --seat-metered arms (#3653)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): guard tolerates Claude Code's cwd-reset footer and recovers after an unreadable init (#3651)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (journal reader strips the cwd-reset footer; an init with no text block no longer locks the session)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: journal reader skips a failed or unreadable init, as invocation() does

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(autoplan): --ceo-only runs the CEO review alone with zero questions (GST-537)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(eval-plan): all-seat pilot, single-attempt arm and --seat-metered CLI wiring

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(eval-plan): summed deviation adds arm deviations, matching the summed band (arms share rows)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (test pins an unpriced seat-metered arm)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (seat-metered declaration fixed per --out, price row still required, a real charge settles as spent)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: classify autoplan-ceo-only in the Codex host metadata

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: a seat-metered arm that settles a charge is projected as a priced arm

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: select autoplan-ceo-only on the skills it runs; say it needs Claude Code

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(skills): /eval-plan --seat-metered and per-arm projection

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(eval-plan): skill prose stays under its context budget and carries no $N token

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: run autoplan-ceo-only in the marathon lane

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(skills): a seat-metered arm leaves the seat set once it settles a charge

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: changelog entry for /autoplan --ceo-only

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): init_failed covers unreadable init output and the cwd-reset line (#3651)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: count the autoplan-ceo-only paid file in the shard inventory

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(autoplan): #3651 tests pass the test-code type ratchet

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(dx): CAP finding titles carry a one-line description (GBRA-223 claim 2)

The DX error contract told both autoplan DX voices to title capped findings
exactly CAP4:<path>:<item>, so gates, tasks and reconcile saw a bare code.
The title is now CAP4:<path>:<item> <what fails>; the grader already parses
that shape.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): reconcile pairs same-file findings that carry no line (GBRA-223 claim 1)

Plan reviews name files, rarely lines, and reconcile associated by location
only when both findings had a line, so findings both voices raised on one
file were filed new and native-only. A same-file pair with a missing line is
now associated as unverified (basis file), never confirmed; a CAP code no
longer counts in title comparison. DRIVER.md names submit --reconcile.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): open Critical and High findings reach the gate as apply items (GBRA-223 claim 3)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a spend cap binds on every unattended host; --estimate-usd persists; DRIVER names --usage-usd (GBRA-223 claim 4)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): run binds complete api results, names a truncated response, --max-tokens; DRIVER documents run (GBRA-223 claim 6)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(autoplan): carry an item-by-item answer when the later gate asks nothing new (GBRA-223 claim 7)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(dx): trim the compact error contract so plan-devex-review stays under its size pin

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pre-landing review fixes (p1b applies finding fixes, api attempts reserve no host default, cross-file title pairing, paired severity counted)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: review fixes (priced api reservations, no file-only twins, two-pass pairing, partial replies keep p1-dependent defaults open, outside refusal degrades)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* P7: record whose words answered a gate (--words-by agent:<id>), never carry an agent's own words, import each row under its own answerer (GEVA-2)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: review pass 2 (p1b needs --rec-hash after recommend; api calls send the output limit they reserve for)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: fragment for the /autoplan field-report wave (GST-672)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: captain record notes --words-by agent attribution

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: follow finding apply items in approval-class and Captain flow tests; bind run-api limit to DEFAULT_MAX_TOKENS

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* wave autoplan: lower the gemini plan-devex-review pin to its exact size after #3781's DX trim

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): CAP titles on different paths never match by title in reconcile or confirm rounds (wave review)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a truncated api call settles its cost and a retry is admitted afresh against the cap

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): an applied finding stays open until the revised plan is reviewed; a shared location alone never twins two findings

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(approval): a plan gate answered in an agent's own words is not owner approval

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(gate): a new answer replaces the earlier answer's authorship and words

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a failed api call with no usage keeps its attempt open; reservations price the bound prompt (wave fix-check)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a spend cap fails closed: only a call refused before sending keeps its attempt; a sent call with unknown usage settles at its reservation

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): the 16384 output default applies to both providers; changelog for the wave's integrator fixes

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): how run settles a failed api call; changelog scopes it to the native voice

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(autoplan): a spend cap fails closed on every exit of run; an unpriced model reserves the default once a cap is set

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): every sent call settles; an unpriced model under a cap reserves the default

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(autoplan): DRIVER and help say a cap reserves in any session kind and how run settles a sent call

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-11-02 01:35:47 +00:00
1 parent 7ab354af94
commit 90a170c941
70 files changed
+1769 -249

No files matched your search

+40
View File
@@ -1,5 +1,45 @@
# Changelog
## [1.92.50.0] - 2026-11-01
### Fixed
- **`/autoplan` no longer locks a Claude Code session after a snapshot init run as `cd <dir> && …` (#3651).** When that `cd` leaves the session's folder, Claude Code adds a `Shell cwd was reset to …` line to the command's output. The guard now removes exactly that one line before reading the init or checkpoint JSON; any other extra text still fails. An init whose output still can't be read now counts as a failed init with a "re-run" hint, so a later clean init in the same session works again. Before, every later phase read was denied with code `evidence` until a new session.
### Fixed
- **`/eval-plan` pilots with a subscription arm can now be approved, and the cost projection is per arm (#3653).** `gstack-eval-plan pilot start --seat-metered <arm>` declares an arm that runs on a subscription with no extra charge (for example `claude -p` on a seat). Its attempts hold none of the spend cap, settle as `unknown` (an arm that settles a real charge is projected like any priced arm), and get their own `PILOT_SEAT_METERED:` line, so the verdict covers the arms that cost money. `pilot report` now projects each arm from its own attempts (one `PILOT_ARM:` line each) and adds them up. Before, it averaged all arms together: a priced arm's cost was multiplied across arms with no price, and two arms at different prices got a band so wide that a pilot that fit the cap was told it exceeded it. The skill also says to stop a seat-metered run at its first quota or rate-limit error, with no retry (idea from #3273 by @merit-blake).
**`/autoplan --ceo-only` runs the CEO review by itself and asks nothing from start to finish (GST-537).** Garry asked for a CEO review that "just automatically does the right thing", and the planning eval needed a gstack CEO arm that a harness can run without a simulated user. Before, /autoplan always ran the Eng review too and stopped at a final approval question, and standalone /plan-ceo-review asks for the mode, offers to chain to the next skill and adds a Codex voice.
### Added
- **`/autoplan --ceo-only`** (`autoplan/sections/ceo-only.md`): runs Phase 0, the outside-reviewer preflight and the CEO phase only. Every question takes its recommendation and gets an audit-trail row; consent and destructive options take the conservative choice. The review mode comes from plan-ceo-review's 0E rule. User challenges are never applied; each one is listed at the top of `<run dir>/ceo-review.md`, with scope added, conservative choices, findings and the audit trail. The run ends with `GSTACK_RESULT: skill=autoplan status=reviewed run=<dir>` and approves nothing.
- **`--no-outside`** with `--ceo-only` skips the outside voice, so one model does the whole review.
- **`gstack-ceo-mode-handoff --ceo-only`** prints `Auto-selected review mode → <mode> (CEO-only run: Step 0E's rule)` instead of crediting a saved preference.
- **`status=reviewed`** joins the `GSTACK_RESULT` statuses (`docs/unattended.md`, "CEO-only runs").
### For contributors
- The paid case `autoplan-ceo-only` fails if the mode asks anything, skips a later phase's file, or leaves new files in HOME outside its folders. It takes 6 to 16 minutes, so it runs in the weekly marathon lane, not on autoplan PRs. The mode needs Claude Code for now.
**/autoplan fixes from a field report (GBRA-223): findings without a line number pair across voices, open Critical and High findings reach the gate, a spend cap binds on any host, `run` works with the API runner, and an earlier item-by-item answer carries forward.** OpenClaw's /autoplan run on a real plan hit six problems at once. We reproduced each on main with a failing test first (GST-672). A follow-up from GEVA-2 adds a record of whose words answered a gate.
### Fixed
- **Plan reviews that name a file but no line now pair across voices.** A same-file pair, with a line missing on either side, is associated as `unverified` instead of both findings counting as new. Title matching ignores a leading DX cap code (`CAP4:<path>:`) and never pairs findings on two different files. The same-file fallback runs only after every title and line match is placed, and a same-file pair never merges two findings into one gate item.
- **DX cap findings carry real titles.** `CAP4` and `CAP6` findings are titled `CAP<n>:<path>:<item> <what fails>`, so the gate shows the problem and not only the item name.
- **Every open Critical or High finding becomes a gate item.** `a` applies the reviewer's fix to the plan and is recommended. `b` carries it into the build as a blocking task. `rec` and `p1b` apply the fix. An unanswered item keeps the plan as written unless the reply is `p1b`, and a partial reply never locks that choice early. A finding carried with `b` stays open, so a rerun that raises it again asks again.
- **A spend cap binds on any host.** With `--spend-cap` set and no `--estimate-usd`, each host-subagent attempt reserves the default $5 per voice, on Capy or off it, in any session kind, and so does an API call whose model has no price. `next --estimate-usd <n>` sets the reservation for the rest of the run. `run` reserves each API call's priced worst case (prompt size plus the output limit) and settles it at its real token cost, so `--spend-cap 0` refuses before any call.
- **`gstack-autoplan run` with `--native-runner api --runner api` reaches the gate.** Results are checked as autoplan results, so they bind. A response cut off at the output limit is named `truncated` and interrupts the run instead of binding a partial review. The default output limit is 16384 tokens for both providers (OpenAI counts reasoning tokens toward it), `--max-tokens <n>` sets it, and an outside result refused at submit leaves the run native-only instead of aborting it.
- **An earlier item-by-item answer carries forward when nothing changed.** When every item on the new gate is carried by origin, the owner's `p1` choice is replayed and their tokens are recorded verbatim as the basis. Any new or changed item still goes to the owner.
- **A spend cap stays closed when a call fails.** For the native voice, a call cut off at its output limit is settled at what it cost, and a retry with a larger `--max-tokens` is admitted against the cap afresh. A call that timed out, dropped, was cancelled or had its result refused after it was sent is settled at its usage, or its full reservation when the usage is unknown, because the provider may have billed it. A call still in flight when the process died is charged its reservation by the next `run`, which then sends a fresh attempt. A call refused before anything was sent (no API key, for example) charges nothing, and the rerun reuses its reservation. A failed outside call is still counted at its full reservation while the phase continues native-only.
- **A finding you chose to fix stays open until the revised plan is reviewed.** If the reviewer raises it again after a revise that missed it, it comes back to the gate instead of being treated as settled. A withdrawn fix leaves its finding open.
- **Two findings that only sit near each other are no longer merged.** A user challenge one line away from an unrelated Critical no longer hides the Critical. Findings are merged only when their titles match or the parent says they are the same problem.
- **A plan gate answered in an agent's own words is not owner approval.** `gstack-ship-receipt` rejects a plan-sha approval citation whose gate was answered with `--words-by agent:<id>`.
- **A later gate answer replaces the earlier answer's authorship.** When someone else answers again, the earlier answerer's words, `words_by` and carry fields are cleared, so the approval record names who actually answered.
### Added
- **`gstack-autoplan answer --words-by agent:<id>`** records that the words came from the answering agent itself, as `answer_words` and `words_by`, never as `owner_reply`. The default stays `owner`. A later gate never carries those words forward, and `gstack-approvals import` lists that row under the agent, while `--by` names only the rows that hold the owner's words.
### Tests
- New failing-first tests in test/autoplan-pairing.test.ts, test/autoplan-gate-judgment.test.ts, test/autoplan-budget.test.ts, test/autoplan-run-api.test.ts (a local fake provider, no key or network), test/autoplan-carry.test.ts and test/approval-record.test.ts.
## [1.92.49.0] - 2026-11-01
**The timeline Stop hook returns within its 2-second budget even when the host keeps stdin open.** The hook read its payload with a blocking read that waits for end of input, so a host that writes the Stop payload and leaves the pipe open held the hook for as long as the pipe stayed open. A local run with the pipe held for 8 seconds took 8 seconds, past the hook's 2-second budget and on the way to Claude Code's 15-second hook window (#3628, reported on Windows). Rewritten from PR #3628 by shahab (mashtii85).
+1 -1
View File
@@ -1 +1 @@
1.92.49.0
1.92.50.0
+1 -1
View File
@@ -1,4 +1,4 @@
# gstack digest v1.92.49.0 — regenerate/re-copy after upgrading gstack
# gstack digest v1.92.50.0 — regenerate/re-copy after upgrading gstack
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
for agent hosts without a full skill install. The full skills add workflows,
+11 -11
View File
@@ -57,28 +57,28 @@ line yourself; gstack never writes the plan.
### Commands
- `gstack-autoplan next --out <dir> --plan <plan.md> [scope flags] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--scaffold]`
first call: init the run, print PHASE:, SCOPE:, DRIVER: and one ATTEMPT: per voice; --max-rounds caps review rounds per phase (default 2 unattended), and an unattended Capy run reserves 5.00 per host attempt unless --estimate-usd <n> says otherwise, so --spend-cap binds.
- `gstack-autoplan next --out <dir> --plan <plan.md> [scope flags] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--estimate-usd <n>] [--scaffold]`
first call: init the run, print PHASE:, SCOPE:, DRIVER: and one ATTEMPT: per voice; --max-rounds caps review rounds per phase (default 2 unattended); a run with --spend-cap, in any session kind (or any unattended Capy run), reserves 5.00 per voice for each attempt, so the cap binds; --estimate-usd <n> sets that reservation and the run keeps it for later phases.
- `gstack-autoplan run --out <dir> --plan <plan.md> --native-model <id> --outside-model <id> --native-runner api --runner api [--max-tokens <n>] [--spend-cap <usd>]`
hosts without subagents: one command calls both models directly (ANTHROPIC_API_KEY, OPENAI_API_KEY) and loops next/submit to the gate; a response cut off at the output limit is reported truncated, so raise --max-tokens (default 16384 for both providers; OpenAI counts reasoning tokens toward it); each api call reserves its priced worst case (a model with no price reserves the per-voice default under a cap), a native call that went out but bound no result settles at its usage, or its full reservation when the usage is unknown, and a failed outside call stays counted at its reservation.
- `gstack-autoplan next --out <dir> [--redispatch <attempt>]`
later calls: open the next phase or reprint open attempts; never redispatches on its own.
- `gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --record <file.md> --model <id> [--attempt <id>]`
bind a free-form review record (canonical fence, one findings table or list, or `No findings.`).
- `gstack-autoplan submit ... --record <file.md> --attest "<reason>" --attempt <id>`
bind a record with no INPUT: line as receipt: attested (a wrong INPUT: always refuses).
- `gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> --model <id> --attempt <id>`
bind a canonical result (the RESULT FORMAT below).
- `gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> --model <id> --attempt <id> [--usage-usd <n>] [--reconcile <jsonl>]`
bind a canonical result (the RESULT FORMAT below); `--usage-usd` settles the attempt at what the reviewer actually cost (else it stays priced unknown at its reservation); on the submit that closes the phase, `--reconcile` takes your own pairing of findings both voices raised, one `{"outside_id","native_id","disposition","resolution"}` row each, over the structural one.
- `gstack-autoplan submit --out <dir> --phase <p> --voice outside --unavailable|--failed "<reason>"`
close the phase native-only when no outside voice can run.
- `gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by human|agent:<id> [--owner-reply <text>|-] [--rec-hash <h>]`
answer the gate with the reply grammar below; an agent answer stores the owner's words verbatim.
- `gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by human|agent:<id> [--owner-reply <text>|-] [--words-by agent:<id>] [--rec-hash <h>]`
answer the gate with the reply grammar below; an agent answer stores the owner's words verbatim, and passes --words-by agent:<id> whenever the words are its own (the default is owner).
- `gstack-autoplan recommend --out <dir> --gate-rev <n> --item <id> --option <k> --basis "<why>"`
change one recommendation before relay; then render and relay again (a `rec` needs the new `--rec-hash`).
- `gstack-autoplan revise --out <dir> --plan <plan v2> --incorporates <decision ids>`
bind the amended plan; reopens every phase whose reviewed plan changed, Eng last.
- `gstack-autoplan revise --out <dir> --settle <decision id|d4>`
discharge a packaging or ordering change with no plan edit; reopens nothing.
- `gstack-autoplan revise --out <dir> --withdraw <decision id>`
drop a pending amendment instead of applying it.
- `gstack-autoplan revise --out <dir> --settle <decision id|d4> | --withdraw <decision id>`
--settle discharges a packaging or ordering change with no plan edit and reopens nothing; --withdraw drops a pending amendment instead of applying it.
- `gstack-autoplan close --out <dir> --phase <p> --deferred "<reason>" --by human|agent:<id>`
close a phase accepted-with-findings: its open findings go to tasks.jsonl, and Critical, urgent or user-challenge findings become gate items; also the recovery from `STOP: reason=budget_exhausted`.
- `gstack-autoplan budget --out <dir> [--spend-cap <usd>] [--max-rounds <n>] --by human|agent:<id>`
@@ -111,7 +111,7 @@ A reply binds to the `gate_rev` you read (`--gate-rev`). Bare `yes`/`no` is unpa
- `rec`: every item takes its recommendation, plan changes included (each prints APPLY:); `all` and `do it` are aliases.
- `as-is`: keep every direction; no plan change.
- `d3b uc1a`: one <id><option> token per item you set; unanswered auto items take their default.
- `d3b uc1a`: one <id><option> token per item you set; unanswered auto items take their default (a finding item keeps the plan as written; `p1b` applies it).
- `rec except d3b`: recommendations, with the listed overrides.
## Run directory
+1
View File
@@ -269,6 +269,7 @@ sections. Read a section in full before doing its step; do not work from memory.
| closing a review phase, after its reviews finish and before announcing completion or loading the next phase (read afresh at each exit) | `sections/phase-close.md` |
| presenting the Final Approval Gate (Phase 4) — the aggregator computes $AGGREGATED_TASKS that the gate message substitutes | `sections/tasks-aggregator.md` |
| the preamble echoed `SESSION_KIND: unattended` (read before Phase 1 and at the Phase 4 gate) | `sections/unattended.md` |
| the invocation names `--ceo-only` (read before any preamble instruction block) | `sections/ceo-only.md` |
---
+8 -2
View File
@@ -17,6 +17,10 @@ type Event = ClaudeParentPublicEvent;
type Use = NativePublicToolEvent & { order: number };
/** Claude Code's reply to a Read of an unchanged file it already returned in full. */
export const DEDUP_REPLY = 'Wasted call — file unchanged since your last Read. Refer to that earlier tool_result instead.';
/** Claude Code appends exactly this one line to a Bash result when the command's `cd` left the session cwd (#3651). */
const SHELL_CWD_RESET = /\nShell cwd was reset to [^\n]+$/;
/** A Bash result's JSON, without that one host footer; anything else must still be strict JSON. */
export function bashJson(text: string): any { return JSON.parse(text.replace(SHELL_CWD_RESET, '')); }
/** The longest Read input or Bash command the index keeps; a literal init is far shorter. */
const INDEX_INPUT_MAX = 4096;
@@ -69,11 +73,13 @@ function select(tests: GuardJournalTests, currentToolUseId: string) {
const replies = (resultsOf.get(init.use.toolUseId) ?? []).flatMap(r => r.events)
.filter(e => e.kind === 'result' && e.toolUseId === init.use.toolUseId && e.order > init.use.order);
let reply: unknown;
try { reply = replies.length === 1 ? JSON.parse(textResult(replies[0]!) ?? 'null') : undefined; } catch { reply = undefined; }
try { reply = replies.length === 1 ? bashJson(textResult(replies[0]!) ?? 'null') : undefined; } catch { reply = undefined; }
// A failed or unreadable init binds nothing and is skipped, as invocation() does.
if (!object(reply)) continue;
// A reused:true init keeps the earlier binding, as invocation() does.
if (object(reply) && reply.reused === true && bound && bound.activePlan === reply.activePlan &&
bound.restorePath === reply.restorePath) continue;
chainStart = init.record; bound = object(reply) ? reply : undefined;
chainStart = init.record; bound = reply;
}
let start = 0;
for (let i = chainStart ?? records.length - 1; i >= 0; i--)
+12 -9
View File
@@ -10,7 +10,7 @@ import { extractImplementationPlan, checkPhaseImplementation, acceptedBlocks, ca
import { autoplanPhaseCompletions } from '../../lib/autoplan-phase-publication';
import { nativePathSpelling, ownedNativePath, sameNativePath, type ClaudeParentPublicEvent } from '../../lib/claude-journal-records';
import { ownedRecordLimit, ownedRetainedLimit, type JournalPrefix, type OwnedRead, type OwnedReadMeasure } from '../../lib/claude-owned-journal';
import { readGuardJournal, textResult, DEDUP_REPLY } from './guard-journal';
import { bashJson, readGuardJournal, textResult, DEDUP_REPLY } from './guard-journal';
import { REASONS, reasonCode, reasonText, type Detail, type ReasonCode } from './guard-reasons';
import { logGuardDecision } from './guard-log';
import { approveOwnedRead, renderSectionBase, userRenderRoot } from './owned-read';
@@ -227,7 +227,7 @@ function checkpointResult(result: Event, tools: Tools, init: Invocation): { phas
if (use?.name !== 'Bash' || use.order >= result.order) return;
const text = textResult(result);
if (text === undefined) return;
const output = JSON.parse(text);
const output = bashJson(text);
if (!object(output) || !['ceo', 'design', 'dx', 'eng'].includes(output.phase) ||
!ownPath(output.snapshotPath) || typeof output.nativePrompt !== 'string' || !object(output.baselineEdits)) return;
const { nativePrompt, baselineEdits, ...identity } = output;
@@ -260,6 +260,7 @@ function initArguments(command: unknown, root: string): string[] | undefined {
const INIT_MISMATCH_CAUSE = 'Snapshot init returned paths that do not match the canonical form of its own command arguments';
const INIT_FAILED_CAUSE = 'The most recent snapshot init call failed; re-run the same init command with the same three paths';
const INIT_UNPARSEABLE_CAUSE = 'The most recent snapshot init printed something other than its JSON result; re-run the same init command with the same three paths, without a leading cd';
/** Diagnostic only, never binds: a Bash call that ran snapshot init in a shape the guard cannot bind (#3045). */
const UNBINDABLE_INIT = /(?:gstack-autoplan-snapshot|SNAPSHOT_TOOL|\$\{?[A-Z_]+\}?"?)\S*\s+init\s/;
@@ -268,8 +269,8 @@ function invocation(events: Event[], root: string): Invocation {
let bound: Invocation | undefined;
let chosen: Record<string, any> | undefined;
let unbindable = false;
// True while the most recent bindable init use errored (a later success clears it).
let lastErrored = false;
// The cause while the most recent bindable init use failed (a later success clears it).
let lastFailure: string | undefined;
const tools = byTool(events);
for (const use of events) {
if (use.kind !== 'use' || use.name !== 'Bash') continue;
@@ -278,10 +279,12 @@ function invocation(events: Event[], root: string): Invocation {
const results = tools.results(use.toolUseId).filter(x => x.order > use.order);
if (results.length !== 1) fail('init_mismatch');
// An errored init published nothing; a later correct init may still bind.
if (results[0]!.isError === true) { lastErrored = true; continue; }
if (results[0]!.isError === true) { lastFailure = INIT_FAILED_CAUSE; continue; }
const text = textResult(results[0]!);
if (text === undefined) fail('init_failed');
const result = JSON.parse(text);
if (text === undefined) { lastFailure = INIT_FAILED_CAUSE; continue; }
let result: any;
// Output that is not the init's JSON bound nothing; a later correct init may still bind.
try { result = bashJson(text); } catch { lastFailure = INIT_UNPARSEABLE_CAUSE; continue; }
// init reports canonical destinations (macOS /tmp is /private/tmp); canonicalize the argv the same way.
let active: string, restore: string;
try { active = canonicalDestination(args[1]!); restore = canonicalDestination(args[2]!); }
@@ -289,13 +292,13 @@ function invocation(events: Event[], root: string): Invocation {
if (!object(result) || !samePath(result.sourcePlan, fs.realpathSync(args[0]!)) || !samePath(result.activePlan, active) ||
!samePath(result.restorePath, restore) || typeof result.reused !== 'boolean' || !positive(result.originalBytes) ||
!/^[a-f0-9]{64}$/.test(result.originalSha256)) fail('init_mismatch', { cause: INIT_MISMATCH_CAUSE });
lastErrored = false;
lastFailure = undefined;
if (result.reused && bound && bound.activePlan === result.activePlan && bound.restorePath === result.restorePath) continue;
chosen = result;
bound = { activePlan: result.activePlan, restorePath: result.restorePath,
originalSha256: result.originalSha256, start: results[0]!.order };
}
if (lastErrored) fail('init_failed', { cause: INIT_FAILED_CAUSE });
if (lastFailure) fail('init_failed', { cause: lastFailure });
if ((!chosen || !bound) && unbindable) fail('init_unbindable');
if (!chosen || !bound) fail('init_required');
const restore = read(bound.restorePath, true), active = read(bound.activePlan);
+41
View File
@@ -0,0 +1,41 @@
<!-- AUTO-GENERATED from ceo-only.md.tmpl — do not edit directly -->
<!-- Regenerate: bun run gen:skill-docs -->
## CEO-only run (`--ceo-only`)
This section governs the whole run and wins over the skeleton. The run reviews
the plan; it never approves it.
**Zero questions.** Never call AskUserQuestion (native or `mcp__*`) or
ExitPlanMode, never ask the user in prose, never wait for a reply, whatever
`SESSION_KIND` says. A preamble instruction block that would ask takes its
non-consenting default (off, not now, skip). Each decision takes its
`(recommended)` option, else the 6 principles; consent, destructive or
irreversible options take the conservative choice. Log each as a Decision Audit
Trail row.
**Scope.** Run Phases 0, 0.5 and 1 only; send "Phases 2, 2.5, 3 and 4 skipped
(CEO-only run)" and record it in ACTIVE_PLAN. With `--no-outside`, skip every
outside dispatch: `outside_status: disabled (--no-outside)`.
**Mode.** Instead of SELECTIVE EXPANSION, use plan-ceo-review 0E step 2's rule
(an explicit mode in the request wins), skip step 3, run
`~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff "<mode>" --ceo-only --decisions "<rows or none>"`
and send its line verbatim.
**Never applied:** a user challenge (keep the direction; list it below).
**Skipped and logged `skipped (ceo-only)`:** office-hours and BENEFITS_FROM
offers, 0E's question, plan archive cleanup, Next Steps chaining and CEO plan
promotion; the row satisfies its checklist entry.
**State.** Write only in `$_AP`, ACTIVE_PLAN and `GSTACK_STATE_ROOT`
(the override; docs/state-root.md).
**Finish.** After Phase 1 closes, write `$_AP/ceo-review.md`: `# CEO-only
review: <plan title>`; a line with the mode, outside-voice status and "Not
approved"; then `## User challenges (direction kept)` (plan says, proposal,
why, cost if wrong), `## Scope added`, `## Conservative choices`,
`## Findings` (severity, section, fix, disposition) and `## Decision Audit
Trail` (copied), each "None." when empty. Send the CEO Completion Summary and
both paths, skip Completion's logs, approval and `/ship`, and end with:
`GSTACK_RESULT: skill=autoplan status=reviewed run=<absolute $_AP>`
+39
View File
@@ -0,0 +1,39 @@
## CEO-only run (`--ceo-only`)
This section governs the whole run and wins over the skeleton. The run reviews
the plan; it never approves it.
**Zero questions.** Never call AskUserQuestion (native or `mcp__*`) or
ExitPlanMode, never ask the user in prose, never wait for a reply, whatever
`SESSION_KIND` says. A preamble instruction block that would ask takes its
non-consenting default (off, not now, skip). Each decision takes its
`(recommended)` option, else the 6 principles; consent, destructive or
irreversible options take the conservative choice. Log each as a Decision Audit
Trail row.
**Scope.** Run Phases 0, 0.5 and 1 only; send "Phases 2, 2.5, 3 and 4 skipped
(CEO-only run)" and record it in ACTIVE_PLAN. With `--no-outside`, skip every
outside dispatch: `outside_status: disabled (--no-outside)`.
**Mode.** Instead of SELECTIVE EXPANSION, use plan-ceo-review 0E step 2's rule
(an explicit mode in the request wins), skip step 3, run
`~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff "<mode>" --ceo-only --decisions "<rows or none>"`
and send its line verbatim.
**Never applied:** a user challenge (keep the direction; list it below).
**Skipped and logged `skipped (ceo-only)`:** office-hours and BENEFITS_FROM
offers, 0E's question, plan archive cleanup, Next Steps chaining and CEO plan
promotion; the row satisfies its checklist entry.
**State.** Write only in `$_AP`, ACTIVE_PLAN and `GSTACK_STATE_ROOT`
(the override; docs/state-root.md).
**Finish.** After Phase 1 closes, write `$_AP/ceo-review.md`: `# CEO-only
review: <plan title>`; a line with the mode, outside-voice status and "Not
approved"; then `## User challenges (direction kept)` (plan says, proposal,
why, cost if wrong), `## Scope added`, `## Conservative choices`,
`## Findings` (severity, section, fix, disposition) and `## Decision Audit
Trail` (copied), each "None." when empty. Send the CEO Completion Summary and
both paths, skip Completion's logs, approval and `/ship`, and end with:
`GSTACK_RESULT: skill=autoplan status=reviewed run=<absolute $_AP>`
+6
View File
@@ -45,6 +45,12 @@
"file": "unattended.md",
"title": "Unattended profile: parent-driven next/submit loop, gate answers, export, terminal line",
"trigger": "the preamble echoed `SESSION_KIND: unattended` (read before Phase 1 and at the Phase 4 gate)"
},
{
"id": "ceo-only",
"file": "ceo-only.md",
"title": "CEO-only run: zero questions, CEO phase only, review report, no gate",
"trigger": "the invocation names `--ceo-only` (read before any preamble instruction block)"
}
]
}
+50 -16
View File
@@ -9,8 +9,8 @@
* gstack-autoplan submit --out <dir> --phase <p> --voice outside --unavailable <reason> | --failed <reason> [--json]
* gstack-autoplan revise --out <dir> --plan <file> [--incorporates <id,...>] [--withdraw <id>] [--settle <id>] [--json] review a revised plan in this run
* gstack-autoplan run --out <dir> --plan <file> --native-model <id> --outside-model <id> [--runner codex-cli|api] [--native-runner api]
* [--deadline <min>] [--spend-cap <usd>] [--estimate-usd <n>] [--json] next in a loop; needs runners for both voices
* gstack-autoplan answer --out <dir> --gate-rev <rev> --reply "<grammar>" --by <human|agent:<id>> [--owner-reply <text>|-] [--rec-hash <h>] [--json]
* [--deadline <min>] [--spend-cap <usd>] [--estimate-usd <n>] [--max-tokens <n>] [--json] next in a loop; needs runners for both voices
* gstack-autoplan answer --out <dir> --gate-rev <rev> --reply "<grammar>" --by <human|agent:<id>> [--owner-reply <text>|-] [--words-by owner|agent:<id>] [--rec-hash <h>] [--json]
* gstack-autoplan recommend --out <dir> --gate-rev <n> --item <id> --option <k> --basis "<why>" [--by <who>]
* gstack-autoplan resume --out <dir> [--from <checkpoint>] [--json] take ownership; intent without a terminal record becomes EXECUTION_UNKNOWN
* gstack-autoplan status --out <dir> [--json] query: the manifest, no GSTACK_RESULT line
@@ -43,11 +43,13 @@ import { capyRelayLine } from '../lib/captain-brief';
import { exportPublic } from '../lib/autoplan-public';
import { migrateRun } from '../lib/autoplan-migrate';
import { reviseRun } from '../lib/autoplan-revise';
import { pendingAmendments, type RunState } from '../lib/autoplan-state';
import { FILES, RunError, currentPhase, initRun, markOutside, nextStep, readState, resumeRun, statusLines, submitResult, withRunLock, writeState } from '../lib/autoplan-run';
import { pendingAmendments, type RunState, type Voice } from '../lib/autoplan-state';
import { FILES, RunError, currentPhase, fitReservation, initRun, markOutside, markSent, nextStep, readState, resumeRun, retireAttempt, statusLines, submitResult, withRunLock, writeState, type NextOptions } from '../lib/autoplan-run';
import { driverFile, type Phase } from '../lib/autoplan-prompts';
import { EXIT, exitTableHelp } from '../lib/headless-artifacts';
import { createRunner, familyConflict, type RunnerId } from '../lib/outside-voice-runner';
import { DEFAULT_MAX_TOKENS, apiTimeoutMs } from '../lib/outside-voice-api';
import { estimateCostUsd, pricingRow } from '../lib/pricing';
import { resultLine } from '../lib/result-codes';
import { release } from '../lib/spend-ledger';
import { COORDINATOR_CONTRACT } from '../lib/coordinator-contract';
@@ -59,7 +61,7 @@ DRIVER: ${driverFile()}
[--no-outside] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--estimate-usd <n>] [--deadline <min>] [--redispatch <attempt>] [--json]
--max-rounds (first next): review rounds per phase (default 2 unattended); at the cap a phase closes accepted-with-findings
--docs-only (first next; also detected when every Implementation-plan path is docs-class): CEO (native voice only) + Eng
--estimate-usd: reserved per new attempt; unattended Capy runs reserve a default for host attempts so --spend-cap binds
--estimate-usd: reserved per new attempt, kept for the rest of the run; with --spend-cap any run (or an unattended run on Capy) reserves a default so the cap binds
--scaffold (first next): a plan missing one of its two sections is reviewed from <out>/plan-source.md with the heading added
--light loads each phase's checklist plus the sections its scope needs (the unattended default); interactive kinds load everything
--no-carry-forward (first next): every later gate revision goes to the owner, even one that asks nothing new
@@ -80,9 +82,12 @@ DRIVER: ${driverFile()}
comes into scope, the edit applies without re-review (APPLIED:); --withdraw (alone or with --plan) drops one;
--settle records a packaging/ordering change that needs no plan edit and reopens nothing (ids: decision id or d4)
run --out <dir> --plan <file> --native-model <id> --outside-model <id> [--runner codex-cli|api] [--native-runner api] ...
next in a loop, only when a runner executes both voices here; --deadline cancels the children it started
next in a loop, only when a runner executes both voices here (hosts without subagents: --native-runner api --runner api);
--deadline cancels the children it started; --max-tokens caps each api call's output (default 16384; OpenAI counts reasoning tokens in it), and a
response cut off at that limit is reported truncated; submit's result checks judge every completed response
answer --out <dir> --gate-rev <rev> --reply "<rec | rec except d3a | d3b uc1a | as-is>" --by <human|agent:<id>> [--json]
--by agent:<id> needs --owner-reply "<owner's words>" (- reads stdin; --owner-reply-file <path>), stored verbatim;
--words-by agent:<id> (the same agent) when those words are its own: recorded as its words, never the owner's, never carried;
--rec-hash <h> binds a rec to the render the owner saw (required once recommend ran); accepted changes leave amend_pending
recommend --out <dir> --gate-rev <n> --item <id> --option <k> --basis "<why>" [--by <who>]
set one item's recommendation before relay (provenance recorded, p1 follows); re-render and relay afterwards
@@ -178,7 +183,7 @@ function emit(json: boolean, lines: string[], payload: Record<string, unknown>):
else process.stdout.write(lines.join('\n') + '\n');
}
function doNext(out: string, rest: string[], json: boolean): number {
function doNext(out: string, rest: string[], json: boolean, price?: NextOptions['price']): number {
const cwd = process.cwd();
const initLines: string[] = [];
const fresh = !fs.existsSync(path.join(out, FILES.run));
@@ -195,7 +200,7 @@ function doNext(out: string, rest: string[], json: boolean): number {
deadlineMin: num(rest, '--deadline'), light: rest.includes('--light'), scaffold: rest.includes('--scaffold'), scope: { ui, developerTool: rest.includes('--developer-tool'), agentPrimary: rest.includes('--agent-primary'), dx: rest.includes('--no-dx') ? false : undefined, ...(rest.includes('--docs-only') ? { docsOnly: true } : {}) } }, initLines);
}
const redispatch = rest.flatMap((a, i) => (a === '--redispatch' && rest[i + 1] ? [rest[i + 1]!] : []));
const r = nextStep({ out, cwd, estimateUsd: num(rest, '--estimate-usd'), redispatch });
const r = nextStep({ out, cwd, estimateUsd: num(rest, '--estimate-usd'), redispatch, price });
moment = fresh ? 'init' : r.attempts.some(a => a.new) || r.lines.some(l => l.startsWith('PHASE_DEFERRED: ')) ? 'transition' : undefined;
const status = r.state.status;
const lines = [...initLines, ...r.lines];
@@ -305,10 +310,25 @@ async function doRun(out: string, rest: string[], json: boolean): Promise<number
if (!['codex-cli', 'api'].includes(outsideRunner)) return usage('--runner must be codex-cli or api for run (host-subagent cannot execute here; use next/submit)');
const conflict = familyConflict(outsideModel, nativeModel);
if (conflict) throw new RunError('MODEL_FAMILY_CONFLICT', conflict, 3);
const maxTokens = num(rest, '--max-tokens');
if (maxTokens !== undefined && (!Number.isInteger(maxTokens) || maxTokens <= 0)) return usage('--max-tokens needs a positive integer');
const deadline = num(rest, '--deadline');
const deadlineAt = deadline ? Date.now() + deadline * 60_000 : undefined;
const modelOf = (voice: Voice) => (voice === 'native' ? nativeModel : outsideModel);
const isApi = (voice: Voice) => (voice === 'native' ? nativeRunner : outsideRunner) === 'api';
// An api call reserves its worst case (prompt bytes / 3 input tokens plus the output limit it is sent) and settles at its usage.
const price = (voice: Voice, promptFile: string) => {
const row = isApi(voice) ? pricingRow(modelOf(voice)) : undefined;
return row ? +((fs.statSync(promptFile).size / 3 * row.input_per_mtok + (maxTokens ?? DEFAULT_MAX_TOKENS) * row.output_per_mtok) / 1e6).toFixed(6) : undefined;
};
const usdOf = (voice: Voice, used: { input_tokens?: number; output_tokens?: number } | undefined) =>
isApi(voice) && pricingRow(modelOf(voice)) && typeof used?.input_tokens === 'number' && typeof used?.output_tokens === 'number' ? estimateCostUsd({ input: used.input_tokens, output: used.output_tokens }, modelOf(voice)) : undefined;
// Every exit after a paid call went out settles its attempt: a failure, a cancel, a refused result, or a process that died
// mid-call (sent_at, found on the next run) settles at its usage, or its reservation when the usage is unknown. Only a call
// refused before any request left this machine bills nothing, so its attempt stays open for the rerun.
const NOT_SENT = new Set(['runner_unavailable', 'cli_missing', 'model_unselected']);
for (let guard = 0; guard < 40; guard++) {
const code = withRunLock(out, () => doNext(out, rest, true));
const code = withRunLock(out, () => doNext(out, rest, true, price));
if (code !== EXIT.ok) return code;
const state = readState(out);
if (state.status !== 'awaiting_result') {
@@ -316,27 +336,39 @@ async function doRun(out: string, rest: string[], json: boolean): Promise<number
process.stdout.write(resultLineFor(state.status, out) + '\n'); return EXIT.ok;
}
const phase = state.phase_order.find(p => state.phases[p]?.snapshot && !state.phases[p]?.closed_at)!;
const lost = (['native', 'outside'] as const).flatMap(voice => (state.phases[phase]![voice]!.attempts ?? []).filter(a => a.state === 'dispatched' && a.sent_at).map(a => [voice, a.attempt] as const));
if (lost.length) { withRunLock(out, () => { for (const [voice, attempt] of lost) retireAttempt(out, phase as Phase, voice, attempt, 'unknown'); }); continue; }
for (const voice of ['native', 'outside'] as const) {
const v = state.phases[phase]![voice]!;
const open = v.attempts?.find(a => a.state === 'dispatched');
if (!open) continue;
const runnerId = voice === 'native' ? nativeRunner : outsideRunner;
const cost = price(voice, open.prompt);
if (cost !== undefined) withRunLock(out, () => fitReservation(out, phase as Phase, voice, open.attempt, cost));
const runner = createRunner(runnerId);
const remaining = deadlineAt ? deadlineAt - Date.now() : 300_000;
const remaining = deadlineAt ? deadlineAt - Date.now() : runnerId === 'api' ? apiTimeoutMs(maxTokens) : 300_000;
if (remaining <= 0) { withRunLock(out, () => { const s = readState(out); s.status = 'interrupted'; writeState(s); }); process.stdout.write(resultLineFor('interrupted', out) + '\n'); return EXIT.fail; }
const timer = deadlineAt ? setTimeout(() => runner.cancel(), remaining) : undefined;
const res = await runner.run({ promptFile: open.prompt, outFile: `${open.result}.raw`, model: voice === 'native' ? nativeModel : outsideModel, timeoutMs: Math.min(remaining, 600_000), cwd: process.cwd(), gate: 'review' });
withRunLock(out, () => markSent(out, phase as Phase, voice, open.attempt, true));
const res = await runner.run({ promptFile: open.prompt, outFile: `${open.result}.raw`, model: voice === 'native' ? nativeModel : outsideModel, timeoutMs: Math.min(remaining, runnerId === 'api' ? apiTimeoutMs(maxTokens) : 600_000), cwd: process.cwd(), gate: 'execution', maxTokens: runnerId === 'api' ? maxTokens ?? DEFAULT_MAX_TOKENS : maxTokens });
if (timer) clearTimeout(timer);
if (!json) process.stdout.write(`VOICE: ${phase} ${voice} ${res.status} runner=${res.runner} model=${res.model ?? 'unknown'}\n`);
if (res.status === 'cancelled') { withRunLock(out, () => { const s = readState(out); s.status = 'interrupted'; writeState(s); }); process.stdout.write(resultLineFor('interrupted', out) + '\n'); return EXIT.fail; }
const retire = () => { if (NOT_SENT.has(res.reason ?? '')) markSent(out, phase as Phase, voice, open.attempt, false); else retireAttempt(out, phase as Phase, voice, open.attempt, usdOf(voice, res.usage) ?? 'unknown'); };
if (res.status === 'cancelled') { withRunLock(out, () => { retire(); const s = readState(out); s.status = 'interrupted'; writeState(s); }); process.stdout.write(resultLineFor('interrupted', out) + '\n'); return EXIT.fail; }
if (res.status !== 'completed' && res.status !== 'unverified') {
if (voice === 'outside') { withRunLock(out, () => markOutside(out, phase as Phase, 'unavailable', `${res.runner} ${res.status}${res.detail ? `: ${res.detail}` : ''}`)); continue; }
process.stderr.write(`gstack-autoplan run: native voice ${res.status}${res.detail ? `: ${res.detail}` : ''}\n`);
withRunLock(out, () => { const s = readState(out); s.status = 'interrupted'; writeState(s); });
process.stderr.write(`gstack-autoplan run: native voice ${res.status}${res.reason ? `: ${res.reason}` : ''}${res.detail ? `: ${res.detail}` : ''}\n`);
withRunLock(out, () => { retire(); const s = readState(out); s.status = 'interrupted'; writeState(s); });
process.stdout.write(resultLineFor('interrupted', out) + '\n');
return EXIT.fail;
}
withRunLock(out, () => doSubmit(out, ['--phase', phase, '--voice', voice, '--result', `${open.result}.raw`, '--model', res.model ?? (voice === 'native' ? nativeModel : outsideModel), '--attempt', open.attempt, '--runner', res.runner], true));
const usd = usdOf(voice, res.usage);
const usage = usd === undefined ? [] : ['--usage-usd', String(usd)];
try { withRunLock(out, () => doSubmit(out, ['--phase', phase, '--voice', voice, '--result', `${open.result}.raw`, '--model', res.model ?? modelOf(voice), '--attempt', open.attempt, '--runner', res.runner, ...usage], true)); }
catch (e) {
if (voice !== 'outside' || !(e instanceof RunError)) { withRunLock(out, () => { retire(); const s = readState(out); s.status = 'interrupted'; writeState(s); }); throw e; }
withRunLock(out, () => markOutside(out, phase as Phase, 'unavailable', `${res.runner} result refused: ${e.code}`));
}
}
}
return EXIT.fail;
@@ -386,10 +418,12 @@ async function main(argv: string[]): Promise<number> {
if (rev === undefined) { const s = readState(out); process.stderr.write(resultLine('gstack-autoplan answer', 'GATE_REV_STALE', `no --gate-rev given; current gate_rev=${s.gate_rev ?? 0}`) + '\n'); return EXIT.refused; }
const replyFile = flag(rest, '--owner-reply-file'); const ownerFlag = flag(rest, '--owner-reply');
if (replyFile !== undefined && ownerFlag !== undefined) return usage('--owner-reply and --owner-reply-file are exclusive');
const wordsBy = flag(rest, '--words-by') ?? 'owner';
if (wordsBy !== 'owner' && (wordsBy !== by || !by.startsWith('agent:'))) return usage(`--words-by must be owner or the answering agent (--by ${by ?? '<unset>'}), got ${wordsBy}`);
const ownerReply = replyFile !== undefined ? fs.readFileSync(replyFile, 'utf8') : ownerFlag === '-' ? fs.readFileSync(0, 'utf8') : ownerFlag;
return mutating(out, json, () => {
const state = readState(out);
const r = answerGate(state, Number(rev), reply, by, { recHash: flag(rest, '--rec-hash'), ownerReply });
const r = answerGate(state, Number(rev), reply, by, { recHash: flag(rest, '--rec-hash'), ownerReply, ...(wordsBy === 'owner' ? {} : { wordsBy }) });
writeState(state);
const lines = [...r.lines];
if (r.outcome === 'reopened' || r.outcome === 'amend_pending') lines.push(nextLine(state, out));
+1 -1
View File
@@ -55,7 +55,7 @@ For each finding: what's wrong, severity (critical/high/medium), and the fix.`,
reviewing this plan. You have NOT seen any prior review. Evaluate:
1. Getting started: how many steps from zero to hello world? What's the TTHW?
2. API/CLI ergonomics: naming consistency, sensible defaults, progressive disclosure?
3. Error handling: Score every error path against the agent-ready error contract (https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or the precise thing a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure that names no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item>" (caps the Error Messages score at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item>" (caps it at 6/10).
3. Error handling: Score every error path against the agent-ready error contract (https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or what a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure naming no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item> <what fails>" (caps Error Messages at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item> <what fails>" (caps it at 6/10).
4. Documentation: copy-paste examples? Information architecture? Interactive elements?
5. Escape hatches: can developers override every opinionated default?
For each finding: what's wrong, severity (critical/high/medium), and the fix.`,
+8 -2
View File
@@ -1,10 +1,11 @@
#!/usr/bin/env bash
# gstack-ceo-mode-handoff — print /plan-ceo-review's Step 0E mode handoff line.
#
# Usage: gstack-ceo-mode-handoff <MODE> [--auto] [--decisions "<rows>"]
# Usage: gstack-ceo-mode-handoff <MODE> [--auto | --ceo-only] [--decisions "<rows>"]
# MODE: HOLD SCOPE | SCOPE EXPANSION | SELECTIVE EXPANSION | SCOPE REDUCTION
# (underscores accepted for spaces)
# --auto the mode came from a saved AUTO_DECIDE preference
# --ceo-only the mode came from 0E's rule in an /autoplan --ceo-only run
# --decisions governing approved rows; defaults to "none"
#
# The handoff used to be chat-only, and the model often composed it in its
@@ -15,10 +16,12 @@ set -euo pipefail
mode="${1:-}"
[ "$#" -gt 0 ] && shift
auto=0
ceo_only=0
decisions="none"
while [ "$#" -gt 0 ]; do
case "$1" in
--auto) auto=1; shift ;;
--ceo-only) ceo_only=1; shift ;;
--decisions)
[ "$#" -ge 2 ] || { echo "gstack-ceo-mode-handoff: --decisions needs a value" >&2; exit 2; }
decisions="$2"; shift 2 ;;
@@ -31,11 +34,14 @@ case "$mode" in
"HOLD SCOPE"|"SCOPE EXPANSION"|"SELECTIVE EXPANSION"|"SCOPE REDUCTION") ;;
*) echo "gstack-ceo-mode-handoff: unknown mode: ${mode:-<empty>}" >&2; exit 2 ;;
esac
[ "$auto$ceo_only" != 11 ] || { echo "gstack-ceo-mode-handoff: --auto and --ceo-only conflict" >&2; exit 2; }
decisions="${decisions//$'\n'/ }"
decisions="${decisions%.}"
[ -n "${decisions// /}" ] || decisions="none"
if [ "$auto" = 1 ]; then
if [ "$ceo_only" = 1 ]; then
echo "Auto-selected review mode → $mode (CEO-only run: Step 0E's rule). Approved decisions: $decisions."
elif [ "$auto" = 1 ]; then
echo "Auto-decided review mode → $mode (your preference). Change with /plan-tune. Approved decisions: $decisions."
else
echo "Mode: $mode; approved decisions: $decisions."
+6 -4
View File
@@ -6,7 +6,7 @@
* gstack-eval-plan arms [--arms <model,model>] [--policy-doc <file>] [--input-tokens N] [--output-tokens N] [--json]
* gstack-eval-plan dry-run --input <rows.jsonl> --arms <model,model> [--fields a,b] [--output-budget N]
* [--retry-prompt <file>] [--max-rows N] [--out <dir>] [--json]
* gstack-eval-plan pilot start --out <dir> --cap <usd> --arms <model[#loadout],...> --dry-run <dry-run.json> [--n 3]
* gstack-eval-plan pilot start --out <dir> --cap <usd> --arms <model[#loadout],...> --dry-run <dry-run.json> [--n 3] [--seat-metered <arm,...>]
* gstack-eval-plan pilot settle --out <dir> --attempt <id> --usd <n|unknown>
* gstack-eval-plan pilot release --out <dir> --attempt <id>
* gstack-eval-plan pilot report --out <dir> --total <items> [--workers W] [--json]
@@ -53,9 +53,11 @@ const HELP = `gstack-eval-plan — preregistration, priced arms, $0 dry run, pri
dry-run --input rows.jsonl --arms m,m [--fields a,b] [--output-budget N] [--retry-prompt f] [--max-rows N] [--cap usd] [--out dir] [--json]
stub model over the real rows; checklist: price table, output budget, retry prompt text,
and with --cap reservation-fits-cap (one call: largest row + output budget, priced)
pilot start --out dir --cap usd --arms m,m#no-gstack --dry-run dry-run.json [--n 3]
pilot start --out dir --cap usd --arms m,m#no-gstack --dry-run dry-run.json [--n 3] [--seat-metered arm,...]
--seat-metered: arms on a subscription with no incremental charge; they reserve $0,
settle unknown, and sit outside the cap and the projection until one settles a charge
pilot settle --out dir --attempt id --usd <n|unknown> pilot release --out dir --attempt id
pilot report --out dir --total items [--workers W] [--json] spend table + extrapolation with a 95% band, then the approval ask
pilot report --out dir --total items [--workers W] [--json] spend table + per-arm extrapolation with a 95% band, then the approval ask
results --check <prereg> <results.md> [--json]
one table in the results file has a filled row for every Arms identity, each no-gstack arm
beside its variants; reads text only (docs/TESTING_INTERNALS.md#no-gstack-arm)
@@ -185,7 +187,7 @@ function doPilot(rest: string[]): number {
if (cap === undefined || !arms.length || !dry || !fs.existsSync(dry)) return usage('pilot start needs --cap <usd> --arms <m,m> --dry-run <existing dry-run.json>');
const rules = lintModels(args, arms.map(model => ({ model, scope: scopeOf(args), where: 'pilot start --arms' })), 'pilot start');
if (!rules.ok) return EXIT.fail;
const r = startPilot({ out, cap, n: num(args, '--n', 3)!, arms, dryRun: dry });
const r = startPilot({ out, cap, n: num(args, '--n', 3)!, arms, seatMetered: csv(flag(args, '--seat-metered')), dryRun: dry });
if (r.error && !r.record) { process.stderr.write(resultLine('gstack-eval-plan pilot start', r.error.code, r.error.message) + '\n'); return EXIT.refused; }
for (const a of r.record.attempts) process.stdout.write(`PILOT_ATTEMPT: ${a.attempt} model=${a.model} estimate=$${a.estimate_usd.toFixed(4)}\n`);
process.stdout.write(`PILOT: status=${r.record.status} reserved=${r.record.attempts.length} cap=$${cap.toFixed(2)} ledger=${path.join(out, 'spend.json')}\n`);
+12 -6
View File
@@ -4,7 +4,7 @@
*
* gstack-outside-voice run --runner codex-cli|api|host-subagent --prompt <file> --out <file>
* [--model <id>] [--result <file>] [--native-model <id>] [--fallback <runner>]
* [--gate review|structured|spec|proposal] [--timeout-ms N] [--cwd <dir>] [--json]
* [--gate review|structured|spec|proposal] [--timeout-ms N] [--max-tokens N] [--cwd <dir>] [--json]
* gstack-outside-voice runners [--model <id>] [--json] the runner table: capabilities and availability here
* gstack-outside-voice family <model-id> the model's family (same-family rule)
*
@@ -20,18 +20,21 @@ import { EXIT, exitTableHelp } from '../lib/headless-artifacts';
import type { OutsideGate } from '../lib/outside-review-result';
import { RUNNER_IDS, createRunner, familyConflict, modelFamily, outsideStatusLine, runnerTable, selectRunner, type RunnerId } from '../lib/outside-voice-runner';
import { outsideConsent } from '../lib/outside-consent';
import { apiTimeoutMs } from '../lib/outside-voice-api';
import { resultLine } from '../lib/result-codes';
const HELP = `gstack-outside-voice — one outside voice, three runners (docs/unattended.md)
run --runner <r> --prompt <file> --out <file> [--model <id>] [--result <file>] [--native-model <id>]
[--fallback <r>] [--gate review|structured|spec|proposal] [--timeout-ms N] [--cwd <dir>] [--json]
[--fallback <r>] [--gate review|structured|spec|proposal] [--timeout-ms N] [--max-tokens N] [--cwd <dir>] [--json]
runners [--model <id>] [--json] the runner table: id, capabilities, available here
family <model-id> print the model family (anthropic, openai, google, ...)
Runners: codex-cli (repository review via codex exec), api (supplied-input review via
OPENAI_API_KEY/ANTHROPIC_API_KEY; needs --model), host-subagent (binds --result from a
cross-family subagent the host ran; needs --model). --native-model enforces the same-family
cross-family subagent the host ran; needs --model). --max-tokens caps the api runner's output
(Anthropic default 16384; its default time limit grows with it); a response cut off at the limit
is unavailable (truncated). --native-model enforces the same-family
rule. GSTACK_OUTSIDE_RUNNER names the fallback when --fallback is absent.
Prints OUTSIDE_STATUS: <status> provider=<runner> model=<id> family=<f> and writes <out>.meta.json.
codex_reviews=disabled (or an unreadable config) refuses every runner before it starts:
@@ -75,9 +78,12 @@ async function main(argv: string[]): Promise<number> {
if (!fs.existsSync(promptFile)) return usage(`prompt file not found: ${promptFile}`);
const gate = (flag(rest, '--gate') ?? 'review') as OutsideGate;
if (!['review', 'structured', 'spec', 'proposal'].includes(gate)) return usage('--gate must be review, structured, spec or proposal');
const maxTokensText = flag(rest, '--max-tokens');
const maxTokens = maxTokensText === undefined ? undefined : Number(maxTokensText);
if (maxTokens !== undefined && (!Number.isInteger(maxTokens) || maxTokens <= 0)) return usage('--max-tokens needs a positive integer');
const timeoutText = flag(rest, '--timeout-ms');
const timeoutMs = timeoutText === undefined ? 300_000 : Number(timeoutText);
if (!Number.isInteger(timeoutMs) || timeoutMs <= 0) return usage('--timeout-ms needs a positive integer');
const timeoutMs = timeoutText === undefined ? undefined : Number(timeoutText);
if (timeoutMs !== undefined && (!Number.isInteger(timeoutMs) || timeoutMs <= 0)) return usage('--timeout-ms needs a positive integer');
const fallbackText = flag(rest, '--fallback') ?? process.env.GSTACK_OUTSIDE_RUNNER;
if (fallbackText !== undefined && !isRunner(fallbackText)) return usage(`--fallback must be one of ${RUNNER_IDS.join(', ')}`);
const model = flag(rest, '--model');
@@ -110,7 +116,7 @@ async function main(argv: string[]): Promise<number> {
const stop = () => runner.cancel();
process.on('SIGTERM', stop);
process.on('SIGINT', stop);
const result = await runner.run({ promptFile: path.resolve(promptFile), outFile: path.resolve(outFile), model, timeoutMs, cwd, gate, resultFile: resultFile ? path.resolve(resultFile) : undefined });
const result = await runner.run({ promptFile: path.resolve(promptFile), outFile: path.resolve(outFile), model, timeoutMs: timeoutMs ?? (choice.runner === 'api' ? apiTimeoutMs(maxTokens) : 300_000), cwd, gate, maxTokens, resultFile: resultFile ? path.resolve(resultFile) : undefined });
fs.mkdirSync(path.dirname(path.resolve(outFile)), { recursive: true });
fs.writeFileSync(`${path.resolve(outFile)}.meta.json`, JSON.stringify(result, null, 2) + '\n');
if (json) process.stdout.write(JSON.stringify(result) + '\n');
+2 -2
View File
@@ -37,8 +37,8 @@ unknown is reserved for existing behavior you could not inspect. In a live audit
you never triggered are unknown, not extrapolated.
**Caps.** Name the path and item behind every cap and show the corrected error line.
- **4/10 (Critical finding, title `CAP4:<path>:<item>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
- **6/10 (High finding, title `CAP6:<path>:<item>`):** a path lacks an applicable fix, recovery, context or trace item.
- **4/10 (Critical finding, title `CAP4:<path>:<item> <what fails>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
- **6/10 (High finding, title `CAP6:<path>:<item> <what fails>`):** a path lacks an applicable fix, recovery, context or trace item.
Every mode, DX TRIAGE included, reports every CAP4 finding.
+7
View File
@@ -257,6 +257,13 @@ the same three paths. An errored init is skipped, so a later successful init
still binds. This denial applies when no init ever succeeded, or when an
errored init follows a good one.
An init whose output is not its JSON result (extra text, or no single text
block) also counts as failed, and the message adds "without a leading cd" to
the re-run hint. The one exception is the `Shell cwd was reset to <dir>` line
Claude Code adds as the last line of a Bash result when the command's `cd`
left the session's folder: the guard removes exactly that line before reading
an init or checkpoint result (#3651).
<a id="init-unbindable"></a>
### `init_unbindable`
Init ran through a shell variable, substitution, chaining, a pipe or a
+1 -1
View File
@@ -172,7 +172,7 @@ not a new ask ([carry-forward](unattended.md#carry-forward)).
**The record.** `gstack-approvals import --autoplan <run dir> --by "<owner>"` (with
`--stack <b1,b2,...>` for a stacked PR set) writes the approval row (date, the ask, the owner's exact words, the lane that answered as relayer,
what it covers); `gstack-approvals render` puts it in the report and the PR body. Before
what it covers; a gate answered with `--words-by agent:<id>` is listed under that agent as its own decision, not the owner's words); `gstack-approvals render` puts it in the report and the PR body. Before
training the PR the controller compares that row's words and `gate_rev` with the ask it
relayed and the reply it carried back; a row that does not match is not READY (shape A).
The brief's done line names the row. A plan approval is not a READY field; the
+6 -2
View File
@@ -962,8 +962,12 @@ blank (`PREREG_INCOMPLETE`). `arms` resolves the `eval-arms` model role against
`lib/pricing.ts` and stops on an unpriced model (`EVAL_PRICE_MISSING`); `dry-run`
runs every row through the stub model for $0 and flags malformed rows
(`EVAL_DRY_RUN_FAILED`); `pilot start|settle|release|report` runs three priced
items, records actual spend, extrapolates to the full set against the cap
(`EVAL_BUDGET_EXHAUSTED`) and prints the spend table the approval question shows.
items, records actual spend, extrapolates each priced arm to the full set against
the cap (`EVAL_BUDGET_EXHAUSTED`) and prints the spend table the approval question
shows. An arm on a subscription with no extra charge is declared with
`pilot start --seat-metered <arm,...>`: it reserves $0, settles as `unknown`, and
stays out of the projection until one of its attempts settles a real charge.
A run stops at the first quota or rate-limit error on such an arm.
`results --check <prereg> <results.md>` refuses a finished results file unless one
table holds a filled row for every arm, each `<model>#no-gstack` row beside its
variants (`EVAL_REPORT_NO_GSTACK_ROW`); no number leaves the run before it passes.
+14
View File
@@ -314,6 +314,20 @@ network.
**Expected result.** `OUTSIDE_STATUS: completed`.
<a id="outside-review-truncated"></a>
### `... outside review unavailable: the reviewer hit its output token limit and the response is cut off (...)`
**Meaning.** The api runner's provider stopped at the output token limit
(Anthropic `stop_reason: max_tokens`, OpenAI `finish_reason: length`), so the
review ends mid-sentence and its findings may be missing.
**Fix.** Re-run with a larger `--max-tokens` (`gstack-outside-voice run` or
`gstack-autoplan run`), or shorten the prompt. `gstack-autoplan run` sends a
default of 16384 to both providers, and OpenAI counts reasoning tokens toward
it; `gstack-outside-voice run` sends that default to Anthropic only.
**Expected result.** `OUTSIDE_STATUS: completed`.
<a id="outside-review-missing-markers"></a>
### `... outside review unavailable: the response lacks the markers this gate requires (...)`
+77 -10
View File
@@ -65,10 +65,12 @@ A run that writes a record ends with one line the parent greps:
GSTACK_RESULT: skill=<name> status=<status> run=<dir>
```
`status` is one of `complete`, `gate_pending`, `incomplete`, `interrupted` or
`refused`; `run` is where the run's record is. `/autoplan`
`status` is one of `complete`, `reviewed`, `gate_pending`, `incomplete`,
`interrupted` or `refused`; `run` is where the run's record is. `/autoplan`
always ends `gate_pending` when it reaches the final gate, because the gate is
written and never approved. Pure queries (`gstack-artifact schema`,
written and never approved. `/autoplan --ceo-only` ends `reviewed`: it runs the
CEO phase alone, asks nothing, writes no gate and approves nothing (see
[CEO-only runs](#ceo-only-runs)). Pure queries (`gstack-artifact schema`,
`gstack-gate render`) print no result line.
Not every workflow writes this line or a run directory today:
@@ -374,7 +376,14 @@ the plan.
refuses to open without one (`CONSENSUS_MISSING`).
3. **Loop `next` until `status=gate_pending`.** Eng always runs last. The gate
(`gate.json`, `gate_rev`) lists user challenges as approval items,
disagreements as auto items and `p1` plan approval; nothing is approved.
disagreements as auto items, each open Critical or High finding as an
auto item (a twin pair is one item; `a` applies the reviewer's fix to the
plan and is recommended, `b` carries it into the build as a blocking task)
and `p1` plan approval; nothing is approved. An unanswered auto item whose
recommendation changes the plan keeps the plan's direction, so only `rec`,
`p1b` (approve with the recommended changes) or an explicit `<id>a` applies
a fix; a finding carried with `b` stays
open, so a rerun that raises it again asks again.
4. **Answer the gate** with the reply grammar and the revision you read:
```bash
$AP answer --out <dir> --gate-rev 1 --reply "rec" --by human # every recommendation, plan changes included (APPLY: ...; amend_pending)
@@ -385,12 +394,18 @@ the plan.
means an agent relayed them, and then `--owner-reply` (or `-` for stdin,
`--owner-reply-file <path>`) stores the owner's words verbatim
(`OWNER_REPLY_REQUIRED`); owner words that parse to different choices are
refused (`GATE_OWNER_REPLY_MISMATCH`). An unattended run refuses an answer
refused (`GATE_OWNER_REPLY_MISMATCH`). The words default to the owner's: an
agent that answers in its own words (a Captain deciding on its own
authority) must pass `--words-by agent:<id>` with its own `--by` id. They
are then recorded as `answer_words` with `words_by`, never as
`owner_reply`; a later gate never carries them (`CARRY: no`), and
`gstack-approvals import` lists that row under the agent, not the owner.
An unattended run refuses an answer
without `--by` (`ANSWER_BY_REQUIRED`, exit 3) and records the value as
`answered_by` on every decision row and in the review record's gate line,
so a gate an agent closed is distinguishable from one the owner closed.
`gstack-autoplan recommend` changes one recommendation before relay; it
bumps the gate's `rec_rev`, and a later `rec` needs the `--rec-hash` from
bumps the gate's `rec_rev`, and a later `rec` or `p1b` needs the `--rec-hash` from
the render the owner saw (`GATE_REC_STALE`). Every `GATE:` line is followed
by `GATE_SPEND:` and `GATE_REC_COST:` (also `gate_cost` in `--json`; see
`autoplan/DRIVER.md`).
@@ -440,7 +455,19 @@ Concurrency and recovery: every state write is a locked read-modify-write
(`test/autoplan-run-recovery.test.ts`, `test/autoplan-run-concurrency.test.ts`,
`test/autoplan-answer-invalidation.test.ts`). `--spend-cap` admits each attempt
through `spend.json` (`SPEND_CAP_EXCEEDED` refuses, exit 3); host-subagent
spend is `unknown` and counted in `worst_case`. `--deadline <min>` makes the
spend is `unknown` and counted in `worst_case`. Under a cap, in any session
kind (or unattended on Capy), each host-subagent attempt, and each api attempt
whose model has no price, reserves the default per voice unless
`next --estimate-usd <n>` sets the reservation, which the run keeps for later
phases (`run` reserves an api attempt's priced worst case, prompt bytes / 3
input tokens plus the output limit, and settles it at its token usage; every
native call that went out and bound no result, whether it failed, was cancelled,
had its result refused, or was in flight when the process died (the next `run`
finds it), settles at its usage, or at its full reservation when the usage is
unknown, while one refused before anything was sent, such as with no API key,
keeps its attempt open for the rerun, and a failed outside call leaves the phase
native-only and stays counted at its reservation); `submit
--usage-usd <n>` settles an attempt at its real cost. `--deadline <min>` makes the
`run` subcommand cancel the children it started and marks the run
`interrupted`.
@@ -520,9 +547,8 @@ A gate revision after the first (`gate_rev` 2 and later, written after a
it when nothing new needs their decision. `gstack-autoplan` prints
`CARRIED: gate_rev=<n> <- gate_rev=<m> words="<their words>" by=<relayer> reason=<why>`,
then the usual `APPROVED:` lines, and journals `gate_carried`. The carry needs
all of: the earlier gate approved (`p1a` or `p1b`) with `rec` or "do it"
(not item tokens), no item there left pending, and the owner's words
recorded; that gate was not itself carried; every accepted change incorporated
all of: the earlier gate approved (`p1a` or `p1b`), no item there left
pending, and the owner's words recorded; that gate was not itself carried; every accepted change incorporated
or settled; the plan file's bytes equal those recorded at the gate or at the
`revise` that took the changes (any later edit refuses, an ELI5-only one
included); no urgent or
@@ -531,6 +557,13 @@ Anything else prints `CARRY: no (<why>)` (journal `carry_refused`) and the gate
goes to the owner as before. `--no-carry-forward` on the first `next` turns it
off for the run.
An earlier answer given item by item (`<id><option>` tokens, not `rec` or "do
it") carries only what the owner already answered: every item on the new gate
must be carried by origin (the same proposal, so the same choice), and only an
unreviewed post-cap delta (below) may be new. The carry replays the owner's own
`p1` choice and records their tokens verbatim as its basis; any new or changed
item, "do it" cases included, goes to the owner.
- **"do it".** When the owner's words were "do it" (their words, not a
`captain:` relay of them), a new `auto` item that changes no scope, cost or
direction, whose recommended option changes no plan text, and is no round
@@ -560,6 +593,40 @@ and `gstack-owner-brief render --captain` opens with "Nothing pending.
Carried forward from gate_rev <m>", followed by any items taken under "do it"
without being shown to the owner.
## CEO-only runs
`/autoplan --ceo-only` runs the CEO review by itself and asks nothing. It runs
in the session that invokes the skill; `gstack-autoplan`, the driver a parent
agent uses, has no CEO-only mode yet, and `run.json` does not accept the
`reviewed` status. It needs Claude Code: other hosts inline autoplan's sections
and do not carry this one yet. It is for a harness or a crew that wants the strategy pass
without the full gauntlet, such as an eval comparing gstack's CEO review with a
plain prompt.
- **What runs:** Phase 0, the outside-reviewer preflight and the CEO phase.
Design, DX, Eng and the final gate do not run.
- **How it decides:** every question takes its recommendation and is logged in
the Decision Audit Trail. The review mode comes from plan-ceo-review's 0E
rule (file count, then greenfield, added capability or fix), not autoplan's
fixed SELECTIVE EXPANSION, and the mode line says `CEO-only run`. Consent,
destructive and irreversible options take the conservative choice.
- **What it never does:** apply a user challenge (the plan's direction stays;
each challenge is listed), chain to another skill, promote the CEO plan, or
approve anything.
- **One model:** add `--no-outside` to skip the outside voice, so every
reviewer runs on the host model.
- **Where it writes:** the run directory (`.gstack/tmp/autoplan/` in the
repository), the reviewed plan, and CEO plans and task files under
`GSTACK_STATE_ROOT`. Point that variable inside a trial folder to keep every
write there.
- **What it leaves:** the reviewed plan (its original is kept at the restore
point), `<run dir>/ceo-review.md` (user challenges, scope added,
conservative choices, findings and the audit trail), and the last line
`GSTACK_RESULT: skill=autoplan status=reviewed run=<dir>`.
`test/skill-e2e-autoplan-ceo-only.test.ts` runs it on a plan that makes the
interactive CEO review ask a question and fails if this mode asks anything.
## Owner brief
`gstack-owner-brief render --out <dir>` is what the owner sees at the final
+12 -4
View File
@@ -138,7 +138,7 @@ Write (or open) the preregistration file. Each section is required and must be f
| Arms | one line per arm: the arm identity first (`<model>` or `<model>#<loadout>`), settings, per-item cost from Step 1b |
| Comparison | `stack: loaded` when any arm loads gstack or another skill stack; otherwise `stack: none` and a `why:` line |
| Decoys | inputs the scorer must reject or score low; how many, how they were made, what a decoy pass proves |
| Stop rules | the spend cap, a bar missed on the pilot, a scorer below its bar, `budget_exhausted` |
| Stop rules | the spend cap, a bar missed on the pilot, a scorer below its bar, `budget_exhausted`, the first quota or rate-limit error on a seat-metered arm |
| Held-out exposure | every held-out set the plan touches: prior exposure (which runs, which models), the reservation on it |
| No-gstack arm | required with `stack: loaded`: `arms:`, `inputs:`, `prompt:`, `panel:`, `report:` (below) |
@@ -254,7 +254,14 @@ anything else (`EVAL_DRY_RUN_FAILED`).
`--out`): admission is `spent + reserved + unknown + estimate <= cap`, under a file lock
shared with any concurrent worker. Run each `PILOT_ATTEMPT:` through the project's own
eval command, then settle it with the actual cost from the provider's usage field, or
`unknown` when the call was seat-metered or CLI-metered:
`unknown` when the call was seat-metered or CLI-metered.
An arm on a subscription with no extra charge (say `claude -p` on a seat) is declared at
start with `--seat-metered <arm,...>`, each also in `--arms`. Its attempts hold none of the
cap, settle as `unknown` and get a `PILOT_SEAT_METERED:` line instead of a projection; an
arm that settles a real charge is projected as priced. The first quota or rate-limit error
on such an arm ends the run: no retry, no new attempt; release the unstarted ones. Settle
each attempt, then report:
```bash
~/.claude/skills/gstack/bin/gstack-eval-plan pilot settle --out <outdir> --attempt <attempt-id> --usd <usd|unknown>
@@ -263,8 +270,9 @@ eval command, then settle it with the actual cost from the provider's usage fiel
`pilot report` prints the spend table (`spent`, `reserved`, `unknown charges`, the worst case
with one in-flight call per worker, the worst-case overrun against the cap) and
`PILOT_PROJECTION:` — mean cost per item, its sample deviation, the projection to
`--total` items across the arms, and a 95% band. Regenerate the table at every later gate
one `PILOT_ARM:` line per money arm (its own mean, deviation, projection and band), then
`PILOT_PROJECTION:` — the arms summed: cost per item, deviation, the projection to
`--total` items, and a 95% band. Regenerate the table at every later gate
of the run. Stop and report, without asking for approval, when the pilot missed a bar from
the preregistration, when `status=budget_exhausted`, or when `verdict=exceeds`.
+12 -4
View File
@@ -60,7 +60,7 @@ Write (or open) the preregistration file. Each section is required and must be f
| Arms | one line per arm: the arm identity first (`<model>` or `<model>#<loadout>`), settings, per-item cost from Step 1b |
| Comparison | `stack: loaded` when any arm loads gstack or another skill stack; otherwise `stack: none` and a `why:` line |
| Decoys | inputs the scorer must reject or score low; how many, how they were made, what a decoy pass proves |
| Stop rules | the spend cap, a bar missed on the pilot, a scorer below its bar, `budget_exhausted` |
| Stop rules | the spend cap, a bar missed on the pilot, a scorer below its bar, `budget_exhausted`, the first quota or rate-limit error on a seat-metered arm |
| Held-out exposure | every held-out set the plan touches: prior exposure (which runs, which models), the reservation on it |
| No-gstack arm | required with `stack: loaded`: `arms:`, `inputs:`, `prompt:`, `panel:`, `report:` (below) |
@@ -176,7 +176,14 @@ anything else (`EVAL_DRY_RUN_FAILED`).
`--out`): admission is `spent + reserved + unknown + estimate <= cap`, under a file lock
shared with any concurrent worker. Run each `PILOT_ATTEMPT:` through the project's own
eval command, then settle it with the actual cost from the provider's usage field, or
`unknown` when the call was seat-metered or CLI-metered:
`unknown` when the call was seat-metered or CLI-metered.
An arm on a subscription with no extra charge (say `claude -p` on a seat) is declared at
start with `--seat-metered <arm,...>`, each also in `--arms`. Its attempts hold none of the
cap, settle as `unknown` and get a `PILOT_SEAT_METERED:` line instead of a projection; an
arm that settles a real charge is projected as priced. The first quota or rate-limit error
on such an arm ends the run: no retry, no new attempt; release the unstarted ones. Settle
each attempt, then report:
```bash
~/.claude/skills/gstack/bin/gstack-eval-plan pilot settle --out <outdir> --attempt <attempt-id> --usd <usd|unknown>
@@ -185,8 +192,9 @@ eval command, then settle it with the actual cost from the provider's usage fiel
`pilot report` prints the spend table (`spent`, `reserved`, `unknown charges`, the worst case
with one in-flight call per worker, the worst-case overrun against the cap) and
`PILOT_PROJECTION:` — mean cost per item, its sample deviation, the projection to
`--total` items across the arms, and a 95% band. Regenerate the table at every later gate
one `PILOT_ARM:` line per money arm (its own mean, deviation, projection and band), then
`PILOT_PROJECTION:` — the arms summed: cost per item, deviation, the projection to
`--total` items, and a 95% band. Regenerate the table at every later gate
of the run. Stop and report, without asking for approval, when the pilot missed a bar from
the preregistration, when `status=budget_exhausted`, or when `verdict=exceeds`.
+1
View File
@@ -78,6 +78,7 @@ function planGate(dir: string, run: Record<string, any>): Resolution {
const p1 = readJsonl(path.join(dir, 'decisions.jsonl')).find(d => d.label === 'P1' && d.gate_rev === rev);
const approved = run.status === 'complete' && typeof gate.plan === 'string' && !['c', 'e'].includes(gate.plan) && p1 && p1.chosen === gate.plan && p1.status !== 'pending';
if (!approved) return { state: 'rejected', why: `run ${run.run} holds this plan but its gate is not APPROVED (status ${run.status}, p1 ${p1?.chosen ?? 'unanswered'})` };
if (p1.words_by) return { state: 'rejected', why: `run ${run.run}: its gate was answered in ${p1.words_by}'s own words, not the owner's` };
return { state: 'resolved', evidence: `plan ${run.run} gate_rev=${rev} p1=${gate.plan} answered_by=${gate.answered_by ?? p1.answered_by ?? 'unrecorded'}`, text: fs.readFileSync(planFile, 'utf8') };
}
+4 -3
View File
@@ -235,7 +235,7 @@ export function renderRecord(read: ReadResult, loc: StoreLocation, o: { heading?
return [...head, ...lines, '', `Source: gstack-approvals record for ${cell(loc.branch)} ${APPROVALS_FOOTER_END}`, ''].join('\n');
}
interface DecisionLike { id?: string; title?: string; status?: string; chosen?: string | null; recommended?: string | null; gate_rev?: number; answered_at?: string; answered_by?: string; reply?: string; owner_reply?: string; carried_from?: string; label?: string; chosen_text?: string;
interface DecisionLike { id?: string; title?: string; status?: string; chosen?: string | null; recommended?: string | null; gate_rev?: number; answered_at?: string; answered_by?: string; reply?: string; owner_reply?: string; answer_words?: string; words_by?: string; carried_from?: string; label?: string; chosen_text?: string;
carried_gate_from?: string; carried_gate_rev?: number; carried_answered_by?: string; carried_under?: string; carry_check?: string }
const stripRef = (t: string) => t.replace(/\s*\[[^\]]*\]\s*$/, '').trim();
@@ -278,7 +278,8 @@ export function importAutoplan(runDir: string, branch: string, by?: string): App
const answerer = first.answered_by ?? 'unknown';
const carried = p1?.carried_gate_from ? p1 : undefined;
const relayed = carried?.carried_answered_by && carried.carried_answered_by !== 'human' ? `, relayed by ${carried.carried_answered_by}` : '';
const who = carried ? `${by ?? 'the owner'} (words from gate_rev ${carried.carried_gate_rev}${relayed}), carried forward by gstack-autoplan`
const who = first.words_by ? `${first.words_by}, its own decision (not the owner's words)`
: carried ? `${by ?? 'the owner'} (words from gate_rev ${carried.carried_gate_rev}${relayed}), carried forward by gstack-autoplan`
: answerer === 'human' ? (by ? `${by}, typed at the gate` : 'the owner, typed at the gate') : `${by ?? 'the owner'}, relayed by ${answerer}`;
const unseen = items.filter(r => r.carried_under).map(r => stripRef(r.title ?? ''));
const checked = [...new Set(items.map(r => r.carry_check).filter((c): c is string => !!c))];
@@ -286,7 +287,7 @@ export function importAutoplan(runDir: string, branch: string, by?: string): App
unseen.length ? `taken under "do it", not seen by the owner: ${unseen.join('; ')}` : '', checked.length ? `post-cap delta checked by ${checked.join(', ')}` : ''].filter(Boolean).join('; ');
const built = buildRow({
ask: `Approve the plan "${planTitle}" (autoplan run ${runId}, ${carried ? `gate_rev ${rev} carried forward from gate_rev ${carried.carried_gate_rev}, ` : ''}${items.length + (p1 ? 1 : 0)} decisions)`.slice(0, CAPS.ask),
words: first.owner_reply ?? first.reply ?? '(no words recorded)',
words: first.owner_reply ?? first.answer_words ?? first.reply ?? '(no words recorded)',
by: who.slice(0, CAPS.by),
scope: scope.slice(0, CAPS.scope),
pending: (stillPending.length ? `${stillPending.length} decision(s) unanswered: ${stillPending.map(r => stripRef(r.title ?? '')).join('; ')}` : outcome === 'pending' ? 'the plan approval itself' : 'none').slice(0, CAPS.pending),
+18 -10
View File
@@ -16,12 +16,12 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { ANSWERED_BY_RE, type GateItem } from './gate-list';
import { settledIds } from './autoplan-confirm';
import type { ConsensusRecord } from './autoplan-reconcile';
import { sameProblem, type ConsensusRecord } from './autoplan-reconcile';
import { FILES, RunError, journal, now, readJournal, voiceRows, type RunState, type RunStop } from './autoplan-state';
import { readLedger, release, setCap, summarize } from './spend-ledger';
export const DEFAULT_MAX_ROUNDS = 2;
/** Reserved per voice for a host-subagent attempt with no `--estimate-usd` in an unattended Capy run. */
/** Reserved per voice for a host-subagent attempt with no `--estimate-usd` in an unattended run with a spend cap or on Capy. */
export const DEFAULT_HOST_ESTIMATE_USD = 5;
/** Round r of a phase: 1 plus the times it was reopened. */
@@ -35,17 +35,20 @@ export const atCap = (state: RunState, phase: string): boolean => { const cap =
/** The phase is about to open past its cap (a rerun-all reply reopened it). */
export const overCap = (state: RunState, phase: string): boolean => { const cap = roundCap(state, phase); return cap !== null && roundOf(state, phase) > cap; };
/** Open High or Critical findings of a closed phase: the deferred counts, else bound rows not settled or answered, an outside row paired with a native one counted once. */
/** Open High or Critical findings of a closed phase: the deferred counts, else bound rows not settled or answered, an outside row naming the same problem (`sameProblem`) as a High or Critical native one counted once. */
function openHigh(state: RunState, phase: string): number {
const ps = state.phases[phase];
if (ps?.deferred) return (ps.deferred.counts.High ?? 0) + (ps.deferred.counts.Critical ?? 0);
const file = path.join(state.out, `${phase}-consensus.json`);
let rows: ConsensusRecord['rows'] = [];
try { rows = fs.existsSync(file) ? (JSON.parse(fs.readFileSync(file, 'utf8')) as ConsensusRecord).rows : []; } catch {}
const paired = new Set(rows.filter(r => r.native_id).map(r => r.outside_id));
const high = (s: string | undefined) => s === 'High' || s === 'Critical';
const native = voiceRows(state, phase, 'native');
const nativeHigh = new Set(native.filter(r => high(r.severity)).map(r => r.id));
const paired = new Set(rows.filter(r => r.native_id && sameProblem(r) && nativeHigh.has(r.native_id)).map(r => r.outside_id));
const answered = settledIds(state);
return [...voiceRows(state, phase, 'native'), ...voiceRows(state, phase, 'outside')]
.filter(r => !paired.has(r.id) && !(r as { settled_by?: string }).settled_by && !answered.has(r.id) && (r.severity === 'High' || r.severity === 'Critical')).length;
return [...native, ...voiceRows(state, phase, 'outside')]
.filter(r => !paired.has(r.id) && !(r as { settled_by?: string }).settled_by && !answered.has(r.id) && high(r.severity)).length;
}
/** At the round cap with High or Critical findings still open, more rounds buy nothing: the owner decides. */
@@ -59,11 +62,16 @@ export function roundsLine(state: RunState): string {
return `ROUNDS: ${state.phase_order.map(cell).join(' ')} max_rounds=${state.max_rounds ?? 'none'} host_spend=unknown`;
}
/** The estimate one new attempt reserves: the flag, else the Capy unattended default, else 0 (host spend unknown, not admitted). */
/**
* The estimate one new attempt reserves: the flag, else the run's persisted `--estimate-usd`, else the default whenever
* the run has a spend cap (any session kind) or an unattended run is on Capy, else 0 (host spend unknown, not admitted).
* `run` prices its api attempts from the price table instead (NextOptions.price); an unpriced model falls back to this.
*/
export function attemptEstimate(state: RunState, flag: number | undefined, env: NodeJS.ProcessEnv): { usd: number; line?: string } {
if (flag !== undefined) return { usd: flag };
if (state.session_kind !== 'unattended' || env.GSTACK_CAPY !== '1') return { usd: 0 };
return { usd: DEFAULT_HOST_ESTIMATE_USD, line: `ESTIMATE: ${DEFAULT_HOST_ESTIMATE_USD.toFixed(2)} per voice reserved for each host-subagent attempt (Capy unattended default; --estimate-usd <n> sets it); actual host spend stays unknown` };
const set = flag ?? state.estimate_usd;
if (set !== undefined) return { usd: set };
if (state.spend_cap_usd === null && (state.session_kind !== 'unattended' || env.GSTACK_CAPY !== '1')) return { usd: 0 };
return { usd: DEFAULT_HOST_ESTIMATE_USD, line: `ESTIMATE: ${DEFAULT_HOST_ESTIMATE_USD.toFixed(2)} per voice reserved for each host-subagent attempt (default under a spend cap, or unattended on Capy; --estimate-usd <n> sets it for the rest of the run); actual host spend stays unknown` };
}
/** Stop the run: terminal `incomplete` with the structured budget_exhausted stop, journaled. Returns the STOP and RECOVER lines. */
+12 -8
View File
@@ -7,11 +7,13 @@
* carried answer is the owner's own earlier words, and every refusal says which condition failed.
*
* A carry needs all of: carry not turned off (`--no-carry-forward`); gate_rev N approved (p1 a or b)
* with the owner's words recorded, and not itself a carry; every change accepted at N
* with the owner's words recorded, and not itself a carry; nothing left pending there; every change accepted at N
* incorporated or settled; the plan file unchanged since the gate or the revise that took those
* changes, and the reviewed plan equal to it; no open urgent finding; and every new item already
* carried. Two kinds of new item do not block: items a "do it" covers (the owner's words are
* "do it": an auto item that changes no scope, cost or direction and is no round grant), and a
* carried. An answer given item by item (tokens, not `rec`) carries only that: every item on
* N+1 already carried by origin, its p1 choice replayed and its tokens recorded verbatim as the
* basis; any other new or changed item goes to the owner (an unreviewed capped delta, below, is the one exception). After `rec` or "do it", two kinds of new
* item do not block: items a "do it" covers (the owner's words are "do it": an auto item that changes no scope, cost or direction and is no round grant), and a
* capped-delta item recommending "accept", which first gets one reviewer on just the delta
* (`CARRY_CHECK:`, answered with `gstack-autoplan check`); a High or Critical finding there
* means no carry, and the gate goes to the owner as before.
@@ -92,7 +94,8 @@ function deltaSince(state: RunState, rev: number): { diff: string; sha: string }
return { diff, sha: sha256(diff) };
}
interface Source { id: string; gate_rev: number; words: string; by?: string }
/** `p1`: an item-token answer's own p1 choice, replayed as given. */
interface Source { id: string; gate_rev: number; words: string; by?: string; p1?: string }
type Plan = { source: Source; doIt: GateItem[]; deltas: GateItem[] } | { reason: string; from?: number };
/** Every condition but the delta check; returns why not, or what the carry takes. */
@@ -105,7 +108,8 @@ function assess(state: RunState, prev: GateList, list: GateList): Plan {
if (unanswered) return { reason: `gate_rev ${from} left ${unanswered.label ?? unanswered.id} unanswered`, from };
if (p1.chosen !== 'a' && p1.chosen !== 'b') return { reason: `gate_rev ${from} answer was p1${p1.chosen}, not an approval`, from };
if (p1.carried_gate_from) return { reason: `gate_rev ${from} was itself carried; the owner answers this one`, from };
if (parseReply(p1.reply ?? '', prev).mode !== 'rec') return { reason: `gate_rev ${from} was answered with item tokens, not rec or "do it"`, from };
const tokens = parseReply(p1.reply ?? '', prev).mode !== 'rec';
if (p1.words_by) return { reason: `gate_rev ${from} was answered in ${p1.words_by}'s own words, not the owner's`, from };
const words = p1.owner_reply ?? (p1.answered_by === 'human' ? p1.reply : undefined);
if (!words?.trim()) return { reason: `no owner words recorded at gate_rev ${from}`, from };
const amendments = (state.amendments ?? []).filter(a => a.gate_rev === from);
@@ -125,10 +129,10 @@ function assess(state: RunState, prev: GateList, list: GateList): Plan {
if (item.id === 'p1' || item.carried) continue;
const option = item.options.find(o => o.key === item.recommended);
if (item.cap?.kind === 'delta' && !option?.changes_plan && item.recommended === 'a') { deltas.push(item); continue; }
if (isDoIt(words) && item.kind === 'auto' && !item.cap && !option?.changes_plan && !['scope', 'cost', 'direction'].includes(item.changes ?? '')) { doIt.push(item); continue; }
if (!tokens && isDoIt(words) && item.kind === 'auto' && !item.cap && !option?.changes_plan && !['scope', 'cost', 'direction'].includes(item.changes ?? '')) { doIt.push(item); continue; }
return { reason: `new item ${item.id}: ${item.title}`, from };
}
return { source: { id: p1.id, gate_rev: from, words, by: p1.answered_by }, doIt, deltas };
return { source: { id: p1.id, gate_rev: from, words, by: p1.answered_by, ...(tokens ? { p1: p1.chosen } : {}) }, doIt, deltas };
}
/** Called by finishGate once gate_rev N+1 is written (status gate_pending): carry, set up the delta check, or say why not. */
@@ -178,7 +182,7 @@ function dispatchCheck(state: RunState, prev: GateList, list: GateList, plan: Ex
function carry(state: RunState, list: GateList, source: Source, doIt: GateItem[], deltas: GateItem[], check?: CarryCheck): CarryRecord {
const rev = list.gate_rev;
const p1 = list.items.find(i => i.id === 'p1')!;
const reply = list.items.map(i => `${i.id}${i.id === 'p1' ? p1.recommended : i.carried?.chosen ?? i.recommended}`).join(' ');
const reply = list.items.map(i => `${i.id}${i.id === 'p1' ? source.p1 ?? p1.recommended : i.carried?.chosen ?? i.recommended}`).join(' ');
const reason = [doIt.length || deltas.length ? '' : 'no new choice', doIt.length ? `"do it" covers ${doIt.map(i => i.id).join(',')}` : '', deltas.length ? `delta check passed on ${deltas.map(i => i.id).join(',')}` : '',
'plan unchanged since the accepted changes'].filter(Boolean).join('; ');
const answered = answerGate(state, rev, reply, 'agent:gstack-autoplan', { carry: { from: source.id, gate_rev: source.gate_rev, words: source.words, by: source.by, reason } });
+12 -5
View File
@@ -13,7 +13,7 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import type { FindingRow } from './headless-artifacts';
import { readJsonl } from './headless-artifacts';
import { jaccard, tokens } from './autoplan-reconcile';
import { titleScore } from './autoplan-reconcile';
import { readDecisions, type PhaseState, type RunState } from './autoplan-state';
export interface DiffLine { op: ' ' | '+' | '-'; text: string; a?: number; b?: number }
@@ -108,11 +108,18 @@ export function reviewRows(state: RunState, prev: PhaseState | undefined): Findi
const earlierRows = (state: RunState, phase: string): FindingRow[] =>
state.history.filter(h => h.event === 'reopened' && h.phase === phase).flatMap(h => reviewRows(state, h.previous as PhaseState | undefined));
/** Earlier finding id -> the answered decision whose gate item named it. */
export function settledIds(state: RunState): Map<string, string> {
/** The origin tag of an open Critical/High finding's gate item: only `a` (its fix applied to the plan) settles it. */
export const FINDING_ORIGIN = 'finding';
/**
* Earlier finding id -> the answered decision whose gate item named it. A finding carried into the build stays open,
* and `findings: false` also leaves an applied one open, so a confirm round re-raising it is never suppressed.
*/
export function settledIds(state: RunState, o: { findings?: boolean } = {}): Map<string, string> {
const out = new Map<string, string>();
for (const d of readDecisions(state.out)) {
if (d.chosen == null || d.status === 'pending' || !d.origin) continue;
if (d.origin.endsWith(`|${FINDING_ORIGIN}`) && (d.chosen !== 'a' || o.findings === false)) continue;
for (const id of (d.origin.split('|')[1] ?? '').split('+')) if (id && id !== 'null') out.set(id, d.id);
}
return out;
@@ -125,13 +132,13 @@ export function settledIds(state: RunState): Map<string, string> {
export function annotateConfirm(state: RunState, phase: string, rows: FindingRow[], ctx: { planName: string; planSource: string; snapshotPath: string }): { correlated: number; settled: number } {
const earlier = earlierRows(state, phase);
if (!earlier.length) return { correlated: 0, settled: 0 };
const settled = settledIds(state);
const settled = settledIds(state, { findings: false });
const added = new Set((phaseDelta(state, phase, ctx.snapshotPath)?.ops ?? []).filter(o => o.op === '+' && o.text.trim()).map(o => o.text.trim()));
const planLines = (read(ctx.planSource) ?? '').split('\n');
const isPlan = (f: FindingRow) => f.file === ctx.planName;
let correlated = 0, settledCount = 0;
for (const row of rows as Array<FindingRow & { correlates?: string[]; settled_by?: string }>) {
const same = earlier.filter(e => jaccard(tokens(e.title), tokens(row.title)) >= 0.6 || (!!e.file && !isPlan(e) && e.file === row.file && e.line === row.line));
const same = earlier.filter(e => titleScore(e.title, row.title) >= 0.6 || (!!e.file && !isPlan(e) && e.file === row.file && e.line === row.line));
if (!same.length) continue;
row.correlates = same.map(e => e.id);
correlated++;
+32 -12
View File
@@ -3,7 +3,7 @@
* (plan B2, B9, B11). `closePhase` reconciles the outside pass against the
* native one from canonical ids (lib/autoplan-reconcile.ts), writes the
* consensus record and the timing row, and marks the phase closed.
* `finishGate` turns disagreements and user challenges into one gate list
* `finishGate` turns disagreements, user challenges and open Critical/High findings into one gate list
* (lib/gate-list.ts: `d<n>` auto items, `uc<n>` approval items, `p1` the plan
* approval), writes decisions.jsonl, tasks.jsonl and the exported files, and
* leaves the run `gate_pending`: unattended never approves. `answerGate`
@@ -20,15 +20,15 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { exportRun } from './autoplan-export';
import { driftLine, reviewHash } from './autoplan-plan-text';
import { consensusRecord, reconcile, remedyKey, renderConsensus, SETTLEABLE_CHANGES, type ConsensusRecord, type ReconciliationOverride, type ReconciliationRow } from './autoplan-reconcile';
import { consensusRecord, reconcile, remedyKey, renderConsensus, sameProblem, SETTLEABLE_CHANGES, type ConsensusRecord, type ReconciliationOverride, type ReconciliationRow } from './autoplan-reconcile';
import type { Phase } from './autoplan-prompts';
import { FILES, RunError, journal as journalRow, now, readDecisions, voiceRows, voiceTerminal, writeDecisions, type Amendment, type RunState, type VoiceState } from './autoplan-state';
import { applyCapChoices, pendingDeltas, releaseUndispatched, roundCap, roundOf } from './autoplan-budget';
import { evaluateCarry, gateEvidence, voidPendingCheck } from './autoplan-carry';
import { buildEli5, capyRelayLine } from './captain-brief';
import { previousReview, reviewRows, settledIds } from './autoplan-confirm';
import { FINDING_ORIGIN, previousReview, reviewRows, settledIds } from './autoplan-confirm';
import { phaseClose } from './autoplan-timing';
import { ANSWERED_BY_RE, applyReply, decisionId, decisionRows, keepDirection, parseReply, pendingHint, recHash, validateGateList, type GateChoice, type GateItem, type GateList, type GateOption, type ParsedReply } from './gate-list';
import { ANSWERED_BY_RE, applyReply, autoDefault, decisionId, decisionRows, keepDirection, parseReply, pendingHint, recHash, validateGateList, type GateChoice, type GateItem, type GateList, type GateOption, type ParsedReply } from './gate-list';
import type { FindingRow } from './headless-artifacts';
import { readJsonl, sha256File } from './headless-artifacts';
import { closeDraftDirectionsFile, loadBriefInputs } from './owner-brief';
@@ -205,14 +205,15 @@ export function buildGate(state: RunState, gateRev: number): GateList {
const byId = new Map([...native, ...outside].map(f => [f.id, f]));
const reran = gateRev > 1 && state.history.some(h => h.event === 'reopened' && h.phase === phase);
const origin = (ids: string, options: GateOption[]) => [phase, ids, ps.closed_at, ps.snapshot, optionHash(options)].join('|');
const push = (item: GateItem, ids: string) => items.push({ ...item, origin: origin(ids, item.options) });
const push = (item: GateItem, ids: string, tag?: string) => items.push({ ...item, origin: [origin(ids, item.options), ...(tag ? [tag] : [])].join('|') });
const challenged = new Set<string>();
const twins = new Map(consensus.rows.filter(r => sameProblem(r) && (r.disposition === 'confirmed' || r.disposition === 'unverified')).map(r => [r.native_id!, r.outside_id]));
for (const row of consensus.rows) {
const o = byId.get(row.outside_id)! as FindingRow & { user_challenge?: boolean; changes?: string; settled_by?: string };
if (o.settled_by) continue;
const n = (row.native_id ? byId.get(row.native_id) : undefined) as (FindingRow & { user_challenge?: boolean; changes?: string }) | undefined;
const ids = `${row.native_id}+${row.outside_id}`;
if (o.user_challenge && n?.user_challenge && row.disposition !== 'new') {
if (o.user_challenge && n?.user_challenge && row.disposition !== 'new' && row.basis !== 'file') {
challenged.add(o.id); challenged.add(n.id);
const title = `${n.title} [${row.native_id} + ${row.outside_id}]`;
if (reran && !n.changes && !o.changes) { push(agentDecision(`d${++d}`, title, phase, 'both voices recommend changing your stated direction'), ids); continue; }
@@ -223,6 +224,7 @@ export function buildGate(state: RunState, gateRev: number): GateList {
continue;
}
if (row.disposition === 'disagree') {
challenged.add(o.id); challenged.add(n!.id);
push({ id: `d${++d}`, title: `${n!.title} [${row.native_id} vs ${row.outside_id}]`, kind: 'auto', recommended: 'a', phase,
why: `voices disagree: ${row.resolution}`, cost: `b reopens ${phase}${phase === 'eng' ? '' : ' and eng'}`,
options: [{ key: 'a', text: `native position (${row.severity.native})`, changes_plan: false }, { key: 'b', text: `outside position (${row.severity.outside})`, changes_plan: true }] }, ids);
@@ -235,6 +237,20 @@ export function buildGate(state: RunState, gateRev: number): GateList {
push({ id: `d${++d}`, title: `${f.title} [${f.id}]`, kind: 'auto', recommended: 'a', phase, why, ...(f.changes ? { changes: f.changes } : {}),
cost: `b reopens ${phase}${phase === 'eng' ? '' : ' and eng'}`, options: [KEEP('keep your direction'), { key: 'b', text: 'accept the change', changes_plan: true }] }, f.id);
}
// An open Critical or High finding the items above do not cover: rec applies its fix to the plan.
const pairedOutside = new Set(twins.values());
for (const f of [...native, ...outside] as Array<FindingRow & { user_challenge?: boolean; settled_by?: string }>) {
if (f.voice === 'outside' && pairedOutside.has(f.id)) continue;
const group = [f, ...(twins.has(f.id) ? [byId.get(twins.get(f.id)!)!] : [])] as Array<FindingRow & { user_challenge?: boolean; settled_by?: string }>;
if (group.some(g => challenged.has(g.id) || g.user_challenge || g.settled_by)) continue;
const severity = group.some(g => g.severity === 'Critical') ? 'Critical' : group.some(g => g.severity === 'High') ? 'High' : null;
if (!severity) continue;
const ids = group.map(g => g.id);
const fixes = [...new Set(group.map(g => g.resolution).filter((r): r is string => !!r))];
push({ id: `d${++d}`, title: `${severity}: ${f.title} [${ids.join(' + ')}]`, kind: 'auto', recommended: 'a', phase, finding: ids,
why: `an open ${severity} finding (${group.map(g => g.voice).join(' + ')}); rec applies its fix to the plan`, cost: `a reopens ${phase}${phase === 'eng' ? '' : ' and eng'} once the plan is revised`,
options: [{ key: 'a', text: `apply the fix to the plan: ${fixes.length ? fixes.join('; ') : `resolve "${f.title}"`}`, changes_plan: true }, { key: 'b', text: 'carry it into the build as a blocking task (no plan change)', changes_plan: false }] }, ids.join('+'), FINDING_ORIGIN);
}
}
items.push(...capItems(state, { d: () => `d${++d}`, uc: () => `uc${++uc}` }));
items.push(planApproval(state, items));
@@ -277,7 +293,7 @@ export function recommendGate(state: RunState, gateRev: number, itemId: string,
writeDecisions(state.out, readDecisions(state.out).map(r => {
const i = updated.get(r.id);
if (!i) return r;
return { ...r, recommended: i.recommended, ...(i.kind === 'auto' ? { default_if_unanswered: i.recommended } : {}), ...(i.recommended_basis ? { recommended_basis: i.recommended_basis } : {}), ...(i.recommended_by ? { recommended_by: i.recommended_by } : {}) };
return { ...r, recommended: i.recommended, ...(i.kind === 'auto' ? { default_if_unanswered: autoDefault(i) } : {}), ...(i.recommended_basis ? { recommended_basis: i.recommended_basis } : {}), ...(i.recommended_by ? { recommended_by: i.recommended_by } : {}) };
}));
const hash = recHash(list);
journalRow(state.out, { event: 'recommended', gate_rev: gateRev, item: itemId, option, basis, by, rec_rev: list.rec_rev, rec_hash: hash });
@@ -344,6 +360,8 @@ export interface AnswerOptions {
recHash?: string;
/** The owner's verbatim words; required for every `--by agent:<id>` answer. */
ownerReply?: string;
/** `agent:<id>` when those words are the answering agent's own: recorded as `answer_words`/`words_by`, never as `owner_reply`, and never carried. */
wordsBy?: string;
/** A whole-gate carry (lib/autoplan-carry.ts): the earlier p1 decision and the owner's words it reuses. Replaces ownerReply and its re-parse. */
carry?: { from: string; gate_rev: number; words: string; by?: string; reason: string };
}
@@ -396,22 +414,24 @@ export function answerGate(state: RunState, gateRev: number, reply: string, answ
const currentHash = recHash(list);
if (o.recHash !== undefined && o.recHash !== currentHash) throw new RunError('GATE_REC_STALE', `--rec-hash ${o.recHash} was rendered before the recommendations changed; current rec_hash=${currentHash}`, 3);
if (parsed.mode === 'rec' && list.rec_rev === undefined && o.recHash === undefined) throw new RunError('GATE_REC_STALE', `this gate was written before \`rec\`/\`all\` took plan changes (gate.json has no rec_rev), so the owner saw a different meaning; render it again (gstack-gate render ${gateFile} --all), relay it, and answer with --rec-hash ${currentHash}, or reply with explicit <id><option> tokens`, 3);
if (parsed.mode === 'rec' && (list.rec_rev ?? 0) > 0 && o.recHash === undefined) throw new RunError('GATE_REC_STALE', `recommend changed this gate (rec_rev=${list.rec_rev}); pass --rec-hash from the render the owner saw (current rec_hash=${currentHash})`, 3);
const takesRecs = parsed.mode === 'rec' || parsed.choices.some(c => c.id === 'p1' && c.chosen === 'b' && !c.prior);
if (takesRecs && (list.rec_rev ?? 0) > 0 && o.recHash === undefined) throw new RunError('GATE_REC_STALE', `recommend changed this gate (rec_rev=${list.rec_rev}); \`rec\` and \`p1b\` take its recommendations, so pass --rec-hash from the render the owner saw (current rec_hash=${currentHash})`, 3);
const ownerParsed = o.ownerReply !== undefined && !o.carry ? checkOwnerReply(list, gateRev, prior, parsed, o.ownerReply) : undefined;
const provenance = o.carry ? { owner_reply: o.carry.words, carried_gate_from: o.carry.from, carried_gate_rev: o.carry.gate_rev, carry_reason: o.carry.reason, ...(o.carry.by ? { carried_answered_by: o.carry.by } : {}) }
: o.ownerReply !== undefined && o.wordsBy ? { answer_words: o.ownerReply, answer_words_parsed: ownerParsed, words_by: o.wordsBy }
: o.ownerReply !== undefined ? { owner_reply: o.ownerReply, owner_reply_parsed: ownerParsed } : {};
const itemOf = (id: string) => list.items.find(i => i.id === id)!;
const optionOf = (c: GateChoice) => itemOf(c.id).options.find(o => o.key === c.chosen)!;
const plan = parsed.choices.find(c => c.id === 'p1');
const lines: string[] = o.carry ? [] : voidPendingCheck(state, `the owner answered gate_rev ${gateRev}`);
const answeredAt = now();
const record = () => writeDecisions(state.out, applyReply(rows, parsed, answeredAt, answeredBy, provenance).map(r => {
const record = (answered = parsed) => writeDecisions(state.out, applyReply(rows, answered, answeredAt, answeredBy, provenance).map(r => {
const c = r.gate_rev === list.gate_rev && r.chosen && !r.chosen_text ? itemOf((r.label ?? '').toLowerCase())?.options.find(o => o.key === r.chosen) : undefined;
return c ? { ...r, chosen_text: c.text } : r;
}));
if (!plan || parsed.pending.length) {
// Approvals unanswered: record what was answered, keep the gate pending; nothing reviewed changes.
record();
// Approvals unanswered: record what was answered, but not an auto default that p1b would change (the final answer decides it).
record({ ...parsed, choices: parsed.choices.filter(c => c.how !== 'auto_default' || c.chosen === itemOf(c.id).recommended) });
const pending = parsed.pending.length ? parsed.pending : ['p1'];
state.gate = { ...(state.gate ?? {}), last_reply: reply, pending };
lines.push(`GATE_PENDING: ${pending.join(', ')} unanswered (${pendingHint(list, pending)})`);
@@ -421,7 +441,7 @@ export function answerGate(state: RunState, gateRev: number, reply: string, answ
const carried = (c: GateChoice) => itemOf(c.id).carried?.chosen === c.chosen;
const changing = proceeds ? parsed.choices.filter(c => c.id !== 'p1' && !carried(c) && itemOf(c.id).reopens !== false && (optionOf(c).changes_plan ?? c.how === 'override')) : [];
const tokens = (cs: GateChoice[]) => cs.map(c => `${c.id}${c.chosen}`);
const overrides = parsed.choices.filter(c => c.how === 'override' && c.id !== 'p1');
const overrides = parsed.choices.filter(c => c.id !== 'p1' && (c.how === 'override' || (c.how === 'auto_default' && c.chosen !== itemOf(c.id).recommended)));
const reopenSet = new Set<Phase>(plan.chosen === 'd' ? state.phase_order : []);
record();
state.gate = { ...(state.gate ?? {}), answered_at: answeredAt, reply, gate_rev: gateRev, plan: plan.chosen, overrides: tokens(overrides), ...(answeredBy ? { answered_by: answeredBy } : {}), ...provenance };
+29 -7
View File
@@ -196,20 +196,30 @@ export interface ReconciliationRow {
outside_id: string; native_id: string | null; disposition: ReconciliationDisposition;
/** Why this disposition: the matched location or title, or the parent's override reason. */
resolution: string;
basis: 'file_line' | 'title' | 'override' | 'none';
basis: 'file_line' | 'file' | 'title' | 'override' | 'none';
severity: { outside: Severity; native: Severity | null };
/** A parent override's one shared proposal for both voices' user challenge; the gate renders it as option b. */
proposal?: string;
}
export interface ReconciliationOverride { outside_id: string; native_id?: string | null; disposition: ReconciliationDisposition; resolution: string; proposal?: string }
export const tokens = (title: string) => new Set(title.toLowerCase().replace(/[^a-z0-9 ]+/g, ' ').split(/\s+/).filter(t => t.length > 2));
/** A DX cap code (`CAP4:<path>:<item>`) names a location, not the problem, so title comparison skips it. */
export const tokens = (title: string) => new Set(title.replace(/^\s*CAP[46]\s*:[^:]*:\s*/i, '').toLowerCase().replace(/[^a-z0-9 ]+/g, ' ').split(/\s+/).filter(t => t.length > 2));
export function jaccard(a: Set<string>, b: Set<string>): number {
if (!a.size || !b.size) return 0;
let inter = 0;
for (const t of a) if (b.has(t)) inter++;
return inter / (a.size + b.size - inter);
}
const capPath = (title: string) => /^\s*CAP[46]\s*:([^:]*):/i.exec(title)?.[1]!.trim();
/** Title similarity. Two CAP-coded titles on different paths name different problems, so they never match. */
export function titleScore(a: string, b: string): number {
const pa = capPath(a), pb = capPath(b);
if (pa !== undefined && pb !== undefined && pa !== pb) return 0;
return jaccard(tokens(a), tokens(b));
}
/** The voices name one problem: a title association or the parent's override. A shared location (file, or file:line with unrelated titles) associates two findings but never makes them one. */
export const sameProblem = (r: ReconciliationRow) => !!r.native_id && (r.basis === 'title' || r.basis === 'override' || (r.basis === 'file_line' && !r.resolution.startsWith('same location')));
const level = (s: Severity) => SEVERITIES.indexOf(s);
/** A remedy compared as words: case, punctuation and spacing ignored. */
export const remedyKey = (text: string | undefined) => (text ?? '').toLowerCase().replace(/[^a-z0-9]+/g, ' ').trim();
@@ -225,7 +235,9 @@ function contradictory(a: string, b: string): boolean {
/**
* Structural reconciliation. Findings associate by a title overlap of at least
* half the words, or by file:line (±3 lines). Association alone never
* half the words (a leading CAP code ignored; never across two different files),
* or by file:line (±3 lines), or, when either finding has no line, by the same
* file. Association alone never
* confirms: a title association is `confirmed` only when both voices state the
* same remedy, `disagree` when severities differ by two or more levels or the
* remedies contradict (one keeps, drops or negates what the other changes),
@@ -250,6 +262,8 @@ export function reconcile(native: FindingRow[], outside: FindingRow[], overrides
}
const used = new Set<string>();
const rows: ReconciliationRow[] = [];
const later: Array<[number, FindingRow]> = [];
const unmatched = (o: FindingRow): ReconciliationRow => ({ outside_id: o.id, native_id: null, disposition: 'new', resolution: 'not raised by the native voice', basis: 'none', severity: { outside: o.severity, native: null } });
for (const o of outside) {
const override = byOutside.get(o.id);
if (override) {
@@ -260,17 +274,17 @@ export function reconcile(native: FindingRow[], outside: FindingRow[], overrides
continue;
}
const near = (n: FindingRow) => !!o.file && o.line !== undefined && n.file === o.file && n.line !== undefined && Math.abs(n.line - o.line) <= 3;
const ot = tokens(o.title);
let match: FindingRow | undefined;
let best = 0;
for (const n of native) {
if (used.has(n.id)) continue;
const score = jaccard(ot, tokens(n.title)) + (near(n) ? 0.01 : 0);
if (used.has(n.id) || (o.file && n.file && o.file !== n.file)) continue;
const score = titleScore(o.title, n.title) + (near(n) ? 0.01 : 0);
if (score >= 0.5 && score > best) { best = score; match = n; }
}
const titled = !!match;
match ??= native.find(n => !used.has(n.id) && near(n));
if (!match) { rows.push({ outside_id: o.id, native_id: null, disposition: 'new', resolution: 'not raised by the native voice', basis: 'none', severity: { outside: o.severity, native: null } }); continue; }
if (!match && o.file) { later.push([rows.length, o]); rows.push(unmatched(o)); continue; }
if (!match) { rows.push(unmatched(o)); continue; }
used.add(match.id);
const basis: ReconciliationRow['basis'] = near(match) ? 'file_line' : 'title';
const where = near(match) ? `${o.file}:${o.line}` : 'same title';
@@ -286,6 +300,14 @@ export function reconcile(native: FindingRow[], outside: FindingRow[], overrides
if (remedyKey(match.resolution) !== remedyKey(o.resolution)) { pair('unverified', `same title, remedies differ (${where}): ${match.title}`); continue; }
pair('confirmed', `both voices, same remedy (${where}): ${match.title}`);
}
// Plan reviews name a file without a line: same file, a line missing on either side, once every title and line match is placed.
for (const [i, o] of later) {
const match = native.filter(n => !used.has(n.id) && n.file === o.file && (o.line === undefined || n.line === undefined))
.reduce<FindingRow | undefined>((b, n) => (!b || titleScore(o.title, n.title) > titleScore(o.title, b.title) ? n : b), undefined);
if (!match) continue;
used.add(match.id);
rows[i] = { outside_id: o.id, native_id: match.id, disposition: 'unverified', resolution: `same file, agreement unverified (${o.file}): ${match.title}`, basis: 'file', severity: { outside: o.severity, native: match.severity } };
}
return { rows, native_only: native.filter(n => !used.has(n.id)).map(n => n.id), errors };
}
+58 -10
View File
@@ -31,7 +31,7 @@ import type { RunStatus } from './headless-artifacts';
import { sha256File } from './headless-artifacts';
import { outsideConsent } from './outside-consent';
import { familyConflict, modelFamily } from './outside-voice-runner';
import { renderSpend, reserve, settle, summarize, readLedger } from './spend-ledger';
import { SpendCapExceeded, release, renderSpend, reserve, settle, summarize, readLedger } from './spend-ledger';
import { DEFAULT_MAX_ROUNDS, attemptEstimate, overCap, resumeStop, roundCap, roundOf, roundsLine, stopForBudget, stopLines } from './autoplan-budget';
import { annotateConfirm, confirmBlock } from './autoplan-confirm';
@@ -142,7 +142,11 @@ export function voiceState(state: RunState, phase: Phase, voice: Voice): VoiceSt
}
const openAttempt = (v: VoiceState) => v.attempts?.find(a => a.state === 'dispatched' || a.state === 'execution_unknown');
export interface NextOptions { out: string; cwd: string; estimateUsd?: number; redispatch?: string[] }
export interface NextOptions {
out: string; cwd: string; estimateUsd?: number; redispatch?: string[];
/** `run`: a voice's priced reservation from its prompt (undefined when unpriced); `--estimate-usd` wins over it. */
price?: (voice: Voice, promptFile: string) => number | undefined;
}
export interface NextResult { state: RunState; phase?: Phase; attempts: Array<{ attempt: string; voice: Voice; prompt: string; result: string; model_family: string; new: boolean }>; lines: string[] }
function openPhase(state: RunState, phase: Phase, cwd: string): void {
@@ -219,15 +223,20 @@ export function nextStep(o: NextOptions): NextResult {
if (ps.outside?.reason === 'docs_only') lines.push(`OUTSIDE: ${phase} not_run reason=docs_only (native voice only)`);
if (ps.sections) lines.push(`Skipped sections: ${ps.sections.skipped.length ? ps.sections.skipped.join(',') : 'none'} (scope: ui=${state.scope.ui ? 'yes' : 'no'},dx=${state.scope.dx ? 'yes' : 'no'},source=${scopeSourceLabel(state.scope)}; checklist=${ps.sections.checklist ? 'loaded' : 'none'})`);
const prompts = ps.prompts!;
if (o.estimateUsd !== undefined) state.estimate_usd = o.estimateUsd;
const estimate = attemptEstimate(state, o.estimateUsd, process.env);
if (estimate.line) lines.push(estimate.line);
const usdFor = (voice: Voice) => (state.estimate_usd === undefined ? o.price?.(voice, prompts[voice]) : undefined) ?? estimate.usd;
if (estimate.line && (['native', 'outside'] as const).some(voice => usdFor(voice) === estimate.usd)) lines.push(estimate.line);
if (first) lines.push(reviewEstimateLine(state, estimate.usd));
const fresh = (['native', 'outside'] as const).filter(voice => { const v = voiceState(state, phase!, voice); const open = openAttempt(v); return !voiceTerminal(v) && (!open || (open.state === 'execution_unknown' && o.redispatch?.includes(open.attempt))); });
const resultOf = (voice: Voice, attempt: string) => path.join(state.out, attemptFile(phase!, voice, attempt.slice(state.run.length + 1), 'md'));
fresh.forEach((voice, i) => bindPromptResult(prompts[voice], resultOf(voice, `${state.run}-a${state.attempt_counter + i + 1}`)));
const before = summarize(readLedger(path.join(state.out, FILES.spend), state.spend_cap_usd));
if (state.spend_cap_usd !== null && fresh.length && before.worst_case_usd + fresh.length * estimate.usd > state.spend_cap_usd + 1e-9) {
const stop = stopForBudget(state, phase, fresh.length * estimate.usd);
const asked = fresh.reduce((sum, voice) => sum + usdFor(voice), 0);
if (state.spend_cap_usd !== null && fresh.length && before.worst_case_usd + asked > state.spend_cap_usd + 1e-9) {
const stop = stopForBudget(state, phase, asked);
writeState(state);
throw new RunError('SPEND_CAP_EXCEEDED', `spent ${before.spent_usd} + reserved ${before.reserved_usd} + unknown ${before.unknown_usd} + ${fresh.length} x estimate ${estimate.usd} > cap ${state.spend_cap_usd}; the run stopped incomplete (budget_exhausted)`, 3, { status: 'incomplete', lines: [...lines, ...stop] });
throw new RunError('SPEND_CAP_EXCEEDED', `spent ${before.spent_usd} + reserved ${before.reserved_usd} + unknown ${before.unknown_usd} + ${fresh.length} attempt(s) reserving ${+asked.toFixed(6)} > cap ${state.spend_cap_usd}; the run stopped incomplete (budget_exhausted)`, 3, { status: 'incomplete', lines: [...lines, ...stop] });
}
for (const voice of ['native', 'outside'] as const) {
const v = voiceState(state, phase, voice);
@@ -243,11 +252,12 @@ export function nextStep(o: NextOptions): NextResult {
continue;
}
const attempt = `${state.run}-a${++state.attempt_counter}`;
try { reserve(path.join(state.out, FILES.spend), attempt, estimate.usd, { cap: state.spend_cap_usd, label: `${phase}/${voice}` }); }
const resultFile = resultOf(voice, attempt);
const usd = usdFor(voice);
try { reserve(path.join(state.out, FILES.spend), attempt, usd, { cap: state.spend_cap_usd, label: `${phase}/${voice}` }); }
catch (e: any) { if (e.code === 'SPEND_CAP_EXCEEDED') { state.attempt_counter--; writeState(state); throw new RunError('SPEND_CAP_EXCEEDED', e.message, 3); } throw e; }
const row: AttemptState = { attempt, state: 'dispatched', prompt: prompts[voice], result: path.join(state.out, attemptFile(phase, voice, attempt.slice(state.run.length + 1), 'md')), estimate_usd: estimate.usd, dispatched_at: now() };
bindPromptResult(row.prompt, row.result);
journal(state.out, { event: 'dispatched', attempt, phase, voice, snapshot: ps.snapshot, prompt: path.relative(state.out, row.prompt), prompt_sha256: sha256File(row.prompt), prompt_bytes: fs.statSync(row.prompt).size, result: path.relative(state.out, row.result), estimate_usd: estimate.usd });
const row: AttemptState = { attempt, state: 'dispatched', prompt: prompts[voice], result: resultFile, estimate_usd: usd, dispatched_at: now() };
journal(state.out, { event: 'dispatched', attempt, phase, voice, snapshot: ps.snapshot, prompt: path.relative(state.out, row.prompt), prompt_sha256: sha256File(row.prompt), prompt_bytes: fs.statSync(row.prompt).size, result: path.relative(state.out, row.result), estimate_usd: usd });
(v.attempts ??= []).push(row);
v.attempt = attempt; v.status = 'running';
result.attempts.push({ attempt, voice, prompt: row.prompt, result: row.result, model_family: familyHint(state, voice), new: true });
@@ -405,6 +415,44 @@ export function closeOutside(state: RunState, phase: Phase, status: 'unavailable
return lines;
}
/** Raise an open attempt's reservation to what the call about to run can cost; the cap still binds (SPEND_CAP_EXCEEDED). */
export function fitReservation(out: string, phase: Phase, voice: Voice, attempt: string, usd: number): void {
const state = readState(out);
const a = voiceState(state, phase, voice).attempts?.find(x => x.attempt === attempt);
if (!a || usd <= a.estimate_usd + 1e-9) return;
const file = path.join(out, FILES.spend);
release(file, attempt);
const label = `${phase}/${voice}`;
try { reserve(file, attempt, usd, { cap: state.spend_cap_usd, label }); }
catch (e) {
reserve(file, attempt, a.estimate_usd, { label });
if (e instanceof SpendCapExceeded) throw new RunError('SPEND_CAP_EXCEEDED', `${attempt} (${phase} ${voice}) was admitted at ${a.estimate_usd}; the call it is about to make can cost ${usd}, past the cap ${state.spend_cap_usd}`, 3);
throw e;
}
a.estimate_usd = usd;
writeState(state);
}
/** Record that `run`'s paid call for an open attempt is going out (true) or never left this machine (false). */
export function markSent(out: string, phase: Phase, voice: Voice, attempt: string, sent: boolean): void {
const state = readState(out);
const a = voiceState(state, phase, voice).attempts?.find(x => x.attempt === attempt);
if (!a) return;
if (sent) a.sent_at = now(); else delete a.sent_at;
writeState(state);
}
/** A paid call that did not bind a result (failed, refused, lost with its process): settle what it cost and supersede its open attempt, so a retry is admitted afresh. */
export function retireAttempt(out: string, phase: Phase, voice: Voice, attempt: string, usd: number | 'unknown'): void {
const state = readState(out);
const a = voiceState(state, phase, voice).attempts?.find(x => x.attempt === attempt);
if (!a || a.state !== 'dispatched') return;
a.state = 'superseded'; a.terminal_at = now();
settle(path.join(out, FILES.spend), attempt, usd);
journal(out, { event: 'superseded', attempt, phase, voice, usage_usd: usd });
writeState(state);
}
/** Mark the outside voice of the current phase unavailable/failed (the runner could not run); closes the phase if native is bound. */
export function markOutside(out: string, phase: Phase, status: 'unavailable' | 'failed', detail: string): RunState {
const state = readState(out);
+2 -2
View File
@@ -13,7 +13,7 @@ import { readJsonl } from './headless-artifacts';
import type { ResultCodeName } from './result-codes';
export type Voice = 'native' | 'outside';
export interface AttemptState { attempt: string; state: 'dispatched' | 'submitted' | 'execution_unknown' | 'cancelled' | 'superseded' | 'deferred'; prompt: string; result: string; estimate_usd: number; dispatched_at: string; terminal_at?: string }
export interface AttemptState { attempt: string; state: 'dispatched' | 'submitted' | 'execution_unknown' | 'cancelled' | 'superseded' | 'deferred'; prompt: string; result: string; estimate_usd: number; dispatched_at: string; terminal_at?: string; /** `run` set it before the paid call went out; an open attempt still carrying it was in flight when its process died. */ sent_at?: string }
export interface VoiceState extends VoiceOutcome {
attempt?: string; attempts?: AttemptState[]; result_sha256?: string; findings?: number; label?: string; usage?: Record<string, unknown>; detail?: string;
/** verified: the INPUT receipt matched the snapshot; attested: a --record bound without one, on the parent's word (attest_reason). */
@@ -58,7 +58,7 @@ export interface DispatchCounts { total: number; by_phase: Record<string, Partia
export interface RunState extends RunManifest {
revision: number; out: string; plan_source: string; active_plan: string; restore: string; scope: Scope; phase_order: Phase[];
phases: Record<string, PhaseState>; native_model?: string; outside_model?: string; outside_enabled: boolean; attempt_counter: number;
spend_cap_usd: number | null; deadline_at?: string; light?: boolean; history: Array<Record<string, unknown>>; created_at: string; updated_at: string;
spend_cap_usd: number | null; /** `next --estimate-usd`, kept for every later attempt. */ estimate_usd?: number; deadline_at?: string; light?: boolean; history: Array<Record<string, unknown>>; created_at: string; updated_at: string;
/** The gstack that last wrote this run (lib/autoplan-migrate.ts); absent on runs written before 1.91.96. */
gstack_version?: string;
/** The --plan file the owner edits (drift hints, revise), the --plan-heading it was cut from, and the scope flags revise re-detects with. */
+4 -4
View File
@@ -14,9 +14,9 @@ export const DX_ERROR_ITEMS = ['what', 'fix', 'recovery', 'code', 'context', 'tr
export const DX_ERROR_DOC = 'https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md';
export const DX_ERROR_COMPACT =
`Score every error path against the agent-ready error contract (${DX_ERROR_DOC}): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or the precise thing a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). ` +
`Score every error path against the agent-ready error contract (${DX_ERROR_DOC}): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or what a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). ` +
'Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. ' +
'Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure that names no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item>" (caps the Error Messages score at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item>" (caps it at 6/10).';
'Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure naming no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item> <what fails>" (caps Error Messages at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item> <what fails>" (caps it at 6/10).';
export const DX_ERROR_CONTRACT = `#### The agent-ready error contract (${DX_ERROR_RUBRIC})
@@ -53,7 +53,7 @@ unknown is reserved for existing behavior you could not inspect. In a live audit
you never triggered are unknown, not extrapolated.
**Caps.** Name the path and item behind every cap and show the corrected error line.
- **4/10 (Critical finding, title \`CAP4:<path>:<item>\`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as \`--yes\`).
- **6/10 (High finding, title \`CAP6:<path>:<item>\`):** a path lacks an applicable fix, recovery, context or trace item.
- **4/10 (Critical finding, title \`CAP4:<path>:<item> <what fails>\`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as \`--yes\`).
- **6/10 (High finding, title \`CAP6:<path>:<item> <what fails>\`):** a path lacks an applicable fix, recovery, context or trace item.
Every mode, DX TRIAGE included, reports every CAP4 finding.`;
+50 -21
View File
@@ -366,15 +366,24 @@ function renderReservationCheck(c: NonNullable<DryRunResult['checks']['reservati
// ---------------------------------------------------------------------------
export const PILOT_FILES = { spend: 'spend.json', pilot: 'pilot.json' } as const;
export interface PilotAttempt { attempt: string; model: string; row: number; estimate_usd: number }
export interface PilotRecord { schema_version: 1; n: number; arms: string[]; cap_usd: number; dry_run: string; attempts: PilotAttempt[]; started_at: string; status: 'reserved' | 'budget_exhausted' }
export interface PilotStartOptions { out: string; cap: number; n: number; arms: string[]; dryRun: string; tokens?: TokenEstimate; now?: Date }
export interface PilotRecord { schema_version: 1; n: number; arms: string[]; seat_metered?: string[]; cap_usd: number; dry_run: string; attempts: PilotAttempt[]; started_at: string; status: 'reserved' | 'budget_exhausted' }
export interface PilotStartOptions { out: string; cap: number; n: number; arms: string[]; seatMetered?: string[]; dryRun: string; tokens?: TokenEstimate; now?: Date }
/** Reserve n attempts per arm under the cap; a refused reservation ends the pilot `budget_exhausted` with nothing else reserved. */
/**
* Reserve n attempts per arm under the cap; a refused reservation ends the pilot `budget_exhausted` with nothing else reserved.
* A seat-metered arm (operator-declared: a subscription call with no incremental charge) reserves $0 and stays outside the projection.
*/
export function startPilot(o: PilotStartOptions): { record: PilotRecord; error?: { code: ResultCodeName; message: string } } {
const dry = JSON.parse(fs.readFileSync(o.dryRun, 'utf8')) as DryRunResult;
if (!dry.pass) return { record: null as unknown as PilotRecord, error: { code: 'EVAL_DRY_RUN_FAILED', message: `${o.dryRun} did not pass; the pilot never starts on a failed dry run` } };
const tokens = o.tokens ?? { input: dry.rows ? Math.ceil(dry.est_input_tokens / dry.rows) : DEFAULT_TOKENS.input, output: dry.checks.output_budget.budget };
const record: PilotRecord = { schema_version: 1, n: o.n, arms: o.arms, cap_usd: o.cap, dry_run: o.dryRun, attempts: [], started_at: (o.now ?? new Date()).toISOString(), status: 'reserved' };
const seat = o.seatMetered ?? [];
const stray = seat.filter(a => !o.arms.includes(a));
if (stray.length) throw new Error(`--seat-metered ${stray.join(',')} needs to name arms listed in --arms`);
const prior = path.join(o.out, PILOT_FILES.pilot);
const priorSeat = fs.existsSync(prior) ? (JSON.parse(fs.readFileSync(prior, 'utf8')) as PilotRecord).seat_metered ?? [] : seat;
if ([...priorSeat].sort().join(',') !== [...seat].sort().join(',')) throw new Error(`${o.out} holds a pilot started with --seat-metered ${priorSeat.join(',') || '(none)'}; seat-metered arms are fixed when a pilot starts, so this one needs a new --out`);
const record: PilotRecord = { schema_version: 1, n: o.n, arms: o.arms, ...(seat.length ? { seat_metered: seat } : {}), cap_usd: o.cap, dry_run: o.dryRun, attempts: [], started_at: (o.now ?? new Date()).toISOString(), status: 'reserved' };
const keyOf = (a: string) => `${armModel(a).replace(/[^A-Za-z0-9.-]/g, '_')}${armLoadout(a) !== null ? `.${armLoadout(a)!.replace(/[^A-Za-z0-9.-]/g, '_')}` : ''}`;
const keys = o.arms.map(keyOf);
const duplicates = [...new Set(o.arms.filter((_, i) => keys.indexOf(keys[i]!) !== i).map(a => o.arms.filter(b => keyOf(b) === keyOf(a)).join(' = ')))];
@@ -383,11 +392,12 @@ export function startPilot(o: PilotStartOptions): { record: PilotRecord; error?:
const ledgerFile = path.join(o.out, PILOT_FILES.spend);
let error: { code: ResultCodeName; message: string } | undefined;
outer: for (const model of o.arms) {
const seated = seat.includes(model);
if (!hasPricing(armModel(model))) { error = { code: 'EVAL_PRICE_MISSING', message: `${armModel(model)} has no row in lib/pricing.ts` }; break; }
for (let i = 1; i <= o.n; i++) {
const estimate = estimateCostUsd(tokens, armModel(model));
const estimate = seated ? 0 : estimateCostUsd(tokens, armModel(model));
const attempt = `pilot-${keyOf(model)}-${i}`;
try { reserve(ledgerFile, attempt, estimate, { cap: o.cap, label: `pilot ${model} #${i}`, now: o.now }); }
try { reserve(ledgerFile, attempt, estimate, { cap: o.cap, label: `pilot ${model} #${i}${seated ? ' (seat-metered)' : ''}`, now: o.now }); }
catch (e) {
if (e instanceof SpendCapExceeded) { record.status = 'budget_exhausted'; error = { code: 'EVAL_BUDGET_EXHAUSTED', message: e.message }; break outer; }
throw e;
@@ -408,31 +418,44 @@ export function releasePilot(out: string, attempt: string): SpendLedger {
/** Two-sided 95% Student t for n-1 degrees of freedom (n settled items); 2.0 past the table. */
const T95: Record<number, number> = { 2: 12.706, 3: 4.303, 4: 3.182, 5: 2.776, 6: 2.571, 7: 2.447, 8: 2.365, 9: 2.306, 10: 2.262 };
export interface Projection { n: number; per_item_usd: number | null; sd_usd: number | null; total: number; projected_usd: number | null; band_usd: [number, number] | null; verdict: 'within' | 'exceeds' | 'unknown' }
export interface ArmProjection { arm: string; n: number; per_item_usd: number | null; sd_usd: number | null; projected_usd: number | null; band_usd: [number, number] | null }
export interface Projection { n: number; per_item_usd: number | null; sd_usd: number | null; total: number; projected_usd: number | null; band_usd: [number, number] | null; verdict: 'within' | 'exceeds' | 'unknown'; arms: ArmProjection[] }
export interface SpendTable { summary: SpendSummary; workers: number; max_estimate_usd: number; worst_case_usd: number; worst_case_overrun_usd: number }
export interface PilotReport { record: PilotRecord; spend: SpendTable; projection: Projection; settled_known: number; settled_unknown: number }
export interface PilotReport { record: PilotRecord; spend: SpendTable; projection: Projection; settled_known: number; settled_unknown: number; seat_metered: Array<{ arm: string; calls: number; settled: number }> }
export function pilotReport(out: string, o: { total: number; workers?: number }): PilotReport {
const record = JSON.parse(fs.readFileSync(path.join(out, PILOT_FILES.pilot), 'utf8')) as PilotRecord;
const ledger = readLedger(path.join(out, PILOT_FILES.spend), record.cap_usd);
const summary = summarize(ledger);
const known = ledger.settled.filter(s => typeof s.usd === 'number').map(s => s.usd as number);
const unknown = ledger.settled.length - known.length;
const armOf = new Map(record.attempts.map(a => [a.attempt, a.model]));
const charged = new Set(ledger.settled.filter(s => typeof s.usd === 'number' && s.usd > 0).map(s => armOf.get(s.attempt)));
const seat = new Set((record.seat_metered ?? []).filter(arm => !charged.has(arm)));
const unknown = ledger.settled.filter(s => s.usd === 'unknown' && !seat.has(armOf.get(s.attempt) ?? '')).length;
const workers = Math.max(1, o.workers ?? 1);
const maxEstimate = Math.max(0, ...record.attempts.map(a => a.estimate_usd));
const worst = summary.worst_case_usd + workers * maxEstimate;
const n = known.length;
const mean = n ? known.reduce((s, x) => s + x, 0) / n : null;
const sd = n > 1 && mean !== null ? Math.sqrt(known.reduce((s, x) => s + (x - mean) ** 2, 0) / (n - 1)) : null;
const perArmTotal = o.total * Math.max(1, record.arms.length);
const projected = mean === null ? null : mean * perArmTotal;
const half = sd !== null && mean !== null ? (T95[n] ?? 2.0) * (sd / Math.sqrt(n)) * perArmTotal : null;
const band: [number, number] | null = projected !== null && half !== null ? [Math.max(0, projected - half), projected + half] : null;
const verdict: Projection['verdict'] = projected === null || band === null || unknown > 0 ? 'unknown' : band[1] + summary.worst_case_usd <= record.cap_usd ? 'within' : 'exceeds';
const arms: ArmProjection[] = record.arms.filter(arm => !seat.has(arm)).map(arm => {
const xs = ledger.settled.filter(s => armOf.get(s.attempt) === arm && typeof s.usd === 'number').map(s => s.usd as number);
const n = xs.length;
const mean = n ? xs.reduce((s, x) => s + x, 0) / n : null;
const sd = n > 1 && mean !== null ? Math.sqrt(xs.reduce((s, x) => s + (x - mean) ** 2, 0) / (n - 1)) : null;
const projected = mean === null ? null : mean * o.total;
const half = sd !== null ? (T95[n] ?? 2.0) * (sd / Math.sqrt(n)) * o.total : null;
return { arm, n, per_item_usd: mean, sd_usd: sd, projected_usd: projected, band_usd: projected !== null && half !== null ? [Math.max(0, projected - half), projected + half] : null };
});
const priced = arms.filter(a => a.per_item_usd !== null);
const n = priced.reduce((s, a) => s + a.n, 0);
const mean = priced.length || !arms.length ? priced.reduce((s, a) => s + a.per_item_usd!, 0) : null;
const sd = mean !== null && priced.every(a => a.sd_usd !== null) ? priced.reduce((s, a) => s + a.sd_usd!, 0) : null;
const projected = mean === null ? null : mean * o.total;
const band: [number, number] | null = projected !== null && priced.every(a => a.band_usd)
? [priced.reduce((s, a) => s + a.band_usd![0], 0), priced.reduce((s, a) => s + a.band_usd![1], 0)] : null;
const verdict: Projection['verdict'] = band === null || unknown > 0 || priced.length < arms.length ? 'unknown' : band[1] + summary.worst_case_usd <= record.cap_usd ? 'within' : 'exceeds';
return {
record, settled_known: n, settled_unknown: unknown,
spend: { summary, workers, max_estimate_usd: maxEstimate, worst_case_usd: worst, worst_case_overrun_usd: Math.max(0, worst - record.cap_usd) },
projection: { n, per_item_usd: mean, sd_usd: sd, total: o.total, projected_usd: projected, band_usd: band, verdict },
projection: { n, per_item_usd: mean, sd_usd: sd, total: o.total, projected_usd: projected, band_usd: band, verdict, arms },
seat_metered: [...seat].map(arm => ({ arm, calls: record.attempts.filter(a => a.model === arm).length, settled: ledger.settled.filter(s => armOf.get(s.attempt) === arm).length })),
};
}
@@ -449,11 +472,17 @@ export function renderSpendTable(t: SpendTable, cap: number): string {
export function renderPilotReport(r: PilotReport): string {
const p = r.projection;
const band = (b: [number, number] | null) => (b ? `[${usd(b[0])}, ${usd(b[1])}]` : 'unknown');
const notReady = r.settled_unknown > 0
? 'unknown charges make the band meaningless; settle them with actual cost, or start a new pilot in a new `--out` with `pilot start --seat-metered <arm>` for an arm on a subscription with no incremental charge'
: 'settle at least 2 attempts per priced arm with actual cost';
return [
`PILOT: n=${r.record.n} arms=${r.record.arms.join(',')} settled=${r.settled_known} unknown=${r.settled_unknown} status=${r.record.status}`,
renderSpendTable(r.spend, r.record.cap_usd),
`PILOT_PROJECTION: items=${p.total} per_item=${usd(p.per_item_usd)} sd=${usd(p.sd_usd)} projected=${usd(p.projected_usd)} band=${p.band_usd ? `[${usd(p.band_usd[0])}, ${usd(p.band_usd[1])}]` : 'unknown'} cap=$${r.record.cap_usd.toFixed(2)} verdict=${p.verdict}`,
p.verdict === 'unknown' ? `APPROVAL: not ready — ${r.settled_known < 2 ? 'settle at least 2 pilot attempts with actual cost' : 'unknown charges make the band meaningless'}; settle, then rerun report` : `APPROVAL: run ${p.total} items at projected ${usd(p.projected_usd)} (95% band ${usd(p.band_usd![0])}–${usd(p.band_usd![1])}) under cap $${r.record.cap_usd.toFixed(2)}: ${p.verdict === 'within' ? 'fits; ask the owner before any further priced call' : 'EXCEEDS the cap; stop, raise the cap on purpose or cut arms or items'}`,
...p.arms.map(a => `PILOT_ARM: arm=${a.arm} n=${a.n} per_item=${usd(a.per_item_usd)} sd=${usd(a.sd_usd)} projected=${usd(a.projected_usd)} band=${band(a.band_usd)}`),
...r.seat_metered.map(s => `PILOT_SEAT_METERED: arm=${s.arm} calls=${s.calls} settled=${s.settled} (declared seat-metered: no incremental charge, outside the cap and the projection; usage-limited)`),
`PILOT_PROJECTION: items=${p.total} known_arms=${p.arms.filter(a => a.per_item_usd !== null).length}/${p.arms.length} per_item=${usd(p.per_item_usd)} sd=${usd(p.sd_usd)} projected=${usd(p.projected_usd)} band=${band(p.band_usd)} cap=$${r.record.cap_usd.toFixed(2)} verdict=${p.verdict}`,
p.verdict === 'unknown' ? `APPROVAL: not ready — ${notReady}; then rerun report` : `APPROVAL: run ${p.total} items at projected ${usd(p.projected_usd)} (95% band ${usd(p.band_usd![0])}–${usd(p.band_usd![1])}) under cap $${r.record.cap_usd.toFixed(2)}: ${p.verdict === 'within' ? 'fits; ask the owner before any further priced call' : 'EXCEEDS the cap; stop, raise the cap on purpose or cut arms or items'}`,
].join('\n') + '\n';
}
+19 -4
View File
@@ -2,7 +2,9 @@
* gate-list — one gate list instead of four-option sequences (plan B9).
* A gate list is numbered items, each with a stable id, a recommended default
* and lettered options; `auto` items take their recommendation when
* unanswered, `approval` items stay pending until answered. One reply
* unanswered (one whose recommendation changes the plan keeps the plan's
* direction instead, unless the reply approves with the recommended changes,
* `p1b`), `approval` items stay pending until answered. One reply
* grammar: `rec` (every item takes its recommendation, plan changes
* included), `<id><option>` tokens (`d3b uc1a`), or `rec except <tokens>`
* (the same operation as listing the overrides). `all`, `approve all`,
@@ -39,6 +41,8 @@ export interface GateItem {
carried?: { from: string; chosen: string; gate_rev: number };
/** A round-cap item (lib/autoplan-budget.ts): `b` grants the phase one more round and reopens it. */
cap?: { phase: string; kind: 'deferred' | 'blocker' | 'delta'; ref?: string };
/** An open Critical or High finding (its ids, a paired twin included): `a` applies the reviewer's fix to the plan. */
finding?: string[];
}
export interface GateList { schema_version: 1; run: string; gate_rev: number; items: GateItem[]; title?: string;
/** Bumped by every `recommend`, so a `rec` relayed before it no longer matches `recHash`. */
@@ -196,6 +200,7 @@ export function parseReply(reply: string, list: GateList, gateRev?: number, prio
const choices: GateChoice[] = [];
const pending: string[] = [];
const how = (item: GateItem, chosen: string) => (chosen === item.recommended ? 'recommended' : 'override') as GateChoice['how'];
const withChanges = overrides.get('p1') === 'b';
for (const item of list.items) {
const chosen = overrides.get(item.id);
const earlier = prior[item.id] ?? item.carried?.chosen;
@@ -203,7 +208,7 @@ export function parseReply(reply: string, list: GateList, gateRev?: number, prio
else if (earlier !== undefined && item.options.some(o => o.key === earlier)) choices.push({ id: item.id, chosen: earlier, kind: item.kind, how: how(item, earlier), prior: true });
else if (asIs) { const keep = keepDirection(item); choices.push({ id: item.id, chosen: keep, kind: item.kind, how: how(item, keep) }); }
else if (rec) choices.push({ id: item.id, chosen: item.recommended, kind: item.kind, how: 'recommended' });
else if (item.kind === 'auto') choices.push({ id: item.id, chosen: item.recommended, kind: item.kind, how: 'auto_default' });
else if (item.kind === 'auto') choices.push({ id: item.id, chosen: withChanges ? item.recommended : autoDefault(item), kind: item.kind, how: 'auto_default' });
else pending.push(item.id);
}
return { ...base, valid: true, choices, pending, mode: rec ? 'rec' : asIs ? 'as-is' : 'tokens' };
@@ -214,6 +219,11 @@ export function keepDirection(item: GateItem): string {
return item.options.find(o => o.changes_plan === false)?.key ?? (item.id === 'p1' && item.options.some(o => o.key === 'a') ? 'a' : item.recommended);
}
/** What an unanswered `auto` item resolves to: its recommendation, unless that changes the plan (then the plan's direction). */
export function autoDefault(item: GateItem): string {
return item.options.find(o => o.key === item.recommended)?.changes_plan ? keepDirection(item) : item.recommended;
}
/** The decisions.jsonl id for a gate item. Item ids restart at every gate revision, so revisions after the first carry `r<rev>-`. */
export function decisionId(run: string, gateRev: number, itemId: string): string {
return gateRev > 1 ? `${run}-r${gateRev}-${itemId}` : `${run}-${itemId}`;
@@ -224,7 +234,7 @@ export function decisionRows(list: GateList, extra: Partial<DecisionRow> = {}):
return list.items.map(item => ({
schema_version: 1, run: list.run, id: decisionId(list.run, list.gate_rev, item.id), label: item.id.toUpperCase(), title: item.title,
options: item.options.map(o => o.key), recommended: item.recommended, kind: item.kind, status: 'pending',
default_if_unanswered: item.kind === 'auto' ? item.recommended : null, gate_rev: list.gate_rev,
default_if_unanswered: item.kind === 'auto' ? autoDefault(item) : null, gate_rev: list.gate_rev,
...(item.phase ? { phase: item.phase } : {}), ...(item.cost ? { cost: item.cost } : {}),
...(item.recommended_basis ? { recommended_basis: item.recommended_basis } : {}), ...(item.changes ? { changes: item.changes } : {}),
...(item.origin ? { origin: item.origin } : {}), ...(item.reopens === false ? { source: 'agent' } : {}), ...extra,
@@ -234,14 +244,19 @@ export function decisionRows(list: GateList, extra: Partial<DecisionRow> = {}):
/** `--by` values: the owner typing tokens, or an agent relaying or deciding them. */
export const ANSWERED_BY_RE = /^(human|agent:[A-Za-z0-9_.:-]+)$/;
/** Who answered and in whose words: a new answer replaces all of it, so an earlier answerer's fields never survive under a later one. */
const PROVENANCE = ['answered_by', 'owner_reply', 'owner_reply_parsed', 'answer_words', 'answer_words_parsed', 'words_by', 'carried_gate_from', 'carried_gate_rev', 'carry_reason', 'carried_answered_by'] as const;
export function applyReply(rows: DecisionRow[], parsed: ParsedReply, answeredAt: string, answeredBy?: string, extra: Partial<DecisionRow> = {}): DecisionRow[] {
if (!parsed.valid) return rows;
const byId = new Map(parsed.choices.map(c => [c.id, c]));
return rows.map(row => {
const choice = byId.get(row.label?.toLowerCase() ?? '') ?? byId.get(row.id.slice(row.run.length + 1).replace(/^r\d+-/, ''));
if (!choice || choice.prior || row.gate_rev !== parsed.gate_rev) return row;
const fresh = { ...row };
for (const k of PROVENANCE) delete (fresh as Record<string, unknown>)[k];
return {
...row, chosen: choice.chosen, answered_at: answeredAt, reply: parsed.reply, ...(answeredBy ? { answered_by: answeredBy } : {}), ...extra,
...fresh, chosen: choice.chosen, answered_at: answeredAt, reply: parsed.reply, ...(answeredBy ? { answered_by: answeredBy } : {}), ...extra,
status: choice.how === 'override' ? 'overridden' : 'approved',
};
});
+5
View File
@@ -48,6 +48,11 @@ export const GATE_OUTCOMES = {
summary: 'the reviewer declined to review',
fix: 're-run the review; if it declines again, review the change natively',
},
truncated: {
state: 'unavailable', anchor: 'outside-review-truncated',
summary: 'the reviewer hit its output token limit and the response is cut off',
fix: 're-run with a larger --max-tokens (or a smaller prompt); a cut-off response never counts as a review',
},
missing_markers: {
state: 'unavailable', anchor: 'outside-review-missing-markers',
summary: 'the response lacks the markers this gate requires',
+4 -2
View File
@@ -23,7 +23,7 @@ export type DecisionKind = 'auto' | 'approval' | 'user_challenge';
export type DecisionStatus = 'pending' | 'approved' | 'overridden' | 'rejected';
export type TaskStatus = 'pending' | 'ready' | 'blocked_on_dependency' | 'blocked_on_decision' | 'in_progress' | 'done';
export type VoiceStatus = 'completed' | 'unavailable' | 'disabled' | 'skipped' | 'failed' | 'pending' | 'running';
export type ResultStatus = 'complete' | 'gate_pending' | 'amend_pending' | 'incomplete' | 'interrupted' | 'refused';
export type ResultStatus = 'complete' | 'reviewed' | 'gate_pending' | 'amend_pending' | 'incomplete' | 'interrupted' | 'refused';
export type RunStatus = ResultStatus | 'awaiting_result' | 'running';
export interface FindingRow {
@@ -41,6 +41,8 @@ export interface DecisionRow {
recommended_basis?: string; changes?: string; origin?: string; carried_from?: string; chosen_text?: string;
/** An agent-relayed answer: the owner's verbatim words, and whether they parsed under the gate grammar. */
owner_reply?: string; owner_reply_parsed?: boolean; recommended_by?: string;
/** An agent's own words (`--words-by agent:<id>`): never the owner's, so never carried or imported as the owner's. */
answer_words?: string; answer_words_parsed?: boolean; words_by?: string;
/** A whole-gate carry (lib/autoplan-carry.ts): the earlier p1 decision whose owner words this answer reuses, its gate_rev, why, who relayed them, items taken under "do it", and the delta check attempt. */
carried_gate_from?: string; carried_gate_rev?: number; carry_reason?: string; carried_answered_by?: string; carried_under?: string; carry_check?: string;
}
@@ -119,7 +121,7 @@ export interface ShipReceipt {
base_red?: { cases: string[]; files: string[]; base: string; lane: string; expires_at: string };
}
export const RESULT_STATUSES: readonly ResultStatus[] = ['complete', 'gate_pending', 'amend_pending', 'incomplete', 'interrupted', 'refused'];
export const RESULT_STATUSES: readonly ResultStatus[] = ['complete', 'reviewed', 'gate_pending', 'amend_pending', 'incomplete', 'interrupted', 'refused'];
const TERMINAL_RUN: readonly string[] = ['complete', 'gate_pending', 'amend_pending', 'refused', 'incomplete'];
const TERMINAL_VOICE: readonly string[] = ['completed', 'unavailable', 'disabled', 'skipped', 'failed'];
const RESOLVED: readonly string[] = ['accepted', 'partially_accepted', 'rejected', 'deferred', 'fixed'];
+11 -5
View File
@@ -13,7 +13,13 @@ import { modelFamily, type ModelFamily } from './outside-voice-runner';
export interface ApiCallRequest { model: string; prompt: string; timeoutMs: number; env?: Record<string, string | undefined>; signal?: AbortSignal; maxTokens?: number }
export interface ApiUsage { input_tokens?: number; output_tokens?: number }
export interface ApiCallResult { text: string; model: string; usage: ApiUsage; provider: 'openai' | 'anthropic'; status: number }
/** `truncated`: the provider stopped at the output token limit (Anthropic `max_tokens`, OpenAI `length`), so the text is cut off. */
export interface ApiCallResult { text: string; model: string; usage: ApiUsage; provider: 'openai' | 'anthropic'; status: number; truncated: boolean }
/** Anthropic requires max_tokens; a full review needs more than 8192. `run` sends this to OpenAI too, where reasoning tokens count toward it. */
export const DEFAULT_MAX_TOKENS = 16_384;
/** The api runner's default time limit: five minutes, or about 40 ms per output token the call may write. */
export const apiTimeoutMs = (maxTokens?: number): number => Math.max(300_000, (maxTokens ?? DEFAULT_MAX_TOKENS) * 40);
export const API_FAMILIES: readonly ModelFamily[] = ['openai', 'anthropic'];
@@ -57,17 +63,17 @@ export async function callModel(req: ApiCallRequest): Promise<ApiCallResult> {
const model = req.model.includes('/') ? req.model.slice(req.model.indexOf('/') + 1) : req.model;
if (auth.provider === 'anthropic') {
const base = (env.ANTHROPIC_BASE_URL ?? 'https://api.anthropic.com').replace(/\/$/, '');
const body = JSON.stringify({ model, max_tokens: req.maxTokens ?? 8192, messages: [{ role: 'user', content: req.prompt }] });
const body = JSON.stringify({ model, max_tokens: req.maxTokens ?? DEFAULT_MAX_TOKENS, messages: [{ role: 'user', content: req.prompt }] });
const { status, json } = await post(`${base}/v1/messages`, { 'x-api-key': auth.key, 'anthropic-version': '2023-06-01' }, body, req, 'anthropic');
if (status < 200 || status >= 300) throw new Error(`anthropic ${status}: ${json?.error?.message ?? json?.raw ?? JSON.stringify(json)}`);
const text = Array.isArray(json?.content) ? json.content.filter((c: any) => c?.type === 'text').map((c: any) => c.text).join('\n') : '';
return { text, model: typeof json?.model === 'string' ? json.model : model, usage: { input_tokens: json?.usage?.input_tokens, output_tokens: json?.usage?.output_tokens }, provider: 'anthropic', status };
return { text, model: typeof json?.model === 'string' ? json.model : model, usage: { input_tokens: json?.usage?.input_tokens, output_tokens: json?.usage?.output_tokens }, provider: 'anthropic', status, truncated: json?.stop_reason === 'max_tokens' };
}
const base = (env.OPENAI_BASE_URL ?? 'https://api.openai.com/v1').replace(/\/$/, '');
const body = JSON.stringify({ model, messages: [{ role: 'user', content: req.prompt }] });
const body = JSON.stringify({ model, messages: [{ role: 'user', content: req.prompt }], ...(req.maxTokens ? { max_completion_tokens: req.maxTokens } : {}) });
const { status, json } = await post(`${base}/chat/completions`, { authorization: `Bearer ${auth.key}` }, body, req, 'openai');
if (status < 200 || status >= 300) throw new Error(`openai ${status}: ${json?.error?.message ?? json?.raw ?? JSON.stringify(json)}`);
const content = json?.choices?.[0]?.message?.content;
const text = typeof content === 'string' ? content : Array.isArray(content) ? content.map((c: any) => c?.text ?? '').join('\n') : '';
return { text, model: typeof json?.model === 'string' ? json.model : model, usage: { input_tokens: json?.usage?.prompt_tokens, output_tokens: json?.usage?.completion_tokens }, provider: 'openai', status };
return { text, model: typeof json?.model === 'string' ? json.model : model, usage: { input_tokens: json?.usage?.prompt_tokens, output_tokens: json?.usage?.completion_tokens }, provider: 'openai', status, truncated: json?.choices?.[0]?.finish_reason === 'length' };
}
+4 -1
View File
@@ -36,6 +36,8 @@ export interface RunnerRequest {
promptFile: string; outFile: string; model?: string; timeoutMs: number; cwd: string; gate: OutsideGate;
/** host-subagent: the file the host's subagent wrote. */
resultFile?: string;
/** api: the output token limit (lib/outside-voice-api.ts DEFAULT_MAX_TOKENS for Anthropic when unset). */
maxTokens?: number;
env?: Record<string, string | undefined>;
/** Directory of the gstack bins (gstack-codex-probe); defaults to this module's sibling bin/. */
binDir?: string;
@@ -199,10 +201,11 @@ class ApiRunner implements OutsideRunner {
if (!avail.ok) return finish({ ...base, status: 'unavailable', exit: 1, reason: 'runner_unavailable', detail: avail.reason }, started);
const { callModel } = await import('./outside-voice-api');
try {
const res = await callModel({ model: req.model!, prompt, timeoutMs: req.timeoutMs, env: req.env, signal: this.controller.signal });
const res = await callModel({ model: req.model!, prompt, timeoutMs: req.timeoutMs, env: req.env, signal: this.controller.signal, maxTokens: req.maxTokens });
fs.mkdirSync(path.dirname(req.outFile), { recursive: true });
fs.writeFileSync(req.outFile, res.text);
const usage: RunnerUsage = { ...res.usage, usd: 'unknown' };
if (res.truncated) return finish({ ...base, model: res.model, family: modelFamily(res.model), status: 'unavailable', exit: 0, usage, reason: 'truncated', detail: `stopped at the output limit after ${res.usage.output_tokens ?? 'an unknown number of'} output tokens; raise --max-tokens` }, started, req.outFile);
const { status, verdict } = classify(res.text, req.gate, { exit: 0 });
return finish({ ...base, model: res.model, family: modelFamily(res.model), status, exit: 0, usage, verdict, ...(verdict.reason ? { reason: verdict.reason } : {}), ...(verdict.detail ? { detail: verdict.detail } : {}) }, started, req.outFile);
} catch (e: any) {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "gstack",
"version": "1.92.49",
"version": "1.92.50",
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
"license": "MIT",
"type": "module",
+2 -2
View File
@@ -76,8 +76,8 @@ unknown is reserved for existing behavior you could not inspect. In a live audit
you never triggered are unknown, not extrapolated.
**Caps.** Name the path and item behind every cap and show the corrected error line.
- **4/10 (Critical finding, title `CAP4:<path>:<item>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
- **6/10 (High finding, title `CAP6:<path>:<item>`):** a path lacks an applicable fix, recovery, context or trace item.
- **4/10 (Critical finding, title `CAP4:<path>:<item> <what fails>`):** a path leaks a secret, signals failure as success, gives an unsafe fix, or blocks waiting for a terminal (hangs, or fails for lack of a terminal without naming a non-interactive alternative such as `--yes`).
- **6/10 (High finding, title `CAP6:<path>:<item> <what fails>`):** a path lacks an applicable fix, recovery, context or trace item.
Every mode, DX TRIAGE included, reports every CAP4 finding.
@@ -162,7 +162,7 @@ persona from 0A.
Load reference: Read the "## Pass 3" section from `~/.claude/skills/gstack/plan-devex-review/dx-hall-of-fame.md`
before scoring: the full contract, calibration examples and the standards mapping.
Score every error path against the agent-ready error contract (https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or the precise thing a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure that names no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item>" (caps the Error Messages score at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item>" (caps it at 6/10).
Score every error path against the agent-ready error contract (https://github.com/garrytan/gstack/blob/main/docs/agent-ready-errors.md): what (operation and actual values), fix (a safe, authorized command or edit on the first or last line, or what a human must provide), recovery (retry, fix and retry, do X first, or ask a human; side effects none/partial/complete/unknown, reconcile before retrying an unknown outcome, a retry bound), code (stable, with a doc anchor), context (versions, config, env var names, the underlying cause; caller-safe only for remote services), trace (unexpected failures only: a compact trace or its file path locally, a correlation id remotely), redaction (no secret values anywhere), channel (the surface's native failure signal: non-zero exit and stderr for a CLI, status plus RFC 9457 body for an API, a typed error for an SDK, isError for an MCP tool). Usage and validation errors rejected before any work need only what, fix, code, redaction and channel; items apply per surface, N/A needs a reason, an item the plan never specifies for a path it introduces is a gap, and unknown is reserved for existing behavior you could not inspect. Report a leaked secret, a failure signalled as success, an unsafe fix, or a block on a terminal (a hang, or a no-terminal failure naming no non-interactive alternative such as --yes) as a Critical finding titled "CAP4:<path>:<item> <what fails>" (caps Error Messages at 4/10); report a missing applicable fix, recovery, context or trace as a High finding titled "CAP6:<path>:<item> <what fails>" (caps it at 6/10).
The "## Pass 3" Hall of Fame section loaded above holds the full contract: the item table,
the per-surface channel rules and the evidence states.
+6 -6
View File
@@ -17,17 +17,17 @@ import type { ResolverFn } from './types';
/** One entry per command line the loop runs, as the field-wave plan specifies them. */
export const AUTOPLAN_DRIVER_COMMANDS: Array<{ cmd: string; does: string }> = [
{ cmd: 'gstack-autoplan next --out <dir> --plan <plan.md> [scope flags] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--scaffold]', does: `first call: init the run, print PHASE:, SCOPE:, DRIVER: and one ATTEMPT: per voice; --max-rounds caps review rounds per phase (default ${DEFAULT_MAX_ROUNDS} unattended), and an unattended Capy run reserves ${DEFAULT_HOST_ESTIMATE_USD.toFixed(2)} per host attempt unless --estimate-usd <n> says otherwise, so --spend-cap binds` },
{ cmd: 'gstack-autoplan next --out <dir> --plan <plan.md> [scope flags] [--light] [--spend-cap <usd>] [--max-rounds <n>] [--estimate-usd <n>] [--scaffold]', does: `first call: init the run, print PHASE:, SCOPE:, DRIVER: and one ATTEMPT: per voice; --max-rounds caps review rounds per phase (default ${DEFAULT_MAX_ROUNDS} unattended); a run with --spend-cap, in any session kind (or any unattended Capy run), reserves ${DEFAULT_HOST_ESTIMATE_USD.toFixed(2)} per voice for each attempt, so the cap binds; --estimate-usd <n> sets that reservation and the run keeps it for later phases` },
{ cmd: 'gstack-autoplan run --out <dir> --plan <plan.md> --native-model <id> --outside-model <id> --native-runner api --runner api [--max-tokens <n>] [--spend-cap <usd>]', does: 'hosts without subagents: one command calls both models directly (ANTHROPIC_API_KEY, OPENAI_API_KEY) and loops next/submit to the gate; a response cut off at the output limit is reported truncated, so raise --max-tokens (default 16384 for both providers; OpenAI counts reasoning tokens toward it); each api call reserves its priced worst case (a model with no price reserves the per-voice default under a cap), a native call that went out but bound no result settles at its usage, or its full reservation when the usage is unknown, and a failed outside call stays counted at its reservation' },
{ cmd: 'gstack-autoplan next --out <dir> [--redispatch <attempt>]', does: 'later calls: open the next phase or reprint open attempts; never redispatches on its own' },
{ cmd: 'gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --record <file.md> --model <id> [--attempt <id>]', does: 'bind a free-form review record (canonical fence, one findings table or list, or `No findings.`)' },
{ cmd: 'gstack-autoplan submit ... --record <file.md> --attest "<reason>" --attempt <id>', does: 'bind a record with no INPUT: line as receipt: attested (a wrong INPUT: always refuses)' },
{ cmd: 'gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> --model <id> --attempt <id>', does: 'bind a canonical result (the RESULT FORMAT below)' },
{ cmd: 'gstack-autoplan submit --out <dir> --phase <p> --voice native|outside --result <file> --model <id> --attempt <id> [--usage-usd <n>] [--reconcile <jsonl>]', does: 'bind a canonical result (the RESULT FORMAT below); `--usage-usd` settles the attempt at what the reviewer actually cost (else it stays priced unknown at its reservation); on the submit that closes the phase, `--reconcile` takes your own pairing of findings both voices raised, one `{"outside_id","native_id","disposition","resolution"}` row each, over the structural one' },
{ cmd: 'gstack-autoplan submit --out <dir> --phase <p> --voice outside --unavailable|--failed "<reason>"', does: 'close the phase native-only when no outside voice can run' },
{ cmd: 'gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by human|agent:<id> [--owner-reply <text>|-] [--rec-hash <h>]', does: 'answer the gate with the reply grammar below; an agent answer stores the owner\'s words verbatim' },
{ cmd: 'gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by human|agent:<id> [--owner-reply <text>|-] [--words-by agent:<id>] [--rec-hash <h>]', does: 'answer the gate with the reply grammar below; an agent answer stores the owner\'s words verbatim, and passes --words-by agent:<id> whenever the words are its own (the default is owner)' },
{ cmd: 'gstack-autoplan recommend --out <dir> --gate-rev <n> --item <id> --option <k> --basis "<why>"', does: 'change one recommendation before relay; then render and relay again (a `rec` needs the new `--rec-hash`)' },
{ cmd: 'gstack-autoplan revise --out <dir> --plan <plan v2> --incorporates <decision ids>', does: 'bind the amended plan; reopens every phase whose reviewed plan changed, Eng last' },
{ cmd: 'gstack-autoplan revise --out <dir> --settle <decision id|d4>', does: 'discharge a packaging or ordering change with no plan edit; reopens nothing' },
{ cmd: 'gstack-autoplan revise --out <dir> --withdraw <decision id>', does: 'drop a pending amendment instead of applying it' },
{ cmd: 'gstack-autoplan revise --out <dir> --settle <decision id|d4> | --withdraw <decision id>', does: '--settle discharges a packaging or ordering change with no plan edit and reopens nothing; --withdraw drops a pending amendment instead of applying it' },
{ cmd: 'gstack-autoplan close --out <dir> --phase <p> --deferred "<reason>" --by human|agent:<id>', does: 'close a phase accepted-with-findings: its open findings go to tasks.jsonl, and Critical, urgent or user-challenge findings become gate items; also the recovery from `STOP: reason=budget_exhausted`' },
{ cmd: 'gstack-autoplan budget --out <dir> [--spend-cap <usd>] [--max-rounds <n>] --by human|agent:<id>', does: 'the owner\'s journaled cap raise; after a budget stop, `resume` continues the run' },
{ cmd: 'gstack-autoplan export --out <dir>', does: 'write the artifact set; then `gstack-artifact validate <dir>/run.json`' },
@@ -40,7 +40,7 @@ export const AUTOPLAN_DRIVER_COMMANDS: Array<{ cmd: string; does: string }> = [
export const GATE_REPLY_FORMS: Array<{ reply: string; means: string }> = [
{ reply: 'rec', means: 'every item takes its recommendation, plan changes included (each prints APPLY:); `all` and `do it` are aliases' },
{ reply: 'as-is', means: 'keep every direction; no plan change' },
{ reply: 'd3b uc1a', means: 'one <id><option> token per item you set; unanswered auto items take their default' },
{ reply: 'd3b uc1a', means: 'one <id><option> token per item you set; unanswered auto items take their default (a finding item keeps the plan as written; `p1b` applies it)' },
{ reply: 'rec except d3b', means: 'recommendations, with the listed overrides' },
];
+22 -4
View File
@@ -16,7 +16,7 @@ import { EXIT } from '../lib/headless-artifacts';
import { extractReceipt } from '../lib/ship-receipt';
import { evaluateApprovals, parseCitations } from '../lib/approval-citations';
import { scopeOf, type ApprovalHit } from '../lib/approval-classes';
import { autoplan, runPhase } from './helpers/autoplan-run-fixture';
import { autoplan, recCarrying, runPhase } from './helpers/autoplan-run-fixture';
import { BASE_POLICY, cleanup, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/restamp-fixture';
const repos: FixtureRepo[] = [];
@@ -228,7 +228,7 @@ describe('gstack-ship-receipt write enforces declared classes (APPROVAL_CITATION
expect(autoplan(['next', '--out', out, '--plan', path.join(dir, 'plan.md'), '--no-ui']).status).toBe(0);
runPhase(out, 'ceo');
runPhase(out, 'eng');
if (approve) expect(autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', 'all']).stdout).toMatch(/^APPROVED: gate_rev=1/m);
if (approve) expect(autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', recCarrying(out)]).stdout).toMatch(/^APPROVED: gate_rev=1/m);
const sha = JSON.parse(fs.readFileSync(path.join(out, 'run.json'), 'utf8')).artifacts['plan.md'].sha256 as string;
return { out, sha: sha.slice(0, 8) };
};
@@ -237,17 +237,35 @@ describe('gstack-ship-receipt write enforces declared classes (APPROVAL_CITATION
const other = run('docs-plan', 'Rewrite the quickstart in docs/quickstart.md with a shorter first step. Tests cover the links.', true);
const ok = write(repo, `Approved by Garry 2026-10-16 via ${good.sha}\n`, ['--approval-run', other.out, '--approval-run', good.out]);
expect(ok.status, ok.err).toBe(EXIT.ok);
expect(JSON.parse(ok.out).approval.classes[0]).toMatchObject({ class: 'data-repair', citation: 'verified', binding: 'paths', via: good.sha, evidence: expect.stringMatching(/^plan run-\d+ gate_rev=1 p1=a answered_by=human$/) });
expect(JSON.parse(ok.out).approval.classes[0]).toMatchObject({ class: 'data-repair', citation: 'verified', binding: 'paths', via: good.sha, evidence: expect.stringMatching(/^plan run-\d+ gate_rev=1 p1=b answered_by=human$/) });
const notApproved = write(repo, `Approved by Garry 2026-10-16 via ${pending.sha}\n`, ['--approval-run', pending.out]);
expect(notApproved.err).toMatch(/holds this plan but its gate is not APPROVED \(status gate_pending, p1 unanswered\).*\(APPROVAL_CITATION_MISSING\)$/m);
const unrelated = write(repo, `Approved by Garry 2026-10-16 via ${other.sha}\n`, ['--approval-run', other.out]);
expect(unrelated.err).toMatch(/plan run-\d+ gate_rev=1 p1=a answered_by=human names neither/);
expect(unrelated.err).toMatch(/plan run-\d+ gate_rev=1 p1=b answered_by=human names neither/);
const noRun = write(repo, `Approved by Garry 2026-10-16 via ${good.sha}\n`);
expect(noRun.err).toMatch(/plan sha [0-9a-f]{8}: pass the autoplan run with --approval-run <dir>.*must attest "for data-repair@[0-9a-f]{12}"/);
fs.appendFileSync(path.join(good.out, 'plan.md'), '\nsrc/commands/repair/later.ts\n');
expect(write(repo, `Approved by Garry 2026-10-16 via ${good.sha}\n`, ['--approval-run', good.out]).err).toMatch(/plan\.md does not match its manifest hash/);
}, 120_000);
test('a plan gate an agent answered in its own words is not the owner\'s approval: a plan-sha citation of it is rejected', () => {
const repo = classRepo();
commit(repo, REPAIR);
const dir = path.join(repo.root, 'self-approved');
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, 'plan.md'), '# self-approved\n\n## Implementation plan\n\nAdd a data-repair command in src/commands/repair/accepted-takes.ts: explicit, preview-first, never at upgrade. Tests cover preview and apply.\n\n## Review record\n');
const out = path.join(dir, 'run');
expect(autoplan(['next', '--out', out, '--plan', path.join(dir, 'plan.md'), '--no-ui']).status).toBe(0);
runPhase(out, 'ceo');
runPhase(out, 'eng');
const reply = recCarrying(out);
expect(autoplan(['answer', '--by', 'agent:test', '--words-by', 'agent:test', '--owner-reply', reply, '--out', out, '--gate-rev', '1', '--reply', reply]).stdout).toMatch(/^APPROVED: gate_rev=1/m);
const sha = (JSON.parse(fs.readFileSync(path.join(out, 'run.json'), 'utf8')).artifacts['plan.md'].sha256 as string).slice(0, 8);
const cited = write(repo, `Approved by Garry 2026-10-16 via ${sha}\n`, ['--approval-run', out]);
expect(cited.status).toBe(EXIT.fail);
expect(cited.err).toMatch(/gate was answered in agent:test's own words, not the owner's.*\(APPROVAL_CITATION_MISSING\)$/m);
}, 120_000);
test('an unresolvable reference counts only as an explicit for <class>@<digest> attestation, recorded unverified, and goes stale when the effects change', () => {
const repo = classRepo();
commit(repo, REPAIR);
+10 -2
View File
@@ -119,13 +119,13 @@ describe('the rendered table is display only', () => {
});
});
function autoplanRun(dir: string, answers: Array<{ by: string; words?: string; at: string; p1: string }>) {
function autoplanRun(dir: string, answers: Array<{ by: string; words?: string; at: string; p1: string; own?: boolean }>) {
const rows = [];
for (const a of answers) {
for (const id of ['d1', 'p1']) rows.push({
schema_version: 1, run: 'run-1', id: `run-1-${id}`, label: id.toUpperCase(), title: id === 'p1' ? 'Plan approval' : 'Split the CLI [run-1-ceo-native-2]',
options: ['a', 'b'], recommended: 'a', kind: id === 'p1' ? 'approval' : 'auto', status: 'approved', gate_rev: 1,
chosen: id === 'p1' ? a.p1 : 'a', answered_at: a.at, reply: a.words ?? 'rec', answered_by: a.by, ...(a.by === 'human' ? {} : { owner_reply: a.words }),
chosen: id === 'p1' ? a.p1 : 'a', answered_at: a.at, reply: a.words ?? 'rec', answered_by: a.by, ...(a.by === 'human' ? {} : a.own ? { answer_words: a.words, words_by: a.by } : { owner_reply: a.words }),
});
}
fs.writeFileSync(path.join(dir, 'decisions.jsonl'), rows.map(r => JSON.stringify(r)).join('\n') + '\n');
@@ -147,6 +147,14 @@ describe('import --autoplan', () => {
expect(bin(['import', '--autoplan', run, '--by', 'Garry'], { cwd, env: { GSTACK_STATE_ROOT: state } }).out).toMatch(/^APPROVAL_IMPORTED: n=0 skipped=2 /);
});
test("each row names its own answerer: --by names only the owner's words, an agent's own words import under the agent (GEVA-2)", () => {
const run = tmp('appr-run-');
autoplanRun(run, [{ by: 'agent:GSTA-36', words: 'Do it', at: '2026-10-20T21:35:42.697Z', p1: 'a' }, { by: 'agent:GSTA-36', words: 'Accepted on the standing approval', at: '2026-10-21T08:00:00.000Z', p1: 'a', own: true }]);
const rows = importAutoplan(run, 'b', 'Garry');
if ('code' in rows) throw new Error(rows.message);
expect(rows.map(r => [r.by, r.words])).toEqual([['Garry, relayed by agent:GSTA-36', 'Do it'], ["agent:GSTA-36, its own decision (not the owner's words)", 'Accepted on the standing approval']]);
});
test('a default run id whose YYYYMMDDHHMMSS stamp passes Luhn still imports (capy-captain-flow flake)', () => {
const run = tmp('appr-run-');
autoplanRun(run, [{ by: 'agent:GSTA-1', words: 'rec', at: '2026-10-24T13:15:04.000Z', p1: 'a' }]);
+18 -16
View File
@@ -13,12 +13,12 @@ import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { attemptsOf, autoplan, journal, readRun, resultOf, review, ROOT, runPhase, SMALL_PLAN, startSmallRun } from './helpers/autoplan-run-fixture';
import { attemptsOf, autoplan, journal, readRun, recCarrying, resultOf, review, ROOT, runPhase, SMALL_PLAN, startSmallRun } from './helpers/autoplan-run-fixture';
const TMP = fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-answer-'));
afterAll(() => fs.rmSync(TMP, { recursive: true, force: true }));
/** A gated run with a disagreement in ceo (taste item d1) and the fixture's user challenges (uc1 ceo, uc2 eng). */
/** A gated run with a disagreement in ceo (taste item d1), the fixture's user challenges (uc1 ceo, uc2 eng) and its open Critical/High findings (d2 ceo, d3 and d4 eng). */
function gatedRun(name: string, extra: string[] = []): string {
const out = startSmallRun(path.join(TMP, name), extra);
const disagree = JSON.stringify([
@@ -60,11 +60,12 @@ function incorporate(out: string, ids: string[], edit: string) {
}
describe('answer', () => {
test('the gate lists the disagreement as an auto item and the user challenges as approval items', () => {
test('the gate lists the disagreement and the open Critical/High findings as auto items and the user challenges as approval items', () => {
const out = gatedRun('shape');
const gate = JSON.parse(fs.readFileSync(path.join(out, 'gate.json'), 'utf8'));
expect(gate.gate_rev).toBe(1);
expect(gate.items.map((i: any) => [i.id, i.kind, i.phase ?? null])).toEqual([['d1', 'auto', 'ceo'], ['uc1', 'approval', 'ceo'], ['uc2', 'approval', 'eng'], ['p1', 'approval', null]]);
expect(gate.items.map((i: any) => [i.id, i.kind, i.phase ?? null])).toEqual([['d1', 'auto', 'ceo'], ['uc1', 'approval', 'ceo'], ['d2', 'auto', 'ceo'], ['uc2', 'approval', 'eng'], ['d3', 'auto', 'eng'], ['d4', 'auto', 'eng'], ['p1', 'approval', null]]);
expect(gate.items.filter((i: any) => i.finding).map((i: any) => i.id)).toEqual(['d2', 'd3', 'd4']);
expect(gate.items[0].why).toMatch(/voices disagree: severity: native High, outside Low/);
}, 30_000);
@@ -86,26 +87,27 @@ describe('answer', () => {
expect(fs.readFileSync(path.join(out, 'run.json'), 'utf8')).toBe(before.run);
expect(fs.readFileSync(path.join(out, 'decisions.jsonl'), 'utf8')).toBe(before.decisions);
expect(journal(out).length).toBe(before.journal);
// partial: only the auto item answered; approval items stay pending; nothing reviewed changes
// partial: only the auto item answered; approval items, and finding items whose default waits on p1 (p1b applies them), stay pending
const partial = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', 'd1a', '--json']);
expect(partial.status, partial.stderr).toBe(0);
expect(JSON.parse(partial.stdout)).toMatchObject({ status: 'gate_pending', outcome: 'gate_pending', pending: ['uc1', 'uc2', 'p1'] });
const rows = decisions(out);
expect(rows.find((r: any) => r.label === 'D1')).toMatchObject({ status: 'approved', chosen: 'a' });
expect(rows.filter((r: any) => r.status === 'pending').map((r: any) => r.label)).toEqual(['UC1', 'UC2', 'P1']);
expect(rows.filter((r: any) => r.status === 'pending').map((r: any) => r.label)).toEqual(['UC1', 'D2', 'UC2', 'D3', 'D4', 'P1']);
}, 30_000);
test('`all` finalizes without another paid phase: status complete, decisions approved, no phase reopened', () => {
test('`all` carrying the open findings finalizes without another paid phase: status complete, decisions answered, no phase reopened', () => {
const out = gatedRun('approve');
const counter = readRun(out).attempt_counter;
const r = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', 'all']);
const reply = recCarrying(out);
const r = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', reply]);
expect(r.status, r.stderr).toBe(0);
expect(r.stdout).toMatch(/^APPROVED: gate_rev=1 as-is$/m);
expect(r.stdout).toContain('GSTACK_RESULT: skill=autoplan status=complete run=' + out);
const run = readRun(out);
expect(run.status).toBe('complete');
expect(run.attempt_counter).toBe(counter);
expect(decisions(out).every((d: any) => d.status === 'approved' && d.reply === 'all')).toBe(true);
expect(decisions(out).every((d: any) => d.status === (/^D[234]$/.test(d.label) ? 'overridden' : 'approved') && d.reply === reply)).toBe(true);
expect(journal(out).filter(j => j.event === 'reopened')).toEqual([]);
const again = autoplan(['next', '--out', out]);
expect(again.stdout).toContain('status=complete');
@@ -114,7 +116,7 @@ describe('answer', () => {
test('a non-recommended choice on a ceo item is recorded without reopening; revise reopens the changed phases, archives their bound files, and the next gate is gate_rev 2', () => {
const out = gatedRun('reopen', ['--no-carry-forward']);
const r = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', 'all except d1b', '--json']);
const r = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', recCarrying(out, 'all', ['d1b']), '--json']);
expect(r.status, r.stderr).toBe(0);
const runId = readRun(out).run;
expect(JSON.parse(r.stdout)).toMatchObject({ status: 'amend_pending', outcome: 'amend_pending', reopened: [], amendments: [`${runId}-d1`] });
@@ -145,12 +147,12 @@ describe('answer', () => {
expect(eng.outside.stdout).toContain('status=gate_pending');
expect(readRun(out).gate_rev).toBe(2);
const rows = decisions(out);
expect(rows.filter((x: any) => x.gate_rev === 1)).toHaveLength(4);
expect(rows.filter((x: any) => x.gate_rev === 1)).toHaveLength(7);
expect(rows.filter((x: any) => x.gate_rev === 2 && x.status === 'pending').length).toBeGreaterThan(0);
const staleAgain = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', 'all']);
expect(staleAgain.status).toBe(3);
// closing gate_rev 2 exports a run that validates: rev-2 rows get their own ids beside rev 1's
const approve = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '2', '--reply', 'all']);
const approve = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '2', '--reply', recCarrying(out)]);
expect(approve.stdout).toContain('GSTACK_RESULT: skill=autoplan status=complete run=' + out);
expect(autoplan(['export', '--out', out]).status).toBe(0);
const ids = decisions(out).map((x: any) => x.id);
@@ -167,7 +169,7 @@ describe('answer', () => {
test('an eng-only override reopens nothing at answer; revise reruns every phase that reviewed the old plan; p1d reopens every phase; p1c keeps the gate; p1e ends incomplete', () => {
const out = gatedRun('eng-only', ['--max-rounds', '3', '--no-carry-forward']);
const engOnly = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', 'all except uc2b', '--json']);
const engOnly = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '1', '--reply', recCarrying(out, 'all', ['uc2b']), '--json']);
const runId = readRun(out).run;
expect(JSON.parse(engOnly.stdout)).toMatchObject({ outcome: 'amend_pending', reopened: [], amendments: [`${runId}-uc2`] });
expect(readRun(out).phases.ceo.closed_at).toBeTruthy();
@@ -178,13 +180,13 @@ describe('answer', () => {
expect(runPhase(out, 'ceo').outside.stdout).toMatch(/PHASE_CLOSED: ceo/);
const eng = runPhase(out, 'eng');
expect(eng.outside.stdout).toContain('status=gate_pending');
const interrogate = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '2', '--reply', 'all except p1c', '--json']);
const interrogate = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '2', '--reply', recCarrying(out, 'all', ['p1c']), '--json']);
expect(JSON.parse(interrogate.stdout)).toMatchObject({ status: 'gate_pending', outcome: 'gate_pending' });
const revise = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '2', '--reply', 'all except p1d', '--json']);
const revise = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '2', '--reply', recCarrying(out, 'all', ['p1d']), '--json']);
expect(JSON.parse(revise.stdout)).toMatchObject({ outcome: 'reopened', reopened: ['ceo', 'eng'] });
runPhase(out, 'ceo');
runPhase(out, 'eng');
const reject = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '3', '--reply', 'all except p1e']);
const reject = autoplan(['answer', '--by', 'human', '--out', out, '--gate-rev', '3', '--reply', recCarrying(out, 'all', ['p1e'])]);
expect(reject.status).toBe(0);
expect(reject.stdout).toContain('GSTACK_RESULT: skill=autoplan status=incomplete run=' + out);
expect(readRun(out).deviations).toContain('plan rejected by the owner at gate_rev 3');
+36 -2
View File
@@ -1,7 +1,7 @@
/**
* Field list 2, W5: the autoplan run budget on the simulated Capy host. An
* unattended Capy run reserves the default per-voice estimate so --spend-cap
* binds, and a refused dispatch stops the run `incomplete` with a structured
* unattended run under --spend-cap (or on Capy) reserves the default per-voice
* estimate so the cap binds, `--estimate-usd` persists for later phases, and a refused dispatch stops the run `incomplete` with a structured
* stop object that export, validate and resume understand; `budget` journals
* the owner's raise. `close --deferred` keeps a dispatched voice's
* reservation, routes an open Critical to the gate and carries open findings
@@ -258,3 +258,37 @@ describe('confirm rounds', () => {
expect(rows[1].settled_by).toBeUndefined();
});
});
describe('the cap binds on every unattended host (GBRA-223 claim 4)', () => {
const OFF_CAPY = { GSTACK_CAPY: '' };
test('a capped unattended run off Capy reserves the default per voice, so the cap stops it; with no cap it reserves nothing', () => {
const dir = path.join(r.host.base, 'offcapy');
const out = path.join(dir, 'run');
const first = r.ap(['next', '--out', out, '--plan', writePlan(dir), '--no-ui', '--spend-cap', '12'], OFF_CAPY);
expect(first.status, first.stderr).toBe(0);
expect(first.stdout).toMatch(/^ESTIMATE: 5\.00 per voice reserved for each host-subagent attempt/m);
expect(spend(out).reserved.map((x: any) => x.estimate_usd)).toEqual([DEFAULT_HOST_ESTIMATE_USD, DEFAULT_HOST_ESTIMATE_USD]);
phaseVia(r, out, 'ceo', NONE);
const stopped = r.ap(['next', '--out', out], OFF_CAPY);
expect(stopped.status).toBe(3);
expect(stopped.stderr).toMatch(/\(SPEND_CAP_EXCEEDED\)$/m);
const freeDir = path.join(r.host.base, 'offcapy-nocap');
const free = path.join(freeDir, 'run');
const uncapped = r.ap(['next', '--out', free, '--plan', writePlan(freeDir), '--no-ui'], OFF_CAPY);
expect(uncapped.status, uncapped.stderr).toBe(0);
expect(uncapped.stdout).not.toMatch(/^ESTIMATE:/m);
expect(spend(free).reserved.map((x: any) => x.estimate_usd)).toEqual([0, 0]);
});
test('--estimate-usd persists: later phases reserve it without the flag', () => {
const s = start(['--spend-cap', '40', '--estimate-usd', '3']);
expect(spend(s.out).reserved.map((x: any) => x.estimate_usd)).toEqual([3, 3]);
expect(readRunJson(s.out).estimate_usd).toBe(3);
phaseVia(r, s.out, 'ceo', NONE);
expect(spend(s.out).settled.map((x: any) => x.reserved_usd)).toEqual([3, 3]);
const eng = r.ap(['next', '--out', s.out]);
expect(eng.status, eng.stderr).toBe(0);
expect(eng.stdout).toMatch(/^PHASE: eng /m);
expect(spend(s.out).reserved.map((x: any) => x.estimate_usd)).toEqual([3, 3]);
});
});
+9 -9
View File
@@ -12,7 +12,7 @@ import { afterAll, describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { RESULT_CODES } from '../lib/result-codes';
import { ROOT, SMALL_PLAN, boundOf, resultOf } from './helpers/autoplan-run-fixture';
import { ROOT, SMALL_PLAN, boundOf, recCarrying, resultOf } from './helpers/autoplan-run-fixture';
import { capyRunner, phaseVia, readJsonl, readRunJson, writePlan, type CapyRunner } from './helpers/autoplan-capy';
const r = capyRunner();
@@ -55,11 +55,11 @@ describe('unattended runs on the Capy host', () => {
phaseVia(r, s.out, 'ceo');
expect(phaseVia(r, s.out, 'eng').outside.stdout).toContain('status=gate_pending');
expectReviewedHashes(s.out);
expect(ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', String(rev), '--reply', 'all except p1d'])).stdout).toMatch(/^REOPENED: ceo,eng \(Eng last\) reason=revise$/m);
expect(ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', String(rev), '--reply', recCarrying(s.out, 'all', ['p1d'])])).stdout).toMatch(/^REOPENED: ceo,eng \(Eng last\) reason=revise$/m);
}
phaseVia(r, s.out, 'ceo');
phaseVia(r, s.out, 'eng');
expect(ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '3', '--reply', 'all'])).stdout).toContain('status=complete');
expect(ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '3', '--reply', recCarrying(s.out)])).stdout).toContain('status=complete');
ok(r.ap(['export', '--out', s.out]));
const v = ok(r.validate(s.out));
expect(v.stdout).toMatch(/^ARTIFACT_VALID: run=\S+ status=complete/m);
@@ -83,7 +83,7 @@ describe('unattended runs on the Capy host', () => {
const gated = ok(r.ap(['submit', '--out', s.out, '--phase', 'eng', '--voice', 'outside', '--unavailable', 'no cross-family subagent on this host']));
expect(gated.stdout).toMatch(/^PHASE_CLOSED: eng .*coverage=native_only/m);
expect(gated.stdout).toContain('status=gate_pending');
expect(ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '1', '--reply', 'all'])).stdout).toContain('status=complete');
expect(ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '1', '--reply', recCarrying(s.out)])).stdout).toContain('status=complete');
ok(r.ap(['export', '--out', s.out]));
expect(ok(r.validate(s.out)).stdout).toMatch(/^ARTIFACT_VALID: run=\S+ status=complete attested=1$/m);
expect(fs.readFileSync(path.join(s.out, 'review-record.md'), 'utf8')).toMatch(/attested/);
@@ -94,7 +94,7 @@ describe('unattended runs on the Capy host', () => {
const s = start();
phaseVia(r, s.out, 'ceo');
phaseVia(r, s.out, 'eng');
ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '1', '--reply', 'all except p1d']));
ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '1', '--reply', recCarrying(s.out, 'all', ['p1d'])]));
phaseVia(r, s.out, 'ceo');
phaseVia(r, s.out, 'eng');
const file = path.join(s.out, 'decisions.jsonl');
@@ -102,7 +102,7 @@ describe('unattended runs on the Capy host', () => {
const manifest = readRunJson(s.out);
delete manifest.gstack_version;
fs.writeFileSync(path.join(s.out, 'run.json'), JSON.stringify(manifest, null, 2) + '\n');
const res = ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '2', '--reply', 'all']));
const res = ok(r.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '2', '--reply', recCarrying(s.out)]));
expect(res.stdout).toMatch(/^GSTACK_VERSION_CHANGED: run unknown -> /m);
expect(res.stdout).toContain('status=complete');
ok(r.ap(['export', '--out', s.out]));
@@ -150,7 +150,7 @@ describe('conformance: one run through every mechanism', () => {
const accept = JSON.parse(fs.readFileSync(path.join(s.out, 'gate.json'), 'utf8')).items.find((i: any) => i.title.startsWith('Gate the backoff')).id;
// partial, then full
expect(ok(c.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '1', '--reply', `${accept}b`])).stdout).toMatch(/^GATE_PENDING: p1/m);
const full = ok(c.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '1', '--reply', 'all']));
const full = ok(c.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '1', '--reply', recCarrying(s.out)]));
expect(full.stdout).toMatch(new RegExp(`^APPLY: ${run}-${accept} `, 'm'));
const pendingNext = c.ap(['next', '--out', s.out]).stderr;
expect(pendingNext).toMatch(/\(PLAN_AMEND_PENDING\)$/m);
@@ -169,10 +169,10 @@ describe('conformance: one run through every mechanism', () => {
for (const voice of ['native', 'outside']) ok(c.ap(['submit', '--out', s.out, '--phase', 'ceo', '--voice', voice, '--result', resultOf(s.out, 'ceo', voice as 'native' | 'outside'), '--model', voice === 'native' ? 'anthropic/claude-opus-5-5' : 'openai/gpt-6.1-sol']));
phaseVia(c, s.out, 'eng');
expect(readRunJson(s.out).gate_rev).toBe(2);
ok(c.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '2', '--reply', 'all except p1d']));
ok(c.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '2', '--reply', recCarrying(s.out, 'all', ['p1d'])]));
phaseVia(c, s.out, 'ceo');
phaseVia(c, s.out, 'eng');
expect(ok(c.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '3', '--reply', 'all'])).stdout).toContain('status=complete');
expect(ok(c.ap(['answer', '--by', 'human', '--out', s.out, '--gate-rev', '3', '--reply', recCarrying(s.out)])).stdout).toContain('status=complete');
ok(c.ap(['export', '--out', s.out]));
expect(ok(c.validate(s.out)).stdout).toMatch(/^ARTIFACT_VALID: run=\S+ status=complete/m);
const final = readRunJson(s.out);
+74 -6
View File
@@ -31,7 +31,7 @@ const decisions = (out: string) => readJsonl(path.join(out, 'decisions.jsonl'));
const journal = (out: string) => readJsonl(path.join(out, 'attempts.jsonl'));
/** Gate 1 with one recommended plan change (uc1b), answered with the owner's words, then revised to take it. */
function answeredAndRevised(args: string[] = [], words = 'Garry: Do it', ceo: { native?: unknown[]; outside?: unknown[] } = AGREE) {
function answeredAndRevised(args: string[] = [], words = 'Garry: Do it', ceo: { native?: unknown[]; outside?: unknown[] } = AGREE, answerArgs: string[] = []) {
const dir = path.join(r.host.base, `carry${++n}`);
const plan = writePlan(dir);
const out = path.join(dir, 'run');
@@ -39,7 +39,7 @@ function answeredAndRevised(args: string[] = [], words = 'Garry: Do it', ceo: {
phaseVia(r, out, 'ceo', ceo);
expect(phaseVia(r, out, 'eng', NONE).outside.stdout).toContain('status=gate_pending');
const run = readRunJson(out).run;
const ans = r.ap(['answer', '--out', out, '--gate-rev', '1', '--reply', 'rec', '--rec-hash', recHashOf(out), '--by', 'agent:gstack-minder', '--owner-reply', words]);
const ans = r.ap(['answer', '--out', out, '--gate-rev', '1', '--reply', 'rec', '--rec-hash', recHashOf(out), '--by', 'agent:gstack-minder', '--owner-reply', words, ...answerArgs]);
expect(ans.status, ans.stderr).toBe(0);
expect(ans.stdout).toMatch(/^APPLY: /m);
const v2 = path.join(dir, 'plan-v2.md');
@@ -50,6 +50,58 @@ function answeredAndRevised(args: string[] = [], words = 'Garry: Do it', ceo: {
}
const recHashOf = (out: string) => recHash(gateOf(out));
describe('whose words answered the gate (GEVA-2, P7)', () => {
test("an agent's own words are recorded as its own, never as the owner's, never carried, and imported under the agent", () => {
const s = answeredAndRevised([], 'Accepted on the standing approval', AGREE, ['--words-by', 'agent:gstack-minder']);
const p1 = decisions(s.out).find(d => d.gate_rev === 1 && d.label === 'P1');
expect(p1).toMatchObject({ answer_words: 'Accepted on the standing approval', words_by: 'agent:gstack-minder' });
expect(p1.owner_reply).toBeUndefined();
phaseVia(r, s.out, 'ceo', NONE);
const eng = phaseVia(r, s.out, 'eng', NONE).outside;
expect(eng.stdout).toMatch(/^CARRY: no \(gate_rev 1 was answered in agent:gstack-minder's own words, not the owner's\)$/m);
expect(eng.stdout).toContain('status=gate_pending');
const rows = importAutoplan(s.out, 'feature/x', 'Garry');
if ('code' in rows) throw new Error(rows.message);
expect(rows[0]).toMatchObject({ by: "agent:gstack-minder, its own decision (not the owner's words)", words: 'Accepted on the standing approval' });
expect(r.ap(['export', '--out', s.out]).status).toBe(0);
expect(r.validate(s.out).status).toBe(0);
});
test('a later answer from someone else replaces the earlier answer\'s authorship and words; nothing stale survives', () => {
const dir = path.join(r.host.base, `carry${++n}`);
const out = path.join(dir, 'run');
expect(r.ap(['next', '--out', out, '--plan', writePlan(dir), '--no-ui']).status).toBe(0);
phaseVia(r, out, 'ceo', NONE);
phaseVia(r, out, 'eng', NONE);
const first = r.ap(['answer', '--out', out, '--gate-rev', '1', '--reply', 'p1c', '--by', 'agent:gstack-minder', '--words-by', 'agent:gstack-minder', '--owner-reply', 'p1c']);
expect(first.status, first.stderr).toBe(0);
const second = r.ap(['answer', '--out', out, '--gate-rev', '1', '--reply', 'p1a', '--by', 'human']);
expect(second.status, second.stderr).toBe(0);
expect(second.stdout).toMatch(/^APPROVED: gate_rev=1/m);
const p1 = decisions(out).find(d => d.gate_rev === 1 && d.label === 'P1');
expect(p1).toMatchObject({ chosen: 'a', answered_by: 'human' });
for (const stale of ['words_by', 'answer_words', 'answer_words_parsed', 'owner_reply', 'owner_reply_parsed']) expect(p1[stale]).toBeUndefined();
const rows = importAutoplan(out, 'feature/x', 'Garry');
if ('code' in rows) throw new Error(rows.message);
expect(rows[0]!.by).not.toContain('agent:gstack-minder');
});
test('--words-by names the owner or the answering agent; anything else is refused', () => {
const dir = path.join(r.host.base, `carry${++n}`);
const out = path.join(dir, 'run');
expect(r.ap(['next', '--out', out, '--plan', writePlan(dir), '--no-ui']).status).toBe(0);
phaseVia(r, out, 'ceo', AGREE);
phaseVia(r, out, 'eng', NONE);
const base = ['answer', '--out', out, '--gate-rev', '1', '--reply', 'rec', '--rec-hash', recHashOf(out), '--owner-reply', 'ok'];
for (const bad of [['--by', 'agent:gstack-minder', '--words-by', 'agent:someone-else'], ['--by', 'human', '--words-by', 'agent:gstack-minder'], ['--by', 'agent:gstack-minder', '--words-by', 'Garry']]) {
const res = r.ap([...base, ...bad]);
expect(res.status, res.stdout).not.toBe(0);
expect(res.stderr).toContain('--words-by must be owner or the answering agent');
}
expect(decisions(out).every(d => d.status === 'pending')).toBe(true);
});
});
describe('carry-forward', () => {
test('a second gate with nothing new carries the owner\'s words: CARRIED, APPROVED, the record names its source', () => {
const s = answeredAndRevised();
@@ -123,20 +175,36 @@ describe('carry-forward', () => {
expect(phaseVia(r, rec.out, 'eng', PKG).outside.stdout).toMatch(/^CARRY: no \(new item \w+: Split the backoff/m);
});
test('an earlier answer in item tokens, or one that left an item unanswered, is not carried', () => {
/** Gate 1 answered item by item (every item's recommendation as tokens) by the owner, then revised to take uc1. */
function tokensAndRevised() {
const dir = path.join(r.host.base, `carry${++n}`);
const out = path.join(dir, 'run');
expect(r.ap(['next', '--out', out, '--plan', writePlan(dir), '--no-ui']).status).toBe(0);
phaseVia(r, out, 'ceo', AGREE);
phaseVia(r, out, 'eng', NONE);
const g = gateOf(out);
const tokens = g.items.map((i: any) => `${i.id}${i.recommended}`).join(' ');
const tokens = gateOf(out).items.map((i: any) => `${i.id}${i.recommended}`).join(' ');
expect(r.ap(['answer', '--out', out, '--gate-rev', '1', '--reply', tokens, '--by', 'human']).status).toBe(0);
const v2 = path.join(dir, 'plan-v2.md');
fs.writeFileSync(v2, V2);
expect(r.ap(['revise', '--out', out, '--plan', v2, '--incorporates', 'uc1']).status).toBe(0);
phaseVia(r, out, 'ceo', NONE);
expect(phaseVia(r, out, 'eng', NONE).outside.stdout).toMatch(/^CARRY: no \(gate_rev 1 was answered with item tokens, not rec or "do it"\)$/m);
return { out, tokens, p1: decisions(out).find(x => x.gate_rev === 1 && x.label === 'P1') };
}
test('an item-token answer carries when the later gate asks nothing new: the owner\'s p1 choice and verbatim tokens are the basis', () => {
const s = tokensAndRevised();
const eng = phaseVia(r, s.out, 'eng', NONE).outside;
expect(eng.status, eng.stderr).toBe(0);
expect(eng.stdout).toContain(`CARRIED: gate_rev=2 <- gate_rev=1 words=${JSON.stringify(s.tokens)} by=human reason=no new choice; plan unchanged since the accepted changes`);
expect(eng.stdout).toMatch(/^APPROVED: gate_rev=2/m);
expect(decisions(s.out).find(x => x.gate_rev === 2 && x.label === 'P1')).toMatchObject({ chosen: s.p1.chosen, carried_gate_from: s.p1.id });
});
test('an item-token answer is not carried past a new item', () => {
const s = tokensAndRevised();
const PKG = { native: [{ id: 'F7', severity: 'Medium', title: 'Split the backoff into its own PR', user_challenge: true, changes: 'packaging', fix: 'own PR' }], outside: [] };
expect(phaseVia(r, s.out, 'eng', PKG).outside.stdout).toMatch(/^CARRY: no \(new item \w+: Split the backoff/m);
expect(readRunJson(s.out).status).toBe('gate_pending');
});
test('an answer that left an approval item pending is not carried', () => {
+123
View File
@@ -0,0 +1,123 @@
import { afterAll, describe, expect, test } from 'bun:test';
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { parseResult } from '../lib/headless-artifacts';
import { CEO_REVIEW_HEADINGS, ceoOnlyEvidence, listFiles } from './helpers/autoplan-ceo-only-evidence';
const ROOT = path.resolve(import.meta.dir, '..');
const read = (rel: string) => fs.readFileSync(path.join(ROOT, rel), 'utf8');
const handoff = (...args: string[]) => spawnSync('bash', [path.join(ROOT, 'bin/gstack-ceo-mode-handoff'), ...args], { encoding: 'utf8', timeout: 10_000 });
describe('/autoplan --ceo-only skill text', () => {
const skill = read('autoplan/SKILL.md');
const section = read('autoplan/sections/ceo-only.md');
test('the section index routes --ceo-only to its section before any preamble block', () => {
expect(skill).toContain('| the invocation names `--ceo-only` (read before any preamble instruction block) | `sections/ceo-only.md` |');
});
test('the section forbids every question channel and names the skipped offers', () => {
for (const s of ['Never call AskUserQuestion (native or `mcp__*`) or\nExitPlanMode', 'never ask the user in prose', 'never wait for a reply',
'office-hours and BENEFITS_FROM', "0E's question", 'Next Steps chaining and CEO plan\npromotion', '**Never applied:** a user challenge'])
expect(section).toContain(s);
});
test('the section runs CEO only, honors --no-outside and the state root, and ends status=reviewed', () => {
expect(section).toContain('Run Phases 0, 0.5 and 1 only');
expect(section).toContain("`outside_status: disabled (--no-outside)`");
expect(section).toContain('`GSTACK_STATE_ROOT`\n(the override');
expect(section).toContain('gstack-ceo-mode-handoff "<mode>" --ceo-only');
expect(section).toContain('`GSTACK_RESULT: skill=autoplan status=reviewed run=<absolute $_AP>`');
for (const h of CEO_REVIEW_HEADINGS) expect(section.replace(/\`\n\`/g, '` `').replace(/\n/g, ' ')).toContain(h.replace(/\n/g, ' '));
});
});
test('a normal /autoplan run never reads sections/ceo-only.md (the parity raise in carve-guards.ts rests on it)', () => {
const skill = read('autoplan/SKILL.md');
expect(skill.split('ceo-only.md').length - 1).toBe(1);
expect(skill).toContain('| the invocation names `--ceo-only` (read before any preamble instruction block) | `sections/ceo-only.md` |');
const others = fs.readdirSync(path.join(ROOT, 'autoplan/sections')).filter(f => f.endsWith('.md') && f !== 'ceo-only.md')
.map(f => `autoplan/sections/${f}`);
const runner = ['bin/gstack-autoplan', 'autoplan/DRIVER.md', ...fs.readdirSync(path.join(ROOT, 'lib')).filter(f => f.startsWith('autoplan-')).map(f => `lib/${f}`)];
for (const rel of [...others, ...runner]) expect(read(rel), rel).not.toContain('ceo-only.md');
});
describe('gstack-ceo-mode-handoff --ceo-only', () => {
test('names the CEO-only rule, not a saved preference', () => {
const r = handoff('SELECTIVE_EXPANSION', '--ceo-only');
expect(r.status).toBe(0);
expect(r.stdout.trim()).toBe("Auto-selected review mode → SELECTIVE EXPANSION (CEO-only run: Step 0E's rule). Approved decisions: none.");
});
test('refuses --auto with --ceo-only', () => {
const r = handoff('HOLD SCOPE', '--auto', '--ceo-only');
expect(r.status).toBe(2);
expect(r.stderr).toContain('--auto and --ceo-only conflict');
});
});
test('GSTACK_RESULT parses status=reviewed', () => {
expect(parseResult('x\nGSTACK_RESULT: skill=autoplan status=reviewed run=/r')).toEqual({ skill: 'autoplan', status: 'reviewed', run: '/r' });
});
describe('ceo-only evidence: a passing run and each negative control', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ceo-only-evidence-'));
afterAll(() => fs.rmSync(root, { recursive: true, force: true }));
const runDir = path.join(root, 'run');
fs.mkdirSync(runDir);
const review = ['# CEO-only review: Pricing tier', '', 'Mode: HOLD SCOPE (ceo-only). Not approved.', '',
'## User challenges (direction kept)', 'None.', '## Scope added', 'None.', '## Conservative choices', 'None.',
'## Findings', '- High: the premise was never checked with developers; interview them first.',
'## Decision Audit Trail', '| # | Phase | Decision |', '|---|---|---|', '| 1 | CEO | Mode HOLD SCOPE |', ''].join('\n');
fs.writeFileSync(path.join(runDir, 'ceo-review.md'), review);
const assistant = (...content: any[]) => ({ type: 'assistant', message: { content } });
const user = (...content: any[]) => ({ type: 'user', message: { content } });
const good = [
assistant({ type: 'tool_use', id: 't1', name: 'Bash', input: { command: '~/.claude/skills/gstack/bin/gstack-ceo-mode-handoff "HOLD SCOPE" --ceo-only --decisions "none"' } }),
user({ type: 'tool_result', tool_use_id: 't1', content: "Auto-selected review mode → HOLD SCOPE (CEO-only run: Step 0E's rule). Approved decisions: none." }),
assistant({ type: 'tool_use', id: 't2', name: 'Agent', input: { prompt: 'native CEO review' } }),
assistant({ type: 'text', text: 'Phases 2, 2.5, 3 and 4 skipped (CEO-only run).' }),
];
const output = `Review done.\nGSTACK_RESULT: skill=autoplan status=reviewed run=${runDir}`;
const base = { transcript: good, output, strayWrites: [] as string[], mustMention: [/developer/i, /interview|talk|validat/i] };
test('a complete run passes', () => {
const e = ceoOnlyEvidence(base);
expect(e.reasons).toEqual([]);
expect(e.runDir).toBe(runDir);
});
const fails = (name: string, patch: Partial<typeof base>, reason: RegExp) => test(name, () => {
const e = ceoOnlyEvidence({ ...base, ...patch });
expect(e.passed).toBe(false);
expect(e.reasons.join('\n')).toMatch(reason);
});
fails('an AskUserQuestion call fails', { transcript: [...good, assistant({ type: 'tool_use', id: 'q', name: 'AskUserQuestion', input: {} })] }, /asked 1 question/);
fails('an MCP AskUserQuestion variant fails', { transcript: [...good, assistant({ type: 'tool_use', id: 'q', name: 'mcp__conductor__AskUserQuestion', input: {} })] }, /asked 1 question/);
fails('ExitPlanMode fails', { transcript: [...good, assistant({ type: 'tool_use', id: 'q', name: 'ExitPlanMode', input: {} })] }, /tool ExitPlanMode/);
fails('a prose question fails', { output: `Which mode would you like me to use?\n${output}` }, /prose: Which mode would you like/);
fails('a wrong terminal status fails', { output: output.replace('status=reviewed', 'status=gate_pending') }, /status=gate_pending, expected reviewed/);
fails('a missing terminal line fails', { output: 'Review done.' }, /no GSTACK_RESULT line/);
fails('headings without a review fail', { transcript: [assistant({ type: 'text', text: 'Done.' })] }, /no gstack-ceo-mode-handoff[\s\S]*no CEO reviewer subagent/);
fails('loading a skipped phase fails', { transcript: [...good, assistant({ type: 'tool_use', id: 'r', name: 'Read', input: { file_path: '/h/.claude/skills/gstack/autoplan/sections/eng-phase.md' } })] }, /loaded a skipped phase/);
fails('loading a skipped phase through Bash fails', { transcript: [...good, assistant({ type: 'tool_use', id: 'b', name: 'Bash', input: { command: 'cat /h/.claude/skills/gstack/autoplan/sections/dx-phase.md' } })] }, /loaded a skipped phase: cat /);
fails('a write outside its roots fails', { strayWrites: ['.config/x.json'] }, /wrote 1 file\(s\) outside its roots, under \.config\/x\.json/);
fails('a review that misses the seeded flaw fails', { mustMention: [/churn/i] }, /never names the seeded flaw/);
test('a missing ceo-review.md fails', () => {
const empty = path.join(root, 'empty');
fs.mkdirSync(empty);
const e = ceoOnlyEvidence({ ...base, output: `GSTACK_RESULT: skill=autoplan status=reviewed run=${empty}` });
expect(e.reasons.join('\n')).toContain('ceo-review.md is missing');
});
test('listFiles skips the declared roots', () => {
const home = path.join(root, 'home');
fs.mkdirSync(path.join(home, 'state'), { recursive: true });
fs.writeFileSync(path.join(home, 'state', 'a'), '');
fs.writeFileSync(path.join(home, 'b'), '');
fs.mkdirSync(path.join(home, '.bun', 'install', 'cache'), { recursive: true });
fs.writeFileSync(path.join(home, '.bun', 'install', 'cache', 'c'), '');
fs.mkdirSync(path.join(home, '.bun', 'bin'), { recursive: true });
fs.writeFileSync(path.join(home, '.bun', 'bin', 'd'), '');
expect(listFiles(home, ['state', '.bun/install/cache'])).toEqual(['.bun/bin/d', 'b']);
});
});
+2 -2
View File
@@ -188,7 +188,7 @@ describe('E16 end to end: every gate-producing command prints the cost, and the
test('the closing submit prints GATE_SPEND/GATE_REC_COST (text) or gate_cost (--json); the owner brief carries the Cost so far sentence and render --captain a ### Cost section above the relaying trailer', () => {
const out = gateRun('plain', {}, { challenge: false, json: true });
const out = gateRun('plain', { severity: 'Medium' }, { challenge: false, json: true });
const brief = spawnSync(process.execPath, [path.join(ROOT, 'bin', 'gstack-owner-brief'), 'render', '--out', out], { cwd: ROOT, env: env(), encoding: 'utf8', timeout: 30_000 });
expect(brief.stdout).toMatch(/Cost so far: spent \d+\.\d\d, worst case \d+\.\d\d \(/);
const captain = spawnSync(process.execPath, [path.join(ROOT, 'bin', 'gstack-owner-brief'), 'render', '--captain', '--out', out, '--allow-incomplete'], { cwd: ROOT, env: env(), encoding: 'utf8', timeout: 30_000 });
@@ -203,7 +203,7 @@ describe('E16 end to end: every gate-producing command prints the cost, and the
});
test('run at a pending gate prints the gate lines once: text mode carries GATE_SPEND/GATE_REC_COST, --json exactly one payload with gate_cost', () => {
const out = gateRun('run-gate', {}, { challenge: false });
const out = gateRun('run-gate', { severity: 'Medium' }, { challenge: false });
const models = ['--native-model', 'anthropic/claude-opus-4-7', '--outside-model', 'openai/gpt-6-astra'];
const text = autoplan(['run', '--out', out, ...models]);
expect(text.status).toBe(0);
+105 -7
View File
@@ -17,8 +17,9 @@ import { afterAll, describe, expect, test } from 'bun:test';
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { ROOT, SMALL_PLAN, boundOf, resultOf } from './helpers/autoplan-run-fixture';
import { ROOT, SMALL_PLAN, boundOf, recCarrying, resultOf } from './helpers/autoplan-run-fixture';
import { capyRunner, phaseVia, readJsonl, readRunJson, writePlan } from './helpers/autoplan-capy';
import { settledIds } from '../lib/autoplan-confirm';
const r = capyRunner();
afterAll(() => r.cleanup());
@@ -119,7 +120,7 @@ describe('accepted changes are amendments', () => {
expect(held.status, held.stderr).toBe(0);
expect(held.stdout).toMatch(/^GATE_PENDING: uc2, p1 unanswered \(reply with uc2a\|uc2b p1a\|p1b\|p1c\|p1d\|p1e or rec\)$/m);
expect(held.stdout).not.toMatch(/^APPLY:/m);
const res = answer(g.out, 1, 'rec');
const res = answer(g.out, 1, recCarrying(g.out, 'rec'));
expect(res.status, res.stderr).toBe(0);
expect(res.stdout).toMatch(new RegExp(`^APPLY: ${esc(g.run)}-uc1 Ship the backoff behind a flag first .* -> accept the change both reviewers recommend.*; edit the plan, then revise$`, 'm'));
expect(res.stdout).not.toMatch(/^REOPENED:/m);
@@ -165,7 +166,7 @@ describe('accepted changes are amendments', () => {
expect(asIs.stdout).not.toMatch(/^APPLY:/m);
expect(decisions(g.out).find(d => d.label === 'UC1')).toMatchObject({ chosen: 'a', status: 'approved' });
const w = gated(AGREE, {});
expect(answer(w.out, 1, 'rec except uc1b').stdout).toMatch(/^APPLY:/m);
expect(answer(w.out, 1, recCarrying(w.out, 'rec', ['uc1b'])).stdout).toMatch(/^APPLY:/m);
const withdraw = r.ap(['revise', '--out', w.out, '--withdraw', `${w.run}-uc1`]);
expect(withdraw.status, withdraw.stderr).toBe(0);
expect(withdraw.stdout).toMatch(/^APPROVED: gate_rev=1 \(every accepted change is in the plan or settled\)$/m);
@@ -179,7 +180,7 @@ describe('record-only changes settle without a plan edit', () => {
test('a packaging change: APPLY says settle, revise --settle reopens nothing and completes, export and validate pass; a scope change never settles', () => {
const g = gated(PACKAGING, {});
expect(gate(g.out).items.find((i: any) => i.id === 'uc1')).toMatchObject({ recommended: 'b', changes: 'packaging' });
const res = answer(g.out, 1, 'rec');
const res = answer(g.out, 1, recCarrying(g.out, 'rec'));
expect(res.status, res.stderr).toBe(0);
expect(res.stdout).toMatch(new RegExp(`^APPLY: ${esc(g.run)}-uc1 .*; no plan edit needed: revise --settle uc1$`, 'm'));
expect(readRunJson(g.out).status).toBe('amend_pending');
@@ -196,7 +197,7 @@ describe('record-only changes settle without a plan edit', () => {
expect(r.ap(['export', '--out', g.out]).status).toBe(0);
expect(r.validate(g.out).status).toBe(0);
const scope = gated(AGREE, {});
expect(answer(scope.out, 1, 'rec except uc1b').stdout).toMatch(/; edit the plan, then revise$/m);
expect(answer(scope.out, 1, recCarrying(scope.out, 'rec', ['uc1b'])).stdout).toMatch(/; edit the plan, then revise$/m);
const refused = r.ap(['revise', '--out', scope.out, '--settle', 'uc1']);
expect(refused.status).toBe(3);
expect(refused.stderr).toMatch(/changes scope; only packaging or ordering changes settle.*\(AMENDMENT_NOT_SETTLEABLE\)$/m);
@@ -229,7 +230,7 @@ describe('mixed amendments: a scope change and a packaging change', () => {
expect(readRunJson(g.out).status).toBe('running');
phaseVia(r, g.out, 'ceo');
expect(phaseVia(r, g.out, 'eng').outside.stdout).toContain('status=gate_pending');
expect(answer(g.out, 2, 'rec').stdout).toContain('status=complete');
expect(answer(g.out, 2, recCarrying(g.out, 'rec')).stdout).toContain('status=complete');
expect(r.validate(g.out).status).toBe(0);
});
test('settle and incorporate in one revise; settling alone on an unreviewed plan edit is refused', () => {
@@ -262,7 +263,7 @@ describe('partial replies resolve against persisted decisions', () => {
test('validate rejects a complete run with a pending approval', () => {
const g = gated({}, {});
expect(answer(g.out, 1, 'all').stdout).toContain('status=complete');
expect(answer(g.out, 1, recCarrying(g.out)).stdout).toContain('status=complete');
const file = path.join(g.out, 'decisions.jsonl');
fs.writeFileSync(file, decisions(g.out).map(d => JSON.stringify(d.label === 'UC2' ? { ...d, status: 'pending', chosen: null } : d)).join('\n') + '\n');
r.ap(['export', '--out', g.out]);
@@ -298,3 +299,100 @@ describe('reruns', () => {
expect(decisions(g.out).find(d => d.gate_rev === 2 && d.label === demoted.id.toUpperCase())).toMatchObject({ chosen: 'b', source: 'agent' });
});
});
describe('open Critical and High findings reach the gate (GBRA-223 claim 3)', () => {
const NONE = { native: [], outside: [] };
const KILL = { native: [{ id: 'F1', severity: 'Critical', title: 'No kill switch for the resolver rollout', fix: 'add a config kill switch' }], outside: [] };
test('each open Critical/High finding is an apply item that rec takes; a twin pair is one item; Medium stays a task', () => {
const g = gated({
native: [{ id: 'F1', severity: 'Critical', title: 'No kill switch for the resolver rollout', fix: 'add a config kill switch' }, { id: 'F2', severity: 'Medium', title: 'Name the flag consistently' }],
outside: [{ id: 'O1', severity: 'High', title: 'No kill switch for the resolver rollout', fix: 'ship behind a kill switch' }],
}, { native: [{ id: 'F3', severity: 'High', title: 'No test covers a hung resolver', file: 'test/resolver.test.ts', fix: 'add a hung-resolver test' }], outside: [] });
const items = gate(g.out).items;
const apply = items.filter((i: any) => i.finding);
expect(apply.map((i: any) => i.title)).toEqual([
expect.stringMatching(/^Critical: No kill switch for the resolver rollout \[.*-ceo-native-1 \+ .*-ceo-outside-1\]$/),
expect.stringMatching(/^High: No test covers a hung resolver \[.*-eng-native-1\]$/),
]);
for (const i of apply) expect(i).toMatchObject({ kind: 'auto', recommended: 'a', options: [{ key: 'a', changes_plan: true }, { key: 'b', changes_plan: false }] });
expect(apply[0].options[0].text).toContain('add a config kill switch');
expect(apply[0].options[0].text).toContain('ship behind a kill switch');
expect(items.find((i: any) => i.id === 'p1')).toMatchObject({ recommended: 'b' });
expect(items.some((i: any) => /Name the flag/.test(i.title))).toBe(false);
const rec = answer(g.out, 1, 'rec');
expect(rec.stdout.match(/^APPLY: /gm)?.length).toBe(2);
expect(readRunJson(g.out).status).toBe('amend_pending');
});
test('a user challenge one line away does not hide an unrelated Critical: a location match alone is not a twin', () => {
const g = gated({
native: [{ id: 'F1', severity: 'Medium', title: 'Package the worker as its own service', file: 'plan.md', line: 10, user_challenge: true, changes: 'scope', fix: 'split the worker out' }],
outside: [{ id: 'O1', severity: 'Critical', title: 'Webhook payload reaches the SQL query unescaped', file: 'plan.md', line: 11, fix: 'use parameterized queries' }],
}, NONE);
const apply = gate(g.out).items.filter((i: any) => i.finding);
expect(apply.map((i: any) => i.title)).toEqual([expect.stringMatching(/^Critical: Webhook payload reaches the SQL query unescaped \[.*-ceo-outside-1\]$/)]);
});
test('b, as-is, or a token reply that leaves the item unanswered carries the finding into the build with no plan change', () => {
for (const pick of [(id: string) => `${id}b p1a`, () => 'as-is', () => 'p1a']) {
const g = gated(KILL, NONE);
const item = gate(g.out).items.find((i: any) => i.finding);
const done = answer(g.out, 1, pick(item.id), true);
expect(JSON.parse(done.stdout)).toMatchObject({ outcome: 'complete', reopened: [], amendments: [] });
expect(decisions(g.out).find(d => d.label === item.id.toUpperCase())).toMatchObject({ chosen: 'b' });
expect(readJsonl(path.join(g.out, 'tasks.jsonl')).some(t => /No kill switch/.test(t.title))).toBe(true);
}
const g = gated(KILL, NONE);
const item = gate(g.out).items.find((i: any) => i.finding);
answer(g.out, 1, 'p1a');
expect(readJsonl(path.join(g.out, 'attempts.jsonl')).find(e => e.event === 'answered')).toMatchObject({ answer_class: 'overrides', overrides: [`${item.id}b`] });
expect(settledIds({ out: g.out } as any).has(item.finding[0])).toBe(false);
});
test('p1b applies the finding fixes its option text lists, and an applied finding is settled', () => {
const g = gated(KILL, NONE);
const item = gate(g.out).items.find((i: any) => i.finding);
expect(gate(g.out).items.find((i: any) => i.id === 'p1').options[1].text).toContain(`${item.id}a`);
expect(answer(g.out, 1, 'p1b').stdout).toMatch(/^APPLY: /m);
expect(decisions(g.out).find(d => d.label === item.id.toUpperCase())).toMatchObject({ chosen: 'a' });
expect(readJsonl(path.join(g.out, 'attempts.jsonl')).find(e => e.event === 'answered')).toMatchObject({ answer_class: 'plan-changes', changes: [`${item.id}a`] });
expect(readRunJson(g.out).status).toBe('amend_pending');
expect(settledIds({ out: g.out } as any).has(item.finding[0])).toBe(true);
expect(settledIds({ out: g.out } as any, { findings: false }).has(item.finding[0])).toBe(false);
});
test('after recommend changes an item, p1b (which takes the recommendations) needs the --rec-hash the owner saw', () => {
const g = gated({ native: [{ id: 'F1', severity: 'High', title: 'Ship the backoff behind a flag first', user_challenge: true, changes: 'scope', fix: 'gate it behind RETRY_BACKOFF=1' }], outside: [] }, NONE);
const item = gate(g.out).items.find((i: any) => i.id !== 'p1' && i.kind === 'auto' && i.options.some((o: any) => o.key === 'b'));
expect(r.ap(['recommend', '--out', g.out, '--gate-rev', '1', '--item', item.id, '--option', 'b', '--basis', 'owner asked for the flag', '--by', 'agent:test']).status).toBe(0);
const stale = answer(g.out, 1, 'p1b');
expect(stale.status).toBe(3);
expect(stale.stderr).toContain('GATE_REC_STALE');
expect(decisions(g.out).every(d => d.status === 'pending')).toBe(true);
});
test('a partial reply records no auto default, so a later p1b still applies the finding fix', () => {
const g = gated({ native: [...AGREE.native, ...KILL.native.map(f => ({ ...f, id: 'F9' }))], outside: AGREE.outside }, NONE);
const item = gate(g.out).items.find((i: any) => i.finding);
const uc = gate(g.out).items.find((i: any) => i.id.startsWith('uc'));
expect(answer(g.out, 1, `${uc.id}a`).stdout).toMatch(/GATE_PENDING: p1/);
expect(decisions(g.out).find(d => d.label === item.id.toUpperCase())?.chosen ?? null).toBeNull();
answer(g.out, 1, 'p1b');
expect(decisions(g.out).find(d => d.label === item.id.toUpperCase())).toMatchObject({ chosen: 'a' });
});
test('a same-file association never merges an unrelated Critical into a user challenge', () => {
const g = gated({ native: [{ id: 'F1', severity: 'Medium', title: 'Split the backoff into its own PR', file: 'lib/worker.ts', user_challenge: true, changes: 'packaging', fix: 'land it separately' }],
outside: [{ id: 'O1', severity: 'Critical', title: 'SQL injection in the lookup query', file: 'lib/worker.ts', fix: 'parameterize the query' }] }, NONE);
expect(gate(g.out).items.some((i: any) => i.finding && /^Critical: SQL injection/.test(i.title))).toBe(true);
});
test('at the round cap, an outside Critical paired by file with an unrelated native Medium still counts as open', () => {
const dir = path.join(r.host.base, `gj${++n}`);
const out = path.join(dir, 'run');
expect(r.ap(['next', '--out', out, '--plan', writePlan(dir), '--no-ui', '--max-rounds', '1']).status).toBe(0);
phaseVia(r, out, 'ceo', { native: [{ id: 'F1', severity: 'Medium', title: 'rename the config key', file: 'src/resolver.ts' }], outside: [{ id: 'O1', severity: 'Critical', title: 'SQL injection in lookup query', file: 'src/resolver.ts' }] });
const eng = phaseVia(r, out, 'eng', NONE).outside.stdout;
expect(eng.split('\n').filter(l => l.startsWith('ESCALATE:'))).toEqual(['ESCALATE: evidence sufficient, decision needed (ceo: 1 High open)']);
});
});
+58 -1
View File
@@ -7,7 +7,7 @@
*/
import { describe, expect, test } from 'bun:test';
import fixture from './fixtures/autoplan-pairing/run-20261014-pairs.json';
import { reconcile } from '../lib/autoplan-reconcile';
import { reconcile, titleScore } from '../lib/autoplan-reconcile';
import { challengeDefault } from '../lib/autoplan-gate';
const pair = (name: string) => fixture.pairs.find(p => p.name.startsWith(name))! as any;
@@ -61,3 +61,60 @@ describe('pairing negative control from a real run', () => {
expect(challengeDefault(ok!, challenge(same.native, 'scope')[0], challenge(same.outside, 'scope')[0])).toMatchObject({ recommended: 'b', basis: 'both voices; same remedy' });
});
});
describe('plan reviews name a file without a line (GBRA-223 claim 1)', () => {
const row = (voice: 'native' | 'outside', n: number, f: Record<string, unknown>) => ({ schema_version: 1, run: 'r', id: `r-dx-${voice}-${n}`, phase: 'dx', voice, disposition: 'open', ...f }) as any;
const native = row('native', 1, { severity: 'Critical', title: 'per-turn resolver call has no hard timeout or fail-open', file: 'src/resolver.ts', resolution: 'add a 2s timeout and fail open' });
test('a CAP-coded outside title on the same file is associated, unverified, never new', () => {
const outside = row('outside', 1, { severity: 'Critical', title: 'CAP4:src/resolver.ts:recovery', file: 'src/resolver.ts', resolution: 'bound the call' });
const { rows, native_only } = reconcile([native], [outside]);
expect(rows[0]).toMatchObject({ disposition: 'unverified', basis: 'file', native_id: native.id });
expect(rows[0]!.resolution).toStartWith('same file, agreement unverified (src/resolver.ts)');
expect(native_only).toEqual([]);
});
test('differently worded titles on the same file associate; the best title among same-file natives wins', () => {
const other = row('native', 2, { severity: 'Medium', title: 'resolver output is not cached between turns', file: 'src/resolver.ts' });
const outside = row('outside', 1, { severity: 'Critical', title: 'Resolver call has no timeout and can hang the turn', file: 'src/resolver.ts' });
expect(reconcile([other, native], [outside]).rows[0]).toMatchObject({ disposition: 'unverified', basis: 'file', native_id: native.id });
});
test('a CAP code is not part of the title comparison', () => {
const outside = row('outside', 1, { severity: 'Critical', title: 'CAP4:src/resolver.ts:recovery per-turn resolver call has no hard timeout or fail-open', file: 'src/resolver.ts', resolution: 'add a 2s timeout and fail open' });
expect(reconcile([native], [outside]).rows[0]).toMatchObject({ disposition: 'confirmed', basis: 'title' });
});
test('controls: two lines far apart, or different files, stay new', () => {
const lined = row('native', 1, { ...native, id: 'r-dx-native-1', line: 10 });
expect(reconcile([lined], [row('outside', 1, { severity: 'High', title: 'retry loop has no bound', file: 'src/resolver.ts', line: 90 })]).rows[0]!.disposition).toBe('new');
expect(reconcile([native], [row('outside', 1, { severity: 'High', title: 'CAP6:src/cli.ts:fix', file: 'src/cli.ts' })]).rows[0]!.disposition).toBe('new');
});
test('a title twin is placed before the same-file fallback, so an unrelated finding cannot take its native', () => {
const n1 = row('native', 1, { severity: 'High', title: 'token refresh race drops the session', file: 'plan.md' });
const o1 = row('outside', 1, { severity: 'High', title: 'migration lacks rollback', file: 'plan.md' });
const o2 = row('outside', 2, { severity: 'High', title: 'token refresh race drops the session', file: 'plan.md' });
const { rows } = reconcile([n1], [o1, o2]);
expect(rows.map(r => [r.outside_id, r.native_id, r.basis])).toEqual([[o1.id, null, 'none'], [o2.id, n1.id, 'title']]);
});
test('control: the same CAP item and wording on two different files never pairs by title', () => {
const a = row('native', 1, { severity: 'High', title: 'CAP6:bin/gstack-upgrade:recovery no retry bound on network failure', file: 'bin/gstack-upgrade' });
const b = row('outside', 1, { severity: 'High', title: 'CAP6:lib/telemetry-sync.ts:recovery no retry bound on network failure', file: 'lib/telemetry-sync.ts' });
expect(reconcile([a], [b]).rows[0]).toMatchObject({ disposition: 'new', native_id: null });
});
test('two CAP titles on different paths never match by title, even when both findings sit on the plan file', () => {
const A = 'CAP6:bin/gstack-eval-plan pilot start:fix no fix line names the command';
const B = 'CAP6:bin/gstack-outside-voice run:fix no fix line names the command';
expect(titleScore(A, B)).toBe(0);
expect(titleScore(A, A)).toBe(1);
expect(titleScore(A, 'no fix line names the command')).toBe(1);
const n = row('native', 1, { severity: 'High', title: A, file: 'plan.md', resolution: 'name the command' });
const o = row('outside', 1, { severity: 'High', title: B, file: 'plan.md', resolution: 'name the command' });
expect(reconcile([n], [o]).rows[0]).toMatchObject({ basis: 'file', disposition: 'unverified' });
const bare = (x: any) => { const { file, ...rest } = x; return rest; };
expect(reconcile([bare(n)], [bare(o)]).rows[0]).toMatchObject({ disposition: 'new', native_id: null });
});
});
+63
View File
@@ -206,6 +206,16 @@ describe('Autoplan parent publication guard', () => {
expect(f.evaluate()).toMatchObject({ allow: false, reason: expect.stringContaining('earlier checkpoint') });
});
test('#3651: a rerun checkpoint whose create result carries the cwd-reset footer still retires the old close', () => {
const { f, tasks } = completedCycle(), create = checkpointWriter(f);
const oldClose = f.events.find(e => e.kind === 'use' && e.toolUseId === 'eng-close')!;
create('eng', 'new-generation');
(f.events.at(-1) as any).content += '\nShell cwd was reset to /home/user/project';
f.read('old-close-replayed', (oldClose as any).input.file_path as string); f.message('Phase 3 complete.');
f.input.tool_input = { file_path: tasks }; f.current();
expect(f.evaluate()).toMatchObject({ allow: false, reason: expect.stringContaining('earlier checkpoint') });
});
for (const status of ['partial', 'failed', 'pending'] as const)
test(`a ${status} rerun close cannot borrow the first cycle's completed report`, () => {
const { f, create, tasks } = completedCycle(), rerun = create('eng', 'rerun');
@@ -1596,3 +1606,56 @@ describe('PAX-3638: symlinked plan directory and superseded failed inits', () =>
expect(f.evaluate()).toMatchObject({ allow: false, reason: expect.stringContaining('(code mutation_pending,') });
});
});
describe('#3651: Claude Code cwd-reset footer on a Bash result', () => {
const FOOTER = '\nShell cwd was reset to /home/user/project';
function withInitTail(tail: string) {
const f = fixture();
(f.events.find(e => e.kind === 'result' && e.toolUseId === 'init') as any).content += tail;
return f;
}
const ready = (f: ReturnType<typeof fixture>) => { f.message(); f.current(); f.reorder(); return f.evaluate(); };
test('an init result carrying the footer still binds', () => {
expect(ready(withInitTail(FOOTER))).toEqual({ allow: true });
});
for (const [name, tail] of [['other trailing text', '\nunexpected trailing output'], ['two footers', `${FOOTER}${FOOTER}`], ['a footer that is not the last line', `${FOOTER}\n`]] as const)
test(`${name} after an init result is init_failed with the re-run hint, not the generic evidence denial`, () => {
const decision = ready(withInitTail(tail)) as any;
expect(decision.allow).toBe(false);
expect(decision.reason).toContain('(code init_failed,');
expect(decision.reason).toContain('re-run');
});
test('#3651: the hook path keeps the first binding when an init is re-run after a typed user message', async () => {
const footer = '\nShell cwd was reset to /home/user/project';
const run = async (tail: string, failed?: unknown) => {
const f = fixture(); (f.events[1] as any).content += tail; f.message();
if (failed !== undefined) { f.use('bad-init', 'Bash', (f.events[0] as any).input); f.result('bad-init', { content: failed }); }
f.use('re-init', 'Bash', (f.events[0] as any).input); f.result('re-init', { content: JSON.stringify({ ...f.init, reused: true }) + tail });
f.current();
const { rows, record } = f.journal();
const at = rows.findIndex(r => Array.isArray(r.message.content) && r.message.content.some((b: any) => b.type === 'tool_use' && ['bad-init', 're-init'].includes(b.id)));
(rows[at - 1]!.message as any).stop_reason = 'end_turn';
rows.splice(at, 0, record('user', 'Looks good, keep going.', { origin: { kind: 'human' }, promptSource: 'typed', promptId: randomUUID() }));
rows.forEach((r, i) => { r.parentUuid = i ? rows[i - 1]!.uuid : null; });
fs.writeFileSync(f.input.transcript_path, rows.map(x => JSON.stringify(x)).join('\n') + '\n');
return withNativeProjectDirectory(f.cwd, () => runPublicationHook(f.input, ROOT));
};
expect(await run('')).toEqual(OWNED_READ_APPROVAL);
expect(await run(footer)).toEqual(OWNED_READ_APPROVAL);
expect(await run('', 'unexpected output')).toEqual(OWNED_READ_APPROVAL);
expect(await run('', [{ type: 'text', text: 'a' }, { type: 'text', text: 'b' }])).toEqual(OWNED_READ_APPROVAL);
});
for (const [name, content] of [['did not parse', undefined], ['had no single text block', [{ type: 'text', text: 'a' }, { type: 'text', text: 'b' }]]] as const)
test(`a later clean init recovers the session after an init whose output ${name}`, () => {
const f = withInitTail('\nunexpected trailing output');
if (content) (f.events[1] as any).content = content;
f.events.splice(2, 0, { ...(structuredClone(f.events[0]!) as any), toolUseId: 'again' },
{ ...(structuredClone(f.events[1]!) as any), toolUseId: 'again', content: JSON.stringify({ ...f.init, reused: true }) });
expect(ready(f)).toEqual({ allow: true });
});
});
+213
View File
@@ -0,0 +1,213 @@
/**
* `gstack-autoplan run` with the api runner for both voices (GBRA-223 claim 6):
* a complete autoplan result (INPUT receipt plus the findings fence) binds and
* the run reaches its gate; a response stopped at the output limit is named
* truncated and interrupts the run. The provider is a local server, so no key
* or network is used.
*/
import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { DEFAULT_MAX_TOKENS } from '../lib/outside-voice-api';
import { AUTOPLAN, ROOT, SMALL_PLAN, env } from './helpers/autoplan-run-fixture';
const TMP = fs.mkdtempSync(path.join(os.tmpdir(), 'autoplan-run-api-'));
let mode: 'ok' | 'cut' | 'bad-outside' | 'bad-native' | 'hang' | 'down' = 'ok';
let calls = 0;
let openaiLimits: unknown[] = [];
let server: ReturnType<typeof Bun.serve>;
beforeAll(() => {
server = Bun.serve({
port: 0,
async fetch(req) {
calls++;
if (mode === 'hang') { await Bun.sleep(5000); return new Response('late', { status: 504 }); }
if (mode === 'down') return new Response('overloaded', { status: 503 });
const body = await req.json();
const prompt = String(body.messages[0].content);
const m = /INPUT: ([a-z]+) ([0-9a-f]{64})/.exec(prompt.slice(prompt.lastIndexOf('RESULT FORMAT')))!;
const text = mode === 'cut'
? `INPUT: ${m[1]} ${m[2]}\n\nA long review that ran out of output tokens before the findings fen`
: `INPUT: ${m[1]} ${m[2]}\n\nComplete review.\n\n\`\`\`gstack-findings\n{"id":"F1","severity":"Medium","title":"Name the retry metric","fix":"add retries_total"}\n\`\`\`\n`;
if (new URL(req.url).pathname.endsWith('/messages')) return Response.json({ model: body.model, stop_reason: mode === 'cut' ? 'max_tokens' : 'end_turn', content: [{ type: 'text', text: mode === 'bad-native' ? `INPUT: ${m[1]} ${m[2]}\n\nA review with no findings fence.` : text }], usage: { input_tokens: 10, output_tokens: 20 } });
openaiLimits.push(body.max_completion_tokens);
return Response.json({ model: body.model, choices: [{ finish_reason: mode === 'cut' ? 'length' : 'stop', message: { content: mode === 'bad-outside' ? `INPUT: ${m[1]} ${m[2]}\n\nA review with no findings fence.` : text } }], usage: { prompt_tokens: 10, completion_tokens: 20 } });
},
});
});
afterAll(() => { server.stop(true); fs.rmSync(TMP, { recursive: true, force: true }); });
async function run(name: string, extra: string[] = [], keys: Record<string, string> = { ANTHROPIC_API_KEY: 'ak-test', OPENAI_API_KEY: 'sk-test' }, nativeModel = 'anthropic/claude-opus-5-5') {
const dir = path.join(TMP, name);
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, 'plan.md'), SMALL_PLAN);
const out = path.join(dir, 'run');
const base = `http://127.0.0.1:${server.port}`;
const proc = Bun.spawn(['bun', AUTOPLAN, 'run', '--out', out, '--plan', path.join(dir, 'plan.md'), '--no-ui', '--no-dx', '--native-model', nativeModel, '--outside-model', 'openai/gpt-6-astra', '--runner', 'api', '--native-runner', 'api', ...extra], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
env: env({ ANTHROPIC_API_KEY: '', OPENAI_API_KEY: '', ...keys, ANTHROPIC_BASE_URL: base, OPENAI_BASE_URL: `${base}/v1`, GSTACK_AUTOPLAN_CHECKPOINT_DIR: '', GSTACK_STATE_ROOT: path.join(TMP, 'state') }),
});
const timer = setTimeout(() => proc.kill('SIGKILL'), 120_000);
const [status, stdout, stderr] = await Promise.all([proc.exited, new Response(proc.stdout).text(), new Response(proc.stderr).text()]);
clearTimeout(timer);
return { status, stdout, stderr, out, state: fs.existsSync(path.join(out, 'run.json')) ? JSON.parse(fs.readFileSync(path.join(out, 'run.json'), 'utf8')) : null };
}
describe('gstack-autoplan run --native-runner api --runner api', () => {
test('a complete result binds both voices in every phase and the run reaches its gate', async () => {
mode = 'ok';
const r = await run('ok');
expect(r.status, r.stderr).toBe(0);
expect(r.stdout).toContain(`GSTACK_RESULT: skill=autoplan status=gate_pending run=${r.out}`);
for (const phase of ['ceo', 'eng']) for (const voice of ['native', 'outside']) expect(r.state.phases[phase][voice]).toMatchObject({ status: 'completed', findings: 1 });
}, 150_000);
test('a spend cap binds api attempts at their priced worst case and settles them at their usage', async () => {
mode = 'ok';
openaiLimits = [];
const r = await run('capped', ['--spend-cap', '8']);
expect(r.status, r.stderr).toBe(0);
expect(r.stdout).toContain(`GSTACK_RESULT: skill=autoplan status=gate_pending run=${r.out}`);
expect(r.stdout).not.toMatch(/^ESTIMATE:/m);
const ledger = JSON.parse(fs.readFileSync(path.join(r.out, 'spend.json'), 'utf8'));
expect(ledger.reserved).toEqual([]);
expect(ledger.settled.length).toBe(4);
for (const s of ledger.settled) expect(s.usd).toBeGreaterThan(0);
expect(openaiLimits.length).toBeGreaterThan(0);
for (const limit of openaiLimits) expect(limit).toBe(DEFAULT_MAX_TOKENS);
calls = 0;
const zero = await run('zero-cap', ['--spend-cap', '0']);
expect(zero.status).toBe(3);
expect(zero.stdout + zero.stderr).toContain('SPEND_CAP_EXCEEDED');
expect(calls).toBe(0);
}, 150_000);
test('an outside result with no findings fence leaves the run native-only instead of aborting it', async () => {
mode = 'bad-outside';
const r = await run('bad-outside');
expect(r.status, r.stderr).toBe(0);
expect(r.state.status).toBe('gate_pending');
expect(r.state.phases.ceo.outside.status).toBe('unavailable');
expect(r.state.phases.ceo.native).toMatchObject({ status: 'completed', findings: 1 });
}, 150_000);
test('--max-tokens needs a positive integer', async () => {
const r = await run('max0', ['--max-tokens', '0']);
expect(r.status).toBe(2);
expect(r.stderr).toContain('--max-tokens needs a positive integer');
expect(r.state).toBeNull();
}, 30_000);
test('a response stopped at the output limit is named truncated and interrupts the run', async () => {
mode = 'cut';
const r = await run('cut');
expect(r.status).toBe(1);
expect(r.stderr).toMatch(/native voice unavailable: truncated: .*output limit/);
expect(r.state.status).toBe('interrupted');
}, 150_000);
test('a truncated call settles what it cost, and a retry with a larger output limit is admitted afresh against the cap', async () => {
mode = 'cut';
const first = await run('cut-retry', ['--spend-cap', '8']);
expect(first.state.status).toBe('interrupted');
const truncated = first.state.phases.ceo.native.attempts[0].attempt;
const ledger = JSON.parse(fs.readFileSync(path.join(first.out, 'spend.json'), 'utf8'));
expect(ledger.reserved.map((r: any) => r.attempt)).not.toContain(truncated);
expect(ledger.settled.find((s: any) => s.attempt === truncated).usd).toBeGreaterThan(0);
mode = 'ok';
calls = 0;
const retry = await run('cut-retry', ['--spend-cap', '8', '--max-tokens', '2000000']);
expect(retry.status).toBe(3);
expect(retry.stdout + retry.stderr).toContain('SPEND_CAP_EXCEEDED');
expect(calls).toBe(0);
}, 150_000);
test('a call refused before any request leaves charges nothing: with no key the attempt stays open and the rerun reuses it at the cap that admitted it', async () => {
mode = 'ok';
calls = 0;
const first = await run('no-key', ['--spend-cap', '8'], { OPENAI_API_KEY: 'sk-test' });
expect(first.state.status).toBe('interrupted');
expect(calls).toBe(0);
const open = first.state.phases.ceo.native.attempts[0];
expect(open.state).toBe('dispatched');
expect(JSON.parse(fs.readFileSync(path.join(first.out, 'spend.json'), 'utf8')).settled).toEqual([]);
const retry = await run('no-key', ['--spend-cap', '8']);
expect(retry.status, retry.stderr).toBe(0);
expect(retry.stdout).toContain(`GSTACK_RESULT: skill=autoplan status=gate_pending run=${retry.out}`);
expect(retry.state.phases.ceo.native.attempts.map((a: any) => a.attempt)).toEqual([open.attempt]);
const ledger = JSON.parse(fs.readFileSync(path.join(retry.out, 'spend.json'), 'utf8'));
expect(ledger.settled.length).toBe(4);
expect(ledger.released ?? []).toEqual([]);
for (const s of ledger.settled) expect(s.label).toMatch(/^(ceo|eng)\/(native|outside)$/);
}, 150_000);
test('a call that was sent and timed out with no usage is charged its reservation, so the cap binds on the rerun', async () => {
mode = 'hang';
const first = await run('timed-out', ['--spend-cap', '8', '--deadline', '0.02']);
expect(first.state.status).toBe('interrupted');
const sent = first.state.phases.ceo.native.attempts[0];
expect(sent.state).toBe('superseded');
const ledger = JSON.parse(fs.readFileSync(path.join(first.out, 'spend.json'), 'utf8'));
expect(ledger.reserved.map((r: any) => r.attempt)).not.toContain(sent.attempt);
expect(ledger.settled.find((s: any) => s.attempt === sent.attempt).usd).toBe('unknown');
}, 150_000);
test('a sent call the provider refused with a 503 is charged its reservation', async () => {
mode = 'down';
const r = await run('refused-503', ['--spend-cap', '8']);
expect(r.state.status).toBe('interrupted');
const sent = r.state.phases.ceo.native.attempts[0];
expect(sent.state).toBe('superseded');
const ledger = JSON.parse(fs.readFileSync(path.join(r.out, 'spend.json'), 'utf8'));
expect(ledger.reserved.map((x: any) => x.attempt)).not.toContain(sent.attempt);
expect(ledger.settled.find((s: any) => s.attempt === sent.attempt).usd).toBe('unknown');
}, 150_000);
test('a paid native call whose result submit refuses is settled at its usage, so the rerun is admitted afresh', async () => {
mode = 'bad-native';
const r = await run('refused-result', ['--spend-cap', '8']);
expect(r.status).not.toBe(0);
const sent = r.state.phases.ceo.native.attempts[0];
expect(sent.state).toBe('superseded');
const ledger = JSON.parse(fs.readFileSync(path.join(r.out, 'spend.json'), 'utf8'));
expect(ledger.reserved.map((x: any) => x.attempt)).not.toContain(sent.attempt);
expect(ledger.settled.find((s: any) => s.attempt === sent.attempt).usd).toBeGreaterThan(0);
}, 150_000);
test('a call in flight when the process died is charged its reservation on the next run, which dispatches a fresh attempt', async () => {
mode = 'hang';
calls = 0;
const dir = path.join(TMP, 'killed');
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, 'plan.md'), SMALL_PLAN);
const out = path.join(dir, 'run');
const base = `http://127.0.0.1:${server.port}`;
const proc = Bun.spawn(['bun', AUTOPLAN, 'run', '--out', out, '--plan', path.join(dir, 'plan.md'), '--no-ui', '--no-dx', '--native-model', 'anthropic/claude-opus-5-5', '--outside-model', 'openai/gpt-6-astra', '--runner', 'api', '--native-runner', 'api', '--spend-cap', '8'], {
cwd: ROOT, stdout: 'ignore', stderr: 'ignore',
env: env({ ANTHROPIC_API_KEY: 'ak-test', OPENAI_API_KEY: 'sk-test', ANTHROPIC_BASE_URL: base, OPENAI_BASE_URL: `${base}/v1`, GSTACK_AUTOPLAN_CHECKPOINT_DIR: '', GSTACK_STATE_ROOT: path.join(TMP, 'state') }),
});
for (let i = 0; i < 200 && calls === 0; i++) await Bun.sleep(50);
expect(calls).toBe(1);
proc.kill('SIGKILL');
await proc.exited;
mode = 'ok';
const retry = await run('killed', ['--spend-cap', '8']);
expect(retry.status, retry.stderr).toBe(0);
const [lost, fresh] = retry.state.phases.ceo.native.attempts;
expect(lost.state).toBe('superseded');
expect(fresh.state).toBe('submitted');
const ledger = JSON.parse(fs.readFileSync(path.join(retry.out, 'spend.json'), 'utf8'));
expect(ledger.settled.find((s: any) => s.attempt === lost.attempt).usd).toBe('unknown');
}, 150_000);
test('with a spend cap an unpriced api model reserves the default in any session kind, so a cap below it refuses before any call', async () => {
mode = 'ok';
calls = 0;
const kind = ['GSTACK', 'SESSION', 'KIND'].join('_');
const r = await run('unpriced', ['--spend-cap', '1'], { ANTHROPIC_API_KEY: 'ak-test', OPENAI_API_KEY: 'sk-test', [kind]: 'spawned' }, 'anthropic/claude-haiku-5-5');
expect(r.status).toBe(3);
expect(r.stdout + r.stderr).toContain('SPEND_CAP_EXCEEDED');
expect(calls).toBe(0);
}, 150_000);
});
+2 -1
View File
@@ -106,7 +106,8 @@ describe('next/submit loop', () => {
expect(v.status, v.stdout + v.stderr).toBe(0);
expect(JSON.parse(v.stdout)).toMatchObject({ valid: true, status: 'gate_pending' });
const gate = JSON.parse(fs.readFileSync(path.join(out, 'gate.json'), 'utf8'));
expect(gate.items.map((i: any) => i.id)).toEqual(['uc1', 'uc2', 'p1']);
expect(gate.items.map((i: any) => i.id)).toEqual(['uc1', 'd1', 'd2', 'uc2', 'd3', 'd4', 'p1']);
expect(gate.items.filter((i: any) => i.finding).map((i: any) => i.title.split(':')[0])).toEqual(['Critical', 'High', 'Critical', 'High']);
const decisions = fs.readFileSync(path.join(out, 'decisions.jsonl'), 'utf8').trim().split('\n').map(l => JSON.parse(l));
expect(decisions.every((d: any) => d.status === 'pending' && d.gate_rev === 1)).toBe(true);
const tasks = fs.readFileSync(path.join(out, 'tasks.jsonl'), 'utf8').trim().split('\n').map(l => JSON.parse(l));
+4 -3
View File
@@ -11,7 +11,7 @@ import { afterAll, describe, expect, test } from 'bun:test';
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { ROOT, resultOf } from './helpers/autoplan-run-fixture';
import { ROOT, recCarrying, resultOf } from './helpers/autoplan-run-fixture';
import { capyRunner, phaseVia, writePlan } from './helpers/autoplan-capy';
const r = capyRunner();
@@ -69,7 +69,8 @@ describe('Capy Captain flow', () => {
expect(j.brief).toStartWith('**Decision needed:** approve the plan "Retry backoff for the webhook worker".');
for (const h of ['### Before → after', '### Why it matters', '### What caused this work', '### Big decisions', '### What we need from you']) expect(j.brief).toContain(h);
const answer = r.ap(['answer', '--out', out, '--gate-rev', String(j.gate_rev), '--reply', 'rec', '--rec-hash', j.rec_hash, '--by', 'agent:GSTA-1', '--owner-reply', 'rec']);
const reply = recCarrying(out, 'rec');
const answer = r.ap(['answer', '--out', out, '--gate-rev', String(j.gate_rev), '--reply', reply, '--rec-hash', j.rec_hash, '--by', 'agent:GSTA-1', '--owner-reply', reply]);
expect(answer.status, answer.stderr + answer.stdout).toBe(0);
expect(answer.stdout).toMatch(/^APPROVED: gate_rev=1/m);
@@ -84,7 +85,7 @@ describe('Capy Captain flow', () => {
expect(bin('gstack-approvals', ['import', '--autoplan', out, '--by', 'Garry'], env, repo).stdout).toMatch(/^APPROVAL_IMPORTED: n=0 skipped=1 /);
const table = bin('gstack-approvals', ['render'], env, repo).stdout;
expect(table).toContain('Approve the plan "Retry backoff for the webhook worker"');
expect(table).toContain('“rec” (Garry, relayed by agent:GSTA-1) | approved |');
expect(table).toContain(`“${reply}” (Garry, relayed by agent:GSTA-1) | approved |`);
expect(fs.readdirSync(path.join(mirror, 'acme-worker')).length).toBe(1);
});
+2 -2
View File
@@ -21,8 +21,8 @@ describe('dx-error-contract-review grader (free controls for the paid case)', ()
});
test('the grader keys on the title and severity shapes the product prompt asks for', () => {
expect(DX_ERROR_COMPACT).toContain('as a Critical finding titled "CAP4:<path>:<item>"');
expect(DX_ERROR_COMPACT).toContain('as a High finding titled "CAP6:<path>:<item>"');
expect(DX_ERROR_COMPACT).toContain('as a Critical finding titled "CAP4:<path>:<item> <what fails>"');
expect(DX_ERROR_COMPACT).toContain('as a High finding titled "CAP6:<path>:<item> <what fails>"');
const rows = DX_ERROR_ITEMS.map((item, index) => ({ id: `F${index}`, severity: 'High', title: `CAP6:P1:${item} gap` }));
expect(capFindings(fence(rows)).map(cap => cap.item)).toEqual([...DX_ERROR_ITEMS]);
});
+2
View File
@@ -20,6 +20,8 @@ describe('agent-ready error contract reaches every DX reviewer', () => {
for (const text of [DX_ERROR_CONTRACT, DX_ERROR_COMPACT]) {
expect(text).toContain('CAP4:<path>:<item>');
expect(text).toContain('CAP6:<path>:<item>');
expect(text).toContain('CAP4:<path>:<item> <what fails>');
expect(text).toContain('CAP6:<path>:<item> <what fails>');
expect(text).toContain('4/10');
expect(text).toContain('6/10');
expect(text).toMatch(/[Uu]sage (and|or) validation error/);
+1 -1
View File
@@ -141,7 +141,7 @@ test('live periodic census fits the declared CI wall including setup', () => {
test('registered allocation is deterministic and preserves every discovered file', () => {
const files = collectPaidTestFiles();
expect(files).toHaveLength(114); // + skill-e2e-planning-wave (GSTA-228); + skill-e2e-model-panel (GSTA-173); + skill-e2e-write-skill (GSTA-133); + skill-e2e-ship-frozen-scope (frozen review scope, GSTA-113 PR 2); + skill-e2e-failure-routing (#3157/#3158); + skill-e2e-dx-error-contract (#3226); + skill-e2e-gate-integrity (HONEST_WORK_GATE_INTEGRITY W8); + skill-e2e-autoplan-schema-canary, -guard-pty and -long-session (Oct 7 wave, CEO-13, ENG-18, CEO-15); + skill-e2e-ship-measure-seeded-flake (v1.91.32.0); W5a: 15 carve wrappers became one case-sharded file; + codex-e2e-multiblock-live and skill-e2e-design-model-smoke (v1.91.30.0);
expect(files).toHaveLength(115); // + skill-e2e-autoplan-ceo-only (GST-537); + skill-e2e-planning-wave (GSTA-228); + skill-e2e-model-panel (GSTA-173); + skill-e2e-write-skill (GSTA-133); + skill-e2e-ship-frozen-scope (frozen review scope, GSTA-113 PR 2); + skill-e2e-failure-routing (#3157/#3158); + skill-e2e-dx-error-contract (#3226); + skill-e2e-gate-integrity (HONEST_WORK_GATE_INTEGRITY W8); + skill-e2e-autoplan-schema-canary, -guard-pty and -long-session (Oct 7 wave, CEO-13, ENG-18, CEO-15); + skill-e2e-ship-measure-seeded-flake (v1.91.32.0); W5a: 15 carve wrappers became one case-sharded file; + codex-e2e-multiblock-live and skill-e2e-design-model-smoke (v1.91.30.0);
expect(files).toContain('test/skill-e2e-ship-skip.test.ts');
const m = livePlan(files);
expect(livePlan([...files].reverse())).toEqual(m);
+1 -1
View File
@@ -25,7 +25,7 @@
"diagram": 4286,
"document-generate": 6013,
"document-release": 4454,
"eval-plan": 4887,
"eval-plan": 5021,
"freeze": 1115,
"gstack": 5350,
"gstack-upgrade": 6297,
+9 -9
View File
@@ -4,53 +4,53 @@
"pins": {
"agy/gstack-autoplan": 109666,
"agy/gstack-plan-design-review": 109116,
"agy/gstack-plan-devex-review": 114958,
"agy/gstack-plan-devex-review": 114957,
"agy/gstack-plan-eng-review": 143590,
"agy/gstack-review": 104276,
"codex/gstack-autoplan": 108409,
"codex/gstack-plan-design-review": 110473,
"codex/gstack-plan-devex-review": 116466,
"codex/gstack-plan-devex-review": 116465,
"codex/gstack-plan-eng-review": 144755,
"codex/gstack-review": 103799,
"copilot/gstack-autoplan": 109109,
"copilot/gstack-plan-design-review": 109382,
"copilot/gstack-plan-devex-review": 114420,
"copilot/gstack-plan-devex-review": 114419,
"copilot/gstack-plan-eng-review": 143755,
"copilot/gstack-review": 103857,
"cursor/gstack-autoplan": 108341,
"cursor/gstack-plan-design-review": 107924,
"cursor/gstack-plan-devex-review": 113690,
"cursor/gstack-plan-devex-review": 113689,
"cursor/gstack-plan-eng-review": 142345,
"cursor/gstack-review": 122147,
"factory/gstack-autoplan": 110619,
"factory/gstack-plan-design-review": 111022,
"factory/gstack-plan-devex-review": 117053,
"factory/gstack-plan-devex-review": 117052,
"factory/gstack-plan-eng-review": 145453,
"factory/gstack-review": 125920,
"gbrain/gstack-autoplan": 108748,
"gbrain/gstack-plan-eng-review": 124972,
"gemini/gstack-autoplan": 108899,
"gemini/gstack-plan-design-review": 109210,
"gemini/gstack-plan-devex-review": 114248,
"gemini/gstack-plan-devex-review": 114247,
"gemini/gstack-plan-eng-review": 143588,
"gemini/gstack-review": 103679,
"hermes/gstack-autoplan": 108751,
"hermes/gstack-plan-eng-review": 125008,
"kiro/gstack-autoplan": 108115,
"kiro/gstack-plan-design-review": 107770,
"kiro/gstack-plan-devex-review": 113542,
"kiro/gstack-plan-devex-review": 113541,
"kiro/gstack-plan-eng-review": 142207,
"kiro/gstack-review": 121971,
"openclaw/gstack-autoplan": 108571,
"openclaw/gstack-plan-eng-review": 123487,
"opencode/gstack-autoplan": 109090,
"opencode/gstack-plan-design-review": 109238,
"opencode/gstack-plan-devex-review": 114298,
"opencode/gstack-plan-devex-review": 114297,
"opencode/gstack-plan-eng-review": 143614,
"opencode/gstack-review": 122769,
"slate/gstack-autoplan": 108228,
"slate/gstack-plan-design-review": 107847,
"slate/gstack-plan-devex-review": 113616,
"slate/gstack-plan-devex-review": 113615,
"slate/gstack-plan-eng-review": 142276,
"slate/gstack-review": 122059
}
@@ -0,0 +1,96 @@
/**
* Evidence for the /autoplan --ceo-only paid case (test/skill-e2e-autoplan-ceo-only.test.ts):
* the run asked nothing, reviewed the plan through the CEO phase, skipped every
* later phase, wrote ceo-review.md, ended status=reviewed, and wrote nothing in
* HOME outside its declared roots. Pure over the transcript and files the test
* collects, so test/autoplan-ceo-only.test.ts proves each check fails on a
* synthetic negative control before any paid run.
*/
import * as fs from 'node:fs';
import * as path from 'node:path';
export const CEO_REVIEW_HEADINGS = [
'## User challenges (direction kept)', '## Scope added', '## Conservative choices', '## Findings', '## Decision Audit Trail',
] as const;
const QUESTION_TOOL = /(^|__)(AskUserQuestion|ExitPlanMode)$/;
const PROSE_QUESTION = /\b(would you like|do you want|should i\b|want me to|shall i\b|which (?:option|mode|one) (?:do|would|should) you|reply with|let me know (?:if|which|whether))/i;
const LATER_PHASE = /autoplan\/sections\/(?:design-phase|dx-phase|eng-phase|tasks-aggregator)\.md/;
const DISPATCH_TOOL = /^(Agent|Task)$/;
export interface CeoOnlyEvidenceInput {
transcript: any[];
/** The session's final result text. */
output: string;
/** New files under HOME outside the declared roots (the test's before/after diff). */
strayWrites: string[];
/** Patterns the reviewed plan or ceo-review.md must contain for the seeded flaw. */
mustMention: RegExp[];
}
export interface CeoOnlyEvidence { passed: boolean; reasons: string[]; runDir?: string; questions: string[] }
function blocks(transcript: any[]): Array<{ role: string; block: any }> {
return transcript.flatMap(event => {
const role = event?.type === 'assistant' ? 'assistant' : event?.type === 'user' ? 'user' : '';
const content = event?.message?.content;
return role && Array.isArray(content) ? content.map((block: any) => ({ role, block })) : [];
});
}
export function ceoOnlyEvidence(input: CeoOnlyEvidenceInput): CeoOnlyEvidence {
const reasons: string[] = [];
const all = blocks(input.transcript);
const uses = all.filter(b => b.role === 'assistant' && b.block?.type === 'tool_use').map(b => b.block);
const texts = all.filter(b => b.role === 'assistant' && b.block?.type === 'text').map(b => String(b.block.text ?? ''));
const results = all.filter(b => b.role === 'user' && b.block?.type === 'tool_result')
.map(b => (typeof b.block.content === 'string' ? b.block.content : JSON.stringify(b.block.content ?? '')));
const questions = [
...uses.filter(u => QUESTION_TOOL.test(String(u.name))).map(u => `tool ${u.name}`),
...[...texts, input.output].flatMap(t => t.split('\n').filter(line => line.trim().endsWith('?') && PROSE_QUESTION.test(line)).map(line => `prose: ${line.trim().slice(0, 160)}`)),
];
if (questions.length) reasons.push(`asked ${questions.length} question(s): ${questions.join(' | ')}`);
const handoff = uses.some(u => u.name === 'Bash' && /gstack-ceo-mode-handoff\b/.test(String(u.input?.command)) && /--ceo-only\b/.test(String(u.input?.command)))
&& results.some(r => r.includes('(CEO-only run: Step 0E'));
if (!handoff) reasons.push('no gstack-ceo-mode-handoff --ceo-only call with its CEO-only line');
if (!uses.some(u => DISPATCH_TOOL.test(String(u.name)))) reasons.push('no CEO reviewer subagent was dispatched');
const later = uses.map(u => String(u.name === 'Bash' ? u.input?.command : (u.input?.file_path ?? u.input?.path ?? u.input?.pattern ?? '')))
.filter(arg => LATER_PHASE.test(arg));
if (later.length) reasons.push(`loaded a skipped phase: ${later.map(arg => arg.slice(0, 160)).join(', ')}`);
const result = /^GSTACK_RESULT: skill=autoplan status=(\S+) run=(.+)$/m.exec(input.output.split('\n').filter(l => l.startsWith('GSTACK_RESULT: ')).pop() ?? '');
if (!result) reasons.push('no GSTACK_RESULT line for autoplan in the final output');
else if (result[1] !== 'reviewed') reasons.push(`GSTACK_RESULT status=${result[1]}, expected reviewed`);
const runDir = result?.[2]?.trim();
const reviewFile = runDir ? path.join(runDir, 'ceo-review.md') : '';
const review = reviewFile && fs.existsSync(reviewFile) ? fs.readFileSync(reviewFile, 'utf8') : undefined;
if (runDir && review === undefined) reasons.push(`${reviewFile} is missing`);
if (review !== undefined) {
const missing = CEO_REVIEW_HEADINGS.filter(h => !review.includes(h));
if (missing.length) reasons.push(`ceo-review.md lacks ${missing.join(', ')}`);
if (!/\|\s*\d+\s*\|/.test(review.slice(review.indexOf('## Decision Audit Trail')))) reasons.push('ceo-review.md has no Decision Audit Trail rows');
const unmet = input.mustMention.filter(re => !re.test(review));
if (unmet.length) reasons.push(`ceo-review.md never names the seeded flaw (${unmet.map(String).join(', ')})`);
}
if (input.strayWrites.length) {
const dirs = [...new Set(input.strayWrites.map(f => f.split('/').slice(0, 2).join('/')))];
reasons.push(`wrote ${input.strayWrites.length} file(s) outside its roots, under ${dirs.join(', ')}: ${input.strayWrites.slice(0, 5).join(', ')}`);
}
return { passed: reasons.length === 0, reasons, runDir, questions };
}
/** Every file under root, relative, skipping the given relative prefixes. */
export function listFiles(root: string, skip: string[] = []): string[] {
const out: string[] = [];
const walk = (dir: string) => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const abs = path.join(dir, entry.name);
const rel = path.relative(root, abs);
if (skip.some(s => rel === s || rel.startsWith(s + path.sep))) continue;
if (entry.isDirectory()) walk(abs); else out.push(rel);
}
};
if (fs.existsSync(root)) walk(root);
return out.sort();
}
+10
View File
@@ -58,6 +58,16 @@ export function resultOf(out: string, phase: string, voice: 'native' | 'outside'
}
/** The sealed copy a completed voice is bound to. */
export const boundOf = (out: string, phase: string, voice: 'native' | 'outside'): string => path.join(out, readRun(out).phases[phase][voice].output);
/**
* A `rec`/`all` reply that carries every open Critical/High finding item into the build (`<id>b`, no plan change),
* so a test about another gate mechanism keeps its flow; `extra` tokens are appended to the except list.
*/
export function recCarrying(out: string, base = 'all', extra: string[] = []): string {
const carry = (JSON.parse(fs.readFileSync(path.join(out, 'gate.json'), 'utf8')).items as Array<{ id: string; finding?: string[] }>).filter(i => i.finding).map(i => `${i.id}b`);
const tokens = [...carry, ...extra];
return tokens.length ? `${base} except ${tokens.join(' ')}` : base;
}
export function attemptsOf(out: string, phase: string): { native: string; outside: string } {
const run = readRun(out);
return { native: run.phases[phase].native.attempt, outside: run.phases[phase].outside.attempt };
+1 -1
View File
@@ -583,7 +583,7 @@ do not launch the downstream skill or open a browser.`,
maxSkeletonBytes: 48_175, // GSTA-96 hoist (2026-10-26): lowered from 70_000 by the shared preamble that moved to sections/preamble.md, keeping the pre-hoist headroom of 4,888 bytes; measured 43,287. Before: Phase-specific outside coverage, native fallback, and harness guard.
minUnionBytes: 85_000, // measured union 86,926
mustContain: ['6 Decision Principles', 'TASTE DECISION', 'USER CHALLENGE', 'consensus', 'Restore Point'],
maxSizeRatio: 1.1511, // Four validated outside invocations replace raw CLI calls; phases keep independent coverage. + Oct 7 wave B3: the Codex probe runs as a command (select-model capture lines, check-sandbox/probe-model/run-with-timeout subcommands) instead of being sourced; measured 114,297 / 101,979 = 1.1208 (2026-10-07). + Oct 7 wave CEO-28/DX-11: review prompts ask for a severity per finding and design proposals use the proposal gate; measured 114,533 / 101,979 = 1.1231 (2026-10-07). + Oct 7 wave guard (CEO-20): conditional `run_in_background` request plus the background-run recovery at each foreground dispatch site, and phase-close publishing each report with the `true autoplan-published <phase>` no-op before the next phase entry in a later message (CEO-1); measured 114,320 / 101,979 = 1.1210 (2026-10-07). Merged with the guard area: measured 115,149 / 101,979 = 1.1291 (2026-10-07). + Oct 8 wave (permission cards after a background reviewer): Step 1 puts RESTORE_PATH, and so every phase artifact, in the project's git-excluded `.gstack/tmp/autoplan/`, which Claude Code reads without a card after the skill turn ends; measured 115,270 / 101,979 = 1.1303 (2026-10-08). + fix wave PR D: the prerequisite offer is skipped in spawned/headless sessions (#1958); measured 115,267 / 101,979 = 1.1303 (2026-10-07). Both merged: measured 115,126 / 101,979 = 1.1289 (2026-10-08). + Honest Work wave merged (W2/W3 gate integrity in review/ship, W4 kernel + DONE line in every tier-2+ skill, W5/W7 in spec/ship): re-measured on the integrated tree (2026-10-10). + Oct 10 multi-agent wave B9/B10/B12: Phase 0 ESTIMATE and the phase-close timing record, the final gate as a gate list (`gstack-gate render/parse`, stable ids, gate_rev) and the URGENT block above the summary, offset by tightening the Phase 4 prose and the tasks-section; measured 114,980 / 101,979 = 1.1275 (2026-10-10). + Oct 10 multi-agent wave PR D1 (reality rows): the reality-rows check line in the CEO and Eng phase files, the Pre-Gate rule (unattended miss is `incomplete`, never a warning) and the two Pre-Gate cells; measured 117,042 / 101,979 = 1.1477 (2026-10-10). + GSTA-96 shared-preamble hoist (owner-approved raise, Garry 2026-10-25: "do it"): the union now holds sections/preamble.md, so a fresh invocation reads the pointer plus the shared file; measured 117,388 / 101,979 = 1.1511 (2026-10-25). Before: 1.149.
maxSizeRatio: 1.1723, // GST-537 /autoplan --ceo-only (GStack Minder approved A1, 2026-10-31): sections/ceo-only.md (2,168 B) plus its Section index row, read only when the invocation names --ceo-only (test/autoplan-ceo-only.test.ts pins that a normal run never reads it); measured 119,545 / 101,979 = 1.1723 (2026-10-31), +2,276 B over main's 117,269 = 1.1499. Before: 1.1511. Four validated outside invocations replace raw CLI calls; phases keep independent coverage. + Oct 7 wave B3: the Codex probe runs as a command (select-model capture lines, check-sandbox/probe-model/run-with-timeout subcommands) instead of being sourced; measured 114,297 / 101,979 = 1.1208 (2026-10-07). + Oct 7 wave CEO-28/DX-11: review prompts ask for a severity per finding and design proposals use the proposal gate; measured 114,533 / 101,979 = 1.1231 (2026-10-07). + Oct 7 wave guard (CEO-20): conditional `run_in_background` request plus the background-run recovery at each foreground dispatch site, and phase-close publishing each report with the `true autoplan-published <phase>` no-op before the next phase entry in a later message (CEO-1); measured 114,320 / 101,979 = 1.1210 (2026-10-07). Merged with the guard area: measured 115,149 / 101,979 = 1.1291 (2026-10-07). + Oct 8 wave (permission cards after a background reviewer): Step 1 puts RESTORE_PATH, and so every phase artifact, in the project's git-excluded `.gstack/tmp/autoplan/`, which Claude Code reads without a card after the skill turn ends; measured 115,270 / 101,979 = 1.1303 (2026-10-08). + fix wave PR D: the prerequisite offer is skipped in spawned/headless sessions (#1958); measured 115,267 / 101,979 = 1.1303 (2026-10-07). Both merged: measured 115,126 / 101,979 = 1.1289 (2026-10-08). + Honest Work wave merged (W2/W3 gate integrity in review/ship, W4 kernel + DONE line in every tier-2+ skill, W5/W7 in spec/ship): re-measured on the integrated tree (2026-10-10). + Oct 10 multi-agent wave B9/B10/B12: Phase 0 ESTIMATE and the phase-close timing record, the final gate as a gate list (`gstack-gate render/parse`, stable ids, gate_rev) and the URGENT block above the summary, offset by tightening the Phase 4 prose and the tasks-section; measured 114,980 / 101,979 = 1.1275 (2026-10-10). + Oct 10 multi-agent wave PR D1 (reality rows): the reality-rows check line in the CEO and Eng phase files, the Pre-Gate rule (unattended miss is `incomplete`, never a warning) and the two Pre-Gate cells; measured 117,042 / 101,979 = 1.1477 (2026-10-10). + GSTA-96 shared-preamble hoist (owner-approved raise, Garry 2026-10-25: "do it"): the union now holds sections/preamble.md, so a fresh invocation reads the pointer plus the shared file; measured 117,388 / 101,979 = 1.1511 (2026-10-25). Before: 1.149.
},
spec: {
skill: 'spec',
+8
View File
@@ -918,6 +918,12 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
'test/fixtures/autoplan-phase-handoff-6714.json','scripts/resolvers/learnings.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/preamble/generate-preamble-bash.ts', 'test/fixtures/plan-scope-recovery-av.json', 'test/fixtures/design-scope-checkpoint-at.json', 'scripts/resolvers/composition.ts', 'autoplan/**', 'lib/claude-journal-records.ts', 'lib/claude-owned-journal.ts', 'codex/**', 'bin/gstack-codex-probe', 'scripts/resolvers/review-dashboard.ts', 'scripts/resolvers/plan-gates.ts', 'scripts/resolvers/spec-review.ts', 'scripts/resolvers/outside-voice-steps.ts', 'scripts/resolvers/review-scope.ts', 'scripts/resolvers/outside-voice.ts', 'scripts/resolvers/design.ts', 'test/skill-e2e-autoplan-dual-voice.test.ts', 'bin/gstack-autoplan-snapshot.ts', 'test/fixtures/autoplan/t-ceo-omitted-obligations.json', 'test/fixtures/autoplan/u-ceo-original-loss.json', 'test/fixtures/autoplan/v-ceo-dangling-references.json',
'scripts/resolvers/design-doc-discovery.ts', 'bin/gstack-design-doc-find', 'bin/gstack-ceo-plan-find', 'plan-ceo-review/**', 'ETHOS.md', 'plan-eng-review/**', 'plan-design-review/**', 'plan-devex-review/**', 'scripts/resolvers/testing.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/tasks-section.ts', 'test/helpers/autoplan-phase-observer.ts', 'scripts/resolve-codex-generation-model.ts', 'scripts/resolvers/constants.ts'],
// /autoplan --ceo-only: zero questions through the CEO phase (GST-537)
'autoplan-ceo-only': ['autoplan/**', 'plan-ceo-review/**', 'ETHOS.md', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble-hoist.ts',
'scripts/resolvers/preamble/generate-ask-user-format.ts', 'bin/gstack-state-root.sh', 'lib/state-root.ts',
'lib/headless-artifacts.ts', 'bin/gstack-ceo-mode-handoff', 'test/helpers/autoplan-ceo-only-evidence.ts', 'test/helpers/hermetic-env.ts',
'test/skill-e2e-autoplan-ceo-only.test.ts', 'test/fixtures/forcing-finding-seeds.ts'],
// Multi-provider benchmark adapters — live API smoke against real claude/codex/gemini CLIs
'benchmark-providers-live': ['bin/gstack-model-benchmark', 'test/helpers/providers/**', 'test/helpers/benchmark-runner.ts', 'test/helpers/pricing.ts', 'test/skill-e2e-benchmark-providers.test.ts'],
@@ -1581,6 +1587,7 @@ export const E2E_TIERS: Record<string, 'gate' | 'periodic' | 'marathon'> = {
// Autoplan — periodic (not yet implemented)
'autoplan-dual-voice': 'periodic',
'autoplan-ceo-only': 'marathon', // Full /autoplan --ceo-only CEO review (349–984 s over 6 runs, GST-537); the gate (I5) and periodic lanes are at their caps
// Multi-provider benchmark — periodic (requires external CLIs + auth, paid)
'benchmark-providers-live': 'periodic',
@@ -1995,6 +2002,7 @@ export const E2E_KINDS: Record<string, 'rule' | 'behavior' | 'judge'> = {
'benchmark-workflow': 'rule',
'setup-deploy-workflow': 'rule',
'autoplan-dual-voice': 'rule',
'autoplan-ceo-only': 'rule',
'benchmark-providers-live': 'rule',
'scrape-match-path': 'behavior',
'scrape-prototype-path': 'behavior',
+43
View File
@@ -240,3 +240,46 @@ describe('api runner (supplied-input review)', () => {
expect(nokey.stderr).toMatch(/OPENAI_API_KEY is not set.*\(OUTSIDE_RUNNER_UNAVAILABLE\)$/m);
});
});
describe('api runner output limits (GBRA-223 claim 6)', () => {
let server: ReturnType<typeof Bun.serve>;
const bodies: Array<{ url: string; body: any }> = [];
beforeAll(() => {
server = Bun.serve({
port: 0,
async fetch(req) {
const url = new URL(req.url);
const body = await req.json();
bodies.push({ url: url.pathname, body });
const cut = String(body.model).includes('cut');
if (url.pathname === '/v1/messages') return Response.json({ model: body.model, stop_reason: cut ? 'max_tokens' : 'end_turn', content: [{ type: 'text', text: cut ? 'INPUT: ceo abc\n\nA long review cut off mid-sent' : REVIEW }], usage: { input_tokens: 10, output_tokens: body.max_tokens } });
return Response.json({ model: body.model, choices: [{ finish_reason: cut ? 'length' : 'stop', message: { content: cut ? 'INPUT: ceo abc\n\nA long review cut off' : REVIEW } }], usage: { prompt_tokens: 10, completion_tokens: 50 } });
},
});
});
afterAll(() => server.stop(true));
const anthropic = () => ({ ...cleanEnv(emptyBin), ANTHROPIC_API_KEY: 'ak-test', ANTHROPIC_BASE_URL: `http://127.0.0.1:${server.port}` });
const openai = () => ({ ...cleanEnv(emptyBin), OPENAI_API_KEY: 'sk-test', OPENAI_BASE_URL: `http://127.0.0.1:${server.port}/v1` });
const out = (name: string) => path.join(TMP, 'limits', name);
test('anthropic asks for 16384 output tokens by default and --max-tokens sets it; openai sends max_completion_tokens only when set', async () => {
expect((await runAsync(['run', '--runner', 'api', '--prompt', promptFile, '--out', out('a1.md'), '--model', 'claude-opus-5-5'], anthropic())).status).toBe(0);
expect((await runAsync(['run', '--runner', 'api', '--prompt', promptFile, '--out', out('a2.md'), '--model', 'claude-opus-5-5', '--max-tokens', '4000'], anthropic())).status).toBe(0);
expect(bodies.filter(b => b.url === '/v1/messages').map(b => b.body.max_tokens)).toEqual([16384, 4000]);
expect((await runAsync(['run', '--runner', 'api', '--prompt', promptFile, '--out', out('o1.md'), '--model', 'gpt-6-astra'], openai())).status).toBe(0);
expect((await runAsync(['run', '--runner', 'api', '--prompt', promptFile, '--out', out('o2.md'), '--model', 'gpt-6-astra', '--max-tokens', '4000'], openai())).status).toBe(0);
expect(bodies.filter(b => b.url === '/v1/chat/completions').map(b => b.body.max_completion_tokens ?? null)).toEqual([null, 4000]);
});
test('a response stopped at the output limit is truncated (exit 1), never a completed or missing-markers review', async () => {
for (const [model, env, tokens] of [['claude-cut', anthropic(), '16384'], ['gpt-cut', openai(), '50']] as const) {
const file = out(`${model}.md`);
const r = await runAsync(['run', '--runner', 'api', '--prompt', promptFile, '--out', file, '--model', model], env);
expect(r.status).toBe(1);
expect(r.stderr).toMatch(/api unavailable \(truncated\): .*output limit/);
const meta = JSON.parse(fs.readFileSync(`${file}.meta.json`, 'utf8'));
expect(meta).toMatchObject({ status: 'unavailable', reason: 'truncated' });
expect(meta.detail).toContain(tokens);
}
});
});
+102
View File
@@ -159,6 +159,108 @@ describe('priced pilot', () => {
expect(r.settled_unknown).toBe(1);
expect(r.projection.verdict).toBe('unknown');
});
test('#3653: each arm is projected on its own, so arms at different prices that fit the cap are not told they exceed it', () => {
const d = tmp();
const out = path.join(d, 'mixed');
const s = startPilot({ out, cap: 2, n: 3, arms: ['gpt-6-astra', 'gpt-5.4-mini'], dryRun: passingDryRun(d) });
expect(s.error).toBeUndefined();
const usd: Record<string, number[]> = { 'gpt-6-astra': [0.001, 0.001, 0.001], 'gpt-5.4-mini': [0.10, 0.11, 0.12] };
for (const a of s.record.attempts) settlePilot(out, a.attempt, usd[a.model]![a.row - 1]!);
const r = pilotReport(out, { total: 10, workers: 1 });
expect(r.projection.projected_usd).toBeCloseTo(1.11, 6);
expect(r.projection.band_usd![0]).toBeCloseTo(1.11 - 4.303 * 0.01 / Math.sqrt(3) * 10, 4);
expect(r.projection.band_usd![1]).toBeCloseTo(1.11 + 4.303 * 0.01 / Math.sqrt(3) * 10, 4);
expect(r.projection.verdict).toBe('within');
expect(r.projection.arms.map(a => [a.arm, a.n])).toEqual([['gpt-6-astra', 3], ['gpt-5.4-mini', 3]]);
expect(renderPilotReport(r)).toContain('PILOT_ARM: arm=gpt-5.4-mini n=3 per_item=$0.1100');
});
test('#3653: a priced arm mean is never multiplied across an arm settled unknown', () => {
const d = tmp();
const out = path.join(d, 'partial');
const s = startPilot({ out, cap: 10, n: 3, arms: ['gpt-6-astra', 'gpt-5.4-mini'], dryRun: passingDryRun(d) });
for (const a of s.record.attempts) settlePilot(out, a.attempt, a.model === 'gpt-6-astra' ? 0.0003 : 'unknown');
const r = pilotReport(out, { total: 400 });
expect(r.projection.projected_usd).toBeCloseTo(0.12, 6);
expect(r.projection.verdict).toBe('unknown');
expect(r.settled_unknown).toBe(3);
const text = renderPilotReport(r);
expect(text).toContain('known_arms=1/2');
expect(text).toContain('--seat-metered');
});
test('#3653: a declared seat-metered arm holds none of the cap, stays out of the verdict and gets its own line', () => {
const d = tmp();
const dry = passingDryRun(d);
const est = startPilot({ out: path.join(d, 'probe'), cap: 100, n: 1, arms: ['gpt-6-astra', 'gpt-5.4-mini'], dryRun: dry }).record.attempts;
const astra = est.find(a => a.model === 'gpt-6-astra')!.estimate_usd, mini = est.find(a => a.model === 'gpt-5.4-mini')!.estimate_usd;
const cap = Number((3 * mini + 2 * astra).toFixed(4));
expect(startPilot({ out: path.join(d, 'undeclared'), cap, n: 3, arms: ['gpt-5.4-mini', 'gpt-6-astra'], dryRun: dry }).error?.code).toBe('EVAL_BUDGET_EXHAUSTED');
const out = path.join(d, 'seat');
const s = startPilot({ out, cap, n: 3, arms: ['gpt-5.4-mini', 'gpt-6-astra'], seatMetered: ['gpt-6-astra'], dryRun: dry });
expect(s.error).toBeUndefined();
expect(s.record.seat_metered).toEqual(['gpt-6-astra']);
expect(s.record.attempts.filter(a => a.model === 'gpt-6-astra').map(a => a.estimate_usd)).toEqual([0, 0, 0]);
for (const a of s.record.attempts) settlePilot(out, a.attempt, a.model === 'gpt-6-astra' ? 'unknown' : 0.0003);
const r = pilotReport(out, { total: 400, workers: 3 });
expect(r.settled_unknown).toBe(0);
expect(r.spend.summary.unknown_usd).toBe(0);
expect(r.spend.max_estimate_usd).toBeCloseTo(mini, 6);
expect(r.projection.projected_usd).toBeCloseTo(0.12, 6);
expect(r.projection.verdict).toBe('within');
expect(r.seat_metered).toEqual([{ arm: 'gpt-6-astra', calls: 3, settled: 3 }]);
expect(renderPilotReport(r)).toContain('PILOT_SEAT_METERED: arm=gpt-6-astra calls=3 settled=3');
});
test('#3653: a charge on a seat-metered attempt still counts as spent; the declaration names listed arms and is fixed for its --out', () => {
const d = tmp();
const dry = passingDryRun(d);
const out = path.join(d, 'seat');
const s = startPilot({ out, cap: 10, n: 1, arms: ['gpt-6-astra'], seatMetered: ['gpt-6-astra'], dryRun: dry });
settlePilot(out, s.record.attempts[0]!.attempt, 0.05);
const charged = pilotReport(out, { total: 1000 });
expect(charged.spend.summary.spent_usd).toBeCloseTo(0.05, 6);
expect(charged.seat_metered).toEqual([]);
expect(charged.projection.arms.map(a => [a.arm, a.n])).toEqual([['gpt-6-astra', 1]]);
expect(charged.projection.verdict).toBe('unknown');
expect(() => startPilot({ out: path.join(d, 'stray'), cap: 10, n: 1, arms: ['gpt-6-astra'], seatMetered: ['gpt-5.4-mini'], dryRun: dry })).toThrow('--seat-metered');
const switched = path.join(d, 'switched');
startPilot({ out: switched, cap: 1, n: 3, arms: ['gpt-6-astra'], seatMetered: ['gpt-6-astra'], dryRun: dry });
expect(() => startPilot({ out: switched, cap: 1, n: 3, arms: ['gpt-6-astra'], dryRun: dry })).toThrow('needs a new --out');
expect(JSON.parse(fs.readFileSync(path.join(switched, 'pilot.json'), 'utf8')).seat_metered).toEqual(['gpt-6-astra']);
});
test('#3653: an all-seat-metered pilot spends nothing and fits; one priced attempt is not enough for a verdict', () => {
const d = tmp();
const dry = passingDryRun(d);
const seat = path.join(d, 'all-seat');
const s = startPilot({ out: seat, cap: 1, n: 3, arms: ['gpt-6-astra'], seatMetered: ['gpt-6-astra'], dryRun: dry });
for (const a of s.record.attempts) settlePilot(seat, a.attempt, 'unknown');
const all = pilotReport(seat, { total: 50 });
expect([all.projection.projected_usd, all.projection.verdict]).toEqual([0, 'within']);
expect(renderPilotReport(all)).toContain('known_arms=0/0');
const one = path.join(d, 'one');
settlePilot(one, startPilot({ out: one, cap: 10, n: 1, arms: ['gpt-6-astra'], dryRun: dry }).record.attempts[0]!.attempt, 0.1);
const r = pilotReport(one, { total: 50 });
expect([r.projection.band_usd, r.projection.verdict]).toEqual([null, 'unknown']);
expect(renderPilotReport(r)).toContain('settle at least 2 attempts per priced arm');
});
test('#3653: a seat-metered arm still needs a price row, as the dry run does', () => {
const d = tmp();
const s = startPilot({ out: path.join(d, 'seat'), cap: 1, n: 2, arms: ['gpt-5.4-mini', 'seat-only-model'], seatMetered: ['seat-only-model'], dryRun: passingDryRun(d) });
expect(s.error?.code).toBe('EVAL_PRICE_MISSING');
});
test('#3653: pilot start --seat-metered reaches the ledger through the CLI', () => {
const d = tmp();
const out = path.join(d, 'cli');
const r = spawnSync(BIN, ['pilot', 'start', '--out', out, '--cap', '10', '--arms', 'gpt-5.4-mini,gpt-6-astra', '--seat-metered', 'gpt-6-astra', '--dry-run', passingDryRun(d)], { cwd: d, encoding: 'utf8', timeout: 30_000 });
expect(r.status).toBe(0);
expect(r.stdout).toContain('PILOT_ATTEMPT: pilot-gpt-6-astra-1 model=gpt-6-astra estimate=$0.0000');
expect(JSON.parse(fs.readFileSync(path.join(out, 'pilot.json'), 'utf8')).seat_metered).toEqual(['gpt-6-astra']);
});
});
// ---------------------------------------------------------------------------
+108
View File
@@ -0,0 +1,108 @@
import { describe, test, expect, beforeEach, afterEach, afterAll } from 'bun:test';
import { spawnSync } from 'child_process';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { ceoOnlyEvidence, listFiles } from './helpers/autoplan-ceo-only-evidence';
import { PTY_LONG_MS } from './helpers/eval-budgets';
import { runSkillTest } from './helpers/session-runner';
import { buildSeedConfig, seedHermeticGstackHome, seedHermeticRuntimeView } from './helpers/hermetic-env';
import { FORCING_FLOOR_CEO } from './fixtures/forcing-finding-seeds';
import {
ROOT, runId, evalsEnabled, describeIfSelected, logCost, recordE2E, copyDirSync, createEvalCollector, finalizeEvalCollector,
} from './helpers/e2e-helpers';
// /autoplan --ceo-only must finish a CEO review without asking anything (GST-537).
// The seed is the plan that makes interactive /plan-ceo-review ask a question
// (test/skill-e2e-plan-ceo-finding-floor.test.ts is the control on the same seed).
// Every check is a pure function over the transcript and files,
// proven against negative controls in test/autoplan-ceo-only.test.ts.
const evalCollector = createEvalCollector('e2e-autoplan-ceo-only');
// Declared roots in the isolated HOME: the state root, the host's own config and temp, and
// Bun's package cache (the runtime writes it under HOME on first use; run gsta215-e2e-2).
const OWNED_HOME_ROOTS = ['gstack-home', '.claude', 'tmp', '.bun/install/cache', '.capy/git-shim.log'];
describeIfSelected('Autoplan CEO-only E2E', ['autoplan-ceo-only'], () => {
let workDir = '';
let stateDir = '';
let env: Record<string, string> = {};
let homeBefore: string[] = [];
const cleanup = () => {
for (const dir of [workDir, stateDir]) if (dir) fs.rmSync(dir, { recursive: true, force: true });
};
beforeEach(() => {
try {
workDir = fs.mkdtempSync(path.join(os.tmpdir(), 'skill-e2e-autoplan-ceo-only-'));
stateDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-autoplan-ceo-only-state-'));
const configDir = path.join(stateDir, '.claude');
const gstackHome = path.join(stateDir, 'gstack-home');
const tempDir = path.join(stateDir, 'tmp');
fs.mkdirSync(configDir);
fs.mkdirSync(gstackHome);
fs.mkdirSync(tempDir, { mode: 0o700 });
fs.writeFileSync(path.join(configDir, '.claude.json'), JSON.stringify(buildSeedConfig({
apiKey: process.env.ANTHROPIC_API_KEY ?? process.env.GSTACK_ANTHROPIC_API_KEY, trustedDirs: [workDir],
})), { mode: 0o600 });
seedHermeticGstackHome(gstackHome);
fs.mkdirSync(path.join(configDir, 'skills'));
seedHermeticRuntimeView(ROOT, path.join(configDir, 'skills', 'gstack'));
env = { HOME: stateDir, CLAUDE_CONFIG_DIR: configDir, GSTACK_HOME: gstackHome, GSTACK_STATE_ROOT: gstackHome,
TMPDIR: tempDir, TEMP: tempDir, TMP: tempDir };
const git = (...args: string[]) => {
const r = spawnSync('git', args, { cwd: workDir, encoding: 'utf8', timeout: 10_000 });
if (r.error || r.status !== 0) throw new Error(`CEO-only fixture setup failed: ${r.error?.message ?? r.stderr}`);
};
git('init', '-b', 'main');
git('config', 'user.email', 'test@test.com');
git('config', 'user.name', 'Test');
// The seed's first line asks for a /tmp plan copy; the plan itself is the rest.
fs.writeFileSync(path.join(workDir, 'PLAN.md'), FORCING_FLOOR_CEO.split('\n').slice(2).join('\n') + '\n');
fs.writeFileSync(path.join(workDir, 'README.md'), '# Acme API\n\nA paid HTTP API with one pricing tier.\n');
git('add', '.');
git('-c', 'commit.gpgsign=false', 'commit', '--no-verify', '-m', 'Plan');
for (const skill of ['autoplan', 'plan-ceo-review']) {
const dest = path.join(workDir, '.claude', 'skills', skill);
fs.mkdirSync(dest, { recursive: true });
fs.copyFileSync(path.join(ROOT, skill, 'SKILL.md'), path.join(dest, 'SKILL.md'));
copyDirSync(path.join(ROOT, skill, 'sections'), path.join(dest, 'sections'));
}
homeBefore = listFiles(stateDir, OWNED_HOME_ROOTS);
} catch (error) {
cleanup();
throw error;
}
});
afterEach(cleanup);
afterAll(() => finalizeEvalCollector(evalCollector));
// Titled by its E2E id so scripts/lib/paid-cases.ts needs no CASE_TEST_NAMES entry.
test.skipIf(!evalsEnabled)('autoplan-ceo-only', async () => {
const result = await runSkillTest({
testName: 'autoplan-ceo-only',
workingDirectory: workDir,
env,
prompt: '/autoplan --ceo-only --no-outside\n\nReview the plan in PLAN.md.',
timeout: PTY_LONG_MS,
allowedTools: ['Bash', 'Read', 'Write', 'Edit', 'Grep', 'Glob', 'Agent', 'Skill'],
maxTurns: 200,
runId,
});
const before = new Set(homeBefore);
const evidence = ceoOnlyEvidence({
transcript: Array.isArray(result.transcript) ? result.transcript : [],
output: result.output,
strayWrites: listFiles(stateDir, OWNED_HOME_ROOTS).filter(f => !before.has(f)),
mustMention: [/developer/i, /talk|interview|validat|evidence|research/i],
});
logCost('autoplan-ceo-only', result);
recordE2E(evalCollector, 'autoplan-ceo-only', 'Autoplan CEO-only E2E', result, {
passed: evidence.passed, ...(evidence.reasons.length ? { error: evidence.reasons.join('; ') } : {}),
});
expect(evidence.reasons).toEqual([]);
}, PTY_LONG_MS + 60_000);
});
+4 -2
View File
@@ -310,8 +310,10 @@ describe('selectTests', () => {
// The formerly keyless periodic AUQ probes drive the CEO skill too.
expect(result.selected).toContain('auq-consistency');
expect(result.selected).toContain('auq-verbose-vs-carved-ab');
expect(result.selected.length).toBe(21);
expect(result.skipped.length).toBe(Object.keys(E2E_TOUCHFILES).length - 21);
// /autoplan --ceo-only runs plan-ceo-review's Phase 1 and its 0E mode rule.
expect(result.selected).toContain('autoplan-ceo-only');
expect(result.selected.length).toBe(22);
expect(result.skipped.length).toBe(Object.keys(E2E_TOUCHFILES).length - 22);
});
test('global touchfile triggers ALL tests', () => {