* Load the Capy crew skills only on Capy machines (GST-1833) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Add changelog entry for Capy-only crew skills (GST-1833) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Move the Capy-only crew tooling into crew/ and run its tests only when needed (GST-1833) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Test that the Capy launch file puts crew/bin on PATH Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Count any ~/.capy as a Capy machine, so a Mac running Capy keeps /captain Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Fix the tests and fixtures that broke when crew tooling moved into crew/ Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Plan every crew test when a pull request's merge-base can't be read Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Add a test that checks each crew-only module left the root tree for crew/ Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Run crew tests on pull requests that change bin/ commands or workflows they drive Review-Finding: p1/specialist-batched#1 reader=testing Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Keep running every crew test on every pull request; the skip comes in a later change Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Keep copied installs and older release bisects working after the crew/ move Setup now copies crew/lib into every host's runtime root beside bin and lib, so Windows installs (which copy instead of linking) can still start gstack-ship-receipt, gstack-pregate and the other commands that import it. The release bisect runs the train-gate script from wherever the checked-out commit has it, so a bisect over commits from before the move still gets a verdict. Review-Finding: p1/outside-voice#1 reader=outside-voice Review-Finding: p1/outside-voice#2 reader=outside-voice Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Say in the changelog that Windows installs keep working after the crew/ move Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Point the harness guide's merge-queue rows at crew/lib/ where those modules moved Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Point the plain-title test at the wave module's new home in crew/ (GST-1833) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
8.3 KiB
gstack on Capy: what crew-only pull requests cost, and the switch for Phase 2
Status: Phase 0 of GST-1833, written 2026-11-07. Nothing here changes how gstack
behaves; it records a measurement for Garry's gate decision and the interface
Phase 2 builds. Approved shape (Garry, 2026-11-07): the crew tools stay in this
repo under crew/ and load only on Capy machines; everyone else gets gstack
without them.
1. What a lighter gate for crew-only pull requests would have saved
Answer: about $735–855 over the 14 days 2026-10-24 to 2026-11-07 on recorded spend, or about $1,900–2,000 if every crew-only pull request had paid what today's selector picks. On today's merge process the paid-eval part is close to $0. That is well under the planning estimate of $2–6k per 14 days.
The lighter gate measured: a pull request whose code changes are all crew-only (tests and docs allowed) runs no public paid evals and gets one review pass.
| Pull requests | Recorded paid evals | Paid evals through today's selector | Extra review rounds beyond the first | |
|---|---|---|---|---|
| Merged in the window | 529 | $8,476 (plus $969 release gate, $300 unattributed) | 42,630 cases, $14,727 | 626 |
| Touching crew code | 260 | $4,467 | $8,933 | 373 |
| Crew-only (the lighter gate's scope) | 54 | $711 | 5,454 cases, $1,878 | 64 |
| Crew plus unclassified repo files only | 22 | $333 | $781 | 23 |
| Mixed (crew plus product or shared code) | 184 | $3,423 | $6,275 | 286 |
- Paid evals, recorded: $711. $245 came from screens those pull requests ran and $465 from their equal share of merge-train full gates. The train share is an upper bound, since a train ran once for up to five pull requests.
- Paid evals, modelled: $1,878. Only 3 of the 54 select zero cases today,
because some crew files (for example
scripts/pr-dependencies.ts) are inputs to every paid case and some crew bins are named in skill prose. - Review: about $24–144 (estimate). 64 extra rounds and about 83 extra reviewer dispatches, priced from 36 measured dispatches in 14 /ship cost logs.
- Widening the scope. Pull requests whose other files are tests or docs
are already in the 54. Adding the 22 whose other files are unclassified
(
TODOS.md,README.md,.gstack/,scripts/lib,scripts/resolvers) raises paid evals to $1,044 recorded and $2,659 modelled, but some of those touch product:scripts/resolversgenerates skill text.
Caveats
- The process changed mid-window. Until 2026-11-04 every pull request rode a merge train that ran the full paid gate. Since direct landing (#3977) a pull request pays for paid evals only when it asks for a screen, and the release gate runs on main every 8 hours whatever each pull request changed. All of the recorded paid saving comes from the train period; the 16 crew-only pull requests merged after 2026-11-04 recorded $0.
- The modelled column uses today's selector, not the one each pull request ran against, and counts gate cases only (a full gate's judges cost about $0.48).
- 24% of recorded dollars are priced from run duration because those trials recorded no cost.
- Review dollars are an order of magnitude: the cost logs carry no pull request number, and 168 merged pull requests have no review line in their body.
- Only paid-eval and review spend is counted: not /ship's own work, /autoplan, free CI minutes or Ubicloud pre-gates.
- The 528/276/62/196 split in the planning inventory differs from the 529/260/54/184
here mainly because
agents-digest/gstack-AGENTS.mdchanges only its version line in about 60% of commits; this count treats that line as version metadata.
Method
First-parent squash merges on origin/main from 2026-10-24T00:00Z to
2026-11-07T23:59Z, each diffed against its parent and classified with the
planning inventory (crew, product, shared, unclassified; tests and docs never
count against a pull request). Recorded spend sums executed trials' cost_usd
from every evals.yml run's trial-outcomes, paid-plan, report-verdict and
train-record artifacts (2,818 runs, 1,872 artifacts). The modelled column is
scripts/gate-cost-replay.ts --since 2026-10-24 --until 2026-11-07 --json at
5f9b7ef with the case cost table from runs 42391221370, 42349387007 and
41751174367. Review rounds come from each pull request body's REVIEW_LOOP
line. Free, read-only, no paid runs.
2. Phase 2: one switch for the crew steps in /ship, /review and /autoplan
Phase 1 moves code no public skill calls into crew/ and makes setup register
the crew skills only on Capy. What is left is crew code that public skills call
today. Phase 2 puts each of those calls behind one switch, so a non-Capy run
skips the crew step and gets a defined result instead.
The switch
- One signal. "Crew loaded" means a Capy machine (
CAPY_MACHINE_CONFIGset or any~/.capyentry, which paired devices have too:gstack_on_capyinbin/gstack-install-registry.sh,onCapyMachineinscripts/discover-skills.ts, both added by #4143) and acrew/folder in the checkout. Nothing else turns crew steps on: notGSTACK_CAPY, not.gstack/ship-policy.json. Tests force either answer with a temporary HOME, astest/crew-skills-on-capy.test.ts(also from #4143) does. - Prose calls go through one wrapper. A new
bin/gstack-crew-step <step> -- <command…>runs the command when crew is loaded. Otherwise it runs nothing, printsCREW_STEP: <step> skipped (not a Capy machine)and exits 0. Skill templates name crew commands only through it, so the generated SKILL.md is the same for every host and a guard test can list every crew call. - Code calls go through one module.
lib/crew-hooks.tsexports the crew steps shared /ship code needs, each with a typed "absent" result. When crew is loaded it importscrew/lib/hooks.tsdynamically; otherwise it returns the absent result without loading any crew module.lib/ship-policy.ts,lib/ship-preflight.ts,lib/ship-bin-common.ts,lib/ship-patch.tsandlib/gstack-post.tsimport that module instead of crew modules. A guard test fails when anything outsidecrew/importscrew/directly.
What each skill does without crew
| Skill and step | Crew command or module today | Without crew |
|---|---|---|
| /ship Step 1 and every boundary check | gstack-ship-budget start, note and handoff; lib/spend-ledger |
No meter; an explicit --token-cap is still honoured by the public core |
| /ship Step 5 and finish | gstack-pregate (remote carry), gstack-ci-carry |
Local checks only (generated-file freshness, secrets, strays), under a public name |
| /ship push and PR body | gstack-ship-receipt, SHIP_RECEIPT line, the Approvals and Ruled-notes lines |
No receipt and no crew lines; the body keeps What changed, Why and Blast radius |
| /ship version | gstack-ship-release, restamp |
The existing VERSION and CHANGELOG path; restamp's rewrite stays public for /land-and-deploy |
| /ship queue ready, red-base pause | gstack-ship-queue ready, lib/base-red |
Skipped |
| /review rulings | gstack-approval-classes |
No approval-class check |
| /autoplan budget and gate | lib/spend-ledger, lib/gate-list (nine modules import them) |
Split: a public budget and gate core stays in lib/; crew keeps the ledger mirror |
| /autoplan owner brief and trail | gstack-owner-brief, lib/captain-brief, gstack-decision-trail, gstack-lane-check |
Skipped; the plan and the gate render as today |
How Phase 2 proves it
- A no-crew install check in CI: a clean HOME without the Capy signal runs /ship's deterministic steps with stub crew commands that record any call, and fails on one. The same job runs with the signal and expects today's calls.
- Byte-identical Capy behaviour: the generated SKILL.md and the PR body a Capy run produces are compared with main's on a recorded fixture.
- Focused evals: /ship and /autoplan, a crew arm and a no-crew arm, under the $10-per-PR cap, with a no-loss bar registered before the run (review changes merge only after their no-loss replay passes).
- Rollout: one pull request per row group (ship budget and receipts; pregate split; autoplan split; review), each landing alone. Rollback is a revert of that pull request; the switch itself keeps today's behaviour on Capy, so a crew run never depends on the new path being right.