From 959c8a6a4fdafabf1186e407b76082b7f07e9f24 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Thu, 29 Oct 2026 18:34:37 -0700 Subject: [PATCH 1/3] v1.91.327.0 docs(capy): capy/README.md, the gstack + Capy software factory with measured numbers (#3622) * docs(capy): capy/README.md, the gstack + Capy software factory: how it fits, measured before/after, tips, how Garry uses it Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(capy/README): footnote's gh pr list query needs --limit to return 197 Without --limit, gh pr list returns at most 30 rows, so the quoted reproduction printed 30, not 197 or 65. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(capy/README): don't name a private repository's Captain The Harness Evals Captain runs a private repository; the public README names only gstack-evals among private repositories. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(capy/README): #3610 is an open fix, not a landed one Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(capy/README): the 2.4-day median covers issues of every kind, not only bugs The 52 are issues linked to a merged PR by closing references; the measurement does not classify them as bugs. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(capy/README): point the /eval-plan link at its section in docs/skills.md Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): [Unreleased] entry for capy/README.md Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(capy/README): the Captain took over on 2026-10-16; 10-15 is the first fix waves Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(capy/README): the 2026-10-15 waves were not gstack's first fix waves docs/designs/ already holds fix-wave plans from October 3, 6 and 12; on October 15 Garry approved two more (fix-wave-2026-10-15 and fix-wave-2-2026-10-15). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(docs/captain.md): keep the controller guide generic; the capy/README.md pointer names no owner Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(capy/README): tighten claims the review noted (stamp rule, UTC days, gbrain attribution, gate-class PRs, severe fixes keep their gates) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(capy/README): everyday words (two weeks, plain-English summary, gut feel, merge commit) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * v1.91.327.0 stamp Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 8 ++ README.md | 1 + VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- capy/README.md | 230 +++++++++++++++++++++++++++++++++ docs/captain.md | 4 +- package.json | 2 +- 7 files changed, 245 insertions(+), 4 deletions(-) create mode 100644 capy/README.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 07ea2940e..0126e53be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,13 @@ # Changelog +## [1.91.327.0] - 2026-10-30 + +**A new page shows how gstack and Capy run as a software factory, with measured numbers.** `capy/README.md` explains how Capy Captains, their crews and gstack's skills fit together, from the plan-first approval loop and `/ship` before READY to merge trains, fix waves, the daily flaky pass and peer Captains. It compares the 14 days after 2026-10-15 with the 14 days before: 197 PRs merged to gstack's main against 65, and 194 versions shipped against 62. It also says what those numbers leave out. The median PR shrank from 1,292 changed lines to 422, and the median time from opening a PR to merging it grew from 1.1 hours to 5.6, because trains batch merges. A footnote gives the windows and the method behind every figure, and the page ends with tips and how Garry uses the setup day to day. + +### Docs +- **`capy/README.md`**: the new page (how the pieces fit, the measured before/after, why we think it works, tips, and how Garry uses it). +- The documentation table in `README.md` and the introduction of `docs/captain.md` link to it. + ## [1.91.326.0] - 2026-10-30 **/ship reviewers now read one packet file each, and the parent reads a findings record of at most 60 lines instead of every reviewer's transcript.** On this PR's own /ship, pass 2's eight reviewers cost 8.6M tokens on the root counter, against 52.6M for pass 1's eleven (pass 2 read the fix range, pass 1 the whole diff, so the two are not directly comparable). diff --git a/README.md b/README.md index 222b96893..5a3cd01a0 100644 --- a/README.md +++ b/README.md @@ -242,6 +242,7 @@ On Windows without Developer Mode (MSYS2 / Git Bash), `setup` falls back to file |-----|---------------| | [Skill Deep Dives](docs/skills.md) | Philosophy, examples, and workflow for every skill (includes Greptile integration) | | [The recommended harness](docs/recommended-harness.md) | gstack + Capy from zero to the first merged train: install, the Captain and its Drive, trains, the daily flaky pass, the twice-weekly release, the factory scorecard, what is Capy-specific, known Capy limits | +| [gstack + Capy: the software factory](capy/README.md) | What we built with Capy Captains and gstack, how the pieces fit, the measured before/after, tips, and how Garry uses it | | [Why gstack](docs/why-gstack.md) | The story, the productivity numbers and their caveats, Karpathy's failure modes, parallel sprints | | [Install details](docs/install.md) | Setup timing, requirements in detail, team mode, the hooks setup registers, agent harness registration | | [Other agents](docs/hosts.md) | Agent harnesses (OpenClaw, Hermes, dots, QM, Muse Code, Grok, any other), OpenClaw and ClawHub, per-host notes, the Codex profile and model tiers, outside-review providers, the design binary's models | diff --git a/VERSION b/VERSION index 10366ab12..5c6442608 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.91.326.0 +1.91.327.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 693798394..b06d270a6 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.91.326.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.91.327.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/capy/README.md b/capy/README.md new file mode 100644 index 000000000..60ed605a1 --- /dev/null +++ b/capy/README.md @@ -0,0 +1,230 @@ +# gstack + Capy: the software factory + +gstack is a set of engineering skills: plan reviews, `/ship`, `/review`, `/qa`, +evals. Capy runs agents in the cloud. It gives each thread its own machine, and it +adds Captains that direct a crew of threads, a shared Drive, and automations. Put together, one +person runs a team of agent engineers across several repositories, and the work +lands on main green, reviewed, and in the order he chose. + +This page is what we built, how the pieces fit, why we think it is the best way +to run a software factory today, what we learned the hard way, and how Garry +actually uses it. The step-by-step setup is in +[docs/recommended-harness.md](../docs/recommended-harness.md); the controller's +reference is [docs/captain.md](../docs/captain.md). + +## The numbers + +The 14 days from the 2026-10-15 fix waves (the Captain took over +the next day) against the 14 days before, measured from git and GitHub[^1]: + +- **197 PRs merged to gstack's main, up from 65 (3.0x).** That is 98.5 a week. +- **194 versions shipped, 13.9 a day, up from 62 (3.1x).** Nearly every merged PR + gets its own version number when it merges. +- **Nearly twice gstack's best two weeks before Capy.** The March launch + shipped 99 versions in its busiest 14 days; these two weeks shipped 194. +- **36 PRs merged and 36 versions shipped in one day** (UTC), on October 25 + and again on October 26. +- **A reported issue is fixed in a median 2.4 days**, from the issue opening to the + fixing PR's merge, across the 52 issues closed by a PR in the window. +- **1,874 PRs merged across eight of Garry's repositories** in the same 14 days. + +What the numbers don't say: PRs got smaller. The median gstack PR fell from +1,292 changed lines to 422, so "3x the PRs" is true and "3x the code" is not. +Small PRs are the point (each one is reviewed, gated and revertible on its +own), but they are not more total output. Time from opening a PR to merging it +went up, from a median of 1.1 hours to 5.6, because trains batch merges. + +## What we did + +Capy threads started writing gstack PRs on 2026-09-15. On 2026-10-15 Garry +approved two fix waves, and the next day he handed gstack's development +to a Capy Captain, GStack Minder. Its standing job is to keep +`garrytan/gstack` moving and main green. Within two weeks there were seven Captains (gstack, gbrain, +alphaclaw, two eval projects, a game, and a magazine), each directing its own +crew of threads. + +Most of what made it work did not exist on day one. It came out of Garry's +replies, one at a time, each turned into a rule and then into gstack itself: + +- "yes merge when green" became the Captain's standing merge approval, and then + [merge trains](../docs/captain.md#merge-trains): one paid gate covers up to + five READY PRs, and the Captain merges them slot by slot at the exact head. +- "you can just run the failed test again and note that it is flaky and we + should have a flaky test fix pass every day" became the one-rerun rule and a + [daily flaky pass](../docs/captain.md#the-daily-flaky-pass) automation. +- "for each plan approval i want a before/after, why it matters, what caused us + to work on this, and big decisions you had to make" became + `gstack-owner-brief` and [the three approval shapes](../docs/captain.md#approvals-three-shapes). +- "also we should really be using the gstack ship skill before merging PRs" + became the `SHIP_RECEIPT` every READY must carry at its head. +- "When you make a new thread or take on new functionality can you remember to + use gstack autoplan to make a plan for it and then eli5 for approval from me?" + became [Plan before build](../docs/captain.md#plan-before-build). +- "Gstack should be the actual holder of how to do things not one off Capy + files" is why all of the above lives in this repository, not in anyone's Drive. + +## How Capy and gstack fit together + +```mermaid +flowchart LR + G[Garry] -->|one-word answers| C[Captain] + C <-->|findings, priorities| P[Peer Captains] + C -->|brief| L1[Lane: plan with /autoplan] + C -->|brief| L2[Lane: build] + C -->|brief| L3[Lane: fix wave] + L1 -->|plain-English plan + rec| C + L2 -->|/ship receipt, READY at head| C + L3 -->|READY| C + C -->|train: one gate, up to 5 PRs| T[(Gate)] + T -->|green| M[(main)] + T -.->|flake records| F[Daily flaky pass] +``` + +**Captains, crews and lanes.** A Captain is a long-running Capy thread that owns +one repository's flow. It starts crew threads (lanes), each on its own machine +and branch with named files, so no two lanes edit the same file. The Captain's +Drive holds the crew's contract (`CREW.md`), its digest, and each lane's report; +every rule in it points back at a gstack doc. Lanes never ask Garry directly. +Everything goes through the Captain, which asks in plain words and brings back +his exact reply. + +**Plan first, with a plain-language yes.** New functionality starts with +`/autoplan`: a CEO review, an engineering review, design and DX when the work +has them, each with a native and a cross-model voice. The gate goes to Garry as +a short plain-English summary with "rec" as the default answer and a "do it" option that takes +every recommendation at full scope. One "do it" has approved eight plans in a +morning, and the lanes built them in parallel. Reviews catch the expensive +mistakes before code exists. One plan learned that 12 tasks were too few to +prove anything, and another that its adapter needed a translator nobody had planned. + +**`/ship` before READY, merge on green.** A lane runs `/ship` at its final +pushed head. That runs the tests, a review sized to the diff, a cross-model +voice where it matters, and writes a receipt bound to that head. The lane sends +one unstamped READY and stops. The Captain batches READY PRs into a train, +reads one full gate, stamps versions at merge, and merges each PR with +`--match-head-commit`, so nothing lands at a head nobody reviewed. A review +finding blocks only when it "would fail a test, break production, or lose data +or secrets" (Garry's floor); the rest goes in the PR body. + +**Fix waves.** When main goes red, lanes hold `/ship`, one owner finds the root +cause, and everyone merges main once. Deferred review findings and test-only +follow-ups get folded into one wave PR instead of one PR each. On one crew a +single audit-pin fix cleared a main-wide red and ten PRs landed within the hour. + +**The daily flaky pass.** A red test is rerun once at the same tree. A pass is +logged as flaky and the PR moves on; a second red is real. Every morning an +automation starts a lane that fixes the worst repeat offender with a PR that +shows it red before green. Nothing is retried into green and no threshold moves. + +**Peer Captains route findings, not chatter.** Captains message each other +with evidence. gstack-evals' replay showed `/review` downgrading real bugs, and +within the hour gstack had a lane on it. Another eval Captain's $5.78 panel found that +OpenClaw refused to load a 78 KB gstack skill, and gstack had a fix up the +same day ([#3610](https://github.com/garrytan/gstack/pull/3610)). When Garry +said "hurry up" to one Captain, another reordered its merge queue for it and +sent back each merge commit. + +**Evals with preregistered bars.** A prompt or skill change that claims to +help is judged by a before/after with fixed trials and a bar written down +before the first paid call ([/eval-plan](../docs/skills.md#eval-plan)), never by gut feel. +Good evals here have a no-skill arm and a length-matched placebo, a check that +the treatment actually arrived, decoys verified clean, and spend quoted from +the meter. They report losses as plainly as wins. One preregistered eval found +that with the skill loaded every time, gstack still lost to the placebo on a +cheap model, and that result now drives gstack's work on its review rubric. +In gbrain, a facts block won by 16 points on a held-out set, and only then +did it go on by default. + +## Why we think it is the best software factory + +- **The owner decides, and only decides.** Garry's job is one-word answers to + plain-language questions. Everything that a command, a measurement or a test + can settle gets settled before it reaches him. +- **Main stays green at volume.** Receipts bound to the exact head, merges with + `--match-head-commit`, trains that test the combination before it lands, and + a backstop that freezes the queue on a red main. +- **Every rule is code.** The rules live in gstack: bins that refuse a stale + receipt, tests that cap prompt sizes, a ship policy file. A new crew inherits + them by installing gstack, not by reading someone's notes. +- **It measures itself.** `gstack-factory-scorecard` reports lead time, gate + cost, red-train rate and flaky events every week against industry benchmarks (DORA), + and says `unavailable` instead of 0 when it can't read something. +- **It is honest about cost.** Paid gates are cheap relative to a broken main + ("it's fine run as many $93 paid gates as you want who cares"), but token + spend per PR is tracked, capped and treated as a bug when it grows. + +## Tips and tricks + +1. **Install gstack on every machine, every session.** The first act of every + thread and subagent is the install check; no `gstack: ok` line means no + gstack workflow on that machine. +2. **Always offer a "do it".** Garry takes recommendations. Make the + recommended reply the one-word default, and never offer "the plan as first + written" as the safe choice. +3. **Ask "worth it?" before three review rounds.** A plan the owner may shelve + gets a cheap check first. A deploy replay that measured 83 seconds of boot + cost killed a plan three review rounds had blessed. +4. **Assign versions at merge, not in the PR.** Version and CHANGELOG + collisions were the second-biggest time sink until READY went unstamped. +5. **Don't merge main every time it moves.** Each merge recreates the + CHANGELOG conflict and voids the receipt. Merge once, at your slot. +6. **Size review to the diff.** A docs PR doesn't need a red team. Full-depth + review on small diffs is how `/ship` once cost about 50M tokens per + PR that needed the full test gate. +7. **Keep threads small.** A long-lived parent context reached 1.36B tokens. + Hand long steps to fresh children and read the token counter instead of + estimating. +8. **Use background operations, not watcher subagents**, and message only on a + decision, a blocker, a READY or a finished phase. Most Captain wakes used to + be "nothing to report". +9. **Measure before you choose.** Time the queue, count the PRs, run the real + system. A runner option that looked cheaper in quiet hours failed in a busy + one. +10. **Severe fixes go first, through the same gates.** In gbrain a + trust-escalation bug went from report to merged fix in about three hours, + with its receipt, green checks and merge owner unchanged. + +## How Garry uses it + +- **He answers in one word, from his phone.** "rec", "do it", "Take all + recommendations". Long answers mean the ask was badly shaped. +- **He asks for explanations, not jargon.** "Eli5 these recommendations and + please note in the future you need to explain." Gate codes and slot numbers + never reach him. +- **He wants to see what changed.** Every merge ask shows before/after images + of what changed and why. +- **He spends money to save time and tokens to save money.** Paid gates are + pre-approved; token waste is a P-1 ("Can we get 95% of the outcome but with + 50% of the token use?"). +- **He pushes for parallelism.** "Make sure we are parallelizing as much as we + can that is practical to do": five to eight lanes on disjoint files whenever + there is a backlog. +- **He turns every annoyance into a rule, and every rule into gstack.** "are you + learning that the ship review loop is too overzealous?" became the blocking + floor the same day. +- **He asks the factory to improve itself.** "Ask all captains for feedback on + how fast or slow things are and if we can maintain quality but go faster", + then "Can you fix GStack to incorporate the feedback". This page came from + the same kind of ask. + +## Where to go next + +- [docs/recommended-harness.md](../docs/recommended-harness.md): set it up, day 0 to the first merged train. +- [docs/captain.md](../docs/captain.md): the controller's reference (READY contract, trains, approvals, digest, flaky pass). +- [capy/skill/gstack/](skill/gstack/): the Capy skill every thread loads, and its [captain.md](skill/gstack/captain.md) for lanes. +- [docs/why-gstack.md](../docs/why-gstack.md): the longer story behind gstack itself. + +[^1]: Windows are UTC, [2026-10-01, 2026-10-15) before and + [2026-10-15, 2026-10-29) after. Merged PRs are GitHub's `mergedAt` with base + `main` (`gh pr list --state merged --limit 1000 --search 'merged:2026-10-15..2026-10-28 base:main'` + returns 197; the same query for 10-01..10-14 returns 65). Versions are distinct + new values of `VERSION` on main's first-parent history. The March comparison is + the busiest 14-day window between 2026-03-01 and 2026-09-15. Issue-to-fix time + links issues to PRs through GitHub's closing references and "fixes #n" lines; + the before window's median (35.9 days) is left out because those PRs were + clearing an old backlog. The eight-repository total covers gstack, gbrain, + gbrain-evals and alphaclaw plus four private repositories, which merged 1,243 + of the 1,874 PRs and were created during the window, so it is volume, not a + speedup. Capy threads were already writing most gstack PRs before the window + (57 of the 65), so this measures the step to Captains, merge trains and + `/ship` before READY, not Capy against no agents. Measured 2026-10-29. diff --git a/docs/captain.md b/docs/captain.md index e955acfb7..a27a01794 100644 --- a/docs/captain.md +++ b/docs/captain.md @@ -8,7 +8,9 @@ controller the same model at run time is [`/captain`](skills.md#captain) (it als to `/crew`); the lane-side rules for a thread working under a Captain are in [`capy/skill/gstack/captain.md`](../capy/skill/gstack/captain.md) and [capy-install.md](capy-install.md#working-under-a-captain). Every file named below has a -copy-paste template under [`captain/templates/`](../captain/templates/). +copy-paste template under [`captain/templates/`](../captain/templates/). For the story of how +one owner's crews run this day to day, with measured numbers, read +[capy/README.md](../capy/README.md). ## The model in one paragraph diff --git a/package.json b/package.json index a15159f16..e973f1654 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.91.326", + "version": "1.91.327", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", From 9184ffd16cfe54f9944ea0a6d5adecd1c72eb60c Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Thu, 29 Oct 2026 20:14:00 -0700 Subject: [PATCH 2/3] v1.91.328.0 ship-queue: screen only when the selector restores every case; unscreened PRs fill train seats (GSTA-165 1a) (#3619) * Narrow screens leave hub files to the train gate; map size-ledger baselines and the careful mask awk Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (pull_request runs stay broad, deleted hubs, READY and summary previews) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG for narrow screens (Unreleased) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: TESTING_INTERNALS describes narrow and broad PR screen policy Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * chore: record test-pr-profile.ts as non-behavioral at HARNESS_VERSION 6 Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship-queue: screen only when the selector restores every case; unscreened PRs fill train seats (GSTA-165 1a) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test: pin the diffs that still screen (unselectable, skill template + bun.lock) Two assertions in test/ship-queue-train-logic.test.ts: a diff the paid selector cannot select recommends --screen with the full gate selection, and a diff that mixes a skill template with bun.lock recommends --screen. GSTA-165 1a, from /ship Steps 7-8. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: CHANGELOG tests and deferrals; P1 TODOs for GSTA-165 follow-ups CHANGELOG [Unreleased]: name the two added tests and list the deferred CREW.md rule 22 text and train-record attribution field. TODOS.md: both deferrals as P1 under Ship. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 13 +++++++++ TODOS.md | 28 ++++++++++++++++++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- docs/ship-policy.md | 13 +++++---- lib/gate-cost-history.ts | 20 +++++--------- lib/ship-queue-train-ready.ts | 29 ++++++++++++++------ lib/ship-queue-train.ts | 14 +++++----- package.json | 2 +- test/gate-cost.test.ts | 4 +-- test/ship-queue-train-logic.test.ts | 41 ++++++++++++++++++++--------- test/ship-queue-train.test.ts | 6 ++--- 12 files changed, 121 insertions(+), 53 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0126e53be..63eb9ec49 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,18 @@ # Changelog +## [1.91.328.0] - 2026-10-30 + +**Most PRs no longer pay for a solo paid screen before the merge train runs the same cases.** `gstack-ship-queue ready` used to recommend `--screen` whenever a diff touched a skill's own cases, which is almost every PR, and the screens (16 to 19 a day at about $32 each) cost more than the train gates that rerun all 137 cases a few hours later. Of 41 full screens between Oct 20 and 29, the 3 that caught a real bug all came from diffs the selector can't narrow (an unknown dependency or a shared runtime input). The recommendation is now `--screen` only in that case; every other PR rides unscreened and the train gate's full census covers it. Because most PRs now ride unscreened, a train also stopped holding them back, so unscreened PRs fill the seats a screened PR leaves. + +### Changed +- **Screen recommendation** (`lib/ship-queue-train-ready.ts`): `--screen` when the paid selector's mode is `full-fallback` or the selector cannot select; otherwise `--no-screen`. The `SCREEN:` lines name the selector's mode and reason. +- **Slot order** (`lib/ship-queue-train.ts` `orderSlots`): screened PRs first, then unscreened PRs fill the remaining seats in READY order. The rule that built a screened-only train and held every unscreened PR once `max_size` of them waited is gone, along with its `HELD: … (unscreened PRs form their own train)` line. `--order` still overrides. +- **Train replay model** (`lib/gate-cost-history.ts` `pickTrain`) seats trains the same way. +- **Not in this change:** a focused paid run of a no-screen PR's own cases, bound to its READY receipt, needs a new `screen_policy: own` route in evals.yml (GSTA-165 1b, deferred). The matching rule 22 text in `captain/templates/CREW.md` is also deferred, as is a train-record field that attributes a red train to the PR that caused it, which the trigger for reverting this default needs (GSTA-165 v2 item 6). + +### Tests +- `test/ship-queue-train-logic.test.ts`: one screened plus five unscreened at size 5 seats five; a full unscreened pool at size 3 seats three; a skill-template diff recommends `--no-screen`; a `bun.lock` diff recommends `--screen` with its full-fallback reason. These tests failed before the change. Two more pin the diffs that still screen. A diff that mixes a skill template with `bun.lock` recommends `--screen`, and a diff the selector cannot select recommends `--screen` with the full gate selection. `test/ship-queue-train.test.ts` and `test/gate-cost.test.ts` change their ordering expectations on purpose. + ## [1.91.327.0] - 2026-10-30 **A new page shows how gstack and Capy run as a software factory, with measured numbers.** `capy/README.md` explains how Capy Captains, their crews and gstack's skills fit together, from the plan-first approval loop and `/ship` before READY to merge trains, fix waves, the daily flaky pass and peer Captains. It compares the 14 days after 2026-10-15 with the 14 days before: 197 PRs merged to gstack's main against 65, and 194 versions shipped against 62. It also says what those numbers leave out. The median PR shrank from 1,292 changed lines to 422, and the median time from opening a PR to merging it grew from 1.1 hours to 5.6, because trains batch merges. A footnote gives the windows and the method behind every figure, and the page ends with tips and how Garry uses the setup day to day. diff --git a/TODOS.md b/TODOS.md index 7d7aa202d..b599ee149 100644 --- a/TODOS.md +++ b/TODOS.md @@ -3587,6 +3587,34 @@ Linux cookie import shipped in v0.11.11.0 (Wave 3). Supports Chrome, Chromium, B ## Ship +### Crew template rule 22: say when a lane picks `--screen` (GSTA-165 1a follow-up) + +**What:** Update the rule 22 text in `captain/templates/CREW.md` (and its `docs/captain.md` home if it repeats the rule) so a lane picks `--screen` only when `gstack-ship-queue ready` recommends it: the paid selector's mode is `full-fallback` or it cannot select. + +**Why:** `gstack-ship-queue ready` now recommends `--no-screen` for every diff the selector can narrow, and unscreened PRs fill train seats. A crew instantiated from the template still reads the older screen guidance, so lanes can keep paying for screens the recommendation no longer asks for. + +**Context:** The GSTA-165 plan's PR 1 (D1) listed this doc change beside the code; PR #3619 shipped the code, `docs/ship-policy.md` and tests and left the template text out. Template rule 22 ("/ship before every READY") names the receipt and its homes but not the screen choice, and rule 6 names `--screen|--no-screen` without saying which to pick. + +Deferred from plan: /home/user/.capy/drive/captain-gstack-minder/plans/GSTA-165.md + +**Effort:** S +**Priority:** P1 +**Depends on:** PR #3619 + +### Train records: attribute a red train to the PR that caused it (GSTA-165 v2 item 6) + +**What:** Add an attribution field to the merge-train record so each red train names the PR (or none, or infra) that caused it. + +**Why:** The trigger for reverting GSTA-165 D1 (screen only full-fallback diffs) is "PR-attributed red trains exceed the forecast (about 10% at size 3, 16% at size 5) by 2x over 20 trains". Without an attribution field nobody can measure it, so the default cannot be revisited on evidence. + +**Context:** Restated in handoffs/GSTA-165/v2-additions.md item 6 and unassigned to a PR. Start from the train record in `lib/ship-queue-train-record.ts` and what `train bisect` writes back to it. + +Deferred from plan: /home/user/.capy/drive/captain-gstack-minder/plans/GSTA-165.md + +**Effort:** M +**Priority:** P1 +**Depends on:** PR #3619 + ### Runtime enforcement of foreground dispatch (PreToolUse hook) **What:** A PreToolUse hook (settings.json) that forces or verifies `run_in_background: false` on Agent tool calls made inside gstack workflows, making the #497/#2440 bug class structurally impossible on Claude Code instead of prose-pinned. diff --git a/VERSION b/VERSION index 5c6442608..084921575 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.91.327.0 +1.91.328.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index b06d270a6..8a81b87b0 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.91.327.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.91.328.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/docs/ship-policy.md b/docs/ship-policy.md index 56407b281..24047fa6b 100644 --- a/docs/ship-policy.md +++ b/docs/ship-policy.md @@ -414,9 +414,12 @@ gstack-ship-queue ready --pr 42 --screen [--broad] # dispatches one paid screen ``` Without a choice it prints the `SCREEN:` selection (with -`screen_selection: gstack-paid`, the skill's own cases and `recommend ---screen` or `--no-screen`: `--screen` when the screen selects a skill's own -cases) and refuses with `READY_SCREEN_CHOICE_REQUIRED`. The price line says +`screen_selection: gstack-paid`, the skill's own cases, the selector's mode and +`recommend --screen` or `--no-screen`: `--screen` only when the selector's +mode is `full-fallback`, an unknown dependency or a shared runtime input that +restores every case, or when it cannot select at all; every other PR rides +unscreened, and the train gate's full census runs its cases on the tree that +merges) and refuses with `READY_SCREEN_CHOICE_REQUIRED`. The price line says the dollar figure is pending the cost replay; until it lands, `SCREEN:` shows case counts and the recommendation only. A lane that already ran focused local cases says so in the READY block and may pass `--no-screen`. `--screen` @@ -466,8 +469,8 @@ gstack-ship-queue train build --prs 41@,42@,43@ ``` It prints `ORDER:` (the slot order to announce: screened PRs first in READY -order, then unscreened; when `max_size` or more unscreened PRs wait they form -their own train and `HELD:` lists them; `--order ,,...` naming every +order, then unscreened PRs fill the remaining seats, and `HELD:` lists the PRs +over `max_size`; `--order ,,...` naming every pinned PR once replaces that rule, for a PR that must merge before another, and the train's later `NEXT:` rebuilds keep it), one `PREFIX: T tree=` per prefix and one `SLOT:` per PR. `--dry-run` adds the diff --git a/lib/gate-cost-history.ts b/lib/gate-cost-history.ts index f6f4b668e..b379de9f3 100644 --- a/lib/gate-cost-history.ts +++ b/lib/gate-cost-history.ts @@ -317,22 +317,16 @@ export interface ArrivalTrainResult { interface Waiting { pr: number; readyMs: number; enqueuedMs: number; screened: boolean; defect: boolean } /** - * The next train, or null to keep waiting: five or more waiting screened (or - * unscreened) PRs dispatch a full train of their own; otherwise a full mixed - * train, screened first; otherwise everything waiting, screened first, once - * the oldest has waited the fill limit. The train with the oldest PR goes first. + * The next train, or null to keep waiting, as orderSlots and fillDue seat it: + * screened PRs first, then unscreened PRs fill the remaining seats, each by + * age; a full train dispatches at once, a smaller one once its oldest PR has + * waited the fill limit. */ export function pickTrain(pool: T[], nowMs: number, size: number, fillMs: number): T[] | null { const byAge = (a: T, b: T) => a.enqueuedMs - b.enqueuedMs; - const screened = pool.filter(x => x.screened).sort(byAge), unscreened = pool.filter(x => !x.screened).sort(byAge); - const options = [screened, unscreened].filter(list => list.length >= size).map(list => list.slice(0, size)); - if (!options.length) { - const mixed = [...screened, ...unscreened]; - if (mixed.length >= size) options.push(mixed.slice(0, size)); - else if (mixed.length && nowMs - Math.min(...mixed.map(x => x.enqueuedMs)) >= fillMs) options.push(mixed); - } - const oldest = (train: T[]) => Math.min(...train.map(x => x.enqueuedMs)); - return options.sort((a, b) => oldest(a) - oldest(b))[0] ?? null; + const mixed = [...pool.filter(x => x.screened).sort(byAge), ...pool.filter(x => !x.screened).sort(byAge)]; + if (mixed.length >= size) return mixed.slice(0, size); + return mixed.length && nowMs - Math.min(...mixed.map(x => x.enqueuedMs)) >= fillMs ? mixed : null; } /** diff --git a/lib/ship-queue-train-ready.ts b/lib/ship-queue-train-ready.ts index 5a04ec381..32a318742 100644 --- a/lib/ship-queue-train-ready.ts +++ b/lib/ship-queue-train-ready.ts @@ -2,8 +2,8 @@ * `ready --pr --screen | --no-screen` under a ship policy with a `train` * block (docs/ship-policy.md#run-a-merge-train; plan GSTA-49 T1). The * READY block is today's (lib/ship-queue.ts ready()); this adds the screen - * selection and the default recommendation (screen when the screen - * selects a skill's own cases), requires an explicit choice + * selection and the default recommendation (screen only when the selector + * restores every case or cannot select), requires an explicit choice * (READY_SCREEN_CHOICE_REQUIRED), dispatches a chosen screen through * eval-bg's nonce with its purpose and policy in the key, and records the * choice as a READY receipt (local state and refs/notes/gstack-trains) that @@ -49,28 +49,41 @@ export function skillOwnCases(selected: readonly string[], touchfiles: Record (touchfiles[id] ?? []).some(glob => skills.some(s => glob.startsWith(`${s}/`)))).sort(); } -/** gstack's paid selector over a diff, as a narrow screen (`--screen` without `--broad`) plans it, loaded only when the policy asks for it. */ +/** + * gstack's paid selector over a diff, as a narrow screen (`--screen` without `--broad`) plans it, loaded only when the + * policy asks for it. The recommendation is --screen only when the selector restores every case (its `full-fallback` + * mode: an unknown dependency or a shared runtime input) or cannot select at all; every other PR rides the train + * unscreened, where the full gate census runs every case on the tree that merges (GSTA-165). + */ export async function gstackPaidSelection(repoRoot: string, baseSha: string, changed: string[], hasTemplate: (dir: string) => boolean): Promise { const { computePaidCaseSelection } = await import('../scripts/lib/paid-select'); const { E2E_TOUCHFILES, LLM_JUDGE_TOUCHFILES } = await import('../test/helpers/touchfiles'); const skills = touchedSkills(changed, hasTemplate); let e2e: string[]; let judges: number; let note = "the narrow screen's selection; --broad adds the dependents of files whose references reach every gate case"; + let why: string; + let everyCase = false; try { - const s = computePaidCaseSelection({ profile: 'pr', rootDir: repoRoot, changedFiles: changed, env: { ...process.env, EVALS_BASE: baseSha, EVALS_ALL: '', EVALS_SCREEN_POLICY: 'narrow' } }).selection; - e2e = s.e2e ?? Object.keys(E2E_TOUCHFILES); - judges = s.judges ? s.judges.length : Object.keys(LLM_JUDGE_TOUCHFILES).length; + const r = computePaidCaseSelection({ profile: 'pr', rootDir: repoRoot, changedFiles: changed, env: { ...process.env, EVALS_BASE: baseSha, EVALS_ALL: '', EVALS_SCREEN_POLICY: 'narrow' } }); + e2e = r.selection.e2e ?? Object.keys(E2E_TOUCHFILES); + judges = r.selection.judges ? r.selection.judges.length : Object.keys(LLM_JUDGE_TOUCHFILES).length; + const mode = r.coverage?.mode ?? 'unknown'; + everyCase = mode === 'full-fallback'; + why = everyCase ? `selector mode full-fallback (${r.reason.slice(0, 200)})` : `selector mode ${mode}: the train gate's full census covers these cases`; } catch (error) { e2e = Object.keys(E2E_TOUCHFILES); judges = Object.keys(LLM_JUDGE_TOUCHFILES).length; note = `the PR profile needs full validation (${error instanceof Error ? error.message.slice(0, 160) : String(error)}): the screen is the full gate selection`; + everyCase = true; + why = 'the selector could not select'; } const own = skillOwnCases(e2e, E2E_TOUCHFILES, skills); - const recommend = own.length ? 'screen' : 'no-screen'; + const recommend = everyCase ? 'screen' : 'no-screen'; return { recommend, lines: [ `SCREEN: selection ${e2e.length} e2e case(s) + ${judges} judge(s) (${note})`, - `SCREEN: skill's own cases ${own.length ? own.slice(0, 12).join(',') + (own.length > 12 ? ` (+${own.length - 12})` : '') : 'none'} → recommend --${recommend}`, + `SCREEN: skill's own cases ${own.length ? own.slice(0, 12).join(',') + (own.length > 12 ? ` (+${own.length - 12})` : '') : 'none'}`, + `SCREEN: ${why} → recommend --${recommend}`, ], }; } diff --git a/lib/ship-queue-train.ts b/lib/ship-queue-train.ts index 5485f9b20..d8077fc16 100644 --- a/lib/ship-queue-train.ts +++ b/lib/ship-queue-train.ts @@ -100,22 +100,22 @@ export const pinList = (slots: Array<{ pr: string; head: string }>) => slots.map export const prsArgs = (slots: Array<{ pr: string; head: string }>, explicit?: boolean) => `--prs ${pinList(slots)}${explicit ? ` --order ${slots.map(s => s.pr).join(',')}` : ''}`; /** - * Slot order: screened PRs first, then unscreened, each in READY order; - * enough unscreened PRs form their own train. An explicit order (`--order`, + * Slot order: screened PRs first, then unscreened PRs fill the remaining + * seats, each in READY order (GSTA-165: most PRs ride unscreened, so a + * screened-only train would hold the pool). An explicit order (`--order`, * every admitted PR once) replaces that rule: slots follow it, the first * `max` ride and the rest are held. */ -export function orderSlots(slots: TrainSlot[], max: number, explicit?: readonly string[]): { train: TrainSlot[]; held: TrainSlot[]; split: boolean } { +export function orderSlots(slots: TrainSlot[], max: number, explicit?: readonly string[]): { train: TrainSlot[]; held: TrainSlot[] } { if (explicit) { const all = explicit.map(pr => slots.find(s => s.pr === pr)!); - return { train: all.slice(0, max), held: all.slice(max), split: false }; + return { train: all.slice(0, max), held: all.slice(max) }; } const byReady = (a: TrainSlot, b: TrainSlot) => a.ready_at.localeCompare(b.ready_at) || Number(a.pr) - Number(b.pr); const screened = slots.filter(s => s.screen.choice === 'screen').sort(byReady); const unscreened = slots.filter(s => s.screen.choice !== 'screen').sort(byReady); - if (screened.length && unscreened.length >= max) return { train: screened.slice(0, max), held: [...screened.slice(max), ...unscreened], split: true }; const all = [...screened, ...unscreened]; - return { train: all.slice(0, max), held: all.slice(max), split: false }; + return { train: all.slice(0, max), held: all.slice(max) }; } /** When a train under its size may dispatch: its oldest READY plus the fill time; null when it may dispatch now. */ @@ -389,7 +389,7 @@ async function newTrain(env: TrainEnv, o: BuildOptions): Promise<{ t: TrainRecor const lines = [ ...admitted.lines, `ORDER: ${order.train.map((s, i) => `${i + 1}:#${s.pr}(${s.screen.choice === 'screen' ? 'screened' : 'unscreened'})`).join(' ')}${explicit ? ' (explicit --order)' : ''}`, - ...(order.held.length ? [`HELD: ${order.held.map(s => '#' + s.pr).join(',')} (${order.split ? 'unscreened PRs form their own train' : `over the train size ${env.train.max_size}`})`] : []), + ...(order.held.length ? [`HELD: ${order.held.map(s => '#' + s.pr).join(',')} (over the train size ${env.train.max_size})`] : []), ]; const baseSha = git(env.ctx.repoRoot, ['rev-parse', `${env.remote}/${env.ctx.base}`]).out; const key = sha256(JSON.stringify({ base: baseSha, slots: order.train.map(s => `${s.pr}@${s.head}`) })); diff --git a/package.json b/package.json index e973f1654..e4bc156c0 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.91.327", + "version": "1.91.328", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/test/gate-cost.test.ts b/test/gate-cost.test.ts index 155a94bfd..b73a1cd53 100644 --- a/test/gate-cost.test.ts +++ b/test/gate-cost.test.ts @@ -344,10 +344,10 @@ describe('READY arrivals and trains', () => { expect([percentile([5, 1, 3, 2, 4], 0.5), percentile([5, 1, 3, 2, 4], 0.9), percentile([], 0.5)]).toEqual([3, 5, 0]); }); - test('train composition: unscreened PRs form their own train at five, else screened first, else the fill limit', () => { + test('train composition: screened first, unscreened fill the remaining seats, else the fill limit', () => { const item = (enqueuedMs: number, screened: boolean) => ({ enqueuedMs, screened }); const pool = [item(0, true), item(1, true), ...[2, 3, 4, 5, 6].map(t => item(t, false))]; - expect(pickTrain(pool, 10, 5, 100)).toEqual(pool.slice(2)); + expect(pickTrain(pool, 10, 5, 100)).toEqual(pool.slice(0, 5)); const mixed = [item(5, false), item(0, true), item(1, false), item(2, true), item(3, true)]; expect(pickTrain(mixed, 10, 5, 100)?.map(x => x.screened)).toEqual([true, true, true, false, false]); expect(pickTrain([item(0, false), item(1, true)], 99, 5, 100)).toBeNull(); diff --git a/test/ship-queue-train-logic.test.ts b/test/ship-queue-train-logic.test.ts index 15bae88b2..284ac052d 100644 --- a/test/ship-queue-train-logic.test.ts +++ b/test/ship-queue-train-logic.test.ts @@ -28,17 +28,17 @@ describe('ordering and fill', () => { const o = orderSlots([slot('4', false, '2026-10-18T01:00:00Z'), slot('3', true, '2026-10-18T03:00:00Z'), slot('2', false, '2026-10-18T00:00:00Z'), slot('1', true, '2026-10-18T02:00:00Z')], 5); expect(o.train.map(s => s.pr)).toEqual(['1', '3', '2', '4']); expect(o.held).toEqual([]); - expect(o.split).toBe(false); }); - test('when as many unscreened PRs wait as fill a train, they form their own train; over-size PRs are held in order', () => { - const unscreened = ['5', '6', '7'].map((pr, i) => slot(pr, false, `2026-10-18T0${i}:00:00Z`)); - const split = orderSlots([slot('9', true, '2026-10-18T05:00:00Z'), ...unscreened], 3); - expect(split.split).toBe(true); - expect(split.train.map(s => s.pr)).toEqual(['9']); - expect(split.held.map(s => s.pr)).toEqual(['5', '6', '7']); + test('unscreened PRs fill the seats a screened PR leaves, however many wait; over-size PRs are held in order', () => { + const unscreened = ['5', '6', '7', '10', '11'].map((pr, i) => slot(pr, false, `2026-10-18T0${i}:00:00Z`)); + const mixed = orderSlots([slot('9', true, '2026-10-18T05:00:00Z'), ...unscreened], 5); + expect(mixed.train.map(s => s.pr)).toEqual(['9', '5', '6', '7', '10']); + expect(mixed.held.map(s => s.pr)).toEqual(['11']); + const atThree = orderSlots([slot('9', true, '2026-10-18T05:00:00Z'), ...unscreened.slice(0, 3)], 3); + expect([atThree.train.map(s => s.pr), atThree.held.map(s => s.pr)]).toEqual([['9', '5', '6'], ['7']]); + expect(orderSlots(unscreened, 3).train.map(s => s.pr)).toEqual(['5', '6', '7']); const over = orderSlots([slot('8', true, '2026-10-18T06:00:00Z'), slot('9', true, '2026-10-18T05:00:00Z'), unscreened[0]!], 2); - expect(over.split).toBe(false); expect(over.train.map(s => s.pr)).toEqual(['9', '8']); expect(over.held.map(s => s.pr)).toEqual(['5']); }); @@ -68,7 +68,7 @@ describe('ordering and fill', () => { expect(orderSlots([docs, catalog], 5, order as string[]).train.map(s => s.pr)).toEqual(['3343', '3337']); const screened = slot('9', true, '2026-10-22T20:00:00Z'); const over = orderSlots([docs, catalog, screened], 2, ['3343', '3337', '9']); - expect([over.train.map(s => s.pr), over.held.map(s => s.pr), over.split]).toEqual([['3343', '3337'], ['9'], false]); + expect([over.train.map(s => s.pr), over.held.map(s => s.pr)]).toEqual([['3343', '3337'], ['9']]); expect(parseOrder(undefined, pins)).toBeUndefined(); for (const bad of ['3343', '3343,3337,3337', '3343,3337,1', '3343,x']) expect(parseOrder(bad, pins)).toEqual({ error: `--order must list every PR in --prs exactly once (--prs has #3337,#3343; got ${bad})` }); expect(prsArgs([{ pr: '3343', head: SHA }, { pr: '3337', head: SHA }], true)).toBe(`--prs 3343@${SHA},3337@${SHA} --order 3343,3337`); @@ -194,15 +194,32 @@ describe('screen recommendation', () => { expect(skillOwnCases(['a', 'b', 'c'], { a: ['review/**'], b: ['lib/**'], c: ['review/SKILL.md'] }, ['review'])).toEqual(['a', 'c']); }); - test("gstack's selector on gstack's own tree: a skill template change recommends --screen, a docs change --no-screen", async () => { + test("gstack's selector on gstack's own tree: only a change that restores every case recommends --screen", async () => { const has = (dir: string) => fs.existsSync(path.join(ROOT, dir, 'SKILL.md.tmpl')); const skill = await gstackPaidSelection(ROOT, 'HEAD', ['review/SKILL.md.tmpl'], has); - expect(skill.recommend).toBe('screen'); + expect(skill.recommend).toBe('no-screen'); expect(skill.lines[0]).toMatch(/^SCREEN: selection \d+ e2e case\(s\) \+ \d+ judge\(s\)/); - expect(skill.lines[1]).toMatch(/^SCREEN: skill's own cases \S+ → recommend --screen$/); + expect(skill.lines[1]).toMatch(/^SCREEN: skill's own cases \S+$/); + expect(skill.lines[2]).toMatch(/^SCREEN: selector mode (pr|dependents): the train gate's full census covers these cases → recommend --no-screen$/); + const shared = await gstackPaidSelection(ROOT, 'HEAD', ['bun.lock'], has); + expect(shared.recommend).toBe('screen'); + expect(shared.lines[2]).toMatch(/^SCREEN: selector mode full-fallback \(.*bun\.lock.*\) → recommend --screen$/); + expect((await gstackPaidSelection(ROOT, 'HEAD', ['review/SKILL.md.tmpl', 'bun.lock'], has)).recommend).toBe('screen'); const docs = await gstackPaidSelection(ROOT, 'HEAD', ['docs/ship-policy.md'], has); expect(docs.recommend).toBe('no-screen'); }, 60_000); + + // Generated by /ship Step 7 coverage audit (GSTA-165 1a). + // Value: protects=a diff the paid selector cannot select (needs full validation) recommends --screen with the full gate selection; + // fails_when=the selector-error branch stops forcing everyCase, so an unselectable diff rides the train unscreened; + // why_new=the existing selector test covers pr, dependents and full-fallback modes only, never the throw; + // seam=none; no_claim=dispatch of the screen itself (pure selection on gstack's own tree; no eval-bg run) + test("gstack's selector on gstack's own tree: a diff it cannot select recommends --screen with the full gate selection", async () => { + const r = await gstackPaidSelection(ROOT, 'HEAD', ['unregistered/nested/SKILL.md'], () => false); + expect(r.recommend).toBe('screen'); + expect(r.lines[0]).toMatch(/^SCREEN: selection \d+ e2e case\(s\) \+ \d+ judge\(s\) \(the PR profile needs full validation \(.*requires full validation.*\): the screen is the full gate selection\)$/); + expect(r.lines[2]).toBe('SCREEN: the selector could not select → recommend --screen'); + }, 60_000); }); describe('the train policy block', () => { diff --git a/test/ship-queue-train.test.ts b/test/ship-queue-train.test.ts index 15e24c277..ff77594b0 100644 --- a/test/ship-queue-train.test.ts +++ b/test/ship-queue-train.test.ts @@ -564,7 +564,7 @@ describe('train build', () => { expect(dispatches(fx, 'free-tests.yml').length).toBe(4); }, 180_000); - test('ordering: screened PRs first in READY order; enough unscreened PRs wait for their own train', () => { + test('ordering: screened PRs first in READY order; unscreened PRs fill the remaining seats', () => { const fx = make({ ...TRAIN_POLICY, train: { ...TRAIN_BLOCK, max_size: 2 } }); const h = ['1', '2', '3'].map(pr => openPr(fx, pr)); readyReceipt(fx, '1', h[0]!, { at: '2026-10-18T08:00:00.000Z' }); @@ -572,8 +572,8 @@ describe('train build', () => { readyReceipt(fx, '3', h[2]!, { at: '2026-10-18T09:00:00.000Z', screen: true }); const r = queue(fx, ['train', 'build', '--prs', `1@${h[0]},2@${h[1]},3@${h[2]}`, '--dry-run']); expect(r.status).toBe(EXIT.ok); - expect(line(r.out, 'ORDER:')).toBe('ORDER: 1:#3(screened)'); - expect(line(r.out, 'HELD:')).toBe('HELD: #2,#1 (unscreened PRs form their own train)'); + expect(line(r.out, 'ORDER:')).toBe('ORDER: 1:#3(screened) 2:#2(unscreened)'); + expect(line(r.out, 'HELD:')).toBe('HELD: #1 (over the train size 2)'); const wide = make(); const w = ['1', '2', '3'].map(pr => openPr(wide, pr)); readyReceipt(wide, '1', w[0]!, { at: '2026-10-18T08:00:00.000Z' }); From ec0010ba86db80ade8f7cb6ef8460982d6e1a0d8 Mon Sep 17 00:00:00 2001 From: garrytan Date: Fri, 30 Oct 2026 03:52:59 +0000 Subject: [PATCH 3/3] fix(ship): reviewers confirm suspected defects outside the hunks: callers, callees, tests, contracts; at most 8 reads in 2 turns (GSTA-167) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- capy/skill/gstack/SKILL.md | 2 +- docs/review-policy.md | 11 ++++++--- lib/review-prompts.ts | 4 ++-- ship/sections/adversarial.md | 2 +- ship/sections/review-army.md | 2 +- ship/sections/test-coverage.md | 2 +- ship/sections/test-coverage.md.tmpl | 2 +- test/review-packet.test.ts | 23 +++++++++++++++++++ .../ship/gsta-167-review-reads.json | 4 ++++ 9 files changed, 42 insertions(+), 10 deletions(-) create mode 100644 test/size-ledger/ship/gsta-167-review-reads.json diff --git a/capy/skill/gstack/SKILL.md b/capy/skill/gstack/SKILL.md index d39c2cf95..ed9ab9eb7 100644 --- a/capy/skill/gstack/SKILL.md +++ b/capy/skill/gstack/SKILL.md @@ -44,7 +44,7 @@ Capy checklist for `/autoplan`: drive `gstack-autoplan`, never a hand adaptation Token caps on /ship: pass a brief's cap as `--token-cap `. /ship reads `cost.tokens` from `tools.thread.get({ thread_id: "this" })` (one execute program returning only that number) into `gstack-ship-budget --reading ` and obeys `verdict=` (`docs/ship-budget.md`). -/ship reviewers on Capy (`docs/review-policy.md#reviewer-packets`): one execute program runs `gstack-review-packet build` with the plan run's flags, creates one subagent per `PACKET:` line (`tools.subagent.create` takes at most 10 per call) with `machine: "shared"` on the lane's model at reasoning `medium` (red-team and adversarial keep the lane's reasoning; a cross-family outside voice stays as it is), and starts at most the free slots of the running-subagent cap, retrying refused starts as slots free. Each brief is the packet brief with that packet's `read=` ranges, plus one more parallel call in turn 1: `install.sh --check --for review-voice` in bash. A reviewer then finishes in about three calls: the parallel reads, at most one turn of extra ranges, one write. Under a token cap, `gstack-ship-budget check --next` takes build's `BUDGET_NEXT:` value. The same program polls `tools.subagent.get` with `transcript: "none"` every ~20 s until every reviewer settles, then runs `gstack-review-packet collect --dir ` and returns only its output; keep one poll program under ~8 minutes and run it again until collect has run (a lost program loses no reviewer), and never read a reviewer's transcript. +/ship reviewers on Capy (`docs/review-policy.md#reviewer-packets`): one execute program runs `gstack-review-packet build` with the plan run's flags, creates one subagent per `PACKET:` line (`tools.subagent.create` takes at most 10 per call) with `machine: "shared"` on the lane's model at reasoning `medium` (red-team and adversarial keep the lane's reasoning; a cross-family outside voice stays as it is), and starts at most the free slots of the running-subagent cap, retrying refused starts as slots free. Each brief is the packet brief with that packet's `read=` ranges, plus one more parallel call in turn 1: `install.sh --check --for review-voice` in bash. A reviewer then finishes in at most four turns: the parallel packet reads, up to two turns that confirm or rule out each suspected defect from what it depends on outside the hunks (callers, callees, covering tests, the config and contracts they read; grep and the read tool, at most 8 reads, batched in parallel), one write. Under a token cap, `gstack-ship-budget check --next` takes build's `BUDGET_NEXT:` value. The same program polls `tools.subagent.get` with `transcript: "none"` every ~20 s until every reviewer settles, then runs `gstack-review-packet collect --dir ` and returns only its output; keep one poll program under ~8 minutes and run it again until collect has run (a lost program loses no reviewer), and never read a reviewer's transcript. ## Plan, approve, ship diff --git a/docs/review-policy.md b/docs/review-policy.md index eae963672..c3e13a772 100644 --- a/docs/review-policy.md +++ b/docs/review-policy.md @@ -349,8 +349,13 @@ writes one packet per `DISPATCH:` line holding: lines), except the testing reviewer and the coverage audit for data under `test/` or `fixtures/`; - at most 8 KB of `--plan-file`, and past learnings for specialists; -- a read budget: at most 2 more named ranges (200 lines at most) of files in the diff, - in one turn, no repository search, no other packets, no transcript; +- a read rule: before reporting, the reviewer confirms or rules out each suspected defect + from what it depends on outside the hunks (the callers and callees of changed functions, + the tests that cover them, the config and contracts they read), with grep and the read + tool, at most 8 reads in at most 2 turns of parallel calls; no other packets, no + transcript (GSTA-167: under a budget of 2 ranges around the hunks, 7 recorded blockers + that needed context outside them were found by no reviewer); an audit keeps its own + budget (files in the diff, their tests and the files the plan names); - its output path and the reply `DONE crit= info=`; its last line is `END OF PACKET `. Each `PACKET:` line carries `budget_role=` (the `gstack-ship-budget` dispatch role: @@ -358,7 +363,7 @@ audits reserve as `specialist`, shards as their parent role) and `read=`, line r under 30,000 bytes each. Capy's read tool returns about 37,800 characters of a file and drops the rest without a notice, and shell output past about 35,000 characters spills to a log, so a reviewer reads its packet with the read tool, one parallel call per range, -and finishes in about three calls. `BUDGET_NEXT: dispatch:` is the value for +and finishes in at most four turns. `BUDGET_NEXT: dispatch:` is the value for `gstack-ship-budget check --next`. A partial build (`--only`) keeps other reviewers' entries only at the same head, diff base and diff content; any other build starts the manifest over and removes the old outputs, except a Red Team build (`--findings`), which refuses diff --git a/lib/review-prompts.ts b/lib/review-prompts.ts index bd74c2c15..1051c5319 100644 --- a/lib/review-prompts.ts +++ b/lib/review-prompts.ts @@ -74,7 +74,7 @@ export function renderFixCheckPrompt(i: FixCheckInput): string { } /** What every reviewer packet allows beyond itself. */ -export const READ_BUDGET = 'Everything you need is in this packet. You may read at most 2 more named line ranges, each at most 200 lines of a file in this diff (code around a hunk, a caller the diff changed), both in one turn as parallel calls. No whole-repository search, no git commands, no other packets, no transcript, no web.'; +export const READ_BUDGET = 'The packet is your starting context: the diff, checklists and contract. Before reporting, confirm or rule out each suspected defect by reading what it depends on outside the hunks: the callers and callees of changed functions, the tests that cover them, and the config/contracts they read. Use grep and the read tool freely for that, batching reads into parallel calls; at most 8 reads, in at most 2 turns. Never stop at the hunk when the defect would show elsewhere. Do not rerun git diff (the diff is below); no other packets, no transcript, no web.'; /** The native adversarial pass writes JSONL like the specialists, plus a classification and its Recommendation line. */ export const ADVERSARIAL_CONTRACT = 'Write each finding as one JSON object per line with the specialist fields (`"specialist":"adversarial"`) plus `"classification":"FIXABLE|INVESTIGATE"`, rated against the same floor (CRITICAL only for a test that would fail, a production break, or a security or data-loss defect); `NO FINDINGS` when nothing would fail. End with the one `Recommendation:` line either way.'; @@ -84,5 +84,5 @@ export const OUTSIDE_CONTRACT = 'Write one finding per line as `[Critical] path: /** The whole brief a packet reviewer receives; the packet carries the rest. */ export function packetBrief(file: string, out: string): string { - return `Turn 1: read ${file} with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turn 2, only if needed: at most 2 more ranges, in parallel. Last turn: write your result to ${out} in one write and reply with only the DONE line it names.`; + return `Turn 1: read ${file} with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turns 2 and 3, only if needed: the reads its Read budget allows, each turn's calls in parallel. Last turn: write your result to ${out} in one write and reply with only the DONE line it names.`; } diff --git a/ship/sections/adversarial.md b/ship/sections/adversarial.md index a0e34faf1..9d22801c0 100644 --- a/ship/sections/adversarial.md +++ b/ship/sections/adversarial.md @@ -98,7 +98,7 @@ Those files are part of the recorded content too. Dispatch via the Agent tool with `run_in_background: false` when available (background is the default since Claude Code v2.1.198); findings must arrive before review concludes. A launch receipt means it went background: await its completion notice. Fresh context avoids checklist bias, but this is the same harness, not an independent model unless runtime identity proves otherwise. -Build Step 11's packets from Step 9's last full-pass plan: `~/.claude/skills/gstack/bin/gstack-review-packet build --only adversarial,outside-voice --base --out-dir "$_PKA"` with `_PKA=.gstack/tmp/review-packets-adv-p$PASS`, that plan run's flags and `--range ` from pass 2 on. Leave `outside-voice` out of `--only` when nothing will run it (`CODEX_MODE` neither `ready` nor `unverified` and no reviewer on another model family): collect counts a built packet with no output as missing coverage. The `adversarial` packet carries the defensive framing, the native adversarial prompt (`ADVERSARIAL_NATIVE`), the diff with fixtures in summary mode and a JSONL contract (`NO FINDINGS` is the expected result for a clean range). Its whole brief is "Turn 1: read with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turn 2, only if needed: at most 2 more ranges, in parallel. Last turn: write your result to in one write and reply with only the DONE line it names." with that packet's `read=` ranges. +Build Step 11's packets from Step 9's last full-pass plan: `~/.claude/skills/gstack/bin/gstack-review-packet build --only adversarial,outside-voice --base --out-dir "$_PKA"` with `_PKA=.gstack/tmp/review-packets-adv-p$PASS`, that plan run's flags and `--range ` from pass 2 on. Leave `outside-voice` out of `--only` when nothing will run it (`CODEX_MODE` neither `ready` nor `unverified` and no reviewer on another model family): collect counts a built packet with no output as missing coverage. The `adversarial` packet carries the defensive framing, the native adversarial prompt (`ADVERSARIAL_NATIVE`), the diff with fixtures in summary mode and a JSONL contract (`NO FINDINGS` is the expected result for a clean range). Its whole brief is "Turn 1: read with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turns 2 and 3, only if needed: the reads its Read budget allows, each turn's calls in parallel. Last turn: write your result to in one write and reply with only the DONE line it names." with that packet's `read=` ranges. Present findings under an `ADVERSARIAL REVIEW (Claude subagent):` header. **FIXABLE findings** are queued for the parent; do not edit during Step 11. **INVESTIGATE findings** are presented as informational. diff --git a/ship/sections/review-army.md b/ship/sections/review-army.md index 6059542be..c391cf004 100644 --- a/ship/sections/review-army.md +++ b/ship/sections/review-army.md @@ -379,7 +379,7 @@ packet; the Red Team's waits for collect (below) and Step 11 builds its own. Launch one subagent per packet via the Agent tool (`subagent_type: "general-purpose"`, `run_in_background: false` when available; A launch receipt means it went background: await its completion notice.) at its `effort=` on the same model -family. The whole brief is "Turn 1: read with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turn 2, only if needed: at most 2 more ranges, in parallel. Last turn: write your result to in one write and reply with only the DONE line it names." with that packet's `read=` ranges. Never paste a diff, +family. The whole brief is "Turn 1: read with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turns 2 and 3, only if needed: the reads its Read budget allows, each turn's calls in parallel. Last turn: write your result to in one write and reply with only the DONE line it names." with that packet's `read=` ranges. Never paste a diff, checklist or finding into a brief. **Fan-out (`--review-fanout auto|serial|parallel` in the prompt; default `auto`).** diff --git a/ship/sections/test-coverage.md b/ship/sections/test-coverage.md index bd2cc6493..487b57666 100644 --- a/ship/sections/test-coverage.md +++ b/ship/sections/test-coverage.md @@ -19,7 +19,7 @@ run `~/.claude/skills/gstack/bin/gstack-review-packet build --audit test-coverag --out-dir .gstack/tmp/review-packets-audit --context .gstack/tmp/audit-context.txt --generation --passes-used ` (the packet holds the prompt below and the diff; an audit is one packet, never sharded) and brief the child only with the packet brief and its `read=` ranges: -"Turn 1: read with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turn 2, only if needed: at most 2 more ranges, in parallel. Last turn: write your result to in one write and reply with only the DONE line it names." +"Turn 1: read with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turns 2 and 3, only if needed: the reads its Read budget allows, each turn's calls in parallel. Last turn: write your result to in one write and reply with only the DONE line it names." Wait for its `out=` file's LAST-line JSON before applying the coverage gate. **Generation allowance:** Maximum 2 generation passes total per invocation. diff --git a/ship/sections/test-coverage.md.tmpl b/ship/sections/test-coverage.md.tmpl index 41841161a..1e6ea6bed 100644 --- a/ship/sections/test-coverage.md.tmpl +++ b/ship/sections/test-coverage.md.tmpl @@ -17,7 +17,7 @@ run `~/.claude/skills/gstack/bin/gstack-review-packet build --audit test-coverag --out-dir .gstack/tmp/review-packets-audit --context .gstack/tmp/audit-context.txt --generation --passes-used ` (the packet holds the prompt below and the diff; an audit is one packet, never sharded) and brief the child only with the packet brief and its `read=` ranges: -"Turn 1: read with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turn 2, only if needed: at most 2 more ranges, in parallel. Last turn: write your result to in one write and reply with only the DONE line it names." +"Turn 1: read with your file-reading tool (never cat: shell output past ~35 KB spills to a log), one parallel call per line range its PACKET line lists as read=; load no skill and read no SKILL.md. The packet is your whole task and ends with END OF PACKET. Turns 2 and 3, only if needed: the reads its Read budget allows, each turn's calls in parallel. Last turn: write your result to in one write and reply with only the DONE line it names." Wait for its `out=` file's LAST-line JSON before applying the coverage gate. **Generation allowance:** Maximum 2 generation passes total per invocation. diff --git a/test/review-packet.test.ts b/test/review-packet.test.ts index a9d23fdfe..5bde93d5b 100644 --- a/test/review-packet.test.ts +++ b/test/review-packet.test.ts @@ -59,6 +59,7 @@ describe('gstack-review-packet build', () => { for (const p of all) expect(p.text).not.toContain('UNRELATED_MARKER'); expect(b.stdout).toMatch(/^PACKETS: n=\d+ total_bytes=\d+ fingerprint=\S+$/m); expect(all.find(p => p.name === 'adversarial')!.text).toContain(P.ADVERSARIAL_NATIVE); + expect(all.find(p => p.name === 'adversarial')!.text).toContain(P.READ_BUDGET); }); test('the defensive framing reaches the security and adversarial packets with fixtures as a stat; testing reads fixtures in full', () => { @@ -87,6 +88,7 @@ describe('gstack-review-packet build', () => { expect(batched.text).toContain(checklist(c)); } expect(batched.text).toContain('Tag each finding with the checklist that found it'); + expect(batched.text).toContain(P.READ_BUDGET); }); test('exclusions are honored: maintainability leaves out docs and tests; testing keeps tests; untracked files are in', () => { @@ -139,6 +141,7 @@ describe('gstack-review-packet build', () => { expect(red.text).toContain(P.DEFENSIVE_REVIEW_FRAMING); expect(red.text).toContain(P.RED_TEAM_OUTPUT); expect(red.text).toContain(checklist('red-team')); + expect(red.text).toContain(P.READ_BUDGET); expect(red.text).not.toContain('+PAYLOAD_MARKER'); expect(red.text).toContain('test/fixture.test.ts'); const manifest = JSON.parse(fs.readFileSync(path.join(r.out, 'manifest.json'), 'utf8')); @@ -155,6 +158,7 @@ describe('gstack-review-packet build', () => { const b = build(r, [...flags, '--fix-list', w('fix.txt', 'FIX_LIST_MARKER'), '--tests', w('tests.txt', '12 pass')]); const fc = packets(b.stdout).find(p => p.role === 'fix-check')!; expect(fc.text).toContain(P.FIX_CHECK_INSTRUCTIONS); + expect(fc.text).toContain(P.READ_BUDGET); expect(fc.text).toContain('FIX_LIST_MARKER'); expect(fc.text).toContain('+export const FIXED = 1;'); expect(fc.text).not.toContain('+export const eps0 = 0;'); @@ -166,6 +170,8 @@ describe('gstack-review-packet build', () => { expect(a.text).toContain('You are running a ship-workflow test coverage audit. Run `git diff main`'); expect(a.text).toContain('Generation: audit-only; passes used: 1 of 2.'); expect(a.text).toContain('+export const zeta19 = 19;'); + expect(a.text).not.toContain(P.READ_BUDGET); + expect(a.text).toContain('Read only files in the diff, their tests and the files the plan names'); expect(build(r, ['--audit', 'plan-completion']).status).toBe(2); }); @@ -391,6 +397,23 @@ describe('the packet prose and the generated SKILL.md prose are one text', () => expect(read('ship/sections/adversarial.md')).toContain(P.packetBrief('', '')); expect(read('ship/sections/test-coverage.md')).toContain(P.packetBrief('', '')); }); + // Value: protects=every reviewer confirms or rules out a suspected defect from what it depends on outside the hunks (callers, callees, covering tests, config and contracts), in bounded batched reads, and the Capy reviewer paragraph and policy say the same; fails_when=the rule shrinks back to a fixed count of ranges around the hunks (GSTA-167 replay: 7 blockers needing outside context found by no arm) or loses its read and turn bound; + // why_new=the build tests only hold the packet equal to READ_BUDGET, never what it allows; seam=none; no_claim=that a model follows the rule + test('the read rule sends reviewers past the hunks to callers, callees, tests and contracts, bounded and batched', () => { + for (const clause of ['confirm or rule out each suspected defect', 'the callers and callees of changed functions, the tests that cover them, and the config/contracts they read', + 'Use grep and the read tool freely', 'batching reads into parallel calls; at most 8 reads, in at most 2 turns', 'Never stop at the hunk when the defect would show elsewhere']) expect(P.READ_BUDGET).toContain(clause); + expect(P.READ_BUDGET).not.toContain('at most 2 more'); + const brief = P.packetBrief('', ''); + expect(brief).not.toContain('2 more ranges'); + expect(brief).toContain('Turns 2 and 3, only if needed: the reads its Read budget allows, each turn\'s calls in parallel.'); + const capy = read('capy/skill/gstack/SKILL.md').replace(/\s+/g, ' '); + expect(capy).not.toContain('one turn of extra ranges'); + expect(capy).toContain('up to two turns that confirm or rule out each suspected defect from what it depends on outside the hunks'); + expect(capy).toContain('at most 8 reads'); + const policy = read('docs/review-policy.md').replace(/\s+/g, ' '); + expect(policy).not.toContain('at most 2 more named ranges'); + expect(policy).toContain('at most 8 reads in at most 2 turns'); + }); test('the outside voice boundary the safety-rule registry pins in the resolver is the packet\'s boundary', () => { const line = read('scripts/resolvers/outside-voice-steps.ts').split('\n').find(l => l.startsWith('const CODEX_BOUNDARY = '))!; expect(line).toBe(`const CODEX_BOUNDARY = '${P.CODEX_BOUNDARY_TEXT}\\\\n\\\\n';`); diff --git a/test/size-ledger/ship/gsta-167-review-reads.json b/test/size-ledger/ship/gsta-167-review-reads.json new file mode 100644 index 000000000..05b8a9fd9 --- /dev/null +++ b/test/size-ledger/ship/gsta-167-review-reads.json @@ -0,0 +1,4 @@ +{ + "bytes": 108, + "why": "GSTA-167: the reviewer packet brief allows up to two batched read turns past the hunks (rendered three times in ship sections)" +}