CalvinBackup/gstack
1
0 Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-11-12 03:30:28 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge main bf2cab3 into gsta258-l1-release-gate (rule 19 grant, GSTA-36)

Browse Source 
Resolve lib/ship-policy.ts against #3920: keep main's stamped
REVIEW_POLICY_KEY_VERSION and its review entry, and append release_gate
to the same POLICY_KEY_INTRODUCED line.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-11-03 19:08:00 +00:00
2 parents c0d33a2195 bf2cab34c5
commit e9277ef4d4
126 files changed
+4486 -457
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/TESTING_INTERNALS.md
+18
View File
@@ -1871,6 +1871,24 @@ own attempts. The arm gets:
- the same budget treatment: it is priced and reserved in the pilot like every
other arm.
**Masking.** On Capy, the owner's user-scope instructions tell every session to
load the user-drive gstack skill, and eval run agents inherit them: in one
checkpoint (24 runs, 3 arms) every run in every arm tried to read that skill in
its first minute. So `## No-gstack arm` also needs three lines, and `prereg
--check` refuses a missing, blank or `none` one (`PREREG_NO_GSTACK_ARM`):
- `masking:` an empty directory bind-mounted over `~/.capy/drive/user-garry-tan`
and `~/.capy/drive/org-gstack-gbrain` before the run agent starts, or a
container that sees no Capy drive;
- `install:` no gstack on the arm's machine (`~/.local/share/gstack`,
`~/.claude/skills`), checked before the run;
- `audit:` a per-arm audit of drive and skill reads.
An arm that read a gstack skill file is contaminated and gets zero credit. A
no-gstack result that ran before this rule without a mask is not evidence for
an R48 verdict until it is re-checked; the evidence ledger records it under
`prior` with `unmasked` ([docs/inventory.md](inventory.md)).
**Reporting.** The no-gstack result is one row beside the variants in every
results table and every verdict, including when it wins. A comparison that
drops the row has not reported its result. `gstack-eval-plan results --check
docs/inventory.md
+249
View File
@@ -0,0 +1,249 @@
# gstack inventory: every part, its load per run and its evidence
Generated by `bin/gstack-inventory` from [evidence.json](inventory/evidence.json), the hand-kept ledger, and lib/context-bill.ts. Measured at commit cd67c41d8 (2026-11-02). This is a dated snapshot: rerun `bin/gstack-inventory` to refresh it; `bin/gstack-inventory --check` validates the ledger. The rule: [ETHOS.md: 9. Earn Its Place](../ETHOS.md#9-earn-its-place).
Token counts are offline estimates (estimate: calibrated bytes/token per content class, about ±40%), not billed tokens.
| Status | Parts |
|---|---|
| helps | 0 |
| no-effect | 1 |
| hurts | 0 |
| untested, with a plan | 17 |
| untested, plan none | 48 |
| safety | 5 |
## Parts by load per run
Load per run: a skill's SKILL.md plus the files it must read on every use (`perInvocationTokens` in lib/context-bill.ts); a shared part's files. Tier 2+ skills also read the shared preamble once per session (its own row). On demand: the skill's other sections/ and references/ files, read when a step asks for them; files a shared row also counts, such as /ship's review sections, appear in both. Always-on: frontmatter every session's skill scanner reads.
| Rank | Part | Kind | Load per run (tokens) | On demand (tokens) | Always-on (tokens) | Loads | Status | Source, plan or reason |
|---|---|---|---|---|---|---|---|---|
| 1 | ship-review-panel | shared | 22,830 | n/a | n/a | every /ship run, by depth | untested | plan: GEVA-2 registry row R56-1 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 2 | design-review | skill | 20,159 | 7,778 | 101 | every use | untested | plan: GEVA-2 registry row R56-9 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 3 | ship | skill | 14,587 | 69,678 | 91 | every use | untested | plan: GEVA-2 registry row R56-4 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 4 | plan-ceo-review | skill | 14,394 | 21,203 | 313 | every use | untested | plan: GEVA-2 registry row R56-5 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 5 | review | skill | 13,990 | 41,146 | 75 | every use | untested | plan: R2 (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-02-geva-122-r2-amendment (#129, main eae6efc), told /review vs a no-gstack arm, Opus 5.5, preset on both arms (grandfathered) |
| 6 | devex-review | skill | 13,911 | 0 | 74 | every use | untested | plan: GEVA-2 registry row R56-11 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 7 | plan-design-review | skill | 13,351 | 12,986 | 79 | every use | untested | plan: GEVA-2 registry row R56-7 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 8 | retro | skill | 12,257 | 1,514 | 225 | every use | untested | plan: GEVA-2 registry row R56-12 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 9 | plan-devex-review | skill | 11,104 | 20,474 | 80 | every use | untested | plan: GEVA-2 registry row R56-6 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 10 | autoplan-phases | shared | 10,709 | n/a | n/a | every /autoplan run, by scope | untested | plan: folded into GEVA-2 registry row R56-3 (autoplan); proposed, not yet preregistered (grandfathered) |
| 11 | office-hours | skill | 10,424 | 23,078 | 280 | every use | untested | plan: none (grandfathered) |
| 12 | autoplan | skill | 10,288 | 18,701 | 394 | every use | untested | plan: GEVA-2 registry row R56-3 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 13 | plan-eng-review | skill | 9,837 | 25,963 | 74 | every use | untested | plan: GEVA-2 registry row R56-2 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 14 | design-consultation | skill | 9,557 | 10,864 | 282 | every use | untested | plan: GEVA-2 registry row R56-10 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 15 | setup-gbrain | skill | 9,099 | 10,921 | 106 | every use | untested | plan: none (grandfathered) |
| 16 | codex | skill | 8,991 | 10,358 | 68 | every use | untested | plan: none (grandfathered) |
| 17 | sync-gbrain | skill | 8,863 | 0 | 85 | every use | untested | plan: none (grandfathered) |
| 18 | design-shotgun | skill | 8,470 | 1,208 | 248 | every use | untested | plan: none (grandfathered) |
| 19 | plan-tune | skill | 8,377 | 0 | 107 | every use | untested | plan: none (grandfathered) |
| 20 | spec | skill | 8,305 | 7,759 | 92 | every use | untested | plan: none (grandfathered) |
| 21 | browse | skill | 8,125 | 5,597 | 81 | every use | untested | plan: none (grandfathered) |
| 22 | design-html | skill | 8,096 | 5,844 | 82 | every use | untested | plan: none (grandfathered) |
| 23 | land-and-deploy | skill | 7,652 | 16,188 | 63 | every use | untested | plan: GEVA-2 registry row R56-8 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 24 | benchmark | skill | 7,430 | 961 | 72 | every use | untested | plan: none (grandfathered) |
| 25 | catalog | shared | 7,291 | n/a | 7,291 | every session | untested | plan: Run D (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-01-geva-125-untold-discovery-prereg (#125, main 9820b14), untold gstack vs placebo vs no gstack, preset vs stub, Opus 5.5, GPT-6.1 Sol and particle-1; also measures /investigate's description (grandfathered) |
| 26 | investigate | skill | 7,086 | 3,729 | 490 | every use | no-effect | sources: GST-814 (HARN-1, 2026-11-02), org-gstack-gbrain handoffs/HARN-14/hard-follow-2026-11/decision-rule.md; R48 CUT-CANDIDATE |
| 27 | canary | skill | 7,034 | 0 | 66 | every use | untested | plan: none (grandfathered) |
| 28 | qa | skill | 6,951 | 13,877 | 74 | every use | untested | plan: none (grandfathered) |
| 29 | scrape | skill | 6,647 | 0 | 80 | every use | untested | plan: none (grandfathered) |
| 30 | captain | skill | 6,522 | 7,856 | 127 | every use | untested | plan: none (grandfathered) |
| 31 | preamble | shared | 5,743 | n/a | n/a | every tier 2+ skill run, inside that skill's load | untested | plan: I1 ceremony cut (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-01-geva-108-ceremony-cut-prereg (#122, main 409ff47) with its G3 amendment (#128), preset, Opus 5.5 and particle-1 (grandfathered) |
| 32 | gstack-upgrade | skill | 5,726 | 0 | 59 | every use | untested | plan: none (grandfathered) |
| 33 | context-restore | skill | 5,715 | 1,734 | 84 | every use | untested | plan: none (grandfathered) |
| 34 | document-generate | skill | 5,466 | 0 | 110 | every use | untested | plan: none (grandfathered) |
| 35 | ios-qa | skill | 5,368 | 1,378 | 84 | every use | untested | plan: none (grandfathered) |
| 36 | skillify | skill | 5,317 | 0 | 80 | every use | untested | plan: none (grandfathered) |
| 37 | capy-skill | shared | 5,309 | n/a | n/a | every Capy session | untested | plan: none (grandfathered) |
| 38 | health | skill | 5,287 | 0 | 68 | every use | untested | plan: none (grandfathered) |
| 39 | setup-deploy | skill | 5,216 | 0 | 73 | every use | untested | plan: none (grandfathered) |
| 40 | qa-only | skill | 5,205 | 3,460 | 68 | every use | untested | plan: none (grandfathered) |
| 41 | eval-plan | skill | 5,021 | 0 | 108 | every use | untested | plan: none (grandfathered) |
| 42 | gstack | skill | 4,951 | n/a | 59 | every use | untested | plan: none (grandfathered) |
| 43 | pair-agent | skill | 4,811 | 0 | 63 | every use | untested | plan: none (grandfathered) |
| 44 | make-pdf | skill | 4,730 | 1,767 | 65 | every use | untested | plan: none (grandfathered) |
| 45 | context-save | skill | 4,637 | 0 | 65 | every use | untested | plan: none (grandfathered) |
| 46 | cso | skill | 4,456 | 4,759 | 95 | every use | safety | reason: security: supported security findings with explicit coverage; kept on its security reason (GEVA-2 marks it exempt; GStack Minder may reclassify it) |
| 47 | open-gstack-browser | skill | 4,429 | 0 | 74 | every use | untested | plan: none (grandfathered) |
| 48 | design-motion | skill | 4,295 | 4,995 | 86 | every use | untested | plan: none (grandfathered) |
| 49 | test-audit | skill | 4,201 | 0 | 82 | every use | untested | plan: none (grandfathered) |
| 50 | document-release | skill | 4,049 | 10,260 | 72 | every use | untested | plan: none (grandfathered) |
| 51 | diagram | skill | 3,895 | 0 | 112 | every use | untested | plan: none (grandfathered) |
| 52 | benchmark-models | skill | 3,822 | 0 | 77 | every use | untested | plan: none (grandfathered) |
| 53 | ubicloud | skill | 3,593 | 0 | 83 | every use | untested | plan: none (grandfathered) |
| 54 | modernize | skill | 3,429 | 0 | 106 | every use | untested | plan: none (grandfathered) |
| 55 | setup-browser-cookies | skill | 3,184 | 0 | 79 | every use | untested | plan: none (grandfathered) |
| 56 | deslop-shared-libs | skill | 3,100 | 0 | 64 | every use | untested | plan: none (grandfathered) |
| 57 | landing-report | skill | 2,971 | 0 | 71 | every use | untested | plan: none (grandfathered) |
| 58 | ship-docs-audit | shared | 2,730 | n/a | n/a | every /ship run | untested | plan: folded into GEVA-2 registry row R56-4 (ship); proposed, not yet preregistered (grandfathered) |
| 59 | verify | skill | 2,679 | 0 | 91 | every use | untested | plan: none (grandfathered) |
| 60 | ios-design-review | skill | 2,367 | 0 | 76 | every use | untested | plan: none (grandfathered) |
| 61 | ios-sync | skill | 2,240 | 0 | 90 | every use | untested | plan: none (grandfathered) |
| 62 | ios-clean | skill | 2,214 | 0 | 86 | every use | untested | plan: none (grandfathered) |
| 63 | write-skill | skill | 2,158 | 795 | 100 | every use | untested | plan: none (grandfathered) |
| 64 | ios-fix | skill | 2,096 | 0 | 69 | every use | untested | plan: none (grandfathered) |
| 65 | learn | skill | 1,989 | 0 | 66 | every use | untested | plan: none (grandfathered) |
| 66 | careful | skill | 1,126 | 0 | 171 | every use | safety | reason: safety guard: warns before destructive commands (rm -rf, DROP TABLE, force-push); kept on that reason, an average score cannot see the rare bad run it stops |
| 67 | freeze | skill | 1,043 | 0 | 234 | every use | safety | reason: safety guard: hard-blocks edits outside one directory; kept on that reason |
| 68 | guard | skill | 908 | 0 | 348 | every use | safety | reason: safety guard: careful and freeze together; kept on that reason |
| 69 | capy-factory | shared | 846 | n/a | n/a | every Capy session | untested | plan: none (grandfathered) |
| 70 | unfreeze | skill | 372 | 0 | 65 | every use | safety | reason: safety guard: the off switch for freeze's edit restriction; kept with the guard it releases |
| 71 | claude-code | skill | n/a | n/a | n/a | every use | untested | plan: none (grandfathered) |
## Measure next
Untested parts, largest load per run first. A fair test runs the claude_code preset, frontier models (Opus 5.5 and GPT-6.1 Sol, plus at most one named low-cost primary) and a masked no-gstack arm ([TESTING_INTERNALS.md](TESTING_INTERNALS.md#no-gstack-arm)). Safety rows are not on this list.
| Order | Part | Load per run (tokens) | Plan |
|---|---|---|---|
| 1 | ship-review-panel | 22,830 | GEVA-2 registry row R56-1 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 2 | design-review | 20,159 | GEVA-2 registry row R56-9 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 3 | ship | 14,587 | GEVA-2 registry row R56-4 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 4 | plan-ceo-review | 14,394 | GEVA-2 registry row R56-5 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 5 | review | 13,990 | R2 (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-02-geva-122-r2-amendment (#129, main eae6efc), told /review vs a no-gstack arm, Opus 5.5, preset on both arms (grandfathered) |
| 6 | devex-review | 13,911 | GEVA-2 registry row R56-11 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 7 | plan-design-review | 13,351 | GEVA-2 registry row R56-7 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 8 | retro | 12,257 | GEVA-2 registry row R56-12 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 9 | plan-devex-review | 11,104 | GEVA-2 registry row R56-6 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 10 | autoplan-phases | 10,709 | folded into GEVA-2 registry row R56-3 (autoplan); proposed, not yet preregistered (grandfathered) |
| 11 | office-hours | 10,424 | none (grandfathered) |
| 12 | autoplan | 10,288 | GEVA-2 registry row R56-3 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 13 | plan-eng-review | 9,837 | GEVA-2 registry row R56-2 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 14 | design-consultation | 9,557 | GEVA-2 registry row R56-10 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 15 | setup-gbrain | 9,099 | none (grandfathered) |
| 16 | codex | 8,991 | none (grandfathered) |
| 17 | sync-gbrain | 8,863 | none (grandfathered) |
| 18 | design-shotgun | 8,470 | none (grandfathered) |
| 19 | plan-tune | 8,377 | none (grandfathered) |
| 20 | spec | 8,305 | none (grandfathered) |
| 21 | browse | 8,125 | none (grandfathered) |
| 22 | design-html | 8,096 | none (grandfathered) |
| 23 | land-and-deploy | 7,652 | GEVA-2 registry row R56-8 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 24 | benchmark | 7,430 | none (grandfathered) |
| 25 | catalog | 7,291 | Run D (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-01-geva-125-untold-discovery-prereg (#125, main 9820b14), untold gstack vs placebo vs no gstack, preset vs stub, Opus 5.5, GPT-6.1 Sol and particle-1; also measures /investigate's description (grandfathered) |
| 26 | canary | 7,034 | none (grandfathered) |
| 27 | qa | 6,951 | none (grandfathered) |
| 28 | scrape | 6,647 | none (grandfathered) |
| 29 | captain | 6,522 | none (grandfathered) |
| 30 | preamble | 5,743 | I1 ceremony cut (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-01-geva-108-ceremony-cut-prereg (#122, main 409ff47) with its G3 amendment (#128), preset, Opus 5.5 and particle-1 (grandfathered) |
| 31 | gstack-upgrade | 5,726 | none (grandfathered) |
| 32 | context-restore | 5,715 | none (grandfathered) |
| 33 | document-generate | 5,466 | none (grandfathered) |
| 34 | ios-qa | 5,368 | none (grandfathered) |
| 35 | skillify | 5,317 | none (grandfathered) |
| 36 | capy-skill | 5,309 | none (grandfathered) |
| 37 | health | 5,287 | none (grandfathered) |
| 38 | setup-deploy | 5,216 | none (grandfathered) |
| 39 | qa-only | 5,205 | none (grandfathered) |
| 40 | eval-plan | 5,021 | none (grandfathered) |
| 41 | gstack | 4,951 | none (grandfathered) |
| 42 | pair-agent | 4,811 | none (grandfathered) |
| 43 | make-pdf | 4,730 | none (grandfathered) |
| 44 | context-save | 4,637 | none (grandfathered) |
| 45 | open-gstack-browser | 4,429 | none (grandfathered) |
| 46 | design-motion | 4,295 | none (grandfathered) |
| 47 | test-audit | 4,201 | none (grandfathered) |
| 48 | document-release | 4,049 | none (grandfathered) |
| 49 | diagram | 3,895 | none (grandfathered) |
| 50 | benchmark-models | 3,822 | none (grandfathered) |
| 51 | ubicloud | 3,593 | none (grandfathered) |
| 52 | modernize | 3,429 | none (grandfathered) |
| 53 | setup-browser-cookies | 3,184 | none (grandfathered) |
| 54 | deslop-shared-libs | 3,100 | none (grandfathered) |
| 55 | landing-report | 2,971 | none (grandfathered) |
| 56 | ship-docs-audit | 2,730 | folded into GEVA-2 registry row R56-4 (ship); proposed, not yet preregistered (grandfathered) |
| 57 | verify | 2,679 | none (grandfathered) |
| 58 | ios-design-review | 2,367 | none (grandfathered) |
| 59 | ios-sync | 2,240 | none (grandfathered) |
| 60 | ios-clean | 2,214 | none (grandfathered) |
| 61 | write-skill | 2,158 | none (grandfathered) |
| 62 | ios-fix | 2,096 | none (grandfathered) |
| 63 | learn | 1,989 | none (grandfathered) |
| 64 | capy-factory | 846 | none (grandfathered) |
| 65 | claude-code | n/a | none (grandfathered) |
## Shared parts and preamble tiers
| Tier | Skills | What loads |
|---|---|---|
| 1 | benchmark, benchmark-models, browse, diagram, gstack, make-pdf, open-gstack-browser, scrape, setup-browser-cookies, verify | a short preamble inline in SKILL.md |
| 2 | canary, captain, context-restore, context-save, design-html, design-motion, design-shotgun, document-generate, document-release, eval-plan, health, investigate, ios-clean, ios-design-review, ios-fix, ios-sync, landing-report, learn, modernize, pair-agent, plan-tune, retro, setup-deploy, setup-gbrain, skillify, sync-gbrain, test-audit, ubicloud, write-skill | the shared sections/preamble.md (row preamble), once per session |
| 3 | autoplan, claude-code, codex, design-consultation, devex-review, ios-qa, office-hours, plan-ceo-review, plan-design-review, plan-devex-review, plan-eng-review, spec | the shared sections/preamble.md (row preamble), once per session |
| 4 | design-review, land-and-deploy, qa, qa-only, review, ship | the shared sections/preamble.md (row preamble), once per session |
| none | careful, cso, deslop-shared-libs, freeze, gstack-upgrade, guard, unfreeze | no preamble |
- **preamble**: shared preamble (sections/preamble.md, identical in every tier 2+ skill) (ship/sections/preamble.md); loads every tier 2+ skill run, inside that skill's load.
- **catalog**: skill catalog (every skill's frontmatter); loads every session.
- **capy-skill**: capy/skill/gstack/SKILL.md (capy/skill/gstack/SKILL.md); loads every Capy session.
- **capy-factory**: capy/skill/gstack/factory.md (capy/skill/gstack/factory.md); loads every Capy session.
- **ship-review-panel**: /ship review panel (review-army and adversarial sections) (ship/sections/review-army.md, ship/sections/adversarial.md); loads every /ship run, by depth.
- **ship-docs-audit**: /ship docs audit (Step 14.5) (ship/sections/documentation.md); loads every /ship run.
- **autoplan-phases**: /autoplan CEO, design, DX and eng phases (autoplan/sections/ceo-phase.md, autoplan/sections/design-phase.md, autoplan/sections/dx-phase.md, autoplan/sections/eng-phase.md); loads every /autoplan run, by scope.
## Paid dispatches per run
Each count is quoted from the file and line named; `bin/gstack-inventory --check` fails when a quoted line moves away.
| Skill | What | Count | Source |
|---|---|---|---|
| ship | Step 9 review, pass 1, light depth, docs or tests only | 1 reader (the core checklist) | docs/review-policy.md:194 |
| ship | Step 9 review, pass 1, light depth, code | 3 readers (core, one batched specialist, native adversarial), plus the outside voice on a diff that publishes numbers | docs/review-policy.md:195 |
| ship | Step 9 review, pass 1, standard depth | the batched specialist, the Red Team at 200 runtime lines, adversarial and outside voice | docs/review-policy.md:196 |
| ship | Step 9 review, pass 1, full depth | the batched specialist on the whole diff and the Red Team always | docs/review-policy.md:197 |
| ship | targeted reader on money, gate, auth or secret code | +1 at every depth | docs/review-policy.md:319 |
| ship | audits (Steps 7, 8 and 14.5) | 3 | lib/ship-budget-calibration.json:3 |
| ship | measured: one /ship child (gstack-evals#31) | 10 review dispatches | lib/ship-budget-calibration.json:43 |
| ship | prior cost of one dispatch on the Capy meter | 7,430,000 tokens | lib/ship-budget-calibration.json:56 |
| autoplan | voices per phase (native and outside) | 2 | autoplan/DRIVER.md.tmpl:25 |
| autoplan | phases per round: CEO and Eng always, Design with UI scope, DX with developer-facing scope | 2 to 4, so 4 to 8 voice dispatches a round | autoplan/SKILL.md.tmpl:99 |
| autoplan | CEO spec review loop | up to 3 launches | autoplan/sections/ceo-phase.md.tmpl:50 |
| autoplan | review rounds per phase (new unattended runs) | up to 2 | lib/autoplan-budget.ts:23 |
| autoplan | reserved per voice on Capy or under a spend cap | $5 | lib/autoplan-budget.ts:25 |
## Earlier results that do not count yet
- `stub`: ran on the Agent SDK stub prompt, not the claude_code preset.
- `small-tier`: Haiku, Sonnet or another non-frontier model only.
- `unmasked`: no mask recorded: the no-gstack arm predates the masking rule and was not re-checked.
- `ceiling`: control at the ceiling, so no difference could show.
- `no-true-no-gstack-arm`: the none arm still had gstack installed.
- `no-exposure`: the model never used the part.
- `not-claude-code`: ran outside Claude Code (Messages API, Codex or Capy).
- `variant-only`: variant against variant, no no-gstack arm.
| Part | Result | Why it does not count | Note |
|---|---|---|---|
| investigate | gstack-evals #109 stacks round 1 TB2 (GEVA-89, GEVA-93), wiki/board/2026-11-01-geva-93-stacks-round-1-tb2-results | stub, no-exposure, unmasked | Opus +3 points [-3, +9] at 1.9x cost; debug report in 0 of 111 trials; particle-1 -5 [-14, +1] |
| investigate | GSTA-12 Lane B quick look 2, wiki/board/2026-10-29-gsta-12-laneb-quick-2-results | stub, unmasked | Opus gstack minus none -7 [-13, -2] at 1.7x cost; did not replicate |
| investigate | GSTA-30 Lane B quick look 3 (#89), wiki/board/2026-10-30-gsta-30-laneb-quick-3-results | stub, small-tier | gstack arms on Sonnet, particle-1, DeepSeek and Haiku; Opus ran the none arm only |
| investigate | HARN-14 replication (harness-evals #181, unmerged) | stub, small-tier | Haiku 5.5: gstack minus none +3.5 [-12.3, +20.0] |
| learn | GSTA-13, GEVA-54 and GEVA-65 Lane A runs (wiki/board/2026-10-28-gsta-13-quick-look-3-results and the GEVA-54 and GEVA-65 posts) | stub, no-true-no-gstack-arm, ceiling | Opus contract tasks 27/30 vs 27/30 |
| learn | GEVA-65 Lane A rerun row 5 (GPT-6.1 Sol in Codex) | not-claude-code, no-true-no-gstack-arm | Sol failed the no-code-change safety task in 5 of 9 skill trials vs 0 of 3 without; small n, a lead to check, not a hurts verdict |
| office-hours | GSTA-13 quick look 3 Lane A, wiki/board/2026-10-28-gsta-13-quick-look-3-results | stub, no-true-no-gstack-arm | contract tasks on Opus: gstack 74% vs none 28%; the tasks grade the skill's own contract, a manipulation check, not lift |
| office-hours | GEVA-65 Lane A rerun, wiki/board/2026-10-31-geva-65-lanea-shakedown-rerun-results | stub, no-true-no-gstack-arm | Opus before 0.67, after 0.76, none 0.36 |
| plan-ceo-review | CEO panel v2 admission (GEVA-142), wiki/board/2026-11-02-geva-142-ceo-panel-v2-admission-results (open PR #131) | ceiling, unmasked | preset, Opus 5.5 only: must-fix recall 1.00 in all three arms; plain invocation ran the procedure in 0 of 16 |
| plan-ceo-review | Planning R1a pilot (GEVA-102), wiki/board/2026-11-01-geva-102-planning-r1a-pilot-stop (#114) | stub, ceiling, unmasked | none-opus 8/8; gstack-opus read the sections file in 0 of 4 |
| review | gstack-evals #108 stacks round 1 review (GEVA-89), wiki/board/2026-11-01-geva-89-stacks-round-1-review-results | stub, no-exposure, unmasked | Opus 5.5 blocking recall 9.5% vs 7.1%, +2.4 points [-6.4, +11.1]; specialists and adversarial pass ran in 0 of 66 sessions |
| review | gstack-evals #120 R1 blind re-judge (GEVA-113), wiki/board/2026-11-01-geva-113-r1-blind-rejudge-results | stub, unmasked | gstack-opus minus none-opus +3.5 points [-9.6, +16.7] |
| review | GSTA-11 quick look 1 review replay, wiki/board/2026-10-28-gsta-11-review-replay-results | not-claude-code, unmasked | /review's prompts through the Messages API; Opus recall 44% vs 39% plain, 12 bugs |
| preamble | GEVA-100 exposure counts, org-gstack-gbrain/research/why-null-stacks-r1.md (GEVA-100) | stub, no-exposure | ceremony sections ran in 0 of 66 Opus review sessions and 6 of 111 Opus TB2 trials; no effect estimate |
| catalog | GSTA-12 Lane B quick look 1, wiki/board/2026-10-29-gsta-12-laneb-quick-results | stub, no-exposure, unmasked | untold skill opened in 0 of 180 gstack-arm trials; Opus gstack minus none +5 [-3, +17] |
| ship-review-panel | gstack-evals #120 R1 blind re-judge (GEVA-113), wiki/board/2026-11-01-geva-113-r1-blind-rejudge-results | stub, not-claude-code, unmasked | arm C minus none +56.9 points [40.5, 72.2]; the none arm ran on the stub, arm C in Capy at another gstack version, on a cohort mined from gstack's own catches |
| ship-review-panel | GSTA-48 review no-loss arm D (#77), wiki/board/2026-10-30-gsta-48-review-noloss-arm-how-to | variant-only, not-claude-code | arm D vs pooled arm C, no no-gstack arm |
## Key list and aliases
71 keys: 64 skills (the folders `discoverTemplates()` finds, the root skill as `gstack`) and 7 shared parts.
- Skills: autoplan, benchmark, benchmark-models, browse, canary, captain, careful, claude-code, codex, context-restore, context-save, cso, design-consultation, design-html, design-motion, design-review, design-shotgun, deslop-shared-libs, devex-review, diagram, document-generate, document-release, eval-plan, freeze, gstack, gstack-upgrade, guard, health, investigate, ios-clean, ios-design-review, ios-fix, ios-qa, ios-sync, land-and-deploy, landing-report, learn, make-pdf, modernize, office-hours, open-gstack-browser, pair-agent, plan-ceo-review, plan-design-review, plan-devex-review, plan-eng-review, plan-tune, qa, qa-only, retro, review, scrape, setup-browser-cookies, setup-deploy, setup-gbrain, ship, skillify, spec, sync-gbrain, test-audit, ubicloud, unfreeze, verify, write-skill.
- Shared: preamble, catalog, capy-skill, capy-factory, ship-review-panel, ship-docs-audit, autoplan-phases.
- Installed names: `gstack-<name>` is key `<name>` (`gstack-ship` is `ship`); `gstack-upgrade` keeps its name; the root skill installs as `gstack`.
- Other names (the ledger's `aliases`): gstack (root skill) → gstack; upgrade → gstack-upgrade; preamble (ceremony) → preamble; always-on skill catalog (descriptions) → catalog; capy/skill/gstack/SKILL.md (Capy context) → capy-skill; capy/skill/gstack/factory.md (Capy context) → capy-factory; ship: review panel (specialist, adversarial, targeted, outside voice, red team) → ship-review-panel; ship: docs audit (Step 14.5) → ship-docs-audit; autoplan: CEO, design, DX and eng phases, outside voice → autoplan-phases.
## Not in this inventory
- Out of scope: openclaw/skills/* and browser-skills/* (not gstack skills an agent loads from this checkout), and connect-chrome (a symlink to open-gstack-browser, counted once).
- Evidence-table row "review: Review Army and adversarial pass": a section inside /review, not its own key; R2 measures it as part of the review row's plan.
- Evidence-table row "ship: QA probes (Step 9.2.1)": a step inside /ship with no separate size; folded into the ship row (GEVA-2 registry row R56-4).
- Evidence-table row "gstack as a whole (installed, any skill)": not a single text part; GEVA-2's real-world outcome lane measures it (gstack-evals plan #111), not a measurement plan for any one part.
docs/inventory/evidence.json
+661
View File
@@ -0,0 +1,661 @@
{
"schema_version": 1,
"note": "R56 evidence ledger (ETHOS.md: 9. Earn Its Place). One row per inventory key (docs/inventory.md lists them); bin/gstack-inventory --check validates it and test/inventory.test.ts runs that check. Statuses and their required fields: helps, no-effect, hurts need sources, models (frontier per R54), prompt: preset, scope and masked: true, and no-effect and hurts also r48 {verdict, ref}; untested needs plan (a reference, or none only when grandfathered), and a new part also reason, owner and measure_by; safety needs reason. prior lists earlier results that do not count yet, with why. Seeded 2026-11-02 from GEVA-2's table (org-gstack-gbrain/research/gstack-part-evidence.json, gstack dac71371a); every row present at that base is grandfathered.",
"seed_source": "org-gstack-gbrain/research/gstack-part-evidence.json (GEVA-156 for GST-915, 2026-11-02, gstack-evals main 409ff47)",
"parts": {
"autoplan": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-3 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: autoplan: untested (never measured)"
},
"benchmark": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: benchmark: untested (never measured)"
},
"benchmark-models": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: benchmark-models: untested (never measured)"
},
"browse": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: browse: untested (never measured)"
},
"canary": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: canary: untested (never measured)"
},
"captain": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: captain: untested (never measured)"
},
"careful": {
"status": "safety",
"grandfathered": true,
"reason": "safety guard: warns before destructive commands (rm -rf, DROP TABLE, force-push); kept on that reason, an average score cannot see the rare bad run it stops",
"seed": "GEVA-2, 2026-11-02: careful: exempt (safety guard)"
},
"claude-code": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: claude-code: untested (never measured)"
},
"codex": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "not in GEVA-2's table (2026-11-02); never measured"
},
"context-restore": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: context-restore: untested (never measured)"
},
"context-save": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: context-save: untested (never measured)"
},
"cso": {
"status": "safety",
"grandfathered": true,
"reason": "security: supported security findings with explicit coverage; kept on its security reason (GEVA-2 marks it exempt; GStack Minder may reclassify it)",
"seed": "GEVA-2, 2026-11-02: cso: exempt (security audit skill, treated as exempt; GStack Minder may reclassify)"
},
"design-consultation": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-10 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: design-consultation: untested (never measured)"
},
"design-html": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: design-html: untested (never measured)"
},
"design-motion": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: design-motion: untested (never measured)"
},
"design-review": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-9 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: design-review: untested (never measured)"
},
"design-shotgun": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: design-shotgun: untested (never measured)"
},
"deslop-shared-libs": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: deslop-shared-libs: untested (never measured)"
},
"devex-review": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-11 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: devex-review: untested (never measured)"
},
"diagram": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: diagram: untested (never measured)"
},
"document-generate": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: document-generate: untested (never measured)"
},
"document-release": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: document-release: untested (never measured)"
},
"eval-plan": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: eval-plan: untested (never measured)"
},
"freeze": {
"status": "safety",
"grandfathered": true,
"reason": "safety guard: hard-blocks edits outside one directory; kept on that reason",
"seed": "GEVA-2, 2026-11-02: freeze: exempt (safety guard)"
},
"gstack": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: gstack (root skill): untested (never measured)",
"aliases": [
"gstack (root skill)"
]
},
"gstack-upgrade": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: upgrade: untested (never measured)",
"aliases": [
"upgrade"
]
},
"guard": {
"status": "safety",
"grandfathered": true,
"reason": "safety guard: careful and freeze together; kept on that reason",
"seed": "GEVA-2, 2026-11-02: guard: exempt (safety guard)"
},
"health": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: health: untested (never measured)"
},
"investigate": {
"status": "no-effect",
"grandfathered": true,
"sources": [
"GST-814 (HARN-1, 2026-11-02)",
"org-gstack-gbrain handoffs/HARN-14/hard-follow-2026-11/decision-rule.md"
],
"models": [
"claude-opus-5-5"
],
"prompt": "preset",
"scope": "skill step procedure follow rate: 0 of 46 gstack sessions ran any step (0 of 30 on the claude_code preset); body loaded every time; the outcome check voided at ceiling (no-gstack fixed 8 of 8)",
"masked": true,
"r48": {
"verdict": "CUT-CANDIDATE",
"ref": "GST-938 (GSTA-251 plans hooks-or-drop)"
},
"plan": "R3 (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-01-geva-101-r48-follow-ups-prereg (#112, main f0b81c0) and its amendment (open PR #133), told /investigate vs no gstack on SWE-bench Verified, preset, Opus 5.5 then GPT-6.1 Sol",
"prior": [
{
"ref": "gstack-evals #109 stacks round 1 TB2 (GEVA-89, GEVA-93), wiki/board/2026-11-01-geva-93-stacks-round-1-tb2-results",
"why": [
"stub",
"no-exposure",
"unmasked"
],
"note": "Opus +3 points [-3, +9] at 1.9x cost; debug report in 0 of 111 trials; particle-1 -5 [-14, +1]"
},
{
"ref": "GSTA-12 Lane B quick look 2, wiki/board/2026-10-29-gsta-12-laneb-quick-2-results",
"why": [
"stub",
"unmasked"
],
"note": "Opus gstack minus none -7 [-13, -2] at 1.7x cost; did not replicate"
},
{
"ref": "GSTA-30 Lane B quick look 3 (#89), wiki/board/2026-10-30-gsta-30-laneb-quick-3-results",
"why": [
"stub",
"small-tier"
],
"note": "gstack arms on Sonnet, particle-1, DeepSeek and Haiku; Opus ran the none arm only"
},
{
"ref": "HARN-14 replication (harness-evals #181, unmerged)",
"why": [
"stub",
"small-tier"
],
"note": "Haiku 5.5: gstack minus none +3.5 [-12.3, +20.0]"
}
],
"seed": "GEVA-2, 2026-11-02: investigate: measurement under way (R3)",
"note": "Only Opus 5.5 (claude-opus-5-5) was measured; GPT-6.1 Sol was not, so this verdict covers one of the two frontier models ETHOS.md 9 names."
},
"ios-clean": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: ios-clean: untested (never measured)"
},
"ios-design-review": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: ios-design-review: untested (never measured)"
},
"ios-fix": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: ios-fix: untested (never measured)"
},
"ios-qa": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: ios-qa: untested (never measured)"
},
"ios-sync": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: ios-sync: untested (never measured)"
},
"land-and-deploy": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-8 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: land-and-deploy: untested (never measured)"
},
"landing-report": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: landing-report: untested (never measured)"
},
"learn": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"prior": [
{
"ref": "GSTA-13, GEVA-54 and GEVA-65 Lane A runs (wiki/board/2026-10-28-gsta-13-quick-look-3-results and the GEVA-54 and GEVA-65 posts)",
"why": [
"stub",
"no-true-no-gstack-arm",
"ceiling"
],
"note": "Opus contract tasks 27/30 vs 27/30"
},
{
"ref": "GEVA-65 Lane A rerun row 5 (GPT-6.1 Sol in Codex)",
"why": [
"not-claude-code",
"no-true-no-gstack-arm"
],
"note": "Sol failed the no-code-change safety task in 5 of 9 skill trials vs 0 of 3 without; small n, a lead to check, not a hurts verdict"
}
],
"seed": "GEVA-2, 2026-11-02: learn: untested (unfair test: SDK stub; no true no-gstack arm; Opus at ceiling; Sol ran in Codex, not Claude Code)"
},
"make-pdf": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: make-pdf: untested (never measured)"
},
"modernize": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: modernize: untested (never measured)"
},
"office-hours": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"prior": [
{
"ref": "GSTA-13 quick look 3 Lane A, wiki/board/2026-10-28-gsta-13-quick-look-3-results",
"why": [
"stub",
"no-true-no-gstack-arm"
],
"note": "contract tasks on Opus: gstack 74% vs none 28%; the tasks grade the skill's own contract, a manipulation check, not lift"
},
{
"ref": "GEVA-65 Lane A rerun, wiki/board/2026-10-31-geva-65-lanea-shakedown-rerun-results",
"why": [
"stub",
"no-true-no-gstack-arm"
],
"note": "Opus before 0.67, after 0.76, none 0.36"
}
],
"seed": "GEVA-2, 2026-11-02: office-hours: untested (unfair test: SDK stub; no true no-gstack arm; tasks grade the skill's own contract, a manipulation check, not lift)"
},
"open-gstack-browser": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: open-gstack-browser: untested (never measured)"
},
"pair-agent": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: pair-agent: untested (never measured)"
},
"plan-ceo-review": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-5 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"prior": [
{
"ref": "CEO panel v2 admission (GEVA-142), wiki/board/2026-11-02-geva-142-ceo-panel-v2-admission-results (open PR #131)",
"why": [
"ceiling",
"unmasked"
],
"note": "preset, Opus 5.5 only: must-fix recall 1.00 in all three arms; plain invocation ran the procedure in 0 of 16"
},
{
"ref": "Planning R1a pilot (GEVA-102), wiki/board/2026-11-01-geva-102-planning-r1a-pilot-stop (#114)",
"why": [
"stub",
"ceiling",
"unmasked"
],
"note": "none-opus 8/8; gstack-opus read the sections file in 0 of 4"
}
],
"r48": {
"verdict": "FIX-METHOD",
"ref": "CEO panel v2 admission (GEVA-142, open PR #131): FIX-METHOD on the panel, a harder panel is planned"
},
"seed": "GEVA-2, 2026-11-02: plan-ceo-review: untested (unfair test: ceiling, control 12 of 12 so no MDD; plain invocation ran the procedure in 0 of 16)"
},
"plan-design-review": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-7 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: plan-design-review: untested (never measured)"
},
"plan-devex-review": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-6 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: plan-devex-review: untested (never measured)"
},
"plan-eng-review": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-2 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: plan-eng-review: untested (never measured)"
},
"plan-tune": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: plan-tune: untested (never measured)"
},
"qa": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: qa: untested (never measured)"
},
"qa-only": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: qa-only: untested (never measured)"
},
"retro": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-12 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: retro: untested (never measured)"
},
"review": {
"status": "untested",
"grandfathered": true,
"plan": "R2 (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-02-geva-122-r2-amendment (#129, main eae6efc), told /review vs a no-gstack arm, Opus 5.5, preset on both arms",
"prior": [
{
"ref": "gstack-evals #108 stacks round 1 review (GEVA-89), wiki/board/2026-11-01-geva-89-stacks-round-1-review-results",
"why": [
"stub",
"no-exposure",
"unmasked"
],
"note": "Opus 5.5 blocking recall 9.5% vs 7.1%, +2.4 points [-6.4, +11.1]; specialists and adversarial pass ran in 0 of 66 sessions"
},
{
"ref": "gstack-evals #120 R1 blind re-judge (GEVA-113), wiki/board/2026-11-01-geva-113-r1-blind-rejudge-results",
"why": [
"stub",
"unmasked"
],
"note": "gstack-opus minus none-opus +3.5 points [-9.6, +16.7]"
},
{
"ref": "GSTA-11 quick look 1 review replay, wiki/board/2026-10-28-gsta-11-review-replay-results",
"why": [
"not-claude-code",
"unmasked"
],
"note": "/review's prompts through the Messages API; Opus recall 44% vs 39% plain, 12 bugs"
}
],
"r48": {
"verdict": "FIX-METHOD",
"ref": "org-gstack-gbrain/research/why-null-stacks-r1.md (GEVA-100); R2 is the fix"
},
"seed": "GEVA-2, 2026-11-02: review: measurement under way (R2)"
},
"scrape": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: scrape: untested (never measured)"
},
"setup-browser-cookies": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: setup-browser-cookies: untested (never measured)"
},
"setup-deploy": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: setup-deploy: untested (never measured)"
},
"setup-gbrain": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: setup-gbrain: untested (never measured)"
},
"ship": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-4 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"seed": "GEVA-2, 2026-11-02: ship: untested (never measured)"
},
"skillify": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: skillify: untested (never measured)"
},
"spec": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: spec: untested (never measured)"
},
"sync-gbrain": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: sync-gbrain: untested (never measured)"
},
"test-audit": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: test-audit: untested (never measured)"
},
"ubicloud": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: ubicloud: untested (never measured)"
},
"unfreeze": {
"status": "safety",
"grandfathered": true,
"reason": "safety guard: the off switch for freeze's edit restriction; kept with the guard it releases",
"seed": "GEVA-2, 2026-11-02: unfreeze: exempt (safety guard (undoes freeze))"
},
"verify": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "not in GEVA-2's table (2026-11-02); never measured"
},
"write-skill": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: write-skill: untested (never measured)"
},
"preamble": {
"status": "untested",
"grandfathered": true,
"plan": "I1 ceremony cut (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-01-geva-108-ceremony-cut-prereg (#122, main 409ff47) with its G3 amendment (#128), preset, Opus 5.5 and particle-1",
"prior": [
{
"ref": "GEVA-100 exposure counts, org-gstack-gbrain/research/why-null-stacks-r1.md (GEVA-100)",
"why": [
"stub",
"no-exposure"
],
"note": "ceremony sections ran in 0 of 66 Opus review sessions and 6 of 111 Opus TB2 trials; no effect estimate"
}
],
"r48": {
"verdict": "CUT-CANDIDATE",
"ref": "org-gstack-gbrain/research/why-null-stacks-r1.md (GEVA-100); taken by Garry as the I1 ceremony-cut eval"
},
"seed": "GEVA-2, 2026-11-02: preamble (ceremony): measurement under way (I1, ceremony-cut eval)",
"aliases": [
"preamble (ceremony)"
]
},
"catalog": {
"status": "untested",
"grandfathered": true,
"plan": "Run D (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-01-geva-125-untold-discovery-prereg (#125, main 9820b14), untold gstack vs placebo vs no gstack, preset vs stub, Opus 5.5, GPT-6.1 Sol and particle-1; also measures /investigate's description",
"prior": [
{
"ref": "GSTA-12 Lane B quick look 1, wiki/board/2026-10-29-gsta-12-laneb-quick-results",
"why": [
"stub",
"no-exposure",
"unmasked"
],
"note": "untold skill opened in 0 of 180 gstack-arm trials; Opus gstack minus none +5 [-3, +17]"
}
],
"seed": "GEVA-2, 2026-11-02: always-on skill catalog (descriptions): measurement under way (Run D)",
"aliases": [
"always-on skill catalog (descriptions)"
]
},
"capy-skill": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: capy/skill/gstack/SKILL.md (Capy context): untested (never measured)",
"aliases": [
"capy/skill/gstack/SKILL.md (Capy context)"
],
"note": "loads only in Capy sessions, which the claude_code preset fair test cannot reach yet; needs a Captain call on how to test it"
},
"capy-factory": {
"status": "untested",
"grandfathered": true,
"plan": "none",
"seed": "GEVA-2, 2026-11-02: capy/skill/gstack/factory.md (Capy context): untested (never measured)",
"aliases": [
"capy/skill/gstack/factory.md (Capy context)"
],
"note": "loads only in Capy sessions, which the claude_code preset fair test cannot reach yet; needs a Captain call on how to test it"
},
"ship-review-panel": {
"status": "untested",
"grandfathered": true,
"plan": "GEVA-2 registry row R56-1 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered)",
"prior": [
{
"ref": "gstack-evals #120 R1 blind re-judge (GEVA-113), wiki/board/2026-11-01-geva-113-r1-blind-rejudge-results",
"why": [
"stub",
"not-claude-code",
"unmasked"
],
"note": "arm C minus none +56.9 points [40.5, 72.2]; the none arm ran on the stub, arm C in Capy at another gstack version, on a cohort mined from gstack's own catches"
},
{
"ref": "GSTA-48 review no-loss arm D (#77), wiki/board/2026-10-30-gsta-48-review-noloss-arm-how-to",
"why": [
"variant-only",
"not-claude-code"
],
"note": "arm D vs pooled arm C, no no-gstack arm"
}
],
"seed": "GEVA-2, 2026-11-02: ship: review panel (specialist, adversarial, targeted, outside voice, red team): untested (unfair test: none arm on the SDK stub; arms from different harnesses and gstack versions; cohort mined from gstack's own catches)",
"aliases": [
"ship: review panel (specialist, adversarial, targeted, outside voice, red team)"
]
},
"ship-docs-audit": {
"status": "untested",
"grandfathered": true,
"plan": "folded into GEVA-2 registry row R56-4 (ship); proposed, not yet preregistered",
"seed": "GEVA-2, 2026-11-02: ship: docs audit (Step 14.5): untested (never measured)",
"aliases": [
"ship: docs audit (Step 14.5)"
]
},
"autoplan-phases": {
"status": "untested",
"grandfathered": true,
"plan": "folded into GEVA-2 registry row R56-3 (autoplan); proposed, not yet preregistered",
"seed": "GEVA-2, 2026-11-02: autoplan: CEO, design, DX and eng phases, outside voice: untested (never measured)",
"aliases": [
"autoplan: CEO, design, DX and eng phases, outside voice"
]
}
},
"unmapped": [
{
"row": "review: Review Army and adversarial pass",
"why": "a section inside /review, not its own key; R2 measures it as part of the review row's plan"
},
{
"row": "ship: QA probes (Step 9.2.1)",
"why": "a step inside /ship with no separate size; folded into the ship row (GEVA-2 registry row R56-4)"
},
{
"row": "gstack as a whole (installed, any skill)",
"why": "not a single text part; GEVA-2's real-world outcome lane measures it (gstack-evals plan #111), not a measurement plan for any one part"
}
]
}
docs/pregate.md
+69 -8
View File
@@ -86,7 +86,7 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`).
| `sealed` | sealed or eval output that must never reach a public destination; read by the pre-push hook and `gstack-pregate sealed` (see [sealed output](#sealed)) |
| `price_file` | repo-relative path of the paid gate's call-plan data file (`GSTACK_EVAL_CALLS v1`); `gstack-eval-plan price-gate` prices from it and never executes anything ([docs/eval-calls.md](eval-calls.md)) |
| `stall_timeout_s` | seconds a test command may print nothing while using under a tenth of one CPU before its process group is killed and the check is `incomplete` (`PREGATE_STALLED`); default 300, `0` disables; `--stall-timeout` overrides; the `--remote` gate is exempt (see [supervision](#supervision)) |
| `timeout_s` | seconds each runner process may take (the unit-file group, each serial file, each guard without its own `timeout_s`, and the remote gate's deadline) when `--timeout` is not passed; 1 to 3600, default 600. `gstack-ship-preflight` and the `finish` fallback run pregate without `--timeout`, so this is how a repo with long lanes sets theirs |
| `timeout_s` | seconds each runner process may take (the unit-file group, each serial file, each guard without its own `timeout_s`, and the remote gate's deadline) when `--timeout` is not passed; 1 to 3600, default 600. `gstack-ship-preflight` forwards its own `--timeout`; without it, and in the `finish` fallback, this is how a repo with long lanes sets theirs |
| `remote_gate` | the repo's remote test gate command; under 8 cores the local test tier becomes a `requires-remote` obligation naming it, and `--remote` runs it (see [remote gate](#remote-gate)) |
| `run_profiles` | name → `{ workflow, events, jobs, artifact, max_age_hours?, os?, skippable? }`: the GitHub Actions runs `clear --run` verifies as evidence on the identical tree; a profile named after a lane (`free-tests.yml/macos-named-regressions`) clears that lane (see [run evidence](#run-evidence)). `--from-run` and the main-push reuse job verify runs against them too ([from-run](#from-run), [CI reuse](#ci-reuse)) |
| `remote_gate_runs` | `{ profiles, inventory, skipped? }`: the run profiles that together clear `pregate.json/remote_gate`, the command that prints the remote gate's test files, one per line, which their runs must cover, and an optional command that prints which selected files the gate itself never runs (see [run evidence](#run-evidence)) |
@@ -173,7 +173,7 @@ The list lives in its own file, not in `pregate.json`, because older pregate rea
## `gstack-pregate`
```
gstack-pregate [--stage preflight|tests|all] [--only <check,..>] [--allow-repo-commands] [--policy-from base|head]
gstack-pregate [--stage preflight|tests|guards|all] [--only <check,..>] [--allow-repo-commands] [--policy-from base|head]
[--timeout <s>] [--stall-timeout <s>] [--remote --spend-cap <usd>] [--run <id>] [--from-run <id|url>] [--no-ci-carry] [--explain] [--json]
gstack-pregate verify | clear --remote <workflow.yml/job> --run-url <url> | lanes | validate | init
```
@@ -765,10 +765,12 @@ once:
such as `"per_file": "bun test --timeout=30000 {files}"`, the same form as the
tests stage's `runner`). That run is recorded under `<label>.skipped`, the
`LANE:` line adds `skipped_here=<n> file(s) tests_ran_here=<n>`, and the CI row
is written only after it passes with a test count above zero and every carried
run still stands (`run_changed` otherwise). A failure there is
`skipped_failed` and a run of zero tests is `skipped_zero_run`; either way the
whole lane runs here. An entry with no `per_file` refuses as
is written only after four checks pass. In order, they are the remainder's
exit (`skipped_failed`), its test count above zero (`skipped_zero_run`), every
carried run again (`run_changed`), and the working tree once more
(`tree_changed`, for an edit made during those checks). An edit made while the
remainder itself runs leaves its ledger row without a tree, so it shows as
`skipped_zero_run`. Any of these refusals runs the whole lane here. An entry with no `per_file` refuses as
`inventory_skipped`, naming the files, and the lane runs here whole. A partly
skipped file ran tests and stays covered. gstack's own Free Tests writer does
not emit the list yet: it starts in the same follow-up pull request that
@@ -1014,12 +1016,27 @@ a branch that does not contain the base tip it tested another tree and
`CI_CARRY` says `none (no_verified_run: … tree_mismatch)`: dispatch Free Tests
on the branch
(`gh workflow run free-tests.yml --ref <branch>`), which tests HEAD's own tree,
and wait again. The preflight stage (generated-file freshness, secrets, strays
and wait again. When the pull_request run tested that merge commit, the
refusal names the command for the run's branch (`fix: gh workflow run
free-tests.yml --ref <branch>, then gstack-ci-carry wait`). The preflight stage (generated-file freshness, secrets, strays
and the other mechanical checks) and the sealed guard are unchanged and always
run here. The macOS and Windows lane
obligations still clear with `gstack-pregate clear --remote <lane> --run-url
<url>` from their CI runs.
<a id="dispatch-carry"></a>**A dispatched run can hide a red merge commit.** A
`workflow_dispatch` run reports its checks on the head commit under the same
job names as the `pull_request` run. So a green dispatch run of HEAD's own tree
can stand in for a required check whose `pull_request` run failed on the merge
commit, and without strict branch
protection ("Require branches to be up to date before merging") the pull
request can merge with that failure hidden (GBEV-3). Turn on strict protection
for the base branch, or give dispatch runs distinct job names (for example a
job `name:` that includes `github.event_name`), so a dispatch run never stands
in for the required `pull_request` check. gstack's own merge trains test the
merged tree before they merge, so gstack is covered; a repo that merges
without a train is not.
**/ship lanes: `ship_lanes_from`.** A lane carries only when the base branch's
`.gstack/pregate.json` maps its exact command (the evidence ledger binds the
command's sha256; a label alone never carries) to the profiles whose runs carry
@@ -1123,6 +1140,9 @@ As a second guard, every `record-ci` row names the policy that mapped it
(`origin/<base>@<sha>:.gstack/pregate.json`), and `gstack-evidence carried`, which
the ship receipt reads, lists only rows whose policy is `origin/<the receipt's
base>`. A row written by a run against another base never lands in a receipt.
`carried` also lists a row only when `check`'s own grading, at the 24-hour age
limit /ship uses, calls it FRESH, so a carry that never reached FRESH (a failed
or zero-run row, or one older than 24 hours) leaves no `ci_carried` entry.
A carried lane prints `LANE: <label> carried run=<id>/<attempt> tests_ran=<n>`
(see above for what `<n>` counts) and writes an explicit CI-sourced ledger row (`gstack-evidence record-ci
@@ -1131,7 +1151,11 @@ itself, so no caller hands it run ids or a test count: `source: "ci"`, its runs
and profiles, the run's finish time as `ts`, bound to the clean tree), never a
fictitious local run. `gstack-evidence check` grades it
like any row and names `source=ci runs=…`, so `finish`'s FRESH check and the
24-hour age limit apply to the CI run's own time. A mapped lane that cannot
24-hour age limit apply to the CI run's own time. The ledger stores a label
with every character outside `A-Za-z0-9._-` turned into `_` (`pnpm lint` is
`pnpm_lint`, `db:test` is `db_test`), and `run`, `record-ci` and `check` all
use that one spelling, so `check --label 'pnpm lint'` finds the row; `ci_carried`
and `CI_CARRIED:` name the stored spelling. A mapped lane that cannot
carry prints `CI_CARRY: none label=<label> (<reason>)` and runs here. Every
other lane runs through `gstack-evidence run --reuse`: the newest row for its
label, command and working tree is reused when it passed with a counted run
@@ -1305,6 +1329,43 @@ guard is a *command* with its own fix line. Older gstack refuses unknown keys
that reads them ships. The worked gbrain configuration is in
[gbrain worked configs](#gbrain-worked-configs).
<a id="prepush"></a>
### Before every push: `--stage guards`
Housekeeping checks cause a large share of red CI runs: a stale render or
golden, a size ratchet, a token budget, a type error. gbrain had 64 of 93 red
heads from these (FOUN-64), and gstack's #3908 went red on the context-budget
ratchet. Each one is quick to run locally, so run them before the push instead
of waiting a CI round to find out:
```
gstack-pregate --stage guards --allow-repo-commands
```
`--stage guards` runs only two checks: `regen` (generated-file freshness) and
`guards`. Each runs only for the files the branch touched, so a docs-only
change runs nothing. It writes `.gstack/tmp/pregate-guards.json`, so it never
replaces the full run's `.gstack/tmp/pregate.json`. It refuses `--only`,
`--remote` and `--from-run`.
`gstack-ship-release push` runs it before every push, early or final, and
prints `GUARDS: <verdict> (<detail>)`. A failing or unfinished guard stops the
push with the guard's fix, and nothing is pushed. A pass already recorded at
HEAD's exact tree is reused instead of rerun: either push's own record or
`finish`'s preflight record. Both live under `.git`, where no commit can
place one. The guards must have checked the tree being pushed, so uncommitted
or untracked files that change what they saw stop the push until you commit or
stash them. A refused run (an unreadable policy, a base that is on origin but
not fetched, a mistyped `--base`) stops the push too; only a remote that has no branch except the one
being pushed skips the guards, as `GUARDS: not run`. Outside /ship, run the
command above yourself before `git push`.
gstack declares its own list in `.gstack/pregate.json`: `typecheck`,
`skill-size`, `parity-suite`, `context-budget`, `module-size`, `goldens` and
`doc-caps`. Generated SKILL.md freshness is the `regen` check's job. Other
repos add their own `guards` the same way; the key needs no newer gstack than
the one that already reads `guards`.
<a id="guard-carry"></a>
### A guard that already passed at this tree does not run again
docs/review-policy.md
+48 -3
View File
@@ -182,6 +182,46 @@ reads everything), one `SKIP:` per reviewer left out, and `--json` with the same
base as `base`).
A plan that fails closed to full exits 0; a diff git cannot list exits 1.
### Who reads (R65, the base policy's `review` key)
The depth says how much review a diff needs; the base branch's `.gstack/ship-policy.json`
`review` key says who does it (`review.panel`, default `all`: today's panel on every class). `gstack-review-plan`
prints `REVIEW_MODE: <lean|panel|platform> panel=<p> reader_model=<m> platform=<on|off>`
and, under `review.platform on`, `PLATFORM_REVIEW: <id> reviewer=capy-ai[bot] head=<sha>`
or `PLATFORM_REVIEW: none (<reason>)`.
Under the opt-in `review.panel risk` (and `review.platform on` under any panel):
| Diff | Pass 1 | After fixes (never more than 2 rounds) |
|---|---|---|
| Risk class: gate class, an R51 class (money, auth, secret, security, data), `eval` (scorer, judge and grader paths) or `policy` (`review.risk_paths`) | The full-depth plan as printed: the batched specialist, Red Team, native adversarial, outside voice and the targeted reader | A fixed blocker re-runs the printed plan on the fixed head as round 2; below-floor fixes get one fix-check |
| Non-risk code | One native adversarial reader (`DISPATCH: adversarial-native single`) at the lane's reasoning, on `review.reader_model` (`lane` by default), carrying the code checklists and the Red Team checklist; it also carries the targeted brief when the diff has a targeted class | A fixed blocker or a gate-class edit gets one fix-check on the touched hunks; below-floor fixes ship as notes |
| Docs and tests | The core read; the outside voice only when the diff publishes numbers | One fix-check after a fixed blocker |
| Docs or tests with no numeric, results or prereg claims, under `review.platform on`, with capy-ai[bot]'s review on the PR at HEAD | No reader: `NEXT: platform` and a planned `SKIP:` for every reader, the core read included | none |
`review.panel all`, the default, keeps today's panel and loop on every class. `risk` is
opt-in because its one-reader shape has not passed its no-loss replay: build C's replay
scored it FIX-METHOD (its readers saw the diff without the repository). `review.panel off` gives
risk class one merged reader too (`DISPATCH: adversarial-native merged`: the native
adversarial brief, the gate-class checklists, the Red Team checklist and the targeted
brief, whose ledger record is the receipt's `review_targeted`). The risk classes come
from paths and changed lines, never from a recorded `review_depth` (FOUN-64). Two
content rules: data and config files (JSON, YAML, TOML, CSV and the like) and analysis
scripts under `docs/`, `results/`, `benchmarks/`, `analysis/` or `notebooks/` are
judged by their path, so a `lease_usd` value or an `os.path.join(sys.argv[1], …)` in a
stats script names no class (HARN-1, harness-evals#231), and binary data such as
`*.sqlite` is data wherever it sits; a docs line that can print a credential (an
`Authorization:` header, a bearer or key-shaped value, an environment key set to a value,
a get or show command for auth config such as `mcp get`) is security (gbrain#8417); code
that reserves, binds, leases or settles spend against a ledger, budget or cap (`bindCheck(ledger, …)`,
`reserveSpend`, `bindCap`) is money (GST-1129, gstack-evals#170). When the matchers miss a class the
lane knows the diff has, `gstack-review-plan --declare-risk <classes>` adds it: a declaration joins the
detected classes, only raises the depth, and prints `RISK_DECLARED: classes=<declared> detected=<found>`.
The outside voice reads its diff from a file the brief names with its sha256; past
2,000,000 bytes it reads a path-scoped slice whose brief lists the files left out.
`--fix-checks <n>` tells the lean loop how many fix-checks ran, so the owner's grant
adds one fix-check after the first and never a second.
**Classes** are unchanged. A mixed diff takes its most restrictive class: **docs**
(markdown no agent or gate reads), **tests** (test files and fixtures), **code**
(everything else, including every `SKILL.md` and template, markdown in a skill
@@ -483,9 +523,14 @@ specialist stats they record). The summary line appends `fix_checks=`,
loop prints the old line.
Beside the counters, `write` records `review_depth` (the depth, runtime lines,
rules hash, base and registry commits, the gate scan bound to the head and the risk classes) and
the queue refuses a receipt below the depth its head needs
([ship-policy.md#ship-receipt](ship-policy.md#ship-receipt)). A stop the owner
rules hash, base and registry commits, the gate scan bound to the head, the risk classes and
`mode`: lean, panel or platform; with `--declare-risk`, `declared` and `detected` beside `risk`, so a
class the lane added shows apart from what the matchers found) and the queue refuses a receipt below the depth its head needs
([ship-policy.md#ship-receipt](ship-policy.md#ship-receipt)). After `NEXT: platform`,
`write --platform-review <id>` records `review_depth: platform` with `platform_review`
(capy-ai[bot]'s review id, reviewer and head); write and the queue accept it only under
`review.platform on`, on a docs or tests diff with no numeric, results or prereg claims,
with the review at that exact head, so a new head needs `/ship` again. A stop the owner
accepted after a fix-check edit goes in `review_loop_exception`, never in prose
under `REVIEW_LOOP:`. Step 16 takes the same record:
`gstack-ship-release finish --review-loop-exception <json|@file>` routes the
docs/skill-routing-and-contracts.md
+6 -6
View File
@@ -1,6 +1,6 @@
# Skill routing and skill contracts
Two opt-in Claude Code hooks that help a model actually run gstack's skills (GST-1088). Both are **off by default**, and with both off nothing changes.
Two Claude Code hooks that help a model actually run gstack's skills (GST-1088). Each has one default, `SWITCH_DEFAULT` in `lib/skill-route.ts`; `gstack-config defaults` prints it. With both off nothing changes.
- **Skill routing** (`skill_routing`, a UserPromptSubmit hook). When a prompt matches an installed skill's `triggers:`, the hook adds one line of context naming that skill, for example "this request fits /investigate. Run /investigate with the Skill tool before you start".
- **Skill contracts** (`skill_contracts`, a Stop hook). A skill can declare the outputs that must exist before it is done. When the model tries to stop without them, the hook blocks once and names what is missing.
@@ -12,9 +12,9 @@ Each switch resolves in this order, and the first one set wins:
1. The `./setup` flag: `--skill-routing` / `--no-skill-routing` and `--skill-contracts` / `--no-skill-contracts` (also `--skill-routing=on|off`). A flag is saved to config, so it survives upgrades.
2. The environment: `GSTACK_SKILL_ROUTING` and `GSTACK_SKILL_CONTRACTS` (`on` or `off`). Not saved.
3. Config: `gstack-config set skill_routing on|off` and `gstack-config set skill_contracts on|off`.
4. The default: off.
4. The default: `SWITCH_DEFAULT` in `lib/skill-route.ts`, the only place it is written. Setup, the hooks, `gstack-config get|list|defaults` and `gstack-skill-route --default <switch>` all read it.
`on` also accepts yes, true and 1; `off` accepts no, false and 0. Setup warns about any other value, uses off for that run, and does not save it.
`on` also accepts yes, true and 1; `off` accepts no, false and 0. Setup warns about any other value, uses the default for that run, and does not save it; the hooks also use the default for such a value. A config file that exists but cannot be read turns both hooks off in setup and in the hooks, whatever the default, because it may hold an `off` you set.
Setup registers a hook only when its switch is on, prints one line the first time it registers it, and removes the registration when the switch is off (or with `--no-team`). The hooks also read the switch every time they run, so this takes effect on the next prompt without re-running setup:
@@ -73,7 +73,7 @@ Every item has an `id` (lowercase letters, digits, `.`, `_`, `-`) and a `kind`,
| kind | Fields | Met when |
|---|---|---|
| `report_labels` | `heading`, `labels[]` | The last assistant message containing the heading has a `Label: value` line for every label, and no value is empty or a placeholder (`[...]`, `<...>`, TBD, TODO, `...`, or a template like `DONE \| BLOCKED`) |
| `check_after_last_edit` | optional `match` (regular expression) | No Edit, Write, MultiEdit or NotebookEdit happened, or a Bash test or repro command ran after the last one. By default a command counts when, with quoted text removed, one of its parts (split on `&&`, `\|\|`, `;`, `\|`) starts with a test runner, build tool or interpreter (bun, npm, pytest, go, cargo, make, node, python, curl, ...) or a script whose name says test, spec, repro, check, verify or lint, directly or behind a wrapper (`bundle exec rspec`, `uv run pytest`, `docker compose run app pytest`, `xvfb-run npm test`, `bash scripts/test.sh`). `git commit --no-verify`, `cat test/a.test.ts` and the shell's `test -f` do not count |
| `check_after_last_edit` | optional `match` (regular expression) | No Edit, Write, MultiEdit or NotebookEdit happened, or a Bash test or repro command ran after the last one. By default a command counts when, with quoted text removed, one of its parts (split on `&&`, `\|\|`, `;`, `\|`) runs a test runner (pytest, jest, vitest, rspec, tsc, ...), a test or check verb (`npm test`, `go test`, `cargo run`, `make check`, `bun run lint`), code through an interpreter (`node repro.js`, `python -c ...`, `bun scripts/repro.ts`), `curl` against a URL, or a script whose name says test, spec, repro, check, verify or lint, directly or behind a wrapper (`bundle exec rspec`, `uv run pytest`, `docker compose run app pytest`, `xvfb-run npm test`, `bash scripts/test.sh`). Installs, builds, cleans and version probes (`bun install`, `npm install`, `go build`, `make clean`, `node --version`), `git commit --no-verify`, `cat test/a.test.ts` and the shell's `test -f` do not count |
| `selected_specialists_dispatched` | optional `marker`, `empty_marker` | The selection line was printed in an assistant message or a Bash result (never a fetched page or file), its names are plain (letters, digits, spaces, `_`, `.`, `-`, at most 40 characters), and each specialist it names has an Agent dispatch that mentions it. The marker is literal text in which a whole-word N stands for a number; the default is `Dispatching N specialists:`. N = 0 (`Dispatching 0 specialists: (small diff)`) or a "specialists skipped" line means zero specialists were selected, which passes. Text in parentheses is not a name (`Dispatching 1 specialists: State Machines (small diff)` selects State Machines) |
| `agent_dispatch` | `match` (regular expression, case-insensitive), optional `marker` | The `marker` line (literal text, like /review's `Dispatching adversarial pass: native`) was printed and an Agent dispatch follows it; otherwise, as the fallback, an Agent dispatch's description or prompt matches `match` |
@@ -109,13 +109,13 @@ The hook checks the most recently loaded skill that declares a contract, and onl
- the session is headless or unattended (`gstack-session-kind`), or
- the turn does not end on a question to the user (a last paragraph ending in "?", an AskUserQuestion call, or `NEEDS_CONTEXT`).
So an interactive session that pauses to ask a question is never blocked. When Claude Code says a Stop hook already continued this turn (`stop_hook_active`), the hook never blocks: after its own block it records the result (`met_after_block` or `miss`). It also allows the stop while Agent dispatches are still running (no result yet, or launched in the background with no completion notice naming that agent), because their outputs may still arrive. Because Claude Code can fire Stop before the turn's last message reaches the transcript file, the hook reads the payload's `last_assistant_message` as the turn's final text.
So an interactive session that pauses to ask a question is never blocked. When Claude Code says a Stop hook already continued this turn (`stop_hook_active`), the hook never blocks: after its own block it records the result (`met_after_block` or `miss`). On a final turn it also allows the stop while Agent dispatches are still running (no result yet, or launched in the background with no completion notice naming that agent), because their outputs may still arrive, and records the run as `deferred`. Once the same agents have been running for 10 minutes since the stop that first saw them, they count as stalled and the contract is checked at the next stop. Because Claude Code can fire Stop before the turn's last message reaches the transcript file, the hook reads the payload's `last_assistant_message` as the turn's final text.
It blocks at most once per skill run. If the model stops again with the contract still unmet, the stop is allowed and the run is recorded as a miss.
### Records
Each Claude Code session has one record, `$GSTACK_STATE_ROOT/skill-contracts/<session id>.json`, written atomically (mode 0600) before any block. Each skill run in it has an outcome: `met`, `met_waived`, `blocked`, `met_after_block` or `miss`. The hook's decisions also go to `$GSTACK_STATE_ROOT/hooks/skill-contract.log`.
Each Claude Code session has one record, `$GSTACK_STATE_ROOT/skill-contracts/<session id>.json`, written atomically (mode 0600) before any block. Each skill run in it has an outcome: `met`, `met_waived`, `blocked`, `met_after_block`, `miss` or `deferred` (stopped while its agents were still running, not yet checked; `report` re-checks it against its transcript). The hook's decisions also go to `$GSTACK_STATE_ROOT/hooks/skill-contract.log`.
## Commands
docs/skills.md
+5 -1
View File
@@ -910,8 +910,12 @@ gstack's own templates and for any skill folder you own.
add nothing a fresh session already does; edit the line that covers the idea instead of appending; put the fact on the
lowest level that reaches the agent in time (description, SKILL.md, `references/`, the tool's `--help`, the tool's
errors); write the rule and put the story in the commit message.
- **Evidence.** A new skill, step, instruction, check or default needs evidence that it helps on frontier models, or a
dated plan to measure it ([ETHOS.md: 9. Earn Its Place](../ETHOS.md#9-earn-its-place)). Commits carry an `Evidence:`
line, and every gstack skill has a row in the evidence ledger, `docs/inventory/evidence.json`, which
[docs/inventory.md](inventory.md) ranks by load per run.
- **Size.** `gstack-skill-bill` prints bytes and estimated tokens per level, and in a gstack checkout the skill's eval
task file (`EVAL TASKS:`). Commits carry
task file (`EVAL TASKS:`) and its ledger row (`EVIDENCE:`). Commits carry
`Skill-Size: <before>→<after> bytes`; a skill more than 10% over its last `Skill-Diet:` commit is DUE, and a red
context-budget ratchet means diet, not a higher ceiling.
- **Drift.** `test/skill-bin-drift.test.ts` fails when a template or rendered skill calls a gstack bin or flag that
docs/standalone-binaries.md
+1 -1
View File
@@ -18,7 +18,7 @@ Beyond the slash-command skills, gstack ships standalone CLIs for workflows that
| `gstack-wtree` | **Working-tree fingerprint** — prints a content hash of what's actually on disk (temp index seeded from the stat cache, ~40x cheaper than a full re-hash; untracked source counts, gitignored scratch doesn't). Identical content fingerprints identically through commits, rebases, amends, and squashes — it's what binds reviews and test evidence to content instead of commit SHAs. |
| `gstack-review-log` | **Review-pass receipts** — `--start <skill>` captures the working-tree fingerprint before a diff review; `'<JSON>' --finish <token>` consumes that single-use, repository/branch/skill-scoped receipt. Binding requires matching start/end content and reviewer-reported `completed:true` and `converged:true`; it is not independent proof that a model read the source. `--check-shared-libs <token>` reads a current finding as JSON on stdin and checks prior Skip decisions against the actual branch, capture and eligible raw source blobs. It returns `reusable:false` when proof is missing or unsafe, including older records without logger-versioned coverage. Final review logging computes shared-code fingerprints and coverage rather than trusting supplied proof. |
| `gstack-review-read` | **Review freshness** — emits review records with computed `review_freshness.status` and `reason`: CURRENT, STALE, or UNVERIFIED for diff reviews. `/ship` and `/land-and-deploy` use the same grade; a matching commit alone never certifies a diff review. [Dashboard rules](skills.md#review-readiness-dashboard). |
| `gstack-evidence` | **Verification-evidence ledger** — `run --label <lane> -- <cmd>` transparently wraps any test command (the child's exit code always passes through) and records what ran against which working-tree fingerprint (tracked and untracked files that are not gitignored, minus an untracked `.bun/` folder at the repo root, which Bun can create during a run); a run whose files changed while it ran is recorded without a fingerprint and with up to five of the changed paths, which `check` names in its STALE reason. `check` grades each label FRESH/STALE/MISSING with `--expect-cmd`, `--max-age`, and `--allow-paths` binding. /ship and /land-and-deploy cite fresh evidence instead of re-running suites. Per-run logs are 0600, capped at 2MB, pruned after 30 days; the ledger and logs stay machine-local by design. |
| `gstack-evidence` | **Verification-evidence ledger** — `run --label <lane> -- <cmd>` transparently wraps any test command (the child's exit code always passes through) and records what ran against which working-tree fingerprint (tracked and untracked files that are not gitignored, minus an untracked `.bun/` folder at the repo root, which Bun can create during a run); a run whose files changed while it ran is recorded without a fingerprint and with up to five of the changed paths, which `check` names in its STALE reason. `check` grades each label FRESH/STALE/MISSING with `--expect-cmd`, `--max-age`, and `--allow-paths` binding. A label is stored and looked up with every character outside `A-Za-z0-9._-` turned into `_`, so `pnpm lint` and `pnpm_lint` name the same row. `carried` lists the CI-sourced rows at the current tree that grade FRESH, which the ship receipt records as `ci_carried`. /ship and /land-and-deploy cite fresh evidence instead of re-running suites. Per-run logs are 0600, capped at 2MB, pruned after 30 days; the ledger and logs stay machine-local by design. |
| `gstack-flaky` | **Flaky-test rerun for any JUnit runner** — `run --junit <file.xml> -- <command>` runs your test command (Playwright, `bun test`, jest, vitest, pytest) and, when at most `--max-reruns` (3) tests failed and the runner made no retries of its own, reruns the whole command once (`--rerun-scope test` opts in to rerunning only the failed tests). A test that fails and then passes on that rerun is a flaky pass: the run stays green and one redacted line per flaky pass or failed rerun goes on a ledger (`GSTACK_FLAKE_LEDGER_JUNIT`, default `<state root>/flake-ledger-junit.jsonl`). A missing or malformed report, a crash or a killed runner never passes. `quarantine add|renew|release|check|report` keeps `.gstack/quarantine.json`: a quarantined test still runs, gets up to 3 attempts of the whole command, passes only on a full green attempt and is never counted as a pass (`run --base <ref>` reads the registry from the base branch). `ids --junit <file>` prints a report's test ids. Exit 0 pass, 1 a test failed, 2 usage, 3 no usable report. [Guide](flaky.md). |
| `gstack-pr-wait` | **PR event watcher** — `gstack-pr-wait <pr> [--repo owner/name] [--until green\|ready\|any-change] [--interval 20] [--timeout 60] [--exclude CHECK,...]` polls one pull request through `gh` and prints one stdout line per real event (a check failed with its link, all checks finished, a comment or review from someone other than the author, a review bot finished on the head, the base moved and its new commits touch this PR's files, BEHIND or DIRTY, merged or closed), with progress on stderr, so a monitor wakes the agent once per event instead of on a timer. `green` gates on every check on the head (an advisory bot's review is never waited for, its checks gate like any other), as `gstack-ci-gate` does; a check the base branch requires (rulesets and classic branch protection) is pending until it appears, never green by omission; two identical polls confirm a check set is complete; `--exclude` leaves a named non-required check out of the verdict as `gstack-ci-gate --exclude` does (a required name is refused and gates). Exit 0 when the condition is met, 1 when it needs action or `--timeout` passes (`PR_WAIT_BLOCKED`), 2 on bad input, a second watcher for the same PR or a local error (`PR_WAIT_REFUSED`). State under `<state root>/pr-wait/`, baseline discarded after 6 hours. Review bots are configuration: `gstack-config set pr_wait_gating_bots greptile,macroscope`, `pr_wait_advisory_bots coderabbit` (advisory bots are reported, never waited on). `/land-and-deploy` Step 3 is one call of it. Design ported from tstack's pr-wait (MIT, Theo Browne; NOTICE.md). [Exit codes](troubleshooting.md#gstack-pr-wait). |
| `gstack-review-bots` | **Review-bot comments** — `fetch [--pr <n>]` reads every root line comment, conversation comment and newest review body from each bot in `pr_wait_gating_bots` and `pr_wait_advisory_bots` (Bot accounts only, all pages, through `gh`) into a private directory with per-bot counts, or prints `REVIEW_BOTS: skip` or `REVIEW_BOTS: error` (never a false zero). `record --dir <dir> --id <comment id> --type fix\|fp\|already-fixed --category <category>` appends one triage outcome to the project and global `review-bot-history.md`, taking the bot and file from the fetched comment. `/review` Step 2.5 and `/ship` Step 10 run it. [Triage reference](../review/greptile-triage.md). |
docs/troubleshooting.md
+10 -7
View File
@@ -2091,8 +2091,8 @@ browse chain 'goto https://example.com | text'
<a id="browse-chromium-path-failed"></a>
### `Chromium at GSTACK_CHROMIUM_PATH=<path> failed to launch: ...`
**Meaning.** Headless browse and make-pdf now use `GSTACK_CHROMIUM_PATH`
(NixOS, macOS 13). That binary could not start.
**Meaning.** Headless browse, headed browse, `browse handoff` and make-pdf use
`GSTACK_CHROMIUM_PATH` (NixOS, macOS 13). That binary could not start.
**Fix.** Point `GSTACK_CHROMIUM_PATH` at a working Chromium, or
`unset GSTACK_CHROMIUM_PATH` to use gstack's bundled one.
@@ -4165,15 +4165,17 @@ refuses to start without a passing check.
**Meaning.** The preregistration says `stack: loaded` (some arm loads gstack or
another skill stack) and its `## No-gstack arm` section is missing, lacks one
of `arms:`, `inputs:`, `prompt:`, `panel:` or `report:`, lists an arm that
of `arms:`, `inputs:`, `prompt:`, `panel:`, `report:`, `masking:`, `install:`
or `audit:` (or says `none` for one of the last three), lists an arm that
is not a `<model>#no-gstack` Arms line, or leaves a model in the comparison
without its own no-gstack arm. Arms match on whole identities, so
`claude-opus-5#no-gstack` does not cover `claude-opus-5-5`.
**Fix.** Add one `<model>#no-gstack` line per model under `## Arms`, list them
on the `arms:` line, and fill the other four keys: the same inputs, a plain
prompt with no skill text, the same panel and judge, and where the result row
goes beside the variants. When no arm loads a skill stack, write
on the `arms:` line, and fill the other keys: the same inputs, a plain
prompt with no skill text, the same panel and judge, where the result row
goes beside the variants, how the run agent is masked from the Capy drives,
the check that no gstack is installed, and the per-arm drive-read audit. When no arm loads a skill stack, write
`stack: none` with a `why:` line instead. The rule is in
[docs/TESTING_INTERNALS.md](TESTING_INTERNALS.md#no-gstack-arm).
@@ -5707,7 +5709,8 @@ closed PRs.
verdict is read from the run itself, never from a lane's report: the
`gate-binding-a<attempt>` artifact evals.yml's report job uploads (verdict,
reconcile exit, flaky cases) and the `report-verdict-a<attempt>` artifact's
`collector-outcomes.json` (the panels that failed the lane and the rule-16
`collector-outcomes.json` (the panels that failed the lane, the report's `✗`
failure lines, which name rule cases that have no panel, and the rule-16
flaky lines). A case the binding calls red already failed twice inside the
run (rule 16 reruns a first failure once), so the red is real. `ready`
refuses with this code when the head's recorded screen run is red, whichever