CalvinBackup/gstack
1
0 Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-11-07 17:28:23 +01:00
Code Issues Packages Projects Releases Wiki Activity

docs(inventory): record L1 verdicts on untold routing and told /investigate (GST-1629) (#4076)

Browse Source 
* docs(inventory): record L1 verdicts on untold routing and told /investigate (GST-1629)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(inventory): regenerate the snapshot with L1's verdicts (GST-1629)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-11-06 14:36:50 +00:00
1 parent 1a06afdc19
commit 274fb34537
3 files changed
+36 -24
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

changelog.d/capy-gst-1629-l1-inventory.md
+1
View File
@@ -0,0 +1 @@
- **The R56 inventory records L1's verdicts on gstack's untold routing and /investigate's told record (GST-1629).** GEVA-2's L1 run (gstack-evals#247; Opus 5.5 on the claude_code preset, 80 SWE-rebench tasks, a no-gstack arm, pins 82db89ab to 699a41cb) is now in `docs/inventory/evidence.json` and the regenerated `docs/inventory.md`: the untold routing package (route hook, contracts and the description rewrite) helps, B1 +21.3 points [+12.5, +30.0], KEEP, and needs routing or contracts on; the description rewrite alone had no effect (0 of 80); told record completeness is untested, B3 +10.0 [0, +20], FIX-METHOD because probe I failed its blind audit; pass rate O5 +5.0 at a 65% control, MDD 18.7, is descriptive only. /investigate keeps its no-effect status and CUT-CANDIDATE, which measured something else.
docs/inventory.md
+23 -22
View File
@@ -1,6 +1,6 @@
# gstack inventory: every part, its load per run and its evidence
Generated by `bin/gstack-inventory` from [evidence.json](inventory/evidence.json), the hand-kept ledger, and lib/context-bill.ts. Measured at commit 56c1fb045 (2026-11-04). This is a dated snapshot: rerun `bin/gstack-inventory` to refresh it; `bin/gstack-inventory --check` validates the ledger. The rule: [ETHOS.md: 9. Earn Its Place](../ETHOS.md#9-earn-its-place).
Generated by `bin/gstack-inventory` from [evidence.json](inventory/evidence.json), the hand-kept ledger, and lib/context-bill.ts. Measured at commit cf1333466 (2026-11-06). This is a dated snapshot: rerun `bin/gstack-inventory` to refresh it; `bin/gstack-inventory --check` validates the ledger. The rule: [ETHOS.md: 9. Earn Its Place](../ETHOS.md#9-earn-its-place).
Token counts are offline estimates (estimate: calibrated bytes/token per content class, about ±40%), not billed tokens.
@@ -19,9 +19,9 @@ Load per run: a skill's SKILL.md plus the files it must read on every use (`perI
| Rank | Part | Kind | Load per run (tokens) | On demand (tokens) | Always-on (tokens) | Loads | Status | Source, plan or reason |
|---|---|---|---|---|---|---|---|---|
| 1 | ship-review-panel | shared | 23,370 | n/a | n/a | every /ship run, by depth | untested | plan: GEVA-2 registry row R56-1 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 1 | ship-review-panel | shared | 23,369 | n/a | n/a | every /ship run, by depth | untested | plan: GEVA-2 registry row R56-1 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 2 | design-review | skill | 20,159 | 7,778 | 101 | every use | untested | plan: GEVA-2 registry row R56-9 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 3 | ship | skill | 14,587 | 70,907 | 91 | every use | untested | plan: GEVA-2 registry row R56-4 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 3 | ship | skill | 14,571 | 71,199 | 91 | every use | untested | plan: GEVA-2 registry row R56-4 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 4 | plan-ceo-review | skill | 14,394 | 21,203 | 313 | every use | untested | plan: GEVA-2 registry row R56-5 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 5 | review | skill | 14,268 | 41,459 | 185 | every use | helps | sources: gstack-evals #207 (baa599a9) wiki/board/2026-11-04-geva-271-r2-after-pin-results, gstack-evals #211 (81daa79) wiki/board/2026-11-04-geva-281-r2-after-pin-results-home-field; R48 KEEP; caveats: one unit: /review and the contract Stop hook were measured together, and neither gets credit alone; 42 of 66 gstack sessions ran the review readers only after the hook blocked them (24 unprompted), and that split is not randomized; home-field bias in gstack's favor: 41 of the 42 blocking bugs in stacks-r1 came from gstack-family reviews, so +22.6 may run high; a suite seeded from other reviewers' bugs would remove the bias (GEVA-2 lists it as future work); development suite: stacks-r1 is a development suite, so the result makes no release claim and holds only for these 33 units at these settings |
| 6 | devex-review | skill | 14,025 | 0 | 74 | every use | untested | plan: GEVA-2 registry row R56-11 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
@@ -47,21 +47,21 @@ Load per run: a skill's SKILL.md plus the files it must read on every use (`perI
| 26 | canary | skill | 7,034 | 0 | 66 | every use | untested | plan: none (grandfathered) |
| 27 | qa | skill | 6,957 | 13,909 | 74 | every use | untested | plan: none (grandfathered) |
| 28 | scrape | skill | 6,647 | 0 | 80 | every use | untested | plan: none (grandfathered) |
| 29 | captain | skill | 6,550 | 9,194 | 127 | every use | untested | plan: none (grandfathered) |
| 29 | captain | skill | 6,550 | 9,517 | 127 | every use | untested | plan: none (grandfathered) |
| 30 | preamble | shared | 5,743 | n/a | n/a | every tier 2+ skill run, inside that skill's load | untested | plan: I1 ceremony cut (GEVA-2, 2026-11-02): gstack-evals wiki/plans/2026-11-01-geva-108-ceremony-cut-prereg (#122, main 409ff47) with its G3 amendment (#128), preset, Opus 5.5 and particle-1 (grandfathered) |
| 31 | gstack-upgrade | skill | 5,726 | 0 | 59 | every use | untested | plan: none (grandfathered) |
| 32 | context-restore | skill | 5,715 | 1,734 | 84 | every use | untested | plan: none (grandfathered) |
| 33 | document-generate | skill | 5,466 | 0 | 110 | every use | untested | plan: none (grandfathered) |
| 34 | ios-qa | skill | 5,368 | 1,378 | 84 | every use | untested | plan: none (grandfathered) |
| 35 | skillify | skill | 5,317 | 0 | 80 | every use | untested | plan: none (grandfathered) |
| 36 | capy-skill | shared | 5,315 | n/a | n/a | every Capy session | untested | plan: none (grandfathered) |
| 36 | capy-skill | shared | 5,295 | n/a | n/a | every Capy session | untested | plan: none (grandfathered) |
| 37 | setup-deploy | skill | 5,216 | 0 | 73 | every use | untested | plan: none (grandfathered) |
| 38 | qa-only | skill | 5,205 | 3,460 | 68 | every use | untested | plan: none (grandfathered) |
| 39 | eval-plan | skill | 5,107 | 0 | 108 | every use | untested | plan: none (grandfathered) |
| 40 | investigate | skill | 5,059 | 3,729 | 598 | every use | no-effect | sources: GST-814 (HARN-1, 2026-11-02), org-gstack-gbrain handoffs/HARN-14/hard-follow-2026-11/decision-rule.md; R48 CUT-CANDIDATE |
| 40 | investigate | skill | 5,059 | 3,729 | 598 | every use | no-effect | sources: GST-814 (HARN-1, 2026-11-02), org-gstack-gbrain handoffs/HARN-14/hard-follow-2026-11/decision-rule.md; R48 CUT-CANDIDATE; caveats: L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, no-gstack arm, pins 82db89ab → 699a41cb), untold: the routing package (route hook, contracts and the description rewrite) helps, untold /investigate opens 17/80 vs 0/80, B1 +21.3 [+12.5, +30.0], R48 KEEP; it needs routing or contracts on, and hook vs contracts aren't split; the description rewrite alone had no effect (off-u 0/80); L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, no-gstack arm, pins 82db89ab → 699a41cb), told: /investigate record completeness with the contract Stop hook is untested, B3 +10.0 [0, +20]; probe I failed its blind audit, R48 FIX-METHOD; the weak item is 'Fix names an edited file', and 5 of 7 misses were fixes written by an inline Python heredoc; L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, no-gstack arm, pins 82db89ab → 699a41cb), pass rate: O5 +5.0 at a 65% control, MDD 18.7, descriptive only |
| 41 | health | skill | 5,041 | 0 | 68 | every use | untested | plan: none (grandfathered) |
| 42 | gstack | skill | 4,990 | n/a | 59 | every use | untested | plan: none (grandfathered) |
| 43 | pair-agent | skill | 4,811 | 0 | 63 | every use | untested | plan: none (grandfathered) |
| 43 | pair-agent | skill | 4,824 | 0 | 63 | every use | untested | plan: none (grandfathered) |
| 44 | make-pdf | skill | 4,730 | 1,767 | 65 | every use | untested | plan: none (grandfathered) |
| 45 | context-save | skill | 4,637 | 0 | 65 | every use | untested | plan: none (grandfathered) |
| 46 | cso | skill | 4,456 | 4,759 | 95 | every use | safety | reason: security: supported security findings with explicit coverage; kept on its security reason (GEVA-2 marks it exempt; GStack Minder may reclassify it) |
@@ -71,9 +71,9 @@ Load per run: a skill's SKILL.md plus the files it must read on every use (`perI
| 50 | document-release | skill | 4,049 | 10,260 | 72 | every use | untested | plan: none (grandfathered) |
| 51 | diagram | skill | 3,895 | 0 | 112 | every use | untested | plan: none (grandfathered) |
| 52 | benchmark-models | skill | 3,822 | 0 | 77 | every use | untested | plan: none (grandfathered) |
| 53 | runaway-stopper | skill | 3,672 | 571 | 78 | every use | untested | plan: docs/evals/runaway-stopper-prereg.md (owner GStack Minder (GSTA-36), measure by 2026-11-24) |
| 54 | ubicloud | skill | 3,638 | 0 | 83 | every use | untested | plan: none (grandfathered) |
| 55 | modernize | skill | 3,418 | 0 | 94 | every use | untested | plan: none (grandfathered) |
| 53 | runaway-stopper | skill | 3,693 | 571 | 78 | every use | untested | plan: docs/evals/runaway-stopper-prereg.md (owner GStack Minder (GSTA-36), measure by 2026-11-24) |
| 54 | ubicloud | skill | 3,676 | 0 | 83 | every use | untested | plan: none (grandfathered) |
| 55 | modernize | skill | 3,526 | 0 | 94 | every use | untested | plan: none (grandfathered) |
| 56 | setup-browser-cookies | skill | 3,289 | 0 | 79 | every use | untested | plan: none (grandfathered) |
| 57 | deslop-shared-libs | skill | 3,100 | 0 | 64 | every use | untested | plan: none (grandfathered) |
| 58 | landing-report | skill | 2,971 | 0 | 71 | every use | untested | plan: none (grandfathered) |
@@ -98,9 +98,9 @@ Untested parts, largest load per run first. A fair test runs the claude_code pre
| Order | Part | Load per run (tokens) | Plan |
|---|---|---|---|
| 1 | ship-review-panel | 23,370 | GEVA-2 registry row R56-1 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 1 | ship-review-panel | 23,369 | GEVA-2 registry row R56-1 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 2 | design-review | 20,159 | GEVA-2 registry row R56-9 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 3 | ship | 14,587 | GEVA-2 registry row R56-4 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 3 | ship | 14,571 | GEVA-2 registry row R56-4 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 4 | plan-ceo-review | 14,394 | GEVA-2 registry row R56-5 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 5 | devex-review | 14,025 | GEVA-2 registry row R56-11 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
| 6 | plan-design-review | 13,351 | GEVA-2 registry row R56-7 (org-gstack-gbrain/research/gstack-part-evidence.json registry_rows; proposed, not yet preregistered) (grandfathered) |
@@ -132,13 +132,13 @@ Untested parts, largest load per run first. A fair test runs the claude_code pre
| 32 | document-generate | 5,466 | none (grandfathered) |
| 33 | ios-qa | 5,368 | none (grandfathered) |
| 34 | skillify | 5,317 | none (grandfathered) |
| 35 | capy-skill | 5,315 | none (grandfathered) |
| 35 | capy-skill | 5,295 | none (grandfathered) |
| 36 | setup-deploy | 5,216 | none (grandfathered) |
| 37 | qa-only | 5,205 | none (grandfathered) |
| 38 | eval-plan | 5,107 | none (grandfathered) |
| 39 | health | 5,041 | none (grandfathered) |
| 40 | gstack | 4,990 | none (grandfathered) |
| 41 | pair-agent | 4,811 | none (grandfathered) |
| 41 | pair-agent | 4,824 | none (grandfathered) |
| 42 | make-pdf | 4,730 | none (grandfathered) |
| 43 | context-save | 4,637 | none (grandfathered) |
| 44 | open-gstack-browser | 4,429 | none (grandfathered) |
@@ -147,9 +147,9 @@ Untested parts, largest load per run first. A fair test runs the claude_code pre
| 47 | document-release | 4,049 | none (grandfathered) |
| 48 | diagram | 3,895 | none (grandfathered) |
| 49 | benchmark-models | 3,822 | none (grandfathered) |
| 50 | runaway-stopper | 3,672 | docs/evals/runaway-stopper-prereg.md (owner GStack Minder (GSTA-36), measure by 2026-11-24) |
| 51 | ubicloud | 3,638 | none (grandfathered) |
| 52 | modernize | 3,418 | none (grandfathered) |
| 50 | runaway-stopper | 3,693 | docs/evals/runaway-stopper-prereg.md (owner GStack Minder (GSTA-36), measure by 2026-11-24) |
| 51 | ubicloud | 3,676 | none (grandfathered) |
| 52 | modernize | 3,526 | none (grandfathered) |
| 53 | setup-browser-cookies | 3,289 | none (grandfathered) |
| 54 | deslop-shared-libs | 3,100 | none (grandfathered) |
| 55 | landing-report | 2,971 | none (grandfathered) |
@@ -188,11 +188,11 @@ Each count is quoted from the file and line named; `bin/gstack-inventory --check
| Skill | What | Count | Source |
|---|---|---|---|
| ship | Step 9 review, pass 1, light depth, docs or tests only | 1 reader (the core checklist) | docs/review-policy.md:295 |
| ship | Step 9 review, pass 1, light depth, code | 3 readers (core, one batched specialist, native adversarial), plus the outside voice on a diff that publishes numbers | docs/review-policy.md:296 |
| ship | Step 9 review, pass 1, standard depth | the batched specialist, the Red Team at 200 runtime lines, adversarial and outside voice | docs/review-policy.md:297 |
| ship | Step 9 review, pass 1, full depth | the batched specialist on the whole diff and the Red Team always | docs/review-policy.md:298 |
| ship | targeted reader on money, gate, auth or secret code | +1 at every depth | docs/review-policy.md:446 |
| ship | Step 9 review, pass 1, light depth, docs or tests only | 1 reader (the core checklist) | docs/review-policy.md:298 |
| ship | Step 9 review, pass 1, light depth, code | 3 readers (core, one batched specialist, native adversarial), plus the outside voice on a diff that publishes numbers | docs/review-policy.md:299 |
| ship | Step 9 review, pass 1, standard depth | the batched specialist, the Red Team at 200 runtime lines, adversarial and outside voice | docs/review-policy.md:300 |
| ship | Step 9 review, pass 1, full depth | the batched specialist on the whole diff and the Red Team always | docs/review-policy.md:301 |
| ship | targeted reader on money, gate, auth or secret code | +1 at every depth | docs/review-policy.md:449 |
| ship | audits (Steps 7, 8 and 14.5) | 3 | lib/ship-budget-calibration.json:3 |
| ship | measured: one /ship child (gstack-evals#31) | 10 review dispatches | lib/ship-budget-calibration.json:43 |
| ship | prior cost of one dispatch on the Capy meter | 7,430,000 tokens | lib/ship-budget-calibration.json:56 |
@@ -248,3 +248,4 @@ Each count is quoted from the file and line named; `bin/gstack-inventory --check
- Evidence-table row "review: Review Army and adversarial pass": a section inside /review, not its own key; R2 measures it as part of the review row's plan.
- Evidence-table row "ship: QA probes (Step 9.2.1)": a step inside /ship with no separate size; folded into the ship row (GEVA-2 registry row R56-4).
- Evidence-table row "gstack as a whole (installed, any skill)": not a single text part; GEVA-2's real-world outcome lane measures it (gstack-evals plan #111), not a measurement plan for any one part.
- Evidence-table row "untold routing package (route hook, contracts and the description rewrite)": two Claude Code hooks and four description lines, not one text part with a key. L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, no-gstack arm, pins 82db89ab → 699a41cb): helps, untold /investigate opens 17/80 vs 0/80, B1 +21.3 [+12.5, +30.0], R48 KEEP; it needs routing or contracts on, and hook vs contracts aren't split; the description rewrite alone had no effect (off-u 0/80). The investigate row carries it as a caveat.
docs/inventory/evidence.json
+12 -2
View File
@@ -231,7 +231,12 @@
}
],
"seed": "GEVA-2, 2026-11-02: investigate: measurement under way (R3)",
"note": "Only Opus 5.5 (claude-opus-5-5) was measured; GPT-6.1 Sol was not, so this verdict covers one of the two frontier models ETHOS.md 9 names."
"note": "Only Opus 5.5 (claude-opus-5-5) was measured; GPT-6.1 Sol was not, so this verdict covers one of the two frontier models ETHOS.md 9 names. L1 (gstack-evals#247, wiki/board/2026-11-06-geva-350-l1-overhaul-results) measured the after pin's untold routing package and told record completeness, not this row's procedure follow rate, so the no-effect status and its CUT-CANDIDATE stand; its three verdicts are this row's caveats. GPT-6.1 Sol has not run L1.",
"caveats": [
"L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, no-gstack arm, pins 82db89ab → 699a41cb), untold: the routing package (route hook, contracts and the description rewrite) helps, untold /investigate opens 17/80 vs 0/80, B1 +21.3 [+12.5, +30.0], R48 KEEP; it needs routing or contracts on, and hook vs contracts aren't split; the description rewrite alone had no effect (off-u 0/80)",
"L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, no-gstack arm, pins 82db89ab → 699a41cb), told: /investigate record completeness with the contract Stop hook is untested, B3 +10.0 [0, +20]; probe I failed its blind audit, R48 FIX-METHOD; the weak item is 'Fix names an edited file', and 5 of 7 misses were fixes written by an inline Python heredoc",
"L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, no-gstack arm, pins 82db89ab → 699a41cb), pass rate: O5 +5.0 at a 65% control, MDD 18.7, descriptive only"
]
},
"ios-clean": {
"status": "untested",
@@ -597,7 +602,8 @@
"seed": "GEVA-2, 2026-11-02: always-on skill catalog (descriptions): measurement under way (Run D)",
"aliases": [
"always-on skill catalog (descriptions)"
]
],
"note": "L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, pins 82db89ab → 699a41cb): the description rewrite alone had no effect (off-u, routing and contracts off with the rewritten descriptions kept: 0/80 untold /investigate opens). The rewrite helps only inside the untold routing package; see the investigate row's caveats and this ledger's unmapped rows."
},
"capy-skill": {
"status": "untested",
@@ -678,6 +684,10 @@
{
"row": "gstack as a whole (installed, any skill)",
"why": "not a single text part; GEVA-2's real-world outcome lane measures it (gstack-evals plan #111), not a measurement plan for any one part"
},
{
"row": "untold routing package (route hook, contracts and the description rewrite)",
"why": "two Claude Code hooks and four description lines, not one text part with a key. L1 (GEVA-2, gstack-evals#247, merged 28be250; Opus 5.5, claude_code preset, 80 SWE-rebench tasks, no-gstack arm, pins 82db89ab → 699a41cb): helps, untold /investigate opens 17/80 vs 0/80, B1 +21.3 [+12.5, +30.0], R48 KEEP; it needs routing or contracts on, and hook vs contracts aren't split; the description rewrite alone had no effect (off-u 0/80). The investigate row carries it as a caveat"
}
]
}