CalvinBackup/gstack
1
0 Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-11-08 09:48:20 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,043 Commits 1,236 Branches 5 Tags
cf4d847e0169e8edb8d37abd00a29855bb2fd380
Commit Graph
3 Commits
Author SHA1 Message Date
Garry Tanandcapy-ai[bot] ade35bc813 v1.94.29.0 R56: everything in gstack earns its place (principle, evidence gates, inventory) (#3837) 
* eval-plan: a no-gstack arm needs masking, install and audit lines

prereg --check now refuses a loaded comparison whose ## No-gstack arm
section lacks masking:, install: or audit:, or says none for one of them
(PREREG_NO_GSTACK_ARM). On Capy, run agents inherit the instruction to load
the user-drive gstack skill, so the arm hides both Capy drives, checks that
no gstack is installed, and audits drive reads; an arm that read a gstack
skill file is contaminated and gets zero credit (TESTING_INTERNALS.md
#no-gstack-arm). The prereg template, /eval-plan, troubleshooting and the
two passing prereg fixtures carry the new lines. eval-plan's paragraph was
dieted to hold its context-budget ceiling.

Evidence: safety: keeps the no-gstack control uncontaminated (Captain order, FOUN-32 via GGST-6)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* inventory: the R56 evidence ledger, gstack-inventory and skill-bill EVIDENCE

docs/inventory/evidence.json holds one row per inventory key: the skill
folders discoverTemplates() finds (root skill as gstack, symlinked folders
excluded) plus seven shared parts, 71 keys. lib/inventory.ts validates each
row by its status's own fields, with named codes (LEDGER_*), and joins the
ledger with lib/context-bill.ts buildBill. bin/gstack-inventory writes
docs/inventory.md, --check validates (exit 1 with codes), --json prints.
Paid dispatches per run for /ship and /autoplan are quoted from their own
docs and found by line, so a moved citation fails --check
(INVENTORY_SOURCE_MISSING). Every current row is grandfathered and seeded
from GEVA-2's table (org-gstack-gbrain/research/gstack-part-evidence.json):
in-flight fair tests and proposed registry rows as plans, earlier unfair
results under prior with why (stub, unmasked, ceiling, ...), five safety
rows. gstack-skill-bill prints the skill's EVIDENCE line.

Evidence: plan none (owner ruling R56, GST-914); the ledger itself is the measurement record
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* ETHOS 9, Earn Its Place: factory, /write-skill and /autoplan carry it

Garry, 2026-11-02: "Also work this principle of only include it if it is
beneficial and good into gstack". ETHOS.md principle 9 states the rule, the
fair test (claude_code preset, frontier models per R54, masked no-gstack
arm), the R48 Why-null cycle, the safety exception, and measure biggest
first, linking docs/inventory.md. factory.md item 8 cites it, trimmed
elsewhere to 3,298 of 3,300 bytes with no rule dropped. /write-skill's edit
ladder step 2 is the evidence rule and section 4 adds the Evidence: commit
line. /autoplan's CEO phase lists each part a plan adds to gstack with its
evidence or owned, dated plan; two redundant lines went to keep autoplan
under its size pins (pins lowered by 17). CONTRIBUTING.md "Adding a new
skill" and docs/skills.md point at the ledger.

Skill-Size: 3273→3666 bytes (write-skill/SKILL.md.tmpl)
Skill-Size: 12738→12721 bytes (autoplan/sections/ceo-phase.md)
Evidence: plan none (owner ruling R56, GST-914); write-skill and autoplan rows stay grandfathered untested
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fixup! inventory: the R56 evidence ledger, gstack-inventory and skill-bill EVIDENCE

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* inventory: list the evidence table's other names and its unmapped rows

Ledger rows carry aliases (the name GEVA-2's table uses where it differs from the key, such as upgrade for gstack-upgrade), and docs/inventory.md lists them beside the installed-name rule. Unmapped rows are labeled as evidence-table rows, and the ranking notes that a skill's on-demand files can also be a shared row's.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: the first inventory snapshot and the R56 changelog fragment

docs/inventory.md is generated by bin/gstack-inventory at the previous commit: 71 parts ranked by load per run, 0 helps, 0 no-effect, 0 hurts, 16 untested with a plan (4 fair tests under way, 12 proposed), 50 untested with plan none, 5 safety.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* inventory: investigate's step procedure is a fair-test CUT-CANDIDATE (GST-814)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: the inventory snapshot after investigate's no-effect row

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* inventory: only the seeded backlog may be grandfathered, so a new part's row is checked (review fix)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* inventory: devex-review and retro name their GEVA-2 registry rows (R56-11, R56-12)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: the inventory snapshot after the review fixes

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* inventory: a concrete plan is a path, gstack-inventory never writes into a gstack install, and the ledger checks aliases, r48 and prior.why (review fixes)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* inventory: investigate's no-effect row notes that only Opus 5.5 was measured

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: the inventory snapshot after the review fixes

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* changelog: R56 counts after investigate's row; grandfathering and the install guard

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: register inventory-grandfathered.json as a free fixture

free-suite (1) at e56d3312c failed FREE_FIXTURES inventory: the pinned grandfather fixture from d0b3028b5 had no paid touchfile and no FREE_FIXTURES entry. Its only reader is test/inventory.test.ts.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
 2026-11-03 17:08:30 +00:00
Garry Tanandcapy-ai[bot] 4eff47c5cf v1.92.2.0 evals: no-gstack arms for the model panel, arm-benchmark, safety and routing evals (#3708) 
* feat(eval-plan): results --check refuses a results table without its no-gstack rows

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(model-panel): a <model>#no-gstack arm per model, identity-keyed cells, one campaign, results.md (GST-468)

Each panel model also runs as <model>#no-gstack: the same fixture, ticket,
tools, limits, route and reservation, with no skill folder and no routing
line. The arm identity keys the ledger attempt, test name, row and summary;
candidate folders are named after the fixture project and the seed commit
has a neutral author. summarize reads one campaign (run id and PREREG.md
sha256) and refuses a duplicate cell; suggestionLine reads the skill arms
only, so the preregistered particle-1 line is unchanged. renderResults writes
results.md, which passes gstack-eval-plan results --check against the new
test/fixtures/model-panel/PREREG.md (supersedes GSTA-173 for later runs).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(arm-benchmark): no-gstack arm on a pinned model, 3 paired trials, both-order paired judge, spend ledger (GST-469)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(evals): no-gstack arm for the safety-rule and router evals (GST-470)

GSTACK_SAFETY_ARM and GSTACK_ROUTER_ARM take a third value, none: the same
fixture, tools and user request with no skill excerpt, rule text, step file
or installed skill. Under none every behavioral check is recorded as an
observation (none-arm-observations.jsonl) and never fails the case or writes
contract-violations.jsonl; harness preconditions still assert.

The four safety prompt builders (ship-evidence, codex-boundary,
codex-consult, design-risk) and the router fixtures move into helpers; the
rule, removed, current and before prompts stay byte-identical, proven
against reference copies of main's code. pair-agent and ios-demo refuse
none with the reason (NO_NONE_ARM). GSTACK_ROUTER_ARM now throws on an
unknown value instead of silently running current.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: changelog fragment for the no-gstack arms in existing evals

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(arm-benchmark): arm-pair paired-judge calibration run (bar met 6/6, $7.22)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(paid-shards): keep loop-named panel and arm-benchmark cases selectable by --case

The lanes moved each cell's testName into helpers, so the static detector no
longer saw a computed registration and B4 failed for all eight cases. Same
runtime test names; the paid runs are unaffected.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(tests): keep FREE_FIXTURES and the placeholder allowlist current for the no-gstack arms

scripts/judge-calibration.ts is now an arm-benchmark paid touchfile, so it can no
longer be a free consumer of test/fixtures/judge-calibration/** (CI free-suite 8).
The eval-plan Step 5 fence names <prereg-path> and <results-path>; allowlist both
as PATH placeholders (CI free-suite 19).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(eval-plan): cover a complete table after a partial one, --json output and the results usage error

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(eval-plan): keep the Step 7 coverage test as reviewed

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: name the changelog fragment after the branch slug

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: arm-benchmark project folder names and the arm-pair calibration corpus

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: keep the judge-calibration fixture README as reviewed

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(skills): describe eval-plan results --check

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
 2026-10-31 22:41:36 +00:00
Garry Tanandcapy-ai[bot] 0cd2b4a9f9 v1.91.373.0 feat(eval-plan): require a no-gstack arm in every head-to-head preregistration (#3690) 
* feat(eval-plan): require a no-gstack arm in every head-to-head preregistration

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(eval-plan): lint loadout arms by their model; refuse arms sharing a ledger attempt

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
 2026-10-31 15:49:11 +00:00