CalvinBackup/gstack
1
0 Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-11-03 07:18:03 +01:00
Code Issues Packages Projects Releases Wiki Activity
Files
1a0be11bf4e0abb20cfd46a0a7657eaecbb13bcd
gstack/docs/designs/gate-diff-calibration.md
T
Garry Tanandcapy-ai[bot] 5b9b8372c4 feat: honest work and gate integrity — gate-edit detector, review/ship gate pass, zero-run green, claims-need-evidence kernel (#3114) 
* docs(designs): honest work and gate integrity plan (autoplan-reviewed)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(e2e): review-gate-integrity rule case registered in touchfiles and the PR profile

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(honesty): claims-need-evidence kernel, no_claim card field, CEO value test, implementer dispatch rules, ACCEPTANCE_EDITED, Honest Work ethos

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(gate-diff): gate-edit detector lib + CLI, patch fixtures, calibration script, tests_ran in gstack-evidence

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(gate-diff): RH-15? only on CI/config floors after calibration

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* feat(review,ship): Gate Integrity category, GATE_SCAN_BLOCK, one-way gate disposition, actor stamping, ZERO-RUN lanes, draft-while-open

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore: re-measure parity and context budgets on the integrated tree

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(touchfiles): gate-diff fixtures and helpers select review-gate-integrity

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: archaeology lint wording, paid-file census, harness-version non-behavioral record

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix: pin review Step 3.5 rename, plan-path placeholder, mktemp template, per-stream evidence tail

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: ACCEPTANCE_EDITED fence substitutes the <plan-path> placeholder

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(gate-diff): shallow boundary commits report no_base instead of a whole-tree inventory; recalibrate

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-10 22:33:01 +00:00

122 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# gate-diff calibration — tool inventory over gstack's last 50 merged PRs
Generated by `bun scripts/gate-diff-calibrate.ts --ref origin/main --count 50 --markdown`
on 2026-10-10 against `origin/main` at `20eb620` (v1.91.68.0), scanning each
commit's first-parent delta exactly as `gstack-gate-diff --commit <sha>` does
(regenerated after the RH-15? demotion and the shallow-boundary parent fix, so
the table reflects the shipped detector).
gstack squash-merges, so the 50 first-parent commits are the 50 landed PRs.
The clone was shallow; `git fetch --deepen=60 origin main` was needed first
(the tool reports a missing parent as `GATE_ERROR=no_base … fix=git fetch --deepen`).
## What this measures, and what it does not
This is the **tool's inventory**: how many hunks the floor lists per PR, how
many are eligible for the read level, how often the 50-hunk / 200 KB read cap
makes a result partial, and how often each tag fires. Tags are factual labels
(a test deleted beside its deleted source is still RH-1), so a tag count is not
a count of problems.
It does **not** measure agent findings, kept/restored/open dispositions, or
whether a reviewer would have wanted to read a tagged hunk. The design's
10-PR hand read of tagged hunks is a human step this table only supports; the
agent-level signal is collected after ship from the persisted review records.
No artifact was written: calibration never touches the state root.
## Observations from the numbers
- **Listed per PR:** median 40.5, p75 267, max 1,276 (the v1.91.8.0 test-audit
wave); eligible median 24.5. This matches the design's own measurement
(median 23–45) and confirms per-hunk disposition is unworkable; the three
levels are the right unit.
- **Partial results:** the read cap was hit in 20 of 50 PRs. For gstack's
test-heavy PRs, "partial (K of M eligible read)" will be the common state,
not the exception.
- **RH-15 (paired, 35 hunks / 14 PRs)** reads well by inspection: real
`timeout-minutes` bumps in workflows, `timeout 5000 -> 30000` in tests, a
judge threshold `4 -> 3`, an exact-count assertion relaxed to a bound, plus
tightenings labelled as such. The direction note makes the read fast.
- **RH-15? (unpaired):** as first specified ("unpaired values are tagged
RH-15? and read") the tag fired on 848 hunks across 41 PRs, dominated by
`spawnSync(…, { timeout: 30_000 })` in new or moved test code; this repository
requires a timeout on every synchronous spawn
(`test/spawnsync-timeout-tripwire.test.ts`), so nearly every test PR carried
dozens of noise tags. **Decision (integrator, per the design's demotion
rule):** `RH-15?` applies only to CI and runner/lint-config files; in test
files unpaired values are inventory and only paired RH-15 is tagged. With that
rule the table below shows 30 hunks across 13 PRs, all in workflow or runner
config. The detector was also tightened to key-value assignments (`key: N`, `key = N`, `key(N)`, `--key N`,
and a gate key opening a nested structure) and to numeric literals with `_`
separators; prose strings and `setTimeout` calls no longer fire.
- **RH-1 (411 hunks / 18 PRs):** 343 of them are the v1.91.8.0 test-retirement
PR; 29 are skip/only idioms (several are conditional
`process.platform === 'linux' ? describe : describe.skip`). Factual, and
exactly the inventory a reviewer of that PR would want listed.
- **RH-16 (80 hunks / 19 PRs):** gstack's many best-effort `catch {}` blocks.
Overlaps slop-scan as the design states; listed under the RH id.
- **RH-12, RH-13, RH-14:** rare (6 / 8 / 3 hunks), plausible on inspection.
- **RH-3 / RH-3?:** did not fire; gstack has no `__snapshots__`/`.golden`
fixtures in this window (its SKILL.md goldens are `.md`, listed by path only).
- Prose (`.md`, `.md.tmpl`) and generated/vendored paths (`dist/`, `vendor/`,
lockfiles, `.min.js`) are floor inventory only; without that rule the
design document itself tagged every RH id it describes.
## Table
| commit | subject | listed | eligible | read | unread | unmatched | unsupported | tags |
|---|---|---:|---:|---:|---:|---:|---:|---|
| 20eb6202fa8e | v1.91.68.0 fix(cso-eval): Claude producers keep the source sealed, run t | 18 | 16 | 16 | 0 | 6 | 0 | - |
| 54efba6dd5a6 | v1.91.67.0 feat(hosts): Antigravity CLI (agy) host (#3089) | 14 | 7 | 7 | 0 | 15 | 0 | - |
| 8bb6d6527076 | v1.91.66.0 fix: /autoplan without permission prompts after background re | 133 | 88 | 50 | 38 | 119 | 0 | RH-1:2 RH-15?:1 RH-16:2 |
| b357b503780f | v1.91.65.0 fix(install): skills only for the agents you installed, every | 12 | 8 | 8 | 0 | 19 | 0 | - |
| 9e9e674b1efa | v1.91.62.0 fix(cso): Rails verification fits two arm64 offline preparati | 1 | 1 | 1 | 0 | 5 | 0 | - |
| 4dfd83bf8c5c | v1.91.60.0 fix(windows): setup degrades instead of dying on a CSO failur | 25 | 8 | 8 | 0 | 17 | 0 | RH-15:1 RH-16:2 |
| 4e5ffbb857c4 | v1.91.59.0 fix(autoplan): bind init through symlinked plan directories a | 6 | 4 | 4 | 0 | 8 | 0 | - |
| 9a1dc81a2b96 | v1.91.54.0 fix: office-hours first sessions get the first-session closin | 13 | 6 | 6 | 0 | 18 | 0 | - |
| d082eccc50d2 | v1.91.52.0 fix(qa): annotations limits shape; qa-quick does less work in | 2 | 1 | 1 | 0 | 7 | 0 | - |
| c7edf46bcf6e | v1.91.47.0 fix(careful,freeze): safety hooks cover PowerShell and Notebo | 11 | 7 | 7 | 0 | 22 | 0 | - |
| 92cfd07a79ed | v1.91.45.0 feat: configurable model tiers and advisory freshness (#3073) | 52 | 26 | 26 | 0 | 60 | 0 | RH-15?:1 RH-16:1 |
| 52eec18c6347 | v1.91.42.0 fix(cso): lift the snapshot file-count ceiling, bound report | 2 | 1 | 1 | 0 | 9 | 0 | RH-16:1 |
| c7554824cbd1 | v1.91.38.0 fix(ios-qa): iPads work, a dropped route no longer relaunches | 17 | 10 | 10 | 0 | 13 | 0 | - |
| 1d1ce39b4e2b | v1.91.36.0 fix(cso): Python and Rails repairs replay, Rails verification | 9 | 4 | 4 | 0 | 11 | 0 | - |
| f67c478f05d8 | v1.91.34.0 fix: /autoplan on current Claude Code and long sessions, quie | 194 | 126 | 50 | 76 | 150 | 0 | RH-1:1 RH-15:1 RH-15?:2 RH-16:1 |
| db745675bdf9 | v1.91.33.0 feat(ship): measurement bar with explained-red and pooled dec | 26 | 18 | 18 | 0 | 16 | 0 | RH-15?:1 |
| 28f1385eac65 | v1.91.32.0 fix: Codex works from zsh, free text never enters shell comma | 120 | 65 | 50 | 15 | 155 | 0 | RH-15:1 RH-16:1 |
| 7697fb298173 | fix(cso): install Rails gems on one thread so exports stop failing (v1.9 | 2 | 0 | 0 | 0 | 7 | 0 | - |
| 55f8151fbe77 | v1.91.30.0 fix: honest eval failure reports, Bun floor with safe auto-up | 414 | 219 | 50 | 169 | 103 | 0 | RH-1:2 RH-13:4 |
| 5885157a90e5 | v1.91.29.0 fix(cso): native runtime qualification passes for Node, Bun, | 21 | 12 | 12 | 0 | 14 | 0 | - |
| c285d88b90d3 | Promote qualified CSO scanner catalog (#3042) | 0 | 0 | 0 | 0 | 1 | 0 | - |
| 10315cf44d1e | v1.91.27.0 fix(cso): runtime staging can finish and a qualified scanner | 11 | 7 | 7 | 0 | 7 | 0 | - |
| 857466ff8b93 | v1.91.25.0 feat(cso): private two-architecture release scoring + self-ch | 55 | 30 | 30 | 0 | 19 | 0 | RH-15?:1 |
| 55fbe77195b1 | v1.91.24.0 test: audit wave — PR evals keep finished work, honest ceilin | 761 | 637 | 50 | 587 | 66 | 0 | RH-1:14 RH-14:2 RH-15:1 RH-15?:8 RH-16:1 |
| f97aefe654fb | v1.91.22.0 feat(cso): reviewed, attested scanner build inputs + fixes so | 5 | 3 | 3 | 0 | 10 | 0 | - |
| 2db0b3adc84b | v1.91.19.0 fix: severe fix wave (truthful gates, no lost memory, every h | 577 | 317 | 50 | 267 | 219 | 0 | RH-1:1 RH-12:1 RH-16:11 |
| 4015c2870b06 | v1.91.18.0 feat: Opus 5.5 cleanup follow-ups — scoped transcript consent | 423 | 110 | 50 | 60 | 82 | 0 | RH-1:2 RH-16:1 |
| fe6d1ae62a42 | v1.91.17.0 fix: mvanhorn wave (checkpoint provenance, never-overwrite de | 24 | 8 | 8 | 0 | 37 | 0 | RH-16:3 |
| ac20ef1e1380 | v1.91.16.0 fix: safe installs and upgrades, skills start everywhere, tru | 420 | 288 | 50 | 238 | 228 | 0 | RH-15:2 RH-15?:2 RH-16:1 |
| 74512c20a7ac | v1.91.15.0 v1.91.15.0: Opus 5.5 prompt cleanup, behavior-level skill tes | 538 | 451 | 50 | 401 | 186 | 0 | RH-1:8 |
| f30b7b788a21 | v1.91.13.0 fix: severe bug wave (autoplan guard, land-and-deploy CI gate | 115 | 66 | 50 | 16 | 56 | 0 | RH-16:1 |
| 7fca42ad8b6c | v1.91.12.0 v1.91.12.0: audit fix wave, ~11-minute paid eval lanes, eval | 712 | 516 | 11 | 505 | 109 | 0 | RH-1:4 RH-12:1 RH-13:1 RH-15:7 RH-15?:7 RH-16:23 |
| df89475b1713 | v1.91.11.0 refactor: one state-root rule, browse route table, shared sha | 310 | 223 | 16 | 207 | 236 | 0 | RH-1:5 RH-13:3 RH-15:1 RH-16:3 |
| 96764e80a641 | v1.91.9.0 feat: test value bar in plan-eng-review, review, qa and ship, | 45 | 33 | 33 | 0 | 37 | 0 | - |
| 943105f1099c | v1.91.8.0 test: smaller suite, derived paid selection, retired never-gre | 905 | 858 | 37 | 821 | 30 | 3 | RH-1:344 RH-15:1 RH-15?:1 |
| dcaea5280055 | v1.91.7.0 feat: add functional QA and pre-publication docs checks (#2983 | 670 | 492 | 50 | 442 | 138 | 0 | RH-1:7 RH-15:4 RH-16:1 |
| 65bfb0ce49da | v1.91.6.0 feat: balance PR eval slices by recorded duration without weak | 2 | 0 | 0 | 0 | 7 | 0 | - |
| 080a655791d4 | v1.91.5.0 feat: balanced free-suite shards, test:ubicloud, and faster PR | 36 | 23 | 23 | 0 | 12 | 0 | RH-1:1 RH-15:1 RH-15?:1 |
| d2a0bbcf4c11 | v1.91.4.0 fix: Windows Opera cookie import wave (#2980, refs #2957) (#29 | 12 | 5 | 5 | 0 | 14 | 0 | - |
| 01593aa67c94 | v1.91.2.0 fix: consolidate gstack reliability wave (#2959) | 269 | 166 | 50 | 116 | 97 | 0 | RH-1:3 RH-15:5 RH-15?:1 RH-16:4 |
| 2a113ae7e623 | v1.91.1.0 fix: harden Impeccable plugin discovery (#2978) | 27 | 3 | 3 | 0 | 6 | 0 | - |
| 7b534d3e9010 | v1.90.2.0 perf: halve local free-suite time and preserve coverage (#2972 | 99 | 56 | 50 | 6 | 21 | 0 | RH-1:1 |
| a84b0b5b6d73 | v1.90.0.0 feat: make browser cookie imports explicit and safe (#2964) | 160 | 74 | 10 | 64 | 35 | 0 | RH-12:3 RH-15:3 RH-15?:1 RH-16:11 |
| 730a1017d1a1 | v1.89.1.0 fix: remove continuous checkpoint commits (#2970) | 98 | 72 | 50 | 22 | 69 | 1 | RH-1:1 |
| 06ed920a9748 | v1.89.0.0 feat: add shared-code extraction audit (#2925) | 267 | 189 | 50 | 139 | 89 | 3 | RH-1:2 RH-15:1 |
| b9706f3635b6 | v1.88.1.0 fix: harden credential boundaries and owned state (#2942) | 59 | 42 | 42 | 0 | 12 | 0 | RH-1:1 RH-12:1 RH-16:8 |
| 636175d3496e | v1.87.6.0 fix: make checks reliable and everyday validation faster (#289 | 1276 | 618 | 50 | 568 | 154 | 2 | RH-1:12 RH-14:1 RH-15:6 RH-15?:3 RH-16:4 |
| 35dd014c5800 | v1.87.5.0 perf: remove idle waits from tests and CI planning (#2897) | 77 | 47 | 47 | 0 | 12 | 0 | - |
| a6b3a57512ca | v1.87.4.0 fix: preserve health failures and disclose coverage (#2882) | 15 | 6 | 6 | 0 | 6 | 0 | - |
| 85b8c038fc00 | v1.87.3.0 fix: bind review evidence to the reviewed tree (#2875) | 32 | 19 | 19 | 0 | 29 | 0 | - |
**Scanned:** 50 of 50 first-parent commits on origin/main; 0 errored.
**Listed per commit:** median 40.5, p75 267, max 1276; eligible median 24.5, p75 126, max 858; partial (read cap hit) in 20 commit(s).
**Tags:** RH-1: 411 hunk(s) in 18 commit(s); RH-12: 6 hunk(s) in 4 commit(s); RH-13: 8 hunk(s) in 3 commit(s); RH-14: 3 hunk(s) in 2 commit(s); RH-15: 35 hunk(s) in 14 commit(s); RH-16: 80 hunk(s) in 19 commit(s); RH-15?: 30 hunk(s) in 13 commit(s).
Reference in new issue View Git Blame Copy Permalink