docs: version flow under changelog fragments; scorecard pass cap matches the two-round review cap (GST-1003)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-11-02 17:48:39 +00:00
1 parent daf661374f
commit 839bec0e15
6 files changed
+30 -20

No files matched your search

+15 -11
View File
@@ -374,18 +374,22 @@ is not a reason to choose MINOR. Choose **MINOR (X.Y+1.0.0)** or **MAJOR
("ridiculous"), such as an incompatible public-interface change or a genuinely
new product-scale release. Make that judgment without another approval question.
Use `bin/gstack-next-version` to check the live release queue before publishing.
If a slot is claimed, advance to the next available version at the chosen bump
level and use `bin/gstack-version-bump` to synchronize release metadata. A higher
base version does not itself require a MINOR bump. Keep the PR ready for Garry to
merge; autonomous version decisions do not authorize merging, deploying or
skipping required validation.
gstack's policy sets `changelog_fragments` (#3654), so a branch never writes a
version and never checks `bin/gstack-next-version` for a slot. The merge
train's stamp (`train land`) assigns base + PATCH at merge. To ask for more,
open the branch's fragment with `bump: minor` (or `major`) front matter. A
higher base version does not itself require a MINOR bump. Keep the PR READY for
the merge train; autonomous version decisions do not authorize merging,
deploying or skipping required validation.
**VERSION and CHANGELOG are branch-scoped.** Each shipping branch gets its own
bump and one entry, written at `/ship` time, that describes what THIS branch
adds versus main, for users, with no branch-internal versions or development
narrative. VERSION stays 4-digit; package.json carries the 3-digit npm
translation owned by `bin/gstack-version-bump`, so never hand-edit it. Read
**The CHANGELOG entry is a branch fragment.** Each shipping branch writes one
entry at `/ship` time as `changelog.d/<branch-slug>.md` (entry text only, no
`## [x]` heading) that describes what THIS branch adds versus main, for users,
with no branch-internal versions or development narrative. A branch never edits
VERSION, package.json or a released CHANGELOG.md section: the merge stamp folds
the fragment into CHANGELOG.md under its version, keeps VERSION 4-digit and
writes package.json's 3-digit npm translation through `bin/gstack-version-bump`
([changelog fragments](docs/ship-policy.md#changelog-fragments)). Read
[docs/CHANGELOG_STYLE.md](docs/CHANGELOG_STYLE.md) BEFORE writing an entry: it
holds the versioning invariant, the merge-main checks, what stays out and the
entry format. Always credit community contributions with
+1 -1
View File
@@ -11,7 +11,7 @@
* gstack-factory-scorecard [--repo owner/name] [--base main] [--since-days 7]
* [--crew login,login] [--gate-workflow evals.yml] [--gate-event workflow_dispatch|any]
* [--free-workflow free-tests.yml] [--train-prefix train/] [--ready-marker <regex>]
* [--pass-marker <regex>] [--pass-cap 3] [--max-artifact-runs 60]
* [--pass-marker <regex>] [--pass-cap 2] [--max-artifact-runs 60]
* [--out-dir <dir>] [--csv <file>] [--json] [--cache-dir <dir>]
* [--fixtures <recorded gh responses.json>] [--now <iso>]
*
+2 -2
View File
@@ -263,7 +263,7 @@ line whose input could not be read says `unavailable (<reason>)` rather than 0.
gstack-factory-scorecard --repo owner/name --since-days 7 \
[--crew login,login] [--gate-workflow evals.yml] [--free-workflow free-tests.yml] \
[--train-prefix train/] [--ready-marker 'READY #\d+|SHIP_RECEIPT:'] [--pass-marker '"passes": (\d+)|passes=(\d+)'] \
[--pass-cap 3] [--out-dir .context/factory-scorecard] [--now <iso>]
[--pass-cap 2] [--out-dir .context/factory-scorecard] [--now <iso>]
```
`--repo` defaults to the origin remote's owner/name on github.com (or
@@ -280,7 +280,7 @@ measured later does not count anything that happened after that week.
| Merged PRs | `merged_at` in the window, base = default branch; crew by `--crew` logins, else author association OWNER/MEMBER/COLLABORATOR | several merges a day, every day; outside contributions land in the same week |
| Lead time, created → merged | PR `created_at` to `merged_at`, median and p90 | median under 24h, p90 under 3 days (DORA "elite" change lead time is under a day) |
| Lead time, READY → merged | first PR comment matching `--ready-marker` at or before `merged_at`; PRs with no such comment are left out and counted | under 2h; every merged PR carries a READY comment |
| Review passes per PR | the largest count `--pass-marker` captures in the PR body (the ship receipt's `"passes": N` once /ship writes it) or a PR comment (`passes=N`, `REVIEW_PASSES: N`), median and p90, plus how many PRs sat at `--pass-cap` (3: pass 1 reviews the whole diff, later passes only the delta, a pass that finds nothing new is the exit) | median 2 passes; none at the cap |
| Review passes per PR | the largest count `--pass-marker` captures in the PR body (the ship receipt's `"passes": N` once /ship writes it) or a PR comment (`passes=N`, `REVIEW_PASSES: N`), median and p90, plus how many PRs reached `--pass-cap` (2, the [review policy](review-policy.md)'s cap of two rounds, a fix-check included: pass 1 reviews the whole diff, pass 2 only the range since, and a pass that finds nothing blocking is the exit) | median 1 pass; few at the cap |
| Gate runs green / red / cancelled | completed runs of the gate workflow (`workflow_dispatch` by default; `--gate-event any`); cost from the run's `report-verdict-a*` artifact (`collector-outcomes.json` `.verdict.costUsd`), else the `**$N** rule/judge cost` or `cost=$N` line of the PR's verdict comment | green on 95%+ of runs, cancelled near 0, cost known for every run |
| Known gate cost per merged PR | known cost / merged PRs; unknown-cost runs make it a floor | known, stable or falling week over week |
| Red-train rate | runs whose head branch starts with `--train-prefix`, cancelled excluded; red = conclusion other than success; red backstops and screens shown beside it | under 5% (each red train is a bisect plus a rebuild, about two gates) |
+9 -3
View File
@@ -112,10 +112,16 @@ refuses the file by name (`POLICY_GSTACK_TOO_OLD`).
`.gstack/ship-policy.example.json` shows every key set the way a serial-merge
repo with a release script, docs mentions and a mirror would set them.
### Queue-side procedure for gstack
<a id="queue-side-procedure-for-gstack"></a>
### Queue-side procedure under `stamp-at-merge` without fragments
Who does what under `stamp-at-merge`, for the PR author and for the thread
running the merge queue:
gstack itself no longer follows these steps. Since #3654 (v1.91.342.0) its
policy sets [`changelog_fragments`](#changelog-fragments), so a gstack PR adds
a fragment and never a version, and trains merge through
[`train land`](#stamped-trains-under-changelog_fragments-train-land). The steps
below are for a repo on `stamp-at-merge` whose policy has no
`changelog_fragments`. Who does what there, for the PR author and for the
thread running the merge queue:
1. **Open.** The branch carries a tentative stamp of the base's version plus
PATCH (`gstack-version-bump write --version <base+patch> --regen-digest`)
+2 -2
View File
@@ -70,7 +70,7 @@ export interface ScorecardOptions {
export const DEFAULTS = {
base: 'main', sinceDays: 7, gateWorkflow: 'evals.yml', gateEvent: 'workflow_dispatch', freeWorkflow: 'free-tests.yml',
trainPrefix: 'train/', readyMarker: 'READY #\\d+|SHIP_RECEIPT:|gstack-ship-receipt',
passMarker: '"passes"\\s*:\\s*(\\d+)|\\bpasses=(\\d+)|\\bREVIEW_PASS(?:ES)?[:= ]\\s*(\\d+)', passCap: 3, maxArtifactRuns: 60,
passMarker: '"passes"\\s*:\\s*(\\d+)|\\bpasses=(\\d+)|\\bREVIEW_PASS(?:ES)?[:= ]\\s*(\\d+)', passCap: 2, maxArtifactRuns: 60,
} as const;
export const CREW_ASSOCIATIONS = new Set(['OWNER', 'MEMBER', 'COLLABORATOR']);
@@ -399,7 +399,7 @@ export const BENCHMARKS: Record<string, string> = {
merged: 'several merges a day, every day; outside contributions land in the same week',
lead_time: 'median under 24h, p90 under 3 days (DORA elite: change lead time under a day)',
ready_to_merge: 'under 2h from READY to merge; every merged PR carries a READY comment',
review_passes: 'median 2 passes, none at the cap (a pass that finds nothing new is the exit; what is left at the cap is named in the PR body)',
review_passes: 'median 1 pass, few at the cap (a pass that finds nothing blocking is the exit; a PR at the cap ships with what is left named in the PR body)',
gates: 'green on 95%+ of runs, cancelled near 0, cost known for every run',
cost_per_merge: 'known, stable or falling week over week',
red_train: 'under 5% of trains red (each red train costs a bisect and a rebuild)',
+1 -1
View File
@@ -59,7 +59,7 @@ describe('factory-scorecard: recorded window', () => {
expect(values.merged).toBe('3 (crew 2, outside 1)');
expect(values.lead_time).toBe('median 8.0h, p90 5.0d');
expect(values.ready_to_merge).toBe('median 1.0h, p90 2.0h (2 of 3 PRs with a READY comment)');
expect(values.review_passes).toBe('median 2, p90 3 (2 of 3 PRs report a pass count; 1 at the cap of 3)');
expect(values.review_passes).toBe('median 2, p90 3 (2 of 3 PRs report a pass count; 2 at the cap of 2)');
expect(values.gates).toBe('2 / 1 / 1; known cost $43.00 on 2 of 4 runs');
expect(values.cost_per_merge).toBe('$14.33 (2 runs without a known cost)');
expect(values.red_train).toBe('50% (1 red of 2 trains; 0 red backstop, 0 red screens)');