diff --git a/CHANGELOG.md b/CHANGELOG.md index c392fa7b6..6d642bc7a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,20 @@ # Changelog +## [Unreleased] + +**A remote gate that ran ahead of the stamp now clears /ship's small-machine obligation; gbrain stops paying a second full gate per PR.** `gstack-pregate clear --remote pregate.json/remote_gate --receipt ` accepts a `PREGATE_REMOTE:` receipt whose tree differs from pregate.json's tree only in the repo's declared stamp paths. gstack proves it itself: a `git diff` of the two trees, every differing file classified by the tree receipt's own rule (`lib/tree-receipt.ts` `classifyFile` under `.gstack/ship-policy.json` on the base branch: `stamp_files`, `stamp_patterns`, and the built-in version-file, manifest-version, lockfile-root-version and changelog-heading forms), so the pregate verdict and `gstack-tree-receipt`'s can never disagree; there is no second stamp list. gbrain's rule lets a head reuse a full gate whose tree differs only in release stamps, but /ship's Step 12 restamp always changed those files, so no gate-ahead run could satisfy `remote_gate` and every gbrain PR ran the whole gate again after /ship (#7621, #7674, #7681: 17–37 min, 64–128 vCPU each). + +### Added +- **`lib/pregate/remote-receipt.ts`** (`substituteStampOnlyTree`): both trees must be objects here (the gated commit fetched, the head tree `bin/gstack-wtree` wrote); a path outside the rule, a stamp path whose hunk is not the exact old → new rewrite, an unfetched gated tree, an unreadable base policy or equal versions refuse with `PREGATE_RECEIPT_INVALID` and a fix naming the offending path (`lib/x.ts (stamp_files: not in the repo's stamp_files allowlist, line 1)`). The tests, failed and exit checks on the receipt are unchanged. +- **The substitution is recorded.** The receipt string gains `accepted-for= stamp-only=` after `tests=`, and pregate.json's obligation carries `substitution: { gated_tree, accepted_for, policy, files: [{ path, rule }] }`; `clear` prints `PREGATE_RECEIPT_SUBSTITUTED: gated tree … accepted for tree … under ; stamp-only: (), …`, `verify` repeats it on its ok line, and a substitution whose `accepted_for` is not pregate.json's tree is `PREGATE_REMOTE_UNCLEARED`. +- **Docs:** [docs/pregate.md#remote-gate-stamp-only](docs/pregate.md#remote-gate-stamp-only) (what is accepted, what is refused, how a consuming repo declares its stamp paths: `.gstack/ship-policy.json` on the base branch, the file the tree receipt already reads, and the four-command sequence that reuses a gate-ahead run); the `PREGATE_RECEIPT_INVALID` and `PREGATE_REMOTE_UNCLEARED` troubleshooting anchors. + +### Changed +- `lib/tree-receipt.ts` exports `classifyFile` (the function is unchanged) so pregate reuses the rule instead of restating it; `PregateRemote` in `lib/headless-artifacts.ts` gains the optional `substitution` field and its schema entry. + +### Tests +- `test/pregate-remote-stamp-only.test.ts`: accept (five stamp files under a `stamp_files` + `stamp_patterns` policy, the recorded substitution equal to `gstack-tree-receipt`'s file verdicts on the same pair, `verify` ok), the unchanged same-tree path, refusals (a path outside the rule, a stamp path with a description edit, an unfetched gated tree, an unreadable receipt file, zero tests, a failure), `verify` on a misapplied substitution, and `clearDecision` without a repository root. + ## [1.91.252.0] - 2026-10-26 **`/retro` can now say whether the factory is getting better, and a new team has one page that takes it from zero to its first merged train on gstack + Capy.** `bin/gstack-factory-scorecard` reads one window (default 7 days) from GitHub through `gh` and prints a Markdown table plus a CSV: merged PRs (crew vs outside contributors), lead time (PR created → merged, and first READY comment → merged), gate runs green / red / cancelled with their known cost, cost per merge, the red-train rate, flaky events from the same two sources the daily flaky pass reads, issues opened vs closed, and releases cut. Each line carries its counting rule and what one of the best software factories would show on it; a line whose input cannot be read says `unavailable ()`, never 0. `docs/recommended-harness.md` is the adoption guide: install, the Captain and its Drive, the first train, the daily flaky pass, the weekly release, the scorecard with gstack's own numbers against the bar, what is Capy-specific, the options where two crews differ, and the known Capy limits as of 2026-10-26. diff --git a/bin/gstack-pregate b/bin/gstack-pregate index 8ce0a2083..72535a70c 100755 --- a/bin/gstack-pregate +++ b/bin/gstack-pregate @@ -33,6 +33,7 @@ import { buildContext, defaultUserConfigPath, describeRefusal, loadedDotenvFiles import { CHECKS, CHECK_IDS, PREGATE_OUT_REL, PREGATE_TIER2, clearDecision, clearObligation, readPregateFile, renderTable, runPregate, verifyPregate, writePregateFile } from '../lib/pregate/index'; import { describeLane, discoverLanes } from '../lib/pregate/workflows'; import { REMOTE_GATE_LANE } from '../lib/pregate/checks/lanes'; +import { describeSubstitution } from '../lib/pregate/remote-receipt'; import { ensureTmpExcluded } from '../lib/restamp'; import { RESULT_CODES, resultCodeSuffix, resultLine } from '../lib/result-codes'; import { DEFAULT_STALL_TIMEOUT_S } from '../lib/pregate/context'; @@ -48,7 +49,9 @@ const HELP = `gstack-pregate — cheap checks before anything paid or remote (do run (default) stage 1 mechanical preflight (${CHECKS.filter(c => c.stage === 'preflight').map(c => c.id).join(', ')}), then stage 2 test execution (${CHECKS.filter(c => c.stage === 'tests').map(c => c.id).join(', ')}) verify publication check against ${PREGATE_OUT_REL}: current tree, no fail or incomplete, every requires-remote obligation cleared clear record the receipt that clears one requires-remote obligation: pregate.json/remote_gate needs - --receipt (the gate's PREGATE_REMOTE: line, checked against the tested tree); a + --receipt (the gate's PREGATE_REMOTE: line, checked against the tested tree; a tree that + differs from pregate.json's only in the base branch's stamp rule is accepted and recorded as a + substitution, docs/pregate.md#remote-gate-stamp-only); a test_backend/ run URL is recorded as attested, not validated lanes print every CI lane with its platform tier (from .github/workflows/*.yml) validate | init the working-tree ${PREGATE_CONFIG_PATH} @@ -209,14 +212,15 @@ function cmdClear(root: string, rest: string[]): number { const out = outPath(root, rest); const read = fs.existsSync(out) ? readPregateFile(out) : { file: null, errors: [] }; if (!read.file) { process.stderr.write(resultLine('gstack-pregate clear', 'PREGATE_STALE', `${out} is missing or invalid; run gstack-pregate first`) + '\n'); return EXIT.fail; } - const decision = clearDecision(read.file, lane, { runUrl: flag(rest, '--run-url'), bundleLane: flag(rest, '--bundle-lane'), receiptPath: flag(rest, '--receipt') }); + const decision = clearDecision(read.file, lane, { runUrl: flag(rest, '--run-url'), bundleLane: flag(rest, '--bundle-lane'), receiptPath: flag(rest, '--receipt') }, root); if ('refused' in decision) { if (decision.refused === 'usage') { process.stderr.write(`gstack-pregate clear: ${decision.message}\n`); return EXIT.usage; } - process.stderr.write(`gstack-pregate clear: receipt refused — ${decision.message}; fix: ${RESULT_CODES.PREGATE_RECEIPT_INVALID.fix} ${resultCodeSuffix('PREGATE_RECEIPT_INVALID')}\n`); + process.stderr.write(`gstack-pregate clear: receipt refused — ${decision.message}; fix: ${decision.fix ?? RESULT_CODES.PREGATE_RECEIPT_INVALID.fix} ${resultCodeSuffix('PREGATE_RECEIPT_INVALID')}\n`); return EXIT.fail; } - const file = clearObligation(read.file, lane, decision.receipt, `gstack-pregate clear ${new Date().toISOString()}`); + const file = clearObligation(read.file, lane, decision.receipt, `gstack-pregate clear ${new Date().toISOString()}`, decision.substitution); writePregateFile(root, out, file); + if (decision.substitution) process.stdout.write(`PREGATE_RECEIPT_SUBSTITUTED: ${describeSubstitution(decision.substitution)}\n`); process.stdout.write(`PREGATE_CLEARED: ${lane} receipt=${decision.receipt} verdict=${file.verdict}${decision.attested ? ` (attested, not validated: ${lane} has no machine-checkable receipt)` : ''}\n`); return EXIT.ok; } diff --git a/docs/pregate.md b/docs/pregate.md index 26315aac2..fbbd80448 100644 --- a/docs/pregate.md +++ b/docs/pregate.md @@ -333,7 +333,9 @@ the supervision above. Close the obligation one of two ways: - after running it, `gstack-pregate clear --remote pregate.json/remote_gate --receipt ` with the gate's output, validated as `--remote` validates it - (a run URL is refused, `PREGATE_RECEIPT_INVALID` for a bad receipt), or + (a run URL is refused, `PREGATE_RECEIPT_INVALID` for a bad receipt; a receipt + for a tree that differs from this one only by the repo's stamp rule is + accepted and recorded, see [below](#remote-gate-stamp-only)), or `gstack-pregate clear --remote test_backend/ --run-url `, which has no machine-checkable receipt and is recorded as `attested:`; or - `gstack-pregate --stage tests --remote --spend-cap --allow-repo-commands`, @@ -350,6 +352,67 @@ the supervision above. Close the obligation one of two ways: `spent, reserved, unknown`: a gate that prints no `spent=` is `unknown`, counted at the cap. + +### A receipt from a stamp-only predecessor tree (gate ahead of the stamp) + +The receipt `clear --receipt` validates names the tree the gate tested, and +pregate.json names the tree being published. A repo whose full gate runs before +its turn in the queue ("gate ahead": the gate tests commit G, then /ship's +restamp moves the version files, so the head H it publishes is G plus the +stamp) could never clear the obligation with that run, and paid a second full +gate on H for a diff that was nothing but stamps. `clear` now accepts that +receipt for H when the two trees differ only as a stamp does: + +- gstack diffs the gated tree against pregate.json's tree + (`git diff --unified=0 `, both trees as objects in this + repository: the gated commit's tree once it is fetched, and the head tree + `bin/gstack-wtree` wrote) and classifies every differing file with the tree + receipt's own rule, `lib/tree-receipt.ts` under `.gstack/ship-policy.json` on + `origin/` ([ship-policy](ship-policy.md#tree-receipt)). That is the rule + `gstack-tree-receipt`, `gstack-pregate regate` and the train's + `verify --train` apply: with `stamp_files` set only those files can be + stamps, each hunk in them must be the exact old → new rewrite under + `stamp_patterns` (literal `{version}` templates) or the built-in forms + (`(was X)`, `since: X`, `vX`), and the version file, `package.json`'s + `version`, a lockfile's root version keys and a CHANGELOG re-heading have + their own built-in rules. There is no second list and no looser rule here: + the verdict `clear` reaches on a pair is the verdict `gstack-tree-receipt + --gated G --current H` reaches. +- Accepted, the obligation's receipt reads `remote-gate tree= + tests= accepted-for= stamp-only= receipt= + sha256=`, and pregate.json records the substitution beside it: + `substitution: { gated_tree, accepted_for, policy, files: [{ path, rule }] }`, + the policy label (`origin/@:.gstack/ship-policy.json`) and every + differing path with the rule that classed it (`version-file`, + `manifest-version`, `lockfile-root-version`, `changelog-heading`, + `stamp_files`, `stamp_paths`, `release_outputs`). `clear` prints the same + as `PREGATE_RECEIPT_SUBSTITUTED: …`, `verify` repeats it on its ok line, and + the ship receipt carries the receipt string. The gate's own tests, failed + and exit checks still apply to the receipt as before. +- Refused, with `PREGATE_RECEIPT_INVALID` and a fix naming the offending + path: any differing path outside the rule (`lib/x.ts (stamp_files: not in the + repo's stamp_files allowlist, line 1)`), a stamp path whose hunk is not the + stamp rewrite (`plugin.json (stamp_files: "…description…" -> "…", line 4)`), + a gated tree that is not an object here (fetch the gated commit first), a + base branch whose policy cannot be read, or two trees at the same version + (a `stamp_files` hunk needs an old → new version to rewrite). The obligation stays uncleared; rerun + the gate on the head, or make the extra change a stamp under the repo's + rule. + +A consuming repo declares its stamp paths once, in `.gstack/ship-policy.json` +on its base branch (`stamp_files`, `stamp_patterns`, `changelog`), the same +file the tree receipt and restamp already read; nothing is declared in +`.gstack/pregate.json` for this. gbrain's is in +[gbrain worked configs](#gbrain-worked-configs). The sequence that reuses a +gate-ahead run, on a machine under 8 cores: + +```bash +git fetch origin # the gated commit's tree must be here +gstack-pregate --allow-repo-commands # on the stamped head: records pregate.json/remote_gate uncleared +gstack-pregate clear --remote pregate.json/remote_gate --receipt +gstack-pregate verify # PREGATE_VERIFY: ok …; pregate.json/remote_gate: gated tree … accepted for tree … +``` + gstack's own example: `.gstack/pregate.json` declares `"remote_gate": "bash scripts/ubicloud/pregate-remote.sh"`. With `UBICLOUD_API_KEY` set the script runs the selected files on one Ubicloud VM diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 4091c3291..cd5a1700c 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -3106,9 +3106,24 @@ reports failures. The obligation stays uncleared. A run URL alone never clears this obligation; a `test_backend/` obligation takes a run URL, recorded as attested rather than validated. +Another tree is accepted in exactly one case: it differs from `pregate.json`'s +tree only in the repo's stamp rule (`.gstack/ship-policy.json` on the base +branch: `stamp_files`, `stamp_patterns`, the built-in version, manifest, +lockfile and CHANGELOG forms), judged by the tree receipt's own classifier on a +`git diff` of the two trees ([the gate ran ahead of the +stamp](pregate.md#remote-gate-stamp-only)). The refusal line then says `and the +trees differ beyond the stamp rule of : (: , line +)`, and the path it names is the one that is not a stamp: a file outside +`stamp_files`, or a stamp file with a hunk that is not the old → new rewrite. +`the gated tree is not an object in this repository` means the gated +commit is not fetched here, so the two trees cannot be compared. + **Fix.** Run the remote gate on the tree pregate tested and pass its output file with `--receipt`, or let `gstack-pregate --stage tests --remote --spend-cap - --allow-repo-commands` run and validate it. + --allow-repo-commands` run and validate it. For a gate-ahead receipt: +fetch the gated commit (`git fetch origin `) and clear again, or make the +named path a stamp under the repo's rule; a receipt is never accepted while any +differing path is a change. ### `PREGATE_STALE` @@ -3126,12 +3141,19 @@ refuses to write the ship receipt otherwise. ### `PREGATE_REMOTE_UNCLEARED` **Meaning.** A `requires-remote` obligation (a platform lane this machine cannot -run, named by `workflow.yml/job`) has no receipt. +run, named by `workflow.yml/job`, or the small-machine `pregate.json/remote_gate` +tier) has no receipt; or a `remote_gate` receipt accepted from a stamp-only +predecessor tree records `accepted_for` as a tree other than `pregate.json`'s +(the file was edited after `clear`), so it clears nothing here. **Fix.** Dispatch that job, then `gstack-pregate clear --remote --run-url ` (or -`--bundle-lane