From 94771d927f29d2c252d97bbc2ff883e76c4d25d1 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Tue, 3 Nov 2026 06:06:33 -0800 Subject: [PATCH 1/4] v1.94.16.0 fix(stamp): declared placeholder restamps are stamps; carry records HEAD's version (GST-1026, GST-1030) (#3892) * fix(stamp): name branch lines by position; carry and finish accept a kept VERSION claim (GST-979, GST-980) BranchSide carries the positions of the branch-added lines and stampOnlyLine takes the removed line position, so a base line whose text duplicates a branch line keeps the old-version floor. At equal versions only the branch own lines may move. stampOnlyText and stampedFiles accept a kept claim for those lines, and the carry leaves a stamp file out of the product comparison when its only change since the reviewed head is a stamp on the branch own rows and the base did not touch it. Failing-first: all five new tests fail on main fc292e22f. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (GST-980 carry reads the stamp file line by line at the branch's positions, and keeps its mode) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): GST-979/980 entry counts the seven tests Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: the receipt carry follows a stamp on the branch's own rows (GST-980) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: keep docs/ship-policy.md at the reviewed text; its GST-980 sentence already states the carry rule Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Revert "docs: keep docs/ship-policy.md at the reviewed text; its GST-980 sentence already states the carry rule" This reverts commit f926ccb0c206ce9b1cf2387f3500558a20102510. * fix(stamp): a declared placeholder restamped on the branch's own row is a stamp; the carry names HEAD's version (GST-1026, GST-1030) stamp_placeholders (default []) lets a repo declare literals such as TBD that its restamp replaces with the new version inside a stamp_patterns template. stampOnlyLine accepts that move only on a branch line named by position; finish's stampedFiles, the receipt carry and the tree receipt share it. The carried receipt's version is HEAD's, read the way write reads it. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (carry keeps a write --version receipt's version without a version source; a placeholder fill composes with a stamp move; a bare {version} template and a blank placeholder take no placeholder) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): GST-1026/1030 fragment names the review fixes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship-receipt): carry masks only package.json version on a pin-bump PR (GST-1030, GBEV-3) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * refactor(ship-policy): stamp key validation moves to lib/ship-policy-stamps.ts for the module-size ratchet Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ship-policy): stamp_placeholders key names the next minor 1.95.0.0 1.94.0.0 shipped as #3901 without the key, so the released-key test reds once main's CHANGELOG is in the tree. The slot stamp moves the tentative value through stamp_paths. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(restamp-mentions): set the stamp file's exec bit through the index so the mode-change carry case holds on Windows Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * unstamp: return VERSION, package.json, CHANGELOG and the digest to main f6d6215's values and restore the changelog fragment after #3892 left train 100 The tentative STAMP_PLACEHOLDERS_KEY_VERSION goes back to 1.95.0.0 for the next slot stamp to move. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 20 +++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- bin/gstack-ship-receipt | 23 +++- docs/ship-policy.md | 1 + lib/restamp.ts | 2 +- lib/ship-bin-common.ts | 9 +- lib/ship-policy-stamps.ts | 35 +++++ lib/ship-policy.ts | 23 ++-- lib/ship-receipt.ts | 6 +- lib/stamp-only.ts | 32 ++++- lib/tree-receipt.ts | 9 +- package.json | 2 +- test/gstack-ship-policy.test.ts | 28 +++- test/gstack-ship-receipt-docs.test.ts | 38 ++++++ test/restamp-mentions.test.ts | 159 +++++++++++++++++++++- test/ship-rereview-triggers.test.ts | 32 +++++ test/tree-receipt-stamp-sensitive.test.ts | 30 ++++ 18 files changed, 414 insertions(+), 39 deletions(-) create mode 100644 lib/ship-policy-stamps.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index b371c3a62..3ee6eb368 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,25 @@ # Changelog +## [1.94.16.0] - 2026-11-03 + +**A declared placeholder restamped to the new version is a stamp, and a carried receipt names the head's version (GST-1026, GST-1030).** gbrain writes a PR's own BEHAVIOR_CHANGES row as `since: 'TBD'` and its `release:restamp` fills in the version. Finish refused that receipt ("behavior, tests or build inputs changed since review"), and the receipt carry and the tree receipt would have refused the same row (gbrain #8444). Separately, a receipt carried after a main merge and a restamp kept the version of the head it was written at (GGST-1). + +### Added +- **`stamp_placeholders` policy key** (default `[]`, so nothing changes for a repo that does not set it): literal placeholders such as `["TBD"]`, at most 16, one line and 32 characters each, never `{version}` and never a version. It needs `min_gstack_version` at the release that introduced it. + +### Fixed +- **`lib/stamp-only.ts`:** `stampOnlyLine` takes the policy's placeholders. On a line the branch added itself (named by its position), a placeholder inside a `stamp_patterns` template may become that template at the new version, alongside any stamp-form version move on the same line. The built-in forms, a bare `{version}` template, base lines (including one whose text duplicates a branch line) and undeclared placeholders are still refused. Finish's `stampedFiles`, the receipt carry and the tree receipt share it. +- **`bin/gstack-ship-receipt` carry:** the carried receipt's `version` is HEAD's, read the way `write` reads it, instead of the version copied from the old receipt. A head with no version source keeps the version `write --version` recorded. +- **`bin/gstack-ship-receipt` carry, pin-bump PRs:** a `package.json` the branch changed as product (a dependency pin) that both main and a restamp moved since review is compared with only its `version` field masked, at all four sides. The receipt carries when nothing else in the file moved, and still refuses when the pin or any other field changed (GBEV-3, gbrain-evals #306). + +### Tests +- `test/tree-receipt-stamp-sensitive.test.ts`, `test/gstack-ship-receipt-docs.test.ts`: the shared predicate and the carry classifier accept gbrain's `since: 'TBD'` → `since: '0.60.553.0'` row and refuse an undeclared placeholder, a base row, a duplicated base copy, a built-in form and a line no template matches. +- `test/ship-rereview-triggers.test.ts`: finish routes the declared row as metadata and stops on the same controls. +- `test/restamp-mentions.test.ts`: the tree receipt and the carry accept the declared row and refuse it undeclared or on a base row; the carried receipt records the head's version after a main merge and a restamp; a version-only restamp of an EVIDENCE.md declared in `stamp_files` carries, and an undeclared one stays product; a pin-bump PR's receipt carries after a main merge and a restamp, and refuses when the pin moved since review. +- `test/gstack-ship-policy.test.ts`: the key's default, floor and validation. + +The EVIDENCE.md refusal (GST-1030's second point) reproduces on main and is already fixed by the GST-980 own-row carry this branch builds on, so this change adds only the test for it. + ## [1.94.15.0] - 2026-11-03 - **A stacked PR keeps its /ship receipt after its parent squash-merges, even when the receipt was written against main (GST-1199).** `gstack-ship-receipt carry` used to refuse with `RECEIPT_CARRY_PRODUCT_CHANGED` once the parent's hunks moved into main, forcing a full re-review of unchanged code. The carry now finds the newest reviewed commit that main already holds (`git merge-tree` into main changes no product path) and compares from there: an unchanged child carries as `stack parent landed`, base edits next to the child's hunks go to `RECEIPT_CARRY_SCOPED_REVIEW`, and any other change still refuses. A parent that never landed never counts, even if the child's merge dropped its change. diff --git a/VERSION b/VERSION index f2224bc7b..e1438391e 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.94.15.0 +1.94.16.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index d3eba6498..18e81912e 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.94.15.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.94.16.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-ship-receipt b/bin/gstack-ship-receipt index 41a4490e5..e6c01c637 100755 --- a/bin/gstack-ship-receipt +++ b/bin/gstack-ship-receipt @@ -507,13 +507,13 @@ function verifiedOutputs(repoRoot: string, base: string, pregateFile: string | u return regen.proven.filter(p => !/[*?[\]\\]/.test(p) && matchesAny(owned, p)); } -interface ProductRules { extra: { metadata: string[]; outputs: readonly string[]; fragments?: string }; metadata: string[]; templates: readonly string[] } +interface ProductRules { extra: { metadata: string[]; outputs: readonly string[]; fragments?: string }; metadata: string[]; templates: readonly string[]; placeholders: readonly string[] } function productRules(repoRoot: string, base: string, outputs: readonly string[]): ProductRules { const load = loadPolicyFromBase(repoRoot, base); const policy: ShipPolicy | null = isUnavailable(load) ? null : load.policy; const fragments = policy?.changelog_fragments?.dir.replace(/^(\.\/)+/, '').replace(/\/+$/, ''); const extra = { metadata: [...(policy?.stamp_files ?? []), ...(policy?.stamp_paths ?? []), ...(policy?.release_outputs ?? []), ...(policy?.changelog ? [policy.changelog] : [])], outputs, fragments }; - return { extra, metadata: [...CARRY_METADATA_GLOBS, ...extra.metadata, ...(fragments ? [`${fragments}/*.md`] : []), ...outputs], templates: policy?.stamp_patterns ?? [] }; + return { extra, metadata: [...CARRY_METADATA_GLOBS, ...extra.metadata, ...(fragments ? [`${fragments}/*.md`] : []), ...outputs], templates: policy?.stamp_patterns ?? [], placeholders: policy?.stamp_placeholders ?? [] }; } /** @@ -540,7 +540,7 @@ function productSpec(repoRoot: string, rules: ProductRules, from: string, to: st const versions = new Map(); const version = (ref: string) => { if (!versions.has(ref)) versions.set(ref, versionAt(ref)); return versions.get(ref)!; }; const stamped = (p: string) => matchesAny(rules.metadata, p) && !matchesAny(rules.extra.outputs, p); - const classify = (p: string, a: string, b: string) => classifyCarryPath(p, rules.extra, stamped(p) ? { before: blobAt(a, p), after: blobAt(b, p), oldVersion: version(a), newVersion: version(b), templates: rules.templates, side: a === from && b === to ? branch?.own(p) : undefined } : undefined); + const classify = (p: string, a: string, b: string) => classifyCarryPath(p, rules.extra, stamped(p) ? { before: blobAt(a, p), after: blobAt(b, p), oldVersion: version(a), newVersion: version(b), templates: rules.templates, placeholders: rules.placeholders, side: a === from && b === to ? branch?.own(p) : undefined } : undefined); const changed = changedRaw.split('\n').filter(Boolean).map(p => ({ path: p, class: classify(p, from, to) })); const excludeGlobs = (globs: readonly string[]) => globs.map(g => `:(exclude,glob)${g}`); const demoted = !branch ? changed.filter(c => c.class === 'product' && matchesAny(rules.metadata, c.path)).map(c => c.path) @@ -556,7 +556,7 @@ function productSpec(repoRoot: string, rules: ProductRules, from: string, to: st const was = before.split('\n'); const now = after.split('\n'); const lines = was.map((_, i) => i).filter(i => was[i] !== now[i]); - return was.length === now.length && lines.length > 0 && lines.every(i => side.at.has(i + 1) && stampOnlyLine(was[i]!, now[i]!, oldVersion, newVersion, rules.templates, side, i + 1)); + return was.length === now.length && lines.length > 0 && lines.every(i => side.at.has(i + 1) && stampOnlyLine(was[i]!, now[i]!, oldVersion, newVersion, rules.templates, side, i + 1, rules.placeholders)); }); return { changed, product: ['.', ...excludeGlobs(CARRY_TEST_GLOBS), ...excludeGlobs(metadataGlobs)], tests: [...CARRY_TEST_GLOBS.map(g => `:(glob)${g}`), ...excludeGlobs(metadataGlobs)], demoted, ownRows }; } @@ -718,6 +718,16 @@ function cmdCarry(rest: string[], json: boolean): number { if (changedProduct.length >= CARRY_NAMED_PATHS) break; const cls = changed.find(c => c.path === p)?.class ?? classifyCarryPath(p, rules.extra); if (cls === 'metadata' && spec.ownRows.includes(p) && !movedBase.has(p)) stampOnly.push(p); + if (cls === 'metadata' && spec.demoted.includes(p) && movedBase.has(p) && /(^|\/)package\.json$/.test(p)) { + const masked = Object.values(sides).map(ref => { + const r = spawnSync('git', ['show', `${ref}:${p}`], { cwd: repoRoot, encoding: 'utf8', timeout: 30_000, maxBuffer: 16 * 1024 * 1024 }); + try { const { version: _, ...rest } = JSON.parse(r.status === 0 ? r.stdout : 'null') ?? {}; return JSON.stringify(rest); } catch { return null; } + }); + const [rb, rv, cb, cv] = masked; + if (masked.some(m => m === null) || rv !== cv || rb !== cb) changedProduct.push(p); + else stampOnly.push(p); + continue; + } if (cls !== 'product') continue; const verdict = comparePath(repoRoot, sides, p); if (verdict === 'changed') changedProduct.push(p); @@ -795,7 +805,10 @@ function cmdCarry(rest: string[], json: boolean): number { depthField = { review_depth: { ...now, depth: deeper(receipt.review_depth.depth, now.depth) }, review_exclude: receiptExclude(review.exclude) }; } // The rebound product_patch_id is HEAD's diff against the carry's base, so a retargeted receipt names that base from now on. - const rebound = { ...receipt, ...(retarget ? { base: base.replace(/^origin\//, '') } : {}), ...(pregateField ?? {}), ...depthField }; + // A receipt that records a version names HEAD's, read as write reads it: a restamp after a base merge moves it (GST-1030). + // A head with no version source keeps the recorded one (write --version named it). + const version = receipt.version === undefined ? undefined : (receiptVersion(repoRoot, head, base) ?? receipt.version); + const rebound = { ...receipt, ...(version !== undefined ? { version } : {}), ...(retarget ? { base: base.replace(/^origin\//, '') } : {}), ...(pregateField ?? {}), ...depthField }; const carried = carryReceipt(rebound, head, plan.carry, { tree: headTree, fromTree, productPatchId: headPatch }); const errors = validateReceipt(carried); if (errors.length) return refused('RECEIPT_CARRY_REFUSED', `the carried receipt does not validate: ${errors[0]!.message} at ${errors[0]!.path}`); diff --git a/docs/ship-policy.md b/docs/ship-policy.md index f25e55b85..6612a8aa3 100644 --- a/docs/ship-policy.md +++ b/docs/ship-policy.md @@ -53,6 +53,7 @@ refuses the file (`POLICY_GSTACK_TOO_OLD`). `validate` ends with | `maintainers` | GitHub logins | `[]` | Maintainer set for `/review` contributor mode; empty means repository push permission. | | `stamp_files` | globs | `[]` | The repo's stamp rule: when set, only these files may differ by a stamp (list the version file, manifests, plugin trees and the files holding the PR's own `since:` rows). It applies before every other stamp rule, so a doc `vX` mention outside it is a difference. | | `stamp_patterns` | literal templates | `[]` | At most 16 templates of at most 128 characters, each with exactly one `{version}` slot (`"version": "{version}"`, `since: '{version}'`, ``); matched literally at a version boundary, never as a regex. The restamp writer, its mention scanner and the tree receipt use them beside the built-in forms. | +| `stamp_placeholders` | literal placeholders | `[]` | At most 16 one-line literals of at most 32 characters, never `{version}` and never a version (`["TBD"]`). On a line the branch added itself, a placeholder inside a `stamp_patterns` template (`since: 'TBD'` under `since: '{version}'`) may become that template at the new version, and finish's route, the receipt carry and the tree receipt read the change as a stamp. The built-in forms never take a placeholder, and neither does a base line, even one whose text duplicates a branch line. The writer does not fill placeholders: the repo's `restamp_command` does. A newer key: set `min_gstack_version` to the release `gstack-ship-policy validate` names. Empty accepts no placeholder. | | `restamp_command` | command or `null` | `null` | The repo's own restamp, run first in the staging worktree; see [restamp_command](#restamp-command). | | `migrations` | `{ dir, id }` or `null` | `null` | The migration directory and a regex with exactly one capture group for the number in a file name (`"^v(\\d+)-"`); `gstack-pregate regate` reports `MIGRATION_COLLISION`. | | `gate_command` | command or `null` | `null` | The repo's full pre-merge gate. `gstack-ship-queue gate-ahead` and `ready` print it for the caller to run (in the background or in CI); gstack never executes it. Needs `min_gstack_version`. | diff --git a/lib/restamp.ts b/lib/restamp.ts index be2c6fc23..9eba5f88e 100644 --- a/lib/restamp.ts +++ b/lib/restamp.ts @@ -313,7 +313,7 @@ export function branchAddedLines(cwd: string, since: string, files: string[], to return out; } -const escapeRegExp = (t: string) => t.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +export const escapeRegExp = (t: string) => t.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); const VERSION_RE = '(?\\d+(?:\\.\\d+){1,3})(?![0-9.])'; /** * The built-in stamp forms as scanners, the version in the `v` group: `(was X)`, `since: X` and `vX`. diff --git a/lib/ship-bin-common.ts b/lib/ship-bin-common.ts index 51bb7d167..e36ef6ed7 100644 --- a/lib/ship-bin-common.ts +++ b/lib/ship-bin-common.ts @@ -113,7 +113,8 @@ export function releasePaths(policy: Record | null, cwd: string, re * added since the merge base with origin/, and not added by the base since): after a lost slot and a merge of the * base its stale stamp is older than `ref`'s VERSION, so it must be at or newer than the fork point's version instead, * the bound gstack-restamp's stamp_paths stage uses (GST-895; the fork version itself since GST-963). A branch line is - * named by its position, and with a kept VERSION only such a line may move (GST-979, GST-980). + * named by its position, and with a kept VERSION only such a line may move (GST-979, GST-980); on it a declared + * `stamp_placeholders` value may also become the new version inside a `stamp_patterns` template (GST-1026). */ export function stampedFiles(policy: Record | null, cwd: string, ref: string, base: string): string[] { const globs = Array.isArray(policy?.stamp_files) ? policy.stamp_files.filter((s: unknown): s is string => typeof s === 'string') : []; @@ -123,11 +124,13 @@ export function stampedFiles(policy: Record | null, cwd: string, re let after = ''; try { after = extractVersion(fs.readFileSync(path.join(cwd, before.rel), 'utf8'), before.rel); } catch { return []; } if (!after) return []; - const templates = Array.isArray(policy?.stamp_patterns) ? policy.stamp_patterns.filter((s: unknown): s is string => typeof s === 'string') : []; + const strings = (v: unknown): string[] => (Array.isArray(v) ? v.filter((s): s is string => typeof s === 'string') : []); + const templates = strings(policy?.stamp_patterns); + const placeholders = strings(policy?.stamp_placeholders); const diff = git(cwd, ['diff', '--unified=0', '--no-color', '--no-ext-diff', '--no-textconv', '--no-renames', ref]); if (diff.status !== 0) return []; const side = branchSides(cwd, `origin/${base}`, 'HEAD', before.version, ref); - const stamp = (rel: string, removed: string, added: string, at: number): boolean => stampOnlyLine(removed, added, before.version, after, templates, side(rel), at); + const stamp = (rel: string, removed: string, added: string, at: number): boolean => stampOnlyLine(removed, added, before.version, after, templates, side(rel), at, placeholders); return parseUnifiedDiff(diff.stdout).filter(f => f.status === 'M' && !f.binary && !f.modeChanged && matchesAny(globs, f.path) && f.hunks.length > 0 && f.hunks.every(h => h.removed.length === h.added.length && h.removed.every((line, i) => stamp(f.path, line, h.added[i]!, h.removedAt[i]!)))).map(f => f.path); } diff --git a/lib/ship-policy-stamps.ts b/lib/ship-policy-stamps.ts new file mode 100644 index 000000000..6e0b144da --- /dev/null +++ b/lib/ship-policy-stamps.ts @@ -0,0 +1,35 @@ +/** + * Validation of the stamp policy keys (stamp_patterns templates and stamp_placeholders), split out of ship-policy.ts + * to keep that module under the module-size ratchet. + */ +import type { PolicyError } from './ship-policy'; +import { parseVersion } from './version-source'; + +const MAX_TEMPLATES = 16; +const MAX_TEMPLATE = 128; +const MAX_PLACEHOLDERS = 16; +const MAX_PLACEHOLDER = 32; +export const VERSION_SLOT = '{version}'; + +export function templateErrors(value: unknown, out: PolicyError[]): void { + if (!Array.isArray(value)) { out.push({ code: 'POLICY_INVALID', path: 'stamp_patterns', message: 'must be an array of literal templates' }); return; } + if (value.length > MAX_TEMPLATES) out.push({ code: 'POLICY_CONTAINMENT', path: 'stamp_patterns', message: `more than ${MAX_TEMPLATES} templates` }); + value.forEach((t, i) => { + const at = `stamp_patterns[${i}]`; + if (typeof t !== 'string') { out.push({ code: 'POLICY_INVALID', path: at, message: 'must be a string' }); return; } + if (t.split(VERSION_SLOT).length !== 2) out.push({ code: 'POLICY_INVALID', path: at, message: `${JSON.stringify(t)} must contain exactly one ${VERSION_SLOT}` }); + else if (t.length > MAX_TEMPLATE || /[\0\r\n]/.test(t)) out.push({ code: 'POLICY_CONTAINMENT', path: at, message: `must be one line of at most ${MAX_TEMPLATE} characters` }); + }); +} + +export function placeholderErrors(value: unknown, out: PolicyError[]): void { + if (!Array.isArray(value)) { out.push({ code: 'POLICY_INVALID', path: 'stamp_placeholders', message: 'must be an array of literal placeholders' }); return; } + if (value.length > MAX_PLACEHOLDERS) out.push({ code: 'POLICY_CONTAINMENT', path: 'stamp_placeholders', message: `more than ${MAX_PLACEHOLDERS} placeholders` }); + value.forEach((p, i) => { + const at = `stamp_placeholders[${i}]`; + if (typeof p !== 'string') { out.push({ code: 'POLICY_INVALID', path: at, message: 'must be a string' }); return; } + if (!p.trim() || p.length > MAX_PLACEHOLDER || /[\0\r\n]/.test(p)) out.push({ code: 'POLICY_CONTAINMENT', path: at, message: `must be one non-empty line of at most ${MAX_PLACEHOLDER} characters` }); + else if (p.includes(VERSION_SLOT)) out.push({ code: 'POLICY_INVALID', path: at, message: `${JSON.stringify(p)} must not contain ${VERSION_SLOT}` }); + else if (parseVersion(p)) out.push({ code: 'POLICY_INVALID', path: at, message: `${JSON.stringify(p)} is a version, not a placeholder` }); + }); +} diff --git a/lib/ship-policy.ts b/lib/ship-policy.ts index 0092b3be4..3fd85545c 100644 --- a/lib/ship-policy.ts +++ b/lib/ship-policy.ts @@ -18,6 +18,8 @@ import * as path from 'node:path'; import { defaultBaseBranch, noDefaultBaseLine } from './default-base'; import { RESULT_CODES, type ResultCodeName } from './result-codes'; import { cmpVersion, parseVersion } from './version-source'; +import { placeholderErrors, templateErrors } from './ship-policy-stamps'; +export { VERSION_SLOT } from './ship-policy-stamps'; export const SHIP_POLICY_PATH = '.gstack/ship-policy.json'; export const SHIP_POLICY_SCHEMA = 'https://github.com/garrytan/gstack/docs/ship-policy.md#schema-v1'; @@ -64,6 +66,8 @@ export interface ShipPolicy { stamp_files: string[]; /** Literal stamp templates with one `{version}` slot, beside the built-in `(was X)`, `since: X`, `vX`; writer and classifier share them. */ stamp_patterns: string[]; + /** Placeholders (`TBD`) a restamp may replace with the new version inside a `stamp_patterns` template, on the branch's own lines only. */ + stamp_placeholders: string[]; /** The repo's own restamp, run first in the staging worktree with the target version; gstack's version writer is skipped. */ restamp_command: string | null; /** Migration numbering: `{ dir, id }` where `id` is a regex with one capture group over the file name. */ @@ -161,6 +165,7 @@ export const DEFAULT_POLICY: ShipPolicy = { maintainers: [], stamp_files: [], stamp_patterns: [], + stamp_placeholders: [], restamp_command: null, migrations: null, gate_command: null, @@ -193,6 +198,7 @@ export const POLICY_EXPLANATIONS: Record = { maintainers: 'GitHub logins treated as maintainers by /review contributor mode; empty uses repository push permission.', stamp_files: 'The repo’s stamp rule: globs of the only files that may differ by a stamp (version file, manifests, plugin trees, the PR’s own since: rows). Empty keeps gstack’s per-file rules; set, it applies before every rule.', stamp_patterns: 'Literal templates with exactly one {version} slot (at most 16, 128 characters each), for example `"version": "{version}"` or `since: \'{version}\'`; the restamp writer and the tree receipt use them beside (was X), since: X and vX.', + stamp_placeholders: 'Literal placeholders (at most 16, one line, 32 characters each, never `{version}` or a version), for example `["TBD"]`: a line the branch added that reads a placeholder in a stamp_patterns template (`since: \'TBD\'`) may become that template at the new version, and finish, the receipt carry and the tree receipt read it as a stamp. Empty (default) accepts no placeholder.', restamp_command: 'The repo’s own restamp, run first in gstack-restamp’s staging worktree (a clean checkout of HEAD) with the target version as its argument and in GSTACK_RESTAMP_VERSION; gstack’s version writer is skipped, output confinement still applies. Needs a clean caller tree and --allow-repo-commands.', migrations: '{ dir, id }: the migration directory and a regex with one capture group for the number in a file name; `gstack-pregate regate` reports MIGRATION_COLLISION against the base branch and every open PR.', gate_command: 'The repo’s full pre-merge gate command. gstack-ship-queue gate-ahead and ready print it for the caller to run (in the background or in CI); gstack never executes it. null when the repo has none.', @@ -236,6 +242,7 @@ export const BUMP_NEVER_KEY_VERSION = '1.91.353.0'; export const BASE_RED_KEY_VERSION = '1.91.374.0'; /** `train.gates` and `train.land` are values of the older train key, so compatErrors checks them by value, as for bump: never. */ export const TRAIN_GATES_KEY_VERSION = '1.92.0.0'; +export const STAMP_PLACEHOLDERS_KEY_VERSION = '1.94.16.0'; export const REVIEW_EXCLUDE_KEY_VERSION = '1.93.33.0'; export const POLICY_KEY_INTRODUCED: Record = { bump: '1.91.73.0', queue_mode: '1.91.73.0', release_tool: '1.91.73.0', release_outputs: '1.91.73.0', stamp_paths: '1.91.73.0', @@ -245,6 +252,7 @@ export const POLICY_KEY_INTRODUCED: Record = { train: TRAIN_KEYS_VERSION, review_scope: REVIEW_SCOPE_KEY_VERSION, docs: DOCS_CAPS_KEY_VERSION, changelog_fragments: CHANGELOG_FRAGMENTS_KEY_VERSION, base_red: BASE_RED_KEY_VERSION, review_exclude_paths: REVIEW_EXCLUDE_KEY_VERSION, + stamp_placeholders: STAMP_PLACEHOLDERS_KEY_VERSION, }; const newer = (a: string, b: string): boolean => cmpVersion(parseVersion(a)!, parseVersion(b)!) > 0; @@ -274,9 +282,6 @@ const MAX_PATTERN = 256; const MAX_REGEX = 512; const MAX_COMMAND = 512; const SHA_RE = /^[0-9a-f]{7,64}$/; -const MAX_TEMPLATES = 16; -const MAX_TEMPLATE = 128; -export const VERSION_SLOT = '{version}'; function hasDotDot(p: string): boolean { return p.split(/[\\/]+/).some(seg => seg === '..'); @@ -343,17 +348,6 @@ function excludeErrors(value: unknown, out: PolicyError[]): void { }); } -function templateErrors(value: unknown, out: PolicyError[]): void { - if (!Array.isArray(value)) { out.push({ code: 'POLICY_INVALID', path: 'stamp_patterns', message: 'must be an array of literal templates' }); return; } - if (value.length > MAX_TEMPLATES) out.push({ code: 'POLICY_CONTAINMENT', path: 'stamp_patterns', message: `more than ${MAX_TEMPLATES} templates` }); - value.forEach((t, i) => { - const at = `stamp_patterns[${i}]`; - if (typeof t !== 'string') { out.push({ code: 'POLICY_INVALID', path: at, message: 'must be a string' }); return; } - if (t.split(VERSION_SLOT).length !== 2) out.push({ code: 'POLICY_INVALID', path: at, message: `${JSON.stringify(t)} must contain exactly one ${VERSION_SLOT}` }); - else if (t.length > MAX_TEMPLATE || /[\0\r\n]/.test(t)) out.push({ code: 'POLICY_CONTAINMENT', path: at, message: `must be one line of at most ${MAX_TEMPLATE} characters` }); - }); -} - function migrationErrors(value: unknown, out: PolicyError[]): void { const m = value as Record | null; if (!m || typeof m !== 'object' || Array.isArray(m)) { out.push({ code: 'POLICY_INVALID', path: 'migrations', message: 'must be { dir, id } or null' }); return; } @@ -497,6 +491,7 @@ export function validatePolicy(raw: unknown, opts: { gstackVersion?: string } = if ('release_outputs' in obj) listErrors('release_outputs', obj.release_outputs, errors, globError); if ('stamp_files' in obj) listErrors('stamp_files', obj.stamp_files, errors, globError); if ('stamp_patterns' in obj) templateErrors(obj.stamp_patterns, errors); + if ('stamp_placeholders' in obj) placeholderErrors(obj.stamp_placeholders, errors); if ('restamp_command' in obj && obj.restamp_command !== null) { const t = obj.restamp_command; if (typeof t !== 'string' || !t.trim()) errors.push({ code: 'POLICY_INVALID', path: 'restamp_command', message: 'must be a non-empty string or null' }); diff --git a/lib/ship-receipt.ts b/lib/ship-receipt.ts index 3f9ff6dcd..27f911951 100644 --- a/lib/ship-receipt.ts +++ b/lib/ship-receipt.ts @@ -192,9 +192,9 @@ const CARRY_RERUNS = ['CI at head (gstack-ship-queue ready requires it); test an /** * The file's text at the reviewed head and at HEAD, with the release version each head carries (from `VERSION`, else - * `package.json`) and the base policy's `stamp_patterns` templates. + * `package.json`) and the base policy's `stamp_patterns` templates and `stamp_placeholders`. */ -export interface CarryChange { before: string | null; after: string | null; oldVersion?: string | null; newVersion?: string | null; templates?: readonly string[]; side?: BranchSide } +export interface CarryChange { before: string | null; after: string | null; oldVersion?: string | null; newVersion?: string | null; templates?: readonly string[]; side?: BranchSide; placeholders?: readonly string[] } /** * A policy-named stamp path's change is a stamp when the new text reads as the @@ -217,7 +217,7 @@ export function stampOnlyText(change: CarryChange): boolean { const now = after.split('\n'); if (was.length !== now.length) return false; const changed = was.map((line, i) => i).filter(i => was[i] !== now[i]); - return changed.length > 0 && changed.every(i => stampOnlyLine(was[i]!, now[i]!, oldVersion, newVersion, change.templates ?? [], change.side, i + 1)); + return changed.length > 0 && changed.every(i => stampOnlyLine(was[i]!, now[i]!, oldVersion, newVersion, change.templates ?? [], change.side, i + 1, change.placeholders ?? [])); } /** diff --git a/lib/stamp-only.ts b/lib/stamp-only.ts index bbb87dc66..53b2b5231 100644 --- a/lib/stamp-only.ts +++ b/lib/stamp-only.ts @@ -4,7 +4,8 @@ * lines the branch wrote itself. Split from restamp.ts (module-size ratchet). */ import { spawnSync } from 'node:child_process'; -import { branchAddedLines, forkVersion, rewriteStamps, stampMentions } from './restamp'; +import { branchAddedLines, escapeRegExp, forkVersion, rewriteStamps, stampMentions } from './restamp'; +import { VERSION_SLOT } from './ship-policy'; import { cmpVersion, parseVersion, versionWidth } from './version-source'; /** @@ -26,12 +27,20 @@ import { cmpVersion, parseVersion, versionWidth } from './version-source'; * stage's bound for its own lines (GST-895, GST-963): finish's stampedFiles, * the tree receipt and the receipt carry all pass the same side. When the * version does not move (a slot that keeps its claimed VERSION), only such a - * branch line may change: the stage moves nothing else then. + * branch line may change: the stage moves nothing else then. On such a branch + * line a declared placeholder (`stamp_placeholders`, GST-1026: gbrain's + * `since: 'TBD'`) may also become the new version, but only inside a + * `stamp_patterns` template with literal text around its slot, never a built-in + * form: with at least one such rewrite, the line must then be the added line, + * or one whose remaining stamp-form moves pass the fork-version bound. */ -export function stampOnlyLine(removed: string, added: string, oldV: string, newV: string, templates: readonly string[] = [], side?: BranchSide, at?: number): boolean { +export function stampOnlyLine(removed: string, added: string, oldV: string, newV: string, templates: readonly string[] = [], side?: BranchSide, at?: number, placeholders: readonly string[] = []): boolean { if (removed === added) return false; if (oldV !== newV && stampOnlyWithin(removed, added, oldV, newV, templates)) return true; - return !!side && at !== undefined && side.at.has(at) && side.added.has(removed) && stampOnlyWithin(removed, added, side.fork, newV, templates); + if (!side || at === undefined || !side.at.has(at) || !side.added.has(removed)) return false; + if (stampOnlyWithin(removed, added, side.fork, newV, templates)) return true; + const filled = fillPlaceholders(removed, newV, templates, placeholders); + return filled.count > 0 && (filled.text === added || stampOnlyWithin(filled.text, added, side.fork, newV, templates)); } /** @@ -64,6 +73,21 @@ export function branchSides(repoRoot: string, baseRef: string, head: string, fal }; } +/** Each declared placeholder rewritten to `newV` inside each template with literal text around its slot (a bare edge at a word boundary). */ +function fillPlaceholders(removed: string, newV: string, templates: readonly string[], placeholders: readonly string[]): { text: string; count: number } { + let text = removed; + let count = 0; + for (const t of templates) { + const [pre, post] = t.split(VERSION_SLOT) as [string, string]; + if (!pre && !post) continue; + for (const p of placeholders) { + const re = new RegExp(`${escapeRegExp(pre)}${pre ? '' : '(? { count++; return pre + newV + post; }); + } + } + return { text, count }; +} + function stampOnlyWithin(removed: string, added: string, oldV: string, newV: string, templates: readonly string[]): boolean { const floor = parseVersion(oldV); const width = versionWidth(oldV); diff --git a/lib/tree-receipt.ts b/lib/tree-receipt.ts index f5d46b053..0611d6074 100644 --- a/lib/tree-receipt.ts +++ b/lib/tree-receipt.ts @@ -78,15 +78,16 @@ export function classifyFiles(repoRoot: string, fileDiffs: FileDiff[], gated: st /** * Every removed/added pair differs only by stamp rewrites to the new version: * the old version, or a tentative one (newer than it) the stamp_paths stage - * moved (#3512), each in a stamp form (`stampOnlyLine`, bounded by the fork version on the branch's own lines). + * moved (#3512), each in a stamp form (`stampOnlyLine`, bounded by the fork version on the branch's own lines, where a + * declared placeholder may also move inside a stamp_patterns template, GST-1026). */ -function stampOnlyHunks(hunks: FileDiff['hunks'], oldV: string, newV: string, templates: readonly string[], side?: BranchSide): { why: string; line: number } | null { +function stampOnlyHunks(hunks: FileDiff['hunks'], oldV: string, newV: string, templates: readonly string[], side?: BranchSide, placeholders: readonly string[] = []): { why: string; line: number } | null { for (const h of hunks) { if (h.removed.length !== h.added.length) return { why: `${h.removed.length} line(s) removed, ${h.added.length} added`, line: h.at }; for (let i = 0; i < h.removed.length; i++) { const r = h.removed[i]!; const a = h.added[i]!; - if (!stampOnlyLine(r, a, oldV, newV, templates, side, h.removedAt[i])) return { why: `${JSON.stringify(r.trim().slice(0, 60))} -> ${JSON.stringify(a.trim().slice(0, 60))}`, line: h.at + i }; + if (!stampOnlyLine(r, a, oldV, newV, templates, side, h.removedAt[i], placeholders)) return { why: `${JSON.stringify(r.trim().slice(0, 60))} -> ${JSON.stringify(a.trim().slice(0, 60))}`, line: h.at + i }; } } return null; @@ -130,7 +131,7 @@ export function classifyFile(repoRoot: string, f: FileDiff, gated: string, curre const rule = matchesAny(policy.stamp_paths, f.path) ? 'stamp_paths' : matchesAny(policy.release_outputs, f.path) ? 'release_outputs' : matchesAny(policy.stamp_files, f.path) ? 'stamp_files' : null; const own = rule ? side?.(f.path) : undefined; if (oldV && newV && (oldV !== newV || own) && rule) { - const bad = stampOnlyHunks(f.hunks, oldV, newV, policy.stamp_patterns, own); + const bad = stampOnlyHunks(f.hunks, oldV, newV, policy.stamp_patterns, own, policy.stamp_placeholders); return bad ? changed(rule, bad.why, bad.line) : stamp(rule); } return changed('other', `${f.hunks.length} hunk(s) outside every stamp rule`); diff --git a/package.json b/package.json index 0f033e81f..62b1079ca 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.94.15", + "version": "1.94.16", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/test/gstack-ship-policy.test.ts b/test/gstack-ship-policy.test.ts index 025b1bb57..e290256a8 100644 --- a/test/gstack-ship-policy.test.ts +++ b/test/gstack-ship-policy.test.ts @@ -8,7 +8,7 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { spawnSync } from 'node:child_process'; import { EXIT } from '../lib/headless-artifacts'; -import { BUMP_NEVER_KEY_VERSION, CHANGELOG_FRAGMENTS_KEY_VERSION, DEFAULT_POLICY, POLICY_KEYS, POLICY_KEY_INTRODUCED, REVIEW_SCOPE_KEY_VERSION, globToRegExp, matchesAny, minimumReader, shipConsequences, validatePolicy } from '../lib/ship-policy'; +import { BUMP_NEVER_KEY_VERSION, CHANGELOG_FRAGMENTS_KEY_VERSION, DEFAULT_POLICY, POLICY_KEYS, POLICY_KEY_INTRODUCED, REVIEW_SCOPE_KEY_VERSION, STAMP_PLACEHOLDERS_KEY_VERSION, globToRegExp, matchesAny, minimumReader, shipConsequences, validatePolicy } from '../lib/ship-policy'; import { fillDue } from '../lib/ship-queue-train'; import { cmpVersion, parseVersion } from '../lib/version-source'; import { BASE_POLICY, cleanup, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/restamp-fixture'; @@ -63,6 +63,32 @@ describe('validatePolicy (schema + containment, pure)', () => { expect(codes({ preregistration_shas: ['not-a-sha'] })).toEqual(['POLICY_CONTAINMENT:preregistration_shas[0]']); }); + // Value: protects=stamp_placeholders defaults to none, needs its reader floor, and admits only short one-line literals + // that are neither the {version} slot nor a version (GST-1026); fails_when=a non-default value validates without + // min_gstack_version, or a version, a slot, a multi-line or an oversized entry is accepted; seam=validatePolicy + test('stamp_placeholders: empty by default, needs its floor, and admits only short literal placeholders (GST-1026)', () => { + const codes = (p: unknown) => validatePolicy(p).map(e => `${e.code}:${e.path}`); + const at = (stamp_placeholders: unknown) => codes({ stamp_placeholders, min_gstack_version: STAMP_PLACEHOLDERS_KEY_VERSION }); + expect(DEFAULT_POLICY.stamp_placeholders).toEqual([]); + expect(POLICY_KEY_INTRODUCED.stamp_placeholders).toBe(STAMP_PLACEHOLDERS_KEY_VERSION); + expect(codes({ stamp_placeholders: [] })).toEqual([]); + expect(codes({ stamp_placeholders: ['TBD'] })).toEqual(['POLICY_INVALID:stamp_placeholders']); + expect(minimumReader({ stamp_placeholders: ['TBD'] })).toBe(STAMP_PLACEHOLDERS_KEY_VERSION); + expect(at(['TBD', 'UNRELEASED'])).toEqual([]); + expect(at('TBD')).toEqual(['POLICY_INVALID:stamp_placeholders']); + expect(at([7])).toEqual(['POLICY_INVALID:stamp_placeholders[0]']); + expect(at(Array.from({ length: 17 }, (_, i) => `P${i}`))).toEqual(['POLICY_CONTAINMENT:stamp_placeholders']); + expect(at(['x'.repeat(33)])).toEqual(['POLICY_CONTAINMENT:stamp_placeholders[0]']); + expect(at(['x'.repeat(32)])).toEqual([]); + expect(at(['TB\nD'])).toEqual(['POLICY_CONTAINMENT:stamp_placeholders[0]']); + expect(at([''])).toEqual(['POLICY_CONTAINMENT:stamp_placeholders[0]']); + expect(at([' '])).toEqual(['POLICY_CONTAINMENT:stamp_placeholders[0]']); + expect(at(['{version}'])).toEqual(['POLICY_INVALID:stamp_placeholders[0]']); + expect(at(['next-{version}'])).toEqual(['POLICY_INVALID:stamp_placeholders[0]']); + expect(at(['1.2.3.0'])).toEqual(['POLICY_INVALID:stamp_placeholders[0]']); + expect(at(['0.60.553'])).toEqual(['POLICY_INVALID:stamp_placeholders[0]']); + }); + // Value: protects=bump never is refused without its floor (an older gstack names the cause) and next to any policy that stamps a version at every merge, reports its reader floor, and prints its Step 12 consequence; fails_when=never validates bare, or beside stamp-at-merge, train or changelog_fragments; // why_new=GST-380 added the value to an older key, which the per-key floor check skips; seam=validatePolicy, minimumReader, shipConsequences; no_claim=Step 12's behavior (test/ship-release.test.ts) test('bump never needs its floor and rules out every policy that stamps at merge', () => { diff --git a/test/gstack-ship-receipt-docs.test.ts b/test/gstack-ship-receipt-docs.test.ts index db89a68fb..03213266c 100644 --- a/test/gstack-ship-receipt-docs.test.ts +++ b/test/gstack-ship-receipt-docs.test.ts @@ -53,6 +53,25 @@ describe('E18: a repository without a version source or CHANGELOG', () => { expect(JSON.parse(fs.readFileSync(path.join(out, 'ship-run.json'), 'utf8')).version).toBe('none'); }); + // Value: protects=a receipt write --version named in a repository with no version source keeps that version through a + // carry (GST-1030 review); fails_when=the carry refuses because HEAD has no readable version, or drops the version; + // seam=bin/gstack-ship-receipt cmdCarry + test('a carry keeps the version write --version named when HEAD has no version source (GST-1030)', () => { + const repo = fixture(); + for (const f of ['VERSION', 'package.json', 'package-lock.json']) fs.rmSync(path.join(repo.work, f)); + repo.git('add', '-A'); repo.git('commit', '-qm', 'changelog only'); repo.git('push', '-q', 'origin', 'main'); + makePr(repo, 'pr-unversioned'); + const body = path.join(repo.root, 'body.md'); + fs.writeFileSync(body, '## Summary\n'); + expect(runBin(repo, 'gstack-ship-receipt', ['write', '--pr', '3', '--base', 'main', '--session-kind', 'x', '--version', '2.0.0', '--body', body]).status).toBe(EXIT.ok); + fs.mkdirSync(path.join(repo.work, 'test'), { recursive: true }); + fs.writeFileSync(path.join(repo.work, 'test', 'a.test.ts'), 'test a\n'); + repo.git('add', '-A'); repo.git('commit', '-qm', 'a test-only change'); + const carried = runBin(repo, 'gstack-ship-receipt', ['carry', '--body', body, '--base', 'main']); + expect(carried.out, carried.err).toContain('CARRY: carried'); + expect(extractReceipt(fs.readFileSync(body, 'utf8')).receipt?.version).toBe('2.0.0'); + }); + test('a repository with a version source keeps its version, and one with only a CHANGELOG records no version rather than "none"', () => { const repo = fixture(); const w = runBin(repo, 'gstack-ship-receipt', ['write', '--base', 'main', '--session-kind', 'x', '--json']); @@ -248,6 +267,25 @@ describe('carry follow-ups: git floor, named refusal, branch-touched walk, digit expect(stampOnlyText({ before, after: lines(key('1.2.3.0'), key('1.2.4.0')), oldVersion: '1.2.4.0', newVersion: '1.2.4.0', templates })).toBe(false); }); + // Value: protects=the carry classifier reads a declared placeholder moved to the new version on the branch's own row + // as a stamp, through the same predicate as finish and the tree receipt (GST-1026); fails_when=stampOnlyText ignores + // the policy's placeholders, or accepts an undeclared one or a base row; seam=lib/ship-receipt.ts stampOnlyText + test('stampOnlyText reads a declared placeholder moved on the branch\'s own row as a stamp, and nothing else (GST-1026)', () => { + const templates = ["since: '{version}'"]; + const row = (v: string, id: string) => ` { since: '${v}', id: '${id}' },`; + const text = (...rows: string[]) => ['export const ROWS = [', ...rows, '];', ''].join('\n'); + const before = text(row('TBD', 'base'), row('TBD', 'mine')); + const side = { fork: '0.60.552.0', added: new Set([row('TBD', 'mine')]), at: new Set([3]) }; + const change = (after: string, extra: Record = {}) => stampOnlyText({ before, after, oldVersion: '0.60.552.0', newVersion: '0.60.553.0', templates, side, placeholders: ['TBD'], ...extra }); + expect(change(text(row('TBD', 'base'), row('0.60.553.0', 'mine')))).toBe(true); + expect(change(text(row('TBD', 'base'), row('0.60.553.0', 'mine')), { oldVersion: '0.60.553.0' })).toBe(true); + expect(change(text(row('TBD', 'base'), row('0.60.553.0', 'mine')), { placeholders: [] })).toBe(false); + expect(change(text(row('TBD', 'base'), row('0.60.553.0', 'mine')), { placeholders: ['WIP'] })).toBe(false); + expect(change(text(row('TBD', 'base'), row('0.60.553.0', 'mine')), { templates: [] })).toBe(false); + expect(change(text(row('TBD', 'base'), row('0.60.553.0', 'mine')), { side: undefined })).toBe(false); + expect(change(text(row('0.60.553.0', 'base'), row('TBD', 'mine')))).toBe(false); + }); + test('planCarry names at most CARRY_NAMED_PATHS paths', () => { const many = Array.from({ length: CARRY_NAMED_PATHS + 3 }, (_, i) => `src/f${i}.ts`); const plan = planCarry({ from: 'a'.repeat(40), head: 'b'.repeat(40), changed: many.map(p => ({ path: p, class: 'product' as const })), productPatchId: { from: 'x', head: 'y' }, testPatchId: { from: '', head: '' }, changedProduct: many, mergeCommits: 0, rangeDiff: '' }); diff --git a/test/restamp-mentions.test.ts b/test/restamp-mentions.test.ts index 68a546a9c..a74641aa0 100644 --- a/test/restamp-mentions.test.ts +++ b/test/restamp-mentions.test.ts @@ -14,6 +14,8 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { EXIT } from '../lib/headless-artifacts'; import { rewriteStamps, tentativeMentions } from '../lib/restamp'; +import { STAMP_PLACEHOLDERS_KEY_VERSION } from '../lib/ship-policy'; +import { extractReceipt } from '../lib/ship-receipt'; import type { GitFixtureTemplate } from './helpers/git-fixture-template'; import { BASE_POLICY, cleanup, cloneFixture, fixtureTemplate, makePr, mergeMain, mergeToMain, readFile, restamp, runBin, type FixtureRepo } from './helpers/restamp-fixture'; @@ -425,10 +427,165 @@ describe('a branch-added row left at the fork version after the base moved (GST- const { repo, body } = carryAfterStamp('restamp-carry-mode-', [row('1.2.3.0', 'base_row')], row('1.2.3.0', 'mine'), [row('1.2.3.0', 'base_row'), '', row('1.2.5.0', 'mine')]); fs.chmodSync(path.join(repo.work, NOTICE), 0o755); - repo.git('commit', '-qam', 'stamp 1.2.5.0, and make the file executable'); + repo.git('add', '-A'); + repo.git('update-index', '--chmod=+x', '--', NOTICE); + repo.git('commit', '-qm', 'stamp 1.2.5.0, and make the file executable'); expect(repo.git('ls-tree', 'HEAD', '--', NOTICE)).toStartWith('100755 '); const carried = runBin(repo, 'gstack-ship-receipt', ['carry', '--body', body, '--base', 'main']); expect(carried.status, carried.out).toBe(EXIT.refused); expect(carried.err).toContain('RECEIPT_CARRY_PRODUCT_CHANGED'); }); }); + +describe('declared placeholders and the receipt carry (GST-1026, GST-1030)', () => { + /** gbrain's policy (GST-1026): the since: template, and TBD declared as a placeholder when `declared`. */ + const placeholderPolicy = (declared: boolean) => ({ ...receiptPolicy(true), ...(declared ? { stamp_placeholders: ['TBD'], min_gstack_version: STAMP_PLACEHOLDERS_KEY_VERSION } : {}) }); + + /** A branch row written as since: 'TBD' at the claim 1.2.4.0, then the stamp to 1.2.5.0 with the row restamped by hand (gbrain's release:restamp). */ + function tbdStamp(prefix: string, pol: Record, ownRow: string): { repo: FixtureRepo; gated: string } { + const repo = makeFixtureRepo({ prefix, policy: pol }); repos.push(repo); + write(repo, NOTICE, notice([row('TBD', 'base_row')])); + repo.git('add', '-A'); repo.git('commit', '-qm', 'notice rows'); repo.git('push', '-q', 'origin', 'main'); + makePr(repo, 'pr-tbd'); + expect(restamp(repo, ['--version', '1.2.4.0']).status).toBe(EXIT.ok); + repo.git('commit', '-qam', 'stamp 1.2.4.0'); + write(repo, NOTICE, notice([row('TBD', 'base_row'), '', ownRow])); + repo.git('commit', '-qam', 'mine: a TBD row'); + const gated = repo.git('rev-parse', 'HEAD'); + expect(restamp(repo, ['--version', '1.2.5.0']).status).toBe(EXIT.ok); + return { repo, gated }; + } + + // Value: protects=the tree receipt (stamp, verify, gate-ahead replay) accepts gbrain's restamp of the branch's own + // since: 'TBD' row when the policy declares TBD, and refuses it undeclared, on a base row, or on a base row whose text + // the branch duplicates (GST-1026); fails_when=classifyFile reads the TBD move as a hand edit, or accepts it off the + // branch's own rows; seam=lib/tree-receipt.ts stampOnlyHunks + test('the tree receipt accepts a declared TBD row restamped on the branch, and refuses the same move on a base row (GST-1026)', () => { + const own = tbdStamp('restamp-tbd-receipt-', placeholderPolicy(true), row('TBD', 'mine')); + write(own.repo, NOTICE, readFile(own.repo, NOTICE).replace(row('TBD', 'mine'), row('1.2.5.0', 'mine'))); + own.repo.git('commit', '-qam', 'stamp 1.2.5.0: the TBD row'); + const accepted = runBin(own.repo, 'gstack-tree-receipt', ['--gated', own.gated, '--base', 'main']); + expect(accepted.out, accepted.err).toContain('TREE: same-modulo-stamps'); + write(own.repo, NOTICE, readFile(own.repo, NOTICE).replace(row('TBD', 'base_row'), row('1.2.5.0', 'base_row'))); + own.repo.git('commit', '-qam', 'move the base TBD row by hand'); + const base = runBin(own.repo, 'gstack-tree-receipt', ['--gated', own.gated, '--base', 'main']); + expect(base.out, base.err).toContain('TREE: changed'); + expect(base.out).toContain("id: 'base_row'"); + }); + + test('the tree receipt refuses an undeclared TBD row restamped on the branch (GST-1026)', () => { + const undeclared = tbdStamp('restamp-tbd-undeclared-', placeholderPolicy(false), row('TBD', 'mine')); + write(undeclared.repo, NOTICE, readFile(undeclared.repo, NOTICE).replace(row('TBD', 'mine'), row('1.2.5.0', 'mine'))); + undeclared.repo.git('commit', '-qam', 'stamp 1.2.5.0: the TBD row'); + const refused = runBin(undeclared.repo, 'gstack-tree-receipt', ['--gated', undeclared.gated, '--base', 'main']); + expect(refused.out, refused.err).toContain('TREE: changed'); + }); + + test('the tree receipt refuses a restamp of the base copy of a TBD row the branch duplicates (GST-1026)', () => { + const dup = tbdStamp('restamp-tbd-dup-', placeholderPolicy(true), row('TBD', 'base_row')); + write(dup.repo, NOTICE, notice([row('1.2.5.0', 'base_row'), '', row('TBD', 'base_row')])); + dup.repo.git('commit', '-qam', 'move the base copy of the duplicated TBD row'); + const copy = runBin(dup.repo, 'gstack-tree-receipt', ['--gated', dup.gated, '--base', 'main']); + expect(copy.out, copy.err).toContain('TREE: changed'); + }); + + /** A receipt written at the claim 1.2.4.0 after `own` changed the branch, then the stamp to 1.2.5.0, uncommitted. */ + function receiptThenStamp(prefix: string, pol: Record, own: (repo: FixtureRepo) => void): { repo: FixtureRepo; body: string } { + const repo = makeFixtureRepo({ prefix, policy: pol }); repos.push(repo); + write(repo, NOTICE, notice([row('TBD', 'base_row')])); + fs.rmSync(path.join(repo.work, 'package-lock.json')); + repo.git('add', '-A'); repo.git('commit', '-qm', 'notice rows, no lockfile'); repo.git('push', '-q', 'origin', 'main'); + makePr(repo, 'pr-carry'); + expect(restamp(repo, ['--version', '1.2.4.0']).status).toBe(EXIT.ok); + repo.git('commit', '-qam', 'stamp 1.2.4.0'); + own(repo); + repo.git('add', '-A'); repo.git('commit', '-qm', 'mine'); + const body = path.join(repo.root, 'body.md'); + fs.writeFileSync(body, '## Summary\n'); + const w = runBin(repo, 'gstack-ship-receipt', ['write', '--pr', '3', '--base', 'main', '--session-kind', 'x', '--body', body]); + expect(w.status, w.err).toBe(EXIT.ok); + expect(restamp(repo, ['--version', '1.2.5.0']).status).toBe(EXIT.ok); + return { repo, body }; + } + const carry = (repo: FixtureRepo, body: string) => runBin(repo, 'gstack-ship-receipt', ['carry', '--body', body, '--base', 'main']); + + // Value: protects=the receipt carry follows gbrain's restamp of the branch's own since: 'TBD' row when the policy + // declares TBD, and refuses it undeclared (GST-1026); fails_when=productSpec's own-row check ignores the policy's + // placeholders and refuses RECEIPT_CARRY_PRODUCT_CHANGED; seam=bin/gstack-ship-receipt productSpec ownRows + test.each([['declared', true], ['undeclared', false]])('the carry and a TBD row restamped on the branch: %s', (_, declared) => { + const { repo, body } = receiptThenStamp(`restamp-tbd-carry-${declared}-`, placeholderPolicy(declared), r => write(r, NOTICE, notice([row('TBD', 'base_row'), '', row('TBD', 'mine')]))); + write(repo, NOTICE, readFile(repo, NOTICE).replace(row('TBD', 'mine'), row('1.2.5.0', 'mine'))); + repo.git('commit', '-qam', 'stamp 1.2.5.0: the TBD row'); + const carried = carry(repo, body); + if (declared) expect(carried.out, carried.err).toContain('CARRY: carried'); + else expect([carried.status, carried.err.includes('RECEIPT_CARRY_PRODUCT_CHANGED')], carried.err).toEqual([EXIT.refused, true]); + }); + + // Value: protects=a carried receipt names the head's version, the one write would record there, after a main merge + // and a restamp (GST-1030 point 1, GGST-1); fails_when=carryReceipt copies version from the old receipt; + // seam=bin/gstack-ship-receipt cmdCarry + test('the carried receipt records the head\'s version after a main merge and a restamp (GST-1030)', () => { + const { repo, body } = receiptThenStamp('restamp-carry-version-', policy(true), r => write(r, 'src.ts', 'export const x = 0;\n')); + repo.git('reset', '-q', '--hard'); + expect(readFile(repo, 'VERSION')).toBe('1.2.4.0\n'); + makePr(repo, 'pr-other'); + expect(restamp(repo, ['--next']).status).toBe(EXIT.ok); + repo.git('commit', '-qam', 'stamp other'); + mergeToMain(repo, 'pr-other'); + repo.git('checkout', '-q', 'pr-carry'); + mergeMain(repo); + expect(restamp(repo, ['--next']).status).toBe(EXIT.ok); + repo.git('commit', '-qam', 'restamp after the lost slot'); + expect(readFile(repo, 'VERSION')).toBe('1.2.5.0\n'); + expect(extractReceipt(fs.readFileSync(body, 'utf8')).receipt?.version).toBe('1.2.4.0'); + const carried = carry(repo, body); + expect(carried.out, carried.err).toContain('CARRY: carried'); + expect(extractReceipt(fs.readFileSync(body, 'utf8')).receipt?.version).toBe('1.2.5.0'); + }); + +/** package.json with its dependency pin set to `pin`, every other field kept. */ + const pinned = (repo: FixtureRepo, pin: string) => { + const pkg = JSON.parse(readFile(repo, 'package.json')); + pkg.dependencies = { ...pkg.dependencies, 'left-pad': pin }; + write(repo, 'package.json', JSON.stringify(pkg, null, 2) + '\n'); + }; + + // Value: protects=a pin-bump PR's receipt carries across a main merge and a restamp when package.json's only change since + // review is its version field, and refuses when the pin itself moved (GST-1030, GBEV-3, gbrain-evals #306); + // fails_when=the carry counts the PR's package.json as product and refuses RECEIPT_CARRY_PRODUCT_CHANGED on a version-only + // move, or lets a changed dependency pin ride; seam=bin/gstack-ship-receipt cmdCarry walk + test.each([['the pin unchanged since review', false], ['the pin moved since review', true]])('the carry and a pin-bump PR after a main merge: %s (GST-1030)', (_, moved) => { + const { repo, body } = receiptThenStamp(`restamp-carry-pin-${moved}-`, policy(true), r => pinned(r, '1.3.0')); + repo.git('reset', '-q', '--hard'); + makePr(repo, 'pr-other'); + expect(restamp(repo, ['--next']).status).toBe(EXIT.ok); + repo.git('commit', '-qam', 'stamp other'); + mergeToMain(repo, 'pr-other'); + repo.git('checkout', '-q', 'pr-carry'); + mergeMain(repo); + expect(restamp(repo, ['--next']).status).toBe(EXIT.ok); + if (moved) pinned(repo, '1.3.1'); + repo.git('commit', '-qam', 'restamp after the lost slot'); + expect(JSON.parse(readFile(repo, 'package.json')).version).toBe('1.2.5'); + const carried = carry(repo, body); + if (moved) expect([carried.status, carried.err.includes('RECEIPT_CARRY_PRODUCT_CHANGED')], carried.err).toEqual([EXIT.refused, true]); + else expect(carried.out, carried.err).toContain('CARRY: carried'); + }); + + // Value: protects=a version-only restamp of EVIDENCE.md is release metadata to the carry when the policy declares it in + // stamp_files, and product when it does not (GST-1030 point 2, GGST-1); fails_when=the carry counts the declared + // file's stamp as a product change (RECEIPT_CARRY_PRODUCT_CHANGED), or lets an undeclared file's version edit ride; + // seam=bin/gstack-ship-receipt productSpec, lib/ship-receipt.ts stampOnlyText + test.each([['declared in stamp_files', true], ['undeclared', false]])('the carry and a version-only restamp of EVIDENCE.md: %s (GST-1030)', (_, declared) => { + const base = policy(true); + const pol = declared ? { ...base, stamp_files: [...base.stamp_files, 'EVIDENCE.md'] } : base; + const { repo, body } = receiptThenStamp(`restamp-carry-evidence-${declared}-`, pol, r => write(r, 'EVIDENCE.md', '# Evidence\n\nVerified on v1.2.4.0 with the free suite\n')); + if (!declared) write(repo, 'EVIDENCE.md', '# Evidence\n\nVerified on v1.2.5.0 with the free suite\n'); + expect(readFile(repo, 'EVIDENCE.md')).toBe('# Evidence\n\nVerified on v1.2.5.0 with the free suite\n'); + repo.git('commit', '-qam', 'stamp 1.2.5.0'); + const carried = carry(repo, body); + if (declared) expect(carried.out, carried.err).toContain('CARRY: carried'); + else expect([carried.status, carried.err.includes('RECEIPT_CARRY_PRODUCT_CHANGED')], carried.err).toEqual([EXIT.refused, true]); + }); +}); + diff --git a/test/ship-rereview-triggers.test.ts b/test/ship-rereview-triggers.test.ts index f83c6755e..3854c7808 100644 --- a/test/ship-rereview-triggers.test.ts +++ b/test/ship-rereview-triggers.test.ts @@ -9,6 +9,7 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { EXIT } from '../lib/headless-artifacts'; import { comparePath } from '../lib/ship-patch'; +import { STAMP_PLACEHOLDERS_KEY_VERSION } from '../lib/ship-policy'; import { extractReceipt, placeReceiptFirst } from '../lib/ship-receipt'; import { cleanup, makeFixtureRepo, runBin, write, type FixtureRepo } from './helpers/pregate-fixture'; import { lineOf, PREFLIGHT, RELEASE, shipFixture, type ShipFx } from './helpers/ship-bin-fixture'; @@ -266,6 +267,37 @@ describe('a restamp after a lost slot (GST-895, GBRA-218, gbrain #8134)', () => const base = kept([['0.9.0.0', 'old'], ['1.0.1.0', 'base'], ['1.0.0.0', 'mine']]); expect([base.status, lineOf(base.stdout, 'STOP')]).toEqual([3, behaviorStop]); }); + + // Value: protects=gbrain's restamp of the PR's own `since: 'TBD'` row to the new version routes as metadata when the + // policy declares TBD in stamp_placeholders (GST-1026, gbrain #8444); an undeclared placeholder, a base row moved the + // same way, a duplicated base copy, or a line no stamp_pattern matches still stops; fails_when=stampedFiles reads the + // move as behavior (gbrain #8444) or accepts it off the branch's own rows; seam=lib/ship-bin-common.ts stampedFiles + test('a declared placeholder on the branch row is metadata; undeclared, a base row or no template stops (GST-1026)', () => { + const declared = (placeholders: string[] | undefined, templates = ["since: '{version}'"]) => JSON.stringify({ stamp_files: ['VERSION', 'src/notice.ts'], stamp_patterns: templates, + ...(placeholders ? { stamp_placeholders: placeholders, min_gstack_version: STAMP_PLACEHOLDERS_KEY_VERSION } : {}) }); + const restamped = (pol: string, baseRows: [string, string][], ownRows: [string, string][], rows: [string, string][]) => { + const f = fx({ '.gstack/ship-policy.json': pol, VERSION: '1.0.0.0\n', 'src/notice.ts': notice(...baseRows) }, + { VERSION: '1.0.1.0\n', 'src/notice.ts': notice(...baseRows, ...ownRows) }); + const reviewed = wtree(f); + const post = snapshot(f); + f.write({ VERSION: '1.0.2.0\n', 'src/notice.ts': notice(...rows) }); f.commit('restamp: TBD rows to 1.0.2.0'); + return finish(f, reviewed, post, ['--reviewed-base', mergeBase(f)]); + }; + const base: [string, string][] = [['0.9.0.0', 'old'], ['TBD', 'base']]; + const metadata = / behavior=0 docs=0 metadata=2 tests=0 /; + const own = restamped(declared(['TBD']), base, [['TBD', 'mine']], [...base, ['1.0.2.0', 'mine']]); + expect(lineOf(own.stdout, 'ROUTE'), own.stdout).toEqual(expect.stringMatching(metadata)); + const undeclared = restamped(declared(undefined), base, [['TBD', 'mine']], [...base, ['1.0.2.0', 'mine']]); + expect([undeclared.status, lineOf(undeclared.stdout, 'STOP')]).toEqual([3, behaviorStop]); + const other = restamped(declared(['WIP']), base, [['TBD', 'mine']], [...base, ['1.0.2.0', 'mine']]); + expect([other.status, lineOf(other.stdout, 'STOP')]).toEqual([3, behaviorStop]); + const baseRow = restamped(declared(['TBD']), base, [['TBD', 'mine']], [['0.9.0.0', 'old'], ['1.0.2.0', 'base'], ['1.0.2.0', 'mine']]); + expect([baseRow.status, lineOf(baseRow.stdout, 'STOP')]).toEqual([3, behaviorStop]); + const dup = restamped(declared(['TBD']), base, [['TBD', 'base']], [['0.9.0.0', 'old'], ['1.0.2.0', 'base'], ['TBD', 'base']]); + expect([dup.status, lineOf(dup.stdout, 'STOP')]).toEqual([3, behaviorStop]); + const noTemplate = restamped(declared(['TBD'], ['version: {version}']), base, [['TBD', 'mine']], [...base, ['1.0.2.0', 'mine']]); + expect([noTemplate.status, lineOf(noTemplate.stdout, 'STOP')]).toEqual([3, behaviorStop]); + }); }); describe('adjacent base edits (GST-384, #3631, founderstar#441)', () => { diff --git a/test/tree-receipt-stamp-sensitive.test.ts b/test/tree-receipt-stamp-sensitive.test.ts index e9c56e660..033f4ea5b 100644 --- a/test/tree-receipt-stamp-sensitive.test.ts +++ b/test/tree-receipt-stamp-sensitive.test.ts @@ -121,6 +121,36 @@ describe('tree receipt: stamps, stamp-sensitive gates and negative fixtures', () expect(stampOnlyLine("mine since: '1.2.3.0'", "mine since: '1.2.4.0'", '1.2.4.0', '1.2.4.0', tpl, mine, 5)).toBe(true); }); + // Value: protects=a declared placeholder on the branch's own row moves to the new version as a stamp, only inside a + // stamp_patterns template (gbrain's `since: 'TBD'` row, GST-1026); fails_when=stampOnlyLine reads the move as a hand + // edit, or accepts an undeclared placeholder, a base row, a duplicated base line, a built-in form or a line no + // template matches; seam=lib/stamp-only.ts stampOnlyLine placeholders + test('a declared placeholder on a branch row moves to the new version inside a stamp_patterns template, and nowhere else (GST-1026)', () => { + const tpl = ["since: '{version}'"]; + const row = " { since: 'TBD', id: 'mine' },"; + const moved = " { since: '0.60.553.0', id: 'mine' },"; + const side = (r: string, at = 10) => ({ fork: '0.60.552.0', added: new Set([r]), at: new Set([at]) }); + const line = (r: string, a: string, opts: { templates?: string[]; at?: number; placeholders?: string[]; oldV?: string } = {}) => + stampOnlyLine(r, a, opts.oldV ?? '0.60.552.0', '0.60.553.0', opts.templates ?? tpl, side(r), opts.at ?? 10, opts.placeholders ?? ['TBD']); + expect(line(row, moved)).toBe(true); + expect(line(row, moved, { oldV: '0.60.553.0' })).toBe(true); + expect(line(row, moved, { placeholders: [] })).toBe(false); + expect(line(row, moved, { placeholders: ['WIP'] })).toBe(false); + expect(line(" { since: 'WIP', id: 'mine' },", moved)).toBe(false); + expect(line(row, moved, { at: 2 })).toBe(false); + expect(stampOnlyLine(row, moved, '0.60.552.0', '0.60.553.0', tpl, undefined, undefined, ['TBD'])).toBe(false); + expect(stampOnlyLine(row, moved, '0.60.552.0', '0.60.553.0', tpl, side(" { since: 'TBD', id: 'other' },"), 10, ['TBD'])).toBe(false); + expect(line(row, moved, { templates: [] })).toBe(false); + expect(line(" { release: 'TBD', id: 'mine' },", " { release: '0.60.553.0', id: 'mine' },")).toBe(false); + expect(line(' { since: TBD, id: mine },', ' { since: 0.60.553.0, id: mine },')).toBe(false); + expect(line(row, " { since: '0.60.553.0', id: 'yours' },")).toBe(false); + expect(line(row, " { since: '0.60.554.0', id: 'mine' },")).toBe(false); + const both = " { since: 'TBD', note: 'Added in v0.60.552.0' },"; + expect(line(both, " { since: '0.60.553.0', note: 'Added in v0.60.553.0' },")).toBe(true); + expect(line(both, " { since: '0.60.553.0', note: 'Added in v0.60.551.0' },")).toBe(false); + expect(line(' // TBD: tidy', ' // 0.60.553.0: tidy', { templates: ['{version}'] })).toBe(false); + }); + test('negative fixture: a dependency pin equal to the old version is `changed`, not a stamp, even inside stamp_paths', () => { const repo = fixture(); const g = gated(repo); From d2e63181d7df5870370ef68f161a8e2bd294b166 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Tue, 3 Nov 2026 06:14:50 -0800 Subject: [PATCH 2/4] v1.94.17.0 ci-carry: guard carry, jobs-mapped lanes, aliases, per-file carry of runner-skipped files, path-skipped jobs (GST-1186, GST-1189, GST-1190) (#3929) * feat(pregate): a guard that already passed at this tree carries from CI or the ledger (GST-1186) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(ci-carry): jobs-mapped lanes, aliases, narrow-lane counts, --simulate proof, policy-base check on carried (GST-1189, GST-1190) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(ci-carry): a file whose every test skipped on CI is not carried; tree.json records skipped_files (GST-1190) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * feat(ci-carry): path-skipped jobs count only under an unchanged gate; find --policy-from head --simulate (GST-1190) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship: name the pre-push guards in Step 17 and the early push (moved from #3923 per GSTA-258) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci-carry: per-file lane carry for runner-skipped files; review fixes (gate inputs, passed ignores CI rows, simulate consent, pull_request-only skips, guard not-owed) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ship: Step 17 names the pre-push guards without a code this branch cannot print Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * pregate: gstack's bun run test lane runs its runner-skipped files with per_file Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci-carry: a per_file remainder must run tests here and the carried runs must still stand (l6c-p2) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * ci-carry: read record-ci from its own last record line; the remainder's count must come from this run's row (l6c-p2 red-team) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(ci-tree-reuse): type the skipped_files fixture for the test typecheck ratchet Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * Revert "pregate: gstack's bun run test lane runs its runner-skipped files with per_file" This reverts commit fafbae5b9bae1a07065bd0706bc6bf5f41a6ef09. * ci-carry: ship the skipped_files reader inert; gstack's writer and per_file move to the adoption PR; Step 17 keeps its refs/heads marker Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs: guard carry, per_file and the too-old message in pregate and troubleshooting (docs audit l6cdocs2) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * gstack-evidence record-ci: recheck the working tree after the post-remainder run recheck (#3929 second fix-check) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * windows-curation: exclude test/ship-ci-carry.test.ts like its ship-bin-fixture siblings (the path-scoped Windows check runs it at its first spawn) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 27 + VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- bin/gstack-ci-carry | 29 +- bin/gstack-doctor-components.sh | 1 + bin/gstack-evidence | 113 ++++- bin/gstack-ship-receipt | 8 +- docs/pregate.md | 191 +++++++- docs/troubleshooting.md | 17 + lib/ci-carry.ts | 111 ++++- lib/pregate/checks/guards.ts | 41 +- lib/pregate/checks/lanes.ts | 7 +- lib/pregate/config.ts | 37 +- lib/pregate/index.ts | 2 +- lib/pregate/run-check.ts | 51 +- lib/result-codes-release.ts | 5 + lib/run-evidence.ts | 57 ++- lib/ship-bin-common.ts | 22 +- lib/ship-preflight.ts | 44 ++ package.json | 2 +- scripts/lib/windows-curation.ts | 4 + ship/SKILL.md | 2 +- ship/SKILL.md.tmpl | 2 +- ship/sections/tests.md | 2 +- ship/sections/tests.md.tmpl | 2 +- test/ci-carry.test.ts | 6 +- test/evidence-reuse.test.ts | 5 +- test/fixtures/golden/agy-ship-SKILL.md | 2 +- test/fixtures/golden/claude-ship-SKILL.md | 2 +- test/fixtures/golden/codex-ship-SKILL.md | 2 +- test/fixtures/golden/factory-ship-SKILL.md | 2 +- test/helpers/fake-gh-runs.ts | 9 +- test/run-evidence.test.ts | 3 +- test/ship-ci-carry.test.ts | 462 +++++++++++++++++- .../ship/gsta258-l6-guard-carry.json | 4 + 35 files changed, 1144 insertions(+), 134 deletions(-) create mode 100644 test/size-ledger/ship/gsta258-l6-guard-carry.json diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ee6eb368..17e5ed851 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,32 @@ # Changelog +## [1.94.17.0] - 2026-11-03 + +- A pregate guard whose exact command already passed at this tree no longer runs again (GST-1186). When the base branch's `ship_lanes_from` maps the command, it is carried from the verified CI run through the same `gstack-evidence record-ci` path /ship lanes use (`GUARD: carried runs=…`); otherwise a counted local ledger pass of that command at the same working tree stands (CI rows count only through a live `record-ci` check) (`GUARD: reused …`, and the new read-only `gstack-evidence passed -- `). Repos whose suites are guards, like founderstar, stop rerunning about six minutes of suites per /ship. See docs/pregate.md#guard-carry. +- Known limit: `ship_lanes_from` is still read from `origin/` only, so the PR that adds a mapping cannot carry from it. +- A `ship_lanes_from` entry can name the CI jobs that run a lane instead of an inventory command: `{ "profiles": [...], "jobs": ["verify", "typecheck"] }`, exactly one of the two (GST-1189). It carries only when every named job succeeded in the newest attempt of the verified run at HEAD's tree, through the same `verifyRun` job check. A missing job refuses as `job_missing` and a skipped one as the new `job_skipped`. Lanes with no test files, like gbrain's verify and typecheck (run 41564913688) or harness-evals' `bunx tsc --noEmit`, can now carry, and they record `tests_ran=n/a`. Fake `check:` entries in `tree.json` can move to this form. A repo declaring `jobs` should raise its `min_gstack_version` to this release first, since older gstack rejects the key. See docs/pregate.md#ship-lanes-jobs. + +- A `ship_lanes_from` entry can declare `aliases`, other exact spellings of the same lane such as `env -u ANTHROPIC_API_KEY bun test tests/` (GST-1190). gstack never strips env prefixes on its own, since an `env -i` run is a different environment. +- A carried lane no longer reports the whole run's test count. An inventory lane reports how many of its files the run covered, labeled as a file count (alphaclaw's 46-test guards lane recorded 9677 before). +- `gstack-ship-preflight --tests-only --policy-from head --simulate` proves a new mapping before it lands: every line starts `SIMULATION:`, no lane runs, no evidence row is written and nothing clears. An inventory-mapped lane runs the branch's inventory command, so the dry run needs `--allow-repo-commands`. A branch's own map never carries, so the first pull request that adopts a mapping pays its full lanes. +- `gstack-evidence carried`, which the ship receipt reads, lists only rows the receipt base's policy mapped, so a run against another base never lands in a receipt. +- An unknown `.gstack/pregate.json` key under a `min_gstack_version` above the running gstack now says `needs gstack >= ; upgrade` (`PREGATE_GSTACK_TOO_OLD`). It still refuses. +- docs/pregate.md#prove-a-mapping documents the proof route, the version floors, and that `tree.json` writers using Bun's JUnit must take the root `testsuite` count. + +- A test file whose every test skipped on the CI runner no longer counts as carried (GST-1190, harness-evals: 75 tests). A `tree.json` that lists such files in `skipped_files` makes the pregate tests stage run them here. A /ship lane or guard whose inventory names one carries the covered files and runs only the skipped ones here, through the `per_file` runner its `ship_lanes_from` entry declares. The CI row is written only after that run passes with a test count above zero and the carried runs still stand (`skipped_failed`, `skipped_zero_run` or `run_changed` otherwise), and an entry without `per_file` refuses as `inventory_skipped`, naming the files. A partly skipped file still counts. An older `tree.json` without the list is read as before. gstack itself changes nothing yet: its own writer and a `per_file` entry for its `bun run test` lane switch on together in a follow-up pull request, after the pull requests now queued on older gstack have landed, because an older gstack rejects the new key in main's policy. + +- A run profile can declare `skippable: { jobs, gate, scripts }` for jobs a path filter skips (GST-1190, truther #1169). A skipped job counts only in a `pull_request` run whose gate job succeeded, with the workflow, the declared `scripts` (every file the gate reads), the paths the gate job names (also under its `working-directory`) and `package.json` when it runs a package script matching `origin/` byte for byte; the lane then prints `not owed: CI skipped at this tree (gate job , workflow unchanged)`. If the branch changed any of them, it controlled the skip, so the lane runs here. +- `gstack-ci-carry find --policy-from head --simulate` is the same dry run for one lane command, every line starting `SIMULATION:`. + +### Tests +- `test/ship-ci-carry.test.ts`: in founderstar's shape (guards only, no lanes), `check-quick` carries from CI, `play-through` reuses a same-tree ledger pass, `assets` runs, and a dirty tree carries nothing. It fails without the guard carry. +- `test/ship-ci-carry.test.ts`: a lane mapped by `jobs` carries when every named job passed at the same tree with no test files, and refuses for a missing job, a skipped job, a rerun whose newest attempt skipped the job, and another tree's run. Those five fail without the change. +- `test/ship-ci-carry.test.ts` (GST-1190): an alias carries and an undeclared env prefix does not; a narrow lane reports its covered file count, not 9677; a jobs-mapped `bunx tsc --noEmit` guard carries with `n/a`; the newer-gstack message; `--simulate` writes and clears nothing and the branch's own map never carries; `carried` drops a row another base mapped. Six of these fail without the change. +- `test/ship-ci-carry.test.ts` (GST-1190 skipped files): a fully skipped inventory file stops the lane carry, an empty list or older `tree.json` carries, and the tests stage runs only the skipped selected file. Both fail without the change. +- `test/ship-ci-carry.test.ts` (path-skipped jobs and `find --simulate`): a skip under an unchanged, green gate is not owed; a changed workflow, a changed gate script or a failed gate job runs the lane here; `find --policy-from head --simulate` prints only SIMULATION lines and writes nothing. All five fail without the change. +- `test/ship-ci-carry.test.ts` (review fixes): a gate script behind `${{ github.workspace }}`, under `working-directory` or behind a package script is compared; the gate is the job, not a same-named step; only a `pull_request` run's skip counts; a skippable profile must declare `scripts`; `passed` never counts a CI row; a path-skipped guard says not owed; `--simulate` runs a branch inventory only with `--allow-repo-commands`. Eight fail without the fixes; a newer-failure veto test covers existing behavior. +- `test/ship-ci-carry.test.ts` (per-file lane carry): the covered file carries and only the skipped file runs through `per_file`, a failing or zero-test remainder, or a run that moved to a new attempt meanwhile, records no CI row and runs the lane whole, and `per_file` needs `{files}` and an inventory entry. All five fail without the change. + ## [1.94.16.0] - 2026-11-03 **A declared placeholder restamped to the new version is a stamp, and a carried receipt names the head's version (GST-1026, GST-1030).** gbrain writes a PR's own BEHAVIOR_CHANGES row as `since: 'TBD'` and its `release:restamp` fills in the version. Finish refused that receipt ("behavior, tests or build inputs changed since review"), and the receipt carry and the tree receipt would have refused the same row (gbrain #8444). Separately, a receipt carried after a main merge and a restamp kept the version of the head it was written at (GGST-1). diff --git a/VERSION b/VERSION index e1438391e..54c4aa075 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.94.16.0 +1.94.17.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index 18e81912e..be1ec0e3d 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.94.16.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.94.17.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-ci-carry b/bin/gstack-ci-carry index c29156107..c5292f5eb 100755 --- a/bin/gstack-ci-carry +++ b/bin/gstack-ci-carry @@ -3,6 +3,7 @@ * gstack-ci-carry — CI as the one full test run (GSTA-258; docs/pregate.md#ci-carry). * * gstack-ci-carry find [--base ] [--profile ]... [--command --allow-repo-commands] [--json] + * [--policy-from head --simulate] * gstack-ci-carry wait [--base ] [--profile ] [--timeout ] [--interval ] * * find: the newest verified run of each profile (default: every profile the base branch's @@ -14,10 +15,9 @@ * carried, 3 still running or no run. Logic: lib/ci-carry.ts; the run checks are lib/run-evidence.ts. */ import { spawnSync } from 'node:child_process'; -import { findCarry, waitForRun } from '../lib/ci-carry'; -import { isConfigUnavailable, loadConfigFromBase } from '../lib/pregate/config'; +import { findCarry, laneMapping, mappingCoverage, waitForRun } from '../lib/ci-carry'; +import { isConfigUnavailable, loadConfigFromBase, loadConfigFromHead } from '../lib/pregate/config'; import { buildRunCheck } from '../lib/pregate/run-check'; -import { runInventory } from '../lib/run-evidence'; import { wtreeFingerprint } from '../lib/script-invocation'; import { resolveBaseBranch } from '../lib/ship-policy'; @@ -25,11 +25,13 @@ const HELP = `gstack-ci-carry — use a verified CI run at HEAD's exact tree ins find [--base ] [--profile ]... [--command --allow-repo-commands] [--json] CI_CARRY: accepted (exit 0) or CI_CARRY: none () (exit 1) + --policy-from head --simulate: the same check under the branch's own map and profiles, every line starting + SIMULATION:; a dry run that proves a new mapping before it lands (find never writes evidence; nothing clears) wait [--base ] [--profile ] [--timeout ] [--interval ] poll the newest run at HEAD (default 1800 s, every 20 s): CI_WAIT: green|red|running|none, then find on green; exit 0 carried, 1 red or not carried, 3 still running or no run -Profiles and ship_lanes_from come from origin/'s .gstack/pregate.json only. Only a clean, committed tree +Profiles and ship_lanes_from come from origin/'s .gstack/pregate.json only (a simulation reads the branch's). Only a clean, committed tree carries; fork runs, stale runs, a newer red or running run on HEAD and a tree.json that misses an inventory file never do. Exit 2 is usage.`; @@ -37,30 +39,35 @@ const argv = process.argv.slice(2); const cmd = argv[0]; const all = (n: string) => argv.flatMap((a, i) => (a === n && argv[i + 1] !== undefined ? [argv[i + 1]!] : [])); const flag = (n: string) => all(n).at(-1); -const out = (s: string) => process.stdout.write(`${s}\n`); +const simulate = argv.includes('--simulate'); +const out = (s: string) => process.stdout.write(`${simulate ? 'SIMULATION: ' : ''}${s}\n`); const usage = (m: string) => { process.stderr.write(`gstack-ci-carry: ${m}\n`); process.exit(2); }; if (!cmd || argv.includes('--help') || argv.includes('-h')) { out(HELP); process.exit(cmd ? 0 : 2); } if (cmd !== 'find' && cmd !== 'wait') usage(`unknown command ${cmd}; run gstack-ci-carry --help`); +const policyFrom = flag('--policy-from') ?? 'base'; +if (policyFrom !== 'base' && policyFrom !== 'head') usage('--policy-from needs base or head'); +if (simulate !== (policyFrom === 'head') || (simulate && cmd !== 'find')) usage('find --policy-from head --simulate go together: a branch\'s own map is only ever simulated, never carried'); const git = (args: string[]) => { const r = spawnSync('git', args, { encoding: 'utf8', timeout: 30_000 }); return r.status === 0 ? r.stdout.trim() : ''; }; const root = git(['rev-parse', '--show-toplevel']); if (!root) usage('not inside a git repository'); const base = resolveBaseBranch(root, flag('--base')); -const loaded = loadConfigFromBase(root, base); -if (isConfigUnavailable(loaded) || loaded.errors.length) { out(`CI_CARRY: none (profiles_unavailable: origin/${base}'s .gstack/pregate.json is unreadable or invalid)`); process.exit(1); } +const loaded = simulate ? loadConfigFromHead(root) : loadConfigFromBase(root, base); +if (simulate) out(`policy ${isConfigUnavailable(loaded) ? 'unavailable' : loaded.label}; a dry run: nothing is written and nothing clears (a carry reads origin/${base}'s map only)`); +if (isConfigUnavailable(loaded) || loaded.errors.length) { out(`CI_CARRY: none (profiles_unavailable: ${simulate ? 'the branch\'s' : `origin/${base}'s`} .gstack/pregate.json is unreadable or invalid)`); process.exit(1); } const command = flag('--command'); -const mapping = command !== undefined ? loaded.config.ship_lanes_from[command] : undefined; -if (command !== undefined && !mapping) { out(`CI_CARRY: none (not_mapped: ${JSON.stringify(command)} is not a key of origin/${base}'s ship_lanes_from)`); process.exit(1); } +const mapping = command !== undefined ? laneMapping(loaded.config.ship_lanes_from, command) : undefined; +if (command !== undefined && !mapping) { out(`CI_CARRY: none (not_mapped: ${JSON.stringify(command)} is neither a key nor a declared alias in origin/${base}'s ship_lanes_from)`); process.exit(1); } if (mapping && !argv.includes('--allow-repo-commands')) usage('--command runs the mapping\'s inventory, a pin-file command: pass --allow-repo-commands'); const profiles = all('--profile').length ? all('--profile') : mapping ? mapping.profiles : [...new Set(Object.values(loaded.config.ship_lanes_from).flatMap(m => m.profiles))]; if (!profiles.length) { out(`CI_CARRY: none (no_profile: origin/${base} maps no lane in ship_lanes_from; pass --profile )`); process.exit(1); } -const check = buildRunCheck(root, null, argv.includes('--allow-repo-commands'), base); +const check = buildRunCheck(root, null, argv.includes('--allow-repo-commands'), base, simulate ? 'head' : 'base'); if ('error' in check) { out(`CI_CARRY: none (profiles_unavailable: ${check.error})`); process.exit(1); } const head = git(['rev-parse', 'HEAD']); const tree = git(['rev-parse', 'HEAD^{tree}']); function find(): number { const wtree = () => { const fp = wtreeFingerprint({ cwd: root }); return 'tree' in fp ? fp.tree : null; }; - const v = findCarry(check as Exclude, { profiles, head, tree, wtree, ...(mapping ? { inventory: () => runInventory(mapping.inventory, root) } : {}) }); + const v = findCarry(check as Exclude, { profiles, head, tree, wtree, ...(mapping ? mappingCoverage(mapping, root) : {}) }); if (argv.includes('--json')) out(JSON.stringify(v)); else out(v.ok ? `CI_CARRY: accepted ${v.receipt}` : `CI_CARRY: none (${v.reason}: ${v.detail})`); return v.ok ? 0 : 1; diff --git a/bin/gstack-doctor-components.sh b/bin/gstack-doctor-components.sh index 23db8f80d..4ec7b9a61 100644 --- a/bin/gstack-doctor-components.sh +++ b/bin/gstack-doctor-components.sh @@ -258,6 +258,7 @@ gstack_result_anchor() { DX_CAPTURE_REDACTION_FAILED) printf '%s' 'dx-capture-redaction-failed' ;; DX_CAPTURE_INTERNAL) printf '%s' 'dx-capture-internal' ;; POLICY_GSTACK_TOO_OLD) printf '%s' 'policy-gstack-too-old' ;; + PREGATE_GSTACK_TOO_OLD) printf '%s' 'pregate-gstack-too-old' ;; QUEUE_NOT_READY) printf '%s' 'queue-not-ready' ;; QUEUE_PROOF_MISSING) printf '%s' 'queue-proof-missing' ;; QUEUE_STAMP_NOT_STAMP_ONLY) printf '%s' 'queue-stamp-not-stamp-only' ;; diff --git a/bin/gstack-evidence b/bin/gstack-evidence index 8fc3e692b..e3d2bbab1 100755 --- a/bin/gstack-evidence +++ b/bin/gstack-evidence @@ -5,7 +5,8 @@ * * gstack-evidence run [--reuse [--max-age ]] --label -- * gstack-evidence record-ci --label --base -- - * gstack-evidence carried [--json] + * gstack-evidence passed -- + * gstack-evidence carried [--base ] [--json] * gstack-evidence check [--label [--expect-cmd ]]... | --all * [--max-age ] [--allow-paths ] * gstack-evidence bundle --out [--label ]... [--require ] [--selection L=]... @@ -65,8 +66,11 @@ * `record-ci` writes the explicit CI-sourced row for a lane CI carried: it * finds and verifies the run itself (lib/ci-carry.ts) for a command the base's * ship_lanes_from maps, then records source "ci", the runs and profiles, the - * run's finish time as ts, bound to HEAD's clean tree; `carried` lists those - * rows at the current tree for the ship receipt. + * run's finish time as ts, bound to HEAD's clean tree, and the policy that + * mapped it (`origin/@:…`); `carried` lists those rows at the + * current tree for the ship receipt, only when that policy is origin/ (GST-1190: a row a run against another base wrote never + * counts). * * MACHINE-LOCAL by design: neither the ledger nor the logs are brain-synced. * A synced record citing an unsynced log would grade FRESH on a machine where @@ -90,10 +94,10 @@ import { resultLine } from "../lib/result-codes"; import { isUnavailable, loadPolicyFromBase, resolveBaseBranch } from "../lib/ship-policy"; import { versionAtRef } from "../lib/restamp"; import { bashCommandInvocation, runScriptSync, wtreeFingerprint } from "../lib/script-invocation"; -import { findCarry } from "../lib/ci-carry"; +import { findCarry, laneMapping, mappingCoverage, perFileCommand } from "../lib/ci-carry"; +import { runChanged } from "../lib/run-evidence"; import { isConfigUnavailable, loadConfigFromBase } from "../lib/pregate/config"; import { buildRunCheck } from "../lib/pregate/run-check"; -import { runInventory } from "../lib/run-evidence"; const BIN_DIR = dirname(Bun.fileURLToPath(import.meta.url)); const LOG_MAX_BYTES = 2 * 1024 * 1024; @@ -122,10 +126,17 @@ interface EvidenceRecord { source?: "ci"; runs?: string[]; profiles?: string[]; + /** The CI jobs a jobs-mapped lane carried from (record-ci, GST-1189); tests_ran is then "n/a". */ + jobs?: string[]; /** The base policy that mapped a carried lane (record-ci), as `origin/@:.gstack/pregate.json`. */ policy?: string; /** The runner's own test count (executed for bun/jest/vitest/pytest/rspec; ran with skips included for gstack's shard runner and the remote gate), 0 for ZERO-RUN, or "unknown" when no supported summary was seen. */ - tests_ran?: number | "unknown"; + tests_ran?: number | "unknown" | "n/a"; + /** "files" when a carried narrow lane's tests_ran counts its inventory files the run covered (record-ci), not tests. */ + tests_unit?: "files"; + /** Why a carried lane was not owed at all: every job it names was path-skipped by an unchanged gate (record-ci, GST-1190). */ + not_owed?: string; + skipped_here?: { label: string; files: string[]; tests_ran: number | "unknown" }; } /** Bytes of streamed output kept for the tests_ran parse (the 2MB log would truncate the summary). */ @@ -598,6 +609,36 @@ function identityDigest(): string { return sha256(JSON.stringify({ bun: version("bun"), node: version("node"), os: process.platform, arch: process.arch, image: process.env.GSTACK_RUNNER_IMAGE || null })); } +/** A local row that passed with a counted run within the max age, under this machine's identity. */ +function countedPass(r: EvidenceRecord, maxAgeHours: number): boolean { + if (r.exit !== 0 || typeof r.tests_ran !== "number" || r.tests_ran <= 0) return false; + const age = Date.now() - Date.parse(r.ts); + if (!(age >= 0 && age <= maxAgeHours * 3600_000)) return false; + return r.identity === identityDigest(); +} + +/** + * `passed -- ` (GST-1186, read-only): did this exact command pass here at the current working tree under any label? The + * newest local row for the command hash and tree decides, so a newer failure vetoes; it must be a counted pass (countedPass). + * CI rows never count: a CI pass counts only through record-ci's live check against the base. Prints PASSED: label= + * source=local ts= and exits 0, else NOT_PASSED: and exits 1. gstack-pregate's guards read it. + */ +function cmdPassed(argv: string[]): number { + const sep = argv.indexOf("--"); + const command = sep < 0 ? "" : argv.slice(sep + 1).join(" "); + if (!command || sep !== 0) { console.error("usage: gstack-evidence passed -- "); return 2; } + const wtree = currentWtree(); + if (!wtree) { console.log("NOT_PASSED: no working-tree fingerprint"); return 1; } + let records: EvidenceRecord[] = []; + try { records = readJsonl(ledgerPath().file); } catch { records = []; } + const cmdSha = sha256(command); + const newest = records.findLast((r) => r.cmd_sha256 === cmdSha && r.wtree === wtree && r.source === undefined); + if (!newest) { console.log(`NOT_PASSED: no local ledger row for this command at tree ${wtree.slice(0, 12)}`); return 1; } + if (!countedPass(newest, 24)) { console.log(`NOT_PASSED: the newest row (label=${newest.label} ts=${newest.ts}) is not a counted pass within 24 h under this runtime`); return 1; } + console.log(`PASSED: label=${newest.label} source=local ts=${newest.reused_from ?? newest.ts}`); + return 0; +} + /** * `run --reuse`: the newest local row for this label, command and working tree decides. A counted pass under the same * identity within the max age is copied (original ts kept) and printed as a run would be; anything else runs the command. @@ -612,10 +653,7 @@ function reused(file: string, label: string, cmdSha: string, maxAgeHours: number return false; } const newest = records.findLast((r) => r.label === label && r.cmd_sha256 === cmdSha && r.wtree === wtree && r.source === undefined); - if (!newest || newest.exit !== 0 || typeof newest.tests_ran !== "number" || newest.tests_ran <= 0) return false; - const age = Date.now() - Date.parse(newest.ts); - if (!(age >= 0 && age <= maxAgeHours * 3600_000)) return false; - if (newest.identity !== identityDigest()) return false; + if (!newest || !countedPass(newest, maxAgeHours)) return false; try { appendJsonl(file, { ...newest, reused_from: newest.reused_from ?? newest.ts }, { mode: 0o600 }); } catch (e: any) { @@ -629,9 +667,11 @@ function reused(file: string, label: string, cmdSha: string, maxAgeHours: number /** * `record-ci`: carry a lane from CI. Its command must be a key of the base branch's ship_lanes_from; the run is found and - * verified here (lib/ci-carry.ts) at HEAD's clean tree, so no caller supplies run ids, a test count or a tree. Never runs the lane. + * verified here (lib/ci-carry.ts) at HEAD's clean tree, so no caller supplies run ids, a test count or a tree. Never runs the lane; + * inventory files every run skipped entirely run here through the entry's per_file runner (label .skipped) first, and the CI + * row is written only when that run passes at the same tree. */ -function cmdRecordCi(argv: string[]): number { +async function cmdRecordCi(argv: string[]): Promise { const sep = argv.indexOf("--"); const opts = sep < 0 ? argv : argv.slice(0, sep); const get = (name: string) => { const i = opts.indexOf(name); return i >= 0 ? opts[i + 1] : undefined; }; @@ -651,17 +691,33 @@ function cmdRecordCi(argv: string[]): number { const loaded = loadConfigFromBase(cwd, base); if (isConfigUnavailable(loaded)) return none("config_unavailable", loaded.detail); if (loaded.errors.length) return none("config_invalid", loaded.errors.map((x) => `${x.path}: ${x.message}`).join("; ")); - const from = loaded.config.ship_lanes_from; - if (!Object.prototype.hasOwnProperty.call(from, command)) return none("not_mapped", `the command is not a key of origin/${base}'s ship_lanes_from`); - const m = from[command]!; + const m = laneMapping(loaded.config.ship_lanes_from, command); + if (!m) return none("not_mapped", `the command is neither a key nor a declared alias in origin/${base}'s ship_lanes_from`); const check = buildRunCheck(cwd, null, true, base); if ("error" in check) return none("profiles_unavailable", check.error); const head = git(["rev-parse", "HEAD"]); const tree = git(["rev-parse", "HEAD^{tree}"]); if (!head || !tree) return none("no_head", "HEAD is unreadable"); const wtree = () => { const fp = wtreeFingerprint({ cwd, timeoutMs: 30000 }); return "tree" in fp ? fp.tree : null; }; - const v = findCarry(check, { profiles: m.profiles, head, tree, wtree, inventory: () => runInventory(m.inventory, cwd) }); + const v = findCarry(check, { profiles: m.profiles, head, tree, wtree, ...mappingCoverage(m, cwd) }); if (!v.ok) return none(v.reason, v.detail); + let here: EvidenceRecord["skipped_here"]; + if (v.remainder?.length) { + const hereLabel = `${label}.skipped`; + console.error(`gstack-evidence record-ci: ${v.remainder.length} inventory file(s) skipped every test in CI; running them here (label=${hereLabel})`); + const hereCommand = perFileCommand(m.per_file!, v.remainder); + const exit = await cmdRun(["--label", hereLabel, "--", hereCommand]); + if (exit !== 0) return none("skipped_failed", `the per_file run of ${v.remainder.length} skipped file(s) exited ${exit} (label=${hereLabel})`); + let rows: EvidenceRecord[] = []; + try { rows = readJsonl(ledgerPath().file); } catch { rows = []; } + here = { label: hereLabel, files: v.remainder, tests_ran: rows.findLast((r) => r.label === hereLabel && r.source === undefined && r.cmd_sha256 === sha256(hereCommand) && r.wtree === tree)?.tests_ran ?? "unknown" }; + if (typeof here.tests_ran !== "number" || here.tests_ran <= 0) return none("skipped_zero_run", `the per_file run of ${v.remainder.length} skipped file(s) ran ${here.tests_ran} tests here (label=${hereLabel}); a lane passes on its runner count, never exit 0 alone`); + for (const e of v.runs) { + const changed = runChanged(check.deps, e.run_id, e.run_attempt); + if (changed) return none("run_changed", changed); + } + if (wtree() !== tree) return none("tree_changed", `the working tree changed while the per_file run of label=${hereLabel} ran`); + } let paths: ReturnType; try { paths = ledgerPath(); @@ -673,15 +729,22 @@ function cmdRecordCi(argv: string[]): number { const runs = v.runs.map((r) => `${r.run_id}/${r.run_attempt}`); const rec: EvidenceRecord = { ts: new Date(v.runs.map((r) => r.finished_at).sort()[0]!).toISOString(), label, command, cmd_sha256: sha256(command), exit: 0, duration_s: 0, - commit: head, tree, dirty: false, wtree: tree, tests_ran: v.tests, source: "ci", runs, profiles: m.profiles, policy: loaded.label, + commit: head, tree, dirty: false, wtree: tree, tests_ran: v.tests, source: "ci", runs, profiles: m.profiles, policy: loaded.label, ...(m.jobs ? { jobs: m.jobs } : {}), ...(v.unit ? { tests_unit: v.unit } : {}), ...(v.notOwed ? { not_owed: v.notOwed } : {}), ...(here ? { skipped_here: here } : {}), }; appendJsonl(paths.file, rec, { mode: 0o600 }); - console.error(`gstack-evidence: recorded label=${label} exit=0 tests_ran=${v.tests} log=- source=ci runs=${runs.join(",")} policy=${loaded.label}`); + console.error(`gstack-evidence: recorded label=${label} exit=0 tests_ran=${v.tests}${v.unit ? ` unit=${v.unit}` : ""} log=- source=ci runs=${runs.join(",")} policy=${loaded.label}${here ? ` skipped_here=${here.files.length} tests_ran_here=${here.tests_ran}` : ""}`); + if (v.notOwed) console.error(`gstack-evidence: not owed: ${v.notOwed}`); return 0; } -/** `carried`: each label's newest row when it is a CI-sourced row at the current working tree, as the receipt's ci_carried entries. */ +/** + * `carried`: each label's newest row when it is a CI-sourced row at the current working tree whose mapping policy is + * origin/ (the receipt's base; default the resolved base branch), as the receipt's ci_carried entries. + */ function cmdCarried(argv: string[]): number { + const at = argv.indexOf("--base"); + if (at >= 0 && !argv[at + 1]) { console.error("usage: gstack-evidence carried [--base ] [--json]"); return 2; } + const policyRef = `origin/${(at >= 0 ? argv[at + 1]! : resolveBaseBranch(git(["rev-parse", "--show-toplevel"]) ?? process.cwd())).replace(/^origin\//, "")}@`; let records: EvidenceRecord[] = []; try { records = readJsonl(ledgerPath().file); @@ -691,7 +754,7 @@ function cmdCarried(argv: string[]): number { const wtree = currentWtree(); const latest = new Map(); for (const r of records) latest.set(r.label, r); - const carried = [...latest.values()].filter((r) => r.source === "ci" && !!wtree && r.wtree === wtree && Array.isArray(r.runs)) + const carried = [...latest.values()].filter((r) => r.source === "ci" && !!wtree && r.wtree === wtree && Array.isArray(r.runs) && typeof r.policy === "string" && r.policy.startsWith(policyRef)) .flatMap((r) => r.runs!.map((ref) => { const [run, attempt] = ref.split("/").map(Number); return { lane: r.label, run, attempt, tree: r.wtree! }; })); if (argv.includes("--json")) console.log(JSON.stringify(carried)); else for (const c of carried) console.log(`CI_CARRIED: ${c.lane} run=${c.run}/${c.attempt} tree=${c.tree.slice(0, 12)}`); @@ -756,10 +819,10 @@ function cmdCheck(argv: string[]): number { allFresh = false; continue; } - const testsRan = typeof latest.tests_ran === "number" ? latest.tests_ran : "unknown"; + const testsRan = typeof latest.tests_ran === "number" || latest.tests_ran === "n/a" ? latest.tests_ran : "unknown"; const changedDuringRun = Array.isArray(latest.wtree_changed) ? latest.wtree_changed.filter((p): p is string => typeof p === "string" && p !== "" && !/[\u0000-\u001f\u007f]/.test(p)).slice(0, 5) : []; const ci = latest.source === "ci" && Array.isArray(latest.runs) ? ` source=ci runs=${latest.runs.filter((s) => /^\d+\/\d+$/.test(s)).join(",")}` : ""; - const detail = `label=${label} exit=${latest.exit} tests_ran=${testsRan} ts=${latest.ts}${latest.log_path ? ` log=${latest.log_path}` : ""}${ci}`; + const detail = `label=${label} exit=${latest.exit} tests_ran=${testsRan}${latest.tests_unit === "files" ? " unit=files" : ""} ts=${latest.ts}${latest.log_path ? ` log=${latest.log_path}` : ""}${ci}`; let verdict: "FRESH" | "STALE" | "ZERO-RUN" = "FRESH"; let reason = ""; if (latest.exit !== 0) { @@ -911,7 +974,9 @@ try { if (sub === "run") { process.exit(await cmdRun(rest)); } else if (sub === "record-ci") { - process.exit(cmdRecordCi(rest)); + process.exit(await cmdRecordCi(rest)); + } else if (sub === "passed") { + process.exit(cmdPassed(rest)); } else if (sub === "carried") { process.exit(cmdCarried(rest)); } else if (sub === "check") { @@ -923,7 +988,7 @@ try { } else if (sub === "verify") { process.exit(cmdVerify(rest)); } else { - console.error("usage: gstack-evidence run|record-ci|carried|check|bundle|ancestor|verify ..."); + console.error("usage: gstack-evidence run|record-ci|passed|carried|check|bundle|ancestor|verify ..."); process.exit(2); } } catch (e: any) { diff --git a/bin/gstack-ship-receipt b/bin/gstack-ship-receipt index e6c01c637..15f1a15f3 100755 --- a/bin/gstack-ship-receipt +++ b/bin/gstack-ship-receipt @@ -175,9 +175,9 @@ function reviewedProduct(repoRoot: string, base: string, head: string, pregateFi return typeof id === 'string' ? { head_tree: tree, product_patch_id: id || 'none' } : { head_tree: tree }; } -/** The lanes CI carried at the checked-out tree, from this machine's evidence ledger (gstack-evidence carried); undefined when none. */ -function ciCarried(): ShipReceipt['ci_carried'] { - const r = runScriptSync(path.join(path.dirname(Bun.fileURLToPath(import.meta.url)), 'gstack-evidence'), ['carried', '--json'], { encoding: 'utf8', timeout: 60_000 }); +/** The lanes CI carried at the checked-out tree under origin/'s mapping, from this machine's evidence ledger (gstack-evidence carried); undefined when none. */ +function ciCarried(base: string): ShipReceipt['ci_carried'] { + const r = runScriptSync(path.join(path.dirname(Bun.fileURLToPath(import.meta.url)), 'gstack-evidence'), ['carried', '--base', base, '--json'], { encoding: 'utf8', timeout: 60_000 }); try { const list = JSON.parse(r.stdout); return Array.isArray(list) && list.length ? list : undefined; } catch { return undefined; } } @@ -271,7 +271,7 @@ function cmdWrite(rest: string[], json: boolean): number { version: flag(rest, '--version') ?? receiptVersion(repoRoot, head, base), base, gated_tree: flag(rest, '--gated-tree') ?? tree?.gated_tree, predecessor: flag(rest, '--predecessor') ?? tree?.predecessor?.recorded_head ?? undefined, tree: tree?.tree, gate_reuse: tree ? `${tree.gate_reuse} (${tree.gate_reuse_reason})` : undefined, - gate: jsonArg(flag(rest, '--gate')), ci: jsonArg(flag(rest, '--ci')), ci_carried: ciCarried(), + gate: jsonArg(flag(rest, '--gate')), ci: jsonArg(flag(rest, '--ci')), ci_carried: ciCarried(base), spend_usd: flag(rest, '--spend-usd') !== undefined ? (Number.isFinite(Number(flag(rest, '--spend-usd'))) ? Number(flag(rest, '--spend-usd')) : flag(rest, '--spend-usd')) : undefined, session_kind: sessionKind, artifacts_consumed: flag(rest, '--artifacts-consumed'), run: flag(rest, '--run'), thread: flag(rest, '--thread'), policy: flag(rest, '--policy') ?? tree?.policy, queue_mode: flag(rest, '--queue-mode'), preregistration: flag(rest, '--preregistration'), history: flag(rest, '--history'), diff --git a/docs/pregate.md b/docs/pregate.md index a6b692927..f0f46da40 100644 --- a/docs/pregate.md +++ b/docs/pregate.md @@ -88,9 +88,9 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`). | `stall_timeout_s` | seconds a test command may print nothing while using under a tenth of one CPU before its process group is killed and the check is `incomplete` (`PREGATE_STALLED`); default 300, `0` disables; `--stall-timeout` overrides; the `--remote` gate is exempt (see [supervision](#supervision)) | | `timeout_s` | seconds each runner process may take (the unit-file group, each serial file, each guard without its own `timeout_s`, and the remote gate's deadline) when `--timeout` is not passed; 1 to 3600, default 600. `gstack-ship-preflight` and the `finish` fallback run pregate without `--timeout`, so this is how a repo with long lanes sets theirs | | `remote_gate` | the repo's remote test gate command; under 8 cores the local test tier becomes a `requires-remote` obligation naming it, and `--remote` runs it (see [remote gate](#remote-gate)) | -| `run_profiles` | name → `{ workflow, events, jobs, artifact, max_age_hours?, os? }`: the GitHub Actions runs `clear --run` verifies as evidence on the identical tree; a profile named after a lane (`free-tests.yml/macos-named-regressions`) clears that lane (see [run evidence](#run-evidence)). `--from-run` and the main-push reuse job verify runs against them too ([from-run](#from-run), [CI reuse](#ci-reuse)) | +| `run_profiles` | name → `{ workflow, events, jobs, artifact, max_age_hours?, os?, skippable? }`: the GitHub Actions runs `clear --run` verifies as evidence on the identical tree; a profile named after a lane (`free-tests.yml/macos-named-regressions`) clears that lane (see [run evidence](#run-evidence)). `--from-run` and the main-push reuse job verify runs against them too ([from-run](#from-run), [CI reuse](#ci-reuse)) | | `remote_gate_runs` | `{ profiles, inventory, skipped? }`: the run profiles that together clear `pregate.json/remote_gate`, the command that prints the remote gate's test files, one per line, which their runs must cover, and an optional command that prints which selected files the gate itself never runs (see [run evidence](#run-evidence)) | -| `ship_lanes_from` | maps each exact /ship lane command to `{ profiles, inventory }`: the run profiles whose verified runs at HEAD's clean tree carry that lane instead of running it, and the command listing its test files, which their runs must cover (see [CI carry](#ci-carry)); read from the base branch only | +| `ship_lanes_from` | maps each exact /ship lane command to `{ profiles, inventory }` or `{ profiles, jobs }`, with optional `aliases` (and, on an inventory entry, an optional `per_file` runner for files CI skipped entirely, see [skipped files](#skipped-files)): the run profiles whose verified runs at HEAD's clean tree carry that lane instead of running it, and either the command listing its test files, which their runs must cover, or the CI jobs that must have succeeded (see [CI carry](#ci-carry)); read from the base branch only | | `allowlist` | `[{ finding, paths?, pattern?, reason }]`: benign findings the `secrets` check accepts and names as allowlisted, by gstack-redact finding id or non-secret category, path globs and a pattern anchored at the finding; never `env.value` (see [allowlist](#allowlist)) | Only the `origin/` copy or the user-owned file below is honoured, so a @@ -259,7 +259,8 @@ Two stages with separate clocks, both printed: `--allow-repo-commands` the selected guards are listed and the check is `incomplete`. Guards are read from the base branch's copy (or the user-owned file when the base carries none) and cannot be - downgraded to warn. See [guards](#guards). + downgraded to warn. A guard whose exact command already passed at this tree + is not run again ([guard carry](#guard-carry)). See [guards](#guards). **Stage 2, test execution:** @@ -710,7 +711,22 @@ other gate change. Each profile names the workflow file, the events it accepts, every job that must have passed (each exactly once), and the artifact holding the run's `tree.json`. `max_age_hours` (1–168, default 24) bounds how old the run may be, and `os` (`Linux`, `macOS`, `Windows`) requires -`tree.json` to record that runner OS. Each workflow uploads `tree.json` from +`tree.json` to record that runner OS. `skippable` +(`{ jobs, gate, scripts }`, GST-1190) lets named jobs be skipped by a path +filter: a skipped job counts only in a `pull_request` run (whose gate diffs +against the pull request's base; a push or dispatch gate diffs against the last +push, which the branch's push history controls), when the `gate` job (the one +whose output decides the skip) succeeded in the same attempt, and the workflow +file plus the gate's inputs are byte-identical to `origin/`. Those inputs +are the declared `scripts`, which must list every repo file the gate reads, +plus every path the gate job's steps name in the base copy of the workflow +(also read under the job's `working-directory` and after a `${{ … }}` +prefix), plus `package.json` when the gate runs a package script. gstack +cannot see what a gate script imports or what a package script runs, so list +those files in `scripts`. Otherwise the pull request controlled the skip +decision, so the job refuses as `job_skipped` and the lane runs here. A lane +whose every named job was skipped this way prints `LANE: