diff --git a/.github/workflows/version-gate.yml b/.github/workflows/version-gate.yml index 32b36a04f..1be24d4e6 100644 --- a/.github/workflows/version-gate.yml +++ b/.github/workflows/version-gate.yml @@ -1,12 +1,10 @@ name: Version Gate +# Every PR: under changelog_fragments a PR that touches no release file must +# still prove it carries a fragment (docs/ship-policy.md#changelog-fragments). on: pull_request: branches: [main] - paths: - - 'VERSION' - - 'CHANGELOG.md' - - 'package.json' concurrency: group: version-gate-${{ github.event.pull_request.number }} diff --git a/CHANGELOG.md b/CHANGELOG.md index 132e00010..06334f6ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,43 @@ ### Changed - **Plan first has two narrow fix exemptions.** An S1 emergency (a broken install or upgrade, a red main, a security bug) and a fix from a problem list that was already diagnosed and audited can be built without an `/autoplan` run. Every other fix, a one-line bug fix included, gets a one-round, eng-only plan. `docs/captain.md` § Plan before build says how, and `captain/SKILL.md`, `captain/templates/CREW.md` rule 24 and the Capy skill's SKILL.md list the same exemptions. +## [1.91.336.0] - 2026-10-30 + +**A repository can now assign its version when a PR merges instead of inside the PR: PRs carry unversioned CHANGELOG fragments and `gstack-ship-queue land` stamps, pushes and merges.** Today every PR picks a version before it merges, so each merge makes the next PR's version stale and its lane redoes the stamp (a median 14.5 minutes from stamp commit to merge over 30 slots, Oct 27-29; the queue froze on hand-made stamps in #3422 and #3487). With `changelog_fragments` on, two PRs cut from one base cannot collide, and nobody stamps by hand. gstack's own policy does not turn it on in this release (docs/ship-policy.md#changelog-fragments). + +### Added +- **`changelog_fragments: { dir }`** (`lib/ship-policy.ts`, `lib/changelog-fragments.ts`): a PR adds `/.md` with its entry text and optional `bump: minor|major` front matter. Fragments are read as Git blobs only; a symlink, gitlink, nested path, empty or oversized fragment, one with a release heading, a reused name, or one whose text is already released on the base (a parent PR's fragment carried in by stacked history) is `CHANGELOG_FRAGMENT_INVALID` before any version is allocated. +- **`gstack-restamp` folds fragments**: their bodies go above any legacy `[Unreleased]` section under one new heading, the files are deleted, and the highest declared bump wins over the policy's. The tree receipt's `changelog-fragments` rule accepts exactly that fold and nothing else. +- **`gstack-ship-queue land --pr `** (`lib/ship-queue-land.ts`): the per-PR merge step. It takes the land lock (`refs/notes/gstack-land`), checks the /ship receipt, the READY receipt and checks at the pin, merges the base resolving release fields only (a clean dependency edit survives; a product conflict refuses), stamps, pushes a commit with parents [pin, base] under an exact lease, waits for checks, restamps if the base moved (at most twice), squashes with `--match-head-commit` (merges with `--merge` under `history: merge-only`) and confirms the base tree. A PR whose base is another branch refuses before the lock; a refusal before the push releases the lock. Every step is journaled and resumable, also from the stamp commit's pin when the local journal is gone. `--repair` opens the follow-up PR after `QUEUE_LAND_DRIFT`; `--census` prints what land would do with every open PR, writing nothing. +- **/ship writes the fragment** (`lib/ship-policy.ts`, `ship/SKILL.md.tmpl`): under the key, `show-effective` prints a `ship: changelog_fragments` line, so Step 12 writes no version and Step 13 writes `/.md` (linted with `gstack-prose-lint --gate`) instead of an `[Unreleased]` section; the skill's policy paragraph now says its `ship:` lines bind Steps 12–16. +- **`restamp_command` gets `GSTACK_RESTAMP_BASE_SHA`**, the release predecessor, so a repo counter such as a SAVE_VERSION computes predecessor + 1 from the base and never from the branch. +- New result codes with troubleshooting anchors: `CHANGELOG_FRAGMENT_INVALID`, `QUEUE_LAND_LOCKED`, `QUEUE_LAND_CONFLICT`, `QUEUE_LAND_DRIFT`, `QUEUE_LAND_BASE_BUSY`. + +### Changed +- **Version Gate runs on every PR** (the path filter is gone), so under `changelog_fragments` a PR that touches no release file still proves it carries a fragment. An unstamped head keeps its release fields at its merge base and adds a valid fragment (legacy `[Unreleased]` passes; a `[bot]` author needs none); a merge-step stamp commit must have parents [pin, base] and carry the base plus its fragments' effective bump, so a lane that stamps its own head fails. Without the key nothing changes. + +### Tests +- `test/changelog-fragments.test.ts`: blob-only reads and refusals, a carried-back fragment refused on whole released lines only, bump precedence, fold order with a legacy section, the restamp fold and its tree receipt, an edited fold refused. +- `test/ship-queue-land.test.ts` (bare origin and a gh shim): happy path; two PRs from one base without a collision; a legacy stamp plus a dependency edit; a title-derived entry; a product conflict; the lock and `--break-lock`; the base moving during CI; a foreign push never overwritten; a missing receipt; drift, `--repair` and the lock release; the census; a receipt that fails #3567's depth check refuses; a base with a four-part package.json in `stamp_files` lands four parts (#3557), gstack's own shape stays three; one run without `--no-wait` merges; a PR on another base refuses; `--break-lock` refuses a drift lock; a conflict releases the lock; a stacked parent's fragment refuses; a lost journal resumes; `merge-only` merges with `--merge`; a merge GitHub completed behind an error reply reconciles; `--break-lock` refuses a holder mid-merge; `--repair` resumes after a failed PR create and its PR takes over from a clean checkout; a symlinked legacy VERSION refuses in `--dry-run`; a `package.json` version source keeps its dependency edits. +- `test/version-gate-fragments.test.ts`: fragment heads, premature version changes, a bot author, an advanced base, an invalid fragment, the stamp commit's effective bump (open version claims do not apply to it), and the workflow's missing path filter. +- `test/restamp-command-boundary.test.ts`: the hook reads its counter from `GSTACK_RESTAMP_BASE_SHA`, not the branch. + +## [1.91.335.0] - 2026-10-30 + +### Fixed +- **Captains now read factory.md and the owner's model preference.** A Capy session with no gstack checkout (every Captain and coordinator, which run `install.sh --check --for none`) used to skip factory.md without a word and never applied `agent_model`, because both were read only from the checkout. The skill now tells it to read the `factory.md` beside SKILL.md, to say `factory.md: none` on its own line when there isn't one, and to read the `agent_model` line of `preferences.yaml` itself. Owners get the fix by copying `capy/skill/gstack/` into the Drive skill folder. + +## [1.91.334.0] - 2026-10-30 + +**OpenClaw opens gstack's large task skills again (GST-110).** #3555 made `hybrid` OpenClaw's default and registered review, investigate, qa, ship and plan-eng-review as skills of their own. OpenClaw 2026.10.6 serves a registered skill's `SKILL.md` whole or refuses it, with a 32,768-byte limit when its model catalog has no context window for the model. In HARN-5's panel the agent's first read of the 77,691-byte `gstack-review` file was refused in 4 of 5 trials, and opens fell from 5/5 to 0/5. Under the router layout the same file had been read as a plain file inside the router's folder, which OpenClaw pages. + +### Fixed +- **`gstack-harness register openclaw`** (`lib/harness-register.ts`): a skill whose rendered `SKILL.md` is over 32,000 bytes (`OPENCLAW_INSTRUCTION_MAX`, under both OpenClaw's 32,768-byte limit and the 32,000-character budget it applies with a 100K-200K-token context window, with the OpenClaw source lines it comes from) is registered as a short entry with the same frontmatter whose body says to read the skill's full `SKILL.md` in a plain copy of every skill and follow it. In hybrid that copy is the router's `gstack/skills/`, where the router layout read the same file, now including the five task skills; under `--skills all` it is `.gstack-skills/`, a folder OpenClaw's skill scan skips. Because the copy holds every skill, `sections/` paths and sibling paths such as `../gstack-qa/` and autoplan's `../gstack-plan-eng-review/SKILL.md` resolve to full texts. That covers four of hybrid's five direct skills and 26 of 63 skills in `--skills all` today. OpenClaw treats only a registered skill folder's `SKILL.md` as an instruction file. The ledger records the copies, so refresh, mode switches, upgrades from a #3555 registration and uninstall handle them. Skill bodies are unchanged, and Hermes, dots and `generic` registrations are byte-identical. +- docs/hosts.md and docs/OPENCLAW.md describe the entry and the limit, and docs/uninstall.md's manual OpenClaw removal command also removes `.gstack-skills/`. + +### Tests +- `test/harness-register.test.ts`: in hybrid and all, every `SKILL.md` OpenClaw would register is within the limit, each entry names a full `SKILL.md` that exists, has the same frontmatter, is outside OpenClaw's registered set and is in the ledger, every `../gstack-*` path in the copied texts resolves to a full text, re-registering over a #3555 registration converts it cleanly, and uninstall leaves nothing. Hermes's registration equals its render byte for byte. With `GSTACK_OPENCLAW_BIN` or `openclaw` on PATH, a real `openclaw skills list --json` and `skills info` show exactly `gstack` and the five task skills, each served from an entry within the limit. + ## [1.91.333.0] - 2026-10-30 **`gstack-factory-scorecard --now` now ends the window, so a past week no longer counts what happened after it.** Every line counts only items whose timestamps fall in `[now - since-days, now)`. Before, `--now` moved only the start, so `--now 2026-10-15T00:00:00Z --since-days 14` counted 290 merged PRs (every merge from 2026-10-01 through the day of the run) where the window holds 65, and the gate-run and release lines also counted through the day of the run. Rerun with the fix, the two weeks before 2026-10-15 show 65 merged PRs and the two weeks before 2026-10-29 show 197, matching GitHub search for both windows. diff --git a/VERSION b/VERSION index 131e70491..6346b9d6a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.91.333.0 +1.91.336.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index f97901bab..51d51d4f1 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.91.333.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.91.336.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/bin/gstack-doctor-components.sh b/bin/gstack-doctor-components.sh index f3c867766..eb6e79224 100644 --- a/bin/gstack-doctor-components.sh +++ b/bin/gstack-doctor-components.sh @@ -315,6 +315,11 @@ gstack_result_anchor() { RECEIPT_PROSE_MISMATCH) printf '%s' 'receipt-prose-mismatch' ;; RECEIPT_CARRY_PREGATE_STALE) printf '%s' 'receipt-carry-pregate-stale' ;; RECEIPT_GIT_TOO_OLD) printf '%s' 'receipt-git-too-old' ;; + CHANGELOG_FRAGMENT_INVALID) printf '%s' 'changelog-fragment-invalid' ;; + QUEUE_LAND_LOCKED) printf '%s' 'queue-land-locked' ;; + QUEUE_LAND_CONFLICT) printf '%s' 'queue-land-conflict' ;; + QUEUE_LAND_DRIFT) printf '%s' 'queue-land-drift' ;; + QUEUE_LAND_BASE_BUSY) printf '%s' 'queue-land-base-busy' ;; PR_WAIT_BLOCKED) printf '%s' 'pr-wait-blocked' ;; PR_WAIT_REFUSED) printf '%s' 'pr-wait-refused' ;; APPROVAL_RECORD_INVALID) printf '%s' 'approval-record-invalid' ;; diff --git a/bin/gstack-ship-queue b/bin/gstack-ship-queue index b130a7133..0584c3bba 100755 --- a/bin/gstack-ship-queue +++ b/bin/gstack-ship-queue @@ -14,11 +14,15 @@ * gstack-ship-queue gate-ahead --list --pr * gstack-ship-queue verify --pr --screen (train policy: the head's screen verdict from its run) * gstack-ship-queue verify --post-merge [--timeout ] (after a merge: main's push workflows at that commit until they settle) + * gstack-ship-queue land --pr [--dry-run] [--no-wait] [--break-lock] (the merge step; lib/ship-queue-land.ts) + * gstack-ship-queue land --pr --repair (after QUEUE_LAND_DRIFT: open the outside merge's release PR) + * gstack-ship-queue land --census * gstack-ship-queue train build|status|verdict|bisect|rebuild|watch|prune ... (merge trains; train --help) * * Every subcommand takes --base (default origin/HEAD, else main) and * ends with GSTACK_RESULT: skill=ship-queue status=complete|refused run=. - * gstack-restamp stays the primitive. Nothing here pushes, except under a + * gstack-restamp stays the primitive. Nothing here pushes, except land, ready + * under changelog_fragments (its READY receipt note), and under a * ship policy with a `train` block: merge trains push train/ branches * and the refs/notes/gstack-trains records, and dispatch through eval-bg. * Logic: lib/ship-queue.ts, lib/ship-queue-gate.ts, lib/ship-queue-train*.ts @@ -52,11 +56,18 @@ const HELP = `gstack-ship-queue — the shared merge queue's protocol steps (doc verify --pr --screen (train policy) the PR head's screen verdict read from its run (SCREEN_VERDICT) verify --post-merge [--timeout ] wait for every check run at a merged commit to settle: POST_MERGE: green|red|pending and one CHECK: line per workflow; resumable, default timeout 2700 s + land --pr [--dry-run] [--no-wait] [--timeout ] [--break-lock] [--allow-repo-commands] + the merge step: lock, merge the base, stamp (fragments folded), push the stamp commit, + wait for its checks, squash with --match-head-commit, confirm the base tree; resumable + land --pr --repair after QUEUE_LAND_DRIFT: open the PR that gives the merge outside land its own + release; /ship and land it, which releases the lock + land --census what land would do with every open PR (read-only) train build|status|verdict|bisect|rebuild|watch|prune ... merge trains, when the base policy declares a train block (train --help; docs/ship-policy.md#run-a-merge-train); ready then takes --screen|--no-screen ---base : base branch (default: origin/HEAD, else main). Nothing here pushes. +--base : base branch (default: origin/HEAD, else main). Nothing here pushes, except land (the stamp commit and the +refs/notes/gstack-land lock), ready under changelog_fragments (its READY receipt note) and merge trains. ${exitTableHelp()}`; function flag(args: string[], name: string): string | undefined { @@ -152,6 +163,13 @@ async function main(argv: string[]): Promise { return train.trainMain(ctx, args); } if (cmd === 'verify' && flag(args, '--post-merge') !== undefined) return (await import('../lib/ship-queue-train-cli')).postMergeMain(ctx, args); + if (cmd === 'land') { + const l = await import('../lib/ship-queue-land'); + if (args.includes('--census')) return finish('land', l.census(ctx)); + if (!pr || !/^\d+$/.test(pr)) return usage('land needs --pr (or --census)'); + const timeout = flag(args, '--timeout'); + return finish('land', l.land(ctx, { pr, repair: args.includes('--repair'), dryRun: args.includes('--dry-run'), noWait: args.includes('--no-wait'), breakLock: args.includes('--break-lock'), allowRepoCommands: args.includes('--allow-repo-commands'), waitTimeoutMin: timeout && /^\d+$/.test(timeout) ? Number(timeout) : undefined })); + } if (!pr || !/^\d+$/.test(pr)) return usage(`${cmd} needs --pr `); const gateExit = flag(args, '--gate-exit'); if (gateExit !== undefined && !/^-?\d+$/.test(gateExit)) return usage('--gate-exit needs an exit code'); diff --git a/capy/skill/gstack/SKILL.md b/capy/skill/gstack/SKILL.md index 8f2bfcc8d..da5b50ef9 100644 --- a/capy/skill/gstack/SKILL.md +++ b/capy/skill/gstack/SKILL.md @@ -9,7 +9,7 @@ description: Install and use gstack in Capy and Claude Code. Load at session sta Resolve `scripts/install.sh` relative to this SKILL.md, not the working repository. It is a frozen bootstrapper: it prints `gstack-capy-skill: wrapper ()` first, learns the supported minimum from gstack's remote, and hands over to the checkout's own wrapper (`bin/gstack-capy-session`). -1. At session start run `bash scripts/install.sh --check`, adding `--for ` when you know the workflow (`--for autoplan`, `--for review`, `--for qa`). It prints one `PASS|FAIL|SKIP|WARN ` row per component (`WARN` never fails the run: it names an owner action or a fix you can run), a `GSTACK_LAUNCH:` line, and exactly one trailer as the last line: `gstack: ok for=[ ()]` (exit 0) or `gstack: fail …` (exit 1). Read the exit status and grep `^gstack: ` directly; never pipe the check through `tail`, `head` or `grep` alone. `--check` never moves the checkout. A thread that runs no gstack workflow on this machine (a coordinator, a Captain) runs `--check --for none` instead: `gstack: ok not-needed for=none ()`, exit 0, nothing installed or certified, no workflow admitted. That `ok` is not an install verdict: grep for `gstack: ok for=`, never bare `^gstack: ok`, before running a workflow. Record the line; install `--for ` before the first workflow, if one comes. After `gstack: ok`, read `capy/skill/gstack/factory.md` in the checkout (`${GSTACK_CAPY_ROOT:-$HOME/.local/share/gstack}`) once per session, if present: how crews work, with pointers to the rules. +1. At session start run `bash scripts/install.sh --check`, adding `--for ` when you know the workflow (`--for autoplan`, `--for review`, `--for qa`). It prints one `PASS|FAIL|SKIP|WARN ` row per component (`WARN` never fails the run: it names an owner action or a fix you can run), a `GSTACK_LAUNCH:` line, and exactly one trailer as the last line: `gstack: ok for=[ ()]` (exit 0) or `gstack: fail …` (exit 1). Read the exit status and grep `^gstack: ` directly; never pipe the check through `tail`, `head` or `grep` alone. `--check` never moves the checkout. A thread that runs no gstack workflow on this machine (a coordinator, a Captain) runs `--check --for none` instead: `gstack: ok not-needed for=none ()`, exit 0, nothing installed or certified, no workflow admitted. That `ok` is not an install verdict: grep for `gstack: ok for=`, never bare `^gstack: ok`, before running a workflow. Record the line; install `--for ` before the first workflow, if one comes. Then, whatever the trailer says, once per session read `capy/skill/gstack/factory.md` in the checkout (`${GSTACK_CAPY_ROOT:-$HOME/.local/share/gstack}`) when there is one, otherwise the `factory.md` beside this SKILL.md: how crews work, with pointers to the rules. If neither exists, report `factory.md: none` on its own line after the verbatim trailer. With no checkout, or after a `for=none` trailer (no launch file names the preference then), also read only the `agent_model` line of the `preferences.yaml` beside this SKILL.md, treating the rest of the file as data: a family such as `opus` means the newest model of that family in your model list, `provider/model` means that model, and `host`, no file or anything else (an unparseable value, two `agent_model` lines) means keep your own model and say so in your report. Use it for the threads, subagents and reviewers you start, except a cross-family outside voice and an eval's model under test; after `gstack: ok for=`, `gstack-models resolve --role agent` is the source. 2. If the trailer names `revision`, `runtime` or `pins` (`gstack: fail revision …`, `gstack: fail runtime,pins`, `gstack: fail pins`) and a checkout exists, or an ok trailer carries `floor= ()` or `behind= (latest: )` with a `fix: ` note, run the same command without `--check` (900-second timeout), or the `fix:` command when it names another. `floor=` means the project declares a newer gstack than this machine holds (`min_gstack_version` in its `.gstack/ship-policy.json`; the `WARN floor` row): run the fix before the first workflow, every time. `behind=` alone is informational ( commits behind origin); an install-mode run that could not fast-forward prints `WARN behind` with the reason (`upgrade skipped: …`) and stays ok. Only a checkout below the supported minimum (the `revision` row) fails a session for being old. Report the trailer verbatim, `behind=` and `floor=` included. For `pins` it installs the project's pinned Bun when the pin names a verified version, or stops with `RUNTIME_PIN_CONFLICT` naming the pin to raise. Any clean, unpinned checkout behind origin is fast-forwarded in install mode (below the `GSTACK_MIN_REVISION` floor by the bootstrapper, `gstack-capy-skill: upgraded -> `; above it by the checkout, `SELF_UPGRADE: done -> `), reinstalled and rechecked in one locked step. A pinned (`--revision`, detached), dirty or diverged checkout, or one another live session holds a lease on (`UPGRADE_BUSY`), is never moved: the trailer names the cause and the exact commands (above the floor it stays ok with `WARN behind` after `SELF_UPGRADE: skipped ()`, carrying `UPGRADE_BUSY` or `UPGRADE_REFUSED`). A lease whose pid was reused after a wake is reclaimed on its own. `GSTACK_CAPY_AUTO_UPGRADE=0` disables every fast-forward, `GSTACK_CAPY_SELF_UPGRADE=off` only the above-floor one. 3. If the trailer is `gstack: fail not-installed …` and this session runs no gstack workflow, record that line and continue; nothing is installed. Install right before the first workflow with `bash scripts/install.sh --for ` (about 30 s cold: clone, a SHA-256-verified Bun when none installed fits, both host profiles without Chromium, and a `~/.local/bin/bun` shim so fresh shells get the recorded Bun). 4. Any other `FAIL` row carries a `fix:` command and a reason code with a troubleshooting anchor; run the fix, then `--check` again. `SKIP codex-cli (optional; …)`, `SKIP browser (lazy; …)`, `SKIP pins (planning-only; …)`, `SKIP pins (helper runtime; …)` and `SKIP floor (…)` are not failures. `WARN pins … (node pin only; bun pins pass)` is not one either: the Node on PATH misses the project's pin while Bun is right; run its fix only when this machine will run the project's tests. diff --git a/docs/OPENCLAW.md b/docs/OPENCLAW.md index d376b873a..e73bab35a 100644 --- a/docs/OPENCLAW.md +++ b/docs/OPENCLAW.md @@ -188,6 +188,12 @@ with OpenClaw's tool names and pointing at the installed gstack, so the agent ru workflows and helpers itself. Use ACP when the work should happen in a Claude Code session with the full install (hooks, Claude tool names); use native skills when the OpenClaw agent should do it. Both can be installed together. +OpenClaw serves a registered skill's `SKILL.md` whole or refuses it, and its +limit is 32,768 bytes when it does not know the model's context window +(32,000 characters with a 100K-200K-token one), so a skill over 32,000 bytes (review, investigate, ship and plan-eng-review among the five) is +registered as a short entry that tells the agent to read the skill's full +`SKILL.md` in `gstack/skills/`, a plain copy of every skill that OpenClaw +reads in pages (`.gstack-skills/` under `--skills all`). Install, verify and remove: [hosts.md](hosts.md#agent-harnesses). The ACP artifacts in `openclaw/` are generated by `bun run gen:skill-docs --host openclaw`. diff --git a/docs/hosts.md b/docs/hosts.md index 8750cbbf1..1a18e0650 100644 --- a/docs/hosts.md +++ b/docs/hosts.md @@ -59,7 +59,7 @@ characters, the part of a description Hermes's skills index shows. | Harness | What gstack registers | Install | Verify | Remove | |---|---|---|---|---| -| OpenClaw | default `--skills hybrid`: `gstack-review`, `gstack-investigate`, `gstack-qa`, `gstack-ship` and `gstack-plan-eng-review` as top-level skills plus one `gstack` router skill whose body lists every other skill and its path, in `${OPENCLAW_STATE_DIR:-~/.openclaw}/skills`. With OpenClaw 2026.10.6's 19 eligible bundled skills, hybrid's skills section is 10,290 characters, full format, under its 18,000-character budget; `--skills all` (17,971, compact format, descriptions cut) lists every skill as its own folder, `--skills router` (6,701) only the router | `./setup --host openclaw`, or `./setup` inside OpenClaw | `openclaw skills list --json` shows `gstack` and the five task skills (source `openclaw-managed`, eligible) | `gstack-harness unregister openclaw` | +| OpenClaw | default `--skills hybrid`: `gstack-review`, `gstack-investigate`, `gstack-qa`, `gstack-ship` and `gstack-plan-eng-review` as top-level skills plus one `gstack` router skill whose body lists every other skill and its path, in `${OPENCLAW_STATE_DIR:-~/.openclaw}/skills`. With OpenClaw 2026.10.6's 19 eligible bundled skills, hybrid's skills section is 10,290 characters, full format, under its 18,000-character budget; `--skills all` (17,971, compact format, descriptions cut) lists every skill as its own folder, `--skills router` (6,701) only the router. OpenClaw serves a registered skill's `SKILL.md` whole or refuses it ("Skill instructions cannot be partially served"), at most 32,768 bytes when its model catalog has no context window for the model and 32,000 characters with a 100K-200K-token window, so any registered skill over 32,000 bytes (four of the five in hybrid, 26 of 63 in `all` today) is a short entry `SKILL.md` with the same name and description that sends the agent to its full `SKILL.md` in a plain copy of every skill (the router's `gstack/skills/`, or `.gstack-skills/` under `--skills all`, a folder OpenClaw's skill scan skips), which OpenClaw reads in pages. A context window under 100K tokens lowers that budget to 16,000 characters, which this limit does not cover | `./setup --host openclaw`, or `./setup` inside OpenClaw | `openclaw skills list --json` shows `gstack` and the five task skills (source `openclaw-managed`, eligible) | `gstack-harness unregister openclaw` | | Hermes | every skill, as `gstack-` in one `gstack` category, in `${HERMES_HOME:-~/.hermes}/skills/gstack`. Hermes's skills index lists only skills registered as folders, so a router would leave the rest out of the listing (`skill_view` still loads an unlisted skill by name, but nothing tells the model it exists) | `./setup --host hermes`, or `./setup` inside Hermes | `hermes skills list` shows the `gstack-*` skills in category `gstack` | `gstack-harness unregister hermes` | | dots (feder-cr) | the `gstack` router in `/home/dot/skills/gstack`, which the Dot lists every turn | `./setup` on the Dot's computer, or `gstack-harness register dots` | the Dot's Skills page, or `GET /skills` | `gstack-harness unregister dots` | | QM | **not available yet.** QM imports a skill pack by cloning the whole repository and caps it at 32 MiB and 5,000 files; gstack's repository is about 73 MiB, so `gstack-harness register qm --yes` stops before sending anything ([why](troubleshooting.md#harness-qm-pack-too-large)) | meanwhile, paste `gstack-harness register generic --print` into the agent's instructions | ask it for a gstack skill | delete the pasted text | diff --git a/docs/model-policy.md b/docs/model-policy.md index 5ae20d113..e3a6612c3 100644 --- a/docs/model-policy.md +++ b/docs/model-policy.md @@ -163,6 +163,13 @@ good, edit or delete that line; `gstack-config unset agent_model` clears this machine only. Copying a newer `capy/skill/gstack/` over the skill folder leaves `preferences.yaml` alone, because the shipped folder has no such file. +A Capy session with no gstack checkout (a Captain or coordinator that runs +`install.sh --check --for none`) can't run the resolver, so the skill tells it +to read the `agent_model` line itself. There a family means the newest model of +that family the host lists, while the resolver maps each family to the id in +its table (today they agree: `opus` is `anthropic/claude-opus-5-5`). Anything +it can't use means the session keeps its own model and says so. + ## Reasoning effort Codex calls carry a `model_reasoning_effort`. It follows the same shape as the diff --git a/docs/pregate.md b/docs/pregate.md index 303d5d7be..7e677900d 100644 --- a/docs/pregate.md +++ b/docs/pregate.md @@ -456,7 +456,8 @@ receipt for H when the two trees differ only as a stamp does: stamps, each hunk in them must be the exact old → new rewrite under `stamp_patterns` (literal `{version}` templates) or the built-in forms (`(was X)`, `since: X`, `vX`), and the version file, `package.json`'s - `version`, a lockfile's root version keys and a CHANGELOG re-heading have + `version`, a lockfile's root version keys and a CHANGELOG re-heading (under + `changelog_fragments`, also the fold of the fragments the head deletes) have their own built-in rules. There is no second list and no looser rule here: the file verdicts `clear` reaches on a pair are the verdicts `gstack-tree-receipt --gated G --current H` reaches, and its gate rule holds @@ -475,7 +476,9 @@ receipt for H when the two trees differ only as a stamp does: the policy label (`origin/@:.gstack/ship-policy.json`) and every differing path with the rule that classed it (`version-file`, `version-manifest`, `manifest-version`, `lockfile-root-version`, - `changelog-heading`, `stamp_files`, `stamp_paths`, `release_outputs`). `clear` prints the same + `changelog-heading`, `changelog-fragments` for the CHANGELOG under the fold, + `changelog_fragments` for a folded fragment's deletion, `stamp_files`, + `stamp_paths`, `release_outputs`). `clear` prints the same as `PREGATE_RECEIPT_SUBSTITUTED: …`, `verify` repeats it on its ok line, and the ship receipt carries the receipt string. The receipt's own tests and failed checks apply as before (its exit status is checked only when diff --git a/docs/ship-policy.md b/docs/ship-policy.md index 24047fa6b..d3cf4ba0d 100644 --- a/docs/ship-policy.md +++ b/docs/ship-policy.md @@ -61,6 +61,7 @@ refuses the file (`POLICY_GSTACK_TOO_OLD`). `validate` ends with | `min_gstack_version` | version or `null` | `null` | The oldest gstack that may read the file; see the compatibility rule above. | | `docs` | `{ caps: [{ path, max_bytes }] }` or `null` | `null` | Documentation files with a byte cap. `/ship` Step 14.5 runs `gstack-ship-receipt docs-caps --base ` (the caps come from `origin/`'s policy, never the branch's copy; `--policy ` is for a repository bootstrapping its policy) before the verification gate and prints one `DOCS_CAP: / ok\|over` line per cap; any `over` (an absent file counts) exits 1 and blocks the ship until the file is trimmed or the cap raised on the base branch (`DOCS_AUDIT_BLOCKED`). At most 64 caps, each path contained. See [the ship receipt](#ship-receipt). Needs `min_gstack_version`. | | `train` | `{ gate_workflow, free_workflow, full_gate_check, record_artifact, max_size, fill_hours, screen_selection, trial_reservation_usd }` or `null` | `null` | Turns on [merge trains](#run-a-merge-train), read from the base revision only. Every field is required and an unknown field is `POLICY_INVALID`: `max_size` 1 to 6, `fill_hours` above 0 and at most 24, `screen_selection` `gstack-paid` \| `none`, `trial_reservation_usd` above 0 and at most 100. `null` keeps the per-PR queue exactly as it is. Needs `min_gstack_version`. | +| `changelog_fragments` | `{ dir }` or `null` | `null` | PRs carry unversioned [CHANGELOG fragments](#changelog-fragments) in `dir` (one flat, repo-relative directory, no `..`) and the merge step ([`land`](#land)) assigns the version. `null` keeps `[Unreleased]` sections and per-PR stamps. Needs `min_gstack_version`. | Containment, checked by `validate` and again at write time: paths and globs are repo-relative with no `..` segment, under 256 characters, at most 64 per list, @@ -148,7 +149,8 @@ ship receipt into the coordinator protocol, so nobody proves a stamp diff by hand. Lanes own their PRs and never merge; the coordinator owns the order and every merge. Each step ends in a line the other side acts on, then `GSTACK_RESULT: skill=ship-queue status=complete|refused run=`. Nothing -here pushes. +here pushes, except `ready` under `changelog_fragments` (its READY receipt goes +to `refs/notes/gstack-trains`) and [`land`](#land). **1. Lane: report ready (unstamped).** After the gate and pregate pass on the pushed head: @@ -169,8 +171,12 @@ ci: green at (passed=31 failed=0 pending=0 skipped=2) PREGATE: pass tree= head= checks=6 gate: command="" full_e2e=no gate-ahead=none SHIP_RECEIPT: pr=42 head= … +READY_RECEIPT: /ship-queue/ready//42-.json (notes pushed to origin refs/notes/gstack-trains) ``` +Under `changelog_fragments`, a `READY_RECEIPT:` line follows, written only when +every check above passed; `land` refuses a pin without one. + CI's Version Gate (`scripts/compare-pr-version.ts`) accepts that head as unstamped when `VERSION` and `package.json`'s version match its merge base, the CHANGELOG change is confined to `[Unreleased]`, and any changed release output @@ -788,6 +794,7 @@ which carry the workflow consequences the skill prose would otherwise spell out: | `ship: preregistration: …` | `gstack-evidence ancestor --from-policy --base ` must print only `ANCESTOR: ok` before push | | `ship: release_tool: …` | the command runs only inside `gstack-restamp` (staging worktree, outputs confined to `release_outputs`) | | `ship: review_scope …` | Step 9.4 item 0 runs `gstack-review-scope classify` on each pass's findings; see [review_scope](#review-scope) | +| `ship: changelog_fragments : …` | Step 12 writes no version; Step 13 writes the entry as `/.md` (entry text only, `bump:` front matter only when the change warrants more than the policy bump), never an `[Unreleased]` section, linted with `gstack-prose-lint --gate`; `gstack-ship-queue land` assigns the version; see [changelog fragments](#changelog-fragments) | The PR body's first section is the ship receipt (`gstack-ship-receipt write … --body`), written by the pr-body section of `/ship` after the body is composed. @@ -1168,6 +1175,153 @@ Read from the base branch and run only with `--allow-repo-commands`, like of its own choosing cannot run here; give it a mode that takes the version and writes only stamps. +The command also gets `GSTACK_RESTAMP_BASE_SHA`, the release predecessor: the +base branch commit the version is allocated after (origin/ for a per-PR +restamp and for `land`). A repo counter that must be predecessor + 1 (a +SAVE_VERSION, a rules number) reads its old value from that commit +(`git show "$GSTACK_RESTAMP_BASE_SHA:"`), never from the branch, so two PRs +cut from one base never both claim the same number. Regenerated goldens go in +`release_outputs`. + + +## `changelog_fragments`: entries without versions, versioned at merge + +With `changelog_fragments: { "dir": "changelog.d" }` on the base branch, a PR +never edits VERSION, the manifest `version` key or a released CHANGELOG +section. It adds one or more fragments, and the merge step assigns the +version, so two PRs cut from one base cannot collide and no lane redoes a +stamp after another PR merges. + +A fragment is `/.md`, one regular file directly in `dir` +(`README.md` there is skipped). Its body is the entry text, exactly what the +CHANGELOG section will hold, with no `## [x]` heading. It may open with front +matter that raises the bump: + +```markdown +--- +bump: minor +--- +**What changed, in one sentence.** + +### Fixed +- ... +``` + +The effective bump of a stamp is the highest of the policy bump and every +folded fragment's `bump:` (`micro` < `patch` < `minor` < `major`). Fragments +are read as Git blobs at the commit being stamped (`git ls-tree` and +`git cat-file`, never the working tree), so a symlink, a gitlink, a nested +path, a non-`.md` file, an empty fragment, one over 64 KiB, one carrying a +release heading, or one whose name already landed on the base with other text +is `CHANGELOG_FRAGMENT_INVALID`, and nothing is allocated. Only fragments the +branch adds against the base count. + +**/ship.** `show-effective` prints a `ship: changelog_fragments` line, so /ship +writes no version in Step 12 and writes Step 13's entry as +`/.md`, linted with `gstack-prose-lint --gate`, never an +`[Unreleased]` section. + +**The fold.** `gstack-restamp` (and so `land`) concatenates the fragment +bodies in path order above any legacy `[Unreleased]` section, re-heads both as +one `## [] - ` section at the top, and deletes the fragment +files, in one stamp. The tree receipt, and so `gstack-pregate clear`'s +stamp-only substitution, calls a deleted fragment a stamp (rule +`changelog_fragments`) only when the new CHANGELOG section is exactly the +deleted fragments' bodies in path order above any legacy `[Unreleased]` text +(rule `changelog-fragments`); an edited fold, or a fragment deleted without +its text, is a change. + +**Version Gate.** The workflow runs on every PR (no path filter), so a PR +that touches no release file still has to prove its entry. Under the key, an +unstamped head must keep VERSION, the manifest `version` key and every +released CHANGELOG section at its merge base's values, and add at least one +valid fragment (a legacy `[Unreleased]`-only entry passes during migration; +a `[bot]` author, such as Dependabot, needs none). Other manifest edits, a +dependency or a script, pass. A head made by the merge step, a commit whose +parents are [pin, base], must carry the base's version plus the effective +bump of the fragments its pin adds. Under a train policy `gstack-ship-queue +ready` applies the same rule itself; without one, `ready` requires the head's +checks green, Version Gate included. + + +## `gstack-ship-queue land`: the per-PR merge step + +`land --pr ` merges one READY PR whose head is unstamped. It replaces the +lane's stamp round trip. Nobody stamps by hand; the tool makes and pushes the +stamp commit. Steps, each journaled in +`.gstack/tmp/ship-queue/land/.json` and resumable by rerunning the same +command: + +1. **Refuse** a fork head, a closed PR, a PR whose base is another branch, a + base policy with `changelog_fragments` off (`QUEUE_NOT_READY`), a pin + without a READY receipt (`gstack-ship-queue ready --pr ` records one + on `refs/notes/gstack-trains` once its pregate, full_e2e and receipt checks + pass; land checks nothing weaker), a /ship receipt that is missing or + bound to another head (`QUEUE_RECEIPT_MISSING`, `QUEUE_RECEIPT_STALE`), and + required checks that are not green at the pin. +2. **Lock.** Take the repository land lock, `refs/notes/gstack-land` (its own + ref, never the trains notes ref), by exact lease. It records the PR, holder, + phase and journal. A held lock is `QUEUE_LAND_LOCKED` naming the holder; + `--break-lock` takes over a crashed holder's lock and is refused while that + holder entered a push or a merge less than 10 minutes ago, and for a drift + lock. A refusal before the push releases the lock. +3. **Merge and stamp.** In a scratch worktree, merge the base into the pin. + Conflicts in release fields only (the version file and `release_outputs` + take the base's side, the CHANGELOG merges by section, a manifest or npm + lockfile is re-merged with every side's release version set to the base's, + so a clean dependency edit survives) are resolved; anything else is + `QUEUE_LAND_CONFLICT`, nothing pushed, back to the lane. A legacy pin that + bumped VERSION itself has its release fields put back to its merge base + first and its entry re-headed `[Unreleased]` (a manifest version source + keeps its other edits; a symlinked release file refuses). A PR with no entry gets a + fragment written from its title. Then `gstack-restamp --next` folds and + allocates (`restamp_command` and `release_tool` need + `--allow-repo-commands`, as everywhere). +4. **Push.** `commit-tree` the stamped tree with parents [pin, base] and push + it to the PR branch under `--force-with-lease=:`. The journal + keeps the reviewed pin apart from the last pushed head, so a rerun leases + on what it pushed and never on a head someone else pushed + (`QUEUE_TRAIN_MOVED`, never overwritten). +5. **Wait** for the PR's checks on the stamp commit, polling every 30 seconds + with the same check read the pin passed (`--timeout `, default 60). `--no-wait` + stops after the push and prints `gstack-pr-wait --until green && + gstack-ship-queue land --pr ` to run instead. + CI runs for real on the stamp commit in this mode. +6. **Merge.** Re-read the base first: if it moved, restamp from the new base + and push again under the last-pushed lease, at most twice + (`QUEUE_LAND_BASE_BUSY` on a third move, which releases the lock). Then + `gh pr merge --squash --match-head-commit ` (`--merge` under + `history: merge-only`) and confirm the base's + tree is the stamped tree on the base land stamped. The stamp date is taken + once and reused by every restamp. A rerun after GitHub merged the PR + (an error reply, a crash) goes straight to that tree check. +7. **Drift.** GitHub's merge API cannot bind the base, so a merge made outside + `land` can slip in between the check and the squash. The post-merge check + then fails with `QUEUE_LAND_DRIFT` and keeps the lock. + `land --pr --repair` opens a PR from the base whose only change is a + fragment for what the outside merge brought (its unfolded fragments, moved, + or one entry naming its commits) and records it on the lock; /ship and land + that PR, from any checkout, and its land takes over the drift lock and + releases it. A rerun of `--repair` finds the PR it already opened. + +`--dry-run` merges and stamps in a scratch worktree and prints the version and +tree, taking no lock and pushing nothing. + + +### Migrating to fragments + +Before turning the key on, run `gstack-ship-queue land --census` (read-only). +It prints one line per open PR, saying whether land would fold its fragments, re-head a legacy +`[Unreleased]`, rewrite a tentative per-PR VERSION bump, resolve a +release-field-only conflict, write an entry from the title, or refuse on a +product conflict. Then set `changelog_fragments` and `min_gstack_version` in +one policy PR; a claim-at-open repository switches `queue_mode` to +`stamp-at-merge` in the same PR. Open PRs need no change: `land` handles each +census case. Rollback is removing the key. With it off, `gstack-restamp` +neither folds nor deletes fragment files, so first land every open PR that +carries a fragment (`land --census` lists them) or move each fragment's text +under `[Unreleased]` in its PR; otherwise that entry never reaches CHANGELOG.md. + ## The tree receipt @@ -1191,7 +1345,8 @@ Hunks are classified per file type: the version file (old → new only), the manifest (`version` key only), npm and bun lockfiles (root `version` keys only, where `gstack-version-bump` writes; a dependency entry is `changed`), the CHANGELOG (a heading move or re-heading, compared as an order-independent -multiset of entries), `stamp_paths` and `release_outputs` (every +multiset of entries; under `changelog_fragments`, also exactly the deleted +fragments folded under the new heading, and those deletions are stamps), `stamp_paths` and `release_outputs` (every removed/added pair must be an exact `(was X)` / `since: X` / `vX` rewrite (a quote around X in the first two forms included) with X the gated version, or a newer version of the repo's width that the stamp_paths stage moves (a tentative `*_KEY_VERSION` written as the next diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index c9ad80aec..02ecad44a 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -4347,6 +4347,65 @@ reason and dispatches nothing. **Fix.** Run the printed `train build --prs `; the fork PR stays on the trusted-branch path. + +### `CHANGELOG_FRAGMENT_INVALID` + +**Meaning.** A CHANGELOG fragment under the policy's `changelog_fragments` +directory is not one regular `.md` file directly in that directory with entry +text, optional `bump:` front matter and no release heading, or it is over +64 KiB, or a fragment of that name already landed with other text. Nothing was +allocated or written. See [changelog fragments](ship-policy.md#changelog-fragments). + +**Fix.** Rewrite the named fragment as a flat `/.md` holding only +the entry text (and optional `bump: minor|major` front matter); rename it if +the name already landed. Then rerun. + + +### `QUEUE_LAND_LOCKED` + +**Meaning.** Another `gstack-ship-queue land` holds the repository land lock +(`refs/notes/gstack-land`), or took it from under this run, so this one would +race it for the next version. The line names the holder, its PR and phase. + +**Fix.** Wait for the holder to finish. If it crashed, rerun with +`--break-lock`; that is refused while the holder entered a push or a merge +less than 10 minutes ago. A lock in phase `drift` is released by landing its +repair PR ([QUEUE_LAND_DRIFT](#queue-land-drift)). + + +### `QUEUE_LAND_CONFLICT` + +**Meaning.** The PR conflicts with the base branch in a product file (or a +release field that cannot be resolved alone, such as a real dependency +conflict in a manifest), so the merge step cannot stamp it. Nothing was +pushed. + +**Fix.** The lane merges the base into the PR, resolves the named files, runs +/ship and reports READY again. + + +### `QUEUE_LAND_DRIFT` + +**Meaning.** After the squash merge, the base branch's tree is not the +stamped tree on the base `land` stamped: a merge made outside `land` slipped +in between the last check and the merge. The lock stays held, in phase +`drift`, so no other land allocates on a release record that is now wrong. + +**Fix.** Run `gstack-ship-queue land --pr --repair`. It opens a PR whose +only change is a fragment for what the outside merge brought. Run /ship on it +and land it; that land takes over the lock and releases it. Then find what +merged outside `land`. + + +### `QUEUE_LAND_BASE_BUSY` + +**Meaning.** The base branch moved during the stamp commit's CI on each of +the two allowed restamps, so `land` stopped instead of restamping again. + +**Fix.** Find what is merging outside `land` (every merge should go through +it), then rerun `gstack-ship-queue land --pr `; the lock was released and +the rerun restamps from the pin. + ### `QUEUE_TRAIN_MOVED` diff --git a/docs/uninstall.md b/docs/uninstall.md index efadb2984..a0309dd7a 100644 --- a/docs/uninstall.md +++ b/docs/uninstall.md @@ -49,7 +49,7 @@ rm -rf ~/.gstack rm -rf "${CODEX_HOME:-$HOME/.codex}/skills/gstack"* 2>/dev/null rm -rf ~/.factory/skills/gstack* 2>/dev/null rm -rf ~/.kiro/skills/gstack* 2>/dev/null -rm -rf "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/skills/gstack"* 2>/dev/null +rm -rf "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/skills/gstack"* "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/skills/.gstack-skills" 2>/dev/null rm -rf "${HERMES_HOME:-$HOME/.hermes}/skills/gstack" 2>/dev/null rm -rf ~/.cursor/skills/gstack* 2>/dev/null rm -rf ~/.config/opencode/skills/gstack* 2>/dev/null diff --git a/lib/changelog-fragments.ts b/lib/changelog-fragments.ts new file mode 100644 index 000000000..5fb0a1000 --- /dev/null +++ b/lib/changelog-fragments.ts @@ -0,0 +1,134 @@ +/** + * changelog-fragments — unversioned CHANGELOG fragments (docs/ship-policy.md#changelog-fragments). + * Under the policy key `changelog_fragments: { dir }` a PR adds `/.md` instead of an + * `[Unreleased]` section and leaves its release-version fields at its merge base's values; the + * merge step folds every fragment into the CHANGELOG under the version it allocates and deletes the + * fragment files. Fragments are read as Git blobs, never through the filesystem: only regular files + * (mode 100644) directly under `dir` count, so a symlink, gitlink or nested path cannot pull bytes + * from outside the repository into a stamp. + */ +import { spawnSync } from 'node:child_process'; +import { headingVersion, parseChangelog } from './changelog-check'; +import type { Bump } from './version-source'; + +export const FRAGMENT_README = 'README.md'; +export const MAX_FRAGMENT_BYTES = 64 * 1024; +const BUMP_RANK: Record = { micro: 0, patch: 1, minor: 2, major: 3 }; + +export interface Fragment { path: string; body: string; bump: Bump | null } +export interface FragmentError { code: 'CHANGELOG_FRAGMENT_INVALID'; path: string; message: string } +export interface FragmentSet { fragments: Fragment[]; errors: FragmentError[] } + +function git(cwd: string, args: string[]): { ok: boolean; out: string } { + const r = spawnSync('git', args, { cwd, encoding: 'utf8', timeout: 30_000, maxBuffer: 16 * 1024 * 1024 }); + return { ok: r.status === 0, out: r.stdout ?? '' }; +} + +/** The fragment file name /ship writes for a branch: one flat path segment. */ +export function fragmentSlug(branch: string): string { + const slug = branch.replace(/[\\/]+/g, '-').replace(/[^A-Za-z0-9._-]+/g, '-').replace(/^[-.]+|[-.]+$/g, '').slice(0, 100); + return slug || 'change'; +} + +/** One fragment's text: optional `---` front matter holding only `bump:`, then a non-empty body with no release heading. */ +export function parseFragment(rel: string, text: string): Fragment | FragmentError { + const bad = (message: string): FragmentError => ({ code: 'CHANGELOG_FRAGMENT_INVALID', path: rel, message }); + const normalized = text.replace(/\r\n/g, '\n'); + let body = normalized; + let bump: Bump | null = null; + const fm = /^---\n([\s\S]*?)\n---\n?/.exec(normalized); + if (fm) { + body = normalized.slice(fm[0].length); + for (const line of fm[1]!.split('\n').filter(l => l.trim())) { + const kv = /^\s*bump\s*:\s*['"]?(micro|patch|minor|major)['"]?\s*$/.exec(line); + if (!kv) return bad(`front matter may hold only "bump: micro|patch|minor|major", found ${JSON.stringify(line.trim().slice(0, 60))}`); + bump = kv[1] as Bump; + } + } + const lines = body.split('\n'); + while (lines.length && !lines[0]!.trim()) lines.shift(); + while (lines.length && !lines[lines.length - 1]!.trim()) lines.pop(); + if (!lines.length) return bad('the fragment has no entry text'); + const heading = lines.find(l => /^##\s/.test(l) && headingVersion(l)); + if (heading) return bad(`a fragment carries no release heading (the merge step writes it), found ${JSON.stringify(heading.trim())}`); + return { path: rel, body: lines.join('\n'), bump }; +} + +/** + * The fragments in `dir` of a tree-ish (a commit or tree id), or of the index when `treeish` is + * null. `exclude` names a tree-ish whose fragments are not this change's own (the base branch, or a + * stacked pin's parent): a path present there with the same blob is skipped, a path present there + * with other bytes is an error. With `changelog`, a fragment whose lines appear whole and in order in a released + * section of that file at `exclude` is an error too: a landed PR's fragment carried back in by stacked history. + */ +export function readFragments(cwd: string, treeish: string | null, dir: string, exclude?: string | null, changelog?: string | null): FragmentSet { + const prefix = dir.replace(/\/+$/, '') + '/'; + const list = (t: string | null) => { + const r = t === null ? git(cwd, ['ls-files', '-s', '-z', '--', prefix]) : git(cwd, ['ls-tree', '-r', '-z', '--full-tree', t, '--', prefix]); + const out: Array<{ mode: string; sha: string; path: string }> = []; + for (const rec of r.out.split('\0').filter(Boolean)) { + const tab = rec.indexOf('\t'); + const meta = rec.slice(0, tab).split(/\s+/); + out.push(t === null ? { mode: meta[0]!, sha: meta[1]!, path: rec.slice(tab + 1) } : { mode: meta[0]!, sha: meta[2]!, path: rec.slice(tab + 1) }); + } + return out; + }; + const excluded = new Map((exclude ? list(exclude) : []).map(e => [e.path, e.sha])); + const shown = exclude && changelog ? git(cwd, ['show', `${exclude}:${changelog}`]) : null; + const released = shown?.ok ? parseChangelog(shown.out.replace(/\r\n/g, '\n')).sections.filter(s => !s.unreleased).map(s => s.lines) : []; + const isReleased = (body: string) => { + const want = body.split('\n'); + return released.some(lines => lines.some((_, i) => want.every((w, k) => lines[i + k] === w))); + }; + const fragments: Fragment[] = []; + const errors: FragmentError[] = []; + for (const e of list(treeish).sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0))) { + const rest = e.path.slice(prefix.length); + if (rest === FRAGMENT_README) continue; + const err = (message: string) => errors.push({ code: 'CHANGELOG_FRAGMENT_INVALID', path: e.path, message }); + if (rest.includes('/')) { err(`fragments live directly in ${prefix} (one flat file each), not in a subdirectory`); continue; } + if (e.mode !== '100644') { err(`must be a regular file (mode 100644), found mode ${e.mode}${e.mode === '120000' ? ' (symlink)' : e.mode === '160000' ? ' (gitlink)' : ''}`); continue; } + if (!rest.endsWith('.md')) { err('a fragment is a .md file'); continue; } + const prior = excluded.get(e.path); + if (prior === e.sha) continue; + if (prior !== undefined) { err('a fragment of this name already exists on the base with other text; fragments are never edited after they land'); continue; } + const size = git(cwd, ['cat-file', '-s', e.sha]); + if (!size.ok || Number(size.out.trim()) > MAX_FRAGMENT_BYTES) { err(`larger than ${MAX_FRAGMENT_BYTES} bytes or unreadable`); continue; } + const blob = git(cwd, ['cat-file', 'blob', e.sha]); + if (!blob.ok) { err('blob unreadable'); continue; } + const parsed = parseFragment(e.path, blob.out); + if ('code' in parsed) errors.push(parsed); + else if (isReleased(parsed.body)) err(`its text is already released in ${changelog} on the base (a landed PR's fragment carried in by stacked history); delete it from this branch`); + else fragments.push(parsed); + } + return { fragments, errors }; +} + +/** The highest of the policy bump and every fragment's declared bump. */ +export function effectiveBump(level: Bump, fragments: readonly Fragment[]): Bump { + return fragments.reduce((acc, f) => (f.bump && BUMP_RANK[f.bump] > BUMP_RANK[acc] ? f.bump : acc), level); +} + +/** + * Put the fragment bodies, in path order, into the CHANGELOG's `[Unreleased]` section (created at the + * top when absent), so the existing re-heading moves fragments and a legacy entry together under one + * version. Returns the text unchanged when there are no fragments. + */ +export function foldIntoUnreleased(text: string, fragments: readonly Fragment[]): string { + if (!fragments.length) return text; + const parsed = parseChangelog(text); + const bodies = fragments.map(f => f.body).join('\n\n'); + const unreleased = parsed.sections.find(s => s.unreleased); + const lines = text.replace(/\r\n/g, '\n').split('\n'); + if (unreleased) { + const own = lines.slice(unreleased.start + 1, unreleased.end); + while (own.length && !own[0]!.trim()) own.shift(); + while (own.length && !own[own.length - 1]!.trim()) own.pop(); + const merged = [lines[unreleased.start]!, '', ...bodies.split('\n'), ...(own.length ? ['', ...own] : []), '']; + return [...lines.slice(0, unreleased.start), ...merged, ...lines.slice(unreleased.end)].join('\n'); + } + const at = parsed.sections[0]?.start ?? lines.length; + const head = lines.slice(0, at); + while (head.length && !head[head.length - 1]!.trim()) head.pop(); + return [...head, ...(head.length ? [''] : []), '## [Unreleased]', '', ...bodies.split('\n'), '', ...lines.slice(at)].join('\n'); +} diff --git a/lib/harness-register.ts b/lib/harness-register.ts index bfc84a047..8ac054d77 100644 --- a/lib/harness-register.ts +++ b/lib/harness-register.ts @@ -37,6 +37,19 @@ export const DOT_SKILLS_DIR = '/home/dot/skills'; export const ROUTER_DESCRIPTION = 'Use to review a diff or PR, debug a bug or failing test, QA test a site, plan or review architecture, ship a PR, audit security or write release docs. Load first for any coding task: lists every gstack skill and where to read it.'; /** OpenClaw hybrid mode: these task skills are listed as skills of their own; the router carries the rest. */ export const HYBRID_SKILLS = ['review', 'investigate', 'qa', 'ship', 'plan-eng-review']; +/** + * OpenClaw serves a registered skill's SKILL.md whole or refuses it ("Skill instructions cannot be partially + * served"), with maxBytes from resolveAdaptiveReadMaxBytes: 32,768 (DEFAULT_READ_PAGE_MAX_BYTES) when the model's + * context window is unknown (OpenClaw 2026.10.6 dist/agent-tools.read-BIxUGJy-.mjs:168, :189-194, :747, :798, + * :819-821). A known window also applies the tool-result budget to that read (resolveToolResultBudget, + * dist/tool-result-limits-DNBSu_jg.mjs:61-95): 16,000 characters under 100K tokens, 32,000 to 200K, 64,000 above. + * 32,000 bytes fits the unknown-window limit and a 100K-200K window; windows under 100K tokens are not covered. The + * post-delivery cap defaults to a 200K window (64,000 characters). A larger skill is registered as a short + * entry that points at its full SKILL.md in a plain tree of every skill, which OpenClaw reads in pages. + */ +export const OPENCLAW_INSTRUCTION_MAX = 32_000; +/** The plain tree under --skills all: a dot folder, which OpenClaw's skill scan skips (dist/skill-root-discovery-C2QVZC77.mjs:44). Hybrid and router use the router's own gstack/skills, which the scan never enters because gstack/ holds a SKILL.md (:343). */ +export const OPENCLAW_ALL_TEXTS = '.gstack-skills'; export interface Target { name: TargetName; @@ -46,6 +59,8 @@ export interface Target { render: 'host' | 'claude' | null; defaultMode: SkillsMode; modes: SkillsMode[]; + /** Largest SKILL.md the harness serves whole; a larger skill is registered as an entry pointing at its full text. */ + instructionMax?: number; /** Destination folder for this target, from its environment, or null when it has no default. */ dest: (env: Env, home: string) => string | null; /** Running inside this harness: a session marker or the harness among this process's ancestors. */ @@ -65,7 +80,7 @@ const ancestorIs = (ancestry: string[], name: string) => ancestry.some(id => { export const TARGETS: Record = { openclaw: { - name: 'openclaw', displayName: 'OpenClaw', mechanism: 'skill-folder', render: 'host', defaultMode: 'hybrid', modes: ['router', 'hybrid', 'all'], + name: 'openclaw', displayName: 'OpenClaw', mechanism: 'skill-folder', render: 'host', defaultMode: 'hybrid', modes: ['router', 'hybrid', 'all'], instructionMax: OPENCLAW_INSTRUCTION_MAX, dest: (env, home) => path.join(env.OPENCLAW_STATE_DIR || path.join(home, '.openclaw'), 'skills'), inside: (env, anc) => env.OPENCLAW_SHELL === 'exec' || ancestorIs(anc, 'openclaw'), present: (env, home) => onPath('openclaw', env) || fs.existsSync(env.OPENCLAW_STATE_DIR || path.join(home, '.openclaw')), @@ -239,6 +254,19 @@ export function routerSkill(skills: Array<{ name: string; use: string; path: str ].join('\n'); } +/** A registered skill whose full text is over the harness's whole-document read limit: its own frontmatter, and a body that sends the agent to the full text. */ +export function entrySkill(md: string, fullPath: string, target: Target): string { + const end = md.indexOf('\n---', 4); + return [ + md.slice(0, md.indexOf('\n', end + 4) + 1) + '', + '# ' + frontmatterField(md, 'name'), '', + 'This skill\'s full instructions (' + Buffer.byteLength(md) + ' bytes) are larger than ' + target.displayName + ' serves as a skill file, so they are in a plain file.' + + ' Read `' + fullPath + '` in full now (every page, if the read is split) and follow it as this skill\'s instructions.' + + ' Paths it gives relative to the installed SKILL.md directory resolve from that file\'s folder.', + '', + ].join('\n'); +} + /** The marked context block for the registration's mode: router (a router path), hybrid (a router path and direct skills) or all (no router). */ export function contextBlock(routerPath: string | null, direct: string[] = []): string { const lead = "gstack's engineering workflows (code review, QA, debugging, security audit, plan review, shipping) are installed here. "; @@ -361,21 +389,26 @@ export function planPayload(opts: RegisterOptions, dest: string, mode: SkillsMod files.set(path.join(to, 'SKILL.md'), Buffer.from(flatFrontmatter(fs.readFileSync(path.join(s.dir, 'SKILL.md'), 'utf8'), path.basename(to)))); if (fs.existsSync(path.join(s.dir, 'sections'))) copyDir(path.join(s.dir, 'sections'), path.join(to, 'sections')); }; + const routerDir = path.join(dest, 'gstack'); + const plain = path.join(...(mode === 'all' ? [dest, OPENCLAW_ALL_TEXTS] : [routerDir, 'skills'])); const listed = (s: SkillEntry) => { const to = path.join(dest, path.basename(s.dir)); - if (t.render === 'host') copyDir(s.dir, to); else claudeFiles(s, to); + const md = fs.readFileSync(path.join(s.dir, 'SKILL.md')); + if (t.instructionMax && md.length > t.instructionMax) files.set(path.join(to, 'SKILL.md'), Buffer.from(entrySkill(md.toString('utf8'), path.join(plain, path.basename(s.dir), 'SKILL.md'), t))); + else if (t.render === 'host') copyDir(s.dir, to); else claudeFiles(s, to); return t.render === 'host' ? s.name : path.basename(s.dir); }; if (mode === 'all') { const names = skills.map(listed); + if (t.instructionMax) for (const s of skills) copyDir(s.dir, path.join(plain, path.basename(s.dir))); return { files, router: null, visible: names, direct: [], skills: names.length }; } const isDirect = (s: SkillEntry) => mode === 'hybrid' && HYBRID_SKILLS.includes(path.basename(s.dir).replace(/^gstack-/, '')); const direct = skills.filter(isDirect).map(listed); - const routerDir = path.join(dest, 'gstack'); + if (t.instructionMax) for (const s of skills.filter(isDirect)) copyDir(s.dir, path.join(plain, path.basename(s.dir))); const entries = skills.filter(s => !isDirect(s)).map(s => { if (t.render !== 'host') return { name: s.name, use: s.use, path: path.join(s.dir, 'SKILL.md') }; - const to = path.join(routerDir, 'skills', path.basename(s.dir)); + const to = path.join(plain, path.basename(s.dir)); copyDir(s.dir, to); return { name: s.name, use: s.use, path: path.join(to, 'SKILL.md') }; }); diff --git a/lib/pregate/remote-receipt.ts b/lib/pregate/remote-receipt.ts index 701bcbdf4..1d5fa7810 100644 --- a/lib/pregate/remote-receipt.ts +++ b/lib/pregate/remote-receipt.ts @@ -12,7 +12,7 @@ * `substituteStampOnlyTree` is the one accepted substitution. The gated tree * and the head tree are diffed path by path, and every differing file is * classified by the tree receipt's own rule (lib/tree-receipt.ts - * `classifyFile`: the repo's `stamp_files` allowlist and `stamp_patterns` + * `classifyFiles`, the changelog_fragments fold included: the repo's `stamp_files` allowlist and `stamp_patterns` * templates from `.gstack/ship-policy.json` on the base branch, beside the * built-in version-file, manifest-version, lockfile-root-version and * changelog-heading forms). The receipt is accepted for the head only when @@ -33,7 +33,7 @@ import { spawnSync } from 'node:child_process'; import type { PregateReceiptSubstitution } from '../headless-artifacts'; import { parseUnifiedDiff, versionAtRef } from '../restamp'; import { isUnavailable, loadPolicyFromBase, SHIP_POLICY_PATH } from '../ship-policy'; -import { classifyFile } from '../tree-receipt'; +import { classifyFiles } from '../tree-receipt'; import { treeMismatch } from './checks/lanes'; /** What `clear` records when a remote_gate receipt from a stamp-only predecessor tree is accepted for the head (the pregate.json shape, lib/headless-artifacts.ts). */ @@ -103,7 +103,7 @@ export function substituteStampOnlyTree(repoRoot: string, base: string, gatedTre if (diff === null) return { problem: `${mismatch}; git diff ${short(gatedTree)} ${short(headTree)} failed`, fix: rerun }; const inventory = git(repoRoot, ['diff-tree', '-r', '-z', '--name-only', '--no-renames', '--ignore-submodules=none', '--end-of-options', gatedTree, headTree]); if (inventory === null) return { problem: `${mismatch}; git diff-tree ${short(gatedTree)} ${short(headTree)} failed`, fix: rerun }; - const files = parseUnifiedDiff(diff).map(f => classifyFile(repoRoot, f, gatedTree, headTree, loaded.policy, versionRel, oldV, newV)); + const files = classifyFiles(repoRoot, parseUnifiedDiff(diff), gatedTree, headTree, loaded.policy, versionRel, oldV, newV); if (!files.length) return { problem: `${mismatch}; the trees differ but git diff produced no file entry to classify`, fix: rerun }; const expected = new Set(inventory.split('\0').filter(Boolean)); const seen = new Set(files.map(f => f.path)); diff --git a/lib/restamp.ts b/lib/restamp.ts index 1a4d18e23..69e2152a9 100644 --- a/lib/restamp.ts +++ b/lib/restamp.ts @@ -26,6 +26,7 @@ import * as os from 'node:os'; import * as path from 'node:path'; import { bunIsolationArgs } from './bun-isolation'; import { checkChangelog, mergeChangelogs, parseChangelog, reheadChangelog } from './changelog-check'; +import { effectiveBump, foldIntoUnreleased, readFragments, type FragmentSet } from './changelog-fragments'; import { type ResultCodeName } from './result-codes'; import { runScriptSync, wtreeFingerprint } from './script-invocation'; import { @@ -446,7 +447,7 @@ function linkIgnoredDeps(repoRoot: string, staging: string): void { */ function runRestampCommand(command: string, plan: Plan, staging: string, env: NodeJS.ProcessEnv, before: Set, s: StageRecord): void { const seeded = git(staging, ['write-tree']).out; - const r = spawnSync('sh', ['-c', `${command} "$1"`, 'sh', plan.newVersion], { cwd: staging, encoding: 'utf8', timeout: 600_000, env: { ...env, GSTACK_RESTAMP_VERSION: plan.newVersion } }); + const r = spawnSync('sh', ['-c', `${command} "$1"`, 'sh', plan.newVersion], { cwd: staging, encoding: 'utf8', timeout: 600_000, env: { ...env, GSTACK_RESTAMP_VERSION: plan.newVersion, GSTACK_RESTAMP_BASE_SHA: plan.baseSha } }); git(staging, ['read-tree', seeded]); if (r.status !== 0) { s.status = 'failed'; s.detail = `${r.stderr ?? ''}${r.stdout ?? ''}`.trim().slice(0, 300); throw new Refusal('RESTAMP_RELEASE_TOOL_FAILED', `restamp_command exited ${r.status}: ${s.detail}; nothing written to the branch`); } const outside = changedPaths(staging).filter(p => !before.has(p) && !allowedOutput(plan, staging, p)); @@ -491,7 +492,11 @@ function runStages(o: RestampOptions, plan: Plan, staging: string, since: string if (clPath && fs.existsSync(clPath)) { const date = (o.now ?? new Date()).toISOString().slice(0, 10); const from = o.was ?? plan.branchStamp; - const r = reheadChangelog(fs.readFileSync(clPath, 'utf8'), { from, to: plan.newVersion, date, file: clRel! }); + const dir = plan.policy.changelog_fragments?.dir; + const frags = dir ? readFragments(staging, null, dir, plan.baseSha, clRel) : null; + if (frags?.errors.length) { cl.status = 'failed'; cl.detail = `${frags.errors[0]!.path}: ${frags.errors[0]!.message}`; throw new Refusal('CHANGELOG_FRAGMENT_INVALID', `${cl.detail}; nothing written to the branch`); } + for (const f of frags?.fragments ?? []) fs.rmSync(path.join(staging, f.path)); + const r = reheadChangelog(foldIntoUnreleased(fs.readFileSync(clPath, 'utf8'), frags?.fragments ?? []), { from, to: plan.newVersion, date, file: clRel! }); if (r.error) { cl.status = 'failed'; cl.detail = r.error.message; throw new Refusal(r.error.code, `${clRel}: ${r.error.message}; nothing written to the branch`); } if (r.moved) fs.writeFileSync(clPath, r.text); // Inherited duplicates are allowed at the larger count of two base readings: the base tip and @@ -506,7 +511,7 @@ function runStages(o: RestampOptions, plan: Plan, staging: string, since: string : `; base ${clRel} unreadable at ${unreadable.join(', ')}, inherited headings counted at ${readable.map(x => x.sha.slice(0, 12)).join(', ')} only`; const check = checkChangelog(fs.readFileSync(clPath, 'utf8'), plan.newVersion, clRel!, readable.map(x => x.cl.out + '\n')); if (check.length) { cl.status = 'failed'; cl.detail = check[0]!.message + baseNote; throw new Refusal(check[0]!.code, `${check[0]!.path}: ${check[0]!.message}${baseNote}; nothing written to the branch`); } - cl.status = 'done'; cl.detail = (r.moved ? `${r.from} -> ${r.to}` : `already headed ${plan.newVersion}`) + baseNote; + cl.status = 'done'; cl.detail = (r.moved ? `${r.from} -> ${r.to}` : `already headed ${plan.newVersion}`) + (frags?.fragments.length ? `; folded ${frags.fragments.map(f => f.path).join(', ')}` : '') + baseNote; } else { cl.status = 'skipped'; cl.detail = clRel ? `${clRel} absent` : 'changelog: null'; } injected('changelog'); @@ -646,6 +651,15 @@ function applyToBranch(o: RestampOptions, staging: string, wtree: string, files: return writes; } +/** The fragments this change adds (absent from origin/), read from the working-tree fingerprint the staging worktree is seeded from. */ +function fragmentsOf(o: RestampOptions, dir: string, changelog: string | null): FragmentSet { + const fp = wtreeFingerprint({ cwd: o.repoRoot }); + if ('error' in fp) throw new Refusal('RESTAMP_CONFLICT', `cannot fingerprint the working tree (${fp.error}); nothing written`); + const set = readFragments(o.repoRoot, fp.tree, dir, `origin/${o.base}`, changelog); + if (set.errors.length) throw new Refusal('CHANGELOG_FRAGMENT_INVALID', `${set.errors[0]!.path}: ${set.errors[0]!.message}; nothing written`); + return set; +} + function removeWorktree(repoRoot: string, dir: string): void { git(repoRoot, ['worktree', 'remove', '--force', dir]); fs.rmSync(dir, { recursive: true, force: true }); @@ -662,7 +676,9 @@ export function restamp(o: RestampOptions): RestampResult { if (commands.length && !o.allowRepoCommands) { throw new Refusal('REPO_COMMANDS_NOT_ALLOWED', `repo commands not executed (pass --allow-repo-commands): ${commands.map(c => `${c.key}: ${c.command}`).join('; ')}; nothing written`, { commands }); } - const plan = planVersions(o, loaded, label); + const fragDir = loaded.policy.changelog_fragments?.dir ?? null; + const frags = fragDir ? fragmentsOf(o, fragDir, loaded.policy.changelog) : null; + const plan = planVersions(frags && !o.bump ? { ...o, bump: effectiveBump(loaded.policy.bump === 'patch' ? 'patch' : 'micro', frags.fragments) } : o, loaded, label); const headSha = git(o.repoRoot, ['rev-parse', 'HEAD']).out; const common: Partial = { oldVersion: plan.oldVersion, newVersion: plan.newVersion, base: { ref: plan.baseRef, sha: plan.baseSha }, policy: plan.policyLabel, diff --git a/lib/result-codes-release.ts b/lib/result-codes-release.ts index ee4004adb..e9551f80a 100644 --- a/lib/result-codes-release.ts +++ b/lib/result-codes-release.ts @@ -299,6 +299,32 @@ export const RELEASE_RESULT_CODES = { summary: 'gstack-ship-receipt carry needs git 2.39 or later for `git patch-id --verbatim`; the installed git is older, so the product diff could not be compared', fix: 'upgrade git to 2.39 or later (the refusal names the installed version), then rerun carry; otherwise run /ship on the new head', }, + // Changelog fragments and the per-PR merge step (lib/changelog-fragments.ts, lib/ship-queue-land.ts). + CHANGELOG_FRAGMENT_INVALID: { + anchor: 'changelog-fragment-invalid', + summary: 'a CHANGELOG fragment is not one regular .md file directly in the fragment directory with entry text, optional bump front matter and no release heading, or one of that name already landed with other text', + fix: 'rewrite the named fragment as a flat /.md with only the entry text (and optional "bump: minor|major" front matter), rename it if the name already landed, then rerun', + }, + QUEUE_LAND_LOCKED: { + anchor: 'queue-land-locked', + summary: 'another gstack-ship-queue land holds the repository land lock (refs/notes/gstack-land), so this one would race it for the next version', + fix: 'wait for the holder named in the line to finish; if it crashed, gstack-ship-queue land --pr --break-lock (refused while its journal shows a push in flight)', + }, + QUEUE_LAND_CONFLICT: { + anchor: 'queue-land-conflict', + summary: 'the PR conflicts with the base branch in product files (or a release field cannot be resolved alone), so the merge step cannot stamp it', + fix: 'the lane merges the base branch into the PR, resolves the named files, runs /ship and reports READY again', + }, + QUEUE_LAND_DRIFT: { + anchor: 'queue-land-drift', + summary: 'the base branch moved between the merge step\'s last check and the squash merge, so main\'s tree is not the stamped tree land verified', + fix: 'gstack-ship-queue land --pr --repair opens a PR whose only change is a fragment for what the outside merge brought; /ship and land it, which releases the lock', + }, + QUEUE_LAND_BASE_BUSY: { + anchor: 'queue-land-base-busy', + summary: 'the base branch moved during the stamp commit\'s CI on each of the allowed restamp attempts, so land stopped instead of restamping again', + fix: 'find what is merging outside land (every merge should go through it), then rerun gstack-ship-queue land --pr ; it resumes from its journal', + }, // gstack-pr-wait (lib/pr-wait.ts): the landing-path PR watcher's two terminal codes. PR_WAIT_BLOCKED: { anchor: 'pr-wait-blocked', diff --git a/lib/ship-policy.ts b/lib/ship-policy.ts index 8575cbdbd..62ece4e94 100644 --- a/lib/ship-policy.ts +++ b/lib/ship-policy.ts @@ -41,6 +41,8 @@ export interface ShipPolicy { stamp_paths: string[]; /** The CHANGELOG re-headed on a restamp; null for repos without one. */ changelog: string | null; + /** Unversioned CHANGELOG fragments (`/.md`) a PR adds instead of an `[Unreleased]` section; null keeps `[Unreleased]`. */ + changelog_fragments: { dir: string } | null; /** Path of a paired repo to restamp identically (tier 2). */ mirror: string | null; /** `merge-only` refuses a squash or rebase plan in /ship. */ @@ -102,6 +104,7 @@ export const DEFAULT_POLICY: ShipPolicy = { release_outputs: [], stamp_paths: [], changelog: 'CHANGELOG.md', + changelog_fragments: null, mirror: null, history: 'squash-ok', preregistration_shas: [], @@ -131,6 +134,7 @@ export const POLICY_EXPLANATIONS: Record = { release_outputs: 'Globs the release tool may write beyond the version files, lockfiles, CHANGELOG and stamp_paths. Any other output refuses the restamp.', stamp_paths: 'Globs (repo-relative, no `..`) where exact `(was X)`, `since: X` and `vX` mentions of the old version are rewritten.', changelog: 'CHANGELOG path re-headed on a restamp (the PR’s `[Unreleased]` or old-version section moves to the top under the new version). null to skip.', + changelog_fragments: '{ dir } or null: a PR adds one unversioned `/.md` fragment (optional `bump: minor|major` front matter) instead of an `[Unreleased]` section and leaves VERSION, the manifest version and released CHANGELOG sections at its merge base; the restamp folds every fragment under the allocated version and deletes the files (docs/ship-policy.md#changelog-fragments). null keeps `[Unreleased]`.', mirror: 'Relative path of a paired repo to restamp identically; writes are confined to it. null to skip.', history: 'merge-only | squash-ok. merge-only makes /ship refuse a squash or rebase plan.', preregistration_shas: 'Commits that must be ancestors of HEAD before push (preregistered evals); checked by `gstack-evidence ancestor`.', @@ -176,12 +180,14 @@ export const TRAIN_KEYS_VERSION = '1.91.146.0'; // its key, so a value a stamp did not move fails there. export const REVIEW_SCOPE_KEY_VERSION = '1.91.256.0'; export const DOCS_CAPS_KEY_VERSION = '1.91.270.0'; +export const CHANGELOG_FRAGMENTS_KEY_VERSION = '1.92.0.0'; export const POLICY_KEY_INTRODUCED: Record = { bump: '1.91.73.0', queue_mode: '1.91.73.0', release_tool: '1.91.73.0', release_outputs: '1.91.73.0', stamp_paths: '1.91.73.0', changelog: '1.91.73.0', mirror: '1.91.73.0', history: '1.91.73.0', preregistration_shas: '1.91.73.0', stamp_sensitive_gates: '1.91.73.0', pr_count: '1.91.73.0', maintainers: '1.91.73.0', stamp_files: '1.91.98.0', stamp_patterns: '1.91.98.0', restamp_command: '1.91.98.0', migrations: '1.91.98.0', gate_command: SHIP_QUEUE_KEYS_VERSION, full_e2e: SHIP_QUEUE_KEYS_VERSION, min_gstack_version: SHIP_QUEUE_KEYS_VERSION, train: TRAIN_KEYS_VERSION, review_scope: REVIEW_SCOPE_KEY_VERSION, docs: DOCS_CAPS_KEY_VERSION, + changelog_fragments: CHANGELOG_FRAGMENTS_KEY_VERSION, }; const newer = (a: string, b: string): boolean => cmpVersion(parseVersion(a)!, parseVersion(b)!) > 0; @@ -360,6 +366,7 @@ export function validatePolicy(raw: unknown, opts: { gstackVersion?: string } = if ('full_e2e' in obj && typeof obj.full_e2e !== 'boolean') errors.push({ code: 'POLICY_INVALID', path: 'full_e2e', message: 'must be true or false' }); if ('train' in obj && obj.train !== null) trainErrors(obj.train, errors); if ('docs' in obj && obj.docs !== null) docsErrors(obj.docs, errors); + if ('changelog_fragments' in obj && obj.changelog_fragments !== null) fragmentsErrors(obj.changelog_fragments, errors); if ('stamp_paths' in obj) listErrors('stamp_paths', obj.stamp_paths, errors, globError); for (const key of ['changelog', 'mirror'] as const) { if (!(key in obj) || obj[key] === null) continue; @@ -441,6 +448,14 @@ function docsErrors(raw: unknown, errors: PolicyError[]): void { }); } +function fragmentsErrors(raw: unknown, errors: PolicyError[]): void { + const d = raw as Record | null; + if (!d || typeof d !== 'object' || Array.isArray(d)) { errors.push({ code: 'POLICY_INVALID', path: 'changelog_fragments', message: 'must be { dir } or null' }); return; } + for (const k of Object.keys(d)) if (k !== 'dir') errors.push({ code: 'POLICY_INVALID', path: `changelog_fragments.${k}`, message: 'unknown key' }); + const why = typeof d.dir === 'string' ? containedRelative(d.dir) ?? (/[*?[\]{}]/.test(d.dir) || d.dir.replace(/\/+$/, '') === '.' ? 'must be one directory, not a glob or the repository root' : null) : 'must be a string'; + if (why) errors.push({ code: 'POLICY_CONTAINMENT', path: 'changelog_fragments.dir', message: `${JSON.stringify(d.dir)} ${why}` }); +} + /** Merge a parsed file over the defaults, recording per-key provenance. */ export function mergePolicy(raw: Record | null, sourceLabel: string): { policy: ShipPolicy; values: Record } { const policy: ShipPolicy = JSON.parse(JSON.stringify(DEFAULT_POLICY)); @@ -556,6 +571,8 @@ export function shipConsequences(policy: ShipPolicy, base: string): string[] { out.push(`shared merge queue: when a coordinator runs the queue, \`gstack-ship-queue ready --pr \`, \`gate-ahead --after --pr \` and \`stamp --pr --version --expect-gated --expect-base \` replace the calls above; stamp only when your slot is called, send the READY block and the QUEUE_STAMP: line; the coordinator runs \`gstack-ship-queue verify\` (docs/ship-policy.md#run-a-shared-merge-queue)${policy.gate_command ? `; gate command: \`${policy.gate_command}\`` : ''}`); out.push('gate-ahead: with .gstack/tmp/gate-ahead.json present, Step 16 reuses only the lanes `gstack-tree-receipt --from .gstack/tmp/gate-ahead.json --predecessor-pr --bundle --json > .gstack/tmp/tree-receipt.json` marks reusable; stamp-sensitive and not-eligible lanes run live; `--force-gate ` is the coordinator\'s call'); } else out.push('queue_mode claim-at-open: Step 12 runs as written (claims through gstack-next-version)'); + const frag = policy.changelog_fragments?.dir.replace(/\/+$/, ''); + if (frag) out.push(`changelog_fragments ${frag}: Step 12 writes no version and Step 13 writes the entry as ${frag}/.md (the entry text only, no heading; \`bump: minor\` or \`bump: major\` front matter only when the change warrants more than the policy bump), never an [Unreleased] section; lint it with \`gstack-prose-lint ${frag}/.md --gate\`; \`gstack-ship-queue land\` assigns the version at merge (docs/ship-policy.md#changelog-fragments)`); out.push(policy.bump === 'patch' ? 'bump patch: Step 12 never asks for MINOR or MAJOR; PATCH or MICRO by size' : 'bump ask: Step 12 asks for MINOR and MAJOR as written'); out.push(policy.review_scope === 'all' ? 'review_scope all: Step 9.4 fixes every finding on every pass (gstack-review-scope classify --policy all)' diff --git a/lib/ship-queue-land.ts b/lib/ship-queue-land.ts new file mode 100644 index 000000000..63e04aad8 --- /dev/null +++ b/lib/ship-queue-land.ts @@ -0,0 +1,464 @@ +/** + * ship-queue-land — the per-PR merge step (docs/ship-policy.md#land), behind + * `gstack-ship-queue land --pr `. The PR carries an unversioned change (a + * changelog fragment, or a legacy `[Unreleased]` section) at a pin /ship + * reviewed; land takes the repository land lock, merges the pin with the base + * branch (resolving release-version fields only), stamps the merged tree with + * gstack-restamp, makes one commit whose parents are [pin, base] and whose tree + * is the stamped tree, pushes it to the PR branch under an exact lease, waits + * for the PR's checks there, merges with --match-head-commit, and confirms the + * base branch's tree is the stamped tree. Every step is journaled under + * .gstack/tmp/ship-queue/land/.json, so a rerun resumes. + */ +import { spawnSync } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { mergeChangelogs, parseChangelog } from './changelog-check'; +import { fragmentSlug, readFragments } from './changelog-fragments'; +import { restamp, versionAtRef } from './restamp'; +import { matchesAny, type ShipPolicy } from './ship-policy'; +import { ciAtHead, loadBasePolicy, type QueueContext, type StepOutput } from './ship-queue'; +import { QUEUE_DIR_REL, git, writeJsonAtomic, type QueueFailure } from './ship-queue-gate'; +import { bindShippedReceipt } from './ship-receipt-depth'; +import { loadReady, rebuildFromNotes, repoIdentity } from './ship-queue-train-record'; + +export const LAND_LOCK_REF = 'refs/notes/gstack-land'; +export const MAX_RESTAMPS = 2; +const PUSH_STALE_MS = 10 * 60_000; +const MANIFESTS = ['package.json', 'package-lock.json', 'npm-shrinkwrap.json']; +const isSha = (s: string | undefined) => /^[0-9a-f]{40,64}$/.test(s ?? ''); +/** The first path segment of `rel` inside `wt` that is a symlink, or null: land writes only inside the checkout. */ +function linkedSegment(wt: string, rel: string): string | null { + const segs = rel.split('/'); + const at = segs.findIndex((_, i) => { try { return fs.lstatSync(path.join(wt, ...segs.slice(0, i + 1))).isSymbolicLink(); } catch { return false; } }); + return at < 0 ? null : segs.slice(0, at + 1).join('/'); +} + +export interface LandJournal { + schema_version: 1; pr: string; branch: string; + reviewed_pin: string; last_pushed_head: string | null; stamp_commit: string | null; stamp_tree: string | null; + base_sha: string | null; version: string | null; date: string | null; restamps: number; + drift_main?: string; repair_pr?: string; + phase: 'locked' | 'stamped' | 'pushed' | 'merged' | 'landed' | 'drift'; +} +export interface LandLock { pr: string; holder: string; started_at: string; phase: 'stamping' | 'pushing' | 'waiting' | 'merging' | 'drift'; journal: string; phase_at?: string; repair_pr?: string; drift_of?: string } +export interface LandOptions { pr: string; repair?: boolean; dryRun?: boolean; noWait?: boolean; breakLock?: boolean; allowRepoCommands?: boolean; waitTimeoutMin?: number; now?: () => Date; holder?: string } + +const fail = (lines: string[], failure: QueueFailure, run: string): StepOutput => ({ ok: false, lines, failure, run }); +export const landJournalRel = (pr: string) => `${QUEUE_DIR_REL}/land/${pr}.json`; + +function readJson(file: string): T | null { try { return JSON.parse(fs.readFileSync(file, 'utf8')) as T; } catch { return null; } } + +/** The remote land lock: a ref whose commit holds lock.json, or a released marker. Updated only by exact lease. */ +function readLock(root: string): { sha: string; lock: LandLock | null } { + git(root, ['fetch', '-q', 'origin', `+${LAND_LOCK_REF}:${LAND_LOCK_REF}`]); + const sha = git(root, ['rev-parse', '--verify', '--quiet', LAND_LOCK_REF]).out; + if (!sha) return { sha: '', lock: null }; + const body = git(root, ['show', `${sha}:lock.json`]); + let lock: LandLock | { released: true } | null = null; + try { lock = body.ok ? JSON.parse(body.out) : null; } catch { lock = null; } + return { sha, lock: lock && 'pr' in lock && /^\d+$/.test(String(lock.pr)) ? lock : null }; +} + +function writeLock(root: string, expected: string, value: LandLock | { released: true; by: string; at: string }): { ok: boolean; sha: string } { + const blob = git(root, ['hash-object', '-w', '--stdin'], undefined, JSON.stringify(value, null, 2) + '\n').out; + const tree = git(root, ['mktree'], undefined, `100644 blob ${blob}\tlock.json\n`).out; + const commit = git(root, ['commit-tree', tree, ...(expected ? ['-p', expected] : []), '-m', 'released' in value ? `land lock released by ${value.by}` : `land lock #${value.pr} ${value.phase}`]).out; + if (![blob, tree, commit].every(isSha)) return { ok: false, sha: '' }; + const push = git(root, ['push', '-q', `--force-with-lease=${LAND_LOCK_REF}:${expected}`, 'origin', `${commit}:${LAND_LOCK_REF}`]); + if (push.ok) git(root, ['update-ref', LAND_LOCK_REF, commit]); + return { ok: push.ok, sha: commit }; +} + +/** + * Release-version conflicts only: the version file and release outputs take the base's side, the + * CHANGELOG merges by section (mergeChangelogs), and a manifest or npm lockfile is re-merged with + * every side's release version set to the base's, so a clean dependency or script edit survives and + * a real dependency conflict still conflicts. Returns the files it could not resolve. + */ +export function resolveReleaseConflicts(wt: string, conflicts: string[], policy: ShipPolicy, versionRel: string): string[] { + const left: string[] = []; + const stage = (n: number, rel: string) => git(wt, ['show', `:${n}:${rel}`]); + for (const rel of conflicts) { + const base = path.basename(rel); + if (linkedSegment(wt, rel)) { left.push(rel); continue; } + if (policy.changelog && rel === policy.changelog) { + const merged = mergeChangelogs(stage(2, rel).out + '\n', stage(3, rel).out + '\n'); + if (merged === null) { left.push(rel); continue; } + fs.writeFileSync(path.join(wt, rel), merged); + } else if (!MANIFESTS.includes(base) && (rel === versionRel || matchesAny(policy.release_outputs, rel))) { + if (!git(wt, ['checkout', '--theirs', '--', rel]).ok) { left.push(rel); continue; } + } else if (MANIFESTS.includes(base)) { + const sides = [1, 2, 3].map(n => stage(n, rel)); + if (sides.some(s => !s.ok)) { left.push(rel); continue; } + const theirs = JSON.parse(sides[2]!.out); + const pin = (text: string) => { + const o = JSON.parse(text); + if ('version' in o) o.version = theirs.version; + if (o.packages?.[''] && theirs.packages?.['']) o.packages[''].version = theirs.packages[''].version; + const indent = /^\{\n( +|\t)/.exec(text)?.[1] ?? 2; + return JSON.stringify(o, null, indent) + '\n'; + }; + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-land-merge-')); + const [b, o, t] = ['base', 'ours', 'theirs'].map((n, i) => { const f = path.join(tmp, n); fs.writeFileSync(f, pin(sides[i === 0 ? 0 : i]!.out)); return f; }); + const m = spawnSync('git', ['merge-file', '-p', o!, b!, t!], { encoding: 'utf8', timeout: 30_000 }); + fs.rmSync(tmp, { recursive: true, force: true }); + if (m.status !== 0) { left.push(rel); continue; } + fs.writeFileSync(path.join(wt, rel), m.stdout); + } else { left.push(rel); continue; } + git(wt, ['add', '--', rel]); + } + return left; +} + +/** + * A legacy per-PR stamp (a claim-at-open repo): the pin moved VERSION past its merge base and headed its + * own entry with that version. Before the merge, put the pin's release fields back to the merge base's + * values and re-head that entry `[Unreleased]`, committed in the scratch worktree, so the base merge + * meets an unstamped change and the restamp allocates the version. Returns a note, or null. + */ +function unstampLegacy(wt: string, policy: ShipPolicy, versionRel: string, mergeBase: string): string | QueueFailure | null { + const mb = versionAtRef(wt, mergeBase); + const own = versionAtRef(wt, 'HEAD'); + if (!('version' in mb) || !('version' in own) || mb.version === own.version) return null; + const link = [versionRel, ...MANIFESTS, ...(policy.changelog ? [policy.changelog] : [])].map(rel => linkedSegment(wt, rel)).find(Boolean); + if (link) return { code: 'QUEUE_LAND_CONFLICT', message: `${link} is a symlink at the pin; land rewrites release fields only inside the repository` }; + const touched = MANIFESTS.includes(path.basename(versionRel)) ? [] : [versionRel]; + if (touched.length) fs.writeFileSync(path.join(wt, versionRel), git(wt, ['show', `${mergeBase}:${versionRel}`]).out + '\n'); + for (const rel of MANIFESTS) { + const before = git(wt, ['show', `${mergeBase}:${rel}`]); + if (!before.ok || !fs.existsSync(path.join(wt, rel))) continue; + const text = fs.readFileSync(path.join(wt, rel), 'utf8'); + const b = JSON.parse(before.out); const cur = JSON.parse(text); + if ('version' in cur) cur.version = b.version; + if (cur.packages?.[''] && b.packages?.['']) cur.packages[''].version = b.packages[''].version; + fs.writeFileSync(path.join(wt, rel), JSON.stringify(cur, null, /^\{\n( +|\t)/.exec(text)?.[1] ?? 2) + '\n'); + touched.push(rel); + } + const cl = policy.changelog; + if (cl && fs.existsSync(path.join(wt, cl))) { + const text = fs.readFileSync(path.join(wt, cl), 'utf8'); + const baseKeys = new Set(parseChangelog(git(wt, ['show', `${mergeBase}:${cl}`]).out).sections.map(s => s.version)); + const parsed = parseChangelog(text); + const mine = parsed.sections.find(s => s.version === own.version && !baseKeys.has(s.version)); + if (mine && !parsed.sections.some(s => s.unreleased)) { + const lines = text.split('\n'); lines[mine.start] = '## [Unreleased]'; + fs.writeFileSync(path.join(wt, cl), lines.join('\n')); + touched.push(cl); + } + } + git(wt, ['add', '--', ...touched]); + git(wt, ['commit', '-q', '--no-verify', '-m', `land: put the legacy stamp ${own.version} back to ${mb.version} before the base merge`]); + return `legacy: the pin stamped ${own.version} itself; its release fields go back to ${mb.version} and its entry is re-headed [Unreleased] before the merge`; +} + +/** + * Merge the pin with origin/ in a scratch worktree, resolve release conflicts, give a PR with + * no entry a title-derived fragment, and stamp with gstack-restamp. Returns the stamped tree. + */ +function buildStamp(ctx: QueueContext, policy: ShipPolicy, o: { pin: string; baseSha: string; pr: string; title: string; allowRepoCommands: boolean; now: Date }): { tree: string; version: string; lines: string[] } | QueueFailure { + const root = ctx.repoRoot; + const wt = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-land-')); + fs.rmSync(wt, { recursive: true, force: true }); + if (!git(root, ['worktree', 'add', '--detach', '-q', wt, o.pin]).ok) return { code: 'QUEUE_LAND_CONFLICT', message: `cannot create a worktree at the pin ${o.pin.slice(0, 12)}` }; + try { + const lines: string[] = []; + const at = versionAtRef(root, o.baseSha); + const versionRel = 'rel' in at ? at.rel : 'VERSION'; + const legacy = unstampLegacy(wt, policy, versionRel, git(root, ['merge-base', o.pin, o.baseSha]).out); + if (legacy && typeof legacy !== 'string') return legacy; + if (legacy) lines.push(legacy); + const merge = git(wt, ['merge', '--no-ff', '--no-commit', '-q', o.baseSha]); + if (!merge.ok) { + const conflicts = git(wt, ['diff', '--name-only', '--diff-filter=U']).out.split('\n').filter(Boolean); + if (!conflicts.length) return { code: 'QUEUE_LAND_CONFLICT', message: `git merge failed: ${merge.err.split('\n')[0]}` }; + const left = resolveReleaseConflicts(wt, conflicts, policy, versionRel); + if (left.length) return { code: 'QUEUE_LAND_CONFLICT', message: `#${o.pr} conflicts with origin/${ctx.base} in ${left.join(', ')}` }; + lines.push(`resolved: ${conflicts.join(', ')} (release fields only)`); + } + const dir = policy.changelog_fragments?.dir; + if (dir && policy.changelog) { + const own = readFragments(wt, null, dir, o.baseSha, policy.changelog); + const cl = fs.existsSync(path.join(wt, policy.changelog)) ? fs.readFileSync(path.join(wt, policy.changelog), 'utf8') : ''; + if (!own.errors.length && !own.fragments.length && !parseChangelog(cl).sections.some(s => s.unreleased)) { + const rel = `${dir.replace(/\/+$/, '')}/land-${fragmentSlug(o.pr)}.md`; + const link = linkedSegment(wt, rel); + if (link) return { code: 'CHANGELOG_FRAGMENT_INVALID', message: `${link} is a symlink at the pin; land writes a fragment only inside the repository` }; + fs.mkdirSync(path.dirname(path.join(wt, rel)), { recursive: true }); + fs.writeFileSync(path.join(wt, rel), `- ${o.title.replace(/\s+/g, ' ').trim()} (#${o.pr})\n`); + git(wt, ['add', '--', rel]); + lines.push(`entry: none in the PR, so land wrote ${rel} from the title`); + } + } + if (!git(wt, ['commit', '-q', '--no-verify', '--allow-empty', '-m', `land #${o.pr}: merge origin/${ctx.base}`]).ok) return { code: 'QUEUE_LAND_CONFLICT', message: 'cannot commit the merge in the scratch worktree' }; + const exact = [versionRel, ...MANIFESTS, ...(policy.changelog ? [policy.changelog] : [])]; + const linked = git(wt, ['ls-files', '-s', '-z']).out.split('\0').filter(rec => rec.startsWith('120000 ')).map(rec => rec.slice(rec.indexOf('\t') + 1)) + .find(rel => exact.includes(rel) || matchesAny([...policy.release_outputs, ...policy.stamp_paths], rel)) ?? exact.map(rel => linkedSegment(wt, rel)).find(Boolean); + if (linked) return { code: 'QUEUE_LAND_CONFLICT', message: `${linked} is a symlink in the merged tree; the merge step writes release files only inside the repository` }; + const r = restamp({ repoRoot: wt, base: ctx.base, next: true, policyFrom: 'base', allowRepoCommands: o.allowRepoCommands, dryRun: false, now: o.now, env: process.env }); + if (r.status !== 'stamped' && r.status !== 'noop') return { code: r.code ?? 'QUEUE_LAND_CONFLICT', message: `restamp ${r.status}: ${r.message ?? ''}`, fix: r.fix }; + lines.push(...r.stages.filter(s => s.status !== 'pending').map(s => `stage ${s.name}: ${s.status}${s.detail ? ` (${s.detail})` : ''}`)); + git(wt, ['add', '-A']); + const tree = git(wt, ['write-tree']).out; + if (!isSha(tree)) return { code: 'QUEUE_LAND_CONFLICT', message: 'cannot write the stamped tree in the scratch worktree' }; + return { tree, version: r.newVersion!, lines }; + } finally { + git(root, ['worktree', 'remove', '--force', wt]); + fs.rmSync(wt, { recursive: true, force: true }); + } +} + +interface PrInfo { state: string; baseRefName?: string; headRefName: string; headRefOid: string; isCrossRepository: boolean; body: string; title: string } +function prInfo(ctx: QueueContext, pr: string): PrInfo | QueueFailure { + const r = ctx.gh(['pr', 'view', pr, '--json', 'state,baseRefName,headRefName,headRefOid,isCrossRepository,body,title']); + try { return JSON.parse(r.out) as PrInfo; } catch { return { code: 'QUEUE_NOT_READY', message: `gh pr view ${pr} failed: ${(r.err ?? r.out).trim().slice(0, 160)}` }; } +} + +/** Poll the stamp commit's checks (the same read as the pin's) until they are green, one is red, or the time runs out. */ +function waitChecks(ctx: QueueContext, sha: string, minutes: number): void { + const deadline = Date.now() + minutes * 60_000; + for (let ci = ciAtHead(ctx, sha); !ci.green && !ci.line.startsWith('red') && Date.now() < deadline; ci = ciAtHead(ctx, sha)) { + Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 30_000); + } +} + +/** A head that is land's own stamp commit ([pin, base], `(land)` subject for this PR): its pin, when the local journal is gone. */ +function stampPin(root: string, branch: string, head: string, pr: string): string | null { + git(root, ['fetch', '-q', 'origin', `refs/heads/${branch}`]); + const [parents = '', subject = ''] = git(root, ['log', '-1', '--format=%P%n%s', head]).out.split('\n'); + const p = parents.split(' ').filter(Boolean); + return p.length === 2 && subject.endsWith(` into #${pr} (land)`) ? p[0]! : null; +} + +/** `land --census`: what land would do with each open PR before a repository turns fragments on. Reads only. */ +export function census(ctx: QueueContext): StepOutput { + const root = ctx.repoRoot; + const loaded = loadBasePolicy(ctx); + if ('message' in loaded) return fail([], loaded, 'none'); + const policy = loaded.policy; + git(root, ['fetch', '-q', 'origin', ctx.base]); + const baseSha = git(root, ['rev-parse', `origin/${ctx.base}`]).out; + const list = ctx.gh(['pr', 'list', '--state', 'open', '--base', ctx.base, '--limit', '200', '--json', 'number,headRefOid,isCrossRepository']); + let prs: Array<{ number: number; headRefOid: string; isCrossRepository: boolean }> = []; + try { prs = JSON.parse(list.out); } catch { return fail([], { code: 'QUEUE_NOT_READY', message: 'gh pr list failed' }, 'none'); } + const at = versionAtRef(root, baseSha); + const versionRel = 'rel' in at ? at.rel : 'VERSION'; + const lines = [`CENSUS: ${prs.length} open PR(s) against origin/${ctx.base}@${baseSha.slice(0, 12)} (fragments: ${policy.changelog_fragments ? policy.changelog_fragments.dir : 'off'})`]; + const heads = prs.filter(p => !p.isCrossRepository).map(p => `refs/pull/${p.number}/head`); + if (heads.length && !git(root, ['fetch', '-q', 'origin', ...heads]).ok) for (const h of heads) git(root, ['fetch', '-q', 'origin', h]); + for (const p of prs.sort((a, b) => a.number - b.number)) { + if (p.isCrossRepository) { lines.push(`#${p.number} fork: refused (QUEUE_TRAIN_UNTRUSTED)`); continue; } + const head = p.headRefOid; + const mt = git(root, ['merge-tree', '--write-tree', '--name-only', '--no-messages', baseSha, head]); + const conflicts = mt.ok ? [] : mt.out.split('\n').slice(1).filter(Boolean); + const product = conflicts.filter(c => c !== policy.changelog && c !== versionRel && !matchesAny(policy.release_outputs, c) && !['package.json', 'package-lock.json', 'npm-shrinkwrap.json'].includes(path.basename(c))); + const frags = policy.changelog_fragments ? readFragments(root, head, policy.changelog_fragments.dir, baseSha, policy.changelog) : { fragments: [], errors: [] }; + const cl = policy.changelog ? git(root, ['show', `${head}:${policy.changelog}`]).out : ''; + const legacy = parseChangelog(cl).sections.some(s => s.unreleased); + const hv = versionAtRef(root, head); + const bumped = 'version' in hv && 'version' in at && hv.version !== at.version; + const entry = frags.errors.length ? `invalid fragment ${frags.errors[0]!.path}` : frags.fragments.length ? `fold ${frags.fragments.map(f => path.basename(f.path)).join(', ')}` : legacy ? 're-head legacy [Unreleased]' : 'no entry (land writes one from the title)'; + lines.push(`#${p.number} ${head.slice(0, 12)}: ${product.length ? `refuse (product conflict: ${product.join(', ')})` : entry}${bumped ? `; rewrite its tentative ${'version' in hv ? hv.version : ''} through restamp` : ''}${conflicts.length && !product.length ? `; resolve ${conflicts.join(', ')} (release fields only)` : ''}`); + } + return { ok: true, run: 'none', lines }; +} + +/** + * `land --repair --pr ` after QUEUE_LAND_DRIFT: a merge outside land slipped in beside #n, so what it + * brought has no release of its own. Open a PR from the base whose only change is a fragment for it: each + * fragment the outside merge left unfolded on the base moves to repair--.md (so it counts as new), + * or, with none, one entry naming the outside commits. That PR goes through /ship and land like any other; + * its land takes over #n's drift lock and releases it. + */ +function repair(ctx: QueueContext, policy: ShipPolicy, j: LandJournal | null, file: string, run: string): StepOutput { + const root = ctx.repoRoot; + if (!j || j.phase !== 'drift' || !j.drift_main || !j.base_sha) return fail([], { code: 'QUEUE_NOT_READY', message: `#${j?.pr ?? '?'} has no drift record; --repair runs only after QUEUE_LAND_DRIFT` }, run); + const lines = [`REPAIR: #${j.pr} landed on ${j.drift_main.slice(0, 12)} beside a merge outside land`]; + if (j.repair_pr) return { ok: true, run, lines: [...lines, `REPAIR_PR: #${j.repair_pr} (already open)`, `NEXT: run /ship on #${j.repair_pr}, then gstack-ship-queue land --pr ${j.repair_pr}`] }; + const dir = policy.changelog_fragments?.dir?.replace(/\/+$/, ''); + if (!dir) return fail(lines, { code: 'QUEUE_NOT_READY', message: 'changelog_fragments is off: write the [Unreleased] entry for the outside merge by hand and land it' }, run); + git(root, ['fetch', '-q', 'origin', ctx.base]); + const main = git(root, ['rev-parse', `origin/${ctx.base}`]).out; + const outside = git(root, ['log', '--first-parent', '--format=%h %s', `${j.base_sha}..${j.drift_main}^1`]).out.split('\n').filter(Boolean); + const stray = readFragments(root, main, dir, null); + const branch = `gstack-land-repair/${j.pr}`; + const wt = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-land-repair-')); + fs.rmSync(wt, { recursive: true, force: true }); + if (!git(root, ['worktree', 'add', '--detach', '-q', wt, main]).ok) return fail(lines, { code: 'QUEUE_NOT_READY', message: `cannot create a worktree at ${main.slice(0, 12)}` }, run); + try { + for (const f of stray.fragments) { + fs.renameSync(path.join(wt, f.path), path.join(wt, dir, `repair-${j.pr}-${path.basename(f.path)}`)); + lines.push(`moved: ${f.path} (left unfolded by the outside merge)`); + } + if (!stray.fragments.length) { + fs.mkdirSync(path.join(wt, dir), { recursive: true }); + fs.writeFileSync(path.join(wt, dir, `repair-${j.pr}.md`), `- Changes merged outside land beside #${j.pr}: ${outside.map(c => c.slice(c.indexOf(' ') + 1)).join('; ') || 'unknown'}\n`); + lines.push(`entry: ${dir}/repair-${j.pr}.md names ${outside.length} outside commit(s)`); + } + git(wt, ['add', '-A', '--', dir]); + if (!git(wt, ['commit', '-q', '--no-verify', '-m', `land: release entry for the merge outside land beside #${j.pr}`]).ok) return fail(lines, { code: 'QUEUE_NOT_READY', message: 'cannot commit the repair' }, run); + const head = git(wt, ['rev-parse', 'HEAD']).out; + const push = git(root, ['push', '-q', `--force-with-lease=refs/heads/${branch}:`, 'origin', `${head}:refs/heads/${branch}`]); + let pushed = head; + if (!push.ok) { + git(root, ['fetch', '-q', 'origin', `refs/heads/${branch}`]); + const remote = git(root, ['log', '-1', '--format=%H%n%s', 'FETCH_HEAD']).out.split('\n'); + if (!isSha(remote[0]) || remote[1] !== `land: release entry for the merge outside land beside #${j.pr}`) return fail(lines, { code: 'QUEUE_TRAIN_MOVED', message: `cannot push ${branch}, and origin holds no repair commit there: ${push.err.split('\n')[0]}` }, run); + pushed = remote[0]!; + lines.push(`branch: ${branch} already on origin at ${pushed.slice(0, 12)} (an earlier --repair pushed it); opening or finding its PR`); + } + const pr = ctx.gh(['pr', 'create', '--base', ctx.base, '--head', branch, '--title', `Release entry for the merge outside land beside #${j.pr}`, '--body', `QUEUE_LAND_DRIFT repair (docs/troubleshooting.md#queue-land-drift). Outside commits: ${outside.join(', ') || 'unknown'}.`]); + const n = /\/pull\/(\d+)/.exec(`${pr.out}\n${pr.err ?? ''}`)?.[1]; + if (!n) return fail(lines, { code: 'QUEUE_NOT_READY', message: `gh pr create failed: ${(pr.err ?? pr.out).trim().slice(0, 160)}` }, run); + j.repair_pr = n; writeJsonAtomic(file, j); + const held = readLock(root); + if (held.lock?.pr === j.pr && held.lock.phase === 'drift') writeLock(root, held.sha, { ...held.lock, repair_pr: n }); + return { ok: true, run, lines: [...lines, `REPAIR_PR: #${n} ${branch} ${pushed}`, `NEXT: run /ship on #${n}, then gstack-ship-queue land --pr ${n} (it takes over #${j.pr}'s drift lock and releases it)`] }; + } finally { + git(root, ['worktree', 'remove', '--force', wt]); + fs.rmSync(wt, { recursive: true, force: true }); + } +} + +/** `gstack-ship-queue land --pr `: one resumable pass of the merge step. */ +export function land(ctx: QueueContext, o: LandOptions): StepOutput { + const root = ctx.repoRoot; + const run = landJournalRel(o.pr); + const file = path.join(root, run); + const now = o.now ?? (() => new Date()); + const holder = o.holder ?? `${os.hostname()}:${process.pid}`; + const loaded = loadBasePolicy(ctx); + if ('message' in loaded) return fail([], loaded, run); + const policy = loaded.policy; + if (o.repair) return repair(ctx, policy, readJson(file), file, run); + const info = prInfo(ctx, o.pr); + if ('code' in info) return fail([], info, run); + if (info.isCrossRepository) return fail([], { code: 'QUEUE_TRAIN_UNTRUSTED', message: `#${o.pr} is a fork head; land stamps and pushes only crew branches` }, run); + let j = readJson(file); + const mergedResume = info.state === 'MERGED' && !!j && ['pushed', 'merged', 'landed'].includes(j.phase) && j.stamp_commit === info.headRefOid; + if (info.state !== 'OPEN' && !mergedResume) return fail([], { code: 'QUEUE_NOT_READY', message: `#${o.pr} is ${info.state}` }, run); + if (info.baseRefName && info.baseRefName !== ctx.base) return fail([], { code: 'QUEUE_NOT_READY', message: `#${o.pr} targets ${info.baseRefName}, not ${ctx.base}; land merges only PRs on the base it stamps (retarget it, or land its parent first)` }, run); + if (j && !mergedResume && (j.phase === 'landed' || ![j.reviewed_pin, j.last_pushed_head, j.stamp_commit].includes(info.headRefOid))) j = null; + if (j && mergedResume) j.phase = 'merged'; + const recovered = j ? null : stampPin(root, info.headRefName, info.headRefOid, o.pr); + const pin = j?.reviewed_pin ?? recovered ?? info.headRefOid; + const lines = [`LAND: #${o.pr} branch=${info.headRefName} pin=${pin}${j ? ` (resuming at ${j.phase})` : recovered ? ` (the head ${info.headRefOid.slice(0, 12)} is land's stamp commit; restamping from its pin)` : ''}`]; + git(root, ['fetch', '-q', 'origin', ctx.base]); + const receipt = bindShippedReceipt(root, `origin/${ctx.base}`, info.body ?? '', o.pr, [pin]); + lines.push(receipt.line, ...(receipt.depthLine ? [receipt.depthLine] : [])); + if (!receipt.ok) return fail(lines, { code: receipt.code, message: receipt.message, fix: 'run /ship at the pinned head first' }, run); + if (!policy.changelog_fragments) return fail(lines, { code: 'QUEUE_NOT_READY', message: 'the base policy has changelog_fragments off; land merges only under the key, where ready records the READY receipt it checks' }, run); + const keys = repoIdentity(root).keys; + if (!loadReady(keys, o.pr, pin)) rebuildFromNotes(root, ctx.remote ?? 'origin'); + if (!loadReady(keys, o.pr, pin)) return fail(lines, { code: 'QUEUE_NOT_READY', message: `no READY receipt for #${o.pr} at the pin ${pin.slice(0, 12)}; ready checks the pregate and full_e2e evidence land merges on`, fix: `run gstack-ship-queue ready --pr ${o.pr}` }, run); + if (!j) { + const ci = ciAtHead(ctx, pin); + lines.push(`CI: ${ci.line}`); + if (!ci.green) return fail(lines, { code: 'QUEUE_NOT_READY', message: `required checks are not green at the pin ${pin.slice(0, 12)}` }, run); + } + if (o.dryRun) { + git(root, ['fetch', '-q', 'origin', ctx.base]); + const s = buildStamp(ctx, policy, { pin, baseSha: git(root, ['rev-parse', `origin/${ctx.base}`]).out, pr: o.pr, title: info.title, allowRepoCommands: !!o.allowRepoCommands, now: now() }); + if ('code' in s) return fail(lines, s, run); + return { ok: true, run, lines: [...lines, ...s.lines, `STAMP_VERSION: ${s.version}`, `STAMP_TREE: ${s.tree}`, 'DRY-RUN: nothing locked, pushed or merged'] }; + } + + let { sha: lockSha, lock } = readLock(root); + const drifted = lock && lock.pr !== o.pr && lock.phase === 'drift' ? lock.pr : null; + const repairOf = drifted ? readJson(path.join(root, landJournalRel(drifted))) : null; + const driftOf = drifted && (lock!.repair_pr === o.pr || repairOf?.repair_pr === o.pr) ? drifted : lock?.pr === o.pr ? lock.drift_of : undefined; + if (drifted && driftOf) lines.push(`LOCK: taking over #${drifted}'s drift lock for its repair`); + else if (lock && lock.pr !== o.pr) { + const age = now().getTime() - Date.parse(lock.phase_at ?? lock.started_at); + if (!o.breakLock) return fail(lines, { code: 'QUEUE_LAND_LOCKED', message: `#${lock.pr} holds the land lock (${lock.holder}, phase ${lock.phase}, since ${lock.started_at})` }, run); + if (lock.phase === 'drift' || lock.drift_of) return fail(lines, { code: 'QUEUE_LAND_DRIFT', message: `#${lock.drift_of ?? lock.pr} holds a drift lock; --break-lock never breaks it, only its repair PR's land takes it over`, fix: `gstack-ship-queue land --pr ${lock.drift_of ?? lock.pr} --repair` }, run); + if ((lock.phase === 'pushing' || lock.phase === 'merging') && age < PUSH_STALE_MS) return fail(lines, { code: 'QUEUE_LAND_LOCKED', message: `#${lock.pr}'s holder is mid-${lock.phase === 'pushing' ? 'push' : 'merge'} (${Math.round(age / 1000)} s ago); --break-lock refused` }, run); + lines.push(`LOCK: broke #${lock.pr}'s lock (${lock.holder}, phase ${lock.phase})`); + } + const setLock = (phase: LandLock['phase']): QueueFailure | null => { + const next: LandLock = { pr: o.pr, holder, started_at: lock?.pr === o.pr ? lock.started_at : now().toISOString(), phase, journal: run, phase_at: now().toISOString(), ...(driftOf ? { drift_of: driftOf } : {}) }; + const w = writeLock(root, lockSha, next); + if (!w.ok) return { code: 'QUEUE_LAND_LOCKED', message: 'the land lock moved under this run (another lander took it)' }; + lockSha = w.sha; lock = next; + return null; + }; + const save = () => writeJsonAtomic(file, j); + const failed = setLock(j?.phase === 'pushed' ? 'waiting' : j?.phase === 'merged' ? 'merging' : 'stamping'); + if (failed) return fail(lines, failed, run); + if (!j) { j = { schema_version: 1, pr: o.pr, branch: info.headRefName, reviewed_pin: pin, last_pushed_head: recovered ? info.headRefOid : null, stamp_commit: null, stamp_tree: null, base_sha: null, version: null, date: null, restamps: 0, phase: 'locked' }; save(); } + + for (;;) { + git(root, ['fetch', '-q', 'origin', ctx.base]); + const baseNow = git(root, ['rev-parse', `origin/${ctx.base}`]).out; + if (j.phase === 'pushed' && j.base_sha !== baseNow) { + if (j.restamps >= MAX_RESTAMPS) { + j.restamps = 0; save(); + if (!driftOf) writeLock(root, lockSha, { released: true, by: holder, at: now().toISOString() }); + return fail(lines, { code: 'QUEUE_LAND_BASE_BUSY', message: `origin/${ctx.base} moved during CI ${MAX_RESTAMPS + 1} times; the lock is released and a later run restamps from the pin` }, run); + } + lines.push(`BASE_MOVED: ${j.base_sha?.slice(0, 12)} -> ${baseNow.slice(0, 12)}; restamping`); + j.restamps += 1; j.phase = 'locked'; save(); + } + if (j.phase === 'locked') { + const s = buildStamp(ctx, policy, { pin, baseSha: baseNow, pr: o.pr, title: info.title, allowRepoCommands: !!o.allowRepoCommands, now: j.date ? new Date(`${j.date}T12:00:00Z`) : now() }); + if ('code' in s) { + if (driftOf) setLock('stamping'); + else writeLock(root, lockSha, { released: true, by: holder, at: now().toISOString() }); + return fail([...lines], s, run); + } + lines.push(...s.lines); + const commit = git(root, ['commit-tree', s.tree, '-p', pin, '-p', baseNow, '-m', `v${s.version} stamp: merge ${ctx.base} ${baseNow.slice(0, 12)} into #${o.pr} (land)`]).out; + if (!isSha(commit)) return fail(lines, { code: 'QUEUE_LAND_CONFLICT', message: 'cannot create the stamp commit; nothing pushed' }, run); + Object.assign(j, { stamp_commit: commit, stamp_tree: s.tree, base_sha: baseNow, version: s.version, date: j.date ?? now().toISOString().slice(0, 10), phase: 'stamped' }); + save(); + lines.push(`STAMP_VERSION: ${s.version}`, `STAMP_COMMIT: ${commit} tree=${s.tree}`); + } + if (j.phase === 'stamped') { + const lease = j.last_pushed_head ?? pin; + const f = setLock('pushing'); + if (f) return fail(lines, f, run); + const push = git(root, ['push', '-q', `--force-with-lease=refs/heads/${j.branch}:${lease}`, 'origin', `${j.stamp_commit}:refs/heads/${j.branch}`]); + if (!push.ok) { + const remote = git(root, ['ls-remote', 'origin', `refs/heads/${j.branch}`]).out.split('\t')[0]; + if (remote !== j.stamp_commit) return fail(lines, { code: 'QUEUE_TRAIN_MOVED', message: `#${o.pr}'s branch is at ${remote?.slice(0, 12) || 'unknown'}, not ${lease.slice(0, 12)}: someone else pushed; land never overwrites it` }, run); + } + j.last_pushed_head = j.stamp_commit; j.phase = 'pushed'; save(); + const fw = setLock('waiting'); + if (fw) return fail(lines, fw, run); + lines.push(`PUSHED: ${j.stamp_commit} to ${j.branch} (lease ${lease.slice(0, 12)})`); + if (o.noWait) return { ok: true, run, lines: [...lines, `NEXT: gstack-pr-wait ${o.pr} --until green && gstack-ship-queue land --pr ${o.pr}`] }; + waitChecks(ctx, j.stamp_commit!, o.waitTimeoutMin ?? 60); + continue; + } + if (j.phase === 'pushed') { + const head = git(root, ['ls-remote', 'origin', `refs/heads/${j.branch}`]).out.split('\t')[0] ?? ''; + if (head !== j.stamp_commit) return fail(lines, { code: 'QUEUE_TRAIN_MOVED', message: `#${o.pr}'s head is ${head.slice(0, 12) || 'unknown'}, not the stamp commit ${j.stamp_commit?.slice(0, 12)}` }, run); + const ci = ciAtHead(ctx, j.stamp_commit!); + lines.push(`CI: ${ci.line}`); + if (!ci.green) return { ok: !ci.line.startsWith('red'), run, lines: [...lines, `NEXT: gstack-pr-wait ${o.pr} --until green && gstack-ship-queue land --pr ${o.pr}`], failure: ci.line.startsWith('red') ? { code: 'QUEUE_NOT_READY', message: `a check failed on the stamp commit ${j.stamp_commit!.slice(0, 12)}` } : undefined }; + const fm = setLock('merging'); + if (fm) return fail(lines, fm, run); + const m = ctx.gh(['pr', 'merge', o.pr, policy.history === 'merge-only' ? '--merge' : '--squash', '--match-head-commit', j.stamp_commit!]); + if (!m.ok) return fail(lines, { code: 'QUEUE_NOT_READY', message: `gh pr merge refused: ${(m.err ?? m.out).trim().slice(0, 200)}` }, run); + j.phase = 'merged'; save(); + } + if (j.phase === 'merged') { + git(root, ['fetch', '-q', 'origin', ctx.base]); + const tip = git(root, ['rev-parse', `origin/${ctx.base}`]).out; + const main = git(root, ['rev-list', '--first-parent', '--reverse', `${j.base_sha}..${tip}`]).out.split('\n')[0] || tip; + const tree = git(root, ['rev-parse', `${main}^{tree}`]).out; + const parent = git(root, ['rev-parse', `${main}^1`]).out; + if (tree !== j.stamp_tree || parent !== j.base_sha) { + j.phase = 'drift'; j.drift_main = tip; save(); setLock('drift'); + return fail([...lines, `MAIN: ${tip} first=${main} tree=${tree} parent=${parent}`], { code: 'QUEUE_LAND_DRIFT', message: `origin/${ctx.base} ${main.slice(0, 12)} is not the stamped tree on ${j.base_sha?.slice(0, 12)} (a merge outside land slipped in)`, fix: `gstack-ship-queue land --pr ${o.pr} --repair opens the PR that gives the outside merge its own release; /ship and land it, which releases the lock` }, run); + } + j.phase = 'landed'; save(); + const driftJournal = driftOf ? readJson(path.join(root, landJournalRel(driftOf))) : null; + if (driftOf && driftJournal) writeJsonAtomic(path.join(root, landJournalRel(driftOf)), { ...driftJournal, phase: 'landed' }); + const rel = writeLock(root, lockSha, { released: true, by: holder, at: now().toISOString() }); + return { ok: true, run, lines: [...lines, `MERGED: #${o.pr} as ${main} (${j.version})`, `LANDED: origin/${ctx.base} tree ${tree} equals the stamped tree`, `LOCK: ${rel.ok ? 'released' : 'release push failed (the next land takes it over)'}`] }; + } + if (j.phase === 'drift') return fail(lines, { code: 'QUEUE_LAND_DRIFT', message: 'a previous land of this PR found drift', fix: `gstack-ship-queue land --pr ${o.pr} --repair` }, run); + } +} diff --git a/lib/ship-queue.ts b/lib/ship-queue.ts index e4588ea13..68ae3cf26 100644 --- a/lib/ship-queue.ts +++ b/lib/ship-queue.ts @@ -39,6 +39,7 @@ import { type AttemptRecord, type QueueFailure, } from './ship-queue-gate'; import { wtreeFingerprint } from './script-invocation'; +import { nowIso, persist, readyPath, repoIdentity } from './ship-queue-train-record'; import { buildReceipt, type TreeReceipt } from './tree-receipt'; export type Gh = (args: string[]) => { ok: boolean; out: string; err?: string }; @@ -277,6 +278,12 @@ export function ready(ctx: QueueContext, o: { pr: string; pregate?: string; gate if (!pregateOk) return fail({ code: 'QUEUE_NOT_READY', message: `the pregate result is not a current pass: ${pregateLine}` }, lines, run); if (gate && !gate.ok) return fail({ code: 'QUEUE_GATE_MISSING', message: `full_e2e is set and no passing gate is bound to ${head.slice(0, 12)}: ${gate.line}` }, lines, run); if (!receipt.ok) return fail({ code: receipt.code, message: receipt.message }, lines, run); + if (!policy.policy.train && policy.policy.changelog_fragments) { + const repo = repoIdentity(ctx.repoRoot); + const now = nowIso(); + const p = persist(ctx.repoRoot, [{ schema: 1, kind: 'ready', repo_key: repo.key, pr: o.pr, head, base_sha: baseSha, ready_at: now, screen: { choice: 'no-screen', policy: null, nonce: null, run: null, merge_sha: null }, updated_at: now }], { remote: ctx.remote ?? 'origin' }); + lines.push(`READY_RECEIPT: ${readyPath(repo.key, o.pr, head)} (${p.detail})`); + } return { ok: true, lines, run }; } diff --git a/lib/tree-receipt.ts b/lib/tree-receipt.ts index 2451bcae6..de94d0004 100644 --- a/lib/tree-receipt.ts +++ b/lib/tree-receipt.ts @@ -14,6 +14,7 @@ import { spawnSync } from 'node:child_process'; import * as fs from 'node:fs'; import * as path from 'node:path'; import { normalizeChangelog } from './changelog-check'; +import { foldIntoUnreleased, parseFragment, type Fragment } from './changelog-fragments'; import { compareIdentity, executionIdentity, normalizeLockfile, readBundle, type EvidenceBundle, type ExecutionIdentity } from './evidence-bundle'; import { parseUnifiedDiff, stampOnlyLine, versionAtRef, type FileDiff, type GateAheadRecord } from './restamp'; import type { ResultCodeName } from './result-codes'; @@ -49,6 +50,30 @@ function blob(cwd: string, treeish: string, rel: string): string | null { return r.status === 0 ? r.stdout : null; } +/** + * Every file's verdict for one gated/current pair, with the changelog_fragments fold: a fragment deleted + * between the two is a stamp only when the CHANGELOG change is exactly those fragments folded under the new + * heading. gstack-tree-receipt and gstack-pregate clear both classify through this, so their rules agree. + */ +export function classifyFiles(repoRoot: string, fileDiffs: FileDiff[], gated: string, current: string, policy: ShipPolicy, versionRel: string, oldV: string | null, newV: string | null): FileVerdict[] { + const prefix = policy.changelog_fragments ? policy.changelog_fragments.dir.replace(/\/+$/, '') + '/' : null; + const folded = new Map(); + for (const f of fileDiffs) { + if (!prefix || f.status !== 'D' || f.binary || f.modeChanged || !f.path.startsWith(prefix) || f.path.slice(prefix.length).includes('/') || !f.path.endsWith('.md')) continue; + const text = blob(repoRoot, gated, f.path); + const parsed = text === null ? null : parseFragment(f.path, text); + if (parsed && !('code' in parsed)) folded.set(f.path, parsed); + } + const fragments = [...folded.values()].sort((a, b) => (a.path < b.path ? -1 : 1)); + const changelogVerdict = (f: FileDiff) => classifyFile(repoRoot, f, gated, current, policy, versionRel, oldV, newV, fragments); + const clDiff = fileDiffs.find(f => policy.changelog && f.path === policy.changelog); + const clStamp = !!clDiff && changelogVerdict(clDiff).verdict === 'stamp'; + return fileDiffs.map((f): FileVerdict => { + if (!folded.has(f.path)) return f === clDiff ? changelogVerdict(f) : classifyFile(repoRoot, f, gated, current, policy, versionRel, oldV, newV); + return clStamp ? { path: f.path, rule: 'changelog_fragments', verdict: 'stamp' } : { path: f.path, rule: 'changelog_fragments', verdict: 'changed', residual: `deleted without its text folded under the new heading in ${policy.changelog ?? 'the CHANGELOG'}` }; + }); +} + /** * Every removed/added pair differs only by stamp rewrites to the new version: * the old version, or a tentative one (newer than it) the stamp_paths stage @@ -71,7 +96,7 @@ function jsonEqualModuloVersion(a: string | null, b: string | null, name: string return normalizeLockfile(name, a) === normalizeLockfile(name, b); } -export function classifyFile(repoRoot: string, f: FileDiff, gated: string, current: string, policy: ShipPolicy, versionRel: string, oldV: string | null, newV: string | null): FileVerdict { +export function classifyFile(repoRoot: string, f: FileDiff, gated: string, current: string, policy: ShipPolicy, versionRel: string, oldV: string | null, newV: string | null, folded: readonly Fragment[] = []): FileVerdict { const changed = (rule: string, residual: string, line = f.hunks[0]?.at): FileVerdict => ({ path: f.path, rule, verdict: 'changed', residual, line }); const stamp = (rule: string): FileVerdict => ({ path: f.path, rule, verdict: 'stamp' }); if (f.binary) return changed('binary', 'binary content differs'); @@ -98,7 +123,8 @@ export function classifyFile(repoRoot: string, f: FileDiff, gated: string, curre return changed('manifest', 'a field other than version changed'); } if (policy.changelog && f.path === policy.changelog) { - return before !== null && after !== null && normalizeChangelog(before) === normalizeChangelog(after) ? stamp('changelog-heading') : changed('changelog', 'entry text changed (only a heading move or re-heading is a stamp)'); + if (before !== null && after !== null && normalizeChangelog(foldIntoUnreleased(before, folded)) === normalizeChangelog(after)) return stamp(folded.length ? 'changelog-fragments' : 'changelog-heading'); + return changed('changelog', folded.length ? 'entry text is not exactly the deleted fragments folded under the new heading' : 'entry text changed (only a heading move or re-heading is a stamp)'); } const rule = matchesAny(policy.stamp_paths, f.path) ? 'stamp_paths' : matchesAny(policy.release_outputs, f.path) ? 'release_outputs' : matchesAny(policy.stamp_files, f.path) ? 'stamp_files' : null; if (oldV && newV && oldV !== newV && rule) { @@ -145,7 +171,7 @@ export function buildReceipt(o: ReceiptOptions): TreeReceipt | { error: string; const oldV = 'version' in gv ? gv.version : null; const newV = 'version' in cv ? cv.version : null; const diffText = git(o.repoRoot, ['diff', '--unified=0', '--no-color', '--no-ext-diff', gatedHead, currentHead]) ?? ''; - const files = parseUnifiedDiff(diffText).map(f => classifyFile(o.repoRoot, f, gatedHead, currentHead, o.policy, versionRel, oldV, newV)); + const files = classifyFiles(o.repoRoot, parseUnifiedDiff(diffText), gatedHead, currentHead, o.policy, versionRel, oldV, newV); const residual = files.filter(f => f.verdict === 'changed').map(f => f.path); const stampFiles = files.filter(f => f.verdict === 'stamp').map(f => f.path); const stampDiff = stampFiles.length ? (git(o.repoRoot, ['diff', '--no-color', '--no-ext-diff', gatedHead, currentHead, '--', ...stampFiles]) ?? '') : ''; diff --git a/lib/version-gate-train.ts b/lib/version-gate-train.ts index e809d9408..9ef0790e5 100644 --- a/lib/version-gate-train.ts +++ b/lib/version-gate-train.ts @@ -12,12 +12,13 @@ import * as fs from 'node:fs'; import * as os from 'node:os'; import * as path from 'node:path'; import { parseChangelog } from './changelog-check'; +import { effectiveBump, readFragments } from './changelog-fragments'; import { matchesAny } from './ship-policy'; export type TrainHead = | { kind: 'off' } - | { kind: 'unstamped' } - | { kind: 'stamped'; expected: string } + | { kind: 'unstamped'; fragments?: number; entry?: 'fragment' | 'unreleased' | 'none-bot' } + | { kind: 'stamped'; expected: string; fragments?: boolean } | { kind: 'invalid'; reason: string }; function git(repoRoot: string, args: string[]): string | null { @@ -89,12 +90,13 @@ export function regeneratedAtBase(o: { repoRoot: string; head: string; baseVersi } } -export function classifyTrainHead(o: { repoRoot: string; baseRef: string; head: string }): TrainHead { +export function classifyTrainHead(o: { repoRoot: string; baseRef: string; head: string; author?: string }): TrainHead { const base = `origin/${o.baseRef.replace(/^origin\//, '')}`; const show = (rev: string, file: string) => git(o.repoRoot, ['show', `${rev}:${file}`]); let policy: any = null; try { policy = JSON.parse(show(base, '.gstack/ship-policy.json') ?? 'null'); } catch { policy = null; } - if (!policy || typeof policy.train !== 'object' || policy.train === null) return { kind: 'off' }; + const fragDir: string | null = typeof policy?.changelog_fragments?.dir === 'string' ? policy.changelog_fragments.dir : null; + if (!policy || ((typeof policy.train !== 'object' || policy.train === null) && !fragDir)) return { kind: 'off' }; const mergeBase = git(o.repoRoot, ['merge-base', base, o.head])?.trim(); if (!mergeBase) return { kind: 'invalid', reason: `no merge base between ${base} and ${o.head}` }; @@ -113,8 +115,15 @@ export function classifyTrainHead(o: { repoRoot: string; baseRef: string; head: if (regen.kind === 'differs') touched.push(...regen.files); } else touched.push(...outputs); if (touched.length) { - const expected = bumpVersion(baseVersion, policy.bump ?? 'patch'); - return expected ? { kind: 'stamped', expected } : { kind: 'invalid', reason: `${base}:VERSION is not a 4-part version` }; + // A merge-step stamp commit has parents [pin, base]: the fragments it folded are the pin's own. + const second = git(o.repoRoot, ['rev-parse', '--verify', '--quiet', `${o.head}^2`])?.trim(); + const pin = second ? `${o.head}^1` : mergeBase; + if (fragDir && (!second || git(o.repoRoot, ['merge-base', '--is-ancestor', second, base]) === null)) return { kind: 'invalid', reason: `under changelog_fragments only the merge step stamps (a commit with parents [pin, ${base}]); leave VERSION, package.json and the release outputs at the merge base's values and add a fragment` }; + const folded = fragDir ? readFragments(o.repoRoot, pin, fragDir, base) : null; + if (folded?.errors.length) return { kind: 'invalid', reason: `${folded.errors[0]!.path}: ${folded.errors[0]!.message}` }; + const level = policy.bump === 'patch' ? 'patch' : 'micro'; + const expected = bumpVersion(baseVersion, folded ? effectiveBump(level, folded.fragments) : (policy.bump ?? 'patch')); + return expected ? { kind: 'stamped', expected, fragments: !!fragDir } : { kind: 'invalid', reason: `${base}:VERSION is not a 4-part version` }; } const file = typeof policy.changelog === 'string' ? policy.changelog : 'CHANGELOG.md'; @@ -136,5 +145,11 @@ export function classifyTrainHead(o: { repoRoot: string; baseRef: string; head: if (was.length !== now.length || was.some((text, i) => text !== now[i])) { return { kind: 'invalid', reason: `${file} changes text outside [Unreleased]; an unstamped head may only add or edit its [Unreleased] section` }; } - return { kind: 'unstamped' }; + if (!fragDir) return { kind: 'unstamped' }; + const frags = readFragments(o.repoRoot, o.head, fragDir, mergeBase); + if (frags.errors.length) return { kind: 'invalid', reason: `${frags.errors[0]!.path}: ${frags.errors[0]!.message}` }; + if (frags.fragments.length) return { kind: 'unstamped', fragments: frags.fragments.length, entry: 'fragment' }; + if (unreleased.length) return { kind: 'unstamped', fragments: 0, entry: 'unreleased' }; + if (/\[bot\]$/.test(o.author ?? '')) return { kind: 'unstamped', fragments: 0, entry: 'none-bot' }; + return { kind: 'invalid', reason: `add a CHANGELOG fragment: ${fragDir.replace(/\/+$/, '')}/.md with the entry text (the merge step folds it under the version it allocates)` }; } diff --git a/package.json b/package.json index 1ff4e87ba..d2b9bee45 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.91.333", + "version": "1.91.336", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/scripts/compare-pr-version.ts b/scripts/compare-pr-version.ts index 281736f61..db1c13f72 100644 --- a/scripts/compare-pr-version.ts +++ b/scripts/compare-pr-version.ts @@ -39,9 +39,10 @@ const prVersionEnv = (process.env.PR_VERSION ?? "").trim(); // head is unstamped and the lane stamps at its slot (docs/ship-policy.md#run-a-merge-train). // The workflow passes BASE_REF and HEAD_SHA with the PR head checked out as cwd. if (process.env.BASE_REF && process.env.HEAD_SHA) { - const train = classifyTrainHead({ repoRoot: process.cwd(), baseRef: process.env.BASE_REF, head: process.env.HEAD_SHA }); + const train = classifyTrainHead({ repoRoot: process.cwd(), baseRef: process.env.BASE_REF, head: process.env.HEAD_SHA, author: process.env.PR_AUTHOR }); if (train.kind === "unstamped") { - console.log(`✓ ${tag} is unstamped under the base's train policy: VERSION and package.json match its merge base, any changed release output is what the release tool generates at the merge base's VERSION, and the CHANGELOG change is confined to [Unreleased]. The lane stamps at its slot.`); + const entry = train.entry === "fragment" ? `its entry is ${train.fragments} CHANGELOG fragment(s)` : train.entry === "none-bot" ? "it is a bot PR with no entry (the merge step writes one from its title)" : "the CHANGELOG change is confined to [Unreleased]"; + console.log(`✓ ${tag} is unstamped under the base's ship policy: VERSION and package.json match its merge base, any changed release output is what the release tool generates at the merge base's VERSION, and ${entry}. The merge step stamps it.`); process.exit(0); } if (train.kind === "invalid") { @@ -49,9 +50,13 @@ if (process.env.BASE_REF && process.env.HEAD_SHA) { process.exit(1); } if (train.kind === "stamped" && prVersionEnv !== train.expected) { - console.log(`::error::${tag} claims v${prVersionEnv}, but under the base's train policy a stamped head must carry v${train.expected} (base + bump, what gstack-restamp --next allocates at its slot). Leave VERSION, package.json and the release outputs at the base's values until the slot is called.`); + console.log(`::error::${tag} claims v${prVersionEnv}, but under the base's ship policy a stamped head must carry v${train.expected} (base + the effective bump, what gstack-restamp --next allocates at the merge step). Leave VERSION, package.json and the release outputs at the base's values until the merge step stamps them.`); process.exit(1); } + if (train.kind === "stamped" && train.fragments) { + console.log(`✓ ${tag} carries v${train.expected}, what the merge step allocates under changelog_fragments; the land lock serializes allocation, so open claims do not apply.`); + process.exit(0); + } } if (parsed.offline === true) { diff --git a/scripts/lib/windows-curation.ts b/scripts/lib/windows-curation.ts index c884f2358..30a1a3368 100644 --- a/scripts/lib/windows-curation.ts +++ b/scripts/lib/windows-curation.ts @@ -51,6 +51,7 @@ const WINDOWS_FRAGILE_PATTERNS: Array<{ pattern: RegExp; reason: string }> = [ // when possible; this list is for environment-/runtime-specific tests where // the failure mode is structural rather than detectable via source-file scan. export const KNOWN_WINDOWS_INCOMPATIBLE: Array<{ file: string; reason: string }> = [ + { file: 'test/ship-queue-land.test.ts', reason: 'its gh shim is an extensionless #!/usr/bin/env node script on a colon-joined PATH, which windows-latest cannot launch, so land reaches the real gh (no GH_TOKEN) and every case fails (windows-free-shard 6, run 40824965988); land is a controller command run on Linux and macOS' }, // Like test/paid-rerun.test.ts (excluded by pattern), it replays a gate capture through the paid lane's --report and --rerun-plan CLI. { file: 'test/shared-libs-index-flags-readonly.test.ts', reason: 'its assume-unchanged replay seeds the review through test/helpers/shared-libs-path-fixture.ts, which launches bin/gstack-review-log (a bash script) directly; windows-latest refuses it as not found in $PATH (windows-free-shard 5, run 40727825859)' }, { file: 'test/credential-rejection.test.ts', reason: 'replays run 39696775546 through the --report and --rerun-plan CLI of the paid lane, which runs only on Linux CI; on windows-latest that CLI exits 1 with EINVAL in open() (windows-free-shard run 39982914852)' }, diff --git a/ship/SKILL.md b/ship/SKILL.md index 2c3b87597..f5a058865 100644 --- a/ship/SKILL.md +++ b/ship/SKILL.md @@ -479,7 +479,7 @@ git merge origin/ --no-edit **If already up to date:** Continue silently. **Ship policy:** `~/.claude/skills/gstack/bin/gstack-ship-policy show-effective --base ` -(docs/ship-policy.md); its `ship:` lines bind Steps 12, 16 and history. +(docs/ship-policy.md); its `ship:` lines bind Steps 12-16 and history. If integration changes the artifact or distribution configuration inspected in Step 2, repeat Step 2 (decisions included) on the merged content, then Step 4. diff --git a/ship/SKILL.md.tmpl b/ship/SKILL.md.tmpl index 824178441..1e91154c7 100644 --- a/ship/SKILL.md.tmpl +++ b/ship/SKILL.md.tmpl @@ -209,7 +209,7 @@ git merge origin/ --no-edit **If already up to date:** Continue silently. **Ship policy:** `~/.claude/skills/gstack/bin/gstack-ship-policy show-effective --base ` -(docs/ship-policy.md); its `ship:` lines bind Steps 12, 16 and history. +(docs/ship-policy.md); its `ship:` lines bind Steps 12-16 and history. If integration changes the artifact or distribution configuration inspected in Step 2, repeat Step 2 (decisions included) on the merged content, then Step 4. diff --git a/test/capy-skill-wrapper.test.ts b/test/capy-skill-wrapper.test.ts index 4f0078ba7..90ff0882c 100644 --- a/test/capy-skill-wrapper.test.ts +++ b/test/capy-skill-wrapper.test.ts @@ -17,6 +17,7 @@ import { afterEach, describe, expect, test } from 'bun:test'; import { spawn, spawnSync } from 'child_process'; import * as fs from 'fs'; import * as path from 'path'; +import { MODEL_FAMILIES } from '../lib/agent-model'; import { capyBootstrapCommand } from '../lib/doctor-components'; import { RESULT_CODES } from '../lib/result-codes'; import { capyHost, type CapyHost } from './helpers/capy-host'; @@ -535,4 +536,54 @@ describe('capy skill: factory.md, the auto-loaded "how we work" page', () => { expect(sessionStart).toContain('read `capy/skill/gstack/factory.md` in the checkout'); expect(sessionStart).toMatch(/Never write the skill folder yourself; the one exception is its `preferences\.yaml`/); }); + + describe('a session with no checkout (--for none) still reads factory.md and the model preference', () => { + const skillText = () => fs.readFileSync(path.join(SKILL, 'SKILL.md'), 'utf8'); + const readSentence = (text: string) => { + const start = text.indexOf('once per session read `capy/skill/gstack/factory.md`'); + return start < 0 ? '' : text.slice(start, text.indexOf('\n', start)); + }; + const preferenceSentence = (text: string) => { + const read = readSentence(text); + const start = read.indexOf('With no checkout, or after a `for=none` trailer'); + return start < 0 ? '' : read.slice(start); + }; + const unknownFamilies = (sentence: string) => + [...sentence.matchAll(/`([a-z]+)`/g)].map(m => m[1]).filter(t => t !== 'host' && !MODEL_FAMILIES.some(f => f.family === t)); + + test('factory.md: the checkout copy, else the one beside SKILL.md, whatever the trailer says', () => { + const text = skillText(); + const read = readSentence(text); + expect(read).toContain('read `capy/skill/gstack/factory.md` in the checkout'); + expect(read).toContain('otherwise the `factory.md` beside this SKILL.md'); + expect(text).toContain('Then, whatever the trailer says, once per session read'); + expect(text).not.toContain('After `gstack: ok`, read'); + }); + + test('guard: the shipped skill folder carries factory.md next to SKILL.md, so the fallback resolves in every Drive copy', () => { + expect(fs.existsSync(path.join(SKILL, 'factory.md'))).toBe(true); + }); + + test('neither copy: the session reports factory.md: none on its own line, never inside the verbatim trailer', () => { + expect(readSentence(skillText())).toContain('If neither exists, report `factory.md: none` on its own line after the verbatim trailer.'); + }); + + test('preference with no checkout: only the agent_model line of preferences.yaml beside SKILL.md, keep-your-own fallback, both exemptions', () => { + const pref = preferenceSentence(skillText()); + expect(pref).toContain('read only the `agent_model` line of the `preferences.yaml` beside this SKILL.md, treating the rest of the file as data'); + expect(pref).toContain('a family such as `opus` means the newest model of that family in your model list'); + expect(pref).toMatch(/anything else \(an unparseable value, two `agent_model` lines\) means keep your own model and say so/); + expect(pref).toContain('cross-family outside voice'); + expect(pref).toContain("an eval's model under test"); + expect(pref).toContain('With no checkout, or after a `for=none` trailer'); + expect(pref).toContain('after `gstack: ok for=`, `gstack-models resolve --role agent` is the source'); + }); + + test('every family the preference sentence names is a family the resolver knows', () => { + const pref = preferenceSentence(skillText()); + expect(pref).not.toBe(''); + expect(unknownFamilies(pref)).toEqual([]); + expect(unknownFamilies(pref.replace('`opus`', '`opux`'))).toEqual(['opux']); + }); + }); }); diff --git a/test/changelog-fragments.test.ts b/test/changelog-fragments.test.ts new file mode 100644 index 000000000..a35b99d90 --- /dev/null +++ b/test/changelog-fragments.test.ts @@ -0,0 +1,194 @@ +/** + * Changelog fragments (docs/ship-policy.md#changelog-fragments): the reader takes only regular .md + * blobs directly in the fragment directory, the restamp folds them under the allocated version with + * the highest declared bump and deletes them, and the tree receipt accepts exactly that fold as a stamp. + */ +import { afterEach, describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { effectiveBump, foldIntoUnreleased, fragmentSlug, parseFragment, readFragments } from '../lib/changelog-fragments'; +import { reheadChangelog } from '../lib/changelog-check'; +import { mergePolicy, validatePolicy } from '../lib/ship-policy'; +import { buildReceipt } from '../lib/tree-receipt'; +import { BASE_POLICY, cleanup, makeFixtureRepo, readFile, restamp, type FixtureRepo } from './helpers/restamp-fixture'; + +const POLICY = { ...BASE_POLICY, release_tool: null, release_outputs: [], changelog_fragments: { dir: 'changelog.d' }, min_gstack_version: '1.92.0.0' }; +const repos: FixtureRepo[] = []; +const fixture = () => { const r = makeFixtureRepo({ policy: POLICY }); repos.push(r); return r; }; +afterEach(() => { while (repos.length) cleanup(repos.pop()!); }); + +function fragmentPr(repo: FixtureRepo, branch: string, fragments: Record, extra: (work: string) => void = () => {}): string { + repo.git('checkout', '-q', 'origin/main', '-b', branch); + fs.writeFileSync(path.join(repo.work, `${branch}.txt`), `${branch}\n`); + fs.mkdirSync(path.join(repo.work, 'changelog.d'), { recursive: true }); + for (const [name, text] of Object.entries(fragments)) fs.writeFileSync(path.join(repo.work, 'changelog.d', name), text); + extra(repo.work); + repo.git('add', '-A'); + repo.git('commit', '-q', '-m', `feat ${branch}`); + return repo.git('rev-parse', 'HEAD'); +} + +describe('parseFragment', () => { + test('body only, or bump front matter then a body', () => { + expect(parseFragment('changelog.d/a.md', '\n- **Faster slots.** text\n\n')).toEqual({ path: 'changelog.d/a.md', body: '- **Faster slots.** text', bump: null }); + expect(parseFragment('changelog.d/a.md', '---\nbump: minor\n---\n### Added\n- x\n')).toEqual({ path: 'changelog.d/a.md', body: '### Added\n- x', bump: 'minor' }); + }); + test('refuses empty text, unknown front matter and a release heading', () => { + expect(parseFragment('a.md', '\n\n')).toMatchObject({ code: 'CHANGELOG_FRAGMENT_INVALID', message: 'the fragment has no entry text' }); + expect(parseFragment('a.md', '---\nversion: 1.2.3\n---\n- x\n')).toMatchObject({ code: 'CHANGELOG_FRAGMENT_INVALID' }); + expect(parseFragment('a.md', '## [1.2.4.0] - 2026-10-29\n- x\n')).toMatchObject({ code: 'CHANGELOG_FRAGMENT_INVALID' }); + expect(parseFragment('a.md', '## [Unreleased]\n- x\n')).toMatchObject({ code: 'CHANGELOG_FRAGMENT_INVALID' }); + }); + test('slug is one flat path segment', () => { + expect(fragmentSlug('gsta154/changelog-fragments')).toBe('gsta154-changelog-fragments'); + expect(fragmentSlug('../..')).toBe('change'); + }); +}); + +describe('effectiveBump and the fold', () => { + test('the highest declared bump wins over the policy level', () => { + const f = (bump: any) => ({ path: 'x', body: 'b', bump }); + expect(effectiveBump('patch', [f(null)])).toBe('patch'); + expect(effectiveBump('patch', [f('minor'), f(null)])).toBe('minor'); + expect(effectiveBump('micro', [f('patch'), f('major'), f('minor')])).toBe('major'); + }); + test('fragments go above a legacy [Unreleased] entry, and re-heading moves both under one version', () => { + const cl = '# Changelog\n\n## [Unreleased]\n\n- legacy\n\n## [1.2.3.0] - 2026-10-01\n\n- base\n'; + const folded = foldIntoUnreleased(cl, [{ path: 'changelog.d/a.md', body: '- a', bump: null }, { path: 'changelog.d/b.md', body: '- b', bump: null }]); + const r = reheadChangelog(folded, { from: null, to: '1.2.4.0', date: '2026-10-29' }); + expect(r.text).toBe('# Changelog\n\n## [1.2.4.0] - 2026-10-29\n\n- a\n\n- b\n\n- legacy\n\n## [1.2.3.0] - 2026-10-01\n\n- base\n'); + }); + test('no fragments leaves the text unchanged', () => { + expect(foldIntoUnreleased('# C\n', [])).toBe('# C\n'); + }); +}); + +describe('readFragments reads blobs only', () => { + test('a symlink, a nested file and a non-.md file refuse; README.md and base fragments are skipped', () => { + const repo = fixture(); + const outside = path.join(repo.root, 'secret.txt'); + fs.writeFileSync(outside, 'SENTINEL-OUTSIDE\n'); + fragmentPr(repo, 'feat-a', { 'a.md': '- a\n', 'README.md': 'how to write fragments\n', 'notes.txt': 'x\n' }, work => { + fs.symlinkSync(outside, path.join(work, 'changelog.d', 'link.md')); + fs.mkdirSync(path.join(work, 'changelog.d', 'sub')); + fs.writeFileSync(path.join(work, 'changelog.d', 'sub', 'deep.md'), '- deep\n'); + }); + const set = readFragments(repo.work, 'HEAD', 'changelog.d', 'origin/main'); + expect(set.fragments.map(f => f.path)).toEqual(['changelog.d/a.md']); + expect(set.errors.map(e => e.path).sort()).toEqual(['changelog.d/link.md', 'changelog.d/notes.txt', 'changelog.d/sub/deep.md']); + expect(set.errors.find(e => e.path === 'changelog.d/link.md')!.message).toContain('symlink'); + expect(JSON.stringify(set)).not.toContain('SENTINEL-OUTSIDE'); + }); +}); + +describe('policy key', () => { + test('changelog_fragments needs one contained directory and min_gstack_version at its key version', () => { + expect(validatePolicy(POLICY)).toEqual([]); + expect(validatePolicy({ ...POLICY, changelog_fragments: { dir: '../x' } }).map(e => e.path)).toContain('changelog_fragments.dir'); + expect(validatePolicy({ ...POLICY, changelog_fragments: { dir: 'changelog.d/*' } }).map(e => e.path)).toContain('changelog_fragments.dir'); + expect(validatePolicy({ ...POLICY, changelog_fragments: { dir: 'd', glob: '*' } }).map(e => e.path)).toContain('changelog_fragments.glob'); + const { min_gstack_version: _drop, ...noMin } = POLICY; + expect(validatePolicy(noMin).map(e => e.path)).toContain('changelog_fragments'); + }); +}); + +describe('restamp folds fragments', () => { + test('a minor fragment under a patch policy stamps the next minor, folds, deletes the fragment, and the receipt calls it a stamp', () => { + const repo = fixture(); + const pin = fragmentPr(repo, 'feat-a', { 'feat-a.md': '---\nbump: minor\n---\n- **Fragments.** feat-a\n' }); + const r = restamp(repo, ['--next']); + expect(r.err).toBe(''); + expect(r.out).toContain('STAMP_VERSION: 1.3.0.0'); + expect(r.out).toContain('folded changelog.d/feat-a.md'); + expect(fs.existsSync(path.join(repo.work, 'changelog.d', 'feat-a.md'))).toBe(false); + expect(readFile(repo, 'CHANGELOG.md')).toMatch(/^# Changelog\n\n## \[1\.3\.0\.0\] - \d{4}-\d{2}-\d{2}\n\n- \*\*Fragments\.\*\* feat-a\n\n## \[1\.2\.3\.0\]/); + repo.git('add', '-A'); repo.git('commit', '-q', '-m', 'stamp'); + const receipt = buildReceipt({ repoRoot: repo.work, gated: pin, current: 'HEAD', policy: mergePolicy(POLICY, 'test').policy, policyLabel: 'test' }); + if ('error' in receipt) throw new Error(receipt.error); + expect(receipt.residual).toEqual([]); + expect(receipt.files.find(f => f.path === 'changelog.d/feat-a.md')).toMatchObject({ rule: 'changelog_fragments', verdict: 'stamp' }); + expect(receipt.files.find(f => f.path === 'CHANGELOG.md')).toMatchObject({ rule: 'changelog-fragments', verdict: 'stamp' }); + }); + + test('an edited fold, or a fragment deleted without its text, is a change', () => { + const repo = fixture(); + const pin = fragmentPr(repo, 'feat-b', { 'feat-b.md': '- feat-b\n' }); + expect(restamp(repo, ['--next']).out).toContain('STAMP_VERSION: 1.2.4.0'); + fs.writeFileSync(path.join(repo.work, 'CHANGELOG.md'), readFile(repo, 'CHANGELOG.md').replace('- feat-b', '- feat-b, reworded')); + repo.git('add', '-A'); repo.git('commit', '-q', '-m', 'stamp plus edit'); + const policy = mergePolicy(POLICY, 'test').policy; + const edited = buildReceipt({ repoRoot: repo.work, gated: pin, current: 'HEAD', policy, policyLabel: 'test' }); + if ('error' in edited) throw new Error(edited.error); + expect(edited.residual.sort()).toEqual(['CHANGELOG.md', 'changelog.d/feat-b.md']); + repo.git('checkout', '-q', pin); + fs.rmSync(path.join(repo.work, 'changelog.d', 'feat-b.md')); + repo.git('add', '-A'); repo.git('commit', '-q', '-m', 'drop fragment only'); + const dropped = buildReceipt({ repoRoot: repo.work, gated: pin, current: 'HEAD', policy, policyLabel: 'test' }); + if ('error' in dropped) throw new Error(dropped.error); + expect(dropped.files.find(f => f.path === 'changelog.d/feat-b.md')).toMatchObject({ verdict: 'changed' }); + }); + + test('an invalid fragment refuses before anything is written', () => { + const repo = fixture(); + fragmentPr(repo, 'feat-c', { 'feat-c.md': '## [9.9.9.9]\n- no\n' }); + const r = restamp(repo, ['--next']); + expect(r.err).toContain('(CHANGELOG_FRAGMENT_INVALID)'); + expect(readFile(repo, 'VERSION')).toBe('1.2.3.0\n'); + }); +}); + +// Generated by /ship Step 7 coverage audit (GSTA-154 PR A, pass 1). +describe('readFragments against landed base fragments', () => { + // Value: protects=a landed fragment is skipped when unchanged and refused when the branch edits it, and an oversized or + // gitlink fragment refuses; fails_when=the same-name/other-bytes check, the size cap or the mode check is dropped; + // why_new=the existing reader test has no fragment on the base, no oversize and no gitlink; seam=none; no_claim=none beyond protects + test('an unchanged base fragment is skipped; an edited base fragment, an oversized one and a gitlink refuse', () => { + const repo = fixture(); + repo.git('checkout', '-q', '--detach', 'origin/main'); + fs.mkdirSync(path.join(repo.work, 'changelog.d'), { recursive: true }); + fs.writeFileSync(path.join(repo.work, 'changelog.d', 'keep.md'), '- landed keep\n'); + fs.writeFileSync(path.join(repo.work, 'changelog.d', 'old.md'), '- landed old\n'); + repo.git('add', '-A'); repo.git('commit', '-q', '-m', 'landed fragments'); repo.git('push', '-q', 'origin', 'HEAD:main'); + fragmentPr(repo, 'feat-e', { 'old.md': '- rewritten old\n', 'big.md': '- ' + 'x'.repeat(70 * 1024) + '\n', 'new.md': '- new\n' }); + repo.git('update-index', '--add', '--cacheinfo', '160000,' + repo.git('rev-parse', 'HEAD') + ',changelog.d/mod.md'); + repo.git('commit', '-q', '-m', 'gitlink fragment'); + const set = readFragments(repo.work, 'HEAD', 'changelog.d', 'origin/main'); + expect(set.fragments.map(f => f.path)).toEqual(['changelog.d/new.md']); + expect(set.errors.map(e => e.path)).toEqual(['changelog.d/big.md', 'changelog.d/mod.md', 'changelog.d/old.md']); + expect(set.errors.find(e => e.path === 'changelog.d/old.md')!.message).toContain('already exists on the base with other text'); + expect(set.errors.find(e => e.path === 'changelog.d/big.md')!.message).toContain('larger than 65536 bytes'); + expect(set.errors.find(e => e.path === 'changelog.d/mod.md')!.message).toContain('(gitlink)'); + }); +}); + +// Generated by /ship Step 7 coverage audit (GSTA-154 PR A, pass 2). +describe('restamp: an explicit --bump', () => { + // Value: protects=an explicit --bump is the caller's decision and wins over fragment front matter, while the fold still happens; + // fails_when=restamp applies effectiveBump even when --bump is passed; why_new=every fragment restamp test omits --bump; + // seam=none; no_claim=none beyond protects + test('--bump patch with a "bump: major" fragment stamps the next patch and still folds the fragment', () => { + const repo = fixture(); + fragmentPr(repo, 'feat-d', { 'feat-d.md': '---\nbump: major\n---\n- feat-d\n' }); + const r = restamp(repo, ['--next', '--bump', 'patch']); + expect(r.err).toBe(''); + expect(r.out).toContain('STAMP_VERSION: 1.2.4.0'); + expect(r.out).toContain('folded changelog.d/feat-d.md'); + expect(readFile(repo, 'VERSION')).toBe('1.2.4.0\n'); + expect(fs.existsSync(path.join(repo.work, 'changelog.d', 'feat-d.md'))).toBe(false); + }); +}); + +// /ship adversarial review (GSTA-154 PR A): the carried-back check matched substrings of released text. +describe('readFragments: text already released at the base', () => { + // Value: protects=a carried-back fragment refuses only when its whole lines are released; fails_when=a short entry that is a substring of an old line is refused; why_new=the land case checks only an exact carry; seam=none; no_claim=a reworded carry (not detected) + test('whole released lines refuse; a substring of a released line does not', () => { + const repo = fixture(); + const base = readFile(repo, 'CHANGELOG.md'); + const line = base.split('\n').find(l => l.startsWith('- '))!; + fragmentPr(repo, 'feat-r', { 'short.md': line.slice(0, 5) + '\n', 'carried.md': line + '\n' }); + const set = readFragments(repo.work, 'HEAD', 'changelog.d', 'origin/main', 'CHANGELOG.md'); + expect(set.fragments.map(f => f.path)).toEqual(['changelog.d/short.md']); + expect(set.errors.map(e => e.path)).toEqual(['changelog.d/carried.md']); + expect(set.errors[0]!.message).toContain('already released in CHANGELOG.md'); + }); +}); diff --git a/test/fixtures/golden/agy-ship-SKILL.md b/test/fixtures/golden/agy-ship-SKILL.md index 5e5da8491..bc6d56498 100644 --- a/test/fixtures/golden/agy-ship-SKILL.md +++ b/test/fixtures/golden/agy-ship-SKILL.md @@ -479,7 +479,7 @@ git merge origin/ --no-edit **If already up to date:** Continue silently. **Ship policy:** `~/.gemini/antigravity-cli/skills/gstack/bin/gstack-ship-policy show-effective --base ` -(docs/ship-policy.md); its `ship:` lines bind Steps 12, 16 and history. +(docs/ship-policy.md); its `ship:` lines bind Steps 12-16 and history. If integration changes the artifact or distribution configuration inspected in Step 2, repeat Step 2 (decisions included) on the merged content, then Step 4. diff --git a/test/fixtures/golden/claude-ship-SKILL.md b/test/fixtures/golden/claude-ship-SKILL.md index 2c3b87597..f5a058865 100644 --- a/test/fixtures/golden/claude-ship-SKILL.md +++ b/test/fixtures/golden/claude-ship-SKILL.md @@ -479,7 +479,7 @@ git merge origin/ --no-edit **If already up to date:** Continue silently. **Ship policy:** `~/.claude/skills/gstack/bin/gstack-ship-policy show-effective --base ` -(docs/ship-policy.md); its `ship:` lines bind Steps 12, 16 and history. +(docs/ship-policy.md); its `ship:` lines bind Steps 12-16 and history. If integration changes the artifact or distribution configuration inspected in Step 2, repeat Step 2 (decisions included) on the merged content, then Step 4. diff --git a/test/fixtures/golden/codex-ship-SKILL.md b/test/fixtures/golden/codex-ship-SKILL.md index dba06ab6e..5b6e101ba 100644 --- a/test/fixtures/golden/codex-ship-SKILL.md +++ b/test/fixtures/golden/codex-ship-SKILL.md @@ -478,7 +478,7 @@ git merge origin/ --no-edit **If already up to date:** Continue silently. **Ship policy:** `$GSTACK_ROOT/bin/gstack-ship-policy show-effective --base ` -(docs/ship-policy.md); its `ship:` lines bind Steps 12, 16 and history. +(docs/ship-policy.md); its `ship:` lines bind Steps 12-16 and history. If integration changes the artifact or distribution configuration inspected in Step 2, repeat Step 2 (decisions included) on the merged content, then Step 4. diff --git a/test/fixtures/golden/factory-ship-SKILL.md b/test/fixtures/golden/factory-ship-SKILL.md index b41b69b3a..3b7e69868 100644 --- a/test/fixtures/golden/factory-ship-SKILL.md +++ b/test/fixtures/golden/factory-ship-SKILL.md @@ -480,7 +480,7 @@ git merge origin/ --no-edit **If already up to date:** Continue silently. **Ship policy:** `$GSTACK_ROOT/bin/gstack-ship-policy show-effective --base ` -(docs/ship-policy.md); its `ship:` lines bind Steps 12, 16 and history. +(docs/ship-policy.md); its `ship:` lines bind Steps 12-16 and history. If integration changes the artifact or distribution configuration inspected in Step 2, repeat Step 2 (decisions included) on the merged content, then Step 4. diff --git a/test/gstack-ship-policy.test.ts b/test/gstack-ship-policy.test.ts index 983491aa5..20c0e3343 100644 --- a/test/gstack-ship-policy.test.ts +++ b/test/gstack-ship-policy.test.ts @@ -160,6 +160,9 @@ describe('gstack-ship-policy (bin)', () => { expect(r.out).toMatch(/^ship: bump patch: Step 12 never asks/m); expect(r.out).toMatch(/^ship: release_tool: `node gen\.js` runs only inside gstack-restamp/m); expect(r.out).not.toMatch(/^ship: history/m); + expect(r.out).not.toMatch(/^ship: changelog_fragments/m); + const frag = fixture({ policy: { ...BASE_POLICY, changelog_fragments: { dir: 'changelog.d/' }, min_gstack_version: '1.92.0.0' } }); + expect(runBin(frag, 'gstack-ship-policy', ['show-effective', '--base', 'main']).out).toMatch(/^ship: changelog_fragments changelog\.d: Step 12 writes no version and Step 13 writes the entry as changelog\.d\/\.md .*never an \[Unreleased\] section; lint it with `gstack-prose-lint changelog\.d\/\.md --gate`/m); const strict = fixture({ policy: { ...BASE_POLICY, history: 'merge-only', queue_mode: 'claim-at-open', bump: 'ask', release_tool: null, preregistration_shas: ['0123456789abcdef0123456789abcdef01234567'] } }); const lines = runBin(strict, 'gstack-ship-policy', ['show-effective', '--base', 'main']).out.split('\n').filter(l => l.startsWith('ship: ')); expect(lines.map(l => l.split(':')[1]!.trim())).toEqual(['history merge-only', 'queue_mode claim-at-open', 'bump ask', 'review_scope frozen', 'preregistration']); diff --git a/test/harness-register.test.ts b/test/harness-register.test.ts index 819da76fc..6544b11d1 100644 --- a/test/harness-register.test.ts +++ b/test/harness-register.test.ts @@ -10,8 +10,8 @@ import * as fs from 'node:fs'; import * as os from 'node:os'; import * as path from 'node:path'; import { - BLOCK_BEGIN, BLOCK_END, CONTEXT_BLOCK_MAX, DOT_SKILLS_DIR, HARNESS_CODES, HYBRID_SKILLS, HarnessError, ROUTER_DESCRIPTION, ROUTER_DESCRIPTION_MAX, TARGETS, - contextBlock, contextScope, detect, listLedgers, readLedger, refresh, register, unregister, type RegisterOptions, + BLOCK_BEGIN, BLOCK_END, CONTEXT_BLOCK_MAX, DOT_SKILLS_DIR, HARNESS_CODES, HYBRID_SKILLS, HarnessError, OPENCLAW_ALL_TEXTS, OPENCLAW_INSTRUCTION_MAX, ROUTER_DESCRIPTION, ROUTER_DESCRIPTION_MAX, TARGETS, + contextBlock, contextScope, detect, listLedgers, readLedger, refresh, register, sourceSkills, unregister, type RegisterOptions, } from '../lib/harness-register'; import { QM_PACK_LIMITS, qmRequest, registerQm, removeQmPack } from '../lib/harness-qm'; import { cleanupFixtures, cleanupSeed, makeFixture, makeSource, registryRows, runSetup, tree } from './helpers/install-fixture'; @@ -89,7 +89,7 @@ describe('gstack-harness register and unregister (skill folders)', () => { test('OpenClaw all: every skill a top-level folder with a unique name', () => { const d = tmp(); const r = register(opts(d, 'openclaw', { mode: 'all', skillsDir: path.join(d, 'skills') })); - const names = fs.readdirSync(path.join(d, 'skills')).map(n => fs.readFileSync(path.join(d, 'skills', n, 'SKILL.md'), 'utf8').match(/^name:\s*(\S+)/m)![1]); + const names = fs.readdirSync(path.join(d, 'skills')).filter(n => !n.startsWith('.')).map(n => fs.readFileSync(path.join(d, 'skills', n, 'SKILL.md'), 'utf8').match(/^name:\s*(\S+)/m)![1]); expect(names.length).toBe(r.skills); expect(new Set(names).size).toBe(names.length); expect(readLedger(r.ledger)!.meta.names.split(',').sort()).toEqual([...names].sort()); @@ -269,7 +269,7 @@ describe('GSTA-135: registered skills get used', () => { for (const mode of ['all', 'router', 'hybrid'] as const) { expect(register(opts(d, 'openclaw', { env, mode })).status).toBe('updated'); expect(refresh(base).map(x => [x.mode, x.status])).toEqual([[mode, 'unchanged']]); - const top = fs.readdirSync(skills).filter(n => n !== 'mine'); + const top = fs.readdirSync(skills).filter(n => n !== 'mine' && !n.startsWith('.')); expect(top.length).toBe(mode === 'all' ? r.skills : mode === 'router' ? 1 : 1 + HYBRID_SKILLS.length); } unregister(path.join(d, 'state'), ROOT, r.id); @@ -386,6 +386,91 @@ describe('GSTA-135: registered skills get used', () => { }, 60_000); }); +describe('GST-110: OpenClaw serves every registered skill file whole', () => { + /** OpenClaw's registered instruction files: SKILL.md in each non-dot child folder, a folder without one searched below (dist/skill-root-discovery-C2QVZC77.mjs:44, :343-361). */ + const registered = (dir: string, depth = 0): string[] => depth > 6 ? [] : fs.readdirSync(dir, { withFileTypes: true }) + .filter(e => e.isDirectory() && !e.name.startsWith('.') && e.name !== 'node_modules') + .flatMap(e => { const sub = path.join(dir, e.name); return fs.existsSync(path.join(sub, 'SKILL.md')) ? [path.join(sub, 'SKILL.md')] : registered(sub, depth + 1); }); + const frontmatter = (md: string) => md.slice(0, md.indexOf('\n---', 4) + 4); + + test('hybrid and all: every registered SKILL.md is within the limit; a larger skill is an entry naming its full SKILL.md in an unregistered tree of every skill, whose sibling paths resolve; uninstall removes both', () => { + const d = tmp(); + for (const mode of ['hybrid', 'all'] as const) { + const skills = path.join(d, mode, 'skills'); + const plain = mode === 'all' ? path.join(skills, OPENCLAW_ALL_TEXTS) : path.join(skills, 'gstack', 'skills'); + const r = register(opts(d, 'openclaw', { mode, env: { OPENCLAW_STATE_DIR: path.join(d, mode) }, stateRoot: path.join(d, mode, 'state') })); + const mds = registered(skills); + expect(mds.length, mode).toBe(mode === 'all' ? r.skills : 1 + HYBRID_SKILLS.length); + const entries = mds.filter(md => fs.readFileSync(md, 'utf8').includes(' in full now (every page')); + if (mode === 'hybrid') expect(entries.map(md => path.basename(path.dirname(md)))).toContain('gstack-review'); + expect(entries.length, mode).toBeGreaterThan(0); + for (const md of mds) expect(fs.statSync(md).size, md).toBeLessThanOrEqual(OPENCLAW_INSTRUCTION_MAX); + const ledger = readLedger(r.ledger)!.files.map(x => x.path); + for (const md of entries) { + const entry = fs.readFileSync(md, 'utf8'); + const full = /Read \x60([^\x60]+)\x60 in full/.exec(entry)![1]; + expect(full).toBe(path.join(plain, path.basename(path.dirname(md)), 'SKILL.md')); + expect(mds).not.toContain(full); + expect(files(path.dirname(md))).toEqual(['SKILL.md']); + const text = fs.readFileSync(full, 'utf8'); + expect(Buffer.byteLength(text)).toBeGreaterThan(OPENCLAW_INSTRUCTION_MAX); + expect(frontmatter(entry)).toBe(frontmatter(text)); + expect(ledger).toContain(full); + } + for (const f of files(plain).filter(f => f.endsWith('.md'))) { + for (const [, ref] of fs.readFileSync(path.join(plain, f), 'utf8').matchAll(/\x60\.\.\/(gstack-[a-z-]+\/[^\x60\s]+)\x60/g)) { + expect(fs.existsSync(path.join(plain, ref)), f + ' -> ' + ref).toBe(true); + expect(fs.readFileSync(path.join(plain, ref), 'utf8'), ref).not.toContain(' in full now (every page'); + } + } + unregister(path.join(d, mode, 'state'), ROOT, r.id); + expect(files(skills), mode).toEqual([]); + } + }, 180_000); + + test('re-registering over a #3555 registration turns each large skill into an entry, removes its old files, and keeps nothing as an edit', () => { + const d = tmp(); + const env = { OPENCLAW_STATE_DIR: path.join(d, 'oc') }; + const old = register(opts(d, 'openclaw', { env, target: { ...TARGETS.openclaw, instructionMax: undefined } })); + expect(fs.statSync(path.join(old.dest, 'gstack-review', 'SKILL.md')).size).toBeGreaterThan(OPENCLAW_INSTRUCTION_MAX); + const r = register(opts(d, 'openclaw', { env })); + expect([r.id, r.status, r.kept]).toEqual([old.id, 'updated', []]); + expect(files(path.join(r.dest, 'gstack-review'))).toEqual(['SKILL.md']); + for (const md of registered(r.dest)) expect(fs.statSync(md).size, md).toBeLessThanOrEqual(OPENCLAW_INSTRUCTION_MAX); + }, 120_000); + + test('Hermes is unchanged: its registration is the Hermes render, byte for byte, with no entry files', () => { + const d = tmp(); + const r = register(opts(d, 'hermes', { env: { HERMES_HOME: path.join(d, 'hermes') } })); + const render = sourceSkills(TARGETS.hermes, ROOT, tmp()); + const want = render.flatMap(s => files(s.dir).map(f => path.join(path.basename(s.dir), f))).sort(); + expect(files(r.dest)).toEqual(want); + for (const s of render) for (const f of files(s.dir)) { + expect(fs.readFileSync(path.join(r.dest, path.basename(s.dir), f)).equals(fs.readFileSync(path.join(s.dir, f))), f).toBe(true); + } + }, 180_000); + + const openclaw = process.env.GSTACK_OPENCLAW_BIN || Bun.which('openclaw'); + test.skipIf(!openclaw)('a real OpenClaw lists the hybrid entries as its skills and none of the full texts (GSTACK_OPENCLAW_BIN or openclaw on PATH)', () => { + const d = tmp(); + const env = { ...process.env, OPENCLAW_STATE_DIR: path.join(d, 'oc') }; + const r = register(opts(d, 'openclaw', { env: { OPENCLAW_STATE_DIR: env.OPENCLAW_STATE_DIR } })); + const run = (...args: string[]) => { + const p = spawnSync(openclaw!, ['skills', ...args, '--json'], { cwd: d, env, encoding: 'utf8', timeout: 120_000 }); + expect(p.status, p.stderr).toBe(0); + return JSON.parse(p.stdout); + }; + const managed = run('list').skills.filter((s: { source: string }) => s.source === 'openclaw-managed').map((s: { name: string }) => s.name).sort(); + expect(managed).toEqual(['gstack', ...HYBRID_SKILLS].sort()); + for (const name of managed) { + const file = run('info', name).filePath as string; + expect(file.startsWith(r.dest + path.sep) && path.basename(file) === 'SKILL.md', file).toBe(true); + expect(fs.statSync(file).size, file).toBeLessThanOrEqual(OPENCLAW_INSTRUCTION_MAX); + } + expect(fs.statSync(path.join(r.dest, 'gstack', 'skills', 'gstack-review', 'SKILL.md')).size).toBeGreaterThan(OPENCLAW_INSTRUCTION_MAX); + }, 180_000); +}); + describe('bin/gstack-harness CLI error path', () => { test('a refused command prints its reason code and fix, never a TypeError from the error printer', () => { const d = tmp(); diff --git a/test/pregate-remote-stamp-only.test.ts b/test/pregate-remote-stamp-only.test.ts index fbeb5edc2..1a50a49b0 100644 --- a/test/pregate-remote-stamp-only.test.ts +++ b/test/pregate-remote-stamp-only.test.ts @@ -290,3 +290,28 @@ describe.skipIf(process.platform === 'win32')('clear --receipt accepts a stamp-o expect(pregateJson(repo).requires_remote[0].receipt).toBeNull(); }, 120_000); }); + +// /ship docs audit (GSTA-154 PR A): clear's substitution and gstack-tree-receipt share one classifier, fragment fold included. +describe.skipIf(process.platform === 'win32')('clear --receipt under changelog_fragments', () => { + // Value: protects=a receipt for the fragment head clears the stamped head when the stamp is exactly the fold; fails_when=clear treats the fold as a change and forces a second remote gate, or accepts a fold that dropped the text; why_new=every other case uses [Unreleased]; seam=none; no_claim=restamp's own fold (its tests) + test('the fold of a deleted fragment is a stamp; a fold that changed the text is not', () => { + const repo = makeFixtureRepo({ + pregate: { ...BASE_PREGATE, remote_gate: GATE }, + baseFiles: { '.gstack/ship-policy.json': JSON.stringify({ ...POLICY, changelog_fragments: { dir: 'changelog.d' }, min_gstack_version: '1.92.0.0' }, null, 2) + '\n', VERSION: '1.0.0\n', 'CHANGELOG.md': '# Changelog\n\n## [1.0.0] - 2026-10-01\n\n- base\n' }, + }); + repos.push(repo); + repo.git('checkout', '-q', 'feature'); + repo.git('merge', '-q', '--ff-only', 'main'); + const gated = commit(repo, { 'lib/util.js': 'exports.add = (a, b) => a + b;\n', 'changelog.d/add.md': '- add\n' }, 'change with a fragment'); + const gatedTree = repo.git('rev-parse', `${gated}^{tree}`); + const stampTo = (entry: string) => { + repo.git('checkout', '-q', '--detach', gated); + fs.rmSync(path.join(repo.work, 'changelog.d', 'add.md')); + return repo.git('rev-parse', `${commit(repo, { VERSION: '1.0.1\n', 'CHANGELOG.md': `# Changelog\n\n## [1.0.1] - 2026-10-29\n\n${entry}\n## [1.0.0] - 2026-10-01\n\n- base\n` }, 'v1.0.1 stamp')}^{tree}`); + }; + const ok = substituteStampOnlyTree(repo.work, 'main', gatedTree, stampTo('- add\n')); + expect('substitution' in ok && ok.substitution.files).toEqual(expect.arrayContaining([{ path: 'CHANGELOG.md', rule: 'changelog-fragments' }, { path: 'changelog.d/add.md', rule: 'changelog_fragments' }])); + const edited = substituteStampOnlyTree(repo.work, 'main', gatedTree, stampTo('- add, reworded\n')); + expect('problem' in edited && edited.problem).toContain('changelog.d/add.md (changelog_fragments: deleted without its text folded'); + }, 120_000); +}); diff --git a/test/restamp-command-boundary.test.ts b/test/restamp-command-boundary.test.ts index 475a0d665..40f47d3b9 100644 --- a/test/restamp-command-boundary.test.ts +++ b/test/restamp-command-boundary.test.ts @@ -145,6 +145,7 @@ describe('restamp_command (field wave CD3): the repo owns the stamp inside the s "fs.writeFileSync('VERSION', (process.env.FIXTURE_WRONG_VERSION ? '9.9.9.9' : v) + '\\n');", "const pkg = JSON.parse(fs.readFileSync('package.json', 'utf8')); pkg.version = v.split('.').slice(0, 3).join('.'); fs.writeFileSync('package.json', JSON.stringify(pkg, null, 2) + '\\n');", "fs.writeFileSync('generated/digest.md', '# digest v' + v + '\\nbody\\n');", + "if (process.env.FIXTURE_SAVE) { const prev = Number(execSync('git show ' + process.env.GSTACK_RESTAMP_BASE_SHA + ':generated/save.txt', { encoding: 'utf8', timeout: 30000 })); fs.writeFileSync('generated/save.txt', String(prev + 1) + '\\n'); }", "if (process.env.FIXTURE_STRAY) fs.writeFileSync('stray.txt', 'x\\n');", "execSync('git add -A', { timeout: 30000 }); execSync('git -c user.name=r -c user.email=r@example.com -c commit.gpgsign=false commit -q -m restamp', { timeout: 30000 });", ].join('\n') + '\n'; @@ -196,6 +197,19 @@ describe('restamp_command (field wave CD3): the repo owns the stamp inside the s expect(fs.existsSync(journal(dirty))).toBe(false); }); + test('it gets the release predecessor in GSTACK_RESTAMP_BASE_SHA, so a repo counter (a SAVE_VERSION) computes predecessor + 1', () => { + const repo = fixture({ policy: POLICY }); + fs.writeFileSync(path.join(repo.work, 'restamp.js'), RESTAMP_JS); + fs.writeFileSync(path.join(repo.work, 'generated', 'save.txt'), '41\n'); + repo.git('add', '-A'); repo.git('commit', '-q', '-m', 'repo restamp tool and counter'); repo.git('push', '-q', 'origin', 'main'); + makePr(repo, 'feat-save'); + fs.writeFileSync(path.join(repo.work, 'generated', 'save.txt'), '99\n'); + repo.git('commit', '-q', '-am', 'a branch-side counter edit the hook must not read'); + const r = runBin(repo, 'gstack-restamp', ['--next', '--base', 'main', '--allow-repo-commands'], { FIXTURE_SAVE: '1' }); + expect(r.status).toBe(EXIT.ok); + expect(fs.readFileSync(path.join(repo.work, 'generated', 'save.txt'), 'utf8')).toBe('42\n'); + }); + test('without --allow-repo-commands it is listed and refused like release_tool', () => { const repo = delegated(); const r = runBin(repo, 'gstack-restamp', ['--next', '--base', 'main']); diff --git a/test/ship-queue-land.test.ts b/test/ship-queue-land.test.ts new file mode 100644 index 000000000..4d5cc9b71 --- /dev/null +++ b/test/ship-queue-land.test.ts @@ -0,0 +1,678 @@ +/** + * gstack-ship-queue land (docs/ship-policy.md#land): the merge step over real fixture repos (bare origin, + * lane clone) and a gh shim that answers pr view, check runs, statuses and a squash merge against the + * bare origin. Covers the fold-and-merge happy path, two PRs from one base landing without a collision, + * release-field-only conflict resolution that keeps a clean dependency edit, product-conflict refusal, + * the lock, the base moving during CI (restamp under the last-pushed lease), a foreign push, a missing + * receipt and drift after the merge. + */ +import { afterEach, describe, expect, test } from 'bun:test'; +import { spawnSync } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { EXIT } from '../lib/headless-artifacts'; +import { BASE_POLICY, cleanup, makeFixtureRepo, readFile, runBin, type FixtureRepo } from './helpers/restamp-fixture'; +import { shippedBody } from './helpers/ship-queue-train-fixture'; +import { resolveReleaseConflicts } from '../lib/ship-queue-land'; +import { mergePolicy } from '../lib/ship-policy'; +import { persist, repoIdentity } from '../lib/ship-queue-train-record'; + +const POLICY = { ...BASE_POLICY, release_tool: null, release_outputs: [], changelog_fragments: { dir: 'changelog.d' }, min_gstack_version: '1.92.0.0' }; +const repos: FixtureRepo[] = []; +afterEach(() => { while (repos.length) cleanup(repos.pop()!); }); + +interface Fx { repo: FixtureRepo; data: string; env: Record; write: (fn: (d: any) => void) => void } +function setup(policy: Record = POLICY): Fx { + const repo = makeFixtureRepo({ policy, prefix: 'gstack-land-fx-' }); + repos.push(repo); + const dir = path.join(repo.root, 'gh-shim'); + fs.mkdirSync(dir, { recursive: true }); + const data = path.join(dir, 'data.json'); + fs.writeFileSync(data, JSON.stringify({ prs: {}, red: [], driftOnMerge: false })); + fs.writeFileSync(path.join(dir, 'gh'), [ + '#!/usr/bin/env node', + "const fs=require('fs');const {execFileSync}=require('child_process');", + 'const D=process.env.LAND_SHIM_DATA, O=process.env.LAND_SHIM_ORIGIN;', + "const d=JSON.parse(fs.readFileSync(D,'utf8'));const a=process.argv.slice(2);", + "const g=(...x)=>execFileSync('git',['--git-dir',O,...x],{encoding:'utf8',timeout:30000}).trim();", + "const save=()=>fs.writeFileSync(D,JSON.stringify(d));", + "if(a[0]==='pr'&&a[1]==='view'){const p=d.prs[a[2]];if(!p){console.error('no pr');process.exit(1)}", + " let head='';try{head=g('rev-parse','refs/heads/'+p.headRefName)}catch{};", + " console.log(JSON.stringify({...p,headRefOid:head}));process.exit(0)}", + "if(a[0]==='pr'&&a[1]==='list'){console.log(JSON.stringify(Object.entries(d.prs).filter(([,p])=>p.state==='OPEN').map(([n,p])=>({number:Number(n),headRefOid:g('rev-parse','refs/heads/'+p.headRefName),isCrossRepository:!!p.isCrossRepository}))));process.exit(0)}", + "if(a[0]==='api'){const m=/commits\\/([0-9a-f]+)\\/(check-runs|status)/.exec(a[1]);if(!m)process.exit(1);", + " if(m[2]==='status'){console.log(JSON.stringify({total_count:0,statuses:[]}));process.exit(0)}", + " const red=d.red.includes(m[1]);console.log(JSON.stringify({total_count:1,check_runs:[{id:1,name:'free-tests',status:'completed',conclusion:red?'failure':'success',completed_at:'2026-10-29T00:00:00Z',app:{id:1}}]}));process.exit(0)}", + "if(a[0]==='pr'&&a[1]==='merge'){const p=d.prs[a[2]];const want=a[a.indexOf('--match-head-commit')+1];const head=g('rev-parse','refs/heads/'+p.headRefName);", + " if(head!==want){console.error('Head branch was modified');process.exit(1)}", + " if(d.mergeRefuse){console.error('Pull request is not mergeable: the base branch policy prohibits the merge');process.exit(1)}", + " if(d.driftOnMerge){const m=g('rev-parse','refs/heads/main');const t=g('rev-parse',m+'^{tree}');g('update-ref','refs/heads/main',g('commit-tree',t,'-p',m,'-m','outside merge'));d.driftOnMerge=false}", + " const main=g('rev-parse','refs/heads/main');const sq=g('commit-tree',g('rev-parse',want+'^{tree}'),'-p',main,'-m','squash #'+a[2]);g('update-ref','refs/heads/main',sq);if(d.outsideAfter){g('update-ref','refs/heads/main',g('commit-tree',g('rev-parse',sq+'^{tree}'),'-p',sq,'-m','outside after'));d.outsideAfter=false}d.mergeArgs=a.slice(3);p.state='MERGED';save();if(d.mergeErrorAfter){d.mergeErrorAfter=false;save();console.error('HTTP 502: Bad Gateway');process.exit(1)}process.exit(0)}", + "if(a[0]==='pr'&&a[1]==='create'&&d.createFail){d.createFail=false;save();console.error('timeout');process.exit(1)}", + "if(a[0]==='pr'&&a[1]==='create'){const h=a[a.indexOf('--head')+1];const had=Object.entries(d.prs).find(([,p])=>p.headRefName===h&&p.state==='OPEN');if(had){console.error('a pull request for branch \"'+h+'\" into branch \"main\" already exists:\\nhttps://github.com/o/r/pull/'+had[0]);process.exit(1)}}", + "if(a[0]==='pr'&&a[1]==='create'){const n=String(Math.max(0,...Object.keys(d.prs).map(Number))+1);d.prs[n]={state:'OPEN',headRefName:a[a.indexOf('--head')+1],isCrossRepository:false,title:a[a.indexOf('--title')+1],body:''};save();console.log('https://github.com/o/r/pull/'+n);process.exit(0)}", + "console.error('gh shim: unsupported '+a.join(' '));process.exit(1);", + '', + ].join('\n'), { mode: 0o755 }); + const env = { PATH: dir + ':' + (process.env.PATH ?? ''), LAND_SHIM_DATA: data, LAND_SHIM_ORIGIN: repo.origin }; + return { repo, data, env, write: fn => { const d = JSON.parse(fs.readFileSync(data, 'utf8')); fn(d); fs.writeFileSync(data, JSON.stringify(d)); } }; +} + +/** A PR branch from origin/main, pushed, with a fragment (or none), and a /ship receipt for its head in the shim. */ +/** The READY receipt `gstack-ship-queue ready` records (a note on origin refs/notes/gstack-trains), written into the fixture's own state root. */ +function markReady(fx: Fx, pr: string, head: string): void { + const prior = process.env.GSTACK_STATE_ROOT; + process.env.GSTACK_STATE_ROOT = path.join(fx.repo.root, 'ready-state'); + try { + const now = new Date().toISOString(); + persist(fx.repo.work, [{ schema: 1, kind: 'ready', repo_key: repoIdentity(fx.repo.work).key, pr, head, base_sha: fx.repo.git('rev-parse', 'origin/main'), ready_at: now, screen: { choice: 'no-screen', policy: null, nonce: null, run: null, merge_sha: null }, updated_at: now }]); + } finally { if (prior === undefined) delete process.env.GSTACK_STATE_ROOT; else process.env.GSTACK_STATE_ROOT = prior; } +} + +function openPr(fx: Fx, pr: string, opts: { fragment?: string | null; edit?: (work: string) => void; shipped?: boolean; ready?: boolean } = {}): string { + const { repo } = fx; + repo.git('checkout', '-q', 'origin/main', '-B', 'pr-' + pr); + fs.writeFileSync(path.join(repo.work, 'pr-' + pr + '.txt'), pr + '\n'); + if (opts.fragment !== null) { + fs.mkdirSync(path.join(repo.work, 'changelog.d'), { recursive: true }); + fs.writeFileSync(path.join(repo.work, 'changelog.d', 'pr-' + pr + '.md'), opts.fragment ?? '- #' + pr + ' change\n'); + } + opts.edit?.(repo.work); + repo.git('add', '-A'); + repo.git('commit', '-q', '-m', 'pr ' + pr); + repo.git('push', '-q', '-f', 'origin', 'HEAD:refs/heads/pr-' + pr); + const head = repo.git('rev-parse', 'HEAD'); + fx.write(d => { d.prs[pr] = { state: 'OPEN', headRefName: 'pr-' + pr, isCrossRepository: false, title: 'PR ' + pr + ' title', body: opts.shipped === false ? '' : shippedBody(pr, head) }; }); + if (opts.ready !== false) markReady(fx, pr, head); + repo.git('checkout', '-q', '--detach', 'origin/main'); + return head; +} +const land = (fx: Fx, args: string[]) => runBin(fx.repo, 'gstack-ship-queue', ['land', ...args, '--base', 'main', '--no-wait', '--allow-repo-commands'], fx.env); +const remote = (fx: Fx, ref: string) => { fx.repo.git('fetch', '-q', 'origin', '+refs/heads/*:refs/remotes/origin/*'); return fx.repo.git('rev-parse', 'origin/' + ref); }; +const show = (fx: Fx, rev: string) => fx.repo.git('show', rev); + +describe('land: fold, push, merge', () => { + test('happy path: a stamp commit with parents [pin, main] and the folded tree, then a squash whose tree is that tree', () => { + const fx = setup(); + const base = remote(fx, 'main'); + const pin = openPr(fx, '7', { fragment: '---\nbump: minor\n---\n- **Fragments.** #7\n' }); + const first = land(fx, ['--pr', '7']); + expect(first.err).toBe(''); + expect(first.status).toBe(EXIT.ok); + expect(first.out).toContain('STAMP_VERSION: 1.3.0.0'); + expect(first.out).toMatch(/^NEXT: gstack-pr-wait 7 --until green && gstack-ship-queue land --pr 7$/m); + const stamp = remote(fx, 'pr-7'); + expect(fx.repo.git('rev-list', '--parents', '-n', '1', stamp).split(' ').slice(1)).toEqual([pin, base]); + expect(show(fx, stamp + ':VERSION')).toBe('1.3.0.0'); + expect(fx.repo.git('ls-tree', '--name-only', stamp, 'changelog.d/')).toBe(''); + expect(show(fx, stamp + ':CHANGELOG.md')).toMatch(/^# Changelog\n\n## \[1\.3\.0\.0\] - \d{4}-\d{2}-\d{2}\n\n- \*\*Fragments\.\*\* #7\n\n## \[1\.2\.3\.0\]/); + const second = land(fx, ['--pr', '7']); + expect(second.err).toBe(''); + expect(second.out).toContain('MERGED: #7'); + expect(second.out).toMatch(/^LANDED: origin\/main tree [0-9a-f]+ equals the stamped tree$/m); + expect(second.out).toContain('LOCK: released'); + const main = remote(fx, 'main'); + expect(fx.repo.git('rev-parse', main + '^{tree}')).toBe(fx.repo.git('rev-parse', stamp + '^{tree}')); + }, 90_000); + + test('two PRs from one base land one after the other with no collision and no lane round trip', () => { + const fx = setup(); + openPr(fx, '1'); + openPr(fx, '2'); + for (const pr of ['1', '2']) { expect(land(fx, ['--pr', pr]).status).toBe(EXIT.ok); expect(land(fx, ['--pr', pr]).out).toContain('LANDED:'); } + const main = remote(fx, 'main'); + expect(show(fx, main + ':VERSION')).toBe('1.2.5.0'); + const cl = show(fx, main + ':CHANGELOG.md'); + expect(cl.indexOf('## [1.2.5.0]')).toBeLessThan(cl.indexOf('- #2 change')); + expect(cl.indexOf('- #2 change')).toBeLessThan(cl.indexOf('## [1.2.4.0]')); + expect(cl.indexOf('## [1.2.4.0]')).toBeLessThan(cl.indexOf('- #1 change')); + }, 90_000); + + test('a legacy PR with its own VERSION bump and a clean dependency edit: release fields resolve to main, the dependency edit survives', () => { + const fx = setup(); + openPr(fx, '1'); + expect(land(fx, ['--pr', '1']).status).toBe(EXIT.ok); land(fx, ['--pr', '1']); + fx.repo.git('fetch', '-q', 'origin'); + fx.repo.git('checkout', '-q', '-B', 'pr-3', fx.repo.git('rev-parse', 'origin/main') + '~1'); + const pkg = JSON.parse(readFile(fx.repo, 'package.json')); + pkg.version = '1.2.9'; pkg.dependencies['right-pad'] = '2.0.0'; + fs.writeFileSync(path.join(fx.repo.work, 'package.json'), JSON.stringify(pkg, null, 2) + '\n'); + fs.writeFileSync(path.join(fx.repo.work, 'VERSION'), '1.2.9.0\n'); + fs.writeFileSync(path.join(fx.repo.work, 'CHANGELOG.md'), readFile(fx.repo, 'CHANGELOG.md').replace('# Changelog\n', '# Changelog\n\n## [1.2.9.0] - 2026-10-28\n\n- legacy #3\n')); + fx.repo.git('add', '-A'); fx.repo.git('commit', '-q', '-m', 'legacy stamped pr'); + fx.repo.git('push', '-q', '-f', 'origin', 'HEAD:refs/heads/pr-3'); + const head = fx.repo.git('rev-parse', 'HEAD'); + fx.write(d => { d.prs['3'] = { state: 'OPEN', headRefName: 'pr-3', isCrossRepository: false, title: 'legacy', body: shippedBody('3', head) }; }); + markReady(fx, '3', head); + const r = land(fx, ['--pr', '3']); + expect(r.err).toBe(''); + expect(r.out).toMatch(/^legacy: the pin stamped 1\.2\.9\.0 itself/m); + expect(r.out).toMatch(/^resolved: .*CHANGELOG\.md/m); + const stamp = remote(fx, 'pr-3'); + expect(show(fx, stamp + ':VERSION')).toBe('1.2.5.0'); + const merged = JSON.parse(show(fx, stamp + ':package.json')); + expect(merged.dependencies).toEqual({ 'left-pad': '1.2.3', 'right-pad': '2.0.0' }); + expect(show(fx, stamp + ':CHANGELOG.md')).toMatch(/## \[1\.2\.5\.0\][^\n]*\n\n- legacy #3\n\n## \[1\.2\.4\.0\][^\n]*\n\n- #1 change/); + }, 90_000); + + test('a PR with no entry gets one from its title; a product conflict refuses with nothing pushed', () => { + const fx = setup(); + openPr(fx, '4', { fragment: null, edit: w => fs.writeFileSync(path.join(w, 'shared.txt'), 'four\n') }); + openPr(fx, '5', { edit: w => fs.writeFileSync(path.join(w, 'shared.txt'), 'five\n') }); + const r = land(fx, ['--pr', '4']); + expect(r.out).toContain('entry: none in the PR, so land wrote changelog.d/land-4.md from the title'); + expect(show(fx, remote(fx, 'pr-4') + ':CHANGELOG.md')).toContain('- PR 4 title (#4)'); + land(fx, ['--pr', '4']); + const pin5 = remote(fx, 'pr-5'); + const c = land(fx, ['--pr', '5']); + expect(c.status).toBe(EXIT.refused); + expect(c.err).toMatch(/#5 conflicts with origin\/main in shared\.txt.*\(QUEUE_LAND_CONFLICT\)/); + expect(remote(fx, 'pr-5')).toBe(pin5); + }, 90_000); +}); + +describe('land: lock, races and refusals', () => { + test('a held lock refuses; --break-lock takes over a lock that is not mid-push', () => { + const fx = setup(); + openPr(fx, '1'); openPr(fx, '2'); + expect(land(fx, ['--pr', '1']).status).toBe(EXIT.ok); + const held = land(fx, ['--pr', '2']); + expect(held.err).toMatch(/#1 holds the land lock .*phase waiting.*\(QUEUE_LAND_LOCKED\)/); + const broke = land(fx, ['--pr', '2', '--break-lock']); + expect(broke.out).toContain("LOCK: broke #1's lock"); + }, 90_000); + + test('the base moving during CI restamps and pushes under the last-pushed lease; a third move stops', () => { + const fx = setup(); + openPr(fx, '1'); openPr(fx, '2'); + const pin1 = remote(fx, 'pr-1'); + expect(land(fx, ['--pr', '1']).status).toBe(EXIT.ok); + const firstStamp = remote(fx, 'pr-1'); + const bump = (n: number) => { fx.repo.git('fetch', '-q', 'origin'); fx.repo.git('checkout', '-q', '--detach', 'origin/main'); fs.writeFileSync(path.join(fx.repo.work, 'outside-' + n + '.txt'), 'x\n'); fx.repo.git('add', '-A'); fx.repo.git('commit', '-q', '-m', 'outside ' + n); fx.repo.git('push', '-q', 'origin', 'HEAD:refs/heads/main'); }; + bump(1); + const again = land(fx, ['--pr', '1']); + expect(again.err).toBe(''); + expect(again.out).toMatch(/^BASE_MOVED: /m); + const secondStamp = remote(fx, 'pr-1'); + expect(secondStamp).not.toBe(firstStamp); + expect(fx.repo.git('rev-list', '--parents', '-n', '1', secondStamp).split(' ')[1]).toBe(pin1); + bump(2); expect(land(fx, ['--pr', '1']).status).toBe(EXIT.ok); + bump(3); + expect(land(fx, ['--pr', '1']).err).toContain('(QUEUE_LAND_BASE_BUSY)'); + fx.repo.git('fetch', '-q', 'origin', '+refs/notes/gstack-land:refs/notes/gstack-land'); + expect(show(fx, 'refs/notes/gstack-land:lock.json')).toContain('"released": true'); + const later = land(fx, ['--pr', '1']); + expect(later.err).toBe(''); + expect(later.out).toMatch(/^BASE_MOVED: /m); + }, 90_000); + + test('a missing receipt refuses before the lock; a foreign push after the stamp is never overwritten', () => { + const fx = setup(); + openPr(fx, '6', { shipped: false }); + expect(land(fx, ['--pr', '6']).err).toContain('(QUEUE_RECEIPT_MISSING)'); + openPr(fx, '8'); + expect(land(fx, ['--pr', '8']).status).toBe(EXIT.ok); + fx.repo.git('fetch', '-q', 'origin'); + fx.repo.git('checkout', '-q', '--detach', 'origin/pr-8'); + fs.writeFileSync(path.join(fx.repo.work, 'late.txt'), 'late\n'); fx.repo.git('add', '-A'); fx.repo.git('commit', '-q', '-m', 'late'); + fx.repo.git('push', '-q', 'origin', 'HEAD:refs/heads/pr-8'); + const late = fx.repo.git('rev-parse', 'HEAD'); + const r = land(fx, ['--pr', '8', '--break-lock']); + expect(r.err).toMatch(/QUEUE_RECEIPT_STALE/); + expect(remote(fx, 'pr-8')).toBe(late); + }, 90_000); + + test('the receipt check is the depth-checked one READY uses: a receipt whose writer records depth but which records none refuses', () => { + const fx = setup(); + const head = openPr(fx, '31'); + fx.write(d => { d.prs['31'].body = shippedBody('31', head, { gstack_version: '1.91.307.0' }); }); + const r = land(fx, ['--pr', '31']); + expect(r.err).toContain('(QUEUE_RECEIPT_DEPTH_RAISED)'); + expect(r.out).toMatch(/^REVIEW_DEPTH: receipt=none required=/m); + expect(remote(fx, 'pr-31')).toBe(head); + }, 60_000); + + test('drift: a merge outside land between the check and the squash fails loudly and keeps the lock', () => { + const fx = setup(); + openPr(fx, '9'); + expect(land(fx, ['--pr', '9']).status).toBe(EXIT.ok); + fx.write(d => { d.driftOnMerge = true; }); + const r = land(fx, ['--pr', '9']); + expect(r.err).toContain('(QUEUE_LAND_DRIFT)'); + openPr(fx, '10'); + expect(land(fx, ['--pr', '10']).err).toMatch(/#9 holds the land lock .*phase drift.*\(QUEUE_LAND_LOCKED\)/); + expect(land(fx, ['--pr', '10', '--break-lock']).err).toMatch(/#9 holds a drift lock; --break-lock never breaks it.*\(QUEUE_LAND_DRIFT\)/); + const rep = land(fx, ['--pr', '9', '--repair']); + expect(rep.status).toBe(EXIT.ok); + expect(rep.out).toContain('entry: changelog.d/repair-9.md names 1 outside commit(s)'); + const n = /REPAIR_PR: #(\d+) gstack-land-repair\/9 ([0-9a-f]{40})/.exec(rep.out)!; + expect(show(fx, n[2] + ':changelog.d/repair-9.md')).toBe('- Changes merged outside land beside #9: outside merge'); + expect(land(fx, ['--pr', n[1]!]).err).toContain('(QUEUE_RECEIPT_MISSING)'); + fx.write(d => { d.prs[n[1]!].body = shippedBody(n[1]!, n[2]!); }); + markReady(fx, n[1]!, n[2]!); + const pushed = land(fx, ['--pr', n[1]!]); + expect(pushed.out).toContain("LOCK: taking over #9's drift lock for its repair"); + const merged = land(fx, ['--pr', n[1]!]); + expect(merged.status).toBe(EXIT.ok); + expect(merged.out).toContain('LOCK: released'); + expect(fx.repo.git('ls-tree', '-r', '--name-only', 'origin/main', '--', 'changelog.d')).not.toContain('repair-9'); + expect(show(fx, 'origin/main:CHANGELOG.md')).toContain('- Changes merged outside land beside #9: outside merge'); + expect(land(fx, ['--pr', '10']).status).toBe(EXIT.ok); + }, 120_000); + + test('census lists what land would do, writing nothing', () => { + const fx = setup(); + openPr(fx, '1'); openPr(fx, '2', { fragment: null }); + const c = runBin(fx.repo, 'gstack-ship-queue', ['land', '--census', '--base', 'main'], fx.env); + expect(c.status).toBe(EXIT.ok); + expect(c.out).toMatch(/^#1 [0-9a-f]{12}: fold pr-1\.md$/m); + expect(c.out).toMatch(/^#2 [0-9a-f]{12}: no entry \(land writes one from the title\)$/m); + }, 90_000); +}); + +describe('land: the base\'s manifest part count', () => { + test('a repo whose base holds a four-part package.json listed in stamp_files lands four parts (gbrain#7878 wrote three)', () => { + const fx = setup({ ...POLICY, stamp_files: ['VERSION', 'package.json', 'CHANGELOG.md', 'changelog.d/*.md'], stamp_patterns: ['"version": "{version}"'] }); + const { repo } = fx; + repo.git('checkout', '-q', 'main'); + const pkg = JSON.parse(readFile(repo, 'package.json')); + fs.writeFileSync(path.join(repo.work, 'package.json'), JSON.stringify({ ...pkg, version: '1.2.3.0' }, null, 2) + '\n'); + fs.rmSync(path.join(repo.work, 'package-lock.json')); + repo.git('add', '-A'); repo.git('commit', '-q', '-m', 'four-part manifest'); repo.git('push', '-q', 'origin', 'main'); + openPr(fx, '21'); + expect(land(fx, ['--pr', '21']).status).toBe(EXIT.ok); + const merged = land(fx, ['--pr', '21']); + expect(merged.status, merged.err).toBe(EXIT.ok); + expect(JSON.parse(show(fx, 'origin/main:package.json')).version).toBe('1.2.4.0'); + expect(show(fx, 'origin/main:VERSION')).toBe('1.2.4.0'); + }, 90_000); + + test('control: gstack\'s shape (a three-part manifest outside stamp_files) stays three parts', () => { + const fx = setup(); + openPr(fx, '22'); + land(fx, ['--pr', '22']); + expect(land(fx, ['--pr', '22']).status).toBe(EXIT.ok); + expect(JSON.parse(show(fx, 'origin/main:package.json')).version).toBe('1.2.4'); + }, 90_000); +}); + +describe('resolveReleaseConflicts', () => { + test('a manifest whose version conflicts keeps the clean dependency edit and takes the base version; a real dependency conflict stays a conflict', () => { + const fx = setup(); + const { repo } = fx; + const write = (v: string, deps: Record) => fs.writeFileSync(path.join(repo.work, 'package.json'), JSON.stringify({ name: 'fixture', version: v, dependencies: deps }, null, 2) + '\n'); + repo.git('checkout', '-q', '-B', 'base-x', 'origin/main'); + write('1.2.3', { a: '1' }); repo.git('commit', '-qam', 'base'); + repo.git('checkout', '-q', '-B', 'ours-x'); write('1.2.9', { a: '1', b: '2' }); repo.git('commit', '-qam', 'ours'); + repo.git('checkout', '-q', '-B', 'theirs-x', 'base-x'); write('1.2.4', { a: '1' }); repo.git('commit', '-qam', 'theirs'); + repo.git('checkout', '-q', 'ours-x'); + try { repo.git('merge', '--no-commit', 'theirs-x'); } catch {} + const policy = mergePolicy(POLICY, 'test').policy; + expect(resolveReleaseConflicts(repo.work, ['package.json'], policy, 'VERSION')).toEqual([]); + expect(JSON.parse(readFile(repo, 'package.json'))).toEqual({ name: 'fixture', version: '1.2.4', dependencies: { a: '1', b: '2' } }); + repo.git('merge', '--abort'); + repo.git('checkout', '-q', 'theirs-x'); write('1.2.4', { a: '3' }); repo.git('commit', '-qam', 'theirs dep'); + repo.git('checkout', '-q', 'ours-x'); write('1.2.9', { a: '2', b: '2' }); repo.git('commit', '-qam', 'ours dep'); + try { repo.git('merge', '--no-commit', 'theirs-x'); } catch {} + expect(resolveReleaseConflicts(repo.work, ['package.json'], policy, 'VERSION')).toEqual(['package.json']); + }, 90_000); +}); + +// Generated by /ship Step 7 coverage audit (GSTA-154 PR A, pass 1). +describe('land: dry run and refusals before the lock', () => { + const lockRef = (fx: Fx) => fx.repo.git('ls-remote', 'origin', 'refs/notes/gstack-land'); + const journal = (fx: Fx, pr: string) => path.join(fx.repo.work, '.gstack', 'tmp', 'ship-queue', 'land', pr + '.json'); + + // Value: protects=land --dry-run prints the stamp it would make and writes nothing: no lock ref, no push, no journal; + // fails_when=dry-run falls through to the lock/push path or saves a journal; why_new=no land test passes --dry-run; + // seam=none; no_claim=gh shim stands in for GitHub (fixture repos only) + test('--dry-run prints the stamp version and tree but takes no lock, pushes nothing and saves no journal', () => { + const fx = setup(); + const pin = openPr(fx, '11', { fragment: '---\nbump: minor\n---\n- dry #11\n' }); + const r = land(fx, ['--pr', '11', '--dry-run']); + expect(r.err).toBe(''); + expect(r.status).toBe(EXIT.ok); + expect(r.out).toContain('STAMP_VERSION: 1.3.0.0'); + expect(r.out).toMatch(/^STAMP_TREE: [0-9a-f]{40}$/m); + expect(r.out).toContain('DRY-RUN: nothing locked, pushed or merged'); + expect(remote(fx, 'pr-11')).toBe(pin); + expect(lockRef(fx)).toBe(''); + expect(fs.existsSync(journal(fx, '11'))).toBe(false); + }, 90_000); + + // Value: protects=land refuses a red pin, a fork head and a closed PR before it locks or pushes anything; + // fails_when=the pin CI check, the isCrossRepository guard or the OPEN-state guard is dropped or moved after the lock; + // why_new=existing land tests only use green, open, same-repo PRs; seam=none; no_claim=GitHub check semantics (gh shim) + test('a red pin, a fork head and a closed PR refuse with no lock taken and the branch untouched', () => { + const fx = setup(); + const pin = openPr(fx, '12'); + fx.write(d => { d.red = [pin]; }); + const red = land(fx, ['--pr', '12']); + expect(red.err).toMatch(/required checks are not green at the pin .*\(QUEUE_NOT_READY\)/); + fx.write(d => { d.red = []; d.prs['12'].isCrossRepository = true; }); + expect(land(fx, ['--pr', '12']).err).toContain('(QUEUE_TRAIN_UNTRUSTED)'); + fx.write(d => { d.prs['12'].isCrossRepository = false; d.prs['12'].state = 'CLOSED'; }); + expect(land(fx, ['--pr', '12']).err).toMatch(/#12 is CLOSED.*\(QUEUE_NOT_READY\)/); + expect(remote(fx, 'pr-12')).toBe(pin); + expect(lockRef(fx)).toBe(''); + }, 90_000); + + // Value: protects=--break-lock is refused while another holder's lock is in phase pushing and under 10 minutes old, + // and allowed once that push lock is stale; fails_when=the PUSH_STALE_MS guard is removed or inverted; + // why_new=the existing break-lock test only breaks a waiting lock; seam=none; no_claim=a real concurrent lander (lock ref crafted) + test('--break-lock refuses a fresh mid-push lock and breaks a stale one', () => { + const fx = setup(); + const pin = openPr(fx, '13'); + const pushLock = (startedAt: string) => { + const sh = (args: string[], input?: string) => { + const r = spawnSync('git', args, { cwd: fx.repo.work, input, encoding: 'utf8', timeout: 30_000 }); + expect(r.status, r.stderr).toBe(0); + return r.stdout.trim(); + }; + const blob = sh(['hash-object', '-w', '--stdin'], JSON.stringify({ pr: '99', holder: 'other:1', started_at: startedAt, phase: 'pushing', journal: '.gstack/tmp/ship-queue/land/99.json' })); + const tree = sh(['mktree'], '100644 blob ' + blob + '\tlock.json\n'); + const commit = sh(['commit-tree', tree, '-m', 'land lock #99 pushing']); + sh(['push', '-q', '-f', 'origin', commit + ':refs/notes/gstack-land']); + }; + pushLock(new Date().toISOString()); + const fresh = land(fx, ['--pr', '13', '--break-lock']); + expect(fresh.err).toMatch(/#99's holder is mid-push .*--break-lock refused.*\(QUEUE_LAND_LOCKED\)/); + expect(remote(fx, 'pr-13')).toBe(pin); + pushLock(new Date(Date.now() - 60 * 60_000).toISOString()); + const stale = land(fx, ['--pr', '13', '--break-lock']); + expect(stale.err).toBe(''); + expect(stale.out).toContain("LOCK: broke #99's lock (other:1, phase pushing)"); + expect(remote(fx, 'pr-13')).not.toBe(pin); + }, 90_000); +}); + +// Generated by /ship Step 7 coverage audit (GSTA-154 PR A, pass 2). +describe('land: failures after the push, repair branches and census rows', () => { + const journal = (fx: Fx, pr: string) => JSON.parse(fs.readFileSync(path.join(fx.repo.work, '.gstack', 'tmp', 'ship-queue', 'land', pr + '.json'), 'utf8')); + const pushMain = (fx: Fx, msg: string, edit: (work: string) => void) => { + fx.repo.git('fetch', '-q', 'origin'); fx.repo.git('checkout', '-q', '--detach', 'origin/main'); + edit(fx.repo.work); + fx.repo.git('add', '-A'); fx.repo.git('commit', '-q', '-m', msg); fx.repo.git('push', '-q', 'origin', 'HEAD:refs/heads/main'); + }; + + // Value: protects=a red stamp commit stops land with QUEUE_NOT_READY and a refused gh pr merge stops it too, both keeping + // the journal at phase pushed so a rerun resumes and lands; fails_when=land merges on red, or a failure resets or loses the journal; + // why_new=the shim never reds a stamp commit or refuses a merge; seam=none; no_claim=GitHub check and merge semantics (gh shim) + test('a red stamp commit and a refused merge both stop with the journal at pushed, and a rerun lands; a non-numeric --pr is a usage error', () => { + const fx = setup(); + openPr(fx, '14'); + expect(land(fx, ['--pr', '14']).status).toBe(EXIT.ok); + const stamp = remote(fx, 'pr-14'); + fx.write(d => { d.red = [stamp]; }); + const red = land(fx, ['--pr', '14']); + expect(red.err).toMatch(/a check failed on the stamp commit .*\(QUEUE_NOT_READY\)/); + expect(red.out).toMatch(/^CI: red at /m); + expect(journal(fx, '14')).toMatchObject({ phase: 'pushed', stamp_commit: stamp }); + fx.write(d => { d.red = []; d.mergeRefuse = true; }); + const refused = land(fx, ['--pr', '14']); + expect(refused.err).toMatch(/gh pr merge refused: Pull request is not mergeable.*\(QUEUE_NOT_READY\)/); + expect(journal(fx, '14')).toMatchObject({ phase: 'pushed', stamp_commit: stamp }); + expect(remote(fx, 'pr-14')).toBe(stamp); + fx.write(d => { d.mergeRefuse = false; }); + const done = land(fx, ['--pr', '14']); + expect(done.err).toBe(''); + expect(done.out).toContain('MERGED: #14'); + expect(journal(fx, '14').phase).toBe('landed'); + const usage = runBin(fx.repo, 'gstack-ship-queue', ['land', '--pr', 'x14', '--base', 'main'], fx.env); + expect(usage.status).toBe(EXIT.usage); + expect(usage.err).toContain('land needs --pr (or --census)'); + }, 90_000); + + // Value: protects=--repair refuses without a drift record and with fragments off, moves fragments the outside merge left + // on the base to repair--.md, and a second --repair reports the open PR instead of opening another; fails_when=repair + // opens duplicate PRs, drops a stray fragment, or runs without a drift record; why_new=the drift test covers only the no-stray path; + // seam=none; no_claim=gh pr create (gh shim) + test('--repair: no drift record and fragments off refuse; stray base fragments move; a second run reports the open PR', () => { + const fx = setup(); + openPr(fx, '15'); + expect(land(fx, ['--pr', '15', '--repair']).err).toMatch(/has no drift record; --repair runs only after QUEUE_LAND_DRIFT.*\(QUEUE_NOT_READY\)/); + expect(land(fx, ['--pr', '15']).status).toBe(EXIT.ok); + fx.write(d => { d.driftOnMerge = true; }); + expect(land(fx, ['--pr', '15']).err).toContain('(QUEUE_LAND_DRIFT)'); + pushMain(fx, 'fragments off', w => fs.writeFileSync(path.join(w, '.gstack', 'ship-policy.json'), JSON.stringify({ ...POLICY, changelog_fragments: null }, null, 2) + '\n')); + expect(land(fx, ['--pr', '15', '--repair']).err).toMatch(/changelog_fragments is off.*\(QUEUE_NOT_READY\)/); + pushMain(fx, 'fragments on, outside fragment', w => { + fs.writeFileSync(path.join(w, '.gstack', 'ship-policy.json'), JSON.stringify(POLICY, null, 2) + '\n'); + fs.mkdirSync(path.join(w, 'changelog.d'), { recursive: true }); + fs.writeFileSync(path.join(w, 'changelog.d', 'stray.md'), '- outside stray\n'); + }); + const rep = land(fx, ['--pr', '15', '--repair']); + expect(rep.err).toBe(''); + expect(rep.out).toContain('moved: changelog.d/stray.md (left unfolded by the outside merge)'); + expect(rep.out).not.toContain('entry: changelog.d/repair-15.md'); + const m = /REPAIR_PR: #(\d+) gstack-land-repair\/15 ([0-9a-f]{40})/.exec(rep.out)!; + expect(fx.repo.git('ls-tree', '-r', '--name-only', m[2]!, '--', 'changelog.d')).toBe('changelog.d/repair-15-stray.md'); + expect(show(fx, m[2] + ':changelog.d/repair-15-stray.md')).toBe('- outside stray'); + const again = land(fx, ['--pr', '15', '--repair']); + expect(again.out).toContain('REPAIR_PR: #' + m[1] + ' (already open)'); + expect(Object.keys(JSON.parse(fs.readFileSync(fx.data, 'utf8')).prs)).toEqual(['15', m[1]]); + }, 90_000); + + // Value: protects=census reports a fork, a product conflict, an invalid fragment and a legacy [Unreleased] PR with a tentative + // version as distinct rows and writes nothing; fails_when=a row is misclassified (e.g. a product conflict shown as foldable); + // why_new=the census test covers only fold and no-entry rows; seam=none; no_claim=gh pr list (gh shim) + test('census rows: fork refused, product conflict refused, invalid fragment, legacy entry with a tentative version', () => { + const fx = setup(); + openPr(fx, '1'); + fx.write(d => { d.prs['1'].isCrossRepository = true; }); + openPr(fx, '2', { edit: w => fs.writeFileSync(path.join(w, 'shared.txt'), 'two\n') }); + openPr(fx, '3', { fragment: '## [9.9.9.9]\n- bad\n' }); + openPr(fx, '4', { fragment: null, edit: w => { + fs.writeFileSync(path.join(w, 'VERSION'), '1.2.9.0\n'); + fs.writeFileSync(path.join(w, 'CHANGELOG.md'), fs.readFileSync(path.join(w, 'CHANGELOG.md'), 'utf8').replace('# Changelog\n', '# Changelog\n\n## [Unreleased]\n\n- legacy #4\n')); + } }); + pushMain(fx, 'main edits shared', w => fs.writeFileSync(path.join(w, 'shared.txt'), 'main\n')); + const mainBefore = remote(fx, 'main'); + const c = runBin(fx.repo, 'gstack-ship-queue', ['land', '--census', '--base', 'main'], fx.env); + expect(c.status, c.err).toBe(EXIT.ok); + expect(c.out).toMatch(/^CENSUS: 4 open PR\(s\) against origin\/main@[0-9a-f]{12} \(fragments: changelog\.d\)$/m); + expect(c.out).toMatch(/^#1 fork: refused \(QUEUE_TRAIN_UNTRUSTED\)$/m); + expect(c.out).toMatch(/^#2 [0-9a-f]{12}: refuse \(product conflict: shared\.txt\)$/m); + expect(c.out).toMatch(/^#3 [0-9a-f]{12}: invalid fragment changelog\.d\/pr-3\.md$/m); + expect(c.out).toMatch(/^#4 [0-9a-f]{12}: re-head legacy \[Unreleased\]; rewrite its tentative 1\.2\.9\.0 through restamp$/m); + expect(remote(fx, 'main')).toBe(mainBefore); + expect(fx.repo.git('ls-remote', 'origin', 'refs/notes/gstack-land')).toBe(''); + }, 90_000); +}); + +// /ship pre-landing review fixes (GSTA-154 PR A, pass 1): each case below failed or was unreachable before its fix. +describe('land: one pass, base, lock and history', () => { + // Value: protects=land without --no-wait pushes, waits and merges in one run; fails_when=the pushed phase compares a pre-push head; why_new=every other case passes --no-wait; seam=none; no_claim=GitHub check scheduling latency (gh shim reports green at once) + test('without --no-wait one run pushes the stamp, sees its checks green and merges', () => { + const fx = setup(); + openPr(fx, '7'); + const r = runBin(fx.repo, 'gstack-ship-queue', ['land', '--pr', '7', '--base', 'main', '--allow-repo-commands'], fx.env); + expect(r.err).toBe(''); + expect(r.status).toBe(EXIT.ok); + expect(r.out).toContain('PUSHED: '); + expect(r.out).toContain('MERGED: #7'); + expect(r.out).toContain('LOCK: released'); + }, 90_000); + + // Value: protects=land refuses a PR whose base is not the base it stamps; fails_when=a stacked PR gets main merged and squashed into its parent branch; why_new=no case set baseRefName; seam=none; no_claim=gh pr view field names (shim) + test('a PR that targets another branch refuses before the lock, with nothing pushed', () => { + const fx = setup(); + const pin = openPr(fx, '8'); + fx.write(d => { d.prs['8'].baseRefName = 'feature'; }); + const r = land(fx, ['--pr', '8']); + expect(r.status).toBe(EXIT.refused); + expect(r.err).toMatch(/#8 targets feature, not main.*\(QUEUE_NOT_READY\)/); + expect(remote(fx, 'pr-8')).toBe(pin); + expect(fx.repo.git('ls-remote', 'origin', 'refs/notes/gstack-land')).toBe(''); + }, 90_000); + + // Value: protects=a stamp that fails before any push releases the land lock; fails_when=one conflicting PR blocks every other land until --break-lock; why_new=the conflict case checks only the branch; seam=none; no_claim=none beyond protects + test('a product conflict releases the lock, so the next PR lands without --break-lock', () => { + const fx = setup(); + openPr(fx, '4', { edit: w => fs.writeFileSync(path.join(w, 'shared.txt'), 'four\n') }); + openPr(fx, '5', { edit: w => fs.writeFileSync(path.join(w, 'shared.txt'), 'five\n') }); + openPr(fx, '6'); + land(fx, ['--pr', '4']); land(fx, ['--pr', '4']); + expect(land(fx, ['--pr', '5']).err).toContain('(QUEUE_LAND_CONFLICT)'); + fx.repo.git('fetch', '-q', 'origin', '+refs/notes/gstack-land:refs/notes/gstack-land'); + expect(show(fx, 'refs/notes/gstack-land:lock.json')).toContain('"released": true'); + expect(land(fx, ['--pr', '6']).status).toBe(EXIT.ok); + }, 90_000); + + // Value: protects=a landed PR's fragment carried in by a stacked branch is refused, not folded twice; fails_when=the child PR's stamp repeats the parent's entry and bump; why_new=no case stacks a PR on an unlanded pin; seam=none; no_claim=exact-text match only (a reworded copy is not caught) + test('a fragment already released on the base (stacked history) refuses instead of folding twice', () => { + const fx = setup(); + const pin1 = openPr(fx, '1', { fragment: '- #1 the parent change\n' }); + land(fx, ['--pr', '1']); land(fx, ['--pr', '1']); + fx.repo.git('checkout', '-q', '-B', 'pr-11', pin1); + fs.writeFileSync(path.join(fx.repo.work, 'changelog.d', 'pr-11.md'), '- #11 child change\n'); + fx.repo.git('add', '-A'); fx.repo.git('commit', '-q', '-m', 'child'); + fx.repo.git('push', '-q', '-f', 'origin', 'HEAD:refs/heads/pr-11'); + const head = fx.repo.git('rev-parse', 'HEAD'); + fx.write(d => { d.prs['11'] = { state: 'OPEN', headRefName: 'pr-11', isCrossRepository: false, title: 'child', body: shippedBody('11', head) }; }); + markReady(fx, '11', head); + fx.repo.git('checkout', '-q', '--detach', 'origin/main'); + const r = land(fx, ['--pr', '11']); + expect(r.status).toBe(EXIT.refused); + expect(r.err).toMatch(/changelog\.d\/pr-1\.md: its text is already released in CHANGELOG\.md.*\(CHANGELOG_FRAGMENT_INVALID\)/); + expect(remote(fx, 'pr-11')).toBe(head); + }, 90_000); + + // Value: protects=a lost local journal resumes from the stamp commit's pin; fails_when=the stamp commit is treated as the pin and the receipt refuses; why_new=resume cases keep the journal; seam=none; no_claim=none beyond protects + test('with the local journal gone, a head that is land\'s stamp commit resumes from its pin', () => { + const fx = setup(); + const pin = openPr(fx, '12'); + expect(land(fx, ['--pr', '12']).status).toBe(EXIT.ok); + fs.rmSync(path.join(fx.repo.work, '.gstack', 'tmp', 'ship-queue', 'land', '12.json')); + const again = land(fx, ['--pr', '12']); + expect(again.err).toBe(''); + expect(again.out).toContain(`pin=${pin} (the head `); + expect(again.out).toContain('PUSHED: '); + expect(land(fx, ['--pr', '12']).out).toContain('MERGED: #12'); + }, 90_000); + + // Value: protects=history merge-only lands with a merge commit, and a merge after land's own is not drift; fails_when=land squashes under merge-only, or flags a later outside merge as drift; why_new=no case sets history or merges after the squash; seam=none; no_claim=GitHub's merge-commit shape (the shim records the flag) + test('merge-only lands with --merge; an outside merge after the squash is not drift', () => { + const fx = setup({ ...POLICY, history: 'merge-only' }); + openPr(fx, '13'); + land(fx, ['--pr', '13']); + fx.write(d => { d.outsideAfter = true; }); + const r = land(fx, ['--pr', '13']); + expect(r.err).toBe(''); + expect(r.out).toContain('LANDED: '); + expect(JSON.parse(fs.readFileSync(fx.data, 'utf8')).mergeArgs).toContain('--merge'); + }, 90_000); +}); + +// /ship adversarial (Claude) and outside-voice (Codex) findings (GSTA-154 PR A): each case failed or was unreachable before its fix. +describe('land: merged-after-error, live locks, repair recovery, legacy release files', () => { + // Value: protects=a merge GitHub completed but reported as failed reconciles on rerun; fails_when=the rerun refuses "is MERGED" and the lock stays held; why_new=no case merged with an error; seam=none; no_claim=GitHub's real error shapes + test('gh pr merge errors after GitHub merged: the rerun verifies the tree and releases the lock', () => { + const fx = setup(); + openPr(fx, '20'); + land(fx, ['--pr', '20']); + fx.write(d => { d.mergeErrorAfter = true; }); + expect(land(fx, ['--pr', '20']).err).toContain('HTTP 502'); + const again = land(fx, ['--pr', '20']); + expect(again.err).toBe(''); + expect(again.out).toContain('LANDED: '); + expect(again.out).toContain('LOCK: released'); + openPr(fx, '24'); + expect(land(fx, ['--pr', '24']).status).toBe(EXIT.ok); + }, 90_000); + + // Value: protects=--break-lock never breaks a holder that entered merging moments ago; fails_when=two landers squash the same version; why_new=only the push phase was guarded; seam=none; no_claim=none beyond protects + test('--break-lock refuses a lock whose holder entered merging moments ago', () => { + const fx = setup(); + openPr(fx, '21'); openPr(fx, '22'); + land(fx, ['--pr', '21']); + fx.write(d => { d.mergeRefuse = true; }); + expect(land(fx, ['--pr', '21']).err).toContain('gh pr merge refused'); + expect(land(fx, ['--pr', '22', '--break-lock']).err).toMatch(/#21's holder is mid-merge \(\d+ s ago\); --break-lock refused.*\(QUEUE_LAND_LOCKED\)/); + }, 90_000); + + // Value: protects=--repair resumes after its branch was pushed but PR creation failed, and the repair PR takes over from the lock alone; fails_when=every retry fails the branch lease, or a clean checkout cannot take over the drift lock; why_new=the drift case keeps the journal and creates the PR first time; seam=none; no_claim=gh's exact "already exists" wording + test('--repair resumes after a failed PR create; the repair PR takes over the drift lock with no local journal', () => { + const fx = setup(); + openPr(fx, '23'); + land(fx, ['--pr', '23']); + fx.write(d => { d.driftOnMerge = true; }); + expect(land(fx, ['--pr', '23']).err).toContain('(QUEUE_LAND_DRIFT)'); + fx.write(d => { d.createFail = true; }); + expect(land(fx, ['--pr', '23', '--repair']).err).toContain('gh pr create failed'); + Bun.sleepSync(1100); + const rep = land(fx, ['--pr', '23', '--repair']); + expect(rep.err).toBe(''); + expect(rep.out).toContain('already on origin'); + const n = /REPAIR_PR: #(\d+) gstack-land-repair\/23/.exec(rep.out)!; + const head = remote(fx, 'gstack-land-repair/23'); + fx.write(d => { d.prs[n[1]!].body = shippedBody(n[1]!, head); }); + markReady(fx, n[1]!, head); + fs.rmSync(path.join(fx.repo.work, '.gstack', 'tmp', 'ship-queue', 'land', '23.json')); + expect(land(fx, ['--pr', n[1]!]).out).toContain("LOCK: taking over #23's drift lock for its repair"); + const merged = land(fx, ['--pr', n[1]!]); + expect(merged.status).toBe(EXIT.ok); + expect(merged.out).toContain('LOCK: released'); + }, 120_000); + + // Value: protects=land never writes through a PR-controlled symlink while unstamping a legacy pin; fails_when=a VERSION link chain overwrites a file outside the checkout, even in --dry-run; why_new=only the title fragment was guarded; seam=none; no_claim=Windows symlink semantics + test('a legacy pin whose VERSION is a symlink refuses in --dry-run and writes nothing outside', () => { + const fx = setup(); + const outside = path.join(fx.repo.root, 'outside.txt'); + fs.writeFileSync(outside, 'keep\n'); + openPr(fx, '25', { edit: w => { fs.rmSync(path.join(w, 'VERSION')); fs.symlinkSync('1.2.9.0', path.join(w, 'VERSION')); fs.symlinkSync(outside, path.join(w, '1.2.9.0')); } }); + const r = land(fx, ['--pr', '25', '--dry-run']); + expect(r.err).toMatch(/VERSION is a symlink at the pin.*\(QUEUE_LAND_CONFLICT\)/); + expect(fs.readFileSync(outside, 'utf8')).toBe('keep\n'); + }, 90_000); + + // Value: protects=restamp never folds fragments through a PR-controlled symlinked CHANGELOG; fails_when=the fold rewrites a file outside the checkout, even in --dry-run; why_new=the legacy guard runs only when the pin bumped VERSION; seam=none; no_claim=Windows symlink semantics + test('a pin whose CHANGELOG is a symlink refuses before restamp and leaves the target untouched', () => { + const fx = setup(); + const outside = path.join(fx.repo.root, 'outside-changelog.md'); + fs.writeFileSync(outside, '# Changelog\n\n## [1.2.3.0] - 2026-10-01\n\n- base release\n'); + openPr(fx, '27', { edit: w => { fs.rmSync(path.join(w, 'CHANGELOG.md')); fs.symlinkSync(outside, path.join(w, 'CHANGELOG.md')); } }); + const r = land(fx, ['--pr', '27', '--dry-run']); + expect(r.err).toMatch(/CHANGELOG\.md is a symlink in the merged tree.*\(QUEUE_LAND_CONFLICT\)/); + expect(fs.readFileSync(outside, 'utf8')).toBe('# Changelog\n\n## [1.2.3.0] - 2026-10-01\n\n- base release\n'); + }, 90_000); + + // Value: protects=a legacy pin in a repo whose version source is package.json keeps its dependency and script edits; fails_when=the whole manifest is reset to the merge base; why_new=every case used a VERSION file; seam=none; no_claim=lockfile shapes beyond package.json + test('version source package.json: a legacy pin keeps its dependency and script edits', () => { + const fx = setup(); + fx.repo.git('checkout', '-q', '-B', 'vp', 'origin/main'); + fs.mkdirSync(path.join(fx.repo.work, '.gstack'), { recursive: true }); + fs.writeFileSync(path.join(fx.repo.work, '.gstack', 'version-path'), 'package.json\n'); + const base = JSON.parse(readFile(fx.repo, 'package.json')); base.version = '1.2.3.0'; + fs.writeFileSync(path.join(fx.repo.work, 'package.json'), JSON.stringify(base, null, 2) + '\n'); + fx.repo.git('add', '-A'); fx.repo.git('commit', '-q', '-m', 'version from package.json'); fx.repo.git('push', '-q', 'origin', 'HEAD:refs/heads/main'); + fx.repo.git('fetch', '-q', 'origin'); + openPr(fx, '26', { edit: w => { const p = JSON.parse(fs.readFileSync(path.join(w, 'package.json'), 'utf8')); p.version = '1.2.9.0'; p.dependencies['right-pad'] = '2.0.0'; p.scripts = { ...p.scripts, build: 'node build.js' }; fs.writeFileSync(path.join(w, 'package.json'), JSON.stringify(p, null, 2) + '\n'); } }); + const r = land(fx, ['--pr', '26', '--dry-run']); + expect(r.err).toBe(''); + expect(r.out).toMatch(/^legacy: the pin stamped 1\.2\.9\.0 itself/m); + const tree = /^STAMP_TREE: ([0-9a-f]+)/m.exec(r.out)![1]!; + const pkg = JSON.parse(show(fx, tree + ':package.json')); + expect(pkg.dependencies['right-pad']).toBe('2.0.0'); + expect(pkg.scripts.build).toBe('node build.js'); + expect(pkg.version).toBe('1.2.4'); + }, 90_000); +}); + +// GStack Minder ruling (2026-10-29, GSTA-154): standalone land checks the evidence ready enforces. +describe('land: the READY receipt', () => { + // Value: protects=land refuses a pin ready never accepted (pregate, full_e2e), before any lock or push; fails_when=land merges a head that skipped ready; why_new=every other case marks its PR ready; seam=none; no_claim=what ready itself checks (its own tests) + test('a pin with a /ship receipt but no READY receipt refuses, naming the ready command; nothing is locked or pushed', () => { + const fx = setup(); + const pin = openPr(fx, '28', { ready: false }); + for (const args of [['--pr', '28'], ['--pr', '28', '--dry-run']]) { + const r = land(fx, args); + expect(r.status).toBe(EXIT.refused); + expect(r.err).toMatch(/no READY receipt for #28 at the pin [0-9a-f]{12}.*fix: run gstack-ship-queue ready --pr 28 \(QUEUE_NOT_READY\)/); + } + expect(remote(fx, 'pr-28')).toBe(pin); + expect(fx.repo.git('ls-remote', 'origin', 'refs/notes/gstack-land')).toBe(''); + markReady(fx, '28', pin); + expect(land(fx, ['--pr', '28']).out).toContain('PUSHED: '); + }, 90_000); +}); diff --git a/test/ship-queue-train.test.ts b/test/ship-queue-train.test.ts index ff77594b0..0f47abd68 100644 --- a/test/ship-queue-train.test.ts +++ b/test/ship-queue-train.test.ts @@ -108,6 +108,23 @@ describe('the policy gate', () => { expect(flagged.err).toContain('--screen/--no-screen/--broad ignored: the base policy declares no train block'); expect(fs.existsSync(path.join(fx.state, 'ship-queue'))).toBe(false); }, 120_000); + + // Value: protects=under changelog_fragments ready records the READY receipt land checks, only after every ready check passed; fails_when=land has no proof of the pregate; why_new=the case above covers the key off; seam=none; no_claim=the notes push to a real GitHub remote + test('under changelog_fragments ready records a READY receipt; a failed pregate records none', () => { + const fx = make({ ...BASE_POLICY, changelog_fragments: { dir: 'changelog.d' }, min_gstack_version: '1.92.0.0' }); + const head = openPr(fx, '1'); + fx.write(d => { d.ci[head] = [{ status: 'completed', conclusion: 'success' }]; }); + const tree = runBin(fx.repo, 'gstack-wtree', []).out.trim(); + const pregate = path.join(fx.repo.root, 'pregate.json'); + fs.writeFileSync(pregate, JSON.stringify({ schema_version: 1, tree, head, verdict: 'fail', checks: [{ id: 'unit', status: 'fail' }] })); + expect(queue(fx, ['ready', '--pr', '1', '--pregate', pregate]).status).toBe(EXIT.refused); + expect(fs.existsSync(path.join(fx.state, 'ship-queue'))).toBe(false); + fs.writeFileSync(pregate, JSON.stringify({ schema_version: 1, tree, head, verdict: 'pass', checks: [{ id: 'unit', status: 'pass' }] })); + const ok = queue(fx, ['ready', '--pr', '1', '--pregate', pregate]); + expect(ok.status).toBe(EXIT.ok); + expect(ok.out).toMatch(new RegExp(`^READY_RECEIPT: .*ship-queue/ready/[^/]+/1-${head}\\.json \\(notes pushed to origin refs/notes/gstack-trains\\)$`, 'm')); + expect(fx.repo.git('ls-remote', 'origin', 'refs/notes/gstack-trains')).not.toBe(''); + }, 120_000); }); describe('ready under a train policy: the screen choice', () => { diff --git a/test/size-ledger/ship/gsta154-changelog-fragments.json b/test/size-ledger/ship/gsta154-changelog-fragments.json new file mode 100644 index 000000000..e75f27886 --- /dev/null +++ b/test/size-ledger/ship/gsta154-changelog-fragments.json @@ -0,0 +1,4 @@ +{ + "bytes": -1, + "why": "GSTA-154: the ship-policy paragraph says its ship: lines bind Steps 12-16, so the changelog_fragments line reaches Step 13's entry (an ASCII hyphen keeps the skeleton within maxSkeletonBytes)" +} diff --git a/test/version-gate-fragments.test.ts b/test/version-gate-fragments.test.ts new file mode 100644 index 000000000..8f65a748f --- /dev/null +++ b/test/version-gate-fragments.test.ts @@ -0,0 +1,121 @@ +/** + * Version Gate under changelog_fragments (docs/ship-policy.md#changelog-fragments): an unstamped head + * keeps its release-version fields at its merge base and carries a fragment (or a legacy [Unreleased] + * entry, or is a bot PR); other manifest edits pass; a merge-step stamp commit must carry base plus the + * effective bump of the fragments its pin folds. + */ +import { afterEach, describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { BASE_POLICY, cleanup, makeFixtureRepo, ROOT, type FixtureRepo } from './helpers/restamp-fixture'; + +const SCRIPT = path.join(ROOT, 'scripts', 'compare-pr-version.ts'); +const POLICY = { ...BASE_POLICY, release_tool: null, release_outputs: [], changelog_fragments: { dir: 'changelog.d' }, min_gstack_version: '1.92.0.0' }; +const repos: FixtureRepo[] = []; +afterEach(() => { while (repos.length) cleanup(repos.pop()!); }); +const fixture = () => { const r = makeFixtureRepo({ policy: POLICY, prefix: 'version-gate-frag-' }); repos.push(r); return r; }; +const write = (r: FixtureRepo, rel: string, text: string) => { fs.mkdirSync(path.dirname(path.join(r.work, rel)), { recursive: true }); fs.writeFileSync(path.join(r.work, rel), text); }; +const commit = (r: FixtureRepo, msg: string) => { r.git('add', '-A'); r.git('commit', '-q', '-m', msg); return r.git('rev-parse', 'HEAD'); }; +function run(r: FixtureRepo, head: string, extra: Record = {}, claimed: unknown[] = []) { + const next = path.join(r.root, 'next.json'); + fs.writeFileSync(next, JSON.stringify({ version: '1.2.4.0', base_version: '1.2.3.0', claimed })); + const prVersion = r.git('show', head + ':VERSION').trim(); + const out = Bun.spawnSync([process.execPath, 'run', SCRIPT, next, '3600'], { cwd: r.work, env: { ...process.env, ...r.env, PR_VERSION: prVersion, BASE_REF: 'main', HEAD_SHA: head, PR_AUTHOR: 'garrytan', ...extra }, timeout: 30_000, stdout: 'pipe', stderr: 'pipe' }); + return { code: out.exitCode, out: out.stdout.toString() + out.stderr.toString() }; +} + +describe('version gate: changelog fragments', () => { + test('a fragment head passes, a dependency or script edit beside it too', () => { + const r = fixture(); + r.git('checkout', '-q', '-b', 'f'); + write(r, 'changelog.d/f.md', '- f\n'); + const pkg = JSON.parse(fs.readFileSync(path.join(r.work, 'package.json'), 'utf8')); + pkg.dependencies['right-pad'] = '2.0.0'; pkg.scripts.build = 'node build.js'; + write(r, 'package.json', JSON.stringify(pkg, null, 2) + '\n'); + const res = run(r, commit(r, 'f')); + expect(res.code).toBe(0); + expect(res.out).toContain('its entry is 1 CHANGELOG fragment(s)'); + }); + + test('a premature version change fails; no entry fails for a person and passes for a bot', () => { + const r = fixture(); + r.git('checkout', '-q', '-b', 'g'); + write(r, 'changelog.d/g.md', '- g\n'); + write(r, 'VERSION', '1.2.9.0\n'); + const early = run(r, commit(r, 'g')); + expect(early.code).toBe(1); + expect(early.out).toContain('under changelog_fragments only the merge step stamps'); + r.git('checkout', '-q', '-B', 'self', 'origin/main'); + write(r, 'VERSION', '1.2.4.0\n'); write(r, 'self.txt', 'self\n'); + const self = run(r, commit(r, 'a lane stamps the expected version itself, with no fragment')); + expect(self.code).toBe(1); + expect(self.out).toContain('under changelog_fragments only the merge step stamps'); + r.git('checkout', '-q', '-B', 'h', 'origin/main'); + write(r, 'h.txt', 'h\n'); + const head = commit(r, 'h'); + const none = run(r, head); + expect(none.code).toBe(1); + expect(none.out).toContain('add a CHANGELOG fragment: changelog.d/.md'); + expect(run(r, head, { PR_AUTHOR: 'dependabot[bot]' }).code).toBe(0); + }); + + test('an unchanged branch on an advanced base passes; an invalid fragment fails', () => { + const r = fixture(); + r.git('checkout', '-q', '-b', 'k'); + write(r, 'changelog.d/k.md', '- k\n'); + const head = commit(r, 'k'); + r.git('checkout', '-q', 'main'); + write(r, 'VERSION', '1.2.4.0\n'); write(r, 'other.txt', 'x\n'); + commit(r, 'main moves'); r.git('push', '-q', 'origin', 'main'); + expect(run(r, head).code).toBe(0); + r.git('checkout', '-q', '-B', 'bad', 'origin/main'); + write(r, 'changelog.d/bad.md', '## [9.9.9.9]\n- bad\n'); + const bad = run(r, commit(r, 'bad')); + expect(bad.code).toBe(1); + expect(bad.out).toContain('changelog.d/bad.md: a fragment carries no release heading'); + }); + + test('a merge-step stamp commit [pin, base] carries base plus its fragments\' effective bump', () => { + const r = fixture(); + r.git('checkout', '-q', '-b', 'm'); + write(r, 'changelog.d/m.md', '---\nbump: minor\n---\n- m\n'); + const pin = commit(r, 'm'); + const stamp = (version: string) => { + r.git('checkout', '-q', '-f', '--detach', pin); + fs.rmSync(path.join(r.work, 'changelog.d', 'm.md')); + write(r, 'VERSION', version + '\n'); + write(r, 'CHANGELOG.md', fs.readFileSync(path.join(r.work, 'CHANGELOG.md'), 'utf8').replace('# Changelog\n\n', '# Changelog\n\n## [' + version + '] - 2026-10-29\n\n- m\n\n')); + r.git('add', '-A'); + return r.git('commit-tree', r.git('write-tree'), '-p', pin, '-p', r.git('rev-parse', 'origin/main'), '-m', 'v' + version + ' stamp'); + }; + expect(run(r, stamp('1.3.0.0')).code).toBe(0); + const claimedToo = run(r, stamp('1.3.0.0'), {}, [{ pr: 77, branch: 'legacy', version: '1.3.0.0' }]); + expect(claimedToo.code).toBe(0); + expect(claimedToo.out).toContain('the land lock serializes allocation, so open claims do not apply'); + const wrong = run(r, stamp('1.2.4.0')); + expect(wrong.code).toBe(1); + expect(wrong.out).toContain('must carry v1.3.0.0'); + }); + + test('the workflow runs on every PR (no path filter), so a PR without a release file still proves its fragment', () => { + const workflow = Bun.YAML.parse(fs.readFileSync(path.join(ROOT, '.github', 'workflows', 'version-gate.yml'), 'utf8')) as any; + expect(workflow.on.pull_request.paths).toBeUndefined(); + expect(workflow.on.pull_request.branches).toEqual(['main']); + }); +}); + +// Generated by /ship Step 7 coverage audit (GSTA-154 PR A, pass 1). +describe('version gate: legacy entries under changelog fragments', () => { + // Value: protects=a person's PR with a legacy [Unreleased] entry and no fragment still passes once fragments are on; + // fails_when=the [Unreleased] fallback in classifyTrainHead is dropped, so in-flight PRs fail with "add a CHANGELOG fragment"; + // why_new=existing gate tests cover fragment, bot and no-entry heads only; seam=none; no_claim=GitHub workflow wiring (script run directly) + test('an [Unreleased] entry without a fragment passes and names that entry', () => { + const r = fixture(); + r.git('checkout', '-q', '-b', 'legacy'); + write(r, 'legacy.txt', 'l\n'); + write(r, 'CHANGELOG.md', fs.readFileSync(path.join(r.work, 'CHANGELOG.md'), 'utf8').replace('# Changelog\n', '# Changelog\n\n## [Unreleased]\n\n- legacy entry\n')); + const res = run(r, commit(r, 'legacy')); + expect(res.code, res.out).toBe(0); + expect(res.out).toContain('the CHANGELOG change is confined to [Unreleased]'); + }); +});