mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-11 03:00:26 +01:00
train S1: merge #3917 at e9277ef4d4 v1.95.0.0
This commit is contained in:
commit
baed2e8f3b
28 files changed
+1588
-58
No files matched your search
+4
-1
@@ -299,7 +299,10 @@ through an urgent one-PR train (or `train build --repair` while the queue is fro
|
||||
refuses `--urgent`); all prefixes green earns exactly one unchanged rebuild;
|
||||
anything else goes to a repair lane. An infra-only census earns one re-dispatch. A
|
||||
backstop gate runs on every push to the base branch and freezes the queue when it finds
|
||||
a tree no green train covered; one repair PR at its exact head clears the freeze.
|
||||
a tree no green train covered; one repair PR at its exact head clears the freeze. A
|
||||
policy with a `release_gate` block swaps that per-push backstop for one full gate on main
|
||||
every 8 hours, whose red verdict holds the stable release instead of freezing the queue
|
||||
([ship-policy.md#release-gate](ship-policy.md#release-gate)).
|
||||
|
||||
**Wave PRs when the queue is long.** Every slot pays its own
|
||||
stamp, its own round of required checks and its own version, one after another. When more
|
||||
|
||||
@@ -4,7 +4,7 @@ Three layers, each owned by gstack:
|
||||
|
||||
| Layer | Where | Job |
|
||||
|---|---|---|
|
||||
| Bootstrapper | `capy/skill/gstack/scripts/install.sh` (copied once into the owner's Drive skill folder; frozen) | clone when absent; fetch origin's default branch and read the supported minimum from `FETCH_HEAD:capy/support.env`; in install mode fast-forward a clean, unpinned checkout below it; exec the full wrapper |
|
||||
| Bootstrapper | `capy/skill/gstack/scripts/install.sh` (copied once into the owner's Drive skill folder; frozen) | clone when absent; fetch origin's default branch and read the supported minimum from `FETCH_HEAD:capy/support.env` (from wrapper 4 a fresh clone or the floor fast-forward follows `stable` instead when the update channel resolves to it and `stable` meets that minimum, see [releases.md](releases.md#channels)); in install mode fast-forward a clean, unpinned checkout below it; exec the full wrapper |
|
||||
| Full wrapper | `bin/gstack-capy-session` (moves with the checkout) | map the session start onto the installer: `--check` certifies (and reports a checkout behind origin), otherwise fast-forward a checkout behind origin (`bin/gstack-upgrade --ff-only-no-reinstall`), then install or repair |
|
||||
| Installer | `bin/gstack-capy-install` | Bun, both host profiles, the doctor, the install verdict, the launch line, the trailer |
|
||||
|
||||
@@ -594,7 +594,10 @@ Wrapper 2 only added optional exports (the trailer's `latest:` note and the
|
||||
`cloned` line). Wrapper 3 answers `--check --for none` itself (a copy at 1 or 2
|
||||
prints `fail not-installed` for a coordinator machine and still works for every
|
||||
other call), so the minimum is 3: every older Drive copy prints the owner
|
||||
action until the owner recopies `capy/skill/gstack/`.
|
||||
action until the owner recopies `capy/skill/gstack/`. Wrapper 4 only adds the
|
||||
update-channel choice for the clone and the floor fast-forward
|
||||
(`GSTACK_UPDATE_CHANNEL=stable|main`, else the checkout's channel, else main);
|
||||
the minimum stays 3, so a copy at 3 keeps working.
|
||||
|
||||
## Exit codes
|
||||
|
||||
|
||||
+17
-5
@@ -29,11 +29,12 @@ it has completed and none failed, and `refs/notes/gstack-trains` holds no
|
||||
active freeze. A cancelled check run counts as red. A commit with a check run
|
||||
still going is skipped and the next older one is considered, back through 7
|
||||
days or 200 first-parent commits. The workflow runs no tests and no paid evals
|
||||
of its own; it only reads what the trains and the backstop already recorded.
|
||||
of its own; it only reads what the trains, the backstop and the release gate
|
||||
already recorded.
|
||||
With nothing green in the window it fails and lists why each commit was
|
||||
skipped.
|
||||
|
||||
**Gated** means one of two pieces of evidence. The first is the
|
||||
**Gated** means one of three pieces of evidence. The first is the
|
||||
`main-backstop` check run: the Main backstop gate workflow publishes it on
|
||||
each push to main from a job of its own, success when a green train record
|
||||
covers the tree or the backstop gated it green, neutral when main moved past
|
||||
@@ -45,11 +46,17 @@ its tree: those entries are the main commits `gstack-ship-queue verify
|
||||
--train` confirmed, so a commit a green train merged is gated even when trains
|
||||
merged faster than the backstop could settle and its own backstop run ended
|
||||
neutral or never ran. The `CANDIDATE` line says which: `gated by
|
||||
main-backstop` or `gated by train <nonce>`. Train evidence relaxes nothing
|
||||
main-backstop` or `gated by train <nonce>`. The third is a GREEN release
|
||||
record (kind `release`) on the commit, written by the release gate
|
||||
([docs/ship-policy.md#release-gate](ship-policy.md#release-gate)) when the
|
||||
policy declares `release_gate`; the line then says `gated by release gate run
|
||||
<id>`. A RED release record with no GREEN one on a later commit that
|
||||
contains it refuses the cut like a freeze. Train evidence relaxes nothing
|
||||
else: a red backstop run, a failed or missing `free-tests` run, any other red
|
||||
check run or a run still going keeps the commit out, and a record in any other
|
||||
state (`red`, `superseded`, `free-green`, …) is not evidence. A commit with
|
||||
neither a successful backstop run nor a train record is not released: a
|
||||
none of the three (no successful backstop run, train record or GREEN release
|
||||
record) is not released: a
|
||||
dependabot push, a policy without a train block, or a tree that was never
|
||||
gated.
|
||||
|
||||
@@ -240,6 +247,10 @@ overrides the key for one command. The resolution lives in one helper,
|
||||
- `bin/gstack-upgrade` and `/gstack-upgrade`: they fast-forward to the channel
|
||||
ref (`UPSTREAM_REF=origin/stable` in the skill's output) and say which channel
|
||||
they followed.
|
||||
- `capy/skill/gstack/scripts/install.sh`, the Capy skill's installer (wrapper
|
||||
revision 4): a fresh clone or a floor upgrade follows `stable` when the channel
|
||||
resolves to it and `stable` meets the support floor, and follows main
|
||||
otherwise. `--revision` still pins an exact commit.
|
||||
|
||||
Two edge cases are handled on purpose:
|
||||
|
||||
@@ -302,11 +313,12 @@ is exercised against real check runs before it merges.
|
||||
|
||||
| Line | Meaning |
|
||||
|---|---|
|
||||
| `CANDIDATE: <sha> (<date>) gated by main-backstop|train <nonce>` | the commit that will be (or was) released, and which evidence gated it (`[hotfix: --ref]` before `gated by` on a `--ref` cut) |
|
||||
| `CANDIDATE: <sha> (<date>) gated by main-backstop|train <nonce>|release gate run <id>` | the commit that will be (or was) released, and which evidence gated it (`[hotfix: --ref]` before `gated by` on a `--ref` cut) |
|
||||
| `NOTHING_NEW: origin/main <sha> is already released as v<X>` | nothing merged since the last release; exit 0 |
|
||||
| `NO_GREEN_COMMIT: ...` with `SKIPPED:` lines | nothing in the window passed the gate; exit 1 (a dry run exits 0: the skips are its answer) |
|
||||
| `REFUSED: non-fast-forward: ...` | the candidate is behind stable; stable only moves forward |
|
||||
| `REFUSED: queue frozen since train ...` | the main backstop froze the queue; clear it first (docs/ship-policy.md) |
|
||||
| `REFUSED: release gate red on <sha> (<cases>) and no gate on a later commit is green; ...` | the release gate's hold: land the revert or fix-forward, then `gstack-release-gate run --ignore-cadence` ([docs/ship-policy.md#release-gate](ship-policy.md#release-gate)) |
|
||||
| `REFUSED: tag v<X> already names <other sha>` | a version was stamped twice; needs a look before anything is cut |
|
||||
| `REFUSED_FLOOR: stable moved to v<X> at <time>, <h> h ago; ...` | inside the 72-hour floor; nothing changed. Exit 1 by hand, 0 on the schedule or a dry run |
|
||||
| `NEXT_ALLOWED: <time> for a hand-run cut; <time> for a scheduled run` | follows every `REFUSED_FLOOR` (and `FLOOR_UNKNOWN`, where it says why there is no time) |
|
||||
|
||||
@@ -1120,6 +1120,88 @@ every lane to the per-PR queue at once. `--urgent` ships one PR alone,
|
||||
`--no-screen` skips the paid screen, `--broad` widens it to the full PR selection,
|
||||
and `train build --prs <list without it>` drops a PR at any step.
|
||||
|
||||
<a id="release-gate"></a>
|
||||
## `release_gate`: one full gate on main every 8 hours
|
||||
|
||||
Trains gate each batch of PRs, and the main backstop gates every push. The
|
||||
release gate replaces the backstop with one full paid gate on main's head at
|
||||
most every `cadence_hours`, and only when main moved since the last gate. It
|
||||
feeds the stable release (docs/releases.md) and never writes `stable` itself:
|
||||
`scripts/release-cut.ts` stays the only writer, with its 72-hour floor and its
|
||||
security and regression exceptions unchanged.
|
||||
|
||||
The surface is off unless main's `.gstack/ship-policy.json` declares the
|
||||
block, with `min_gstack_version` at least the release that introduced the key
|
||||
(validation names it):
|
||||
|
||||
```json
|
||||
"release_gate": {
|
||||
"cadence_hours": 8, "gate_workflow": "evals.yml", "branch": "main",
|
||||
"stable_ref": "stable", "check_name": "main-backstop",
|
||||
"on_red": "hold-release", "bisect_spend_cap_usd": 40
|
||||
}
|
||||
```
|
||||
|
||||
Every field is required. `cadence_hours` runs from 8 (the workflow's
|
||||
schedule) to 168. `check_name` is the check run release-cut reads as gated,
|
||||
so gstack keeps `main-backstop`. `bisect_spend_cap_usd` caps one red gate's
|
||||
bisect (0 skips it).
|
||||
|
||||
**What runs.** `.github/workflows/release-gate.yml` fires every 8 hours and
|
||||
on a manual dispatch from main. Without the block it stops after a git-and-jq
|
||||
policy read, so no paid work starts and nothing is written. With the block it runs
|
||||
`gstack-release-gate run`, which:
|
||||
|
||||
1. first finishes an open gate on an older commit (a run that outlived its
|
||||
budget, or a red gate's unfinished bisect), so main moving never orphans a
|
||||
verdict; only then does it look at main's head. An open gate older than
|
||||
three cadences stops holding up the head;
|
||||
2. skips when main has not moved since `stable` or since the last gate, or
|
||||
when the last gate is younger than `cadence_hours` (`--ignore-cadence`
|
||||
overrides only the age);
|
||||
3. writes a `release` record (`gstack-trains:release:<repo>:<sha>`) to
|
||||
`refs/notes/gstack-trains` before it dispatches, so a lost run resumes;
|
||||
4. dispatches the full gate on that sha and reads its verdict and flake
|
||||
records. A case that failed once and passed its one rule-16 rerun is green
|
||||
and is listed as flaky;
|
||||
5. records GREEN, RED or INCOMPLETE. An INCOMPLETE gate is re-dispatched once
|
||||
by the next run, then left for a person;
|
||||
6. on RED, bisects the red cases over main's first-parent commits since the
|
||||
last green gate (at most 4 rounds, within the spend cap) and records the
|
||||
first bad commit;
|
||||
7. publishes `check_name` on the gated commit from a separate job that holds
|
||||
`checks: write` and runs no repository code: success for a green gate,
|
||||
failure for a red one, and neutral when the gate has no verdict yet (still
|
||||
running at the budget, or incomplete), so a later green on the same commit
|
||||
is not undercut by an earlier failure.
|
||||
|
||||
**What release-cut reads.** A GREEN release record on a commit counts as
|
||||
gated, the same as a successful `main-backstop` check run. A RED record
|
||||
with no GREEN record on a later commit that contains it holds every cut, like a
|
||||
queue freeze: the `REFUSED` line names the red sha and cases. Land the revert
|
||||
or the fix-forward, then gate again with `gstack-release-gate run
|
||||
--ignore-cadence` (or dispatch the workflow). Nothing else clears a hold.
|
||||
|
||||
**The per-push backstop stands down.** `evals-main-backstop.yml` keeps its
|
||||
push trigger. While the block is set, its policy step reports `train=false`,
|
||||
so a push dispatches no gate, freezes nothing and publishes no check run. Main
|
||||
is never gated twice. Trains keep running until the train lane retires them.
|
||||
|
||||
```bash
|
||||
gstack-release-gate run [--dry-run] [--sha <sha>] [--ignore-cadence]
|
||||
gstack-release-gate status # the newest record, the hold if any, pending commits
|
||||
gstack-release-gate bisect [--sha <sha>] [--spend-cap <usd>] [--dry-run]
|
||||
```
|
||||
|
||||
Common flags: `--repo`, `--policy <file>` (read the block from a file instead
|
||||
of main, for a dry run or a proof before adoption), `--remote`,
|
||||
`--summary`, `--outputs`, `--poll-seconds`, `--budget-minutes`. Every line
|
||||
starts with `RELEASE:`. Exit 0 means gated, skipped, inert or a dry run; 1
|
||||
means held (red), failed, incomplete or a refused policy; 2 is a usage error.
|
||||
|
||||
**Escape hatch.** Remove the block on main. The per-push backstop gates the
|
||||
next push again, and release-cut reads its check runs as before.
|
||||
|
||||
<a id="ship"></a>
|
||||
## What `/ship` reads from it
|
||||
|
||||
|
||||
Reference in new issue
Block a user