train S1: merge #3917 at e9277ef4d4 v1.95.0.0

This commit is contained in:
garrytan committed 2000-01-01 00:00:00 +00:00
commit baed2e8f3b
28 files changed
+1588 -58

No files matched your search

+4 -1
View File
@@ -299,7 +299,10 @@ through an urgent one-PR train (or `train build --repair` while the queue is fro
refuses `--urgent`); all prefixes green earns exactly one unchanged rebuild;
anything else goes to a repair lane. An infra-only census earns one re-dispatch. A
backstop gate runs on every push to the base branch and freezes the queue when it finds
a tree no green train covered; one repair PR at its exact head clears the freeze.
a tree no green train covered; one repair PR at its exact head clears the freeze. A
policy with a `release_gate` block swaps that per-push backstop for one full gate on main
every 8 hours, whose red verdict holds the stable release instead of freezing the queue
([ship-policy.md#release-gate](ship-policy.md#release-gate)).
**Wave PRs when the queue is long.** Every slot pays its own
stamp, its own round of required checks and its own version, one after another. When more
+5 -2
View File
@@ -4,7 +4,7 @@ Three layers, each owned by gstack:
| Layer | Where | Job |
|---|---|---|
| Bootstrapper | `capy/skill/gstack/scripts/install.sh` (copied once into the owner's Drive skill folder; frozen) | clone when absent; fetch origin's default branch and read the supported minimum from `FETCH_HEAD:capy/support.env`; in install mode fast-forward a clean, unpinned checkout below it; exec the full wrapper |
| Bootstrapper | `capy/skill/gstack/scripts/install.sh` (copied once into the owner's Drive skill folder; frozen) | clone when absent; fetch origin's default branch and read the supported minimum from `FETCH_HEAD:capy/support.env` (from wrapper 4 a fresh clone or the floor fast-forward follows `stable` instead when the update channel resolves to it and `stable` meets that minimum, see [releases.md](releases.md#channels)); in install mode fast-forward a clean, unpinned checkout below it; exec the full wrapper |
| Full wrapper | `bin/gstack-capy-session` (moves with the checkout) | map the session start onto the installer: `--check` certifies (and reports a checkout behind origin), otherwise fast-forward a checkout behind origin (`bin/gstack-upgrade --ff-only-no-reinstall`), then install or repair |
| Installer | `bin/gstack-capy-install` | Bun, both host profiles, the doctor, the install verdict, the launch line, the trailer |
@@ -594,7 +594,10 @@ Wrapper 2 only added optional exports (the trailer's `latest:` note and the
`cloned` line). Wrapper 3 answers `--check --for none` itself (a copy at 1 or 2
prints `fail not-installed` for a coordinator machine and still works for every
other call), so the minimum is 3: every older Drive copy prints the owner
action until the owner recopies `capy/skill/gstack/`.
action until the owner recopies `capy/skill/gstack/`. Wrapper 4 only adds the
update-channel choice for the clone and the floor fast-forward
(`GSTACK_UPDATE_CHANNEL=stable|main`, else the checkout's channel, else main);
the minimum stays 3, so a copy at 3 keeps working.
## Exit codes
+17 -5
View File
@@ -29,11 +29,12 @@ it has completed and none failed, and `refs/notes/gstack-trains` holds no
active freeze. A cancelled check run counts as red. A commit with a check run
still going is skipped and the next older one is considered, back through 7
days or 200 first-parent commits. The workflow runs no tests and no paid evals
of its own; it only reads what the trains and the backstop already recorded.
of its own; it only reads what the trains, the backstop and the release gate
already recorded.
With nothing green in the window it fails and lists why each commit was
skipped.
**Gated** means one of two pieces of evidence. The first is the
**Gated** means one of three pieces of evidence. The first is the
`main-backstop` check run: the Main backstop gate workflow publishes it on
each push to main from a job of its own, success when a green train record
covers the tree or the backstop gated it green, neutral when main moved past
@@ -45,11 +46,17 @@ its tree: those entries are the main commits `gstack-ship-queue verify
--train` confirmed, so a commit a green train merged is gated even when trains
merged faster than the backstop could settle and its own backstop run ended
neutral or never ran. The `CANDIDATE` line says which: `gated by
main-backstop` or `gated by train <nonce>`. Train evidence relaxes nothing
main-backstop` or `gated by train <nonce>`. The third is a GREEN release
record (kind `release`) on the commit, written by the release gate
([docs/ship-policy.md#release-gate](ship-policy.md#release-gate)) when the
policy declares `release_gate`; the line then says `gated by release gate run
<id>`. A RED release record with no GREEN one on a later commit that
contains it refuses the cut like a freeze. Train evidence relaxes nothing
else: a red backstop run, a failed or missing `free-tests` run, any other red
check run or a run still going keeps the commit out, and a record in any other
state (`red`, `superseded`, `free-green`, …) is not evidence. A commit with
neither a successful backstop run nor a train record is not released: a
none of the three (no successful backstop run, train record or GREEN release
record) is not released: a
dependabot push, a policy without a train block, or a tree that was never
gated.
@@ -240,6 +247,10 @@ overrides the key for one command. The resolution lives in one helper,
- `bin/gstack-upgrade` and `/gstack-upgrade`: they fast-forward to the channel
ref (`UPSTREAM_REF=origin/stable` in the skill's output) and say which channel
they followed.
- `capy/skill/gstack/scripts/install.sh`, the Capy skill's installer (wrapper
revision 4): a fresh clone or a floor upgrade follows `stable` when the channel
resolves to it and `stable` meets the support floor, and follows main
otherwise. `--revision` still pins an exact commit.
Two edge cases are handled on purpose:
@@ -302,11 +313,12 @@ is exercised against real check runs before it merges.
| Line | Meaning |
|---|---|
| `CANDIDATE: <sha> (<date>) gated by main-backstop|train <nonce>` | the commit that will be (or was) released, and which evidence gated it (`[hotfix: --ref]` before `gated by` on a `--ref` cut) |
| `CANDIDATE: <sha> (<date>) gated by main-backstop|train <nonce>|release gate run <id>` | the commit that will be (or was) released, and which evidence gated it (`[hotfix: --ref]` before `gated by` on a `--ref` cut) |
| `NOTHING_NEW: origin/main <sha> is already released as v<X>` | nothing merged since the last release; exit 0 |
| `NO_GREEN_COMMIT: ...` with `SKIPPED:` lines | nothing in the window passed the gate; exit 1 (a dry run exits 0: the skips are its answer) |
| `REFUSED: non-fast-forward: ...` | the candidate is behind stable; stable only moves forward |
| `REFUSED: queue frozen since train ...` | the main backstop froze the queue; clear it first (docs/ship-policy.md) |
| `REFUSED: release gate red on <sha> (<cases>) and no gate on a later commit is green; ...` | the release gate's hold: land the revert or fix-forward, then `gstack-release-gate run --ignore-cadence` ([docs/ship-policy.md#release-gate](ship-policy.md#release-gate)) |
| `REFUSED: tag v<X> already names <other sha>` | a version was stamped twice; needs a look before anything is cut |
| `REFUSED_FLOOR: stable moved to v<X> at <time>, <h> h ago; ...` | inside the 72-hour floor; nothing changed. Exit 1 by hand, 0 on the schedule or a dry run |
| `NEXT_ALLOWED: <time> for a hand-run cut; <time> for a scheduled run` | follows every `REFUSED_FLOOR` (and `FLOOR_UNKNOWN`, where it says why there is no time) |
+82
View File
@@ -1120,6 +1120,88 @@ every lane to the per-PR queue at once. `--urgent` ships one PR alone,
`--no-screen` skips the paid screen, `--broad` widens it to the full PR selection,
and `train build --prs <list without it>` drops a PR at any step.
<a id="release-gate"></a>
## `release_gate`: one full gate on main every 8 hours
Trains gate each batch of PRs, and the main backstop gates every push. The
release gate replaces the backstop with one full paid gate on main's head at
most every `cadence_hours`, and only when main moved since the last gate. It
feeds the stable release (docs/releases.md) and never writes `stable` itself:
`scripts/release-cut.ts` stays the only writer, with its 72-hour floor and its
security and regression exceptions unchanged.
The surface is off unless main's `.gstack/ship-policy.json` declares the
block, with `min_gstack_version` at least the release that introduced the key
(validation names it):
```json
"release_gate": {
"cadence_hours": 8, "gate_workflow": "evals.yml", "branch": "main",
"stable_ref": "stable", "check_name": "main-backstop",
"on_red": "hold-release", "bisect_spend_cap_usd": 40
}
```
Every field is required. `cadence_hours` runs from 8 (the workflow's
schedule) to 168. `check_name` is the check run release-cut reads as gated,
so gstack keeps `main-backstop`. `bisect_spend_cap_usd` caps one red gate's
bisect (0 skips it).
**What runs.** `.github/workflows/release-gate.yml` fires every 8 hours and
on a manual dispatch from main. Without the block it stops after a git-and-jq
policy read, so no paid work starts and nothing is written. With the block it runs
`gstack-release-gate run`, which:
1. first finishes an open gate on an older commit (a run that outlived its
budget, or a red gate's unfinished bisect), so main moving never orphans a
verdict; only then does it look at main's head. An open gate older than
three cadences stops holding up the head;
2. skips when main has not moved since `stable` or since the last gate, or
when the last gate is younger than `cadence_hours` (`--ignore-cadence`
overrides only the age);
3. writes a `release` record (`gstack-trains:release:<repo>:<sha>`) to
`refs/notes/gstack-trains` before it dispatches, so a lost run resumes;
4. dispatches the full gate on that sha and reads its verdict and flake
records. A case that failed once and passed its one rule-16 rerun is green
and is listed as flaky;
5. records GREEN, RED or INCOMPLETE. An INCOMPLETE gate is re-dispatched once
by the next run, then left for a person;
6. on RED, bisects the red cases over main's first-parent commits since the
last green gate (at most 4 rounds, within the spend cap) and records the
first bad commit;
7. publishes `check_name` on the gated commit from a separate job that holds
`checks: write` and runs no repository code: success for a green gate,
failure for a red one, and neutral when the gate has no verdict yet (still
running at the budget, or incomplete), so a later green on the same commit
is not undercut by an earlier failure.
**What release-cut reads.** A GREEN release record on a commit counts as
gated, the same as a successful `main-backstop` check run. A RED record
with no GREEN record on a later commit that contains it holds every cut, like a
queue freeze: the `REFUSED` line names the red sha and cases. Land the revert
or the fix-forward, then gate again with `gstack-release-gate run
--ignore-cadence` (or dispatch the workflow). Nothing else clears a hold.
**The per-push backstop stands down.** `evals-main-backstop.yml` keeps its
push trigger. While the block is set, its policy step reports `train=false`,
so a push dispatches no gate, freezes nothing and publishes no check run. Main
is never gated twice. Trains keep running until the train lane retires them.
```bash
gstack-release-gate run [--dry-run] [--sha <sha>] [--ignore-cadence]
gstack-release-gate status # the newest record, the hold if any, pending commits
gstack-release-gate bisect [--sha <sha>] [--spend-cap <usd>] [--dry-run]
```
Common flags: `--repo`, `--policy <file>` (read the block from a file instead
of main, for a dry run or a proof before adoption), `--remote`,
`--summary`, `--outputs`, `--poll-seconds`, `--budget-minutes`. Every line
starts with `RELEASE:`. Exit 0 means gated, skipped, inert or a dry run; 1
means held (red), failed, incomplete or a refused policy; 2 is a usage error.
**Escape hatch.** Remove the block on main. The per-push backstop gates the
next push again, and release-cut reads its check runs as before.
<a id="ship"></a>
## What `/ship` reads from it