mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-05 00:03:15 +01:00
docs: ship-policy, troubleshooting and captain for train gates review fixes
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
583cc1b372
commit
c6cb48c27a
4 files changed
+52
-18
No files matched your search
File renamed without changes.
+3
-2
@@ -248,7 +248,8 @@ a push included. The sequence that works unattended on Capy:
|
||||
([base_red](ship-policy.md#base-red)): the lane is waiting, not failed. It
|
||||
registers `gstack-base-red wait --pause <file>` as a background operation,
|
||||
tells the controller which cases hold it, and on `BASE_GREEN` runs
|
||||
`/ship --resume <record>`; no READY goes out on the conditional receipt.
|
||||
`/ship --resume <record>`; no READY goes out on the conditional receipt,
|
||||
except on a train with `gates` ([Merge trains for other repos](#merge-trains-for-other-repos)).
|
||||
9. READY: `gstack-ship-queue ready --pr <n> --pregate <pregate.json> --screen|--no-screen`;
|
||||
send the printed block.
|
||||
10. After the slot call, the stamp commit keeps the pin's receipt; re-READY with the same
|
||||
@@ -338,7 +339,7 @@ the verdict and lands each landable slot in order. Lanes READY with
|
||||
`ready --pr <n> --no-screen`; there is no screen and no slot call. A lane whose
|
||||
/ship paused on a red base READYs with `--base-red --fixer <pr>`, its fixer with
|
||||
`--fixes-base-red <cases>`, and the train seats the fixer at slot 1. The `update` commit `train land` pushes to a PR
|
||||
branch (the gated tree on [pin, main]) is the one exception to CREW rule 19's "never merge
|
||||
branch (the gated tree on [pin, main]) is the one exception to CREW rule 14's "never merge
|
||||
the base into a PR": it is tool-made, recorded before the push, and pushed under the pin's
|
||||
lease, like a stamped train's stamp commit. A peer crew
|
||||
adopting this starts from
|
||||
|
||||
+16
-8
@@ -60,7 +60,7 @@ refuses the file (`POLICY_GSTACK_TOO_OLD`). `validate` ends with
|
||||
| `review_scope` | `frozen` \| `diff-plus-blocking` \| `all` | `frozen` | The finding scope of `/ship`'s review loop; see [review_scope](#review-scope). `frozen` fixes the diff on pass 1, files blocking findings in unchanged code as issues, and later passes read only the range since the previous pass. `diff-plus-blocking` is the same but fixes a blocking finding in unchanged code in this PR instead of filing it. `all` is the earlier loop: every finding is fixed on every pass. Needs `min_gstack_version` when not `frozen`. |
|
||||
| `min_gstack_version` | version or `null` | `null` | The oldest gstack that may read the file; see the compatibility rule above. |
|
||||
| `docs` | `{ caps: [{ path, max_bytes }] }` or `null` | `null` | Documentation files with a byte cap. `/ship` Step 14.5 runs `gstack-ship-receipt docs-caps --base <base>` (the caps come from `origin/<base>`'s policy, never the branch's copy; `--policy <file>` is for a repository bootstrapping its policy) before the verification gate and prints one `DOCS_CAP: <path> <bytes>/<max> ok\|over` line per cap; any `over` (an absent file counts) exits 1 and blocks the ship until the file is trimmed or the cap raised on the base branch (`DOCS_AUDIT_BLOCKED`). At most 64 caps, each path contained. See [the ship receipt](#ship-receipt). Needs `min_gstack_version`. |
|
||||
| `train` | `{ gate_workflow, free_workflow, full_gate_check, record_artifact, max_size, fill_hours, screen_selection, trial_reservation_usd }` or `null` | `null` | Turns on [merge trains](#run-a-merge-train), read from the base revision only. Every field is required and an unknown field is `POLICY_INVALID`: `max_size` 1 to 6, `fill_hours` above 0 and at most 24, `screen_selection` `gstack-paid` \| `none`, `trial_reservation_usd` above 0 and at most 100. `null` keeps the per-PR queue exactly as it is. Needs `min_gstack_version`. |
|
||||
| `train` | `{ gate_workflow, free_workflow, full_gate_check, record_artifact, max_size, fill_hours, screen_selection, trial_reservation_usd, land? }`, `{ gates, land, max_size, fill_hours }` or `null` | `null` | Turns on [merge trains](#run-a-merge-train), read from the base revision only. Every field is required and an unknown field is `POLICY_INVALID`: `max_size` 1 to 6, `fill_hours` above 0 and at most 24, `screen_selection` `gstack-paid` \| `none`, `trial_reservation_usd` above 0 and at most 100. A block with `gates` ([any repository](#train-gates)) takes `max_size` 1 to 5 and needs `min_gstack_version` at least `TRAIN_GATES_KEY_VERSION`. `null` keeps the per-PR queue exactly as it is. Needs `min_gstack_version`. |
|
||||
| `base_red` | `{ setup, select, timeout_sec }` or `null` | `null` | How `/ship` Step 5 proves a failure is the base branch's own before pausing on it; see [base_red](#base-red). Read from the base revision only. Needs `min_gstack_version`. |
|
||||
| `changelog_fragments` | `{ dir, required_checks? }` or `null` | `null` | PRs carry unversioned [CHANGELOG fragments](#changelog-fragments) in `dir` (one flat, repo-relative directory, no `..`) and the merge step ([`land`](#land)) assigns the version. `required_checks` (up to 50 distinct names) are the checks [`train land`](#stamped-trains-under-changelog_fragments-train-land) requires present and green on a stamp commit. `null` keeps `[Unreleased]` sections and per-PR stamps. Needs `min_gstack_version`. |
|
||||
|
||||
@@ -479,7 +479,9 @@ prefixes.
|
||||
rerun is GREEN with a `FLAKY:` line, red again is real, and both attempts'
|
||||
jobs stay in the record. `post_merge_checks` names checks that must succeed
|
||||
on main after each land before the next slot lands (`verify --post-merge
|
||||
<sha> --checks <names>`; `train watch --land` runs it between slots). A workflow whose concurrency
|
||||
<sha> --checks <names>`; `train land` checks the previous slot's merge and
|
||||
refuses the next slot until they are green, and `train watch --land` runs it
|
||||
between slots). A workflow whose concurrency
|
||||
group is not keyed by `github.ref` (or the nonce) gets one prefix at a
|
||||
time.
|
||||
- **`command` gates.** Each prefix is checked out clean and detached and runs
|
||||
@@ -487,8 +489,8 @@ prefixes.
|
||||
(standard-16, `UBI_OWNER` required). The VM's `git write-tree` must equal
|
||||
the prefix tree, and exit 0 is GREEN. Only names in `pass_env` reach the VM
|
||||
(`UBICLOUD_*` never). Each run reserves its timeout plus 10 minutes at the
|
||||
list price, and one `spend_cap_usd` covers every run of the train, reruns
|
||||
included (3 dollars covers six prefixes and one rerun at 1800 s; `train init`
|
||||
list price, and each command gate's `spend_cap_usd` covers every run of that
|
||||
gate in the train, reruns included (3 dollars covers six prefixes and one rerun at 1800 s; `train init`
|
||||
prints that default and `train doctor` flags a cap below a full train). The
|
||||
next run past the cap refuses (`QUEUE_TRAIN_SPEND_CAP`); raising the cap is a
|
||||
policy commit that moves main, so it is followed by a new train.
|
||||
@@ -545,7 +547,8 @@ declares fragments nor stamps at merge. After the block lands on the base,
|
||||
`train doctor` re-checks it: a missing workflow, nonce input or run-name, a
|
||||
required job no workflow defines, a concurrency group not keyed by ref, a push
|
||||
trigger that matches `train-prefix/` refs, a workflow never green on the base,
|
||||
and a merge method GitHub has turned off (`QUEUE_TRAIN_DOCTOR_FINDINGS`).
|
||||
a command gate cap below what a full train reserves, a merge method GitHub has
|
||||
turned off, and `rebase` as the merge method (`QUEUE_TRAIN_DOCTOR_FINDINGS`).
|
||||
Neither command writes anything.
|
||||
|
||||
<a id="ship-before-ready"></a>
|
||||
@@ -883,7 +886,7 @@ repair. When verify confirms the repair merged, it prints `FREEZE: cleared`
|
||||
and `NEXT:` builds a train of the PRs still unmerged.
|
||||
|
||||
**5b. Coordinator: after the merge.** `gstack-ship-queue verify --post-merge
|
||||
<sha> [--timeout <s>]` (the merged commit's full 40-character SHA) watches
|
||||
<sha> [--timeout <s>] [--checks <a,b>]` (the merged commit's full 40-character SHA) watches
|
||||
every check run attached to it (the push workflows: free tests, the Windows
|
||||
subset, the main backstop) and every commit status until none is still
|
||||
running, prints one `CHECK: <name> <conclusion>` per check and
|
||||
@@ -892,7 +895,10 @@ Green needs at least one completed check and no failure; `neutral` and
|
||||
`skipped` (the backstop superseding itself) count as skipped; a pending
|
||||
commit status keeps the commit pending and a failed one makes it red. A
|
||||
failure is `QUEUE_POST_MERGE_RED` naming the checks; a timeout (default 2700 s) is
|
||||
`QUEUE_TRAIN_PENDING` with the same command as `NEXT:`. It keeps no process
|
||||
`QUEUE_TRAIN_PENDING` with the same command as `NEXT:`. With `--checks` (a
|
||||
workflows gate's `post_merge_checks`), each named check must also run and
|
||||
succeed: one that did not succeed is `QUEUE_POST_MERGE_RED`, and one that
|
||||
never ran is `QUEUE_TRAIN_PENDING` at the timeout. It keeps no process
|
||||
state, so rerunning it after a machine restart continues the watch.
|
||||
|
||||
**6. Clean up.** `gstack-ship-queue train prune [--train <nonce>] [--dry-run]`
|
||||
@@ -1189,7 +1195,9 @@ expiry), finishes every other step, and writes its receipt with `--base-red
|
||||
`expires_at`) makes it **conditional**. `gstack-ship-receipt result` prints
|
||||
`SHIP_PAUSED: base-red …` and `status=incomplete`; `ready`, `train build`,
|
||||
`verify --train`, `carry` and every other receipt consumer refuse a conditional
|
||||
receipt with `QUEUE_RECEIPT_CONDITIONAL`.
|
||||
receipt with `QUEUE_RECEIPT_CONDITIONAL`. The one exception is a train with
|
||||
`gates`: `ready --base-red --fixer <pr>` READYs the PR to ride behind its fixer
|
||||
([base-red PRs](#train-gates)).
|
||||
|
||||
To resume, a parent runs `gstack-base-red wait --pause <file>` as a background
|
||||
operation. On each new base head (`git ls-remote`) it reruns `setup` and
|
||||
|
||||
+33
-8
@@ -4555,9 +4555,12 @@ one-PR train.
|
||||
**Meaning.** The train's run (free tests, the full gate or the bisect) has
|
||||
not finished, so there is no verdict yet. The same code covers a `train watch`
|
||||
that reached its timeout, `verify --pr <n> --screen` on a screen run still
|
||||
going, `verify --post-merge` on checks that have not settled, and a
|
||||
`ready --dispatch-ci` workflow run still going at its `--timeout`; each prints
|
||||
the command to rerun as `NEXT:`.
|
||||
going, `verify --post-merge` on checks that have not settled, a
|
||||
`ready --dispatch-ci` workflow run still going at its `--timeout`, a `train
|
||||
build` or `train verdict` while another process holds the train's lease or
|
||||
runs one of its command gates, and a `train land` whose previous slot's
|
||||
post-merge checks have not settled; each prints the command to rerun as
|
||||
`NEXT:`.
|
||||
|
||||
**Fix.** Rerun the printed `NEXT:` command later. Nothing is dispatched again
|
||||
while a run is pending.
|
||||
@@ -4574,6 +4577,10 @@ record. An incomplete gate is never green.
|
||||
one re-dispatch EVAL_POLICY.infraRedispatch allows (`train rebuild --train
|
||||
<nonce> --infra`) and report both runs. Otherwise diagnose the run.
|
||||
|
||||
On a train with `gates` it means a prefix gate is incomplete (a required job
|
||||
that was skipped or never ran, for example); the `PREFIX:` line names the
|
||||
run. `train rebuild` refuses, so diagnose that run and build a new train.
|
||||
|
||||
<a id="queue-train-red"></a>
|
||||
### `QUEUE_TRAIN_RED`
|
||||
|
||||
@@ -4584,6 +4591,11 @@ verdict.
|
||||
**Fix.** Bisect the red cases (`train bisect --train <nonce> --spend-cap
|
||||
<usd>`). After a rebuild's second red, route the cases to a repair lane.
|
||||
|
||||
On a train with `gates` there is no bisect: T<i> is the first red prefix after
|
||||
a green one and every later prefix is red, so slot i is the culprit. The slots
|
||||
before it land (`train land`, or `train watch --land`), the culprit goes back
|
||||
to its lane, and `NEXT:` builds the rest into a new train.
|
||||
|
||||
<a id="queue-train-red-on-main"></a>
|
||||
### `QUEUE_TRAIN_RED_ON_MAIN`
|
||||
|
||||
@@ -4593,6 +4605,9 @@ included: main is broken, not the train's PRs.
|
||||
**Fix.** Fix forward on main first (a one-PR train), then rebuild this train
|
||||
unchanged.
|
||||
|
||||
On a train with `gates`, no bisect runs: T0 (main) is red on its own gate run,
|
||||
or red on anything other than exactly slot 1's `FIXES_BASE_RED` cases.
|
||||
|
||||
<a id="queue-train-unattributed"></a>
|
||||
### `QUEUE_TRAIN_UNATTRIBUTED`
|
||||
|
||||
@@ -4603,6 +4618,10 @@ monotone. One trial per prefix cannot tell a flake from a break.
|
||||
<nonce>`), reporting both runs. Any other pattern: a repair lane, then a
|
||||
changed train once the repair merges.
|
||||
|
||||
On a train with `gates` it means a red prefix is followed by a green one.
|
||||
Repair the red cases or seat the PRs in another order (`--order`); `train
|
||||
rebuild` refuses, so this train never runs again unchanged.
|
||||
|
||||
<a id="queue-train-bisect-capped"></a>
|
||||
### `QUEUE_TRAIN_BISECT_CAPPED`
|
||||
|
||||
@@ -4687,8 +4706,8 @@ again by its lane ([ship-policy.md](ship-policy.md#train-gates)).
|
||||
<a id="queue-train-spend-cap"></a>
|
||||
### `QUEUE_TRAIN_SPEND_CAP`
|
||||
|
||||
**Meaning.** A command gate's next Ubicloud run would pass the train's one
|
||||
spend cap, or `UBI_OWNER` is not set. Nothing more ran; finished runs keep
|
||||
**Meaning.** A command gate's next Ubicloud run would pass that gate's spend
|
||||
cap for the train, or `UBI_OWNER` is not set. Nothing more ran; finished runs keep
|
||||
their outcomes.
|
||||
|
||||
**Fix.** Set `UBI_OWNER` to your thread code and rerun `train build --train
|
||||
@@ -4725,7 +4744,8 @@ stamp-at-merge` on the base branch, then run `train init` again.
|
||||
`FINDING:` line names one problem: a gate workflow missing, without the
|
||||
`nonce` input or run-name, a required job it does not define, a global
|
||||
concurrency group, a push trigger on `train-prefix/` refs, a workflow never
|
||||
green on the base, or a merge method GitHub refuses.
|
||||
green on the base, a command gate cap below what a full train reserves, a
|
||||
merge method GitHub refuses, or `rebase` as the merge method.
|
||||
|
||||
**Fix.** Fix each finding on the base branch (`train init` prints the
|
||||
snippet), then run `train doctor` again.
|
||||
@@ -4813,7 +4833,9 @@ to fix this: the stamp comes only when the slot is called.
|
||||
**Meaning.** The PR body's ship receipt carries a `base_red` block: /ship's
|
||||
Step 5 failed on cases `gstack-base-red` proved are the base branch's own, so
|
||||
/ship paused before READY ([base_red](ship-policy.md#base-red)). Nothing
|
||||
READYs, trains, carries or merges on it.
|
||||
READYs, trains, carries or merges on it, except on a train with `gates`, where
|
||||
`ready --base-red --fixer <pr>` READYs it to ride behind its fixer at slot 1
|
||||
([base-red PRs](ship-policy.md#train-gates)).
|
||||
|
||||
**Fix.** Wait for the base to pass those cases (`gstack-base-red wait --pause
|
||||
<file>`), then run `/ship --resume <record>`, which reruns Step 5 and rewrites
|
||||
@@ -5005,7 +5027,10 @@ or `--timeout` (default 2700 s) passes, prints `POST_MERGE: <sha12>
|
||||
green|red|pending|none (passed= failed= pending= skipped=)` and one
|
||||
`CHECK: <name> <conclusion|status>` line per check, and a timeout is
|
||||
`QUEUE_TRAIN_PENDING` with the same command as `NEXT:`, so rerunning it after
|
||||
an interruption continues the watch.
|
||||
an interruption continues the watch. With `--checks <a,b>` (a workflows
|
||||
gate's `post_merge_checks`, which `train land` and `train watch --land` pass),
|
||||
each named check must also succeed; one that never ran is `QUEUE_TRAIN_PENDING`
|
||||
at the timeout.
|
||||
|
||||
**Fix.** Read the red run named on its `CHECK:` line. Repair on a new PR or
|
||||
roll back the merge; `gstack-ship-queue train status` names the freeze if the
|
||||
|
||||
Reference in new issue
Block a user