mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-12 19:44:48 +01:00
fix(pregate): remote gate finishes before pregate's deadline
pregate kills the remote gate at --timeout with a 5 s grace, which could cut a VM teardown short. pregate-remote.sh now reads the enclosing deadline (PREGATE_TIMEOUT_S, else the gstack-pregate --timeout it runs under, else 600), reserves 120 s for teardown plus 15 s margin, and tells the VM to stop at the earlier of that point and the cap's wall; a deadline that leaves under 300 s refuses before any VM (PREGATE_REMOTE_TIMEOUT_TOO_SHORT). It names the VM up front and prints the exact ubi-runner.sh down line, and the body is one braced block so an edit to the script mid-run cannot change what the run executes (keyed run 5 died after teardown with '$2: unbound variable' for that reason). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
3e14ca12f6
commit
8fa3e0d766
3 files changed
+99
-23
No files matched your search
+19
-8
@@ -251,10 +251,20 @@ it runs the same files locally in parallel shards
|
||||
selection (skill-e2e and the other paid globs in `test/helpers/paid-test-set.ts`)
|
||||
are named and left to the paid PR gate; a selection of only paid files reports
|
||||
`tests=0`, which never clears the obligation. The command is the same either way
|
||||
(`--remote` always needs a cap; the keyless path spends nothing). `--timeout
|
||||
2700` keeps pregate's default 600 s deadline from cutting off a slow VM run
|
||||
inside its 45-minute wall (the VM's teardown runs in its own session, so a
|
||||
pregate timeout never strands it):
|
||||
(`--remote` always needs a cap; the keyless path spends nothing).
|
||||
|
||||
pregate kills the gate at its `--timeout` (default 600 s) with a 5 s grace, so
|
||||
the script finishes before that deadline: it reads the deadline from
|
||||
`PREGATE_TIMEOUT_S`, else from the `gstack-pregate` command it runs under (its
|
||||
`--timeout`, else 600), reserves 120 s for teardown plus 15 s margin, and tells
|
||||
the VM to stop at whichever comes first, that point or the cap's wall (at most
|
||||
45 minutes, minus 3 for teardown). A deadline that leaves the VM under 300 s
|
||||
refuses before any VM (`PREGATE_REMOTE_TIMEOUT_TOO_SHORT`). At create time it
|
||||
prints the VM's name and the exact
|
||||
`bash scripts/ubicloud/ubi-runner.sh down <name>` line to destroy it if the
|
||||
run is killed before its teardown line. The whole script is parsed before it
|
||||
runs, so editing it during a run changes nothing. `--timeout 2700` gives a
|
||||
whole-tier run room inside the 45-minute wall:
|
||||
|
||||
```bash
|
||||
gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-commands
|
||||
@@ -262,9 +272,9 @@ gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-
|
||||
|
||||
Measured (October 2026): keyed, all 1,280 free files on standard-16 took 404 s
|
||||
from start to teardown (create 24 s, setup 72 s, tests 250 s) for $0.06.
|
||||
Keyless on 4 cores, 98 files took 103 s, and a 798-file pregate selection took
|
||||
3,550 s in an earlier wave, so keyless is practical for small selections only. A large selection on a small
|
||||
machine needs a key, or a `--timeout` longer than the run. Keyless also runs in
|
||||
Keyless on 4 cores, 98 files took 103 s, and GSTA-35's 798-file selection took
|
||||
3,550 s, so keyless is practical for small selections only. A large selection
|
||||
on a small machine needs a key, or a `--timeout` longer than the run. Keyless also runs in
|
||||
your environment, not CI's: a file that needs what setup-free-suite.sh installs
|
||||
(Playwright's Chromium, Xvfb) fails locally until you install it.
|
||||
|
||||
@@ -280,7 +290,8 @@ Every refusal line starts with a reason code:
|
||||
| `PREGATE_REMOTE_PAID_ONLY` | 0 | every selected file is a paid test file; `tests=0` cannot clear |
|
||||
| `PREGATE_REMOTE_CAP_TOO_LOW` | 5 | the cap cannot buy a 20-minute run at the size's price; nothing created |
|
||||
| `PREGATE_REMOTE_NO_PRICE` | 5 | `UBI_SIZE` has no list price in the script |
|
||||
| `PREGATE_REMOTE_INFRA` | 4 | create, setup, SSH or the wall timeout failed; the VM, its teardown line and `spent=` follow |
|
||||
| `PREGATE_REMOTE_TIMEOUT_TOO_SHORT` | 5 | pregate's deadline leaves the VM under 300 s after the 135 s teardown reserve; nothing created |
|
||||
| `PREGATE_REMOTE_INFRA` | 4 | create, setup, SSH or the stop deadline failed; the VM, its teardown line and `spent=` follow |
|
||||
| `PREGATE_REMOTE_INTERRUPTED` | 130 | TERM or INT; ubi-runner tears the VM down on its own |
|
||||
|
||||
<a id="guards"></a>
|
||||
|
||||
Reference in new issue
Block a user