fix(pregate): remote gate finishes before pregate's deadline

pregate kills the remote gate at --timeout with a 5 s grace, which could cut a
VM teardown short. pregate-remote.sh now reads the enclosing deadline
(PREGATE_TIMEOUT_S, else the gstack-pregate --timeout it runs under, else 600),
reserves 120 s for teardown plus 15 s margin, and tells the VM to stop at the
earlier of that point and the cap's wall; a deadline that leaves under 300 s
refuses before any VM (PREGATE_REMOTE_TIMEOUT_TOO_SHORT). It names the VM up
front and prints the exact ubi-runner.sh down line, and the body is one braced
block so an edit to the script mid-run cannot change what the run executes
(keyed run 5 died after teardown with '$2: unbound variable' for that reason).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-17 20:42:49 +00:00
1 parent 3e14ca12f6
commit 8fa3e0d766
3 files changed
+99 -23

No files matched your search

+19 -8
View File
@@ -251,10 +251,20 @@ it runs the same files locally in parallel shards
selection (skill-e2e and the other paid globs in `test/helpers/paid-test-set.ts`)
are named and left to the paid PR gate; a selection of only paid files reports
`tests=0`, which never clears the obligation. The command is the same either way
(`--remote` always needs a cap; the keyless path spends nothing). `--timeout
2700` keeps pregate's default 600 s deadline from cutting off a slow VM run
inside its 45-minute wall (the VM's teardown runs in its own session, so a
pregate timeout never strands it):
(`--remote` always needs a cap; the keyless path spends nothing).
pregate kills the gate at its `--timeout` (default 600 s) with a 5 s grace, so
the script finishes before that deadline: it reads the deadline from
`PREGATE_TIMEOUT_S`, else from the `gstack-pregate` command it runs under (its
`--timeout`, else 600), reserves 120 s for teardown plus 15 s margin, and tells
the VM to stop at whichever comes first, that point or the cap's wall (at most
45 minutes, minus 3 for teardown). A deadline that leaves the VM under 300 s
refuses before any VM (`PREGATE_REMOTE_TIMEOUT_TOO_SHORT`). At create time it
prints the VM's name and the exact
`bash scripts/ubicloud/ubi-runner.sh down <name>` line to destroy it if the
run is killed before its teardown line. The whole script is parsed before it
runs, so editing it during a run changes nothing. `--timeout 2700` gives a
whole-tier run room inside the 45-minute wall:
```bash
gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-commands
@@ -262,9 +272,9 @@ gstack-pregate --stage tests --remote --spend-cap 2 --timeout 2700 --allow-repo-
Measured (October 2026): keyed, all 1,280 free files on standard-16 took 404 s
from start to teardown (create 24 s, setup 72 s, tests 250 s) for $0.06.
Keyless on 4 cores, 98 files took 103 s, and a 798-file pregate selection took
3,550 s in an earlier wave, so keyless is practical for small selections only. A large selection on a small
machine needs a key, or a `--timeout` longer than the run. Keyless also runs in
Keyless on 4 cores, 98 files took 103 s, and GSTA-35's 798-file selection took
3,550 s, so keyless is practical for small selections only. A large selection
on a small machine needs a key, or a `--timeout` longer than the run. Keyless also runs in
your environment, not CI's: a file that needs what setup-free-suite.sh installs
(Playwright's Chromium, Xvfb) fails locally until you install it.
@@ -280,7 +290,8 @@ Every refusal line starts with a reason code:
| `PREGATE_REMOTE_PAID_ONLY` | 0 | every selected file is a paid test file; `tests=0` cannot clear |
| `PREGATE_REMOTE_CAP_TOO_LOW` | 5 | the cap cannot buy a 20-minute run at the size's price; nothing created |
| `PREGATE_REMOTE_NO_PRICE` | 5 | `UBI_SIZE` has no list price in the script |
| `PREGATE_REMOTE_INFRA` | 4 | create, setup, SSH or the wall timeout failed; the VM, its teardown line and `spent=` follow |
| `PREGATE_REMOTE_TIMEOUT_TOO_SHORT` | 5 | pregate's deadline leaves the VM under 300 s after the 135 s teardown reserve; nothing created |
| `PREGATE_REMOTE_INFRA` | 4 | create, setup, SSH or the stop deadline failed; the VM, its teardown line and `spent=` follow |
| `PREGATE_REMOTE_INTERRUPTED` | 130 | TERM or INT; ubi-runner tears the VM down on its own |
<a id="guards"></a>