diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f56e1cb2..14fd3b568 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,10 +7,10 @@ ### Added - **`ubicloud/SKILL.md.tmpl`, `bin/gstack-ubi`**: the skill: set/unset prerequisite checks (never `env`), `UBI_OWNER` on every machine, a watched launch (a waited-on background operation or `gstack-detach`), the quoted recovery line, one committed verify per repository, how to read exit 124 and a quota refusal, and the resume checklist. Classified `CLAUDE_ONLY` in `lib/doctor-components.ts`; routed from the root `SKILL.md`; listed in README, AGENTS.md, docs/skills.md and docs/PROJECT_STRUCTURE.md. - **TTL names and a reaper**: a generated VM name ends in `-t` (`UBI_TTL_HOURS`, default 4; 0 omits it). `reap` destroys every VM in the project whose TTL ended more than ten minutes ago, whichever owner created it, and every `up` runs it first (`UBI_REAP=0` skips it). Names without a TTL, including the ones `pregate-remote.sh` builds, are never reaped. -- **Quota preflight**: `up` adds the size's vCPUs to the project's running VMs and refuses before the create request when that exceeds the known ceiling (`UBI_QUOTA_VCPUS`, or the maximum the last API refusal printed, remembered in the state root); `UBI_QUOTA_WAIT=` polls for room instead of refusing. The per-owner `usage` table is printed either way. +- **Quota preflight**: `up` adds the size's vCPUs to the project's running VMs and refuses before the create request when that exceeds the known ceiling (`UBI_QUOTA_VCPUS`, or the maximum the last API refusal printed, remembered in the state root); `UBI_QUOTA_WAIT=` polls for room instead of refusing. Both refusals (the preflight's and the API's) print the per-owner `usage` table. - **Watchdog and heartbeat**: `run`, `verify` and `ssh NAME COMMAND` stream the remote output (both streams kept in the VM's state dir) through a watchdog: a heartbeat every `UBI_HEARTBEAT` seconds (300), and a command silent for `UBI_IDLE_TIMEOUT` seconds (1800) or older than `UBI_RUN_TIMEOUT` (0, unlimited) is killed in its own session with exit 124 and its last 40 lines on stderr. `run` also prints the VM's TTL and its `down` recovery command as soon as the VM is ready. -- **`verify`**: runs the repository's committed `.gstack/ubicloud.json` (setup, command, size, location, storage, image, env, pass, pull; flags and a trailing `-- COMMAND` override it) and refuses without the file or a command, so every lane runs one definition instead of its own wrapper. The file is repository content and gets no reach beyond the checkout: `setup` and each `pull` destination must be relative paths that resolve inside `--src` (absolute, `~` and `..` paths are refused before any API call), and a name under `pass` is forwarded only when the command line grants it (`--pass NAME`, or every listed name with `--trust-config`). `--pass` never forwards `UBICLOUD_*`, from the file or the command line. -- **Watched commands keep their stdin and exit codes**: `ssh NAME COMMAND` hands the caller's stdin to the remote command (`printf payload | gstack-ubi ssh NAME 'cat > file'` wrote an empty file once the watchdog backgrounded the client), a command that finished while the watchdog read the clock is never reported as a timeout, a hung setup script exits 124 like a hung command instead of 1, Ctrl-C on a watched command takes the ssh session down with it, and when the watchdog fires the runner also ends the command's session on the VM (`setsid` there, leader pid in `~/.ubirun--.sid`, `pkill -s` bounded to 20 s), since killing the local client alone leaves a silent remote command running until it next writes. +- **`verify`**: runs the repository's committed `.gstack/ubicloud.json` (setup, command, size, location, storage, image, env, pass, pull; flags and a trailing `-- COMMAND` override it) and refuses without the file or a command, so every lane runs one definition instead of its own wrapper. The file is repository content and gets no reach beyond the checkout: `setup` and each `pull` destination must be relative paths that resolve inside `--src` (absolute, `~` and `..` paths are refused before any API call, and under `verify` every pull destination is resolved again, symlinks followed, right before its pull, so a symlink an earlier pull planted cannot redirect a later one; a refused pull makes the run exit 1), and a name under `pass` is forwarded only when the command line grants it (`--pass NAME`, or every listed name with `--trust-config`; `verify` parses its options by arity, so an option value can never act as that flag). `--pass` never forwards `UBICLOUD_*`, from the file or the command line. +- **Watched commands keep their stdin and exit codes**: `ssh NAME COMMAND` hands the caller's stdin to the remote command (`printf payload | gstack-ubi ssh NAME 'cat > file'` wrote an empty file once the watchdog backgrounded the client), a command that finished while the watchdog read the clock, or in the instant before its kill, keeps its own exit status instead of 124, a hung setup script exits 124 like a hung command instead of 1, Ctrl-C on a watched command takes the ssh session down with it, and when the watchdog fires the runner also ends the command's session on the VM (`setsid` there, leader pid in `~/.ubirun--.sid`, kept until the cancel reads it so a background child the command left behind is still reachable, `pkill -s` bounded to 20 s), since killing the local client alone leaves a silent remote command running until it next writes. - **Quota ceiling and reaper outcomes**: the maximum a refusal cached is trusted for a day, then the API's own answer decides again; the refusal line says where the ceiling came from and names `UBI_QUOTA_VCPUS`, and its `vCPU quota exhausted` wording is the one `pregate-remote.sh` retries once on. `reap` returns 1 when the list fails (nothing destroyed) or an expired VM could not be destroyed, instead of 0 either way; a TTL suffix is read in base 10 (`UBI_TTL_HOURS` takes 0 to 999 with no leading zero). ### Tests diff --git a/docs/TESTING_INTERNALS.md b/docs/TESTING_INTERNALS.md index a6f9d5cfd..d632bac80 100644 --- a/docs/TESTING_INTERNALS.md +++ b/docs/TESTING_INTERNALS.md @@ -962,16 +962,18 @@ the CSO private-state checks), AppArmor blocks the unprivileged user namespaces Chromium's sandbox needs, and `clang` and `python3-venv` are absent (required by the Dia readiness and Python runner tests). -VMs are named `ubirun---`; `` is `UBI_OWNER` (set it -to the thread id on a Capy machine), else a stable per-machine id. Every exit +VMs are named `ubirun---`, plus a `-t` TTL suffix +(below); `` is `UBI_OWNER` (set it to the thread id on a Capy machine; +lowercase letters and digits, at most 12), else a stable per-machine id. Every exit path destroys the client's own VM (`trap` on EXIT, INT, TERM, HUP and QUIT), and `down` fails loudly (`FAILED to destroy / (HTTP )`) unless the VM is confirmed gone. `list --mine` shows this owner's VMs, `usage` the project's, and `records` the VM-id scratch file (`UBI_VM_RECORD`, default `~/.capy/work/ubi-runner/vms.tsv`, else `/vms.tsv`): every VM is recorded there before the create call and forgotten only after a confirmed -destroy, so a killed runner leaves a name to clean up. Nothing sweeps stale VMs -by default, because the project quota is shared with other agents' runners; set +destroy, so a killed runner leaves a name to clean up. No age-based sweep runs +by default, because the project quota is shared with other agents' runners (the +TTL reaper below destroys only a VM whose own `-t` TTL has ended); set `UBI_GC_HOURS` to a positive number (or run `gc HOURS`; `gc 0` is a no-op) to sweep only this owner's VMs older than that. For other commands, use the runner directly: `scripts/ubicloud/ubi-runner.sh run --setup