From 703b0049ebbef6be23e687a2ae9b18d24c5798fc Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Tue, 27 Oct 2026 10:39:03 -0700 Subject: [PATCH] v1.91.280.0 fix(flaky): ubi-runner quota wait counts down; pregate TERM teardown and design-lint linearity measure what they mean (GSTA-128) (#3517) * fix(flaky): ubi-runner quota wait counts its budget down; pregate TERM teardown and design-lint linearity checks measure what they mean (GSTA-128) Daily flaky pass, free-tier timing entries from the 2026-10-27 ledger. - test/ubicloud-runner.test.ts quota case (Free Tests 40474956830, 40480063084, 40488875742): quota_preflight pinned a wall-clock deadline before its first usage call, so a slow fake-CLI spawn on a loaded runner shaved the printed budget to "waiting up to 2s more". The budget now counts seconds spent waiting for room; usage reads between sleeps no longer eat it. Loaded local loop: 8/12 and 6/12 green before, 12/12 after. - test/ubicloud-pregate-remote.test.ts TERM teardown (40487321907 shard 13): the stub's `>>` creates the .down file before echo writes the line, and the test polled the inode, then read "". Both teardown waits now wait for the written line. - test/design-lint-fifth-cycle.test.ts tracked-caps linearity (40494745147 shard 15): the 8000-label run was timed once by wall clock on a shared runner (515 ms against the 500 ms floor). It now measures the lint's own CPU time, fastest of three runs, against the same unchanged bound. - The hung-setup watchdog assertion now carries the runner's stderr, so its next red names the cause (one unexplained exit 2 in 46 local runs). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(ubi-runner): prove the quota wait countdown; CHANGELOG entry for the free-tier timing repairs (GSTA-128) The existing WAIT=3 quota case passes under both the old wall-clock deadline and the new countdown because its fake usage read is instant. The new case slows the usage read to 1.2 s and asserts the printed 2 s budget (origin/main prints 1 s), then stubs sleep for a 70 s wait to see the 30 s steps and the release when the peer VM disappears. /ship Step 13 adds the [Unreleased] CHANGELOG entry; the train re-heads it at the slot. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * test(ubi-runner): assert the quota wait budget on its log line, not on elapsed wall time (GSTA-128) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- CHANGELOG.md | 15 ++++++++++++ VERSION | 2 +- agents-digest/gstack-AGENTS.md | 2 +- package.json | 2 +- scripts/ubicloud/ubi-runner.sh | 11 +++++---- test/design-lint-fifth-cycle.test.ts | 5 +++- test/ubicloud-pregate-remote.test.ts | 10 ++++---- test/ubicloud-runner.test.ts | 34 ++++++++++++++++++++++++++-- 8 files changed, 67 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2dd5ca389..1f4ee90ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,20 @@ # Changelog +## [1.91.280.0] - 2026-10-27 + +**Three free-tier tests measured something other than what they assert, and the Ubicloud quota wait shaved its own budget.** The daily flaky pass (GSTA-128) traced the free tier's open timing reds to measurement defects, not product ones: `quota_preflight` in `scripts/ubicloud/ubi-runner.sh` pinned its deadline to the wall clock before the first usage call, so a slow CLI spawn on a loaded runner printed "waiting up to 2s more" where the test expects the full 3 s; the pregate-remote teardown test read the `.down` file as soon as it existed, before the stub's `echo` had written the line; and the design-lint linearity check timed one wall-clock run of the 8000-label lint against a 500 ms floor on a shared runner. Under eight busy loops on four cores the quota case went from 8 of 12 green to 12 of 12. No ms bound, threshold, trial count or retry changed. + +### Fixed +- **`scripts/ubicloud/ubi-runner.sh`**: the quota wait budget is a countdown of the seconds spent sleeping for room. Usage reads between sleeps do not eat it, the first log line names the whole `UBI_QUOTA_WAIT`, `UBI_QUOTA_WAIT=0` still refuses on the first poll, and the wait still releases when usage drops. + +### Tests +- `test/ubicloud-pregate-remote.test.ts` waits for the written teardown line (`exit …` or `term …` ending in a newline), not for the file to exist. +- `test/design-lint-fifth-cycle.test.ts` measures the lint's own CPU time, fastest of three runs, against the unchanged `max(2.5 × t4000, 500 ms)` bound. A quadratic regression at 8000 labels would still measure about 690 ms CPU, above the floor, so the check keeps catching what it was written for. +- `test/ubicloud-runner.test.ts` attaches the runner's stderr to the hung-setup watchdog assertion, so the next red of that case names its cause. A new quota case proves the countdown. A usage read that takes 1.2 s keeps the printed 2 s budget (origin/main prints 1 s), a 70 s wait sleeps 30 s twice, and the wait releases when the peer VM disappears. + +### Gate integrity +- Four hunks listed, three read, none a relaxation. The teardown wait is stricter, the watchdog assertion is unchanged with a message added, and the linearity measure keeps its bound (measurement above). + ## [1.91.279.0] - 2026-10-27 **A licence line or a changelog entry no longer buys a full paid gate, the review loop stops re-reporting findings the owners have already ruled below the floor, and three follow-ups from the Blast radius ship land.** #3499: every PR that touched `NOTICE.md`, `LICENSE`, `licenses/` or `ETHOS.md` restored every gate case and judge because no paid case was mapped to them (#3493's docs-only PR planned 82 cases, about $30, for a notice and a changelog entry). #3504 and #3485: the same classes of finding (a schema lagging its writer, a helper nobody calls, a re-dispatch, a refusal one step late) reached the owners again on every ship after being judged below the floor once. GSTA-121's ship left three gaps. The Captain's merge-ask template and `docs/skills.md` lacked the `## Blast radius` item the PR body now carries, an 11-digit GitHub Actions run id after a run word was redacted from an approval record as a phone number, and the remote free gate failed with `PREGATE_REMOTE_INFRA` when a reviewer ran git in the checkout while the gate tarred `.git`. diff --git a/VERSION b/VERSION index 057e0ed5e..3d95d3946 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.91.279.0 +1.91.280.0 diff --git a/agents-digest/gstack-AGENTS.md b/agents-digest/gstack-AGENTS.md index cd16452a1..15ac5e105 100644 --- a/agents-digest/gstack-AGENTS.md +++ b/agents-digest/gstack-AGENTS.md @@ -1,4 +1,4 @@ -# gstack digest v1.91.279.0 — regenerate/re-copy after upgrading gstack +# gstack digest v1.91.280.0 — regenerate/re-copy after upgrading gstack Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed for agent hosts without a full skill install. The full skills add workflows, diff --git a/package.json b/package.json index 2b5266344..d0c1012d7 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.91.279", + "version": "1.91.280", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/scripts/ubicloud/ubi-runner.sh b/scripts/ubicloud/ubi-runner.sh index 19fb9435e..ce71e52bd 100755 --- a/scripts/ubicloud/ubi-runner.sh +++ b/scripts/ubicloud/ubi-runner.sh @@ -393,7 +393,7 @@ vcpus() { [[ "$1" =~ -([0-9]+)$ ]] && echo "${BASH_REMATCH[1]}"; } # maximum the last refusal printed; without either the API's own refusal rules. QUOTA_CACHE_MAX_AGE=86400 quota_preflight() { - local location=$1 name=$2 size=$3 need max used wait_until remaining source="UBI_QUOTA_VCPUS" cache="$STATE_ROOT/quota-max" + local location=$1 name=$2 size=$3 need max used remaining step source="UBI_QUOTA_VCPUS" cache="$STATE_ROOT/quota-max" need=$(vcpus "$size") || return 0 max=$QUOTA_VCPUS if [ -z "$max" ] && [ -f "$cache" ]; then @@ -406,19 +406,22 @@ quota_preflight() { fi fi [[ "$max" =~ ^[0-9]+$ ]] || return 0 - wait_until=$(( $(date +%s) + QUOTA_WAIT )) + # The budget counts seconds spent waiting for room, not the usage calls + # between them: a slow first usage read must not shave the printed budget. + remaining=$QUOTA_WAIT while :; do used=$(cmd_usage 2>/dev/null | awk '$1=="total" {print $3}') [[ "$used" =~ ^[0-9]+$ ]] || return 0 [ $(( used + need )) -gt "$max" ] || return 0 - remaining=$(( wait_until - $(date +%s) )) if [ "$remaining" -le 0 ]; then log "quota preflight refused $location/$name ($size): it needs $need vCPUs and the project's VMs already use $used of $max (ceiling from $source; other usage such as managed runners counts on top)" cmd_usage >&2 || true die "create refused before any request: vCPU quota exhausted (quota full); wait for running VMs to finish (UBI_QUOTA_WAIT= waits here), run $0 reap, use a smaller size, or set UBI_QUOTA_VCPUS to the project's real ceiling" fi log "quota: $used of $max vCPUs in use and $need needed; waiting up to ${remaining}s more for room" - sleep $(( remaining < 30 ? remaining : 30 )) + step=$(( remaining < 30 ? remaining : 30 )) + sleep "$step" + remaining=$(( remaining - step )) done } diff --git a/test/design-lint-fifth-cycle.test.ts b/test/design-lint-fifth-cycle.test.ts index cb3ef458f..730a5edd6 100644 --- a/test/design-lint-fifth-cycle.test.ts +++ b/test/design-lint-fifth-cycle.test.ts @@ -36,6 +36,9 @@ const quiet = { warn: () => {} }; const run = (html: string, files: Record = {}): Finding[] => { const dir = tree({ 'index.html': html, ...files }); return lintStatic([dir], { cwd: dir, ...quiet }).findings; }; const only = (f: Finding[], rule: string) => f.filter((x) => x.rule === rule); const timed = (fn: () => void): number => { const t = performance.now(); fn(); return performance.now() - t; }; +// A linearity check measures the lint's own CPU time, fastest of three runs: wall time on a loaded shard carries scheduling waits that are not the lint's cost. +const cpuTimed = (fn: () => void): number => { const c = process.cpuUsage(); fn(); const d = process.cpuUsage(c); return (d.user + d.system) / 1000; }; +const fastest = (fn: () => void, trials = 3): number => Math.min(...Array.from({ length: trials }, () => cpuTimed(fn))); describe('design-lint fifth cycle: dead markup and hostile inputs', () => { test('an emoji placeholder inside an HTML comment is dead', () => { @@ -63,7 +66,7 @@ describe('design-lint fifth cycle: dead markup and hostile inputs', () => { }); test('a page of many tracked-caps labels lints in linear time and a well-formed JSX file still yields its classes', () => { - const times = [4000, 8000].map((n) => { const dir = tree({ 'index.html': page('', '

Hello world

' + 'Label one\n'.repeat(n) + '
') }); return timed(() => lintStatic([dir], { cwd: dir, ...quiet })); }); + const times = [4000, 8000].map((n) => { const dir = tree({ 'index.html': page('', '

Hello world

' + 'Label one\n'.repeat(n) + '
') }); return fastest(() => lintStatic([dir], { cwd: dir, ...quiet })); }); expect(times[1]).toBeLessThan(Math.max(times[0] * 2.5, 500)); const doc = parseDoc('Page.tsx', 'export default function P(){ return (
x
); }'); expect(doc.classAt.size + doc.virtuals.length).toBeGreaterThan(0); diff --git a/test/ubicloud-pregate-remote.test.ts b/test/ubicloud-pregate-remote.test.ts index 1cb8820c0..3c51d1ef5 100644 --- a/test/ubicloud-pregate-remote.test.ts +++ b/test/ubicloud-pregate-remote.test.ts @@ -285,21 +285,23 @@ describe('pregate-remote.sh keyed path (stub ubi-runner)', () => { while (!cond()) { if (Date.now() > deadline) throw new Error('condition not met in time'); await Bun.sleep(50); } }; const vmLog = (f: Fixture) => { try { return fs.readFileSync(fs.readdirSync(path.join(f.repo, '.context/pregate-remote')).map(d => path.join(f.repo, '.context/pregate-remote', d, 'run.log'))[0]!, 'utf8'); } catch { return ''; } }; + // The stub's `>>` creates the file before echo writes the line; wait for the line itself. + const down = (f: Fixture) => { try { return fs.readFileSync(f.calls + '.down', 'utf8'); } catch { return ''; } }; const killed = fixture(); const a = run(killed); await until(() => vmLog(killed).includes('ubi-runner: running on'), 30_000); process.kill(-a.pid, 'SIGKILL'); await a.exited; - await until(() => fs.existsSync(killed.calls + '.down'), 30_000); - expect(fs.readFileSync(killed.calls + '.down', 'utf8')).toMatch(/^exit ubirun-stub-\d+-[0-9a-f]{8}\n$/); + await until(() => down(killed).endsWith('\n'), 30_000); + expect(down(killed)).toMatch(/^exit ubirun-stub-\d+-[0-9a-f]{8}\n$/); const termed = fixture(); const b = run(termed); await until(() => vmLog(termed).includes('ubi-runner: running on'), 30_000); process.kill(b.pid, 'SIGTERM'); expect(await b.exited).toBe(130); expect(await new Response(b.stdout).text()).toContain('interrupted: ubi-runner is tearing the VM down on its own'); - await until(() => fs.existsSync(termed.calls + '.down'), 10_000); - expect(fs.readFileSync(termed.calls + '.down', 'utf8')).toMatch(/^term ubirun-stub-\d+-[0-9a-f]{8}\n$/); + await until(() => down(termed).endsWith('\n'), 10_000); + expect(down(termed)).toMatch(/^term ubirun-stub-\d+-[0-9a-f]{8}\n$/); const edited = fixture(); const c = run(edited); await until(() => vmLog(edited).includes('ubi-runner: running on'), 30_000); diff --git a/test/ubicloud-runner.test.ts b/test/ubicloud-runner.test.ts index 8adacf278..4a9e2a04c 100644 --- a/test/ubicloud-runner.test.ts +++ b/test/ubicloud-runner.test.ts @@ -383,7 +383,7 @@ describe('ubi-runner run, verify and quota against a fake API and a fake ssh', ( ' *\'"create"\'*) if [ -n ' + JSON.stringify(opts.refuse ?? '') + ' ]; then printf \'%s\' ' + JSON.stringify(opts.refuse ?? '') + ' > "$out"; printf 400; else echo "$loc $name" >> "$listed"; : > "$out"; printf 200; fi ;;\n' + ' *destroy*) : > "$out"; grep -v " $name$" "$listed" > "$listed.tmp"; mv "$listed.tmp" "$listed"; printf 200 ;;\n' + ' *\'"list"\'*) cat "$listed" > "$out"; printf 200 ;;\n' + - ' *\'"show"\'*) if grep -q " $name$" "$listed"; then printf \'state: running\\nsize: standard-16\\nip4: 192.0.2.1\\n\' > "$out"; printf 200; else : > "$out"; printf 404; fi ;;\n' + + ' *\'"show"\'*) [ -z "${FAKE_SHOW_DELAY:-}" ] || sleep "$FAKE_SHOW_DELAY"; if grep -q " $name$" "$listed"; then printf \'state: running\\nsize: standard-16\\nip4: 192.0.2.1\\n\' > "$out"; printf 200; else : > "$out"; printf 404; fi ;;\n' + ' *) : > "$out"; printf 200 ;;\n' + 'esac\n'); writeFileSync(join(bin, 'ssh'), '#!/usr/bin/env bash\n' + @@ -432,7 +432,7 @@ describe('ubi-runner run, verify and quota against a fake API and a fake ssh', ( const setup = join(f.root, 'setup.sh'); writeFileSync(setup, 'echo hi\n'); const r = run(f, ['run', '--src', f.root, '--setup', setup, '--', 'make test'], { UBI_IDLE_TIMEOUT: '2' }); const err = r.stderr.toString(); - expect(r.exitCode).toBe(124); + expect(r.exitCode, err).toBe(124); expect(err).toMatch(/watchdog: setup printed nothing for \d+s \(UBI_IDLE_TIMEOUT=2\)/); expect(err).toContain('setup failed (exit 124)'); expect(f.ssh()).not.toContain('bash -lc'); @@ -607,6 +607,36 @@ describe('ubi-runner run, verify and quota against a fake API and a fake ssh', ( } finally { rmSync(f.root, { recursive: true, force: true }); } }, 90_000); + // Value: protects=the quota wait budget counts seconds slept, not usage reads: a slow usage read keeps the printed budget, a wait above 30 s counts down in 30 s steps, and the wait releases when room appears; fails_when=the budget goes back to a wall-clock deadline taken before the first usage read (the first log line shrinks to 1s), the step cap or the decrement is lost, or room found mid-wait still refuses; why_new=the WAIT=3 case above passes under both the old deadline and the new countdown because its fake usage read is instant and it never sees a second step; seam=none (the fixture's bin/ is already first on PATH, so sleep and curl are stubbed there); no_claim=real elapsed time of the long wait (the 70 s case stubs sleep) + test('quota wait: a slow usage read does not shave the printed budget; a long wait counts down in 30s steps and releases when room appears', () => { + const slow = fakeVm({ listed: 'eu-central-h1 ubirun-peer-9999999999-cccc\n' }); + try { + writeFileSync(join(slow.state, 'quota-max'), '31\n'); + const r = run(slow, ['up'], { UBI_QUOTA_WAIT: '2', FAKE_SHOW_DELAY: '1.2' }); + const err = r.stderr.toString(); + expect(r.exitCode, err).toBe(1); + expect(err).toMatch(/quota: 16 of 31 vCPUs in use and 16 needed; waiting up to 2s more for room/); + expect(err).toContain('create refused before any request: vCPU quota exhausted'); + expect(slow.calls()).not.toContain('"create"'); + } finally { rmSync(slow.root, { recursive: true, force: true }); } + + const f = fakeVm({ listed: 'eu-central-h1 ubirun-peer-9999999999-cccc\n' }); + try { + writeFileSync(join(f.state, 'quota-max'), '31\n'); + const sleepLog = join(f.root, 'sleep.log'); + writeFileSync(join(f.root, 'bin/sleep'), '#!/usr/bin/env bash\necho "$1" >> ' + JSON.stringify(sleepLog) + '\n[ "$(wc -l < ' + JSON.stringify(sleepLog) + ')" -ne 2 ] || : > ' + JSON.stringify(join(f.root, 'listed')) + '\n'); + chmodSync(join(f.root, 'bin/sleep'), 0o755); + const r = run(f, ['up'], { UBI_QUOTA_WAIT: '70' }); + const err = r.stderr.toString(); + expect(r.exitCode, err).toBe(0); + expect(err).toMatch(/waiting up to 70s more for room/); + expect(err).toMatch(/waiting up to 40s more for room/); + expect(err).not.toContain('quota preflight refused'); + expect(readFileSync(sleepLog, 'utf8').split('\n').slice(0, 2)).toEqual(['30', '30']); + expect(f.calls()).toContain('"create"'); + } finally { rmSync(f.root, { recursive: true, force: true }); } + }, 90_000); + test('reap reports a failed list and a failed destroy as failures; UBI_TTL_HOURS rejects leading zeros and more than three digits', () => { const f = fakeVm({ listed: 'eu-central-h1 ubirun-other-1000000000-bbbb-t1\n' }); try {