/ship's test step says when it hits its time limit, and long test file lists no longer overflow the remote command (GST-2958) (#4429)

* Say when /ship's test step hit its time limit, and keep long file lists off the remote test command line

gstack-ship-preflight stopped gstack-pregate after a fixed hour and printed
'PREGATE: unreadable (exit 143)'. The whole step's budget is now
--step-timeout <s> (default 3600), and a stop at it prints
'PREGATE: time limit hit' with the way out: push and let CI's full run
carry the tests, or a larger budget.

The test_backend command pregate prints for a small machine put every
selected file into one argument, which Linux refuses past 128 KiB. The
files now reach the VM in a setup script named by its content.

Also adds an arg-limit troubleshooting entry and says the Windows job has
eight shards.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Read the small-machine file list from the setup script in the hidden-folder path test

The remote command now names a setup script instead of the files, so the
test checks that the script lists the ./-prefixed path.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Keep a test path with a newline out of the remote file list

A selected test path containing a newline, or equal to the list's end marker, could end the setup script's file list early and run the rest as shell on the test VM. Such paths now keep the quoted inline command.

Review-Finding: p1/outside-voice#1 reader=outside-voice
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Mark the preflight half of the Step 5 time limit to-do as done

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-11-17 13:13:19 +00:00
1 parent e8c8625bd6
commit aeb74eb00b
11 files changed
+146 -20

No files matched your search

+1 -1
View File
@@ -46,7 +46,7 @@
### P2/P3: pregate head-policy follow-ups (filed 2026-10-31, from the GST-419/GST-420 /ship review)
- **`finish` fallback drops head-policy lanes (P2)** — `lib/ship-release.ts` reruns pregate on the stamped tree under the base policy, so after a version bump a lane the branch added under `--policy-from head` is gone and `finish` reports `PREGATE: fail (rerun on the stamped tree)`. GSTA-194 owns that file and has the note. **Effort:** S. **Priority:** P2.
- **The 3600 s preflight and `finish` wrapper deadline (P3)** — it caps the whole pregate run, so several runners that each fit `timeout_s` can still be cut off together. Derive the wrapper deadline from `timeout_s` and the lane count, or document the cap. **Effort:** S. **Priority:** P3.
- **The 3600 s `finish` wrapper deadline (P3)** — it caps the whole pregate rerun, so several runners that each fit `timeout_s` can still be cut off together. Derive the wrapper deadline from `timeout_s` and the lane count, or document the cap. Preflight's half is done: `gstack-ship-preflight --step-timeout <s>` sets that budget and a stop prints `PREGATE: time limit hit` (#4429). **Effort:** S. **Priority:** P3.
- **A bootstrap head policy can still certify itself without `--remote` (P3)** — with no trusted policy, the head file may set `remote_gate: null`, or produce an obligation that `gstack-pregate clear --receipt` accepts from the branch's own gate. Decide what a bootstrap `pregate.json` may certify, then enforce it in `verify` from the recorded policy label instead of per entry point. The `--remote` refusal also prints the generic `PREGATE_HEAD_POLICY_LOOSENS` fix line, which does not apply when there is no trusted policy. **Effort:** S. **Priority:** P3.
### P3: `/codex --effort` follow-ups (filed 2026-10-11, from the #3146 autoplan review)
@@ -0,0 +1,2 @@
- **/ship's test step says when it ran out of time, and you can set its budget.** `gstack-ship-preflight` used to stop gstack-pregate after a fixed hour and print `PREGATE: unreadable (exit 143)`, which hid the cause. The whole step's budget is now `--step-timeout <s>` (default 3600 s), and a stop at that limit prints `PREGATE: time limit hit` with the way out: push and let CI's full run carry the tests, or give the step a larger budget.
- **The remote test command no longer puts every file on one command line.** On a small machine with a remote `test_backend`, the command pregate prints for gstack's Ubicloud runner used to carry every selected test file in one argument, which Linux refuses past 128 KiB (`E2BIG`). The files now go to the VM in a setup script, so the command stays short however many files are selected. docs/troubleshooting.md has a new `arg-limit` entry for this kind of failure.
+8 -4
View File
@@ -103,7 +103,7 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`).
| `sealed` | sealed or eval output that must never reach a public destination; read by the pre-push hook and `gstack-pregate sealed` (see [sealed output](#sealed)) |
| `price_file` | repo-relative path of the paid gate's call-plan data file (`GSTACK_EVAL_CALLS v1`); `gstack-eval-plan price-gate` prices from it and never executes anything ([docs/eval-calls.md](eval-calls.md)) |
| `stall_timeout_s` | seconds a test command may print nothing while using under a tenth of one CPU before its process group is killed and the check is `incomplete` (`PREGATE_STALLED`); default 300, `0` disables; `--stall-timeout` overrides; the `--remote` gate is exempt (see [supervision](#supervision)) |
| `timeout_s` | seconds each runner process may take (the unit-file group, each serial file, each guard without its own `timeout_s`, and the remote gate's deadline) when `--timeout` is not passed; 1 to 3600, default 600. `gstack-ship-preflight` forwards its own `--timeout`; without it, and in the `finish` fallback, this is how a repo with long lanes sets theirs |
| `timeout_s` | seconds each runner process may take (the unit-file group, each serial file, each guard without its own `timeout_s`, and the remote gate's deadline) when `--timeout` is not passed; 1 to 3600, default 600. `gstack-ship-preflight` forwards its own `--timeout`; without it, and in the `finish` fallback, this is how a repo with long lanes sets theirs. The whole pregate under `gstack-ship-preflight` has a separate budget, `--step-timeout <s>` (1 to 86400, default 3600): a pregate stopped there prints `PREGATE: time limit hit` and a STOP naming the way out, push and let CI's full run at the tree carry the tests ([CI carry](#ci-carry)), or a larger `--step-timeout` |
| `remote_gate` | the repo's remote test gate command; under 8 cores the local test tier becomes a `requires-remote` obligation naming it, and `--remote` runs it (see [remote gate](#remote-gate)) |
| `run_profiles` | a map from each profile name to `{ workflow, events, jobs, artifact, max_age_hours?, os?, skippable? }`: the GitHub Actions runs `clear --run` verifies as evidence on the identical tree; a profile named after a lane (`free-tests.yml/macos-named-regressions`) clears that lane (see [run evidence](#run-evidence)). `--from-run` and the reuse job verify runs against them too ([from-run](#from-run), [CI reuse](#ci-reuse)) |
| `remote_gate_runs` | `{ profiles, inventory, skipped? }`: the run profiles that together clear `pregate.json/remote_gate`, the command that prints the remote gate's test files, one per line, which their runs must cover, and an optional command that prints which selected files the gate itself never runs (see [run evidence](#run-evidence)) |
@@ -563,7 +563,11 @@ test_backend` is not `local`, the lanes check runs nothing locally: it records
the selected local files as a `requires-remote` obligation with the command to
run, `pregate.json/remote_gate` (the declared command) or
`test_backend/<backend>` (gstack's `scripts/ubicloud/ubi-runner.sh run --src
<repo> -- '<runner> <files>'`). Without either, the tier runs locally under
<repo> --setup <file> -- '<runner>'`, where the setup script, kept under
`<state root>/projects/<slug>/pregate/remote-files/` and named by its content,
writes the selected files to `~/.pregate-files.txt` on the VM and the runner
reads them from there; the list never goes on the command line, which reaches
the VM as one argument, see [arg-limit](troubleshooting.md#arg-limit)). Without either, the tier runs locally under
the supervision above. Close the obligation one of four ways:
- by default, when the base declares `remote_gate_runs`, let CI clear it: the
@@ -1000,7 +1004,7 @@ conclusion. On main the free tier's Windows smoke lane is
Two narrower Windows lanes are owed only when a branch selects their own test
files: `free-tests.yml/cso-windows-launcher` (cleared by a `free-tests.yml`
run) and `windows-setup-e2e.yml/windows-setup` (cleared by a
`windows-setup-e2e.yml` run). The six-shard
`windows-setup-e2e.yml` run). The eight-shard
`windows-free-tests.yml/windows-free-shard` job is `manual`, so it is never an
obligation. A branch cut before
`windows-smoke.yml` landed still owes `windows-free-tests.yml/windows-free-shard`
@@ -1862,7 +1866,7 @@ failure the branch caused carries `cause: "diff"` (`PREGATE_LANE_FAILED`,
| Kind | Evidence |
|---|---|
| `missing-binary` | exit 127, `command not found`, a spawn `ENOENT`, a runner tool `not on PATH` (dart, flutter, cargo, a package manager, `python3`, or `pytest` when python3 cannot import it), Playwright's browsers not installed |
| `arg-limit` | `E2BIG`, `Argument list too long`: a command line or environment over the system's limit. gstack splits a lane's file list to stay under it, so this is a gstack bug to report |
| `arg-limit` | `E2BIG`, `Argument list too long`: a command line or environment over the system's limit. gstack splits a lane's file list to stay under it, so this is a gstack bug to report ([troubleshooting](troubleshooting.md#arg-limit)) |
| `network` | `ENOTFOUND`, `ECONNREFUSED`, `ECONNRESET`, `ETIMEDOUT`, `EAI_AGAIN`, `Could not resolve host`, `fetch failed`, a TLS handshake or certificate failure |
| `quota` | `quota`, `429 Too Many Requests`, `rate limit`, `insufficient_quota` (a vCPU quota on the remote backend) |
| `disk` | `ENOSPC`, `No space left on device`, `EDQUOT` |
+19
View File
@@ -4360,6 +4360,25 @@ or on the comment line directly above it. The marker waives only that line, and
the check lists it as `waived:`. Don't use `warn: ["literals"]` for this, since
that turns the check off for every constant.
<a id="arg-limit"></a>
### `PREGATE_ENVIRONMENT` with `environment: arg-limit — E2BIG` (or `Argument list too long`)
**Meaning.** A command gstack started had a command line or environment over
the system's limit, so it never ran and no test was counted. Linux caps one
argument at 128 KiB and a whole command line plus environment at about 2 MiB,
macOS caps the total at 1 MiB, and Windows caps a command line at 32,767
characters. The usual trigger is a long file list: thousands of selected test
files put into one argument.
**Fix.** gstack keeps file lists out of single arguments: a local lane splits
its files into runner processes of about 96 KB each (24 KB on Windows), and
the `test_backend` command for the remote gate passes them in a setup script
(docs/pregate.md#remote-gate). So this cause is a gstack bug: report it with
the `environment:` line and the lane or command it names. Until it is fixed,
run the lane remotely (push and let CI's full run carry it, or
`gstack-pregate --remote`), or select fewer files. When the environment, not
the file list, is what's large, unset the big variables before the run.
<a id="pregate-no-lane"></a>
### `PREGATE_NO_LANE`
+30 -3
View File
@@ -30,7 +30,7 @@
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { baseInputs, sha256 } from '../context';
import { baseInputs, defaultScratchRoot, sha256 } from '../context';
import { PREGATE_CONFIG_PATH } from '../config';
import { classifyCommandEnvironment, classifyEnvironment, environmentLine, environmentResult, launchCause } from '../environment';
import { checkScratch } from '../run-scratch';
@@ -498,12 +498,39 @@ const GSTACK_ROOT = path.resolve(import.meta.dir, '..', '..', '..');
*/
function remoteObligation(ctx: PregateContext, sel: Selection): RemoteObligation {
const declared = ctx.config.remote_gate;
const runner = detectRunner(ctx, sel.gateTests);
const command = declared ?? `${GSTACK_ROOT}/scripts/ubicloud/ubi-runner.sh run --src ${quote(ctx.repoRoot)} -- ${quote(runner ? runner.replace('{files}', testFileArgs(runner, sel.gateTests).map(quote).join(' ')) : '<test runner> <selected files>')}`;
const command = declared ?? backendCommand(ctx, sel.gateTests);
const [workflow, job] = declared ? REMOTE_GATE_LANE.split('/') as [string, string] : ['test_backend', ctx.testBackend];
return { lane: `${workflow}/${job}`, platform: 'remote', workflow, job, tests: sel.gateTests, receipt: null, command };
}
/**
* The test backend's command: gstack's Ubicloud runner, with the selected files in a setup script that writes them one per line
* on the VM. The command reaches the VM as one argument and Linux caps one argument at 128 KiB (#4424), so the list never goes in
* it. The script lives under the state root, named by its content, so the same selection prints the same command every run and a
* same-tree clear still matches it; an unwritable state root keeps it in this run's scratch. A path the one-per-line list can't
* hold (a newline, or the list's end marker) keeps the quoted inline form, so no path can end the list early and run as shell.
*/
function backendCommand(ctx: PregateContext, files: readonly string[]): string {
const ubi = `${GSTACK_ROOT}/scripts/ubicloud/ubi-runner.sh run --src ${quote(ctx.repoRoot)}`;
const runner = detectRunner(ctx, files);
if (!runner) return `${ubi} -- ${quote('<test runner> <selected files>')}`;
const args = testFileArgs(runner, files);
if (args.some(f => f.includes('\n') || f === 'PREGATE_FILES')) return `${ubi} -- ${quote(runner.replace('{files}', () => args.map(quote).join(' ')))}`;
const setup = `cat > ~/.pregate-files.txt <<'PREGATE_FILES'\n${args.map(f => `${f}\n`).join('')}PREGATE_FILES\n`;
const name = `${sha256(setup).slice(0, 16)}.sh`;
let file: string;
try {
const dir = path.join(path.dirname(defaultScratchRoot(ctx.repoRoot, ctx.env)), 'remote-files');
fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
file = path.join(dir, name);
fs.writeFileSync(file, setup);
} catch {
file = path.join(checkScratch(ctx.scratch, 'remote'), name);
fs.writeFileSync(file, setup);
}
return `${ubi} --setup ${quote(file)} -- ${quote(`mapfile -t PREGATE_FILES < ~/.pregate-files.txt && ${runner.replace('{files}', () => '"${PREGATE_FILES[@]}"')}`)}`;
}
export const REMOTE_GATE_LANE = 'pregate.json/remote_gate';
/** How many leading lines of a remote gate's output are read for an environment cause: its preamble, before any test output could stream. */
export const REMOTE_PREAMBLE_LINES = 40;
+3 -2
View File
@@ -19,9 +19,10 @@ import { ghSpawnSync } from './gh-poll';
import { extractVersion } from './version-source';
import { BIN_DIR, runBashSync, runScriptSync, scriptInvocation } from './script-invocation';
export interface Run { status: number | null; stdout: string; stderr: string }
/** `timedOut`: the call's own time limit stopped the process (spawnSync's ETIMEDOUT), whatever exit status it then gave. */
export interface Run { status: number | null; stdout: string; stderr: string; timedOut?: boolean }
const done = (r: { status: number | null; stdout?: string | Buffer | null; stderr?: string | Buffer | null; error?: Error }): Run =>
({ status: r.status, stdout: String(r.stdout ?? ''), stderr: String(r.stderr ?? '') + (r.error ? r.error.message : '') });
({ status: r.status, stdout: String(r.stdout ?? ''), stderr: String(r.stderr ?? '') + (r.error ? r.error.message : ''), ...((r.error as NodeJS.ErrnoException | undefined)?.code === 'ETIMEDOUT' ? { timedOut: true } : {}) });
export function git(cwd: string, args: string[], timeout = 120_000): Run {
return done(spawnSync('git', args, { cwd, encoding: 'utf8', timeout, maxBuffer: 64 * 1024 * 1024, env: { ...process.env, GIT_TERMINAL_PROMPT: '0' } }));
+13 -3
View File
@@ -30,6 +30,8 @@ export const PREFLIGHT_HELP = `gstack-ship-preflight — /ship Steps 1-3 and Ste
(a branch that adds the lane for a file it touches); anything else is PREGATE_HEAD_POLICY_LOOSENS
--timeout <s> Step 5's pregate --timeout for each runner process, 1 to 3600; unset, the base .gstack/pregate.json
timeout_s applies, else 600 s, and a longer lane is PREGATE_INCOMPLETE
--step-timeout <s> the whole Step 5 pregate's budget (every check and lane together), 1 to 86400, default 3600; at the
limit it is stopped and prints PREGATE: time limit hit with the way out (push and let CI's full run carry it)
gstack-ship-preflight --tests-only --base <base> --policy-from head --simulate [--allow-repo-commands] --lane <label>=<command>...
a dry run that proves a new ship_lanes_from mapping: which lanes the branch's own map would carry from a verified
CI run at this tree. Every line starts SIMULATION:; no lane runs, no evidence or ledger row is written and no
@@ -195,8 +197,13 @@ function merge(cwd: string, base: string, r: Report): boolean {
return true;
}
async function tests(cwd: string, base: string, list: { label: string; command: string }[], policyFrom: string, timeout: string | undefined, run: { concurrency: number; line: string }, r: Report): Promise<void> {
const pg = bin(cwd, 'gstack-pregate', ['--allow-repo-commands', '--base', base, '--policy-from', policyFrom, ...(timeout ? ['--timeout', timeout] : [])], 3_600_000);
async function tests(cwd: string, base: string, list: { label: string; command: string }[], policyFrom: string, timeout: string | undefined, step: { seconds: number; source: string }, run: { concurrency: number; line: string }, r: Report): Promise<void> {
const pg = bin(cwd, 'gstack-pregate', ['--allow-repo-commands', '--base', base, '--policy-from', policyFrom, ...(timeout ? ['--timeout', timeout] : [])], step.seconds * 1000);
if (pg.timedOut) {
r.add(`PREGATE: time limit hit (Step 5's whole pregate ran past its ${step.seconds} s budget, ${step.source}, and was stopped with exit ${pg.status ?? 'none'}; no verdict)`);
r.stop(`the tests are longer than this step's ${step.seconds} s budget on this machine: push, let CI's full run at this tree carry them (gstack-ci-carry wait --base ${base}, then rerun --tests-only; gbrain docs/ci-event-parity.md#ship-lanes), or rerun --tests-only with --step-timeout <s> above ${step.seconds}`);
return;
}
const refusal = pg.status === 3 ? /\((PREGATE_[A-Z_]+|POLICY_[A-Z_]+)[;)]/.exec(pg.stderr)?.[1] : undefined;
const verdict = refusal ? `refused (${refusal})` : /^PREGATE: (.+?) tree=/m.exec(pg.stdout + pg.stderr)?.[1] ?? `unreadable (exit ${pg.status})`;
r.add(`PREGATE: ${verdict}`);
@@ -263,6 +270,9 @@ export async function preflight(argv: readonly string[], cwd: string): Promise<{
if (!['base', 'head'].includes(policyFrom)) return { text: 'gstack-ship-preflight: --policy-from needs base|head\n', exit: 2 };
const timeout = flag('--timeout');
if (has('--timeout') && !(timeout && /^[1-9]\d*$/.test(timeout) && Number(timeout) <= 3600)) return { text: 'gstack-ship-preflight: --timeout needs whole seconds from 1 to 3600\n', exit: 2 };
const stepTimeout = flag('--step-timeout');
if (has('--step-timeout') && !(stepTimeout && /^[1-9]\d*$/.test(stepTimeout) && Number(stepTimeout) <= 86_400)) return { text: 'gstack-ship-preflight: --step-timeout needs whole seconds from 1 to 86400\n', exit: 2 };
const step = stepTimeout ? { seconds: Number(stepTimeout), source: '--step-timeout' } : { seconds: 3600, source: 'the default; set --step-timeout' };
if (!base || typeof list === 'string') return { text: `gstack-ship-preflight: ${typeof list === 'string' ? list : '--base <base> is required'}\n`, exit: 2 };
const run = flag('--run');
if (has('--run') && !(run && RUN_ID.test(run))) return { text: 'gstack-ship-preflight: --run needs a run id: letters, digits, dot, underscore or dash, at most 128\n', exit: 2 };
@@ -273,7 +283,7 @@ export async function preflight(argv: readonly string[], cwd: string): Promise<{
if (typeof concurrency === 'string') return { text: `gstack-ship-preflight: ${concurrency}\n`, exit: 2 };
const r = new Report();
if (has('--tests-only') || integrate(cwd, base, run, r)) {
if (list.length) await tests(cwd, base, list, policyFrom, timeout, { concurrency, line: `LANES: ${concurrency === 1 ? 'serial' : 'parallel'} (${mode ? '--lanes' : `${os.availableParallelism()} cores`})` }, r);
if (list.length) await tests(cwd, base, list, policyFrom, timeout, step, { concurrency, line: `LANES: ${concurrency === 1 ? 'serial' : 'parallel'} (${mode ? '--lanes' : `${os.availableParallelism()} cores`})` }, r);
else if (r.exit === 0) r.next('Step 4: confirm the test framework, then rerun with --tests-only --lane <label>=<command> per suite (Step 5)');
}
return { text: r.render(), exit: r.exit };
+3 -2
View File
@@ -73,8 +73,9 @@ describe.skipIf(process.platform === 'win32')('lanes check on a test under a hid
const result = await lanes(DETECTED, { [TEST]: GREEN }, [], { cores: 4, testBackend: 'ubicloud' });
expect(result.status).toBe('requires-remote');
const run = result.lines?.find(l => l.startsWith('run: ')) ?? '';
expect(run).toContain('./.github/scripts/select.test.mjs');
expect(run).not.toMatch(/'\.github\/scripts/);
const listed = fs.readFileSync(/--setup '([^']+)'/.exec(run)![1]!, 'utf8').split('\n');
expect(listed).toContain('./.github/scripts/select.test.mjs');
expect(listed).not.toContain('.github/scripts/select.test.mjs');
}, 120_000);
// Value: protects=the lane still runs and judges a hidden-folder test; fails_when=the fix passes the lane without running the test
+36 -4
View File
@@ -6,6 +6,9 @@
* one test per file argument, so the lane's count proves every file reached a runner.
*/
import { afterEach, describe, expect, test } from 'bun:test';
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { argBudget, argChunks } from '../lib/pregate/checks/lanes';
import { buildContext } from '../lib/pregate/context';
import { classifyEnvironment } from '../lib/pregate/environment';
@@ -24,18 +27,18 @@ const RUNNER = [
'process.exit(failed ? 1 : 0);',
].join('\n');
async function lanesOver(tests: string[]) {
async function pregateOver(tests: string[], machine: { cores: number; testBackend: string }) {
const repo = makeFixtureRepo({
pregate: { ...BASE_PREGATE, runner: 'node scripts/count.js {files}', dependencies: { ...BASE_PREGATE.dependencies, 'src/big/**': tests } },
baseFiles: Object.fromEntries([...tests.map(t => [t, "require('node:test');\n"]), ['src/big/.keep', ''], ['scripts/count.js', `${RUNNER}\n`]]),
});
repos.push(repo);
commit(repo, { 'src/big/f.js': 'exports.n = 1;\n' }, 'touch the big tree');
const loaded = buildContext({ repoRoot: repo.work, base: 'main', policyFrom: 'base', allowRepoCommands: true, env: { ...process.env, ...repo.env }, progress: () => {}, cores: 64, testBackend: 'local' });
const loaded = buildContext({ repoRoot: repo.work, base: 'main', policyFrom: 'base', allowRepoCommands: true, env: { ...process.env, ...repo.env }, progress: () => {}, ...machine });
if (!('ctx' in loaded)) throw new Error('fixture context refused');
const file = await runPregate(loaded.ctx, { stage: 'tests', only: ['lanes'] });
return file.checks.find(c => c.id === 'lanes')!;
return runPregate(loaded.ctx, { stage: 'tests', only: ['lanes'] });
}
const lanesOver = async (tests: string[]) => (await pregateOver(tests, { cores: 64, testBackend: 'local' })).checks.find(c => c.id === 'lanes')!;
describe.skipIf(process.platform === 'win32')('a local lane whose file list is over the per-argument limit', () => {
// Value: protects=a unit group whose quoted file list is over Linux's 128 KiB per-argument cap still launches and every file
@@ -54,6 +57,35 @@ describe.skipIf(process.platform === 'win32')('a local lane whose file list is o
}, 120_000);
});
describe.skipIf(process.platform === 'win32')('the test backend command for a file list over the per-argument limit', () => {
// Value: protects=on a small machine with a remote test_backend, the printed command passes the selected files in a setup
// script, so it stays short and, run the way ubi-runner runs it on the VM (setup, then bash -c <command>), every file reaches
// the runner; fails_when=the files go back into the command, which reaches the VM as one argument (E2BIG on Linux);
// why_new=the remote-gate half of #4424; seam=runPregate --only lanes, the setup file, bash; no_claim=ubi-runner's transport
test('the files travel in the setup script, the command stays short, and running both reaches every file', async () => {
const tests = Array.from({ length: FILES }, (_, i) => name(i));
const quoted = tests.map(t => `'${t}'`).join(' ');
expect(quoted.length).toBeGreaterThan(128 * 1024);
if (process.platform === 'linux') expect((spawnSync('bash', ['-c', `true ${quoted}`], { timeout: 30_000 }).error as NodeJS.ErrnoException | undefined)?.code).toBe('E2BIG');
const file = await pregateOver(tests, { cores: 4, testBackend: 'ubicloud' });
expect(file.checks.find(c => c.id === 'lanes')!.status).toBe('requires-remote');
const command = file.requires_remote!.find(o => o.lane === 'test_backend/ubicloud')!.command!;
expect(command.length).toBeLessThan(2048);
const argv = spawnSync('bash', ['-c', 'eval "set -- $1"; printf "%s\\0" "$@"', '_', command], { encoding: 'utf8', timeout: 30_000 }).stdout.split('\0').slice(0, -1);
expect(argv.slice(1, 3)).toEqual(['run', '--src']);
expect(argv[4]).toBe('--setup');
expect(argv[6]).toBe('--');
const setup = fs.readFileSync(argv[5]!, 'utf8');
expect(setup.split('\n').filter(l => l.startsWith('test/big/'))).toEqual(tests);
const vm = repos.at(-1)!;
const env = { ...process.env, HOME: fs.mkdtempSync(path.join(vm.root, 'vm-home-')) };
expect(spawnSync('bash', [argv[5]!], { cwd: vm.work, env, timeout: 30_000 }).status).toBe(0);
const ran = spawnSync('bash', ['-c', argv[7]!], { cwd: vm.work, env, encoding: 'utf8', timeout: 60_000 });
expect(ran.stdout).toContain(`ℹ tests ${FILES}\n`);
expect(ran.status).toBe(0);
}, 120_000);
});
describe('argChunks and argBudget', () => {
// Value: protects=the split keeps order, drops nothing and keeps each chunk's quoted text under the budget, with an oversized
// argument alone in its chunk, and the budget stays under Linux's per-argument cap and shrinks with the environment;
+11 -1
View File
@@ -68,11 +68,21 @@ describe('small machines record the test tier for the remote gate (C4)', () => {
expect(viaBackend.lanes.status).toBe('requires-remote');
const obligation = viaBackend.file.requires_remote![0]!;
expect(obligation.lane).toBe('test_backend/ubicloud');
expect(obligation.command).toMatch(/scripts\/ubicloud\/ubi-runner\.sh run --src '.*\/work' -- 'node --test '\\''test\/util\.test\.js'\\'''$/);
expect(obligation.command).toMatch(/scripts\/ubicloud\/ubi-runner\.sh run --src '.*\/work' --setup '(.*\/remote-files\/[0-9a-f]{16}\.sh)' -- 'mapfile -t PREGATE_FILES < ~\/\.pregate-files\.txt && node --test "\$\{PREGATE_FILES\[@\]\}"'$/);
expect(fs.readFileSync(/--setup '([^']+)'/.exec(obligation.command!)![1]!, 'utf8')).toBe("cat > ~/.pregate-files.txt <<'PREGATE_FILES'\ntest/util.test.js\nPREGATE_FILES\n");
expect((await lanesOn(backend, { cores: 4, testBackend: 'local' })).lanes.status).toBe('pass');
expect((await lanesOn(gated(), { cores: 16, testBackend: 'local' })).lanes.status).toBe('pass');
});
test.skipIf(process.platform === 'win32')('a selected test path the file list cannot hold (a newline) keeps the quoted inline command, so it cannot end the list and run as shell', async () => {
const backend = gated(BASE_PREGATE);
const odd = 'test/odd\nPREGATE_FILES\nname.test.js';
commit(backend, { [odd]: "require('node:test')('ok', () => {});\n" });
const command = (await lanesOn(backend, { cores: 4, testBackend: 'ubicloud' })).file.requires_remote![0]!.command!;
expect(command).not.toContain('--setup');
expect(command).toContain(odd);
});
test('`clear` on the remote_gate obligation needs the gate\'s PREGATE_REMOTE receipt for the tested tree; a run URL or a bad receipt leaves it uncleared', async () => {
const repo = gated();
const fileAt = path.join(repo.work, '.gstack/tmp/pregate.json');
+20
View File
@@ -412,6 +412,26 @@ describe('gstack-ship-preflight Step 5 pregate: head policy and timeout_s (GST-4
expect(u.stdout + u.stderr).toContain('--timeout needs whole seconds from 1 to 3600');
}
});
// Value: protects=a caller sets the whole Step 5 pregate's budget, and a kill at that limit says the limit was hit and names the
// way out instead of "unreadable (exit 143)"; fails_when=the budget is a fixed hour, or a timed-out pregate reads as unreadable;
// why_new=Vorflux via FOUN-32: gbrain's unit and serial suite ran past the fixed hour on 8-core Linux; seam=gstack-pregate under
// preflight's spawn limit; no_claim=the finish fallback's own pregate limit
test('--step-timeout bounds the whole Step 5 pregate, and a kill at the limit says so with the way out', () => {
const slow = "const test = require('node:test');\ntest('slow', async () => { await new Promise(r => setTimeout(r, 30000)); });\n";
const f = fx(base({ timeout_s: 600 }), { 'test/slow.serial.test.js': slow });
const r = f.run(PREFLIGHT, ['--tests-only', '--base', 'main', '--step-timeout', '3', '--lane', LANE]);
expect(r.status).toBe(3);
expect(lineOf(r.stdout, 'PREGATE')).toMatch(/^PREGATE: time limit hit \(Step 5's whole pregate ran past its 3 s budget, --step-timeout, and was stopped with exit \S+; no verdict\)$/);
expect(lineOf(r.stdout, 'STOP')).toBe("STOP: the tests are longer than this step's 3 s budget on this machine: push, let CI's full run at this tree carry them (gstack-ci-carry wait --base main, then rerun --tests-only; gbrain docs/ci-event-parity.md#ship-lanes), or rerun --tests-only with --step-timeout <s> above 3");
expect(r.stdout).not.toContain('unreadable');
expect(r.stdout).not.toContain('LANE: tests');
for (const bad of ['0', '86401', '1.5', 'x']) {
const u = f.run(PREFLIGHT, ['--tests-only', '--base', 'main', '--step-timeout', bad, '--lane', LANE]);
expect(u.status).toBe(2);
expect(u.stdout + u.stderr).toContain('--step-timeout needs whole seconds from 1 to 86400');
}
});
});
describe('gstack-ship-preflight during an open /ship run (GST-1204)', () => {