Let a repo keep its tier-only test folders out of /ship's local test run (GST-2500) (#4305)

* Let a declared test folder skip /ship's local test run

A [] declaration on a glob naming a test directory now leaves the selected
tests under it unrun in pregate's local lane, with one recorded line per glob.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Fix the declared test tree test's type

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Only skip tests for a [] on a folder inside a test directory, and only in the local lane

A [] on the whole test directory (test/**) is a common way to clear helpers and
must keep running mapped tests. The skip now applies to the local lane only, after
always-run tests join, so remote obligations and always-run tripwires are unchanged.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Keep a [] on a single test file from skipping that test

Review-Finding: p1/outside-voice#1 reader=outside-voice
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Say the remote gate also leaves out declared test tree tests

Review-Finding: p1/adversarial-native#6 reader=adversarial-native
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Keep always-run and remote-gate tests when a test tree is declared local-only

A [] on a test folder no longer drops an always-run test, no longer counts when it comes only from a branch's bootstrap policy, and no longer removes tests the remote gate or test backend owes. Tiers ran now names only tiers that ran a test.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* Reword the declared test tree release note into plain sentences

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-11-12 02:24:21 +00:00
1 parent b1f78301db
commit b9bbc31c5c
8 files changed
+174 -14

No files matched your search

@@ -0,0 +1,4 @@
---
important: blocker
---
- **A repo can keep its tier-only test trees out of /ship's local lane.** A pull request touching tests that only run under a separate tier (alphaclaw's `tests/live/**` and `tests/container/**`, which its test runner skips unless a tier flag is set) made the pregate pick those tests, and the files that import their helpers, and run them locally; the runner found no test files, the lane failed, and /ship wrote no receipt. Declaring `[]` for such a glob in `.gstack/pregate.json` used to do nothing for test files, because a test always selects itself. Now a `[]` on a folder inside a test directory (`tests/live/**`; the directory is `test/`, `tests/`, `spec/` or `__tests__/`) keeps the selected tests under it out of the local lane, and pregate prints one `declared test tree` line per glob with the count it left out, recorded with the rest of the lanes result. Nothing changes without such a declaration. Tests outside the glob still run, a `[]` on any other glob, the whole `test/**` included, never skips a test, and as before a new `[]` counts only once it is on the base branch (or in your own user policy), never from a branch's bootstrap policy. With one, a test `.gstack/always-run.json` names still runs, and whenever the remote gate, the test backend or a platform lane is owed, it still covers every selected test, because only the local run leaves them out.
+2 -2
View File
@@ -91,8 +91,8 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`).
| Key | Meaning |
|---|---|
| `runner` | the local test command with `{files}` (gstack: `bun test --timeout=30000 {files}`, the 30 s per-test timeout CI uses); set, the local lane gives it only the selected files its tool runs, judged by extension (JS/TS files for `bun test`, `node --test`, `deno test`, vitest, jest, mocha, ava or playwright; `.py` files for pytest; each matched as a whole word, so `npx vitest run` counts and `tests/node/run.sh` does not; every file for any other command, such as a `node scripts/run-tests.js` dispatcher, or one naming both), and the lanes detail names each file it cannot run as not run and not counted as covered, so a `*_test.py` never reaches a local `bun test` (no Python lane runs beside a declared runner; the remote gate and `test_backend` commands still receive the whole selection); unset, it is detected (bun, vitest or jest for JS/TS files, pytest when python3 can import it (else `pytest` or `python3` is named missing), `node --test`, and `flutter test`/`dart test` per Dart package root, through fvm when that package or an ancestor pins it with `.fvmrc` or `.fvm/fvm_config.json` and fvm is on PATH; `cargo test -p <name>` per Rust crate, run from the root of the workspace the crate is a member of (Cargo's `members` globs, `exclude` and `package.workspace`), else the crate directory; a JS/TS test whose nearest package.json is a nested package declaring vitest or jest (a dependency or its `test` script) runs from that package, with package-relative paths, through the package manager its `packageManager` field or the nearest lockfile up to the repo root names: `pnpm exec`, `yarn`, `bun x`, else `npx`; a JS/TS test file that imports `@playwright/test` at the start of a line runs as `playwright test --reporter=list` from its nearest package through that same package manager, whatever the unit runner is, a type-only import (`import type …`, or one whose every specifier is `type X`) does not count, file arguments are passed as escaped regexes anchored to the path (Playwright reads them as regexes), and a run where every test fails and Playwright says its browser is not installed is `PREGATE_ENVIRONMENT` (a partial install, where some tests pass, stays the diff's failure); every other test keeps the root detection, split by language when no one runner fits them all, so `.mjs` and `.py` root tests each get their own runner); nothing detected, a Dart SDK, cargo or package manager missing from PATH, a `.dart` file with no `pubspec.yaml` above it, a `.rs` file outside any crate, or one in a crate whose name Cargo would reject (anything but letters, digits, `_` and `-`), is `incomplete` (the runnable groups still run, so a real failure still fails) |
| `dependencies` | glob → test files for subprocess targets, runtime reads and generated inputs the import graph cannot see; `[]` declares "no lane needed" (docs, licences). A `[]` that matches a deleted test does not skip it: the deleted test still runs its owning suite, and a file naming it that `[]` covers (a durations list) counts as covered. With this key present, even empty, the policy [declares lanes](#lane-requirements) |
| `runner` | the local test command with `{files}` (gstack: `bun test --timeout=30000 {files}`, the 30 s per-test timeout CI uses); set, the local lane gives it only the selected files its tool runs, judged by extension (JS/TS files for `bun test`, `node --test`, `deno test`, vitest, jest, mocha, ava or playwright; `.py` files for pytest; each matched as a whole word, so `npx vitest run` counts and `tests/node/run.sh` does not; every file for any other command, such as a `node scripts/run-tests.js` dispatcher, or one naming both), and the lanes detail names each file it cannot run as not run and not counted as covered, so a `*_test.py` never reaches a local `bun test` (no Python lane runs beside a declared runner; the remote gate and `test_backend` commands still receive the whole selection, a declared test tree's tests included, since that exclusion is local-only); unset, it is detected (bun, vitest or jest for JS/TS files, pytest when python3 can import it (else `pytest` or `python3` is named missing), `node --test`, and `flutter test`/`dart test` per Dart package root, through fvm when that package or an ancestor pins it with `.fvmrc` or `.fvm/fvm_config.json` and fvm is on PATH; `cargo test -p <name>` per Rust crate, run from the root of the workspace the crate is a member of (Cargo's `members` globs, `exclude` and `package.workspace`), else the crate directory; a JS/TS test whose nearest package.json is a nested package declaring vitest or jest (a dependency or its `test` script) runs from that package, with package-relative paths, through the package manager its `packageManager` field or the nearest lockfile up to the repo root names: `pnpm exec`, `yarn`, `bun x`, else `npx`; a JS/TS test file that imports `@playwright/test` at the start of a line runs as `playwright test --reporter=list` from its nearest package through that same package manager, whatever the unit runner is, a type-only import (`import type …`, or one whose every specifier is `type X`) does not count, file arguments are passed as escaped regexes anchored to the path (Playwright reads them as regexes), and a run where every test fails and Playwright says its browser is not installed is `PREGATE_ENVIRONMENT` (a partial install, where some tests pass, stays the diff's failure); every other test keeps the root detection, split by language when no one runner fits them all, so `.mjs` and `.py` root tests each get their own runner); nothing detected, a Dart SDK, cargo or package manager missing from PATH, a `.dart` file with no `pubspec.yaml` above it, a `.rs` file outside any crate, or one in a crate whose name Cargo would reject (anything but letters, digits, `_` and `-`), is `incomplete` (the runnable groups still run, so a real failure still fails) |
| `dependencies` | glob → test files for subprocess targets, runtime reads and generated inputs the import graph cannot see; `[]` declares "no lane needed" (docs, licences). A `[]` on a folder inside a test directory (`test/`, `tests/`, `spec/` or `__tests__/`, as in `"tests/live/**": []`; never the whole directory, so `"test/**": []` changes nothing) is the repo's statement that the tests under it run in their own tier, not in /ship's local lane: pregate keeps every selected test under that glob out of the local lane, whether touched or picked through an import (a test `.gstack/always-run.json` names still runs; the remote side is unchanged: the remote gate, `test_backend` and any remote platform lane that selects the test still owe it; and only a base-branch or user-owned policy's `[]` counts, never a bootstrap head file read under `--policy-from` head when neither exists), and prints `declared test tree <glob>: N test file(s) not run in the local lane` in the `lanes` lines it records; tests outside the glob still run, and a `[]` on any other glob never skips a test. A `[]` that matches a deleted test does not skip it: the deleted test still runs its owning suite, and a file naming it that `[]` covers (a durations list) counts as covered. With this key present, even empty, the policy [declares lanes](#lane-requirements) |
| `empty_lanes` | extra globs for the docs/no-tests empty lane. Any Markdown file already qualifies with no pin file when no test depends on it (the same import graph as other files: a quoted path, a `join()` of path segments, an import, or a `dependencies` entry). reStructuredText and AsciiDoc count as Markdown here. README, CHANGELOG, CONTRIBUTING, LICENSE, AGENTS and CLAUDE qualify even without `.md`, and so does licence text under any vendored name (`roboto-LICENSE.txt`, `LICENSE-MIT`, `licenses/*.txt`). Source code and files under a test directory never qualify. A test that reads the file still selects that test |
| `selection` | `workflow.yml/job` → a command printing the lane's file list, for matrices a planner computes (gstack: `scripts/test-free-shards.ts --list`) |
| `warn` | check ids downgraded to warn; `secrets` and required lanes never. `"lanes"` downgrades only a touched file with no lane: the selected tests still run, a red lane stays `fail` (`PREGATE_LANE_FAILED`) and an unfinished one `incomplete`; a green run is `warn` `PREGATE_NO_LANE` naming the unmapped files |
+11 -9
View File
@@ -441,6 +441,7 @@ function describe(ctx: PregateContext, sel: Selection, mode: NoLaneMode): string
for (const r of sel.removed) lines.push(`removed (no lane owed): ${r.file} — ${r.note}`);
if (sel.emptyLane.length) lines.push(`docs/no-tests (empty lane): ${sel.emptyLane.join(' ')}`);
if (sel.inert.length) lines.push(`no-tests (assets and tool config): ${sel.inert.join(' ')}`);
for (const d of sel.declaredTier) lines.push(`declared test tree ${d.glob}: ${d.tests.length} test file(s) not run in the local lane (declared [] in ${ctx.configLabel}; the repo runs them in their own tier)`);
if (sel.stampOnly.length) lines.push(`version-stamp-only (no lane needed): ${sel.stampOnly.join(' ')} — only version values changed against the merge base; a manifest whose other content changed still needs a lane`);
for (const r of sel.remote) lines.push(`requires-remote ${r.lane} (${r.platform}): ${r.tests.length} test(s) — ${r.tests.slice(0, 8).join(' ')}${r.tests.length > 8 ? ' …' : ''}`);
lines.push(`Tiers ran: ${sel.tiersRan.join(' ') || '(none)'}; requires-remote: ${[...new Set(sel.remote.map(r => r.platform))].join(' ') || 'none'}`);
@@ -458,10 +459,10 @@ const GSTACK_ROOT = path.resolve(import.meta.dir, '..', '..', '..');
*/
function remoteObligation(ctx: PregateContext, sel: Selection): RemoteObligation {
const declared = ctx.config.remote_gate;
const runner = detectRunner(ctx, sel.localTests);
const command = declared ?? `${GSTACK_ROOT}/scripts/ubicloud/ubi-runner.sh run --src ${quote(ctx.repoRoot)} -- ${quote(runner ? runner.replace('{files}', sel.localTests.map(quote).join(' ')) : '<test runner> <selected files>')}`;
const runner = detectRunner(ctx, sel.gateTests);
const command = declared ?? `${GSTACK_ROOT}/scripts/ubicloud/ubi-runner.sh run --src ${quote(ctx.repoRoot)} -- ${quote(runner ? runner.replace('{files}', sel.gateTests.map(quote).join(' ')) : '<test runner> <selected files>')}`;
const [workflow, job] = declared ? REMOTE_GATE_LANE.split('/') as [string, string] : ['test_backend', ctx.testBackend];
return { lane: `${workflow}/${job}`, platform: 'remote', workflow, job, tests: sel.localTests, receipt: null, command };
return { lane: `${workflow}/${job}`, platform: 'remote', workflow, job, tests: sel.gateTests, receipt: null, command };
}
export const REMOTE_GATE_LANE = 'pregate.json/remote_gate';
@@ -508,7 +509,7 @@ async function runRemote(ctx: PregateContext, sel: Selection, base: CheckResult)
const log = path.join(dir, 'remote.log');
const filesFile = path.join(dir, 'files.txt');
fs.writeFileSync(log, '');
fs.writeFileSync(filesFile, sel.localTests.join('\n') + '\n');
fs.writeFileSync(filesFile, sel.gateTests.join('\n') + '\n');
const r = await supervise({ command: obligation.command!, cwd: ctx.repoRoot, env: { ...ctx.env, PREGATE_TREE: ctx.wtree, PREGATE_FILES_FILE: filesFile, PREGATE_SPEND_CAP_USD: String(cap), PREGATE_TIMEOUT_S: String(Math.floor(ctx.timeoutMs / 1000)) }, log, deadline: Date.now() + ctx.timeoutMs, stallMs: 0, lastLine: true, label: `remote gate ${obligation.lane}`, progress: ctx.progress });
const output = readRange(log, r.start, r.end);
const reported = parseRemoteReport(output);
@@ -542,17 +543,18 @@ async function runSelected(ctx: PregateContext, sel: Selection, base: CheckResul
: ctx.touched.length ? 'nothing touched selects a test' : 'no touched files';
return { ...base, detail };
}
if (ctx.remote && sel.localTests.length) return runRemote(ctx, sel, base);
const smallMachine = ctx.cores < SMALL_MACHINE_CORES && sel.localTests.length > 0 && (ctx.config.remote_gate || ctx.testBackend !== 'local');
if (ctx.remote && sel.gateTests.length) return runRemote(ctx, sel, base);
const smallMachine = ctx.cores < SMALL_MACHINE_CORES && sel.gateTests.length > 0 && (ctx.config.remote_gate || ctx.testBackend !== 'local');
if (smallMachine) {
const obligation = remoteObligation(ctx, sel);
const route = obligation.lane === REMOTE_GATE_LANE && ctx.config.remote_gate_runs && ctx.ciCarry !== false
? `a verified CI run at this tree clears it: push, then gstack-ci-carry wait --base ${ctx.base} and rerun gstack-pregate --allow-repo-commands (a pull_request run of a branch behind its base tested the merge, another tree: dispatch the workflow on the branch); or opt in to the VM: gstack-pregate --remote --spend-cap <usd> --allow-repo-commands`
: `run it, then ${clearHint(obligation.lane)}, or gstack-pregate --remote --spend-cap <usd> --allow-repo-commands`;
return { ...base, status: 'requires-remote', remote: [...sel.remote, obligation], lines: [...lines, `requires-remote ${obligation.lane}: ${ctx.cores} cores (under ${SMALL_MACHINE_CORES}); ${sel.localTests.length} local test file(s) recorded for the remote gate, not run here`, `run: ${obligation.command}`], detail: `${sel.localTests.length} local test file(s) recorded as requires-remote ${obligation.lane} (${ctx.cores} cores, under ${SMALL_MACHINE_CORES}); ${route}` };
return { ...base, status: 'requires-remote', remote: [...sel.remote, obligation], lines: [...lines, `requires-remote ${obligation.lane}: ${ctx.cores} cores (under ${SMALL_MACHINE_CORES}); ${sel.gateTests.length} local test file(s) recorded for the remote gate, not run here`, `run: ${obligation.command}`], detail: `${sel.gateTests.length} local test file(s) recorded as requires-remote ${obligation.lane} (${ctx.cores} cores, under ${SMALL_MACHINE_CORES}); ${route}` };
}
if (!ctx.allowRepoCommands) return { ...base, status: 'incomplete', code: 'REPO_COMMANDS_NOT_ALLOWED', detail: `${sel.localTests.length} local test file(s) selected but not run (pass --allow-repo-commands)`, fix: 'rerun with --allow-repo-commands (what /ship passes)' };
if (sel.localTests.length === 0) return { ...base, status: sel.remote.length ? 'requires-remote' : 'pass', detail: `no local lane; ${sel.remote.length} requires-remote obligation(s)` };
const tier = sel.declaredTier.reduce((n, d) => n + d.tests.length, 0);
if (sel.localTests.length === 0) return { ...base, status: sel.remote.length ? 'requires-remote' : 'pass', detail: `no local lane; ${sel.remote.length} requires-remote obligation(s)${tier ? `; ${tier} test file(s) in declared test trees not run here` : ''}` };
// Runnable groups run even when other files are unclaimed: a real failure is reported as one, and an all-green run is still incomplete.
const plan = planRuns(ctx, sel.localTests);
const gap = plan.unclaimed.length ? planGap(ctx, plan, sel.localTests.length) : null;
@@ -636,7 +638,7 @@ async function run(ctx: PregateContext): Promise<CheckResult> {
}
const reuse = ctx.fromRun ? fromRun(ctx, sel) : null;
if (reuse) ctx.progress(reuse.line);
const rest = reuse?.passed.length ? { ...sel, localTests: sel.localTests.filter(t => !reuse.passed.includes(t)) } : sel;
const rest = reuse?.passed.length ? { ...sel, localTests: sel.localTests.filter(t => !reuse.passed.includes(t)), gateTests: sel.gateTests.filter(t => !reuse.passed.includes(t)) } : sel;
const ran = await runSelected(ctx, rest, base);
const reused = !reuse ? ran : { ...ran, lines: [reuse.line, ...(ran.lines ?? [])], ...(reuse.receipt ? { from_run: { receipt: reuse.receipt, files: reuse.passed }, detail: `${reuse.passed.length} of ${sel.localTests.length} local test file(s) passed in the --from-run run; ${ran.detail ?? ran.status}` } : {}) };
const result = withRemoved(reused);
+3 -1
View File
@@ -218,6 +218,7 @@ export function buildContext(o: BuildContextOptions): ContextLoad {
let configLoad: ConfigLoad = loadConfigFromBase(o.repoRoot, base, o.knownChecks);
const explain: string[] = [];
let absentUserPolicy: string | undefined;
let headBootstrap = false;
if (o.userConfigPath && !isConfigUnavailable(configLoad) && configLoad.kind === 'none') {
const rejected = userConfigRejection(o.repoRoot, o.userConfigPath);
const user = rejected ? null : loadConfigFromUser(o.userConfigPath, o.knownChecks);
@@ -228,6 +229,7 @@ export function buildContext(o: BuildContextOptions): ContextLoad {
if (isConfigUnavailable(configLoad)) return { refused: { code: 'POLICY_SOURCE_UNAVAILABLE', message: configLoad.detail } };
if (o.policyFrom === 'head' && !configLoad.errors.length) {
if (o.remote && configLoad.kind === 'none') return { refused: { code: 'PREGATE_HEAD_POLICY_LOOSENS', message: '--remote under --policy-from head needs a trusted policy (the base copy or the user-owned file); with none, the head file is a bootstrap and its remote_gate cannot clear its own obligation' } };
headBootstrap = configLoad.kind === 'none';
const head = headPolicy(o.repoRoot, configLoad, mergeBase, base, o.knownChecks ?? []);
if ('loosens' in head) return { refused: { code: 'PREGATE_HEAD_POLICY_LOOSENS', message: head.loosens.join('; ') } };
configLoad = head.load;
@@ -250,7 +252,7 @@ export function buildContext(o: BuildContextOptions): ContextLoad {
return {
ctx: {
repoRoot: o.repoRoot, base, baseRef, baseSha: baseSha.out, mergeBase, head, wtree, touched: touchedFiles(o.repoRoot, mergeBase),
config, configLabel: configLoad.label, configKind: configLoad.kind, policyDeclaresLanes: configLoad.declaresLanes !== false, scratch, configFile: configLoad.kind === 'user' && o.userConfigPath ? o.userConfigPath : PREGATE_CONFIG_PATH, ...(configLoad.digest ? { configDigest: configLoad.digest } : {}), ...(absentUserPolicy ? { absentUserPolicy } : {}), registry: registryLoad.registry as Registry, registryLabel: registryLoad.source.label,
config, configLabel: configLoad.label, configKind: configLoad.kind, policyDeclaresLanes: configLoad.declaresLanes !== false, scratch, configFile: configLoad.kind === 'user' && o.userConfigPath ? o.userConfigPath : PREGATE_CONFIG_PATH, ...(configLoad.digest ? { configDigest: configLoad.digest } : {}), ...(absentUserPolicy ? { absentUserPolicy } : {}), ...(headBootstrap ? { headBootstrap } : {}), registry: registryLoad.registry as Registry, registryLabel: registryLoad.source.label,
lanes, platform, allowRepoCommands: o.allowRepoCommands, timeoutMs: o.timeoutMs ?? (config.timeout_s ?? 600) * 1000, timeoutSource: o.timeoutMs !== undefined ? '--timeout' : config.timeout_s ? `timeout_s from ${configLoad.label}` : 'the 600 s default',
stallTimeoutMs: o.stallTimeoutMs ?? (config.stall_timeout_s ?? DEFAULT_STALL_TIMEOUT_S) * 1000,
progress: o.progress ?? (line => process.stderr.write(`${line}\n`)),
+22 -2
View File
@@ -23,6 +23,9 @@ import type { Lane } from './workflows';
/** Whole-tree tests every selection that runs a test must include (GST-98). Kept out of pregate.json so older readers, which reject unknown keys, keep working. */
export const ALWAYS_RUN_PATH = '.gstack/always-run.json';
/** A glob naming a folder inside a test directory (`tests/live/**`, never `test/**` or a single file): its `[]` declaration keeps the tests under it out of the local lane. */
const TEST_SUBTREE = /(?:^|\/)(?:test|tests|spec|__tests__)\/[^*?[{/]+\//;
/** The `tests` globs of one `.gstack/always-run.json` text; `label` names the copy in errors. */
export function parseAlwaysRun(text: string, label: string = ALWAYS_RUN_PATH): string[] {
let parsed: unknown;
@@ -68,10 +71,14 @@ export interface Selection {
inert: string[];
/** Release manifests whose diff against the merge base changes only version values (lib/pregate/stamp-only.ts): waived, no lane needed. */
stampOnly: string[];
/** Local-lane tests a `[]` declaration on a folder inside a test directory (TEST_SUBTREE) kept out, per glob: the repo runs them in their own tier. Never an always-run test; they stay in `tests`, `gateTests` and any remote obligation. */
declaredTier: Array<{ glob: string; tests: string[] }>;
/** Every test the touched files select, sorted. */
tests: string[];
/** Tests that run on this machine. */
localTests: string[];
/** The local selection before declared test trees were left out: what the remote gate and `test_backend` owe, since that exclusion is local-only. */
gateTests: string[];
/** Platform lanes this machine cannot run that select part of the set. */
remote: RemoteObligation[];
/** Tiers of the local lanes the selection touches. */
@@ -283,8 +290,10 @@ export function selectLanes(ctx: PregateContext): Selection {
}
noLane.push(file);
}
const alwaysRun = new Set<string>();
if (tests.size || noLane.length || seeds.has(ALWAYS_RUN_PATH)) {
const always = alwaysRunTests(ctx.repoRoot, [...graph.tests].filter(t => !goneSet.has(t)), readAtMergeBase(ctx, ALWAYS_RUN_PATH, ['100644', '100755']));
for (const t of always.tests) alwaysRun.add(t);
for (const t of always.tests) if (!tests.has(t)) { tests.add(t); provenance.push(`${t} (always_run, ${always.baseOnly.includes(t) ? 'merge base ' : ''}${ALWAYS_RUN_PATH})`); }
for (const glob of always.unmatched) provenance.push(`${ALWAYS_RUN_PATH}: ${glob} matches no test file`);
}
@@ -293,6 +302,8 @@ export function selectLanes(ctx: PregateContext): Selection {
const remote: RemoteObligation[] = [];
const tiers = new Set<string>();
const selected = [...tests].sort();
const declaredTier = ctx.headBootstrap ? [] : Object.entries(ctx.config.dependencies ?? {}).filter(([glob, list]) => list.length === 0 && TEST_SUBTREE.test(glob) && /\/\*\*?$/.test(glob)).map(([glob]) => ({ glob, tests: [] as string[] }));
const tierOf = (t: string) => alwaysRun.has(t) ? undefined : declaredTier.find(d => matchesAny([d.glob], t));
let anyLocalAll = false;
for (const lane of ctx.lanes.filter(l => l.runsTests && !l.manual)) {
const set = laneFileSet(ctx, lane);
@@ -304,12 +315,21 @@ export function selectLanes(ctx: PregateContext): Selection {
continue;
}
if (set === 'all') anyLocalAll = true;
if (inLane.length) { tiers.add(lane.tier === 'platform' ? 'unit' : lane.tier); for (const t of inLane) local.add(t); }
if (inLane.some(t => !tierOf(t))) tiers.add(lane.tier === 'platform' ? 'unit' : lane.tier);
for (const t of inLane) local.add(t);
}
if (anyLocalAll || ctx.lanes.filter(l => l.runsTests && !l.manual && l.tier !== 'platform').length === 0) for (const t of selected) local.add(t);
for (const t of selected) if (!local.has(t) && !remote.some(r => r.tests.includes(t))) local.add(t);
const gateTests = [...local].sort();
for (const t of gateTests) {
const tier = tierOf(t);
if (!tier) continue;
tier.tests.push(t);
local.delete(t);
provenance.push(`${t}: not run in the local lane; ${tier.glob} is declared [] in ${PREGATE_CONFIG_PATH} (its own tier)`);
}
if (local.size && tiers.size === 0) tiers.add('unit');
return { noLane: noLane.sort(), removed: removed.sort((a, b) => a.file.localeCompare(b.file)), emptyLane: emptyLane.sort(), inert: inert.sort(), stampOnly: stampOnly.sort(), tests: selected, localTests: [...local].sort(), remote, tiersRan: [...tiers].sort(), laneSets, provenance };
return { noLane: noLane.sort(), removed: removed.sort((a, b) => a.file.localeCompare(b.file)), emptyLane: emptyLane.sort(), inert: inert.sort(), stampOnly: stampOnly.sort(), declaredTier: declaredTier.filter(d => d.tests.length), tests: selected, localTests: [...local].sort(), gateTests, remote, tiersRan: [...tiers].sort(), laneSets, provenance };
}
export function isTest(file: string): boolean { return isTestFile(file); }
+2
View File
@@ -36,6 +36,8 @@ export interface PregateContext {
configLabel: string;
/** Where the policy came from: the base branch, the unreviewed head, the user-owned file, or nowhere (defaults). */
configKind: 'base' | 'head' | 'user' | 'none';
/** A `--policy-from head` run with no base or user policy: the head file is a bootstrap, so its `[]` declarations on test folders drop nothing. */
headBootstrap?: boolean;
/** The active policy has a `dependencies`, `empty_lanes`, `selection` or `local` key (config LANE_KEYS); false keeps the no-policy lane verdicts. Absent counts as declared. */
policyDeclaresLanes?: boolean;
/** This run's scratch directory under the state root (lib/pregate/run-scratch.ts): logs and scan files, never os.tmpdir(). */
+1
View File
@@ -79,6 +79,7 @@ export const KNOWN_WINDOWS_INCOMPATIBLE: Array<{ file: string; reason: string }>
{ file: 'test/pregate-docs-lane-review.test.ts', reason: 'spawns bin/gstack-pregate through test/helpers/pregate-fixture.ts runBin (shebang via CreateProcess); the docs-lane selection rules it drives are POSIX-run like the other pregate fixtures' },
{ file: 'test/pregate-dart.test.ts', reason: 'its dart, flutter and fvm stand-ins are bash scripts on a colon-joined PATH, which Bun.which and CreateProcess cannot use on Windows, and its containment case needs a POSIX symlink; the Dart lane code it drives is POSIX-run like the other pregate fixtures' },
{ file: 'test/pregate-non-js.test.ts', reason: 'spawns bin/gstack-pregate through test/helpers/pregate-fixture.ts runBin and expects the Linux-host lanes verdict, like test/pregate-docs-lane-review.test.ts and test/pregate-selection.test.ts' },
{ file: 'test/pregate-declared-test-tree.test.ts', reason: 'spawns bin/gstack-pregate through test/helpers/pregate-fixture.ts runBin and expects the Linux-host lanes verdict (the outside test runs in the local lane), like test/pregate-selection.test.ts' },
{ file: 'test/pregate-remote-gbrain.test.ts', reason: 'spawns bin/gstack-pregate through test/helpers/pregate-fixture.ts runBin (shebang via CreateProcess) and runs the gbrain-shaped ci:ubicloud package script through the POSIX lane runner, like test/pregate-remote.test.ts' },
{ file: 'test/pregate-remote-stamp-only.test.ts', reason: 'expects the Linux-host lanes verdict like test/pregate-remote.test.ts (on Windows the fixture\'s Linux lane is a requires-remote obligation) and skips itself on win32 (describe.skipIf), so the curated lane would run an empty file (#3485, #3486)' },
// C-pregate: these drive bin/gstack-issue-links and bin/gstack-ship-receipt through
+129
View File
@@ -0,0 +1,129 @@
/**
* GST-2500: a `[]` declaration on a folder inside a test directory
* (`tests/live/**`, never the whole `test/**`) keeps the selected tests under it
* out of the local lane, and pregate says so. alphaclaw's tests/live/** and
* tests/container/** need their own tier; before this the lane ran them and the
* repo's runner found no test files. Fixture code only, never alphaclaw's.
*/
import { afterEach, describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { EXIT } from '../lib/headless-artifacts';
import { lanesCheck } from '../lib/pregate/checks/lanes';
import { buildContext } from '../lib/pregate/context';
import { BASE_PREGATE, cleanup, commit, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/pregate-fixture';
const TIER_TEST = "const test = require('node:test');\nconst { v } = require('./fixture');\ntest('needs the live tier', () => { throw new Error('tier test ran in the local lane ' + v); });\n";
const OUTSIDE_TEST = "const test = require('node:test');\nconst { v } = require('../tests/live/fixture');\ntest('reads the live fixture', () => { if (v !== 1) throw new Error('bad'); });\n";
const SCRIPT_TEST = "const test = require('node:test');\ntest('a test under a non-test glob', () => { throw new Error('script test ran'); });\n";
const BASE_FILES = { 'tests/live/fixture.js': 'exports.v = 1;\n', 'tests/live/x.test.js': TIER_TEST, 'test/outside.test.js': OUTSIDE_TEST, 'scripts/z.test.js': SCRIPT_TEST };
const repos: FixtureRepo[] = [];
afterEach(() => { while (repos.length) cleanup(repos.pop()!); });
const fixture = (declared: Record<string, string[]>) => {
const repo = makeFixtureRepo({ pregate: { ...BASE_PREGATE, dependencies: { ...BASE_PREGATE.dependencies, ...declared } }, baseFiles: BASE_FILES });
repos.push(repo);
return repo;
};
const lanes = (repo: FixtureRepo) => runBin(repo, 'gstack-pregate', ['--base', 'main', '--allow-repo-commands', '--only', 'lanes', '--explain']);
const LINE = /lanes: declared test tree tests\/live\/\*\*: 1 test file\(s\) not run in the local lane \(declared \[\] in origin\/main@[0-9a-f]{12}:\.gstack\/pregate\.json; the repo runs them in their own tier\)/;
describe('declared test trees', () => {
test('a touched test under a [] test-tree glob is not run and the drop is printed and recorded', () => {
const repo = fixture({ 'tests/live/**': [] });
commit(repo, { 'tests/live/x.test.js': TIER_TEST + '\n' });
const r = lanes(repo);
expect(r.out).not.toContain('tier test ran in the local lane');
expect(r.status).toBe(EXIT.ok);
expect(r.out).toMatch(LINE);
const recorded = JSON.parse(fs.readFileSync(path.join(repo.work, '.gstack', 'tmp', 'pregate.json'), 'utf8')).checks.find((c: { id: string }) => c.id === 'lanes');
expect(recorded.lines.map((l: string) => `lanes: ${l}`).join('\n')).toMatch(LINE);
expect(r.out).toContain('tests/live/x.test.js: not run in the local lane; tests/live/** is declared [] in .gstack/pregate.json (its own tier)');
expect(r.out).toContain('no local lane; 0 requires-remote obligation(s); 1 test file(s) in declared test trees not run here');
expect(r.out).not.toContain('has no test lane');
});
test('a test outside the glob that imports a file inside it still runs; the tier test it shares the file with does not', () => {
const repo = fixture({ 'tests/live/**': [] });
commit(repo, { 'tests/live/fixture.js': 'exports.v = 1; \n' });
const r = lanes(repo);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toContain('test/outside.test.js ← tests/live/fixture.js');
expect(r.out).toMatch(LINE);
expect(r.out).toContain("ran: node --test 'test/outside.test.js'");
expect(r.out).not.toContain('tier test ran in the local lane');
});
test('with no declaration, a glob that does not match, or a [] on the whole test directory, the tier test is still selected and run', () => {
for (const declared of [{}, { 'tests/container/**': [] }, { 'tests/**': [] }, { 'tests/live/x.test.js': [] }] as Array<Record<string, string[]>>) {
const repo = fixture(declared);
commit(repo, { 'tests/live/x.test.js': TIER_TEST + '\n' });
const r = lanes(repo);
expect(r.status).not.toBe(EXIT.ok);
expect(r.out).toContain('tests/live/x.test.js (itself a test)');
expect(r.out).not.toContain('declared test tree');
}
});
test('touching only a declared-tier test still runs the always_run tests', () => {
const repo = makeFixtureRepo({ pregate: { ...BASE_PREGATE, dependencies: { ...BASE_PREGATE.dependencies, 'tests/live/**': [] } }, baseFiles: { ...BASE_FILES, 'test/tree-tripwire.test.js': "const test = require('node:test');\ntest('scans the tree', () => {});\n", '.gstack/always-run.json': JSON.stringify({ tests: ['test/*-tripwire.test.js'] }) } });
repos.push(repo);
commit(repo, { 'tests/live/x.test.js': TIER_TEST + '\n' });
const r = lanes(repo);
expect(r.status).toBe(EXIT.ok);
expect(r.out).toContain('test/tree-tripwire.test.js (always_run, .gstack/always-run.json)');
expect(r.out).toContain("ran: node --test 'test/tree-tripwire.test.js'");
expect(r.out).toMatch(LINE);
expect(r.out).not.toContain('tier test ran in the local lane');
});
test('a [] for a test folder added only on the branch is refused under --policy-from head', () => {
const repo = fixture({});
commit(repo, { '.gstack/pregate.json': JSON.stringify({ ...BASE_PREGATE, dependencies: { ...BASE_PREGATE.dependencies, 'tests/live/**': [] } }) + '\n', 'tests/live/x.test.js': TIER_TEST + '\n' });
const r = runBin(repo, 'gstack-pregate', ['--base', 'main', '--allow-repo-commands', '--only', 'lanes', '--policy-from', 'head']);
expect(r.status).toBe(EXIT.refused);
expect(r.err).toContain('(PREGATE_HEAD_POLICY_LOOSENS');
});
test('an always_run test inside the declared tree still runs, whether always_run added it or the diff selected it first', () => {
for (const touched of <Array<Record<string, string>>>[{ 'lib/util.js': 'exports.add = (a, b) => a + b + 0;\n' }, { 'tests/live/x.test.js': TIER_TEST + '\n' }]) {
const repo = makeFixtureRepo({ pregate: { ...BASE_PREGATE, dependencies: { ...BASE_PREGATE.dependencies, 'tests/live/**': [] } }, baseFiles: { ...BASE_FILES, '.gstack/always-run.json': JSON.stringify({ tests: ['tests/live/**'] }) } });
repos.push(repo);
commit(repo, touched);
const r = lanes(repo);
expect(r.status).not.toBe(EXIT.ok);
expect(r.out).toContain('tier test ran in the local lane');
expect(r.out).not.toContain('declared test tree');
}
});
test('with no base or user policy, a [] the branch adds in its bootstrap policy drops nothing under --policy-from head', () => {
const repo = makeFixtureRepo({ pregate: null, baseFiles: BASE_FILES });
repos.push(repo);
commit(repo, { '.gstack/pregate.json': JSON.stringify({ ...BASE_PREGATE, dependencies: { ...BASE_PREGATE.dependencies, 'tests/live/**': [] } }) + '\n', 'tests/live/x.test.js': TIER_TEST + '\n' });
const r = runBin(repo, 'gstack-pregate', ['--base', 'main', '--allow-repo-commands', '--only', 'lanes', '--policy-from', 'head']);
expect(r.status).not.toBe(EXIT.ok);
expect(r.out).toContain('tier test ran in the local lane');
expect(r.out).not.toContain('declared test tree');
});
test('on a small machine the remote gate still owes a declared-tree test: the exclusion is local-only', async () => {
const repo = makeFixtureRepo({ pregate: { ...BASE_PREGATE, remote_gate: 'node scripts/remote-gate.js', dependencies: { ...BASE_PREGATE.dependencies, 'tests/live/**': [] } }, baseFiles: BASE_FILES });
repos.push(repo);
commit(repo, { 'tests/live/x.test.js': TIER_TEST + '\n' });
const loaded = buildContext({ repoRoot: repo.work, base: 'main', policyFrom: 'base', allowRepoCommands: true, env: { ...process.env, ...repo.env }, progress: () => {}, cores: 4, testBackend: 'local' });
if (!('ctx' in loaded)) throw new Error('fixture context refused');
const r = await lanesCheck.run(loaded.ctx);
expect(r.status).toBe('requires-remote');
expect(r.remote?.find(o => o.lane === 'pregate.json/remote_gate')?.tests).toEqual(['tests/live/x.test.js']);
});
test('a [] declaration on a glob that names no test directory keeps running the tests under it', () => {
const repo = fixture({ 'scripts/**': [] });
commit(repo, { 'scripts/z.test.js': SCRIPT_TEST + '\n' });
const r = lanes(repo);
expect(r.status).not.toBe(EXIT.ok);
expect(r.out).toContain('scripts/z.test.js (itself a test)');
expect(r.out).not.toContain('declared test tree');
});
});