CalvinBackup/gstack
1
0 Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-11-01 14:34:42 +01:00
Code Issues Packages Projects Releases Wiki Activity

v1.91.197.0 fix(pregate): Rust crate lanes and nested JS package runners (#3344, port of #3346) (#3380)

Browse Source 
* fix(pregate): Rust crate lanes and nested JS package runners

lanes reported PREGATE_NO_LANE for every touched .rs file and ran a nested
pnpm+vitest package's tests as `bun test` from the repo root.

- Rust: a .rs file of a crate (nearest Cargo.toml with [package]) is a test
  under <crate>/tests/ or when it holds a #[cfg(test)]; every .rs file of a
  crate and its Cargo.toml select the crate's tests and those of every crate
  depending on it (lib/pregate/rust.ts). Selected tests run per crate as
  `cargo test -p <name>` from the workspace root, judged on the summed
  `test result:` lines (doc-tests included); zero tests stays ZERO-RUN and a
  missing cargo is incomplete, like a missing Dart SDK.
- JS/TS: a test owned by a nested package.json that declares vitest or jest
  runs from that package with package-relative paths, through the package
  manager its packageManager field or nearest lockfile names (pnpm exec,
  yarn, bun x, else npx). Root detection and a declared runner are unchanged.
- init notes that Rust tests run per crate; docs/pregate.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pregate): parse Cargo manifests as TOML; follow workspace membership

The line scan of Cargo.toml missed dependents (a false green) and picked the
wrong workspace:
- an inherited renamed dependency (`base = { workspace = true }` with
  `[workspace.dependencies] base = { package = "alpha", path = ... }`);
- dotted keys (`alpha.path = "../a"`), `[dependencies.<name>]` tables and
  target-specific tables, matched by key name only;
- a crate the workspace excludes ran `cargo test -p` from the workspace root.

Manifests are now parsed with Bun.TOML. A dependency resolves by path
(relative to the manifest declaring it, or to the workspace root when
inherited), else by package name. A crate runs from the workspace it is a
member of under Cargo's rules (package.workspace, the nearest [workspace],
members globs, exclude, literal members, path dependencies of members),
else from its own directory. Nothing runs cargo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(pregate): the Rust fixture's cargo shim wins PATH on Windows and beside a real cargo

windows-free-shard (5) failed 'a touched src file runs its crate...' and
'zero tests is ZERO-RUN...' deterministically: the shim PATH was joined with
':' so Git Bash never saw the shim dir, and the runner's real cargo ran
against the fixture (exit 1, no test result lines). On POSIX hosts where
Bun and cargo share a directory (/usr/local/bin on Capy's image), the lane
puts Bun's directory first and the real cargo wins the same way.

Join with path.delimiter, add cargo.cmd for Bun.which on Windows, and on
POSIX put the shim first again in the lane's own bash through BASH_ENV.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): quote the crate name in cargoRunner

The crate name is read from the repo's Cargo.toml and was interpolated into
'cargo test -p <name>', which the lane runs through bash. A name like
'evil;touch pwned' ran the second command. Quote it like every file and cwd
the lane already passes; a new end-to-end case checks the name reaches cargo
as one argument and runs nothing.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(pregate): the nested-package pnpm shim wins PATH beside a real pnpm

'a touched file in a nested pnpm+vitest package runs vitest there' ran the
host's real pnpm wherever Bun and pnpm share a directory (/usr/local/bin on
Capy's image): the lane puts Bun's directory first, so pnpm installed vitest
from the network, dirtied the fixture tree and the check came back
incomplete. Put the shim first again in the lane's bash through BASH_ENV,
as the Rust fixture does.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): pregate Rust crate lanes and nested package runners (#3344)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): a crate name Cargo would reject never reaches the lane's shell

Port review (outside voice): quoting alone was not enough, because the lane
expands {files} after the runner string is built, so a crate named
'evil{files}' spliced the quoted file list into the quoted name and a file
name like tests/$(cmd).rs then ran. A name outside Cargo's grammar
(letters, digits, _ and -) now gets no runner: its files are unclaimed and
the lane is incomplete, as Cargo itself would refuse the manifest. A valid
name, Unicode included, still runs quoted.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): match Cargo members globs per path component, without a backtracking regex

Port review (outside voice): members globs went through ship-policy's
matchesAny, which escapes [...] classes (crates/[ab] matched nothing, so the
crate was no member, lost its inherited [workspace.dependencies] path and its
dependents' tests were never selected: a false green) and backtracks
exponentially on a hostile glob such as '*a*a...*ab' (over 5 s for one crate,
before any runner deadline). A small dynamic-programming matcher handles *,
?, [..] with negation and ranges, and ** across components, in
O(pattern x path).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): a plain #[test] in src is a Rust test; bound the cfg attribute scan

Port review (outside voice): only #[cfg(test)] modules and tests/ files
counted, so a dependent crate whose tests are plain #[test] or #[tokio::test]
functions in src never ran when its dependency changed (a false green). The
cfg scan also went quadratic on a run of unclosed '#[cfg(' (about 3 s on
120 KB). Test attributes now count, and the cfg predicate and its strings are
length-bounded so the scan stays linear.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(pregate): Rust test attributes, workspace membership and rejected crate names; changelog for the port review fixes

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): match members globs by code point in one pass, and admit every crate past a size cap

Second port review round (outside voice): the matcher counted UTF-16 units, so
'crates/?' missed a crate directory named with a supplementary character and
dropped its inherited dependencies (a false green); and each unmatched '['
rescanned the rest of the glob (5 s for one crate with a million of them).
Tokenize by code point with a precomputed next-']' table, and past 4096
characters of members globs admit every crate under the workspace: a wrong
member fails loudly in cargo, a dropped one is silent. The new supervised
probe kills a super-linear matcher at 30 s instead of hanging the file.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): scan cfg attributes once without truncating them; an unclosable one counts as a test

Second port review round (outside voice): the length bound from the first
round dropped a valid cfg(any(test, feature = "<257 chars>")), so its crate's
tests were no longer selected (a false green). Read each attribute once to
its closing ']' and resume after it, which keeps the scan linear without a
bound; an attribute the scan cannot close (an unterminated string, no ']')
counts as a test, which selects more, never fewer.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): second port review round (code-point member globs, unbounded linear cfg scan)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): past the members size cap, a crate's run root is undecided and its tests unclaimed

Third port review round (outside voice): admitting every crate past the cap
made a non-member run as 'cargo test -p <name>' from the workspace root,
where a same-named member answered and its passing tests stood in for the
orphan's failing one (a false green). Keep the conservative membership for
dependency edges, but leave such a crate's workspace undecided (null):
planRuns leaves its files unclaimed and the lane incomplete.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(pregate): the Rust end-to-end runs put the cargo shim under Windows' own Path key

windows-free-shard (5) still failed both end-to-end cases at fe99b200 with
the same symptom (the runner's real cargo, exit 1, no test result lines).
runBin merges process.env, whose Windows key is 'Path', with the shim's
'PATH', so the child got two path variables and read the original. Write the
shim under the key process.env actually uses, as test/helpers/unreachable-gbrain.ts
and sync-gbrain-readiness-fixture.ts do.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(pregate): run the cargo stand-in cases on POSIX only

windows-free-shard (5) still failed both end-to-end cargo-lane cases after the
delimiter, cargo.cmd and Path-key repairs: the lane's Git Bash exits 1 with an
empty log, and that log is not visible from CI. Skip the two cases that run the
bash stand-in on win32 (test.skipIf, as other POSIX-shim pregate fixtures are
handled) and drop the unproven cargo.cmd and Path-key changes. Selection,
planning and the hostile-name case still run on Windows.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* v1.91.197.0 stamp

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: vilano93 <244097767+vilano93@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tancapy-ai[bot]vilano93Claude Opus 5.5
 authored and GitHub committed 2026-10-25 06:25:53 +00:00
1 parent 080b752140
commit c739dfa494
11 files changed
+873 -29
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CHANGELOG.md
+22
View File
@@ -1,5 +1,27 @@
# Changelog
## [1.91.197.0] - 2026-10-25
**Pregate gives touched Rust files a test lane, and runs a nested pnpm or vitest package with its own runner (#3344).** In a mixed Cargo and JavaScript repo, every touched `.rs` file failed `lanes` with "has no test lane", so /ship could not get a receipt, and a nested package's vitest tests ran under `bun test` from the repo root (about 98 false failures in the reporter's repo). Ported from #3346 by vilano93, whose two commits carry the feature; our commits on top close four findings from a security-focused review of the port and make the new fixtures hermetic.
### Fixed
- **`lib/pregate/rust.ts`** (new, vilano93): a crate is the nearest Cargo.toml with `[package]`. Its `.rs` files under `tests/` or holding a `#[cfg(test)]` are tests, and every `.rs` file of a crate and its Cargo.toml select that crate's tests and those of every crate depending on it. Manifests are parsed as TOML, so renamed, inherited, dotted, table-form and target-specific dependencies all count, and a crate runs from the workspace it is a member of under Cargo's rules (members, exclude, `package.workspace`). Nothing runs cargo to build the graph.
- **`lib/pregate/checks/lanes.ts`** (vilano93): selected Rust tests run per crate as `cargo test -p <name>` from the workspace root and are judged on the summed `test result:` lines; zero tests is ZERO-RUN and a missing cargo is incomplete. A JS/TS test owned by a nested package.json that declares vitest or jest runs from that package through the package manager its `packageManager` field or nearest lockfile names (`pnpm exec`, `yarn`, `bun x`, else `npx`).
- **`lib/pregate/checks/lanes.ts`**: the crate name comes from the repo's Cargo.toml and reaches the lane's shell. Before, a crate named `evil;touch pwned` ran `touch pwned` in the workspace, and a name containing `{files}` spliced the file list into the command. A name outside Cargo's grammar (letters, digits, `_`, `-`) now gets no runner, so its files are unclaimed and the lane is incomplete, as Cargo would refuse the manifest anyway; a valid name is quoted like every file and cwd. The lane only runs with `--allow-repo-commands`, under which the repo's tests run anyway, but no other repo input reached the shell unquoted.
- **`lib/pregate/rust.ts`**: Cargo `members` globs are matched per path component and by Unicode code point, in one pass. The first cut used the policy glob matcher, which matched `[ab]` literally (a member listed that way lost its inherited workspace dependencies, so its tests were never selected) and backtracked for seconds on a hostile glob. Past 4096 characters of `members` globs, membership is undecided: every crate under the workspace counts as a member for dependency edges (more tests, never fewer), but its own tests are unclaimed and the lane is incomplete, because running `cargo test -p <name>` from the wrong root can test a same-named member instead.
- **`lib/pregate/rust.ts`**: a plain `#[test]` or `#[tokio::test]` function in `src/` makes the file a test, so a dependent crate tested that way runs when its dependency changes. The `#[cfg(...)]` scan reads each attribute once, up to its closing `]`, so a file of unclosed attributes no longer takes seconds, and an attribute it cannot close counts as a test (more tests selected, never fewer).
- **`lib/pregate/init-draft.ts`** (vilano93): `init` notes that Rust tests run per crate.
### Tests
- `test/pregate-rust.test.ts` (new, vilano93) covers the crate model, the dependency forms, workspace membership, the cargo summary parser and three end-to-end lane runs. New cases: crates named `evil;touch pwned-by-crate-name` and `evil{files}` never reach the shell (nothing is created, both are unclaimed) while `naïve_crate-2` still runs quoted; `crates/[ab]`, `nested/**/leaf` and `unicode/?` (a 📦 directory) members inherit workspace dependencies; a plain `#[test]` dependent and a `cfg(any(test, feature = "<300 chars>"))` are tests; and a same-named non-member past the size cap is unclaimed rather than run as the member; a hostile members glob, a million unmatched `[` and 100,000 unclosed `#[cfg(` run in a supervised Bun that kills a super-linear version at 30 s. Each failed before its fix.
- The cargo stand-in is hermetic on POSIX: where Bun and cargo share a directory, the lane puts Bun's directory first and the real cargo ran against the fixture, so the shim PATH now uses the platform delimiter and the lane's bash puts the shim first again through `BASH_ENV`. On windows-free-shard (5) the two cases that run the stand-in failed deterministically (`local lane exited 1` with an empty log) and still did after the delimiter and a `Path`-key repair, so they are skipped on Windows (`test.skipIf(process.platform === 'win32')`, as other POSIX-shim pregate fixtures are); the selection, planning and hostile-name cases still run there.
- `test/pregate-selection.test.ts` (vilano93) covers nested pnpm/yarn/vitest/jest packages. Its `pnpm` stand-in gets the same `BASH_ENV` fix: on a host with a real pnpm beside Bun, the end-to-end case installed vitest from the network and came back incomplete.
### Gate integrity
- One coverage change to review: the two new end-to-end cargo-lane cases in `test/pregate-rust.test.ts` are skipped on Windows (they never passed there); Linux and macOS run them, and the rest of the file runs on Windows.
- No threshold, budget, timeout, trial count or case kind changed. Pregate's lanes now map and run tests for `.rs` files that previously failed closed with "no test lane", and nested vitest/jest packages run under their own runner instead of `bun test`. A `.rs` file outside any crate, a crate with no tests, and a missing cargo or package manager still fail closed or stay incomplete; the tests above cover each.
- No new dependency (Cargo.toml is parsed with Bun's built-in TOML parser), no network call in pregate itself, no workflow change. As with the root runner detection on main (`npx vitest run`), a nested package's detected runner (`npx`, `bun x`) may fetch a missing tool when it runs under `--allow-repo-commands`.
## [1.91.196.0] - 2026-10-25
**`/land-and-deploy` no longer stops parallel PRs with a false "VERSION drift detected".** Step 3.4 compared the branch VERSION with the next free patch slot (`gstack-next-version --bump patch --exclude-pr`) under every ship policy. So it stopped a stamp-at-merge PR whenever a sibling shared its tentative stamp, a claim-at-open PR whenever a sibling had claimed a higher slot, and every MICRO bump even with no other PR open. Reported with a full reproduction by Evgenii Lopatin (@exGeni) in #3318.
VERSION
+1 -1
View File
@@ -1 +1 @@
1.91.196.0
1.91.197.0
agents-digest/gstack-AGENTS.md
+1 -1
View File
@@ -1,4 +1,4 @@
# gstack digest v1.91.196.0 — regenerate/re-copy after upgrading gstack
# gstack digest v1.91.197.0 — regenerate/re-copy after upgrading gstack
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
for agent hosts without a full skill install. The full skills add workflows,
docs/pregate.md
+11 -4
View File
@@ -59,7 +59,7 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`).
| Key | Meaning |
|---|---|
| `runner` | the local test command with `{files}` (gstack: `bun test {files}`); unset, it is detected (bun, vitest, jest, pytest, `node --test`, and `flutter test`/`dart test` per Dart package root, through fvm when that package or an ancestor pins it with `.fvmrc` or `.fvm/fvm_config.json` and fvm is on PATH); nothing detected, a Dart SDK missing from PATH, or a `.dart` file with no `pubspec.yaml` above it is `incomplete` (the runnable groups still run, so a real failure still fails) |
| `runner` | the local test command with `{files}` (gstack: `bun test {files}`); unset, it is detected (bun, vitest, jest, pytest, `node --test`, and `flutter test`/`dart test` per Dart package root, through fvm when that package or an ancestor pins it with `.fvmrc` or `.fvm/fvm_config.json` and fvm is on PATH; `cargo test -p <name>` per Rust crate, run from the root of the workspace the crate is a member of (Cargo's `members` globs, `exclude` and `package.workspace`), else the crate directory; a JS/TS test whose nearest package.json is a nested package declaring vitest or jest (a dependency or its `test` script) runs from that package, with package-relative paths, through the package manager its `packageManager` field or the nearest lockfile up to the repo root names: `pnpm exec`, `yarn`, `bun x`, else `npx`; every other JS/TS test keeps the root detection); nothing detected, a Dart SDK, cargo or package manager missing from PATH, a `.dart` file with no `pubspec.yaml` above it, a `.rs` file outside any crate, or one in a crate whose name Cargo would reject (anything but letters, digits, `_` and `-`), is `incomplete` (the runnable groups still run, so a real failure still fails) |
| `dependencies` | glob → test files for subprocess targets, runtime reads and generated inputs the import graph cannot see; `[]` declares "no lane needed" (docs, licences) |
| `empty_lanes` | extra globs for the docs/no-tests empty lane. Any Markdown file already qualifies with no pin file when no test depends on it (the same import graph as other files: a quoted path, a `join()` of path segments, an import, or a `dependencies` entry). reStructuredText and AsciiDoc count as Markdown here. README, CHANGELOG, CONTRIBUTING, LICENSE, AGENTS and CLAUDE qualify even without `.md`, and so does licence text under any vendored name (`roboto-LICENSE.txt`, `LICENSE-MIT`, `licenses/*.txt`). Source code and files under a test directory never qualify. A test that reads the file still selects that test |
| `selection` | `workflow.yml/job` → a command printing the lane's file list, for matrices a planner computes (gstack: `scripts/test-free-shards.ts --list`) |
@@ -135,7 +135,13 @@ Two stages with separate clocks, both printed:
one has no lane until you declare `"**/integration_test/**": []` (and
`"**/test_driver/**": []`) under `dependencies` and let CI run them; Dart files, device tests
included, are source code, so a `dependencies` `[]` declaration clears them and `empty_lanes`
does not), transitively through
does not); in a Rust crate (the nearest Cargo.toml with a `[package]` name) a `.rs` file is a
test when it sits under `<crate>/tests/`, holds a `#[cfg(test)]` (any cfg predicate naming
`test`) or a test attribute (`#[test]`, `#[tokio::test]`), and every `.rs` file of the crate and its Cargo.toml select the crate's tests and
those of every crate whose manifest depends on it, directly or through other crates (cargo
runs a crate's tests, not a file, so the lane is the crate; a crate with no tests still has
no lane, and a workspace-only Cargo.toml or Cargo.lock needs a `dependencies` entry),
transitively through
non-test files; deleted and renamed files select the tests that named them.
An extensionless JS/TS import (relative, a tsconfig/jsconfig `paths` alias or
`baseUrl`) tries the exact path, then `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`,
@@ -172,7 +178,8 @@ Two stages with separate clocks, both printed:
that step's `working-directory`, else the job's or workflow's
`defaults.run.working-directory`): local ones run through the runner under
`set -o pipefail`, judged on the runner's summary line (zero tests ran is a
failure, a `| tail`-hidden exit is still an exit); a platform lane this
failure, a `| tail`-hidden exit is still an exit; cargo's `test result:` lines, one per
test binary and doc-tests included, are summed); a platform lane this
machine cannot run becomes a `requires-remote` obligation naming
`workflow.yml/job`. `--explain` prints each test's provenance and every
uncertain mapping (a planner-computed matrix without a `selection` manifest
@@ -234,7 +241,7 @@ test-tier command:
Per-file lines come from a progress adapter where the runner's reporter
provides them: `bun test` (its `<file>:` headers) is the only adapter today;
a one-file run (each serial file) reports its file from the launch for any
runner. Other runners (node --test, vitest, jest, pytest, flutter test, dart test) get heartbeats
runner. Other runners (node --test, vitest, jest, pytest, flutter test, dart test, cargo test) get heartbeats
only. A heartbeat follows each 60 s with no progress line;
- is killed as **stalled** when its log does not grow for the stall timeout
(`--stall-timeout <s>`, else `stall_timeout_s`, else 300; `0` disables): the
lib/pregate/checks/lanes.ts
+114 -17
View File
@@ -16,9 +16,10 @@ import * as os from 'node:os';
import * as path from 'node:path';
import { baseInputs, sha256 } from '../context';
import { PREGATE_CONFIG_PATH } from '../config';
import { isDeviceDart, readRepoFile } from '../import-graph';
import { isDeviceDart, listRepoFiles, readRepoFile } from '../import-graph';
import { selectLanes, type Selection } from '../lane-selection';
import { launchRuntime } from '../runtime';
import { cargoModel, type CargoModel } from '../rust';
import { adapterFor, readRange, supervise, type Counter, type Supervised } from '../supervise';
import { matchesAny } from '../../ship-policy';
import type { CheckDef, CheckResult, PregateContext, RemoteObligation } from '../types';
@@ -42,21 +43,24 @@ export function detectRunner(ctx: Pick<PregateContext, 'config' | 'repoRoot'>, f
/** One runner invocation: `files` relative to `cwd` (repo-relative, `''` for the root). */
export interface RunGroup { runner: string; cwd: string; files: string[] }
/** The nearest directory at or above `file` holding a pubspec.yaml, repo-relative; null outside any Dart package. `seen` memoises per directory. */
function dartRoot(repoRoot: string, file: string, seen: Map<string, string | null>): string | null {
/** The nearest directory at or above `file` holding `marker`, repo-relative; null when none does. `seen` memoises per directory (one map per marker). */
function nearestRoot(repoRoot: string, file: string, marker: string, seen: Map<string, string | null>): string | null {
const walked: string[] = [];
let found: string | null = null;
for (let d = path.posix.dirname(file); ; d = path.posix.dirname(d)) {
const rel = d === '.' ? '' : d;
if (seen.has(rel)) { found = seen.get(rel)!; break; }
walked.push(rel);
if (fs.existsSync(path.join(repoRoot, rel, 'pubspec.yaml'))) { found = rel; break; }
if (fs.existsSync(path.join(repoRoot, rel, marker))) { found = rel; break; }
if (d === '.' || d === '/' || d === '') break;
}
for (const w of walked) seen.set(w, found);
return found;
}
/** The nearest directory at or above `file` holding a pubspec.yaml, repo-relative; null outside any Dart package. */
function dartRoot(repoRoot: string, file: string, seen: Map<string, string | null>): string | null { return nearestRoot(repoRoot, file, 'pubspec.yaml', seen); }
/** An fvm pin (`.fvmrc` or `.fvm/fvm_config.json`) at the package root `root` or any ancestor up to the repo root. */
function fvmPinned(repoRoot: string, root: string): boolean {
for (let d = root; ; d = path.posix.dirname(d)) {
@@ -72,6 +76,9 @@ function fvmPinnedFor(repoRoot: string, files: readonly string[]): boolean {
return files.some(f => { const root = f.endsWith('.dart') ? dartRoot(repoRoot, f, seen) : null; return root !== null && fvmPinned(repoRoot, root); });
}
/** Whether `bin` is on the PATH of `env` (the PATH the runner will get). */
const onPath = (bin: string, env: NodeJS.ProcessEnv) => !!Bun.which(bin, { PATH: env.PATH ?? '' });
/**
* `flutter test` for a package on the Flutter SDK, else `dart test`; through fvm
* when the package or an ancestor up to the repo root pins one and fvm is on PATH. `missing` names the tool when
@@ -82,48 +89,129 @@ export function dartRunner(repoRoot: string, root: string, env: NodeJS.ProcessEn
const pubspec = readRepoFile(repoRoot, path.posix.join(root, 'pubspec.yaml')) ?? '';
// `sdk: flutter` (the key optionally quoted) under any dependency name (flutter, flutter_test, ...), block style or inside a flow mapping a key opens, comments stripped.
const tool = /^[ \t]*(['"]?)sdk\1:[ \t]*['"]?flutter\b|^[ \t]*[\w-]+:[ \t]*\{(?:[^\n]*[{,])?[ \t]*(['"]?)sdk\2:[ \t]*['"]?flutter\b/m.test(pubspec.replace(/(^|\s)#.*$/gm, '$1')) ? 'flutter' : 'dart';
const onPath = (bin: string) => !!Bun.which(bin, { PATH: env.PATH ?? '' });
const fvm = fvmPinned(repoRoot, root) && onPath('fvm');
if (!fvm && !onPath(tool)) return { missing: tool };
const fvm = fvmPinned(repoRoot, root) && onPath('fvm', env);
if (!fvm && !onPath(tool, env)) return { missing: tool };
return { runner: `${fvm ? 'fvm ' : ''}${tool} test {files}` };
}
/** The run plan: the groups that can run, the files no runner claims, Dart SDKs missing from PATH, and `.dart` files with no pubspec.yaml above. */
/**
* `cargo test -p <name>` for a crate (cargo runs a crate's tests, never a file list), or the missing tool when cargo is
* not on PATH. The name comes from the repo's Cargo.toml and reaches a shell and the `{files}` expansion: a name outside
* Cargo's grammar (letters, digits, `_`, `-`) gets no runner (null), and a valid one is still quoted like every file and cwd.
*/
function cargoRunner(name: string, env: NodeJS.ProcessEnv): { runner: string } | { missing: string } | null {
if (!/^[\p{ID_Continue}-]+$/u.test(name)) return null;
return onPath('cargo', env) ? { runner: `cargo test -p ${quote(name)}` } : { missing: 'cargo' };
}
type PackageJson = { packageManager?: string; devDependencies?: Record<string, string>; dependencies?: Record<string, string>; scripts?: Record<string, string> };
function readPackageJson(repoRoot: string, dir: string): PackageJson | null {
try { return JSON.parse(readRepoFile(repoRoot, path.posix.join(dir, 'package.json')) ?? 'null'); } catch { return null; }
}
const LOCKFILES: Array<[string, string]> = [['pnpm-lock.yaml', 'pnpm'], ['yarn.lock', 'yarn'], ['bun.lock', 'bun'], ['bun.lockb', 'bun'], ['package-lock.json', 'npm'], ['npm-shrinkwrap.json', 'npm']];
const PM_EXEC: Record<string, string> = { pnpm: 'pnpm exec', yarn: 'yarn', bun: 'bun x', npm: 'npx' };
/** The package manager for a package: the first `packageManager` field or lockfile from its directory up to the repo root; npm when none. */
function packageManager(repoRoot: string, dir: string): string {
for (let d = dir; ; d = path.posix.dirname(d)) {
const rel = d === '.' ? '' : d;
const declared = /^(pnpm|yarn|bun|npm)@/.exec(readPackageJson(repoRoot, rel)?.packageManager ?? '')?.[1];
if (declared) return declared;
const lock = LOCKFILES.find(([f]) => fs.existsSync(path.join(repoRoot, rel, f)));
if (lock) return lock[1];
if (rel === '') return 'npm';
}
}
/**
* The runner a nested package (not the repo root) declares for itself: vitest
* or jest in its dependencies or its `test` script, through the package manager
* its `packageManager` field or nearest lockfile names; null when it declares
* neither, so its files stay with the root runner. `missing` names a package
* manager that is not on PATH.
*/
function packageRunner(repoRoot: string, dir: string, env: NodeJS.ProcessEnv): { runner: string } | { missing: string } | null {
const pkg = readPackageJson(repoRoot, dir);
if (!pkg) return null;
const deps = { ...pkg.dependencies, ...pkg.devDependencies };
const script = pkg.scripts?.test ?? '';
const tool = deps.vitest || /\bvitest\b/.test(script) ? 'vitest run' : deps.jest || /\bjest\b/.test(script) ? 'jest' : null;
if (!tool) return null;
const pm = packageManager(repoRoot, dir);
const bin = pm === 'npm' ? 'npx' : pm;
if (!onPath(bin, env)) return { missing: bin };
return { runner: `${PM_EXEC[pm]} ${tool} {files}` };
}
/** The run plan: the groups that can run, the files no runner claims, tools missing from PATH, and `.dart` files with no pubspec.yaml above. */
export interface RunPlan { groups: RunGroup[]; unclaimed: string[]; missing: string[]; noPubspec: string[] }
/**
* The invocations that run `files`. A declared runner takes every file. Otherwise
* Dart tests run from their own package root (a sub-package's imports resolve
* against its own pubspec), and the rest go to the detected runner. Files no
* runner claims (a Dart SDK missing from PATH, a `.dart` file outside any
* package, no detected runner) are `unclaimed`: the lane is `incomplete`, never a pass.
* against its own pubspec); Rust tests run per crate as `cargo test -p <name>`
* from the crate's workspace root; JS/TS tests owned by a nested package that
* declares vitest or jest run from that package; the rest go to the detected
* runner from the repo root. Files no runner claims (a tool missing from PATH,
* a `.dart` file outside any package, a `.rs` file outside any crate or in a
* crate whose name Cargo would reject, no detected runner) are `unclaimed`: the lane is `incomplete`, never a pass.
*/
export function planRuns(ctx: PregateContext, files: readonly string[]): RunPlan {
if (ctx.config.runner) return { groups: [{ runner: ctx.config.runner, cwd: '', files: [...files] }], unclaimed: [], missing: [], noPubspec: [] };
const groups: RunGroup[] = [];
const dart = new Map<string, string[]>();
const cargo = new Map<string, { name: string; cwd: string; files: string[] }>();
const packages = new Map<string, string[]>();
const seen = new Map<string, string | null>();
const seenPackage = new Map<string, string | null>();
let model: CargoModel | null = null;
const pkgRunners = new Map<string, ReturnType<typeof packageRunner>>();
const rest: string[] = [];
const noPubspec: string[] = [];
const unclaimed: string[] = [];
const missing = new Set<string>();
const add = <T>(map: Map<string, T[]>, key: string, value: T) => { let list = map.get(key); if (!list) map.set(key, (list = [])); list.push(value); };
const under = (root: string, f: string) => (root ? f.slice(root.length + 1) : f);
for (const f of files) {
if (f.endsWith('.rs')) {
model ??= cargoModel(listRepoFiles(ctx.repoRoot), rel => readRepoFile(ctx.repoRoot, rel));
const crate = model.crateOf(f);
const cwd = crate?.workspace ?? null;
if (!crate || cwd === null) { unclaimed.push(f); continue; }
const key = `${cwd}\0${crate.name}`;
if (!cargo.has(key)) cargo.set(key, { name: crate.name, cwd, files: [] });
cargo.get(key)!.files.push(under(cwd, f));
continue;
}
const pkg = /\.[cm]?[jt]sx?$/.test(f) ? nearestRoot(ctx.repoRoot, f, 'package.json', seenPackage) : null;
if (pkg) {
if (!pkgRunners.has(pkg)) pkgRunners.set(pkg, packageRunner(ctx.repoRoot, pkg, ctx.env));
const r = pkgRunners.get(pkg);
if (r && 'missing' in r) { missing.add(r.missing); unclaimed.push(f); continue; }
if (r) { add(packages, pkg, under(pkg, f)); continue; }
}
if (!f.endsWith('.dart')) { rest.push(f); continue; }
const root = dartRoot(ctx.repoRoot, f, seen);
if (root === null) { noPubspec.push(f); continue; }
let list = dart.get(root);
if (!list) dart.set(root, (list = []));
list.push(root ? f.slice(root.length + 1) : f);
add(dart, root, under(root, f));
}
const unclaimed: string[] = [];
const missing = new Set<string>();
for (const [root, list] of [...dart].sort(([a], [b]) => a.localeCompare(b))) {
const r = dartRunner(ctx.repoRoot, root, ctx.env);
if ('missing' in r) { missing.add(r.missing); unclaimed.push(...list.map(f => (root ? `${root}/${f}` : f))); } else groups.push({ runner: r.runner, cwd: root, files: list });
}
for (const [, c] of [...cargo].sort(([a], [b]) => a.localeCompare(b))) {
const r = cargoRunner(c.name, ctx.env);
if (r && 'runner' in r) { groups.push({ runner: r.runner, cwd: c.cwd, files: c.files }); continue; }
if (r) missing.add(r.missing);
unclaimed.push(...c.files.map(f => (c.cwd ? `${c.cwd}/${f}` : f)));
}
if (rest.length) {
const runner = detectRunner(ctx, rest);
if (runner) groups.push({ runner, cwd: '', files: rest });
else unclaimed.push(...rest);
}
for (const [root, list] of [...packages].sort(([a], [b]) => a.localeCompare(b))) groups.push({ runner: (pkgRunners.get(root) as { runner: string }).runner, cwd: root, files: list });
return { groups, unclaimed: [...unclaimed, ...noPubspec], missing: [...missing].sort(), noPubspec };
}
@@ -153,9 +241,18 @@ function parseDart(output: string): RunnerSummary | null {
return d ? { ran: Number(d[1]) + Number(d[2] ?? 0), failed: Number(d[2] ?? 0) } : null;
}
/** The runner's own summary line: bun, node --test, vitest, jest, pytest, bats, dart/flutter test. A bats or dart/flutter runner's summary is tried first; for any other runner those formats are a last resort. `ran: null` when not recognised. Colour codes are dropped and a carriage return ends a line, so a coloured or redrawn verdict still reads. */
/** cargo test: one `test result: ok|FAILED. N passed; M failed; …` line per test binary (unit tests, each integration test, doc-tests), summed; ignored and filtered-out tests did not run. */
function parseCargo(output: string): RunnerSummary | null {
const lines = [...output.matchAll(/^test result: \w+\. (\d+) passed; (\d+) failed;/gm)];
if (!lines.length) return null;
const failed = lines.reduce((n, m) => n + Number(m[2]), 0);
return { ran: lines.reduce((n, m) => n + Number(m[1]), 0) + failed, failed };
}
/** The runner's own summary line: bun, node --test, vitest, jest, pytest, bats, dart/flutter test, cargo test. A bats, dart/flutter or cargo runner's summary is tried first (cargo's only there: its `N passed` lines would misread as another runner's); for any other runner the bats and dart formats are a last resort. `ran: null` when not recognised. Colour codes are dropped and a carriage return ends a line, so a coloured or redrawn verdict still reads. */
export function parseSummary(raw: string, runner = ''): RunnerSummary {
const output = raw.replace(/\x1b\[[0-9;?]*[A-Za-z]/g, '').replace(/\r/g, '\n');
if (/(?<![\w-])cargo\s+test(?![\w-])/.test(runner)) return parseCargo(output) ?? { ran: null, failed: null };
if (/(?<![\w-])bats(?![\w-])/.test(runner)) { const b = parseBats(output); if (b) return b; }
const dartRun = /(?<![\w-])(?:dart|flutter)(?![\w-])/.test(runner);
if (dartRun) { const d = parseDart(output); if (d) return d; }
lib/pregate/import-graph.ts
+8 -1
View File
@@ -12,7 +12,10 @@
* 3. the `bin/<name>` → `test/<name>*.test.*` convention (with and without
* the `gstack-` prefix);
* 4. explicit `.gstack/pregate.json` `dependencies` declarations;
* 5. in a Django project, the files Django loads by name (lib/pregate/django.ts).
* 5. in a Django project, the files Django loads by name (lib/pregate/django.ts);
* 6. in a Cargo crate, every `.rs` file and the Cargo.toml select the crate's
* tests and its dependents' (lib/pregate/rust.ts); in-source
* `#[cfg(test)]` files and `<crate>/tests/` files are the tests.
* Dart directives (`import`/`export`/`part`, conditional URIs included) resolve
* `package:<name>/x` through every tracked pubspec.yaml to `<root>/lib/x`.
* Uncertainty selects more tests, never fewer: a dependency on a directory
@@ -25,6 +28,7 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { matchesAny } from '../ship-policy';
import { djangoEdges } from './django';
import { rustEdges } from './rust';
export const SOURCE_EXT = /\.(?:[cm]?[jt]sx?|py|rb|sh|bash|go|rs|dart)$/;
export const TEST_FILE_RE = /(?:^|\/)(?:test|tests|spec|__tests__)\/.*\.(?:test|spec)\.[cm]?[jt]sx?$|(?:^|\/)[^/]*\.(?:test|spec)\.[cm]?[jt]sx?$|(?:^|\/)test_[^/]*\.py$|_test\.(?:py|go|rb)$|_spec\.rb$|(?:^|\/)[^/]*\.bats$|(?:^|\/)test\/(?:.*\/)?[^/]*_test\.dart$/;
@@ -465,6 +469,9 @@ export function buildGraph(repoRoot: string, opts: BuildOptions = {}): Graph {
for (const { target, via } of referencesIn(files, dirs, rel, source, config, graph.tests.has(rel) ? unresolved : undefined, packages)) addEdge(graph, target, rel, via);
if (unresolved.length) graph.unresolved!.set(rel, unresolved);
}
const rust = rustEdges(all, rel => readRepoFile(repoRoot, rel, maxBytes));
for (const t of rust.tests) graph.tests.add(t);
for (const e of rust.edges) addEdge(graph, e.dependency, e.dependent, e.via);
const django = djangoEdges({ files: [...files], tests: graph.tests, read: rel => pySources.get(rel) ?? readRepoFile(repoRoot, rel, maxBytes), code: pythonCode, resolveDotted: dotted => pythonPatchTarget(files, dotted) });
for (const e of django) addEdge(graph, e.dependency, e.dependent, 'framework');
for (const test of graph.tests) {
lib/pregate/init-draft.ts
+3 -2
View File
@@ -72,13 +72,14 @@ export function draftConfig(repoRoot: string): InitDraft {
}
const groups = new Map<string, string[]>();
for (const t of tests) { const lang = t.endsWith('.dart') ? 'dart' : t.endsWith('.py') ? 'python' : /\.[cm]?[jt]sx?$/.test(t) ? 'javascript' : 'other'; groups.set(lang, [...(groups.get(lang) ?? []), t]); }
for (const t of tests) { const lang = t.endsWith('.dart') ? 'dart' : t.endsWith('.rs') ? 'rust' : t.endsWith('.py') ? 'python' : /\.[cm]?[jt]sx?$/.test(t) ? 'javascript' : 'other'; groups.set(lang, [...(groups.get(lang) ?? []), t]); }
const notes = [...groups].map(([lang, list]) => {
if (lang === 'dart') return `${list.length} Dart test file(s) run per package with flutter test or dart test`;
if (lang === 'rust') return `${list.length} Rust test file(s) run per crate with cargo test -p`;
const runner = detectRunner({ repoRoot, config: DEFAULT_CONFIG }, list);
return runner ? `${lang} tests detected at run time: ${runner}` : `no runner detected for ${list.length} ${lang} test file(s); set runner (with {files})`;
});
const mixed = [...groups.keys()].filter(l => l !== 'dart').length > 1 ? '; a selection mixing these languages needs runner set' : '';
const mixed = [...groups.keys()].filter(l => l !== 'dart' && l !== 'rust').length > 1 ? '; a selection mixing these languages needs runner set' : '';
const runnerNote = notes.length ? notes.join('; ') + mixed : 'no test files found';
const selectionTodo = discoverLanes(repoRoot).filter(l => l.runsTests && l.matrixExpression && l.testFiles.length === 0).map(l => l.id);
lib/pregate/rust.ts
+258
View File
@@ -0,0 +1,258 @@
/**
* pregate/rust — Cargo crates for the import graph and the `lanes` runner.
* A crate is the nearest directory holding a Cargo.toml with a `[package]`
* name. Its `.rs` files are tests when they sit under `<crate>/tests/` or hold
* a `#[cfg(test)]` (in-source unit tests). Cargo runs tests per crate, never
* per file, so the graph maps at crate level: every `.rs` file of a crate,
* and its Cargo.toml, select that crate's tests and the tests of every crate
* that depends on it, transitively.
* Manifests are parsed as TOML (Bun.TOML, as lib/model-policy.ts does), so
* dotted keys, `[dependencies.<name>]` tables, target-specific tables and
* `{ workspace = true }` inheritance read as Cargo reads them. A dependency
* resolves by `path` (relative to the manifest declaring it) to a tracked
* crate, else by package name (`package = ` or the key); a name match only
* widens. The workspace a crate runs in follows Cargo's membership rules:
* `package.workspace`, else the nearest `[workspace]` above it, when that
* workspace is the crate's own manifest, lists it in `members` or reaches it
* by a path dependency, and does not `exclude` it; otherwise the crate
* directory. Nothing here runs cargo: pregate reads files only.
*/
import * as path from 'node:path';
/** Reads a repo-relative file's text, or null when it is absent or unreadable. */
export type ReadFile = (rel: string) => string | null;
type Table = Record<string, unknown>;
const isTable = (v: unknown): v is Table => typeof v === 'object' && v !== null && !Array.isArray(v);
const strings = (v: unknown): string[] => (Array.isArray(v) ? v.filter((x): x is string => typeof x === 'string') : []);
const DEP_TABLES = ['dependencies', 'dev-dependencies', 'build-dependencies'] as const;
/** One dependency: the crate directory its `path` names (repo-relative, normalized) and the package name it asks for. */
interface DepSpec { path: string | null; name: string }
export interface Crate {
root: string;
name: string;
/** Where `cargo test -p <name>` runs: the root of the workspace it is a member of, else the crate directory; null when membership is undecided. */
workspace: string | null;
/** Roots of the tracked crates it depends on. */
deps: string[];
}
export interface CargoModel { crates: Map<string, Crate>; crateOf(file: string): Crate | null }
interface Manifest { dir: string; toml: Table }
const dirOf = (manifest: string) => (path.posix.dirname(manifest) === '.' ? '' : path.posix.dirname(manifest));
/** `dir` joined with a manifest-relative path, normalized; null when it leaves the repository. */
function joinRel(dir: string, rel: string): string | null {
const joined = path.posix.normalize(path.posix.join(dir || '.', rel.replace(/\\/g, '/'))).replace(/\/+$/, '');
if (joined === '..' || joined.startsWith('../') || path.posix.isAbsolute(joined)) return null;
return joined === '.' ? '' : joined;
}
/** One glob component (`*`, `?`, `[...]` with `!`/`^` negation and ranges, else literal) against one path component, by Unicode code point and dynamic programming: O(pattern × name), no regex backtracking. */
function componentMatch(glob: string, name: string): boolean {
const g = Array.from(glob);
const s = Array.from(name);
const nextClose = new Array<number>(g.length + 1).fill(-1);
for (let i = g.length - 1; i >= 0; i--) nextClose[i] = g[i] === ']' ? i : nextClose[i + 1]!;
const tokens: Array<'*' | ((c: string) => boolean)> = [];
for (let i = 0; i < g.length; i++) {
const ch = g[i]!;
const negate = ch === '[' && /[!^]/.test(g[i + 1] ?? '');
const close = ch === '[' ? nextClose[i + (negate ? 3 : 2)] ?? -1 : -1;
if (ch === '*') tokens.push('*');
else if (ch === '?') tokens.push(() => true);
else if (close > 0) {
const body = g.slice(i + (negate ? 2 : 1), close).map(c => c.codePointAt(0)!);
tokens.push(c => {
const p = c.codePointAt(0)!;
let hit = false;
for (let j = 0; j < body.length; j++) {
if (body[j + 1] === 0x2d && j + 2 < body.length) { hit ||= p >= body[j]! && p <= body[j + 2]!; j += 2; } else hit ||= p === body[j];
}
return hit !== negate;
});
i = close;
} else tokens.push(c => c === ch);
}
let row = s.map(() => false).concat(false);
row[0] = true;
for (const t of tokens) {
const next = row.map(() => false);
next[0] = t === '*' && row[0]!;
for (let j = 1; j <= s.length; j++) next[j] = t === '*' ? row[j]! || next[j - 1]! : row[j - 1]! && t(s[j - 1]!);
row = next;
}
return row[s.length]!;
}
/** A Cargo `members` glob against a workspace-relative crate directory, component by component; `**` spans any number of components. */
function memberMatch(glob: string, rel: string): boolean {
const p = rel.split('/').filter(Boolean);
let row = p.map(() => false).concat(false);
row[0] = true;
for (const part of glob.split('/').filter(Boolean)) {
const next = row.map(() => false);
next[0] = part === '**' && row[0]!;
for (let j = 1; j <= p.length; j++) next[j] = part === '**' ? row[j]! || next[j - 1]! : row[j - 1]! && componentMatch(part, p[j - 1]!);
row = next;
}
return row[p.length]!;
}
/**
* Past this many characters of `members` globs a workspace's membership is undecided: every crate under it counts as a
* member for dependency edges (more tests, never fewer), but where its tests run is unknown, so they are unclaimed and the
* lane is incomplete (running from the wrong root can test a same-named member instead).
*/
const MAX_MEMBER_GLOB_CHARS = 4096;
const inside = (dir: string, root: string) => root === '' || dir === root || dir.startsWith(`${root}/`);
const relTo = (dir: string, root: string) => (root === '' ? dir : dir === root ? '' : dir.slice(root.length + 1));
/** Every dependency of a manifest: `[*dependencies]` and `[target.<cfg>.*dependencies]`, `{ workspace = true }` read from `ws`'s `[workspace.dependencies]` (its paths relative to the workspace root). */
function depSpecs(m: Manifest, ws: Manifest | null): DepSpec[] {
const tables: Table[] = [];
for (const key of DEP_TABLES) if (isTable(m.toml[key])) tables.push(m.toml[key] as Table);
if (isTable(m.toml.target)) for (const cfg of Object.values(m.toml.target)) if (isTable(cfg)) for (const key of DEP_TABLES) if (isTable(cfg[key])) tables.push(cfg[key] as Table);
const inherited = ws && isTable(ws.toml.workspace) && isTable(ws.toml.workspace.dependencies) ? ws.toml.workspace.dependencies : {};
const out: DepSpec[] = [];
for (const table of tables) {
for (const [key, raw] of Object.entries(table)) {
const fromWorkspace = isTable(raw) && raw.workspace === true;
const spec = fromWorkspace ? inherited[key] : raw;
const renamed = [spec, raw].find(s => isTable(s) && typeof s.package === 'string') as Table | undefined;
const p = isTable(spec) && typeof spec.path === 'string' ? joinRel(fromWorkspace ? ws!.dir : m.dir, spec.path) : null;
out.push({ path: p, name: (renamed?.package as string | undefined) ?? key });
}
}
return out;
}
/** The tracked crates among `files` (the paths `git ls-files` reports), what each depends on, and the workspace each runs in. */
export function cargoModel(files: readonly string[], read: ReadFile): CargoModel {
const manifests = new Map<string, Manifest>();
for (const f of files) {
if (path.posix.basename(f) !== 'Cargo.toml') continue;
const text = read(f);
if (text === null) continue;
try { const toml = Bun.TOML.parse(text); if (isTable(toml)) manifests.set(dirOf(f), { dir: dirOf(f), toml }); } catch { /* an unparsable manifest names no crate */ }
}
const nameOf = (dir: string) => { const pkg = manifests.get(dir)?.toml.package; return isTable(pkg) && typeof pkg.name === 'string' ? pkg.name : null; };
const roots = [...manifests.keys()].filter(d => nameOf(d) !== null);
const workspaces = [...manifests.values()].filter(m => isTable(m.toml.workspace));
const byName = new Map<string, string[]>();
for (const r of roots) byName.set(nameOf(r)!, [...(byName.get(nameOf(r)!) ?? []), r]);
// `package.workspace` names the workspace; otherwise Cargo consults only the nearest [workspace] at or above the crate.
const candidate = (root: string): Manifest | null => {
const pkg = manifests.get(root)!.toml.package as Table;
if (typeof pkg.workspace === 'string') { const w = manifests.get(joinRel(root, pkg.workspace) ?? '\0'); return w && isTable(w.toml.workspace) ? w : null; }
return workspaces.filter(w => inside(root, w.dir)).sort((a, b) => b.dir.length - a.dir.length)[0] ?? null;
};
const memberCache = new Map<string, Set<string>>();
const undecided = new Set<string>();
const membersOf = (ws: Manifest): Set<string> => {
const cached = memberCache.get(ws.dir);
if (cached) return cached;
const conf = ws.toml.workspace as Table;
const norm = (g: string) => joinRel('', g) ?? '\0';
const globs = strings(conf.members).map(norm);
const exclude = strings(conf.exclude).map(norm);
const own = new Set(roots.filter(r => candidate(r)?.dir === ws.dir));
const excluded = (r: string) => exclude.some(e => { const rel = relTo(r, ws.dir); return rel === e || rel.startsWith(`${e}/`); });
// Cargo's rules: the root package and a literally listed member always belong; a glob match or a path dependency of a
// member inside the workspace belongs unless `exclude` names it or a directory above it.
const literal = (r: string) => globs.some(g => !/[*?[]/.test(g) && relTo(r, ws.dir) === g);
const oversized = globs.join('').length > MAX_MEMBER_GLOB_CHARS;
const members = new Set([...own].filter(r => r === ws.dir || literal(r) || ((oversized || globs.some(g => memberMatch(g, relTo(r, ws.dir)))) && !excluded(r))));
if (oversized) for (const r of members) if (r !== ws.dir && !literal(r)) undecided.add(r);
memberCache.set(ws.dir, members);
const queue = [...members];
while (queue.length) {
for (const dep of depSpecs(manifests.get(queue.shift()!)!, ws)) {
if (dep.path === null || members.has(dep.path) || !own.has(dep.path) || !inside(dep.path, ws.dir) || excluded(dep.path)) continue;
members.add(dep.path);
queue.push(dep.path);
}
}
return members;
};
// A path names one crate; without a path (or when it names no tracked crate) every crate of that package name is a candidate.
const resolve = (specs: DepSpec[]) => [...new Set(specs.flatMap(s => (s.path !== null && nameOf(s.path) !== null ? [s.path] : byName.get(s.name) ?? [])))];
const crates = new Map<string, Crate>();
for (const root of roots) {
const ws = candidate(root);
const member = ws !== null && membersOf(ws).has(root) ? ws : null;
crates.set(root, { root, name: nameOf(root)!, workspace: undecided.has(root) ? null : member?.dir ?? root, deps: resolve(depSpecs(manifests.get(root)!, member)).filter(d => d !== root) });
}
const crateOf = (file: string): Crate | null => {
for (let d = path.posix.dirname(file); ; d = path.posix.dirname(d)) {
const c = crates.get(d === '.' ? '' : d);
if (c) return c;
if (d === '.' || d === '/' || d === '') return null;
}
};
return { crates, crateOf };
}
/**
* Whether Rust source declares tests: a `#[test]` or `#[<path>::test]` attribute (a plain test function, `#[tokio::test]`),
* or a `#[cfg(...)]` whose predicate names `test` outside a string (`cfg(test)`, `cfg(all(test, unix))`; never
* `cfg(feature = "test")`). One forward scan: each attribute is read once to its `]`, and one this scan cannot close
* (an unterminated string, no `]` before the end) counts as a test, which selects more, never fewer.
*/
export function hasCfgTest(source: string): boolean {
if (/#\s*\[\s*(?:[A-Za-z_]\w*\s*::\s*)*test\s*[\]()]/.test(source)) return true;
const open = /#\s*\[\s*cfg\s*\(/g;
for (let m = open.exec(source); m; m = open.exec(source)) {
let predicate = '';
let i = open.lastIndex;
for (; i < source.length && source[i] !== ']'; i++) {
if (source[i] !== '"') { predicate += source[i]; continue; }
const close = source.indexOf('"', i + 1);
if (close < 0) return true;
predicate += '""';
i = close;
}
if (i >= source.length) return true;
const body = predicate.trimEnd();
if (body.endsWith(')') && /\btest\b/.test(body.slice(0, -1))) return true;
open.lastIndex = i + 1;
}
return false;
}
export interface RustGraph {
tests: string[];
edges: Array<{ dependency: string; dependent: string; via: 'import' | 'convention' }>;
}
/**
* The Rust part of the reverse graph over the tracked `files`: each crate's
* test files, an edge from every `.rs` file of a crate to its Cargo.toml (the
* crate's node), and an edge from each Cargo.toml to the tests of its crate
* (`convention`) and of every crate depending on it, transitively (`import`).
*/
export function rustEdges(files: readonly string[], read: ReadFile): RustGraph {
const { crates, crateOf } = cargoModel(files, read);
const out: RustGraph = { tests: [], edges: [] };
if (!crates.size) return out;
const testsOf = new Map<string, string[]>([...crates.keys()].map(k => [k, []]));
const manifestOf = (root: string) => (root ? `${root}/Cargo.toml` : 'Cargo.toml');
for (const f of files) {
if (!f.endsWith('.rs')) continue;
const crate = crateOf(f);
if (!crate) continue;
out.edges.push({ dependency: f, dependent: manifestOf(crate.root), via: 'convention' });
if (f.startsWith(crate.root ? `${crate.root}/tests/` : 'tests/') || hasCfgTest(read(f) ?? '')) { out.tests.push(f); testsOf.get(crate.root)!.push(f); }
}
const dependents = new Map<string, Set<string>>();
for (const c of crates.values()) for (const d of c.deps) dependents.set(d, (dependents.get(d) ?? new Set()).add(c.root));
for (const root of crates.keys()) {
const reach = new Set<string>([root]);
const queue = [root];
while (queue.length) for (const next of dependents.get(queue.shift()!) ?? []) if (!reach.has(next)) { reach.add(next); queue.push(next); }
for (const r of reach) for (const t of testsOf.get(r)!) out.edges.push({ dependency: manifestOf(root), dependent: t, via: r === root ? 'convention' : 'import' });
}
return out;
}
package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "gstack",
"version": "1.91.196",
"version": "1.91.197",
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
"license": "MIT",
"type": "module",
test/pregate-rust.test.ts
+369
View File
@@ -0,0 +1,369 @@
/**
* C8 `lanes` for a Rust repo: a `.rs` file of a crate (the nearest Cargo.toml
* with `[package]`) is a test when it sits under `<crate>/tests/` or holds a
* `#[cfg(test)]`; every `.rs` file of a crate and its Cargo.toml select that
* crate's tests and the tests of every crate depending on it; selected tests
* run as `cargo test -p <name>` from the workspace root, judged on the summed
* `test result:` lines. The end-to-end fixture carries no workflows, so every
* selected test is a local lane on any host platform; `cargo` is a PATH shim.
*/
import { afterEach, describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import { spawnSync } from 'node:child_process';
import * as path from 'node:path';
import { bunIsolationArgs } from '../lib/bun-isolation';
import { EXIT } from '../lib/headless-artifacts';
import { parseSummary, planRuns } from '../lib/pregate/checks/lanes';
import { DEFAULT_CONFIG } from '../lib/pregate/config';
import { buildGraph, testsFor } from '../lib/pregate/import-graph';
import { draftConfig } from '../lib/pregate/init-draft';
import { cargoModel, hasCfgTest, rustEdges } from '../lib/pregate/rust';
import type { PregateContext } from '../lib/pregate/types';
import { cleanup, commit, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/pregate-fixture';
const repos: FixtureRepo[] = [];
afterEach(() => { while (repos.length) cleanup(repos.pop()!); });
const RUST_FILES: Record<string, string> = {
'engine/Cargo.toml': '[workspace]\nresolver = "3"\nmembers = ["crates/*"]\n\n[workspace.dependencies]\nalpha = { path = "crates/alpha" }\n',
'engine/crates/alpha/Cargo.toml': '[package]\nname = "alpha"\nversion = "0.1.0"\nedition = "2024"\n',
'engine/crates/alpha/src/lib.rs': 'pub mod util;\npub fn one() -> u32 { 1 }\n\n#[cfg(test)]\nmod tests {\n #[test]\n fn one_is_one() { assert_eq!(super::one(), 1); }\n}\n',
'engine/crates/alpha/src/util.rs': 'pub fn two() -> u32 { 2 }\n',
'engine/crates/alpha/src/flags.rs': '#[cfg(feature = "test")]\npub fn only_with_the_test_feature() {}\n',
'engine/crates/alpha/tests/it.rs': '#[test]\nfn it() { assert_eq!(alpha::one(), 1); }\n',
'engine/crates/beta/Cargo.toml': '[package]\nname = "beta-core"\nversion = "0.1.0"\n\n[dependencies]\nalpha = { workspace = true } # the shared base\n',
'engine/crates/beta/src/lib.rs': 'pub fn b() -> u32 { alpha::one() }\n\n#[cfg(all(test, unix))]\nmod tests {\n #[test]\n fn b() {}\n}\n',
'engine/crates/gamma/Cargo.toml': '[package]\nname = "gamma"\nversion = "0.1.0"\n\n[target.\'cfg(unix)\'.dev-dependencies.base]\npackage = "beta-core"\npath = "../beta"\n',
'engine/crates/gamma/src/lib.rs': 'pub fn g() {}\n',
'engine/crates/gamma/tests/g.rs': '#[test]\nfn g() {}\n',
'engine/crates/lonely/Cargo.toml': '[package]\nname = "lonely"\nversion = "0.1.0"\n',
'engine/crates/lonely/src/lib.rs': 'pub fn alone() {}\n',
'standalone/Cargo.toml': '[package]\nname = "standalone"\nversion = "0.1.0"\n',
'standalone/src/main.rs': 'fn main() {}\n\n#[cfg(test)]\nmod tests {\n #[test]\n fn t() {}\n}\n',
'tools/tests/loose.rs': 'fn main() {}\n',
};
const ALPHA_TESTS = ['engine/crates/alpha/src/lib.rs', 'engine/crates/alpha/tests/it.rs'];
/** The fixture plus RUST_FILES on main, with no base policy and no workflows, so every selected test runs locally. */
function rustRepo(): FixtureRepo {
const repo = makeFixtureRepo({ pregate: null });
repos.push(repo);
commit(repo, { ...RUST_FILES, '.github/workflows/free-tests.yml': null, '.github/workflows/windows.yml': null }, 'rust');
repo.git('push', '-q', 'origin', 'HEAD:main');
repo.git('fetch', '-q', 'origin');
return repo;
}
const CARGO_OK = [
'echo "ran in $(basename "$PWD"): $*"',
'echo "running 2 tests"',
'echo "test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s"',
'echo " Doc-tests $3"',
'echo "test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s"',
].join('\n');
/**
* A `cargo` on PATH running `body` (default: reports its cwd and argv, then two passed tests and an empty doc-test binary).
* The lane puts its Bun's directory first on PATH, and that directory can hold a real cargo (/usr/local/bin,
* /opt/homebrew/bin), so on POSIX the lane's bash puts the shim first again through BASH_ENV.
*/
function cargoShim(repo: FixtureRepo, body = CARGO_OK): Record<string, string> {
const dir = path.join(repo.root, 'shim');
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(path.join(dir, 'cargo'), `#!/usr/bin/env bash\n${body}\n`, { mode: 0o755 });
fs.writeFileSync(path.join(dir, 'path.sh'), `PATH='${dir}':"$PATH"\n`);
const PATH = [dir, process.env.PATH ?? ''].join(path.delimiter);
return process.platform === 'win32' ? { PATH } : { PATH, BASH_ENV: path.join(dir, 'path.sh') };
}
const rustTests = (tests: Iterable<string>) => [...tests].filter(t => t.endsWith('.rs')).sort();
describe('Rust import graph', () => {
// Value: protects=in-source `#[cfg(test)]` modules and `<crate>/tests/` files are tests only inside a crate, and a cfg naming a
// feature called "test" is not; fails_when=Rust tests are never recognised (every touched .rs is PREGATE_NO_LANE) or any
// tests/*.rs counts without a Cargo.toml; why_new=no Rust support existed; seam=none; no_claim=a Rust parser (a cfg attribute scan)
test('cfg(test) files and <crate>/tests/ files are tests; a loose tests/*.rs and a cfg(feature = "test") file are not', () => {
const g = buildGraph(rustRepo().work);
expect(rustTests(g.tests)).toEqual([...ALPHA_TESTS, 'engine/crates/beta/src/lib.rs', 'engine/crates/gamma/tests/g.rs', 'standalone/src/main.rs']);
});
// Value: protects=a src file without its own tests selects its crate's tests and, transitively, the tests of crates depending on it
// (workspace = true, a renamed target-specific dev-dependency); an in-source test selects itself; a crate with no tests stays
// no-lane; fails_when=the mapping narrows to the file, misses dependents, or invents a lane; why_new=no Rust mapping existed;
// seam=none; no_claim=cargo's own resolver (manifest dependency keys are read)
test('a crate file selects its crate and every crate depending on it; a crate without tests has no lane', () => {
const g = buildGraph(rustRepo().work);
const everyAlphaDependent = [...ALPHA_TESTS, 'engine/crates/beta/src/lib.rs', 'engine/crates/gamma/tests/g.rs'];
expect(testsFor(g, 'engine/crates/alpha/src/util.rs').tests).toEqual(everyAlphaDependent);
expect(testsFor(g, 'engine/crates/alpha/src/lib.rs').tests).toEqual(everyAlphaDependent);
expect(testsFor(g, 'engine/crates/alpha/Cargo.toml').tests).toEqual(everyAlphaDependent);
expect(testsFor(g, 'engine/crates/beta/src/lib.rs').tests).toEqual(['engine/crates/beta/src/lib.rs', 'engine/crates/gamma/tests/g.rs']);
expect(testsFor(g, 'engine/crates/gamma/src/lib.rs').tests).toEqual(['engine/crates/gamma/tests/g.rs']);
expect(testsFor(g, 'standalone/src/main.rs').tests).toEqual(['standalone/src/main.rs']);
expect(testsFor(g, 'engine/crates/lonely/src/lib.rs').tests).toEqual([]);
expect(testsFor(g, 'tools/tests/loose.rs').tests).toEqual([]);
});
// Value: protects=`gstack-pregate init` tells a Rust repo its tests run per crate (no runner to declare), and Rust does not make a
// JS selection "mixed"; fails_when=init tells the user to set a runner for Rust tests, which would override the per-crate plan;
// why_new=init grouped .rs tests as "other"; seam=none; no_claim=none beyond protects
test('init notes that Rust tests run per crate with cargo test, not that a runner must be set', () => {
const note = draftConfig(rustRepo().work).runnerNote;
expect(note).toContain('5 Rust test file(s) run per crate with cargo test -p');
expect(note).not.toContain('needs runner set');
expect(note).not.toMatch(/no runner detected for \d+ (?:other|rust)/);
});
});
describe('Rust runner and summary (pure)', () => {
// Value: protects=cargo prints one `test result:` line per test binary (unit, each integration test, doc-tests); all are summed,
// ignored and filtered tests are not runs, and an all-zero run is ZERO-RUN; fails_when=only the first or last line counts, or the
// generic `N passed` fallback reads cargo output; why_new=cargo output was unparsed (ran: null); seam=none; no_claim=a real cargo
test('cargo test: every test result line counts, doc-tests included; all zeros are a zero run', () => {
const cargo = 'cargo test -p alpha';
const ok = [
' Compiling alpha v0.1.0', ' Running unittests src/lib.rs (target/debug/deps/alpha-1)', '', 'running 2 tests', 'test tests::a ... ok', 'test tests::b ... ok', '',
'test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s', '',
' Running tests/it.rs (target/debug/deps/it-2)', '', 'running 2 tests', 'test it ... ok', 'test slow ... ignored', '',
'test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s', '',
' Doc-tests alpha', '', 'running 3 tests', 'test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.31s',
].join('\n');
expect(parseSummary(ok, cargo)).toEqual({ ran: 6, failed: 0 });
const failed = 'test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s\n\nfailures:\n tests::b\n\ntest result: FAILED. 1 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s\n\nerror: test failed, to rerun pass `-p alpha --lib`';
expect(parseSummary(failed, cargo)).toEqual({ ran: 7, failed: 2 });
const zero = 'running 0 tests\n\ntest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s\n\n Doc-tests alpha\n\nrunning 0 tests\n\ntest result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s';
expect(parseSummary(zero, cargo)).toEqual({ ran: 0, failed: 0 });
expect(parseSummary('\x1b[1m\x1b[32mtest result: ok\x1b[0m. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s', cargo)).toEqual({ ran: 5, failed: 0 });
expect(parseSummary('error[E0425]: cannot find value `x` in this scope\nerror: could not compile `alpha`', cargo)).toEqual({ ran: null, failed: null });
});
// Value: protects=each selected crate runs once as `cargo test -p <name>` from its workspace root (else its own directory), crates
// group apart, a .rs file outside any crate is unclaimed, and a missing cargo is incomplete; fails_when=.rs files reach the JS
// runner, run from the repo root, merge across crates, or a missing cargo is judged as a test failure (exit 127); why_new=no cargo
// runner existed; seam=none; no_claim=the runners themselves (plan only)
test('without a declared runner, Rust tests group by crate from the workspace root; a declared runner takes every file', () => {
const repo = rustRepo();
const ctx = { repoRoot: repo.work, config: DEFAULT_CONFIG, env: cargoShim(repo) } as unknown as PregateContext;
const files = [...ALPHA_TESTS, 'engine/crates/beta/src/lib.rs', 'standalone/src/main.rs', 'tools/tests/loose.rs'];
expect(planRuns(ctx, files)).toEqual({ groups: [
{ runner: "cargo test -p 'alpha'", cwd: 'engine', files: ['crates/alpha/src/lib.rs', 'crates/alpha/tests/it.rs'] },
{ runner: "cargo test -p 'beta-core'", cwd: 'engine', files: ['crates/beta/src/lib.rs'] },
{ runner: "cargo test -p 'standalone'", cwd: 'standalone', files: ['src/main.rs'] },
], unclaimed: ['tools/tests/loose.rs'], missing: [], noPubspec: [] });
const noCargo = { ...ctx, env: { PATH: path.join(repo.root, 'empty-path') } } as PregateContext;
expect(planRuns(noCargo, ALPHA_TESTS)).toEqual({ groups: [], unclaimed: ALPHA_TESTS, missing: ['cargo'], noPubspec: [] });
const declared = { ...ctx, config: { ...DEFAULT_CONFIG, runner: 'bash scripts/test.sh {files}' } } as PregateContext;
expect(planRuns(declared, files).groups).toEqual([{ runner: 'bash scripts/test.sh {files}', cwd: '', files }]);
});
});
/**
* Manifest shapes a line scan misreads (review of the first cut): a renamed
* dependency inherited from `[workspace.dependencies]`, TOML dotted keys, the
* `[dependencies.<name>]` table form, target-specific tables, and a crate the
* workspace excludes. Dependencies resolve by path first, then by package name.
*/
const MANIFEST_FILES: Record<string, string> = {
'Cargo.toml': '[workspace]\nmembers = ["crates/*"]\nexclude = ["crates/helper"]\n\n[workspace.dependencies]\nbase = { package = "alpha", path = "crates/a" }\n',
'crates/a/Cargo.toml': '[package]\nname = "alpha"\nversion = "0.1.0"\n',
'crates/a/src/lib.rs': 'pub fn a() {}\n',
'crates/a/tests/a.rs': '#[test]\nfn a() {}\n',
'crates/beta/Cargo.toml': '[package]\nname = "beta"\nversion = "0.1.0"\n\n[dependencies]\nbase = { workspace = true }\n',
'crates/beta/tests/b.rs': '#[test]\nfn b() {}\n',
'crates/dotted/Cargo.toml': '[package]\nname = "dotted"\nversion = "0.1.0"\n\n[dependencies]\nfoo.path = "../a"\n',
'crates/dotted/tests/d.rs': '#[test]\nfn d() {}\n',
'crates/table/Cargo.toml': '[package]\nname = "table"\nversion = "0.1.0"\n\n[dev-dependencies.whatever]\npath = "../a"\n',
'crates/table/tests/t.rs': '#[test]\nfn t() {}\n',
'crates/targeted/Cargo.toml': '[package]\nname = "targeted"\nversion = "0.1.0"\n\n[target.\'cfg(unix)\'.build-dependencies]\nrenamed = { path = "../a" }\n',
'crates/targeted/tests/x.rs': '#[test]\nfn x() {}\n',
'crates/plain/Cargo.toml': '[package]\nname = "plain"\nversion = "0.1.0"\n',
'crates/plain/tests/p.rs': '#[test]\nfn p() {}\n',
'crates/helper/Cargo.toml': '[package]\nname = "helper"\nversion = "0.1.0"\n',
'crates/helper/src/lib.rs': '#[cfg(test)]\nmod tests {\n #[test]\n fn h() {}\n}\n',
};
function manifestRepo(): FixtureRepo {
const repo = makeFixtureRepo({ pregate: null });
repos.push(repo);
commit(repo, MANIFEST_FILES, 'cargo manifests');
return repo;
}
describe('Cargo manifests are parsed as TOML', () => {
// Value: protects=every dependency spelling Cargo accepts links a crate to its dependents: an inherited renamed workspace dependency,
// a dotted `foo.path` key, a `[dev-dependencies.<name>]` table and a target-specific build dependency, all resolved by path;
// fails_when=the manifest is read by a line scan again and alpha's change misses these dependents (a false green);
// why_new=the first cut matched dependency keys against package names only; seam=none; no_claim=cargo's own resolver
test('renamed, inherited, dotted, table-form and target-specific dependencies all select the dependent crates', () => {
const g = buildGraph(manifestRepo().work);
expect(testsFor(g, 'crates/a/src/lib.rs').tests).toEqual(['crates/a/tests/a.rs', 'crates/beta/tests/b.rs', 'crates/dotted/tests/d.rs', 'crates/table/tests/t.rs', 'crates/targeted/tests/x.rs']);
});
// Value: protects=a crate the workspace excludes runs from its own directory, a member from the workspace root; fails_when=the
// nearest [workspace] is taken without checking members/exclude and `cargo test -p helper` runs where helper is no member
// (cargo fails, or tests a same-named member); why_new=membership was not read; seam=none; no_claim=a real cargo (plan only)
test('an excluded crate runs from its own directory; a member runs from the workspace root', () => {
const repo = manifestRepo();
const ctx = { repoRoot: repo.work, config: DEFAULT_CONFIG, env: cargoShim(repo) } as unknown as PregateContext;
expect(planRuns(ctx, ['crates/beta/tests/b.rs', 'crates/helper/src/lib.rs']).groups).toEqual([
{ runner: "cargo test -p 'beta'", cwd: '', files: ['crates/beta/tests/b.rs'] },
{ runner: "cargo test -p 'helper'", cwd: 'crates/helper', files: ['src/lib.rs'] },
]);
});
});
describe('Cargo membership and test detection (port review)', () => {
const inMemory = (files: Record<string, string>) => [Object.keys(files), (f: string) => files[f] ?? null] as const;
const pkg = (name: string, deps = '') => `[package]\nname = "${name}"\nversion = "0.1.0"\n${deps}`;
const dependentsOf = (files: Record<string, string>, manifest: string) => rustEdges(...inMemory(files)).edges.filter(e => e.dependency === manifest).map(e => e.dependent).sort();
// Value: protects=Cargo's member globs ([ab] classes, ** spans) admit the crates Cargo admits, so a member inherits
// [workspace.dependencies] and its tests are selected; fails_when=a character class is matched literally, the crate is
// no member, the inherited path dependency is dropped and alpha's change misses beta's tests (a false green);
// why_new=port review found classes escaped; seam=none; no_claim=cargo's own glob crate
test('a members glob with a character class or ** admits the crate, so an inherited dependency links its dependent', () => {
const files = {
'Cargo.toml': '[workspace]\nmembers = ["crates/[ab]", "nested/**/leaf", "unicode/?"]\n\n[workspace.dependencies]\nbase = { package = "alpha", path = "crates/a" }\n',
'crates/a/Cargo.toml': pkg('alpha'), 'crates/a/tests/a.rs': '#[test]\nfn a() {}\n',
'crates/b/Cargo.toml': pkg('beta', '\n[dependencies]\nbase = { workspace = true }\n'), 'crates/b/tests/b.rs': '#[test]\nfn b() {}\n',
'crates/c/Cargo.toml': pkg('gamma'),
'nested/x/y/leaf/Cargo.toml': pkg('leaf'),
'unicode/📦/Cargo.toml': pkg('boxed'),
};
const { crates } = cargoModel(...inMemory(files));
expect(['crates/a', 'crates/b', 'crates/c', 'nested/x/y/leaf', 'unicode/📦'].map(r => crates.get(r)!.workspace)).toEqual(['', '', 'crates/c', '', '']);
expect(dependentsOf(files, 'crates/a/Cargo.toml')).toEqual(['crates/a/tests/a.rs', 'crates/b/tests/b.rs']);
});
// Value: protects=a plain #[test] function or a #[tokio::test] in src is a Rust test, so a dependent crate tested that way
// runs when its dependency changes; fails_when=only cfg(test) modules and tests/ count and the dependent's failing test
// never runs (a false green); why_new=port review; seam=none; no_claim=custom [[test]] targets outside tests/
test('a plain #[test] or #[tokio::test] in src is a test; cfg(feature = "test") and #[testing] are not', () => {
expect(['#[test]\nfn a() {}', '#[tokio::test(flavor = "multi_thread")]\nasync fn a() {}', '#[ test ]\nfn a() {}'].map(hasCfgTest)).toEqual([true, true, true]);
expect(['#[cfg(feature = "test")]\nfn a() {}', '#[testing]\nfn a() {}', '// attests', '#[cfg(feature = "a]b")]\nfn a() {}'].map(hasCfgTest)).toEqual([false, false, false, false]);
expect(hasCfgTest(`#[cfg(any(test, feature = "${'f'.repeat(300)}"))]\nmod t {}`)).toBe(true);
const files = {
'a/Cargo.toml': pkg('a'), 'a/src/lib.rs': 'pub fn one() -> u32 { 1 }\n#[cfg(test)]\nmod t { #[test] fn one() {} }\n',
'b/Cargo.toml': pkg('b', '\n[dependencies]\na = { path = "../a" }\n'), 'b/src/lib.rs': 'pub fn two() -> u32 { a::one() + 1 }\n#[test]\nfn two() { assert_eq!(super::two(), 2); }\n',
};
expect(dependentsOf(files, 'a/Cargo.toml')).toEqual(['a/src/lib.rs', 'b/src/lib.rs']);
});
// Value: protects=past the members-glob size cap, where a crate's tests run is undecided, so its files are unclaimed (the
// lane incomplete) instead of running `cargo test -p <name>` from a root where a same-named member answers; fails_when=an
// oversized workspace admits every crate as runnable and the orphan's failing test is replaced by the member's passing
// ones (a false green); why_new=third port review round; seam=none; no_claim=a real cargo (plan only)
test('past the members size cap a same-named non-member is unclaimed, not run as the member', () => {
const repo = makeFixtureRepo({ pregate: null });
repos.push(repo);
commit(repo, {
'Cargo.toml': `[workspace]\nmembers = ["[${'m'.repeat(4100)}]ember"]\n`,
'member/Cargo.toml': pkg('same'), 'member/tests/m.rs': '#[test]\nfn m() {}\n',
'orphan/Cargo.toml': pkg('same'), 'orphan/tests/t.rs': '#[test]\nfn t() { panic!() }\n',
}, 'oversized members');
const ctx = { repoRoot: repo.work, config: DEFAULT_CONFIG, env: cargoShim(repo) } as unknown as PregateContext;
expect(planRuns(ctx, ['orphan/tests/t.rs'])).toEqual({ groups: [], unclaimed: ['orphan/tests/t.rs'], missing: [], noPubspec: [] });
});
// Value: protects=a hostile members glob or a file of unclosed cfg attributes cannot stall lane selection, which runs before
// any runner deadline; each probe runs in its own supervised Bun so a regression is killed and fails instead of hanging the
// file; fails_when=member globs backtrack again (an exponential '*a*a…b', or a rescan per unmatched '[') or the cfg scan
// rescans from every start (quadratic: about 80 s on 600 KB, against tens of ms now); why_new=port review; seam=none;
// no_claim=a wall-clock budget for selection (the 30 s kill only separates linear from super-linear)
const supervised = (code: string) => spawnSync(process.execPath, [...bunIsolationArgs(), '-e', `import { cargoModel, hasCfgTest } from ${JSON.stringify(path.join(import.meta.dir, '..', 'lib', 'pregate', 'rust.ts'))};\n${code}`], { encoding: 'utf8', timeout: 30_000 });
test('a hostile members glob is matched without backtracking', () => {
const r = supervised(`
const pkg = n => '[package]\\nname = "' + n + '"\\nversion = "0.1.0"\\n';
const run = (members, dirs) => { const files = { 'Cargo.toml': '[workspace]\\nmembers = ' + JSON.stringify(members) + '\\n' }; dirs.forEach((d, i) => { files[d + '/Cargo.toml'] = pkg('k' + i); }); return cargoModel(Object.keys(files), f => files[f] ?? null); };
const exp = run(['*a'.repeat(20) + 'b'], ['a'.repeat(40)]).crates.get('a'.repeat(40)).workspace;
const brackets = run(['['.repeat(1000000)], Array.from({ length: 10 }, (_, i) => 'c' + i)).crates.get('c0').workspace;
console.log(JSON.stringify([exp, brackets]));`);
expect(r.error).toBeUndefined();
expect(r.stdout.trim()).toBe(JSON.stringify(['a'.repeat(40), null]));
}, 60_000);
test('a run of unclosed cfg attributes is scanned once and counts as a test', () => {
const r = supervised(`console.log(JSON.stringify([hasCfgTest('#[cfg('.repeat(100000)), hasCfgTest('#[cfg(unix)] fn a() {}\\n'.repeat(50000))]));`);
expect(r.error).toBeUndefined();
expect(r.stdout.trim()).toBe('[true,false]');
}, 60_000);
});
describe('gstack-pregate on a Rust workspace', () => {
const run = (repo: FixtureRepo, env: Record<string, string>) => runBin(repo, 'gstack-pregate', ['--base', 'main', '--allow-repo-commands', '--stage', 'tests', '--only', 'lanes'], env);
// The cargo stand-in is a bash script the lane's Git Bash must find before the runner's real cargo; on windows-latest it
// never did (three PATH repairs, the lane still exited 1 with an empty log), so the two cases that run it are POSIX-only,
// like test/pregate-dart.test.ts's stand-ins. Selection, planning and the hostile-name case still run on Windows.
const posixShim = test.skipIf(process.platform === 'win32');
const log = (out: string) => fs.readFileSync(/log: (\S+)/.exec(out)![1]!, 'utf8');
// Value: protects=the reported case end to end: a touched src file of a workspace crate is not PREGATE_NO_LANE, its crate and the
// dependent crates run with `cargo test -p` from the workspace root, and the summed count is reported; fails_when=any link of
// graph, plan or summary regresses; why_new=no Rust run existed; seam=none; no_claim=a real cargo (PATH shim reports cwd and argv)
posixShim('a touched src file runs its crate and the crates depending on it from the workspace root', () => {
const repo = rustRepo();
commit(repo, { 'engine/crates/alpha/src/util.rs': 'pub fn two() -> u32 { 1 + 1 }\n' });
const r = run(repo, cargoShim(repo));
expect(r.out).not.toContain('has no test lane');
expect(r.out).toMatch(/^tests {6}lanes {5}pass .*6 tests ran in 4 file\(s\), exit 0/m);
expect(r.out).toContain("ran: (cd 'engine' && cargo test -p 'alpha') && (cd 'engine' && cargo test -p 'beta-core') && (cd 'engine' && cargo test -p 'gamma')");
expect(log(r.out)).toContain('ran in engine: test -p alpha');
expect(r.status).toBe(EXIT.ok);
}, 60_000);
// Value: protects=a crate whose test binaries all report 0 tests is ZERO-RUN (exit 0 is not a pass), and a failing cargo run is
// PREGATE_LANE_FAILED with its counts; fails_when=cargo's summary is unparsed (count unavailable passes) or the exit is ignored;
// why_new=the pure parser test does not run the lane; seam=none; no_claim=a real cargo (PATH shim)
posixShim('zero tests is ZERO-RUN; a failing cargo run is PREGATE_LANE_FAILED', () => {
const repo = rustRepo();
commit(repo, { 'engine/crates/gamma/src/lib.rs': 'pub fn g() { }\n' });
const zero = run(repo, cargoShim(repo, 'echo "test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s"'));
expect(zero.status).not.toBe(EXIT.ok);
expect(zero.out).toMatch(/^tests {6}lanes {5}fail .*ZERO-RUN: the runner reported 0 tests in \(cd 'engine' && cargo test -p 'gamma'\)/m);
const red = run(repo, cargoShim(repo, 'echo "test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s"\nexit 101'));
expect(red.status).not.toBe(EXIT.ok);
expect(red.out).toMatch(/^tests {6}lanes {5}fail .*local lane exited 101 \(1 failed of 2 ran; 1 files; pipefail\)/m);
}, 60_000);
// Value: protects=a crate name is data from the repo's Cargo.toml that reaches the lane's shell and the {files} expansion,
// so a name Cargo would reject never runs (its files are unclaimed, the lane incomplete) and a valid one, Unicode
// included, still runs quoted; fails_when=the name is interpolated unchecked (`;touch` runs a command, `{files}` splices
// the quoted file list into the name); why_new=the crate runner was new; seam=none; no_claim=a real cargo (PATH shim)
test('a crate name with shell syntax or {files} never reaches the shell; a valid Unicode name still runs', () => {
const repo = rustRepo();
const lib = '#[cfg(test)]\nmod tests {\n #[test]\n fn t() {}\n}\n';
commit(repo, {
'engine/crates/evil/Cargo.toml': '[package]\nname = "evil;touch pwned-by-crate-name"\nversion = "0.1.0"\n',
'engine/crates/evil/src/lib.rs': lib,
'engine/crates/splice/Cargo.toml': '[package]\nname = "evil{files}"\nversion = "0.1.0"\n',
'engine/crates/splice/src/lib.rs': lib,
'engine/crates/uni/Cargo.toml': '[package]\nname = "naïve_crate-2"\nversion = "0.1.0"\n',
'engine/crates/uni/src/lib.rs': lib,
}, 'odd crate names');
repo.git('push', '-q', 'origin', 'HEAD:main');
repo.git('fetch', '-q', 'origin');
const env = cargoShim(repo);
const ctx = { repoRoot: repo.work, config: DEFAULT_CONFIG, env } as unknown as PregateContext;
expect(planRuns(ctx, ['engine/crates/evil/src/lib.rs', 'engine/crates/splice/src/lib.rs', 'engine/crates/uni/src/lib.rs'])).toEqual({
groups: [{ runner: "cargo test -p 'naïve_crate-2'", cwd: 'engine', files: ['crates/uni/src/lib.rs'] }],
unclaimed: ['engine/crates/evil/src/lib.rs', 'engine/crates/splice/src/lib.rs'], missing: [], noPubspec: [],
});
commit(repo, { 'engine/crates/evil/src/lib.rs': lib.replace('fn t() {}', 'fn t() { }') });
const r = run(repo, env);
expect(fs.existsSync(path.join(repo.work, 'engine', 'pwned-by-crate-name'))).toBe(false);
expect(r.out).not.toContain('cargo test -p');
expect(r.out).toContain('engine/crates/evil/src/lib.rs');
expect(r.status).not.toBe(EXIT.ok);
}, 60_000);
// Value: protects=a crate with no tests still fails closed (no lane), so the crate mapping never invents coverage;
// fails_when=crate membership alone counts as a lane; why_new=negative control for the new mapping; seam=none; no_claim=none
test('a touched file of a crate without tests still has no test lane', () => {
const repo = rustRepo();
commit(repo, { 'engine/crates/lonely/src/lib.rs': 'pub fn alone() { }\n' });
const r = run(repo, cargoShim(repo));
expect(r.status).not.toBe(EXIT.ok);
expect(r.out).toContain('FAIL lanes — engine/crates/lonely/src/lib.rs has no test lane');
}, 60_000);
});
test/pregate-selection.test.ts
+85 -2
View File
@@ -14,11 +14,12 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { EXIT } from '../lib/headless-artifacts';
import { changedLiterals, constantsIn } from '../lib/pregate/checks/literals';
import { parseSummary } from '../lib/pregate/checks/lanes';
import { validateConfig } from '../lib/pregate/config';
import { parseSummary, planRuns } from '../lib/pregate/checks/lanes';
import { DEFAULT_CONFIG, validateConfig } from '../lib/pregate/config';
import { isDocsEmptyLane } from '../lib/pregate/docs-lane';
import { buildGraph, testsFor } from '../lib/pregate/import-graph';
import { CHECK_IDS } from '../lib/pregate/index';
import type { PregateContext } from '../lib/pregate/types';
import { parseWorkflow } from '../lib/pregate/workflows';
import { BASE_PREGATE, cleanup, commit, makeFixtureRepo, runBin, write, type FixtureRepo } from './helpers/pregate-fixture';
@@ -564,3 +565,85 @@ describe('gstack-pregate (bin)', () => {
expect(runBin(repo, 'gstack-pregate', ['validate']).status).toBe(EXIT.ok);
});
});
describe('nested JS packages run their own runner from their own directory', () => {
const APP = {
'app/package.json': JSON.stringify({ name: 'app', scripts: { test: 'vitest run' }, devDependencies: { vitest: '^3.2.0' } }) + '\n',
'app/pnpm-lock.yaml': "lockfileVersion: '9.0'\n",
'app/src/sum.ts': 'export const sum = (a: number, b: number) => a + b;\n',
'app/src/sum.test.ts': "import { expect, test } from 'vitest';\nimport { sum } from './sum';\ntest('sum', () => expect(sum(1, 2)).toBe(3));\n",
};
/**
* A `pnpm` (and `yarn`) on PATH that reports its cwd and argv and prints a vitest summary for the files it is given.
* The lane puts its Bun's directory first on PATH, and that directory can hold a real pnpm (/usr/local/bin), which
* would install from the network; the lane's bash puts the shim first again through BASH_ENV.
*/
const pmShim = (repo: FixtureRepo): { PATH: string; BASH_ENV: string } => {
const dir = path.join(repo.root, 'shim');
fs.mkdirSync(dir, { recursive: true });
for (const tool of ['pnpm', 'yarn']) fs.writeFileSync(path.join(dir, tool), '#!/usr/bin/env bash\necho "ran in $(basename "$PWD"): $*"\necho " Test Files 1 passed (1)"\necho " Tests 3 passed (3)"\n', { mode: 0o755 });
fs.writeFileSync(path.join(dir, 'path.sh'), `PATH='${dir}':"$PATH"\n`);
return { PATH: `${dir}:${process.env.PATH ?? ''}`, BASH_ENV: path.join(dir, 'path.sh') };
};
const ctxFor = (repo: FixtureRepo, env: Record<string, string>) => ({ repoRoot: repo.work, config: DEFAULT_CONFIG, env }) as unknown as PregateContext;
// Value: protects=a test owned by a nested package that declares vitest runs from that package with its lockfile's package manager
// and package-relative paths, while root-owned tests keep the root runner; fails_when=the root package.json alone decides again
// (the reported `bun test` from the repo root: vi.stubGlobal missing); why_new=only root package.json detection was tested;
// seam=none; no_claim=the runners themselves (plan only)
test('a nested vitest package plans its own group with the lockfile\'s package manager; root tests keep the root runner', () => {
const repo = fixture({ pregate: null, baseFiles: APP });
const ctx = ctxFor(repo, pmShim(repo));
expect(planRuns(ctx, ['app/src/sum.test.ts', 'test/util.test.js'])).toEqual({ groups: [
{ runner: 'node --test {files}', cwd: '', files: ['test/util.test.js'] },
{ runner: 'pnpm exec vitest run {files}', cwd: 'app', files: ['src/sum.test.ts'] },
], unclaimed: [], missing: [], noPubspec: [] });
// The lockfile is found walking up, and a packageManager field outranks it.
fs.rmSync(path.join(repo.work, 'app/pnpm-lock.yaml'));
write(repo.work, 'yarn.lock', '# yarn lockfile v1\n');
expect(planRuns(ctx, ['app/src/sum.test.ts']).groups).toEqual([{ runner: 'yarn vitest run {files}', cwd: 'app', files: ['src/sum.test.ts'] }]);
write(repo.work, 'app/package.json', JSON.stringify({ name: 'app', packageManager: 'pnpm@9.15.0', devDependencies: { jest: '^29.0.0' } }) + '\n');
expect(planRuns(ctx, ['app/src/sum.test.ts']).groups).toEqual([{ runner: 'pnpm exec jest {files}', cwd: 'app', files: ['src/sum.test.ts'] }]);
// Value: protects=a package manager missing from PATH is incomplete naming it, never exit 127 judged as a test failure;
// fails_when=the PATH check is dropped; why_new=new detection path; seam=none; no_claim=a real pnpm install
expect(planRuns(ctxFor(repo, { PATH: path.join(repo.root, 'empty-path') }), ['app/src/sum.test.ts'])).toEqual({ groups: [], unclaimed: ['app/src/sum.test.ts'], missing: ['pnpm'], noPubspec: [] });
fs.rmSync(path.join(repo.work, 'yarn.lock'));
write(repo.work, 'app/package.json', JSON.stringify({ name: 'app', devDependencies: { vitest: '^3.2.0' } }) + '\n');
expect(planRuns(ctx, ['app/src/sum.test.ts']).groups).toEqual([{ runner: 'npx vitest run {files}', cwd: 'app', files: ['src/sum.test.ts'] }]);
});
// Value: protects=repos where root detection works today are unchanged: a nested package that declares no test runner stays with the
// root runner from the root, a root vitest stays `npx vitest run` from the root, and a declared runner takes every file;
// fails_when=every nested package.json splits a group or the root command changes; why_new=negative controls for the new
// nested detection; seam=none; no_claim=the runners themselves (plan only)
test('negative controls: a nested package without a test runner, a root vitest repo and a declared runner keep today\'s plan', () => {
const repo = fixture({ pregate: null, baseFiles: { ...APP, 'app/package.json': JSON.stringify({ name: 'app', scripts: { build: 'tsc' } }) + '\n' } });
const ctx = ctxFor(repo, pmShim(repo));
write(repo.work, 'bun.lock', '{}\n');
expect(planRuns(ctx, ['app/src/sum.test.ts', 'test/util.test.js']).groups).toEqual([{ runner: 'bun test {files}', cwd: '', files: ['app/src/sum.test.ts', 'test/util.test.js'] }]);
fs.rmSync(path.join(repo.work, 'bun.lock'));
write(repo.work, 'package.json', JSON.stringify({ name: 'fixture', devDependencies: { vitest: '^3.2.0' } }) + '\n');
write(repo.work, 'pnpm-lock.yaml', "lockfileVersion: '9.0'\n");
expect(planRuns(ctx, ['app/src/sum.test.ts', 'test/util.test.js']).groups).toEqual([{ runner: 'npx vitest run {files}', cwd: '', files: ['app/src/sum.test.ts', 'test/util.test.js'] }]);
const declared = { ...ctx, config: { ...DEFAULT_CONFIG, runner: 'bun test {files}' } } as PregateContext;
write(repo.work, 'app/package.json', APP['app/package.json']);
expect(planRuns(declared, ['app/src/sum.test.ts']).groups).toEqual([{ runner: 'bun test {files}', cwd: '', files: ['app/src/sum.test.ts'] }]);
});
// Value: protects=the reported case end to end: a touched source file in a nested pnpm+vitest package runs `pnpm exec vitest run`
// in that package with the package-relative path and passes on vitest's summary; fails_when=the lane runs from the repo root or
// with the root runner; why_new=no nested package run existed; seam=none; no_claim=a real pnpm or vitest (PATH shim)
test('a touched file in a nested pnpm+vitest package runs vitest there', () => {
const repo = fixture({ pregate: null, baseFiles: APP });
// No workflows, so the Windows lane's unknown matrix cannot turn the selection into a requires-remote obligation.
commit(repo, { '.github/workflows/free-tests.yml': null, '.github/workflows/windows.yml': null }, 'no workflows');
repo.git('push', '-q', 'origin', 'HEAD:main');
repo.git('fetch', '-q', 'origin');
commit(repo, { 'app/src/sum.ts': 'export const sum = (a: number, b: number) => b + a;\n' });
const r = runBin(repo, 'gstack-pregate', ['--base', 'main', '--allow-repo-commands', '--stage', 'tests', '--only', 'lanes'], pmShim(repo));
expect(r.out).toMatch(/^tests {6}lanes {5}pass .*3 tests ran in 1 file\(s\), exit 0/m);
expect(r.out).toContain("ran: (cd 'app' && pnpm exec vitest run 'src/sum.test.ts')");
expect(fs.readFileSync(/log: (\S+)/.exec(r.out)![1]!, 'utf8')).toContain('ran in app: exec vitest run src/sum.test.ts');
expect(r.status).toBe(EXIT.ok);
}, 60_000);
});