Files
gstack/lib/pregate/import-graph.ts
T
c739dfa494 v1.91.197.0 fix(pregate): Rust crate lanes and nested JS package runners (#3344, port of #3346) (#3380)
* fix(pregate): Rust crate lanes and nested JS package runners

lanes reported PREGATE_NO_LANE for every touched .rs file and ran a nested
pnpm+vitest package's tests as `bun test` from the repo root.

- Rust: a .rs file of a crate (nearest Cargo.toml with [package]) is a test
  under <crate>/tests/ or when it holds a #[cfg(test)]; every .rs file of a
  crate and its Cargo.toml select the crate's tests and those of every crate
  depending on it (lib/pregate/rust.ts). Selected tests run per crate as
  `cargo test -p <name>` from the workspace root, judged on the summed
  `test result:` lines (doc-tests included); zero tests stays ZERO-RUN and a
  missing cargo is incomplete, like a missing Dart SDK.
- JS/TS: a test owned by a nested package.json that declares vitest or jest
  runs from that package with package-relative paths, through the package
  manager its packageManager field or nearest lockfile names (pnpm exec,
  yarn, bun x, else npx). Root detection and a declared runner are unchanged.
- init notes that Rust tests run per crate; docs/pregate.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pregate): parse Cargo manifests as TOML; follow workspace membership

The line scan of Cargo.toml missed dependents (a false green) and picked the
wrong workspace:
- an inherited renamed dependency (`base = { workspace = true }` with
  `[workspace.dependencies] base = { package = "alpha", path = ... }`);
- dotted keys (`alpha.path = "../a"`), `[dependencies.<name>]` tables and
  target-specific tables, matched by key name only;
- a crate the workspace excludes ran `cargo test -p` from the workspace root.

Manifests are now parsed with Bun.TOML. A dependency resolves by path
(relative to the manifest declaring it, or to the workspace root when
inherited), else by package name. A crate runs from the workspace it is a
member of under Cargo's rules (package.workspace, the nearest [workspace],
members globs, exclude, literal members, path dependencies of members),
else from its own directory. Nothing runs cargo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(pregate): the Rust fixture's cargo shim wins PATH on Windows and beside a real cargo

windows-free-shard (5) failed 'a touched src file runs its crate...' and
'zero tests is ZERO-RUN...' deterministically: the shim PATH was joined with
':' so Git Bash never saw the shim dir, and the runner's real cargo ran
against the fixture (exit 1, no test result lines). On POSIX hosts where
Bun and cargo share a directory (/usr/local/bin on Capy's image), the lane
puts Bun's directory first and the real cargo wins the same way.

Join with path.delimiter, add cargo.cmd for Bun.which on Windows, and on
POSIX put the shim first again in the lane's own bash through BASH_ENV.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): quote the crate name in cargoRunner

The crate name is read from the repo's Cargo.toml and was interpolated into
'cargo test -p <name>', which the lane runs through bash. A name like
'evil;touch pwned' ran the second command. Quote it like every file and cwd
the lane already passes; a new end-to-end case checks the name reaches cargo
as one argument and runs nothing.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(pregate): the nested-package pnpm shim wins PATH beside a real pnpm

'a touched file in a nested pnpm+vitest package runs vitest there' ran the
host's real pnpm wherever Bun and pnpm share a directory (/usr/local/bin on
Capy's image): the lane puts Bun's directory first, so pnpm installed vitest
from the network, dirtied the fixture tree and the check came back
incomplete. Put the shim first again in the lane's bash through BASH_ENV,
as the Rust fixture does.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): pregate Rust crate lanes and nested package runners (#3344)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): a crate name Cargo would reject never reaches the lane's shell

Port review (outside voice): quoting alone was not enough, because the lane
expands {files} after the runner string is built, so a crate named
'evil{files}' spliced the quoted file list into the quoted name and a file
name like tests/$(cmd).rs then ran. A name outside Cargo's grammar
(letters, digits, _ and -) now gets no runner: its files are unclaimed and
the lane is incomplete, as Cargo itself would refuse the manifest. A valid
name, Unicode included, still runs quoted.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): match Cargo members globs per path component, without a backtracking regex

Port review (outside voice): members globs went through ship-policy's
matchesAny, which escapes [...] classes (crates/[ab] matched nothing, so the
crate was no member, lost its inherited [workspace.dependencies] path and its
dependents' tests were never selected: a false green) and backtracks
exponentially on a hostile glob such as '*a*a...*ab' (over 5 s for one crate,
before any runner deadline). A small dynamic-programming matcher handles *,
?, [..] with negation and ranges, and ** across components, in
O(pattern x path).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): a plain #[test] in src is a Rust test; bound the cfg attribute scan

Port review (outside voice): only #[cfg(test)] modules and tests/ files
counted, so a dependent crate whose tests are plain #[test] or #[tokio::test]
functions in src never ran when its dependency changed (a false green). The
cfg scan also went quadratic on a run of unclosed '#[cfg(' (about 3 s on
120 KB). Test attributes now count, and the cfg predicate and its strings are
length-bounded so the scan stays linear.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(pregate): Rust test attributes, workspace membership and rejected crate names; changelog for the port review fixes

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): match members globs by code point in one pass, and admit every crate past a size cap

Second port review round (outside voice): the matcher counted UTF-16 units, so
'crates/?' missed a crate directory named with a supplementary character and
dropped its inherited dependencies (a false green); and each unmatched '['
rescanned the rest of the glob (5 s for one crate with a million of them).
Tokenize by code point with a precomputed next-']' table, and past 4096
characters of members globs admit every crate under the workspace: a wrong
member fails loudly in cargo, a dropped one is silent. The new supervised
probe kills a super-linear matcher at 30 s instead of hanging the file.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): scan cfg attributes once without truncating them; an unclosable one counts as a test

Second port review round (outside voice): the length bound from the first
round dropped a valid cfg(any(test, feature = "<257 chars>")), so its crate's
tests were no longer selected (a false green). Read each attribute once to
its closing ']' and resume after it, which keeps the scan linear without a
bound; an attribute the scan cannot close (an unterminated string, no ']')
counts as a test, which selects more, never fewer.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs(changelog): second port review round (code-point member globs, unbounded linear cfg scan)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(pregate): past the members size cap, a crate's run root is undecided and its tests unclaimed

Third port review round (outside voice): admitting every crate past the cap
made a non-member run as 'cargo test -p <name>' from the workspace root,
where a same-named member answered and its passing tests stood in for the
orphan's failing one (a false green). Keep the conservative membership for
dependency edges, but leave such a crate's workspace undecided (null):
planRuns leaves its files unclaimed and the lane incomplete.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(pregate): the Rust end-to-end runs put the cargo shim under Windows' own Path key

windows-free-shard (5) still failed both end-to-end cases at fe99b200 with
the same symptom (the runner's real cargo, exit 1, no test result lines).
runBin merges process.env, whose Windows key is 'Path', with the shim's
'PATH', so the child got two path variables and read the original. Write the
shim under the key process.env actually uses, as test/helpers/unreachable-gbrain.ts
and sync-gbrain-readiness-fixture.ts do.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test(pregate): run the cargo stand-in cases on POSIX only

windows-free-shard (5) still failed both end-to-end cargo-lane cases after the
delimiter, cargo.cmd and Path-key repairs: the lane's Git Bash exits 1 with an
empty log, and that log is not visible from CI. Skip the two cases that run the
bash stand-in on win32 (test.skipIf, as other POSIX-shim pregate fixtures are
handled) and drop the unproven cargo.cmd and Path-key changes. Selection,
planning and the hostile-name case still run on Windows.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* v1.91.197.0 stamp

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: vilano93 <244097767+vilano93@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-25 06:25:53 +00:00

536 lines
31 KiB
TypeScript

/**
* pregate/import-graph — which test files depend on which repo files (plan
* C8 `lanes`). The reverse graph unions four sources, each kept as provenance
* for `--explain`:
* 1. static imports and requires (Bun.Transpiler.scanImports), and Python
* `import` / `from … import` statements resolved from the repository root
* or, when relative, from the importing file's package;
* 2. repo-path string literals, including `join('a', 'b')` segment runs
* (a test that spawns `bin/gstack-doctor` names it in a string, never an
* import) and literal directories (`test/fixtures/x` → `x/**`), and in
* Python, dotted `mock.patch` targets (`'pkg.mod.Class.method'`);
* 3. the `bin/<name>` → `test/<name>*.test.*` convention (with and without
* the `gstack-` prefix);
* 4. explicit `.gstack/pregate.json` `dependencies` declarations;
* 5. in a Django project, the files Django loads by name (lib/pregate/django.ts);
* 6. in a Cargo crate, every `.rs` file and the Cargo.toml select the crate's
* tests and its dependents' (lib/pregate/rust.ts); in-source
* `#[cfg(test)]` files and `<crate>/tests/` files are the tests.
* Dart directives (`import`/`export`/`part`, conditional URIs included) resolve
* `package:<name>/x` through every tracked pubspec.yaml to `<root>/lib/x`.
* Uncertainty selects more tests, never fewer: a dependency on a directory
* selects every test under it; a touched file with no path to a test is the
* caller's `no lane` failure, not a pass. test/helpers/touchfile-closure.ts
* (helpers and fixtures only, a lower bound by its own words) is the precedent.
*/
import { spawnSync } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { matchesAny } from '../ship-policy';
import { djangoEdges } from './django';
import { rustEdges } from './rust';
export const SOURCE_EXT = /\.(?:[cm]?[jt]sx?|py|rb|sh|bash|go|rs|dart)$/;
export const TEST_FILE_RE = /(?:^|\/)(?:test|tests|spec|__tests__)\/.*\.(?:test|spec)\.[cm]?[jt]sx?$|(?:^|\/)[^/]*\.(?:test|spec)\.[cm]?[jt]sx?$|(?:^|\/)test_[^/]*\.py$|_test\.(?:py|go|rb)$|_spec\.rb$|(?:^|\/)[^/]*\.bats$|(?:^|\/)test\/(?:.*\/)?[^/]*_test\.dart$/;
export interface Edge { from: string; via: 'import' | 'literal' | 'convention' | 'declared' | 'generated' | 'framework' }
export interface Graph {
files: Set<string>;
tests: Set<string>;
/** dependency → dependents with the kind of edge. */
reverse: Map<string, Edge[]>;
/** Test file → import specifiers that looked like repo paths but resolved nowhere (for the no-lane explanation). */
unresolved?: Map<string, string[]>;
/** tsconfig/jsconfig `extends` hops that were not followed (for the no-lane explanation). */
aliasNotes?: string[];
}
const scanner = new Bun.Transpiler({ loader: 'tsx' });
// `( ) [ ] +` appear in framework route paths: Next.js route groups `(app)`,
// dynamic segments `[id]` / `[[...slug]]`, SvelteKit `+page.svelte`. A literal
// that matches more text only gains an edge when it resolves to a repo file.
const QUOTED = /['"`]([A-Za-z0-9_@][A-Za-z0-9_./@()[\]+-]*)['"`]/g;
const JOIN = /\bjoin\(([^()]*)\)/g;
const RESOLVE_EXT = ['', '.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs', '.mts', '.cts', '/index.ts', '/index.tsx', '/index.js', '/index.jsx', '/index.mjs'];
// A statement starts a line or follows `;` or a compound statement's `:`. The module spec is
// non-empty and starts with a dot or a word character, so no two repetitions can share a dot or a space.
const PY_IMPORT = /(?:^|[;:])[ \t\f]*(?:from[ \t]+(\.+(?:\w[\w.]*)?|\w[\w.]*)[ \t]+import[ \t]+(\([^)]*\)|[^\n;]+)|import[ \t]+([^\n;]+))/gm;
const PY_DOTTED = /^[A-Za-z_]\w*(?:\.[A-Za-z_]\w*)+$/;
/** Flutter's device-only directories: their Dart files need a device, so they are never tests here (a touched one has no lane until declared `[]`). */
const DEVICE_TEST_DIR = /(?:^|\/)(?:integration_test|test_driver)\//;
export function isDeviceDart(rel: string): boolean { return rel.endsWith('.dart') && DEVICE_TEST_DIR.test(rel); }
export function isTestFile(rel: string): boolean { return TEST_FILE_RE.test(rel) && !isDeviceDart(rel); }
/**
* A repo file's text, or null when it is not a regular file, is a symlink
* resolving outside the repo, is over `maxBytes` (default 1 MiB), or is
* unreadable. A committed symlink to a device or FIFO, or a directory named
* like the file, is skipped rather than read (a read could hang or exhaust memory).
*/
export function readRepoFile(repoRoot: string, rel: string, maxBytes = 1024 * 1024): string | null {
const abs = path.join(repoRoot, rel);
try {
if (fs.lstatSync(abs).isSymbolicLink()) {
const real = fs.realpathSync(abs);
const top = fs.realpathSync(repoRoot);
if (real !== top && !real.startsWith(top + path.sep)) return null;
}
const st = fs.statSync(abs);
if (!st.isFile() || st.size > maxBytes) return null;
return fs.readFileSync(abs, 'utf8');
} catch { return null; }
}
/** Dart package name → its roots (`''` for the repository root), from every tracked pubspec.yaml; a name two pubspecs share keeps both, in ls-files order. */
export function dartPackages(repoRoot: string, files: Iterable<string>): Map<string, string[]> {
const out = new Map<string, string[]>();
for (const f of files) {
if (path.posix.basename(f) !== 'pubspec.yaml') continue;
const text = readRepoFile(repoRoot, f);
if (text === null) continue;
const name = /^name:\s*['"]?([A-Za-z_][\w]*)/m.exec(text.replace(/^\uFEFF/, ''))?.[1];
const root = path.posix.dirname(f);
if (name) out.set(name, [...(out.get(name) ?? []), root === '.' ? '' : root]);
}
return out;
}
/** The repo files one Dart directive URI names: `package:<name>/x` → the tracked `<root>/lib/x` of the importer's own package (the deepest pubspec root of any name containing `fromFile`) when that package is `<name>` and holds it, otherwise every tracked `<root>/lib/x` of `<name>` (an ambiguous name selects more, never fewer); a relative URI from `fromFile`; `dart:` and external packages name none. */
function resolveDartUri(files: Set<string>, packages: Map<string, string[]>, fromFile: string, uri: string): string[] {
const pkg = /^package:([\w]+)\/(.+)$/.exec(uri);
if (pkg) {
const roots = packages.get(pkg[1]!) ?? [];
const targetIn = (root: string) => normalize(path.posix.join(root, 'lib', pkg[2]!));
const own = [...packages.values()].flat().filter(r => r === '' || fromFile.startsWith(`${r}/`)).sort((a, b) => b.length - a.length)[0];
if (own !== undefined && roots.includes(own) && files.has(targetIn(own))) return [targetIn(own)];
return roots.map(targetIn).filter(t => files.has(t));
}
if (/^[a-z][\w+.-]*:/.test(uri)) return [];
const target = normalize(path.posix.join(path.posix.dirname(fromFile), uri));
return files.has(target) ? [target] : [];
}
/**
* The URIs of every `import`/`export`/`part` directive (never `part of`), comments blanked. A directive runs from its keyword at
* a line start or after a `;`, metadata (`@deprecated`) allowed before it, to the next unquoted `;`, across any number of lines whatever their indentation (a conditional import's
* `if (…) 'uri'` clause may start a line). The scan resumes after each directive, so it is one linear pass; a directive missing its
* `;` reads to the end of the file, which can only add edges.
*/
export function dartDirectiveUris(source: string): string[] {
const code = blankDartComments(source.replace(/^\uFEFF/, ''));
const start = /(?:^|;)[ \t]*(?:@[\w.$]+(?:\([^;]*?\))?\s+)*(?:import|export|part)\b(?!\s+of\b)/gm;
const uris: string[] = [];
for (let m = start.exec(code); m; m = start.exec(code)) {
const from = m.index + m[0].length;
let end = from;
for (let quote = ''; end < code.length && (quote || code[end] !== ';'); end++) {
const c = code[end]!;
if (quote) { if (c === '\\') end++; else if (c === quote || c === '\n') quote = ''; } else if (c === "'" || c === '"') quote = c;
}
for (const u of code.slice(from, end).matchAll(/'([^'\n]+)'|"([^"\n]+)"/g)) uris.push((u[1] ?? u[2])!);
start.lastIndex = Math.max(end, from);
}
return uris;
}
/** Dart source with line and (nested) block comments blanked to spaces (newlines kept, so line structure holds), quote-aware, in one linear pass. */
export function blankDartComments(source: string): string {
let out = '';
let quote = '';
for (let i = 0; i < source.length; i++) {
const c = source[i]!;
if (quote) { out += c; if (c === '\\') { out += source[i + 1] ?? ''; i++; } else if (c === quote || c === '\n') quote = ''; continue; }
if (c === "'" || c === '"') { quote = c; out += c; continue; }
const line = source.startsWith('//', i);
if (!line && !source.startsWith('/*', i)) { out += c; continue; }
let stop: number;
if (line) { const e = source.indexOf('\n', i); stop = e < 0 ? source.length : e; }
else {
// Dart block comments nest.
let depth = 0;
for (stop = i; stop < source.length;) {
if (source.startsWith('/*', stop)) { depth++; stop += 2; } else if (source.startsWith('*/', stop)) { depth--; stop += 2; if (!depth) break; } else stop++;
}
}
out += source.slice(i, stop).replace(/[^\n]/g, ' ');
i = stop - 1;
}
return out;
}
export function listRepoFiles(repoRoot: string): string[] {
const r = spawnSync('git', ['ls-files', '-co', '--exclude-standard', '-z'], { cwd: repoRoot, encoding: 'utf8', timeout: 60_000, maxBuffer: 64 * 1024 * 1024 });
return (r.stdout ?? '').split('\0').filter(p => p && !p.startsWith('node_modules/'));
}
function normalize(p: string): string { return path.posix.normalize(p.replace(/\\/g, '/')).replace(/^\.\//, ''); }
/** One tsconfig/jsconfig `compilerOptions.paths` entry: `@/*` → [`./*`], already joined to its base. */
export interface PathAlias { prefix: string; suffix: string; wildcard: boolean; targets: string[] }
/** The effective `paths` of the root config, its `baseUrl` (repo-relative, or null) and what could not be read. */
export interface PathAliases { aliases: PathAlias[]; baseUrl: string | null; notes: string[] }
/** How many relative `extends` hops are followed from the root tsconfig/jsconfig. */
export const MAX_TSCONFIG_EXTENDS = 4;
/** JSONC → JSON: drop comments, and a comma whose next token closes an object or array, outside strings. */
function stripJsonc(text: string): string {
const skipTrivia = (from: number): number => {
let j = from;
while (j < text.length) {
if (/\s/.test(text[j]!)) { j++; continue; }
if (text.startsWith('//', j)) { const e = text.indexOf('\n', j); j = e < 0 ? text.length : e; continue; }
if (text.startsWith('/*', j)) { const e = text.indexOf('*/', j + 2); j = e < 0 ? text.length : e + 2; continue; }
break;
}
return j;
};
let out = '';
for (let i = 0; i < text.length; i++) {
const c = text[i]!;
if (c === '"') { const end = /"(?:[^"\\]|\\.)*"/y; end.lastIndex = i; const m = end.exec(text); const s = m ? m[0] : text.slice(i); out += s; i += s.length - 1; continue; }
if (c === '/' && text[i + 1] === '/') { while (i < text.length && text[i] !== '\n') i++; out += '\n'; continue; }
if (c === '/' && text[i + 1] === '*') { const e = text.indexOf('*/', i + 2); i = e < 0 ? text.length : e + 1; continue; }
if (c === ',' && (text[skipTrivia(i + 1)] === '}' || text[skipTrivia(i + 1)] === ']')) continue;
out += c;
}
return out;
}
interface Located<T> { value: T; dir: string }
interface ConfigOptions { baseUrl?: Located<string>; paths?: Located<Record<string, unknown>> }
/** `baseUrl` and `paths` of one config after its relative `extends` chain (later entries and the config itself override). */
function effectiveOptions(repoRoot: string, rel: string, depth: number, seen: Set<string>, notes: string[]): ConfigOptions | null {
let json: { extends?: unknown; compilerOptions?: { baseUrl?: unknown; paths?: unknown } };
try { json = JSON.parse(stripJsonc(fs.readFileSync(path.join(repoRoot, rel), 'utf8'))) ?? {}; } catch { return null; }
const dir = path.posix.dirname(rel);
let options: ConfigOptions = {};
const parents = typeof json.extends === 'string' ? [json.extends] : Array.isArray(json.extends) ? json.extends.filter((e): e is string => typeof e === 'string') : [];
for (const parent of parents) {
if (!parent.startsWith('./') && !parent.startsWith('../')) { notes.push(`${rel} extends ${parent}: not a relative path, not followed`); continue; }
if (depth >= MAX_TSCONFIG_EXTENDS) { notes.push(`${rel} extends ${parent}: not followed (tsconfig extends depth limit ${MAX_TSCONFIG_EXTENDS})`); continue; }
let target = normalize(path.posix.join(dir, parent));
if (!target.endsWith('.json') && !fs.existsSync(path.join(repoRoot, target))) target += '.json';
if (target.startsWith('../') || seen.has(target)) { notes.push(`${rel} extends ${parent}: outside the repo or a cycle, not followed`); continue; }
const inherited = effectiveOptions(repoRoot, target, depth + 1, new Set([...seen, target]), notes);
if (inherited) options = { ...options, ...inherited };
else notes.push(`${rel} extends ${parent}: unreadable`);
}
const own = json.compilerOptions ?? {};
if (typeof own.baseUrl === 'string') options.baseUrl = { value: own.baseUrl, dir };
if (own.paths && typeof own.paths === 'object') options.paths = { value: own.paths as Record<string, unknown>, dir };
return options;
}
/**
* The root tsconfig.json (else jsconfig.json) `paths`, relative to the repo,
* following relative `extends` up to MAX_TSCONFIG_EXTENDS hops. Targets join
* `baseUrl` when one is set, else the directory of the config declaring
* `paths`. What is not followed is recorded in `notes` for the no-lane
* explanation; a missing alias leaves no edge (so `no lane`), never a wrong one.
*/
export function readPathAliases(repoRoot: string): PathAliases {
for (const name of ['tsconfig.json', 'jsconfig.json']) {
const notes: string[] = [];
const options = effectiveOptions(repoRoot, name, 0, new Set([name]), notes);
if (!options) continue;
const baseUrl = options.baseUrl ? normalize(path.posix.join(options.baseUrl.dir, options.baseUrl.value)) : null;
const base = baseUrl ?? options.paths?.dir ?? '.';
const aliases: PathAlias[] = [];
for (const [pattern, targets] of Object.entries(options.paths?.value ?? {})) {
if (!Array.isArray(targets)) continue;
const star = pattern.indexOf('*');
const resolved = targets.filter((t): t is string => typeof t === 'string').map(t => normalize(path.posix.join(base, t)));
aliases.push(star < 0 ? { prefix: pattern, suffix: '', wildcard: false, targets: resolved } : { prefix: pattern.slice(0, star), suffix: pattern.slice(star + 1), wildcard: true, targets: resolved });
}
return { aliases, baseUrl, notes };
}
return { aliases: [], baseUrl: null, notes: [] };
}
const NO_ALIASES: PathAliases = { aliases: [], baseUrl: null, notes: [] };
/**
* TypeScript's pattern choice: an exact pattern, else the matching wildcard
* with the longest prefix. `chosen` is that pattern's expanded targets in
* order; `others` are the other matching patterns' targets.
*/
function matchAliases(aliases: readonly PathAlias[], ref: string): { chosen: string[] | null; others: string[] } {
const expand = (a: PathAlias) => a.wildcard ? a.targets.map(t => normalize(t.replace('*', ref.slice(a.prefix.length, ref.length - a.suffix.length)))) : a.targets;
const matching = aliases.filter(a => a.wildcard ? ref.startsWith(a.prefix) && ref.endsWith(a.suffix) && ref.length >= a.prefix.length + a.suffix.length : ref === a.prefix);
if (matching.length === 0) return { chosen: null, others: [] };
const best = matching.find(a => !a.wildcard) ?? matching.reduce((a, b) => (b.prefix.length > a.prefix.length ? b : a));
return { chosen: expand(best), others: matching.filter(a => a !== best).flatMap(expand) };
}
/** One candidate repo path → the file it names (trying extensions), `dir/**`, a bare bin, or null. */
function resolveCandidate(files: Set<string>, dirs: Set<string>, c: string, literal: boolean): string | null {
if (c.startsWith('../') || c === '..' || c === '.' || c === '') return null;
for (const ext of RESOLVE_EXT) if (files.has(c + ext)) return c + ext;
// A directory literal needs two segments (`test/fixtures/x`): a bare word is a name, not a path.
if (dirs.has(c) && (!literal || c.includes('/'))) return `${c}/**`;
// A bare bin name in a test (spawned through PATH or a helper's bin dir) names bin/<name>.
if (literal && !c.includes('/') && files.has(`bin/${c}`)) return `bin/${c}`;
return null;
}
/**
* A module specifier or literal → the repo files it names. A relative ref
* and an alias whose chosen pattern has a target that exists name one file
* (targets tried in order). Otherwise no single target is established, so
* every plausible reading (other matching patterns, `baseUrl`, the repo
* root) contributes: uncertainty selects more tests, never fewer.
*/
function resolveRefs(files: Set<string>, dirs: Set<string>, fromFile: string, ref: string, literal = false, config: PathAliases = NO_ALIASES): string[] {
if (!ref || ref.startsWith('node:') || ref.startsWith('bun:')) return [];
if (ref.startsWith('.')) { const t = resolveCandidate(files, dirs, normalize(path.posix.join(path.posix.dirname(fromFile), ref)), literal); return t ? [t] : []; }
const { chosen, others } = matchAliases(config.aliases, ref);
for (const c of chosen ?? []) { const t = resolveCandidate(files, dirs, c, literal); if (t) return [t]; }
if (!chosen && others.length === 0 && ref.startsWith('@') && !ref.includes('/')) return [];
const plausible = [...others, ...(config.baseUrl ? [normalize(path.posix.join(config.baseUrl, ref))] : []), normalize(ref)];
return [...new Set(plausible.map(c => resolveCandidate(files, dirs, c, literal)).filter((t): t is string => t !== null))];
}
/** A specifier worth naming when it resolves nowhere: relative, aliased, or alias-shaped (`@/`, `~`, `#`); never a package. */
function pathLike(config: PathAliases, ref: string): boolean {
if (ref.startsWith('.') || /^(?:@\/|~|#)/.test(ref)) return true;
const { chosen, others } = matchAliases(config.aliases, ref);
return chosen !== null || others.length > 0;
}
const pathDepths = new WeakMap<Set<string>, number>();
/**
* Segments in the repo's deepest path. No module name resolves deeper, so
* dotted-name resolution is bounded by the repo rather than by the source.
*/
function maxPathDepth(files: Set<string>): number {
let depth = pathDepths.get(files);
if (depth === undefined) { depth = 0; for (const f of files) depth = Math.max(depth, f.split('/').length); pathDepths.set(files, depth); }
return depth;
}
/** The leading dotted name of each comma-separated `name [as alias]` clause. */
function pyNames(list: string): string[] {
return list.replace(/[()]/g, '').split(',').map(p => p.trim().split(/\s+/)[0]!).filter(n => n && n !== '*');
}
/** `from <dots><rest>` → the absolute dotted package it names, or null when it climbs above the repository root. */
function pyFromBase(rel: string, spec: string): string | null {
const dots = /^\.*/.exec(spec)![0].length;
const rest = spec.slice(dots);
if (!dots) return rest;
const pkg = path.posix.dirname(rel).split('/').filter(s => s !== '.');
if (dots - 1 > pkg.length) return null;
return [...pkg.slice(0, pkg.length - (dots - 1)), ...(rest ? [rest] : [])].join('.');
}
/**
* Python source with every string blanked to `''` and every comment dropped, in
* one linear pass, so a `#` inside a string and a quote inside a comment both
* stay inert. Backslash escapes are honoured; an unterminated string runs to
* its line end, or to the end of the file when triple-quoted.
*/
export function pythonCode(source: string): string {
const parts: string[] = [];
let from = 0;
for (let i = 0; i < source.length;) {
const c = source[i]!;
if (c === '#') { parts.push(source.slice(from, i)); const end = source.indexOf('\n', i); i = from = end < 0 ? source.length : end; continue; }
if (c !== '"' && c !== "'") { i++; continue; }
parts.push(source.slice(from, i), "''");
const close = source.startsWith(c + c + c, i) ? c + c + c : c;
let j = i + close.length;
while (j < source.length && !source.startsWith(close, j) && (close.length === 3 || source[j] !== '\n')) j += source[j] === '\\' ? 2 : 1;
i = from = Math.min(source.length, source.startsWith(close, j) ? j + close.length : j);
}
parts.push(source.slice(from));
return parts.join('');
}
/** Dotted modules a Python source imports; `from a import b` also yields `a.b`, which may be a submodule. */
function pythonImports(rel: string, source: string): string[] {
const code = pythonCode(source).replace(/\\\r?\n/g, ' ');
const out: string[] = [];
for (const m of code.matchAll(PY_IMPORT)) {
if (m[3]) { out.push(...pyNames(m[3])); continue; }
const base = pyFromBase(rel, m[1]!);
if (base === null) continue;
if (base) out.push(base);
for (const name of pyNames(m[2]!)) out.push(base ? `${base}.${name}` : name);
}
return out;
}
/**
* A dotted module → its repo files (`a/b/__init__.py`, which Python prefers,
* and `a/b.py`; both when both exist) plus the package `__init__.py` files
* Python executes on the way; [] for a module outside the repo (stdlib,
* site-packages), including one deeper than any repo path.
*/
function pythonModuleFiles(files: Set<string>, dotted: string): string[] {
const parts = dotted.split('.');
if (parts.length > maxPathDepth(files) || parts.some(p => !p)) return [];
const p = parts.join('/');
const found = [`${p}/__init__.py`, `${p}.py`].filter(f => files.has(f));
if (!found.length) return [];
const inits = parts.slice(0, -1).map((_, i) => `${parts.slice(0, i + 1).join('/')}/__init__.py`).filter(f => files.has(f));
return [...found, ...inits];
}
/** `'pkg.mod.Class.attr'` (a `mock.patch` target) → the longest dotted prefix that is a repo module. */
function pythonPatchTarget(files: Set<string>, ref: string): string[] {
const parts = ref.split('.').slice(0, maxPathDepth(files));
for (let n = parts.length; n >= 1; n--) {
const hit = pythonModuleFiles(files, parts.slice(0, n).join('.'));
if (hit.length) return hit;
}
return [];
}
function addEdge(graph: Graph, dependency: string, dependent: string, via: Edge['via']): void {
if (dependency === dependent) return;
const list = graph.reverse.get(dependency) ?? [];
if (!list.some(e => e.from === dependent && e.via === via)) { list.push({ from: dependent, via }); graph.reverse.set(dependency, list); }
}
/**
* Every dependency reference one source file makes: imports, requires, quoted
* repo paths and join() segment runs. Import specifiers that resolve nowhere
* and look like repo paths go to `unresolved` when given.
*/
export function referencesIn(files: Set<string>, dirs: Set<string>, rel: string, source: string, config: PathAliases = NO_ALIASES, unresolved?: string[], packages: Map<string, string[]> = new Map()): Array<{ target: string; via: Edge['via'] }> {
const out: Array<{ target: string; via: Edge['via'] }> = [];
if (rel.endsWith('.dart')) {
for (const uri of dartDirectiveUris(source)) for (const t of resolveDartUri(files, packages, rel, uri)) out.push({ target: t, via: 'import' });
}
const importOf = (spec: string) => {
const targets = resolveRefs(files, dirs, rel, spec, false, config);
for (const target of targets) out.push({ target, via: 'import' });
if (targets.length === 0 && unresolved && pathLike(config, spec) && !unresolved.includes(spec)) unresolved.push(spec);
};
if (/\.[cm]?[jt]sx?$/.test(rel)) {
let imports: Array<{ path: string }> = [];
try { imports = scanner.scanImports(source); } catch { imports = []; }
for (const { path: spec } of imports) importOf(spec);
// The transpiler erases type-only imports; a types module still binds its importers' contracts.
for (const m of source.matchAll(/^\s*(?:import|export)\s+type\b[^;'"]*?\bfrom\s+['"]([^'"]+)['"]/gm)) importOf(m[1]!);
}
const python = rel.endsWith('.py');
if (python) for (const mod of pythonImports(rel, source)) for (const t of pythonModuleFiles(files, mod)) out.push({ target: t, via: 'import' });
// Literals carry dependencies only from the test tree and from bin/ and shell
// sources (a test spawns a bin; a bash bin calls a sibling). Prose and
// manifests name everything and would make every file a hub.
if (!scansLiterals(rel)) return out;
for (const m of source.matchAll(QUOTED)) {
const targets = resolveRefs(files, dirs, rel, m[1]!, true, config);
for (const target of targets) out.push({ target, via: 'literal' });
if (python && PY_DOTTED.test(m[1]!)) for (const f of pythonPatchTarget(files, m[1]!)) out.push({ target: f, via: 'literal' });
}
for (const m of source.matchAll(JOIN)) {
const segments = [...m[1]!.matchAll(/['"`]([^'"`]+)['"`]/g)].map(s => s[1]!);
if (segments.length < 2) continue;
const joined = segments.join('/');
const direct = resolveRefs(files, dirs, rel, joined, true);
for (const target of direct.length ? direct : resolveRefs(files, dirs, rel, './' + joined, true)) out.push({ target, via: 'literal' });
}
return out;
}
/** Test-tree files (tests, helpers, fixtures), bin/ entrypoints and shell sources carry literal dependencies. */
export function scansLiterals(rel: string): boolean {
return /(?:^|\/)(?:test|tests|spec|__tests__)\//.test(rel) || isTestFile(rel) || rel.startsWith('bin/') || /\.(?:sh|bash)$/.test(rel);
}
export interface BuildOptions {
declarations?: Record<string, string[]>; maxBytes?: number;
/** Paths that no longer exist (deleted or renamed-from) but that tests may still name; they resolve as dependencies so their consumers are found. */
extraFiles?: string[];
}
/** The whole-repo reverse graph. Files over maxBytes (default 2 MiB) are skipped: generated bundles, not tests. */
export function buildGraph(repoRoot: string, opts: BuildOptions = {}): Graph {
const all = listRepoFiles(repoRoot);
const files = new Set([...all, ...(opts.extraFiles ?? [])]);
const dirs = new Set<string>();
for (const f of all) { let d = path.posix.dirname(f); while (d && d !== '.') { dirs.add(d); d = path.posix.dirname(d); } }
const config = readPathAliases(repoRoot);
const graph: Graph = { files, tests: new Set(all.filter(isTestFile)), reverse: new Map(), unresolved: new Map(), aliasNotes: config.notes };
const maxBytes = opts.maxBytes ?? 2 * 1024 * 1024;
const packages = dartPackages(repoRoot, all);
const pySources = new Map<string, string>();
for (const rel of all) {
if (!SOURCE_EXT.test(rel) && !rel.startsWith('bin/')) continue;
let source: string;
try { const st = fs.statSync(path.join(repoRoot, rel)); if (!st.isFile() || st.size > maxBytes) continue; source = fs.readFileSync(path.join(repoRoot, rel), 'utf8'); } catch { continue; }
if (rel.endsWith('.py')) pySources.set(rel, source);
const unresolved: string[] = [];
for (const { target, via } of referencesIn(files, dirs, rel, source, config, graph.tests.has(rel) ? unresolved : undefined, packages)) addEdge(graph, target, rel, via);
if (unresolved.length) graph.unresolved!.set(rel, unresolved);
}
const rust = rustEdges(all, rel => readRepoFile(repoRoot, rel, maxBytes));
for (const t of rust.tests) graph.tests.add(t);
for (const e of rust.edges) addEdge(graph, e.dependency, e.dependent, e.via);
const django = djangoEdges({ files: [...files], tests: graph.tests, read: rel => pySources.get(rel) ?? readRepoFile(repoRoot, rel, maxBytes), code: pythonCode, resolveDotted: dotted => pythonPatchTarget(files, dotted) });
for (const e of django) addEdge(graph, e.dependency, e.dependent, 'framework');
for (const test of graph.tests) {
const m = /^(?:[\w.-]+\/)*?test\/([\w-]+)\.(?:test|spec)\./.exec(test);
if (!m) continue;
// test/gstack-ship-receipt-census.test.ts names bin/gstack-ship-receipt-census, then bin/gstack-ship-receipt, then bin/gstack-ship, …
const parts = m[1]!.split('-');
for (let n = parts.length; n >= 1; n--) {
const name = parts.slice(0, n).join('-');
for (const bin of [`bin/${name}`, `bin/gstack-${name}`, `bin/${name}.ts`, `bin/gstack-${name}.ts`]) if (files.has(bin)) addEdge(graph, bin, test, 'convention');
}
}
for (const [glob, tests] of Object.entries(opts.declarations ?? {})) {
for (const f of files) if (matchesAny([glob], f)) for (const t of tests) addEdge(graph, f, t, 'declared');
}
return graph;
}
export interface LaneMapping { file: string; tests: string[]; provenance: string[]; declaredNone: boolean }
/**
* Tests that depend on `file`, transitively through non-test files and
* through `dir/**` edges. `declarations` with an empty list declare "no lane
* needed" and win when nothing else maps.
*/
export function testsFor(graph: Graph, file: string, declarations: Record<string, string[]> = {}, maxDepth = 12): LaneMapping {
const tests = new Set<string>();
const provenance: string[] = [];
const seen = new Set<string>([file]);
const queue: Array<{ node: string; chain: string[] }> = [{ node: file, chain: [file] }];
const dirEdges = [...graph.reverse.keys()].filter(k => k.endsWith('/**'));
while (queue.length) {
const { node, chain } = queue.shift()!;
if (chain.length > maxDepth) continue;
const edges = [...(graph.reverse.get(node) ?? [])];
for (const dir of dirEdges) if (node.startsWith(dir.slice(0, -3) + '/')) edges.push(...(graph.reverse.get(dir) ?? []).map(e => ({ ...e, via: e.via })));
for (const e of edges) {
if (seen.has(e.from)) continue;
seen.add(e.from);
if (graph.tests.has(e.from)) { tests.add(e.from); provenance.push(`${e.from} ← ${[...chain].reverse().join(' ← ')} (${e.via})`); }
else queue.push({ node: e.from, chain: [...chain, e.from] });
}
}
if (graph.tests.has(file)) { tests.add(file); provenance.push(`${file} (itself a test)`); }
const declaredNone = tests.size === 0 && Object.entries(declarations).some(([glob, list]) => list.length === 0 && matchesAny([glob], file));
if (declaredNone) provenance.push(`${file}: declared no lane needed (.gstack/pregate.json dependencies)`);
if (tests.size === 0 && !declaredNone) provenance.push(...noLaneExplanation(graph, file));
return { file, tests: [...tests].sort(), provenance, declaredNone };
}
/** Why the graph may have missed a file's tests: unresolved repo-path specifiers in test files and unfollowed `extends`. */
function noLaneExplanation(graph: Graph, file: string): string[] {
const seen = [...(graph.unresolved ?? new Map<string, string[]>())].sort(([a], [b]) => a.localeCompare(b));
const notes = graph.aliasNotes ?? [];
if (seen.length === 0 && notes.length === 0) return [];
const lines = notes.map(n => `${file}: path aliases may be incomplete: ${n}`);
const shown = seen.slice(0, 10).map(([test, specs]) => `${test}: ${specs.slice(0, 5).map(s => `'${s}'`).join(' ')}${specs.length > 5 ? ' …' : ''}`);
if (shown.length) lines.push(`${file}: no lane; specifiers in test files that resolve to no repo file: ${shown.join('; ')}${seen.length > 10 ? `; +${seen.length - 10} more test file(s)` : ''}`);
lines.push(`${file}: if a test covers it, declare the file under dependencies in .gstack/pregate.json, or report the resolver gap`);
return lines;
}