mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-03 15:27:56 +01:00
v1.93.15.0 fix(pregate): pytest, Playwright and mixed-language runners; literals stops flagging stray digits (#3598, #3492) (#3741)
* fix(pregate): runner detection for pytest, Playwright and mixed-language roots; literals binds one-character values A missing pytest is PREGATE_ENVIRONMENT, not the diff's failure (#3598). Playwright specs run under playwright test, with missing browsers reported as the machine's gap (#3629). Root tests in two languages each get their runner, and a repo's bun/vitest/jest claims only JS/TS (#3600). literals counts a one-character old value only in a statement naming the constant, and a call-site name also needs the module (#3492, gstack-evals#97). Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(pregate): name the import-type shapes the Playwright detection skips Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: escape Playwright file filters; skip type-only Playwright imports; pytest probe and summary read hardening Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix(pregate): linear Playwright import scan; comments and no-space imports detected Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(pregate): Playwright missing-browsers exception and literals continuation rule Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
4b43d4fd43
commit
90ccf30e2a
13 files changed
+353
-30
No files matched your search
@@ -1,5 +1,18 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## [1.93.15.0] - 2026-11-02
|
||||||
|
|
||||||
|
**Pregate stops blaming a clean branch for a missing pytest, Playwright specs and mixed-language root tests, and the literals check stops flagging stray digits (#3598, #3492, gstack-evals#97).** A machine without pytest reported "fix the in-branch failure". Playwright specs in a vitest repo went to vitest, which found no tests and failed. Root tests in `.mjs` and `.py` got no runner at all. And any lone `2` in a file that named a changed constant (a comment, a `$2` in a shell stub, a fixture) failed `literals`. A wrapper bump from 3 to 4 would have raised 5 false hits here, and gstack-evals#97 hit 11. Rewritten from outside PRs #3639 and #3600 by @vilano93 and #3629 by @nfitzsimmons.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **Missing pytest is the machine's gap.** Before planning `python3 -m pytest`, `lanes` asks that python3 whether it can import pytest (10 s probe, from the repo root). If it can't, the file is unclaimed and the lane is `PREGATE_ENVIRONMENT` (missing-binary, `pytest` or `python3`): "install pytest on this machine and rerun, or run the lane remotely; the diff is not at fault".
|
||||||
|
- **Playwright specs run under Playwright.** A JS/TS test file that imports `@playwright/test` at the start of a line runs as `<pm exec> playwright test --reporter=list` from its nearest package. Other tests keep their runner. A declared `runner` still takes every file. Playwright's summary is read count line by count line. Failed and interrupted count as failed, flaky and passed as passed, and skipped as not run. A run where every test fails and Playwright says its browser is not installed is `PREGATE_ENVIRONMENT` naming `npx playwright install`.
|
||||||
|
- **Mixed-language root tests each get a runner.** A repo's bun, vitest or jest now claims only JS/TS files. When no one runner fits every root test, they are split by language and each part is detected on its own. So `.mjs` with `.py` runs `node --test` and pytest, and a `.py` test in a vitest repo gets pytest instead of vitest. `gstack-pregate init` drops its "a selection mixing these languages needs runner set" note.
|
||||||
|
- **`literals` binds a one-character old value to its statement.** The value counts on a line that names the constant, or on up to three lines that continue that statement, never on a comment line. Version-history rows count as before. A lowercase call-site name (`attempts`, `limit`) also needs the changed module named in the file. Values of two or more characters are unchanged.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **docs/pregate.md** describes the new detection and the literals binding. **docs/troubleshooting.md** names the Playwright missing-browsers case under `PREGATE_ENVIRONMENT`. **docs/state-root.md** lists `pregate/runs/<run id>/`.
|
||||||
|
|
||||||
## [1.93.14.0] - 2026-11-02
|
## [1.93.14.0] - 2026-11-02
|
||||||
|
|
||||||
Free Tests no longer goes red with no named file when one test hangs. Each `free-suite` job's GitHub timeout used to be a fixed 20 minutes, shorter than the runner's own wall for a 10-shard plan (about 37 minutes), so GitHub killed the job before the runner could name the hung file or retry it, and no shard result was uploaded. The plan now emits each shard's job timeout as its runner wall plus the flaky-retry pass plus the job's measured setup (`--ci-plan --job-setup-seconds`), 49 minutes per shard today, and fails loudly above 90 minutes instead of truncating. See docs/TESTING_INTERNALS.md#runners-how-the-suites-execute-2026-08-overhaul. Closes #3850.
|
Free Tests no longer goes red with no named file when one test hangs. Each `free-suite` job's GitHub timeout used to be a fixed 20 minutes, shorter than the runner's own wall for a 10-shard plan (about 37 minutes), so GitHub killed the job before the runner could name the hung file or retry it, and no shard result was uploaded. The plan now emits each shard's job timeout as its runner wall plus the flaky-retry pass plus the job's measured setup (`--ci-plan --job-setup-seconds`), 49 minutes per shard today, and fails loudly above 90 minutes instead of truncating. See docs/TESTING_INTERNALS.md#runners-how-the-suites-execute-2026-08-overhaul. Closes #3850.
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# gstack digest v1.93.14.0 — regenerate/re-copy after upgrading gstack
|
# gstack digest v1.93.15.0 — regenerate/re-copy after upgrading gstack
|
||||||
|
|
||||||
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
||||||
for agent hosts without a full skill install. The full skills add workflows,
|
for agent hosts without a full skill install. The full skills add workflows,
|
||||||
|
|||||||
+16
-4
@@ -74,7 +74,7 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`).
|
|||||||
|
|
||||||
| Key | Meaning |
|
| Key | Meaning |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `runner` | the local test command with `{files}` (gstack: `bun test --timeout=30000 {files}`, the 30 s per-test timeout CI uses); unset, it is detected (bun, vitest, jest, pytest, `node --test`, and `flutter test`/`dart test` per Dart package root, through fvm when that package or an ancestor pins it with `.fvmrc` or `.fvm/fvm_config.json` and fvm is on PATH; `cargo test -p <name>` per Rust crate, run from the root of the workspace the crate is a member of (Cargo's `members` globs, `exclude` and `package.workspace`), else the crate directory; a JS/TS test whose nearest package.json is a nested package declaring vitest or jest (a dependency or its `test` script) runs from that package, with package-relative paths, through the package manager its `packageManager` field or the nearest lockfile up to the repo root names: `pnpm exec`, `yarn`, `bun x`, else `npx`; every other JS/TS test keeps the root detection); nothing detected, a Dart SDK, cargo or package manager missing from PATH, a `.dart` file with no `pubspec.yaml` above it, a `.rs` file outside any crate, or one in a crate whose name Cargo would reject (anything but letters, digits, `_` and `-`), is `incomplete` (the runnable groups still run, so a real failure still fails) |
|
| `runner` | the local test command with `{files}` (gstack: `bun test --timeout=30000 {files}`, the 30 s per-test timeout CI uses); unset, it is detected (bun, vitest or jest for JS/TS files, pytest when python3 can import it (else `pytest` or `python3` is named missing), `node --test`, and `flutter test`/`dart test` per Dart package root, through fvm when that package or an ancestor pins it with `.fvmrc` or `.fvm/fvm_config.json` and fvm is on PATH; `cargo test -p <name>` per Rust crate, run from the root of the workspace the crate is a member of (Cargo's `members` globs, `exclude` and `package.workspace`), else the crate directory; a JS/TS test whose nearest package.json is a nested package declaring vitest or jest (a dependency or its `test` script) runs from that package, with package-relative paths, through the package manager its `packageManager` field or the nearest lockfile up to the repo root names: `pnpm exec`, `yarn`, `bun x`, else `npx`; a JS/TS test file that imports `@playwright/test` at the start of a line runs as `playwright test --reporter=list` from its nearest package through that same package manager, whatever the unit runner is, a type-only import (`import type …`, or one whose every specifier is `type X`) does not count, file arguments are passed as escaped regexes anchored to the path (Playwright reads them as regexes), and a run where every test fails and Playwright says its browser is not installed is `PREGATE_ENVIRONMENT` (a partial install, where some tests pass, stays the diff's failure); every other test keeps the root detection, split by language when no one runner fits them all, so `.mjs` and `.py` root tests each get their own runner); nothing detected, a Dart SDK, cargo or package manager missing from PATH, a `.dart` file with no `pubspec.yaml` above it, a `.rs` file outside any crate, or one in a crate whose name Cargo would reject (anything but letters, digits, `_` and `-`), is `incomplete` (the runnable groups still run, so a real failure still fails) |
|
||||||
| `dependencies` | glob → test files for subprocess targets, runtime reads and generated inputs the import graph cannot see; `[]` declares "no lane needed" (docs, licences). A `[]` that matches a deleted test does not skip it: the deleted test still runs its owning suite, and a file naming it that `[]` covers (a durations list) counts as covered. With this key present, even empty, the policy [declares lanes](#lane-requirements) |
|
| `dependencies` | glob → test files for subprocess targets, runtime reads and generated inputs the import graph cannot see; `[]` declares "no lane needed" (docs, licences). A `[]` that matches a deleted test does not skip it: the deleted test still runs its owning suite, and a file naming it that `[]` covers (a durations list) counts as covered. With this key present, even empty, the policy [declares lanes](#lane-requirements) |
|
||||||
| `empty_lanes` | extra globs for the docs/no-tests empty lane. Any Markdown file already qualifies with no pin file when no test depends on it (the same import graph as other files: a quoted path, a `join()` of path segments, an import, or a `dependencies` entry). reStructuredText and AsciiDoc count as Markdown here. README, CHANGELOG, CONTRIBUTING, LICENSE, AGENTS and CLAUDE qualify even without `.md`, and so does licence text under any vendored name (`roboto-LICENSE.txt`, `LICENSE-MIT`, `licenses/*.txt`). Source code and files under a test directory never qualify. A test that reads the file still selects that test |
|
| `empty_lanes` | extra globs for the docs/no-tests empty lane. Any Markdown file already qualifies with no pin file when no test depends on it (the same import graph as other files: a quoted path, a `join()` of path segments, an import, or a `dependencies` entry). reStructuredText and AsciiDoc count as Markdown here. README, CHANGELOG, CONTRIBUTING, LICENSE, AGENTS and CLAUDE qualify even without `.md`, and so does licence text under any vendored name (`roboto-LICENSE.txt`, `LICENSE-MIT`, `licenses/*.txt`). Source code and files under a test directory never qualify. A test that reads the file still selects that test |
|
||||||
| `selection` | `workflow.yml/job` → a command printing the lane's file list, for matrices a planner computes (gstack: `scripts/test-free-shards.ts --list`) |
|
| `selection` | `workflow.yml/job` → a command printing the lane's file list, for matrices a planner computes (gstack: `scripts/test-free-shards.ts --list`) |
|
||||||
@@ -220,6 +220,14 @@ Two stages with separate clocks, both printed:
|
|||||||
other line holding the old value still fail. Only the row itself is exempt:
|
other line holding the old value still fail. Only the row itself is exempt:
|
||||||
the old value after its closing bracket on the same line still fails. Rows in object form (`{ v: 45 }`) or spread over several
|
the old value after its closing bracket on the same line still fails. Rows in object form (`{ v: 45 }`) or spread over several
|
||||||
lines are not recognized as history rows.
|
lines are not recognized as history rows.
|
||||||
|
A one-character old value (`2`) counts only in a statement that names the
|
||||||
|
constant: on the name's line, or on up to three lines that continue it (the
|
||||||
|
line before ends with `(`, `[`, `{`, `,`, `.`, `=` or another operator, or the
|
||||||
|
line starts with `.`, `)`, `]`, `}` or an operator), never on a comment line, plus the history rows
|
||||||
|
above. A `$2` in a shell stub, a comment, or `export default 2` in a fixture
|
||||||
|
elsewhere in the file is not a consumer. A call-site name such as `attempts`
|
||||||
|
or `limit` is an ordinary word, so for it the file must also name the changed
|
||||||
|
module.
|
||||||
|
|
||||||
- `timing`: added or changed test lines that compare an elapsed wall-clock or
|
- `timing`: added or changed test lines that compare an elapsed wall-clock or
|
||||||
CPU time against a bound, or sleep and then assert, print
|
CPU time against a bound, or sleep and then assert, print
|
||||||
@@ -437,7 +445,7 @@ test-tier command:
|
|||||||
Per-file lines come from a progress adapter where the runner's reporter
|
Per-file lines come from a progress adapter where the runner's reporter
|
||||||
provides them: `bun test` (its `<file>:` headers) is the only adapter today;
|
provides them: `bun test` (its `<file>:` headers) is the only adapter today;
|
||||||
a one-file run (each serial file) reports its file from the launch for any
|
a one-file run (each serial file) reports its file from the launch for any
|
||||||
runner. Other runners (node --test, vitest, jest, pytest, flutter test, dart test, cargo test) get heartbeats
|
runner. Other runners (node --test, vitest, jest, pytest, playwright test, flutter test, dart test, cargo test) get heartbeats
|
||||||
only. A heartbeat follows each 60 s with no progress line;
|
only. A heartbeat follows each 60 s with no progress line;
|
||||||
- is killed as **stalled** when its log does not grow for the stall timeout
|
- is killed as **stalled** when its log does not grow for the stall timeout
|
||||||
(`--stall-timeout <s>`, else `stall_timeout_s`, else 300; `0` disables): the
|
(`--stall-timeout <s>`, else `stall_timeout_s`, else 300; `0` disables): the
|
||||||
@@ -994,7 +1002,7 @@ failure the branch caused carries `cause: "diff"` (`PREGATE_LANE_FAILED`,
|
|||||||
|
|
||||||
| Kind | Evidence |
|
| Kind | Evidence |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `missing-binary` | exit 127, `command not found`, a spawn `ENOENT`, a runner tool `not on PATH` (dart, flutter, cargo, a package manager) |
|
| `missing-binary` | exit 127, `command not found`, a spawn `ENOENT`, a runner tool `not on PATH` (dart, flutter, cargo, a package manager, `python3`, or `pytest` when python3 cannot import it), Playwright's browsers not installed |
|
||||||
| `network` | `ENOTFOUND`, `ECONNREFUSED`, `ECONNRESET`, `ETIMEDOUT`, `EAI_AGAIN`, `Could not resolve host`, `fetch failed`, a TLS handshake or certificate failure |
|
| `network` | `ENOTFOUND`, `ECONNREFUSED`, `ECONNRESET`, `ETIMEDOUT`, `EAI_AGAIN`, `Could not resolve host`, `fetch failed`, a TLS handshake or certificate failure |
|
||||||
| `quota` | `quota`, `429 Too Many Requests`, `rate limit`, `insufficient_quota` (a vCPU quota on the remote backend) |
|
| `quota` | `quota`, `429 Too Many Requests`, `rate limit`, `insufficient_quota` (a vCPU quota on the remote backend) |
|
||||||
| `disk` | `ENOSPC`, `No space left on device`, `EDQUOT` |
|
| `disk` | `ENOSPC`, `No space left on device`, `EDQUOT` |
|
||||||
@@ -1005,7 +1013,11 @@ The classifier is conservative. A local lane that exited non-zero is read from
|
|||||||
its output **only when its runner reported no test summary** (nothing ran) and
|
its output **only when its runner reported no test summary** (nothing ran) and
|
||||||
the runner was not seen starting a test file; a run whose summary counts `N
|
the runner was not seen starting a test file; a run whose summary counts `N
|
||||||
ran, M failed` is the diff's failure whatever its log says, so a test that
|
ran, M failed` is the diff's failure whatever its log says, so a test that
|
||||||
failed on `ECONNREFUSED` is still a red test. A remote gate that exited
|
failed on `ECONNREFUSED` is still a red test. One exception: when every
|
||||||
|
failed run is a Playwright run in which every test failed and Playwright says
|
||||||
|
its browsers are not installed, the lane is `missing-binary` with `proof:
|
||||||
|
"output"` (a partial install, where some tests pass, stays the diff's
|
||||||
|
failure). A remote gate that exited
|
||||||
without a `PREGATE_REMOTE:` line is read from its **preamble only** (its first
|
without a `PREGATE_REMOTE:` line is read from its **preamble only** (its first
|
||||||
40 lines, before any test output could stream): a VM refused for quota or an
|
40 lines, before any test output could stream): a VM refused for quota or an
|
||||||
unreachable API there never ran the tests, so it is environment and the
|
unreachable API there never ran the tests, so it is environment and the
|
||||||
|
|||||||
+3
-1
@@ -105,7 +105,9 @@ lives in `projects/<slug>/`. `bin/gstack-slug` picks the slug from the git
|
|||||||
- Without a remote, the project root's directory name is used.
|
- Without a remote, the project root's directory name is used.
|
||||||
|
|
||||||
A bucket can also hold a user-owned `pregate.json` policy, for a repo that keeps
|
A bucket can also hold a user-owned `pregate.json` policy, for a repo that keeps
|
||||||
`.gstack/` out of git (see [pregate v1](pregate.md#pregate-v1)).
|
`.gstack/` out of git (see [pregate v1](pregate.md#pregate-v1)), and
|
||||||
|
`pregate/runs/<run id>/`, the scratch each pregate run writes (lane and guard
|
||||||
|
logs, the remote gate's file list); old runs are pruned when a new one starts.
|
||||||
|
|
||||||
The first slug gstack resolves in a directory sticks (it is cached under
|
The first slug gstack resolves in a directory sticks (it is cached under
|
||||||
`slug-cache/`), so a project keeps its bucket when it adds a remote later.
|
`slug-cache/`), so a project keeps its bucket when it adds a remote later.
|
||||||
|
|||||||
@@ -3932,7 +3932,10 @@ and the table prints `environment: <kind> — <evidence>`. A test that failed an
|
|||||||
whose log also names a connection error is **not** this code: the runner's own
|
whose log also names a connection error is **not** this code: the runner's own
|
||||||
summary counted a failure, so it is `PREGATE_LANE_FAILED` with `cause:
|
summary counted a failure, so it is `PREGATE_LANE_FAILED` with `cause:
|
||||||
"diff"`; nor is a runner, guard or gate script whose own binary is present but
|
"diff"`; nor is a runner, guard or gate script whose own binary is present but
|
||||||
exits 127. `proof: "launch"` is certain (the command could not start, its
|
exits 127. One exception: a Playwright lane where every test failed and
|
||||||
|
Playwright says its browsers are not installed is `missing-binary` with
|
||||||
|
`proof: "output"` (`npx playwright install`); a partial install stays the
|
||||||
|
diff's failure. `proof: "launch"` is certain (the command could not start, its
|
||||||
binary is off PATH, the runtime or a gstack tool refused); `proof: "output"`
|
binary is off PATH, the runtime or a gstack tool refused); `proof: "output"`
|
||||||
means a rule matched the command's text before any test was counted, which the
|
means a rule matched the command's text before any test was counted, which the
|
||||||
diff's own output could have written, so the line says `(inferred from output)`
|
diff's own output could have written, so the line says `(inferred from output)`
|
||||||
|
|||||||
+81
-15
@@ -27,6 +27,7 @@
|
|||||||
* Incidents: p0-4-pipe-hidden-failure (`false | tail` read as green),
|
* Incidents: p0-4-pipe-hidden-failure (`false | tail` read as green),
|
||||||
* p0-4-windows-lane-unrun (a Windows-only change merged with its lane unrun).
|
* p0-4-windows-lane-unrun (a Windows-only change merged with its lane unrun).
|
||||||
*/
|
*/
|
||||||
|
import { spawnSync } from 'node:child_process';
|
||||||
import * as fs from 'node:fs';
|
import * as fs from 'node:fs';
|
||||||
import * as path from 'node:path';
|
import * as path from 'node:path';
|
||||||
import { baseInputs, sha256 } from '../context';
|
import { baseInputs, sha256 } from '../context';
|
||||||
@@ -43,17 +44,21 @@ import { verifyFromRun } from '../../run-evidence';
|
|||||||
import { matchesAny } from '../../ship-policy';
|
import { matchesAny } from '../../ship-policy';
|
||||||
import type { CheckDef, CheckResult, PregateContext, RemoteObligation } from '../types';
|
import type { CheckDef, CheckResult, PregateContext, RemoteObligation } from '../types';
|
||||||
|
|
||||||
/** The test command with `{files}`; detected from the repo when the config does not name one. */
|
/** The runner a `.py`-only selection gets; it needs the pytest module importable by python3 (`pytestMissing`). */
|
||||||
|
const PYTEST = 'python3 -m pytest {files}';
|
||||||
|
|
||||||
|
/** The test command with `{files}`; detected from the repo when the config does not name one. The repo's bun, vitest or jest claims JS/TS files only. */
|
||||||
export function detectRunner(ctx: Pick<PregateContext, 'config' | 'repoRoot'>, files: readonly string[]): string | null {
|
export function detectRunner(ctx: Pick<PregateContext, 'config' | 'repoRoot'>, files: readonly string[]): string | null {
|
||||||
if (ctx.config.runner) return ctx.config.runner;
|
if (ctx.config.runner) return ctx.config.runner;
|
||||||
const has = (f: string) => fs.existsSync(path.join(ctx.repoRoot, f));
|
const has = (f: string) => fs.existsSync(path.join(ctx.repoRoot, f));
|
||||||
let pkg: { devDependencies?: Record<string, string>; dependencies?: Record<string, string>; scripts?: Record<string, string> } = {};
|
let pkg: { devDependencies?: Record<string, string>; dependencies?: Record<string, string>; scripts?: Record<string, string> } = {};
|
||||||
try { pkg = JSON.parse(fs.readFileSync(path.join(ctx.repoRoot, 'package.json'), 'utf8')); } catch { /* no package.json */ }
|
try { pkg = JSON.parse(fs.readFileSync(path.join(ctx.repoRoot, 'package.json'), 'utf8')); } catch { /* no package.json */ }
|
||||||
const deps = { ...pkg.dependencies, ...pkg.devDependencies };
|
const deps = { ...pkg.dependencies, ...pkg.devDependencies };
|
||||||
if (has('bunfig.toml') || has('bun.lock') || has('bun.lockb')) return 'bun test {files}';
|
const jsts = files.every(f => /\.[cm]?[jt]sx?$/.test(f));
|
||||||
if (deps.vitest) return 'npx vitest run {files}';
|
if (jsts && (has('bunfig.toml') || has('bun.lock') || has('bun.lockb'))) return 'bun test {files}';
|
||||||
if (deps.jest) return 'npx jest {files}';
|
if (jsts && deps.vitest) return 'npx vitest run {files}';
|
||||||
if (files.every(f => /\.py$/.test(f)) && files.length) return 'python3 -m pytest {files}';
|
if (jsts && deps.jest) return 'npx jest {files}';
|
||||||
|
if (files.every(f => /\.py$/.test(f)) && files.length) return PYTEST;
|
||||||
if (files.every(f => /\.[cm]?js$/.test(f)) && files.length) return 'node --test {files}';
|
if (files.every(f => /\.[cm]?js$/.test(f)) && files.length) return 'node --test {files}';
|
||||||
if (files.every(f => /\.[cm]?[jt]sx?$/.test(f)) && files.length && Bun.which('bun')) return 'bun test {files}';
|
if (files.every(f => /\.[cm]?[jt]sx?$/.test(f)) && files.length && Bun.which('bun')) return 'bun test {files}';
|
||||||
return null;
|
return null;
|
||||||
@@ -113,6 +118,18 @@ export function dartRunner(repoRoot: string, root: string, env: NodeJS.ProcessEn
|
|||||||
return { runner: `${fvm ? 'fvm ' : ''}${tool} test {files}` };
|
return { runner: `${fvm ? 'fvm ' : ''}${tool} test {files}` };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The tool `python3 -m pytest` lacks: `python3` off the PATH of `env`, or `pytest` when that python3, run from the repo root as the
|
||||||
|
* runner is, cannot import it (a `pytest` script on PATH may belong to another interpreter); null when both are there, or when the
|
||||||
|
* probe itself times out or dies, so the run decides.
|
||||||
|
*/
|
||||||
|
function pytestMissing(repoRoot: string, env: NodeJS.ProcessEnv): string | null {
|
||||||
|
const python = Bun.which('python3', { PATH: env.PATH ?? '' });
|
||||||
|
if (!python) return 'python3';
|
||||||
|
const r = spawnSync(python, ['-c', 'import importlib.util, sys; sys.exit(3 if importlib.util.find_spec("pytest") is None else 0)'], { cwd: repoRoot, env, stdio: 'ignore', timeout: 10_000 });
|
||||||
|
return r.status === 3 ? 'pytest' : null;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* `cargo test -p <name>` for a crate (cargo runs a crate's tests, never a file list), or the missing tool when cargo is
|
* `cargo test -p <name>` for a crate (cargo runs a crate's tests, never a file list), or the missing tool when cargo is
|
||||||
* not on PATH. The name comes from the repo's Cargo.toml and reaches a shell and the `{files}` expansion: a name outside
|
* not on PATH. The name comes from the repo's Cargo.toml and reaches a shell and the `{files}` expansion: a name outside
|
||||||
@@ -163,6 +180,23 @@ function packageRunner(repoRoot: string, dir: string, env: NodeJS.ProcessEnv): {
|
|||||||
return { runner: `${PM_EXEC[pm]} ${tool} {files}` };
|
return { runner: `${PM_EXEC[pm]} ${tool} {files}` };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether a JS/TS test imports a value from Playwright's runner in a statement that starts a line: `import … from '@playwright/test'`
|
||||||
|
* (one line or several), a bare import, or `require`. A type-only import (`import type …`, or every specifier `type X`) and a string or
|
||||||
|
* fixture body that mentions the import mid-line do not count.
|
||||||
|
*/
|
||||||
|
function importsPlaywright(text: string): boolean {
|
||||||
|
const code = text.replace(/\/\*[\s\S]*?\*\/|\/\/[^\n]*/g, '');
|
||||||
|
if (/^[\t ]*(?:import[\t ]*|(?:const|let|var)\b[^=\n]*=[\t ]*require\([\t ]*)['"]@playwright\/test['"]/m.test(code)) return true;
|
||||||
|
for (const m of code.matchAll(/^[\t ]*import(?=[\s{*])([^'"`;]*?)\bfrom[\t ]*['"]@playwright\/test['"]/gm)) {
|
||||||
|
const spec = m[1]!.trim();
|
||||||
|
if (!/^type(?=[\s{*])/.test(spec) && spec.replace(/[{}]/g, '').split(',').some(s => s.trim() && !/^type\s/.test(s.trim()))) return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/** Playwright's own text when its browsers are not installed on this machine. */
|
||||||
|
const PLAYWRIGHT_NO_BROWSERS = /^.*(?:Executable doesn't exist at|download new browsers).*$/m;
|
||||||
|
|
||||||
/** The run plan: the groups that can run, the files no runner claims, tools missing from PATH, and `.dart` files with no pubspec.yaml above. */
|
/** The run plan: the groups that can run, the files no runner claims, tools missing from PATH, and `.dart` files with no pubspec.yaml above. */
|
||||||
export interface RunPlan { groups: RunGroup[]; unclaimed: string[]; missing: string[]; noPubspec: string[] }
|
export interface RunPlan { groups: RunGroup[]; unclaimed: string[]; missing: string[]; noPubspec: string[] }
|
||||||
|
|
||||||
@@ -171,8 +205,11 @@ export interface RunPlan { groups: RunGroup[]; unclaimed: string[]; missing: str
|
|||||||
* Dart tests run from their own package root (a sub-package's imports resolve
|
* Dart tests run from their own package root (a sub-package's imports resolve
|
||||||
* against its own pubspec); Rust tests run per crate as `cargo test -p <name>`
|
* against its own pubspec); Rust tests run per crate as `cargo test -p <name>`
|
||||||
* from the crate's workspace root; JS/TS tests owned by a nested package that
|
* from the crate's workspace root; JS/TS tests owned by a nested package that
|
||||||
* declares vitest or jest run from that package; the rest go to the detected
|
* declares vitest or jest run from that package; a JS/TS test that imports
|
||||||
* runner from the repo root. Files no runner claims (a tool missing from PATH,
|
* `@playwright/test` runs through `playwright test` from its nearest package;
|
||||||
|
* the rest go to the detected runner from the repo root, split by language
|
||||||
|
* when no one runner fits them all (`.mjs` + `.py` runs both), and pytest only
|
||||||
|
* when python3 can import it. Files no runner claims (a tool missing from PATH,
|
||||||
* a `.dart` file outside any package, a `.rs` file outside any crate or in a
|
* a `.dart` file outside any package, a `.rs` file outside any crate or in a
|
||||||
* crate whose name Cargo would reject, no detected runner) are `unclaimed`: the lane is `incomplete`, never a pass.
|
* crate whose name Cargo would reject, no detected runner) are `unclaimed`: the lane is `incomplete`, never a pass.
|
||||||
*/
|
*/
|
||||||
@@ -182,6 +219,7 @@ export function planRuns(ctx: PregateContext, files: readonly string[]): RunPlan
|
|||||||
const dart = new Map<string, string[]>();
|
const dart = new Map<string, string[]>();
|
||||||
const cargo = new Map<string, { name: string; cwd: string; files: string[] }>();
|
const cargo = new Map<string, { name: string; cwd: string; files: string[] }>();
|
||||||
const packages = new Map<string, string[]>();
|
const packages = new Map<string, string[]>();
|
||||||
|
const playwright = new Map<string, string[]>();
|
||||||
const seen = new Map<string, string | null>();
|
const seen = new Map<string, string | null>();
|
||||||
const seenPackage = new Map<string, string | null>();
|
const seenPackage = new Map<string, string | null>();
|
||||||
let model: CargoModel | null = null;
|
let model: CargoModel | null = null;
|
||||||
@@ -204,6 +242,7 @@ export function planRuns(ctx: PregateContext, files: readonly string[]): RunPlan
|
|||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
const pkg = /\.[cm]?[jt]sx?$/.test(f) ? nearestRoot(ctx.repoRoot, f, 'package.json', seenPackage) : null;
|
const pkg = /\.[cm]?[jt]sx?$/.test(f) ? nearestRoot(ctx.repoRoot, f, 'package.json', seenPackage) : null;
|
||||||
|
if (pkg !== null && importsPlaywright(readRepoFile(ctx.repoRoot, f) ?? '')) { add(playwright, pkg, under(pkg, f)); continue; }
|
||||||
if (pkg) {
|
if (pkg) {
|
||||||
if (!pkgRunners.has(pkg)) pkgRunners.set(pkg, packageRunner(ctx.repoRoot, pkg, ctx.env));
|
if (!pkgRunners.has(pkg)) pkgRunners.set(pkg, packageRunner(ctx.repoRoot, pkg, ctx.env));
|
||||||
const r = pkgRunners.get(pkg);
|
const r = pkgRunners.get(pkg);
|
||||||
@@ -225,12 +264,24 @@ export function planRuns(ctx: PregateContext, files: readonly string[]): RunPlan
|
|||||||
if (r) missing.add(r.missing);
|
if (r) missing.add(r.missing);
|
||||||
unclaimed.push(...c.files.map(f => (c.cwd ? `${c.cwd}/${f}` : f)));
|
unclaimed.push(...c.files.map(f => (c.cwd ? `${c.cwd}/${f}` : f)));
|
||||||
}
|
}
|
||||||
if (rest.length) {
|
const language = (f: string) => (/\.py$/.test(f) ? 'py' : /\.[cm]?js$/.test(f) ? 'js' : /\.[cm]?[jt]sx?$/.test(f) ? 'ts' : f);
|
||||||
const runner = detectRunner(ctx, rest);
|
const parts = !rest.length ? [] : detectRunner(ctx, rest) ? [rest] : [...Map.groupBy(rest, language).values()];
|
||||||
if (runner) groups.push({ runner, cwd: '', files: rest });
|
const byRunner = new Map<string, string[]>();
|
||||||
else unclaimed.push(...rest);
|
for (const part of parts) {
|
||||||
|
const runner = detectRunner(ctx, part);
|
||||||
|
const lacking = runner === PYTEST ? pytestMissing(ctx.repoRoot, ctx.env) : null;
|
||||||
|
if (lacking) missing.add(lacking);
|
||||||
|
if (runner && !lacking) for (const f of part) add(byRunner, runner, f);
|
||||||
|
else unclaimed.push(...part);
|
||||||
}
|
}
|
||||||
|
for (const [runner, list] of byRunner) groups.push({ runner, cwd: '', files: list });
|
||||||
for (const [root, list] of [...packages].sort(([a], [b]) => a.localeCompare(b))) groups.push({ runner: (pkgRunners.get(root) as { runner: string }).runner, cwd: root, files: list });
|
for (const [root, list] of [...packages].sort(([a], [b]) => a.localeCompare(b))) groups.push({ runner: (pkgRunners.get(root) as { runner: string }).runner, cwd: root, files: list });
|
||||||
|
for (const [root, list] of [...playwright].sort(([a], [b]) => a.localeCompare(b))) {
|
||||||
|
const pm = packageManager(ctx.repoRoot, root);
|
||||||
|
const bin = pm === 'npm' ? 'npx' : pm;
|
||||||
|
if (onPath(bin, ctx.env)) groups.push({ runner: `${PM_EXEC[pm]} playwright test --reporter=list {files}`, cwd: root, files: list });
|
||||||
|
else { missing.add(bin); unclaimed.push(...list.map(f => (root ? `${root}/${f}` : f))); }
|
||||||
|
}
|
||||||
return { groups, unclaimed: [...unclaimed, ...noPubspec], missing: [...missing].sort(), noPubspec };
|
return { groups, unclaimed: [...unclaimed, ...noPubspec], missing: [...missing].sort(), noPubspec };
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -269,11 +320,20 @@ function parseCargo(output: string): RunnerSummary | null {
|
|||||||
return { ran: lines.reduce((n, m) => n + Number(m[1]), 0) + failed, failed };
|
return { ran: lines.reduce((n, m) => n + Number(m[1]), 0) + failed, failed };
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The runner's own summary line: bun, node --test, vitest, jest, pytest, bats, dart/flutter test, cargo test. A bats, dart/flutter or cargo runner's summary is tried first (cargo's only there: its `N passed` lines would misread as another runner's); for any other runner the bats and dart formats are a last resort. `ran: null` when not recognised. Colour codes are dropped and a carriage return ends a line, so a coloured or redrawn verdict still reads. */
|
/** playwright test: the last `N failed`, `N interrupted`, `N flaky`, `N passed` and `N skipped` lines (its closing summary, with test names between them; a test's own output earlier does not count); skipped tests did not run. */
|
||||||
|
function parsePlaywright(output: string): RunnerSummary | null {
|
||||||
|
if (!/^\s*\d+ (?:passed|failed|flaky|interrupted|skipped)\b/m.test(output)) return null;
|
||||||
|
const count = (word: string) => Number([...output.matchAll(new RegExp(`^\\s*(\\d+) ${word}\\b`, 'gm'))].pop()?.[1] ?? 0);
|
||||||
|
const failed = count('failed') + count('interrupted');
|
||||||
|
return { ran: count('passed') + count('flaky') + failed, failed };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The runner's own summary line: bun, node --test, vitest, jest, pytest, bats, dart/flutter test, cargo test, playwright test. A bats, dart/flutter, playwright or cargo runner's summary is tried first (cargo's only there: its `N passed` lines would misread as another runner's); for any other runner the bats and dart formats are a last resort. `ran: null` when not recognised. Colour codes are dropped and a carriage return ends a line, so a coloured or redrawn verdict still reads. */
|
||||||
export function parseSummary(raw: string, runner = ''): RunnerSummary {
|
export function parseSummary(raw: string, runner = ''): RunnerSummary {
|
||||||
const output = raw.replace(/\x1b\[[0-9;?]*[A-Za-z]/g, '').replace(/\r/g, '\n');
|
const output = raw.replace(/\x1b\[[0-9;?]*[A-Za-z]/g, '').replace(/\r/g, '\n');
|
||||||
if (/(?<![\w-])cargo\s+test(?![\w-])/.test(runner)) return parseCargo(output) ?? { ran: null, failed: null };
|
if (/(?<![\w-])cargo\s+test(?![\w-])/.test(runner)) return parseCargo(output) ?? { ran: null, failed: null };
|
||||||
if (/(?<![\w-])bats(?![\w-])/.test(runner)) { const b = parseBats(output); if (b) return b; }
|
if (/(?<![\w-])bats(?![\w-])/.test(runner)) { const b = parseBats(output); if (b) return b; }
|
||||||
|
if (/(?<![\w-])playwright\s+test(?![\w-])/.test(runner)) { const p = parsePlaywright(output); if (p) return p; }
|
||||||
const dartRun = /(?<![\w-])(?:dart|flutter)(?![\w-])/.test(runner);
|
const dartRun = /(?<![\w-])(?:dart|flutter)(?![\w-])/.test(runner);
|
||||||
if (dartRun) { const d = parseDart(output); if (d) return d; }
|
if (dartRun) { const d = parseDart(output); if (d) return d; }
|
||||||
let m: RegExpExecArray | null;
|
let m: RegExpExecArray | null;
|
||||||
@@ -287,19 +347,22 @@ export function parseSummary(raw: string, runner = ''): RunnerSummary {
|
|||||||
return (dartRun ? null : parseDart(output)) ?? parseBats(output) ?? { ran: null, failed: null };
|
return (dartRun ? null : parseDart(output)) ?? parseBats(output) ?? { ran: null, failed: null };
|
||||||
}
|
}
|
||||||
|
|
||||||
interface LocalRun { exit: number | null; outcome: Supervised['outcome']; signal: string | null; observedStart: boolean; ran: number | null; failed: number | null; tail: string[]; runner: string; command: string; files: string[]; lastFile: string | null; ms: number; error?: string }
|
interface LocalRun { exit: number | null; outcome: Supervised['outcome']; signal: string | null; observedStart: boolean; ran: number | null; failed: number | null; tail: string[]; runner: string; command: string; files: string[]; lastFile: string | null; ms: number; error?: string; noBrowsers?: string }
|
||||||
|
|
||||||
const quote = (f: string) => `'${f.replace(/'/g, `'\\''`)}'`;
|
const quote = (f: string) => `'${f.replace(/'/g, `'\\''`)}'`;
|
||||||
|
|
||||||
async function runOnce(ctx: PregateContext, env: NodeJS.ProcessEnv, g: RunGroup, files: string[], log: string, counter: Counter): Promise<LocalRun> {
|
async function runOnce(ctx: PregateContext, env: NodeJS.ProcessEnv, g: RunGroup, files: string[], log: string, counter: Counter): Promise<LocalRun> {
|
||||||
const joined = files.map(quote).join(' ');
|
// playwright test reads each file argument as a case-insensitive regex: escape it and anchor it to a path segment, so `a[1].spec.ts` runs that file, not `a1.spec.ts`.
|
||||||
|
const args = /(?<![\w-])playwright\s+test(?![\w-])/.test(g.runner) ? files.map(f => `(?:^|/)${f.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`) : files;
|
||||||
|
const joined = args.map(quote).join(' ');
|
||||||
const run = g.runner.replace('{files}', () => joined);
|
const run = g.runner.replace('{files}', () => joined);
|
||||||
const command = g.cwd ? `(cd ${quote(g.cwd)} && ${run})` : run;
|
const command = g.cwd ? `(cd ${quote(g.cwd)} && ${run})` : run;
|
||||||
const shown = g.cwd ? files.map(f => `${g.cwd}/${f}`) : files;
|
const shown = g.cwd ? files.map(f => `${g.cwd}/${f}`) : files;
|
||||||
const r = await supervise({ command: run, cwd: path.join(ctx.repoRoot, g.cwd), env: { ...env, CI: env.CI ?? '1' }, log, deadline: Date.now() + ctx.timeoutMs, stallMs: ctx.stallTimeoutMs, label: `lanes (${shown.length === 1 ? shown[0] : `${shown.length} files`})`, files, adapter: adapterFor(g.runner), counter, progress: ctx.progress });
|
const r = await supervise({ command: run, cwd: path.join(ctx.repoRoot, g.cwd), env: { ...env, CI: env.CI ?? '1' }, log, deadline: Date.now() + ctx.timeoutMs, stallMs: ctx.stallTimeoutMs, label: `lanes (${shown.length === 1 ? shown[0] : `${shown.length} files`})`, files, adapter: adapterFor(g.runner), counter, progress: ctx.progress });
|
||||||
const output = readRange(log, r.start, r.end);
|
const output = readRange(log, r.start, r.end);
|
||||||
const summary = parseSummary(output, g.runner);
|
const summary = parseSummary(output, g.runner);
|
||||||
return { exit: r.exit, outcome: r.outcome, signal: r.signal, observedStart: r.observedStart, ...summary, tail: output.trim().split('\n').slice(-25), runner: g.runner, command, files: shown, lastFile: r.lastFile && g.cwd ? `${g.cwd}/${r.lastFile}` : r.lastFile, ms: r.ms, ...(r.error ? { error: r.error } : {}) };
|
const noBrowsers = /(?<![\w-])playwright\s+test(?![\w-])/.test(g.runner) ? PLAYWRIGHT_NO_BROWSERS.exec(output)?.[0].trim().slice(0, 200) : undefined;
|
||||||
|
return { ...(noBrowsers ? { noBrowsers } : {}), exit: r.exit, outcome: r.outcome, signal: r.signal, observedStart: r.observedStart, ...summary, tail: output.trim().split('\n').slice(-25), runner: g.runner, command, files: shown, lastFile: r.lastFile && g.cwd ? `${g.cwd}/${r.lastFile}` : r.lastFile, ms: r.ms, ...(r.error ? { error: r.error } : {}) };
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -496,6 +559,9 @@ async function runSelected(ctx: PregateContext, sel: Selection, base: CheckResul
|
|||||||
// only when the runner's binary is off PATH.
|
// only when the runner's binary is off PATH.
|
||||||
const environment = r.ran === null && r.bad && !r.bad.observedStart ? classifyCommandEnvironment(r.bad.runner, r.tail.join('\n'), r.exit, runtime.env) : null;
|
const environment = r.ran === null && r.bad && !r.bad.observedStart ? classifyCommandEnvironment(r.bad.runner, r.tail.join('\n'), r.exit, runtime.env) : null;
|
||||||
if (environment) return environmentResult(withRun, environment, `the runner exited ${r.exit} before any test was counted (${total} files): ${environment.evidence}`);
|
if (environment) return environmentResult(withRun, environment, `the runner exited ${r.exit} before any test was counted (${total} files): ${environment.evidence}`);
|
||||||
|
// Playwright counts a test whose browser is not installed as failed; when every test of every failed run failed and Playwright says so, the machine lacks the browsers.
|
||||||
|
const failedRuns = r.runs.filter(x => x.exit !== 0);
|
||||||
|
if (failedRuns.every(x => x.noBrowsers && x.failed !== null && x.failed === x.ran)) return environmentResult(withRun, { kind: 'missing-binary', evidence: `Playwright browsers not installed (npx playwright install): ${failedRuns[0]!.noBrowsers}`, proof: 'output' }, `every Playwright test failed to launch its browser (${r.failed} of ${r.ran}; ${total} files)`);
|
||||||
return { ...withRun, status: 'fail', cause: 'diff', code: 'PREGATE_LANE_FAILED', detail: `local lane exited ${r.exit} (${r.failed ?? '?'} failed of ${r.ran ?? '?'} ran; ${total} files; pipefail)`, fix: `read ${r.log}, fix the in-branch failure, rerun gstack-pregate --stage tests` };
|
return { ...withRun, status: 'fail', cause: 'diff', code: 'PREGATE_LANE_FAILED', detail: `local lane exited ${r.exit} (${r.failed ?? '?'} failed of ${r.ran ?? '?'} ran; ${total} files; pipefail)`, fix: `read ${r.log}, fix the in-branch failure, rerun gstack-pregate --stage tests` };
|
||||||
}
|
}
|
||||||
if (r.ran === 0) return { ...withRun, status: 'fail', cause: 'diff', code: 'PREGATE_LANE_FAILED', detail: `ZERO-RUN: the runner reported 0 tests${r.zero ? ` in ${r.zero}` : ''} (${total} selected file(s))`, fix: 'a bad selector, a missing file or a skipped describe; exit 0 is not a pass' };
|
if (r.ran === 0) return { ...withRun, status: 'fail', cause: 'diff', code: 'PREGATE_LANE_FAILED', detail: `ZERO-RUN: the runner reported 0 tests${r.zero ? ` in ${r.zero}` : ''} (${total} selected file(s))`, fix: 'a bad selector, a missing file or a skipped describe; exit 0 is not a pass' };
|
||||||
|
|||||||
@@ -69,7 +69,12 @@ function unquote(v: string): string { return /^['"`]/.test(v) ? v.slice(1, -1) :
|
|||||||
* `/retry.ts.snap`, or `lib/install.ts` for `install.sh`, is not the module (GST-820): a bare stem such as `install` or `retry` is an
|
* `/retry.ts.snap`, or `lib/install.ts` for `install.sh`, is not the module (GST-820): a bare stem such as `install` or `retry` is an
|
||||||
* English word in half the test tree, and a one-character old value (`2`)
|
* English word in half the test tree, and a one-character old value (`2`)
|
||||||
* sits in every test, so that value needs the constant's name (a wrapper
|
* sits in every test, so that value needs the constant's name (a wrapper
|
||||||
* version 2 → 3 flagged 575 consumers through the stem alone).
|
* version 2 → 3 flagged 575 consumers through the stem alone), and counts only
|
||||||
|
* in a statement that names it: on the name's line, or on a line that
|
||||||
|
* continues it (the line before ends open, `(` `,` `.` `=`, or the line
|
||||||
|
* starts with `.` `)` `]`), comments excluded (#3492: `$2`, "from revision 2",
|
||||||
|
* `export default 2`). A call-site name (`attempts`, `limit`) is an ordinary
|
||||||
|
* word, so for it the file must also name the module (gstack-evals#97).
|
||||||
*
|
*
|
||||||
* A version-history row (`[45, "<digest>"]`, the old value first in a
|
* A version-history row (`[45, "<digest>"]`, the old value first in a
|
||||||
* bracketed row) is skipped when the same table, a run of consecutive
|
* bracketed row) is skipped when the same table, a run of consecutive
|
||||||
@@ -94,12 +99,23 @@ export function consumersOf(repoRoot: string, testFiles: readonly string[], chan
|
|||||||
const moduleRe = new RegExp(`(?:^|[/\\\\])${escape(stem)}(?=[^\\w.-]|$|\\.${dts}(?:${exts})(?![\\w.-])|\\.(?!\\w))|(?<![\\w-])${escape(base)}(?![\\w-]|\\.\\w)`, 'm');
|
const moduleRe = new RegExp(`(?:^|[/\\\\])${escape(stem)}(?=[^\\w.-]|$|\\.${dts}(?:${exts})(?![\\w.-])|\\.(?!\\w))|(?<![\\w-])${escape(base)}(?![\\w-]|\\.\\w)`, 'm');
|
||||||
const nameRe = new RegExp(`\\b${escape(change.name)}\\b`);
|
const nameRe = new RegExp(`\\b${escape(change.name)}\\b`);
|
||||||
const nameRequired = value.length < 2;
|
const nameRequired = value.length < 2;
|
||||||
|
const callSite = !/^[A-Z]/.test(change.name);
|
||||||
|
const comment = (l: string) => /^\s*(?:\/\/|#|\*|\/\*|<!--)/.test(l);
|
||||||
|
const continues = (prev: string, cur: string) => /[([{,.=+\-*/&|?:\\]\s*$/.test(prev) || /^\s*[.)\]}?:&|+]/.test(cur);
|
||||||
const hits: string[] = [];
|
const hits: string[] = [];
|
||||||
for (const rel of testFiles) {
|
for (const rel of testFiles) {
|
||||||
let text: string;
|
let text: string;
|
||||||
try { text = fs.readFileSync(path.join(repoRoot, rel), 'utf8'); } catch { continue; }
|
try { text = fs.readFileSync(path.join(repoRoot, rel), 'utf8'); } catch { continue; }
|
||||||
if (!(nameRe.test(text) || (!nameRequired && moduleRe.test(text)))) continue;
|
if (nameRequired ? !(nameRe.test(text) && (!callSite || moduleRe.test(text))) : !(nameRe.test(text) || moduleRe.test(text))) continue;
|
||||||
const lines = text.split('\n');
|
const lines = text.split('\n');
|
||||||
|
const named = (i: number) => {
|
||||||
|
if (comment(lines[i]!)) return false;
|
||||||
|
for (let j = i; j >= 0 && i - j < 4; j--) {
|
||||||
|
if (j < i && (comment(lines[j]!) || !continues(lines[j]!, lines[j + 1]!))) return false;
|
||||||
|
if (nameRe.test(lines[j]!)) return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
};
|
||||||
const table: number[] = [];
|
const table: number[] = [];
|
||||||
const tableHasNew: boolean[] = [];
|
const tableHasNew: boolean[] = [];
|
||||||
for (let i = 0; i < lines.length; i++) {
|
for (let i = 0; i < lines.length; i++) {
|
||||||
@@ -115,7 +131,7 @@ export function consumersOf(repoRoot: string, testFiles: readonly string[], chan
|
|||||||
if (extended === undefined) { const base = baseText?.(rel); extended = base === null || (base !== undefined && !newRowRe.test(base)); }
|
if (extended === undefined) { const base = baseText?.(rel); extended = base === null || (base !== undefined && !newRowRe.test(base)); }
|
||||||
return extended;
|
return extended;
|
||||||
};
|
};
|
||||||
for (let i = 0; i < lines.length; i++) if (valueRe.test(lines[i]!) && !skip(i)) { hits.push(`${rel}:${i + 1}`); break; }
|
for (let i = 0; i < lines.length; i++) if (valueRe.test(lines[i]!) && (!nameRequired || oldRowRe.test(lines[i]!) || named(i)) && !skip(i)) { hits.push(`${rel}:${i + 1}`); break; }
|
||||||
}
|
}
|
||||||
return hits;
|
return hits;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -79,8 +79,7 @@ export function draftConfig(repoRoot: string): InitDraft {
|
|||||||
const runner = detectRunner({ repoRoot, config: DEFAULT_CONFIG }, list);
|
const runner = detectRunner({ repoRoot, config: DEFAULT_CONFIG }, list);
|
||||||
return runner ? `${lang} tests detected at run time: ${runner}` : `no runner detected for ${list.length} ${lang} test file(s); set runner (with {files})`;
|
return runner ? `${lang} tests detected at run time: ${runner}` : `no runner detected for ${list.length} ${lang} test file(s); set runner (with {files})`;
|
||||||
});
|
});
|
||||||
const mixed = [...groups.keys()].filter(l => l !== 'dart' && l !== 'rust').length > 1 ? '; a selection mixing these languages needs runner set' : '';
|
const runnerNote = notes.length ? notes.join('; ') : 'no test files found';
|
||||||
const runnerNote = notes.length ? notes.join('; ') + mixed : 'no test files found';
|
|
||||||
|
|
||||||
const selectionTodo = discoverLanes(repoRoot).filter(l => l.runsTests && l.matrixExpression && l.testFiles.length === 0).map(l => l.id);
|
const selectionTodo = discoverLanes(repoRoot).filter(l => l.runsTests && l.matrixExpression && l.testFiles.length === 0).map(l => l.id);
|
||||||
const config = { $schema: PREGATE_CONFIG_SCHEMA, runner: null, dependencies, empty_lanes: [], selection: {}, warn: [], strays: { allow: [] } };
|
const config = { $schema: PREGATE_CONFIG_SCHEMA, runner: null, dependencies, empty_lanes: [], selection: {}, warn: [], strays: { allow: [] } };
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "gstack",
|
"name": "gstack",
|
||||||
"version": "1.93.14",
|
"version": "1.93.15",
|
||||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
|
|||||||
@@ -12,8 +12,11 @@ import { afterEach, describe, expect, test } from 'bun:test';
|
|||||||
import * as fs from 'node:fs';
|
import * as fs from 'node:fs';
|
||||||
import * as path from 'node:path';
|
import * as path from 'node:path';
|
||||||
import { EXIT } from '../lib/headless-artifacts';
|
import { EXIT } from '../lib/headless-artifacts';
|
||||||
|
import { planRuns } from '../lib/pregate/checks/lanes';
|
||||||
|
import { DEFAULT_CONFIG } from '../lib/pregate/config';
|
||||||
import { classifyCommandEnvironment, classifyEnvironment, commandBinary } from '../lib/pregate/environment';
|
import { classifyCommandEnvironment, classifyEnvironment, commandBinary } from '../lib/pregate/environment';
|
||||||
import { verifyPregate } from '../lib/pregate/index';
|
import { verifyPregate } from '../lib/pregate/index';
|
||||||
|
import type { PregateContext } from '../lib/pregate/types';
|
||||||
import { BASE_PREGATE, cleanup, commit, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/pregate-fixture';
|
import { BASE_PREGATE, cleanup, commit, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/pregate-fixture';
|
||||||
|
|
||||||
const repos: FixtureRepo[] = [];
|
const repos: FixtureRepo[] = [];
|
||||||
@@ -171,6 +174,83 @@ describe('environment failures are reported apart from diff failures', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('a detected pytest runner needs pytest importable by python3', () => {
|
||||||
|
const PY_TEST = { 'tests/test_alpha.py': 'import unittest\n\nclass T(unittest.TestCase):\n def test_a(self):\n self.assertEqual(1 + 1, 2)\n' };
|
||||||
|
/** A directory holding only a `python3` whose `-c` probe exits `probeExit` (0: pytest importable, 3: not, anything else: the probe itself failed). */
|
||||||
|
const python3Shim = (repo: FixtureRepo, probeExit: number): string => {
|
||||||
|
const dir = path.join(repo.root, `py-${probeExit}`);
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(dir, 'python3'), `#!/bin/sh\n[ "$1" = "-c" ] && exit ${probeExit}\necho "unexpected: python3 $*"\nexit 3\n`, { mode: 0o755 });
|
||||||
|
return dir;
|
||||||
|
};
|
||||||
|
const ctxFor = (repo: FixtureRepo, PATH: string) => ({ repoRoot: repo.work, config: DEFAULT_CONFIG, env: { PATH } }) as unknown as PregateContext;
|
||||||
|
|
||||||
|
// Value: protects=a `.py` selection plans `python3 -m pytest` only when python3 can import pytest; otherwise the tool to install is
|
||||||
|
// named (`pytest`, or `python3` when it is off PATH) and the file is unclaimed; fails_when=the pytest branch plans the runner
|
||||||
|
// unchecked again (#3598); why_new=pnpm/cargo/dart had a pre-run check, pytest none; seam=none; no_claim=a real pytest install
|
||||||
|
test('plans pytest when python3 imports it, else names pytest or python3 as missing', () => {
|
||||||
|
const repo = makeFixtureRepo({ pregate: null, baseFiles: PY_TEST }); repos.push(repo);
|
||||||
|
const files = ['tests/test_alpha.py'];
|
||||||
|
expect(planRuns(ctxFor(repo, python3Shim(repo, 0)), files)).toEqual({ groups: [{ runner: 'python3 -m pytest {files}', cwd: '', files }], unclaimed: [], missing: [], noPubspec: [] });
|
||||||
|
expect(planRuns(ctxFor(repo, python3Shim(repo, 3)), files)).toEqual({ groups: [], unclaimed: files, missing: ['pytest'], noPubspec: [] });
|
||||||
|
// A probe that crashes (a repo-root module shadowing importlib exits 1) is not read as a missing pytest: the run decides.
|
||||||
|
expect(planRuns(ctxFor(repo, python3Shim(repo, 1)), files).groups).toEqual([{ runner: 'python3 -m pytest {files}', cwd: '', files }]);
|
||||||
|
expect(planRuns(ctxFor(repo, path.join(repo.root, 'empty-path')), files)).toEqual({ groups: [], unclaimed: files, missing: ['python3'], noPubspec: [] });
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=the reported case end to end: a machine without pytest gets PREGATE_ENVIRONMENT missing-binary naming pytest and
|
||||||
|
// "the diff is not at fault", and the lane never runs; fails_when=it is PREGATE_LANE_FAILED with "fix the in-branch failure"
|
||||||
|
// (#3598); why_new=no pytest environment case existed; seam=none; no_claim=a real pytest install
|
||||||
|
test('a machine without pytest is PREGATE_ENVIRONMENT naming pytest, never PREGATE_LANE_FAILED', () => {
|
||||||
|
const repo = makeFixtureRepo({ pregate: null, baseFiles: PY_TEST }); repos.push(repo);
|
||||||
|
commit(repo, { 'tests/test_alpha.py': `${PY_TEST['tests/test_alpha.py']}# touched\n` });
|
||||||
|
const r = runBin(repo, 'gstack-pregate', ['--base', 'main', '--allow-repo-commands', '--stage', 'tests', '--only', 'lanes'], { PATH: `${python3Shim(repo, 3)}:${process.env.PATH ?? ''}` });
|
||||||
|
expect(r.err).toContain('(PREGATE_ENVIRONMENT)');
|
||||||
|
expect(r.err).not.toContain('PREGATE_LANE_FAILED');
|
||||||
|
expect(r.out).not.toContain('unexpected: python3');
|
||||||
|
const c = check(repo, 'lanes');
|
||||||
|
expect([c.status, c.code, c.cause, c.environment.kind]).toEqual(['incomplete', 'PREGATE_ENVIRONMENT', 'environment', 'missing-binary']);
|
||||||
|
expect(c.environment.evidence).toContain('pytest');
|
||||||
|
expect(c.fix).toContain('install pytest');
|
||||||
|
expect(c.fix).toContain('the diff is not at fault');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('Playwright without its browsers is the machine, not the diff', () => {
|
||||||
|
const SPEC = { 'package.json': JSON.stringify({ name: 'fixture', devDependencies: { '@playwright/test': '^1.48.0' } }) + '\n', 'tests/e2e/home.spec.ts': "import { test } from '@playwright/test';\ntest('home', async ({ page }) => { await page.goto('/'); });\ntest('about', async ({ page }) => { await page.goto('/about'); });\n" };
|
||||||
|
const NO_BROWSERS = ['Running 2 tests using 1 worker', ' x 1 tests/e2e/home.spec.ts:2:5 > home (12ms)', ' x 2 tests/e2e/home.spec.ts:3:5 > about (8ms)', '', ' 1) tests/e2e/home.spec.ts:2:5 > home', " Error: browserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/chromium_headless_shell-1169/chrome-linux/headless_shell", ' Looks like Playwright Test or Playwright was just installed or updated.', ' Please run the following command to download new browsers:', ' npx playwright install', ' 2 failed', ' tests/e2e/home.spec.ts:2:5 > home', ' tests/e2e/home.spec.ts:3:5 > about'];
|
||||||
|
const ASSERTION = ['Running 2 tests using 1 worker', ' 1) tests/e2e/home.spec.ts:2:5 > home', ' Error: expect(received).toBe(expected)', ' 1 failed', ' tests/e2e/home.spec.ts:2:5 > home', ' 1 passed (2.0s)'];
|
||||||
|
/** An `npx` first on PATH (and again through BASH_ENV, since the lane puts its Bun's directory first) that prints `lines` and exits 1. */
|
||||||
|
const npxShim = (repo: FixtureRepo, lines: string[]) => {
|
||||||
|
const dir = path.join(repo.root, 'npx-shim');
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(dir, 'out.txt'), lines.join('\n') + '\n');
|
||||||
|
fs.writeFileSync(path.join(dir, 'npx'), `#!/bin/sh\ncat '${path.join(dir, 'out.txt')}'\nexit 1\n`, { mode: 0o755 });
|
||||||
|
fs.writeFileSync(path.join(dir, 'path.sh'), `PATH='${dir}':"$PATH"\n`);
|
||||||
|
return { PATH: `${dir}:${process.env.PATH ?? ''}`, BASH_ENV: path.join(dir, 'path.sh') };
|
||||||
|
};
|
||||||
|
|
||||||
|
// Value: protects=a Playwright run whose every failure is a missing browser is PREGATE_ENVIRONMENT missing-binary naming
|
||||||
|
// `npx playwright install`, while an ordinary assertion failure stays the diff's; fails_when=the browsers case is PREGATE_LANE_FAILED
|
||||||
|
// "fix the in-branch failure" again (the #3598 misreading for Playwright); why_new=Playwright had no runner before #3629;
|
||||||
|
// seam=none; no_claim=a real Playwright install
|
||||||
|
test('every test failing to launch its browser is PREGATE_ENVIRONMENT; an assertion failure is PREGATE_LANE_FAILED', () => {
|
||||||
|
const repo = makeFixtureRepo({ pregate: null, baseFiles: SPEC }); repos.push(repo);
|
||||||
|
commit(repo, { 'tests/e2e/home.spec.ts': `${SPEC['tests/e2e/home.spec.ts']}// touched\n` });
|
||||||
|
const args = ['--base', 'main', '--allow-repo-commands', '--stage', 'tests', '--only', 'lanes'];
|
||||||
|
const r = runBin(repo, 'gstack-pregate', args, npxShim(repo, NO_BROWSERS));
|
||||||
|
expect(r.err).toContain('(PREGATE_ENVIRONMENT)');
|
||||||
|
const c = check(repo, 'lanes');
|
||||||
|
expect([c.status, c.code, c.cause, c.environment.kind]).toEqual(['incomplete', 'PREGATE_ENVIRONMENT', 'environment', 'missing-binary']);
|
||||||
|
expect(c.environment.evidence).toContain('Playwright browsers not installed (npx playwright install)');
|
||||||
|
expect(c.lines.join('\n')).toContain("playwright test --reporter=list '(?:^|/)tests/e2e/home\\.spec\\.ts$'");
|
||||||
|
runBin(repo, 'gstack-pregate', args, npxShim(repo, ASSERTION));
|
||||||
|
const d = check(repo, 'lanes');
|
||||||
|
expect([d.status, d.code]).toEqual(['fail', 'PREGATE_LANE_FAILED']);
|
||||||
|
expect(d.detail).toContain('1 failed of 2 ran');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('verify: a mixed incomplete record stays PREGATE_INCOMPLETE and names the environment-blocked check', () => {
|
describe('verify: a mixed incomplete record stays PREGATE_INCOMPLETE and names the environment-blocked check', () => {
|
||||||
test('one environment-caused check beside a plain incomplete one is not promoted to PREGATE_ENVIRONMENT', () => {
|
test('one environment-caused check beside a plain incomplete one is not promoted to PREGATE_ENVIRONMENT', () => {
|
||||||
const file: any = { schema_version: 1, tree: 'T', checks: [
|
const file: any = { schema_version: 1, tree: 'T', checks: [
|
||||||
|
|||||||
@@ -175,7 +175,7 @@ describe('gstack-pregate init drafts a policy from the repo (#3268)', () => {
|
|||||||
expect(r.out).toContain('PREGATE_CONFIG_WRITTEN: .gstack/pregate.json');
|
expect(r.out).toContain('PREGATE_CONFIG_WRITTEN: .gstack/pregate.json');
|
||||||
expect(r.out).toMatch(/PREGATE_INIT: languages python \d+, javascript \d+/);
|
expect(r.out).toMatch(/PREGATE_INIT: languages python \d+, javascript \d+/);
|
||||||
expect(r.out).toContain('python tests detected at run time: python3 -m pytest {files}');
|
expect(r.out).toContain('python tests detected at run time: python3 -m pytest {files}');
|
||||||
expect(r.out).toContain('a selection mixing these languages needs runner set');
|
expect(r.out).not.toContain('needs runner set');
|
||||||
expect(r.out).not.toContain('TODO src/version.js');
|
expect(r.out).not.toContain('TODO src/version.js');
|
||||||
expect(r.out).toContain('TODO tools/release.py: no test lane and no nearby test');
|
expect(r.out).toContain('TODO tools/release.py: no test lane and no nearby test');
|
||||||
expect(r.out).not.toContain('TODO orders/services.py');
|
expect(r.out).not.toContain('TODO orders/services.py');
|
||||||
|
|||||||
@@ -258,6 +258,46 @@ describe('literals (pure)', () => {
|
|||||||
expect(consumersOf(root, all, { name: 'TIMEOUT_MS', old: '5000', new: '9000', file: 'lib/retry.js' })).toEqual(['test/retry.test.ts:1', 'test/retry-path.test.ts:2', 'test/retry-bare.test.ts:2']);
|
expect(consumersOf(root, all, { name: 'TIMEOUT_MS', old: '5000', new: '9000', file: 'lib/retry.js' })).toEqual(['test/retry.test.ts:1', 'test/retry-path.test.ts:2', 'test/retry-bare.test.ts:2']);
|
||||||
fs.rmSync(root, { recursive: true, force: true });
|
fs.rmSync(root, { recursive: true, force: true });
|
||||||
});
|
});
|
||||||
|
// Value: protects=a one-character old value counts only in a statement that names the constant (its line, or lines that continue it),
|
||||||
|
// so comments, shell positionals and fixture bodies elsewhere in a file that names it are not consumers, while a formatter-split
|
||||||
|
// assertion still is; fails_when=any lone digit in such a file is reported again
|
||||||
|
// (#3492: five false hits on a wrapper 2 -> 3 bump, the same five on 3 -> 4 at v1.91.376.0); why_new=only file-level gating was
|
||||||
|
// tested; seam=none; no_claim=longer values (module-gated as before)
|
||||||
|
test('consumersOf: a one-character old value counts only in a statement that names the constant (#3492)', () => {
|
||||||
|
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-literals-bound-'));
|
||||||
|
const nl = String.fromCharCode(10);
|
||||||
|
const files: Record<string, string> = {
|
||||||
|
'scripts/install.sh': ['#!/usr/bin/env bash', '# handing the held lock over (GSTACK_DEMO_LOCK_HELD=1) and,', "# from revision 2, origin's VERSION read in this run", 'GSTACK_DEMO_WRAPPER=3', ''].join(nl),
|
||||||
|
'test/prose.test.ts': ["const rev = read(BOOT).match(/^GSTACK_DEMO_WRAPPER=(\\d+)$/m);", '', '// Field list 2 item 13: lanes relay only the trailer', ''].join(nl),
|
||||||
|
'test/stub.test.ts': ["const env = { GSTACK_DEMO_WRAPPER: '9' };", 'const script = [', " 'bash \"$1\" touch --root \"$2\" --pid $$ || exit 9',", '];', ''].join(nl),
|
||||||
|
'test/fixture.test.ts': ['// covers the GSTACK_DEMO_WRAPPER readers', "write('src/a.js', 'export default 1;');", "write('src/b.js', 'export default 2;');", ''].join(nl),
|
||||||
|
'test/same-line.test.ts': ['expect(GSTACK_DEMO_WRAPPER).toBe(2);', ''].join(nl),
|
||||||
|
'test/later.test.ts': ['// readers of the wrapper revision', "it('x', () => { const n = 2; });", 'expect(env.GSTACK_DEMO_WRAPPER).toBe(2);', ''].join(nl),
|
||||||
|
'test/split.test.ts': ['expect(read().GSTACK_DEMO_WRAPPER)', ' .toBe(2);', ''].join(nl),
|
||||||
|
'test/split-args.test.ts': ['assert.strictEqual(', ' mod.GSTACK_DEMO_WRAPPER,', ' 2,', ');', ''].join(nl),
|
||||||
|
'test/adjacent.test.ts': ['const rev = mod.GSTACK_DEMO_WRAPPER;', 'const two = 2;', ''].join(nl),
|
||||||
|
};
|
||||||
|
for (const [rel, text] of Object.entries(files)) { fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true }); fs.writeFileSync(path.join(root, rel), text); }
|
||||||
|
expect(consumersOf(root, Object.keys(files), { name: 'GSTACK_DEMO_WRAPPER', old: '2', new: '3', file: 'scripts/install.sh' })).toEqual(['test/same-line.test.ts:1', 'test/later.test.ts:3', 'test/split.test.ts:2', 'test/split-args.test.ts:3']);
|
||||||
|
fs.rmSync(root, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
// Value: protects=a one-character call-site literal (`attempts: 1`) counts only in a file that also names the changed module, on a line
|
||||||
|
// naming the call-site name, because `attempts`, `limit` and `cap` are ordinary words across a test tree; fails_when=any file with
|
||||||
|
// the word and a lone `1` is reported again (gstack-evals#97: `retry: { attempts: 1 }` -> `JUDGE_RETRY` raised 11 false hits,
|
||||||
|
// GSTA-2); why_new=call-site names had only the constant-name gate; seam=none; no_claim=constant names (previous test)
|
||||||
|
test('consumersOf: a one-character call-site literal needs the changed module in the file and the name on its line (gstack-evals#97)', () => {
|
||||||
|
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-literals-call-'));
|
||||||
|
const nl = String.fromCharCode(10);
|
||||||
|
const files: Record<string, string> = {
|
||||||
|
'tests/grid.test.ts': ['const grid = { tasks: ["a"], attempts: 1, n_concurrent: 4 };', ''].join(nl),
|
||||||
|
'tests/replay.test.ts': ['// attempts are counted per finding', 'expect(r.findings.length).toBe(1);', ''].join(nl),
|
||||||
|
'tests/proxy.test.ts': ["import { retryBudget } from '../lanes/proxy';", 'const cell = { trial: 1, attempt: 1 };', 'expect(retryBudget().attempts).toBeGreaterThan(0);', ''].join(nl),
|
||||||
|
'tests/judge-run.test.ts': ["import { run } from '../lanes/review-noloss/judge-run';", 'const n = 1;', 'expect(run({ retry: { attempts: 1 } }).sent).toBe(n);', ''].join(nl),
|
||||||
|
};
|
||||||
|
for (const [rel, text] of Object.entries(files)) { fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true }); fs.writeFileSync(path.join(root, rel), text); }
|
||||||
|
expect(consumersOf(root, Object.keys(files), { name: 'attempts', old: '1', new: '3', file: 'lanes/review-noloss/judge-run.ts' })).toEqual(['tests/judge-run.test.ts:3']);
|
||||||
|
fs.rmSync(root, { recursive: true, force: true });
|
||||||
|
});
|
||||||
test('consumersOf: a history row is skipped only when this branch added the new-value row to the same table; other old-value lines still count (GST-344)', () => {
|
test('consumersOf: a history row is skipped only when this branch added the new-value row to the same table; other old-value lines still count (GST-344)', () => {
|
||||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-literals-history-'));
|
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-literals-history-'));
|
||||||
const nl = String.fromCharCode(10);
|
const nl = String.fromCharCode(10);
|
||||||
@@ -301,6 +341,13 @@ describe('literals (pure)', () => {
|
|||||||
expect(parseSummary('Tests: 1 failed, 7 passed, 8 total')).toEqual({ ran: 8, failed: 1 });
|
expect(parseSummary('Tests: 1 failed, 7 passed, 8 total')).toEqual({ ran: 8, failed: 1 });
|
||||||
expect(parseSummary('===== 3 passed in 0.1s =====')).toEqual({ ran: 3, failed: 0 });
|
expect(parseSummary('===== 3 passed in 0.1s =====')).toEqual({ ran: 3, failed: 0 });
|
||||||
expect(parseSummary('No tests found')).toEqual({ ran: 0, failed: 0 });
|
expect(parseSummary('No tests found')).toEqual({ ran: 0, failed: 0 });
|
||||||
|
// Playwright lists the failed tests between its counts, so each count line is read on its own.
|
||||||
|
expect(parseSummary(' 2 failed\n [e2e] › a.spec.ts:9:1 › one\n [e2e] › b.spec.ts:4:1 › two\n 1 flaky\n [e2e] › c.spec.ts:2:1 › three\n 3 skipped\n 135 passed (3.0m)', 'npx playwright test {files}')).toEqual({ ran: 138, failed: 2 });
|
||||||
|
expect(parseSummary(' 137 passed (1.1m)', 'npx playwright test {files}')).toEqual({ ran: 137, failed: 0 });
|
||||||
|
expect(parseSummary(' 3 failed\n [e2e] › a.spec.ts:9:1 › one\n 1 interrupted\n 2 did not run', 'npx playwright test {files}')).toEqual({ ran: 4, failed: 4 });
|
||||||
|
expect(parseSummary(' 5 skipped', 'npx playwright test {files}')).toEqual({ ran: 0, failed: 0 });
|
||||||
|
// A test's own output that looks like a count is not the summary: the last count lines are.
|
||||||
|
expect(parseSummary(' 0 passed\n 1 failed\n [e2e] › a.spec.ts:9:1 › one\n 4 passed (2.0s)', 'npx playwright test {files}')).toEqual({ ran: 5, failed: 1 });
|
||||||
expect(parseSummary('ok')).toEqual({ ran: null, failed: null });
|
expect(parseSummary('ok')).toEqual({ ran: null, failed: null });
|
||||||
});
|
});
|
||||||
test('config: unknown keys, secrets in warn, a runner without {files} and an uncontained glob are PREGATE_CONFIG_INVALID', () => {
|
test('config: unknown keys, secrets in warn, a runner without {files} and an uncontained glob are PREGATE_CONFIG_INVALID', () => {
|
||||||
@@ -440,6 +487,19 @@ describe('gstack-pregate (bin)', () => {
|
|||||||
expect(pregate(repo, ['--stage', 'preflight', '--only', 'literals']).status).toBe(EXIT.ok);
|
expect(pregate(repo, ['--stage', 'preflight', '--only', 'literals']).status).toBe(EXIT.ok);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Value: protects=a real stale one-character constant still fails literals end to end after the #3492 narrowing, and an incidental
|
||||||
|
// digit in the same file is not what it reports; fails_when=the statement binding drops a named assertion; why_new=only
|
||||||
|
// multi-character values had a bin-level control; seam=none; no_claim=two-character values
|
||||||
|
test('#3492 control: a one-character constant change still fails literals on the test that asserts the old value by name', () => {
|
||||||
|
const repo = fixture();
|
||||||
|
commit(repo, { 'test/nightly/retries.test.js': "const test = require('node:test');\nconst assert = require('node:assert');\n// covers 3 retry shapes\nconst { MAX_RETRIES } = require('../../lib/retry.js');\ntest('retries', () => assert.strictEqual(\n MAX_RETRIES,\n 3,\n));\n" });
|
||||||
|
commit(repo, { 'lib/retry.js': "exports.MAX_RETRIES = 4;\nexports.TIMEOUT_MS = 5000;\nexports.retry = (fn) => fn();\n" });
|
||||||
|
const r = pregate(repo, ['--stage', 'preflight', '--only', 'literals']);
|
||||||
|
expect(r.status).toBe(EXIT.fail);
|
||||||
|
expect(r.out).toContain('literals: test/nightly/retries.test.js:7: MAX_RETRIES still 3 (now 4 in lib/retry.js)');
|
||||||
|
expect(r.err).toContain('(PREGATE_LITERALS)');
|
||||||
|
});
|
||||||
|
|
||||||
test('GST-344: a history table the branch extends passes literals through the merge-base read; one it does not extend fails', () => {
|
test('GST-344: a history table the branch extends passes literals through the merge-base read; one it does not extend fails', () => {
|
||||||
const history = (...ns: number[]) => ["const { MAX_RETRIES } = require('../lib/retry.js');", 'const versions = [', ...ns.map(n => ` [${n}, 'digest-${n}'],`), '];', ''].join('\n');
|
const history = (...ns: number[]) => ["const { MAX_RETRIES } = require('../lib/retry.js');", 'const versions = [', ...ns.map(n => ` [${n}, 'digest-${n}'],`), '];', ''].join('\n');
|
||||||
const repo = fixture({ baseFiles: { 'test/history.test.js': history(2, 3) } });
|
const repo = fixture({ baseFiles: { 'test/history.test.js': history(2, 3) } });
|
||||||
@@ -767,6 +827,78 @@ describe('nested JS packages run their own runner from their own directory', ()
|
|||||||
expect(planRuns(ctx, ['app/src/sum.test.ts']).groups).toEqual([{ runner: 'npx vitest run {files}', cwd: 'app', files: ['src/sum.test.ts'] }]);
|
expect(planRuns(ctx, ['app/src/sum.test.ts']).groups).toEqual([{ runner: 'npx vitest run {files}', cwd: 'app', files: ['src/sum.test.ts'] }]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
/** `env` with a `python3` first on PATH whose `-c` probe exits `probeExit` (0: pytest importable, 3: not). */
|
||||||
|
const withPython = (repo: FixtureRepo, env: { PATH: string; BASH_ENV: string }, probeExit: number) => {
|
||||||
|
const dir = path.join(repo.root, `py-${probeExit}`);
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(dir, 'python3'), `#!/bin/sh\n[ "$1" = "-c" ] && exit ${probeExit}\nexit 3\n`, { mode: 0o755 });
|
||||||
|
return { ...env, PATH: `${dir}:${env.PATH}` };
|
||||||
|
};
|
||||||
|
|
||||||
|
// Value: protects=root-owned tests in two languages each get their language's runner instead of the whole lane going incomplete, and a
|
||||||
|
// language whose runner this machine lacks is named while the other still runs; fails_when=detection again needs one language
|
||||||
|
// across every root file (#3600: `.mjs` + `.py` was PREGATE_INCOMPLETE asking for one `runner` for both); why_new=only
|
||||||
|
// single-language root selections were tested; seam=none; no_claim=the runners themselves (plan only)
|
||||||
|
test('root tests in two languages plan one group per language; a file no runner claims stays unclaimed', () => {
|
||||||
|
const repo = fixture({ pregate: null, baseFiles: APP });
|
||||||
|
const ctx = ctxFor(repo, withPython(repo, pmShim(repo), 0));
|
||||||
|
const mixed = ['scripts/doku-gate.test.mjs', 'scripts/test_assets.py', 'scripts/version-gate.test.mjs'];
|
||||||
|
expect(planRuns(ctx, mixed)).toEqual({ groups: [
|
||||||
|
{ runner: 'node --test {files}', cwd: '', files: ['scripts/doku-gate.test.mjs', 'scripts/version-gate.test.mjs'] },
|
||||||
|
{ runner: 'python3 -m pytest {files}', cwd: '', files: ['scripts/test_assets.py'] },
|
||||||
|
], unclaimed: [], missing: [], noPubspec: [] });
|
||||||
|
expect(planRuns(ctx, ['scripts/doku-gate.test.mjs', 'test/run.bats'])).toEqual({ groups: [
|
||||||
|
{ runner: 'node --test {files}', cwd: '', files: ['scripts/doku-gate.test.mjs'] },
|
||||||
|
], unclaimed: ['test/run.bats'], missing: [], noPubspec: [] });
|
||||||
|
expect(planRuns(ctxFor(repo, withPython(repo, pmShim(repo), 3)), mixed)).toEqual({ groups: [
|
||||||
|
{ runner: 'node --test {files}', cwd: '', files: ['scripts/doku-gate.test.mjs', 'scripts/version-gate.test.mjs'] },
|
||||||
|
], unclaimed: ['scripts/test_assets.py'], missing: ['pytest'], noPubspec: [] });
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=a repo-wide bun or vitest claims only JS/TS files, so a root `.py` test beside them gets pytest instead of being handed to
|
||||||
|
// the JS runner; fails_when=bun.lock or a vitest dependency again takes every extension; why_new=the split was only tested with no
|
||||||
|
// repo runner; seam=none; no_claim=the runners themselves (plan only)
|
||||||
|
test('a repo-wide bun or vitest runner claims JS/TS only; a root .py test beside it gets pytest; parts with one runner run together', () => {
|
||||||
|
const repo = fixture({ pregate: null, baseFiles: APP });
|
||||||
|
const ctx = ctxFor(repo, withPython(repo, pmShim(repo), 0));
|
||||||
|
write(repo.work, 'package.json', JSON.stringify({ name: 'fixture', devDependencies: { vitest: '^3.2.0' } }) + '\n');
|
||||||
|
expect(planRuns(ctx, ['test/a.test.ts', 'scripts/b.test.mjs', 'scripts/test_c.py', 'test/run.bats'])).toEqual({ groups: [
|
||||||
|
{ runner: 'npx vitest run {files}', cwd: '', files: ['test/a.test.ts', 'scripts/b.test.mjs'] },
|
||||||
|
{ runner: 'python3 -m pytest {files}', cwd: '', files: ['scripts/test_c.py'] },
|
||||||
|
], unclaimed: ['test/run.bats'], missing: [], noPubspec: [] });
|
||||||
|
expect(planRuns(ctx, ['test/a.test.ts', 'scripts/b.test.mjs']).groups).toEqual([{ runner: 'npx vitest run {files}', cwd: '', files: ['test/a.test.ts', 'scripts/b.test.mjs'] }]);
|
||||||
|
write(repo.work, 'bun.lock', '');
|
||||||
|
expect(planRuns(ctx, ['test/a.test.ts', 'scripts/test_c.py']).groups).toEqual([
|
||||||
|
{ runner: 'bun test {files}', cwd: '', files: ['test/a.test.ts'] },
|
||||||
|
{ runner: 'python3 -m pytest {files}', cwd: '', files: ['scripts/test_c.py'] },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Value: protects=a Playwright spec in a repo whose unit runner is vitest runs through `playwright test`, never vitest (which finds
|
||||||
|
// no test files and exits 1 on a diff that touches only specs); fails_when=the repo-wide runner takes every JS/TS test again (#3629);
|
||||||
|
// why_new=no detection knew Playwright; seam=none; no_claim=a real Playwright run (plan only)
|
||||||
|
test('a spec importing @playwright/test runs through playwright test from its package; the other tests keep their runner', () => {
|
||||||
|
const repo = fixture({ pregate: null, baseFiles: APP });
|
||||||
|
const ctx = ctxFor(repo, pmShim(repo));
|
||||||
|
write(repo.work, 'package.json', JSON.stringify({ name: 'fixture', devDependencies: { vitest: '^3.2.0', '@playwright/test': '^1.48.0' } }) + '\n');
|
||||||
|
write(repo.work, 'tests/e2e/home.spec.ts', "import { expect, test } from '@playwright/test';\ntest('home', async ({ page }) => expect(page).toBeTruthy());\n");
|
||||||
|
write(repo.work, 'app/e2e/app.spec.ts', "import {\n test, // the runner's fixture\n /* it's */ expect,\n} from \"@playwright/test\";\ntest('app', () => {});\n");
|
||||||
|
write(repo.work, 'tests/e2e/tight.spec.ts', "import{test}from'@playwright/test';\ntest('tight', () => {});\n");
|
||||||
|
write(repo.work, 'test/pad.test.js', 'import' + ' '.repeat(5_000) + '\n');
|
||||||
|
write(repo.work, 'test/util.test.js', "import { test } from 'vitest';\nimport type { Page } from '@playwright/test';\nimport type {\n Locator,\n} from '@playwright/test';\nimport { type BrowserContext } from '@playwright/test';\nconst spec = \"import { test } from '@playwright/test';\";\ntest('fixture text and type imports only', () => {});\n");
|
||||||
|
expect(planRuns(ctx, ['tests/e2e/home.spec.ts', 'tests/e2e/tight.spec.ts', 'test/util.test.js', 'test/pad.test.js', 'app/src/sum.test.ts', 'app/e2e/app.spec.ts'])).toEqual({ groups: [
|
||||||
|
{ runner: 'npx vitest run {files}', cwd: '', files: ['test/util.test.js', 'test/pad.test.js'] },
|
||||||
|
{ runner: 'pnpm exec vitest run {files}', cwd: 'app', files: ['src/sum.test.ts'] },
|
||||||
|
{ runner: 'npx playwright test --reporter=list {files}', cwd: '', files: ['tests/e2e/home.spec.ts', 'tests/e2e/tight.spec.ts'] },
|
||||||
|
{ runner: 'pnpm exec playwright test --reporter=list {files}', cwd: 'app', files: ['e2e/app.spec.ts'] },
|
||||||
|
], unclaimed: [], missing: [], noPubspec: [] });
|
||||||
|
// A declared runner still takes every file, Playwright specs included.
|
||||||
|
const declared = { ...ctx, config: { ...DEFAULT_CONFIG, runner: 'bun test {files}' } } as PregateContext;
|
||||||
|
expect(planRuns(declared, ['tests/e2e/home.spec.ts']).groups).toEqual([{ runner: 'bun test {files}', cwd: '', files: ['tests/e2e/home.spec.ts'] }]);
|
||||||
|
// A package manager missing from PATH leaves the spec unclaimed naming it.
|
||||||
|
expect(planRuns(ctxFor(repo, { PATH: path.join(repo.root, 'empty-path') }), ['app/e2e/app.spec.ts'])).toEqual({ groups: [], unclaimed: ['app/e2e/app.spec.ts'], missing: ['pnpm'], noPubspec: [] });
|
||||||
|
});
|
||||||
|
|
||||||
// Value: protects=repos where root detection works today are unchanged: a nested package that declares no test runner stays with the
|
// Value: protects=repos where root detection works today are unchanged: a nested package that declares no test runner stays with the
|
||||||
// root runner from the root, a root vitest stays `npx vitest run` from the root, and a declared runner takes every file;
|
// root runner from the root, a root vitest stays `npx vitest run` from the root, and a declared runner takes every file;
|
||||||
// fails_when=every nested package.json splits a group or the root command changes; why_new=negative controls for the new
|
// fails_when=every nested package.json splits a group or the root command changes; why_new=negative controls for the new
|
||||||
|
|||||||
Reference in new issue
Block a user