mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-10 18:50:18 +01:00
pregate: a diff cannot clear itself as an environment failure; a taken --run id gets its own directory
Adversarial review of the environment classifier and the scratch layout: - exit 127 and 'command not found' are missing-binary only when the command's own binary is off PATH (runner, guard and remote gate alike); a script the diff controls that exits 127 is the diff's failure - a local lane with no test summary is classified from its output only when the runner was not seen starting a test file; a remote gate only from its first 40 lines, before any test output could stream - environment.proof records launch (certain) or output (inferred); an output-inferred cause never says 'the diff is not at fault', its line says '(inferred from output)', and verify names such checks - a runner killed by a signal is PREGATE_INCOMPLETE, not missing-binary - a --run id whose directory exists takes <id>-2 .. -9 instead of sharing (and truncating) a live run's logs - engine.* findings are never allowlistable; a pattern entry the scanner gave no position for is reported as not tested instead of silently inert - docs: synopsis --run, no-policy lanes wording, verify's code list, guard exit 127 carve-out, PREGATE_SECRETS allowlist pointer, scratch Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
e890d0d225
commit
f2a3246905
18 files changed
+299
-103
No files matched your search
+3
-3
@@ -6,9 +6,9 @@
|
||||
|
||||
### Added
|
||||
- **No policy → warn plus a starter, never a hard `PREGATE_NO_LANE`** (`lib/pregate/starter.ts`, `lib/pregate/checks/lanes.ts`). When the base branch carries no `.gstack/pregate.json` and no user-owned policy exists, a touched file with no lane is a `warn` row under the new code `PREGATE_NO_POLICY`; the selected tests still run (a red lane is still `fail`, an unknown platform matrix is still a `requires-remote` obligation), and the table prints a starter policy drafted from the repository: the detected runner, the test directories, the CI lanes (naming a planner-computed matrix that needs `selection`), the JSON between `PREGATE_STARTER: begin` and `end` with a `dependencies` entry per unmapped touched file mapped to the tests nearest it or sharing its stem (never `[]`), and a `TODO` line per file nothing covers. `run` writes nothing; `gstack-pregate init` is the explicit write. Once any policy exists (base copy, user-owned file, or `--policy-from head`), an unmapped file is `PREGATE_NO_LANE` as before.
|
||||
- **`allowlist` in `.gstack/pregate.json`** (`lib/pregate/allowlist.ts`, `lib/pregate/config.ts`, `lib/pregate/checks/secrets.ts`): `[{ finding, paths?, pattern?, reason }]`, where `finding` is a gstack-redact finding id or one of the categories `pii`, `internal`, `hygiene`, `legal`; `paths` are contained globs; `pattern` is a regex matched at the finding's position in the added line and ending at a token boundary; one scope is required and `reason` always is (64 entries at most). The `secrets` row says `N allowlisted (<ids>)`, lists each allowlisted finding with its entry and reason, and records the entries' digest as `inputs.allowlist`. **`env.value` (a credential-named environment variable's real value in the added lines, #3449) and the whole `secret` category can never be allowlisted:** validation refuses the entry (`PREGATE_CONFIG_INVALID` at `allowlist[i].finding`) and the matcher never consults the list for `env.value`, so a stale or hand-edited policy cannot silence it. An unparsed scanner report keeps its exit-code verdict.
|
||||
- **Environment failures reported apart from diff failures** (`lib/pregate/environment.ts`; new code `PREGATE_ENVIRONMENT` with the troubleshooting anchor `pregate-environment`). A runner or guard binary missing from PATH (exit 127, `command not found`, a spawn `ENOENT`, a tool `not on PATH`), a network error or a quota refusal before any test was counted, a full disk, an unverified runtime or gstack's own scanner not finishing is `incomplete` with `cause: "environment"` and `environment: { kind, evidence }` in `pregate.json`, an `environment: <kind> — <evidence>` table line and a fix that says the diff is not at fault; `PREGATE_LANE_FAILED`, `PREGATE_GUARD_FAILED` and `PREGATE_SECRETS` carry `cause: "diff"`. The classifier is conservative: a local lane that exited non-zero is read from its output only when its runner reported no test summary; a counted failure is the diff's whatever its log says; a remote gate is environment only when it printed no `PREGATE_REMOTE:` line; stalls and timeouts keep their codes. `gstack-pregate verify` prints `PREGATE_ENVIRONMENT: <n> check(s) blocked by the environment (<kinds>), not by the diff` when every incomplete check is environment-caused and names the kinds beside `PREGATE_INCOMPLETE` otherwise, so a lane waives by cause, never by test name.
|
||||
- **One scratch directory per run under the state root** (`lib/pregate/run-scratch.ts`): `<state root>/projects/<slug>/pregate/runs/<run id>/` (0700; `--run <id>` names it, else `<UTC stamp>-<pid>-<4 hex>`) holds the secrets scan's added-line copies (deleted after the scan), `lanes/lanes.log`, `guards/<id>.log` and the remote gate's log and file list, never `os.tmpdir()`. `pregate.json` records it as `scratch`, the table prints `scratch: <dir>`, `--explain` names it; runs older than 7 days under the same project are pruned when a new run starts; an unwritable state root falls back to a private per-run directory under `os.tmpdir()` and says so.
|
||||
- **`allowlist` in `.gstack/pregate.json`** (`lib/pregate/allowlist.ts`, `lib/pregate/config.ts`, `lib/pregate/checks/secrets.ts`): `[{ finding, paths?, pattern?, reason }]`, where `finding` is a gstack-redact finding id or one of the categories `pii`, `internal`, `hygiene`, `legal`; `paths` are contained globs; `pattern` is a regex matched at the finding's position in the added line and ending at a token boundary; one scope is required and `reason` always is (64 entries at most). The `secrets` row says `N allowlisted (<ids>)`, lists each allowlisted finding with its entry and reason, and records the entries' digest as `inputs.allowlist`. **`env.value` (a credential-named environment variable's real value in the added lines, #3449) and the whole `secret` category can never be allowlisted:** validation refuses the entry (`PREGATE_CONFIG_INVALID` at `allowlist[i].finding`) and the matcher never consults the list for `env.value`, so a stale or hand-edited policy cannot silence it; the scanner's own `engine.*` refusals (an oversize input fails closed) are refused and kept the same way. A `pattern` entry that named a finding the scanner gave no position for is reported as not tested (the finding is kept) instead of silently inert. An unparsed scanner report keeps its exit-code verdict.
|
||||
- **Environment failures reported apart from diff failures** (`lib/pregate/environment.ts`; new code `PREGATE_ENVIRONMENT` with the troubleshooting anchor `pregate-environment`). A runner or guard binary missing from PATH (exit 127, `command not found`, a spawn `ENOENT`, a tool `not on PATH`), a network error or a quota refusal before any test was counted, a full disk, an unverified runtime or gstack's own scanner not finishing is `incomplete` with `cause: "environment"` and `environment: { kind, evidence }` in `pregate.json`, an `environment: <kind> — <evidence>` table line and a fix; `PREGATE_LANE_FAILED`, `PREGATE_GUARD_FAILED` and `PREGATE_SECRETS` carry `cause: "diff"`. `environment.proof` records how the cause was established: `launch` (a spawn failure, a binary off PATH, a runtime or tool refusal; the fix ends "the diff is not at fault") or `output` (a rule matched the command's text before any test was counted, which the diff's own output could have written; the table line says `(inferred from output)` and the fix asks for confirmation on another machine or with `--remote`). The classifier is conservative: a local lane that exited non-zero is read from its output only when its runner reported no test summary and was not seen starting a test file; a counted failure is the diff's whatever its log says; exit 127 and `command not found` are `missing-binary` only when the command's own binary is off PATH, so a runner, guard or gate script the diff controls that exits 127 is the diff's; a remote gate is environment only when it printed no `PREGATE_REMOTE:` line and the cause sits in its first 40 lines, before any test output could stream; a runner killed by a signal, stalls and timeouts keep `PREGATE_INCOMPLETE` / `PREGATE_STALLED`. `gstack-pregate verify` prints `PREGATE_ENVIRONMENT: <n> check(s) blocked by the environment (<kinds>), not by the diff` when every incomplete check is environment-caused, naming any check inferred from output alone, and names the kinds beside `PREGATE_INCOMPLETE` otherwise, so a lane waives by cause and proof, never by test name.
|
||||
- **One scratch directory per run under the state root** (`lib/pregate/run-scratch.ts`): `<state root>/projects/<slug>/pregate/runs/<run id>/` (0700; `--run <id>` names it, else `<UTC stamp>-<pid>-<4 hex>`) holds the secrets scan's added-line copies (deleted after the scan), `lanes/lanes.log`, `guards/<id>.log` and the remote gate's log and file list, never `os.tmpdir()`. `pregate.json` records it as `scratch`, the table prints `scratch: <dir>`, a taken `--run` name takes a numbered sibling (`<id>-2` … `-9`) rather than sharing the directory; runs older than 7 days under the same project are pruned when a new run starts; an unwritable state root falls back to a private per-run directory under `os.tmpdir()` and says so.
|
||||
|
||||
### Fixed
|
||||
- **`literals` no longer floods on a common module stem or a one-digit value** (`lib/pregate/checks/literals.ts` `consumersOf`). A test file counted as a consumer of a changed constant when it merely contained the source file's stem as a bare word (`install`, `retry`), so `GSTACK_CAPY_WRAPPER=2 → 3` in `capy/skill/gstack/scripts/install.sh` flagged 575 test files (`PREGATE_LITERALS`; GSTA-114, #3475). The module now counts only as a path fragment (`/install.sh`, `/retry.js`, `/retry'`), and an old value shorter than two characters needs the constant's name in the file. The same edit now flags the 5 files that name `GSTACK_CAPY_WRAPPER`. Regression: `test/pregate-selection.test.ts` "consumersOf: a module stem is a path fragment…".
|
||||
|
||||
+1
-1
@@ -75,7 +75,7 @@ const HELP = `gstack-pregate — cheap checks before anything paid or remote (do
|
||||
PREGATE_REMOTE: tree=<tested tree> tests=<n> failed=0 line with n > 0
|
||||
--spend-cap <usd> the cap --remote passes to the gate (default \$GSTACK_SPEND_CAP_USD); spend is reported as
|
||||
spent, reserved and unknown
|
||||
--run <id> the run label recorded in pregate.json and the name of this run's scratch directory
|
||||
--run <id> the run label recorded in pregate.json and the name of this run's scratch directory (a taken name gets a numbered sibling, <id>-2 … -9)
|
||||
(<state root>/projects/<slug>/pregate/runs/<id>: lane and guard logs, scan files; default a unique id)
|
||||
--explain print each selected test's provenance and each uncertain mapping
|
||||
--json one JSON object on stdout; a refusal before any check runs (unavailable base, invalid pin
|
||||
|
||||
+53
-23
@@ -136,7 +136,7 @@ Review each entry before committing the file on the base branch.
|
||||
|
||||
```
|
||||
gstack-pregate [--stage preflight|tests|all] [--only <check,..>] [--allow-repo-commands] [--policy-from base|head]
|
||||
[--timeout <s>] [--stall-timeout <s>] [--remote --spend-cap <usd>] [--explain] [--json]
|
||||
[--timeout <s>] [--stall-timeout <s>] [--remote --spend-cap <usd>] [--run <id>] [--explain] [--json]
|
||||
gstack-pregate verify | clear --remote <workflow.yml/job> --run-url <url> | lanes | validate | init
|
||||
```
|
||||
|
||||
@@ -168,7 +168,9 @@ Two stages with separate clocks, both printed:
|
||||
touched file (renames included) runs its `command` in its own process, under
|
||||
the verified launch Bun, with its own `timeout_s` (default `--timeout`). A
|
||||
non-zero exit fails pregate naming the guard and its `fix`
|
||||
(`PREGATE_GUARD_FAILED`); a timeout is `incomplete`; without
|
||||
(`PREGATE_GUARD_FAILED`); a guard that could not start, or whose own binary
|
||||
is off PATH (exit 127), is the machine's, `PREGATE_ENVIRONMENT`
|
||||
(`incomplete`), not a guard failure; a timeout is `incomplete`; without
|
||||
`--allow-repo-commands` the selected guards are listed and the check is
|
||||
`incomplete`. Guards are read from the base branch's copy (or the
|
||||
user-owned file when the base carries none) and cannot be
|
||||
@@ -197,8 +199,11 @@ Two stages with separate clocks, both printed:
|
||||
`baseUrl`) tries the exact path, then `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`,
|
||||
`.cjs`, `.mts`, `.cts`, then `index.ts`, `index.tsx`, `index.js`, `index.jsx`
|
||||
and `index.mjs` inside the directory; the first file that exists wins.
|
||||
A touched source file with no lane is `FAIL lanes — <file> has no test lane`,
|
||||
never a pass. Markdown that no test depends on is a docs/no-tests empty lane
|
||||
A touched source file with no lane is `FAIL lanes — <file> has no test lane`
|
||||
once any policy exists (the base copy, the user-owned file or `--policy-from
|
||||
head`); with no policy anywhere it is the `warn` `PREGATE_NO_POLICY` with a
|
||||
starter printed ([no policy](#no-policy)); never a pass. Markdown that no
|
||||
test depends on is a docs/no-tests empty lane
|
||||
instead (`docs/no-tests: N documentation file(s)`), including when the base
|
||||
branch has no `.gstack/pregate.json`. The dependency is the same import graph
|
||||
as every other file, so a test that reads the Markdown (a fixture or a doc)
|
||||
@@ -286,7 +291,8 @@ The record is `.gstack/tmp/pregate.json` (`gstack-artifact validate --as pregate
|
||||
**Publication.** `gstack-ship-receipt write --pregate .gstack/tmp/pregate.json`
|
||||
(Step 18) refuses unless the record is current for the tree being published
|
||||
(`PREGATE_STALE`: rerun after the final stamp), green (`PREGATE_LANE_FAILED`,
|
||||
`PREGATE_INCOMPLETE`) and every `requires-remote` obligation carries a receipt
|
||||
`PREGATE_INCOMPLETE`, or `PREGATE_ENVIRONMENT` when every incomplete check is
|
||||
the machine's) and every `requires-remote` obligation carries a receipt
|
||||
(`PREGATE_REMOTE_UNCLEARED`: dispatch the job, then
|
||||
`gstack-pregate clear --remote <workflow.yml/job> --run-url <url>`). The ship
|
||||
receipt then carries `pregate` and `requires_remote`. `gstack-pregate verify`
|
||||
@@ -475,7 +481,13 @@ those, each with its scope and its reason:
|
||||
credential-named variable of the environment pregate ran in (#3449), found in
|
||||
the added lines whatever its shape. Validation refuses an entry naming it
|
||||
(`PREGATE_CONFIG_INVALID`), and the matcher never consults the list for it,
|
||||
so a stale or hand-edited policy cannot silence it either.
|
||||
so a stale or hand-edited policy cannot silence it either. The scanner's own
|
||||
`engine.*` refusals (an oversize input fails closed) are refused and kept the
|
||||
same way: they are the scanner declining its input, not a finding in it.
|
||||
- A `pattern` entry needs the finding's position. When the scanner gave none,
|
||||
the entry is inert for that finding and the row says so (`allowlist[0]
|
||||
env.kv not tested on env.kv (<file>): the scanner gave no position for its
|
||||
pattern; the finding is kept`) rather than silently keeping it.
|
||||
|
||||
The `secrets` row says `N allowlisted (<ids>)` and lists each allowlisted
|
||||
finding with the entry (`allowlisted env.kv (evals/receipts/t1.json, added line
|
||||
@@ -493,7 +505,7 @@ its command because of the machine (truther #617: a root runner with a vitest
|
||||
config that found no file, a corpus fetch that stalled on one host and passed
|
||||
on another), the check is `incomplete` under its own code,
|
||||
**`PREGATE_ENVIRONMENT`**, with `cause: "environment"` and an
|
||||
`environment: { kind, evidence }` object in `pregate.json`, an
|
||||
`environment: { kind, evidence, proof }` object in `pregate.json`, an
|
||||
`environment: <kind> — <evidence>` table line, and a fix that says so. A
|
||||
failure the branch caused carries `cause: "diff"` (`PREGATE_LANE_FAILED`,
|
||||
`PREGATE_GUARD_FAILED`, `PREGATE_SECRETS`). Kinds:
|
||||
@@ -508,23 +520,38 @@ failure the branch caused carries `cause: "diff"` (`PREGATE_LANE_FAILED`,
|
||||
| `tool` | gstack's own scanner (gstack-redact) did not finish or exited outside its contract |
|
||||
|
||||
The classifier is conservative. A local lane that exited non-zero is read from
|
||||
its output **only when its runner reported no test summary** (nothing ran); a
|
||||
run whose summary counts `N ran, M failed` is the diff's failure whatever its
|
||||
log says, so a test that failed on `ECONNREFUSED` is still a red test. A
|
||||
remote gate that exited without a `PREGATE_REMOTE:` line and whose output
|
||||
names a quota or network error never ran the tests, so it is environment and
|
||||
the obligation stays uncleared. A guard that ran and said no is the diff's; a
|
||||
guard whose command the shell could not find is the machine's. A stall or a
|
||||
timeout keeps `PREGATE_STALLED` / `PREGATE_INCOMPLETE`: a hang has no
|
||||
single cause.
|
||||
its output **only when its runner reported no test summary** (nothing ran) and
|
||||
the runner was not seen starting a test file; a run whose summary counts `N
|
||||
ran, M failed` is the diff's failure whatever its log says, so a test that
|
||||
failed on `ECONNREFUSED` is still a red test. A remote gate that exited
|
||||
without a `PREGATE_REMOTE:` line is read from its **preamble only** (its first
|
||||
40 lines, before any test output could stream): a VM refused for quota or an
|
||||
unreachable API there never ran the tests, so it is environment and the
|
||||
obligation stays uncleared, while a `quota` in a streamed test name is not. A
|
||||
guard that ran and said no is the diff's. Exit 127 and `command not found`
|
||||
count as `missing-binary` only when the command's **own** binary is off
|
||||
PATH: a runner, guard or gate script the diff controls that exits 127 (a
|
||||
package script naming a tool that is not installed) is the diff's failure. A
|
||||
stall or a timeout keeps `PREGATE_STALLED` / `PREGATE_INCOMPLETE`: a hang
|
||||
has no single cause, and so does a runner killed by a signal.
|
||||
|
||||
`proof` records how the cause was established. `launch` is certain: the
|
||||
command could not start, its binary is off PATH, the runtime or a gstack tool
|
||||
refused; the fix ends `the diff is not at fault`. `output` means a rule
|
||||
matched the command's text before any test was counted, which the diff's own
|
||||
output could have written (a test that prints `fetch failed` and exits the
|
||||
process before the summary); the table line says `(inferred from output)` and
|
||||
the fix asks for confirmation on another machine or with `--remote` before
|
||||
the diff is treated as clear. Neither proof ever passes the check.
|
||||
|
||||
`gstack-pregate verify` reports `PREGATE_ENVIRONMENT: <n> check(s) blocked by
|
||||
the environment (<kinds>), not by the diff` when every incomplete check is
|
||||
environment-caused, and names the kinds beside the generic
|
||||
`PREGATE_INCOMPLETE` otherwise; `--json` carries `cause` and `environment`
|
||||
per check. A lane that waives such a run cites the kind and the evidence, not a
|
||||
test name, and clears the lane remotely (`--remote`, or `gstack-pregate clear`
|
||||
with a run URL) rather than on the broken machine. Incomplete never passes.
|
||||
environment-caused, naming any check whose cause was inferred from output
|
||||
alone, and names the kinds beside the generic `PREGATE_INCOMPLETE` otherwise;
|
||||
`--json` carries `cause` and `environment` per check. A lane that waives such
|
||||
a run cites the kind, the evidence and the proof, not a test name, and clears
|
||||
the lane remotely (`--remote`, or `gstack-pregate clear` with a run URL)
|
||||
rather than on the broken machine. Incomplete never passes.
|
||||
|
||||
<a id="scratch"></a>
|
||||
## Scratch: one directory per run, under the state root
|
||||
@@ -537,8 +564,11 @@ guard's log (`guards/<id>.log`) and the remote gate's log and file list
|
||||
(`remote/`) live there, never under `os.tmpdir()`: two pregates in parallel
|
||||
(two lanes on one machine, a subagent beside its parent) never share a path,
|
||||
and the log a `fix:` line names survives a host that wipes `/tmp` between
|
||||
wakes. `pregate.json` records the directory as `scratch`, the table prints
|
||||
`scratch: <dir>`, and `--explain` names it. Runs older than 7 days under the
|
||||
wakes. `pregate.json` records the directory as `scratch` and the table prints
|
||||
`scratch: <dir>`. A `--run <id>` whose directory already exists (a finished run
|
||||
under the same name, or a second run beside a live one) is never shared, since
|
||||
a second run would truncate the first's logs: the run takes `<id>-2` … `-9`
|
||||
and the record names the directory it got. Runs older than 7 days under the
|
||||
same project are pruned when a new run starts (the current run never is). An
|
||||
unwritable state root falls back to a private directory under `os.tmpdir()`,
|
||||
still one per run, and `--explain` says so.
|
||||
|
||||
+21
-7
@@ -2997,7 +2997,11 @@ names it).
|
||||
`secret` category, in the lines this branch adds.
|
||||
|
||||
**Fix.** Rotate the credential, remove it from the added lines (history
|
||||
included), rerun. Not downgradable.
|
||||
included), rerun. Not downgradable. A benign finding (a dummy token, a run id
|
||||
read as a phone number, a vendor's public key) is named in the policy's
|
||||
`allowlist` by finding id or non-secret category with a path glob or an
|
||||
anchored pattern and a reason ([docs/pregate.md](pregate.md#allowlist));
|
||||
`env.value`, the whole `secret` category and `engine.*` never.
|
||||
|
||||
<a id="pregate-strays"></a>
|
||||
### `PREGATE_STRAYS`
|
||||
@@ -3141,20 +3145,27 @@ diff: a runner or guard binary missing from PATH (exit 127, `command not
|
||||
found`, a tool `not on PATH`), a network error or a quota refusal before any
|
||||
test was counted, a full disk, an unverified runtime, or gstack's own scanner
|
||||
not finishing. The check is `incomplete` (never pass) with `cause:
|
||||
"environment"` and `environment: { kind, evidence }` in `pregate.json`, and
|
||||
the table prints `environment: <kind> — <evidence>`. A test that failed and
|
||||
"environment"` and `environment: { kind, evidence, proof }` in `pregate.json`,
|
||||
and the table prints `environment: <kind> — <evidence>`. A test that failed and
|
||||
whose log also names a connection error is **not** this code: the runner's own
|
||||
summary counted a failure, so it is `PREGATE_LANE_FAILED` with `cause:
|
||||
"diff"`. Kinds and the rule: [docs/pregate.md](pregate.md#environment).
|
||||
"diff"`; nor is a runner, guard or gate script whose own binary is present but
|
||||
exits 127. `proof: "launch"` is certain (the command could not start, its
|
||||
binary is off PATH, the runtime or a gstack tool refused); `proof: "output"`
|
||||
means a rule matched the command's text before any test was counted, which the
|
||||
diff's own output could have written, so the line says `(inferred from output)`
|
||||
and the fix asks for confirmation elsewhere. Kinds and the rule:
|
||||
[docs/pregate.md](pregate.md#environment).
|
||||
|
||||
**Fix.** Repair what the `environment:` line names on this machine (install the
|
||||
tool, restore the network or the quota, free the disk, repair the runtime), or
|
||||
run the lane where it can run: `gstack-pregate --stage tests --remote
|
||||
--spend-cap <usd> --allow-repo-commands`, or the CI job and `gstack-pregate
|
||||
clear --remote <workflow.yml/job> --run-url <url>`. A lane that waives the run
|
||||
cites the kind and the evidence, never a test name; `verify` prints
|
||||
cites the kind, the evidence and the proof, never a test name; `verify` prints
|
||||
`PREGATE_ENVIRONMENT: <n> check(s) blocked by the environment (<kinds>), not by
|
||||
the diff` so a coordinator can tell the two apart.
|
||||
the diff`, naming any check inferred from output alone, so a coordinator can
|
||||
tell the two apart and knows which to confirm elsewhere first.
|
||||
|
||||
<a id="pregate-stale"></a>
|
||||
### `PREGATE_STALE`
|
||||
@@ -3397,7 +3408,10 @@ CLI golden test) from the base branch's `.gstack/pregate.json`, or from the
|
||||
user-owned file when the base carries none, matched a
|
||||
file the branch touched and exited non-zero. Guards cover what the import graph
|
||||
cannot see: manifests, subprocess goldens, registries. They cannot be
|
||||
downgraded to warn.
|
||||
downgraded to warn. A guard that could not start, or whose own binary is off
|
||||
PATH (exit 127), is the machine's: `PREGATE_ENVIRONMENT` (`incomplete`), not
|
||||
this code; a guard whose binary is present but whose script exits 127 is this
|
||||
code.
|
||||
|
||||
**Fix.** Read the guard's lines in the table (exit code, output tail, log path)
|
||||
and do what its `fix` line says, commit, then rerun
|
||||
|
||||
@@ -68,7 +68,7 @@ export interface PregateCheck {
|
||||
// C-pregate (C8): stage, the input identity the check ran against, the fix clause and result code, and per-item lines.
|
||||
stage?: 'preflight' | 'tests'; inputs?: Record<string, string>; fix?: string; code?: string; lines?: string[];
|
||||
// Lane D (GSTA-112): a non-pass names its cause; an environment cause carries its kind and evidence.
|
||||
cause?: 'diff' | 'environment'; environment?: { kind: string; evidence: string };
|
||||
cause?: 'diff' | 'environment'; environment?: { kind: string; evidence: string; proof?: 'launch' | 'output' };
|
||||
}
|
||||
export interface PregateRemote { lane: string; platform: string; workflow: string; job: string; tests: string[]; receipt: string | null; cleared_by?: string; command?: string }
|
||||
export interface PregateFile {
|
||||
|
||||
+27
-10
@@ -11,7 +11,7 @@
|
||||
*/
|
||||
import { matchesAny } from '../ship-policy';
|
||||
import { sha256 } from './context';
|
||||
import { ALLOWLISTABLE_CATEGORIES, NON_ALLOWLISTABLE, type AllowlistEntry } from './config';
|
||||
import { ALLOWLISTABLE_CATEGORIES, nonAllowlistable, type AllowlistEntry } from './config';
|
||||
|
||||
export interface AllowlistFinding {
|
||||
id: string; category: string; severity: string; file: string; line?: number; col?: number;
|
||||
@@ -26,31 +26,48 @@ function names(entry: AllowlistEntry, f: AllowlistFinding): boolean {
|
||||
return (ALLOWLISTABLE_CATEGORIES as readonly string[]).includes(entry.finding) && entry.finding === f.category;
|
||||
}
|
||||
|
||||
/** Whether a finding has the position a `pattern` needs: the added line and a 1-based column. */
|
||||
const positioned = (f: AllowlistFinding) => f.lineText !== undefined && !!f.col;
|
||||
|
||||
/** The pattern, anchored at the finding's column in its added line and ending at a token boundary (so a short pattern never covers the head of a longer value); a finding with no position never matches a pattern. */
|
||||
function patternHits(entry: AllowlistEntry, f: AllowlistFinding): boolean {
|
||||
if (entry.pattern === undefined) return true;
|
||||
if (f.lineText === undefined || !f.col) return false;
|
||||
if (!positioned(f)) return false;
|
||||
let re: RegExp;
|
||||
try { re = new RegExp(`^(?:${entry.pattern})(?![\\w-])`); } catch { return false; }
|
||||
return re.test(f.lineText.slice(f.col - 1));
|
||||
return re.test(f.lineText!.slice(f.col! - 1));
|
||||
}
|
||||
|
||||
/** Split findings into the ones an entry covers (with the entry) and the ones kept; env.value is always kept. */
|
||||
export function applyAllowlist<F extends AllowlistFinding>(findings: readonly F[], entries: readonly AllowlistEntry[]): { kept: F[]; allowlisted: Allowlisted<F>[] } {
|
||||
/**
|
||||
* Split findings into the ones an entry covers (with the entry) and the ones
|
||||
* kept; env.value and engine.* are always kept. `untested` lists kept findings
|
||||
* that a pattern entry named by id and path but could not test because the
|
||||
* scanner gave no position, so the table says why the entry was inert.
|
||||
*/
|
||||
export function applyAllowlist<F extends AllowlistFinding>(findings: readonly F[], entries: readonly AllowlistEntry[]): { kept: F[]; allowlisted: Allowlisted<F>[]; untested: Allowlisted<F>[] } {
|
||||
const kept: F[] = [];
|
||||
const allowlisted: Allowlisted<F>[] = [];
|
||||
const untested: Allowlisted<F>[] = [];
|
||||
for (const f of findings) {
|
||||
const index = (NON_ALLOWLISTABLE as readonly string[]).includes(f.id) ? -1
|
||||
: entries.findIndex(e => names(e, f) && (!e.paths || matchesAny(e.paths, f.file)) && patternHits(e, f));
|
||||
if (index < 0) kept.push(f);
|
||||
else allowlisted.push({ finding: f, index, entry: entries[index]! });
|
||||
const scoped = (e: AllowlistEntry) => names(e, f) && (!e.paths || matchesAny(e.paths, f.file));
|
||||
const index = nonAllowlistable(f.id) ? -1 : entries.findIndex(e => scoped(e) && patternHits(e, f));
|
||||
if (index >= 0) { allowlisted.push({ finding: f, index, entry: entries[index]! }); continue; }
|
||||
kept.push(f);
|
||||
if (nonAllowlistable(f.id) || positioned(f)) continue;
|
||||
const named = entries.findIndex(e => e.pattern !== undefined && scoped(e));
|
||||
if (named >= 0) untested.push({ finding: f, index: named, entry: entries[named]! });
|
||||
}
|
||||
return { kept, allowlisted };
|
||||
return { kept, allowlisted, untested };
|
||||
}
|
||||
|
||||
/** sha256 of the entries, for the check's `inputs.allowlist`. */
|
||||
export const allowlistDigest = (entries: readonly AllowlistEntry[]) => sha256(JSON.stringify(entries));
|
||||
|
||||
/** One table line per finding a pattern entry named but could not test: the finding is kept. */
|
||||
export function describeUntested(a: Allowlisted): string {
|
||||
return `allowlist[${a.index}] ${a.entry.finding} not tested on ${a.finding.id} (${a.finding.file}): the scanner gave no position for its pattern; the finding is kept`;
|
||||
}
|
||||
|
||||
/** One table line per allowlisted finding: what, where, and which entry (with its reason) covered it. */
|
||||
export function describeAllowlisted(a: Allowlisted): string {
|
||||
const where = a.finding.line ? ` (${a.finding.file}, added line ${a.finding.line})` : ` (${a.finding.file})`;
|
||||
|
||||
@@ -17,7 +17,7 @@ import * as path from 'node:path';
|
||||
import { matchesAny } from '../../ship-policy';
|
||||
import type { Guard } from '../config';
|
||||
import { baseInputs, sha256 } from '../context';
|
||||
import { classifyEnvironment, environmentLine, environmentResult } from '../environment';
|
||||
import { classifyCommandEnvironment, classifyEnvironment, environmentLine, environmentResult, launchCause } from '../environment';
|
||||
import { launchRuntime } from '../runtime';
|
||||
import { checkScratch } from '../run-scratch';
|
||||
import { readRange, supervise } from '../supervise';
|
||||
@@ -32,9 +32,9 @@ async function runGuard(ctx: PregateContext, guard: Guard, env: NodeJS.ProcessEn
|
||||
fs.writeFileSync(log, '');
|
||||
const r = await supervise({ command: guard.command, cwd: ctx.repoRoot, env: { ...env, CI: env.CI ?? '1' }, log, deadline: Date.now() + timeoutMs, stallMs: ctx.stallTimeoutMs, label: `guard ${guard.id}`, progress: ctx.progress });
|
||||
const output = readRange(log, r.start, r.end).trim();
|
||||
// A guard that could not start, or whose shell found no such command (exit 127), is the machine's failure; a guard that ran and said no is the diff's,
|
||||
// whatever its output mentions (a test about rate limits must not read as a quota refusal).
|
||||
const environment = r.outcome === 'spawn-failed' ? classifyEnvironment(r.error ?? '', null) ?? { kind: 'missing-binary' as const, evidence: r.error ?? 'the guard could not be started' } : r.outcome === 'exited' && r.exit === 127 ? classifyEnvironment(output, r.exit) : null;
|
||||
// A guard that could not start, or whose own binary is off PATH (exit 127), is the machine's failure; a guard that ran and said no is the diff's,
|
||||
// whatever its output mentions (a test about rate limits must not read as a quota refusal, and a `bun run lint` whose script names a missing tool is the diff's).
|
||||
const environment = r.outcome === 'spawn-failed' ? { ...(classifyEnvironment(r.error ?? '', null) ?? launchCause('missing-binary', r.error ?? 'the guard could not be started')), proof: 'launch' as const } : r.outcome === 'exited' && r.exit === 127 ? classifyCommandEnvironment(guard.command, output, r.exit, env) : null;
|
||||
return { guard, exit: r.exit, timedOut: r.outcome !== 'exited' && r.outcome !== 'spawn-failed', stalled: r.outcome === 'stalled', tail: output ? output.split('\n').slice(-15) : [], log, environment };
|
||||
}
|
||||
|
||||
@@ -50,7 +50,7 @@ async function run(ctx: PregateContext): Promise<CheckResult> {
|
||||
return { ...base, status: 'incomplete', code: 'REPO_COMMANDS_NOT_ALLOWED', lines: listed, detail: `${selected.length} guard(s) not run (pass --allow-repo-commands): ${selected.map(g => g.id).join(', ')}`, fix: 'rerun with --allow-repo-commands (what a lane runs before reporting done, and what /ship passes)' };
|
||||
}
|
||||
const runtime = launchRuntime(ctx.repoRoot, ctx.env);
|
||||
if (runtime.refusal) return { ...base, ...runtime.refusal, cause: 'environment', environment: { kind: 'runtime', evidence: runtime.refusal.detail }, lines: [`runtime: ${runtime.label}`, environmentLine({ kind: 'runtime', evidence: runtime.refusal.detail })] };
|
||||
if (runtime.refusal) return { ...base, ...runtime.refusal, cause: 'environment', environment: launchCause('runtime', runtime.refusal.detail), lines: [`runtime: ${runtime.label}`, environmentLine(launchCause('runtime', runtime.refusal.detail))] };
|
||||
const dir = checkScratch(ctx.scratch, 'guards');
|
||||
const runs: GuardRun[] = [];
|
||||
for (const g of selected) runs.push(await runGuard(ctx, g, runtime.env, dir));
|
||||
|
||||
+24
-12
@@ -21,7 +21,7 @@ import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { baseInputs, sha256 } from '../context';
|
||||
import { PREGATE_CONFIG_PATH } from '../config';
|
||||
import { classifyEnvironment, environmentLine, environmentResult } from '../environment';
|
||||
import { classifyCommandEnvironment, classifyEnvironment, environmentLine, environmentResult, launchCause } from '../environment';
|
||||
import { checkScratch } from '../run-scratch';
|
||||
import { starterPolicy } from '../starter';
|
||||
import { isDeviceDart, listRepoFiles, readRepoFile } from '../import-graph';
|
||||
@@ -228,7 +228,7 @@ function planGap(ctx: PregateContext, plan: RunPlan, total: number): Pick<CheckR
|
||||
const { unclaimed, missing, noPubspec } = plan;
|
||||
const detail = `no test runner detected for ${unclaimed.length} of ${total} selected file(s): ${unclaimed.slice(0, 8).join(' ')}${unclaimed.length > 8 ? ' …' : ''}${missing.length ? `; not on PATH: ${missing.join(', ')}` : ''}${noPubspec.length ? `; no pubspec.yaml above ${noPubspec.length} .dart file(s)` : ''}`;
|
||||
// A tool missing from PATH is this machine's gap, not the diff's; no runner declared or no pubspec is the repo's.
|
||||
if (missing.length) return { code: 'PREGATE_ENVIRONMENT', cause: 'environment', environment: { kind: 'missing-binary', evidence: `not on PATH: ${missing.join(', ')}` }, detail, fix: `install ${missing.join(' and ')}${fvmPinnedFor(ctx.repoRoot, unclaimed) ? ' (or fvm, which this repo pins)' : ''} on this machine and rerun, or run the lane remotely; the diff is not at fault` };
|
||||
if (missing.length) return { code: 'PREGATE_ENVIRONMENT', cause: 'environment', environment: launchCause('missing-binary', `not on PATH: ${missing.join(', ')}`), detail, fix: `install ${missing.join(' and ')}${fvmPinnedFor(ctx.repoRoot, unclaimed) ? ' (or fvm, which this repo pins)' : ''} on this machine and rerun, or run the lane remotely; the diff is not at fault` };
|
||||
const fix = noPubspec.length ? `commit the Dart package's pubspec.yaml, or declare \`runner\` (with {files}) in ${ctx.configFile}`
|
||||
: `declare \`runner\` (with {files}) in ${ctx.configFile}`;
|
||||
return { code: 'PREGATE_INCOMPLETE', detail, fix };
|
||||
@@ -276,7 +276,7 @@ export function parseSummary(raw: string, runner = ''): RunnerSummary {
|
||||
return (dartRun ? null : parseDart(output)) ?? parseBats(output) ?? { ran: null, failed: null };
|
||||
}
|
||||
|
||||
interface LocalRun { exit: number | null; outcome: Supervised['outcome']; ran: number | null; failed: number | null; tail: string[]; command: string; files: string[]; lastFile: string | null; ms: number; error?: string }
|
||||
interface LocalRun { exit: number | null; outcome: Supervised['outcome']; signal: string | null; observedStart: boolean; ran: number | null; failed: number | null; tail: string[]; runner: string; command: string; files: string[]; lastFile: string | null; ms: number; error?: string }
|
||||
|
||||
const quote = (f: string) => `'${f.replace(/'/g, `'\\''`)}'`;
|
||||
|
||||
@@ -288,7 +288,7 @@ async function runOnce(ctx: PregateContext, env: NodeJS.ProcessEnv, g: RunGroup,
|
||||
const r = await supervise({ command: run, cwd: path.join(ctx.repoRoot, g.cwd), env: { ...env, CI: env.CI ?? '1' }, log, deadline: Date.now() + ctx.timeoutMs, stallMs: ctx.stallTimeoutMs, label: `lanes (${shown.length === 1 ? shown[0] : `${shown.length} files`})`, files, adapter: adapterFor(g.runner), counter, progress: ctx.progress });
|
||||
const output = readRange(log, r.start, r.end);
|
||||
const summary = parseSummary(output, g.runner);
|
||||
return { exit: r.exit, outcome: r.outcome, ...summary, tail: output.trim().split('\n').slice(-25), command, files: shown, lastFile: r.lastFile && g.cwd ? `${g.cwd}/${r.lastFile}` : r.lastFile, ms: r.ms, ...(r.error ? { error: r.error } : {}) };
|
||||
return { exit: r.exit, outcome: r.outcome, signal: r.signal, observedStart: r.observedStart, ...summary, tail: output.trim().split('\n').slice(-25), runner: g.runner, command, files: shown, lastFile: r.lastFile && g.cwd ? `${g.cwd}/${r.lastFile}` : r.lastFile, ms: r.ms, ...(r.error ? { error: r.error } : {}) };
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -329,11 +329,16 @@ function unfinished(ctx: PregateContext, r: Awaited<ReturnType<typeof runLocal>>
|
||||
if (stalled.length) {
|
||||
return { code: 'PREGATE_STALLED', detail: `no test output for ${ctx.stallTimeoutMs / 1000} s; process group killed; last file started: ${stalled.map(x => x.lastFile ?? `(none reported; ${x.files.length} file(s))`).join(', ')}${rest}`, fix: `read ${r.log} for the stalled file; fix the hang, or raise --stall-timeout (stall_timeout_s in ${ctx.configFile}; 0 disables) for a test that is quiet for longer` };
|
||||
}
|
||||
if (!timedOut.length) {
|
||||
const launch = r.runs.find(x => x.outcome === 'spawn-failed');
|
||||
const environment = classifyEnvironment(launch?.error ?? '', null) ?? { kind: 'missing-binary' as const, evidence: launch?.error ?? 'the runner could not be started' };
|
||||
const launch = r.runs.find(x => x.outcome === 'spawn-failed');
|
||||
if (launch) {
|
||||
const environment = { ...(classifyEnvironment(launch.error ?? '', null) ?? launchCause('missing-binary', launch.error ?? 'the runner could not be started')), proof: 'launch' as const };
|
||||
return { environment, detail: `the runner could not be launched (${total} files${rest}): ${environment.evidence}` };
|
||||
}
|
||||
if (!timedOut.length) {
|
||||
// The runner started and died without an exit code (a signal: OOM kill, SIGSEGV, an outside kill): incomplete, not the machine's missing binary.
|
||||
const killed = r.runs.find(x => x.outcome === 'exited' && x.exit === null);
|
||||
return { code: 'PREGATE_INCOMPLETE', detail: `the runner process ended without an exit code (killed by ${killed?.signal ?? 'a signal'}; ${total} files; last file started: ${killed?.lastFile ?? 'none reported'}${rest})`, fix: `read ${r.log}; a runner killed by a signal (out of memory, an outside kill) never passes; rerun, or split the selection` };
|
||||
}
|
||||
return { code: 'PREGATE_INCOMPLETE', detail: `a runner process timed out after ${ctx.timeoutMs} ms (--timeout applies to each runner process; ${total} files; last file started: ${timedOut.map(x => x.lastFile ?? `(none reported; ${x.files.length} file(s))`).join(', ')}${rest})`, fix: 'raise --timeout or split the selection; a timed-out lane never passes' };
|
||||
}
|
||||
|
||||
@@ -365,6 +370,8 @@ function remoteObligation(ctx: PregateContext, sel: Selection): RemoteObligation
|
||||
}
|
||||
|
||||
export const REMOTE_GATE_LANE = 'pregate.json/remote_gate';
|
||||
/** How many leading lines of a remote gate's output are read for an environment cause: its preamble, before any test output could stream. */
|
||||
export const REMOTE_PREAMBLE_LINES = 40;
|
||||
/** The `clear` command for an obligation: the remote gate's own output for remote_gate, a run URL otherwise. */
|
||||
export const clearHint = (lane: string) => `gstack-pregate clear --remote ${lane} ${lane === REMOTE_GATE_LANE ? '--receipt <gate output file>' : '--run-url <url>'}`;
|
||||
export interface RemoteReport { tree: string; tests: number; failed: number; spent?: string }
|
||||
@@ -412,8 +419,11 @@ async function runRemote(ctx: PregateContext, sel: Selection, base: CheckResult)
|
||||
const why = r.outcome !== 'exited' ? `the remote gate did not finish (${r.outcome === 'stalled' ? `no output for ${ctx.stallTimeoutMs / 1000} s` : r.outcome})`
|
||||
: !reported ? `the remote gate exited ${r.exit} without a PREGATE_REMOTE: line`
|
||||
: remoteReportProblem(reported, ctx.wtree) ?? (r.exit !== 0 ? `the remote gate exited ${r.exit}` : null);
|
||||
// A gate that printed no report and names a quota, network or missing-tool error never ran the tests: the environment, not the diff.
|
||||
const environment = why && !reported && r.outcome !== 'stalled' && r.outcome !== 'timeout' ? classifyEnvironment(r.outcome === 'spawn-failed' ? r.error ?? '' : output, r.exit) : null;
|
||||
// A gate that printed no report and names a quota, network or missing-tool error in its preamble never ran the tests: the environment, not the diff.
|
||||
// Only the gate's first lines are read (a VM refusal or a network error comes before any test streams); exit 127 counts only when the gate's own binary is off PATH.
|
||||
const environment = why && !reported && r.outcome !== 'stalled' && r.outcome !== 'timeout'
|
||||
? r.outcome === 'spawn-failed' ? { ...(classifyEnvironment(r.error ?? '', null) ?? launchCause('missing-binary', r.error ?? 'the remote gate could not be started')), proof: 'launch' as const } : classifyCommandEnvironment(obligation.command!, output.split('\n').slice(0, REMOTE_PREAMBLE_LINES).join('\n'), r.exit, ctx.env)
|
||||
: null;
|
||||
if (environment) return environmentResult({ ...base, inputs, lines: runLines, remote: [...sel.remote, obligation] }, environment, `${why}; requires-remote ${obligation.lane} stays uncleared`);
|
||||
if (why) return { ...base, inputs, lines: runLines, status: 'incomplete', remote: [...sel.remote, obligation], code: r.outcome === 'stalled' ? 'PREGATE_STALLED' : 'PREGATE_INCOMPLETE', detail: `${why}; requires-remote ${obligation.lane} stays uncleared`, fix: `read ${log}; the obligation clears only on PREGATE_REMOTE: tree=${ctx.wtree.slice(0, 12)}… tests>0 failed=0 with exit 0` };
|
||||
const cleared = { ...obligation, receipt: `remote-gate tree=${ctx.wtree.slice(0, 12)} tests=${reported!.tests}`, cleared_by: `gstack-pregate --remote ${new Date().toISOString()}` };
|
||||
@@ -445,7 +455,7 @@ async function runSelected(ctx: PregateContext, sel: Selection, base: CheckResul
|
||||
const { groups } = plan;
|
||||
const total = groups.reduce((n, g) => n + g.files.length, 0);
|
||||
const runtime = launchRuntime(ctx.repoRoot, ctx.env);
|
||||
if (runtime.refusal) return { ...base, ...runtime.refusal, cause: 'environment', environment: { kind: 'runtime', evidence: runtime.refusal.detail }, lines: [`runtime: ${runtime.label}`, environmentLine({ kind: 'runtime', evidence: runtime.refusal.detail })] };
|
||||
if (runtime.refusal) return { ...base, ...runtime.refusal, cause: 'environment', environment: launchCause('runtime', runtime.refusal.detail), lines: [`runtime: ${runtime.label}`, environmentLine(launchCause('runtime', runtime.refusal.detail))] };
|
||||
const r = await runLocal(ctx, runtime.env, groups);
|
||||
const adapters = [...new Set(groups.map(g => adapterFor(g.runner)).filter(Boolean))];
|
||||
const runLines = [...lines, `runtime: ${runtime.label}`, `progress: ${adapters.join(', ') || 'none (no progress adapter for this runner; serial files and heartbeats only)'}`, `ran: ${r.command.length > 200 ? `${r.command.slice(0, 200)}…` : r.command}`, `log: ${r.log}`, ...r.tail.map(l => ` ${l}`)];
|
||||
@@ -455,8 +465,10 @@ async function runSelected(ctx: PregateContext, sel: Selection, base: CheckResul
|
||||
return why.environment ? environmentResult(withRun, why.environment, why.detail!) : { ...withRun, status: 'incomplete', ...why };
|
||||
}
|
||||
if (r.exit !== 0) {
|
||||
// The runner's own summary decides: a counted failure is the diff's. Only a run that reported no test at all is read from its output.
|
||||
const environment = r.ran === null ? classifyEnvironment(r.tail.join('\n'), r.exit) : null;
|
||||
// The runner's own summary decides: a counted failure is the diff's. Only a run that reported no test at all, and whose runner was not seen
|
||||
// starting a test file (a test that exits the process before the summary cannot relabel itself), is read from its output; exit 127 counts
|
||||
// only when the runner's binary is off PATH.
|
||||
const environment = r.ran === null && r.bad && !r.bad.observedStart ? classifyCommandEnvironment(r.bad.runner, r.tail.join('\n'), r.exit, runtime.env) : null;
|
||||
if (environment) return environmentResult(withRun, environment, `the runner exited ${r.exit} before any test was counted (${total} files): ${environment.evidence}`);
|
||||
return { ...withRun, status: 'fail', cause: 'diff', code: 'PREGATE_LANE_FAILED', detail: `local lane exited ${r.exit} (${r.failed ?? '?'} failed of ${r.ran ?? '?'} ran; ${total} files; pipefail)`, fix: `read ${r.log}, fix the in-branch failure, rerun gstack-pregate --stage tests` };
|
||||
}
|
||||
|
||||
@@ -11,9 +11,9 @@ import { spawnSync } from 'node:child_process';
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { runScriptSync } from '../../script-invocation';
|
||||
import { allowlistDigest, applyAllowlist, describeAllowlisted, type AllowlistFinding } from '../allowlist';
|
||||
import { allowlistDigest, applyAllowlist, describeAllowlisted, describeUntested, type AllowlistFinding } from '../allowlist';
|
||||
import { baseInputs, sha256 } from '../context';
|
||||
import { environmentResult } from '../environment';
|
||||
import { environmentResult, launchCause } from '../environment';
|
||||
import { checkScratch } from '../run-scratch';
|
||||
import type { CheckDef, CheckResult, PregateContext } from '../types';
|
||||
|
||||
@@ -71,19 +71,19 @@ function run(ctx: PregateContext): CheckResult {
|
||||
fs.mkdirSync(path.dirname(file), { mode: 0o700 });
|
||||
fs.writeFileSync(file, part.text, { mode: 0o600 });
|
||||
const r = runScriptSync(path.join(BIN_DIR, 'gstack-redact'), ['--from-file', file, '--json', '--max-bytes', String(64 * 1024 * 1024)], { cwd: ctx.repoRoot, encoding: 'utf8', timeout: Math.min(ctx.timeoutMs, 300_000), maxBuffer: 64 * 1024 * 1024, env: ctx.env });
|
||||
if (r.error || r.status === null) return environmentResult(base, { kind: 'tool', evidence: `gstack-redact did not finish on ${part.file || 'added lines'} (${r.error?.message ?? 'killed'})` }, `gstack-redact did not finish on ${part.file || 'added lines'}; an incomplete scan never passes`);
|
||||
if (r.status !== 0 && r.status !== 2 && r.status !== 3) return environmentResult(base, { kind: 'tool', evidence: `gstack-redact exited ${r.status} on ${part.file || 'added lines'}: ${(r.stderr ?? '').trim().split('\n').pop() ?? ''}` }, `gstack-redact exited ${r.status}; an incomplete scan never passes`);
|
||||
if (r.error || r.status === null) return environmentResult(base, launchCause('tool', `gstack-redact did not finish on ${part.file || 'added lines'} (${r.error?.message ?? 'killed'})`), `gstack-redact did not finish on ${part.file || 'added lines'}; an incomplete scan never passes`);
|
||||
if (r.status !== 0 && r.status !== 2 && r.status !== 3) return environmentResult(base, launchCause('tool', `gstack-redact exited ${r.status} on ${part.file || 'added lines'}: ${(r.stderr ?? '').trim().split('\n').pop() ?? ''}`), `gstack-redact exited ${r.status}; an incomplete scan never passes`);
|
||||
high ||= r.status === 3;
|
||||
medium ||= r.status === 2;
|
||||
const lines = part.text.split('\n');
|
||||
try { findings.push(...(JSON.parse(r.stdout).findings ?? []).map((f: Omit<Finding, 'file'>) => ({ ...f, id: f.id ?? 'finding', category: f.category ?? '?', severity: f.severity ?? '?', file: part.file, lineText: f.line ? lines[f.line - 1] : undefined }))); } catch { parsedAll = false; }
|
||||
}
|
||||
// The allowlist applies only to findings the scanner reported in full; an unparsed report keeps its exit-code verdict.
|
||||
const applied = parsedAll && ctx.config.allowlist.length ? applyAllowlist(findings, ctx.config.allowlist) : { kept: findings, allowlisted: [] };
|
||||
const applied = parsedAll && ctx.config.allowlist.length ? applyAllowlist(findings, ctx.config.allowlist) : { kept: findings, allowlisted: [], untested: [] };
|
||||
const kept = applied.kept;
|
||||
if (parsedAll) { high = kept.some(f => f.severity === 'HIGH'); medium = kept.some(f => f.severity === 'MEDIUM'); }
|
||||
const render = (f: Scanned) => `${f.severity} ${f.id} [${f.category}]: ${f.description ?? ''}${f.line ? ` (${f.file ? `${f.file}, ` : ''}added line ${f.line})` : ''}`;
|
||||
const allowLines = applied.allowlisted.map(describeAllowlisted);
|
||||
const allowLines = [...applied.allowlisted.map(describeAllowlisted), ...applied.untested.map(describeUntested)];
|
||||
const allowNote = applied.allowlisted.length ? `; ${applied.allowlisted.length} allowlisted (${[...new Set(applied.allowlisted.map(a => a.finding.id))].join(', ')})` : '';
|
||||
// HIGH and any MEDIUM credential fail; a MEDIUM hygiene/internal/pii/legal
|
||||
// hit in code (a `.local` property reads as a hostname) is listed as warn:
|
||||
|
||||
@@ -58,6 +58,8 @@ export const PREGATE_CONFIG_SCHEMA = 'https://github.com/garrytan/gstack/docs/pr
|
||||
export const NON_DOWNGRADABLE = ['secrets', 'guards'] as const;
|
||||
/** Finding ids (or categories) no allowlist entry may name: the real value of a credential-named environment variable (#3449), and the secret wildcard. */
|
||||
export const NON_ALLOWLISTABLE = ['env.value', 'secret'] as const;
|
||||
/** Whether no entry may name this finding id: the list above, plus the scanner's own `engine.*` refusals (an oversize input fails closed and is never benign). */
|
||||
export const nonAllowlistable = (id: string) => (NON_ALLOWLISTABLE as readonly string[]).includes(id) || id.startsWith('engine.');
|
||||
/** Categories an allowlist entry may name whole; a secret is always named by its id. */
|
||||
export const ALLOWLISTABLE_CATEGORIES = ['pii', 'internal', 'hygiene', 'legal'] as const;
|
||||
export const DEFAULT_SERIAL_PATTERNS = ['**/*.serial.test.*'];
|
||||
@@ -139,7 +141,7 @@ function allowlistErrors(raw: unknown, errors: ConfigError[]): void {
|
||||
const o = e as Record<string, unknown>;
|
||||
for (const k of Object.keys(o)) if (!ALLOWLIST_KEYS.includes(k)) bad(`${at}.${k}`, `unknown key; valid keys: ${ALLOWLIST_KEYS.join(', ')}`);
|
||||
if (typeof o.finding !== 'string' || !FINDING_ID.test(o.finding)) bad(`${at}.finding`, 'must be a gstack-redact finding id (env.kv, google.api_key, pii.phone.e164, …) or one of the categories pii, internal, hygiene, legal');
|
||||
else if ((NON_ALLOWLISTABLE as readonly string[]).includes(o.finding)) bad(`${at}.finding`, `${o.finding} cannot be allowlisted (${o.finding === 'env.value' ? 'the real value of a credential-named environment variable is never benign' : 'name the secret finding by its id, never the whole category'})`);
|
||||
else if (nonAllowlistable(o.finding)) bad(`${at}.finding`, `${o.finding} cannot be allowlisted (${o.finding === 'env.value' ? 'the real value of a credential-named environment variable is never benign' : o.finding === 'secret' ? 'name the secret finding by its id, never the whole category' : 'an engine.* finding is the scanner refusing its input, not a finding in it'})`);
|
||||
if (!('paths' in o) && !('pattern' in o)) bad(at, 'needs paths, pattern or both: an entry with no scope would allowlist every finding of that kind');
|
||||
if ('paths' in o) {
|
||||
if (!Array.isArray(o.paths) || o.paths.length === 0 || o.paths.some(p => typeof p !== 'string')) bad(`${at}.paths`, 'must be a non-empty array of globs');
|
||||
@@ -323,7 +325,7 @@ export function explainConfig(): string {
|
||||
'price_file: repo-relative path of the GSTACK_EVAL_CALLS v1 call-plan file `gstack-eval-plan price-gate` prices a paid gate from; never executed (docs/eval-calls.md)',
|
||||
'stall_timeout_s: seconds without test output before the lanes check kills the runner\'s process group and records it incomplete (default 300; 0 disables; --stall-timeout overrides)',
|
||||
'remote_gate: the repo\'s remote test gate command; under 8 cores the test tier becomes a requires-remote obligation naming it, and gstack-pregate --remote runs it (it must print PREGATE_REMOTE: tree=<tree> tests=<n> failed=<n>) (docs/pregate.md#remote-gate)',
|
||||
`allowlist: [{ finding, paths?, pattern?, reason }]: benign findings the secrets check accepts and names as allowlisted. finding is a gstack-redact id (env.kv, google.api_key, pii.phone.e164, …) or a category (${ALLOWLISTABLE_CATEGORIES.join(', ')}); paths are globs; pattern is a regex matched at the finding's position in the added line; one of the two is required, reason always. Never ${NON_ALLOWLISTABLE.join(' or ')} (docs/pregate.md#allowlist)`,
|
||||
`allowlist: [{ finding, paths?, pattern?, reason }]: benign findings the secrets check accepts and names as allowlisted. finding is a gstack-redact id (env.kv, google.api_key, pii.phone.e164, …) or a category (${ALLOWLISTABLE_CATEGORIES.join(', ')}); paths are globs; pattern is a regex matched at the finding's position in the added line; one of the two is required, reason always. Never ${NON_ALLOWLISTABLE.join(' or ')} or engine.* (docs/pregate.md#allowlist)`,
|
||||
'no .gstack/pregate.json on the base: <state root>/projects/<slug>/pregate.json is read instead (state root: GSTACK_STATE_ROOT, else GSTACK_HOME, else GSTACK_STATE_DIR, else a gstack plugin data dir, else ~/.gstack, per docs/state-root.md; run --explain prints the resolved path; outside the repo, so a PR cannot reach it; a path inside the checkout is ignored)',
|
||||
].join('\n') + '\n';
|
||||
}
|
||||
+44
-12
@@ -8,10 +8,16 @@
|
||||
* branch's fault. The classifier reads a command's output and exit code only;
|
||||
* the caller decides when to ask it: a local lane whose runner printed a test
|
||||
* summary is the diff's failure whatever its log says, and only a run with no
|
||||
* summary (nothing ran) is classified from its output.
|
||||
* summary (nothing ran) is classified from its output, and only before any
|
||||
* test file started. A missing binary is the machine's only when the command's
|
||||
* own binary is off PATH: a script the diff controls that exits 127 or prints
|
||||
* "command not found" is the diff's failure (`classifyCommandEnvironment`).
|
||||
*/
|
||||
import type { CheckResult, EnvironmentCause, EnvironmentKind } from './types';
|
||||
|
||||
/** A cause established at launch: a spawn failure, a binary off PATH, a runtime or tool refusal. */
|
||||
export const launchCause = (kind: EnvironmentKind, evidence: string): EnvironmentCause => ({ kind, evidence, proof: 'launch' });
|
||||
|
||||
const ANSI = /\x1b\[[0-9;?]*[A-Za-z]/g;
|
||||
/** The shell's own "no such command" text (POSIX and cmd.exe); exit 127 says the same without text. */
|
||||
const NOT_FOUND = /command not found|is not recognized as an internal or external command/;
|
||||
@@ -33,28 +39,54 @@ function evidenceLine(text: string, re: RegExp): string {
|
||||
* ordinary failure. Exit 127 (the shell found no such command) is
|
||||
* missing-binary with or without text; a disk rule outranks quota (EDQUOT is a
|
||||
* disk quota); otherwise the first rule in order whose text appears wins.
|
||||
* Exit 127 is proved at launch; a text match is `proof: 'output'`, which the
|
||||
* diff's own output could have written, and the fix says so.
|
||||
*/
|
||||
export function classifyEnvironment(output: string, exit: number | null): EnvironmentCause | null {
|
||||
const text = output.replace(ANSI, '');
|
||||
if (exit === 127 || NOT_FOUND.test(text)) return { kind: 'missing-binary', evidence: evidenceLine(text, RULES[0]!.re) || 'exit 127: the shell found no such command' };
|
||||
if (exit === 127 || NOT_FOUND.test(text)) return { kind: 'missing-binary', evidence: evidenceLine(text, RULES[0]!.re) || 'exit 127: the shell found no such command', proof: exit === 127 ? 'launch' : 'output' };
|
||||
const disk = RULES.find(r => r.kind === 'disk')!;
|
||||
if (disk.re.test(text)) return { kind: 'disk', evidence: evidenceLine(text, disk.re) };
|
||||
for (const rule of RULES) if (rule.kind !== 'disk' && rule.re.test(text)) return { kind: rule.kind, evidence: evidenceLine(text, rule.re) };
|
||||
if (disk.re.test(text)) return { kind: 'disk', evidence: evidenceLine(text, disk.re), proof: 'output' };
|
||||
for (const rule of RULES) if (rule.kind !== 'disk' && rule.re.test(text)) return { kind: rule.kind, evidence: evidenceLine(text, rule.re), proof: 'output' };
|
||||
return null;
|
||||
}
|
||||
|
||||
/** The binary a shell command runs: the first word after a leading `(cd … &&` and any `NAME=value` assignments. */
|
||||
export function commandBinary(command: string): string | null {
|
||||
const words = command.replace(/^\(\s*cd\s+(?:'[^']*'|"[^"]*"|\S+)\s*&&\s*/, '').trim().split(/\s+/);
|
||||
return words.find(w => w && !/^[A-Za-z_][A-Za-z0-9_]*=/.test(w)) ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* `classifyEnvironment` for a command that started and exited: missing-binary
|
||||
* holds only when the command's own binary is off `env.PATH`. A runner or
|
||||
* guard that resolved and then exited 127, or printed "command not found" from
|
||||
* a script the diff controls, is the diff's failure.
|
||||
*/
|
||||
export function classifyCommandEnvironment(command: string, output: string, exit: number | null, env: NodeJS.ProcessEnv): EnvironmentCause | null {
|
||||
const cause = classifyEnvironment(output, exit);
|
||||
if (cause?.kind !== 'missing-binary') return cause;
|
||||
const bin = commandBinary(command);
|
||||
return bin && Bun.which(bin, { PATH: env.PATH ?? '' }) ? null : { ...cause, proof: 'launch' };
|
||||
}
|
||||
|
||||
const FIX: Record<EnvironmentKind, string> = {
|
||||
'missing-binary': 'install the named tool on this machine (or put it on PATH), or run the lane remotely (--remote, or clear it with a run URL); the diff is not at fault',
|
||||
network: 'check the network, proxy or registry this machine reaches, then rerun; or run the lane remotely; the diff is not at fault',
|
||||
quota: 'wait for the quota or raise it, free the backend\'s capacity, then rerun; the diff is not at fault',
|
||||
disk: 'free disk space (or move the state root) and rerun; the diff is not at fault',
|
||||
runtime: 'repair the runtime the line names (the install verdict or the pinned Bun), then rerun; the diff is not at fault',
|
||||
tool: 'repair or reinstall the named gstack tool, then rerun; the diff is not at fault',
|
||||
'missing-binary': 'install the named tool on this machine (or put it on PATH), or run the lane remotely (--remote, or clear it with a run URL)',
|
||||
network: 'check the network, proxy or registry this machine reaches, then rerun; or run the lane remotely',
|
||||
quota: 'wait for the quota or raise it, free the backend\'s capacity, then rerun',
|
||||
disk: 'free disk space (or move the state root) and rerun',
|
||||
runtime: 'repair the runtime the line names (the install verdict or the pinned Bun), then rerun',
|
||||
tool: 'repair or reinstall the named gstack tool, then rerun',
|
||||
};
|
||||
/** The fix's last clause: a launch-proved cause clears the diff; an output-inferred one asks for confirmation elsewhere, since the diff's own output could have written the line. */
|
||||
export const PROOF_NOTE = {
|
||||
launch: 'the diff is not at fault',
|
||||
output: 'inferred from the output alone (no test was counted), which the diff could have written: confirm on another machine or with --remote before treating the diff as clear',
|
||||
} as const;
|
||||
|
||||
/** An `incomplete` result under PREGATE_ENVIRONMENT: cause, kind, evidence, an `environment:` line and the kind's fix. */
|
||||
export function environmentResult(base: CheckResult, cause: EnvironmentCause, detail: string): CheckResult {
|
||||
return { ...base, status: 'incomplete', code: 'PREGATE_ENVIRONMENT', cause: 'environment', environment: cause, detail, fix: FIX[cause.kind], lines: [...(base.lines ?? []), environmentLine(cause)] };
|
||||
return { ...base, status: 'incomplete', code: 'PREGATE_ENVIRONMENT', cause: 'environment', environment: cause, detail, fix: `${FIX[cause.kind]}; ${PROOF_NOTE[cause.proof]}`, lines: [...(base.lines ?? []), environmentLine(cause)] };
|
||||
}
|
||||
|
||||
export const environmentLine = (cause: EnvironmentCause) => `environment: ${cause.kind} — ${cause.evidence}`;
|
||||
export const environmentLine = (cause: EnvironmentCause) => `environment: ${cause.kind} — ${cause.evidence}${cause.proof === 'output' ? ' (inferred from output)' : ''}`;
|
||||
@@ -139,7 +139,11 @@ export function verifyPregate(repoRoot: string, file: PregateFile, treeNow: Wtre
|
||||
const incomplete = file.checks.filter(c => c.status === 'incomplete' || c.status === 'not_installed');
|
||||
// An environment cause is named apart from the diff: a lane waives it by kind, never by test name.
|
||||
const environment = incomplete.filter(c => c.cause === 'environment');
|
||||
if (environment.length === incomplete.length && environment.length) return { ok: false, code: 'PREGATE_ENVIRONMENT', message: `${environment.length} check(s) blocked by the environment (${[...new Set(environment.map(c => c.environment?.kind ?? 'unknown'))].join(', ')}), not by the diff: ${environment.map(c => `${c.id}: ${c.environment?.evidence ?? c.detail}`).join('; ')}`, current, uncleared };
|
||||
if (environment.length === incomplete.length && environment.length) {
|
||||
const inferred = environment.filter(c => c.environment?.proof === 'output');
|
||||
const caveat = inferred.length ? `; ${inferred.map(c => c.id).join(', ')} inferred from output alone (no test was counted): confirm on another machine or with --remote before treating the diff as clear` : '';
|
||||
return { ok: false, code: 'PREGATE_ENVIRONMENT', message: `${environment.length} check(s) blocked by the environment (${[...new Set(environment.map(c => c.environment?.kind ?? 'unknown'))].join(', ')}), not by the diff: ${environment.map(c => `${c.id}: ${c.environment?.evidence ?? c.detail}`).join('; ')}${caveat}`, current, uncleared };
|
||||
}
|
||||
if (incomplete.length) return { ok: false, code: 'PREGATE_INCOMPLETE', message: `${incomplete.length} check(s) incomplete: ${incomplete.map(c => c.id).join(', ')}${environment.length ? ` (blocked by the environment: ${environment.map(c => `${c.id} ${c.environment?.kind}`).join(', ')})` : ''}`, current, uncleared };
|
||||
if (uncleared.length) return { ok: false, code: 'PREGATE_REMOTE_UNCLEARED', message: `${uncleared.length} requires-remote obligation(s) without a receipt: ${uncleared.map(r => r.lane).join(', ')}`, current, uncleared };
|
||||
return { ok: true, message: `pregate ${file.verdict} is current for tree ${file.tree.slice(0, 12)}`, current, uncleared };
|
||||
|
||||
@@ -3,9 +3,10 @@
|
||||
* (Lane D, GSTA-112): `<state root>/projects/<slug>/pregate/runs/<run id>/`.
|
||||
* Every check keeps its working files there (the secrets scan's added-line
|
||||
* copies, the lane and guard logs, the remote gate's file list) instead of
|
||||
* os.tmpdir(): two pregates running at once never share a path, and the log a
|
||||
* `fix:` line names survives a host that wipes /tmp between wakes (gbrain 16).
|
||||
* The record names the directory. Runs older than the retention are pruned
|
||||
* os.tmpdir(): two pregates running at once never share a path (a taken
|
||||
* `--run <id>` takes a numbered sibling), and the log a `fix:` line names
|
||||
* survives a host that wipes /tmp between wakes (gbrain 16). The record names
|
||||
* the directory. Runs older than the retention are pruned
|
||||
* best-effort when a new run starts; the current run is never pruned.
|
||||
*/
|
||||
import { randomBytes } from 'node:crypto';
|
||||
@@ -23,14 +24,26 @@ export function scratchRunId(requested?: string, pid: number = process.pid): str
|
||||
return `${stamp}-${pid}-${randomBytes(2).toString('hex')}`;
|
||||
}
|
||||
|
||||
/** `<runsRoot>/<run id>`, created 0700 (and its parents), after pruning old siblings. */
|
||||
/** How many numbered siblings a taken run id may fall back to before the run refuses. */
|
||||
const MAX_RUN_ID_SIBLINGS = 9;
|
||||
|
||||
/**
|
||||
* `<runsRoot>/<run id>`, created 0700 (and its parents), after pruning old
|
||||
* siblings. The leaf is created without `recursive`, so a directory that
|
||||
* already exists (a second `--run <id>` beside a live run, or a finished one
|
||||
* under the same name) is never shared: the run takes `<run id>-2` … `-9`
|
||||
* instead, and the record names the directory it got.
|
||||
*/
|
||||
export function createScratch(runsRoot: string, runId: string): string {
|
||||
fs.mkdirSync(runsRoot, { recursive: true, mode: 0o700 });
|
||||
pruneScratch(runsRoot, runId);
|
||||
const dir = path.join(runsRoot, runId);
|
||||
fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
|
||||
try { fs.chmodSync(dir, 0o700); } catch { /* best effort on hosts without POSIX modes */ }
|
||||
return dir;
|
||||
for (let n = 1; n <= MAX_RUN_ID_SIBLINGS; n++) {
|
||||
const dir = path.join(runsRoot, n === 1 ? runId : `${runId}-${n}`);
|
||||
try { fs.mkdirSync(dir, { mode: 0o700 }); } catch (e: any) { if (e?.code === 'EEXIST') continue; throw e; }
|
||||
try { fs.chmodSync(dir, 0o700); } catch { /* best effort on hosts without POSIX modes */ }
|
||||
return dir;
|
||||
}
|
||||
throw new Error(`run directory ${path.join(runsRoot, runId)} and its ${MAX_RUN_ID_SIBLINGS - 1} numbered siblings all exist; choose another --run id`);
|
||||
}
|
||||
|
||||
/** Remove sibling run directories older than the retention, never `keep` (the current run); every failure is ignored. */
|
||||
|
||||
@@ -54,6 +54,8 @@ export interface Supervised {
|
||||
start: number;
|
||||
end: number;
|
||||
lastFile: string | null;
|
||||
/** Whether the progress adapter saw a test file start in the runner's own output (a single file is marked started by the supervisor, which proves nothing). */
|
||||
observedStart: boolean;
|
||||
error?: string;
|
||||
ms: number;
|
||||
}
|
||||
@@ -109,6 +111,7 @@ export function supervise(o: SuperviseOptions): Promise<Supervised> {
|
||||
let lastProgress = started;
|
||||
let current: { file: string; at: number; failed: boolean } | null = null;
|
||||
let lastFile: string | null = null;
|
||||
let observedStart = false;
|
||||
const emit = (line: string) => { lastProgress = Date.now(); o.progress(line); };
|
||||
const startFile = (file: string) => { current = { file, at: Date.now(), failed: false }; lastFile = file; emit(`PREGATE_TEST: ${counter.done + 1}/${counter.total} ${file} start`); };
|
||||
const finishFile = (failed?: boolean) => {
|
||||
@@ -125,7 +128,7 @@ export function supervise(o: SuperviseOptions): Promise<Supervised> {
|
||||
partial = lines.pop() ?? '';
|
||||
for (const line of lines) {
|
||||
const header = /^(?:::group::)?(\S.*):$/.exec(line);
|
||||
if (header && files.has(header[1]!)) { finishFile(); startFile(header[1]!); continue; }
|
||||
if (header && files.has(header[1]!)) { observedStart = true; finishFile(); startFile(header[1]!); continue; }
|
||||
if (/^\s*\d+\s+pass$/.test(line) || /^Ran \d+ tests? across/.test(line) || /^\s*\d+\s+tests? failed:?$/.test(line)) { finishFile(); continue; }
|
||||
if (current && /^\(fail\)\s/.test(line)) current.failed = true;
|
||||
}
|
||||
@@ -134,11 +137,11 @@ export function supervise(o: SuperviseOptions): Promise<Supervised> {
|
||||
return new Promise(resolve => {
|
||||
let fd: number;
|
||||
try { fd = fs.openSync(o.log, 'a'); }
|
||||
catch (e: any) { resolve({ outcome: 'spawn-failed', exit: null, signal: null, start, end: start, lastFile: null, error: e.message, ms: 0 }); return; }
|
||||
catch (e: any) { resolve({ outcome: 'spawn-failed', exit: null, signal: null, start, end: start, lastFile: null, observedStart: false, error: e.message, ms: 0 }); return; }
|
||||
const inv = bashCommandInvocation(`set -o pipefail; ${o.command}`);
|
||||
if (!inv.ok) {
|
||||
fs.closeSync(fd);
|
||||
resolve({ outcome: 'spawn-failed', exit: null, signal: null, start, end: start, lastFile: null, error: inv.error, ms: Date.now() - started });
|
||||
resolve({ outcome: 'spawn-failed', exit: null, signal: null, start, end: start, lastFile: null, observedStart: false, error: inv.error, ms: Date.now() - started });
|
||||
return;
|
||||
}
|
||||
let child: ReturnType<typeof spawn>;
|
||||
@@ -146,7 +149,7 @@ export function supervise(o: SuperviseOptions): Promise<Supervised> {
|
||||
child = spawn(inv.cmd, inv.argv, { cwd: o.cwd, env: o.env, stdio: ['ignore', fd, fd], detached: process.platform !== 'win32', windowsHide: true });
|
||||
} catch (e: any) {
|
||||
fs.closeSync(fd);
|
||||
resolve({ outcome: 'spawn-failed', exit: null, signal: null, start, end: start, lastFile: null, error: e.message, ms: Date.now() - started });
|
||||
resolve({ outcome: 'spawn-failed', exit: null, signal: null, start, end: start, lastFile: null, observedStart: false, error: e.message, ms: Date.now() - started });
|
||||
return;
|
||||
}
|
||||
fs.closeSync(fd);
|
||||
@@ -170,7 +173,7 @@ export function supervise(o: SuperviseOptions): Promise<Supervised> {
|
||||
readNew();
|
||||
const done: Supervised['outcome'] = error ? 'spawn-failed' : (outcome ?? 'exited');
|
||||
if (current) finishFile(done !== 'exited' || exit !== 0 ? true : undefined);
|
||||
resolve({ outcome: done, exit: done === 'exited' ? exit : null, signal, start, end: size, lastFile, error, ms: Date.now() - started });
|
||||
resolve({ outcome: done, exit: done === 'exited' ? exit : null, signal, start, end: size, lastFile, observedStart, error, ms: Date.now() - started });
|
||||
};
|
||||
const timer = setInterval(() => {
|
||||
const now = Date.now();
|
||||
|
||||
@@ -95,7 +95,9 @@ export interface CheckResult extends PregateCheck {
|
||||
}
|
||||
|
||||
export type EnvironmentKind = 'missing-binary' | 'network' | 'quota' | 'disk' | 'runtime' | 'tool';
|
||||
export interface EnvironmentCause { kind: EnvironmentKind; evidence: string }
|
||||
/** How the cause was established: `launch` (the command could not start, its binary is off PATH, the runtime or a gstack tool refused), or `output` (a rule matched the command's text before any test was counted, which the diff's own output could have written). */
|
||||
export type EnvironmentProof = 'launch' | 'output';
|
||||
export interface EnvironmentCause { kind: EnvironmentKind; evidence: string; proof: EnvironmentProof }
|
||||
|
||||
export interface CheckDef {
|
||||
id: string;
|
||||
|
||||
@@ -46,9 +46,15 @@ describe('allowlist: policy validation', () => {
|
||||
expect(codes({ allowlist: [{ finding: 'env.kv', pattern: 'a'.repeat(257), reason: 'x' }] })).toEqual(['PREGATE_CONFIG_INVALID:allowlist[0].pattern']);
|
||||
// A pattern entry never covers a finding the scanner gave no position; a paths-only entry does.
|
||||
const positionless: AllowlistFinding = { id: 'env.kv', category: 'secret', severity: 'MEDIUM', file: 'evals/x.json' };
|
||||
expect(applyAllowlist([positionless], [TRIAL_ENTRY]).kept).toHaveLength(1);
|
||||
const inert = applyAllowlist([positionless], [TRIAL_ENTRY]);
|
||||
expect(inert.kept).toHaveLength(1);
|
||||
expect(inert.untested.map(u => [u.index, u.finding.id])).toEqual([[0, 'env.kv']]);
|
||||
expect(applyAllowlist([positionless], [{ finding: 'env.kv', paths: ['evals/**'], reason: 'r' }]).allowlisted).toHaveLength(1);
|
||||
expect(NON_ALLOWLISTABLE).toContain('env.value');
|
||||
// The scanner's own refusal (an oversize input fails closed) is never a benign finding: refused by validation, kept by the matcher.
|
||||
expect(codes({ allowlist: [{ finding: 'engine.input_too_large', paths: ['**'], reason: 'x' }] })).toEqual(['PREGATE_CONFIG_INVALID:allowlist[0].finding']);
|
||||
const refusal: AllowlistFinding = { id: 'engine.input_too_large', category: 'engine', severity: 'HIGH', file: 'big.json' };
|
||||
expect(applyAllowlist([refusal], [{ finding: 'engine.input_too_large', paths: ['**'], reason: 'x' }]).kept).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ import { afterEach, describe, expect, test } from 'bun:test';
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { EXIT } from '../lib/headless-artifacts';
|
||||
import { classifyEnvironment } from '../lib/pregate/environment';
|
||||
import { classifyCommandEnvironment, classifyEnvironment, commandBinary } from '../lib/pregate/environment';
|
||||
import { verifyPregate } from '../lib/pregate/index';
|
||||
import { BASE_PREGATE, cleanup, commit, makeFixtureRepo, runBin, type FixtureRepo } from './helpers/pregate-fixture';
|
||||
|
||||
@@ -30,10 +30,10 @@ describe('classifyEnvironment', () => {
|
||||
// seam=none; no_claim=completeness of the vocabulary
|
||||
test('names missing-binary, network, quota and disk, and stays null for an ordinary failure', () => {
|
||||
expect(classifyEnvironment('bash: line 1: vitest: command not found', 127)?.kind).toBe('missing-binary');
|
||||
expect(classifyEnvironment('', 127)).toEqual({ kind: 'missing-binary', evidence: 'exit 127: the shell found no such command' });
|
||||
expect(classifyEnvironment('write failed: Disk quota exceeded (EDQUOT)', 1)).toEqual({ kind: 'disk', evidence: 'write failed: Disk quota exceeded (EDQUOT)' });
|
||||
expect(classifyEnvironment('', 127)).toEqual({ kind: 'missing-binary', evidence: 'exit 127: the shell found no such command', proof: 'launch' });
|
||||
expect(classifyEnvironment('write failed: Disk quota exceeded (EDQUOT)', 1)).toEqual({ kind: 'disk', evidence: 'write failed: Disk quota exceeded (EDQUOT)', proof: 'output' });
|
||||
expect(classifyEnvironment("'nope' is not recognized as an internal or external command", 1)?.kind).toBe('missing-binary');
|
||||
expect(classifyEnvironment('Error: getaddrinfo ENOTFOUND registry.example.invalid', 1)).toEqual({ kind: 'network', evidence: 'getaddrinfo ENOTFOUND registry.example.invalid' });
|
||||
expect(classifyEnvironment('Error: getaddrinfo ENOTFOUND registry.example.invalid', 1)).toEqual({ kind: 'network', evidence: 'getaddrinfo ENOTFOUND registry.example.invalid', proof: 'output' });
|
||||
expect(classifyEnvironment('curl: (7) Failed to connect to host port 443: Connection refused', 7)?.kind).toBe('network');
|
||||
expect(classifyEnvironment('ubicloud: vcpu quota exceeded (528/512)', 1)?.kind).toBe('quota');
|
||||
expect(classifyEnvironment('HTTP 429 Too Many Requests', 1)?.kind).toBe('quota');
|
||||
@@ -41,6 +41,20 @@ describe('classifyEnvironment', () => {
|
||||
expect(classifyEnvironment('AssertionError: expected 3 to be 4', 1)).toBeNull();
|
||||
expect(classifyEnvironment('✖ fetches the thing\n ECONNREFUSED 127.0.0.1:3000\nℹ tests 1\nℹ fail 1', 1)?.kind).toBe('network');
|
||||
});
|
||||
|
||||
// Value: protects=exit 127 (or "command not found" text) is the machine's only when the command's own binary is off PATH: a script
|
||||
// the diff controls that exits 127 is the diff's; fails_when=a present runner exiting 127 reads as missing-binary; why_new=the veto
|
||||
// is new (adversarial review); seam=none; no_claim=none
|
||||
test('a missing binary is the machine\'s only when the command\'s own binary is off PATH', () => {
|
||||
expect(commandBinary("(cd 'pkg' && UBI_OWNER=x node run.js a b)")).toBe('node');
|
||||
expect(commandBinary('FOO=1 BAR=2 bun test x')).toBe('bun');
|
||||
expect(commandBinary('')).toBeNull();
|
||||
const env = { PATH: process.env.PATH };
|
||||
expect(classifyCommandEnvironment('node scripts/x.js', '', 127, env)).toBeNull();
|
||||
expect(classifyCommandEnvironment('node scripts/x.js', 'bash: vitest: command not found', 127, env)).toBeNull();
|
||||
expect(classifyCommandEnvironment('gstack-fixture-runner-that-does-not-exist x', '', 127, env)).toMatchObject({ kind: 'missing-binary', proof: 'launch' });
|
||||
expect(classifyCommandEnvironment('node scripts/x.js', 'Error: getaddrinfo ENOTFOUND registry.example.invalid', 1, env)).toMatchObject({ kind: 'network', proof: 'output' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('environment failures are reported apart from diff failures', () => {
|
||||
@@ -55,8 +69,9 @@ describe('environment failures are reported apart from diff failures', () => {
|
||||
expect(r.err).toContain('(PREGATE_ENVIRONMENT)');
|
||||
expect(r.err).not.toContain('PREGATE_LANE_FAILED');
|
||||
const c = check(repo, 'lanes');
|
||||
expect([c.status, c.code, c.cause, c.environment.kind]).toEqual(['incomplete', 'PREGATE_ENVIRONMENT', 'environment', 'missing-binary']);
|
||||
expect([c.status, c.code, c.cause, c.environment.kind, c.environment.proof]).toEqual(['incomplete', 'PREGATE_ENVIRONMENT', 'environment', 'missing-binary', 'launch']);
|
||||
expect(c.environment.evidence).toContain('gstack-fixture-runner-that-does-not-exist');
|
||||
expect(c.fix).toContain('the diff is not at fault');
|
||||
expect(r.out).toContain('environment: missing-binary');
|
||||
const verify = runBin(repo, 'gstack-pregate', ['verify']);
|
||||
expect(verify.status).toBe(EXIT.fail);
|
||||
@@ -82,6 +97,41 @@ describe('environment failures are reported apart from diff failures', () => {
|
||||
expect(d.environment).toBeUndefined();
|
||||
});
|
||||
|
||||
// Value: protects=a test in the diff cannot clear itself: a runner whose binary is present but exits 127 is PREGATE_LANE_FAILED
|
||||
// (diff), and a bun test that prints a network line and exits the process before the summary is at most an output-inferred
|
||||
// environment cause whose fix and verify message ask for confirmation elsewhere, never "the diff is not at fault";
|
||||
// fails_when=exit 127 from a present runner reads as missing-binary, or the spoof's fix clears the diff; why_new=the binary veto
|
||||
// and the launch/output proof are new (adversarial review); seam=none; no_claim=a spoof with no summary cannot be told from a
|
||||
// real network failure by the output alone, which is exactly what the proof records
|
||||
test('a present runner that exits 127 is the diff\'s; a test that exits with a network line is only an output-inferred cause', () => {
|
||||
const exit127 = fixture({ ...BASE_PREGATE, runner: 'node scripts/x.js {files}' }, { 'scripts/x.js': 'process.exit(127);\n' });
|
||||
commit(exit127, UTIL_EDIT);
|
||||
pregate(exit127, []);
|
||||
const c = check(exit127, 'lanes');
|
||||
expect([c.status, c.code, c.cause, c.environment]).toEqual(['fail', 'PREGATE_LANE_FAILED', 'diff', undefined]);
|
||||
if (!Bun.which('bun')) return;
|
||||
const spoof = fixture({ ...BASE_PREGATE, runner: 'bun test {files}' }, { 'test/util.test.js': "const { test } = require('bun:test');\nrequire('../lib/util.js');\ntest('spoof', () => { console.log('Received: fetch failed'); process.exit(1); });\n" });
|
||||
commit(spoof, UTIL_EDIT);
|
||||
pregate(spoof, []);
|
||||
const s = check(spoof, 'lanes');
|
||||
expect([s.status, s.code, s.environment.kind, s.environment.proof]).toEqual(['incomplete', 'PREGATE_ENVIRONMENT', 'network', 'output']);
|
||||
expect(s.fix).not.toContain('the diff is not at fault');
|
||||
expect(s.fix).toContain('confirm on another machine or with --remote');
|
||||
expect(s.lines.join('\n')).toContain('environment: network — Received: fetch failed (inferred from output)');
|
||||
const verify = runBin(spoof, 'gstack-pregate', ['verify']);
|
||||
expect(verify.err).toContain('lanes inferred from output alone');
|
||||
});
|
||||
|
||||
// Value: protects=a remote gate is classified from its preamble only: quota text streamed from the tests past the first lines
|
||||
// is not the environment; fails_when=a test name with "quota" waives the gate; why_new=new (adversarial review); seam=none; no_claim=none
|
||||
test('a remote gate whose test stream names a quota past the preamble is PREGATE_INCOMPLETE, not environment', () => {
|
||||
const repo = fixture({ ...BASE_PREGATE, remote_gate: 'node scripts/gate.js' }, { 'scripts/gate.js': "for (let i = 0; i < 60; i++) console.log('ok test ' + i);\nconsole.log('✗ billing > enforces quota per user');\nprocess.exit(1);\n" });
|
||||
commit(repo, UTIL_EDIT);
|
||||
pregate(repo, ['--remote', '--spend-cap', '1']);
|
||||
const c = check(repo, 'lanes');
|
||||
expect([c.status, c.code, c.environment]).toEqual(['incomplete', 'PREGATE_INCOMPLETE', undefined]);
|
||||
});
|
||||
|
||||
// Value: protects=a remote gate refused by a quota (no PREGATE_REMOTE line) is environment/quota and the obligation stays
|
||||
// uncleared; fails_when=it is a generic PREGATE_INCOMPLETE; why_new=new; seam=none; no_claim=none
|
||||
test('a remote gate that hit a quota is PREGATE_ENVIRONMENT quota and leaves the obligation uncleared', () => {
|
||||
@@ -107,6 +157,12 @@ describe('environment failures are reported apart from diff failures', () => {
|
||||
expect([g.status, g.code, g.cause]).toEqual(['fail', 'PREGATE_GUARD_FAILED', 'diff']);
|
||||
expect(g.lines.join('\n')).toContain('environment: missing-binary');
|
||||
expect(g.lines.join('\n')).not.toContain('red environment:');
|
||||
// A guard whose own binary is present but whose script exits 127 (a package script naming a missing tool) is the diff's.
|
||||
const present = fixture({ ...BASE_PREGATE, guards: [{ id: 'script', when: ['lib/**'], command: 'node -e "process.exit(127)"', fix: 'fix it' }] });
|
||||
commit(present, UTIL_EDIT);
|
||||
pregate(present, ['--only', 'guards']);
|
||||
const p = check(present, 'guards');
|
||||
expect([p.status, p.code, p.cause, p.environment]).toEqual(['fail', 'PREGATE_GUARD_FAILED', 'diff', undefined]);
|
||||
const only = fixture({ ...BASE_PREGATE, guards: [{ id: 'absent', when: ['lib/**'], command: 'gstack-fixture-guard-that-does-not-exist --check' }] });
|
||||
commit(only, UTIL_EDIT);
|
||||
pregate(only, ['--only', 'guards']);
|
||||
|
||||
@@ -43,6 +43,11 @@ describe('per-run scratch under the state root', () => {
|
||||
pregate(repo, ['--run', 'run-b', '--only', 'lanes']);
|
||||
expect(readOut(repo).scratch).toBe(path.join(runsDir(repo), 'run-b'));
|
||||
expect(fs.existsSync(path.join(dirA, 'lanes', 'lanes.log'))).toBe(true);
|
||||
// A taken --run id is never shared (a second run would truncate the first's logs): it takes a numbered sibling and the record names it.
|
||||
const again = pregate(repo, ['--run', 'run-a', '--only', 'lanes']);
|
||||
expect(readOut(repo).scratch).toBe(`${dirA}-2`);
|
||||
expect(again.out).toContain(`scratch: ${dirA}-2`);
|
||||
expect(fs.existsSync(path.join(dirA, 'guards', 'echo.log'))).toBe(true);
|
||||
});
|
||||
|
||||
// Value: protects=without --run the id is unique per process and the secrets scan files are gone after the scan; fails_when=two
|
||||
|
||||
Reference in new issue
Block a user