mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-02 23:14:33 +01:00
v1.93.18.0 fix(pregate): a green Free Tests run at the pregate tree clears gstack's remote gate (GST-1024) (#3883)
* pregate: gstack declares remote_gate_runs, so a green Free Tests run at the pregate tree clears remote_gate (GST-1024) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * fix: pre-landing review fixes (GST-1024: refuse a paid-only selection; validate skipped) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * docs(changelog): GST-1024 entry names all four tests Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
68410828fd
commit
8af121fd4c
11 files changed
+244
-26
No files matched your search
@@ -22,6 +22,11 @@
|
||||
"ci-reuse/free-tests": { "workflow": "free-tests.yml", "events": ["pull_request", "workflow_dispatch"], "jobs": ["free-plan", "typecheck", "quick-bun-default", "cso-macos-launcher", "macos-named-regressions", "cso-windows-launcher", "cso-docker-integration", "free-tests"], "artifact": "tested-tree", "os": "Linux" },
|
||||
"ci-reuse/windows-free-tests": { "workflow": "windows-free-tests.yml", "events": ["pull_request", "workflow_dispatch"], "jobs": ["windows-plan", "windows-free-tests"], "artifact": "tested-tree", "os": "Windows" }
|
||||
},
|
||||
"remote_gate_runs": {
|
||||
"profiles": ["ci-reuse/free-tests"],
|
||||
"inventory": "bun --no-env-file --config=/dev/null run scripts/test-free-shards.ts --list | sed -n 's/^ //p'",
|
||||
"skipped": "bun --no-env-file --config=/dev/null -e 'import { isPaidTestFile } from \"./test/helpers/paid-test-set.ts\"; import * as fs from \"node:fs\"; for (const f of fs.readFileSync(process.env.PREGATE_FILES_FILE, \"utf8\").split(/\\r?\\n/)) if (f && isPaidTestFile(f)) console.log(f);'"
|
||||
},
|
||||
"warn": [],
|
||||
"strays": { "allow": ["modules/loadouts.json", "modules/loadouts.schema.json"] }
|
||||
}
|
||||
@@ -1,5 +1,16 @@
|
||||
# Changelog
|
||||
|
||||
## [1.93.18.0] - 2026-11-02
|
||||
|
||||
**A green Free Tests run at the pregate tree now clears gstack's remote gate, so a slow Ubicloud VM no longer blocks a PR whose CI already passed (GST-1024).** docs/pregate.md described `gstack-pregate clear --remote pregate.json/remote_gate --run <id|url> --allow-repo-commands`, but gstack's own `.gstack/pregate.json` declared no `remote_gate_runs`, so the route always refused. On #3869, every check passed at 3f2bcf3 (Free Tests run 41484634592), while two remote gate runs at the same tree failed on bun timeouts in files the PR doesn't touch.
|
||||
|
||||
### Added
|
||||
- **`.gstack/pregate.json` `remote_gate_runs`:** the `ci-reuse/free-tests` profile, with `scripts/test-free-shards.ts --list` as the inventory (the complete free suite the Free Tests plan shards).
|
||||
- **`remote_gate_runs.skipped`** (optional, `lib/run-evidence.ts`, `lib/pregate/index.ts`, `lib/pregate/run-check.ts`): a command that reads the selected files from `PREGATE_FILES_FILE` and prints the ones the remote gate itself never runs. Those files need no run, and the receipt records them as `skipped=<n>`. An inventory file stays required even when `skipped` names it, and a failing command refuses with `skipped_failed`. When every selected file is skipped, the clear refuses with `paid_only`, just as the gate's own `tests=0` report never clears. gstack's command prints the selected paid test files, the same split `scripts/ubicloud/pregate-remote.sh` makes. Without it, run 41484634592 still refused: all 71 of the selected files it didn't cover are paid.
|
||||
|
||||
### Tests
|
||||
- `test/pregate-remote-gate-runs.test.ts`: gstack's inventory equals the Free Tests plan's file set, and its skipped command prints exactly the paid files of a mixed list. In a fixture with gstack's real profile, a green Free Tests `pull_request` run clears the gate. A run at another tree, a run of another workflow, a failed or cancelled run, a fork run, a merge commit behind its base, an uncovered inventory file, a skipped list naming an inventory file, a failing skipped command, a paid-only selection, and a declaration without `skipped` all refuse. Malformed `skipped` values fail config validation. On main, all four tests fail.
|
||||
|
||||
## [1.93.17.0] - 2026-11-02
|
||||
|
||||
- `gstack-ship-queue wave plan` now leaves out a READY head that does not merge onto main. It prints `NOT_MERGEABLE: #<n> (...)` with the conflicting files, the same way `NOT_GREEN` works for red CI. A conflict only in the CHANGELOG doesn't count, since the train merges those. Before, such a head was planned into a wave and the wave build stopped on it.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# gstack digest v1.93.17.0 — regenerate/re-copy after upgrading gstack
|
||||
# gstack digest v1.93.18.0 — regenerate/re-copy after upgrading gstack
|
||||
|
||||
Behavioral rules from gstack (https://github.com/garrytan/gstack), compressed
|
||||
for agent hosts without a full skill install. The full skills add workflows,
|
||||
|
||||
+29
-4
@@ -89,7 +89,7 @@ the byte-pinned `/ship` renders in `test/fixtures/golden/`).
|
||||
| `timeout_s` | seconds each runner process may take (the unit-file group, each serial file, each guard without its own `timeout_s`, and the remote gate's deadline) when `--timeout` is not passed; 1 to 3600, default 600. `gstack-ship-preflight` and the `finish` fallback run pregate without `--timeout`, so this is how a repo with long lanes sets theirs |
|
||||
| `remote_gate` | the repo's remote test gate command; under 8 cores the local test tier becomes a `requires-remote` obligation naming it, and `--remote` runs it (see [remote gate](#remote-gate)) |
|
||||
| `run_profiles` | name → `{ workflow, events, jobs, artifact, max_age_hours?, os? }`: the GitHub Actions runs `clear --run` verifies as evidence on the identical tree; a profile named after a lane (`free-tests.yml/macos-named-regressions`) clears that lane (see [run evidence](#run-evidence)). `--from-run` and the main-push reuse job verify runs against them too ([from-run](#from-run), [CI reuse](#ci-reuse)) |
|
||||
| `remote_gate_runs` | `{ profiles, inventory }`: the run profiles that together clear `pregate.json/remote_gate`, and the command that prints the remote gate's test files, one per line, which their runs must cover (see [run evidence](#run-evidence)) |
|
||||
| `remote_gate_runs` | `{ profiles, inventory, skipped? }`: the run profiles that together clear `pregate.json/remote_gate`, the command that prints the remote gate's test files, one per line, which their runs must cover, and an optional command that prints which selected files the gate itself never runs (see [run evidence](#run-evidence)) |
|
||||
| `allowlist` | `[{ finding, paths?, pattern?, reason }]`: benign findings the `secrets` check accepts and names as allowlisted, by gstack-redact finding id or non-secret category, path globs and a pattern anchored at the finding; never `env.value` (see [allowlist](#allowlist)) |
|
||||
|
||||
Only the `origin/<base>` copy or the user-owned file below is honoured, so a
|
||||
@@ -742,6 +742,16 @@ Then, for the lane:
|
||||
while the working tree is pregate.json's tree (checked before and after it
|
||||
runs); an empty or failing inventory refuses, and it runs under `pipefail`,
|
||||
so a pipeline whose first command fails refuses too.
|
||||
- When `remote_gate_runs` declares `skipped`, that command runs the same way,
|
||||
with `PREGATE_FILES_FILE` naming a file of the obligation's selected test
|
||||
files, one per line, and prints the ones the remote gate itself never runs
|
||||
(gstack's gate leaves paid test files to the paid PR gate). Those files need
|
||||
no run, and the receipt counts them as `skipped=<n>`. A file the inventory
|
||||
prints stays required even when `skipped` names it, and a failing `skipped`
|
||||
command refuses (`skipped_failed`). When `skipped` names every selected file,
|
||||
no selected test ran, so the clear refuses (`paid_only`), just as the gate's
|
||||
own `tests=0` report never clears. Without `skipped`, every selected file
|
||||
must appear in a run's `files`.
|
||||
- Every run is read once more after the inventory: a run that moved to a new
|
||||
attempt or stopped being a success meanwhile refuses (`run_changed`).
|
||||
|
||||
@@ -753,13 +763,14 @@ verification is named with its own reason (`fork_run`, `event_mismatch`,
|
||||
`os_mismatch`, `api_error`, …); the clear-level codes are `base_mismatch`,
|
||||
`no_verified_run`, `tree_mismatch` (runs on different trees),
|
||||
`tree_not_stamp_only`, `no_tests`, `inventory_failed`, `inventory_empty`,
|
||||
`inventory_uncovered` (the remote gate) and `tests_uncovered` (a lane's own
|
||||
`skipped_failed`, `paid_only`, `inventory_uncovered` (the remote gate) and `tests_uncovered` (a lane's own
|
||||
selected tests). A wrong run count, a repeated run or a lane with no
|
||||
profile is a usage error (exit 2), not a REUSE refusal.
|
||||
|
||||
Accepted, `clear` prints `REUSE: accepted <lane> <receipt>`, and the receipt
|
||||
reads `remote-gate tree=<t12> tests=<n> runs=<id>/<attempt>,… profiles=<p>,…
|
||||
heads=<h12> verified=<time>` (with `accepted-for=` and the substitution for a
|
||||
heads=<h12> verified=<time>` (with `skipped=<n>` after `tests=` when the
|
||||
`skipped` command excused selected files, and `accepted-for=` and the substitution for a
|
||||
stamp-only predecessor). A lane with its own profile gets `run tree=… runs=…`
|
||||
the same way; for such a lane `--run-url` is verified as `--run` is, and
|
||||
`--bundle-lane` is refused. A lane with no profile still clears by
|
||||
@@ -804,7 +815,21 @@ every non-matrix job and the `free-tests` aggregate, `os` Linux) and
|
||||
aggregate, `os` Windows). Each accepts `pull_request` and `workflow_dispatch`
|
||||
runs and reads the `tested-tree` artifact, which both aggregates upload only
|
||||
after their strict verification passes. The main-push reuse job and
|
||||
`--from-run` read them. No lane has a profile, so every macOS and Windows lane
|
||||
`--from-run` read them, and `remote_gate_runs` names `ci-reuse/free-tests`,
|
||||
so a green Free Tests run at pregate.json's tree clears
|
||||
`pregate.json/remote_gate` without a Ubicloud VM:
|
||||
|
||||
```bash
|
||||
gstack-pregate clear --remote pregate.json/remote_gate --run <free-tests run id or url> --allow-repo-commands
|
||||
```
|
||||
|
||||
Its `inventory` is `scripts/test-free-shards.ts --list` (the complete free
|
||||
suite, the same set the Free Tests plan shards), and its `skipped` command
|
||||
prints the selected files `test/helpers/paid-test-set.ts` marks paid, the
|
||||
same split `scripts/ubicloud/pregate-remote.sh` makes before it runs. A
|
||||
`pull_request` run counts only when the branch already contains its base
|
||||
(the merge commit's tree is the head's), so a branch behind main still needs
|
||||
the gate or a fresh run. No lane has a profile, so every macOS and Windows lane
|
||||
still clears by `--run-url`, checked for its workflow file and a green
|
||||
conclusion. On main the free tier's Windows smoke lane is
|
||||
`windows-smoke.yml/windows-smoke`, and a `windows-smoke.yml` run clears it.
|
||||
|
||||
@@ -40,9 +40,11 @@
|
||||
* run_profiles name → `{ workflow, events, jobs, artifact, max_age_hours?, os? }`:
|
||||
* the GitHub Actions runs `clear --run` accepts as evidence
|
||||
* (lib/run-evidence.ts); a profile named after a lane clears it
|
||||
* remote_gate_runs `{ profiles, inventory }`: the run profiles that together
|
||||
* clear the remote gate obligation on the identical tree, and
|
||||
* the command listing the gate's test files they must cover
|
||||
* remote_gate_runs `{ profiles, inventory, skipped? }`: the run profiles that
|
||||
* together clear the remote gate obligation on the identical
|
||||
* tree, the command listing the gate's test files they must
|
||||
* cover, and the command naming the selected files the gate
|
||||
* itself never runs (gstack: paid files)
|
||||
* allowlist `{ finding, paths?, pattern?, reason }` entries the `secrets`
|
||||
* check accepts as benign: a gstack-redact finding id or a
|
||||
* non-secret category, scoped by path globs and/or a regex
|
||||
@@ -354,7 +356,7 @@ export function explainConfig(): string {
|
||||
'timeout_s: seconds each runner process may take (the unit-file group, each serial file, each guard without its own timeout_s, and the remote gate\'s deadline) when --timeout is not passed; 1 to 3600, default 600',
|
||||
'remote_gate: the repo\'s remote test gate command; under 8 cores the test tier becomes a requires-remote obligation naming it, and gstack-pregate --remote runs it (it must print PREGATE_REMOTE: tree=<tree> tests=<n> failed=<n>) (docs/pregate.md#remote-gate)',
|
||||
'run_profiles: name → { workflow, events, jobs, artifact, max_age_hours?, os? }; the GitHub Actions runs gstack-pregate clear --run verifies as evidence on the identical tree (docs/pregate.md#run-evidence)',
|
||||
'remote_gate_runs: { profiles, inventory }; the run profiles that together clear the remote gate obligation, and the command listing the gate\'s test files they must cover (docs/pregate.md#run-evidence)',
|
||||
'remote_gate_runs: { profiles, inventory, skipped? }; the run profiles that together clear the remote gate obligation, the command listing the gate\'s test files they must cover, and an optional command printing which selected files (PREGATE_FILES_FILE) the gate itself does not run, which then need no run (docs/pregate.md#run-evidence)',
|
||||
`allowlist: [{ finding, paths?, pattern?, reason }]: benign findings the secrets check accepts and names as allowlisted. finding is a gstack-redact id (env.kv, google.api_key, pii.phone.e164, …) or a category (${ALLOWLISTABLE_CATEGORIES.join(', ')}); paths are globs; pattern is a regex matched at the finding's position in the added line; one of the two is required, reason always. Never ${NON_ALLOWLISTABLE.join(' or ')} or engine.* (docs/pregate.md#allowlist)`,
|
||||
'no .gstack/pregate.json on the base: <state root>/projects/<slug>/pregate.json is read instead (state root: GSTACK_STATE_ROOT, else GSTACK_HOME, else GSTACK_STATE_DIR, else a gstack plugin data dir, else ~/.gstack, per docs/state-root.md; run --explain prints the resolved path; outside the repo, so a PR cannot reach it; a path inside the checkout is ignored)',
|
||||
].join('\n') + '\n';
|
||||
|
||||
+13
-3
@@ -205,7 +205,7 @@ function userPolicyChange(file: PregateFile): string | null {
|
||||
|
||||
export interface ClearInput { runUrl?: string; bundleLane?: string; receiptPath?: string; runs?: string[]; evidence?: RunCheck }
|
||||
/** What a run clear needs: the base branch's run profiles, the verifier's dependencies, the clock, and the remote gate's inventory. */
|
||||
export interface RunCheck { repo: string; deps: RunDeps; profiles: Record<string, RunProfile>; gateRuns: RemoteGateRuns | null; now: Date; inventory: (command: string, tree: string) => string[] | { error: string } }
|
||||
export interface RunCheck { repo: string; deps: RunDeps; profiles: Record<string, RunProfile>; gateRuns: RemoteGateRuns | null; now: Date; inventory: (command: string, tree: string, selected?: string[]) => string[] | { error: string } }
|
||||
export type ClearDecision = { receipt: string; attested: boolean; substitution?: PregateReceiptSubstitution } | { refused: 'usage' | 'PREGATE_RECEIPT_INVALID'; message: string; fix?: string };
|
||||
|
||||
/**
|
||||
@@ -312,7 +312,8 @@ function treeSubstitution(file: PregateFile, gatedTree: string, lane: string, re
|
||||
/**
|
||||
* `clear --run`: GitHub Actions runs as the obligation's evidence (lib/run-evidence.ts). The remote gate needs one
|
||||
* verified run per `remote_gate_runs` profile, all on one tree, whose tree.json file lists together cover every file
|
||||
* the gate's inventory command prints; any other lane needs one verified run of the profile named after it. The run
|
||||
* the gate's inventory command prints and every selected file except those its skipped command says the gate itself
|
||||
* does not run (never an inventory file); any other lane needs one verified run of the profile named after it. The run
|
||||
* tree must be pregate.json's tree or a stamp-only predecessor of it, exactly as a gate report's tree must.
|
||||
*/
|
||||
function clearByRuns(file: PregateFile, lane: string, refs: string[], check: RunCheck | undefined, repoRoot: string): ClearDecision {
|
||||
@@ -359,10 +360,19 @@ function clearByRuns(file: PregateFile, lane: string, refs: string[], check: Run
|
||||
const idle = evidence.filter(e => (e.tests ?? e.files.length) === 0);
|
||||
if (idle.length) return { refused: 'PREGATE_RECEIPT_INVALID', message: `REUSE: refused reason=no_tests — ${idle.map(e => `run ${e.run_id} (${e.profile})`).join(', ')} records no tests; ${lane} stays uncleared` };
|
||||
const required = new Set(file.requires_remote?.find(r => r.lane === lane)?.tests ?? []);
|
||||
let skippedCount = 0;
|
||||
if (isGate) {
|
||||
const listed = check.inventory(check.gateRuns!.inventory, file.tree);
|
||||
if ('error' in listed) return { refused: 'PREGATE_RECEIPT_INVALID', message: `REUSE: refused reason=inventory_failed — the remote gate's inventory command failed: ${listed.error}; ${lane} stays uncleared` };
|
||||
if (!listed.length) return { refused: 'PREGATE_RECEIPT_INVALID', message: `REUSE: refused reason=inventory_empty — the remote gate's inventory command listed no test files; ${lane} stays uncleared` };
|
||||
const skipCommand = check.gateRuns!.skipped;
|
||||
if (skipCommand && required.size) {
|
||||
const skipped = check.inventory(skipCommand, file.tree, [...required]);
|
||||
if ('error' in skipped) return { refused: 'PREGATE_RECEIPT_INVALID', message: `REUSE: refused reason=skipped_failed — the remote gate's skipped command failed: ${skipped.error}; ${lane} stays uncleared` };
|
||||
const inventory = new Set(listed);
|
||||
for (const f of skipped) if (!inventory.has(f) && required.delete(f)) skippedCount++;
|
||||
if (!required.size) return { refused: 'PREGATE_RECEIPT_INVALID', message: `REUSE: refused reason=paid_only — the remote gate skips every selected test file (${skippedCount}), so no selected test ran; like the gate's own tests=0 report, that never clears; ${lane} stays uncleared` };
|
||||
}
|
||||
for (const f of listed) required.add(f);
|
||||
}
|
||||
const missing = uncoveredFiles([...required], evidence);
|
||||
@@ -375,7 +385,7 @@ function clearByRuns(file: PregateFile, lane: string, refs: string[], check: Run
|
||||
}
|
||||
const tests = evidence.reduce((n, e) => n + (e.tests ?? e.files.length), 0);
|
||||
const accepted = sub.substitution ? ` accepted-for=${file.tree.slice(0, 12)} stamp-only=${sub.substitution.files.length}` : '';
|
||||
return { receipt: `${isGate ? 'remote-gate' : 'run'} tree=${trees[0]!.slice(0, 12)} tests=${tests}${accepted} ${runReceiptFields(evidence, check.now)}`, attested: false, substitution: sub.substitution };
|
||||
return { receipt: `${isGate ? 'remote-gate' : 'run'} tree=${trees[0]!.slice(0, 12)} tests=${tests}${skippedCount ? ` skipped=${skippedCount}` : ''}${accepted} ${runReceiptFields(evidence, check.now)}`, attested: false, substitution: sub.substitution };
|
||||
}
|
||||
|
||||
/** Record the receipt that clears one obligation (a run URL, an evidence-bundle lane label, or a remote gate report, with the stamp-only substitution when one was accepted). */
|
||||
|
||||
@@ -1,12 +1,16 @@
|
||||
/**
|
||||
* The run-evidence inputs one consumer needs (GSTA-156): the base branch's run
|
||||
* profiles, this repository on GitHub, the verifier's gh- and git-backed
|
||||
* dependencies, and the remote gate's inventory command (a pin-file command,
|
||||
* so it runs only with --allow-repo-commands). Shared by `gstack-pregate
|
||||
* dependencies, and the remote gate's inventory and skipped commands (pin-file
|
||||
* commands, so they run only with --allow-repo-commands; skipped reads the
|
||||
* selected files from PREGATE_FILES_FILE). Shared by `gstack-pregate
|
||||
* clear` and `gstack-pregate --from-run`, so both read profiles from the
|
||||
* base and nowhere else.
|
||||
*/
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import * as fs from 'node:fs';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { ghRunDeps, runInventory } from '../run-evidence';
|
||||
import { wtreeFingerprint } from '../script-invocation';
|
||||
import { resolveBaseBranch } from '../ship-policy';
|
||||
@@ -28,13 +32,19 @@ export function buildRunCheck(root: string, lane: string | null, allowRepoComman
|
||||
if (!repo) return { error: 'origin is not a github.com repository; run evidence is read from GitHub Actions', profiled: lane === null ? false : lane === REMOTE_GATE_LANE ? !!loaded.config.remote_gate_runs : !!loaded.config.run_profiles[lane] };
|
||||
return {
|
||||
repo, deps: ghRunDeps(repo, root), profiles: loaded.config.run_profiles, gateRuns: loaded.config.remote_gate_runs, now: new Date(),
|
||||
inventory: (command, tree) => {
|
||||
inventory: (command, tree, selected) => {
|
||||
if (!allowRepoCommands) return { error: 'the inventory command is a pin-file command; pass --allow-repo-commands' };
|
||||
const at = () => { const fp = wtreeFingerprint({ cwd: root }); return 'tree' in fp ? fp.tree : null; };
|
||||
if (at() !== tree) return { error: `the working tree is not pregate.json's tree ${tree.slice(0, 12)}; the inventory must list that tree's gate files` };
|
||||
const listed = runInventory(command, root);
|
||||
if (at() !== tree) return { error: `the working tree changed while ${command} ran` };
|
||||
return listed;
|
||||
const dir = selected && fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-pregate-skipped-'));
|
||||
try {
|
||||
if (dir) fs.writeFileSync(path.join(dir, 'selected.txt'), selected!.map(f => `${f}\n`).join(''));
|
||||
const listed = runInventory(command, root, dir ? { PREGATE_FILES_FILE: path.join(dir, 'selected.txt') } : undefined);
|
||||
if (at() !== tree) return { error: `the working tree changed while ${command} ran` };
|
||||
return listed;
|
||||
} finally {
|
||||
if (dir) fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
+7
-6
@@ -43,7 +43,7 @@ import { sha256Hex, writeReceipt } from './egress-receipt';
|
||||
import { runBashSync } from './script-invocation';
|
||||
|
||||
export interface RunProfile { workflow: string; events: string[]; jobs: string[]; artifact: string; max_age_hours?: number; os?: string }
|
||||
export interface RemoteGateRuns { profiles: string[]; inventory: string }
|
||||
export interface RemoteGateRuns { profiles: string[]; inventory: string; skipped?: string }
|
||||
export interface ApiRun {
|
||||
id: number; run_attempt: number; status: string; conclusion: string | null; event: string; head_sha: string;
|
||||
path?: string; created_at?: string; run_started_at?: string; updated_at?: string;
|
||||
@@ -108,15 +108,16 @@ export function runProfileErrors(profiles: unknown, gateRuns: unknown): Array<{
|
||||
}
|
||||
if (gateRuns !== undefined && gateRuns !== null) {
|
||||
const g = gateRuns as Record<string, unknown>;
|
||||
if (!g || typeof g !== 'object' || Array.isArray(g)) errors.push({ path: 'remote_gate_runs', message: 'must be { profiles: [names], inventory: command }' });
|
||||
if (!g || typeof g !== 'object' || Array.isArray(g)) errors.push({ path: 'remote_gate_runs', message: 'must be { profiles: [names], inventory: command, skipped?: command }' });
|
||||
else {
|
||||
for (const k of Object.keys(g)) if (k !== 'profiles' && k !== 'inventory') errors.push({ path: `remote_gate_runs.${k}`, message: 'unknown key; valid keys: profiles, inventory' });
|
||||
for (const k of Object.keys(g)) if (k !== 'profiles' && k !== 'inventory' && k !== 'skipped') errors.push({ path: `remote_gate_runs.${k}`, message: 'unknown key; valid keys: profiles, inventory, skipped' });
|
||||
if (!Array.isArray(g.profiles) || g.profiles.length === 0 || g.profiles.some(p => typeof p !== 'string')) errors.push({ path: 'remote_gate_runs.profiles', message: 'must be a non-empty array of run_profiles names' });
|
||||
else {
|
||||
for (const p of g.profiles as string[]) if (!names.has(p)) errors.push({ path: 'remote_gate_runs.profiles', message: `${JSON.stringify(p)} is not a run_profiles name` });
|
||||
if (new Set(g.profiles).size !== g.profiles.length) errors.push({ path: 'remote_gate_runs.profiles', message: 'names each profile once' });
|
||||
}
|
||||
if (typeof g.inventory !== 'string' || !g.inventory.trim() || g.inventory.length > 512 || /[\0\r\n]/.test(g.inventory)) errors.push({ path: 'remote_gate_runs.inventory', message: 'must be one command line under 512 characters that prints the remote gate\'s test files, one per line' });
|
||||
if ('skipped' in g && (typeof g.skipped !== 'string' || !g.skipped.trim() || g.skipped.length > 512 || /[\0\r\n]/.test(g.skipped))) errors.push({ path: 'remote_gate_runs.skipped', message: 'must be one command line under 512 characters that prints which selected test files (PREGATE_FILES_FILE) the remote gate itself does not run, one per line' });
|
||||
}
|
||||
}
|
||||
return errors;
|
||||
@@ -201,9 +202,9 @@ export function runChanged(deps: RunDeps, id: number, attempt: number): string |
|
||||
return again.run_attempt === attempt && again.status === 'completed' && again.conclusion === 'success' ? null : `run ${id} moved to attempt ${again.run_attempt} (${again.status === 'completed' ? again.conclusion : again.status}) while it was being verified`;
|
||||
}
|
||||
|
||||
/** The remote gate's inventory: `command`'s non-empty output lines under pipefail, so a pipeline whose producer fails refuses instead of listing part of the gate. */
|
||||
export function runInventory(command: string, cwd: string): string[] | { error: string } {
|
||||
const r = runBashSync(`set -o pipefail\n${command}`, { cwd, encoding: 'utf8', timeout: 300_000, maxBuffer: 64 * 1024 * 1024 });
|
||||
/** The remote gate's inventory (or skipped list, with PREGATE_FILES_FILE in `env`): `command`'s non-empty output lines under pipefail, so a pipeline whose producer fails refuses instead of listing part of the gate. */
|
||||
export function runInventory(command: string, cwd: string, env?: Record<string, string>): string[] | { error: string } {
|
||||
const r = runBashSync(`set -o pipefail\n${command}`, { cwd, encoding: 'utf8', timeout: 300_000, maxBuffer: 64 * 1024 * 1024, ...(env && { env: { ...process.env, ...env } }) });
|
||||
if (r.status !== 0 || r.error) return { error: `${command} exited ${r.status ?? r.error}: ${(r.stderr ?? '').trim().slice(0, 200)}` };
|
||||
return [...new Set(r.stdout.split('\n').map(l => l.trim()).filter(Boolean))];
|
||||
}
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gstack",
|
||||
"version": "1.93.17",
|
||||
"version": "1.93.18",
|
||||
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
/**
|
||||
* GST-1024: gstack's own .gstack/pregate.json declares remote_gate_runs, so a green Free Tests run at the pregate tree
|
||||
* clears pregate.json/remote_gate through `gstack-pregate clear --run`. The declaration's commands run against this
|
||||
* checkout (the inventory must be exactly the free suite CI plans, the skipped list exactly the paid files
|
||||
* pregate-remote.sh leaves to the paid gate); the clear runs through buildRunCheck in a fixture repository with this
|
||||
* repository's real run profile and a fake Actions API shaped like a real Free Tests pull_request run (41484634592).
|
||||
*/
|
||||
import { afterEach, describe, expect, test } from 'bun:test';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import * as fs from 'node:fs';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { bunIsolationArgs } from '../lib/bun-isolation';
|
||||
import { buildContext } from '../lib/pregate/context';
|
||||
import { clearDecision, runPregate, type RunCheck } from '../lib/pregate/index';
|
||||
import { buildRunCheck } from '../lib/pregate/run-check';
|
||||
import { runInventory, runProfileErrors, type ApiRun, type RemoteGateRuns, type RunProfile } from '../lib/run-evidence';
|
||||
import { isPaidTestFile } from './helpers/paid-test-set';
|
||||
import { BASE_PREGATE, cleanup, commit, makeFixtureRepo, type FixtureRepo } from './helpers/pregate-fixture';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const REAL = JSON.parse(fs.readFileSync(path.join(ROOT, '.gstack/pregate.json'), 'utf8')) as { run_profiles: Record<string, RunProfile>; remote_gate_runs?: RemoteGateRuns };
|
||||
const LANE = 'pregate.json/remote_gate';
|
||||
const PROFILE = 'ci-reuse/free-tests';
|
||||
const PAID = 'test/skill-e2e-ship-gate.test.ts';
|
||||
|
||||
const dirs: string[] = [];
|
||||
const repos: FixtureRepo[] = [];
|
||||
afterEach(() => {
|
||||
while (repos.length) cleanup(repos.pop()!);
|
||||
while (dirs.length) fs.rmSync(dirs.pop()!, { recursive: true, force: true });
|
||||
});
|
||||
const tmp = () => { const d = fs.mkdtempSync(path.join(os.tmpdir(), 'gst1024-')); dirs.push(d); return d; };
|
||||
const lines = (text: string) => text.split('\n').filter(Boolean);
|
||||
|
||||
describe.skipIf(process.platform === 'win32')('gstack declares remote_gate_runs (GST-1024)', () => {
|
||||
test('the declaration names the Free Tests profile; its inventory is exactly the free suite CI plans, and its skipped list is exactly the selected paid files', () => {
|
||||
const g = REAL.remote_gate_runs;
|
||||
expect(g?.profiles).toEqual([PROFILE]);
|
||||
expect(REAL.run_profiles[PROFILE]).toMatchObject({ workflow: 'free-tests.yml', artifact: 'tested-tree', os: 'Linux' });
|
||||
expect(runProfileErrors(REAL.run_profiles, g)).toEqual([]);
|
||||
const listed = runInventory(g!.inventory, ROOT);
|
||||
if ('error' in listed) throw new Error(listed.error);
|
||||
const planFile = path.join(tmp(), 'plan.json');
|
||||
const plan = spawnSync('bun', [...bunIsolationArgs(), 'run', 'scripts/test-free-shards.ts', '--ci-plan', planFile, '--shards', '11', '--shard-jobs', '2'], { cwd: ROOT, encoding: 'utf8', timeout: 120_000 });
|
||||
expect(plan.status).toBe(0);
|
||||
const planned = (JSON.parse(fs.readFileSync(planFile, 'utf8')) as { shards: Array<{ files: string[] }> }).shards.flatMap(s => s.files);
|
||||
expect(listed.length).toBeGreaterThan(1000);
|
||||
expect([...listed].sort()).toEqual([...new Set(planned)].sort());
|
||||
expect(listed.filter(isPaidTestFile)).toEqual([]);
|
||||
const selected = path.join(tmp(), 'selected.txt');
|
||||
fs.writeFileSync(selected, [listed[0], PAID, 'test/codex-e2e-plan-format.test.ts', listed[1]].join('\n') + '\n');
|
||||
const skipped = runInventory(g!.skipped!, ROOT, { PREGATE_FILES_FILE: selected });
|
||||
expect(skipped).toEqual([PAID, 'test/codex-e2e-plan-format.test.ts']);
|
||||
}, 180_000);
|
||||
});
|
||||
|
||||
const INVENTORY = 'printf "test/tool-cli.test.js\\ntest/other.test.js\\n"';
|
||||
const SKIPPED = 'grep -E "^test/skill-e2e-" "$PREGATE_FILES_FILE" || true';
|
||||
|
||||
function fixture(gateRuns: Record<string, unknown>): FixtureRepo {
|
||||
const repo = makeFixtureRepo({ pregate: { ...BASE_PREGATE, remote_gate: 'node scripts/remote-gate.js', run_profiles: REAL.run_profiles, remote_gate_runs: gateRuns } });
|
||||
repos.push(repo);
|
||||
repo.git('checkout', '-q', 'main');
|
||||
commit(repo, { 'scripts/remote-gate.js': "console.log('PREGATE_REMOTE: tree=' + process.env.PREGATE_TREE + ' tests=3 failed=0');\n" }, 'remote gate');
|
||||
repo.git('push', '-q', 'origin', 'main');
|
||||
repo.git('checkout', '-q', 'feature');
|
||||
repo.git('merge', '-q', '--ff-only', 'main');
|
||||
commit(repo, { 'lib/util.js': 'exports.add = (a, b) => a + b + 0;\n' }, 'the real change');
|
||||
return repo;
|
||||
}
|
||||
|
||||
/** A Free Tests pull_request run as GitHub reports it: the profile's jobs, a tested-tree artifact naming the merge commit. */
|
||||
interface Run { head: string; files: string[]; conclusion?: string; path?: string; headRepo?: string; mergeTree?: string }
|
||||
function gateCheck(repo: FixtureRepo, run: Run): RunCheck {
|
||||
const built = buildRunCheck(repo.work, LANE, true, 'main');
|
||||
if ('error' in built) throw new Error(built.error);
|
||||
const profile = REAL.run_profiles[PROFILE]!;
|
||||
const merge = 'f'.repeat(40);
|
||||
const finished = new Date(Date.now() - 3600_000).toISOString();
|
||||
const api: ApiRun = {
|
||||
id: 41484634592, run_attempt: 1, status: 'completed', conclusion: run.conclusion ?? 'success', event: 'pull_request', head_sha: run.head,
|
||||
path: run.path ?? `.github/workflows/${profile.workflow}`, created_at: finished, updated_at: finished,
|
||||
repository: { full_name: 'fx/fx' }, head_repository: { full_name: run.headRepo ?? 'fx/fx' },
|
||||
};
|
||||
const tree = (sha: string) => repo.git('rev-parse', `${sha}^{tree}`);
|
||||
return {
|
||||
...built,
|
||||
repo: 'fx/fx',
|
||||
deps: {
|
||||
run: id => { if (id !== api.id) throw new Error('HTTP 404'); return api; },
|
||||
runsForHead: () => [api],
|
||||
jobs: () => profile.jobs.map(name => ({ name, conclusion: 'success', completed_at: finished })),
|
||||
artifact: () => ({ run_id: api.id, run_attempt: 1, event: 'pull_request', sha: merge, head_sha: run.head, tree: tree(run.head), workflow: profile.workflow, os: ['Linux', 'macOS', 'Windows'], files: run.files, tests: run.files.length * 20 }),
|
||||
commitTree: sha => (sha === merge ? run.mergeTree ?? tree(run.head) : tree(sha)),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function gatedFile(repo: FixtureRepo) {
|
||||
const loaded = buildContext({ repoRoot: repo.work, base: 'main', policyFrom: 'base', allowRepoCommands: true, env: { ...process.env, ...repo.env }, progress: () => {}, cores: 4, testBackend: 'local' });
|
||||
if (!('ctx' in loaded)) throw new Error('fixture context refused');
|
||||
const file = await runPregate(loaded.ctx, { stage: 'tests' });
|
||||
const gate = file.requires_remote?.find(r => r.lane === LANE);
|
||||
if (!gate) throw new Error('no remote gate obligation');
|
||||
gate.tests = [...(gate.tests ?? []), PAID];
|
||||
repo.git('config', 'remote.origin.url', 'https://github.com/fx/fx.git');
|
||||
return { file, selected: gate.tests.filter(f => f !== PAID) };
|
||||
}
|
||||
|
||||
describe.skipIf(process.platform === 'win32')('clear --run with the Free Tests profile (GST-1024)', () => {
|
||||
const RUN = 'https://github.com/fx/fx/actions/runs/41484634592';
|
||||
const msg = (d: ReturnType<typeof clearDecision>) => ('refused' in d ? `${d.refused}: ${d.message}` : `cleared ${d.receipt}`);
|
||||
|
||||
test('a green Free Tests run at the pregate tree clears the gate; the selected paid file the gate skips is counted, not required', async () => {
|
||||
const repo = fixture({ profiles: [PROFILE], inventory: INVENTORY, skipped: SKIPPED });
|
||||
const head = repo.git('rev-parse', 'HEAD');
|
||||
const { file, selected } = await gatedFile(repo);
|
||||
const files = [...new Set([...selected, 'test/tool-cli.test.js', 'test/other.test.js'])];
|
||||
const d = clearDecision(file, LANE, { runs: [RUN], evidence: gateCheck(repo, { head, files }) }, repo.work);
|
||||
expect(msg(d)).toStartWith(`cleared remote-gate tree=${file.tree.slice(0, 12)} tests=${files.length * 20} skipped=1 runs=41484634592/1 profiles=${PROFILE} heads=${head.slice(0, 12)} verified=`);
|
||||
}, 120_000);
|
||||
|
||||
test('refusals: no skipped command, a paid-only selection, the wrong tree, the wrong workflow, a failed or cancelled run, a fork run, a merge behind its base, an uncovered inventory file, a skipped list that names an inventory file', async () => {
|
||||
const repo = fixture({ profiles: [PROFILE], inventory: INVENTORY, skipped: SKIPPED });
|
||||
const older = repo.git('rev-parse', 'HEAD~1');
|
||||
const head = repo.git('rev-parse', 'HEAD');
|
||||
const { file, selected } = await gatedFile(repo);
|
||||
const files = [...new Set([...selected, 'test/tool-cli.test.js', 'test/other.test.js'])];
|
||||
const clear = (run: Run, over: Partial<RunCheck> = {}) => msg(clearDecision(file, LANE, { runs: [RUN], evidence: { ...gateCheck(repo, run), ...over } }, repo.work));
|
||||
expect(clear({ head, files }, { gateRuns: { profiles: [PROFILE], inventory: INVENTORY } })).toMatch(new RegExp(`reason=inventory_uncovered — 1 of the \\d+ test file\\(s\\) ${LANE} must cover appear in no run's tree.json \\(${PAID}\\)`));
|
||||
expect(clear({ head: older, files })).toMatch(/^PREGATE_RECEIPT_INVALID: REUSE: refused reason=tree_not_stamp_only — .*lib\/util\.js/);
|
||||
expect(clear({ head, files, path: '.github/workflows/windows-free-tests.yml' })).toContain('reason=workflow_mismatch run 41484634592 attempt 1 ran .github/workflows/windows-free-tests.yml, not .github/workflows/free-tests.yml');
|
||||
expect(clear({ head, files, conclusion: 'failure' })).toContain('reason=run_not_success run 41484634592 attempt 1 concluded failure');
|
||||
expect(clear({ head, files, conclusion: 'cancelled' })).toContain('reason=run_not_success run 41484634592 attempt 1 concluded cancelled');
|
||||
expect(clear({ head, files, headRepo: 'attacker/fx' })).toContain('reason=fork_run run 41484634592 attempt 1 ran for attacker/fx, not fx/fx');
|
||||
expect(clear({ head, files, mergeTree: repo.git('rev-parse', `${older}^{tree}`) })).toContain('reason=tree_mismatch');
|
||||
expect(clear({ head, files: files.filter(f => f !== 'test/other.test.js') })).toContain("reason=inventory_uncovered — 1 of the");
|
||||
expect(clear({ head, files: files.filter(f => f !== 'test/other.test.js') }, { gateRuns: { profiles: [PROFILE], inventory: INVENTORY, skipped: `${SKIPPED}; echo test/other.test.js` } })).toContain('(test/other.test.js)');
|
||||
expect(clear({ head, files }, { gateRuns: { profiles: [PROFILE], inventory: INVENTORY, skipped: 'exit 3' } })).toContain('reason=skipped_failed');
|
||||
const paidOnly = structuredClone(file);
|
||||
paidOnly.requires_remote!.find(r => r.lane === LANE)!.tests = [PAID];
|
||||
expect(msg(clearDecision(paidOnly, LANE, { runs: [RUN], evidence: gateCheck(repo, { head, files }) }, repo.work))).toBe(`PREGATE_RECEIPT_INVALID: REUSE: refused reason=paid_only — the remote gate skips every selected test file (1), so no selected test ran; like the gate's own tests=0 report, that never clears; ${LANE} stays uncleared`);
|
||||
}, 120_000);
|
||||
|
||||
test('remote_gate_runs.skipped must be one command line, and an unknown key names it as valid', () => {
|
||||
const profiles = { p: { workflow: 'w.yml', events: ['push'], jobs: ['j'], artifact: 'a' } };
|
||||
for (const skipped of [3, '', ' ', 'a\nb', 'x'.repeat(513)]) {
|
||||
expect(runProfileErrors(profiles, { profiles: ['p'], inventory: 'echo t', skipped }).map(e => e.path)).toEqual(['remote_gate_runs.skipped']);
|
||||
}
|
||||
expect(runProfileErrors(profiles, { profiles: ['p'], inventory: 'echo t', skipped: 'echo s' })).toEqual([]);
|
||||
expect(runProfileErrors(profiles, { profiles: ['p'], inventory: 'echo t', skip: 'echo s' })).toEqual([{ path: 'remote_gate_runs.skip', message: 'unknown key; valid keys: profiles, inventory, skipped' }]);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user