Files
gstack/lib/pregate/config.ts
T
garrytanandcapy-ai[bot] d6f4ea9f12 fix(pregate): test stage never hangs silently; small machines record the remote gate (Lane C, GBRA-95)
C1 diagnosis (gbrain 92f0ea87, a core edit selecting 3,683 local files, 358
serial; Bun 1.4.2 and 1.4.3 identical): --timeout was applied per runner
spawn (unit group plus one per serial file), so the check was unbounded, and
spawnSync buffered all output until it returned: 360 s with 0 bytes on stdout
and stderr. A backgrounded child holding stdout made spawnSync wait the full
timeout and then report exit 0 (a silent wait, then green). Not the Bun 1.4.2
spawnSync hang.

C2/C-b: test-tier commands (lanes runner, guards, the --remote gate) run via
lib/pregate/supervise.ts: own process group (setsid), stdout/stderr appended
to a log file (no pipes), async supervision; CheckDef.run may return a
promise and runPregate/withFingerprint await it (a rejection is incomplete).
Leftover group members are killed after exit; SIGINT/SIGTERM to pregate
SIGKILLs running groups, writes no pregate.json, exits 130/143. Windows:
taskkill /T /F (reparented or detached descendants can survive; documented).

C3/C-a: --timeout bounds the whole check; groups past the deadline are not
started. --stall-timeout <s> (default stall_timeout_s, else 300; 0 disables)
kills a group whose log stops growing and records PREGATE_STALLED naming the
last file started. Live stderr: PREGATE_TEST per file (bun test adapter; any
runner for one-file runs), PREGATE_HEARTBEAT after 60 s without progress,
PREGATE: tests <verdict>. --json stdout unchanged.

C4/C-c: under 8 cores, a base-branch remote_gate or a non-local test_backend
records the local tier as a requires-remote obligation with its command
(pregate.json/remote_gate or test_backend/<backend>). --remote runs the
declared remote_gate now (needs --allow-repo-commands, never --policy-from
head) under --spend-cap/GSTACK_SPEND_CAP_USD; it clears only on
PREGATE_REMOTE: tree=<tested tree> tests>0 failed=0 with exit 0, and reports
spent/reserved/unknown.

G1: new code PREGATE_STALLED has its result-codes row, troubleshooting anchor
and the installer-form error line (CODE; url#anchor).

Files (G2): lib/pregate/supervise.ts (new), lib/pregate/types.ts,
lib/pregate/context.ts, lib/pregate/index.ts, lib/pregate/checks/lanes.ts,
lib/pregate/checks/guards.ts, lib/pregate/config.ts, lib/headless-artifacts.ts,
lib/result-codes.ts, bin/gstack-pregate, bin/gstack-doctor-components.sh
(generated), docs/pregate.md, docs/troubleshooting.md,
test/pregate-supervise.test.ts (new), test/pregate-remote.test.ts (new).

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
2026-10-15 00:23:34 +00:00

235 lines
16 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* pregate/config — `.gstack/pregate.json` (plan C8): the repo's declarations
* for the pre-gate. Read from the committed copy on origin/<base> under the
* C1 trust boundary (lib/ship-policy.ts loaders are the model); file presence
* never authorizes a command. Keys:
*
* runner the test command with a `{files}` placeholder (default: detected)
* dependencies glob → test files for subprocess targets, runtime reads and
* generated inputs the import graph cannot see; `[]` declares
* "no lane needed" for docs and the like
* selection `<workflow.yml>/<job>` → the runner-owned command that lists
* the lane's test files when the matrix is planner-computed
* warn check ids downgraded to warn (never `secrets`; `lanes` keeps
* its required-lane and requires-remote verdicts)
* strays.allow root files or new top-level directories that are intentional
* guards `{ id, when: [globs], command, fix?, timeout_s? }`: a repo
* command that must pass when a touched file matches `when`
* (manifest checks, CLI goldens); never downgradable
* serial_patterns globs of test files the lanes check runs one process each
* (default: every `*.serial.test.*` file)
* sealed the sealed-output guard's rules (lib/sealed-guard.ts; read by
* the pre-push hook and `gstack-pregate sealed`, not by `run`)
* price_file the call-plan data file `gstack-eval-plan price-gate` prices
* a paid gate from (GSTACK_EVAL_CALLS v1; never executed)
* stall_timeout_s seconds without test output before the lanes check kills
* the runner's process group (default 300; 0 disables)
* remote_gate the repo's remote test gate command: under 8 cores the test
* tier is a requires-remote obligation naming it, and
* `--remote` runs it (it prints `PREGATE_REMOTE: tree=… tests=… failed=…`)
*/
import * as fs from 'node:fs';
import * as path from 'node:path';
import { spawnSync } from 'node:child_process';
import type { ResultCodeName } from '../result-codes';
import { sealedErrors } from '../sealed-guard';
import { containedRelative, globToRegExp } from '../ship-policy';
export const PREGATE_CONFIG_PATH = '.gstack/pregate.json';
export const PREGATE_CONFIG_SCHEMA = 'https://github.com/garrytan/gstack/docs/pregate.md#pregate-v1';
export const NON_DOWNGRADABLE = ['secrets', 'guards'] as const;
export const DEFAULT_SERIAL_PATTERNS = ['**/*.serial.test.*'];
export interface PregateConfig {
$schema?: string;
runner: string | null;
dependencies: Record<string, string[]>;
selection: Record<string, string>;
warn: string[];
strays: { allow: string[] };
guards: Guard[];
serial_patterns: string[];
price_file: string | null;
stall_timeout_s: number | null;
remote_gate: string | null;
}
export interface Guard { id: string; when: string[]; command: string; fix?: string; timeout_s?: number }
export const DEFAULT_CONFIG: PregateConfig = { runner: null, dependencies: {}, selection: {}, warn: [], strays: { allow: [] }, guards: [], serial_patterns: DEFAULT_SERIAL_PATTERNS, price_file: null, stall_timeout_s: null, remote_gate: null };
export interface ConfigError { code: ResultCodeName; path: string; message: string }
const MAX_LIST = 256;
const MAX_COMMAND = 512;
const KEYS = ['runner', 'dependencies', 'selection', 'warn', 'strays', 'guards', 'serial_patterns', 'price_file', 'sealed', 'stall_timeout_s', 'remote_gate'];
const MAX_GUARDS = 32;
const GUARD_KEYS = ['id', 'when', 'command', 'fix', 'timeout_s'];
function globError(g: string): string | null {
const why = containedRelative(g);
if (why) return why;
try { globToRegExp(g); } catch (e: any) { return e.message; }
return null;
}
function commandError(c: unknown): string | null {
if (typeof c !== 'string' || !c.trim()) return 'must be a non-empty string';
if (c.length > MAX_COMMAND || /[\0\r\n]/.test(c)) return `must be one line under ${MAX_COMMAND} characters`;
return null;
}
function guardErrors(raw: unknown, errors: ConfigError[]): void {
const bad = (p: string, message: string) => errors.push({ code: 'PREGATE_CONFIG_INVALID', path: p, message });
if (!Array.isArray(raw)) { bad('guards', 'must be an array of { id, when, command, fix?, timeout_s? }'); return; }
if (raw.length > MAX_GUARDS) bad('guards', `more than ${MAX_GUARDS} guards`);
const seen = new Set<string>();
raw.forEach((g, i) => {
const at = `guards[${i}]`;
if (!g || typeof g !== 'object' || Array.isArray(g)) { bad(at, 'must be an object { id, when, command, fix?, timeout_s? }'); return; }
const o = g as Record<string, unknown>;
for (const k of Object.keys(o)) if (!GUARD_KEYS.includes(k)) bad(`${at}.${k}`, `unknown key; valid keys: ${GUARD_KEYS.join(', ')}`);
if (typeof o.id !== 'string' || !/^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/.test(o.id)) bad(`${at}.id`, 'must match [A-Za-z0-9][A-Za-z0-9._-]{0,63}');
else if (seen.has(o.id)) bad(`${at}.id`, `duplicate guard id ${o.id}`);
else seen.add(o.id);
if (!Array.isArray(o.when) || o.when.length === 0 || o.when.some(w => typeof w !== 'string')) bad(`${at}.when`, 'must be a non-empty array of globs');
else (o.when as string[]).forEach((w, j) => { const why = globError(w); if (why) bad(`${at}.when[${j}]`, `${JSON.stringify(w)} ${why}`); });
const cmd = commandError(o.command);
if (cmd) bad(`${at}.command`, cmd);
if ('fix' in o && commandError(o.fix)) bad(`${at}.fix`, commandError(o.fix)!);
if ('timeout_s' in o && (!Number.isInteger(o.timeout_s) || (o.timeout_s as number) < 1 || (o.timeout_s as number) > 3600)) bad(`${at}.timeout_s`, 'must be an integer number of seconds from 1 to 3600');
});
}
export function validateConfig(raw: unknown, knownChecks: readonly string[] = []): ConfigError[] {
const errors: ConfigError[] = [];
if (!raw || typeof raw !== 'object' || Array.isArray(raw)) return [{ code: 'PREGATE_CONFIG_INVALID', path: '', message: 'config must be a JSON object' }];
const obj = raw as Record<string, unknown>;
for (const key of Object.keys(obj)) if (key !== '$schema' && !KEYS.includes(key)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: key, message: `unknown key; valid keys: ${KEYS.join(', ')}` });
if ('runner' in obj && obj.runner !== null) {
const why = commandError(obj.runner);
if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'runner', message: why });
else if (!(obj.runner as string).includes('{files}')) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'runner', message: 'must contain the {files} placeholder' });
}
if ('price_file' in obj && obj.price_file !== null) {
const why = typeof obj.price_file === 'string' ? containedRelative(obj.price_file) : 'must be a repo-relative path or null';
if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'price_file', message: why });
}
if ('stall_timeout_s' in obj && obj.stall_timeout_s !== null && (!Number.isInteger(obj.stall_timeout_s) || (obj.stall_timeout_s as number) < 0 || (obj.stall_timeout_s as number) > 86_400)) {
errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'stall_timeout_s', message: 'must be an integer number of seconds from 0 (disabled) to 86400, or null' });
}
if ('remote_gate' in obj && obj.remote_gate !== null) {
const why = commandError(obj.remote_gate);
if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'remote_gate', message: why });
}
if ('guards' in obj) guardErrors(obj.guards, errors);
if ('sealed' in obj) for (const e of sealedErrors(obj.sealed)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: e.path, message: e.message });
if ('serial_patterns' in obj) {
const list = obj.serial_patterns;
if (!Array.isArray(list) || list.some(g => typeof g !== 'string')) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'serial_patterns', message: 'must be an array of globs' });
else (list as string[]).forEach((g, i) => { const why = globError(g); if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: `serial_patterns[${i}]`, message: `${JSON.stringify(g)} ${why}` }); });
}
if ('dependencies' in obj) {
const deps = obj.dependencies;
if (!deps || typeof deps !== 'object' || Array.isArray(deps)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'dependencies', message: 'must be an object of glob → test file list' });
else {
const entries = Object.entries(deps as Record<string, unknown>);
if (entries.length > MAX_LIST) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'dependencies', message: `more than ${MAX_LIST} entries` });
for (const [glob, tests] of entries) {
const why = globError(glob);
if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: `dependencies[${JSON.stringify(glob)}]`, message: `${JSON.stringify(glob)} ${why}` });
if (!Array.isArray(tests) || tests.some(t => typeof t !== 'string')) { errors.push({ code: 'PREGATE_CONFIG_INVALID', path: `dependencies[${JSON.stringify(glob)}]`, message: 'must be an array of test file paths ([] declares no lane needed)' }); continue; }
(tests as string[]).forEach((t, i) => { const w = containedRelative(t); if (w) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: `dependencies[${JSON.stringify(glob)}][${i}]`, message: `${JSON.stringify(t)} ${w}` }); });
}
}
}
if ('selection' in obj) {
const sel = obj.selection;
if (!sel || typeof sel !== 'object' || Array.isArray(sel)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'selection', message: 'must be an object of <workflow.yml>/<job> → command' });
else for (const [lane, command] of Object.entries(sel as Record<string, unknown>)) {
if (!/^[A-Za-z0-9_.-]+\.ya?ml\/[A-Za-z0-9_-]+$/.test(lane)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: `selection[${JSON.stringify(lane)}]`, message: 'key must be <workflow file>/<job id>' });
const why = commandError(command);
if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: `selection[${JSON.stringify(lane)}]`, message: why });
}
}
if ('warn' in obj) {
if (!Array.isArray(obj.warn) || obj.warn.some(w => typeof w !== 'string')) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'warn', message: 'must be an array of check ids' });
else for (const w of obj.warn as string[]) {
if ((NON_DOWNGRADABLE as readonly string[]).includes(w)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'warn', message: `${w} cannot be downgraded to warn` });
else if (knownChecks.length && !knownChecks.includes(w)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'warn', message: `unknown check ${w}; known: ${knownChecks.join(', ')}` });
}
}
if ('strays' in obj) {
const s = obj.strays as Record<string, unknown> | null;
if (!s || typeof s !== 'object' || Array.isArray(s)) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'strays', message: 'must be { allow: [globs] }' });
else {
for (const k of Object.keys(s)) if (k !== 'allow') errors.push({ code: 'PREGATE_CONFIG_INVALID', path: `strays.${k}`, message: 'unknown key' });
if ('allow' in s) {
if (!Array.isArray(s.allow) || s.allow.some(a => typeof a !== 'string')) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: 'strays.allow', message: 'must be an array of globs' });
else (s.allow as string[]).forEach((g, i) => { const why = globError(g); if (why) errors.push({ code: 'PREGATE_CONFIG_INVALID', path: `strays.allow[${i}]`, message: `${JSON.stringify(g)} ${why}` }); });
}
}
}
return errors;
}
export function parseConfig(text: string, knownChecks: readonly string[] = []): { config: PregateConfig; errors: ConfigError[] } {
let raw: unknown;
try { raw = JSON.parse(text); }
catch (e: any) { return { config: DEFAULT_CONFIG, errors: [{ code: 'PREGATE_CONFIG_INVALID', path: PREGATE_CONFIG_PATH, message: `not valid JSON: ${e.message}` }] }; }
const errors = validateConfig(raw, knownChecks);
if (errors.length) return { config: DEFAULT_CONFIG, errors };
const o = raw as Partial<PregateConfig>;
return { config: { $schema: o.$schema, runner: o.runner ?? null, dependencies: o.dependencies ?? {}, selection: o.selection ?? {}, warn: o.warn ?? [], strays: { allow: o.strays?.allow ?? [] }, guards: o.guards ?? [], serial_patterns: o.serial_patterns ?? DEFAULT_SERIAL_PATTERNS, price_file: o.price_file ?? null, stall_timeout_s: o.stall_timeout_s ?? null, remote_gate: o.remote_gate ?? null }, errors: [] };
}
export type ConfigLoad = { config: PregateConfig; errors: ConfigError[]; present: boolean; label: string; kind: 'base' | 'head' | 'none' } | { unavailable: true; base: string; detail: string };
function git(cwd: string, args: string[]): { ok: boolean; out: string } {
const r = spawnSync('git', args, { cwd, encoding: 'utf8', timeout: 30_000 });
return { ok: r.status === 0, out: (r.stdout ?? '').trim() };
}
export function loadConfigFromBase(repoRoot: string, base: string, knownChecks: readonly string[] = []): ConfigLoad {
const ref = `origin/${base.replace(/^origin\//, '')}`;
const sha = git(repoRoot, ['rev-parse', '--verify', '--quiet', `${ref}^{commit}`]);
if (!sha.ok || !sha.out) return { unavailable: true, base: base.replace(/^origin\//, ''), detail: `${ref} is not a resolvable commit` };
const show = git(repoRoot, ['show', `${sha.out}:${PREGATE_CONFIG_PATH}`]);
if (!show.ok) return { config: DEFAULT_CONFIG, errors: [], present: false, label: `${ref}@${sha.out.slice(0, 12)} (no ${PREGATE_CONFIG_PATH})`, kind: 'none' };
return { ...parseConfig(show.out, knownChecks), present: true, label: `${ref}@${sha.out.slice(0, 12)}:${PREGATE_CONFIG_PATH}`, kind: 'base' };
}
export function loadConfigFromHead(repoRoot: string, knownChecks: readonly string[] = []): ConfigLoad {
const file = path.join(repoRoot, PREGATE_CONFIG_PATH);
if (!fs.existsSync(file)) return { config: DEFAULT_CONFIG, errors: [], present: false, label: `head (no ${PREGATE_CONFIG_PATH})`, kind: 'none' };
return { ...parseConfig(fs.readFileSync(file, 'utf8'), knownChecks), present: true, label: `head (unreviewed) ${PREGATE_CONFIG_PATH}`, kind: 'head' };
}
export function isConfigUnavailable(load: ConfigLoad): load is { unavailable: true; base: string; detail: string } {
return (load as any).unavailable === true;
}
export function exampleConfigJson(): string {
const example = {
$schema: PREGATE_CONFIG_SCHEMA,
runner: null,
dependencies: { 'bin/example-tool': ['test/example-tool.test.ts'], 'docs/**': [] },
selection: {},
warn: [],
strays: { allow: [] },
};
return JSON.stringify(example, null, 2) + '\n';
}
export function explainConfig(): string {
return [
'runner: the test command with a {files} placeholder; null detects bun test / vitest / jest / pytest from the repo',
'dependencies: glob → test files the import graph cannot see (subprocess targets, runtime reads, generated inputs); [] declares no lane needed',
'selection: <workflow.yml>/<job> → the runner-owned command that lists that lane’s test files when its matrix is planner-computed',
`warn: check ids downgraded to warn; never ${NON_DOWNGRADABLE.join(', ')}; lanes keeps its required-lane and requires-remote verdicts`,
'strays.allow: root files or new top-level directories that are intentional',
'guards: [{ id, when: [globs], command, fix?, timeout_s? }]: each guard whose globs match a touched file runs in its own process (--allow-repo-commands); a failure fails pregate with the guard id and fix; never downgradable',
`serial_patterns: test-file globs the lanes check runs one process per file, apart from the unit files (default ${DEFAULT_SERIAL_PATTERNS.join(', ')})`,
'sealed: { paths, transcripts, answer_markers, seed_patterns, local_paths, visibility }: sealed or eval output the pre-push hook and `gstack-pregate sealed` refuse to publish to a public destination (docs/pregate.md#sealed)',
'price_file: repo-relative path of the GSTACK_EVAL_CALLS v1 call-plan file `gstack-eval-plan price-gate` prices a paid gate from; never executed (docs/eval-calls.md)',
'stall_timeout_s: seconds without test output before the lanes check kills the runner\'s process group and records it incomplete (default 300; 0 disables; --stall-timeout overrides)',
'remote_gate: the repo\'s remote test gate command; under 8 cores the test tier becomes a requires-remote obligation naming it, and gstack-pregate --remote runs it (it must print PREGATE_REMOTE: tree=<tree> tests=<n> failed=<n>) (docs/pregate.md#remote-gate)',
].join('\n') + '\n';
}