fix: price-gate --gate-file keeps free text out of the /eval-plan fence; shortest dependency traces; bounded test spawns

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-13 01:57:57 +00:00
1 parent 30d3a68e50
commit c0b56d7932
11 files changed
+81 -22

No files matched your search

+2 -1
View File
@@ -7,7 +7,7 @@
### What this means for you
- **Repo guards run in `gstack-pregate`.** `.gstack/pregate.json` takes `guards`: `{id, when: [globs], command, fix}`. When a changed file matches `when`, the guard runs in its own process with a timeout, and a failure names the guard and its fix. Guards cover the checks an import graph can't see, such as a serial-test manifest row or CLI goldens that spawn the binary. Required guards can't be downgraded, and without `--allow-repo-commands` the run is `incomplete`, never a pass. The lanes check runs on the verified Bun from the install's launch directory and runs serial test files one process each. Unknown flags exit 2 instead of being ignored.
- **`gstack-eval-plan price-gate` prices a paid gate without running it.** It reads a versioned call-plan file (`--plan-file`, or `price_file` in the base branch's policy) that is bound to the gate command, head and inputs, and prints `GATE_PRICE:` with every precondition the file declares. Nothing runs before approval. A gate that declares no plan is `EVAL_GATE_UNPRICED`; `--allow-unpriced "<reason>"` approves it as a recorded owner override, never as permission to spend. A stale or malformed plan file refuses. The new `reservation-fits-cap` check catches a single call whose reservation can't fit under the smoke cap. `/eval-plan` prices every paid gate this way at approval. The wire format and a reference emitter are in `docs/eval-calls.md`.
- **`gstack-eval-plan price-gate` prices a paid gate without running it.** It reads a versioned call-plan file (`--plan-file`, or `price_file` in the base branch's policy) that is bound to the gate command, head and inputs, and prints `GATE_PRICE:` with every precondition the file declares. Nothing runs before approval. A gate that declares no plan is `EVAL_GATE_UNPRICED`; `--allow-unpriced "<reason>"` approves it as a recorded owner override, never as permission to spend. A stale or malformed plan file refuses. The new `reservation-fits-cap` check catches a single call whose reservation can't fit under the smoke cap. `/eval-plan` prices every paid gate this way at approval, passing the gate command as a file (`--gate-file`) so a command with quotes or `$` never goes through a shell string. The wire format and a reference emitter are in `docs/eval-calls.md`.
- **Current prices.** Claude Sonnet 5.5, Gemini 3.x Flash and the gbrain embedding models are priced from the providers' pages (source and date in the table), and Claude Opus 4.7 is at its current list price ($5/$25 per million tokens).
- **`gstack-pregate regate --against <gated-head>` says whether a restamped head needs a new gate.** It computes a fresh tree receipt and prints `STAMP_ONLY:`, one `LANE:` line per gate lane (reusable, rerun or not-reusable), `MIGRATION_COLLISION:` against the base and every other open PR, and `REGATE: needed|not-needed`. `not-needed` requires gate evidence for the gated head, an unchanged execution identity, no stamp-sensitive rerun and no unknown collision lookup. Bare `--against` is an alias.
- **The repo's stamp rule is the stamp rule.** Ship policy takes `stamp_files` (the files allowed to differ) and literal `stamp_patterns` with one `{version}` slot, such as `since: '{version}'`. A pure restamp of a repo whose versions sit in quoted `since:` rows, package manifests or stamp comments now classifies as stamps. `restamp_command` runs the repo's own restamp inside restamp's staging worktree with the target version, on a clean tree; a failure or a wrong-version output writes nothing. The `migrations` key (directory and id pattern) feeds the collision check.
@@ -16,6 +16,7 @@
### For contributors
- `scripts/pr-dependencies.ts` traces breadth-first, so a trace names the shortest chain from a case's touchfiles to a file (selection is unchanged).
- New: `lib/pregate/checks/guards.ts`, `lib/pregate/regate.ts`, `lib/pregate/runtime.ts`, `lib/sealed-guard.ts`; docs `docs/eval-calls.md`, `docs/pregate.md#regate` and `#gbrain-worked-configs`.
- New result codes with troubleshooting anchors include `EVAL_GATE_UNPRICED`, `PREGATE_GUARD_FAILED`, `SEALED_OUTPUT_REFUSED` and `MIGRATION_COLLISION`.
- Pregate and ship-policy validators still refuse unknown keys, so a repo should commit `guards`, `sealed`, `price_file`, `stamp_files`, `stamp_patterns`, `restamp_command` or `migrations` only after its machines run this release.
+8 -5
View File
@@ -10,7 +10,7 @@
* gstack-eval-plan pilot settle --out <dir> --attempt <id> --usd <n|unknown>
* gstack-eval-plan pilot release --out <dir> --attempt <id>
* gstack-eval-plan pilot report --out <dir> --total <items> [--workers W] [--json]
* gstack-eval-plan price-gate --gate "<exact gate command>" [--plan-file <file>] [--base <branch>] [--cap-usd N]
* gstack-eval-plan price-gate --gate-file <file> | --gate "<exact gate command>" [--plan-file <file>] [--base <branch>] [--cap-usd N]
* [--smoke-cap-usd N] [--allow-unpriced "<reason>"] [--out <dir>] [--json]
*
* The dry run spends nothing (stub model) and prints the checklist: price-table
@@ -47,7 +47,7 @@ const HELP = `gstack-eval-plan — preregistration, priced arms, $0 dry run, pri
pilot start --out dir --cap usd --arms m,m --dry-run dry-run.json [--n 3]
pilot settle --out dir --attempt id --usd <n|unknown> pilot release --out dir --attempt id
pilot report --out dir --total items [--workers W] [--json] spend table + extrapolation with a 95% band, then the approval ask
price-gate --gate "<exact gate command>" [--plan-file f] [--base b] [--cap-usd N] [--smoke-cap-usd N] [--allow-unpriced "<reason>"] [--out dir] [--json]
price-gate --gate-file <file> | --gate "<exact gate command>" [--plan-file f] [--base b] [--cap-usd N] [--smoke-cap-usd N] [--allow-unpriced "<reason>"] [--out dir] [--json]
price a paid gate from its GSTACK_EVAL_CALLS v1 call-plan file (--plan-file, else the base
policy's price_file at HEAD); checks the gate/head/input binding, executes nothing, writes a
receipt (default .gstack/tmp/price-gate/); docs/eval-calls.md
@@ -150,7 +150,7 @@ function doPilot(rest: string[]): number {
return usage(`unknown pilot command ${sub ?? ''}`);
}
const PRICE_GATE_FLAGS = new Set(['--gate', '--plan-file', '--base', '--cap-usd', '--smoke-cap-usd', '--allow-unpriced', '--out']);
const PRICE_GATE_FLAGS = new Set(['--gate', '--gate-file', '--plan-file', '--base', '--cap-usd', '--smoke-cap-usd', '--allow-unpriced', '--out']);
function doPriceGate(rest: string[]): number {
for (let i = 0; i < rest.length; i++) {
@@ -158,8 +158,11 @@ function doPriceGate(rest: string[]): number {
if (!PRICE_GATE_FLAGS.has(rest[i]!) || rest[i + 1] === undefined) return usage(`price-gate: unknown flag or missing value ${rest[i]}`);
i++;
}
const gate = flag(rest, '--gate');
if (!gate) return usage('price-gate needs --gate "<exact gate command>"');
const gateFile = flag(rest, '--gate-file');
if (gateFile && flag(rest, '--gate') !== undefined) return usage('price-gate takes --gate or --gate-file, not both');
if (gateFile && !fs.existsSync(gateFile)) return usage(`--gate-file ${gateFile} does not exist`);
const gate = gateFile ? fs.readFileSync(gateFile, 'utf8').replace(/\r?\n$/, '') : flag(rest, '--gate');
if (!gate?.trim()) return usage('price-gate needs the exact gate command: --gate-file <file> (written with your file-write tool) or --gate "<command>"');
const top = spawnSync('git', ['rev-parse', '--show-toplevel'], { encoding: 'utf8' });
if (top.status !== 0) return usage('price-gate runs inside the git repository whose gate it prices');
const repoRoot = top.stdout.trim();
+5 -1
View File
@@ -11,9 +11,13 @@ that it still describes what is being approved, prices it from
(`docs/designs/GSTACK_FIELD_WAVE_2026_10_12.md`, gate decision d5b as amended).
```bash
gstack-eval-plan price-gate --gate "<exact gate command>" --cap-usd 20 [--smoke-cap-usd 2] [--plan-file <file>]
gstack-eval-plan price-gate --gate-file <file> --cap-usd 20 [--smoke-cap-usd 2] [--plan-file <file>]
```
`--gate-file` reads the exact gate command from a file (one trailing newline is
dropped), so a command with quotes or `$` never passes through a shell string;
`--gate "<command>"` takes it inline. Give one of the two.
## Where the file comes from
1. `--plan-file <file>`: an operator-given file, read from disk.
+15 -2
View File
@@ -462,10 +462,23 @@ the preregistration, when `status=budget_exhausted`, or when `verdict=exceeds`.
## Step 4: The approval ask
Price every paid gate the plan names (a benchmark or eval command that spends) from the
repo's call-plan file; nothing is executed (`docs/eval-calls.md` in gstack):
repo's call-plan file; nothing is executed (`docs/eval-calls.md` in gstack). The gate
command is free text, so it reaches the pricer only as a file:
```bash
~/.claude/skills/gstack/bin/gstack-eval-plan price-gate --gate "<exact gate command>" --cap-usd <cap-usd>
_GT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.gstack/tmp"
mkdir -p "$_GT" && chmod 700 "$_GT" || { echo "Not sent: cannot create $_GT for the text file." >&2; exit 1; }
_EX=$(git rev-parse --git-path info/exclude 2>/dev/null) && mkdir -p "$(dirname "$_EX")" && { grep -qxF '/.gstack/tmp/' "$_EX" 2>/dev/null || echo '/.gstack/tmp/' >> "$_EX"; }
GATE_FILE=$(mktemp "${_GT:?}/gate-command.XXXXXX") || { echo "Not sent: mktemp failed in $_GT." >&2; exit 1; }; echo "GATE_FILE: $GATE_FILE (name: ${GATE_FILE##*/})"
```
Write the text into each printed file with your file-write tool (Claude Code's Write tool needs a Read of the empty file first), exactly as it should appear. The text never goes into a shell command, heredoc or quoted argument. If a write fails or is refused, do not send: print the cause, the file path and the command below for sending by hand.
Write the exact gate command, one line, into the printed file, then price it:
```bash
GATE_FILE="$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.gstack/tmp/<gate-file-name>"
~/.claude/skills/gstack/bin/gstack-eval-plan price-gate --gate-file "$GATE_FILE" --cap-usd <cap-usd>
```
Each `GATE_PRICE:` line goes into the ask. `EVAL_GATE_UNPRICED`, `EVAL_PLAN_STALE`,
+8 -2
View File
@@ -134,10 +134,16 @@ the preregistration, when `status=budget_exhausted`, or when `verdict=exceeds`.
## Step 4: The approval ask
Price every paid gate the plan names (a benchmark or eval command that spends) from the
repo's call-plan file; nothing is executed (`docs/eval-calls.md` in gstack):
repo's call-plan file; nothing is executed (`docs/eval-calls.md` in gstack). The gate
command is free text, so it reaches the pricer only as a file:
{{FREE_TEXT_FILE:GATE_FILE=gate-command}}
Write the exact gate command, one line, into the printed file, then price it:
```bash
~/.claude/skills/gstack/bin/gstack-eval-plan price-gate --gate "<exact gate command>" --cap-usd <cap-usd>
GATE_FILE="$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.gstack/tmp/<gate-file-name>"
~/.claude/skills/gstack/bin/gstack-eval-plan price-gate --gate-file "$GATE_FILE" --cap-usd <cap-usd>
```
Each `GATE_PRICE:` line goes into the ask. `EVAL_GATE_UNPRICED`, `EVAL_PLAN_STALE`,
+3 -3
View File
@@ -201,9 +201,9 @@ export function deriveDependencies(maps: DependencyMaps, root: string): DerivedD
};
const closure = (entries: Iterable<string>, parents?: Map<string, string>): Set<string> => {
const seen = new Set<string>();
const queue = [...entries].filter(file => trackedSet.has(file));
while (queue.length) {
const file = queue.pop()!;
const queue = [...entries].filter(file => trackedSet.has(file)).reverse();
for (let head = 0; head < queue.length; head++) {
const file = queue[head]!;
if (seen.has(file)) continue;
seen.add(file);
for (const next of refs(file)) if (!seen.has(next) && !/\.test\.tsx?$/.test(next)) { queue.push(next); if (parents && !parents.has(next)) parents.set(next, file); }
+1
View File
@@ -102,6 +102,7 @@ export const IDENTIFIER_PLACEHOLDERS: Record<string, IdentifierGrammar> = {
// Paths printed by earlier gstack blocks or chosen by the skill.
'<body-file-name>': { grammar: ID, what: 'basename of the mktemp body file printed by the previous block' },
'<title-file-name>': { grammar: ID, what: 'basename of the mktemp title file printed by the previous block' },
'<gate-file-name>': { grammar: ID, what: 'basename of the mktemp gate-command file printed by the previous block (/eval-plan price-gate)' },
'<reply-file-name>': { grammar: ID, what: 'basename of the mktemp reply file printed by the previous block' },
'<redact-file-name>': { grammar: ID, what: 'basename of the mktemp draft file printed by the free-text block' },
'<approach-file-name>': { grammar: ID, what: 'basename of the mktemp approach file printed by the free-text block' },
+11
View File
@@ -77,6 +77,17 @@ describe('reference derivation on a fixture tree', () => {
expect(referenceText('data.json', '{"a":"lib/x.ts"}')).toBe('');
});
test('trace names the shortest chain: a template that invokes a bin directly wins over a long import path to it', () => {
const repo = fixtureRepo({
'test/skill-e2e-gamma.test.ts': '', 'gamma/SKILL.md.tmpl': 'Run bin/gstack-target.\n', 'gamma/sections/a.md.tmpl': 'See lib/one.ts.\n',
'lib/one.ts': "import './two';\n", 'lib/two.ts': "import './three';\n", 'lib/three.ts': "export const x = 'bin/gstack-target';\n",
'bin/gstack-target': '#!/bin/sh\n', 'package.json': '{}',
});
const traced = deriveDependencies({ e2eTouchfiles: { gamma: ['test/skill-e2e-gamma.test.ts', 'gamma/**'] }, judgeTouchfiles: {}, globalTouchfiles: [] }, repo);
expect(traced.trace('gamma', 'bin/gstack-target')).toEqual(['gamma/SKILL.md.tmpl', 'bin/gstack-target']);
expect(traced.trace('gamma', 'lib/three.ts')).toEqual(['gamma/sections/a.md.tmpl', 'lib/one.ts', 'lib/two.ts', 'lib/three.ts']);
});
test('a host config feeds only cases whose test names the host', () => {
const named = deriveDependencies({ ...maps, e2eTouchfiles: { ...maps.e2eTouchfiles, beta: ['test/skill-e2e-beta.test.ts'] } },
fixtureRepo({ 'test/skill-e2e-beta.test.ts': "run({ host: 'acme' });\n", 'hosts/acme.ts': '', 'hosts/zeta.ts': '', 'scripts/build.sh': '', 'scripts/resolvers/index.ts': '', 'package.json': '{}' }));
+25 -5
View File
@@ -172,7 +172,7 @@ function scratchRepo(env: Record<string, string>): { repo: string; head: string;
const repo = path.join(env.TMPDIR!, 'repo');
fs.mkdirSync(repo, { recursive: true });
const git = (...args: string[]) => {
const r = spawnSync('git', args, { cwd: repo, env, encoding: 'utf8' });
const r = spawnSync('git', args, { cwd: repo, env, encoding: 'utf8', timeout: 30_000 });
if (r.status !== 0) throw new Error(`git ${args.join(' ')}: ${r.stderr}`);
return r.stdout.trim();
};
@@ -190,7 +190,7 @@ const twoRows = [
];
function runGate(host: CapyHost, cwd: string, args: string[], extraEnv: Record<string, string> = {}) {
const r = spawnSync(BIN, ['price-gate', ...args], { cwd, env: { ...host.env, ...extraEnv }, encoding: 'utf8' });
const r = spawnSync(BIN, ['price-gate', ...args], { cwd, env: { ...host.env, ...extraEnv }, encoding: 'utf8', timeout: 30_000 });
return { code: r.status, out: r.stdout ?? '', err: r.stderr ?? '' };
}
@@ -254,9 +254,29 @@ describe('price-gate: binding, pricing and the receipt (bin, simulated Capy host
expect(typeof receipt.gstack_revision).toBe('string');
});
test('--gate-file prices the gate command the agent wrote to a file, byte for byte as --gate does; empty, missing or both refuse', () => {
const { repo, head, rowsSha } = scratchRepo(host.env);
const quoted = `${GATE} --label "it's \\"quoted\\" \\$HOME"`;
const file = writePlan(repo, { gate: quoted, head, inputs: [{ path: 'rows.jsonl', sha256: rowsSha }], preconditions: [], rows: twoRows });
const gateFile = path.join(host.base, 'gate-command.txt');
fs.writeFileSync(gateFile, quoted + '\n');
const viaFile = runGate(host, repo, ['--plan-file', file, '--gate-file', gateFile, '--out', path.join(host.base, 'a')]);
expect(viaFile.err).toBe('');
expect(viaFile.code).toBe(0);
const viaFlag = runGate(host, repo, ['--plan-file', file, '--gate', quoted, '--out', path.join(host.base, 'b')]);
const total = (out: string) => out.split('\n').find(l => l.startsWith('GATE_PRICE: '))!.replace(/ receipt=.*/, '');
expect(total(viaFile.out)).toBe(total(viaFlag.out));
const receipt = JSON.parse(fs.readFileSync(viaFile.out.split('receipt=')[1]!.split('\n')[0]!, 'utf8'));
expect(receipt.gate_command).toBe(quoted);
fs.writeFileSync(gateFile, '\n');
expect(runGate(host, repo, ['--plan-file', file, '--gate-file', gateFile]).code).toBe(2);
expect(runGate(host, repo, ['--plan-file', file, '--gate-file', path.join(host.base, 'absent')]).code).toBe(2);
expect(runGate(host, repo, ['--plan-file', file, '--gate', quoted, '--gate-file', gateFile]).code).toBe(2);
});
test('the base policy price_file is read from the HEAD commit; a plan committed on top of its own head stays fresh until another file changes', () => {
const { repo, rowsSha } = scratchRepo(host.env);
const git = (...args: string[]) => spawnSync('git', args, { cwd: repo, env: host.env, encoding: 'utf8' }).stdout.trim();
const git = (...args: string[]) => spawnSync('git', args, { cwd: repo, env: host.env, encoding: 'utf8', timeout: 30_000 }).stdout.trim();
fs.mkdirSync(path.join(repo, '.gstack'), { recursive: true });
fs.writeFileSync(path.join(repo, '.gstack', 'pregate.json'), JSON.stringify({ price_file: 'evals/gate.calls' }));
git('add', '.gstack/pregate.json'); git('commit', '-q', '-m', 'policy');
@@ -282,7 +302,7 @@ describe('price-gate: binding, pricing and the receipt (bin, simulated Capy host
test('a declared price_file missing at HEAD refuses EVAL_PLAN_STALE (not overridable); an unresolvable base or an invalid pregate.json exits 3', () => {
const { repo } = scratchRepo(host.env);
const git = (...args: string[]) => spawnSync('git', args, { cwd: repo, env: host.env, encoding: 'utf8' }).stdout.trim();
const git = (...args: string[]) => spawnSync('git', args, { cwd: repo, env: host.env, encoding: 'utf8', timeout: 30_000 }).stdout.trim();
const out = path.join(host.base, 'receipts');
let r = runGate(host, repo, ['--gate', GATE, '--base', 'main', '--out', out, '--allow-unpriced', 'owner said so']);
expect(r.code).toBe(3);
@@ -354,7 +374,7 @@ describe('price-gate: binding, pricing and the receipt (bin, simulated Capy host
test('no plan file, or a file with no call lines, is EVAL_GATE_UNPRICED exit 1; --allow-unpriced exits 0 with a recorded override', () => {
const { repo, head, rowsSha } = scratchRepo(host.env);
spawnSync('git', ['update-ref', 'refs/remotes/origin/main', head], { cwd: repo, env: host.env });
spawnSync('git', ['update-ref', 'refs/remotes/origin/main', head], { cwd: repo, env: host.env, timeout: 30_000 });
const out = path.join(host.base, 'receipts');
let r = runGate(host, repo, ['--gate', GATE, '--base', 'main', '--out', out]);
expect(r.code).toBe(1);
+2 -2
View File
@@ -138,7 +138,7 @@ describe('release-tool contract', () => {
describe('restamp_command (field wave CD3): the repo owns the stamp inside the staging worktree', () => {
const RESTAMP_JS = [
"const fs = require('fs'); const { execSync } = require('child_process');",
"if (execSync('git status --porcelain', { encoding: 'utf8' }).trim()) { console.error('FAIL: the working tree is not clean'); process.exit(1); }",
"if (execSync('git status --porcelain', { encoding: 'utf8', timeout: 30000 }).trim()) { console.error('FAIL: the working tree is not clean'); process.exit(1); }",
"const v = process.argv[2];",
"if (!v || v !== process.env.GSTACK_RESTAMP_VERSION) { console.error('version argument and GSTACK_RESTAMP_VERSION disagree'); process.exit(2); }",
"if (process.env.FIXTURE_FAIL) process.exit(7);",
@@ -146,7 +146,7 @@ describe('restamp_command (field wave CD3): the repo owns the stamp inside the s
"const pkg = JSON.parse(fs.readFileSync('package.json', 'utf8')); pkg.version = v.split('.').slice(0, 3).join('.'); fs.writeFileSync('package.json', JSON.stringify(pkg, null, 2) + '\\n');",
"fs.writeFileSync('generated/digest.md', '# digest v' + v + '\\nbody\\n');",
"if (process.env.FIXTURE_STRAY) fs.writeFileSync('stray.txt', 'x\\n');",
"execSync('git add -A'); execSync('git -c user.name=r -c user.email=r@example.com -c commit.gpgsign=false commit -q -m restamp');",
"execSync('git add -A', { timeout: 30000 }); execSync('git -c user.name=r -c user.email=r@example.com -c commit.gpgsign=false commit -q -m restamp', { timeout: 30000 });",
].join('\n') + '\n';
const POLICY = { ...BASE_POLICY, release_tool: null, restamp_command: 'node restamp.js' };
+1 -1
View File
@@ -47,7 +47,7 @@ function scratch(baseFiles: Record<string, string | Buffer>): Scratch {
fs.mkdirSync(work); fs.mkdirSync(state);
const env = { HOME: root, GSTACK_HOME: state, GSTACK_STATE_ROOT: state, GIT_CONFIG_GLOBAL: path.join(root, 'gitconfig'), GIT_CONFIG_NOSYSTEM: '1' };
fs.writeFileSync(env.GIT_CONFIG_GLOBAL, '[user]\n\tname = t\n\temail = t@example.com\n[commit]\n\tgpgsign = false\n[init]\n\tdefaultBranch = main\n');
spawnSync('git', ['init', '-q', '--bare', origin], { env: { ...process.env, ...env } });
spawnSync('git', ['init', '-q', '--bare', origin], { env: { ...process.env, ...env }, timeout: 30_000 });
sh(work, env, ['init', '-q', '-b', 'main']);
write(work, { 'README.md': '# evals\n', ...baseFiles });
sh(work, env, ['add', '-A']); sh(work, env, ['commit', '-q', '-m', 'base']);