feat(eval-plan): the approval step prices every paid gate with price-gate

Step 4 runs gstack-eval-plan price-gate for each paid gate the plan names and
puts its GATE_PRICE line in the ask; the dry-run checklist documents
reservation-fits-cap under --cap.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-13 00:51:37 +00:00
1 parent 958c7b34c1
commit b26b000ee9
2 files changed
+28 -2

No files matched your search

+14 -1
View File
@@ -428,6 +428,8 @@ the budget the run will use. The checklist printed must be all `ok`:
spends output tokens thinking, and a smaller budget truncates the answer the scorer reads.
- `CHECK: retry-prompt` — the exact text the run sends on a retry, printed verbatim; a retry
with no text is a repeat of the same failure at the same price.
- `CHECK: reservation-fits-cap` (with `--cap <cap-usd>`) — one call, the largest row plus the
output budget, priced, fits under the cap; a provider refuses a reservation the cap cannot hold.
- `DRY_RUN_ROW:` lines name rows missing a field; fix the rows, not the prompt.
`DRY_RUN_RESULT: pass` writes `dry-run.json` to `--out`; the pilot refuses to start on
@@ -459,8 +461,19 @@ the preregistration, when `status=budget_exhausted`, or when `verdict=exceeds`.
## Step 4: The approval ask
Price every paid gate the plan names (a benchmark or eval command that spends) from the
repo's call-plan file; nothing is executed (`docs/eval-calls.md` in gstack):
```bash
~/.claude/skills/gstack/bin/gstack-eval-plan price-gate --gate "<exact gate command>" --cap-usd <cap-usd>
```
Each `GATE_PRICE:` line goes into the ask. `EVAL_GATE_UNPRICED`, `EVAL_PLAN_STALE`,
`EVAL_PRICE_MISSING` or `EVAL_GATE_OVER_CAP` means that gate is not approvable as priced;
`--allow-unpriced "<reason>"` only on the owner's word.
Present, in this order and nothing before it: the `PILOT_PROJECTION:` line, the spend
table, the pilot's per-bar results, and the preregistration's stop rules. Then ask one
table, each gate's `GATE_PRICE:` line, the pilot's per-bar results, and the preregistration's stop rules. Then ask one
question with the projection as the recommended option only when `verdict=within`:
- A) Run `<total-items>` items at the projected cost (recommended when within the cap)
+14 -1
View File
@@ -100,6 +100,8 @@ the budget the run will use. The checklist printed must be all `ok`:
spends output tokens thinking, and a smaller budget truncates the answer the scorer reads.
- `CHECK: retry-prompt` — the exact text the run sends on a retry, printed verbatim; a retry
with no text is a repeat of the same failure at the same price.
- `CHECK: reservation-fits-cap` (with `--cap <cap-usd>`) — one call, the largest row plus the
output budget, priced, fits under the cap; a provider refuses a reservation the cap cannot hold.
- `DRY_RUN_ROW:` lines name rows missing a field; fix the rows, not the prompt.
`DRY_RUN_RESULT: pass` writes `dry-run.json` to `--out`; the pilot refuses to start on
@@ -131,8 +133,19 @@ the preregistration, when `status=budget_exhausted`, or when `verdict=exceeds`.
## Step 4: The approval ask
Price every paid gate the plan names (a benchmark or eval command that spends) from the
repo's call-plan file; nothing is executed (`docs/eval-calls.md` in gstack):
```bash
~/.claude/skills/gstack/bin/gstack-eval-plan price-gate --gate "<exact gate command>" --cap-usd <cap-usd>
```
Each `GATE_PRICE:` line goes into the ask. `EVAL_GATE_UNPRICED`, `EVAL_PLAN_STALE`,
`EVAL_PRICE_MISSING` or `EVAL_GATE_OVER_CAP` means that gate is not approvable as priced;
`--allow-unpriced "<reason>"` only on the owner's word.
Present, in this order and nothing before it: the `PILOT_PROJECTION:` line, the spend
table, the pilot's per-bar results, and the preregistration's stop rules. Then ask one
table, each gate's `GATE_PRICE:` line, the pilot's per-bar results, and the preregistration's stop rules. Then ask one
question with the projection as the recommended option only when `verdict=within`:
- A) Run `<total-items>` items at the projected cost (recommended when within the cap)