From bbf48aa7ef195459afcf7d7840af7073cd050734 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Wed, 18 Nov 2026 19:29:02 +0800 Subject: [PATCH] Unattended /autoplan records the owner's relayed gate reply instead of refusing it (GST-3068) (#4484) An unattended /autoplan run that's handed the owner's reply at the final gate now records it and finishes, the same way whichever instruction file the agent reads. Before, autoplan/SKILL.md said "never approve" while autoplan/DRIVER.md said to record the relayed reply, so the outcome depended on which file the model followed. Nothing to do; this fixes existing instructions and adds no new feature. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --- autoplan/DRIVER.md | 4 +- autoplan/DRIVER.md.tmpl | 4 +- autoplan/SKILL.md | 18 ++---- autoplan/SKILL.md.tmpl | 18 ++---- autoplan/sections/unattended.md | 10 +-- autoplan/sections/unattended.md.tmpl | 10 +-- bin/gstack-skill-start | 2 +- ...ed-autoplan-records-relayed-owner-reply.md | 1 + docs/unattended.md | 7 ++- scripts/resolvers/autoplan-driver.ts | 5 ++ scripts/resolvers/index.ts | 3 +- test/autoplan-unattended-gate-rule.test.ts | 61 +++++++++++++++++++ test/fixtures/skill-md-size-allowlist.json | 24 ++++---- test/gstack-skill-start.test.ts | 3 + test/host-conformance.test.ts | 2 +- 15 files changed, 119 insertions(+), 53 deletions(-) create mode 100644 changelog.d/unattended-autoplan-records-relayed-owner-reply.md create mode 100644 test/autoplan-unattended-gate-rule.test.ts diff --git a/autoplan/DRIVER.md b/autoplan/DRIVER.md index 91d970bc2..2a42ab5ad 100644 --- a/autoplan/DRIVER.md +++ b/autoplan/DRIVER.md @@ -29,8 +29,8 @@ line yourself; gstack never writes the plan. **Kinds (R65), first `next` only.** a subagent on another model family (or `gstack-outside-voice run`). Each writes `result=`. The first `next` also prints `REVIEW_ESTIMATE:` (most attempts and reservations the review can take). **Outside voice, unattended.** A stored `codex_reviews: enabled` is the owner's standing consent to send each phase's review input to the outside model (`next` prints `OUTSIDE_CONSENT: given by codex_reviews=enabled`), so run the outside attempt: the preamble's unattended consent rule does not cover it, and you never close it `--unavailable` over consent or data leaving the machine. The runner itself enforces `disabled` or an unreadable config (`OUTSIDE: not_run reason=consent_missing`, no outside attempt); an unset `codex_reviews` keeps its existing handling. Run the voice in the foreground, inside this turn: `gstack-outside-voice run ... --timeout-ms 540000` with the tool's own timeout above it (Claude Code Bash `timeout: 600000`), or a subagent you await. Never `run_in_background`, never a monitor, never end the turn to wait for it: an unattended session ends with its turn. When the voice fails, `submit --voice outside --unavailable ""` with the matching line and carry on in the same turn, the native voice still counting: **Auth failure:** If stderr contains "auth", "login", "unauthorized", or "API key": "Codex authentication failed. Run `codex login` to authenticate." **Timeout:** "Codex timed out after 9 minutes and was terminated; this pass produced NO findings." A timed-out pass is MISSING COVERAGE, not a clean bill — say so explicitly rather than continuing as if Codex had reviewed. **Empty response:** "Codex returned no response. Stderr: ." 2. Submit each voice exactly once (`BOUND:`). Both voices in closes the phase (`PHASE_CLOSED:`). 3. Repeat `next` until `status=gate_pending` (`GATE: gate_rev=`); render it with `gstack-gate render /gate.json --all` and relay every page with the two cost lines printed under `GATE:`: `GATE_SPEND:` (spent, worst case, cap, headroom, attempts; host-subagent attempts are priced unknown and counted at their reservation, so quote their real figure from the host's cost view) and `GATE_REC_COST:` (the phases `rec` can reopen and the estimate for them, an upper bound; `gate_cost` in `--json`). - You never approve the gate for the owner. Offer `rec` (every recommendation, plan changes included) as the default. -4. `answer` with the owner's reply and `--by human` (the owner typed the tokens) or `--by agent: --owner-reply ""` (you relayed them); without `--by` an + Offer `rec` (every recommendation, plan changes included) as the default. +4. **The gate, unattended.** Never approve it on your own judgment: with no owner reply, the run stops at `status=gate_pending`. Record an owner's reply relayed to you yourself, never handing them the command: `gstack-autoplan answer --out --gate-rev --reply "" --by agent: --owner-reply-file `. `--by human` means the owner typed the tokens and needs no words file; without `--by` an unattended run refuses (`ANSWER_BY_REQUIRED`). `APPLY: -> <change>` lines are accepted plan changes (status `amend_pending`): edit the plan, then `revise` (the printed `NEXT:` names it), or `revise --settle <id>` when the line says no plan edit is needed. Until diff --git a/autoplan/DRIVER.md.tmpl b/autoplan/DRIVER.md.tmpl index c9f84eb4a..b8c3aa855 100644 --- a/autoplan/DRIVER.md.tmpl +++ b/autoplan/DRIVER.md.tmpl @@ -26,8 +26,8 @@ line yourself; gstack never writes the plan. **Kinds (R65), first `next` only.** a subagent on another model family (or `gstack-outside-voice run`). Each writes `result=`. The first `next` also prints `REVIEW_ESTIMATE:` (most attempts and reservations the review can take). {{AUTOPLAN_UNATTENDED_VOICE}} 2. Submit each voice exactly once (`BOUND:`). Both voices in closes the phase (`PHASE_CLOSED:`). 3. Repeat `next` until `status=gate_pending` (`GATE: gate_rev=<n>`); render it with `gstack-gate render <dir>/gate.json --all` and relay every page with the two cost lines printed under `GATE:`: `GATE_SPEND:` (spent, worst case, cap, headroom, attempts; host-subagent attempts are priced unknown and counted at their reservation, so quote their real figure from the host's cost view) and `GATE_REC_COST:` (the phases `rec` can reopen and the estimate for them, an upper bound; `gate_cost` in `--json`). - You never approve the gate for the owner. Offer `rec` (every recommendation, plan changes included) as the default. -4. `answer` with the owner's reply and `--by human` (the owner typed the tokens) or `--by agent:<id> --owner-reply "<their words>"` (you relayed them); without `--by` an + Offer `rec` (every recommendation, plan changes included) as the default. +4. {{AUTOPLAN_UNATTENDED_GATE}} `--by human` means the owner typed the tokens and needs no words file; without `--by` an unattended run refuses (`ANSWER_BY_REQUIRED`). `APPLY: <id> <title> -> <change>` lines are accepted plan changes (status `amend_pending`): edit the plan, then `revise` (the printed `NEXT:` names it), or `revise --settle <id>` when the line says no plan edit is needed. Until diff --git a/autoplan/SKILL.md b/autoplan/SKILL.md index fea3f5ca2..5773e89b6 100644 --- a/autoplan/SKILL.md +++ b/autoplan/SKILL.md @@ -302,10 +302,10 @@ outside reviewer: yes; run evals: yes; reduce scope on a complete plan: no). 3. **Codex disagreements** — the outside reviewer recommends differently and has a valid point. **User Challenge** — Claude and Codex both recommend changing the -user's stated direction: merge, split, add or remove features/skills/workflows. -NEVER auto-decide these. At the final approval gate give the original direction, -proposed change, reasoning, blind spots and cost of being wrong (Phase 4 gate item); -flag agreed security/feasibility risks. The user's direction stands until they approve the change. +user's stated direction (merge, split, add or remove features/skills/workflows), or a +premise is clearly wrong. Never auto-decide User Challenges: each is a Phase 4 `uc<n>` +approval, never mid-run (the user has context models lack); flag agreed security/feasibility +risks. The user's direction stands until they approve the change. --- @@ -365,10 +365,6 @@ full interactive depth; answer each question using the 6 principles. context default. Use the 6 principles for missing recommendations/ties. On principle disagreement, take the recommendation and surface it as Taste at the final gate. -**Never auto-decide User Challenges:** both models agree to change the user's -direction/settled decisions, or a premise is clearly wrong. Ask once at the Final -Approval Gate, never mid-run; the user has context models lack. - Read referenced code/diffs/files; decide every issue. Produce all required diagrams, tables, registries and artifacts on disk or in the plan. LOG decisions, record ALL accepted obligations below and run `amend-input` before continuing. @@ -669,8 +665,8 @@ unparsed or stale replies are re-presented (`gate_rev`+1), never guessed. `rec` the plan as first written. First match: any `pending` id = approve nothing (bare `p1a` leaves `uc<n>` pending; re-parse the owner's whole reply); `p1c`/`p1d`/`p1e` = C/D/E; a chosen `changes_plan` option (or an unflagged override) -= B on a `d<n>`, B2 on a `uc<n>`; else = A. Under `SESSION_KIND: unattended` Read `sections/unattended.md`, end here -with its `GSTACK_RESULT` line and never approve. += B on a `d<n>`, B2 on a `uc<n>`; else = A. Under `SESSION_KIND: unattended` +Read `sections/unattended.md` and settle the gate by its gate rule instead. **Option handling:** - A: mark APPROVED, run `~/.claude/skills/gstack/bin/gstack-owner-brief close --out "$_AP" --plan <ACTIVE_PLAN>`, write review logs, suggest /ship @@ -730,5 +726,3 @@ Present a phase coverage table (CEO, design, DX, eng): host, outside provider/st native completion, findings, partial coverage, with actual counts. **Implementation model:** relay model/source from `"$HOME/.claude/skills/gstack/bin/gstack-models" resolve --role implementation --provider anthropic`. gstack cannot change this session. Recommend only; no spawn or config edits unless asked. On error, relay its repair, not a model. [Policy setup](https://github.com/garrytan/gstack/blob/main/docs/model-policy.md). - -Suggest `/ship` when ready to create the PR. diff --git a/autoplan/SKILL.md.tmpl b/autoplan/SKILL.md.tmpl index a4b0433d8..4b5537faf 100644 --- a/autoplan/SKILL.md.tmpl +++ b/autoplan/SKILL.md.tmpl @@ -86,10 +86,10 @@ outside reviewer: yes; run evals: yes; reduce scope on a complete plan: no). 3. **{{OUTSIDE_LABEL}} disagreements** — the outside reviewer recommends differently and has a valid point. **User Challenge** — {{NATIVE_LABEL}} and {{OUTSIDE_LABEL}} both recommend changing the -user's stated direction: merge, split, add or remove features/skills/workflows. -NEVER auto-decide these. At the final approval gate give the original direction, -proposed change, reasoning, blind spots and cost of being wrong (Phase 4 gate item); -flag agreed security/feasibility risks. The user's direction stands until they approve the change. +user's stated direction (merge, split, add or remove features/skills/workflows), or a +premise is clearly wrong. Never auto-decide User Challenges: each is a Phase 4 `uc<n>` +approval, never mid-run (the user has context models lack); flag agreed security/feasibility +risks. The user's direction stands until they approve the change. --- @@ -149,10 +149,6 @@ full interactive depth; answer each question using the 6 principles. context default. Use the 6 principles for missing recommendations/ties. On principle disagreement, take the recommendation and surface it as Taste at the final gate. -**Never auto-decide User Challenges:** both models agree to change the user's -direction/settled decisions, or a premise is clearly wrong. Ask once at the Final -Approval Gate, never mid-run; the user has context models lack. - Read referenced code/diffs/files; decide every issue. Produce all required diagrams, tables, registries and artifacts on disk or in the plan. LOG decisions, record ALL accepted obligations below and run `amend-input` before continuing. @@ -403,8 +399,8 @@ unparsed or stale replies are re-presented (`gate_rev`+1), never guessed. `rec` the plan as first written. First match: any `pending` id = approve nothing (bare `p1a` leaves `uc<n>` pending; re-parse the owner's whole reply); `p1c`/`p1d`/`p1e` = C/D/E; a chosen `changes_plan` option (or an unflagged override) -= B on a `d<n>`, B2 on a `uc<n>`; else = A. Under `SESSION_KIND: unattended` Read `sections/unattended.md`, end here -with its `GSTACK_RESULT` line and never approve. += B on a `d<n>`, B2 on a `uc<n>`; else = A. Under `SESSION_KIND: unattended` +Read `sections/unattended.md` and settle the gate by its gate rule instead. **Option handling:** - A: mark APPROVED, run `~/.claude/skills/gstack/bin/gstack-owner-brief close --out "$_AP" --plan <ACTIVE_PLAN>`, write review logs, suggest /ship @@ -464,5 +460,3 @@ Present a phase coverage table (CEO, design, DX, eng): host, outside provider/st native completion, findings, partial coverage, with actual counts. {{IMPLEMENTATION_MODEL_HANDOFF}} - -Suggest `/ship` when ready to create the PR. diff --git a/autoplan/sections/unattended.md b/autoplan/sections/unattended.md index a3d181f6b..fffa3d9ac 100644 --- a/autoplan/sections/unattended.md +++ b/autoplan/sections/unattended.md @@ -5,8 +5,10 @@ `SESSION_KIND: unattended`: a parent drives the run through `~/.claude/skills/gstack/bin/gstack-autoplan` (DRIVER.md; docs/unattended.md) and reads artifacts, never prose. Phases close -themselves, Eng runs last, the outside voice (none when lean) runs per DRIVER.md -(enabled `codex_reviews` is consent; foreground only), -and the run stops at `status=gate_pending`: never approve. `gstack-artifact -validate <dir>/run.json` is the completion check. End with +themselves, Eng runs last, and the outside voice (none when lean) runs per DRIVER.md +(enabled `codex_reviews` is consent; foreground only). + +**The gate, unattended.** Never approve it on your own judgment: with no owner reply, the run stops at `status=gate_pending`. Record an owner's reply relayed to you yourself, never handing them the command: `gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by agent:<id> --owner-reply-file <file of their words>`. + +`gstack-artifact validate <dir>/run.json` is the completion check. End with `GSTACK_RESULT: skill=autoplan status=<status> run=<dir>`. diff --git a/autoplan/sections/unattended.md.tmpl b/autoplan/sections/unattended.md.tmpl index e5a2cd6f8..03407417f 100644 --- a/autoplan/sections/unattended.md.tmpl +++ b/autoplan/sections/unattended.md.tmpl @@ -3,8 +3,10 @@ `SESSION_KIND: unattended`: a parent drives the run through `~/.claude/skills/gstack/bin/gstack-autoplan` (DRIVER.md; docs/unattended.md) and reads artifacts, never prose. Phases close -themselves, Eng runs last, the outside voice (none when lean) runs per DRIVER.md -(enabled `codex_reviews` is consent; foreground only), -and the run stops at `status=gate_pending`: never approve. `gstack-artifact -validate <dir>/run.json` is the completion check. End with +themselves, Eng runs last, and the outside voice (none when lean) runs per DRIVER.md +(enabled `codex_reviews` is consent; foreground only). + +{{AUTOPLAN_UNATTENDED_GATE}} + +`gstack-artifact validate <dir>/run.json` is the completion check. End with `GSTACK_RESULT: skill=autoplan status=<status> run=<dir>`. diff --git a/bin/gstack-skill-start b/bin/gstack-skill-start index 7bc9f2ad5..c371df7b3 100755 --- a/bin/gstack-skill-start +++ b/bin/gstack-skill-start @@ -750,7 +750,7 @@ fi # end interactive-onboarding guard (kind != spawned/unattended) if [ "$_SESSION_KIND" = "unattended" ]; then _UA_NONE="This skill writes no run directory and prints no GSTACK_RESULT line, so do not invent either; end at its first pending item and list that item, with every recommended option you took, in your final report." case "$SKILL_NAME" in - autoplan) _UA_RECORD="This skill writes a run directory through gstack-autoplan (plan.md, review-record.md, tasks.jsonl, decisions.jsonl, findings.jsonl, timing.json, run.json, gate.json): append each recommended choice to decisions.jsonl with kind \"auto\" and each pending one with kind \"approval\" and status \"pending\". The final approval gate is written as a gate list (gstack-gate render) and this run ENDS there with \`GSTACK_RESULT: skill=autoplan status=gate_pending run=<dir>\`." ;; + autoplan) _UA_RECORD="This skill writes a run directory through gstack-autoplan (plan.md, review-record.md, tasks.jsonl, decisions.jsonl, findings.jsonl, timing.json, run.json, gate.json): append each recommended choice to decisions.jsonl with kind \"auto\" and each pending one with kind \"approval\" and status \"pending\". The final approval gate is written as a gate list (gstack-gate render) and this run ENDS there with \`GSTACK_RESULT: skill=autoplan status=gate_pending run=<dir>\`. The one exception: when the owner's own reply to that gate is relayed to you, record it with \`gstack-autoplan answer --by agent:<id> --owner-reply-file <file>\` as sections/unattended.md says, never on your own judgment and never handed back to the owner to run." ;; eval-plan) _UA_RECORD="This run ENDS at the Step 4 spend approval, left pending, with \`GSTACK_RESULT: skill=eval-plan status=gate_pending run=<outdir>\`, where <outdir> is the pilot's --out directory. No helper writes a gate.json for this skill yet, so the parent reads the pilot report and spend ledger in <outdir>." ;; ship) _UA_RECORD="This skill ends with \`gstack-ship-receipt result\` (Step 22), which writes <state root>/ship/runs/<utc>-<head12>/ship-run.json (a /ship record, not a run manifest) and prints \`GSTACK_RESULT: skill=ship status=complete head=<sha> run=<dir>\`; a run that stops earlier prints \`GSTACK_RESULT: skill=ship status=refused run=-\` itself. The one publication allowed here: when an open PR for this branch exists and HEAD is its pushed head, update that PR's body and title with the receipt through the pr-body section's existing-PR path (never pr-create, never a push, and never \`gh pr ready\`: the draft-state items there concern an early PR this run never opened); otherwise the handoff section writes the receipt to .gstack/tmp/ship-receipt.md and the parent reads it there." ;; review) _UA_RECORD="$_UA_NONE The parent reads the review-log rows (\`gstack-review-read\`)." ;; diff --git a/changelog.d/unattended-autoplan-records-relayed-owner-reply.md b/changelog.d/unattended-autoplan-records-relayed-owner-reply.md new file mode 100644 index 000000000..f0393beb6 --- /dev/null +++ b/changelog.d/unattended-autoplan-records-relayed-owner-reply.md @@ -0,0 +1 @@ +- **Unattended /autoplan now records the owner's relayed gate reply instead of refusing it.** `autoplan/SKILL.md` told an unattended session to end at the final gate and never approve, while `autoplan/DRIVER.md` told it to record a relayed owner reply with `gstack-autoplan answer --by agent:<id> --owner-reply`, so whether a run reached `complete` depended on which file the model followed. Both now render one rule from `scripts/resolvers/autoplan-driver.ts`: never approve on your own judgment, stop at `gate_pending` when there is no owner reply, and record the owner's own relayed words with `--owner-reply-file`, never handing the command back to the owner. `gstack-skill-start`'s unattended block names the same exception for /autoplan. diff --git a/docs/unattended.md b/docs/unattended.md index e7b4163c2..c45a1dcd4 100644 --- a/docs/unattended.md +++ b/docs/unattended.md @@ -70,8 +70,11 @@ GSTACK_RESULT: skill=<name> status=<status> run=<dir> `status` is one of `complete`, `reviewed`, `gate_pending`, `incomplete`, `interrupted` or `refused`; `run` is where the run's record is. `/autoplan` -always ends `gate_pending` when it reaches the final gate, because the gate is -written and never approved. `/autoplan --ceo-only` ends `reviewed`: it runs the +ends `gate_pending` when it reaches the final gate with no owner reply, because +the session never approves the gate on its own judgment; when the owner's own +reply is relayed to it, the session records it with `gstack-autoplan answer +--by agent:<id> --owner-reply-file <file>` (see the driver steps below). +`/autoplan --ceo-only` ends `reviewed`: it runs the CEO phase alone, asks nothing, writes no gate and approves nothing (see [CEO-only runs](#ceo-only-runs)). Pure queries (`gstack-artifact schema`, `gstack-gate render`) print no result line. diff --git a/scripts/resolvers/autoplan-driver.ts b/scripts/resolvers/autoplan-driver.ts index e24cb6dcb..755130a55 100644 --- a/scripts/resolvers/autoplan-driver.ts +++ b/scripts/resolvers/autoplan-driver.ts @@ -83,3 +83,8 @@ export const generateAutoplanUnattendedVoice: ResolverFn = ctx => [ "Run the voice in the foreground, inside this turn: `gstack-outside-voice run ... --timeout-ms 540000` with the tool's own timeout above it (Claude Code Bash `timeout: 600000`), or a subagent you await. Never `run_in_background`, never a monitor, never end the turn to wait for it: an unattended session ends with its turn. When the voice fails, `submit --voice outside --unavailable \"<line>\"` with the matching line and carry on in the same turn, the native voice still counting:", ...outsideVoiceFailurePolicy(ctx, { timeoutMinutes: 9, onTimeout: 'missing-coverage', stderrOnEmpty: true, fallback: 'none', escape: 0 }).split('\n').map(l => l.replace(/^- /, '')), ].join(' '); + +/** GST-3068 (GEVA-549): the one gate rule for unattended /autoplan, rendered into the unattended profile and DRIVER.md so the two can't contradict. An agent never approves on its own judgment; a relayed owner reply is recorded with the owner's words bound. */ +export const AUTOPLAN_UNATTENDED_GATE_RULE = '**The gate, unattended.** Never approve it on your own judgment: with no owner reply, the run stops at `status=gate_pending`. Record an owner\'s reply relayed to you yourself, never handing them the command: `gstack-autoplan answer --out <dir> --gate-rev <n> --reply "<reply>" --by agent:<id> --owner-reply-file <file of their words>`.'; + +export const generateAutoplanUnattendedGate: ResolverFn = () => AUTOPLAN_UNATTENDED_GATE_RULE; diff --git a/scripts/resolvers/index.ts b/scripts/resolvers/index.ts index e68656396..5bee5bac8 100644 --- a/scripts/resolvers/index.ts +++ b/scripts/resolvers/index.ts @@ -53,7 +53,7 @@ import { generateImplementerDispatchRules } from './dispatch-rules'; import { generatePlanRealityRows } from './plan-reality'; import { generateSidewaysSweep, generateFlakeFixNumber } from './sideways'; import { generateCoordinatorContract } from './coordinator-contract'; -import { generateAutoplanDriverCodes, generateAutoplanDriverCommands, generateAutoplanDriverFiles, generateAutoplanDriverGate, generateAutoplanDriverPhases, generateAutoplanDriverResultFormat, generateAutoplanUnattendedVoice } from './autoplan-driver'; +import { generateAutoplanDriverCodes, generateAutoplanDriverCommands, generateAutoplanDriverFiles, generateAutoplanDriverGate, generateAutoplanDriverPhases, generateAutoplanDriverResultFormat, generateAutoplanUnattendedGate, generateAutoplanUnattendedVoice } from './autoplan-driver'; import { generateShipDriverStops } from './ship-driver'; import { generateQAScope, generateQAExploratory, generateQAFunctional, generateQAResource, generateQAReview, generateQAReviewPreflight, generateQAMethodReads } from './qa'; import { generateVoiceTriggerRouting } from './voice-triggers'; @@ -194,6 +194,7 @@ export const RESOLVERS: Record<string, ResolverFn> = { AUTOPLAN_DRIVER_FILES: generateAutoplanDriverFiles, AUTOPLAN_DRIVER_CODES: generateAutoplanDriverCodes, AUTOPLAN_UNATTENDED_VOICE: generateAutoplanUnattendedVoice, + AUTOPLAN_UNATTENDED_GATE: generateAutoplanUnattendedGate, SHIP_DRIVER_STOPS: generateShipDriverStops, SECTION, SECTION_INDEX, diff --git a/test/autoplan-unattended-gate-rule.test.ts b/test/autoplan-unattended-gate-rule.test.ts new file mode 100644 index 000000000..fcc924ce3 --- /dev/null +++ b/test/autoplan-unattended-gate-rule.test.ts @@ -0,0 +1,61 @@ +/** + * GST-3068 (GEVA-549, day-one eval at e671a531): unattended /autoplan read two rules at the + * final gate. autoplan/SKILL.md said "end here with its GSTACK_RESULT line and never approve"; + * autoplan/DRIVER.md said to record a relayed owner reply with --by agent:<id> --owner-reply. + * An A session followed DRIVER.md and reached status=complete; an S-full session followed + * SKILL.md, refused the owner's relayed "rec", told the owner to run the command and stayed + * gate_pending. One rule (AUTOPLAN_UNATTENDED_GATE_RULE) now renders into the unattended + * profile and DRIVER.md, and SKILL.md's gate step defers to it. These checks read the + * generated files, so a template edit that brings back an unconditional "never approve" fails. + */ +import { describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import { AUTOPLAN_UNATTENDED_GATE_RULE } from '../scripts/resolvers/autoplan-driver'; + +const ROOT = path.resolve(import.meta.dir, '..'); +const read = (rel: string) => fs.readFileSync(path.join(ROOT, rel), 'utf8'); +const flat = (s: string) => s.replace(/\s+/g, ' '); +const rule = flat(AUTOPLAN_UNATTENDED_GATE_RULE); + +/** Sentences that forbid approving with no condition: a "never approve" that neither limits itself to the agent's own judgment nor names an exception. */ +const unconditionalNeverApprove = (text: string) => + flat(text).split(/(?<=[.;])\s+/).filter(s => /never approve/i.test(s) && !/own judgment|only exception/i.test(s)); + +const GEVA_549_SKILL_SENTENCE = 'Under `SESSION_KIND: unattended` Read `sections/unattended.md`, end here with its `GSTACK_RESULT` line and never approve.'; +const GEVA_549_PROFILE_SENTENCE = 'and the run stops at `status=gate_pending`: never approve.'; + +describe('unattended /autoplan states one gate rule', () => { + test('the detector flags the two sentences GEVA-549 ran on (negative control)', () => { + expect(unconditionalNeverApprove(GEVA_549_SKILL_SENTENCE)).toHaveLength(1); + expect(unconditionalNeverApprove(GEVA_549_PROFILE_SENTENCE)).toHaveLength(1); + expect(unconditionalNeverApprove(rule)).toEqual([]); + }); + + test('the rule forbids approving on the agent\'s own judgment and records a relayed owner reply with the words bound', () => { + expect(rule).toContain('Never approve it on your own judgment: with no owner reply, the run stops at `status=gate_pending`.'); + expect(rule).toContain("Record an owner's reply relayed to you yourself, never handing them the command"); + expect(rule).toContain('--by agent:<id> --owner-reply-file <file of their words>'); + }); + + test('the unattended profile and DRIVER.md carry the rule verbatim, and SKILL.md\'s gate step defers to it', () => { + for (const rel of ['autoplan/sections/unattended.md', 'autoplan/DRIVER.md']) expect({ rel, has: flat(read(rel)).includes(rule) }).toEqual({ rel, has: true }); + const skill = flat(read('autoplan/SKILL.md')); + expect(skill).toContain('else = A. Under `SESSION_KIND: unattended` Read `sections/unattended.md` and settle the gate by its gate rule instead.'); + expect(skill).not.toContain(GEVA_549_SKILL_SENTENCE); + expect(flat(read('autoplan/sections/unattended.md'))).not.toContain(GEVA_549_PROFILE_SENTENCE); + }); + + test('no generated text an unattended /autoplan session reads says never approve without the condition', () => { + for (const rel of ['autoplan/SKILL.md', 'autoplan/sections/unattended.md', 'autoplan/DRIVER.md']) { + expect({ rel, unconditional: unconditionalNeverApprove(read(rel)) }).toEqual({ rel, unconditional: [] }); + } + }); + + test('every flag the rule names is one gstack-autoplan answer parses', () => { + const bin = read('bin/gstack-autoplan'); + const flags = [...new Set(AUTOPLAN_UNATTENDED_GATE_RULE.match(/--[a-z][a-z-]+/g) ?? [])]; + expect(flags).toEqual(['--out', '--gate-rev', '--reply', '--by', '--owner-reply-file']); + for (const f of flags) expect({ f, parsed: bin.includes(`'${f}'`) }).toEqual({ f, parsed: true }); + }); +}); diff --git a/test/fixtures/skill-md-size-allowlist.json b/test/fixtures/skill-md-size-allowlist.json index 1935d573d..872cede12 100644 --- a/test/fixtures/skill-md-size-allowlist.json +++ b/test/fixtures/skill-md-size-allowlist.json @@ -2,53 +2,53 @@ "_comment": "Generated SKILL.md files over the limit, pinned at their exact character count (#3199). test/skill-md-size-guard.test.ts fails when a pinned skill grows past its pin or shrinks below it: lower the pin when a skill shrinks and delete it once the skill is within the limit. Raising a pin is an owner decision. The shared-preamble hoist (GSTA-96) moved every tier 2+ skill's shared preamble into sections/preamble.md; the pins below are what remains.", "limit": 100000, "pins": { - "agy/gstack-autoplan": 109449, + "agy/gstack-autoplan": 109152, "agy/gstack-plan-design-review": 109103, "agy/gstack-plan-devex-review": 114944, "agy/gstack-plan-eng-review": 143530, "agy/gstack-review": 104723, - "codex/gstack-autoplan": 108192, + "codex/gstack-autoplan": 107895, "codex/gstack-plan-design-review": 110460, "codex/gstack-plan-devex-review": 116452, "codex/gstack-plan-eng-review": 144695, "codex/gstack-review": 104107, - "copilot/gstack-autoplan": 108892, + "copilot/gstack-autoplan": 108595, "copilot/gstack-plan-design-review": 109369, "copilot/gstack-plan-devex-review": 114406, "copilot/gstack-plan-eng-review": 143695, "copilot/gstack-review": 104304, - "cursor/gstack-autoplan": 108124, + "cursor/gstack-autoplan": 107827, "cursor/gstack-plan-design-review": 107911, "cursor/gstack-plan-devex-review": 113676, "cursor/gstack-plan-eng-review": 142285, "cursor/gstack-review": 122776, - "factory/gstack-autoplan": 110402, + "factory/gstack-autoplan": 110105, "factory/gstack-plan-design-review": 111009, "factory/gstack-plan-devex-review": 117039, "factory/gstack-plan-eng-review": 145393, "factory/gstack-review": 126549, - "gbrain/gstack-autoplan": 108531, + "gbrain/gstack-autoplan": 108234, "gbrain/gstack-plan-eng-review": 124932, - "gemini/gstack-autoplan": 108682, + "gemini/gstack-autoplan": 108385, "gemini/gstack-plan-design-review": 109197, "gemini/gstack-plan-devex-review": 114234, "gemini/gstack-plan-eng-review": 143528, "gemini/gstack-review": 104126, - "hermes/gstack-autoplan": 108536, + "hermes/gstack-autoplan": 108239, "hermes/gstack-plan-eng-review": 124885, - "kiro/gstack-autoplan": 107898, + "kiro/gstack-autoplan": 107601, "kiro/gstack-plan-design-review": 107757, "kiro/gstack-plan-devex-review": 113528, "kiro/gstack-plan-eng-review": 142147, "kiro/gstack-review": 122600, - "openclaw/gstack-autoplan": 108354, + "openclaw/gstack-autoplan": 108057, "openclaw/gstack-plan-eng-review": 123428, - "opencode/gstack-autoplan": 108873, + "opencode/gstack-autoplan": 108576, "opencode/gstack-plan-design-review": 109225, "opencode/gstack-plan-devex-review": 114284, "opencode/gstack-plan-eng-review": 143554, "opencode/gstack-review": 123398, - "slate/gstack-autoplan": 108011, + "slate/gstack-autoplan": 107714, "slate/gstack-plan-design-review": 107834, "slate/gstack-plan-devex-review": 113602, "slate/gstack-plan-eng-review": 142216, diff --git a/test/gstack-skill-start.test.ts b/test/gstack-skill-start.test.ts index f3f8be40c..07bd7ca20 100644 --- a/test/gstack-skill-start.test.ts +++ b/test/gstack-skill-start.test.ts @@ -425,6 +425,9 @@ describe('gstack-skill-start behavior', () => { expect(autoplan).toContain('gate list (gstack-gate render)'); expect(autoplan).toMatch(/decisions\.jsonl with kind "auto"/); expect(autoplan).toContain('gate.json'); + // GST-3068 (GEVA-549): the general "never approve a gate" rule defers to the record rule's exception; autoplan's names the relayed owner reply. + expect(autoplan).toContain("The one exception: when the owner's own reply to that gate is relayed to you, record it with `gstack-autoplan answer --by agent:<id> --owner-reply-file <file>` as sections/unattended.md says, never on your own judgment and never handed back to the owner to run."); + for (const [skill, text] of Object.entries(blocks)) if (skill !== 'autoplan') expect({ skill, relays: text.includes('--owner-reply') }).toEqual({ skill, relays: false }); const evalPlan = blocks['eval-plan']; expect(evalPlan).toContain('`GSTACK_RESULT: skill=eval-plan status=gate_pending run=<outdir>`'); expect(evalPlan).toContain('No helper writes a gate.json'); diff --git a/test/host-conformance.test.ts b/test/host-conformance.test.ts index 56d6bb725..8695a55d8 100644 --- a/test/host-conformance.test.ts +++ b/test/host-conformance.test.ts @@ -224,7 +224,7 @@ describe.skipIf(process.platform === 'win32')('host conformance kit', () => { expect(runSetup(f, join(src, 'setup'), ['--host', 'claude']).status).toBe(0); const sections = join(f.home, '.claude/skills/autoplan/sections'); expect(lstatSync(sections).isSymbolicLink(), sections).toBe(true); - expect(readFileSync(join(f.home, '.claude/skills/autoplan/SKILL.md'), 'utf8')).toContain('Read `sections/unattended.md`, end here'); + expect(readFileSync(join(f.home, '.claude/skills/autoplan/SKILL.md'), 'utf8')).toContain('Read `sections/unattended.md` and settle the gate'); rmSync(join(src, 'autoplan/sections/unattended.md')); const status = runSetup(f, join(src, 'setup'), ['--status']); expect(status.status, status.stdout + status.stderr).toBe(1);