* feat: retire /investigate after a no-gain eval (GST-1717)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(skill-start): core routing offer drops every non-core rule line (GST-1717)
The retired /investigate rule became 'Bugs/errors → fix them directly', which no
longer matches '→ invoke /', so proactive=core leaked it into the offered section.
Match any '- … → ' rule line.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* revert(review): keep review/checklist.md at its no-loss-pinned bytes (GST-1717)
The pin names replay GSTA-224 H1; a reviewer checklist edit merges only after a
passing no-loss replay (crew rule 15). The /investigate flake line stays until a
replayed repin.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test: repair free-suite fallout from retiring /investigate (GST-1717)
CI run 42340956269 at 83a8a9a: drop fixtures, ledger rows, allowlist entries and
counts for the retired skill and its deleted paid files; record the harness
change as non-behavioral; the test-of-test detector accepts path.join(ROOT, 'bin');
uncovered-prompt fixtures use a tracked prompt, since an untracked path now
reads as a deleted one.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(relink): back up a retired skill's real SKILL.md before cleaning it (GST-1717)
With the skill's source gone, the customized-file check never ran, so a
banner-carrying local copy of investigate/SKILL.md was deleted instead of moved
to the backup root (red-team review, pass 1).
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(qm): restore the review checklist's flake-line rewrite (GST-1717)
review/checklist.md keeps its pinned /investigate line, so the QM edition needs
its replaceText rule back; qm/ stays fresh against the generator.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(changelog): name the setup flag break, the relink backup and the pinned checklist line (GST-1717)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(setup,relink): back up a retired skill's real SKILL.md instead of deleting it (GST-1717)
Scoped fix round under grant a-360dd62bd46a (outside-voice blocking finding and
adversarial #1, #4). setup's prune compares a copied host entry with the render
before removing that render, and backs up a copy that differs or whose render is
gone; relink no longer takes the marker's rm -rf for a skill whose source is
gone, so a real SKILL.md is backed up and the user's files stay. setup reports a
failed remove-source of the retired gate hooks on stderr.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(pr-profile): a changed prompt counts as deleted only when it is gone from disk (GST-1717)
An untracked prompt file on disk still needs prompt validation (targeted review
finding); harness change recorded as non-behavioral.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(changelog): exact backup wording, the hook-removal warning and GST-1744 (GST-1717)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(setup): one backup folder per host for a retired skill's copies (GST-1717)
Under grant a-c95576b8b1f9 (fix-check#1): every copy host moved its
gstack-investigate/SKILL.md to the same backup path, so a later host's copy
overwrote an earlier host's edits. A sandboxed ./setup --host factory run showed
gen-skill-docs removes the retired render before the prune, so every real copy
is backed up; each host now gets its own folder.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(changelog): say exactly what setup backs up for a retired skill (GST-1717)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* docs(changelog): end the two colon sentences prose-lint flagged (GST-1717)
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(retired-skills): name plan-eng-review once behind a pregate-literal waiver (GST-1717)
pregate's literals check read the GSTACK_RETIRED_SKILLS change as an old
value still in the tests. plan-eng-review is still retired; investigate was
appended. The fixture names it once, with the waiver.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
16 KiB
Skill gates: opt-in enforcement hooks
gstack's advisory skills say what a careful agent does; a gate makes one rule hold. Each gate is a Claude Code hook that is off unless you turn it on, never changes the skill's text, and leaves a countable marker for every decision it makes (GST-1461). They follow the contract hook in skill-routing-and-contracts.md: the same switch order, the same per-session record, the same fail-open rule.
| Gate | Switch | What it holds |
|---|---|---|
| /plan-ceo-review | ceo_review_gate |
After /plan-ceo-review loads, the final stop (of the session or of a review subagent) waits until the review's output file ranks the top risks |
| /autoplan | autoplan_gate |
After /autoplan loads, the session's final stop waits until every applicable phase has published its report and the Final Approval Gate is written (GST-1666) |
Turn a gate on
The switch resolves like the contract hook's, and the first one set wins:
- The
./setupflag:--ceo-review-gate/--no-ceo-review-gate(also--ceo-review-gate=on|off), or--autoplan-gate/--no-autoplan-gate(also--autoplan-gate=on|off). A flag is saved to config. - The environment:
GSTACK_CEO_REVIEW_GATE=on|offorGSTACK_AUTOPLAN_GATE=on|off. Not saved. - Config:
gstack-config set ceo_review_gate on|offorgstack-config set autoplan_gate on|off. - The default,
SWITCH_DEFAULTinlib/skill-route.ts: off for both.
Setup registers a gate's shims in ~/.claude/settings.json when its switch is on (Stop and SubagentStop for /plan-ceo-review; Stop for /autoplan) and removes them when it is off. The hooks also read the switch on every call, so gstack-config set ceo_review_gate off takes effect on the next stop.
Headless runs
For claude -p or an eval harness, skip settings.json:
gstack-skill-gates settings --ceo-review --autoplan > gates.json
claude -p "<task>" --settings gates.json
Name any of the gates. The JSON registers their hooks from this checkout and sets each named gate's switch (GSTACK_CEO_REVIEW_GATE=on, GSTACK_AUTOPLAN_GATE=on) through its env block. Under the Agent SDK, pass the same JSON as settings or include user in settingSources after running setup; with neither, or with --bare, no hook loads and no fired marker appears. Check every session with gstack-skill-gates report --session <id> --expect armed (exit codes).
The /investigate gate was retired with /investigate (GST-1717). ./setup removes its hooks from ~/.claude/settings.json whatever investigate_gate says.
The /plan-ceo-review gate
Once a session loads /plan-ceo-review (the Skill tool, a typed /plan-ceo-review, or a Read of its installed SKILL.md), its final stop is held until the review's output ranks the top risks. Only a final stop counts, by the contract hook's rule (isFinalTurn in lib/skill-contract.ts): a STATUS line other than NEEDS_CONTEXT, or a turn that does not end waiting on the user. A stop that asks the user a question (a closing ?, AskUserQuestion, STATUS: NEEDS_CONTEXT) in an interactive session is never blocked; in a session no human answers (gstack-session-kind says headless or unattended) it counts as final.
What meets it. One output file, modified after the load (its mtime is later than the load record's transcript timestamp), with a markdown heading whose text contains Findings or Top risks (any level, any case), followed inside that section by a numbered list that starts at 1 and has at least one item that is not a placeholder ([risk], TBD, ...). A bullet list, a list starting at 2, a list under the next same-level heading, or a section of placeholders does not count:
## Findings
1. Payment retries can double-charge when the provider times out.
2. No rollback path for the new orders schema.
Which files count, in order, stopping at the first one that meets it:
GSTACK_CEO_REVIEW_OUTPUT, when set (relative paths resolve against the session's cwd).- Files the run wrote with Write, Edit or MultiEdit.
- Markdown files in the git repository at the session's cwd that differ from HEAD or are untracked and not ignored.
$GSTACK_STATE_ROOT/projects/<slug>/ceo-plans/*.md, with the slug frombin/gstack-slug.
Bash commands are never parsed: a review written with a heredoc or tee is found by the repository and ceo-plans scans.
Like the /autoplan gate, the hook first re-reads the transcript until it holds the stop's last_assistant_message, for at most 2 seconds or half its deadline (settledTranscript in lib/skill-gates.ts). Claude Code can fire Stop before it writes the turn's last lines, so a single read could miss the Write of a review kept outside the repository (a plan under ~/.claude/plans) and hold a finished run, or miss the load itself and let an unfinished one through.
The block. Its reason starts with the blocked via=stop marker and says exactly what to write and where: a ## Findings section ranking the top risks as a numbered list, in GSTACK_CEO_REVIEW_OUTPUT, else the last markdown file the run wrote, else the last plan file it read (never a file in a skills/ directory, judged from the repository root so a project that itself sits under a skills/ folder still counts, such as the review's own methodology sections; such a file never releases the gate either), plus each file checked and why it fell short. It never names the off switch.
The cap. A run is blocked at most 3 times. The next final stop without ranked risks goes through with an exhausted marker and exhausted_at in the record. stop_hook_active (Claude Code sets it once a Stop hook has blocked the turn) does not skip the check; the cap ends the repeats. After released or exhausted the gate does nothing for the rest of that run; a new load starts a new run.
Subagents. The SubagentStop shim reads the payload's agent_transcript_path (Claude Code 2.1 sends it with agent_id) and gates the subagent only when that subagent's own transcript loaded /plan-ceo-review, so an Explore or test subagent finishing in a session that ran the review is never touched. Its records are sidechain records, parsed as such. A subagent always ends on a final turn (no human answers it), and its run is counted under agent in the record. A payload without the path, or with an empty one, is left alone.
The /autoplan gate
Opus 5.5 told to run /autoplan in a one-shot session with no human spent about 20 times a plain session and still showed none of autoplan's phases in an eval (GEVA-2's G admission, 0 of 23): nothing held the session until the pipeline finished, and a session could stop after any phase. Autoplan's own publication guard (autoplan/bin/phase-publication-hook) only denies entering a phase before the previous one published; this gate holds the stop. It never changes autoplan's text or what an un-gated run does.
Once a session loads /autoplan (the Skill tool, a typed /autoplan, or a Read of its installed SKILL.md), its final stop is held until the run is complete. Final stops are judged as for /plan-ceo-review: a stop that asks the user in an interactive session is never held, and in a session no human answers it counts as final.
What meets it. One run is the transcript from the load to the next /autoplan load. A full run is complete when:
- Phase 0 ran scope: a successful
gstack-autoplan-snapshot init(itsscopefield) orscoperesult withuiRequiredanddxRequired. It decides the applicable phases: CEO and Eng always, Design whenuiRequired, DX whendxRequired. - Every applicable phase published, as
sections/phase-close.mdstep 6 says: assistant text with itsPhase N completeline, then a successful Bash no-optrue autoplan-published <phase>. A no-op with no report since the previous one, or a failed one, does not count. - The Phase 4 Final Approval Gate is written: a successful
gstack-gate rendercall, or a Write or Edit of agate.json.
A --ceo-only run needs Phase 1 published and the GSTACK_RESULT: skill=autoplan status=reviewed end line from sections/ceo-only.md.
Claude Code can fire Stop before it writes the turn's last lines to the transcript (the smoke run saw a scope call and a phase publication land about 200 ms late), so the hook re-reads the transcript until it holds the stop's last_assistant_message, for at most 2 seconds or half its deadline, and judges what is there then.
What it never holds. A stop while a native reviewer launched in the background has not reported back (autoplan's "Waiting for " barrier); the next final stop after the notification is checked. A run that drives the gstack-autoplan runner (next, submit, answer or export), which keeps its own record (docs/unattended.md): it gets one skipped reason=runner marker and is never blocked.
The block. Its reason starts with the blocked via=stop missing=<step> published=<phases> marker and names the next step: run Phase 0 (missing=scope), continue at a phase's driver Read when it has not started, finish a started phase's close and publish its report (missing=ceo|design|dx|eng), write and render the gate (missing=gate), or end a --ceo-only run (missing=end). It lists the applicable and published phases and never names the off switch.
The cap and the record. As for /plan-ceo-review: at most 3 blocks per run, then the next final stop goes through with exhausted missing=<step>; stop_hook_active does not skip the check. The session record keeps each run's phases and published as of its last final stop, so gstack-skill-gates report --session <id> --expect armed --json says per session whether /autoplan armed, how often it was held, and whether it was released or exhausted. An eval can read exposure from that record instead of from the model's wording.
Markers
Every decision is one marker line:
GSTACK_GATE <hook> <event> session=<session id> [key=value ...]
| Event | When | Fields |
|---|---|---|
fired |
the gate armed for a skill run (once per run; for /plan-ceo-review and /autoplan, its first final stop) | skill, on (the hook event) |
blocked |
each refusal | skill, via=stop, files (for /plan-ceo-review's via=stop, the candidate files checked, or none), missing and published (/autoplan: the next missing step and the phases published so far), n (blocks so far in this run) |
released |
an output file ranks the top risks, or the /autoplan run is complete (once per run) | skill, file (the output file's name) or phases (the phases /autoplan published), blocked |
exhausted |
/plan-ceo-review and /autoplan: a fourth final stop after 3 blocks, which goes through | skill, missing (/autoplan), blocked |
skipped |
the gate could not arm (once per session and reason), or an /autoplan run drives the gstack-autoplan runner (once per run) | skill, `reason=no-git |
error |
a hook error; the call or stop went through | code, why |
Where each one lands:
- Session record (the count source):
$GSTACK_STATE_ROOT/skill-gates/<session id>/record.json, mode 0600, written atomically under the session lock. Per gate and skill run it holdsfired_at,released_at,file,blocked,exhausted_at,agent(a subagent's run),phasesandpublished(/autoplan), plus the gate'serrorsandskippedreasons.gstack-skill-gates reportreads these. - Trace log:
$GSTACK_STATE_ROOT/hooks/ceo-review-gate.logandhooks/autoplan-gate.log, one line per marker in the contract hook's shape (<time> session=<id> event=<event> key=value ...). It rotates at 1 MiB, so count from the record. - Transcript: a block puts the marker on the first line of its reason, which Claude Code shows the model (a
hook_blocking_errorattachment for Stop and SubagentStop).fired,released,exhaustedanderrorridesystemMessage, which the transcript records as ahook_system_messageattachment and the model never sees, so counting them does not change the session.
Session directories untouched for 7 days are deleted the next time a gate arms.
Errors
A gate never blocks because it broke. On any error it writes an error marker with a code to the record, the trace log, stderr and the transcript (systemMessage), and exits 0 with no decision, so the call or stop goes through.
| Code | Cause | Fix |
|---|---|---|
GATE_BAD_PAYLOAD |
the hook input is not a JSON object with session_id and transcript_path, or is over 1 MiB |
a Claude Code version change; update gstack |
GATE_TRANSCRIPT |
the transcript is missing, over 32 MiB, or not a shape the parser knows, or the /plan-ceo-review load record has no timestamp | see skill-routing-and-contracts.md |
GATE_TIMEOUT |
the hook ran past its 5 second deadline (GSTACK_SKILL_GATE_DEADLINE_MS changes it, up to 20 seconds; a larger value is clamped to 20) |
a very large transcript or a slow disk |
GATE_STATE |
the session record could not be read or written | make $GSTACK_STATE_ROOT writable, or delete skill-gates/<session id>/ |
GATE_LOCKED |
another hook held the session lock for over 3 seconds | transient; a lock older than 30 seconds is taken over |
GATE_SLUG |
/plan-ceo-review: bin/gstack-slug failed, so the ceo-plans directory could not be checked |
run gstack-slug in the repository; GSTACK_PROJECT_SLUG pins it |
GATE_UNEXPECTED |
any other throw | report it with the trace log line |
Hook timeout: a killed hook cannot fail closed
Claude Code kills a hook that runs past the timeout in its registration and then lets the stop or tool call through as if the hook had allowed it. It writes nothing to the stream output about the kill, and the hook settings have no option that makes a timed-out hook block. A gate killed this way leaves no error marker either, because its own deadline never fired, so gstack-skill-gates report cannot see it. There is no way for the gate to fail closed after a kill.
So every gate registration outlives the gate. HOOK_TIMEOUT_S in lib/skill-gates.ts is the longest deadline (20 seconds) plus a 5 second margin for Bun startup and the GATE_TIMEOUT marker write, rounded up: 25 seconds. gstack-skill-gates settings writes it on every hook, and ./setup reads it from gstack-skill-gates hook-timeout; re-running setup updates an older 10 second registration. A slow scan then ends in the gate's own GATE_TIMEOUT, which is recorded and makes report exit 3. Keep your own settings files at or above this number.
Tested with Claude Code 2.1.318 through claude -p --settings only: a Stop hook killed at a 3 second timeout let the stop through, and the same hook with a 15 second timeout blocked it (GST-1615).
Exit codes
gstack-skill-gates report [--session <id>] [--expect armed] [--json] lists each session's gate counts: runs, fired, blocked, released, exhausted, observed and errors.
| Exit | Code | Meaning |
|---|---|---|
| 0 | done | |
| 1 | SKILL_GATES_USAGE |
a bad argument |
| 2 | GATE_NOT_ARMED |
--expect armed and a session has no fired marker: the hooks never loaded (see Headless runs) or the skill never ran |
| 3 | GATE_ERRORED |
a session recorded a hook error, so some calls or stops went through unchecked |
Headless smoke
bun scripts/skill-gates-smoke.ts runs the real Claude Code CLI against a local stand-in for the Anthropic Messages API that plays two scripted sessions. The first loads /plan-ceo-review, writes plan.md with no Findings and ends its turn (the Stop is blocked), then appends a ranked Findings section through a Bash heredoc and ends again (released). The second loads /autoplan, runs Phase 0 scope, publishes Phase 1 and ends its turn (the Stop is blocked with missing=eng), then publishes Phase 3, writes gate.json and ends again (released). No model is called and nothing is spent. It fetches the pinned CLI (CLAUDE_CODE_VERSION in the script, verified at 2.1.318) with npm into a temp directory at run time and never stores it; --claude <path> uses one you have. It checks the record, the trace log, the transcript attachments, that systemMessage markers never reach a request, and that the CEO review was released by plan.md. Run it after a Claude Code upgrade.