v1.48.0.0 feat: AskUserQuestion split rule + runtime AUTO_DECIDE carve-out (#1740)

* feat(preamble): add "Handling 5+ options — split, never drop" rule

Agents repeatedly hit Conductor's 4-option AskUserQuestion cap and
silently drop one option to fit, shrinking the user's decision space.
This rule names the bug and gives two compliant shapes: batch into
≤4-groups (for coherent alternatives) or split into N sequential
per-option calls (for independent scope items, default).

Inline preamble subsection is ~15 lines (rule + buckets + pointer).
Full reference with worked examples, Hold/dependency semantics, and
final-summary validation lives in docs/askuserquestion-split.md.
The agent loads the docs file on demand when N>4.

Per-option call shape: D<N>.k header, ELI10, Recommendation, kind-note
(no completeness score — decision actions, not coverage), Include /
Defer / Cut / Hold buckets. Hold stops the chain immediately; the
final D<N>.final call validates dependencies and confirms the
assembled scope.

question_ids: <skill>-split-<option-slug> (kebab-case ASCII, ≤64
chars). Also fixes orphan "12. " prefix on the existing CJK rule.

Tier-2+ skills inherit via the existing resolver. SKILL.md regenerated
for all 41 affected skills + 3 golden fixtures. Net diff per SKILL.md:
~34 lines (vs ~110 for the full inline version).

6 tests pin the inline contract (4-option cap, buckets, D-numbering,
docs pointer, runtime AUTO_DECIDE gate reference, orphan 12 regression).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(question-pref): runtime AUTO_DECIDE carve-out for *-split-* ids

Split chains (per-option AskUserQuestion calls emitted by the new
"Handling 5+ options" rule) must never be silently auto-approved
via /plan-tune preferences. The user's option set is sacred.

Layer 1 (mechanism): unique <skill>-split-<option-slug> ids prevent
cross-option preference leakage. Layer 2 (this commit): the runtime
checker `gstack-question-preference --check` detects any id matching
*-split-* and forces ASK_NORMALLY even when never-ask or
ask-only-for-one-way preferences exist for that exact id. An
explanatory note tells the user their preference was bypassed and why.

7 tests pin the carve-out: no-pref baseline, never-ask override,
explanatory note text, ask-only-for-one-way override, always-ask
(no note), non-split id containing "split" word (negative case for
regex specificity), multi-skill split id formats.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(e2e): split-overflow regression for /plan-ceo-review

Periodic-tier E2E test that catches the original failure mode the
user complained about: 5+ options for ONE decision must split into
N sequential AskUserQuestion calls, not drop one to fit Conductor's
4-option cap.

Fixture: 5 independent chat-platform integration candidates
(Slack/Discord/Teams/Telegram/Mattermost), each carrying its own
include/defer/cut decision. Floor = 4 review-phase AUQs (standard
[N-1] tolerance band). Pre-fix "drop to 4 + 1 dropped" fails this
floor.

Wired into test/helpers/touchfiles.ts: tier periodic, depends on
plan-ceo-review/**, the new preamble subsection, the question-pref
binary (for the carve-out), and the runner helper. touchfiles.test.ts
expected count bumped 21 → 22 to account for the new entry.

Cost: ~$0.30/run when EVALS_TIER=periodic. Skips silently otherwise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: post-merge regen + rebase size-budget baseline to v1.47.0.0

After merging origin/main (v1.45 → v1.47), three things needed cleanup:

1. spec/SKILL.md (main's new skill) regenerated to include our split-vs-drop
   preamble subsection — same mechanical regen as the other 41 tier-2+ skills.
2. Three golden ship fixtures refreshed to capture main's GSTACK_PLAN_MODE
   block + /spec routing entry + jargon-list.json refactor.
3. docs/skills.md — added /spec table row that main's PR (#1698/#1733) shipped
   without. Pre-existing failure on main; this PR catches and fixes.

Also rebased test/skill-size-budget.test.ts from v1.44.1 → v1.47.0.0 baseline.
Main's v1.46 (catalog tokens trim) + v1.47 (/spec skill) pushed the v1.44.1
anchor past the 5% ratchet to ×1.059 — pre-existing failure on main. This
PR captures a fresh parity-baseline-v1.47.0.0.json and re-anchors the test
there. Historical v1.44.1.json and v1.46.0.0.json retained in test/fixtures/
for reference. Our subsection contributes ~0.1% of the post-rebase corpus.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v1.48.0.0)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-05-26 23:43:07 -07:00
committed by GitHub
parent f8bb59094d
commit a6fb31726c
60 changed files with 2902 additions and 62 deletions
+56
View File
@@ -120,3 +120,59 @@ export const FORCING_BATCHING_ENG = [
'iterate the payload to recompute the dependency graph. Could cache the',
'graph on the first attempt; not planned.',
].join('\n');
/**
* Split-overflow regression seed (periodic tier).
*
* Catches the original failure mode the user complained about: when the
* agent has 5+ options for ONE conceptual decision, it must split into N
* sequential AskUserQuestion calls (or batch into compatible ≤4-groups),
* NOT drop an option arbitrarily to fit Conductor's 4-option cap.
*
* Fixture shape: 5 independent platform-integration candidates for ONE
* scope decision. Each is independent (no dependencies between them) so
* the natural compliant shape is a per-option split chain at parent D<N>.
*
* Used by test/skill-e2e-plan-ceo-split-overflow.test.ts to assert the
* agent fires >= 4 review-phase AUQs (floor uses the standard [N-1]
* tolerance band, accounting for one expected scope-reduction-or-merge
* call before the per-option chain begins).
*
* Pre-fix behavior: agent fires 1 AUQ with 4 options, "trims" the 5th
* via prose ("E5 is the largest lift and a natural follow-up; moving to
* TODOs without asking"). That's the bug. Floor of 4 detects it.
*/
export const FORCING_SPLIT_OVERFLOW_CEO = [
'Please review this plan and help me decide scope. Write your plan-mode plan to /tmp/gstack-test-plan-ceo-split-overflow.md (use Edit/Write to that exact path).',
'',
'# Plan: Pick which chat-platform integrations to ship this quarter',
'',
'We have engineering bandwidth for at most 2-3 integrations this quarter.',
'I need your help deciding which to prioritize. Below are 5 candidates,',
'each fully independent of the others (no shared infrastructure, no',
'dependencies between them). For each, the user can independently decide:',
'include in this scope, defer to next quarter, or cut entirely.',
'',
'## E1) Slack — DM bot for incident alerts',
'Build cost: ~2 weeks. Existing Slack auth flow we can reuse. High user',
'demand (top customer request in Q2 survey, ~40% of asks).',
'',
'## E2) Discord — guild bot for community channels',
'Build cost: ~3 weeks. Greenfield integration, no existing auth. Medium',
'demand (~15% of asks, but loud community).',
'',
'## E3) Microsoft Teams — webhook + bot framework',
'Build cost: ~4 weeks. Enterprise customers specifically asked for this.',
'Highest revenue impact per user but smallest user count (~5% of asks).',
'',
'## E4) Telegram — bot API integration',
'Build cost: ~1 week. Simplest API surface. Low strategic value but',
'cheap win (~8% of asks, mostly from international users).',
'',
'## E5) Mattermost — REST plugin',
'Build cost: ~2 weeks. Self-hosted enterprise users. Niche but locked-in',
'segment (~3% of asks but all from high-ARR accounts).',
'',
'Please walk me through each candidate and help me decide include/defer/cut',
'per option. I want individual decisions per candidate, not a bundled pick.',
].join('\n');
+80 -1
View File
@@ -107,6 +107,19 @@ _CHECKPOINT_MODE=$(~/.claude/skills/gstack/bin/gstack-config get checkpoint_mode
_CHECKPOINT_PUSH=$(~/.claude/skills/gstack/bin/gstack-config get checkpoint_push 2>/dev/null || echo "false")
echo "CHECKPOINT_MODE: $_CHECKPOINT_MODE"
echo "CHECKPOINT_PUSH: $_CHECKPOINT_PUSH"
# Plan-mode hint for skills like /spec that branch behavior on plan-mode state.
# Claude Code exposes plan mode via system reminders; we detect best-effort
# from CLAUDE_PLAN_FILE (set by the harness when plan mode is active) and
# fall back to "inactive". Codex hosts and Claude execution mode both end up
# inactive, which is the safe default (defaults to file+execute pipeline).
if [ -n "${CLAUDE_PLAN_FILE:-}${GSTACK_PLAN_MODE_FORCE:-}" ]; then
export GSTACK_PLAN_MODE="active"
elif [ "${GSTACK_PLAN_MODE:-}" = "active" ]; then
export GSTACK_PLAN_MODE="active"
else
export GSTACK_PLAN_MODE="inactive"
fi
echo "GSTACK_PLAN_MODE: $GSTACK_PLAN_MODE"
[ -n "$OPENCLAW_SESSION" ] && echo "SPAWNED_SESSION: true" || true
```
@@ -238,6 +251,7 @@ Key routing rules:
- Ship/deploy/PR → invoke /ship or /land-and-deploy
- Save progress → invoke /context-save
- Resume context → invoke /context-restore
- Author a backlog-ready spec/issue → invoke /spec
```
Then commit the change: `git add CLAUDE.md && git commit -m "chore: add gstack skill routing rules to CLAUDE.md"`
@@ -324,7 +338,36 @@ Effort both-scales: when an option involves effort, label both human-team and CC
Net line closes the tradeoff. Per-skill instructions may add stricter rules.
12. **Non-ASCII characters — write directly, never \u-escape.** When any
### Handling 5+ options — split, never drop
AskUserQuestion caps every call at **4 options**. With 5+ real options, NEVER
drop, merge, or silently defer one to fit. Pick a compliant shape:
- **Batch into ≤4-groups** — for coherent alternatives (e.g. version bumps,
layout variants). One call, 5th surfaced only if first 4 don't fit.
- **Split per-option** — for independent scope items (e.g. "ship E1..E6?").
Fire N sequential calls, one per option. Default to this when unsure.
Per-option call shape: `D<N>.k` header (e.g. D3.1..D3.5), ELI10 per option,
Recommendation, kind-note (no completeness score — Include/Defer/Cut/Hold are
decision actions), and 4 buckets:
**A) Include**, **B) Defer**, **C) Cut**, **D) Hold** (stop chain, discuss).
After the chain, fire `D<N>.final` to validate the assembled set (reprompt
dependency conflicts) and confirm shipping it. Use `D<N>.revise-<k>` to
revise one option without re-running the chain.
For N>6, fire a `D<N>.0` meta-AskUserQuestion first (proceed / narrow / batch).
question_ids for split chains: `<skill>-split-<option-slug>` (kebab-case ASCII,
≤64 chars, `-2`/`-3` suffix on collision). The runtime checker
(`bin/gstack-question-preference`) refuses `never-ask` on any `*-split-*` id,
so split chains are never AUTO_DECIDE-eligible — the user's option set is sacred.
**Full rule + worked examples + Hold/dependency semantics:** see
`docs/askuserquestion-split.md` in the gstack repo. Read on demand when N>4.
**Non-ASCII characters — write directly, never \u-escape.** When any
string field (question, option label, option description) contains
Chinese (繁體/簡體), Japanese, Korean, or other non-ASCII text, emit
the literal UTF-8 characters in the JSON string. **Never escape them
@@ -357,6 +400,9 @@ Before calling AskUserQuestion, verify:
- [ ] Net line closes the decision
- [ ] You are calling the tool, not writing prose
- [ ] Non-ASCII characters (CJK / accents) written directly, NOT \u-escaped
- [ ] If you had 5+ options, you split (or batched into ≤4-groups) — did NOT drop any
- [ ] If you split, you checked dependencies between options before firing the chain
- [ ] If a per-option Hold fires, you stopped the chain immediately (didn't queue)
## Artifacts Sync (skill start)
@@ -2926,6 +2972,39 @@ you missed it.>
<If no plan file: "No plan file detected.">
<If plan items deferred: list deferred items>
## Linked Spec
<Auto-detect: look for /spec archives matching this branch via:
eval "$(${ctx.paths.binDir}/gstack-paths)"
eval "$(${ctx.paths.binDir}/gstack-slug)"
CURRENT_BRANCH=$(git branch --show-current)
SPEC_ARCHIVES="$GSTACK_STATE_ROOT/projects/$SLUG/specs"
# Find newest archive whose spec_branch frontmatter matches current branch (or one of its
# parents — if spec spawned worktree spec/<slug>-$$, the spawned worktree IS where /ship runs).
SPEC_FILE=$(grep -l "^spec_branch: $CURRENT_BRANCH$" "$SPEC_ARCHIVES"/*.md 2>/dev/null | head -1)
[ -z "$SPEC_FILE" ] && exit # no spec; omit this section entirely
SPEC_ISSUE=$(grep "^spec_issue_number:" "$SPEC_FILE" | cut -d' ' -f2)
[ -z "$SPEC_ISSUE" ] && exit # spec archive exists but no issue number; omit
# CONDITIONAL Closes #N (codex F4): only add when Plan Completion above is "complete".
# If the plan completion gate from Step 8 reports any deferred or failed items, emit:
# "Linked to #$SPEC_ISSUE (partial delivery — NOT auto-closing; close manually after follow-up)"
# If Plan Completion is fully complete, emit:
# "Closes #$SPEC_ISSUE"
# and include the Closes #N line in the PR body so GitHub auto-closes on merge.>
<Format:
Closes #<N>
This PR delivers the spec at <archive path relative to repo root>.
Spec filed: <spec_filed_at from frontmatter>>
<If partial delivery, emit instead:
Linked to #<N> (partial delivery — not auto-closing).
Deferred items: <list from Plan Completion>.
Close #<N> manually after follow-up lands.>
<If no /spec archive matches this branch: omit this entire section.>
## Verification Results
<If verification ran: summary from Step 8.1 (N PASS, M FAIL, K SKIPPED)>
<If skipped: reason (no plan, no server, no verification section)>
+80 -1
View File
@@ -93,6 +93,19 @@ _CHECKPOINT_MODE=$($GSTACK_BIN/gstack-config get checkpoint_mode 2>/dev/null ||
_CHECKPOINT_PUSH=$($GSTACK_BIN/gstack-config get checkpoint_push 2>/dev/null || echo "false")
echo "CHECKPOINT_MODE: $_CHECKPOINT_MODE"
echo "CHECKPOINT_PUSH: $_CHECKPOINT_PUSH"
# Plan-mode hint for skills like /spec that branch behavior on plan-mode state.
# Claude Code exposes plan mode via system reminders; we detect best-effort
# from CLAUDE_PLAN_FILE (set by the harness when plan mode is active) and
# fall back to "inactive". Codex hosts and Claude execution mode both end up
# inactive, which is the safe default (defaults to file+execute pipeline).
if [ -n "${CLAUDE_PLAN_FILE:-}${GSTACK_PLAN_MODE_FORCE:-}" ]; then
export GSTACK_PLAN_MODE="active"
elif [ "${GSTACK_PLAN_MODE:-}" = "active" ]; then
export GSTACK_PLAN_MODE="active"
else
export GSTACK_PLAN_MODE="inactive"
fi
echo "GSTACK_PLAN_MODE: $GSTACK_PLAN_MODE"
[ -n "$OPENCLAW_SESSION" ] && echo "SPAWNED_SESSION: true" || true
```
@@ -224,6 +237,7 @@ Key routing rules:
- Ship/deploy/PR → invoke /ship or /land-and-deploy
- Save progress → invoke /context-save
- Resume context → invoke /context-restore
- Author a backlog-ready spec/issue → invoke /spec
```
Then commit the change: `git add CLAUDE.md && git commit -m "chore: add gstack skill routing rules to CLAUDE.md"`
@@ -310,7 +324,36 @@ Effort both-scales: when an option involves effort, label both human-team and CC
Net line closes the tradeoff. Per-skill instructions may add stricter rules.
12. **Non-ASCII characters — write directly, never \u-escape.** When any
### Handling 5+ options — split, never drop
AskUserQuestion caps every call at **4 options**. With 5+ real options, NEVER
drop, merge, or silently defer one to fit. Pick a compliant shape:
- **Batch into ≤4-groups** — for coherent alternatives (e.g. version bumps,
layout variants). One call, 5th surfaced only if first 4 don't fit.
- **Split per-option** — for independent scope items (e.g. "ship E1..E6?").
Fire N sequential calls, one per option. Default to this when unsure.
Per-option call shape: `D<N>.k` header (e.g. D3.1..D3.5), ELI10 per option,
Recommendation, kind-note (no completeness score — Include/Defer/Cut/Hold are
decision actions), and 4 buckets:
**A) Include**, **B) Defer**, **C) Cut**, **D) Hold** (stop chain, discuss).
After the chain, fire `D<N>.final` to validate the assembled set (reprompt
dependency conflicts) and confirm shipping it. Use `D<N>.revise-<k>` to
revise one option without re-running the chain.
For N>6, fire a `D<N>.0` meta-AskUserQuestion first (proceed / narrow / batch).
question_ids for split chains: `<skill>-split-<option-slug>` (kebab-case ASCII,
≤64 chars, `-2`/`-3` suffix on collision). The runtime checker
(`bin/gstack-question-preference`) refuses `never-ask` on any `*-split-*` id,
so split chains are never AUTO_DECIDE-eligible — the user's option set is sacred.
**Full rule + worked examples + Hold/dependency semantics:** see
`docs/askuserquestion-split.md` in the gstack repo. Read on demand when N>4.
**Non-ASCII characters — write directly, never \u-escape.** When any
string field (question, option label, option description) contains
Chinese (繁體/簡體), Japanese, Korean, or other non-ASCII text, emit
the literal UTF-8 characters in the JSON string. **Never escape them
@@ -343,6 +386,9 @@ Before calling AskUserQuestion, verify:
- [ ] Net line closes the decision
- [ ] You are calling the tool, not writing prose
- [ ] Non-ASCII characters (CJK / accents) written directly, NOT \u-escaped
- [ ] If you had 5+ options, you split (or batched into ≤4-groups) — did NOT drop any
- [ ] If you split, you checked dependencies between options before firing the chain
- [ ] If a per-option Hold fires, you stopped the chain immediately (didn't queue)
## Artifacts Sync (skill start)
@@ -2536,6 +2582,39 @@ you missed it.>
<If no plan file: "No plan file detected.">
<If plan items deferred: list deferred items>
## Linked Spec
<Auto-detect: look for /spec archives matching this branch via:
eval "$(${ctx.paths.binDir}/gstack-paths)"
eval "$(${ctx.paths.binDir}/gstack-slug)"
CURRENT_BRANCH=$(git branch --show-current)
SPEC_ARCHIVES="$GSTACK_STATE_ROOT/projects/$SLUG/specs"
# Find newest archive whose spec_branch frontmatter matches current branch (or one of its
# parents — if spec spawned worktree spec/<slug>-$$, the spawned worktree IS where /ship runs).
SPEC_FILE=$(grep -l "^spec_branch: $CURRENT_BRANCH$" "$SPEC_ARCHIVES"/*.md 2>/dev/null | head -1)
[ -z "$SPEC_FILE" ] && exit # no spec; omit this section entirely
SPEC_ISSUE=$(grep "^spec_issue_number:" "$SPEC_FILE" | cut -d' ' -f2)
[ -z "$SPEC_ISSUE" ] && exit # spec archive exists but no issue number; omit
# CONDITIONAL Closes #N (codex F4): only add when Plan Completion above is "complete".
# If the plan completion gate from Step 8 reports any deferred or failed items, emit:
# "Linked to #$SPEC_ISSUE (partial delivery — NOT auto-closing; close manually after follow-up)"
# If Plan Completion is fully complete, emit:
# "Closes #$SPEC_ISSUE"
# and include the Closes #N line in the PR body so GitHub auto-closes on merge.>
<Format:
Closes #<N>
This PR delivers the spec at <archive path relative to repo root>.
Spec filed: <spec_filed_at from frontmatter>>
<If partial delivery, emit instead:
Linked to #<N> (partial delivery — not auto-closing).
Deferred items: <list from Plan Completion>.
Close #<N> manually after follow-up lands.>
<If no /spec archive matches this branch: omit this entire section.>
## Verification Results
<If verification ran: summary from Step 8.1 (N PASS, M FAIL, K SKIPPED)>
<If skipped: reason (no plan, no server, no verification section)>
+80 -1
View File
@@ -95,6 +95,19 @@ _CHECKPOINT_MODE=$($GSTACK_BIN/gstack-config get checkpoint_mode 2>/dev/null ||
_CHECKPOINT_PUSH=$($GSTACK_BIN/gstack-config get checkpoint_push 2>/dev/null || echo "false")
echo "CHECKPOINT_MODE: $_CHECKPOINT_MODE"
echo "CHECKPOINT_PUSH: $_CHECKPOINT_PUSH"
# Plan-mode hint for skills like /spec that branch behavior on plan-mode state.
# Claude Code exposes plan mode via system reminders; we detect best-effort
# from CLAUDE_PLAN_FILE (set by the harness when plan mode is active) and
# fall back to "inactive". Codex hosts and Claude execution mode both end up
# inactive, which is the safe default (defaults to file+execute pipeline).
if [ -n "${CLAUDE_PLAN_FILE:-}${GSTACK_PLAN_MODE_FORCE:-}" ]; then
export GSTACK_PLAN_MODE="active"
elif [ "${GSTACK_PLAN_MODE:-}" = "active" ]; then
export GSTACK_PLAN_MODE="active"
else
export GSTACK_PLAN_MODE="inactive"
fi
echo "GSTACK_PLAN_MODE: $GSTACK_PLAN_MODE"
[ -n "$OPENCLAW_SESSION" ] && echo "SPAWNED_SESSION: true" || true
```
@@ -226,6 +239,7 @@ Key routing rules:
- Ship/deploy/PR → invoke /ship or /land-and-deploy
- Save progress → invoke /context-save
- Resume context → invoke /context-restore
- Author a backlog-ready spec/issue → invoke /spec
```
Then commit the change: `git add CLAUDE.md && git commit -m "chore: add gstack skill routing rules to CLAUDE.md"`
@@ -312,7 +326,36 @@ Effort both-scales: when an option involves effort, label both human-team and CC
Net line closes the tradeoff. Per-skill instructions may add stricter rules.
12. **Non-ASCII characters — write directly, never \u-escape.** When any
### Handling 5+ options — split, never drop
AskUserQuestion caps every call at **4 options**. With 5+ real options, NEVER
drop, merge, or silently defer one to fit. Pick a compliant shape:
- **Batch into ≤4-groups** — for coherent alternatives (e.g. version bumps,
layout variants). One call, 5th surfaced only if first 4 don't fit.
- **Split per-option** — for independent scope items (e.g. "ship E1..E6?").
Fire N sequential calls, one per option. Default to this when unsure.
Per-option call shape: `D<N>.k` header (e.g. D3.1..D3.5), ELI10 per option,
Recommendation, kind-note (no completeness score — Include/Defer/Cut/Hold are
decision actions), and 4 buckets:
**A) Include**, **B) Defer**, **C) Cut**, **D) Hold** (stop chain, discuss).
After the chain, fire `D<N>.final` to validate the assembled set (reprompt
dependency conflicts) and confirm shipping it. Use `D<N>.revise-<k>` to
revise one option without re-running the chain.
For N>6, fire a `D<N>.0` meta-AskUserQuestion first (proceed / narrow / batch).
question_ids for split chains: `<skill>-split-<option-slug>` (kebab-case ASCII,
≤64 chars, `-2`/`-3` suffix on collision). The runtime checker
(`bin/gstack-question-preference`) refuses `never-ask` on any `*-split-*` id,
so split chains are never AUTO_DECIDE-eligible — the user's option set is sacred.
**Full rule + worked examples + Hold/dependency semantics:** see
`docs/askuserquestion-split.md` in the gstack repo. Read on demand when N>4.
**Non-ASCII characters — write directly, never \u-escape.** When any
string field (question, option label, option description) contains
Chinese (繁體/簡體), Japanese, Korean, or other non-ASCII text, emit
the literal UTF-8 characters in the JSON string. **Never escape them
@@ -345,6 +388,9 @@ Before calling AskUserQuestion, verify:
- [ ] Net line closes the decision
- [ ] You are calling the tool, not writing prose
- [ ] Non-ASCII characters (CJK / accents) written directly, NOT \u-escaped
- [ ] If you had 5+ options, you split (or batched into ≤4-groups) — did NOT drop any
- [ ] If you split, you checked dependencies between options before firing the chain
- [ ] If a per-option Hold fires, you stopped the chain immediately (didn't queue)
## Artifacts Sync (skill start)
@@ -2914,6 +2960,39 @@ you missed it.>
<If no plan file: "No plan file detected.">
<If plan items deferred: list deferred items>
## Linked Spec
<Auto-detect: look for /spec archives matching this branch via:
eval "$(${ctx.paths.binDir}/gstack-paths)"
eval "$(${ctx.paths.binDir}/gstack-slug)"
CURRENT_BRANCH=$(git branch --show-current)
SPEC_ARCHIVES="$GSTACK_STATE_ROOT/projects/$SLUG/specs"
# Find newest archive whose spec_branch frontmatter matches current branch (or one of its
# parents — if spec spawned worktree spec/<slug>-$$, the spawned worktree IS where /ship runs).
SPEC_FILE=$(grep -l "^spec_branch: $CURRENT_BRANCH$" "$SPEC_ARCHIVES"/*.md 2>/dev/null | head -1)
[ -z "$SPEC_FILE" ] && exit # no spec; omit this section entirely
SPEC_ISSUE=$(grep "^spec_issue_number:" "$SPEC_FILE" | cut -d' ' -f2)
[ -z "$SPEC_ISSUE" ] && exit # spec archive exists but no issue number; omit
# CONDITIONAL Closes #N (codex F4): only add when Plan Completion above is "complete".
# If the plan completion gate from Step 8 reports any deferred or failed items, emit:
# "Linked to #$SPEC_ISSUE (partial delivery — NOT auto-closing; close manually after follow-up)"
# If Plan Completion is fully complete, emit:
# "Closes #$SPEC_ISSUE"
# and include the Closes #N line in the PR body so GitHub auto-closes on merge.>
<Format:
Closes #<N>
This PR delivers the spec at <archive path relative to repo root>.
Spec filed: <spec_filed_at from frontmatter>>
<If partial delivery, emit instead:
Linked to #<N> (partial delivery — not auto-closing).
Deferred items: <list from Plan Completion>.
Close #<N> manually after follow-up lands.>
<If no /spec archive matches this branch: omit this entire section.>
## Verification Results
<If verification ran: summary from Step 8.1 (N PASS, M FAIL, K SKIPPED)>
<If skipped: reason (no plan, no server, no verification section)>
+633
View File
@@ -0,0 +1,633 @@
{
"tag": "v1.47.0.0",
"capturedAt": "2026-05-27T05:50:57.656Z",
"capturedFromCommit": "e08e5fa8",
"capturedFromBranch": "garrytan/askuserquestion-split-on-overflow",
"totalSkills": 52,
"totalCorpusBytes": 3090887,
"estTotalCatalogTokens": 4116,
"topHeaviest": [
{
"skill": "ship",
"skillMdBytes": 166782,
"skillMdLines": 3099,
"estTokens": 41696,
"tmplBytes": 50495,
"descriptionLen": 291,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "plan-ceo-review",
"skillMdBytes": 132488,
"skillMdLines": 2197,
"estTokens": 33122,
"tmplBytes": 63393,
"descriptionLen": 794,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "office-hours",
"skillMdBytes": 112842,
"skillMdLines": 2066,
"estTokens": 28211,
"tmplBytes": 55466,
"descriptionLen": 860,
"hasGateEval": true,
"hasPeriodicEval": false
},
{
"skill": "plan-design-review",
"skillMdBytes": 107855,
"skillMdLines": 1928,
"estTokens": 26964,
"tmplBytes": 28624,
"descriptionLen": 218,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "plan-devex-review",
"skillMdBytes": 106167,
"skillMdLines": 2119,
"estTokens": 26542,
"tmplBytes": 35680,
"descriptionLen": 250,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "plan-eng-review",
"skillMdBytes": 103009,
"skillMdLines": 1762,
"estTokens": 25752,
"tmplBytes": 26234,
"descriptionLen": 231,
"hasGateEval": true,
"hasPeriodicEval": true
},
{
"skill": "spec",
"skillMdBytes": 102629,
"skillMdLines": 2141,
"estTokens": 25657,
"tmplBytes": 28429,
"descriptionLen": 282,
"hasGateEval": true,
"hasPeriodicEval": false
},
{
"skill": "design-review",
"skillMdBytes": 95654,
"skillMdLines": 1932,
"estTokens": 23914,
"tmplBytes": 11674,
"descriptionLen": 304,
"hasGateEval": true,
"hasPeriodicEval": false
},
{
"skill": "review",
"skillMdBytes": 94048,
"skillMdLines": 1762,
"estTokens": 23512,
"tmplBytes": 14099,
"descriptionLen": 205,
"hasGateEval": true,
"hasPeriodicEval": false
},
{
"skill": "land-and-deploy",
"skillMdBytes": 91886,
"skillMdLines": 1856,
"estTokens": 22972,
"tmplBytes": 48624,
"descriptionLen": 160,
"hasGateEval": true,
"hasPeriodicEval": false
}
],
"skills": {
"autoplan": {
"skill": "autoplan",
"skillMdBytes": 90870,
"skillMdLines": 1784,
"estTokens": 22718,
"tmplBytes": 45271,
"descriptionLen": 366,
"hasGateEval": true,
"hasPeriodicEval": true
},
"benchmark": {
"skill": "benchmark",
"skillMdBytes": 33266,
"skillMdLines": 747,
"estTokens": 8317,
"tmplBytes": 9378,
"descriptionLen": 213,
"hasGateEval": true,
"hasPeriodicEval": false
},
"benchmark-models": {
"skill": "benchmark-models",
"skillMdBytes": 29333,
"skillMdLines": 622,
"estTokens": 7333,
"tmplBytes": 6631,
"descriptionLen": 217,
"hasGateEval": false,
"hasPeriodicEval": false
},
"browse": {
"skill": "browse",
"skillMdBytes": 48018,
"skillMdLines": 929,
"estTokens": 12005,
"tmplBytes": 10805,
"descriptionLen": 181,
"hasGateEval": true,
"hasPeriodicEval": false
},
"canary": {
"skill": "canary",
"skillMdBytes": 47105,
"skillMdLines": 990,
"estTokens": 11776,
"tmplBytes": 8033,
"descriptionLen": 180,
"hasGateEval": true,
"hasPeriodicEval": false
},
"careful": {
"skill": "careful",
"skillMdBytes": 2551,
"skillMdLines": 68,
"estTokens": 638,
"tmplBytes": 2435,
"descriptionLen": 315,
"hasGateEval": false,
"hasPeriodicEval": false
},
"codex": {
"skill": "codex",
"skillMdBytes": 79620,
"skillMdLines": 1519,
"estTokens": 19905,
"tmplBytes": 34143,
"descriptionLen": 187,
"hasGateEval": true,
"hasPeriodicEval": false
},
"context-restore": {
"skill": "context-restore",
"skillMdBytes": 41493,
"skillMdLines": 848,
"estTokens": 10373,
"tmplBytes": 5255,
"descriptionLen": 238,
"hasGateEval": true,
"hasPeriodicEval": false
},
"context-save": {
"skill": "context-save",
"skillMdBytes": 45690,
"skillMdLines": 966,
"estTokens": 11423,
"tmplBytes": 9293,
"descriptionLen": 168,
"hasGateEval": true,
"hasPeriodicEval": false
},
"cso": {
"skill": "cso",
"skillMdBytes": 77397,
"skillMdLines": 1451,
"estTokens": 19349,
"tmplBytes": 35158,
"descriptionLen": 196,
"hasGateEval": true,
"hasPeriodicEval": false
},
"design-consultation": {
"skill": "design-consultation",
"skillMdBytes": 79222,
"skillMdLines": 1561,
"estTokens": 19806,
"tmplBytes": 25899,
"descriptionLen": 888,
"hasGateEval": true,
"hasPeriodicEval": false
},
"design-html": {
"skill": "design-html",
"skillMdBytes": 66547,
"skillMdLines": 1449,
"estTokens": 16637,
"tmplBytes": 22567,
"descriptionLen": 233,
"hasGateEval": false,
"hasPeriodicEval": false
},
"design-review": {
"skill": "design-review",
"skillMdBytes": 95654,
"skillMdLines": 1932,
"estTokens": 23914,
"tmplBytes": 11674,
"descriptionLen": 304,
"hasGateEval": true,
"hasPeriodicEval": false
},
"design-shotgun": {
"skill": "design-shotgun",
"skillMdBytes": 62836,
"skillMdLines": 1311,
"estTokens": 15709,
"tmplBytes": 13331,
"descriptionLen": 786,
"hasGateEval": false,
"hasPeriodicEval": false
},
"devex-review": {
"skill": "devex-review",
"skillMdBytes": 64413,
"skillMdLines": 1233,
"estTokens": 16103,
"tmplBytes": 7984,
"descriptionLen": 201,
"hasGateEval": false,
"hasPeriodicEval": false
},
"document-generate": {
"skill": "document-generate",
"skillMdBytes": 52987,
"skillMdLines": 1176,
"estTokens": 13247,
"tmplBytes": 15093,
"descriptionLen": 334,
"hasGateEval": false,
"hasPeriodicEval": false
},
"document-release": {
"skill": "document-release",
"skillMdBytes": 58251,
"skillMdLines": 1235,
"estTokens": 14563,
"tmplBytes": 20362,
"descriptionLen": 192,
"hasGateEval": true,
"hasPeriodicEval": false
},
"freeze": {
"skill": "freeze",
"skillMdBytes": 3154,
"skillMdLines": 92,
"estTokens": 789,
"tmplBytes": 3038,
"descriptionLen": 503,
"hasGateEval": false,
"hasPeriodicEval": false
},
"gstack-upgrade": {
"skill": "gstack-upgrade",
"skillMdBytes": 10817,
"skillMdLines": 285,
"estTokens": 2704,
"tmplBytes": 10667,
"descriptionLen": 163,
"hasGateEval": true,
"hasPeriodicEval": false
},
"guard": {
"skill": "guard",
"skillMdBytes": 3297,
"skillMdLines": 91,
"estTokens": 824,
"tmplBytes": 3181,
"descriptionLen": 686,
"hasGateEval": false,
"hasPeriodicEval": false
},
"health": {
"skill": "health",
"skillMdBytes": 47916,
"skillMdLines": 1014,
"estTokens": 11979,
"tmplBytes": 11617,
"descriptionLen": 184,
"hasGateEval": true,
"hasPeriodicEval": false
},
"investigate": {
"skill": "investigate",
"skillMdBytes": 50409,
"skillMdLines": 1012,
"estTokens": 12602,
"tmplBytes": 11561,
"descriptionLen": 1379,
"hasGateEval": true,
"hasPeriodicEval": false
},
"ios-clean": {
"skill": "ios-clean",
"skillMdBytes": 41045,
"skillMdLines": 813,
"estTokens": 10261,
"tmplBytes": 3851,
"descriptionLen": 252,
"hasGateEval": false,
"hasPeriodicEval": false
},
"ios-design-review": {
"skill": "ios-design-review",
"skillMdBytes": 41631,
"skillMdLines": 815,
"estTokens": 10408,
"tmplBytes": 4417,
"descriptionLen": 209,
"hasGateEval": false,
"hasPeriodicEval": false
},
"ios-fix": {
"skill": "ios-fix",
"skillMdBytes": 40760,
"skillMdLines": 811,
"estTokens": 10190,
"tmplBytes": 3574,
"descriptionLen": 187,
"hasGateEval": false,
"hasPeriodicEval": false
},
"ios-qa": {
"skill": "ios-qa",
"skillMdBytes": 47271,
"skillMdLines": 931,
"estTokens": 11818,
"tmplBytes": 10090,
"descriptionLen": 223,
"hasGateEval": true,
"hasPeriodicEval": false
},
"ios-sync": {
"skill": "ios-sync",
"skillMdBytes": 40737,
"skillMdLines": 804,
"estTokens": 10184,
"tmplBytes": 3544,
"descriptionLen": 269,
"hasGateEval": false,
"hasPeriodicEval": false
},
"land-and-deploy": {
"skill": "land-and-deploy",
"skillMdBytes": 91886,
"skillMdLines": 1856,
"estTokens": 22972,
"tmplBytes": 48624,
"descriptionLen": 160,
"hasGateEval": true,
"hasPeriodicEval": false
},
"landing-report": {
"skill": "landing-report",
"skillMdBytes": 43985,
"skillMdLines": 874,
"estTokens": 10996,
"tmplBytes": 6806,
"descriptionLen": 195,
"hasGateEval": false,
"hasPeriodicEval": false
},
"learn": {
"skill": "learn",
"skillMdBytes": 41722,
"skillMdLines": 891,
"estTokens": 10431,
"tmplBytes": 5594,
"descriptionLen": 178,
"hasGateEval": true,
"hasPeriodicEval": false
},
"make-pdf": {
"skill": "make-pdf",
"skillMdBytes": 29450,
"skillMdLines": 663,
"estTokens": 7363,
"tmplBytes": 5106,
"descriptionLen": 177,
"hasGateEval": false,
"hasPeriodicEval": false
},
"office-hours": {
"skill": "office-hours",
"skillMdBytes": 112842,
"skillMdLines": 2066,
"estTokens": 28211,
"tmplBytes": 55466,
"descriptionLen": 860,
"hasGateEval": true,
"hasPeriodicEval": false
},
"open-gstack-browser": {
"skill": "open-gstack-browser",
"skillMdBytes": 46131,
"skillMdLines": 954,
"estTokens": 11533,
"tmplBytes": 7702,
"descriptionLen": 204,
"hasGateEval": false,
"hasPeriodicEval": false
},
"pair-agent": {
"skill": "pair-agent",
"skillMdBytes": 46939,
"skillMdLines": 1010,
"estTokens": 11735,
"tmplBytes": 8548,
"descriptionLen": 167,
"hasGateEval": false,
"hasPeriodicEval": false
},
"plan-ceo-review": {
"skill": "plan-ceo-review",
"skillMdBytes": 132488,
"skillMdLines": 2197,
"estTokens": 33122,
"tmplBytes": 63393,
"descriptionLen": 794,
"hasGateEval": true,
"hasPeriodicEval": true
},
"plan-design-review": {
"skill": "plan-design-review",
"skillMdBytes": 107855,
"skillMdLines": 1928,
"estTokens": 26964,
"tmplBytes": 28624,
"descriptionLen": 218,
"hasGateEval": true,
"hasPeriodicEval": true
},
"plan-devex-review": {
"skill": "plan-devex-review",
"skillMdBytes": 106167,
"skillMdLines": 2119,
"estTokens": 26542,
"tmplBytes": 35680,
"descriptionLen": 250,
"hasGateEval": true,
"hasPeriodicEval": true
},
"plan-eng-review": {
"skill": "plan-eng-review",
"skillMdBytes": 103009,
"skillMdLines": 1762,
"estTokens": 25752,
"tmplBytes": 26234,
"descriptionLen": 231,
"hasGateEval": true,
"hasPeriodicEval": true
},
"plan-tune": {
"skill": "plan-tune",
"skillMdBytes": 51717,
"skillMdLines": 1077,
"estTokens": 12929,
"tmplBytes": 15586,
"descriptionLen": 325,
"hasGateEval": true,
"hasPeriodicEval": false
},
"qa": {
"skill": "qa",
"skillMdBytes": 73863,
"skillMdLines": 1622,
"estTokens": 18466,
"tmplBytes": 12701,
"descriptionLen": 218,
"hasGateEval": true,
"hasPeriodicEval": false
},
"qa-only": {
"skill": "qa-only",
"skillMdBytes": 56421,
"skillMdLines": 1194,
"estTokens": 14105,
"tmplBytes": 3851,
"descriptionLen": 165,
"hasGateEval": true,
"hasPeriodicEval": false
},
"retro": {
"skill": "retro",
"skillMdBytes": 82889,
"skillMdLines": 1750,
"estTokens": 20722,
"tmplBytes": 42427,
"descriptionLen": 648,
"hasGateEval": true,
"hasPeriodicEval": false
},
"review": {
"skill": "review",
"skillMdBytes": 94048,
"skillMdLines": 1762,
"estTokens": 23512,
"tmplBytes": 14099,
"descriptionLen": 205,
"hasGateEval": true,
"hasPeriodicEval": false
},
"scrape": {
"skill": "scrape",
"skillMdBytes": 43641,
"skillMdLines": 887,
"estTokens": 10910,
"tmplBytes": 5220,
"descriptionLen": 167,
"hasGateEval": true,
"hasPeriodicEval": false
},
"setup-browser-cookies": {
"skill": "setup-browser-cookies",
"skillMdBytes": 26618,
"skillMdLines": 594,
"estTokens": 6655,
"tmplBytes": 2724,
"descriptionLen": 222,
"hasGateEval": false,
"hasPeriodicEval": false
},
"setup-deploy": {
"skill": "setup-deploy",
"skillMdBytes": 43927,
"skillMdLines": 919,
"estTokens": 10982,
"tmplBytes": 7780,
"descriptionLen": 197,
"hasGateEval": true,
"hasPeriodicEval": false
},
"setup-gbrain": {
"skill": "setup-gbrain",
"skillMdBytes": 78394,
"skillMdLines": 1704,
"estTokens": 19599,
"tmplBytes": 42245,
"descriptionLen": 323,
"hasGateEval": true,
"hasPeriodicEval": false
},
"ship": {
"skill": "ship",
"skillMdBytes": 166782,
"skillMdLines": 3099,
"estTokens": 41696,
"tmplBytes": 50495,
"descriptionLen": 291,
"hasGateEval": true,
"hasPeriodicEval": true
},
"skillify": {
"skill": "skillify",
"skillMdBytes": 53534,
"skillMdLines": 1168,
"estTokens": 13384,
"tmplBytes": 15107,
"descriptionLen": 233,
"hasGateEval": true,
"hasPeriodicEval": false
},
"spec": {
"skill": "spec",
"skillMdBytes": 102629,
"skillMdLines": 2141,
"estTokens": 25657,
"tmplBytes": 28429,
"descriptionLen": 282,
"hasGateEval": true,
"hasPeriodicEval": false
},
"sync-gbrain": {
"skill": "sync-gbrain",
"skillMdBytes": 50156,
"skillMdLines": 1028,
"estTokens": 12539,
"tmplBytes": 13996,
"descriptionLen": 299,
"hasGateEval": false,
"hasPeriodicEval": false
},
"unfreeze": {
"skill": "unfreeze",
"skillMdBytes": 1504,
"skillMdLines": 49,
"estTokens": 376,
"tmplBytes": 1386,
"descriptionLen": 199,
"hasGateEval": false,
"hasPeriodicEval": false
}
}
}
+59
View File
@@ -103,6 +103,65 @@ describe('--check with preferences set', () => {
});
});
// Split-chain carve-out: question_ids matching <skill>-split-<option-slug>
// must always ASK_NORMALLY regardless of stored preferences.
// See scripts/resolvers/preamble/generate-ask-user-format.ts
// "Handling 5+ options — split, never drop" for the surrounding mechanism.
describe('--check split-chain carve-out (*-split-* always ASK_NORMALLY)', () => {
function setPref(id: string, pref: string) {
return run('--write', JSON.stringify({ question_id: id, preference: pref, source: 'plan-tune' }));
}
test('split-id without preference → ASK_NORMALLY', () => {
const r = run('--check', 'plan-ceo-review-split-e4-detect-mappings');
expect(r.stdout.trim()).toContain('ASK_NORMALLY');
});
test('split-id + never-ask → ASK_NORMALLY (carve-out overrides preference)', () => {
setPref('plan-ceo-review-split-e4-detect-mappings', 'never-ask');
const r = run('--check', 'plan-ceo-review-split-e4-detect-mappings');
expect(r.stdout).toContain('ASK_NORMALLY');
expect(r.stdout).not.toContain('AUTO_DECIDE');
});
test('split-id + never-ask → emits explanatory note', () => {
setPref('plan-ceo-review-split-e4-detect-mappings', 'never-ask');
const r = run('--check', 'plan-ceo-review-split-e4-detect-mappings');
expect(r.stdout).toContain('split-chain per-option calls always ASK_NORMALLY');
expect(r.stdout).toContain('never-ask');
});
test('split-id + ask-only-for-one-way → ASK_NORMALLY (carve-out overrides preference)', () => {
setPref('ship-split-version-bump', 'ask-only-for-one-way');
const r = run('--check', 'ship-split-version-bump');
expect(r.stdout).toContain('ASK_NORMALLY');
expect(r.stdout).not.toContain('AUTO_DECIDE');
});
test('split-id + always-ask → ASK_NORMALLY (no note since preference agrees)', () => {
setPref('plan-eng-review-split-add-test', 'always-ask');
const r = run('--check', 'plan-eng-review-split-add-test');
expect(r.stdout.trim()).toContain('ASK_NORMALLY');
expect(r.stdout).not.toContain('does not apply');
});
test('non-split id that just happens to contain "split" word is NOT carved out', () => {
// The carve-out matches `-split-` (kebab-cased), not the substring "split".
// A question id like `qa-splitscreen-test` (hypothetical) would not match.
// Verify by using a never-ask pref that should fire AUTO_DECIDE.
setPref('qa-splitscreen-test', 'never-ask');
const r = run('--check', 'qa-splitscreen-test');
expect(r.stdout.trim()).toContain('AUTO_DECIDE');
});
test('multiple split-id formats: skill-split-anything matches', () => {
setPref('autoplan-split-ceo-finding-7', 'never-ask');
const r = run('--check', 'autoplan-split-ceo-finding-7');
expect(r.stdout).toContain('ASK_NORMALLY');
expect(r.stdout).not.toContain('AUTO_DECIDE');
});
});
// -----------------------------------------------------------------------
// --write
// -----------------------------------------------------------------------
+2
View File
@@ -149,6 +149,7 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
// confirm" plan write. runPlanSkillFloorCheck cannot detect that shape
// (it exits on first AUQ); runPlanSkillCounting can.
'plan-eng-multi-finding-batching': ['plan-eng-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-eng-multi-finding-batching.test.ts'],
'plan-ceo-split-overflow': ['plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'bin/gstack-question-preference', 'test/helpers/claude-pty-runner.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-ceo-split-overflow.test.ts'],
'brain-privacy-gate': ['scripts/resolvers/preamble/generate-brain-sync-block.ts', 'scripts/resolvers/preamble.ts', 'bin/gstack-brain-sync', 'bin/gstack-artifacts-init', 'bin/gstack-config', 'test/helpers/agent-sdk-runner.ts'],
// /setup-gbrain Path 4 (Remote MCP) — happy + bad-token end-to-end via
@@ -479,6 +480,7 @@ export const E2E_TIERS: Record<string, 'gate' | 'periodic'> = {
'plan-design-finding-floor': 'gate',
'plan-devex-finding-floor': 'gate',
'plan-eng-multi-finding-batching': 'periodic',
'plan-ceo-split-overflow': 'periodic',
// Privacy gate for gstack-brain-sync — periodic (non-deterministic LLM call,
// costs ~$0.30-$0.50 per run, not needed on every commit)
+42
View File
@@ -119,3 +119,45 @@ describe('generateAskUserFormat — v1.7.0.0 Pros/Cons format', () => {
expect(out).toMatch(/Per-skill instructions may add/);
});
});
describe('generateAskUserFormat — 5+ option split rule (slim inline + docs pointer)', () => {
const out = generateAskUserFormat(makeCtx());
// 5 highest-signal pins. The full rule lives in
// docs/askuserquestion-split.md; this contract only checks what the
// inline subsection MUST surface so the agent can act without
// reading the docs file for routine 5-option splits.
test('forbids dropping options to fit the 4-option cap', () => {
expect(out).toMatch(/caps every call at \*\*4 options\*\*/);
expect(out).toMatch(/NEVER\s+drop, merge, or silently defer/);
});
test('names the Include / Defer / Cut / Hold buckets', () => {
expect(out).toMatch(/A\) Include/);
expect(out).toMatch(/B\) Defer/);
expect(out).toMatch(/C\) Cut/);
expect(out).toMatch(/D\) Hold/);
});
test('specifies D<N>.k child numbering and D<N>.final summary', () => {
expect(out).toContain('D<N>.k');
expect(out).toContain('D<N>.final');
});
test('AUTO_DECIDE is gated at runtime, not just collision-resistance', () => {
expect(out).toContain('bin/gstack-question-preference');
expect(out).toContain('*-split-*');
expect(out).toContain('never AUTO_DECIDE-eligible');
});
test('points to docs/askuserquestion-split.md for the full rule', () => {
expect(out).toContain('docs/askuserquestion-split.md');
expect(out).toMatch(/Read on demand when N>4/);
});
test('regression: orphan "12." prefix removed from CJK rule', () => {
expect(out).not.toContain('12. **Non-ASCII');
expect(out).toContain('**Non-ASCII characters');
});
});
@@ -0,0 +1,108 @@
/**
* /plan-ceo-review split-overflow regression (periodic, paid, real-PTY).
*
* Catches the original failure mode the user complained about: when the
* agent has 5+ options for ONE conceptual decision, it must split into N
* sequential AskUserQuestion calls (or batch into compatible 4-groups),
* NOT drop an option arbitrarily to fit Conductor's 4-option cap.
*
* Pre-fix reasoning trace from the user transcript that motivated this:
* "I'm hitting Conductor's limit of 4 options in the AUQ, so I need
* to cut one. E4 is the largest lift and probably beyond scope...
* Trimming: E4. Moving to TODOs without asking. Re-firing with 4."
*
* The fixture seeds 5 independent scope candidates (chat-platform
* integrations) each carries an independent include/defer/cut decision.
* With the split rule active, the natural compliant shape is a per-option
* chain at parent D<N>; the test asserts the agent fires at least
* [N-1] review-phase AUQs (standard tolerance band from the existing
* finding-count tests, which accounts for one expected scope-reduction
* call before the per-option chain begins).
*
* Why a separate test from skill-e2e-plan-ceo-finding-count and
* skill-e2e-plan-eng-multi-finding-batching:
* - finding-count tests fire one AUQ per finding (Architecture, Code
* Quality, etc) they exercise the "one issue per call" rule, not
* the "5+ options for ONE decision" split rule.
* - This test fixtures ONE scope decision with 5 options inside it,
* which is exactly the shape that hits Conductor's 4-option cap and
* triggers the new split-vs-drop guidance.
*
* Tier: periodic (~25 min, ~$0.30-$5.00/run depending on agent path).
* Sequential by default.
*/
import { describe, test } from 'bun:test';
import * as fs from 'node:fs';
import {
runPlanSkillCounting,
ceoStep0Boundary,
} from './helpers/claude-pty-runner';
import { FORCING_SPLIT_OVERFLOW_CEO } from './fixtures/forcing-finding-seeds';
const shouldRun = !!process.env.EVALS && process.env.EVALS_TIER === 'periodic';
const describeE2E = shouldRun ? describe : describe.skip;
const N = 5;
const FLOOR = N - 1; // 4 — must fire at least one AUQ per non-dropped option
const PLAN_PATH = '/tmp/gstack-test-plan-ceo-split-overflow.md';
describeE2E('/plan-ceo-review split-overflow regression (periodic)', () => {
test(
`5-option scope decision emits >= ${FLOOR} review-phase AskUserQuestions (no dropping)`,
async () => {
try {
fs.rmSync(PLAN_PATH, { force: true });
} catch {
/* best-effort */
}
const obs = await runPlanSkillCounting({
skillName: 'plan-ceo-review',
slashCommand: '/plan-ceo-review',
followUpPrompt: FORCING_SPLIT_OVERFLOW_CEO,
isLastStep0AUQ: ceoStep0Boundary,
reviewCountCeiling: N + 3, // hard cap above floor + tolerance
cwd: process.cwd(),
timeoutMs: 1_500_000, // 25 min
env: { QUESTION_TUNING: 'false', EXPLAIN_LEVEL: 'default' },
});
try {
if (!['plan_ready', 'completion_summary', 'ceiling_reached'].includes(obs.outcome)) {
throw new Error(
`split-overflow test FAILED: outcome=${obs.outcome}\n` +
`step0=${obs.step0Count} review=${obs.reviewCount} elapsed=${obs.elapsedMs}ms\n` +
`--- evidence (last 3KB) ---\n${obs.evidence}`,
);
}
if (obs.reviewCount < FLOOR) {
throw new Error(
`SPLIT-OVERFLOW REGRESSION: reviewCount=${obs.reviewCount} < FLOOR=${FLOOR}.\n` +
`Agent surfaced fewer review-phase AUQs than independent scope options.\n` +
`This is the original drop-to-fit-4-options failure mode:\n` +
` expected: ${N} per-option calls (or compliant ≤4-group batching with follow-up)\n` +
` got: ${obs.reviewCount} call(s)\n` +
`Most likely the agent dropped one option to fit Conductor's 4-option\n` +
`cap, the exact bug scripts/resolvers/preamble/generate-ask-user-format.ts\n` +
`"Handling 5+ options — split, never drop" exists to prevent.\n` +
`Review-phase fingerprints:\n` +
obs.fingerprints
.filter((f) => !f.preReview)
.map((f) => ` - "${f.promptSnippet.slice(0, 80)}"`)
.join('\n') +
`\n--- evidence (last 3KB) ---\n${obs.evidence}`,
);
}
} finally {
try {
fs.rmSync(PLAN_PATH, { force: true });
} catch {
/* best-effort */
}
}
},
1_700_000,
);
});
+16 -11
View File
@@ -1,15 +1,20 @@
/**
* Per-skill SKILL.md size budget regression (v1.46.0.0 T5).
*
* Asserts that no skill's generated SKILL.md grew beyond the v1.44.1
* Asserts that no skill's generated SKILL.md grew beyond the v1.47.0.0
* baseline. Catches preamble/resolver changes that bloat skills back to
* the pre-compression size. Free pure file IO + JSON diff.
*
* Baseline rebased v1.44.1 v1.47.0.0 in the AskUserQuestion split-rule
* PR after main merged GSTACK_PLAN_MODE + /spec, pushing the v1.44.1
* anchor past the 5% ratchet. Historical v1.44.1.json and v1.46.0.0.json
* are retained in test/fixtures/ for reference.
*
* Why a separate test from skill-budget-regression.test.ts: that one
* compares LIVE eval runs (tool calls, turns, cost); this one compares
* static SKILL.md sizes. Both gate-tier.
*
* The baseline lives at test/fixtures/parity-baseline-v1.44.1.json,
* The baseline lives at test/fixtures/parity-baseline-v1.47.0.0.json,
* captured by scripts/capture-baseline.ts before any Phase A work landed.
*
* Override:
@@ -30,7 +35,7 @@ import { captureBaseline, type ParityBaseline } from './helpers/capture-parity-b
import { logBudgetOverride } from './helpers/budget-override';
const REPO_ROOT = path.resolve(import.meta.dir, '..');
const BASELINE_PATH = path.join(REPO_ROOT, 'test', 'fixtures', 'parity-baseline-v1.44.1.json');
const BASELINE_PATH = path.join(REPO_ROOT, 'test', 'fixtures', 'parity-baseline-v1.47.0.0.json');
// Default per-skill ratio is 1.05 (5% growth tolerance). T4 catalog trim
// MOVES text from frontmatter (always-loaded catalog) to a body section
@@ -49,11 +54,11 @@ interface Regression {
}
describe('SKILL.md size budget regression (gate, free)', () => {
test('parity-baseline-v1.44.1.json exists', () => {
test('parity-baseline-v1.47.0.0.json exists', () => {
expect(fs.existsSync(BASELINE_PATH)).toBe(true);
});
test('no skill exceeds v1.44.1 baseline size × ratio', () => {
test('no skill exceeds v1.47.0.0 baseline size × ratio', () => {
const baseline: ParityBaseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf-8'));
const current = captureBaseline({ repoRoot: REPO_ROOT });
@@ -94,7 +99,7 @@ describe('SKILL.md size budget regression (gate, free)', () => {
` ${r.skill}: ${r.beforeBytes}${r.afterBytes} bytes (×${r.growth.toFixed(2)})`,
).join('\n');
throw new Error(
`${regressions.length} skill(s) regressed past v1.44.1 baseline × ${RATIO}:\n${msg}\n` +
`${regressions.length} skill(s) regressed past v1.47.0.0 baseline × ${RATIO}:\n${msg}\n` +
`Override: set GSTACK_SIZE_BUDGET_OVERRIDE_REASON="why this is OK" to allow and audit-log.`,
);
});
@@ -120,7 +125,7 @@ describe('SKILL.md size budget regression (gate, free)', () => {
return;
}
throw new Error(
`Total corpus regressed past v1.44.1 baseline × ${RATIO}: ` +
`Total corpus regressed past v1.47.0.0 baseline × ${RATIO}: ` +
`${baseline.totalCorpusBytes}${current.totalCorpusBytes} bytes (×${ratio.toFixed(3)}). ` +
`Override: set GSTACK_SIZE_BUDGET_OVERRIDE_REASON to allow.`,
);
@@ -130,13 +135,13 @@ describe('SKILL.md size budget regression (gate, free)', () => {
* Gap E (v1.46.0.0): per-skill min-size floor.
*
* The existing skill-coverage-floor enforces body 200 bytes, which is
* a tiny noise floor. A skill that was 100 KB at v1.44.1 and shrinks to
* a tiny noise floor. A skill that was 100 KB at v1.47.0.0 and shrinks to
* 250 bytes passes that check despite losing 99.75% of content. The
* parity-suite content invariants cover this for 10 hand-picked skills
* (cso, ship, plan-ceo, etc.); the remaining 41 skills had no per-skill
* shrinkage floor.
*
* Floor: 80% of the v1.44.1 baseline. v1.46 actual shrinkage is <1% per
* Floor: 80% of the v1.47.0.0 baseline. v1.46 actual shrinkage is <1% per
* skill, so this is a comfortable ceiling that still catches accidental
* mass deletion (e.g., a refactor that strips the body of a skill).
*
@@ -146,7 +151,7 @@ describe('SKILL.md size budget regression (gate, free)', () => {
* skeletons. When that lands, add them to SECTIONS_EXTRACTED so the floor
* relaxes for them.
*/
test('no skill shrinks past 80% of v1.44.1 baseline (catches accidental body strip)', () => {
test('no skill shrinks past 80% of v1.47.0.0 baseline (catches accidental body strip)', () => {
const baseline: ParityBaseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf-8'));
const current = captureBaseline({ repoRoot: REPO_ROOT });
const MIN_RATIO = 0.80; // a skill at <80% of its v1.44 size signals mass-deletion
@@ -187,7 +192,7 @@ describe('SKILL.md size budget regression (gate, free)', () => {
` ${u.skill}: ${u.beforeBytes}${u.afterBytes} bytes (×${u.ratio.toFixed(2)} — below ${MIN_RATIO} floor)`,
).join('\n');
throw new Error(
`${undershoots.length} skill(s) shrunk past v1.44.1 × ${MIN_RATIO} floor:\n${msg}\n` +
`${undershoots.length} skill(s) shrunk past v1.47.0.0 × ${MIN_RATIO} floor:\n${msg}\n` +
`This usually signals accidental body strip (e.g., a resolver returning empty, a ` +
`template losing a section). If the shrinkage is intentional (e.g., the skill moved ` +
`to the sections/ pattern), add it to SECTIONS_EXTRACTED in this test. Override: ` +
+6 -2
View File
@@ -105,8 +105,12 @@ describe('selectTests', () => {
expect(result.selected).toContain('auto-decide-preserved');
// v1.27+ gate-tier reviewCount-floor regression for transcript bug
expect(result.selected).toContain('plan-ceo-finding-floor');
expect(result.selected.length).toBe(21);
expect(result.skipped.length).toBe(Object.keys(E2E_TOUCHFILES).length - 21);
// garrytan/askuserquestion-split-on-overflow: split-overflow periodic
// E2E test also depends on plan-ceo-review/** (5-option scope decision
// regression for the "drop to fit 4 options" failure mode).
expect(result.selected).toContain('plan-ceo-split-overflow');
expect(result.selected.length).toBe(22);
expect(result.skipped.length).toBe(Object.keys(E2E_TOUCHFILES).length - 22);
});
test('global touchfile triggers ALL tests', () => {