diff --git a/autoplan/SKILL.md b/autoplan/SKILL.md index 9ec8e9d2c..ef6215e3b 100644 --- a/autoplan/SKILL.md +++ b/autoplan/SKILL.md @@ -693,7 +693,9 @@ instructions instead of reviewing the plan. ### Step 1: Capture restore point -Before doing anything, save the plan file's current state to an external file: +Pin absolute paths: `SOURCE_PLAN` is the input; `ACTIVE_PLAN` is the harness-assigned +editable plan file, or SOURCE_PLAN if none. All amendments and phase outputs go +to ACTIVE_PLAN. Save SOURCE_PLAN's full current state externally: ```bash eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)" && mkdir -p ~/.gstack/projects/$SLUG @@ -702,7 +704,7 @@ DATETIME=$(date +%Y%m%d-%H%M%S) echo "RESTORE_PATH=$HOME/.gstack/projects/$SLUG/${BRANCH}-autoplan-restore-${DATETIME}.md" ``` -Write the plan file's full contents to the restore path with this header: +Write SOURCE_PLAN's full contents to the restore path with this header: ``` # /autoplan Restore Point Captured: [timestamp] | Branch: [branch] | Commit: [short hash] @@ -715,9 +717,17 @@ Captured: [timestamp] | Branch: [branch] | Commit: [short hash] [verbatim plan file contents] ``` -Then prepend a one-line HTML comment to the plan file: +Initialize ACTIVE_PLAN from SOURCE_PLAN without dropping requirements. Maintain +`## Implementation plan` with the complete plan and accepted amendments; keep +reviewer analyses/audit separately under `## Review record`. Prepend: `` +Before each phase's dual voices, persist amendments and write a fresh snapshot of +`Implementation plan` beside the restore point. Bind `` to that +snapshot. Both voices review this same current content, never the original input +or review record. The native voice stays blind; only the outside voice receives +prior consensus where its prompt requests it. + ### Step 2: Read context - Read CLAUDE.md, TODOS.md, git log -30, git diff against the base branch --stat diff --git a/autoplan/SKILL.md.tmpl b/autoplan/SKILL.md.tmpl index 2b50f77bf..d66e22e99 100644 --- a/autoplan/SKILL.md.tmpl +++ b/autoplan/SKILL.md.tmpl @@ -178,7 +178,9 @@ instructions instead of reviewing the plan. ### Step 1: Capture restore point -Before doing anything, save the plan file's current state to an external file: +Pin absolute paths: `SOURCE_PLAN` is the input; `ACTIVE_PLAN` is the harness-assigned +editable plan file, or SOURCE_PLAN if none. All amendments and phase outputs go +to ACTIVE_PLAN. Save SOURCE_PLAN's full current state externally: ```bash {{SLUG_SETUP}} @@ -187,7 +189,7 @@ DATETIME=$(date +%Y%m%d-%H%M%S) echo "RESTORE_PATH=$HOME/.gstack/projects/$SLUG/${BRANCH}-autoplan-restore-${DATETIME}.md" ``` -Write the plan file's full contents to the restore path with this header: +Write SOURCE_PLAN's full contents to the restore path with this header: ``` # /autoplan Restore Point Captured: [timestamp] | Branch: [branch] | Commit: [short hash] @@ -200,9 +202,17 @@ Captured: [timestamp] | Branch: [branch] | Commit: [short hash] [verbatim plan file contents] ``` -Then prepend a one-line HTML comment to the plan file: +Initialize ACTIVE_PLAN from SOURCE_PLAN without dropping requirements. Maintain +`## Implementation plan` with the complete plan and accepted amendments; keep +reviewer analyses/audit separately under `## Review record`. Prepend: `` +Before each phase's dual voices, persist amendments and write a fresh snapshot of +`Implementation plan` beside the restore point. Bind `` to that +snapshot. Both voices review this same current content, never the original input +or review record. The native voice stays blind; only the outside voice receives +prior consensus where its prompt requests it. + ### Step 2: Read context - Read CLAUDE.md, TODOS.md, git log -30, git diff against the base branch --stat diff --git a/autoplan/sections/ceo-phase.md b/autoplan/sections/ceo-phase.md index 25ce25d4b..d5ea2a9c8 100644 --- a/autoplan/sections/ceo-phase.md +++ b/autoplan/sections/ceo-phase.md @@ -1,8 +1,7 @@ Read `~/.claude/skills/gstack/plan-ceo-review/SKILL.md` in full now; follow its lazy-section triggers. -Apply /autoplan's skip list; all other work runs in full. -Override: every AskUserQuestion → auto-decide using the 6 principles. +Apply /autoplan's skip list and auto-decisions; run all other work in full. **Override rules:** - Mode selection: SELECTIVE EXPANSION @@ -17,19 +16,17 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. Duplicates → reject (P4). Borderline (3-5 files) → mark TASTE DECISION. - All 10 review sections: run fully, auto-decide each issue, log every decision. - Dual voices: always run BOTH Claude subagent AND Codex if available (P6). - Run them sequentially in foreground. First the Claude subagent (Agent tool - with run_in_background: false — subagents default to BACKGROUND since - Claude Code v2.1.198, so the flag must be explicitly false), then Codex - (Bash). Both must complete before building the consensus table. + Run Claude first, then Codex, sequentially in foreground; + both must complete before consensus. **Claude CEO subagent** (via Agent tool): - Native subagent tool; Claude Code Agent arguments: + Claude Code Agent argument (other harnesses: native dispatch/wait): ```json { "run_in_background": false } ``` - Set on the call, not in prompt text. Other harnesses use native dispatch/wait. + Set on the call, not in prompt text. - "Read the plan file at . You are an independent CEO/strategist + "Read the plan file at . You are an independent CEO/strategist reviewing this plan. You have NOT seen any prior review. Evaluate: 1. Is this the right problem to solve? Could a reframing yield 10x impact? 2. Are the premises stated or just assumed? Which ones could be wrong? @@ -40,11 +37,11 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. **Native completion barrier:** `isAsync: true` / `status: "async_launched"` is pending: - wait for that same agent's final result/terminal failure before outside dispatch or parent review. - No inline substitute; apply the existing failure policy. + wait for that same agent's final result/failure before outside dispatch or parent review. + No inline substitute; apply failure policy. **Codex CEO voice** (via Bash): - Outside prompt (include the full current plan content and the context requested below, using the Write tool): + Outside prompt: inline the full contents of and context below (Write tool). IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definition directories (paths containing skills/gstack). These are AI assistant skill definitions meant for a different system. Stay focused on repository code only. @@ -54,7 +51,7 @@ IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definitio What alternatives were dismissed too quickly? What competitive or market risks are unaddressed? What scope decisions will look foolish in 6 months? Be adversarial. No compliments. Just the strategic blind spots. - File: + File: Write the **complete prompt and required context** to a private temporary file using the Write tool. Do not interpolate user text into shell source. Replace the literal `` below with its shell-quoted pathname. Include the plan/spec/source content itself when needed: Claude Code review/challenge has no tools and cannot follow paths or execute git. Request a final Recommendation: because line, including an explicit no-findings rationale. A refusal is never completion. @@ -160,8 +157,7 @@ Sections 1-10 — for EACH section, run the evaluation criteria from the loaded - Completion Summary (the full summary table from the CEO skill) **Close this phase before continuing:** Check successful Write/Edit results for -all required plan/artifact outputs and both reviewers' terminal status (including -unavailable/disabled coverage). Do not announce completion while required work remains. +all required outputs and both reviewers' terminal status (including unavailable/disabled). Do not announce completion while required work remains. Emit this filled-in summary as an actual assistant message: **Phase 1 complete.** diff --git a/autoplan/sections/ceo-phase.md.tmpl b/autoplan/sections/ceo-phase.md.tmpl index ee627b3a9..854687e0e 100644 --- a/autoplan/sections/ceo-phase.md.tmpl +++ b/autoplan/sections/ceo-phase.md.tmpl @@ -1,6 +1,5 @@ Read {{AUTOPLAN_REVIEW_FILE:plan-ceo-review}} in full now; follow its lazy-section triggers. -Apply /autoplan's skip list; all other work runs in full. -Override: every AskUserQuestion → auto-decide using the 6 principles. +Apply /autoplan's skip list and auto-decisions; run all other work in full. **Override rules:** - Mode selection: SELECTIVE EXPANSION @@ -15,19 +14,17 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. Duplicates → reject (P4). Borderline (3-5 files) → mark TASTE DECISION. - All 10 review sections: run fully, auto-decide each issue, log every decision. - Dual voices: always run BOTH {{NATIVE_LABEL}} subagent AND {{OUTSIDE_LABEL}} if available (P6). - Run them sequentially in foreground. First the {{NATIVE_LABEL}} subagent (Agent tool - with run_in_background: false — subagents default to BACKGROUND since - Claude Code v2.1.198, so the flag must be explicitly false), then {{OUTSIDE_LABEL}} - (Bash). Both must complete before building the consensus table. + Run {{NATIVE_LABEL}} first, then {{OUTSIDE_LABEL}}, sequentially in foreground; + both must complete before consensus. **{{NATIVE_LABEL}} CEO subagent** (via Agent tool): - Native subagent tool; Claude Code Agent arguments: + Claude Code Agent argument (other harnesses: native dispatch/wait): ```json { "run_in_background": false } ``` - Set on the call, not in prompt text. Other harnesses use native dispatch/wait. + Set on the call, not in prompt text. - "Read the plan file at . You are an independent CEO/strategist + "Read the plan file at . You are an independent CEO/strategist reviewing this plan. You have NOT seen any prior review. Evaluate: 1. Is this the right problem to solve? Could a reframing yield 10x impact? 2. Are the premises stated or just assumed? Which ones could be wrong? @@ -38,11 +35,11 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. **Native completion barrier:** `isAsync: true` / `status: "async_launched"` is pending: - wait for that same agent's final result/terminal failure before outside dispatch or parent review. - No inline substitute; apply the existing failure policy. + wait for that same agent's final result/failure before outside dispatch or parent review. + No inline substitute; apply failure policy. **{{OUTSIDE_LABEL}} CEO voice** (via Bash): - Outside prompt (include the full current plan content and the context requested below, using the Write tool): + Outside prompt: inline the full contents of and context below (Write tool). IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definition directories (paths containing skills/gstack). These are AI assistant skill definitions meant for a different system. Stay focused on repository code only. @@ -52,7 +49,7 @@ IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definitio What alternatives were dismissed too quickly? What competitive or market risks are unaddressed? What scope decisions will look foolish in 6 months? Be adversarial. No compliments. Just the strategic blind spots. - File: + File: {{OUTSIDE_INVOCATION:autoplan}} @@ -119,8 +116,7 @@ Sections 1-10 — for EACH section, run the evaluation criteria from the loaded - Completion Summary (the full summary table from the CEO skill) **Close this phase before continuing:** Check successful Write/Edit results for -all required plan/artifact outputs and both reviewers' terminal status (including -unavailable/disabled coverage). Do not announce completion while required work remains. +all required outputs and both reviewers' terminal status (including unavailable/disabled). Do not announce completion while required work remains. Emit this filled-in summary as an actual assistant message: **Phase 1 complete.** diff --git a/autoplan/sections/design-phase.md b/autoplan/sections/design-phase.md index b3802f09b..9db25ab26 100644 --- a/autoplan/sections/design-phase.md +++ b/autoplan/sections/design-phase.md @@ -1,8 +1,7 @@ Read `~/.claude/skills/gstack/plan-design-review/SKILL.md` in full now; follow its lazy-section triggers. -Apply /autoplan's skip list; all other work runs in full. -Override: every AskUserQuestion → auto-decide using the 6 principles. +Apply /autoplan's skip list and auto-decisions; run all other work in full. **Override rules:** - Focus areas: all relevant dimensions (P1) @@ -11,14 +10,14 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. - Design system alignment: auto-fix if DESIGN.md exists and fix is obvious - Dual voices: always run BOTH Claude subagent AND Codex if available (P6). - **Claude design subagent** (via Agent tool, `run_in_background: false` — same foreground contract as Phase 1): - Native subagent tool; Claude Code Agent arguments: + **Claude design subagent** (native tool, foreground): + Claude Code Agent argument (other harnesses: native dispatch/wait): ```json { "run_in_background": false } ``` - Set on the call, not in prompt text. Other harnesses use native dispatch/wait. + Set on the call, not in prompt text. - "Read the plan file at . You are an independent senior product designer + "Read the plan file at . You are an independent senior product designer reviewing this plan. You have NOT seen any prior review. Evaluate: 1. Information hierarchy: what does the user see first, second, third? Is it right? 2. Missing states: loading, empty, error, success, partial — which are unspecified? @@ -30,15 +29,15 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. **Native completion barrier:** `isAsync: true` / `status: "async_launched"` is pending: - wait for that same agent's final result/terminal failure before outside dispatch or parent review. - No inline substitute; apply the existing failure policy. + wait for that same agent's final result/failure before outside dispatch or parent review. + No inline substitute; apply failure policy. **Codex design voice** (via Bash): - Outside prompt (include the full current plan content and the context requested below, using the Write tool): + Outside prompt: inline the full contents of and context below (Write tool). IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definition directories (paths containing skills/gstack). These are AI assistant skill definitions meant for a different system. Stay focused on repository code only. - Read the plan file at . Evaluate this plan's + Read the plan file at . Evaluate this plan's UI/UX design decisions. Also consider these findings from the CEO review phase: @@ -117,8 +116,7 @@ For this phase (design), retain the historical review-log skill identifier. Add DISAGREE items from scorecard → raised in the relevant pass with both perspectives. **Close this phase before continuing:** Check successful Write/Edit results for -all required plan/artifact outputs and both reviewers' terminal status (including -unavailable/disabled coverage). Do not announce completion while required work remains. +all required outputs and both reviewers' terminal status (including unavailable/disabled). Do not announce completion while required work remains. Emit this filled-in summary as an actual assistant message: **Phase 2 complete.** diff --git a/autoplan/sections/design-phase.md.tmpl b/autoplan/sections/design-phase.md.tmpl index 9d5c9c14d..14b9f6fd7 100644 --- a/autoplan/sections/design-phase.md.tmpl +++ b/autoplan/sections/design-phase.md.tmpl @@ -1,6 +1,5 @@ Read {{AUTOPLAN_REVIEW_FILE:plan-design-review}} in full now; follow its lazy-section triggers. -Apply /autoplan's skip list; all other work runs in full. -Override: every AskUserQuestion → auto-decide using the 6 principles. +Apply /autoplan's skip list and auto-decisions; run all other work in full. **Override rules:** - Focus areas: all relevant dimensions (P1) @@ -9,14 +8,14 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. - Design system alignment: auto-fix if DESIGN.md exists and fix is obvious - Dual voices: always run BOTH {{NATIVE_LABEL}} subagent AND {{OUTSIDE_LABEL}} if available (P6). - **{{NATIVE_LABEL}} design subagent** (via Agent tool, `run_in_background: false` — same foreground contract as Phase 1): - Native subagent tool; Claude Code Agent arguments: + **{{NATIVE_LABEL}} design subagent** (native tool, foreground): + Claude Code Agent argument (other harnesses: native dispatch/wait): ```json { "run_in_background": false } ``` - Set on the call, not in prompt text. Other harnesses use native dispatch/wait. + Set on the call, not in prompt text. - "Read the plan file at . You are an independent senior product designer + "Read the plan file at . You are an independent senior product designer reviewing this plan. You have NOT seen any prior review. Evaluate: 1. Information hierarchy: what does the user see first, second, third? Is it right? 2. Missing states: loading, empty, error, success, partial — which are unspecified? @@ -28,15 +27,15 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. **Native completion barrier:** `isAsync: true` / `status: "async_launched"` is pending: - wait for that same agent's final result/terminal failure before outside dispatch or parent review. - No inline substitute; apply the existing failure policy. + wait for that same agent's final result/failure before outside dispatch or parent review. + No inline substitute; apply failure policy. **{{OUTSIDE_LABEL}} design voice** (via Bash): - Outside prompt (include the full current plan content and the context requested below, using the Write tool): + Outside prompt: inline the full contents of and context below (Write tool). IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definition directories (paths containing skills/gstack). These are AI assistant skill definitions meant for a different system. Stay focused on repository code only. - Read the plan file at . Evaluate this plan's + Read the plan file at . Evaluate this plan's UI/UX design decisions. Also consider these findings from the CEO review phase: @@ -76,8 +75,7 @@ Outer tool timeout: 720000ms. On any failed invocation or incomplete review, mar DISAGREE items from scorecard → raised in the relevant pass with both perspectives. **Close this phase before continuing:** Check successful Write/Edit results for -all required plan/artifact outputs and both reviewers' terminal status (including -unavailable/disabled coverage). Do not announce completion while required work remains. +all required outputs and both reviewers' terminal status (including unavailable/disabled). Do not announce completion while required work remains. Emit this filled-in summary as an actual assistant message: **Phase 2 complete.** diff --git a/autoplan/sections/dx-phase.md b/autoplan/sections/dx-phase.md index 069d95613..a808b7de5 100644 --- a/autoplan/sections/dx-phase.md +++ b/autoplan/sections/dx-phase.md @@ -1,8 +1,7 @@ Read `~/.claude/skills/gstack/plan-devex-review/SKILL.md` in full now; follow its lazy-section triggers. -Apply /autoplan's skip list; all other work runs in full. -Override: every AskUserQuestion → auto-decide using the 6 principles. +Apply /autoplan's skip list and auto-decisions; run all other work in full. **Override rules:** - Mode selection: DX POLISH @@ -15,14 +14,14 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. - DX taste decisions (e.g., opinionated defaults vs flexibility): mark TASTE DECISION - Dual voices: always run BOTH Claude subagent AND Codex if available (P6). - **Claude DX subagent** (via Agent tool, `run_in_background: false` — same foreground contract as Phase 1): - Native subagent tool; Claude Code Agent arguments: + **Claude DX subagent** (native tool, foreground): + Claude Code Agent argument (other harnesses: native dispatch/wait): ```json { "run_in_background": false } ``` - Set on the call, not in prompt text. Other harnesses use native dispatch/wait. + Set on the call, not in prompt text. - "Read the plan file at . You are an independent DX engineer + "Read the plan file at . You are an independent DX engineer reviewing this plan. You have NOT seen any prior review. Evaluate: 1. Getting started: how many steps from zero to hello world? What's the TTHW? 2. API/CLI ergonomics: naming consistency, sensible defaults, progressive disclosure? @@ -34,15 +33,15 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. **Native completion barrier:** `isAsync: true` / `status: "async_launched"` is pending: - wait for that same agent's final result/terminal failure before outside dispatch or parent review. - No inline substitute; apply the existing failure policy. + wait for that same agent's final result/failure before outside dispatch or parent review. + No inline substitute; apply failure policy. **Codex DX voice** (via Bash): - Outside prompt (include the full current plan content and the context requested below, using the Write tool): + Outside prompt: inline the full contents of and context below (Write tool). IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definition directories (paths containing skills/gstack). These are AI assistant skill definitions meant for a different system. Stay focused on repository code only. - Read the plan file at . Evaluate this plan's developer experience. + Read the plan file at . Evaluate this plan's developer experience. Also consider these findings from prior review phases: CEO: @@ -145,8 +144,7 @@ Missing voice = N/A (not CONFIRMED). Single critical finding from one voice = fl - TTHW assessment with target **Close this phase before continuing:** Check successful Write/Edit results for -all required plan/artifact outputs and both reviewers' terminal status (including -unavailable/disabled coverage). Do not announce completion while required work remains. +all required outputs and both reviewers' terminal status (including unavailable/disabled). Do not announce completion while required work remains. Emit this filled-in summary as an actual assistant message: **Phase 2.5 complete.** diff --git a/autoplan/sections/dx-phase.md.tmpl b/autoplan/sections/dx-phase.md.tmpl index 4fead4207..06e5bfa02 100644 --- a/autoplan/sections/dx-phase.md.tmpl +++ b/autoplan/sections/dx-phase.md.tmpl @@ -1,6 +1,5 @@ Read {{AUTOPLAN_REVIEW_FILE:plan-devex-review}} in full now; follow its lazy-section triggers. -Apply /autoplan's skip list; all other work runs in full. -Override: every AskUserQuestion → auto-decide using the 6 principles. +Apply /autoplan's skip list and auto-decisions; run all other work in full. **Override rules:** - Mode selection: DX POLISH @@ -13,14 +12,14 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. - DX taste decisions (e.g., opinionated defaults vs flexibility): mark TASTE DECISION - Dual voices: always run BOTH {{NATIVE_LABEL}} subagent AND {{OUTSIDE_LABEL}} if available (P6). - **{{NATIVE_LABEL}} DX subagent** (via Agent tool, `run_in_background: false` — same foreground contract as Phase 1): - Native subagent tool; Claude Code Agent arguments: + **{{NATIVE_LABEL}} DX subagent** (native tool, foreground): + Claude Code Agent argument (other harnesses: native dispatch/wait): ```json { "run_in_background": false } ``` - Set on the call, not in prompt text. Other harnesses use native dispatch/wait. + Set on the call, not in prompt text. - "Read the plan file at . You are an independent DX engineer + "Read the plan file at . You are an independent DX engineer reviewing this plan. You have NOT seen any prior review. Evaluate: 1. Getting started: how many steps from zero to hello world? What's the TTHW? 2. API/CLI ergonomics: naming consistency, sensible defaults, progressive disclosure? @@ -32,15 +31,15 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. **Native completion barrier:** `isAsync: true` / `status: "async_launched"` is pending: - wait for that same agent's final result/terminal failure before outside dispatch or parent review. - No inline substitute; apply the existing failure policy. + wait for that same agent's final result/failure before outside dispatch or parent review. + No inline substitute; apply failure policy. **{{OUTSIDE_LABEL}} DX voice** (via Bash): - Outside prompt (include the full current plan content and the context requested below, using the Write tool): + Outside prompt: inline the full contents of and context below (Write tool). IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definition directories (paths containing skills/gstack). These are AI assistant skill definitions meant for a different system. Stay focused on repository code only. - Read the plan file at . Evaluate this plan's developer experience. + Read the plan file at . Evaluate this plan's developer experience. Also consider these findings from prior review phases: CEO: @@ -104,8 +103,7 @@ Missing voice = N/A (not CONFIRMED). Single critical finding from one voice = fl - TTHW assessment with target **Close this phase before continuing:** Check successful Write/Edit results for -all required plan/artifact outputs and both reviewers' terminal status (including -unavailable/disabled coverage). Do not announce completion while required work remains. +all required outputs and both reviewers' terminal status (including unavailable/disabled). Do not announce completion while required work remains. Emit this filled-in summary as an actual assistant message: **Phase 2.5 complete.** diff --git a/autoplan/sections/eng-phase.md b/autoplan/sections/eng-phase.md index d19c9df49..a29366394 100644 --- a/autoplan/sections/eng-phase.md +++ b/autoplan/sections/eng-phase.md @@ -1,21 +1,20 @@ Read `~/.claude/skills/gstack/plan-eng-review/SKILL.md` in full now; follow its lazy-section triggers. -Apply /autoplan's skip list; all other work runs in full. -Override: every AskUserQuestion → auto-decide using the 6 principles. +Apply /autoplan's skip list and auto-decisions; run all other work in full. **Override rules:** - Scope challenge: never reduce (P2) - Dual voices: always run BOTH Claude subagent AND Codex if available (P6). - **Claude eng subagent** (via Agent tool, `run_in_background: false` — same foreground contract as Phase 1): - Native subagent tool; Claude Code Agent arguments: + **Claude eng subagent** (native tool, foreground): + Claude Code Agent argument (other harnesses: native dispatch/wait): ```json { "run_in_background": false } ``` - Set on the call, not in prompt text. Other harnesses use native dispatch/wait. + Set on the call, not in prompt text. - "Read the plan file at . You are an independent senior engineer + "Read the plan file at . You are an independent senior engineer reviewing this plan. You have NOT seen any prior review. Evaluate: 1. Architecture: Is the component structure sound? Coupling concerns? 2. Edge cases: What breaks under 10x load? What's the nil/empty/error path? @@ -27,11 +26,11 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. **Native completion barrier:** `isAsync: true` / `status: "async_launched"` is pending: - wait for that same agent's final result/terminal failure before outside dispatch or parent review. - No inline substitute; apply the existing failure policy. + wait for that same agent's final result/failure before outside dispatch or parent review. + No inline substitute; apply failure policy. **Codex eng voice** (via Bash): - Outside prompt (include the full current plan content and the context requested below, using the Write tool): + Outside prompt: inline the full contents of and context below (Write tool). IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definition directories (paths containing skills/gstack). These are AI assistant skill definitions meant for a different system. Stay focused on repository code only. @@ -43,7 +42,7 @@ IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definitio Design: DX: - File: + File: Write the **complete prompt and required context** to a private temporary file using the Write tool. Do not interpolate user text into shell source. Replace the literal `` below with its shell-quoted pathname. Include the plan/spec/source content itself when needed: Claude Code review/challenge has no tools and cannot follow paths or execute git. Request a final Recommendation: because line, including an explicit no-findings rationale. A refusal is never completion. @@ -154,8 +153,7 @@ Missing voice = N/A (not CONFIRMED). Single critical finding from one voice = fl - TODOS.md updates (collected from all phases) **Close this phase before continuing:** Check successful Write/Edit results for -all required plan/artifact outputs and both reviewers' terminal status (including -unavailable/disabled coverage). Do not announce completion while required work remains. +all required outputs and both reviewers' terminal status (including unavailable/disabled). Do not announce completion while required work remains. Emit this filled-in summary as an actual assistant message: **Phase 3 complete.** diff --git a/autoplan/sections/eng-phase.md.tmpl b/autoplan/sections/eng-phase.md.tmpl index b4f528ff4..b22678a79 100644 --- a/autoplan/sections/eng-phase.md.tmpl +++ b/autoplan/sections/eng-phase.md.tmpl @@ -1,19 +1,18 @@ Read {{AUTOPLAN_REVIEW_FILE:plan-eng-review}} in full now; follow its lazy-section triggers. -Apply /autoplan's skip list; all other work runs in full. -Override: every AskUserQuestion → auto-decide using the 6 principles. +Apply /autoplan's skip list and auto-decisions; run all other work in full. **Override rules:** - Scope challenge: never reduce (P2) - Dual voices: always run BOTH {{NATIVE_LABEL}} subagent AND {{OUTSIDE_LABEL}} if available (P6). - **{{NATIVE_LABEL}} eng subagent** (via Agent tool, `run_in_background: false` — same foreground contract as Phase 1): - Native subagent tool; Claude Code Agent arguments: + **{{NATIVE_LABEL}} eng subagent** (native tool, foreground): + Claude Code Agent argument (other harnesses: native dispatch/wait): ```json { "run_in_background": false } ``` - Set on the call, not in prompt text. Other harnesses use native dispatch/wait. + Set on the call, not in prompt text. - "Read the plan file at . You are an independent senior engineer + "Read the plan file at . You are an independent senior engineer reviewing this plan. You have NOT seen any prior review. Evaluate: 1. Architecture: Is the component structure sound? Coupling concerns? 2. Edge cases: What breaks under 10x load? What's the nil/empty/error path? @@ -25,11 +24,11 @@ Override: every AskUserQuestion → auto-decide using the 6 principles. **Native completion barrier:** `isAsync: true` / `status: "async_launched"` is pending: - wait for that same agent's final result/terminal failure before outside dispatch or parent review. - No inline substitute; apply the existing failure policy. + wait for that same agent's final result/failure before outside dispatch or parent review. + No inline substitute; apply failure policy. **{{OUTSIDE_LABEL}} eng voice** (via Bash): - Outside prompt (include the full current plan content and the context requested below, using the Write tool): + Outside prompt: inline the full contents of and context below (Write tool). IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definition directories (paths containing skills/gstack). These are AI assistant skill definitions meant for a different system. Stay focused on repository code only. @@ -41,7 +40,7 @@ IMPORTANT: Do NOT read or execute any SKILL.md files or files in skill definitio Design: DX: - File: + File: {{OUTSIDE_INVOCATION:autoplan}} @@ -113,8 +112,7 @@ Missing voice = N/A (not CONFIRMED). Single critical finding from one voice = fl - TODOS.md updates (collected from all phases) **Close this phase before continuing:** Check successful Write/Edit results for -all required plan/artifact outputs and both reviewers' terminal status (including -unavailable/disabled coverage). Do not announce completion while required work remains. +all required outputs and both reviewers' terminal status (including unavailable/disabled). Do not announce completion while required work remains. Emit this filled-in summary as an actual assistant message: **Phase 3 complete.** diff --git a/plan-ceo-review/sections/review-sections.md b/plan-ceo-review/sections/review-sections.md index e93669d03..2dcaa1e9d 100644 --- a/plan-ceo-review/sections/review-sections.md +++ b/plan-ceo-review/sections/review-sections.md @@ -511,6 +511,11 @@ Follow the AskUserQuestion format from the Preamble above. Additional rules for * Describe the problem concretely, with file and line references. * Present 2-3 options, including "do nothing" where reasonable. * For each option: effort, risk, and maintenance burden in one line. +* Before calling AskUserQuestion, draft the recommended option as a complete remedy + for this one issue. Its offered description must state the rescue behavior, + verification, and failure visibility needed for that fix. Include those details + in the option itself. Omit irrelevant work, and keep independent findings and + new TODOs in their own questions. * **Map the reasoning to my engineering preferences above.** One sentence connecting your recommendation to a specific preference. * Label with issue NUMBER + option LETTER (e.g., "3A", "3B"). * **Zero findings:** if a section has zero findings, state "No issues, moving on" and proceed. Otherwise, use AskUserQuestion for each finding — a finding with an "obvious fix" is still a finding and still needs user approval before any change lands in the plan. diff --git a/plan-ceo-review/sections/review-sections.md.tmpl b/plan-ceo-review/sections/review-sections.md.tmpl index 7610ad299..a68cb8b34 100644 --- a/plan-ceo-review/sections/review-sections.md.tmpl +++ b/plan-ceo-review/sections/review-sections.md.tmpl @@ -287,6 +287,11 @@ Follow the AskUserQuestion format from the Preamble above. Additional rules for * Describe the problem concretely, with file and line references. * Present 2-3 options, including "do nothing" where reasonable. * For each option: effort, risk, and maintenance burden in one line. +* Before calling AskUserQuestion, draft the recommended option as a complete remedy + for this one issue. Its offered description must state the rescue behavior, + verification, and failure visibility needed for that fix. Include those details + in the option itself. Omit irrelevant work, and keep independent findings and + new TODOs in their own questions. * **Map the reasoning to my engineering preferences above.** One sentence connecting your recommendation to a specific preference. * Label with issue NUMBER + option LETTER (e.g., "3A", "3B"). * **Zero findings:** if a section has zero findings, state "No issues, moving on" and proceed. Otherwise, use AskUserQuestion for each finding — a finding with an "obvious fix" is still a finding and still needs user approval before any change lands in the plan. diff --git a/plan-devex-review/SKILL.md b/plan-devex-review/SKILL.md index c082394de..60c2a73e8 100644 --- a/plan-devex-review/SKILL.md +++ b/plan-devex-review/SKILL.md @@ -799,6 +799,10 @@ The core principle: **gather evidence and force decisions BEFORE scoring, not du scoring.** Steps 0A through 0G build the evidence base. Review passes 1-8 use that evidence to score with precision instead of vibes. +**Decision cadence, including Step 0:** One unresolved DX issue per AskUserQuestion +call. Never batch issues into a call's `questions` array. Wait for each answer. +Keep persona, empathy, and mode confirmations in separate calls from issue approvals. + ### 0A. Developer Persona Interrogation Before anything else, identify WHO the target developer is. Different developers have @@ -1006,8 +1010,8 @@ For each stage (Discover, Install, Hello World, Real Usage, Debug, Upgrade): or tells the developer to install it. A [persona] without Docker will see [specific error or nothing]." -3. **AskUserQuestion per friction point.** One question per friction point found. - Do NOT batch multiple friction points into one question. +3. **AskUserQuestion per friction point.** One separate tool call per friction point. + Do NOT batch friction points into one question or into different questions in one call. > "Journey Stage: INSTALL > diff --git a/plan-devex-review/SKILL.md.tmpl b/plan-devex-review/SKILL.md.tmpl index 247382fed..4636d703e 100644 --- a/plan-devex-review/SKILL.md.tmpl +++ b/plan-devex-review/SKILL.md.tmpl @@ -148,6 +148,10 @@ The core principle: **gather evidence and force decisions BEFORE scoring, not du scoring.** Steps 0A through 0G build the evidence base. Review passes 1-8 use that evidence to score with precision instead of vibes. +**Decision cadence, including Step 0:** One unresolved DX issue per AskUserQuestion +call. Never batch issues into a call's `questions` array. Wait for each answer. +Keep persona, empathy, and mode confirmations in separate calls from issue approvals. + ### 0A. Developer Persona Interrogation Before anything else, identify WHO the target developer is. Different developers have @@ -355,8 +359,8 @@ For each stage (Discover, Install, Hello World, Real Usage, Debug, Upgrade): or tells the developer to install it. A [persona] without Docker will see [specific error or nothing]." -3. **AskUserQuestion per friction point.** One question per friction point found. - Do NOT batch multiple friction points into one question. +3. **AskUserQuestion per friction point.** One separate tool call per friction point. + Do NOT batch friction points into one question or into different questions in one call. > "Journey Stage: INSTALL > diff --git a/test/autoplan-chain-fixture.test.ts b/test/autoplan-chain-fixture.test.ts new file mode 100644 index 000000000..734b3a791 --- /dev/null +++ b/test/autoplan-chain-fixture.test.ts @@ -0,0 +1,29 @@ +import { expect, test } from 'bun:test'; +import { readFileSync } from 'node:fs'; +import { resolve } from 'node:path'; +import { E2E_TOUCHFILES, selectTests } from './helpers/touchfiles'; + +const root = resolve(import.meta.dir, '..'); +const read = (file: string) => readFileSync(resolve(root, file), 'utf8'); +const fixture = 'test/fixtures/plans/autoplan-dashboard.md'; + +test('the chain fixture retains the complete original UI/API scope', () => { + const original = read('test/fixtures/plans/ui-heavy-feature.md'); + const complete = read(fixture); + expect(complete.startsWith(original + '\n')).toBe(true); + // This supplements dependency facts; it does not supply a completed review, + // prescribe its decisions, or pre-build the feature exercised by the chain. + expect(complete).not.toMatch(/Phase \d|GSTACK REVIEW REPORT|AUTO-DECIDE|all findings resolved/i); + expect(complete).toContain('there are no dashboard-specific tests yet'); + expect(complete).toContain('not completed work'); +}); + +test('the new fixture is isolated to the chain and its selection dependencies', () => { + expect(read('test/skill-e2e-autoplan-chain.test.ts')).toContain("'plans', 'autoplan-dashboard.md'"); + expect(read('test/skill-e2e-plan-design-with-ui.test.ts')).toContain("'plans', 'ui-heavy-feature.md'"); + for (const file of [fixture, 'test/autoplan-chain-fixture.test.ts']) { + expect(selectTests([file], E2E_TOUCHFILES).selected).toEqual(['autoplan-chain-pty']); + } + expect(selectTests(['test/fixtures/plans/ui-heavy-feature.md'], E2E_TOUCHFILES).selected) + .toEqual(['plan-design-with-ui-scope']); +}); diff --git a/test/autoplan-phase-order.test.ts b/test/autoplan-phase-order.test.ts index 1443c3097..87de06dcf 100644 --- a/test/autoplan-phase-order.test.ts +++ b/test/autoplan-phase-order.test.ts @@ -126,3 +126,33 @@ describe('autoplan phase execution checkpoints', () => { expect(dx).not.toContain('Eng: '); }); }); + + +describe('autoplan current implementation-plan identity', () => { + test('pins the assigned active plan and keeps accepted amendments separate from review analyses', () => { + const intake = read('autoplan/SKILL.md.tmpl').split('## Phase 0: Intake')[1]?.split('### Step 2:')[0] ?? ''; + expect(intake).toContain('`ACTIVE_PLAN` is the harness-assigned'); + expect(intake).toContain('All amendments and phase outputs go\nto ACTIVE_PLAN'); + expect(intake).toContain("Save SOURCE_PLAN's full current state"); + expect(intake).toContain('without dropping requirements'); + expect(intake).toContain('complete plan and accepted amendments'); + expect(intake).toContain('reviewer analyses/audit separately'); + expect(intake).toContain("Before each phase's dual voices, persist amendments"); + expect(intake).toContain('Both voices review this same current content'); + expect(intake).toContain('never the original input\nor review record'); + expect(intake).toContain('The native voice stays blind'); + }); + + test('every native and outside call site binds the fresh snapshot, retaining requested outside consensus', () => { + for (const phase of ['ceo', 'design', 'dx', 'eng']) { + const section = read(`autoplan/sections/${phase}-phase.md.tmpl`); + expect(section).toContain('Read the plan file at '); + expect(section).toContain('Outside prompt: inline the full contents of '); + expect(section).not.toContain(''); + expect(section).toContain('You have NOT seen any prior review'); + } + const eng = read('autoplan/sections/eng-phase.md.tmpl'); + expect(eng).toContain('NO prior-phase context — subagent must be truly independent'); + expect(eng).toContain('DX: { test('answers the captured combined decline action once, regardless of option order', () => { const seen = new Set(); - expect(autoplanRoutingSetupInput(F_SETUP_CAPTURE, seen)).toBe('1\r'); + expect(autoplanRoutingSetupInput(F_SETUP_CAPTURE, seen)).toBe('1'); expect(autoplanRoutingSetupInput(F_SETUP_CAPTURE, seen)).toBeNull(); const reordered = F_SETUP_CAPTURE.replace('❯1.AddroutingrulestoCLAUDE.md', '❯1.Nothanks,skip') .replace('2.Nothanks,skip', '2.AddroutingrulestoCLAUDE.md'); - expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2\r'); - expect(autoplanRoutingSetupInput(F_SETUP_CAPTURE.replace('Nothanks,skip', 'No thanks, skip—invoke skills manually'), new Set())).toBe('1\r'); + expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2'); + expect(autoplanRoutingSetupInput(F_SETUP_CAPTURE.replace('Nothanks,skip', 'No thanks, skip—invoke skills manually'), new Set())).toBe('1'); }); test('does not infer a routing answer from damaged, ambiguous, or unrelated setup choices', () => { @@ -125,11 +125,11 @@ describe('autoplan routing setup handling', () => { test('answers the fresh retry manual-invocation setup once in either option order', () => { const seen = new Set(); - expect(autoplanRoutingSetupInput(FRESH_RETRY_CAPTURE, seen)).toBe('1\r'); + expect(autoplanRoutingSetupInput(FRESH_RETRY_CAPTURE, seen)).toBe('1'); expect(autoplanRoutingSetupInput(FRESH_RETRY_CAPTURE, seen)).toBeNull(); const reordered = FRESH_RETRY_CAPTURE.replace('❯1.Addroutingrules(Recommended)', '❯1.Nothanks,manualinvocation') .replace('2.Nothanks,manualinvocation', '2.Addroutingrules(Recommended)'); - expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2\r'); + expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2'); }); test('requires opposed manual setup actions and rejects ambiguous or unrelated choices', () => { @@ -155,7 +155,7 @@ describe('autoplan routing setup handling', () => { test('answers the captured setup once, using the full question identity', () => { const seen = new Set(); - expect(autoplanRoutingSetupInput(CAPTURE, seen)).toBe('1\r'); + expect(autoplanRoutingSetupInput(CAPTURE, seen)).toBe('1'); expect(autoplanRoutingSetupInput(CAPTURE, seen)).toBeNull(); expect(autoplanRoutingSetupInput(CAPTURE.replace('works best', 'works best'), seen)).toBeNull(); }); @@ -163,25 +163,25 @@ describe('autoplan routing setup handling', () => { test('chooses Add routing rules by label when option order changes', () => { const reordered = CAPTURE.replace('❯1.Addroutingrules(Recommended)', '❯1.Nothanks') .replace('2.Nothanks', '2.Add routing rules (Recommended)'); - expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2\r'); + expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2'); }); test('accepts the full option labels captured from the subsequent live setup prompt', () => { const fullLabels = CAPTURE.replace('Addroutingrules(Recommended)', 'Add routing rules to CLAUDE.md (Recommended)') .replace('2.Nothanks', "2.No thanks, I'll invoke skills manually"); - expect(autoplanRoutingSetupInput(fullLabels, new Set())).toBe('1\r'); + expect(autoplanRoutingSetupInput(fullLabels, new Set())).toBe('1'); expect(autoplanRoutingSetupInput(fullLabels.replace('CLAUDE.md (Recommended)', 'product routes (Recommended)'), new Set())).toBeNull(); expect(autoplanRoutingSetupInput(fullLabels.replace("I'll invoke skills manually", 'delete the existing rules'), new Set())).toBeNull(); }); test('answers the current captured CLAUDE.md setup, including reordered choices, once', () => { const seen = new Set(); - expect(autoplanRoutingSetupInput(CURRENT_CAPTURE, seen)).toBe('1\r'); + expect(autoplanRoutingSetupInput(CURRENT_CAPTURE, seen)).toBe('1'); expect(autoplanRoutingSetupInput(CURRENT_CAPTURE, seen)).toBeNull(); const reordered = CURRENT_CAPTURE.replace('❯1.AddtoCLAUDE.md(recommended)', '❯1.Skip—invokemanually') .replace('2.Skip—invokemanually', '2.AddtoCLAUDE.md(recommended)'); - expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2\r'); - expect(autoplanRoutingSetupInput(CURRENT_CAPTURE.replace('to CLAUDE.md?', "to this project's CLAUDE.md?"), new Set())).toBe('1\r'); + expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2'); + expect(autoplanRoutingSetupInput(CURRENT_CAPTURE.replace('to CLAUDE.md?', "to this project's CLAUDE.md?"), new Set())).toBe('1'); }); test('the current wording still requires both explicit setup choices and the CLAUDE.md target', () => { @@ -197,17 +197,17 @@ describe('autoplan routing setup handling', () => { test('recognizes the native A retry packet with its abbreviated manual-decline label', () => { const retry = CAPTURE.replace('Addroutingrules(Recommended)', 'Add to CLAUDE.md (Recommended)') .replace('2.Nothanks', '2.No thanks, manual'); - expect(autoplanRoutingSetupInput(retry, new Set())).toBe('1\r'); + expect(autoplanRoutingSetupInput(retry, new Set())).toBe('1'); expect(autoplanRoutingSetupInput(retry.replace('No thanks, manual', 'No thanks, delete it'), new Set())).toBeNull(); }); test('answers the exact B timeout menu by its routing label, in either order', () => { const seen = new Set(); - expect(autoplanRoutingSetupInput(B_CAPTURE, seen)).toBe('1\r'); + expect(autoplanRoutingSetupInput(B_CAPTURE, seen)).toBe('1'); expect(autoplanRoutingSetupInput(B_CAPTURE, seen)).toBeNull(); const reordered = B_CAPTURE.replace('❯1.AddroutingrulestoCLAUDE.md(Recommended)', '❯1.Nothanks,invokemanually') .replace('2.Nothanks,invokemanually', '2.AddroutingrulestoCLAUDE.md(Recommended)'); - expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2\r'); + expect(autoplanRoutingSetupInput(reordered, new Set())).toBe('2'); expect(autoplanRoutingSetupInput(B_CAPTURE.replace('Nothanks,invokemanually', 'Nothanks,deletethefilemanually'), new Set())).toBeNull(); expect(autoplanRoutingSetupInput(B_CAPTURE.replace('Add gstack skill routing rules to CLAUDE.md?', 'Which routing design should the application use?'), new Set())).toBeNull(); }); @@ -221,13 +221,13 @@ describe('autoplan routing setup handling', () => { ]; for (const opening of openings) { const frame = CURRENT_CAPTURE.replace('Add gstack skill routing rules to CLAUDE.md? ', opening); - expect(autoplanRoutingSetupInput(frame, new Set()), opening).toBe('1\r'); + expect(autoplanRoutingSetupInput(frame, new Set()), opening).toBe('1'); } }); test('recognizes an intact setup qid with an explicit CLAUDE.md action and opposed manual decline', () => { const frame = CURRENT_CAPTURE.replace('Add gstack skill routing rules to CLAUDE.md?', 'Configure this project’s CLAUDE.md?'); - expect(autoplanRoutingSetupInput(frame, new Set())).toBe('1\r'); + expect(autoplanRoutingSetupInput(frame, new Set())).toBe('1'); expect(autoplanRoutingSetupInput(frame.replace('gstack-qid:routing-injection', 'gstack-qid:product-routing'), new Set())).toBeNull(); expect(autoplanRoutingSetupInput(frame.replace('AddtoCLAUDE.md(recommended)', 'Acceptrecommendation'), new Set())).toBeNull(); expect(autoplanRoutingSetupInput(frame.replace('Skip—invokemanually', 'Deferthisfinding'), new Set())).toBeNull(); @@ -290,16 +290,16 @@ const G_ROUTING_CAPTURE = [ describe('autoplan routing action survives courtesy repaint', () => { test('selects the explicit Add action once in the captured G menu, in both orders', () => { const seen = new Set(); - expect(autoplanRoutingSetupInput(G_ROUTING_CAPTURE, seen)).toBe('1\r'); + expect(autoplanRoutingSetupInput(G_ROUTING_CAPTURE, seen)).toBe('1'); expect(autoplanRoutingSetupInput(G_ROUTING_CAPTURE, seen)).toBeNull(); const reversed = G_ROUTING_CAPTURE.replace('❯1.AddroutingrulestoCLAUDE.md', "❯1.N thanks, I'll invokeskillsmanually") .replace("2. N thanks, I'll invokeskillsmanually", '2.AddroutingrulestoCLAUDE.md'); - expect(autoplanRoutingSetupInput(reversed, new Set())).toBe('2\r'); + expect(autoplanRoutingSetupInput(reversed, new Set())).toBe('2'); }); test('the actual manual-invocation action needs no courtesy formula', () => { for (const action of ['Manual invocation', 'Invoke skills manually', "I'll invoke skills manually", 'Thanks, invoke manually']) { - expect(autoplanRoutingSetupInput(G_ROUTING_CAPTURE.replace("N thanks, I'll invokeskillsmanually", action), new Set()), action).toBe('1\r'); + expect(autoplanRoutingSetupInput(G_ROUTING_CAPTURE.replace("N thanks, I'll invokeskillsmanually", action), new Set()), action).toBe('1'); } }); diff --git a/test/devex-count-fixture.test.ts b/test/devex-count-fixture.test.ts index e24965360..2b3612612 100644 --- a/test/devex-count-fixture.test.ts +++ b/test/devex-count-fixture.test.ts @@ -172,3 +172,43 @@ describe('DevEx calibrated fixture instructions', () => { expect(suppliedContext).not.toMatch(/(?:exactly|at least|at most)\s+(?:five|5)|(?:five|5)[- ]findings?|4[-–]7|reviewCount|CEILING|FLOOR/i); }); }); + + +describe('native first-local-run CI decisions', () => { + const question = + 'D3 \u2014 Journey Stage: FIRST RESULT \u2014 5-minute CI gate makes the <2min TTHW target mathematically unreachable\n\nELI10: On every first local run, the SDK blocks for 5 minutes waiting for a remote CI check (docs/current-contracts.md). There is no skip flag. The TTHW study measured EvalKit at 6 minutes total (docs/benchmarks.md). The agreed target is under 2 minutes. With a mandatory 5-minute wait baked in, you cannot reach that target \u2014 the CI gate alone exceeds it. Competitors: A=2min, B=4min, C=3min. EvalKit currently loses on TTHW.\n\nStakes if we pick wrong: If the target stays <2min but the gate stays too, the benchmark is aspirational theatre. If the gate stays and the target is adjusted, the competitive position is weaker.\n\nRecommendation: A \u2014 add a local skip path. The CI gate adds real value in production CI, but blocking local first-runs is the wrong tradeoff for an SDK that wants sub-2min TTHW.\nNote: options differ in kind, not coverage \u2014 no completeness score.\n\n'; + test('the answered first-local-run CI gate is substantive, including its plural variant', () => { + for (const text of [ + question, + question.replace('first local run', 'first local runs'), + ]) { + const fp = call(text); + fp.nativeCall!.questions[0]!.header = 'CI gate TTHW'; + expect(isDevexReviewIssue(fp)).toBe(true); + } + }); + test('an unanswered CI tab and an administrative recap never create coverage', () => { + const fp = call('Does the empathy narrative match reality?'); + fp.nativeCall!.questions[0]!.header = 'Empathy check'; + fp.nativeCall!.questions.push({ + header: 'CI gate TTHW', + question, + options: [{ label: 'Skip CI' }, { label: 'Keep CI' }], + }); + fp.nativeCall!.unansweredQuestionIndices = [1]; + expect(isDevexReviewIssue(fp)).toBe(false); + fp.nativeCall!.answers![question] = 'Skip CI'; + fp.nativeCall!.unansweredQuestionIndices = []; + expect(isDevexReviewIssue(fp)).toBe(true); + const recap = call(question); + recap.nativeCall!.questions[0]!.header = 'Empathy check'; + expect(isDevexReviewIssue(recap)).toBe(false); + expect( + isDevexReviewIssue( + call( + 'The production CI gate waits five minutes. Change the release check?', + ), + ), + ).toBe(false); + }); +}); diff --git a/test/fixtures/plans/autoplan-dashboard.md b/test/fixtures/plans/autoplan-dashboard.md new file mode 100644 index 000000000..e02429274 --- /dev/null +++ b/test/fixtures/plans/autoplan-dashboard.md @@ -0,0 +1,77 @@ +# Plan: User Dashboard Page + +## Context +We're shipping a new user dashboard at `/dashboard` showing recent activity, +notifications panel, and quick-action buttons. Users land here after login. + +## UI Scope +- New React page component `UserDashboard.tsx` at `src/pages/` +- Three new sub-components: `ActivityFeed`, `NotificationsPanel`, `QuickActions` +- Tailwind CSS for layout, mobile-first responsive (breakpoints: sm/md/lg) +- Empty state, loading skeleton, error state for each panel +- Hover states + focus-visible outlines on every interactive element +- Modal dialog for "Mark all as read" on notifications panel +- Toast notification system for action feedback + +## Backend +- New REST endpoint `GET /api/dashboard` returns `{ activity, notifications, quickActions }` +- Backed by existing PostgreSQL tables; no schema changes + +## Out of scope +- Dark mode (separate plan) +- Personalization / customization (separate plan) + +## Existing product and application contracts + +This is the existing single-role member workspace, not a new product or a new +onboarding flow. Members currently visit three separate pages after login to +resume work, check alerts, and inspect recent changes. In the team's last task +walkthrough, finding the next item took a median 75 seconds. The dashboard's +success measure is login-to-first-completed-task time, targeting 45 seconds, +with completed-task rate and permission-error rate as guardrails. Existing +analytics records login, action start, action completion, and permission errors; +the new page still needs its own exposure and interaction instrumentation. + +Activity is the immutable audit history of workspace changes. Notifications are +member-specific alerts with persistent read state; acknowledging an alert does +not alter audit history. The existing action registry supplies three actions +(create an item, resume assigned work, invite a member), with stable IDs, labels, +route targets, and server-side eligibility predicates. These are links into +existing workflows; action ranking and a new configuration service do not exist. + +The application already uses cookie sessions and workspace membership middleware. +Its request context supplies the authenticated member and workspace IDs. Existing +repository methods apply both IDs where appropriate; callers do not accept a +workspace ID from query parameters. Mutations already require CSRF tokens. The +new dashboard endpoint must compose these methods and follow the same boundaries; +its handler, authorization integration, and failure paths have not been written. + +Existing list methods return the latest 20 records plus a cursor and have indexed +workspace/member and created-at access paths. The existing full activity and +notification pages own older-page navigation. The member-scoped bulk-read API is +idempotent and marks only notifications at or before the supplied snapshot time, +so later arrivals remain unread. Existing HTTP clients expose typed unauthenticated, +forbidden, validation, retryable-service, and network errors. Each dashboard panel +still needs to map these results to its loading, empty, error, retry, and success +states; the aggregate endpoint's response composition and partial-failure behavior +remain new implementation work. No schema migration or new mutation API is needed. + +The app already has Tailwind spacing/color/type tokens, a responsive page shell, +buttons, links, and a dialog primitive with focus trapping, Escape dismissal, and +focus return. These primitives do not implement any dashboard panel, confirmation +flow, or toast system. The new modal and toast feedback must also work with keyboard +and screen readers; existing accessibility policy requires named controls, a live +region for nonblocking feedback, sufficient contrast, and reduced-motion support. +The dashboard still needs its own layout, content hierarchy, mobile behavior, and +state-specific copy at sm/md/lg breakpoints. + +Vitest, React Testing Library, and Playwright already run in CI. Existing fixtures +cover authenticated members, another workspace, empty lists, and service failures; +there are no dashboard-specific tests yet. Existing staging feature flags and +request/error metrics support a member-cohort rollout and rollback to the current +landing page. The dashboard's rollout criteria, endpoint performance checks, +interaction tests, and accessibility verification must be specified and added. + +All dashboard screen, panel, aggregate-endpoint, modal, and toast work listed above +is new. The existing contracts describe dependencies to reuse, not completed work +or prior approval of an implementation approach. diff --git a/test/helpers/auq-sdk-capture.ts b/test/helpers/auq-sdk-capture.ts index 993976e6c..fd613d7cc 100644 --- a/test/helpers/auq-sdk-capture.ts +++ b/test/helpers/auq-sdk-capture.ts @@ -278,7 +278,8 @@ Rules for this run: - At any decision point that would call AskUserQuestion, silently pick the skill's recommended option and continue. Do NOT stop to ask. - This skill's body has been carved into on-demand sections/. When the skill gives a STOP-Read directive (for example "Read \`.../sections/\` and execute it in full"), you MUST actually Read that sections/ file with the Read tool BEFORE doing the work it covers. Do not work from memory. - Do NOT run git, gh, commit, push, or any mutating command. -- When the workflow is complete, write the skill's final output (the full review report / ship plan, including any required report table) to ${outFile}.${nativeReviewRule}`; +- When the workflow is complete, write the skill's final output (the full review report / ship plan, including any required report table) to ${outFile}.${nativeReviewRule} +- After all required writes are complete, return a brief completion message and STOP. Do not reproduce the full report in the final response.`; let result: SkillTestResult; try { diff --git a/test/helpers/autoplan-setup-question.ts b/test/helpers/autoplan-setup-question.ts index b30ad1e23..1448bb818 100644 --- a/test/helpers/autoplan-setup-question.ts +++ b/test/helpers/autoplan-setup-question.ts @@ -1,8 +1,10 @@ -import { capturePlanCountQuestion } from './claude-pty-runner'; +import { capturePlanCountQuestion, planCountQuestionInput } from './claude-pty-runner'; + +import type { NativePlanQuestionCall } from './plan-count-transcript'; /** Answer only gstack's routing setup prompt in the isolated autoplan fixture. */ -export function autoplanRoutingSetupInput(visible: string, seen: Set): string | null { - const question = capturePlanCountQuestion(visible, seen, 0, true); +export function autoplanRoutingSetupInput(visible: string, seen: Set, pending?: NativePlanQuestionCall): string | null { + const question = capturePlanCountQuestion(visible, seen, 0, true, pending); if (!question) return null; // The model rephrases the setup question's closing sentence. Its routing @@ -44,5 +46,5 @@ export function autoplanRoutingSetupInput(visible: string, seen: Set): s const claudeTarget = /CLAUDE\.md/i.test(prompt); const quotedPremise = /\b(?:plan|spec|document)\s+(?:quotes?|cites?|references?)\b/i.test(primary); if (!claudeTarget || quotedPremise || (!routingId && !routingPremise)) return null; - return `${add[0]!.index}\r`; + return planCountQuestionInput(visible, question, add[0]!.index); } diff --git a/test/helpers/ceo-mode-option.ts b/test/helpers/ceo-mode-option.ts index 64bca3e25..be4cd4097 100644 --- a/test/helpers/ceo-mode-option.ts +++ b/test/helpers/ceo-mode-option.ts @@ -7,6 +7,7 @@ import { auqFingerprint, classifyPlanCountFrame, isNumberedOptionListVisible, + matchesNativePlanQuestion, planCountSubmissionInput, type AskUserQuestionFingerprint, } from './claude-pty-runner'; @@ -52,13 +53,13 @@ const permissionStates = new WeakMap, { file: ReturnType; other: Set; }>(); -function ceoPermissionAction(visible: string, seenQuestions: Set): 'grant' | 'handled' | null { +function ceoPermissionAction(visible: string, seenQuestions: Set, completionHistory = visible): 'grant' | 'handled' | null { let state = permissionStates.get(seenQuestions); if (!state) { state = { file: createPlanCountPermissionGuard(), other: new Set() }; permissionStates.set(seenQuestions, state); } - const file = state.file(visible); + const file = state.file(visible, completionHistory); if (file !== null) return file; if (classifyPlanCountFrame(visible) !== 'permission') return null; const signature = auqFingerprint(parseQuestionPrompt(visible), parseNumberedOptions(visible)); @@ -72,15 +73,17 @@ export function nextCeoModeNavigation( visible: string, targetMode: CeoMode, seenQuestions: Set, + pending?: NativePlanQuestionCall, + completionHistory = visible, ): ModeNavigationAction { - const permission = ceoPermissionAction(visible, seenQuestions); + const permission = pending && matchesNativePlanQuestion(visible, pending) ? null : ceoPermissionAction(visible, seenQuestions, completionHistory); if (permission !== null) return permission === 'grant' ? { kind: 'permission', input: '1\r' } : { kind: 'wait' }; const frame = classifyPlanCountFrame(visible); const submission = frame === null ? planCountSubmissionInput(visible) : null; if (submission !== null) return { kind: 'submission', input: submission }; if (!isNumberedOptionListVisible(visible)) return { kind: 'wait' }; - const question = capturePlanCountQuestion(visible, seenQuestions, 0, true); + const question = capturePlanCountQuestion(visible, seenQuestions, 0, true, pending); if (!question) return { kind: 'wait' }; const index = findCeoModeOption(question.options, targetMode); return index === null ? { kind: 'question', question } : { kind: 'mode', index, question }; @@ -176,10 +179,12 @@ export function nextCeoPostureContinuation( selectionStartedAt: number, seenQuestions: Set, alreadyContinued: boolean, + completionHistory = visible, ): 'permission' | 'question' | null { - const permission = ceoPermissionAction(visible, seenQuestions); + const pending = transcript.calls.find(call => !call.answered && !call.failed); + const permission = pending && matchesNativePlanQuestion(visible, pending) ? null : ceoPermissionAction(visible, seenQuestions, completionHistory); if (permission !== null) return permission === 'grant' ? 'permission' : null; if (alreadyContinued || !nativeCeoModeAnswer(transcript, targetMode, selectionStartedAt)) return null; - const action = nextCeoModeNavigation(visible, targetMode, seenQuestions); + const action = nextCeoModeNavigation(visible, targetMode, seenQuestions, pending, completionHistory); return action.kind === 'question' ? 'question' : null; } diff --git a/test/helpers/claude-pty-runner.ts b/test/helpers/claude-pty-runner.ts index ad96eb051..a1652d054 100644 --- a/test/helpers/claude-pty-runner.ts +++ b/test/helpers/claude-pty-runner.ts @@ -1070,6 +1070,8 @@ export interface AskUserQuestionFingerprint { administrative?: 'completion-handoff'; /** Lossless source metadata for observed calls; UI-only fingerprints omit it. */ nativeCall?: NativePlanQuestionCall; + /** Active tab for UI answering; completed coverage still counts the whole call once. */ + nativeQuestionIndex?: number; } /** Full question text feeds routing/phase predicates before diagnostics truncate it. */ @@ -1274,16 +1276,61 @@ function matchesClippedNativeQuestion(visible: string, call: NativePlanQuestionC if (suffix.split('\n').filter(line => line.trim()).length < 2 || displayed.length < 160 || displayed.length > native.length || !native.endsWith(displayed)) return false; const menu = visible.slice(cursor.index); - const footer = /Enter\s*to\s*select\s*·\s*↑\/↓\s*to\s*navigate\s*·\s*Esc\s*to\s*cancel/i.exec(menu); + const footer = /Enter\s*to\s*select\s*·\s*(?:↑\/↓\s*to\s*navigate|Tab\/Arrow\s*keys\s*to\s*navigate)\s*·\s*Esc\s*to\s*cancel/i.exec(menu); if (!footer || !/^[\s│┃─━└┘]*$/.test(menu.slice(footer.index + footer[0].length))) return false; const options = parseNumberedOptions(visible); - return options.length === question.options.length && options.every((option, index) => - option.index === index + 1 && exact(option.label) === exact(question.options[index]!.label)); + const offered = options.slice(0, question.options.length); + const controls = options.slice(question.options.length); + return offered.length === question.options.length && offered.every((option, index) => + option.index === index + 1 && exact(option.label) === exact(question.options[index]!.label)) && + controls.length <= 2 && controls.every((option, index) => option.index === question.options.length + index + 1 && + (index === 0 ? /^Typesomething\.?$/ : /^Chataboutthis$/).test(exact(option.label))); +} + +/** Match a pending packet's displayed tab by its full question and offered choices. */ +function nativePacketQuestionIndex(visible: string, call: NativePlanQuestionCall): number | null { + if (call.failed || call.questions.length < 2) return null; + const normalized = stripPtyResidue(visible).replace(/\r+\n?/g, '\n'); + const cursor = [...normalized.matchAll(/❯\s*1\./g)].at(-1); + if (!cursor) return null; + const before = normalized.slice(0, cursor.index); + const bar = [...before.matchAll(/←[^\n]*[☐☒][^\n]*✔\s*Submit\s*→/g)].at(-1); + if (!bar) { + const clipped = call.questions.flatMap((question, index) => + matchesClippedNativeQuestion(normalized, {...call, questions: [question]}) ? [index] : []); + return clipped.length === 1 ? clipped[0]! : null; + } + if (!/Enter\s*to\s*select\s*·\s*Tab\/Arrow\s*keys\s*to\s*navigate\s*·\s*Esc\s*to\s*cancel/i.test(normalized.slice(cursor.index))) return null; + const compact = (value: string) => value.replace(/^[\t │┃]+/gm, '').replace(/\s+/g, ''); + const body = compact(before.slice(bar.index + bar[0].length)); + const options = parseNumberedOptions(normalized); + const matched = call.questions.flatMap((q, index) => body === compact(q.question) && + q.options.every((option, i) => options[i]?.index === i + 1 && compact(options[i]!.label) === compact(option.label)) ? [index] : []); + return matched.length === 1 ? matched[0]! : null; +} + +/** A native numeric shortcut accepts immediately; never queue Enter on the next tab. */ +export function planCountQuestionInput(visible: string, fp: AskUserQuestionFingerprint, index: number): string { + if (!Number.isInteger(index) || index < 1 || index > 9) throw new RangeError(`Invalid numbered option: ${index}`); + const native = fp.nativeCall?.questions[fp.nativeQuestionIndex ?? 0]; + if (native) return native.multiSelect === true ? `${index}\r` : String(index); + // The native multi-select renderer uses the same footer, but marks rows + // with [ ]/[✓]. Without metadata, preserve that existing protocol. + if (/❯?\s*[1-9]\.\s*\[[ ✓✔xX]\]/m.test(visible)) return `${index}\r`; + // Native JSONL may flush only after submission. Its complete tab bar and + // navigation footer establish the input protocol without counting coverage. + const packet = /←[^\r\n]*[☐☒][^\r\n]*✔\s*Submit\s*→[\s\S]*❯\s*1\./.test(visible) && + /Enter\s*to\s*select\s*·\s*Tab\/Arrow\s*keys\s*to\s*navigate\s*·\s*Esc\s*to\s*cancel/i.test(visible); + const single = /(?:^|[\r\n])[\t │┃]*[☐□][^\r\n]+[\s\S]*❯\s*1\./.test(visible) && + /Enter\s*to\s*select\s*·\s*↑\/↓\s*to\s*navigate\s*·\s*Esc\s*to\s*cancel/i.test(visible); + const panel = packet || single; + return panel ? String(index) : `${index}\r`; } /** Match native question identity before permission text can choose an answer. */ export function matchesNativePlanQuestion(visible: string, call: NativePlanQuestionCall): boolean { - if (call.failed || call.questions.length !== 1) return false; + if (call.failed) return false; + if (call.questions.length !== 1) return nativePacketQuestionIndex(visible, call) !== null; const normalized = stripPtyResidue(visible).replace(/\r+\n?/g, '\n'); const tail = normalized.slice(-4096); const cursor = [...tail.matchAll(/❯\s*1\./g)].at(-1); @@ -1323,7 +1370,15 @@ export function capturePlanCountQuestion( if (!cursor) return null; if (pending && !pending.answered && !pending.failed && matchesNativePlanQuestion(visible, pending)) { + const activeIndex = pending.questions.length === 1 ? 0 : nativePacketQuestionIndex(visible, pending)!; const fp = nativePlanCallFingerprint(pending, observedAtMs, preReview); + fp.nativeQuestionIndex = activeIndex; + if (pending.questions.length > 1) { + const question = pending.questions[activeIndex]!; + fp.signature += `:question:${activeIndex}`; + fp.promptSnippet = `${question.header} ${question.question}`; + fp.options = question.options.map((option, i) => ({index: i + 1, label: option.label})); + } const renderedOptions = parseNumberedOptions(visible); const renderedPrompt = parseQuestionPrompt(visible); const renderedSignature = renderedOptions.length >= 2 && renderedPrompt @@ -2873,8 +2928,9 @@ export async function runPlanSkillCounting(opts: { const pickIdx = prerequisitePick ?? callerPick ?? (isFirstAUQ && opts.firstAUQPick ? opts.firstAUQPick(routing) : defaultPick); isFirstAUQ = false; - if (prerequisitePick !== null || callerPick !== null) await selectPtyNumberedOption(session, pickIdx); - else session.send(`${pickIdx}\r`); + const questionInput = planCountQuestionInput(visible, fp, pickIdx); + if (questionInput.includes('\r') && (prerequisitePick !== null || callerPick !== null)) await selectPtyNumberedOption(session, pickIdx); + else session.send(questionInput); // Give the agent a beat to advance to the next state. await Bun.sleep(2000); diff --git a/test/helpers/devex-count-fixture.ts b/test/helpers/devex-count-fixture.ts index 67d75efbb..2c69ef284 100644 --- a/test/helpers/devex-count-fixture.ts +++ b/test/helpers/devex-count-fixture.ts @@ -131,7 +131,7 @@ function substantiveIssue({ header, question }: { header: string; question: stri if (administrativeQuestion(header, question)) return false; const normalized = `${header} ${question}`.replace(/\s+/g, ' '); const ciGate = /\b(?:CI|continuous integration)\b/i.test(normalized) - && /\b(?:first[- ]run|first eval(?:uation)?|local eval(?:uation)?|hello world)\b/i.test(normalized) + && /\b(?:first[- ](?:local[- ])?runs?|first eval(?:uation)?|local eval(?:uation)?|hello world)\b/i.test(normalized) && /\b(?:mandatory|required|blocks?|five[- ]minute|5[- ]min(?:ute)?|wait|gate)\b/i.test(normalized); const argumentsReversed = /\brun_eval\b/i.test(normalized) && /\brun_batch\b/i.test(normalized) && /\b(?:revers\w*|inconsisten\w*|swapp\w*|different|order|positional)\b/i.test(normalized); diff --git a/test/helpers/touchfiles-data.ts b/test/helpers/touchfiles-data.ts index ed818ae72..43f464726 100644 --- a/test/helpers/touchfiles-data.ts +++ b/test/helpers/touchfiles-data.ts @@ -139,7 +139,7 @@ export const E2E_TOUCHFILES: Record = { // Each one tests behavior the SDK harness can't observe (rendered TTY, // numbered-option lists, multi-phase ordering, idempotency state echo). 'auq-format-gate': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completeness-section.ts', 'scripts/resolvers/preamble.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/session-runner.ts', 'test/helpers/llm-judge.ts', 'test/skill-e2e-ask-user-question-format-compliance.test.ts'], - 'plan-ceo-mode-routing': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/ceo-mode-option.ts', 'test/ceo-mode-option.test.ts', 'test/pty-option-selection.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/skill-e2e-plan-ceo-mode-routing.test.ts'], + 'plan-ceo-mode-routing': ['plan-ceo-review/**', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble.ts', 'test/helpers/claude-pty-runner.ts', 'test/plan-count-native-input.test.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/ceo-mode-option.ts', 'test/ceo-mode-option.test.ts', 'test/pty-option-selection.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/skill-e2e-plan-ceo-mode-routing.test.ts'], 'plan-design-with-ui-scope': ['plan-design-review/**', 'test/fixtures/plans/ui-heavy-feature.md', 'test/helpers/claude-pty-runner.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/skill-e2e-plan-design-with-ui.test.ts'], 'ship-idempotency-pty': ['ship/**', 'bin/gstack-next-version', 'bin/gstack-version-bump', 'scripts/resolvers/sections.ts', 'lib/worktree.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/skill-e2e-ship-idempotency.test.ts'], 'tpa-present': ['scripts/resolvers/third-party-actions.ts', 'ship/SKILL.md.tmpl', 'ship/sections/apple-release.md.tmpl', 'scripts/gen-skill-docs.ts', 'test/helpers/session-runner.ts', 'test/skill-e2e-third-party-actions.test.ts'], @@ -154,16 +154,16 @@ export const E2E_TOUCHFILES: Record = { // the selector sets GSTACK_CARVE_SKILL= to scope cost to the changed // skill (D-CODEX A). Touching the registry/helper or sections.ts runs all. 'carve-section-loading': ['scripts/resolvers/composition.ts', 'test/autoplan-review-discovery.test.ts', 'test/autoplan-phase-order.test.ts', 'design-html/**', 'design-shotgun/**', 'qa/**', 'browse/**', 'retro/**', 'autoplan/**', 'spec/**', 'setup-gbrain/**', 'review/**', 'codex/**', 'land-and-deploy/**', 'plan-eng-review/**', 'plan-design-review/**', 'plan-devex-review/**', 'office-hours/**', 'document-release/**', 'design-consultation/**', 'cso/**', 'test/helpers/carve-guards.ts', 'scripts/resolvers/sections.ts', 'scripts/resolvers/redact-doc.ts', 'scripts/gen-skill-docs.ts', 'test/helpers/auq-sdk-capture.ts', 'test/helpers/session-runner.ts', 'test/session-runner-tools.test.ts', 'test/carve-section-loading.test.ts'], - 'autoplan-chain-pty': ['scripts/resolvers/composition.ts', 'test/autoplan-review-discovery.test.ts', 'test/autoplan-phase-order.test.ts', 'autoplan/**', 'plan-ceo-review/**', 'plan-design-review/**', 'plan-eng-review/**', 'plan-devex-review/**', 'test/fixtures/plans/ui-heavy-feature.md', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/autoplan-setup-question.ts', 'test/autoplan-setup-question.test.ts', 'test/helpers/autoplan-phase-observer.ts', 'test/autoplan-phase-observer.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/skill-e2e-autoplan-chain.test.ts'], + 'autoplan-chain-pty': ['scripts/resolvers/composition.ts', 'test/autoplan-review-discovery.test.ts', 'test/autoplan-phase-order.test.ts', 'autoplan/**', 'plan-ceo-review/**', 'plan-design-review/**', 'plan-eng-review/**', 'plan-devex-review/**', 'test/fixtures/plans/autoplan-dashboard.md', 'test/autoplan-chain-fixture.test.ts', 'test/helpers/claude-pty-runner.ts', 'test/plan-count-native-input.test.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/autoplan-setup-question.ts', 'test/autoplan-setup-question.test.ts', 'test/helpers/autoplan-phase-observer.ts', 'test/autoplan-phase-observer.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/skill-e2e-autoplan-chain.test.ts'], // Per-finding AskUserQuestion count + review-report-at-bottom assertion. // Each test drives its skill end-to-end; touchfiles include preamble + // completion-status resolvers because they affect question cadence and // terminal output (the regression surface this test catches). - 'plan-ceo-finding-count': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/plan-count-completion.test.ts', 'test/helpers/ceo-completion-handoff.ts', 'test/ceo-completion-handoff.test.ts', 'test/fixtures/ceo-completion-handoff-calls.json', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/skill-e2e-plan-ceo-finding-count.test.ts'], - 'plan-eng-finding-count': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-eng-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/plan-count-completion.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/skill-e2e-plan-eng-finding-count.test.ts'], - 'plan-design-finding-count': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-design-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/plan-count-completion.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/skill-e2e-plan-design-finding-count.test.ts'], - 'plan-devex-finding-count': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-devex-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/plan-count-completion.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/helpers/devex-count-fixture.ts', 'test/devex-count-fixture.test.ts', 'test/skill-e2e-plan-devex-finding-count.test.ts'], + 'plan-ceo-finding-count': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/plan-count-native-input.test.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/plan-count-completion.test.ts', 'test/helpers/ceo-completion-handoff.ts', 'test/ceo-completion-handoff.test.ts', 'test/fixtures/ceo-completion-handoff-calls.json', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/skill-e2e-plan-ceo-finding-count.test.ts'], + 'plan-eng-finding-count': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-eng-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/plan-count-native-input.test.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/plan-count-completion.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/skill-e2e-plan-eng-finding-count.test.ts'], + 'plan-design-finding-count': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-design-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/plan-count-native-input.test.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/plan-count-completion.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/skill-e2e-plan-design-finding-count.test.ts'], + 'plan-devex-finding-count': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-devex-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'test/helpers/claude-pty-runner.ts', 'test/plan-count-native-input.test.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/plan-count-completion.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/helpers/devex-count-fixture.ts', 'test/devex-count-fixture.test.ts', 'test/skill-e2e-plan-devex-finding-count.test.ts'], // Gate-tier reviewCount-floor counterparts. Catch the May 2026 transcript // bug (model wrote a plan-mode plan and ExitPlanMode'd without firing any @@ -180,8 +180,8 @@ export const E2E_TOUCHFILES: Record = { // a model fires one AUQ then batches the rest into a "## Decisions to // confirm" plan write. runPlanSkillFloorCheck cannot detect that shape // (it exits on first AUQ); runPlanSkillCounting can. - 'plan-eng-multi-finding-batching': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-eng-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-eng-multi-finding-batching.test.ts'], - 'plan-ceo-split-overflow': ['bin/gstack-config', 'plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'bin/gstack-question-preference', 'test/helpers/claude-pty-runner.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-ceo-split-overflow.test.ts'], + 'plan-eng-multi-finding-batching': ['bin/gstack-config', 'bin/gstack-skill-start', 'bin/gstack-skill-end', 'plan-eng-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'scripts/resolvers/preamble/generate-completion-status.ts', 'scripts/resolvers/review.ts', 'test/helpers/claude-pty-runner.ts', 'test/plan-count-native-input.test.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-eng-multi-finding-batching.test.ts'], + 'plan-ceo-split-overflow': ['bin/gstack-config', 'plan-ceo-review/**', 'scripts/resolvers/preamble.ts', 'scripts/resolvers/preamble/generate-ask-user-format.ts', 'bin/gstack-question-preference', 'test/helpers/claude-pty-runner.ts', 'test/plan-count-native-input.test.ts', 'test/helpers/pty-screen.ts', 'test/pty-screen.test.ts', 'test/pty-screen-session.test.ts', 'test/fixtures/pty-screen/**', 'test/helpers/hermetic-skill-runtime.ts', 'test/hermetic-skill-runtime.test.ts', 'test/helpers/claude-pty-runner.unit.test.ts', 'test/helpers/pty-trust-dialog.ts', 'test/pty-trust-dialog.test.ts', 'test/helpers/plan-count-transcript.ts', 'test/plan-count-transcript.test.ts', 'test/helpers/plan-count-artifacts.ts', 'test/plan-count-artifacts.test.ts', 'test/helpers/eval-store.ts', 'test/helpers/plan-count-fixture.ts', 'test/plan-count-fixture.test.ts', 'test/fixtures/forcing-finding-seeds.ts', 'test/skill-e2e-plan-ceo-split-overflow.test.ts'], 'brain-privacy-gate': ['bin/gstack-skill-start', 'bin/gstack-skill-end', 'scripts/resolvers/preamble/generate-brain-sync-block.ts', 'scripts/resolvers/preamble.ts', 'bin/gstack-brain-sync', 'bin/gstack-artifacts-init', 'bin/gstack-config', 'test/helpers/agent-sdk-runner.ts', 'test/skill-e2e-brain-privacy-gate.test.ts'], // /setup-gbrain Path 4 (Remote MCP) — happy + bad-token end-to-end via diff --git a/test/plan-count-completion.test.ts b/test/plan-count-completion.test.ts index 889f3d123..485096c4a 100644 --- a/test/plan-count-completion.test.ts +++ b/test/plan-count-completion.test.ts @@ -469,7 +469,7 @@ process.stdin.resume(); expect(fs.readFileSync(output, 'utf8')).toBe(REPORT); const recorded = fs.readFileSync(events, 'utf8').trim().split('\n').map(line => JSON.parse(line)); expect(recorded.map(e => e.name)).toEqual(['streamed-heading', 'pending-question', 'answered-second', 'partial-report', 'complete-report', ...(terminal === 'plan_ready' ? ['valid-native-ready'] : [])]); - expect(fs.readFileSync(record, 'utf8').trim().split('\n').map(line => JSON.parse(line))).toEqual(['/plan-eng-review\r', '1\r']); + expect(fs.readFileSync(record, 'utf8').trim().split('\n').map(line => JSON.parse(line))).toEqual(['/plan-eng-review\r', '1']); if (terminal === 'plan_ready') expect(observation.transcript.planReadyRequests.map(r => r.failed)).toEqual([true, false]); } finally { clearTimeout(timer); child.kill('SIGKILL'); await child.exited; fs.rmSync(dir, { recursive: true, force: true }); } })); diff --git a/test/plan-count-fixture.test.ts b/test/plan-count-fixture.test.ts index b5f983a79..4bdbc2054 100644 --- a/test/plan-count-fixture.test.ts +++ b/test/plan-count-fixture.test.ts @@ -256,8 +256,15 @@ const filePermission = () => '\nDo you want to make this edit to gstack-test-pla '❯1.Yes\n2.Yes,andswitchtoacceptedits(auto-approvefileeditsandcommonfilecommands)forthissession;Yes,and\n' + 'alwaysallowaccessto/tmp/fixtureforthissession\n3.No\nEsctocancel·Tabtoamend\n'; process.stdin.on('data', (data) => { - const input = data.toString('utf8'); - record({ type: 'input', data: input }); + const rawInput = data.toString('utf8'); + record({ type: 'input', data: rawInput }); + // These complete native panels accept a numeric shortcut immediately. + // Keep the fixture's existing answer handler sentinel internal; assertions + // below still check the exact bytes the real driver sent. + const instantNative = ['native-permission-policy', 'damaged-menu'].includes(process.env.FIXTURE_MODE) + || (process.env.FIXTURE_MODE === 'permission-lifecycle' && permissionStage === 'question') + || (process.env.FIXTURE_MODE === 'prerequisite' && (question === 2 || process.env.FIXTURE_CUSTOM !== 'true')); + const input = instantNative && /^[1-9]$/.test(rawInput) ? rawInput + '\r' : rawInput; if (!firstInput) { if (process.env.FIXTURE_MODE === 'native-permission-policy') { if (permissionStage === 'done') { record({ type: 'unexpected-policy-input', input }); return; } @@ -566,11 +573,12 @@ await Bun.write(${JSON.stringify(resultPath)}, JSON.stringify({ results, onboard expect(startup.skill).toContain(`name: ${item.skillName}`); expect(startup.sections).toBe(fs.readFileSync(path.join(ROOT, item.skillName, 'sections/review-sections.md'), 'utf8')); expect(events.filter((event) => event.type === 'input').map((event) => event.data).join('')) - .toBe(`/${item.skillName}\r` + (item.mode === 'prerequisite' ? `${item.custom ? 1 : 2}\r${item.skipIndex}\r` - : item.mode === 'permission-lifecycle' ? '1\r1\r2\r' + .toBe(`/${item.skillName}\r` + (item.mode === 'prerequisite' ? `${item.custom ? '1\r' : '2'}${item.skipIndex}` + : item.mode === 'permission-lifecycle' ? '1\r1\r2' : item.mode === 'damaged-submit' ? '\x1b[Z2\r\r' : item.mode === 'batched-finding' ? '2\r1\r' : item.mode === 'batched-mode' ? '1\r' - : ['direct-finding', 'failed-call', 'permission', 'late-mode', 'damaged-menu', 'native-permission-policy'].includes(item.mode) ? '2\r' : '')); + : ['damaged-menu', 'native-permission-policy'].includes(item.mode) ? '2' + : ['direct-finding', 'failed-call', 'permission', 'late-mode'].includes(item.mode) ? '2\r' : '')); expect(fs.existsSync(startup.cwd)).toBe(false); expect(fs.existsSync(startup.stateRoot)).toBe(false); expect(() => process.kill(startup.pid, 0)).toThrow(); diff --git a/test/plan-count-native-input.test.ts b/test/plan-count-native-input.test.ts new file mode 100644 index 000000000..250b86e31 --- /dev/null +++ b/test/plan-count-native-input.test.ts @@ -0,0 +1,504 @@ +import { describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { pathToFileURL } from 'node:url'; +import { + capturePlanCountQuestion, + planCountQuestionInput, +} from './helpers/claude-pty-runner'; +import type { NativePlanQuestionCall } from './helpers/plan-count-transcript'; +import { + nextCeoModeNavigation, + nextCeoPostureContinuation, + hasNativePostAnswerCeoPosture, +} from './helpers/ceo-mode-option'; +import { autoplanRoutingSetupInput } from './helpers/autoplan-setup-question'; + +const questions = [ + { + header: 'Routing rules', + question: 'Add skill routing rules? ', + options: [{ label: 'Enable' }, { label: 'Skip' }], + }, + { + header: 'Cross-project', + question: 'Enable cross-project learnings?', + options: [{ label: 'Enable' }, { label: 'Skip' }], + }, +]; +function menu(index: number) { + return ( + '← ☐ Routing rules ☐ Cross-project ✔ Submit →\n│ ' + + questions[index]!.question + + '\n❯1.Enable\n2.Skip\n3.Type something.\n4.Chat about this\nEnter to select · Tab/Arrow keys to navigate · Esc to cancel\n' + ); +} +const pending = (): NativePlanQuestionCall => ({ + sessionId: 'native', + toolUseId: 'packet', + answered: false, + questions, +}); + +describe('native AUQ accepts one action per displayed question', () => { + test('mode permission history releases a fresh identical viewport grant', () => { + const seen = new Set(); + const permission = 'Do you want to create PLAN.md?\n❯1.Yes\n2.Yes, and switch to accept edits\n3.No\nEsc to cancel · Tab to amend\n'; + expect(nextCeoModeNavigation(permission, 'HOLD SCOPE', seen, undefined, '')).toEqual({ + kind: 'permission', input: '1\r', + }); + expect(nextCeoModeNavigation(permission, 'HOLD SCOPE', seen, undefined, '').kind).toBe('wait'); + const completed = '⎿ Wrote1line\n'; + expect(nextCeoModeNavigation(permission, 'HOLD SCOPE', seen, undefined, completed)).toEqual({ + kind: 'permission', input: '1\r', + }); + expect(nextCeoModeNavigation(permission, 'HOLD SCOPE', seen, undefined, completed).kind).toBe('wait'); + }); + test('each native tab has separate UI identity, while retaining one call identity', () => { + const seen = new Set(); + const first = capturePlanCountQuestion(menu(0), seen, 0, true, pending())!; + const second = capturePlanCountQuestion(menu(1), seen, 1, true, pending())!; + expect(first.nativeQuestionIndex).toBe(0); + expect(second.nativeQuestionIndex).toBe(1); + expect(first.signature).not.toBe(second.signature); + expect(first.nativeCall!.toolUseId).toBe(second.nativeCall!.toolUseId); + expect(planCountQuestionInput(menu(0), first, 2)).toBe('2'); + expect(planCountQuestionInput(menu(1), second, 1)).toBe('1'); + expect( + capturePlanCountQuestion(menu(0), seen, 2, true, pending()), + ).toBeNull(); + expect( + capturePlanCountQuestion(menu(1), seen, 3, true, pending()), + ).toBeNull(); + }); + test('a delayed native record binds an already answered UI tab without another input', () => { + const seen = new Set(); + const ui = capturePlanCountQuestion(menu(0), seen, 0, true)!; + expect(ui.nativeCall).toBeUndefined(); + expect(planCountQuestionInput(menu(0), ui, 1)).toBe('1'); + expect( + capturePlanCountQuestion(menu(0), seen, 1, true, pending()), + ).toBeNull(); + expect( + capturePlanCountQuestion(menu(1), seen, 2, true, pending())! + .nativeQuestionIndex, + ).toBe(1); + }); + test('shared headers or options cannot borrow an unrelated native question', () => { + const foreign = menu(0).replace( + questions[0]!.question, + 'Which release should we inspect?', + ); + const fp = capturePlanCountQuestion( + foreign, + new Set(), + 0, + true, + pending(), + )!; + expect(fp.nativeCall).toBeUndefined(); + const prose = 'Which release?\n❯1.First\n2.Second\n'; + const plain = capturePlanCountQuestion(prose, new Set(), 0, true)!; + expect(planCountQuestionInput(prose, plain, 2)).toBe('2\r'); + }); + test('known multi-select keeps its existing protocol', () => { + const fp = capturePlanCountQuestion( + menu(0), + new Set(), + 0, + true, + pending(), + )!; + fp.nativeCall = { + ...fp.nativeCall!, + questions: fp.nativeCall!.questions.map((q) => ({ + ...q, + multiSelect: true, + })), + }; + expect(planCountQuestionInput(menu(0), fp, 1)).toBe('1\r'); + }); + test('a native checkbox packet with delayed metadata keeps its existing input protocol', () => { + for (const mark of [' ', '✓']) { + const screen = menu(0) + .replace('1.Enable', `1.[${mark}] Enable`) + .replace('2.Skip', '2.[ ] Skip'); + const fp = capturePlanCountQuestion(screen, new Set(), 0, true)!; + expect(fp.nativeCall).toBeUndefined(); + expect(planCountQuestionInput(screen, fp, 1)).toBe('1\r'); + } + }); + test('CEO mode and its next native question accept one digit; permission stays separate', () => { + const mode = { + header: 'Review mode', + question: 'Which review posture? ', + options: [{ label: 'HOLD SCOPE' }, { label: 'SCOPE EXPANSION' }], + }; + const call = { ...pending(), questions: [mode] }; + const screen = + '☐ Review mode\n' + + mode.question + + '\n❯1.HOLD SCOPE\n2.SCOPE EXPANSION\nEnter to select · ↑/↓ to navigate · Esc to cancel\n'; + const seen = new Set(); + const action = nextCeoModeNavigation(screen, 'SCOPE EXPANSION', seen, call); + expect(action.kind).toBe('mode'); + if (action.kind !== 'mode') throw Error('Expected native mode'); + expect(planCountQuestionInput(screen, action.question, action.index)).toBe( + '2', + ); + expect( + nextCeoModeNavigation(screen, 'SCOPE EXPANSION', seen, call).kind, + ).toBe('wait'); + const selected = { + ...call, + answered: true, + answeredAt: '2026-09-08T20:00:01Z', + answers: { [mode.question]: 'SCOPE EXPANSION' }, + }; + const next = { + ...pending(), + toolUseId: 'next', + questions: [ + { + header: 'Expansion', + question: 'Which idea would create 10x delight?', + options: [{ label: 'First idea' }, { label: 'Second idea' }], + }, + ], + }; + const transcript = { + status: 'ready' as const, + calls: [selected, next], + assistantMessages: [], + }; + const nextScreen = + '☐ Expansion\n' + + next.questions[0]!.question + + '\n❯1.First idea\n2.Second idea\nEnter to select · ↑/↓ to navigate · Esc to cancel\n'; + expect( + hasNativePostAnswerCeoPosture( + transcript, + 'SCOPE EXPANSION', + /delight/, + Date.parse('2026-09-08T20:00:00Z'), + ), + ).toBe(false); + expect( + nextCeoPostureContinuation( + nextScreen, + transcript, + 'SCOPE EXPANSION', + Date.parse('2026-09-08T20:00:00Z'), + new Set(), + false, + ), + ).toBe('question'); + const fp = capturePlanCountQuestion(nextScreen, new Set(), 0, false, next)!; + expect(planCountQuestionInput(nextScreen, fp, 1)).toBe('1'); + const permission = + 'Do you want to create PLAN.md?\n❯1.Yes\n2.Yes, and switch to accept edits\n3.No\nEsc to cancel · Tab to amend\n'; + const grant = nextCeoModeNavigation( + permission, + 'SCOPE EXPANSION', + new Set(), + next, + ); + expect(grant).toEqual({ kind: 'permission', input: '1\r' }); + }); + test('autoplan routing uses the native shortcut with early or delayed metadata, while prose is unchanged', () => { + const q = { + header: 'Routing rules', + question: + "gstack works best when your project's CLAUDE.md includes skill routing rules. Add them? ", + options: [ + { label: 'Add routing rules (Recommended)' }, + { label: 'Skip — manual invocation' }, + ], + }; + const native = { ...pending(), questions: [q] }; + const screen = + '☐ Routing rules\n' + + q.question + + '\n❯1.Add routing rules (Recommended)\n2.Skip — manual invocation\nEnter to select · ↑/↓ to navigate · Esc to cancel\n'; + expect(autoplanRoutingSetupInput(screen, new Set(), native)).toBe('1'); + expect(autoplanRoutingSetupInput(screen, new Set())).toBe('1'); + expect( + autoplanRoutingSetupInput( + screen.replace( + 'Enter to select · ↑/↓ to navigate · Esc to cancel\n', + '', + ), + new Set(), + ), + ).toBe('1\r'); + expect( + autoplanRoutingSetupInput( + screen.replace( + q.question, + 'Should the app change its product routing?', + ), + new Set(), + native, + ), + ).toBeNull(); + }); + + test.skipIf(process.platform === 'win32')( + 'real PTY completes two and four tabs once without skipping, overshooting or queuing Enter', + async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'native-auq-input-')); + const fake = path.join(dir, 'fake-claude'); + const worker = path.join(dir, 'worker.ts'); + const results = path.join(dir, 'results.json'); + const cases = [ + { name: 'two-tabs', count: 2, late: false, permission: true }, + { + name: 'four-tabs', + count: 4, + late: false, + permission: false, + long: true, + }, + { name: 'late-packet', count: 2, late: true, permission: false }, + ].map((item) => ({ + ...item, + record: path.join(dir, item.name + '.jsonl'), + })); + fs.writeFileSync( + fake, + `#!${process.execPath}\n` + + String.raw` +import * as fs from 'node:fs'; +import * as path from 'node:path'; +const item = JSON.parse(process.env.NATIVE_INPUT_CASE); +const record = (event) => + fs.appendFileSync(item.record, JSON.stringify(event) + '\n'); +const sessionId = 'native-input-' + process.pid; +const project = path.join(process.env.CLAUDE_CONFIG_DIR, 'projects', sessionId); +fs.mkdirSync(project, { recursive: true }); +const native = (role, content, extra = {}) => + fs.appendFileSync( + path.join(project, sessionId + '.jsonl'), + JSON.stringify({ + cwd: process.cwd(), + sessionId, + isSidechain: false, + message: { role, content }, + ...extra, + }) + '\n', + ); +native('assistant', [{ type: 'text', text: 'Fixture started.' }]); +record({ type: 'startup', pid: process.pid, cwd: process.cwd() }); +const questions = Array.from({ length: item.count }, (_, i) => ({ + header: 'Decision ' + i, + question: + (item.long && i === 0 + ? 'Additional concrete context for this decision. '.repeat(140) + : '') + + 'Which implementation fixes gap ' + + i + + '? ', + options: [{ label: 'First remedy' }, { label: 'Second remedy' }], +})); +const ask = () => + native('assistant', [ + { + type: 'tool_use', + id: 'packet', + name: 'AskUserQuestion', + input: { questions }, + }, + ]); +let started = false, + permission = item.permission, + index = 0, + answers = {}, + done = false; +const render = () => { + let screen = ''; + if (permission) + screen = + 'Do you want to create PLAN.md?\n❯1.Yes\n2.Yes, and switch to accept edits\n3.No\nEsc to cancel · Tab to amend\n'; + else if (index < questions.length) + screen = + '← ' + + questions + .map((q) => (answers[q.question] ? '☒' : '☐') + ' ' + q.header) + .join(' ') + + ' ✔ Submit →\n│ ' + + questions[index].question + + '\n❯1.First remedy\n2.Second remedy\n3.Type something.\n4.Chat about this\nEnter to select · Tab/Arrow keys to navigate · Esc to cancel\n'; + else if (index === questions.length) + screen = + '← ' + + questions + .map((q) => (answers[q.question] ? '☒' : '☐') + ' ' + q.header) + .join(' ') + + ' ✔ Submit →\nReview your answers\n' + + (Object.keys(answers).length < questions.length + ? '⚠ You have not answered all questions\n' + : '') + + 'Ready to submit your answers?\n❯1.Submit answers\n2.Cancel\n'; + record({ + type: 'screen', + index, + answers: Object.keys(answers), + blank: !screen, + }); + process.stdout.write('\x1b[2J\x1b[H' + screen.replace(/\n/g, '\r\n')); +}; +process.stdin.setRawMode?.(true); +process.stdin.on('data', (data) => { + const input = data.toString(); + record({ type: 'input', input }); + if (!started) { + started = true; + if (!item.late) ask(); + render(); + return; + } + if (done) { + record({ type: 'unexpected-after-completion', input }); + return; + } + if (permission) { + if (input !== '1\r') throw Error('permission protocol changed'); + permission = false; + render(); + return; + } + const rendered = index; + if (input === '\x1b[Z') { + index = Math.max(0, index - 1); + render(); + return; + } + if (rendered === questions.length) { + if (input !== '\r') throw Error('submit protocol changed'); + if (item.late) ask(); + native( + 'user', + [{ type: 'tool_result', tool_use_id: 'packet', content: 'Answered.' }], + { toolUseResult: { answers } }, + ); + record({ type: 'submitted', answers }); + done = true; + process.stdout.write('\x1b[2J\x1b[HGSTACK REVIEW REPORT\r\n'); + return; + } + // CLI2.1.257's numeric shortcut and select:accept both call onChange, + // whose set-answer reducer increments the current question index. Events + // in one chunk retain the rendered question closure until the next paint. + for (const key of input) { + if (rendered >= questions.length) break; + if (/[1-2]/.test(key) || key === '\r') { + answers[questions[rendered].question] = + questions[rendered].options[key === '2' ? 1 : 0].label; + index++; + } + } + render(); +}); +process.on('SIGINT', () => process.exit(0)); +process.stdin.resume(); +`, + ); + fs.chmodSync(fake, 0o755); + fs.writeFileSync( + worker, + `import {runPlanSkillCounting} from ${JSON.stringify(pathToFileURL(path.resolve(import.meta.dir, 'helpers/claude-pty-runner.ts')).href)}; +const cases=${JSON.stringify(cases)}; +const results = await Promise.all(cases.map(async item => ({ + name: item.name, + observation: await runPlanSkillCounting({ + skillName: 'plan-eng-review', + slashCommand: '/plan-eng-review', + followUpPrompt: 'Review only this fixture.', + isLastStep0AUQ: () => false, + isReviewAUQ: () => true, + firstAUQPick: () => 2, + reviewCountCeiling: 8, + timeoutMs: 22000, + env: { NATIVE_INPUT_CASE: JSON.stringify(item) }, + }), +}))); +await Bun.write(${JSON.stringify(results)},JSON.stringify(results));`, + ); + const child = Bun.spawn([process.execPath, worker], { + env: { + ...process.env, + BROWSE_TERMINAL_BINARY: fake, + EVALS_HERMETIC: '1', + }, + stdout: 'pipe', + stderr: 'pipe', + }); + const timer = setTimeout(() => child.kill('SIGKILL'), 35000); + try { + const [code, out, err] = await Promise.all([ + child.exited, + new Response(child.stdout).text(), + new Response(child.stderr).text(), + ]); + expect(code, out + err).toBe(0); + const report = JSON.parse(fs.readFileSync(results, 'utf8')); + for (const item of cases) { + const row = report.find((r) => r.name === item.name); + expect(row.observation.outcome, JSON.stringify(row.observation)).toBe( + 'completion_summary', + ); + expect(row.observation.reviewCount).toBe(1); + expect(row.observation.transcript.calls).toHaveLength(1); + expect( + row.observation.transcript.calls[0].unansweredQuestionIndices, + ).toEqual([]); + const events = fs + .readFileSync(item.record, 'utf8') + .trim() + .split('\n') + .map((line) => JSON.parse(line)); + expect( + events.filter((e) => e.type === 'input').map((e) => e.input), + ).toEqual([ + '/plan-eng-review\r', + ...(item.permission ? ['1\r'] : []), + '2', + ...Array(item.count - 1).fill('1'), + '\r', + ]); + expect(events.filter((e) => e.type === 'screen' && e.blank)).toEqual( + [], + ); + expect( + events.filter((e) => e.type === 'unexpected-after-completion'), + ).toEqual([]); + const submitted = events.filter((e) => e.type === 'submitted'); + expect(submitted).toHaveLength(1); + expect(Object.values(submitted[0].answers)).toEqual([ + 'Second remedy', + ...Array(item.count - 1).fill('First remedy'), + ]); + expect(() => process.kill(events[0].pid, 0)).toThrow(); + expect(fs.existsSync(events[0].cwd)).toBe(false); + } + } finally { + clearTimeout(timer); + child.kill('SIGKILL'); + for (const item of cases) { + if (!fs.existsSync(item.record)) continue; + const pid = JSON.parse( + fs.readFileSync(item.record, 'utf8').split('\n')[0]!, + ).pid; + try { + process.kill(pid, 'SIGKILL'); + } catch {} + } + fs.rmSync(dir, { recursive: true, force: true }); + } + }, + 40000, + ); +}); diff --git a/test/pty-screen.test.ts b/test/pty-screen.test.ts index 12569cb21..a601b9074 100644 --- a/test/pty-screen.test.ts +++ b/test/pty-screen.test.ts @@ -123,7 +123,7 @@ describe('owned PTY viewport', () => { expect(current.split('\n').map(line => line.trimEnd())).toEqual(frame.expected.map((line: string) => line.trimEnd())); expect(current).toContain("2. No thanks, I'll invoke skills manually"); expect(stripAnsi(frame.update)).toContain("2. N thanks, I'll invokeskillsmanually"); - expect(autoplanRoutingSetupInput(current, new Set())).toBe('1\r'); + expect(autoplanRoutingSetupInput(current, new Set())).toBe('1'); } finally { await screen.dispose(); } }); } diff --git a/test/run-in-background-guidance.test.ts b/test/run-in-background-guidance.test.ts index bd0f1e42d..47c537f1d 100644 --- a/test/run-in-background-guidance.test.ts +++ b/test/run-in-background-guidance.test.ts @@ -9,7 +9,8 @@ import * as path from 'path'; // producing a foreground run — the review army and autoplan dual-voice // steps silently launched specialists in the background and merged before // they completed. The only guidance that works post-2.1.198 is an explicit -// `run_in_background: false` on the Agent call. +// `run_in_background: false` (prose) or `"run_in_background": false` (JSON) +// on the Agent call. // // This tripwire pins the corrected phrasing in the generated skill output // and fails if the inverted form ever comes back through a template or @@ -59,6 +60,14 @@ const GENERATED_WITH_GUIDANCE = [ // SKILL.md so the regression can't migrate to another skill unnoticed. const INVERTED = /do not use\s+`?run_in_background`?/i; +// Both spellings describe the same boolean Agent argument. Keep the key and +// false token bounded so an unrelated key or quoted/string value cannot pass. +const EXPLICIT_FOREGROUND = /(?:\brun_in_background\b|"run_in_background")\s*:\s*false\b/; +function hasForegroundGuidance(content: string): boolean { + return EXPLICIT_FOREGROUND.test(content) && !INVERTED.test(content); +} + + function allGeneratedSkillFiles(): string[] { const out: string[] = []; for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) { @@ -80,10 +89,30 @@ function allGeneratedSkillFiles(): string[] { } describe('run_in_background guidance (#2440)', () => { + test('recognizes prose and actual JSON false without accepting missing, true, or inverted guidance', () => { + for (const guidance of [ + 'Pass `run_in_background: false` on the Agent call.', + 'Native subagent tool; Claude Code Agent arguments:\n```json\n{ "run_in_background": false }\n```\nSet on the call, not in prompt text.', + '{\n "run_in_background" :\n false\n}', + ]) expect(hasForegroundGuidance(guidance), guidance).toBe(true); + for (const guidance of [ + 'Dispatch via the Agent tool (foreground).', + 'run_in_background: true', + '{ "run_in_background": true }', + '{ "run_in_background": "false" }', + '{ "run_in_background": null }', + 'other_run_in_background: false', + 'run_in_background: falsehood', + 'Do NOT use `run_in_background`.', + 'Do NOT use run_in_background: false.', + 'Do NOT use `run_in_background`. { "run_in_background": false }', + ]) expect(hasForegroundGuidance(guidance), guidance).toBe(false); + }); + test('foreground-required skills instruct run_in_background: false explicitly', () => { for (const rel of GENERATED_WITH_GUIDANCE) { const content = fs.readFileSync(path.join(ROOT, rel), 'utf-8'); - expect(content).toContain('run_in_background: false'); + expect(hasForegroundGuidance(content), rel).toBe(true); } }); @@ -147,7 +176,7 @@ describe('run_in_background guidance (#2440)', () => { const rel = path.relative(ROOT, file).split(path.sep).join('/'); if (BACKGROUND_OK[rel]) continue; const content = fs.readFileSync(file, 'utf-8'); - if (DISPATCH_IMPERATIVE.test(content) && !content.includes('run_in_background: false')) { + if (DISPATCH_IMPERATIVE.test(content) && !hasForegroundGuidance(content)) { throw new Error( `${rel} contains an Agent-dispatch imperative (or bare "foreground" prose) but never states ` + '`run_in_background: false` — pin the flag at the dispatch site or add a reasoned BACKGROUND_OK ' + diff --git a/test/session-runner-tools.test.ts b/test/session-runner-tools.test.ts index 527007e46..bfcb0f002 100644 --- a/test/session-runner-tools.test.ts +++ b/test/session-runner-tools.test.ts @@ -52,11 +52,12 @@ const FAKE_CLAUDE = String.raw` outsideDisabled: /^codex_reviews:\s*disabled\s*$/m.test(config), }; fs.writeFileSync('observed.json', JSON.stringify(observed)); + const outputFile = fs.existsSync('active-plan-output') ? 'PLAN.md' : 'REPORT.md'; console.log(JSON.stringify({ type: 'system', subtype: 'init' })); if (fs.existsSync('fail-cli')) { const failure = fs.readFileSync('fail-cli', 'utf8'); const report = '## GSTACK REVIEW REPORT\nA report written before the run failed.\n'; - fs.writeFileSync('REPORT.md', report); + fs.writeFileSync(outputFile, report); console.log(JSON.stringify({ type: 'result', subtype: failure === 'error' ? 'error_during_execution' : 'success', is_error: failure !== 'nonzero', result: report, @@ -71,7 +72,7 @@ const FAKE_CLAUDE = String.raw` fs.writeFileSync('work-ready', ''); while (!fs.existsSync('release-report')) await Bun.sleep(5); } - fs.writeFileSync('REPORT.md', '## GSTACK REVIEW REPORT\nFull native review fixture.\nOutside review: ' + fs.writeFileSync(outputFile, '## GSTACK REVIEW REPORT\nFull native review fixture.\nOutside review: ' + (observed.outsideDisabled ? 'disabled' : 'default') + '\n'); console.log(JSON.stringify({ type: 'result', subtype: 'success', result: JSON.stringify(observed) })); } @@ -241,6 +242,24 @@ describe.skipIf(process.platform === 'win32')('session-runner explicit tool avai }); }); + test('the active plan can hold the final report without a second output file', async () => { + await withFakeClaude(async (dir, observed) => { + fs.writeFileSync(path.join(dir, 'PLAN.md'), '# Original plan\nA seeded defect.\n'); + fs.writeFileSync(path.join(dir, 'active-plan-output'), ''); + const result = await captureSectionReads({ + planDir: dir, skillName: 'plan-ceo-review', scenario: 'Review PLAN.md in full', + reportFile: 'PLAN.md', reportMarker: /^## GSTACK REVIEW REPORT\s*$/m, + testName: 'section-active-plan', nativeReviewOnly: true, timeout: 5_000, + }); + expect(result.reportProduced).toBe(true); + expect(result.output).toBe(fs.readFileSync(path.join(dir, 'PLAN.md'), 'utf8')); + expect(fs.existsSync(path.join(dir, 'REPORT.md'))).toBe(false); + expect(observed().prompt).toContain('to ' + path.join(dir, 'PLAN.md') + '.'); + expect(observed().prompt).toContain('After all required writes are complete'); + expect(result.readSections.has('review-sections.md')).toBe(true); + }); + }); + test('native-only capture gives its child real isolated disabled config and cleans only that state', async () => { await withFakeClaude(async (dir, observed) => { const hostState = path.join(dir, 'host-state'); diff --git a/test/skill-e2e-autoplan-chain.test.ts b/test/skill-e2e-autoplan-chain.test.ts index 030db1e2f..de6d024f9 100644 --- a/test/skill-e2e-autoplan-chain.test.ts +++ b/test/skill-e2e-autoplan-chain.test.ts @@ -46,7 +46,7 @@ import { createPlanCountSnapshotWriter } from './helpers/plan-count-artifacts'; const describeE2E = describeE2ETier('periodic'); const ROOT = path.resolve(import.meta.dir, '..'); -const UI_FIXTURE = path.join(ROOT, 'test', 'fixtures', 'plans', 'ui-heavy-feature.md'); +const UI_FIXTURE = path.join(ROOT, 'test', 'fixtures', 'plans', 'autoplan-dashboard.md'); function diagnosticTail(text: string): string { return stripVTControlCharacters(text) @@ -58,7 +58,8 @@ describeE2E('/autoplan chain ordering (periodic)', () => { test( 'phase completions are ordered: Phase 1 (CEO) before Phase 3 (Eng), Phase 2 (Design) between when present', async () => { - // UI-heavy fixture so Phase 2 runs. + // Chain-only fixture retains all new UI/API work and supplies existing + // application contracts; the shared design-scope fixture stays unchanged. const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-autoplan-chain-')); try { const gitRun = (args: string[]) => @@ -150,9 +151,10 @@ describeE2E('/autoplan chain ordering (periodic)', () => { // CLAUDE.md. Answer only that recognized setup prompt; review and // taste decisions remain autoplan's responsibility. The helper // deduplicates the complete question before returning an input. - const setupInput = autoplanRoutingSetupInput(visible, seenSetupQuestions); + const setupInput = autoplanRoutingSetupInput(visible, seenSetupQuestions, transcript.calls.find(call => !call.answered && !call.failed)); if (setupInput !== null) { - await selectPtyNumberedOption(session, Number(setupInput.trim())); + if (setupInput.includes('\r')) await selectPtyNumberedOption(session, Number(setupInput.trim())); + else session.send(setupInput); await Bun.sleep(2000); continue; } diff --git a/test/skill-e2e-plan-ceo-finding-count.test.ts b/test/skill-e2e-plan-ceo-finding-count.test.ts index 3931bbfce..e3e8b79c2 100644 --- a/test/skill-e2e-plan-ceo-finding-count.test.ts +++ b/test/skill-e2e-plan-ceo-finding-count.test.ts @@ -77,6 +77,13 @@ const planCeo5Findings = (planPath: string) => [ '## Existing contracts retained', 'The existing ingress middleware verifies the Stripe signature against the', 'raw request body and rejects invalid signatures before invoking handlers.', + 'The existing ingress forwards only `payment_intent.succeeded` events to', + 'this handler; other Stripe event types are acknowledged without invoking it.', + 'The existing payload adapter exposes `event.data.object.metadata.user_id`', + 'as `request.params.userId`. This params object is the parsed body-data map,', + 'not URL query/path parameters; all users share one webhook URL.', + 'The adapter forwards that external string unchanged. It does not cast,', + 'escape, or SQL-sanitize it; a valid signature does not make it safe for SQL.', 'The existing webhook event guard deduplicates deliveries by Stripe event ID,', 'and an existing per-user transaction lock serializes payment updates.', 'The new handler runs inside those unchanged guards; this plan does not', diff --git a/test/skill-e2e-plan-ceo-mode-routing.test.ts b/test/skill-e2e-plan-ceo-mode-routing.test.ts index e4f447908..5a7cd68dd 100644 --- a/test/skill-e2e-plan-ceo-mode-routing.test.ts +++ b/test/skill-e2e-plan-ceo-mode-routing.test.ts @@ -38,6 +38,9 @@ import { isNumberedOptionListVisible, isPlanReadyVisible, selectPtyNumberedOption, + planCountQuestionInput, + capturePlanCountQuestion, + type AskUserQuestionFingerprint, type ClaudePtySession, } from './helpers/claude-pty-runner'; import { hasNativePostAnswerCeoPosture, nextCeoModeNavigation, nextCeoPostureContinuation } from './helpers/ceo-mode-option'; @@ -89,8 +92,9 @@ async function navigateToModeAskUserQuestion( session: ClaudePtySession, since: number, targetMode: ModeCase['mode'], + cwd: string, opts: { maxNav?: number; budgetMs?: number } = {}, -): Promise<{ modeIndex: number; visibleAtMode: string }> { +): Promise<{ modeIndex: number; visibleAtMode: string; question: AskUserQuestionFingerprint }> { // /plan-ceo-review's mode AskUserQuestion (Step 0F) sits behind several preamble // and Step 0A-0C-bis gates: telemetry, proactive, routing, vendoring, // brain privacy, office-hours offer, premise challenge (3 questions), @@ -109,8 +113,10 @@ async function navigateToModeAskUserQuestion( ); } await Bun.sleep(2000); - const visible = session.visibleSince(since); - const action = nextCeoModeNavigation(visible, targetMode, seenQuestions); + const visible = await session.currentScreen(); + const transcript = session.hermeticConfigDir ? readPlanCountTranscript(session.hermeticConfigDir, cwd) : null; + const pending = transcript?.calls.find(call => !call.answered && !call.failed); + const action = nextCeoModeNavigation(visible, targetMode, seenQuestions, pending, session.visibleText()); if (action.kind === 'wait') continue; // Native permission and multi-question Submit menus are controls, not // review questions, so neither consumes the navigation question budget. @@ -120,7 +126,7 @@ async function navigateToModeAskUserQuestion( continue; } if (action.kind === 'mode') { - return { modeIndex: action.index, visibleAtMode: visible }; + return { modeIndex: action.index, visibleAtMode: visible, question: action.question }; } // Not the mode AskUserQuestion — answer with option 1 (recommended) and continue. @@ -132,7 +138,7 @@ async function navigateToModeAskUserQuestion( ); } priorAnswered++; - session.send('1\r'); + session.send(planCountQuestionInput(visible, action.question, 1)); // Give the agent a beat to advance before re-polling. await Bun.sleep(2000); } @@ -155,17 +161,20 @@ describeE2E('/plan-ceo-review mode routing (gate)', () => { permissionMode: 'plan', timeoutMs: CAPTURE_LONG_MS, seedSkills: true, + observeScreen: true, }); await Bun.sleep(8000); const since = session.mark(); session.send('/plan-ceo-review\r'); - const { modeIndex, visibleAtMode } = await navigateToModeAskUserQuestion(session, since, c.mode); + const { modeIndex, visibleAtMode, question } = await navigateToModeAskUserQuestion(session, since, c.mode, fixture.cwd); - // The digit redraws the menu before Enter confirms it. Start after - // confirmation so its mode names cannot count as assistant posture. + // Native shortcuts accept immediately. Preserve the captured question + // identity; only the legacy prose-menu protocol needs Enter. const selectionStartedAt = Date.now(); - await selectPtyNumberedOption(session, modeIndex); + const modeInput = planCountQuestionInput(visibleAtMode, question, modeIndex); + if (modeInput.includes('\r')) await selectPtyNumberedOption(session, modeIndex); + else session.send(modeInput); const sincePick = session.mark(); // Wait for downstream evidence: either next AskUserQuestion or plan_ready or @@ -193,11 +202,19 @@ describeE2E('/plan-ceo-review mode routing (gate)', () => { postureMatched = true; break; } - const continuation = nextCeoPostureContinuation(downstreamSnapshot, transcript, - c.mode, selectionStartedAt, seenDownstream, continuedQuestion); + const currentInput = await session.currentScreen(); + const continuation = nextCeoPostureContinuation(currentInput, transcript, + c.mode, selectionStartedAt, seenDownstream, continuedQuestion, session.visibleText()); if (continuation !== null) { if (continuation === 'question') continuedQuestion = true; - await selectPtyNumberedOption(session, 1); + if (continuation === 'permission') await selectPtyNumberedOption(session, 1); + else { + const pending = transcript.calls.find(call => !call.answered && !call.failed); + const question = capturePlanCountQuestion(currentInput, new Set(), 0, false, pending)!; + const input = planCountQuestionInput(currentInput, question, 1); + if (input.includes('\r')) await selectPtyNumberedOption(session, 1); + else session.send(input); + } continue; } // Don't bail early on plan_ready alone — the posture text may diff --git a/test/skill-e2e-plan-ceo-review-section-loading.test.ts b/test/skill-e2e-plan-ceo-review-section-loading.test.ts index 4e53ab655..fdc3513f6 100644 --- a/test/skill-e2e-plan-ceo-review-section-loading.test.ts +++ b/test/skill-e2e-plan-ceo-review-section-loading.test.ts @@ -59,7 +59,10 @@ describeE2E('/plan-ceo-review section-loading E2E (periodic, SDK capture)', () = planDir, skillName: 'plan-ceo-review', scenario: - 'Review the plan in PLAN.md. Hold the current scope (HOLD SCOPE mode) — do not challenge or expand scope. Run the full CEO review and produce the review report.', + 'Review the plan in PLAN.md. Hold the current scope (HOLD SCOPE mode) — do not challenge or expand scope. Run the full CEO review. PLAN.md is both the active plan and final output: preserve and amend its plan content, then include the full review report there.', + // The skill appends its report to the active plan. Use that same + // artifact so the capture does not request a second report write. + reportFile: 'PLAN.md', requiredSections: REQUIRED_SECTIONS, reportMarker: /^## GSTACK REVIEW REPORT\s*$/m, testName: 'plan-ceo-section-loading',