test(plan-mode-no-op): the CEO sub-test reviews a fixed pasted plan

Without a plan, plan-ceo-review reviewed the checkout's branch diff, so its
pre-question work (audit and landscape searches) grew with whatever PR the
gate ran on. On this PR it timed out at 300s in 3 of 10 trials while still
researching the diff (ship-measure baseline 7/10). With a fixed small plan,
the same way the devex sub-test already runs, it measured 10/10 (CEO
sub-test 83-255s). No budget or threshold changed.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
garrytanandcapy-ai[bot] committed 2026-10-07 02:52:46 +00:00
1 parent d6250a058a
commit 36c0cf1579
1 file changed
+16
+16
View File
@@ -26,6 +26,12 @@
* pasted developer-facing plan: without one it reviews the checkout's
* branch diff, and a tests-only diff legitimately ends at its
* applicability gate, which is neither 'asked' nor 'plan_ready'.
* - plan-ceo-review reviews a pasted plan too. Without one it reviews the
* checkout's branch diff, so its work before the first question (the
* pre-review audit and landscape searches) grew with whatever PR the gate
* ran on: on a 1,900-line PR it timed out at 300s in 3 of 10 measured
* trials while still researching that diff. A fixed small plan keeps the
* work the same on every branch.
* - named-target case: a pasted draft (initialPlanContent) IS an
* explicitly-named target, so the gate must NOT ask — and the review
* must actually consume the pasted content.
@@ -62,6 +68,15 @@ weekly export emails. One new component, one route, one test file.
- test/${SEED_TOKEN}.test.tsx (new)
`;
const CEO_PLAN = `
# Plan: weekly digest email
## Scope
Send each workspace owner a weekly digest email listing the tasks closed
that week. One scheduled job, one email template, one opt-out toggle in
settings.
`;
const DEVEX_PLAN = `
# Plan: tasks export command
@@ -96,6 +111,7 @@ describeE2E('plan-mode-info no-op outside plan mode (gate regression)', () => {
? { extraArgs: ['--disallowedTools', 'AskUserQuestion'], requireProseEvidence: true }
: {}),
...(skillName === 'plan-devex-review' ? { initialPlanContent: DEVEX_PLAN } : {}),
...(skillName === 'plan-ceo-review' ? { initialPlanContent: CEO_PLAN } : {}),
});
if (obs.outcome === 'silent_write' || obs.outcome === 'exited' || obs.outcome === 'timeout') {