fix(evals): parse single-logical-line AskUserQuestions in the PTY runner

When the PTY reflows a boxed AUQ, ALL options land on ONE logical line
after stripAnsi — parseNumberedOptions parsed one option per line, found
only '1.', and the >=2 check failed forever while the correct question
sat on screen (plan-design-with-ui timed out this way twice, with the
rendered scope-gate AUQ visible in both failure buffers). The cursor
line is now parsed as a stream of ascending N. tokens; DEC cursor-
visibility residue is stripped before matching; plan-design-with-ui's
budgets grow to fit observed ~6min preamble+thinking latency. Pinned by
test/pty-auq-single-line.test.ts using the real failure buffers; all 142
existing parser-consumer unit tests still green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-13 10:34:25 -07:00
co-authored by Claude Fable 5
parent bd11416d80
commit 828b229900
3 changed files with 120 additions and 27 deletions
+7 -3
View File
@@ -44,7 +44,7 @@ describeE2E('/plan-design-review with UI scope (gate)', () => {
const session = await launchClaudePty({
permissionMode: 'plan',
cwd: ROOT,
timeoutMs: 480_000,
timeoutMs: 720_000,
seedSkills: true,
});
@@ -71,7 +71,11 @@ describeE2E('/plan-design-review with UI scope (gate)', () => {
`Reference plan file: ${fixtureRelPath}\r`
);
const budgetMs = 360_000;
// 600s, not 360s: the skill preamble (update-check, session bookkeeping,
// learnings) plus extended model thinking can take ~6 minutes before the
// scope-gate AskUserQuestion renders — a 360s budget expired seconds
// before the (correct) AUQ appeared in the observed failure transcript.
const budgetMs = 600_000;
const start = Date.now();
let lastPermSig = '';
while (Date.now() - start < budgetMs) {
@@ -146,6 +150,6 @@ describeE2E('/plan-design-review with UI scope (gate)', () => {
);
}
},
540_000,
780_000,
);
});