test(pty): detect stripAnsi-collapsed prose AUQs + judge spinner-precedence

The plan-eng/plan-design plan-mode + finding-floor smokes timed out even when
the skill HAD rendered a complete prose AskUserQuestion and was waiting: the PTY
strips cursor-positioning escapes, collapsing the option newlines/spaces so
"A) ..." arrives as "A(recommended)" / "-B:" and "Reply with A, B, or C" as
"ReplywithA,B,orC". Every line-anchored detector (Patterns 1-3) returns false on
those bytes, so proseAUQEverObserved never latched and the run timed out on a
question that was already on screen.

Add Pattern 4/5: a two-signal collapsed-form detector — a reply/recommendation
marker (space-insensitive "reply with [A-D]", "Recommendation:", or
"(recommended)") AND 2+ distinct A-D letters each punctuated by ) : or (. The
conjunction is what separates a real AUQ from incidental report prose; verified
true on the verbatim failing-run buffers where Patterns 1-3 return false.

Also fix the Haiku judge spinner bias: of 614 verdicts, 569 were 'working' and
95 of those noted a question was visible — Claude Code keeps the spinner
animating at an idle prose decision, so the judge coin-flipped. Add a precedence
override: when an option list AND a Recommendation/Reply instruction are both
visible, classify WAITING even with spinner glyphs. Kept the strict dual-signal
gate (never option-list-alone) so auto-decide-preserved doesn't flip.

5 unit tests pin the two-signal contract (2 true on real collapsed bytes, 3
false guards). 90 -> 95 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-06-18 00:58:50 -07:00
co-authored by Claude Opus 4.8
parent f436aab0db
commit be67fcce0f
2 changed files with 86 additions and 1 deletions
@@ -349,6 +349,51 @@ What should we do about this?
// The 1. cursor gate fires first — native list handling owns this.
expect(isProseAUQVisible(sample)).toBe(false);
});
// Pattern 4/5: collapsed-form prose AUQ. stripAnsi destroys the newlines +
// inter-word spaces, so a real prose AUQ arrives collapsed and defeats the
// line-anchored Patterns 1-3. These are the dominant Shape-B render mode in
// the plan-design smoke + floor timeouts — verbatim de-spinnered bytes from
// the real failing runs (bdm3sucql.output).
test('matches the real collapsed floor render (colon-delimited, Pattern 4/5)', () => {
const sample =
'The review is blocked on D1—reply withA, B, r Cabovetocontinue:' +
'- A(recommended): Spec thefull P1AskUserQuestioncopy in this review' +
'-B:LeaveP1copytotheimplementerwithstructuralrequirements' +
'C: Add a placeholder template to the plan';
expect(isProseAUQVisible(sample)).toBe(true);
});
test('matches the real collapsed plan-mode render (Recommendation + collapsed A)/B), Pattern 4/5)', () => {
const sample =
'Recommendation:A—writethecopynow.(recommended)A) Writ the fullcopy in thisdesign review— now.' +
'(recommended) Completeness:10/10 B) Leveit to theimplemente — task spec is enough.' +
'Reply withA (write the copy now)orB(leavetoimplementer)';
expect(isProseAUQVisible(sample)).toBe(true);
});
test('collapsed-form requires BOTH signals — single B) + word "recommendation" stays false', () => {
// Only one punctuated letter marker: the two-signal contract is not met.
const sample =
'We should consider option B) here. My recommendation is to do it now.';
expect(isProseAUQVisible(sample)).toBe(false);
});
test('collapsed-form requires letter punctuation — comma-only "ReplywithA,B,orC" stays false', () => {
// Reply-instruction present, but the letters carry no ) : or ( punctuation,
// so they could be incidental enumerations in running prose. Stays false.
const sample = 'ReplywithA,B,orC';
expect(isProseAUQVisible(sample)).toBe(false);
});
test('collapsed-form does not regress the existing FP guard (see option B) ... point A))', () => {
// The classic citation FP: a model referencing prior options in prose.
// No reply-instruction / recommendation marker on its own line, so the
// collapsed-form signal does not fire either.
const sample =
'As noted (see option B) above, and the earlier point A) we discussed, this is fine.';
expect(isProseAUQVisible(sample)).toBe(false);
});
});
describe('classifyVisible (runtime path through the runner classifier)', () => {