test: PTY harness handles clipped reviews and bundled setup tabs; AUQ judge uses structured output; design-consultation carve declines optional outside voices

- ceo mode routing: a Submit review taller than the viewport, a setup tab
  bundled after the mode tab, and a clip through the mode question each hung
  or misread the run; the native answer is still verified after Submit.
- judgeRecommendation requests a 1-5 enum schema; a malformed Haiku reply had
  scored substance 0 for a 4/5 brief. Judge failures now propagate.
- carve section-loading for design-consultation declines the optional outside
  voices (a supported path) and treats DESIGN.md as the report; timeout unchanged.
The Step 0E handoff defect is not fixed (0/15 samples across four wordings,
none shipped) and is filed in TODOS.
This commit is contained in:
garrytan committed 2026-09-30 12:52:56 +00:00
1 parent dfe5e733fb
commit a27365bffe
16 files changed
+893 -28

No files matched your search

+4 -7
View File
@@ -53,7 +53,8 @@ export function scoreAuqFormat(text: string): { present: number; total: number;
* whether the ORIGINAL used the literal "because" — a soft style signal, since
* the format spec prefers it and the voice rule forbids the em-dash form.
*
* This does NOT touch judgeRecommendation or its pinned fixtures.
* This does NOT touch judgeRecommendation or its pinned fixtures. A judge
* failure propagates with its cause; it is never reported as substance 0.
*/
export async function gradeAuqRecommendation(
text: string,
@@ -75,12 +76,8 @@ export async function gradeAuqRecommendation(
}
}
try {
const r = await judgeRecommendation(graded);
return { substance: r.reason_substance, present: r.present, hadLiteralBecause, reason: r.reason_text };
} catch {
return { substance: 0, present: !!recLine, hadLiteralBecause, reason: '' };
}
const r = await judgeRecommendation(graded);
return { substance: r.reason_substance, present: r.present, hadLiteralBecause, reason: r.reason_text };
}
/**