test: judge plan-count completion on structured evidence, not wording

Replaying run 36385945043's two Design attempts showed the existing routes
rejected correct endings: attempt 1 at the typed-completion path field
('- Reviewed plan written to …' is not a 'Plan written to' line), attempt 2
at the leading-fence veto (its final message opens with the dashboard).

nativePlanTerminalPreconditions is the structural prefix of
hasNativePlanTerminal (behavior unchanged). structuredPlanCompletion adds,
inside the existing nativeSummary branch: a complete report (Design
binding for Design), a completed review-log row for the expected skill
appended during this attempt under the child's GSTACK_HOME/project slug
(resolved with bin/gstack-slug) and stamped with the fixture commit, timed
between the report/last answer (second resolution) and the final native
message, a final message with stop_reason end_turn (now carried on public
transcript messages), and no visible question or permission prompt.

Timeout summaries add idleFor and lastTerminalCandidate. Terminal and throw
captures copy the plan file and review-log rows into the artifact
directory; copies are best-effort and recorded in evidence-copy.json.
Free regressions: both captured Design endings (trimmed fixture with
provenance; report, row and end_turn reconstructed and labelled), the
negative controls, and real-PTY completion/timeout runs through the real
review logger.
This commit is contained in:
garrytan committed 2026-09-29 15:04:28 +00:00
1 parent a0aa1e8f39
commit f1eeec384e
6 files changed
+1172 -32

No files matched your search

+4 -2
View File
@@ -40,7 +40,8 @@ export interface NativePublicToolEvent {
export interface PlanCountTranscript {
status: 'missing' | 'ready' | 'error';
calls: NativePlanQuestionCall[];
assistantMessages: Array<{ sessionId: string; text: string; timestamp: string }>;
/** stopReason is the native record's stop_reason when it carries one (e.g. end_turn). */
assistantMessages: Array<{ sessionId: string; text: string; timestamp: string; stopReason?: string }>;
/** Actual native plan-mode approval requests; pending is the UI gate, never an AUQ. */
planReadyRequests?: Array<{ sessionId: string; toolUseId: string; timestamp: string; failed: boolean; source?: 'pre_tool_use' }>;
error?: string;
@@ -348,7 +349,8 @@ export function readPlanCountTranscript(configDir: string, cwd: string,
const text = block.type === 'text' && typeof block.text === 'string' && block.text.trim()
? block.text : publicNarrationText(block);
if (text) {
assistantMessages.push({ sessionId: record.sessionId, text, timestamp: record.timestamp });
assistantMessages.push({ sessionId: record.sessionId, text, timestamp: record.timestamp,
...(typeof record.message.stop_reason === 'string' ? { stopReason: record.message.stop_reason } : {}) });
ordered({ kind: 'message', sessionId: record.sessionId, text, timestamp: record.timestamp });
}
}