fix(evals): cut path variance at its measured sources

- gstack-qa-evidence capture prints startedAt/completedAt/durationMs and, for
  --deadline captures, remainingMs; the functional report takes durations from
  them. The section clock notice asks for one clock read up front instead of one
  after every checkpoint (QA runs spent 7-14% of tool calls on date -u).
- ship plan-completion: skip the audit dispatch when discovery already found no
  plan (the dispatch-vs-skip conflict produced an optional 60-100 s subagent).
- materialize/checkpoint validation errors state the expected schema, so a
  rejected annotations file is fixable in one call instead of blocking the phase.
- session-runner counts turns from the transcript when a run times out, so
  timeouts stop reporting 'turn 0'.
This commit is contained in:
garrytan committed 2026-09-30 15:59:58 +00:00
1 parent a8a32e3216
commit 79092b22ea
8 files changed
+36 -22

No files matched your search

+3 -2
View File
@@ -293,7 +293,7 @@ Runner entry UTC: ${new Date(startTime).toISOString()}
Hard deadline UTC: ${new Date(deadline).toISOString()}
Completion reserve starts UTC: ${new Date(deadline - reserve).toISOString()}
Setup, CLI startup and API queueing consume this same window; it never resets.
Before source Reads and after each saved checkpoint, use Bash to run exactly \`date -u +%Y-%m-%dT%H:%M:%SZ\`. Compare that observed UTC time with the times above. When remaining time is at most ${reserve / 1000} seconds, prioritize the remaining required completion outputs and verification. No required content or gate may be skipped. If the clock read fails, report timing unavailable; do not invent remaining time or restart the deadline.`;
Before source Reads, use Bash to run exactly \`date -u +%Y-%m-%dT%H:%M:%SZ\`. After each saved checkpoint, compare the latest evidence capture's printed completedAt with the times above; run that clock read again only when no capture has completed since your last clock read. When remaining time is at most ${reserve / 1000} seconds, prioritize the remaining required completion outputs and verification. No required content or gate may be skipped. If the clock read fails, report timing unavailable; do not invent remaining time or restart the deadline.`;
systemPrompt = systemPrompt ? `${systemPrompt}\n\n${notice}` : notice;
}
@@ -696,7 +696,8 @@ Before source Reads and after each saved checkpoint, use Bash to run exactly \`d
}
// Cost from result line (exact) or estimate from chars
const turnsUsed = resultLine?.num_turns || 0;
const turnsUsed = resultLine?.num_turns
|| new Set(transcript.filter(event => event.type === 'assistant' && !event.parent_tool_use_id).map(event => event.message?.id)).size;
const estimatedCost = resultLine?.total_cost_usd || 0;
const inputChars = prompt.length;
const outputChars = (resultLine?.result || '').length;