evals: SDK runner default Opus -> Sonnet (D1a)

agent-sdk-runner defaulted to Opus 4.7 while session-runner (the claude
-p path) defaulted to Sonnet — an inconsistency between the two runners,
not a decision anyone made. Unpinned tests were implicitly asserting the
expensive model. The 30+ tests that genuinely need Opus already pin it
via opts.model. Re-baselined by the periodic run in this branch's final
verification; regressors get explicit Opus pins.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-15 09:14:36 -07:00
co-authored by Claude Fable 5
parent d464412010
commit 07f7d30e80
2 changed files with 7 additions and 4 deletions
+3 -3
View File
@@ -191,7 +191,7 @@ This is a capture test, not an interactive session. Skip any system-audit / envi
timeout: 240_000,
testName: opts.testName,
runId: opts.runId,
model: opts.model ?? 'claude-opus-4-7',
model: opts.model ?? 'claude-sonnet-4-6', // D1a: align with session-runner default; Opus is opt-in
});
try {
@@ -253,7 +253,7 @@ Rules for this run:
timeout: opts.timeout ?? 300_000,
testName: opts.testName,
runId: opts.runId,
model: opts.model ?? 'claude-opus-4-7',
model: opts.model ?? 'claude-sonnet-4-6', // D1a: align with session-runner default; Opus is opt-in
});
const readSections = new Set<string>();
@@ -334,7 +334,7 @@ Write the verbatim text of that AskUserQuestion (the full decision brief: title,
timeout: 240_000,
testName: opts.testName,
runId: opts.runId,
model: opts.model ?? 'claude-opus-4-7',
model: opts.model ?? 'claude-sonnet-4-6', // D1a: align with session-runner default; Opus is opt-in
});
try {