test: qa-quick sets a caller probe limit and drains past its session budget

C2 raised Browser Quick's native budget to 180 s. qa-quick is the only gate case
on the native budget, and its wall went 194 s (main periodic, pre-C2) -> 254 s and
274 s (this PR's runs) -> 300 s session timeout (run 37242691913): the agent now
probes every navigation target with a checkpoint, then needs ~80 s to report.
Like qa-only-no-fix, the eval now sets a caller probe limit (30 s), scopes the
fixture, and gives bun's outer timeout the session drain margin instead of racing
the 300 s session. The product's 180 s budget and the session budget are unchanged.
Diagnostic rerun passed in 195 s.
This commit is contained in:
garrytan committed 2026-10-05 00:06:55 +00:00
1 parent adc683e349
commit f8dc64031e
1 file changed
+4 -1
+4 -1
View File
@@ -68,6 +68,9 @@ qa is a carved skill: when SKILL.md tells you to Read ~/.claude/skills/gstack/qa
Skip the preamble bash block, lake intro, telemetry, and contributor mode sections — go straight to the QA workflow.
Run a Quick-depth QA test on ${testServer.url}/basic.html
This fixture is one static page with two same-origin navigation links (/page1, /page2) and one external link; Quick scope is that page and its two same-origin links.
As the caller, I set a 30-second total probe limit for this run (shorter than Browser Quick's own budget), so start the deadline with SECONDS=30.
Write concise checkpoints and a concise report, retaining required evidence, scores and untested-coverage fields; cite evidence IDs instead of repeating commands.
Do NOT use AskUserQuestion — run Quick tier directly.
Do NOT try to start a server or discover ports — the URL above is ready.
Write your report to ${qaDir}/qa-reports/qa-report.md`,
@@ -88,7 +91,7 @@ Write your report to ${qaDir}/qa-reports/qa-report.md`,
}
// Accept error_max_turns — the agent doing thorough QA work is not a failure
expect(['success', 'error_max_turns']).toContain(result.exitReason);
}, CAPTURE_MS);
}, CAPTURE_MS + SESSION_DRAIN_GRACE_MS + 5_000);
}, browserSelected);
// --- QA-Only E2E (report-only, no fixes) ---