From f8dc64031e9ad514cee5575843ba53b2da1e0fe1 Mon Sep 17 00:00:00 2001 From: garrytan Date: Mon, 5 Oct 2026 00:06:55 +0000 Subject: [PATCH] test: qa-quick sets a caller probe limit and drains past its session budget C2 raised Browser Quick's native budget to 180 s. qa-quick is the only gate case on the native budget, and its wall went 194 s (main periodic, pre-C2) -> 254 s and 274 s (this PR's runs) -> 300 s session timeout (run 37242691913): the agent now probes every navigation target with a checkpoint, then needs ~80 s to report. Like qa-only-no-fix, the eval now sets a caller probe limit (30 s), scopes the fixture, and gives bun's outer timeout the session drain margin instead of racing the 300 s session. The product's 180 s budget and the session budget are unchanged. Diagnostic rerun passed in 195 s. --- test/skill-e2e-qa-workflow.test.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/test/skill-e2e-qa-workflow.test.ts b/test/skill-e2e-qa-workflow.test.ts index 39617321b..57d8e8988 100644 --- a/test/skill-e2e-qa-workflow.test.ts +++ b/test/skill-e2e-qa-workflow.test.ts @@ -68,6 +68,9 @@ qa is a carved skill: when SKILL.md tells you to Read ~/.claude/skills/gstack/qa Skip the preamble bash block, lake intro, telemetry, and contributor mode sections — go straight to the QA workflow. Run a Quick-depth QA test on ${testServer.url}/basic.html +This fixture is one static page with two same-origin navigation links (/page1, /page2) and one external link; Quick scope is that page and its two same-origin links. +As the caller, I set a 30-second total probe limit for this run (shorter than Browser Quick's own budget), so start the deadline with SECONDS=30. +Write concise checkpoints and a concise report, retaining required evidence, scores and untested-coverage fields; cite evidence IDs instead of repeating commands. Do NOT use AskUserQuestion — run Quick tier directly. Do NOT try to start a server or discover ports — the URL above is ready. Write your report to ${qaDir}/qa-reports/qa-report.md`, @@ -88,7 +91,7 @@ Write your report to ${qaDir}/qa-reports/qa-report.md`, } // Accept error_max_turns — the agent doing thorough QA work is not a failure expect(['success', 'error_max_turns']).toContain(result.exitReason); - }, CAPTURE_MS); + }, CAPTURE_MS + SESSION_DRAIN_GRACE_MS + 5_000); }, browserSelected); // --- QA-Only E2E (report-only, no fixes) ---