mirror of
https://github.com/garrytan/gstack.git
synced 2026-10-31 14:06:50 +01:00
test: qa-quick sets a caller probe limit and drains past its session budget
C2 raised Browser Quick's native budget to 180 s. qa-quick is the only gate case on the native budget, and its wall went 194 s (main periodic, pre-C2) -> 254 s and 274 s (this PR's runs) -> 300 s session timeout (run 37242691913): the agent now probes every navigation target with a checkpoint, then needs ~80 s to report. Like qa-only-no-fix, the eval now sets a caller probe limit (30 s), scopes the fixture, and gives bun's outer timeout the session drain margin instead of racing the 300 s session. The product's 180 s budget and the session budget are unchanged. Diagnostic rerun passed in 195 s.
This commit is contained in:
1 parent
adc683e349
commit
f8dc64031e
1 file changed
+4
-1
@@ -68,6 +68,9 @@ qa is a carved skill: when SKILL.md tells you to Read ~/.claude/skills/gstack/qa
|
||||
Skip the preamble bash block, lake intro, telemetry, and contributor mode sections — go straight to the QA workflow.
|
||||
|
||||
Run a Quick-depth QA test on ${testServer.url}/basic.html
|
||||
This fixture is one static page with two same-origin navigation links (/page1, /page2) and one external link; Quick scope is that page and its two same-origin links.
|
||||
As the caller, I set a 30-second total probe limit for this run (shorter than Browser Quick's own budget), so start the deadline with SECONDS=30.
|
||||
Write concise checkpoints and a concise report, retaining required evidence, scores and untested-coverage fields; cite evidence IDs instead of repeating commands.
|
||||
Do NOT use AskUserQuestion — run Quick tier directly.
|
||||
Do NOT try to start a server or discover ports — the URL above is ready.
|
||||
Write your report to ${qaDir}/qa-reports/qa-report.md`,
|
||||
@@ -88,7 +91,7 @@ Write your report to ${qaDir}/qa-reports/qa-report.md`,
|
||||
}
|
||||
// Accept error_max_turns — the agent doing thorough QA work is not a failure
|
||||
expect(['success', 'error_max_turns']).toContain(result.exitReason);
|
||||
}, CAPTURE_MS);
|
||||
}, CAPTURE_MS + SESSION_DRAIN_GRACE_MS + 5_000);
|
||||
}, browserSelected);
|
||||
|
||||
// --- QA-Only E2E (report-only, no fixes) ---
|
||||
|
||||
Reference in new issue
Block a user