mirror of
https://github.com/garrytan/gstack.git
synced 2026-11-20 23:23:26 +01:00
v1.91.52.0 fix(qa): annotations limits shape; qa-quick does less work in the same 300 s budget (#3081)
* test(qa-quick): run the fixture in a committed git repository
/qa runs inside a repository: its clean-tree check and the evidence
recorder's revision come from git. The qa-quick working directory was a
bare temp dir, so all 15 CI transcripts read (runs 37374409928 to
37744583272) hit 'revision is required when git rev-parse HEAD is
unavailable' on the first materialize and spent a retry (6-46 s) plus a
learnings-log of the pitfall. qa-only-no-fix already commits its fixture.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* test(qa-quick): scope Quick to the fixture page, not its 404 links
basic.html's /page1 and /page2 links 404 on the test server. With the
links in scope, the 15 CI transcripts spent 55-75 s of the 30 s caller
probe window and its wrap-up navigating, screenshotting and checkpointing
those 404s, then triaging and reporting them as issues. The case proves
/qa Quick completes end to end (fallback browser, deadline guard,
checkpoint, evidence materialize, report); link-following and defect
detection are covered by the qa-b6/b7/b8 and qa-fix-loop cases. The
links are now listed as untested coverage.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* fix(qa): say that annotations.json limits is a non-empty string array
exploratory.md §4 gave the annotations shape as {evidence: [...], limits}
with 'checkpoints in limits', but never said what type limits is. The
materialize validator (lib/qa-evidence.ts) wants a non-empty array of
strings. In 11 of 15 qa-quick CI transcripts the agent's first
annotations.json wrote limits as an object, failed with 'need limits
(non-empty string array)', and spent a retry. The section now shows the
shape with a one-line example and says it is never an object. Shared by
/qa and /qa-only through the QA_EXPLORATORY resolver.
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
* v1.91.52.0 fix(qa): annotations limits shape; qa-quick does less work in the same 300 s budget
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
---------
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
c7edf46bcf
commit
d082eccc50
8 files changed
+46
-7
No files matched your search
@@ -49,6 +49,13 @@ describeIfSelected('QA skill E2E', ['qa-quick'], () => {
|
||||
|
||||
// Create report directory
|
||||
fs.mkdirSync(path.join(qaDir, 'qa-reports'), { recursive: true });
|
||||
|
||||
for (const args of [['init', '-b', 'main'], ['config', 'user.email', 'test@test.com'], ['config', 'user.name', 'Test'],
|
||||
['add', '.'], ['commit', '-m', 'initial']]) {
|
||||
const git = spawnSync('git', args, { cwd: qaDir, stdio: 'pipe', timeout: 5000 });
|
||||
if (git.error) throw git.error;
|
||||
if (git.status !== 0) throw new Error(`git ${args.join(' ')} failed: ${git.stderr}`);
|
||||
}
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
@@ -68,7 +75,7 @@ qa is a carved skill: when SKILL.md tells you to Read ~/.claude/skills/gstack/qa
|
||||
Skip the preamble bash block, lake intro, telemetry, and contributor mode sections — go straight to the QA workflow.
|
||||
|
||||
Run a Quick-depth QA test on ${testServer.url}/basic.html
|
||||
This fixture is one static page with two same-origin navigation links (/page1, /page2) and one external link; Quick scope is that page and its two same-origin links.
|
||||
This fixture is one static page with two same-origin navigation links (/page1, /page2) and one external link. Quick scope is that page only: its load, console health and rendered content. Do not follow or probe its links; list them as untested coverage.
|
||||
As the caller, I set a 30-second total probe limit for this run (shorter than Browser Quick's own budget), so start the deadline with SECONDS=30.
|
||||
Write concise checkpoints and a concise report, retaining required evidence, scores and untested-coverage fields; cite evidence IDs instead of repeating commands.
|
||||
Do NOT use AskUserQuestion — run Quick tier directly.
|
||||
|
||||
Reference in new issue
Block a user