Phase 3: /qa skill — systematic QA testing with health scores

New /qa skill for systematic web app QA testing. Three modes:
- full: 5-10 documented issues with screenshots and repro steps
- quick: 30-second smoke test with health score
- regression: compare against saved baseline

Includes issue taxonomy (7 categories, 4 severity levels), structured
report template, health score rubric (weighted across 7 categories),
framework detection guidance (Next.js, Rails, WordPress, SPA).

Also adds browse/bin/find-browse (DRY binary discovery using git
rev-parse), .gstack/ to .gitignore, and updated TODO roadmap.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-03-12 13:34:06 -07:00
parent d827276a8d
commit 6ff6d5b6d3
6 changed files with 516 additions and 17 deletions
+45 -17
View File
@@ -7,24 +7,46 @@
- [x] Snapshot command with ref-based element selection
- [x] Snapshot tests
## Phase 2: Enhanced Browser
- [ ] Annotated screenshots (--annotate flag, numbered labels on elements mapped to refs)
- [ ] Snapshot diffing (compare before/after accessibility trees, verify actions worked)
- [ ] Dialog handling (dialog accept/dismiss prevents browser lockup)
- [ ] File upload (upload <sel> <files>)
- [ ] Cursor-interactive elements (-C flag, detect divs with cursor:pointer/onclick/tabindex)
- [ ] Element state checks (is visible/enabled/checked <sel>)
## Phase 2: Enhanced Browser (v0.2.0) ✅
- [x] Annotated screenshots (--annotate flag, ref labels overlaid on screenshot)
- [x] Snapshot diffing (--diff flag, unified diff against previous snapshot)
- [x] Dialog handling (auto-accept/dismiss, dialog buffer, prevents browser lockup)
- [x] File upload (upload <sel> <files>)
- [x] Cursor-interactive elements (-C flag, cursor:pointer/onclick/tabindex scan)
- [x] Element state checks (is visible/hidden/enabled/disabled/checked/editable/focused)
- [x] CircularBuffer — O(1) ring buffer for console/network/dialog (was O(n) array+shift)
- [x] Async buffer flush with Bun.write() (was appendFileSync)
- [x] Health check with page.evaluate('1') + 2s timeout
- [x] Playwright error wrapping — actionable messages for AI agents
- [x] Fix useragent — context recreation preserves cookies/storage/URLs
- [x] DRY: getCleanText exported, command sets in chain updated
- [x] 148 integration tests (was ~63)
## Phase 3: QA Testing Agent (dogfood skill)
- [ ] SKILL.md — 6-phase workflow: Initialize → Authenticate → Orient → Explore → Document → Wrap up
- [ ] Issue taxonomy reference (7 categories: visual, functional, UX, content, performance, console, accessibility)
- [ ] Severity classification (critical/high/medium/low)
- [ ] Exploration checklist per page
- [ ] Report template (structured markdown with per-issue evidence)
- [ ] Repro-first philosophy: every issue gets evidence before moving on
- [ ] Two evidence tiers: interactive bugs (video + step-by-step screenshots), static bugs (single annotated screenshot)
- [ ] Video recording (record start/stop for WebM capture via Playwright)
- [ ] Key guidance: 5-10 well-documented issues per session, depth over breadth, write incrementally
## Phase 3: QA Testing Agent (v0.3.0)
- [x] `/qa` SKILL.md — 6-phase workflow: Initialize → Authenticate → Orient → Explore → Document → Wrap up
- [x] Issue taxonomy reference (7 categories: visual, functional, UX, content, performance, console, accessibility)
- [x] Severity classification (critical/high/medium/low)
- [x] Exploration checklist per page
- [x] Report template (structured markdown with per-issue evidence)
- [x] Repro-first philosophy: every issue gets evidence before moving on
- [x] Two evidence tiers: interactive bugs (multi-step screenshots), static bugs (single annotated screenshot)
- [x] Key guidance: 5-10 well-documented issues per session, depth over breadth, write incrementally
- [x] Three modes: full (systematic), quick (30-second smoke test), regression (compare against baseline)
- [x] Framework detection guidance (Next.js, Rails, WordPress, SPA)
- [x] Health score rubric (7 categories, weighted average)
- [x] `wait --networkidle` / `wait --load` / `wait --domcontentloaded`
- [x] `console --errors` (filter to error/warning only)
- [x] `cookie-import <json-file>` (bulk cookie import with auto-fill domain)
- [x] `browse/bin/find-browse` (DRY binary discovery across skills)
- [ ] Video recording (deferred to Phase 5 — recreateContext destroys page state)
## Phase 3.5: Visual PR Annotations + S3 Upload
- [ ] `/setup-gstack-upload` skill (configure S3 bucket for image hosting)
- [ ] `browse/bin/gstack-upload` helper (upload file to S3, return public URL)
- [ ] `/ship` Step 7.5: visual verification with screenshots in PR body
- [ ] `/review` Step 4.5: visual review with annotated screenshots in PR
- [ ] WebM → GIF conversion (ffmpeg) for video evidence in PRs
- [ ] README documentation for visual PR annotations
## Phase 4: Skill + Browser Integration
- [ ] ship + browse: post-deploy verification
@@ -51,6 +73,7 @@
- [ ] Sessions (isolated browser instances with separate cookies/storage/history)
- [ ] State persistence (save/load cookies + localStorage to JSON files)
- [ ] Auth vault (encrypted credential storage, referenced by name, LLM never sees passwords)
- [ ] Video recording (record start/stop — needs sessions for clean context lifecycle)
- [ ] retro + browse: deployment health tracking
- Screenshot production state
- Check perf metrics (page load times)
@@ -67,6 +90,11 @@
- [ ] Streaming (WebSocket live preview for pair browsing)
- [ ] CDP mode (connect to already-running Chrome/Electron apps)
## Future Ideas
- [ ] Trend tracking across QA runs — compare baseline.json over time, detect regressions (P2, S)
- [ ] CI/CD integration — `/qa` as GitHub Action step, fail PR if health score drops (P2, M)
- [ ] Accessibility audit mode — `--a11y` flag for focused accessibility testing (P3, S)
## Ideas & Notes
- Browser is the nervous system — every skill should be able to see, interact with, and verify the web
- Skills are the product; the browser enables them