mirror of
https://github.com/garrytan/gstack.git
synced 2026-07-20 22:40:58 +02:00
Phase 3: /qa skill — systematic QA testing with health scores
New /qa skill for systematic web app QA testing. Three modes: - full: 5-10 documented issues with screenshots and repro steps - quick: 30-second smoke test with health score - regression: compare against saved baseline Includes issue taxonomy (7 categories, 4 severity levels), structured report template, health score rubric (weighted across 7 categories), framework detection guidance (Next.js, Rails, WordPress, SPA). Also adds browse/bin/find-browse (DRY binary discovery using git rev-parse), .gstack/ to .gitignore, and updated TODO roadmap. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -7,24 +7,46 @@
|
||||
- [x] Snapshot command with ref-based element selection
|
||||
- [x] Snapshot tests
|
||||
|
||||
## Phase 2: Enhanced Browser
|
||||
- [ ] Annotated screenshots (--annotate flag, numbered labels on elements mapped to refs)
|
||||
- [ ] Snapshot diffing (compare before/after accessibility trees, verify actions worked)
|
||||
- [ ] Dialog handling (dialog accept/dismiss — prevents browser lockup)
|
||||
- [ ] File upload (upload <sel> <files>)
|
||||
- [ ] Cursor-interactive elements (-C flag, detect divs with cursor:pointer/onclick/tabindex)
|
||||
- [ ] Element state checks (is visible/enabled/checked <sel>)
|
||||
## Phase 2: Enhanced Browser (v0.2.0) ✅
|
||||
- [x] Annotated screenshots (--annotate flag, ref labels overlaid on screenshot)
|
||||
- [x] Snapshot diffing (--diff flag, unified diff against previous snapshot)
|
||||
- [x] Dialog handling (auto-accept/dismiss, dialog buffer, prevents browser lockup)
|
||||
- [x] File upload (upload <sel> <files>)
|
||||
- [x] Cursor-interactive elements (-C flag, cursor:pointer/onclick/tabindex scan)
|
||||
- [x] Element state checks (is visible/hidden/enabled/disabled/checked/editable/focused)
|
||||
- [x] CircularBuffer — O(1) ring buffer for console/network/dialog (was O(n) array+shift)
|
||||
- [x] Async buffer flush with Bun.write() (was appendFileSync)
|
||||
- [x] Health check with page.evaluate('1') + 2s timeout
|
||||
- [x] Playwright error wrapping — actionable messages for AI agents
|
||||
- [x] Fix useragent — context recreation preserves cookies/storage/URLs
|
||||
- [x] DRY: getCleanText exported, command sets in chain updated
|
||||
- [x] 148 integration tests (was ~63)
|
||||
|
||||
## Phase 3: QA Testing Agent (dogfood skill)
|
||||
- [ ] SKILL.md — 6-phase workflow: Initialize → Authenticate → Orient → Explore → Document → Wrap up
|
||||
- [ ] Issue taxonomy reference (7 categories: visual, functional, UX, content, performance, console, accessibility)
|
||||
- [ ] Severity classification (critical/high/medium/low)
|
||||
- [ ] Exploration checklist per page
|
||||
- [ ] Report template (structured markdown with per-issue evidence)
|
||||
- [ ] Repro-first philosophy: every issue gets evidence before moving on
|
||||
- [ ] Two evidence tiers: interactive bugs (video + step-by-step screenshots), static bugs (single annotated screenshot)
|
||||
- [ ] Video recording (record start/stop for WebM capture via Playwright)
|
||||
- [ ] Key guidance: 5-10 well-documented issues per session, depth over breadth, write incrementally
|
||||
## Phase 3: QA Testing Agent (v0.3.0)
|
||||
- [x] `/qa` SKILL.md — 6-phase workflow: Initialize → Authenticate → Orient → Explore → Document → Wrap up
|
||||
- [x] Issue taxonomy reference (7 categories: visual, functional, UX, content, performance, console, accessibility)
|
||||
- [x] Severity classification (critical/high/medium/low)
|
||||
- [x] Exploration checklist per page
|
||||
- [x] Report template (structured markdown with per-issue evidence)
|
||||
- [x] Repro-first philosophy: every issue gets evidence before moving on
|
||||
- [x] Two evidence tiers: interactive bugs (multi-step screenshots), static bugs (single annotated screenshot)
|
||||
- [x] Key guidance: 5-10 well-documented issues per session, depth over breadth, write incrementally
|
||||
- [x] Three modes: full (systematic), quick (30-second smoke test), regression (compare against baseline)
|
||||
- [x] Framework detection guidance (Next.js, Rails, WordPress, SPA)
|
||||
- [x] Health score rubric (7 categories, weighted average)
|
||||
- [x] `wait --networkidle` / `wait --load` / `wait --domcontentloaded`
|
||||
- [x] `console --errors` (filter to error/warning only)
|
||||
- [x] `cookie-import <json-file>` (bulk cookie import with auto-fill domain)
|
||||
- [x] `browse/bin/find-browse` (DRY binary discovery across skills)
|
||||
- [ ] Video recording (deferred to Phase 5 — recreateContext destroys page state)
|
||||
|
||||
## Phase 3.5: Visual PR Annotations + S3 Upload
|
||||
- [ ] `/setup-gstack-upload` skill (configure S3 bucket for image hosting)
|
||||
- [ ] `browse/bin/gstack-upload` helper (upload file to S3, return public URL)
|
||||
- [ ] `/ship` Step 7.5: visual verification with screenshots in PR body
|
||||
- [ ] `/review` Step 4.5: visual review with annotated screenshots in PR
|
||||
- [ ] WebM → GIF conversion (ffmpeg) for video evidence in PRs
|
||||
- [ ] README documentation for visual PR annotations
|
||||
|
||||
## Phase 4: Skill + Browser Integration
|
||||
- [ ] ship + browse: post-deploy verification
|
||||
@@ -51,6 +73,7 @@
|
||||
- [ ] Sessions (isolated browser instances with separate cookies/storage/history)
|
||||
- [ ] State persistence (save/load cookies + localStorage to JSON files)
|
||||
- [ ] Auth vault (encrypted credential storage, referenced by name, LLM never sees passwords)
|
||||
- [ ] Video recording (record start/stop — needs sessions for clean context lifecycle)
|
||||
- [ ] retro + browse: deployment health tracking
|
||||
- Screenshot production state
|
||||
- Check perf metrics (page load times)
|
||||
@@ -67,6 +90,11 @@
|
||||
- [ ] Streaming (WebSocket live preview for pair browsing)
|
||||
- [ ] CDP mode (connect to already-running Chrome/Electron apps)
|
||||
|
||||
## Future Ideas
|
||||
- [ ] Trend tracking across QA runs — compare baseline.json over time, detect regressions (P2, S)
|
||||
- [ ] CI/CD integration — `/qa` as GitHub Action step, fail PR if health score drops (P2, M)
|
||||
- [ ] Accessibility audit mode — `--a11y` flag for focused accessibility testing (P3, S)
|
||||
|
||||
## Ideas & Notes
|
||||
- Browser is the nervous system — every skill should be able to see, interact with, and verify the web
|
||||
- Skills are the product; the browser enables them
|
||||
|
||||
Reference in New Issue
Block a user