CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-09-16 09:55:29 +02:00
Code Issues Packages Projects Releases Wiki Activity

refactor(qa): /qa and /qa-only drive Aside, fall back to $B

Browse Source 
QA_METHODOLOGY runs every phase as Aside scripts (orient, explore, document, re-test, mobile viewport via CDP emulation, links via HEAD fetch); the authenticate phase is 'you are already signed in'; a 13th rule requires consent before mutating actions on non-local targets; the fallback section translates each step onto $B. The qa E2E tests run on whichever engine is present.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Sina
2026-09-05 16:48:37 -04:00
co-authored by Claude Fable 5.1
parent 0ded0c8df1
commit 181f9d863b
7 changed files with 205 additions and 118 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qa/SKILL.md.tmpl
+37 -21
View File
@@ -57,7 +57,7 @@ You are a QA engineer AND a bug-fix engineer. Test web applications like a real
| Mode | full | `--regression .gstack/qa-reports/baseline.json` |
| Output dir | `.gstack/qa-reports/` | `Output to /tmp/qa` |
| Scope | Full app (or diff-scoped) | `Focus on the billing page` |
| Auth | None | `Sign in to user@example.com`, `Import cookies from cookies.json` |
| Auth | Your Aside session (already signed in) | If a sign-in wall appears, you sign in yourself in Aside — no credentials in chat (see BROWSER SETUP). Fallback browser only: /setup-browser-cookies or `$B handoff` |
**Tiers determine which issues get fixed:**
- **Quick:** Fix critical + high severity only
@@ -66,12 +66,6 @@ You are a QA engineer AND a bug-fix engineer. Test web applications like a real
**If no URL is given and you're on a feature branch:** Automatically enter **diff-aware mode** (see Modes below). This is the most common case — the user just shipped code on a branch and wants to verify it works.
**CDP mode detection:** Before starting, check if the browse server is connected to the user's real browser:
```bash
$B status 2>/dev/null | grep -q "Mode: cdp" && echo "CDP_MODE=true" || echo "CDP_MODE=false"
```
If `CDP_MODE=true`: skip cookie import prompts (the real browser already has cookies), skip user-agent overrides (real browser has real user-agent), and skip headless detection workarounds. The user's real auth sessions are already available.
**Check for clean working tree:**
```bash
@@ -90,9 +84,11 @@ RECOMMENDATION: Choose A because uncommitted work should be preserved as a commi
After the user chooses, execute their choice (commit or stash), then continue with setup.
**Find the browse binary:**
**Browser: Aside**
{{BROWSE_SETUP}}
{{ASIDE_SETUP}}
{{BROWSE_FALLBACK}}
**Check test framework (bootstrap if needed):**
@@ -101,7 +97,8 @@ After the user chooses, execute their choice (commit or stash), then continue wi
**Create output directories:**
```bash
mkdir -p .gstack/qa-reports/screenshots
REPORT_DIR=".gstack/qa-reports"
mkdir -p "$REPORT_DIR/screenshots"
```
---
@@ -137,11 +134,11 @@ Record baseline health score at end of Phase 6 (per the Health Score Rubric in t
.gstack/qa-reports/
├── qa-report-{domain}-{YYYY-MM-DD}.md # Structured report
├── screenshots/
│ ├── initial.png # Landing page annotated screenshot
│ ├── issue-001-step-1.png # Per-issue evidence
│ ├── issue-001-result.png
│ ├── issue-001-before.png # Before fix (if fixed)
│ ├── issue-001-after.png # After fix (if fixed)
│ ├── initial.jpg # Landing page screenshot
│ ├── issue-001-step-1.jpg # Per-issue evidence
│ ├── issue-001-result.jpg
│ ├── issue-002.png # Annotated screenshot (static bugs)
│ ├── issue-001-after.jpg # After fix (if fixed); the Phase 5 evidence is the before
│ └── ...
└── baseline.json # For regression mode
```
@@ -209,17 +206,36 @@ git commit -m "fix(qa): ISSUE-NNN — short description"
### 8d. Re-test
- Navigate back to the affected page
- Take **before/after screenshot pair**
- Take **before/after screenshot pair** — the Phase 5 evidence is the before; capture the after now
- Check console for errors
- Use `snapshot -D` to verify the change had the expected effect
- Compare the snapshot tree and `CONSOLE_ERRORS=` against the Phase 5 evidence to verify the change had the expected effect
One flow, one script (tabs close when the script ends, so re-navigate from the URL):
```bash
$B goto <affected-url>
$B screenshot "$REPORT_DIR/screenshots/issue-NNN-after.png"
$B console --errors
$B snapshot -D
aside repl '
const HOOK = `(() => { window.__gstackErrs = window.__gstackErrs || []; const oe = console.error; console.error = (...a) => { window.__gstackErrs.push(a.map(String).join(" ")); oe.apply(console, a); }; window.addEventListener("error", e => window.__gstackErrs.push("uncaught: " + e.message)); })()`;
const pg = await openTab("about:blank");
await pg._sendToTarget("Page.addScriptToEvaluateOnNewDocument", { source: HOOK });
await pg.goto("<affected-url>");
const s = await snapshot(pg, { interactive: true });
console.log(s.tree);
console.log("CONSOLE_ERRORS=" + JSON.stringify(await pg.evaluate(() => window.__gstackErrs)));
await pg.screenshot({ path: "issue-NNN-after.jpg", type: "jpeg", quality: 60, fullPage: true });
console.log("ASIDE_DIR=" + pwd);
await closeTab(pg);
console.log("GSTACK_STEP_OK");
'
```
Then copy the evidence out of the `ASIDE_DIR` the script printed:
```bash
cp "<ASIDE_DIR>/issue-NNN-after.jpg" "$REPORT_DIR/screenshots/issue-NNN-after.jpg"
```
Read `$REPORT_DIR/screenshots/issue-NNN-after.jpg` so the user sees the after state inline. If the bug needed an interaction to reproduce, re-run the Phase 5 Drive-a-flow script instead and compare its `DIFF` and `CONSOLE_ERRORS=` lines with the original evidence.
### 8e. Classify
- **verified**: re-test confirms the fix works, no new errors introduced
qa/references/issue-taxonomy.md
+4 -4
View File
@@ -75,11 +75,11 @@
For each page visited during a QA session:
1. **Visual scan** — Take annotated screenshot (`snapshot -i -a -o`). Look for layout issues, broken images, alignment.
1. **Visual scan** — Take a screenshot (the Read-a-page script; `annotatedScreenshot(pg)` when you need ref labels). Look for layout issues, broken images, alignment.
2. **Interactive elements** — Click every button, link, and control. Does each do what it says?
3. **Forms** — Fill and submit. Test empty submission, invalid data, edge cases (long text, special characters).
3. **Forms** — Fill and submit (non-local target: consent first — rule 13). Test empty submission, invalid data, edge cases (long text, special characters).
4. **Navigation** — Check all paths in/out. Breadcrumbs, back button, deep links, mobile menu.
5. **States** — Check empty state, loading state, error state, full/overflow state.
6. **Console** — Run `console --errors` after interactions. Any new JS errors or failed requests?
6. **Console** — Print `CONSOLE_ERRORS=` after interactions. Any new JS errors or failed requests?
7. **Responsiveness** — If relevant, check mobile and tablet viewports.
8. **Auth boundaries** — What happens when logged out? Different user roles?
8. **Auth boundaries** — Never sign the user out or switch accounts yourself. If the signed-out or other-role view matters, ask the user to sign out / switch in Aside and re-run the page scripts.
qa/templates/qa-report-template.md
+4 -5
View File
@@ -64,11 +64,10 @@
**Repro Steps:**
1. Navigate to {URL}
![Step 1](screenshots/issue-001-step-1.png)
![Step 1](screenshots/issue-001-step-1.jpg)
2. {Action}
![Step 2](screenshots/issue-001-step-2.png)
3. **Observe:** {what goes wrong}
![Result](screenshots/issue-001-result.png)
![Result](screenshots/issue-001-result.jpg)
---
@@ -81,8 +80,8 @@
### Before/After Evidence
#### ISSUE-NNN: {title}
**Before:** ![Before](screenshots/issue-NNN-before.png)
**After:** ![After](screenshots/issue-NNN-after.png)
**Before:** ![Before](screenshots/issue-NNN-result.jpg) — the Phase 5 evidence (`issue-NNN.png` for a static bug)
**After:** ![After](screenshots/issue-NNN-after.jpg)
---