gstack/qa/sections/qa-patterns.md

Modes

Diff-aware (automatic when on a feature branch with no URL)

This is the primary mode for developers verifying their work. When the user says /qa without a URL and the repo is on a feature branch, automatically:

1. Analyze the branch diff to understand what changed:

   git diff main...HEAD --name-only
   git log main..HEAD --oneline
   

2. Identify affected pages/routes from the changed files:

   • Controller/route files → which URL paths they serve
   • View/template/component files → which pages render them
   • Model/service files → which pages use those models (check controllers that reference them)
   • CSS/style files → which pages include those stylesheets
   • API endpoints → call them with the session's own cookies from one aside repl script:

     aside repl '
     const pg = await openTab("<base-url>");
     const r = await fetch("<base-url>/api/...", { method: "GET" });
     console.log("API_STATUS=" + r.status);
     console.log("API_BODY_START"); console.log((await r.text()).slice(0, 4000)); console.log("API_BODY_END");
     await closeTab(pg); console.log("GSTACK_STEP_OK");
     '
     

   • Static pages (markdown, HTML) → navigate to them directly

   If no obvious pages/routes are identified from the diff: Do not skip browser testing. The user invoked /qa because they want browser-based verification. Fall back to Quick mode — navigate to the homepage, follow the top 5 navigation targets, check console for errors, and test any interactive elements found. Backend, config, and infrastructure changes affect app behavior — always verify the app still works.

3. Detect the running app — probe common local dev ports (no browser needed to find a port):

   for p in 3000 4000 8080; do curl -sI --max-time 3 "http://localhost:$p" >/dev/null 2>&1 && echo "Found app on :$p"; done
   

   Open the first URL that answers in Aside. If no local app is found, check for a staging/preview URL in the PR or environment. If nothing works, ask the user for the URL.

4. Test each affected page/route:

   • Navigate to the page (the Read-a-page script in Phase 3)
   • Take a screenshot
   • Check console for errors (the CONSOLE_ERRORS= line)
   • If the change was interactive (forms, buttons, flows), test the interaction end-to-end
   • Snapshot before acting and print the diff after (the Drive-a-flow script in Phase 5) to verify the change had the expected effect

5. Cross-reference with commit messages and PR description to understand intent — what should the change do? Verify it actually does that.

6. Check TODOS.md (if it exists) for known bugs or issues related to the changed files. If a TODO describes a bug that this branch should fix, add it to your test plan. If you find a new bug during QA that isn't in TODOS.md, note it in the report.

7. Report findings scoped to the branch changes:

   • "Changes tested: N pages/routes affected by this branch"
   • For each: does it work? Screenshot evidence.
   • Any regressions on adjacent pages?

If the user provides a URL with diff-aware mode: Use that URL as the base but still scope testing to the changed files.

Full (default when URL is provided)

Systematic exploration. Visit every reachable page. Document 5-10 well-evidenced issues. Produce health score. Takes 5-15 minutes depending on app size.

Quick (--quick)

30-second smoke test. Visit homepage + top 5 navigation targets. Check: page loads? Console errors? Broken links? Produce health score. No detailed issue documentation.

Regression (--regression <baseline>)

Run full mode, then load baseline.json from a previous run. Diff: which issues are fixed? Which are new? What's the score delta? Append regression section to report.

---

Workflow

Phase 1: Initialize

1. Confirm Aside is READY (see BROWSER SETUP above). For any non-READY result, the Browser fallback section applies: find $B there and translate every aside repl script below through its table.
2. Create output directories
3. Copy report template from qa/templates/qa-report-template.md to output dir
4. Start timer for duration tracking

Phase 2: Authenticate (if needed)

Aside is the user's real browser, so the session is already signed in wherever the user is signed in. You never authenticate — the user does. In the fallback browser there is no session to inherit: import one with /setup-browser-cookies, or $B handoff for a human sign-in and $B resume when they're done.

If a sign-in wall appears: stop and tell the user: "Sign in to in Aside yourself (open it in a new Aside tab), then tell me you're done." Then re-run the step — the browser's cookies now apply. Never type passwords, one-time codes, or payment details, and never read or print cookies, tokens, or localStorage.

If 2FA/OTP is required: The user completes it in the Aside window, then tells you to continue.

If CAPTCHA blocks you: Tell the user: "Please complete the CAPTCHA in Aside, then tell me to continue."

Phase 3: Orient

Get a map of the application. One script reads the landing page — console errors from load, the interactive snapshot tree, the visible text, and a screenshot:

aside repl '
const HOOK = `(() => { window.__gstackErrs = window.__gstackErrs || []; const oe = console.error; console.error = (...a) => { window.__gstackErrs.push(a.map(String).join(" ")); oe.apply(console, a); }; window.addEventListener("error", e => window.__gstackErrs.push("uncaught: " + e.message)); window.addEventListener("unhandledrejection", e => window.__gstackErrs.push("unhandledrejection: " + (e.reason && e.reason.message || e.reason))); })()`;
const pg = await openTab("about:blank");
await pg._sendToTarget("Page.addScriptToEvaluateOnNewDocument", { source: HOOK });
await pg.goto("<target-url>");
const s = await snapshot(pg, { interactive: true });
console.log(s.tree);
console.log("CONSOLE_ERRORS=" + JSON.stringify(await pg.evaluate(() => window.__gstackErrs)));
console.log("TEXT_START"); console.log((await pg.evaluate(() => document.body.innerText)).slice(0, 20000)); console.log("TEXT_END");
await pg.screenshot({ path: "initial.jpg", type: "jpeg", quality: 60, fullPage: true });
console.log("ASIDE_DIR=" + pwd);
await closeTab(pg);
console.log("GSTACK_STEP_OK");
'

Then copy the screenshot out of the printed directory and show it: cp "<ASIDE_DIR>/initial.jpg" "$REPORT_DIR/screenshots/initial.jpg", then Read it.

Map the navigation structure with the links script (same-origin; HEAD status checks only on a LOCAL target — on a real site the user's cookies would ride every request, so links print as LINK ? unfetched):

aside repl '
const pg = await openTab("<target-url>");
const links = await pg.evaluate(() => [...new Set([...document.querySelectorAll("a[href]")].map(a => a.href))].filter(h => new URL(h).origin === location.origin && !/logout|signout|delete|remove|cancel|unsubscribe/i.test(h)));
const local = await pg.evaluate(() => /^(localhost|127\.0\.0\.1|0\.0\.0\.0|::1|\[::1\])$|\.(localhost|test)$/.test(location.hostname));
for (const l of links) { if (!local) { console.log("LINK ?", l); continue; } const r = await fetch(l, { method: "HEAD" }).catch(e => ({ status: "ERR " + e.message })); console.log("LINK", r.status, l); }
await closeTab(pg); console.log("GSTACK_STEP_OK");
'

Every LINK line with a 4xx/5xx or ERR status is a broken link for the Links score; LINK ? lines were not fetched (non-local target) and count as unverified, not broken.

Detect framework (note in report metadata):

• __next in HTML or _next/data requests → Next.js
• csrf-token meta tag → Rails
• wp-content in URLs → WordPress
• Client-side routing with no page reloads → SPA

For SPAs: The links script may return few results because navigation is client-side. Use snapshot(pg, { interactive: true }) to find nav elements (buttons, menu items) instead.

Phase 4: Explore

Visit pages systematically. At each page, run the Read-a-page script from Phase 3 against the page URL with page-<name>.jpg as the screenshot path, copy it into $REPORT_DIR/screenshots/, and Read it.

Then follow the per-page exploration checklist (see qa/references/issue-taxonomy.md):

1. Visual scan — Look at the screenshot for layout issues (use the annotated-screenshot script when you need ref labels on the page)
2. Interactive elements — Click buttons, links, controls. Do they work?
3. Forms — Fill and submit. Test empty, invalid, edge cases
4. Navigation — Check all paths in and out
5. States — Empty state, loading, error, overflow
6. Console — Any new JS errors after interactions? Print CONSOLE_ERRORS= after every action
7. Responsiveness — Check the mobile viewport if relevant:

   aside repl '
   const pg = await openTab("<page-url>");
   await pg._sendToTarget("Emulation.setDeviceMetricsOverride", { width: 375, height: 812, deviceScaleFactor: 2, mobile: true });
   await sleep(300);
   await pg.screenshot({ path: "page-mobile.jpg", type: "jpeg", quality: 60, fullPage: true });
   await pg._sendToTarget("Emulation.clearDeviceMetricsOverride", {});
   console.log("ASIDE_DIR=" + pwd); await closeTab(pg); console.log("GSTACK_STEP_OK");
   '
   

Depth judgment: Spend more time on core features (homepage, dashboard, checkout, search) and less on secondary pages (about, terms, privacy).

Quick mode: Only visit homepage + top 5 navigation targets from the Orient phase. Skip the per-page checklist — just check: loads? Console errors? Broken links visible?

Phase 5: Document

Document each issue immediately when found — don't batch them.

Two evidence tiers:

Interactive bugs (broken flows, dead buttons, form failures) — one script per flow, because tabs close when the script ends:

1. Take a screenshot before the action
2. Perform the action
3. Take a screenshot showing the result
4. Print the snapshot diff to show what changed
5. Write repro steps referencing screenshots

aside repl '
const HOOK = `(() => { window.__gstackErrs = window.__gstackErrs || []; const oe = console.error; console.error = (...a) => { window.__gstackErrs.push(a.map(String).join(" ")); oe.apply(console, a); }; window.addEventListener("error", e => window.__gstackErrs.push("uncaught: " + e.message)); })()`;
const pg = await openTab("about:blank");
await pg._sendToTarget("Page.addScriptToEvaluateOnNewDocument", { source: HOOK });
await pg.goto("<page-url>");
await snapshot(pg, { interactive: true });                            // baseline for .diff; refs like e12 name the elements
await pg.screenshot({ path: "issue-001-step-1.jpg", type: "jpeg", quality: 60 });
await pg.locator("e12").click();                                       // or pg.fill("#email", "qa@example.com"), pg.getByRole("button", { name: "Save" }).click()
await sleep(500);                                                      // or await pg.waitForSelector("#done"); await pg.waitForURL(/dashboard/)
const s = await snapshot(pg);
console.log("DIFF_START"); console.log(s.diff); console.log("DIFF_END");
console.log("URL=" + pg.url());
console.log("CONSOLE_ERRORS=" + JSON.stringify(await pg.evaluate(() => window.__gstackErrs)));
await pg.screenshot({ path: "issue-001-result.jpg", type: "jpeg", quality: 60 });
console.log("ASIDE_DIR=" + pwd);
await closeTab(pg);
console.log("GSTACK_STEP_OK");
'

Copy both screenshots out of the printed ASIDE_DIR into $REPORT_DIR/screenshots/ and Read them.

Static bugs (typos, layout issues, missing images):

1. Take a single annotated screenshot showing the problem
2. Describe what's wrong

aside repl '
const pg = await openTab("<page-url>");
const a = await annotatedScreenshot(pg);
await fs.writeFile(path.join(pwd, "issue-002.png"), Buffer.from(a.base64Image, "base64"));
console.log("ASIDE_DIR=" + pwd); await closeTab(pg); console.log("GSTACK_STEP_OK");
'

Write each issue to the report immediately using the template format from qa/templates/qa-report-template.md.

Phase 6: Wrap Up

1. Compute health score using the rubric below
2. Write "Top 3 Things to Fix" — the 3 highest-severity issues
3. Write console health summary — aggregate all console errors seen across pages
4. Update severity counts in the summary table
5. Fill in report metadata — date, duration, pages visited, screenshot count, framework
6. Save baseline — write baseline.json with:

   {
     "date": "YYYY-MM-DD",
     "url": "<target>",
     "healthScore": N,
     "issues": [{ "id": "ISSUE-001", "title": "...", "severity": "...", "category": "..." }],
     "categoryScores": { "console": N, "links": N, ... }
   }
   

Regression mode: After writing the report, load the baseline file. Compare:

• Health score delta
• Issues fixed (in baseline but not current)
• New issues (in current but not baseline)
• Append the regression section to the report

---

Health Score Rubric

Compute each category score (0-100), then take the weighted average.

Counting

• Deduplicate the same root cause across pages. Use one primary category, first applicable: Links (navigation), Accessibility (access barriers), Functional (behavior), Performance (speed), Visual (layout), Content (copy), UX (friction), Console (remaining errors). No double deductions.
• Exclude untested categories; label partial scores provisional with coverage. None tested: "not scored". Compare only identical coverage.

Console (weight: 15%)

Deduplicate reproducible errors/exceptions by message+source across pages. Exclude warnings, info, and defects scored elsewhere.

• 0 errors → 100
• 1-3 errors → 70
• 4-10 errors → 40
• 11+ errors → 10

Links (weight: 10%)

Count unique broken destinations, including client-side routes: repeatable 4xx/5xx, missing routes/anchors, or timeouts. Exclude expected auth redirects and resource/API requests.

• 0 broken → 100
• Each broken link → -15 (minimum 0)

Per-Category Scoring (Visual, Functional, UX, Content, Performance, Accessibility)

Start at 100; deduct per finding:

• Critical issue → -25
• High issue → -15
• Medium issue → -8
• Low issue → -3 Floor: 0.

Use the highest applicable severity; record impact/workaround:

• Critical: data loss, security/privacy exposure, or core app unusable for all users.
• High: core/major task blocked without a workaround.
• Medium: task impaired but a workaround exists.
• Low: cosmetic/copy/friction issue without lost task completion. Console/Links use counts instead.

Weights

Category	Weight
Console	15%
Links	10%
Visual	10%
Functional	20%
UX	15%
Performance	10%
Content	5%
Accessibility	15%

Final Score

Use decimal weights (15% = 0.15): score = Σ (category_score × weight) / Σ tested weights. Round only the final score to the nearest integer (0.5 rounds up).

---

Framework-Specific Guidance

Next.js

• Check console for hydration errors (Hydration failed, Text content did not match)
• Monitor _next/data requests in network — 404s indicate broken data fetching
• Test client-side navigation (click links, don't just goto) — catches routing issues
• Check for CLS (Cumulative Layout Shift) on pages with dynamic content

Rails

• Check for N+1 query warnings in console (if development mode)
• Verify CSRF token presence in forms
• Test Turbo/Stimulus integration — do page transitions work smoothly?
• Check for flash messages appearing and dismissing correctly

WordPress

• Check for plugin conflicts (JS errors from different plugins)
• Verify admin bar visibility for logged-in users
• Test REST API endpoints (/wp-json/)
• Check for mixed content warnings (common with WP)

General SPA (React, Vue, Angular)

• Use snapshot(pg, { interactive: true }) for navigation — the links script misses client-side routes
• Check for stale state (navigate away and back — does data refresh?)
• Test browser back/forward — does the app handle history correctly?
• Check for memory leaks (monitor console after extended use)

---

Important Rules

1. Repro is everything. Every issue needs at least one screenshot. No exceptions.
2. Verify before documenting. Retry the issue once to confirm it's reproducible, not a fluke.
3. Never include credentials. You never type them — the user signs in inside Aside. Write [REDACTED] if a repro step has to mention one.
4. Write incrementally. Append each issue to the report as you find it. Don't batch.
5. Never read source code. Test as a user, not a developer.
6. Check console after every interaction. JS errors that don't surface visually are still bugs.
7. Test like a user. Use realistic data. Walk through complete workflows end-to-end.
8. Depth over breadth. 5-10 well-documented issues with evidence > 20 vague descriptions.
9. Never delete output files. Screenshots and reports accumulate — that's intentional.
10. Use annotatedScreenshot(pg) when the tree misses a clickable element. Ref labels drawn on the page find clickable divs the accessibility tree skips; then click by ref or CSS selector.
11. Show screenshots to the user. After every script that saves a screenshot, cp it out of the printed ASIDE_DIR into $REPORT_DIR/screenshots/ and use the Read tool on the copied file so the user can see it inline. This is critical — without it, screenshots are invisible to the user.
12. Never refuse to use the browser. When the user invokes /qa or /qa-only, they are requesting browser-based testing in Aside. Never suggest evals, unit tests, curl, or other alternatives as a substitute. Even if the diff appears to have no UI changes, backend changes affect app behavior — always open the app in the browser and test.
13. Mutating actions on a non-local target need consent. Submitting, creating, deleting, purchasing, or changing settings on anything that is not LOCAL follows the "Invocation is consent to LOOK, not to ACT" rule in BROWSER SETUP — one AskUserQuestion per run, before the first such action.