## Modes ### Diff-aware (automatic when on a feature branch with no URL) This is the **primary mode** for developers verifying their work. When the user says `/qa` without a URL and the repo is on a feature branch, automatically: 1. **Analyze the branch diff** to understand what changed: ```bash git diff main...HEAD --name-only git log main..HEAD --oneline ``` 2. **Identify affected pages/routes** from the changed files: - Controller/route files → which URL paths they serve - View/template/component files → which pages render them - Model/service files → which pages use those models (check controllers that reference them) - CSS/style files → which pages include those stylesheets - API endpoints → call them with the session's own cookies from one `aside repl` script: ```bash aside repl ' const pg = await openTab(""); const r = await fetch("/api/...", { method: "GET" }); console.log("API_STATUS=" + r.status); console.log("API_BODY_START"); console.log((await r.text()).slice(0, 4000)); console.log("API_BODY_END"); await closeTab(pg); console.log("GSTACK_STEP_OK"); ' ``` - Static pages (markdown, HTML) → navigate to them directly **If no obvious pages/routes are identified from the diff:** Do not skip browser testing. The user invoked /qa because they want browser-based verification. Fall back to Quick mode — navigate to the homepage, follow the top 5 navigation targets, check console for errors, and test any interactive elements found. Backend, config, and infrastructure changes affect app behavior — always verify the app still works. 3. **Detect the running app** — probe common local dev ports (no browser needed to find a port): ```bash for p in 3000 4000 8080; do curl -sI --max-time 3 "http://localhost:$p" >/dev/null 2>&1 && echo "Found app on :$p"; done ``` Open the first URL that answers in Aside. If no local app is found, check for a staging/preview URL in the PR or environment. If nothing works, ask the user for the URL. 4. **Test each affected page/route:** - Navigate to the page (the Read-a-page script in Phase 3) - Take a screenshot - Check console for errors (the `CONSOLE_ERRORS=` line) - If the change was interactive (forms, buttons, flows), test the interaction end-to-end - Snapshot before acting and print the diff after (the Drive-a-flow script in Phase 5) to verify the change had the expected effect 5. **Cross-reference with commit messages and PR description** to understand *intent* — what should the change do? Verify it actually does that. 6. **Check TODOS.md** (if it exists) for known bugs or issues related to the changed files. If a TODO describes a bug that this branch should fix, add it to your test plan. If you find a new bug during QA that isn't in TODOS.md, note it in the report. 7. **Report findings** scoped to the branch changes: - "Changes tested: N pages/routes affected by this branch" - For each: does it work? Screenshot evidence. - Any regressions on adjacent pages? **If the user provides a URL with diff-aware mode:** Use that URL as the base but still scope testing to the changed files. ### Full (default when URL is provided) Systematic exploration. Visit every reachable page. Document 5-10 well-evidenced issues. Produce health score. Takes 5-15 minutes depending on app size. ### Quick (`--quick`) 30-second smoke test. Visit homepage + top 5 navigation targets. Check: page loads? Console errors? Broken links? Produce health score. No detailed issue documentation. ### Regression (`--regression `) Run full mode, then load `baseline.json` from a previous run. Diff: which issues are fixed? Which are new? What's the score delta? Append regression section to report. --- ## Workflow ### Phase 1: Initialize 1. Confirm Aside is READY (see BROWSER SETUP above). If it printed `NEEDS_ASIDE` or `ASIDE_NOT_RUNNING`, the Browser fallback section applies: find `$B` there and translate every `aside repl` script below through its table. 2. Create output directories 3. Copy report template from `qa/templates/qa-report-template.md` to output dir 4. Start timer for duration tracking ### Phase 2: Authenticate (if needed) Aside is the user's real browser, so the session is already signed in wherever the user is signed in. You never authenticate — the user does. In the fallback browser there is no session to inherit: import one with /setup-browser-cookies, or `$B handoff` for a human sign-in and `$B resume` when they're done. **If a sign-in wall appears:** stop and tell the user: "Sign in to in Aside yourself (open it in a new Aside tab), then tell me you're done." Then re-run the step — the browser's cookies now apply. Never type passwords, one-time codes, or payment details, and never read or print cookies, tokens, or localStorage. **If 2FA/OTP is required:** The user completes it in the Aside window, then tells you to continue. **If CAPTCHA blocks you:** Tell the user: "Please complete the CAPTCHA in Aside, then tell me to continue." ### Phase 3: Orient Get a map of the application. One script reads the landing page — console errors from load, the interactive snapshot tree, the visible text, and a screenshot: ```bash aside repl ' const HOOK = `(() => { window.__gstackErrs = window.__gstackErrs || []; const oe = console.error; console.error = (...a) => { window.__gstackErrs.push(a.map(String).join(" ")); oe.apply(console, a); }; window.addEventListener("error", e => window.__gstackErrs.push("uncaught: " + e.message)); window.addEventListener("unhandledrejection", e => window.__gstackErrs.push("unhandledrejection: " + (e.reason && e.reason.message || e.reason))); })()`; const pg = await openTab("about:blank"); await pg._sendToTarget("Page.addScriptToEvaluateOnNewDocument", { source: HOOK }); await pg.goto(""); const s = await snapshot(pg, { interactive: true }); console.log(s.tree); console.log("CONSOLE_ERRORS=" + JSON.stringify(await pg.evaluate(() => window.__gstackErrs))); console.log("TEXT_START"); console.log((await pg.evaluate(() => document.body.innerText)).slice(0, 20000)); console.log("TEXT_END"); await pg.screenshot({ path: "initial.jpg", type: "jpeg", quality: 60, fullPage: true }); console.log("ASIDE_DIR=" + pwd); await closeTab(pg); console.log("GSTACK_STEP_OK"); ' ``` Then copy the screenshot out of the printed directory and show it: `cp "/initial.jpg" "$REPORT_DIR/screenshots/initial.jpg"`, then Read it. Map the navigation structure with the links script (same-origin; HEAD status checks only on a LOCAL target — on a real site the user's cookies would ride every request, so links print as `LINK ?` unfetched): ```bash aside repl ' const pg = await openTab(""); const links = await pg.evaluate(() => [...new Set([...document.querySelectorAll("a[href]")].map(a => a.href))].filter(h => new URL(h).origin === location.origin && !/logout|signout|delete|remove|cancel|unsubscribe/i.test(h))); const local = await pg.evaluate(() => /^(localhost|127\.0\.0\.1|0\.0\.0\.0|::1|\[::1\])$|\.(localhost|test)$/.test(location.hostname)); for (const l of links) { if (!local) { console.log("LINK ?", l); continue; } const r = await fetch(l, { method: "HEAD" }).catch(e => ({ status: "ERR " + e.message })); console.log("LINK", r.status, l); } await closeTab(pg); console.log("GSTACK_STEP_OK"); ' ``` Every `LINK` line with a 4xx/5xx or `ERR` status is a broken link for the Links score; `LINK ?` lines were not fetched (non-local target) and count as unverified, not broken. **Detect framework** (note in report metadata): - `__next` in HTML or `_next/data` requests → Next.js - `csrf-token` meta tag → Rails - `wp-content` in URLs → WordPress - Client-side routing with no page reloads → SPA **For SPAs:** The links script may return few results because navigation is client-side. Use `snapshot(pg, { interactive: true })` to find nav elements (buttons, menu items) instead. ### Phase 4: Explore Visit pages systematically. At each page, run the Read-a-page script from Phase 3 against the page URL with `page-.jpg` as the screenshot path, copy it into `$REPORT_DIR/screenshots/`, and Read it. Then follow the **per-page exploration checklist** (see `qa/references/issue-taxonomy.md`): 1. **Visual scan** — Look at the screenshot for layout issues (use the annotated-screenshot script when you need ref labels on the page) 2. **Interactive elements** — Click buttons, links, controls. Do they work? 3. **Forms** — Fill and submit. Test empty, invalid, edge cases 4. **Navigation** — Check all paths in and out 5. **States** — Empty state, loading, error, overflow 6. **Console** — Any new JS errors after interactions? Print `CONSOLE_ERRORS=` after every action 7. **Responsiveness** — Check the mobile viewport if relevant: ```bash aside repl ' const pg = await openTab(""); await pg._sendToTarget("Emulation.setDeviceMetricsOverride", { width: 375, height: 812, deviceScaleFactor: 2, mobile: true }); await sleep(300); await pg.screenshot({ path: "page-mobile.jpg", type: "jpeg", quality: 60, fullPage: true }); await pg._sendToTarget("Emulation.clearDeviceMetricsOverride", {}); console.log("ASIDE_DIR=" + pwd); await closeTab(pg); console.log("GSTACK_STEP_OK"); ' ``` **Depth judgment:** Spend more time on core features (homepage, dashboard, checkout, search) and less on secondary pages (about, terms, privacy). **Quick mode:** Only visit homepage + top 5 navigation targets from the Orient phase. Skip the per-page checklist — just check: loads? Console errors? Broken links visible? ### Phase 5: Document Document each issue **immediately when found** — don't batch them. **Two evidence tiers:** **Interactive bugs** (broken flows, dead buttons, form failures) — one script per flow, because tabs close when the script ends: 1. Take a screenshot before the action 2. Perform the action 3. Take a screenshot showing the result 4. Print the snapshot diff to show what changed 5. Write repro steps referencing screenshots ```bash aside repl ' const HOOK = `(() => { window.__gstackErrs = window.__gstackErrs || []; const oe = console.error; console.error = (...a) => { window.__gstackErrs.push(a.map(String).join(" ")); oe.apply(console, a); }; window.addEventListener("error", e => window.__gstackErrs.push("uncaught: " + e.message)); })()`; const pg = await openTab("about:blank"); await pg._sendToTarget("Page.addScriptToEvaluateOnNewDocument", { source: HOOK }); await pg.goto(""); await snapshot(pg, { interactive: true }); // baseline for .diff; refs like e12 name the elements await pg.screenshot({ path: "issue-001-step-1.jpg", type: "jpeg", quality: 60 }); await pg.locator("e12").click(); // or pg.fill("#email", "qa@example.com"), pg.getByRole("button", { name: "Save" }).click() await sleep(500); // or await pg.waitForSelector("#done"); await pg.waitForURL(/dashboard/) const s = await snapshot(pg); console.log("DIFF_START"); console.log(s.diff); console.log("DIFF_END"); console.log("URL=" + pg.url()); console.log("CONSOLE_ERRORS=" + JSON.stringify(await pg.evaluate(() => window.__gstackErrs))); await pg.screenshot({ path: "issue-001-result.jpg", type: "jpeg", quality: 60 }); console.log("ASIDE_DIR=" + pwd); await closeTab(pg); console.log("GSTACK_STEP_OK"); ' ``` Copy both screenshots out of the printed `ASIDE_DIR` into `$REPORT_DIR/screenshots/` and Read them. **Static bugs** (typos, layout issues, missing images): 1. Take a single annotated screenshot showing the problem 2. Describe what's wrong ```bash aside repl ' const pg = await openTab(""); const a = await annotatedScreenshot(pg); await fs.writeFile(path.join(pwd, "issue-002.png"), Buffer.from(a.base64Image, "base64")); console.log("ASIDE_DIR=" + pwd); await closeTab(pg); console.log("GSTACK_STEP_OK"); ' ``` **Write each issue to the report immediately** using the template format from `qa/templates/qa-report-template.md`. ### Phase 6: Wrap Up 1. **Compute health score** using the rubric below 2. **Write "Top 3 Things to Fix"** — the 3 highest-severity issues 3. **Write console health summary** — aggregate all console errors seen across pages 4. **Update severity counts** in the summary table 5. **Fill in report metadata** — date, duration, pages visited, screenshot count, framework 6. **Save baseline** — write `baseline.json` with: ```json { "date": "YYYY-MM-DD", "url": "", "healthScore": N, "issues": [{ "id": "ISSUE-001", "title": "...", "severity": "...", "category": "..." }], "categoryScores": { "console": N, "links": N, ... } } ``` **Regression mode:** After writing the report, load the baseline file. Compare: - Health score delta - Issues fixed (in baseline but not current) - New issues (in current but not baseline) - Append the regression section to the report --- ## Health Score Rubric Compute each category score (0-100), then take the weighted average. ### Console (weight: 15%) - 0 errors → 100 - 1-3 errors → 70 - 4-10 errors → 40 - 10+ errors → 10 ### Links (weight: 10%) - 0 broken → 100 - Each broken link → -15 (minimum 0) ### Per-Category Scoring (Visual, Functional, UX, Content, Performance, Accessibility) Each category starts at 100. Deduct per finding: - Critical issue → -25 - High issue → -15 - Medium issue → -8 - Low issue → -3 Minimum 0 per category. ### Weights | Category | Weight | |----------|--------| | Console | 15% | | Links | 10% | | Visual | 10% | | Functional | 20% | | UX | 15% | | Performance | 10% | | Content | 5% | | Accessibility | 15% | ### Final Score `score = Σ (category_score × weight)` --- ## Framework-Specific Guidance ### Next.js - Check console for hydration errors (`Hydration failed`, `Text content did not match`) - Monitor `_next/data` requests in network — 404s indicate broken data fetching - Test client-side navigation (click links, don't just `goto`) — catches routing issues - Check for CLS (Cumulative Layout Shift) on pages with dynamic content ### Rails - Check for N+1 query warnings in console (if development mode) - Verify CSRF token presence in forms - Test Turbo/Stimulus integration — do page transitions work smoothly? - Check for flash messages appearing and dismissing correctly ### WordPress - Check for plugin conflicts (JS errors from different plugins) - Verify admin bar visibility for logged-in users - Test REST API endpoints (`/wp-json/`) - Check for mixed content warnings (common with WP) ### General SPA (React, Vue, Angular) - Use `snapshot(pg, { interactive: true })` for navigation — the links script misses client-side routes - Check for stale state (navigate away and back — does data refresh?) - Test browser back/forward — does the app handle history correctly? - Check for memory leaks (monitor console after extended use) --- ## Important Rules 1. **Repro is everything.** Every issue needs at least one screenshot. No exceptions. 2. **Verify before documenting.** Retry the issue once to confirm it's reproducible, not a fluke. 3. **Never include credentials.** You never type them — the user signs in inside Aside. Write `[REDACTED]` if a repro step has to mention one. 4. **Write incrementally.** Append each issue to the report as you find it. Don't batch. 5. **Never read source code.** Test as a user, not a developer. 6. **Check console after every interaction.** JS errors that don't surface visually are still bugs. 7. **Test like a user.** Use realistic data. Walk through complete workflows end-to-end. 8. **Depth over breadth.** 5-10 well-documented issues with evidence > 20 vague descriptions. 9. **Never delete output files.** Screenshots and reports accumulate — that's intentional. 10. **Use `annotatedScreenshot(pg)` when the tree misses a clickable element.** Ref labels drawn on the page find clickable divs the accessibility tree skips; then click by ref or CSS selector. 11. **Show screenshots to the user.** After every script that saves a screenshot, `cp` it out of the printed `ASIDE_DIR` into `$REPORT_DIR/screenshots/` and use the Read tool on the copied file so the user can see it inline. This is critical — without it, screenshots are invisible to the user. 12. **Never refuse to use the browser.** When the user invokes /qa or /qa-only, they are requesting browser-based testing in Aside. Never suggest evals, unit tests, curl, or other alternatives as a substitute. Even if the diff appears to have no UI changes, backend changes affect app behavior — always open the app in the browser and test. 13. **Mutating actions on a non-local target need consent.** Submitting, creating, deleting, purchasing, or changing settings on anything that is not LOCAL follows the "Invocation is consent to LOOK, not to ACT" rule in BROWSER SETUP — one AskUserQuestion per run, before the first such action.