Files
gstack/browse/test/snapshot.test.ts
T
Garry Tan f7b95329c1 feat: Phase 3.5 — cookie import, QA testing, team retro (v0.3.1) (#29)
* Phase 2: Enhanced browser — dialog handling, upload, state checks, snapshots

- CircularBuffer O(1) ring buffer for console/network/dialog (was O(n) array+shift)
- Async buffer flush with Bun.write() (was appendFileSync)
- Dialog auto-accept/dismiss with buffer + prompt text support
- File upload command (upload <sel> <file...>)
- Element state checks (is visible/hidden/enabled/disabled/checked/editable/focused)
- Annotated screenshots with ref labels overlaid (-a flag)
- Snapshot diffing against previous snapshot (-D flag)
- Cursor-interactive element scan for non-ARIA clickables (-C flag)
- Snapshot scoping depth limit (-d N flag)
- Health check with page.evaluate + 2s timeout
- Playwright error wrapping — actionable messages for AI agents
- Fix useragent — context recreation preserves cookies/storage/URLs
- wait --networkidle / --load / --domcontentloaded flags
- console --errors filter (error + warning only)
- cookie-import <json-file> with auto-fill domain from page URL
- 166 integration tests (was ~63)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Phase 2: Rewrite SKILL.md as QA playbook + command reference

Reorient SKILL.md files from raw command reference to QA-first playbook
with 10 workflow patterns (test user flows, verify deployments, dogfood
features, responsive layouts, file upload, forms, dialogs, compare pages).
Compact command reference tables at the bottom.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Phase 3: /qa skill — systematic QA testing with health scores

New /qa skill for systematic web app QA testing. Three modes:
- full: 5-10 documented issues with screenshots and repro steps
- quick: 30-second smoke test with health score
- regression: compare against saved baseline

Includes issue taxonomy (7 categories, 4 severity levels), structured
report template, health score rubric (weighted across 7 categories),
framework detection guidance (Next.js, Rails, WordPress, SPA).

Also adds browse/bin/find-browse (DRY binary discovery using git
rev-parse), .gstack/ to .gitignore, and updated TODO roadmap.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Bump to v0.3.0 — Phase 2 + Phase 3 changelog

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: cookie-import-browser — Chromium cookie decryption module + tests

Pure logic module for reading and decrypting cookies from macOS Chromium
browsers (Comet, Chrome, Arc, Brave, Edge). Supports v10 AES-128-CBC
encryption with macOS Keychain access, PBKDF2 key derivation, and
per-browser key caching. 18 unit tests with encrypted cookie fixtures.

* feat: cookie picker web UI + route handler

Two-panel dark-theme picker served from the browse server. Left panel
shows source browser domains with search and import buttons. Right panel
shows imported domains with trash buttons. No cookie values exposed.
6 API endpoints, importedDomains Set tracking, inline clearCookies.

* feat: wire cookie-import-browser into browse server

Add cookie-picker route dispatch (no auth, localhost-only), add
cookie-import-browser to WRITE_COMMANDS and CHAIN_WRITE, add serverPort
property to BrowserManager, add write command with two modes (picker UI
vs --domain direct import), update CLI help text.

* chore: /setup-browser-cookies skill + docs (Phase 3.5)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: bump version and changelog (v0.3.1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security: redact sensitive values from command output (PR #21)

type no longer echoes text (reports character count), cookie redacts
value with ****, header redacts Authorization/Cookie/X-API-Key/X-Auth-Token,
storage set drops value, forms redacts password fields. Prevents secrets
from persisting in LLM transcripts. 7 new tests.

Credit: fredluz (PR #21)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security: path traversal prevention for screenshot/pdf/eval (PR #26)

Add validateOutputPath() for screenshot/pdf/responsive (restricts to
/tmp and cwd) and validateReadPath() for eval (blocks .. sequences and
absolute paths outside safe dirs). 7 new tests.

Credit: Jah-yee (PR #26)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: auto-install Playwright Chromium in setup (PR #22)

Setup now verifies Playwright can launch Chromium, and auto-installs
it via `bunx playwright install chromium` if missing. Exits non-zero
if build or Chromium launch fails.

Credit: AkbarDevop (PR #22)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security: fix path validation bypass, CORS restriction, cookie-import path check

- startsWith('/tmp') matched '/tmpevil' — now requires trailing slash
- CORS Access-Control-Allow-Origin changed from * to http://127.0.0.1:<port>
- cookie-import now validates file paths (was missing validateReadPath)
- 3 new tests for prefix collision and cookie-import path traversal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review informational issues + add regression tests

- Add cookie-import to CHAIN_WRITE set for chain command routing
- Add path validation to snapshot -a -o output path
- Fix package.json version to match 0.3.1
- Use crypto.randomUUID() for temp DB paths (unpredictable filenames)
- Add regression tests for chain cookie-import and snapshot path validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add /qa, /setup-browser-cookies to README + update BROWSER.md

- Add /qa and /setup-browser-cookies to skills table, install/update/uninstall blurbs
- Add dedicated README sections for both new skills with usage examples
- Update demo workflow to show cookie import → QA → browse flow
- Update BROWSER.md: cookie import commands, new source files, test count (203)
- Update skill count from 6 to 8

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: team-aware /retro v2.0 — per-person praise and growth opportunities

- Identify current user via git config, orient narrative as "you" vs teammates
- Add per-author metrics: commits, LOC, focus areas, commit type mix, sessions
- New "Your Week" section with personal deep-dive for whoever runs the command
- New "Team Breakdown" with per-person praise and growth opportunities
- Track AI-assisted commits via Co-Authored-By trailers
- Personal + team shipping streaks
- Tone: praise like a 1:1, growth like investment advice, never compare negatively

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add Conductor parallel sessions section to README

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 00:31:41 -07:00

419 lines
18 KiB
TypeScript

/**
* Snapshot command tests
*
* Tests: accessibility tree snapshots, ref-based element selection,
* ref invalidation on navigation, and ref resolution in commands.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { startTestServer } from './test-server';
import { BrowserManager } from '../src/browser-manager';
import { handleReadCommand } from '../src/read-commands';
import { handleWriteCommand } from '../src/write-commands';
import { handleMetaCommand } from '../src/meta-commands';
import * as fs from 'fs';
let testServer: ReturnType<typeof startTestServer>;
let bm: BrowserManager;
let baseUrl: string;
const shutdown = async () => {};
beforeAll(async () => {
testServer = startTestServer(0);
baseUrl = testServer.url;
bm = new BrowserManager();
await bm.launch();
});
afterAll(() => {
try { testServer.server.stop(); } catch {}
setTimeout(() => process.exit(0), 500);
});
// ─── Snapshot Output ────────────────────────────────────────────
describe('Snapshot', () => {
test('snapshot returns accessibility tree with refs', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const result = await handleMetaCommand('snapshot', [], bm, shutdown);
expect(result).toContain('@e');
expect(result).toContain('[heading]');
expect(result).toContain('"Snapshot Test"');
expect(result).toContain('[button]');
expect(result).toContain('[link]');
});
test('snapshot -i returns only interactive elements', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const result = await handleMetaCommand('snapshot', ['-i'], bm, shutdown);
expect(result).toContain('[button]');
expect(result).toContain('[link]');
expect(result).toContain('[textbox]');
// Should NOT contain non-interactive roles like heading or paragraph
expect(result).not.toContain('[heading]');
});
test('snapshot -c returns compact output', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const full = await handleMetaCommand('snapshot', [], bm, shutdown);
const compact = await handleMetaCommand('snapshot', ['-c'], bm, shutdown);
// Compact should have fewer lines (empty structural elements removed)
const fullLines = full.split('\n').length;
const compactLines = compact.split('\n').length;
expect(compactLines).toBeLessThanOrEqual(fullLines);
});
test('snapshot -d 2 limits depth', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const shallow = await handleMetaCommand('snapshot', ['-d', '2'], bm, shutdown);
const deep = await handleMetaCommand('snapshot', [], bm, shutdown);
// Shallow should have fewer or equal lines
expect(shallow.split('\n').length).toBeLessThanOrEqual(deep.split('\n').length);
});
test('snapshot -s "#main" scopes to selector', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const scoped = await handleMetaCommand('snapshot', ['-s', '#main'], bm, shutdown);
// Should contain elements inside #main
expect(scoped).toContain('[button]');
expect(scoped).toContain('"Submit"');
// Should NOT contain elements outside #main (like nav links)
expect(scoped).not.toContain('"Internal Link"');
});
test('snapshot on page with no interactive elements', async () => {
// Navigate to about:blank which has minimal content
await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm);
const result = await handleMetaCommand('snapshot', ['-i'], bm, shutdown);
// basic.html has links, so this should find those
expect(result).toContain('[link]');
});
test('second snapshot generates fresh refs', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const snap1 = await handleMetaCommand('snapshot', [], bm, shutdown);
const snap2 = await handleMetaCommand('snapshot', [], bm, shutdown);
// Both should have @e1 (refs restart from 1)
expect(snap1).toContain('@e1');
expect(snap2).toContain('@e1');
});
});
// ─── Ref-Based Interaction ──────────────────────────────────────
describe('Ref resolution', () => {
test('click @ref works after snapshot', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const snap = await handleMetaCommand('snapshot', ['-i'], bm, shutdown);
// Find a button ref
const buttonLine = snap.split('\n').find(l => l.includes('[button]') && l.includes('"Submit"'));
expect(buttonLine).toBeDefined();
const refMatch = buttonLine!.match(/@(e\d+)/);
expect(refMatch).toBeDefined();
const ref = `@${refMatch![1]}`;
const result = await handleWriteCommand('click', [ref], bm);
expect(result).toContain('Clicked');
});
test('fill @ref works after snapshot', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const snap = await handleMetaCommand('snapshot', ['-i'], bm, shutdown);
// Find a textbox ref (Username)
const textboxLine = snap.split('\n').find(l => l.includes('[textbox]') && l.includes('"Username"'));
expect(textboxLine).toBeDefined();
const refMatch = textboxLine!.match(/@(e\d+)/);
expect(refMatch).toBeDefined();
const ref = `@${refMatch![1]}`;
const result = await handleWriteCommand('fill', [ref, 'testuser'], bm);
expect(result).toContain('Filled');
});
test('hover @ref works after snapshot', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const snap = await handleMetaCommand('snapshot', ['-i'], bm, shutdown);
const linkLine = snap.split('\n').find(l => l.includes('[link]'));
expect(linkLine).toBeDefined();
const refMatch = linkLine!.match(/@(e\d+)/);
const ref = `@${refMatch![1]}`;
const result = await handleWriteCommand('hover', [ref], bm);
expect(result).toContain('Hovered');
});
test('html @ref returns innerHTML', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const snap = await handleMetaCommand('snapshot', [], bm, shutdown);
// Find a heading ref
const headingLine = snap.split('\n').find(l => l.includes('[heading]') && l.includes('"Snapshot Test"'));
expect(headingLine).toBeDefined();
const refMatch = headingLine!.match(/@(e\d+)/);
const ref = `@${refMatch![1]}`;
const result = await handleReadCommand('html', [ref], bm);
expect(result).toContain('Snapshot Test');
});
test('css @ref returns computed CSS', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const snap = await handleMetaCommand('snapshot', [], bm, shutdown);
const headingLine = snap.split('\n').find(l => l.includes('[heading]') && l.includes('"Snapshot Test"'));
const refMatch = headingLine!.match(/@(e\d+)/);
const ref = `@${refMatch![1]}`;
const result = await handleReadCommand('css', [ref, 'font-family'], bm);
expect(result).toBeTruthy();
});
test('attrs @ref returns element attributes', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const snap = await handleMetaCommand('snapshot', ['-i'], bm, shutdown);
const textboxLine = snap.split('\n').find(l => l.includes('[textbox]') && l.includes('"Username"'));
const refMatch = textboxLine!.match(/@(e\d+)/);
const ref = `@${refMatch![1]}`;
const result = await handleReadCommand('attrs', [ref], bm);
expect(result).toContain('id');
});
});
// ─── Ref Invalidation ───────────────────────────────────────────
describe('Ref invalidation', () => {
test('stale ref after goto returns clear error', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
await handleMetaCommand('snapshot', ['-i'], bm, shutdown);
// Navigate away — should invalidate refs
await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm);
// Try to use old ref
try {
await handleWriteCommand('click', ['@e1'], bm);
expect(true).toBe(false); // Should not reach here
} catch (err: any) {
expect(err.message).toContain('not found');
expect(err.message).toContain('snapshot');
}
});
test('refs cleared on page navigation', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
await handleMetaCommand('snapshot', ['-i'], bm, shutdown);
expect(bm.getRefCount()).toBeGreaterThan(0);
// Navigate
await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm);
expect(bm.getRefCount()).toBe(0);
});
});
// ─── Snapshot Diffing ──────────────────────────────────────────
describe('Snapshot diff', () => {
test('first snapshot -D stores baseline', async () => {
// Clear any previous snapshot
bm.setLastSnapshot(null);
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const result = await handleMetaCommand('snapshot', ['-D'], bm, shutdown);
expect(result).toContain('no previous snapshot');
expect(result).toContain('baseline');
});
test('snapshot -D shows diff after change', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
// Take first snapshot
await handleMetaCommand('snapshot', [], bm, shutdown);
// Modify DOM
await handleReadCommand('js', ['document.querySelector("h1").textContent = "Changed Title"'], bm);
// Take diff
const diff = await handleMetaCommand('snapshot', ['-D'], bm, shutdown);
expect(diff).toContain('---');
expect(diff).toContain('+++');
expect(diff).toContain('previous snapshot');
expect(diff).toContain('current snapshot');
});
test('snapshot -D with identical page shows no changes', async () => {
await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm);
await handleMetaCommand('snapshot', [], bm, shutdown);
const diff = await handleMetaCommand('snapshot', ['-D'], bm, shutdown);
// All lines should be unchanged (prefixed with space)
const lines = diff.split('\n').filter(l => l.startsWith('+') || l.startsWith('-'));
// Header lines start with --- and +++ so filter those
const contentChanges = lines.filter(l => !l.startsWith('---') && !l.startsWith('+++'));
expect(contentChanges.length).toBe(0);
});
});
// ─── Annotated Screenshots ─────────────────────────────────────
describe('Annotated screenshots', () => {
test('snapshot -a creates annotated screenshot', async () => {
const screenshotPath = '/tmp/browse-test-annotated.png';
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const result = await handleMetaCommand('snapshot', ['-a', '-o', screenshotPath], bm, shutdown);
expect(result).toContain('annotated screenshot');
expect(result).toContain(screenshotPath);
expect(fs.existsSync(screenshotPath)).toBe(true);
const stat = fs.statSync(screenshotPath);
expect(stat.size).toBeGreaterThan(1000);
fs.unlinkSync(screenshotPath);
});
test('snapshot -a uses default path', async () => {
const defaultPath = '/tmp/browse-annotated.png';
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const result = await handleMetaCommand('snapshot', ['-a'], bm, shutdown);
expect(result).toContain('annotated screenshot');
expect(fs.existsSync(defaultPath)).toBe(true);
fs.unlinkSync(defaultPath);
});
test('snapshot -a -i only annotates interactive', async () => {
const screenshotPath = '/tmp/browse-test-annotated-i.png';
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const result = await handleMetaCommand('snapshot', ['-i', '-a', '-o', screenshotPath], bm, shutdown);
expect(result).toContain('[button]');
expect(result).toContain('[link]');
expect(result).toContain('annotated screenshot');
if (fs.existsSync(screenshotPath)) fs.unlinkSync(screenshotPath);
});
test('annotation overlays are cleaned up', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
await handleMetaCommand('snapshot', ['-a'], bm, shutdown);
// Check that overlays are removed
const overlays = await handleReadCommand('js', ['document.querySelectorAll(".__browse_annotation__").length'], bm);
expect(overlays).toBe('0');
// Clean up default file
try { fs.unlinkSync('/tmp/browse-annotated.png'); } catch {}
});
});
// ─── Cursor-Interactive ────────────────────────────────────────
describe('Cursor-interactive', () => {
test('snapshot -C finds cursor:pointer elements', async () => {
await handleWriteCommand('goto', [baseUrl + '/cursor-interactive.html'], bm);
const result = await handleMetaCommand('snapshot', ['-C'], bm, shutdown);
expect(result).toContain('cursor-interactive');
expect(result).toContain('@c');
expect(result).toContain('cursor:pointer');
});
test('snapshot -C includes onclick elements', async () => {
await handleWriteCommand('goto', [baseUrl + '/cursor-interactive.html'], bm);
const result = await handleMetaCommand('snapshot', ['-C'], bm, shutdown);
expect(result).toContain('onclick');
});
test('snapshot -C includes tabindex elements', async () => {
await handleWriteCommand('goto', [baseUrl + '/cursor-interactive.html'], bm);
const result = await handleMetaCommand('snapshot', ['-C'], bm, shutdown);
expect(result).toContain('tabindex');
});
test('@c ref is clickable', async () => {
await handleWriteCommand('goto', [baseUrl + '/cursor-interactive.html'], bm);
const snap = await handleMetaCommand('snapshot', ['-C'], bm, shutdown);
// Find a @c ref
const cLine = snap.split('\n').find(l => l.includes('@c'));
if (cLine) {
const refMatch = cLine.match(/@(c\d+)/);
if (refMatch) {
const result = await handleWriteCommand('click', [`@${refMatch[1]}`], bm);
expect(result).toContain('Clicked');
}
}
});
test('snapshot -C on page with no cursor elements', async () => {
await handleWriteCommand('goto', [baseUrl + '/empty.html'], bm);
const result = await handleMetaCommand('snapshot', ['-C'], bm, shutdown);
// Should not contain cursor-interactive section
expect(result).not.toContain('cursor-interactive');
});
test('snapshot -i -C combines both modes', async () => {
await handleWriteCommand('goto', [baseUrl + '/cursor-interactive.html'], bm);
const result = await handleMetaCommand('snapshot', ['-i', '-C'], bm, shutdown);
// Should have interactive elements (button, link)
expect(result).toContain('[button]');
expect(result).toContain('[link]');
// And cursor-interactive section
expect(result).toContain('cursor-interactive');
});
});
// ─── Snapshot Error Paths ───────────────────────────────────────
describe('Snapshot errors', () => {
test('unknown flag throws', async () => {
try {
await handleMetaCommand('snapshot', ['--bogus'], bm, shutdown);
expect(true).toBe(false);
} catch (err: any) {
expect(err.message).toContain('Unknown snapshot flag');
}
});
test('-d without number throws', async () => {
try {
await handleMetaCommand('snapshot', ['-d'], bm, shutdown);
expect(true).toBe(false);
} catch (err: any) {
expect(err.message).toContain('Usage');
}
});
test('-s without selector throws', async () => {
try {
await handleMetaCommand('snapshot', ['-s'], bm, shutdown);
expect(true).toBe(false);
} catch (err: any) {
expect(err.message).toContain('Usage');
}
});
test('-s with nonexistent selector throws', async () => {
await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm);
try {
await handleMetaCommand('snapshot', ['-s', '#nonexistent-element-12345'], bm, shutdown);
expect(true).toBe(false);
} catch (err: any) {
expect(err.message).toContain('Selector not found');
}
});
test('-o without path throws', async () => {
try {
await handleMetaCommand('snapshot', ['-o'], bm, shutdown);
expect(true).toBe(false);
} catch (err: any) {
expect(err.message).toContain('Usage');
}
});
});
// ─── Combined Flags ─────────────────────────────────────────────
describe('Snapshot combined flags', () => {
test('-i -c -d 2 combines all filters', async () => {
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
const result = await handleMetaCommand('snapshot', ['-i', '-c', '-d', '2'], bm, shutdown);
// Should be filtered to interactive, compact, shallow
expect(result).toContain('[button]');
expect(result).toContain('[link]');
// Should NOT contain deep nested non-interactive elements
expect(result).not.toContain('[heading]');
});
test('closetab last tab auto-creates new', async () => {
// Get down to 1 tab
const tabs = await bm.getTabListWithTitles();
for (let i = 1; i < tabs.length; i++) {
await bm.closeTab(tabs[i].id);
}
expect(bm.getTabCount()).toBe(1);
// Close the last tab
const lastTab = (await bm.getTabListWithTitles())[0];
await bm.closeTab(lastTab.id);
// Should have auto-created a new tab
expect(bm.getTabCount()).toBe(1);
});
});