mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-02 11:45:20 +02:00
Garry Tan
5205070299
* refactor: extract command registry to commands.ts, add SNAPSHOT_FLAGS metadata
- NEW: browse/src/commands.ts — command sets + COMMAND_DESCRIPTIONS + load-time validation (zero side effects)
- server.ts imports from commands.ts instead of declaring sets inline
- snapshot.ts: SNAPSHOT_FLAGS array drives parseSnapshotArgs (metadata-driven, no duplication)
- All 186 existing tests pass
* feat: SKILL.md template system with auto-generated command references
- SKILL.md.tmpl + browse/SKILL.md.tmpl with {{COMMAND_REFERENCE}} and {{SNAPSHOT_FLAGS}} placeholders
- scripts/gen-skill-docs.ts generates SKILL.md from templates (supports --dry-run)
- Build pipeline runs gen:skill-docs before binary compilation
- Generated files have AUTO-GENERATED header, committed to git
* test: Tier 1 static validation — 34 tests for SKILL.md command correctness
- test/helpers/skill-parser.ts: extracts $B commands from code blocks, validates against registry
- test/skill-parser.test.ts: 13 parser/validator unit tests
- test/skill-validation.test.ts: 13 tests validating all SKILL.md files + registry consistency
- test/gen-skill-docs.test.ts: 8 generator tests (categories, sorting, freshness)
* feat: DX tools (skill:check, dev:skill) + Tier 2 E2E test scaffolding
- scripts/skill-check.ts: health summary for all SKILL.md files (commands, templates, freshness)
- scripts/dev-skill.ts: watch mode for template development
- test/helpers/session-runner.ts: Agent SDK wrapper for E2E skill tests
- test/skill-e2e.test.ts: 2 E2E tests + 3 stubs (auto-skip inside Claude Code sessions)
- E2E tests must run from plain terminal: SKILL_E2E=1 bun test test/skill-e2e.test.ts
* ci: SKILL.md freshness check on push/PR + TODO updates
- .github/workflows/skill-docs.yml: fails if generated SKILL.md files are stale
- TODO.md: add E2E cost tracking and model pinning to future ideas
* fix: restore rich descriptions lost in auto-generation
- Snapshot flags: add back value hints (-d <N>, -s <sel>, -o <path>)
- Snapshot flags: restore parenthetical context (@e refs, @c refs, etc.)
- Commands: is → includes valid states enum
- Commands: console → notes --errors filter behavior
- Commands: press → lists common keys (Enter, Tab, Escape)
- Commands: cookie-import-browser → describes picker UI
- Commands: dialog-accept → specifies alert/confirm/prompt
- Tips: restore → arrow (was downgraded to ->)
* test: quality evals for generated SKILL.md descriptions
Catches the exact regressions we shipped and caught in review:
- Snapshot flags must include value hints (-d <N>, -s <sel>, -o <path>)
- is command must list all valid states (visible/hidden/enabled/...)
- press command must list example keys (Enter, Tab, Escape)
- console command must describe --errors behavior
- Snapshot -i must mention @e refs, -C must mention @c refs
- All descriptions must be >= 8 chars (no empty stubs)
- Tips section must use → not ->
* feat: LLM-as-judge evals for SKILL.md documentation quality
4 eval tests using Anthropic API (claude-haiku, ~$0.01-0.03/run):
- Command reference table: clarity/completeness/actionability >= 4/5
- Snapshot flags section: same thresholds
- browse/SKILL.md overall quality
- Regression: generated version must score >= hand-maintained baseline
Requires ANTHROPIC_API_KEY. Auto-skips without it.
Run: bun run test:eval (or ANTHROPIC_API_KEY=sk-... bun test test/skill-llm-eval.test.ts)
* chore: bump version to 0.3.3, update changelog
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add ARCHITECTURE.md, update CLAUDE.md and CONTRIBUTING.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: conductor.json lifecycle hooks + .env propagation across worktrees
bin/dev-setup now copies .env from main worktree so API keys carry
over to Conductor workspaces automatically. conductor.json wires up
setup and archive hooks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: complete CHANGELOG for v0.3.3 (architecture, conductor, .env)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
6.9 KiB
6.9 KiB
TODO — gstack roadmap
Phase 1: Foundations (v0.2.0)
- Rename to gstack
- Restructure to monorepo layout
- Setup script for skill symlinks
- Snapshot command with ref-based element selection
- Snapshot tests
Phase 2: Enhanced Browser (v0.2.0) ✅
- Annotated screenshots (--annotate flag, ref labels overlaid on screenshot)
- Snapshot diffing (--diff flag, unified diff against previous snapshot)
- Dialog handling (auto-accept/dismiss, dialog buffer, prevents browser lockup)
- File upload (upload )
- Cursor-interactive elements (-C flag, cursor:pointer/onclick/tabindex scan)
- Element state checks (is visible/hidden/enabled/disabled/checked/editable/focused)
- CircularBuffer — O(1) ring buffer for console/network/dialog (was O(n) array+shift)
- Async buffer flush with Bun.write() (was appendFileSync)
- Health check with page.evaluate('1') + 2s timeout
- Playwright error wrapping — actionable messages for AI agents
- Fix useragent — context recreation preserves cookies/storage/URLs
- DRY: getCleanText exported, command sets in chain updated
- 148 integration tests (was ~63)
Phase 3: QA Testing Agent (v0.3.0)
/qaSKILL.md — 6-phase workflow: Initialize → Authenticate → Orient → Explore → Document → Wrap up- Issue taxonomy reference (7 categories: visual, functional, UX, content, performance, console, accessibility)
- Severity classification (critical/high/medium/low)
- Exploration checklist per page
- Report template (structured markdown with per-issue evidence)
- Repro-first philosophy: every issue gets evidence before moving on
- Two evidence tiers: interactive bugs (multi-step screenshots), static bugs (single annotated screenshot)
- Key guidance: 5-10 well-documented issues per session, depth over breadth, write incrementally
- Three modes: full (systematic), quick (30-second smoke test), regression (compare against baseline)
- Framework detection guidance (Next.js, Rails, WordPress, SPA)
- Health score rubric (7 categories, weighted average)
wait --networkidle/wait --load/wait --domcontentloadedconsole --errors(filter to error/warning only)cookie-import <json-file>(bulk cookie import with auto-fill domain)browse/bin/find-browse(DRY binary discovery across skills)- Video recording (deferred to Phase 5 — recreateContext destroys page state)
Phase 3.5: Browser Cookie Import (v0.3.x)
cookie-import-browsercommand (Chromium cookie DB decryption)- Cookie picker web UI (served from browse server)
/setup-browser-cookiesskill- Unit tests with encrypted cookie fixtures (18 tests)
- Browser registry (Comet, Chrome, Arc, Brave, Edge)
Phase 3.6: Visual PR Annotations + S3 Upload
/setup-gstack-uploadskill (configure S3 bucket for image hosting)browse/bin/gstack-uploadhelper (upload file to S3, return public URL)/shipStep 7.5: visual verification with screenshots in PR body/reviewStep 4.5: visual review with annotated screenshots in PR- WebM → GIF conversion (ffmpeg) for video evidence in PRs
- README documentation for visual PR annotations
Phase 4: Skill + Browser Integration
- ship + browse: post-deploy verification
- Browse staging/preview URL after push
- Screenshot key pages
- Check console for JS errors
- Compare staging vs prod via snapshot diff
- Include verification screenshots in PR body
- STOP if critical errors found
- review + browse: visual diff review
- Browse PR's preview deploy
- Annotated screenshots of changed pages
- Compare against production visually
- Check responsive layouts (mobile/tablet/desktop)
- Verify accessibility tree hasn't regressed
- deploy-verify skill: lightweight post-deploy smoke test
- Hit key URLs, verify 200s
- Screenshot critical pages
- Console error check
- Compare against baseline snapshots
- Pass/fail with evidence
Phase 5: State & Sessions
- Bundle server.ts into compiled binary (eliminate resolveServerScript() fallback chain entirely) (P2, M)
- v20 encryption format support (AES-256-GCM) — future Chromium versions may change from v10
- Sessions (isolated browser instances with separate cookies/storage/history)
- State persistence (save/load cookies + localStorage to JSON files)
- Auth vault (encrypted credential storage, referenced by name, LLM never sees passwords)
- Video recording (record start/stop — needs sessions for clean context lifecycle)
- retro + browse: deployment health tracking
- Screenshot production state
- Check perf metrics (page load times)
- Count console errors across key pages
- Track trends over retro window
Phase 6: Advanced Browser
- Iframe support (frame , frame main)
- Semantic locators (find role/label/text/placeholder/testid with actions)
- Device emulation presets (set device "iPhone 16 Pro")
- Network mocking/routing (intercept, block, mock requests)
- Download handling (click-to-download with path control)
- Content safety (--max-output truncation, --allowed-domains)
- Streaming (WebSocket live preview for pair browsing)
- CDP mode (connect to already-running Chrome/Electron apps)
Future Ideas
- Linux/Windows cookie decryption (GNOME Keyring / kwallet / DPAPI)
- Trend tracking across QA runs — compare baseline.json over time, detect regressions (P2, S)
- CI/CD integration —
/qaas GitHub Action step, fail PR if health score drops (P2, M) - Accessibility audit mode —
--a11yflag for focused accessibility testing (P3, S) - Greptile training feedback loop — export suppression patterns to Greptile team for model improvement (P3, S)
- E2E test cost tracking — track cumulative API spend, warn if over threshold (P3, S)
- E2E model pinning — pin E2E tests to claude-sonnet-4-6 for cost efficiency, add retry:2 for flaky LLM (P2, XS)
Ideas & Notes
- Browser is the nervous system — every skill should be able to see, interact with, and verify the web
- Skills are the product; the browser enables them
- One repo, one install, entire AI engineering workflow
- Bun compiled binary matches Rust CLI performance for this use case (bottleneck is Chromium, not CLI parsing)
- Accessibility tree snapshots use ~200-400 tokens vs ~3000-5000 for full DOM — critical for AI context efficiency
- Locator map approach for refs: store Map<string, Locator> on BrowserManager, no DOM mutation, no CSP issues
- Snapshot scoping (-i, -c, -d, -s flags) is critical for performance on large pages
- All new commands follow existing pattern: add to command set, add switch case, return string