mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-02 03:35:09 +02:00
Garry Tan
5205070299
* refactor: extract command registry to commands.ts, add SNAPSHOT_FLAGS metadata
- NEW: browse/src/commands.ts — command sets + COMMAND_DESCRIPTIONS + load-time validation (zero side effects)
- server.ts imports from commands.ts instead of declaring sets inline
- snapshot.ts: SNAPSHOT_FLAGS array drives parseSnapshotArgs (metadata-driven, no duplication)
- All 186 existing tests pass
* feat: SKILL.md template system with auto-generated command references
- SKILL.md.tmpl + browse/SKILL.md.tmpl with {{COMMAND_REFERENCE}} and {{SNAPSHOT_FLAGS}} placeholders
- scripts/gen-skill-docs.ts generates SKILL.md from templates (supports --dry-run)
- Build pipeline runs gen:skill-docs before binary compilation
- Generated files have AUTO-GENERATED header, committed to git
* test: Tier 1 static validation — 34 tests for SKILL.md command correctness
- test/helpers/skill-parser.ts: extracts $B commands from code blocks, validates against registry
- test/skill-parser.test.ts: 13 parser/validator unit tests
- test/skill-validation.test.ts: 13 tests validating all SKILL.md files + registry consistency
- test/gen-skill-docs.test.ts: 8 generator tests (categories, sorting, freshness)
* feat: DX tools (skill:check, dev:skill) + Tier 2 E2E test scaffolding
- scripts/skill-check.ts: health summary for all SKILL.md files (commands, templates, freshness)
- scripts/dev-skill.ts: watch mode for template development
- test/helpers/session-runner.ts: Agent SDK wrapper for E2E skill tests
- test/skill-e2e.test.ts: 2 E2E tests + 3 stubs (auto-skip inside Claude Code sessions)
- E2E tests must run from plain terminal: SKILL_E2E=1 bun test test/skill-e2e.test.ts
* ci: SKILL.md freshness check on push/PR + TODO updates
- .github/workflows/skill-docs.yml: fails if generated SKILL.md files are stale
- TODO.md: add E2E cost tracking and model pinning to future ideas
* fix: restore rich descriptions lost in auto-generation
- Snapshot flags: add back value hints (-d <N>, -s <sel>, -o <path>)
- Snapshot flags: restore parenthetical context (@e refs, @c refs, etc.)
- Commands: is → includes valid states enum
- Commands: console → notes --errors filter behavior
- Commands: press → lists common keys (Enter, Tab, Escape)
- Commands: cookie-import-browser → describes picker UI
- Commands: dialog-accept → specifies alert/confirm/prompt
- Tips: restore → arrow (was downgraded to ->)
* test: quality evals for generated SKILL.md descriptions
Catches the exact regressions we shipped and caught in review:
- Snapshot flags must include value hints (-d <N>, -s <sel>, -o <path>)
- is command must list all valid states (visible/hidden/enabled/...)
- press command must list example keys (Enter, Tab, Escape)
- console command must describe --errors behavior
- Snapshot -i must mention @e refs, -C must mention @c refs
- All descriptions must be >= 8 chars (no empty stubs)
- Tips section must use → not ->
* feat: LLM-as-judge evals for SKILL.md documentation quality
4 eval tests using Anthropic API (claude-haiku, ~$0.01-0.03/run):
- Command reference table: clarity/completeness/actionability >= 4/5
- Snapshot flags section: same thresholds
- browse/SKILL.md overall quality
- Regression: generated version must score >= hand-maintained baseline
Requires ANTHROPIC_API_KEY. Auto-skips without it.
Run: bun run test:eval (or ANTHROPIC_API_KEY=sk-... bun test test/skill-llm-eval.test.ts)
* chore: bump version to 0.3.3, update changelog
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add ARCHITECTURE.md, update CLAUDE.md and CONTRIBUTING.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: conductor.json lifecycle hooks + .env propagation across worktrees
bin/dev-setup now copies .env from main worktree so API keys carry
over to Conductor workspaces automatically. conductor.json wires up
setup and archive hooks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: complete CHANGELOG for v0.3.3 (architecture, conductor, .env)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
120 lines
6.9 KiB
Markdown
120 lines
6.9 KiB
Markdown
# TODO — gstack roadmap
|
|
|
|
## Phase 1: Foundations (v0.2.0)
|
|
- [x] Rename to gstack
|
|
- [x] Restructure to monorepo layout
|
|
- [x] Setup script for skill symlinks
|
|
- [x] Snapshot command with ref-based element selection
|
|
- [x] Snapshot tests
|
|
|
|
## Phase 2: Enhanced Browser (v0.2.0) ✅
|
|
- [x] Annotated screenshots (--annotate flag, ref labels overlaid on screenshot)
|
|
- [x] Snapshot diffing (--diff flag, unified diff against previous snapshot)
|
|
- [x] Dialog handling (auto-accept/dismiss, dialog buffer, prevents browser lockup)
|
|
- [x] File upload (upload <sel> <files>)
|
|
- [x] Cursor-interactive elements (-C flag, cursor:pointer/onclick/tabindex scan)
|
|
- [x] Element state checks (is visible/hidden/enabled/disabled/checked/editable/focused)
|
|
- [x] CircularBuffer — O(1) ring buffer for console/network/dialog (was O(n) array+shift)
|
|
- [x] Async buffer flush with Bun.write() (was appendFileSync)
|
|
- [x] Health check with page.evaluate('1') + 2s timeout
|
|
- [x] Playwright error wrapping — actionable messages for AI agents
|
|
- [x] Fix useragent — context recreation preserves cookies/storage/URLs
|
|
- [x] DRY: getCleanText exported, command sets in chain updated
|
|
- [x] 148 integration tests (was ~63)
|
|
|
|
## Phase 3: QA Testing Agent (v0.3.0)
|
|
- [x] `/qa` SKILL.md — 6-phase workflow: Initialize → Authenticate → Orient → Explore → Document → Wrap up
|
|
- [x] Issue taxonomy reference (7 categories: visual, functional, UX, content, performance, console, accessibility)
|
|
- [x] Severity classification (critical/high/medium/low)
|
|
- [x] Exploration checklist per page
|
|
- [x] Report template (structured markdown with per-issue evidence)
|
|
- [x] Repro-first philosophy: every issue gets evidence before moving on
|
|
- [x] Two evidence tiers: interactive bugs (multi-step screenshots), static bugs (single annotated screenshot)
|
|
- [x] Key guidance: 5-10 well-documented issues per session, depth over breadth, write incrementally
|
|
- [x] Three modes: full (systematic), quick (30-second smoke test), regression (compare against baseline)
|
|
- [x] Framework detection guidance (Next.js, Rails, WordPress, SPA)
|
|
- [x] Health score rubric (7 categories, weighted average)
|
|
- [x] `wait --networkidle` / `wait --load` / `wait --domcontentloaded`
|
|
- [x] `console --errors` (filter to error/warning only)
|
|
- [x] `cookie-import <json-file>` (bulk cookie import with auto-fill domain)
|
|
- [x] `browse/bin/find-browse` (DRY binary discovery across skills)
|
|
- [ ] Video recording (deferred to Phase 5 — recreateContext destroys page state)
|
|
|
|
## Phase 3.5: Browser Cookie Import (v0.3.x)
|
|
- [x] `cookie-import-browser` command (Chromium cookie DB decryption)
|
|
- [x] Cookie picker web UI (served from browse server)
|
|
- [x] `/setup-browser-cookies` skill
|
|
- [x] Unit tests with encrypted cookie fixtures (18 tests)
|
|
- [x] Browser registry (Comet, Chrome, Arc, Brave, Edge)
|
|
|
|
## Phase 3.6: Visual PR Annotations + S3 Upload
|
|
- [ ] `/setup-gstack-upload` skill (configure S3 bucket for image hosting)
|
|
- [ ] `browse/bin/gstack-upload` helper (upload file to S3, return public URL)
|
|
- [ ] `/ship` Step 7.5: visual verification with screenshots in PR body
|
|
- [ ] `/review` Step 4.5: visual review with annotated screenshots in PR
|
|
- [ ] WebM → GIF conversion (ffmpeg) for video evidence in PRs
|
|
- [ ] README documentation for visual PR annotations
|
|
|
|
## Phase 4: Skill + Browser Integration
|
|
- [ ] ship + browse: post-deploy verification
|
|
- Browse staging/preview URL after push
|
|
- Screenshot key pages
|
|
- Check console for JS errors
|
|
- Compare staging vs prod via snapshot diff
|
|
- Include verification screenshots in PR body
|
|
- STOP if critical errors found
|
|
- [ ] review + browse: visual diff review
|
|
- Browse PR's preview deploy
|
|
- Annotated screenshots of changed pages
|
|
- Compare against production visually
|
|
- Check responsive layouts (mobile/tablet/desktop)
|
|
- Verify accessibility tree hasn't regressed
|
|
- [ ] deploy-verify skill: lightweight post-deploy smoke test
|
|
- Hit key URLs, verify 200s
|
|
- Screenshot critical pages
|
|
- Console error check
|
|
- Compare against baseline snapshots
|
|
- Pass/fail with evidence
|
|
|
|
## Phase 5: State & Sessions
|
|
- [ ] Bundle server.ts into compiled binary (eliminate resolveServerScript() fallback chain entirely) (P2, M)
|
|
- [ ] v20 encryption format support (AES-256-GCM) — future Chromium versions may change from v10
|
|
- [ ] Sessions (isolated browser instances with separate cookies/storage/history)
|
|
- [ ] State persistence (save/load cookies + localStorage to JSON files)
|
|
- [ ] Auth vault (encrypted credential storage, referenced by name, LLM never sees passwords)
|
|
- [ ] Video recording (record start/stop — needs sessions for clean context lifecycle)
|
|
- [ ] retro + browse: deployment health tracking
|
|
- Screenshot production state
|
|
- Check perf metrics (page load times)
|
|
- Count console errors across key pages
|
|
- Track trends over retro window
|
|
|
|
## Phase 6: Advanced Browser
|
|
- [ ] Iframe support (frame <sel>, frame main)
|
|
- [ ] Semantic locators (find role/label/text/placeholder/testid with actions)
|
|
- [ ] Device emulation presets (set device "iPhone 16 Pro")
|
|
- [ ] Network mocking/routing (intercept, block, mock requests)
|
|
- [ ] Download handling (click-to-download with path control)
|
|
- [ ] Content safety (--max-output truncation, --allowed-domains)
|
|
- [ ] Streaming (WebSocket live preview for pair browsing)
|
|
- [ ] CDP mode (connect to already-running Chrome/Electron apps)
|
|
|
|
## Future Ideas
|
|
- [ ] Linux/Windows cookie decryption (GNOME Keyring / kwallet / DPAPI)
|
|
- [ ] Trend tracking across QA runs — compare baseline.json over time, detect regressions (P2, S)
|
|
- [ ] CI/CD integration — `/qa` as GitHub Action step, fail PR if health score drops (P2, M)
|
|
- [ ] Accessibility audit mode — `--a11y` flag for focused accessibility testing (P3, S)
|
|
- [ ] Greptile training feedback loop — export suppression patterns to Greptile team for model improvement (P3, S)
|
|
- [ ] E2E test cost tracking — track cumulative API spend, warn if over threshold (P3, S)
|
|
- [ ] E2E model pinning — pin E2E tests to claude-sonnet-4-6 for cost efficiency, add retry:2 for flaky LLM (P2, XS)
|
|
|
|
## Ideas & Notes
|
|
- Browser is the nervous system — every skill should be able to see, interact with, and verify the web
|
|
- Skills are the product; the browser enables them
|
|
- One repo, one install, entire AI engineering workflow
|
|
- Bun compiled binary matches Rust CLI performance for this use case (bottleneck is Chromium, not CLI parsing)
|
|
- Accessibility tree snapshots use ~200-400 tokens vs ~3000-5000 for full DOM — critical for AI context efficiency
|
|
- Locator map approach for refs: store Map<string, Locator> on BrowserManager, no DOM mutation, no CSP issues
|
|
- Snapshot scoping (-i, -c, -d, -s flags) is critical for performance on large pages
|
|
- All new commands follow existing pattern: add to command set, add switch case, return string
|