mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-08 06:26:45 +02:00
Merge branch 'main' into garrytan/team-supabase-store
Brings in 55 commits from main (v0.12.x–v0.13.5.0): Factory Droid compat, prompt injection defense, user sovereignty, security audit, design binary, skill namespacing, modular resolvers, Chrome sidebar, and more. Conflict resolution: - .agents/ SKILL.md files: deleted (main moved to .factory/) - 8 .tmpl templates: accepted main (new features: CDP mode, design tools, global retro, parallelization, distribution checks, plan audits) - scripts/gen-skill-docs.ts: accepted main's modular resolver refactor - test/helpers/session-runner.ts: accepted main + layered back CostEntry tracking from team branch - Generated SKILL.md files: regenerated via bun run gen:skill-docs - Updated tests to match main's gstack-slug output (2 lines, no PROJECTS_DIR) and review log mechanism (gstack-review-log, not $BRANCH.jsonl) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
@@ -1,5 +1,19 @@
|
||||
# TODOS
|
||||
|
||||
## Sidebar Security
|
||||
|
||||
### ML Prompt Injection Classifier
|
||||
|
||||
**What:** Add DeBERTa-v3-base-prompt-injection-v2 via @huggingface/transformers v4 (WASM backend) as an ML defense layer for the Chrome sidebar. Reusable `browse/src/security.ts` module with `checkInjection()` API. Includes canary tokens, attack logging, shield icon, special telemetry (AskUserQuestion on detection even when telemetry off), and BrowseSafe-bench red team test harness (3,680 adversarial cases from Perplexity).
|
||||
|
||||
**Why:** PR 1 fixes the architecture (command allowlist, XML framing, Opus default). But attackers can still trick Claude into navigating to phishing sites or exfiltrating visible page data via allowed browse commands. The ML classifier catches prompt injection patterns that architectural controls can't see. 94.8% accuracy, 99.6% recall, ~50-100ms inference via WASM. Defense-in-depth.
|
||||
|
||||
**Context:** Full design doc with industry research, open source tool landscape, Codex review findings, and ambitious Bun-native vision (5ms inference via FFI + Apple Accelerate): [`docs/designs/ML_PROMPT_INJECTION_KILLER.md`](docs/designs/ML_PROMPT_INJECTION_KILLER.md). CEO plan with scope decisions: `~/.gstack/projects/garrytan-gstack/ceo-plans/2026-03-28-sidebar-prompt-injection-defense.md`.
|
||||
|
||||
**Effort:** L (human: ~2 weeks / CC: ~3-4 hours)
|
||||
**Priority:** P0
|
||||
**Depends on:** Sidebar security fix PR (command allowlist + XML framing + arg fix) landing first
|
||||
|
||||
## Builder Ethos
|
||||
|
||||
### First-time Search Before Building intro
|
||||
@@ -14,6 +28,26 @@
|
||||
**Priority:** P2
|
||||
**Depends on:** Blog post about Search Before Building
|
||||
|
||||
## Chrome DevTools MCP Integration
|
||||
|
||||
### Real Chrome session access
|
||||
|
||||
**What:** Integrate Chrome DevTools MCP to connect to the user's real Chrome session with real cookies, real state, no Playwright middleman.
|
||||
|
||||
**Why:** Right now, headed mode launches a fresh Chromium profile. Users must log in manually or import cookies. Chrome DevTools MCP connects to the user's actual Chrome ... instant access to every authenticated site. This is the future of browser automation for AI agents.
|
||||
|
||||
**Context:** Google shipped Chrome DevTools MCP in Chrome 146+ (June 2025). It provides screenshots, console messages, performance traces, Lighthouse audits, and full page interaction through the user's real browser. gstack should use it for real-session access while keeping Playwright for headless CI/testing workflows.
|
||||
|
||||
Potential new skills:
|
||||
- `/debug-browser`: JS error tracing with source-mapped stack traces
|
||||
- `/perf-debug`: performance traces, Core Web Vitals, network waterfall
|
||||
|
||||
May replace `/setup-browser-cookies` for most use cases since the user's real cookies are already there.
|
||||
|
||||
**Effort:** L (human: ~2 weeks / CC: ~2 hours)
|
||||
**Priority:** P0
|
||||
**Depends on:** Chrome 146+, DevTools MCP server installed
|
||||
|
||||
## Browse
|
||||
|
||||
### Bundle server.ts into compiled binary
|
||||
@@ -60,17 +94,14 @@
|
||||
**Effort:** S
|
||||
**Priority:** P3
|
||||
|
||||
### State persistence
|
||||
### State persistence — SHIPPED
|
||||
|
||||
**What:** Save/load cookies + localStorage to JSON files for reproducible test sessions.
|
||||
~~**What:** Save/load cookies + localStorage to JSON files for reproducible test sessions.~~
|
||||
|
||||
**Why:** Enables "resume where I left off" for QA sessions and repeatable auth states.
|
||||
`$B state save/load` ships in v0.12.1.0. V1 saves cookies + URLs only (not localStorage, which breaks on load-before-navigate). Files at `.gstack/browse-states/{name}.json` with 0o600 permissions. Load replaces session (closes all pages first). Name sanitized to `[a-zA-Z0-9_-]`.
|
||||
|
||||
**Context:** The `saveState()`/`restoreState()` helpers from the handoff feature (browser-manager.ts) already capture cookies + localStorage + sessionStorage + URLs. Adding file I/O on top is ~20 lines.
|
||||
|
||||
**Effort:** S
|
||||
**Priority:** P3
|
||||
**Depends on:** Sessions
|
||||
**Remaining:** V2 localStorage support (needs pre-navigation injection strategy).
|
||||
**Completed:** v0.12.1.0 (2026-03-26)
|
||||
|
||||
### Auth vault
|
||||
|
||||
@@ -82,14 +113,13 @@
|
||||
**Priority:** P3
|
||||
**Depends on:** Sessions, state persistence
|
||||
|
||||
### Iframe support
|
||||
### Iframe support — SHIPPED
|
||||
|
||||
**What:** `frame <sel>` and `frame main` commands for cross-frame interaction.
|
||||
~~**What:** `frame <sel>` and `frame main` commands for cross-frame interaction.~~
|
||||
|
||||
**Why:** Many web apps use iframes (embeds, payment forms, ads). Currently invisible to browse.
|
||||
`$B frame` ships in v0.12.1.0. Supports CSS selector, @ref, `--name`, and `--url` pattern matching. Execution target abstraction (`getActiveFrameOrPage()`) across all read/write/snapshot commands. Frame context cleared on navigation, tab switch, resume. Detached frame auto-recovery. Page-only operations (goto, screenshot, viewport) throw clear error when in frame context.
|
||||
|
||||
**Effort:** M
|
||||
**Priority:** P4
|
||||
**Completed:** v0.12.1.0 (2026-03-26)
|
||||
|
||||
### Semantic locators
|
||||
|
||||
@@ -145,26 +175,90 @@
|
||||
**Effort:** L
|
||||
**Priority:** P4
|
||||
|
||||
### CDP mode
|
||||
### Headed mode with Chrome extension — SHIPPED
|
||||
|
||||
**What:** Connect to already-running Chrome/Electron apps via Chrome DevTools Protocol.
|
||||
`$B connect` launches Playwright's bundled Chromium in headed mode with the gstack Chrome extension auto-loaded. `$B handoff` now produces the same result (extension + side panel). Sidebar chat gated behind `--chat` flag.
|
||||
|
||||
**Why:** Test production apps, Electron apps, and existing browser sessions without launching new instances.
|
||||
### `$B watch` — SHIPPED
|
||||
|
||||
**Effort:** M
|
||||
Claude observes user browsing in passive read-only mode with periodic snapshots. `$B watch stop` exits with summary. Mutation commands blocked during watch.
|
||||
|
||||
### Sidebar scout / file drop relay — SHIPPED
|
||||
|
||||
Sidebar agent writes structured messages to `.context/sidebar-inbox/`. Workspace agent reads via `$B inbox`. Message format: `{type, timestamp, page, userMessage, sidebarSessionId}`.
|
||||
|
||||
### Multi-agent tab isolation
|
||||
|
||||
**What:** Two Claude sessions connect to the same browser, each operating on different tabs. No cross-contamination.
|
||||
|
||||
**Why:** Enables parallel /qa + /design-review on different tabs in the same browser.
|
||||
|
||||
**Context:** Requires tab ownership model for concurrent headed connections. Playwright may not cleanly support two persistent contexts. Needs investigation.
|
||||
|
||||
**Effort:** L (human: ~2 weeks / CC: ~2 hours)
|
||||
**Priority:** P3
|
||||
**Depends on:** Headed mode (shipped)
|
||||
|
||||
### Sidebar agent needs Write tool + better error visibility
|
||||
|
||||
**What:** Two issues with the sidebar agent (`sidebar-agent.ts`): (1) `--allowedTools` is hardcoded to `Bash,Read,Glob,Grep`, missing `Write`. Claude can't create files (like CSVs) when asked. (2) When Claude errors or returns empty, the sidebar UI shows nothing, just a green dot. No error message, no "I tried but failed", nothing.
|
||||
|
||||
**Why:** Users ask "write this to a CSV" and the sidebar silently can't. Then they think it's broken. The UI needs to surface errors visibly, and Claude needs the tools to actually do what's asked.
|
||||
|
||||
**Context:** `sidebar-agent.ts:163` hardcodes `--allowedTools`. The event relay (`handleStreamEvent`) handles `agent_done` and `agent_error` but the extension's sidepanel.js may not be rendering error states. The sidebar should show "Error: ..." or "Claude finished but produced no output" instead of staying on the green dot forever.
|
||||
|
||||
**Effort:** S (human: ~2h / CC: ~10min)
|
||||
**Priority:** P1
|
||||
**Depends on:** None
|
||||
|
||||
### Chrome Web Store publishing
|
||||
|
||||
**What:** Publish the gstack browse Chrome extension to Chrome Web Store for easier install.
|
||||
|
||||
**Why:** Currently sideloaded via chrome://extensions. Web Store makes install one-click.
|
||||
|
||||
**Effort:** S
|
||||
**Priority:** P4
|
||||
**Depends on:** Chrome extension proving value via sideloading
|
||||
|
||||
### Linux/Windows cookie decryption
|
||||
### Linux cookie decryption — PARTIALLY SHIPPED
|
||||
|
||||
**What:** GNOME Keyring / kwallet / DPAPI support for non-macOS cookie import.
|
||||
~~**What:** GNOME Keyring / kwallet / DPAPI support for non-macOS cookie import.~~
|
||||
|
||||
**Why:** Cross-platform cookie import. Currently macOS-only (Keychain).
|
||||
Linux cookie import shipped in v0.11.11.0 (Wave 3). Supports Chrome, Chromium, Brave, Edge on Linux with GNOME Keyring (libsecret) and "peanuts" fallback. Windows DPAPI support remains deferred.
|
||||
|
||||
**Effort:** L
|
||||
**Remaining:** Windows cookie decryption (DPAPI). Needs complete rewrite — PR #64 was 1346 lines and stale.
|
||||
|
||||
**Effort:** L (Windows only)
|
||||
**Priority:** P4
|
||||
**Completed (Linux):** v0.11.11.0 (2026-03-23)
|
||||
|
||||
## Ship
|
||||
|
||||
### GitLab support for /land-and-deploy
|
||||
|
||||
**What:** Add GitLab MR merge + CI polling support to `/land-and-deploy` skill. Currently uses `gh pr view`, `gh pr checks`, `gh pr merge`, and `gh run list/view` in 15+ places — each needs a GitLab conditional path using `glab ci status`, `glab mr merge`, etc.
|
||||
|
||||
**Why:** Without this, GitLab users can `/ship` (create MR) but can't `/land-and-deploy` (merge + verify). Completes the GitLab story end-to-end.
|
||||
|
||||
**Context:** `/retro`, `/ship`, and `/document-release` now support GitLab via the multi-platform `BASE_BRANCH_DETECT` resolver. `/land-and-deploy` has deeper GitHub-specific semantics (merge queues, required checks via `gh pr checks`, deploy workflow polling) that have different shapes on GitLab. The `glab` CLI (v1.90.0) supports `glab mr merge`, `glab ci status`, `glab ci view` but with different output formats and no merge queue concept.
|
||||
|
||||
**Effort:** L
|
||||
**Priority:** P2
|
||||
**Depends on:** None (BASE_BRANCH_DETECT multi-platform resolver is already done)
|
||||
|
||||
### Multi-commit CHANGELOG completeness eval
|
||||
|
||||
**What:** Add a periodic E2E eval that creates a branch with 5+ commits spanning 3+ themes (features, cleanup, infra), runs /ship's Step 5 CHANGELOG generation, and verifies the CHANGELOG mentions all themes.
|
||||
|
||||
**Why:** The bug fixed in v0.11.22 (garrytan/ship-full-commit-coverage) showed that /ship's CHANGELOG generation biased toward recent commits on long branches. The prompt fix adds a cross-check, but no test exercises the multi-commit failure mode. The existing `ship-local-workflow` E2E only uses a single-commit branch.
|
||||
|
||||
**Context:** Would be a `periodic` tier test (~$4/run, non-deterministic since it tests LLM instruction-following). Setup: create bare remote, clone, add 5+ commits across different themes on a feature branch, run Step 5 via `claude -p`, verify CHANGELOG output covers all themes. Pattern: `ship-local-workflow` in `test/skill-e2e-workflow.test.ts`.
|
||||
|
||||
**Effort:** M
|
||||
**Priority:** P3
|
||||
**Depends on:** None
|
||||
|
||||
### Ship log — persistent record of /ship runs
|
||||
|
||||
**What:** Append structured JSON entry to `.gstack/ship-log.json` at end of every /ship run (version, date, branch, PR URL, review findings, Greptile stats, todos completed, test results).
|
||||
@@ -449,17 +543,18 @@ No S3 needed.
|
||||
**Depends on:** Video recording
|
||||
|
||||
|
||||
### GitHub Actions eval upload
|
||||
|
||||
**What:** Run eval suite in CI, upload result JSON as artifact, post summary comment on PR.
|
||||
### Extend worktree isolation to Claude E2E tests
|
||||
|
||||
**Why:** CI integration catches quality regressions before merge and provides persistent eval records per PR.
|
||||
**What:** Add `useWorktree?: boolean` option to `runSkillTest()` so any Claude E2E test can opt into worktree mode for full repo context instead of tmpdir fixtures.
|
||||
|
||||
**Context:** Requires `ANTHROPIC_API_KEY` in CI secrets. Cost is ~$4/run. Eval persistence system (v0.3.6) writes JSON to `~/.gstack-dev/evals/` — CI would upload as GitHub Actions artifacts and use `eval:compare` to post delta comment.
|
||||
**Why:** Some Claude E2E tests (CSO audit, review-sql-injection) create minimal fake repos but would produce more realistic results with full repo context. The infrastructure exists (`describeWithWorktree()` in e2e-helpers.ts) — this extends it to the session-runner level.
|
||||
|
||||
**Effort:** M
|
||||
**Priority:** P2
|
||||
**Depends on:** Eval persistence (shipped in v0.3.6)
|
||||
**Context:** WorktreeManager shipped in v0.11.12.0. Currently only Gemini/Codex tests use worktrees. Claude tests use planted-bug fixture repos which are correct for their purpose, but new tests that want real repo context can use `describeWithWorktree()` today. This TODO is about making it even easier via a flag on `runSkillTest()`.
|
||||
|
||||
**Effort:** M (human: ~2 days / CC: ~20 min)
|
||||
**Priority:** P3
|
||||
**Depends on:** Worktree isolation (shipped v0.11.12.0)
|
||||
|
||||
### E2E model pinning — SHIPPED
|
||||
|
||||
@@ -543,6 +638,30 @@ Shipped: Default model changed to Sonnet for structure tests (~30), Opus retaine
|
||||
|
||||
Shipped as v0.5.0 on main. Includes `/plan-design-review` (report-only design audit), `/qa-design-review` (audit + fix loop), and `/design-consultation` (interactive DESIGN.md creation). `{{DESIGN_METHODOLOGY}}` resolver provides shared 80-item design audit checklist.
|
||||
|
||||
### Design outside voices in /plan-eng-review
|
||||
|
||||
**What:** Extend the parallel dual-voice pattern (Codex + Claude subagent) to /plan-eng-review's architecture review section.
|
||||
|
||||
**Why:** The design beachhead (v0.11.3.0) proves cross-model consensus works for subjective reviews. Architecture reviews have similar subjectivity in tradeoff decisions.
|
||||
|
||||
**Context:** Depends on learnings from the design beachhead. If the litmus scorecard format proves useful, adapt it for architecture dimensions (coupling, scaling, reversibility).
|
||||
|
||||
**Effort:** S
|
||||
**Priority:** P3
|
||||
**Depends on:** Design outside voices shipped (v0.11.3.0)
|
||||
|
||||
### Outside voices in /qa visual regression detection
|
||||
|
||||
**What:** Add Codex design voice to /qa for detecting visual regressions during bug-fix verification.
|
||||
|
||||
**Why:** When fixing bugs, the fix can introduce visual regressions that code-level checks miss. Codex could flag "the fix broke the responsive layout" during re-test.
|
||||
|
||||
**Context:** Depends on /qa having design awareness. Currently /qa focuses on functional testing.
|
||||
|
||||
**Effort:** M
|
||||
**Priority:** P3
|
||||
**Depends on:** Design outside voices shipped (v0.11.3.0)
|
||||
|
||||
## Document-Release
|
||||
|
||||
### Auto-invoke /document-release from /ship — SHIPPED
|
||||
@@ -576,6 +695,20 @@ Shipped in v0.8.3. Step 8.5 added to `/ship` — after creating the PR, `/ship`
|
||||
**Depends on:** gstack-diff-scope (shipped)
|
||||
|
||||
|
||||
## Codex
|
||||
|
||||
### Codex→Claude reverse buddy check skill
|
||||
|
||||
**What:** A Codex-native skill (`.agents/skills/gstack-claude/SKILL.md`) that runs `claude -p` to get an independent second opinion from Claude — the reverse of what `/codex` does today from Claude Code.
|
||||
|
||||
**Why:** Codex users deserve the same cross-model challenge that Claude users get via `/codex`. Currently the flow is one-way (Claude→Codex). Codex users have no way to get a Claude second opinion.
|
||||
|
||||
**Context:** The `/codex` skill template (`codex/SKILL.md.tmpl`) shows the pattern — it wraps `codex exec` with JSONL parsing, timeout handling, and structured output. The reverse skill would wrap `claude -p` with similar infrastructure. Would be generated into `.agents/skills/gstack-claude/` by `gen-skill-docs --host codex`.
|
||||
|
||||
**Effort:** M (human: ~2 weeks / CC: ~30 min)
|
||||
**Priority:** P1
|
||||
**Depends on:** None
|
||||
|
||||
## Completeness
|
||||
|
||||
### Completeness metrics dashboard
|
||||
@@ -624,8 +757,50 @@ Shipped in v0.6.5. TemplateContext in gen-skill-docs.ts bakes skill name into pr
|
||||
**Priority:** P3
|
||||
**Depends on:** Telemetry data showing freeze hook fires in real /investigate sessions
|
||||
|
||||
## Factory Droid
|
||||
|
||||
### Browse MCP server for Factory Droid
|
||||
|
||||
**What:** Expose gstack's browse binary and key workflows as an MCP server that Factory Droid connects to natively. Factory users would run /mcp, add the gstack server, and get browse, QA, and review capabilities as Factory tools.
|
||||
|
||||
**Why:** Factory already supports 40+ MCP servers in its registry. Getting gstack's browse binary listed there is a distribution play. Nobody else has a real compiled browser binary as an MCP tool. This is the thing that makes gstack uniquely valuable on Factory Droid.
|
||||
|
||||
**Context:** Option A (--host factory compatibility shim) ships first in v0.13.4.0. Option B is the follow-up that provides deeper integration. The browse binary is already a stateless CLI, so wrapping it as an MCP server is straightforward (stdin/stdout JSON-RPC). Each browse command becomes an MCP tool.
|
||||
|
||||
**Effort:** L (human: ~1 week / CC: ~5 hours)
|
||||
**Priority:** P1
|
||||
**Depends on:** --host factory (Option A, shipping in v0.13.4.0)
|
||||
|
||||
### .agent/skills/ dual output for cross-agent compatibility
|
||||
|
||||
**What:** Factory also reads from `<repo>/.agent/skills/` as a cross-agent compatibility path. Could output there in addition to `.factory/skills/` for broader reach across other agents that use the `.agent` convention.
|
||||
|
||||
**Why:** Multiple AI agents beyond Factory may adopt the `.agent/skills/` convention. Outputting there too would give free compatibility.
|
||||
|
||||
**Effort:** S
|
||||
**Priority:** P3
|
||||
**Depends on:** --host factory
|
||||
|
||||
### Custom Droid definitions alongside skills
|
||||
|
||||
**What:** Factory has "custom droids" (subagents with tool restrictions, model selection, autonomy levels). Could ship `gstack-qa.md` droid configs alongside skills that restrict tools to read-only + execute for safety.
|
||||
|
||||
**Why:** Deeper Factory integration. Droid configs give Factory users tighter control over what gstack skills can do.
|
||||
|
||||
**Effort:** M
|
||||
**Priority:** P3
|
||||
**Depends on:** --host factory
|
||||
|
||||
## Completed
|
||||
|
||||
### CI eval pipeline (v0.9.9.0)
|
||||
- GitHub Actions eval upload on Ubicloud runners ($0.006/run)
|
||||
- Within-file test concurrency (test() → testConcurrentIfSelected())
|
||||
- Eval artifact upload + PR comment with pass/fail + cost
|
||||
- Baseline comparison via artifact download from main
|
||||
- EVALS_CONCURRENCY=40 for ~6min wall clock (was ~18min)
|
||||
**Completed:** v0.9.9.0
|
||||
|
||||
### Deploy pipeline (v0.9.8.0)
|
||||
- /land-and-deploy — merge PR, wait for CI/deploy, canary verification
|
||||
- /canary — post-deploy monitoring loop with anomaly detection
|
||||
|
||||
Reference in New Issue
Block a user