mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-02 03:35:09 +02:00
7665adf4fe
* feat: CDP connect — control real Chrome/Comet via Playwright Add `connectCDP()` to BrowserManager: connects to a running browser via Chrome DevTools Protocol. All existing browse commands work unchanged through Playwright's abstraction layer. - chrome-launcher.ts: browser discovery, CDP probe, auto-relaunch with rollback - browser-manager.ts: connectCDP(), mode guards (close/closeTab/recreateContext/handoff), auto-reconnect on browser restart, getRefMap() for extension API - server.ts: CDP branch in start(), /health gains mode field, /refs endpoint, idle timer only resets on /command (not passive endpoints) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: browse connect/disconnect/focus CLI commands - connect: pre-server command that discovers browser, starts server in CDP mode - disconnect: drops CDP connection, restarts in headless mode - focus: brings browser window to foreground via osascript (macOS) - status: now shows Mode: cdp | launched | headed - startServer() accepts extra env vars for CDP URL/port passthrough Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: CDP-aware skill templates — skip cookie import in real browser mode Skills now check `$B status` for CDP mode and skip: - /qa: cookie import prompt, user-agent override, headless workarounds - /design-review: cookie import for authenticated pages - /setup-browser-cookies: returns "not needed" in CDP mode Regenerated SKILL.md files from updated templates. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: activity streaming — SSE endpoint for Chrome extension Side Panel Real-time browse command feed via Server-Sent Events: - activity.ts: ActivityEntry type, CircularBuffer (capacity 1000), privacy filtering (redacts passwords, auth tokens, sensitive URL params), cursor-based gap detection, async subscriber notification - server.ts: /activity/stream SSE, /activity/history REST, handleCommand instrumented with command_start/command_end events - 18 unit tests for filterArgs privacy, emitActivity, subscribe lifecycle Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Chrome extension Side Panel + Conductor API proposal Chrome extension (Manifest V3, sideload): - Side Panel with live activity feed, @ref overlays, dark terminal aesthetic - Background worker: health polling, SSE relay, ref fetching - Popup: port config, connection status, side panel launcher - Content script: floating ref panel with @ref badges Conductor API proposal (docs/designs/CONDUCTOR_SESSION_API.md): - SSE endpoint for full Claude Code session mirroring in Side Panel - Discovery via HTTP endpoint (not filesystem — extensions can't read files) TODOS.md: add $B watch, multi-agent tabs, cross-platform CDP, Web Store publishing. Mark CDP mode as shipped. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: detect Conductor runtime, skip osascript quit for sandboxed apps macOS App Management blocks Electron apps (Conductor) from quitting other apps via osascript. Now detects the runtime environment: - terminal/claude-code/codex: can manage apps freely - conductor: prints manual restart instructions + polls for 60s detectRuntime() checks env vars and parent process. When Chrome needs restart but we can't quit it, prints step-by-step instructions and waits for the user to restart Chrome with --remote-debugging-port. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: detect Conductor via actual env vars (CONDUCTOR_WORKSPACE_NAME) Previous detection checked CONDUCTOR_WORKSPACE_ID which doesn't exist. Conductor sets CONDUCTOR_WORKSPACE_NAME, CONDUCTOR_BIN_DIR, CONDUCTOR_PORT, and __CFBundleIdentifier=com.conductor.app. Check these FIRST because Conductor sessions also have ANTHROPIC_API_KEY (which was matching claude-code). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: connection status pill — floating indicator when gstack controls Chrome Small pill in bottom-right corner of every page: "● gstack · 3 refs" Shows when connected via CDP, fades to 30% opacity after 3s, full on hover. Disappears entirely when disconnected. Background worker now notifies content scripts on connect/disconnect state changes so the pill appears/disappears without polling. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Chrome requires --user-data-dir for remote debugging Chrome refuses --remote-debugging-port without an explicit --user-data-dir. Add userDataDir to BrowserBinary registry (macOS Application Support paths) and pass it in both auto-launch and manual restart instructions. Fix double-quoting in CLI manual restart instructions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Chrome must be fully quit before launching with --remote-debugging-port Chrome refuses to enable CDP on its default profile when another instance is running (even with explicit --user-data-dir). The only reliable path: fully quit Chrome first, then relaunch with the flag. Updated instructions to emphasize this clearly with verification step. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: bin/chrome-cdp — quit Chrome and relaunch with CDP in one command Quits Chrome gracefully, waits for full exit, relaunches with --remote-debugging-port, polls until CDP is ready. Usage: chrome-cdp [port] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use Playwright channel:chrome instead of broken connectOverCDP Playwright's connectOverCDP hangs with Chrome 146 due to CDP protocol version mismatch. Switch to channel:'chrome' which uses Playwright's native pipe protocol to launch the system Chrome binary directly. This is simpler and more reliable: - No CDP port discovery needed - No --remote-debugging-port or --user-data-dir hassles - $B connect just works — launches real Chrome headed window - All Playwright APIs (snapshot, click, fill) work unchanged bin/chrome-cdp updated with symlinked profile approach (kept for manual CDP use cases, but $B connect no longer needs it). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: green border + gstack label on controlled Chrome window Injects a 2px green border and small "gstack" label on every page loaded in the controlled Chrome window via context.addInitScript(). Users can instantly tell which Chrome window Claude controls. Also fixes close() for channel:chrome mode (uses browser.close() not browser.disconnect() which doesn't exist). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: cleanup chrome-launcher runtime detection, remove puppeteer-core dep Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(design): redesign controlled Chrome indicator Replace crude green border + label with polished indicator: - 2px shimmer gradient at top edge (green→cyan→green, 3s loop) - Floating pill bottom-right with frosted glass bg, fades to 25% opacity after 4s so it doesn't compete with page content - prefers-reduced-motion disables shimmer animation - Much more subtle — looks like a developer tool, not broken CSS Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: document real browser mode + Chrome extension in BROWSER.md and README.md BROWSER.md: new sections for connect/disconnect/focus commands, Chrome extension Side Panel install, CDP-aware skills, activity streaming. Updated command reference table, key components, env vars, source map. README.md: updated /browse description, added "Real browser mode" to What's New section. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: step-by-step Chrome extension install guide in BROWSER.md Replace terse bullet points with numbered walkthrough covering: developer mode toggle, load unpacked, macOS file picker tip (Cmd+Shift+G), pin extension, configure port, open side panel. Added troubleshooting section. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Cmd+Shift+. tip for hidden folders in macOS file picker macOS hides folders starting with . by default. Added both shortcuts: Cmd+Shift+G (paste path directly) and Cmd+Shift+. (show hidden files). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: integrate hidden folder tips into the install flow naturally Move Cmd+Shift+G and Cmd+Shift+. tips inline with the file picker step instead of as a separate tip block after it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: auto-load Chrome extension when $B connect launches Chrome Extension auto-loads via --load-extension flag — no manual chrome://extensions install needed. findExtensionPath() checks repo root, global install, and dev paths. Also adds bin/gstack-extension helper for manual install in regular Chrome, and rewrites BROWSER.md install docs with auto-load as primary path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: /connect-chrome skill — one command to launch Chrome with Side Panel New skill that runs $B connect, verifies the connection, guides the user to open the Side Panel, and demos the live activity feed. Extension auto-loads via --load-extension so no manual chrome://extensions install needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use launchPersistentContext for Chrome extension loading Playwright's chromium.launch() silently ignores --load-extension. Switch to launchPersistentContext with ignoreDefaultArgs to remove --disable-extensions flag. Use bundled Chromium (real Chrome blocks unpacked extensions). Fixed port 34567 for CDP mode so the extension auto-connects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: sync extension to DESIGN.md — amber accent, zinc neutrals, grain texture Import design system from gstack-website. Update all extension colors: green (#4ade80) → amber (#F59E0B/#FBBF24), zinc gray neutrals, grain texture overlay. Regenerate icons as amber "G" monogram on dark background. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: sidebar chat with Claude Code — icon opens side panel directly Replace popup flyout with direct side panel open on icon click. Primary UI is now a chat interface that sends messages to Claude Code via file queue. Activity/Refs tabs moved behind a debug toggle in the footer. Command bar with history, auto-poll for responses, amber design system. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: sidebar agent — Claude-powered chat backend via file queue Add /sidebar-command, /sidebar-response, and /sidebar-chat endpoints to the browse server. sidebar-agent.ts watches the command queue file, spawns claude -p with browse context for each message, and streams responses back to the sidebar chat. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove duplicate gstack pill overlay, hide crash restore bubble The addInitScript indicator and the extension's content script were both injecting bottom-right pills, causing duplicates. Remove the pill from addInitScript (extension handles it). Replace --restore-last-session with --hide-crash-restore-bubble to suppress the "Chromium didn't shut down correctly" dialog. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: state file authority — CDP server cannot be silently replaced Hardens the connect/disconnect lifecycle: - ensureServer() refuses to auto-start headless when CDP server is alive - $B connect does full cleanup: SIGTERM → 2s → SIGKILL, profile locks, state - shutdown() cleans Chromium SingletonLock/Socket/Cookie files - uncaughtException/unhandledRejection handlers do emergency cleanup This prevents the bug where a headless server overwrites the CDP server's state file, causing $B commands to hit the wrong browser. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: sidebar agent streaming events + session state management Enhance sidebar-agent.ts with: - Live streaming of claude -p events (tool_use, text, result) to sidebar - Session state file for BROWSE_STATE_FILE propagation to claude subprocess - Improved logging (stderr, exit codes, event types) - stdin.end() to prevent claude waiting for input - summarizeToolInput() with path shortening for compact sidebar display Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: sidebar chat UI — streaming events, agent status, reconnect retry Sidebar panel improvements: - Chat tab renders streaming agent events (tool_use, text, result) - Thinking dots animation while agent processes - Agent error display with styled error blocks - tryConnect() with 2s retry loop for initial connection - Debug tabs (Activity/Refs) hidden behind gear toggle - Clear chat button - Compact tool call display with path shortening Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: server-integrated sidebar agent with sessions and message queue Move the sidebar agent from a separate bun process into server.ts: - Agent spawns claude -p directly when messages arrive via /sidebar-command - In-memory chat buffer backed by per-session chat.jsonl on disk - Session manager: create, load, persist, list sessions - Message queue (cap 5) with agent status tracking (idle/processing/hung) - Stop/kill endpoints with queue dismiss support - /health now returns agent status + session info - All sidebar endpoints require Bearer auth - Agent killed on server shutdown - 120s timeout detects hung claude processes Eliminates: file-queue polling, separate sidebar-agent.ts process, stale auth tokens, state file conflicts between processes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: extension auth + token flow for server-integrated agent Update Chrome extension to use Bearer auth on all sidebar endpoints: - background.js captures auth token from /health, exposes via getToken msg - background.js sets openPanelOnActionClick for direct side panel access - sidepanel.js gets token from background, sends in all fetch headers - Health broadcasts include token so sidebar auto-authenticates - Removes popup from manifest — icon click opens side panel directly Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: self-healing sidebar — reconnect banner, state machine, copy button Sidebar UI now handles disconnection gracefully: - Connection state machine: connected → reconnecting → dead - Amber pulsing banner during reconnect (2s retry, 30 attempts) - Red "Server offline" banner with Reconnect + Copy /connect-chrome buttons - Green "Reconnected" toast that fades after 3s on successful reconnect - Copy button lets user paste /connect-chrome into any Claude Code session Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: crash handling — save session, kill agent, distinct exit codes Hardened shutdown/crash behavior: - Browser disconnect exits with code 2 (distinct from crash code 1) - emergencyCleanup kills agent subprocess and saves session state - Clean shutdown saves session before exit (chat history persists) - Clear user message on browser disconnect: "Run $B connect to reconnect" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: worktree-per-session isolation for sidebar agent Each sidebar session gets an isolated git worktree so the agent's file operations don't conflict with the user's working directory: - createWorktree() creates detached HEAD worktree in ~/.gstack/worktrees/ - Falls back to main cwd for non-git repos or on creation failure - Handles collision cleanup from prior crashes - removeWorktree() cleans up on session switch and shutdown - worktreePath persisted in session.json Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(qa): ISSUE-001 — disconnect blocked by CDP guard in ensureServer $B disconnect was routed through ensureServer() which refused to start a headless server when a CDP state file existed. Disconnect is now handled before ensureServer() (like connect), with force-kill + cleanup fallback when the CDP server is unresponsive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve claude binary path for daemon-spawned agent The browse server runs as a daemon and may not inherit the user's shell PATH. Add findClaudeBin() that checks ~/.local/bin/claude (standard install location), which claude, and common system paths. Shows a clear error in the sidebar chat if claude CLI is not found. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve claude symlinks + check Conductor bundled binary posix_spawn fails on symlinks in compiled bun binaries. Now: - Checks Conductor app's bundled binary first (not a symlink) - Scans ~/.local/share/claude/versions/ for direct versioned binaries - Uses fs.realpathSync() to resolve symlinks before spawning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: compiled bun binary cannot posix_spawn — use external agent process Compiled bun binaries fail posix_spawn on ALL executables (even /bin/bash). The server now writes to an agent queue file, and a separate non-compiled bun process (sidebar-agent.ts) reads the queue, spawns claude, and POSTs events back via /sidebar-agent/event. Changes: - server.ts: spawnClaude writes to queue file instead of spawning directly - server.ts: new /sidebar-agent/event endpoint for agent → server relay - server.ts: fix result event field name (event.text vs event.result) - sidebar-agent.ts: rewritten to poll queue file, relay events via HTTP - cli.ts: $B connect auto-starts sidebar-agent as non-compiled bun process Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: loading spinner on sidebar open while connecting to server Shows an amber spinner with "Connecting..." when the sidebar first opens, replacing the empty state. After the first successful /sidebar-chat poll: - If chat history exists: renders it immediately - If no history: shows the welcome message Prevents the jarring empty-then-populated flash on sidebar open. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: zero-friction side panel — auto-open on install, pill is clickable Three changes to eliminate manual side panel setup: - Auto-open side panel on extension install/update (onInstalled listener) - gstack pill (bottom-right) is now clickable — opens the side panel - Pill has pointer-events: auto so clicks always register (was: none) User no longer needs to find the puzzle piece icon, pin the extension, or know the side panel exists. It opens automatically on first launch and can be re-opened by clicking the floating gstack pill. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: kill CDP naming, delete chrome-launcher.ts dead code The connectCDP() method and connectionMode: 'cdp' naming was a legacy artifact — real Chrome was tried but failed (silently blocks --load-extension), so the implementation already used Playwright's bundled Chromium via launchPersistentContext(). The naming was misleading. Changes: - Delete chrome-launcher.ts (361 LOC) — only import was in unreachable attemptReconnect() method - Delete dead attemptReconnect() and reconnecting field - Delete preExistingTabIds (was for protecting real Chrome tabs we never connect to) - Rename connectCDP() → launchHeaded() - Rename connectionMode: 'cdp' → 'headed' across all files - Replace BROWSE_CDP_URL/BROWSE_CDP_PORT env vars with BROWSE_HEADED=1 - Regenerate SKILL.md files for updated command descriptions - Move BrowserManager unit tests to browser-manager-unit.test.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: converge handoff into connect — extension loads on handoff Handoff now uses launchPersistentContext() with extension auto-loading, same as the connect/launchHeaded() path. This means when the agent gets stuck (2FA, CAPTCHA) and hands off to the user, the Chrome extension + side panel are available automatically. Before: handoff used chromium.launch() + newContext() — no extension After: handoff uses chromium.launchPersistentContext() — extension loads Also sets connectionMode to 'headed' and disables dialog auto-accept on handoff, matching connect behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: gate sidebar chat behind --chat flag $B connect (default): headed Chromium + extension with Activity + Refs tabs only. No separate agent spawned. Clean, no confusion. $B connect --chat: same + Chat tab with standalone claude -p agent. Shows experimental banner: "Standalone mode — this is a separate agent from your workspace." Implementation: - cli.ts: parse --chat, set BROWSE_SIDEBAR_CHAT env, conditionally spawn sidebar-agent - server.ts: gate /sidebar-* routes behind chatEnabled, return 403 when disabled, include chatEnabled in /health response - sidepanel.js: applyChatEnabled() hides/shows Chat tab + banner - background.js: forward chatEnabled from health response - sidepanel.html/css: experimental banner with amber styling Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: file drop relay + $B inbox command Sidebar agent now writes structured messages to .context/sidebar-inbox/ when processing user input. The workspace agent can read these via $B inbox to see what the user reported from the browser. File drop format: .context/sidebar-inbox/{timestamp}-observation.json { type, timestamp, page: {url}, userMessage, sidebarSessionId } Atomic writes (tmp + rename) prevent partial reads. $B inbox --clear removes messages after display. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: $B watch — passive observation mode Claude enters read-only mode and captures periodic snapshots (every 5s) while the user browses. Mutation commands (click, fill, etc.) are blocked during watch. $B watch stop exits and returns a summary with the last snapshot. Requires headed mode ($B connect). This is the inverse of the scout pattern — the workspace agent watches through the browser instead of the sidebar relaying to it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add coverage for sidebar-agent, file-drop, and watch mode 33 new tests covering: - Sidebar agent queue parsing (valid/malformed/empty JSONL) - writeToInbox file drop (directory creation, atomic writes, JSON format) - Inbox command (display, sorting, --clear, malformed file handling) - Watch mode state machine (start/stop cycles, snapshots, duration) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: TODOS cleanup + Chrome vs Chromium exploration doc - Update TODOS.md: mark CDP mode, $B watch, sidebar scout as SHIPPED - Delete dead "cross-platform CDP browser discovery" TODO - Rename dependencies from "CDP connect" to "headed mode" - Add docs/designs/CHROME_VS_CHROMIUM_EXPLORATION.md memorializing the architecture exploration and decision to use Playwright Chromium Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Conductor Chrome sidebar integration design doc Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: sidebar-agent validates cwd before spawning claude The queue entry may reference a worktree that was cleaned up between sessions. Now falls back to process.cwd() if the path doesn't exist, preventing silent spawn failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gen-skill-docs resolver merge + preamble tier gate + plan file discovery The local RESOLVERS record in gen-skill-docs.ts was shadowing the imported canonical resolvers, causing stale test coverage and preamble generators to be used instead of the authoritative versions in resolvers/. Changes: - Merge imported RESOLVERS with local overrides (spread + override pattern) - Fix preamble tier gate: tier 1 skills no longer get AskUserQuestion format - Make plan file discovery host-agnostic (search multiple plan dirs) - Add missing E2E tier entries for ship/review plan completion tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: ungate sidebar agent + raise timeout to 5 minutes (v0.12.0) Sidebar chat is now always available in headed mode — no --chat flag needed. Agent tasks get 5 minutes instead of 2, enabling multi-page workflows like navigating directories and filling forms across pages. Changes: - cli.ts: remove --chat flag, always set BROWSE_SIDEBAR_CHAT=1, always spawn agent - server.ts: remove chatEnabled gate (403 response), raise AGENT_TIMEOUT_MS to 300s - sidebar-agent.ts: raise child process timeout from 120s to 300s Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: headed mode + sidebar agent documentation (v0.12.0) - README: sidebar agent section, personal automation example (school parent portal), two auth paths (manual login + cookie import), DevTools MCP mention - BROWSER.md: sidebar agent section with usage, timeout, session isolation, authentication, and random delay documentation - connect-chrome template: add sidebar chat onboarding step - CHANGELOG: v0.12.0 entry covering headed mode, sidebar agent, extension - VERSION: bump to 0.12.0.0 - TODOS: Chrome DevTools MCP integration as P0 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate SKILL.md files Generated from updated templates + resolver merge. Key changes: - Tier 1 skills no longer include AskUserQuestion format section - Ship/review skills now include coverage gate with thresholds - Connect-chrome skill includes sidebar chat onboarding step - Plan file discovery uses host-agnostic paths Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate Codex connect-chrome skill Updated preamble with proactive prompt and sidebar chat onboarding step. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: network idle, state persistence, iframe support, chain pipe format (v0.12.1.0) (#516) * feat: network idle detection + chain pipe format - Upgrade click/fill/select from domcontentloaded to networkidle wait (2s timeout, best-effort). Catches XHR/fetch triggered by interactions. - Add pipe-delimited format to chain as JSON fallback: $B chain 'goto url | click @e5 | snapshot -ic' - Add post-loop networkidle wait in chain when last command was a write. - Frame-aware: commands use target (getActiveFrameOrPage) for locator ops, page-only ops (goto/back/forward/reload) guard against frame context. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: $B state save/load + $B frame — new browse commands - state save/load: persist cookies + URLs to .gstack/browse-states/{name}.json File perms 0o600, name sanitized to [a-zA-Z0-9_-]. V1 skips localStorage (breaks on load-before-navigate). Load replaces session via closeAllPages(). - frame: switch command context to iframe via CSS selector, @ref, --name, or --url. 'frame main' returns to main frame. Execution target abstraction (getActiveFrameOrPage) across read-commands, snapshot, and write-commands. - Frame context cleared on tab switch, navigation, resume, and handoff. - Snapshot shows [Context: iframe src="..."] header when in frame. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add tests for network idle, chain pipe format, state, and frame - Network idle: click on fetch button waits for XHR, static click is fast - Chain pipe: pipe-delimited commands, quoted args, JSON still works - State: save/load round-trip, name sanitization, missing state error - Frame: switch to iframe + back, snapshot context header, fill in frame, goto-in-frame guard, usage error New fixtures: network-idle.html (fetch + static buttons), iframe.html (srcdoc) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review fixes — iframe ref scoping, detached frame recovery, state validation - snapshot.ts: ref locators, cursor-interactive scan, and cursor locator now use target (frame-aware) instead of page — fixes @ref clicking in iframes - browser-manager.ts: getActiveFrameOrPage auto-recovers from detached frames via isDetached() check - meta-commands.ts: state load resets activeFrame, elementHandle disposed after contentFrame(), state file schema validation (cookies + pages arrays), filter empty pipe segments in chain tokenizer - write-commands.ts: upload command uses target.locator() for frame support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate SKILL.md files + rebuild binary Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.12.1.0) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
935 lines
32 KiB
TypeScript
935 lines
32 KiB
TypeScript
/**
|
|
* Browser lifecycle manager
|
|
*
|
|
* Chromium crash handling:
|
|
* browser.on('disconnected') → log error → process.exit(1)
|
|
* CLI detects dead server → auto-restarts on next command
|
|
* We do NOT try to self-heal — don't hide failure.
|
|
*
|
|
* Dialog handling:
|
|
* page.on('dialog') → auto-accept by default → store in dialog buffer
|
|
* Prevents browser lockup from alert/confirm/prompt
|
|
*
|
|
* Context recreation (useragent):
|
|
* recreateContext() saves cookies/storage/URLs, creates new context,
|
|
* restores state. Falls back to clean slate on any failure.
|
|
*/
|
|
|
|
import { chromium, type Browser, type BrowserContext, type BrowserContextOptions, type Page, type Locator, type Cookie } from 'playwright';
|
|
import { addConsoleEntry, addNetworkEntry, addDialogEntry, networkBuffer, type DialogEntry } from './buffers';
|
|
import { validateNavigationUrl } from './url-validation';
|
|
|
|
export interface RefEntry {
|
|
locator: Locator;
|
|
role: string;
|
|
name: string;
|
|
}
|
|
|
|
export interface BrowserState {
|
|
cookies: Cookie[];
|
|
pages: Array<{
|
|
url: string;
|
|
isActive: boolean;
|
|
storage: { localStorage: Record<string, string>; sessionStorage: Record<string, string> } | null;
|
|
}>;
|
|
}
|
|
|
|
export class BrowserManager {
|
|
private browser: Browser | null = null;
|
|
private context: BrowserContext | null = null;
|
|
private pages: Map<number, Page> = new Map();
|
|
private activeTabId: number = 0;
|
|
private nextTabId: number = 1;
|
|
private extraHeaders: Record<string, string> = {};
|
|
private customUserAgent: string | null = null;
|
|
|
|
/** Server port — set after server starts, used by cookie-import-browser command */
|
|
public serverPort: number = 0;
|
|
|
|
// ─── Ref Map (snapshot → @e1, @e2, @c1, @c2, ...) ────────
|
|
private refMap: Map<string, RefEntry> = new Map();
|
|
|
|
// ─── Snapshot Diffing ─────────────────────────────────────
|
|
// NOT cleared on navigation — it's a text baseline for diffing
|
|
private lastSnapshot: string | null = null;
|
|
|
|
// ─── Dialog Handling ──────────────────────────────────────
|
|
private dialogAutoAccept: boolean = true;
|
|
private dialogPromptText: string | null = null;
|
|
|
|
// ─── Handoff State ─────────────────────────────────────────
|
|
private isHeaded: boolean = false;
|
|
private consecutiveFailures: number = 0;
|
|
|
|
// ─── Watch Mode ─────────────────────────────────────────
|
|
private watching = false;
|
|
public watchInterval: ReturnType<typeof setInterval> | null = null;
|
|
private watchSnapshots: string[] = [];
|
|
private watchStartTime: number = 0;
|
|
|
|
// ─── Headed State ────────────────────────────────────────
|
|
private connectionMode: 'launched' | 'headed' = 'launched';
|
|
private intentionalDisconnect = false;
|
|
|
|
getConnectionMode(): 'launched' | 'headed' { return this.connectionMode; }
|
|
|
|
// ─── Watch Mode Methods ─────────────────────────────────
|
|
isWatching(): boolean { return this.watching; }
|
|
|
|
startWatch(): void {
|
|
this.watching = true;
|
|
this.watchSnapshots = [];
|
|
this.watchStartTime = Date.now();
|
|
}
|
|
|
|
stopWatch(): { snapshots: string[]; duration: number } {
|
|
this.watching = false;
|
|
if (this.watchInterval) {
|
|
clearInterval(this.watchInterval);
|
|
this.watchInterval = null;
|
|
}
|
|
const snapshots = this.watchSnapshots;
|
|
const duration = Date.now() - this.watchStartTime;
|
|
this.watchSnapshots = [];
|
|
this.watchStartTime = 0;
|
|
return { snapshots, duration };
|
|
}
|
|
|
|
addWatchSnapshot(snapshot: string): void {
|
|
this.watchSnapshots.push(snapshot);
|
|
}
|
|
|
|
/**
|
|
* Find the gstack Chrome extension directory.
|
|
* Checks: repo root /extension, global install, dev install.
|
|
*/
|
|
private findExtensionPath(): string | null {
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const candidates = [
|
|
// Relative to this source file (dev mode: browse/src/ -> ../../extension)
|
|
path.resolve(__dirname, '..', '..', 'extension'),
|
|
// Global gstack install
|
|
path.join(process.env.HOME || '', '.claude', 'skills', 'gstack', 'extension'),
|
|
// Git repo root (detected via BROWSE_STATE_FILE location)
|
|
(() => {
|
|
const stateFile = process.env.BROWSE_STATE_FILE || '';
|
|
if (stateFile) {
|
|
const repoRoot = path.resolve(path.dirname(stateFile), '..');
|
|
return path.join(repoRoot, '.claude', 'skills', 'gstack', 'extension');
|
|
}
|
|
return '';
|
|
})(),
|
|
].filter(Boolean);
|
|
|
|
for (const candidate of candidates) {
|
|
try {
|
|
if (fs.existsSync(path.join(candidate, 'manifest.json'))) {
|
|
return candidate;
|
|
}
|
|
} catch {}
|
|
}
|
|
return null;
|
|
}
|
|
|
|
/**
|
|
* Get the ref map for external consumers (e.g., /refs endpoint).
|
|
*/
|
|
getRefMap(): Array<{ ref: string; role: string; name: string }> {
|
|
const refs: Array<{ ref: string; role: string; name: string }> = [];
|
|
for (const [ref, entry] of this.refMap) {
|
|
refs.push({ ref, role: entry.role, name: entry.name });
|
|
}
|
|
return refs;
|
|
}
|
|
|
|
async launch() {
|
|
// ─── Extension Support ────────────────────────────────────
|
|
// BROWSE_EXTENSIONS_DIR points to an unpacked Chrome extension directory.
|
|
// Extensions only work in headed mode, so we use an off-screen window.
|
|
const extensionsDir = process.env.BROWSE_EXTENSIONS_DIR;
|
|
const launchArgs: string[] = [];
|
|
let useHeadless = true;
|
|
|
|
// Docker/CI: Chromium sandbox requires unprivileged user namespaces which
|
|
// are typically disabled in containers. Detect container environment and
|
|
// add --no-sandbox automatically.
|
|
if (process.env.CI || process.env.CONTAINER) {
|
|
launchArgs.push('--no-sandbox');
|
|
}
|
|
|
|
if (extensionsDir) {
|
|
launchArgs.push(
|
|
`--disable-extensions-except=${extensionsDir}`,
|
|
`--load-extension=${extensionsDir}`,
|
|
'--window-position=-9999,-9999',
|
|
'--window-size=1,1',
|
|
);
|
|
useHeadless = false; // extensions require headed mode; off-screen window simulates headless
|
|
console.log(`[browse] Extensions loaded from: ${extensionsDir}`);
|
|
}
|
|
|
|
this.browser = await chromium.launch({
|
|
headless: useHeadless,
|
|
// On Windows, Chromium's sandbox fails when the server is spawned through
|
|
// the Bun→Node process chain (GitHub #276). Disable it — local daemon
|
|
// browsing user-specified URLs has marginal sandbox benefit.
|
|
chromiumSandbox: process.platform !== 'win32',
|
|
...(launchArgs.length > 0 ? { args: launchArgs } : {}),
|
|
});
|
|
|
|
// Chromium crash → exit with clear message
|
|
this.browser.on('disconnected', () => {
|
|
console.error('[browse] FATAL: Chromium process crashed or was killed. Server exiting.');
|
|
console.error('[browse] Console/network logs flushed to .gstack/browse-*.log');
|
|
process.exit(1);
|
|
});
|
|
|
|
const contextOptions: BrowserContextOptions = {
|
|
viewport: { width: 1280, height: 720 },
|
|
};
|
|
if (this.customUserAgent) {
|
|
contextOptions.userAgent = this.customUserAgent;
|
|
}
|
|
this.context = await this.browser.newContext(contextOptions);
|
|
|
|
if (Object.keys(this.extraHeaders).length > 0) {
|
|
await this.context.setExtraHTTPHeaders(this.extraHeaders);
|
|
}
|
|
|
|
// Create first tab
|
|
await this.newTab();
|
|
}
|
|
|
|
// ─── Headed Mode ─────────────────────────────────────────────
|
|
/**
|
|
* Launch Playwright's bundled Chromium in headed mode with the gstack
|
|
* Chrome extension auto-loaded. Uses launchPersistentContext() which
|
|
* is required for extension loading (launch() + newContext() can't
|
|
* load extensions).
|
|
*
|
|
* The browser launches headed with a visible window — the user sees
|
|
* every action Claude takes in real time.
|
|
*/
|
|
async launchHeaded(): Promise<void> {
|
|
// Clear old state before repopulating
|
|
this.pages.clear();
|
|
this.refMap.clear();
|
|
this.nextTabId = 1;
|
|
|
|
// Find the gstack extension directory for auto-loading
|
|
const extensionPath = this.findExtensionPath();
|
|
const launchArgs = ['--hide-crash-restore-bubble'];
|
|
if (extensionPath) {
|
|
launchArgs.push(`--disable-extensions-except=${extensionPath}`);
|
|
launchArgs.push(`--load-extension=${extensionPath}`);
|
|
}
|
|
|
|
// Launch headed Chromium via Playwright's persistent context.
|
|
// Extensions REQUIRE launchPersistentContext (not launch + newContext).
|
|
// Real Chrome (executablePath/channel) silently blocks --load-extension,
|
|
// so we use Playwright's bundled Chromium which reliably loads extensions.
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const userDataDir = path.join(process.env.HOME || '/tmp', '.gstack', 'chromium-profile');
|
|
fs.mkdirSync(userDataDir, { recursive: true });
|
|
|
|
this.context = await chromium.launchPersistentContext(userDataDir, {
|
|
headless: false,
|
|
args: launchArgs,
|
|
viewport: null, // Use browser's default viewport (real window size)
|
|
// Playwright adds flags that block extension loading
|
|
ignoreDefaultArgs: [
|
|
'--disable-extensions',
|
|
'--disable-component-extensions-with-background-pages',
|
|
],
|
|
});
|
|
this.browser = this.context.browser();
|
|
this.connectionMode = 'headed';
|
|
this.intentionalDisconnect = false;
|
|
|
|
// Inject visual indicator — subtle top-edge amber gradient
|
|
// Extension's content script handles the floating pill
|
|
const indicatorScript = () => {
|
|
const injectIndicator = () => {
|
|
if (document.getElementById('gstack-ctrl')) return;
|
|
|
|
const topLine = document.createElement('div');
|
|
topLine.id = 'gstack-ctrl';
|
|
topLine.style.cssText = `
|
|
position: fixed; top: 0; left: 0; right: 0; height: 2px;
|
|
background: linear-gradient(90deg, #F59E0B, #FBBF24, #F59E0B);
|
|
background-size: 200% 100%;
|
|
animation: gstack-shimmer 3s linear infinite;
|
|
pointer-events: none; z-index: 2147483647;
|
|
opacity: 0.8;
|
|
`;
|
|
|
|
const style = document.createElement('style');
|
|
style.textContent = `
|
|
@keyframes gstack-shimmer {
|
|
0% { background-position: 200% 0; }
|
|
100% { background-position: -200% 0; }
|
|
}
|
|
@media (prefers-reduced-motion: reduce) {
|
|
#gstack-ctrl { animation: none !important; }
|
|
}
|
|
`;
|
|
|
|
document.documentElement.appendChild(style);
|
|
document.documentElement.appendChild(topLine);
|
|
};
|
|
if (document.readyState === 'loading') {
|
|
document.addEventListener('DOMContentLoaded', injectIndicator);
|
|
} else {
|
|
injectIndicator();
|
|
}
|
|
};
|
|
await this.context.addInitScript(indicatorScript);
|
|
|
|
// Persistent context opens a default page — adopt it instead of creating a new one
|
|
const existingPages = this.context.pages();
|
|
if (existingPages.length > 0) {
|
|
const page = existingPages[0];
|
|
const id = this.nextTabId++;
|
|
this.pages.set(id, page);
|
|
this.activeTabId = id;
|
|
this.wirePageEvents(page);
|
|
// Inject indicator on restored page (addInitScript only fires on new navigations)
|
|
try { await page.evaluate(indicatorScript); } catch {}
|
|
} else {
|
|
await this.newTab();
|
|
}
|
|
|
|
// Browser disconnect handler — exit code 2 distinguishes from crashes (1)
|
|
if (this.browser) {
|
|
this.browser.on('disconnected', () => {
|
|
if (this.intentionalDisconnect) return;
|
|
console.error('[browse] Real browser disconnected (user closed or crashed).');
|
|
console.error('[browse] Run `$B connect` to reconnect.');
|
|
process.exit(2);
|
|
});
|
|
}
|
|
|
|
// Headed mode defaults
|
|
this.dialogAutoAccept = false; // Don't dismiss user's real dialogs
|
|
this.isHeaded = true;
|
|
this.consecutiveFailures = 0;
|
|
}
|
|
|
|
async close() {
|
|
if (this.browser || (this.connectionMode === 'headed' && this.context)) {
|
|
if (this.connectionMode === 'headed') {
|
|
// Headed/persistent context mode: close the context (which closes the browser)
|
|
this.intentionalDisconnect = true;
|
|
if (this.browser) this.browser.removeAllListeners('disconnected');
|
|
await Promise.race([
|
|
this.context ? this.context.close() : Promise.resolve(),
|
|
new Promise(resolve => setTimeout(resolve, 5000)),
|
|
]).catch(() => {});
|
|
} else {
|
|
// Launched mode: close the browser we spawned
|
|
this.browser.removeAllListeners('disconnected');
|
|
await Promise.race([
|
|
this.browser.close(),
|
|
new Promise(resolve => setTimeout(resolve, 5000)),
|
|
]).catch(() => {});
|
|
}
|
|
this.browser = null;
|
|
}
|
|
}
|
|
|
|
/** Health check — verifies Chromium is connected AND responsive */
|
|
async isHealthy(): Promise<boolean> {
|
|
if (!this.browser || !this.browser.isConnected()) return false;
|
|
try {
|
|
const page = this.pages.get(this.activeTabId);
|
|
if (!page) return true; // connected but no pages — still healthy
|
|
await Promise.race([
|
|
page.evaluate('1'),
|
|
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 2000)),
|
|
]);
|
|
return true;
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
|
|
// ─── Tab Management ────────────────────────────────────────
|
|
async newTab(url?: string): Promise<number> {
|
|
if (!this.context) throw new Error('Browser not launched');
|
|
|
|
// Validate URL before allocating page to avoid zombie tabs on rejection
|
|
if (url) {
|
|
await validateNavigationUrl(url);
|
|
}
|
|
|
|
const page = await this.context.newPage();
|
|
const id = this.nextTabId++;
|
|
this.pages.set(id, page);
|
|
this.activeTabId = id;
|
|
|
|
// Wire up console/network/dialog capture
|
|
this.wirePageEvents(page);
|
|
|
|
if (url) {
|
|
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 15000 });
|
|
}
|
|
|
|
return id;
|
|
}
|
|
|
|
async closeTab(id?: number): Promise<void> {
|
|
const tabId = id ?? this.activeTabId;
|
|
const page = this.pages.get(tabId);
|
|
if (!page) throw new Error(`Tab ${tabId} not found`);
|
|
|
|
await page.close();
|
|
this.pages.delete(tabId);
|
|
|
|
// Switch to another tab if we closed the active one
|
|
if (tabId === this.activeTabId) {
|
|
const remaining = [...this.pages.keys()];
|
|
if (remaining.length > 0) {
|
|
this.activeTabId = remaining[remaining.length - 1];
|
|
} else {
|
|
// No tabs left — create a new blank one
|
|
await this.newTab();
|
|
}
|
|
}
|
|
}
|
|
|
|
switchTab(id: number): void {
|
|
if (!this.pages.has(id)) throw new Error(`Tab ${id} not found`);
|
|
this.activeTabId = id;
|
|
this.activeFrame = null; // Frame context is per-tab
|
|
}
|
|
|
|
getTabCount(): number {
|
|
return this.pages.size;
|
|
}
|
|
|
|
async getTabListWithTitles(): Promise<Array<{ id: number; url: string; title: string; active: boolean }>> {
|
|
const tabs: Array<{ id: number; url: string; title: string; active: boolean }> = [];
|
|
for (const [id, page] of this.pages) {
|
|
tabs.push({
|
|
id,
|
|
url: page.url(),
|
|
title: await page.title().catch(() => ''),
|
|
active: id === this.activeTabId,
|
|
});
|
|
}
|
|
return tabs;
|
|
}
|
|
|
|
// ─── Page Access ───────────────────────────────────────────
|
|
getPage(): Page {
|
|
const page = this.pages.get(this.activeTabId);
|
|
if (!page) throw new Error('No active page. Use "browse goto <url>" first.');
|
|
return page;
|
|
}
|
|
|
|
getCurrentUrl(): string {
|
|
try {
|
|
return this.getPage().url();
|
|
} catch {
|
|
return 'about:blank';
|
|
}
|
|
}
|
|
|
|
// ─── Ref Map ──────────────────────────────────────────────
|
|
setRefMap(refs: Map<string, RefEntry>) {
|
|
this.refMap = refs;
|
|
}
|
|
|
|
clearRefs() {
|
|
this.refMap.clear();
|
|
}
|
|
|
|
/**
|
|
* Resolve a selector that may be a @ref (e.g., "@e3", "@c1") or a CSS selector.
|
|
* Returns { locator } for refs or { selector } for CSS selectors.
|
|
*/
|
|
async resolveRef(selector: string): Promise<{ locator: Locator } | { selector: string }> {
|
|
if (selector.startsWith('@e') || selector.startsWith('@c')) {
|
|
const ref = selector.slice(1); // "e3" or "c1"
|
|
const entry = this.refMap.get(ref);
|
|
if (!entry) {
|
|
throw new Error(
|
|
`Ref ${selector} not found. Run 'snapshot' to get fresh refs.`
|
|
);
|
|
}
|
|
const count = await entry.locator.count();
|
|
if (count === 0) {
|
|
throw new Error(
|
|
`Ref ${selector} (${entry.role} "${entry.name}") is stale — element no longer exists. ` +
|
|
`Run 'snapshot' for fresh refs.`
|
|
);
|
|
}
|
|
return { locator: entry.locator };
|
|
}
|
|
return { selector };
|
|
}
|
|
|
|
/** Get the ARIA role for a ref selector, or null for CSS selectors / unknown refs. */
|
|
getRefRole(selector: string): string | null {
|
|
if (selector.startsWith('@e') || selector.startsWith('@c')) {
|
|
const entry = this.refMap.get(selector.slice(1));
|
|
return entry?.role ?? null;
|
|
}
|
|
return null;
|
|
}
|
|
|
|
getRefCount(): number {
|
|
return this.refMap.size;
|
|
}
|
|
|
|
// ─── Snapshot Diffing ─────────────────────────────────────
|
|
setLastSnapshot(text: string | null) {
|
|
this.lastSnapshot = text;
|
|
}
|
|
|
|
getLastSnapshot(): string | null {
|
|
return this.lastSnapshot;
|
|
}
|
|
|
|
// ─── Dialog Control ───────────────────────────────────────
|
|
setDialogAutoAccept(accept: boolean) {
|
|
this.dialogAutoAccept = accept;
|
|
}
|
|
|
|
getDialogAutoAccept(): boolean {
|
|
return this.dialogAutoAccept;
|
|
}
|
|
|
|
setDialogPromptText(text: string | null) {
|
|
this.dialogPromptText = text;
|
|
}
|
|
|
|
getDialogPromptText(): string | null {
|
|
return this.dialogPromptText;
|
|
}
|
|
|
|
// ─── Viewport ──────────────────────────────────────────────
|
|
async setViewport(width: number, height: number) {
|
|
await this.getPage().setViewportSize({ width, height });
|
|
}
|
|
|
|
// ─── Extra Headers ─────────────────────────────────────────
|
|
async setExtraHeader(name: string, value: string) {
|
|
this.extraHeaders[name] = value;
|
|
if (this.context) {
|
|
await this.context.setExtraHTTPHeaders(this.extraHeaders);
|
|
}
|
|
}
|
|
|
|
// ─── User Agent ────────────────────────────────────────────
|
|
setUserAgent(ua: string) {
|
|
this.customUserAgent = ua;
|
|
}
|
|
|
|
getUserAgent(): string | null {
|
|
return this.customUserAgent;
|
|
}
|
|
|
|
// ─── Lifecycle helpers ───────────────────────────────
|
|
/**
|
|
* Close all open pages and clear the pages map.
|
|
* Used by state load to replace the current session.
|
|
*/
|
|
async closeAllPages(): Promise<void> {
|
|
for (const page of this.pages.values()) {
|
|
await page.close().catch(() => {});
|
|
}
|
|
this.pages.clear();
|
|
this.clearRefs();
|
|
}
|
|
|
|
// ─── Frame context ─────────────────────────────────
|
|
private activeFrame: import('playwright').Frame | null = null;
|
|
|
|
setFrame(frame: import('playwright').Frame | null): void {
|
|
this.activeFrame = frame;
|
|
}
|
|
|
|
getFrame(): import('playwright').Frame | null {
|
|
return this.activeFrame;
|
|
}
|
|
|
|
/**
|
|
* Returns the active frame if set, otherwise the current page.
|
|
* Use this for operations that work on both Page and Frame (locator, evaluate, etc.).
|
|
*/
|
|
getActiveFrameOrPage(): import('playwright').Page | import('playwright').Frame {
|
|
// Auto-recover from detached frames (iframe removed/navigated)
|
|
if (this.activeFrame?.isDetached()) {
|
|
this.activeFrame = null;
|
|
}
|
|
return this.activeFrame ?? this.getPage();
|
|
}
|
|
|
|
// ─── State Save/Restore (shared by recreateContext + handoff) ─
|
|
/**
|
|
* Capture browser state: cookies, localStorage, sessionStorage, URLs, active tab.
|
|
* Skips pages that fail storage reads (e.g., already closed).
|
|
*/
|
|
async saveState(): Promise<BrowserState> {
|
|
if (!this.context) throw new Error('Browser not launched');
|
|
|
|
const cookies = await this.context.cookies();
|
|
const pages: BrowserState['pages'] = [];
|
|
|
|
for (const [id, page] of this.pages) {
|
|
const url = page.url();
|
|
let storage = null;
|
|
try {
|
|
storage = await page.evaluate(() => ({
|
|
localStorage: { ...localStorage },
|
|
sessionStorage: { ...sessionStorage },
|
|
}));
|
|
} catch {}
|
|
pages.push({
|
|
url: url === 'about:blank' ? '' : url,
|
|
isActive: id === this.activeTabId,
|
|
storage,
|
|
});
|
|
}
|
|
|
|
return { cookies, pages };
|
|
}
|
|
|
|
/**
|
|
* Restore browser state into the current context: cookies, pages, storage.
|
|
* Navigates to saved URLs, restores storage, wires page events.
|
|
* Failures on individual pages are swallowed — partial restore is better than none.
|
|
*/
|
|
async restoreState(state: BrowserState): Promise<void> {
|
|
if (!this.context) throw new Error('Browser not launched');
|
|
|
|
// Restore cookies
|
|
if (state.cookies.length > 0) {
|
|
await this.context.addCookies(state.cookies);
|
|
}
|
|
|
|
// Re-create pages
|
|
let activeId: number | null = null;
|
|
for (const saved of state.pages) {
|
|
const page = await this.context.newPage();
|
|
const id = this.nextTabId++;
|
|
this.pages.set(id, page);
|
|
this.wirePageEvents(page);
|
|
|
|
if (saved.url) {
|
|
await page.goto(saved.url, { waitUntil: 'domcontentloaded', timeout: 15000 }).catch(() => {});
|
|
}
|
|
|
|
if (saved.storage) {
|
|
try {
|
|
await page.evaluate((s: { localStorage: Record<string, string>; sessionStorage: Record<string, string> }) => {
|
|
if (s.localStorage) {
|
|
for (const [k, v] of Object.entries(s.localStorage)) {
|
|
localStorage.setItem(k, v);
|
|
}
|
|
}
|
|
if (s.sessionStorage) {
|
|
for (const [k, v] of Object.entries(s.sessionStorage)) {
|
|
sessionStorage.setItem(k, v);
|
|
}
|
|
}
|
|
}, saved.storage);
|
|
} catch {}
|
|
}
|
|
|
|
if (saved.isActive) activeId = id;
|
|
}
|
|
|
|
// If no pages were saved, create a blank one
|
|
if (this.pages.size === 0) {
|
|
await this.newTab();
|
|
} else {
|
|
this.activeTabId = activeId ?? [...this.pages.keys()][0];
|
|
}
|
|
|
|
// Clear refs — pages are new, locators are stale
|
|
this.clearRefs();
|
|
}
|
|
|
|
/**
|
|
* Recreate the browser context to apply user agent changes.
|
|
* Saves and restores cookies, localStorage, sessionStorage, and open pages.
|
|
* Falls back to a clean slate on any failure.
|
|
*/
|
|
async recreateContext(): Promise<string | null> {
|
|
if (this.connectionMode === 'headed') {
|
|
throw new Error('Cannot recreate context in headed mode. Use disconnect first.');
|
|
}
|
|
if (!this.browser || !this.context) {
|
|
throw new Error('Browser not launched');
|
|
}
|
|
|
|
try {
|
|
// 1. Save state
|
|
const state = await this.saveState();
|
|
|
|
// 2. Close old pages and context
|
|
for (const page of this.pages.values()) {
|
|
await page.close().catch(() => {});
|
|
}
|
|
this.pages.clear();
|
|
await this.context.close().catch(() => {});
|
|
|
|
// 3. Create new context with updated settings
|
|
const contextOptions: BrowserContextOptions = {
|
|
viewport: { width: 1280, height: 720 },
|
|
};
|
|
if (this.customUserAgent) {
|
|
contextOptions.userAgent = this.customUserAgent;
|
|
}
|
|
this.context = await this.browser.newContext(contextOptions);
|
|
|
|
if (Object.keys(this.extraHeaders).length > 0) {
|
|
await this.context.setExtraHTTPHeaders(this.extraHeaders);
|
|
}
|
|
|
|
// 4. Restore state
|
|
await this.restoreState(state);
|
|
|
|
return null; // success
|
|
} catch (err: unknown) {
|
|
// Fallback: create a clean context + blank tab
|
|
try {
|
|
this.pages.clear();
|
|
if (this.context) await this.context.close().catch(() => {});
|
|
|
|
const contextOptions: BrowserContextOptions = {
|
|
viewport: { width: 1280, height: 720 },
|
|
};
|
|
if (this.customUserAgent) {
|
|
contextOptions.userAgent = this.customUserAgent;
|
|
}
|
|
this.context = await this.browser!.newContext(contextOptions);
|
|
await this.newTab();
|
|
this.clearRefs();
|
|
} catch {
|
|
// If even the fallback fails, we're in trouble — but browser is still alive
|
|
}
|
|
return `Context recreation failed: ${err instanceof Error ? err.message : String(err)}. Browser reset to blank tab.`;
|
|
}
|
|
}
|
|
|
|
// ─── Handoff: Headless → Headed ─────────────────────────────
|
|
/**
|
|
* Hand off browser control to the user by relaunching in headed mode.
|
|
*
|
|
* Flow (launch-first-close-second for safe rollback):
|
|
* 1. Save state from current headless browser
|
|
* 2. Launch NEW headed browser
|
|
* 3. Restore state into new browser
|
|
* 4. Close OLD headless browser
|
|
* If step 2 fails → return error, headless browser untouched
|
|
*/
|
|
async handoff(message: string): Promise<string> {
|
|
if (this.connectionMode === 'headed' || this.isHeaded) {
|
|
return `HANDOFF: Already in headed mode at ${this.getCurrentUrl()}`;
|
|
}
|
|
if (!this.browser || !this.context) {
|
|
throw new Error('Browser not launched');
|
|
}
|
|
|
|
// 1. Save state from current browser
|
|
const state = await this.saveState();
|
|
const currentUrl = this.getCurrentUrl();
|
|
|
|
// 2. Launch new headed browser with extension (same as launchHeaded)
|
|
// Uses launchPersistentContext so the extension auto-loads.
|
|
let newContext: BrowserContext;
|
|
try {
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const extensionPath = this.findExtensionPath();
|
|
const launchArgs = ['--hide-crash-restore-bubble'];
|
|
if (extensionPath) {
|
|
launchArgs.push(`--disable-extensions-except=${extensionPath}`);
|
|
launchArgs.push(`--load-extension=${extensionPath}`);
|
|
console.log(`[browse] Handoff: loading extension from ${extensionPath}`);
|
|
} else {
|
|
console.log('[browse] Handoff: extension not found — headed mode without side panel');
|
|
}
|
|
|
|
const userDataDir = path.join(process.env.HOME || '/tmp', '.gstack', 'chromium-profile');
|
|
fs.mkdirSync(userDataDir, { recursive: true });
|
|
|
|
newContext = await chromium.launchPersistentContext(userDataDir, {
|
|
headless: false,
|
|
args: launchArgs,
|
|
viewport: null,
|
|
ignoreDefaultArgs: [
|
|
'--disable-extensions',
|
|
'--disable-component-extensions-with-background-pages',
|
|
],
|
|
timeout: 15000,
|
|
});
|
|
} catch (err: unknown) {
|
|
const msg = err instanceof Error ? err.message : String(err);
|
|
return `ERROR: Cannot open headed browser — ${msg}. Headless browser still running.`;
|
|
}
|
|
|
|
// 3. Restore state into new headed browser
|
|
try {
|
|
// Swap to new browser/context before restoreState (it uses this.context)
|
|
const oldBrowser = this.browser;
|
|
|
|
this.context = newContext;
|
|
this.browser = newContext.browser();
|
|
this.pages.clear();
|
|
this.connectionMode = 'headed';
|
|
|
|
if (Object.keys(this.extraHeaders).length > 0) {
|
|
await newContext.setExtraHTTPHeaders(this.extraHeaders);
|
|
}
|
|
|
|
// Register crash handler on new browser
|
|
if (this.browser) {
|
|
this.browser.on('disconnected', () => {
|
|
if (this.intentionalDisconnect) return;
|
|
console.error('[browse] FATAL: Chromium process crashed or was killed. Server exiting.');
|
|
process.exit(1);
|
|
});
|
|
}
|
|
|
|
await this.restoreState(state);
|
|
this.isHeaded = true;
|
|
this.dialogAutoAccept = false; // User controls dialogs in headed mode
|
|
|
|
// 4. Close old headless browser (fire-and-forget)
|
|
oldBrowser.removeAllListeners('disconnected');
|
|
oldBrowser.close().catch(() => {});
|
|
|
|
return [
|
|
`HANDOFF: Browser opened at ${currentUrl}`,
|
|
`MESSAGE: ${message}`,
|
|
`STATUS: Waiting for user. Run 'resume' when done.`,
|
|
].join('\n');
|
|
} catch (err: unknown) {
|
|
// Restore failed — close the new context, keep old state
|
|
await newContext.close().catch(() => {});
|
|
const msg = err instanceof Error ? err.message : String(err);
|
|
return `ERROR: Handoff failed during state restore — ${msg}. Headless browser still running.`;
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Resume AI control after user handoff.
|
|
* Clears stale refs and resets failure counter.
|
|
* The meta-command handler calls handleSnapshot() after this.
|
|
*/
|
|
resume(): void {
|
|
this.clearRefs();
|
|
this.resetFailures();
|
|
this.activeFrame = null;
|
|
}
|
|
|
|
getIsHeaded(): boolean {
|
|
return this.isHeaded;
|
|
}
|
|
|
|
// ─── Auto-handoff Hint (consecutive failure tracking) ───────
|
|
incrementFailures(): void {
|
|
this.consecutiveFailures++;
|
|
}
|
|
|
|
resetFailures(): void {
|
|
this.consecutiveFailures = 0;
|
|
}
|
|
|
|
getFailureHint(): string | null {
|
|
if (this.consecutiveFailures >= 3 && !this.isHeaded) {
|
|
return `HINT: ${this.consecutiveFailures} consecutive failures. Consider using 'handoff' to let the user help.`;
|
|
}
|
|
return null;
|
|
}
|
|
|
|
// ─── Console/Network/Dialog/Ref Wiring ────────────────────
|
|
private wirePageEvents(page: Page) {
|
|
// Clear ref map on navigation — refs point to stale elements after page change
|
|
// (lastSnapshot is NOT cleared — it's a text baseline for diffing)
|
|
page.on('framenavigated', (frame) => {
|
|
if (frame === page.mainFrame()) {
|
|
this.clearRefs();
|
|
this.activeFrame = null; // Navigation invalidates frame context
|
|
}
|
|
});
|
|
|
|
// ─── Dialog auto-handling (prevents browser lockup) ─────
|
|
page.on('dialog', async (dialog) => {
|
|
const entry: DialogEntry = {
|
|
timestamp: Date.now(),
|
|
type: dialog.type(),
|
|
message: dialog.message(),
|
|
defaultValue: dialog.defaultValue() || undefined,
|
|
action: this.dialogAutoAccept ? 'accepted' : 'dismissed',
|
|
response: this.dialogAutoAccept ? (this.dialogPromptText ?? undefined) : undefined,
|
|
};
|
|
addDialogEntry(entry);
|
|
|
|
try {
|
|
if (this.dialogAutoAccept) {
|
|
await dialog.accept(this.dialogPromptText ?? undefined);
|
|
} else {
|
|
await dialog.dismiss();
|
|
}
|
|
} catch {
|
|
// Dialog may have been dismissed by navigation — ignore
|
|
}
|
|
});
|
|
|
|
page.on('console', (msg) => {
|
|
addConsoleEntry({
|
|
timestamp: Date.now(),
|
|
level: msg.type(),
|
|
text: msg.text(),
|
|
});
|
|
});
|
|
|
|
page.on('request', (req) => {
|
|
addNetworkEntry({
|
|
timestamp: Date.now(),
|
|
method: req.method(),
|
|
url: req.url(),
|
|
});
|
|
});
|
|
|
|
page.on('response', (res) => {
|
|
// Find matching request entry and update it (backward scan)
|
|
const url = res.url();
|
|
const status = res.status();
|
|
for (let i = networkBuffer.length - 1; i >= 0; i--) {
|
|
const entry = networkBuffer.get(i);
|
|
if (entry && entry.url === url && !entry.status) {
|
|
networkBuffer.set(i, { ...entry, status, duration: Date.now() - entry.timestamp });
|
|
break;
|
|
}
|
|
}
|
|
});
|
|
|
|
// Capture response sizes via response finished
|
|
page.on('requestfinished', async (req) => {
|
|
try {
|
|
const res = await req.response();
|
|
if (res) {
|
|
const url = req.url();
|
|
const body = await res.body().catch(() => null);
|
|
const size = body ? body.length : 0;
|
|
for (let i = networkBuffer.length - 1; i >= 0; i--) {
|
|
const entry = networkBuffer.get(i);
|
|
if (entry && entry.url === url && !entry.size) {
|
|
networkBuffer.set(i, { ...entry, size });
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
} catch {}
|
|
});
|
|
}
|
|
}
|