mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-09 06:45:46 +02:00
8ca950f6f1
* feat: token registry for multi-agent browser access Per-agent scoped tokens with read/write/admin/meta command categories, domain glob restrictions, rate limiting, expiry, and revocation. Setup key exchange for the /pair-agent ceremony (5-min one-time key → 24h session token). Idempotent exchange handles tunnel drops. 39 tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: integrate token registry + scoped auth into browse server Server changes for multi-agent browser access: - /connect endpoint: setup key exchange for /pair-agent ceremony - /token endpoint: root-only minting of scoped sub-tokens - /token/:clientId DELETE: revoke agent tokens - /agents endpoint: list connected agents (root-only) - /health: strips root token when tunnel is active (P0 security fix) - /command: scope/rate/domain checks via token registry before dispatch - Idle timer skips shutdown when tunnel is active Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: ngrok tunnel integration + @ngrok/ngrok dependency BROWSE_TUNNEL=1 env var starts an ngrok tunnel after Bun.serve(). Reads NGROK_AUTHTOKEN from env or ~/.gstack/ngrok.env. Reads NGROK_DOMAIN for dedicated domain (stable URL). Updates state file with tunnel URL. Feasibility spike confirmed: SDK works in compiled Bun binary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: tab isolation for multi-agent browser access Add per-tab ownership tracking to BrowserManager. Scoped agents must create their own tab via newtab before writing. Unowned tabs (pre-existing, user-opened) are root-only for writes. Read access always allowed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: tab enforcement + POST /pair endpoint + activity attribution Server-side tab ownership check blocks scoped agents from writing to unowned tabs. Special-case newtab records ownership for scoped tokens. POST /pair endpoint creates setup keys for the pairing ceremony. Activity events now include clientId for attribution. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: pair-agent CLI command + instruction block generator One command to pair a remote agent: $B pair-agent. Creates a setup key via POST /pair, prints a copy-pasteable instruction block with curl commands. Smart tunnel fallback (tunnel URL > auto-start > localhost). Flags: --for HOST, --local HOST, --admin, --client NAME. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: tab isolation + instruction block generator tests 14 tests covering tab ownership lifecycle (access checks, unowned tabs, transferTab) and instruction block generator (scopes, URLs, admin flag, troubleshooting section). Fix server-auth test that used fragile sliceBetween boundaries broken by new endpoints. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.15.9.0) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: CSO security fixes — token leak, domain bypass, input validation 1. Remove root token from /health endpoint entirely (CSO #1 CRITICAL). Origin header is spoofable. Extension reads from ~/.gstack/.auth.json. 2. Add domain check for newtab URL (CSO #5). Previously only goto was checked, allowing domain-restricted agents to bypass via newtab. 3. Validate scope values, rateLimit, expiresSeconds in createToken() (CSO #4). Rejects invalid scopes and negative values. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: /pair-agent skill — syntactic sugar for browser sharing Users remember /pair-agent, not $B pair-agent. The skill walks through agent selection (OpenClaw, Hermes, Codex, Cursor, generic), local vs remote setup, tunnel configuration, and includes platform-specific notes for each agent type. Wraps the CLI command with context. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: remote browser access reference for paired agents Full API reference, snapshot→@ref pattern, scopes, tab isolation, error codes, ngrok setup, and same-machine shortcuts. The instruction block points here for deeper reading. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: improved instruction block with snapshot→@ref pattern The paste-into-agent instruction block now teaches the snapshot→@ref workflow (the most powerful browsing pattern), shows the server URL prominently, and uses clearer formatting. Tests updated to match. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: smart ngrok detection + auto-tunnel in pair-agent The pair-agent command now checks ngrok's native config (not just ~/.gstack/ngrok.env) and auto-starts the tunnel when ngrok is available. The skill template walks users through ngrok install and auth if not set up, instead of just printing a dead localhost URL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: on-demand tunnel start via POST /tunnel/start pair-agent now auto-starts the ngrok tunnel without restarting the server. New POST /tunnel/start endpoint reads authtoken from env, ~/.gstack/ngrok.env, or ngrok's native config. CLI detects ngrok availability and calls the endpoint automatically. Zero manual steps when ngrok is installed and authed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pair-agent skill must output the instruction block verbatim Added CRITICAL instruction: the agent MUST output the full instruction block so the user can copy it. Previously the agent could summarize over it, leaving the user with nothing to paste. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: scoped tokens rejected on /command — auth gate ordering bug The blanket validateAuth() gate (root-only) sat above the /command endpoint, rejecting all scoped tokens with 401 before they reached getTokenInfo(). Moved /command above the gate so both root and scoped tokens are accepted. This was the bug Wintermute hit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: pair-agent auto-launches headed mode before pairing When pair-agent detects headless mode, it auto-switches to headed (visible Chromium window) so the user can watch what the remote agent does. Use --headless to skip this. Fixed compiled binary path resolution (process.execPath, not process.argv[1] which is virtual /$bunfs/ in Bun compiled binaries). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: comprehensive tests for auth ordering, tunnel, ngrok, headed mode 16 new tests covering: - /command sits above blanket auth gate (Wintermute bug) - /command uses getTokenInfo not validateAuth - /tunnel/start requires root, checks native ngrok config, returns already_active - /pair creates setup keys not session tokens - Tab ownership checked before command dispatch - Activity events include clientId - Instruction block teaches snapshot→@ref pattern - pair-agent auto-headed mode, process.execPath, --headless skip - isNgrokAvailable checks all 3 sources (gstack env, env var, native config) - handlePairAgent calls /tunnel/start not server restart Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: chain scope bypass + /health info leak when tunneled 1. Chain command now pre-validates ALL subcommand scopes before executing any. A read+meta token can no longer escalate to admin via chain (eval, js, cookies were dispatched without scope checks). tokenInfo flows through handleMetaCommand into the chain handler. Rejects entire chain if any subcommand fails. 2. /health strips sensitive fields (currentUrl, agent.currentMessage, session) when tunnel is active. Only operational metadata (status, mode, uptime, tabs) exposed to the internet. Previously anyone reaching the ngrok URL could surveil browsing activity. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: tout /pair-agent as headline feature in CHANGELOG + README Lead with what it does for the user: type /pair-agent, paste into your other agent, done. First time AI agents from different companies can coordinate through a shared browser with real security boundaries. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: expand /pair-agent, /design-shotgun, /design-html in README Each skill gets a real narrative paragraph explaining the workflow, not just a table cell. design-shotgun: visual exploration with taste memory. design-html: production HTML with Pretext computed layout. pair-agent: cross-vendor AI agent coordination through shared browser. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: split handleCommand into handleCommandInternal + HTTP wrapper Chain subcommands now route through handleCommandInternal for full security enforcement (scope, domain, tab ownership, rate limiting, content wrapping). Adds recursion guard for nested chains, rate-limit exemption for chain subcommands, and activity event suppression (1 event per chain, not per sub). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add content-security.ts with datamarking, envelope, and filter hooks Four-layer prompt injection defense for pair-agent browser sharing: - Datamarking: session-scoped watermark for text exfiltration detection - Content envelope: trust boundary wrapping with ZWSP marker escaping - Content filter hooks: extensible filter pipeline with warn/block modes - Built-in URL blocklist: requestbin, pipedream, webhook.site, etc. BROWSE_CONTENT_FILTER env var controls mode: off|warn|block (default: warn) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: centralize content wrapping in handleCommandInternal response path Single wrapping location replaces fragmented per-handler wrapping: - Scoped tokens: content filters + datamarking + enhanced envelope - Root tokens: existing basic wrapping (backward compat) - Chain subcommands exempt from top-level wrapping (wrapped individually) - Adds 'attrs' to PAGE_CONTENT_COMMANDS (ARIA value exposure defense) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: hidden element stripping for scoped token text extraction Detects CSS-hidden elements (opacity, font-size, off-screen, same-color, clip-path) and ARIA label injection patterns. Marks elements with data-gstack-hidden, extracts text from a clean clone (no DOM mutation), then removes markers. Only active for scoped tokens on text command. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: snapshot split output format for scoped tokens Scoped tokens get a split snapshot: trusted @refs section (for click/fill) separated from untrusted web content in an envelope. Ref names truncated to 50 chars in trusted section. Root tokens unchanged (backward compat). Resume command also uses split format for scoped tokens. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add SECURITY section to pair-agent instruction block Instructs remote agents to treat content inside untrusted envelopes as potentially malicious. Lists common injection phrases to watch for. Directs agents to only use @refs from the trusted INTERACTIVE ELEMENTS section, not from page content. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add 4 prompt injection test fixtures - injection-visible.html: visible injection in product review text - injection-hidden.html: 7 CSS hiding techniques + ARIA injection + false positive - injection-social.html: social engineering in legitimate-looking content - injection-combined.html: all attack types + envelope escape attempt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: comprehensive content security tests (47 tests) Covers all 4 defense layers: - Datamarking: marker format, session consistency, text-only application - Content envelope: wrapping, ZWSP marker escaping, filter warnings - Content filter hooks: URL blocklist, custom filters, warn/block modes - Instruction block: SECURITY section content, ordering, generation - Centralized wrapping: source-level verification of integration - Chain security: recursion guard, rate-limit exemption, activity suppression - Hidden element stripping: 7 CSS techniques, ARIA injection, false positives - Snapshot split format: scoped vs root output, resume integration Also fixes: visibility:hidden detection, case-insensitive ARIA pattern matching. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pair-agent skill compliance + fix all 16 pre-existing test failures Root cause: pair-agent was added without completing the gen-skill-docs compliance checklist. All 16 failures traced back to this. Fixes: - Sync package.json version to VERSION (0.15.9.0) - Add "(gstack)" to pair-agent description for discoverability - Add pair-agent to Codex path exception (legitimately documents ~/.codex/) - Add CLI_COMMANDS (status, pair-agent, tunnel) to skill parser allowlist - Regenerate SKILL.md for all hosts (claude, codex, factory, kiro, etc.) - Update golden file baselines for ship skill - Fix relink tests: pass GSTACK_INSTALL_DIR to auto-relink calls so they use the fast mock install instead of scanning real ~/.claude/skills/gstack Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.15.12.0) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: E2E exit reason precedence + worktree prune race condition Two fixes for E2E test reliability: 1. session-runner.ts: error_max_turns was misclassified as error_api because is_error flag was checked before subtype. Now known subtypes like error_max_turns are preserved even when is_error is set. The is_error override only applies when subtype=success (API failure). 2. worktree.ts: pruneStale() now skips worktrees < 1 hour old to avoid deleting worktrees from concurrent test runs still in progress. Previously any second test execution would kill the first's worktrees. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore token in /health for localhost extension auth The CSO security fix stripped the token from /health to prevent leaking when tunneled. But the extension needs it to authenticate on localhost. Now returns token only when not tunneled (safe: localhost-only path). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: verify /health token is localhost-only, never served through tunnel Updated tests to match the restored token behavior: - Test 1: token assignment exists AND is inside the !tunnelActive guard - Test 1b: tunnel branch (else block) does not contain AUTH_TOKEN Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add security rationale for token in /health on localhost Explains why this is an accepted risk (no escalation over file-based token access), CORS protection, and tunnel guard. Prevents future CSO scans from stripping it without providing an alternative auth path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: verify tunnel is alive before returning URL to pair-agent Root cause: when ngrok dies externally (pkill, crash, timeout), the server still reports tunnelActive=true with a dead URL. pair-agent prints an instruction block pointing at a dead tunnel. The remote agent gets "endpoint offline" and the user has to manually restart everything. Three-layer fix: - Server /pair endpoint: probes tunnel URL before returning it. If dead, resets tunnelActive/tunnelUrl and returns null (triggers CLI restart). - Server /tunnel/start: probes cached tunnel before returning already_active. If dead, falls through to restart ngrok automatically. - CLI pair-agent: double-checks tunnel URL from server before printing instruction block. Falls through to auto-start on failure. 4 regression tests verify all three probe points + CLI verification. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add POST /batch endpoint for multi-command batching Remote agents controlling GStack Browser through a tunnel pay 2-5s of latency per HTTP round-trip. A typical "navigate and read" takes 4 sequential commands = 10-20 seconds. The /batch endpoint collapses N commands into a single HTTP round-trip, cutting a 20-tab crawl from ~60s to ~5s. Sequential execution through the full security pipeline (scope, domain, tab ownership, content wrapping). Rate limiting counts the batch as 1 request. Activity events emitted at batch level, not per-command. Max 50 commands per batch. Nested batches rejected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add source-level security tests for /batch endpoint 8 tests verifying: auth gate placement, scoped token support, max command limit, nested batch rejection, rate limiting bypass, batch-level activity events, command field validation, and tabId passthrough. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: correct CHANGELOG date from 2026-04-06 to 2026-04-05 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: consolidate Hermes into generic HTTP option in pair-agent Hermes doesn't have a host-specific config — it uses the same generic curl instructions as any other agent. Removing the dedicated option simplifies the menu and eliminates a misleading distinction. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump VERSION to 0.15.14.0, add CHANGELOG entry for batch endpoint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate pair-agent/SKILL.md after main merge Vendoring deprecation section from main's template wasn't reflected in the generated file. Fixes check-freshness CI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: checkTabAccess uses options object, add own-only tab policy Refactors checkTabAccess(tabId, clientId, isWrite) to use an options object { isWrite?, ownOnly? }. Adds tabPolicy === 'own-only' support in the server command dispatch — scoped tokens with this policy are restricted to their own tabs for all commands, not just writes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add --domain flag to pair-agent CLI for domain restrictions Allows passing --domain to pair-agent to restrict the remote agent's navigation to specific domains (comma-separated). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * revert: remove batch commands CHANGELOG entry and VERSION bump The batch endpoint work belongs on the browser-batch-multitab branch (port-louis), not this branch. Reverting VERSION to 0.15.14.0. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: adopt main's headed-mode /health token serving Our merge kept the old !tunnelActive guard which conflicted with main's security-audit-r2 tests that require no currentUrl/currentMessage in /health. Adopts main's approach: serve token conditionally based on headed mode or chrome-extension origin. Updates server-auth tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: improve snapshot flags docs completeness for LLM judge Adds $B placeholder explanation, explicit syntax line, and detailed flag behavior (-d depth values, -s CSS selector syntax, -D unified diff format and baseline persistence, -a screenshot vs text output relationship). Fixes snapshot flags reference LLM eval scoring completeness < 4. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1201 lines
44 KiB
TypeScript
1201 lines
44 KiB
TypeScript
/**
|
|
* Browser lifecycle manager
|
|
*
|
|
* Chromium crash handling:
|
|
* browser.on('disconnected') → log error → process.exit(1)
|
|
* CLI detects dead server → auto-restarts on next command
|
|
* We do NOT try to self-heal — don't hide failure.
|
|
*
|
|
* Dialog handling:
|
|
* page.on('dialog') → auto-accept by default → store in dialog buffer
|
|
* Prevents browser lockup from alert/confirm/prompt
|
|
*
|
|
* Context recreation (useragent):
|
|
* recreateContext() saves cookies/storage/URLs, creates new context,
|
|
* restores state. Falls back to clean slate on any failure.
|
|
*/
|
|
|
|
import { chromium, type Browser, type BrowserContext, type BrowserContextOptions, type Page, type Locator, type Cookie } from 'playwright';
|
|
import { addConsoleEntry, addNetworkEntry, addDialogEntry, networkBuffer, type DialogEntry } from './buffers';
|
|
import { validateNavigationUrl } from './url-validation';
|
|
|
|
export interface RefEntry {
|
|
locator: Locator;
|
|
role: string;
|
|
name: string;
|
|
}
|
|
|
|
export interface BrowserState {
|
|
cookies: Cookie[];
|
|
pages: Array<{
|
|
url: string;
|
|
isActive: boolean;
|
|
storage: { localStorage: Record<string, string>; sessionStorage: Record<string, string> } | null;
|
|
}>;
|
|
}
|
|
|
|
export class BrowserManager {
|
|
private browser: Browser | null = null;
|
|
private context: BrowserContext | null = null;
|
|
private pages: Map<number, Page> = new Map();
|
|
private activeTabId: number = 0;
|
|
private nextTabId: number = 1;
|
|
private extraHeaders: Record<string, string> = {};
|
|
private customUserAgent: string | null = null;
|
|
|
|
/** Server port — set after server starts, used by cookie-import-browser command */
|
|
public serverPort: number = 0;
|
|
|
|
// ─── Tab Ownership (multi-agent isolation) ──────────────
|
|
// Maps tabId → clientId. Unowned tabs (not in this map) are root-only for writes.
|
|
private tabOwnership: Map<number, string> = new Map();
|
|
|
|
// ─── Ref Map (snapshot → @e1, @e2, @c1, @c2, ...) ────────
|
|
private refMap: Map<string, RefEntry> = new Map();
|
|
|
|
// ─── Snapshot Diffing ─────────────────────────────────────
|
|
// NOT cleared on navigation — it's a text baseline for diffing
|
|
private lastSnapshot: string | null = null;
|
|
|
|
// ─── Dialog Handling ──────────────────────────────────────
|
|
private dialogAutoAccept: boolean = true;
|
|
private dialogPromptText: string | null = null;
|
|
|
|
// ─── Handoff State ─────────────────────────────────────────
|
|
private isHeaded: boolean = false;
|
|
private consecutiveFailures: number = 0;
|
|
|
|
// ─── Watch Mode ─────────────────────────────────────────
|
|
private watching = false;
|
|
public watchInterval: ReturnType<typeof setInterval> | null = null;
|
|
private watchSnapshots: string[] = [];
|
|
private watchStartTime: number = 0;
|
|
|
|
// ─── Headed State ────────────────────────────────────────
|
|
private connectionMode: 'launched' | 'headed' = 'launched';
|
|
private intentionalDisconnect = false;
|
|
|
|
getConnectionMode(): 'launched' | 'headed' { return this.connectionMode; }
|
|
|
|
// ─── Watch Mode Methods ─────────────────────────────────
|
|
isWatching(): boolean { return this.watching; }
|
|
|
|
startWatch(): void {
|
|
this.watching = true;
|
|
this.watchSnapshots = [];
|
|
this.watchStartTime = Date.now();
|
|
}
|
|
|
|
stopWatch(): { snapshots: string[]; duration: number } {
|
|
this.watching = false;
|
|
if (this.watchInterval) {
|
|
clearInterval(this.watchInterval);
|
|
this.watchInterval = null;
|
|
}
|
|
const snapshots = this.watchSnapshots;
|
|
const duration = Date.now() - this.watchStartTime;
|
|
this.watchSnapshots = [];
|
|
this.watchStartTime = 0;
|
|
return { snapshots, duration };
|
|
}
|
|
|
|
addWatchSnapshot(snapshot: string): void {
|
|
this.watchSnapshots.push(snapshot);
|
|
}
|
|
|
|
/**
|
|
* Find the gstack Chrome extension directory.
|
|
* Checks: repo root /extension, global install, dev install.
|
|
*/
|
|
private findExtensionPath(): string | null {
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const candidates = [
|
|
// Explicit override via env var (used by GStack Browser.app bundle)
|
|
process.env.BROWSE_EXTENSIONS_DIR || '',
|
|
// Relative to this source file (dev mode: browse/src/ -> ../../extension)
|
|
path.resolve(__dirname, '..', '..', 'extension'),
|
|
// Global gstack install
|
|
path.join(process.env.HOME || '', '.claude', 'skills', 'gstack', 'extension'),
|
|
// Git repo root (detected via BROWSE_STATE_FILE location)
|
|
(() => {
|
|
const stateFile = process.env.BROWSE_STATE_FILE || '';
|
|
if (stateFile) {
|
|
const repoRoot = path.resolve(path.dirname(stateFile), '..');
|
|
return path.join(repoRoot, '.claude', 'skills', 'gstack', 'extension');
|
|
}
|
|
return '';
|
|
})(),
|
|
].filter(Boolean);
|
|
|
|
for (const candidate of candidates) {
|
|
try {
|
|
if (fs.existsSync(path.join(candidate, 'manifest.json'))) {
|
|
return candidate;
|
|
}
|
|
} catch {}
|
|
}
|
|
return null;
|
|
}
|
|
|
|
/**
|
|
* Get the ref map for external consumers (e.g., /refs endpoint).
|
|
*/
|
|
getRefMap(): Array<{ ref: string; role: string; name: string }> {
|
|
const refs: Array<{ ref: string; role: string; name: string }> = [];
|
|
for (const [ref, entry] of this.refMap) {
|
|
refs.push({ ref, role: entry.role, name: entry.name });
|
|
}
|
|
return refs;
|
|
}
|
|
|
|
async launch() {
|
|
// ─── Extension Support ────────────────────────────────────
|
|
// BROWSE_EXTENSIONS_DIR points to an unpacked Chrome extension directory.
|
|
// Extensions only work in headed mode, so we use an off-screen window.
|
|
const extensionsDir = process.env.BROWSE_EXTENSIONS_DIR;
|
|
const launchArgs: string[] = [];
|
|
let useHeadless = true;
|
|
|
|
// Docker/CI: Chromium sandbox requires unprivileged user namespaces which
|
|
// are typically disabled in containers. Detect container environment and
|
|
// add --no-sandbox automatically.
|
|
if (process.env.CI || process.env.CONTAINER) {
|
|
launchArgs.push('--no-sandbox');
|
|
}
|
|
|
|
if (extensionsDir) {
|
|
launchArgs.push(
|
|
`--disable-extensions-except=${extensionsDir}`,
|
|
`--load-extension=${extensionsDir}`,
|
|
'--window-position=-9999,-9999',
|
|
'--window-size=1,1',
|
|
);
|
|
useHeadless = false; // extensions require headed mode; off-screen window simulates headless
|
|
console.log(`[browse] Extensions loaded from: ${extensionsDir}`);
|
|
}
|
|
|
|
this.browser = await chromium.launch({
|
|
headless: useHeadless,
|
|
// On Windows, Chromium's sandbox fails when the server is spawned through
|
|
// the Bun→Node process chain (GitHub #276). Disable it — local daemon
|
|
// browsing user-specified URLs has marginal sandbox benefit.
|
|
chromiumSandbox: process.platform !== 'win32',
|
|
...(launchArgs.length > 0 ? { args: launchArgs } : {}),
|
|
});
|
|
|
|
// Chromium crash → exit with clear message
|
|
this.browser.on('disconnected', () => {
|
|
console.error('[browse] FATAL: Chromium process crashed or was killed. Server exiting.');
|
|
console.error('[browse] Console/network logs flushed to .gstack/browse-*.log');
|
|
process.exit(1);
|
|
});
|
|
|
|
const contextOptions: BrowserContextOptions = {
|
|
viewport: { width: 1280, height: 720 },
|
|
};
|
|
if (this.customUserAgent) {
|
|
contextOptions.userAgent = this.customUserAgent;
|
|
}
|
|
this.context = await this.browser.newContext(contextOptions);
|
|
|
|
if (Object.keys(this.extraHeaders).length > 0) {
|
|
await this.context.setExtraHTTPHeaders(this.extraHeaders);
|
|
}
|
|
|
|
// Create first tab
|
|
await this.newTab();
|
|
}
|
|
|
|
// ─── Headed Mode ─────────────────────────────────────────────
|
|
/**
|
|
* Launch Playwright's bundled Chromium in headed mode with the gstack
|
|
* Chrome extension auto-loaded. Uses launchPersistentContext() which
|
|
* is required for extension loading (launch() + newContext() can't
|
|
* load extensions).
|
|
*
|
|
* The browser launches headed with a visible window — the user sees
|
|
* every action Claude takes in real time.
|
|
*/
|
|
async launchHeaded(authToken?: string): Promise<void> {
|
|
// Clear old state before repopulating
|
|
this.pages.clear();
|
|
this.refMap.clear();
|
|
this.nextTabId = 1;
|
|
|
|
// Find the gstack extension directory for auto-loading
|
|
const extensionPath = this.findExtensionPath();
|
|
const launchArgs = [
|
|
'--hide-crash-restore-bubble',
|
|
// Anti-bot-detection: remove the navigator.webdriver flag that Playwright sets.
|
|
// Sites like Google and NYTimes check this to block automation browsers.
|
|
'--disable-blink-features=AutomationControlled',
|
|
];
|
|
if (extensionPath) {
|
|
launchArgs.push(`--disable-extensions-except=${extensionPath}`);
|
|
launchArgs.push(`--load-extension=${extensionPath}`);
|
|
// Write auth token for extension bootstrap.
|
|
// Write to ~/.gstack/.auth.json (not the extension dir, which may be read-only
|
|
// in .app bundles and breaks codesigning).
|
|
if (authToken) {
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const gstackDir = path.join(process.env.HOME || '/tmp', '.gstack');
|
|
fs.mkdirSync(gstackDir, { recursive: true });
|
|
const authFile = path.join(gstackDir, '.auth.json');
|
|
try {
|
|
fs.writeFileSync(authFile, JSON.stringify({ token: authToken, port: this.serverPort || 34567 }), { mode: 0o600 });
|
|
} catch (err: any) {
|
|
console.warn(`[browse] Could not write .auth.json: ${err.message}`);
|
|
}
|
|
}
|
|
}
|
|
|
|
// Launch headed Chromium via Playwright's persistent context.
|
|
// Extensions REQUIRE launchPersistentContext (not launch + newContext).
|
|
// Real Chrome (executablePath/channel) silently blocks --load-extension,
|
|
// so we use Playwright's bundled Chromium which reliably loads extensions.
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const userDataDir = path.join(process.env.HOME || '/tmp', '.gstack', 'chromium-profile');
|
|
fs.mkdirSync(userDataDir, { recursive: true });
|
|
|
|
// Support custom Chromium binary via GSTACK_CHROMIUM_PATH env var.
|
|
// Used by GStack Browser.app to point at the bundled Chromium.
|
|
const executablePath = process.env.GSTACK_CHROMIUM_PATH || undefined;
|
|
|
|
// Rebrand Chromium → GStack Browser in macOS menu bar / Dock / Cmd+Tab.
|
|
// Patch the Chromium .app's Info.plist so macOS shows our name.
|
|
// This works for both dev mode (system Playwright cache) and .app bundle.
|
|
const chromePath = executablePath || chromium.executablePath();
|
|
try {
|
|
// Walk up from binary to the .app's Info.plist
|
|
// e.g. .../Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing
|
|
// → .../Google Chrome for Testing.app/Contents/Info.plist
|
|
const chromeContentsDir = path.resolve(path.dirname(chromePath), '..');
|
|
const chromePlist = path.join(chromeContentsDir, 'Info.plist');
|
|
if (fs.existsSync(chromePlist)) {
|
|
const plistContent = fs.readFileSync(chromePlist, 'utf-8');
|
|
if (plistContent.includes('Google Chrome for Testing')) {
|
|
const patched = plistContent
|
|
.replace(/Google Chrome for Testing/g, 'GStack Browser');
|
|
fs.writeFileSync(chromePlist, patched);
|
|
}
|
|
// Replace Chromium's Dock icon with ours (Chromium's process owns the Dock icon)
|
|
const iconCandidates = [
|
|
path.join(__dirname, '..', '..', 'scripts', 'app', 'icon.icns'), // repo dev mode
|
|
path.join(process.env.HOME || '', '.claude', 'skills', 'gstack', 'scripts', 'app', 'icon.icns'), // global install
|
|
];
|
|
const iconSrc = iconCandidates.find(p => fs.existsSync(p));
|
|
if (iconSrc) {
|
|
const chromeResources = path.join(chromeContentsDir, 'Resources');
|
|
// Read original icon name from plist
|
|
const iconMatch = plistContent.match(/<key>CFBundleIconFile<\/key>\s*<string>([^<]+)<\/string>/);
|
|
let origIcon = iconMatch ? iconMatch[1] : 'app';
|
|
if (!origIcon.endsWith('.icns')) origIcon += '.icns';
|
|
const destIcon = path.join(chromeResources, origIcon);
|
|
try { fs.copyFileSync(iconSrc, destIcon); } catch { /* non-fatal */ }
|
|
}
|
|
}
|
|
} catch {
|
|
// Non-fatal: app name just stays as Chrome for Testing
|
|
}
|
|
|
|
// Build custom user agent: keep Chrome version for site compatibility,
|
|
// but replace "Chrome for Testing" branding with "GStackBrowser"
|
|
let customUA: string | undefined;
|
|
if (!this.customUserAgent) {
|
|
// Detect Chrome version from the Chromium binary
|
|
const chromePath = executablePath || chromium.executablePath();
|
|
try {
|
|
const versionProc = Bun.spawnSync([chromePath, '--version'], {
|
|
stdout: 'pipe', stderr: 'pipe', timeout: 5000,
|
|
});
|
|
const versionOutput = versionProc.stdout.toString().trim();
|
|
// Output like: "Google Chrome for Testing 145.0.6422.0" or "Chromium 145.0.6422.0"
|
|
const versionMatch = versionOutput.match(/(\d+\.\d+\.\d+\.\d+)/);
|
|
const chromeVersion = versionMatch ? versionMatch[1] : '131.0.0.0';
|
|
customUA = `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/${chromeVersion} Safari/537.36 GStackBrowser`;
|
|
} catch {
|
|
// Fallback: generic modern Chrome UA
|
|
customUA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 GStackBrowser';
|
|
}
|
|
}
|
|
|
|
this.context = await chromium.launchPersistentContext(userDataDir, {
|
|
headless: false,
|
|
args: launchArgs,
|
|
viewport: null, // Use browser's default viewport (real window size)
|
|
userAgent: this.customUserAgent || customUA,
|
|
...(executablePath ? { executablePath } : {}),
|
|
// Playwright adds flags that block extension loading
|
|
ignoreDefaultArgs: [
|
|
'--disable-extensions',
|
|
'--disable-component-extensions-with-background-pages',
|
|
],
|
|
});
|
|
this.browser = this.context.browser();
|
|
this.connectionMode = 'headed';
|
|
this.intentionalDisconnect = false;
|
|
|
|
// ─── Anti-bot-detection stealth patches ───────────────────────
|
|
// Playwright's Chromium is detected by sites like Google/NYTimes via:
|
|
// 1. navigator.webdriver = true (handled by --disable-blink-features above)
|
|
// 2. Missing plugins array (real Chrome has PDF viewer, etc.)
|
|
// 3. Missing languages
|
|
// 4. CDP runtime detection (window.cdc_* variables)
|
|
// 5. Permissions API returning 'denied' for notifications
|
|
await this.context.addInitScript(() => {
|
|
// Fake plugins array (real Chrome has at least PDF Viewer)
|
|
Object.defineProperty(navigator, 'plugins', {
|
|
get: () => {
|
|
const plugins = [
|
|
{ name: 'PDF Viewer', filename: 'internal-pdf-viewer', description: 'Portable Document Format' },
|
|
{ name: 'Chrome PDF Viewer', filename: 'internal-pdf-viewer', description: '' },
|
|
{ name: 'Chromium PDF Viewer', filename: 'internal-pdf-viewer', description: '' },
|
|
];
|
|
(plugins as any).namedItem = (name: string) => plugins.find(p => p.name === name) || null;
|
|
(plugins as any).refresh = () => {};
|
|
return plugins;
|
|
},
|
|
});
|
|
|
|
// Fake languages (Playwright sometimes sends empty)
|
|
Object.defineProperty(navigator, 'languages', {
|
|
get: () => ['en-US', 'en'],
|
|
});
|
|
|
|
// Remove CDP runtime artifacts that automation detectors look for
|
|
// cdc_ prefixed vars are injected by ChromeDriver/CDP
|
|
const cleanup = () => {
|
|
for (const key of Object.keys(window)) {
|
|
if (key.startsWith('cdc_') || key.startsWith('__webdriver')) {
|
|
try { delete (window as any)[key]; } catch {}
|
|
}
|
|
}
|
|
};
|
|
cleanup();
|
|
// Re-clean after a tick in case they're injected late
|
|
setTimeout(cleanup, 0);
|
|
|
|
// Override Permissions API to return 'prompt' for notifications
|
|
// (automation browsers return 'denied' which is a fingerprint)
|
|
const originalQuery = window.navigator.permissions?.query;
|
|
if (originalQuery) {
|
|
(window.navigator.permissions as any).query = (params: any) => {
|
|
if (params.name === 'notifications') {
|
|
return Promise.resolve({ state: 'prompt', onchange: null } as PermissionStatus);
|
|
}
|
|
return originalQuery.call(window.navigator.permissions, params);
|
|
};
|
|
}
|
|
});
|
|
|
|
// Inject visual indicator — subtle top-edge amber gradient
|
|
// Extension's content script handles the floating pill
|
|
const indicatorScript = () => {
|
|
const injectIndicator = () => {
|
|
if (document.getElementById('gstack-ctrl')) return;
|
|
|
|
const topLine = document.createElement('div');
|
|
topLine.id = 'gstack-ctrl';
|
|
topLine.style.cssText = `
|
|
position: fixed; top: 0; left: 0; right: 0; height: 2px;
|
|
background: linear-gradient(90deg, #F59E0B, #FBBF24, #F59E0B);
|
|
background-size: 200% 100%;
|
|
animation: gstack-shimmer 3s linear infinite;
|
|
pointer-events: none; z-index: 2147483647;
|
|
opacity: 0.8;
|
|
`;
|
|
|
|
const style = document.createElement('style');
|
|
style.textContent = `
|
|
@keyframes gstack-shimmer {
|
|
0% { background-position: 200% 0; }
|
|
100% { background-position: -200% 0; }
|
|
}
|
|
@media (prefers-reduced-motion: reduce) {
|
|
#gstack-ctrl { animation: none !important; }
|
|
}
|
|
`;
|
|
|
|
document.documentElement.appendChild(style);
|
|
document.documentElement.appendChild(topLine);
|
|
};
|
|
if (document.readyState === 'loading') {
|
|
document.addEventListener('DOMContentLoaded', injectIndicator);
|
|
} else {
|
|
injectIndicator();
|
|
}
|
|
};
|
|
await this.context.addInitScript(indicatorScript);
|
|
|
|
// Track user-created tabs automatically (Cmd+T, link opens in new tab, etc.)
|
|
this.context.on('page', (page) => {
|
|
const id = this.nextTabId++;
|
|
this.pages.set(id, page);
|
|
this.activeTabId = id;
|
|
this.wirePageEvents(page);
|
|
// Inject indicator on the new tab
|
|
page.evaluate(indicatorScript).catch(() => {});
|
|
console.log(`[browse] New tab detected (id=${id}, total=${this.pages.size})`);
|
|
});
|
|
|
|
// Persistent context opens a default page — adopt it instead of creating a new one
|
|
const existingPages = this.context.pages();
|
|
if (existingPages.length > 0) {
|
|
const page = existingPages[0];
|
|
const id = this.nextTabId++;
|
|
this.pages.set(id, page);
|
|
this.activeTabId = id;
|
|
this.wirePageEvents(page);
|
|
// Inject indicator on restored page (addInitScript only fires on new navigations)
|
|
try { await page.evaluate(indicatorScript); } catch {}
|
|
} else {
|
|
await this.newTab();
|
|
}
|
|
|
|
// Browser disconnect handler — exit code 2 distinguishes from crashes (1)
|
|
if (this.browser) {
|
|
this.browser.on('disconnected', () => {
|
|
if (this.intentionalDisconnect) return;
|
|
console.error('[browse] Real browser disconnected (user closed or crashed).');
|
|
console.error('[browse] Run `$B connect` to reconnect.');
|
|
process.exit(2);
|
|
});
|
|
}
|
|
|
|
// Headed mode defaults
|
|
this.dialogAutoAccept = false; // Don't dismiss user's real dialogs
|
|
this.isHeaded = true;
|
|
this.consecutiveFailures = 0;
|
|
}
|
|
|
|
async close() {
|
|
if (this.browser || (this.connectionMode === 'headed' && this.context)) {
|
|
if (this.connectionMode === 'headed') {
|
|
// Headed/persistent context mode: close the context (which closes the browser)
|
|
this.intentionalDisconnect = true;
|
|
if (this.browser) this.browser.removeAllListeners('disconnected');
|
|
await Promise.race([
|
|
this.context ? this.context.close() : Promise.resolve(),
|
|
new Promise(resolve => setTimeout(resolve, 5000)),
|
|
]).catch(() => {});
|
|
} else {
|
|
// Launched mode: close the browser we spawned
|
|
this.browser.removeAllListeners('disconnected');
|
|
await Promise.race([
|
|
this.browser.close(),
|
|
new Promise(resolve => setTimeout(resolve, 5000)),
|
|
]).catch(() => {});
|
|
}
|
|
this.browser = null;
|
|
}
|
|
}
|
|
|
|
/** Health check — verifies Chromium is connected AND responsive */
|
|
async isHealthy(): Promise<boolean> {
|
|
if (!this.browser || !this.browser.isConnected()) return false;
|
|
try {
|
|
const page = this.pages.get(this.activeTabId);
|
|
if (!page) return true; // connected but no pages — still healthy
|
|
await Promise.race([
|
|
page.evaluate('1'),
|
|
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 2000)),
|
|
]);
|
|
return true;
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
|
|
// ─── Tab Management ────────────────────────────────────────
|
|
async newTab(url?: string, clientId?: string): Promise<number> {
|
|
if (!this.context) throw new Error('Browser not launched');
|
|
|
|
// Validate URL before allocating page to avoid zombie tabs on rejection
|
|
if (url) {
|
|
await validateNavigationUrl(url);
|
|
}
|
|
|
|
const page = await this.context.newPage();
|
|
const id = this.nextTabId++;
|
|
this.pages.set(id, page);
|
|
this.activeTabId = id;
|
|
|
|
// Record tab ownership for multi-agent isolation
|
|
if (clientId) {
|
|
this.tabOwnership.set(id, clientId);
|
|
}
|
|
|
|
// Wire up console/network/dialog capture
|
|
this.wirePageEvents(page);
|
|
|
|
if (url) {
|
|
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 15000 });
|
|
}
|
|
|
|
return id;
|
|
}
|
|
|
|
async closeTab(id?: number): Promise<void> {
|
|
const tabId = id ?? this.activeTabId;
|
|
const page = this.pages.get(tabId);
|
|
if (!page) throw new Error(`Tab ${tabId} not found`);
|
|
|
|
await page.close();
|
|
this.pages.delete(tabId);
|
|
this.tabOwnership.delete(tabId);
|
|
|
|
// Switch to another tab if we closed the active one
|
|
if (tabId === this.activeTabId) {
|
|
const remaining = [...this.pages.keys()];
|
|
if (remaining.length > 0) {
|
|
this.activeTabId = remaining[remaining.length - 1];
|
|
} else {
|
|
// No tabs left — create a new blank one
|
|
await this.newTab();
|
|
}
|
|
}
|
|
}
|
|
|
|
switchTab(id: number, opts?: { bringToFront?: boolean }): void {
|
|
if (!this.pages.has(id)) throw new Error(`Tab ${id} not found`);
|
|
this.activeTabId = id;
|
|
this.activeFrame = null; // Frame context is per-tab
|
|
// Only bring to front when explicitly requested (user-initiated tab switch).
|
|
// Internal tab pinning (BROWSE_TAB) should NOT steal focus.
|
|
if (opts?.bringToFront !== false) {
|
|
const page = this.pages.get(id);
|
|
if (page) page.bringToFront().catch(() => {});
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Sync activeTabId to match the tab whose URL matches the Chrome extension's
|
|
* active tab. Called on every /sidebar-tabs poll so manual tab switches in
|
|
* the browser are detected within ~2s.
|
|
*/
|
|
syncActiveTabByUrl(activeUrl: string): void {
|
|
if (!activeUrl || this.pages.size <= 1) return;
|
|
// Try exact match first, then fuzzy match (origin+pathname, ignoring query/fragment)
|
|
let fuzzyId: number | null = null;
|
|
let activeOriginPath = '';
|
|
try {
|
|
const u = new URL(activeUrl);
|
|
activeOriginPath = u.origin + u.pathname;
|
|
} catch {}
|
|
|
|
for (const [id, page] of this.pages) {
|
|
try {
|
|
const pageUrl = page.url();
|
|
// Exact match — best case
|
|
if (pageUrl === activeUrl && id !== this.activeTabId) {
|
|
this.activeTabId = id;
|
|
this.activeFrame = null;
|
|
return;
|
|
}
|
|
// Fuzzy match — origin+pathname (handles query param / fragment differences)
|
|
if (activeOriginPath && fuzzyId === null && id !== this.activeTabId) {
|
|
try {
|
|
const pu = new URL(pageUrl);
|
|
if (pu.origin + pu.pathname === activeOriginPath) {
|
|
fuzzyId = id;
|
|
}
|
|
} catch {}
|
|
}
|
|
} catch {}
|
|
}
|
|
// Fall back to fuzzy match
|
|
if (fuzzyId !== null) {
|
|
this.activeTabId = fuzzyId;
|
|
this.activeFrame = null;
|
|
}
|
|
}
|
|
|
|
getActiveTabId(): number {
|
|
return this.activeTabId;
|
|
}
|
|
|
|
getTabCount(): number {
|
|
return this.pages.size;
|
|
}
|
|
|
|
// ─── Tab Ownership (multi-agent isolation) ──────────────
|
|
|
|
/** Get the owner of a tab, or null if unowned (root-only for writes). */
|
|
getTabOwner(tabId: number): string | null {
|
|
return this.tabOwnership.get(tabId) || null;
|
|
}
|
|
|
|
/**
|
|
* Check if a client can access a tab.
|
|
* If ownOnly or isWrite is true, requires ownership.
|
|
* Otherwise (reads), allow by default.
|
|
*/
|
|
checkTabAccess(tabId: number, clientId: string, options: { isWrite?: boolean; ownOnly?: boolean } = {}): boolean {
|
|
if (clientId === 'root') return true;
|
|
const owner = this.tabOwnership.get(tabId);
|
|
if (options.ownOnly || options.isWrite) {
|
|
if (!owner) return false;
|
|
return owner === clientId;
|
|
}
|
|
return true;
|
|
}
|
|
|
|
/** Transfer tab ownership to a different client. */
|
|
transferTab(tabId: number, toClientId: string): void {
|
|
if (!this.pages.has(tabId)) throw new Error(`Tab ${tabId} not found`);
|
|
this.tabOwnership.set(tabId, toClientId);
|
|
}
|
|
|
|
async getTabListWithTitles(): Promise<Array<{ id: number; url: string; title: string; active: boolean }>> {
|
|
const tabs: Array<{ id: number; url: string; title: string; active: boolean }> = [];
|
|
for (const [id, page] of this.pages) {
|
|
tabs.push({
|
|
id,
|
|
url: page.url(),
|
|
title: await page.title().catch(() => ''),
|
|
active: id === this.activeTabId,
|
|
});
|
|
}
|
|
return tabs;
|
|
}
|
|
|
|
// ─── Page Access ───────────────────────────────────────────
|
|
getPage(): Page {
|
|
const page = this.pages.get(this.activeTabId);
|
|
if (!page) throw new Error('No active page. Use "browse goto <url>" first.');
|
|
return page;
|
|
}
|
|
|
|
getCurrentUrl(): string {
|
|
try {
|
|
return this.getPage().url();
|
|
} catch {
|
|
return 'about:blank';
|
|
}
|
|
}
|
|
|
|
// ─── Ref Map ──────────────────────────────────────────────
|
|
setRefMap(refs: Map<string, RefEntry>) {
|
|
this.refMap = refs;
|
|
}
|
|
|
|
clearRefs() {
|
|
this.refMap.clear();
|
|
}
|
|
|
|
/**
|
|
* Resolve a selector that may be a @ref (e.g., "@e3", "@c1") or a CSS selector.
|
|
* Returns { locator } for refs or { selector } for CSS selectors.
|
|
*/
|
|
async resolveRef(selector: string): Promise<{ locator: Locator } | { selector: string }> {
|
|
if (selector.startsWith('@e') || selector.startsWith('@c')) {
|
|
const ref = selector.slice(1); // "e3" or "c1"
|
|
const entry = this.refMap.get(ref);
|
|
if (!entry) {
|
|
throw new Error(
|
|
`Ref ${selector} not found. Run 'snapshot' to get fresh refs.`
|
|
);
|
|
}
|
|
const count = await entry.locator.count();
|
|
if (count === 0) {
|
|
throw new Error(
|
|
`Ref ${selector} (${entry.role} "${entry.name}") is stale — element no longer exists. ` +
|
|
`Run 'snapshot' for fresh refs.`
|
|
);
|
|
}
|
|
return { locator: entry.locator };
|
|
}
|
|
return { selector };
|
|
}
|
|
|
|
/** Get the ARIA role for a ref selector, or null for CSS selectors / unknown refs. */
|
|
getRefRole(selector: string): string | null {
|
|
if (selector.startsWith('@e') || selector.startsWith('@c')) {
|
|
const entry = this.refMap.get(selector.slice(1));
|
|
return entry?.role ?? null;
|
|
}
|
|
return null;
|
|
}
|
|
|
|
getRefCount(): number {
|
|
return this.refMap.size;
|
|
}
|
|
|
|
// ─── Snapshot Diffing ─────────────────────────────────────
|
|
setLastSnapshot(text: string | null) {
|
|
this.lastSnapshot = text;
|
|
}
|
|
|
|
getLastSnapshot(): string | null {
|
|
return this.lastSnapshot;
|
|
}
|
|
|
|
// ─── Dialog Control ───────────────────────────────────────
|
|
setDialogAutoAccept(accept: boolean) {
|
|
this.dialogAutoAccept = accept;
|
|
}
|
|
|
|
getDialogAutoAccept(): boolean {
|
|
return this.dialogAutoAccept;
|
|
}
|
|
|
|
setDialogPromptText(text: string | null) {
|
|
this.dialogPromptText = text;
|
|
}
|
|
|
|
getDialogPromptText(): string | null {
|
|
return this.dialogPromptText;
|
|
}
|
|
|
|
// ─── Viewport ──────────────────────────────────────────────
|
|
async setViewport(width: number, height: number) {
|
|
await this.getPage().setViewportSize({ width, height });
|
|
}
|
|
|
|
// ─── Extra Headers ─────────────────────────────────────────
|
|
async setExtraHeader(name: string, value: string) {
|
|
this.extraHeaders[name] = value;
|
|
if (this.context) {
|
|
await this.context.setExtraHTTPHeaders(this.extraHeaders);
|
|
}
|
|
}
|
|
|
|
// ─── User Agent ────────────────────────────────────────────
|
|
setUserAgent(ua: string) {
|
|
this.customUserAgent = ua;
|
|
}
|
|
|
|
getUserAgent(): string | null {
|
|
return this.customUserAgent;
|
|
}
|
|
|
|
// ─── Lifecycle helpers ───────────────────────────────
|
|
/**
|
|
* Close all open pages and clear the pages map.
|
|
* Used by state load to replace the current session.
|
|
*/
|
|
async closeAllPages(): Promise<void> {
|
|
for (const page of this.pages.values()) {
|
|
await page.close().catch(() => {});
|
|
}
|
|
this.pages.clear();
|
|
this.clearRefs();
|
|
}
|
|
|
|
// ─── Frame context ─────────────────────────────────
|
|
private activeFrame: import('playwright').Frame | null = null;
|
|
|
|
setFrame(frame: import('playwright').Frame | null): void {
|
|
this.activeFrame = frame;
|
|
}
|
|
|
|
getFrame(): import('playwright').Frame | null {
|
|
return this.activeFrame;
|
|
}
|
|
|
|
/**
|
|
* Returns the active frame if set, otherwise the current page.
|
|
* Use this for operations that work on both Page and Frame (locator, evaluate, etc.).
|
|
*/
|
|
getActiveFrameOrPage(): import('playwright').Page | import('playwright').Frame {
|
|
// Auto-recover from detached frames (iframe removed/navigated)
|
|
if (this.activeFrame?.isDetached()) {
|
|
this.activeFrame = null;
|
|
}
|
|
return this.activeFrame ?? this.getPage();
|
|
}
|
|
|
|
// ─── State Save/Restore (shared by recreateContext + handoff) ─
|
|
/**
|
|
* Capture browser state: cookies, localStorage, sessionStorage, URLs, active tab.
|
|
* Skips pages that fail storage reads (e.g., already closed).
|
|
*/
|
|
async saveState(): Promise<BrowserState> {
|
|
if (!this.context) throw new Error('Browser not launched');
|
|
|
|
const cookies = await this.context.cookies();
|
|
const pages: BrowserState['pages'] = [];
|
|
|
|
for (const [id, page] of this.pages) {
|
|
const url = page.url();
|
|
let storage = null;
|
|
try {
|
|
storage = await page.evaluate(() => ({
|
|
localStorage: { ...localStorage },
|
|
sessionStorage: { ...sessionStorage },
|
|
}));
|
|
} catch {}
|
|
pages.push({
|
|
url: url === 'about:blank' ? '' : url,
|
|
isActive: id === this.activeTabId,
|
|
storage,
|
|
});
|
|
}
|
|
|
|
return { cookies, pages };
|
|
}
|
|
|
|
/**
|
|
* Restore browser state into the current context: cookies, pages, storage.
|
|
* Navigates to saved URLs, restores storage, wires page events.
|
|
* Failures on individual pages are swallowed — partial restore is better than none.
|
|
*/
|
|
async restoreState(state: BrowserState): Promise<void> {
|
|
if (!this.context) throw new Error('Browser not launched');
|
|
|
|
// Restore cookies
|
|
if (state.cookies.length > 0) {
|
|
await this.context.addCookies(state.cookies);
|
|
}
|
|
|
|
// Re-create pages
|
|
let activeId: number | null = null;
|
|
for (const saved of state.pages) {
|
|
const page = await this.context.newPage();
|
|
const id = this.nextTabId++;
|
|
this.pages.set(id, page);
|
|
this.wirePageEvents(page);
|
|
|
|
if (saved.url) {
|
|
// Validate the saved URL before navigating — the state file is user-writable and
|
|
// a tampered URL could navigate to cloud metadata endpoints or file:// URIs.
|
|
try {
|
|
await validateNavigationUrl(saved.url);
|
|
} catch (err: any) {
|
|
console.warn(`[browse] Skipping invalid URL in state file: ${saved.url} — ${err.message}`);
|
|
continue;
|
|
}
|
|
await page.goto(saved.url, { waitUntil: 'domcontentloaded', timeout: 15000 }).catch(() => {});
|
|
}
|
|
|
|
if (saved.storage) {
|
|
try {
|
|
await page.evaluate((s: { localStorage: Record<string, string>; sessionStorage: Record<string, string> }) => {
|
|
if (s.localStorage) {
|
|
for (const [k, v] of Object.entries(s.localStorage)) {
|
|
localStorage.setItem(k, v);
|
|
}
|
|
}
|
|
if (s.sessionStorage) {
|
|
for (const [k, v] of Object.entries(s.sessionStorage)) {
|
|
sessionStorage.setItem(k, v);
|
|
}
|
|
}
|
|
}, saved.storage);
|
|
} catch {}
|
|
}
|
|
|
|
if (saved.isActive) activeId = id;
|
|
}
|
|
|
|
// If no pages were saved, create a blank one
|
|
if (this.pages.size === 0) {
|
|
await this.newTab();
|
|
} else {
|
|
this.activeTabId = activeId ?? [...this.pages.keys()][0];
|
|
}
|
|
|
|
// Clear refs — pages are new, locators are stale
|
|
this.clearRefs();
|
|
}
|
|
|
|
/**
|
|
* Recreate the browser context to apply user agent changes.
|
|
* Saves and restores cookies, localStorage, sessionStorage, and open pages.
|
|
* Falls back to a clean slate on any failure.
|
|
*/
|
|
async recreateContext(): Promise<string | null> {
|
|
if (this.connectionMode === 'headed') {
|
|
throw new Error('Cannot recreate context in headed mode. Use disconnect first.');
|
|
}
|
|
if (!this.browser || !this.context) {
|
|
throw new Error('Browser not launched');
|
|
}
|
|
|
|
try {
|
|
// 1. Save state
|
|
const state = await this.saveState();
|
|
|
|
// 2. Close old pages and context
|
|
for (const page of this.pages.values()) {
|
|
await page.close().catch(() => {});
|
|
}
|
|
this.pages.clear();
|
|
await this.context.close().catch(() => {});
|
|
|
|
// 3. Create new context with updated settings
|
|
const contextOptions: BrowserContextOptions = {
|
|
viewport: { width: 1280, height: 720 },
|
|
};
|
|
if (this.customUserAgent) {
|
|
contextOptions.userAgent = this.customUserAgent;
|
|
}
|
|
this.context = await this.browser.newContext(contextOptions);
|
|
|
|
if (Object.keys(this.extraHeaders).length > 0) {
|
|
await this.context.setExtraHTTPHeaders(this.extraHeaders);
|
|
}
|
|
|
|
// 4. Restore state
|
|
await this.restoreState(state);
|
|
|
|
return null; // success
|
|
} catch (err: unknown) {
|
|
// Fallback: create a clean context + blank tab
|
|
try {
|
|
this.pages.clear();
|
|
if (this.context) await this.context.close().catch(() => {});
|
|
|
|
const contextOptions: BrowserContextOptions = {
|
|
viewport: { width: 1280, height: 720 },
|
|
};
|
|
if (this.customUserAgent) {
|
|
contextOptions.userAgent = this.customUserAgent;
|
|
}
|
|
this.context = await this.browser!.newContext(contextOptions);
|
|
await this.newTab();
|
|
this.clearRefs();
|
|
} catch {
|
|
// If even the fallback fails, we're in trouble — but browser is still alive
|
|
}
|
|
return `Context recreation failed: ${err instanceof Error ? err.message : String(err)}. Browser reset to blank tab.`;
|
|
}
|
|
}
|
|
|
|
// ─── Handoff: Headless → Headed ─────────────────────────────
|
|
/**
|
|
* Hand off browser control to the user by relaunching in headed mode.
|
|
*
|
|
* Flow (launch-first-close-second for safe rollback):
|
|
* 1. Save state from current headless browser
|
|
* 2. Launch NEW headed browser
|
|
* 3. Restore state into new browser
|
|
* 4. Close OLD headless browser
|
|
* If step 2 fails → return error, headless browser untouched
|
|
*/
|
|
async handoff(message: string): Promise<string> {
|
|
if (this.connectionMode === 'headed' || this.isHeaded) {
|
|
return `HANDOFF: Already in headed mode at ${this.getCurrentUrl()}`;
|
|
}
|
|
if (!this.browser || !this.context) {
|
|
throw new Error('Browser not launched');
|
|
}
|
|
|
|
// 1. Save state from current browser
|
|
const state = await this.saveState();
|
|
const currentUrl = this.getCurrentUrl();
|
|
|
|
// 2. Launch new headed browser with extension (same as launchHeaded)
|
|
// Uses launchPersistentContext so the extension auto-loads.
|
|
let newContext: BrowserContext;
|
|
try {
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const extensionPath = this.findExtensionPath();
|
|
const launchArgs = ['--hide-crash-restore-bubble'];
|
|
if (extensionPath) {
|
|
launchArgs.push(`--disable-extensions-except=${extensionPath}`);
|
|
launchArgs.push(`--load-extension=${extensionPath}`);
|
|
// Auth token is served via /health endpoint now (no file write needed).
|
|
// Extension reads token from /health on connect.
|
|
console.log(`[browse] Handoff: loading extension from ${extensionPath}`);
|
|
} else {
|
|
console.log('[browse] Handoff: extension not found — headed mode without side panel');
|
|
}
|
|
|
|
const userDataDir = path.join(process.env.HOME || '/tmp', '.gstack', 'chromium-profile');
|
|
fs.mkdirSync(userDataDir, { recursive: true });
|
|
|
|
newContext = await chromium.launchPersistentContext(userDataDir, {
|
|
headless: false,
|
|
args: launchArgs,
|
|
viewport: null,
|
|
ignoreDefaultArgs: [
|
|
'--disable-extensions',
|
|
'--disable-component-extensions-with-background-pages',
|
|
],
|
|
timeout: 15000,
|
|
});
|
|
} catch (err: unknown) {
|
|
const msg = err instanceof Error ? err.message : String(err);
|
|
return `ERROR: Cannot open headed browser — ${msg}. Headless browser still running.`;
|
|
}
|
|
|
|
// 3. Restore state into new headed browser
|
|
try {
|
|
// Swap to new browser/context before restoreState (it uses this.context)
|
|
const oldBrowser = this.browser;
|
|
|
|
this.context = newContext;
|
|
this.browser = newContext.browser();
|
|
this.pages.clear();
|
|
this.connectionMode = 'headed';
|
|
|
|
if (Object.keys(this.extraHeaders).length > 0) {
|
|
await newContext.setExtraHTTPHeaders(this.extraHeaders);
|
|
}
|
|
|
|
// Register crash handler on new browser
|
|
if (this.browser) {
|
|
this.browser.on('disconnected', () => {
|
|
if (this.intentionalDisconnect) return;
|
|
console.error('[browse] FATAL: Chromium process crashed or was killed. Server exiting.');
|
|
process.exit(1);
|
|
});
|
|
}
|
|
|
|
await this.restoreState(state);
|
|
this.isHeaded = true;
|
|
this.dialogAutoAccept = false; // User controls dialogs in headed mode
|
|
|
|
// 4. Close old headless browser (fire-and-forget)
|
|
oldBrowser.removeAllListeners('disconnected');
|
|
oldBrowser.close().catch(() => {});
|
|
|
|
return [
|
|
`HANDOFF: Browser opened at ${currentUrl}`,
|
|
`MESSAGE: ${message}`,
|
|
`STATUS: Waiting for user. Run 'resume' when done.`,
|
|
].join('\n');
|
|
} catch (err: unknown) {
|
|
// Restore failed — close the new context, keep old state
|
|
await newContext.close().catch(() => {});
|
|
const msg = err instanceof Error ? err.message : String(err);
|
|
return `ERROR: Handoff failed during state restore — ${msg}. Headless browser still running.`;
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Resume AI control after user handoff.
|
|
* Clears stale refs and resets failure counter.
|
|
* The meta-command handler calls handleSnapshot() after this.
|
|
*/
|
|
resume(): void {
|
|
this.clearRefs();
|
|
this.resetFailures();
|
|
this.activeFrame = null;
|
|
}
|
|
|
|
getIsHeaded(): boolean {
|
|
return this.isHeaded;
|
|
}
|
|
|
|
// ─── Auto-handoff Hint (consecutive failure tracking) ───────
|
|
incrementFailures(): void {
|
|
this.consecutiveFailures++;
|
|
}
|
|
|
|
resetFailures(): void {
|
|
this.consecutiveFailures = 0;
|
|
}
|
|
|
|
getFailureHint(): string | null {
|
|
if (this.consecutiveFailures >= 3 && !this.isHeaded) {
|
|
return `HINT: ${this.consecutiveFailures} consecutive failures. Consider using 'handoff' to let the user help.`;
|
|
}
|
|
return null;
|
|
}
|
|
|
|
// ─── Console/Network/Dialog/Ref Wiring ────────────────────
|
|
private wirePageEvents(page: Page) {
|
|
// Track tab close — remove from pages map, switch to another tab
|
|
page.on('close', () => {
|
|
for (const [id, p] of this.pages) {
|
|
if (p === page) {
|
|
this.pages.delete(id);
|
|
console.log(`[browse] Tab closed (id=${id}, remaining=${this.pages.size})`);
|
|
// If the closed tab was active, switch to another
|
|
if (this.activeTabId === id) {
|
|
const remaining = [...this.pages.keys()];
|
|
this.activeTabId = remaining.length > 0 ? remaining[remaining.length - 1] : 0;
|
|
}
|
|
break;
|
|
}
|
|
}
|
|
});
|
|
|
|
// Clear ref map on navigation — refs point to stale elements after page change
|
|
// (lastSnapshot is NOT cleared — it's a text baseline for diffing)
|
|
page.on('framenavigated', (frame) => {
|
|
if (frame === page.mainFrame()) {
|
|
this.clearRefs();
|
|
this.activeFrame = null; // Navigation invalidates frame context
|
|
}
|
|
});
|
|
|
|
// ─── Dialog auto-handling (prevents browser lockup) ─────
|
|
page.on('dialog', async (dialog) => {
|
|
const entry: DialogEntry = {
|
|
timestamp: Date.now(),
|
|
type: dialog.type(),
|
|
message: dialog.message(),
|
|
defaultValue: dialog.defaultValue() || undefined,
|
|
action: this.dialogAutoAccept ? 'accepted' : 'dismissed',
|
|
response: this.dialogAutoAccept ? (this.dialogPromptText ?? undefined) : undefined,
|
|
};
|
|
addDialogEntry(entry);
|
|
|
|
try {
|
|
if (this.dialogAutoAccept) {
|
|
await dialog.accept(this.dialogPromptText ?? undefined);
|
|
} else {
|
|
await dialog.dismiss();
|
|
}
|
|
} catch {
|
|
// Dialog may have been dismissed by navigation — ignore
|
|
}
|
|
});
|
|
|
|
page.on('console', (msg) => {
|
|
addConsoleEntry({
|
|
timestamp: Date.now(),
|
|
level: msg.type(),
|
|
text: msg.text(),
|
|
});
|
|
});
|
|
|
|
page.on('request', (req) => {
|
|
addNetworkEntry({
|
|
timestamp: Date.now(),
|
|
method: req.method(),
|
|
url: req.url(),
|
|
});
|
|
});
|
|
|
|
page.on('response', (res) => {
|
|
// Find matching request entry and update it (backward scan)
|
|
const url = res.url();
|
|
const status = res.status();
|
|
for (let i = networkBuffer.length - 1; i >= 0; i--) {
|
|
const entry = networkBuffer.get(i);
|
|
if (entry && entry.url === url && !entry.status) {
|
|
networkBuffer.set(i, { ...entry, status, duration: Date.now() - entry.timestamp });
|
|
break;
|
|
}
|
|
}
|
|
});
|
|
|
|
// Capture response sizes via response finished
|
|
page.on('requestfinished', async (req) => {
|
|
try {
|
|
const res = await req.response();
|
|
if (res) {
|
|
const url = req.url();
|
|
const body = await res.body().catch(() => null);
|
|
const size = body ? body.length : 0;
|
|
for (let i = networkBuffer.length - 1; i >= 0; i--) {
|
|
const entry = networkBuffer.get(i);
|
|
if (entry && entry.url === url && !entry.size) {
|
|
networkBuffer.set(i, { ...entry, size });
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
} catch {}
|
|
});
|
|
}
|
|
}
|