feat(browse): Puppeteer parity — load-html, screenshot --selector, viewport --scale, file:// (v1.1.0.0) (#1062)

* feat(browse): TabSession loadedHtml + command aliases + DX polish primitives Adds the foundation layer for Puppeteer-parity features: - TabSession.loadedHtml + setTabContent/getLoadedHtml/clearLoadedHtml — enables load-html content to survive context recreation (viewport --scale) via in-memory replay. ASCII lifecycle diagram in the source explains the clear-before-navigation contract. - COMMAND_ALIASES + canonicalizeCommand() helper — single source of truth for name aliases (setcontent / set-content / setContent → load-html), consumed by server dispatch and chain prevalidation. - buildUnknownCommandError() pure function — rich error messages with Levenshtein-based "Did you mean" suggestions (distance ≤ 2, input length ≥ 4 to skip 2-letter noise) and NEW_IN_VERSION upgrade hints. - load-html registered in WRITE_COMMANDS + SCOPE_WRITE so scoped write tokens can use it. - screenshot and viewport descriptions updated for upcoming flags. - New browse/test/dx-polish.test.ts (15 tests): alias canonicalization, Levenshtein threshold + alphabetical tiebreak, short-input guard, NEW_IN_VERSION upgrade hint, alias + scope integration invariants. No consumers yet — pure additive foundation. Safe to bisect on its own. * feat(browse): accept file:// in goto with smart cwd/home-relative parsing Extends validateNavigationUrl to accept file:// URLs scoped to safe dirs (cwd + TEMP_DIR) via the existing validateReadPath policy. The workhorse is a new normalizeFileUrl() helper that handles non-standard relative forms BEFORE the WHATWG URL parser sees them: file:///abs/path.html → unchanged file://./docs/page.html → file://<cwd>/docs/page.html file://~/Documents/page.html → file://<HOME>/Documents/page.html file://docs/page.html → file://<cwd>/docs/page.html file://localhost/abs/path → unchanged file://host.example.com/... → rejected (UNC/network) file:// and file:/// → rejected (would list a directory) Host heuristic rejects segments with '.', ':', '\\', '%', IPv6 brackets, or Windows drive-letter patterns — so file://docs.v1/page.html, file://127.0.0.1/x, file://[::1]/x, and file://C:/Users/x are explicit errors. Uses fileURLToPath() + pathToFileURL() from node:url (never string-concat) so URL escapes like %20 decode correctly and Node rejects encoded-slash traversal (%2F..%2F) outright. Signature change: validateNavigationUrl now returns Promise<string> (the normalized URL) instead of Promise<void>. Existing callers that ignore the return value still compile — they just don't benefit from smart-parsing until updated in follow-up commits. Callers will be migrated in the next few commits (goto, diff, newTab, restoreState). Rewrites the url-validation test file: updates existing tests for the new return type, adds 20+ new tests covering every normalizeFileUrl shape variant, URL-encoding edge cases, and path-traversal rejection. References: codex consult v3 P1 findings on URL parser semantics and fileURLToPath. * feat(browse): BrowserManager deviceScaleFactor + setContent replay + file:// plumbing Three tightly-coupled changes to BrowserManager, all in service of the Puppeteer-parity workflow: 1. deviceScaleFactor + currentViewport tracking. New private fields (default scale=1, viewport=1280x720) + setDeviceScaleFactor(scale, w, h) method. deviceScaleFactor is a context-level Playwright option — changing it requires recreateContext(). The method validates (finite number, 1-3 cap, headed-mode rejected), stores new values, calls recreateContext(), and rolls back the fields on failure so a bad call doesn't leave inconsistent state. Context options at all three sites (launch, recreate happy path, recreate fallback) now honor the stored values instead of hardcoding 1280x720. 2. BrowserState.loadedHtml + loadedHtmlWaitUntil. saveState captures per-tab loadedHtml from the session; restoreState replays it via newSession. setTabContent() — NOT bare page.setContent() — so TabSession.loadedHtml is rehydrated and survives *subsequent* scale changes. In-memory only, never persisted to disk (HTML may contain secrets or customer data). 3. newTab + restoreState now consume validateNavigationUrl's normalized return value. file://./x, file://~/x, and bare-segment forms now take effect at every navigation site, not just the top-level goto command. Together these enable: load-html → viewport --scale 2 → viewport --scale 1.5 → screenshot, with content surviving both context recreations. Codex v2 P0 flagged that bare page.setContent in restoreState would lose content on the second scale change — this commit implements the rehydration path. References: codex v2 P0 (TabSession rehydration), codex v3 P1 (4-caller return value), plan Feature 3 + Feature 4. * feat(browse): load-html, screenshot --selector, viewport --scale, alias dispatch Wires the new handlers and dispatch logic that the previous commits made possible: write-commands.ts - New 'load-html' case: validateReadPath for safe-dir scoping, stat-based actionable errors (not found, directory, oversize), extension allowlist (.html/.htm/.xhtml/.svg), magic-byte sniff with UTF-8 BOM strip accepting any <[a-zA-Z!?] markup opener (not just <!doctype — bare fragments like <div>...</div> work for setContent), 50MB cap via GSTACK_BROWSE_MAX_HTML_BYTES override, frame-context rejection. Calls session.setTabContent() so replay metadata is rehydrated. - viewport command extended: optional [<WxH>], optional [--scale <n>], scale-only variant reads current size via page.viewportSize(). Invalid scale (NaN, Infinity, empty, out of 1-3) throws with named value. Headed mode rejected explicitly. - clearLoadedHtml() called BEFORE goto/back/forward/reload navigation (not after) so a timed-out goto post-commit doesn't leave stale metadata that could resurrect on a later context recreation. Codex v2 P1 catch. - goto uses validateNavigationUrl's normalized return value. meta-commands.ts - screenshot --selector <css> flag: explicit element-screenshot form. Rejects alongside positional selector (both = error), preserves --clip conflict at line 161, composes with --base64 at lines 168-174. - chain canonicalizes each step with canonicalizeCommand — step shape is now { rawName, name, args } so prevalidation, dispatch, WRITE_COMMANDS.has, watch blocking, and result labels all use canonical names while audit labels show 'rawName→name' when aliased. Codex v3 P2 catch — prior shape only canonicalized at prevalidation and diverged everywhere else. - diff command consumes validateNavigationUrl return value for both URLs. server.ts - Command canonicalization inserted immediately after parse, before scope / watch / tab-ownership / content-wrapping checks. rawCommand preserved for future audit (not wired into audit log in this commit — follow-up). - Unknown-command handler replaced with buildUnknownCommandError() from commands.ts — produces 'Unknown command: X. Did you mean Y?' with optional upgrade hint for NEW_IN_VERSION entries. security-audit-r2.test.ts - Updated chain-loop marker from 'for (const cmd of commands)' to 'for (const c of commands)' to match the new chain step shape. Same isWatching + BLOCKED invariants still asserted. * chore: bump version and changelog (v1.1.0.0) - VERSION: 1.0.0.0 → 1.1.0.0 (MINOR bump — new user-facing commands) - package.json: matching version bump - CHANGELOG.md: new 1.1.0.0 entry describing load-html, screenshot --selector, viewport --scale, file:// support, setContent replay, and DX polish in user voice with a dedicated Security section for file:// safe-dirs policy - browse/SKILL.md.tmpl: adds pattern #12 "Render local HTML", pattern #13 "Retina screenshots", and a full Puppeteer → browse cheatsheet with side-by- side API mapping and a worked tweet-renderer migration example - browse/SKILL.md + SKILL.md: regenerated from templates via `bun run gen:skill-docs` to reflect the new command descriptions Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: pre-landing review fixes (9 findings from specialist + adversarial review) Adversarial review (Claude subagent + Codex) surfaced 9 bugs across CRITICAL/HIGH severity. All fixed: 1. tab-session.ts:setTabContent — state mutation moved AFTER the setContent await. Prior order left phantom HTML in replay metadata if setContent threw (timeout, browser crash), which a later viewport --scale would silently replay. Now loadedHtml is only recorded on successful load. 2. browser-manager.ts:setDeviceScaleFactor — rollback now forces a second recreateContext after restoring the old fields. The fallback path in the original recreateContext builds a blank context using whatever this.deviceScaleFactor/currentViewport hold at that moment (which were the NEW values we were trying to apply). Rolling back the fields without a second recreate left the live context at new-scale while state tracked old-scale. Now: restore fields, force re-recreate with old values, only if that ALSO fails do we return a combined error. 3. commands.ts:buildUnknownCommandError — Levenshtein tiebreak simplified to 'd <= 2 && d < bestDist' (strict less). Candidates are pre-sorted alphabetically, so first equal-distance wins by default. The prior '(d === bestDist && best !== undefined && cand < best)' clause was dead code. 4. tab-session.ts:onMainFrameNavigated — now clears loadedHtml, not just refs + frame. Without this, a user who load-html'd then clicked a link (or had a form submit / JS redirect / OAuth flow) would retain the stale replay metadata. The next viewport --scale would silently revert the tab to the ORIGINAL loaded HTML, losing whatever the post-navigation content was. Silent data corruption. Browser-emitted navigations trigger this path via wirePageEvents. 5. browser-manager.ts:saveState + restoreState — tab ownership now flows through BrowserState.owner. Without this, a scoped agent's viewport --scale would strand them: tab IDs change during recreate, ownership map held stale IDs, owner lookup failed. New IDs had no owner, so writes without tabId were denied (DoS). Worse, if the agent sent a stale tabId the server's swallowed-tab-switch-error path would let the command hit whatever tab was currently active (cross-tab authz bypass). Now: clear ownership before restore, re-add per-tab with new IDs. 6. meta-commands.ts:state load — disk-loaded state.pages is now explicit allowlist (url, isActive, storage:null) instead of object spread. Spreading accepted loadedHtml, loadedHtmlWaitUntil, and owner from a user-writable state file, letting a tampered state.json smuggle HTML past load-html's safe-dirs / extension / magic-byte / 50MB-cap validators, or forge tab ownership. Now stripped at the boundary. 7. url-validation.ts:normalizeFileUrl — preserves query string + fragment across normalization. file://./app.html?route=home#login previously resolved to a filesystem path that URL-encoded '?' as %3F and '#' as %23, or (for absolute forms) pathToFileURL dropped them entirely. SPAs and fixture URLs with query params 404'd or loaded the wrong route. Now: split on ?/# before path resolution, reattach after. 8. url-validation.ts:validateNavigationUrl — reattaches parsed.search + parsed.hash to the normalized file:// URL. Same fix at the main validator for absolute paths that go through fileURLToPath round-trip. 9. server.ts:writeAuditEntry — audit entries now include aliasOf when the user typed an alias ('setcontent' → cmd: 'load-html', aliasOf: 'setcontent'). Previously the isAliased variable was computed but dropped, losing the raw input from the forensic trail. Completes the plan's codex v3 P2 requirement. Also added bm.getCurrentViewport() and switched 'viewport --scale'- without-size to read from it (more reliable than page.viewportSize() on headed/transition contexts). Tests pass: exit 0, no failures. Build clean. * test: integration coverage for load-html, screenshot --selector, viewport --scale, replay, aliases Adds 28 Playwright-integration tests that close the coverage gap flagged by the ship-workflow coverage audit (50% → expected ~80%+). **load-html (12 tests):** - happy path loads HTML file, page text matches - bare HTML fragments (<div>...</div>) accepted, not just full documents - missing file arg throws usage - non-.html extension rejected by allowlist - /etc/passwd.html rejected by safe-dirs policy - ENOENT path rejected with actionable "not found" error - directory target rejected - binary file (PNG magic bytes) disguised as .html rejected by magic-byte check - UTF-8 BOM stripped before magic-byte check — BOM-prefixed HTML accepted - --wait-until networkidle exercises non-default branch - invalid --wait-until value rejected - unknown flag rejected **screenshot --selector (5 tests):** - --selector flag captures element, validates Screenshot saved (element) - conflicts with positional selector (both = error) - conflicts with --clip (mutually exclusive) - composes with --base64 (returns data:image/png;base64,...) - missing value throws usage **viewport --scale (5 tests):** - WxH --scale 2 produces PNG with 2x element dimensions (parses IHDR bytes 16-23) - --scale without WxH keeps current size + applies scale - non-finite value (abc) throws "not a finite number" - out-of-range (4, 0.5) throws "between 1 and 3" - missing value throws **setContent replay across context recreation (3 tests):** - load-html → viewport --scale 2: content survives (hits setTabContent replay path) - double cycle 2x → 1.5x: content still survives (proves TabSession rehydration) - goto after load-html clears replay: subsequent viewport --scale does NOT resurrect the stale HTML (validates the onMainFrameNavigated fix) **Command aliases (2 tests):** - setcontent routes to load-html via chain canonicalization - set-content (hyphenated) also routes — both end-to-end through chain dispatch Fixture paths use /tmp (SAFE_DIRECTORIES entry) instead of $TMPDIR which is /var/folders/... on macOS and outside the safe-dirs boundary. Chain result labels use rawName→name format when an alias is resolved (matches the meta-commands.ts chain refactor). Full suite: exit 0, 223/223 pass. * docs: update BROWSER.md + CHANGELOG for v1.1.0.0 BROWSER.md: - Command reference table updated: goto now lists file:// support, load-html added to Navigate row, viewport flagged with --scale option, screenshot row shows --selector + --base64 flags - Screenshot modes table adds the fifth mode (element crop via --selector flag) and notes the tag-selector-not-caught-positionally gotcha - New "Retina screenshots — viewport --scale" subsection explains deviceScaleFactor mechanics, context recreation side effects, and headed-mode rejection - New "Loading local HTML — goto file:// vs load-html" subsection explains the two paths, their tradeoffs (URL state, relative asset resolution), the safe-dirs policy, extension allowlist + magic-byte sniff, 50MB cap, setContent replay across recreateContext, and the alias routing (setcontent → load-html before scope check) CHANGELOG.md (v1.1.0.0 security section expanded, no existing content removed): - State files cannot smuggle HTML or forge tab ownership (allowlist on disk-loaded page fields) - Audit log records aliasOf when a canonical command was reached via an alias (setcontent → load-html) - load-html content clears on real navigations (clicks, form submits, JS redirects) — not just explicit goto. Also notes SPA query/fragment preservation for goto file:// Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-08 00:58:05 +02:00 · 2026-04-18 23:25:33 +08:00
parent 4d2c8d94d0
commit c15b805cd8
20 changed files with 1439 additions and 92 deletions
@@ -31,6 +31,18 @@ export interface BrowserState {
    url: string;
    isActive: boolean;
    storage: { localStorage: Record<string, string>; sessionStorage: Record<string, string> } | null;
+    /**
+     * HTML content loaded via load-html (setContent), replayed after context recreation.
+     * In-memory only — never persisted to disk (HTML may contain secrets or customer data).
+     */
+    loadedHtml?: string;
+    loadedHtmlWaitUntil?: 'load' | 'domcontentloaded' | 'networkidle';
+    /**
+     * Tab owner clientId for multi-agent isolation. Survives context recreation so
+     * scoped agents don't get locked out of their own tabs after viewport --scale.
+     * In-memory only.
+     */
+    owner?: string;
  }>;
 }

@@ -44,6 +56,14 @@ export class BrowserManager {
  private extraHeaders: Record<string, string> = {};
  private customUserAgent: string | null = null;

+  // ─── Viewport + deviceScaleFactor (context options) ──────────
+  // Tracked at the manager level so recreateContext() preserves them.
+  // deviceScaleFactor is a *context* option, not a page-level setter — changes
+  // require recreateContext(). Viewport width/height can change on-page, but we
+  // track the latest so context recreation restores it instead of hardcoding 1280x720.
+  private deviceScaleFactor: number = 1;
+  private currentViewport: { width: number; height: number } = { width: 1280, height: 720 };
+
  /** Server port — set after server starts, used by cookie-import-browser command */
  public serverPort: number = 0;

@@ -197,7 +217,8 @@ export class BrowserManager {
    });

    const contextOptions: BrowserContextOptions = {
-      viewport: { width: 1280, height: 720 },
+      viewport: { width: this.currentViewport.width, height: this.currentViewport.height },
+      deviceScaleFactor: this.deviceScaleFactor,
    };
    if (this.customUserAgent) {
      contextOptions.userAgent = this.customUserAgent;
@@ -550,9 +571,12 @@ export class BrowserManager {
  async newTab(url?: string, clientId?: string): Promise<number> {
    if (!this.context) throw new Error('Browser not launched');

-    // Validate URL before allocating page to avoid zombie tabs on rejection
+    // Validate URL before allocating page to avoid zombie tabs on rejection.
+    // Use the normalized return value for navigation — it handles file://./x and
+    // file://<segment> cwd-relative forms that the standard URL parser doesn't.
+    let normalizedUrl: string | undefined;
    if (url) {
-      await validateNavigationUrl(url);
+      normalizedUrl = await validateNavigationUrl(url);
    }

    const page = await this.context.newPage();
@@ -569,8 +593,8 @@ export class BrowserManager {
    // Wire up console/network/dialog capture
    this.wirePageEvents(page);

-    if (url) {
-      await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 15000 });
+    if (normalizedUrl) {
+      await page.goto(normalizedUrl, { waitUntil: 'domcontentloaded', timeout: 15000 });
    }

    return id;
@@ -792,6 +816,7 @@ export class BrowserManager {

  // ─── Viewport ──────────────────────────────────────────────
  async setViewport(width: number, height: number) {
+    this.currentViewport = { width, height };
    await this.getPage().setViewportSize({ width, height });
  }

@@ -858,10 +883,21 @@ export class BrowserManager {
          sessionStorage: { ...sessionStorage },
        }));
      } catch {}
+
+      // Capture load-html content so a later context recreation (viewport --scale)
+      // can replay it via setTabContent. Never persisted to disk.
+      const session = this.tabSessions.get(id);
+      const loaded = session?.getLoadedHtml();
+      // Preserve tab ownership through recreation so scoped agents aren't locked out.
+      const owner = this.tabOwnership.get(id);
+
      pages.push({
        url: url === 'about:blank' ? '' : url,
        isActive: id === this.activeTabId,
        storage,
+        loadedHtml: loaded?.html,
+        loadedHtmlWaitUntil: loaded?.waitUntil,
+        owner,
      });
    }

@@ -881,25 +917,49 @@ export class BrowserManager {
      await this.context.addCookies(state.cookies);
    }

+    // Clear stale ownership — the old tab IDs are gone. We'll re-add per-tab
+    // owners below as each saved tab gets a fresh ID. Without this reset, old
+    // tabId → clientId entries would linger and match new tabs with the same
+    // sequential IDs, silently granting ownership to the wrong clients.
+    this.tabOwnership.clear();
+
    // Re-create pages
    let activeId: number | null = null;
    for (const saved of state.pages) {
      const page = await this.context.newPage();
      const id = this.nextTabId++;
      this.pages.set(id, page);
-      this.tabSessions.set(id, new TabSession(page));
+      const newSession = new TabSession(page);
+      this.tabSessions.set(id, newSession);
      this.wirePageEvents(page);

-      if (saved.url) {
-        // Validate the saved URL before navigating — the state file is user-writable and
-        // a tampered URL could navigate to cloud metadata endpoints or file:// URIs.
+      // Restore tab ownership for the new ID — preserves scoped-agent isolation
+      // across context recreation (viewport --scale, user-agent change, handoff).
+      if (saved.owner) {
+        this.tabOwnership.set(id, saved.owner);
+      }
+
+      if (saved.loadedHtml) {
+        // Replay load-html content via setTabContent — this rehydrates
+        // TabSession.loadedHtml so the next saveState sees it. page.setContent()
+        // alone would restore the DOM but lose the replay metadata.
        try {
-          await validateNavigationUrl(saved.url);
+          await newSession.setTabContent(saved.loadedHtml, { waitUntil: saved.loadedHtmlWaitUntil });
+        } catch (err: any) {
+          console.warn(`[browse] Failed to replay loadedHtml for tab ${id}: ${err.message}`);
+        }
+      } else if (saved.url) {
+        // Validate the saved URL before navigating — the state file is user-writable and
+        // a tampered URL could navigate to cloud metadata endpoints. Use the normalized
+        // return value so file:// forms get consistent treatment with live goto.
+        let normalizedUrl: string;
+        try {
+          normalizedUrl = await validateNavigationUrl(saved.url);
        } catch (err: any) {
          console.warn(`[browse] Skipping invalid URL in state file: ${saved.url} — ${err.message}`);
          continue;
        }
-        await page.goto(saved.url, { waitUntil: 'domcontentloaded', timeout: 15000 }).catch(() => {});
+        await page.goto(normalizedUrl, { waitUntil: 'domcontentloaded', timeout: 15000 }).catch(() => {});
      }

      if (saved.storage) {
@@ -960,7 +1020,8 @@ export class BrowserManager {

      // 3. Create new context with updated settings
      const contextOptions: BrowserContextOptions = {
-        viewport: { width: 1280, height: 720 },
+        viewport: { width: this.currentViewport.width, height: this.currentViewport.height },
+        deviceScaleFactor: this.deviceScaleFactor,
      };
      if (this.customUserAgent) {
        contextOptions.userAgent = this.customUserAgent;
@@ -983,7 +1044,8 @@ export class BrowserManager {
        if (this.context) await this.context.close().catch(() => {});

        const contextOptions: BrowserContextOptions = {
-          viewport: { width: 1280, height: 720 },
+          viewport: { width: this.currentViewport.width, height: this.currentViewport.height },
+          deviceScaleFactor: this.deviceScaleFactor,
        };
        if (this.customUserAgent) {
          contextOptions.userAgent = this.customUserAgent;
@@ -998,6 +1060,63 @@ export class BrowserManager {
    }
  }

+  /**
+   * Change deviceScaleFactor + viewport size atomically.
+   *
+   * deviceScaleFactor is a context-level option, so Playwright requires a full context
+   * recreation. This method validates the input, stores the new values, calls
+   * recreateContext(), and rolls back the fields on failure so a bad call doesn't
+   * leave the manager in an inconsistent state.
+   *
+   * Returns null on success, or an error string if the new context couldn't be built
+   * (state may have been lost, per recreateContext's fallback behavior).
+   */
+  async setDeviceScaleFactor(scale: number, width: number, height: number): Promise<string | null> {
+    if (!Number.isFinite(scale)) {
+      throw new Error(`viewport --scale: value must be a finite number, got ${scale}`);
+    }
+    if (scale < 1 || scale > 3) {
+      throw new Error(`viewport --scale: value must be between 1 and 3 (gstack policy cap), got ${scale}`);
+    }
+    if (this.connectionMode === 'headed') {
+      throw new Error('viewport --scale is not supported in headed mode — scale is controlled by the real browser window.');
+    }
+
+    const prevScale = this.deviceScaleFactor;
+    const prevViewport = { ...this.currentViewport };
+    this.deviceScaleFactor = scale;
+    this.currentViewport = { width, height };
+
+    const err = await this.recreateContext();
+    if (err !== null) {
+      // recreateContext's fallback path built a blank context using the NEW scale +
+      // viewport (the fields we just set). Rolling the fields back without a second
+      // recreate would leave the live context at new-scale while state says old-scale.
+      // Roll back fields FIRST, then force a second recreate against the old values
+      // so live state matches tracked state.
+      this.deviceScaleFactor = prevScale;
+      this.currentViewport = prevViewport;
+      const rollbackErr = await this.recreateContext();
+      if (rollbackErr !== null) {
+        // Second recreate also failed — we're in a clean blank slate via fallback, but
+        // with old scale. Return the original error so the caller sees the primary failure.
+        return `${err} (rollback also encountered: ${rollbackErr})`;
+      }
+      return err;
+    }
+    return null;
+  }
+
+  /** Read current deviceScaleFactor (for tests + debug). */
+  getDeviceScaleFactor(): number {
+    return this.deviceScaleFactor;
+  }
+
+  /** Read current tracked viewport (for tests + `viewport --scale` size fallback). */
+  getCurrentViewport(): { width: number; height: number } {
+    return { ...this.currentViewport };
+  }
+
  // ─── Handoff: Headless → Headed ─────────────────────────────
  /**
   * Hand off browser control to the user by relaunching in headed mode.