mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-09 14:38:59 +02:00
* fix(ci): free-tests lane actually runs the make-pdf e2e gates The 9 make-pdf/test/e2e gate tests probe make-pdf/dist/pdf, browse/dist/browse, and the diagram-render bundle, then self-skip when absent. The required free-tests lane never built any of them, so the gates silently skipped on Linux for their entire life (verified: 9 of 14 skip, exit 0). make-pdf-gate.yml's justification for deleting its Linux leg claimed the free lane covered this — it didn't. - new build:gates script: exactly the three artifacts the gates probe (full bun run build compiles five binaries; ~60-90s tax on the only required check is not warranted) - free-tests.yml: build:gates step + poppler-utils + fonts-noto-color-emoji (fonts must precede the first browse daemon launch — Chromium snapshots fontconfig at startup; verified live: a warm daemon renders tofu, a fresh one embeds NotoColorEmoji) - make-pdf/test/e2e/ci-prereqs.test.ts: GSTACK_EXPECT_BINARIES=1 (set by the workflow) inverts the skip polarity in CI — dropping the build step or poppler fails the lane instead of re-opening the silent-skip hole Pre-flight: all 9 gates green on Linux locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): kill the three zero-test eval jobs (hollow green) - delete the vestigial e2e-codex / e2e-gemini matrix rows: both files are whole-file periodic-tier, so with no row tier: they ran ZERO tests and reported green on every PR (~2 min of runner each, pure false confidence; the periodic lane owns those suites) - e2e-pty-plan-smoke gains tier: gate — its two files are whole-file describeE2ETier('gate'), so the job burned ~7 min of container setup then skipped every describe - KNOWN_TIER_UNSET burned down to empty; the ratchet stays armed so a future row/file tier mismatch fails the suite instead of shipping hollow green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): least-privilege permissions + fork-safe concurrency keys - evals.yml / evals-periodic.yml evals jobs: explicit contents:read + packages:read (container-image pull) and persist-credentials:false — the jobs that execute PR-authored code with three provider API keys ran on the repo-default token grant with the token written into .git/config - permissions blocks for the 4 workflows that had none (skill-docs, make-pdf-gate, windows-free-tests, windows-setup-e2e) - fork-safe concurrency keys: actionlint, skill-docs, make-pdf-gate, windows-setup-e2e switch from head_ref to PR-number keying — a bare branch name carries no fork prefix, so same-name branches from two forks shared one group and cancelled each other's runs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): one bun version everywhere + drift tripwire Lanes disagreed four ways: 1.3.13 (free-tests, windows, Dockerfile.ci), latest (quality-gate, make-pdf-gate), unpinned (skill-docs, version-gate — setup-bun installs latest), 1.3.10 (.gitlab-ci.yml). Different Bun versions change the runner output shapes the strict classifiers regex-match, spawn semantics, and shell parsing — a lane on a different Bun tests a different product; Dockerfile.ci's own comment records this class biting once already (silent 1.3.13/1.3.14 drift). All surfaces pinned to 1.3.13; test/bun-version-drift.test.ts scans every workflow setup-bun stanza + Dockerfile.ci + .gitlab-ci.yml and fails on any mismatch or unpinned stanza. skill-docs also gains --frozen-lockfile (was bare bun install). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(ci): bind the three-way image-tag hashFiles() expressions evals.yml, evals-periodic.yml, and ci-image.yml each compute the CI image tag from hashFiles('.github/docker/Dockerfile.ci', 'bun.lock', 'patches/**') — synced by comment only (TODOS.md 'CI three-way image-tag drift'). If one input list drifts, that workflow computes a different tag for the same content: eval lanes silently rebuild the image every run, or ci-image prebuilds a tag nobody looks up. The test extracts each tag-computation site and fails on any mismatch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): ci-image stops rebuilding the identical image every ship - package.json out of the trigger paths: the tag hash deliberately excludes it (version bumps every ship), so every merge rebuilt and re-pushed the IDENTICAL tag (~2m26s for zero content change); patches/** added (it IS a tag input) - manifest existence check (mirrors evals.yml): tag already exists → skip the build - concurrency group: two rapid main pushes raced pushing the same :latest/:buildcache tags - cron staggered 06:00→04:00 Monday: it shared the exact minute with evals-periodic, which could race a half-pushed tag or duplicate the build - timeout-minutes: 30 (was unbounded → 360-min default for a hung docker build) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): quality-gate drops the 74s full-history checkout fetch-depth:0 cost 74 of the job's 92 seconds; the three gates it feeds take ~12s combined. Shallow checkout + exact-SHA fetches for the diff's base/head (an exact-SHA fetch, not a guessed depth — long-lived branches and merge queues still resolve), with a --deepen fallback for push events whose 'before' is unusable. timeout right-sized 20→10 min. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): small-lane batch — timeouts, right-sizing, windows cache warm-start - timeout-minutes on the 6 remaining unbounded jobs (actionlint 5, skill-docs 10, version-gate 10, make-pdf-gate 15, pr-title-sync 5, evals build-image 15) — a hung step sat on GitHub's 360-min default - right-size measured-over-long timeouts: dependency-review 10→5, windows-setup-e2e 15→10 - dependency-review: 2-core runner (28s API call on an 8-core box) and drop .github/workflows/** from its trigger paths (workflow edits have no dependencies to review) - windows caches gain restore-keys: a lockfile bump paid the 26s/43s restore for a guaranteed cold miss Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): scope GSTACK_HOME to each file's execution window Five files assigned process.env.GSTACK_HOME at module scope. Shard processes evaluate sibling modules before running their tests, so the assignment leaked into every other file in the shard — the damage was already visible in defensive workarounds (relink.test.ts:28 'fresh install test saw a neighbor's skill_prefix'; cdp-e2e's own comment documents a sibling's temp dir baked into artifacts). Pattern: save original, assign in beforeAll, restore in afterAll (cdp-e2e already restored but still assigned at load — its window now matches the others). GSTACK_TELEMETRY_OFF and GSTACK_PROJECT_SLUG get the same treatment where they rode along. Victim files' defenses stay in place (cheap insurance). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: tripwire against module-scope GSTACK_HOME assignments Column-0 assignment of GSTACK_HOME / GSTACK_STATE_ROOT in any tracked *.test.ts fails with the file:line and the fix (beforeAll + afterAll restore). Kills the cross-file env-leak class the previous commit swept. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): e2e-harness-audit derives its skill census from disk The hand-maintained 39-name SKILL_GLOBS list had drifted to 39 of 54 SKILL.md.tmpl on disk. No live gap today (none of the 15 unlisted skills is interactive), but the next interactive skill would have landed unguarded with zero signal. The audit now walks top-level dirs for SKILL.md.tmpl (statSync so symlinked dirs like connect-chrome count), so new skills are in scope the commit they appear. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(evals): judges honor the eval-model resolution chain + real 429 backoff callJudge inlined GSTACK_EVAL_MODEL_JUDGE || sonnet, silently ignoring the global GSTACK_EVAL_MODEL override every other eval call site honors via lib/eval-model.ts. New 'judge' kind in DEFAULTS (sonnet — the D1a pin-on-regressors calibration stands; model CHOICE unchanged) and callJudge resolves through it: explicit arg > GSTACK_EVAL_MODEL_JUDGE > GSTACK_EVAL_MODEL > default. 429 handling upgraded from one fixed 1s retry (reliably lost races at CI concurrency) to three jittered exponential retries (~1s/4s/16s), honoring the server's retry-after when present. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): the two expect(true) paid stubs become test.todo skill-e2e-spec-execute (600s budget) and skill-llm-eval-spec (300s) reported PASS on every periodic run while asserting nothing. Deleting them would remove the periodic-tier selector surface they exist to register (diff-based selection for spec/ changes), so they become test.todo — reported as todo/skip, never pass — with the v1.1 implementation specs kept in-file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): reactivate 5 quarantined browse tests (2 security) extension-sender-auth's two privileged-message denial tests (content script + missing sender.url — the extension's security boundary) and snapshot's three skips were quarantined 'pre-existing' failures. Root cause: machine-local state on the quarantining dev machines — the test and gate code are byte-identical between the quarantining commit (410b4928) and HEAD, and all five pass deterministically on a clean checkout (68/68 across both files, multiple runs). No assertions weakened, no product changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(evals): activate the 4 paid test files that could never run anywhere carve-section-loading, codex-e2e-plan-format, codex-e2e-recommendation-substance, and llm-judge-recommendation gated on EVALS/tier (free suite loads them as describe.skip) but their names fell outside PAID_TEST_GLOBS, so no paid lane ever selected them — net execution zero, forever. The existing matrix tripwire filtered on isPaidTestFile() first, so it was blind to exactly this class (the same bug that hid the pre-split monolith's gate tests for ~8 releases). - PAID_TEST_GLOBS: codex-e2e* + skill-llm-eval* wildcards (replacing exact names) + llm-judge-recommendation + carve-section-loading; package.json's six test-script glob lists mirrored - codex-e2e-plan-format gains the explicit periodic tier gate its siblings carry (external-service rule) — without it the sharded runner's no-guard default would spawn Codex in the gate tier per PR - eval:bg:periodic --timeout 32400→37800: the census growth pushed the periodic worst case to 35910s; the old value had 270s of headroom BEFORE this change and would now kill healthy runs mid-flight - new test/paid-orphan-tripwire.test.ts: any EVALS/tier-gated test file outside the globs fails the free suite (reasoned SCANNER_EXEMPT for the gate helpers + meta-tests) — the class-killer - paid-shards pins updated: the four orphans now assert INSIDE the census Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(browse): restrictDirectoryPermissions warns and skips symlinked dirs Closes the Windows Free Tests red: recent lane failures showed a platform-unguarded POSIX mode-bit assertion ('Expected: 493' — a symlink-skip test) from PR-branch variants; the KNOWN_WINDOWS_SAFE force-include reason ('mode-bitmask hits are POSIX-branch only') did not hold for that shape, and main had neither the guard nor the behavior. - product: lstat first; a symlinked dir gets a warning and a skip on both platforms — chmod AND icacls dereference the link, so restricting through a symlink hardens an unvetted target (and /inheritance:r could lock out its real owner). All callers already treat hardening as best-effort (try/catch). - test: the symlink regression test, platform-aware — symlinkSync in the house try/catch skip pattern (Windows runners without Developer Mode can't create symlinks), mode-bit assertion guarded off win32, behavior assertions (no throw, warning text, target readable) everywhere; POSIX still proves the skip (0o755 unchanged, not 0o700) - KNOWN_WINDOWS_SAFE reason updated to the now-true premise 20/20 pass on Linux. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): unique tmp dirs for plan artifacts + audited live-repo cwd sites Six paid PTY tests wrote their expected plan artifact to a FIXED shared /tmp path ('/tmp/gstack-test-plan-<mode>.md') and rmSync'd it in finally — under --retry 1, EVALS_JOBS>1, or two concurrent worktrees, a sibling's cleanup deletes this run's artifact and the D19 'agent did not produce expected plan file' assertion fires spuriously. Each test now mkdtemps its own dir, interpolates the unique path into the agent prompt (fixture-sourced prompts get a replaceAll + drift guard that throws if the fixture's literal ever moves), and cleans up its own dir. The 18 cwd:-into-the-live-repo sites were audited: all deliberate (skill registry + hermetic pre-trusted dir, in-repo gen renders, git history reads, slug resolution) — each now carries a '// LIVE-REPO CWD: <reason>' comment so the next audit can tell deliberate from accidental. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): trim the seven over-wall 1700s timeouts to the 1500s physical ceiling 1,700,000ms (28.3 min) exceeded every wall these tests run inside: the 25-min CI job timeout and the 1800s sharded-runner wall (which also leaves --retry 1 zero room for a second attempt). Budget above the wall is fiction, not headroom — a test that actually used it produced a job-level kill (no bun summary, no artifact) instead of a clean per-test timeout. No recorded p95 exists for this family (they are being retiered to periodic in the re-platform wave); the trim stops at the physical ceiling rather than guessing lower. Final policy lands in the Wave-2 eval-budgets constants module. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(gen): main() guard — importing gen-skill-docs no longer regenerates the tree The generator's whole body executed at module load, so any import of it (test/gen-skill-docs.test.ts pulls assertSinglePreamble via require(); test/catalog-trim.test.ts imports helpers) regenerated all 71 SKILL.md in place — the root cause of half the TREE_MUTATING serial-shard entries (hazard class #2532). The body now lives in an exported main(): number behind if (import.meta.main). Semantics preserved exactly: failure exits are immediate (matching the old top-level process.exit), success leaves the event loop to drain so the llms.txt fire-and-forget IIFE finishes its write, and the module stays synchronous/require()-able. Proofs: byte-identical --host all output (git status clean), --dry-run stale-tree still exits 1 (the skill-docs freshness lane depends on it), and the new test/gen-skill-docs-import-purity.test.ts pins load-time purity via a subprocess probe (mtime-based, so a dirty worktree can't false-fail). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(gen): --out-dir renders every host, outputs-only --out-dir was Claude-host-only (gen-skill-docs.ts:842), which forced the codex/factory-regenerating tests (gen-skill-docs, skill-validation, host-config) to mutate the live tree — the reason they sit in the TREE_MUTATING serial shard. The flag now mirrors ALL outputs into the out-dir: external-host trees (.agents/.factory/... via processExternalHost), external section files, openclaw docs, and gstack/llms.txt (a catalog-mode render must never rewrite the tracked index). OUTPUTS ONLY — inputs (templates, sections/, host configs) are always read from ROOT, so an empty out-dir can never feed the render. rewriteSectionBase stays Claude-only (external hosts have their own path grammar). Proofs: in-place --host all is byte-identical (tree clean); --host all --out-dir <mkdtemp> renders the full multi-host tree with ROOT untouched; gen-skill-docs-out-dir tests + 415/415 gen-skill-docs.test.ts green (bin/dev-setup's claude rendering byte-compat). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(evals): every E2E key's dep list names its own declaring test file 129-of-177 keys omitted their own test file, so editing only a test's prompt or assertions selected NOTHING — the changed test never ran on the change that changed it. 135 keys self-registered (110 E2E + 25 LLM-judge), resolved by strict declaration evidence (testName:/ testIfSelected/judge call sites), with skill-name false positives excluded. e2e-tier-alignment's warn-only branch for unregistered files is now a hard failure with a 4-entry KNOWN_UNREGISTERED ratchet (template- literal testNames, fail-open-safe) + a burn-down test so the set only shrinks. Selection sanity: a one-file diff on skill-e2e-qa-workflow now selects its 4 tests (was 0); skill-llm-eval 0 → 25. Known follow-ups (filed): 15 E2E + 2 judge PHANTOM keys select tests that exist nowhere; codex-e2e-plan-format's testIfSelected names have no map keys (run-all only). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(evals): ratchet the 8 newly-visible gate-matrix gaps The self-registration sweep made these eight files' gate-tier keys visible to the census for the first time — their gate tests run in NO CI lane today (pre-existing hole, newly measurable). Ratcheted into KNOWN_MATRIX_GAPS with the burn-down note: the paid-lane re-platform runs every gate file by construction and retires this ratchet class. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(test): duration-aware LPT shard packing for the free suite Hash sharding balances file COUNTS (1.15x spread) but not cost — the Playwright-launching files landed 4/3/4/1/2/1 across 6 shards, giving a measured 28s–97s shard spread and ~40s of idle tail on every run. Full-suite mode now packs by recorded per-file durations (longest-processing-time-first) when the committed seed scripts/free-test-durations.json exists. - ONE store, no overlay: the seed is refreshed occasionally via the new --record-durations mode (each file timed in its own child — exact, and immune to bun's stream buffering, where silent passers print no header to timestamp); GSTACK_FREE_TEST_DURATIONS overrides the path for experiments; CI never records - seed is a hint: missing → silent hash-shard fallback; corrupt (bad merge) → one warning + fallback; unknown files → 75th-percentile pessimism so a surprise long-runner can't recreate the tail - packed shards get duration-aware walls (max(base, predicted x 3)) — LPT decouples count from cost BY DESIGN, so the 5s/file heuristic would undersize a shard holding few expensive files - one log line per shard (files + predicted seconds) so packing regressions are diagnosable from any run log - the --shard CI-matrix path is untouched: stable hash indices are its contract - successor note in-code: bun >=1.3.14 ships native --timings/--shard LPT — swap this packer when the repo unpins 1.3.13 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): decouple slop:diff from bun run test; quality-gate runs it per PR 'bun run test' silently appended up to two 120s npx slop-scan runs plus a git worktree add/remove after the suite (2>/dev/null || true) — invisible in the documented '~90-100s' timing and pure friction in the pre-commit loop. Decoupling is not coverage removal: quality-gate.yml now runs slop:diff on every PR (advisory, matching its in-repo 'never blocking' contract), and /review already invokes it explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(test): eval-budgets timeout tiers + fit/ceiling policy test Five named tiers (JUDGE 120s / CAPTURE 300s / CAPTURE_LONG 600s / PTY 900s / PTY_LONG 1200s) replace hand-ratcheted sprawl (46x300s, 46x120s, 44x360s, 44x180s, 27x240s, 19x150s, 13x420s, 12x600s...), much of it inflated to paper over the old 40-way in-shard concurrency that the sharded runner's 1-file-per-shard model kills. Policy test pins: every tier fits the shard wall minus 120s overhead (the structural fix for budgets-above-the-wall fiction), tiers stay ordered, and no paid literal exceeds PTY_LONG x1.25 — oversized tests get split, not budgeted past the wall. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(test): shared runBin helper for bin-script unit tests ~36 free test files each carry a near-identical local run() (spawnSync + utf-8 + {status, stdout, stderr}) differing only in env composition, cwd, and timeout. runBin absorbs the invariant core; options carry the variance (gstackHome sets BOTH GSTACK_HOME and GSTACK_STATE_DIR — the config-precedence trap several locals rediscovered independently; home for $HOME-anchored bins; input/trim/timeout/maxBuffer). Free-test-only by design so it never becomes a de facto global touchfile. Migration of the 36 call sites lands separately (mechanical batches). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): runBin trim assertion — trim shapes stream ends, not interior Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(test): mechanical sweep — 298 paid-test timeouts onto eval-budget tiers 69 files, both shapes (trailing bun-test budgets and runner timeout/timeoutMs options), ROUND-UP ONLY so nothing that passed can start failing: 75 → JUDGE_MS, 137 → CAPTURE_MS, 74 → CAPTURE_LONG_MS, 9 → PTY_MS, 3 → PTY_LONG_MS. Raw >=60s literal count in the paid scope: 395 → 97, of which 51 are non-timeout noise (fixture dates, run IDs) and 46 are enumerated justified holds (comment-carrying calibrated budgets, poll-loop constants, utility spawn waits, and the seven physical-ceiling 1_500_000 sites). The eval-budgets policy ratchet keeps the residue from regrowing. Known collapse: where an inner runner budget and its enclosing test budget now share a tier, the old stagger is gone — an overrun surfaces as a bun test timeout instead of a graceful runner timeout (diagnosability trade, not a correctness one). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: coverage fill — 95 tests for six zero-coverage surfaces - eval CLI family (eval-list/compare/summary + eval-select smoke): the primary interface to eval results had no tests; isolation via a fake gstack-slug under a mkdtemp HOME (the scripts' real resolution path — they do NOT honor GSTACK_EVAL_DIR; only EvalCollector does). Pinned current behavior: eval-list does NOT exclude _partial runs (documented improvement candidate) - slop-diff (runs on every /review + quality-gate): fixture git repo + first-on-PATH npx stub (never downloads real slop-scan); no-diff early exit, missing-scanner fallback, fingerprint line-insensitivity, merge-base worktree scan - bin/gstack-code-intelligence CLI arg surface (lib was covered, the 284-line CLI wasn't): select/consent/suggest/index/search gating; pinned: --help routes to usage failure exit 1 (no handler) - browse media-extract: the page.evaluate callback exercised in-process against a mock DOM (no exports added) — lazy-src fallback chain, HLS/DASH detection, bg-image url() parsing, 500-element cap - browse session-cookie-store: factory contract (cookieName/ttlMs/ maxSessions eviction, cross-store isolation, mint→validate round-trip); store is in-memory — no fs cases exist - lib/version-source direct unit tests (gstack-version-bump.test.ts spawns the bin, never imports the lib): parse/format/cmp/bump coercion, npm 4→3 translation, #2501 mangled-JSON regression class All hermetic (mkdtemp homes, runBin child isolation); windows curation correctly partitions the six. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(test): first runBin migration batch (3 of ~36 run() duplicates) explain-level-config, benchmark-cli, evidence move onto the shared helper; each file's remaining special-case spawnSync sites (raw-buffer probes, env-scrub probes) stay put deliberately. 55/55 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(evals): paid shards spool to disk + shared runShardChild lifecycle - runPaidShard no longer buffers whole 30-min stream-json streams in RAM (x concurrent jobs): every byte tees to a per-shard log file (slug-named, path printed at START for mid-run inspection and on the FAILED terminal line); failures print a 64KiB tail read back from disk; passing shards stay quiet (the file is the record) — the free runner's proven contract. Classification unchanged: the strict classifier still sees every byte first. - the ~35 duplicated spawn/group-kill/wall-timer/finally-reap lines move into runShardChild in test-strict-output.ts (detached-per- platform spawn, signal forwarding, SIGKILL group kill at the wall, drain-before-verdict); designed so the free runner can migrate later - expectedFiles drift fixed toward ENFORCEMENT: the injected-command exemption is gone — a fake command exiting 0 without bun's terminal summary now reads FAILED (pinned: silent-pass → failed) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(evals): parent-computed selection propagates to shard children The sharded runner computed diff selection once, then each of its 48-73 children recomputed it at module load — including, on touchfiles-diff branches, a per-child bun subprocess evaluating the old data file (20s timeout each). The parent now serializes {version, selected, reason} as EVALS_SELECTION_JSON into the shard env; e2e-helpers adopts it at load. Fail-open preserved: any parse/shape violation → ONE stderr warning + local recompute; absent env → silent local compute (non-sharded entrypoints unchanged). Drift test pins parent→child round-trip to identical selection decisions plus the malformed/absent cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): kill the four worst fixed sleeps (300s/30s/30s/20s) - watchdog.test: the 20s blind wait for one production parent-watchdog tick becomes BROWSE_PARENT_WATCHDOG_INTERVAL_MS=250 (new env knob in server.ts, NaN-safe, production default unchanged) + polls for the boot line and the tick's stay-alive log — strictly stronger (the old form never proved a tick observed the parent death). 24s → 3.6s. - stop-dead-daemon / terminal-agent-owner-watchdog: the 300s/30s stand-in child lifetimes become stdin-EOF-bound — the child can never self-exit mid-test on a slow runner (spurious-failure class) and self-reaps instantly if the test dies (no 300s orphans). Node-compat stdin APIs (owner-watchdog runs on the Windows lane). - browser-skill-commands: the sleeper fixture's 30s self-time becomes 8s (no stdin pipe exists in runToFiles) — far above the 1s product timeout it must outlive, below the test ceiling, so a timeout-kill regression fails on clean assertions instead of an opaque bun timeout; added: stdout must NOT contain 'done'. 45/45 green across the four files + server tripwires. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): gen-skill-docs + catalog-trim leave the serial mutator shard gen-skill-docs.test.ts's 15 in-place generator spawns now render into mkdtemp out-dirs (gitignored-artifact reads repointed; the handshake scan's silent console.warn degrade became a hard assertion); its tracked-tree reads (freshness dry-run, SKILL.md content pins) stay reads. catalog-trim needed no change beyond the earlier main() guard — its import is now side-effect-free (pinned by the import-purity test). Both TREE_MUTATING entries deleted in this commit, per the transition rule: an entry leaves in the same commit as the file's last in-place write. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): skill-validation renders codex host into an out-dir Its 3 in-place --host codex regeneration sites collapse into one module-level --out-dir render; assertions untouched. TREE_MUTATING entry deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): host-config self-provisions goldens (ordering dependency severed) Its goldens were 'produced by gen-skill-docs.test.ts' with a when-missing beforeAll fallback that wrote the live tree — an inter-test ordering dependency the serial shard hid. It now renders codex+factory UNCONDITIONALLY into its own out-dir and reads goldens only from there (the Claude golden deliberately keeps reading tracked ship/SKILL.md — a read; out-dir claude renders repoint section-base paths by design). TREE_MUTATING entry deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): gbrain-detection-override drops mutate-then-git-restore regenAndSnapshot renders --host claude --out-dir <mkdtemp> (+ --respect-detection) and snapshots probes from the out-dir. The git-restore machinery is deleted outright — it restored only PROBE_FILES of the 71 files each call wrote, so a stale tree kept the other 68 dirty (the partial-restore bug), and its 'no output-path arg' comment had been false since --out-dir landed. TREE_MUTATING entry deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): catalog-mode-full renders to out-dir; restore machinery deleted The full-catalog smoke no longer rewrites all 71 SKILL.md then regenerates to restore (with its 'CRITICAL: failed to restore' prayer path) — it renders into a mkdtemp and additionally asserts tracked ship/SKILL.md is byte-unchanged. TREE_MUTATING entry deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): idempotency proof strengthens to two-out-dir recursive diff Two renders into two separate out-dirs, EVERY file diffed byte-for-byte (claude-only and --host all; normalization only for each dir's own sanctioned section-base repoint; presence-sanity lists guard against a vacuous empty-dir pass) — strictly stronger than the old in-place double-regen that sampled 5 files. TREE_MUTATING entry deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): spec-template-sync compares an out-dir render, not an in-place one TREE_MUTATING entry deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(test): the serial tree-mutating shard dissolves — TREE_MUTATING is empty Zero mutators remain (all eight render into out-dirs now), so the four ratchet READERS (parity caps, size budgets, carve parity/ordering) get a quiet tree by construction in any shard and rejoin the parallel phase. The ~35-40s serial tail on every full-suite run is gone. The mechanism stays: a future test that genuinely must write shared artifacts in place earns an entry with a reason and is serialized again; the census pin still fails on renamed keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(gen): out-dir byte-identity + tree-clean pins for external hosts codex render: porcelain unchanged AND out-dir gstack-ship/SKILL.md byte-identical to a fresh in-place render (+openai.yaml presence); --host all render: exit 0, porcelain unchanged, claude + .agents + .factory + llms.txt + openclaw docs all present in the out-dir. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(test): commit the initial free-test durations seed (496 files) Recorded via --record-durations on a quiescent tree: 479s serial total, p50 92ms / p90 1.8s / max 31.4s — the top-heavy cost shape LPT packing exists for. A hint, not a contract: refresh opportunistically with bun run test:free --record-durations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(evals): planner/executor/report modes — the CI re-platform surface One PLANNER computes diff selection + the slice plan ONCE and writes a manifest (--emit-plan <path> --slices K); K executors consume it (--plan <path> --slice i), never self-selecting, and write slice-result artifacts; a REPORT reconciles results against the manifest (--report <dir>) fail-closed: a slice whose artifact never landed is a FAILURE, a planned shard nobody reported fails, wrong-slice/duplicate/cross-tier results fail. Kills per-slice selector divergence and hollow-lane aggregation at the root. - hollow-shard guard: under EVALS_ALL, exit 0 with ZERO executed tests (bun's 'Ran N tests' now captured by the classifier — additive) is 'passed-empty' and fails the run; selective runs keep it 'passed' with one warning (in-file diff/tier self-skips are legitimate there); unknown counts are never guessed hollow - retry parity: --retry 1 default + RETRY_OVERRIDES literals for the three files whose old matrix rows earned retries: 2 (stale entries pinned against disk) - live smoke: gate plan = 48 shards across 6 slices; report mode exits 1 on a fabricated missing slice, 0 when complete Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ci): sliced paid lane (planner -> 6 executors -> fail-closed report) The parity-phase re-platform: evals.yml gains a second, sliced lane driven by scripts/test-paid-shards.ts — the SAME engine local eval:bg:gate uses, so CI and local share one selection engine. - plan-slices: ONE planner (fetch-depth 0 — the only job needing history) emits the manifest; selection fails open to run-all, never per-slice (the divergence class is structurally dead) - eval-slices: 6-way matrix consuming the manifest; PTY seed + skill-registration steps run unconditionally (idempotent — a sliced lane cannot key them on suite names); aggregate spawn budget 6 x EVALS_JOBS=2 x EVALS_CONCURRENCY=2 = 24 lane-wide (the matrix's 40-way per row queued session startup behind 39 siblings — the timeout-flake family root); slice results + spooled shard logs uploaded as artifacts - slices-report: reconciles slice artifacts against the manifest FAIL-CLOSED via --report — a slice whose artifact never landed, or a planned shard nobody reported, is a failure, not an absence - sequenced needs: evals so provider concurrency never doubles while both lanes coexist; the matrix + its ratchets are deleted after demonstrated parity (intersection + expected-additions comparison) - workflow_dispatch gains evals_all (default true) for parity runs and post-merge smokes — a dispatch can never silently select zero Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ci): weekly periodic lane runs EVERY periodic test + gate census backstop evals-periodic.yml re-platforms onto the sharded runner: planner manifest → 6 executor slices → FAIL-CLOSED report. This IS the coverage contract: all ~70 periodic-tier files weekly (EVALS_ALL=1), killing the silent-rot class where a hard-coded 9-file matrix left ~57 files running NOWHERE (the autoplan E2E rotted invisibly for months). - test/helpers/periodic-exclude-data.ts: reasoned exclusions in their OWN literals file (deliberately not touchfiles-data — map-diff evaluates old versions of that file standalone). Every entry carries reason + tracking with a re-entry condition; the runner surfaces each exclusion per run; policy test pins real-file + non-empty fields. Initial: ship-idempotency + brain-privacy-gate (documented-red, never green) and skill-e2e-ios (manual hardware). The TODOS 'sidebar E2E trio' turned out already deleted — only tombstone tests remain. - gate-census job: weekly EVALS_ALL gate-tier run — PR lanes are diff-billed, so without this the full gate census might never execute anywhere; with the hollow-shard guard it is a census-health check (exit 0 + zero executed tests fails), not just a test run. - failure notification is a concrete gh issue UPSERT (one tracking issue, commented per red week — never issue-per-week spam), with issues:write scoped to the report job. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: TESTING_INTERNALS covers the 2026-08 runner overhaul LPT-packed free suite + --record-durations, the emptied TREE_MUTATING mechanism, the sharded paid runner as the single selection engine, CI planner/executor/report with the fail-closed report and hollow-shard guard, the weekly coverage contract + exclusions policy, and the eval-budgets timeout tiers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(CLAUDE.md): testing prose matches the overhauled runners - bun run test: duration-packed shards + --record-durations; the trailing serial tree-mutating shard no longer exists - two-tier system: the sliced CI lanes (one engine local+CI), the weekly all-periodic coverage contract + exclusions, the gate census - periodic detach timeout 32400 → 37800 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(TODOS): close the absorbed test-infra items, file the overhaul follow-ups Closed with receipts: the periodic coverage contract (implemented as full weekly coverage + exclusions), the eval-harness observability P1 (verified already landed: heartbeat, incremental _partial persistence, live stderr + eval-watch), and the sidebar trio (already deleted — tombstones remain). Filed: matrix deletion after parity, the required-check maintainer decision, browse /tmp-namespace hardening, PTY boot-readiness waits, the single typed test registry, bun-native LPT swap, runBin/free-runner migrations, eval-list partial exclusion, phantom key cleanup, duration-weighted slicing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * v1.73.0.0: test/CI overhaul — green means green, suites restructured for speed Version + release notes for the audit-and-overhaul branch: every silently-skipping or never-running test class fixed and tripwired, the free suite duration-packed with the serial mutator shard dissolved, the paid lane re-platformed onto the sharded runner (planner/slices/ fail-closed report, parity phase), the weekly all-periodic coverage contract, eval-budget timeout tiers, and 95 new coverage tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): first-live-run fixes — executor history + two environment-blind assertions The sliced lane's first run (PR #2721) did its job: the planner and report worked, the manifest governed, and every failure had a name. Three were fixable on the spot: - executor + gate-census checkouts get fetch-depth: 0 — files with SELF-derived selection (the LLM-judge map, routing) walk git at module load, and selection is deliberately fail-closed on git errors, so the shallow checkout crashed those shards ('ambiguous argument main...HEAD'). The manifest still governs WHICH shards run. - landscape --toc gate: the exact toBe(3) landscape-page count was font-metric-dependent (3 on Amazon Linux, 2 on ubuntu CI — the same disease the file's own page-index comment warns about). Now a comparative invariant: --toc must not CHANGE the landscape count vs a baseline render. - paid-run-manifest parse test builds its manifest under EVALS_ALL so it never walks git (proven with GIT_DIR=/nonexistent). Remaining first-run failures are newly-exposed rot in gate files that had never executed in CI (skillify D1 refusal, session-intelligence context-restore, one tpa-apple-ban retry flake) — being probed separately; they are the lane WORKING, not the lane failing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(TODOS): file the three first-execution findings from the sliced lane's live run Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * v1.74.0.0: queue-advance — #2722 claims the v1.73.0.0 slot The version gate caught a live queue collision (its whole job); same MINOR bump level, next free slot per bin/gstack-next-version. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): per-shard CHROMIUM_PROFILE — the collision class duration packing exposed Nine test files launch in-process persistent contexts or daemons that default to the SHARED ~/.gstack/chromium-profile. Two concurrent shard processes on one profile dir kill each other's browser — observed live on CI once duration packing recomposed shards: handoff's launchPersistentContext died 'Target page, context or browser has been closed' (--user-data-dir=~/.gstack/chromium-profile in the call log) while a sibling shard's daemon logged 'Chromium process crashed'. Hash sharding had masked the collision by chance placement; handoff passes standalone everywhere. Fix at the runner, not per file: each shard child gets CHROMIUM_PROFILE=<shard-state>/chromium-profile (the documented env knob, same isolation idea as the existing per-shard TMPDIR). Files within a shard run serially, so sharing the per-shard profile is safe; config.test's resolution-order tests save/restore the env around their assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): landscape --toc gate asserts promotion PRESENCE, not counts Two rounds of CI receipts: the exact toBe(3) was font-metric-coupled (3 on Amazon Linux, 2 on ubuntu), and the baseline-comparison repair then failed 2-vs-3 across renders SECONDS apart in one CI job while the sibling no-toc test saw 3 — per-render image-promotion timing makes any count assertion here a coin flip. The sibling test owns exact promotion counts; this test's actual invariant is that --toc does not break the promotion machinery: >=1 landscape page + the TOC rendered. Also drops the second render (halves the test's runtime). Flaky per-render image promotion itself is worth its own look — noted in TODOS with these receipts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(TODOS): file the per-render image-promotion nondeterminism (receipts from PR #2721) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): per-FILE Chromium profiles for the nine in-process launcher files Completes the profile-isolation work: the per-shard CHROMIUM_PROFILE stopped cross-shard kills; these nine files launch in-process persistent contexts and could still collide with a lingering daemon a sibling file spawned on the SAME shard profile. Each now scopes a mkdtemp profile via beforeAll/afterAll (the module-scope-tripwire-safe pattern), cleaned up per file. All nine green solo and in combined runs, except the pre-existing commands+snapshot pairing — proven identical WITH and WITHOUT these edits (baseline receipts) — which is the daemon-lifecycle follow-up now extended in TODOS with this session's receipts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(browse): Chromium-crash exit is daemon-only — embedded launches never kill their host handleChromiumDisconnect unconditionally process.exit()ed. Correct for the standalone daemon (its supervisor/user must notice); suicidal when a TEST launches BrowserManager in-process: a mid-suite Chromium death exited the whole bun shard with no terminal summary — the exact truncation class the strict runner flags (observed live: CI shard 1 oneb233299died at cache-concurrent-refresh right after a daemon-spawning gate test; with this fix the same pairing runs to completion and REPORTS instead of dying). The standalone entrypoint opts in via markDaemonProcess() under server.ts's import.meta.main gate — the same embedder contract its signal handlers already use (gbrowser phoenix keeps its own handlers). Embedded contexts now get the disconnect log line and continue. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): context-restore assertion is evidence-based, not prose-matching The test failed twice per run in TWO CI cycles while passing locally 4/4: the prompt said 'present the content' and the check grepped the FINAL message for exact phrases — local runs quoted the file, CI runs paraphrased ('the most recent context is from branch-b...') and the substring check lost the coin flip. - prompt now demands machine-checkable output: the newest file's '## Working on:' heading VERBATIM + a literal 'RESTORED: <filename>' marker (the mtime-scramble and cross-branch subject matter untouched) - assertion ordered strongest-first: RESTORED marker → legacy content phrases → tool-call corroboration (Read/Bash input naming the newer file, credited ONLY when the older file was never read — a both-files run must still present the right one) - the older-file negative got STRONGER: an explicit RESTORED marker naming the older file fails even if wintermute words appear elsewhere - sibling scan: context-recovery-artifacts got the additive prompt-side treatment only (quote the matched literals verbatim); its lenient 1-of-6 assertion deliberately unchanged 3/3 consecutive local green with all evidence classes firing (marker=true, content=true, toolNewer=true, toolOlder=false). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): skillify family — HOME==cwd broke project-skill registration Root cause (forensically pinned from stream-json init events + a kill-after-init probe): with HOME set EQUAL to the child's cwd, claude resolves <cwd>/.claude/skills as the PERSONAL skills directory and the seeded project-tier skills never register — the Skill tool returned 'Unknown skill'. The provenance-refusal test then improvised a refusal whose wording missed the regex (the deterministic CI+local red); the happy-path and approval-reject siblings passed only because their agents self-recovered by Reading SKILL.md manually — silently not exercising the Skill-tool path at all. All three tests now use HOME=<workDir>/home (a fresh subdir keeps the override's intent: child ~/.gstack writes land in the assertable sandbox, without the cwd collision). Refusal test additionally: a 'not registered/unknown skill' tripwire (a not-loaded skill can never pass as a refusal) and the refusal regex now matches assistant text only — the skill BODY echoed into the transcript contains the exact refusal message, so the old full-surface match could pass vacuously once the skill loaded. Sibling disk assertions sweep both $HOME/.gstack and cwd .gstack roots (positives and negatives). Verified paid: refusal 2x consecutive green with the skill's EXACT message rendered ('Launching skill: skillify' in-transcript), then the full file 5/5 green (~$1.35) with both siblings driving real Skill calls (25-27 turns each). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(TODOS): two of three first-execution findings fixed (skillify family, context-restore) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): context-restore gets a private home — the REAL root cause was fixture sharing The evidence-based assertion fix was treating a symptom. The slice artifact's embedded transcript showed the CI agent restoring 20260829-context-save-skill-test.md — the checkpoint the SIBLING context-save test wrote into the SHARED gstackHome checkpoints dir, which by filename-prefix ordering genuinely IS the newest. The agent behaved CORRECTLY; the test's fixture set was open to concurrent sibling writes, and bun --concurrent ordering differs between CI (save finished first) and local (restore listed first) — the entire local-green/CI-red split explained. The restore test now uses its own .gstack-restore-home (the whole home moves, not just the handed path — an agent deriving the dir from GSTACK_HOME/projects/<slug> must land in the closed set too). Full file 4/4 paid green with all evidence flags firing. Also: the on-failure shard-log artifact glob uploaded nothing — the Fix-bun-temp step points TMPDIR at /home/runner/.cache, so the spool lands there, not /tmp. Both eval workflows now glob both locations (this gap is why diagnosing THIS failure required digging transcripts out of the slice-results artifact). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(evidence): carry the real index mtime onto gstack-wtree's temp copy The stat-cache seed (cp of the real index) stamped the temp index "now", which defeats git's racy-git protection: an entry is only re-hashed when its cached mtime is not older than the index file itself, so a same-size rewrite landing in the same second as the last real index write looked non-racy, kept its stale stat-cache entry, and vanished from the fingerprint — evidence stayed FRESH after a source change. This is the CI flake in test/evidence.test.ts "allow-paths carve-out" (sub-second alignment on fast runners: expected STALE exit 1, got FRESH exit 0). touch -r restores the original index timestamp, reinstating the exact racy window git itself uses. Deterministic regression pin in test/review-log.test.ts reproduces the miss with pinned zero-nsec timestamps (fails on the old script, passes now); receipts: manual probe shows the fresh-stamped copy returning the clean tree for a same-size 'hello'→'howdy' rewrite while the mtime-carried copy detects it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): landscape gate bounds the promotion count instead of pinning 3 The alt-hinted image promotion rides the per-render measurement race already filed in TODOS (2-vs-3 landscape pages on renders seconds apart — CI receipts from PR #2721, now reproduced locally). Pin the two deterministic promotions as the floor and the three promotable blocks as the ceiling (anything above 3 means the veto leaked); the veto/portrait assertions remain exact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Test <test@test.com>
3372 lines
156 KiB
TypeScript
3372 lines
156 KiB
TypeScript
/**
|
||
* gstack browse server — persistent Chromium daemon
|
||
*
|
||
* Architecture:
|
||
* Bun.serve HTTP on localhost → routes commands to Playwright
|
||
* Console/network/dialog buffers: CircularBuffer in-memory + async disk flush
|
||
* Chromium crash → server EXITS with clear error (CLI auto-restarts)
|
||
* Auto-shutdown after BROWSE_IDLE_TIMEOUT (default 30 min)
|
||
*
|
||
* State:
|
||
* State file: <project-root>/.gstack/browse.json (set via BROWSE_STATE_FILE env)
|
||
* Log files: <project-root>/.gstack/browse-{console,network,dialog}.log
|
||
* Port: random 10000-60000 (or BROWSE_PORT env for debug override)
|
||
*/
|
||
|
||
import { BrowserManager, markDaemonProcess } from './browser-manager';
|
||
import { handleReadCommand, hasOutArg } from './read-commands';
|
||
import { handleWriteCommand } from './write-commands';
|
||
import { handleMetaCommand } from './meta-commands';
|
||
import { handleCookiePickerRoute, hasActivePicker } from './cookie-picker-routes';
|
||
import { COMMAND_DESCRIPTIONS, PAGE_CONTENT_COMMANDS, DOM_CONTENT_COMMANDS, wrapUntrustedContent, canonicalizeCommand, buildUnknownCommandError, ALL_COMMANDS } from './commands';
|
||
import {
|
||
wrapUntrustedPageContent, datamarkContent,
|
||
runContentFilters, type ContentFilterResult,
|
||
markHiddenElements, getCleanTextWithStripping, cleanupHiddenMarkers,
|
||
} from './content-security';
|
||
import { isSidecarAvailable, scanWithSidecar } from './security-sidecar-client';
|
||
import { writeSecureFile, mkdirSecure, appendSecureFile } from './file-permissions';
|
||
import { handleSnapshot, SNAPSHOT_FLAGS } from './snapshot';
|
||
import {
|
||
initRegistry, validateToken as validateScopedToken, checkScope, checkDomain,
|
||
checkRate, createToken, createSetupKey, exchangeSetupKey, revokeToken,
|
||
listTokens, recordCommand,
|
||
isRootToken, checkConnectRateLimit, type TokenInfo, type ScopeCategory,
|
||
DEFAULT_PAIR_SCOPES, InvalidScopeError, ReservedClientIdError, assertValidClientId,
|
||
assertValidTokenOptions, revokeSetupKeys, getClientSession, grantReducesAccess,
|
||
} from './token-registry';
|
||
import { validateTempPath } from './path-security';
|
||
import { resolveConfig, ensureStateDir, readVersionHash, resolveChromiumProfile, cleanSingletonLocks, isPairAgentEnabled } from './config';
|
||
import {
|
||
isSessionPersistEnabled, persistSessionState, restoreSessionState,
|
||
sessionPersistIntervalMs, SESSION_STATE_FILE,
|
||
} from './session-persist';
|
||
import { emitActivity, subscribe, getActivityAfter, getActivityHistory, getSubscriberCount } from './activity';
|
||
import { createSseEndpoint } from './sse-helpers';
|
||
import { initAuditLog, writeAuditEntry } from './audit';
|
||
import { inspectElement, modifyStyle, resetModifications, getModificationHistory, detachSession, type InspectorResult } from './cdp-inspector';
|
||
// Bun.spawn used instead of child_process.spawn (compiled bun binaries
|
||
// fail posix_spawn on all executables including /bin/bash)
|
||
import { safeUnlink, safeUnlinkQuiet, safeKill } from './error-handling';
|
||
import {
|
||
findAvailablePort, formatExplicitPortUnavailableError, formatRandomPortUnavailableError,
|
||
} from './port-allocator';
|
||
import { readAgentRecord, killAgentByRecord, agentRecordPath, spawnTerminalAgent } from './terminal-agent-control';
|
||
import { isProcessAlive } from './error-handling';
|
||
import { sanitizeBody, stripLoneSurrogateEscapes, stripLoneSurrogates, sanitizeReplacer } from './sanitize';
|
||
import { startSocksBridge, testUpstream, type BridgeHandle } from './socks-bridge';
|
||
import { parseProxyConfig, toUpstreamConfig, ProxyConfigError } from './proxy-config';
|
||
import { writeReceipt } from '../../lib/egress-receipt';
|
||
import { redactProxyUrl } from './proxy-redact';
|
||
import { shouldSpawnXvfb, pickFreeDisplay, spawnXvfb, xvfbInstallHint, type XvfbHandle } from './xvfb';
|
||
import { logTunnelDenial } from './tunnel-denial-log';
|
||
import {
|
||
mintSseSessionToken, validateSseSessionToken, extractSseCookie,
|
||
buildSseSetCookie, SSE_COOKIE_NAME,
|
||
} from './sse-session-cookie';
|
||
import {
|
||
mintPtySessionToken, buildPtySetCookie, revokePtySessionToken,
|
||
} from './pty-session-cookie';
|
||
import {
|
||
mintLease, validateLease, refreshLease, revokeLease,
|
||
} from './pty-session-lease';
|
||
import * as fs from 'fs';
|
||
import * as net from 'net';
|
||
import * as path from 'path';
|
||
import * as crypto from 'crypto';
|
||
|
||
// ─── Unicode Sanitization ───────────────────────────────────────
|
||
// Unpaired UTF-16 surrogate halves (\uD800–\uDFFF) in page DOM text, OCR
|
||
// output, and other CDP-sourced strings are rejected by JSON consumers
|
||
// downstream (Anthropic API in particular: "no low surrogate in string").
|
||
// The sanitizers live in sanitize.ts (single source of truth, shared with
|
||
// sse-helpers.ts and the read/snapshot pipeline): `stripLoneSurrogates`
|
||
// replaces lone halves with U+FFFD (valid pairs like emoji survive), and
|
||
// `sanitizeReplacer` runs it on every string value inside JSON.stringify.
|
||
//
|
||
// INVARIANT: every server egress path that ships page-content strings MUST
|
||
// route through the sanitizer. handleCommandInternal wraps the final
|
||
// cr.result string (text/plain bodies carry lone surrogates verbatim;
|
||
// JSON.stringify already escapes them). The SSE producers stringify with
|
||
// `sanitizeReplacer` so payload string fields get cleaned BEFORE escaping.
|
||
// Plain post-stringify regex is a no-op there because JSON.stringify
|
||
// converts \uD800 → "\\ud800" — the regex can't see the surrogate after
|
||
// that point.
|
||
|
||
// ─── Config ─────────────────────────────────────────────────────
|
||
const config = resolveConfig();
|
||
ensureStateDir(config);
|
||
initAuditLog(config.auditLog);
|
||
|
||
// ─── Auth ───────────────────────────────────────────────────────
|
||
// activeShutdown points to the factory-scoped shutdown function once
|
||
// buildFetchHandler has been called. Module-level timers (idle check, parent
|
||
// watchdog) and signal handlers route through activeShutdown so they close
|
||
// the cfg-provided browserManager rather than a stale module-level reference.
|
||
// Null before the first buildFetchHandler call, which is correct: nothing to
|
||
// shut down yet.
|
||
let activeShutdown: ((code?: number) => Promise<void>) | null = null;
|
||
|
||
// AUTH_TOKEN is injectable via process.env.AUTH_TOKEN so embedders
|
||
// (gbrowser's gbd daemon spawn) can pre-allocate the token and hand it to
|
||
// the Bun child via env.
|
||
//
|
||
// Validation: require >= 16 chars after stripping ALL unicode whitespace
|
||
// (not just ASCII — .trim() misses U+200B / U+FEFF / U+00A0 / etc., which
|
||
// would otherwise let a misconfigured embedder ship a one-character BOM as
|
||
// the bearer secret). Reject tokens that are too short or contain only
|
||
// whitespace; fall back to randomUUID so the security boundary is never
|
||
// silently weakened by misconfiguration.
|
||
function sanitizeAuthToken(raw: string | undefined): string | null {
|
||
if (!raw) return null;
|
||
const stripped = raw.replace(/[\s -]/g, '');
|
||
if (stripped.length < 16) return null;
|
||
return stripped;
|
||
}
|
||
// AUTH_TOKEN const + module-level initRegistry call deleted in v1.35.0.0.
|
||
// buildFetchHandler now owns auth state end-to-end: cfg.authToken is the
|
||
// single source of truth, factory body calls initRegistry(cfg.authToken),
|
||
// and factory-scoped validateAuth closes over the same value. start() reads
|
||
// env once via resolveConfigFromEnv() and threads the result through.
|
||
const BROWSE_PORT = parseInt(process.env.BROWSE_PORT || '0', 10);
|
||
const IDLE_TIMEOUT_MS = parseInt(process.env.BROWSE_IDLE_TIMEOUT || '1800000', 10); // 30 min
|
||
|
||
/**
|
||
* Port the local listener bound to. Set once the daemon picks a port.
|
||
* Used by `$B skill run` to point spawned skill scripts at the daemon over
|
||
* loopback. Module-level so handleCommandInternal can read it without threading
|
||
* the port through every dispatch.
|
||
*/
|
||
let LOCAL_LISTEN_PORT: number = 0;
|
||
// Sidebar chat is always enabled in headed mode (ungated in v0.12.0)
|
||
|
||
// ─── Tunnel State ───────────────────────────────────────────────
|
||
//
|
||
// Dual-listener architecture: the daemon binds TWO HTTP listeners when a
|
||
// tunnel is active. The local listener serves bootstrap + CLI + sidebar
|
||
// (never exposed to ngrok). The tunnel listener serves only the pairing
|
||
// ceremony and scoped-token command endpoints (the ONLY port ngrok forwards).
|
||
//
|
||
// Security property comes from physical port separation: a tunnel caller
|
||
// cannot reach bootstrap endpoints because they live on a different TCP
|
||
// socket, not because of any per-request check.
|
||
let tunnelActive = false;
|
||
let tunnelUrl: string | null = null;
|
||
let tunnelListener: any = null; // ngrok listener handle
|
||
let tunnelServer: ReturnType<typeof Bun.serve> | null = null; // tunnel HTTP listener
|
||
|
||
/** Which HTTP listener accepted this request. */
|
||
export type Surface = 'local' | 'tunnel';
|
||
|
||
/**
|
||
* Factory contract for embedders (gbrowser phoenix overlay).
|
||
*
|
||
* Today the CLI calls `start()` which reads env vars and binds Bun.serve
|
||
* itself. Embedders building on this server as a submodule (gbrowser's
|
||
* fd-passing gbd architecture) need to inject auth + ports + a
|
||
* BrowserManager they pre-launched, and own the listener themselves.
|
||
*
|
||
* Status: v1 surfaces this type as documentation. AUTH_TOKEN env-injection
|
||
* is already live (see ~L70). `start()` is exported and the kickoff /
|
||
* signal-handler registration is gated on `import.meta.main`, so phoenix
|
||
* can `import { start } from '.../server'` without auto-starting. Full
|
||
* `buildFetchHandler` extraction lands in a follow-up; see plan
|
||
* `/Users/garrytan/.claude/plans/system-instruction-you-are-working-swirling-fountain.md`
|
||
* Part 1.
|
||
*/
|
||
export interface ServerConfig {
|
||
/** Bearer token clients must present. Today injected via AUTH_TOKEN env. */
|
||
authToken: string;
|
||
/** Local listener port. Used in /welcome URL + state-file. */
|
||
browsePort: number;
|
||
/** Result of resolveConfig() — stateDir, auditLog, stateFile. */
|
||
config: ReturnType<typeof resolveConfig>;
|
||
/** Pre-launched BrowserManager. Caller owns lifecycle. */
|
||
browserManager: BrowserManager;
|
||
// NOTE: per-factory idleTimeoutMs and chromiumProfile were deleted — they
|
||
// were documented but never read (the idle timer, activity state, and
|
||
// shutdown target are module-global, so per-factory wiring would lie for
|
||
// any embedder running >1 handler). Real support belongs to the deferred
|
||
// server.ts singleton/route-table refactor. Until then: BROWSE_IDLE_TIMEOUT
|
||
// and CHROMIUM_PROFILE env are the honest knobs.
|
||
/** Caller-owned. shutdown() does NOT call xvfb.stop(); caller is responsible. */
|
||
xvfb?: XvfbHandle | null;
|
||
/** Caller-owned. shutdown() does NOT call proxyBridge.close(); caller is responsible. */
|
||
proxyBridge?: BridgeHandle | null;
|
||
startTime: number;
|
||
/**
|
||
* Overlay hook. Runs AFTER gstack resolves auth and BEFORE route dispatch.
|
||
* Invalid tokens are auto-rejected at the gstack layer (401 returned
|
||
* before hook fires), so the hook only ever sees valid TokenInfo or null
|
||
* (no token presented). Returning a Response short-circuits gstack
|
||
* dispatch; returning null falls through.
|
||
*/
|
||
beforeRoute?: (req: Request, surface: Surface, auth: TokenInfo | null) => Promise<Response | null>;
|
||
/**
|
||
* Whether gstack owns the lifecycle of the terminal-agent process and its
|
||
* discovery files (`<stateDir>/terminal-port`, `<stateDir>/terminal-internal-token`,
|
||
* `<stateDir>/terminal-agent-pid`).
|
||
*
|
||
* When true (default), shutdown() runs four side effects:
|
||
* 1. Identity-based kill via `killAgentByRecord(readAgentRecord(stateDir))`
|
||
* (v1.44+). Only signals the PID recorded by THIS daemon's agent.
|
||
* Replaced the historical `pkill -f terminal-agent\.ts` regex that
|
||
* matched sibling gstack sessions on the same host — see
|
||
* terminal-agent-control.ts for rationale.
|
||
* 2. `safeUnlinkQuiet(<stateDir>/terminal-port)`
|
||
* 3. `safeUnlinkQuiet(<stateDir>/terminal-internal-token)`
|
||
* 4. `safeUnlinkQuiet(<stateDir>/terminal-agent-pid)` (the v1.44 record)
|
||
*
|
||
* This is correct for gstack's CLI path, which spawns `terminal-agent.ts` as
|
||
* the producer of those files (see cli.ts:1037-1063).
|
||
*
|
||
* Embedders (gbrowser phoenix overlay, future hosts) that run their own PTY
|
||
* server and write those files themselves should pass `false`. When `false`,
|
||
* the embedder owns BOTH the agent process AND all three discovery files.
|
||
* Note that terminal-agent.ts's own SIGTERM cleanup removes `terminal-port`
|
||
* and `terminal-agent-pid` (the agent writes both at boot), so embedders
|
||
* that pre-launch their own agent must ensure their cleanup matches.
|
||
*
|
||
* Polarity note: this differs from `xvfb?` and `proxyBridge?`, which gate by
|
||
* the *presence* of a caller-owned handle (presence ⇒ don't close). This
|
||
* field gates by an explicit boolean because there is no handle object —
|
||
* the terminal-agent is started elsewhere (cli.ts), and shutdown's only
|
||
* reference is the PID record + the file paths.
|
||
*/
|
||
ownsTerminalAgent?: boolean;
|
||
}
|
||
|
||
/**
|
||
* Return shape of buildFetchHandler() — fetch handlers + lifecycle helpers
|
||
* embedders need to drive their own Bun.serve binding. See ServerConfig.
|
||
*/
|
||
export interface ServerHandle {
|
||
fetchLocal: (req: Request, server: any) => Promise<Response>;
|
||
fetchTunnel: (req: Request, server: any) => Promise<Response>;
|
||
/**
|
||
* Drains buffers, kills terminal-agent, closes browser, clears intervals,
|
||
* removes state files. Does NOT stop bound Bun.Server listeners — call
|
||
* stopListeners() for that. CLI relies on process.exit() to drop sockets.
|
||
*/
|
||
shutdown: (exitCode?: number) => Promise<void>;
|
||
/**
|
||
* Graceful listener stop for embedders. Calls server.stop(true) on each
|
||
* passed Bun.Server. CLI doesn't need this (process.exit handles it).
|
||
*/
|
||
stopListeners: (local: any, tunnel?: any) => Promise<void>;
|
||
}
|
||
|
||
/**
|
||
* Build a ServerConfig-shaped object from process.env. Used by gstack's
|
||
* own CLI when running `bun run dev` or the compiled binary directly.
|
||
* Embedders construct their own ServerConfig explicitly.
|
||
*
|
||
* Reads env, calls resolveConfig(). Does NOT bind a listener or call
|
||
* initAuditLog/initRegistry — those happen inside the buildFetchHandler
|
||
* lifecycle.
|
||
*/
|
||
export function resolveConfigFromEnv(): Omit<ServerConfig, 'browserManager' | 'startTime'> & {
|
||
config: ReturnType<typeof resolveConfig>;
|
||
} {
|
||
return {
|
||
// Same sanitizer as the module-level AUTH_TOKEN: strips ALL unicode
|
||
// whitespace and rejects tokens shorter than 16 chars so a misconfigured
|
||
// embedder can't ship a BOM/zero-width as the bearer secret.
|
||
authToken: sanitizeAuthToken(process.env.AUTH_TOKEN) || crypto.randomUUID(),
|
||
browsePort: parseInt(process.env.BROWSE_PORT || '0', 10),
|
||
config: resolveConfig(),
|
||
};
|
||
}
|
||
|
||
/**
|
||
* Paths reachable over the tunnel surface. Everything else returns 404.
|
||
*
|
||
* `/connect` is the only unauthenticated tunnel endpoint — POST for setup-key
|
||
* exchange, GET for an `{alive: true}` probe used by /pair and /tunnel/start
|
||
* to detect dead ngrok tunnels. Other paths in this set require a scoped
|
||
* token via Authorization: Bearer.
|
||
*
|
||
* Updating this set is a deliberate security decision. Every addition widens
|
||
* the tunnel attack surface.
|
||
*/
|
||
const TUNNEL_PATHS = new Set<string>([
|
||
'/connect',
|
||
'/command',
|
||
]);
|
||
|
||
/**
|
||
* The gstack sidebar extension's pinned Chrome extension ID. Derived from
|
||
* the "key" field in extension/manifest.json (first 16 bytes of SHA-256 of
|
||
* the DER public key, hex nibbles mapped 0-9a-f → a-p). Reproduce with:
|
||
* bun browse/scripts/extension-id.ts
|
||
* POST /extension-token releases AUTH_TOKEN only to an Origin of exactly
|
||
* `chrome-extension://<this id>`. If the manifest keypair is ever rotated,
|
||
* this constant must be updated in the same commit.
|
||
*/
|
||
export const GSTACK_EXTENSION_ID = 'dgbkdbjebeiblbajiilljmhjdpmiglep';
|
||
|
||
/**
|
||
* Commands reachable via POST /command over the tunnel surface. A paired
|
||
* remote agent can drive the browser (goto, click, text, etc.) but cannot
|
||
* configure the daemon, bootstrap new sessions, import cookies, or reach
|
||
* extension-inspector state. This allowlist maps to the eng-review decision
|
||
* logged in the CEO plan for sec-wave v1.6.0.0.
|
||
*/
|
||
export const TUNNEL_COMMANDS = new Set<string>([
|
||
// Original 17
|
||
'goto', 'click', 'text', 'screenshot',
|
||
'html', 'links', 'forms', 'accessibility',
|
||
'attrs', 'media', 'data',
|
||
'scroll', 'press', 'type', 'select', 'wait', 'eval',
|
||
// Tab + navigation primitives operator docs and CLI hints already promised
|
||
'newtab', 'tabs', 'back', 'forward', 'reload',
|
||
// Read/inspect/write operators paired agents need to be useful
|
||
'snapshot', 'fill', 'url', 'closetab',
|
||
]);
|
||
|
||
/**
|
||
* Pure gate: returns true iff the command is reachable over the tunnel surface.
|
||
* Extracted from the inline /command handler so the gate logic is unit-testable
|
||
* without standing up an HTTP listener. Behavior is identical to the inline
|
||
* check; the function canonicalizes the command (so aliases hit the same set)
|
||
* and returns false for null/undefined input.
|
||
*
|
||
* `args` is consulted so an `--out` invocation (e.g. `eval --out <file>`) is
|
||
* NEVER tunnel-dispatchable: `--out` turns an otherwise-readable command into a
|
||
* local-disk WRITE, and the tunnel surface never grants disk-write capability to
|
||
* remote paired agents. Omitting `args` preserves the old command-only behavior.
|
||
*/
|
||
export function canDispatchOverTunnel(command: string | undefined | null, args?: string[]): boolean {
|
||
if (typeof command !== 'string' || command.length === 0) return false;
|
||
if (Array.isArray(args) && hasOutArg(args)) return false;
|
||
const cmd = canonicalizeCommand(command);
|
||
return TUNNEL_COMMANDS.has(cmd);
|
||
}
|
||
|
||
/**
|
||
* Read ngrok authtoken from env var, ~/.gstack/ngrok.env, or ngrok's native
|
||
* config files. Returns null if nothing found. Shared between the
|
||
* /tunnel/start handler and the BROWSE_TUNNEL=1 auto-start flow.
|
||
*/
|
||
function resolveNgrokAuthtoken(): string | null {
|
||
let authtoken = process.env.NGROK_AUTHTOKEN;
|
||
if (authtoken) return authtoken;
|
||
|
||
const home = process.env.HOME || '';
|
||
const ngrokEnvPath = path.join(home, '.gstack', 'ngrok.env');
|
||
if (fs.existsSync(ngrokEnvPath)) {
|
||
try {
|
||
const envContent = fs.readFileSync(ngrokEnvPath, 'utf-8');
|
||
const match = envContent.match(/^NGROK_AUTHTOKEN=(.+)$/m);
|
||
if (match) return match[1].trim();
|
||
} catch {}
|
||
}
|
||
|
||
const ngrokConfigs = [
|
||
path.join(home, 'Library', 'Application Support', 'ngrok', 'ngrok.yml'),
|
||
path.join(home, '.config', 'ngrok', 'ngrok.yml'),
|
||
path.join(home, '.ngrok2', 'ngrok.yml'),
|
||
];
|
||
for (const conf of ngrokConfigs) {
|
||
try {
|
||
const content = fs.readFileSync(conf, 'utf-8');
|
||
const match = content.match(/authtoken:\s*(.+)/);
|
||
if (match) return match[1].trim();
|
||
} catch {}
|
||
}
|
||
return null;
|
||
}
|
||
|
||
/**
|
||
* Tear down the tunnel: close the ngrok listener and stop the tunnel-surface
|
||
* Bun.serve listener. Safe to call with nothing running. Always clears
|
||
* tunnel state regardless of individual close failures.
|
||
*/
|
||
async function closeTunnel(): Promise<void> {
|
||
try { if (tunnelListener) await tunnelListener.close(); } catch {}
|
||
try { if (tunnelServer) tunnelServer.stop(true); } catch {}
|
||
tunnelListener = null;
|
||
tunnelServer = null;
|
||
tunnelUrl = null;
|
||
tunnelActive = false;
|
||
}
|
||
|
||
/**
|
||
* Result of startTunnel(). `stage` tells the caller which half failed so it
|
||
* can keep its distinct error surface: 'bind' = the tunnel-surface Bun.serve
|
||
* listener could not bind (nothing to clean up), 'ngrok' = anything after the
|
||
* bind (ngrok forward, egress receipt, state-file write) — startTunnel has
|
||
* already torn down both ngrok and the Bun listener by the time it returns.
|
||
*/
|
||
type StartTunnelResult =
|
||
| { ok: true; url: string }
|
||
| { ok: false; stage: 'bind' | 'ngrok'; error: Error };
|
||
|
||
/**
|
||
* Start the ngrok tunnel using the dual-listener pattern: bind a dedicated
|
||
* tunnel-surface listener on an ephemeral 127.0.0.1 port and point
|
||
* ngrok.forward() at THAT port — the local listener (which serves
|
||
* /extension-token, /cookie-picker, /inspector/*, welcome, etc.) is never
|
||
* exposed to ngrok. Shared by the /tunnel/start route handler (which passes
|
||
* its in-closure makeFetchHandler('tunnel')) and the BROWSE_TUNNEL=1
|
||
* auto-start flow in start() (which passes handle.fetchTunnel from the
|
||
* factory). The BROWSE_TUNNEL_LOCAL_ONLY=1 test path does NOT use this
|
||
* helper — it binds the tunnel surface with no ngrok forwarding at all.
|
||
*
|
||
* Hard fail on listener bind (`stage: 'bind'`) — NEVER fall back to the
|
||
* local port, which would silently defeat the whole security property.
|
||
*
|
||
* On success, sets the module tunnel state (tunnelListener / tunnelUrl /
|
||
* tunnelServer / tunnelActive) and records the tunnel in the state file.
|
||
*/
|
||
async function startTunnel(opts: {
|
||
fetchHandler: (req: Request, server: any) => Promise<Response>;
|
||
authtoken: string;
|
||
consent: string;
|
||
}): Promise<StartTunnelResult> {
|
||
// Bind the tunnel listener on an ephemeral port. HARD FAIL if this
|
||
// errors — never fall back to the local port.
|
||
let boundTunnel: ReturnType<typeof Bun.serve>;
|
||
try {
|
||
boundTunnel = Bun.serve({
|
||
port: 0,
|
||
hostname: '127.0.0.1',
|
||
fetch: opts.fetchHandler,
|
||
});
|
||
} catch (err: any) {
|
||
return { ok: false, stage: 'bind', error: err };
|
||
}
|
||
const tunnelPort = boundTunnel.port;
|
||
|
||
// Point ngrok at the TUNNEL port (not the local port). If this fails,
|
||
// tear the listener back down so we don't leak sockets.
|
||
try {
|
||
const ngrok = await import('@ngrok/ngrok');
|
||
const domain = process.env.NGROK_DOMAIN;
|
||
const forwardOpts: any = { addr: tunnelPort, authtoken: opts.authtoken };
|
||
if (domain) forwardOpts.domain = domain;
|
||
|
||
// Egress receipt BEFORE the tunnel session opens, fail-closed: a
|
||
// writeReceipt failure lands in this catch, which tears the tunnel
|
||
// listener back down and refuses the start. One receipt per session
|
||
// open; browse command behavior over the tunnel is unchanged.
|
||
writeReceipt({
|
||
sink: 'browse-tunnel',
|
||
host: domain || 'connect.ngrok-agent.com',
|
||
payloadClass: 'tunnel-session-open (scoped-token browser-command surface)',
|
||
bytes: 0,
|
||
sha256: null,
|
||
consent: opts.consent,
|
||
});
|
||
|
||
tunnelListener = await ngrok.forward(forwardOpts);
|
||
tunnelUrl = tunnelListener.url();
|
||
tunnelServer = boundTunnel;
|
||
tunnelActive = true;
|
||
console.log(`[browse] Tunnel listener bound on 127.0.0.1:${tunnelPort}, ngrok → ${tunnelUrl}`);
|
||
|
||
// Update state file
|
||
const stateContent = JSON.parse(fs.readFileSync(config.stateFile, 'utf-8'));
|
||
stateContent.tunnel = { url: tunnelUrl, domain: domain || null, startedAt: new Date().toISOString() };
|
||
const tmpState = tmpStatePath();
|
||
fs.writeFileSync(tmpState, JSON.stringify(stateContent, null, 2), { mode: 0o600 });
|
||
fs.renameSync(tmpState, config.stateFile);
|
||
|
||
return { ok: true, url: tunnelUrl! };
|
||
} catch (err: any) {
|
||
// Clean up BOTH ngrok and the Bun listener on failure. If
|
||
// ngrok.forward() succeeded but tunnelListener.url() or the
|
||
// state-file write threw, we'd otherwise leak an active ngrok
|
||
// session on the user's account.
|
||
try { if (tunnelListener) await tunnelListener.close(); } catch {}
|
||
try { boundTunnel.stop(true); } catch {}
|
||
tunnelListener = null;
|
||
return { ok: false, stage: 'ngrok', error: err };
|
||
}
|
||
}
|
||
|
||
// Module-level validateAuth deleted in v1.35.0.0. Factory-scoped equivalent
|
||
// in buildFetchHandler closes over cfg.authToken so every internal auth check
|
||
// sees the same token the routes receive.
|
||
|
||
/**
|
||
* Terminal-agent discovery. The non-compiled bun process at
|
||
* `browse/src/terminal-agent.ts` writes its chosen port to
|
||
* `<stateDir>/terminal-port` and the loopback handshake token to
|
||
* `<stateDir>/terminal-internal-token` once it boots. Read on demand —
|
||
* lazy so we don't break tests that don't spawn the agent.
|
||
*/
|
||
function readTerminalPort(): number | null {
|
||
try {
|
||
const f = path.join(path.dirname(config.stateFile), 'terminal-port');
|
||
const v = parseInt(fs.readFileSync(f, 'utf-8').trim(), 10);
|
||
return Number.isFinite(v) && v > 0 ? v : null;
|
||
} catch { return null; }
|
||
}
|
||
function readTerminalInternalToken(): string | null {
|
||
try {
|
||
const f = path.join(path.dirname(config.stateFile), 'terminal-internal-token');
|
||
const t = fs.readFileSync(f, 'utf-8').trim();
|
||
return t.length > 16 ? t : null;
|
||
} catch { return null; }
|
||
}
|
||
|
||
/**
|
||
* Push a freshly-minted PTY cookie token to the terminal-agent so its
|
||
* /ws upgrade can validate the cookie. v1.44+: also pushes the bound
|
||
* sessionId so the agent can route /internal/restart and (Commit 3)
|
||
* re-attach back to the same PtySession. Loopback POST authenticated
|
||
* with the internal token written by the agent at startup. If the agent
|
||
* isn't up yet, the extension just retries /pty-session.
|
||
*/
|
||
async function grantPtyToken(token: string, sessionId?: string): Promise<boolean> {
|
||
const port = readTerminalPort();
|
||
const internal = readTerminalInternalToken();
|
||
if (!port || !internal) return false;
|
||
try {
|
||
const resp = await fetch(`http://127.0.0.1:${port}/internal/grant`, {
|
||
method: 'POST',
|
||
headers: {
|
||
'Content-Type': 'application/json',
|
||
'Authorization': `Bearer ${internal}`,
|
||
},
|
||
body: JSON.stringify(sessionId ? { token, sessionId } : { token }),
|
||
signal: AbortSignal.timeout(2000),
|
||
});
|
||
return resp.ok;
|
||
} catch { return false; }
|
||
}
|
||
|
||
/**
|
||
* Ask the terminal-agent to dispose the PtySession bound to `sessionId`.
|
||
* Scoped to one caller's session — sibling tabs/agents untouched. Used by
|
||
* /pty-restart and /pty-dispose. Returns true on agent ack.
|
||
*/
|
||
async function restartPtySession(sessionId: string): Promise<boolean> {
|
||
const port = readTerminalPort();
|
||
const internal = readTerminalInternalToken();
|
||
if (!port || !internal) return false;
|
||
try {
|
||
const resp = await fetch(`http://127.0.0.1:${port}/internal/restart`, {
|
||
method: 'POST',
|
||
headers: {
|
||
'Content-Type': 'application/json',
|
||
'Authorization': `Bearer ${internal}`,
|
||
},
|
||
body: JSON.stringify({ sessionId }),
|
||
signal: AbortSignal.timeout(5000),
|
||
});
|
||
return resp.ok;
|
||
} catch { return false; }
|
||
}
|
||
|
||
/** Extract bearer token from request. Returns the token string or null. */
|
||
function extractToken(req: Request): string | null {
|
||
const header = req.headers.get('authorization');
|
||
if (!header?.startsWith('Bearer ')) return null;
|
||
return header.slice(7);
|
||
}
|
||
|
||
/** Validate token and return TokenInfo. Returns null if invalid/expired. */
|
||
function getTokenInfo(req: Request): TokenInfo | null {
|
||
const token = extractToken(req);
|
||
if (!token) return null;
|
||
return validateScopedToken(token);
|
||
}
|
||
|
||
/** Check if request is from root token (local use). */
|
||
function isRootRequest(req: Request): boolean {
|
||
const token = extractToken(req);
|
||
return token !== null && isRootToken(token);
|
||
}
|
||
|
||
// ─── Help text (auto-generated from COMMAND_DESCRIPTIONS) ────────
|
||
function generateHelpText(): string {
|
||
// Group commands by category
|
||
const groups = new Map<string, string[]>();
|
||
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
||
const display = meta.usage || cmd;
|
||
const list = groups.get(meta.category) || [];
|
||
list.push(display);
|
||
groups.set(meta.category, list);
|
||
}
|
||
|
||
const categoryOrder = [
|
||
'Navigation', 'Reading', 'Interaction', 'Inspection',
|
||
'Visual', 'Snapshot', 'Meta', 'Tabs', 'Server',
|
||
];
|
||
|
||
const lines = ['gstack browse — headless browser for AI agents', '', 'Commands:'];
|
||
for (const cat of categoryOrder) {
|
||
const cmds = groups.get(cat);
|
||
if (!cmds) continue;
|
||
lines.push(` ${(cat + ':').padEnd(15)}${cmds.join(', ')}`);
|
||
}
|
||
|
||
// Snapshot flags from source of truth
|
||
lines.push('');
|
||
lines.push('Snapshot flags:');
|
||
const flagPairs: string[] = [];
|
||
for (const flag of SNAPSHOT_FLAGS) {
|
||
const label = flag.valueHint ? `${flag.short} ${flag.valueHint}` : flag.short;
|
||
flagPairs.push(`${label} ${flag.long}`);
|
||
}
|
||
// Print two flags per line for compact display
|
||
for (let i = 0; i < flagPairs.length; i += 2) {
|
||
const left = flagPairs[i].padEnd(28);
|
||
const right = flagPairs[i + 1] || '';
|
||
lines.push(` ${left}${right}`);
|
||
}
|
||
|
||
return lines.join('\n');
|
||
}
|
||
|
||
// ─── Buffer (from buffers.ts) ────────────────────────────────────
|
||
import { consoleBuffer, networkBuffer, dialogBuffer, addConsoleEntry, addNetworkEntry, addDialogEntry, type LogEntry, type NetworkEntry, type DialogEntry } from './buffers';
|
||
export { consoleBuffer, networkBuffer, dialogBuffer, addConsoleEntry, addNetworkEntry, addDialogEntry, type LogEntry, type NetworkEntry, type DialogEntry };
|
||
|
||
const CONSOLE_LOG_PATH = config.consoleLog;
|
||
const NETWORK_LOG_PATH = config.networkLog;
|
||
const DIALOG_LOG_PATH = config.dialogLog;
|
||
|
||
/**
|
||
* Per-process state-file temp path. The state-file write pattern is
|
||
* `writeFileSync(tmp, ...) → renameSync(tmp, stateFile)` for atomicity,
|
||
* but a shared `${stateFile}.tmp` filename means two concurrent writers
|
||
* (cold-start race when N CLIs hit a fresh repo simultaneously, parallel
|
||
* /tunnel/start handlers, or a combination) collide on the rename: the
|
||
* first writer's renameSync moves the shared temp file out of the way,
|
||
* the second writer's writeFileSync re-creates it, the second rename
|
||
* then races with the first writer's already-renamed state. Worst case
|
||
* the second renameSync throws ENOENT mid-air, killing one of the
|
||
* spawning daemons during startup.
|
||
*
|
||
* Per-process suffix (pid + 4 random bytes) makes each writer's temp
|
||
* path unique. The atomic rename still gives last-writer-wins semantics
|
||
* for the final state.json content; the only behavior change is that
|
||
* concurrent writers no longer kill each other on the rename.
|
||
*/
|
||
function tmpStatePath(): string {
|
||
return `${config.stateFile}.tmp.${process.pid}.${crypto.randomBytes(4).toString('hex')}`;
|
||
}
|
||
|
||
|
||
// ─── Sidebar agent / chat state ripped ──────────────────────────────
|
||
let lastConsoleFlushed = 0;
|
||
let lastNetworkFlushed = 0;
|
||
let lastDialogFlushed = 0;
|
||
let flushInProgress = false;
|
||
|
||
async function flushBuffers() {
|
||
if (flushInProgress) return; // Guard against concurrent flush
|
||
flushInProgress = true;
|
||
|
||
try {
|
||
// Console buffer
|
||
const newConsoleCount = consoleBuffer.totalAdded - lastConsoleFlushed;
|
||
if (newConsoleCount > 0) {
|
||
const entries = consoleBuffer.last(Math.min(newConsoleCount, consoleBuffer.length));
|
||
const lines = entries.map(e =>
|
||
`[${new Date(e.timestamp).toISOString()}] [${e.level}] ${e.text}`
|
||
).join('\n') + '\n';
|
||
appendSecureFile(CONSOLE_LOG_PATH, lines);
|
||
lastConsoleFlushed = consoleBuffer.totalAdded;
|
||
}
|
||
|
||
// Network buffer
|
||
const newNetworkCount = networkBuffer.totalAdded - lastNetworkFlushed;
|
||
if (newNetworkCount > 0) {
|
||
const entries = networkBuffer.last(Math.min(newNetworkCount, networkBuffer.length));
|
||
const lines = entries.map(e =>
|
||
`[${new Date(e.timestamp).toISOString()}] ${e.method} ${e.url} → ${e.status || 'pending'} (${e.duration || '?'}ms, ${e.size || '?'}B)`
|
||
).join('\n') + '\n';
|
||
appendSecureFile(NETWORK_LOG_PATH, lines);
|
||
lastNetworkFlushed = networkBuffer.totalAdded;
|
||
}
|
||
|
||
// Dialog buffer
|
||
const newDialogCount = dialogBuffer.totalAdded - lastDialogFlushed;
|
||
if (newDialogCount > 0) {
|
||
const entries = dialogBuffer.last(Math.min(newDialogCount, dialogBuffer.length));
|
||
const lines = entries.map(e =>
|
||
`[${new Date(e.timestamp).toISOString()}] [${e.type}] "${e.message}" → ${e.action}${e.response ? ` "${e.response}"` : ''}`
|
||
).join('\n') + '\n';
|
||
appendSecureFile(DIALOG_LOG_PATH, lines);
|
||
lastDialogFlushed = dialogBuffer.totalAdded;
|
||
}
|
||
} catch (err: any) {
|
||
console.error('[browse] Buffer flush failed:', err.message);
|
||
} finally {
|
||
flushInProgress = false;
|
||
}
|
||
}
|
||
|
||
// Flush every 1 second
|
||
const flushInterval = setInterval(flushBuffers, 1000);
|
||
|
||
// ─── Idle Timer ────────────────────────────────────────────────
|
||
let lastActivity = Date.now();
|
||
|
||
function resetIdleTimer() {
|
||
lastActivity = Date.now();
|
||
}
|
||
|
||
// Named for behavioral testing via __testInternals__. The factory tests in
|
||
// server-factory.test.ts call this directly so the idle-shutdown path can be
|
||
// exercised without waiting 60s for the interval to fire.
|
||
function idleCheckTick() {
|
||
// Headed mode: the user is looking at the browser. Never auto-die.
|
||
// Only shut down when the user explicitly disconnects or closes the window.
|
||
// Reads via the activeBrowserManager indirection so embedders that pass
|
||
// their own BrowserManager into buildFetchHandler hit the right instance.
|
||
if (activeBrowserManager.getConnectionMode() === 'headed') return;
|
||
// Tunnel mode: remote agents may send commands sporadically. Never auto-die.
|
||
if (tunnelActive) return;
|
||
if (Date.now() - lastActivity > IDLE_TIMEOUT_MS) {
|
||
console.log(`[browse] Idle for ${IDLE_TIMEOUT_MS / 1000}s, shutting down`);
|
||
activeShutdown?.();
|
||
}
|
||
}
|
||
const idleCheckInterval = setInterval(idleCheckTick, 60_000);
|
||
|
||
// Test-only surface for server-factory.test.ts. Lets the dual-instance
|
||
// idle-timer behavior be exercised deterministically without mutating
|
||
// Date.now (which would interact with the leaked module-level setInterval).
|
||
// Production code must never import this — see `idle timer + onDisconnect
|
||
// dual-instance fix` describe block for usage.
|
||
export const __testInternals__ = {
|
||
idleCheckTick,
|
||
// Watchdog seams (watchdog.test.ts): drive the 15s poll against an
|
||
// arbitrary (dead) PID, trigger the handoff-promotion suppression exactly
|
||
// as onHeadedPromotion does, and reset the latches between tests.
|
||
parentWatchdogTick,
|
||
suppressHeadedParentShutdown,
|
||
resetParentWatchdogState: () => { headedParentShutdownSuppressed = false; parentGone = false; },
|
||
setTunnelActive: (v: boolean) => { tunnelActive = v; },
|
||
setLastActivity: (t: number) => { lastActivity = t; },
|
||
formatExplicitPortUnavailableError,
|
||
formatRandomPortUnavailableError,
|
||
// Reset the module-level shutdown latch so tests that drive shutdown to
|
||
// completion (process.exit-stubbed) can be followed by tests that also
|
||
// need shutdown to fire. Without this, the second test's shutdown
|
||
// returns early at the `if (isShuttingDown) return;` guard.
|
||
resetShutdownState: () => { isShuttingDown = false; },
|
||
};
|
||
|
||
// ─── Parent-Process Watchdog ────────────────────────────────────────
|
||
// When the spawning CLI process (e.g. a Claude Code session) exits, this
|
||
// server can become an orphan — keeping chrome-headless-shell alive and
|
||
// causing console-window flicker on Windows. Poll the parent PID every 15s
|
||
// and self-terminate if it is gone.
|
||
//
|
||
// Headed mode (BROWSE_HEADED=1 or BROWSE_PARENT_PID=0): The user controls
|
||
// the browser window lifecycle. The CLI exits immediately after connect,
|
||
// so the watchdog would kill the server prematurely. Disabled in both cases
|
||
// as defense-in-depth — the CLI sets PID=0 for headed mode, and the server
|
||
// also checks BROWSE_HEADED in case a future launcher forgets.
|
||
// Cleanup happens via browser disconnect event or $B disconnect.
|
||
const BROWSE_PARENT_PID = parseInt(process.env.BROWSE_PARENT_PID || '0', 10);
|
||
// Outer gate: if the spawner explicitly marks this as headed (env var set at
|
||
// launch time), skip registering the watchdog entirely. Cheaper than entering
|
||
// the closure every 15s. The CLI's connect path sets BROWSE_HEADED=1 + PID=0,
|
||
// so this branch is the normal path for /open-gstack-browser.
|
||
const IS_HEADED_WATCHDOG = process.env.BROWSE_HEADED === '1';
|
||
// Runtime promotion to headed (`handoff`) must NOT clear this interval — the
|
||
// same tick is the tunnel-orphan reaper, and idle timeout is disabled in
|
||
// tunnel mode, so parent death is the ONLY thing that reaps an
|
||
// internet-exposed daemon after handoff → resume → /pair-agent. Promotion
|
||
// sets this suppress flag instead; the tick re-reads it (and tunnelActive)
|
||
// every pass. See suppressHeadedParentShutdown() below.
|
||
let headedParentShutdownSuppressed = false;
|
||
// Latch for the one-time "parent exited, staying alive" log line.
|
||
let parentGone = false;
|
||
// Named + parameterized (default: the boot-time env PID) so watchdog.test.ts
|
||
// can drive the tick deterministically via __testInternals__, mirroring
|
||
// idleCheckTick above. setInterval invokes it with no args in production.
|
||
function parentWatchdogTick(parentPid: number = BROWSE_PARENT_PID): void {
|
||
try {
|
||
process.kill(parentPid, 0); // signal 0 = existence check only, no signal sent
|
||
} catch {
|
||
// Parent exited. Resolution order:
|
||
// 1. Active cookie picker (one-time code or session live)? Stay alive
|
||
// regardless of mode — tearing down the server mid-import leaves the
|
||
// picker UI with a stale "Failed to fetch" error.
|
||
// 2. Headed (unless suppressed by a runtime promotion) / tunnel mode?
|
||
// Shutdown. The idle timeout doesn't apply in these modes (see
|
||
// idleCheckInterval above — both early-return), so ignoring parent
|
||
// death here would leak orphan daemons after /pair-agent or
|
||
// /open-gstack-browser sessions.
|
||
// 3. Normal (headless) mode, or headed-by-promotion? Stay alive. Claude
|
||
// Code's Bash tool kills the parent shell between invocations, and a
|
||
// promoted daemon's user owns the window lifecycle. The idle timeout
|
||
// (30 min) handles eventual cleanup.
|
||
if (hasActivePicker()) return;
|
||
const headed = activeBrowserManager.getConnectionMode() === 'headed'
|
||
&& !headedParentShutdownSuppressed;
|
||
if (headed || tunnelActive) {
|
||
console.log(`[browse] Parent process ${parentPid} exited in ${headed ? 'headed' : 'tunnel'} mode, shutting down`);
|
||
activeShutdown?.();
|
||
} else if (!parentGone) {
|
||
parentGone = true;
|
||
console.log(`[browse] Parent process ${parentPid} exited (server stays alive, idle timeout will clean up)`);
|
||
}
|
||
}
|
||
}
|
||
// Poll cadence. Env-overridable as a test seam: watchdog.test.ts shrinks it
|
||
// (250ms) so a free-tier test can observe a real tick deciding on a dead
|
||
// parent instead of sleeping through the 15s production cadence. Production
|
||
// launchers never set this; unparsable or non-positive values fall back to 15s.
|
||
const rawWatchdogIntervalMs = parseInt(process.env.BROWSE_PARENT_WATCHDOG_INTERVAL_MS || '', 10);
|
||
const PARENT_WATCHDOG_INTERVAL_MS =
|
||
Number.isFinite(rawWatchdogIntervalMs) && rawWatchdogIntervalMs > 0
|
||
? rawWatchdogIntervalMs
|
||
: 15_000;
|
||
if (BROWSE_PARENT_PID > 0 && !IS_HEADED_WATCHDOG) {
|
||
setInterval(parentWatchdogTick, PARENT_WATCHDOG_INTERVAL_MS);
|
||
} else if (IS_HEADED_WATCHDOG) {
|
||
console.log('[browse] Parent-process watchdog disabled (headed mode)');
|
||
} else if (BROWSE_PARENT_PID === 0) {
|
||
console.log('[browse] Parent-process watchdog disabled (BROWSE_PARENT_PID=0)');
|
||
}
|
||
|
||
/**
|
||
* Suppress the headed-mode parent-death shutdown after a runtime promotion.
|
||
*
|
||
* The watchdog's contract is "headless daemons outlive their parent, headed ones
|
||
* do not" — reasonable at boot, when mode is fixed by env. `handoff` breaks that
|
||
* assumption: it swaps in a headed context on a RUNNING daemon
|
||
* (browser-manager.ts, connectionMode = 'headed') without a restart, so a daemon
|
||
* that legitimately registered a watchdog is suddenly on the fatal side of the
|
||
* branch. The parent is typically a short-lived shell — Claude Code's Bash tool
|
||
* kills one after every invocation — so the next 15s poll shuts the daemon down,
|
||
* discarding whatever the user was handed off to do, such as a login.
|
||
*
|
||
* Once promoted, the user owns the window lifecycle exactly as if the daemon had
|
||
* been started headed, which is the case the env guards already exempt.
|
||
*
|
||
* A flag, NOT clearInterval: the tick doubles as the tunnel-orphan reaper
|
||
* (its `tunnelActive` branch), and idle timeout is disabled in tunnel mode —
|
||
* clearing the whole interval here left handoff → resume → /pair-agent with
|
||
* an internet-exposed daemon nothing could ever reap. After promotion, parent
|
||
* death no longer kills the daemon for BEING HEADED, but still kills it when
|
||
* a tunnel is active.
|
||
*/
|
||
function suppressHeadedParentShutdown(): void {
|
||
if (headedParentShutdownSuppressed) return;
|
||
headedParentShutdownSuppressed = true;
|
||
console.log('[browse] Parent-death headed shutdown suppressed (promoted to headed at runtime); watchdog stays armed as the tunnel-orphan reaper');
|
||
}
|
||
|
||
// ─── Command Sets (from commands.ts — single source of truth) ───
|
||
import { READ_COMMANDS, WRITE_COMMANDS, META_COMMANDS } from './commands';
|
||
export { READ_COMMANDS, WRITE_COMMANDS, META_COMMANDS };
|
||
|
||
/**
|
||
* Whether an invocation should be treated as a WRITE for capability gating
|
||
* (scope, watch-mode block, tab ownership, tunnel). A command is a write if it
|
||
* mutates state (`WRITE_COMMANDS`) OR it carries an `--out` flag — `js`/`eval
|
||
* --out` writes the evaluate result to local disk, so the capability is
|
||
* per-invocation, not per-command-name. This deliberately does NOT change
|
||
* dispatch routing: `js`/`eval` still route to `handleReadCommand`; only the
|
||
* security gates consult this.
|
||
*/
|
||
function isWriteInvocation(command: string, args: string[]): boolean {
|
||
return WRITE_COMMANDS.has(command) || hasOutArg(args);
|
||
}
|
||
|
||
// ─── Inspector State (in-memory) ──────────────────────────────
|
||
let inspectorData: InspectorResult | null = null;
|
||
let inspectorTimestamp: number = 0;
|
||
|
||
// Inspector SSE subscribers
|
||
type InspectorSubscriber = (event: any) => void;
|
||
const inspectorSubscribers = new Set<InspectorSubscriber>();
|
||
|
||
/** Diagnostic accessor used by the $B memory snapshot. */
|
||
export function getInspectorSubscriberCount(): number {
|
||
return inspectorSubscribers.size;
|
||
}
|
||
|
||
function emitInspectorEvent(event: any): void {
|
||
for (const notify of inspectorSubscribers) {
|
||
queueMicrotask(() => {
|
||
try { notify(event); } catch (err: any) {
|
||
console.error('[browse] Inspector event subscriber threw:', err.message);
|
||
}
|
||
});
|
||
}
|
||
}
|
||
|
||
// ─── Server ────────────────────────────────────────────────────
|
||
const browserManager = new BrowserManager();
|
||
// Declared here rather than beside suppressHeadedParentShutdown: that function
|
||
// sits with the watchdog it gates, which is above this line, and binding it up
|
||
// there would touch `browserManager` in its temporal dead zone — aborting
|
||
// module evaluation and leaving every later const uninitialized.
|
||
browserManager.onHeadedPromotion = suppressHeadedParentShutdown;
|
||
// Indirection for embedders. Module-level handlers (idleCheckTick, parent
|
||
// watchdog, SIGTERM) read activeBrowserManager so that buildFetchHandler can
|
||
// retarget them at a caller-supplied BrowserManager. Symmetric with the
|
||
// existing `let activeShutdown` pattern at module scope (line ~113).
|
||
// Without this, embedders like gbrowser hit the dead module-level instance
|
||
// whose connectionMode never leaves 'launched' — and headed mode never
|
||
// short-circuits idle-shutdown.
|
||
let activeBrowserManager: BrowserManager = browserManager;
|
||
// When the user closes the headed browser window, run full cleanup
|
||
// (kill terminal agent, save session, remove profile locks, delete state file)
|
||
// before exiting. Exit code 0 means user-initiated clean quit (Cmd+Q on
|
||
// macOS) so process supervisors like gbrowser's gbd skip the restart loop;
|
||
// 2 means a real crash that should respawn. The fallback `?? 2` preserves
|
||
// legacy crash semantics for any caller that invokes onDisconnect without
|
||
// an explicit code. This is the safety-net default for the CLI flow before
|
||
// any buildFetchHandler call rebinds onDisconnect onto the cfg instance.
|
||
browserManager.onDisconnect = (code) => activeShutdown?.(code ?? 2);
|
||
let isShuttingDown = false;
|
||
// Session-persist ticker handle. Registered in start() (module scope so the
|
||
// factory's shutdown() can reach it), cleared by shutdown() BEFORE the final
|
||
// snapshot — a tick landing during browser teardown would otherwise overwrite
|
||
// the good final snapshot with a degraded one (zero tabs).
|
||
let sessionPersistInterval: ReturnType<typeof setInterval> | null = null;
|
||
|
||
// Port allocation lives in port-allocator.ts (#2314, decision 8) so the
|
||
// terminal-agent shares the SAME fixed 10000-60000 scan range instead of
|
||
// binding port:0 into the OS ephemeral range. The imports at the top of
|
||
// this file re-expose the pieces __testInternals__ pins.
|
||
|
||
// Find port: explicit BROWSE_PORT, or random in 10000-60000
|
||
function findPort(): Promise<number> {
|
||
return findAvailablePort(BROWSE_PORT);
|
||
}
|
||
|
||
/**
|
||
* Translate Playwright errors into actionable messages for AI agents.
|
||
*/
|
||
function wrapError(err: any): string {
|
||
const msg = err.message || String(err);
|
||
// Timeout errors
|
||
if (err.name === 'TimeoutError' || msg.includes('Timeout') || msg.includes('timeout')) {
|
||
if (msg.includes('locator.click') || msg.includes('locator.fill') || msg.includes('locator.hover')) {
|
||
return `Element not found or not interactable within timeout. Check your selector or run 'snapshot' for fresh refs.`;
|
||
}
|
||
if (msg.includes('page.goto') || msg.includes('Navigation')) {
|
||
return `Page navigation timed out. The URL may be unreachable or the page may be loading slowly.`;
|
||
}
|
||
return `Operation timed out: ${msg.split('\n')[0]}`;
|
||
}
|
||
// Multiple elements matched
|
||
if (msg.includes('resolved to') && msg.includes('elements')) {
|
||
return `Selector matched multiple elements. Be more specific or use @refs from 'snapshot'.`;
|
||
}
|
||
// Pass through other errors
|
||
return msg;
|
||
}
|
||
|
||
/** Internal command result — used by handleCommand and chain subcommand routing */
|
||
interface CommandResult {
|
||
status: number;
|
||
result: string;
|
||
headers?: Record<string, string>;
|
||
json?: boolean; // true if result is JSON (errors), false for text/plain
|
||
}
|
||
|
||
/**
|
||
* Core command execution logic. Returns a structured result instead of HTTP Response.
|
||
* Used by both the HTTP handler (handleCommand) and chain subcommand routing.
|
||
*
|
||
* Options:
|
||
* skipRateCheck: true when called from chain (chain counts as 1 request)
|
||
* skipActivity: true when called from chain (chain emits 1 event for all subcommands)
|
||
* chainDepth: recursion guard — reject nested chains (depth > 0 means inside a chain)
|
||
*/
|
||
async function handleCommandInternalImpl(
|
||
body: { command: string; args?: string[]; tabId?: number },
|
||
tokenInfo?: TokenInfo | null,
|
||
opts?: { skipRateCheck?: boolean; skipActivity?: boolean; chainDepth?: number },
|
||
): Promise<CommandResult> {
|
||
const { args = [], tabId } = body;
|
||
const rawCommand = body.command;
|
||
|
||
if (!rawCommand) {
|
||
return { status: 400, result: JSON.stringify({ error: 'Missing "command" field' }), json: true };
|
||
}
|
||
|
||
// ─── Alias canonicalization (before scope, watch, tab-ownership, dispatch) ─
|
||
// Agent-friendly names like 'setcontent' route to canonical 'load-html'. Must
|
||
// happen BEFORE scope check so a read-scoped token calling 'setcontent' is still
|
||
// rejected (load-html lives in SCOPE_WRITE). Audit logging preserves rawCommand
|
||
// so the trail records what the agent actually typed.
|
||
const command = canonicalizeCommand(rawCommand);
|
||
const isAliased = command !== rawCommand;
|
||
|
||
// ─── Recursion guard: reject nested chains ──────────────────
|
||
if (command === 'chain' && (opts?.chainDepth ?? 0) > 0) {
|
||
return { status: 400, result: JSON.stringify({ error: 'Nested chain commands are not allowed' }), json: true };
|
||
}
|
||
|
||
// ─── Scope check (for scoped tokens) ──────────────────────────
|
||
if (tokenInfo && tokenInfo.clientId !== 'root') {
|
||
if (!checkScope(tokenInfo, command)) {
|
||
return {
|
||
status: 403, json: true,
|
||
result: JSON.stringify({
|
||
error: `Command "${command}" not allowed by your token scope`,
|
||
hint: `Your scopes: ${tokenInfo.scopes.join(', ')}. Ask the user to re-pair without --restrict for full page access, or with --control for browser control commands.`,
|
||
}),
|
||
};
|
||
}
|
||
|
||
// `--out` writes the evaluate result to local disk, which is a WRITE
|
||
// capability distinct from the JS-exec (admin) capability js/eval need.
|
||
// Require write scope so an admin-but-not-write token can't write files.
|
||
if (hasOutArg(args) && !tokenInfo.scopes.includes('write')) {
|
||
return {
|
||
status: 403, json: true,
|
||
result: JSON.stringify({
|
||
error: `"--out" writes to disk and requires the "write" scope`,
|
||
hint: `Your scopes: ${tokenInfo.scopes.join(', ')}. Re-pair with write access to use --out.`,
|
||
}),
|
||
};
|
||
}
|
||
|
||
// Domain check for navigation commands
|
||
if ((command === 'goto' || command === 'newtab') && args[0]) {
|
||
if (!checkDomain(tokenInfo, args[0])) {
|
||
return {
|
||
status: 403, json: true,
|
||
result: JSON.stringify({
|
||
error: `Domain not allowed by your token scope`,
|
||
hint: `Allowed domains: ${tokenInfo.domains?.join(', ') || 'none configured'}`,
|
||
}),
|
||
};
|
||
}
|
||
}
|
||
|
||
// Rate check (skipped for chain subcommands — chain counts as 1 request)
|
||
if (!opts?.skipRateCheck) {
|
||
const rateResult = checkRate(tokenInfo);
|
||
if (!rateResult.allowed) {
|
||
return {
|
||
status: 429, json: true,
|
||
result: JSON.stringify({
|
||
error: 'Rate limit exceeded',
|
||
hint: `Max ${tokenInfo.rateLimit} requests/second. Retry after ${rateResult.retryAfterMs}ms.`,
|
||
}),
|
||
headers: { 'Retry-After': String(Math.ceil((rateResult.retryAfterMs || 1000) / 1000)) },
|
||
};
|
||
}
|
||
}
|
||
|
||
// Record command execution for idempotent key exchange tracking
|
||
if (!opts?.skipRateCheck && tokenInfo.token) recordCommand(tokenInfo.token);
|
||
}
|
||
|
||
// Pin to a specific tab if requested (set by BROWSE_TAB env var, e.g. per-tab agent contexts).
|
||
// This prevents parallel agents from interfering with each other's tab context.
|
||
// Safe because Bun's event loop is single-threaded — no concurrent handleCommand.
|
||
let savedTabId: number | null = null;
|
||
if (tabId !== undefined && tabId !== null) {
|
||
savedTabId = browserManager.getActiveTabId();
|
||
// bringToFront: false — internal tab pinning must NOT steal window focus
|
||
try { browserManager.switchTab(tabId, { bringToFront: false }); } catch (err: any) {
|
||
console.warn('[browse] Failed to pin tab', tabId, ':', err.message);
|
||
}
|
||
}
|
||
|
||
// ─── Tab ownership check (own-only tokens / pair-agent isolation) ──
|
||
//
|
||
// Only `own-only` tokens (pair-agent over tunnel) are bound to their own
|
||
// tabs. `shared` tokens — the default for skill spawns and local scoped
|
||
// clients — can drive any tab; the capability gate (scope checks above)
|
||
// and rate limits already constrain what they can do.
|
||
//
|
||
// Skip for `newtab` — it creates a tab rather than accessing one.
|
||
if (command !== 'newtab' && tokenInfo && tokenInfo.clientId !== 'root' && tokenInfo.tabPolicy === 'own-only') {
|
||
const targetTab = tabId ?? browserManager.getActiveTabId();
|
||
if (!browserManager.checkTabAccess(targetTab, tokenInfo.clientId, { isWrite: isWriteInvocation(command, args), ownOnly: true })) {
|
||
return {
|
||
status: 403, json: true,
|
||
result: JSON.stringify({
|
||
error: 'Tab not owned by your agent. Use newtab to create your own tab.',
|
||
hint: `Tab ${targetTab} is owned by ${browserManager.getTabOwner(targetTab) || 'root'}. Your agent: ${tokenInfo.clientId}.`,
|
||
}),
|
||
};
|
||
}
|
||
}
|
||
|
||
// ─── newtab with ownership for scoped tokens ──────────────
|
||
if (command === 'newtab' && tokenInfo && tokenInfo.clientId !== 'root') {
|
||
const newId = await browserManager.newTab(args[0] || undefined, tokenInfo.clientId);
|
||
return {
|
||
status: 200, json: true,
|
||
result: JSON.stringify({
|
||
tabId: newId,
|
||
owner: tokenInfo.clientId,
|
||
hint: 'Include "tabId": ' + newId + ' in subsequent commands to target this tab.',
|
||
}),
|
||
};
|
||
}
|
||
|
||
// Block mutation commands while watching (read-only observation mode).
|
||
// `--out` invocations count as mutations (they write the result to disk).
|
||
if (browserManager.isWatching() && isWriteInvocation(command, args)) {
|
||
return {
|
||
status: 400, json: true,
|
||
result: JSON.stringify({ error: 'Cannot run mutation commands while watching. Run `$B watch stop` first.' }),
|
||
};
|
||
}
|
||
|
||
// Activity: emit command_start (skipped for chain subcommands)
|
||
const startTime = Date.now();
|
||
if (!opts?.skipActivity) {
|
||
emitActivity({
|
||
type: 'command_start',
|
||
command,
|
||
args,
|
||
url: browserManager.getCurrentUrl(),
|
||
tabs: browserManager.getTabCount(),
|
||
mode: browserManager.getConnectionMode(),
|
||
clientId: tokenInfo?.clientId,
|
||
});
|
||
}
|
||
|
||
try {
|
||
let result: string;
|
||
|
||
const session = browserManager.getActiveSession();
|
||
|
||
// Per-request warnings collected during hidden-element detection,
|
||
// surfaced into the envelope the LLM sees. Carries across the read
|
||
// phase into the centralized wrap block below.
|
||
let hiddenContentWarnings: string[] = [];
|
||
|
||
if (READ_COMMANDS.has(command)) {
|
||
const isScoped = tokenInfo && tokenInfo.clientId !== 'root';
|
||
// Hidden-element / ARIA-injection detection for every scoped
|
||
// DOM-reading channel (text, html, links, forms, accessibility,
|
||
// attrs, data, media, ux-audit). Previously only `text` received
|
||
// stripping; other channels let hidden injection payloads reach
|
||
// the LLM despite the envelope wrap. Detections become CONTENT
|
||
// WARNINGS on the outgoing envelope so the model can see what it
|
||
// would have otherwise trusted silently.
|
||
if (isScoped && DOM_CONTENT_COMMANDS.has(command)) {
|
||
const page = session.getPage();
|
||
try {
|
||
const strippedDescs = await markHiddenElements(page);
|
||
if (strippedDescs.length > 0) {
|
||
console.warn(`[browse] Content security: ${strippedDescs.length} hidden elements flagged on ${command} for ${tokenInfo.clientId}`);
|
||
hiddenContentWarnings = strippedDescs.slice(0, 8).map(d =>
|
||
`hidden content: ${d.slice(0, 120)}`,
|
||
);
|
||
if (strippedDescs.length > 8) {
|
||
hiddenContentWarnings.push(`hidden content: +${strippedDescs.length - 8} more flagged elements`);
|
||
}
|
||
}
|
||
if (command === 'text') {
|
||
const target = session.getActiveFrameOrPage();
|
||
result = await getCleanTextWithStripping(target);
|
||
} else {
|
||
result = await handleReadCommand(command, args, session, browserManager);
|
||
}
|
||
} finally {
|
||
await cleanupHiddenMarkers(page);
|
||
}
|
||
} else {
|
||
result = await handleReadCommand(command, args, session, browserManager);
|
||
}
|
||
} else if (WRITE_COMMANDS.has(command)) {
|
||
result = await handleWriteCommand(command, args, session, browserManager);
|
||
} else if (META_COMMANDS.has(command)) {
|
||
// Pass chain depth + executeCommand callback so chain routes subcommands
|
||
// through the full security pipeline (scope, domain, tab, wrapping).
|
||
const chainDepth = (opts?.chainDepth ?? 0);
|
||
// shutdown is factory-scoped (deleted from module scope in v1.35.0.0);
|
||
// route the call through activeShutdown which buildFetchHandler assigns.
|
||
const shutdownFn = () => activeShutdown ? activeShutdown() : Promise.resolve();
|
||
result = await handleMetaCommand(command, args, browserManager, shutdownFn, tokenInfo, {
|
||
chainDepth,
|
||
daemonPort: LOCAL_LISTEN_PORT,
|
||
executeCommand: (body, ti) => handleCommandInternal(body, ti, {
|
||
skipRateCheck: true, // chain counts as 1 request
|
||
skipActivity: true, // chain emits 1 event for all subcommands
|
||
chainDepth: chainDepth + 1, // recursion guard
|
||
}),
|
||
});
|
||
// Start periodic snapshot interval when watch mode begins
|
||
if (command === 'watch' && args[0] !== 'stop' && browserManager.isWatching()) {
|
||
const watchInterval = setInterval(async () => {
|
||
if (!browserManager.isWatching()) {
|
||
clearInterval(watchInterval);
|
||
return;
|
||
}
|
||
try {
|
||
const snapshot = await handleSnapshot(['-i'], browserManager.getActiveSession());
|
||
browserManager.addWatchSnapshot(snapshot);
|
||
} catch {
|
||
// Page may be navigating — skip this snapshot
|
||
}
|
||
}, 5000);
|
||
browserManager.watchInterval = watchInterval;
|
||
}
|
||
} else if (command === 'help') {
|
||
const helpText = generateHelpText();
|
||
return { status: 200, result: helpText };
|
||
} else {
|
||
// Use the rich unknown-command helper: names the input, suggests the closest
|
||
// match via Levenshtein (≤ 2 distance, ≥ 4 chars input), and appends an upgrade
|
||
// hint if the command is listed in NEW_IN_VERSION.
|
||
return {
|
||
status: 400, json: true,
|
||
result: JSON.stringify({
|
||
error: buildUnknownCommandError(rawCommand, ALL_COMMANDS),
|
||
hint: `Available commands: ${[...READ_COMMANDS, ...WRITE_COMMANDS, ...META_COMMANDS].sort().join(', ')}`,
|
||
}),
|
||
};
|
||
}
|
||
|
||
// ─── Centralized content wrapping (single location for all commands) ───
|
||
// Scoped tokens: content filter + enhanced envelope + datamarking
|
||
// Root tokens: basic untrusted content wrapper (backward compat)
|
||
// Chain exempt from top-level wrapping (each subcommand wrapped individually)
|
||
if (PAGE_CONTENT_COMMANDS.has(command) && command !== 'chain') {
|
||
const isScoped = tokenInfo && tokenInfo.clientId !== 'root';
|
||
if (isScoped) {
|
||
// Run content filters
|
||
const filterResult: ContentFilterResult = runContentFilters(
|
||
result, browserManager.getCurrentUrl(), command,
|
||
);
|
||
if (filterResult.blocked) {
|
||
return { status: 403, json: true, result: JSON.stringify({ error: filterResult.message }) };
|
||
}
|
||
// Datamark text command output only (not html, forms, or structured data)
|
||
if (command === 'text') {
|
||
result = datamarkContent(result);
|
||
}
|
||
// Enhanced envelope wrapping for scoped tokens.
|
||
// Merge per-request hidden-element warnings with content-filter
|
||
// warnings so both reach the LLM through the same CONTENT
|
||
// WARNINGS header.
|
||
const combinedWarnings = [...filterResult.warnings, ...hiddenContentWarnings];
|
||
result = wrapUntrustedPageContent(
|
||
result, command,
|
||
combinedWarnings.length > 0 ? combinedWarnings : undefined,
|
||
);
|
||
} else {
|
||
// Root token: basic wrapping (backward compat, Decision 2)
|
||
result = wrapUntrustedContent(result, browserManager.getCurrentUrl());
|
||
}
|
||
}
|
||
|
||
// Activity: emit command_end (skipped for chain subcommands)
|
||
const successDuration = Date.now() - startTime;
|
||
if (!opts?.skipActivity) {
|
||
emitActivity({
|
||
type: 'command_end',
|
||
command,
|
||
args,
|
||
url: browserManager.getCurrentUrl(),
|
||
duration: successDuration,
|
||
status: 'ok',
|
||
result: result,
|
||
tabs: browserManager.getTabCount(),
|
||
mode: browserManager.getConnectionMode(),
|
||
clientId: tokenInfo?.clientId,
|
||
});
|
||
}
|
||
|
||
writeAuditEntry({
|
||
ts: new Date().toISOString(),
|
||
cmd: command,
|
||
aliasOf: isAliased ? rawCommand : undefined,
|
||
args: args.join(' '),
|
||
origin: browserManager.getCurrentUrl(),
|
||
durationMs: successDuration,
|
||
status: 'ok',
|
||
hasCookies: browserManager.hasCookieImports(),
|
||
mode: browserManager.getConnectionMode(),
|
||
});
|
||
|
||
browserManager.resetFailures();
|
||
// Restore original active tab if we pinned to a specific one
|
||
if (savedTabId !== null) {
|
||
try { browserManager.switchTab(savedTabId, { bringToFront: false }); } catch (restoreErr: any) {
|
||
console.warn('[browse] Failed to restore tab after command:', restoreErr.message);
|
||
}
|
||
}
|
||
return { status: 200, result };
|
||
} catch (err: any) {
|
||
// Restore original active tab even on error
|
||
if (savedTabId !== null) {
|
||
try { browserManager.switchTab(savedTabId, { bringToFront: false }); } catch (restoreErr: any) {
|
||
console.warn('[browse] Failed to restore tab after error:', restoreErr.message);
|
||
}
|
||
}
|
||
|
||
// Activity: emit command_end (error) — skipped for chain subcommands
|
||
const errorDuration = Date.now() - startTime;
|
||
if (!opts?.skipActivity) {
|
||
emitActivity({
|
||
type: 'command_end',
|
||
command,
|
||
args,
|
||
url: browserManager.getCurrentUrl(),
|
||
duration: errorDuration,
|
||
status: 'error',
|
||
error: err.message,
|
||
tabs: browserManager.getTabCount(),
|
||
mode: browserManager.getConnectionMode(),
|
||
clientId: tokenInfo?.clientId,
|
||
});
|
||
}
|
||
|
||
writeAuditEntry({
|
||
ts: new Date().toISOString(),
|
||
cmd: command,
|
||
aliasOf: isAliased ? rawCommand : undefined,
|
||
args: args.join(' '),
|
||
origin: browserManager.getCurrentUrl(),
|
||
durationMs: errorDuration,
|
||
status: 'error',
|
||
error: err.message,
|
||
hasCookies: browserManager.hasCookieImports(),
|
||
mode: browserManager.getConnectionMode(),
|
||
});
|
||
|
||
browserManager.incrementFailures();
|
||
let errorMsg = wrapError(err);
|
||
const hint = browserManager.getFailureHint();
|
||
if (hint) errorMsg += '\n' + hint;
|
||
return { status: 500, result: JSON.stringify({ error: errorMsg }), json: true };
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Sanitizing wrapper around handleCommandInternalImpl. ALL callers (single-command
|
||
* HTTP, batch loop, scoped-token dispatch) go through this so the lone-surrogate
|
||
* sanitization happens once at the architectural choke point, not per-leaf.
|
||
* Do not bypass this by calling handleCommandInternalImpl directly.
|
||
*/
|
||
async function handleCommandInternal(
|
||
body: { command: string; args?: string[]; tabId?: number },
|
||
tokenInfo?: TokenInfo | null,
|
||
opts?: { skipRateCheck?: boolean; skipActivity?: boolean; chainDepth?: number },
|
||
): Promise<CommandResult> {
|
||
const cr = await handleCommandInternalImpl(body, tokenInfo, opts);
|
||
return { ...cr, result: stripLoneSurrogates(cr.result) };
|
||
}
|
||
|
||
/**
|
||
* Build the HTTP response from a CommandResult. Pure function so it can be
|
||
* unit-tested without spinning up the server (#1440). Defense in depth on top
|
||
* of handleCommandInternal's choke-point sanitization: this catches any
|
||
* \uXXXX JSON-escape surrogate forms that the raw-codepoint regex above
|
||
* misses when the body has already been JSON-stringified.
|
||
*/
|
||
export function buildCommandResponse(cr: CommandResult): Response {
|
||
const contentType = cr.json ? 'application/json' : 'text/plain';
|
||
const safeBody = typeof cr.result === 'string' ? sanitizeBody(cr.result, !!cr.json) : cr.result;
|
||
return new Response(safeBody, {
|
||
status: cr.status,
|
||
headers: { 'Content-Type': contentType, ...cr.headers },
|
||
});
|
||
}
|
||
|
||
/** HTTP wrapper — converts CommandResult to Response. Used by the /command
|
||
* route dispatcher (line ~2158). The wrapper layer exists so
|
||
* `buildCommandResponse` is independently unit-testable (v1.38.1.0).
|
||
*/
|
||
async function handleCommand(body: any, tokenInfo?: TokenInfo | null): Promise<Response> {
|
||
const cr = await handleCommandInternal(body, tokenInfo);
|
||
return buildCommandResponse(cr);
|
||
}
|
||
|
||
// Module-level shutdown function deleted in v1.39.0.0; it now lives inside
|
||
// the buildFetchHandler closure so it closes the cfg-provided browserManager.
|
||
// Signal handlers below call activeShutdown which buildFetchHandler assigns.
|
||
|
||
// Handle signals
|
||
//
|
||
// Node passes the signal name (e.g. 'SIGTERM') as the first arg to listeners.
|
||
// Wrap calls so activeShutdown receives no args — otherwise the string gets
|
||
// passed as exitCode and process.exit() coerces it to NaN, exiting with code 1
|
||
// instead of 0. (Caught in v0.18.1.0 #1025.)
|
||
//
|
||
// Gated on `import.meta.main` so embedders (gbrowser phoenix) that import
|
||
// server.ts as a submodule can register their own signal handlers without
|
||
// fighting with gstack's. CLI path is unchanged.
|
||
if (import.meta.main) {
|
||
// Standalone daemon: a Chromium crash must exit THIS process (its
|
||
// supervisor/user notices); embedders and in-process test launches must
|
||
// never be exited by browser-manager's disconnect handler.
|
||
markDaemonProcess();
|
||
// SIGINT (Ctrl+C): user intentionally stopping → shutdown.
|
||
process.on('SIGINT', () => activeShutdown?.());
|
||
// SIGHUP (terminal hangup): with handleSIGHUP:false at the three launch
|
||
// sites (#2220), Playwright no longer closes Chromium when this process
|
||
// gets hung up on — this handler is now the ONLY Chromium cleanup on
|
||
// SIGHUP (ENG-OV4). Route to the same shutdown path as SIGINT:
|
||
// activeShutdown closes the browser, releases ports, and removes the
|
||
// state file. Without it, a hangup would leak a live Chromium.
|
||
process.on('SIGHUP', () => activeShutdown?.());
|
||
// SIGTERM behavior depends on mode:
|
||
// - Normal (headless) mode: Claude Code's Bash sandbox fires SIGTERM when the
|
||
// parent shell exits between tool invocations. Ignoring it keeps the server
|
||
// alive across $B calls. Idle timeout (30 min) handles eventual cleanup.
|
||
// - Headed / tunnel mode: idle timeout doesn't apply in these modes. Respect
|
||
// SIGTERM so external tooling (systemd, supervisord, CI) can shut cleanly
|
||
// without waiting forever. Ctrl+C and /stop still work either way.
|
||
// - Active cookie picker: never tear down mid-import regardless of mode —
|
||
// would strand the picker UI with "Failed to fetch."
|
||
process.on('SIGTERM', () => {
|
||
if (hasActivePicker()) {
|
||
console.log('[browse] Received SIGTERM but cookie picker is active, ignoring to avoid stranding the picker UI');
|
||
return;
|
||
}
|
||
const headed = activeBrowserManager.getConnectionMode() === 'headed';
|
||
if (headed || tunnelActive) {
|
||
console.log(`[browse] Received SIGTERM in ${headed ? 'headed' : 'tunnel'} mode, shutting down`);
|
||
activeShutdown?.();
|
||
} else {
|
||
console.log('[browse] Received SIGTERM (ignoring — use /stop or Ctrl+C for intentional shutdown)');
|
||
}
|
||
});
|
||
// Windows: taskkill /F bypasses SIGTERM, but 'exit' fires for some shutdown paths.
|
||
// Defense-in-depth — primary cleanup is the CLI's stale-state detection via health check.
|
||
if (process.platform === 'win32') {
|
||
process.on('exit', () => {
|
||
safeUnlinkQuiet(config.stateFile);
|
||
});
|
||
}
|
||
}
|
||
|
||
// Emergency cleanup for crashes (OOM, uncaught exceptions, browser disconnect)
|
||
function emergencyCleanup() {
|
||
if (isShuttingDown) return;
|
||
isShuttingDown = true;
|
||
// Xvfb cleanup MUST happen before state-file deletion. spawnXvfb detaches
|
||
// the child, so without this, an uncaught exception leaves the Xvfb
|
||
// running with no PID record — orphan accumulates and eventually
|
||
// exhausts the :99-:120 display range. Read the state file FIRST,
|
||
// call cleanupXvfb (validates cmdline + start-time before kill), THEN
|
||
// delete the state file.
|
||
try {
|
||
if (fs.existsSync(config.stateFile)) {
|
||
const raw = fs.readFileSync(config.stateFile, 'utf-8');
|
||
const state = JSON.parse(raw);
|
||
if (state.xvfbPid && state.xvfbStartTime) {
|
||
// Lazy import — emergencyCleanup may run on platforms where
|
||
// ./xvfb's Linux-specific helpers fail to load. Best effort.
|
||
try {
|
||
const { cleanupXvfb } = require('./xvfb');
|
||
cleanupXvfb({
|
||
pid: state.xvfbPid,
|
||
startTime: state.xvfbStartTime,
|
||
display: state.xvfbDisplay || ':99',
|
||
});
|
||
} catch { /* best effort */ }
|
||
}
|
||
}
|
||
} catch { /* state file unparseable — fall through to lock + state cleanup */ }
|
||
|
||
// Clean Chromium profile locks via the shared helper (defensive guard
|
||
// refuses to operate on unrecognized profile dirs).
|
||
cleanSingletonLocks(resolveChromiumProfile());
|
||
safeUnlinkQuiet(config.stateFile);
|
||
}
|
||
// Same import.meta.main gate as SIGINT/SIGTERM — embedders register their
|
||
// own crash handlers.
|
||
if (import.meta.main) {
|
||
process.on('uncaughtException', (err) => {
|
||
console.error('[browse] FATAL uncaught exception:', err.message);
|
||
emergencyCleanup();
|
||
process.exit(1);
|
||
});
|
||
process.on('unhandledRejection', (err: any) => {
|
||
console.error('[browse] FATAL unhandled rejection:', err?.message || err);
|
||
emergencyCleanup();
|
||
process.exit(1);
|
||
});
|
||
}
|
||
|
||
// ─── Start ─────────────────────────────────────────────────────
|
||
/**
|
||
* Entry point for `bun run dev` and the compiled binary.
|
||
*
|
||
* Exported so embedders (gbrowser phoenix overlay) can call it
|
||
* directly with env vars set, bypassing the module-level `import.meta.main`
|
||
* gate. Phoenix's eventual fd-passing path will use `buildFetchHandler`
|
||
* directly; until that lands, calling `start()` from a non-main entry is
|
||
* supported via env (AUTH_TOKEN, BROWSE_PORT, BROWSE_OWN_SIGNALS).
|
||
*/
|
||
/**
|
||
* Build a request handler set for the browse daemon. Embedders (gbrowser
|
||
* phoenix overlay) call this directly with their own cfg to compose overlay
|
||
* routes via cfg.beforeRoute, pass a pre-launched cfg.browserManager, and
|
||
* opt out of terminal-agent teardown via cfg.ownsTerminalAgent (default
|
||
* true, set to false when the embedder runs its own PTY server). The CLI
|
||
* path calls this through start() with env-derived defaults and explicit
|
||
* cfg.ownsTerminalAgent: true — externally-observable behavior is identical.
|
||
*
|
||
* Auth state lives ENTIRELY inside the factory closure: cfg.authToken is the
|
||
* single source of truth for the bearer secret, factory-scoped validateAuth
|
||
* closes over it, and factory-scoped shutdown closes the cfg-provided
|
||
* browserManager. Module-level lifecycle singletons (LOCAL_LISTEN_PORT,
|
||
* tunnelActive, inspector state) intentionally STAY at module scope; see
|
||
* the v1.35.0.0 CHANGELOG entry for the architectural rationale.
|
||
*
|
||
* The returned ServerHandle is callable directly. Bun.serve is the caller's
|
||
* responsibility — embedders may fd-pass; CLI uses Bun.serve normally.
|
||
*/
|
||
export function buildFetchHandler(cfg: ServerConfig): ServerHandle {
|
||
if (!cfg.authToken || cfg.authToken.length < 16) {
|
||
throw new Error('buildFetchHandler: cfg.authToken must be a non-empty string >= 16 chars');
|
||
}
|
||
if (!cfg.browserManager) {
|
||
throw new Error('buildFetchHandler: cfg.browserManager is required');
|
||
}
|
||
|
||
// Re-run init with cfg-provided values. ensureStateDir is idempotent
|
||
// (mkdir -p); initAuditLog is idempotent (sets a module string);
|
||
// initRegistry is idempotent for same-token, throws for different-token.
|
||
// Owning init here (instead of at module load) means cfg.authToken is the
|
||
// single source of truth for the registry root token.
|
||
ensureStateDir(cfg.config);
|
||
initAuditLog(cfg.config.auditLog);
|
||
initRegistry(cfg.authToken);
|
||
|
||
const { authToken, browserManager: cfgBrowserManager, startTime, beforeRoute, browsePort } = cfg;
|
||
// Strict opt-out: only explicit `false` flips the gate. Any other value
|
||
// (undefined, truthy non-bool from a JS caller bypassing TS, etc.) defaults
|
||
// to gstack-owns. Matches the "default-true preserves CLI bit-for-bit"
|
||
// premise even under malformed cfg.
|
||
const ownsTerminalAgent = cfg.ownsTerminalAgent === false ? false : true;
|
||
|
||
// ─── Terminal-Agent Watchdog (v1.44+) ─────────────────────────────
|
||
//
|
||
// The terminal-agent process can die independently of the server: SIGKILL
|
||
// from the OS OOM killer, an uncaught exception under load, an external
|
||
// `pkill` from a sibling debugging session. Pre-v1.44 the sidebar would
|
||
// see the broken connection and stay broken until the user reloaded.
|
||
// Now: 60s ticker checks the recorded agent PID, respawns via the shared
|
||
// spawnTerminalAgent helper if dead.
|
||
//
|
||
// Identity-based — uses readAgentRecord + isProcessAlive, NOT a process
|
||
// name probe. Critical: prevents respawning around a slow-but-alive agent
|
||
// (which would create split-brain — two agents writing the port file,
|
||
// tokens diverging between them, mystery PTY upgrade failures).
|
||
//
|
||
// Crash-loop guard: 3 respawn attempts inside 60s → stop trying and emit
|
||
// a one-line error. Manual `forceRestart` from the sidebar clears the
|
||
// history (the user is the explicit signal to retry).
|
||
//
|
||
// Only active when ownsTerminalAgent === true. Embedders that pre-launch
|
||
// their own PTY server (gbrowser phoenix overlay) must not be auto-respawned
|
||
// by us — their lifecycle is their concern.
|
||
let agentWatchdogInterval: ReturnType<typeof setInterval> | null = null;
|
||
const respawnHistory: number[] = [];
|
||
const AGENT_WATCHDOG_TICK_MS = parseInt(
|
||
process.env.GSTACK_AGENT_WATCHDOG_TICK_MS || '60000',
|
||
10,
|
||
);
|
||
const RESPAWN_GUARD_MAX = 3;
|
||
// The guard window MUST span enough ticks for RESPAWN_GUARD_MAX respawns to
|
||
// land inside it. This was a fixed 60_000 against a 60_000 tick, so at most
|
||
// ONE respawn could ever be in the window and `respawnHistory.length >= 3`
|
||
// was unreachable — the guard could not fire at the default tick rate, and a
|
||
// steady one-per-tick leak ran unbounded instead of stopping after 3. Scale
|
||
// with the tick so the intent ("3 crashes in quick succession → stop") holds
|
||
// at any tick value: 3 respawns within 5 ticks trips it.
|
||
const RESPAWN_GUARD_WINDOW_MS = Math.max(
|
||
60_000,
|
||
AGENT_WATCHDOG_TICK_MS * (RESPAWN_GUARD_MAX + 2),
|
||
);
|
||
let agentRespawnGuardTripped = false;
|
||
|
||
if (ownsTerminalAgent) {
|
||
agentWatchdogInterval = setInterval(() => {
|
||
if (isShuttingDown) return;
|
||
if (agentRespawnGuardTripped) return;
|
||
const stateDir = path.dirname(cfg.config.stateFile);
|
||
const record = readAgentRecord(stateDir);
|
||
// If the record exists and the PID is alive, the agent is healthy
|
||
// (or at least still answering signal 0). Slow-but-alive agents
|
||
// intentionally fall through here — split-brain is worse than
|
||
// unresponsiveness, and slow recovery is handled by the user via
|
||
// restart.
|
||
if (record && isProcessAlive(record.pid)) return;
|
||
// Either no record (never spawned, or cleaned up after crash) or
|
||
// PID is dead. Try to respawn.
|
||
const now = Date.now();
|
||
while (respawnHistory.length && now - respawnHistory[0] > RESPAWN_GUARD_WINDOW_MS) {
|
||
respawnHistory.shift();
|
||
}
|
||
if (respawnHistory.length >= RESPAWN_GUARD_MAX) {
|
||
agentRespawnGuardTripped = true;
|
||
console.error(
|
||
`[browse] terminal-agent respawn guard tripped (${RESPAWN_GUARD_MAX} crashes in ${RESPAWN_GUARD_WINDOW_MS / 1000}s) — manual restart required`,
|
||
);
|
||
return;
|
||
}
|
||
respawnHistory.push(now);
|
||
try {
|
||
const pid = spawnTerminalAgent({
|
||
stateFile: cfg.config.stateFile,
|
||
serverPort: cfg.browsePort,
|
||
ownerPid: process.pid,
|
||
cwd: cfg.config.projectDir,
|
||
});
|
||
if (pid) {
|
||
console.log(`[browse] terminal-agent respawned by watchdog (PID: ${pid})`);
|
||
} else {
|
||
console.warn('[browse] terminal-agent respawn skipped — script not found on disk');
|
||
}
|
||
} catch (err: any) {
|
||
console.warn('[browse] terminal-agent respawn failed:', err?.message || err);
|
||
}
|
||
}, AGENT_WATCHDOG_TICK_MS);
|
||
// Detach the watchdog timer from Node's event-loop ref count so a
|
||
// healthy idle process can still exit cleanly if everything else is
|
||
// also unref'd. Bun's setInterval returns a Timer with unref().
|
||
(agentWatchdogInterval as any)?.unref?.();
|
||
}
|
||
|
||
// Factory-scoped validateAuth. Closes over cfg.authToken so every internal
|
||
// auth check sees the same token the routes receive. Module-level
|
||
// validateAuth was deleted in v1.35.0.0.
|
||
function validateAuth(req: Request): boolean {
|
||
const header = req.headers.get('authorization');
|
||
if (header === null) return false;
|
||
// Constant-time compare so a byte-by-byte early-exit can't leak the token
|
||
// prefix via response timing. timingSafeEqual requires equal-length inputs,
|
||
// so the length check gates it (the length itself is not secret).
|
||
const got = Buffer.from(header);
|
||
const want = Buffer.from(`Bearer ${authToken}`);
|
||
return got.length === want.length && crypto.timingSafeEqual(got, want);
|
||
}
|
||
|
||
// Factory-scoped shutdown. Closes the cfg-provided browserManager so
|
||
// embedders that pass their own BrowserManager get correct teardown.
|
||
// Module-level shutdown was deleted in v1.35.0.0.
|
||
async function shutdown(exitCode: number = 0) {
|
||
if (isShuttingDown) return;
|
||
isShuttingDown = true;
|
||
|
||
console.log('[browse] Shutting down...');
|
||
if (ownsTerminalAgent) {
|
||
// Identity-based kill (v1.44+). Replaces the v1.43- `pkill -f
|
||
// terminal-agent\.ts` regex teardown which matched sibling gstack
|
||
// sessions on the same host. Only the PID recorded in
|
||
// `<stateDir>/terminal-agent-pid` by THIS daemon's agent is signaled.
|
||
try {
|
||
const stateDir = path.dirname(config.stateFile);
|
||
const record = readAgentRecord(stateDir);
|
||
if (record) killAgentByRecord(record, 'SIGTERM');
|
||
} catch (err: any) {
|
||
console.warn('[browse] Failed to kill terminal-agent:', err.message);
|
||
}
|
||
safeUnlinkQuiet(path.join(path.dirname(config.stateFile), 'terminal-port'));
|
||
safeUnlinkQuiet(path.join(path.dirname(config.stateFile), 'terminal-internal-token'));
|
||
safeUnlinkQuiet(agentRecordPath(path.dirname(config.stateFile)));
|
||
}
|
||
try { detachSession(); } catch (err: any) {
|
||
console.warn('[browse] Failed to detach CDP session:', err.message);
|
||
}
|
||
inspectorSubscribers.clear();
|
||
if (cfgBrowserManager.isWatching()) cfgBrowserManager.stopWatch();
|
||
clearInterval(flushInterval);
|
||
clearInterval(idleCheckInterval);
|
||
if (agentWatchdogInterval) clearInterval(agentWatchdogInterval);
|
||
// Stop the session-persist ticker BEFORE the final snapshot below —
|
||
// paired with the isShuttingDown gate inside the tick, this guarantees
|
||
// no interval snapshot can race the final one during teardown.
|
||
if (sessionPersistInterval) {
|
||
clearInterval(sessionPersistInterval);
|
||
sessionPersistInterval = null;
|
||
}
|
||
await flushBuffers();
|
||
|
||
// Final session snapshot before the browser goes away (#778). Best
|
||
// effort with a hard 2s deadline: shutdown must never hang on a wedged
|
||
// page.evaluate — after the deadline we proceed to browser close and let
|
||
// the previous interval snapshot stand (atomic writes guarantee it's
|
||
// intact). The .catch is attached to the persist promise itself so a
|
||
// late rejection after losing the race can't become an unhandled
|
||
// rejection.
|
||
if (isSessionPersistEnabled()) {
|
||
const finalSnapshot = persistSessionState(cfgBrowserManager, path.join(config.stateDir, SESSION_STATE_FILE))
|
||
.catch((err: any) => {
|
||
console.warn(`[browse] SESSION_PERSIST_FAILED at shutdown: ${err?.message ?? err}`);
|
||
});
|
||
await Promise.race([
|
||
finalSnapshot,
|
||
new Promise<void>((resolve) => setTimeout(resolve, 2_000)),
|
||
]);
|
||
}
|
||
|
||
await cfgBrowserManager.close();
|
||
|
||
cleanSingletonLocks(resolveChromiumProfile());
|
||
safeUnlinkQuiet(config.stateFile);
|
||
process.exit(exitCode);
|
||
}
|
||
|
||
// Named lifecycle helper (matches closeTunnel style). Logs failures so
|
||
// future debugging isn't blind to a stuck listener.
|
||
async function stopListeners(local: any, tunnel?: any) {
|
||
try { if (local?.stop) local.stop(true); }
|
||
catch (err: any) { console.warn('[browse] local listener stop failed:', err?.message || err); }
|
||
try { if (tunnel?.stop) tunnel.stop(true); }
|
||
catch (err: any) { console.warn('[browse] tunnel listener stop failed:', err?.message || err); }
|
||
}
|
||
|
||
// Register this handle's shutdown as the active one. Module-level
|
||
// handlers (idleCheckInterval, parent watchdog, onDisconnect, signal
|
||
// handlers) call activeShutdown so they reach THIS shutdown, not a stale
|
||
// module reference. Critical for embedders whose cfg.browserManager
|
||
// differs from the module-level instance.
|
||
activeShutdown = shutdown;
|
||
|
||
// Retarget the BrowserManager indirection at the cfg-instance so the
|
||
// module-level idleCheckTick + parent watchdog + SIGTERM handler all read
|
||
// the right connectionMode. Without this, headed embedders auto-shutdown
|
||
// after 30 min of HTTP idle because the dead module-level instance still
|
||
// reports connectionMode === 'launched'.
|
||
activeBrowserManager = cfgBrowserManager;
|
||
// Same reason as above: the watchdog reads activeBrowserManager, so the
|
||
// instance that can promote itself to headed must be the one that can
|
||
// suppress the headed parent-death branch. An embedder-supplied manager
|
||
// otherwise promotes silently and the watchdog keeps shutting down on a
|
||
// promotion it can no longer see.
|
||
cfgBrowserManager.onHeadedPromotion = suppressHeadedParentShutdown;
|
||
|
||
// Wire the cfg-instance's onDisconnect to run shutdown when the user
|
||
// closes the headed browser window. CHAIN any caller-provided handler
|
||
// instead of overwriting it: gbrowser may have set its own onDisconnect
|
||
// before calling buildFetchHandler (e.g. for snapshot/log work that needs
|
||
// to run before the process exits). Caller errors are logged but never
|
||
// block gstack shutdown — defensive symmetry with the safeUnlinkQuiet /
|
||
// safeKill philosophy in error-handling.ts.
|
||
const callerOnDisconnect = cfgBrowserManager.onDisconnect;
|
||
cfgBrowserManager.onDisconnect = async (code) => {
|
||
if (callerOnDisconnect) {
|
||
try { await callerOnDisconnect(code); }
|
||
catch (err: any) {
|
||
console.warn('[browse] caller onDisconnect threw:', err?.message ?? err);
|
||
}
|
||
}
|
||
await activeShutdown?.(code ?? 2);
|
||
};
|
||
|
||
// Substitute cfgBrowserManager for module-level browserManager in the
|
||
// dispatcher body so all browser-state reads/writes go through the cfg
|
||
// instance. Other module-level references (handleCommand, getTokenInfo,
|
||
// isRootRequest, etc.) take the token as a parameter and are passed
|
||
// `authToken` (the cfg-derived value) explicitly.
|
||
const browserManager = cfgBrowserManager;
|
||
|
||
|
||
const makeFetchHandler = (surface: Surface) => async (req: Request): Promise<Response> => {
|
||
const url = new URL(req.url);
|
||
|
||
// ─── Tunnel surface filter (runs before any route dispatch) ──
|
||
if (surface === 'tunnel') {
|
||
const isGetConnect = req.method === 'GET' && url.pathname === '/connect';
|
||
const allowed = TUNNEL_PATHS.has(url.pathname);
|
||
if (!allowed && !isGetConnect) {
|
||
logTunnelDenial(req, url, 'path_not_on_tunnel');
|
||
return new Response(JSON.stringify({ error: 'Not found' }), {
|
||
status: 404, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
if (isRootRequest(req)) {
|
||
logTunnelDenial(req, url, 'root_token_on_tunnel');
|
||
return new Response(JSON.stringify({
|
||
error: 'Root token rejected on tunnel surface',
|
||
hint: 'Remote agents must pair via /connect to receive a scoped token.',
|
||
}), { status: 403, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
if (url.pathname !== '/connect' && !getTokenInfo(req)) {
|
||
logTunnelDenial(req, url, 'missing_scoped_token');
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
}
|
||
|
||
// beforeRoute overlay hook (v1.35.0.0). Runs AFTER the tunnel surface
|
||
// filter and BEFORE per-route dispatch. Pre-resolves bearer auth once
|
||
// so the hook receives TokenInfo | null. Note: getTokenInfo returns null
|
||
// for both missing AND invalid bearer — see the ServerConfig.beforeRoute
|
||
// JSDoc for the security implications.
|
||
if (beforeRoute) {
|
||
const auth = getTokenInfo(req);
|
||
const overlayResp = await beforeRoute(req, surface, auth);
|
||
if (overlayResp) return overlayResp;
|
||
}
|
||
|
||
// GET /connect — alive probe. Unauth on both surfaces. Used by /pair
|
||
// and /tunnel/start to detect dead ngrok tunnels via the tunnel URL,
|
||
// since /health is not tunnel-reachable under the dual-listener design.
|
||
//
|
||
// Shares the same rate limit as POST /connect — otherwise a tunnel
|
||
// caller can probe unlimited GETs and lock out nothing, which makes
|
||
// the endpoint a free daemon-enumeration surface.
|
||
if (url.pathname === '/connect' && req.method === 'GET') {
|
||
if (!checkConnectRateLimit()) {
|
||
return new Response(JSON.stringify({ error: 'Rate limited' }), {
|
||
status: 429, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
return new Response(JSON.stringify({ alive: true }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// Cookie picker routes — HTML page unauthenticated, data/action routes require auth
|
||
if (url.pathname.startsWith('/cookie-picker')) {
|
||
return handleCookiePickerRoute(url, req, browserManager, authToken);
|
||
}
|
||
|
||
// Welcome page — served when GStack Browser launches in headed mode
|
||
if (url.pathname === '/welcome') {
|
||
const welcomePath = (() => {
|
||
// Gate GSTACK_SLUG on a strict regex BEFORE interpolating it into
|
||
// the filesystem path. Without this, a slug like "../../etc/passwd"
|
||
// would resolve to ~/.gstack/projects/../../etc/passwd/... — path
|
||
// traversal. Not exploitable today (attacker needs local env-var
|
||
// access), but the gate is one regex and buys us defense-in-depth.
|
||
const rawSlug = process.env.GSTACK_SLUG || 'unknown';
|
||
const slug = /^[a-z0-9_-]+$/.test(rawSlug) ? rawSlug : 'unknown';
|
||
const homeDir = process.env.HOME || process.env.USERPROFILE || '/tmp';
|
||
const projectWelcome = `${homeDir}/.gstack/projects/${slug}/designs/welcome-page-20260331/finalized.html`;
|
||
if (fs.existsSync(projectWelcome)) return projectWelcome;
|
||
// Fallback: built-in welcome page from gstack install. Reject
|
||
// SKILL_ROOT values containing '..' for the same defense-in-depth
|
||
// reason as the GSTACK_SLUG regex above. Not exploitable today
|
||
// (env set at install time), but the gate is one check.
|
||
const rawSkillRoot = process.env.GSTACK_SKILL_ROOT || `${homeDir}/.claude/skills/gstack`;
|
||
if (rawSkillRoot.includes('..')) return null;
|
||
const builtinWelcome = `${rawSkillRoot}/browse/src/welcome.html`;
|
||
if (fs.existsSync(builtinWelcome)) return builtinWelcome;
|
||
return null;
|
||
})();
|
||
if (welcomePath) {
|
||
try {
|
||
const html = require('fs').readFileSync(welcomePath, 'utf-8');
|
||
return new Response(html, { headers: { 'Content-Type': 'text/html; charset=utf-8' } });
|
||
} catch (err: any) {
|
||
console.error('[browse] Failed to read welcome page:', welcomePath, err.message);
|
||
}
|
||
}
|
||
// No welcome page found — serve a simple fallback (avoid ERR_UNSAFE_REDIRECT on Windows)
|
||
return new Response(
|
||
`<!DOCTYPE html><html><head><title>GStack Browser</title>
|
||
<style>body{background:#111;color:#fff;font-family:system-ui;display:flex;align-items:center;justify-content:center;height:100vh;margin:0;}
|
||
.msg{text-align:center;opacity:.7;}.gold{color:#f5a623;font-size:2em;margin-bottom:12px;}</style></head>
|
||
<body><div class="msg"><div class="gold">◈</div><p>GStack Browser ready.</p><p style="font-size:.85em">Waiting for commands from Claude Code.</p></div></body></html>`,
|
||
{ status: 200, headers: { 'Content-Type': 'text/html; charset=utf-8' } }
|
||
);
|
||
}
|
||
|
||
// ─── POST /extension-token — pinned-origin token bootstrap ──────
|
||
//
|
||
// The ONLY endpoint that hands out AUTH_TOKEN. GET /health used to
|
||
// carry the token (headed mode + any chrome-extension:// Origin),
|
||
// which meant ANY extension — or any localhost caller in headed
|
||
// mode — could read the root token. Now the token is released only
|
||
// to the one extension identity we ship: the Origin header must be
|
||
// exactly `chrome-extension://<GSTACK_EXTENSION_ID>`, where the ID
|
||
// is pinned by the "key" field in extension/manifest.json (derive
|
||
// it with `bun browse/scripts/extension-id.ts`). Chrome sets Origin
|
||
// on cross-origin POSTs from extension contexts and web pages
|
||
// cannot forge a chrome-extension:// Origin.
|
||
//
|
||
// Local listener only: NEVER added to TUNNEL_PATHS, so the tunnel
|
||
// surface 404s it by default-deny.
|
||
if (url.pathname === '/extension-token' && req.method === 'POST') {
|
||
// Defense-in-depth alongside the 127.0.0.1 bind: a DNS-rebinding
|
||
// page can't present a localhost Host header. Host arrives as
|
||
// '127.0.0.1:34567', so parse out the hostname — never compare
|
||
// the raw header (which carries the port) against a literal.
|
||
let hostname: string | null = null;
|
||
try {
|
||
hostname = new URL(`http://${req.headers.get('host') ?? ''}`).hostname;
|
||
} catch (err) {
|
||
if (!(err instanceof TypeError)) throw err; // TypeError = malformed Host
|
||
}
|
||
const originOk =
|
||
req.headers.get('origin') === `chrome-extension://${GSTACK_EXTENSION_ID}`;
|
||
const hostOk = hostname === '127.0.0.1' || hostname === 'localhost';
|
||
if (!originOk || !hostOk) {
|
||
// No detail in the body — don't teach a probing caller which
|
||
// check failed.
|
||
return new Response(JSON.stringify({ error: 'Forbidden' }), {
|
||
status: 403, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
return new Response(JSON.stringify({ token: authToken }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// Health check — no auth required, does NOT reset idle timer.
|
||
// NEVER carries a token in any mode: token bootstrap is
|
||
// POST /extension-token (pinned extension Origin) and shell auth
|
||
// is POST /pty-session. Liveness/status only.
|
||
if (url.pathname === '/health') {
|
||
const healthy = await browserManager.isHealthy();
|
||
return new Response(JSON.stringify({
|
||
status: healthy ? 'healthy' : 'unhealthy',
|
||
mode: browserManager.getConnectionMode(),
|
||
uptime: Math.floor((Date.now() - startTime) / 1000),
|
||
tabs: browserManager.getTabCount(),
|
||
// No `security` field (#2557): the only writer of the status it
|
||
// reported (sidebar-agent.ts's session-state file) went away with
|
||
// the chat path, so it read from a file nothing wrote — reporting
|
||
// a permanent 'inactive', or a stale false-green 'protected'
|
||
// wherever an old state file survived on disk. The live defenses
|
||
// (content-security L1-L3, the L4 sidecar on /pty-inject-scan)
|
||
// report through their own call sites, not through /health.
|
||
// Terminal-agent discovery. ONLY a port number — never a token.
|
||
// Tokens flow via the /pty-session HttpOnly cookie path. See
|
||
// `pty-session-cookie.ts` for the rationale (codex outside-voice
|
||
// finding #2: don't reuse this endpoint for shell auth).
|
||
terminalPort: readTerminalPort(),
|
||
}), {
|
||
status: 200,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// ─── /pty-session — mint sessionId + lease + attachToken ─────────
|
||
//
|
||
// v1.44+ four-tuple shape:
|
||
// { terminalPort, sessionId, attachToken, leaseExpiresAt }
|
||
//
|
||
// - sessionId : stable, non-secret. Safe to log. Identifies "this
|
||
// terminal" across re-attaches.
|
||
// - attachToken : short-lived (30 min wall, single attach in practice
|
||
// since the agent revokes on WS close). Bearer for
|
||
// the /ws upgrade.
|
||
// - leaseExpiresAt: client-visible deadline for the lease. Re-attach
|
||
// only works inside this window.
|
||
//
|
||
// The lease + attachToken are minted together so a successful
|
||
// /pty-session is one round trip. Re-attach mints a fresh attachToken
|
||
// for the SAME sessionId via /pty-session/reattach.
|
||
//
|
||
// NEVER added to TUNNEL_PATHS — the tunnel surface 404s any
|
||
// /pty-session attempt by default-deny.
|
||
if (url.pathname === '/pty-session' && req.method === 'POST') {
|
||
if (!validateAuth(req)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const port = readTerminalPort();
|
||
if (!port) {
|
||
return new Response(JSON.stringify({
|
||
error: 'terminal-agent not ready',
|
||
}), { status: 503, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
const lease = mintLease();
|
||
const minted = mintPtySessionToken();
|
||
const granted = await grantPtyToken(minted.token, lease.sessionId);
|
||
if (!granted) {
|
||
revokePtySessionToken(minted.token);
|
||
revokeLease(lease.sessionId);
|
||
return new Response(JSON.stringify({
|
||
error: 'failed to grant terminal session',
|
||
}), { status: 503, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
return new Response(JSON.stringify({
|
||
terminalPort: port,
|
||
sessionId: lease.sessionId,
|
||
attachToken: minted.token,
|
||
leaseExpiresAt: lease.expiresAt,
|
||
// Legacy alias — extensions still on the v1.43 wire shape keep
|
||
// working. Drop after one minor release once dogfood confirms.
|
||
ptySessionToken: minted.token,
|
||
expiresAt: minted.expiresAt,
|
||
}), {
|
||
status: 200,
|
||
headers: {
|
||
'Content-Type': 'application/json',
|
||
'Set-Cookie': buildPtySetCookie(minted.token),
|
||
},
|
||
});
|
||
}
|
||
|
||
// ─── /pty-session/reattach — mint fresh attachToken for existing sessionId
|
||
//
|
||
// Used by Commit 3's re-attach loop on the client. Validates the
|
||
// lease (rejects unknown/expired sessionId with 410 Gone), mints a
|
||
// fresh short-lived attachToken bound to the same sessionId, and
|
||
// pushes it to the agent. The client opens a new WS with the new
|
||
// token; the agent matches the sessionId binding and re-attaches
|
||
// to the existing PtySession (kept alive for the 60s detach
|
||
// window — Commit 3 wires that side).
|
||
if (url.pathname === '/pty-session/reattach' && req.method === 'POST') {
|
||
if (!validateAuth(req)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const port = readTerminalPort();
|
||
if (!port) {
|
||
return new Response(JSON.stringify({ error: 'terminal-agent not ready' }), {
|
||
status: 503, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
let body: any;
|
||
try { body = await req.json(); } catch { body = null; }
|
||
const sessionId = typeof body?.sessionId === 'string' ? body.sessionId : null;
|
||
const v = sessionId ? validateLease(sessionId) : { ok: false };
|
||
if (!v.ok) {
|
||
// 410 Gone — session window has closed (lease expired or never
|
||
// existed). Client must fall back to /pty-session for a brand-new
|
||
// session.
|
||
return new Response(JSON.stringify({ error: 'lease expired or unknown' }), {
|
||
status: 410, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const minted = mintPtySessionToken();
|
||
const granted = await grantPtyToken(minted.token, sessionId!);
|
||
if (!granted) {
|
||
revokePtySessionToken(minted.token);
|
||
return new Response(JSON.stringify({ error: 'failed to grant attach token' }), {
|
||
status: 503, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
return new Response(JSON.stringify({
|
||
terminalPort: port,
|
||
sessionId,
|
||
attachToken: minted.token,
|
||
leaseExpiresAt: v.ok ? v.expiresAt : 0,
|
||
}), { status: 200, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
|
||
// ─── /pty-restart — one-transaction kill + fresh mint ────────────
|
||
//
|
||
// The Restart button. Synchronously disposes the caller's existing
|
||
// PtySession on the agent, revokes the old lease, mints a fresh
|
||
// sessionId + lease + attachToken, and returns the new 4-tuple in
|
||
// one response. Zero race window between kill and mint (codex T2
|
||
// + D8 of the eng review).
|
||
if (url.pathname === '/pty-restart' && req.method === 'POST') {
|
||
if (!validateAuth(req)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const port = readTerminalPort();
|
||
if (!port) {
|
||
return new Response(JSON.stringify({ error: 'terminal-agent not ready' }), {
|
||
status: 503, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
let body: any;
|
||
try { body = await req.json(); } catch { body = null; }
|
||
const oldSessionId = typeof body?.sessionId === 'string' ? body.sessionId : null;
|
||
// Best-effort dispose. Missing/unknown sessionId is non-fatal —
|
||
// the client may be doing a "restart from scratch" with no prior
|
||
// session (e.g. ENDED state). The fresh mint always proceeds.
|
||
if (oldSessionId) {
|
||
await restartPtySession(oldSessionId);
|
||
revokeLease(oldSessionId);
|
||
}
|
||
const lease = mintLease();
|
||
const minted = mintPtySessionToken();
|
||
const granted = await grantPtyToken(minted.token, lease.sessionId);
|
||
if (!granted) {
|
||
revokePtySessionToken(minted.token);
|
||
revokeLease(lease.sessionId);
|
||
return new Response(JSON.stringify({ error: 'failed to grant terminal session' }), {
|
||
status: 503, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
return new Response(JSON.stringify({
|
||
terminalPort: port,
|
||
sessionId: lease.sessionId,
|
||
attachToken: minted.token,
|
||
leaseExpiresAt: lease.expiresAt,
|
||
}), { status: 200, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
|
||
// ─── /pty-dispose — explicit teardown (pagehide / browser quit) ──
|
||
//
|
||
// sendBeacon-compatible: accepts the auth token in the BODY so the
|
||
// extension's pagehide handler can fire it without setting headers
|
||
// (sendBeacon doesn't support custom headers). Codex T3 fix —
|
||
// without this, every browser quit + sidebar close leaves a zombie
|
||
// PTY alive for the 60s detach window (Commit 3).
|
||
if (url.pathname === '/pty-dispose' && req.method === 'POST') {
|
||
let body: any;
|
||
try { body = await req.json(); } catch { body = null; }
|
||
const authTokenFromBody = typeof body?.authToken === 'string' ? body.authToken : null;
|
||
// Accept either header bearer OR body authToken. Both must match
|
||
// the root auth token; otherwise reject.
|
||
const headerToken = extractToken(req);
|
||
const authedByHeader = headerToken !== null && headerToken === authToken;
|
||
const authedByBody = authTokenFromBody !== null && authTokenFromBody === authToken;
|
||
if (!authedByHeader && !authedByBody) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const sessionId = typeof body?.sessionId === 'string' ? body.sessionId : null;
|
||
if (sessionId) {
|
||
await restartPtySession(sessionId);
|
||
revokeLease(sessionId);
|
||
}
|
||
return new Response(JSON.stringify({ ok: true }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// ─── /internal/lease-refresh — loopback from terminal-agent on keepalive
|
||
//
|
||
// T6 PTY-only idle reset (codex outside-voice fix): the headless
|
||
// daemon's idle timer must reset only on active PTY usage, not on
|
||
// every passive SSE consumer. Terminal-agent calls this endpoint
|
||
// (lazily, only when its cached lease is within 5 min of expiry)
|
||
// on its 25s keepalive cycle. Refreshing the lease here also bumps
|
||
// lastActivity so the daemon stays alive while a sidebar terminal
|
||
// is actively in use.
|
||
//
|
||
// INTERNAL endpoint — bound to the root authToken so an external
|
||
// caller can't refresh another user's lease. Body: {sessionId}.
|
||
if (url.pathname === '/internal/lease-refresh' && req.method === 'POST') {
|
||
if (!validateAuth(req)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
let body: any;
|
||
try { body = await req.json(); } catch { body = null; }
|
||
const sessionId = typeof body?.sessionId === 'string' ? body.sessionId : null;
|
||
const r = sessionId ? refreshLease(sessionId) : { ok: false };
|
||
if (!r.ok) {
|
||
return new Response(JSON.stringify({ error: 'lease expired or unknown' }), {
|
||
status: 410, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
// T6: PTY activity resets the daemon idle timer.
|
||
resetIdleTimer();
|
||
return new Response(JSON.stringify({ ok: true, expiresAt: r.expiresAt }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// ─── /pty-inject-scan — pre-inject prompt-injection scan for the
|
||
// extension's gstackInjectToTerminal callers. The extension routes
|
||
// every page-derived text through this endpoint BEFORE writing to
|
||
// the PTY (#1370). Local-only by intent: not added to the tunnel
|
||
// allowlist; root-token auth required. Sidecar absence degrades to
|
||
// L4 unavailable (extension shows WARN + user confirm per D7).
|
||
if (url.pathname === '/pty-inject-scan' && req.method === 'POST') {
|
||
if (!validateAuth(req)) {
|
||
return new Response(
|
||
JSON.stringify({ error: 'Unauthorized' }, sanitizeReplacer),
|
||
{ status: 401, headers: { 'Content-Type': 'application/json' } },
|
||
);
|
||
}
|
||
// 64KB request cap. Defense against accidentally posting an
|
||
// entire page DOM into the PTY path.
|
||
const contentLength = Number(req.headers.get('content-length') || '0');
|
||
if (contentLength > 64 * 1024) {
|
||
return new Response(
|
||
JSON.stringify({ error: 'payload-too-large', limit: 65536 }, sanitizeReplacer),
|
||
{ status: 413, headers: { 'Content-Type': 'application/json' } },
|
||
);
|
||
}
|
||
let body: { text?: unknown; origin?: unknown } = {};
|
||
try {
|
||
body = (await req.json()) as { text?: unknown; origin?: unknown };
|
||
} catch {
|
||
return new Response(
|
||
JSON.stringify({ error: 'malformed-json' }, sanitizeReplacer),
|
||
{ status: 400, headers: { 'Content-Type': 'application/json' } },
|
||
);
|
||
}
|
||
const text = typeof body.text === 'string' ? body.text : '';
|
||
const origin = typeof body.origin === 'string' ? body.origin : 'unknown';
|
||
if (text.length === 0) {
|
||
return new Response(
|
||
JSON.stringify({ error: 'missing-text' }, sanitizeReplacer),
|
||
{ status: 400, headers: { 'Content-Type': 'application/json' } },
|
||
);
|
||
}
|
||
|
||
// L1-L3 honest accounting (codex review correction):
|
||
// - URL blocklist forced to BLOCK in PTY context (override
|
||
// BROWSE_CONTENT_FILTER default — page-derived text in the
|
||
// REPL is a higher-risk surface than ordinary tool output).
|
||
// - L4 ML classifier via the sidecar when available.
|
||
// - L1-L3 envelope/datamarking is INFORMATIONAL only; the
|
||
// verdict is driven by the URL blocklist + L4.
|
||
// See CLAUDE.md "Sidebar security stack" + plan §"L1-L3 honest
|
||
// accounting".
|
||
let verdict: 'PASS' | 'WARN' | 'BLOCK' = 'PASS';
|
||
const reasons: string[] = [];
|
||
|
||
// Quick URL-blocklist check (re-uses the security module's
|
||
// pure-string helpers — no @huggingface/transformers dep).
|
||
// Pattern: text containing a known bad-actor domain → BLOCK.
|
||
if (/(\bbit\.ly|\btinyurl\.com|\bdiscord\.gg)/i.test(text)) {
|
||
verdict = 'BLOCK';
|
||
reasons.push('url-blocklist');
|
||
}
|
||
|
||
// L4 sidecar scan if available.
|
||
const sidecarAvail = isSidecarAvailable();
|
||
let l4: { available: boolean; verdict?: unknown; error?: string } = {
|
||
available: sidecarAvail.available,
|
||
};
|
||
if (sidecarAvail.available && verdict !== 'BLOCK') {
|
||
try {
|
||
const { verdict: layerVerdict } = await scanWithSidecar(text, {
|
||
timeoutMs: 5000,
|
||
});
|
||
l4 = { available: true, verdict: layerVerdict };
|
||
// LayerSignal shape: { verdict: 'safe'|'suspicious'|'unsafe', ... }
|
||
const lv = (layerVerdict as { verdict?: string })?.verdict;
|
||
if (lv === 'unsafe') {
|
||
verdict = 'BLOCK';
|
||
reasons.push('l4-unsafe');
|
||
} else if (lv === 'suspicious') {
|
||
verdict = 'WARN';
|
||
reasons.push('l4-suspicious');
|
||
}
|
||
} catch (err) {
|
||
l4 = {
|
||
available: false,
|
||
error: err instanceof Error ? err.message : String(err),
|
||
};
|
||
// L4 failure during scan: degrade to WARN per D7.
|
||
if (verdict === 'PASS') {
|
||
verdict = 'WARN';
|
||
reasons.push('l4-unavailable');
|
||
}
|
||
}
|
||
} else if (!sidecarAvail.available && verdict === 'PASS') {
|
||
verdict = 'WARN';
|
||
reasons.push(`l4-unavailable:${sidecarAvail.reason ?? 'unknown'}`);
|
||
}
|
||
|
||
// BLOCK decisions are surfaced in the response shape; the
|
||
// existing writeDecision audit log is tab-scoped (per-page) and
|
||
// doesn't fit the PTY surface. The extension logs the BLOCK
|
||
// event into its own activity feed on receipt, which keeps the
|
||
// audit signal observable without bolting a new attempts.jsonl
|
||
// onto the server.
|
||
|
||
return new Response(
|
||
JSON.stringify(
|
||
{ verdict, reasons, l4, datamark: '<untrusted-page-content>' },
|
||
sanitizeReplacer,
|
||
),
|
||
{ status: 200, headers: { 'Content-Type': 'application/json' } },
|
||
);
|
||
}
|
||
|
||
// ─── /connect — setup key exchange for /pair-agent ceremony ────
|
||
if (url.pathname === '/connect' && req.method === 'POST') {
|
||
if (!checkConnectRateLimit()) {
|
||
return new Response(JSON.stringify({
|
||
error: 'Too many connection attempts. Wait 1 minute.',
|
||
}), { status: 429, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
try {
|
||
const connectBody = await req.json() as { setup_key?: string };
|
||
if (!connectBody.setup_key) {
|
||
return new Response(JSON.stringify({ error: 'Missing setup_key' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const session = exchangeSetupKey(connectBody.setup_key);
|
||
if (!session) {
|
||
return new Response(JSON.stringify({
|
||
error: 'Invalid, expired, or already-used setup key',
|
||
}), { status: 401, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
console.log(`[browse] Remote agent connected: ${session.clientId} (scopes: ${session.scopes.join(',')})`);
|
||
return new Response(JSON.stringify({
|
||
token: session.token,
|
||
expires: session.expiresAt,
|
||
scopes: session.scopes,
|
||
agent: session.clientId,
|
||
}), { status: 200, headers: { 'Content-Type': 'application/json' } });
|
||
} catch {
|
||
return new Response(JSON.stringify({ error: 'Invalid request body' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
}
|
||
|
||
// ─── /token — mint scoped tokens (root-only) ──────────────────
|
||
if (url.pathname === '/token' && req.method === 'POST') {
|
||
if (!isRootRequest(req)) {
|
||
return new Response(JSON.stringify({
|
||
error: 'Only the root token can mint sub-tokens',
|
||
}), { status: 403, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
try {
|
||
const tokenBody = await req.json() as any;
|
||
if (!tokenBody.clientId) {
|
||
return new Response(JSON.stringify({ error: 'Missing clientId' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const session = createToken({
|
||
clientId: tokenBody.clientId,
|
||
scopes: tokenBody.scopes,
|
||
domains: tokenBody.domains,
|
||
tabPolicy: tokenBody.tabPolicy,
|
||
rateLimit: tokenBody.rateLimit,
|
||
expiresSeconds: tokenBody.expiresSeconds,
|
||
});
|
||
return new Response(JSON.stringify({
|
||
token: session.token,
|
||
expires: session.expiresAt,
|
||
scopes: session.scopes,
|
||
agent: session.clientId,
|
||
}), { status: 200, headers: { 'Content-Type': 'application/json' } });
|
||
} catch (err) {
|
||
// Name the caller's typo (bad scope, negative rateLimit, reserved
|
||
// clientId) instead of hiding it behind the generic body error.
|
||
if (err instanceof InvalidScopeError || err instanceof ReservedClientIdError) {
|
||
return new Response(JSON.stringify({ error: err.message }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
return new Response(JSON.stringify({ error: 'Invalid request body' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
}
|
||
|
||
// ─── /token/:clientId — revoke a scoped token (root-only) ─────
|
||
if (url.pathname.startsWith('/token/') && req.method === 'DELETE') {
|
||
if (!isRootRequest(req)) {
|
||
return new Response(JSON.stringify({ error: 'Root token required' }), {
|
||
status: 403, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
// decodeURIComponent so CLI-encoded names (spaces, UTF-8) round-trip.
|
||
let clientId: string;
|
||
try {
|
||
clientId = decodeURIComponent(url.pathname.slice('/token/'.length));
|
||
} catch {
|
||
return new Response(JSON.stringify({ error: 'Malformed client ID encoding' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const revoked = revokeToken(clientId);
|
||
// Release tabs UNCONDITIONALLY: ownership outlives the token (it clears
|
||
// only on tab close), so a client whose token already expired can still
|
||
// own tabs. Gating release on a revoke hit would orphan that ownership
|
||
// and let a same-name re-pair inherit an authenticated tab.
|
||
const tabsReleased = browserManager.releaseClientTabs(clientId).length;
|
||
if (!revoked && tabsReleased === 0) {
|
||
return new Response(JSON.stringify({ error: `Agent "${clientId}" not found` }), {
|
||
status: 404, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
console.log(`[browse] Revoked ${revoked} token(s), released ${tabsReleased} tab(s) for: ${clientId}`);
|
||
return new Response(JSON.stringify({ revoked: clientId, tokens_deleted: revoked, tabs_released: tabsReleased }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// ─── /agents — list connected agents (root-only) ──────────────
|
||
if (url.pathname === '/agents' && req.method === 'GET') {
|
||
if (!isRootRequest(req)) {
|
||
return new Response(JSON.stringify({ error: 'Root token required' }), {
|
||
status: 403, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
// includeSetup: pending (unexchanged) setup keys are live grants the
|
||
// operator must be able to see — without them, revoking a paired-but-
|
||
// never-connected agent "works" while the list shows nothing.
|
||
const agents = listTokens({ includeSetup: true }).map(t => ({
|
||
clientId: t.clientId,
|
||
scopes: t.scopes,
|
||
domains: t.domains,
|
||
expiresAt: t.expiresAt,
|
||
commandCount: t.commandCount,
|
||
createdAt: t.createdAt,
|
||
pending: t.type === 'setup',
|
||
}));
|
||
return new Response(JSON.stringify({ agents }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// ─── /pair — create setup key for pair-agent ceremony (root-only) ───
|
||
if (url.pathname === '/pair' && req.method === 'POST') {
|
||
if (!isRootRequest(req)) {
|
||
return new Response(JSON.stringify({ error: 'Root token required' }), {
|
||
status: 403, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
try {
|
||
const pairBody = await req.json() as any;
|
||
// Reject a reserved/invalid clientId up front (createSetupKey enforces
|
||
// it too, but this makes the 400 unambiguous and skips the teardown).
|
||
if (pairBody.clientId !== undefined) assertValidClientId(pairBody.clientId);
|
||
// Default: DEFAULT_PAIR_SCOPES (full page access). The trust boundary
|
||
// is the pairing ceremony itself, not the scope. --control adds
|
||
// browser-wide destructive commands (stop, restart, disconnect).
|
||
// --restrict limits scope — but can never grant control: that scope
|
||
// stays behind the explicit control flag.
|
||
if (!pairBody.control && !pairBody.admin
|
||
&& Array.isArray(pairBody.scopes) && pairBody.scopes.includes('control')) {
|
||
return new Response(JSON.stringify({
|
||
error: 'The control scope requires the control flag (--control); it cannot be granted via a scopes list.',
|
||
}), { status: 400, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
const scopes = pairBody.control || pairBody.admin
|
||
? [...DEFAULT_PAIR_SCOPES, 'control' as const]
|
||
: ((pairBody.scopes || [...DEFAULT_PAIR_SCOPES]) as ScopeCategory[]);
|
||
// D1: a re-pair supersedes prior grants. ALWAYS drop stale setup keys
|
||
// so a superseded broad key can never be exchanged — this closes the
|
||
// shadow-key hole where a narrowing re-pair before the agent connects
|
||
// would otherwise leave the old broad key live. Revoke the live
|
||
// SESSION only when the new grant actually reduces access, so a
|
||
// broaden/refresh never strands a working agent mid-task. Compare
|
||
// against the resolved grant (not raw pairBody) so dropping 'control'
|
||
// or a default re-pair is classified correctly. Revoke runs BEFORE
|
||
// createSetupKey — revokeToken deletes all of a clientId's tokens, so
|
||
// minting first would nuke the fresh key.
|
||
const grant = {
|
||
scopes: [...scopes] as ScopeCategory[],
|
||
domains: pairBody.domains as string[] | undefined,
|
||
rateLimit: pairBody.rateLimit ?? 10,
|
||
tabPolicy: 'own-only' as const,
|
||
};
|
||
// Validate BEFORE any revoke (createSetupKey validates too, but that
|
||
// runs after the teardown below). A bad scope or negative rateLimit
|
||
// must 400 without knocking a live session offline — otherwise a
|
||
// reducing re-pair with a typo (--restrict red) destroys the session
|
||
// and mints no replacement.
|
||
assertValidTokenOptions(grant.scopes, grant.rateLimit);
|
||
const priorSession = pairBody.clientId ? getClientSession(pairBody.clientId) : null;
|
||
let superseded: { tokens_deleted: number; tabs_released: number } | undefined;
|
||
if (priorSession && grantReducesAccess(priorSession, grant)) {
|
||
const tokensDeleted = revokeToken(pairBody.clientId);
|
||
const tabsReleased = browserManager.releaseClientTabs(pairBody.clientId).length;
|
||
superseded = { tokens_deleted: tokensDeleted, tabs_released: tabsReleased };
|
||
console.log(`[browse] Superseded ${tokensDeleted} token(s), released ${tabsReleased} tab(s) for reducing re-pair: ${pairBody.clientId}`);
|
||
} else if (pairBody.clientId) {
|
||
revokeSetupKeys(pairBody.clientId);
|
||
// No live session, but tab ownership outlives token expiry: free any
|
||
// tabs orphaned by an expired session so this re-pair can't inherit
|
||
// an earlier incarnation's authenticated pages (mirrors DELETE
|
||
// /token's unconditional release). A live-session broaden keeps its
|
||
// tabs — the working agent still owns them.
|
||
if (!priorSession) browserManager.releaseClientTabs(pairBody.clientId);
|
||
}
|
||
const setupKey = createSetupKey({
|
||
clientId: pairBody.clientId,
|
||
scopes: [...scopes],
|
||
domains: pairBody.domains,
|
||
rateLimit: pairBody.rateLimit,
|
||
});
|
||
// Verify tunnel is actually alive before reporting it (ngrok may have died externally).
|
||
// Probe via GET /connect — under dual-listener /health is NOT on the tunnel allowlist,
|
||
// so the old probe would return 404 and always mark the tunnel as dead.
|
||
let verifiedTunnelUrl: string | null = null;
|
||
if (tunnelActive && tunnelUrl) {
|
||
try {
|
||
const probe = await fetch(`${tunnelUrl}/connect`, {
|
||
method: 'GET',
|
||
headers: { 'ngrok-skip-browser-warning': 'true' },
|
||
signal: AbortSignal.timeout(5000),
|
||
});
|
||
if (probe.ok) {
|
||
verifiedTunnelUrl = tunnelUrl;
|
||
} else {
|
||
console.warn(`[browse] Tunnel probe failed (HTTP ${probe.status}), marking tunnel as dead`);
|
||
await closeTunnel();
|
||
}
|
||
} catch {
|
||
console.warn('[browse] Tunnel probe timed out or unreachable, marking tunnel as dead');
|
||
await closeTunnel();
|
||
}
|
||
}
|
||
return new Response(JSON.stringify({
|
||
setup_key: setupKey.token,
|
||
expires_at: setupKey.expiresAt,
|
||
scopes: setupKey.scopes,
|
||
tunnel_url: verifiedTunnelUrl,
|
||
server_url: `http://127.0.0.1:${browsePort}`,
|
||
...(superseded ? { superseded } : {}),
|
||
}), { status: 200, headers: { 'Content-Type': 'application/json' } });
|
||
} catch (err) {
|
||
// Name the caller's typo (bad scope, negative rateLimit, reserved
|
||
// clientId) instead of hiding it behind the generic body error.
|
||
if (err instanceof InvalidScopeError || err instanceof ReservedClientIdError) {
|
||
return new Response(JSON.stringify({ error: err.message }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
return new Response(JSON.stringify({ error: 'Invalid request body' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
}
|
||
|
||
// ─── /tunnel/start — start ngrok tunnel on demand (root-only) ──
|
||
//
|
||
// Dual-listener model: binds a SECOND Bun.serve listener on an
|
||
// ephemeral 127.0.0.1 port dedicated to tunnel traffic, then points
|
||
// ngrok.forward() at THAT port. The existing local listener (which
|
||
// serves /extension-token, /cookie-picker, /inspector/*, welcome, etc.)
|
||
// is never exposed to ngrok.
|
||
//
|
||
// Hard fail if the tunnel listener bind fails — NEVER fall back to
|
||
// the local port, which would silently defeat the whole security
|
||
// property.
|
||
if (url.pathname === '/tunnel/start' && req.method === 'POST') {
|
||
if (!isRootRequest(req)) {
|
||
return new Response(JSON.stringify({ error: 'Root token required' }), {
|
||
status: 403, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
if (!isPairAgentEnabled()) {
|
||
// Consent-on-first-use: the /pair-agent skill asks once and sets the
|
||
// key; a direct API caller gets the same hint instead of a tunnel.
|
||
return new Response(JSON.stringify({
|
||
error: 'pair-agent is off (tunnel exposes this browser beyond the machine)',
|
||
hint: 'enable once with: gstack-config set pair_agent on — or run /pair-agent, which asks for consent and sets it',
|
||
}), { status: 403, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
if (tunnelActive && tunnelUrl && tunnelServer) {
|
||
// Verify tunnel is still alive before returning cached URL.
|
||
// Probe GET /connect (the only unauth-reachable path on the tunnel
|
||
// surface); /health is NOT tunnel-reachable under dual-listener.
|
||
try {
|
||
const probe = await fetch(`${tunnelUrl}/connect`, {
|
||
method: 'GET',
|
||
headers: { 'ngrok-skip-browser-warning': 'true' },
|
||
signal: AbortSignal.timeout(5000),
|
||
});
|
||
if (probe.ok) {
|
||
return new Response(JSON.stringify({ url: tunnelUrl, already_active: true }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
} catch {}
|
||
// Tunnel is dead — tear down cleanly before restarting
|
||
console.warn('[browse] Cached tunnel is dead, restarting...');
|
||
await closeTunnel();
|
||
}
|
||
|
||
// 1) Resolve ngrok authtoken from env / .gstack / native config
|
||
const authtoken = resolveNgrokAuthtoken();
|
||
if (!authtoken) {
|
||
return new Response(JSON.stringify({
|
||
error: 'No ngrok authtoken found',
|
||
hint: 'Run: ngrok config add-authtoken YOUR_TOKEN',
|
||
}), { status: 400, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
|
||
// 2) Bind the tunnel listener + open ngrok via the shared helper
|
||
// (see startTunnel — hard-fails the bind, cleans up both ngrok
|
||
// and the Bun listener on any post-bind failure).
|
||
const started = await startTunnel({
|
||
fetchHandler: makeFetchHandler('tunnel'),
|
||
authtoken,
|
||
consent: 'pair_agent=on (isPairAgentEnabled gate at /tunnel/start)',
|
||
});
|
||
if (!started.ok) {
|
||
return new Response(JSON.stringify({
|
||
error: started.stage === 'bind'
|
||
? `Failed to bind tunnel listener: ${started.error.message}`
|
||
: `Failed to open ngrok tunnel: ${started.error.message}`,
|
||
}), { status: 500, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
return new Response(JSON.stringify({ url: started.url }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// ─── SSE session cookie mint (auth required) ──────────────────
|
||
//
|
||
// Issues a short-lived view-only token in an HttpOnly SameSite=Strict
|
||
// cookie so EventSource calls can authenticate without putting the
|
||
// root token in a URL. The returned cookie is valid ONLY on the SSE
|
||
// endpoints (/activity/stream, /inspector/events); it is not a
|
||
// scoped token and cannot be used against /command.
|
||
//
|
||
// The extension calls this once at bootstrap with the root Bearer
|
||
// header, then opens EventSource with `withCredentials: true` which
|
||
// sends the cookie back automatically.
|
||
if (url.pathname === '/sse-session' && req.method === 'POST') {
|
||
if (!validateAuth(req)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const minted = mintSseSessionToken();
|
||
return new Response(JSON.stringify({
|
||
expiresAt: minted.expiresAt,
|
||
cookie: SSE_COOKIE_NAME,
|
||
}), {
|
||
status: 200,
|
||
headers: {
|
||
'Content-Type': 'application/json',
|
||
'Set-Cookie': buildSseSetCookie(minted.token),
|
||
},
|
||
});
|
||
}
|
||
|
||
// Refs endpoint — auth required, does NOT reset idle timer
|
||
if (url.pathname === '/refs') {
|
||
if (!validateAuth(req)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const refs = browserManager.getRefMap();
|
||
return new Response(JSON.stringify({
|
||
refs,
|
||
url: browserManager.getCurrentUrl(),
|
||
mode: browserManager.getConnectionMode(),
|
||
}), {
|
||
status: 200,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// Activity stream — SSE, auth required, does NOT reset idle timer
|
||
if (url.pathname === '/activity/stream') {
|
||
// Auth: Bearer header OR view-only SSE session cookie (EventSource
|
||
// can't send Authorization headers, so the extension fetches a cookie
|
||
// via POST /sse-session first, then opens EventSource with
|
||
// withCredentials: true). The ?token= query param is NO LONGER
|
||
// accepted — URLs leak to logs/referer/history. See N1 in the
|
||
// v1.6.0.0 security wave plan.
|
||
const cookieToken = extractSseCookie(req);
|
||
if (!validateAuth(req) && !validateSseSessionToken(cookieToken)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const afterId = parseInt(url.searchParams.get('after') || '0', 10);
|
||
// Cleanup contract (abort + enqueue-fail + heartbeat-fail, all
|
||
// idempotent) lives in createSseEndpoint; sanitizeReplacer is
|
||
// applied to every JSON.stringify inside the helper, so
|
||
// page-content-derived fields (URLs, command args, errors)
|
||
// stay surrogate-safe per CLAUDE.md egress invariant.
|
||
return createSseEndpoint(req, {
|
||
initialReplay: (send) => {
|
||
const { entries, gap, gapFrom, availableFrom } = getActivityAfter(afterId);
|
||
if (gap) send('gap', { gapFrom, availableFrom });
|
||
for (const entry of entries) send('activity', entry);
|
||
},
|
||
subscribe,
|
||
liveEventName: 'activity',
|
||
});
|
||
}
|
||
|
||
// Activity history — REST, auth required, does NOT reset idle timer
|
||
if (url.pathname === '/activity/history') {
|
||
if (!validateAuth(req)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const limit = parseInt(url.searchParams.get('limit') || '50', 10);
|
||
const { entries, totalAdded } = getActivityHistory(limit);
|
||
return new Response(JSON.stringify({ entries, totalAdded, subscribers: getSubscriberCount() }), {
|
||
status: 200,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
|
||
// ─── Batch endpoint — N commands, 1 HTTP round-trip ─────────────
|
||
// Accepts both root AND scoped tokens (same as /command).
|
||
// Executes commands sequentially through the full security pipeline.
|
||
// Designed for remote agents where tunnel latency dominates.
|
||
if (url.pathname === '/batch' && req.method === 'POST') {
|
||
const tokenInfo = getTokenInfo(req);
|
||
if (!tokenInfo) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
resetIdleTimer();
|
||
const body = await req.json();
|
||
const { commands } = body;
|
||
|
||
if (!Array.isArray(commands) || commands.length === 0) {
|
||
return new Response(JSON.stringify({ error: '"commands" must be a non-empty array' }), {
|
||
status: 400,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
if (commands.length > 50) {
|
||
return new Response(JSON.stringify({ error: 'Max 50 commands per batch' }), {
|
||
status: 400,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
const startTime = Date.now();
|
||
emitActivity({
|
||
type: 'command_start',
|
||
command: 'batch',
|
||
args: [`${commands.length} commands`],
|
||
url: browserManager.getCurrentUrl(),
|
||
tabs: browserManager.getTabCount(),
|
||
mode: browserManager.getConnectionMode(),
|
||
clientId: tokenInfo?.clientId,
|
||
});
|
||
|
||
const results: Array<{ index: number; status: number; result: string; command: string; tabId?: number }> = [];
|
||
for (let i = 0; i < commands.length; i++) {
|
||
const cmd = commands[i];
|
||
if (!cmd || typeof cmd.command !== 'string') {
|
||
results.push({ index: i, status: 400, result: JSON.stringify({ error: 'Missing "command" field' }), command: '' });
|
||
continue;
|
||
}
|
||
// Reject nested batches
|
||
if (cmd.command === 'batch') {
|
||
results.push({ index: i, status: 400, result: JSON.stringify({ error: 'Nested batch commands are not allowed' }), command: 'batch' });
|
||
continue;
|
||
}
|
||
const cr = await handleCommandInternal(
|
||
{ command: cmd.command, args: cmd.args, tabId: cmd.tabId },
|
||
tokenInfo,
|
||
{ skipRateCheck: true, skipActivity: true },
|
||
);
|
||
// Sanitize lone surrogates per-result (#1440 — /batch bypasses the
|
||
// handleCommand chokepoint, so it needs its own sanitization).
|
||
const safeResult = typeof cr.result === 'string' ? sanitizeBody(cr.result, !!cr.json) : cr.result;
|
||
results.push({
|
||
index: i,
|
||
status: cr.status,
|
||
result: safeResult,
|
||
command: cmd.command,
|
||
tabId: cmd.tabId,
|
||
});
|
||
}
|
||
|
||
const duration = Date.now() - startTime;
|
||
emitActivity({
|
||
type: 'command_end',
|
||
command: 'batch',
|
||
args: [`${commands.length} commands`],
|
||
url: browserManager.getCurrentUrl(),
|
||
duration,
|
||
status: 'ok',
|
||
result: `${results.filter(r => r.status === 200).length}/${commands.length} succeeded`,
|
||
tabs: browserManager.getTabCount(),
|
||
mode: browserManager.getConnectionMode(),
|
||
clientId: tokenInfo?.clientId,
|
||
});
|
||
|
||
// Sanitize the JSON envelope a second time (defense in depth) — catches
|
||
// any \uXXXX escape sequences for lone surrogates that survived the
|
||
// per-result pass.
|
||
const batchBody = stripLoneSurrogateEscapes(JSON.stringify({
|
||
results,
|
||
duration,
|
||
total: commands.length,
|
||
succeeded: results.filter(r => r.status === 200).length,
|
||
failed: results.filter(r => r.status !== 200).length,
|
||
}));
|
||
return new Response(batchBody, {
|
||
status: 200,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// ─── File serving endpoint (for remote agents to retrieve downloaded files) ────
|
||
if (url.pathname === '/file' && req.method === 'GET') {
|
||
const tokenInfo = getTokenInfo(req);
|
||
if (!tokenInfo) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const filePath = url.searchParams.get('path');
|
||
if (!filePath) {
|
||
return new Response(JSON.stringify({ error: 'Missing "path" query parameter' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
try {
|
||
validateTempPath(filePath);
|
||
} catch (err: any) {
|
||
return new Response(JSON.stringify({ error: err.message }), {
|
||
status: 403, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
if (!fs.existsSync(filePath)) {
|
||
return new Response(JSON.stringify({ error: 'File not found' }), {
|
||
status: 404, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const stat = fs.statSync(filePath);
|
||
if (stat.size > 200 * 1024 * 1024) {
|
||
return new Response(JSON.stringify({ error: 'File too large (max 200MB)' }), {
|
||
status: 413, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const ext = path.extname(filePath).toLowerCase();
|
||
const MIME_MAP: Record<string, string> = {
|
||
'.png': 'image/png', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg',
|
||
'.gif': 'image/gif', '.webp': 'image/webp', '.svg': 'image/svg+xml',
|
||
'.avif': 'image/avif',
|
||
'.mp4': 'video/mp4', '.webm': 'video/webm', '.mov': 'video/quicktime',
|
||
'.mp3': 'audio/mpeg', '.wav': 'audio/wav', '.ogg': 'audio/ogg',
|
||
'.pdf': 'application/pdf', '.json': 'application/json',
|
||
'.html': 'text/html', '.txt': 'text/plain', '.mhtml': 'message/rfc822',
|
||
};
|
||
const contentType = MIME_MAP[ext] || 'application/octet-stream';
|
||
resetIdleTimer();
|
||
return new Response(Bun.file(filePath), {
|
||
headers: {
|
||
'Content-Type': contentType,
|
||
'Content-Length': String(stat.size),
|
||
'Content-Disposition': `inline; filename="${path.basename(filePath)}"`,
|
||
'Cache-Control': 'no-cache',
|
||
},
|
||
});
|
||
}
|
||
|
||
// ─── Command endpoint (accepts both root AND scoped tokens) ────
|
||
// Must be checked BEFORE the blanket root-only auth gate below,
|
||
// because scoped tokens from /connect are valid for /command.
|
||
if (url.pathname === '/command' && req.method === 'POST') {
|
||
const tokenInfo = getTokenInfo(req);
|
||
if (!tokenInfo) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
resetIdleTimer();
|
||
const body = await req.json() as any;
|
||
// Tunnel surface: only commands in TUNNEL_COMMANDS are allowed.
|
||
// Paired remote agents drive the browser but cannot configure the
|
||
// daemon, launch new browsers, import cookies, or rotate tokens.
|
||
if (surface === 'tunnel') {
|
||
if (!canDispatchOverTunnel(body?.command, body?.args)) {
|
||
logTunnelDenial(req, url, `disallowed_command:${body?.command}`);
|
||
return new Response(JSON.stringify({
|
||
error: `Command '${body?.command}' is not allowed over the tunnel surface`,
|
||
hint: `Tunnel commands: ${[...TUNNEL_COMMANDS].sort().join(', ')}. Note: --out (disk write) is never allowed over the tunnel.`,
|
||
}), { status: 403, headers: { 'Content-Type': 'application/json' } });
|
||
}
|
||
}
|
||
return handleCommand(body, tokenInfo);
|
||
}
|
||
|
||
// ─── Auth-required endpoints (root token only) ─────────────────
|
||
|
||
if (!validateAuth(req)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// ─── Inspector endpoints ──────────────────────────────────────
|
||
|
||
// POST /inspector/pick — receive element pick from extension, run CDP inspection
|
||
if (url.pathname === '/inspector/pick' && req.method === 'POST') {
|
||
const body = await req.json();
|
||
const { selector, activeTabUrl } = body;
|
||
if (!selector) {
|
||
return new Response(JSON.stringify({ error: 'Missing selector' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
try {
|
||
const page = browserManager.getPage();
|
||
const result = await inspectElement(page, selector);
|
||
inspectorData = result;
|
||
inspectorTimestamp = Date.now();
|
||
// Also store on browserManager for CLI access
|
||
(browserManager as any)._inspectorData = result;
|
||
(browserManager as any)._inspectorTimestamp = inspectorTimestamp;
|
||
emitInspectorEvent({ type: 'pick', selector, timestamp: inspectorTimestamp });
|
||
return new Response(JSON.stringify(result), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
} catch (err: any) {
|
||
return new Response(JSON.stringify({ error: err.message }), {
|
||
status: 500, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
}
|
||
|
||
// GET /inspector — return latest inspector data
|
||
if (url.pathname === '/inspector' && req.method === 'GET') {
|
||
if (!inspectorData) {
|
||
return new Response(JSON.stringify({ data: null }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const stale = inspectorTimestamp > 0 && (Date.now() - inspectorTimestamp > 60000);
|
||
return new Response(JSON.stringify({ data: inspectorData, timestamp: inspectorTimestamp, stale }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// POST /inspector/apply — apply a CSS modification
|
||
if (url.pathname === '/inspector/apply' && req.method === 'POST') {
|
||
const body = await req.json();
|
||
const { selector, property, value } = body;
|
||
if (!selector || !property || value === undefined) {
|
||
return new Response(JSON.stringify({ error: 'Missing selector, property, or value' }), {
|
||
status: 400, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
try {
|
||
const page = browserManager.getPage();
|
||
const mod = await modifyStyle(page, selector, property, value);
|
||
emitInspectorEvent({ type: 'apply', modification: mod, timestamp: Date.now() });
|
||
return new Response(JSON.stringify(mod), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
} catch (err: any) {
|
||
return new Response(JSON.stringify({ error: err.message }), {
|
||
status: 500, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
}
|
||
|
||
// POST /inspector/reset — clear all modifications
|
||
if (url.pathname === '/inspector/reset' && req.method === 'POST') {
|
||
try {
|
||
const page = browserManager.getPage();
|
||
await resetModifications(page);
|
||
emitInspectorEvent({ type: 'reset', timestamp: Date.now() });
|
||
return new Response(JSON.stringify({ ok: true }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
} catch (err: any) {
|
||
return new Response(JSON.stringify({ error: err.message }), {
|
||
status: 500, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
}
|
||
|
||
// GET /inspector/history — return modification list
|
||
if (url.pathname === '/inspector/history' && req.method === 'GET') {
|
||
return new Response(JSON.stringify({ history: getModificationHistory() }), {
|
||
status: 200, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// GET /memory — diagnostic snapshot (auth required, does NOT reset idle).
|
||
// Same auth model as /activity/stream and /inspector/events: Bearer header
|
||
// OR view-only SSE-session cookie. Does NOT extend /health (which is
|
||
// unauthenticated liveness-only — token bootstrap moved to the pinned
|
||
// POST /extension-token); a separate endpoint with the standard SSE auth
|
||
// keeps /health free of anything worth stealing.
|
||
if (url.pathname === '/memory' && req.method === 'GET') {
|
||
const cookieToken = extractSseCookie(req);
|
||
if (!validateAuth(req) && !validateSseSessionToken(cookieToken)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
const { buildMemorySnapshotJson } = await import('./memory-command');
|
||
const snapshot = await buildMemorySnapshotJson(cfgBrowserManager);
|
||
// sanitizeReplacer is required at every SSE/JSON egress that ships
|
||
// page-content-derived strings — tab.url and tab.title come from
|
||
// page content, so lone-surrogate bytes from broken emoji or
|
||
// mid-emoji splits could otherwise reach the sidebar / Claude API.
|
||
return new Response(JSON.stringify(snapshot, sanitizeReplacer), {
|
||
status: 200,
|
||
headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
|
||
// GET /inspector/events — SSE for inspector state changes (auth required)
|
||
if (url.pathname === '/inspector/events' && req.method === 'GET') {
|
||
// Same auth model as /activity/stream: Bearer OR view-only cookie.
|
||
// ?token= query param dropped (see N1 in the v1.6.0.0 security plan).
|
||
const cookieToken = extractSseCookie(req);
|
||
if (!validateAuth(req) && !validateSseSessionToken(cookieToken)) {
|
||
return new Response(JSON.stringify({ error: 'Unauthorized' }), {
|
||
status: 401, headers: { 'Content-Type': 'application/json' },
|
||
});
|
||
}
|
||
// Cleanup contract (abort + enqueue-fail + heartbeat-fail,
|
||
// idempotent) lives in createSseEndpoint; sanitizeReplacer is
|
||
// applied to every JSON.stringify inside the helper. The
|
||
// inspector subscriber set stays here because it's also written
|
||
// to by emitInspectorEvent above.
|
||
return createSseEndpoint(req, {
|
||
initialReplay: inspectorData
|
||
? (send) => send('state', { data: inspectorData, timestamp: inspectorTimestamp })
|
||
: undefined,
|
||
subscribe: (notify) => {
|
||
inspectorSubscribers.add(notify);
|
||
return () => inspectorSubscribers.delete(notify);
|
||
},
|
||
liveEventName: 'inspector',
|
||
});
|
||
}
|
||
|
||
return new Response('Not found', { status: 404 });
|
||
};
|
||
|
||
return {
|
||
fetchLocal: makeFetchHandler('local'),
|
||
fetchTunnel: makeFetchHandler('tunnel'),
|
||
shutdown,
|
||
stopListeners,
|
||
};
|
||
}
|
||
|
||
export async function start() {
|
||
// Clear old log files
|
||
safeUnlink(CONSOLE_LOG_PATH);
|
||
safeUnlink(NETWORK_LOG_PATH);
|
||
safeUnlink(DIALOG_LOG_PATH);
|
||
|
||
const port = await findPort();
|
||
LOCAL_LISTEN_PORT = port;
|
||
|
||
// ─── Proxy config (D8 + codex F5) ──────────────────────────────
|
||
// BROWSE_PROXY_URL is set by the CLI when --proxy was passed. For SOCKS5
|
||
// with auth, we run a local 127.0.0.1 bridge that relays to the
|
||
// authenticated upstream (Chromium can't do SOCKS5 auth itself). For
|
||
// HTTP/HTTPS or unauthenticated SOCKS5, we pass the URL directly to
|
||
// Chromium's proxy.server option.
|
||
let proxyBridge: BridgeHandle | null = null;
|
||
const proxyUrl = process.env.BROWSE_PROXY_URL;
|
||
if (proxyUrl) {
|
||
let parsed;
|
||
try {
|
||
parsed = parseProxyConfig({
|
||
proxyUrl,
|
||
envUser: process.env.BROWSE_PROXY_USER,
|
||
envPass: process.env.BROWSE_PROXY_PASS,
|
||
});
|
||
} catch (err) {
|
||
if (err instanceof ProxyConfigError) {
|
||
console.error(`[browse] error: ${err.message} (${err.hint})`);
|
||
process.exit(1);
|
||
}
|
||
throw err;
|
||
}
|
||
|
||
if (parsed.scheme === 'socks5' && parsed.hasAuth) {
|
||
// Pre-flight: verify upstream accepts our creds before launching
|
||
// Chromium. 5s budget, 3 retries with 500ms backoff (D4: handles VPN
|
||
// warm-up race). On failure, exit with redacted error.
|
||
console.log(`[browse] Testing SOCKS5 upstream ${redactProxyUrl(proxyUrl)}...`);
|
||
try {
|
||
const test = await testUpstream({
|
||
upstream: toUpstreamConfig(parsed),
|
||
budgetMs: 5000,
|
||
retries: 3,
|
||
backoffMs: 500,
|
||
});
|
||
console.log(`[browse] [proxy] upstream test ok in ${test.ms}ms (${test.attempts} attempt${test.attempts === 1 ? '' : 's'})`);
|
||
} catch (err) {
|
||
const msg = err instanceof Error ? err.message : String(err);
|
||
console.error(`[browse] [proxy] FAIL upstream ${redactProxyUrl(proxyUrl)}: ${msg}`);
|
||
process.exit(1);
|
||
}
|
||
|
||
proxyBridge = await startSocksBridge({ upstream: toUpstreamConfig(parsed) });
|
||
console.log(`[browse] [proxy] bridge listening on 127.0.0.1:${proxyBridge.port}`);
|
||
browserManager.setProxyConfig({ server: `socks5://127.0.0.1:${proxyBridge.port}` });
|
||
} else {
|
||
// HTTP/HTTPS or unauth SOCKS5 — pass through to Chromium directly.
|
||
browserManager.setProxyConfig({
|
||
server: `${parsed.scheme}://${parsed.host}:${parsed.port}`,
|
||
...(parsed.userId ? { username: parsed.userId } : {}),
|
||
...(parsed.password ? { password: parsed.password } : {}),
|
||
});
|
||
console.log(`[browse] [proxy] using ${redactProxyUrl(proxyUrl)} (pass-through to Chromium)`);
|
||
}
|
||
|
||
// Tear down bridge on shutdown.
|
||
process.on('exit', () => {
|
||
if (proxyBridge) {
|
||
proxyBridge.close().catch(() => { /* shutting down anyway */ });
|
||
}
|
||
});
|
||
}
|
||
|
||
// ─── Xvfb auto-spawn (Linux + headed + no DISPLAY) ─────────────
|
||
// codex F2: walk display range to pick a free one (never hardcode :99);
|
||
// record start-time alongside PID so cleanup can validate ownership and
|
||
// not kill a recycled PID.
|
||
let xvfb: XvfbHandle | null = null;
|
||
const xvfbDecision = shouldSpawnXvfb(process.env, process.platform);
|
||
if (xvfbDecision.spawn) {
|
||
const displayNum = pickFreeDisplay();
|
||
if (displayNum == null) {
|
||
console.error('[browse] no free X display in range :99-:120 — refusing to clobber existing X servers');
|
||
process.exit(1);
|
||
}
|
||
try {
|
||
xvfb = await spawnXvfb(displayNum);
|
||
process.env.DISPLAY = xvfb.display;
|
||
console.log(`[browse] [xvfb] spawned on ${xvfb.display} (pid ${xvfb.pid})`);
|
||
} catch (err) {
|
||
const msg = err instanceof Error ? err.message : String(err);
|
||
console.error(`[browse] [xvfb] FAILED: ${msg}`);
|
||
console.error(`[browse] [xvfb] hint: ${xvfbInstallHint()}`);
|
||
process.exit(1);
|
||
}
|
||
process.on('exit', () => { try { xvfb?.close(); } catch { /* shutting down */ } });
|
||
} else if (process.env.BROWSE_HEADED === '1') {
|
||
console.log(`[browse] [xvfb] skipped: ${xvfbDecision.reason}`);
|
||
}
|
||
|
||
// Read env once — single source of truth for authToken (and other env).
|
||
// Threaded through launchHeaded, buildFetchHandler, and the state file
|
||
// write so all consumers see the same value. v1.34.x's module-level
|
||
// AUTH_TOKEN const was deleted in v1.35.0.0.
|
||
const envCfg = resolveConfigFromEnv();
|
||
|
||
// Launch browser (headless or headed with extension)
|
||
// BROWSE_HEADLESS_SKIP=1 skips browser launch entirely (for HTTP-only testing)
|
||
const skipBrowser = process.env.BROWSE_HEADLESS_SKIP === '1';
|
||
if (!skipBrowser) {
|
||
const headed = process.env.BROWSE_HEADED === '1';
|
||
if (headed) {
|
||
await browserManager.launchHeaded(envCfg.authToken);
|
||
console.log(`[browse] Launched headed Chromium with extension`);
|
||
} else {
|
||
await browserManager.launch();
|
||
}
|
||
}
|
||
|
||
const startTime = Date.now();
|
||
|
||
// ─── Build the request handlers via buildFetchHandler factory ───
|
||
// CLI path passes env-derived values; no beforeRoute hook. Phoenix uses
|
||
// the same factory with its own cfg + overlay hook.
|
||
const handle = buildFetchHandler({
|
||
...envCfg,
|
||
browsePort: port, // actual bound port (resolveConfigFromEnv default is 0)
|
||
browserManager, // module-level instance, same as today
|
||
xvfb,
|
||
proxyBridge,
|
||
startTime,
|
||
ownsTerminalAgent: true, // CLI spawns terminal-agent.ts itself (see cli.ts:1037-1063)
|
||
});
|
||
|
||
const server = Bun.serve({
|
||
port,
|
||
hostname: '127.0.0.1',
|
||
fetch: handle.fetchLocal,
|
||
});
|
||
|
||
// Write state file (atomic: write .tmp then rename)
|
||
const state: Record<string, unknown> = {
|
||
pid: process.pid,
|
||
port,
|
||
token: envCfg.authToken,
|
||
startedAt: new Date().toISOString(),
|
||
serverPath: path.resolve(import.meta.dir, 'server.ts'),
|
||
binaryVersion: readVersionHash() || undefined,
|
||
mode: browserManager.getConnectionMode(),
|
||
// D2 daemon-mismatch detection: CLI computes the same hash from its
|
||
// resolved flags and refuses if it differs from this stored value.
|
||
...(process.env.BROWSE_CONFIG_HASH ? { configHash: process.env.BROWSE_CONFIG_HASH } : {}),
|
||
// Xvfb child PID + start-time + display so disconnect (or a future
|
||
// daemon launch on this state file) can validate-then-cleanup orphans
|
||
// without clobbering a recycled PID.
|
||
...(xvfb ? { xvfbPid: xvfb.pid, xvfbStartTime: xvfb.startTime, xvfbDisplay: xvfb.display } : {}),
|
||
};
|
||
const tmpFile = tmpStatePath();
|
||
fs.writeFileSync(tmpFile, JSON.stringify(state, null, 2), { mode: 0o600 });
|
||
fs.renameSync(tmpFile, config.stateFile);
|
||
|
||
browserManager.serverPort = port;
|
||
|
||
// ─── Opt-in session persistence (#778 class) ─────────────────
|
||
// BROWSE_PERSIST_STATE=1: restore cookies/storage/tabs from the last
|
||
// snapshot, then keep snapshotting on an interval. Launched mode only —
|
||
// the headed persistent profile owns its own state. The final snapshot at
|
||
// clean shutdown lives in buildFetchHandler's shutdown().
|
||
//
|
||
// Runs AFTER Bun.serve() + the state-file write, in the BACKGROUND:
|
||
// restore re-creates tabs sequentially with up-to-15s goto timeouts while
|
||
// the CLI's readiness probe gives up at 8s — one slow/unreachable saved
|
||
// URL must never make every `$B` command report "Server failed to start".
|
||
// Fire-and-forget: a restore failure is logged and never affects the
|
||
// daemon.
|
||
if (!skipBrowser && isSessionPersistEnabled() && browserManager.getConnectionMode() === 'launched') {
|
||
const sessionStatePath = path.join(config.stateDir, SESSION_STATE_FILE);
|
||
restoreSessionState(browserManager, sessionStatePath)
|
||
.then((restored) => {
|
||
if (restored) {
|
||
// Counts come from the deserialized snapshot itself — no extra
|
||
// saveState() round-trip against pages that may still be loading.
|
||
console.log(`[browse] Session state restored: ${restored.cookies.length} cookies / ${restored.pages.length} tabs (BROWSE_PERSIST_STATE=1)`);
|
||
} else {
|
||
console.log('[browse] Session persistence on; no prior state — fresh session (BROWSE_PERSIST_STATE=1)');
|
||
}
|
||
})
|
||
.catch((err: any) => {
|
||
console.warn(`[browse] SESSION_RESTORE_FAILED: ${err?.message ?? err}`);
|
||
});
|
||
let persistWarned = false;
|
||
// In-flight guard: never start a new snapshot while the previous one is
|
||
// still pending (a slow page.evaluate would otherwise pile up ticks).
|
||
let persistInFlight = false;
|
||
sessionPersistInterval = setInterval(() => {
|
||
// Shutdown gate (belt; shutdown()'s clearInterval is the suspenders):
|
||
// a tick that fires during browser teardown snapshots a degraded state
|
||
// (zero tabs) over the good final snapshot.
|
||
if (isShuttingDown) return;
|
||
if (persistInFlight) return; // skip the tick
|
||
persistInFlight = true;
|
||
persistSessionState(browserManager, sessionStatePath)
|
||
.catch((err: any) => {
|
||
// Warn once — a full disk must not spam the log every 30s, and a
|
||
// snapshot failure must never kill the daemon (R3).
|
||
if (!persistWarned) {
|
||
persistWarned = true;
|
||
console.warn(`[browse] SESSION_PERSIST_FAILED: ${err?.message ?? err} (further failures suppressed)`);
|
||
}
|
||
})
|
||
.finally(() => { persistInFlight = false; });
|
||
}, sessionPersistIntervalMs());
|
||
(sessionPersistInterval as any)?.unref?.();
|
||
}
|
||
|
||
// Navigate to welcome page if in headed mode and still on about:blank
|
||
if (browserManager.getConnectionMode() === 'headed') {
|
||
try {
|
||
const currentUrl = browserManager.getCurrentUrl();
|
||
if (currentUrl === 'about:blank' || currentUrl === '') {
|
||
const page = browserManager.getPage();
|
||
page.goto(`http://127.0.0.1:${port}/welcome`, { timeout: 3000 }).catch((err: any) => {
|
||
console.warn('[browse] Failed to navigate to welcome page:', err.message);
|
||
});
|
||
}
|
||
} catch (err: any) {
|
||
console.warn('[browse] Welcome page navigation setup failed:', err.message);
|
||
}
|
||
}
|
||
|
||
// Clean up stale state files (older than 7 days)
|
||
try {
|
||
const stateDir = path.join(config.stateDir, 'browse-states');
|
||
if (fs.existsSync(stateDir)) {
|
||
const SEVEN_DAYS = 7 * 24 * 60 * 60 * 1000;
|
||
for (const file of fs.readdirSync(stateDir)) {
|
||
const filePath = path.join(stateDir, file);
|
||
const stat = fs.statSync(filePath);
|
||
if (Date.now() - stat.mtimeMs > SEVEN_DAYS) {
|
||
fs.unlinkSync(filePath);
|
||
console.log(`[browse] Deleted stale state file: ${file}`);
|
||
}
|
||
}
|
||
}
|
||
} catch (err: any) {
|
||
console.warn('[browse] Failed to clean stale state files:', err.message);
|
||
}
|
||
|
||
console.log(`[browse] Server running on http://127.0.0.1:${port} (PID: ${process.pid})`);
|
||
console.log(`[browse] State file: ${config.stateFile}`);
|
||
console.log(`[browse] Idle timeout: ${IDLE_TIMEOUT_MS / 1000}s`);
|
||
|
||
// ─── Tunnel startup (optional) ────────────────────────────────
|
||
// Start ngrok tunnel if BROWSE_TUNNEL=1 is set. Uses the dual-listener
|
||
// pattern: bind a dedicated tunnel listener on an ephemeral port and
|
||
// point ngrok.forward() at IT, not the local daemon port.
|
||
if (process.env.BROWSE_TUNNEL === '1' && !isPairAgentEnabled()) {
|
||
console.error('[browse] BROWSE_TUNNEL=1 ignored: pair-agent is off. Enable once with: gstack-config set pair_agent on');
|
||
} else if (process.env.BROWSE_TUNNEL === '1') {
|
||
const authtoken = resolveNgrokAuthtoken();
|
||
if (!authtoken) {
|
||
console.error('[browse] BROWSE_TUNNEL=1 but no NGROK_AUTHTOKEN found. Set it via env var or ~/.gstack/ngrok.env');
|
||
} else {
|
||
// Shared startTunnel helper: binds the tunnel listener, opens ngrok,
|
||
// and on any failure tears down BOTH ngrok and the Bun listener so we
|
||
// don't leak an ngrok session if the error happened after
|
||
// ngrok.forward() resolved.
|
||
const started = await startTunnel({
|
||
fetchHandler: handle.fetchTunnel,
|
||
authtoken,
|
||
consent: 'pair_agent=on (isPairAgentEnabled gate, BROWSE_TUNNEL=1)',
|
||
});
|
||
if (!started.ok) {
|
||
console.error(`[browse] Failed to start tunnel: ${started.error.message}`);
|
||
}
|
||
}
|
||
} else if (process.env.BROWSE_TUNNEL_LOCAL_ONLY === '1') {
|
||
// Test-only: bind the dual-listener tunnel surface on 127.0.0.1 with NO
|
||
// ngrok forwarding. Lets paid evals exercise the surface==='tunnel' gate
|
||
// without an ngrok authtoken or live network. Production tunneling still
|
||
// requires BROWSE_TUNNEL=1 + a valid authtoken above.
|
||
try {
|
||
const boundTunnel = Bun.serve({
|
||
port: 0,
|
||
hostname: '127.0.0.1',
|
||
fetch: handle.fetchTunnel,
|
||
});
|
||
tunnelServer = boundTunnel;
|
||
tunnelActive = true;
|
||
const tunnelPort = boundTunnel.port;
|
||
console.log(`[browse] Tunnel listener bound (local-only test mode) on 127.0.0.1:${tunnelPort}`);
|
||
const stateContent = JSON.parse(fs.readFileSync(config.stateFile, 'utf-8'));
|
||
stateContent.tunnelLocalPort = tunnelPort;
|
||
const tmpState = tmpStatePath();
|
||
fs.writeFileSync(tmpState, JSON.stringify(stateContent, null, 2), { mode: 0o600 });
|
||
fs.renameSync(tmpState, config.stateFile);
|
||
} catch (err: any) {
|
||
console.error(`[browse] BROWSE_TUNNEL_LOCAL_ONLY=1 listener bind failed: ${err.message}`);
|
||
}
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Test-only. Resets the module-level shutdown latch so a second test case
|
||
* can exercise shutdown() in the same process. Mirrors __resetRegistry in
|
||
* token-registry.ts. shutdown() short-circuits when isShuttingDown is true
|
||
* (see line near the start of shutdown), so without this, tests that call
|
||
* shutdown() more than once silently no-op after the first call.
|
||
*
|
||
* DO NOT call from production code. Defeats the shutdown re-entry guard,
|
||
* which can race process.exit with cfgBrowserManager.close() and the pkill /
|
||
* safeUnlinkQuiet side effects. The `__` prefix is the convention; nothing
|
||
* enforces it. If you find yourself reaching for this outside a test file,
|
||
* the right fix is to make isShuttingDown factory-scoped instead.
|
||
*/
|
||
export function __resetShuttingDown(): void {
|
||
isShuttingDown = false;
|
||
}
|
||
|
||
// Auto-kickoff only when this module is the entry point. Embedders
|
||
// (gbrowser phoenix overlay) import { start, buildFetchHandler, ... }
|
||
// without triggering the listener-binding side effects.
|
||
if (import.meta.main) {
|
||
start().catch((err) => {
|
||
console.error(`[browse] Failed to start: ${err.message}`);
|
||
// Write error to disk for the CLI to read — on Windows, the CLI can't capture
|
||
// stderr because the server is launched with detached: true, stdio: 'ignore'.
|
||
try {
|
||
const errorLogPath = path.join(config.stateDir, 'browse-startup-error.log');
|
||
mkdirSecure(config.stateDir);
|
||
writeSecureFile(errorLogPath, `${new Date().toISOString()} ${err.message}\n${err.stack || ''}\n`);
|
||
} catch {
|
||
// stateDir may not exist — nothing more we can do
|
||
}
|
||
process.exit(1);
|
||
});
|
||
}
|