Files
gstack/ARCHITECTURE.md
T
Garry Tan 54d4cde773 security: tunnel dual-listener + SSRF + envelope + path wave (v1.6.0.0) (#1137)
* refactor(security): loosen /connect rate limit from 3/min to 300/min

Setup keys are 24 random bytes (unbruteforceable), so a tight rate limit
does not meaningfully prevent key guessing. It exists only to cap
bandwidth, CPU, and log-flood damage from someone who discovered the
ngrok URL. A legitimate pair-agent session hits /connect once; 300/min
is 60x that pattern and never hit accidentally.

3/min caused pairing to fail on any retry flow (network blip, second
paired client) with no upside. Per-IP tracking was considered and
rejected — adds a bounded Map + LRU for defense already adequate at the
global layer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(security): add tunnel-denial-log module for attack visibility

Append-only log of tunnel-surface auth denials to
~/.gstack/security/attempts.jsonl. Gives operators visibility into who
is probing tunneled daemons so the next security wave can be driven by
real attack data instead of speculation.

Design notes:
- Async via fs.promises.appendFile. Never appendFileSync — blocking the
  event loop on every denial during a flood is what an attacker wants
  (prior learning: sync-audit-log-io, 10/10 confidence).
- In-process rate cap at 60 writes/minute globally. Excess denials are
  counted in memory but not written to disk — prevents disk DoS.
- Writes to the same ~/.gstack/security/attempts.jsonl used by the
  prompt-injection attempt log. File rotation is handled by the existing
  security pipeline (10MB, 5 generations).

No consumers in this commit; wired up in the dual-listener refactor that
follows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(security): dual-listener tunnel architecture

The /health endpoint leaked AUTH_TOKEN to any caller that hit the ngrok
URL (spoofing chrome-extension:// origin, or catching headed mode).
Surfaced by @garagon in PR #1026; the original fix was header-inference
on the single port. Codex's outside-voice review during /plan-ceo-review
called that approach brittle (ngrok header behavior could change, local
proxies would false-positive), and pushed for the structural fix.

This is that fix. Stop making /health a root-token bootstrap endpoint on
any surface the tunnel can reach. The server now binds two HTTP
listeners when a tunnel is active. The local listener (extension, CLI,
sidebar) stays on 127.0.0.1 and is never exposed to ngrok. ngrok
forwards only to the tunnel listener, which serves only /connect
(unauth, rate-limited) and /command with a locked allowlist of
browser-driving commands. Security property comes from physical port
separation, not from header inference — a tunnel caller cannot reach
/health or /cookie-picker or /inspector because they live on a
different TCP socket.

What this commit adds to browse/src/server.ts:
  * Surface type ('local' | 'tunnel') and TUNNEL_PATHS +
    TUNNEL_COMMANDS allowlists near the top of the file.
  * makeFetchHandler(surface) factory replacing the single fetch arrow;
    closure-captures the surface so the filter that runs before route
    dispatch knows which socket accepted the request.
  * Tunnel filter at dispatch entry: 404s anything not on TUNNEL_PATHS,
    403s root-token bearers with a clear pairing hint, 401s non-/connect
    requests that lack a scoped token. Every denial is logged via
    logTunnelDenial (from tunnel-denial-log).
  * GET /connect alive probe (unauth on both surfaces) so /pair and
    /tunnel/start can detect dead ngrok tunnels without reaching
    /health — /health is no longer tunnel-reachable.
  * Lazy tunnel listener lifecycle. /tunnel/start binds a dedicated
    Bun.serve on an ephemeral port, points ngrok.forward at THAT port
    (not the local port), hard-fails on bind error (no local fallback),
    tears down cleanly on ngrok failure. BROWSE_TUNNEL=1 startup uses
    the same pattern.
  * closeTunnel() helper — single teardown path for both the ngrok
    listener and the tunnel Bun.serve listener.
  * resolveNgrokAuthtoken() helper — shared authtoken lookup across
    /tunnel/start and BROWSE_TUNNEL=1 startup (was duplicated).
  * TUNNEL_COMMANDS check in /command dispatch: on the tunnel surface,
    commands outside the allowlist return 403 with a list of allowed
    commands as a hint.
  * Probe paths in /pair and /tunnel/start migrated from /health to
    GET /connect — the only unauth path reachable on the tunnel surface
    under the new architecture.

Test updates in browse/test/server-auth.test.ts:
  * /pair liveness-verify test: assert via closeTunnel() helper instead
    of the inline `tunnelActive = false; tunnelUrl = null` lines that
    the helper subsumes.
  * /tunnel/start cached-tunnel test: same closeTunnel() adaptation.

Credit
  Derived from PR #1026 by @garagon — thanks for flagging the critical
  bug that drove the architectural rewrite. The per-request
  isTunneledRequest approach from #1026 is superseded by physical port
  separation here; the underlying report remains the root cause for the
  entire v1.6.0.0 wave.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(security): add source-level guards for dual-listener architecture

23 source-level assertions that keep future contributors from silently
widening the tunnel surface during a routine refactor. Covers:

  * Surface type + tunnelServer state variable shape
  * TUNNEL_PATHS is a closed set of /connect, /command, /sidebar-chat
    (and NOT /health, /welcome, /cookie-picker, /inspector/*, /pair,
    /token, /refs, /activity/stream, /tunnel/{start,stop})
  * TUNNEL_COMMANDS includes browser-driving ops only (and NOT
    launch-browser, tunnel-start, token-mint, cookie-import, etc.)
  * makeFetchHandler(surface) factory exists and is wired to both
    listeners with the correct surface parameter
  * Tunnel filter runs BEFORE any route dispatch, with 404/403/401
    responses and logged denials for each reason
  * GET /connect returns {alive: true} unauth
  * /command dispatch enforces TUNNEL_COMMANDS on tunnel surface
  * closeTunnel() helper tears down ngrok + Bun.serve listener
  * /tunnel/start binds on ephemeral port, points ngrok at TUNNEL_PORT
    (not local port), hard-fails on bind error (no fallback), probes
    cached tunnel via GET /connect (not /health), tears down on
    ngrok.forward failure
  * BROWSE_TUNNEL=1 startup uses the dual-listener pattern
  * logTunnelDenial wired for all three denial reasons
  * /connect rate limit is 300/min, not 3/min

All 23 tests pass. Behavioral integration tests (spawn subprocess, real
network) live in the E2E suite that lands later in this wave.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* security: gate download + scrape through validateNavigationUrl (SSRF)

The `goto` command was correctly wired through validateNavigationUrl,
but `download` and `scrape` called page.request.fetch(url, ...) directly.
A caller with the default write scope could hit the /command endpoint
and ask the daemon to fetch http://169.254.169.254/latest/meta-data/
(AWS IMDSv1) or the GCP/Azure/internal equivalents. The response body
comes back as base64 or lands on disk where GET /file serves it.

Fix: call validateNavigationUrl(url) immediately before each
page.request.fetch() call site in download and in the scrape loop.
Same blocklist that already protects `goto`: file://, javascript:,
data:, chrome://, cloud metadata (IPv4 all encodings, IPv6 ULA,
metadata.*.internal).

Tests: extend browse/test/url-validation.test.ts with a source-level
guard that walks every `await page.request.fetch(` call site and
asserts a validateNavigationUrl call precedes it within the same
branch. Regression trips before code review if a future refactor
drops the gate.

* security: route splitForScoped through envelope sentinel escape

The scoped-token snapshot path in snapshot.ts built its untrusted
block by pushing the raw accessibility-tree lines between the literal
`═══ BEGIN UNTRUSTED WEB CONTENT ═══` / `═══ END UNTRUSTED WEB CONTENT ═══`
sentinels. The full-page wrap path in content-security.ts already
applied a zero-width-space escape on those exact strings to prevent
sentinel injection, but the scoped path skipped it.

Net effect: a page whose rendered text contains the literal sentinel
can close the envelope early from inside untrusted content and forge
a fake "trusted" block for the LLM. That includes fabricating
interactive `@eN` references the agent will act on.

Fix:
  * Extract the zero-width-space escape into a named, exported helper
    `escapeEnvelopeSentinels(content)` in content-security.ts.
  * Have `wrapUntrustedPageContent` call it (behavior unchanged on
    that path — same bytes out).
  * Import the helper in snapshot.ts and map it over `untrustedLines`
    in the `splitForScoped` branch before pushing the BEGIN sentinel.

Tests: add a describe block in content-security.test.ts that covers
  * `escapeEnvelopeSentinels` defuses BEGIN and END markers;
  * `escapeEnvelopeSentinels` leaves normal text untouched;
  * `wrapUntrustedPageContent` still emits exactly one real envelope
    pair when hostile content contains forged sentinels;
  * snapshot.ts imports the helper;
  * the scoped-snapshot branch calls `escapeEnvelopeSentinels` before
    pushing the BEGIN sentinel (source-level regression — if a future
    refactor reorders this, the test trips).

* security: extend hidden-element detection to all DOM-reading channels

The Confusion Protocol envelope wrap (`wrapUntrustedPageContent`)
covers every scoped PAGE_CONTENT_COMMAND, but the hidden-element
ARIA-injection detection layer only ran for `text`. Other DOM-reading
channels (html, links, forms, accessibility, attrs, data, media,
ux-audit) returned their output through the envelope with no hidden-
content filter, so a page serving a display:none div that instructs
the agent to disregard prior system messages, or an aria-label that
claims to put the LLM in admin mode, leaked the injection payload on
any non-text channel. The envelope alone does not mitigate this, and
the page itself never rendered the hostile content to the human
operator.

Fix:
  * New export `DOM_CONTENT_COMMANDS` in commands.ts — the subset of
    PAGE_CONTENT_COMMANDS that derives its output from the live DOM.
    Console and dialog stay out; they read separate runtime state.
  * server.ts runs `markHiddenElements` + `cleanupHiddenMarkers` for
    every scoped command in this set. `text` keeps its existing
    `getCleanTextWithStripping` path (hidden elements physically
    stripped before the read). All other channels keep their output
    format but emit flagged elements as CONTENT WARNINGS on the
    envelope, so the LLM sees what it would otherwise have consumed
    silently.
  * Hidden-element descriptions merge into `combinedWarnings`
    alongside content-filter warnings before the wrap call.

Tests: new describe block in content-security.test.ts covering
  * `DOM_CONTENT_COMMANDS` export shape and channel membership;
  * dispatch gates on `DOM_CONTENT_COMMANDS.has(command)`, not the
    literal `text` string;
  * hiddenContentWarnings plumbs into `combinedWarnings` and reaches
    wrapUntrustedPageContent;
  * DOM_CONTENT_COMMANDS is a strict subset of PAGE_CONTENT_COMMANDS.

Existing datamarking, envelope wrap, centralized-wrapping, and chain
security suites stay green (52 pass, 0 fail).

* security: validate --from-file payload paths for parity with direct paths

The direct `load-html <file>` path runs every caller-supplied file path
through validateReadPath() so reads stay confined to SAFE_DIRECTORIES
(cwd, TEMP_DIR). The `load-html --from-file <payload.json>` shortcut
and its sibling `pdf --from-file <payload.json>` skipped that check and
went straight to fs.readFileSync(). An MCP caller that picks the
payload path (or any caller whose payload argument is reachable from
attacker-influenced text) could use --from-file as a read-anywhere
escape hatch for the safe-dirs policy.

Fix: call validateReadPath(path.resolve(payloadPath)) before readFileSync
at both sites. Error surface mirrors the direct-path branch so ops and
agent errors stay consistent.

Test coverage in browse/test/from-file-path-validation.test.ts:
  - source-level: validateReadPath precedes readFileSync in the load-html
    --from-file branch (write-commands.ts) and the pdf --from-file parser
    (meta-commands.ts)
  - error-message parity: both sites reference SAFE_DIRECTORIES

Related security audit pattern: R3 F002 (validateNavigationUrl gap on
download/scrape) and R3 F008 (markHiddenElements gap on 10 DOM commands)
were the same shape — a defense that existed on the primary code path
but not its shortcut sibling. This PR closes the same class of gap on
the --from-file shortcuts.

* fix(design): escape url.origin when injecting into served HTML

serve.ts injected url.origin into a single-quoted JS string in
the response body. A local request with a crafted Host header
(e.g. Host: "evil'-alert(1)-'x") would break out of the string
and execute JS in the 127.0.0.1:<port> origin opened by the
design board. Low severity — bound to localhost, requires a
local attacker — but no reason not to escape.

Fix: JSON.stringify(url.origin) produces a properly quoted,
escaped JS string literal in one call.

Also includes Prettier reformatting (single→double quotes,
trailing commas, line wrapping) applied by the repo's
PostToolUse formatter hook. Security change is the one line
in the HTML injection; everything else is whitespace/style.

* fix(scripts): drop shell:true from slop-diff npx invocations

spawnSync('npx', [...], { shell: true }) invokes /bin/sh -c
with the args concatenated, subjecting them to shell parsing
(word splitting, glob expansion, metacharacter interpretation).
No user input reaches these calls today, so not exploitable —
but the posture is wrong: npx + shell args should be direct.

Fix: scope shell:true to process.platform === 'win32' where
npx is actually a .cmd requiring the shell. POSIX runs the
npx binary directly with array-form args.

Also includes Prettier reformatting (single→double quotes,
trailing commas, line wrapping) applied by the repo's
PostToolUse formatter hook. Security-relevant change is just
the two shell:true -> shell: process.platform === 'win32'
lines; everything else is whitespace/style.

* security(E3): gate GSTACK_SLUG on /welcome path traversal

The /welcome handler interpolates GSTACK_SLUG directly into the filesystem
path used to locate the project-local welcome page. Without validation, a
slug like "../../etc/passwd" would resolve to
~/.gstack/projects/../../etc/passwd/designs/welcome-page-20260331/finalized.html
— classic path traversal.

Not exploitable today: GSTACK_SLUG is set by the gstack CLI at daemon launch,
and an attacker would already need local env-var access to poison it. But
the gate is one regex (^[a-z0-9_-]+$), and a defense-in-depth pass costs us
nothing when the cost of being wrong is arbitrary file read via /welcome.

Fall back to the safe 'unknown' literal when the slug fails validation —
same fallback the code already uses when GSTACK_SLUG is unset. No behavior
change for legitimate slugs (they all match the regex).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* security(N1): replace ?token= SSE auth with HttpOnly session cookie

Activity stream and inspector events SSE endpoints accepted the root
AUTH_TOKEN via `?token=` query param (EventSource can't send Authorization
headers). URLs leak to browser history, referer headers, server logs,
crash reports, and refactoring accidents. Codex flagged this during the
/plan-ceo-review outside voice pass.

New auth model: the extension calls POST /sse-session with a Bearer token
and receives a view-only session cookie (HttpOnly, SameSite=Strict, 30-min
TTL). EventSource is opened with `withCredentials: true` so the browser
sends the cookie back on the SSE connection. The ?token= query param is
GONE — no more URL-borne secrets.

Scope isolation (prior learning cookie-picker-auth-isolation, 10/10
confidence): the SSE session cookie grants access to /activity/stream and
/inspector/events ONLY. The token is never valid against /command, /token,
or any mutating endpoint. A leaked cookie can watch activity; it cannot
execute browser commands.

Components
  * browse/src/sse-session-cookie.ts — registry: mint/validate/extract/
    build-cookie. 256-bit tokens, 30-min TTL, lazy expiry pruning,
    no imports from token-registry (scope isolation enforced by module
    boundary).
  * browse/src/server.ts — POST /sse-session mint endpoint (requires
    Bearer). /activity/stream and /inspector/events now accept Bearer
    OR the session cookie, and reject ?token= query param.
  * extension/sidepanel.js — ensureSseSessionCookie() bootstrap call,
    EventSource opened with withCredentials:true on both SSE endpoints.
    Tested via the source guards; behavioral test is the E2E pairing
    flow that lands later in the wave.
  * browse/test/sse-session-cookie.test.ts — 20 unit tests covering
    mint entropy, TTL enforcement, cookie flag invariants, cookie
    parsing from multi-cookie headers, and scope-isolation contract
    guard (module must not import token-registry).
  * browse/test/server-auth.test.ts — existing /activity/stream auth
    test updated to assert the new cookie-based gate and the absence
    of the ?token= query param.

Cookie flag choices:
  * HttpOnly: token not readable from page JS (mitigates XSS
    exfiltration).
  * SameSite=Strict: cookie not sent on cross-site requests (mitigates
    CSRF). Fine for SSE because the extension connects to 127.0.0.1
    directly.
  * Path=/: cookie scoped to the whole origin.
  * Max-Age=1800: 30 minutes, matches TTL. Extension re-mints on
    reconnect when daemon restarts.
  * Secure NOT set: daemon binds to 127.0.0.1 over plain HTTP. Adding
    Secure would block the browser from ever sending the cookie back.
    Add Secure when gstack ships over HTTPS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* security(N2): document Windows v20 ABE elevation path on CDP port

The existing comment around the cookie-import-browser --remote-debugging-port
launch claimed "threat model: no worse than baseline." That's wrong on
Windows with App-Bound Encryption v20. A same-user local process that
opens the cookie SQLite DB directly CANNOT decrypt v20 values (DPAPI
context is bound to the browser process). The CDP port lets them bypass
that: connect to the debug port, call Network.getAllCookies inside Chrome,
walk away with decrypted v20 cookies.

The correct fix is to switch from TCP --remote-debugging-port to
--remote-debugging-pipe so the CDP transport is a stdio pipe, not a
socket. That requires restructuring the CDP WebSocket client in this
module and Playwright doesn't expose the pipe transport out of the box.
Non-trivial, deferred from the v1.6.0.0 wave.

This commit updates the comment to correctly describe the threat and
points at the tracking issue. No code change to the launch itself.
Follow-up: #1136.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(E2): document dual-listener tunnel architecture in ARCHITECTURE.md

Adds an explicit per-endpoint disposition table to the Security model
section, covering the v1.6.0.0 dual-listener refactor. Every HTTP
endpoint now has a documented local-vs-tunnel answer. Future audits
(and future contributors wondering "is it safe to add X to the tunnel
surface?") can read this instead of reverse-engineering server.ts.

Also documents:
  * Why physical port separation beats per-request header inference
    (ngrok behavior drift, local proxies can forge headers, etc.)
  * Tunnel surface denial logging → ~/.gstack/security/attempts.jsonl
  * SSE session cookie model (gstack_sse, 30-min TTL, stream-scope only,
    module-boundary-enforced scope isolation)
  * N2 non-goal for Windows v20 ABE via CDP port (tracking #1136)

No code changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(E1): end-to-end pair-agent flow against a spawned daemon

Spawns the browse daemon as a subprocess with BROWSE_HEADLESS_SKIP=1 so
the HTTP layer runs without a real browser.  Exercises:

  * GET /health — token delivery for chrome-extension origin, withheld
    otherwise (the F1 + PR #1026 invariant)
  * GET /connect — alive probe returns {alive:true} unauth
  * POST /pair — root Bearer required (403 without), returns setup_key
  * POST /connect — setup_key exchange mints a distinct scoped token
  * POST /command — 401 without auth
  * POST /sse-session — Bearer required, Set-Cookie has HttpOnly +
    SameSite=Strict (the N1 invariant)
  * GET /activity/stream — 401 without auth
  * GET /activity/stream?token= — 401 (the old ?token= query param is
    REJECTED, which is the whole point of N1)
  * GET /welcome — serves HTML, does not leak /etc/passwd content under
    the default 'unknown' slug (E3 regex gate)

12 behavioral tests, ~220ms end-to-end, no network dependencies, no
ngrok, no real browser.  This is the receipt for the wave's central
'pair-agent still works + the security boundary holds' claim.

Tunnel-port binding (/tunnel/start) is deliberately NOT exercised here
— it requires an ngrok authtoken and live network.  The dual-listener
route allowlist is covered by source-level guards in
dual-listener.test.ts; behavioral tunnel testing belongs in a separate
paid-evals harness.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* release(v1.6.0.0): bump VERSION + CHANGELOG for security wave

Architectural bump, not patch: dual-listener HTTP refactor changes the
daemon's tunnel-exposure model.  See CHANGELOG for the full release
summary (~950 words) covering the five root causes this wave closes:

  1. /health token leak over ngrok (F1 + E3 + test infra)
  2. /cookie-picker + /inspector exposed over the tunnel (F1)
  3. ?token=<ROOT> in SSE URLs leaking to logs/referer/history (N1)
  4. /welcome GSTACK_SLUG path traversal (E3)
  5. Windows v20 ABE elevation via CDP port (N2 — documented non-goal,
     tracked as #1136)

Plus the base PRs: SSRF gate (#1029), envelope sentinel escape (#1031),
DOM-channel hidden-element coverage (#1032), --from-file path validation
(#1103), and 2 commits from #1073 (@theqazi).

VERSION + package.json bumped to 1.6.0.0.  CHANGELOG entry covers
credits (@garagon, @Hybirdss, @HMAKT99, @theqazi), review lineage (CEO
→ Codex outside voice → Eng), and the non-goal tracking issue.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: pre-landing review findings (4 auto-fixes)

Addresses 4 findings from the Claude adversarial subagent on the
v1.6.0.0 security wave diff.  No user-visible behavior change; all
are defense-in-depth hardening of newly-introduced code.

1. GET /connect rate-limited (was POST-only) [HIGH conf 8/10]
   Attacker discovering the ngrok URL could probe unlimited GETs for
   daemon enumeration.  Now shares the global /connect counter.

2. ngrok listener leak on tunnel startup failure [MEDIUM conf 8/10]
   If ngrok.forward() resolved but tunnelListener.url() or the
   state-file write threw, the Bun listener was torn down but the
   ngrok session was leaked.  Fixed in BOTH /tunnel/start and
   BROWSE_TUNNEL=1 startup paths.

3. GSTACK_SKILL_ROOT path-traversal gate [MEDIUM conf 8/10]
   Symmetric with E3's GSTACK_SLUG regex gate — reject values
   containing '..' before interpolating into the welcome-page path.

4. SSE session registry pruning [LOW conf 7/10]
   pruneExpired() only checked 10 entries per mint call.  Now runs
   on every validate too, checks 20 entries, with a hard 10k cap as
   backstop.  Prevents registry growth under sustained extension
   reconnect pressure.

Tests remain green (56/56 in sse-session-cookie + dual-listener +
pair-agent-e2e suites).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update project documentation for v1.6.0.0

Reflect the dual-listener tunnel architecture, SSE session cookies,
SSRF guards, and Windows v20 ABE non-goal across the three docs
users actually read for remote-agent and browser auth context:

- docs/REMOTE_BROWSER_ACCESS.md: rewrote Architecture diagram for
  dual listeners, fixed /connect rate limit (3/min → 300/min),
  removed stale "/health requires no auth" (now 404 on tunnel),
  added SSE cookie auth, expanded Security Model with tunnel
  allowlist, SSRF guards, /welcome path traversal defense, and
  the Windows v20 ABE tracking note.
- BROWSER.md: added dual-listener paragraph to Authentication and
  linked to ARCHITECTURE.md endpoint table. Replaced the stale
  ?token= SSE auth note with the HttpOnly gstack_sse cookie flow.
- CLAUDE.md: added Transport-layer security section above the
  sidebar prompt-injection stack so contributors editing server.ts,
  sse-session-cookie.ts, or tunnel-denial-log.ts see the load-bearing
  module boundaries before touching them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(make-pdf): write --from-file payload to /tmp, not os.tmpdir()

make-pdf's browseClient wrote its --from-file payload to os.tmpdir(),
which is /var/folders/... on macOS. v1.6.0.0's PR #1103 cherry-pick
tightened browse load-html --from-file to validate against the
safe-dirs allowlist ([TEMP_DIR, cwd] where TEMP_DIR is '/tmp' on
macOS/Linux, os.tmpdir() on Windows). This closed a CLI/API parity
gap but broke make-pdf on macOS because /var/folders/... is outside
the allowlist.

Fix: mirror browse's TEMP_DIR convention — use '/tmp' on non-Windows,
os.tmpdir() on Windows. The make-pdf-gate CI failure on macOS-latest
(run 72440797490) is caused by exactly this: the payload file was
rejected by validateReadPath.

Verified locally: the combined-gate e2e test now passes after
rebuilding make-pdf/dist/pdf.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sidebar): killAgent resets per-tab state; align tests with current agent event format

Two pre-existing bugs surfaced while running the full e2e suite on the
sec-wave branch.  Both pre-date v1.6.0.0 (same failures on main at
e23ff280) but blocked the ship verification, so fixing now.

### Bug 1: killAgent leaked stale per-tab state

`killAgent()` reset the legacy globals (agentProcess, agentStatus,
etc.) but never touched the per-tab `tabAgents` Map.  Meanwhile
`/sidebar-command` routes on `tabState.status` from that Map, not the
legacy globals.  Consequence: after a kill (including the implicit
kill in `/sidebar-session/new`), the next /sidebar-command on the
same tab saw `tabState.status === 'processing'` and fell into the
queue branch, silently NOT spawning an agent.  Integration tests that
called resetState between cases all failed with empty queues.

Fix: when targetTabId is supplied, reset that one tab's state; when
called without a tab (session-new, full kill), reset ALL tab states.
Matches the semantic boundary already used for the cancel-file write.

### Bug 2: sidebar-integration tests drifted from current event format

`agent events appear in /sidebar-chat` posted the raw Claude streaming
format (`{type: 'assistant', message: {content: [...]}}`) but
`processAgentEvent` in server.ts only handles the simplified types
that sidebar-agent.ts pre-processes into (text, text_delta, tool_use,
result, agent_error, security_event).  The architecture moved
pre-processing into sidebar-agent.ts at some point and this test
never got updated.  Fixed by sending the pre-processed `{type:
'text', text: '...'}` format — which is actually what the server sees
in production.

Also removed the `entry.prompt` URL-containment check in the
queue-write test.  The URL is carried on entry.pageUrl (metadata) by
design: the system prompt tells Claude to run `browse url` to fetch
the actual page rather than trust any URL in the prompt body.  That's
the URL-based prompt-injection defense.  The prompt SHOULD NOT
contain the URL, so the test assertion was wrong for the current
security posture.

### Verification

- `bun test browse/test/sidebar-integration.test.ts` → 13/13 pass
  (was 6/13 on both main and branch before this commit)
- Full `bun run test` → exit 0, zero fail markers
- No behavior change for production sidebar flows: killAgent was
  already supposed to return the agent to idle; it just wasn't fully
  doing so.  Per-tab reset now matches the documented semantics.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: gus <gustavoraularagon@gmail.com>
Co-authored-by: Mohammed Qazi <10266060+theqazi@users.noreply.github.com>
2026-04-21 21:58:27 -07:00

29 KiB
Raw Blame History

Architecture

This document explains why gstack is built the way it is. For setup and commands, see CLAUDE.md. For contributing, see CONTRIBUTING.md.

The core idea

gstack gives Claude Code a persistent browser and a set of opinionated workflow skills. The browser is the hard part — everything else is Markdown.

The key insight: an AI agent interacting with a browser needs sub-second latency and persistent state. If every command cold-starts a browser, you're waiting 3-5 seconds per tool call. If the browser dies between commands, you lose cookies, tabs, and login sessions. So gstack runs a long-lived Chromium daemon that the CLI talks to over localhost HTTP.

Claude Code                     gstack
─────────                      ──────
                               ┌──────────────────────┐
  Tool call: $B snapshot -i    │  CLI (compiled binary)│
  ─────────────────────────→   │  • reads state file   │
                               │  • POST /command      │
                               │    to localhost:PORT   │
                               └──────────┬───────────┘
                                          │ HTTP
                               ┌──────────▼───────────┐
                               │  Server (Bun.serve)   │
                               │  • dispatches command  │
                               │  • talks to Chromium   │
                               │  • returns plain text  │
                               └──────────┬───────────┘
                                          │ CDP
                               ┌──────────▼───────────┐
                               │  Chromium (headless)   │
                               │  • persistent tabs     │
                               │  • cookies carry over  │
                               │  • 30min idle timeout  │
                               └───────────────────────┘

First call starts everything (~3s). Every call after: ~100-200ms.

Why Bun

Node.js would work. Bun is better here for three reasons:

  1. Compiled binaries. bun build --compile produces a single ~58MB executable. No node_modules at runtime, no npx, no PATH configuration. The binary just runs. This matters because gstack installs into ~/.claude/skills/ where users don't expect to manage a Node.js project.

  2. Native SQLite. Cookie decryption reads Chromium's SQLite cookie database directly. Bun has new Database() built in — no better-sqlite3, no native addon compilation, no gyp. One less thing that breaks on different machines.

  3. Native TypeScript. The server runs as bun run server.ts during development. No compilation step, no ts-node, no source maps to debug. The compiled binary is for deployment; source files are for development.

  4. Built-in HTTP server. Bun.serve() is fast, simple, and doesn't need Express or Fastify. The server handles ~10 routes total. A framework would be overhead.

The bottleneck is always Chromium, not the CLI or server. Bun's startup speed (~1ms for the compiled binary vs ~100ms for Node) is nice but not the reason we chose it. The compiled binary and native SQLite are.

The daemon model

Why not start a browser per command?

Playwright can launch Chromium in ~2-3 seconds. For a single screenshot, that's fine. For a QA session with 20+ commands, it's 40+ seconds of browser startup overhead. Worse: you lose all state between commands. Cookies, localStorage, login sessions, open tabs — all gone.

The daemon model means:

  • Persistent state. Log in once, stay logged in. Open a tab, it stays open. localStorage persists across commands.
  • Sub-second commands. After the first call, every command is just an HTTP POST. ~100-200ms round-trip including Chromium's work.
  • Automatic lifecycle. The server auto-starts on first use, auto-shuts down after 30 minutes idle. No process management needed.

State file

The server writes .gstack/browse.json (atomic write via tmp + rename, mode 0o600):

{ "pid": 12345, "port": 34567, "token": "uuid-v4", "startedAt": "...", "binaryVersion": "abc123" }

The CLI reads this file to find the server. If the file is missing or the server fails an HTTP health check, the CLI spawns a new server. On Windows, PID-based process detection is unreliable in Bun binaries, so the health check (GET /health) is the primary liveness signal on all platforms.

Port selection

Random port between 10000-60000 (retry up to 5 on collision). This means 10 Conductor workspaces can each run their own browse daemon with zero configuration and zero port conflicts. The old approach (scanning 9400-9409) broke constantly in multi-workspace setups.

Version auto-restart

The build writes git rev-parse HEAD to browse/dist/.version. On each CLI invocation, if the binary's version doesn't match the running server's binaryVersion, the CLI kills the old server and starts a new one. This prevents the "stale binary" class of bugs entirely — rebuild the binary, next command picks it up automatically.

Security model

Localhost only

The HTTP server binds to 127.0.0.1, not 0.0.0.0. It's not reachable from the network.

Dual-listener tunnel architecture (v1.6.0.0)

When a user runs pair-agent --client, the daemon starts an ngrok tunnel so a remote paired agent can drive the browser. Exposing the full daemon surface to the internet (even behind a random ngrok subdomain) meant /health leaked the root token on any Origin spoof, and /cookie-picker embedded the token into HTML that any caller could fetch.

The fix is two HTTP listeners, not one:

  • Local listener (127.0.0.1:LOCAL_PORT) — always bound. Serves bootstrap (/health with token delivery), /cookie-picker, /inspector/*, /welcome, /refs, the sidebar-agent API, and the full command surface. Never forwarded.
  • Tunnel listener (127.0.0.1:TUNNEL_PORT) — bound lazily on /tunnel/start, torn down on /tunnel/stop. Serves a locked allowlist: /connect (pairing ceremony, unauth + rate-limited), /command (scoped tokens only, further restricted to a browser-driving command allowlist), and /sidebar-chat. Everything else 404s.

ngrok forwards only the tunnel port. The security property comes from physical port separation: a tunnel caller cannot reach /health or /cookie-picker because those paths don't exist on that TCP socket. Header inference (check x-forwarded-for, check origin) is unreliable (ngrok header behavior changes; local proxies can add these headers); socket separation isn't.

Endpoint Local listener Tunnel listener Notes
GET /health public (no token unless headed/extension) 404 Token bootstrap for extension happens locally only
GET /connect public ({alive:true}) public ({alive:true}) Probe path for tunnel liveness
POST /connect public (rate-limited 300/min) public (rate-limited) Setup-key exchange for pair-agent
POST /command auth (Bearer root OR scoped) auth (scoped only, allowlisted commands) Root token on tunnel = 403
POST /sidebar-chat auth auth Lets remote agent post into local sidebar
POST /pair root-only 404 Pairing mint — local operator action
POST /tunnel/{start,stop} root-only 404 Daemon configuration
POST /token, DELETE /token/:id root-only 404 Scoped token mint/revoke
GET /cookie-picker, GET /cookie-picker/* public UI, auth API 404 Local-only — reads local browser DBs
GET /inspector, /inspector/events, etc. auth 404 Extension callback, local-only
GET /welcome public 404 GStack Browser landing page, local-only
GET /refs auth 404 Ref map — internal state
GET /activity/stream Bearer OR HttpOnly gstack_sse cookie 404 SSE. ?token= query param no longer accepted
GET /inspector/events Bearer OR HttpOnly gstack_sse cookie 404 SSE. Same cookie as /activity/stream
POST /sse-session auth (Bearer) 404 Mints the view-only 30-min SSE session cookie

Tunnel surface denial logs. Every rejection on the tunnel listener (path_not_on_tunnel, root_token_on_tunnel, missing_scoped_token, disallowed_command:*) is recorded asynchronously to ~/.gstack/security/attempts.jsonl with timestamp, source IP (from x-forwarded-for), path, and method. Rate-capped at 60 writes/min globally to prevent log-flood DoS. Shares the attempt log with the prompt-injection scanner.

SSE session cookies. EventSource can't send Authorization headers, so the extension POSTs /sse-session once at bootstrap with the root Bearer and receives a 30-minute view-only cookie (gstack_sse, HttpOnly, SameSite=Strict). The cookie is valid ONLY for /activity/stream and /inspector/events — it is NOT a scoped token and cannot be used on /command. Scope isolation is enforced by the module boundary: sse-session-cookie.ts has no imports from token-registry.ts.

Non-goal in this wave (tracked as #1136): the cookie-import-browser path launches Chrome with --remote-debugging-port=<random>. On Windows with App-Bound Encryption v20, a same-user local process can connect to that port and exfiltrate decrypted v20 cookies — an elevation path relative to reading the SQLite DB directly (which can't decrypt v20 without DPAPI context). Fix direction is --remote-debugging-pipe instead of TCP; requires restructuring the CDP client.

Bearer token auth

Every server session generates a random UUID token, written to the state file with mode 0o600 (owner-only read). Every HTTP request that mutates browser state must include Authorization: Bearer <token>. If the token doesn't match, the server returns 401.

This prevents other processes on the same machine from talking to your browse server. The cookie picker UI (/cookie-picker) and health check (/health) are exempt on the local listener — they're 127.0.0.1-bound and don't execute commands. On the tunnel listener nothing is exempt except /connect.

Cookies are the most sensitive data gstack handles. The design:

  1. Keychain access requires user approval. First cookie import per browser triggers a macOS Keychain dialog. The user must click "Allow" or "Always Allow." gstack never silently accesses credentials.

  2. Decryption happens in-process. Cookie values are decrypted in memory (PBKDF2 + AES-128-CBC), loaded into the Playwright context, and never written to disk in plaintext. The cookie picker UI never displays cookie values — only domain names and counts.

  3. Database is read-only. gstack copies the Chromium cookie DB to a temp file (to avoid SQLite lock conflicts with the running browser) and opens it read-only. It never modifies your real browser's cookie database.

  4. Key caching is per-session. The Keychain password + derived AES key are cached in memory for the server's lifetime. When the server shuts down (idle timeout or explicit stop), the cache is gone.

  5. No cookie values in logs. Console, network, and dialog logs never contain cookie values. The cookies command outputs cookie metadata (domain, name, expiry) but values are truncated.

Shell injection prevention

The browser registry (Comet, Chrome, Arc, Brave, Edge) is hardcoded. Database paths are constructed from known constants, never from user input. Keychain access uses Bun.spawn() with explicit argument arrays, not shell string interpolation.

Prompt injection defense (sidebar agent)

The Chrome sidebar agent has tools (Bash, Read, Glob, Grep, WebFetch) and reads hostile web pages, so it's the part of gstack most exposed to prompt injection. Defense is layered, not single-point.

  1. L1-L3 content security (browse/src/content-security.ts). Runs on every page-content command and every tool output: datamarking, hidden-element strip, ARIA regex, URL blocklist, and a trust-boundary envelope wrapper. Applied at both the server and the agent.

  2. L4 ML classifier — TestSavantAI (browse/src/security-classifier.ts). A 22MB BERT-small ONNX model (int8 quantized) bundled with the agent. Runs locally, no network. Scans every user message and every Read/Glob/Grep/WebFetch tool output before Claude sees it. Opt-in 721MB DeBERTa-v3 ensemble via GSTACK_SECURITY_ENSEMBLE=deberta.

  3. L4b transcript classifier. A Claude Haiku pass that looks at the full conversation shape (user message, tool calls, tool output), not just text. Gated by LOG_ONLY: 0.40 so most clean traffic skips the paid call.

  4. L5 canary token (browse/src/security.ts). A random token injected into the system prompt at session start. Rolling-buffer detection across text_delta and input_json_delta streams catches the token if it shows up anywhere in Claude's output, tool arguments, URLs, or file writes. Deterministic BLOCK — if the token leaks, the attacker convinced Claude to reveal the system prompt, and the session ends.

  5. L6 ensemble combiner (combineVerdict). BLOCK requires agreement from two ML classifiers at >= WARN (0.60), not a single confident hit. This is the Stack Overflow instruction-writing false-positive mitigation. On tool-output scans, single-layer high confidence BLOCKs directly — the content wasn't user-authored, so the FP concern doesn't apply.

Critical constraint: security-classifier.ts runs only in the sidebar-agent process, never in the compiled browse binary. @huggingface/transformers v4 requires onnxruntime-node, which fails dlopen from Bun compile's temp extract directory. Only the pure-string pieces (canary inject/check, verdict combiner, attack log, status) are in security.ts, which is safe to import from server.ts.

Env knobs: GSTACK_SECURITY_OFF=1 is a real kill switch (skips ML scan, canary still injects). Model cache at ~/.gstack/models/testsavant-small/ (112MB, first run) and ~/.gstack/models/deberta-v3-injection/ (721MB, opt-in only). Attack log at ~/.gstack/security/attempts.jsonl (salted sha256 + domain, rotates at 10MB, 5 generations). Per-device salt at ~/.gstack/security/device-salt (0600), cached in-process to survive FS-unwritable environments.

Visibility. The sidebar header shows a shield icon (green/amber/red) polled via /sidebar-chat. A centered banner appears on canary leak or BLOCK verdict with the exact layer scores. bin/gstack-security-dashboard aggregates local attempts; supabase/functions/community-pulse aggregates opt-in community telemetry across users.

The ref system

Refs (@e1, @e2, @c1) are how the agent addresses page elements without writing CSS selectors or XPath.

How it works

1. Agent runs: $B snapshot -i
2. Server calls Playwright's page.accessibility.snapshot()
3. Parser walks the ARIA tree, assigns sequential refs: @e1, @e2, @e3...
4. For each ref, builds a Playwright Locator: getByRole(role, { name }).nth(index)
5. Stores Map<string, RefEntry> on the BrowserManager instance (role + name + Locator)
6. Returns the annotated tree as plain text

Later:
7. Agent runs: $B click @e3
8. Server resolves @e3 → Locator → locator.click()

Why Locators, not DOM mutation

The obvious approach is to inject data-ref="@e1" attributes into the DOM. This breaks on:

  • CSP (Content Security Policy). Many production sites block DOM modification from scripts.
  • React/Vue/Svelte hydration. Framework reconciliation can strip injected attributes.
  • Shadow DOM. Can't reach inside shadow roots from the outside.

Playwright Locators are external to the DOM. They use the accessibility tree (which Chromium maintains internally) and getByRole() queries. No DOM mutation, no CSP issues, no framework conflicts.

Ref lifecycle

Refs are cleared on navigation (the framenavigated event on the main frame). This is correct — after navigation, all locators are stale. The agent must run snapshot again to get fresh refs. This is by design: stale refs should fail loudly, not click the wrong element.

Ref staleness detection

SPAs can mutate the DOM without triggering framenavigated (e.g. React router transitions, tab switches, modal opens). This makes refs stale even though the page URL didn't change. To catch this, resolveRef() performs an async count() check before using any ref:

resolveRef(@e3) → entry = refMap.get("e3")
                → count = await entry.locator.count()
                → if count === 0: throw "Ref @e3 is stale — element no longer exists. Run 'snapshot' to get fresh refs."
                → if count > 0: return { locator }

This fails fast (~5ms overhead) instead of letting Playwright's 30-second action timeout expire on a missing element. The RefEntry stores role and name metadata alongside the Locator so the error message can tell the agent what the element was.

Cursor-interactive refs (@c)

The -C flag finds elements that are clickable but not in the ARIA tree — things styled with cursor: pointer, elements with onclick attributes, or custom tabindex. These get @c1, @c2 refs in a separate namespace. This catches custom components that frameworks render as <div> but are actually buttons.

Logging architecture

Three ring buffers (50,000 entries each, O(1) push):

Browser events → CircularBuffer (in-memory) → Async flush to .gstack/*.log

Console messages, network requests, and dialog events each have their own buffer. Flushing happens every 1 second — the server appends only new entries since the last flush. This means:

  • HTTP request handling is never blocked by disk I/O
  • Logs survive server crashes (up to 1 second of data loss)
  • Memory is bounded (50K entries × 3 buffers)
  • Disk files are append-only, readable by external tools

The console, network, and dialog commands read from the in-memory buffers, not disk. Disk files are for post-mortem debugging.

SKILL.md template system

The problem

SKILL.md files tell Claude how to use the browse commands. If the docs list a flag that doesn't exist, or miss a command that was added, the agent hits errors. Hand-maintained docs always drift from code.

The solution

SKILL.md.tmpl          (human-written prose + placeholders)
       ↓
gen-skill-docs.ts      (reads source code metadata)
       ↓
SKILL.md               (committed, auto-generated sections)

Templates contain the workflows, tips, and examples that require human judgment. Placeholders are filled from source code at build time:

Placeholder Source What it generates
{{COMMAND_REFERENCE}} commands.ts Categorized command table
{{SNAPSHOT_FLAGS}} snapshot.ts Flag reference with examples
{{PREAMBLE}} gen-skill-docs.ts Startup block: update check, session tracking, contributor mode, AskUserQuestion format
{{BROWSE_SETUP}} gen-skill-docs.ts Binary discovery + setup instructions
{{BASE_BRANCH_DETECT}} gen-skill-docs.ts Dynamic base branch detection for PR-targeting skills (ship, review, qa, plan-ceo-review)
{{QA_METHODOLOGY}} gen-skill-docs.ts Shared QA methodology block for /qa and /qa-only
{{DESIGN_METHODOLOGY}} gen-skill-docs.ts Shared design audit methodology for /plan-design-review and /design-review
{{REVIEW_DASHBOARD}} gen-skill-docs.ts Review Readiness Dashboard for /ship pre-flight
{{TEST_BOOTSTRAP}} gen-skill-docs.ts Test framework detection, bootstrap, CI/CD setup for /qa, /ship, /design-review
{{CODEX_PLAN_REVIEW}} gen-skill-docs.ts Optional cross-model plan review (Codex or Claude subagent fallback) for /plan-ceo-review and /plan-eng-review
{{DESIGN_SETUP}} resolvers/design.ts Discovery pattern for $D design binary, mirrors {{BROWSE_SETUP}}
{{DESIGN_SHOTGUN_LOOP}} resolvers/design.ts Shared comparison board feedback loop for /design-shotgun, /plan-design-review, /design-consultation
{{UX_PRINCIPLES}} resolvers/design.ts User behavioral foundations (scanning, satisficing, goodwill reservoir, trunk test) for /design-html, /design-shotgun, /design-review, /plan-design-review
{{GBRAIN_CONTEXT_LOAD}} resolvers/gbrain.ts Brain-first context search with keyword extraction, health awareness, and data-research routing. Injected into 10 brain-aware skills. Suppressed on non-brain hosts.
{{GBRAIN_SAVE_RESULTS}} resolvers/gbrain.ts Post-skill brain persistence with entity enrichment, throttle handling, and per-skill save instructions. 8 skill-specific save formats.

This is structurally sound — if a command exists in code, it appears in docs. If it doesn't exist, it can't appear.

The preamble

Every skill starts with a {{PREAMBLE}} block that runs before the skill's own logic. It handles five things in a single bash command:

  1. Update check — calls gstack-update-check, reports if an upgrade is available.
  2. Session tracking — touches ~/.gstack/sessions/$PPID and counts active sessions (files modified in the last 2 hours). When 3+ sessions are running, all skills enter "ELI16 mode" — every question re-grounds the user on context because they're juggling windows.
  3. Operational self-improvement — at the end of every skill session, the agent reflects on failures (CLI errors, wrong approaches, project quirks) and logs operational learnings to the project's JSONL file for future sessions.
  4. AskUserQuestion format — universal format: context, question, RECOMMENDATION: Choose X because ___, lettered options. Consistent across all skills.
  5. Search Before Building — before building infrastructure or unfamiliar patterns, search first. Three layers of knowledge: tried-and-true (Layer 1), new-and-popular (Layer 2), first-principles (Layer 3). When first-principles reasoning reveals conventional wisdom is wrong, the agent names the "eureka moment" and logs it. See ETHOS.md for the full builder philosophy.

Why committed, not generated at runtime?

Three reasons:

  1. Claude reads SKILL.md at skill load time. There's no build step when a user invokes /browse. The file must already exist and be correct.
  2. CI can validate freshness. gen:skill-docs --dry-run + git diff --exit-code catches stale docs before merge.
  3. Git blame works. You can see when a command was added and in which commit.

Template test tiers

Tier What Cost Speed
1 — Static validation Parse every $B command in SKILL.md, validate against registry Free <2s
2 — E2E via claude -p Spawn real Claude session, run each skill, check for errors ~$3.85 ~20min
3 — LLM-as-judge Sonnet scores docs on clarity/completeness/actionability ~$0.15 ~30s

Tier 1 runs on every bun test. Tiers 2+3 are gated behind EVALS=1. The idea is: catch 95% of issues for free, use LLMs only for judgment calls.

Command dispatch

Commands are categorized by side effects:

  • READ (text, html, links, console, cookies, ...): No mutations. Safe to retry. Returns page state.
  • WRITE (goto, click, fill, press, ...): Mutates page state. Not idempotent.
  • META (snapshot, screenshot, tabs, chain, ...): Server-level operations that don't fit neatly into read/write.

This isn't just organizational. The server uses it for dispatch:

if (READ_COMMANDS.has(cmd))   handleReadCommand(cmd, args, bm)
if (WRITE_COMMANDS.has(cmd))  handleWriteCommand(cmd, args, bm)
if (META_COMMANDS.has(cmd))   handleMetaCommand(cmd, args, bm, shutdown)

The help command returns all three sets so agents can self-discover available commands.

Error philosophy

Errors are for AI agents, not humans. Every error message must be actionable:

  • "Element not found" → "Element not found or not interactable. Run snapshot -i to see available elements."
  • "Selector matched multiple elements" → "Selector matched multiple elements. Use @refs from snapshot instead."
  • Timeout → "Navigation timed out after 30s. The page may be slow or the URL may be wrong."

Playwright's native errors are rewritten through wrapError() to strip internal stack traces and add guidance. The agent should be able to read the error and know what to do next without human intervention.

Crash recovery

The server doesn't try to self-heal. If Chromium crashes (browser.on('disconnected')), the server exits immediately. The CLI detects the dead server on the next command and auto-restarts. This is simpler and more reliable than trying to reconnect to a half-dead browser process.

E2E test infrastructure

Session runner (test/helpers/session-runner.ts)

E2E tests spawn claude -p as a completely independent subprocess — not via the Agent SDK, which can't nest inside Claude Code sessions. The runner:

  1. Writes the prompt to a temp file (avoids shell escaping issues)
  2. Spawns sh -c 'cat prompt | claude -p --output-format stream-json --verbose'
  3. Streams NDJSON from stdout for real-time progress
  4. Races against a configurable timeout
  5. Parses the full NDJSON transcript into structured results

The parseNDJSON() function is pure — no I/O, no side effects — making it independently testable.

Observability data flow

  skill-e2e-*.test.ts
        │
        │ generates runId, passes testName + runId to each call
        │
  ┌─────┼──────────────────────────────┐
  │     │                              │
  │  runSkillTest()              evalCollector
  │  (session-runner.ts)         (eval-store.ts)
  │     │                              │
  │  per tool call:              per addTest():
  │  ┌──┼──────────┐              savePartial()
  │  │  │          │                   │
  │  ▼  ▼          ▼                   ▼
  │ [HB] [PL]    [NJ]          _partial-e2e.json
  │  │    │        │             (atomic overwrite)
  │  │    │        │
  │  ▼    ▼        ▼
  │ e2e-  prog-  {name}
  │ live  ress   .ndjson
  │ .json .log
  │
  │  on failure:
  │  {name}-failure.json
  │
  │  ALL files in ~/.gstack-dev/
  │  Run dir: e2e-runs/{runId}/
  │
  │         eval-watch.ts
  │              │
  │        ┌─────┴─────┐
  │     read HB     read partial
  │        └─────┬─────┘
  │              ▼
  │        render dashboard
  │        (stale >10min? warn)

Split ownership: session-runner owns the heartbeat (current test state), eval-store owns partial results (completed test state). The watcher reads both. Neither component knows about the other — they share data only through the filesystem.

Non-fatal everything: All observability I/O is wrapped in try/catch. A write failure never causes a test to fail. The tests themselves are the source of truth; observability is best-effort.

Machine-readable diagnostics: Each test result includes exit_reason (success, timeout, error_max_turns, error_api, exit_code_N), timeout_at_turn, and last_tool_call. This enables jq queries like:

jq '.tests[] | select(.exit_reason == "timeout") | .last_tool_call' ~/.gstack-dev/evals/_partial-e2e.json

Eval persistence (test/helpers/eval-store.ts)

The EvalCollector accumulates test results and writes them in two ways:

  1. Incremental: savePartial() writes _partial-e2e.json after each test (atomic: write .tmp, fs.renameSync). Survives kills.
  2. Final: finalize() writes a timestamped eval file (e.g. e2e-20260314-143022.json). The partial file is never cleaned up — it persists alongside the final file for observability.

eval:compare diffs two eval runs. eval:summary aggregates stats across all runs in ~/.gstack-dev/evals/.

Test tiers

Tier What Cost Speed
1 — Static validation Parse $B commands, validate against registry, observability unit tests Free <5s
2 — E2E via claude -p Spawn real Claude session, run each skill, scan for errors ~$3.85 ~20min
3 — LLM-as-judge Sonnet scores docs on clarity/completeness/actionability ~$0.15 ~30s

Tier 1 runs on every bun test. Tiers 2+3 are gated behind EVALS=1. The idea: catch 95% of issues for free, use LLMs only for judgment calls and integration testing.

What's intentionally not here

  • No WebSocket streaming. HTTP request/response is simpler, debuggable with curl, and fast enough. Streaming would add complexity for marginal benefit.
  • No MCP protocol. MCP adds JSON schema overhead per request and requires a persistent connection. Plain HTTP + plain text output is lighter on tokens and easier to debug.
  • No multi-user support. One server per workspace, one user. The token auth is defense-in-depth, not multi-tenancy.
  • No Windows/Linux cookie decryption. macOS Keychain is the only supported credential store. Linux (GNOME Keyring/kwallet) and Windows (DPAPI) are architecturally possible but not implemented.
  • No iframe auto-discovery. $B frame supports cross-frame interaction (CSS selector, @ref, --name, --url matching), but the ref system does not auto-crawl iframes during snapshot. You must explicitly enter a frame context first.