Files
gstack/browse/test/token-registry.test.ts
Garry Tan b73f364411 feat: browser data platform for AI agents (v0.16.0.0) (#907)
* refactor: extract path-security.ts shared module

validateOutputPath, validateReadPath, and SAFE_DIRECTORIES were duplicated
across write-commands.ts, meta-commands.ts, and read-commands.ts. Extract
to a single shared module with re-exports for backward compatibility.

Also adds validateTempPath() for the upcoming GET /file endpoint (TEMP_DIR
only, not cwd, to prevent remote agents from reading project files).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: default paired agents to full access, split SCOPE_CONTROL

The trust boundary for paired agents is the pairing ceremony itself, not
the scope. An agent with write scope can already click anything and navigate
anywhere. Gating js/cookies behind --admin was security theater.

Changes:
- Default pair scopes: read+write+admin+meta (was read+write)
- New SCOPE_CONTROL for browser-wide destructive ops (stop, restart,
  disconnect, state, handoff, resume, connect)
- --admin flag now grants control scope (backward compat)
- New --restrict flag for limited access (e.g., --restrict read)
- Updated hint text: "re-pair with --control" instead of "--admin"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add media and data commands for page content extraction

media command: discovers all img/video/audio/background-image elements
on the page. Returns JSON with URLs, dimensions, srcset, loading state,
HLS/DASH detection. Supports --images/--videos/--audio filters and
optional CSS selector scoping.

data command: extracts structured data embedded in pages (JSON-LD,
Open Graph, Twitter Cards, meta tags). One command returns product
prices, article metadata, social share info without DOM scraping.

Both are READ scope with untrusted content wrapping.
Shared media-extract.ts helper for reuse by the upcoming scrape command.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add download, scrape, and archive commands

download: fetch any URL or @ref element to disk using browser session
cookies via page.request.fetch(). Supports blob: URLs via in-page
base64 conversion. --base64 flag returns inline data URI (cap 10MB).
Detects HLS/DASH and rejects with yt-dlp hint.

scrape: bulk media download composing media discovery + download loop.
Sequential with 100ms delay, URL deduplication, configurable --limit.
Writes manifest.json with per-file metadata for machine consumption.

archive: saves complete page as MHTML via CDP Page.captureSnapshot.
No silent fallback -- errors clearly if CDP unavailable.

All three are WRITE scope (write to disk, blocked in watch mode).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add GET /file endpoint for remote agent file retrieval

Remote paired agents can now retrieve downloaded files over HTTP.
TEMP_DIR only (not cwd) to prevent project file exfiltration.

- Bearer token auth (root or scoped with read scope)
- Path validation via validateTempPath() (symlink-aware)
- 200MB size cap
- Extension-based MIME detection
- Zero-copy streaming via Bun.file()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add scroll --times N for automated repeated scrolling

Extends the scroll command with --times N flag for infinite feed
scraping. Scrolls N times with configurable --wait delay (default
1000ms) between each scroll for content loading.

Usage: scroll --times 10
       scroll --times 5 --wait 2000
       scroll --times 3 .feed-container

Composable with scrape: scroll to load content, then scrape images.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add network response body capture (--capture/--export/--bodies)

The killer feature for social media scraping. Extends the existing
network command to intercept API response bodies:

  network --capture [--filter graphql]  # start capturing
  network --capture stop                # stop
  network --export /tmp/api.jsonl       # export as JSONL
  network --bodies                      # show summary

Uses page.on('response') listener with URL pattern filtering.
SizeCappedBuffer (50MB total, 5MB per-entry cap) evicts oldest
entries when full. Binary responses stored as base64, text as-is.

This lets agents tap Instagram's GraphQL API, TikTok's hydration
data, and any SPA's internal API responses instead of fragile DOM
scraping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add screenshot --base64 for inline image return

Returns data:image/png;base64,... instead of writing to disk.
Cap at 10MB. Works with all screenshot modes (element, clip, viewport).

Eliminates the two-step screenshot+file-serve dance for remote agents.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add data platform tests and media fixture

Tests for SizeCappedBuffer (eviction, export, summary), validateTempPath
(TEMP_DIR only, rejects cwd), command registration (all new commands in
correct scope sets), and MIME mapping source checks.

Rich HTML fixture with: standard images, lazy-loaded images, srcset,
video with sources + HLS, audio, CSS background-images, JSON-LD,
Open Graph, Twitter Cards, and meta tags.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: regenerate SKILL.md with Extraction category

Add Extraction category to browse command table ordering. Regenerate
SKILL.md files to include media, data, download, scrape, archive
commands in the generated documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.16.0.0)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 00:41:55 -07:00

404 lines
14 KiB
TypeScript

import { describe, it, expect, beforeEach } from 'bun:test';
import {
initRegistry, getRootToken, isRootToken,
createToken, createSetupKey, exchangeSetupKey,
validateToken, checkScope, checkDomain, checkRate,
revokeToken, rotateRoot, listTokens, recordCommand,
serializeRegistry, restoreRegistry, checkConnectRateLimit,
SCOPE_READ, SCOPE_WRITE, SCOPE_ADMIN, SCOPE_CONTROL, SCOPE_META,
} from '../src/token-registry';
describe('token-registry', () => {
beforeEach(() => {
// rotateRoot clears all tokens and rate buckets, then initRegistry sets the root
rotateRoot();
initRegistry('root-token-for-tests');
});
describe('root token', () => {
it('identifies root token correctly', () => {
expect(isRootToken('root-token-for-tests')).toBe(true);
expect(isRootToken('not-root')).toBe(false);
});
it('validates root token with full scopes', () => {
const info = validateToken('root-token-for-tests');
expect(info).not.toBeNull();
expect(info!.clientId).toBe('root');
expect(info!.scopes).toEqual(['read', 'write', 'admin', 'meta', 'control']);
expect(info!.rateLimit).toBe(0);
});
});
describe('createToken', () => {
it('creates a session token with defaults', () => {
const info = createToken({ clientId: 'test-agent' });
expect(info.token).toStartWith('gsk_sess_');
expect(info.clientId).toBe('test-agent');
expect(info.type).toBe('session');
expect(info.scopes).toEqual(['read', 'write']);
expect(info.tabPolicy).toBe('own-only');
expect(info.rateLimit).toBe(10);
expect(info.expiresAt).not.toBeNull();
expect(info.commandCount).toBe(0);
});
it('creates token with custom scopes', () => {
const info = createToken({
clientId: 'admin-agent',
scopes: ['read', 'write', 'admin'],
rateLimit: 20,
expiresSeconds: 3600,
});
expect(info.scopes).toEqual(['read', 'write', 'admin']);
expect(info.rateLimit).toBe(20);
});
it('creates token with indefinite expiry', () => {
const info = createToken({
clientId: 'forever',
expiresSeconds: null,
});
expect(info.expiresAt).toBeNull();
});
it('overwrites existing token for same clientId', () => {
const first = createToken({ clientId: 'agent-1' });
const second = createToken({ clientId: 'agent-1' });
expect(first.token).not.toBe(second.token);
expect(validateToken(first.token)).toBeNull();
expect(validateToken(second.token)).not.toBeNull();
});
});
describe('setup key exchange', () => {
it('creates setup key with 5-minute expiry', () => {
const setup = createSetupKey({});
expect(setup.token).toStartWith('gsk_setup_');
expect(setup.type).toBe('setup');
expect(setup.usesRemaining).toBe(1);
});
it('exchanges setup key for session token', () => {
const setup = createSetupKey({ clientId: 'remote-1' });
const session = exchangeSetupKey(setup.token);
expect(session).not.toBeNull();
expect(session!.token).toStartWith('gsk_sess_');
expect(session!.clientId).toBe('remote-1');
expect(session!.type).toBe('session');
});
it('setup key is single-use', () => {
const setup = createSetupKey({});
exchangeSetupKey(setup.token);
// Second exchange with 0 commands should be idempotent
const second = exchangeSetupKey(setup.token);
expect(second).not.toBeNull(); // idempotent — session has 0 commands
});
it('idempotent exchange fails after commands are executed', () => {
const setup = createSetupKey({});
const session = exchangeSetupKey(setup.token);
// Simulate command execution
recordCommand(session!.token);
// Now re-exchange should fail
const retry = exchangeSetupKey(setup.token);
expect(retry).toBeNull();
});
it('rejects expired setup key', () => {
const setup = createSetupKey({});
// Manually expire it
const info = validateToken(setup.token);
if (info) {
(info as any).expiresAt = new Date(Date.now() - 1000).toISOString();
}
const session = exchangeSetupKey(setup.token);
expect(session).toBeNull();
});
it('rejects unknown setup key', () => {
expect(exchangeSetupKey('gsk_setup_nonexistent')).toBeNull();
});
it('rejects session token as setup key', () => {
const session = createToken({ clientId: 'test' });
expect(exchangeSetupKey(session.token)).toBeNull();
});
});
describe('validateToken', () => {
it('validates active session token', () => {
const created = createToken({ clientId: 'valid' });
const info = validateToken(created.token);
expect(info).not.toBeNull();
expect(info!.clientId).toBe('valid');
});
it('rejects unknown token', () => {
expect(validateToken('gsk_sess_unknown')).toBeNull();
});
it('rejects expired token', async () => {
// expiresSeconds: 0 creates a token that expires at creation time
const created = createToken({ clientId: 'expiring', expiresSeconds: 0 });
// Wait 1ms so the expiry is definitively in the past
await new Promise(r => setTimeout(r, 2));
expect(validateToken(created.token)).toBeNull();
});
});
describe('checkScope', () => {
it('allows read commands with read scope', () => {
const info = createToken({ clientId: 'reader', scopes: ['read'] });
expect(checkScope(info, 'snapshot')).toBe(true);
expect(checkScope(info, 'text')).toBe(true);
expect(checkScope(info, 'html')).toBe(true);
});
it('denies write commands with read-only scope', () => {
const info = createToken({ clientId: 'reader', scopes: ['read'] });
expect(checkScope(info, 'click')).toBe(false);
expect(checkScope(info, 'goto')).toBe(false);
expect(checkScope(info, 'fill')).toBe(false);
});
it('denies admin commands without admin scope', () => {
const info = createToken({ clientId: 'normal', scopes: ['read', 'write'] });
expect(checkScope(info, 'eval')).toBe(false);
expect(checkScope(info, 'js')).toBe(false);
expect(checkScope(info, 'cookies')).toBe(false);
expect(checkScope(info, 'storage')).toBe(false);
});
it('allows admin commands with admin scope', () => {
const info = createToken({ clientId: 'admin', scopes: ['read', 'write', 'admin'] });
expect(checkScope(info, 'eval')).toBe(true);
expect(checkScope(info, 'cookies')).toBe(true);
});
it('allows chain with meta scope', () => {
const info = createToken({ clientId: 'meta', scopes: ['read', 'meta'] });
expect(checkScope(info, 'chain')).toBe(true);
});
it('denies chain without meta scope', () => {
const info = createToken({ clientId: 'no-meta', scopes: ['read'] });
expect(checkScope(info, 'chain')).toBe(false);
});
it('root token allows everything', () => {
const root = validateToken('root-token-for-tests')!;
expect(checkScope(root, 'eval')).toBe(true);
expect(checkScope(root, 'state')).toBe(true);
expect(checkScope(root, 'stop')).toBe(true);
});
it('denies destructive commands without admin scope', () => {
const info = createToken({ clientId: 'normal', scopes: ['read', 'write'] });
expect(checkScope(info, 'useragent')).toBe(false);
expect(checkScope(info, 'state')).toBe(false);
expect(checkScope(info, 'handoff')).toBe(false);
expect(checkScope(info, 'stop')).toBe(false);
});
});
describe('checkDomain', () => {
it('allows any domain when no restrictions', () => {
const info = createToken({ clientId: 'unrestricted' });
expect(checkDomain(info, 'https://evil.com')).toBe(true);
});
it('matches exact domain', () => {
const info = createToken({ clientId: 'exact', domains: ['myapp.com'] });
expect(checkDomain(info, 'https://myapp.com/page')).toBe(true);
expect(checkDomain(info, 'https://evil.com')).toBe(false);
});
it('matches wildcard domain', () => {
const info = createToken({ clientId: 'wild', domains: ['*.myapp.com'] });
expect(checkDomain(info, 'https://api.myapp.com/v1')).toBe(true);
expect(checkDomain(info, 'https://myapp.com')).toBe(true);
expect(checkDomain(info, 'https://evil.com')).toBe(false);
});
it('root allows all domains', () => {
const root = validateToken('root-token-for-tests')!;
expect(checkDomain(root, 'https://anything.com')).toBe(true);
});
it('denies invalid URLs', () => {
const info = createToken({ clientId: 'strict', domains: ['myapp.com'] });
expect(checkDomain(info, 'not-a-url')).toBe(false);
});
});
describe('checkRate', () => {
it('allows requests under limit', () => {
const info = createToken({ clientId: 'rated', rateLimit: 10 });
for (let i = 0; i < 10; i++) {
expect(checkRate(info).allowed).toBe(true);
}
});
it('denies requests over limit', () => {
const info = createToken({ clientId: 'limited', rateLimit: 3 });
checkRate(info);
checkRate(info);
checkRate(info);
const result = checkRate(info);
expect(result.allowed).toBe(false);
expect(result.retryAfterMs).toBeGreaterThan(0);
});
it('root is unlimited', () => {
const root = validateToken('root-token-for-tests')!;
for (let i = 0; i < 100; i++) {
expect(checkRate(root).allowed).toBe(true);
}
});
});
describe('revokeToken', () => {
it('revokes existing token', () => {
const info = createToken({ clientId: 'to-revoke' });
expect(revokeToken('to-revoke')).toBe(true);
expect(validateToken(info.token)).toBeNull();
});
it('returns false for non-existent client', () => {
expect(revokeToken('no-such-client')).toBe(false);
});
});
describe('rotateRoot', () => {
it('generates new root and invalidates all tokens', () => {
const oldRoot = getRootToken();
createToken({ clientId: 'will-die' });
const newRoot = rotateRoot();
expect(newRoot).not.toBe(oldRoot);
expect(isRootToken(newRoot)).toBe(true);
expect(isRootToken(oldRoot)).toBe(false);
expect(listTokens()).toHaveLength(0);
});
});
describe('listTokens', () => {
it('lists active session tokens', () => {
createToken({ clientId: 'a' });
createToken({ clientId: 'b' });
createSetupKey({}); // setup keys not listed
expect(listTokens()).toHaveLength(2);
});
});
describe('serialization', () => {
it('serializes and restores registry', () => {
createToken({ clientId: 'persist-1', scopes: ['read'] });
createToken({ clientId: 'persist-2', scopes: ['read', 'write', 'admin'] });
const state = serializeRegistry();
expect(Object.keys(state.agents)).toHaveLength(2);
// Clear and restore
rotateRoot();
initRegistry('new-root');
restoreRegistry(state);
const restored = listTokens();
expect(restored).toHaveLength(2);
expect(restored.find(t => t.clientId === 'persist-1')?.scopes).toEqual(['read']);
});
});
describe('connect rate limit', () => {
it('allows up to 3 attempts per minute', () => {
// Reset by creating a new module scope (can't easily reset static state)
// Just verify the function exists and returns boolean
const result = checkConnectRateLimit();
expect(typeof result).toBe('boolean');
});
});
describe('scope coverage', () => {
it('every command in commands.ts is covered by a scope', () => {
// Import the command sets to verify coverage
const allInScopes = new Set([
...SCOPE_READ, ...SCOPE_WRITE, ...SCOPE_ADMIN, ...SCOPE_CONTROL, ...SCOPE_META,
]);
// chain is a special case (checked via meta scope but dispatches subcommands)
allInScopes.add('chain');
// These commands don't need scope coverage (server control, handled separately)
const exemptFromScope = new Set(['status', 'snapshot']);
// snapshot appears in both READ and META (it's read-safe)
// Verify dangerous commands are in admin scope
expect(SCOPE_ADMIN.has('eval')).toBe(true);
expect(SCOPE_ADMIN.has('js')).toBe(true);
expect(SCOPE_ADMIN.has('cookies')).toBe(true);
expect(SCOPE_ADMIN.has('storage')).toBe(true);
expect(SCOPE_ADMIN.has('useragent')).toBe(true);
// Browser-wide destructive commands moved to SCOPE_CONTROL
expect(SCOPE_CONTROL.has('state')).toBe(true);
expect(SCOPE_CONTROL.has('handoff')).toBe(true);
expect(SCOPE_CONTROL.has('stop')).toBe(true);
expect(SCOPE_CONTROL.has('restart')).toBe(true);
expect(SCOPE_CONTROL.has('disconnect')).toBe(true);
// Verify safe read commands are NOT in admin
expect(SCOPE_ADMIN.has('text')).toBe(false);
expect(SCOPE_ADMIN.has('snapshot')).toBe(false);
expect(SCOPE_ADMIN.has('screenshot')).toBe(false);
});
});
// ─── CSO Fix #4: Input validation ──────────────────────────────
describe('Input validation (CSO finding #4)', () => {
it('rejects invalid scope values', () => {
expect(() => createToken({
clientId: 'test-invalid-scope',
scopes: ['read', 'bogus' as any],
})).toThrow('Invalid scope: bogus');
});
it('rejects negative rateLimit', () => {
expect(() => createToken({
clientId: 'test-neg-rate',
rateLimit: -1,
})).toThrow('rateLimit must be >= 0');
});
it('rejects negative expiresSeconds', () => {
expect(() => createToken({
clientId: 'test-neg-expire',
expiresSeconds: -100,
})).toThrow('expiresSeconds must be >= 0 or null');
});
it('accepts null expiresSeconds (indefinite)', () => {
const token = createToken({
clientId: 'test-indefinite',
expiresSeconds: null,
});
expect(token.expiresAt).toBeNull();
});
it('accepts zero rateLimit (unlimited)', () => {
const token = createToken({
clientId: 'test-unlimited-rate',
rateLimit: 0,
});
expect(token.rateLimit).toBe(0);
});
it('accepts valid scopes', () => {
const token = createToken({
clientId: 'test-valid-scopes',
scopes: ['read', 'write', 'admin', 'meta'],
});
expect(token.scopes).toEqual(['read', 'write', 'admin', 'meta']);
});
});
});