v1.91.8.0 test: smaller suite, derived paid selection, retired never-green evals (#2994)

* test: delete test-infrastructure dead code (G)

- exit-propagation drives the runner's real strict verdict
  (BunTestOutputClassifier + strictTestExitCode); delete the unused
  shardRunLooksTruncated predicate.
- delete skill-coverage-matrix registry + its gate (nothing reads it; the
  floor already iterates skillCensus()).
- delete touchfiles-facade export-parity tests (Bun fails missing imports
  at link time) and the duplicated E2E_TIERS tier-value test.
- delete brain-cache-spec TRANSPORT_DEFAULT_POLICY, SKILL_RUN_RETENTION_DAYS
  and the now-unused BrainTrustPolicy type with their literal tests.
  AUTOPLAN_PREFLIGHT_BUDGET_BYTES stays: skill-preflight-budget enforces it
  against real resolver output.
- delete audit-compliance's JSDoc-comment grep.

* test: replace product tests that fake the product with real-boundary tests (F)

- design: serve.test.ts drove an inline mirror server; now two tests run the
  real serve() on an ephemeral port (reload confinement, submit exit 0).
- setup-gbrain: rollback + voyage tests execute the template-extracted init
  blocks (3 sites) instead of drifted local bash copies.
- terminal-agent: internalHandler source greps replaced by a behavioral
  /internal/grant + /internal/revoke auth matrix (no/wrong/valid token).
- /health: server-security-surface and the server-auth / security-audit-r2 /
  sidebar-tabs source greps fold into one liveness-only check on the real
  body; the L4 sidecar wiring gets a behavioral /pty-inject-scan test.
- delete tautologies (browser-manager onDisconnect, memory-command #12),
  ios swiftui tap fixture self-check, memory-ingest put_page grep, detach
  source greps, sidebar-agent absence pins, dead-CSS pins + the dead CSS,
  security-audit-r2 Task 1 + the test-only meta-commands re-export,
  duplicate generated-SKILL.md checks.
- make-pdf coverage-gaps cases move into their owner test files.

* test: delete tests of dead eval code (A)

- A1: the retired Eng lexical oracle (evaluateEngSeedCoverage,
  isEngSeedDecisionAUQ), the completion-handoff detector and the retained
  corpus had no paid caller since v1.87.6; delete their 26 replay files,
  ~2.6k helper LOC and fixtures, and the dead blocks in 8 mixed files
  (live hasNativePlanTerminal / batching assertions stay).
- A2: dead viewport approvers in autoplan-artifact-permission and their 11
  replay files + fixtures; recorder/launcher cases stay.
- A3: never-wired oracles and seeders (autoplan-phase-order,
  eng-finding-fixture, ceo-paired-fixture, design-ui-scope,
  plan-skill-completion, pty-current-screen, required-reads,
  transcript-section-logger); plan-seed-submission now decodes through the
  production createPtyScreen; section manifests name their actual guard.
- A4: zero-reference helper exports, plus execGit and invokeAndObserve
  found by the reachability pass.
- 52 fixtures orphaned by the deletions; touchfile and selection-table
  entries for every deleted path.

* test: clean up the paid eval lane (B1-B4, B6, B7)

- B1: delete paid files that assert nothing or cannot pass meaningfully:
  skill-llm-eval-spec and skill-e2e-spec-execute (test.todo), gemini-e2e
  (+ gemini-session-runner; no gemini CLI in CI), ship-idempotency (red
  since v1.63), the two opus-4-7 *-sonnet overlay wrappers, conductor-prose
  (+ its source-evaluation replay), codex-e2e-plan-format; drop their keys,
  scripts and census rows.
- B2: skill-llm-eval grades browse/sections/command-list.md with one union
  judge that also carries the baseline score pin; regression-vs-baseline
  deleted (paid run: pass, c4/c4/a4).
- B3: memory-pipeline, ios-qa, ios-qa-swift-build and plan-tune-cathedral
  make no model calls; renamed out of the paid glob so they run on every
  PR. Swift builds need GSTACK_TEST_SWIFT=1; device stub deleted.
- B4: codex-e2e*, outside-voice, aside and ios-device cannot run in the CI
  image; excluded from the weekly lane with a tracked re-entry condition.
- B6: fold opus-47's negative routing controls into skill-routing-e2e
  journey-negatives (paid run: 3/3 unrouted) and delete the file.
- B7: delete the never-green brain-privacy-gate eval; a free
  gstack-skill-start test now proves consent precedes artifacts egress.

* test: retire the finding-count cluster and trim its helpers (C)

- C0/C1: the five never-green evals (skill-e2e-autoplan-chain and
  skill-e2e-plan-{ceo,eng,design,devex}-finding-count) failed on harness and
  budget, never on skill behavior; delete them, their touchfile/tier ids,
  AUTOPLAN_CHAIN_BUDGET and the dedicated eighth periodic slice (--slices 7).
- C2: delete the helper groups whose only paid consumers were those files
  (11 modules), trim claude-pty-runner and eng-seeded-coverage to the paid
  closure, and delete the free replay tests whose assertions exercised only
  that dead code (89 files, 135 orphaned fixtures). Blocks that used dead code
  only as input for a live subject keep their assertions: the multiSelect
  default moved to plan-review-decisions, runner PTY tests use inline caller
  policies, and the timer-safe budget checks moved to eng-finding-retry-budget.
- The eight production-touching files stay except ceo-current-decision-record
  (its template read only feeds the retired counter).
- CARVE_GUARDS.autoplan is behavioral 'none'; TODOS records the lost chain
  and per-finding cadence coverage with their re-entry tests.

* test: fold per-incident replay series into their detector owners (D)

Twelve detector families move into one owner test each: 73 incident files
become describe blocks in ceo-section-loading-fixture (stale-fill race),
model-overlays, coverage-audit-evidence, autoplan-phase-observer,
native-auto-decide, outside-voice-evidence, eng-first-review,
plan-count-completion, plan-count-file-permission, ceo-mode-option,
plan-scope-selection and plan-count-prerequisite. Each block keeps its original
code and fixture, so every case still runs; only tests asserting the incident
file's own touchfile registration are dropped (41). Touchfile lists that named
an incident now name its owner.

* test: start the plan-count history PTY on its readiness marker (H)

The fake CLI prints a startup marker and the runner waits for it instead of the
fixed 8 s startup sleep (8.6 s -> 0.9 s locally). eng-semantic-terminal's
sleeping registration cases went with C; plan-count-timeout keeps the fixed wait
because it asserts deadline behavior.

* test: derive paid touchfiles from each eval's static closure (E)

touchfiles.test.ts now checks, per key, that the paid file's static
test/helpers and test/fixtures closure (plus fixture paths it names in string
literals) is covered, and names the file, path, chain and key to fix when it is
not. Free *.test.ts files are no longer touchfiles, so editing a free replay
test stops selecting paid evals: 950 entries removed, 653 real closure paths
added. The hand-copied inventories go: periodic-fixture-selection,
fake-impeccable-touchfiles and 45 per-file selection examples. Selection for
the sample edits (plan-eng-review template, claude-pty-runner,
plan-count-fixture, gstack-config) loses no case under either profile.
CONTRIBUTING documents the rule and its lower bound.

* test: skip hollow tier shards and census judges in the paid planner (B5)

A paid file is now skipped for a tier lane only when every E2E id it registers
is known statically and none has that tier; ids come from the touchfile
registrations and literal testName/*IfSelected arguments, so a comment or
skill path that quotes another id cannot unschedule it, and computed names
keep today's scheduling. --list and the manifest show each skip as
"skipped: no E2E_TIERS id has tier <tier>". The weekly gate census drops the
LLM judges (--skip-judges); they still run in the periodic census and PR gate
lanes. Gate lane 52 -> 42 files, census 41; periodic 77 -> 69.

* test: run seven paid evals on the current default capture model (B8)

skill-e2e-{auq-matrix,plan-format,qa-bugs,retro,workflow} pinned
claude-opus-4-7 and skill-e2e-office-hours plus -brain-writeback pinned
claude-sonnet-4-6; none tests a historical model, so they now capture with
resolveEvalModel('capture'), and the free harness tests that execute these
registrations receive the same resolver. The paid re-pin run passed all of
them. skill-e2e-{design,office-hours-phase4,plan-prosons,plan} keep
claude-opus-4-7: six of their cases failed on the default model (three
timeouts, a missing report file, a format miss and a posture score of 3), so
per the plan's fallback they keep their pins with a TODOS entry. The pre-spend
estimate and drop threshold are in docs/test-audit-2026-09.md.

* test: guard the reduced suite against new test-of-test files

- test/test-of-test-ratchet.test.ts records the 228 free tests that import only
  test/ code and fails on a new one, naming the owner test to extend instead;
  a stale baseline entry fails with the remove instruction.
- test/helpers/resolve-repo-path.ts is the one specifier/literal resolver for
  the ratchet and the touchfile closure invariant, with its own unit tests.
- CONTRIBUTING "Test tiers" describes the paid-failure workflow (fix, then one
  row in the detector's owner test) and the ratchet; TEST_PORTFOLIO gains the
  detector -> owner-test table and no longer claims an Autoplan chain eval.
- TODOS: automatic exclusion policy for chronically red periodic files (P3),
  the deferred native-completion table collapse, the unused CEO payment
  seeder; the PTY readiness item is narrowed to the paid runner.
- docs/test-audit-2026-09.md collects the triage, security mapping, inventories,
  selection proof, behavior-commit decisions and retained false positives.

* v1.91.8.0 test: smaller suite, derived paid selection, retired never-green evals

Release metadata for the test-reduction branch: VERSION 1.91.8.0 (1.91.7.0 is
claimed by #2983), CHANGELOG with the measured before/after table and a
contributor section, durations re-recorded on Ubicloud standard-16 (857 files,
0 failures), the agents digest, CONTRIBUTING's after-measurement row, the B8
fallback TODOS entry, and the after metrics, kept-vs-plan notes, B8 run and
census estimate in docs/test-audit-2026-09.md.

* fix(ubicloud): skip retrieval globs that match nothing instead of reporting a failed pull

* test: count issue-numbered Design findings in the UI-scope gate eval
This commit is contained in:
Garry Tan authored and GitHub committed 2026-09-29 08:58:49 -07:00
1 parent dcaea52800
commit 943105f109
668 files changed
+11998 -102381

No files matched your search

+1 -1
View File
@@ -2,7 +2,7 @@
"$schema": "https://gstack.dev/schemas/section-manifest.json",
"skill": "browse",
"version": 1,
"note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's prose is the ONLY place that decides WHEN to read a section; required-reads live in the E2E fixtures. No machine predicate here — see docs/designs/v2_PLAN.md:663.",
"note": "PASSIVE registry (v2 plan T9 / CM2). Fields are IDs, file paths, human titles, and human-readable trigger text ONLY. The skeleton's prose is the ONLY place that decides WHEN to read a section; required section reads are checked by test/carve-section-loading-browse.test.ts. No machine predicate here — see docs/designs/v2_PLAN.md:663.",
"sections": [
{
"id": "command-list",
-2
View File
@@ -12,8 +12,6 @@ import { validateNavigationUrl } from './url-validation';
import { checkScope, type TokenInfo } from './token-registry';
import { validateOutputPath, validateReadPath, SAFE_DIRECTORIES, escapeRegExp } from './path-security';
import { guardScreenshotBuffer, guardScreenshotPath } from './screenshot-size-guard';
// Re-export for backward compatibility (tests import from meta-commands)
export { validateOutputPath, escapeRegExp } from './path-security';
import * as Diff from 'diff';
import * as fs from 'fs';
import * as path from 'path';
-36
View File
@@ -192,42 +192,6 @@ describe('resolveDisconnectCause', () => {
});
});
// ─── onDisconnect exit-code propagation (regression test) ──────────
//
// The contract: BrowserManager.onDisconnect is called with the resolved
// exit code (0 for clean Cmd+Q, 2 for crash). server.ts then forwards
// that code to activeShutdown(), which exits the process.
//
// Without this propagation, the headed-mode user-visible Cmd+Q respawn
// bug returns: server.ts hardcoded `activeShutdown?.(2)` ignores the
// resolved 0 and gbrowser's gbd HealthMonitor treats the clean quit as
// a crash, restarting the window.
describe('BrowserManager.onDisconnect exit-code propagation', () => {
it('signature accepts an optional exitCode argument', async () => {
const { BrowserManager } = await import('../src/browser-manager');
const bm = new BrowserManager();
const calls: Array<number | undefined> = [];
bm.onDisconnect = (code?: number) => { calls.push(code); };
bm.onDisconnect(0);
bm.onDisconnect(2);
bm.onDisconnect(undefined);
expect(calls).toEqual([0, 2, undefined]);
});
it('server.ts callback forwards exitCode when provided, falls back to 2', async () => {
// Mirror the production wiring in browse/src/server.ts so a refactor
// that drops the forward (e.g. reverting to `() => activeShutdown?.(2)`)
// fails CI before the user-visible bug returns.
const shutdownCalls: number[] = [];
const activeShutdown = (code: number) => { shutdownCalls.push(code); };
const onDisconnect = (code?: number) => activeShutdown(code ?? 2);
onDisconnect(0);
onDisconnect(2);
onDisconnect(undefined);
expect(shutdownCalls).toEqual([0, 2, 2]);
});
});
// ─── Stealth injected on EVERY launch path (regression tripwire) ───
//
// applyStealth must run on launch() (headless), launchHeaded(), AND
+23
View File
@@ -101,6 +101,29 @@ describe('GET /health never carries a token (IRON RULE)', () => {
});
});
describe('GET /health is liveness-only', () => {
beforeEach(() => __resetRegistry());
// Folds the former server-auth / security-audit-r2 / sidebar-tabs /
// server-security-surface source greps into one check on the real body.
// #2557: no `security` field (its only data source had no writer).
const FORBIDDEN = ['token', 'security', 'currentUrl', 'currentMessage', 'agentStatus', 'messageQueue', 'agentStartTime', 'chatEnabled'];
for (const [label, browserManager, headers] of [
['default mode', () => new BrowserManager(), {}],
['headed mode + pinned extension Origin', headedBrowserManager, { Origin: PINNED_ORIGIN }],
] as const) {
test(`${label}: no token, security, browsing-state or chat fields; terminal port survives`, async () => {
const handle = buildFetchHandler(makeConfig({ browserManager: browserManager() }));
const resp = await handle.fetchLocal(new Request('http://127.0.0.1:34567/health', { headers }), null);
expect(resp.status).toBe(200);
const body = await resp.json() as Record<string, unknown>;
expect(FORBIDDEN.filter((key) => key in body)).toEqual([]);
expect('terminalPort' in body).toBe(true);
});
}
});
describe('POST /extension-token pinned-origin bootstrap', () => {
beforeEach(() => __resetRegistry());
-28
View File
@@ -158,34 +158,6 @@ describe('handleMemoryCommand', () => {
expect(result).toContain('Chromium processes: (unavailable — see notes)');
});
test('12. text mode renders modificationHistory with evicted-count when > 0', async () => {
// formatSnapshotText is what we're really testing here — exercise it
// directly with a known snapshot so the live collectStructureStats
// doesn't override the fixture values.
const mod = await import('../src/memory-command');
// formatSnapshotText is private; reach via re-rendering through
// --json mode then visually validating the JSON shape. The text-mode
// renderer is exercised by test 13 below with live (zero) values.
const stats = makeStructureStats();
stats.modificationHistory = { current: 200, cap: 200, evicted: 47 };
// Synthesize a "would-render" snapshot to assert the eviction note shape.
const renderedExpected =
'modificationHistory: 200 / 200 entries (47 evicted since reset)';
// Since formatSnapshotText isn't exported, validate the format
// contract by re-implementing the line and asserting our expectation
// matches the canonical format. This pins the user-visible string
// shape — a renderer change to drop the "evicted since reset" suffix
// would fail this assertion.
const evicted = stats.modificationHistory.evicted;
const current = stats.modificationHistory.current;
const cap = stats.modificationHistory.cap;
const expected =
`modificationHistory: ${current} / ${cap} entries` +
(evicted > 0 ? ` (${evicted} evicted since reset)` : '');
expect(expected).toBe(renderedExpected);
void mod;
});
test('13. text mode renders modificationHistory line shape', async () => {
const { handleMemoryCommand } = await import('../src/memory-command');
const result = await handleMemoryCommand([], makeFakeBm(makeSnapshot()));
+1 -1
View File
@@ -1,6 +1,6 @@
import { beforeAll, describe, it, expect } from 'bun:test';
import { chromium } from 'playwright';
import { validateOutputPath } from '../src/meta-commands';
import { validateOutputPath } from '../src/path-security';
import { validateReadPath, SENSITIVE_COOKIE_NAME, SENSITIVE_COOKIE_VALUE } from '../src/read-commands';
import { BLOCKED_METADATA_HOSTS } from '../src/url-validation';
import { mkdirSync, mkdtempSync, rmSync, symlinkSync, unlinkSync, writeFileSync, realpathSync } from 'fs';
+72 -1
View File
@@ -11,7 +11,8 @@
*/
import { describe, test, expect } from 'bun:test';
import { readFileSync } from 'fs';
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'fs';
import { tmpdir } from 'os';
import { join } from 'path';
const SERVER_SRC = readFileSync(
@@ -74,3 +75,73 @@ describe('/pty-inject-scan — server.ts static invariants', () => {
expect(SERVER_SRC).not.toContain("from './security-classifier'");
});
});
// Behavioral: the real buildFetchHandler consumes the L4 sidecar verdict.
// The sidecar client is replaced with mock.module inside a child `bun test`
// process, so the module mock cannot leak into other files of a shard.
describe('/pty-inject-scan — L4 sidecar verdict drives the response', () => {
test('unsafe → BLOCK, suspicious → WARN, unavailable → WARN (D7), blocklisted URL skips L4', async () => {
const dir = mkdtempSync(join(tmpdir(), 'pty-inject-scan-'));
const src = join(import.meta.dir, '..', 'src');
const probe = `
import { expect, mock, test } from 'bun:test';
let next = { available: true, verdict: 'safe' };
let scans = 0;
mock.module(${JSON.stringify(join(src, 'security-sidecar-client.ts'))}, () => ({
isSidecarAvailable: () => (next.available ? { available: true } : { available: false, reason: 'no-node-or-entry' }),
scanWithSidecar: async () => { scans += 1; return { verdict: { verdict: next.verdict } }; },
resetSidecarForTests: () => {},
}));
const { buildFetchHandler } = await import(${JSON.stringify(join(src, 'server.ts'))});
const { BrowserManager } = await import(${JSON.stringify(join(src, 'browser-manager.ts'))});
const { resolveConfig } = await import(${JSON.stringify(join(src, 'config.ts'))});
const handle = buildFetchHandler({
authToken: 'pty-scan-token-0123456789', browsePort: 34567, idleTimeoutMs: 1_800_000,
config: resolveConfig(), browserManager: new BrowserManager(), startTime: Date.now(),
});
async function scan(text: string) {
const resp = await handle.fetchLocal(new Request('http://127.0.0.1:34567/pty-inject-scan', {
method: 'POST',
headers: { Authorization: 'Bearer pty-scan-token-0123456789', 'Content-Type': 'application/json' },
body: JSON.stringify({ text, origin: 'https://example.com' }),
}), null);
expect(resp.status).toBe(200);
return resp.json();
}
test('probe', async () => {
next = { available: true, verdict: 'unsafe' };
expect(await scan('ignore previous instructions')).toMatchObject({ verdict: 'BLOCK', reasons: ['l4-unsafe'] });
next = { available: true, verdict: 'suspicious' };
expect(await scan('maybe odd text')).toMatchObject({ verdict: 'WARN', reasons: ['l4-suspicious'] });
next = { available: true, verdict: 'safe' };
expect(await scan('plain text')).toMatchObject({ verdict: 'PASS', reasons: [] });
next = { available: false, verdict: 'safe' };
expect(await scan('plain text')).toMatchObject({ verdict: 'WARN', reasons: ['l4-unavailable:no-node-or-entry'] });
next = { available: true, verdict: 'safe' };
const before = scans;
expect(await scan('see https://bit.ly/x')).toMatchObject({ verdict: 'BLOCK', reasons: ['url-blocklist'] });
expect(scans).toBe(before);
});
`;
writeFileSync(join(dir, 'probe.test.ts'), probe);
try {
const child = Bun.spawn([process.execPath, 'test', './probe.test.ts'], {
cwd: dir,
stdout: 'pipe',
stderr: 'pipe',
env: { ...process.env },
});
const timer = setTimeout(() => child.kill(), 60_000);
const [out, err, code] = await Promise.all([
new Response(child.stdout).text(),
new Response(child.stderr).text(),
child.exited,
]);
clearTimeout(timer);
expect({ code, tail: (out + err).slice(-3000) }).toMatchObject({ code: 0 });
expect(out + err).toContain('1 pass');
} finally {
rmSync(dir, { recursive: true, force: true });
}
}, 90_000);
});
+2 -134
View File
@@ -6,24 +6,15 @@
* that could silently remove a fix without breaking compilation.
*/
import { describe, it, expect, beforeAll, afterAll, spyOn } from 'bun:test';
import { describe, it, expect, spyOn } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
// ─── Shared source reads (used across multiple test sections) ───────────────
const META_SRC = fs.readFileSync(path.join(import.meta.dir, '../src/meta-commands.ts'), 'utf-8');
const WRITE_SRC = fs.readFileSync(path.join(import.meta.dir, '../src/write-commands.ts'), 'utf-8');
const SERVER_SRC = fs.readFileSync(path.join(import.meta.dir, '../src/server.ts'), 'utf-8');
// sidebar-agent.ts was ripped (chat queue replaced by interactive PTY).
// AGENT_SRC kept as empty string so the legacy describe block below skips
// without crashing module load on a missing file.
const AGENT_SRC = (() => {
try { return fs.readFileSync(path.join(import.meta.dir, '../src/sidebar-agent.ts'), 'utf-8'); }
catch { return ''; }
})();
const SNAPSHOT_SRC = fs.readFileSync(path.join(import.meta.dir, '../src/snapshot.ts'), 'utf-8');
const PATH_SECURITY_SRC = fs.readFileSync(path.join(import.meta.dir, '../src/path-security.ts'), 'utf-8');
// ─── Helper ─────────────────────────────────────────────────────────────────
@@ -121,104 +112,6 @@ describe('Task 2: CSS value validator blocks dangerous patterns', () => {
});
});
// ─── Task 1: Harden validateOutputPath to use realpathSync ──────────────────
describe('Task 1: validateOutputPath uses realpathSync', () => {
describe('source-level checks', () => {
it('path-security.ts validateOutputPath contains realpathSync', () => {
const fn = extractFunction(PATH_SECURITY_SRC, 'validateOutputPath');
expect(fn).toBeTruthy();
expect(fn).toContain('realpathSync');
});
it('path-security.ts SAFE_DIRECTORIES resolves with realpathSync', () => {
const safeBlock = sliceBetween(PATH_SECURITY_SRC, 'const SAFE_DIRECTORIES', ';');
expect(safeBlock).toContain('realpathSync');
});
it('meta-commands.ts re-exports validateOutputPath from path-security', () => {
expect(META_SRC).toContain("from './path-security'");
expect(META_SRC).toContain('validateOutputPath');
});
it('write-commands.ts imports validateOutputPath from path-security', () => {
expect(WRITE_SRC).toContain("from './path-security'");
expect(WRITE_SRC).toContain('validateOutputPath');
});
});
describe('behavioral checks', () => {
let tmpDir: string;
let symlinkPath: string;
beforeAll(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-sec-test-'));
symlinkPath = path.join(tmpDir, 'evil-link');
try {
fs.symlinkSync('/etc', symlinkPath);
} catch {
symlinkPath = '';
}
});
afterAll(() => {
try {
if (symlinkPath) fs.unlinkSync(symlinkPath);
fs.rmdirSync(tmpDir);
} catch {
// best-effort cleanup
}
});
it('meta-commands validateOutputPath rejects path through /etc symlink', async () => {
if (!symlinkPath) {
console.warn('Skipping: symlink creation failed');
return;
}
const mod = await import('../src/meta-commands.ts');
const attackPath = path.join(symlinkPath, 'passwd');
expect(() => mod.validateOutputPath(attackPath)).toThrow();
});
it('realpathSync on symlink-to-/etc resolves to /etc (out of safe dirs)', () => {
if (!symlinkPath) {
console.warn('Skipping: symlink creation failed');
return;
}
const resolvedLink = fs.realpathSync(symlinkPath);
// macOS: /etc -> /private/etc
expect(resolvedLink).toBe(fs.realpathSync('/etc'));
const TEMP_DIR_VAL = process.platform === 'win32' ? os.tmpdir() : '/tmp';
const safeDirs = [TEMP_DIR_VAL, process.cwd()].map(d => {
try { return fs.realpathSync(d); } catch { return d; }
});
const passwdReal = path.join(resolvedLink, 'passwd');
const isSafe = safeDirs.some(d => passwdReal === d || passwdReal.startsWith(d + path.sep));
expect(isSafe).toBe(false);
});
it('meta-commands validateOutputPath accepts legitimate tmpdir paths', async () => {
const mod = await import('../src/meta-commands.ts');
// Use /tmp (which resolves to /private/tmp on macOS) — matches SAFE_DIRECTORIES
const tmpBase = process.platform === 'darwin' ? '/tmp' : os.tmpdir();
const legitimatePath = path.join(tmpBase, 'gstack-screenshot.png');
expect(() => mod.validateOutputPath(legitimatePath)).not.toThrow();
});
it('meta-commands validateOutputPath accepts paths in cwd', async () => {
const mod = await import('../src/meta-commands.ts');
const cwdPath = path.join(process.cwd(), 'output.png');
expect(() => mod.validateOutputPath(cwdPath)).not.toThrow();
});
it('meta-commands validateOutputPath rejects paths outside safe dirs', async () => {
const mod = await import('../src/meta-commands.ts');
expect(() => mod.validateOutputPath('/home/user/secret.png')).toThrow(/Path must be within/);
expect(() => mod.validateOutputPath('/var/log/access.log')).toThrow(/Path must be within/);
});
});
});
// ─── Round-2 review findings: applyStyle CSS check ──────────────────────────
describe('Round-2 finding 1: extension applyStyle blocks dangerous CSS values', () => {
@@ -298,19 +191,6 @@ describe('Round-2 finding 2: snapshot.ts annotated path uses realpathSync', () =
// traversal in browse-server's tab-state writer is covered by
// browse/test/terminal-agent.test.ts (handleTabState atomic-write tests).
// ─── Task 5: /health endpoint must not expose sensitive fields ───────────────
describe('/health endpoint security', () => {
it('must not expose currentMessage', () => {
const block = sliceBetween(SERVER_SRC, "url.pathname === '/health'", "url.pathname === '/refs'");
expect(block).not.toContain('currentMessage');
});
it('must not expose currentUrl', () => {
const block = sliceBetween(SERVER_SRC, "url.pathname === '/health'", "url.pathname === '/refs'");
expect(block).not.toContain('currentUrl');
});
});
// ─── Task 6: frame --url ReDoS fix ──────────────────────────────────────────
describe('frame --url ReDoS fix', () => {
@@ -325,9 +205,7 @@ describe('frame --url ReDoS fix', () => {
});
it('escapeRegExp neutralizes catastrophic patterns (behavioral)', async () => {
const mod = await import('../src/meta-commands.ts');
const { escapeRegExp } = mod as any;
expect(typeof escapeRegExp).toBe('function');
const { escapeRegExp } = await import('../src/path-security.ts');
const evil = '(a+)+$';
const escaped = escapeRegExp(evil);
const start = Date.now();
@@ -429,10 +307,6 @@ describe('Task 10: responsive screenshot path validation', () => {
expect(validateIdx).toBeLessThan(screenshotIdx);
});
it('results.push is present in the loop block (loop structure intact)', () => {
const block = sliceBetween(META_SRC, 'for (const vp of viewports)', 'Restore original viewport');
expect(block).toContain('results.push');
});
});
// ─── Task 11: State load — cookie + page URL validation ──────────────────────
@@ -538,12 +412,6 @@ describe('Task 17: viewport dimensions and wait timeouts are clamped', () => {
expect(block).toMatch(/Math\.min|Math\.max/);
});
it('viewport case uses rawW/rawH before clamping (not direct destructure)', () => {
const block = sliceBetween(WRITE_SRC, "case 'viewport':", "case 'cookie':");
expect(block).toContain('rawW');
expect(block).toContain('rawH');
});
it('wait case (networkidle branch) clamps timeout with MAX_WAIT_MS', () => {
const block = sliceBetween(WRITE_SRC, "case 'wait':", "case 'viewport':");
expect(block).toBeTruthy();
+3 -2
View File
@@ -243,8 +243,9 @@ describe('canary', () => {
// /health reported a false-green 'protected' indefinitely. The surfaces they
// covered (SessionState, read/writeSessionState, getStatus, the /health
// security field, the sidepanel SEC shield) were dead since the PTY terminal
// rewrite and are now removed. server-security-surface.test.ts pins the
// removal + the live L4 wiring.
// rewrite and are now removed. extension-token.test.ts ("GET /health is
// liveness-only") pins the removal on the real /health body;
// pty-inject-scan.test.ts pins the live L4 sidecar wiring behaviorally.
// ─── URL domain extraction ───────────────────────────────────
-18
View File
@@ -22,17 +22,6 @@ function sliceBetween(source: string, startMarker: string, endMarker: string): s
}
describe('Server auth security', () => {
// Test 1 (IRON RULE, inverted in v1.62): /health NEVER serves a token in
// ANY mode. Both carve-outs (headed-mode disjunct + chrome-extension://
// Origin disjunct) are gone. Token bootstrap moved to POST /extension-token
// with a pinned extension Origin.
test('/health never serves a token — no headed-mode or chrome-extension carve-out', () => {
const healthBlock = sliceBetween(SERVER_SRC, "url.pathname === '/health'", "url.pathname === '/connect'");
expect(healthBlock).not.toContain('token: authToken');
expect(healthBlock).not.toContain("getConnectionMode() === 'headed'");
expect(healthBlock).not.toContain("startsWith('chrome-extension://')");
});
// Test 1a: the pinned-origin bootstrap endpoint exists and gates on both
// the exact extension Origin and a loopback Host.
test('POST /extension-token gates on pinned Origin and loopback Host', () => {
@@ -47,13 +36,6 @@ describe('Server auth security', () => {
expect(tokenBlock).toContain('403');
});
// Test 1b: /health does not expose sensitive browsing state
test('/health does not expose currentUrl or currentMessage', () => {
const healthBlock = sliceBetween(SERVER_SRC, "url.pathname === '/health'", "url.pathname === '/connect'");
expect(healthBlock).not.toContain('currentUrl');
expect(healthBlock).not.toContain('currentMessage');
});
// Test 1c: newtab must check domain restrictions (CSO finding #5)
// Domain check for newtab is now unified with goto in the scope check section:
// (command === 'goto' || command === 'newtab') && args[0] → checkDomain
@@ -1,86 +0,0 @@
/**
* #2557 / ENG-OV9: pins the dead-shield removal AND the live L4 wiring.
*
* The removed surface: /health's `security` field read getStatus(), whose
* only data source (~/.gstack/security/session-state.json) lost its only
* writer when sidebar-agent.ts was ripped — so /health reported a permanent
* 'inactive' or, wherever an old state file survived, a stale FALSE-GREEN
* 'protected' ("no threats detected" when the real state was "not
* measured"). Same fail-open class as #2026.
*
* The kept surface (ENG-OV9): security.ts is NOT dead — server.ts's
* /pty-inject-scan path is the live L4 consumer (sidecar scan + URL
* blocklist + datamark envelope), and security.ts's pure combiner/canary
* exports stay. This test pins both directions so a future "cleanup" can't
* silently take the live half, and a future re-feed of /health.security
* from LIVE signals (isSidecarAvailable, content filters) must update this
* test deliberately rather than resurrect the state-file path.
*
* Source-level, same style as windows-spawn-hide.test.ts.
*/
import { describe, expect, test } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
const SRC = (f: string) => fs.readFileSync(path.join(import.meta.dir, '../src', f), 'utf-8');
describe('#2557: dead shield surface stays dead', () => {
test('/health carries no security field and server.ts does not import getStatus', () => {
const server = SRC('server.ts');
expect(server).not.toMatch(/security:\s*getSecurityStatus\(\)/);
expect(server).not.toMatch(/getStatus as getSecurityStatus/);
// The SECURITY session-state file must not be read anywhere in src/ —
// that file has no writer, so any reader is a false-signal feed.
// (session-persist.ts's per-project <stateDir>/session-state.json is a
// different, live file — only the ~/.gstack/security/ one is dead.)
for (const f of fs.readdirSync(path.join(import.meta.dir, '../src')).filter((x) => x.endsWith('.ts'))) {
const code = SRC(f).replace(/\/\*[\s\S]*?\*\//g, '').replace(/^\s*\/\/.*$/gm, '').replace(/^\s*\*.*$/gm, '');
const refs = /security[/'",\s][^\n]{0,80}session-state\.json/.test(code);
expect({ file: f, refs }).toEqual({ file: f, refs: false });
}
});
test('security.ts no longer exports the unfed status surface', () => {
const security = SRC('security.ts');
expect(security).not.toMatch(/export function getStatus/);
expect(security).not.toMatch(/export function (read|write)SessionState/);
expect(security).not.toMatch(/export interface SessionState/);
expect(security).not.toMatch(/export interface StatusDetail/);
});
test('the sidepanel shield markup is gone', () => {
const html = fs.readFileSync(path.join(import.meta.dir, '../../extension/sidepanel.html'), 'utf-8');
const css = fs.readFileSync(path.join(import.meta.dir, '../../extension/sidepanel.css'), 'utf-8');
expect(html).not.toContain('security-shield');
expect(css).not.toMatch(/\.security-shield\s*\{/);
});
});
describe('ENG-OV9: the LIVE L4 path is untouched', () => {
test('server.ts still consumes the sidecar on the inject-scan path', () => {
const server = SRC('server.ts');
expect(server).toContain("from './security-sidecar-client'");
expect(server).toMatch(/isSidecarAvailable/);
expect(server).toMatch(/scanWithSidecar\(/);
});
test('security.ts keeps the pure combiner + canary exports', () => {
const security = SRC('security.ts');
expect(security).toMatch(/export const THRESHOLDS/);
expect(security).toMatch(/export function combineVerdict/);
expect(security).toMatch(/export function generateCanary/);
expect(security).toMatch(/export function injectCanary/);
expect(security).toMatch(/export function checkCanaryInStructure/);
expect(security).toMatch(/export function extractDomain/);
});
test('/health stays liveness-only: no token in any mode (regression wall from v1.63)', () => {
const server = SRC('server.ts');
// The /health handler block must not interpolate a token.
const healthIdx = server.indexOf("url.pathname === '/health'");
expect(healthIdx).toBeGreaterThan(0);
const healthBlock = server.slice(healthIdx, healthIdx + 1500);
expect(healthBlock).not.toMatch(/token:\s*[^n]/i);
});
});
-24
View File
@@ -198,19 +198,6 @@ describe('server.ts: chat / sidebar-agent endpoints are gone', () => {
expect(SERVER_SRC).not.toMatch(/^interface ChatEntry/m);
expect(SERVER_SRC).not.toMatch(/^interface SidebarSession/m);
});
test('/health no longer surfaces agentStatus or messageQueue length', () => {
const health = SERVER_SRC.slice(SERVER_SRC.indexOf("url.pathname === '/health'"));
const slice = health.slice(0, 2000);
expect(slice).not.toContain('agentStatus');
expect(slice).not.toContain('messageQueue');
expect(slice).not.toContain('agentStartTime');
// chatEnabled is gone entirely — the chat pane no longer exists in any
// extension build, so /health stopped advertising a chat mode.
expect(slice).not.toContain('chatEnabled');
// terminalPort survives.
expect(slice).toContain('terminalPort');
});
});
describe('cli.ts: sidebar-agent is no longer spawned', () => {
@@ -240,17 +227,6 @@ describe('cli.ts: sidebar-agent is no longer spawned', () => {
});
});
describe('files: sidebar-agent.ts and its tests are deleted', () => {
test('browse/src/sidebar-agent.ts is gone', () => {
expect(fs.existsSync(path.join(import.meta.dir, '../src/sidebar-agent.ts'))).toBe(false);
});
test('sidebar-agent test files are gone', () => {
expect(fs.existsSync(path.join(import.meta.dir, 'sidebar-agent.test.ts'))).toBe(false);
expect(fs.existsSync(path.join(import.meta.dir, 'sidebar-agent-roundtrip.test.ts'))).toBe(false);
});
});
describe('manifest: ws permission + xterm-safe CSP', () => {
test('host_permissions covers ws localhost', () => {
expect(MANIFEST.host_permissions).toContain('ws://127.0.0.1:*/');
-71
View File
@@ -182,43 +182,6 @@ describe('browser tab bar (sidepanel.css)', () => {
});
});
// ─── Sidebar CSS tests ──────────────────────────────────────────
describe('sidebar CSS (sidepanel.css)', () => {
const css = fs.readFileSync(path.join(ROOT, '..', 'extension', 'sidepanel.css'), 'utf-8');
test('stop button style exists', () => {
expect(css).toContain('.stop-btn');
});
test('stop button uses error color', () => {
const stopBtnSection = css.slice(
css.indexOf('.stop-btn {'),
css.indexOf('}', css.indexOf('.stop-btn {')) + 1,
);
expect(stopBtnSection).toContain('--error');
});
test('experimental-banner no longer uses amber warning colors', () => {
const bannerSection = css.slice(
css.indexOf('.experimental-banner {'),
css.indexOf('}', css.indexOf('.experimental-banner {')) + 1,
);
// Should not be amber/warning anymore
expect(bannerSection).not.toContain('245, 158, 11, 0.15');
expect(bannerSection).not.toContain('#F59E0B');
});
test('tool description uses system font not mono', () => {
const toolSection = css.slice(
css.indexOf('.agent-tool {'),
css.indexOf('}', css.indexOf('.agent-tool {')) + 1,
);
expect(toolSection).toContain('font-system');
expect(toolSection).not.toContain('font-mono');
});
});
// ─── Inspector message allowlist fix ────────────────────────────
describe('inspector message allowlist fix', () => {
@@ -491,11 +454,6 @@ describe('tab switching does not steal focus', () => {
const serverSrc = fs.readFileSync(path.join(ROOT, 'src', 'server.ts'), 'utf-8');
const bmSrc = fs.readFileSync(path.join(ROOT, 'src', 'browser-manager.ts'), 'utf-8');
test('switchTab has bringToFront option', () => {
expect(bmSrc).toContain('bringToFront?: boolean');
expect(bmSrc).toContain('bringToFront !== false');
});
test('handleCommand tab pinning does NOT steal focus', () => {
// All switchTab calls in handleCommand should use bringToFront: false
const handleFn = serverSrc.slice(
@@ -1004,41 +962,12 @@ describe('BROWSE_NO_AUTOSTART (sidebar headless prevention)', () => {
// chat-queue rip (PR #1216) — /command and /batch reset the timer and are
// covered by that factory suite.
// ─── Shutdown kills the terminal-agent (server.ts) ──────────────
describe('shutdown cleanup (server.ts)', () => {
const serverSrc = fs.readFileSync(path.join(ROOT, 'src', 'server.ts'), 'utf-8');
test('shutdown kills the terminal-agent via identity-based kill (no pkill)', () => {
// v1.44+ identity-based teardown: only the PID recorded by THIS
// daemon's agent is signaled. The pre-v1.44 `pkill -f terminal-agent`
// regex killed sibling gstack sessions on the same host (also pinned
// by browse/test/terminal-agent-pid-identity.test.ts).
const shutdownFn = serverSrc.slice(
serverSrc.indexOf('async function shutdown('),
serverSrc.indexOf('try { detachSession()', serverSrc.indexOf('async function shutdown(')),
);
expect(shutdownFn).toContain('stopAgentByRecord');
expect(shutdownFn).toContain('isOurAgent(record, process.pid)');
expect(shutdownFn).toContain('readAgentRecord');
// No pkill CALL — the word may appear in the explanatory comment, so
// match invocation shapes only. The repo-wide reintroduction tripwire
// is browse/test/terminal-agent-pid-identity.test.ts.
expect(shutdownFn).not.toMatch(/(?:spawnSync|execSync|\$)\(\s*['"`]pkill/);
});
});
// ─── Cookie button in sidebar footer ────────────────────────────
describe('cookie import button (sidebar)', () => {
const html = fs.readFileSync(path.join(ROOT, '..', 'extension', 'sidepanel.html'), 'utf-8');
const js = fs.readFileSync(path.join(ROOT, '..', 'extension', 'sidepanel.js'), 'utf-8');
test('quick actions toolbar has cookies button', () => {
expect(html).toContain('id="chat-cookies-btn"');
expect(html).toContain('Cookies');
});
test('cookies button navigates to cookie-picker', () => {
expect(js).toContain("'chat-cookies-btn'");
expect(js).toContain('cookie-picker');
@@ -13,19 +13,6 @@ import * as path from 'path';
const AGENT_TS = path.resolve(import.meta.path, '..', '..', 'src', 'terminal-agent.ts');
describe('terminal-agent detach + re-attach (v1.44+ Commit 3)', () => {
test('1. PtySession carries ring buffer + alt-screen + detach state', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
const i = src.indexOf('interface PtySession {');
const j = src.indexOf('\n}', i);
const block = src.slice(i, j);
expect(block).toContain('liveWs: any | null');
expect(block).toContain('ringBuffer: Buffer[]');
expect(block).toContain('ringBufferBytes: number');
expect(block).toContain('altScreenActive: boolean');
expect(block).toContain('detached: boolean');
expect(block).toContain('detachTimer:');
});
test('2. RING_BUFFER_MAX_BYTES default is 1 MB, env-overridable', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
expect(src).toContain('GSTACK_PTY_RING_BUFFER_BYTES');
@@ -38,36 +25,6 @@ describe('terminal-agent detach + re-attach (v1.44+ Commit 3)', () => {
expect(src).toContain("'60000'");
});
test('4. appendToRingBuffer evicts oldest frames past the cap', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
expect(src).toMatch(/function appendToRingBuffer\(/);
// Eviction loop: must keep at least one frame even at extreme caps
// (otherwise a single oversized frame would empty the buffer).
expect(src).toMatch(/session\.ringBufferBytes > RING_BUFFER_MAX_BYTES/);
expect(src).toContain('session.ringBuffer.length > 1');
expect(src).toContain('session.ringBuffer.shift()');
});
test('5. alt-screen tracking watches for CSI ?1049h / CSI ?1049l', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
// Canonical xterm enter/exit alt-screen sequences. Must update
// session.altScreenActive so the replay prelude knows.
expect(src).toContain('\\x1b[?1049h');
expect(src).toContain('\\x1b[?1049l');
expect(src).toContain('session.altScreenActive');
});
test('6. buildReplayPayload prefixes soft-reset (+ alt-screen if active)', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
expect(src).toMatch(/function buildReplayPayload\(/);
// DECSTR soft reset — re-defaults character attributes after the
// client's RIS clears the xterm buffer.
expect(src).toContain('\\x1b[!p');
// Conditionally re-enter alt-screen if claude was in a tool-call
// (alt-screen mode) at detach.
expect(src).toContain('session.altScreenActive');
});
test('7. WS open() re-attaches when sessionId already lives in sessionsById', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
const block = sliceBetween(src, 'open(ws) {', 'message(ws, raw) {');
@@ -115,6 +115,50 @@ describe('terminal-agent: /internal/grant', () => {
});
});
describe('terminal-agent: /internal/grant and /internal/revoke bearer auth', () => {
function post(route: 'grant' | 'revoke', token: string, authorization?: string): Promise<Response> {
const headers: Record<string, string> = { 'Content-Type': 'application/json' };
if (authorization !== undefined) headers.Authorization = authorization;
return fetch(`http://127.0.0.1:${agentPort}/internal/${route}`, {
method: 'POST',
headers,
body: JSON.stringify({ token }),
});
}
function wsStatus(token: string): Promise<number> {
return fetch(`http://127.0.0.1:${agentPort}/ws`, {
headers: { 'Origin': 'chrome-extension://abc123', 'Cookie': `gstack_pty=${token}` },
}).then((r) => r.status);
}
for (const route of ['grant', 'revoke'] as const) {
test(`${route}: no token → 403, wrong token → 403, valid internal token → 200`, async () => {
const target = `auth-matrix-${route}-token-long-enough`;
expect((await post(route, target)).status).toBe(403);
expect((await post(route, target, 'Bearer wrong-token')).status).toBe(403);
expect((await post(route, target, `Bearer ${internalToken}`)).status).toBe(200);
});
}
test('an unauthenticated revoke leaves the grant usable; an authenticated revoke removes it', async () => {
const token = 'revoke-auth-token-at-least-seventeen';
expect((await grantToken(token)).status).toBe(200);
expect(await wsStatus(token)).not.toBe(401);
expect((await post('revoke', token)).status).toBe(403);
expect((await post('revoke', token, 'Bearer wrong-token')).status).toBe(403);
expect(await wsStatus(token)).not.toBe(401);
expect((await post('revoke', token, `Bearer ${internalToken}`)).status).toBe(200);
expect(await wsStatus(token)).toBe(401);
});
test('an unauthenticated grant does not register the token', async () => {
const token = 'forged-grant-token-at-least-seventeen';
expect((await post('grant', token, 'Bearer wrong-token')).status).toBe(403);
expect(await wsStatus(token)).toBe(401);
});
});
describe('terminal-agent: /ws gates', () => {
test('rejects upgrade attempts without an extension Origin', async () => {
const resp = await fetch(`http://127.0.0.1:${agentPort}/ws`);
@@ -1,51 +0,0 @@
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
// Static-grep tripwire for the v1.44 internalHandler refactor.
//
// /internal/grant and /internal/revoke were copies of the same dance:
// bearer-auth → x-browse-gen check → req.json().then(...).catch(...).
// internalHandler<T>(req, fn) collapses that into a single helper call.
// This test fails CI if the helper goes away or the existing routes
// regress to inline auth + JSON parse boilerplate. Wiring tests
// (token grant/revoke behavior) already live in
// browse/test/terminal-agent-integration.test.ts.
const AGENT_TS = path.resolve(import.meta.path, '..', '..', 'src', 'terminal-agent.ts');
describe('terminal-agent internalHandler refactor (v1.44+)', () => {
test('1. internalHandler<T> exists with the documented signature', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
expect(src).toMatch(/async function internalHandler<T>\s*\(/);
// Body must include the auth gate, body parse, and result coercion.
expect(src).toContain('checkInternalAuth(req)');
expect(src).toContain('await req.json()');
expect(src).toContain('instanceof Response');
});
test('2. /internal/grant routes through internalHandler', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
// Match the route handler block.
const block = sliceBetween(src, "url.pathname === '/internal/grant'", "url.pathname === '/internal/revoke'");
expect(block).toContain('internalHandler(req');
// Must NOT have the old inline pattern (would be a regression).
expect(block).not.toContain('req.headers.get(\'authorization\')');
expect(block).not.toContain('req.json().then(');
});
test('3. /internal/revoke routes through internalHandler', () => {
const src = fs.readFileSync(AGENT_TS, 'utf-8');
const block = sliceBetween(src, "url.pathname === '/internal/revoke'", "url.pathname === '/internal/healthz'");
expect(block).toContain('internalHandler(req');
expect(block).not.toContain('req.json().then(');
});
});
function sliceBetween(source: string, start: string, end: string): string {
const i = source.indexOf(start);
if (i === -1) throw new Error(`marker not found: ${start}`);
const j = source.indexOf(end, i + start.length);
if (j === -1) throw new Error(`end marker not found: ${end}`);
return source.slice(i, j);
}