Files
gstack/browse/test/browser-skills-e2e.test.ts
T
be66d4f06f fix(browse): capture browser-skill subprocess output via temp files, not pipes (core of #2559)
Under a loaded parent, the FIRST piped Bun.spawn in a process
intermittently yields an empty stderr even though the child wrote it and
exited 0 — measured identically with readers-attached-before-exit and with
a manual getReader() drain, so it's loss inside the async pipe plumbing,
not read ordering. It flaked `$B skill test` (bun test writes its banner to
stdout and the pass/fail summary to stderr, so a dropped stderr silently
degraded the result to just the banner) and would blank a skill's JSON
result on `$B skill run` while still reporting success.

New runToFiles() points the child's stdout/stderr at temp files via
Bun.file() (never raw fds — closing self-opened fds around a spawn tripped
Bun's fd bookkeeping into a stray epoll_ctl EBADF), awaits exit, then reads
the files: the kernel has flushed everything by child exit, so the
post-exit read is complete, and chatty children can't stall on a full pipe
buffer. Both handleTest and spawnSkill route through it (timeout + capped
read preserved via timeoutMs/maxStdoutBytes). Bun.spawnSync would also
capture reliably but would deadlock: a spawned skill calls back into this
same daemon on GSTACK_PORT.

The `tests passed for "<name>"` fallback is gone — a passing bun test
always prints a summary, so exit 0 with no output means the run was NOT
captured, and handleTest now throws instead of fabricating success. The
E2E assertion checks both stream halves (banner + summary + "Ran N tests")
instead of the loose alternation whose `tests passed` branch matched the
synthetic fallback vacuously. A static tripwire pins the structure:
runToFiles owns the module's ONLY Bun.spawn, and no site reads child
output via stdout:'pipe' / new Response(proc.stdout) / getReader().

Scope: the PR's repo-wide test-file sweep is deliberately not absorbed —
this is the core only, per the wave plan.

Tests: browser-skill-commands + browser-skills-e2e + browser-skill-write
74 pass, 0 fail.

Re-derived from PR #2559 by @frederik-kaster-noygear.

Co-authored-by: Frederik Kaster <frederik.kaster@noygear.ai>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 10:59:08 -07:00

102 lines
4.7 KiB
TypeScript

/**
* browser-skills E2E — exercise the full dispatch path against the bundled
* `hackernews-frontpage` reference skill. Verifies:
*
* - $B skill list resolves the bundled tier and surfaces hackernews-frontpage
* - $B skill show returns the SKILL.md
* - $B skill test runs script.test.ts (which itself runs against the bundled
* fixture) and reports pass
*
* Coverage gap intentionally NOT here: $B skill run end-to-end against the
* bundled skill goes to live news.ycombinator.com and would be flaky. The
* spawnSkill lifecycle (env scrub, scoped token, timeout, stdout cap) is
* already covered by browse/test/browser-skill-commands.test.ts using inline
* scripts.
*/
import { describe, test, expect, beforeAll } from 'bun:test';
import { handleSkillCommand } from '../src/browser-skill-commands';
import { listBrowserSkills, defaultTierPaths } from '../src/browser-skills';
import { initRegistry, __resetRegistry } from '../src/token-registry';
beforeAll(() => {
// __resetRegistry zeroes rootToken so the new initRegistry mismatch guard
// doesn't fire. rotateRoot would leave a UUID in rootToken and the next
// initRegistry call would throw.
__resetRegistry();
initRegistry('e2e-root-token');
});
describe('browser-skills E2E — bundled hackernews-frontpage', () => {
test('defaultTierPaths resolves bundled tier to <repo>/browser-skills/', () => {
const tiers = defaultTierPaths();
expect(tiers.bundled).toMatch(/\/browser-skills$/);
// Bundled tier should exist on disk (the reference skill is shipped).
expect(require('fs').existsSync(tiers.bundled)).toBe(true);
});
test('listBrowserSkills() returns hackernews-frontpage at bundled tier', () => {
const skills = listBrowserSkills();
const hn = skills.find(s => s.name === 'hackernews-frontpage');
expect(hn).toBeTruthy();
expect(hn!.tier).toBe('bundled');
expect(hn!.frontmatter.host).toBe('news.ycombinator.com');
expect(hn!.frontmatter.trusted).toBe(true);
expect(hn!.frontmatter.triggers).toContain('scrape hn frontpage');
});
test('$B skill list dispatches and includes hackernews-frontpage', async () => {
const result = await handleSkillCommand(['list'], { port: 0 });
expect(result).toContain('hackernews-frontpage');
expect(result).toContain('bundled');
expect(result).toContain('news.ycombinator.com');
});
test('$B skill show hackernews-frontpage prints the SKILL.md', async () => {
const result = await handleSkillCommand(['show', 'hackernews-frontpage'], { port: 0 });
expect(result).toContain('host: news.ycombinator.com');
expect(result).toContain('trusted: true');
expect(result).toContain('Hacker News front-page scraper');
expect(result).toContain('triggers:');
});
test('$B skill show <missing> errors clearly', async () => {
await expect(handleSkillCommand(['show', 'nonexistent-skill-xyz'], { port: 0 }))
.rejects.toThrow(/not found in any tier/);
});
test('$B skill help prints usage', async () => {
const result = await handleSkillCommand([], { port: 0 });
expect(result).toContain('Usage');
expect(result).toContain('list');
expect(result).toContain('show');
expect(result).toContain('run');
});
test('$B skill rm cannot tombstone bundled tier (read-only)', async () => {
// The bundled hackernews-frontpage skill is shipped read-only; rm targets
// user tiers (project default, --global). Attempting rm on a name that
// only exists in bundled should error with "not found".
await expect(handleSkillCommand(['rm', 'hackernews-frontpage', '--global'], { port: 0 }))
.rejects.toThrow(/not found/);
});
// The `test` subcommand spawns `bun test script.test.ts` in the skill dir.
// It takes ~1s. Run it last so other assertions are quick.
test('$B skill test hackernews-frontpage runs script.test.ts and reports pass', async () => {
const result = await handleSkillCommand(['test', 'hackernews-frontpage'], { port: 0 });
// `bun test` splits its report across streams: the version banner goes to
// stdout, the pass/fail summary to stderr. handleSkillCommand must return
// both, so assert on each stream's half.
//
// This used to flake under full-suite load: capturing the child through
// pipes dropped stderr on the first piped spawn in the process, so the
// result was just the banner. The old `13 pass|0 fail|tests passed` regex
// also had a `tests passed` alternative that matched a synthetic fallback
// string, which would have passed vacuously on an empty capture.
expect(result).toMatch(/bun test v/); // stdout half
expect(result).toMatch(/\b0 fail\b/); // stderr half
expect(result).toMatch(/Ran \d+ tests/);
}, 30_000);
});