mirror of
https://github.com/garrytan/gstack.git
synced 2026-08-29 09:20:39 +02:00
* fix(ship): name the /document-release subagent at every Step 18 decision point The v1.54.0.0 carve moved Step 18 (documentation sync) into ship/sections/pr-body.md and the Claude-host skeleton stopped saying "document-release" anywhere in the workflow body — the dispatch became invisible at exactly the moments an agent decides whether to open the section. Restore visibility at three touchpoints, all subagent-framed (never bare-slash-framed, which would invite an inline Skill invocation that bypasses the fresh-context subagent + JSON contract): - manifest trigger (renders into the section-index row AND the STOP pointer): "dispatching the /document-release subagent to sync docs (Step 18) and then creating or updating the PR/MR (Step 19)" - Step 17 handoff line names Step 18's dispatch explicitly - new hoisted doc-sync invariant beside the PR-title invariant: the dispatch itself is never skipped; only a failed subagent is non-blocking Pin it in carve-guards: 'the /document-release subagent' (all three touchpoints) + 'dispatches the /document-release subagent' (invariant) must stay in the skeleton; the carved imperative 'Dispatch /document-release as a subagent' must stay carved. Skeleton cap 91,600 → 92,300 (measured 91,764; trigger renders twice). Goldens regenerated for all three hosts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: pin the ship→document-release Step 18 wiring with a free tripwire Five substring/structure asserts across the carved section, the Claude skeleton's three touchpoints, the manifest trigger, and the codex/factory goldens (inlined Step 18 ordered before Step 19). Claude-golden asserts deliberately omitted: host-config.test.ts already enforces golden == generated byte-for-byte. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: gate-tier E2E proving /ship dispatches the document-release subagent New skill-e2e-ship-docsync: a live agent gets the sliced Step 17→19 tail of the generated ship skeleton in a bare-remote git fixture (Steps 0-16 "done"), under a fake HOME so the STOP pointer and the Step 18 subagent prompt resolve to planted copies, with a stub document-release skill that returns the empty-result JSON contract. Hard assert: an Agent/Task tool-call matching /document-release/i exists in result.toolCalls and precedes any `gh pr create`. Neutral prompt (no STOP-Read priming, no document-release mention — the prompt echoes into the transcript, so asserts read toolCalls only). Hardening from review: throw-on-marker-drift fixture slice; per-test GSTACK_HOME + .redact-prepush-prompted marker (routes Step 17's credential guard to its silent branch — the hermetic GSTACK_HOME pin defeats a HOME-only override); 480s/540s timeouts (nested subagent adds wall clock the 300s sibling never carried); 'timeout' accepted in exitReason only because the dispatch assert is independently hard; whole-file describeE2ETier('gate') composed with diff selection (keeps the file out of the periodic shard census, which sits at its ceiling, and under the hard tier-alignment invariant). Registered as 'ship-docsync' in E2E_TOUCHFILES + E2E_TIERS (gate) in the same commit — touchfiles.test.ts rejects either half landing first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: fix stale document-release TODOS entry + three review-deferred items The SHIPPED entry still described the deleted Step 8.5 post-PR cat-delegation design from v0.8.4; replace with the current Step 18 subagent design and its test pins. Add the three P3 items deferred from the v1.69 plan review: dispatch receipt enforcement, land-and-deploy→canary dispatch-pin pattern, and the periodic shard-census boundary. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: pre-landing review fixes Testing-specialist findings, all mechanical: (1) pin the E2E fixture's git branch (-b main / init.defaultBranch=main) and assert every setup command's exit status so operator git config can't silently corrupt a paid run; (2) tighten the dispatch matcher to Step 18-prompt-specific markers (document-release/SKILL.md | executing the /document-release workflow) so a subagent merely quoting section text can't false-pass the regression assert (verified against recorded burn-in transcripts); (3) replace the subsumed carve-guards anchor with three non-overlapping per-touchpoint anchors (gerund/imperative/3rd-person) so each touchpoint is independently enforced. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: red-team review fixes Five informational findings: TODOS shard-census arithmetic corrected (census is 67 with one free ungated slot; the SECOND ungated file trips the floor) and version pointer fixed (v0.18.2.0, not v0.18.1.0); the free tripwire now pins the two dispatch-matcher marker strings so a pr-body prompt reword fails the free suite instead of surfacing as a paid-tier mystery; the E2E matcher gains a section-paste exclusion (scaffold strings disqualify) — verified against all recorded runs; the E2E header documents the tierless test:evals invisibility tradeoff. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: adversarial review fixes Pin the E2E matcher's two EXCLUSION markers in the free tripwire (an unpinned 'Parent processing:' reword would silently deaden the section-paste guard while every test stayed green); add an ordering pin (the hoisted doc-sync invariant must sit above the pr-body STOP pointer — presence-only anchors can't catch drift below it); plant a third cwd-relative pr-body copy inside the fixture repo, gitignored so the agent never tries to commit test scaffolding. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: bump version and changelog (v1.70.1.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: CHANGELOG accuracy fixes from the doc-release review Three factual corrections the Step 18 doc subagent caught in the fresh v1.70.1.0 entry: 5 tripwire tests (not 6), cost floor $0.63 per the cited eval store (not $0.59), and the visibility claim scoped to decision points (the re-run checklist mention survived the carve). Plus the E2E header's stale pending-burn-in note replaced with the observed numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: raise bun-polyfill subprocess budget to 60s for degraded Windows runners The 50ms-sleep test blew the 20s budget on BOTH bun retry attempts on PR #2700's windows-latest runner (run 32989821401) — sustained AV/runner pressure, not just the documented cold-start. Same flake passed-on-rerun on the prompt-token-load-reduction branch yesterday. Budget only; every assertion still checks exact output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): run the ship-docsync gate E2E in the evals matrix + silent-skip tripwire The evals.yml matrix is hand-enumerated and the Run step never exported EVALS_TIER, so the new whole-file-gated ship-docsync E2E would have self-skipped even with a row — a hollow green one layer deeper than the documented rehomed-monolith incident. Add the e2e-ship-docsync row with a row-level `tier: gate` property, exported as EVALS_TIER by the Run step (empty = unset for every existing row: all readers are `=== '<tier>'` or truthiness). New free tripwire test/evals-workflow-matrix.test.ts ratchets the class: matrix files must exist; gate-hosting files must have a row; whole-file-gated matrix files must carry a matching row tier; and the burn-down lists enforce their own cleanup. It enumerates the PRE-EXISTING holes found while wiring this (8 gate-hosting files with no row; codex/gemini rows running zero tests; the pty-plan-smoke row hollow since its files adopted describeE2ETier) — tracked in TODOS as the CI gate-lane hollow-coverage burn-down. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
270 lines
13 KiB
TypeScript
270 lines
13 KiB
TypeScript
import { describe, test, expect, afterAll, setDefaultTimeout } from 'bun:test';
|
|
import * as path from 'path';
|
|
|
|
// Every test here spawnSync's a `node` child; Windows CI cold-start (AV scan,
|
|
// first-touch of node.exe) alone can blow bun's 5s default — observed 5,007ms
|
|
// on a 50ms sleep test. 20s was still not enough: on 2026-08-26 (PR #2700,
|
|
// run 32989821401) the 50ms sleep test blew 20s on BOTH bun retry attempts on
|
|
// a degraded windows-latest runner, so cold-start alone doesn't explain it —
|
|
// sustained AV/runner pressure does. Subprocess budget, not assertion
|
|
// looseness: every assertion still checks exact output, only the slowness
|
|
// allowance grows.
|
|
setDefaultTimeout(60_000);
|
|
|
|
// Load the polyfill into a fresh object (don't clobber globalThis.Bun)
|
|
const polyfillPath = path.resolve(import.meta.dir, '../src/bun-polyfill.cjs');
|
|
|
|
describe('bun-polyfill', () => {
|
|
// We test the polyfill by requiring it in a subprocess under Node.js
|
|
// since it's designed for Node, not Bun.
|
|
|
|
test('Bun.sleep resolves after delay', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
(async () => {
|
|
const start = Date.now();
|
|
await Bun.sleep(50);
|
|
const elapsed = Date.now() - start;
|
|
console.log(elapsed >= 40 ? 'OK' : 'TOO_FAST');
|
|
})();
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
expect(result.stdout.toString().trim()).toBe('OK');
|
|
expect(result.exitCode).toBe(0);
|
|
});
|
|
|
|
test('Bun.spawnSync runs a command and returns stdout', () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
const r = Bun.spawnSync(['echo', 'hello'], { stdout: 'pipe' });
|
|
console.log(r.stdout.toString().trim());
|
|
console.log('exit:' + r.exitCode);
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
const lines = result.stdout.toString().trim().split('\n');
|
|
expect(lines[0]).toBe('hello');
|
|
expect(lines[1]).toBe('exit:0');
|
|
});
|
|
|
|
test('Bun.spawn launches a process with pid', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
const p = Bun.spawn(['echo', 'test'], { stdio: ['pipe', 'pipe', 'pipe'] });
|
|
console.log(typeof p.pid === 'number' ? 'HAS_PID' : 'NO_PID');
|
|
console.log(typeof p.kill === 'function' ? 'HAS_KILL' : 'NO_KILL');
|
|
console.log(typeof p.unref === 'function' ? 'HAS_UNREF' : 'NO_UNREF');
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
const lines = result.stdout.toString().trim().split('\n');
|
|
expect(lines[0]).toBe('HAS_PID');
|
|
expect(lines[1]).toBe('HAS_KILL');
|
|
expect(lines[2]).toBe('HAS_UNREF');
|
|
});
|
|
|
|
// Bun.spawn parity: `proc.exited` is a Promise resolving to the exit code.
|
|
// The DPAPI helper and isBrowserRunning both `await proc.exited`; without
|
|
// it the awaits resolve immediately to `undefined` and the caller reads
|
|
// stdout before the child has produced it — surfacing as a silent failure.
|
|
test('Bun.spawn exposes proc.exited that resolves to the exit code', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
(async () => {
|
|
const p = Bun.spawn(['node', '-e', 'process.exit(0)'], { stdio: ['ignore', 'ignore', 'ignore'] });
|
|
console.log(typeof p.exited === 'object' && typeof p.exited.then === 'function' ? 'IS_PROMISE' : 'NOT_PROMISE');
|
|
console.log('exit:' + await p.exited);
|
|
})();
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
const lines = result.stdout.toString().trim().split('\n');
|
|
expect(lines[0]).toBe('IS_PROMISE');
|
|
expect(lines[1]).toBe('exit:0');
|
|
});
|
|
|
|
test('Bun.spawn proc.exited reflects non-zero exit codes', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
(async () => {
|
|
const p = Bun.spawn(['node', '-e', 'process.exit(3)'], { stdio: ['ignore', 'ignore', 'ignore'] });
|
|
console.log('exit:' + await p.exited);
|
|
})();
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
expect(result.stdout.toString().trim()).toBe('exit:3');
|
|
});
|
|
|
|
test('Bun.spawn proc.exited resolves before reading stdout (no race)', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
(async () => {
|
|
// Real-world pattern: write to stdout, then exit. Awaiting proc.exited
|
|
// before reading must guarantee the bytes are flushed.
|
|
const p = Bun.spawn(['node', '-e', 'process.stdout.write("ready"); process.exit(0)'], {
|
|
stdio: ['ignore', 'pipe', 'ignore']
|
|
});
|
|
const code = await p.exited;
|
|
const out = await new Response(p.stdout).text();
|
|
console.log(out + ':' + code);
|
|
})();
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
expect(result.stdout.toString().trim()).toBe('ready:0');
|
|
});
|
|
|
|
// Spawn-failure case: Node emits 'error' but not 'exit' when the binary
|
|
// is missing, so listening only for 'exit' hangs `await proc.exited`
|
|
// forever. The lifecycle promise must resolve on either event.
|
|
test('Bun.spawn proc.exited resolves on spawn failure (missing binary)', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
(async () => {
|
|
const p = Bun.spawn(['this-binary-does-not-exist-zzz-' + Date.now()], {
|
|
stdio: ['ignore', 'pipe', 'pipe']
|
|
});
|
|
const code = await Promise.race([
|
|
p.exited,
|
|
new Promise((_, r) => setTimeout(() => r(new Error('timeout')), 3000))
|
|
]).catch(() => 'TIMEOUT');
|
|
console.log('exit:' + code);
|
|
})();
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
// Anything other than 'TIMEOUT' (and ideally a non-zero number) means the
|
|
// lifecycle promise resolved on the spawn error.
|
|
const out = result.stdout.toString().trim();
|
|
expect(out).not.toBe('exit:TIMEOUT');
|
|
expect(out).toMatch(/^exit:\d+$/);
|
|
});
|
|
|
|
// Signal-exit branch: Bun reports 128 + signal number when a child is killed
|
|
// by a signal (code === null). Skipped on Windows, whose kill() semantics
|
|
// don't produce the POSIX 128+n mapping.
|
|
test.skipIf(process.platform === 'win32')('Bun.spawn proc.exited maps a killing signal to 128+signal', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
(async () => {
|
|
const p = Bun.spawn(['node', '-e', 'setInterval(() => {}, 1000)'], { stdio: ['ignore', 'ignore', 'ignore'] });
|
|
setTimeout(() => p.kill('SIGTERM'), 150);
|
|
console.log('exit:' + await p.exited);
|
|
})();
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
// SIGTERM = 15 → 128 + 15 = 143.
|
|
expect(result.stdout.toString().trim()).toBe('exit:143');
|
|
});
|
|
|
|
// GSTACK_SPAWN_MAX_BUFFER caps the drain so a runaway child can't OOM the
|
|
// server. Past the cap, the pipe keeps flowing (child doesn't block) but
|
|
// further bytes are dropped. Set a small cap, write more than that, assert
|
|
// the captured stdout equals the cap and the child exits cleanly.
|
|
test('Bun.spawn caps buffered output at GSTACK_SPAWN_MAX_BUFFER', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
process.env.GSTACK_SPAWN_MAX_BUFFER = '${1024}';
|
|
require(${JSON.stringify(polyfillPath)});
|
|
(async () => {
|
|
// Child writes 10 KB; cap is 1 KB; drained output should be exactly 1 KB
|
|
// and exit should still resolve cleanly (child not back-pressured to death).
|
|
const p = Bun.spawn(
|
|
['node', '-e', 'process.stdout.write("y".repeat(10 * 1024)); process.exit(0)'],
|
|
{ stdio: ['ignore', 'pipe', 'ignore'] }
|
|
);
|
|
const code = await Promise.race([
|
|
p.exited,
|
|
new Promise((_, r) => setTimeout(() => r(new Error('timeout')), 3000))
|
|
]).catch(() => 'TIMEOUT');
|
|
const out = await new Response(p.stdout).text();
|
|
console.log(out.length + ':' + code);
|
|
})();
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
expect(result.stdout.toString().trim()).toBe('1024:0');
|
|
});
|
|
|
|
// Regression for the pipe-blocking case: if the child writes more than the
|
|
// OS pipe buffer (~16-64 KB) and the polyfill doesn't drain eagerly, the
|
|
// child blocks in write() and `exit` never fires. 1 MB is well past every
|
|
// OS pipe buffer size. Pre-fix this test hangs forever; post-fix it returns
|
|
// in <500ms. Bun's default per-test timeout is 5s — generous here.
|
|
test('Bun.spawn drains large stdout so proc.exited still resolves', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
(async () => {
|
|
const ONE_MB = 1024 * 1024;
|
|
// Exit in the write callback, not straight after write(): on modern
|
|
// Node a pipe write past the OS buffer is async, and process.exit()
|
|
// right after write() truncates at ~64 KB even with a live reader.
|
|
// The callback only fires once the full MB is flushed — which still
|
|
// requires the parent to drain, so the regression (no eager drain →
|
|
// child blocks → timeout) is still caught.
|
|
const p = Bun.spawn(
|
|
['node', '-e', 'process.stdout.write("x".repeat(' + ONE_MB + '), () => process.exit(0))'],
|
|
{ stdio: ['ignore', 'pipe', 'ignore'] }
|
|
);
|
|
const code = await Promise.race([
|
|
p.exited,
|
|
new Promise((_, r) => setTimeout(() => r(new Error('timeout')), 10000))
|
|
]).catch(e => 'TIMEOUT');
|
|
const out = await new Response(p.stdout).text();
|
|
console.log(out.length + ':' + code);
|
|
})().catch((e) => { console.log('THREW:' + e.message); });
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
expect(result.stdout.toString().trim()).toBe('1048576:0');
|
|
}, 15000);
|
|
|
|
test('Bun.serve creates an HTTP server that responds', async () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
require(${JSON.stringify(polyfillPath)});
|
|
const server = Bun.serve({
|
|
port: 0, // Note: polyfill uses port directly, so we pick one
|
|
hostname: '127.0.0.1',
|
|
fetch(req) {
|
|
return new Response(JSON.stringify({ ok: true }), {
|
|
headers: { 'Content-Type': 'application/json' },
|
|
});
|
|
},
|
|
});
|
|
// The polyfill doesn't support port 0, so we test the object shape
|
|
console.log(typeof server.stop === 'function' ? 'HAS_STOP' : 'NO_STOP');
|
|
console.log(typeof server.port === 'number' ? 'HAS_PORT' : 'NO_PORT');
|
|
server.stop();
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
const lines = result.stdout.toString().trim().split('\n');
|
|
expect(lines[0]).toBe('HAS_STOP');
|
|
expect(lines[1]).toBe('HAS_PORT');
|
|
});
|
|
|
|
// windowsHide is the one option where Node's default is the opposite of
|
|
// Bun's: Node shows the child's console window, Bun hides it. Dropping it
|
|
// in translation makes every spawned child pop a window on Windows, which
|
|
// is the platform this whole file exists for. Both shims are covered, and
|
|
// an explicit windowsHide:false must survive forwarding (#2523 + #2539).
|
|
test('Bun.spawn defaults windowsHide to true', () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
const cp = require('child_process');
|
|
const orig = cp.spawn;
|
|
let seen;
|
|
cp.spawn = (c, a, o) => { seen = o; return orig(c, a, o); };
|
|
require(${JSON.stringify(polyfillPath)});
|
|
Bun.spawn(['node', '-e', ''], { stdio: ['ignore', 'ignore', 'ignore'] });
|
|
console.log('windowsHide:' + seen.windowsHide);
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
expect(result.stdout.toString().trim()).toBe('windowsHide:true');
|
|
});
|
|
|
|
test('Bun.spawnSync defaults windowsHide to true', () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
const cp = require('child_process');
|
|
const orig = cp.spawnSync;
|
|
let seen;
|
|
cp.spawnSync = (c, a, o) => { seen = o; return orig(c, a, o); };
|
|
require(${JSON.stringify(polyfillPath)});
|
|
Bun.spawnSync(['node', '-e', '']);
|
|
console.log('windowsHide:' + seen.windowsHide);
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
expect(result.stdout.toString().trim()).toBe('windowsHide:true');
|
|
});
|
|
|
|
test('an explicit windowsHide:false is honored', () => {
|
|
const result = Bun.spawnSync(['node', '-e', `
|
|
const cp = require('child_process');
|
|
const orig = cp.spawn;
|
|
let seen;
|
|
cp.spawn = (c, a, o) => { seen = o; return orig(c, a, o); };
|
|
require(${JSON.stringify(polyfillPath)});
|
|
Bun.spawn(['node', '-e', ''], { stdio: ['ignore', 'ignore', 'ignore'], windowsHide: false });
|
|
console.log('windowsHide:' + seen.windowsHide);
|
|
`], { stdout: 'pipe', stderr: 'pipe' });
|
|
expect(result.stdout.toString().trim()).toBe('windowsHide:false');
|
|
});
|
|
});
|