Round 5 of windows-free-tests fixes. Round 4 caught Playwright launchers
but two more failure shapes appeared in shard 5:
1. test/diff-scope.test.ts uses `import { join }` (destructured) and
`join(import.meta.dir, '..', 'bin', 'gstack-diff-scope')`. My round-3
pattern only matched `path.join(...)` — the destructured form slipped
through. Tightened the pattern to match the literal `, 'bin', '<name>'`
path-segment shape regardless of whether it's `path.join` or `join`
directly.
2. browse/test/sidebar-integration.test.ts spawns the browse server via
`spawn(['bun', 'run', server.ts])` with BROWSE_HEADLESS_SKIP=1. The
Bun-run-server.ts path is the same Playwright-on-Windows broken path
that the windows-free-tests job intentionally avoids — the server-node.mjs
route only kicks in for the compiled binary, not direct Bun runs of the
TypeScript source. Added a BROWSE_HEADLESS_SKIP / spawn-bun-run pattern.
Curated Windows subset: 72 → 73 tests (~57% of free suite). Net up by 1
because the tightened bin pattern released one test that was a false
positive in the loose `path\\.join` form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round 4 of windows-free-tests fixes. Round 3 cleared shard 2 except for
browse/test/batch.test.ts:35 which calls `await bm.launch()` and triggers
Playwright Chromium launch. The windows-latest runner doesn't have
Chromium installed (browser bring-up is a separate concern, tracked by
PR #1238 windows-pty-bun-pty-fix).
Fix: extend WINDOWS_FRAGILE_PATTERNS with `await \\w+\\.launch\\(` matcher.
Catches batch.test.ts plus 7 sibling tests (commands, compare-board,
content-security, handoff, security-live-playwright, security-sidepanel-dom,
snapshot — most already excluded by other patterns).
Curated Windows subset: 76 → 72 tests (~56% of free suite). Net curation
across all 4 rounds: 56 of 128 free tests excluded, each with a logged
reason. The 56 excluded fall into 6 buckets — POSIX shells, raw /tmp/,
chmod/xargs, file mode bitmasks, forward-slash path assertions, bin/
shebang spawns, and Playwright launches — all tracked as a P4 follow-up
TODO for full Windows parity.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round 3 of windows-free-tests fixes. Round 2 (LF gitattributes + server-node.mjs
build) cleared shard 1 entirely (skill-collision-sentinel and tab-isolation
green). Shard 2 surfaced two more issues:
1. browse/test/claude-bin.test.ts:50 — the "PATH-resolvable override" test
creates a fake binary 'fake-claude-cli' (no extension) and expects
Bun.which to find it. On Windows, Bun.which probes PATHEXT extensions
(.cmd, .exe, .bat) — a bare-name file is not discoverable. Production
behavior is correct; the test was Mac/Linux-shaped.
Fix: branch on process.platform. On Windows, write 'fake-claude-cli.cmd'
with a Windows batch payload instead of a POSIX shebang script.
2. test/gstack-question-log.test.ts (and 18 sibling tests) — spawn a bash
shebang script via spawnSync(BIN, args). Git Bash on Windows can run
`bash /path/to/script` but spawnSync invokes CreateProcess directly,
which doesn't parse #!/usr/bin/env bash. All these tests are
Windows-fragile and can't run as-is.
Fix: extend WINDOWS_FRAGILE_PATTERNS with `path.join(.., 'bin', ..)`
detector. Curates 19 additional tests (benchmark-cli, brain-sync,
builder-profile, explain-level-config, gbrain-*, gstack-question-*,
hook-scripts, learnings, plan-tune, review-log, secret-sink-harness,
taste-engine, telemetry, timeline, uninstall).
Curated Windows subset: 95 → 76 tests (~59% of free suite). Still
meaningful Windows coverage. The 52 excluded tests are tracked as a
follow-up TODO for full Windows parity (shebang-bin spawns + POSIX file
modes + raw /tmp/ etc).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First windows-free-tests CI run surfaced 34 failures across two patterns:
1. Tests that init a temp git repo via execSync('git commit ...') — Windows
runner has no default git user.email/user.name, so the commit fails.
Fix: add a "Configure git identity" step to .github/workflows/windows-free-tests.yml
that sets a CI-only identity globally.
2. Tests that use POSIX-only APIs unconditionally:
- file-mode bitmask checks (`stat.mode & 0o600`, `mode & 0o111`) — Windows
fakes mode bits and these assertions don't compose
- hardcoded forward-slash path assertions (`file.endsWith('/tab-42.json')`)
— Windows path separators are '\\'
Fix: extend WINDOWS_FRAGILE_PATTERNS in scripts/test-free-shards.ts to
detect both. 8 additional tests now excluded from the curated Windows
subset with logged reasons:
- browse/test/security-review-flow.test.ts (file mode)
- browse/test/security-sidepanel-dom.test.ts (forward-slash path)
- browse/test/url-validation.test.ts (forward-slash path)
- test/gbrain-repo-policy.test.ts (file mode)
- test/relink.test.ts (file mode)
- test/skill-validation.test.ts (file mode — single assertion at :934)
- test/team-mode.test.ts (file mode — also kills its 30 git-init beforeEach failures)
- test/upgrade-migration-v1.test.ts (file mode)
Curated Windows subset: 103 → 95 tests (still ~74% of free suite). All
14 test-free-shards unit tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex's v1.18.0.0 review flagged that a windows-latest matrix entry on the
existing Linux-container evals.yml workflow can't work as a drop-in, and that
the free test suite has POSIX-bound dependencies a sharded runner doesn't fix
on its own. This commit takes McGluut's test-free-shards.ts (190 LOC), adds a
Windows-fragility scan, and runs the curated subset on a separate non-container
windows-latest job.
scripts/test-free-shards.ts:
- Enumeration + paid-eval filtering + stable-hash sharding (FNV-1a). Adapted
from McGluut/gstack fork.
- Upstream-original: --windows-only filter scans each test's content for
POSIX-bound patterns: hardcoded /bin/sh, spawn('sh', ...), bash -c, raw
/tmp/, chmod, xargs, which claude. Files matching are excluded with the
reason logged. Currently filters 25 of 128 free tests; remaining 103 run
on windows-latest.
.github/workflows/windows-free-tests.yml:
- Separate non-container job (NOT a matrix entry on evals.yml). Runs:
bun run test:windows # curated subset
bun test browse/test/claude-bin.test.ts # PATHEXT+overrides on Windows
bun test test/gstack-paths.test.ts # state-root resolution
package.json: new test:free + test:windows scripts.
Honest about scope (codex-flagged): this does NOT make the full free suite
Windows-safe. The 25 excluded tests need POSIX-only surfaces ported off shell
primitives (test/ship-version-sync.test.ts:72 hardcodes /bin/bash, etc).
Tracked as a P4 follow-up TODO. Full Windows parity is the next wave; this
release ships the curated lane.
Tests: test/test-free-shards.test.ts has 14 unit tests covering enumeration,
paid-eval filtering, Windows-fragility detection (POSIX patterns + safe code),
and stable sharding determinism.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Garry Tan