ci: Linux free-test lane — ~400 files get CI coverage for the first time

New required, secretless free-tests job: the canonical runner's single
'bun test --parallel' invocation with strict-output classification on
ubicloud-standard-8. The free suite previously ran on NO Linux CI — only
a curated Windows subset ran anywhere — so every 'tests pass' claim
about main rested on contributors running them locally.

Secretless by design (no API keys; fork PRs finally get real test
signal) and pinned by test/free-tests-workflow-wiring.test.ts: canonical
runner invoked, zero secrets.* references, pull_request never
pull_request_target, and matrix-count/--shards agreement if anyone
switches to the sharded fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-15 08:24:20 -07:00
co-authored by Claude Fable 5
parent 41160a14ed
commit f23b2263bc
2 changed files with 131 additions and 0 deletions
+75
View File
@@ -0,0 +1,75 @@
name: Free Tests
# The free suite (~400 files: test/, browse/test/, make-pdf/test/, design/test/)
# had ZERO Linux CI coverage before this lane — only a curated Windows subset
# ran anywhere. This job runs the whole thing through the canonical runner
# (scripts/test-free-shards.ts): one `bun test --parallel` invocation with
# strict-output classification, so a truncated or summary-less run can never
# report green.
#
# Deliberately SECRETLESS: free tests make no API calls, so this lane gets no
# provider keys at all — least privilege, and fork PRs get real test signal
# here (the eval matrix skips fork PRs because repository secrets can't reach
# them). test/free-tests-workflow-wiring.test.ts fails CI if a secret sneaks in.
#
# This is a REQUIRED check from day one (branch protection lists it). If it's
# red, fix or quarantine-with-issue — don't make it advisory; an advisory lane
# is permanent false comfort.
#
# Sizing note (decision V3): single --parallel job first. If PR runs show it
# slower than the eval matrix wall, switch to a matrix of
# `--shards N --shard i` jobs (indices are stable, empty shards no-op).
on:
pull_request:
branches: [main]
workflow_dispatch:
concurrency:
group: free-tests-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
free-tests:
runs-on: ubicloud-standard-8
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.13
- uses: actions/cache@v4
with:
path: ~/.bun/install/cache
key: linux-bun-${{ hashFiles('bun.lock') }}
- name: Install dependencies
run: bun install --frozen-lockfile
- uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: linux-playwright-${{ hashFiles('bun.lock') }}
# Cache restores browser binaries; install is still required for system
# deps and is a fast no-op for already-present browsers.
- name: Install Playwright Chromium
run: npx playwright install --with-deps chromium
- name: Configure git identity (tests init temp repos)
run: |
git config --global user.email "free-tests-ci@gstack.test"
git config --global user.name "Free Tests CI"
git config --global init.defaultBranch main
- name: Generate host SKILL.md outputs (.agents, .factory)
# Golden-file tests read generated host outputs that are gitignored.
run: bun run gen:skill-docs --host all
- name: Build server-node bundle (loaded by browse cli imports)
run: bash browse/scripts/build-node-server.sh
- name: Run free suite
run: bun run test:free
+56
View File
@@ -0,0 +1,56 @@
/**
* Static tripwire for .github/workflows/free-tests.yml — the Linux free-suite
* lane. Pins the three properties that made the lane worth having:
*
* 1. It invokes the CANONICAL runner (bun run test:free), not a raw
* `bun test <dirs>` glob — the runner owns TEST_ROOTS and strict-output
* classification, so a truncated run can't report green.
* 2. It is SECRETLESS: free tests make no API calls, and keeping keys out
* means fork PRs get real signal here. Any `secrets.` reference is a
* regression.
* 3. It triggers on `pull_request` (never `pull_request_target`, which
* would hand a fork PR the base repo's context).
*
* Same wiring-tripwire class as test/hermetic-wiring.test.ts.
*/
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
const WORKFLOW = path.resolve(import.meta.dir, '..', '.github', 'workflows', 'free-tests.yml');
describe('free-tests workflow wiring', () => {
const source = fs.readFileSync(WORKFLOW, 'utf-8');
test('workflow exists and invokes the canonical runner', () => {
expect(source).toContain('bun run test:free');
expect(source).not.toMatch(/run:\s*bun test\s/);
});
test('secretless: no secrets reach the free lane', () => {
expect(source).not.toContain('secrets.');
expect(source).not.toContain('ANTHROPIC_API_KEY');
expect(source).not.toContain('OPENAI_API_KEY');
});
test('pull_request trigger, never pull_request_target', () => {
expect(source).toContain('pull_request:');
expect(source).not.toContain('pull_request_target');
});
test('if sharded (matrix), the matrix count matches --shards N', () => {
// Single-job --parallel mode has no matrix — vacuously fine. If someone
// switches to the shard matrix (the V3 fallback), the two encodings of
// the shard count must agree or CI silently drops files.
const shardsFlag = source.match(/--shards\s+(\d+)/);
const matrix = source.match(/shard:\s*\[([^\]]+)\]/);
if (shardsFlag || matrix) {
expect(shardsFlag, 'matrix present but no --shards N flag').toBeTruthy();
expect(matrix, '--shards N present but no shard matrix').toBeTruthy();
const count = parseInt(shardsFlag![1], 10);
const entries = matrix![1].split(',').map(s => s.trim()).filter(Boolean);
expect(entries.length).toBe(count);
}
});
});