mirror of
https://github.com/garrytan/gstack.git
synced 2026-08-21 13:37:14 +02:00
ci: Linux free-test lane — ~400 files get CI coverage for the first time
New required, secretless free-tests job: the canonical runner's single 'bun test --parallel' invocation with strict-output classification on ubicloud-standard-8. The free suite previously ran on NO Linux CI — only a curated Windows subset ran anywhere — so every 'tests pass' claim about main rested on contributors running them locally. Secretless by design (no API keys; fork PRs finally get real test signal) and pinned by test/free-tests-workflow-wiring.test.ts: canonical runner invoked, zero secrets.* references, pull_request never pull_request_target, and matrix-count/--shards agreement if anyone switches to the sharded fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
41160a14ed
commit
f23b2263bc
@@ -0,0 +1,75 @@
|
|||||||
|
name: Free Tests
|
||||||
|
|
||||||
|
# The free suite (~400 files: test/, browse/test/, make-pdf/test/, design/test/)
|
||||||
|
# had ZERO Linux CI coverage before this lane — only a curated Windows subset
|
||||||
|
# ran anywhere. This job runs the whole thing through the canonical runner
|
||||||
|
# (scripts/test-free-shards.ts): one `bun test --parallel` invocation with
|
||||||
|
# strict-output classification, so a truncated or summary-less run can never
|
||||||
|
# report green.
|
||||||
|
#
|
||||||
|
# Deliberately SECRETLESS: free tests make no API calls, so this lane gets no
|
||||||
|
# provider keys at all — least privilege, and fork PRs get real test signal
|
||||||
|
# here (the eval matrix skips fork PRs because repository secrets can't reach
|
||||||
|
# them). test/free-tests-workflow-wiring.test.ts fails CI if a secret sneaks in.
|
||||||
|
#
|
||||||
|
# This is a REQUIRED check from day one (branch protection lists it). If it's
|
||||||
|
# red, fix or quarantine-with-issue — don't make it advisory; an advisory lane
|
||||||
|
# is permanent false comfort.
|
||||||
|
#
|
||||||
|
# Sizing note (decision V3): single --parallel job first. If PR runs show it
|
||||||
|
# slower than the eval matrix wall, switch to a matrix of
|
||||||
|
# `--shards N --shard i` jobs (indices are stable, empty shards no-op).
|
||||||
|
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
branches: [main]
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: free-tests-${{ github.head_ref || github.run_id }}
|
||||||
|
cancel-in-progress: true
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
free-tests:
|
||||||
|
runs-on: ubicloud-standard-8
|
||||||
|
timeout-minutes: 20
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- uses: oven-sh/setup-bun@v2
|
||||||
|
with:
|
||||||
|
bun-version: 1.3.13
|
||||||
|
|
||||||
|
- uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: ~/.bun/install/cache
|
||||||
|
key: linux-bun-${{ hashFiles('bun.lock') }}
|
||||||
|
|
||||||
|
- name: Install dependencies
|
||||||
|
run: bun install --frozen-lockfile
|
||||||
|
|
||||||
|
- uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: ~/.cache/ms-playwright
|
||||||
|
key: linux-playwright-${{ hashFiles('bun.lock') }}
|
||||||
|
|
||||||
|
# Cache restores browser binaries; install is still required for system
|
||||||
|
# deps and is a fast no-op for already-present browsers.
|
||||||
|
- name: Install Playwright Chromium
|
||||||
|
run: npx playwright install --with-deps chromium
|
||||||
|
|
||||||
|
- name: Configure git identity (tests init temp repos)
|
||||||
|
run: |
|
||||||
|
git config --global user.email "free-tests-ci@gstack.test"
|
||||||
|
git config --global user.name "Free Tests CI"
|
||||||
|
git config --global init.defaultBranch main
|
||||||
|
|
||||||
|
- name: Generate host SKILL.md outputs (.agents, .factory)
|
||||||
|
# Golden-file tests read generated host outputs that are gitignored.
|
||||||
|
run: bun run gen:skill-docs --host all
|
||||||
|
|
||||||
|
- name: Build server-node bundle (loaded by browse cli imports)
|
||||||
|
run: bash browse/scripts/build-node-server.sh
|
||||||
|
|
||||||
|
- name: Run free suite
|
||||||
|
run: bun run test:free
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
/**
|
||||||
|
* Static tripwire for .github/workflows/free-tests.yml — the Linux free-suite
|
||||||
|
* lane. Pins the three properties that made the lane worth having:
|
||||||
|
*
|
||||||
|
* 1. It invokes the CANONICAL runner (bun run test:free), not a raw
|
||||||
|
* `bun test <dirs>` glob — the runner owns TEST_ROOTS and strict-output
|
||||||
|
* classification, so a truncated run can't report green.
|
||||||
|
* 2. It is SECRETLESS: free tests make no API calls, and keeping keys out
|
||||||
|
* means fork PRs get real signal here. Any `secrets.` reference is a
|
||||||
|
* regression.
|
||||||
|
* 3. It triggers on `pull_request` (never `pull_request_target`, which
|
||||||
|
* would hand a fork PR the base repo's context).
|
||||||
|
*
|
||||||
|
* Same wiring-tripwire class as test/hermetic-wiring.test.ts.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, test, expect } from 'bun:test';
|
||||||
|
import * as fs from 'fs';
|
||||||
|
import * as path from 'path';
|
||||||
|
|
||||||
|
const WORKFLOW = path.resolve(import.meta.dir, '..', '.github', 'workflows', 'free-tests.yml');
|
||||||
|
|
||||||
|
describe('free-tests workflow wiring', () => {
|
||||||
|
const source = fs.readFileSync(WORKFLOW, 'utf-8');
|
||||||
|
|
||||||
|
test('workflow exists and invokes the canonical runner', () => {
|
||||||
|
expect(source).toContain('bun run test:free');
|
||||||
|
expect(source).not.toMatch(/run:\s*bun test\s/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('secretless: no secrets reach the free lane', () => {
|
||||||
|
expect(source).not.toContain('secrets.');
|
||||||
|
expect(source).not.toContain('ANTHROPIC_API_KEY');
|
||||||
|
expect(source).not.toContain('OPENAI_API_KEY');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pull_request trigger, never pull_request_target', () => {
|
||||||
|
expect(source).toContain('pull_request:');
|
||||||
|
expect(source).not.toContain('pull_request_target');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('if sharded (matrix), the matrix count matches --shards N', () => {
|
||||||
|
// Single-job --parallel mode has no matrix — vacuously fine. If someone
|
||||||
|
// switches to the shard matrix (the V3 fallback), the two encodings of
|
||||||
|
// the shard count must agree or CI silently drops files.
|
||||||
|
const shardsFlag = source.match(/--shards\s+(\d+)/);
|
||||||
|
const matrix = source.match(/shard:\s*\[([^\]]+)\]/);
|
||||||
|
if (shardsFlag || matrix) {
|
||||||
|
expect(shardsFlag, 'matrix present but no --shards N flag').toBeTruthy();
|
||||||
|
expect(matrix, '--shards N present but no shard matrix').toBeTruthy();
|
||||||
|
const count = parseInt(shardsFlag![1], 10);
|
||||||
|
const entries = matrix![1].split(',').map(s => s.trim()).filter(Boolean);
|
||||||
|
expect(entries.length).toBe(count);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user