v1.91.105.0 fix(tests): the free suite stops needing its retry pass; test:health tells the truth (#3154) (#3212)

* test: turn git auto-maintenance off in free fixtures, with a runner backstop

git 2.29+ starts a detached `git maintenance run --auto` after every commit;
it creates and deletes .git/objects/maintenance.lock after the commit returns.
Diagnosed on 47d639a (git 2.55.0):

- gstack-docs-candidate (ledger 37742176952/16): 4/150 replicated snapshots
  saw the .git listing change, every one `-objects/maintenance.lock`; 0/150
  with the keys. repo() now sets maintenance.auto=false and gc.auto=0.
- plan-design-sdk-fixture (ledger 37807608021/6): under commit load 18/150
  runs failed the collector check; in every one Bun's rmSync(recursive, force)
  returned without throwing and left the scratch repo behind. 0/100 with the
  keys in the fixture's replacement git env, which never sees the runner's.
- qa-functional-observer (ledger 37800857251/4): under load 6/200 observer
  setups threw ENOENT lstat .git/objects/maintenance.lock; 0/400 with the keys
  in the inherited env. The observer and its fixture (22 paid cases) are not
  edited; the runner backstop covers them.

runFreeShard appends the two keys after any inherited GIT_CONFIG_* pairs of
each shard child (a malformed inherited count is left for git to reject).
A GIT_TRACE commit shows 3 maintenance lines with defaults and 0 in the
runner child env; the new test pins that and the env merge cases.
Failure messages now name the changed .git entries and surviving fixture
directories; no assertion changed.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: synchronize the zombie publication-lock test on a framed PID line

Ledger 38153371796/3 and 38161906980/3: line 69 `existsSync(ready)` failed.
The test polled 300 x 10 ms for the Python parent's readiness file, then
another 300 x 10 ms of /proc for state Z; under shard load the fixed count
expired. existsSync also raced the write (an empty file read as pid 0).

The parent now waits for its child with os.waitid(P_PID, WEXITED|WNOWAIT),
which leaves it unreaped in state Z, and prints the PID with flush=True. The
test reads exactly one newline-framed line from the parent's stdout under the
test's own (unchanged) 10 s timeout, rejects a malformed or non-positive PID,
and makes one /proc read that must show Z. Early exit, a malformed line or a
timeout reports the parent's exit status and stderr tail; the owned parent is
always killed in cleanup.

Intentional assertion replacement: the readiness-file existence check and the
polling loop are replaced by the framed-line requirement (a positive PID) and
one /proc read; `expect(state).toBe('Z')` and every lock-recovery assertion
are unchanged.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: stop the next-version smoke test from calling live GitHub

Ledger 37804770972/9: "CLI runs against real repo and emits parseable JSON"
failed with `JSON Parse error: Unexpected EOF` after 30032 ms. The smoke test
ran the real CLI against live `gh`; bin/gstack-next-version's per-PR
`gh api` calls run one after another (spawnSync inside its async workers), so
27 open PRs passed the 30 s spawn timeout and stdout was empty. Measured with
a delayed fake gh: 8 PRs x 0.5 s = 4108 ms; 27 PRs x 1.2 s = 32637 ms. The
CLI defect is filed as a P1 follow-up, not fixed here.

The test keeps the real repository and the real CLI and puts a named fake gh
first on PATH. It logs every argv, answers exactly the calls the CLI makes
(auth status only when origin is not github.com, pr view exit 1, canned pr
list, repo view and contents with one same-owner PR claiming 1.6.4.0) and
exits 2 with a FAKE_GH_UNHANDLED marker for anything else; glab exits 1. The
test now checks the exit code with stderr before JSON.parse, the exact call
set, no marker, no glab call, host github, offline false, no fallback, no
warnings, the canned claim, and version 1.6.5.0. Every existing assertion is
kept; the spawn timeout is unchanged.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(test-health): report the Windows wall over PR runs and name the seed's real prerequisite

free-wall read windows-free-tests.yml with `branch: 'main'`, but that
workflow runs only on pull requests and workflow_dispatch, so the "main" runs
it averaged were dispatch duration recordings. It now uses listWorkflowRuns'
existing `event: 'pull_request'` filter and prints the population inline:
`windows-free-tests (successful PR runs) 7.1 min (23 runs)`, or `no
successful PR runs in window`. The free-tests half is byte-identical.

The CEO-10 fix text stops offering a "CI recording" (none exists for the Linux
seed) and names UBICLOUD_API_KEY; test:ubicloud --record-durations copies the
seed back into the checkout, and contributors without the key leave the
refresh to a maintainer. docs/TESTING_INTERNALS.md drops "or a CI recording"
in the same change.

Intentional pin change: test/test-health-report.test.ts free-wall value
`windows-free-tests 0.0 min (1 runs)` -> `windows-free-tests (successful PR
runs) 0.0 min (1 runs)`; the fixture now carries a PR run and an excluded
dispatch run, plus a new case for the empty and mixed populations.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* fix(timeline-read): reject missing and unparseable --since/--branch values

`--since` and `--branch` read an unbound $2 under `set -u` and crashed with
exit 1 when given no value, and an unparseable --since ("yesterday", "2
weeks") silently returned unfiltered history because the Bun regex was
unanchored and fell through. Unknown arguments were silently ignored.

--since is now parsed once in bash (no `date -d`, no bash-4 features) into a
cutoff in epoch ms, and the Bun regex is deleted. Accepted forms: "<N>
minute|hour|day|week|month[s] ago" (case-insensitive, as before), "<N>h|d|w"
(the form gstack-global-discover documents) and ISO dates YYYY-MM-DD or
YYYY-MM-DDTHH:MM:SSZ (UTC). In #1723's shape, before any read:

  --since must be a relative time like "7 days ago" (got no value); accepted: ...
  --since must be a relative time like "7 days ago" (got 'yesterday'); accepted: ...
  --branch must be a branch name (got flag '--limit')

with "got an empty value" for ''. Unknown arguments exit 2 with the usage
line; --help/-h prints it and exits 0. Tests extend the --limit table in
test/timeline.test.ts and run every case on empty and populated history,
plus each accepted --since form filtering old events; the quoted-branch and
no-filter tests are unchanged.

Behavior change: unrecognized arguments (previously ignored) now exit 2.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* docs: record the due audit follow-ups with verdicts, and file wave-2 follow-ups

docs/test-audit-2026-10.md gains a dated 2026-10-15 section from a no-spend
`bun run test:health --since-days 7`, with the 2026-10-12 figures from #3154
kept as history and a verdict per target: full-fallback 45.9% (open; the
audit's next step is touchfile mappings for bin/, lib/, hosts/, make-pdf/),
spend per merged PR $73.52 (verdict unavailable: the pre-change baseline,
recomputed with collectHealth at now=2026-10-04 behind an upper-bound client,
holds no trial-outcomes artifacts because the recorder shipped with the
audit), cancelled slice-minutes 1,545/week, push-to-verdict p50 11.4 min
(missed against 3-5 min), flaky-pass 6.1% (missed; this wave repairs it).
Census reliability is not due until 2026-10-26; the interim figure is
recorded and the repair stays with #3153/#3200.

TODOS.md: one appended dated section (P1 gstack-next-version serial fetch with
the 4160 ms measurement, P1 standing seed recorder, P1 full-fallback/spend,
P1 dated flake follow-ups, P2 createQAFunctionalFixture maintenance keys, P2
diagnostic-mode HOME/state-root artifacts). The timeline-read bullet was
deleted with its fix.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* chore(release): tentative stamp v1.91.105.0 with its CHANGELOG entry

Stamp-at-merge policy (.gstack/ship-policy.json): the branch carries a
tentative PATCH stamp and its CHANGELOG section directly under
`# Changelog`; the queue restamps at merge time. Base 47d639a is 1.91.101.0,
but the live slot scan shows 1.91.102.0 (#3190), 1.91.103.0 (#3200) and
1.91.104.0 (#3209) claimed, so this branch takes 1.91.105.0. Written with
`gstack-restamp --version 1.91.105.0 --base main --expect-base 47d639a
--allow-repo-commands` (VERSION, package.json's npm translation, the agents
digest via the policy's release tool, and the CHANGELOG header).

* test(seed): refresh the free-suite duration seed from a Ubicloud recording (#3154)

Recorded with `bun run test:ubicloud --record-durations` (standard-16,
eu-central-h1) on this branch, which equals origin/main 47d639a plus this
wave; the recorder exited 0 and wrote 1219 durations (1055 before, 164 new,
none removed; serial 2439 s -> 3202 s). `test:health` CEO-10 now reports
0 unseeded files.

Two earlier recordings did not count: one failed on an unrelated fixed-sleep
race in test/plan-seed-submission.test.ts (filed P2, not edited here), the
other failed free-seed-ratchet because it ran against the first attempt's
pulled seed. No file was excluded.

free-seed-ratchet: three files are over the 60 s ceiling in this recording
and are allowlisted with a split todo (setup-install-registry 82 s,
host-renders-prune 64 s, setup-codex-scope 60.4 s); the split follow-up and
the plan-seed-submission race are filed in the appended TODOS section.

The ratchet test now names scripts/free-test-seed-allowlist.json literally
(a pin on SEED_ALLOWLIST_FILE): pregate's lanes check maps data files to
tests through string literals, and the allowlist had no lane
(PREGATE_NO_LANE). Additive assertion; nothing weakened.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

* test: give the git-maintenance backstop test a Windows-safe empty global config

windows-free-shard (2) on #3212 (run 38832498861) failed two cases in
test/test-free-shards-git-maintenance.test.ts, both serially reproduced by
the retry pass: Git for Windows rejects GIT_CONFIG_GLOBAL=os.devNull
(`fatal: unable to access '\\.\nul': Invalid argument`), so the malformed-count
case saw that error instead of git's GIT_CONFIG_COUNT rejection, and the traced
commit's `git init` exited nonzero. Point GIT_CONFIG_GLOBAL at a file that does
not exist in the scratch HOME, as test/ceo-plan-mode-fixture.test.ts does;
git reads a missing global config as empty on every platform. No assertion
changed.

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>

---------

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
This commit is contained in:
Garry Tanandcapy-ai[bot] authored and GitHub committed 2026-10-16 13:29:38 -07:00
1 parent 47d639abff
commit a889ef9393
20 files changed
+1778 -1115

No files matched your search

+1
View File
@@ -95,6 +95,7 @@ describe('seed ceiling and shrink-only allowlist', () => {
});
test('the committed seed holds the ceiling and the allowlist only shrinks', () => {
expect(SEED_ALLOWLIST_FILE).toBe('scripts/free-test-seed-allowlist.json');
const seed = JSON.parse(fs.readFileSync(path.join(ROOT, FREE_TEST_DURATIONS_FILE), 'utf8')) as { durations: Record<string, number> };
const allowlist = parseSeedAllowlist(fs.readFileSync(path.join(ROOT, SEED_ALLOWLIST_FILE), 'utf8'));
const explicitBase = process.env.GSTACK_FREE_SEED_BASE;
+5 -2
View File
@@ -21,6 +21,7 @@ function repo() {
fs.mkdirSync(path.join(dir, 'handbook'), { recursive: true });
git(root, 'init', '-q', '-b', 'main', dir);
git(dir, 'config', 'user.email', 't@t'); git(dir, 'config', 'user.name', 't'); git(dir, 'config', 'commit.gpgsign', 'false');
git(dir, 'config', 'maintenance.auto', 'false'); git(dir, 'config', 'gc.auto', '0');
fs.writeFileSync(path.join(dir, 'app.ts'), 'export const a = 1;\n');
fs.writeFileSync(path.join(dir, 'handbook', 'guide.md'), '# Guide\n');
git(dir, 'add', '.'); git(dir, 'commit', '-qm', 'base');
@@ -38,7 +39,7 @@ describe('gstack-docs-candidate', () => {
test('snapshot records base, HEAD, index, NUL-safe path lists and content hashes without touching the repository', () => {
const { dir, record } = repo();
const gitDir = path.join(dir, '.git');
const before = fs.readdirSync(gitDir, { recursive: true }).length;
const before = fs.readdirSync(gitDir, { recursive: true }) as string[];
const indexBefore = fs.readFileSync(path.join(gitDir, 'index'));
const result = run(dir, 'snapshot', '--out', record, '--audit-id', 'a1', '--mode', 'edit', '--base', 'main', '--docs', 'handbook');
expect(result.status, result.stderr).toBe(0);
@@ -51,7 +52,9 @@ describe('gstack-docs-candidate', () => {
expect(saved.hashes['app.ts']).toBe(git(dir, 'hash-object', 'app.ts'));
expect(saved.index).toHaveLength(2);
expect(fs.statSync(record).mode & 0o777).toBe(0o600);
expect(fs.readdirSync(gitDir, { recursive: true }).length).toBe(before);
const after = fs.readdirSync(gitDir, { recursive: true }) as string[];
const changed = [...before.filter(entry => !after.includes(entry)).map(entry => `-${entry}`), ...after.filter(entry => !before.includes(entry)).map(entry => `+${entry}`)];
expect(after.length, `.git entries changed during snapshot: ${changed.join(', ') || '(same names)'}`).toBe(before.length);
expect(fs.readFileSync(path.join(gitDir, 'index')).equals(indexBefore)).toBe(true);
});
+83 -28
View File
@@ -932,35 +932,90 @@ describe("width pinned on failed base read (3-digit repos)", () => {
describe("integration (smoke)", () => {
const SMOKE_SCRIPT = join(import.meta.dir, "..", "bin", "gstack-next-version");
// Bumps timeout to 30s — the test spawns a real `bun run` subprocess that
// does a `gh pr list` against the live GitHub API to inspect claimed slots.
// Network latency makes 5s tight on developer machines.
const FAKE_OWNER = "fixture-owner";
const CLAIM = { pr: 41, branch: "fixture/claim", version: "1.6.4.0", url: "https://github.com/fixture-owner/repo/pull/41" };
// A named fake `gh` first on PATH answers every call the CLI makes in this
// checkout and logs each argv; anything else exits 2 so a fixture gap cannot
// silently change the path. The live GitHub API is never called.
function fakeGh(dir: string): { calls: () => string[][]; glabCalls: () => string } {
const log = join(dir, "gh-calls.log");
const glabLog = join(dir, "glab-calls.log");
const prList = JSON.stringify([{ number: CLAIM.pr, headRefName: CLAIM.branch, headRepositoryOwner: { login: FAKE_OWNER }, url: CLAIM.url, isDraft: false }]);
const content = Buffer.from(`${CLAIM.version}\n`).toString("base64");
writeFileSync(join(dir, "gh"), [
"#!/bin/sh",
`printf '%s\\037' "$@" >> '${log}'; printf '\\n' >> '${log}'`,
'case "$*" in',
' "auth status") exit 0 ;;',
' "pr view --json number -q .number") exit 1 ;;',
` "pr list --state open --base main --limit 200 --json number,headRefName,headRepositoryOwner,url,isDraft") printf '%s\\n' '${prList}'; exit 0 ;;`,
` "repo view --json owner -q .owner.login") echo '${FAKE_OWNER}'; exit 0 ;;`,
` "api repos/{owner}/{repo}/contents/VERSION?ref=${encodeURIComponent(CLAIM.branch)} -q .content") echo '${content}'; exit 0 ;;`,
"esac",
`echo "FAKE_GH_UNHANDLED $*" >> '${log}'`,
'echo "FAKE_GH_UNHANDLED $*" >&2',
"exit 2",
"",
].join("\n"), { mode: 0o755 });
writeFileSync(join(dir, "glab"), `#!/bin/sh\necho "$*" >> '${glabLog}'\nexit 1\n`, { mode: 0o755 });
return {
calls: () => readFileSync(log, "utf8").trimEnd().split("\n").map((line) => line.split("\x1f").filter(Boolean)),
glabCalls: () => { try { return readFileSync(glabLog, "utf8"); } catch { return ""; } },
};
}
test("CLI runs against real repo and emits parseable JSON", async () => {
const proc = Bun.spawnSync([
"bun",
"run",
"./bin/gstack-next-version",
"--base",
"main",
"--bump",
"patch",
"--current-version",
"1.6.3.0",
"--workspace-root",
"null", // skip sibling scan in CI
], { timeout: 30_000 });
const out = new TextDecoder().decode(proc.stdout);
const parsed = JSON.parse(out);
expect(parsed).toHaveProperty("version");
expect(parseVersion(parsed.version)).not.toBeNull();
expect(parsed).toHaveProperty("bump", "patch");
expect(parsed).toHaveProperty("host");
expect(["github", "gitlab", "unknown"]).toContain(parsed.host);
expect(parsed).toHaveProperty("claimed");
expect(Array.isArray(parsed.claimed)).toBe(true);
expect(parsed).toHaveProperty("siblings");
expect(parsed.siblings).toEqual([]); // --workspace-root null disabled scanning
expect(parsed).toHaveProperty("version_path", "VERSION"); // default when no config + no flag
const stubDir = mkdtempSync(join(tmpdir(), "nextver-smoke-gh-"));
try {
const gh = fakeGh(stubDir);
const proc = Bun.spawnSync([
"bun",
"run",
"./bin/gstack-next-version",
"--base",
"main",
"--bump",
"patch",
"--current-version",
"1.6.3.0",
"--workspace-root",
"null", // skip sibling scan in CI
], { timeout: 30_000, env: { ...process.env, PATH: `${stubDir}:${process.env.PATH ?? ""}` } });
const stderr = new TextDecoder().decode(proc.stderr);
expect(proc.exitCode, stderr).toBe(0);
const out = new TextDecoder().decode(proc.stdout);
const parsed = JSON.parse(out);
expect(parsed).toHaveProperty("version");
expect(parseVersion(parsed.version)).not.toBeNull();
expect(parsed).toHaveProperty("bump", "patch");
expect(parsed).toHaveProperty("host");
expect(["github", "gitlab", "unknown"]).toContain(parsed.host);
expect(parsed).toHaveProperty("claimed");
expect(Array.isArray(parsed.claimed)).toBe(true);
expect(parsed).toHaveProperty("siblings");
expect(parsed.siblings).toEqual([]); // --workspace-root null disabled scanning
expect(parsed).toHaveProperty("version_path", "VERSION"); // default when no config + no flag
const remote = execFileSync("git", ["remote", "get-url", "origin"], { cwd: join(import.meta.dir, ".."), encoding: "utf8", timeout: 10_000 }).trim();
expect(gh.calls()).toEqual([
...(remote.includes("github.com") ? [] : [["auth", "status"]]),
["pr", "view", "--json", "number", "-q", ".number"],
["pr", "list", "--state", "open", "--base", "main", "--limit", "200", "--json", "number,headRefName,headRepositoryOwner,url,isDraft"],
["repo", "view", "--json", "owner", "-q", ".owner.login"],
["api", `repos/{owner}/{repo}/contents/VERSION?ref=${encodeURIComponent(CLAIM.branch)}`, "-q", ".content"],
]);
expect(gh.calls().flat().join(" ")).not.toContain("FAKE_GH_UNHANDLED");
expect(stderr).not.toContain("FAKE_GH_UNHANDLED");
expect(gh.glabCalls()).toBe("");
expect(parsed.host).toBe("github");
expect(parsed.offline).toBe(false);
expect(parsed.fallback).toBeNull();
expect(parsed.warnings).toEqual([]);
expect(parsed.claimed).toEqual([CLAIM]); // the canned PR's owner matches the canned repo owner
expect(parsed.version).toBe("1.6.5.0"); // the canned claim on 1.6.4.0 is skipped
} finally {
rmSync(stubDir, { recursive: true, force: true });
}
}, 30_000); // Headroom over the 4-5s wall time of the spawned process under load
test("CLI runs with --version-path and surfaces it in JSON output", async () => {
+10 -3
View File
@@ -18,6 +18,7 @@ async function exercise(mode: 'success' | 'max-turns' | 'first-timeout' | 'secon
const scratch = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'legacy-design-free-')));
const home = path.join(scratch, 'home'); fs.mkdirSync(home);
const env = { PATH: process.env.PATH ?? '', HOME: home, GIT_CONFIG_NOSYSTEM: '1',
GIT_CONFIG_COUNT: '2', GIT_CONFIG_KEY_0: 'maintenance.auto', GIT_CONFIG_VALUE_0: 'false', GIT_CONFIG_KEY_1: 'gc.auto', GIT_CONFIG_VALUE_1: '0',
GIT_CONFIG_GLOBAL: process.platform === 'win32' ? 'NUL' : '/dev/null',
...(process.env.SystemRoot ? { SystemRoot: process.env.SystemRoot } : {}) };
const callbacks: Array<{ name: string; fn: () => Promise<void>; timeout: number }> = [];
@@ -25,6 +26,8 @@ async function exercise(mode: 'success' | 'max-turns' | 'first-timeout' | 'secon
const recordBudgets: number[] = [];
let launched = false, sdkSignal: AbortSignal | undefined;
let inputVerified = false, saved = '', resultError: unknown;
const removals: string[] = [];
const survivors = () => `surviving fixture dirs: ${dirs.filter(dir => fs.existsSync(dir)).join(', ') || '(none)'}; removals: ${removals.join('; ') || '(none)'}`;
const owned = (p: string) => {
const full = path.resolve(p);
expect(full.startsWith(scratch + path.sep)).toBe(true);
@@ -43,13 +46,17 @@ async function exercise(mode: 'success' | 'max-turns' | 'first-timeout' | 'secon
},
copyFileSync(a: string, b: string) { owned(b); fs.copyFileSync(a, b); },
cpSync(a: string, b: string, opts: any) { owned(b); fs.cpSync(a, b, opts); },
rmSync(p: string, opts: any) { owned(p); fs.rmSync(p, opts); },
rmSync(p: string, opts: any) {
owned(p);
try { fs.rmSync(p, opts); } catch (error) { removals.push(`${p}: ${error}`); throw error; }
if (fs.existsSync(p)) removals.push(`${p}: rmSync returned but the path survived`);
},
};
const args = {
ROOT, fs: localFs, path, os: { tmpdir: () => scratch }, expect,
CAPTURE_MS, CAPTURE_LONG_MS, OFFICE_HOURS_BUN_GRACE_MS, resolveEvalModel, runId: 'free-legacy-design',
evalCollector: { addTest(entry: any) {
expect(dirs.every(dir => !fs.existsSync(dir))).toBe(true);
expect(dirs.every(dir => !fs.existsSync(dir)), survivors()).toBe(true);
rows.push(entry);
} },
runRecordedOfficeHoursAttempt: (options: Parameters<typeof runRecordedOfficeHoursAttempt>[0]) => {
@@ -134,7 +141,7 @@ async function exercise(mode: 'success' | 'max-turns' | 'first-timeout' | 'secon
expect(callbacks).toHaveLength(1); expect(callbacks[0]!.timeout).toBe(CAPTURE_LONG_MS);
try { await callbacks[0]!.fn(); } catch (error) { resultError = error; }
expect(inputVerified).toBe(true); expect(dirs).toHaveLength(1);
expect(dirs.every(dir => !fs.existsSync(dir))).toBe(true);
expect(dirs.every(dir => !fs.existsSync(dir)), survivors()).toBe(true);
expect(recordBudgets).toEqual([CAPTURE_LONG_MS - OFFICE_HOURS_BUN_GRACE_MS]);
expect(sdkSignal?.aborted).toBe(true);
return { resultError, rows, saved };
@@ -0,0 +1,106 @@
/**
* The free runner's git auto-maintenance backstop: every shard child gets
* maintenance.auto=false and gc.auto=0 appended to its inherited GIT_CONFIG_*
* pairs, so a fixture commit cannot leave a detached `git maintenance run`
* creating and deleting .git/objects/maintenance.lock under a later listing,
* watch or removal. Fixtures that build their own git env set the keys
* themselves (docs/TESTING_INTERNALS.md#flake-ledger).
*/
import { afterEach, describe, expect, test } from 'bun:test';
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { spawnSync } from 'node:child_process';
import { disableGitAutoMaintenance, runFreeShard } from '../scripts/test-free-shards';
const roots: string[] = [];
afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); });
const scratch = () => {
const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'free-git-maintenance-')));
roots.push(root);
return root;
};
const gitConfigEnv = (env: NodeJS.ProcessEnv) => Object.fromEntries(Object.entries(env).filter(([key]) => key.startsWith('GIT_CONFIG_')).sort());
const cleanGitEnv = (home: string): NodeJS.ProcessEnv => ({
PATH: process.env.PATH ?? '', HOME: home, GIT_CONFIG_NOSYSTEM: '1', GIT_CONFIG_GLOBAL: path.join(home, 'no-global-gitconfig'),
...(process.env.SystemRoot ? { SystemRoot: process.env.SystemRoot } : {}),
});
/** The GIT_CONFIG_* entries the real runFreeShard child received. */
async function childGitConfig(env: NodeJS.ProcessEnv | undefined): Promise<NodeJS.ProcessEnv> {
const out = path.join(scratch(), 'child-env.json');
const probe = `require('fs').writeFileSync(${JSON.stringify(out)}, JSON.stringify(Object.fromEntries(Object.entries(process.env).filter(([k]) => k.startsWith('GIT_CONFIG_')))));`
+ `console.log('Ran 1 tests across 1 files. [1.00ms]');`;
const outcome = await runFreeShard(['probe'], 1, 1, { env, quiet: true, log: () => {},
commandFor: () => ({ command: process.execPath, args: ['-e', probe] }) });
expect(outcome.status).toBe('passed');
return gitConfigEnv(JSON.parse(fs.readFileSync(out, 'utf8')));
}
const OFF = { maintenance: ['maintenance.auto', 'false'], gc: ['gc.auto', '0'] } as const;
const appended = (start: number) => ({
[`GIT_CONFIG_KEY_${start}`]: OFF.maintenance[0], [`GIT_CONFIG_VALUE_${start}`]: OFF.maintenance[1],
[`GIT_CONFIG_KEY_${start + 1}`]: OFF.gc[0], [`GIT_CONFIG_VALUE_${start + 1}`]: OFF.gc[1],
});
describe('free runner: git auto-maintenance off for every shard child', () => {
test('absent and zero inherited GIT_CONFIG_COUNT both start the pairs at index 0', async () => {
const home = scratch();
expect(await childGitConfig(cleanGitEnv(home))).toEqual(gitConfigEnv({ ...cleanGitEnv(home), GIT_CONFIG_COUNT: '2', ...appended(0) }));
expect(await childGitConfig({ ...cleanGitEnv(home), GIT_CONFIG_COUNT: '0' }))
.toEqual(gitConfigEnv({ ...cleanGitEnv(home), GIT_CONFIG_COUNT: '2', ...appended(0) }));
});
test('a nonzero inherited count keeps its pairs and appends after them', async () => {
const env = { ...cleanGitEnv(scratch()), GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: 'user.name', GIT_CONFIG_VALUE_0: 'Fixture' };
expect(await childGitConfig(env)).toEqual(gitConfigEnv({ ...env, GIT_CONFIG_COUNT: '3', ...appended(1) }));
});
test('an explicit options.env and the default process.env are both covered, and neither caller object changes', async () => {
const env = { ...cleanGitEnv(scratch()), GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: 'core.autocrlf', GIT_CONFIG_VALUE_0: 'false' };
const envBefore = structuredClone(env);
const processBefore = gitConfigEnv(process.env);
await childGitConfig(env);
expect(env).toEqual(envBefore);
const inherited = Number(process.env.GIT_CONFIG_COUNT || '0');
expect(await childGitConfig(undefined)).toEqual({ ...processBefore, GIT_CONFIG_COUNT: String(inherited + 2), ...appended(inherited) });
expect(gitConfigEnv(process.env)).toEqual(processBefore);
});
test('a malformed inherited count is passed through untouched and git still rejects it', async () => {
const env = { ...cleanGitEnv(scratch()), GIT_CONFIG_COUNT: 'abc', GIT_CONFIG_KEY_0: 'user.name', GIT_CONFIG_VALUE_0: 'Fixture' };
const child = await childGitConfig(env);
expect(child).toEqual(gitConfigEnv(env));
const result = spawnSync('git', ['config', '--get', 'user.name'], { env: { ...env, ...child }, encoding: 'utf8', timeout: 10_000 });
expect(result.status, result.stderr).not.toBe(0);
expect(result.stderr).toContain('GIT_CONFIG_COUNT');
});
test('a traced commit in the runner child env starts no git maintenance run; the default env does', async () => {
const root = scratch();
const base = cleanGitEnv(root);
const traced = (env: NodeJS.ProcessEnv, name: string) => {
const repo = path.join(root, name);
const git = (...args: string[]) => {
const result = spawnSync('git', args, { cwd: repo, env: { ...env, GIT_TRACE: '1' }, encoding: 'utf8', timeout: 10_000 });
expect(result.status, result.stderr).toBe(0);
return result.stderr;
};
fs.mkdirSync(repo);
git('init', '-q', '-b', 'main');
git('config', 'user.email', 't@t'); git('config', 'user.name', 't'); git('config', 'commit.gpgsign', 'false');
fs.writeFileSync(path.join(repo, 'a.txt'), 'a\n');
git('add', 'a.txt');
return git('commit', '-qm', 'traced').split('\n').filter(line => /maintenance run|gc --auto/.test(line));
};
expect(traced(base, 'defaults').length, 'control: the default env must show the auto-maintenance child').toBeGreaterThan(0);
const runnerEnv = { ...base, ...(await childGitConfig(base)) };
expect(traced(runnerEnv, 'runner')).toEqual([]);
});
test('disableGitAutoMaintenance mutates only the env it is given', () => {
const env: NodeJS.ProcessEnv = { GIT_CONFIG_COUNT: ' 2 ', GIT_CONFIG_KEY_0: 'a.b', GIT_CONFIG_VALUE_0: '1', GIT_CONFIG_KEY_1: 'c.d', GIT_CONFIG_VALUE_1: '2' };
disableGitAutoMaintenance(env);
expect(env).toEqual({ GIT_CONFIG_KEY_0: 'a.b', GIT_CONFIG_VALUE_0: '1', GIT_CONFIG_KEY_1: 'c.d', GIT_CONFIG_VALUE_1: '2', GIT_CONFIG_COUNT: '4', ...appended(2) });
});
});
+20 -3
View File
@@ -149,6 +149,10 @@ describe('enforced checks and the tracking issue (DX-9)', () => {
const failing = unseededCheck(freeRoot(2, 6));
expect(failing).toMatchObject({ status: 'fail', detail: expect.stringContaining('stale seed: 6 unseeded file(s) > 5') });
expect(failing.fix).toContain('bun run test:ubicloud --record-durations');
expect(failing.fix).toContain('UBICLOUD_API_KEY');
expect(failing.fix).toContain('copies scripts/free-test-durations.json back into this checkout');
expect(failing.fix).toContain('leave the refresh to a maintainer');
expect(failing.fix).not.toContain('CI recording');
expect(unseededCheck(freeRoot(2, 5))).toMatchObject({ status: 'pass', detail: expect.stringContaining('5 unseeded file(s) (limit 5)') });
});
@@ -162,7 +166,8 @@ describe('enforced checks and the tracking issue (DX-9)', () => {
});
describe('collectHealth over recorded fixture data', () => {
function fixture() {
function fixture(windowsRuns: unknown[] = [run(70, { event: 'pull_request', head_branch: 'feature' }),
run(71, { event: 'workflow_dispatch', run_started_at: day(2), updated_at: day(1) })]) {
const prTrials = formatTrialOutcomes([trialRecord('gate-a', 'passed', { tier: 'gate', lane: 'gate/pr', cost_usd: 2.5 }),
trialRecord('gate-b', 'passed', { tier: 'gate', lane: 'gate/pr', execution: 'reused', cost_usd: 0 })]);
const census = formatTrialOutcomes([
@@ -184,7 +189,7 @@ describe('collectHealth over recorded fixture data', () => {
[/workflows\/free-tests\.yml\/runs/, () => ({ workflow_runs: [run(60), run(61, { conclusion: 'failure' })] })],
[/runs\/60\/artifacts/, () => ({ artifacts: [{ id: 61, name: 'flake-ledger-2', size_in_bytes: 10 }] })],
[/runs\/61\/artifacts/, () => ({ artifacts: [] })],
[/workflows\/windows-free-tests\.yml\/runs/, () => ({ workflow_runs: [run(70)] })],
[/workflows\/windows-free-tests\.yml\/runs/, () => ({ workflow_runs: windowsRuns })],
], {
11: { 'manifest.json': JSON.stringify({ prCoverage: { mode: 'full-fallback' } }) },
12: { 'trial-outcomes.jsonl': prTrials },
@@ -207,7 +212,7 @@ describe('collectHealth over recorded fixture data', () => {
expect(value('census-redispatch').value).toBe('0 of 1 census(es) re-dispatched; longest paid slice job 20.0 min');
expect(value('quarantine-evidence').value).toContain('insufficient evidence (not the same as healthy)');
expect(value('free-flaky-pass-rate').value).toStartWith('50.0% (1/2 main runs');
expect(value('free-wall').value).toBe('free-tests 0.0 min (1 runs); windows-free-tests 0.0 min (1 runs)');
expect(value('free-wall').value).toBe('free-tests 0.0 min (1 runs); windows-free-tests (successful PR runs) 0.0 min (1 runs)');
expect(value('free-serial').value).toBe('3 s over 3 files');
for (const entry of report.metrics.filter(entry => entry.id !== 'net-complexity')) {
expect(entry.rule.length).toBeGreaterThan(10);
@@ -218,6 +223,18 @@ describe('collectHealth over recorded fixture data', () => {
expect(formatHealth(report)).toContain('rule: share of evals.yml pull_request runs');
});
test('the Windows wall counts only successful pull_request runs and says so when there are none', () => {
const wall = (windowsRuns: unknown[]) => collectHealth({ client: fixture(windowsRuns), repo: 'o/r', now: NOW, sinceDays: 7, cacheDir: tmp('health-cache-'), root: freeRoot(3, 0) })
.metrics.find(entry => entry.id === 'free-wall')!;
const pr = (id: number, minutes: number, extra: Record<string, unknown> = {}) => run(id, { event: 'pull_request', head_branch: `pr-${id}`,
run_started_at: day(1), updated_at: new Date(Date.parse(day(1)) + minutes * 60_000).toISOString(), ...extra });
expect(wall([pr(70, 6), pr(71, 7), pr(74, 8), pr(72, 30, { conclusion: 'failure' }), run(73, { event: 'workflow_dispatch', run_started_at: day(3), updated_at: day(1) })]).value)
.toBe('free-tests 0.0 min (1 runs); windows-free-tests (successful PR runs) 7.0 min (3 runs)');
expect(wall([run(73, { event: 'workflow_dispatch', run_started_at: day(3), updated_at: day(1) })]).value)
.toBe('free-tests 0.0 min (1 runs); windows-free-tests (successful PR runs) no successful PR runs in window');
expect(wall([]).rule).toContain('workflow_dispatch duration recordings excluded');
});
test('a rate-limited API degrades every remote metric to a named unavailable reason, never 0, and keeps the issue open', () => {
const client = fakeClient([[/./, () => { throw new Error('gh api failed: HTTP 403: API rate limit exceeded for installation ID 1'); }]]);
const report = collectHealth({ client, repo: 'o/r', now: NOW, sinceDays: 7, cacheDir: tmp('health-cache-'), root: freeRoot(3, 0) });
+60
View File
@@ -222,4 +222,64 @@ describe('gstack-timeline-read', () => {
});
}
}
// --since/--branch used to read an unbound $2 (exit 1), and an unparseable
// --since silently returned unfiltered history.
const badFilters: Array<[string, string[], string]> = [
['--since missing value', ['--since'], '--since must be a relative time like "7 days ago" (got no value)'],
['--since empty value', ['--since', ''], '--since must be a relative time like "7 days ago" (got an empty value)'],
['--since followed by a flag', ['--since', '--limit', '5'], "(got flag '--limit')"],
['--since yesterday', ['--since', 'yesterday'], "--since must be a relative time like \"7 days ago\" (got 'yesterday')"],
['--since 2 weeks', ['--since', '2 weeks'], "(got '2 weeks')"],
['--since 7days', ['--since', '7days'], "(got '7days')"],
['--since impossible month', ['--since', '2026-13-01'], "(got '2026-13-01')"],
['--since unanchored', ['--since', 'about 7 days ago, roughly'], "(got 'about 7 days ago, roughly')"],
['--branch missing value', ['--branch'], '--branch must be a branch name (got no value)'],
['--branch empty value', ['--branch', ''], '--branch must be a branch name (got an empty value)'],
['--branch followed by a flag', ['--branch', '--limit', '5'], "--branch must be a branch name (got flag '--limit')"],
['unknown flag', ['--bogus'], "unknown argument '--bogus'"],
['stray argument', ['stray'], "unknown argument 'stray'"],
];
for (const history of ['empty', 'populated'] as const) {
for (const [label, args, message] of badFilters) {
test(`${label} exits 2 before any read on ${history} history`, () => {
if (history === 'populated') {
runLog(JSON.stringify({ skill: 'review', event: 'completed', branch: 'main', ts: '2026-03-28T10:00:00Z' }));
}
const result = runReadStatus(args);
expect(result.status, result.stderr).toBe(2);
expect(result.stdout).toBe('');
expect(result.stderr).toContain(message);
if (label.startsWith('--since')) expect(result.stderr).toContain('accepted: "<N> minute|hour|day|week|month[s] ago", "<N>h|d|w", or an ISO date');
if (label === 'unknown flag' || label === 'stray argument') expect(result.stderr).toContain('Usage: gstack-timeline-read');
});
}
test(`--help prints the usage line and exits 0 on ${history} history`, () => {
if (history === 'populated') {
runLog(JSON.stringify({ skill: 'review', event: 'completed', branch: 'main', ts: '2026-03-28T10:00:00Z' }));
}
const result = runReadStatus(['--help']);
expect(result.status).toBe(0);
expect(result.stdout.trim()).toBe('Usage: gstack-timeline-read [--since "7 days ago"|7d|YYYY-MM-DD] [--limit N] [--branch NAME]');
});
const yesterday = new Date(Date.now() - 86_400_000).toISOString().slice(0, 10);
for (const since of ['7 days ago', '3 Hours ago', '7d', '24h', '1w', yesterday, `${yesterday}T00:00:00Z`]) {
test(`--since '${since}' keeps recent events and drops old ones on ${history} history`, () => {
if (history === 'populated') {
runLog(JSON.stringify({ skill: 'review', event: 'completed', branch: 'main', ts: '2020-01-01T10:00:00Z' }));
runLog(JSON.stringify({ skill: 'ship', event: 'completed', branch: 'main', ts: new Date(Date.now() - 3_600_000).toISOString() }));
}
const result = runReadStatus(['--since', since]);
expect(result.status, result.stderr).toBe(0);
expect(result.stderr).toBe('');
if (history === 'empty') expect(result.stdout).toBe('');
else {
expect(result.stdout).toContain('/ship completed');
expect(result.stdout).not.toContain('/review completed');
}
});
}
}
});