fix(memory-helpers): a slow gitleaks probe no longer disables secret scanning

`gitleaksAvailable()` cached every failure the same way, so a 2s timeout on
`gitleaks version` was recorded as "the binary is absent" for the rest of the
process. One busy moment and the whole ingest ran unscanned behind a single
stderr line — a fail-open outcome decided by machine load rather than by
anything about the machine's setup. The caller only acts on
`scanner === "gitleaks"`, so every later file was written with no scan and no
second warning.

The probe now classifies three outcomes. ENOENT (and a present-but-unusable
binary: bad exit, EACCES) stays cached — that is a fact about the box, and
re-probing it per file would be waste. A timeout gets one retry on a 10s
budget, and if that also expires nothing is cached: the file is reported
unscanned, the warning says so in those words, and the next file probes again.

Observed under the 7-way sharded free-test runner, where spawning a shell
script inside a temp bin dir took longer than the 2s budget.

Tests: the retry path, the no-cache-on-timeout path (the second call must
re-probe), and the cached-absent path. The fake gitleaks hangs for 30s rather
than racing a short sleep against a short budget, and the budgets are chosen so
load cannot flip an outcome: 30s where the retry MUST answer, 800ms where the
probe MUST expire. An earlier draft used 1s/5s and flaked under the same shard
runner this commit is about. The existing probe test pinned `detect` to
calls[1], which a retry breaks; it now asserts the order instead of the index.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111Mq3JGwZDcstn5wYcbhSw
This commit is contained in:
Denis Zjukow
2026-08-31 20:56:32 +00:00
committed by Garry Tan
co-authored by Claude Opus 5
parent 5424ac5fe0
commit 8c7ff15fc4
2 changed files with 221 additions and 8 deletions
+86 -7
View File
@@ -126,25 +126,88 @@ export function canonicalizeRemote(url: string | null | undefined): string {
// ── Public: secretScanFile (gitleaks wrapper) ─────────────────────────────
let _gitleaksAvailability: boolean | null = null;
let _gitleaksWarned = false;
function gitleaksAvailable(): boolean {
if (_gitleaksAvailability !== null) return _gitleaksAvailability;
// Probe budgets. The first is short because the common answers (a real
// gitleaks, or ENOENT) are both immediate; the second is generous because by
// then we know the box is busy, not that the binary is absent.
const GITLEAKS_PROBE_MS = 2_000;
const GITLEAKS_RETRY_MS = 10_000;
let _probeMs = GITLEAKS_PROBE_MS;
let _retryMs = GITLEAKS_RETRY_MS;
/**
* Probe outcome. "slow" is the load case: the binary may well be installed,
* the machine just did not get around to answering. It is deliberately NOT
* folded into "absent" — see gitleaksAvailable().
*/
type GitleaksProbe = "ok" | "absent" | "slow";
function probeGitleaks(timeoutMs: number): GitleaksProbe {
try {
execFileSync("gitleaks", ["version"], {
env: process.env,
stdio: "ignore",
timeout: 2_000,
timeout: timeoutMs,
});
return "ok";
} catch (err) {
const e = err as NodeJS.ErrnoException & { killed?: boolean; signal?: string };
if (e?.code === "ENOENT") return "absent";
// execFileSync kills the child when the budget runs out: `killed` with a
// SIGTERM on POSIX, ETIMEDOUT on runtimes that surface errno instead.
if (e?.killed === true || e?.signal === "SIGTERM" || e?.code === "ETIMEDOUT") {
return "slow";
}
// Present but unusable (non-zero exit, EACCES). Practically the same as
// absent, and equally permanent for this process.
return "absent";
}
}
/**
* Is gitleaks usable? Answers are cached for the process — EXCEPT a timeout.
*
* Caching a timeout was a fail-open bug: one busy moment (observed under the
* 7-way sharded test runner, where spawning a shell script took over 2s) set
* availability to false for the whole run, and every later file was ingested
* unscanned behind a single stderr line. A missing binary is a fact and stays
* cached; a slow answer is a condition and gets retried on the next file.
*/
function gitleaksAvailable(): boolean {
if (_gitleaksAvailability !== null) return _gitleaksAvailability;
let probe = probeGitleaks(_probeMs);
if (probe === "slow") probe = probeGitleaks(_retryMs);
if (probe === "ok") {
_gitleaksAvailability = true;
} catch {
_gitleaksAvailability = false;
// Only warn once per process — Lane E will vendor the binary.
return true;
}
if (probe === "slow") {
// No cache write: leave the question open for the next call.
if (!_gitleaksWarned) {
_gitleaksWarned = true;
process.stderr.write(
"[gstack-memory-helpers] gitleaks did not answer in " +
`${Math.round((_probeMs + _retryMs) / 1000)}s (machine under load); ` +
"this file goes unscanned and the probe retries on the next one.\n"
);
}
return false;
}
_gitleaksAvailability = false;
// Only warn once per process — Lane E will vendor the binary.
if (!_gitleaksWarned) {
_gitleaksWarned = true;
process.stderr.write(
"[gstack-memory-helpers] gitleaks not in PATH; secret scanning disabled. " +
"Run /setup-gbrain to install (or `brew install gitleaks`).\n"
);
}
return _gitleaksAvailability;
return false;
}
/**
@@ -513,4 +576,20 @@ function logErrorContext(entry: ErrorContextEntry): void {
// Test-only export for resetting the gitleaks availability cache between tests.
export function _resetGitleaksAvailabilityCache(): void {
_gitleaksAvailability = null;
_gitleaksWarned = false;
_probeMs = GITLEAKS_PROBE_MS;
_retryMs = GITLEAKS_RETRY_MS;
}
// Test-only: shrink the probe budgets so the slow path can be exercised
// without a multi-second sleep in the suite. Reset restores the defaults.
export function _setGitleaksProbeTimeouts(first: number, second: number): void {
_probeMs = first;
_retryMs = second;
}
// Test-only: read the cache without triggering a probe. `null` means the
// question is still open — which is the whole point of the timeout path.
export function _gitleaksCacheState(): boolean | null {
return _gitleaksAvailability;
}