Files
gstack/lib/gstack-memory-helpers.ts
T
01593aa67c v1.91.2.0 fix: consolidate gstack reliability wave (#2959)
* fix(memory-ingest): --scan-secrets scans the rendered page and fails closed

--scan-secrets ran gitleaks on the raw transcript .jsonl, then imported a
page rendered from it. gitleaks' assignment rules don't match across a
JSON-escaped quote (KEY=\"v\" on disk), so a secret the rendered page
shows as KEY="v" was imported unflagged. And the gate skipped a file only
on scanner "gitleaks" with findings, so a scan that errored (non-zero
exit, 16MB maxBuffer overflow on a file with many findings, unparseable
report) or could not run (gitleaks missing, slow-probe cooldown) imported
the file unscanned.

Scan the rendered page body, the exact bytes writeStaged() writes, via a
new secretScanText() helper, and skip the file whenever the scan did not
complete. Skipped files stay out of the state file, so the next run
retries them. Reword the helper warnings and setup-gbrain/memory.md,
which described the fail-open as intended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(test): reconcile Bun failure markers and footer counts

* fix(sync-gbrain): verify source-scoped reads without mutation

* fix(test): recognize grounded TTHW target choices structurally

* fix(aside): make the readiness probe work under zsh and report why it failed

The probe built its deadline into `_T` and expanded it unquoted, so
`$_T aside repl …` only worked in a shell that word-splits. zsh does not: it
looked for a command literally named "gtimeout 30", the probe answered
ASIDE_NOT_RUNNING with Aside installed and ready, and every browsing skill
fell back to the bundled Chromium in silence. zsh is the macOS default and
Aside is macOS-only, so on a stock Mac the probe could never report READY.

The deadline becomes a function, `_gs_d`. It receives the command as "$@",
already split, so sh, bash and zsh all behave the same, and the gtimeout →
timeout → perl alarm chain is unchanged. A 4th arm runs the call unbounded
when none of the three is present, which is what the empty `_T` did before.
Not `eval`: it re-parses the string, so the parens and `;` of the perl arm
become syntax and that arm dies in bash *and* zsh — on a stock Mac, the arm
that actually runs.

On failure the probe now prints the CLI's reason after ASIDE_NOT_RUNNING:,
the shape gstack-render already uses: the first line that starts with a
capital letter, i.e. the CLI's own sentence or Node's `Error:` line below its
loader frame. "Not running" covers states with different fixes — no window
open for the profile, a NODE_OPTIONS preload that kills the CLI — and a bare
verdict sent all of them to "open the Aside app". The BROWSER SETUP prose
quotes that reason before asking the user to open the app.

The text pin asserted the broken invocation verbatim, so it now pins the
function and asserts neither `$_T aside repl` nor an eval form comes back. A
second test executes the rendered probe in sh, bash and zsh on each of the
four deadline arms with stubbed binaries on a narrowed PATH, plus two failing
CLIs: one that prints its own sentence, one that crashes like Node with the
useful line below the frame.

The deadline function costs zero bytes against the lines it replaces; the
reason costs 53 per copy of the probe (44 where the reworded BROWSER SETUP
line gives 9 back). That moves four guards by the measured amount:
plan-devex-review's skeleton cap to 68,550 (measured 68,544), plan-ceo-review's
skeleton cap to 80,150 (measured 80,111) and union ratio to 1.081 (measured
1.0803), and plan-eng-review's union ratio to 1.151 (measured 1.1504).

Fixes #2842, #2941.

* Clarify engineering review startup and decision flow

* Fix Windows readiness fixture PATH and command shim

* fix(test): recognize grounded TTHW target choices structurally

* Clarify engineering review startup and decision flow

* fix(test): restrict QA-only fixture tools to its no-Edit contract

* v1.90.0.0 fix(sync-gbrain): guard readiness verdicts and refresh metadata

* fix(browse): validate canonical upload targets

* fix(gbrain): classify structured PGLite busy response

* fix(browse): preserve native extension runtime APIs

* Fix displayless browser handoff ownership

* Accept unique installed autoplan methodology aliases

* fix(skills): preserve positional literals during installation

* fix(browse): checksum installer contents through stdin

* fix(test): normalize Windows checksum fixture paths

* test: emulate unavailable shasum in Windows checksum fixture

* fix(investigate): preserve owned freeze lifecycle

* fix(review): preserve N+1 retry and Red Team completion

* fix: bound Aside readiness and preserve safe fallback

* test: exercise setup and Chromium on native ARM

* fix: preserve install ownership and ARM browser selection

* Fix gbrain ingest scan boundaries and seed observation

* Refresh managed ship hooks and supervise expanded paid census

* Reject resumed gbrain pages excluded by current policy

* Recover zombie agent locks safely and enable CI Python venv

* Repair paid actor declarations and Aside pitch assertions

* Bump consolidated wave to next free minor release

* Clarify CEO review admin choices and option tradeoffs

* Preserve CEO mode handoff anchors in clarified workflow

* Make Windows portability fixtures use shell-native paths

* Restore ARM Bun alias and clarify ship review gates

* Refresh ship workflow golden snapshots

* Fix Windows DX documentation controls without piped stdin

* Decode Codex child pipes without Bun's encoded-stream stall

* Bound DX pre-review audit before product questions

* Clarify trusted review-start read in paid revalidation

* Bump consolidated wave to next free minor release

* Clarify CEO review admin choices and option tradeoffs

* Preserve CEO mode handoff anchors in clarified workflow

* Make Windows portability fixtures use shell-native paths

* Restore ARM Bun alias and clarify ship review gates

* Refresh ship workflow golden snapshots

* Fix Windows DX documentation controls without piped stdin

* Decode Codex child pipes without Bun's encoded-stream stall

* Bound DX pre-review audit before product questions

* Clarify trusted review-start read in paid revalidation

* Reconcile new main planning flow and paid judge census

* fix: reconcile rebased planning and source-bound validation

* test: pin cookie workflow judge to scored Sonnet model

* fix: keep terminal agent boot out of module imports

* fix: preserve pending-question uncertainty in engineering review

* fix: stabilize Windows reliability-wave fixtures

* fix: clarify design consultation research workflow

* fix: preserve independent design consultation inputs

* fix: resolve design taste scope and browser research guidance

* fix: make consultation opt-in preflight unambiguous

* test: await native Edge owner readiness or terminal result

---------

Co-authored-by: Bruce Krysiak <brucek@alum.mit.edu>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Antonio Vitalic <antoninte99@gmail.com>
2026-09-26 18:57:53 -04:00

655 lines
24 KiB
TypeScript

/**
* gstack-memory-helpers — shared helpers for the V1 memory ingest + retrieval pipeline.
*
* Imported by:
* - bin/gstack-memory-ingest.ts (Lane A)
* - bin/gstack-gbrain-sync.ts (Lane B)
* - bin/gstack-brain-context-load.ts (Lane C)
* - scripts/gen-skill-docs.ts (manifest validation)
*
* Design refs in the plan:
* §"Eng review additions" — DRY refactor (Section 1A)
* §"V1 final scope clarification" — schema_version: 1 standardization (Section 2A)
* ED1 — engine-tier cache lives in ~/.gstack/.gbrain-engine-cache.json (60s TTL)
*
* NOTE: secretScanFile() currently shells out to `gitleaks` from PATH; the vendored
* binary install is part of Lane E (setup-gbrain). When gitleaks is missing, the
* helper warns once and returns an empty findings list with scanner="missing".
* An empty list means "clean" only when scanner === "gitleaks" — callers that
* gate writes on a scan must treat "missing" and "error" as unscanned.
*/
import { existsSync, readFileSync, writeFileSync, mkdirSync, mkdtempSync, rmSync, statSync } from "fs";
import { appendJsonl } from "./jsonl-store";
import { gbrainConfigDir, isExecTimeout } from "./gbrain-exec";
import { dirname, join } from "path";
import { execFileSync } from "child_process";
import { homedir, tmpdir } from "os";
// ── Types ──────────────────────────────────────────────────────────────────
export interface SecretFinding {
rule_id: string;
description: string;
line: number;
redacted_match: string;
}
export interface SecretScanResult {
scanned: boolean;
findings: SecretFinding[];
scanner: "gitleaks" | "missing" | "error";
}
export type EngineTier = "pglite" | "supabase" | "unknown";
export interface EngineDetect {
engine: EngineTier;
supabase_url?: string;
detected_at: number;
schema_version: 1;
}
export interface GbrainManifestQuery {
id: string;
kind: "vector" | "list" | "filesystem";
render_as: string;
// kind=vector
query?: string;
// kind=list
filter?: Record<string, unknown>;
sort?: string;
// kind=filesystem
glob?: string;
tail?: number;
// common
limit?: number;
}
export interface GbrainManifest {
schema: number; // gbrain.schema in frontmatter; V1 = 1
context_queries: GbrainManifestQuery[];
}
export interface ErrorContextEntry {
ts: string;
op: string;
duration_ms: number;
outcome: "ok" | "error";
error?: string;
schema_version: 1;
last_writer: string;
}
// ── Public: canonicalizeRemote ────────────────────────────────────────────
/**
* Normalize a git remote URL to a canonical form: `host/org/repo` (no scheme,
* no trailing `.git`). Used as the dedup key for cross-Mac transcript routing
* (per ED1 — gbrain-side session_id dedup uses repo as a tag).
*
* Examples:
* https://github.com/garrytan/gstack.git → github.com/garrytan/gstack
* git@github.com:garrytan/gstack.git → github.com/garrytan/gstack
* ssh://git@gitlab.com/foo/bar → gitlab.com/foo/bar
* (empty / null) → ""
*/
export function canonicalizeRemote(url: string | null | undefined): string {
if (!url) return "";
let s = url.trim();
if (!s) return "";
// strip surrounding quotes that some configs add
s = s.replace(/^['"]|['"]$/g, "");
// git@host:path/repo → host/path/repo
const scpMatch = s.match(/^[^@\s]+@([^:]+):(.+)$/);
if (scpMatch) {
s = `${scpMatch[1]}/${scpMatch[2]}`;
} else {
// strip scheme (https://, ssh://, git://, http://)
s = s.replace(/^[a-z][a-z0-9+.-]*:\/\//i, "");
// strip user@ prefix on URL-style remotes
s = s.replace(/^[^@\/]+@/, "");
}
// strip trailing slash(es) first, so a URL written with a trailing slash
// still matches the `.git$` suffix below (e.g. ".../repo.git/" must
// canonicalize to ".../repo", not ".../repo.git").
s = s.replace(/\/+$/, "");
// strip trailing .git
s = s.replace(/\.git$/i, "");
// re-strip trailing slash(es): a path remote ending in a `.git` directory
// component ("/repo/.git") exposes a new trailing slash once `.git` is
// stripped, which would split the repo into a second identity.
s = s.replace(/\/+$/, "");
// collapse multiple slashes (after path normalization)
s = s.replace(/\/{2,}/g, "/");
return s.toLowerCase();
}
// ── Public: secretScanFile (gitleaks wrapper) ─────────────────────────────
let _gitleaksAvailability: boolean | null = null;
// Two flags, not one: "slow" and "absent" are different messages with
// different remediations, and a slow warning early in a run must not
// suppress the permanent "not in PATH; scanning disabled" warning later.
let _gitleaksSlowWarned = false;
let _gitleaksAbsentWarned = false;
// Per-run cooldown: retrying a slow probe on EVERY file re-pays up to
// probe+retry (12s default) per file — an 887-file ingest on a loaded box
// spent hours asking the same slow question. After this many consecutive
// slow answers the run stops probing; the availability cache is still never
// written (slow != absent — the next PROCESS probes fresh).
const GITLEAKS_SLOW_PROBE_LIMIT = 3;
let _gitleaksConsecutiveSlow = 0;
let _gitleaksCooldownWarned = false;
// Probe budgets. The first is short because the common answers (a real
// gitleaks, or ENOENT) are both immediate; the second is generous because by
// then we know the box is busy, not that the binary is absent.
const GITLEAKS_PROBE_MS = 2_000;
const GITLEAKS_RETRY_MS = 10_000;
let _probeMs = GITLEAKS_PROBE_MS;
let _retryMs = GITLEAKS_RETRY_MS;
/**
* Probe outcome. "slow" is the load case: the binary may well be installed,
* the machine just did not get around to answering. It is deliberately NOT
* folded into "absent" — see gitleaksAvailable().
*/
type GitleaksProbe = "ok" | "absent" | "slow";
function probeGitleaks(timeoutMs: number): GitleaksProbe {
try {
execFileSync("gitleaks", ["version"], {
env: process.env,
stdio: "ignore",
timeout: timeoutMs,
});
return "ok";
} catch (err) {
const e = err as NodeJS.ErrnoException;
if (e?.code === "ENOENT") return "absent";
if (isExecTimeout(err)) return "slow";
// Present but unusable (non-zero exit, EACCES). Practically the same as
// absent, and equally permanent for this process.
return "absent";
}
}
/**
* Is gitleaks usable? Answers are cached for the process — EXCEPT a timeout.
*
* Caching a timeout was a fail-open bug: one busy moment (observed under the
* 7-way sharded test runner, where spawning a shell script took over 2s) set
* availability to false for the whole run, and every later file was ingested
* unscanned behind a single stderr line. A missing binary is a fact and stays
* cached; a slow answer is a condition and gets retried on the next file.
*/
function gitleaksAvailable(): boolean {
if (_gitleaksAvailability !== null) return _gitleaksAvailability;
if (_gitleaksConsecutiveSlow >= GITLEAKS_SLOW_PROBE_LIMIT) {
if (!_gitleaksCooldownWarned) {
_gitleaksCooldownWarned = true;
process.stderr.write(
"[gstack-memory-helpers] gitleaks did not answer in " +
`${GITLEAKS_SLOW_PROBE_LIMIT} consecutive probes; skipping the probe ` +
"for the rest of this run — remaining files cannot be scanned. Re-run " +
"when the machine is less loaded to scan them.\n"
);
}
return false;
}
let probe = probeGitleaks(_probeMs);
if (probe === "slow") probe = probeGitleaks(_retryMs);
if (probe === "ok") {
_gitleaksAvailability = true;
_gitleaksConsecutiveSlow = 0;
return true;
}
if (probe === "slow") {
// No cache write: leave the question open for the next call — but count
// it, so a persistently loaded box stops paying probe+retry per file.
_gitleaksConsecutiveSlow++;
if (!_gitleaksSlowWarned) {
_gitleaksSlowWarned = true;
process.stderr.write(
"[gstack-memory-helpers] gitleaks did not answer in " +
`${Math.round((_probeMs + _retryMs) / 1000)}s (machine under load); ` +
"this file could not be scanned and the probe retries on the next one.\n"
);
}
return false;
}
_gitleaksAvailability = false;
// Only warn once per process — Lane E will vendor the binary.
if (!_gitleaksAbsentWarned) {
_gitleaksAbsentWarned = true;
process.stderr.write(
"[gstack-memory-helpers] gitleaks not in PATH; files cannot be secret-scanned. " +
"Run /setup-gbrain to install (or `brew install gitleaks`).\n"
);
}
return false;
}
/**
* Scan a file for embedded secrets using gitleaks. Returns findings list
* (empty if clean). When gitleaks is not in PATH, returns scanned=false with
* scanner="missing" — caller decides whether to skip the file or proceed.
*
* Per D19: gitleaks runs at ingest time before any put_page / put_file write.
* Replaces the inadequate regex scanner in bin/gstack-brain-sync (which only
* applies to staged git diffs).
*/
export function secretScanFile(path: string): SecretScanResult {
if (!existsSync(path)) {
return { scanned: false, findings: [], scanner: "error" };
}
if (!gitleaksAvailable()) {
return { scanned: false, findings: [], scanner: "missing" };
}
let dir: string | undefined;
try {
dir = mkdtempSync(join(tmpdir(), "gstack-secret-report-"));
const report = join(dir, "report.json");
writeFileSync(report, "", { mode: 0o600, flag: "wx" });
const maxReportBytes = 16 * 1024 * 1024;
execFileSync(
"gitleaks",
["detect", "--no-git", "--source", path, "--report-format", "json", "--report-path", report, "--exit-code", "0"],
{ env: process.env, stdio: "ignore", timeout: 60_000, killSignal: "SIGKILL" }
);
if (statSync(report).size > maxReportBytes) {
return { scanned: false, findings: [], scanner: "error" };
}
const parsed = JSON.parse(readFileSync(report, "utf-8"));
if (!Array.isArray(parsed) || !parsed.every((f) =>
f && typeof f.RuleID === "string" && f.RuleID.length > 0 &&
typeof f.Description === "string" && Number.isInteger(f.StartLine) && f.StartLine > 0 &&
(f.Secret === undefined || typeof f.Secret === "string") &&
(f.Match === undefined || typeof f.Match === "string")
)) {
return { scanned: false, findings: [], scanner: "error" };
}
const findings: SecretFinding[] = parsed.map((f) => ({
rule_id: f.RuleID,
description: f.Description,
line: f.StartLine,
redacted_match: redactMatch(f.Secret || f.Match || ""),
}));
return { scanned: true, findings, scanner: "gitleaks" };
} catch (err) {
return {
scanned: false,
findings: [],
scanner: "error",
};
} finally {
if (dir) rmSync(dir, { recursive: true, force: true });
}
}
/**
* Scan in-memory text — e.g. a rendered page body — by writing it to a
* private temp file and running secretScanFile() on it. Scan the exact bytes
* you are about to write, not the file they were rendered from: gitleaks'
* assignment rules don't match across a JSON-escaped quote, so a .jsonl
* transcript line holding `KEY=\"value\"` scans clean while the rendered
* page's `KEY="value"` is a finding. A temp file that can't be written
* returns scanner="error", never an empty "clean" result.
*/
export function secretScanText(text: string): SecretScanResult {
let dir: string | undefined;
try {
dir = mkdtempSync(join(tmpdir(), "gstack-secret-scan-"));
const file = join(dir, "page.md");
writeFileSync(file, text, { encoding: "utf-8", mode: 0o600 });
return secretScanFile(file);
} catch {
return { scanned: false, findings: [], scanner: "error" };
} finally {
if (dir) rmSync(dir, { recursive: true, force: true });
}
}
function redactMatch(s: string): string {
if (!s) return "";
if (s.length <= 8) return "[REDACTED]";
return `${s.slice(0, 4)}...${s.slice(-4)}`;
}
// ── Public: detectEngineTier (cached) ─────────────────────────────────────
const ENGINE_CACHE_TTL_MS = 60 * 1000;
function gstackHome(): string {
return process.env.GSTACK_HOME || join(homedir(), ".gstack");
}
function engineCachePath(): string {
return join(gstackHome(), ".gbrain-engine-cache.json");
}
function errorLogPath(): string {
return join(gstackHome(), ".gbrain-errors.jsonl");
}
/**
* Detect which gbrain engine is active (PGLite vs Supabase) and cache the
* answer for 60s in ~/.gstack/.gbrain-engine-cache.json. Caching avoids
* fork+exec'ing `gbrain doctor --json` on every skill start.
*
* Per ED1 (state files local-only): this cache is gitignored from the brain
* repo. Per Section 2A: schema_version: 1 + last_writer field for forensic
* tracing.
*/
export function detectEngineTier(): EngineDetect {
// Try cache first
if (existsSync(engineCachePath())) {
try {
const stat = statSync(engineCachePath());
const ageMs = Date.now() - stat.mtimeMs;
if (ageMs < ENGINE_CACHE_TTL_MS) {
const cached = JSON.parse(readFileSync(engineCachePath(), "utf-8")) as EngineDetect;
if (cached.schema_version === 1) return cached;
}
} catch {
// Cache corrupt; fall through to fresh detect.
}
}
const fresh = freshDetectEngineTier();
try {
mkdirSync(dirname(engineCachePath()), { recursive: true });
writeFileSync(
engineCachePath(),
JSON.stringify({ ...fresh, last_writer: "gstack-memory-helpers.detectEngineTier" }, null, 2),
"utf-8"
);
} catch {
// Cache write failure is non-fatal.
}
return fresh;
}
// Returns gbrain's config.json path, honoring GBRAIN_HOME env var with a
// fallback to ~/.gbrain. Resolution matches gbrain's own configDir()
// contract (#2521): GBRAIN_HOME is a parent dir, `.gbrain` is appended.
// gbrain >=0.25 dropped the top-level `engine` field
// from doctor output, so this file is the only reliable source for engine
// detection on that version. See #1415.
function gbrainConfigPath(): string {
return join(gbrainConfigDir(process.env), "config.json");
}
// Best-effort JSONL append to ~/.gstack/.gbrain-errors.jsonl. Never throws.
function logGbrainError(kind: string, detail: string): void {
try {
const path = errorLogPath();
mkdirSync(dirname(path), { recursive: true });
appendJsonl(path, { ts: new Date().toISOString(), kind, detail: detail.slice(0, 500) });
} catch { /* logging is best-effort */ }
}
function freshDetectEngineTier(): EngineDetect {
const now = Date.now();
let parsed: Record<string, unknown> | null = null;
// execFileSync (not execSync) avoids shell redirection — portable to
// environments where `2>/dev/null` is bash-specific. The stdio array
// suppresses stderr without invoking a shell.
try {
const out = execFileSync("gbrain", ["doctor", "--json", "--fast"], {
encoding: "utf-8",
timeout: 5000,
stdio: ["ignore", "pipe", "ignore"],
});
parsed = JSON.parse(out);
} catch (err: unknown) {
// execFileSync throws on non-zero exit; stdout is still on the error
// object. gbrain doctor exits 1 whenever health_score < 100, which is
// essentially always on fresh installs (resolver_health warnings are
// normal). Recover stdout and re-parse. See #1415.
try {
const stdout = (err as { stdout?: Buffer | string })?.stdout ?? "";
const stdoutStr = typeof stdout === "string" ? stdout : stdout.toString("utf-8");
if (stdoutStr) parsed = JSON.parse(stdoutStr);
} catch (parseErr) {
logGbrainError("doctor_parse_failure", String(parseErr));
}
}
let engine: EngineTier =
parsed?.engine === "supabase" ? "supabase" :
parsed?.engine === "pglite" ? "pglite" : "unknown";
// gbrain >=0.25 ships schema_version:2 doctor output which dropped the
// top-level `engine` field. Fall back to gbrain's config.json (respects
// GBRAIN_HOME). "supabase" here means "remote postgres" — gbrain config
// uses engine:"postgres" for real Supabase AND any other remote postgres
// (e.g. local-postgres-for-testing). Downstream sync code treats them the
// same, so the label compression is intentional.
if (engine === "unknown") {
try {
const cfg = JSON.parse(readFileSync(gbrainConfigPath(), "utf-8"));
if (cfg?.engine === "pglite") engine = "pglite";
else if (cfg?.engine === "postgres" || cfg?.database_url) engine = "supabase";
} catch (cfgErr) {
logGbrainError("config_read_failure", String(cfgErr));
}
}
return {
engine,
supabase_url: parsed?.supabase_url as string | undefined,
detected_at: now,
schema_version: 1,
};
}
// ── Public: parseSkillManifest ────────────────────────────────────────────
/**
* Parse the `gbrain:` section out of a SKILL.md.tmpl frontmatter block.
* Returns null if no manifest is declared OR if the file has no frontmatter.
*
* Schema validation (full kind/required-fields check) lives in
* scripts/gen-skill-docs.ts and runs at generation time. This parser is the
* runtime read path used by gstack-brain-context-load; it tolerates extra
* fields and relies on validation having already happened upstream.
*/
export function parseSkillManifest(skillFilePath: string): GbrainManifest | null {
if (!existsSync(skillFilePath)) return null;
const content = readFileSync(skillFilePath, "utf-8");
const frontmatter = extractFrontmatter(content);
if (!frontmatter) return null;
const gbrain = extractGbrainBlock(frontmatter);
if (!gbrain) return null;
return gbrain;
}
function extractFrontmatter(content: string): string | null {
// Supports both `---\n...\n---` (YAML) and `+++\n...\n+++` (TOML, rare).
const yamlMatch = content.match(/^---\s*\n([\s\S]*?)\n---\s*\n/);
if (yamlMatch) return yamlMatch[1];
return null;
}
function extractGbrainBlock(frontmatter: string): GbrainManifest | null {
// Naive YAML extraction — finds the `gbrain:` key and parses its sub-tree.
// Real YAML parsing avoided to keep zero-deps; gen-skill-docs validates the
// shape strictly at build time.
const lines = frontmatter.split("\n");
const start = lines.findIndex((l) => /^gbrain\s*:/.test(l));
if (start === -1) return null;
// Collect indented lines under `gbrain:` until next top-level key or EOF
const block: string[] = [];
for (let i = start + 1; i < lines.length; i++) {
const line = lines[i];
if (/^[A-Za-z_][A-Za-z0-9_-]*\s*:/.test(line)) break; // next top-level key
block.push(line);
}
const text = block.join("\n");
// Extract schema number
const schemaMatch = text.match(/\n\s*schema\s*:\s*(\d+)/);
const schema = schemaMatch ? parseInt(schemaMatch[1], 10) : 1;
// Extract context_queries items
const queries: GbrainManifestQuery[] = [];
const cqMatch = text.match(/\n\s*context_queries\s*:\s*\n([\s\S]+)/);
if (cqMatch) {
const cqText = cqMatch[1];
// Split using a positive lookahead so each chunk begins with the list-item dash.
// Pattern: line starting with 4-6 spaces + "-" + whitespace.
const rawItems = cqText.split(/(?=^[ ]{4,6}-\s)/m);
const items = rawItems.filter((s) => /^[ ]{4,6}-\s/.test(s));
for (const item of items) {
const q: Partial<GbrainManifestQuery> = {};
// Strip the leading list-item marker so id/kind/etc. regexes can use line-start.
const body = item.replace(/^[ ]{4,6}-\s+/, " ");
const idM = body.match(/(?:^|\n)\s*id\s*:\s*([^\n]+)/);
const kindM = body.match(/(?:^|\n)\s*kind\s*:\s*([^\n]+)/);
const renderM = body.match(/(?:^|\n)\s*render_as\s*:\s*"?([^"\n]+?)"?\s*$/m);
const queryM = body.match(/(?:^|\n)\s*query\s*:\s*"?([^"\n]+?)"?\s*$/m);
const limitM = body.match(/(?:^|\n)\s*limit\s*:\s*(\d+)/);
const globM = body.match(/(?:^|\n)\s*glob\s*:\s*"?([^"\n]+?)"?\s*$/m);
const sortM = body.match(/(?:^|\n)\s*sort\s*:\s*([^\n]+)/);
const tailM = body.match(/(?:^|\n)\s*tail\s*:\s*(\d+)/);
const filterMap = parseFilterMap(body);
if (idM) q.id = idM[1].trim();
if (kindM) {
const k = kindM[1].trim();
if (k === "vector" || k === "list" || k === "filesystem") q.kind = k;
}
if (renderM) q.render_as = renderM[1].trim();
if (queryM) q.query = queryM[1].trim();
if (limitM) q.limit = parseInt(limitM[1], 10);
if (globM) q.glob = globM[1].trim();
if (sortM) q.sort = sortM[1].trim();
if (tailM) q.tail = parseInt(tailM[1], 10);
if (filterMap) q.filter = filterMap;
if (q.id && q.kind && q.render_as) {
queries.push(q as GbrainManifestQuery);
}
}
}
return { schema, context_queries: queries };
}
/**
* Parse a nested `filter:` block map out of a single context_queries item body.
*
* The block is a YAML map nested under the `filter:` key:
*
* filter:
* type: timeline
* tags_contains: "repo:{repo_slug}"
*
* Each sub-key sits one indent level deeper than `filter:`. Surrounding quotes
* are stripped and template vars ({repo_slug}, now-7d, ...) are left intact for
* downstream substitution, matching how dispatchList stringifies each value
* into a `--filter k=v` argument. Returns undefined when there is no `filter:`
* block or it is empty.
*/
function parseFilterMap(body: string): Record<string, string> | undefined {
const lines = body.split("\n");
const filterIdx = lines.findIndex((l) => /^\s*filter\s*:\s*$/.test(l));
if (filterIdx === -1) return undefined;
const filterIndent = lines[filterIdx].match(/^\s*/)![0].length;
const filter: Record<string, string> = {};
for (let i = filterIdx + 1; i < lines.length; i++) {
const line = lines[i];
if (line.trim() === "") continue; // tolerate blank lines within the block
const indent = line.match(/^\s*/)![0].length;
if (indent <= filterIndent) break; // dedent to a sibling key ends the block
const kv = line.match(/^\s*([A-Za-z0-9_]+)\s*:\s*"?(.*?)"?\s*$/);
if (kv) filter[kv[1]] = kv[2].trim();
}
return Object.keys(filter).length > 0 ? filter : undefined;
}
// ── Public: withErrorContext ──────────────────────────────────────────────
const ERROR_LOG_PATH = join(gstackHome(), ".gbrain-errors.jsonl");
/**
* Wrap an op with structured error logging. Logs success/failure + duration
* to ~/.gstack/.gbrain-errors.jsonl for forensic debugging. Replaces ad-hoc
* try/catch sites across the three Bun helpers (Section 2B).
*
* On error: the error is RE-THROWN after logging — caller still owns flow.
*/
export async function withErrorContext<T>(
op: string,
fn: () => T | Promise<T>,
caller: string = "unknown"
): Promise<T> {
const t0 = Date.now();
try {
const result = await fn();
logErrorContext({
ts: new Date().toISOString(),
op,
duration_ms: Date.now() - t0,
outcome: "ok",
schema_version: 1,
last_writer: caller,
});
return result;
} catch (err) {
logErrorContext({
ts: new Date().toISOString(),
op,
duration_ms: Date.now() - t0,
outcome: "error",
error: err instanceof Error ? err.message : String(err),
schema_version: 1,
last_writer: caller,
});
throw err;
}
}
function logErrorContext(entry: ErrorContextEntry): void {
try {
const path = errorLogPath();
mkdirSync(dirname(path), { recursive: true });
appendJsonl(path, entry);
} catch {
// Logging failure is non-fatal — never block the op.
}
}
// Test-only export for resetting the gitleaks availability cache between tests.
export function _resetGitleaksAvailabilityCache(): void {
_gitleaksAvailability = null;
_gitleaksSlowWarned = false;
_gitleaksAbsentWarned = false;
_gitleaksConsecutiveSlow = 0;
_gitleaksCooldownWarned = false;
_probeMs = GITLEAKS_PROBE_MS;
_retryMs = GITLEAKS_RETRY_MS;
}
// Test-only: shrink the probe budgets so the slow path can be exercised
// without a multi-second sleep in the suite. Reset restores the defaults.
export function _setGitleaksProbeTimeouts(first: number, second: number): void {
_probeMs = first;
_retryMs = second;
}
// Test-only: read the cache without triggering a probe. `null` means the
// question is still open — which is the whole point of the timeout path.
export function _gitleaksCacheState(): boolean | null {
return _gitleaksAvailability;
}