Files
gstack/lib/gstack-memory-helpers.ts
T
f30b7b788a v1.91.13.0 fix: severe bug wave (autoplan guard, land-and-deploy CI gate, project identity, gbrain, review gates) (#3013)
* fix(land-and-deploy): never merge over red, pending or missing CI (#2995)

/land-and-deploy gated only on required checks, so a repo with none
(every GitHub Free private repo, any repo without branch protection)
skipped the CI wait and could merge over red CI. The verdict now comes
from bin/gstack-ci-gate, a tested helper bound to the head SHA: all
checks by default, cancel/unknown buckets fail, NO_CHECKS needs explicit
per-head approval, and both merge commands (including --auto) run only
after a PASS for the same SHA they pin with --match-head-commit.

Co-authored-by: harshitgavita-07 <harshit.gavita@gmail.com>

* fix(slug): give nested-group projects their own state bucket and verify checkpoint identity (#3003, #2901)

gstack-slug kept only the last two remote path segments, so
customer-a/product/repo and customer-b/product/repo shared one bucket
and /context-restore could present another project's checkpoint.

- One canonical-remote rule (bash + TS, golden vectors) used by all six
  resolvers: lowercased host, no userinfo/port/.git, Azure DevOps and
  Bitbucket Server normalized. 3+ segment remotes get
  <last-two>-<16 hex>; two-segment remotes (all of GitHub) are unchanged.
- Versioned sticky cache (v2:) keeps cache hits git-free.
- context-save stamps the credential-free remote and root;
  context-restore only treats this project's checkpoints as latest,
  reports foreign/unverified ones first, and treats a different root
  with the same remote (Conductor handoff) as information.
- gstack-slug --adopt-legacy copies only files the user confirms.
- question-preference-hook reads the same versioned cache, so never-ask
  preferences fire; unknown identity asks instead of auto-deciding.

* fix(gbrain): brain context loads, managed sync works, memory sync stops duplicating (#2883, #2985, #2922)

- brain-context-load called 'gbrain list_pages' (an MCP tool name), so
  every skill's brain context was 0 bytes. It now uses 'gbrain list'
  with gbrain's real filter flags, and prints one line when queries fail.
- The code-stage sync always passes --no-pull (managed gbrain >= 0.51
  refuses without it); a gbrain that rejects the flag fails with an
  upgrade message and is never retried without it. stderr is streamed
  with a bounded tail and quoted in failure summaries.
- /sync-gbrain gains --sources (and GSTACK_MEMORY_INGEST_SOURCES); by
  default it skips types already synced through a federated curated
  source, so curated pages stop being re-duplicated.

Co-authored-by: Vitor Cepeda Lopes <vitor@vitorcepedalopes.com>
Co-authored-by: harshitgavita-07 <harshit.gavita@gmail.com>

* fix(codex,cso): clean Codex reviews stop failing; semgrep and trivy SARIF results import (#2769, #3011)

- Codex review gate is PASS / FAIL / UNVERIFIED. Untagged output (what
  a clean 'codex review --base' prints) is UNVERIFIED, not FAIL, and
  never a false PASS; P2/P3-only output passes.
- import-sarif resolves semgrep's undeclared %SRCROOT% and trivy's
  ROOTPATH (including git-URL targets) through explicit, captured
  mappings; declared bases must sit inside the source root or the
  run's checkout, URIs are percent-decoded, and rejected results are
  counted with rule, base and path instead of silently dropped.

* fix: version strings are not IPs, failed update checks are not 'up to date', private security reporting (#2856, #2786, #2984)

- The pre-push redact hook exempts a root VERSION file that is only a
  version string and the version token in CHANGELOG release headings;
  the engine allowlist is untouched.
- gstack-update-check caches an unreadable remote as CHECK_FAILED with
  a short TTL instead of UP_TO_DATE; /gstack-upgrade never reports
  'latest' after a failed check.
- SECURITY.md points reporters at GitHub private vulnerability
  reporting.
- pr-title-sync (pull_request_target, write token) pins actions/checkout
  to a reviewed SHA.

* fix(autoplan): publication guard accepts real Claude Code journals (#3007, #3009, #2968, #2977, #2986, #3006)

The guard required a session journal to start with a user message, but
Claude Code starts it with SessionStart hook attachments (and a forked
compacted session starts with a compact_boundary), so every phase was
denied for anyone with a SessionStart hook. Windows path spellings
(C:/, /c/, drive case) were rejected too.

- Reader accepts the captured startup, compact, clear and fork shapes
  (claude 2.1.284 fixture), computes root uniqueness over every
  candidate, and returns typed reason codes instead of one string.
- Hard denials (competing root, foreign cwd, sidechain, agent, cycle,
  malformed, identity) say what happened and how to run the reviews by
  hand; unflushed journals still retry.
- A well-formed, stable journal whose root shape is unrecognized now
  warns ('phase publication was NOT verified') and continues, without
  overriding Claude Code's own permission check; no bypass switch.
- Windows paths fold before validation, so '..' is still rejected.
- Periodic drift canary runs the reader on fresh journals from the
  pinned claude.

Co-authored-by: Charles M Berg <77517285+CharlesBerg85@users.noreply.github.com>
Co-authored-by: laurenbdaniels-coder <264343335+laurenbdaniels-coder@users.noreply.github.com>
Co-authored-by: Jordan Mayer <176445995+maydaycyber@users.noreply.github.com>
Co-authored-by: elmesery-dev <elmesery.dev@gmail.com>

* v1.91.13.0: severe bug fix wave release notes, stale gbrain verbs in docs

* test: seed the versioned slug cache in the memory-injection deny case (#2901)

The preference hook now resolves the project bucket from gstack-slug's
v2 slug-cache entry instead of basename(cwd); this fixture still wrote
preferences under the basename bucket without the entry the skill
preamble always writes first.

* fix: full-suite follow-ups for the fix wave

- gstack-slug's --help heredoc is over 512B, so it gets the BASH_COMPAT=50
  pipe-deadlock guard like the other bins.
- Refresh the codex/factory ship goldens for the plan-gates slug change.
- The periodic lane gains the autoplan journal drift canary (107 paid
  files); raise the eval-slices cap to 25 so every slice still starts
  at once until its recorded duration lets the planner pack it.

* fix(plan-eng-review): clearer Step 0 sequencing, write routes and section outputs

The workflow judge scored plan-eng-review's instructions at
actionability 3.67 (bar 4) on text unchanged from main. Resolve the
ambiguities it named together, without changing any gate or output:
Step 0 names the Read's order and keeps one STOP; the pending-call,
read-only, forbidden-write and failed-save rules become one Write
routes block; header/label limits and grid checks are concrete;
Performance gets a checklist; each section has a stated output shape;
Lake Score is defined. Same total length (parity cap).

---------

Co-authored-by: harshitgavita-07 <harshit.gavita@gmail.com>
Co-authored-by: Vitor Cepeda Lopes <vitor@vitorcepedalopes.com>
Co-authored-by: Charles M Berg <77517285+CharlesBerg85@users.noreply.github.com>
Co-authored-by: laurenbdaniels-coder <264343335+laurenbdaniels-coder@users.noreply.github.com>
Co-authored-by: Jordan Mayer <176445995+maydaycyber@users.noreply.github.com>
Co-authored-by: elmesery-dev <elmesery.dev@gmail.com>
2026-10-02 11:11:00 -07:00

656 lines
24 KiB
TypeScript

/**
* gstack-memory-helpers — shared helpers for the V1 memory ingest + retrieval pipeline.
*
* Imported by:
* - bin/gstack-memory-ingest.ts (Lane A)
* - bin/gstack-gbrain-sync.ts (Lane B)
* - bin/gstack-brain-context-load.ts (Lane C)
* - scripts/gen-skill-docs.ts (manifest validation)
*
* Design refs in the plan:
* §"Eng review additions" — DRY refactor (Section 1A)
* §"V1 final scope clarification" — schema_version: 1 standardization (Section 2A)
* ED1 — engine-tier cache lives in ~/.gstack/.gbrain-engine-cache.json (60s TTL)
*
* NOTE: secretScanFile() currently shells out to `gitleaks` from PATH; the vendored
* binary install is part of Lane E (setup-gbrain). When gitleaks is missing, the
* helper warns once and returns an empty findings list with scanner="missing".
* An empty list means "clean" only when scanner === "gitleaks" — callers that
* gate writes on a scan must treat "missing" and "error" as unscanned.
*/
import { existsSync, readFileSync, writeFileSync, mkdirSync, mkdtempSync, rmSync, statSync } from "fs";
import { appendJsonl } from "./jsonl-store";
import { gbrainConfigDir, isExecTimeout } from "./gbrain-exec";
import { dirname, join } from "path";
import { execFileSync } from "child_process";
import { tmpdir } from "os";
import { resolveStateRoot } from "./state-root";
// ── Types ──────────────────────────────────────────────────────────────────
export interface SecretFinding {
rule_id: string;
description: string;
line: number;
redacted_match: string;
}
export interface SecretScanResult {
scanned: boolean;
findings: SecretFinding[];
scanner: "gitleaks" | "missing" | "error";
}
export type EngineTier = "pglite" | "supabase" | "unknown";
export interface EngineDetect {
engine: EngineTier;
supabase_url?: string;
detected_at: number;
schema_version: 1;
}
export interface GbrainManifestQuery {
id: string;
kind: "vector" | "list" | "filesystem";
render_as: string;
// kind=vector
query?: string;
// kind=list
filter?: Record<string, unknown>;
sort?: string;
// kind=filesystem
glob?: string;
tail?: number;
// common
limit?: number;
}
export interface GbrainManifest {
schema: number; // gbrain.schema in frontmatter; V1 = 1
context_queries: GbrainManifestQuery[];
}
export interface ErrorContextEntry {
ts: string;
op: string;
duration_ms: number;
outcome: "ok" | "error";
error?: string;
schema_version: 1;
last_writer: string;
}
// ── Public: canonicalizeRemote ────────────────────────────────────────────
/**
* Normalize a git remote URL to a canonical form: `host/org/repo` (no scheme,
* no trailing `.git`). Used as the dedup key for cross-Mac transcript routing
* (per ED1 — gbrain-side session_id dedup uses repo as a tag).
*
* Examples:
* https://github.com/garrytan/gstack.git → github.com/garrytan/gstack
* git@github.com:garrytan/gstack.git → github.com/garrytan/gstack
* ssh://git@gitlab.com/foo/bar → gitlab.com/foo/bar
* (empty / null) → ""
*/
export function canonicalizeRemote(url: string | null | undefined): string {
if (!url) return "";
let s = url.trim();
if (!s) return "";
// strip surrounding quotes that some configs add
s = s.replace(/^['"]|['"]$/g, "");
// git@host:path/repo → host/path/repo
const scpMatch = s.match(/^[^@\s]+@([^:]+):(.+)$/);
if (scpMatch) {
s = `${scpMatch[1]}/${scpMatch[2]}`;
} else {
// strip scheme (https://, ssh://, git://, http://)
s = s.replace(/^[a-z][a-z0-9+.-]*:\/\//i, "");
// strip user@ prefix on URL-style remotes
s = s.replace(/^[^@\/]+@/, "");
}
// strip trailing slash(es) first, so a URL written with a trailing slash
// still matches the `.git$` suffix below (e.g. ".../repo.git/" must
// canonicalize to ".../repo", not ".../repo.git").
s = s.replace(/\/+$/, "");
// strip trailing .git
s = s.replace(/\.git$/i, "");
// re-strip trailing slash(es): a path remote ending in a `.git` directory
// component ("/repo/.git") exposes a new trailing slash once `.git` is
// stripped, which would split the repo into a second identity.
s = s.replace(/\/+$/, "");
// collapse multiple slashes (after path normalization)
s = s.replace(/\/{2,}/g, "/");
return s.toLowerCase();
}
// ── Public: secretScanFile (gitleaks wrapper) ─────────────────────────────
let _gitleaksAvailability: boolean | null = null;
// Two flags, not one: "slow" and "absent" are different messages with
// different remediations, and a slow warning early in a run must not
// suppress the permanent "not in PATH; scanning disabled" warning later.
let _gitleaksSlowWarned = false;
let _gitleaksAbsentWarned = false;
// Per-run cooldown: retrying a slow probe on EVERY file re-pays up to
// probe+retry (12s default) per file — an 887-file ingest on a loaded box
// spent hours asking the same slow question. After this many consecutive
// slow answers the run stops probing; the availability cache is still never
// written (slow != absent — the next PROCESS probes fresh).
const GITLEAKS_SLOW_PROBE_LIMIT = 3;
let _gitleaksConsecutiveSlow = 0;
let _gitleaksCooldownWarned = false;
// Probe budgets. The first is short because the common answers (a real
// gitleaks, or ENOENT) are both immediate; the second is generous because by
// then we know the box is busy, not that the binary is absent.
const GITLEAKS_PROBE_MS = 2_000;
const GITLEAKS_RETRY_MS = 10_000;
let _probeMs = GITLEAKS_PROBE_MS;
let _retryMs = GITLEAKS_RETRY_MS;
/**
* Probe outcome. "slow" is the load case: the binary may well be installed,
* the machine just did not get around to answering. It is deliberately NOT
* folded into "absent" — see gitleaksAvailable().
*/
type GitleaksProbe = "ok" | "absent" | "slow";
function probeGitleaks(timeoutMs: number): GitleaksProbe {
try {
execFileSync("gitleaks", ["version"], {
env: process.env,
stdio: "ignore",
timeout: timeoutMs,
});
return "ok";
} catch (err) {
const e = err as NodeJS.ErrnoException;
if (e?.code === "ENOENT") return "absent";
if (isExecTimeout(err)) return "slow";
// Present but unusable (non-zero exit, EACCES). Practically the same as
// absent, and equally permanent for this process.
return "absent";
}
}
/**
* Is gitleaks usable? Answers are cached for the process — EXCEPT a timeout.
*
* Caching a timeout was a fail-open bug: one busy moment (observed under the
* 7-way sharded test runner, where spawning a shell script took over 2s) set
* availability to false for the whole run, and every later file was ingested
* unscanned behind a single stderr line. A missing binary is a fact and stays
* cached; a slow answer is a condition and gets retried on the next file.
*/
function gitleaksAvailable(): boolean {
if (_gitleaksAvailability !== null) return _gitleaksAvailability;
if (_gitleaksConsecutiveSlow >= GITLEAKS_SLOW_PROBE_LIMIT) {
if (!_gitleaksCooldownWarned) {
_gitleaksCooldownWarned = true;
process.stderr.write(
"[gstack-memory-helpers] gitleaks did not answer in " +
`${GITLEAKS_SLOW_PROBE_LIMIT} consecutive probes; skipping the probe ` +
"for the rest of this run — remaining files cannot be scanned. Re-run " +
"when the machine is less loaded to scan them.\n"
);
}
return false;
}
let probe = probeGitleaks(_probeMs);
if (probe === "slow") probe = probeGitleaks(_retryMs);
if (probe === "ok") {
_gitleaksAvailability = true;
_gitleaksConsecutiveSlow = 0;
return true;
}
if (probe === "slow") {
// No cache write: leave the question open for the next call — but count
// it, so a persistently loaded box stops paying probe+retry per file.
_gitleaksConsecutiveSlow++;
if (!_gitleaksSlowWarned) {
_gitleaksSlowWarned = true;
process.stderr.write(
"[gstack-memory-helpers] gitleaks did not answer in " +
`${Math.round((_probeMs + _retryMs) / 1000)}s (machine under load); ` +
"this file could not be scanned and the probe retries on the next one.\n"
);
}
return false;
}
_gitleaksAvailability = false;
// Only warn once per process — Lane E will vendor the binary.
if (!_gitleaksAbsentWarned) {
_gitleaksAbsentWarned = true;
process.stderr.write(
"[gstack-memory-helpers] gitleaks not in PATH; files cannot be secret-scanned. " +
"Run /setup-gbrain to install (or `brew install gitleaks`).\n"
);
}
return false;
}
/**
* Scan a file for embedded secrets using gitleaks. Returns findings list
* (empty if clean). When gitleaks is not in PATH, returns scanned=false with
* scanner="missing" — caller decides whether to skip the file or proceed.
*
* Per D19: gitleaks runs at ingest time before any put_page / put_file write.
* Replaces the inadequate regex scanner in bin/gstack-brain-sync (which only
* applies to staged git diffs).
*/
export function secretScanFile(path: string): SecretScanResult {
if (!existsSync(path)) {
return { scanned: false, findings: [], scanner: "error" };
}
if (!gitleaksAvailable()) {
return { scanned: false, findings: [], scanner: "missing" };
}
let dir: string | undefined;
try {
dir = mkdtempSync(join(tmpdir(), "gstack-secret-report-"));
const report = join(dir, "report.json");
writeFileSync(report, "", { mode: 0o600, flag: "wx" });
const maxReportBytes = 16 * 1024 * 1024;
execFileSync(
"gitleaks",
["detect", "--no-git", "--source", path, "--report-format", "json", "--report-path", report, "--exit-code", "0"],
{ env: process.env, stdio: "ignore", timeout: 60_000, killSignal: "SIGKILL" }
);
if (statSync(report).size > maxReportBytes) {
return { scanned: false, findings: [], scanner: "error" };
}
const parsed = JSON.parse(readFileSync(report, "utf-8"));
if (!Array.isArray(parsed) || !parsed.every((f) =>
f && typeof f.RuleID === "string" && f.RuleID.length > 0 &&
typeof f.Description === "string" && Number.isInteger(f.StartLine) && f.StartLine > 0 &&
(f.Secret === undefined || typeof f.Secret === "string") &&
(f.Match === undefined || typeof f.Match === "string")
)) {
return { scanned: false, findings: [], scanner: "error" };
}
const findings: SecretFinding[] = parsed.map((f) => ({
rule_id: f.RuleID,
description: f.Description,
line: f.StartLine,
redacted_match: redactMatch(f.Secret || f.Match || ""),
}));
return { scanned: true, findings, scanner: "gitleaks" };
} catch (err) {
return {
scanned: false,
findings: [],
scanner: "error",
};
} finally {
if (dir) rmSync(dir, { recursive: true, force: true });
}
}
/**
* Scan in-memory text — e.g. a rendered page body — by writing it to a
* private temp file and running secretScanFile() on it. Scan the exact bytes
* you are about to write, not the file they were rendered from: gitleaks'
* assignment rules don't match across a JSON-escaped quote, so a .jsonl
* transcript line holding `KEY=\"value\"` scans clean while the rendered
* page's `KEY="value"` is a finding. A temp file that can't be written
* returns scanner="error", never an empty "clean" result.
*/
export function secretScanText(text: string): SecretScanResult {
let dir: string | undefined;
try {
dir = mkdtempSync(join(tmpdir(), "gstack-secret-scan-"));
const file = join(dir, "page.md");
writeFileSync(file, text, { encoding: "utf-8", mode: 0o600 });
return secretScanFile(file);
} catch {
return { scanned: false, findings: [], scanner: "error" };
} finally {
if (dir) rmSync(dir, { recursive: true, force: true });
}
}
function redactMatch(s: string): string {
if (!s) return "";
if (s.length <= 8) return "[REDACTED]";
return `${s.slice(0, 4)}...${s.slice(-4)}`;
}
// ── Public: detectEngineTier (cached) ─────────────────────────────────────
const ENGINE_CACHE_TTL_MS = 60 * 1000;
function gstackHome(): string {
return resolveStateRoot();
}
function engineCachePath(): string {
return join(gstackHome(), ".gbrain-engine-cache.json");
}
function errorLogPath(): string {
return join(gstackHome(), ".gbrain-errors.jsonl");
}
/**
* Detect which gbrain engine is active (PGLite vs Supabase) and cache the
* answer for 60s in ~/.gstack/.gbrain-engine-cache.json. Caching avoids
* fork+exec'ing `gbrain doctor --json` on every skill start.
*
* Per ED1 (state files local-only): this cache is gitignored from the brain
* repo. Per Section 2A: schema_version: 1 + last_writer field for forensic
* tracing.
*/
export function detectEngineTier(): EngineDetect {
// Try cache first
if (existsSync(engineCachePath())) {
try {
const stat = statSync(engineCachePath());
const ageMs = Date.now() - stat.mtimeMs;
if (ageMs < ENGINE_CACHE_TTL_MS) {
const cached = JSON.parse(readFileSync(engineCachePath(), "utf-8")) as EngineDetect;
if (cached.schema_version === 1) return cached;
}
} catch {
// Cache corrupt; fall through to fresh detect.
}
}
const fresh = freshDetectEngineTier();
try {
mkdirSync(dirname(engineCachePath()), { recursive: true });
writeFileSync(
engineCachePath(),
JSON.stringify({ ...fresh, last_writer: "gstack-memory-helpers.detectEngineTier" }, null, 2),
"utf-8"
);
} catch {
// Cache write failure is non-fatal.
}
return fresh;
}
// Returns gbrain's config.json path, honoring GBRAIN_HOME env var with a
// fallback to ~/.gbrain. Resolution matches gbrain's own configDir()
// contract (#2521): GBRAIN_HOME is a parent dir, `.gbrain` is appended.
// gbrain >=0.25 dropped the top-level `engine` field
// from doctor output, so this file is the only reliable source for engine
// detection on that version. See #1415.
function gbrainConfigPath(): string {
return join(gbrainConfigDir(process.env), "config.json");
}
// Best-effort JSONL append to ~/.gstack/.gbrain-errors.jsonl. Never throws.
function logGbrainError(kind: string, detail: string): void {
try {
const path = errorLogPath();
mkdirSync(dirname(path), { recursive: true });
appendJsonl(path, { ts: new Date().toISOString(), kind, detail: detail.slice(0, 500) });
} catch { /* logging is best-effort */ }
}
function freshDetectEngineTier(): EngineDetect {
const now = Date.now();
let parsed: Record<string, unknown> | null = null;
// execFileSync (not execSync) avoids shell redirection — portable to
// environments where `2>/dev/null` is bash-specific. The stdio array
// suppresses stderr without invoking a shell.
try {
const out = execFileSync("gbrain", ["doctor", "--json", "--fast"], {
encoding: "utf-8",
timeout: 5000,
stdio: ["ignore", "pipe", "ignore"],
});
parsed = JSON.parse(out);
} catch (err: unknown) {
// execFileSync throws on non-zero exit; stdout is still on the error
// object. gbrain doctor exits 1 whenever health_score < 100, which is
// essentially always on fresh installs (resolver_health warnings are
// normal). Recover stdout and re-parse. See #1415.
try {
const stdout = (err as { stdout?: Buffer | string })?.stdout ?? "";
const stdoutStr = typeof stdout === "string" ? stdout : stdout.toString("utf-8");
if (stdoutStr) parsed = JSON.parse(stdoutStr);
} catch (parseErr) {
logGbrainError("doctor_parse_failure", String(parseErr));
}
}
let engine: EngineTier =
parsed?.engine === "supabase" ? "supabase" :
parsed?.engine === "pglite" ? "pglite" : "unknown";
// gbrain >=0.25 ships schema_version:2 doctor output which dropped the
// top-level `engine` field. Fall back to gbrain's config.json (respects
// GBRAIN_HOME). "supabase" here means "remote postgres" — gbrain config
// uses engine:"postgres" for real Supabase AND any other remote postgres
// (e.g. local-postgres-for-testing). Downstream sync code treats them the
// same, so the label compression is intentional.
if (engine === "unknown") {
try {
const cfg = JSON.parse(readFileSync(gbrainConfigPath(), "utf-8"));
if (cfg?.engine === "pglite") engine = "pglite";
else if (cfg?.engine === "postgres" || cfg?.database_url) engine = "supabase";
} catch (cfgErr) {
logGbrainError("config_read_failure", String(cfgErr));
}
}
return {
engine,
supabase_url: parsed?.supabase_url as string | undefined,
detected_at: now,
schema_version: 1,
};
}
// ── Public: parseSkillManifest ────────────────────────────────────────────
/**
* Parse the `gbrain:` section out of a SKILL.md.tmpl frontmatter block.
* Returns null if no manifest is declared OR if the file has no frontmatter.
*
* Schema validation (full kind/required-fields check) lives in
* scripts/gen-skill-docs.ts and runs at generation time. This parser is the
* runtime read path used by gstack-brain-context-load; it tolerates extra
* fields and relies on validation having already happened upstream.
*/
export function parseSkillManifest(skillFilePath: string): GbrainManifest | null {
if (!existsSync(skillFilePath)) return null;
const content = readFileSync(skillFilePath, "utf-8");
const frontmatter = extractFrontmatter(content);
if (!frontmatter) return null;
const gbrain = extractGbrainBlock(frontmatter);
if (!gbrain) return null;
return gbrain;
}
function extractFrontmatter(content: string): string | null {
// Supports both `---\n...\n---` (YAML) and `+++\n...\n+++` (TOML, rare).
const yamlMatch = content.match(/^---\s*\n([\s\S]*?)\n---\s*\n/);
if (yamlMatch) return yamlMatch[1];
return null;
}
function extractGbrainBlock(frontmatter: string): GbrainManifest | null {
// Naive YAML extraction — finds the `gbrain:` key and parses its sub-tree.
// Real YAML parsing avoided to keep zero-deps; gen-skill-docs validates the
// shape strictly at build time.
const lines = frontmatter.split("\n");
const start = lines.findIndex((l) => /^gbrain\s*:/.test(l));
if (start === -1) return null;
// Collect indented lines under `gbrain:` until next top-level key or EOF
const block: string[] = [];
for (let i = start + 1; i < lines.length; i++) {
const line = lines[i];
if (/^[A-Za-z_][A-Za-z0-9_-]*\s*:/.test(line)) break; // next top-level key
block.push(line);
}
const text = block.join("\n");
// Extract schema number
const schemaMatch = text.match(/\n\s*schema\s*:\s*(\d+)/);
const schema = schemaMatch ? parseInt(schemaMatch[1], 10) : 1;
// Extract context_queries items
const queries: GbrainManifestQuery[] = [];
const cqMatch = text.match(/\n\s*context_queries\s*:\s*\n([\s\S]+)/);
if (cqMatch) {
const cqText = cqMatch[1];
// Split using a positive lookahead so each chunk begins with the list-item dash.
// Pattern: line starting with 4-6 spaces + "-" + whitespace.
const rawItems = cqText.split(/(?=^[ ]{4,6}-\s)/m);
const items = rawItems.filter((s) => /^[ ]{4,6}-\s/.test(s));
for (const item of items) {
const q: Partial<GbrainManifestQuery> = {};
// Strip the leading list-item marker so id/kind/etc. regexes can use line-start.
const body = item.replace(/^[ ]{4,6}-\s+/, " ");
const idM = body.match(/(?:^|\n)\s*id\s*:\s*([^\n]+)/);
const kindM = body.match(/(?:^|\n)\s*kind\s*:\s*([^\n]+)/);
const renderM = body.match(/(?:^|\n)\s*render_as\s*:\s*"?([^"\n]+?)"?\s*$/m);
const queryM = body.match(/(?:^|\n)\s*query\s*:\s*"?([^"\n]+?)"?\s*$/m);
const limitM = body.match(/(?:^|\n)\s*limit\s*:\s*(\d+)/);
const globM = body.match(/(?:^|\n)\s*glob\s*:\s*"?([^"\n]+?)"?\s*$/m);
const sortM = body.match(/(?:^|\n)\s*sort\s*:\s*([^\n]+)/);
const tailM = body.match(/(?:^|\n)\s*tail\s*:\s*(\d+)/);
const filterMap = parseFilterMap(body);
if (idM) q.id = idM[1].trim();
if (kindM) {
const k = kindM[1].trim();
if (k === "vector" || k === "list" || k === "filesystem") q.kind = k;
}
if (renderM) q.render_as = renderM[1].trim();
if (queryM) q.query = queryM[1].trim();
if (limitM) q.limit = parseInt(limitM[1], 10);
if (globM) q.glob = globM[1].trim();
if (sortM) q.sort = sortM[1].trim();
if (tailM) q.tail = parseInt(tailM[1], 10);
if (filterMap) q.filter = filterMap;
if (q.id && q.kind && q.render_as) {
queries.push(q as GbrainManifestQuery);
}
}
}
return { schema, context_queries: queries };
}
/**
* Parse a nested `filter:` block map out of a single context_queries item body.
*
* The block is a YAML map nested under the `filter:` key:
*
* filter:
* type: timeline
* tags_contains: "repo:{repo_slug}"
*
* Each sub-key sits one indent level deeper than `filter:`. Surrounding quotes
* are stripped and template vars ({repo_slug}, now-7d, ...) are left intact for
* downstream substitution; dispatchList maps each key onto a `gbrain list`
* flag. Returns undefined when there is no `filter:`
* block or it is empty.
*/
function parseFilterMap(body: string): Record<string, string> | undefined {
const lines = body.split("\n");
const filterIdx = lines.findIndex((l) => /^\s*filter\s*:\s*$/.test(l));
if (filterIdx === -1) return undefined;
const filterIndent = lines[filterIdx].match(/^\s*/)![0].length;
const filter: Record<string, string> = {};
for (let i = filterIdx + 1; i < lines.length; i++) {
const line = lines[i];
if (line.trim() === "") continue; // tolerate blank lines within the block
const indent = line.match(/^\s*/)![0].length;
if (indent <= filterIndent) break; // dedent to a sibling key ends the block
const kv = line.match(/^\s*([A-Za-z0-9_]+)\s*:\s*"?(.*?)"?\s*$/);
if (kv) filter[kv[1]] = kv[2].trim();
}
return Object.keys(filter).length > 0 ? filter : undefined;
}
// ── Public: withErrorContext ──────────────────────────────────────────────
const ERROR_LOG_PATH = join(gstackHome(), ".gbrain-errors.jsonl");
/**
* Wrap an op with structured error logging. Logs success/failure + duration
* to ~/.gstack/.gbrain-errors.jsonl for forensic debugging. Replaces ad-hoc
* try/catch sites across the three Bun helpers (Section 2B).
*
* On error: the error is RE-THROWN after logging — caller still owns flow.
*/
export async function withErrorContext<T>(
op: string,
fn: () => T | Promise<T>,
caller: string = "unknown"
): Promise<T> {
const t0 = Date.now();
try {
const result = await fn();
logErrorContext({
ts: new Date().toISOString(),
op,
duration_ms: Date.now() - t0,
outcome: "ok",
schema_version: 1,
last_writer: caller,
});
return result;
} catch (err) {
logErrorContext({
ts: new Date().toISOString(),
op,
duration_ms: Date.now() - t0,
outcome: "error",
error: err instanceof Error ? err.message : String(err),
schema_version: 1,
last_writer: caller,
});
throw err;
}
}
function logErrorContext(entry: ErrorContextEntry): void {
try {
const path = errorLogPath();
mkdirSync(dirname(path), { recursive: true });
appendJsonl(path, entry);
} catch {
// Logging failure is non-fatal — never block the op.
}
}
// Test-only export for resetting the gitleaks availability cache between tests.
export function _resetGitleaksAvailabilityCache(): void {
_gitleaksAvailability = null;
_gitleaksSlowWarned = false;
_gitleaksAbsentWarned = false;
_gitleaksConsecutiveSlow = 0;
_gitleaksCooldownWarned = false;
_probeMs = GITLEAKS_PROBE_MS;
_retryMs = GITLEAKS_RETRY_MS;
}
// Test-only: shrink the probe budgets so the slow path can be exercised
// without a multi-second sleep in the suite. Reset restores the defaults.
export function _setGitleaksProbeTimeouts(first: number, second: number): void {
_probeMs = first;
_retryMs = second;
}
// Test-only: read the cache without triggering a probe. `null` means the
// question is still open — which is the whole point of the timeout path.
export function _gitleaksCacheState(): boolean | null {
return _gitleaksAvailability;
}