Files
gstack/lib/gbrain-local-status.ts
T
Garry Tan 3bef43bc5a v1.55.0.0 fix wave: gbrain data-loss guards + browser crash-loop + 6 more (#1808)
* fix(jsonl-merge): make equal-ts resolution converge across machines

The JSONL append merge driver sorted timestamped entries by (0, ts) with no
further tiebreaker. Equal-ts entries then fell back to stable-sort insertion
order (base, ours, theirs), but git assigns the local side to "ours", so two
machines resolving the same conflict emitted equal-ts lines in opposite order.
The merged files diverged and never converged. gstack-telemetry-log uses
second-granularity timestamps, so same-ts collisions are routine.

Add the line content as the final sort tiebreaker so the order is total and
side-independent. Add a regression test that runs the driver with the two
sides swapped and asserts identical output.

* fix(gen-skill-docs): quote frontmatter descriptions with interior colons (#1778)

Generated SKILL.md frontmatter emitted the catalog-trimmed description: as a
plain YAML scalar. A description with an interior ": " (e.g. "Ship workflow:
detect...") parses as a nested mapping under strict YAML loaders, so Codex/OpenAI
skill loading rejected those skills.

applyCatalogTrim now routes the value through toYamlInlineScalar, which quotes
(via JSON.stringify) only when a plain scalar would be invalid — interior ": ",
inline " #", leading indicator char, or surrounding whitespace. Strings that are
already valid plain scalars pass through unchanged to keep regen diffs small.

The frontmatter test now parses every generated block (Claude + Codex hosts) with
Bun.YAML.parse instead of string-checking that name:/description: substrings exist,
so the regression can't reappear. Runs under `bun test` (already in CI).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(skills): regenerate SKILL.md after frontmatter quoting fix (#1778)

9 catalog-trimmed descriptions whose values contain an interior colon or inline-
comment marker are now quoted. Generated output only; rerun of bun run gen:skill-docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(gbrain-sources): centralize sources-list shape handling in parseSourcesList (#1576)

#1576's crash in sourceLocalPath was already fixed in v1.42.0.0 (dual-shape
handling). But the readers disagreed: sourceLocalPath accepted both the wrapped
{sources:[...]} object (v0.20+) and a bare array, while probeSource and
sourcePageCount accepted only the wrapped shape. Extract one parseSourcesList()
normalizer and route all three through it, so the shape assumption lives in a
single place. This is also the base the #1734 remote_url audit builds on.

parseSourcesList returns [] for null/garbage rather than throwing; callers treat
'no rows' as absent. New test/gbrain-sources-parse.test.ts pins both shapes plus
the garbage paths and confirms config.remote_url survives for the audit.

#1576 is closeable as already-fixed in v1.42.0.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(gbrain): spawn gbrain + brain-sync through a shell on Windows (#1731)

On Windows, bun/npm install gbrain as a gbrain.cmd/.ps1 shim and gstack-brain-sync
is a bash shebang script. spawnSync/spawn/execFileSync resolve neither without a
shell, so the child spawn failed ENOENT — on the sync orchestrator this surfaced
as 'brain-sync exited undefined' (#1731).

Add NEEDS_SHELL_ON_WINDOWS (process.platform === 'win32') in gbrain-exec and pass
it as shell: to every gbrain/brain-sync child spawn: spawnGbrain, spawnGbrainAsync,
execGbrainText (gbrain-exec), the two sources-list/remove/add spawns (gbrain-sources),
the version + probe spawns (gbrain-local-status), and the two brain-sync spawns in
the orchestrator. POSIX keeps the cheaper no-shell path.

macOS/Linux CI can't exercise the Windows path, so test/gbrain-spawn-windows-shell.ts
is a static-grep tripwire: it fails CI if a gbrain/brain-sync spawn is added without
the shell flag.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(catalog-trim): expect YAML-quoted descriptions with interior colons (#1778)

The quoting fix wraps colon-bearing catalog descriptions in double quotes;
two catalog-trim assertions still pinned the old unquoted form. Tolerate the
optional quotes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(gbrain-sync): defensive guards against destructive gbrain ops (#1734)

The orchestrator shelled out to gbrain's destructive subcommands as if they were
safe. gbrain can rm-rf a user's working tree during an autopilot race (its own
bug, upstream gbrain #1526); gstack now defends itself. New lib/gbrain-guards.ts
gates the two destructive reach points, all checked immediately before the op:

- Autopilot refuse (multi-signal, affirmative-only): refuse a destructive op when
  a live 'gbrain autopilot' process (primary) or a known autopilot lock file
  (secondary; checked under both GBRAIN_HOME and ~/.gbrain since gbrain #1226
  ignores GBRAIN_HOME) is present. No signal → proceed; inability to introspect
  never bricks a normal sync.
- sources remove: routed through safeSourcesRemove → decideSourceRemove. Fail
  CLOSED — refuse to remove a user-managed source (remote_url set, local_path
  outside gbrain's clones) when gbrain has no --keep-storage to protect the files
  (it doesn't in 0.41.x). Also fail closed when the source list can't be read.
  Path containment uses realpath so a symlink can't smuggle a delete out of clones.
- sync --strategy code: decideCodeSync refuses URL-managed sources (remote_url
  set) unless --allow-reclone is passed, since the walk can auto-reclone (rm-rf).

Capability detection memoizes per process keyed to gbrain's identity (no stale
persistent cache); --keep-storage can't be probed (generic help) so it defaults
unsupported → fail closed. Every guard surfaces a visible reason; autopilot/reclone
refusals fail the code stage (verdict ERR) rather than silently skipping protection.

test/gbrain-guards.test.ts covers all branches hermetically (injected rows + probe
overrides): autopilot signals, fail-closed remove, keep-storage path, reclone gate,
realpath/symlink containment. Supersedes #1736 (which guarded a nonexistent path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(sync-gbrain): warn against running during autopilot; prefer --path sources (#1734)

Adds a Safety note to the /sync-gbrain guidance (template + regenerated SKILL.md +
this repo's CLAUDE.md): don't run while autopilot is active, and prefer
`gbrain sources add --path` over URL-managed sources, which can auto-reclone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(memory-ingest): configurable import timeout + resume-on-timeout messaging (#1611)

The gbrain import (the long pole on big brains) had a hardcoded 30-min timeout,
so large memory corpora got SIGTERM'd mid-import on /sync-gbrain --full. Make it
configurable via GSTACK_INGEST_TIMEOUT_MS (default 30 min, validated 1min–24h).

gstack can't drive gbrain's internal resume, but the existing SIGTERM forwarder
already preserves gbrain's import-checkpoint.json, so the next run resumes. On a
timeout we now say so explicitly ('checkpoint preserved — re-run /sync-gbrain to
resume, raise GSTACK_INGEST_TIMEOUT_MS for big brains') instead of surfacing a
bare 'exited null'. True gstack-driven ingest-resume is deferred to gbrain
(.context/gbrain-asks.md).

Also guards the module's main() behind import.meta.main so resolveImportTimeoutMs
is unit-testable; the orchestrator runs it as a subprocess where main still fires.
New test/memory-ingest-timeout.test.ts pins default/override/invalid resolution.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(browse): stop the headed daemon crash-loop + silent headless downgrade (#1781)

A headed session against a beacon-heavy page (analytics/extension load) could tip
the single-threaded daemon into a self-inflicted crash-loop: a brief HTTP stall
was read as a crash, the restart didn't clear the dead Chromium's SingletonLock,
the relaunch failed, and the session silently came back headless. Four fixes:

1. Busy-vs-dead (sendCommand): on a connection error, if the process is alive give
   /health a bounded probe (3x/250ms) and just retry the command — never kill+restart
   a live-but-busy server. A 30s timeout now reports 'busy, not restarting' when the
   process is alive instead of exiting into a kill cycle.
2. Profile-lock cleanup on (re)start: startServer reaps the orphaned Chromium holding
   the SingletonLock and clears Singleton{Lock,Socket,Cookie} before relaunch, so the
   auto-restart path gets the same clean profile the manual connect preamble did.
3. Headed persistence: the restart env reapplies BROWSE_HEADED from this invocation OR
   the persisted server state (mode==='headed'), so a restart from a plain command
   never downgrades a headed window to invisible headless. Extracted to buildRestartEnv.
4. Force-clean disconnect reaps the Chromium child tree (via the SingletonLock PID) so
   the next connect starts clean instead of fighting an orphan.

Plus macOS window surfacing: connect + focus raise 'Google Chrome for Testing' to the
active Space (best-effort osascript) with a Mission Control hint — the first thing
users read as 'I can't see the browser'.

Shared lock helpers (chromiumProfileDir / cleanChromiumProfileLocks / killOrphanChromium)
dedupe the connect, disconnect, and restart paths. browse/test/restart-env.test.ts pins
the headed-persistence decision; the full crash-loop repro is an E2E (periodic).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(gbrain-install): remove the v0.18.2 pin, install latest + version floor + doctor self-test (#1744)

The installer pinned gbrain at v0.18.2 while gbrain shipped v0.41.x — ~23 versions
behind. Remove the hard pin: a fresh clone now stays on the latest default-branch
HEAD. --pinned-commit <sha> still pins for reproducibility.

Unpinning removes the version gate the pin provided, so add two install-time gates
that fail closed (exit 3, matching the existing PATH-shadow/version-mismatch posture):
- MIN_GBRAIN_VERSION floor (0.20.0, the sources-list/federated surface gstack needs):
  refuse an install below it.
- gbrain doctor --fast self-test when a brain config already exists (re-install /
  detected clone): refuse to leave a broken gbrain in place. Pre-init installs skip
  it; the full /sync-gbrain --dry-run self-test runs from /setup-gbrain after init.

Docs updated (USING_GBRAIN_WITH_GSTACK.md no longer says 'edit PINNED_COMMIT').
Detect-install tests bump the success-path fixtures above the floor and add a
below-floor exit-3 test. The gbrain-side asks (root #1526 fix, --keep-storage,
remove-lease, capability command, ingest-resume, integration CI) are written to
.context/gbrain-asks.md for filing against garrytan/gbrain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(#1778): update claude-ship golden + catalog-mode assertions for quoted descriptions

ship's catalog description ('Ship workflow: detect...') has an interior colon, so
the #1778 fix now YAML-quotes it. Refresh the claude-ship golden baseline to the
quoted output and make the catalog-mode-full trim/restore assertions quote-tolerant.
codex/factory ship goldens are unaffected (they use block-scalar descriptions).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(gen-skill-docs): use function replacer so a $ in a description can't corrupt frontmatter (#1778)

String.prototype.replace treats $&/$1/$` in the replacement as patterns. A future
skill description containing $ (e.g. referencing $B/$D) would silently corrupt the
generated frontmatter. Use a function replacer. Behavior-preserving for all current
descriptions (regen produces no diff).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v1.55.0.0)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(gbrain): document configurable memory-ingest timeout for v1.55.0.0

USING_GBRAIN_WITH_GSTACK.md: note GSTACK_INGEST_TIMEOUT_MS (default 30 min,
1 min-24h range) on the /sync-gbrain memory stage, plus checkpoint-resume on
timeout. Fills the reference gap left by the configurable-import-timeout fix
(#1611) shipped in v1.55.0.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Jayesh Betala <jayesh.betala7@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 14:57:07 -07:00

287 lines
9.5 KiB
TypeScript

/**
* gbrain-local-status — classify the local gbrain engine into 5 states.
*
* Shared between bin/gstack-gbrain-detect (preamble probe on every skill start)
* and bin/gstack-gbrain-sync.ts (orchestrator SKIP-when-not-ok semantics).
* Single source of truth: same probe, same classification, same cache.
*
* Per the split-engine plan (D2 + D8):
* - Probe: `gbrain sources list --json`. Cheap (~80ms), actually hits the DB.
* Uses the same stderr patterns as lib/gbrain-sources.ts:66-67.
* - Cache: 60s TTL at ~/.gstack/.gbrain-local-status-cache.json, keyed on
* {home, path_hash, gbrain_bin_path, gbrain_version, config_mtime}.
* - --no-cache bypass: /setup-gbrain and /sync-gbrain pass it after any
* state-mutating operation so the next read sees fresh status.
*
* No-cli → gbrain not on PATH.
* Missing → CLI present, ~/.gbrain/config.json absent.
* Broken-config → config exists but `gbrain sources list` fails with config parse error
* (or any non-recognized error — defensive default per codex #8).
* Broken-db → config exists, DB unreachable per stderr classification.
* Ok → DB reachable, sources list returned valid JSON.
*/
import { execFileSync } from "child_process";
import {
createHash,
} from "crypto";
import {
existsSync,
mkdirSync,
readFileSync,
renameSync,
statSync,
writeFileSync,
} from "fs";
import { homedir } from "os";
import { dirname, join } from "path";
import { buildGbrainEnv, NEEDS_SHELL_ON_WINDOWS } from "./gbrain-exec";
export type LocalEngineStatus =
| "ok"
| "no-cli"
| "missing-config"
| "broken-config"
| "broken-db";
export interface ClassifyOptions {
/** Bypass the 60s cache. Used after any state-mutating operation. */
noCache?: boolean;
/** Env override for the spawned `gbrain` (used by tests to point at a fake binary). */
env?: NodeJS.ProcessEnv;
}
interface CacheEntry {
// Local-cache schema version, controlled by gstack. Not to be confused
// with `gbrain doctor --json` output schema_version (gbrain v0.25+ emits
// schema_version: 2). Doctor-output parsing lives in
// lib/gstack-memory-helpers.ts:freshDetectEngineTier and accepts both
// doctor-output versions. This cache stays strictly at version 1 — a
// future shape change here requires an explicit migration.
schema_version: 1;
status: LocalEngineStatus;
cached_at: number;
/** Cache invariants — entry is invalidated if any of these change between writes. */
key: {
home: string;
path_hash: string;
gbrain_bin_path: string;
gbrain_version: string;
config_mtime: number; // 0 when config absent
config_size: number; // 0 when config absent
};
}
export const CACHE_TTL_MS = 60_000;
export const PROBE_TIMEOUT_MS = 5_000;
/** Effective user home — respects HOME env override (used by tests). */
function userHome(): string {
return process.env.HOME || homedir();
}
/** Cache path computed fresh on each call so tests can mutate GSTACK_HOME per case. */
export function cacheFilePath(): string {
return join(
process.env.GSTACK_HOME || join(userHome(), ".gstack"),
".gbrain-local-status-cache.json",
);
}
function gbrainConfigPath(): string {
return join(userHome(), ".gbrain", "config.json");
}
function hashPath(p: string): string {
return createHash("sha256").update(p).digest("hex").slice(0, 16);
}
/**
* Resolve the absolute path of `gbrain` on PATH. Returns null when missing.
* Memoized per-process keyed on PATH so detect's call and the classifier's
* call share one fork-exec (~200ms saved per skill preamble).
*/
const _gbrainBinCache = new Map<string, string | null>();
export function resolveGbrainBin(env?: NodeJS.ProcessEnv): string | null {
const e = env ?? process.env;
const key = e.PATH || "";
if (_gbrainBinCache.has(key)) return _gbrainBinCache.get(key)!;
let result: string | null = null;
try {
execFileSync("gbrain", ["--version"], {
encoding: "utf-8",
timeout: 2_000,
stdio: ["ignore", "ignore", "ignore"],
env: e,
shell: NEEDS_SHELL_ON_WINDOWS, // #1731: gbrain is a .cmd shim on Windows
});
result = "gbrain";
} catch {
result = null;
}
_gbrainBinCache.set(key, result);
return result;
}
/** Memoized per-process. */
const _gbrainVersionCache = new Map<string, string>();
export function readGbrainVersion(env?: NodeJS.ProcessEnv): string {
const e = env ?? process.env;
const key = `${e.PATH || ""}|${resolveGbrainBin(e) || ""}`;
if (_gbrainVersionCache.has(key)) return _gbrainVersionCache.get(key)!;
let result = "";
try {
const out = execFileSync("gbrain", ["--version"], {
encoding: "utf-8",
timeout: 2_000,
stdio: ["ignore", "pipe", "ignore"],
env: e,
shell: NEEDS_SHELL_ON_WINDOWS, // #1731: gbrain is a .cmd shim on Windows
});
result = out.trim().split("\n")[0] || "";
} catch {
result = "";
}
_gbrainVersionCache.set(key, result);
return result;
}
function configFingerprint(): { mtime: number; size: number } {
try {
const st = statSync(gbrainConfigPath());
return { mtime: Math.floor(st.mtimeMs), size: st.size };
} catch {
return { mtime: 0, size: 0 };
}
}
function buildCacheKey(
gbrainBin: string | null,
gbrainVersion: string,
env?: NodeJS.ProcessEnv,
): CacheEntry["key"] {
const e = env ?? process.env;
const config = configFingerprint();
return {
home: e.HOME || "",
path_hash: hashPath(e.PATH || ""),
gbrain_bin_path: gbrainBin || "",
gbrain_version: gbrainVersion,
config_mtime: config.mtime,
config_size: config.size,
};
}
function keysEqual(a: CacheEntry["key"], b: CacheEntry["key"]): boolean {
return (
a.home === b.home &&
a.path_hash === b.path_hash &&
a.gbrain_bin_path === b.gbrain_bin_path &&
a.gbrain_version === b.gbrain_version &&
a.config_mtime === b.config_mtime &&
a.config_size === b.config_size
);
}
function readCache(key: CacheEntry["key"]): LocalEngineStatus | null {
if (!existsSync(cacheFilePath())) return null;
try {
const raw = JSON.parse(readFileSync(cacheFilePath(), "utf-8")) as CacheEntry;
if (raw.schema_version !== 1) return null;
if (Date.now() - raw.cached_at > CACHE_TTL_MS) return null;
if (!keysEqual(raw.key, key)) return null;
return raw.status;
} catch {
return null;
}
}
function writeCache(status: LocalEngineStatus, key: CacheEntry["key"]): void {
const entry: CacheEntry = {
schema_version: 1,
status,
cached_at: Date.now(),
key,
};
try {
mkdirSync(dirname(cacheFilePath()), { recursive: true });
const tmp = cacheFilePath() + ".tmp." + process.pid;
writeFileSync(tmp, JSON.stringify(entry, null, 2), "utf-8");
renameSync(tmp, cacheFilePath());
} catch {
// Cache write failure is non-fatal — we re-probe next call.
}
}
/**
* Probe via `gbrain sources list --json`. Classify the outcome.
*
* Pattern strings ("Cannot connect to database", "config.json") are deliberately
* the same strings used in lib/gbrain-sources.ts:66-67. If gbrain reworks its
* error messages, classifier returns broken-config defensively (codex #8).
*/
function freshClassify(env?: NodeJS.ProcessEnv): LocalEngineStatus {
// 1. CLI on PATH?
const gbrainBin = resolveGbrainBin(env);
if (!gbrainBin) return "no-cli";
// 2. Config file present?
if (!existsSync(gbrainConfigPath())) return "missing-config";
// 3. Probe gbrain sources list.
//
// Seed DATABASE_URL from ~/.gbrain/config.json (via buildGbrainEnv, the
// same helper the sync orchestrator uses in lib/gbrain-exec.ts). Without
// this, Bun autoloads a project's .env when the probe runs inside a repo
// that defines its own DATABASE_URL (e.g. an app DB on a different port),
// gbrain connects to the wrong DB, and the classifier falsely reports
// broken-db. This also makes the result cwd-independent, so the 60s cache
// can no longer propagate a poisoned negative to clean directories.
try {
execFileSync("gbrain", ["sources", "list", "--json"], {
encoding: "utf-8",
timeout: PROBE_TIMEOUT_MS,
stdio: ["ignore", "pipe", "pipe"],
env: buildGbrainEnv({ baseEnv: env ?? process.env }),
shell: NEEDS_SHELL_ON_WINDOWS, // #1731: gbrain is a .cmd shim on Windows
});
return "ok";
} catch (err) {
const e = err as NodeJS.ErrnoException & { stderr?: Buffer | string };
const stderr = (e.stderr ? e.stderr.toString() : "") || "";
// ENOENT can happen if gbrain disappeared between resolveGbrainBin and now.
if (e.code === "ENOENT") return "no-cli";
// Pattern match against gbrain's known error strings. Order matters:
// "Cannot connect to database" is the more specific DB-unreachable signal.
if (stderr.includes("Cannot connect to database")) return "broken-db";
if (stderr.includes("config.json")) return "broken-config";
// Defensive default per codex #8: unrecognized failures classify as
// broken-config so the user sees the raw stderr surfaced upstream.
return "broken-config";
}
}
/**
* Classify the local gbrain engine status. Cached for 60s; bypassable.
*
* Returns one of 5 states. Never throws — failure modes are surfaced as states.
*/
export function localEngineStatus(opts: ClassifyOptions = {}): LocalEngineStatus {
const env = opts.env ?? process.env;
const gbrainBin = resolveGbrainBin(env);
const gbrainVersion = gbrainBin ? readGbrainVersion(env) : "";
const key = buildCacheKey(gbrainBin, gbrainVersion, env);
if (!opts.noCache) {
const cached = readCache(key);
if (cached) return cached;
}
const fresh = freshClassify(env);
writeCache(fresh, key);
return fresh;
}