mirror of
https://github.com/garrytan/gstack.git
synced 2026-09-12 16:08:59 +02:00
* feat: add optional Memorable workflow memory (cherry picked from commit6bd5d477b2) * fix: refuse the second registration, and say what leaves the machine Two things the first cut got wrong. Memorable's own installer registers the same UserPromptSubmit hook, under its own name and outside gstack's table. `memorable start`, `memorable setup` and `memorable install-hooks` all do it, and that is the documented way to install the CLI, so on most machines it is already there before gstack is asked. Registering ours beside it ran the same command twice on every prompt: context injected twice, and the session captured twice against the user's own extraction allowance. `enable` now looks for it and refuses, naming the entry and the file it lives in; `status` says who registered it rather than reporting none. Matched on the command rather than on a tag, for the reason the hook table already gives: Claude Code rewrites settings and private tags do not survive it. The removal instruction says to delete the entry by hand because Memorable has no command that removes its own hook. `uninstall-hooks` is not a command in 0.5.18; it answers "unknown command". The README said "Memorable, not gstack, owns the captured data and any network access", which answers the question by pointing away from it. It now carries a per-command table of exactly what leaves the machine, in the shape the adopted gbrain section uses, and it is explicit that the hook makes no network call of its own, that every row is the third-party CLI acting under its own consent, and that `gstack-egress` will therefore not show any of it. Under it, the split between what gstack pin-tests (the gating and the wiring) and what is Memorable's claim (storage, sending, and what disable and forget erase). The CHANGELOG entry is removed. This file has never carried an [Unreleased] heading; every entry is a version and a date, written at release. The text is in the pull request for whoever cuts the next one. Three tests added: enable refuses and touches neither consent nor settings when Memorable already holds the hook, status names that registration, and a foreign UserPromptSubmit hook is not mistaken for Memorable's. (cherry picked from commite0899afa8c) * docs: a guide for the Memorable bridge, in the gbrain-sync shape README carries the section and the egress table; this is the page it links to for anyone who wants the whole thing. Same shape as docs/gbrain-sync.md, which is the closest thing in the repo: an optional integration whose interesting questions are all about what leaves the machine and how to turn it off. What it covers that the README cannot at that length: that the hook sees every Claude Code prompt rather than only the ones a skill produced; that capture is a separate consent from this bridge, so turning the bridge off does not turn capture off; what to do when Memorable has already registered the hook itself, which is the common case because its own installer does it; and why the hook has no loud failure path. The egress table is repeated here rather than linked, because the sentence it is answering ("what does this send") is the one somebody arrives on this page already asking. Co-authored-by: Advaiyt Sane <advaiyt.sane@gmail.com> Co-authored-by: Nikhil Krishnaswamy <krishnaswamynikhil@gmail.com> (cherry picked from commit5c108cc0f7) * feat(settings-hook): identity-aware remove-source + read-only list-items remove-source used to inspect only entries still carrying the _gstack_source tag. Claude Code strips that tag when it rewrites settings.json, so an off switch built on remove-source alone silently no-oped on exactly the entries it was written for. Removal is now driven by KNOWN_HOOKS identity for the requested source (tagged or not), keeps the tagged-single-item legacy-stray rule, never touches another source's items, and leaves entries with nothing of ours byte-identical. list-items is the read-only view of the same identity table: one JSON string literal per matching hook command, filters (--owned-by, --command-regex as a JavaScript RegExp) applied inside the JS, empty stdout for no match, and the mutating verbs' exit codes (1 usage, 3 unparseable settings, 4 unexpected shape) so callers can decide mutations from its output without parsing raw command strings. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(config): memorable_recall consent key (on|off, default off, reject-and-preserve) The gstack-side gate for the Memorable UserPromptSubmit bridge. `on` lets a Claude Code hook hand every prompt to a third-party binary, so the key follows the codex_reviews rule: an invalid value is rejected and the stored value kept, never coerced in either direction. Registered in all four places gstack-config keeps in sync (annotated header, DEFAULTS table, the set validator, and both enumeration loops). Memorable's own capture consent (`memorable enable`) is a separate thing gstack never sets. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(hooks): memorable-user-prompt-hook.ts — consent gate, deny veto, HIGH-tier pre-scan, fail-closed receipt, trust envelope; runExternal in spawn-bin The PR's hook exec'd the vendor binary with the full environment and passed its stdout to Claude verbatim. It is now the house pattern: a fail-open bash shim over a .ts twin that (1) gates on the memorable_recall consent key, (2) skips repos whose trust policy is deny or read-only, (3) scans the prompt (raw bytes and decoded string leaves) and refuses to hand over a HIGH-tier credential shape, (4) writes a fail-closed egress receipt naming the local executable it ran, (5) spawns the vendor in its own process group with an allowlisted environment and group-kills it on timeout, (6) accepts only a string additionalContext back, caps it at 8 KiB on a UTF-8 boundary and wraps it in the trust envelope, and (7) records an `output-written` outcome after the stdout write completes. One deadline clock (4.5 s) undercuts Claude Code's 5 s kill and bounds both ledger writes through the new lockBudgetMs option on writeReceipt/writeOutcome (default unchanged). spawn-bin gains runExternal for external executables (detached group, stderr drained, stdin EPIPE handled, stdout capped, win32 refused). The wiring test pins the sink fail-closed and sweeps hosts/. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(gstack-memorable): canonical hook path, no vendor consent, --timeout 5, identity-based status, verified disable, lifecycle lock enable used to bake the hook path from whatever tree the CLI ran in and to run the vendor's own `memorable enable` (its consent for storing AND uploading session traces) before registering anything. It now resolves the canonical install like setup does and refuses when that install does not carry this bridge (version and hook-twin check), registers through the canonical hook manager with --timeout 5, records gstack's own consent in memorable_recall, never executes the vendor, and restores the captured prior state if consent cannot be recorded. disable flips the gate first, removes the entry by identity (tag or no tag), verifies both states and reports partial failure instead of a blended success. status reads only: resolution path, gate, registration by identity (gstack / vendor-own / both / unknown), mismatch lines, receipt count, recent hook errors. enable and disable serialise under a lock with stale takeover. Windows is refused (TODOS.md D21). Exit codes mirror the hook manager (3/4/5). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(setup): --no-team sweep keeps the opt-in gstack-memorable hook `./setup --no-team` finishes its teardown with `prune-stale --all`, which removes every KNOWN_HOOKS item. The Memorable bridge hook is a user-registered opt-in unrelated to team mode, exactly like verify-gate, so it joins the sweep exclusion list. The verify-gate pin now accepts the comma-extended list; a schema-aware case proves the exclusion keeps both opt-ins (tagged or tag-stripped) while the uninstall sweep still removes them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(uninstall): named gstack-memorable arm, vendor-consent notice, honest kept config The identity sweep already removed the Memorable bridge hook as an unnamed stray. It now has a named arm like every other source, so the summary says what went, and says plainly that Memorable's own consent (if the user granted it) is theirs to revoke. Under --keep-state the kept config is set memorable_recall=off so it never claims a hook that is gone. The canonical-paths pins cover the sixth KNOWN_HOOKS row and the new uninstall source. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(egress): memorable-recall row in gstack-egress grants `gstack-egress grants` promises every standing consent in force with the command that revokes it. The Memorable bridge's memorable_recall key is one, so it gets a row: off by default, granted only when `gstack-memorable enable` set it, revoked by `gstack-memorable disable`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(memorable): accurate bridge guide; README row, Docs table, privacy pointer; PROJECT_STRUCTURE The README section becomes one row in the Standalone binaries table (the shape every other binary uses) plus a Docs-table row and one Privacy bullet saying that optional third-party bridges are off by default and receipted. The guide now separates the two consents, says what gstack hands to the vendor binary and what the receipt can and cannot attest, attributes every statement about the vendor's network activity to the vendor, describes the hook manager accurately (identity via KNOWN_HOOKS, list-items vs list-sources, rollback is whole-file), states the Claude Code hot-reload behaviour, adds a troubleshooting runbook, and keeps the contributors' credit in the CHANGELOG idiom. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(todos): Memorable bridge follow-ups Filed from the CEO and eng reviews of the bridge fix-up: the generic third-party hook seam, Windows support (deferred whole), the envelope kind parameter, a vendor payload-minimization contract, a latency and timeout revisit, resolver and canonical-root consolidation, a non-interactive MEDIUM-tier redaction policy for hooks, and adopting list-items at setup's plan-tune check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(settings-hook): list-items --owned-by with --command-regex intersects When both filters are given, an item must satisfy both: owned by the requested source AND matching the pattern. Before, the regex branch skipped every owned row, so the combination could never match. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(gstack-memorable): compat_check reads list-items output before grepping Under pipefail, piping the probe straight into grep -q let a non-zero probe exit mask the match, so a hook manager without list-items was sometimes reported as compatible. Capture the output, then grep. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: coverage for the memorable bridge (remove-source regression for every KNOWN_HOOKS source) - settings-hook: identity removal pinned for each source in KNOWN_HOOKS; list-items unknown flag and combined --owned-by/--command-regex - gstack-memorable: enable/disable failure paths (lock give-up exit 5 with the test-only lock timeout override, consent-write failures guarded by canRevokeWrites, canonical-version mismatch, no-bun status) - hook: non-object JSON, missing cwd, non-ASCII bytes, held-open stdin, shim without bun, stripControl, resolveVendor, runExternal ENOENT - egress-receipt: lockBudgetMs 0 and writeOutcome on garbage input - uninstall: no memorable entry present reports nothing removed Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(memorable): gbrain backend note and the settings-rewrite race Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * perf(redact-engine): line/col by binary search over a per-scan line index lineColAt walked the input from offset 0 for every finding, so a match-dense input (a pasted log full of emails and IPs) cost O(findings x bytes): 128 KiB took ~400 ms and 900 KiB tens of seconds. The line starts are now indexed once per scan, on the first finding, and each finding is a binary search. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(gbrain-repo-policy-client): repoPolicyTier accepts a spawn timeout The policy script spawn was fixed at 10 s, more than twice the memorable hook's whole budget. Callers on their own deadline pass what they can afford; a timeout reads as unreadable and polarity stays the caller's. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(spawn-bin): runExternal resolves on the child's exit, keeps stdin errors advisory - A vendor that exits 0 but leaves a background child holding its pipes was held to the deadline, group-killed and reported as a timeout with its answer dropped. 'exit' now starts a short stdout drain, then resolves with the real exit code and kills whatever still holds the group. - EPIPE on the child's stdin (it answered before reading a large input) is reported as stdinError, separate from error, so a delivered answer is not classified as a spawn failure. - Stdio streams are destroyed and the child unref'd on resolve so a straggler cannot pin the hook process; tail/grace/drain sizes are named constants. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(hooks): memorable hook closes the review army's gaps - Vendor failures are logged even with empty stderr (a silently hanging vendor taxed every prompt invisibly); the stderr tail is withheld when the redaction engine finds a credential or PII shape in it; hook-errors.log is created 0600. - Trust-policy veto fails closed when git cannot run or answer in time (it read as 'no remote' before); the policy script spawn is bounded by the hook's clock; a payload cwd that is not a directory falls back. - Each secret scan is admitted by the deadline clock (the engine's cost grows with match density); stdin is decoded once. - The pre-spawn gate re-check logs a config failure instead of swallowing it; an incomplete stdin read is named as such, not as 'not JSON'. - Carriage returns are stripped with the other controls. - The vendor env allowlist adds the standard proxy, TLS and XDG variables so a vendor behind a corporate proxy or private CA still reaches its service. - A stdin EPIPE on a delivered answer is recorded in the outcome, not treated as a spawn error. - Stage caps and the truncation marker are named constants; a test-only GSTACK_MEMORABLE_TEST_BUDGET_MS can shorten (never widen) the budget. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(gstack-memorable): lock staleness from the directory mtime; honest messages - A contender that looked between the holder's mkdir and its ts write read a missing ts as 0, called the lock stale and reclaimed it; staleness now comes from the lock directory's own mtime (the settings-hook idiom). - The ensure-event failure is no longer labelled 'warning'; the consent-write rollback message says what was actually kept; a removal that left no entry is reported on stdout, not as an error; receipts are counted from the filtered JSON array, not a formatting artefact; the resolution order and lock tuning are named once. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(uninstall): memorable_recall goes off whether or not state is kept gstack-config resolves its root through GSTACK_STATE_ROOT/GSTACK_HOME, which can differ from the STATE_DIR uninstall removes; a full uninstall could leave memorable_recall=on in a config that survived. Flipped only when currently on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: pin the review-army fixes for the memorable bridge Nonce-scoped orphan checks (the system-wide ps grep could see another shard's sleeper); exit-with-lingering-grandchild; advisory stdin EPIPE; withheld stderr; vendor timeout logged with empty stderr; CR stripping; budget seam; rate-limit expiry and 0600 log; unreadable policy store fails closed; file-as-cwd fallback; mtime-based lock staleness and the mkdir gap. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(memorable): vendor environment allowlist and stderr policy; two follow-ups Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(hooks): memorable hook second-pass review fixes - Trust-policy lookup fails closed on any git failure that is not 'no such remote' or 'not a git repository' (a corrupt or unreadable .git/config and dubious ownership exit 128 and used to read as 'no remote'). - pickAdditionalContext takes the first complete top-level JSON object, so a vendor whose background helper appends a line to stdout (or prints a banner first) does not lose its answer. - The hook-errors.log rate limiter keys on a stable string (a vendor's timestamped stderr no longer defeats it); the log is chmod 0600 on every append because sibling hooks create the same file without a mode. - Scan admission is sized by payload bytes (scan() is uninterruptible). - The receipt payload class is a stable token; the prose moved to the docs. - Header, constants and comments match the behaviour (silent skips vs logged refusals; HIGH/MEDIUM withholding; STAGE_CAP_MS scope; runExternal in the spawn-bin header; the ledger lock budget doc). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(gstack-memorable): stale-lock takeover by atomic rename; comments match behaviour Two contenders that both saw a stale lock could both reclaim it with rm -rf; the settings-hook idiom (mv to a private name, exactly one winner) is used instead. The hook-manager fallback comment now says every verb falls back. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: second-pass coverage for the memorable bridge Policy lookup outcomes (plain dir, repo without policy, corrupt .git/config fails closed); repoPolicyTier timeoutMs; line/col at line starts, after blank lines, CRLF and first char; tolerant first-JSON-object parsing; keyed rate limit; uninstall never creates a config just to say off and flips consent in a GSTACK_STATE_ROOT outside the removed state dir. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(memorable): silent skips vs logged refusals; payload class token; D21/D24 anchors Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(hooks): memorable hook survives host termination and brace-bearing banners - The bash shim runs bun as a job and forwards SIGTERM/SIGINT/SIGHUP (bash holds a signal until a foreground child exits); the .ts kills the in-flight vendor's process group on the way out (runExternal exposes the group kill through onSpawn), so a hook the host terminates cannot leave the vendor running with the prompt on its stdin. - The tolerant stdout parser tries every complete top-level object (bounded) and takes the first carrying a string additionalContext, so a banner with braces or quotes, or a progress object, no longer costs the answer. - git runs with LC_ALL=C and the not-a-repository check is anchored to the start of its message: a localized git or a repository path containing the phrase can no longer flip the lookup. - The rate limiter remembers up to 32 live keys, so alternating failures cost two lines, not one per prompt. - Unicode format characters (bidi overrides, zero-width spaces) are stripped from vendor text at egress; the zero-width joiner stays for emoji. - A killed child (timeout, ENOBUFS) resolves on exit without the stdout drain, and the post-kill grace is 100 ms, so the timeout outcome fits the reserve. - The ledger size warning, which the host discards from an exit-0 hook's stderr, is logged where status looks. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(gstack-memorable): failed stale-lock takeover reaches the give-up; disable runs without gstack-config; status shows ledger size Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: host termination kills the vendor group; brace banners, decoys and format characters; non-reclaimable stale lock gives up; ledger line in status Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(memorable): the vendor dies with a terminated hook Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: deterministic stdin EPIPE case for runExternal (child closes stdin, stays alive) Under parallel shard load a child that merely exits fast raced the write and the EPIPE was not always observed; closing the read end first makes it so. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(hooks): outside-model review fixes for the memorable hook and runExternal - Nothing in the vendor's process group outlives the call: the group is killed on every resolve, the clean 'close' path included (a helper the vendor forked with redirected stdio ran on unsupervised before). - A child that already exited when the deadline fires keeps its result; the deadline only ends the drain instead of rewriting a completed exit as a timeout. - The decoded-leaf scan reports when its node/depth bound cut the walk short and the hook refuses the hand-off as unscanned; object keys are scanned too. - git for the trust-policy lookup runs without inherited GIT_* selectors (GIT_DIR, GIT_WORK_TREE, GIT_CONFIG_*), so it inspects the session's repository and no other; a host cancellation kills that git as well. - An unmatched brace in a vendor banner no longer hides the answer after it. - The stderr tail is scanned whole before it is cropped for the log, so a credential's prefix cannot be cropped away from its secret half. - The vendor override reads an empty GSTACK_MEMORABLE_BIN as unset, exactly as bin/gstack-memorable does, so enable checks the binary the hook runs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(gstack-memorable): stale-lock reclaim checks the inode it judged and the owner's liveness; status reports a failed receipt query as unknown Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(uninstall): revoke memorable_recall independently of the hook manager and name a failed revocation Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: group kill on clean close, walk exhaustion refused, GIT_DIR cannot bypass the deny, unmatched-brace banner, whole-tail scan, empty-override parity, uninstall revocation without the hook manager Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(memorable): the process-group guarantee and its setsid boundary Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(release): v1.83.0.0 — Memorable recall bridge, identity-aware hook removal, faster redaction line/col CHANGELOG entry for the Memorable workflow-memory bridge (opt-in, off by default, Claude Code only), the identity-aware remove-source and read-only list-items in the hook manager, the memorable-recall egress sink, runExternal, lockBudgetMs, and the binary-search line/col index in the redaction engine. No migration. Contributed by @AdvaiytSane and @NIkhil-cmd-cmd (#2831). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: stdin EPIPE observation is scheduling-dependent under shard load; pin the invariant instead A delivered answer is never reclassified as a spawn error; when the EPIPE is observed it is reported as stdinError. Whether it is observed before the child's exit resolves the call is not something the test can force. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: list the memorable-recall sink among the fail-closed egress sinks CLAUDE.md and ARCHITECTURE.md enumerate the receipt sinks that refuse to send when the ledger cannot be written; the Memorable bridge's per-prompt hand-off is pinned fail-closed in test/egress-receipt-wiring.test.ts but was missing from both lists. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test: memorable bridge tests pass on a runner that enforces file modes; scanner-proof key fixture - The consent-write-failure tests make the state dir read-only, which also blocked the bridge lock directory (exit 5 before the path under test); the locks dir is pre-created so only the consent write fails. - The unreadable-store test leaves a 0600 directory behind (the policy script chmods the store path); cleanup restores the search bit and the suite's afterEach reopens directories before removing. - The AWS-key-shaped fixture is built by concatenation, as every sibling test does, so the CI credential gate does not read it as a live key. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(gstack-memorable): name an unwritable state directory when the lock cannot be created Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: AdvaiytSane <advaiyt.sane@gmail.com> Co-authored-by: Nikhil Krishnaswamy <krishnaswamynikhil@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Fable 5.1
AdvaiytSane
Nikhil Krishnaswamy
Claude Fable 5.1
AdvaiytSane
Nikhil Krishnaswamy
parent
0530392821
commit
caba78fefa
+20
@@ -0,0 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bash shim — Claude Code hooks run `command` strings via /bin/sh, so this
|
||||
# wrapper makes the TypeScript hook executable via bun. Settings.json
|
||||
# references this file directly (registered by bin/gstack-memorable enable,
|
||||
# never by ./setup).
|
||||
#
|
||||
# FAIL-OPEN: a third-party recall bridge must never block a prompt. Every
|
||||
# failure path — bun missing, script crash — still exits 0 with empty stdout.
|
||||
#
|
||||
# bun runs as a job, not a foreground child: bash holds a SIGTERM until a
|
||||
# foreground child exits, so a host that terminates this shim would leave the
|
||||
# vendor process running with the prompt on its stdin. Forwarded, the signal
|
||||
# reaches the .ts, which kills the vendor's process group on its way out.
|
||||
# `<&0` keeps stdin: bash hands background jobs /dev/null otherwise.
|
||||
HERE="$(cd "$(dirname "$0")" && pwd)" || exit 0
|
||||
bun "$HERE/memorable-user-prompt-hook.ts" <&0 &
|
||||
_child=$!
|
||||
trap 'kill -TERM "$_child" 2>/dev/null' TERM INT HUP
|
||||
wait "$_child" 2>/dev/null || wait "$_child" 2>/dev/null || true
|
||||
exit 0
|
||||
@@ -0,0 +1,593 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* memorable-user-prompt-hook — gstack-mediated bridge from Claude Code's
|
||||
* UserPromptSubmit event to the third-party `memorable` CLI (memorable.sh).
|
||||
*
|
||||
* The vendor's own installer registers `memorable hook user-prompt` directly.
|
||||
* Registering it THROUGH gstack instead buys the user what gstack gives every
|
||||
* other off-machine sink: an explicit consent key, a receipt per attempted
|
||||
* send, a secret pre-scan, a trust envelope around what comes back, healing
|
||||
* and clean removal. This file is that mediation.
|
||||
*
|
||||
* stdin JSON -> cap 1 MiB -> parse -> MEMORABLE=0? -> gate memorable_recall == on?
|
||||
* -> win32? -> trust policy (deny / read-only veto, by session cwd; fail-closed)
|
||||
* -> HIGH-tier secret scan (raw bytes AND decoded string leaves, each admitted by the clock)
|
||||
* -> resolve vendor -> budget >= 500 ms? -> gate re-check
|
||||
* -> receipt (fail-closed: no receipt, no send)
|
||||
* -> VENDOR SPAWN (own process group, allowlisted env, group-killed on timeout)
|
||||
* -> parse vendor JSON -> additionalContext only -> control-strip
|
||||
* -> 8 KiB cap (UTF-8 boundary) -> trust envelope -> stdout (awaited)
|
||||
* -> outcome (bounded by the same clock) -> exit 0
|
||||
* every REFUSAL above is: one rate-limited line in hook-errors.log, empty stdout, exit 0
|
||||
* (gate off, MEMORABLE=0, empty or non-object stdin are silent: nothing was refused).
|
||||
*
|
||||
* CONTRACT
|
||||
* - ALWAYS exits 0 with either one hookSpecificOutput JSON or nothing. The
|
||||
* vendor can never block a prompt or speak as gstack: only a string
|
||||
* `hookSpecificOutput.additionalContext` is accepted from its output.
|
||||
* - One deadline clock (BUDGET_MS) undercuts Claude Code's 5 s hook kill;
|
||||
* every stage, the two ledger writes and the secret scans included, gets
|
||||
* min(cap, remaining). A receipt with no outcome means the host killed us
|
||||
* or the clock ran out (reported as `unknown`), never success.
|
||||
* - Fail-closed on the receipt: if the ledger cannot be written, recall is
|
||||
* skipped for that prompt. What the receipt attests is the bytes handed to
|
||||
* a LOCAL binary running with the user's privileges (host `local:<path>`);
|
||||
* what that binary sends is the vendor's claim.
|
||||
* - The vendor sees an allowlisted environment (PATH, HOME, locale, TMP,
|
||||
* the standard proxy/TLS/XDG variables, MEMORABLE*), never Claude Code's
|
||||
* full env (which can carry API keys).
|
||||
* - The vendor's stderr reaches hook-errors.log only when the redaction
|
||||
* engine finds no HIGH- or MEDIUM-tier shape in it (a CLI that echoes its
|
||||
* input on a parse error would otherwise copy the prompt into the log).
|
||||
* The log is chmod 0600 on every append (sibling hooks share the file).
|
||||
* - If the host terminates the hook mid-flight (SIGTERM/SIGINT/SIGHUP,
|
||||
* forwarded by the bash shim), the vendor's process group is killed on
|
||||
* the way out; the receipt then stands with outcome `unknown`.
|
||||
* - Windows is refused here (no process groups to contain the vendor);
|
||||
* bin/gstack-memorable enable refuses there too. TODOS.md: Windows support (D21).
|
||||
*
|
||||
* Pure helpers are exported for unit tests; main() runs only under import.meta.main.
|
||||
*/
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { runBin, runExternal } from './spawn-bin';
|
||||
import {
|
||||
LEDGER_WARN_BYTES, egressLedgerPath, ledgerSizeWarning, resolveEgressHome, sha256Hex, writeOutcome, writeReceipt,
|
||||
} from '../../../lib/egress-receipt';
|
||||
import { wrapUntrustedTrackerContent } from '../../../lib/tracker-guard';
|
||||
import { scan } from '../../../lib/redact-engine';
|
||||
import { hasRepoPolicyStore, repoPolicyTier } from '../../../lib/gbrain-repo-policy-client';
|
||||
|
||||
export const BUDGET_MS = 4500;
|
||||
export const STDIN_CAP_BYTES = 1024 * 1024;
|
||||
export const OUTPUT_CAP_BYTES = 8192;
|
||||
/** Left on the clock for post-processing, the stdout write and one ledger append after the vendor. */
|
||||
export const RESERVE_MS = 300;
|
||||
/** Below this many ms left before the spawn, the vendor is not started at all. */
|
||||
export const MIN_SPAWN_MS = 500;
|
||||
/** Cap for each pre-spawn subprocess stage (stdin read, gate, git, policy); always min(cap, remaining). */
|
||||
export const STAGE_CAP_MS = 1000;
|
||||
/** Cap for the pre-spawn gate re-check. */
|
||||
export const RECHECK_CAP_MS = 500;
|
||||
/** Below this many ms left, the outcome append is skipped (the receipt stands, outcome reads as unknown). */
|
||||
export const OUTCOME_MIN_MS = 80;
|
||||
/** Kept back from the clock when an outcome append is given the rest of it. */
|
||||
export const OUTCOME_RESERVE_MS = 50;
|
||||
export const LOG_RATE_LIMIT_MS = 10 * 60 * 1000;
|
||||
export const ENVELOPE_SOURCE = 'memorable recall (third-party)';
|
||||
export const SINK = 'memorable-recall';
|
||||
export const CONSENT = 'memorable_recall=on';
|
||||
export const RESOLUTION_ORDER = 'GSTACK_MEMORABLE_BIN, MEMORABLE_BIN, ~/.memorable/bin/memorable, PATH';
|
||||
/** Receipt payload class: a stable token (the prose lives in docs/memorable-workflow-memory.md), so a per-prompt sink does not repeat a sentence per line. */
|
||||
export const PAYLOAD_CLASS = 'claude-user-prompt-json->local-vendor-cli';
|
||||
const HOOK_NAME = 'memorable-user-prompt-hook';
|
||||
/** Per-KiB allowance added to the scan admission check: ~1.5x the measured worst case of scan(). */
|
||||
const SCAN_MS_PER_KIB = 1;
|
||||
/** Distinct rate-limit keys remembered at once (the marker file is rewritten on every log line). */
|
||||
const RATE_LIMIT_KEYS = 32;
|
||||
/** Candidate objects tried by the tolerant stdout parser before giving up (bounds a hostile brace soup). */
|
||||
const JSON_CANDIDATES = 64;
|
||||
const GIT_MAX_BUFFER = 64 * 1024;
|
||||
const TRUNCATION_MARKER = `[truncated by gstack at ${OUTPUT_CAP_BYTES / 1024} KiB]`;
|
||||
|
||||
/** Milliseconds left on a deadline that started at startMs. Pure; unit-tested. */
|
||||
export function budgetFor(startMs: number, nowMs: number, cap: number = BUDGET_MS): number {
|
||||
return Math.max(0, startMs + cap - nowMs);
|
||||
}
|
||||
|
||||
/** The deadline: BUDGET_MS, or a test-only override that can only shorten it. */
|
||||
export function budgetMs(env: Record<string, string | undefined> = process.env): number {
|
||||
const raw = env.GSTACK_MEMORABLE_TEST_BUDGET_MS;
|
||||
const n = raw ? Number(raw) : NaN;
|
||||
return Number.isFinite(n) && n > 0 ? Math.min(n, BUDGET_MS) : BUDGET_MS;
|
||||
}
|
||||
|
||||
/** Truncate to maxBytes of UTF-8 without splitting a multibyte character. */
|
||||
export function capUtf8(text: string, maxBytes: number): { text: string; truncated: boolean } {
|
||||
const buf = Buffer.from(text, 'utf8');
|
||||
if (buf.length <= maxBytes) return { text, truncated: false };
|
||||
let end = maxBytes;
|
||||
while (end > 0 && (buf[end] & 0xc0) === 0x80) end--; // back off to a UTF-8 boundary
|
||||
return { text: buf.subarray(0, end).toString('utf8'), truncated: true };
|
||||
}
|
||||
|
||||
// C0 controls minus tab (9) and newline (10), plus DEL. Carriage return (13)
|
||||
// is stripped too: a CR can visually overwrite earlier text in a rendering of
|
||||
// the injected context while staying one line for the envelope. Built from
|
||||
// char codes so the source file itself carries no control bytes.
|
||||
const cc = (n: number): string => String.fromCharCode(n);
|
||||
const CONTROL_RE = new RegExp(`[${cc(0)}-${cc(8)}${cc(11)}-${cc(31)}${cc(127)}]`, 'g');
|
||||
// Unicode format characters (bidi overrides, zero-width spaces, soft hyphens)
|
||||
// hide text from a reader while the model still sees it; the envelope detects
|
||||
// them but emits the original, so this sink strips them at egress. The
|
||||
// zero-width joiner stays: emoji sequences need it.
|
||||
const FORMAT_RE = /(?!\u200D)\p{Cf}/gu;
|
||||
|
||||
/** Strip control and format characters except newline, tab and ZWJ (the envelope handles the rest). */
|
||||
export function stripControl(text: string): string {
|
||||
return text.replace(CONTROL_RE, '').replace(FORMAT_RE, '');
|
||||
}
|
||||
|
||||
/**
|
||||
* Every string leaf of a parsed JSON value, bounded so a hostile payload
|
||||
* cannot monopolize the clock. `exhausted` is true when the bound cut the
|
||||
* walk short: the caller must then treat the payload as unscanned (and refuse
|
||||
* the hand-off), never as clean.
|
||||
*/
|
||||
export function stringLeavesBounded(value: unknown, maxNodes = 10_000, maxDepth = 32): { leaves: string[]; exhausted: boolean } {
|
||||
const leaves: string[] = [];
|
||||
let nodes = 0;
|
||||
let exhausted = false;
|
||||
const walk = (v: unknown, depth: number): void => {
|
||||
if (nodes++ > maxNodes || depth > maxDepth) { exhausted = true; return; }
|
||||
if (typeof v === 'string') { leaves.push(v); return; }
|
||||
if (Array.isArray(v)) { for (const item of v) walk(item, depth + 1); return; }
|
||||
if (v && typeof v === 'object') {
|
||||
// keys are forwarded bytes too; a credential can sit in one
|
||||
for (const [k, item] of Object.entries(v as Record<string, unknown>)) { leaves.push(k); walk(item, depth + 1); }
|
||||
}
|
||||
};
|
||||
walk(value, 0);
|
||||
return { leaves, exhausted };
|
||||
}
|
||||
|
||||
/** The leaves alone (see stringLeavesBounded). */
|
||||
export function stringLeaves(value: unknown, maxNodes = 10_000, maxDepth = 32): string[] {
|
||||
return stringLeavesBounded(value, maxNodes, maxDepth).leaves;
|
||||
}
|
||||
|
||||
// Identity, locale and temp; the standard proxy, TLS and XDG knobs the vendor
|
||||
// needs to reach its own service through the user's proxy or private CA (it
|
||||
// already gets them from the user's shell); plus MEMORABLE* (its own knobs,
|
||||
// matched by prefix below). Never API keys, never GSTACK_* or CLAUDE_*.
|
||||
const ENV_ALLOW = new Set([
|
||||
'PATH', 'HOME', 'USER', 'LOGNAME', 'SHELL', 'LANG', 'TERM', 'TMPDIR', 'TEMP', 'TMP',
|
||||
'HTTP_PROXY', 'HTTPS_PROXY', 'NO_PROXY', 'http_proxy', 'https_proxy', 'no_proxy',
|
||||
'SSL_CERT_FILE', 'SSL_CERT_DIR', 'NODE_EXTRA_CA_CERTS',
|
||||
'XDG_CONFIG_HOME', 'XDG_DATA_HOME', 'XDG_CACHE_HOME', 'XDG_STATE_HOME',
|
||||
]);
|
||||
|
||||
/** The vendor's environment: an allowlist, never Claude Code's full env. */
|
||||
export function vendorEnv(env: Record<string, string | undefined>): Record<string, string> {
|
||||
const out: Record<string, string> = {};
|
||||
for (const [k, v] of Object.entries(env)) {
|
||||
if (v == null) continue;
|
||||
if (ENV_ALLOW.has(k) || k.startsWith('LC_') || k.startsWith('MEMORABLE')) out[k] = v;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Index of the `}` closing the object that opens at `start`, or -1 when it never closes. */
|
||||
function balancedObjectEnd(raw: string, start: number): number {
|
||||
let depth = 0;
|
||||
let inString = false;
|
||||
let escaped = false;
|
||||
for (let i = start; i < raw.length; i++) {
|
||||
const ch = raw[i];
|
||||
if (inString) {
|
||||
if (escaped) escaped = false;
|
||||
else if (ch === '\\') escaped = true;
|
||||
else if (ch === '"') inString = false;
|
||||
continue;
|
||||
}
|
||||
if (ch === '"') inString = true;
|
||||
else if (ch === '{') depth++;
|
||||
else if (ch === '}' && --depth === 0) return i;
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every complete top-level JSON object in `raw`, in order (at most
|
||||
* JSON_CANDIDATES attempts). A vendor whose background helper logs a line to
|
||||
* the inherited stdout after the answer, or prints a banner before it (even
|
||||
* one with braces in it), must not cost the user the answer. Whole-input JSON
|
||||
* is the normal case and is yielded alone.
|
||||
*/
|
||||
export function* jsonObjects(raw: string): Generator<unknown> {
|
||||
try { yield JSON.parse(raw); return; } catch { /* fall through to the scan */ }
|
||||
let start = raw.indexOf('{');
|
||||
for (let tries = 0; start >= 0 && tries < JSON_CANDIDATES; tries++) {
|
||||
const end = balancedObjectEnd(raw, start);
|
||||
let parsed: unknown;
|
||||
let ok = false;
|
||||
// An unbalanced candidate (a lone brace in a banner) is skipped like an
|
||||
// unparsable one: the complete object after it must still be found.
|
||||
if (end >= 0) { try { parsed = JSON.parse(raw.slice(start, end + 1)); ok = true; } catch { /* a brace in prose */ } }
|
||||
if (ok) { yield parsed; start = raw.indexOf('{', end + 1); }
|
||||
else start = raw.indexOf('{', start + 1);
|
||||
}
|
||||
}
|
||||
|
||||
/** The first complete top-level JSON object in `raw`, or null. */
|
||||
export function firstJsonObject(raw: string): unknown {
|
||||
for (const obj of jsonObjects(raw)) return obj;
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Only a string hookSpecificOutput.additionalContext survives; decision/continue/systemMessage are dropped. */
|
||||
export function pickAdditionalContext(raw: string): string | null {
|
||||
for (const parsed of jsonObjects(raw)) {
|
||||
const hso = (parsed as { hookSpecificOutput?: { additionalContext?: unknown } } | null)?.hookSpecificOutput;
|
||||
const ctx = hso?.additionalContext;
|
||||
if (typeof ctx === 'string' && ctx.length > 0) return ctx;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Cap + envelope: the text Claude will see. */
|
||||
export function renderContext(vendorText: string): string {
|
||||
const { text, truncated } = capUtf8(stripControl(vendorText), OUTPUT_CAP_BYTES);
|
||||
const body = truncated ? `${text}\n${TRUNCATION_MARKER}` : text;
|
||||
return wrapUntrustedTrackerContent(body, ENVELOPE_SOURCE);
|
||||
}
|
||||
|
||||
/**
|
||||
* The vendor's stderr tail as it may appear in hook-errors.log: control-stripped,
|
||||
* whitespace-collapsed, last 300 chars, and WITHHELD when the redaction engine
|
||||
* finds a HIGH or MEDIUM shape in it (a CLI that echoes its input on a parse
|
||||
* error would otherwise copy prompt text into a log the pre-scan only cleared
|
||||
* of HIGH-tier shapes).
|
||||
*/
|
||||
export function safeStderrTail(tail: string): string {
|
||||
// Scan everything runExternal kept, THEN crop for the log: cropping first
|
||||
// could cut a credential's identifying prefix off and log its secret half.
|
||||
const whole = stripControl(tail).replace(/\s+/g, ' ').trim();
|
||||
if (!whole) return '';
|
||||
const r = scan(whole, { repoVisibility: 'unknown' });
|
||||
const n = r.counts.HIGH + r.counts.MEDIUM;
|
||||
return r.oversize || n > 0 ? `[stderr withheld: ${n} redaction finding(s)]` : whole.slice(-300);
|
||||
}
|
||||
|
||||
function stripQuotes(v: string): string {
|
||||
return v.trim().replace(/^"(.*)"$/, '$1');
|
||||
}
|
||||
|
||||
function executable(p: string): boolean {
|
||||
try {
|
||||
const st = fs.statSync(p);
|
||||
if (!st.isFile()) return false;
|
||||
if (process.platform !== 'win32') fs.accessSync(p, fs.constants.X_OK);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function isDirectory(p: string): boolean {
|
||||
try { return fs.statSync(p).isDirectory(); } catch { return false; }
|
||||
}
|
||||
|
||||
/**
|
||||
* GSTACK_MEMORABLE_BIN -> MEMORABLE_BIN -> ~/.memorable/bin/memorable -> PATH.
|
||||
* An explicit override that does not resolve is an error (null), never a
|
||||
* fall-through to something else (lib/claude-bin.ts contract).
|
||||
*/
|
||||
export function resolveVendor(env: Record<string, string | undefined>, homeDir: string): string | null {
|
||||
// Empty means unset, exactly as bash's ${GSTACK_MEMORABLE_BIN:-${MEMORABLE_BIN:-}} reads it
|
||||
// in bin/gstack-memorable: the binary enable checked is the binary the hook runs.
|
||||
const override = (env.GSTACK_MEMORABLE_BIN ?? '').trim() || (env.MEMORABLE_BIN ?? '').trim();
|
||||
if (override) {
|
||||
const o = stripQuotes(override);
|
||||
const resolved = path.isAbsolute(o) ? o : (Bun.which(o) ?? null);
|
||||
return resolved && executable(resolved) ? resolved : null;
|
||||
}
|
||||
const pinned = path.join(homeDir, '.memorable', 'bin', 'memorable');
|
||||
if (executable(pinned)) return pinned;
|
||||
const onPath = Bun.which('memorable');
|
||||
return onPath && executable(onPath) ? onPath : null;
|
||||
}
|
||||
|
||||
function stateRoot(): string {
|
||||
return process.env.GSTACK_STATE_ROOT || process.env.GSTACK_HOME || process.env.GSTACK_STATE_DIR
|
||||
|| path.join(os.homedir(), '.gstack');
|
||||
}
|
||||
|
||||
/**
|
||||
* Best-effort, rate-limited: a message with the same `key` (default: the
|
||||
* message itself) within LOG_RATE_LIMIT_MS is not re-logged, so a vendor that
|
||||
* fails on every prompt with a different timestamp in its stderr still costs
|
||||
* one line per ten minutes, and two alternating failures cost two. The marker
|
||||
* (up to RATE_LIMIT_KEYS live `digest:ts` lines) is per hook so hooks never
|
||||
* contend. The log is chmod 0600 on every append: sibling hooks create the
|
||||
* same file without a mode, and it can name the session's cwd and vendor
|
||||
* diagnostics.
|
||||
*/
|
||||
export function logHookError(msg: string, nowMs: number = Date.now(), key: string = msg): void {
|
||||
try {
|
||||
const root = stateRoot();
|
||||
fs.mkdirSync(root, { recursive: true });
|
||||
const marker = path.join(root, `hook-errors.${HOOK_NAME}.last`);
|
||||
const digest = sha256Hex(key).slice(0, 16);
|
||||
const live: string[] = [];
|
||||
try {
|
||||
for (const line of fs.readFileSync(marker, 'utf8').split('\n')) {
|
||||
const [d, ts] = line.trim().split(':');
|
||||
if (!d || !ts || nowMs - Number(ts) >= LOG_RATE_LIMIT_MS) continue;
|
||||
if (d === digest) return;
|
||||
live.push(line.trim());
|
||||
}
|
||||
} catch { /* no marker yet */ }
|
||||
live.push(`${digest}:${nowMs}`);
|
||||
fs.writeFileSync(marker, `${live.slice(-RATE_LIMIT_KEYS).join('\n')}\n`, { mode: 0o600 });
|
||||
const log = path.join(root, 'hook-errors.log');
|
||||
fs.appendFileSync(log, `${new Date(nowMs).toISOString()} ${HOOK_NAME}: ${msg}\n`, { mode: 0o600 });
|
||||
if (process.platform !== 'win32') { try { fs.chmodSync(log, 0o600); } catch { /* not ours to tighten */ } }
|
||||
} catch {
|
||||
// best-effort; never block the session because logging failed
|
||||
}
|
||||
}
|
||||
|
||||
function readStdin(maxBytes: number, timeoutMs: number): Promise<{ buf: Buffer; oversize: boolean; timedOut: boolean }> {
|
||||
return new Promise((resolve) => {
|
||||
const chunks: Buffer[] = [];
|
||||
let total = 0;
|
||||
let done = false;
|
||||
let oversize = false;
|
||||
const finish = (timedOut: boolean): void => {
|
||||
if (done) return;
|
||||
done = true;
|
||||
clearTimeout(timer);
|
||||
try { process.stdin.destroy(); } catch { /* already closed */ }
|
||||
resolve({ buf: Buffer.concat(chunks), oversize, timedOut });
|
||||
};
|
||||
const timer = setTimeout(() => finish(true), Math.max(1, timeoutMs));
|
||||
process.stdin.on('data', (d: Buffer | string) => {
|
||||
if (done) return;
|
||||
const chunk = typeof d === 'string' ? Buffer.from(d, 'utf8') : d;
|
||||
total += chunk.length;
|
||||
if (total > maxBytes) { oversize = true; finish(false); return; }
|
||||
chunks.push(chunk);
|
||||
});
|
||||
process.stdin.on('end', () => finish(false));
|
||||
process.stdin.on('error', () => finish(false));
|
||||
});
|
||||
}
|
||||
|
||||
function gateIsOn(timeoutMs: number): 'on' | 'off' | 'error' {
|
||||
const r = runBin('gstack-config', ['get', 'memorable_recall'], { encoding: 'utf8', timeout: Math.max(1, timeoutMs), env: process.env });
|
||||
if (r.status !== 0) return 'error';
|
||||
return String(r.stdout ?? '').trim() === 'on' ? 'on' : 'off';
|
||||
}
|
||||
|
||||
/**
|
||||
* git exit 2 = no such remote; exit 128 whose message STARTS with this text
|
||||
* (git is run with LC_ALL=C so the text is English) = not inside a repository.
|
||||
* Anchored, never a substring test: other exit-128 messages echo the
|
||||
* repository path, which a directory name could make carry the phrase.
|
||||
*/
|
||||
const GIT_NO_REMOTE = 2;
|
||||
const GIT_FATAL = 128;
|
||||
const NOT_A_REPO_RE = /^fatal: not a git repository\b/;
|
||||
|
||||
/** Kills the in-flight detached child (git for the policy lookup, then the vendor); read by the signal handlers. */
|
||||
let killInflight: (() => void) | null = null;
|
||||
|
||||
/**
|
||||
* git's environment for the policy lookup: English messages, and NO inherited
|
||||
* GIT_* repository selectors (GIT_DIR, GIT_WORK_TREE, GIT_COMMON_DIR,
|
||||
* GIT_CONFIG_*, ...): cwd does not override them, so an inherited GIT_DIR
|
||||
* would make the veto inspect a different repository than the one the
|
||||
* session works in. Exported for tests.
|
||||
*/
|
||||
export function gitEnv(env: Record<string, string | undefined>): Record<string, string> {
|
||||
const out: Record<string, string> = {};
|
||||
for (const [k, v] of Object.entries(env)) {
|
||||
if (v == null || k.startsWith('GIT_')) continue;
|
||||
out[k] = v;
|
||||
}
|
||||
out.LC_ALL = 'C'; out.LANGUAGE = ''; out.LC_MESSAGES = 'C';
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Trust-policy veto by the session's repo (keyed by its origin remote).
|
||||
* Every half fails CLOSED once a store exists: a git that could not run or
|
||||
* answer in time, a git that could not read the repository (corrupt or
|
||||
* unreadable config, dubious ownership: exit 128 without "not a git
|
||||
* repository"), or a store that could not be read is a failed lookup
|
||||
* (`error`), never "no remote". Only "no such remote" and "not a repository"
|
||||
* mean nothing can be set for this directory.
|
||||
*/
|
||||
async function policyVeto(cwd: string, timeoutMs: number, remaining: () => number): Promise<'ok' | 'skip' | 'error'> {
|
||||
if (!hasRepoPolicyStore()) return 'ok';
|
||||
const git = await runExternal('git', ['remote', 'get-url', 'origin'], {
|
||||
cwd, timeoutMs: Math.max(1, timeoutMs), maxBuffer: GIT_MAX_BUFFER, env: gitEnv(process.env),
|
||||
onSpawn: (kill) => { killInflight = kill; }, // a host cancellation must not leak this git either
|
||||
});
|
||||
killInflight = null;
|
||||
if (git.timedOut || git.error) return 'error';
|
||||
if (git.status === GIT_NO_REMOTE) return 'ok';
|
||||
if (git.status === GIT_FATAL && NOT_A_REPO_RE.test(git.stderrTail.trim())) return 'ok';
|
||||
if (git.status !== 0) return 'error';
|
||||
const url = git.stdout.toString('utf8').trim();
|
||||
if (!url) return 'ok';
|
||||
const res = repoPolicyTier(url, process.env, Math.max(1, Math.min(STAGE_CAP_MS, remaining())));
|
||||
if (res.error) return 'error';
|
||||
// `deny` and `read-only` are the tiers a user picks so a repo's content
|
||||
// never lands in a shared store; a third-party memory service is one.
|
||||
return res.tier === 'deny' || res.tier === 'read-only' ? 'skip' : 'ok';
|
||||
}
|
||||
|
||||
function writeStdout(text: string): Promise<void> {
|
||||
return new Promise((resolve) => { process.stdout.write(text, () => resolve()); });
|
||||
}
|
||||
|
||||
|
||||
/** Best-effort: the ledger's size warning goes to stderr, which the host discards for an exit-0 hook; log it where `status` looks. */
|
||||
function noteLedgerSize(): void {
|
||||
try {
|
||||
const ledger = egressLedgerPath(resolveEgressHome());
|
||||
const size = fs.statSync(ledger).size;
|
||||
if (size > LEDGER_WARN_BYTES) logHookError(ledgerSizeWarning(ledger, size).replace(/\s+/g, ' ').trim(), Date.now(), 'ledger-size');
|
||||
} catch { /* no ledger yet */ }
|
||||
}
|
||||
|
||||
export async function main(): Promise<void> {
|
||||
const start = Date.now();
|
||||
const cap = budgetMs();
|
||||
const remaining = (): number => budgetFor(start, Date.now(), cap);
|
||||
|
||||
const stdin = await readStdin(STDIN_CAP_BYTES, Math.min(STAGE_CAP_MS, remaining()));
|
||||
if (stdin.oversize) { logHookError(`oversize: stdin exceeded ${STDIN_CAP_BYTES / (1024 * 1024)} MiB, recall skipped`); return; }
|
||||
const raw = stdin.buf;
|
||||
if (raw.length === 0) return;
|
||||
const rawText = raw.toString('utf8');
|
||||
let payload: unknown;
|
||||
try { payload = JSON.parse(rawText); } catch {
|
||||
logHookError(stdin.timedOut
|
||||
? 'stdin was not closed within the read budget (incomplete JSON), recall skipped'
|
||||
: 'stdin was not JSON, recall skipped');
|
||||
return;
|
||||
}
|
||||
if (!payload || typeof payload !== 'object') return;
|
||||
|
||||
if (process.env.MEMORABLE === '0') return; // the vendor's own kill switch
|
||||
|
||||
const gate = gateIsOn(Math.min(STAGE_CAP_MS, remaining()));
|
||||
if (gate === 'error') { logHookError('gstack-config get memorable_recall failed, recall skipped (fail-closed)'); return; }
|
||||
if (gate !== 'on') return;
|
||||
|
||||
if (process.platform === 'win32') { logHookError('Windows is not supported by this bridge yet (TODOS.md D21), recall skipped'); return; }
|
||||
|
||||
const payloadCwd = (payload as { cwd?: unknown }).cwd;
|
||||
const cwd = typeof payloadCwd === 'string' && isDirectory(payloadCwd) ? payloadCwd : process.cwd();
|
||||
const veto = await policyVeto(cwd, Math.min(STAGE_CAP_MS, remaining()), remaining);
|
||||
if (veto === 'skip') { logHookError(`trust policy for ${cwd} is deny or read-only, recall skipped`, Date.now(), 'trust policy skip'); return; }
|
||||
if (veto === 'error') { logHookError('trust policy lookup failed (store or repository unreadable), recall skipped (fail-closed)'); return; }
|
||||
|
||||
// HIGH-tier pre-scan over the raw text and the decoded string leaves (a JSON
|
||||
// escape must not hide a key). scan() is synchronous and uninterruptible and
|
||||
// its cost is roughly linear in bytes, so each scan is admitted by the clock
|
||||
// with a size-derived allowance: a scan we cannot afford skips recall
|
||||
// instead of letting the host kill us mid-scan.
|
||||
const walked = stringLeavesBounded(payload);
|
||||
if (walked.exhausted) {
|
||||
// A payload too deep or too wide to walk is unscanned, not clean.
|
||||
logHookError('refused:payload-too-complex: the prompt JSON exceeded the scan walk bounds, nothing handed to the vendor');
|
||||
return;
|
||||
}
|
||||
const leaves = walked.leaves.join('\n');
|
||||
for (const text of [rawText, leaves]) {
|
||||
const allowance = MIN_SPAWN_MS + Math.ceil(Buffer.byteLength(text, 'utf8') / 1024) * SCAN_MS_PER_KIB;
|
||||
if (remaining() < allowance) { logHookError('budget-exhausted before the secret scan, recall skipped'); return; }
|
||||
const result = scan(text, { repoVisibility: 'unknown' });
|
||||
if (result.oversize || result.counts.HIGH > 0) {
|
||||
logHookError('refused:redaction-high: the prompt carries a HIGH-tier credential shape, nothing handed to the vendor');
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
const vendor = resolveVendor(process.env, os.homedir());
|
||||
if (!vendor) { logHookError(`memorable CLI not found (checked ${RESOLUTION_ORDER}), recall skipped`); return; }
|
||||
|
||||
if (remaining() < MIN_SPAWN_MS) { logHookError('budget-exhausted before the vendor spawn, recall skipped'); return; }
|
||||
const again = gateIsOn(Math.min(RECHECK_CAP_MS, remaining()));
|
||||
if (again === 'error') { logHookError('gstack-config get memorable_recall failed on the pre-spawn re-check, recall skipped (fail-closed)'); return; }
|
||||
if (again !== 'on') return; // a disable that landed while we worked wins
|
||||
|
||||
let receiptId: string;
|
||||
try {
|
||||
const { id } = writeReceipt({
|
||||
sink: SINK,
|
||||
host: `local:${vendor}`,
|
||||
payloadClass: PAYLOAD_CLASS,
|
||||
bytes: raw.length,
|
||||
sha256: sha256Hex(raw),
|
||||
consent: CONSENT,
|
||||
lockBudgetMs: Math.max(0, remaining() - RESERVE_MS),
|
||||
});
|
||||
receiptId = id;
|
||||
noteLedgerSize();
|
||||
} catch (err) {
|
||||
// fail-closed: no receipt, no send
|
||||
const why = err instanceof Error ? err.message : String(err);
|
||||
logHookError(`refused:receipt-unwritable: ${why}`);
|
||||
process.stderr.write(`gstack: memorable recall skipped, the egress receipt could not be written (${why}). See gstack-egress.\n`);
|
||||
return;
|
||||
}
|
||||
if (remaining() < MIN_SPAWN_MS) {
|
||||
logHookError('budget-exhausted after the receipt, recall skipped');
|
||||
try { writeOutcome({ receipt: receiptId, status: 'budget-exhausted', lockBudgetMs: Math.max(0, remaining() - OUTCOME_RESERVE_MS) }); } catch { /* bookkeeping */ }
|
||||
return;
|
||||
}
|
||||
|
||||
const gstackMs = Date.now() - start;
|
||||
// VENDOR SPAWN: everything above is gstack's own boundary; from here the bytes are the vendor's.
|
||||
const r = await runExternal(vendor, ['hook', 'user-prompt'], {
|
||||
input: raw,
|
||||
timeoutMs: Math.max(1, remaining() - RESERVE_MS),
|
||||
maxBuffer: 1024 * 1024,
|
||||
env: vendorEnv(process.env),
|
||||
cwd,
|
||||
onSpawn: (kill) => { killInflight = kill; },
|
||||
});
|
||||
killInflight = null;
|
||||
|
||||
let status: string;
|
||||
let delivered = false;
|
||||
if (r.timedOut) status = 'timeout';
|
||||
else if (r.error) status = `spawn-error:${r.error}`;
|
||||
else if (r.status !== 0) status = `exit:${r.status} injected=no`;
|
||||
else {
|
||||
const ctx = pickAdditionalContext(r.stdout.toString('utf8'));
|
||||
if (!ctx) status = 'exit:0 injected=no';
|
||||
else {
|
||||
const rendered = renderContext(ctx);
|
||||
const out = JSON.stringify({ hookSpecificOutput: { hookEventName: 'UserPromptSubmit', additionalContext: rendered } });
|
||||
await writeStdout(out);
|
||||
delivered = true;
|
||||
status = `exit:0 output-written bytes=${Buffer.byteLength(rendered, 'utf8')} gstack_ms=${gstackMs}`;
|
||||
}
|
||||
}
|
||||
// A vendor that exited before reading its stdin (EPIPE) is advisory when it
|
||||
// still answered; the outcome records it, the answer is kept.
|
||||
if (r.stdinError) status += ` stdin=${r.stdinError}`;
|
||||
if (!delivered && (r.timedOut || r.error || r.status !== 0)) {
|
||||
// Logged whether or not the vendor said anything: a silently hanging
|
||||
// vendor taxes every prompt and must show up in `gstack-memorable status`.
|
||||
const tail = safeStderrTail(r.stderrTail);
|
||||
logHookError(`vendor ${status}${tail ? `: ${tail}` : ''}`, Date.now(), `vendor ${status}`);
|
||||
}
|
||||
// The vendor's timeout already left RESERVE_MS on the clock for exactly
|
||||
// this: the stdout write above and one bounded ledger append.
|
||||
if (remaining() > OUTCOME_MIN_MS) {
|
||||
try { writeOutcome({ receipt: receiptId, status, lockBudgetMs: Math.max(0, remaining() - OUTCOME_RESERVE_MS) }); } catch { /* the receipt is the invariant; the outcome is bookkeeping */ }
|
||||
}
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
for (const sig of ['SIGTERM', 'SIGINT', 'SIGHUP'] as const) {
|
||||
process.on(sig, () => {
|
||||
// The host is ending us (its 5 s hook kill, or a session teardown): the
|
||||
// vendor must not outlive the hook that spawned it.
|
||||
if (killInflight) { killInflight(); killInflight = null; }
|
||||
logHookError(`terminated by ${sig} mid-flight; the vendor process group was killed, the receipt (if any) reads unknown`, Date.now(), 'terminated');
|
||||
process.exit(0);
|
||||
});
|
||||
}
|
||||
main()
|
||||
.catch((err) => logHookError(`unexpected: ${err instanceof Error ? (err.stack ?? err.message) : String(err)}`))
|
||||
.finally(() => { process.exitCode = 0; });
|
||||
}
|
||||
@@ -8,11 +8,15 @@
|
||||
* is the correct conversion. (ENOENT before the bin ever ran.)
|
||||
* 2. `bin/gstack-*` are extensionless bash scripts. Windows has no shebang
|
||||
* support, so they must be handed to bash explicitly.
|
||||
*
|
||||
* Also home to runExternal: the contained runner for EXTERNAL executables
|
||||
* (third-party binaries a hook hands data to; see its doc comment). Unlike
|
||||
* runBin it refuses win32, because its guarantee is process-group containment.
|
||||
*/
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
import { spawnSync, type SpawnSyncOptions } from 'child_process';
|
||||
import { spawn, spawnSync, type SpawnSyncOptions } from 'child_process';
|
||||
|
||||
// Forward slashes on purpose: Bun's spawnSync on Windows returns ENOENT for a
|
||||
// backslash exe path containing spaces.
|
||||
@@ -41,3 +45,155 @@ export function runBin(name: string, args: string[], opts: SpawnSyncOptions) {
|
||||
? spawnSync(bashExe(), [bin, ...args], opts)
|
||||
: spawnSync(bin, args, opts);
|
||||
}
|
||||
|
||||
/** Kept tail of the child's stderr, for the caller's error log. */
|
||||
const STDERR_TAIL_BYTES = 500;
|
||||
/** After a group kill, how long to wait for 'exit'/'close' before resolving anyway (kept under the callers' post-spawn reserve). */
|
||||
const KILL_GRACE_MS = 100;
|
||||
/** After the direct child exits, how long to keep draining stdout before resolving. */
|
||||
const EXIT_DRAIN_MS = 150;
|
||||
|
||||
export interface RunExternalOptions {
|
||||
/** bytes written to the child's stdin, then stdin is closed */
|
||||
input?: Buffer | string;
|
||||
/** wall-clock limit; on expiry the child's whole process group is SIGKILLed */
|
||||
timeoutMs: number;
|
||||
/** stdout cap in bytes; exceeding it kills the group and reports error 'ENOBUFS' (default 1 MiB) */
|
||||
maxBuffer?: number;
|
||||
/** the child's COMPLETE environment (callers allowlist; never pass process.env for a third-party binary) */
|
||||
env?: Record<string, string | undefined>;
|
||||
cwd?: string;
|
||||
/** called once the child is running with a function that SIGKILLs its whole process group (for a caller's signal handler) */
|
||||
onSpawn?: (killGroup: () => void) => void;
|
||||
/** test seam: override process.platform */
|
||||
platform?: NodeJS.Platform;
|
||||
}
|
||||
|
||||
export interface RunExternalResult {
|
||||
status: number | null;
|
||||
signal: NodeJS.Signals | null;
|
||||
stdout: Buffer;
|
||||
/** last STDERR_TAIL_BYTES of stderr, for the error log — never forwarded */
|
||||
stderrTail: string;
|
||||
/** 'EPLATFORM' (win32 unsupported), 'ENOBUFS', 'ETIMEDOUT', or a spawn errno */
|
||||
error?: string;
|
||||
/**
|
||||
* errno from writing the child's stdin (EPIPE when it exits before reading
|
||||
* a large input). Advisory and separate from `error`: a child that exited 0
|
||||
* with output still answered.
|
||||
*/
|
||||
stdinError?: string;
|
||||
timedOut: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run an EXTERNAL executable (not a gstack bin) with containment a hook can
|
||||
* rely on:
|
||||
* - `detached: true` makes the child a process-group leader, so a timeout
|
||||
* kills the whole group (`process.kill(-pid)`) — a fork-style vendor shim
|
||||
* cannot outlive the reported timeout the way a bare child kill allows.
|
||||
* - resolves when the DIRECT child exits (after a short stdout drain), not
|
||||
* only on 'close': a child that exits 0 but leaves a background process
|
||||
* holding its pipes gets its output delivered and the straggler group-
|
||||
* killed, instead of being reported as a timeout with its answer dropped.
|
||||
* - NOTHING in the group outlives the call: the group is killed on every
|
||||
* resolve, including a clean 'close' (a helper the child forked with its
|
||||
* stdio redirected would otherwise run on unsupervised). A child that
|
||||
* must leave a daemon behind has to setsid it; that is the child's
|
||||
* explicit choice, visible in its own code, not an accident of ours.
|
||||
* - a child that has already exited when the deadline fires keeps its
|
||||
* result: the deadline then ends the drain, it does not rewrite a
|
||||
* completed exit as a timeout.
|
||||
* - stderr is drained continuously (an undrained pipe blocks a noisy child
|
||||
* before it writes stdout) and only its tail is kept, never forwarded.
|
||||
* - stdin gets an error listener, so a child that exits before reading a
|
||||
* large input surfaces EPIPE as `stdinError`, not an unhandled event.
|
||||
* - stdout is capped; the cap kills the group and reports ENOBUFS.
|
||||
* - win32 is refused ('EPLATFORM'): there are no process groups to kill, so
|
||||
* the containment guarantee cannot be given (Windows support for the
|
||||
* bridges that use this is tracked in TODOS.md).
|
||||
* Async on purpose: spawnSync can only signal the direct child.
|
||||
*/
|
||||
export function runExternal(exe: string, args: string[], opts: RunExternalOptions): Promise<RunExternalResult> {
|
||||
const platform = opts.platform ?? process.platform;
|
||||
const maxBuffer = opts.maxBuffer ?? 1024 * 1024;
|
||||
const empty = (error: string): RunExternalResult =>
|
||||
({ status: null, signal: null, stdout: Buffer.alloc(0), stderrTail: '', error, timedOut: false });
|
||||
if (platform === 'win32') return Promise.resolve(empty('EPLATFORM'));
|
||||
return new Promise((resolve) => {
|
||||
let child: ReturnType<typeof spawn>;
|
||||
try {
|
||||
child = spawn(exe, args, {
|
||||
detached: true,
|
||||
cwd: opts.cwd,
|
||||
env: opts.env as NodeJS.ProcessEnv | undefined,
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
});
|
||||
} catch (e) {
|
||||
resolve(empty((e as NodeJS.ErrnoException)?.code ?? 'ESPAWN'));
|
||||
return;
|
||||
}
|
||||
const chunks: Buffer[] = [];
|
||||
let total = 0;
|
||||
let stderrTail = '';
|
||||
let error: string | undefined;
|
||||
let stdinError: string | undefined;
|
||||
let timedOut = false;
|
||||
let done = false;
|
||||
let exited: { code: number | null; signal: NodeJS.Signals | null } | null = null;
|
||||
let graceTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
let drainTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
const killGroup = (): void => {
|
||||
try { if (child.pid) process.kill(-child.pid, 'SIGKILL'); } catch { /* group already gone */ }
|
||||
try { child.kill('SIGKILL'); } catch { /* already gone */ }
|
||||
};
|
||||
const finish = (status: number | null, signal: NodeJS.Signals | null): void => {
|
||||
if (done) return;
|
||||
done = true;
|
||||
clearTimeout(timer);
|
||||
if (graceTimer) clearTimeout(graceTimer);
|
||||
if (drainTimer) clearTimeout(drainTimer);
|
||||
// Nothing in the group outlives the call; a straggler holding our pipes
|
||||
// must not pin this process either.
|
||||
killGroup();
|
||||
for (const s of [child.stdout, child.stderr, child.stdin]) { try { s?.destroy(); } catch { /* closed */ } }
|
||||
try { child.unref(); } catch { /* fine */ }
|
||||
resolve({ status, signal, stdout: Buffer.concat(chunks), stderrTail, ...(stdinError ? { stdinError } : {}), error, timedOut });
|
||||
};
|
||||
const timer = setTimeout(() => {
|
||||
// The child already answered and exited; the deadline only ends the drain.
|
||||
if (exited) { finish(exited.code, exited.signal); return; }
|
||||
timedOut = true;
|
||||
error = error ?? 'ETIMEDOUT';
|
||||
killGroup();
|
||||
// If 'exit' never arrives (a grandchild holding the pipes open past the
|
||||
// kill), resolve anyway: the caller's own deadline is what matters.
|
||||
graceTimer = setTimeout(() => finish(null, 'SIGKILL'), KILL_GRACE_MS);
|
||||
}, Math.max(1, opts.timeoutMs));
|
||||
opts.onSpawn?.(killGroup);
|
||||
child.on('error', (e) => { error = (e as NodeJS.ErrnoException)?.code ?? 'ESPAWN'; finish(null, null); });
|
||||
child.stdout?.on('data', (d: Buffer) => {
|
||||
if (done) return;
|
||||
total += d.length;
|
||||
if (total > maxBuffer) { error = 'ENOBUFS'; killGroup(); return; }
|
||||
chunks.push(d);
|
||||
});
|
||||
child.stderr?.on('data', (d: Buffer) => { stderrTail = (stderrTail + d.toString('utf8')).slice(-STDERR_TAIL_BYTES); });
|
||||
child.on('exit', (code, signal) => {
|
||||
exited = { code, signal };
|
||||
if (done) return;
|
||||
// A killed child (timeout, ENOBUFS) has nothing worth draining: resolve
|
||||
// now so the caller keeps its post-spawn reserve.
|
||||
if (timedOut || error) { finish(code, signal); return; }
|
||||
// stdio may still be open (a background grandchild inherited the pipes):
|
||||
// drain what the child itself wrote, then resolve with its real exit and
|
||||
// kill whatever is still holding the group.
|
||||
drainTimer = setTimeout(() => { killGroup(); finish(code, signal); }, EXIT_DRAIN_MS);
|
||||
});
|
||||
child.on('close', (code, signal) => finish(code, signal));
|
||||
if (child.stdin) {
|
||||
child.stdin.on('error', (e) => { stdinError = stdinError ?? ((e as NodeJS.ErrnoException)?.code ?? 'EPIPE'); });
|
||||
if (opts.input !== undefined) child.stdin.end(opts.input); else child.stdin.end();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user