Files
gstack/bin/gstack-brain-restore
T
Garry Tan 9dbaf906cf feat(v1.9.0.0): gbrain-sync — cross-machine gstack memory (#1151)
* feat(gbrain-sync): queue primitives + writer shims

Adds bin/gstack-brain-enqueue (atomic append to sync queue) and
bin/gstack-jsonl-merge (git merge driver, ts-sort with SHA-256 fallback).
Wires one backgrounded enqueue call into learnings-log, timeline-log,
review-log, and developer-profile --migrate. question-log and
question-preferences stay local per Codex v2 decision.

gstack-config gains gbrain_sync_mode (off/artifacts-only/full) and
gbrain_sync_mode_prompted keys, plus GSTACK_HOME env alignment so
tests don't leak into real ~/.gstack/config.yaml.

* feat(gbrain-sync): --once drain + secret scan + push

bin/gstack-brain-sync is the core sync binary. Subcommands: --once
(drain queue, allowlist-filter, privacy-class-filter, secret-scan
staged diff, commit with template, push with fetch+merge retry),
--status, --skip-file <path>, --drop-queue --yes, --discover-new
(cursor-based detection of artifact writes that skip the shim).

Secret regex families: AWS keys, GitHub tokens (ghp_/gho_/ghu_/ghs_/
ghr_/github_pat_), OpenAI sk-, PEM blocks, JWTs, bearer-token-in-JSON.
On hit: unstage, preserve queue, print remediation hint (--skip-file
or edit), exit clean. No daemon — invoked by preamble at skill
boundaries.

* feat(gbrain-sync): init, restore, uninstall, consumer registry

bin/gstack-brain-init: idempotent first-run. git init ~/.gstack/,
.gitignore=*, canonical .brain-allowlist + .brain-privacy-map.json,
pre-commit secret-scan hook (defense-in-depth), merge driver registration
via git config, gh repo create --private OR arbitrary --remote <url>,
initial push, ~/.gstack-brain-remote.txt for new-machine discovery,
GBrain consumer registration via HTTP POST.

bin/gstack-brain-restore: safe new-machine bootstrap. Refuses clobber
of existing allowlisted files, clones to staging, rsync-copies tracked
files, re-registers merge drivers (required — not cloned from remote),
rehydrates consumers.json, prompts for per-consumer tokens.

bin/gstack-brain-uninstall: clean off-ramp. Removes .git + .brain-*
files + consumers.json + config keys. Preserves user data (learnings,
plans, retros, profile). Optional --delete-remote for GitHub repos.

bin/gstack-brain-consumer + bin/gstack-brain-reader (symlink alias):
registry management. Internal 'consumer' term; user-facing 'reader'
per DX review decision.

* feat(gbrain-sync): preamble block — privacy gate + boundary sync

scripts/resolvers/preamble/generate-brain-sync-block.ts emits bash that
runs at every skill invocation:
- Detects ~/.gstack-brain-remote.txt on machines without local .git
  and surfaces a restore-available hint (does NOT auto-run restore).
- Runs gstack-brain-sync --once at skill start to drain any pending
  writes (and at skill end via prose instruction).
- Once-per-day auto-pull (cached via .brain-last-pull) for append-only
  JSONL files.
- Emits BRAIN_SYNC: status line every skill run.

Also emits prose for the host LLM to fire the one-time privacy
stop-gate (full / artifacts-only / off) when gbrain is detected and
gbrain_sync_mode_prompted is false. Wired into preamble.ts composition.

* test(gbrain-sync): 27-test consolidated suite

test/brain-sync.test.ts covers:
- Config: validation, defaults, GSTACK_HOME env isolation
- Enqueue: no-op gates, skip list, concurrent atomicity, JSON escape
- JSONL merge driver: 3-way + ts-sort + SHA-256 fallback
- Init + sync: canonical file creation, merge driver registration,
  push-reject + fetch+merge retry path
- Init refuses different remote (idempotency)
- Cross-machine restore round-trip (machine A write → machine B sees)
- Secret scan across all 6 regex families (AWS, GH, OpenAI, PEM, JWT,
  bearer-JSON). --skip-file unblock remediation
- Uninstall removes sync config, preserves user data
- --discover-new idempotence via mtime+size cursor

Behaviors verified via integration smokes during implementation. Known
follow-up: bun-test 5s default timeout needs 30s wrapper for
spawnSync-heavy tests.

* docs(gbrain-sync): user guide + error lookup + README section

docs/gbrain-sync.md: setup walkthrough, privacy modes, cross-machine
workflow, secret protection, two-machine conflict handling, uninstall,
troubleshooting reference.

docs/gbrain-sync-errors.md: problem/cause/fix index for every
user-visible error. Patterned on Rust's error docs + Stripe's API
error reference.

README.md: 'Cross-machine memory with GBrain sync' section near the
top (discovery moment), plus docs-table entry.

* chore: bump version and changelog (v1.7.0.0)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: regenerate SKILL.md files for gbrain-sync preamble block

Re-runs bun run gen:skill-docs after adding generateBrainSyncBlock
to scripts/resolvers/preamble.ts in a2aa8a07. CI check-freshness
caught the drift. All 36 SKILL.md files regenerated with the new
skill-start bash block + privacy-gate prose + skill-end sync
instructions baked in.

* fix(test): session-awareness reads AskUserQuestion Format from a Tier 2+ SKILL.md

The test was reading ROOT/SKILL.md (browse skill, Tier 1) which never
contained '## AskUserQuestion Format' — that section is only emitted
for Tier 2+ skills by scripts/resolvers/preamble.ts. As a result the
agent was prompted with an empty format guide and only emitted
'RECOMMENDATION' intermittently, making the test flaky.

Pre-existing on main (same ROOT/SKILL.md shape there) — surfaced now
because the agent run didn't hit the RECOMMENDATION/recommend/option a
fallback strings in this particular attempt.

Fix: read from office-hours/SKILL.md (Tier 3, always has the section)
with a fallback that scans for the first top-level skill dir whose
SKILL.md contains the header. Future template moves won't break this
test again.

* chore: bump to v1.9.0.0 for gbrain-sync landing

Changes just the VERSION + package.json + CHANGELOG header (1.7.0.0 → 1.9.0.0
and date 2026-04-22 → 2026-04-23). No code changes. User call: land gbrain-sync
as a bigger-signal release above main's 1.6.4.0, skipping 1.8.0.0.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 17:54:54 -07:00

236 lines
7.5 KiB
Bash
Executable File

#!/usr/bin/env bash
# gstack-brain-restore — bootstrap a new machine from an existing brain repo.
#
# Usage:
# gstack-brain-restore [<git-remote-url>]
#
# If no URL is given, reads from ~/.gstack-brain-remote.txt (written by
# gstack-brain-init on the original machine). Copy that file to the new
# machine before running this command.
#
# Safety gates (refuses with clear message):
# - ~/.gstack/.git already exists with a DIFFERENT remote
# - ~/.gstack/ contains non-allowlisted, non-gitignored user files
# that would be clobbered by restore
#
# What it does:
# 1. Clone the remote to a staging directory
# 2. Validate the repo is gstack-brain-shaped (.brain-allowlist, .gitattributes)
# 3. rsync-copy tracked files into ~/.gstack/ with skip-if-same-hash
# 4. Move staging's .git into ~/.gstack/.git
# 5. Register local git config merge drivers (they don't clone from remote)
# 6. Rehydrate consumers.json endpoints; prompt for tokens
#
# Env:
# GSTACK_HOME — override ~/.gstack
set -euo pipefail
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
REMOTE_FILE="$HOME/.gstack-brain-remote.txt"
REMOTE_URL="${1:-}"
if [ -z "$REMOTE_URL" ]; then
if [ -f "$REMOTE_FILE" ]; then
REMOTE_URL=$(head -1 "$REMOTE_FILE" | tr -d '[:space:]')
fi
fi
if [ -z "$REMOTE_URL" ]; then
cat >&2 <<EOF
gstack-brain-restore: no remote URL provided.
Provide one of:
gstack-brain-restore <git-url>
or put the URL in $REMOTE_FILE (copy from the original machine)
EOF
exit 1
fi
# ---- safety gates ----
if [ -d "$GSTACK_HOME/.git" ]; then
EXISTING_REMOTE=$(git -C "$GSTACK_HOME" remote get-url origin 2>/dev/null || echo "")
if [ -n "$EXISTING_REMOTE" ] && [ "$EXISTING_REMOTE" != "$REMOTE_URL" ]; then
cat >&2 <<EOF
gstack-brain-restore: ~/.gstack/.git already points at:
$EXISTING_REMOTE
You asked to restore from:
$REMOTE_URL
Refusing to overwrite. Run 'gstack-brain-uninstall' first or pass a matching URL.
EOF
exit 1
fi
fi
# ---- clone to staging ----
STAGING=$(mktemp -d "${TMPDIR:-/tmp}/gstack-brain-restore.XXXXXX")
trap 'rm -rf "$STAGING" 2>/dev/null' EXIT
echo "Cloning $REMOTE_URL to staging..."
if ! git clone --quiet "$REMOTE_URL" "$STAGING/repo" 2>/dev/null; then
echo "Clone failed. Check:" >&2
echo " - URL is correct: $REMOTE_URL" >&2
echo " - Auth: gh auth status (github) / glab auth status (gitlab)" >&2
exit 1
fi
# ---- validate shape ----
if [ ! -f "$STAGING/repo/.brain-allowlist" ] || [ ! -f "$STAGING/repo/.gitattributes" ]; then
cat >&2 <<EOF
gstack-brain-restore: $REMOTE_URL does not look like a gstack-brain repo.
Missing: .brain-allowlist and/or .gitattributes
This command only works on repos created by gstack-brain-init.
EOF
exit 1
fi
# ---- validate target ~/.gstack/ has no non-gitignored user files ----
mkdir -p "$GSTACK_HOME"
if [ ! -d "$GSTACK_HOME/.git" ]; then
# No existing git → check if we'd clobber anything allowlisted.
# Read the new allowlist globs and see if any existing files would collide.
CLOBBER_RISK=$(python3 - "$GSTACK_HOME" "$STAGING/repo/.brain-allowlist" <<'PYEOF'
import sys, os, fnmatch
home, allowlist_path = sys.argv[1:3]
try:
with open(allowlist_path) as f:
globs = [l.strip() for l in f if l.strip() and not l.lstrip().startswith('#')]
except FileNotFoundError:
globs = []
risks = []
for root, dirs, files in os.walk(home):
dirs[:] = [d for d in dirs if d != '.git']
for name in files:
full = os.path.join(root, name)
rel = os.path.relpath(full, home)
for g in globs:
if fnmatch.fnmatchcase(rel, g):
risks.append(rel)
break
for r in risks[:5]:
print(r)
if len(risks) > 5:
print(f"...and {len(risks) - 5} more")
sys.exit(0 if not risks else 2)
PYEOF
) || true
if [ -n "$CLOBBER_RISK" ]; then
cat >&2 <<EOF
gstack-brain-restore: ~/.gstack/ has existing allowlisted files that would
be clobbered by restore:
$CLOBBER_RISK
Back these up first, or run this command on a machine with an empty
~/.gstack/. If these files are from an earlier gstack session on THIS
machine, you probably want to run gstack-brain-init instead (to create a
new brain repo with this machine's state).
EOF
exit 1
fi
fi
# ---- copy tracked files in ----
echo "Copying tracked files into ~/.gstack/ ..."
# Use git-ls-tree to get exact tracked file list (avoids staged/untracked files).
cd "$STAGING/repo"
git ls-tree -r --name-only HEAD | while IFS= read -r rel_path; do
src="$STAGING/repo/$rel_path"
dst="$GSTACK_HOME/$rel_path"
mkdir -p "$(dirname "$dst")"
# Skip if identical (content hash). Otherwise copy.
if [ -f "$dst" ] && cmp -s "$src" "$dst"; then
continue
fi
cp "$src" "$dst"
done
# ---- move .git into place ----
if [ -d "$GSTACK_HOME/.git" ]; then
# Existing .git with matching remote — just fetch + fast-forward.
git -C "$GSTACK_HOME" fetch origin >/dev/null 2>&1 || true
else
mv "$STAGING/repo/.git" "$GSTACK_HOME/.git"
fi
# ---- register merge drivers (local git config; don't survive clones) ----
git -C "$GSTACK_HOME" config merge.jsonl-append.driver "$SCRIPT_DIR/gstack-jsonl-merge %O %A %B"
git -C "$GSTACK_HOME" config merge.jsonl-append.name "gstack JSONL append-only merger"
git -C "$GSTACK_HOME" config merge.union.driver "cat %A %B > %A.merged && mv %A.merged %A"
git -C "$GSTACK_HOME" config merge.union.name "union concat"
# ---- install pre-commit hook (same as init) ----
HOOK="$GSTACK_HOME/.git/hooks/pre-commit"
mkdir -p "$(dirname "$HOOK")"
cat > "$HOOK" <<'HOOK_EOF'
#!/usr/bin/env bash
set -uo pipefail
python3 -c "
import sys, re, subprocess
try:
out = subprocess.check_output(['git', 'diff', '--cached'], stderr=subprocess.DEVNULL).decode('utf-8', 'replace')
except Exception:
sys.exit(0)
patterns = [
('aws-access-key', re.compile(r'AKIA[0-9A-Z]{16}')),
('github-token', re.compile(r'\b(gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,})')),
('openai-key', re.compile(r'\bsk-[A-Za-z0-9_-]{20,}')),
('pem-block', re.compile(r'-----BEGIN [A-Z ]{3,}-----')),
('jwt', re.compile(r'\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b')),
('bearer-token-json',
re.compile(r'\"(authorization|api[_-]?key|apikey|token|secret|password)\"\s*:\s*\"[A-Za-z0-9_./+=-]{16,}\"',
re.IGNORECASE)),
]
for name, rx in patterns:
if rx.search(out):
sys.stderr.write(f'gstack-brain pre-commit: refusing commit — {name} detected.\n')
sys.exit(1)
sys.exit(0)
"
HOOK_EOF
chmod +x "$HOOK"
# ---- rehydrate consumers, prompt for tokens ----
if [ -f "$GSTACK_HOME/consumers.json" ]; then
echo ""
echo "Consumer registry restored. Tokens are machine-local and NOT synced."
echo "Run these for each consumer to re-enter tokens:"
python3 - "$GSTACK_HOME/consumers.json" <<'PYEOF'
import sys, json
try:
with open(sys.argv[1]) as f:
data = json.load(f)
except Exception:
sys.exit(0)
for c in data.get("consumers", []):
name = c.get("name", "")
token_ref = c.get("token_ref", f"{name}_token")
print(f" gstack-config set {token_ref} <your-token>")
PYEOF
fi
# ---- write remote helper file if missing ----
if [ ! -f "$REMOTE_FILE" ]; then
echo "$REMOTE_URL" > "$REMOTE_FILE"
chmod 600 "$REMOTE_FILE"
echo ""
echo "Wrote $REMOTE_FILE for future skill-run auto-detection."
fi
cat <<EOF
gstack-brain-restore complete.
Local: $GSTACK_HOME
Remote: $REMOTE_URL
Next skill run will ask about privacy mode (one-time question) and then
sync automatically at skill boundaries.
Status anytime: gstack-brain-sync --status
EOF