mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-02 03:35:09 +02:00
feat(v1.9.0.0): gbrain-sync — cross-machine gstack memory (#1151)
* feat(gbrain-sync): queue primitives + writer shims
Adds bin/gstack-brain-enqueue (atomic append to sync queue) and
bin/gstack-jsonl-merge (git merge driver, ts-sort with SHA-256 fallback).
Wires one backgrounded enqueue call into learnings-log, timeline-log,
review-log, and developer-profile --migrate. question-log and
question-preferences stay local per Codex v2 decision.
gstack-config gains gbrain_sync_mode (off/artifacts-only/full) and
gbrain_sync_mode_prompted keys, plus GSTACK_HOME env alignment so
tests don't leak into real ~/.gstack/config.yaml.
* feat(gbrain-sync): --once drain + secret scan + push
bin/gstack-brain-sync is the core sync binary. Subcommands: --once
(drain queue, allowlist-filter, privacy-class-filter, secret-scan
staged diff, commit with template, push with fetch+merge retry),
--status, --skip-file <path>, --drop-queue --yes, --discover-new
(cursor-based detection of artifact writes that skip the shim).
Secret regex families: AWS keys, GitHub tokens (ghp_/gho_/ghu_/ghs_/
ghr_/github_pat_), OpenAI sk-, PEM blocks, JWTs, bearer-token-in-JSON.
On hit: unstage, preserve queue, print remediation hint (--skip-file
or edit), exit clean. No daemon — invoked by preamble at skill
boundaries.
* feat(gbrain-sync): init, restore, uninstall, consumer registry
bin/gstack-brain-init: idempotent first-run. git init ~/.gstack/,
.gitignore=*, canonical .brain-allowlist + .brain-privacy-map.json,
pre-commit secret-scan hook (defense-in-depth), merge driver registration
via git config, gh repo create --private OR arbitrary --remote <url>,
initial push, ~/.gstack-brain-remote.txt for new-machine discovery,
GBrain consumer registration via HTTP POST.
bin/gstack-brain-restore: safe new-machine bootstrap. Refuses clobber
of existing allowlisted files, clones to staging, rsync-copies tracked
files, re-registers merge drivers (required — not cloned from remote),
rehydrates consumers.json, prompts for per-consumer tokens.
bin/gstack-brain-uninstall: clean off-ramp. Removes .git + .brain-*
files + consumers.json + config keys. Preserves user data (learnings,
plans, retros, profile). Optional --delete-remote for GitHub repos.
bin/gstack-brain-consumer + bin/gstack-brain-reader (symlink alias):
registry management. Internal 'consumer' term; user-facing 'reader'
per DX review decision.
* feat(gbrain-sync): preamble block — privacy gate + boundary sync
scripts/resolvers/preamble/generate-brain-sync-block.ts emits bash that
runs at every skill invocation:
- Detects ~/.gstack-brain-remote.txt on machines without local .git
and surfaces a restore-available hint (does NOT auto-run restore).
- Runs gstack-brain-sync --once at skill start to drain any pending
writes (and at skill end via prose instruction).
- Once-per-day auto-pull (cached via .brain-last-pull) for append-only
JSONL files.
- Emits BRAIN_SYNC: status line every skill run.
Also emits prose for the host LLM to fire the one-time privacy
stop-gate (full / artifacts-only / off) when gbrain is detected and
gbrain_sync_mode_prompted is false. Wired into preamble.ts composition.
* test(gbrain-sync): 27-test consolidated suite
test/brain-sync.test.ts covers:
- Config: validation, defaults, GSTACK_HOME env isolation
- Enqueue: no-op gates, skip list, concurrent atomicity, JSON escape
- JSONL merge driver: 3-way + ts-sort + SHA-256 fallback
- Init + sync: canonical file creation, merge driver registration,
push-reject + fetch+merge retry path
- Init refuses different remote (idempotency)
- Cross-machine restore round-trip (machine A write → machine B sees)
- Secret scan across all 6 regex families (AWS, GH, OpenAI, PEM, JWT,
bearer-JSON). --skip-file unblock remediation
- Uninstall removes sync config, preserves user data
- --discover-new idempotence via mtime+size cursor
Behaviors verified via integration smokes during implementation. Known
follow-up: bun-test 5s default timeout needs 30s wrapper for
spawnSync-heavy tests.
* docs(gbrain-sync): user guide + error lookup + README section
docs/gbrain-sync.md: setup walkthrough, privacy modes, cross-machine
workflow, secret protection, two-machine conflict handling, uninstall,
troubleshooting reference.
docs/gbrain-sync-errors.md: problem/cause/fix index for every
user-visible error. Patterned on Rust's error docs + Stripe's API
error reference.
README.md: 'Cross-machine memory with GBrain sync' section near the
top (discovery moment), plus docs-table entry.
* chore: bump version and changelog (v1.7.0.0)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore: regenerate SKILL.md files for gbrain-sync preamble block
Re-runs bun run gen:skill-docs after adding generateBrainSyncBlock
to scripts/resolvers/preamble.ts in a2aa8a07. CI check-freshness
caught the drift. All 36 SKILL.md files regenerated with the new
skill-start bash block + privacy-gate prose + skill-end sync
instructions baked in.
* fix(test): session-awareness reads AskUserQuestion Format from a Tier 2+ SKILL.md
The test was reading ROOT/SKILL.md (browse skill, Tier 1) which never
contained '## AskUserQuestion Format' — that section is only emitted
for Tier 2+ skills by scripts/resolvers/preamble.ts. As a result the
agent was prompted with an empty format guide and only emitted
'RECOMMENDATION' intermittently, making the test flaky.
Pre-existing on main (same ROOT/SKILL.md shape there) — surfaced now
because the agent run didn't hit the RECOMMENDATION/recommend/option a
fallback strings in this particular attempt.
Fix: read from office-hours/SKILL.md (Tier 3, always has the section)
with a fallback that scans for the first top-level skill dir whose
SKILL.md contains the header. Future template moves won't break this
test again.
* chore: bump to v1.9.0.0 for gbrain-sync landing
Changes just the VERSION + package.json + CHANGELOG header (1.7.0.0 → 1.9.0.0
and date 2026-04-22 → 2026-04-23). No code changes. User call: land gbrain-sync
as a bigger-signal release above main's 1.6.4.0, skipping 1.8.0.0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Executable
+196
@@ -0,0 +1,196 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-brain-consumer — manage the consumer (reader) registry.
|
||||
#
|
||||
# Consumer = a reader that ingests the gstack-brain git repo as a source of
|
||||
# session memory. v1 primary consumer is GBrain; later versions can register
|
||||
# Codex, OpenClaw, or third-party readers.
|
||||
#
|
||||
# NOTE ON NAMING: internally this helper uses "consumer" (correct data-model
|
||||
# term). User-facing copy and the alias `gstack-brain-reader` use "reader"
|
||||
# (matches user mental model: "what's reading my brain?").
|
||||
#
|
||||
# Usage:
|
||||
# gstack-brain-consumer add <name> --ingest-url <url> --token <token>
|
||||
# gstack-brain-consumer list
|
||||
# gstack-brain-consumer remove <name>
|
||||
# gstack-brain-consumer test <name>
|
||||
#
|
||||
# Env:
|
||||
# GSTACK_HOME — override ~/.gstack
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
CONSUMERS_FILE="$GSTACK_HOME/consumers.json"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
|
||||
|
||||
ensure_file() {
|
||||
mkdir -p "$GSTACK_HOME"
|
||||
if [ ! -f "$CONSUMERS_FILE" ]; then
|
||||
echo '{"consumers": []}' > "$CONSUMERS_FILE"
|
||||
fi
|
||||
}
|
||||
|
||||
get_remote_url() {
|
||||
git -C "$GSTACK_HOME" remote get-url origin 2>/dev/null || echo ""
|
||||
}
|
||||
|
||||
sub_add() {
|
||||
local name="" url="" token=""
|
||||
local positional=""
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--ingest-url) url="$2"; shift 2 ;;
|
||||
--token) token="$2"; shift 2 ;;
|
||||
--) shift; break ;;
|
||||
-*) echo "Unknown flag: $1" >&2; exit 1 ;;
|
||||
*) positional="$1"; shift ;;
|
||||
esac
|
||||
done
|
||||
name="$positional"
|
||||
if [ -z "$name" ] || [ -z "$url" ]; then
|
||||
echo "Usage: gstack-brain-consumer add <name> --ingest-url <url> [--token <token>]" >&2
|
||||
exit 1
|
||||
fi
|
||||
ensure_file
|
||||
# Upsert in consumers.json, store token in gstack-config under `<name>_token`.
|
||||
python3 - "$CONSUMERS_FILE" "$name" "$url" <<'PYEOF'
|
||||
import sys, json
|
||||
path, name, url = sys.argv[1:4]
|
||||
try:
|
||||
with open(path) as f:
|
||||
data = json.load(f)
|
||||
except Exception:
|
||||
data = {"consumers": []}
|
||||
entry = {"name": name, "ingest_url": url, "status": "unknown", "token_ref": f"{name}_token"}
|
||||
cs = data.setdefault("consumers", [])
|
||||
for i, c in enumerate(cs):
|
||||
if c.get("name") == name:
|
||||
cs[i] = entry
|
||||
break
|
||||
else:
|
||||
cs.append(entry)
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
f.write("\n")
|
||||
print(f"registered consumer: {name}")
|
||||
PYEOF
|
||||
if [ -n "$token" ]; then
|
||||
"$CONFIG_BIN" set "${name}_token" "$token"
|
||||
echo "token stored: gstack-config get ${name}_token to retrieve"
|
||||
fi
|
||||
# Attempt registration with remote (HTTP POST).
|
||||
sub_test "$name"
|
||||
}
|
||||
|
||||
sub_list() {
|
||||
if [ ! -f "$CONSUMERS_FILE" ]; then
|
||||
echo '{"consumers": []}'
|
||||
return 0
|
||||
fi
|
||||
cat "$CONSUMERS_FILE"
|
||||
}
|
||||
|
||||
sub_remove() {
|
||||
local name="${1:-}"
|
||||
if [ -z "$name" ]; then
|
||||
echo "Usage: gstack-brain-consumer remove <name>" >&2
|
||||
exit 1
|
||||
fi
|
||||
ensure_file
|
||||
python3 - "$CONSUMERS_FILE" "$name" <<'PYEOF'
|
||||
import sys, json
|
||||
path, name = sys.argv[1:3]
|
||||
try:
|
||||
with open(path) as f:
|
||||
data = json.load(f)
|
||||
except Exception:
|
||||
data = {"consumers": []}
|
||||
before = len(data.get("consumers", []))
|
||||
data["consumers"] = [c for c in data.get("consumers", []) if c.get("name") != name]
|
||||
after = len(data["consumers"])
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
f.write("\n")
|
||||
print(f"removed: {before - after} entry(ies)")
|
||||
PYEOF
|
||||
}
|
||||
|
||||
sub_test() {
|
||||
local name="${1:-}"
|
||||
if [ -z "$name" ]; then
|
||||
echo "Usage: gstack-brain-consumer test <name>" >&2
|
||||
exit 1
|
||||
fi
|
||||
ensure_file
|
||||
# Look up the consumer by name.
|
||||
local info
|
||||
info=$(python3 - "$CONSUMERS_FILE" "$name" <<'PYEOF'
|
||||
import sys, json
|
||||
path, name = sys.argv[1:3]
|
||||
try:
|
||||
with open(path) as f:
|
||||
data = json.load(f)
|
||||
except Exception:
|
||||
data = {"consumers": []}
|
||||
for c in data.get("consumers", []):
|
||||
if c.get("name") == name:
|
||||
print(c.get("ingest_url", ""))
|
||||
sys.exit(0)
|
||||
sys.exit(1)
|
||||
PYEOF
|
||||
) || { echo "No such consumer: $name" >&2; exit 1; }
|
||||
|
||||
local url="$info"
|
||||
local token
|
||||
token=$("$CONFIG_BIN" get "${name}_token" 2>/dev/null || echo "")
|
||||
if [ -z "$url" ] || [ -z "$token" ]; then
|
||||
echo "consumer '$name': url or token missing; cannot test"
|
||||
return 0
|
||||
fi
|
||||
local repo_url
|
||||
repo_url=$(get_remote_url)
|
||||
echo "Testing $name at ${url%/}/ingest-repo ..."
|
||||
local resp
|
||||
resp=$(curl -sS -X POST "${url%/}/ingest-repo" \
|
||||
-H "Authorization: Bearer $token" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data "{\"repo_url\":\"$repo_url\"}" \
|
||||
-w "\n%{http_code}" 2>&1 || echo -e "\ncurl-error")
|
||||
local code
|
||||
code=$(echo "$resp" | tail -1)
|
||||
if [ "$code" = "200" ] || [ "$code" = "201" ] || [ "$code" = "204" ]; then
|
||||
echo "ok (HTTP $code)"
|
||||
# Update status in consumers.json.
|
||||
python3 - "$CONSUMERS_FILE" "$name" "ok" <<'PYEOF'
|
||||
import sys, json
|
||||
path, name, status = sys.argv[1:4]
|
||||
with open(path) as f: data = json.load(f)
|
||||
for c in data.get("consumers", []):
|
||||
if c.get("name") == name:
|
||||
c["status"] = status
|
||||
with open(path, "w") as f: json.dump(data, f, indent=2); f.write("\n")
|
||||
PYEOF
|
||||
else
|
||||
echo "failed (HTTP $code)"
|
||||
python3 - "$CONSUMERS_FILE" "$name" "error" <<'PYEOF'
|
||||
import sys, json
|
||||
path, name, status = sys.argv[1:4]
|
||||
with open(path) as f: data = json.load(f)
|
||||
for c in data.get("consumers", []):
|
||||
if c.get("name") == name:
|
||||
c["status"] = status
|
||||
with open(path, "w") as f: json.dump(data, f, indent=2); f.write("\n")
|
||||
PYEOF
|
||||
fi
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
add) shift; sub_add "$@" ;;
|
||||
list) sub_list ;;
|
||||
remove) shift; sub_remove "$@" ;;
|
||||
test) shift; sub_test "$@" ;;
|
||||
--help|-h|"") sed -n '2,20p' "$0" | sed 's/^# \{0,1\}//' ;;
|
||||
*) echo "Unknown subcommand: $1" >&2; exit 1 ;;
|
||||
esac
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-brain-enqueue — atomically append a path to the GBrain sync queue.
|
||||
#
|
||||
# Usage:
|
||||
# gstack-brain-enqueue <file-path>
|
||||
#
|
||||
# Called by writer scripts (gstack-learnings-log, gstack-timeline-log, etc.)
|
||||
# after their local write. Fire-and-forget; failures are silent (never blocks
|
||||
# the writer). Queue is drained by `gstack-brain-sync --once` invoked from the
|
||||
# preamble at skill START and END boundaries.
|
||||
#
|
||||
# No-op when:
|
||||
# - gbrain_sync_mode is off (the default)
|
||||
# - ~/.gstack/.git doesn't exist (feature not initialized)
|
||||
# - <file-path> matches a line in ~/.gstack/.brain-skip.txt
|
||||
#
|
||||
# Env:
|
||||
# GSTACK_HOME — override ~/.gstack state directory (aligns with writers).
|
||||
# Tests use GSTACK_HOME=/tmp/test-$$ for isolation.
|
||||
#
|
||||
# Concurrency: POSIX append is atomic up to PIPE_BUF (~4KB Linux, 512 BSD).
|
||||
# Queue lines are ~200 bytes, safe under concurrent callers.
|
||||
|
||||
# No `-e` — writer shims rely on this never failing loudly.
|
||||
set -uo pipefail
|
||||
|
||||
FILE="${1:-}"
|
||||
[ -z "$FILE" ] && exit 0
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
QUEUE="$GSTACK_HOME/.brain-queue.jsonl"
|
||||
SKIP_FILE="$GSTACK_HOME/.brain-skip.txt"
|
||||
|
||||
# Fast exits: no git repo, no sync.
|
||||
[ ! -d "$GSTACK_HOME/.git" ] && exit 0
|
||||
|
||||
# Check sync mode. off → silent no-op.
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" 2>/dev/null && pwd)"
|
||||
MODE=$("$SCRIPT_DIR/gstack-config" get gbrain_sync_mode 2>/dev/null || echo off)
|
||||
[ "$MODE" = "off" ] && exit 0
|
||||
|
||||
# User-maintained skip list (for secret-scan false positives).
|
||||
if [ -f "$SKIP_FILE" ]; then
|
||||
if grep -Fxq "$FILE" "$SKIP_FILE" 2>/dev/null; then
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# JSON-escape the file path (backslash + quotes only; paths shouldn't have other specials).
|
||||
ESC_FILE=$(printf '%s' "$FILE" | sed 's/\\/\\\\/g; s/"/\\"/g')
|
||||
TS=$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || echo "")
|
||||
|
||||
printf '{"file":"%s","ts":"%s"}\n' "$ESC_FILE" "$TS" >> "$QUEUE" 2>/dev/null
|
||||
|
||||
exit 0
|
||||
Executable
+360
@@ -0,0 +1,360 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-brain-init — set up ~/.gstack/ as a git repo that syncs to GBrain.
|
||||
#
|
||||
# Usage:
|
||||
# gstack-brain-init [--remote <url>]
|
||||
#
|
||||
# Interactive by default. Pass --remote to skip the remote prompt.
|
||||
#
|
||||
# Idempotent: safe to re-run. If ~/.gstack/.git already exists AND points at
|
||||
# the same remote, reconfigures drivers/hooks/attributes without clobbering
|
||||
# history. If it points at a DIFFERENT remote, refuses and suggests
|
||||
# `gstack-brain-uninstall` first.
|
||||
#
|
||||
# What it does:
|
||||
# 1. git init ~/.gstack/ (or verify existing repo points at the right remote)
|
||||
# 2. Write .gitignore = "*" (ignore everything; allowlist is explicit)
|
||||
# 3. Write .brain-allowlist (canonical paths to sync)
|
||||
# 4. Write .brain-privacy-map.json (paths → privacy class)
|
||||
# 5. Write .gitattributes (register JSONL + union merge drivers)
|
||||
# 6. git config merge.jsonl-append.driver + merge.union.driver
|
||||
# 7. Install .git/hooks/pre-commit (defense-in-depth secret scan)
|
||||
# 8. Prompt for remote (default: gh repo create --private gstack-brain-$USER)
|
||||
# 9. Initial commit + push
|
||||
# 10. Write ~/.gstack-brain-remote.txt (URL-only, safe to share)
|
||||
# 11. Register GBrain consumer (HTTP POST if GBRAIN_URL set; else defer)
|
||||
#
|
||||
# Env:
|
||||
# GSTACK_HOME — override ~/.gstack
|
||||
# GBRAIN_URL — GBrain ingest endpoint base URL (for consumer registration)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
|
||||
REMOTE_FILE="$HOME/.gstack-brain-remote.txt"
|
||||
CONSUMERS_FILE="$GSTACK_HOME/consumers.json"
|
||||
|
||||
REMOTE_URL=""
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--remote) REMOTE_URL="$2"; shift 2 ;;
|
||||
--help|-h) sed -n '2,32p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) echo "Unknown flag: $1" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# ---- preconditions ----
|
||||
mkdir -p "$GSTACK_HOME"
|
||||
|
||||
EXISTING_REMOTE=""
|
||||
if [ -d "$GSTACK_HOME/.git" ]; then
|
||||
EXISTING_REMOTE=$(git -C "$GSTACK_HOME" remote get-url origin 2>/dev/null || echo "")
|
||||
if [ -n "$EXISTING_REMOTE" ] && [ -n "$REMOTE_URL" ] && [ "$EXISTING_REMOTE" != "$REMOTE_URL" ]; then
|
||||
cat >&2 <<EOF
|
||||
gstack-brain-init: ~/.gstack/ is already a git repo pointing at:
|
||||
$EXISTING_REMOTE
|
||||
|
||||
You asked to init with:
|
||||
$REMOTE_URL
|
||||
|
||||
Refusing to overwrite. To switch remotes, first run:
|
||||
gstack-brain-uninstall
|
||||
|
||||
(or edit the remote manually with: git -C ~/.gstack remote set-url origin <url>)
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- choose the remote ----
|
||||
if [ -z "$REMOTE_URL" ] && [ -n "$EXISTING_REMOTE" ]; then
|
||||
REMOTE_URL="$EXISTING_REMOTE"
|
||||
echo "Using existing remote: $REMOTE_URL"
|
||||
fi
|
||||
|
||||
if [ -z "$REMOTE_URL" ]; then
|
||||
# Interactive prompt. Default: gh repo create (if available).
|
||||
echo "gstack-brain-init will create a private git repo that holds your"
|
||||
echo "gstack session memory across machines and lets GBrain index it."
|
||||
echo
|
||||
if command -v gh >/dev/null 2>&1 && gh auth status >/dev/null 2>&1; then
|
||||
DEFAULT_NAME="gstack-brain-${USER:-$(whoami)}"
|
||||
echo "Default: gh will create a private repo named '$DEFAULT_NAME' under your account."
|
||||
printf "Press Enter to accept, or paste a custom git URL: "
|
||||
read -r REPLY || REPLY=""
|
||||
if [ -z "$REPLY" ]; then
|
||||
echo "Creating GitHub repo: $DEFAULT_NAME ..."
|
||||
if ! gh repo create "$DEFAULT_NAME" --private --description "gstack session memory" --source "$GSTACK_HOME" 2>/dev/null; then
|
||||
# Maybe the repo already exists; try to fetch its URL.
|
||||
REMOTE_URL=$(gh repo view "$DEFAULT_NAME" --json sshUrl -q .sshUrl 2>/dev/null || echo "")
|
||||
if [ -z "$REMOTE_URL" ]; then
|
||||
echo "Failed to create or find '$DEFAULT_NAME'. Try --remote <url>." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "Repo already exists; using $REMOTE_URL"
|
||||
else
|
||||
REMOTE_URL=$(gh repo view "$DEFAULT_NAME" --json sshUrl -q .sshUrl 2>/dev/null || echo "")
|
||||
fi
|
||||
else
|
||||
REMOTE_URL="$REPLY"
|
||||
fi
|
||||
else
|
||||
echo "(gh CLI not found or not authenticated; provide a git URL directly)"
|
||||
printf "Paste a private git URL (e.g. git@github.com:you/gstack-brain.git): "
|
||||
read -r REMOTE_URL || REMOTE_URL=""
|
||||
if [ -z "$REMOTE_URL" ]; then
|
||||
echo "No URL provided. Aborting." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- verify remote reachable ----
|
||||
echo "Verifying remote connectivity: $REMOTE_URL"
|
||||
if ! git ls-remote "$REMOTE_URL" >/dev/null 2>&1; then
|
||||
cat >&2 <<EOF
|
||||
Remote not reachable: $REMOTE_URL
|
||||
This could mean:
|
||||
- Wrong URL
|
||||
- Not authenticated (GitHub: gh auth status; GitLab: glab auth status)
|
||||
- Network issue
|
||||
Fix and re-run gstack-brain-init.
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ---- git init ----
|
||||
if [ ! -d "$GSTACK_HOME/.git" ]; then
|
||||
git -C "$GSTACK_HOME" init -q -b main 2>/dev/null || git -C "$GSTACK_HOME" init -q
|
||||
# If -b main wasn't supported, rename.
|
||||
git -C "$GSTACK_HOME" branch -M main 2>/dev/null || true
|
||||
fi
|
||||
|
||||
if [ -z "$(git -C "$GSTACK_HOME" remote 2>/dev/null)" ]; then
|
||||
git -C "$GSTACK_HOME" remote add origin "$REMOTE_URL"
|
||||
else
|
||||
git -C "$GSTACK_HOME" remote set-url origin "$REMOTE_URL"
|
||||
fi
|
||||
|
||||
# ---- write canonical files (idempotent) ----
|
||||
cat > "$GSTACK_HOME/.gitignore" <<'EOF'
|
||||
# gstack-brain sync: ignore-everything base. Paths are included explicitly via
|
||||
# .brain-allowlist and `git add -f` from gstack-brain-sync. Do not edit.
|
||||
*
|
||||
EOF
|
||||
|
||||
cat > "$GSTACK_HOME/.brain-allowlist" <<'EOF'
|
||||
# Canonical allowlist of paths that gstack-brain-sync will publish.
|
||||
# One glob per line. Anything not matching stays local.
|
||||
# Do not edit directly; managed by gstack-brain-init. User additions go below
|
||||
# the marker and survive re-init.
|
||||
projects/*/learnings.jsonl
|
||||
projects/*/*-reviews.jsonl
|
||||
projects/*/ceo-plans/*.md
|
||||
projects/*/ceo-plans/*/*.md
|
||||
projects/*/designs/*.md
|
||||
projects/*/designs/*/*.md
|
||||
projects/*/timeline.jsonl
|
||||
retros/*.md
|
||||
developer-profile.json
|
||||
builder-journey.md
|
||||
builder-profile.jsonl
|
||||
# NOT synced (per Codex v2 review — machine-local UX state):
|
||||
# projects/*/question-preferences.json (per-machine UX preferences)
|
||||
# projects/*/question-log.jsonl (audit/derivation log stays with preferences)
|
||||
# projects/*/question-events.jsonl (same)
|
||||
# ---- USER ADDITIONS BELOW ---- (survives re-init; above is managed)
|
||||
EOF
|
||||
|
||||
cat > "$GSTACK_HOME/.brain-privacy-map.json" <<'EOF'
|
||||
[
|
||||
{"pattern": "projects/*/learnings.jsonl", "class": "artifact"},
|
||||
{"pattern": "projects/*/*-reviews.jsonl", "class": "artifact"},
|
||||
{"pattern": "projects/*/ceo-plans/*.md", "class": "artifact"},
|
||||
{"pattern": "projects/*/ceo-plans/*/*.md", "class": "artifact"},
|
||||
{"pattern": "projects/*/designs/*.md", "class": "artifact"},
|
||||
{"pattern": "projects/*/designs/*/*.md", "class": "artifact"},
|
||||
{"pattern": "retros/*.md", "class": "artifact"},
|
||||
{"pattern": "builder-journey.md", "class": "artifact"},
|
||||
{"pattern": "projects/*/timeline.jsonl", "class": "behavioral"},
|
||||
{"pattern": "developer-profile.json", "class": "behavioral"},
|
||||
{"pattern": "builder-profile.jsonl", "class": "behavioral"}
|
||||
]
|
||||
EOF
|
||||
|
||||
cat > "$GSTACK_HOME/.gitattributes" <<'EOF'
|
||||
# gstack-brain: merge drivers for cross-machine sync conflicts.
|
||||
# Matching driver must be registered in local git config; gstack-brain-init
|
||||
# and gstack-brain-restore run `git config merge.<name>.driver ...` after init.
|
||||
*.jsonl merge=jsonl-append
|
||||
retros/*.md merge=union
|
||||
projects/*/designs/**/*.md merge=union
|
||||
projects/*/ceo-plans/**/*.md merge=union
|
||||
EOF
|
||||
|
||||
# ---- register merge drivers in local git config ----
|
||||
git -C "$GSTACK_HOME" config merge.jsonl-append.driver "$SCRIPT_DIR/gstack-jsonl-merge %O %A %B"
|
||||
git -C "$GSTACK_HOME" config merge.jsonl-append.name "gstack JSONL append-only merger"
|
||||
git -C "$GSTACK_HOME" config merge.union.driver "cat %A %B > %A.merged && mv %A.merged %A"
|
||||
git -C "$GSTACK_HOME" config merge.union.name "union concat"
|
||||
|
||||
# ---- install pre-commit hook (defense-in-depth) ----
|
||||
HOOK="$GSTACK_HOME/.git/hooks/pre-commit"
|
||||
mkdir -p "$(dirname "$HOOK")"
|
||||
cat > "$HOOK" <<'HOOK_EOF'
|
||||
#!/usr/bin/env bash
|
||||
# gstack-brain pre-commit hook — secret-scan defense-in-depth.
|
||||
# The primary scanner runs inside gstack-brain-sync BEFORE staging. This hook
|
||||
# catches any manual `git commit` a user might accidentally run against the
|
||||
# brain repo.
|
||||
set -uo pipefail
|
||||
|
||||
python3 -c "
|
||||
import sys, re, subprocess
|
||||
try:
|
||||
out = subprocess.check_output(['git', 'diff', '--cached'], stderr=subprocess.DEVNULL).decode('utf-8', 'replace')
|
||||
except Exception:
|
||||
sys.exit(0)
|
||||
|
||||
patterns = [
|
||||
('aws-access-key', re.compile(r'AKIA[0-9A-Z]{16}')),
|
||||
('github-token', re.compile(r'\b(gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,})')),
|
||||
('openai-key', re.compile(r'\bsk-[A-Za-z0-9_-]{20,}')),
|
||||
('pem-block', re.compile(r'-----BEGIN [A-Z ]{3,}-----')),
|
||||
('jwt', re.compile(r'\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b')),
|
||||
('bearer-token-json',
|
||||
re.compile(r'\"(authorization|api[_-]?key|apikey|token|secret|password)\"\s*:\s*\"[A-Za-z0-9_./+=-]{16,}\"',
|
||||
re.IGNORECASE)),
|
||||
]
|
||||
for name, rx in patterns:
|
||||
if rx.search(out):
|
||||
sys.stderr.write(f'gstack-brain pre-commit: refusing commit — {name} detected in staged diff.\n')
|
||||
sys.stderr.write('Either edit the offending file, or if intentional, run:\n')
|
||||
sys.stderr.write(' gstack-brain-sync --skip-file <path> (to permanently exclude)\n')
|
||||
sys.exit(1)
|
||||
sys.exit(0)
|
||||
"
|
||||
HOOK_EOF
|
||||
chmod +x "$HOOK"
|
||||
|
||||
# ---- initial commit (idempotent; skips if already committed) ----
|
||||
cd "$GSTACK_HOME"
|
||||
git add -f .gitignore .brain-allowlist .brain-privacy-map.json .gitattributes
|
||||
# Only commit if the index has changes from HEAD (if there is a HEAD).
|
||||
if git rev-parse HEAD >/dev/null 2>&1; then
|
||||
if ! git diff --cached --quiet 2>/dev/null; then
|
||||
git -c user.email="gstack@localhost" -c user.name="gstack-brain-init" \
|
||||
commit -q -m "chore: gstack-brain-init (refresh sync config)"
|
||||
fi
|
||||
else
|
||||
# First commit ever.
|
||||
git -c user.email="gstack@localhost" -c user.name="gstack-brain-init" \
|
||||
commit -q -m "chore: gstack-brain-init"
|
||||
fi
|
||||
|
||||
# ---- initial push ----
|
||||
if ! git push -q -u origin main 2>/dev/null; then
|
||||
# Maybe the default branch is master, or the remote has existing content.
|
||||
# Try to resolve: fetch + fast-forward merge + push.
|
||||
CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
|
||||
if git fetch origin 2>/dev/null && git pull --ff-only origin "$CURRENT_BRANCH" 2>/dev/null; then
|
||||
git push -q -u origin "$CURRENT_BRANCH" || {
|
||||
echo "Push to $REMOTE_URL failed. The remote may have divergent content." >&2
|
||||
echo "Try: cd ~/.gstack && git pull --rebase origin $CURRENT_BRANCH && git push origin $CURRENT_BRANCH" >&2
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
# Couldn't fetch/merge; print what to do.
|
||||
echo "Push to $REMOTE_URL failed and fetch/merge didn't help." >&2
|
||||
echo "Manual recovery: cd ~/.gstack && git status, then push once conflicts are resolved." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- write the remote-url helper file (outside ~/.gstack/, survives restore) ----
|
||||
echo "$REMOTE_URL" > "$REMOTE_FILE"
|
||||
chmod 600 "$REMOTE_FILE"
|
||||
|
||||
# ---- register GBrain consumer ----
|
||||
mkdir -p "$GSTACK_HOME"
|
||||
CONSUMER_STATUS="pending"
|
||||
GBRAIN_URL_VAL="${GBRAIN_URL:-$("$CONFIG_BIN" get gbrain_url 2>/dev/null || echo "")}"
|
||||
GBRAIN_TOKEN_VAL="${GBRAIN_TOKEN:-$("$CONFIG_BIN" get gbrain_token 2>/dev/null || echo "")}"
|
||||
|
||||
if [ -n "$GBRAIN_URL_VAL" ] && [ -n "$GBRAIN_TOKEN_VAL" ]; then
|
||||
# Try the HTTP handoff.
|
||||
HTTP_RESP=$(curl -sS -X POST "${GBRAIN_URL_VAL%/}/ingest-repo" \
|
||||
-H "Authorization: Bearer $GBRAIN_TOKEN_VAL" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data "{\"repo_url\":\"$REMOTE_URL\"}" \
|
||||
-w "\n%{http_code}" 2>&1 || echo -e "\ncurl-error")
|
||||
HTTP_CODE=$(echo "$HTTP_RESP" | tail -1)
|
||||
if [ "$HTTP_CODE" = "200" ] || [ "$HTTP_CODE" = "201" ] || [ "$HTTP_CODE" = "204" ]; then
|
||||
CONSUMER_STATUS="ok"
|
||||
echo "GBrain consumer registered: $GBRAIN_URL_VAL"
|
||||
else
|
||||
echo "GBrain ingest endpoint returned HTTP $HTTP_CODE; will retry on next skill run."
|
||||
fi
|
||||
elif [ -z "$GBRAIN_URL_VAL" ]; then
|
||||
echo "(GBRAIN_URL not configured; skipping consumer registration. Set it with:"
|
||||
echo " gstack-config set gbrain_url <url>"
|
||||
echo " gstack-config set gbrain_token <token>"
|
||||
echo " then run: gstack-brain-consumer add gbrain --ingest-url <url> --token <token>)"
|
||||
fi
|
||||
|
||||
# Write consumers.json — the canonical registry. Tokens are NOT stored here;
|
||||
# they stay in gstack-config (machine-local). This file IS synced so a new
|
||||
# machine knows which consumers exist and can prompt for tokens.
|
||||
python3 - "$CONSUMERS_FILE" "$GBRAIN_URL_VAL" "$CONSUMER_STATUS" <<'PYEOF'
|
||||
import sys, json, os
|
||||
path, url, status = sys.argv[1:4]
|
||||
try:
|
||||
with open(path) as f:
|
||||
data = json.load(f)
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
data = {"consumers": []}
|
||||
# Upsert GBrain entry.
|
||||
entry = {"name": "gbrain", "ingest_url": url, "status": status, "token_ref": "gbrain_token"}
|
||||
updated = False
|
||||
for i, c in enumerate(data.get("consumers", [])):
|
||||
if c.get("name") == "gbrain":
|
||||
data["consumers"][i] = entry
|
||||
updated = True
|
||||
break
|
||||
if not updated:
|
||||
data.setdefault("consumers", []).append(entry)
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
f.write("\n")
|
||||
PYEOF
|
||||
|
||||
# Stage and commit consumers.json in the same session.
|
||||
cd "$GSTACK_HOME"
|
||||
git add -f consumers.json 2>/dev/null || true
|
||||
if ! git diff --cached --quiet 2>/dev/null; then
|
||||
git -c user.email="gstack@localhost" -c user.name="gstack-brain-init" \
|
||||
commit -q -m "chore: register GBrain consumer"
|
||||
git push -q origin HEAD 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# ---- done ----
|
||||
cat <<EOF
|
||||
|
||||
gstack-brain-init complete.
|
||||
Repo: $GSTACK_HOME (git)
|
||||
Remote: $REMOTE_URL
|
||||
Remote URL also saved at: $REMOTE_FILE
|
||||
|
||||
Sync happens automatically at the start and end of each skill (no daemon).
|
||||
Check status anytime with:
|
||||
gstack-brain-sync --status
|
||||
|
||||
To activate sync, the next skill you run will ask you one question about
|
||||
privacy mode (sync everything / artifacts only / off).
|
||||
|
||||
New machine? On the other laptop, put a copy of:
|
||||
$REMOTE_FILE
|
||||
in that machine's home directory, then run: gstack-brain-restore
|
||||
EOF
|
||||
Symlink
+1
@@ -0,0 +1 @@
|
||||
gstack-brain-consumer
|
||||
Executable
+235
@@ -0,0 +1,235 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-brain-restore — bootstrap a new machine from an existing brain repo.
|
||||
#
|
||||
# Usage:
|
||||
# gstack-brain-restore [<git-remote-url>]
|
||||
#
|
||||
# If no URL is given, reads from ~/.gstack-brain-remote.txt (written by
|
||||
# gstack-brain-init on the original machine). Copy that file to the new
|
||||
# machine before running this command.
|
||||
#
|
||||
# Safety gates (refuses with clear message):
|
||||
# - ~/.gstack/.git already exists with a DIFFERENT remote
|
||||
# - ~/.gstack/ contains non-allowlisted, non-gitignored user files
|
||||
# that would be clobbered by restore
|
||||
#
|
||||
# What it does:
|
||||
# 1. Clone the remote to a staging directory
|
||||
# 2. Validate the repo is gstack-brain-shaped (.brain-allowlist, .gitattributes)
|
||||
# 3. rsync-copy tracked files into ~/.gstack/ with skip-if-same-hash
|
||||
# 4. Move staging's .git into ~/.gstack/.git
|
||||
# 5. Register local git config merge drivers (they don't clone from remote)
|
||||
# 6. Rehydrate consumers.json endpoints; prompt for tokens
|
||||
#
|
||||
# Env:
|
||||
# GSTACK_HOME — override ~/.gstack
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
|
||||
REMOTE_FILE="$HOME/.gstack-brain-remote.txt"
|
||||
|
||||
REMOTE_URL="${1:-}"
|
||||
if [ -z "$REMOTE_URL" ]; then
|
||||
if [ -f "$REMOTE_FILE" ]; then
|
||||
REMOTE_URL=$(head -1 "$REMOTE_FILE" | tr -d '[:space:]')
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$REMOTE_URL" ]; then
|
||||
cat >&2 <<EOF
|
||||
gstack-brain-restore: no remote URL provided.
|
||||
|
||||
Provide one of:
|
||||
gstack-brain-restore <git-url>
|
||||
or put the URL in $REMOTE_FILE (copy from the original machine)
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ---- safety gates ----
|
||||
if [ -d "$GSTACK_HOME/.git" ]; then
|
||||
EXISTING_REMOTE=$(git -C "$GSTACK_HOME" remote get-url origin 2>/dev/null || echo "")
|
||||
if [ -n "$EXISTING_REMOTE" ] && [ "$EXISTING_REMOTE" != "$REMOTE_URL" ]; then
|
||||
cat >&2 <<EOF
|
||||
gstack-brain-restore: ~/.gstack/.git already points at:
|
||||
$EXISTING_REMOTE
|
||||
|
||||
You asked to restore from:
|
||||
$REMOTE_URL
|
||||
|
||||
Refusing to overwrite. Run 'gstack-brain-uninstall' first or pass a matching URL.
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- clone to staging ----
|
||||
STAGING=$(mktemp -d "${TMPDIR:-/tmp}/gstack-brain-restore.XXXXXX")
|
||||
trap 'rm -rf "$STAGING" 2>/dev/null' EXIT
|
||||
|
||||
echo "Cloning $REMOTE_URL to staging..."
|
||||
if ! git clone --quiet "$REMOTE_URL" "$STAGING/repo" 2>/dev/null; then
|
||||
echo "Clone failed. Check:" >&2
|
||||
echo " - URL is correct: $REMOTE_URL" >&2
|
||||
echo " - Auth: gh auth status (github) / glab auth status (gitlab)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ---- validate shape ----
|
||||
if [ ! -f "$STAGING/repo/.brain-allowlist" ] || [ ! -f "$STAGING/repo/.gitattributes" ]; then
|
||||
cat >&2 <<EOF
|
||||
gstack-brain-restore: $REMOTE_URL does not look like a gstack-brain repo.
|
||||
Missing: .brain-allowlist and/or .gitattributes
|
||||
|
||||
This command only works on repos created by gstack-brain-init.
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ---- validate target ~/.gstack/ has no non-gitignored user files ----
|
||||
mkdir -p "$GSTACK_HOME"
|
||||
if [ ! -d "$GSTACK_HOME/.git" ]; then
|
||||
# No existing git → check if we'd clobber anything allowlisted.
|
||||
# Read the new allowlist globs and see if any existing files would collide.
|
||||
CLOBBER_RISK=$(python3 - "$GSTACK_HOME" "$STAGING/repo/.brain-allowlist" <<'PYEOF'
|
||||
import sys, os, fnmatch
|
||||
home, allowlist_path = sys.argv[1:3]
|
||||
try:
|
||||
with open(allowlist_path) as f:
|
||||
globs = [l.strip() for l in f if l.strip() and not l.lstrip().startswith('#')]
|
||||
except FileNotFoundError:
|
||||
globs = []
|
||||
risks = []
|
||||
for root, dirs, files in os.walk(home):
|
||||
dirs[:] = [d for d in dirs if d != '.git']
|
||||
for name in files:
|
||||
full = os.path.join(root, name)
|
||||
rel = os.path.relpath(full, home)
|
||||
for g in globs:
|
||||
if fnmatch.fnmatchcase(rel, g):
|
||||
risks.append(rel)
|
||||
break
|
||||
for r in risks[:5]:
|
||||
print(r)
|
||||
if len(risks) > 5:
|
||||
print(f"...and {len(risks) - 5} more")
|
||||
sys.exit(0 if not risks else 2)
|
||||
PYEOF
|
||||
) || true
|
||||
if [ -n "$CLOBBER_RISK" ]; then
|
||||
cat >&2 <<EOF
|
||||
gstack-brain-restore: ~/.gstack/ has existing allowlisted files that would
|
||||
be clobbered by restore:
|
||||
|
||||
$CLOBBER_RISK
|
||||
|
||||
Back these up first, or run this command on a machine with an empty
|
||||
~/.gstack/. If these files are from an earlier gstack session on THIS
|
||||
machine, you probably want to run gstack-brain-init instead (to create a
|
||||
new brain repo with this machine's state).
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- copy tracked files in ----
|
||||
echo "Copying tracked files into ~/.gstack/ ..."
|
||||
# Use git-ls-tree to get exact tracked file list (avoids staged/untracked files).
|
||||
cd "$STAGING/repo"
|
||||
git ls-tree -r --name-only HEAD | while IFS= read -r rel_path; do
|
||||
src="$STAGING/repo/$rel_path"
|
||||
dst="$GSTACK_HOME/$rel_path"
|
||||
mkdir -p "$(dirname "$dst")"
|
||||
# Skip if identical (content hash). Otherwise copy.
|
||||
if [ -f "$dst" ] && cmp -s "$src" "$dst"; then
|
||||
continue
|
||||
fi
|
||||
cp "$src" "$dst"
|
||||
done
|
||||
|
||||
# ---- move .git into place ----
|
||||
if [ -d "$GSTACK_HOME/.git" ]; then
|
||||
# Existing .git with matching remote — just fetch + fast-forward.
|
||||
git -C "$GSTACK_HOME" fetch origin >/dev/null 2>&1 || true
|
||||
else
|
||||
mv "$STAGING/repo/.git" "$GSTACK_HOME/.git"
|
||||
fi
|
||||
|
||||
# ---- register merge drivers (local git config; don't survive clones) ----
|
||||
git -C "$GSTACK_HOME" config merge.jsonl-append.driver "$SCRIPT_DIR/gstack-jsonl-merge %O %A %B"
|
||||
git -C "$GSTACK_HOME" config merge.jsonl-append.name "gstack JSONL append-only merger"
|
||||
git -C "$GSTACK_HOME" config merge.union.driver "cat %A %B > %A.merged && mv %A.merged %A"
|
||||
git -C "$GSTACK_HOME" config merge.union.name "union concat"
|
||||
|
||||
# ---- install pre-commit hook (same as init) ----
|
||||
HOOK="$GSTACK_HOME/.git/hooks/pre-commit"
|
||||
mkdir -p "$(dirname "$HOOK")"
|
||||
cat > "$HOOK" <<'HOOK_EOF'
|
||||
#!/usr/bin/env bash
|
||||
set -uo pipefail
|
||||
python3 -c "
|
||||
import sys, re, subprocess
|
||||
try:
|
||||
out = subprocess.check_output(['git', 'diff', '--cached'], stderr=subprocess.DEVNULL).decode('utf-8', 'replace')
|
||||
except Exception:
|
||||
sys.exit(0)
|
||||
patterns = [
|
||||
('aws-access-key', re.compile(r'AKIA[0-9A-Z]{16}')),
|
||||
('github-token', re.compile(r'\b(gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,})')),
|
||||
('openai-key', re.compile(r'\bsk-[A-Za-z0-9_-]{20,}')),
|
||||
('pem-block', re.compile(r'-----BEGIN [A-Z ]{3,}-----')),
|
||||
('jwt', re.compile(r'\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b')),
|
||||
('bearer-token-json',
|
||||
re.compile(r'\"(authorization|api[_-]?key|apikey|token|secret|password)\"\s*:\s*\"[A-Za-z0-9_./+=-]{16,}\"',
|
||||
re.IGNORECASE)),
|
||||
]
|
||||
for name, rx in patterns:
|
||||
if rx.search(out):
|
||||
sys.stderr.write(f'gstack-brain pre-commit: refusing commit — {name} detected.\n')
|
||||
sys.exit(1)
|
||||
sys.exit(0)
|
||||
"
|
||||
HOOK_EOF
|
||||
chmod +x "$HOOK"
|
||||
|
||||
# ---- rehydrate consumers, prompt for tokens ----
|
||||
if [ -f "$GSTACK_HOME/consumers.json" ]; then
|
||||
echo ""
|
||||
echo "Consumer registry restored. Tokens are machine-local and NOT synced."
|
||||
echo "Run these for each consumer to re-enter tokens:"
|
||||
python3 - "$GSTACK_HOME/consumers.json" <<'PYEOF'
|
||||
import sys, json
|
||||
try:
|
||||
with open(sys.argv[1]) as f:
|
||||
data = json.load(f)
|
||||
except Exception:
|
||||
sys.exit(0)
|
||||
for c in data.get("consumers", []):
|
||||
name = c.get("name", "")
|
||||
token_ref = c.get("token_ref", f"{name}_token")
|
||||
print(f" gstack-config set {token_ref} <your-token>")
|
||||
PYEOF
|
||||
fi
|
||||
|
||||
# ---- write remote helper file if missing ----
|
||||
if [ ! -f "$REMOTE_FILE" ]; then
|
||||
echo "$REMOTE_URL" > "$REMOTE_FILE"
|
||||
chmod 600 "$REMOTE_FILE"
|
||||
echo ""
|
||||
echo "Wrote $REMOTE_FILE for future skill-run auto-detection."
|
||||
fi
|
||||
|
||||
cat <<EOF
|
||||
|
||||
gstack-brain-restore complete.
|
||||
Local: $GSTACK_HOME
|
||||
Remote: $REMOTE_URL
|
||||
|
||||
Next skill run will ask about privacy mode (one-time question) and then
|
||||
sync automatically at skill boundaries.
|
||||
|
||||
Status anytime: gstack-brain-sync --status
|
||||
EOF
|
||||
Executable
+447
@@ -0,0 +1,447 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-brain-sync — drain queue, commit allowlisted paths, push to remote.
|
||||
#
|
||||
# Usage:
|
||||
# gstack-brain-sync --once drain queue, commit, push (default)
|
||||
# gstack-brain-sync --status print sync health as JSON
|
||||
# gstack-brain-sync --skip-file <p> add <p> to ~/.gstack/.brain-skip.txt
|
||||
# gstack-brain-sync --drop-queue --yes clear queue without committing
|
||||
# gstack-brain-sync --discover-new scan allowlist dirs, enqueue changed files
|
||||
#
|
||||
# Invoked by the preamble at skill START and END boundaries. No persistent
|
||||
# daemon. Typical run <1s when queue empty; ~200-800ms with network push.
|
||||
#
|
||||
# Singleton enforcement: flock on ~/.gstack/.brain-sync.lock. Concurrent
|
||||
# invocations queue and serialize.
|
||||
#
|
||||
# Env:
|
||||
# GSTACK_HOME — override ~/.gstack (aligns with writers).
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
QUEUE="$GSTACK_HOME/.brain-queue.jsonl"
|
||||
ALLOWLIST="$GSTACK_HOME/.brain-allowlist"
|
||||
PRIVACY_MAP="$GSTACK_HOME/.brain-privacy-map.json"
|
||||
SKIP_FILE="$GSTACK_HOME/.brain-skip.txt"
|
||||
STATUS_FILE="$GSTACK_HOME/.brain-sync-status.json"
|
||||
LAST_PUSH_FILE="$GSTACK_HOME/.brain-last-push"
|
||||
LOCK_FILE="$GSTACK_HOME/.brain-sync.lock"
|
||||
DISCOVER_CURSOR="$GSTACK_HOME/.brain-discover-cursor"
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
|
||||
|
||||
# Remote-specific hint for auth errors (branch on origin URL).
|
||||
remote_auth_hint() {
|
||||
local url
|
||||
url=$(git -C "$GSTACK_HOME" remote get-url origin 2>/dev/null || echo "")
|
||||
case "$url" in
|
||||
*github.com*|*@github.*) echo "run: gh auth status (and gh auth refresh if needed)" ;;
|
||||
*gitlab*) echo "run: glab auth status" ;;
|
||||
*) echo "check 'git remote -v' and your credentials" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
write_status() {
|
||||
# args: status_code message [extra_json_blob]
|
||||
local code="$1"
|
||||
local msg="$2"
|
||||
local extra="${3:-{\}}"
|
||||
local ts
|
||||
ts=$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || echo "")
|
||||
python3 - "$STATUS_FILE" "$code" "$msg" "$ts" "$extra" <<'PYEOF' 2>/dev/null || true
|
||||
import json, sys
|
||||
path, code, msg, ts, extra = sys.argv[1:6]
|
||||
try:
|
||||
extra_obj = json.loads(extra) if extra else {}
|
||||
except Exception:
|
||||
extra_obj = {}
|
||||
data = {"status": code, "message": msg, "ts": ts, **extra_obj}
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f)
|
||||
f.write("\n")
|
||||
PYEOF
|
||||
}
|
||||
|
||||
# Read config; return 0 if sync active, 1 otherwise.
|
||||
sync_active() {
|
||||
if [ ! -d "$GSTACK_HOME/.git" ]; then
|
||||
return 1
|
||||
fi
|
||||
local mode
|
||||
mode=$("$CONFIG_BIN" get gbrain_sync_mode 2>/dev/null || echo off)
|
||||
[ "$mode" = "off" ] && return 1
|
||||
return 0
|
||||
}
|
||||
|
||||
# Secret regex families — stdin scan. Exits 0 clean, 1 if hit.
|
||||
# Echoes the matching pattern family name on hit. Uses python3 -c (not
|
||||
# heredoc) so sys.stdin stays available for the diff content.
|
||||
secret_scan_stdin() {
|
||||
python3 -c "
|
||||
import sys, re
|
||||
patterns = [
|
||||
('aws-access-key', re.compile(r'AKIA[0-9A-Z]{16}')),
|
||||
('github-token', re.compile(r'\\b(gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,})')),
|
||||
('openai-key', re.compile(r'\\bsk-[A-Za-z0-9_-]{20,}')),
|
||||
('pem-block', re.compile(r'-----BEGIN [A-Z ]{3,}-----')),
|
||||
('jwt', re.compile(r'\\beyJ[A-Za-z0-9_-]{10,}\\.[A-Za-z0-9_-]{10,}\\.[A-Za-z0-9_-]{10,}\\b')),
|
||||
('bearer-token-json',
|
||||
re.compile(r'\"(authorization|api[_-]?key|apikey|token|secret|password)\"\\s*:\\s*\"[A-Za-z0-9_./+=-]{16,}\"',
|
||||
re.IGNORECASE)),
|
||||
]
|
||||
text = sys.stdin.read()
|
||||
for name, rx in patterns:
|
||||
m = rx.search(text)
|
||||
if m:
|
||||
snippet = m.group(0)
|
||||
if len(snippet) > 30:
|
||||
snippet = snippet[:30] + '...'
|
||||
print(name + ':' + snippet)
|
||||
sys.exit(1)
|
||||
sys.exit(0)
|
||||
"
|
||||
}
|
||||
|
||||
# Compute matched allowlisted, privacy-filtered path set from queue.
|
||||
# Output: newline-delimited relative paths that should be staged.
|
||||
compute_paths_to_stage() {
|
||||
local mode="$1"
|
||||
python3 - "$GSTACK_HOME" "$QUEUE" "$ALLOWLIST" "$PRIVACY_MAP" "$SKIP_FILE" "$mode" <<'PYEOF'
|
||||
import sys, json, os, fnmatch, glob
|
||||
|
||||
gstack_home, queue, allowlist_path, privacy_path, skip_path, mode = sys.argv[1:7]
|
||||
|
||||
def load_lines(path):
|
||||
try:
|
||||
with open(path) as f:
|
||||
return [l.strip() for l in f if l.strip() and not l.lstrip().startswith("#")]
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
|
||||
def load_privacy_map(path):
|
||||
try:
|
||||
with open(path) as f:
|
||||
data = json.load(f)
|
||||
# Expected: [{"pattern": "glob", "class": "artifact" | "behavioral"}]
|
||||
return data if isinstance(data, list) else []
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
return []
|
||||
|
||||
allowlist_globs = load_lines(allowlist_path)
|
||||
privacy_map = load_privacy_map(privacy_path)
|
||||
skip_lines = set(load_lines(skip_path))
|
||||
|
||||
# Read queue; collect unique file paths.
|
||||
queue_paths = set()
|
||||
try:
|
||||
with open(queue) as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
obj = json.loads(line)
|
||||
p = obj.get("file")
|
||||
if isinstance(p, str):
|
||||
queue_paths.add(p)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
|
||||
def path_matches_any(path, globs):
|
||||
for pattern in globs:
|
||||
if fnmatch.fnmatchcase(path, pattern):
|
||||
return True
|
||||
return False
|
||||
|
||||
def privacy_class(path, mapping):
|
||||
for entry in mapping:
|
||||
pat = entry.get("pattern")
|
||||
if pat and fnmatch.fnmatchcase(path, pat):
|
||||
return entry.get("class", "artifact")
|
||||
# Default class when no pattern matches: artifact (safe default).
|
||||
return "artifact"
|
||||
|
||||
# mode filter: 'off' → nothing; 'artifacts-only' → only artifact class;
|
||||
# 'full' → both classes.
|
||||
def mode_allows(cls, mode):
|
||||
if mode == "off":
|
||||
return False
|
||||
if mode == "artifacts-only":
|
||||
return cls == "artifact"
|
||||
return True # full
|
||||
|
||||
final = []
|
||||
for p in sorted(queue_paths):
|
||||
if p in skip_lines:
|
||||
continue
|
||||
# Must be under GSTACK_HOME root. Reject absolute + reject ../ escape.
|
||||
if p.startswith("/") or ".." in p.split("/"):
|
||||
continue
|
||||
# Must match at least one allowlist glob.
|
||||
if not path_matches_any(p, allowlist_globs):
|
||||
continue
|
||||
# Must survive privacy mode filter.
|
||||
cls = privacy_class(p, privacy_map)
|
||||
if not mode_allows(cls, mode):
|
||||
continue
|
||||
# Must exist on disk — can't stage what isn't there.
|
||||
if not os.path.exists(os.path.join(gstack_home, p)):
|
||||
continue
|
||||
final.append(p)
|
||||
|
||||
for p in final:
|
||||
print(p)
|
||||
PYEOF
|
||||
}
|
||||
|
||||
subcmd_once() {
|
||||
if ! sync_active; then
|
||||
# Silent no-op when feature not initialized / disabled.
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Singleton lock via atomic mkdir. `flock(1)` isn't on macOS by default;
|
||||
# `mkdir` is atomic on every POSIX filesystem. If another --once is already
|
||||
# running, skip (don't wait) — the next skill boundary will catch up.
|
||||
local lock_dir="${LOCK_FILE}.d"
|
||||
if ! mkdir "$lock_dir" 2>/dev/null; then
|
||||
# Is the lock stale? Check the pidfile inside. If process is dead, clear it.
|
||||
if [ -f "$lock_dir/pid" ]; then
|
||||
local lock_pid
|
||||
lock_pid=$(cat "$lock_dir/pid" 2>/dev/null || echo "")
|
||||
if [ -n "$lock_pid" ] && ! kill -0 "$lock_pid" 2>/dev/null; then
|
||||
# Stale lock — clear and retry once.
|
||||
rm -rf "$lock_dir" 2>/dev/null || true
|
||||
if ! mkdir "$lock_dir" 2>/dev/null; then
|
||||
exit 0
|
||||
fi
|
||||
else
|
||||
# Lock is held by a live process.
|
||||
exit 0
|
||||
fi
|
||||
else
|
||||
# Lock dir without pidfile — treat as held; don't touch.
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
echo "$$" > "$lock_dir/pid" 2>/dev/null || true
|
||||
|
||||
local mode
|
||||
mode=$("$CONFIG_BIN" get gbrain_sync_mode 2>/dev/null || echo off)
|
||||
|
||||
local paths_file
|
||||
paths_file=$(mktemp /tmp/brain-sync-paths.XXXXXX) || { rm -rf "$lock_dir" 2>/dev/null; write_status "error" "mktemp failed"; exit 1; }
|
||||
# Single trap covers both: lock cleanup AND tempfile cleanup.
|
||||
trap 'rm -f "$paths_file" 2>/dev/null; rm -rf "$lock_dir" 2>/dev/null || true' EXIT INT TERM
|
||||
|
||||
compute_paths_to_stage "$mode" > "$paths_file"
|
||||
if [ ! -s "$paths_file" ]; then
|
||||
# Nothing to stage. Clear any stale queue entries and exit.
|
||||
: > "$QUEUE"
|
||||
write_status "idle" "no allowlisted changes in queue"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Stage with git add -f (forces past .gitignore=*) explicit paths only.
|
||||
while IFS= read -r p; do
|
||||
[ -z "$p" ] && continue
|
||||
git -C "$GSTACK_HOME" add -f -- "$p" 2>/dev/null || true
|
||||
done < "$paths_file"
|
||||
|
||||
# Secret-scan staged diff.
|
||||
local scan_out
|
||||
scan_out=$(git -C "$GSTACK_HOME" diff --cached 2>/dev/null | secret_scan_stdin || true)
|
||||
if [ -n "$scan_out" ]; then
|
||||
# Hit — unstage, preserve queue, write loud status.
|
||||
git -C "$GSTACK_HOME" reset HEAD -- . >/dev/null 2>&1 || true
|
||||
local hint
|
||||
hint="secret pattern detected ($scan_out). Remediation: review the staged file, then run: gstack-brain-sync --skip-file <path> OR edit the content."
|
||||
write_status "blocked" "$hint"
|
||||
echo "BRAIN_SYNC: blocked: $scan_out" >&2
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Commit with template message.
|
||||
local n ts
|
||||
n=$(wc -l < "$paths_file" | tr -d ' ')
|
||||
ts=$(date -u +%Y-%m-%dT%H:%M:%SZ)
|
||||
local msg="sync: $n file(s) | $ts"
|
||||
git -C "$GSTACK_HOME" -c user.email="gstack@localhost" -c user.name="gstack-brain-sync" \
|
||||
commit -q -m "$msg" 2>/dev/null || {
|
||||
# Nothing to commit (e.g. all files already committed).
|
||||
: > "$QUEUE"
|
||||
write_status "idle" "queue drained but no new changes to commit"
|
||||
exit 0
|
||||
}
|
||||
|
||||
# Push. On reject, fetch + merge (merge driver handles JSONL) + retry once.
|
||||
local push_err
|
||||
push_err=$(git -C "$GSTACK_HOME" push origin HEAD 2>&1 >/dev/null) || {
|
||||
# Check if this is an auth error first — no point retrying.
|
||||
if echo "$push_err" | grep -qiE "auth|permission|403|401|forbidden"; then
|
||||
local hint
|
||||
hint=$(remote_auth_hint)
|
||||
write_status "push_failed" "push failed: auth error. fix: $hint"
|
||||
echo "BRAIN_SYNC: push failed: auth. fix: $hint" >&2
|
||||
# Queue cleared because the commit exists locally; next push will send it.
|
||||
: > "$QUEUE"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Try a fetch-and-merge + retry.
|
||||
if git -C "$GSTACK_HOME" fetch origin 2>/dev/null; then
|
||||
local branch
|
||||
branch=$(git -C "$GSTACK_HOME" rev-parse --abbrev-ref HEAD 2>/dev/null || echo main)
|
||||
if git -C "$GSTACK_HOME" merge --no-edit "origin/$branch" >/dev/null 2>&1; then
|
||||
if git -C "$GSTACK_HOME" push origin HEAD 2>/dev/null; then
|
||||
: > "$QUEUE"
|
||||
date -u +%Y-%m-%dT%H:%M:%SZ > "$LAST_PUSH_FILE"
|
||||
write_status "ok" "pushed $n file(s) after rebase"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
write_status "push_failed" "push failed: $(printf '%s' "$push_err" | head -1)"
|
||||
: > "$QUEUE"
|
||||
exit 0
|
||||
}
|
||||
|
||||
# Success: clear queue, update last-push.
|
||||
: > "$QUEUE"
|
||||
date -u +%Y-%m-%dT%H:%M:%SZ > "$LAST_PUSH_FILE"
|
||||
write_status "ok" "pushed $n file(s)"
|
||||
exit 0
|
||||
}
|
||||
|
||||
subcmd_status() {
|
||||
if [ -f "$STATUS_FILE" ]; then
|
||||
cat "$STATUS_FILE"
|
||||
else
|
||||
echo '{"status":"unknown","message":"no status file yet"}'
|
||||
fi
|
||||
# Supplemental info (not in status file).
|
||||
local queue_depth=0
|
||||
[ -f "$QUEUE" ] && queue_depth=$(wc -l < "$QUEUE" | tr -d ' ')
|
||||
local last_push="never"
|
||||
[ -f "$LAST_PUSH_FILE" ] && last_push=$(cat "$LAST_PUSH_FILE" 2>/dev/null || echo never)
|
||||
local mode
|
||||
mode=$("$CONFIG_BIN" get gbrain_sync_mode 2>/dev/null || echo off)
|
||||
printf '{"queue_depth":%s,"last_push":"%s","mode":"%s"}\n' "$queue_depth" "$last_push" "$mode"
|
||||
}
|
||||
|
||||
subcmd_skip_file() {
|
||||
local path="${1:-}"
|
||||
if [ -z "$path" ]; then
|
||||
echo "Usage: gstack-brain-sync --skip-file <path>" >&2
|
||||
exit 1
|
||||
fi
|
||||
mkdir -p "$GSTACK_HOME"
|
||||
# Avoid duplicate entries.
|
||||
if [ -f "$SKIP_FILE" ] && grep -Fxq "$path" "$SKIP_FILE"; then
|
||||
echo "already in skip list: $path"
|
||||
exit 0
|
||||
fi
|
||||
echo "$path" >> "$SKIP_FILE"
|
||||
echo "added to skip list: $path"
|
||||
echo "(future writers will not enqueue this path; existing queue entries ignored on next --once)"
|
||||
}
|
||||
|
||||
subcmd_drop_queue() {
|
||||
local force="${1:-}"
|
||||
if [ "$force" != "--yes" ]; then
|
||||
echo "Refusing: --drop-queue discards pending syncs. Pass --yes to confirm." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$QUEUE" ]; then
|
||||
echo "queue already empty"
|
||||
exit 0
|
||||
fi
|
||||
local n
|
||||
n=$(wc -l < "$QUEUE" | tr -d ' ')
|
||||
: > "$QUEUE"
|
||||
echo "dropped $n queue entries"
|
||||
}
|
||||
|
||||
subcmd_discover_new() {
|
||||
if ! sync_active; then
|
||||
exit 0
|
||||
fi
|
||||
# Walk allowlist globs; enqueue any file where mtime+size differs from cursor.
|
||||
python3 - "$GSTACK_HOME" "$ALLOWLIST" "$DISCOVER_CURSOR" "$SCRIPT_DIR/gstack-brain-enqueue" <<'PYEOF' 2>/dev/null || true
|
||||
import sys, os, json, glob, fnmatch, subprocess, hashlib
|
||||
|
||||
gstack_home, allowlist_path, cursor_path, enqueue_bin = sys.argv[1:5]
|
||||
|
||||
def load_lines(path):
|
||||
try:
|
||||
with open(path) as f:
|
||||
return [l.strip() for l in f if l.strip() and not l.lstrip().startswith("#")]
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
|
||||
def load_cursor(path):
|
||||
try:
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
return {}
|
||||
|
||||
def save_cursor(path, data):
|
||||
try:
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
allowlist = load_lines(allowlist_path)
|
||||
cursor = load_cursor(cursor_path)
|
||||
new_cursor = dict(cursor)
|
||||
|
||||
# Walk all files under gstack_home, match against allowlist.
|
||||
for root, dirs, files in os.walk(gstack_home):
|
||||
# Skip .git and .brain-* state files.
|
||||
if ".git" in root.split(os.sep):
|
||||
continue
|
||||
for name in files:
|
||||
full = os.path.join(root, name)
|
||||
rel = os.path.relpath(full, gstack_home)
|
||||
if rel.startswith(".brain-"):
|
||||
continue
|
||||
matched = any(fnmatch.fnmatchcase(rel, pat) for pat in allowlist)
|
||||
if not matched:
|
||||
continue
|
||||
try:
|
||||
st = os.stat(full)
|
||||
key = f"{int(st.st_mtime)}:{st.st_size}"
|
||||
except OSError:
|
||||
continue
|
||||
prev = cursor.get(rel)
|
||||
if prev != key:
|
||||
# Enqueue via the shim (respects sync mode + skip list).
|
||||
subprocess.run([enqueue_bin, rel], check=False)
|
||||
new_cursor[rel] = key
|
||||
|
||||
save_cursor(cursor_path, new_cursor)
|
||||
PYEOF
|
||||
}
|
||||
|
||||
# -------- dispatch --------
|
||||
case "${1:-}" in
|
||||
--once|"") subcmd_once ;;
|
||||
--status) subcmd_status ;;
|
||||
--skip-file) shift; subcmd_skip_file "${1:-}" ;;
|
||||
--drop-queue) shift; subcmd_drop_queue "${1:-}" ;;
|
||||
--discover-new) subcmd_discover_new ;;
|
||||
--help|-h)
|
||||
sed -n '2,18p' "$0" | sed 's/^# \{0,1\}//'
|
||||
;;
|
||||
*)
|
||||
echo "Unknown subcommand: $1" >&2
|
||||
echo "Run: gstack-brain-sync --help" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
Executable
+145
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-brain-uninstall — clean off-ramp for gstack-brain sync.
|
||||
#
|
||||
# Usage:
|
||||
# gstack-brain-uninstall [--yes] [--delete-remote]
|
||||
#
|
||||
# Removes the git layer from ~/.gstack/ and clears sync config. Your local
|
||||
# gstack memory (learnings, timelines, etc.) is NOT touched — this is an
|
||||
# uninstall-sync command, not a delete-data command.
|
||||
#
|
||||
# Flags:
|
||||
# --yes Skip the confirmation prompt.
|
||||
# --delete-remote Also delete the GitHub repo via `gh repo delete`
|
||||
# (interactive unless --yes is also passed).
|
||||
#
|
||||
# What it removes (in ~/.gstack/):
|
||||
# .git/ — the sync repo's git data
|
||||
# .gitignore — canonical ignore-all marker
|
||||
# .gitattributes — merge driver declarations
|
||||
# .brain-allowlist — sync path list
|
||||
# .brain-privacy-map.json — sync privacy classifier
|
||||
# .brain-queue.jsonl — pending queue
|
||||
# .brain-discover-cursor — discover-new cursor
|
||||
# .brain-last-push — timestamp marker
|
||||
# .brain-skip.txt — user-maintained skip list
|
||||
# .brain-sync.lock.d/ — lock dir (if present)
|
||||
# .brain-sync-status.json — health status
|
||||
# consumers.json — consumer/reader registry
|
||||
#
|
||||
# What it clears (via gstack-config):
|
||||
# gbrain_sync_mode → off
|
||||
# gbrain_sync_mode_prompted → false (so user re-prompts on re-init)
|
||||
#
|
||||
# What it does NOT touch:
|
||||
# Project data (projects/*, retros/*, developer-profile.json, etc.)
|
||||
# Consumer tokens in gstack-config (<name>_token keys)
|
||||
# ~/.gstack-brain-remote.txt in your home directory
|
||||
# The actual remote git repo (unless --delete-remote)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
|
||||
REMOTE_FILE="$HOME/.gstack-brain-remote.txt"
|
||||
|
||||
ASSUME_YES=0
|
||||
DELETE_REMOTE=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--yes|-y) ASSUME_YES=1; shift ;;
|
||||
--delete-remote) DELETE_REMOTE=1; shift ;;
|
||||
--help|-h) sed -n '2,30p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) echo "Unknown flag: $1" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ ! -d "$GSTACK_HOME/.git" ]; then
|
||||
echo "gstack-brain-uninstall: nothing to do (~/.gstack/.git doesn't exist)."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
REMOTE_URL=$(git -C "$GSTACK_HOME" remote get-url origin 2>/dev/null || echo "")
|
||||
|
||||
# ---- confirmation ----
|
||||
if [ "$ASSUME_YES" != "1" ]; then
|
||||
cat <<EOF
|
||||
This will remove gstack-brain sync from this machine:
|
||||
- Remove ~/.gstack/.git and sync config files
|
||||
- Clear gbrain_sync_mode in gstack-config
|
||||
- Remote: $REMOTE_URL will be $([ "$DELETE_REMOTE" = "1" ] && echo "DELETED" || echo "kept")
|
||||
|
||||
Local memory (learnings, plans, etc.) is NOT touched.
|
||||
|
||||
EOF
|
||||
printf "Proceed? [y/N] "
|
||||
read -r reply
|
||||
case "$reply" in
|
||||
y|Y|yes|Yes) ;;
|
||||
*) echo "Aborted."; exit 0 ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# ---- delete remote if requested ----
|
||||
if [ "$DELETE_REMOTE" = "1" ] && [ -n "$REMOTE_URL" ]; then
|
||||
case "$REMOTE_URL" in
|
||||
*github.com*|*@github*)
|
||||
if command -v gh >/dev/null 2>&1; then
|
||||
# Extract owner/repo from URL.
|
||||
REPO_SLUG=$(echo "$REMOTE_URL" | sed -E 's#.*[:/]([^/:]+/[^/]+)(\.git)?$#\1#' | sed 's/\.git$//')
|
||||
if [ -n "$REPO_SLUG" ]; then
|
||||
echo "Deleting GitHub repo: $REPO_SLUG"
|
||||
if [ "$ASSUME_YES" = "1" ]; then
|
||||
gh repo delete "$REPO_SLUG" --yes 2>/dev/null || echo "gh repo delete failed; continuing local uninstall"
|
||||
else
|
||||
gh repo delete "$REPO_SLUG" 2>/dev/null || echo "gh repo delete failed; continuing local uninstall"
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "--delete-remote requires the gh CLI. Skipping remote deletion."
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
echo "--delete-remote only supports github.com remotes. Delete manually if needed: $REMOTE_URL"
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# ---- remove sync files ----
|
||||
echo "Removing git layer and sync config files..."
|
||||
rm -rf "$GSTACK_HOME/.git" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.gitignore" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.gitattributes" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-allowlist" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-privacy-map.json" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-queue.jsonl" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-discover-cursor" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-last-push" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-last-pull" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-skip.txt" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-sync-status.json" 2>/dev/null || true
|
||||
rm -rf "$GSTACK_HOME/.brain-sync.lock.d" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/consumers.json" 2>/dev/null || true
|
||||
|
||||
# ---- clear config keys ----
|
||||
"$CONFIG_BIN" set gbrain_sync_mode off >/dev/null 2>&1 || true
|
||||
"$CONFIG_BIN" set gbrain_sync_mode_prompted false >/dev/null 2>&1 || true
|
||||
|
||||
# ---- leave remote-helper file alone unless user asked to delete remote ----
|
||||
if [ "$DELETE_REMOTE" = "1" ]; then
|
||||
rm -f "$REMOTE_FILE" 2>/dev/null || true
|
||||
else
|
||||
if [ -f "$REMOTE_FILE" ]; then
|
||||
echo "(keeping $REMOTE_FILE — remove manually if you want to forget the URL)"
|
||||
fi
|
||||
fi
|
||||
|
||||
cat <<EOF
|
||||
|
||||
gstack-brain uninstall complete.
|
||||
Sync is off. ~/.gstack/ is a plain directory again.
|
||||
Your project data, learnings, and profile are untouched.
|
||||
|
||||
To re-enable sync later: gstack-brain-init
|
||||
EOF
|
||||
+26
-4
@@ -8,10 +8,11 @@
|
||||
# gstack-config defaults — show just the defaults table
|
||||
#
|
||||
# Env overrides (for testing):
|
||||
# GSTACK_STATE_DIR — override ~/.gstack state directory
|
||||
# GSTACK_HOME — override ~/.gstack state directory (aligns with writer scripts)
|
||||
# GSTACK_STATE_DIR — legacy alias for GSTACK_HOME (kept for backwards compat)
|
||||
set -euo pipefail
|
||||
|
||||
STATE_DIR="${GSTACK_STATE_DIR:-$HOME/.gstack}"
|
||||
STATE_DIR="${GSTACK_HOME:-${GSTACK_STATE_DIR:-$HOME/.gstack}}"
|
||||
CONFIG_FILE="$STATE_DIR/config.yaml"
|
||||
|
||||
# Annotated header for new config files. Written once on first `set`.
|
||||
@@ -59,6 +60,19 @@ CONFIG_HEADER='# gstack configuration — edit freely, changes take effect on ne
|
||||
# # Unknown values default to "default" with a warning.
|
||||
# # See docs/designs/PLAN_TUNING_V1.md for rationale.
|
||||
#
|
||||
# ─── GBrain sync (v1.7+) ─────────────────────────────────────────────
|
||||
# gbrain_sync_mode: off # off | artifacts-only | full
|
||||
# # off — no sync (default)
|
||||
# # artifacts-only — sync plans/designs/retros/learnings only
|
||||
# # (skip behavioral data: question-log,
|
||||
# # developer-profile, timeline)
|
||||
# # full — sync everything allowlisted
|
||||
# # Set by the first-run privacy stop-gate. See docs/gbrain-sync.md.
|
||||
#
|
||||
# gbrain_sync_mode_prompted: false
|
||||
# # Set to true once the privacy gate has asked the user.
|
||||
# # Flip back to false to be re-prompted.
|
||||
#
|
||||
# ─── Advanced ────────────────────────────────────────────────────────
|
||||
# codex_reviews: enabled # disabled = skip Codex adversarial reviews in /ship
|
||||
# gstack_contributor: false # true = file field reports when gstack misbehaves
|
||||
@@ -83,6 +97,8 @@ lookup_default() {
|
||||
gstack_contributor) echo "false" ;;
|
||||
skip_eng_review) echo "false" ;;
|
||||
cross_project_learnings) echo "" ;; # intentionally empty → unset triggers first-time prompt
|
||||
gbrain_sync_mode) echo "off" ;;
|
||||
gbrain_sync_mode_prompted) echo "false" ;;
|
||||
*) echo "" ;;
|
||||
esac
|
||||
}
|
||||
@@ -114,6 +130,10 @@ case "${1:-}" in
|
||||
echo "Warning: explain_level '$VALUE' not recognized. Valid values: default, terse. Using default." >&2
|
||||
VALUE="default"
|
||||
fi
|
||||
if [ "$KEY" = "gbrain_sync_mode" ] && [ "$VALUE" != "off" ] && [ "$VALUE" != "artifacts-only" ] && [ "$VALUE" != "full" ]; then
|
||||
echo "Warning: gbrain_sync_mode '$VALUE' not recognized. Valid values: off, artifacts-only, full. Using off." >&2
|
||||
VALUE="off"
|
||||
fi
|
||||
mkdir -p "$STATE_DIR"
|
||||
# Write annotated header on first creation
|
||||
if [ ! -f "$CONFIG_FILE" ]; then
|
||||
@@ -142,7 +162,8 @@ case "${1:-}" in
|
||||
echo "# ─── Active values (including defaults for unset keys) ───"
|
||||
for KEY in proactive routing_declined telemetry auto_upgrade update_check \
|
||||
skill_prefix checkpoint_mode checkpoint_push codex_reviews \
|
||||
gstack_contributor skip_eng_review; do
|
||||
gstack_contributor skip_eng_review gbrain_sync_mode \
|
||||
gbrain_sync_mode_prompted; do
|
||||
VALUE=$(grep -E "^${KEY}:" "$CONFIG_FILE" 2>/dev/null | tail -1 | awk '{print $2}' | tr -d '[:space:]' || true)
|
||||
SOURCE="default"
|
||||
if [ -n "$VALUE" ]; then
|
||||
@@ -157,7 +178,8 @@ case "${1:-}" in
|
||||
echo "# gstack-config defaults"
|
||||
for KEY in proactive routing_declined telemetry auto_upgrade update_check \
|
||||
skill_prefix checkpoint_mode checkpoint_push codex_reviews \
|
||||
gstack_contributor skip_eng_review; do
|
||||
gstack_contributor skip_eng_review gbrain_sync_mode \
|
||||
gbrain_sync_mode_prompted; do
|
||||
printf ' %-24s %s\n' "$KEY:" "$(lookup_default "$KEY")"
|
||||
done
|
||||
;;
|
||||
|
||||
@@ -101,6 +101,10 @@ do_migrate() {
|
||||
mv "$TMPOUT" "$PROFILE_FILE"
|
||||
trap - EXIT
|
||||
|
||||
# gbrain-sync: enqueue the migrated file for cross-machine sync (no-op if off).
|
||||
SCRIPT_DIR_E="$(cd "$(dirname "$0")" && pwd)"
|
||||
"$SCRIPT_DIR_E/gstack-brain-enqueue" "developer-profile.json" 2>/dev/null &
|
||||
|
||||
# Archive the legacy file.
|
||||
local TS
|
||||
TS="$(date +%Y-%m-%d-%H%M%S)"
|
||||
|
||||
Executable
+88
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-jsonl-merge — git merge driver for append-only JSONL files.
|
||||
#
|
||||
# Usage (called by git, not by users):
|
||||
# gstack-jsonl-merge <base> <ours> <theirs>
|
||||
#
|
||||
# Registered in local git config by bin/gstack-brain-init and
|
||||
# bin/gstack-brain-restore:
|
||||
# git config merge.jsonl-append.driver \
|
||||
# "$GSTACK_BIN/gstack-jsonl-merge %O %A %B"
|
||||
#
|
||||
# Behavior:
|
||||
# Concatenate base + ours + theirs, dedup exact-duplicate lines, sort by
|
||||
# ISO "ts" field when present, fall back to SHA-256 of the line for
|
||||
# deterministic order. Write result to <ours> (the %A file per the git
|
||||
# merge-driver contract).
|
||||
#
|
||||
# Two machines appending to the same JSONL file between pushes produces
|
||||
# a same-line conflict at the file tail. This driver resolves it cleanly:
|
||||
# both appends survive, ordered by wall-clock timestamp where available,
|
||||
# content hash otherwise.
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 — merge succeeded, result written to <ours>
|
||||
# 1 — error; git treats as conflict and stops the merge
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
if [ "$#" -lt 3 ]; then
|
||||
echo "gstack-jsonl-merge: expected 3 args (base ours theirs), got $#" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BASE="$1"
|
||||
OURS="$2"
|
||||
THEIRS="$3"
|
||||
|
||||
TMP=$(mktemp /tmp/gstack-jsonl-merge.XXXXXX) || exit 1
|
||||
trap 'rm -f "$TMP" 2>/dev/null || true' EXIT
|
||||
|
||||
python3 - "$BASE" "$OURS" "$THEIRS" > "$TMP" <<'PYEOF'
|
||||
import sys, json, hashlib
|
||||
|
||||
paths = sys.argv[1:4] # base, ours, theirs
|
||||
seen = {} # line content -> sort_key
|
||||
|
||||
for path in paths:
|
||||
try:
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
line = line.rstrip('\n')
|
||||
if not line:
|
||||
continue
|
||||
if line in seen:
|
||||
continue
|
||||
# Prefer ISO ts field for sort; fall back to SHA-256.
|
||||
sort_key = None
|
||||
try:
|
||||
obj = json.loads(line)
|
||||
ts = obj.get('ts') or obj.get('timestamp')
|
||||
if isinstance(ts, str):
|
||||
sort_key = (0, ts)
|
||||
except (json.JSONDecodeError, ValueError, TypeError):
|
||||
pass
|
||||
if sort_key is None:
|
||||
h = hashlib.sha256(line.encode('utf-8')).hexdigest()
|
||||
sort_key = (1, h)
|
||||
seen[line] = sort_key
|
||||
except FileNotFoundError:
|
||||
# Absent base / absent ours / absent theirs are all valid.
|
||||
continue
|
||||
except OSError:
|
||||
# Permission / IO errors are fatal — caller sees non-zero exit.
|
||||
sys.exit(1)
|
||||
|
||||
# Timestamp-ordered entries first (group 0), then hash-ordered (group 1).
|
||||
for line, _ in sorted(seen.items(), key=lambda item: item[1]):
|
||||
print(line)
|
||||
PYEOF
|
||||
|
||||
_PYEXIT=$?
|
||||
if [ "$_PYEXIT" != "0" ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mv "$TMP" "$OURS" || exit 1
|
||||
trap - EXIT
|
||||
exit 0
|
||||
@@ -84,3 +84,6 @@ if [ $? -ne 0 ] || [ -z "$VALIDATED" ]; then
|
||||
fi
|
||||
|
||||
echo "$VALIDATED" >> "$GSTACK_HOME/projects/$SLUG/learnings.jsonl"
|
||||
|
||||
# gbrain-sync: enqueue for cross-machine sync (no-op if sync is off).
|
||||
"$SCRIPT_DIR/gstack-brain-enqueue" "projects/$SLUG/learnings.jsonl" 2>/dev/null &
|
||||
|
||||
@@ -165,3 +165,7 @@ if [ $VALIDATE_RC -ne 0 ] || [ -z "$VALIDATED" ]; then
|
||||
fi
|
||||
|
||||
echo "$VALIDATED" >> "$GSTACK_HOME/projects/$SLUG/question-log.jsonl"
|
||||
|
||||
# NOTE: question-log.jsonl is deliberately NOT enqueued for gbrain-sync.
|
||||
# Per Codex v2 review, audit/derivation data stays local alongside the
|
||||
# question-preferences.json it annotates.
|
||||
|
||||
@@ -16,3 +16,6 @@ if ! printf '%s' "$INPUT" | bun -e "JSON.parse(await Bun.stdin.text())" 2>/dev/n
|
||||
fi
|
||||
|
||||
echo "$INPUT" >> "$GSTACK_HOME/projects/$SLUG/$BRANCH-reviews.jsonl"
|
||||
|
||||
# gbrain-sync: enqueue for cross-machine sync (no-op if sync is off).
|
||||
"$SCRIPT_DIR/gstack-brain-enqueue" "projects/$SLUG/$BRANCH-reviews.jsonl" 2>/dev/null &
|
||||
|
||||
@@ -2,7 +2,10 @@
|
||||
# gstack-timeline-log — append a timeline event to the project timeline
|
||||
# Usage: gstack-timeline-log '{"skill":"review","event":"started","branch":"main"}'
|
||||
#
|
||||
# Session timeline: local-only, never sent anywhere.
|
||||
# Session timeline: local by default. If the user enables `gbrain_sync_mode`
|
||||
# with the `full` (not `artifacts-only`) privacy tier — via the first-run
|
||||
# stop-gate from `gstack-brain-init` or the preamble — timeline events are
|
||||
# published to the user's private GBrain sync repo. See docs/gbrain-sync.md.
|
||||
# Required fields: skill, event (started|completed).
|
||||
# Optional: branch, outcome, duration_s, session, ts.
|
||||
# Validation failure → skip silently (non-blocking).
|
||||
@@ -32,3 +35,6 @@ if ! printf '%s' "$INPUT" | bun -e "const j=JSON.parse(await Bun.stdin.text());
|
||||
fi
|
||||
|
||||
echo "$INPUT" >> "$GSTACK_HOME/projects/$SLUG/timeline.jsonl"
|
||||
|
||||
# gbrain-sync: enqueue for cross-machine sync (no-op if sync is off).
|
||||
"$SCRIPT_DIR/gstack-brain-enqueue" "projects/$SLUG/timeline.jsonl" 2>/dev/null &
|
||||
|
||||
Reference in New Issue
Block a user