mirror of https://github.com/garrytan/gstack.git synced 2026-05-01 19:25:10 +02:00

Files

T

Garry Tan 675717e320 v1.17.0.0: setup-gbrain wireup ships the gbrain federation surface (#1234 )

* feat: gstack-gbrain-source-wireup helper + 13 unit tests

The new bin/gstack-gbrain-source-wireup is the single helper that registers
the gstack brain repo as a gbrain federated source via `git worktree`, runs
incremental sync, and supports --uninstall + --probe + --strict modes.

Replaces the dead `consumers.json + ingest_url + /ingest-repo` HTTP wireup
introduced in v1.12.0.0 — that endpoint never shipped on the gbrain side.
The federation surface (`gbrain sources` / `gbrain sync`) shipped in gbrain
v0.18.0; this helper adapts to its actual semantics (no `sources update`, so
path drift recovery is `remove + re-add`; no `--install-cron` either, so
freshness rides on the existing skill-end push hook).

Source-id derivation is multi-fallback: ~/.gstack/.git origin URL →
~/.gstack-brain-remote.txt → --source-id flag. This makes `--uninstall`
work even after `~/.gstack/.git` is destroyed by the parent uninstall script.

Worktree is `--detach`ed at $GSTACK_HOME's HEAD because main is already
checked out there; advance is a re-checkout of the parent's current HEAD,
not a `git pull`. Divergence recovery removes + re-adds the worktree.

Test suite covers 13 cases: fresh-state registration, idempotent re-runs,
drift recovery, --strict failure modes, source-id fallback chain, --probe
non-mutation, sync errors, and --uninstall. Fake gbrain on $PATH, real git
ops at GSTACK_HOME tmp dir.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: wire setup-gbrain + brain-restore + brain-uninstall to use the helper

setup-gbrain Step 7 now invokes gstack-gbrain-source-wireup --strict after
gstack-brain-init + gbrain_sync_mode is set. Strict mode means the user sees
the failure rather than silently ending up with an unwired brain.

bin/gstack-brain-init drops 60 lines of dead code: the HTTP POST to
${GBRAIN_URL}/ingest-repo, the GBRAIN_URL_VAL/GBRAIN_TOKEN_VAL probes, the
consumers.json writer, and the chore commit step. CONSUMERS_FILE variable
declaration removed. The closing message no longer points at the dead
gstack-brain-consumer add path.

bin/gstack-brain-restore drops the 18-line consumers.json token-rehydration
block (was a no-op for the only consumer that ever existed). Adds a
best-effort wireup invocation after the brain-repo clone so 2nd-Mac restore
gets gbrain federation automatically. Failure prints a stderr WARNING but
does not abort the restore — restore's primary job is the git clone.

bin/gstack-brain-uninstall calls the helper's --uninstall mode (which
removes the gbrain source registration, the git worktree, and the
future-launchd-plist stub) before the existing legacy consumers.json
removal. Ordering is fragile-by-design: helper derives source-id via
multi-fallback so it works even after .git is destroyed.

bin/gstack-brain-consumer gets a DEPRECATED header note. Stays in the tree
for one cycle of grace; removal in v1.13.0.0.

setup-gbrain/SKILL.md is regenerated from the .tmpl via gen:skill-docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: v1.12.3.0 migration — wire existing brain-sync repos into gbrain

Idempotent migration script. For users who already opted into brain-sync
before this release (gbrain_sync_mode != off, ~/.gstack/.git exists), runs
the new gstack-gbrain-source-wireup helper so their existing brain repo
becomes searchable via gbrain immediately on /gstack-upgrade.

Skip conditions (each ends with exit 0):
- HOME unset or empty (defensive)
- gbrain_sync_mode = off or empty (user opted out)
- no ~/.gstack/.git (brain-init never ran)
- helper missing on disk (broken install)

No --strict on the helper invocation: missing or old gbrain is a benign
skip during a batch upgrade rather than a blocker.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v1.12.3.0: setup-gbrain wireup ships the gbrain federation surface

Bumps VERSION 1.12.2.0 → 1.12.3.0 with a release-notes-format entry in
CHANGELOG.md. After upgrade, the placeholder consumers.json wireup is gone,
gbrain sources + sync + skill-end hook is the new path, your gstack memory
is actually searchable in gbrain.

The CHANGELOG entry follows the release-summary format from CLAUDE.md:
two-line bold headline, lead paragraph naming what shipped, "verify after
upgrade" command block readers can run on their own brain to see the
delta, then the standard Itemized changes / What this means / For
contributors sections.

Three pre-existing test failures on this branch are flagged in the
contributor section: the GSTACK_HOME isolation test (reads Garry's actual
~/.gstack/config.yaml), the 2MB tracked-binary test (security-bench
fixtures > 2MB), and the Opus 4.7 pacing-directive test (overlay text
drifted). All three were verified to fail on the base branch too — out
of scope for this PR, follow-up needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: helper locks GBRAIN_DATABASE_URL at startup, defends against config rewrites

The wireup helper previously read ~/.gbrain/config.json on every gbrain
subprocess invocation. On Garry's Mac, multiple concurrent test runs and
agent integrations were rewriting that file mid-sync, redirecting the
wireup at the wrong brain partway through a 4-min initial import.

This commit adds a `--database-url <url>` flag to the helper and locks
the URL at startup. Precedence:
1. --database-url flag (explicit caller intent)
2. GBRAIN_DATABASE_URL / DATABASE_URL env (CI / manual override)
3. read once from ~/.gbrain/config.json (default)

Whichever wins gets exported as GBRAIN_DATABASE_URL for every child
`gbrain` invocation. Per gbrain's loadConfig at src/core/config.ts:53,
env-var URLs override the file URL — so a process that flips config.json
between two of our gbrain calls can't redirect us. Defense-in-depth:
once the URL is locked, the wireup completes against the original brain
even under hostile filesystem conditions.

setup-gbrain/SKILL.md.tmpl Step 7 now reads the URL out of config.json
once (via python3 inline) and passes it explicitly with --database-url,
so even the very first wireup call is decoupled from config.json mutability.

Three new test cases cover the lock behavior:
- --database-url flag is exported to child gbrain calls
- falls back to ~/.gbrain/config.json when no flag and no env
- flag overrides env GBRAIN_DATABASE_URL and config.json values

The fake gbrain in the test suite now records GBRAIN_DATABASE_URL alongside
each call so tests can assert the helper exported the locked URL.

Total test count: 13 → 16 passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump v1.12.3.0 references to v1.15.1.0 to match merged-with-main release

Internal-only renames after merging origin/main bumped this branch's release
target from v1.12.3.0 → v1.15.1.0:

- gstack-upgrade/migrations/v1.12.3.0.sh → v1.15.1.0.sh (rename + log-prefix
bump from "[v1.12.3.0]" to "[v1.15.1.0]")
- bin/gstack-brain-consumer header: "DEPRECATED in v1.12.3.0" → "DEPRECATED in
v1.15.1.0"; removal target bumped from v1.13.0.0 → v1.16.0.0 (next minor
after v1.15.1.0).
- bin/gstack-brain-uninstall: "no longer written ... since v1.12.3.0" →
"since v1.15.1.0".

No behavior change. Test suite still 16/16 passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: 10 new cases close coverage gaps (helper defensive paths + migration)

/ship Step 7 coverage audit reported 48% (22/46 branches). Added 10 cases
covering the highest-impact gaps:

Helper (test/gstack-gbrain-source-wireup.test.ts, +3 cases → 19 total):
- --uninstall when gbrain is missing: best-effort exit 0, worktree still cleaned
- --no-pull skips HEAD advance on existing worktree (was untested)
- Stray non-git directory at worktree path is cleaned up + worktree created

Migration (test/gstack-upgrade-migration-v1_15_1_0.test.ts, NEW, 7 cases):
- HOME unset → defensive exit 0
- gbrain_sync_mode=off → exit 0 silently
- gbrain_sync_mode unset → exit 0 silently
- no ~/.gstack/.git → exit 0 silently
- helper missing on PATH → warning + exit 0
- happy path → invokes helper without --strict
- helper exits non-zero → migration prints retry hint, still exits 0 (non-blocking)

Also syncs package.json version from 1.15.0.0 → 1.15.1.0 to match VERSION
file (DRIFT_STALE_PKG repair from /ship Step 12 idempotency check; was a
manual-edit-bypass artifact from the merge step).

Coverage estimate: 48% → ~75%. Mainline + migration script + key defensive
paths all exercised. 26 tests total covering the new code surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: pre-landing review auto-fixes (5 correctness + observability)

/ship Step 9 review surfaced 9 INFORMATIONAL findings on the new helper +
migration. Five auto-fixed with no behavior regression (26/26 tests pass):

bin/gstack-gbrain-source-wireup:
- Version compare: put floor "0.18.0" first in `sort -V` stdin so equal-or-
greater $v always sorts to position 2. Stable across sort implementations.
- _worktree_add_detached: drop `2>/dev/null` on the `worktree add`, surface
git's stderr through `prefix` so users see WHY adds fail (disk, perms).
- ensure_worktree: same observability fix on the `git checkout --detach` path
during HEAD-advance, so users see the actual git error before recovery.
- do_probe: replace `[ -d X ] || [ -f X ] && set=present` (precedence trap —
the `&&` short-circuits when the dir branch fails) with explicit if-block.
- do_probe: capture `check_source_state`'s return code explicitly via
`set +e; ...; rc=$?; set -e`. `$?` after an `if`/`elif` chain is fragile
under set -e and may not reach the elif under some shell versions.
- do_wireup: same explicit return-code capture for `ensure_worktree`. The
prior `ensure_worktree || { if [ $? = 2 ]; ...` pattern relied on `$?`
reflecting the function's return after `||`, which is implementation-defined.

gstack-upgrade/migrations/v1.15.1.0.sh:
- Trim whitespace from `gstack-config get gbrain_sync_mode` output via
`tr -d '[:space:]'`. Trailing newlines would mis-classify "off\n" as a
non-empty non-off mode and incorrectly invoke the helper.

Skipped findings (cosmetic / out of scope):
- `python3 -c` reads `~/.gbrain/config.json` via `expanduser` instead of
the helper's `$GBRAIN_CONFIG` variable (cosmetic; HONORS HOME override).
- Long sync-failure error message could truncate to last N lines (cosmetic
log readability).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: adversarial review hardening (rm safety, jq probe, secret redaction, multi-Mac)

/ship Step 11 adversarial review surfaced 7 CRITICAL issues. Five fixed
inline (no behavior regression, 26/26 tests still pass):

bin/gstack-gbrain-source-wireup:

1. **rm -rf path validation** (was: F-c-CRITICAL 9/10).
Added `safe_rm_worktree` helper that refuses any path not strictly under
$HOME/, plus dangerous-path allowlist for /, /Users, $HOME root. Replaces
raw `rm -rf "$WORKTREE"` calls (lines 161, 169 originally). If user sets
GSTACK_BRAIN_WORKTREE="" or "/", the helper now dies cleanly instead of
nuking the home dir or root.

2. **jq dependency probe** (was: F-c-CRITICAL 9/10).
`check_source_state` now hard-fails with a clear message if jq is missing,
instead of silently returning "absent" → re-add → die-on-duplicate. Plus
trims whitespace from jq output (`tr -d '[:space:]'`) to defend against
gbrain emitting `\n` for missing fields. Header comment claimed jq was a
transitive dep; now we enforce it.

3. **Python heredoc warns on JSON parse failure** (was: F-c-CRITICAL 8/10).
Previously `except Exception: pass` silently swallowed malformed JSON,
leaving _locked_url empty and defeating the URL-lock defense. Now writes
the parse error to a temp file and warns the user that the URL was not
locked. Also passes the config path via env var (GBRAIN_CONFIG_PATH)
instead of hardcoded `~/.gbrain/config.json`, respecting any HOME override.

4. **Multi-Mac source-id collision fix** (was: F-c-CRITICAL 9/10).
When `check_source_state` returns 1 (source exists at different path), the
helper used to remove + re-add. Two Macs sharing one Supabase brain would
ping-pong the local_path metadata on every sync. Now: if the existing
path's basename matches the local worktree's basename (likely another
machine's local copy of the SAME brain repo), skip re-registration and
sync against the local worktree. gbrain stores pages by content; metadata
is informational. No more ping-pong.

5. **Redact DB URL from sync-failure error message** (was: F-c-CRITICAL 7/10).
`gbrain sync` failures used to echo the full stderr (which can contain
the postgres connection string with password) into the user's terminal
and any log redirect. Now we sed-replace any `postgres://...` with
`postgres://***REDACTED***` before the die() call, and only show the
last 10 lines.

Bonus minor fix: `die()` now uses `$1` instead of `$*` for the warn
message, so the exit-code arg ($2) doesn't get appended to the warning text.

Acknowledged-but-deferred:
- GBRAIN_DATABASE_URL env exposure on Linux via /proc/$PID/environ. This is
a Linux-only concern; gstack is Mac-targeted today and macOS restricts
process env reads. Document as a follow-up if Linux support lands.
- gbrain version parser brittleness if gbrain switches to "v0.18.0" prefix.
Defensive only; current gbrain output matches `gbrain X.Y.Z` exactly.
- bash 3.2 PIPESTATUS reliability. Tests pass on the host bash version (3.2+
via macOS); modern bash 5.x is widely available.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: sync gbrain-source-wireup helper into USING_GBRAIN + gbrain-sync

USING_GBRAIN_WITH_GSTACK.md: add gstack-gbrain-source-wireup row to the bin
helpers table — describes federation registration via `gbrain sources add` +
worktree, lists flags, calls out it replaces the dead consumers.json/ingest-repo
HTTP wireup.

docs/gbrain-sync.md: replace the `gstack-brain-reader add --ingest-url` step
in gstack-brain-init's flow (which targeted the never-shipped /ingest-repo
endpoint) with the real flow — federate via gbrain sources + worktree, point
to bin/gstack-gbrain-source-wireup.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* v1.16.1.0: rebump after queue-collision (PR #1233 took v1.16.0.0)

CI's "Check VERSION is not stale vs queue" job (job 73105686380) failed
with: "VERSION drift: PR #1234 claims v1.15.1.0 but the queue has moved —
next free slot is v1.16.1.0." PR #1233 (garrytan/browserharness) entered
the queue claiming v1.16.0.0 between when this branch's prior /ship ran
and when CI evaluated, so v1.15.1.0 is stale. Rebumping on top.

Files updated:
- VERSION 1.15.1.0 → 1.16.1.0
- package.json 1.15.1.0 → 1.16.1.0
- CHANGELOG.md heading + Before/After columns 1.15.1.0 → 1.16.1.0
- CHANGELOG removal target (consumers.json + config keys) 1.16.0.0 → 1.17.0.0
- gstack-upgrade/migrations/v1.15.1.0.sh → renamed v1.16.1.0.sh + log prefix
- bin/gstack-brain-consumer "DEPRECATED in" + "removal in" 1.15.1.0/1.16.0.0 → 1.16.1.0/1.17.0.0
- bin/gstack-brain-uninstall "since vX.Y.Z.W" 1.15.1.0 → 1.16.1.0
- test/gstack-upgrade-migration-v1_15_1_0.test.ts → renamed v1_16_1_0.test.ts

No behavior change. 26/26 wireup + migration tests still pass on the rename.
Full bun test suite: exit 0, 0 failures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v1.17.0.0: rebump again — bump-detection now classifies branch as MINOR

CI's version-stale check (job 73106360896) failed: PR #1234 claims v1.16.1.0
but the queue moved to v1.17.0.0. Root cause: bumping 1.15.1.0 → 1.16.1.0
to dodge the prior collision turned the branch's diff classification from
PATCH (1.15.0 → 1.15.1) into MINOR (1.15.0 → 1.16.x). detect-bump.ts now
sees MINOR, gstack-next-version walks the MINOR lane past #1233's
v1.16.0.0 claim, and the next free slot is v1.17.0.0.

Honestly accurate per CLAUDE.md scale-aware bumps: this branch IS a
MINOR ("substantial new capability shipped — skill, harness, command,
big refactor"). The new helper + migration + integration totals ~1200
lines added across 11 files with 26 new tests. PATCH was always the
wrong honest classification; the queue collision forced the right
answer.

Files updated:
- VERSION 1.16.1.0 → 1.17.0.0
- package.json 1.16.1.0 → 1.17.0.0
- CHANGELOG.md heading + After column 1.16.1.0 → 1.17.0.0
- CHANGELOG removal targets 1.17.0.0 → 1.18.0.0
- gstack-upgrade/migrations/v1.16.1.0.sh → renamed v1.17.0.0.sh + log prefix
- bin/gstack-brain-consumer "DEPRECATED in" + "removal in" 1.16.1.0/1.17.0.0 → 1.17.0.0/1.18.0.0
- bin/gstack-brain-uninstall "since vX.Y.Z.W" 1.16.1.0 → 1.17.0.0
- test/gstack-upgrade-migration-v1_16_1_0.test.ts → renamed v1_17_0_0.test.ts

26/26 tests still pass. No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-04-28 01:17:54 -07:00

6.7 KiB

Raw Blame History

Cross-machine memory with GBrain sync

gstack writes a lot of useful state to ~/.gstack/ — learnings, retros, CEO plans, design docs, developer profile. By default, all of that dies when you switch laptops. GBrain sync pushes a curated subset to a private git repo so your memory follows you across machines and becomes indexable by GBrain.

What you get

Work on machine A, pick up seamlessly on machine B.
Your learnings, plans, and designs are visible in GBrain (if you use it).
A clean off-ramp (gstack-brain-uninstall) that never touches your data.
No daemon, no system service, no background process.

What does NOT leave your machine

By design, these stay local even when sync is on:

Credentials: .auth.json, auth-token.json, sidebar-sessions/, security/device-salt, consumer tokens in config.yaml
Machine-specific state: Chromium profiles, ONNX model weights, caches, eval-cache, CDP-profile, one-time prompt markers (.welcome-seen, .telemetry-prompted, .vendoring-warned-*, etc.)
Question-preferences: per-machine UX preferences (question-preferences.json, question-log.jsonl, question-events.jsonl).

The exact allowlist lives in ~/.gstack/.brain-allowlist. The CLI manages it; you can append your own entries below the marker line.

First-run setup (30–90 seconds)

gstack-brain-init

The command:

Turns ~/.gstack/ into a git repo.
Asks for a remote URL (default: gh repo create --private gstack-brain-$USER). Any git remote works — GitHub, GitLab, Gitea, self-hosted.
Pushes an initial commit with just the config.
Writes ~/.gstack-brain-remote.txt (URL-only, no secrets — safe to copy to another machine).
Wires the gstack-brain repo into your local gbrain as a federated source (via gbrain sources add + git worktree) so gbrain search can index your synced learnings, plans, and designs. Implementation lives in bin/gstack-gbrain-source-wireup. The old gstack-brain-reader add --ingest-url ... HTTP path was removed in v1.15.1.0 — it depended on a /ingest-repo endpoint gbrain never shipped.

After init, the next skill you run will ask you ONE question about privacy mode:

Everything allowlisted (recommended): learnings, reviews, plans, designs, retros, timelines, and developer profile all sync.
Only artifacts: plans, designs, retros, learnings — skip behavioral data (timelines, developer profile).
Decline: keep everything local. You can turn sync on later with gstack-config set gbrain_sync_mode full.

Your answer is persisted. You won't be asked again.

Cross-machine workflow

On machine A: run gstack-brain-init once. That's it — every skill invocation now drains the sync queue at its start and end boundaries (~200–800 ms network pause per skill).

On machine B:

Copy ~/.gstack-brain-remote.txt from machine A to machine B (password manager, dotfile repo, USB stick — your call).

Run any gstack skill. The preamble sees the URL file and prints:

BRAIN_SYNC: brain repo detected: <url>
BRAIN_SYNC: run 'gstack-brain-restore' to pull your cross-machine memory

Run gstack-brain-restore. That clones the repo, rehydrates your learnings/plans/retros, and re-registers the git merge drivers.
Re-enter consumer tokens (they're machine-local and NOT synced — gstack-config set gbrain_token <your-token>).
Next skill: your yesterday-on-machine-A learning surfaces. That's the magical moment.

Status, health, and queue depth

gstack-brain-sync --status

Shows: last successful push, pending queue depth, any sync blocks, and the current privacy mode.

Every skill run prints a BRAIN_SYNC: line near the top of the preamble output. Scan it for problems.

Privacy modes in detail

Mode	What syncs
`off`	Nothing (default).
`artifacts-only`	Plans, designs, retros, learnings, reviews. Skips timelines + developer-profile.
`full`	Everything in the allowlist, including behavioral state.

Change anytime with:

gstack-config set gbrain_sync_mode full
gstack-config set gbrain_sync_mode off

Secret protection

Every commit is scanned for credential-shaped content before it leaves your machine. Blocked patterns include:

AWS access keys (AKIA…)
GitHub tokens (ghp_, gho_, ghu_, ghs_, ghr_, github_pat_)
OpenAI keys (sk-…)
PEM blocks (-----BEGIN …-----)
JWTs (eyJ…)
Bearer tokens in JSON ("authorization": "…", "api_key": "…", etc.)

If a scan hits, sync stops, the queue is preserved, and your preamble prints:

BRAIN_SYNC: blocked: <pattern-family>:<snippet>

To remediate:

Review the offending file.
If the match is a false positive on content you explicitly want to sync, run gstack-brain-sync --skip-file <path> to permanently exclude that path.
Otherwise, edit the file to remove the secret and re-run any skill.

There's a defense-in-depth hook at ~/.gstack/.git/hooks/pre-commit that runs the same scan if you manually git commit against the repo.

Two-machine conflicts

If you write on machine A and machine B the same day, both will push append commits. Git's default would conflict at the file tail, but the .jsonl and markdown files are registered with custom merge drivers:

JSONL files use a sort-and-dedup driver that orders appends by ISO timestamp (falls back to SHA-256 hash of each line for determinism).
Markdown artifacts (retros, plans, designs) use a union merge driver that concatenates both sides.

You shouldn't see conflict prompts. If you do (a real semantic conflict, like two machines editing the same plan), git will stop and prompt.

Cross-machine pull cadence

The preamble runs git fetch + git merge --ff-only once per 24 hours (cached via ~/.gstack/.brain-last-pull). You don't need to think about this — it happens automatically at the first skill invocation each day.

Uninstall

gstack-brain-uninstall

This:

Removes ~/.gstack/.git/ and all .brain-* config files.
Clears gbrain_sync_mode in gstack-config.
Does NOT touch your learnings, plans, retros, or developer profile.

Add --delete-remote to also delete the private GitHub repo (GitHub only, uses gh repo delete).

Re-init anytime with gstack-brain-init.

Troubleshooting

See gbrain-sync-errors.md for an index of every error message gstack-brain may print, with problem / cause / fix for each.

Under the hood

For the architectural decisions behind this feature (allowlist vs denylist, daemon vs preamble-boundary sync, JSONL merge driver, privacy stop-gate), see the approved plan in the gstack plans directory.

6.7 KiB Raw Blame History Unescape Escape

Cross-machine memory with GBrain sync

What you get

What does NOT leave your machine

First-run setup (30–90 seconds)

Cross-machine workflow

Status, health, and queue depth

Privacy modes in detail

Secret protection

Two-machine conflicts

Cross-machine pull cadence

Uninstall

Troubleshooting

Under the hood

6.7 KiB

Raw Blame History