diff --git a/README.md b/README.md index 05001dce..1d63004d 100644 --- a/README.md +++ b/README.md @@ -359,12 +359,42 @@ I open sourced how I build software. You can fork it and make it your own. > Come work at YC — [ycombinator.com/software](https://ycombinator.com/software) > Extremely competitive salary and equity. San Francisco, Dogpatch District. +## Cross-machine memory with GBrain sync + +gstack accumulates a lot of useful state on your laptop: learnings, CEO +plans, design docs, retros, developer profile. Today, all of that dies when +you switch machines. **GBrain sync** optionally pushes a curated, secret-scanned +subset to a private git repo so your memory follows you, and (if you use +GBrain) becomes indexable there. + +One command to turn it on: + +```bash +gstack-brain-init +``` + +That creates a private GitHub repo (or any git remote you prefer — +GitLab, Gitea, self-hosted). Every skill run syncs the queue at its +start and end boundaries. No daemon, no background process. A one-time +privacy prompt asks how much you want to share (everything allowlisted / +artifacts only / off). Secret-shaped content (AWS keys, GitHub tokens, +PEM blocks, JWTs, etc.) is blocked from sync before it leaves your +machine. + +New machine? Copy `~/.gstack-brain-remote.txt` over, run +`gstack-brain-restore`, and yesterday's learnings surface on today's +laptop. + +Full guide: [docs/gbrain-sync.md](docs/gbrain-sync.md) • +Error index: [docs/gbrain-sync-errors.md](docs/gbrain-sync-errors.md) + ## Docs | Doc | What it covers | |-----|---------------| | [Skill Deep Dives](docs/skills.md) | Philosophy, examples, and workflow for every skill (includes Greptile integration) | | [Builder Ethos](ETHOS.md) | Builder philosophy: Boil the Lake, Search Before Building, three layers of knowledge | +| [GBrain Sync](docs/gbrain-sync.md) | Cross-machine memory setup, privacy modes, troubleshooting | | [Architecture](ARCHITECTURE.md) | Design decisions and system internals | | [Browser Reference](BROWSER.md) | Full command reference for `/browse` | | [Contributing](CONTRIBUTING.md) | Dev setup, testing, contributor mode, and dev mode | diff --git a/docs/gbrain-sync-errors.md b/docs/gbrain-sync-errors.md new file mode 100644 index 00000000..52120a8b --- /dev/null +++ b/docs/gbrain-sync-errors.md @@ -0,0 +1,214 @@ +# gbrain-sync error lookup + +Every error message `gstack-brain-*` can print, with problem, cause, and fix. + +Search this file by the prefix after `BRAIN_SYNC:` or by the binary name in +the command output. + +--- + +## `BRAIN_SYNC: brain repo detected: ` + +**Problem.** You're on a machine that has `~/.gstack-brain-remote.txt` (copied +from another machine) but no local git repo at `~/.gstack/.git`. + +**Cause.** You've set up GBrain sync elsewhere and your gstack hasn't been +restored on this machine yet. + +**Fix.** +```bash +gstack-brain-restore +``` +This pulls the repo into `~/.gstack/` and re-registers merge drivers. + +If you don't want to restore here, dismiss the hint with: +```bash +gstack-config set gbrain_sync_mode_prompted true +``` + +--- + +## `BRAIN_SYNC: blocked: :` + +**Problem.** Sync stopped because the secret scanner detected credential-shaped +content in a staged file. The queue is preserved; nothing was pushed. + +**Cause.** One of the pre-commit secret patterns matched the file contents — +likely an AWS key, GitHub token, OpenAI key, PEM block, JWT, or bearer token +embedded in JSON. + +**Fix (three options).** + +1. **If it's a real secret**: edit the offending file to remove the secret, + then re-run any skill to retry sync. + +2. **If the pattern is a false positive** (e.g., your learning contains a + GitHub token pattern in an example string that you *want* to publish): + ```bash + gstack-brain-sync --skip-file + ``` + This permanently excludes the path from future syncs. + +3. **If you want to abandon this sync batch entirely** (start fresh): + ```bash + gstack-brain-sync --drop-queue --yes + ``` + This clears the queue without committing. Future writes will re-populate + it normally. + +--- + +## `BRAIN_SYNC: push failed: auth.` + +**Problem.** Git push was rejected because your auth with the remote expired +or is missing. + +**Cause.** The remote is unreachable with current credentials. + +**Fix.** Refresh auth based on your remote: + +- **GitHub**: `gh auth status` (then `gh auth refresh` if needed) +- **GitLab**: `glab auth status` +- **Other**: `git remote -v` + check SSH keys or credential helper + +After fixing auth, run any skill to retry sync automatically. + +--- + +## `BRAIN_SYNC: push failed: ` + +**Problem.** Push failed for a reason other than auth. The first line of +git's error appears after the colon. + +**Cause.** Could be network issue, rejected push (remote ahead), server 500, +or repo access revoked. + +**Fix.** Look at `~/.gstack/.brain-sync-status.json` for more detail, or run: +```bash +cd ~/.gstack && git status && git push origin HEAD +``` +to see git's full error. The queue is cleared after any push attempt, but +your local commit still exists — the next skill run will retry the push. + +--- + +## `gstack-brain-init: ~/.gstack/.git is already a git repo pointing at ` + +**Problem.** You tried to init with a remote URL that doesn't match the +existing one. + +**Cause.** You already ran `gstack-brain-init` with a different remote. + +**Fix.** Either: + +- Use the existing remote: run `gstack-brain-init` without `--remote`, or + with the matching URL. +- Switch remotes: `gstack-brain-uninstall` first, then re-init with the new + URL. This does not delete your data. + +--- + +## `Remote not reachable: ` + +**Problem.** Init couldn't reach the git remote to verify connectivity. + +**Cause.** Wrong URL, missing auth, network issue. + +**Fix.** Test manually: +```bash +git ls-remote +``` +If that fails, check: +- URL spelling +- GitHub: `gh auth status` +- GitLab: `glab auth status` +- Private network / VPN / DNS + +--- + +## `gstack-brain-init: failed to create or find ''` + +**Problem.** Auto-repo-creation via `gh repo create` failed and the repo +isn't discoverable via `gh repo view` either. + +**Cause.** `gh` is unauthenticated, a repo with that name already exists +owned by someone else, or your GitHub account hit a quota. + +**Fix.** +```bash +gh auth status +``` +If unauth'd, run `gh auth login`. If the repo name collides, pass a different +name: +```bash +gstack-brain-init --remote git@github.com:YOURUSER/custom-name.git +``` + +--- + +## `gstack-brain-restore: ~/.gstack/.git already points at ` + +**Problem.** You tried to restore from a URL that doesn't match the existing +git config. + +**Cause.** Stale `.git` from a previous init with a different remote. + +**Fix.** `gstack-brain-uninstall`, then re-run `gstack-brain-restore `. + +--- + +## `gstack-brain-restore: ~/.gstack/ has existing allowlisted files that would be clobbered` + +**Problem.** You're trying to restore, but `~/.gstack/` already contains +learnings or plans that would be overwritten. + +**Cause.** Either (a) this machine has accumulated state from a pre-sync +gstack session, or (b) a previous failed restore left partial state. + +**Fix (three options).** + +1. **If this machine's state should become the new truth**: run + `gstack-brain-init` instead of restore — this creates a brand-new brain + repo from this machine's state. + +2. **If you want to adopt the remote and discard this machine's state**: + back up `~/.gstack/projects/` first, then remove the offending files and + re-run restore. + +3. **If you want to merge**: there's no automatic merge for this. Manually + copy learnings from `~/.gstack/` into your running gstack on a machine + with sync already on, then restore here. + +--- + +## `gstack-brain-restore: does not look like a gstack-brain repo` + +**Problem.** The clone succeeded but the repo is missing `.brain-allowlist` +and `.gitattributes`. + +**Cause.** You pointed restore at a random git repo, or someone deleted the +canonical config files from the brain repo. + +**Fix.** Verify the URL. If it's correct, run `gstack-brain-init --remote +` to re-seed the canonical config. + +--- + +## Nothing is syncing but I expect it to + +**Not an error, but a common gotcha.** Check in order: + +1. `gstack-brain-sync --status` — is mode `off`? +2. `~/.gstack/.git` exists? +3. `gstack-config get gbrain_sync_mode` — should be `full` or `artifacts-only`. +4. The file you expect to sync — is it in the allowlist? + `cat ~/.gstack/.brain-allowlist` +5. Privacy class filter — if mode is `artifacts-only`, behavioral files + (timelines, developer-profile) are intentionally skipped. + +If all those look right, run: +```bash +gstack-brain-sync --discover-new +gstack-brain-sync --once +``` +to force a drain. diff --git a/docs/gbrain-sync.md b/docs/gbrain-sync.md new file mode 100644 index 00000000..02e9dd4c --- /dev/null +++ b/docs/gbrain-sync.md @@ -0,0 +1,188 @@ +# Cross-machine memory with GBrain sync + +gstack writes a lot of useful state to `~/.gstack/` — learnings, retros, CEO +plans, design docs, developer profile. By default, all of that dies when you +switch laptops. **GBrain sync** pushes a curated subset to a private git +repo so your memory follows you across machines and becomes indexable by +GBrain. + +## What you get + +- Work on machine A, pick up seamlessly on machine B. +- Your learnings, plans, and designs are visible in GBrain (if you use it). +- A clean off-ramp (`gstack-brain-uninstall`) that never touches your data. +- No daemon, no system service, no background process. + +## What does NOT leave your machine + +By design, these stay local even when sync is on: + +- Credentials: `.auth.json`, `auth-token.json`, `sidebar-sessions/`, + `security/device-salt`, consumer tokens in `config.yaml` +- Machine-specific state: Chromium profiles, ONNX model weights, + caches, eval-cache, CDP-profile, one-time prompt markers + (`.welcome-seen`, `.telemetry-prompted`, `.vendoring-warned-*`, etc.) +- Question-preferences: per-machine UX preferences + (`question-preferences.json`, `question-log.jsonl`, `question-events.jsonl`). + +The exact allowlist lives in `~/.gstack/.brain-allowlist`. The CLI manages +it; you can append your own entries below the marker line. + +## First-run setup (30–90 seconds) + +```bash +gstack-brain-init +``` + +The command: + +1. Turns `~/.gstack/` into a git repo. +2. Asks for a remote URL (default: `gh repo create --private + gstack-brain-$USER`). Any git remote works — GitHub, GitLab, Gitea, + self-hosted. +3. Pushes an initial commit with just the config. +4. Writes `~/.gstack-brain-remote.txt` (URL-only, no secrets — + safe to copy to another machine). +5. Registers GBrain as a reader if `GBRAIN_URL` + `GBRAIN_TOKEN` are + configured. Otherwise you can add readers later with + `gstack-brain-reader add --ingest-url --token `. + +After init, the **next skill you run** will ask you ONE question about +privacy mode: + +- **Everything allowlisted (recommended)**: learnings, reviews, plans, + designs, retros, timelines, and developer profile all sync. +- **Only artifacts**: plans, designs, retros, learnings — skip + behavioral data (timelines, developer profile). +- **Decline**: keep everything local. You can turn sync on later with + `gstack-config set gbrain_sync_mode full`. + +Your answer is persisted. You won't be asked again. + +## Cross-machine workflow + +On machine A: run `gstack-brain-init` once. That's it — every skill +invocation now drains the sync queue at its start and end boundaries +(~200–800 ms network pause per skill). + +On machine B: + +1. Copy `~/.gstack-brain-remote.txt` from machine A to machine B + (password manager, dotfile repo, USB stick — your call). +2. Run any gstack skill. The preamble sees the URL file and prints: + ``` + BRAIN_SYNC: brain repo detected: + BRAIN_SYNC: run 'gstack-brain-restore' to pull your cross-machine memory + ``` +3. Run `gstack-brain-restore`. That clones the repo, rehydrates your + learnings/plans/retros, and re-registers the git merge drivers. +4. Re-enter consumer tokens (they're machine-local and NOT synced — + `gstack-config set gbrain_token `). +5. Next skill: your yesterday-on-machine-A learning surfaces. That's the + magical moment. + +## Status, health, and queue depth + +```bash +gstack-brain-sync --status +``` + +Shows: last successful push, pending queue depth, any sync blocks, and the +current privacy mode. + +Every skill run prints a `BRAIN_SYNC:` line near the top of the preamble +output. Scan it for problems. + +## Privacy modes in detail + +| Mode | What syncs | +|------|------------| +| `off` | Nothing (default). | +| `artifacts-only` | Plans, designs, retros, learnings, reviews. Skips timelines + developer-profile. | +| `full` | Everything in the allowlist, including behavioral state. | + +Change anytime with: +```bash +gstack-config set gbrain_sync_mode full +gstack-config set gbrain_sync_mode off +``` + +## Secret protection + +Every commit is scanned for credential-shaped content before it leaves +your machine. Blocked patterns include: + +- AWS access keys (`AKIA…`) +- GitHub tokens (`ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_`, `github_pat_`) +- OpenAI keys (`sk-…`) +- PEM blocks (`-----BEGIN …-----`) +- JWTs (`eyJ…`) +- Bearer tokens in JSON (`"authorization": "…"`, `"api_key": "…"`, etc.) + +If a scan hits, sync stops, the queue is preserved, and your preamble +prints: + +``` +BRAIN_SYNC: blocked: : +``` + +To remediate: + +1. Review the offending file. +2. If the match is a false positive on content you explicitly want to + sync, run `gstack-brain-sync --skip-file ` to permanently + exclude that path. +3. Otherwise, edit the file to remove the secret and re-run any skill. + +There's a defense-in-depth hook at `~/.gstack/.git/hooks/pre-commit` that +runs the same scan if you manually `git commit` against the repo. + +## Two-machine conflicts + +If you write on machine A and machine B the same day, both will push +append commits. Git's default would conflict at the file tail, but the +`.jsonl` and markdown files are registered with custom merge drivers: + +- JSONL files use a sort-and-dedup driver that orders appends by ISO + timestamp (falls back to SHA-256 hash of each line for determinism). +- Markdown artifacts (retros, plans, designs) use a union merge driver + that concatenates both sides. + +You shouldn't see conflict prompts. If you do (a real semantic conflict, +like two machines editing the same plan), git will stop and prompt. + +## Cross-machine pull cadence + +The preamble runs `git fetch` + `git merge --ff-only` once per 24 hours +(cached via `~/.gstack/.brain-last-pull`). You don't need to think about +this — it happens automatically at the first skill invocation each day. + +## Uninstall + +```bash +gstack-brain-uninstall +``` + +This: + +- Removes `~/.gstack/.git/` and all `.brain-*` config files. +- Clears `gbrain_sync_mode` in `gstack-config`. +- Does NOT touch your learnings, plans, retros, or developer profile. + +Add `--delete-remote` to also delete the private GitHub repo (GitHub only, +uses `gh repo delete`). + +Re-init anytime with `gstack-brain-init`. + +## Troubleshooting + +See [gbrain-sync-errors.md](gbrain-sync-errors.md) for an index of every +error message gstack-brain may print, with problem / cause / fix for each. + +## Under the hood + +For the architectural decisions behind this feature (allowlist vs +denylist, daemon vs preamble-boundary sync, JSONL merge driver, privacy +stop-gate), see the +[approved plan](../system-instruction-you-are-working-jaunty-kahn.md) in +the gstack plans directory.