mirror of
https://github.com/garrytan/gstack.git
synced 2026-08-23 14:32:33 +02:00
* fix(plan-tune): reject never-ask on one-way ids at --write --check already ignored those prefs; --write still stored them and --stats counted them as a working NEVER_ASK. Refuse the write and count leftover on-disk prefs as INERT_ONE_WAY. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix: gstack-config get returns "" with exit 0 for keys that have no default Skill preambles read configuration with VAR=$(gstack-config get <key> 2>/dev/null || echo "<default>") and that fallback only fires on a non-zero exit. lookup_default ended in a catch-all that echoed "" and returned 0, so for any key missing from the table VAR came back empty and the default written right there in the preamble was unreachable. The skill then branched on a value it never specified: "skip entirely if QUESTION_TUNING is false", reached with QUESTION_TUNING="". Four keys that skills actually read had no entry and took that path: question_tuning -> callers assume "false" repo_mode -> callers assume "unknown" team_mode -> callers assume "false" transcript_ingest_mode -> callers assume "off" Each default above is the value the call sites already substitute in their own `|| echo` fallback, so this only makes reachable what was already intended. The catch-all now returns non-zero. That is deliberately scoped to the unknown-key arm alone: keys whose default is intentionally empty still exit 0, because "" is their real answer and their callers depend on it -- cross_project_learnings ("unset triggers the first-time prompt"), redact_repo_visibility ("empty falls through to gh/glab detection"), salience_allowlist, user_slug_at_*. Making every empty answer an error would have broken those. test/gstack-config-defaults.test.ts pins the class rather than the four instances: it parses the case arms and asserts every `gstack-config get <key>` site in the tree is covered, so adding a read without a default fails CI. It also pins the exit-code contract in both directions. Verified failing against the pre-fix script, where it names exactly those four keys. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(redact): a typo'd subcommand no longer exits 0 having done nothing main() recognised exactly two subcommands and let everything else fall through to the stdin scan. On empty stdin that prints "(no findings)" and exits 0, so: $ gstack-redact install-prepush-hooks # plural typo gstack-redact scan — repo UNKNOWN (no findings) $ echo $? 0 No hook was installed, and the operator has every reason to believe the credential guard is armed. A guard that silently no-ops must never exit 0. Two smaller faults in the same dispatch, both of which lead people here: - There was no --help handler, so `gstack-redact --help` fell through to the scanner. Piping a credential to it scanned the secret and exited 3. - With no piped input and no --from-file, readInput() blocks on readSync(fd 0) until an EOF that an interactive terminal never sends. That prints nothing at all, so it reads as a hang rather than as "this is a filter, feed it". Now: --help/-h/help prints usage and exits 0; an unrecognised positional prints the offender and exits 1; a TTY with nothing piped in prints usage instead of blocking. "scan" stays accepted, because the human output header reads "gstack-redact scan — repo …" and that is what people type. Usage errors exit 1, deliberately not 2 or 3. Those mean MEDIUM and HIGH findings and callers gate dispatch on them, so a usage error exiting 2 would be read as "medium findings — prompt the user". A test pins that. Tests: 4 written failing first, then fixed. Full suite 7,722 pass / 0 fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(browse): one ambiguous ref no longer kills the whole annotated screenshot `snapshot -a` exits 1 with "Selector matched multiple elements" on most real pages, so /qa, /canary and /land-and-deploy silently produce reports whose screenshots do not exist. Plain `screenshot <path>` is unaffected. Refs are built as getByRole(role, {name}) and disambiguated with .nth() when role+name repeats. That disambiguation cannot fire for a node with NO accessible name: the locator degrades to getByRole(role) with no name filter, and the count driving .nth() is taken from the FILTERED aria snapshot while getByRole matches the unfiltered DOM. Measured on a live page: the tree surfaced 2 unnamed paragraphs, the DOM had 9. Landmarks (banner/main/contentinfo) and paragraphs are correctly unnamed per ARIA, so this is the common case rather than an edge case. boundingBox() then hits Playwright strict mode, and the catch allowlisted only timeout/closed/Target/Execution-context messages — so the strict-mode error was re-thrown and aborted every remaining annotation. Two changes: - `.first()` before boundingBox(), so an ambiguous ref draws a box on its first match instead of aborting. The heatmap path below has always tolerated this via a bare `catch {}`; annotate was the only path that could be killed outright. - the catch no longer re-throws on unrecognised messages. A box we cannot measure is a box we do not draw, never a reason to lose the rest of the page. Set BROWSE_DEBUG to see what was skipped. Also: `-o` passed without `-a`/`-H` was silently ignored (exit 0, no file), which reads as "screenshots are broken" rather than "you forgot a flag". It now warns and points at `browse screenshot <path>`. Verified by rebuilding both ways against the same page with 51 refs present: before — "Selector matched multiple elements", no file written after — exit 0, 229KB PNG Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(version-bump): missing or empty VERSION no longer repairs a fabricated 0.0.0.0 into package.json repair now fails with exit 2 when the VERSION file is absent or empty instead of folding to DEFAULT ("0.0.0.0") — which passed VERSION_RE and regressed package.json below where it started. classify gains an additive versionFileExists field so /ship can tell a real 0.0.0.0 from a fabricated one. Re-derived from PR #2612 under the generated-file screening rule. Fixes #2600 (repair half; the path-configurability half landed in v1.67 via #2531). Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(memory-ingest): --probe counts post-attribution, through the same gate --bulk uses probeMode previously stat'd every walked file, so setup-gbrain gated its silent bulk ingest on pre-filter counts that the write path would never ingest (#2394). The attribution decision now lives in ONE shared gate (sessionIsAttributable — cheap-parse: cwd extraction + memoized resolveGitRemote, never a full page build) used by BOTH probeMode and preparePages, so the two stages' post-attribution counts are structurally identical. ProbeReport gains skipped_unattributed; the probe prints what it excluded and --include-unattributed restores raw counts. The parity is pinned at the prepare stage (probe post-attribution == transcripts reaching import), deliberately NOT == final written. Re-derived from PR #2612 under the generated-file screening rule; the shared-gate design and the remote memo are additions from the plan review. Fixes #2394. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(browse): allow CPU and network throttling for performance measurement Adds Emulation.setCPUThrottlingRate and Network.emulateNetworkConditions to CDP_ALLOWLIST. Motivation: diagnosing a real "uploads take 1-2 minutes" report, the only machine available was a fast developer workstation. Client-side processing measured 1.4s where the user experienced minutes, so the conclusion had to be reached arithmetically rather than observed. Throttling would have let the measurement reproduce the reporter's conditions directly. Both fit the existing posture rather than widening it: - Emulation already allows setDeviceMetricsOverride, clearDeviceMetricsOverride and setUserAgentOverride, which are equally mutating and scoped to the tab. - Neither method reads page content. setCPUThrottlingRate affects only timing; emulateNetworkConditions constrains traffic rather than inspecting it, so no request bodies, headers or cookies are exposed. Both are output: 'trusted' because they return no page-derived data. scope 'tab' for both, matching the surrounding Emulation entries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(session-update): lock pidfile records the live holder; hard TTL bounds every wedge (#2613) echo $$ inside the backgrounded subshell recorded the PARENT hook's PID — which exits immediately — so every subsequent session judged the lock stale and rm -rf'd a LIVE holder's lock, letting concurrent updaters run over each other. The pidfile now records ${BASHPID:-$(sh -c 'echo $PPID')} (macOS bash 3.2 has no BASHPID; the sh child's PPID is exactly this subshell). Staleness is now two independent detectors: PID liveness (as before, but against the real holder), and a 30-minute hard TTL on the heartbeat mtime — reclaimed regardless of kill -0, so a recycled PID or hung holder can't wedge the lock forever. The holder touches the pidfile after the pull and after setup, so a legitimately-slow run keeps itself alive. Empty and missing pidfiles are respected inside the TTL window (the mkdir→echo race) and reclaimed past it. Fixes #2613. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(browse): explicit windowsHide on every Bun.spawn site + census tripwire (#2575 residual) Bun.spawn sites were structurally outside the windowsHide census (it swept child_process bindings only). The runtime was already safe — native Bun hides consoles by default and bun-polyfill.cjs defaults windowsHide !== false since #2523/#2539 — but implicit defaults are exactly what regress silently. Every Bun.spawn/spawnSync in browse/src now carries the explicit flag (harmless on unix-only sites like Xvfb/xattr/open), and a second SWEEP in windows-spawn-hide.test.ts fails CI on any new flagless Bun.spawn site. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gbrain): brain worktree advances on the daily sync — no more silently stale brains (#2516) The daily pull refreshed only ~/.gstack itself, never the detached worktree at ~/.gstack-brain-worktree that gbrain actually indexes — so after setup the brain served stale pages forever unless setup-gbrain/sync-gbrain happened to run. brain-sync --once now advances the worktree once per 24h behind an ATTEMPT stamp (.brain-worktree-last-advance — a persistently-failing advance warns once a day, not at every skill boundary), inside the existing run lock and before any ingest step touches the worktree. The new gstack-gbrain-source-wireup --advance-only is built for the unattended cadence: git-only (no gbrain prereqs), pins every operation to the managed worktree (refuses paths that are not worktrees of the artifacts repo), refuses dirty worktrees, and never runs the force-remove recovery — a cron path must not be able to delete local changes. A static pin keeps the force-remove out. docs/gbrain-sync.md stops overclaiming the old cadence. Fixes #2516. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(memory-ingest): honor the per-remote deny/read-only trust policy (#2392) Transcript ingest now respects the same trust store as code import — the gate existed only in gstack-gbrain-sync's runCodeImport, so memory-ingest happily ingested transcripts from deny-listed repos. preparePages filters prepared transcript pages through ONE batch policy lookup (new 'get --batch' verb on bin/gstack-gbrain-repo-policy — the script owns URL normalization; the client adds repoPolicyTierBatch, one spawn for all distinct remotes, so large corpora never pay a 10s-timeout subprocess per remote). Outcomes match code-import semantics: read-only → clean skip (skipped_policy_readonly), deny → counted refusal (skipped_policy_deny), corrupted/unreadable store → HARD ERROR before any write (state, staging, egress receipt, and import all untouched) with the recovery command named — policy corruption must never read as successful ingestion. Artifacts are never policy-filtered (their git_remote is a project slug, not a remote). Fixes #2392. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(config): repo_mode keeps its empty no-default semantics (#2611 follow-up) The ported defaults table synthesized repo_mode → "unknown", but EMPTY is load-bearing for that key: gstack-repo-mode treats any non-empty answer as a user override and skips its own repo classification — the synthesized default turned the classifier into dead code (REPO_MODE=unknown everywhere; caught by test/gstack-repo-mode.test.ts via the wave's cross-agent blame protocol). repo_mode joins the empty-is-real carve-outs (empty output, exit 0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pair-agent): consent before killing a healthy headless daemon The pair-agent headed switch spawned 'connect --force-restart' unconditionally — auto-killing a live headless daemon (open tabs, cookies, logins) in direct contradiction of the iron rule it sits beside ('only an explicit --force-restart may kill a live daemon'). The CLI now captures daemon liveness BEFORE ensureServer (which can itself boot a fresh daemon) and relaunches only when the user passed --force-restart to pair-agent; otherwise it prints the tab count and continues against the existing daemon. The /pair-agent skill gains a matching one-way-door consent question (template half rides the wave's template block). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gbrain-status): MCP scoping is per-project, and project-local beats user scope hasRemoteOnlyGbrainMcp scanned EVERY project's mcpServers in ~/.claude.json, so one project's remote gbrain registration reclassified broken local engines as thin-client machine-wide. It now reads user scope plus only the cwd's nearest-ancestor project key. The precedence itself was verified empirically and hermetically (fake HOME + CLAUDE_CONFIG_DIR fixtures, claude 2.1.233): with both scopes defining gbrain, 'claude mcp get gbrain' reports Scope: Local config — PROJECT-LOCAL WINS. Both in-repo consumers assumed the opposite; brain-cache's endpoint resolution flips to nearest-ancestor-project-first, and the stale user-first pin in brain-cache-roundtrip now pins the verified precedence. (The user-first jq in the brain-sync preamble resolver gets the same swap in the template block.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(slug): gstack-slug matches remote-slug's owner-repo canonical form (live misfile bug) Found live during this wave's CEO review: bin/gstack-slug emitted SLUG=garrytan for this garrytan/gstack worktree while remote-slug correctly gave garrytan-gstack — decisions, timeline, ceo-plans, and learnings were filing into the wrong project store (observed polluting Context Recovery with another repo's decisions). Root cause: a stray empty ~/.git directory made the walk-up crown $HOME as the outermost project root; the remote lookup ran only against that root, failed silently, and the basename fallback cached 'garrytan' sticky. NOT worktree-specific — any strong marker on a non-repo ancestor triggered it. Fix: the walk now finds the outermost ancestor whose .git actually resolves an origin remote and derives owner-repo with remote-slug's byte-identical parse; marker-only ancestors keep anchoring the basename fallback but can no longer shadow a real remote. A new cache self-heal recomputes the poisoned shape (cached == basename of a marker root while a remote-bearing repo exists below), preserving legit #2212 stickiness. Nested-repo walk-up, no-remote and non-git fallbacks, and the SLUG=/BRANCH= eval contract are unchanged, pinned by a 10-case parity suite. Store migration for pre-fix data is tracked in TODOS.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(brain-sync): per-record spool dir — the enqueue/drain race dies structurally Producers appended lines to .brain-queue.jsonl while the drain re-read and os.replace'd it; the in-code comment admitted a lockless append between the re-read and the replace was lost. Locks and rename-rotation designs were both reviewed and rejected (each retained a tail race); the shipped design is a maildir-style spool: one FILE per record in .brain-queue.d/ (tmp + atomic rename), the drain snapshots filenames, processes, and deletes exactly what it snapshotted. Writer and drainer never share an inode — nothing to race. Semantics: at-least-once (a crash between process and unlink re-drains; downstream content-hash dedup absorbs duplicates); retained (privacy-held) records keep their files; unparseable records are kept + warned, never destroyed. Legacy .brain-queue.jsonl migrates atomically on the next drain (crash-leftover .migrating files recovered too); status/drop-queue count both surfaces; discover-new writes spool records and advances its cursor per-record-written. The preamble's queue-depth line switches to spool count in this wave's template block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bin-context): native slug fallback walks up like bash gstack-slug slugFromEnvironment derived the slug from the INNERMOST repo's origin while bash gstack-slug walks to the outermost project root — nested/vendored repos split their stores across the bash/native boundary (win32 hits the native path constantly). The native fallback now ports _outermost_project_root faithfully (strong/weak markers, outermost-strong-wins, 64-depth cap, fixed-point termination) plus the full resolution order: env override → walk-up → sticky cache with the #1125 self-heal → remote get-url → basename. Twelve mirrored scenarios drive BOTH implementations against the same fixtures and pin identical slugs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(next-version): git fallback queries the live remote, never mutates, and keeps 3-digit width The degraded path counted every remote-tracking ref on every remote — stale experiment branches and second remotes inflated version allocation, and a failed base read flipped 3-digit repos to 4-digit slots. Now: ls-remote --heads origin first (GIT_TERMINAL_PROMPT=0, 5s timeout, zero local ref mutation); on failure, local refs/remotes/origin ONLY with an explicit stale-refs warning; a failed base read zeroes at the LOCAL version file's width so a 3-digit repo allocates 0.0.1, not 0.0.1.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(setup): hooks register the global-install path and re-point stale ones Registering hooks from a dev worktree baked that worktree's absolute path into settings.json — deleting the worktree left a dead hook erroring on every session stop, and the presence-only dedup (list-sources | grep) could never re-point it. setup's hook paths now route through _hook_install_path (global install preferred, source dir fallback), and the new ensure-event verb on gstack-settings-hook compares the registered command payload against canonical: identical → no write, different → single atomic replacement (never zero or two registrations). The plan-tune hooks had the same stale pattern and get the same fix without re-triggering their consent prompt. Also hardened: bun 1.3.13 turns an uncaught sync fs error in bun -e into a SILENT exit 0 — the registrar's write path now catches, prints, and exits 1, so a failed update can never report fake-green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(preamble): learnings capture is unconditional at completion (#2402) 43 of 44 learnings entries came from explicit /learn — the completion-status prose read 'if you discovered a durable project quirk... log it', which models treated as optional. The step now ALWAYS runs: review the session for durable learnings, log each one, and state 'No durable learnings this session' explicitly when the review comes up empty — an empty result, never a skipped step. Re-derived from PR #2612 under the generated-file screening rule. Fixes #2402. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scrape): untrusted-content warning on the page-fetching skills (#2441) /scrape and /skillify consumed page content with zero injection guidance — the CHANGELOG claimed coverage the skills didn't have. The warning now lives in ONE exported const (UNTRUSTED_CONTENT_WARNING in resolvers/browse.ts), embedded in the browse COMMAND_REFERENCE as before AND injected standalone into both skills via the new {{UNTRUSTED_CONTENT_WARNING}} token — single source, wording can never drift between surfaces. Re-derived from PR #2612 under the generated-file screening rule. (Structural isolation for skillify-generated code is tracked as its own TODO.) Fixes #2441. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): checklist paths resolve from the installed skill root (#2518) /review Step 2 read .claude/skills/review/checklist.md — a path relative to the TARGET repo, which only resolves in gstack's own checkout. Every checklist/greptile-triage/TODOS-format reference (six across five templates — two more than the issue named, same class) now uses the installed-root form ~/.claude/skills/gstack/review/... that the templates' other references already use. The install-root class itself (non-default install dirs) is #1882, deliberately its own PR. Fixes #2518. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pair-agent): one-way-door consent question before a daemon relaunch (template half) The skill flow now checks daemon liveness before Step 4 and asks an explicit one-way-door question (tabs/cookies/logins are lost) before passing --force-restart — never proceeding on a vague reply. Pairs with the CLI-half commit that stopped pair-agent auto-killing live daemons. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(codex): resume does not amortize the ~21K session prelude (#2387) Measured (#2387): every codex exec call pays Codex's session prelude, and a resumed call came in slightly ABOVE a fresh one — resume buys continuity, never token savings. The skill now says so where the resume flow lives: prefer one codex call per skill, batch questions into it. Fixes #2387. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(upgrade): fast-forward first; reset --hard only behind a proved-safe gate (#2517) /gstack-upgrade went straight to stash + reset --hard origin/main. Now it tries git pull --ff-only --autostash first (the same policy session-update's auto-upgrade uses). The destructive fallback runs unprompted ONLY when both git status --porcelain AND git rev-list origin/main..HEAD are empty — a clean tree with unpushed local commits is NOT safe, reset destroys them. Anything else requires an explicit one-way-door confirmation that lists every dirty file and unpushed commit being discarded. Fixes #2517. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(preamble): brain-sync block counts the spool queue and resolves MCP project-first Two resolver halves deferred from earlier wave commits: the queue-depth line counts .brain-queue.d/*.json spool records (plus legacy lines until the drain migrates them), and GBRAIN_MCP_ENTRY_JQ swaps its operands to nearest-ancestor-project-first — matching the empirically verified Claude Code precedence (project-local beats user scope) instead of the backwards user-first assumption. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate SKILL.md docs + golden fixtures (single regen for the template block) Pure generator output for the six template/resolver commits above (learnings capture, untrusted-content warning, review paths, pair-agent consent, codex resume note, upgrade ff-only, brain-sync block) — bun run gen:skill-docs + --host codex + --host factory, with the three ship golden fixtures refreshed per the documented procedure. The three sidecar-path pins in gen-skill-docs.test.ts move to the new installed-root/$GSTACK_ROOT contract (#2518). Restores template freshness; full suite green from here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: TODOS.md — strike the six wave-fixed residuals, add two follow-ups The v1.67 adversarial-review residuals section shrinks to the one item the wave couldn't reach (iOS tap routing — needs real-device verification). New entries: skillify structural isolation (a prose warning is not a boundary for page-derived generated code) and the slug store migration (pre-fix sessions on stray-marker machines filed data under the degraded slug; post-fix reads go to the correct store, so history needs a merge/alias). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: align cross-cutting pins with the wave's contracts Three suites pinned pre-wave behavior: browse's gstack-config test asserted the old unknown-key ''/exit-0 shape (#2611 made it exit 1); the Windows-paths suite pinned O_APPEND enqueue atomicity (the spool design satisfies the same invariant via tmp + os.replace, one file per record — pinned in its new form); and nine carve-guard skeleton ceilings absorbed the #2402 unconditional-learnings prose (~450B per skill), bumped with measured values per the guard's own protocol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: re-anchor the referenced-path scanner self-check to the gstack-rooted review refs The self-check pinned the review checklist as a class-1 alias-relative ref; #2518 moved those refs to the installed gstack root (class 2). The guard now proves the scanner sees them in their new class, so the class-2 assertion can't go vacuous. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: pin the wave's prose-tier behaviors (ship coverage-audit gap closure) The coverage audit found one regression-shaped gap: nothing pinned that the upgrade template's ff-only pull precedes the gated reset --hard (#2517) — a future template edit reverting to reset-first would fail nothing. Pinned: the ordering, the FF_OK gate, and the unpushed-commits check. Also pinned the two minor gaps: the {{UNTRUSTED_CONTENT_WARNING}} injection points in scrape/skillify (#2441) and brain-uninstall's spool-dir cleanup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: pre-landing review round — 8 auto-fixes + 8 accepted findings hardened The ship review army (4 specialists + red-team + checklist, 29 findings) produced 8 mechanical auto-fixes and 11 decisions; the accepted set: - win32 slug parity completed: lib/bin-context.ts gains the remote-first outermost walk + degraded-cache self-heal the bash side got this wave — the two implementations now agree on the stray-marker live-bug shape, pinned by shared fixtures (multi-specialist 9/10 finding). - probe honors the plan's bounded-read decision: 256KB prefix, extraction semantics mirrored from parseTranscriptJsonl so probe/prepare can never diverge on the same file (>1MB transcript test). - policy normalize parity: bash normalize() now matches canonicalizeRemote on .git/-trailing and uppercase-.GIT shapes (7-shape corpus pinned two ways) — a deny for those shapes could previously slip the transcript gate. - session-update reclaim is TOCTOU-safe (atomic mv-aside on both branches). - settings-hook: unparseable settings.json errors instead of being replaced with {}; ensure-event keys on (event, source) so matcher changes update in place — never zero or two registrations. - dot-only slug guard at both parse sites (hostile 'url = ..' can't escape projects/); enqueue tmp-file janitor (1h TTL, inside the drain lock); brain-sync .migrating never clobbered; drop-queue/status count .migrating; snapshot -o warning correct + surfaced in diff mode; version-bump test order-dependence removed; uninstall clears the advance stamp. Deferred with record: slug heal-probe cost sentinel (P3 TODO), FF_OK conflation (noted, misdiagnosis-only). 270 pass / 0 fail across the 10 touched suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: adversarial round — the P0 finalize fail-safe and 12 hardened findings Three adversarial passes (Claude fresh-context, Codex chaos, Codex structured with P1 gate) on the full wave diff. Multi-source findings, all fixed: - P0: finalize_queue is now explicit-delete-only — a record is unlinked ONLY when classification proves it staged or dropped; a classifier crash, a missing class file, or a malformed pulled .brain-privacy-map.json (which previously nuked the whole snapshotted queue, remotely triggerable) now retains everything, warns, and re-drains next run. load_privacy_map treats corrupt maps as retain-all, never as empty. - next-version cannot silently drop a live claim: unreadable advertised refs get a targeted --depth=1 fetch + retry; still-unreadable claims surface as UNKNOWN warnings instead of duplicate-version silence. - session-update lock: ownership-checked EXIT trap (a TTL-reclaimed holder can no longer delete the new holder's lock) + a 5-min background heartbeat so a legitimately-slow pull/setup is never reclaimed while alive. - ensure-event collapses ALL same-(event,source) duplicates to one canonical entry; unique per-process tmp path; setup call sites surface (not swallow) the hardened refusals. - memory-ingest: --limit counts only policy-permitted pages (denied records no longer starve permitted ones); --probe applies the same policy filter as --bulk (skipped_policy_* fields on the report). - version-bump repair accepts a genuine literal 0.0.0.0 VERSION file. - slug heal restricted to the stray-.git shape — package.json-anchored wrapper roots keep their legit sticky identity (#2212 preserved). - brain-sync: idle fast path sees leftover .migrating records; unparseable spool records quarantine instead of warning forever; migration comment stops overclaiming the transition-window race. - CDP throttling justifications document override persistence (callers own restoration), pinned in the allowlist test. Deferred with record: deny retroactivity for already-ingested pages (P2 TODO, same semantics as the code-import gate); legacy-migration tail race (transition-window, requires pre-spool writers). 288 pass / 0 fail across the 10 touched suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate SKILL.md docs + goldens (Windows-separator jq fix) Pure generator output for the brain-sync block's jq ancestor match now accepting backslash-formed Windows project keys — previously project-scoped brains were invisible on Windows while the TS scope resolvers saw them. Golden ship fixtures refreshed per the documented procedure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: codex verify-pass residuals — chunked cwd read, post-filter partial count, migrating depth The verify re-review passed the P1 gate (0 P1s) and left three residuals, all applied: transcriptCwdFromPrefix reads in chunks until one complete record (4MB cap) so a giant first prompt can't truncate mid-JSON and break probe/bulk parity; partial_pages derives from the FINAL prepared set instead of the whole scanned corpus; the preamble queue-depth line counts leftover .brain-queue.jsonl.migrating records like the status path does (regen + goldens included). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: bump version and changelog (v1.68.0.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: update project documentation for v1.68.0.0 BROWSER.md: fix the $B cdp example (positional JSON params, not --json; depth is the real CDP param) and add the new perf-throttling examples (Emulation.setCPUThrottlingRate, Network.emulateNetworkConditions) with their clear-override counterparts. USING_GBRAIN_WITH_GSTACK.md: the state-files table row for the sync queue now names the maildir-style spool dir .brain-queue.d/ that replaced .brain-queue.jsonl this release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: align memory-pipeline probe pins with the #2394 stage-count contract The paid-tier E2E pinned the pre-fix contract (probe headline = raw discovered). Probe now counts post-attribution — the same gate --bulk uses — with an explicit unattributed-skip line. Adds the --include-unattributed companion pin so all 9 fixtures stay accounted for. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(next-version): batch missing-tip fetches — one bounded round trip, never a per-branch crawl The targeted-fetch retry for branches whose advertised tip has no local object ran ONE git fetch per branch (10s cap each). On a shallow clone against a busy remote that crawls the network for minutes — CI's shard deadline killed the free suite mid-file. Missing tips now collect into a single batched shallow fetch (15s cap); refs still missing after the batch (one unservable ref fails the whole transfer) get a capped per-branch retry, and anything past the cap warns as an UNKNOWN claim instead of fetching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(next-version): pin the batched fetch + make the offline-contract tests hermetic Two new G2 pins: N unfetched claim branches resolve with exactly ONE fetch spawn (PATH-shimmed git counts invocations), and one unservable ref no longer poisons the batch — live claims resolve via the bounded retry while only the ghost warns UNKNOWN. The #2545 offline-contract tests now run the CLI in a local fixture repo instead of the repo's own checkout: the checkout path did a live ls-remote against the real origin (operator-network-dependent, and the CI shard-deadline hang). The online-contract test gains a succeeding gh stub, so fallback:null is asserted deterministically instead of only when the operator happens to be authed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(redact-cli): derive the synthetic AWS-key fixture — no contiguous credential literal in source The CI quality gate scans every ADDED diff line with the redact engine, so the #2610 port's raw fixture literals failed the very gate they exist to test. The fixture is now assembled at runtime; the scanner still receives the identical bytes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(next-version): pin the fixture's host via origin-URL sniff — kills the last environment dependence The hermetic offline-contract fixture had no origin remote, so detectHost() fell through to auth probes: a machine with glab authed passed via the gitlab path while a bare CI runner read host:unknown (offline stays false there) and failed. The fixture now pushes to a local bare origin at a path containing github.com — the URL sniff pins host:github identically everywhere, asserted explicitly in both tests, with every git call still local. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: y$un_ <forrest.sun527@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: benjamin beres <benjamin.beres@bienpreter.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Ricky <ricky@kinokostudio.com.hk> Co-authored-by: Connex Client Access <paul@paulkortman.com> Co-authored-by: henbima <henbima@gmail.com>
963 lines
49 KiB
TypeScript
963 lines
49 KiB
TypeScript
/**
|
||
* gbrain-sync integration tests.
|
||
*
|
||
* Covers the core cross-machine memory sync feature end-to-end:
|
||
* - bin/gstack-config gbrain keys (validation, isolation)
|
||
* - bin/gstack-brain-enqueue (atomicity, skip list, no-op gates)
|
||
* - bin/gstack-jsonl-merge (3-way, ts-sort, hash-fallback)
|
||
* - bin/gstack-brain-sync --once (drain, commit, push, secret-scan, skip-file)
|
||
* - bin/gstack-artifacts-init + --restore round-trip
|
||
* - bin/gstack-brain-uninstall preserves user data
|
||
* - env isolation (GSTACK_HOME never bleeds into real ~/.gstack/config.yaml)
|
||
*
|
||
* Runs each test against a temp GSTACK_HOME and a local bare git repo as
|
||
* a fake remote. No live GitHub, no live GBrain.
|
||
*/
|
||
|
||
import { describe, test as _test, expect, beforeEach, afterEach } from 'bun:test';
|
||
|
||
// Boost timeout: brain-sync tests spawn git, network-ls-remote, and 10-way
|
||
// parallel processes — 5s default is too tight.
|
||
const test = (name: string, fn: any) => _test(name, fn, 30000);
|
||
import * as fs from 'fs';
|
||
import * as path from 'path';
|
||
import * as os from 'os';
|
||
import { spawnSync } from 'child_process';
|
||
|
||
const ROOT = path.resolve(import.meta.dir, '..');
|
||
const BIN = path.join(ROOT, 'bin');
|
||
|
||
let tmpHome: string;
|
||
let bareRemote: string;
|
||
|
||
function run(argv: string[], opts: { env?: Record<string, string>; input?: string } = {}) {
|
||
const bin = argv[0];
|
||
const full = bin.startsWith('/') ? bin : path.join(BIN, bin);
|
||
const res = spawnSync(full, argv.slice(1), {
|
||
// HOME is overridden too: gstack-artifacts-init writes
|
||
// $HOME/.gstack-artifacts-remote.txt (plain $HOME, not GSTACK_HOME), so
|
||
// without this every free-suite run clobbers the operator's real
|
||
// artifacts-remote pointer. Keep it inside tmpHome, which afterEach removes.
|
||
env: { ...process.env, HOME: tmpHome, GSTACK_HOME: tmpHome, ...(opts.env || {}) },
|
||
encoding: 'utf-8',
|
||
input: opts.input,
|
||
cwd: ROOT,
|
||
});
|
||
return { stdout: res.stdout || '', stderr: res.stderr || '', status: res.status ?? -1 };
|
||
}
|
||
|
||
function git(args: string[], cwd?: string) {
|
||
const res = spawnSync('git', args, { cwd: cwd || tmpHome, encoding: 'utf-8' });
|
||
return { stdout: res.stdout || '', stderr: res.stderr || '', status: res.status ?? -1 };
|
||
}
|
||
|
||
// ---- spool helpers (maildir-style queue: one FILE per record) ----
|
||
// Writers create <epoch>-<pid>-<uniq>.json under .brain-queue.d/ via tmp +
|
||
// atomic rename; the drain deletes exactly the files it snapshotted. The
|
||
// legacy single-file .brain-queue.jsonl exists only as a migration source.
|
||
const spoolDir = () => path.join(tmpHome, '.brain-queue.d');
|
||
const spoolFiles = () =>
|
||
fs.existsSync(spoolDir())
|
||
? fs.readdirSync(spoolDir()).filter((f) => f.endsWith('.json')).sort()
|
||
: [];
|
||
const spoolText = () =>
|
||
spoolFiles()
|
||
.map((f) => fs.readFileSync(path.join(spoolDir(), f), 'utf-8'))
|
||
.join('');
|
||
let spoolSeq = 0;
|
||
function seedSpool(record: string): string {
|
||
fs.mkdirSync(spoolDir(), { recursive: true });
|
||
spoolSeq += 1;
|
||
const name = `${Math.floor(Date.now() / 1000)}-${process.pid}-t${spoolSeq}.json`;
|
||
fs.writeFileSync(path.join(spoolDir(), name), record.endsWith('\n') ? record : record + '\n');
|
||
return name;
|
||
}
|
||
|
||
beforeEach(() => {
|
||
tmpHome = fs.mkdtempSync(path.join(os.tmpdir(), 'brain-sync-home-'));
|
||
bareRemote = fs.mkdtempSync(path.join(os.tmpdir(), 'brain-sync-remote-'));
|
||
spawnSync('git', ['init', '--bare', '-q', '-b', 'main', bareRemote]);
|
||
});
|
||
|
||
afterEach(() => {
|
||
fs.rmSync(tmpHome, { recursive: true, force: true });
|
||
fs.rmSync(bareRemote, { recursive: true, force: true });
|
||
// Clean up any remote-helper file init may have written. run() now pins
|
||
// HOME to tmpHome so these land inside the removed temp dir, but scrub the
|
||
// real home too as defense in depth — and cover BOTH the legacy brain-remote
|
||
// name and the current artifacts-remote name (init writes the latter).
|
||
for (const name of ['.gstack-brain-remote.txt', '.gstack-artifacts-remote.txt']) {
|
||
const remoteFile = path.join(os.homedir(), name);
|
||
// Only remove if it points at OUR bare remote (don't clobber a real user file).
|
||
try {
|
||
const contents = fs.readFileSync(remoteFile, 'utf-8').trim();
|
||
if (contents === bareRemote) fs.unlinkSync(remoteFile);
|
||
} catch {}
|
||
}
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// Config key validation + env isolation
|
||
// ---------------------------------------------------------------
|
||
describe('gstack-config gbrain keys', () => {
|
||
test('default artifacts_sync_mode is off', () => {
|
||
const r = run(['gstack-config', 'get', 'artifacts_sync_mode']);
|
||
expect(r.status).toBe(0);
|
||
expect(r.stdout.trim()).toBe('off');
|
||
});
|
||
|
||
test('default artifacts_sync_mode_prompted is false', () => {
|
||
const r = run(['gstack-config', 'get', 'artifacts_sync_mode_prompted']);
|
||
expect(r.stdout.trim()).toBe('false');
|
||
});
|
||
|
||
test('accepts full / artifacts-only / off', () => {
|
||
for (const val of ['full', 'artifacts-only', 'off']) {
|
||
const set = run(['gstack-config', 'set', 'artifacts_sync_mode', val]);
|
||
expect(set.status).toBe(0);
|
||
const get = run(['gstack-config', 'get', 'artifacts_sync_mode']);
|
||
expect(get.stdout.trim()).toBe(val);
|
||
}
|
||
});
|
||
|
||
test('invalid artifacts_sync_mode value warns + defaults', () => {
|
||
const r = run(['gstack-config', 'set', 'artifacts_sync_mode', 'bogus']);
|
||
expect(r.stderr).toContain('not recognized');
|
||
const get = run(['gstack-config', 'get', 'artifacts_sync_mode']);
|
||
expect(get.stdout.trim()).toBe('off');
|
||
});
|
||
|
||
test('GSTACK_HOME overrides real config dir', () => {
|
||
// Real ~/.gstack/config.yaml must not change, regardless of what it
|
||
// already contains on the developer's machine.
|
||
const realConfig = path.join(os.homedir(), '.gstack', 'config.yaml');
|
||
const before = fs.existsSync(realConfig) ? fs.readFileSync(realConfig, 'utf-8') : null;
|
||
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
|
||
// The override actually took effect — temp config got the new value.
|
||
const tempConfig = fs.readFileSync(path.join(tmpHome, 'config.yaml'), 'utf-8');
|
||
expect(tempConfig).toContain('artifacts_sync_mode: full');
|
||
|
||
// Real ~/.gstack/config.yaml must not be touched.
|
||
const after = fs.existsSync(realConfig) ? fs.readFileSync(realConfig, 'utf-8') : null;
|
||
expect(after).toBe(before);
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// Enqueue behavior
|
||
// ---------------------------------------------------------------
|
||
describe('gstack-brain-enqueue', () => {
|
||
test('no-op when feature not initialized', () => {
|
||
const r = run(['gstack-brain-enqueue', 'projects/foo/learnings.jsonl']);
|
||
expect(r.status).toBe(0);
|
||
expect(fs.existsSync(spoolDir())).toBe(false);
|
||
expect(fs.existsSync(path.join(tmpHome, '.brain-queue.jsonl'))).toBe(false);
|
||
});
|
||
|
||
test('no-op when mode is off (even if .git exists)', () => {
|
||
fs.mkdirSync(path.join(tmpHome, '.git'), { recursive: true });
|
||
const r = run(['gstack-brain-enqueue', 'projects/foo/learnings.jsonl']);
|
||
expect(r.status).toBe(0);
|
||
expect(fs.existsSync(spoolDir())).toBe(false);
|
||
});
|
||
|
||
test('enqueues one spool file when mode is full and .git exists', () => {
|
||
fs.mkdirSync(path.join(tmpHome, '.git'), { recursive: true });
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
run(['gstack-brain-enqueue', 'projects/foo/learnings.jsonl']);
|
||
const files = spoolFiles();
|
||
expect(files.length).toBe(1);
|
||
// Sortable maildir name: <epoch>-<pid>-<uniq>.json.
|
||
expect(files[0]).toMatch(/^\d+-\d+-\d+\.json$/);
|
||
const obj = JSON.parse(fs.readFileSync(path.join(spoolDir(), files[0]), 'utf-8').trim());
|
||
expect(obj.file).toBe('projects/foo/learnings.jsonl');
|
||
expect(obj.ts).toBeTruthy();
|
||
// No tmp-file droppings left behind.
|
||
expect(fs.readdirSync(spoolDir()).filter((f) => f.startsWith('.tmp-')).length).toBe(0);
|
||
});
|
||
|
||
test('skip list honored', () => {
|
||
fs.mkdirSync(path.join(tmpHome, '.git'), { recursive: true });
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-skip.txt'), 'projects/foo/secret.jsonl\n');
|
||
run(['gstack-brain-enqueue', 'projects/foo/secret.jsonl']);
|
||
run(['gstack-brain-enqueue', 'projects/foo/ok.jsonl']);
|
||
expect(spoolText()).not.toContain('secret.jsonl');
|
||
expect(spoolText()).toContain('ok.jsonl');
|
||
});
|
||
|
||
test('concurrent enqueues all land (one spool file per record)', async () => {
|
||
fs.mkdirSync(path.join(tmpHome, '.git'), { recursive: true });
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
const procs = [];
|
||
for (let i = 0; i < 10; i++) {
|
||
procs.push(new Promise<void>((resolve) => {
|
||
const r = spawnSync(path.join(BIN, 'gstack-brain-enqueue'), [`file-${i}.jsonl`], {
|
||
env: { ...process.env, GSTACK_HOME: tmpHome },
|
||
encoding: 'utf-8',
|
||
});
|
||
resolve();
|
||
}));
|
||
}
|
||
await Promise.all(procs);
|
||
expect(spoolFiles().length).toBe(10);
|
||
for (let i = 0; i < 10; i++) {
|
||
expect(spoolText()).toContain(`file-${i}.jsonl`);
|
||
}
|
||
});
|
||
|
||
test('no args does not crash', () => {
|
||
const r = run(['gstack-brain-enqueue']);
|
||
expect(r.status).toBe(0);
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// JSONL merge driver
|
||
// ---------------------------------------------------------------
|
||
describe('gstack-jsonl-merge', () => {
|
||
test('3-way merge dedups + sorts by ts', () => {
|
||
const base = path.join(tmpHome, 'base.jsonl');
|
||
const ours = path.join(tmpHome, 'ours.jsonl');
|
||
const theirs = path.join(tmpHome, 'theirs.jsonl');
|
||
fs.writeFileSync(base, '');
|
||
fs.writeFileSync(ours, '{"x":1,"ts":"2026-01-01T10:00:00Z"}\n{"x":2,"ts":"2026-01-01T11:00:00Z"}\n');
|
||
fs.writeFileSync(theirs, '{"x":3,"ts":"2026-01-01T09:00:00Z"}\n{"x":2,"ts":"2026-01-01T11:00:00Z"}\n');
|
||
const r = run([path.join(BIN, 'gstack-jsonl-merge'), base, ours, theirs]);
|
||
expect(r.status).toBe(0);
|
||
const lines = fs.readFileSync(ours, 'utf-8').trim().split('\n');
|
||
expect(lines.length).toBe(3);
|
||
expect(lines[0]).toContain('"x":3'); // earliest ts
|
||
expect(lines[2]).toContain('"x":2'); // latest ts
|
||
});
|
||
|
||
test('falls back to hash order for lines without ts', () => {
|
||
const base = path.join(tmpHome, 'base.jsonl');
|
||
const ours = path.join(tmpHome, 'ours.jsonl');
|
||
const theirs = path.join(tmpHome, 'theirs.jsonl');
|
||
fs.writeFileSync(base, '');
|
||
fs.writeFileSync(ours, '{"a":1}\n{"a":2}\n');
|
||
fs.writeFileSync(theirs, '{"a":3}\n{"a":2}\n');
|
||
run([path.join(BIN, 'gstack-jsonl-merge'), base, ours, theirs]);
|
||
const lines = fs.readFileSync(ours, 'utf-8').trim().split('\n');
|
||
expect(lines.length).toBe(3);
|
||
// Order is deterministic (sha256 of each line).
|
||
const again = spawnSync(path.join(BIN, 'gstack-jsonl-merge'), [base, ours, theirs]);
|
||
// (re-running doesn't change the order since same input → same output)
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// Init + sync + restore round-trip
|
||
// ---------------------------------------------------------------
|
||
describe('init + sync + restore round-trip', () => {
|
||
test('init creates canonical files + registers drivers', () => {
|
||
const r = run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
expect(r.status).toBe(0);
|
||
expect(fs.existsSync(path.join(tmpHome, '.git'))).toBe(true);
|
||
expect(fs.existsSync(path.join(tmpHome, '.gitignore'))).toBe(true);
|
||
expect(fs.existsSync(path.join(tmpHome, '.brain-allowlist'))).toBe(true);
|
||
expect(fs.existsSync(path.join(tmpHome, '.brain-privacy-map.json'))).toBe(true);
|
||
expect(fs.existsSync(path.join(tmpHome, '.gitattributes'))).toBe(true);
|
||
expect(fs.existsSync(path.join(tmpHome, '.git/hooks/pre-commit'))).toBe(true);
|
||
// Merge driver registered in local git config.
|
||
const cfg = git(['config', '--get', 'merge.jsonl-append.driver']);
|
||
expect(cfg.stdout).toContain('gstack-jsonl-merge');
|
||
});
|
||
|
||
test('refuses init on different remote', () => {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
const otherRemote = fs.mkdtempSync(path.join(os.tmpdir(), 'brain-other-'));
|
||
spawnSync('git', ['init', '--bare', '-q', '-b', 'main', otherRemote]);
|
||
const r = run(['gstack-artifacts-init', '--remote', otherRemote]);
|
||
expect(r.status).not.toBe(0);
|
||
expect(r.stderr).toContain('already a git repo pointing at');
|
||
fs.rmSync(otherRemote, { recursive: true, force: true });
|
||
});
|
||
|
||
test('full sync: init → enqueue → --once → commit pushed', () => {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'),
|
||
'{"skill":"x","insight":"y","ts":"2026-04-22T10:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
// Check the remote got the commit.
|
||
const log = spawnSync('git', ['--git-dir=' + bareRemote, 'log', '--oneline'], { encoding: 'utf-8' });
|
||
expect(log.stdout).toMatch(/sync: 1 file/);
|
||
});
|
||
|
||
test('restore round-trip: writes on machine A visible on machine B', () => {
|
||
// Machine A.
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'myproj'), { recursive: true });
|
||
const aLearning = '{"skill":"x","insight":"machine A wisdom","ts":"2026-04-22T10:00:00Z"}\n';
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/myproj/learnings.jsonl'), aLearning);
|
||
run(['gstack-brain-enqueue', 'projects/myproj/learnings.jsonl']);
|
||
run(['gstack-brain-sync', '--once']);
|
||
|
||
// Machine B (new temp home).
|
||
const machineB = fs.mkdtempSync(path.join(os.tmpdir(), 'brain-machineB-'));
|
||
const r = run(['gstack-brain-restore', bareRemote], {
|
||
env: { GSTACK_HOME: machineB },
|
||
});
|
||
expect(r.status).toBe(0);
|
||
const restored = fs.readFileSync(path.join(machineB, 'projects/myproj/learnings.jsonl'), 'utf-8');
|
||
expect(restored).toContain('machine A wisdom');
|
||
// Merge drivers re-registered on B.
|
||
const cfg = spawnSync('git', ['-C', machineB, 'config', '--get', 'merge.jsonl-append.driver'], { encoding: 'utf-8' });
|
||
expect(cfg.stdout).toContain('gstack-jsonl-merge');
|
||
fs.rmSync(machineB, { recursive: true, force: true });
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// Secret scan: all regex families block
|
||
// ---------------------------------------------------------------
|
||
describe('gstack-brain-sync secret scan', () => {
|
||
const SECRETS: [string, string][] = [
|
||
['aws-access-key', 'AKIAABCDEFGHIJKLMNOP'],
|
||
['github-token-ghp', 'ghp_abcdefghij1234567890abcdef1234567890'],
|
||
['github-token-github-pat', 'github_pat_11ABCDEFG1234567890_abcdef'],
|
||
['openai-key', 'sk-abcdefghij1234567890abcdef1234567890'],
|
||
['pem-block', '-----BEGIN PRIVATE KEY-----'],
|
||
['jwt', 'eyJ0eXAiOiJKV1QiLCJh.eyJzdWIiOiIxMjM0NTY3.SflKxwRJSMeKKF30oGTbU'],
|
||
['bearer-json', '"authorization":"Bearer abcdef1234567890abcdef1234567890"'],
|
||
];
|
||
|
||
for (const [name, content] of SECRETS) {
|
||
test(`blocks ${name}`, () => {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'),
|
||
`{"leaked":"${content}"}\n`);
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0); // exits clean even when blocked
|
||
// No new commit should have been created.
|
||
const log = git(['log', '--oneline']);
|
||
expect(log.stdout.split('\n').filter(Boolean).length).toBeLessThanOrEqual(3);
|
||
// Status file should report blocked.
|
||
const status = JSON.parse(fs.readFileSync(path.join(tmpHome, '.brain-sync-status.json'), 'utf-8'));
|
||
expect(status.status).toBe('blocked');
|
||
});
|
||
}
|
||
|
||
test('--skip-file unblocks specific file', () => {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
const leakPath = 'projects/p/leaked.jsonl';
|
||
fs.writeFileSync(path.join(tmpHome, leakPath),
|
||
'{"gh":"ghp_abcdefghij1234567890abcdef1234567890"}\n');
|
||
run(['gstack-brain-enqueue', leakPath]);
|
||
run(['gstack-brain-sync', '--once']); // blocked
|
||
run(['gstack-brain-sync', '--skip-file', leakPath]);
|
||
// Any future enqueue of this path should no-op.
|
||
run(['gstack-brain-enqueue', leakPath]);
|
||
const skip = fs.readFileSync(path.join(tmpHome, '.brain-skip.txt'), 'utf-8');
|
||
expect(skip).toContain(leakPath);
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// Egress receipt gate: receipt-before-commit, queue intact on refusal
|
||
// ---------------------------------------------------------------
|
||
describe('gstack-brain-sync egress receipt gate', () => {
|
||
test('refused receipt leaves the queue intact, makes no commit, and next run retries', () => {
|
||
if (process.platform === 'win32' || process.getuid?.() === 0) return; // chmod is advisory there
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'),
|
||
'{"skill":"x","insight":"y","ts":"2026-04-22T10:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
const commitsBefore = git(['rev-list', '--count', 'HEAD']).stdout.trim();
|
||
|
||
// Make the receipt unwritable: security dir exists but is read-only.
|
||
// (artifacts-init may have created it already — mkdirSync's mode is a
|
||
// no-op on an existing dir, so chmod explicitly.)
|
||
fs.mkdirSync(path.join(tmpHome, 'security'), { recursive: true });
|
||
fs.chmodSync(path.join(tmpHome, 'security'), 0o500);
|
||
try {
|
||
const refused = run(['gstack-brain-sync', '--once']);
|
||
expect(refused.status).toBe(1);
|
||
// DX contract: problem + cause + fix, plain language.
|
||
expect(refused.stderr).toContain('NOT sent');
|
||
expect(refused.stderr).toContain('EGRESS_RECEIPT_FAILED');
|
||
expect(refused.stderr).toContain('Fix: chmod -R u+w');
|
||
expect(refused.stderr).toContain('ATTEMPTS to send off-machine');
|
||
// Spool intact (receipt is written BEFORE finalize consumes records).
|
||
expect(spoolText()).toContain('projects/p/learnings.jsonl');
|
||
// No local commit was created.
|
||
expect(git(['rev-list', '--count', 'HEAD']).stdout.trim()).toBe(commitsBefore);
|
||
// Nothing reached the remote.
|
||
const remoteLog = spawnSync('git', ['--git-dir=' + bareRemote, 'log', '--oneline'], { encoding: 'utf-8' });
|
||
expect(remoteLog.stdout).not.toMatch(/sync: 1 file/);
|
||
const status = JSON.parse(fs.readFileSync(path.join(tmpHome, '.brain-sync-status.json'), 'utf-8'));
|
||
expect(status.status).toBe('push_failed');
|
||
expect(status.message).toContain('EGRESS_RECEIPT_FAILED');
|
||
} finally {
|
||
fs.chmodSync(path.join(tmpHome, 'security'), 0o700);
|
||
}
|
||
|
||
// Next run (ledger writable again) drains the intact queue and pushes.
|
||
const retry = run(['gstack-brain-sync', '--once']);
|
||
expect(retry.status).toBe(0);
|
||
const log = spawnSync('git', ['--git-dir=' + bareRemote, 'log', '--oneline'], { encoding: 'utf-8' });
|
||
expect(log.stdout).toMatch(/sync: 1 file/);
|
||
});
|
||
|
||
test('successful push writes a git-class receipt before the send', () => {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'),
|
||
'{"skill":"x","insight":"y","ts":"2026-04-22T10:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
const ledger = fs.readFileSync(path.join(tmpHome, 'security', 'egress.jsonl'), 'utf-8');
|
||
const records = ledger.trim().split('\n').map((l) => JSON.parse(l));
|
||
const pushReceipt = records.find((rec) => rec.sink === 'brain-sync' && rec.payload_class === 'curated-memory-git-push');
|
||
expect(pushReceipt).toBeTruthy();
|
||
expect(pushReceipt.sha256).toBeNull(); // git owns the bytes
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// Uninstall preserves user data
|
||
// ---------------------------------------------------------------
|
||
describe('gstack-brain-uninstall', () => {
|
||
test('removes sync config but preserves learnings/project data', () => {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'user-data'), { recursive: true });
|
||
const preservedContent = '{"keep":"me","ts":"2026-04-22T12:00:00Z"}\n';
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/user-data/learnings.jsonl'), preservedContent);
|
||
const r = run(['gstack-brain-uninstall', '--yes']);
|
||
expect(r.status).toBe(0);
|
||
expect(fs.existsSync(path.join(tmpHome, '.git'))).toBe(false);
|
||
expect(fs.existsSync(path.join(tmpHome, '.gitignore'))).toBe(false);
|
||
expect(fs.existsSync(path.join(tmpHome, '.brain-allowlist'))).toBe(false);
|
||
expect(fs.existsSync(path.join(tmpHome, 'consumers.json'))).toBe(false);
|
||
// Project data preserved.
|
||
const preserved = fs.readFileSync(path.join(tmpHome, 'projects/user-data/learnings.jsonl'), 'utf-8');
|
||
expect(preserved).toBe(preservedContent);
|
||
// Config key reset.
|
||
const mode = run(['gstack-config', 'get', 'artifacts_sync_mode']);
|
||
expect(mode.stdout.trim()).toBe('off');
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// --discover-new: cursor-based change detection
|
||
// ---------------------------------------------------------------
|
||
describe('gstack-brain-sync --discover-new', () => {
|
||
test('enqueues new allowlisted files as spool records; idempotent on re-run', () => {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.mkdirSync(path.join(tmpHome, 'retros'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'retros/week-1.md'), '# retro\n');
|
||
run(['gstack-brain-sync', '--discover-new']);
|
||
expect(spoolText()).toContain('retros/week-1.md');
|
||
// Clear the spool, run again — idempotent (no new records).
|
||
for (const f of spoolFiles()) fs.unlinkSync(path.join(spoolDir(), f));
|
||
run(['gstack-brain-sync', '--discover-new']);
|
||
expect(spoolFiles().length).toBe(0);
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// Enqueue tmp janitor: a writer killed between its tmp write and the
|
||
// atomic rename orphans a .tmp-* file forever (it never becomes a
|
||
// record, nothing else touches it). The drain reaps ones older than
|
||
// 1 hour, inside its lock; fresh ones (in-flight enqueues) survive.
|
||
// ---------------------------------------------------------------
|
||
describe('enqueue tmp janitor', () => {
|
||
test('an orphaned .tmp-* older than 1h is reaped on --once; a fresh one survives', () => {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
|
||
fs.mkdirSync(spoolDir(), { recursive: true });
|
||
|
||
const oldTmp = path.join(spoolDir(), '.tmp-99999-x1');
|
||
fs.writeFileSync(oldTmp, '{"file":"projects/p/learnings.jsonl"}\n');
|
||
const past = new Date(Date.now() - 2 * 3600 * 1000);
|
||
fs.utimesSync(oldTmp, past, past);
|
||
|
||
const freshTmp = path.join(spoolDir(), '.tmp-99999-x2');
|
||
fs.writeFileSync(freshTmp, '{"file":"projects/p/learnings.jsonl"}\n');
|
||
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(fs.existsSync(oldTmp)).toBe(false); // orphan reaped
|
||
expect(fs.existsSync(freshTmp)).toBe(true); // in-flight write untouched
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// #2549 queue integrity: classified drops, privacy retention,
|
||
// surgical rewrite, unpushed-commit detector
|
||
// ---------------------------------------------------------------
|
||
describe('#2549 queue integrity', () => {
|
||
function initWithMode(mode: string) {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', mode]);
|
||
}
|
||
const statusJson = () => JSON.parse(fs.readFileSync(path.join(tmpHome, '.brain-sync-status.json'), 'utf-8'));
|
||
|
||
test('privacy-held entries are RETAINED and classified, not wiped as "no allowlisted changes"', () => {
|
||
// timeline.jsonl is class=behavioral; artifacts-only mode holds it.
|
||
initWithMode('artifacts-only');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/timeline.jsonl'), '{"skill":"x","event":"started"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/timeline.jsonl']);
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
// The exact #2549 repro: the old code truncated the queue here and said
|
||
// "no allowlisted changes in queue". The record must survive, and the
|
||
// status must attribute the hold honestly.
|
||
expect(spoolText()).toContain('projects/p/timeline.jsonl');
|
||
const s = statusJson();
|
||
expect(s.status).toBe('idle');
|
||
expect(s.message).toContain('privacy-held retained');
|
||
expect(s.message).not.toContain('no allowlisted changes');
|
||
});
|
||
|
||
test('unmatched and missing entries drop WITH counts and a 0600 drops sidecar', () => {
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
// Unmatched: no allowlist glob covers .txt scratch files.
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/scratch.txt'), 'x\n');
|
||
seedSpool('{"file":"projects/p/scratch.txt"}');
|
||
// Missing: allowlisted name that does not exist on disk.
|
||
seedSpool('{"file":"projects/p/learnings.jsonl"}');
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
expect(spoolText()).not.toContain('scratch.txt');
|
||
expect(spoolText()).not.toContain('learnings.jsonl');
|
||
const s = statusJson();
|
||
expect(s.message).toContain('1 unmatched dropped');
|
||
expect(s.message).toContain('1 missing dropped');
|
||
const drops = path.join(tmpHome, '.brain-sync-drops.json');
|
||
expect(fs.existsSync(drops)).toBe(true);
|
||
if (process.platform !== 'win32') {
|
||
expect(fs.statSync(drops).mode & 0o777).toBe(0o600);
|
||
}
|
||
const detail = JSON.parse(fs.readFileSync(drops, 'utf-8'));
|
||
expect(detail.dropped.unmatched).toContain('projects/p/scratch.txt');
|
||
expect(detail.dropped.missing).toContain('projects/p/learnings.jsonl');
|
||
});
|
||
|
||
test('an unparseable legacy queue line migrates as-is and is quarantined, never destroyed', () => {
|
||
// The line lands in the legacy single-file queue (pre-spool writer);
|
||
// migration converts it verbatim to a spool record, and the drain moves
|
||
// what it cannot parse into quarantine (never deletes it, and never
|
||
// leaves it re-warning at every boundary).
|
||
initWithMode('full');
|
||
fs.appendFileSync(path.join(tmpHome, '.brain-queue.jsonl'), 'not json at all\n');
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
const qDir = path.join(spoolDir(), 'quarantine');
|
||
expect(fs.existsSync(qDir)).toBe(true);
|
||
const qFiles = fs.readdirSync(qDir);
|
||
expect(qFiles.length).toBe(1);
|
||
expect(fs.readFileSync(path.join(qDir, qFiles[0]), 'utf-8')).toContain('not json at all');
|
||
});
|
||
|
||
test('finalize: a synced record leaves the spool while a held sibling survives the same drain', () => {
|
||
// Proves finalize is a per-record delete, not a truncation: two records
|
||
// drain in one --once, one stages+pushes, one is mode-held.
|
||
initWithMode('artifacts-only');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"x","insight":"y","ts":"2026-01-01T00:00:00Z"}\n');
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/timeline.jsonl'), '{"skill":"x","event":"started"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
run(['gstack-brain-enqueue', 'projects/p/timeline.jsonl']);
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
expect(spoolText()).not.toContain('learnings.jsonl'); // synced, removed
|
||
expect(spoolText()).toContain('timeline.jsonl'); // held, retained
|
||
const log = spawnSync('git', ['--git-dir=' + bareRemote, 'log', '--oneline'], { encoding: 'utf-8' });
|
||
expect(log.stdout).toMatch(/sync: 1 file/);
|
||
});
|
||
|
||
test('push failure retains the commit locally and the run-start detector re-pushes it', () => {
|
||
initWithMode('full');
|
||
// Establish origin/main so the detector has a remote ref to compare.
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"a","ts":"2026-01-01T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
|
||
// Reject the next push at the remote (pre-receive hook exits 1 with an
|
||
// auth-shaped message so the auth branch is exercised too).
|
||
const hook = path.join(bareRemote, 'hooks', 'pre-receive');
|
||
fs.writeFileSync(hook, '#!/bin/sh\necho "403 forbidden" >&2\nexit 1\n');
|
||
fs.chmodSync(hook, 0o755);
|
||
|
||
fs.appendFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"b","ts":"2026-01-02T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
const fail = run(['gstack-brain-sync', '--once']);
|
||
expect(fail.status).toBe(0);
|
||
const s = statusJson();
|
||
expect(s.status).toBe('push_failed');
|
||
expect(s.message).toContain('commit retained locally');
|
||
// Drained record left the spool — it lives in the local commit now.
|
||
expect(spoolText()).not.toContain('learnings.jsonl');
|
||
// The commit exists locally, ahead of origin.
|
||
const ahead = git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim();
|
||
expect(Number(ahead)).toBeGreaterThan(0);
|
||
|
||
// Remote healthy again: an EMPTY-queue run must still deliver the
|
||
// stranded commit (the detector, not the drain, pushes it).
|
||
fs.rmSync(hook);
|
||
const retry = run(['gstack-brain-sync', '--once']);
|
||
expect(retry.status).toBe(0);
|
||
const log = spawnSync('git', ['--git-dir=' + bareRemote, 'log', '--oneline'], { encoding: 'utf-8' });
|
||
expect(log.stdout).toMatch(/sync: 1 file/);
|
||
expect(git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim()).toBe('0');
|
||
});
|
||
|
||
test('receipt refusal at the detector skips the retry without wedging the drain', () => {
|
||
if (process.platform === 'win32' || process.getuid?.() === 0) return; // chmod advisory there
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"a","ts":"2026-01-01T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
|
||
// Strand a commit: reject pushes, drain once.
|
||
const hook = path.join(bareRemote, 'hooks', 'pre-receive');
|
||
fs.writeFileSync(hook, '#!/bin/sh\nexit 1\n');
|
||
fs.chmodSync(hook, 0o755);
|
||
fs.appendFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"b","ts":"2026-01-02T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
fs.rmSync(hook);
|
||
|
||
// Break receipts. The detector's retry must be SKIPPED (no wedge), and
|
||
// the run must still exit 0 with nothing else to do.
|
||
fs.mkdirSync(path.join(tmpHome, 'security'), { recursive: true });
|
||
fs.chmodSync(path.join(tmpHome, 'security'), 0o500);
|
||
try {
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
// Commit still stranded (retry skipped, not attempted unreceipted).
|
||
expect(Number(git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim())).toBeGreaterThan(0);
|
||
} finally {
|
||
fs.chmodSync(path.join(tmpHome, 'security'), 0o700);
|
||
}
|
||
|
||
// Receipts healthy: detector delivers. The refused attempt above stamped
|
||
// the 10-minute throttle (deliberately — refusals must not busy-loop the
|
||
// network at every skill boundary), so model the interval passing.
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-last-push-attempt'), '0');
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim()).toBe('0');
|
||
});
|
||
|
||
test('detector attempts are throttled to one per interval', () => {
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"a","ts":"2026-01-01T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
|
||
// Strand a commit behind a rejecting remote.
|
||
const hook = path.join(bareRemote, 'hooks', 'pre-receive');
|
||
fs.writeFileSync(hook, '#!/bin/sh\nexit 1\n');
|
||
fs.chmodSync(hook, 0o755);
|
||
fs.appendFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"b","ts":"2026-01-02T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
fs.rmSync(hook);
|
||
|
||
// First empty-queue run: detector attempts (stamps the throttle), pushes.
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
const stamp1 = fs.readFileSync(path.join(tmpHome, '.brain-last-push-attempt'), 'utf-8');
|
||
expect(Number(stamp1)).toBeGreaterThan(0);
|
||
expect(git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim()).toBe('0');
|
||
|
||
// Strand another; an immediate second run must NOT attempt (stamp fresh).
|
||
fs.writeFileSync(hook, '#!/bin/sh\nexit 1\n');
|
||
fs.chmodSync(hook, 0o755);
|
||
fs.appendFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"c","ts":"2026-01-03T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
fs.rmSync(hook);
|
||
const stampBefore = fs.readFileSync(path.join(tmpHome, '.brain-last-push-attempt'), 'utf-8');
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
// Throttled: stamp unchanged, commit still stranded.
|
||
expect(fs.readFileSync(path.join(tmpHome, '.brain-last-push-attempt'), 'utf-8')).toBe(stampBefore);
|
||
expect(Number(git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim())).toBeGreaterThan(0);
|
||
|
||
// Interval passed: delivers.
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-last-push-attempt'), '0');
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim()).toBe('0');
|
||
});
|
||
|
||
test('an interleaved user commit disables the detector push (exclusive author gate)', () => {
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"a","ts":"2026-01-01T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
|
||
// Strand a bot commit behind a rejecting remote.
|
||
const hook = path.join(bareRemote, 'hooks', 'pre-receive');
|
||
fs.writeFileSync(hook, '#!/bin/sh\nexit 1\n');
|
||
fs.chmodSync(hook, 0o755);
|
||
fs.appendFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"b","ts":"2026-01-02T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
fs.rmSync(hook);
|
||
|
||
// A user manually commits in ~/.gstack on top of the stranded bot commit.
|
||
expect(git(['-c', 'user.name=Garry', '-c', 'user.email=garry@example.com',
|
||
'-c', 'commit.gpgsign=false',
|
||
'commit', '--allow-empty', '-m', 'manual note']).status).toBe(0);
|
||
|
||
// Interval passed, remote healthy, queue empty: the detector must STILL
|
||
// refuse — `push origin HEAD` would publish the user's commit uninvited.
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-last-push-attempt'), '0');
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(Number(git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim())).toBe(2);
|
||
|
||
// A REAL drain still rides the user commit along, as before — the gate
|
||
// scopes only the detector's autonomous retry, not user-initiated syncs.
|
||
fs.appendFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"c","ts":"2026-01-03T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(git(['rev-list', '--count', 'origin/main..HEAD']).stdout.trim()).toBe('0');
|
||
});
|
||
});
|
||
|
||
// ---------------------------------------------------------------
|
||
// C12 spool queue: per-record files kill the enqueue/drain race.
|
||
// One FILE per record under .brain-queue.d/ — writer and drainer never
|
||
// share an inode, so the lockless append-vs-rewrite race is structurally
|
||
// gone. Crash semantics are at-least-once (unfinalized records re-drain).
|
||
// ---------------------------------------------------------------
|
||
describe('C12 spool queue', () => {
|
||
function initWithMode(mode: string) {
|
||
run(['gstack-artifacts-init', '--remote', bareRemote]);
|
||
run(['gstack-config', 'set', 'artifacts_sync_mode', mode]);
|
||
}
|
||
const remoteLog = () =>
|
||
spawnSync('git', ['--git-dir=' + bareRemote, 'log', '--oneline'], { encoding: 'utf-8' }).stdout;
|
||
|
||
test('two rapid enqueues of different paths create two spool files; one drain syncs both', () => {
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.mkdirSync(path.join(tmpHome, 'retros'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"x","ts":"2026-01-01T00:00:00Z"}\n');
|
||
fs.writeFileSync(path.join(tmpHome, 'retros/week-1.md'), '# retro\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
run(['gstack-brain-enqueue', 'retros/week-1.md']);
|
||
expect(spoolFiles().length).toBe(2);
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
expect(spoolFiles().length).toBe(0);
|
||
expect(remoteLog()).toMatch(/sync: 2 file/);
|
||
});
|
||
|
||
test('a record created after a drain survives untouched and drains on the NEXT --once', () => {
|
||
// Structural form of the concurrent-append test: finalize deletes only
|
||
// snapshot-manifest files, so a record the drain never listed cannot be
|
||
// touched — whether it lands mid-drain or after.
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.mkdirSync(path.join(tmpHome, 'retros'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"a","ts":"2026-01-01T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(spoolFiles().length).toBe(0);
|
||
// New record arrives (a writer that raced the previous drain).
|
||
fs.writeFileSync(path.join(tmpHome, 'retros/week-1.md'), '# retro\n');
|
||
run(['gstack-brain-enqueue', 'retros/week-1.md']);
|
||
const [pending] = spoolFiles();
|
||
expect(pending).toBeTruthy();
|
||
const pendingContent = fs.readFileSync(path.join(spoolDir(), pending), 'utf-8');
|
||
expect(pendingContent).toContain('retros/week-1.md');
|
||
// Untouched by the completed drain; the NEXT drain delivers it.
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(spoolFiles().length).toBe(0);
|
||
expect(remoteLog()).toMatch(/sync: 1 file/);
|
||
});
|
||
|
||
test('at-least-once: a drain that fails before finalize leaves every spool file for the next run', () => {
|
||
if (process.platform === 'win32' || process.getuid?.() === 0) return; // chmod advisory there
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.mkdirSync(path.join(tmpHome, 'retros'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"a","ts":"2026-01-01T00:00:00Z"}\n');
|
||
fs.writeFileSync(path.join(tmpHome, 'retros/week-1.md'), '# retro\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
run(['gstack-brain-enqueue', 'retros/week-1.md']);
|
||
const seeded = spoolFiles();
|
||
expect(seeded.length).toBe(2);
|
||
|
||
// Break the egress-receipt ledger: the drain fails AFTER staging but
|
||
// BEFORE any commit or finalize — simulating a crash mid-drain.
|
||
fs.mkdirSync(path.join(tmpHome, 'security'), { recursive: true });
|
||
fs.chmodSync(path.join(tmpHome, 'security'), 0o500);
|
||
try {
|
||
const refused = run(['gstack-brain-sync', '--once']);
|
||
expect(refused.status).toBe(1);
|
||
// The exact same spool files are still present — nothing consumed.
|
||
expect(spoolFiles()).toEqual(seeded);
|
||
} finally {
|
||
fs.chmodSync(path.join(tmpHome, 'security'), 0o700);
|
||
}
|
||
|
||
// Next run re-drains the surviving records.
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(spoolFiles().length).toBe(0);
|
||
expect(remoteLog()).toMatch(/sync: 2 file/);
|
||
});
|
||
|
||
test('legacy migration: .brain-queue.jsonl lines convert to spool records, nothing lost', () => {
|
||
// Pre-spool writers appended to the single-file queue. Three lines: two
|
||
// stageable artifacts, one behavioral (mode-held under artifacts-only).
|
||
initWithMode('artifacts-only');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.mkdirSync(path.join(tmpHome, 'retros'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"x","ts":"2026-01-01T00:00:00Z"}\n');
|
||
fs.writeFileSync(path.join(tmpHome, 'retros/week-1.md'), '# retro\n');
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/timeline.jsonl'), '{"skill":"x","event":"started"}\n');
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-queue.jsonl'),
|
||
'{"file":"projects/p/learnings.jsonl","ts":"2026-01-01T00:00:00Z"}\n' +
|
||
'{"file":"retros/week-1.md","ts":"2026-01-01T00:00:01Z"}\n' +
|
||
'{"file":"projects/p/timeline.jsonl","ts":"2026-01-01T00:00:02Z"}\n');
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
// Legacy file consumed; no .migrating remnant.
|
||
expect(fs.existsSync(path.join(tmpHome, '.brain-queue.jsonl'))).toBe(false);
|
||
expect(fs.existsSync(path.join(tmpHome, '.brain-queue.jsonl.migrating'))).toBe(false);
|
||
// Both artifacts synced; the behavioral record survives as a spool file.
|
||
expect(remoteLog()).toMatch(/sync: 2 file/);
|
||
expect(spoolText()).toContain('projects/p/timeline.jsonl');
|
||
expect(spoolText()).not.toContain('learnings.jsonl');
|
||
});
|
||
|
||
test('an unparseable spool record is quarantined with a warning; the drain continues', () => {
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"x","ts":"2026-01-01T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
const badFile = seedSpool('this is not json');
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
expect(r.stderr).toContain('unparseable');
|
||
// The good sibling synced; the unreadable record was never destroyed —
|
||
// it moved to quarantine so it stops re-warning at every boundary.
|
||
expect(remoteLog()).toMatch(/sync: 1 file/);
|
||
expect(spoolFiles()).toEqual([]);
|
||
const qPath = path.join(spoolDir(), 'quarantine', badFile);
|
||
expect(fs.existsSync(qPath)).toBe(true);
|
||
expect(fs.readFileSync(qPath, 'utf-8')).toContain('this is not json');
|
||
});
|
||
|
||
test('--status queue_depth counts spool records plus unmigrated legacy lines', () => {
|
||
initWithMode('full');
|
||
seedSpool('{"file":"projects/p/a.jsonl","ts":"t"}');
|
||
seedSpool('{"file":"projects/p/b.jsonl","ts":"t"}');
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-queue.jsonl'), '{"file":"projects/p/c.jsonl","ts":"t"}\n');
|
||
const r = run(['gstack-brain-sync', '--status']);
|
||
expect(r.status).toBe(0);
|
||
const supplemental = JSON.parse(r.stdout.trim().split('\n').pop()!);
|
||
expect(supplemental.queue_depth).toBe(3);
|
||
});
|
||
|
||
test('G1: a malformed pulled privacy map (["bad"]) holds the queue — warns, deletes NOTHING, next run re-drains', () => {
|
||
// Remotely triggerable kill vector: the privacy map arrives via the
|
||
// artifacts-repo pull. A non-dict entry used to raise mid-classification
|
||
// AFTER the snapshot manifest was written, and the old finalize polarity
|
||
// ("delete unless retained") then unlinked EVERY snapshotted record.
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"x","ts":"2026-01-01T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
const seeded = spoolFiles();
|
||
expect(seeded.length).toBe(1);
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-privacy-map.json'), '["bad"]');
|
||
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
expect(r.stderr).toContain('privacy map');
|
||
// Zero records deleted; nothing pushed.
|
||
expect(spoolFiles()).toEqual(seeded);
|
||
expect(remoteLog()).not.toMatch(/sync:/);
|
||
|
||
// Fix the map: the surviving queue re-drains and syncs.
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-privacy-map.json'), '[]');
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(spoolFiles().length).toBe(0);
|
||
expect(remoteLog()).toMatch(/sync: 1 file/);
|
||
});
|
||
|
||
test('G1: a classifier that dies AFTER the snapshot write consumes nothing (call-site exit check + explicit-delete finalize)', () => {
|
||
// A dict entry with a non-string pattern passes the shape filter but
|
||
// raises inside fnmatch DURING classification — the post-manifest crash
|
||
// window (same shape as ENOSPC/OOM mid-run). The call site must see the
|
||
// nonzero exit, warn, skip finalize, and leave everything queued.
|
||
initWithMode('full');
|
||
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
|
||
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'), '{"skill":"x","ts":"2026-01-01T00:00:00Z"}\n');
|
||
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
|
||
const seeded = spoolFiles();
|
||
expect(seeded.length).toBe(1);
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-privacy-map.json'), '[{"pattern": 123}]');
|
||
|
||
const r = run(['gstack-brain-sync', '--once']);
|
||
expect(r.status).toBe(0);
|
||
expect(r.stderr).toContain('classification failed');
|
||
expect(spoolFiles()).toEqual(seeded); // zero records deleted
|
||
expect(remoteLog()).not.toMatch(/sync:/);
|
||
const status = JSON.parse(fs.readFileSync(path.join(tmpHome, '.brain-sync-status.json'), 'utf-8'));
|
||
expect(status.status).toBe('error');
|
||
expect(status.message).toContain('queue preserved');
|
||
|
||
// Fix the map: the surviving queue re-drains and syncs.
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-privacy-map.json'), '[]');
|
||
expect(run(['gstack-brain-sync', '--once']).status).toBe(0);
|
||
expect(spoolFiles().length).toBe(0);
|
||
expect(remoteLog()).toMatch(/sync: 1 file/);
|
||
});
|
||
|
||
test('G1: finalize is explicit-delete-only and the fast path is .migrating-aware (static pins)', () => {
|
||
const src = fs.readFileSync(path.join(BIN, 'gstack-brain-sync'), 'utf-8');
|
||
// The compute call site checks the python exit status before finalizing.
|
||
expect(src).toMatch(/if ! compute_paths_to_stage /);
|
||
// finalize_queue takes the staged-paths file and deletes only staged ∪ dropped.
|
||
expect(src).toContain('deletable = staged | dropped');
|
||
expect(src).toContain('if p not in deletable:');
|
||
// The empty fast path also treats a leftover .migrating file as non-idle.
|
||
expect(src).toMatch(/spool_has_records && \[ ! -s "\$QUEUE" \] && \[ ! -s "\$QUEUE\.migrating" \]/);
|
||
});
|
||
|
||
test('--drop-queue keeps the --yes gate and counts spool + legacy entries', () => {
|
||
initWithMode('full');
|
||
seedSpool('{"file":"projects/p/a.jsonl","ts":"t"}');
|
||
seedSpool('{"file":"projects/p/b.jsonl","ts":"t"}');
|
||
fs.writeFileSync(path.join(tmpHome, '.brain-queue.jsonl'), '{"file":"projects/p/c.jsonl","ts":"t"}\n');
|
||
const refused = run(['gstack-brain-sync', '--drop-queue']);
|
||
expect(refused.status).toBe(1);
|
||
expect(refused.stderr).toContain('--yes');
|
||
expect(spoolFiles().length).toBe(2);
|
||
const dropped = run(['gstack-brain-sync', '--drop-queue', '--yes']);
|
||
expect(dropped.status).toBe(0);
|
||
expect(dropped.stdout).toContain('dropped 3 queue entries');
|
||
expect(spoolFiles().length).toBe(0);
|
||
expect(fs.readFileSync(path.join(tmpHome, '.brain-queue.jsonl'), 'utf-8')).toBe('');
|
||
const again = run(['gstack-brain-sync', '--drop-queue', '--yes']);
|
||
expect(again.stdout).toContain('queue already empty');
|
||
});
|
||
});
|