mirror of
https://github.com/garrytan/gstack.git
synced 2026-08-23 14:32:33 +02:00
v1.68.0.0 fix: next tracker wave — 16 verified fixes in, 90 stale PRs and 21 issues closed with receipts (#2632)
* fix(plan-tune): reject never-ask on one-way ids at --write --check already ignored those prefs; --write still stored them and --stats counted them as a working NEVER_ASK. Refuse the write and count leftover on-disk prefs as INERT_ONE_WAY. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix: gstack-config get returns "" with exit 0 for keys that have no default Skill preambles read configuration with VAR=$(gstack-config get <key> 2>/dev/null || echo "<default>") and that fallback only fires on a non-zero exit. lookup_default ended in a catch-all that echoed "" and returned 0, so for any key missing from the table VAR came back empty and the default written right there in the preamble was unreachable. The skill then branched on a value it never specified: "skip entirely if QUESTION_TUNING is false", reached with QUESTION_TUNING="". Four keys that skills actually read had no entry and took that path: question_tuning -> callers assume "false" repo_mode -> callers assume "unknown" team_mode -> callers assume "false" transcript_ingest_mode -> callers assume "off" Each default above is the value the call sites already substitute in their own `|| echo` fallback, so this only makes reachable what was already intended. The catch-all now returns non-zero. That is deliberately scoped to the unknown-key arm alone: keys whose default is intentionally empty still exit 0, because "" is their real answer and their callers depend on it -- cross_project_learnings ("unset triggers the first-time prompt"), redact_repo_visibility ("empty falls through to gh/glab detection"), salience_allowlist, user_slug_at_*. Making every empty answer an error would have broken those. test/gstack-config-defaults.test.ts pins the class rather than the four instances: it parses the case arms and asserts every `gstack-config get <key>` site in the tree is covered, so adding a read without a default fails CI. It also pins the exit-code contract in both directions. Verified failing against the pre-fix script, where it names exactly those four keys. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(redact): a typo'd subcommand no longer exits 0 having done nothing main() recognised exactly two subcommands and let everything else fall through to the stdin scan. On empty stdin that prints "(no findings)" and exits 0, so: $ gstack-redact install-prepush-hooks # plural typo gstack-redact scan — repo UNKNOWN (no findings) $ echo $? 0 No hook was installed, and the operator has every reason to believe the credential guard is armed. A guard that silently no-ops must never exit 0. Two smaller faults in the same dispatch, both of which lead people here: - There was no --help handler, so `gstack-redact --help` fell through to the scanner. Piping a credential to it scanned the secret and exited 3. - With no piped input and no --from-file, readInput() blocks on readSync(fd 0) until an EOF that an interactive terminal never sends. That prints nothing at all, so it reads as a hang rather than as "this is a filter, feed it". Now: --help/-h/help prints usage and exits 0; an unrecognised positional prints the offender and exits 1; a TTY with nothing piped in prints usage instead of blocking. "scan" stays accepted, because the human output header reads "gstack-redact scan — repo …" and that is what people type. Usage errors exit 1, deliberately not 2 or 3. Those mean MEDIUM and HIGH findings and callers gate dispatch on them, so a usage error exiting 2 would be read as "medium findings — prompt the user". A test pins that. Tests: 4 written failing first, then fixed. Full suite 7,722 pass / 0 fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(browse): one ambiguous ref no longer kills the whole annotated screenshot `snapshot -a` exits 1 with "Selector matched multiple elements" on most real pages, so /qa, /canary and /land-and-deploy silently produce reports whose screenshots do not exist. Plain `screenshot <path>` is unaffected. Refs are built as getByRole(role, {name}) and disambiguated with .nth() when role+name repeats. That disambiguation cannot fire for a node with NO accessible name: the locator degrades to getByRole(role) with no name filter, and the count driving .nth() is taken from the FILTERED aria snapshot while getByRole matches the unfiltered DOM. Measured on a live page: the tree surfaced 2 unnamed paragraphs, the DOM had 9. Landmarks (banner/main/contentinfo) and paragraphs are correctly unnamed per ARIA, so this is the common case rather than an edge case. boundingBox() then hits Playwright strict mode, and the catch allowlisted only timeout/closed/Target/Execution-context messages — so the strict-mode error was re-thrown and aborted every remaining annotation. Two changes: - `.first()` before boundingBox(), so an ambiguous ref draws a box on its first match instead of aborting. The heatmap path below has always tolerated this via a bare `catch {}`; annotate was the only path that could be killed outright. - the catch no longer re-throws on unrecognised messages. A box we cannot measure is a box we do not draw, never a reason to lose the rest of the page. Set BROWSE_DEBUG to see what was skipped. Also: `-o` passed without `-a`/`-H` was silently ignored (exit 0, no file), which reads as "screenshots are broken" rather than "you forgot a flag". It now warns and points at `browse screenshot <path>`. Verified by rebuilding both ways against the same page with 51 refs present: before — "Selector matched multiple elements", no file written after — exit 0, 229KB PNG Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(version-bump): missing or empty VERSION no longer repairs a fabricated 0.0.0.0 into package.json repair now fails with exit 2 when the VERSION file is absent or empty instead of folding to DEFAULT ("0.0.0.0") — which passed VERSION_RE and regressed package.json below where it started. classify gains an additive versionFileExists field so /ship can tell a real 0.0.0.0 from a fabricated one. Re-derived from PR #2612 under the generated-file screening rule. Fixes #2600 (repair half; the path-configurability half landed in v1.67 via #2531). Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(memory-ingest): --probe counts post-attribution, through the same gate --bulk uses probeMode previously stat'd every walked file, so setup-gbrain gated its silent bulk ingest on pre-filter counts that the write path would never ingest (#2394). The attribution decision now lives in ONE shared gate (sessionIsAttributable — cheap-parse: cwd extraction + memoized resolveGitRemote, never a full page build) used by BOTH probeMode and preparePages, so the two stages' post-attribution counts are structurally identical. ProbeReport gains skipped_unattributed; the probe prints what it excluded and --include-unattributed restores raw counts. The parity is pinned at the prepare stage (probe post-attribution == transcripts reaching import), deliberately NOT == final written. Re-derived from PR #2612 under the generated-file screening rule; the shared-gate design and the remote memo are additions from the plan review. Fixes #2394. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(browse): allow CPU and network throttling for performance measurement Adds Emulation.setCPUThrottlingRate and Network.emulateNetworkConditions to CDP_ALLOWLIST. Motivation: diagnosing a real "uploads take 1-2 minutes" report, the only machine available was a fast developer workstation. Client-side processing measured 1.4s where the user experienced minutes, so the conclusion had to be reached arithmetically rather than observed. Throttling would have let the measurement reproduce the reporter's conditions directly. Both fit the existing posture rather than widening it: - Emulation already allows setDeviceMetricsOverride, clearDeviceMetricsOverride and setUserAgentOverride, which are equally mutating and scoped to the tab. - Neither method reads page content. setCPUThrottlingRate affects only timing; emulateNetworkConditions constrains traffic rather than inspecting it, so no request bodies, headers or cookies are exposed. Both are output: 'trusted' because they return no page-derived data. scope 'tab' for both, matching the surrounding Emulation entries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(session-update): lock pidfile records the live holder; hard TTL bounds every wedge (#2613) echo $$ inside the backgrounded subshell recorded the PARENT hook's PID — which exits immediately — so every subsequent session judged the lock stale and rm -rf'd a LIVE holder's lock, letting concurrent updaters run over each other. The pidfile now records ${BASHPID:-$(sh -c 'echo $PPID')} (macOS bash 3.2 has no BASHPID; the sh child's PPID is exactly this subshell). Staleness is now two independent detectors: PID liveness (as before, but against the real holder), and a 30-minute hard TTL on the heartbeat mtime — reclaimed regardless of kill -0, so a recycled PID or hung holder can't wedge the lock forever. The holder touches the pidfile after the pull and after setup, so a legitimately-slow run keeps itself alive. Empty and missing pidfiles are respected inside the TTL window (the mkdir→echo race) and reclaimed past it. Fixes #2613. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(browse): explicit windowsHide on every Bun.spawn site + census tripwire (#2575 residual) Bun.spawn sites were structurally outside the windowsHide census (it swept child_process bindings only). The runtime was already safe — native Bun hides consoles by default and bun-polyfill.cjs defaults windowsHide !== false since #2523/#2539 — but implicit defaults are exactly what regress silently. Every Bun.spawn/spawnSync in browse/src now carries the explicit flag (harmless on unix-only sites like Xvfb/xattr/open), and a second SWEEP in windows-spawn-hide.test.ts fails CI on any new flagless Bun.spawn site. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gbrain): brain worktree advances on the daily sync — no more silently stale brains (#2516) The daily pull refreshed only ~/.gstack itself, never the detached worktree at ~/.gstack-brain-worktree that gbrain actually indexes — so after setup the brain served stale pages forever unless setup-gbrain/sync-gbrain happened to run. brain-sync --once now advances the worktree once per 24h behind an ATTEMPT stamp (.brain-worktree-last-advance — a persistently-failing advance warns once a day, not at every skill boundary), inside the existing run lock and before any ingest step touches the worktree. The new gstack-gbrain-source-wireup --advance-only is built for the unattended cadence: git-only (no gbrain prereqs), pins every operation to the managed worktree (refuses paths that are not worktrees of the artifacts repo), refuses dirty worktrees, and never runs the force-remove recovery — a cron path must not be able to delete local changes. A static pin keeps the force-remove out. docs/gbrain-sync.md stops overclaiming the old cadence. Fixes #2516. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(memory-ingest): honor the per-remote deny/read-only trust policy (#2392) Transcript ingest now respects the same trust store as code import — the gate existed only in gstack-gbrain-sync's runCodeImport, so memory-ingest happily ingested transcripts from deny-listed repos. preparePages filters prepared transcript pages through ONE batch policy lookup (new 'get --batch' verb on bin/gstack-gbrain-repo-policy — the script owns URL normalization; the client adds repoPolicyTierBatch, one spawn for all distinct remotes, so large corpora never pay a 10s-timeout subprocess per remote). Outcomes match code-import semantics: read-only → clean skip (skipped_policy_readonly), deny → counted refusal (skipped_policy_deny), corrupted/unreadable store → HARD ERROR before any write (state, staging, egress receipt, and import all untouched) with the recovery command named — policy corruption must never read as successful ingestion. Artifacts are never policy-filtered (their git_remote is a project slug, not a remote). Fixes #2392. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(config): repo_mode keeps its empty no-default semantics (#2611 follow-up) The ported defaults table synthesized repo_mode → "unknown", but EMPTY is load-bearing for that key: gstack-repo-mode treats any non-empty answer as a user override and skips its own repo classification — the synthesized default turned the classifier into dead code (REPO_MODE=unknown everywhere; caught by test/gstack-repo-mode.test.ts via the wave's cross-agent blame protocol). repo_mode joins the empty-is-real carve-outs (empty output, exit 0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pair-agent): consent before killing a healthy headless daemon The pair-agent headed switch spawned 'connect --force-restart' unconditionally — auto-killing a live headless daemon (open tabs, cookies, logins) in direct contradiction of the iron rule it sits beside ('only an explicit --force-restart may kill a live daemon'). The CLI now captures daemon liveness BEFORE ensureServer (which can itself boot a fresh daemon) and relaunches only when the user passed --force-restart to pair-agent; otherwise it prints the tab count and continues against the existing daemon. The /pair-agent skill gains a matching one-way-door consent question (template half rides the wave's template block). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gbrain-status): MCP scoping is per-project, and project-local beats user scope hasRemoteOnlyGbrainMcp scanned EVERY project's mcpServers in ~/.claude.json, so one project's remote gbrain registration reclassified broken local engines as thin-client machine-wide. It now reads user scope plus only the cwd's nearest-ancestor project key. The precedence itself was verified empirically and hermetically (fake HOME + CLAUDE_CONFIG_DIR fixtures, claude 2.1.233): with both scopes defining gbrain, 'claude mcp get gbrain' reports Scope: Local config — PROJECT-LOCAL WINS. Both in-repo consumers assumed the opposite; brain-cache's endpoint resolution flips to nearest-ancestor-project-first, and the stale user-first pin in brain-cache-roundtrip now pins the verified precedence. (The user-first jq in the brain-sync preamble resolver gets the same swap in the template block.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(slug): gstack-slug matches remote-slug's owner-repo canonical form (live misfile bug) Found live during this wave's CEO review: bin/gstack-slug emitted SLUG=garrytan for this garrytan/gstack worktree while remote-slug correctly gave garrytan-gstack — decisions, timeline, ceo-plans, and learnings were filing into the wrong project store (observed polluting Context Recovery with another repo's decisions). Root cause: a stray empty ~/.git directory made the walk-up crown $HOME as the outermost project root; the remote lookup ran only against that root, failed silently, and the basename fallback cached 'garrytan' sticky. NOT worktree-specific — any strong marker on a non-repo ancestor triggered it. Fix: the walk now finds the outermost ancestor whose .git actually resolves an origin remote and derives owner-repo with remote-slug's byte-identical parse; marker-only ancestors keep anchoring the basename fallback but can no longer shadow a real remote. A new cache self-heal recomputes the poisoned shape (cached == basename of a marker root while a remote-bearing repo exists below), preserving legit #2212 stickiness. Nested-repo walk-up, no-remote and non-git fallbacks, and the SLUG=/BRANCH= eval contract are unchanged, pinned by a 10-case parity suite. Store migration for pre-fix data is tracked in TODOS.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(brain-sync): per-record spool dir — the enqueue/drain race dies structurally Producers appended lines to .brain-queue.jsonl while the drain re-read and os.replace'd it; the in-code comment admitted a lockless append between the re-read and the replace was lost. Locks and rename-rotation designs were both reviewed and rejected (each retained a tail race); the shipped design is a maildir-style spool: one FILE per record in .brain-queue.d/ (tmp + atomic rename), the drain snapshots filenames, processes, and deletes exactly what it snapshotted. Writer and drainer never share an inode — nothing to race. Semantics: at-least-once (a crash between process and unlink re-drains; downstream content-hash dedup absorbs duplicates); retained (privacy-held) records keep their files; unparseable records are kept + warned, never destroyed. Legacy .brain-queue.jsonl migrates atomically on the next drain (crash-leftover .migrating files recovered too); status/drop-queue count both surfaces; discover-new writes spool records and advances its cursor per-record-written. The preamble's queue-depth line switches to spool count in this wave's template block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bin-context): native slug fallback walks up like bash gstack-slug slugFromEnvironment derived the slug from the INNERMOST repo's origin while bash gstack-slug walks to the outermost project root — nested/vendored repos split their stores across the bash/native boundary (win32 hits the native path constantly). The native fallback now ports _outermost_project_root faithfully (strong/weak markers, outermost-strong-wins, 64-depth cap, fixed-point termination) plus the full resolution order: env override → walk-up → sticky cache with the #1125 self-heal → remote get-url → basename. Twelve mirrored scenarios drive BOTH implementations against the same fixtures and pin identical slugs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(next-version): git fallback queries the live remote, never mutates, and keeps 3-digit width The degraded path counted every remote-tracking ref on every remote — stale experiment branches and second remotes inflated version allocation, and a failed base read flipped 3-digit repos to 4-digit slots. Now: ls-remote --heads origin first (GIT_TERMINAL_PROMPT=0, 5s timeout, zero local ref mutation); on failure, local refs/remotes/origin ONLY with an explicit stale-refs warning; a failed base read zeroes at the LOCAL version file's width so a 3-digit repo allocates 0.0.1, not 0.0.1.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(setup): hooks register the global-install path and re-point stale ones Registering hooks from a dev worktree baked that worktree's absolute path into settings.json — deleting the worktree left a dead hook erroring on every session stop, and the presence-only dedup (list-sources | grep) could never re-point it. setup's hook paths now route through _hook_install_path (global install preferred, source dir fallback), and the new ensure-event verb on gstack-settings-hook compares the registered command payload against canonical: identical → no write, different → single atomic replacement (never zero or two registrations). The plan-tune hooks had the same stale pattern and get the same fix without re-triggering their consent prompt. Also hardened: bun 1.3.13 turns an uncaught sync fs error in bun -e into a SILENT exit 0 — the registrar's write path now catches, prints, and exits 1, so a failed update can never report fake-green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(preamble): learnings capture is unconditional at completion (#2402) 43 of 44 learnings entries came from explicit /learn — the completion-status prose read 'if you discovered a durable project quirk... log it', which models treated as optional. The step now ALWAYS runs: review the session for durable learnings, log each one, and state 'No durable learnings this session' explicitly when the review comes up empty — an empty result, never a skipped step. Re-derived from PR #2612 under the generated-file screening rule. Fixes #2402. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scrape): untrusted-content warning on the page-fetching skills (#2441) /scrape and /skillify consumed page content with zero injection guidance — the CHANGELOG claimed coverage the skills didn't have. The warning now lives in ONE exported const (UNTRUSTED_CONTENT_WARNING in resolvers/browse.ts), embedded in the browse COMMAND_REFERENCE as before AND injected standalone into both skills via the new {{UNTRUSTED_CONTENT_WARNING}} token — single source, wording can never drift between surfaces. Re-derived from PR #2612 under the generated-file screening rule. (Structural isolation for skillify-generated code is tracked as its own TODO.) Fixes #2441. Contributed by @Lockyer228 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): checklist paths resolve from the installed skill root (#2518) /review Step 2 read .claude/skills/review/checklist.md — a path relative to the TARGET repo, which only resolves in gstack's own checkout. Every checklist/greptile-triage/TODOS-format reference (six across five templates — two more than the issue named, same class) now uses the installed-root form ~/.claude/skills/gstack/review/... that the templates' other references already use. The install-root class itself (non-default install dirs) is #1882, deliberately its own PR. Fixes #2518. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pair-agent): one-way-door consent question before a daemon relaunch (template half) The skill flow now checks daemon liveness before Step 4 and asks an explicit one-way-door question (tabs/cookies/logins are lost) before passing --force-restart — never proceeding on a vague reply. Pairs with the CLI-half commit that stopped pair-agent auto-killing live daemons. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(codex): resume does not amortize the ~21K session prelude (#2387) Measured (#2387): every codex exec call pays Codex's session prelude, and a resumed call came in slightly ABOVE a fresh one — resume buys continuity, never token savings. The skill now says so where the resume flow lives: prefer one codex call per skill, batch questions into it. Fixes #2387. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(upgrade): fast-forward first; reset --hard only behind a proved-safe gate (#2517) /gstack-upgrade went straight to stash + reset --hard origin/main. Now it tries git pull --ff-only --autostash first (the same policy session-update's auto-upgrade uses). The destructive fallback runs unprompted ONLY when both git status --porcelain AND git rev-list origin/main..HEAD are empty — a clean tree with unpushed local commits is NOT safe, reset destroys them. Anything else requires an explicit one-way-door confirmation that lists every dirty file and unpushed commit being discarded. Fixes #2517. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(preamble): brain-sync block counts the spool queue and resolves MCP project-first Two resolver halves deferred from earlier wave commits: the queue-depth line counts .brain-queue.d/*.json spool records (plus legacy lines until the drain migrates them), and GBRAIN_MCP_ENTRY_JQ swaps its operands to nearest-ancestor-project-first — matching the empirically verified Claude Code precedence (project-local beats user scope) instead of the backwards user-first assumption. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate SKILL.md docs + golden fixtures (single regen for the template block) Pure generator output for the six template/resolver commits above (learnings capture, untrusted-content warning, review paths, pair-agent consent, codex resume note, upgrade ff-only, brain-sync block) — bun run gen:skill-docs + --host codex + --host factory, with the three ship golden fixtures refreshed per the documented procedure. The three sidecar-path pins in gen-skill-docs.test.ts move to the new installed-root/$GSTACK_ROOT contract (#2518). Restores template freshness; full suite green from here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: TODOS.md — strike the six wave-fixed residuals, add two follow-ups The v1.67 adversarial-review residuals section shrinks to the one item the wave couldn't reach (iOS tap routing — needs real-device verification). New entries: skillify structural isolation (a prose warning is not a boundary for page-derived generated code) and the slug store migration (pre-fix sessions on stray-marker machines filed data under the degraded slug; post-fix reads go to the correct store, so history needs a merge/alias). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: align cross-cutting pins with the wave's contracts Three suites pinned pre-wave behavior: browse's gstack-config test asserted the old unknown-key ''/exit-0 shape (#2611 made it exit 1); the Windows-paths suite pinned O_APPEND enqueue atomicity (the spool design satisfies the same invariant via tmp + os.replace, one file per record — pinned in its new form); and nine carve-guard skeleton ceilings absorbed the #2402 unconditional-learnings prose (~450B per skill), bumped with measured values per the guard's own protocol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: re-anchor the referenced-path scanner self-check to the gstack-rooted review refs The self-check pinned the review checklist as a class-1 alias-relative ref; #2518 moved those refs to the installed gstack root (class 2). The guard now proves the scanner sees them in their new class, so the class-2 assertion can't go vacuous. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: pin the wave's prose-tier behaviors (ship coverage-audit gap closure) The coverage audit found one regression-shaped gap: nothing pinned that the upgrade template's ff-only pull precedes the gated reset --hard (#2517) — a future template edit reverting to reset-first would fail nothing. Pinned: the ordering, the FF_OK gate, and the unpushed-commits check. Also pinned the two minor gaps: the {{UNTRUSTED_CONTENT_WARNING}} injection points in scrape/skillify (#2441) and brain-uninstall's spool-dir cleanup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: pre-landing review round — 8 auto-fixes + 8 accepted findings hardened The ship review army (4 specialists + red-team + checklist, 29 findings) produced 8 mechanical auto-fixes and 11 decisions; the accepted set: - win32 slug parity completed: lib/bin-context.ts gains the remote-first outermost walk + degraded-cache self-heal the bash side got this wave — the two implementations now agree on the stray-marker live-bug shape, pinned by shared fixtures (multi-specialist 9/10 finding). - probe honors the plan's bounded-read decision: 256KB prefix, extraction semantics mirrored from parseTranscriptJsonl so probe/prepare can never diverge on the same file (>1MB transcript test). - policy normalize parity: bash normalize() now matches canonicalizeRemote on .git/-trailing and uppercase-.GIT shapes (7-shape corpus pinned two ways) — a deny for those shapes could previously slip the transcript gate. - session-update reclaim is TOCTOU-safe (atomic mv-aside on both branches). - settings-hook: unparseable settings.json errors instead of being replaced with {}; ensure-event keys on (event, source) so matcher changes update in place — never zero or two registrations. - dot-only slug guard at both parse sites (hostile 'url = ..' can't escape projects/); enqueue tmp-file janitor (1h TTL, inside the drain lock); brain-sync .migrating never clobbered; drop-queue/status count .migrating; snapshot -o warning correct + surfaced in diff mode; version-bump test order-dependence removed; uninstall clears the advance stamp. Deferred with record: slug heal-probe cost sentinel (P3 TODO), FF_OK conflation (noted, misdiagnosis-only). 270 pass / 0 fail across the 10 touched suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: adversarial round — the P0 finalize fail-safe and 12 hardened findings Three adversarial passes (Claude fresh-context, Codex chaos, Codex structured with P1 gate) on the full wave diff. Multi-source findings, all fixed: - P0: finalize_queue is now explicit-delete-only — a record is unlinked ONLY when classification proves it staged or dropped; a classifier crash, a missing class file, or a malformed pulled .brain-privacy-map.json (which previously nuked the whole snapshotted queue, remotely triggerable) now retains everything, warns, and re-drains next run. load_privacy_map treats corrupt maps as retain-all, never as empty. - next-version cannot silently drop a live claim: unreadable advertised refs get a targeted --depth=1 fetch + retry; still-unreadable claims surface as UNKNOWN warnings instead of duplicate-version silence. - session-update lock: ownership-checked EXIT trap (a TTL-reclaimed holder can no longer delete the new holder's lock) + a 5-min background heartbeat so a legitimately-slow pull/setup is never reclaimed while alive. - ensure-event collapses ALL same-(event,source) duplicates to one canonical entry; unique per-process tmp path; setup call sites surface (not swallow) the hardened refusals. - memory-ingest: --limit counts only policy-permitted pages (denied records no longer starve permitted ones); --probe applies the same policy filter as --bulk (skipped_policy_* fields on the report). - version-bump repair accepts a genuine literal 0.0.0.0 VERSION file. - slug heal restricted to the stray-.git shape — package.json-anchored wrapper roots keep their legit sticky identity (#2212 preserved). - brain-sync: idle fast path sees leftover .migrating records; unparseable spool records quarantine instead of warning forever; migration comment stops overclaiming the transition-window race. - CDP throttling justifications document override persistence (callers own restoration), pinned in the allowlist test. Deferred with record: deny retroactivity for already-ingested pages (P2 TODO, same semantics as the code-import gate); legacy-migration tail race (transition-window, requires pre-spool writers). 288 pass / 0 fail across the 10 touched suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate SKILL.md docs + goldens (Windows-separator jq fix) Pure generator output for the brain-sync block's jq ancestor match now accepting backslash-formed Windows project keys — previously project-scoped brains were invisible on Windows while the TS scope resolvers saw them. Golden ship fixtures refreshed per the documented procedure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: codex verify-pass residuals — chunked cwd read, post-filter partial count, migrating depth The verify re-review passed the P1 gate (0 P1s) and left three residuals, all applied: transcriptCwdFromPrefix reads in chunks until one complete record (4MB cap) so a giant first prompt can't truncate mid-JSON and break probe/bulk parity; partial_pages derives from the FINAL prepared set instead of the whole scanned corpus; the preamble queue-depth line counts leftover .brain-queue.jsonl.migrating records like the status path does (regen + goldens included). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: bump version and changelog (v1.68.0.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: update project documentation for v1.68.0.0 BROWSER.md: fix the $B cdp example (positional JSON params, not --json; depth is the real CDP param) and add the new perf-throttling examples (Emulation.setCPUThrottlingRate, Network.emulateNetworkConditions) with their clear-override counterparts. USING_GBRAIN_WITH_GSTACK.md: the state-files table row for the sync queue now names the maildir-style spool dir .brain-queue.d/ that replaced .brain-queue.jsonl this release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: align memory-pipeline probe pins with the #2394 stage-count contract The paid-tier E2E pinned the pre-fix contract (probe headline = raw discovered). Probe now counts post-attribution — the same gate --bulk uses — with an explicit unattributed-skip line. Adds the --include-unattributed companion pin so all 9 fixtures stay accounted for. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(next-version): batch missing-tip fetches — one bounded round trip, never a per-branch crawl The targeted-fetch retry for branches whose advertised tip has no local object ran ONE git fetch per branch (10s cap each). On a shallow clone against a busy remote that crawls the network for minutes — CI's shard deadline killed the free suite mid-file. Missing tips now collect into a single batched shallow fetch (15s cap); refs still missing after the batch (one unservable ref fails the whole transfer) get a capped per-branch retry, and anything past the cap warns as an UNKNOWN claim instead of fetching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(next-version): pin the batched fetch + make the offline-contract tests hermetic Two new G2 pins: N unfetched claim branches resolve with exactly ONE fetch spawn (PATH-shimmed git counts invocations), and one unservable ref no longer poisons the batch — live claims resolve via the bounded retry while only the ghost warns UNKNOWN. The #2545 offline-contract tests now run the CLI in a local fixture repo instead of the repo's own checkout: the checkout path did a live ls-remote against the real origin (operator-network-dependent, and the CI shard-deadline hang). The online-contract test gains a succeeding gh stub, so fallback:null is asserted deterministically instead of only when the operator happens to be authed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(redact-cli): derive the synthetic AWS-key fixture — no contiguous credential literal in source The CI quality gate scans every ADDED diff line with the redact engine, so the #2610 port's raw fixture literals failed the very gate they exist to test. The fixture is now assembled at runtime; the scanner still receives the identical bytes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(next-version): pin the fixture's host via origin-URL sniff — kills the last environment dependence The hermetic offline-contract fixture had no origin remote, so detectHost() fell through to auth probes: a machine with glab authed passed via the gitlab path while a bare CI runner read host:unknown (offline stays false there) and failed. The fixture now pushes to a local bare origin at a path containing github.com — the URL sniff pins host:github identically everywhere, asserted explicitly in both tests, with every git call still local. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: y$un_ <forrest.sun527@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: benjamin beres <benjamin.beres@bienpreter.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Ricky <ricky@kinokostudio.com.hk> Co-authored-by: Connex Client Access <paul@paulkortman.com> Co-authored-by: henbima <henbima@gmail.com>
This commit is contained in:
co-authored by
Claude Fable 5
y$un_
Cursor
benjamin beres
Ricky
Connex Client Access
henbima
parent
60e51342b5
commit
28d59ad56c
+30
-23
@@ -127,15 +127,19 @@ function sha8(input: string): string {
|
||||
* stable identity hash. Used to detect when the user switches brains
|
||||
* (different endpoint → different cache).
|
||||
*
|
||||
* Reads BOTH registration scopes in ~/.claude.json (#2499): user scope
|
||||
* (.mcpServers.gbrain) first, then project scope
|
||||
* Reads BOTH registration scopes in ~/.claude.json (#2499): project scope
|
||||
* (.projects["/abs/path"].mcpServers.gbrain — what `claude mcp add`
|
||||
* WITHOUT --scope user writes), preferring the nearest ancestor of cwd
|
||||
* (longest matching project key) so nested repos resolve to their own
|
||||
* brain. Before the project-scope read, two different project-scoped
|
||||
* brains both hashed to 'local', so switching between them never
|
||||
* invalidated the cache — the exact scenario this function exists to
|
||||
* catch.
|
||||
* WITHOUT --scope user writes) first, preferring the nearest ancestor of
|
||||
* cwd (longest matching project key) so nested repos resolve to their own
|
||||
* brain, then user scope (.mcpServers.gbrain) as the fallback. That order
|
||||
* is Claude Code's own name-conflict precedence (local beats user) —
|
||||
* verified empirically against claude 2.1.233 with a hermetic fake $HOME:
|
||||
* `claude mcp get gbrain` reports "Scope: Local config" and the
|
||||
* project-local URL when both scopes define the name — so the hash tracks
|
||||
* the endpoint the project actually talks to. Before the project-scope
|
||||
* read, two different project-scoped brains both hashed to 'local', so
|
||||
* switching between them never invalidated the cache — the exact scenario
|
||||
* this function exists to catch.
|
||||
*
|
||||
* Params exist for tests; production callers use the defaults.
|
||||
*/
|
||||
@@ -163,9 +167,11 @@ interface McpEntryish {
|
||||
}
|
||||
|
||||
/**
|
||||
* User-scope gbrain entry, else the nearest-ancestor project-scope entry
|
||||
* for cwd (#2499). Path-boundary-aware: /a/repo never matches /a/repo2.
|
||||
* Both separators are accepted so Windows project keys resolve.
|
||||
* Nearest-ancestor project-scope gbrain entry for cwd, else the user-scope
|
||||
* entry (#2499). Project-local first — Claude Code's own precedence for a
|
||||
* same-name conflict (see detectEndpointHash's docstring for the empirical
|
||||
* evidence). Path-boundary-aware: /a/repo never matches /a/repo2. Both
|
||||
* separators are accepted so Windows project keys resolve.
|
||||
*/
|
||||
function resolveGbrainMcpEntry(
|
||||
cfg: unknown,
|
||||
@@ -175,20 +181,21 @@ function resolveGbrainMcpEntry(
|
||||
mcpServers?: Record<string, McpEntryish>;
|
||||
projects?: Record<string, { mcpServers?: Record<string, McpEntryish> }>;
|
||||
} | null;
|
||||
if (root?.mcpServers?.gbrain) return root.mcpServers.gbrain;
|
||||
const projects = root?.projects;
|
||||
if (!projects || typeof projects !== 'object') return undefined;
|
||||
let best: { key: string; entry: McpEntryish } | undefined;
|
||||
for (const [key, val] of Object.entries(projects)) {
|
||||
if (!val || typeof val !== 'object') continue;
|
||||
const entry = val.mcpServers?.gbrain;
|
||||
if (!entry || typeof entry !== 'object') continue;
|
||||
const isAncestor =
|
||||
cwd === key || cwd.startsWith(`${key}/`) || cwd.startsWith(`${key}\\`);
|
||||
if (!isAncestor) continue;
|
||||
if (!best || key.length > best.key.length) best = { key, entry };
|
||||
if (projects && typeof projects === 'object') {
|
||||
let best: { key: string; entry: McpEntryish } | undefined;
|
||||
for (const [key, val] of Object.entries(projects)) {
|
||||
if (!val || typeof val !== 'object') continue;
|
||||
const entry = val.mcpServers?.gbrain;
|
||||
if (!entry || typeof entry !== 'object') continue;
|
||||
const isAncestor =
|
||||
cwd === key || cwd.startsWith(`${key}/`) || cwd.startsWith(`${key}\\`);
|
||||
if (!isAncestor) continue;
|
||||
if (!best || key.length > best.key.length) best = { key, entry };
|
||||
}
|
||||
if (best) return best.entry;
|
||||
}
|
||||
return best?.entry;
|
||||
return root?.mcpServers?.gbrain;
|
||||
}
|
||||
|
||||
// ──────────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
# gstack-brain-enqueue — atomically append a path to the GBrain sync queue.
|
||||
# gstack-brain-enqueue — write a path record into the GBrain sync spool.
|
||||
#
|
||||
# Usage:
|
||||
# gstack-brain-enqueue <file-path>
|
||||
#
|
||||
# Called by writer scripts (gstack-learnings-log, gstack-timeline-log, etc.)
|
||||
# after their local write. Fire-and-forget; failures are silent (never blocks
|
||||
# the writer). Queue is drained by `gstack-brain-sync --once` invoked from the
|
||||
# preamble at skill START and END boundaries.
|
||||
# the writer). The spool is drained by `gstack-brain-sync --once` invoked from
|
||||
# the preamble at skill START and END boundaries.
|
||||
#
|
||||
# No-op when:
|
||||
# - artifacts_sync_mode is off (the default)
|
||||
@@ -18,8 +18,12 @@
|
||||
# GSTACK_HOME — override ~/.gstack state directory (aligns with writers).
|
||||
# Tests use GSTACK_HOME=/tmp/test-$$ for isolation.
|
||||
#
|
||||
# Concurrency: POSIX append is atomic up to PIPE_BUF (~4KB Linux, 512 BSD).
|
||||
# Queue lines are ~200 bytes, safe under concurrent callers.
|
||||
# Concurrency: maildir-style spool — one FILE per record under
|
||||
# .brain-queue.d/, created via tmp-file + atomic rename. Writer and drainer
|
||||
# never share an inode, so there is no append/rewrite race by construction
|
||||
# (the legacy single-file .brain-queue.jsonl append could race the drain's
|
||||
# rewrite). Filenames are <epoch>-<pid>-<uniq>.json, so a sorted listing is
|
||||
# chronological.
|
||||
|
||||
# No `-e` — writer shims rely on this never failing loudly.
|
||||
set -uo pipefail
|
||||
@@ -28,7 +32,7 @@ FILE="${1:-}"
|
||||
[ -z "$FILE" ] && exit 0
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
QUEUE="$GSTACK_HOME/.brain-queue.jsonl"
|
||||
SPOOL="$GSTACK_HOME/.brain-queue.d"
|
||||
SKIP_FILE="$GSTACK_HOME/.brain-skip.txt"
|
||||
|
||||
# Fast exits: no git repo, no sync.
|
||||
@@ -50,6 +54,11 @@ fi
|
||||
ESC_FILE=$(printf '%s' "$FILE" | sed 's/\\/\\\\/g; s/"/\\"/g')
|
||||
TS=$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || echo "")
|
||||
|
||||
printf '{"file":"%s","ts":"%s"}\n' "$ESC_FILE" "$TS" >> "$QUEUE" 2>/dev/null
|
||||
# One spool file per record: tmp write + atomic rename. Any failure exits 0
|
||||
# silently (fire-and-forget contract), cleaning up the tmp file.
|
||||
mkdir -p "$SPOOL" 2>/dev/null || exit 0
|
||||
TMP="$SPOOL/.tmp-$$-$RANDOM"
|
||||
printf '{"file":"%s","ts":"%s"}\n' "$ESC_FILE" "$TS" > "$TMP" 2>/dev/null || { rm -f "$TMP" 2>/dev/null; exit 0; }
|
||||
mv -f "$TMP" "$SPOOL/$(date +%s)-$$-$RANDOM.json" 2>/dev/null || rm -f "$TMP" 2>/dev/null
|
||||
|
||||
exit 0
|
||||
|
||||
+347
-112
@@ -20,6 +20,13 @@
|
||||
set -uo pipefail
|
||||
|
||||
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
|
||||
# Maildir-style spool: one FILE per record, <epoch>-<pid>-<uniq>.json.
|
||||
# Writers (gstack-brain-enqueue, --discover-new) create records via tmp-file
|
||||
# + atomic rename; the drain deletes exactly the files it snapshotted. No
|
||||
# shared inode between writer and drainer → no append/rewrite race.
|
||||
QUEUE_DIR="$GSTACK_HOME/.brain-queue.d"
|
||||
# Legacy single-file queue: kept ONLY for migration. Pre-spool writers
|
||||
# appended lines here; migrate_legacy_queue converts them to spool files.
|
||||
QUEUE="$GSTACK_HOME/.brain-queue.jsonl"
|
||||
ALLOWLIST="$GSTACK_HOME/.brain-allowlist"
|
||||
PRIVACY_MAP="$GSTACK_HOME/.brain-privacy-map.json"
|
||||
@@ -120,7 +127,91 @@ sys.exit(0)
|
||||
"
|
||||
}
|
||||
|
||||
# Compute matched allowlisted, privacy-filtered path set from queue.
|
||||
# True (0) if the spool holds at least one record file.
|
||||
spool_has_records() {
|
||||
local f
|
||||
for f in "$QUEUE_DIR"/*.json; do
|
||||
[ -e "$f" ] && return 0
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# Convert one legacy queue file's lines into spool record files (tmp +
|
||||
# os.replace, one file per line). Reads the file TWICE before unlinking: a
|
||||
# pre-rename writer can still append through its already-open fd after our
|
||||
# rename, and those appends land in the renamed file — the second pass
|
||||
# NARROWS the tail-race window (transition-only: it applies to pre-spool
|
||||
# writers, and a writer that appends after the second read but before the
|
||||
# unlink can still lose that line; spool-native writers are immune).
|
||||
# Unparseable lines migrate as-is; finalize_queue quarantines + warns on them.
|
||||
convert_legacy_file() {
|
||||
local legacy="$1"
|
||||
python3 - "$legacy" "$QUEUE_DIR" <<'PYEOF' 2>/dev/null || true
|
||||
import os, sys, time
|
||||
|
||||
legacy, spool = sys.argv[1:3]
|
||||
|
||||
def read_lines(path):
|
||||
try:
|
||||
with open(path) as f:
|
||||
return [l.rstrip("\r\n") for l in f if l.strip()]
|
||||
except (FileNotFoundError, OSError):
|
||||
return []
|
||||
|
||||
seq = 0
|
||||
def write_spool(line):
|
||||
global seq
|
||||
seq += 1
|
||||
tmp = os.path.join(spool, f".tmp-{os.getpid()}-m{seq}")
|
||||
with open(tmp, "w") as f:
|
||||
f.write(line + "\n")
|
||||
os.replace(tmp, os.path.join(spool, f"{int(time.time())}-{os.getpid()}-m{seq}.json"))
|
||||
|
||||
written = set()
|
||||
for _pass in (1, 2): # second read closes the pre-rename-fd tail race
|
||||
for line in read_lines(legacy):
|
||||
if line not in written: # identical duplicates collapse, as the old rewrite did
|
||||
write_spool(line)
|
||||
written.add(line)
|
||||
os.unlink(legacy)
|
||||
PYEOF
|
||||
}
|
||||
|
||||
# Legacy migration (transition window only). If the single-file queue holds
|
||||
# records, atomically rename it aside and convert each line to a spool file.
|
||||
# A concurrent OLD writer that recreates a fresh legacy file after the rename
|
||||
# simply gets migrated on the NEXT drain — nothing is lost, only deferred one
|
||||
# boundary. Runs inside the run lock, before the drain reads the spool.
|
||||
migrate_legacy_queue() {
|
||||
local migrating="$QUEUE.migrating"
|
||||
# Crash leftover: a prior migration renamed but died before unlink. Some of
|
||||
# its lines may already exist as spool files — re-converting duplicates is
|
||||
# safe (at-least-once; the drain dedups paths per snapshot and downstream
|
||||
# content-hash dedup absorbs re-syncs). Losing the file would not be. If
|
||||
# the conversion itself fails, the file stays for the next run (never rm a
|
||||
# non-empty .migrating file outside convert_legacy_file's own unlink).
|
||||
if [ -f "$migrating" ]; then
|
||||
if [ -s "$migrating" ]; then
|
||||
mkdir -p "$QUEUE_DIR" 2>/dev/null || return 0
|
||||
convert_legacy_file "$migrating"
|
||||
else
|
||||
rm -f "$migrating" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
# If the leftover STILL holds records, the conversion failed (e.g. python3
|
||||
# unavailable). The mv below would overwrite it and destroy those records —
|
||||
# exactly the never-destroy invariant above. Defer this run's migration;
|
||||
# the next run retries both files.
|
||||
[ -s "$migrating" ] && return 0
|
||||
if [ -s "$QUEUE" ]; then
|
||||
mkdir -p "$QUEUE_DIR" 2>/dev/null || return 0
|
||||
mv -f "$QUEUE" "$migrating" 2>/dev/null || return 0
|
||||
convert_legacy_file "$migrating"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# Compute matched allowlisted, privacy-filtered path set from the spool.
|
||||
# Output: newline-delimited relative paths that should be staged.
|
||||
#
|
||||
# #2549: every non-staged queue entry is CLASSIFIED, never silently discarded.
|
||||
@@ -132,13 +223,20 @@ sys.exit(0)
|
||||
# attempt, no allowlist glob, not on disk) and are removed WITH a counted
|
||||
# status — the old behavior truncated the whole queue and reported every one
|
||||
# of these, including privacy holds, as "no allowlisted changes".
|
||||
#
|
||||
# Spool snapshot ($3): the sorted list of spool record filenames read here is
|
||||
# written to the snapshot manifest, one filename per line. finalize_queue
|
||||
# deletes exactly the manifest's files and never touches records created
|
||||
# after this listing — a concurrent enqueue is a separate file by
|
||||
# construction, so it simply rides to the next drain.
|
||||
compute_paths_to_stage() {
|
||||
local mode="$1"
|
||||
local class_file="${2:-}"
|
||||
python3 - "$GSTACK_HOME" "$QUEUE" "$ALLOWLIST" "$PRIVACY_MAP" "$SKIP_FILE" "$mode" "$class_file" <<'PYEOF'
|
||||
local snapshot_file="${3:-}"
|
||||
python3 - "$GSTACK_HOME" "$QUEUE_DIR" "$ALLOWLIST" "$PRIVACY_MAP" "$SKIP_FILE" "$mode" "$class_file" "$snapshot_file" <<'PYEOF'
|
||||
import sys, json, os, fnmatch, glob
|
||||
|
||||
gstack_home, queue, allowlist_path, privacy_path, skip_path, mode, class_file = sys.argv[1:8]
|
||||
gstack_home, spool_dir, allowlist_path, privacy_path, skip_path, mode, class_file, snapshot_file = sys.argv[1:9]
|
||||
|
||||
def load_lines(path):
|
||||
try:
|
||||
@@ -148,39 +246,67 @@ def load_lines(path):
|
||||
return []
|
||||
|
||||
def load_privacy_map(path):
|
||||
# Returns (entries, corrupt). Non-dict entries are filtered out
|
||||
# defensively — the map may be PULLED from the artifacts remote, so a
|
||||
# malformed entry like ["bad"] is remotely triggerable and used to raise
|
||||
# mid-classification (after the snapshot manifest was written), which the
|
||||
# old finalize turned into a full queue wipe. Any malformed shape also
|
||||
# marks the map CORRUPT: privacy classification cannot be trusted, so the
|
||||
# caller holds every queued record instead of guessing (a corrupt privacy
|
||||
# map silently treated as empty would over-share behavioral data).
|
||||
try:
|
||||
with open(path) as f:
|
||||
data = json.load(f)
|
||||
# Expected: [{"pattern": "glob", "class": "artifact" | "behavioral"}]
|
||||
return data if isinstance(data, list) else []
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
return []
|
||||
except FileNotFoundError:
|
||||
return [], False
|
||||
except json.JSONDecodeError:
|
||||
return [], True
|
||||
if not isinstance(data, list):
|
||||
return [], True
|
||||
# Expected: [{"pattern": "glob", "class": "artifact" | "behavioral"}]
|
||||
entries = [e for e in data if isinstance(e, dict)]
|
||||
return entries, len(entries) != len(data)
|
||||
|
||||
allowlist_globs = load_lines(allowlist_path)
|
||||
privacy_map = load_privacy_map(privacy_path)
|
||||
privacy_map, privacy_corrupt = load_privacy_map(privacy_path)
|
||||
# Normalize skip entries to the POSIX form queued paths use, so a backslash
|
||||
# entry in .brain-skip.txt still matches on Windows. The drain is the safety
|
||||
# boundary that actually stages files, so it must normalize identically to
|
||||
# discover_new — otherwise an explicitly-skipped file gets committed.
|
||||
skip_lines = {s.replace(os.sep, "/") for s in load_lines(skip_path)}
|
||||
|
||||
# Read queue; collect unique file paths.
|
||||
queue_paths = set()
|
||||
# Snapshot the spool: sorted (= chronological, filenames are epoch-first)
|
||||
# list of record files at read time. Records that appear after this listing
|
||||
# belong to the NEXT drain. Files we cannot read stay OUT of the manifest so
|
||||
# finalize never deletes a record this drain didn't actually consume.
|
||||
try:
|
||||
with open(queue) as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
obj = json.loads(line)
|
||||
p = obj.get("file")
|
||||
if isinstance(p, str):
|
||||
queue_paths.add(p)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
snapshot = sorted(n for n in os.listdir(spool_dir) if n.endswith(".json"))
|
||||
except (FileNotFoundError, NotADirectoryError):
|
||||
snapshot = []
|
||||
|
||||
queue_paths = set()
|
||||
consumed = []
|
||||
for name in snapshot:
|
||||
try:
|
||||
with open(os.path.join(spool_dir, name)) as f:
|
||||
line = f.readline().strip()
|
||||
except OSError:
|
||||
continue
|
||||
consumed.append(name)
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
obj = json.loads(line)
|
||||
p = obj.get("file")
|
||||
if isinstance(p, str):
|
||||
queue_paths.add(p)
|
||||
except json.JSONDecodeError:
|
||||
continue # unparseable record: finalize keeps + warns
|
||||
|
||||
if snapshot_file:
|
||||
with open(snapshot_file, "w") as f:
|
||||
for name in consumed:
|
||||
f.write(name + "\n")
|
||||
|
||||
def path_matches_any(path, globs):
|
||||
for pattern in globs:
|
||||
@@ -207,6 +333,14 @@ def mode_allows(cls, mode):
|
||||
|
||||
final = []
|
||||
classified = {"retained": [], "dropped": {"skipped": [], "invalid": [], "unmatched": [], "missing": []}}
|
||||
if privacy_corrupt:
|
||||
# Fail-safe: with an untrustworthy privacy map, stage NOTHING and drop
|
||||
# NOTHING — retain every queued record until the map is fixed. The next
|
||||
# drain re-classifies from scratch.
|
||||
print("BRAIN_SYNC: warning: privacy map at " + privacy_path +
|
||||
" is malformed — holding all queued records until it is fixed", file=sys.stderr)
|
||||
classified["retained"] = sorted(queue_paths)
|
||||
queue_paths = set()
|
||||
for p in sorted(queue_paths):
|
||||
if p in skip_lines:
|
||||
classified["dropped"]["skipped"].append(p)
|
||||
@@ -241,22 +375,35 @@ for p in final:
|
||||
PYEOF
|
||||
}
|
||||
|
||||
# #2549: surgical queue rewrite — replaces every whole-queue truncation
|
||||
# (`: > "$QUEUE"`). Re-reads the LIVE queue at rewrite time (a writer may have
|
||||
# enqueued while we were staging/pushing — those entries must survive; the old
|
||||
# truncation destroyed them) and keeps every line whose file is either
|
||||
# retained (privacy/mode-held) or not part of this drain at all. Atomic
|
||||
# tmp+mv in the same directory. Dropped-path detail goes to a 0600 sidecar so
|
||||
# the status line can stay content-free (counts only).
|
||||
rewrite_queue() {
|
||||
local paths_file="$1" # staged (drained) paths, one per line
|
||||
local class_file="$2" # classification JSON from compute_paths_to_stage
|
||||
# Fail-open by design (a failed rewrite self-corrects next run: re-stage →
|
||||
# Finalize the drain: delete exactly the spool record files this drain
|
||||
# consumed (per the snapshot manifest) AND positively classified. Deletion is
|
||||
# EXPLICIT-DELETE-ONLY: a record is unlinked only when its path appears in
|
||||
# (staged paths ∪ classified dropped). The old polarity ("delete unless
|
||||
# retained") turned a missing/unparseable classification into retained=∅ and
|
||||
# wiped every snapshotted record — remotely triggerable via a malformed
|
||||
# pulled privacy map that raised AFTER the manifest write. Now a
|
||||
# missing/unparseable class_file or paths_file deletes NOTHING (warn +
|
||||
# return), and a path the classification never mentions stays queued.
|
||||
# The predecessor (a shared-file queue rewrite) had a lockless-append race
|
||||
# between its live re-read and the os.replace; with one file per record that
|
||||
# race class is structurally gone — a concurrent enqueue is a separate file
|
||||
# the snapshot never listed, so finalize cannot touch it. Crash semantics are
|
||||
# at-least-once: a drain that dies before finalize leaves its spool files in
|
||||
# place and the next run re-drains them; downstream content-hash dedup
|
||||
# absorbs the duplicates. Unparseable records move to $QUEUE_DIR/quarantine/
|
||||
# (never deleted) so they stop re-warning at every boundary. Dropped-path
|
||||
# detail goes to a 0600 sidecar so the status line can stay content-free
|
||||
# (counts only).
|
||||
finalize_queue() {
|
||||
local snapshot_file="$1" # spool filenames this drain consumed, one per line
|
||||
local class_file="$2" # classification JSON from compute_paths_to_stage
|
||||
local paths_file="$3" # staged paths (compute_paths_to_stage stdout), one per line
|
||||
# Fail-open by design (a failed finalize self-corrects next run: re-stage →
|
||||
# nothing-to-commit), but say so — a silent failure here would let the
|
||||
# subsequent "ok/idle" status claim a drain that did not happen.
|
||||
python3 - "$QUEUE" "$paths_file" "$class_file" "$GSTACK_HOME/.brain-sync-drops.json" <<'PYEOF' || echo "BRAIN_SYNC: warning: queue rewrite failed — entries retained; next run re-drains" >&2
|
||||
python3 - "$QUEUE_DIR" "$snapshot_file" "$class_file" "$paths_file" "$GSTACK_HOME/.brain-sync-drops.json" <<'PYEOF' || echo "BRAIN_SYNC: warning: queue finalize failed — entries retained; next run re-drains" >&2
|
||||
import json, os, sys, time
|
||||
queue, paths_file, class_file, drops_file = sys.argv[1:5]
|
||||
spool_dir, snapshot_file, class_file, paths_file, drops_file = sys.argv[1:6]
|
||||
|
||||
def lines(path):
|
||||
try:
|
||||
@@ -265,46 +412,59 @@ def lines(path):
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
|
||||
staged = set(lines(paths_file))
|
||||
# Explicit-delete-only inputs. Either input unreadable → delete NOTHING.
|
||||
try:
|
||||
with open(class_file) as f:
|
||||
classified = json.load(f)
|
||||
if not isinstance(classified, dict):
|
||||
raise ValueError("classification is not an object")
|
||||
except Exception:
|
||||
classified = {"retained": [], "dropped": {}}
|
||||
retained = set(classified.get("retained", []))
|
||||
print("BRAIN_SYNC: warning: classification unreadable — no queue records deleted; next run re-drains", file=sys.stderr)
|
||||
sys.exit(0)
|
||||
try:
|
||||
with open(paths_file) as f:
|
||||
staged = {l.strip() for l in f if l.strip()}
|
||||
except Exception:
|
||||
print("BRAIN_SYNC: warning: staged-paths file unreadable — no queue records deleted; next run re-drains", file=sys.stderr)
|
||||
sys.exit(0)
|
||||
|
||||
dropped = set()
|
||||
for group in (classified.get("dropped", {}) or {}).values():
|
||||
dropped.update(group)
|
||||
processed = staged | dropped
|
||||
deletable = staged | dropped
|
||||
|
||||
kept = []
|
||||
seen_lines = set()
|
||||
unparseable = 0
|
||||
# LIVE re-read narrows (not fully closes) the concurrent-append window: the
|
||||
# lockless enqueue can still land on the old inode between this read and the
|
||||
# os.replace below. Vastly better than the old whole-queue truncation.
|
||||
for line in lines(queue):
|
||||
if line in seen_lines:
|
||||
continue # identical duplicate lines collapse on rewrite
|
||||
for name in lines(snapshot_file):
|
||||
full = os.path.join(spool_dir, name)
|
||||
try:
|
||||
p = json.loads(line).get("file")
|
||||
with open(full) as f:
|
||||
rec = f.readline().strip()
|
||||
except OSError:
|
||||
continue # unreadable now: leave it for the next drain
|
||||
p = None
|
||||
try:
|
||||
p = json.loads(rec).get("file")
|
||||
except Exception:
|
||||
pass
|
||||
if not isinstance(p, str):
|
||||
# Never destroy what we can't read — but don't leave it re-warning at
|
||||
# every boundary either: move it aside for inspection.
|
||||
unparseable += 1
|
||||
kept.append(line) # unparseable line: keep, never destroy
|
||||
seen_lines.add(line)
|
||||
try:
|
||||
qdir = os.path.join(spool_dir, "quarantine")
|
||||
os.makedirs(qdir, exist_ok=True)
|
||||
os.replace(full, os.path.join(qdir, name))
|
||||
except OSError:
|
||||
pass # quarantine move failed — leave in place; next run retries
|
||||
continue
|
||||
if not isinstance(p, str) or p in retained or p not in processed:
|
||||
kept.append(line)
|
||||
seen_lines.add(line)
|
||||
if p not in deletable:
|
||||
continue # retained / unclassified: stays queued (explicit-delete-only)
|
||||
try:
|
||||
os.unlink(full) # staged or dropped: fully processed
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
if unparseable:
|
||||
import sys as _sys
|
||||
print(f"BRAIN_SYNC: {unparseable} unparseable queue line(s) held (inspect {queue})", file=_sys.stderr)
|
||||
|
||||
tmp = queue + ".tmp." + str(os.getpid())
|
||||
with open(tmp, "w") as f:
|
||||
for l in kept:
|
||||
f.write(l + "\n")
|
||||
os.replace(tmp, queue)
|
||||
print(f"BRAIN_SYNC: {unparseable} unparseable spool record(s) moved to quarantine (inspect {os.path.join(spool_dir, 'quarantine')})", file=sys.stderr)
|
||||
|
||||
if dropped:
|
||||
fd = os.open(drops_file, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||
@@ -374,9 +534,43 @@ subcmd_once() {
|
||||
# the lock removal.
|
||||
trap 'rm -rf "$lock_dir" 2>/dev/null || true' EXIT INT TERM
|
||||
|
||||
# Convert any legacy single-file queue lines into spool records before the
|
||||
# drain reads the spool (transition window for pre-spool writers).
|
||||
migrate_legacy_queue
|
||||
|
||||
# Janitor: reap orphaned enqueue temp files. A writer killed between its
|
||||
# tmp write and the atomic rename leaves `.tmp-*` behind forever — it never
|
||||
# becomes a record and nothing else touches it. One hour is far beyond any
|
||||
# live writer's write→rename window, so a fresh tmp (an in-flight enqueue)
|
||||
# is never touched. Runs inside the run lock, so it can't race the drain.
|
||||
find "$QUEUE_DIR" -maxdepth 1 -type f -name '.tmp-*' -mmin +60 -delete 2>/dev/null || true
|
||||
|
||||
local mode
|
||||
mode=$("$CONFIG_BIN" get artifacts_sync_mode 2>/dev/null || echo off)
|
||||
|
||||
# #2516: advance the brain worktree gbrain indexes to the artifacts repo's
|
||||
# HEAD once a day — previously it only moved when setup-gbrain / sync-gbrain
|
||||
# / brain-restore ran, so brains silently served stale code forever. Runs
|
||||
# inside THIS run lock (never concurrent with the ingest steps below) and
|
||||
# before they touch the worktree. Attempt-throttled: the stamp is written on
|
||||
# ATTEMPT, so a persistently-failing advance warns once per 24h, not at
|
||||
# every skill boundary. The advance itself refuses dirty or unmanaged
|
||||
# worktrees and never force-removes (see gstack-gbrain-source-wireup).
|
||||
if [ -e "${GSTACK_BRAIN_WORKTREE:-$HOME/.gstack-brain-worktree}" ]; then
|
||||
local adv_stamp adv_now adv_last adv_age
|
||||
adv_stamp="$GSTACK_HOME/.brain-worktree-last-advance"
|
||||
adv_now=$(date +%s)
|
||||
adv_last=$(cat "$adv_stamp" 2>/dev/null || echo 0)
|
||||
case "$adv_last" in ''|*[!0-9]*) adv_last=0 ;; esac
|
||||
adv_age=$(( adv_now - adv_last ))
|
||||
if [ "$adv_age" -ge 86400 ]; then
|
||||
echo "$adv_now" > "$adv_stamp" 2>/dev/null || true
|
||||
if ! "$SCRIPT_DIR/gstack-gbrain-source-wireup" --advance-only 1>&2; then
|
||||
echo "BRAIN_SYNC: warning: brain worktree advance failed — gbrain may be indexing stale code (run gstack-gbrain-source-wireup to repair)" >&2
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# #2549 unpushed-commit detector: a prior drain may have COMMITTED but
|
||||
# failed to push (auth blip, offline). The data was never lost — it sits in
|
||||
# a local commit — but nothing re-pushed it until NEW changes arrived.
|
||||
@@ -426,27 +620,42 @@ subcmd_once() {
|
||||
fi
|
||||
|
||||
# Empty-queue fast path: this is the steady state at every skill boundary.
|
||||
# Skipping compute/rewrite here is safe — with zero queue lines there is
|
||||
# nothing to classify, retain, or drop, and a concurrent append after this
|
||||
# check simply waits for the next boundary. (The detector above already ran:
|
||||
# its whole point is re-pushing stranded commits when the queue is empty.)
|
||||
# The lock-release trap installed at acquisition covers this exit.
|
||||
if [ ! -s "$QUEUE" ]; then
|
||||
# Skipping compute/finalize here is safe — with zero spool records there is
|
||||
# nothing to classify, retain, or drop, and a record created after this
|
||||
# check simply waits for the next boundary. The legacy file is checked too:
|
||||
# an OLD writer may have recreated it after the migration above (it gets
|
||||
# migrated next run, but the depth is honest now) — and so is a leftover
|
||||
# .migrating file: if its conversion failed above (e.g. python3 missing),
|
||||
# records are still pending, so "idle" would be dishonest. (The detector
|
||||
# above already ran: its whole point is re-pushing stranded commits when
|
||||
# the queue is empty.) The lock-release trap installed at acquisition
|
||||
# covers this exit.
|
||||
if ! spool_has_records && [ ! -s "$QUEUE" ] && [ ! -s "$QUEUE.migrating" ]; then
|
||||
write_status "idle" "queue empty"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
local paths_file class_file
|
||||
local paths_file class_file snapshot_file
|
||||
paths_file=$(mktemp /tmp/brain-sync-paths.XXXXXX) || { rm -rf "$lock_dir" 2>/dev/null; write_status "error" "mktemp failed"; exit 1; }
|
||||
class_file=$(mktemp /tmp/brain-sync-class.XXXXXX) || { rm -f "$paths_file"; rm -rf "$lock_dir" 2>/dev/null; write_status "error" "mktemp failed"; exit 1; }
|
||||
snapshot_file=$(mktemp /tmp/brain-sync-snapshot.XXXXXX) || { rm -f "$paths_file" "$class_file"; rm -rf "$lock_dir" 2>/dev/null; write_status "error" "mktemp failed"; exit 1; }
|
||||
# Single trap covers all: lock cleanup AND tempfile cleanup.
|
||||
trap 'rm -f "$paths_file" "$class_file" 2>/dev/null; rm -rf "$lock_dir" 2>/dev/null || true' EXIT INT TERM
|
||||
trap 'rm -f "$paths_file" "$class_file" "$snapshot_file" 2>/dev/null; rm -rf "$lock_dir" 2>/dev/null || true' EXIT INT TERM
|
||||
|
||||
compute_paths_to_stage "$mode" "$class_file" > "$paths_file"
|
||||
# Fail-safe (G1): a classifier that dies mid-run (ENOSPC/OOM/SIGKILL, or a
|
||||
# shape the defensive filters don't cover) may have already written the
|
||||
# snapshot manifest but no classification. Finalizing on that state is what
|
||||
# used to wipe the queue — so on a nonzero exit, warn loudly, do NOT call
|
||||
# finalize_queue, and leave everything queued for the next drain.
|
||||
if ! compute_paths_to_stage "$mode" "$class_file" "$snapshot_file" > "$paths_file"; then
|
||||
echo "BRAIN_SYNC: warning: queue classification failed — no records consumed; next run re-drains" >&2
|
||||
write_status "error" "classification failed; queue preserved (next run retries)"
|
||||
exit 0
|
||||
fi
|
||||
if [ ! -s "$paths_file" ]; then
|
||||
# Nothing stageable. Rewrite the queue (retained entries + concurrent
|
||||
# appends survive; classified drops removed) instead of truncating it.
|
||||
rewrite_queue "$paths_file" "$class_file"
|
||||
# Nothing stageable. Finalize the snapshot (retained entries survive;
|
||||
# classified drops removed; records created after the snapshot untouched).
|
||||
finalize_queue "$snapshot_file" "$class_file" "$paths_file"
|
||||
local summary
|
||||
summary=$(queue_summary "$class_file")
|
||||
write_status "idle" "no stageable changes${summary:+ ($summary)}"
|
||||
@@ -497,8 +706,8 @@ subcmd_once() {
|
||||
git -C "$GSTACK_HOME" -c user.email="gstack@localhost" -c user.name="gstack-brain-sync" \
|
||||
commit -q -m "$msg" 2>/dev/null || {
|
||||
# Nothing to commit (e.g. all files already committed). The drained
|
||||
# paths leave the queue; retained + concurrent entries survive (#2549).
|
||||
rewrite_queue "$paths_file" "$class_file"
|
||||
# records leave the spool; retained + post-snapshot records survive.
|
||||
finalize_queue "$snapshot_file" "$class_file" "$paths_file"
|
||||
write_status "idle" "queue drained but no new changes to commit"
|
||||
exit 0
|
||||
}
|
||||
@@ -512,10 +721,10 @@ subcmd_once() {
|
||||
hint=$(remote_auth_hint)
|
||||
write_status "push_failed" "push failed: auth error; commit retained locally, will retry next run. fix: $hint"
|
||||
echo "BRAIN_SYNC: push failed: auth. fix: $hint" >&2
|
||||
# Drained paths leave the queue — they live in the local commit, which
|
||||
# Drained records leave the spool — they live in the local commit, which
|
||||
# the run-start detector re-pushes next time (#2549). Retained +
|
||||
# concurrent entries survive the rewrite.
|
||||
rewrite_queue "$paths_file" "$class_file"
|
||||
# post-snapshot records survive the finalize.
|
||||
finalize_queue "$snapshot_file" "$class_file" "$paths_file"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -529,7 +738,7 @@ subcmd_once() {
|
||||
if git -C "$GSTACK_HOME" merge --no-edit "origin/$branch" >/dev/null 2>&1; then
|
||||
if GSTACK_HOME="$GSTACK_HOME" _receipted_git closed brain-sync "$push_host" curated-memory-git-push "artifacts_sync_mode!=off" \
|
||||
bash -c 'git -C "$1" push origin HEAD 2>/dev/null' _ "$GSTACK_HOME"; then
|
||||
rewrite_queue "$paths_file" "$class_file"
|
||||
finalize_queue "$snapshot_file" "$class_file" "$paths_file"
|
||||
date -u +%Y-%m-%dT%H:%M:%SZ > "$LAST_PUSH_FILE"
|
||||
write_status "ok" "pushed $n file(s) after rebase"
|
||||
exit 0
|
||||
@@ -538,12 +747,12 @@ subcmd_once() {
|
||||
fi
|
||||
# Commit exists locally; the run-start detector re-pushes it next time.
|
||||
write_status "push_failed" "push failed: $(printf '%s' "$push_err" | head -1); commit retained locally, will retry next run"
|
||||
rewrite_queue "$paths_file" "$class_file"
|
||||
finalize_queue "$snapshot_file" "$class_file" "$paths_file"
|
||||
exit 0
|
||||
}
|
||||
|
||||
# Success: drained paths leave the queue (retained + concurrent survive).
|
||||
rewrite_queue "$paths_file" "$class_file"
|
||||
# Success: drained records leave the spool (retained + post-snapshot survive).
|
||||
finalize_queue "$snapshot_file" "$class_file" "$paths_file"
|
||||
date -u +%Y-%m-%dT%H:%M:%SZ > "$LAST_PUSH_FILE"
|
||||
write_status "ok" "pushed $n file(s)"
|
||||
exit 0
|
||||
@@ -555,9 +764,15 @@ subcmd_status() {
|
||||
else
|
||||
echo '{"status":"unknown","message":"no status file yet"}'
|
||||
fi
|
||||
# Supplemental info (not in status file).
|
||||
local queue_depth=0
|
||||
[ -f "$QUEUE" ] && queue_depth=$(wc -l < "$QUEUE" | tr -d ' ')
|
||||
# Supplemental info (not in status file). Depth = spool record files plus
|
||||
# any not-yet-migrated legacy queue lines (transition window), including a
|
||||
# crash-leftover .migrating file — its records are still pending too.
|
||||
local queue_depth spool_depth legacy_depth
|
||||
spool_depth=$(ls "$QUEUE_DIR"/*.json 2>/dev/null | wc -l | tr -d ' ')
|
||||
legacy_depth=0
|
||||
[ -f "$QUEUE" ] && legacy_depth=$(wc -l < "$QUEUE" | tr -d ' ')
|
||||
[ -f "$QUEUE.migrating" ] && legacy_depth=$(( legacy_depth + $(wc -l < "$QUEUE.migrating" | tr -d ' ') ))
|
||||
queue_depth=$(( spool_depth + legacy_depth ))
|
||||
local last_push="never"
|
||||
[ -f "$LAST_PUSH_FILE" ] && last_push=$(cat "$LAST_PUSH_FILE" 2>/dev/null || echo never)
|
||||
local mode
|
||||
@@ -588,13 +803,31 @@ subcmd_drop_queue() {
|
||||
echo "Refusing: --drop-queue discards pending syncs. Pass --yes to confirm." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$QUEUE" ]; then
|
||||
# Remove spool record files, then truncate any legacy queue remnant —
|
||||
# including a crash-leftover .migrating file, whose records would otherwise
|
||||
# resurrect on the next drain via migrate_legacy_queue after the user
|
||||
# explicitly discarded the queue.
|
||||
local n=0 f
|
||||
for f in "$QUEUE_DIR"/*.json; do
|
||||
[ -e "$f" ] || continue
|
||||
rm -f "$f" 2>/dev/null && n=$(( n + 1 ))
|
||||
done
|
||||
if [ -f "$QUEUE" ]; then
|
||||
local legacy_n
|
||||
legacy_n=$(wc -l < "$QUEUE" | tr -d ' ')
|
||||
n=$(( n + legacy_n ))
|
||||
: > "$QUEUE"
|
||||
fi
|
||||
if [ -f "$QUEUE.migrating" ]; then
|
||||
local mig_n
|
||||
mig_n=$(wc -l < "$QUEUE.migrating" | tr -d ' ')
|
||||
n=$(( n + mig_n ))
|
||||
rm -f "$QUEUE.migrating" 2>/dev/null || true
|
||||
fi
|
||||
if [ "$n" -eq 0 ]; then
|
||||
echo "queue already empty"
|
||||
exit 0
|
||||
fi
|
||||
local n
|
||||
n=$(wc -l < "$QUEUE" | tr -d ' ')
|
||||
: > "$QUEUE"
|
||||
echo "dropped $n queue entries"
|
||||
}
|
||||
|
||||
@@ -604,11 +837,11 @@ subcmd_discover_new() {
|
||||
fi
|
||||
# Walk allowlist globs; enqueue any file where mtime+size differs from cursor.
|
||||
python3 - "$GSTACK_HOME" "$ALLOWLIST" "$DISCOVER_CURSOR" <<'PYEOF' 2>/dev/null || true
|
||||
import sys, os, json, fnmatch
|
||||
import sys, os, json, fnmatch, time
|
||||
from datetime import datetime, timezone
|
||||
|
||||
gstack_home, allowlist_path, cursor_path = sys.argv[1:4]
|
||||
queue_path = os.path.join(gstack_home, ".brain-queue.jsonl")
|
||||
spool_dir = os.path.join(gstack_home, ".brain-queue.d")
|
||||
skip_path = os.path.join(gstack_home, ".brain-skip.txt")
|
||||
|
||||
def load_lines(path):
|
||||
@@ -666,34 +899,36 @@ for root, dirs, files in os.walk(gstack_home):
|
||||
if cursor.get(rel) != key:
|
||||
to_enqueue.append((rel, key))
|
||||
|
||||
# Append to the queue directly. The previous implementation shelled out to
|
||||
# Write spool records directly. The previous implementation shelled out to
|
||||
# gstack-brain-enqueue once per file, but Windows Python cannot exec a
|
||||
# bash-shebang script (the spawn fails with a fork error), so discovery
|
||||
# enqueued nothing on Windows even after the path-match fix above.
|
||||
# Writing the queue line here is platform-agnostic; the drain step
|
||||
# Writing the record here is platform-agnostic; the drain step
|
||||
# (compute_paths_to_stage) still re-applies the skip-list + privacy filters.
|
||||
if to_enqueue:
|
||||
ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
written = []
|
||||
try:
|
||||
# One atomic append per record (O_APPEND, each line < PIPE_BUF), matching
|
||||
# gstack-brain-enqueue's concurrency contract so a writer-shim append
|
||||
# running in parallel can't interleave mid-record. Buffered text writes
|
||||
# don't guarantee that. Compact separators match the shim's JSON shape.
|
||||
fd = os.open(queue_path, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o644)
|
||||
try:
|
||||
for rel, key in to_enqueue:
|
||||
rec = json.dumps({"file": rel, "ts": ts}, separators=(",", ":"))
|
||||
os.write(fd, (rec + "\n").encode("utf-8"))
|
||||
finally:
|
||||
os.close(fd)
|
||||
# One spool FILE per record (tmp write + atomic os.replace), matching
|
||||
# gstack-brain-enqueue's maildir contract: writers and the drain never
|
||||
# share an inode, so a parallel writer or drain can't race this.
|
||||
# Compact separators match the shim's JSON shape.
|
||||
os.makedirs(spool_dir, exist_ok=True)
|
||||
for i, (rel, key) in enumerate(to_enqueue):
|
||||
rec = json.dumps({"file": rel, "ts": ts}, separators=(",", ":"))
|
||||
tmp = os.path.join(spool_dir, f".tmp-{os.getpid()}-d{i}")
|
||||
with open(tmp, "w") as f:
|
||||
f.write(rec + "\n")
|
||||
os.replace(tmp, os.path.join(spool_dir, f"{int(time.time())}-{os.getpid()}-d{i}.json"))
|
||||
written.append((rel, key))
|
||||
except OSError:
|
||||
# Queue write failed (disk full, AV file lock). Leave the cursor
|
||||
# unadvanced so these files are retried on the next discover instead of
|
||||
# being silently recorded as synced (which loses the change until the
|
||||
# file next changes).
|
||||
to_enqueue = []
|
||||
# Spool write failed (disk full, AV file lock). Leave the cursor
|
||||
# unadvanced for unwritten records so they are retried on the next
|
||||
# discover instead of being silently recorded as synced (which loses
|
||||
# the change until the file next changes).
|
||||
pass
|
||||
# Advance the cursor only for records actually written.
|
||||
for rel, key in to_enqueue:
|
||||
for rel, key in written:
|
||||
new_cursor[rel] = key
|
||||
|
||||
save_cursor(cursor_path, new_cursor)
|
||||
|
||||
@@ -19,9 +19,11 @@
|
||||
# .gitattributes — merge driver declarations
|
||||
# .brain-allowlist — sync path list
|
||||
# .brain-privacy-map.json — sync privacy classifier
|
||||
# .brain-queue.jsonl — pending queue
|
||||
# .brain-queue.d/ — pending spool (one file per record)
|
||||
# .brain-queue.jsonl — legacy pending queue (pre-spool)
|
||||
# .brain-discover-cursor — discover-new cursor
|
||||
# .brain-last-push — timestamp marker
|
||||
# .brain-worktree-last-advance — daily worktree-advance stamp (#2516)
|
||||
# .brain-skip.txt — user-maintained skip list
|
||||
# .brain-sync.lock.d/ — lock dir (if present)
|
||||
# .brain-sync-status.json — health status
|
||||
@@ -118,10 +120,13 @@ rm -f "$GSTACK_HOME/.gitignore" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.gitattributes" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-allowlist" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-privacy-map.json" 2>/dev/null || true
|
||||
rm -rf "$GSTACK_HOME/.brain-queue.d" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-queue.jsonl" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-queue.jsonl.migrating" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-discover-cursor" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-last-push" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-last-pull" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-worktree-last-advance" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-skip.txt" 2>/dev/null || true
|
||||
rm -f "$GSTACK_HOME/.brain-sync-status.json" 2>/dev/null || true
|
||||
rm -rf "$GSTACK_HOME/.brain-sync.lock.d" 2>/dev/null || true
|
||||
|
||||
+27
-2
@@ -161,7 +161,27 @@ lookup_default() {
|
||||
brain_trust_policy*) echo "unset" ;;
|
||||
salience_allowlist) echo "" ;;
|
||||
user_slug_at_*) echo "" ;;
|
||||
*) echo "" ;;
|
||||
# Read by skill preambles but missing from this table, so they fell through
|
||||
# to the catch-all and came back "" with exit 0. Values below are the ones
|
||||
# the callers already assume in their own `|| echo "<default>"` fallback.
|
||||
question_tuning) echo "false" ;;
|
||||
team_mode) echo "false" ;;
|
||||
transcript_ingest_mode) echo "off" ;;
|
||||
# repo_mode: EMPTY is load-bearing — gstack-repo-mode treats any non-empty
|
||||
# answer as a user override and skips its own classification entirely, so
|
||||
# a synthesized "unknown" default turns the classifier into dead code.
|
||||
# Empty + exit 0 = "no override set, go classify".
|
||||
repo_mode) echo "" ;;
|
||||
# Unknown key: exit non-zero instead of printing "". The fallback pattern
|
||||
# the preambles use,
|
||||
# VAR=$(gstack-config get <key> 2>/dev/null || echo "<default>")
|
||||
# only fires on a non-zero exit, so a catch-all echoing "" with exit 0 left
|
||||
# VAR empty and the written default unreachable.
|
||||
# Deliberately *only* the unknown-key path: the keys above whose default is
|
||||
# intentionally empty (cross_project_learnings, salience_allowlist,
|
||||
# user_slug_at_*, redact_repo_visibility) keep exit 0, because "" is their
|
||||
# real answer and their callers rely on it.
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
@@ -297,7 +317,12 @@ case "${1:-}" in
|
||||
fi
|
||||
VALUE=$(read_config_value "$KEY" || true)
|
||||
if [ -z "$VALUE" ]; then
|
||||
VALUE=$(lookup_default "$KEY")
|
||||
# lookup_default exits non-zero for a key it does not know. Propagate
|
||||
# that, so the caller's `|| echo "<default>"` can fire. A known key whose
|
||||
# default is empty still exits 0 and prints "".
|
||||
if ! VALUE=$(lookup_default "$KEY"); then
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
printf '%s' "$VALUE"
|
||||
;;
|
||||
|
||||
@@ -7,6 +7,13 @@
|
||||
# if no URL is passed. Exits 0 with one of: read-write, read-only,
|
||||
# deny, unset.
|
||||
#
|
||||
# gstack-gbrain-repo-policy get --batch
|
||||
# Read remote URLs from stdin (one per line); print one tier per line
|
||||
# in input order: read-write, read-only, deny, or none (no entry / no
|
||||
# store). A corrupt store is a hard error (exit 2), NEVER quarantined:
|
||||
# batch callers are unattended ingest gates that must fail closed
|
||||
# rather than bypass a set policy.
|
||||
#
|
||||
# gstack-gbrain-repo-policy set <remote-url> <read-write|read-only|deny>
|
||||
# Persist a tier for the given remote. Exits 0 on success.
|
||||
#
|
||||
@@ -85,14 +92,23 @@ normalize() {
|
||||
case "$head" in
|
||||
*:*) url=$(printf '%s' "$url" | sed 's|:|/|') ;;
|
||||
esac
|
||||
# Lowercase BEFORE the suffix strips so a `.GIT` suffix still strips —
|
||||
# parity with lib/gstack-memory-helpers' canonicalizeRemote, which strips
|
||||
# `.git` case-insensitively. GitHub and most hosts are case-insensitive on
|
||||
# paths anyway; collapsing avoids duplicate entries for "Foo/Bar" vs
|
||||
# "foo/bar". (Parity is pinned by test/gbrain-repo-policy-client.test.ts:
|
||||
# a key set through THIS normalize must be found via the canonicalized form
|
||||
# memory-ingest passes to `get --batch`.)
|
||||
url=$(printf '%s' "$url" | tr '[:upper:]' '[:lower:]')
|
||||
# Strip trailing slash(es) FIRST, so ".git/" still loses its suffix (same
|
||||
# order as canonicalizeRemote — slash-first, then .git, then re-strip).
|
||||
while [ "${url%/}" != "$url" ]; do url="${url%/}"; done
|
||||
# Strip trailing .git
|
||||
url="${url%.git}"
|
||||
# Strip trailing /
|
||||
url="${url%/}"
|
||||
# Lowercase the whole thing. GitHub and most hosts are case-insensitive on
|
||||
# paths anyway; collapsing avoids duplicate entries for "Foo/Bar" vs
|
||||
# "foo/bar".
|
||||
printf '%s\n' "$url" | tr '[:upper:]' '[:lower:]'
|
||||
# Re-strip trailing slash(es): a path remote ending in a `.git` directory
|
||||
# component ("/repo/.git") exposes a new trailing slash once .git is gone.
|
||||
while [ "${url%/}" != "$url" ]; do url="${url%/}"; done
|
||||
printf '%s\n' "$url"
|
||||
}
|
||||
|
||||
# ensure_file — create the policy file if missing, migrate if legacy.
|
||||
@@ -161,8 +177,50 @@ ensure_file() {
|
||||
fi
|
||||
}
|
||||
|
||||
# get --batch — bulk lookup for ingest gates. One URL per stdin line, one
|
||||
# tier per stdout line, input order preserved. Reuses normalize() (the same
|
||||
# code path single `get` uses) per line. Prints `none` where single `get`
|
||||
# prints `unset` — batch consumers (lib/gbrain-repo-policy-client.ts) speak
|
||||
# the RepoPolicyTierValue vocabulary directly.
|
||||
#
|
||||
# Corruption polarity differs from single `get` ON PURPOSE: interactive
|
||||
# `get` quarantines a corrupt store and starts fresh because /setup-gbrain
|
||||
# re-asks the user; a batch caller is an unattended ingest gate with nobody
|
||||
# to re-ask, so silently quarantining would BYPASS a set deny policy. Batch
|
||||
# fails hard (exit 2) instead and names the recovery path.
|
||||
cmd_get_batch() {
|
||||
require_jq
|
||||
if [ ! -f "$POLICY_FILE" ]; then
|
||||
# No store = no policy was ever set. Every URL is `none`; don't create
|
||||
# the file just for a read (matches cmd_list).
|
||||
while IFS= read -r url || [ -n "$url" ]; do
|
||||
printf 'none\n'
|
||||
done
|
||||
return 0
|
||||
fi
|
||||
if ! jq empty "$POLICY_FILE" 2>/dev/null; then
|
||||
die "policy store $POLICY_FILE is corrupt (invalid JSON) — refusing batch read. Inspect with: gstack-gbrain-repo-policy list; re-run /setup-gbrain to rebuild the store."
|
||||
fi
|
||||
# Valid JSON from here, so ensure_file only performs the legacy
|
||||
# allow → read-write migration (never the quarantine branch).
|
||||
ensure_file
|
||||
local url key
|
||||
while IFS= read -r url || [ -n "$url" ]; do
|
||||
key=$(normalize "$url")
|
||||
if [ -z "$key" ]; then
|
||||
printf 'none\n'
|
||||
continue
|
||||
fi
|
||||
jq -r --arg key "$key" '.[$key] // "none"' "$POLICY_FILE"
|
||||
done
|
||||
}
|
||||
|
||||
cmd_get() {
|
||||
local url="${1:-}"
|
||||
if [ "$url" = "--batch" ]; then
|
||||
cmd_get_batch
|
||||
return 0
|
||||
fi
|
||||
if [ -z "$url" ]; then
|
||||
url=$(git remote get-url origin 2>/dev/null || true)
|
||||
if [ -z "$url" ]; then
|
||||
@@ -221,7 +279,7 @@ case "${1:-}" in
|
||||
set) shift; cmd_set "$@" ;;
|
||||
list) shift; cmd_list "$@" ;;
|
||||
normalize) shift; cmd_normalize "$@" ;;
|
||||
--help|-h|help) sed -n '2,47p' "$0" | sed 's/^# \{0,1\}//' ;;
|
||||
--help|-h|help) sed -n '2,54p' "$0" | sed 's/^# \{0,1\}//' ;;
|
||||
"") die "usage: gstack-gbrain-repo-policy {get|set|list|normalize|--help}" ;;
|
||||
*) die "unknown subcommand: $1" ;;
|
||||
esac
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
# gstack-gbrain-source-wireup --uninstall [--source-id <id>]
|
||||
# [--database-url <url>]
|
||||
# gstack-gbrain-source-wireup --probe
|
||||
# gstack-gbrain-source-wireup --advance-only # daily unattended worktree advance (#2516)
|
||||
# gstack-gbrain-source-wireup --help
|
||||
#
|
||||
# Exit codes:
|
||||
@@ -64,6 +65,7 @@ while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--uninstall) MODE="uninstall"; shift ;;
|
||||
--probe) MODE="probe"; shift ;;
|
||||
--advance-only) MODE="advance-only"; shift ;;
|
||||
--strict) STRICT=1; shift ;;
|
||||
--no-pull) NO_PULL=1; shift ;;
|
||||
--source-id) SOURCE_ID="$2"; shift 2 ;;
|
||||
@@ -336,6 +338,50 @@ do_wireup() {
|
||||
echo "pages_synced=$(echo "$sync_out" | grep -oE '[0-9]+ pages? imported' | head -1 || echo 'incremental')"
|
||||
}
|
||||
|
||||
do_advance_only() {
|
||||
# Daily unattended advance (#2516): the brain worktree gbrain indexes only
|
||||
# moved when setup-gbrain / sync-gbrain / brain-restore ran, so brains
|
||||
# silently served stale code. This mode is git-only (no gbrain prereqs) and
|
||||
# SAFE for a cron cadence: it refuses dirty worktrees and NEVER runs
|
||||
# ensure_worktree's force-remove recovery — an unattended path must not be
|
||||
# able to delete local worktree changes. All git ops are pinned to
|
||||
# $GSTACK_HOME / $WORKTREE, never cwd-derived.
|
||||
[ -d "$GSTACK_HOME/.git" ] || { warn "advance-only: no artifacts repo at $GSTACK_HOME; nothing to advance"; exit 0; }
|
||||
if [ ! -d "$WORKTREE/.git" ] && [ ! -f "$WORKTREE/.git" ]; then
|
||||
warn "advance-only: no managed worktree at $WORKTREE (run the setup-gbrain wireup first)"
|
||||
exit 0
|
||||
fi
|
||||
# Managed-marker check: refuse anything that is not a worktree OF the
|
||||
# artifacts repo — a misconfigured GSTACK_BRAIN_WORKTREE pointing at a user
|
||||
# repo must never be advanced/detached.
|
||||
local gitdir home_git
|
||||
gitdir=$(git -C "$WORKTREE" rev-parse --absolute-git-dir 2>/dev/null || echo "")
|
||||
# Physical path for the comparison: rev-parse returns resolved paths, while
|
||||
# $GSTACK_HOME may reach the same place through a symlink (macOS /var/folders).
|
||||
home_git=$(cd "$GSTACK_HOME/.git" 2>/dev/null && pwd -P || echo "$GSTACK_HOME/.git")
|
||||
case "$gitdir" in
|
||||
"$home_git/worktrees/"*) : ;;
|
||||
*) warn "advance-only: $WORKTREE is not a worktree of $GSTACK_HOME (gitdir: ${gitdir:-unreadable}); refusing"; exit 0 ;;
|
||||
esac
|
||||
if [ -n "$(git -C "$WORKTREE" status --porcelain 2>/dev/null)" ]; then
|
||||
warn "advance-only: worktree at $WORKTREE has local changes; refusing to advance them away"
|
||||
exit 0
|
||||
fi
|
||||
local sha cur
|
||||
sha=$(git -C "$GSTACK_HOME" rev-parse HEAD 2>/dev/null) || { warn "advance-only: cannot read parent HEAD"; exit 0; }
|
||||
cur=$(git -C "$WORKTREE" rev-parse HEAD 2>/dev/null || echo "")
|
||||
if [ "$cur" = "$sha" ]; then
|
||||
echo "advance-only: up-to-date at $sha"
|
||||
return 0
|
||||
fi
|
||||
if ( cd "$WORKTREE" && git checkout --detach "$sha" 2>&1 | prefix; exit "${PIPESTATUS[0]}" ); then
|
||||
echo "advance-only: advanced $WORKTREE to $sha"
|
||||
else
|
||||
warn "advance-only: could not advance $WORKTREE to $sha; NOT force-resetting on the unattended path. Run gstack-gbrain-source-wireup to repair."
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
do_uninstall() {
|
||||
local id
|
||||
id=$(derive_source_id) || die "cannot derive source id; pass --source-id <id> explicitly" 3
|
||||
@@ -356,7 +402,8 @@ do_uninstall() {
|
||||
}
|
||||
|
||||
case "$MODE" in
|
||||
probe) do_probe ;;
|
||||
wireup) do_wireup ;;
|
||||
uninstall) do_uninstall ;;
|
||||
probe) do_probe ;;
|
||||
wireup) do_wireup ;;
|
||||
uninstall) do_uninstall ;;
|
||||
advance-only) do_advance_only ;;
|
||||
esac
|
||||
|
||||
+338
-8
@@ -68,6 +68,7 @@ import {
|
||||
import { execGbrainText, spawnGbrainAsync } from "../lib/gbrain-exec";
|
||||
import { writeReceipt } from "../lib/egress-receipt";
|
||||
import { checkOwnedStagingDir, STAGING_MARKER } from "../lib/staging-guard";
|
||||
import { hasRepoPolicyStore, repoPolicyTierBatch } from "../lib/gbrain-repo-policy-client";
|
||||
|
||||
// ── Types ──────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -141,6 +142,14 @@ interface ProbeReport {
|
||||
new_count: number;
|
||||
updated_count: number;
|
||||
unchanged_count: number;
|
||||
skipped_unattributed: number;
|
||||
/**
|
||||
* #2392 parity: transcripts whose remote's trust tier is `deny` /
|
||||
* `read-only`. Probe applies the SAME per-remote policy filter --bulk
|
||||
* applies, so its ingestible counts match what --bulk would write.
|
||||
*/
|
||||
skipped_policy_deny: number;
|
||||
skipped_policy_readonly: number;
|
||||
estimate_minutes: number;
|
||||
}
|
||||
|
||||
@@ -149,6 +158,14 @@ interface BulkResult {
|
||||
skipped_secret: number;
|
||||
skipped_dedup: number;
|
||||
skipped_unattributed: number;
|
||||
/**
|
||||
* #2392: transcripts skipped because their git remote's trust tier in
|
||||
* ~/.gstack/gbrain-repo-policy.json is `read-only` (search allowed, page
|
||||
* writes never — and transcript ingest writes pages).
|
||||
*/
|
||||
skipped_policy_readonly: number;
|
||||
/** #2392: transcripts skipped because their remote's trust tier is `deny`. */
|
||||
skipped_policy_deny: number;
|
||||
failed: number;
|
||||
duration_ms: number;
|
||||
partial_pages: number;
|
||||
@@ -677,8 +694,21 @@ function extractContentText(rec: any): string {
|
||||
return "";
|
||||
}
|
||||
|
||||
// Memo: probe and prepare both resolve remotes per-transcript, and transcripts
|
||||
// share a small set of cwds — without this an 11.7K-file probe would spawn git
|
||||
// 11.7K times instead of once per distinct cwd.
|
||||
const REMOTE_MEMO = new Map<string, string>();
|
||||
|
||||
function resolveGitRemote(cwd: string): string {
|
||||
if (!cwd) return "";
|
||||
const memo = REMOTE_MEMO.get(cwd);
|
||||
if (memo !== undefined) return memo;
|
||||
const resolved = resolveGitRemoteUncached(cwd);
|
||||
REMOTE_MEMO.set(cwd, resolved);
|
||||
return resolved;
|
||||
}
|
||||
|
||||
function resolveGitRemoteUncached(cwd: string): string {
|
||||
try {
|
||||
// execFileSync (no shell) so `cwd` cannot trigger command substitution.
|
||||
// Transcript JSONL records are an untrusted surface (a poisoned `.cwd`
|
||||
@@ -900,6 +930,14 @@ interface PreparedPage {
|
||||
/** Carry-through fields for state recording on success. */
|
||||
page_slug: string;
|
||||
partial: boolean;
|
||||
/** Memory type — the per-remote policy filter (#2392) applies to transcripts only. */
|
||||
type: MemoryType;
|
||||
/**
|
||||
* Canonical git remote ("host/org/repo") for transcript pages; undefined
|
||||
* for artifacts (whose PageRecord.git_remote is a project slug, not a
|
||||
* remote — artifacts are never policy-filtered).
|
||||
*/
|
||||
git_remote?: string;
|
||||
}
|
||||
|
||||
interface StagingResult {
|
||||
@@ -1046,6 +1084,105 @@ export function readNewFailures(
|
||||
|
||||
// ── Main ingest passes ─────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* The ONE attribution gate (#2394): a transcript is attributable iff its cwd
|
||||
* resolves to a git remote. Both probeMode (via transcriptCwdFromPrefix +
|
||||
* resolveGitRemote — the same memoized resolver) and preparePages route
|
||||
* through THIS logic, so the two stages' post-attribution counts are
|
||||
* structurally identical — the parity the probe report promises.
|
||||
*/
|
||||
function sessionIsAttributable(cwd: string | undefined | null): boolean {
|
||||
if (!cwd) return false;
|
||||
return resolveGitRemote(cwd) !== "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Bounded prefix for the probe's cheap-parse (plan C7): transcripts run to
|
||||
* tens of MB, and the probe only needs the cwd, which both agent formats put
|
||||
* on the FIRST records. 256KB is orders of magnitude past any real header.
|
||||
*/
|
||||
const TRANSCRIPT_PROBE_MAX_BYTES = 256 * 1024;
|
||||
|
||||
/**
|
||||
* Lightweight cwd extraction for the probe: reads a BOUNDED prefix (first
|
||||
* 256KB, never the whole file — plan C7: the probe must stay a cheap parse on
|
||||
* multi-MB transcripts) and extracts the cwd with EXACTLY
|
||||
* parseTranscriptJsonl's rules. The caller resolves attribution/policy via
|
||||
* resolveGitRemote (memoized). Avoids the full parse (body rendering, message
|
||||
* counting) because probe only needs the cwd.
|
||||
*
|
||||
* Extraction MIRRORS parseTranscriptJsonl (the single source of truth for
|
||||
* cwd semantics — keep the two in lockstep):
|
||||
* - the first PARSEABLE line decides the format (Codex: type=session_meta
|
||||
* or payload.id; else Claude Code);
|
||||
* - Codex cwd comes from that FIRST record ONLY (payload.cwd || cwd) —
|
||||
* a cwd appearing only on a later record is NOT used, exactly as
|
||||
* parseTranscriptJsonl ignores it, so probe and prepare can never
|
||||
* diverge on the same file;
|
||||
* - Claude Code cwd comes from the first record that carries one;
|
||||
* - unparseable lines are skipped (the truncated-tail case included).
|
||||
*
|
||||
* Non-transcript types (artifacts) always pass — the attribution filter in
|
||||
* preparePages only applies to transcripts (#2394).
|
||||
*/
|
||||
function transcriptCwdFromPrefix(path: string): string {
|
||||
// Chunked read until the prefix contains at least one COMPLETE record
|
||||
// (newline), up to the hard cap — a first record larger than one chunk
|
||||
// (giant pasted prompt) must not truncate mid-JSON and mis-classify a
|
||||
// session --bulk would accept (probe/bulk parity).
|
||||
let raw: string;
|
||||
try {
|
||||
const fd = openSync(path, "r");
|
||||
try {
|
||||
const chunk = Buffer.alloc(TRANSCRIPT_PROBE_MAX_BYTES);
|
||||
let acc = "";
|
||||
let offset = 0;
|
||||
const HARD_CAP = TRANSCRIPT_PROBE_MAX_BYTES * 16; // 4MB ceiling
|
||||
while (offset < HARD_CAP) {
|
||||
const n = readSync(fd, chunk, 0, chunk.length, offset);
|
||||
if (n <= 0) break;
|
||||
acc += chunk.toString("utf-8", 0, n);
|
||||
offset += n;
|
||||
if (acc.includes("\n")) break; // at least one complete record
|
||||
}
|
||||
raw = acc;
|
||||
} finally {
|
||||
closeSync(fd);
|
||||
}
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
const lines = raw.split("\n").filter((l) => l.trim().length > 0);
|
||||
if (lines.length === 0) return "";
|
||||
|
||||
let cwd = "";
|
||||
let sawFirstParseable = false;
|
||||
for (const line of lines) {
|
||||
let rec: any;
|
||||
try {
|
||||
rec = JSON.parse(line);
|
||||
} catch {
|
||||
continue; // mirrors parseTranscriptJsonl: unparseable lines are skipped
|
||||
}
|
||||
if (!sawFirstParseable) {
|
||||
sawFirstParseable = true;
|
||||
// Format detection mirrors parseTranscriptJsonl's `first` record check.
|
||||
const isCodex = rec?.type === "session_meta" || rec?.payload?.id != null;
|
||||
if (isCodex) {
|
||||
// Codex: cwd comes from the session_meta FIRST record only.
|
||||
cwd = rec.payload?.cwd || rec.cwd || "";
|
||||
break;
|
||||
}
|
||||
}
|
||||
// Claude Code: first record with a cwd wins (the first record included).
|
||||
if (rec?.cwd) {
|
||||
cwd = rec.cwd;
|
||||
break;
|
||||
}
|
||||
}
|
||||
return cwd;
|
||||
}
|
||||
|
||||
async function probeMode(args: CliArgs): Promise<ProbeReport> {
|
||||
const state = loadState();
|
||||
const ctx = makeWalkContext(args, state);
|
||||
@@ -1066,8 +1203,56 @@ async function probeMode(args: CliArgs): Promise<ProbeReport> {
|
||||
let newCount = 0;
|
||||
let updatedCount = 0;
|
||||
let unchangedCount = 0;
|
||||
let skippedUnattributed = 0;
|
||||
let skippedPolicyDeny = 0;
|
||||
let skippedPolicyReadonly = 0;
|
||||
|
||||
// Two-phase walk (#2392 parity): collect candidates first (remembering each
|
||||
// transcript's resolved remote), THEN apply the same per-remote policy
|
||||
// filter --bulk applies via one repoPolicyTierBatch spawn. Counting during
|
||||
// the walk would report policy-denied transcripts as ingestible — probe's
|
||||
// numbers must match what --bulk would actually write.
|
||||
const candidates: Array<{ path: string; type: MemoryType; remote: string }> = [];
|
||||
for (const { path, type } of walkAllSources(ctx)) {
|
||||
// Apply the same attribution filter preparePages uses (#2394):
|
||||
// skip transcripts with no resolvable git remote unless --include-unattributed.
|
||||
let remote = "";
|
||||
if (type === "transcript") {
|
||||
const cwd = transcriptCwdFromPrefix(path);
|
||||
remote = cwd ? resolveGitRemote(cwd) : "";
|
||||
if (!args.includeUnattributed && remote === "") {
|
||||
skippedUnattributed++;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
candidates.push({ path, type, remote });
|
||||
}
|
||||
|
||||
// Batch policy check — same hasRepoPolicyStore fast path as preparePages:
|
||||
// no store on disk → zero policy work. Only transcripts with a resolved
|
||||
// remote are policy-filtered; artifacts never are (#2392). A missing or
|
||||
// errored verdict counts as "none" here — probe is read-only and must not
|
||||
// hard-fail the way the write path does.
|
||||
if (hasRepoPolicyStore()) {
|
||||
const remotes = [...new Set(candidates.filter((c) => c.type === "transcript" && c.remote).map((c) => c.remote))];
|
||||
if (remotes.length > 0) {
|
||||
const verdicts = repoPolicyTierBatch(remotes);
|
||||
for (let i = candidates.length - 1; i >= 0; i--) {
|
||||
const c = candidates[i];
|
||||
if (c.type !== "transcript" || !c.remote) continue;
|
||||
const tier = verdicts.get(c.remote)?.tier ?? "none";
|
||||
if (tier === "deny") {
|
||||
skippedPolicyDeny++;
|
||||
candidates.splice(i, 1);
|
||||
} else if (tier === "read-only") {
|
||||
skippedPolicyReadonly++;
|
||||
candidates.splice(i, 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const { path, type } of candidates) {
|
||||
totalFiles++;
|
||||
let size = 0;
|
||||
try {
|
||||
@@ -1096,6 +1281,9 @@ async function probeMode(args: CliArgs): Promise<ProbeReport> {
|
||||
new_count: newCount,
|
||||
updated_count: updatedCount,
|
||||
unchanged_count: unchangedCount,
|
||||
skipped_unattributed: skippedUnattributed,
|
||||
skipped_policy_deny: skippedPolicyDeny,
|
||||
skipped_policy_readonly: skippedPolicyReadonly,
|
||||
estimate_minutes: estimateMinutes,
|
||||
};
|
||||
}
|
||||
@@ -1131,8 +1319,16 @@ function preparePages(
|
||||
skippedSecret: number;
|
||||
skippedDedup: number;
|
||||
skippedUnattributed: number;
|
||||
skippedPolicyReadonly: number;
|
||||
skippedPolicyDeny: number;
|
||||
parseFailed: number;
|
||||
partialPages: number;
|
||||
/**
|
||||
* #2392: set when the per-remote policy store EXISTS but could not be
|
||||
* read (corrupt file, spawn failure). The caller must abort before any
|
||||
* writes — proceeding would bypass a possibly-set deny policy.
|
||||
*/
|
||||
policyError?: string;
|
||||
} {
|
||||
const prepared: PreparedPage[] = [];
|
||||
let skippedSecret = 0;
|
||||
@@ -1141,8 +1337,16 @@ function preparePages(
|
||||
let parseFailed = 0;
|
||||
let partialPages = 0;
|
||||
|
||||
// --limit semantics: "stop after N pages WRITTEN" = N policy-eligible pages.
|
||||
// When a per-remote policy store exists, eligibility is only known after the
|
||||
// batch policy check below, so the walk must not stop early — a denied-first
|
||||
// corpus would otherwise consume the limit and starve permitted pages. With
|
||||
// no store on disk, every prepared page is eligible and the in-loop break
|
||||
// keeps --limit cheap.
|
||||
const policyStoreExists = hasRepoPolicyStore();
|
||||
|
||||
for (const { path, type } of walkAllSources(ctx)) {
|
||||
if (args.limit !== null && prepared.length >= args.limit) break;
|
||||
if (args.limit !== null && !policyStoreExists && prepared.length >= args.limit) break;
|
||||
|
||||
if (args.mode === "incremental" && !fileChangedSinceState(path, state)) {
|
||||
skippedDedup++;
|
||||
@@ -1176,16 +1380,15 @@ function preparePages(
|
||||
parseFailed++;
|
||||
continue;
|
||||
}
|
||||
if (!args.includeUnattributed && !session.cwd) {
|
||||
// The SAME gate probeMode uses (#2394) — routing both through
|
||||
// sessionIsAttributable is what makes probe counts trustworthy.
|
||||
// (Semantically identical to the old two-step check: no cwd, or a cwd
|
||||
// whose remote resolves empty, both rendered git_remote "_unattributed".)
|
||||
if (!args.includeUnattributed && !sessionIsAttributable(session.cwd)) {
|
||||
skippedUnattributed++;
|
||||
continue;
|
||||
}
|
||||
page = buildTranscriptPage(path, session);
|
||||
if (!args.includeUnattributed && page.git_remote === "_unattributed") {
|
||||
skippedUnattributed++;
|
||||
continue;
|
||||
}
|
||||
if (page.partial) partialPages++;
|
||||
} else {
|
||||
page = buildArtifactPage(path, type);
|
||||
}
|
||||
@@ -1201,16 +1404,89 @@ function preparePages(
|
||||
rendered_body: renderPageBody(page),
|
||||
page_slug: page.slug,
|
||||
partial: page.partial ?? false,
|
||||
type,
|
||||
// Only transcripts carry a real remote; buildArtifactPage's git_remote
|
||||
// is a project slug, and artifacts are never policy-filtered (#2392).
|
||||
git_remote: type === "transcript" ? page.git_remote : undefined,
|
||||
});
|
||||
}
|
||||
|
||||
// #2392: per-remote trust policy for transcript pages — the same store the
|
||||
// code-import gate honors (bin/gstack-gbrain-sync.ts). One batch spawn for
|
||||
// all distinct remotes in the run; no store on disk → zero policy work.
|
||||
// Runs AFTER the loop because preparePages accumulates fully in memory (no
|
||||
// writes happen until the caller stages), so filtering here is still
|
||||
// strictly before any write.
|
||||
let finalPrepared = prepared;
|
||||
let skippedPolicyReadonly = 0;
|
||||
let skippedPolicyDeny = 0;
|
||||
let policyError: string | undefined;
|
||||
if (policyStoreExists) {
|
||||
const remotes = [
|
||||
...new Set(
|
||||
prepared
|
||||
.filter((p) => p.type === "transcript" && p.git_remote)
|
||||
.map((p) => p.git_remote as string),
|
||||
),
|
||||
];
|
||||
if (remotes.length > 0) {
|
||||
const verdicts = repoPolicyTierBatch(remotes);
|
||||
// The store EXISTS (checked above), so an unreadable/spawn-failed
|
||||
// result is a HARD ERROR — match the fail-closed polarity of
|
||||
// gstack-gbrain-sync's code-import gate: never bypass a set policy.
|
||||
const broken = remotes.find((r) => {
|
||||
const v = verdicts.get(r);
|
||||
return !v || v.error !== undefined;
|
||||
});
|
||||
if (broken) {
|
||||
const kind = verdicts.get(broken)?.error === "spawn-failed"
|
||||
? "the policy helper could not be spawned (bash missing from PATH?)"
|
||||
: "the policy store could not be read (corrupt file?)";
|
||||
policyError =
|
||||
`repo policy store exists but ${kind} — refusing transcript ingest rather than ` +
|
||||
`bypassing a possibly-set deny policy. Inspect with: gstack-gbrain-repo-policy list; ` +
|
||||
`re-run /setup-gbrain if the store is corrupt.`;
|
||||
} else {
|
||||
finalPrepared = prepared.filter((p) => {
|
||||
if (p.type !== "transcript" || !p.git_remote) return true;
|
||||
const tier = verdicts.get(p.git_remote)?.tier ?? "none";
|
||||
if (tier === "read-only") {
|
||||
// Honoring an explicit user setting (search allowed, page writes
|
||||
// never) — transcript ingest writes pages, so skip.
|
||||
skippedPolicyReadonly++;
|
||||
return false;
|
||||
}
|
||||
if (tier === "deny") {
|
||||
skippedPolicyDeny++;
|
||||
return false;
|
||||
}
|
||||
return true; // read-write, or none (no policy set for this remote)
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --limit applies AFTER policy filtering, over permitted pages only. In the
|
||||
// no-store fast path the walk already stopped at the limit, so this slice
|
||||
// is a no-op there.
|
||||
if (args.limit !== null && finalPrepared.length > args.limit) {
|
||||
finalPrepared = finalPrepared.slice(0, args.limit);
|
||||
}
|
||||
|
||||
// Derived from the FINAL set: partial counts must describe pages that are
|
||||
// actually eligible and within the limit, not the whole scanned corpus.
|
||||
partialPages = finalPrepared.filter((p) => p.partial).length;
|
||||
|
||||
return {
|
||||
prepared,
|
||||
prepared: finalPrepared,
|
||||
skippedSecret,
|
||||
skippedDedup,
|
||||
skippedUnattributed,
|
||||
skippedPolicyReadonly,
|
||||
skippedPolicyDeny,
|
||||
parseFailed,
|
||||
partialPages,
|
||||
policyError,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1557,6 +1833,25 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
let written = 0;
|
||||
let failed = 0;
|
||||
|
||||
// #2392 HARD ERROR: the policy store exists but could not be consulted.
|
||||
// Abort before ANY write — state recording, staging, gbrain import — so a
|
||||
// corrupt store can never silently bypass a set deny/read-only policy.
|
||||
if (prep.policyError) {
|
||||
console.error(`[memory-ingest] ERR: ${prep.policyError}`);
|
||||
return {
|
||||
written: 0,
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed: prep.parseFailed + prep.prepared.length,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
system_error: prep.policyError,
|
||||
};
|
||||
}
|
||||
|
||||
if (args.noWrite) {
|
||||
// --no-write: skip the gbrain import call but still record state for
|
||||
// prepared pages (treat them as ingested for dedup purposes). Matches
|
||||
@@ -1585,6 +1880,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed: prep.parseFailed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -1601,6 +1898,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed: prep.parseFailed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -1616,6 +1915,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed: prep.parseFailed + prep.prepared.length,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -1763,6 +2064,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -1802,6 +2105,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -1838,6 +2143,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -1856,6 +2163,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -1884,6 +2193,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -1936,6 +2247,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -2015,6 +2328,8 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
|
||||
skipped_secret: prep.skippedSecret,
|
||||
skipped_dedup: prep.skippedDedup,
|
||||
skipped_unattributed: prep.skippedUnattributed,
|
||||
skipped_policy_readonly: prep.skippedPolicyReadonly,
|
||||
skipped_policy_deny: prep.skippedPolicyDeny,
|
||||
failed: failed + prep.parseFailed,
|
||||
duration_ms: Date.now() - t0,
|
||||
partial_pages: prep.partialPages,
|
||||
@@ -2042,6 +2357,15 @@ function printProbeReport(r: ProbeReport, json: boolean): void {
|
||||
console.log(`New (never ingested): ${r.new_count}`);
|
||||
console.log(`Updated (mtime/hash): ${r.updated_count}`);
|
||||
console.log(`Unchanged: ${r.unchanged_count}`);
|
||||
if (r.skipped_unattributed > 0) {
|
||||
console.log(`Skipped (unattributed): ${r.skipped_unattributed} (no git remote; use --include-unattributed to include)`);
|
||||
}
|
||||
if (r.skipped_policy_deny > 0) {
|
||||
console.log(`Skipped (policy deny): ${r.skipped_policy_deny} (remote tier is deny; change with: gstack-gbrain-repo-policy set <remote> read-write)`);
|
||||
}
|
||||
if (r.skipped_policy_readonly > 0) {
|
||||
console.log(`Skipped (policy read-only): ${r.skipped_policy_readonly} (remote tier is read-only; transcript ingest writes pages)`);
|
||||
}
|
||||
console.log("By type:");
|
||||
for (const [t, v] of Object.entries(r.by_type)) {
|
||||
if (v.count > 0) {
|
||||
@@ -2058,6 +2382,12 @@ function printBulkResult(r: BulkResult, args: CliArgs): void {
|
||||
console.log(` skipped (dedup): ${r.skipped_dedup}`);
|
||||
console.log(` skipped (secret-scan): ${r.skipped_secret}`);
|
||||
console.log(` skipped (unattrib): ${r.skipped_unattributed}`);
|
||||
if (r.skipped_policy_readonly > 0) {
|
||||
console.log(` skipped (policy read-only): ${r.skipped_policy_readonly} (remote tier is read-only; transcript ingest writes pages)`);
|
||||
}
|
||||
if (r.skipped_policy_deny > 0) {
|
||||
console.log(` skipped (policy deny): ${r.skipped_policy_deny} (change with: gstack-gbrain-repo-policy set <remote> read-write)`);
|
||||
}
|
||||
console.log(` failed: ${r.failed}`);
|
||||
console.log(` duration: ${(r.duration_ms / 1000).toFixed(1)}s`);
|
||||
if (args.benchmark) {
|
||||
|
||||
+151
-32
@@ -147,19 +147,36 @@ function detectHost(): "github" | "gitlab" | "unknown" {
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
function readBaseVersion(base: string, versionPath: string, warnings: string[]): string {
|
||||
// When the base-version read fails we assume a zero base — but a literal
|
||||
// "0.0.0.0" is 4-digit, which flips versionWidth() to 4 and hands a 3-digit
|
||||
// repo a 4-digit slot (the exact width class of bug #2501 fixed in parsing).
|
||||
// The LOCAL version file at versionPath knows the repo's own width; shape the
|
||||
// zero from it. No local file either → keep the 4-digit default.
|
||||
function zeroBaseAtLocalWidth(versionPath: string, repoRoot: string): string {
|
||||
try {
|
||||
const local = extractVersion(readFileSync(join(repoRoot, versionPath), "utf8"), versionPath);
|
||||
if (local && parseVersion(local) && versionWidth(local) === 3) return "0.0.0";
|
||||
} catch {
|
||||
// unreadable/absent local version file — 4-digit default below
|
||||
}
|
||||
return "0.0.0.0";
|
||||
}
|
||||
|
||||
function readBaseVersion(base: string, versionPath: string, repoRoot: string, warnings: string[]): string {
|
||||
// git fetch is best-effort; we tolerate failure and fall back to whatever
|
||||
// origin/<base> currently points at.
|
||||
runCommand("git", ["fetch", "origin", base, "--quiet"], 10000);
|
||||
const r = runCommand("git", ["show", `origin/${base}:${versionPath}`]);
|
||||
if (!r.ok) {
|
||||
warnings.push(`could not read ${versionPath} at origin/${base}; assuming 0.0.0.0`);
|
||||
return "0.0.0.0";
|
||||
const assumed = zeroBaseAtLocalWidth(versionPath, repoRoot);
|
||||
warnings.push(`could not read ${versionPath} at origin/${base}; assuming ${assumed}`);
|
||||
return assumed;
|
||||
}
|
||||
const v = extractVersion(r.stdout, versionPath);
|
||||
if (!v) {
|
||||
warnings.push(`${versionPath} at origin/${base} has no readable version; assuming 0.0.0.0`);
|
||||
return "0.0.0.0";
|
||||
const assumed = zeroBaseAtLocalWidth(versionPath, repoRoot);
|
||||
warnings.push(`${versionPath} at origin/${base} has no readable version; assuming ${assumed}`);
|
||||
return assumed;
|
||||
}
|
||||
return v;
|
||||
}
|
||||
@@ -460,40 +477,141 @@ function autoDetectExcludePR(): number | null {
|
||||
// v0.1.57.0. Auditing that repo's history found FOUR such pairs going back
|
||||
// three weeks, so the silent fallback had been mis-allocating for a while.
|
||||
//
|
||||
// Git already knows what the API was asked for. Remote-tracking refs carry
|
||||
// each branch's VERSION file, and the base's own history records every version
|
||||
// already shipped. Neither needs a token, a network round-trip, or a working
|
||||
// `gh`. So "offline" degrades the QUEUE VIEW (no PR numbers, no draft status)
|
||||
// without degrading the ALLOCATION.
|
||||
// Git already knows what the API was asked for. `git ls-remote --heads origin`
|
||||
// returns the remote's LIVE branch list with zero local mutation (no fetch, no
|
||||
// ref updates), each branch's VERSION file is readable from the local object
|
||||
// store, and the base's own history records every version already shipped.
|
||||
// None of it needs a token or a working `gh`. So "offline" degrades the QUEUE
|
||||
// VIEW (no PR numbers, no draft status) without degrading the ALLOCATION.
|
||||
function fetchGitClaimed(
|
||||
base: string,
|
||||
versionPath: string,
|
||||
warnings: string[],
|
||||
): ClaimedPR[] {
|
||||
const claims: ClaimedPR[] = [];
|
||||
const baseShort = base.replace(/^origin\//, "");
|
||||
|
||||
// 1. Every remote-tracking branch's VERSION file. These are the open PRs'
|
||||
// branches, whether or not the API can be reached to enumerate them.
|
||||
// Read through extractVersion so a JSON version-path (#2501) resolves on
|
||||
// remote refs too, and the branch's own width is preserved in the claim.
|
||||
const refs = runCommand("git", [
|
||||
"for-each-ref",
|
||||
"--format=%(refname:short)",
|
||||
"refs/remotes",
|
||||
]);
|
||||
if (refs.ok) {
|
||||
const baseShort = base.replace(/^origin\//, "");
|
||||
for (const ref of refs.stdout.split("\n").map((r) => r.trim()).filter(Boolean)) {
|
||||
if (ref.endsWith("/HEAD")) continue;
|
||||
if (ref === base || ref.replace(/^origin\//, "") === baseShort) continue;
|
||||
const show = runCommand("git", ["show", `${ref}:${versionPath}`]);
|
||||
if (!show.ok) continue;
|
||||
const raw = extractVersion(show.stdout, versionPath);
|
||||
if (!raw || !parseVersion(raw)) continue;
|
||||
claims.push({ pr: 0, branch: ref, version: raw });
|
||||
// 1. The version-claim branches. FIRST try `git ls-remote --heads origin`:
|
||||
// fresh remote data, zero local mutation. This scopes claims to branches
|
||||
// that actually EXIST on origin right now — the previous implementation
|
||||
// counted every remote-tracking ref on EVERY remote, so stale local refs
|
||||
// (deleted PR branches, an unrelated `upstream` remote) inflated the
|
||||
// claim set and pushed the allocation further than the real queue.
|
||||
// GIT_TERMINAL_PROMPT=0 + a 5s timeout keep a dead/credential-prompting
|
||||
// remote from hanging the allocator.
|
||||
const lsRemote = spawnSync("git", ["ls-remote", "--heads", "origin"], {
|
||||
encoding: "utf8",
|
||||
timeout: 5000,
|
||||
env: { ...process.env, GIT_TERMINAL_PROMPT: "0" },
|
||||
});
|
||||
const lsOk = lsRemote.status === 0 && !lsRemote.error;
|
||||
|
||||
// Each candidate carries the LIVE tip sha when it came from ls-remote, so
|
||||
// the VERSION read prefers the fresh commit (present locally after any
|
||||
// prior fetch/clone) and only falls back to the local remote-tracking ref.
|
||||
const candidates: { branch: string; sha?: string }[] = [];
|
||||
if (lsOk) {
|
||||
for (const line of (lsRemote.stdout ?? "").split("\n")) {
|
||||
const m = line.trim().match(/^([0-9a-f]{40,64})\trefs\/heads\/(.+)$/);
|
||||
if (!m) continue;
|
||||
if (m[2] === baseShort) continue;
|
||||
candidates.push({ branch: m[2], sha: m[1] });
|
||||
}
|
||||
} else {
|
||||
warnings.push("git for-each-ref failed; branch claims unavailable");
|
||||
// Degraded twice over: no host API AND no reachable remote. Fall back to
|
||||
// the LOCAL refs/remotes/origin snapshot ONLY (never other remotes — an
|
||||
// `upstream` remote's branches are not claims against OUR queue).
|
||||
warnings.push(
|
||||
"git ls-remote origin failed; using stale local refs/remotes/origin — " +
|
||||
"branches deleted on the remote may still be counted as claims (run `git fetch --prune origin` to refresh)",
|
||||
);
|
||||
const refs = runCommand("git", [
|
||||
"for-each-ref",
|
||||
"--format=%(refname:short)",
|
||||
"refs/remotes/origin",
|
||||
]);
|
||||
if (refs.ok) {
|
||||
for (const ref of refs.stdout.split("\n").map((r) => r.trim()).filter(Boolean)) {
|
||||
if (ref.endsWith("/HEAD")) continue;
|
||||
const branch = ref.replace(/^origin\//, "");
|
||||
if (branch === baseShort) continue;
|
||||
candidates.push({ branch });
|
||||
}
|
||||
} else {
|
||||
warnings.push("git for-each-ref failed; branch claims unavailable");
|
||||
}
|
||||
}
|
||||
|
||||
// Read each candidate's VERSION through extractVersion so a JSON
|
||||
// version-path (#2501) resolves on remote refs too, and the branch's own
|
||||
// width is preserved in the claim. Reads are LOCAL-first; branches whose
|
||||
// advertised tip has no local object are collected and resolved with ONE
|
||||
// batched shallow fetch below. A per-branch fetch loop here once crawled
|
||||
// a busy remote for minutes on a shallow CI clone (dozens of sequential
|
||||
// network fetches, 10s cap each) — the total network budget must be one
|
||||
// bounded round trip regardless of branch count.
|
||||
const readClaim = (branch: string, sha?: string): "claimed" | "not-a-claim" | "object-missing" => {
|
||||
let show = sha ? runCommand("git", ["show", `${sha}:${versionPath}`]) : { ok: false, stdout: "", stderr: "" };
|
||||
if (!show.ok) {
|
||||
// Live tip not fetched yet (or no sha in the fallback path): best-effort
|
||||
// read from the local remote-tracking ref.
|
||||
show = runCommand("git", ["show", `refs/remotes/origin/${branch}:${versionPath}`]);
|
||||
}
|
||||
if (!show.ok) {
|
||||
if (!sha) return "not-a-claim";
|
||||
// Distinguish "object missing" from "branch has no VERSION file".
|
||||
return runCommand("git", ["cat-file", "-e", sha]).ok ? "not-a-claim" : "object-missing";
|
||||
}
|
||||
const raw = extractVersion(show.stdout, versionPath);
|
||||
if (!raw || !parseVersion(raw)) return "not-a-claim";
|
||||
claims.push({ pr: 0, branch: `origin/${branch}`, version: raw });
|
||||
return "claimed";
|
||||
};
|
||||
|
||||
const pending: { branch: string; sha?: string }[] = [];
|
||||
for (const { branch, sha } of candidates) {
|
||||
if (readClaim(branch, sha) === "object-missing") pending.push({ branch, sha });
|
||||
}
|
||||
if (pending.length > 0) {
|
||||
// ls-remote advertises SHAs without objects: a branch pushed after our
|
||||
// last fetch has NO local object. The pre-#2545 `continue` silently
|
||||
// dropped a LIVE claim — the exact duplicate-allocation this fallback
|
||||
// exists to prevent. One shallow batched fetch (no prompts, no tags,
|
||||
// bounded) brings every missing tip local in a single round trip.
|
||||
spawnSync(
|
||||
"git",
|
||||
["fetch", "origin", ...pending.map((p) => `refs/heads/${p.branch}`), "--depth=1", "--no-tags"],
|
||||
{ encoding: "utf8", timeout: 15000, env: { ...process.env, GIT_TERMINAL_PROMPT: "0" } },
|
||||
);
|
||||
// One unservable ref (dangling sha on the server) fails the WHOLE batch
|
||||
// transfer, so refs still missing get a bounded per-branch retry — that
|
||||
// isolates a poisoned ref without reopening the unbounded fetch crawl
|
||||
// (per-branch-only fetching once ground a shallow CI clone against a
|
||||
// busy remote for minutes). Anything past the cap is warned, not fetched.
|
||||
const RETRY_CAP = 8;
|
||||
let retries = 0;
|
||||
for (const { branch, sha } of pending) {
|
||||
let outcome = readClaim(branch, sha);
|
||||
if (outcome === "object-missing" && retries < RETRY_CAP) {
|
||||
retries++;
|
||||
spawnSync(
|
||||
"git",
|
||||
["fetch", "origin", `refs/heads/${branch}`, "--depth=1", "--no-tags"],
|
||||
{ encoding: "utf8", timeout: 5000, env: { ...process.env, GIT_TERMINAL_PROMPT: "0" } },
|
||||
);
|
||||
outcome = readClaim(branch, sha);
|
||||
}
|
||||
if (outcome === "object-missing") {
|
||||
// STILL unreadable (fetch failed, retry cap hit, or the tip moved
|
||||
// between ls-remote and fetch) — never skip silently. Surface it as
|
||||
// an UNKNOWN claim so the caller knows the allocation may be unsafe.
|
||||
warnings.push(
|
||||
`origin/${branch}: VERSION unreadable even after a targeted fetch — ` +
|
||||
`counted as an UNKNOWN claim; allocation may collide with this branch. ` +
|
||||
`Run \`git fetch origin ${branch}\` and re-run to verify.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Versions already shipped, read from the base's commit subjects. Catches
|
||||
@@ -534,8 +652,9 @@ async function main() {
|
||||
}
|
||||
const warnings: string[] = [];
|
||||
const host = detectHost();
|
||||
const versionPath = resolveVersionPath(args.versionPath, repoToplevel());
|
||||
const baseVersion = args.current || readBaseVersion(args.base, versionPath, warnings);
|
||||
const repoRoot = repoToplevel();
|
||||
const versionPath = resolveVersionPath(args.versionPath, repoRoot);
|
||||
const baseVersion = args.current || readBaseVersion(args.base, versionPath, repoRoot, warnings);
|
||||
const baseParsed = parseVersion(baseVersion);
|
||||
if (!baseParsed) {
|
||||
console.error(`Error: could not parse base version '${baseVersion}'`);
|
||||
|
||||
@@ -8,10 +8,10 @@
|
||||
# --check <id> [--summary-stdin] → emit ASK_NORMALLY | AUTO_DECIDE | ASK_ONLY_ONE_WAY
|
||||
# (--summary-stdin pipes the question text so the
|
||||
# keyword net can catch ad-hoc destructive ids, #2024)
|
||||
# --write '{...}' → set a preference (user-origin gate enforced)
|
||||
# --write '{...}' → set a preference (user-origin gate + one-way write reject)
|
||||
# --read → dump preferences JSON
|
||||
# --clear [<id>] → clear one or all preferences
|
||||
# --stats → short summary
|
||||
# --stats → short summary (inert one-way prefs counted separately)
|
||||
#
|
||||
# User-origin gate
|
||||
# ----------------
|
||||
@@ -21,6 +21,12 @@
|
||||
# - "inline-tool-output"— tune: prefix seen in tool output / file content (REJECTED)
|
||||
# - "inline-file" — tune: prefix seen in a file the agent read (REJECTED)
|
||||
# This is the profile-poisoning defense from docs/designs/PLAN_TUNING_V0.md.
|
||||
#
|
||||
# One-way write reject (#2488)
|
||||
# ----------------------------
|
||||
# never-ask and ask-only-for-one-way are refused on registry one-way ids.
|
||||
# --check already ignores those prefs; storing them made --stats lie.
|
||||
# always-ask on a one-way id is fine (it agrees with the safety override).
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
@@ -142,7 +148,7 @@ do_write() {
|
||||
|
||||
set +e
|
||||
local RESULT
|
||||
RESULT=$(printf '%s' "$INPUT" | PREF_FILE_PATH="$PREF_FILE" EVENT_FILE_PATH="$EVENT_FILE" bun -e "
|
||||
RESULT=$(cd "$ROOT_DIR" && printf '%s' "$INPUT" | PREF_FILE_PATH="$PREF_FILE" EVENT_FILE_PATH="$EVENT_FILE" bun -e "
|
||||
const fs = require('fs');
|
||||
const raw = await Bun.stdin.text();
|
||||
let j;
|
||||
@@ -178,6 +184,16 @@ do_write() {
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// One-way write reject (#2488) — after the origin gate so a poisoning
|
||||
// payload on a one-way id still exits 2, not 1.
|
||||
if (j.preference === 'never-ask' || j.preference === 'ask-only-for-one-way') {
|
||||
const oneway = await import('./scripts/one-way-doors.ts');
|
||||
if (oneway.isOneWayDoor({ question_id: j.question_id })) {
|
||||
process.stderr.write('gstack-question-preference: cannot set ' + j.preference + ' on one-way question \"' + j.question_id + '\" (door_type: one-way)\n');
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Optional free_text — sanitize (no injection patterns, no newlines, <=300 chars)
|
||||
if (j.free_text !== undefined) {
|
||||
if (typeof j.free_text !== 'string') {
|
||||
@@ -267,20 +283,29 @@ do_clear() {
|
||||
# -----------------------------------------------------------------------
|
||||
do_stats() {
|
||||
ensure_file
|
||||
cat "$PREF_FILE" | bun -e "
|
||||
const prefs = JSON.parse(await Bun.stdin.text());
|
||||
const entries = Object.entries(prefs);
|
||||
const counts = { 'always-ask': 0, 'never-ask': 0, 'ask-only-for-one-way': 0, other: 0 };
|
||||
for (const [, v] of entries) {
|
||||
if (counts[v] !== undefined) counts[v]++;
|
||||
else counts.other++;
|
||||
}
|
||||
console.log('TOTAL: ' + entries.length);
|
||||
console.log('ALWAYS_ASK: ' + counts['always-ask']);
|
||||
console.log('NEVER_ASK: ' + counts['never-ask']);
|
||||
console.log('ASK_ONLY_ONE_WAY: ' + counts['ask-only-for-one-way']);
|
||||
if (counts.other) console.log('OTHER: ' + counts.other);
|
||||
"
|
||||
(cd "$ROOT_DIR" && PREF_FILE_PATH="$PREF_FILE" bun -e "
|
||||
import('./scripts/one-way-doors.ts').then((oneway) => {
|
||||
const fs = require('fs');
|
||||
const prefs = JSON.parse(fs.readFileSync(process.env.PREF_FILE_PATH, 'utf-8'));
|
||||
const entries = Object.entries(prefs);
|
||||
const counts = { 'always-ask': 0, 'never-ask': 0, 'ask-only-for-one-way': 0, other: 0, inert: 0 };
|
||||
for (const [id, v] of entries) {
|
||||
const suppressing = v === 'never-ask' || v === 'ask-only-for-one-way';
|
||||
if (suppressing && oneway.isOneWayDoor({ question_id: id })) {
|
||||
counts.inert++;
|
||||
continue;
|
||||
}
|
||||
if (counts[v] !== undefined) counts[v]++;
|
||||
else counts.other++;
|
||||
}
|
||||
console.log('TOTAL: ' + entries.length);
|
||||
console.log('ALWAYS_ASK: ' + counts['always-ask']);
|
||||
console.log('NEVER_ASK: ' + counts['never-ask']);
|
||||
console.log('ASK_ONLY_ONE_WAY: ' + counts['ask-only-for-one-way']);
|
||||
console.log('INERT_ONE_WAY: ' + counts.inert);
|
||||
if (counts.other) console.log('OTHER: ' + counts.other);
|
||||
}).catch(err => { console.error('stats:', err.message); process.exit(1); });
|
||||
")
|
||||
}
|
||||
|
||||
case "$CMD" in
|
||||
|
||||
@@ -199,13 +199,67 @@ function humanTable(findings: Finding[]): string {
|
||||
return rows.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Usage. Exits 0 when asked for (--help), 1 when the invocation was wrong.
|
||||
*
|
||||
* Deliberately NOT 2 or 3: those mean MEDIUM and HIGH findings, and callers
|
||||
* gate dispatch on them (see the exit-code table at the top). A usage error
|
||||
* that exited 2 would be read as "medium findings — prompt the user".
|
||||
*/
|
||||
function printUsage(code: number): never {
|
||||
const out = code === 0 ? process.stdout : process.stderr;
|
||||
out.write(
|
||||
"gstack-redact — scan text for secrets/PII/legal content.\n" +
|
||||
"\n" +
|
||||
"Reads the text to scan from STDIN, or from --from-file PATH. It is a\n" +
|
||||
"filter: with nothing piped in it has nothing to scan.\n" +
|
||||
"\n" +
|
||||
" git diff | gstack-redact --repo-visibility private\n" +
|
||||
" gstack-redact --from-file notes.md --json\n" +
|
||||
"\n" +
|
||||
"Subcommands:\n" +
|
||||
" install-prepush-hook install the managed git pre-push credential guard\n" +
|
||||
" uninstall-prepush-hook remove it\n" +
|
||||
"\n" +
|
||||
"Flags: --json --repo-visibility V --from-file PATH --allowlist PATH\n" +
|
||||
" --self-email EMAIL --repo-public-emails PATH --auto-redact IDS\n" +
|
||||
" --max-bytes N\n" +
|
||||
"\n" +
|
||||
"Exit: 0 clean · 1 usage error · 2 MEDIUM present · 3 HIGH present\n",
|
||||
);
|
||||
process.exit(code);
|
||||
}
|
||||
|
||||
function main() {
|
||||
// Subcommands (positional, not flags).
|
||||
const sub = process.argv[2];
|
||||
if (sub === "install-prepush-hook") return installPrepushHook();
|
||||
if (sub === "uninstall-prepush-hook") return uninstallPrepushHook();
|
||||
if (sub === "--help" || sub === "-h" || sub === "help") return printUsage(0);
|
||||
|
||||
// An unrecognized POSITIONAL is a typo, not input. This used to fall through
|
||||
// to the stdin scan, which on empty stdin prints "(no findings)" and exits 0
|
||||
// — so `install-prepush-hooks` (plural) installed nothing and still looked
|
||||
// like success, leaving the credential guard absent while the operator
|
||||
// believed it was armed. A guard that no-ops must never exit 0.
|
||||
//
|
||||
// "scan" is exempt: the human output header reads "gstack-redact scan —
|
||||
// repo …", so people reasonably type it. It stays an alias for the default.
|
||||
// Flags start with "-" and are parsed further down, so only bare words land
|
||||
// here.
|
||||
if (sub !== undefined && sub !== "scan" && !sub.startsWith("-")) {
|
||||
process.stderr.write(`gstack-redact: unknown subcommand "${sub}"\n\n`);
|
||||
return printUsage(1);
|
||||
}
|
||||
|
||||
const opts = buildOpts();
|
||||
|
||||
// Nothing piped in and no --from-file: readInput() below blocks on
|
||||
// readSync(fd 0) until EOF, which on an interactive terminal never comes.
|
||||
// That prints nothing at all and is indistinguishable from a crash or a
|
||||
// slow scan. Show usage instead of hanging silently.
|
||||
if (!arg("--from-file") && process.stdin.isTTY) return printUsage(1);
|
||||
|
||||
const input = readInput();
|
||||
|
||||
// Auto-redact mode: print redacted body to stdout, diff to stderr, exit 0.
|
||||
|
||||
@@ -54,29 +54,78 @@ fi
|
||||
mkdir -p "$STATE_DIR"
|
||||
|
||||
# ── Acquire lockfile (skip if another session is running setup) ──
|
||||
#
|
||||
# Staleness has two independent detectors (#2613):
|
||||
# 1. PID liveness — the pidfile records the HOLDER subshell's PID and a
|
||||
# dead PID means reclaim. ($BASHPID, never $$: $$ expands to the PARENT
|
||||
# hook's PID even inside this backgrounded subshell, and the parent
|
||||
# exits immediately — so every later session judged the lock stale and
|
||||
# rm -rf'd a LIVE holder's lock, letting concurrent updaters in.)
|
||||
# 2. Hard TTL on the heartbeat mtime — reclaim regardless of kill -0, so a
|
||||
# recycled PID or a hung holder can't wedge the lock forever. The
|
||||
# holder touches the pidfile at step boundaries (after the pull, after
|
||||
# setup), so a legitimately-slow run keeps itself alive. The TTL also
|
||||
# bounds the missing/empty-pidfile states: inside the window they mean
|
||||
# "just acquired, between mkdir and echo" and are respected.
|
||||
LOCK_TTL_MINUTES=30
|
||||
lock_is_expired() {
|
||||
_hb="$LOCK_DIR/pid"
|
||||
[ -f "$_hb" ] || _hb="$LOCK_DIR"
|
||||
[ -n "$(find "$_hb" -maxdepth 0 -mmin +$LOCK_TTL_MINUTES 2>/dev/null)" ]
|
||||
}
|
||||
# Reclaim is TOCTOU-safe via atomic mv-aside: `rm -rf` then `mkdir` lets TWO
|
||||
# contenders both judge the lock stale, both remove it, and both win the
|
||||
# mkdir (one rm can land between the other's rm and mkdir). `mv` of the lock
|
||||
# dir is atomic — exactly one contender's mv succeeds; the loser's mv fails
|
||||
# (ENOENT) and it backs off. The winner reaps the moved-aside dir at leisure.
|
||||
if ! mkdir "$LOCK_DIR" 2>/dev/null; then
|
||||
# Lock exists — check if stale (PID dead)
|
||||
if [ -f "$LOCK_DIR/pid" ]; then
|
||||
if lock_is_expired; then
|
||||
mv "$LOCK_DIR" "$LOCK_DIR.reap.$$" 2>/dev/null || { log_entry "SKIP lock_contested"; exit 0; }
|
||||
rm -rf "$LOCK_DIR.reap.$$" 2>/dev/null
|
||||
mkdir "$LOCK_DIR" 2>/dev/null || { log_entry "SKIP lock_contested"; exit 0; }
|
||||
log_entry "RECLAIMED lock_ttl_expired"
|
||||
elif [ -f "$LOCK_DIR/pid" ]; then
|
||||
LOCK_PID=$(cat "$LOCK_DIR/pid" 2>/dev/null || echo 0)
|
||||
if [ "$LOCK_PID" -gt 0 ] 2>/dev/null && ! kill -0 "$LOCK_PID" 2>/dev/null; then
|
||||
# Stale lock — remove and re-acquire
|
||||
rm -rf "$LOCK_DIR" 2>/dev/null
|
||||
# Stale lock — mv aside atomically (see reclaim note above), re-acquire
|
||||
mv "$LOCK_DIR" "$LOCK_DIR.reap.$$" 2>/dev/null || { log_entry "SKIP lock_contested"; exit 0; }
|
||||
rm -rf "$LOCK_DIR.reap.$$" 2>/dev/null
|
||||
mkdir "$LOCK_DIR" 2>/dev/null || { log_entry "SKIP lock_contested"; exit 0; }
|
||||
else
|
||||
# Live holder — or an empty/non-numeric pidfile inside the TTL
|
||||
# window (the -gt test fails on garbage, landing here by design).
|
||||
log_entry "SKIP locked_by=$LOCK_PID"
|
||||
exit 0
|
||||
fi
|
||||
else
|
||||
# Missing pidfile inside the TTL window: just-acquired (mkdir→echo race).
|
||||
log_entry "SKIP locked_no_pid"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# Write PID for stale lock detection
|
||||
echo $$ > "$LOCK_DIR/pid" 2>/dev/null
|
||||
# Write the HOLDER's PID for stale lock detection (see #2613 note above;
|
||||
# macOS ships bash 3.2 with no BASHPID — the sh child's $PPID IS this
|
||||
# subshell, so the fallback is exact there). MYPID is captured once at
|
||||
# write time so the trap below can prove ownership before removing.
|
||||
MYPID="${BASHPID:-$(sh -c 'echo $PPID')}"
|
||||
echo "$MYPID" > "$LOCK_DIR/pid" 2>/dev/null
|
||||
|
||||
# Clean up lock on exit
|
||||
trap 'rm -rf "$LOCK_DIR" 2>/dev/null' EXIT
|
||||
# In-flight heartbeat: the step-boundary touches below only fire AFTER the
|
||||
# pull / setup return, so a legitimately-slow step (cold clone, huge setup)
|
||||
# older than the TTL got reclaimed while ALIVE. This background loop
|
||||
# freshens the pidfile mtime every 5 minutes for as long as we still own
|
||||
# the lock (ownership re-checked each beat: if another updater reclaimed
|
||||
# and wrote its own pid, the loop exits instead of touching THEIR file).
|
||||
( while :; do sleep 300; [ "$(cat "$LOCK_DIR/pid" 2>/dev/null)" = "$MYPID" ] || exit 0; touch "$LOCK_DIR/pid" 2>/dev/null; done ) &
|
||||
HB_PID=$!
|
||||
|
||||
# Clean up lock on exit — ownership-checked: after a TTL reclaim by another
|
||||
# updater, $LOCK_DIR belongs to the NEW holder, and an unconditional rm -rf
|
||||
# here would delete the live holder's lock (cascading reclaims). Remove the
|
||||
# lock ONLY while $LOCK_DIR/pid still contains MYPID; always stop the
|
||||
# heartbeat.
|
||||
trap 'kill "$HB_PID" 2>/dev/null; [ "$(cat "$LOCK_DIR/pid" 2>/dev/null)" = "$MYPID" ] && rm -rf "$LOCK_DIR" 2>/dev/null' EXIT
|
||||
|
||||
# ── Pull latest ──
|
||||
OLD_HEAD=$(git -C "$GSTACK_DIR" rev-parse HEAD 2>/dev/null)
|
||||
@@ -94,6 +143,9 @@ fi
|
||||
PULL_EXIT=$?
|
||||
NEW_HEAD=$(git -C "$GSTACK_DIR" rev-parse HEAD 2>/dev/null)
|
||||
|
||||
# Heartbeat: pull done — keep the TTL clock fresh for the setup step.
|
||||
touch "$LOCK_DIR/pid" 2>/dev/null
|
||||
|
||||
# Record check time regardless of outcome
|
||||
date +%s > "$THROTTLE_FILE" 2>/dev/null
|
||||
|
||||
@@ -132,6 +184,8 @@ fi
|
||||
( cd "$GSTACK_DIR" && ./setup -q ) >/dev/null 2>&1 || {
|
||||
log_entry "SETUP_FAILED"
|
||||
}
|
||||
# Heartbeat: setup done (either way) — refresh the TTL clock.
|
||||
touch "$LOCK_DIR/pid" 2>/dev/null
|
||||
else
|
||||
log_entry "SETUP_SKIPPED bun_missing"
|
||||
fi
|
||||
|
||||
+113
-16
@@ -10,13 +10,24 @@
|
||||
# 2. Schema-aware (plan-tune cathedral T3 — supports PreToolUse + PostToolUse):
|
||||
# gstack-settings-hook add-event --event <SessionStart|PreToolUse|PostToolUse> \
|
||||
# --command <cmd> --source <tag> [--matcher <regex>] [--timeout <s>]
|
||||
# gstack-settings-hook ensure-event --event ... --command ... --source ... [--matcher ...] [--timeout <s>]
|
||||
# gstack-settings-hook remove-source --source <tag>
|
||||
# gstack-settings-hook diff-event --event ... --command ... --source ... [--matcher ...]
|
||||
# gstack-settings-hook rollback # restore latest backup
|
||||
# gstack-settings-hook list-sources # show all gstack-tagged hook entries
|
||||
#
|
||||
# ensure-event is the update-in-place verb: same flags as add-event, but it
|
||||
# first compares the REGISTERED payload for (event, matcher, source) against
|
||||
# the requested one. Identical → no write, no backup ("unchanged"). Different
|
||||
# → the single matching entry is replaced via one atomic tmp+rename, so a
|
||||
# failed update can never leave zero or two registrations. This is what heals
|
||||
# a stale absolute hook path (e.g. a deleted dev worktree) baked into
|
||||
# settings.json by an earlier setup — presence-only dedup never re-pointed it.
|
||||
#
|
||||
# Every add-event/remove-source writes a backup to ~/.claude/settings.json.bak.<ts>
|
||||
# before mutating (Codex correction — silent settings.json mutation is wrong).
|
||||
# before mutating (Codex correction — silent settings.json mutation is wrong);
|
||||
# ensure-event backs up only when it actually mutates, so a no-op re-run of
|
||||
# ./setup doesn't churn backup files.
|
||||
#
|
||||
# Dedup: legacy `add`/`remove` dedupe by the historical `gstack-session-update`
|
||||
# substring. Schema-aware `add-event` dedupes by (event, matcher, _gstack_source) so
|
||||
@@ -34,6 +45,7 @@ Usage:
|
||||
gstack-settings-hook add <hook-command> # legacy SessionStart add
|
||||
gstack-settings-hook remove <hook-command> # legacy SessionStart remove
|
||||
gstack-settings-hook add-event --event <name> --command <cmd> --source <tag> [--matcher <re>] [--timeout <s>]
|
||||
gstack-settings-hook ensure-event --event <name> --command <cmd> --source <tag> [--matcher <re>] [--timeout <s>]
|
||||
gstack-settings-hook remove-source --source <tag>
|
||||
gstack-settings-hook diff-event --event <name> --command <cmd> --source <tag> [--matcher <re>] [--timeout <s>]
|
||||
gstack-settings-hook rollback
|
||||
@@ -71,7 +83,17 @@ case "$ACTION" in
|
||||
const settingsPath = process.env.GSTACK_SETTINGS_PATH;
|
||||
const hookCmd = process.env.GSTACK_HOOK_CMD;
|
||||
let settings = {};
|
||||
try { settings = JSON.parse(fs.readFileSync(settingsPath, "utf8")); } catch {}
|
||||
// An EXISTING file that does not parse must never be rewritten: the
|
||||
// old catch{} folded it to {} and the atomic write below replaced the
|
||||
// user permissions/env/other hooks with just ours. Refuse loudly.
|
||||
if (fs.existsSync(settingsPath)) {
|
||||
try { settings = JSON.parse(fs.readFileSync(settingsPath, "utf8")); }
|
||||
catch (e) {
|
||||
console.error("error: " + settingsPath + " exists but is not valid JSON (" +
|
||||
(e && e.message ? e.message : e) + "); refusing to rewrite it. Fix or move the file, then re-run.");
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
if (!settings.hooks) settings.hooks = {};
|
||||
if (!settings.hooks.SessionStart) settings.hooks.SessionStart = [];
|
||||
const exists = settings.hooks.SessionStart.some(entry =>
|
||||
@@ -82,10 +104,10 @@ case "$ACTION" in
|
||||
hooks: [{ type: "command", command: hookCmd }]
|
||||
});
|
||||
}
|
||||
const tmp = settingsPath + ".tmp";
|
||||
const tmp = settingsPath + ".tmp." + process.pid; // per-process: parallel writers must not share a tmp
|
||||
fs.writeFileSync(tmp, JSON.stringify(settings, null, 2) + "\n");
|
||||
fs.renameSync(tmp, settingsPath);
|
||||
' 2>/dev/null
|
||||
'
|
||||
;;
|
||||
|
||||
remove)
|
||||
@@ -108,13 +130,13 @@ case "$ACTION" in
|
||||
if (settings.hooks.SessionStart.length === 0) delete settings.hooks.SessionStart;
|
||||
if (Object.keys(settings.hooks).length === 0) delete settings.hooks;
|
||||
}
|
||||
const tmp = settingsPath + ".tmp";
|
||||
const tmp = settingsPath + ".tmp." + process.pid; // per-process: parallel writers must not share a tmp
|
||||
fs.writeFileSync(tmp, JSON.stringify(settings, null, 2) + "\n");
|
||||
fs.renameSync(tmp, settingsPath);
|
||||
' 2>/dev/null
|
||||
;;
|
||||
|
||||
add-event|diff-event)
|
||||
add-event|diff-event|ensure-event)
|
||||
EVENT=""
|
||||
COMMAND=""
|
||||
SOURCE=""
|
||||
@@ -132,7 +154,7 @@ case "$ACTION" in
|
||||
esac
|
||||
done
|
||||
if [ -z "$EVENT" ] || [ -z "$COMMAND" ] || [ -z "$SOURCE" ]; then
|
||||
echo "add-event/diff-event require --event, --command, --source" >&2
|
||||
echo "add-event/ensure-event/diff-event require --event, --command, --source" >&2
|
||||
exit 1
|
||||
fi
|
||||
case "$EVENT" in
|
||||
@@ -144,6 +166,8 @@ case "$ACTION" in
|
||||
fi
|
||||
DIFF_ONLY=""
|
||||
if [ "$ACTION" = "diff-event" ]; then DIFF_ONLY=1; fi
|
||||
ENSURE=""
|
||||
if [ "$ACTION" = "ensure-event" ]; then ENSURE=1; fi
|
||||
GSTACK_SETTINGS_PATH="$SETTINGS_FILE" \
|
||||
GSTACK_EVENT="$EVENT" \
|
||||
GSTACK_COMMAND="$COMMAND" \
|
||||
@@ -151,6 +175,7 @@ case "$ACTION" in
|
||||
GSTACK_MATCHER="$MATCHER" \
|
||||
GSTACK_TIMEOUT="$TIMEOUT" \
|
||||
GSTACK_DIFF_ONLY="$DIFF_ONLY" \
|
||||
GSTACK_ENSURE="$ENSURE" \
|
||||
bun -e '
|
||||
const fs = require("fs");
|
||||
const settingsPath = process.env.GSTACK_SETTINGS_PATH;
|
||||
@@ -160,23 +185,52 @@ case "$ACTION" in
|
||||
const matcher = process.env.GSTACK_MATCHER || "";
|
||||
const timeoutRaw = process.env.GSTACK_TIMEOUT || "";
|
||||
const diffOnly = process.env.GSTACK_DIFF_ONLY === "1";
|
||||
const ensure = process.env.GSTACK_ENSURE === "1";
|
||||
|
||||
let settings = {};
|
||||
try { settings = JSON.parse(fs.readFileSync(settingsPath, "utf8")); } catch {}
|
||||
// An EXISTING file that does not parse must never be rewritten: the
|
||||
// old catch{} folded it to {} and the atomic write below replaced the
|
||||
// user permissions/env/other hooks with just ours. Refuse loudly.
|
||||
if (fs.existsSync(settingsPath)) {
|
||||
try { settings = JSON.parse(fs.readFileSync(settingsPath, "utf8")); }
|
||||
catch (e) {
|
||||
console.error("error: " + settingsPath + " exists but is not valid JSON (" +
|
||||
(e && e.message ? e.message : e) + "); refusing to rewrite it. Fix or move the file, then re-run.");
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
const before = JSON.stringify(settings, null, 2);
|
||||
|
||||
if (!settings.hooks) settings.hooks = {};
|
||||
if (!settings.hooks[event]) settings.hooks[event] = [];
|
||||
|
||||
// Identity key is (event, source): any existing entry carrying OUR
|
||||
// source tag for this event IS the entry to compare/update — a matcher
|
||||
// change must update it in place, never push a SECOND gstack entry
|
||||
// (the old key included the matcher, so a future matcher change would
|
||||
// have duplicated the registration). Untagged legacy entries are still
|
||||
// adopted when both matcher and command line up.
|
||||
const matchesEntry = (entry) => {
|
||||
if (entry._gstack_source === source) return true;
|
||||
const sameMatcher = (entry.matcher || "") === matcher;
|
||||
const sameCommand = entry.hooks && entry.hooks[0] && entry.hooks[0].command === cmd;
|
||||
const sameSource = entry._gstack_source === source;
|
||||
return sameMatcher && (sameSource || sameCommand);
|
||||
return sameMatcher && sameCommand;
|
||||
};
|
||||
|
||||
let existing = settings.hooks[event].find(matchesEntry);
|
||||
// Collect ALL matches, not just the first: pre-existing installs can
|
||||
// carry two entries with the same (event, _gstack_source) from the old
|
||||
// matcher-keyed dedup. `.find()` updated only the first and left the
|
||||
// stale twin running forever. Keep ONE canonical entry (the first),
|
||||
// remove the rest in the same atomic write.
|
||||
const matched = settings.hooks[event].filter(matchesEntry);
|
||||
let existing = matched.length > 0 ? matched[0] : undefined;
|
||||
let collapsed = 0;
|
||||
if (matched.length > 1) {
|
||||
const extras = new Set(matched.slice(1));
|
||||
settings.hooks[event] = settings.hooks[event].filter((e) => !extras.has(e));
|
||||
collapsed = matched.length - 1;
|
||||
}
|
||||
const hookEntry = { type: "command", command: cmd };
|
||||
if (timeoutRaw) {
|
||||
const n = Number(timeoutRaw);
|
||||
@@ -186,6 +240,10 @@ case "$ACTION" in
|
||||
if (existing) {
|
||||
existing.hooks = [hookEntry];
|
||||
existing._gstack_source = source;
|
||||
// Keep the matcher current too — under the (event, source) key the
|
||||
// matched entry may carry a stale matcher.
|
||||
if (matcher) existing.matcher = matcher;
|
||||
else delete existing.matcher;
|
||||
} else {
|
||||
const newEntry = { _gstack_source: source, hooks: [hookEntry] };
|
||||
if (matcher) newEntry.matcher = matcher;
|
||||
@@ -202,10 +260,49 @@ case "$ACTION" in
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const tmp = settingsPath + ".tmp";
|
||||
fs.writeFileSync(tmp, after + "\n");
|
||||
fs.renameSync(tmp, settingsPath);
|
||||
console.log("OK: " + event + " hook registered (source: " + source + ")");
|
||||
if (ensure && before === after) {
|
||||
// Registered payload already matches the canonical one — no write, no
|
||||
// backup, no churn. Re-running ./setup stays a true no-op.
|
||||
console.log("OK: " + event + " hook unchanged (source: " + source + ")");
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
try {
|
||||
if (ensure && fs.existsSync(settingsPath)) {
|
||||
// Mirrors backup_settings (bash) — but only when a write actually
|
||||
// happens, so a no-op ensure-event never creates backup files.
|
||||
const d = new Date();
|
||||
const pad = (n) => String(n).padStart(2, "0");
|
||||
const ts = "" + d.getFullYear() + pad(d.getMonth() + 1) + pad(d.getDate()) +
|
||||
"-" + pad(d.getHours()) + pad(d.getMinutes()) + pad(d.getSeconds());
|
||||
fs.copyFileSync(settingsPath, settingsPath + ".bak." + ts);
|
||||
fs.writeFileSync(settingsPath + ".bak-latest", settingsPath + ".bak." + ts + "\n");
|
||||
}
|
||||
|
||||
// Atomic tmp+rename: the settings file is either the old JSON (with
|
||||
// the old single registration) or the new JSON (with the replaced
|
||||
// one) — a failed update can never leave zero or two registrations.
|
||||
// Per-process tmp suffix: a fixed settings.json.tmp let two parallel
|
||||
// writers consume one another. (No apostrophes here: this JS lives
|
||||
// inside a bash single-quoted string.)
|
||||
const tmp = settingsPath + ".tmp." + process.pid;
|
||||
fs.writeFileSync(tmp, after + "\n");
|
||||
fs.renameSync(tmp, settingsPath);
|
||||
} catch (e) {
|
||||
// Explicit catch + exit 1: bun -e has been observed (1.3.13) to turn
|
||||
// an uncaught sync fs error into a SILENT exit 0, which would let a
|
||||
// failed update masquerade as success to the caller.
|
||||
console.error("error: could not update " + settingsPath + ": " + (e && e.message ? e.message : e));
|
||||
process.exit(1);
|
||||
}
|
||||
if (collapsed > 0) {
|
||||
console.error("collapsed " + collapsed + " duplicate (event, source) hook entr" + (collapsed === 1 ? "y" : "ies") + " for " + event + " (source: " + source + ")");
|
||||
}
|
||||
if (ensure && existing) {
|
||||
console.log("OK: " + event + " hook re-pointed (source: " + source + ")");
|
||||
} else {
|
||||
console.log("OK: " + event + " hook registered (source: " + source + ")");
|
||||
}
|
||||
'
|
||||
;;
|
||||
|
||||
@@ -239,7 +336,7 @@ case "$ACTION" in
|
||||
if (settings.hooks[event].length === 0) delete settings.hooks[event];
|
||||
}
|
||||
if (Object.keys(settings.hooks).length === 0) delete settings.hooks;
|
||||
const tmp = settingsPath + ".tmp";
|
||||
const tmp = settingsPath + ".tmp." + process.pid; // per-process: parallel writers must not share a tmp
|
||||
fs.writeFileSync(tmp, JSON.stringify(settings, null, 2) + "\n");
|
||||
fs.renameSync(tmp, settingsPath);
|
||||
console.log("OK: removed " + removed + " hook entry/entries tagged source=" + source);
|
||||
|
||||
+100
-13
@@ -13,11 +13,30 @@
|
||||
# Without this walk-up, running gstack-slug from a subdir whose only
|
||||
# "marker" is a deploy artifact silently resolves to the subdir's basename,
|
||||
# misfiling all session state under a phantom slug. (2026-05-25 bug fix.)
|
||||
# 2. If the resolved project root has a git remote, derive the slug from it.
|
||||
# 2. Derive the slug from the canonical git remote: the OUTERMOST ancestor
|
||||
# that is an actual git repo (`.git` directory, or `.git` FILE for
|
||||
# worktrees/submodules) with an `origin` remote wins. The slug is
|
||||
# `owner-repo`, parsed EXACTLY like browse/bin/remote-slug so the two
|
||||
# bins can never disagree on a canonical-remote repo. Marker-only
|
||||
# ancestors that are NOT remote-bearing repos (a stray empty ~/.git,
|
||||
# a stray package.json in $HOME) still anchor the basename FALLBACK,
|
||||
# but they can no longer shadow a real remote. (2026-08-17 bug fix:
|
||||
# a stray empty ~/.git made the walk-up pick $HOME as project root;
|
||||
# $HOME has no origin, so EVERY repo under it degraded to
|
||||
# SLUG=<username> — one shared bucket for all projects.)
|
||||
# 3. Otherwise use the basename of the resolved project root.
|
||||
# 4. If no project root was found anywhere on the chain, fall back to the
|
||||
# basename of $(pwd) (preserves prior behavior for plain folders).
|
||||
#
|
||||
# MIGRATION NOTE (2026-08-17): sessions run BEFORE the remote-first fix above,
|
||||
# in repos below a stray marker-bearing ancestor, filed their session state
|
||||
# (decisions / timeline / ceo-plans / learnings) under the DEGRADED slug —
|
||||
# ~/.gstack/projects/<ancestor-basename>/ (e.g. `garrytan`) instead of the
|
||||
# canonical ~/.gstack/projects/<owner-repo>/. The slug cache self-heals on the
|
||||
# next invocation (see 1b), but already-written store data does NOT move.
|
||||
# Whether/how to merge those stores is tracked in TODOS.md — do not add data
|
||||
# migration code here.
|
||||
#
|
||||
# Caching is self-healing: a cache entry for the literal pwd that differs from
|
||||
# the freshly-computed slug gets opportunistically rewritten (single-shot, key-
|
||||
# local — never sweeps other entries). This lets pre-existing poisoned caches
|
||||
@@ -112,6 +131,32 @@ _outermost_project_root() {
|
||||
fi
|
||||
}
|
||||
|
||||
# 1a. Outermost REMOTE-BEARING repo root: walk the same ancestor chain and
|
||||
# track the outermost dir that has a `.git` entry (directory for normal
|
||||
# clones, FILE for git-worktrees/submodules — `git -C` resolves a
|
||||
# worktree's remote through its main clone) AND whose `origin` remote
|
||||
# resolves. This is the canonical-identity walk: a marker-only ancestor
|
||||
# with no resolvable origin (stray empty ~/.git, stray package.json)
|
||||
# cannot win here, so it cannot hijack remote-derived identity the way
|
||||
# it can hijack the marker walk above. Nested-repo semantics preserved:
|
||||
# an inner repo under an outer canonical-remote repo still resolves to
|
||||
# the OUTER repo's remote (outermost wins), same as before.
|
||||
# Note: git spawns only at `.git`-bearing ancestors — typically one.
|
||||
_outermost_remote_repo() {
|
||||
local dir="$1"
|
||||
local outermost="" parent="" depth=0
|
||||
while [[ -n "$dir" && "$dir" != "/" && $depth -lt 64 ]]; do
|
||||
if [[ -e "$dir/.git" ]] && git -C "$dir" remote get-url origin >/dev/null 2>&1; then
|
||||
outermost="$dir"
|
||||
fi
|
||||
parent=$(dirname "$dir")
|
||||
[[ "$parent" == "$dir" ]] && break # dirname fixed point (C:/, ., //srv)
|
||||
dir="$parent"
|
||||
depth=$((depth + 1))
|
||||
done
|
||||
printf '%s' "$outermost"
|
||||
}
|
||||
|
||||
# Only compute the project root if we don't already have a slug (env override
|
||||
# took precedence). The walk is cheap (~10 stats on the deepest realistic cwd).
|
||||
PROJECT_ROOT=""
|
||||
@@ -119,35 +164,77 @@ if [[ -z "$SLUG" ]]; then
|
||||
PROJECT_ROOT=$(_outermost_project_root "$PROJECT_DIR")
|
||||
fi
|
||||
|
||||
# Lazy, memoized remote discovery. Needed on exactly two paths: fresh
|
||||
# resolution (no usable cache) and the degraded-ancestor heal check below.
|
||||
# Gating it keeps ordinary cache hits git-spawn-free.
|
||||
REMOTE_ROOT=""
|
||||
REMOTE_URL=""
|
||||
_REMOTE_RESOLVED=0
|
||||
_resolve_remote() {
|
||||
if [[ "$_REMOTE_RESOLVED" -eq 1 ]]; then return 0; fi
|
||||
_REMOTE_RESOLVED=1
|
||||
REMOTE_ROOT=$(_outermost_remote_repo "$PROJECT_DIR")
|
||||
if [[ -n "$REMOTE_ROOT" ]]; then
|
||||
REMOTE_URL=$(git -C "$REMOTE_ROOT" remote get-url origin 2>/dev/null) || REMOTE_URL=""
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# 1b. Cached identity is STICKY (#2212): a project that used gstack before it
|
||||
# adopted a git remote keeps its pre-origin slug — recomputing from the
|
||||
# remote here would rename the project mid-life and orphan everything
|
||||
# under ~/.gstack/projects/<slug>/. The ONE exception is the provable
|
||||
# old-bug shape (#1125): the pre-walk-up resolver cached basename(pwd)
|
||||
# for a SUBDIRECTORY of the real project — if the cached value equals this
|
||||
# pwd's basename while the walk-up says pwd is NOT the project root, the
|
||||
# cache came from that bug, not from legitimate identity; fall through and
|
||||
# recompute so it heals.
|
||||
# under ~/.gstack/projects/<slug>/. TWO provable bug shapes are exempt
|
||||
# and fall through to recompute (self-heal):
|
||||
# - Old-bug shape (#1125): the pre-walk-up resolver cached basename(pwd)
|
||||
# for a SUBDIRECTORY of the real project — cached == pwd basename while
|
||||
# the walk-up says pwd is NOT the project root.
|
||||
# - Degraded-ancestor shape (2026-08-17), STRAY-REPO shape ONLY: the
|
||||
# pre-remote-first resolver cached basename(PROJECT_ROOT) for an
|
||||
# ancestor anchored by a .git entry whose origin does NOT resolve (the
|
||||
# stray empty ~/.git live bug) while a remote-bearing repo BELOW it
|
||||
# exists. A marker root anchored by package.json / pyproject etc. with
|
||||
# NO .git is legit #2212 sticky identity (a monorepo wrapper that used
|
||||
# gstack before its inner dir grew a remote) and must NOT be healed.
|
||||
# Legit remote-adopting stickiness is safe too: there the repo that
|
||||
# adopted the remote IS the marker root (REMOTE_ROOT == PROJECT_ROOT),
|
||||
# so the heal never fires.
|
||||
if [[ -z "$SLUG" && -f "$CACHE_FILE" ]]; then
|
||||
_CACHED=$(cat "$CACHE_FILE" 2>/dev/null | tr -cd 'a-zA-Z0-9._-')
|
||||
if [[ -n "$_CACHED" ]]; then
|
||||
_PWD_BASE=$(basename "$PROJECT_DIR" | tr -cd 'a-zA-Z0-9._-')
|
||||
_ROOT_BASE=""
|
||||
if [[ -n "$PROJECT_ROOT" ]]; then
|
||||
_ROOT_BASE=$(basename "$PROJECT_ROOT" | tr -cd 'a-zA-Z0-9._-')
|
||||
fi
|
||||
if [[ "$_CACHED" == "$_PWD_BASE" && -n "$PROJECT_ROOT" && "$PROJECT_ROOT" != "$PROJECT_DIR" ]]; then
|
||||
: # old-bug shape — recompute below and self-heal the cache
|
||||
elif [[ -n "$PROJECT_ROOT" && "$_CACHED" == "$_ROOT_BASE" && -e "$PROJECT_ROOT/.git" ]] \
|
||||
&& ! git -C "$PROJECT_ROOT" remote get-url origin >/dev/null 2>&1 \
|
||||
&& { _resolve_remote; [[ -n "$REMOTE_URL" && "$REMOTE_ROOT" != "$PROJECT_ROOT" ]]; }; then
|
||||
: # degraded-ancestor (stray-repo) shape — recompute below and self-heal the cache
|
||||
else
|
||||
SLUG="$_CACHED"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# 2. If we found a project root and it has a git remote, derive slug from the
|
||||
# remote URL (existing logic — kept verbatim, just rooted at PROJECT_ROOT
|
||||
# instead of $PWD so a subdir without its own remote inherits the parent's).
|
||||
if [[ -z "$SLUG" && -n "$PROJECT_ROOT" ]]; then
|
||||
REMOTE_URL=$(git -C "$PROJECT_ROOT" remote get-url origin 2>/dev/null) || REMOTE_URL=""
|
||||
# 2. Canonical remote-derived slug. Sourced from the outermost remote-bearing
|
||||
# repo (see 1a) — NOT from PROJECT_ROOT, which may be a marker-only
|
||||
# ancestor with no remote. Parse kept byte-identical to
|
||||
# browse/bin/remote-slug (strip trailing .git, then owner/repo → owner-repo)
|
||||
# so the two bins agree on every canonical-remote repo, worktrees included.
|
||||
# Parity pinned by test/gstack-slug-parity.test.ts.
|
||||
if [[ -z "$SLUG" ]]; then
|
||||
_resolve_remote
|
||||
if [[ -n "$REMOTE_URL" ]]; then
|
||||
RAW_SLUG=$(printf '%s' "$REMOTE_URL" | sed 's|.*[:/]\([^/]*/[^/]*\)\.git$|\1|;s|.*[:/]\([^/]*/[^/]*\)$|\1|' | tr '/' '-')
|
||||
RAW_SLUG=$(printf '%s' "${REMOTE_URL%.git}" | sed -E 's#.*[:/]([^/]+)/([^/]+)$#\1-\2#')
|
||||
SLUG=$(printf '%s' "$RAW_SLUG" | tr -cd 'a-zA-Z0-9._-')
|
||||
# Dot-only / degenerate guard: a hostile origin like `url = ..` (git
|
||||
# accepts it) passes sed unchanged and would become SLUG=".." — filing
|
||||
# state one level ABOVE ~/.gstack/projects/. Reject empty/"."/".."/
|
||||
# slash-bearing slugs and fall through to the basename fallback below.
|
||||
# (tr -cd already deletes "/", so */* is belt-and-braces.)
|
||||
case "$SLUG" in ""|.|..|*/*) SLUG="" ;; esac
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -313,6 +313,11 @@ function cmdClassify(args: string[], cwd: string): void {
|
||||
// the version itself.
|
||||
const expectedPkg = jsonSource ? current : npmVersion(current);
|
||||
const state = classifyState(current, baseV, pkg.exists, pkg.version, expectedPkg);
|
||||
// Surface version-file absence so callers (and /ship) can tell "version is
|
||||
// genuinely 0.0.0.0" from "we made up 0.0.0.0 because the file is missing"
|
||||
// (#2600). Without this, the DRIFT_STALE_PKG dispatch on a missing VERSION
|
||||
// would feed repair a fabricated version that passes the shape check.
|
||||
const versionFileExists = existsSync(versionPath);
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
state,
|
||||
@@ -322,6 +327,7 @@ function cmdClassify(args: string[], cwd: string): void {
|
||||
pkgExists: pkg.exists,
|
||||
pkgPath: pkg.exists ? relative(cwd, pkgPath) : null,
|
||||
expectedPkgVersion: pkg.exists ? expectedPkg : null,
|
||||
versionFileExists,
|
||||
}) + "\n",
|
||||
);
|
||||
// DRIFT_UNEXPECTED is a real, decidable state — the caller stops on it, but the
|
||||
@@ -442,7 +448,41 @@ function cmdRepair(args: string[], cwd: string): void {
|
||||
);
|
||||
return;
|
||||
}
|
||||
// Guard: if the VERSION file does not exist, readVersionFile folds that into
|
||||
// DEFAULT ("0.0.0.0") — a structurally valid but fabricated version. The
|
||||
// shape check below (VERSION_RE) would pass it, and we would write 0.0.0
|
||||
// into package.json, regressing it below where it started (#2600).
|
||||
if (!existsSync(versionPath)) {
|
||||
fail(
|
||||
`VERSION file not found at ${versionRel}. ` +
|
||||
"Cannot repair package.json without a real version to sync. " +
|
||||
"Pass --version-path or set .gstack/version-path if the file lives elsewhere.",
|
||||
2,
|
||||
);
|
||||
}
|
||||
const current = readVersionFile(versionPath, versionRel);
|
||||
// Guard against readVersionFile folding "file exists but is empty / unparsable"
|
||||
// into DEFAULT ("0.0.0.0") — same data-corruption pathway as file-missing (#2600).
|
||||
// A fabricated version must never propagate into package.json. But DEFAULT is
|
||||
// ambiguous: a VERSION file that GENUINELY reads "0.0.0.0" (a brand-new repo)
|
||||
// is a legitimate version, not the sentinel. Disambiguate on the raw bytes:
|
||||
// if the trimmed file content itself matches the version shape, proceed with
|
||||
// the repair; reject only when the raw content is empty or unparseable.
|
||||
if (current === DEFAULT) {
|
||||
let rawTrimmed = "";
|
||||
try {
|
||||
rawTrimmed = readFileSync(versionPath, "utf-8").trim();
|
||||
} catch {
|
||||
rawTrimmed = "";
|
||||
}
|
||||
if (!VERSION_RE.test(rawTrimmed)) {
|
||||
fail(
|
||||
`VERSION file at ${versionRel} is empty or contains no parsable version. ` +
|
||||
"Cannot repair package.json with a fabricated version.",
|
||||
2,
|
||||
);
|
||||
}
|
||||
}
|
||||
if (!VERSION_RE.test(current)) {
|
||||
fail(
|
||||
`VERSION file contents (${current}) do not match MAJOR.MINOR.PATCH[.MICRO]. ` +
|
||||
|
||||
Reference in New Issue
Block a user