mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-08 14:34:49 +02:00
f44de365c55c78f9dcb0b26f40647d82d9c93706
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Garry Tan
|
c7aefc1abd |
v1.26.5.0 fix wave: gbrain ingest writer (hybrid frontmatter) + gbrain-valid source ids (#1344)
* fix: use correct `gbrain put <slug>` CLI verb in memory ingest
`put_page` is the MCP tool name, not a CLI subcommand. The actual
gbrain verb is `put <slug>` with content via stdin and tags in YAML
frontmatter. Every transcript / memory ingest fails today on clean
installs.
Switch to the right verb and inject title/type/tags into the
frontmatter that buildTranscriptPage / buildArtifactPage already
produce.
Bundled in the same function:
- timeout: 30s → 60s. Auto-link reconciliation hits 30s once the
brain has a few hundred pages.
- maxBuffer: 1MB → 16MB. Without it Node truncates gbrain's stderr
and callers see only `Command failed:` with no detail.
- Surface stderr/stdout in the returned error instead of the bare
exception.
Verified: bun test test/gstack-memory-ingest.test.ts -> 15/15 pass.
bun test on the three test files touching this path -> 362/362.
* fix(sync-gbrain): generate gbrain-valid source ids for repos with dots or long names
`deriveCodeSourceId` previously concatenated the canonicalized remote with only `/`
and whitespace stripped, leaving dots from hostnames (`github.com`) and no length
cap. gbrain rejects any source id containing characters outside [a-z0-9-] or longer
than 32 chars, so `github.com/<org>/<repo>` produced `gstack-code-github.com-<org>-<repo>`
(40 chars, plus dots) and registration failed:
code source registration failed: Invalid source id
"gstack-code-github.com-radubach-platform". Must be 1-32 lowercase alnum
chars with optional interior hyphens.
Fix:
- Drop the host segment (`github.com` is the same for nearly every user and just
consumes the 32-char budget). Use only the last two path segments (org-repo).
- Sanitize any remaining non-alnum to hyphens, then collapse and trim.
- For genuinely long org/repo names that still exceed the budget, keep the tail
(most distinctive end of the slug) and append a 6-char sha1 hash for collision
resistance.
Adds a regression test that spawns the CLI in temp git repos with controlled
remotes (dot in hostname, SCP-style, multi-dot host, long names forcing
hash-truncation) and asserts every derived id is ≤32 chars and matches the
gbrain validator regex.
* fix(memory-ingest): hybrid frontmatter writer + tightened gbrain availability probe
PR #1328 (merged in the prior commit) correctly injects title/type/tags
into the YAML frontmatter that buildTranscriptPage already prepends. But
buildArtifactPage emits raw markdown without frontmatter, so design-docs,
learnings, and builder-profile-entries were landing in gbrain with empty
title/type/tags. Add the no-frontmatter wrap branch so artifact pages get
the same metadata the inject branch provides for transcripts.
Also bring in gbrainAvailable()'s --help probe (originally proposed in
PR #1341 by Alex Medina), with the regex tightened from /(^|\s)put(\s|$)/m
to /^\s+put\s/m. Anchoring on the indented subcommand format gbrain's
help actually uses keeps the probe from matching "put" appearing as
prose in help text, while still failing fast with one clean error if a
future gbrain renames or removes the put subcommand.
Updates the V1.5 NOTE doc block at the top of the file to describe the
current put-via-stdin shape rather than the legacy put_page flag form.
Co-Authored-By: Alex Medina <oficina@puntoverdemc.com>
* test+fix(memory-ingest): strengthen regression tests, fix inject for malformed-close frontmatter
Imports the shim-based regression tests from PR #1341 (Alex Medina) and
strengthens them to assert title, type, and tags actually arrive in put
stdin — not just `agent: claude-code`. Asserting the metadata fields
matches the regression class that's caused this fix wave: writers can
"succeed" while metadata is silently lost. The original PR #1341 tests
would have passed even with title/type/tags missing.
Strengthening the test surfaced a deeper issue. buildTranscriptPage joins
frontmatter array elements with "\n" and does not append a trailing
newline, so the close fence is "\n---<content>" directly, not "\n---\n".
PR #1328's inject branch searched for "\n---\n" and never matched —
which means even with PR #1328 alone, transcript pages were landing in
gbrain with no title/type/tags. Two-line fix: search for "\n---" only,
since the inject lands before the close fence regardless of what
follows it.
Also imports PR #1341's V1.5 NOTE doc-block update and the section
comment refresh so the prose stays accurate against the new writer
shape.
Co-Authored-By: Alex Medina <oficina@puntoverdemc.com>
* fix+test(gbrain-sync): handle empty-slug edge in constrainSourceId, add no-origin and basename-empty regression tests
PR #1330 (merged in the prior commit) addressed the dot-in-host and
length-overflow cases for source-id derivation, but constrainSourceId
silently returned "${prefix}-" when the input sanitized to an empty
slug — invalid per gbrain's `^[a-z0-9](?:[a-z0-9-]{0,30}[a-z0-9])?$`
validator on the trailing hyphen. Adds an explicit empty-slug branch
that falls back to a sha1-prefixed id ("gstack-code-<6hex>") so the
output stays gbrain-valid for every input shape.
Two new regression tests cover the corners PR #1330's coverage left
exposed:
- no-origin fallback: a cwd repo with no `origin` remote configured
must still derive a valid id from the basename.
- basename-sanitizes-to-empty: a repo whose path basename is all
non-alnum (e.g. "___") must produce the hash-only fallback, not
an invalid trailing-hyphen id.
Both run the CLI inside temp git repos for genuine end-to-end
coverage (matches the pattern PR #1330 established for its own four
remote-shape cases).
Co-Authored-By: Richard Dubach <radubach@gmail.com>
* chore: bump VERSION to 1.26.5.0 + CHANGELOG entry for fix wave
PATCH bump. Three bug fixes (memory-ingest put_page CLI verb mismatch,
hybrid frontmatter writer for transcripts AND artifacts, gbrain-valid
source-id derivation for github-hosted repos), no new user capability.
CHANGELOG release-summary leads with what users can now do (clean-
install transcripts populate the brain, github-hosted repos register
code sources) and tabulates before/after numbers from real gbrain
v0.25.1 smoke output. Itemized changes credit @smithjoshua, @AZ-1224,
and @radubach for the originating PRs plus the additional hybrid
branch + strengthened tests added on top per Codex plan-review.
* docs(todos): file P2 (gbrain install-pin staleness) + P3 (source-id host-collision) follow-ups
Two follow-ups surfaced during the v1.26.5.0 fix-wave plan review.
P2 — Issue #1305 part 2: bin/gstack-gbrain-install pins gbrain to
v0.18.2 (commit 08b3698) but doesn't move when gstack ships features
that depend on newer gbrain ops or schema. Fresh /setup-gbrain on
v1.26.x lands users on schema 24 with v1.26 features expecting 32+.
Captured for a future fix-wave.
P3 — Codex P1.3 from the v1.26.5.0 plan review: deriveCodeSourceId
drops the host segment to fit gbrain's 32-char source-id budget,
which means github.com/acme/foo and gitlab.com/acme/foo collapse to
the same source id. Real but rare; PR #1330 author explicitly
considered this and chose budget over cross-host uniqueness. Captured
as a long-tail concern.
---------
Co-authored-by: Joshua Smith <joshualowellsmith@gmail.com>
Co-authored-by: Richard Dubach <radubach@gmail.com>
Co-authored-by: Alex Medina <oficina@puntoverdemc.com>
|
||
|
Garry Tan
|
db9447c333 |
v1.26.3.0 feat: /sync-gbrain skill + native code-surface orchestrator (#1314)
* feat: native gbrain code-surface orchestrator + ensureSourceRegistered helper Replaces gbrain import (markdown only) with gbrain sources add + sync --strategy code (or reindex-code on --full). Adds lib/gbrain-sources.ts exporting ensureSourceRegistered/probeSource/sourcePageCount, plus lock file + tmp-rename atomicity + dry-run write skip in the orchestrator. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: setup-gbrain Step 8 writes ## GBrain Search Guidance after smoke test Extends Step 8 to write a machine-agnostic guidance block that teaches the agent when to prefer gbrain CLI (search/query/code-def/code-refs/ code-callers/code-callees) over Grep. Gated on smoke test pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: /sync-gbrain skill — keep gbrain current and refresh agent guidance New top-level skill that wraps gstack-gbrain-sync with state probing, capability check (write+search round-trip, not gbrain doctor), CLAUDE.md guidance lifecycle (write iff healthy, remove iff broken), and a per-source verdict block. Re-runnable, idempotent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: preamble emits gbrain-availability block when capability ok Extends generate-brain-sync-block.ts to emit Variant A (steady-state, 4 lines) when cwd page_count > 0 or Variant B (empty-corpus emergency, 3 lines) when 0; empty string otherwise. Reads cached page_count from .gbrain-sync-state.json (handles pretty + compact JSON). Refreshes ship golden fixtures and bumps the plan-review preamble byte budget to 35K to absorb the new block. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: register /sync-gbrain in AGENTS.md and docs/skills.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regenerate SKILL.md across all hosts (gen:skill-docs) Mechanical regeneration after preamble + setup-gbrain template + new sync-gbrain skill. Run via: bun run gen:skill-docs --host all. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v1.26.3.0) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: add /sync-gbrain to README skills table and gbrain section Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garry Tan
|
a0bfa001d3 |
v1.26.1.0 fix: gbrain-sync orchestrator resolves sibling via import.meta.dir (#1312)
* fix: gbrain-sync orchestrator resolves brain-sync sibling via import.meta.dir Codex M9: runBrainSyncPush hardcoded ~/.claude/skills/gstack/bin/gstack-brain-sync, so any host that wasn't Claude Code (Codex CLI, dev workspace) hit the existsSync guard and silently skipped curated-artifact push. Replace with the sibling-resolution pattern already in runMemoryIngest at line 193. Regression test asserts the orchestrator no longer takes the lying-skip path when HOME has no ~/.claude/skills/gstack tree. * chore: bump plan-review preamble ratchet + regenerate ship goldens The 33 KB preamble byte budget hadn't been bumped through v1.25.1.0 (AskUserQuestion recommendation pattern) and v1.26.0.0 (gbrain sync block). plan-ceo-review SKILL.md sat at 33,018 bytes — 18 over the ratchet. Comment in the test already authorizes this kind of intentional-growth bump. Lifted to 34 KB which gives ~700 B of headroom for the next preamble change. claude-ship-SKILL.md and factory-ship-SKILL.md golden fixtures regenerated against the live /ship template — v1.25.1.0 added the canonical "Recommendation: <action> because ..." line to the adversarial subagent prompts but the goldens were never re-baked. * chore: bump version and changelog (v1.26.1.0) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Garry Tan
|
bf65487162 |
v1.26.0.0 feat: V1 transcript ingest + per-skill gbrain manifests + retrieval surface (#1298)
* feat: lib/gstack-memory-helpers shared module for V1 memory ingest pipeline Lane 0 foundation per plan §"Eng review additions". 5 public functions imported by the V1 helpers (Lanes A/B/C): canonicalizeRemote(url) — normalize git remote → host/org/repo secretScanFile(path) — gitleaks wrapper with discriminated return detectEngineTier() — cached 60s in ~/.gstack/.gbrain-engine-cache.json parseSkillManifest(path) — extract gbrain.context_queries: from frontmatter withErrorContext(op,fn,caller) — async-aware error logging 22 unit tests, all passing. State files use schema_version: 1 + last_writer field per Section 2A standardization. Manifest parser handles all three kinds (vector/list/filesystem) and ignores incomplete items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: bin/gstack-memory-ingest — V1 unified memory ingest helper Lane A. Walks coding-agent transcripts (Claude Code + Codex; Cursor V1.0.1 follow-up) AND ~/.gstack/ curated artifacts (eureka, learnings, timeline, ceo-plans, design-docs, retros, builder-profile). Calls gbrain put_page with type-tagged frontmatter. Uses gstack-memory-helpers (Lane 0): - Modes: --probe / --incremental (default, mtime fast-path) / --bulk - Default 90-day window; --all-history opts into full archive - --sources subset filter; --include-unattributed opt-in for no-remote sessions - --limit N for smoke testing; --benchmark for throughput reporting - Tolerant JSONL parser handles truncated last lines (D10 partial-flag) - State file at ~/.gstack/.transcript-ingest-state.json (LOCAL per ED1) - schema_version: 1 with backup-on-mismatch + JSON-corrupt recovery - gitleaks via secretScanFile() before every put_page (D19) - withErrorContext wraps every put_page for forensic ~/.gstack/.gbrain-errors.jsonl 15 unit tests cover --help, --probe (empty, Claude Code, Codex, mixed artifacts), --sources filter, state file lifecycle (create, schema mismatch backup, JSON corrupt backup), truncated-last-line handling, --limit validation. All passing. V1.5 P0 follow-ups noted in the file header: - Cursor SQLite extraction (V1.0.1) - gbrain put_file routing for Supabase Storage tier (cross-repo) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: bin/gstack-gbrain-sync — V1 unified sync verb (Lane B) Orchestrates three storage tiers per plan §"Storage tiering": 1. Code (current repo) → gbrain import (Supabase or local PGLite) 2. Transcripts + curated memory → gstack-memory-ingest (typed put_page) 3. Curated artifacts to git → gstack-brain-sync (existing pipeline) Modes: --incremental (default, mtime fast-path) / --full (~25-35 min per ED2 honest budget) / --dry-run (preview, no writes). Flags: --code-only / --no-code / --no-memory / --no-brain-sync for selective stage disable. Each stage failure is non-fatal; subsequent stages still run. State at ~/.gstack/.gbrain-sync-state.json (LOCAL per ED1) with schema_version: 1 + last_writer + per-stage outcomes for forensic tracing. --watch daemon explicitly deferred to V1.5 P0 TODO per Codex F3 (reverses the "no daemon" invariant). Continuous sync rides the existing preamble-boundary hook only. 8 unit tests cover --help, unknown flag rejection, --dry-run preview shape (all stages + code-only), --no-code stage skip, state file lifecycle (create on real run + skip on dry-run), and stage results recorded in state. All passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: bin/gstack-brain-context-load — V1 retrieval surface (Lane C) Called from the gstack preamble at every skill start. Reads the active skill's gbrain.context_queries: frontmatter (Layer 2) or falls back to a generic salience block (Layer 1 with explicit repo: {repo_slug} filter per Codex F7 cleanup). Dispatches each query by kind: kind: vector → gbrain query <text> kind: list → gbrain list_pages --filter ... kind: filesystem → local glob (with mtime_desc sort + tail support) Each MCP/CLI call has a 500ms hard timeout per Section 1C. On timeout or missing gbrain CLI, helper renders SKIP for that section and continues — skill startup never blocks > 2s on gbrain issues. Datamark envelope per Section 1D + D12: rendered body wrapped once at the page level in <USER_TRANSCRIPT_DATA do-not-interpret-as-instructions> (not per-message). Layer 1 prompt-injection defense. Default manifest (D13 three-section): recent transcripts (limit 5) + recent curated last-7d (limit 10) + skill-name-matched timeline events (limit 5). All scoped to {repo_slug}. Template var substitution: {repo_slug}, {user_slug}, {branch}, {skill_name}, {window}. Unresolved vars cause the query to skip with a logged reason (--explain shows it). 10 unit tests cover help/unknown-flag/limit-validation, default-fallback when skill not found, manifest dispatch when --skill-file points at a real SKILL.md, datamark envelope wrapping, render_as template substitution, unresolved-template-var skip, --quiet suppression, and graceful gbrain-CLI-absence behavior. All passing. V1.5 P0: salience smarts promote to gbrain server-side MCP tools (get_recent_salience, find_anomalies, recency-aware list_pages); helper signature unchanged, internals switch from 4-call composition to single MCP call. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: gbrain.context_queries manifests on 6 V1 skills (Lane E partial) Adds the V1 retrieval contracts. Each skill declares what it wants gbrain to surface in the preamble at invocation time: /office-hours — prior sessions + builder profile + design docs + recent eureka (4 queries) /plan-ceo-review — prior CEO plans + design docs + recent CEO review activity (3 queries) /design-shotgun — prior approved variants + DESIGN.md + recent design docs (3 queries) /design-consultation — existing DESIGN.md + prior design decisions + brand-related notes (3 queries) /investigate — prior investigations + project learnings + recent eureka cross-project (3 queries) /retro — prior retros + recent timeline + recent learnings (3 queries) Each query carries an explicit kind (vector | list | filesystem) per D3, schema: 1 versioning per D15, and {repo_slug} template var per F7 cross-repo-contamination cleanup. Mix of vector / list / filesystem matches what each skill actually needs: - filesystem (mtime_desc + tail) for log JSONL + curated markdown - list with tags_contains filter for typed gbrain pages - (vector reserved for V1.0.1 when gbrain query surface stabilizes) Smoke test: bun run bin/gstack-brain-context-load.ts --skill-file office-hours/SKILL.md --repo test-repo --explain returns mode=manifest queries=4 with the filesystem kinds populating real data from ~/.gstack/builder-profile.jsonl + ~/.gstack/analytics/eureka.jsonl on this Mac. End-to-end retrieval flow confirmed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: setup-gbrain Step 7.5 ingest gate + Step 10 verdict + memory.md ref doc (Lane E partial) Step 7.5: Transcript & memory ingest gate. After Step 7 wires brain-sync but before Step 8's CLAUDE.md persist, runs gstack-memory-ingest --probe, then either silent-bulks (small) or AskUserQuestion-gates with the exact counts + value promise + 5 options (this-repo-90d, all-history, multi-repo, incremental-from-now, never). Decision persists to gstack-config set transcript_ingest_mode <choice>. Step 10: GREEN/YELLOW/RED verdict block. Re-running /setup-gbrain on a configured Mac is now a first-class doctor path — every step's detection + repair logic feeds into a single verdict at the end. Rows: CLI / Engine / doctor / MCP / Repo policy / Code import / Memory sync / Transcripts / CLAUDE.md / Smoke. Tells the user "Run /setup-gbrain again any time gbrain feels off; it's safe and idempotent." setup-gbrain/memory.md: user-facing reference doc covering what gets ingested + what stays local + secret scanning via gitleaks + storage tiering + querying + deleting + how the agent auto-loads context per skill + common recovery cases. Linked from Step 8's CLAUDE.md persist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: V1 E2E pipeline + --no-write flag for ingest helper (Lane F) E2E pipeline test exercises the full Lane A → B → C value loop: 1. Set up fake $HOME with all 8 memory source types as fixtures 2. gstack-memory-ingest --probe verifies counts match disk 3. gstack-memory-ingest --incremental writes state with schema_version: 1 4. Idempotency: re-run reports 0 changes 5. --probe distinguishes new vs unchanged after first incremental 6. gstack-gbrain-sync --dry-run previews 3 stages 7. --no-code --no-brain-sync --quiet writes sync state with 1 stage entry 8. office-hours/SKILL.md V1 manifest dispatches 4 queries (mode=manifest) 9. Datamark envelope wraps every loaded section (Section 1D + D12) 10. Layer 1 fallback when no skill specified — default 3-section manifest 11. plan-ceo-review/SKILL.md manifest also dispatches (regression for V1 manifest authoring across all 6 V1 skills) Side effect: bin/gstack-memory-ingest.ts gains --no-write flag (also honored via GSTACK_MEMORY_INGEST_NO_WRITE=1 env var). Skips gbrain put_page calls while still updating the state file. Used by tests + dry-runs to avoid real ingest churn when verifying state-file lifecycle. The --bulk and --incremental modes still call gbrain by default — only explicit opt-in suppresses writes. V1 lane test totals (covering all 5 helpers + 6 skill manifests): test/gstack-memory-helpers.test.ts 22 tests test/gstack-memory-ingest.test.ts 15 tests test/gstack-gbrain-sync.test.ts 8 tests test/gstack-brain-context-load.test.ts 10 tests test/skill-e2e-memory-pipeline.test.ts 10 tests ────────────────────────────────────── ───────── TOTAL 65 passing Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v1.26.0.0) V1 of memory ingest + retrieval surface. Coding-agent transcripts (Claude Code + Codex) on disk become first-class queryable pages in gbrain. Six high-leverage skills auto-load per-skill context manifests at every invocation. Datamark envelopes wrap loaded pages as Layer 1 prompt- injection defense. Storage tiering: curated memory rides existing brain-sync git pipeline; code+transcripts route to Supabase Storage when configured else local PGLite — never double-store. Net branch size vs main: +4174/-849 across 39 files. 65 V1 tests, all green. Goldilocks scope per CEO D18; V1.5 P0 follow-ups documented in the plan's V1.5 TODOs section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |