fix: correct adapters against real running providers (round 2)

Ran real backends in isolated environments (real Postgres-backed gbrain,
live Sourcebot v6.5.0 with anonymous access, real graphify 0.9.23). All three
now index + search end-to-end. Fixes:

- gbrain: RESTORE `--strategy code` in refresh — round 1 removed it on a --help
  misread, which silently stopped code from ever being indexed. refresh now runs
  the verified two-pass (`sync`, then `sync --strategy code --full`). Pinned by a
  new test so the regression can't return.
- sourcebot: an API key is NOT required for local use — anonymous access
  (FORCE_ENABLE_ANONYMOUS_ACCESS=true) serves /api/search keyless (verified). Key
  stays optional; only the messaging changed (anonymous-access first, key as
  fallback) plus a note that a local repo needs remote.origin.url to index.
- graphify: correct the docstring — for CODE both `graphify <dir>` and
  `graphify update` are AST-only (no LLM); the LLM only renames clusters and
  ingests non-code, which our parser ignores. No LLM mode; local=true is correct.

Docs: capability matrix + "Verified against real environments" updated to record
all three proven end-to-end and to correct the two first-round mistakes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sinabina
2026-07-21 18:01:17 -07:00
co-authored by Claude Opus 4.8
parent a524d860a8
commit ab7989c6c1
5 changed files with 100 additions and 49 deletions
@@ -123,9 +123,9 @@ consent; egress requires a separate explicit step.
| Op | GBrain (recommend first) | Sourcebot | Graphify |
|----|--------------------------|-----------|----------|
| `register_source` | ✓ `sources add` | ✓ local `git` connection in config.json | ✓ `graphify update <dir>` (local, no LLM) |
| `refresh` | ✓ `sync --source` | ✓ auto (config change + reindexIntervalMs) | ✓ `graphify update <dir>` |
| `search` | ✓ `gbrain search` (federated corpora) | ✓ `POST /api/search` + Bearer key (v5) | ✓ `graphify query "<q>" --graph <graph.json>` |
| `register_source` | ✓ `sources add --federated` | ✓ local `git` connection in config.json | ✓ `graphify update <dir>` (local, no LLM) |
| `refresh` | ✓ `sync` + `sync --strategy code --full` | ✓ auto (config change + reindexIntervalMs) | ✓ `graphify update <dir>` |
| `search` | ✓ `gbrain search` (federated corpora) | ✓ `POST /api/search` (keyless w/ anonymous access; Bearer key optional) | ✓ `graphify query "<q>" --graph <graph.json>` |
| `status` | ✓ `sources list` + page_count | ~ partial (server liveness) | ~ partial (graph.json present + node count) |
| `add` | ✓ `put <slug>` | ✗ declines | ✗ declines |
| `delete` | ✓ `delete <slug>` | ✗ declines | ✗ declines |
@@ -140,13 +140,17 @@ All three are driven directly from the runtime — no MCP client:
(`sources add`/`sync`). Advertises all seven capabilities. **Recommended
first.**
- **Sourcebot** (`github.com/sourcebot-dev/sourcebot`, YC Fall 2025): self-hosted
whole-repo regex search. `register_source` adds a local `{ "type": "git", "url":
"file:///path" }` connection to the server's `config.json` (it re-indexes on
config change); `search` is `POST {baseUrl}/api/search`; `status` probes that
endpoint. Declines `add`/`delete`/`export`. **Sourcebot v5 gates `/api/search`
behind auth**, so the adapter sends `Authorization: Bearer <SOURCEBOT_API_KEY>`.
A loopback `baseUrl` keeps content on the machine (local); a remote one requires
egress consent.
whole-repo regex search, deployed via Docker Compose (bundled server + Postgres
+ Redis; no supported non-Docker path). `register_source` adds a local `{ "type":
"git", "url": "file:///path" }` connection to the server's `config.json` (it
re-indexes on config change; a local repo needs a `remote.origin.url` or it is
skipped); `search` is `POST {baseUrl}/api/search`; `status` probes that endpoint.
Declines `add`/`delete`/`export`. It is a **local** tool — indexed code stays on
your machine — and an **API key is optional**: a local instance with anonymous
access (`FORCE_ENABLE_ANONYMOUS_ACCESS=true`) serves `/api/search` keyless. The
adapter sends `Authorization: Bearer <SOURCEBOT_API_KEY>` only when a key is set.
A loopback `baseUrl` keeps content on the machine (local=true); a remote one
requires egress consent.
- **Graphify** (`github.com/Graphify-Labs/graphify`, YC-backed): local
tree-sitter code graph via the `graphify` CLI. The adapter uses **`graphify
update <dir>`** — the local, no-LLM build (writes `graphify-out/graph.json`);
@@ -236,30 +240,45 @@ a later phase precisely so the first slices do not trigger the
## Verified against real environments
The three adapters were tested against the real tools in isolated environments
(parallel agents, one worktree each), not just unit fakes. What that surfaced and
fixed:
All three adapters were driven against the real tools in isolated environments
(parallel agents, one worktree each), and all three now index + search a real repo
end-to-end. Two rounds ran, because the first round's fixes included a mistake that
only real execution caught — recorded here honestly.
- **Graphify (real install, graphify 0.9.23).** The first cut ran `graphify <dir>`
— which invokes an LLM extraction backend (needs a key + network), breaking the
"local, no egress" promise — and parsed a made-up output format. Fixed to the
real local `graphify update` build and a parser written against the real
`NODE`/`EDGE` output (file at `src=`/`at=`). Also fixed `search` ignoring the
indexed repo (now persists the indexed root) and `options` mislabeling an
installed-but-unindexed provider as unavailable.
- **Sourcebot (live v5 in Docker).** Endpoint, request body, and response parsing
were correct against the real server. But v5 gates `/api/search` behind auth, so
the adapter got HTTP 401; added `Authorization: Bearer` support and made `status`
stop following the login redirect (it was falsely reporting "ready").
- **GBrain (real gbrain 0.42.56, pglite engine).** The engine was broken on the
host (upstream macOS WASM bug), which exposed two bugs: engine-down failures were
reported as hard `PROVIDER_ERROR` with a raw stack dump instead of a clean
`PROVIDER_UNAVAILABLE` degrade (fixed), and the adapter sent flags the real
`gbrain` CLI does not define (`sync --strategy`, `search --source`) — corrected
to the real surface.
- **GBrain — real Postgres+pgvector (Docker), gbrain 0.42.56 — PROVEN.** The
default pglite/WASM engine is broken on macOS (upstream garrytan/gbrain#223), so
the working recipe points gbrain at a real Postgres via `DATABASE_URL`. Real
end-to-end search returned the actual code definition
(`[0.88] src-checksum-ts … export statement computeChecksum`). Real execution
caught a **regression I had introduced**: I removed `--strategy code` from
`refresh` based on a `--help` misread, which silently stopped code from ever
being indexed (only docs were). Restored to the verified two-pass
(`sync`, then `sync --strategy code --full`); `--federated` registration is
load-bearing for global search. Also fixed earlier: engine-down now degrades to
`PROVIDER_UNAVAILABLE` (one-line message) instead of `PROVIDER_ERROR` + a WASM
stack dump.
- **Sourcebot — live v6.5.0 (Docker) — PROVEN keyless.** Endpoint, body, and
response parsing were correct against the real server. Correcting an earlier
wrong conclusion: Sourcebot does **not** require an API key for local use —
enabling anonymous access (`FORCE_ENABLE_ANONYMOUS_ACCESS=true`) serves
`/api/search` keyless, verified with a real hit through the CLI with no key set.
The key stays optional; the only fix was messaging (point users to anonymous
access first, key as fallback) plus a note that a local repo needs a
`remote.origin.url` to be indexed. It is a local tool (code stays on the
machine; a boot telemetry ping unless `SOURCEBOT_TELEMETRY_DISABLED=true`).
- **Graphify — real install, graphify 0.9.23 — PROVEN.** Correcting an earlier
wrong claim of mine: for **code**, `graphify <dir>` and `graphify update <dir>`
produce the identical AST graph with **no LLM call**; the LLM only renames
community clusters and ingests non-code docs, adding zero nodes/edges, and our
parser discards the field it touches. So there is deliberately no LLM mode, and
`local=true` is correct. The adapter uses `graphify update`; real `index`+search
returned correct `file:line` refs. Also fixed: `search` now reads the indexed
repo's graph (persisted root), and `options` reports an installed provider as
available.
Live end-to-end index+search-with-results was proven for Graphify and Sourcebot;
GBrain's was blocked only by the host's broken engine, not by adapter code.
The larger lesson, kept on the record: a `--help` reading or a single agent's
conclusion is not proof — running the real tool is. It reversed two of my
first-round calls (the gbrain flag removal and the graphify LLM claim).
## What this does NOT change
+9 -5
View File
@@ -90,11 +90,15 @@ export class GbrainProvider implements CodeProvider {
async refresh(source: SourceRef, opts: OpOptions = {}): Promise<SourceStatus> {
assertEgressConsent(this, opts);
// `gbrain sync` has no `--strategy` flag (verified against gbrain 0.42.x --help).
this.#assertOk(spawnGbrain(["sync", "--source", source.id], {
baseEnv: opts.env,
timeout: opts.timeout ?? DEFAULT_TIMEOUT_MS,
}));
const timeout = opts.timeout ?? DEFAULT_TIMEOUT_MS;
// Two passes, verified end-to-end against real Postgres-backed gbrain 0.42.56:
// 1. default sync (markdown strategy) — indexes docs.
// 2. `sync --strategy code` — the ACTUAL code-indexing pass. Without it code
// is never indexed (the whole point of a code provider); `code-def` stays
// "not_built" and search only finds incidental doc mentions. `--full`
// forces it past the per-source checkpoint the markdown pass advanced.
this.#assertOk(spawnGbrain(["sync", "--source", source.id], { baseEnv: opts.env, timeout }));
this.#assertOk(spawnGbrain(["sync", "--source", source.id, "--strategy", "code", "--full"], { baseEnv: opts.env, timeout }));
return this.status(source, opts);
}
+12 -6
View File
@@ -1,12 +1,18 @@
/**
* Graphify adapter — real CLI integration (github.com/Graphify-Labs/graphify).
*
* Graphify is a LOCAL tree-sitter knowledge graph. The genuinely local, no-LLM
* build is `graphify update <dir>` — it writes `<dir>/graphify-out/graph.json`
* with NO embeddings and NO network (verified against graphify 0.9.23). NOTE:
* the bare `graphify <dir>` build instead runs LLM semantic extraction (a gemini
* backend needing an API key + network), so this adapter deliberately uses
* `graphify update`, which keeps the "local, no egress consent" invariant true.
* Graphify is a LOCAL tree-sitter knowledge graph. For CODE, `graphify <dir>`
* and `graphify update <dir>` produce the SAME AST graph with NO LLM and NO
* network (verified against graphify 0.9.23 — both emit `AST extraction on N
* code files`, all node origins `ast`). The LLM backend (openai/gemini) is only
* used to RENAME community clusters (`graphify label` / `cluster-only`) and to
* ingest non-code docs (`graphify add`); it adds zero nodes/edges, and our parser
* discards the `community=` field it touches — so an LLM mode would send code
* off-machine for no change in search output, and is intentionally not offered.
*
* This adapter uses `graphify update <dir>` (writes `<dir>/graphify-out/graph.json`
* and does clustering in one shot) and stays fully local — nothing leaves the
* machine, so `local = true` and no egress consent is needed.
*
* Query is `graphify query "<q>" --graph <dir>/graphify-out/graph.json`; the
* `--graph` flag points at the built graph so search never depends on cwd.
+13 -6
View File
@@ -46,9 +46,11 @@ export interface SourcebotOptions {
/** Path to the server's config.json (for register_source). Defaults to SOURCEBOT_CONFIG. */
configPath?: string;
/**
* API key for the Sourcebot REST API. Defaults to SOURCEBOT_API_KEY. Sourcebot
* v5 gates `/api/search` behind auth (`Authorization: Bearer <key>`); without
* it, `search` gets HTTP 401. Generate one in Settings -> API Keys.
* OPTIONAL API key for the Sourcebot REST API. Defaults to SOURCEBOT_API_KEY.
* A local instance with anonymous access enabled
* (`FORCE_ENABLE_ANONYMOUS_ACCESS=true`) serves `/api/search` with NO key —
* verified keyless against a real Sourcebot v6.5.0. Only set a key when your
* instance is login-gated; it is sent as `Authorization: Bearer <key>`.
*/
apiKey?: string;
/** Injectable fetch for tests. */
@@ -93,7 +95,12 @@ export class SourcebotProvider implements CodeProvider {
return this.capabilities.has(capability);
}
/** Add the repo as a local `git` connection in Sourcebot's config.json. */
/**
* Add the repo as a local `git` connection in Sourcebot's config.json.
* NOTE: Sourcebot silently SKIPS a local repo that has no `remote.origin.url`
* (logs "Skipping <path> - remote.origin.url not found"); a freshly `git init`'d
* repo must set an origin before it will index.
*/
async registerSource(repo: RepoRef, opts: OpOptions = {}): Promise<SourceStatus> {
assertEgressConsent(this, opts); // no-op when the server is loopback (local)
if (!this.#configPath) {
@@ -148,7 +155,7 @@ export class SourcebotProvider implements CodeProvider {
opts.timeout ?? DEFAULT_TIMEOUT_MS,
);
if (res.status === 401 || res.status === 403) {
return { id: "*", state: "unknown", partial: true, detail: "reachable but not authenticated (set SOURCEBOT_API_KEY)" };
return { id: "*", state: "unknown", partial: true, detail: "reachable but login-gated (enable anonymous access with FORCE_ENABLE_ANONYMOUS_ACCESS=true for local use, or set SOURCEBOT_API_KEY)" };
}
return { id: "*", state: res.ok ? "ready" : "unknown", partial: true, detail: `HTTP ${res.status}` };
} catch {
@@ -168,7 +175,7 @@ export class SourcebotProvider implements CodeProvider {
throw new CodeProviderError("PROVIDER_UNAVAILABLE", `Sourcebot unreachable at ${this.#baseUrl}: ${(err as Error).message}`, this.id);
}
if (res.status === 401 || res.status === 403) {
throw new CodeProviderError("PROVIDER_UNAVAILABLE", `Sourcebot requires authentication; set SOURCEBOT_API_KEY (HTTP ${res.status})`, this.id);
throw new CodeProviderError("PROVIDER_UNAVAILABLE", `Sourcebot is login-gated (HTTP ${res.status}); enable anonymous access (FORCE_ENABLE_ANONYMOUS_ACCESS=true) for local use, or set SOURCEBOT_API_KEY`, this.id);
}
if (!res.ok) throw new CodeProviderError("PROVIDER_ERROR", `Sourcebot ${path} returned HTTP ${res.status}`, this.id);
try {
+15
View File
@@ -281,6 +281,21 @@ exit 1
expect(hits).toEqual([{ ref: "src/x.ts", score: 0.88, snippet: "match", kind: "document" }]);
});
test("refresh runs the code-indexing pass (`sync --strategy code --full`)", async () => {
// Real gbrain only indexes code when `sync --strategy code` runs; without it
// code-def stays not_built. Pin that the adapter issues that pass.
const marker = path.join(homeDir, "sync-calls.log");
writeShim(`#!/usr/bin/env bash
if [ "$1" = "sync" ]; then printf '%s\\n' "$*" >> "${marker}"; exit 0; fi
if [ "$1" = "sources" ]; then echo '{"sources":[{"id":"code","local_path":"/r","page_count":1}]}'; exit 0; fi
exit 1
`);
await new GbrainProvider().refresh({ id: "code" }, { env: env(), consented: true });
const log = fs.readFileSync(marker, "utf-8");
expect(log).toContain("--strategy code");
expect(log).toContain("--full");
});
test("engine-down (pglite WASM) degrades to PROVIDER_UNAVAILABLE, not PROVIDER_ERROR", async () => {
// Reproduces garrytan/gbrain#223: engine fails to init; must degrade cleanly.
writeShim(`#!/usr/bin/env bash