Ran real backends in isolated environments (real Postgres-backed gbrain,
live Sourcebot v6.5.0 with anonymous access, real graphify 0.9.23). All three
now index + search end-to-end. Fixes:
- gbrain: RESTORE `--strategy code` in refresh — round 1 removed it on a --help
misread, which silently stopped code from ever being indexed. refresh now runs
the verified two-pass (`sync`, then `sync --strategy code --full`). Pinned by a
new test so the regression can't return.
- sourcebot: an API key is NOT required for local use — anonymous access
(FORCE_ENABLE_ANONYMOUS_ACCESS=true) serves /api/search keyless (verified). Key
stays optional; only the messaging changed (anonymous-access first, key as
fallback) plus a note that a local repo needs remote.origin.url to index.
- graphify: correct the docstring — for CODE both `graphify <dir>` and
`graphify update` are AST-only (no LLM); the LLM only renames clusters and
ingests non-code, which our parser ignores. No LLM mode; local=true is correct.
Docs: capability matrix + "Verified against real environments" updated to record
all three proven end-to-end and to correct the two first-round mistakes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update the capability matrix and provider notes to the verified real interfaces
(graphify update, Sourcebot v5 Bearer auth) and add a "Verified against real
environments" section documenting what the live tests found and fixed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite the Graphify and Sourcebot expectations against the real formats captured
from live tools (graphify NODE/EDGE query output; Sourcebot v5 response + Bearer
auth), add a gbrain engine-down -> PROVIDER_UNAVAILABLE degrade test and a
no-phantom-`--source` search assertion, and cover the persisted indexed-root path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
search was running in cwd and missing the repo you indexed. Persist the indexed
path per provider in the selection store and resolve it back so `search` reads
the same graph `index` built. Graphify availability now checks `graphify --version`
(installed = selectable) instead of "a graph already exists here", and the CLI
keys Graphify sources on the repo path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Parallel real-environment tests (graphify 0.9.23, Sourcebot v5 in Docker,
gbrain 0.42.56) surfaced real mismatches:
- graphify: build via `graphify update <dir>` (the local, no-LLM path) instead
of `graphify <dir>` (which runs an LLM backend needing a key + network, so the
old path wasn't actually local); query via `graphify query --graph <graph.json>`
so it reads the indexed graph regardless of cwd; parse the real NODE/EDGE output
(file lives at src=/at=) instead of an invented format.
- sourcebot: Sourcebot v5 gates /api/search behind auth — send
`Authorization: Bearer <SOURCEBOT_API_KEY>`; treat 401/403 as PROVIDER_UNAVAILABLE;
make status probe /api/search without following the login redirect.
- gbrain: degrade engine/DB init failures (e.g. pglite WASM, garrytan/gbrain#223)
to PROVIDER_UNAVAILABLE with a one-line message instead of PROVIDER_ERROR + a raw
stack dump; drop flags the real CLI doesn't define (`sync --strategy`,
`search --source`); align put/delete to stdin, export to brain-wide.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capability matrix, result parsers, selection store + per-repo consent +
provider-OFF, egress gating, and each adapter end-to-end against a fake CLI
shim (gbrain, graphify) or injected fetch + temp config.json (sourcebot),
plus PROVIDER_UNAVAILABLE degrade for every provider. 19 tests, no live
tools required.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
User-facing command tying the contract together: `options`/`status` show
providers with GBrain first and live availability, `select <provider>`
persists the choice, `consent [path]` records per-repo indexing consent,
`index [path]` registers + indexes the repo with the selected provider
(refusing non-local providers until consented), and `search <query>` runs a
query, degrading with a clear message when the provider is unavailable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
contract.ts: repo-oriented interface (four required ops, three optional),
typed CodeProviderError, egress + capability consent guards.
Three real, runtime-drivable adapters:
- GbrainProvider: gbrain CLI (reuses lib/gbrain-exec + lib/gbrain-sources),
all seven capabilities.
- GraphifyProvider: graphify CLI — `graphify <dir>` builds the local graph,
`graphify query` searches it, export reads graphify-out/graph.json. Fully
local; never auto-installed.
- SourcebotProvider: self-hosted server over HTTP — register writes a local
git connection to config.json, search is POST /api/search, status is a
liveness probe. Loopback base URL = local (no egress); remote = consent.
selection.ts persists the chosen provider + per-repo indexing consent under
$GSTACK_HOME. picker.ts recommends GBrain first, resolves the selected
provider or null (provider-OFF), and probes live availability. Every adapter
degrades to PROVIDER_UNAVAILABLE when its tool/server is absent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Design for the OPTIONAL code-intelligence provider contract that lets a
user pick how their codebase is indexed and searched, replacing gstack's
~17k LOC of bespoke GBrain glue. Repo-oriented ops (register_source/
refresh/search/status required; add/delete/export optional), a per-provider
capability matrix, GBrain-recommended-first selection, repo-scoped + install
consent, and a phased rollout that keeps gstack fully functional with no
provider selected. All three providers are driven from the runtime via CLI
or HTTP — no MCP client.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Points readers to docs/gstack-2/STATUS.md as the authoritative completion
state. No VERSION bump or release claim while status is BLOCKED.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The adapters parsed each vendor's proprietary JSON stream (Claude json,
Codex JSONL, Gemini stream-json) to extract tokens/tool-calls, and a
per-model pricing table turned tokens into cost. That coupling was the
brittle hardcoding GStack 2 exists to avoid — it broke every time a vendor
reshuffled its output, and it duplicated what any tool that instruments the
real model call already does. Braintrust owns scoring; it can't see a CLI
subprocess's tokens anyway, so computing cost ourselves meant maintaining
both a parser and a price table forever.
Now each adapter runs the CLI in plain-text mode and returns stdout. Scoring
is unchanged (Braintrust reads the text). RunResult drops tokens/toolCalls;
the comparison table drops the Tokens/Cost columns. Deletes pricing.ts, all
three JSON parsers, and the Gemini stream-schema parser + its test. Gemini
auth detection (env/OAuth/.env) is kept — that's not schema parsing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JSON-driven XCUITest runner plus SwiftUI/UIKit fixture apps that drive a
real app by bundle id. The runner resolves the actual switch inside a
SwiftUI Toggle wrapper, and when a center tap misses the full-width row it
retries on the control's trailing edge via an element-anchored normalized
offset (adaptive, not a raw screen coordinate), then verifies the switch
value actually changed. A tap that silently does nothing is now reported
as an interaction failure, not a false product defect. Validated on the
simulator and on a physical iPhone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Standalone planner that turns the IOSQAFlow JSON contract into an
xcodebuild XCUITest invocation for simulator or physical device, using
semantic selectors (identifier/role/label) and no coordinate taps.
Pure plan generation; covered by 7 unit tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A capability whose binary launches while the managed runtime hard-fails
(e.g. skill-API mismatch) was reported as `degraded`/ok:true/exit 0, diverging
from plain `gstack doctor` (ok:false/exit 1) for the identical report. Split
the branch so runtime `warn` stays `degraded` and runtime `fail` maps to
`failed`, and add a regression test for the runtime-fail + capability-pass case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Braintrust now owns benchmark scoring, experiments, comparison, and
reporting. GStack keeps only the CLI-agent adapters (the unavoidable shim)
plus operational metrics the CLIs report. runProviderBenchmark wraps each
adapter as a Braintrust Eval task; a deterministic required-terms scorer
replaces the in-house evaluation logic and the optional autoevals ClosedQA
judge replaces judge.ts.
Runs local by default under bun: with no BRAINTRUST_API_KEY it sets
noSendLogs and ships nothing; setting the key opts into the cloud dashboard.
An empty output with zero tokens is thrown so a silent auth/CLI failure
can't masquerade as a 0.0 score.
Deletes runner.ts and judge.ts (and their test-helper re-export shims);
rewires bin/gstack-model-benchmark and adapts the benchmark tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The benchmark adapter only recognized GOOGLE_API_KEY and parsed the legacy
text/usage event shape. Recognize GEMINI_API_KEY and ~/.gemini/.env, parse
the current content/stats schema, ignore echoed user messages, and run
report-only plan mode with --skip-trust for disposable workspaces.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The contract previously named Context.dev the only authorized external
service and barred provider marketplaces. Loosen it so optional, off-by-
default, consent-gated eval backends (Braintrust and the like) are allowed;
keep the consent + typed-failure hygiene.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backing the model benchmark with Braintrust's local eval runner and its
autoevals scorer library instead of an in-house scoring engine.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New advisories landed after the integration work: fast-uri (High) and hono
(3x Medium) picked up fixed releases, and a Medium surfaced on
@hono/node-server. Bump the fast-uri and hono override pins to their fixed
patch releases (3.1.3, 4.12.27), clearing four findings.
The remaining @hono/node-server advisory is reachable only through the unused
@modelcontextprotocol/sdk transitive (no source imports it, no Hono server is
started); its only fix is a major bump the SDK pins against. Record that
assessment in .osv-scanner.toml so the scheduled scan stays honest instead of
alarming on an unreachable path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Create the browser smoke page under GITHUB_WORKSPACE so the runtime's local-file trust boundary permits it on every runner, and advance the immutable bootstrap channel to RC3.
Use an allowed local file URL for the browser smoke test, centralize keyless archive signing after native builds so Windows ARM64 does not require an unavailable Cosign binary, and advance the immutable bootstrap channel to RC2.
Publish signed prerelease artifacts from v2.0.0-rc.* tags, bind bootstrap trust to the immutable RC tag, make missing-release errors actionable, and install the six public skills from the canonical subpath.
Detect host-native browser tools before offering the isolated local Chromium fallback, add a common readiness fixture, harden managed browser startup, and verify standards installs expose one canonical QA skill.
Time Attakc