Braintrust now owns benchmark scoring, experiments, comparison, and
reporting. GStack keeps only the CLI-agent adapters (the unavoidable shim)
plus operational metrics the CLIs report. runProviderBenchmark wraps each
adapter as a Braintrust Eval task; a deterministic required-terms scorer
replaces the in-house evaluation logic and the optional autoevals ClosedQA
judge replaces judge.ts.
Runs local by default under bun: with no BRAINTRUST_API_KEY it sets
noSendLogs and ships nothing; setting the key opts into the cloud dashboard.
An empty output with zero tokens is thrown so a silent auth/CLI failure
can't masquerade as a 0.0 score.
Deletes runner.ts and judge.ts (and their test-helper re-export shims);
rewires bin/gstack-model-benchmark and adapts the benchmark tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The benchmark adapter only recognized GOOGLE_API_KEY and parsed the legacy
text/usage event shape. Recognize GEMINI_API_KEY and ~/.gemini/.env, parse
the current content/stats schema, ignore echoed user messages, and run
report-only plan mode with --skip-trust for disposable workspaces.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
