Files
gstack/test/gstack-gbrain-detect-mcp-mode.test.ts
T
Garry Tan f44de365c5 v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351)
* feat: gstack-gbrain-mcp-verify helper for remote MCP probe

Probes a remote gbrain MCP endpoint with bearer auth. POSTs initialize,
classifies failures into NETWORK / AUTH / MALFORMED with one-line
remediation hints, and runs a tools/list capability probe to detect
sources_add MCP support (forward-compat for when gbrain ships URL ingest).

Token consumed from GBRAIN_MCP_TOKEN env, never argv. Required to set
both 'application/json' AND 'text/event-stream' in Accept; that gotcha
costs 10 minutes of debugging when missed (regression-tested).

Live-verified against wintermute (gbrain v0.27.1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: gstack-artifacts-init + gstack-artifacts-url helpers

artifacts-init replaces brain-init with provider choice (gh / glab /
manual), per-user gstack-artifacts-$USER repo, HTTPS-canonical storage in
~/.gstack-artifacts-remote.txt, and a "send this to your brain admin"
hookup printout. Always prints the command, never auto-executes — gbrain
v0.26.x has no admin-scope MCP probe (codex Finding #3).

artifacts-url centralizes HTTPS↔SSH/host/owner-repo conversion so callers
don't each string-mangle (codex Finding #10). The remote-conflict check in
artifacts-init compares at the canonical level so re-running with HTTPS
input doesn't trip on a stored SSH URL for the same logical repo.

The "URL form not supported" branch prints a two-line clone-then-path
form for gbrain v0.26.x; the supported branch is a one-liner with --url
ready for when gbrain ships URL ingest.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: extend gstack-gbrain-detect with mcp_mode + artifacts_remote

Adds two new fields to detect's JSON output:

- gbrain_mcp_mode: local-stdio | remote-http | none
  Resolved via 3-tier fallback (codex Finding D3): claude mcp get --json
  → claude mcp list text-grep → ~/.claude.json jq read. If Anthropic moves
  the file format, the first two tiers absorb it.

- gstack_artifacts_remote: HTTPS URL from ~/.gstack-artifacts-remote.txt
  Falls back to ~/.gstack-brain-remote.txt during the v1.27.0.0 migration
  window so detect doesn't return empty between upgrade and migration.

Existing detect tests still pass (15/15). New 19 tests cover every fallback
tier independently, plus a schema regression for /sync-gbrain compat.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: setup-gbrain Path 4 (remote MCP) + artifacts rename

Path 4 lets users paste an HTTPS MCP URL + bearer token and registers it
as an HTTP-transport MCP without needing a local gbrain CLI install. The
flow:

- Step 2 gains a fourth option (Remote gbrain MCP)
- Step 4 adds Path 4 sub-flow: collect URL, secret-read bearer, verify
  via gstack-gbrain-mcp-verify (NETWORK / AUTH / MALFORMED classifier)
- Step 5 (local doctor), Step 7.5 (transcript ingest), Step 5a's stdio
  branch all skip on Path 4
- Step 5a adds an HTTP+bearer registration form: claude mcp add
  --transport http --header "Authorization: Bearer ..."
- Step 7 renamed "session memory sync" → "artifacts sync" and now calls
  gstack-artifacts-init (which always prints the brain-admin hookup
  command — no auto-execute, codex Finding #3)
- Step 8 CLAUDE.md block branches: remote-http includes URL + server
  version (never the token); local-stdio keeps engine + config-file
- Step 9 smoke test on Path 4 prints the curl-equivalent for
  post-restart verification (MCP tools aren't visible mid-session)
- Step 10 verdict block has separate templates per mode

Idempotency: re-running with gbrain_mcp_mode=remote-http already in
detect output skips Step 2 entirely and goes to verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: rename gbrain_sync_mode → artifacts_sync_mode (v1.27.0.0 prep)

Hard rename, no dual-read alias (codex Finding D4). The on-disk migration
script (Phase C, separate commit) renames the config key in users'
~/.gstack/config.yaml and any CLAUDE.md blocks.

Touched call sites:
- bin/gstack-config defaults + validation + list/defaults output
- bin/gstack-gbrain-detect (gstack_brain_sync_mode field still emitted
  with the same name for downstream-tool compat; reads new key)
- bin/gstack-brain-sync, bin/gstack-brain-enqueue, bin/gstack-brain-uninstall
- bin/gstack-timeline-log (comment ref)
- scripts/resolvers/preamble/generate-brain-sync-block.ts: renames key,
  branches on gbrain_mcp_mode=remote-http to emit "ARTIFACTS_SYNC:
  remote-mode (managed by brain server <host>)" instead of the local
  mode/queue/last_push line (codex Finding #11)
- bin/gstack-brain-restore + bin/gstack-gbrain-source-wireup: read
  ~/.gstack-artifacts-remote.txt with ~/.gstack-brain-remote.txt fallback
  during the migration window
- bin/gstack-artifacts-init: tolerant of unrecognized URL forms (local
  paths, file://, self-hosted gitea) so test infrastructure and unusual
  remotes work without canonicalization
- test/brain-sync.test.ts: gstack-brain-init → gstack-artifacts-init
- test/skill-e2e-brain-privacy-gate.test.ts: artifacts_sync_mode keys
- test/gen-skill-docs.test.ts: budget 35K → 36.5K for the new MCP-mode
  probe in the preamble resolver
- health/SKILL.md.tmpl, sync-gbrain/SKILL.md.tmpl: comment + verdict line

Hard delete:
- bin/gstack-brain-init (replaced by bin/gstack-artifacts-init in v1.27.0.0)
- test/gstack-brain-init-gh-mock.test.ts (replaced by gstack-artifacts-init.test.ts)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: regenerate SKILL.md files after artifacts-sync rename

Mechanical regen via \`bun run gen:skill-docs --host all\`. All */SKILL.md
files reflect the renamed config key (gbrain_sync_mode →
artifacts_sync_mode), the renamed remote-helper file
(~/.gstack-artifacts-remote.txt with brain fallback), the renamed init
script (gstack-artifacts-init), and the new ARTIFACTS_SYNC: remote-mode
status line that fires when a remote-http MCP is registered.

Golden fixtures (test/fixtures/golden/*-ship-SKILL.md) refreshed to match
the regenerated default-ship output.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: v1.27.0.0 migration — gstack-brain → gstack-artifacts rename

Journaled, interruption-safe migration. Six steps, each writes to
~/.gstack/.migrations/v1.27.0.0.journal on success; re-entry resumes
from the next un-done step. On final success, journal is replaced by
~/.gstack/.migrations/v1.27.0.0.done.

Steps:
1. gh_repo_renamed       gh/glab repo rename gstack-brain-$USER →
                         gstack-artifacts-$USER (idempotent: detects
                         already-renamed and skips)
2. remote_txt_renamed    mv ~/.gstack-brain-remote.txt → artifacts file,
                         rewriting URL path to match the new repo name
3. config_key_renamed    sed -i in ~/.gstack/config.yaml flips
                         gbrain_sync_mode → artifacts_sync_mode
4. claude_md_block       sed flips "- Memory sync:" → "- Artifacts sync:"
                         in cwd CLAUDE.md and ~/.gstack/CLAUDE.md
5. sources_swapped       gbrain sources add NEW (verify) → remove OLD
                         (codex Finding #6: add-before-remove ordering,
                         no downtime window). On remote-MCP mode, prints
                         commands for the brain admin instead of executing.
6. done                  touchfile + delete journal

User opt-out: any "n" or "skip-for-now" answer at the initial prompt
writes a marker file that prevents re-prompting; user can re-invoke
via /setup-gbrain --rerun-migration.

11 unit tests cover: nothing-to-migrate, GitHub happy path, idempotent
re-run, journal-resume mid-flight, remote-MCP print-only path,
add-before-remove ordering verification, add-fail → old source stays
registered, CLAUDE.md field rewrite.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: regression suite + E2E for v1.27.0.0 rename

Three new regression tests guard the rename's blast radius (per codex
Findings #1, #8, #9, #12):

- test/no-stale-gstack-brain-refs.test.ts: greps bin/, scripts/, *.tmpl,
  test/ for forbidden identifiers (gstack-brain-init, gbrain_sync_mode);
  fails CI if any non-allowlisted file references them.
- test/post-rename-doc-regen.test.ts: confirms gen-skill-docs output has
  no stale references in any */SKILL.md (the cross-product blind spot).
- test/setup-gbrain-path4-structure.test.ts: structural lint over the
  Path 4 prose contract — STOP gates after verify failure, never-write-
  token rules, mode-aware CLAUDE.md block, bearer always via env-var.

Two new gate-tier E2E tests (deterministic stub HTTP server, fixed inputs):

- test/skill-e2e-setup-gbrain-remote.test.ts: Path 4 happy path. Stubs
  an HTTP MCP server, drives the skill via Agent SDK with a stubbed
  bearer, asserts claude.json gets the http MCP entry, CLAUDE.md gets
  the remote-http block, the secret token NEVER leaks to CLAUDE.md.
- test/skill-e2e-setup-gbrain-bad-token.test.ts: stub server returns 401;
  asserts the AUTH classifier hint surfaces, no MCP registration occurs,
  CLAUDE.md is unchanged. Regression guard for the "verify failed → STOP"
  rule.

touchfiles.ts: setup-gbrain-remote and setup-gbrain-bad-token added at
gate-tier so CI catches Path 4 regressions on every PR.

Plus a few comment refs flipped: bin/gstack-jsonl-merge, bin/gstack-timeline-log
(legacy gstack-brain-init mentions in headers).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* release: v1.27.0.0 — /setup-gbrain Path 4 + brain → artifacts rename

Bumps VERSION 1.26.4.0 → 1.27.0.0 (MINOR per CLAUDE.md scale-aware bump
guidance: ~1500 line net change including a new path in /setup-gbrain,
two new bin helpers, a journaled migration, 59 new tests, and a config
key rename across the codebase).

CHANGELOG entry covers: Path 4 (Remote MCP) end-to-end, the brain →
artifacts rename, the journaled migration, the verify-helper error
classifier, the artifacts-init multi-host provider choice. Includes
the canonical Garry-voice headline + numbers table + audience close
per the release-summary format.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: demote setup-gbrain Path 4 E2E to periodic-tier

The Agent SDK E2E tests for Path 4 (skill-e2e-setup-gbrain-remote and
skill-e2e-setup-gbrain-bad-token) are inherently non-deterministic —
the model interprets "follow Path 4 only" prompts flexibly and can
skip Step 8 (CLAUDE.md write) or shortcut past the verify helper, which
makes the gate-tier assertions flaky.

The deterministic gate coverage for Path 4 is in
test/setup-gbrain-path4-structure.test.ts: a fast structural lint that
catches AUQ-pacing regressions and prose contract drift in <200ms with
zero token spend. That test is the right tool for catching the failure
mode the gate-tier was meant to guard against.

The Agent SDK E2E tests stay available on-demand for periodic-tier runs
(EVALS=1 EVALS_TIER=periodic bun test test/skill-e2e-setup-gbrain-*.test.ts).
Also tightened the verify-error assertion to the literal field shape
("error_class": "AUTH") instead of a substring match that false-matches
the parent claude session's "needs-auth" MCP discovery markers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: sync package.json version to 1.27.0.0

VERSION was bumped to 1.27.0.0 in f6ec11eb but package.json was not
updated in the same commit. The gen-skill-docs.test.ts assertion
"package.json version matches VERSION file" caught the drift.

This is the DRIFT_STALE_PKG case the /ship Step 12 idempotency check
is designed for; the fix is the documented sync-only repair (no
re-bump, package.json synced to existing VERSION).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 19:37:53 -07:00

276 lines
8.7 KiB
TypeScript

/**
* gstack-gbrain-detect — gbrain_mcp_mode + gstack_artifacts_remote tests.
*
* The script has a 3-tier fallback chain for resolving gbrain_mcp_mode:
* 1. `claude mcp get gbrain --json` (preferred — public CLI surface)
* 2. `claude mcp list` text-grep (older claude versions without --json)
* 3. `~/.claude.json` jq read (fallback if claude binary is absent)
*
* Each layer is tested by mocking the layer it depends on. Per codex
* Finding #3 (defense-in-depth ordering): if Anthropic moves the
* ~/.claude.json file format, the first two tiers should still work.
*/
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { spawnSync } from 'child_process';
const ROOT = path.resolve(import.meta.dir, '..');
const DETECT_BIN = path.join(ROOT, 'bin', 'gstack-gbrain-detect');
let tmpHome: string;
let fakeBinDir: string;
function makeFakeClaude(opts: {
hasGetJson?: boolean;
getJsonOutput?: string; // raw JSON string
hasMcpList?: boolean;
mcpListOutput?: string;
exitOnAll?: number; // if set, claude always exits with this code
}) {
const { hasGetJson, getJsonOutput, hasMcpList, mcpListOutput, exitOnAll } = opts;
const script = `#!/bin/bash
${exitOnAll !== undefined ? `exit ${exitOnAll}` : ''}
case "$1 $2" in
"mcp get")
if [ "$3" = "gbrain" ] && [ "$4" = "--json" ]; then
${hasGetJson ? `cat <<'JSON'
${getJsonOutput || '{}'}
JSON` : 'exit 1'}
exit 0
fi
;;
"mcp list")
${hasMcpList ? `cat <<'EOM'
${mcpListOutput || ''}
EOM` : 'exit 1'}
exit 0
;;
esac
exit 1
`;
fs.writeFileSync(path.join(fakeBinDir, 'claude'), script, { mode: 0o755 });
}
function runDetect(extraEnv: Record<string, string> = {}): { code: number; json: any; stderr: string } {
const realPath = process.env.PATH ?? '';
const r = spawnSync(DETECT_BIN, [], {
env: {
// Put fakeBinDir first so our claude shim wins; include the project bin
// for any sibling scripts and standard paths for jq/etc.
PATH: `${fakeBinDir}:${path.join(ROOT, 'bin')}:${realPath}`,
HOME: tmpHome,
GSTACK_HOME: path.join(tmpHome, '.gstack'),
...extraEnv,
},
encoding: 'utf-8',
});
let json: any = null;
try {
json = JSON.parse(r.stdout || '{}');
} catch {
json = null;
}
return { code: r.status ?? -1, json, stderr: r.stderr || '' };
}
beforeEach(() => {
tmpHome = fs.mkdtempSync(path.join(os.tmpdir(), 'detect-mcp-mode-'));
fakeBinDir = fs.mkdtempSync(path.join(os.tmpdir(), 'detect-fake-bin-'));
});
afterEach(() => {
fs.rmSync(tmpHome, { recursive: true, force: true });
fs.rmSync(fakeBinDir, { recursive: true, force: true });
});
describe('gbrain_mcp_mode — Tier 1: claude mcp get --json', () => {
test('type=http → remote-http', () => {
makeFakeClaude({
hasGetJson: true,
getJsonOutput: JSON.stringify({ type: 'http', url: 'https://example.com/mcp' }),
});
const r = runDetect();
expect(r.code).toBe(0);
expect(r.json.gbrain_mcp_mode).toBe('remote-http');
});
test('type=stdio → local-stdio', () => {
makeFakeClaude({
hasGetJson: true,
getJsonOutput: JSON.stringify({ type: 'stdio', command: '/usr/local/bin/gbrain' }),
});
expect(runDetect().json.gbrain_mcp_mode).toBe('local-stdio');
});
test('type=sse → remote-http', () => {
makeFakeClaude({
hasGetJson: true,
getJsonOutput: JSON.stringify({ type: 'sse', url: 'https://example.com/sse' }),
});
expect(runDetect().json.gbrain_mcp_mode).toBe('remote-http');
});
test('no type field but has url → remote-http (newer claude shape)', () => {
makeFakeClaude({
hasGetJson: true,
getJsonOutput: JSON.stringify({ url: 'https://example.com/mcp' }),
});
expect(runDetect().json.gbrain_mcp_mode).toBe('remote-http');
});
test('no type field but has command → local-stdio', () => {
makeFakeClaude({
hasGetJson: true,
getJsonOutput: JSON.stringify({ command: '/path/to/gbrain' }),
});
expect(runDetect().json.gbrain_mcp_mode).toBe('local-stdio');
});
});
describe('gbrain_mcp_mode — Tier 2: claude mcp list text-grep', () => {
test('falls back to mcp list when get --json fails', () => {
makeFakeClaude({
hasGetJson: false,
hasMcpList: true,
mcpListOutput: 'gbrain: https://wintermute.tail554574.ts.net:3131/mcp (HTTP) - ✓ Connected',
});
expect(runDetect().json.gbrain_mcp_mode).toBe('remote-http');
});
test('mcp list text-grep with stdio entry → local-stdio', () => {
makeFakeClaude({
hasGetJson: false,
hasMcpList: true,
mcpListOutput: 'gbrain: /usr/local/bin/gbrain serve - ✓ Connected',
});
expect(runDetect().json.gbrain_mcp_mode).toBe('local-stdio');
});
test('mcp list with no gbrain entry → none', () => {
makeFakeClaude({
hasGetJson: false,
hasMcpList: true,
mcpListOutput: 'posthog: https://mcp.posthog.com/mcp (HTTP)\nslack: https://slack.com/mcp (HTTP)',
});
expect(runDetect().json.gbrain_mcp_mode).toBe('none');
});
});
describe('gbrain_mcp_mode — Tier 3: ~/.claude.json jq read', () => {
test('reads mcpServers.gbrain.type=url → remote-http', () => {
// No fake claude binary; force fallback to file read.
fs.writeFileSync(
path.join(tmpHome, '.claude.json'),
JSON.stringify({
mcpServers: { gbrain: { type: 'url', url: 'https://example.com/mcp' } },
})
);
expect(runDetect().json.gbrain_mcp_mode).toBe('remote-http');
});
test('reads mcpServers.gbrain.type=stdio → local-stdio', () => {
fs.writeFileSync(
path.join(tmpHome, '.claude.json'),
JSON.stringify({
mcpServers: { gbrain: { type: 'stdio', command: '/path/gbrain' } },
})
);
expect(runDetect().json.gbrain_mcp_mode).toBe('local-stdio');
});
test('infers from url field if type is missing', () => {
fs.writeFileSync(
path.join(tmpHome, '.claude.json'),
JSON.stringify({
mcpServers: { gbrain: { url: 'https://example.com/mcp' } },
})
);
expect(runDetect().json.gbrain_mcp_mode).toBe('remote-http');
});
test('infers from command field if type is missing', () => {
fs.writeFileSync(
path.join(tmpHome, '.claude.json'),
JSON.stringify({
mcpServers: { gbrain: { command: '/path/gbrain' } },
})
);
expect(runDetect().json.gbrain_mcp_mode).toBe('local-stdio');
});
test('no gbrain entry in ~/.claude.json → none', () => {
fs.writeFileSync(
path.join(tmpHome, '.claude.json'),
JSON.stringify({ mcpServers: { posthog: { type: 'url', url: 'https://x' } } })
);
expect(runDetect().json.gbrain_mcp_mode).toBe('none');
});
});
describe('gbrain_mcp_mode — no info anywhere', () => {
test('no claude binary AND no ~/.claude.json → none', () => {
// No fake claude, no file.
expect(runDetect().json.gbrain_mcp_mode).toBe('none');
});
});
describe('gstack_artifacts_remote', () => {
test('reads ~/.gstack-artifacts-remote.txt when present', () => {
fs.writeFileSync(
path.join(tmpHome, '.gstack-artifacts-remote.txt'),
'https://github.com/garrytan/gstack-artifacts-garrytan\n'
);
expect(runDetect().json.gstack_artifacts_remote).toBe(
'https://github.com/garrytan/gstack-artifacts-garrytan'
);
});
test('migration-window fallback: reads ~/.gstack-brain-remote.txt if artifacts file is missing', () => {
fs.writeFileSync(
path.join(tmpHome, '.gstack-brain-remote.txt'),
'git@github.com:garrytan/gstack-brain-garrytan.git\n'
);
expect(runDetect().json.gstack_artifacts_remote).toBe(
'git@github.com:garrytan/gstack-brain-garrytan.git'
);
});
test('artifacts file wins over brain file when both exist', () => {
fs.writeFileSync(
path.join(tmpHome, '.gstack-artifacts-remote.txt'),
'https://github.com/x/new\n'
);
fs.writeFileSync(
path.join(tmpHome, '.gstack-brain-remote.txt'),
'https://github.com/x/old\n'
);
expect(runDetect().json.gstack_artifacts_remote).toBe('https://github.com/x/new');
});
test('empty when neither file exists', () => {
expect(runDetect().json.gstack_artifacts_remote).toBe('');
});
});
describe('schema regression', () => {
test('output JSON has all expected keys (sync-gbrain compat)', () => {
const r = runDetect();
expect(r.code).toBe(0);
const keys = Object.keys(r.json).sort();
expect(keys).toEqual([
'gbrain_config_exists',
'gbrain_doctor_ok',
'gbrain_engine',
'gbrain_mcp_mode',
'gbrain_on_path',
'gbrain_version',
'gstack_artifacts_remote',
'gstack_brain_git',
'gstack_brain_sync_mode',
]);
});
});