From a1675f839055baafa5ceace1f5bba0d3205a1422 Mon Sep 17 00:00:00 2001 From: ezl-keygraph Date: Fri, 7 Aug 2026 00:28:23 +0530 Subject: [PATCH] feat(cli): support any Pi provider via generic SHANNON_AI_API_KEY (#415) * feat(cli): support any Pi provider via generic SHANNON_AI_API_KEY * docs(cli): point users to pi.dev/models for provider and model ids * docs: document generic provider path and pi.dev catalogue --- .env.example | 23 ++++--- CLAUDE.md | 2 +- README.md | 4 +- apps/cli/src/commands/setup.ts | 89 +++++++++++++++++++++------ apps/cli/src/config/resolver.ts | 31 ++++++++-- apps/cli/src/config/writer.ts | 2 + apps/cli/src/env.ts | 52 ++++++++++------ apps/cli/src/model-spec.ts | 41 ++++++------ apps/worker/src/ai/models.ts | 73 +++++++++++++--------- apps/worker/src/services/preflight.ts | 20 ++++-- docs/ai-providers.md | 43 ++++++++----- 11 files changed, 257 insertions(+), 123 deletions(-) diff --git a/.env.example b/.env.example index e972a1e..cd22ef8 100644 --- a/.env.example +++ b/.env.example @@ -3,16 +3,16 @@ # Defaults to anthropic:claude-sonnet-4-6. # --- Anthropic --------------------------------------------------------------- -ANTHROPIC_API_KEY=your-api-key-here +SHANNON_AI_API_KEY=your-api-key-here SHANNON_AI_MODEL=anthropic:claude-sonnet-4-6 # CLAUDE_CODE_OAUTH_TOKEN=your-oauth-token-here # --- OpenAI ------------------------------------------------------------------ -# OPENAI_API_KEY=your-api-key-here +# SHANNON_AI_API_KEY=your-api-key-here # SHANNON_AI_MODEL=openai:gpt-5.5 # --- xAI --------------------------------------------------------------------- -# XAI_API_KEY=your-api-key-here +# SHANNON_AI_API_KEY=your-api-key-here # SHANNON_AI_MODEL=xai:grok-4.5 # --- AWS Bedrock ------------------------------------------------------------- @@ -24,20 +24,27 @@ SHANNON_AI_MODEL=anthropic:claude-sonnet-4-6 # --- Custom Base URL --------------------------------------------------------- # Route through a proxy or gateway (LiteLLM, an internal endpoint). # Pick the block matching the API dialect your gateway speaks, and uncomment all -# three lines. The provider prefix picks the dialect and which key is sent; the -# model id is whatever name your gateway serves it under. +# three lines. The provider prefix picks the dialect; the model id is whatever +# name your gateway serves it under. # Anthropic compatible - Anthropic Messages: -# ANTHROPIC_API_KEY=your-gateway-key-here +# SHANNON_AI_API_KEY=your-gateway-key-here # SHANNON_AI_BASE_URL=https://llm-gateway.example.com # SHANNON_AI_MODEL=anthropic:claude-sonnet-4-6 # OpenAI compatible - Chat Completions (default) or Responses: -# OPENAI_API_KEY=your-gateway-key-here +# SHANNON_AI_API_KEY=your-gateway-key-here # SHANNON_AI_BASE_URL=https://llm-gateway.example.com/v1 # SHANNON_AI_MODEL=openai:gpt-5.5 # SHANNON_AI_OPENAI_FORMAT=responses -# --- Other ------------------------------------------------------------------- +# --- Other provider ---------------------------------------------------------- +# Any other provider the Pi harness supports. Name it in SHANNON_AI_MODEL and +# supply the key via the generic SHANNON_AI_API_KEY. Pi validates the provider +# and model at preflight. +# SHANNON_AI_MODEL=openrouter:moonshotai/kimi-k3 +# SHANNON_AI_API_KEY=your-api-key-here + +# --- Misc -------------------------------------------------------------------- # Forward /etc/hosts entries into the worker container. # SHANNON_FORWARD_HOSTS=false diff --git a/CLAUDE.md b/CLAUDE.md index 44d0f4d..681061c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -153,7 +153,7 @@ Durable workflow orchestration with crash recovery, queryable progress, intellig ### Supporting Systems - **Configuration** — YAML configs in `apps/worker/configs/` with JSON Schema validation (`config-schema.json`). Supports auth settings (MFA/TOTP), URL/code rule scoping (`rules.avoid`/`rules.focus`), run-scope steering (`vuln_classes`, `exploit`), free-form `rules_of_engagement`, and post-hoc `report` options (`min_severity`, `min_confidence`, `guidance`, and `sarif` to emit a SARIF 2.1.0 log via `apps/worker/src/services/sarif-renderer.ts`; exploit-only). `code_path` avoid rules are enforced via the `@gotgenes/pi-permission-system` extension: `apps/worker/src/temporal/activities.ts:syncCodePathDenyRules` writes a global `path` deny config once per workflow (`apps/worker/src/ai/pi/permission-system.ts:syncPermissionSystemConfig`), and the executor loads the extension when that config is present (`apps/worker/src/ai/pi/pi-executor.ts`), so denies fire across every tool and child `task` session. `vuln_classes`/`exploit` scope is locked into `session.json` on first run; resumes with a different scope fail fast (`persistOrValidateRunScope`). Credential resolution — local mode: env vars → `./.env`; npx mode: env vars → `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) - **Prompts** — Per-phase templates in `apps/worker/prompts/` with variable substitution (`{{TARGET_URL}}`, `{{CONFIG_CONTEXT}}`). Shared partials in `apps/worker/prompts/shared/` via `apps/worker/src/services/prompt-manager.ts`, including `_code-path-rules.txt` (focus/avoid `[FILE]`/`[GLOB]` routing) and `_rules-of-engagement.txt` (free-text engagement rules). When `exploit: false`, `apps/worker/src/services/findings-renderer.ts` deterministically converts each `*_exploitation_queue.json` into a `*_findings.md` for report assembly — no LLM in the loop -- **Agent Harness (pi)** — Uses the **pi harness** (`@earendil-works/pi-coding-agent`, requires Node ≥ 22.19) via `apps/worker/src/ai/pi/pi-executor.ts` (`runPiPrompt` → `createAgentSession`). Retry is split in `apps/worker/src/ai/pi/retry-settings.ts`: pi's agent-level loop is off so Temporal owns agent restarts, while `provider.maxRetries` stays on — pi reads the `provider` block independently of the `enabled` flag — so transport faults are absorbed in-session rather than costing a full agent re-run. `maxRetryDelayMs` is left at pi's 60s default. One model runs every phase, named by `SHANNON_AI_MODEL=:` (default `anthropic:claude-sonnet-4-6`). `apps/worker/src/ai/models.ts` parses the spec — splitting on the **first** colon only, so Bedrock IDs keep theirs — and resolves it through pi's `ModelRuntime`. pi ships the `CredentialStore` interface but no in-memory implementation (its own reads `auth.json` from disk), so `RuntimeCredentialStore` in that file supplies one: credentials arrive as env vars in an ephemeral container and must never touch disk. `createModelRuntime(providerId, apiKey)` builds the runtime; `allowModelNetwork` stays at its default `false` so a scan never blocks on a catalog refresh. `resolveModelSelection()` is **async** because `ModelRuntime.create()` is. Supported providers (all pi-ai provider ids): `anthropic`, `openai`, `xai`, `amazon-bedrock`. Each provider's API key env var is declared once in `PROVIDER_API_KEY_ENV` — Shannon uses each vendor's own variable name (`OPENAI_API_KEY`, `XAI_API_KEY`, …), never an invented one; Bedrock's entry is `AWS_BEARER_TOKEN_BEDROCK`, paired with `AWS_REGION`, which preflight requires separately as provider config rather than a credential. `SHANNON_AI_BASE_URL` overrides the endpoint for any provider (proxies/gateways); the credential is unchanged. `pointAtGateway` (`apps/worker/src/ai/models.ts`) applies the one dialect change: behind a base URL, `openai` follows `SHANNON_AI_OPENAI_FORMAT` (`chat-completions` default, or `responses`). On `chat-completions` it switches the API to `openai-completions` and drops the catalogue's Responses-shaped `compat` block so pi's `detectCompat` derives completions settings; on `responses` the descriptor is unchanged but for the endpoint. `resolveGatewayFormat` rejects the variable when the provider is not `openai` or no base URL is set, since it cannot take effect there. All other providers keep their API. The CLI mirrors the accepted values in `apps/cli/src/model-spec.ts`, forwards the variable in `COMMON_FORWARD_VARS`, and maps it to `openai.format` in config.toml. `buildEnvFlags` forwards only the selected provider's credential into the worker container. The CLI mirrors the parse rule and the provider/credential tables in `apps/cli/src/model-spec.ts` (it cannot import from the worker package); the two must stay in sync. pi ships no JSON-schema output or `Task`/`TodoWrite` built-ins, so structured queues are captured via a `submit_exploitation_queue` custom tool (`apps/worker/src/ai/queue-schemas.ts`), and `task` (child sessions scoped to `read`, `grep`, `find`, `ls`, `write`, and `bash` — no nested `task` or collector tools; `CHILD_TOOLS` in `apps/worker/src/ai/pi/task-tool.ts`) + `todo_write` (`apps/worker/src/ai/pi/session-tools.ts`) are provided as custom tools; the per-phase collectors are pi custom tools (TypeBox `defineTool` in `apps/worker/src/collectors/`). Shannon sets no thinking configuration at all — no `thinkingLevel` is passed to any `createAgentSession` call, so pi's own default applies. There is no adaptive-thinking support and no `CLAUDE_ADAPTIVE_THINKING` / `core.adaptive_thinking` setting. Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login +- **Agent Harness (pi)** — Uses the **pi harness** (`@earendil-works/pi-coding-agent`, requires Node ≥ 22.19) via `apps/worker/src/ai/pi/pi-executor.ts` (`runPiPrompt` → `createAgentSession`). Retry is split in `apps/worker/src/ai/pi/retry-settings.ts`: pi's agent-level loop is off so Temporal owns agent restarts, while `provider.maxRetries` stays on — pi reads the `provider` block independently of the `enabled` flag — so transport faults are absorbed in-session rather than costing a full agent re-run. `maxRetryDelayMs` is left at pi's 60s default. One model runs every phase, named by `SHANNON_AI_MODEL=:` (default `anthropic:claude-sonnet-4-6`). `apps/worker/src/ai/models.ts` parses the spec — splitting on the **first** colon only, so Bedrock IDs keep theirs — and resolves it through pi's `ModelRuntime`. pi ships the `CredentialStore` interface but no in-memory implementation (its own reads `auth.json` from disk), so `RuntimeCredentialStore` in that file supplies one: credentials arrive as env vars in an ephemeral container and must never touch disk. `createModelRuntime(providerId, apiKey)` builds the runtime; `allowModelNetwork` stays at its default `false` so a scan never blocks on a catalog refresh. `resolveModelSelection()` is **async** because `ModelRuntime.create()` is. Any pi-ai provider id is accepted — `parseModelSpec` no longer rejects against a hardcoded list, so pi's registry is the authority (an unknown provider/model surfaces as a clear "not found in pi registry" error at preflight, which points to the browsable catalogue at `pi.dev/models` — `PI_CATALOG_URL` in `apps/worker/src/ai/models.ts`, appended to the not-found errors and shown in the setup wizard's "Other provider" hint). Four providers are **curated** (`CURATED_PROVIDERS`: `anthropic`, `openai`, `xai`, `amazon-bedrock`) with their own credential variables, config sections, and setup flows; each provider's API key env var is declared once in `PROVIDER_API_KEY_ENV` — Shannon uses each vendor's own variable name (`OPENAI_API_KEY`, `XAI_API_KEY`, …), never an invented one; Bedrock's entry is `AWS_BEARER_TOKEN_BEDROCK`, paired with `AWS_REGION`, which preflight requires separately as provider config rather than a credential. Any other provider uses the **generic** credential path: `SHANNON_AI_API_KEY` (`GENERIC_API_KEY_ENV`) supplies the key for any provider whose credential is a plain API key. Curated providers' own variables take precedence over it, and it also works as a fallback for them — Bedrock is the sole exception (it authenticates through its AWS_ variables, so the generic key never stands in for it). The CLI forwards `SHANNON_AI_API_KEY` in `COMMON_FORWARD_VARS` (it is provider-neutral, binding to whatever `SHANNON_AI_MODEL` names, so the "only one provider configured" guard counts only named credentials), and stores it under a generic `[provider]` config.toml section (`provider.api_key`). `npx @keygraph/shannon setup` exposes this as the "Other provider" option: free-text provider id + model id + key (a curated provider id is rejected there, since it has its own option). `SHANNON_AI_BASE_URL` overrides the endpoint for any provider (proxies/gateways); the credential is unchanged. `pointAtGateway` (`apps/worker/src/ai/models.ts`) applies the one dialect change: behind a base URL, `openai` follows `SHANNON_AI_OPENAI_FORMAT` (`chat-completions` default, or `responses`). On `chat-completions` it switches the API to `openai-completions` and drops the catalogue's Responses-shaped `compat` block so pi's `detectCompat` derives completions settings; on `responses` the descriptor is unchanged but for the endpoint. `resolveGatewayFormat` rejects the variable when the provider is not `openai` or no base URL is set, since it cannot take effect there. All other providers keep their API. The CLI mirrors the accepted values in `apps/cli/src/model-spec.ts`, forwards the variable in `COMMON_FORWARD_VARS`, and maps it to `openai.format` in config.toml. `buildEnvFlags` forwards only the selected provider's credential into the worker container. The CLI mirrors the parse rule and the provider/credential tables in `apps/cli/src/model-spec.ts` (it cannot import from the worker package); the two must stay in sync. pi ships no JSON-schema output or `Task`/`TodoWrite` built-ins, so structured queues are captured via a `submit_exploitation_queue` custom tool (`apps/worker/src/ai/queue-schemas.ts`), and `task` (child sessions scoped to `read`, `grep`, `find`, `ls`, `write`, and `bash` — no nested `task` or collector tools; `CHILD_TOOLS` in `apps/worker/src/ai/pi/task-tool.ts`) + `todo_write` (`apps/worker/src/ai/pi/session-tools.ts`) are provided as custom tools; the per-phase collectors are pi custom tools (TypeBox `defineTool` in `apps/worker/src/collectors/`). Shannon sets no thinking configuration at all — no `thinkingLevel` is passed to any `createAgentSession` call, so pi's own default applies. There is no adaptive-thinking support and no `CLAUDE_ADAPTIVE_THINKING` / `core.adaptive_thinking` setting. Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login - **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds only the human-facing report (`Security-Assessment-Report.md`, `FINAL_REPORT_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`workspaces`/`logs`) without migration. Resuming a pre-restructure workspace upgrades it in place first: `migrateLegacyWorkspaceLayout` (`apps/cli/src/commands/start.ts`) renames the flat deliverables/logs/session entries into `.shannon/` (carrying the deliverables `.git` along) before the overlay dirs are mounted, so resume finds the old checkpoints instead of re-running every agent. The report is surfaced by copying the assembled `comprehensive_security_assessment_report.md` from the deliverables dir to the run root (`copyReportToRunRoot` in `apps/worker/src/services/reporting.ts`). WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive - **Deliverables** — Saved to `.shannon/deliverables/` in the target repo via the `save-deliverable` CLI script (`apps/worker/src/scripts/save-deliverable.ts`) - **Workspaces & Resume** — Named workspaces via `-w ` or auto-named from URL+timestamp. Resume detects completed agents via `session.json`. `loadResumeState()` in `apps/worker/src/temporal/activities.ts` validates deliverable existence, restores git checkpoints, and cleans up incomplete deliverables. Workspace listing via `apps/worker/src/temporal/workspaces.ts` diff --git a/README.md b/README.md index 8c07fd6..9c72302 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ Sample penetration test reports from intentionally vulnerable applications, prod - **Docker**: required for the worker container. - **Node.js 18+**: required for the recommended `npx` workflow. -- **AI provider credentials**: Anthropic, OpenAI, xAI, or AWS Bedrock. Claude models are recommended. For suggested model IDs per provider, plus gateways and custom base URLs, see [AI providers](docs/ai-providers.md#suggested-models). +- **AI provider credentials**: Anthropic, OpenAI, xAI, or AWS Bedrock. Any other [Pi-supported provider](https://pi.dev/models) is technically supported too, but not recommended. Claude models are recommended. For suggested model IDs per provider, plus gateways and custom base URLs, see [AI providers](docs/ai-providers.md#suggested-models). - **Cyber safeguards cleared with your provider**: Anthropic and OpenAI apply real-time safeguards to cyber-security workloads, which can interrupt a scan mid-run. Complete their guidance for legitimate security testers before your first run - see [AI providers](docs/ai-providers.md#cyber-safeguards-do-this-before-your-first-scan). ### Run Shannon @@ -190,7 +190,7 @@ Use these guides for operational detail: | --- | --- | | [Source build and CLI commands](docs/development.md) | Cloning, building, common commands, output paths, and local development. | | [Configuration](docs/configuration.md) | Authenticated testing, login flows, rules of engagement, and report filters. | -| [AI providers](docs/ai-providers.md) | Selecting the model, the supported providers (Anthropic, OpenAI, xAI, AWS Bedrock), and custom gateways. | +| [AI providers](docs/ai-providers.md) | Selecting the model, the supported providers (Anthropic, OpenAI, xAI, AWS Bedrock, and any other Pi-supported provider), and custom gateways. | | [Platforms and networking](docs/platforms.md) | Windows/WSL2, Linux, macOS, Docker networking, local apps, and custom hostnames. | | [Workspaces and resuming](docs/workspaces.md) | Naming workspaces, resuming interrupted scans, and workspace storage. | | [Safety and limitations](docs/safety.md) | Authorized-use requirements, non-production guidance, mutative effects, cost, and model caveats. | diff --git a/apps/cli/src/commands/setup.ts b/apps/cli/src/commands/setup.ts index fe4308d..11bd69e 100644 --- a/apps/cli/src/commands/setup.ts +++ b/apps/cli/src/commands/setup.ts @@ -10,13 +10,14 @@ import os from 'node:os'; import path from 'node:path'; import * as p from '@clack/prompts'; import { type ShannonConfig, saveConfig } from '../config/writer.js'; -import { type OpenAiFormat, type ProviderId, SUPPORTED_PROVIDERS } from '../model-spec.js'; +import { CURATED_PROVIDERS, type CuratedProviderId, isCuratedProvider, type OpenAiFormat } from '../model-spec.js'; import { requireInteractive } from '../tty.js'; const SHANNON_HOME = path.join(os.homedir(), '.shannon'); const CUSTOM_MODEL = '__custom__'; const CUSTOM_BASE_URL = '__custom_base_url__'; +const OTHER_PROVIDER = '__other_provider__'; /** * Wire formats reachable through the gateway route. The format picks the provider @@ -39,28 +40,34 @@ const GATEWAY_DIALECTS: readonly { { value: 'openai-responses', label: 'OpenAI Responses', provider: 'openai', format: 'responses' }, ]; -/** Suggested models per provider, best-first. Free-text entry accepts any model in the provider's catalogue. */ -const MODEL_SUGGESTIONS: Readonly> = { +/** Suggested models per curated provider, best-first. Free-text entry accepts any model in the provider's catalogue. */ +const MODEL_SUGGESTIONS: Readonly> = { anthropic: ['claude-sonnet-4-6', 'claude-opus-4-8', 'claude-opus-4-7', 'claude-haiku-4-5-20251001'], openai: ['gpt-5.6-sol', 'gpt-5.5', 'gpt-5.4'], xai: ['grok-4.5'], 'amazon-bedrock': ['us.anthropic.claude-sonnet-4-6', 'us.anthropic.claude-opus-4-8', 'us.anthropic.claude-opus-4-7'], }; -/** Placeholder shown in the free-text model ID prompt. */ -const MODEL_ID_PLACEHOLDER: Readonly> = { +/** Placeholder shown in the free-text model ID prompt, per curated provider. */ +const MODEL_ID_PLACEHOLDER: Readonly> = { anthropic: 'claude-sonnet-4-6', openai: 'gpt-5.6-sol', xai: 'grok-4.5', 'amazon-bedrock': 'us.anthropic.claude-opus-4-8', }; +/** Model ID placeholder for a provider, absent when the provider is not curated. */ +function modelIdPlaceholder(provider: string): string | undefined { + return isCuratedProvider(provider) ? MODEL_ID_PLACEHOLDER[provider] : undefined; +} + export async function setup(): Promise { requireInteractive('setup', 'For non-interactive use, export credentials as env vars (e.g. ANTHROPIC_API_KEY).'); p.intro('Shannon Setup'); // 1. Select provider. "Custom Base URL" is a route, not a provider — it asks - // which API dialect the gateway speaks and configures that provider. + // which API dialect the gateway speaks and configures that provider. "Other + // provider" reaches any pi-supported provider Shannon does not curate. const selected = await p.select({ message: 'Select your AI provider', options: [ @@ -69,14 +76,17 @@ export async function setup(): Promise { { value: 'xai' as const, label: 'xAI', hint: 'Grok models' }, { value: 'amazon-bedrock' as const, label: 'AWS Bedrock', hint: 'Claude models via AWS' }, { value: CUSTOM_BASE_URL as typeof CUSTOM_BASE_URL, label: 'Custom Base URL', hint: 'your own proxy or gateway' }, + { + value: OTHER_PROVIDER as typeof OTHER_PROVIDER, + label: 'Other provider', + hint: 'any other Pi-supported provider', + }, ], }); if (p.isCancel(selected)) return cancelAndExit(); // 2. Credentials — and, on the gateway route, the endpoint and its dialect. - const gateway = selected === CUSTOM_BASE_URL ? await setupGateway() : undefined; - const provider = gateway?.provider ?? (selected as ProviderId); - const config = gateway?.config ?? (await setupProvider(provider)); + const { provider, config, gateway } = await setupSelection(selected); // 3. The model that runs every phase. const modelId = await promptModel(provider); @@ -95,7 +105,27 @@ export async function setup(): Promise { p.outro('Run `npx @keygraph/shannon start` to begin a scan.'); } -async function setupProvider(provider: ProviderId): Promise { +interface Selection { + provider: string; + config: ShannonConfig; + gateway?: GatewaySetup; +} + +/** Resolve the provider selection into a provider id and its credential config. */ +async function setupSelection( + selected: CuratedProviderId | typeof CUSTOM_BASE_URL | typeof OTHER_PROVIDER, +): Promise { + if (selected === CUSTOM_BASE_URL) { + const gateway = await setupGateway(); + return { provider: gateway.provider, config: gateway.config, gateway }; + } + if (selected === OTHER_PROVIDER) { + return setupOtherProvider(); + } + return { provider: selected, config: await setupProvider(selected) }; +} + +async function setupProvider(provider: CuratedProviderId): Promise { switch (provider) { case 'amazon-bedrock': return setupBedrock(); @@ -108,6 +138,27 @@ async function setupProvider(provider: ProviderId): Promise { } } +/** + * Any pi provider Shannon does not curate. The id is free text — the worker's + * preflight validates it — and the key is stored generically as SHANNON_AI_API_KEY. + */ +async function setupOtherProvider(): Promise { + p.log.info('Browse supported providers and models at https://pi.dev/models'); + const provider = await p.text({ + message: 'Provider ID', + validate: (value) => { + const id = value?.trim(); + if (!id) return 'Provider ID is required'; + if (isCuratedProvider(id)) return `${id} has its own option.`; + return undefined; + }, + }); + if (p.isCancel(provider)) return cancelAndExit(); + + const apiKey = await promptSecret('Enter the API key'); + return { provider: provider.trim(), config: { provider: { api_key: apiKey } } }; +} + // === Provider Setup Flows === async function setupAnthropic(): Promise { @@ -143,7 +194,7 @@ async function setupBedrock(): Promise { } interface GatewaySetup { - provider: ProviderId; + provider: CuratedProviderId; config: ShannonConfig; baseUrl: string; format?: OpenAiFormat; @@ -195,11 +246,11 @@ async function setupGateway(): Promise { * Ask for the one model that runs every phase. Providers with suggestions offer a * pick list with a free-text escape hatch; the rest go straight to free text. */ -async function promptModel(provider: ProviderId): Promise { - const suggestions = MODEL_SUGGESTIONS[provider]; +async function promptModel(provider: string): Promise { + const suggestions = isCuratedProvider(provider) ? MODEL_SUGGESTIONS[provider] : []; if (suggestions.length === 0) { - return promptModelId(provider, MODEL_ID_PLACEHOLDER[provider]); + return promptModelId(provider, modelIdPlaceholder(provider)); } const choice = await p.select({ @@ -212,7 +263,7 @@ async function promptModel(provider: ProviderId): Promise { if (p.isCancel(choice)) return cancelAndExit(); if (choice === CUSTOM_MODEL) { - return promptModelId(provider, MODEL_ID_PLACEHOLDER[provider]); + return promptModelId(provider, modelIdPlaceholder(provider)); } return choice as string; } @@ -222,13 +273,13 @@ async function promptModel(provider: ProviderId): Promise { * one. Bedrock model IDs carry their own colons (`…-v1:0`), so only a genuine * provider id counts as a prefix. */ -function conflictingProviderPrefix(provider: ProviderId, value: string): string | undefined { +function conflictingProviderPrefix(provider: string, value: string): string | undefined { const separator = value.indexOf(':'); if (separator === -1) return undefined; const head = value.slice(0, separator); if (head === provider) return undefined; - return (SUPPORTED_PROVIDERS as readonly string[]).includes(head) ? head : undefined; + return (CURATED_PROVIDERS as readonly string[]).includes(head) ? head : undefined; } /** @@ -236,10 +287,10 @@ function conflictingProviderPrefix(provider: ProviderId, value: string): string * and the caller pairs it with the provider — pasting a full `:` * spec just has its redundant prefix dropped. */ -async function promptModelId(provider: ProviderId, placeholder: string): Promise { +async function promptModelId(provider: string, placeholder?: string): Promise { const modelId = await p.text({ message: 'Model ID', - placeholder, + ...(placeholder && { placeholder }), validate: (value) => { if (!value) return 'Model ID is required'; const conflicting = conflictingProviderPrefix(provider, value); diff --git a/apps/cli/src/config/resolver.ts b/apps/cli/src/config/resolver.ts index f1ba735..6bf218a 100644 --- a/apps/cli/src/config/resolver.ts +++ b/apps/cli/src/config/resolver.ts @@ -9,7 +9,13 @@ import fs from 'node:fs'; import { parse as parseTOML } from 'smol-toml'; import { getConfigFile } from '../home.js'; import { getMode } from '../mode.js'; -import { DEFAULT_MODEL_SPEC, type ProviderId, parseModelSpec } from '../model-spec.js'; +import { + type CuratedProviderId, + DEFAULT_MODEL_SPEC, + GENERIC_API_KEY_ENV, + isCuratedProvider, + parseModelSpec, +} from '../model-spec.js'; // === TOML ↔ Env Mapping === @@ -42,16 +48,22 @@ const CONFIG_MAP: readonly ConfigMapping[] = [ // Bedrock { env: 'AWS_REGION', toml: 'bedrock.region', type: 'string' }, { env: 'AWS_BEARER_TOKEN_BEDROCK', toml: 'bedrock.token', type: 'string' }, + + // Generic — credential for any provider Shannon does not curate + { env: GENERIC_API_KEY_ENV, toml: 'provider.api_key', type: 'string' }, ] as const; -/** TOML section holding each provider's credentials, keyed by provider id. */ -const PROVIDER_SECTIONS: Readonly> = { +/** TOML section holding each curated provider's credentials, keyed by provider id. */ +const PROVIDER_SECTIONS: Readonly> = { anthropic: 'anthropic', openai: 'openai', xai: 'xai', 'amazon-bedrock': 'bedrock', }; +/** TOML section holding the generic credential for uncurated providers. */ +const GENERIC_PROVIDER_SECTION = 'provider'; + // === TOML Parsing === type TOMLValue = string | number | boolean; @@ -128,9 +140,18 @@ function buildSchema(): Map> { /** * Check that the section backing the selected provider carries a usable * credential. `core.model` names the provider, so only that section is required; - * other providers' sections are ignored and never forwarded. + * other providers' sections are ignored and never forwarded. An uncurated + * provider draws its credential from the generic [provider] section. */ -function validateProviderFields(config: TOMLConfig, providerId: ProviderId, errors: string[]): void { +function validateProviderFields(config: TOMLConfig, providerId: string, errors: string[]): void { + if (!isCuratedProvider(providerId)) { + const section = config[GENERIC_PROVIDER_SECTION] as Record | undefined; + if (!section || !Object.keys(section).includes('api_key')) { + errors.push(`[${GENERIC_PROVIDER_SECTION}] requires api_key for provider "${providerId}"`); + } + return; + } + const sectionName = PROVIDER_SECTIONS[providerId]; const section = config[sectionName] as Record | undefined; const keys = section ? Object.keys(section) : []; diff --git a/apps/cli/src/config/writer.ts b/apps/cli/src/config/writer.ts index fb8ea59..44e893c 100644 --- a/apps/cli/src/config/writer.ts +++ b/apps/cli/src/config/writer.ts @@ -13,6 +13,8 @@ export interface ShannonConfig { openai?: { api_key?: string; format?: string }; xai?: { api_key?: string }; bedrock?: { region?: string; token?: string }; + /** Generic credential for any provider Shannon does not curate. Maps to SHANNON_AI_API_KEY. */ + provider?: { api_key?: string }; } // === File Operations === diff --git a/apps/cli/src/env.ts b/apps/cli/src/env.ts index 0821a34..75ef6ab 100644 --- a/apps/cli/src/env.ts +++ b/apps/cli/src/env.ts @@ -9,25 +9,35 @@ import dotenv from 'dotenv'; import { resolveConfig } from './config/resolver.js'; import { getMode } from './mode.js'; import { + CURATED_PROVIDERS, + type CuratedProviderId, + GENERIC_API_KEY_ENV, + isCuratedProvider, PROVIDER_API_KEY_ENV, PROVIDER_CREDENTIAL_HINT, PROVIDER_EXTRA_ENV, - type ProviderId, resolveModelSpec, - SUPPORTED_PROVIDERS, } from './model-spec.js'; /** * Variables forwarded to every worker container regardless of provider. Each is * forwarded only when set, so an unused one never appears in the container. + * SHANNON_AI_API_KEY rides along because it is provider-neutral. */ -const COMMON_FORWARD_VARS = ['SHANNON_AI_MODEL', 'SHANNON_AI_BASE_URL', 'SHANNON_AI_OPENAI_FORMAT'] as const; +const COMMON_FORWARD_VARS = [ + 'SHANNON_AI_MODEL', + 'SHANNON_AI_BASE_URL', + 'SHANNON_AI_OPENAI_FORMAT', + GENERIC_API_KEY_ENV, +] as const; /** * Credential variables for one provider. Only the selected provider's entries are - * forwarded, so a key for an unused provider never enters the scan container. + * forwarded, so a key for an unused provider never enters the scan container. An + * uncurated provider has none — it relies on the common SHANNON_AI_API_KEY. */ -function providerForwardVars(providerId: ProviderId): readonly string[] { +function providerForwardVars(providerId: string): readonly string[] { + if (!isCuratedProvider(providerId)) return []; return [...PROVIDER_API_KEY_ENV[providerId], ...PROVIDER_EXTRA_ENV[providerId]]; } @@ -70,22 +80,23 @@ interface CredentialValidation { error?: string; } -/** - * Whether the selected provider has a usable credential in the environment. Any - * one API key satisfies a key-based provider; Bedrock instead needs every one of - * its AWS_ vars. - */ -function hasCredential(providerId: ProviderId): boolean { +/** Whether a curated provider has its own named credential set (API key plus any extra var). */ +function hasNamedCredential(providerId: CuratedProviderId): boolean { const apiKeys = PROVIDER_API_KEY_ENV[providerId]; - if (apiKeys.length > 0 && !apiKeys.some((name) => Boolean(process.env[name]))) { - return false; - } + if (!apiKeys.some((name) => Boolean(process.env[name]))) return false; return PROVIDER_EXTRA_ENV[providerId].every((name) => Boolean(process.env[name])); } -/** Every provider that currently has a complete credential in the environment. */ -function configuredProviders(): ProviderId[] { - return SUPPORTED_PROVIDERS.filter((providerId) => hasCredential(providerId)); +/** Whether the selected provider has a credential. Bedrock needs its AWS_ vars; the generic key never stands in for it. */ +function hasCredential(providerId: string): boolean { + if (providerId === 'amazon-bedrock') return hasNamedCredential('amazon-bedrock'); + if (isCuratedProvider(providerId) && hasNamedCredential(providerId)) return true; + return Boolean(process.env[GENERIC_API_KEY_ENV]); +} + +/** Curated providers with a named credential. The generic key is neutral, so it never counts toward ambiguity. */ +function configuredProviders(): CuratedProviderId[] { + return CURATED_PROVIDERS.filter((providerId) => hasNamedCredential(providerId)); } /** @@ -93,7 +104,7 @@ function configuredProviders(): ProviderId[] { * Runs before any Docker work so mistakes fail immediately. */ export function validateCredentials(): CredentialValidation { - // 1. Model selection must parse and name a supported provider + // 1. Model selection must parse into a provider and model id const spec = resolveModelSpec(); if (typeof spec === 'string') { return { valid: false, error: spec }; @@ -101,9 +112,12 @@ export function validateCredentials(): CredentialValidation { // 2. The selected provider must have a credential if (!hasCredential(spec.providerId)) { + const requirement = isCuratedProvider(spec.providerId) + ? PROVIDER_CREDENTIAL_HINT[spec.providerId] + : GENERIC_API_KEY_ENV; const hint = getMode() === 'local' - ? `Set ${PROVIDER_CREDENTIAL_HINT[spec.providerId]} in .env or export it.` + ? `Set ${requirement} in .env or export it.` : `Export the variables or run 'npx @keygraph/shannon setup'.`; return { valid: false, diff --git a/apps/cli/src/model-spec.ts b/apps/cli/src/model-spec.ts index cc1ec28..2173a78 100644 --- a/apps/cli/src/model-spec.ts +++ b/apps/cli/src/model-spec.ts @@ -6,24 +6,35 @@ * rule are duplicated here deliberately and must stay in sync. */ -/** Providers Shannon can currently reach. Each is a pi-ai provider id. */ -export const SUPPORTED_PROVIDERS = ['anthropic', 'openai', 'xai', 'amazon-bedrock'] as const; +/** + * Providers Shannon curates with their own credential variables, config sections, + * and setup flows. Any other pi provider is reachable via the generic credential + * path. Mirrors CURATED_PROVIDERS in apps/worker/src/ai/models.ts. + */ +export const CURATED_PROVIDERS = ['anthropic', 'openai', 'xai', 'amazon-bedrock'] as const; -export type ProviderId = (typeof SUPPORTED_PROVIDERS)[number]; +export type CuratedProviderId = (typeof CURATED_PROVIDERS)[number]; + +export function isCuratedProvider(value: string): value is CuratedProviderId { + return (CURATED_PROVIDERS as readonly string[]).includes(value); +} + +/** Generic API key, honored for any provider Shannon does not curate. Mirrors the worker. */ +export const GENERIC_API_KEY_ENV = 'SHANNON_AI_API_KEY'; /** - * Env vars carrying each provider's API key, in precedence order. Any one of them - * satisfies the provider. Mirrors PROVIDER_API_KEY_ENV in apps/worker/src/ai/models.ts. + * Env vars carrying each curated provider's API key, in precedence order. Any one of + * them satisfies the provider. Mirrors PROVIDER_API_KEY_ENV in apps/worker/src/ai/models.ts. */ -export const PROVIDER_API_KEY_ENV: Readonly> = { +export const PROVIDER_API_KEY_ENV: Readonly> = { anthropic: ['ANTHROPIC_API_KEY', 'CLAUDE_CODE_OAUTH_TOKEN'], openai: ['OPENAI_API_KEY'], xai: ['XAI_API_KEY'], 'amazon-bedrock': ['AWS_BEARER_TOKEN_BEDROCK'], }; -/** Additional env vars a provider requires beyond its API key. All must be set. */ -export const PROVIDER_EXTRA_ENV: Readonly> = { +/** Additional env vars a curated provider requires beyond its API key. All must be set. */ +export const PROVIDER_EXTRA_ENV: Readonly> = { anthropic: [], openai: [], xai: [], @@ -31,7 +42,7 @@ export const PROVIDER_EXTRA_ENV: Readonly> }; /** Human-readable credential requirement, used in "nothing configured" errors. */ -export const PROVIDER_CREDENTIAL_HINT: Readonly> = { +export const PROVIDER_CREDENTIAL_HINT: Readonly> = { anthropic: 'ANTHROPIC_API_KEY (or CLAUDE_CODE_OAUTH_TOKEN)', openai: 'OPENAI_API_KEY', xai: 'XAI_API_KEY', @@ -51,18 +62,15 @@ export const OPENAI_FORMATS = ['chat-completions', 'responses'] as const; export type OpenAiFormat = (typeof OPENAI_FORMATS)[number]; export interface ModelSpec { - providerId: ProviderId; + providerId: string; modelId: string; } -function isSupportedProvider(value: string): value is ProviderId { - return (SUPPORTED_PROVIDERS as readonly string[]).includes(value); -} - /** * Parse a `:` spec. Splits on the first colon only, so colons * inside a model ID survive (`amazon-bedrock:us.anthropic.claude-opus-4-5-20251101-v1:0`). - * Returns an error string rather than throwing, for the CLI's validation flow. + * The provider id is passed through as given — the worker's preflight validates it + * against pi. Returns an error string rather than throwing, for the CLI's flow. */ export function parseModelSpec(spec: string): ModelSpec | string { const trimmed = spec.trim(); @@ -74,9 +82,6 @@ export function parseModelSpec(spec: string): ModelSpec | string { const modelId = trimmed.slice(separator + 1).trim(); if (!providerId || !modelId) return malformed; - if (!isSupportedProvider(providerId)) { - return `Unsupported provider "${providerId}" in SHANNON_AI_MODEL. Supported providers: ${SUPPORTED_PROVIDERS.join(', ')}`; - } return { providerId, modelId }; } diff --git a/apps/worker/src/ai/models.ts b/apps/worker/src/ai/models.ts index eab2f3e..e9ddec8 100644 --- a/apps/worker/src/ai/models.ts +++ b/apps/worker/src/ai/models.ts @@ -24,18 +24,29 @@ import type { Api, Credential, CredentialInfo, CredentialStore, Model } from '@earendil-works/pi-ai'; import { ModelRuntime } from '@earendil-works/pi-coding-agent'; -/** Providers Shannon can currently reach. Each is a pi-ai provider id. */ -export const SUPPORTED_PROVIDERS = ['anthropic', 'openai', 'xai', 'amazon-bedrock'] as const; +/** + * Providers Shannon curates with their own credential variables, config sections, + * and setup flows. Each is a pi-ai provider id; any other pi provider is still + * reachable through the generic credential path below. + */ +export const CURATED_PROVIDERS = ['anthropic', 'openai', 'xai', 'amazon-bedrock'] as const; -export type ProviderId = (typeof SUPPORTED_PROVIDERS)[number]; +export type CuratedProviderId = (typeof CURATED_PROVIDERS)[number]; + +function isCuratedProvider(value: string): value is CuratedProviderId { + return (CURATED_PROVIDERS as readonly string[]).includes(value); +} + +/** Generic API key, honored for any provider Shannon does not curate. */ +export const GENERIC_API_KEY_ENV = 'SHANNON_AI_API_KEY'; /** - * Env vars carrying each provider's API key, in precedence order. Shannon does not - * invent credential names — these are the variables each provider's own tooling - * uses. Bedrock pairs its bearer token with AWS_REGION, which is provider config - * rather than a credential. + * Env vars carrying each curated provider's API key, in precedence order. Shannon + * does not invent credential names — these are the variables each provider's own + * tooling uses. Bedrock pairs its bearer token with AWS_REGION, which is provider + * config rather than a credential. */ -export const PROVIDER_API_KEY_ENV: Readonly> = { +export const PROVIDER_API_KEY_ENV: Readonly> = { anthropic: ['ANTHROPIC_API_KEY', 'CLAUDE_CODE_OAUTH_TOKEN'], openai: ['OPENAI_API_KEY'], xai: ['XAI_API_KEY'], @@ -45,6 +56,9 @@ export const PROVIDER_API_KEY_ENV: Readonly:` ids. */ +export const PI_CATALOG_URL = 'https://pi.dev/models'; + /** * Wire formats an OpenAI-compatible gateway may serve, named by * SHANNON_AI_OPENAI_FORMAT. Only `openai` offers a choice: every other supported @@ -82,18 +96,14 @@ export function resolveOpenAiFormat(): OpenAiFormat | undefined { } export interface ModelSpec { - providerId: ProviderId; + providerId: string; modelId: string; } -function isSupportedProvider(value: string): value is ProviderId { - return (SUPPORTED_PROVIDERS as readonly string[]).includes(value); -} - /** - * Parse a `:` spec. Splits on the first colon only, so - * colons inside a model ID survive. Throws with the supported provider list on - * a malformed or unknown provider. + * Parse a `:` spec. Splits on the first colon only, so colons + * inside a model ID survive. The provider id is passed through as given — pi's + * registry validates it later — so this throws only on a malformed spec. */ export function parseModelSpec(spec: string): ModelSpec { const trimmed = spec.trim(); @@ -112,11 +122,6 @@ export function parseModelSpec(spec: string): ModelSpec { `SHANNON_AI_MODEL must be ":", got "${trimmed}". Example: ${DEFAULT_MODEL_SPEC}`, ); } - if (!isSupportedProvider(providerId)) { - throw new Error( - `Unsupported provider "${providerId}" in SHANNON_AI_MODEL. Supported providers: ${SUPPORTED_PROVIDERS.join(', ')}`, - ); - } return { providerId, modelId }; } @@ -133,17 +138,25 @@ export interface ProviderCredentials { apiKey?: string; } -/** Collect the API key and optional endpoint override for a provider. */ -export function resolveProviderCredentials(providerId: ProviderId): ProviderCredentials { +/** + * Collect the API key and optional endpoint override for a provider. A curated + * provider's own variables win, then the generic SHANNON_AI_API_KEY. Bedrock is + * excluded — it authenticates through its AWS_ variables, which pi reads directly. + */ +export function resolveProviderCredentials(providerId: string): ProviderCredentials { const credentials: ProviderCredentials = {}; - for (const name of PROVIDER_API_KEY_ENV[providerId]) { + const namedVars = isCuratedProvider(providerId) ? PROVIDER_API_KEY_ENV[providerId] : []; + for (const name of namedVars) { const value = process.env[name]; if (value) { credentials.apiKey = value; break; } } + if (!credentials.apiKey && providerId !== 'amazon-bedrock' && process.env[GENERIC_API_KEY_ENV]) { + credentials.apiKey = process.env[GENERIC_API_KEY_ENV]; + } if (process.env.SHANNON_AI_BASE_URL) credentials.baseUrl = process.env.SHANNON_AI_BASE_URL; return credentials; @@ -203,7 +216,7 @@ export interface ModelSelection { model: Model; modelRuntime: ModelRuntime; modelId: string; - providerId: ProviderId; + providerId: string; } /** @@ -218,7 +231,7 @@ export interface ModelSelection { * then describes the format in use. Every other provider has one API and only * changes address. */ -function pointAtGateway(model: Model, providerId: ProviderId, baseUrl: string, format: OpenAiFormat): Model { +function pointAtGateway(model: Model, providerId: string, baseUrl: string, format: OpenAiFormat): Model { if (providerId !== 'openai') return { ...model, baseUrl }; if (format === 'responses') return { ...model, baseUrl, api: OPENAI_FORMATS.responses }; @@ -240,7 +253,7 @@ function pointAtGateway(model: Model, providerId: ProviderId, baseUrl: stri */ export function resolveModel( modelRuntime: ModelRuntime, - providerId: ProviderId, + providerId: string, modelId: string, baseUrl: string | undefined, format: OpenAiFormat = DEFAULT_OPENAI_FORMAT, @@ -265,7 +278,7 @@ export function resolveModel( * are configured, so it is rejected outside that combination rather than * silently ignored. */ -export function resolveGatewayFormat(providerId: ProviderId, baseUrl: string | undefined): OpenAiFormat { +export function resolveGatewayFormat(providerId: string, baseUrl: string | undefined): OpenAiFormat { const configured = resolveOpenAiFormat(); if (!configured) return DEFAULT_OPENAI_FORMAT; @@ -296,7 +309,9 @@ export async function resolveModelSelection(): Promise { const model = resolveModel(modelRuntime, providerId, modelId, credentials.baseUrl, format); if (!model) { - throw new Error(`Model not found in pi registry: provider="${providerId}" model="${modelId}"`); + throw new Error( + `Model not found in pi registry: provider="${providerId}" model="${modelId}". Browse valid providers and models at ${PI_CATALOG_URL}.`, + ); } return { diff --git a/apps/worker/src/services/preflight.ts b/apps/worker/src/services/preflight.ts index b2fbc74..b687ccc 100644 --- a/apps/worker/src/services/preflight.ts +++ b/apps/worker/src/services/preflight.ts @@ -36,10 +36,12 @@ import { } from '@earendil-works/pi-coding-agent'; import { glob } from 'zx'; import { + type CuratedProviderId, createModelRuntime, + GENERIC_API_KEY_ENV, type ModelSpec, type OpenAiFormat, - type ProviderId, + PI_CATALOG_URL, resolveGatewayFormat, resolveModel, resolveModelSpec, @@ -277,16 +279,22 @@ async function probeCredentialsWithPi( return ok(undefined); } -/** Credential env var a provider reads, for "credential missing" messages. */ -const PROVIDER_CREDENTIAL_HINT: Readonly> = { +/** Credential env var a curated provider reads, for "credential missing" messages. */ +const PROVIDER_CREDENTIAL_HINT: Readonly> = { anthropic: 'ANTHROPIC_API_KEY (or CLAUDE_CODE_OAUTH_TOKEN)', openai: 'OPENAI_API_KEY', xai: 'XAI_API_KEY', 'amazon-bedrock': 'AWS_BEARER_TOKEN_BEDROCK and AWS_REGION', }; +/** Which variable to set when a provider's credential is missing. */ +function credentialHint(providerId: string): string { + const curated = (PROVIDER_CREDENTIAL_HINT as Record)[providerId]; + return curated ?? GENERIC_API_KEY_ENV; +} + /** Human-readable label for which credential path a run is using. */ -function describeAuth(providerId: ProviderId, baseUrl: string | undefined): string { +function describeAuth(providerId: string, baseUrl: string | undefined): string { if (baseUrl) return `custom endpoint (${baseUrl})`; if (providerId === 'amazon-bedrock') return 'Bedrock bearer token'; return `${providerId} API key`; @@ -338,7 +346,7 @@ async function validateCredentials(logger: ActivityLogger): Promise 0 || (!isBedrock && !credentials.apiKey)) { return err( new PentestError( - `No credentials found for provider "${spec.providerId}". Set ${PROVIDER_CREDENTIAL_HINT[spec.providerId]} in .env.`, + `No credentials found for provider "${spec.providerId}". Set ${credentialHint(spec.providerId)} in .env.`, 'config', false, { providerId: spec.providerId, ...(missing.length > 0 && { missing }) }, @@ -356,7 +364,7 @@ async function validateCredentials(logger: ActivityLogger): Promise [!NOTE] -> Only the **first** colon separates the provider from the model ID, so Bedrock IDs that contain colons work unchanged: `amazon-bedrock:us.anthropic.claude-opus-4-5-20251101-v1:0`. +### Any other provider + +Shannon accepts any provider and model present in the Pi harness catalogue. Browse them at [pi.dev/models](https://pi.dev/models). These are technically supported but not recommended. Claude models are best-supported (see the note below). + +```bash +export SHANNON_AI_API_KEY=your-api-key # the provider's API key +export SHANNON_AI_MODEL=openrouter:moonshotai/kimi-k3 # : +``` + +This path covers providers whose credential is a single API key. Providers that need more than that are not currently supported. + +`npx @keygraph/shannon setup` exposes this as the **Other provider** option. > [!IMPORTANT] > Claude models are the best-supported option. Shannon's evaluations, internal testing, and agent harness are tuned for Claude. Other models are permitted and validated against the harness catalogue, but may not follow Shannon's instructions or tool-use constraints as reliably. Use them at your own risk. @@ -58,21 +70,21 @@ The pattern is learned once: export the provider's key, name the model. Two line Anthropic (default): ```bash -export ANTHROPIC_API_KEY=sk-ant-... +export SHANNON_AI_API_KEY=sk-ant-... export SHANNON_AI_MODEL=anthropic:claude-sonnet-4-6 ``` OpenAI: ```bash -export OPENAI_API_KEY=sk-... +export SHANNON_AI_API_KEY=sk-... export SHANNON_AI_MODEL=openai:gpt-5.6-sol ``` xAI: ```bash -export XAI_API_KEY=xai-... +export SHANNON_AI_API_KEY=xai-... export SHANNON_AI_MODEL=xai:grok-4.5 ``` @@ -96,16 +108,16 @@ To route model traffic through your own infrastructure — a corporate proxy, an | Gateway serves | Model prefix | API key | | --- | --- | --- | -| Anthropic Messages | `anthropic:` | `ANTHROPIC_API_KEY` | -| OpenAI Chat Completions | `openai:` | `OPENAI_API_KEY` | -| OpenAI Responses | `openai:` + `SHANNON_AI_OPENAI_FORMAT=responses` | `OPENAI_API_KEY` | +| Anthropic Messages | `anthropic:` | `SHANNON_AI_API_KEY` | +| OpenAI Chat Completions | `openai:` | `SHANNON_AI_API_KEY` | +| OpenAI Responses | `openai:` + `SHANNON_AI_OPENAI_FORMAT=responses` | `SHANNON_AI_API_KEY` | The model ID is whatever name your gateway serves it under; it does not have to exist in Shannon's catalogue. Anthropic Messages: ```bash -export ANTHROPIC_API_KEY=sk-ant-... +export SHANNON_AI_API_KEY=sk-ant-... export SHANNON_AI_MODEL=anthropic:claude-sonnet-4-6 export SHANNON_AI_BASE_URL=https://llm-gateway.example.com ``` @@ -113,7 +125,7 @@ export SHANNON_AI_BASE_URL=https://llm-gateway.example.com OpenAI Chat Completions: ```bash -export OPENAI_API_KEY=sk-... +export SHANNON_AI_API_KEY=sk-... export SHANNON_AI_MODEL=openai:gpt-5.6-sol export SHANNON_AI_BASE_URL=https://llm-gateway.example.com/v1 ``` @@ -136,8 +148,7 @@ The variable is rejected in preflight where it cannot take effect: with a non-`o Checks run before a scan starts, so mistakes fail immediately rather than partway through a run: -- **Provider** — always validated against the providers Shannon's harness knows. An unrecognised provider is rejected with the valid list. -- **Model ID** — validated against the harness catalogue for that provider, so a typo is caught instantly. +- **Provider and model ID** — validated against the Pi harness catalogue. An unknown provider or model ID fails preflight with a pointer to [pi.dev/models](https://pi.dev/models). A custom base URL exempts the model ID, since a gateway may serve its own names. - **Credential presence** — always validated for the selected provider. - **Credential validity** — one minimal request against the model the scan will use, so a rejected key, an exhausted quota, or a model the account cannot reach fails before any agent runs. Bedrock included: its bearer token and region go through the same probe.