diff --git a/CLAUDE.md b/CLAUDE.md index 560966e9..c668603c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -165,7 +165,7 @@ Around those phases: - **Configuration** — YAML configs in `apps/worker/configs/` use the closed JSON Schema in `config-schema.json`. Every fresh scan runs the fixed five analysis classes; there is no public class selector. `agentic_sast.enabled` is the only public agentic-SAST setting. Finding reconciliation runs on every scan and has no public setting of its own. Config also supports authentication (MFA/TOTP), URL/code rule scoping (`rules.avoid`/`rules.focus`), `exploit`, free-form `rules_of_engagement`, and post-hoc `report` options (`min_severity`, `min_confidence`, `guidance`, and exploit-only `sarif` output via `apps/worker/src/services/sarif-renderer.ts`, on by default for exploit runs and opt out with `report.sarif: "false"`). `code_path` avoid rules are enforced via the `@gotgenes/pi-permission-system` extension: `apps/worker/src/temporal/activities.ts:syncCodePathDenyRules` writes a global `path` deny config once per workflow (`apps/worker/src/ai/pi/permission-system.ts:syncPermissionSystemConfig`), and the executor loads the extension when that config is present (`apps/worker/src/ai/pi/pi-executor.ts`), so denies fire across every tool and child `task` session. Credential resolution — local mode: env vars → `./.env`; npx mode: env vars → `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) - **Agentic SAST progress** — Capella runs as a child workflow, so its activities are absent from the parent's `pendingActivities` and invisible to the CLI. The child signals each stage boundary up via `capellaStageProgress` (`apps/worker/src/temporal/shared.ts`); the parent's handler validates the payload and writes the child-supplied `startedAt` and `durationMs` directly to `operationalStages['agentic-sast:']`, so both the live `getProgress` query and the terminal result carry per-stage rows. Signalling is best-effort and every failure is swallowed — a closed or unreachable parent must never fail a SAST run. `CAPELLA_STAGE_LABELS` in `apps/worker/src/ai/sast/types.ts` is the one label table, shared by the scan log and the status tree; `CAPELLA_PROGRESS_STAGES` omits `export`, which runs no model and so never becomes a row. Scans predating the signal keep the aggregate `agentic-sast` span and render as a bare phase line - **Prompts** — Per-phase templates in `apps/worker/prompts/` with variable substitution (`{{TARGET_URL}}`, `{{CONFIG_CONTEXT}}`). Shared partials in `apps/worker/prompts/shared/` via `apps/worker/src/services/prompt-manager.ts`, including `_code-path-rules.txt` (focus/avoid `[FILE]`/`[GLOB]` routing) and `_rules-of-engagement.txt` (free-text engagement rules). When `exploit: false`, `apps/worker/src/services/findings-renderer.ts` deterministically converts each `*_exploitation_queue.json` into a `*_findings.md` for report assembly — no LLM in the loop -- **Agent Harness (pi)** — Uses the **pi harness** (`@earendil-works/pi-coding-agent`, requires Node ≥ 22.19) via `apps/worker/src/ai/pi/pi-executor.ts` (`runPiPrompt` → `createAgentSession`). Retry is split in `apps/worker/src/ai/pi/retry-settings.ts`: pi's agent-level loop is off so Temporal owns agent restarts, while `provider.maxRetries` stays on — pi reads the `provider` block independently of the `enabled` flag — so transport faults are absorbed in-session rather than costing a full agent re-run. `maxRetryDelayMs` is left at pi's 60s default. One model runs every phase, named by `SHANNON_AI_MODEL=:` (default `anthropic:claude-sonnet-4-6`). `apps/worker/src/ai/models.ts` parses the spec — splitting on the **first** colon only, so Bedrock IDs keep theirs — and resolves it through pi's `ModelRuntime`. pi ships the `CredentialStore` interface but no in-memory implementation (its own reads `auth.json` from disk), so `RuntimeCredentialStore` in that file supplies one: credentials arrive as env vars in an ephemeral container and must never touch disk. `createModelRuntime(providerId, apiKey)` builds the runtime; `allowModelNetwork` stays at its default `false` so a scan never blocks on a catalog refresh. `resolveModelSelection()` is **async** because `ModelRuntime.create()` is. Any pi-ai provider id is accepted — `parseModelSpec` no longer rejects against a hardcoded list, so pi's registry is the authority (an unknown provider/model surfaces as a clear "not found in pi registry" error at preflight, which points to the browsable catalogue at `pi.dev/models` — `PI_CATALOG_URL` in `apps/worker/src/ai/models.ts`, appended to the not-found errors and shown in the setup wizard's "Other provider" hint). Four providers are **curated** (`CURATED_PROVIDERS`: `anthropic`, `openai`, `xai`, `amazon-bedrock`) with their own credential variables, config sections, and setup flows; each provider's API key env var is declared once in `PROVIDER_API_KEY_ENV` — Shannon uses each vendor's own variable name (`OPENAI_API_KEY`, `XAI_API_KEY`, …), never an invented one; Bedrock's entry is `AWS_BEARER_TOKEN_BEDROCK`, paired with `AWS_REGION`, which preflight requires separately as provider config rather than a credential. Any other provider uses the **generic** credential path: `SHANNON_AI_API_KEY` (`GENERIC_API_KEY_ENV`) supplies the key for any provider whose credential is a plain API key. Curated providers' own variables take precedence over it, and it also works as a fallback for them — Bedrock is the sole exception (it authenticates through its AWS_ variables, so the generic key never stands in for it). The CLI forwards `SHANNON_AI_API_KEY` in `COMMON_FORWARD_VARS` (it is provider-neutral, binding to whatever `SHANNON_AI_MODEL` names, so the "only one provider configured" guard counts only named credentials), and stores it under a generic `[provider]` config.toml section (`provider.api_key`). `npx @keygraph/shannon setup` exposes this as the "Other provider" option: free-text provider id + model id + key (a curated provider id is rejected there, since it has its own option). A model too new for the pinned pi release does not require an SDK bump: `--models-config ` mounts a pi `models.json` read-only at `/app/models.json`. The mount is the entire CLI→worker protocol: nothing is forwarded through the environment, and `modelsConfigPath()` detects the file at that fixed path, exactly as `piAuthPresent()` detects the pi auth mount whose flag is likewise not forwarded (`MODELS_CONFIG_CONTAINER_PATH` in the CLI and `MODELS_CONFIG_PATH` in `apps/worker/src/paths.ts` must stay in sync). `createModelRuntime` always names `modelsPath` explicitly — the mounted path, or **`null` when no config was supplied**, which switches models.json off outright. It is never left to pi's default of `/models.json`, because that dir is shared with the pi auth mount, so a file landing there must not silently contribute model definitions to a scan that did not ask for one. `modelsStorePath` is pinned to the agent dir alongside it, since pi otherwise derives it from `dirname(modelsPath)` and would try to write beside a read-only mount. Custom definitions merge over the built-in catalogue: a matching model id replaces the built-in entry, a new id is added alongside, and `modelOverrides` adjusts a built-in without replacing the provider's list. Omitted fields take pi's defaults (`contextWindow` 128000, `maxTokens` 16384), so a large-context model needs them stated. Credentials are unaffected: `RuntimeCredentialStore` outranks any `apiKey` the file carries, so a snippet pasted from `pi.dev/models` can keep its placeholder key while the real secret stays in the environment — and pi's `!command` config-value form is never reached through `apiKey`. Preflight reports `modelRuntime.getError()` **before** the "model not found" check, because a file that fails to parse, fails schema validation, or composes badly otherwise leaves pi with an empty or fallback provider and surfaces as a model-id error that blames `SHANNON_AI_MODEL`. `SHANNON_AI_BASE_URL` overrides the endpoint for any provider (proxies/gateways); the credential and API dialect are unchanged. It is applied after model resolution, so it also wins over a `baseUrl` set in a model config. A base URL only changes the address — `resolveModel` (`apps/worker/src/ai/models.ts`) carries it onto the model descriptor and nothing else, and it grants **no exemption from registry validation**: the model id must resolve for every provider and endpoint alike. A gateway serving a model under its own name, like one newer than the pinned pi release, is described in a `--models-config` file, which puts a real descriptor in the registry rather than guessing one. pi's builtin `openai` provider serves a single API (Responses) and dispatches on the provider rather than `model.api`, so an `openai:` gateway run always speaks Responses; a Chat-Completions-only gateway must be reached through a completions-native provider such as `openrouter` instead. `buildEnvFlags` forwards only the selected provider's credential into the worker container. The CLI mirrors the parse rule and the provider/credential tables in `apps/cli/src/model-spec.ts` (it cannot import from the worker package); the two must stay in sync. pi ships no JSON-schema output or `Task`/`TodoWrite` built-ins, so structured queues are captured via a `submit_exploitation_queue` custom tool (`apps/worker/src/ai/queue-schemas.ts`), and `task` (child sessions scoped to `read`, `grep`, `find`, `ls`, `write`, and `bash` — no nested `task` or collector tools; `CHILD_TOOLS` in `apps/worker/src/ai/pi/task-tool.ts`) + `todo_write` (`apps/worker/src/ai/pi/session-tools.ts`) are provided as custom tools; the per-phase collectors are pi custom tools (TypeBox `defineTool` in `apps/worker/src/collectors/`). Shannon sets no thinking configuration at all — no `thinkingLevel` is passed to any `createAgentSession` call, so pi's own default applies. There is no adaptive-thinking support and no `CLAUDE_ADAPTIVE_THINKING` / `core.adaptive_thinking` setting. Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login +- **Agent Harness (pi)** — Uses the **pi harness** (`@earendil-works/pi-coding-agent`, requires Node ≥ 22.19) via `apps/worker/src/ai/pi/pi-executor.ts` (`runPiPrompt` → `createAgentSession`). Retry is split in `apps/worker/src/ai/pi/retry-settings.ts`: pi's agent-level loop is off so Temporal owns agent restarts, while `provider.maxRetries` stays on — pi reads the `provider` block independently of the `enabled` flag — so transport faults are absorbed in-session rather than costing a full agent re-run. `maxRetryDelayMs` is left at pi's 60s default. One model runs every phase, named by `SHANNON_AI_MODEL=:` (default `anthropic:claude-sonnet-4-6`). `apps/worker/src/ai/models.ts` parses the spec — splitting on the **first** colon only, so Bedrock IDs keep theirs — and resolves it through pi's `ModelRuntime`. pi ships the `CredentialStore` interface but no in-memory implementation (its own reads `auth.json` from disk), so `RuntimeCredentialStore` in that file supplies one: credentials arrive as env vars in an ephemeral container and must never touch disk. `createModelRuntime(providerId, apiKey)` builds the runtime with `allowModelNetwork: true`, so `ModelRuntime.create()` refreshes the model catalogue over the network at scan start and a freshly released model resolves without a `--models-config` file. The fetch is bounded (10s) and falls back to the static catalogue on timeout, so an unreachable catalogue endpoint cannot hang the scan. The refresh does not override a `--models-config`: pi reloads and re-applies that file as a config overlay on every refresh (it reloads `this.config` at the top of `refresh()`), so custom definitions still win over the fetched catalogue; the merge semantics below are unchanged, just layered over a fresher base. `resolveModelSelection()` is **async** because `ModelRuntime.create()` is. Any pi-ai provider id is accepted — `parseModelSpec` no longer rejects against a hardcoded list, so pi's registry is the authority (an unknown provider/model surfaces as a clear "not found in pi registry" error at preflight, which points to the browsable catalogue at `pi.dev/models` — `PI_CATALOG_URL` in `apps/worker/src/ai/models.ts`, appended to the not-found errors and shown in the setup wizard's "Other provider" hint). Four providers are **curated** (`CURATED_PROVIDERS`: `anthropic`, `openai`, `xai`, `amazon-bedrock`) with their own credential variables, config sections, and setup flows; each provider's API key env var is declared once in `PROVIDER_API_KEY_ENV` — Shannon uses each vendor's own variable name (`OPENAI_API_KEY`, `XAI_API_KEY`, …), never an invented one; Bedrock's entry is `AWS_BEARER_TOKEN_BEDROCK`, paired with `AWS_REGION`, which preflight requires separately as provider config rather than a credential. Any other provider uses the **generic** credential path: `SHANNON_AI_API_KEY` (`GENERIC_API_KEY_ENV`) supplies the key for any provider whose credential is a plain API key. Curated providers' own variables take precedence over it, and it also works as a fallback for them — Bedrock is the sole exception (it authenticates through its AWS_ variables, so the generic key never stands in for it). The CLI forwards `SHANNON_AI_API_KEY` in `COMMON_FORWARD_VARS` (it is provider-neutral, binding to whatever `SHANNON_AI_MODEL` names, so the "only one provider configured" guard counts only named credentials), and stores it under a generic `[provider]` config.toml section (`provider.api_key`). `npx @keygraph/shannon setup` exposes this as the "Other provider" option: free-text provider id + model id + key (a curated provider id is rejected there, since it has its own option). A model pi's catalogue does not carry, such as a self-hosted model, is reachable without an SDK bump: `--models-config ` mounts a pi `models.json` read-only at `/app/models.json`. The mount is the entire CLI→worker protocol: nothing is forwarded through the environment, and `modelsConfigPath()` detects the file at that fixed path, exactly as `piAuthPresent()` detects the pi auth mount whose flag is likewise not forwarded (`MODELS_CONFIG_CONTAINER_PATH` in the CLI and `MODELS_CONFIG_PATH` in `apps/worker/src/paths.ts` must stay in sync). `createModelRuntime` always names `modelsPath` explicitly — the mounted path, or **`null` when no config was supplied**, which switches models.json off outright. It is never left to pi's default of `/models.json`, because that dir is shared with the pi auth mount, so a file landing there must not silently contribute model definitions to a scan that did not ask for one. `modelsStorePath` is pinned to the agent dir alongside it, since pi otherwise derives it from `dirname(modelsPath)` and would try to write beside a read-only mount. Custom definitions merge over the built-in catalogue: a matching model id replaces the built-in entry, a new id is added alongside, and `modelOverrides` adjusts a built-in without replacing the provider's list. Omitted fields take pi's defaults (`contextWindow` 128000, `maxTokens` 16384), so a large-context model needs them stated. Credentials are unaffected: `RuntimeCredentialStore` outranks any `apiKey` the file carries, so a snippet pasted from `pi.dev/models` can keep its placeholder key while the real secret stays in the environment — and pi's `!command` config-value form is never reached through `apiKey`. Preflight reports `modelRuntime.getError()` **before** the "model not found" check, because a file that fails to parse, fails schema validation, or composes badly otherwise leaves pi with an empty or fallback provider and surfaces as a model-id error that blames `SHANNON_AI_MODEL`. `SHANNON_AI_BASE_URL` overrides the endpoint for any provider (proxies/gateways); the credential and API dialect are unchanged. It is applied after model resolution, so it also wins over a `baseUrl` set in a model config. A base URL only changes the address — `resolveModel` (`apps/worker/src/ai/models.ts`) carries it onto the model descriptor and nothing else, and it grants **no exemption from registry validation**: the model id must resolve for every provider and endpoint alike. A gateway serving a model under its own name is described in a `--models-config` file, which puts a real descriptor in the registry rather than guessing one. pi's builtin `openai` provider serves a single API (Responses) and dispatches on the provider rather than `model.api`, so an `openai:` gateway run always speaks Responses; a Chat-Completions-only gateway must be reached through a completions-native provider such as `openrouter` instead. `buildEnvFlags` forwards only the selected provider's credential into the worker container. The CLI mirrors the parse rule and the provider/credential tables in `apps/cli/src/model-spec.ts` (it cannot import from the worker package); the two must stay in sync. pi ships no JSON-schema output or `Task`/`TodoWrite` built-ins, so structured queues are captured via a `submit_exploitation_queue` custom tool (`apps/worker/src/ai/queue-schemas.ts`), and `task` (child sessions scoped to `read`, `grep`, `find`, `ls`, `write`, and `bash` — no nested `task` or collector tools; `CHILD_TOOLS` in `apps/worker/src/ai/pi/task-tool.ts`) + `todo_write` (`apps/worker/src/ai/pi/session-tools.ts`) are provided as custom tools; the per-phase collectors are pi custom tools (TypeBox `defineTool` in `apps/worker/src/collectors/`). Shannon sets no thinking configuration at all — no `thinkingLevel` is passed to any `createAgentSession` call, so pi's own default applies. There is no adaptive-thinking support and no `CLAUDE_ADAPTIVE_THINKING` / `core.adaptive_thinking` setting. Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login - **Pi Credential Reuse** — `SHANNON_USE_PI_AUTH=1` opts into reusing the host's Pi login, including an `openai-codex` ChatGPT Plus/Pro subscription (`SHANNON_AI_MODEL=openai-codex:`) or an `xai` Grok subscription (`SHANNON_AI_MODEL=xai:`); the mechanism is provider-agnostic and works for any Pi login. `apps/cli/src/env.ts` requires `~/.pi/agent/auth.json`; `start.ts` passes its path to `spawnWorker`, which mounts only that file read-write at `/tmp/.pi/agent/auth.json`. The flag itself is not forwarded: the worker detects the file with `piAuthPresent()` and passes its path to `ModelRuntime.create`. CLI and worker API-key presence checks are skipped on this path, but the normal preflight model probe still validates the credential. The image and UID-remapping entrypoint keep `/tmp/.pi/agent` owned by `pentest` so adjacent Pi/Shannon configuration remains writable. Refreshed OAuth state is persisted to the host for subsequent scans. - **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds the human-facing report in both formats (`Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`, `FINAL_REPORT_PDF_FILENAME`/`FINAL_REPORT_MD_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`scans`/`logs`) without migration. A pre-restructure workspace cannot be resumed: `classifyWorkspaceLaunch` (`apps/cli/src/commands/start.ts`) requires `.shannon/launch.json`, and its absence fails the launch as "created by an earlier version of Shannon" before anything on disk is touched. There is no in-place migration — the workspace's files and report are left untouched, and the operator starts a new scan under a different `-w` name. The report agent writes structured findings to `report.json`, from which `report-renderer.ts` renders the assembled markdown and `report-json-adapter.ts` produces the Typst-shaped JSON that `pdf-renderer.ts` compiles into `comprehensive_security_assessment_report.pdf` using the bundled `apps/worker/templates/typst/report.typ` template (the `typst` binary is installed in the worker image). `copyReportToRunRoot` (`apps/worker/src/services/reporting.ts`) surfaces both the PDF and the markdown to the run root as `Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`; the deliverables-dir copies remain as the git-checkpointed sources. PDF compilation is best-effort — a failure is logged and the run still completes. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive. Every combined-log line is also projected into a per-agent file under `.shannon/agents/.log` (one per pipeline agent, one per Capella stage; subagents fold into the parent's file, and a stage's concurrent sessions share its file with an inline session label). The projection boundary is `apps/worker/src/audit/actor-projection.ts` (`projectActor` maps a `TraceActor` to its combined prefix and owning file slug — slugs come only from closed fields); fan-out is best-effort and never blocks the canonical combined log. A lifecycle owner holds a `LogStream` lease per agent file (the pipeline agent's `logAgent` span, or a Capella stage activity's `try/finally`) so per-line writes ride the reference count; `CapellaStageTrace.drain()` flushes a stage's trace queue before its activity returns. The CLI tails one file with `shannon logs --agent ` (`--list-agents` to enumerate); the default `shannon logs` path is unchanged - **Deliverables** — Saved to `.shannon/deliverables/` in the target repo via the `save-deliverable` CLI script (`apps/worker/src/scripts/save-deliverable.ts`) diff --git a/README.md b/README.md index bdac34c0..57b66ab2 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,7 @@ These reports are from Shannon Open Source scans of Photoview 2.4.0, one of the - **Docker**: required for the worker container. - **Node.js 18+**: required for the recommended `npx` workflow. -- **AI provider credentials**: Shannon runs on Anthropic, OpenAI, xAI, AWS Bedrock, and [any other provider](docs/ai-providers.md#any-other-provider) in the harness catalogue — each of which you can point at a proxy or LLM gateway through a [custom base URL](docs/ai-providers.md#custom-base-url), and a model the catalogue does not yet carry can be described with a [custom model configuration](docs/ai-providers.md#custom-model-configuration). You bring your own key, and Keygraph never proxies your model traffic. Shannon is provider-agnostic. See [AI providers](docs/ai-providers.md#suggested-models) for suggested model IDs. +- **AI provider credentials**: Shannon runs on Anthropic, OpenAI, xAI, AWS Bedrock, and [any other provider](docs/ai-providers.md#any-other-provider) in the harness catalogue — each of which you can point at a proxy or LLM gateway through a [custom base URL](docs/ai-providers.md#custom-base-url), and a model the catalogue does not carry can be described with a [custom model configuration](docs/ai-providers.md#custom-model-configuration). You bring your own key, and Keygraph never proxies your model traffic. Shannon is provider-agnostic. See [AI providers](docs/ai-providers.md#suggested-models) for suggested model IDs. - **Cyber safeguards cleared with your provider**: Anthropic and OpenAI apply real-time safeguards to cyber-security workloads, which can interrupt a scan mid-run. Complete their guidance for legitimate security testers before your first run - see [AI providers](docs/ai-providers.md#cyber-safeguards-do-this-before-your-first-scan). @@ -375,7 +375,7 @@ Yes. Shannon emits SARIF 2.1.0, the OASIS standard format for static analysis re ### Which AI providers does Shannon support? -Anthropic, OpenAI, xAI, and AWS Bedrock are built in and configured directly by provider ID. Beyond those, Shannon runs on any provider in the Pi harness catalogue, named the same `:` way. Any provider can be pointed at a proxy or LLM gateway through a custom base URL, which overrides only the endpoint and keeps that provider's API dialect. A model the catalogue does not yet carry, such as one released after Shannon's pinned harness version, runs without waiting for a Shannon release. Describe it in a [custom model configuration](docs/ai-providers.md#custom-model-configuration) file and pass it with `--models-config`. Shannon uses a single unified model setting throughout a pentest. +Anthropic, OpenAI, xAI, and AWS Bedrock are built in and configured directly by provider ID. Beyond those, Shannon runs on any provider in the Pi harness catalogue, named the same `:` way. Any provider can be pointed at a proxy or LLM gateway through a custom base URL, which overrides only the endpoint and keeps that provider's API dialect. A model the catalogue does not carry, such as one a router or gateway serves under its own ID, or a self-hosted model, is described in a [custom model configuration](docs/ai-providers.md#custom-model-configuration) file and passed with `--models-config`. Shannon uses a single unified model setting throughout a pentest. ### Can I run Shannon on a local or self-hosted model? diff --git a/apps/worker/src/ai/models.ts b/apps/worker/src/ai/models.ts index 8c0ff2f7..96408b82 100644 --- a/apps/worker/src/ai/models.ts +++ b/apps/worker/src/ai/models.ts @@ -20,7 +20,9 @@ * Resolution returns a pi `Model` plus the `ModelRuntime` that owns its auth, * built over an in-memory credential store primed from the environment. * - * A model too new for the pinned pi release is reachable by passing its descriptor in a + * The catalogue is refreshed over the network at scan start, so a newly released model + * on a catalogue provider resolves on its own. A model the catalogue does not carry, such + * as a router model under its own id, or a self-hosted server, is described in a * pi `models.json` (the CLI's `--models-config`), which merges over the catalogue. The * credential store below outranks any `apiKey` that file carries, so it describes the * model while the environment still supplies the secret. @@ -216,13 +218,18 @@ function modelsStorePath(): string { } /** - * Build a ModelRuntime whose only credential is the one supplied. Model catalogs - * stay offline (`allowModelNetwork` defaults to false) so a scan never blocks on - * a catalog refresh. + * Build a ModelRuntime whose only credential is the one supplied. `allowModelNetwork` + * refreshes the model catalogue over the network at scan start, so the registry reflects + * models the pinned pi build predates. The fetch is bounded and falls back to the static + * catalogue on timeout, so an unreachable endpoint cannot hang the scan. A mounted + * `--models-config` overlays the catalogue and is reloaded on every refresh, so its + * definitions take precedence. * * `modelsPath` is always explicit, never pi's default of `/models.json`: with no * `--models-config` it is null, which switches models.json off outright, so a stray file in * that shared dir cannot feed model definitions to a run that did not ask for them. + * `modelsStorePath` is pinned to the writable agent dir, replacing pi's default + * `dirname(modelsPath)` (a read-only mount) as the fetched catalogue's store. * * When the host's pi auth.json is present, the runtime reads it instead: pi's * disk-backed store resolves the credential. The mount is writable so OAuth @@ -233,6 +240,8 @@ export async function createModelRuntime(providerId: string, apiKey: string | un const modelSources = { modelsPath: modelsPath ?? null, ...(modelsPath ? { modelsStorePath: modelsStorePath() } : {}), + allowModelNetwork: true, + modelRefreshTimeoutMs: 10_000, }; if (piAuthPresent()) { @@ -254,9 +263,8 @@ export interface ModelSelection { * * The model must exist in the runtime's registry, whether or not an endpoint override * is in play — a base URL changes the address and nothing else. A gateway serving a - * model under its own name, or one newer than the pinned pi release, is described in a - * `--models-config` file, which puts a real descriptor in the registry rather than - * guessing one from an unrelated model. + * model under its own name is described in a `--models-config` file, which puts a real + * descriptor in the registry rather than guessing one from an unrelated model. */ export function resolveModel( modelRuntime: ModelRuntime, diff --git a/apps/worker/src/services/preflight.ts b/apps/worker/src/services/preflight.ts index 2375f5be..f5ac376a 100644 --- a/apps/worker/src/services/preflight.ts +++ b/apps/worker/src/services/preflight.ts @@ -374,7 +374,7 @@ async function validateCredentials(logger: ActivityLogger): Promise:` way. Any provider can be pointed at a proxy or LLM gateway through a custom base URL, which overrides only the endpoint and keeps that provider's API dialect. A model the catalogue does not yet carry, such as one released after Shannon's pinned harness version, runs without waiting for a Shannon release. Describe it in a [custom model configuration](docs/ai-providers.md#custom-model-configuration) file and pass it with `--models-config`. Shannon uses a single unified model setting throughout a pentest. +Anthropic, OpenAI, xAI, and AWS Bedrock are built in and configured directly by provider ID. Beyond those, Shannon runs on any provider in the Pi harness catalogue, named the same `:` way. Any provider can be pointed at a proxy or LLM gateway through a custom base URL, which overrides only the endpoint and keeps that provider's API dialect. A model the catalogue does not carry, such as one a router or gateway serves under its own ID, or a self-hosted model, is described in a [custom model configuration](docs/ai-providers.md#custom-model-configuration) file and passed with `--models-config`. Shannon uses a single unified model setting throughout a pentest. ### Can I run Shannon on a local or self-hosted model? @@ -792,7 +792,7 @@ export SHANNON_AI_BASE_URL=https://llm-gateway.example.com # optional: route thr This path covers providers whose credential is a single API key. Providers that need more than that are not currently supported. -A model the catalogue does not yet carry, such as one released after Shannon's pinned Pi version, is reachable by describing it yourself. See [Custom model configuration](#custom-model-configuration). +A model the catalogue does not carry is reachable by describing it yourself. See [Custom model configuration](#custom-model-configuration). `npx @keygraph/shannon setup` exposes this as the **Other provider** option. @@ -895,7 +895,7 @@ export SHANNON_AI_BASE_URL=https://llm-gateway.example.com/v1 ## Custom model configuration -A model released after Shannon's pinned Pi version is not in the harness catalogue yet, so `SHANNON_AI_MODEL` alone cannot reach it. Rather than wait for a Shannon release, describe the model yourself and pass the file with `--models-config`: +A custom model configuration is a Pi `models.json` file that describes a model the harness catalogue does not carry: one a router or gateway serves under its own ID, or a local server (see [Local and self-hosted models](#local-and-self-hosted-models)). You pass it with `--models-config`, and Shannon merges its definitions over the catalogue so `SHANNON_AI_MODEL` can then name the model like any other: ```bash npx @keygraph/shannon start -u https://example.com -r /path/to/repo --models-config ./models.json