diff --git a/CLAUDE.md b/CLAUDE.md index ce627364..d2df8c25 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -151,8 +151,15 @@ Durable workflow orchestration with crash recovery, queryable progress, intellig 4. **Exploitation** (5 parallel agents, conditional) — Exploits confirmed vulnerabilities 5. **Reporting** (`report`) — Executive-level security report +Around those phases: + +- Optional agentic static analysis runs before the pentest when `agentic_sast.enabled` is `"true"`, as a child workflow. +- After each class's analysis, reconciliation groups its findings into exploitation tasks. +- Findings outside the five classes form an internal `other` class with its own exploitation agent (`other-exploit`). +- A scan can finish `completed`, `partial`, `failed`, or `cancelled`; `partial` carries an ordered set of reasons. + ### Supporting Systems -- **Configuration** — YAML configs in `apps/worker/configs/` with JSON Schema validation (`config-schema.json`). Supports auth settings (MFA/TOTP), URL/code rule scoping (`rules.avoid`/`rules.focus`), run-scope steering (`vuln_classes`, `exploit`), free-form `rules_of_engagement`, and post-hoc `report` options (`min_severity`, `min_confidence`, `guidance`, and `sarif` for a SARIF 2.1.0 log via `apps/worker/src/services/sarif-renderer.ts`, on by default for exploit runs and opt out with `report.sarif: false`). `code_path` avoid rules are enforced via the `@gotgenes/pi-permission-system` extension: `apps/worker/src/temporal/activities.ts:syncCodePathDenyRules` writes a global `path` deny config once per workflow (`apps/worker/src/ai/pi/permission-system.ts:syncPermissionSystemConfig`), and the executor loads the extension when that config is present (`apps/worker/src/ai/pi/pi-executor.ts`), so denies fire across every tool and child `task` session. `vuln_classes`/`exploit` scope is locked into `session.json` on first run; resumes with a different scope fail fast (`persistOrValidateRunScope`). Credential resolution — local mode: env vars → `./.env`; npx mode: env vars → `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) +- **Configuration** — YAML configs in `apps/worker/configs/` use the closed JSON Schema in `config-schema.json`. Every fresh scan runs the fixed five analysis classes; there is no public class selector. `agentic_sast.enabled` is the only public agentic-SAST setting. Finding reconciliation runs on every scan and has no public setting of its own. Config also supports authentication (MFA/TOTP), URL/code rule scoping (`rules.avoid`/`rules.focus`), `exploit`, free-form `rules_of_engagement`, and post-hoc `report` options (`min_severity`, `min_confidence`, `guidance`, and exploit-only `sarif` output via `apps/worker/src/services/sarif-renderer.ts`). `code_path` avoid rules are enforced via the `@gotgenes/pi-permission-system` extension: `apps/worker/src/temporal/activities.ts:syncCodePathDenyRules` writes a global `path` deny config once per workflow (`apps/worker/src/ai/pi/permission-system.ts:syncPermissionSystemConfig`), and the executor loads the extension when that config is present (`apps/worker/src/ai/pi/pi-executor.ts`), so denies fire across every tool and child `task` session. Credential resolution — local mode: env vars → `./.env`; npx mode: env vars → `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) - **Prompts** — Per-phase templates in `apps/worker/prompts/` with variable substitution (`{{TARGET_URL}}`, `{{CONFIG_CONTEXT}}`). Shared partials in `apps/worker/prompts/shared/` via `apps/worker/src/services/prompt-manager.ts`, including `_code-path-rules.txt` (focus/avoid `[FILE]`/`[GLOB]` routing) and `_rules-of-engagement.txt` (free-text engagement rules). When `exploit: false`, `apps/worker/src/services/findings-renderer.ts` deterministically converts each `*_exploitation_queue.json` into a `*_findings.md` for report assembly — no LLM in the loop - **Agent Harness (pi)** — Uses the **pi harness** (`@earendil-works/pi-coding-agent`, requires Node ≥ 22.19) via `apps/worker/src/ai/pi/pi-executor.ts` (`runPiPrompt` → `createAgentSession`). Retry is split in `apps/worker/src/ai/pi/retry-settings.ts`: pi's agent-level loop is off so Temporal owns agent restarts, while `provider.maxRetries` stays on — pi reads the `provider` block independently of the `enabled` flag — so transport faults are absorbed in-session rather than costing a full agent re-run. `maxRetryDelayMs` is left at pi's 60s default. One model runs every phase, named by `SHANNON_AI_MODEL=:` (default `anthropic:claude-sonnet-4-6`). `apps/worker/src/ai/models.ts` parses the spec — splitting on the **first** colon only, so Bedrock IDs keep theirs — and resolves it through pi's `ModelRuntime`. pi ships the `CredentialStore` interface but no in-memory implementation (its own reads `auth.json` from disk), so `RuntimeCredentialStore` in that file supplies one: credentials arrive as env vars in an ephemeral container and must never touch disk. `createModelRuntime(providerId, apiKey)` builds the runtime; `allowModelNetwork` stays at its default `false` so a scan never blocks on a catalog refresh. `resolveModelSelection()` is **async** because `ModelRuntime.create()` is. Any pi-ai provider id is accepted — `parseModelSpec` no longer rejects against a hardcoded list, so pi's registry is the authority (an unknown provider/model surfaces as a clear "not found in pi registry" error at preflight, which points to the browsable catalogue at `pi.dev/models` — `PI_CATALOG_URL` in `apps/worker/src/ai/models.ts`, appended to the not-found errors and shown in the setup wizard's "Other provider" hint). Four providers are **curated** (`CURATED_PROVIDERS`: `anthropic`, `openai`, `xai`, `amazon-bedrock`) with their own credential variables, config sections, and setup flows; each provider's API key env var is declared once in `PROVIDER_API_KEY_ENV` — Shannon uses each vendor's own variable name (`OPENAI_API_KEY`, `XAI_API_KEY`, …), never an invented one; Bedrock's entry is `AWS_BEARER_TOKEN_BEDROCK`, paired with `AWS_REGION`, which preflight requires separately as provider config rather than a credential. Any other provider uses the **generic** credential path: `SHANNON_AI_API_KEY` (`GENERIC_API_KEY_ENV`) supplies the key for any provider whose credential is a plain API key. Curated providers' own variables take precedence over it, and it also works as a fallback for them — Bedrock is the sole exception (it authenticates through its AWS_ variables, so the generic key never stands in for it). The CLI forwards `SHANNON_AI_API_KEY` in `COMMON_FORWARD_VARS` (it is provider-neutral, binding to whatever `SHANNON_AI_MODEL` names, so the "only one provider configured" guard counts only named credentials), and stores it under a generic `[provider]` config.toml section (`provider.api_key`). `npx @keygraph/shannon setup` exposes this as the "Other provider" option: free-text provider id + model id + key (a curated provider id is rejected there, since it has its own option). `SHANNON_AI_BASE_URL` overrides the endpoint for any provider (proxies/gateways); the credential is unchanged. `pointAtGateway` (`apps/worker/src/ai/models.ts`) applies the one dialect change: behind a base URL, `openai` follows `SHANNON_AI_OPENAI_FORMAT` (`chat-completions` default, or `responses`). On `chat-completions` it switches the API to `openai-completions` and drops the catalogue's Responses-shaped `compat` block so pi's `detectCompat` derives completions settings; on `responses` the descriptor is unchanged but for the endpoint. `resolveGatewayFormat` rejects the variable when the provider is not `openai` or no base URL is set, since it cannot take effect there. All other providers keep their API. The CLI mirrors the accepted values in `apps/cli/src/model-spec.ts`, forwards the variable in `COMMON_FORWARD_VARS`, and maps it to `openai.format` in config.toml. `buildEnvFlags` forwards only the selected provider's credential into the worker container. The CLI mirrors the parse rule and the provider/credential tables in `apps/cli/src/model-spec.ts` (it cannot import from the worker package); the two must stay in sync. pi ships no JSON-schema output or `Task`/`TodoWrite` built-ins, so structured queues are captured via a `submit_exploitation_queue` custom tool (`apps/worker/src/ai/queue-schemas.ts`), and `task` (child sessions scoped to `read`, `grep`, `find`, `ls`, `write`, and `bash` — no nested `task` or collector tools; `CHILD_TOOLS` in `apps/worker/src/ai/pi/task-tool.ts`) + `todo_write` (`apps/worker/src/ai/pi/session-tools.ts`) are provided as custom tools; the per-phase collectors are pi custom tools (TypeBox `defineTool` in `apps/worker/src/collectors/`). Shannon sets no thinking configuration at all — no `thinkingLevel` is passed to any `createAgentSession` call, so pi's own default applies. There is no adaptive-thinking support and no `CLAUDE_ADAPTIVE_THINKING` / `core.adaptive_thinking` setting. Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login - **Pi Credential Reuse** — `SHANNON_USE_PI_AUTH=1` opts into reusing the host's Pi login, including an `openai-codex` ChatGPT Plus/Pro subscription selected with `SHANNON_AI_MODEL=openai-codex:`. `apps/cli/src/env.ts` requires `~/.pi/agent/auth.json`; `start.ts` passes its path to `spawnWorker`, which mounts only that file read-write at `/tmp/.pi/agent/auth.json`. The flag itself is not forwarded: the worker detects the file with `piAuthPresent()` and passes its path to `ModelRuntime.create`. CLI and worker API-key presence checks are skipped on this path, but the normal preflight model probe still validates the credential. The image and UID-remapping entrypoint keep `/tmp/.pi/agent` owned by `pentest` so adjacent Pi/Shannon configuration remains writable. Refreshed OAuth state is persisted to the host for subsequent scans. diff --git a/apps/cli/src/commands/start.ts b/apps/cli/src/commands/start.ts index 9f114fc0..d94e7382 100644 --- a/apps/cli/src/commands/start.ts +++ b/apps/cli/src/commands/start.ts @@ -18,6 +18,7 @@ import { commandPrefix, isLocal } from '../mode.js'; import { resolveModelSpec } from '../model-spec.js'; import { expandHome, + FINAL_REPORT_MD_FILENAME, FINAL_REPORT_PDF_FILENAME, INTERNAL_DIR, resolveConfig, @@ -43,81 +44,216 @@ export interface StartArgs { version: string; } +const LAUNCH_STATE_SCHEMA_VERSION = 1 as const; +const LAUNCH_STATE_FILENAME = 'launch.json'; +const FIXED_CLASSES = ['injection', 'xss', 'auth', 'authz', 'ssrf'] as const; + /** - * Upgrade a pre-restructure workspace (flat layout, no INTERNAL_DIR) before it is mounted, - * so resume finds the old deliverables and their git checkpoints instead of re-running every - * agent. For a legacy run every top-level entry is internal, so move them all into INTERNAL_DIR - * (a same-filesystem rename carries the deliverables .git along). + * CLI-owned launch record at INTERNAL_DIR/launch.json, written once when a workspace is + * created and never rewritten. It pins the customer output destination so a resume with a + * different -o cannot silently redirect the final report. The worker does not read it. */ -function migrateLegacyWorkspaceLayout(workspacePath: string): void { - const legacySessionJson = path.join(workspacePath, 'session.json'); - const internalPath = path.join(workspacePath, INTERNAL_DIR); - if (!fs.existsSync(legacySessionJson) || fs.existsSync(internalPath)) { - return; +interface LaunchState { + readonly schema_version: typeof LAUNCH_STATE_SCHEMA_VERSION; + readonly customer_output_path?: string; +} + +export interface WorkspaceLaunchDecision { + readonly isResume: boolean; + readonly outputDir?: string; +} + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === 'object' && !Array.isArray(value); +} + +function arraysEqual(left: readonly unknown[], right: readonly unknown[]): boolean { + return left.length === right.length && left.every((value, index) => value === right[index]); +} + +/** + * Hand-rolled twin of the worker's durable-state validator in + * apps/worker/src/types/run-state.ts, which owns the session.json.durableScanState shape. + * Each array check accepts two variants because the worker appends 'other' and + * 'other-exploit' only after the other pipeline admits findings. If the worker's shape + * changes and this twin lags, resume fails fast as incompatible instead of launching a + * worker against state it would misread. + */ +function isCurrentDurableState(value: unknown): boolean { + if (!isRecord(value) || value.schema_version !== 1 || typeof value.exploit !== 'boolean') return false; + if (!Array.isArray(value.participating_classes) || !Array.isArray(value.expected_agents)) return false; + + const participating = value.participating_classes; + const validParticipation = + arraysEqual(participating, FIXED_CLASSES) || arraysEqual(participating, [...FIXED_CLASSES, 'other']); + if (!validParticipation) return false; + + const baselineAgents = ['pre-recon', 'recon', ...FIXED_CLASSES.map((name) => `${name}-vuln`)]; + if (value.exploit) baselineAgents.push(...FIXED_CLASSES.map((name) => `${name}-exploit`)); + baselineAgents.push('report'); + const expected = value.expected_agents; + return arraysEqual(expected, baselineAgents) || arraysEqual(expected, [...baselineAgents, 'other-exploit']); +} + +/** One refusal for damaged CLI-owned or worker-owned workspace records, whichever reads first. */ +const DAMAGED_RECORDS_MESSAGE = + "This workspace's internal records are damaged and it cannot be resumed. Its report files are untouched. Start a new scan with a different -w name."; + +const NEWER_RELEASE_MESSAGE = + 'This workspace was created by a newer version of Shannon. Upgrade Shannon, or start a new scan with a different -w name.'; + +function readJsonFile(filePath: string): unknown { + try { + return JSON.parse(fs.readFileSync(filePath, 'utf8')); + } catch { + fail(DAMAGED_RECORDS_MESSAGE); + } +} + +function readLaunchState(filePath: string): LaunchState { + if (!fs.existsSync(filePath)) { + fail( + 'This workspace was created by an earlier version of Shannon and cannot be resumed. Its files and report are untouched. Start a new scan with a different -w name.', + ); + } + const value = readJsonFile(filePath); + if (!isRecord(value)) fail(NEWER_RELEASE_MESSAGE); + // Unknown keys mean a newer release wrote this workspace; refuse rather than half-read it. + const keys = Object.keys(value).sort(); + const keysAreValid = keys.every((key) => key === 'customer_output_path' || key === 'schema_version'); + const customerPath = value.customer_output_path; + const pathIsValid = + customerPath === undefined || + (typeof customerPath === 'string' && path.isAbsolute(customerPath) && path.resolve(customerPath) === customerPath); + if (value.schema_version !== LAUNCH_STATE_SCHEMA_VERSION || !keysAreValid || !pathIsValid) { + fail(NEWER_RELEASE_MESSAGE); + } + return { + schema_version: LAUNCH_STATE_SCHEMA_VERSION, + ...(typeof customerPath === 'string' && { customer_output_path: customerPath }), + }; +} + +/** + * Decide fresh-versus-resume from on-disk state alone, before start() mutates anything. + * A fresh launch requires the workspace directory to be absent or empty; a resume requires + * current-release session state, a matching target URL, and a customer output path that + * agrees with the recorded one. Every other combination fails the launch, so a typo in + * -w or -o stops here instead of spawning a worker into the wrong workspace. + */ +export function classifyWorkspaceLaunch( + workspacePath: string, + expectedUrl: string, + requestedOutputDir: string | undefined, +): WorkspaceLaunchDecision { + const sessionPath = resolveRunFile(workspacePath, 'session.json'); + const sessionExists = fs.existsSync(sessionPath); + if (!sessionExists) { + if (fs.existsSync(workspacePath) && fs.readdirSync(workspacePath).length > 0) { + fail( + 'This directory is not a Shannon workspace, or its scan state is missing. Start a new scan with a different -w name.', + ); + } + return { isResume: false, ...(requestedOutputDir !== undefined && { outputDir: requestedOutputDir }) }; } - fs.mkdirSync(internalPath, { recursive: true }); - for (const entry of fs.readdirSync(workspacePath)) { - if (entry === INTERNAL_DIR) { - continue; + const launchPath = path.join(workspacePath, INTERNAL_DIR, LAUNCH_STATE_FILENAME); + const launch = readLaunchState(launchPath); + const session = readJsonFile(sessionPath); + if (!isRecord(session) || !isRecord(session.session) || session.session.webUrl !== expectedUrl) { + fail( + 'This workspace was created for a different target URL, so it cannot be resumed against this one. Check -u, or start a new scan with a different -w name.', + ); + } + if (!isCurrentDurableState(session.durableScanState)) { + fail( + "This workspace's scan state cannot be read by this version. Its files are untouched. Start a new scan with a different -w name.", + ); + } + + const storedOutputDir = launch.customer_output_path; + if (requestedOutputDir !== undefined && requestedOutputDir !== storedOutputDir) { + fail( + 'This workspace already copies its report to a different location than the -o path you passed. Re-run without -o to keep the original location, or start a new scan with a different -w name.', + ); + } + return { isResume: true, ...(storedOutputDir !== undefined && { outputDir: storedOutputDir }) }; +} + +/** + * Crash-safe single write: exclusive temp file (pid plus random suffix keeps concurrent + * starts apart), fsync, rename into place, then directory fsync so the entry survives a + * host crash. Callers invoke this only for a fresh workspace; an existing launch.json is + * the resume contract and must never be replaced. + */ +export function writeLaunchStateAtomically(internalPath: string, outputDir: string | undefined): void { + const finalPath = path.join(internalPath, LAUNCH_STATE_FILENAME); + const temporaryPath = path.join(internalPath, `${LAUNCH_STATE_FILENAME}.tmp-${process.pid}-${randomSuffix()}`); + const launchState: LaunchState = { + schema_version: LAUNCH_STATE_SCHEMA_VERSION, + ...(outputDir !== undefined && { customer_output_path: outputDir }), + }; + const descriptor = fs.openSync(temporaryPath, 'wx', 0o600); + try { + fs.writeFileSync(descriptor, `${JSON.stringify(launchState, null, 2)}\n`, 'utf8'); + fs.fsyncSync(descriptor); + } finally { + fs.closeSync(descriptor); + } + try { + fs.renameSync(temporaryPath, finalPath); + const directory = fs.openSync(internalPath, 'r'); + try { + fs.fsyncSync(directory); + } finally { + fs.closeSync(directory); } - fs.renameSync(path.join(workspacePath, entry), path.join(internalPath, entry)); + } catch (error) { + fs.rmSync(temporaryPath, { force: true }); + throw error; } - console.log(`Migrated workspace to ${INTERNAL_DIR}/ layout: ${workspacePath}`); } export async function start(args: StartArgs): Promise { - // 1. Initialize state directories and load env + // 1. Resolve non-mutating inputs and classify the workspace before changing it. initHome(); loadEnv(); - - // 2. Validate credentials const creds = validateCredentials(); if (!creds.valid) { fail(creds.error ?? 'Invalid credentials'); } - - // 3. Resolve paths const repo = resolveRepo(args.repo); const config = args.config ? resolveConfig(args.config) : undefined; + const workspacesDir = getWorkspacesDir(); + const workspace = + args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`; + const workspacePath = path.join(workspacesDir, workspace); + const requestedOutputDir = args.output ? path.resolve(expandHome(args.output)) : undefined; + const launchDecision = classifyWorkspaceLaunch(workspacePath, args.url, requestedOutputDir); - // Inputs are valid — show the splash before the Docker/Temporal setup work. - // Skip it off a real terminal (e.g. CI) so piped/logged output stays clean. + // 2. Inputs are valid; initialize shared infrastructure. if (stdoutIsTerminal()) { displaySplash(isLocal() ? undefined : args.version); } - - // 4. Ensure workspaces dir is writable by container user (UID 1001) - const workspacesDir = getWorkspacesDir(); fs.mkdirSync(workspacesDir, { recursive: true }); fs.chmodSync(workspacesDir, 0o777); - - // 5. Ensure Docker and the worker image are available (pull/build prints its own progress). ensureDocker(); ensureImage(args.version); - - // One spinner spans the whole launch: bringing up Temporal and registering the worker. const spinner = p.spinner(); spinner.start('Starting scan'); await ensureInfra(spinner); - // 6. Generate unique task queue and container name + // 3. Generate the invocation identity. const suffix = randomSuffix(); const taskQueue = `shannon-${suffix}`; const containerName = `shannon-worker-${suffix}`; - // 7. Generate workspace name if not provided - const workspace = - args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`; - - // 8. Create writable overlay directories (mounted over :ro repo paths inside container) + // 4. Create writable overlay directories after resume validation has succeeded. // The run dir and its INTERNAL_DIR must be 0o777 so the container user can create audit // subdirs and the overlay backing dirs. - const workspacePath = path.join(workspacesDir, workspace); const internalPath = path.join(workspacePath, INTERNAL_DIR); fs.mkdirSync(workspacePath, { recursive: true }); fs.chmodSync(workspacePath, 0o777); - migrateLegacyWorkspaceLayout(workspacePath); fs.mkdirSync(internalPath, { recursive: true }); fs.chmodSync(internalPath, 0o777); for (const dir of ['deliverables', 'scratchpad', '.playwright-cli', '.playwright']) { @@ -125,24 +261,37 @@ export async function start(args: StartArgs): Promise { fs.mkdirSync(dirPath, { recursive: true }); fs.chmodSync(dirPath, 0o777); } + if (!launchDecision.isResume) { + writeLaunchStateAtomically(internalPath, launchDecision.outputDir); + } - // 9. Pre-create overlay mount points (:ro mounts can't auto-create them) + // 5. Pre-create overlay mount points (:ro mounts cannot create them). const shannonDir = path.join(repo.hostPath, '.shannon'); for (const dir of ['deliverables', 'scratchpad', '.playwright-cli']) { fs.mkdirSync(path.join(shannonDir, dir), { recursive: true }); } fs.mkdirSync(path.join(repo.hostPath, '.playwright'), { recursive: true }); - // 10. Resolve output directory - const outputDir = args.output ? path.resolve(expandHome(args.output)) : undefined; + // 6. Create the validated customer-copy destination, if configured. + const outputDir = launchDecision.outputDir; if (outputDir) { fs.mkdirSync(outputDir, { recursive: true }); } - // 11. Resolve prompts directory (local mode only) + // 7. Resolve prompts and capture the pre-launch resume counter. const promptsDir = isLocal() ? path.resolve('apps/worker/prompts') : undefined; + const sessionJson = resolveRunFile(workspacePath, 'session.json'); + const isResume = launchDecision.isResume; + let initialResumeCount = 0; + if (isResume) { + // Docker and Temporal startup sit between this read and the classification that validated the + // same file, so a file that changed in between is a workspace-state failure, not a CLI bug. + const session = readJsonFile(sessionJson); + const attempts = isRecord(session) && isRecord(session.session) ? session.session.resumeAttempts : undefined; + initialResumeCount = Array.isArray(attempts) ? attempts.length : 0; + } - // 12. Spawn worker container + // 8. Spawn the worker container. const proc = spawnWorker({ version: args.version, url: args.url, @@ -171,24 +320,16 @@ export async function start(args: StartArgs): Promise { process.exit(1); } - // Detect whether this is a fresh workspace or a resume by checking session.json existence - const sessionJson = resolveRunFile(path.join(workspacesDir, workspace), 'session.json'); - const isResume = fs.existsSync(sessionJson); - let initialResumeCount = 0; - if (isResume) { - try { - const session = JSON.parse(fs.readFileSync(sessionJson, 'utf-8')); - initialResumeCount = session.session?.resumeAttempts?.length ?? 0; - } catch { - // Corrupted file — worker will handle validation - } - } - let started = false; + // Set when the startup poll times out but session.json already holds durable state this + // release understands: the workflow is executing, so the exit handler must not stop its + // worker. An operator abort is a different intent and still stops it. + let scanRunningUnconfirmed = false; + // Stop the worker only if the scan hasn't registered yet (e.g. Ctrl-C mid-startup). let cleaned = false; - const cleanup = (): void => { + const stopWorker = (): void => { if (cleaned || started) return; cleaned = true; spinner.stop('Stopping scan'); @@ -202,14 +343,17 @@ export async function start(args: StartArgs): Promise { } }; process.on('SIGINT', () => { - cleanup(); + stopWorker(); process.exit(0); }); process.on('SIGTERM', () => { - cleanup(); + stopWorker(); process.exit(0); }); - process.on('exit', cleanup); + process.on('exit', () => { + if (scanRunningUnconfirmed) return; + stopWorker(); + }); // Poll for the workflow to register in session.json; the spinner resolves once it does. spinner.message('Waiting for the scan to start'); @@ -236,10 +380,52 @@ export async function start(args: StartArgs): Promise { await sleep(2000); } + if (classifyStartupTimeout(sessionJson) === 'scan-running') { + scanRunningUnconfirmed = true; + spinner.error('The scan started, but this CLI could not confirm it'); + printUnconfirmedScanHint(workspace, taskQueue, containerName); + process.exit(1); + } + spinner.error('Timed out waiting for the scan to start'); process.exit(1); } +/** + * Read the startup timeout: 'scan-running' when session.json already holds durable state this + * release understands, which only the worker writes and only after Temporal began executing the + * workflow; 'unregistered' when nothing proves the scan started. The distinction decides whether + * timing out may stop the worker container. + */ +export function classifyStartupTimeout(sessionJsonPath: string): 'unregistered' | 'scan-running' { + let session: unknown; + try { + session = JSON.parse(fs.readFileSync(sessionJsonPath, 'utf-8')); + } catch { + return 'unregistered'; + } + if (!isRecord(session) || !isCurrentDurableState(session.durableScanState)) { + return 'unregistered'; + } + return 'scan-running'; +} + +/** Point the operator at a scan that is running but whose startup this CLI could not confirm. */ +function printUnconfirmedScanHint(workspace: string, taskQueue: string, containerName: string): void { + console.log(''); + console.log(' The scan is running and was left alone; only its startup confirmation is missing.'); + console.log(''); + console.log(` Workspace: ${workspace}`); + console.log(` Task queue: ${taskQueue}`); + console.log(` Container: ${containerName}`); + console.log(''); + console.log(' Inspect it:'); + console.log(` Live logs: ${commandPrefix()} logs ${workspace}`); + console.log(` Worker logs: docker logs ${containerName}`); + console.log(' Dashboard: http://localhost:8233'); + console.log(''); +} + /** * Follow a just-started scan (for `--follow`, aimed at CI): stream its log while Temporal drives * completion, then exit on the workflow outcome — 0 if the assessment ran, 1 if the scan failed. @@ -329,7 +515,7 @@ function printInfo(args: StartArgs, workspace: string, repoPath: string, workspa return; } - const reportPath = path.join(workspacesDir, workspace, FINAL_REPORT_PDF_FILENAME); + const reportDir = path.join(workspacesDir, workspace); // When following, the scan log streams inline next, so the "run these to watch it" hints // would only contradict that. @@ -343,6 +529,8 @@ function printInfo(args: StartArgs, workspace: string, repoPath: string, workspa console.log(''); console.log(' Report (when the scan finishes):'); - console.log(` ${reportPath}`); + console.log(` ${reportDir}${path.sep}`); + console.log(` ${FINAL_REPORT_PDF_FILENAME}`); + console.log(` ${FINAL_REPORT_MD_FILENAME}`); console.log(''); } diff --git a/apps/cli/src/commands/status.ts b/apps/cli/src/commands/status.ts index 5ef3a040..ab3765a1 100644 --- a/apps/cli/src/commands/status.ts +++ b/apps/cli/src/commands/status.ts @@ -15,7 +15,13 @@ import { type RenderInput, renderScan } from '../scan/render.js'; import { toStatusJson } from '../scan/status-json.js'; import { resolveWorkflowId } from '../session.js'; import { displaySplash } from '../splash.js'; -import { describeScan, getTerminalOutcome, queryProgress, type ScanDescription } from '../temporal-client.js'; +import { + ActivityMirrorError, + describeScan, + getTerminalOutcome, + queryProgress, + type ScanDescription, +} from '../temporal-client.js'; import { stdoutIsTerminal, supportsColor } from '../tty.js'; import { getVersion } from '../version.js'; @@ -30,6 +36,24 @@ function isTerminalStatus(status: string): boolean { return status !== 'RUNNING' && status !== 'UNSPECIFIED'; } +/** + * Read one scan description, telling the two failure modes apart. A stale activity mirror + * carries its own message and needs a CLI update; anything else is a read that did not reach + * a usable answer, which is most often Temporal being down. + */ +async function readScanDescription(workflowId: string): Promise { + try { + return await describeScan(workflowId); + } catch (error) { + if (error instanceof ActivityMirrorError) fail(error.message); + fail( + "Could not read this scan's progress.", + 'If Temporal is not running, start a scan to bring it up. If it is running, this build of the CLI', + 'does not recognise part of the scan and needs updating.', + ); + } +} + // Match SGR color escapes (ESC[…m) so a line's on-screen width excludes them. Built from the ESC // char code so the source carries no literal control character. const ANSI_PATTERN = new RegExp(`${String.fromCharCode(27)}\\[[0-9;]*m`, 'g'); @@ -128,7 +152,7 @@ async function watch(workspace: string, workflowId: string): Promise { }, RENDER_MS); for (;;) { - const desc = await describeScan(workflowId); + const desc = await readScanDescription(workflowId); if (!desc) { clearInterval(ticker); fail(`Scan "${workspace}" is no longer in Temporal.`); @@ -158,12 +182,7 @@ export async function status(workspace: string, opts: { readonly json: boolean } // follows the current resume, not the superseded original. Fresh scans: the name is the id. const workflowId = resolveWorkflowId(workspace) ?? workspace; - let desc: ScanDescription | null; - try { - desc = await describeScan(workflowId); - } catch { - fail('Could not reach Temporal at 127.0.0.1:7233.', 'Start Temporal (it comes up with a scan) and try again.'); - } + const desc = await readScanDescription(workflowId); if (!desc) { fail( diff --git a/apps/cli/src/docker.ts b/apps/cli/src/docker.ts index 421bca00..8f317633 100644 --- a/apps/cli/src/docker.ts +++ b/apps/cli/src/docker.ts @@ -345,7 +345,7 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess { args.push('-v', `${opts.config.hostPath}:${opts.config.containerPath}:ro`); } - // Output directory for deliverables copy + // Customer-copy destination. The workflow surfaces only final report artifacts here. if (opts.outputDir) { args.push('-v', `${opts.outputDir}:/app/output`); } diff --git a/apps/cli/src/paths.ts b/apps/cli/src/paths.ts index a1314fc4..ff3bc014 100644 --- a/apps/cli/src/paths.ts +++ b/apps/cli/src/paths.ts @@ -42,6 +42,12 @@ export const INTERNAL_DIR = '.shannon'; */ export const FINAL_REPORT_PDF_FILENAME = 'Security-Assessment-Report.pdf'; +/** + * Customer-facing Markdown report name at the run root. + * Must match FINAL_REPORT_MD_FILENAME in the worker package. + */ +export const FINAL_REPORT_MD_FILENAME = 'Security-Assessment-Report.md'; + /** * Resolve a run-directory file (e.g. session.json, workflow.log), preferring the * current INTERNAL_DIR location and falling back to the legacy run-root location diff --git a/apps/cli/src/scan/derive.ts b/apps/cli/src/scan/derive.ts index c3de3f2e..3c20453d 100644 --- a/apps/cli/src/scan/derive.ts +++ b/apps/cli/src/scan/derive.ts @@ -8,7 +8,13 @@ */ import type { RunningAgent } from '../temporal-client.js'; -import { agentClass, PIPELINE, type PipelineState } from './pipeline.js'; +import { + agentClass, + type OperationalStageState, + operationFamilyKey, + type PipelineState, + pipelineForState, +} from './pipeline.js'; import type { RenderInput } from './render.js'; export type RunState = 'pending' | 'running' | 'completed' | 'failed' | 'skipped'; @@ -22,6 +28,8 @@ export interface DerivedAgent { readonly durationMs: number | null; readonly runningElapsedMs: number | null; readonly attempt: number | null; + /** The step a running operation row is currently on, merged in from its child activity. */ + readonly detail?: string; readonly error?: string; } @@ -48,12 +56,12 @@ function isAgentActive(name: string, state: PipelineState | null, running: Set, resolved: boolean): RunState { if (running.has(name)) return 'running'; @@ -99,16 +107,17 @@ export function phaseGlyphState(states: readonly RunState[]): RunState { * class had anything to exploit), not still pending. */ export function deriveAgentStates(input: RenderInput): Map { - const runningSet = new Set(input.running.map((r) => r.agent)); + const pipeline = pipelineForState(input.state); + const runningSet = new Set(input.running.filter((runner) => runner.kind === 'agent').map((runner) => runner.agent)); const terminal = isTerminal(input.temporalStatus); let frontier = -1; - PIPELINE.forEach((phase, idx) => { + pipeline.forEach((phase, idx) => { if (phase.agents.some((a) => isAgentActive(a.name, input.state, runningSet))) frontier = idx; }); const states = new Map(); - for (const [phaseIdx, phase] of PIPELINE.entries()) { + for (const [phaseIdx, phase] of pipeline.entries()) { const resolved = terminal || phaseIdx < frontier; for (const agent of phase.agents) { states.set(agent.name, agentState(agent.name, input.state, runningSet, resolved)); @@ -117,6 +126,52 @@ export function deriveAgentStates(input: RenderInput): Map { return states; } +/** Which operation families have a running parent stage, and the step to show on it. */ +interface OperationFamilyView { + /** Families whose parent stage row already represents their child activities. */ + readonly runningFamilies: ReadonlySet; + /** Family to current step, present only where the child activities agree on one. */ + readonly stepByFamily: ReadonlyMap; +} + +/** + * Resolve the parent stage rows that own their family's child activities. A family only + * resolves to a step when its running children agree: several classes reconcile at once and + * their pending activities carry no class, so a family caught mid-stride shows its parent + * rows without a step rather than attributing one to the wrong class. + */ +function operationFamilyView( + running: readonly RunningAgent[], + persistedOperations: readonly OperationalStageState[], +): OperationFamilyView { + const runningFamilies = new Set( + persistedOperations + .filter((operation) => operation.status === 'running') + .map((operation) => operationFamilyKey(operation.key)), + ); + + const labelsByFamily = new Map>(); + for (const runner of running) { + if (runner.kind !== 'operation' || runner.parentKey === undefined) continue; + if (!runningFamilies.has(runner.parentKey)) continue; + const labels = labelsByFamily.get(runner.parentKey) ?? new Set(); + labels.add(runner.label); + labelsByFamily.set(runner.parentKey, labels); + } + + const stepByFamily = new Map(); + for (const [family, labels] of labelsByFamily) { + const [onlyLabel] = labels; + if (labels.size === 1 && onlyLabel !== undefined) stepByFamily.set(family, lowercaseFirst(onlyLabel)); + } + return { runningFamilies, stepByFamily }; +} + +/** Progress labels are written to start a row; as a detail they continue a sentence. */ +function lowercaseFirst(label: string): string { + return label.charAt(0).toLowerCase() + label.slice(1); +} + /** * Full structured view of the pipeline: every agent's state plus the raw * metrics/timing needed to present it, and each phase's collapsed state. @@ -124,8 +179,9 @@ export function deriveAgentStates(input: RenderInput): Map { export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] { const states = deriveAgentStates(input); const byAgent = new Map(input.running.map((r) => [r.agent, r])); + const pipeline = pipelineForState(input.state); - return PIPELINE.map((phase) => { + const agentPhases = pipeline.map((phase) => { const agents = phase.agents.map((a): DerivedAgent => { const state = states.get(a.name) ?? 'pending'; const metrics = input.state?.agentMetrics[a.name]; @@ -150,6 +206,57 @@ export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] agents, }; }); + + // Operational rows merge two sources: stages the worker has persisted (durable truth, + // including terminal outcomes) and pending activities whose stage record has not landed + // yet. Persisted keys win, so a stage is never listed twice while the two views overlap. + const persistedOperations = Object.values(input.state?.operationalStages ?? {}); + const persistedKeys = new Set(persistedOperations.map((operation) => operation.key)); + const { runningFamilies, stepByFamily } = operationFamilyView(input.running, persistedOperations); + const unpersistedRunning = input.running + .filter((runner) => runner.kind === 'operation' && !persistedKeys.has(runner.agent)) + // A child activity whose family already has a running parent stage is that stage's current + // step, not separate work: the parent row below represents it, with the step as its detail + // where the family's children agree on one. Without such a parent it keeps its own row. + .filter((runner) => runner.parentKey === undefined || !runningFamilies.has(runner.parentKey)) + .map((runner) => ({ + key: runner.agent, + label: runner.label, + status: 'running' as const, + ...(runner.startedAt !== undefined && { startedAt: runner.startedAt }), + ...(runner.lastFailure !== undefined && { error: runner.lastFailure }), + })); + const operationalAgents: DerivedAgent[] = [...persistedOperations, ...unpersistedRunning].map((operation) => { + const runner = byAgent.get(operation.key); + const operationState = operation.status as RunState; + const persistedDurationMs = 'durationMs' in operation ? (operation.durationMs ?? null) : null; + const detail = operationState === 'running' ? stepByFamily.get(operationFamilyKey(operation.key)) : undefined; + return { + name: operation.key, + label: operation.label, + state: operationState, + durationMs: operationState === 'completed' ? persistedDurationMs : null, + runningElapsedMs: + operationState === 'running' && operation.startedAt !== undefined ? now - operation.startedAt : null, + attempt: operationState === 'running' ? (runner?.attempt ?? null) : null, + ...(detail !== undefined && { detail }), + ...(operation.error !== undefined && { error: operation.error }), + }; + }); + + // The synthetic phase appears only when there is operational work to show, so a scan + // with no recorded operational stages keeps the plain agent tree. + if (operationalAgents.length === 0) return agentPhases; + return [ + ...agentPhases, + { + key: 'operational-work', + label: 'Background work', + parallel: true, + state: phaseGlyphState(operationalAgents.map((operation) => operation.state)), + agents: operationalAgents, + }, + ]; } export { agentError }; diff --git a/apps/cli/src/scan/pipeline.ts b/apps/cli/src/scan/pipeline.ts index fe877ede..dd09d639 100644 --- a/apps/cli/src/scan/pipeline.ts +++ b/apps/cli/src/scan/pipeline.ts @@ -8,6 +8,7 @@ * - apps/worker/src/temporal/activities.ts (the run*Agent activity names → `activityType`) * - apps/worker/src/temporal/shared.ts (PipelineState / PipelineSummary) * - apps/worker/src/types/metrics.ts (AgentMetrics) + * - apps/worker/src/types/run-state.ts (PartialReasonView) */ export interface AgentSpec { @@ -26,6 +27,18 @@ export interface PhaseSpec { readonly agents: readonly AgentSpec[]; } +export interface ActivityProgressSpec { + readonly key: string; + readonly label: string; + readonly kind: 'agent' | 'operation'; + /** + * Operation rows whose work is already represented by a persisted parent stage. The parent + * owns the row; this activity supplies the step shown as its detail. Parent stage keys are + * the family key itself or the family key followed by ':' and a class or stage suffix. + */ + readonly parentKey?: string; +} + /** The pipeline phases in execution order, each with its agents. */ export const PIPELINE: readonly PhaseSpec[] = [ { @@ -80,9 +93,175 @@ export const PIPELINE: readonly PhaseSpec[] = [ }, ]; -/** Temporal activity type name → canonical agent name, for mapping pendingActivities. */ +const OTHER_EXPLOIT_AGENT: AgentSpec = { + name: 'other-exploit', + label: 'other', + activityType: 'runOtherExploitAgent', +}; + +/** + * Shape the static PIPELINE to one scan's durable truth. expectedAgents, persisted by the + * worker at scan start, names every exploit agent the scan can ever run: exploit rows it + * excludes are dropped, 'other-exploit' is appended only once the other pipeline has + * admitted findings, and a phase left with no agents disappears entirely. Without state + * (the scan has not initialized durable state yet) the full static pipeline is the best + * available guess. + */ +export function pipelineForState(state: PipelineState | null): readonly PhaseSpec[] { + if (state?.expectedAgents === undefined) return PIPELINE; + const expected = new Set(state.expectedAgents); + return PIPELINE.map((phase) => { + if (phase.key !== 'exploitation') return phase; + const agents = phase.agents.filter((agent) => expected.has(agent.name)); + if (expected.has(OTHER_EXPLOIT_AGENT.name)) agents.push(OTHER_EXPLOIT_AGENT); + return { ...phase, agents }; + }).filter((phase) => phase.agents.length > 0); +} + +const AGENT_ACTIVITY_PROGRESS: Readonly> = Object.fromEntries( + [...PIPELINE.flatMap((phase) => phase.agents), OTHER_EXPLOIT_AGENT].map((agent) => [ + agent.activityType, + { key: agent.name, label: agent.label, kind: 'agent' }, + ]), +); + +/** Families whose per-class or per-stage work is already carried by one persisted stage row. */ +const RECONCILIATION_PARENT_KEY = 'reconciliation'; +const AGENTIC_SAST_PARENT_KEY = 'agentic-sast'; + +// Every production activity that is not an agent run must have a row here. describeScan +// throws on an unmapped activity type, so adding a worker activity without updating this +// table breaks `shannon status` loudly instead of hiding the new work. The authoritative +// name lists live in apps/worker/src/temporal/worker.ts, +// apps/worker/src/temporal/reconcile-activity-types.ts, and +// apps/worker/src/ai/sast/capella/temporal/activity-types.ts. +const OPERATION_ACTIVITY_PROGRESS: Readonly> = { + runPreflightValidation: { key: 'preflight', label: 'Preflight validation', kind: 'operation' }, + syncPlaywrightStealthConfig: { key: 'preflight', label: 'Browser setup', kind: 'operation' }, + initDeliverableGit: { key: 'scan-initialization', label: 'Initialize deliverables', kind: 'operation' }, + syncCodePathDenyRules: { key: 'scan-initialization', label: 'Apply source rules', kind: 'operation' }, + initializeDurableScanState: { key: 'durable-state', label: 'Saving scan state', kind: 'operation' }, + persistOtherOutcome: { key: 'other-pipeline', label: 'Including other findings', kind: 'operation' }, + initializeReportProgress: { key: 'report:initialize', label: 'Initialize report state', kind: 'operation' }, + renumberClassFindings: { key: 'report:renumber', label: 'Renumber findings', kind: 'operation' }, + assembleReportActivity: { key: 'report:assemble', label: 'Assemble report inputs', kind: 'operation' }, + compactReportFindings: { key: 'report:compact', label: 'Compact report findings', kind: 'operation' }, + persistCanonicalReportProgress: { key: 'report:checkpoint', label: 'Saving report progress', kind: 'operation' }, + finalizeReportOutputs: { key: 'report:finalize', label: 'Finalize report outputs', kind: 'operation' }, + persistFinalizedReportProgress: { key: 'report:terminal', label: 'Saving final report state', kind: 'operation' }, + surfaceReportOutputs: { key: 'report:surface', label: 'Surface customer report', kind: 'operation' }, + checkExploitationQueue: { key: 'queue-check', label: 'Check exploitation queue', kind: 'operation' }, + loadResumeState: { key: 'resume-validation', label: 'Validate resume state', kind: 'operation' }, + restoreGitCheckpoint: { key: 'resume-restore', label: 'Restore checkpoint', kind: 'operation' }, + registerResumeAttempt: { key: 'resume-registration', label: 'Register resume', kind: 'operation' }, + recordResumeAttempt: { key: 'resume-registration', label: 'Record resume', kind: 'operation' }, + logPhaseTransition: { key: 'audit-log', label: 'Update audit log', kind: 'operation' }, + logWorkflowComplete: { key: 'audit-log', label: 'Finalize audit log', kind: 'operation' }, + saveCheckpoint: { key: 'checkpoint', label: 'Save checkpoint', kind: 'operation' }, + seedEmptyProducerQueue: { key: 'other-pipeline', label: 'Preparing other findings', kind: 'operation' }, + prepareClassReconciliation: { + key: 'reconciliation', + label: 'Preparing findings', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + enrichClassSastObservations: { + key: 'reconciliation', + label: 'Adding code context', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + formClassExploitTasks: { + key: 'reconciliation', + label: 'Grouping into test cases', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + materializeClassExploitTasks: { + key: 'reconciliation', + label: 'Writing test cases', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + publishClassReconciliationOss: { + key: 'reconciliation', + label: 'Saving results', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + capellaArchitecture: { + key: 'agentic-sast:architecture', + label: 'Mapping architecture', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaThreatModel: { + key: 'agentic-sast:threat-model', + label: 'Modelling threats', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaPlan: { + key: 'agentic-sast:plan', + label: 'Planning the review', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaResearch: { + key: 'agentic-sast:research', + label: 'Researching code', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaDedupe: { + key: 'agentic-sast:dedupe', + label: 'Merging duplicates', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaReview: { + key: 'agentic-sast:review', + label: 'Reviewing findings', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaCritic: { + key: 'agentic-sast:critic', + label: 'Critiquing findings', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaConfirm: { + key: 'agentic-sast:confirm', + label: 'Confirming findings', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaCalibrate: { + key: 'agentic-sast:calibrate', + label: 'Calibrating risk', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaExport: { + key: 'agentic-sast:export', + label: 'Exporting findings', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, +}; + +/** Complete production activity mirror. Unknown names are errors, never hidden progress. */ +export const ACTIVITY_TO_PROGRESS: Readonly> = Object.freeze({ + ...AGENT_ACTIVITY_PROGRESS, + ...OPERATION_ACTIVITY_PROGRESS, +}); + +/** Agent-only projection of ACTIVITY_TO_PROGRESS: activity type name to canonical agent name. */ export const ACTIVITY_TO_AGENT: Readonly> = Object.fromEntries( - PIPELINE.flatMap((phase) => phase.agents.map((agent) => [agent.activityType, agent.name])), + Object.entries(ACTIVITY_TO_PROGRESS) + .filter(([, progress]) => progress.kind === 'agent') + .map(([activityType, progress]) => [activityType, progress.key]), ); /** The vuln/exploit class of an agent (e.g. "authz-vuln" → "authz"), for failedPipelines matching. */ @@ -100,11 +279,36 @@ export interface AgentMetrics { readonly skipped?: boolean; } +export interface OperationalStageState { + readonly key: string; + readonly label: string; + readonly status: 'pending' | 'running' | 'completed' | 'failed' | 'skipped'; + readonly startedAt?: number; + readonly durationMs?: number; + readonly error?: string; +} + +/** Family key a persisted operational stage belongs to, e.g. `reconciliation:xss` to `reconciliation`. */ +export function operationFamilyKey(stageKey: string): string { + const separator = stageKey.indexOf(':'); + return separator === -1 ? stageKey : stageKey.slice(0, separator); +} + export interface PipelineSummary { readonly totalCostUsd: number; readonly totalDurationMs: number; // Wall-clock (end - start) readonly totalTurns: number; readonly agentCount: number; + /** False when operational (Capella/reconciliation) spend is known to be incomplete. */ + readonly usageAccountingComplete?: boolean; +} + +/** One durable degradation reason with its derived safe message (mirror of PartialReasonView). */ +export interface PartialReasonView { + readonly code: string; + readonly vulnerabilityClass?: string; + readonly stage?: string; + readonly message: string; } export type PipelineStatus = 'running' | 'completed' | 'failed' | 'cancelled' | 'partial'; @@ -114,10 +318,27 @@ export interface PipelineState { readonly currentPhase: string | null; readonly currentAgent: string | null; readonly completedAgents: string[]; + readonly expectedAgents?: string[]; + readonly participatingClasses?: string[]; readonly failedPipelines: { vulnType: string; error: string }[]; + readonly failedReconciliations?: { vulnerabilityClass: string; error: string }[]; readonly failedAgent: string | null; readonly error: string | null; readonly startTime: number; readonly agentMetrics: Record; + readonly operationalMetrics?: Record; + readonly operationalStages?: Record; + /** `error` is the worker's sanitized failure sentence, safe to print verbatim. */ + readonly agenticSast?: { + readonly status: string; + readonly durationMs?: number; + /** Reader-facing name of the failed stage, already projected by the worker. */ + readonly failedStageLabel?: string; + readonly error?: string; + readonly errorCode?: string; + }; + readonly nonFatalFailures?: { readonly phase: string; readonly error: string }[]; + /** Ordered durable degradation reasons with safe messages; empty or absent for full success. */ + readonly partialReasons?: readonly PartialReasonView[]; readonly summary: PipelineSummary | null; } diff --git a/apps/cli/src/scan/render.ts b/apps/cli/src/scan/render.ts index 7e3ac62e..aad2b960 100644 --- a/apps/cli/src/scan/render.ts +++ b/apps/cli/src/scan/render.ts @@ -10,9 +10,8 @@ import { BOLD, DIM, GOLD, paint, RED, YELLOW } from '../colors.js'; import { commandPrefix } from '../mode.js'; import type { RunningAgent } from '../temporal-client.js'; -import { agentError, deriveAgentStates, isTerminal, phaseGlyphState, type RunState, scanElapsedMs } from './derive.js'; -import { inlineFailureReason } from './failure.js'; -import { PIPELINE, type PipelineState } from './pipeline.js'; +import { derivePipeline, isTerminal, type RunState, scanElapsedMs } from './derive.js'; +import type { PipelineState } from './pipeline.js'; export interface RenderInput { readonly workspace: string; @@ -95,6 +94,12 @@ const STATE_COLOR: Record = { skipped: COLORS.dim, }; +/** Column width for an agent or background-work label inside a phase. */ +const AGENT_LABEL_WIDTH = 18; + +/** Inline budget for a failure sentence, wide enough to carry a whole first sentence. */ +const FAILURE_DETAIL_WIDTH = 120; + /** Braille spinner frames for running agents — the clack loader style. */ const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'] as const; @@ -112,13 +117,14 @@ function statusBadge(input: RenderInput, opts: RenderOptions): string { const workflowStatus = input.state?.status; if (!isTerminal(input.temporalStatus)) return paint('running', COLORS.gold, opts.color); if (workflowStatus === 'partial') return paint('partial', COLORS.yellow, opts.color); + if (workflowStatus === 'cancelled') return paint('cancelled', COLORS.yellow, opts.color); if (input.temporalStatus === 'COMPLETED') return paint('completed', COLORS.gold, opts.color); if (input.temporalStatus === 'TERMINATED') return paint('stopped', COLORS.yellow, opts.color); if (input.temporalStatus === 'CANCELLED' || input.temporalStatus === 'CANCELED') { return paint('cancelled', COLORS.yellow, opts.color); } if (input.temporalStatus === 'TIMED_OUT') return paint('timed out', COLORS.red, opts.color); - return paint('FAILED', COLORS.red, opts.color); + return paint('failed', COLORS.red, opts.color); } // === Line builders === @@ -129,6 +135,7 @@ function agentMeta( runner: RunningAgent | undefined, error: string | undefined, opts: RenderOptions, + step?: string, ): string { if (state === 'completed') { const duration = metrics?.durationMs != null ? formatDuration(metrics.durationMs) : 'done'; @@ -136,12 +143,13 @@ function agentMeta( } if (state === 'running') { const parts = ['running']; + if (step !== undefined) parts.push(step); if (runner?.startedAt !== undefined) parts.push(formatDuration(opts.now - runner.startedAt)); if (runner && runner.attempt > 1) parts.push(`retry ${runner.attempt}`); return paint(parts.join(' · '), COLORS.gold, opts.color); } if (state === 'failed') { - const detail = error ? ` · ${truncate(error, 46)}` : ''; + const detail = error ? ` · ${truncate(error, FAILURE_DETAIL_WIDTH)}` : ''; return paint(`failed${detail}`, COLORS.red, opts.color); } if (state === 'skipped') return paint('skipped', COLORS.dim, opts.color); @@ -163,18 +171,20 @@ function phaseMeta(states: readonly RunState[], inPlay: number, parallel: boolea /** Render the full progress frame as one string (no trailing newline). */ export function renderScan(input: RenderInput, opts: RenderOptions): string { const byAgent = new Map(input.running.map((r) => [r.agent, r])); - const stateMap = deriveAgentStates(input); + const phases = derivePipeline(input, opts.now); const lines: string[] = ['', ...headerLines(input, opts), '']; - const metaFor = (name: string, state: RunState): string => - agentMeta(state, input.state?.agentMetrics[name], byAgent.get(name), agentError(name, input.state, byAgent), opts); // Only agents that have actually entered play are shown; pending/skipped ones stay hidden. const inPlay = (s: RunState): boolean => s === 'running' || s === 'completed' || s === 'failed'; - for (const phase of PIPELINE) { - const states = phase.agents.map((a) => stateMap.get(a.name) ?? 'pending'); + for (const phase of phases) { + const states = phase.agents.map((agent) => agent.state); const playing = states.filter(inPlay).length; - const phaseRunState: RunState = phaseGlyphState(states); + const phaseRunState = phase.state; + const metaFor = (agent: (typeof phase.agents)[number]): string => { + const metrics = agent.durationMs === null ? undefined : { durationMs: agent.durationMs }; + return agentMeta(agent.state, metrics, byAgent.get(agent.name), agent.error, opts, agent.detail); + }; // A single-agent phase carries that agent's own duration/cost on the phase line once it // starts; a parallel phase gets a "k/N done" summary over the agents in play. @@ -182,7 +192,7 @@ export function renderScan(input: RenderInput, opts: RenderOptions): string { const firstState = states[0]; const phaseMetaStr = !phase.parallel && first && firstState && inPlay(firstState) - ? metaFor(first.name, firstState) + ? metaFor(first) : phaseMeta(states, playing, phase.parallel, opts); lines.push(` ${glyph(phaseRunState, opts)} ${phase.label.padEnd(26)}${phaseMetaStr}`); @@ -191,7 +201,9 @@ export function renderScan(input: RenderInput, opts: RenderOptions): string { const agent = phase.agents[i]; const state = states[i]; if (!agent || !state || !inPlay(state)) continue; - lines.push(` ${glyph(state, opts)} ${agent.label.padEnd(18)}${metaFor(agent.name, state)}`); + // Two trailing spaces before padding, so a label wider than the column still separates + // from its meta text; a label inside the column pads to the same width as before. + lines.push(` ${glyph(state, opts)} ${`${agent.label} `.padEnd(AGENT_LABEL_WIDTH)}${metaFor(agent)}`); } } @@ -223,15 +235,44 @@ function footerLines(input: RenderInput, opts: RenderOptions): string[] { if (isTerminal(input.temporalStatus) && input.state?.summary) { const wall = formatDuration(input.state.summary.totalDurationMs); - return ['', ` Time Taken ${wall}`]; + const lines = ['', ` Time Taken ${wall}`]; + + // A partial scan names each durable degradation reason through its safe message, + // so the operator never has to guess why the badge is not "completed". + const reasons = input.state.partialReasons ?? []; + if (reasons.length > 0) { + lines.push('', ` ${paint('Why this scan is partial:', COLORS.yellow, opts.color)}`); + for (const reason of reasons) { + lines.push(paint(` - ${reason.message}`, COLORS.dim, opts.color)); + } + // The safe message names what degraded; these three name the agentic-SAST failure + // behind it, under the same labels the scan log and worker output use. + const agenticSast = input.state.agenticSast; + if (agenticSast?.status === 'failed') { + if (agenticSast.failedStageLabel !== undefined) { + lines.push(paint(` Agentic SAST stopped at: ${agenticSast.failedStageLabel}`, COLORS.dim, opts.color)); + } + if (agenticSast.error !== undefined) { + lines.push(paint(` What happened: ${agenticSast.error}`, COLORS.dim, opts.color)); + } + if (agenticSast.errorCode !== undefined) { + lines.push(paint(` Reference code (for a bug report): ${agenticSast.errorCode}`, COLORS.dim, opts.color)); + } + } + } + if (input.state.summary.usageAccountingComplete === false) { + lines.push( + paint(' Cost is incomplete — some background work is not included in this total.', COLORS.dim, opts.color), + ); + } + return lines; } const logsValue = `${prefix} logs ${input.workspace}`; const temporalValue = temporalDashboardUrl(input.workflowId); if (isTerminal(input.temporalStatus)) { - const rawReason = input.failureMessage ?? input.state?.error; - const reason = rawReason ? inlineFailureReason(rawReason) : 'no result recorded'; + const reason = input.failureMessage ?? input.state?.error ?? 'no result recorded'; return [ footerDivider(opts), paint( diff --git a/apps/cli/src/scan/status-json.ts b/apps/cli/src/scan/status-json.ts index 4229758a..dd26a1b3 100644 --- a/apps/cli/src/scan/status-json.ts +++ b/apps/cli/src/scan/status-json.ts @@ -8,6 +8,7 @@ import type { DerivedPhase } from './derive.js'; import { derivePipeline, isTerminal, scanElapsedMs } from './derive.js'; +import type { PartialReasonView } from './pipeline.js'; import type { RenderInput } from './render.js'; /** Coarse scan status token, mirroring the human status badge in machine-friendly form. */ @@ -27,6 +28,12 @@ export interface StatusJson { readonly endedAt?: string; /** Failure text when a failed scan left no readable state. */ readonly failureMessage?: string; + /** Ordered durable degradation reasons with safe messages; present only when non-empty. */ + readonly partialReasons?: readonly PartialReasonView[]; + /** Agentic SAST outcome, with the worker's sanitized failure sentence and bounded code. */ + readonly agenticSast?: { readonly status: string; readonly error?: string; readonly errorCode?: string }; + /** False when operational (Capella/reconciliation) spend is known to be incomplete. */ + readonly usageAccountingComplete?: boolean; readonly phases: readonly DerivedPhase[]; } @@ -34,6 +41,7 @@ export interface StatusJson { function deriveStatus(input: RenderInput): ScanStatus { if (!isTerminal(input.temporalStatus)) return 'running'; if (input.state?.status === 'partial') return 'partial'; + if (input.state?.status === 'cancelled') return 'cancelled'; switch (input.temporalStatus) { case 'COMPLETED': @@ -53,6 +61,9 @@ function deriveStatus(input: RenderInput): ScanStatus { /** Build the JSON snapshot for a scan at instant `now`. */ export function toStatusJson(input: RenderInput, now: number): StatusJson { const elapsedMs = scanElapsedMs(input, now); + const partialReasons = input.state?.partialReasons ?? []; + const agenticSast = input.state?.agenticSast; + const usageAccountingComplete = input.state?.summary?.usageAccountingComplete; return { workspace: input.workspace, @@ -63,6 +74,17 @@ export function toStatusJson(input: RenderInput, now: number): StatusJson { ...(input.startedAt !== undefined && { startedAt: new Date(input.startedAt).toISOString() }), ...(input.endedAt !== undefined && { endedAt: new Date(input.endedAt).toISOString() }), ...(input.failureMessage !== undefined && { failureMessage: input.failureMessage }), + ...(partialReasons.length > 0 && { partialReasons }), + // Present only when agentic SAST actually ran; a disabled scan omits the key entirely. + ...(agenticSast !== undefined && + agenticSast.status !== 'disabled' && { + agenticSast: { + status: agenticSast.status, + ...(agenticSast.error !== undefined && { error: agenticSast.error }), + ...(agenticSast.errorCode !== undefined && { errorCode: agenticSast.errorCode }), + }, + }), + ...(usageAccountingComplete !== undefined && { usageAccountingComplete }), phases: derivePipeline(input, now), }; } diff --git a/apps/cli/src/temporal-client.ts b/apps/cli/src/temporal-client.ts index 5dd4e8d7..3d493062 100644 --- a/apps/cli/src/temporal-client.ts +++ b/apps/cli/src/temporal-client.ts @@ -9,7 +9,7 @@ import { setTimeout as sleep } from 'node:timers/promises'; import { Client, Connection, WorkflowFailedError, WorkflowNotFoundError } from '@temporalio/client'; -import { ACTIVITY_TO_AGENT, type PipelineState } from './scan/pipeline.js'; +import { ACTIVITY_TO_PROGRESS, type PipelineState } from './scan/pipeline.js'; const ADDRESS = '127.0.0.1:7233'; const NAMESPACE = 'default'; @@ -20,11 +20,30 @@ const TERMINAL_STATUSES: ReadonlySet = new Set(['COMPLETED', 'FAILED', ' export interface RunningAgent { readonly agent: string; + readonly label: string; + /** 'agent' rows join the static pipeline tree; 'operation' rows feed the background-work phase. */ + readonly kind: 'agent' | 'operation'; + /** Set when a persisted parent stage owns this row; the label then reads as that stage's step. */ + readonly parentKey?: string; readonly attempt: number; readonly startedAt?: number; readonly lastFailure?: string; } +/** + * The CLI's activity mirror does not know an activity type the running scan is using, so the + * progress tree cannot be rendered completely. Distinct from a Temporal connection failure. + */ +export class ActivityMirrorError extends Error { + override name = 'ActivityMirrorError' as const; + + constructor(activityType: string) { + super( + `This version of the Shannon command line does not recognise part of the running scan\n(${activityType}). Update Shannon, or watch the scan with: shannon logs `, + ); + } +} + /** Convert a proto ITimestamp (seconds is a Long) to epoch millis. */ function timestampMs( ts: { seconds?: { toString(): string } | number | null; nanos?: number | null } | null, @@ -66,12 +85,19 @@ export async function describeScan(workflowId: string): Promise( @@ -69,7 +73,7 @@ export async function runSastEnrichmentBatch( status: 'failed', usage, message: 'SAST enrichment did not return one complete submit_result call', - terminal: isTerminalProviderFailure(result.errorMessage), + terminal: isTerminalProviderFailure(result), }; } diff --git a/apps/worker/src/ai/sast/capella/sarif-exporter.ts b/apps/worker/src/ai/sast/capella/sarif-exporter.ts index d16c76d3..4f4fd94c 100644 --- a/apps/worker/src/ai/sast/capella/sarif-exporter.ts +++ b/apps/worker/src/ai/sast/capella/sarif-exporter.ts @@ -214,7 +214,7 @@ export async function exportCapellaFindings( const warnings: string[] = []; const validFindings = rawFindings.filter(isExportableFinding); const invalidCount = rawFindings.length - validFindings.length; - if (invalidCount > 0) warnings.push(`${invalidCount} invalid finding(s) were excluded`); + if (invalidCount > 0) warnings.push(`${invalidCount} agentic SAST findings were malformed and left out.`); const gated = validFindings.filter(passesExportGate); const exported = gated @@ -224,9 +224,13 @@ export async function exportCapellaFindings( }) .sort((left, right) => compareText(left.id, right.id)); const excludedCount = gated.length - exported.length; - if (excludedCount > 0) warnings.push(`${excludedCount} finding(s) matched code-path exclusions`); + if (excludedCount > 0) { + warnings.push(`${excludedCount} agentic SAST findings were in paths your config told Shannon to avoid.`); + } if (validFindings.length > 0 && exported.length === 0) { - warnings.push(`all ${validFindings.length} valid finding record(s) were dropped before export`); + warnings.push( + 'Every agentic SAST finding was excluded, so no static-analysis results reached the pentest. Check the avoid rules in your config file.', + ); } const sarifDocument = buildCapellaSarif(exported, options.repositoryLabel); diff --git a/apps/worker/src/ai/structured-generation.ts b/apps/worker/src/ai/structured-generation.ts index cb4907af..0554e5a6 100644 --- a/apps/worker/src/ai/structured-generation.ts +++ b/apps/worker/src/ai/structured-generation.ts @@ -17,6 +17,12 @@ export interface StructuredGenerationRequest { signal?: AbortSignal; } +/** Typed classification of a failed provider request, set whenever `errorMessage` is. */ +export interface StructuredGenerationProviderFailure { + readonly type: 'AuthenticationError' | 'ConfigurationError' | 'AgentExecutionError'; + readonly retryable: boolean; +} + /** Host-neutral outcome of one structured generation request. */ export interface StructuredGenerationResult { stopReason: 'toolUse' | 'stop' | 'length' | 'error' | 'aborted'; @@ -27,6 +33,8 @@ export interface StructuredGenerationResult { costUsd: number; }; errorMessage?: string; + /** Consumers branch on this typed flag, never on `errorMessage` text. */ + providerFailure?: StructuredGenerationProviderFailure; } /** Host-supplied transport that makes exactly one model request per call. */ diff --git a/apps/worker/src/audit/audit-session.ts b/apps/worker/src/audit/audit-session.ts index 39dc2038..06294b5d 100644 --- a/apps/worker/src/audit/audit-session.ts +++ b/apps/worker/src/audit/audit-session.ts @@ -14,11 +14,22 @@ import { PentestError } from '../services/error-handling.js'; import { ErrorCode } from '../types/errors.js'; import type { AgentEndResult } from '../types/index.js'; +import type { AgentMetrics } from '../types/metrics.js'; +import { + type DurableScanState, + type MiscellaneousOutcome, + type PartialReason, + type ReportProgress, + type ReportSarifDisposition, + RunStateError, + type StoredPdfProvenance, +} from '../types/run-state.js'; import { SessionMutex } from '../utils/concurrency.js'; +import { fileExists } from '../utils/file-io.js'; import { formatTimestamp } from '../utils/formatting.js'; import { AgentLogger } from './logger.js'; import { MetricsTracker } from './metrics-tracker.js'; -import { initializeAuditStructure, type SessionMetadata } from './utils.js'; +import { generateSessionJsonPath, initializeAuditStructure, type SessionMetadata } from './utils.js'; import { type AgentLogDetails, WorkflowLogger, type WorkflowSummary } from './workflow-logger.js'; // Global mutex instance @@ -170,6 +181,33 @@ export class AuditSession { * End agent execution (mutex-protected) */ async endAgent(agentName: string, result: AgentEndResult): Promise { + await this.finishAgentLogs(agentName, result); + + // 3. Acquire mutex before touching session.json + const unlock = await sessionMutex.lock(this.sessionId); + try { + // 4. Reload-then-write inside mutex to prevent lost updates during parallel phases + await this.metricsTracker.reload(); + await this.metricsTracker.endAgent(agentName, result); + } finally { + unlock(); + } + } + + /** Record a successful report-model attempt as a nonterminal durable draft. */ + async endReportDraft(result: AgentEndResult): Promise { + await this.finishAgentLogs('report', result); + + const unlock = await sessionMutex.lock(this.sessionId); + try { + await this.metricsTracker.reload(); + return await this.metricsTracker.recordReportDraft(result); + } finally { + unlock(); + } + } + + private async finishAgentLogs(agentName: string, result: AgentEndResult): Promise { // 1. Finalize agent log and close the stream if (this.currentLogger) { await this.currentLogger.logEvent('agent_end', { @@ -195,13 +233,128 @@ export class AuditSession { ...(result.error !== undefined && { error: result.error }), }; await this.workflowLogger.logAgent(agentName, 'end', agentLogDetails); + } + + /** + * Initialize fresh durable state or validate a resume record without reconstructing it. + * + * This is the first activity of every run, so it is also where a fresh workspace's + * session.json is created. It therefore takes the workflow id explicitly: initializing + * without one would persist a session with no `originalWorkflowId`, and later calls load + * the existing file rather than rewriting identity, leaving the scan unresolvable. + */ + async initializeDurableScanState( + workflowId: string, + exploit: boolean, + context: 'fresh' | 'resume', + ): Promise { + if (context === 'resume' && !(await fileExists(generateSessionJsonPath(this.sessionMetadata)))) { + throw new RunStateError('IncompatibleWorkspaceError', 'session-json-missing-on-resume'); + } + await this.initialize(workflowId); - // 3. Acquire mutex before touching session.json const unlock = await sessionMutex.lock(this.sessionId); try { - // 4. Reload-then-write inside mutex to prevent lost updates during parallel phases await this.metricsTracker.reload(); - await this.metricsTracker.endAgent(agentName, result); + return await this.metricsTracker.initializeDurableScanState(exploit, context); + } finally { + unlock(); + } + } + + /** Return a validated snapshot of durable execution state. */ + async getDurableScanState(): Promise { + await this.ensureInitialized(); + const unlock = await sessionMutex.lock(this.sessionId); + try { + await this.metricsTracker.reload(); + return this.metricsTracker.getDurableScanState(); + } finally { + unlock(); + } + } + + /** Persist a `miscellaneous` branch outcome under the session lock. */ + async updateMiscellaneousOutcome(outcome: MiscellaneousOutcome): Promise { + await this.ensureInitialized(); + const unlock = await sessionMutex.lock(this.sessionId); + try { + await this.metricsTracker.reload(); + return await this.metricsTracker.updateMiscellaneousOutcome(outcome); + } finally { + unlock(); + } + } + + /** Persist the ordered renumber-failure set and durable partial reasons before assembly. */ + async initializeReportProgress( + failedClasses: readonly import('../types/reconciliation.js').ReconciliationClass[], + partialReasons: readonly PartialReason[], + ): Promise { + await this.ensureInitialized(); + const unlock = await sessionMutex.lock(this.sessionId); + try { + await this.metricsTracker.reload(); + return await this.metricsTracker.initializeReportProgress(failedClasses, partialReasons); + } finally { + unlock(); + } + } + + /** Persist the post-compaction canonical report checkpoint without terminal success. */ + async recordCanonicalReportCheckpoint( + checkpoint: string, + appendReasons: readonly PartialReason[] = [], + ): Promise { + await this.ensureInitialized(); + const unlock = await sessionMutex.lock(this.sessionId); + try { + await this.metricsTracker.reload(); + return await this.metricsTracker.recordCanonicalReportCheckpoint(checkpoint, appendReasons); + } finally { + unlock(); + } + } + + /** Atomically mark report finalized and successful after external proof validation. */ + async finalizeReportProgress( + finalCheckpoint: string, + manifestSha256: string, + terminal: { + readonly sarifDisposition: ReportSarifDisposition; + readonly pdfProvenance: StoredPdfProvenance | null; + readonly partialReasons: readonly PartialReason[]; + }, + ): Promise { + await this.ensureInitialized(); + const unlock = await sessionMutex.lock(this.sessionId); + try { + await this.metricsTracker.reload(); + return await this.metricsTracker.finalizeReportProgress(finalCheckpoint, manifestSha256, terminal); + } finally { + unlock(); + } + } + + /** Return an invalid model draft to pending without erasing its billable attempt. */ + async rollbackReportDraft(): Promise { + await this.ensureInitialized(); + const unlock = await sessionMutex.lock(this.sessionId); + try { + await this.metricsTracker.reload(); + return await this.metricsTracker.rollbackReportDraft(); + } finally { + unlock(); + } + } + + /** Read persisted report metrics for model-skip resume. */ + async getReportMetrics(): Promise { + await this.ensureInitialized(); + const unlock = await sessionMutex.lock(this.sessionId); + try { + await this.metricsTracker.reload(); + return this.metricsTracker.getReportMetrics(); } finally { unlock(); } diff --git a/apps/worker/src/audit/metrics-tracker.ts b/apps/worker/src/audit/metrics-tracker.ts index 060ffa47..68e3ca59 100644 --- a/apps/worker/src/audit/metrics-tracker.ts +++ b/apps/worker/src/audit/metrics-tracker.ts @@ -15,6 +15,21 @@ import { PentestError } from '../services/error-handling.js'; import { AGENT_PHASE_MAP, type PhaseName } from '../session-manager.js'; import { ErrorCode } from '../types/errors.js'; import type { AgentEndResult, AgentName } from '../types/index.js'; +import type { AgentMetrics } from '../types/metrics.js'; +import { + appendPartialReasons, + createInitialDurableScanState, + type DurableScanState, + isDurableScanState, + isOrderedPartialReasonSet, + type MiscellaneousOutcome, + type PartialReason, + type ReportProgress, + type ReportSarifDisposition, + RunStateError, + recordMiscellaneousOutcome, + type StoredPdfProvenance, +} from '../types/run-state.js'; import { atomicWrite, fileExists, readJson } from '../utils/file-io.js'; import { calculatePercentage, formatTimestamp } from '../utils/formatting.js'; import { generateSessionJsonPath, type SessionMetadata } from './utils.js'; @@ -78,6 +93,7 @@ interface SessionData { phases: Record; agents: Record; }; + durableScanState?: DurableScanState; } interface ActiveTimer { @@ -176,51 +192,11 @@ export class MetricsTracker { ); } - // 1. Initialize agent metrics if first time seeing this agent - const existingAgent = this.data.metrics.agents[agentName]; - const agent = existingAgent ?? { - status: 'in-progress' as const, - attempts: [], - final_duration_ms: 0, - total_cost_usd: 0, - total_input_tokens: 0, - total_output_tokens: 0, - total_cache_read_tokens: 0, - total_cache_write_tokens: 0, - }; - this.data.metrics.agents[agentName] = agent; - - // 2. Build attempt record with optional model/error fields - const attempt: AttemptData = { - attempt_number: result.attemptNumber, - duration_ms: result.duration_ms, - cost_usd: result.cost_usd, - success: result.success, - timestamp: formatTimestamp(), - ...(result.input_tokens !== undefined && { input_tokens: result.input_tokens }), - ...(result.output_tokens !== undefined && { output_tokens: result.output_tokens }), - ...(result.cache_read_tokens !== undefined && { cache_read_tokens: result.cache_read_tokens }), - ...(result.cache_write_tokens !== undefined && { cache_write_tokens: result.cache_write_tokens }), - ...(result.turns !== undefined && { turns: result.turns }), - }; - - if (result.model) { - attempt.model = result.model; + if (agentName === 'report' && result.success) { + throw new RunStateError('DurableStateConflictError', 'report-success-requires-terminal-promotion'); } - if (result.error) { - attempt.error = result.error; - } - - // 3. Append attempt to history - agent.attempts.push(attempt); - - // 4. Recalculate totals across all attempts (includes failures) - agent.total_cost_usd = agent.attempts.reduce((sum, a) => sum + a.cost_usd, 0); - agent.total_input_tokens = agent.attempts.reduce((sum, a) => sum + (a.input_tokens ?? 0), 0); - agent.total_output_tokens = agent.attempts.reduce((sum, a) => sum + (a.output_tokens ?? 0), 0); - agent.total_cache_read_tokens = agent.attempts.reduce((sum, a) => sum + (a.cache_read_tokens ?? 0), 0); - agent.total_cache_write_tokens = agent.attempts.reduce((sum, a) => sum + (a.cache_write_tokens ?? 0), 0); + const agent = this.appendAttempt(agentName, result); // 5. Update agent status based on outcome if (result.success) { @@ -235,6 +211,11 @@ export class MetricsTracker { if (result.checkpoint) { agent.checkpoint = result.checkpoint; } + + if (agentName === 'miscellaneous-exploit') { + const durableState = this.requireDurableScanState(); + this.data.durableScanState = recordMiscellaneousOutcome(durableState, 'completed'); + } } else { // A non-final failed attempt stays in-progress (Temporal will retry); only the // terminal attempt (or an unqualified failure) marks the agent failed. @@ -251,6 +232,319 @@ export class MetricsTracker { await this.save(); } + /** Initialize or validate the schema-1 state without reconstructing a missing resume record. */ + async initializeDurableScanState(exploit: boolean, context: 'fresh' | 'resume'): Promise { + const data = this.requireData(); + const existing = data.durableScanState; + if (existing !== undefined) { + if (!isDurableScanState(existing)) { + throw new RunStateError('CorruptedSessionError', 'durable-state-malformed'); + } + if (existing.exploit !== exploit) { + throw new RunStateError('IncompatibleWorkspaceError', 'exploit-mode-changed'); + } + return structuredClone(existing); + } + + if (context === 'resume') { + throw new RunStateError('IncompatibleWorkspaceError', 'durable-state-missing-on-resume'); + } + const hasRecordedWork = + Object.keys(data.metrics.agents).length > 0 || (data.session.resumeAttempts?.length ?? 0) > 0; + if (hasRecordedWork) { + throw new RunStateError('CorruptedSessionError', 'durable-state-missing-after-work'); + } + + const initialized = createInitialDurableScanState(exploit); + data.durableScanState = initialized; + await this.save(); + return structuredClone(initialized); + } + + /** Return validated durable state. */ + getDurableScanState(): DurableScanState { + return structuredClone(this.requireDurableScanState()); + } + + /** Persist the internal `miscellaneous` result and append its agent only for actionable exploitation. */ + async updateMiscellaneousOutcome(outcome: MiscellaneousOutcome): Promise { + const data = this.requireData(); + const next = recordMiscellaneousOutcome(this.requireDurableScanState(), outcome); + if (!isDurableScanState(next)) { + throw new RunStateError('DurableStateConflictError', 'miscellaneous-outcome-produced-invalid-state'); + } + data.durableScanState = next; + await this.save(); + return structuredClone(next); + } + + /** Persist the complete failed-class set and durable partial reasons before report assembly. */ + async initializeReportProgress( + failedClasses: readonly import('../types/reconciliation.js').ReconciliationClass[], + partialReasons: readonly PartialReason[], + ): Promise { + const data = this.requireData(); + const durableState = this.requireDurableScanState(); + if (!isOrderedPartialReasonSet(partialReasons)) { + throw new RunStateError('DurableStateConflictError', 'report-pending-reasons-invalid'); + } + if (durableState.report !== undefined) { + if (!this.arraysEqual(durableState.report.renumber_failed_classes, failedClasses)) { + throw new RunStateError('DurableStateConflictError', 'report-failed-class-set-changed'); + } + // A lost-acknowledgement re-drive adopts the same set; a resume may append newly + // observed reasons, but never removes a durable one. Append preserves every existing + // member, so an unchanged length means nothing new was observed. + const merged = appendPartialReasons(durableState.report.partial_reasons, partialReasons); + if (merged.length === durableState.report.partial_reasons.length) { + return structuredClone(durableState.report); + } + const report: ReportProgress = { ...durableState.report, partial_reasons: merged }; + const next = { ...durableState, report }; + if (!isDurableScanState(next)) { + throw new RunStateError('DurableStateConflictError', 'report-pending-reasons-conflict'); + } + data.durableScanState = next; + await this.save(); + return structuredClone(report); + } + + const report: ReportProgress = { + stage: 'pending', + renumber_failed_classes: [...failedClasses], + partial_reasons: appendPartialReasons([], partialReasons), + }; + const next = { ...durableState, report }; + if (!isDurableScanState(next)) { + throw new RunStateError('DurableStateConflictError', 'report-pending-invalid'); + } + data.durableScanState = next; + await this.save(); + return structuredClone(report); + } + + /** Record billable report-model metrics and a real Git checkpoint without terminal success. */ + async recordReportDraft(result: AgentEndResult): Promise { + const data = this.requireData(); + const checkpoint = result.checkpoint; + if (!result.success || checkpoint === undefined) { + throw new RunStateError('DurableStateConflictError', 'report-draft-requires-success-checkpoint'); + } + const durableState = this.requireDurableScanState(); + const current = durableState.report; + if (current === undefined || current.stage === 'finalized') { + throw new RunStateError('DurableStateConflictError', 'report-draft-invalid-source-stage'); + } + if (current.stage === 'draft') { + if (current.model_checkpoint !== checkpoint) { + throw new RunStateError('DurableStateConflictError', 'report-model-checkpoint-conflict'); + } + return structuredClone(current); + } + + const agent = this.appendAttempt('report', result); + agent.status = 'in-progress'; + agent.final_duration_ms = result.duration_ms; + agent.checkpoint = checkpoint; + if (result.model !== undefined) { + agent.model = result.model; + } else { + delete agent.model; + } + + const report: ReportProgress = { + stage: 'draft', + renumber_failed_classes: [...current.renumber_failed_classes], + partial_reasons: [...current.partial_reasons], + model_checkpoint: checkpoint, + }; + const next = { ...durableState, report }; + if (!isDurableScanState(next)) { + throw new RunStateError('DurableStateConflictError', 'report-draft-invalid'); + } + data.durableScanState = next; + this.activeTimers.delete('report'); + this.recalculateAggregations(); + await this.save(); + return structuredClone(report); + } + + /** Record the post-compaction canonical checkpoint while keeping report nonterminal. */ + async recordCanonicalReportCheckpoint( + checkpoint: string, + appendReasons: readonly PartialReason[] = [], + ): Promise { + const data = this.requireData(); + const durableState = this.requireDurableScanState(); + const current = durableState.report; + if (current?.stage === 'finalized') { + if (current.canonical_checkpoint !== checkpoint) { + throw new RunStateError('DurableStateConflictError', 'report-canonical-checkpoint-conflict'); + } + return structuredClone(current); + } + if (current?.stage !== 'draft') { + throw new RunStateError('DurableStateConflictError', 'report-canonical-invalid-source-stage'); + } + const mergedReasons = appendPartialReasons(current.partial_reasons, appendReasons); + if (current.canonical_checkpoint !== undefined) { + if (current.canonical_checkpoint !== checkpoint) { + throw new RunStateError('DurableStateConflictError', 'report-canonical-checkpoint-conflict'); + } + if (mergedReasons.length === current.partial_reasons.length) { + return structuredClone(current); + } + } + + const report: ReportProgress = { + ...current, + partial_reasons: mergedReasons, + canonical_checkpoint: checkpoint, + }; + const next = { ...durableState, report }; + if (!isDurableScanState(next)) { + throw new RunStateError('DurableStateConflictError', 'report-canonical-invalid'); + } + data.durableScanState = next; + await this.save(); + return structuredClone(report); + } + + /** + * Promote a verified finalization commit to the only terminal report state. + * + * `final_checkpoint` and the manifest digest are strict match-or-conflict fields. The SARIF + * disposition and its `report_sarif_failed` reason are derived from the committed manifest, + * partial reasons stay append-only, and the PDF provenance is replaceable after finalization. + */ + async finalizeReportProgress( + finalCheckpoint: string, + manifestSha256: string, + terminal: { + readonly sarifDisposition: ReportSarifDisposition; + readonly pdfProvenance: StoredPdfProvenance | null; + readonly partialReasons: readonly PartialReason[]; + }, + ): Promise { + const data = this.requireData(); + const durableState = this.requireDurableScanState(); + const current = durableState.report; + if (current?.stage === 'finalized') { + if (current.final_checkpoint !== finalCheckpoint || current.finalization_manifest_sha256 !== manifestSha256) { + throw new RunStateError('DurableStateConflictError', 'report-final-checkpoint-conflict'); + } + if (current.sarif_disposition !== terminal.sarifDisposition) { + throw new RunStateError('DurableStateConflictError', 'report-final-disposition-conflict'); + } + const adopted: ReportProgress = { + ...current, + partial_reasons: appendPartialReasons(current.partial_reasons, terminal.partialReasons), + ...(terminal.pdfProvenance !== null ? { pdf_provenance: terminal.pdfProvenance } : {}), + }; + if (terminal.pdfProvenance === null && 'pdf_provenance' in adopted) { + const { pdf_provenance: _removed, ...withoutProvenance } = adopted; + return await this.persistFinalizedReport(data, durableState, withoutProvenance as ReportProgress); + } + return await this.persistFinalizedReport(data, durableState, adopted); + } + if (current?.stage !== 'draft' || current.canonical_checkpoint === undefined) { + throw new RunStateError('DurableStateConflictError', 'report-final-invalid-source-stage'); + } + + const sarifReasons: readonly PartialReason[] = + terminal.sarifDisposition === 'render_failed' ? [{ code: 'report_sarif_failed' }] : []; + const report: ReportProgress = { + stage: 'finalized', + renumber_failed_classes: [...current.renumber_failed_classes], + partial_reasons: appendPartialReasons(current.partial_reasons, [...terminal.partialReasons, ...sarifReasons]), + model_checkpoint: current.model_checkpoint, + canonical_checkpoint: current.canonical_checkpoint, + final_checkpoint: finalCheckpoint, + finalization_manifest_sha256: manifestSha256, + sarif_disposition: terminal.sarifDisposition, + ...(terminal.pdfProvenance !== null && { pdf_provenance: terminal.pdfProvenance }), + }; + + const agent = data.metrics.agents.report; + if (agent === undefined || agent.attempts.length === 0) { + throw new RunStateError('DurableStateConflictError', 'report-final-without-model-metrics'); + } + const persisted = await this.persistFinalizedReport(data, durableState, report, () => { + agent.status = 'success'; + agent.checkpoint = finalCheckpoint; + const latestAttempt = agent.attempts.at(-1); + agent.final_duration_ms = latestAttempt?.duration_ms ?? agent.final_duration_ms; + this.recalculateAggregations(); + }); + return persisted; + } + + private async persistFinalizedReport( + data: SessionData, + durableState: DurableScanState, + report: ReportProgress, + beforeSave?: () => void, + ): Promise { + const next = { ...durableState, report }; + if (!isDurableScanState(next)) { + throw new RunStateError('DurableStateConflictError', 'report-final-invalid'); + } + beforeSave?.(); + data.durableScanState = next; + await this.save(); + return structuredClone(report); + } + + /** Roll back only report state after a coherent draft shape fails checkpoint validation. */ + async rollbackReportDraft(): Promise { + const data = this.requireData(); + const durableState = this.requireDurableScanState(); + const current = durableState.report; + if (current?.stage !== 'draft') { + throw new RunStateError('DurableStateConflictError', 'report-draft-rollback-invalid-source-stage'); + } + const report: ReportProgress = { + stage: 'pending', + renumber_failed_classes: [...current.renumber_failed_classes], + partial_reasons: [...current.partial_reasons], + }; + const agent = data.metrics.agents.report; + if (agent !== undefined) { + agent.status = 'in-progress'; + delete agent.checkpoint; + delete agent.model; + } + data.durableScanState = { ...durableState, report }; + this.recalculateAggregations(); + await this.save(); + return structuredClone(report); + } + + /** Return persisted report metrics for a coherent draft/finalized model-skip path. */ + getReportMetrics(): AgentMetrics { + const durableState = this.requireDurableScanState(); + if (durableState.report?.stage !== 'draft' && durableState.report?.stage !== 'finalized') { + throw new RunStateError('DurableStateConflictError', 'report-metrics-before-draft'); + } + const agent = this.requireData().metrics.agents.report; + if (agent === undefined || agent.attempts.length === 0) { + throw new RunStateError('CorruptedSessionError', 'report-draft-metrics-missing'); + } + const latest = agent.attempts.at(-1); + return { + durationMs: agent.final_duration_ms, + inputTokens: agent.total_input_tokens, + outputTokens: agent.total_output_tokens, + cacheReadTokens: agent.total_cache_read_tokens, + cacheWriteTokens: agent.total_cache_write_tokens, + costUsd: agent.total_cost_usd, + numTurns: agent.attempts.reduce((sum, attempt) => sum + (attempt.turns ?? 0), 0), + ...(latest?.model !== undefined && { model: latest.model }), + ...(agent.checkpoint !== undefined && { checkpoint: agent.checkpoint }), + skipped: true, + }; + } + /** * Update session status */ @@ -294,6 +588,12 @@ export class MetricsTracker { this.data.session.resumeAttempts = []; } + // A lost-acknowledgement re-drive of the same resume adopts the earlier record instead + // of appending a duplicate row for the same workflow id. + if (this.data.session.resumeAttempts.some((attempt) => attempt.workflowId === workflowId)) { + return; + } + // Add new resume attempt const resumeAttempt: ResumeAttempt = { workflowId, @@ -399,4 +699,64 @@ export class MetricsTracker { async reload(): Promise { this.data = await readJson(this.sessionJsonPath); } + + private requireData(): SessionData { + if (this.data === null) { + throw new RunStateError('CorruptedSessionError', 'metrics-tracker-not-initialized'); + } + return this.data; + } + + private requireDurableScanState(): DurableScanState { + const durableState = this.requireData().durableScanState; + if (durableState === undefined) { + throw new RunStateError('CorruptedSessionError', 'durable-state-missing'); + } + if (!isDurableScanState(durableState)) { + throw new RunStateError('CorruptedSessionError', 'durable-state-malformed'); + } + return durableState; + } + + private appendAttempt(agentName: string, result: AgentEndResult): AgentAuditMetrics { + const data = this.requireData(); + const existingAgent = data.metrics.agents[agentName]; + const agent = existingAgent ?? { + status: 'in-progress' as const, + attempts: [], + final_duration_ms: 0, + total_cost_usd: 0, + total_input_tokens: 0, + total_output_tokens: 0, + total_cache_read_tokens: 0, + total_cache_write_tokens: 0, + }; + data.metrics.agents[agentName] = agent; + + const attempt: AttemptData = { + attempt_number: result.attemptNumber, + duration_ms: result.duration_ms, + cost_usd: result.cost_usd, + success: result.success, + timestamp: formatTimestamp(), + ...(result.input_tokens !== undefined && { input_tokens: result.input_tokens }), + ...(result.output_tokens !== undefined && { output_tokens: result.output_tokens }), + ...(result.cache_read_tokens !== undefined && { cache_read_tokens: result.cache_read_tokens }), + ...(result.cache_write_tokens !== undefined && { cache_write_tokens: result.cache_write_tokens }), + ...(result.turns !== undefined && { turns: result.turns }), + ...(result.model !== undefined && { model: result.model }), + ...(result.error !== undefined && { error: result.error }), + }; + agent.attempts.push(attempt); + agent.total_cost_usd = agent.attempts.reduce((sum, entry) => sum + entry.cost_usd, 0); + agent.total_input_tokens = agent.attempts.reduce((sum, entry) => sum + (entry.input_tokens ?? 0), 0); + agent.total_output_tokens = agent.attempts.reduce((sum, entry) => sum + (entry.output_tokens ?? 0), 0); + agent.total_cache_read_tokens = agent.attempts.reduce((sum, entry) => sum + (entry.cache_read_tokens ?? 0), 0); + agent.total_cache_write_tokens = agent.attempts.reduce((sum, entry) => sum + (entry.cache_write_tokens ?? 0), 0); + return agent; + } + + private arraysEqual(left: readonly T[], right: readonly T[]): boolean { + return left.length === right.length && left.every((value, index) => value === right[index]); + } } diff --git a/apps/worker/src/audit/workflow-logger.ts b/apps/worker/src/audit/workflow-logger.ts index eb5a9b66..66a5e13f 100644 --- a/apps/worker/src/audit/workflow-logger.ts +++ b/apps/worker/src/audit/workflow-logger.ts @@ -29,12 +29,33 @@ export interface AgentMetricsSummary { costUsd: number | null; } +/** Mirror of the derived partial-reason view; the safe message is already resolved. */ +export interface WorkflowSummaryPartialReason { + readonly code: string; + readonly message: string; + readonly vulnerabilityClass?: string; + readonly stage?: string; +} + export interface WorkflowSummary { status: 'completed' | 'failed' | 'cancelled' | 'partial'; totalDurationMs: number; totalCostUsd: number; + /** Agents that actually ran. Mutually exclusive from `skippedAgents`. */ completedAgents: string[]; + /** Expected agents that never ran because their class had nothing to exploit. */ + skippedAgents?: readonly string[]; agentMetrics: Record; + /** Ordered durable degradation reasons; present and non-empty exactly for partial runs. */ + partialReasons?: readonly WorkflowSummaryPartialReason[]; + /** False when operational (Capella/reconciliation) spend is known to be incomplete. */ + usageAccountingComplete?: boolean; + /** Reader-facing name of the stage a failed agentic-SAST run stopped at. */ + agenticSastFailedStage?: string; + /** Sanitized failure sentence from a failed agentic-SAST run; safe for operator output. */ + agenticSastFailureMessage?: string; + /** Bounded machine code from a failed agentic-SAST run, when one was preserved. */ + agenticSastErrorCode?: string; error?: string; } @@ -322,7 +343,22 @@ export class WorkflowLogger { async logWorkflowComplete(summary: WorkflowSummary): Promise { await this.ensureInitialized(); - const status = summary.status === 'completed' ? 'COMPLETED' : 'FAILED'; + // Each terminal status prints its own header so partial and cancelled runs are never + // mislabelled as full successes or failures. The CLI log tailer stops on exactly these + // headings (COMPLETION_PATTERN in apps/cli/src/commands/logs.ts); adding one here without + // adding it there strands `shannon logs` on a finished scan. + const STATUS_HEADERS: Record = { + completed: 'COMPLETED', + partial: 'PARTIAL', + cancelled: 'CANCELLED', + failed: 'FAILED', + }; + const status = STATUS_HEADERS[summary.status]; + + // completedAgents and skippedAgents are mutually exclusive: an agent that was skipped + // because its class had nothing to exploit is tracked only in skippedAgents. + const skippedAgents = summary.skippedAgents ?? []; + const ranCount = summary.completedAgents.length; const lines: string[] = [ '', @@ -333,14 +369,36 @@ export class WorkflowLogger { `Status: ${summary.status}`, `Duration: ${formatDuration(summary.totalDurationMs)}`, `Total Cost: $${summary.totalCostUsd.toFixed(4)}`, - `Agents: ${summary.completedAgents.length} completed`, + `Agents: ${ranCount} ran, ${skippedAgents.length} skipped`, ]; + if (summary.usageAccountingComplete === false) { + lines.push('Cost Note: Cost is incomplete — some background work is not included in this total.'); + } if (summary.error) { lines.push(this.formatErrorBlock(summary.error).trimEnd()); } - if (summary.completedAgents.length > 0) { + if (summary.partialReasons !== undefined && summary.partialReasons.length > 0) { + lines.push(''); + lines.push('Why this scan is partial:'); + for (const reason of summary.partialReasons) { + lines.push(` - ${reason.message}`); + } + // The reason above says what degraded; these three name the agentic-SAST failure + // behind it, under the same labels the terminal and worker output use. + if (summary.agenticSastFailedStage !== undefined) { + lines.push(` Agentic SAST stopped at: ${summary.agenticSastFailedStage}`); + } + if (summary.agenticSastFailureMessage !== undefined) { + lines.push(` What happened: ${summary.agenticSastFailureMessage}`); + } + if (summary.agenticSastErrorCode !== undefined) { + lines.push(` Reference code (for a bug report): ${summary.agenticSastErrorCode}`); + } + } + + if (summary.completedAgents.length > 0 || skippedAgents.length > 0) { lines.push(''); lines.push('Agent Breakdown:'); @@ -354,6 +412,12 @@ export class WorkflowLogger { lines.push(` - ${agentName}`); } } + for (const agentName of skippedAgents) { + lines.push(` - ${agentName} (skipped — nothing to exploit)`); + } + } + for (const agentName of skippedAgents) { + lines.push(` - ${agentName} (skipped — nothing to exploit)`); } lines.push('================================================================================'); diff --git a/apps/worker/src/config-parser.ts b/apps/worker/src/config-parser.ts index fc3c62c5..d229a69d 100644 --- a/apps/worker/src/config-parser.ts +++ b/apps/worker/src/config-parser.ts @@ -10,15 +10,21 @@ import type { FormatsPlugin } from 'ajv-formats'; import yaml from 'js-yaml'; import { fs } from 'zx'; import { PentestError } from './services/error-handling.js'; -import { - ALL_VULN_CLASSES, - type Authentication, - type Config, - type DistributedConfig, - type Rule, -} from './types/config.js'; +import type { Authentication, Config, DistributedConfig, Rule } from './types/config.js'; import { ErrorCode } from './types/errors.js'; +/** + * Parses and validates scan configuration YAML against config-schema.json, then + * distributes it into the plain values consumed by prompts and services. + * + * The schema is closed: every object in config-schema.json sets `additionalProperties: + * false`, so an unrecognized field anywhere in the config is a hard validation failure + * rather than a silently ignored typo. There is no public way to select which analysis + * classes run; the schema only exposes steering knobs (rules, authentication, + * agentic_sast.enabled, exploit, report, rules_of_engagement) on top of the fixed + * five-class pipeline. + */ + // Handle ESM/CJS interop for ajv-formats using require const require = createRequire(import.meta.url); const addFormats: FormatsPlugin = require('ajv-formats'); @@ -42,6 +48,10 @@ try { }); } +// Free-text config fields (description, rules_of_engagement, rule values, login fields, +// report.guidance) get interpolated verbatim into agent prompts via prompt-manager.ts. +// These patterns block the more obvious ways a scan config could smuggle markup, script +// URLs, or path traversal into that prompt text or into a rendered value. const DANGEROUS_PATTERNS: RegExp[] = [ /\.\.\//, // Path traversal /[<>]/, // HTML/XML injection @@ -312,6 +322,9 @@ export const parseConfigYAML = (yamlContent: string): Config => { return config as Config; }; +// Runs before schema validation so a renamed field fails with a specific "renamed to X" +// message instead of the generic "additionalProperties" rejection the closed schema +// would otherwise produce for the old field name. function checkDeprecatedFields(config: Config): void { const messages: string[] = []; @@ -387,7 +400,7 @@ const validateConfig = (config: Config): void => { !!config.rules || !!config.authentication || !!config.description || - !!config.vuln_classes || + !!config.agentic_sast || config.exploit !== undefined || !!config.report || !!config.rules_of_engagement; @@ -673,9 +686,10 @@ export const distributeConfig = (config: Config | null): DistributedConfig => { const authentication = config?.authentication || null; const description = config?.description?.trim() || ''; - const vuln_classes = - config?.vuln_classes && config.vuln_classes.length > 0 ? [...config.vuln_classes] : [...ALL_VULN_CLASSES]; - + // The schema types boolean-shaped fields (exploit, report.sarif, agentic_sast.enabled) + // as a string enum ("true"/"false") rather than JSON boolean, since YAML's FAILSAFE_SCHEMA + // parses bareword true/false as strings. The string comparison here is intentional, not + // a leftover from a looser type. const exploit = config?.exploit !== undefined ? config.exploit === 'true' : true; const report = { @@ -693,7 +707,7 @@ export const distributeConfig = (config: Config | null): DistributedConfig => { focus: focus.map(sanitizeRule), authentication: authentication ? sanitizeAuthentication(authentication) : null, description, - vuln_classes, + ...(config?.agentic_sast?.enabled === 'true' && { agenticSast: true as const }), exploit, report, rules_of_engagement, diff --git a/apps/worker/src/paths.ts b/apps/worker/src/paths.ts index 8b52b99a..b52ea0d3 100644 --- a/apps/worker/src/paths.ts +++ b/apps/worker/src/paths.ts @@ -46,6 +46,9 @@ export const REPORT_JSON_FILENAME = 'report.json'; /** SARIF 2.1.0 log, written for exploit=true runs unless report.sarif is set to false. */ export const SARIF_FILENAME = 'report.sarif'; +/** Deterministic receipt for the canonical report finalization commit. */ +export const REPORT_FINALIZATION_MANIFEST_FILENAME = 'report_finalization_manifest.json'; + /** * Resolve the session.json path for a run directory, preferring the current * `.shannon/` location and falling back to the legacy run-root location so diff --git a/apps/worker/src/services/agent-execution.ts b/apps/worker/src/services/agent-execution.ts index 95f60869..5eedcde2 100644 --- a/apps/worker/src/services/agent-execution.ts +++ b/apps/worker/src/services/agent-execution.ts @@ -33,6 +33,7 @@ import type { AgentEndResult } from '../types/audit.js'; import { ErrorCode, type PentestErrorType } from '../types/errors.js'; import type { AgentMetrics } from '../types/metrics.js'; import { err, isErr, ok, type Result } from '../types/result.js'; +import { assertFixedAnalysisScope } from '../types/run-state.js'; import { getAgentGitPaths } from './agent-git-paths.js'; import type { ConfigLoaderService } from './config-loader.js'; import { PentestError } from './error-handling.js'; @@ -51,11 +52,14 @@ export interface AgentExecutionInput { configYAML?: string | undefined; pipelineTestingMode?: boolean | undefined; attemptNumber: number; + /** Workflow-resolved fixed scope; prompt generation never derives this from public config. */ + analysisClasses: readonly import('../types/config.js').VulnClass[]; promptDir?: string | undefined; customTools?: import('@earendil-works/pi-coding-agent').ToolDefinition[]; failedClasses?: readonly import('../types/config.js').VulnClass[] | undefined; // Renders the deliverable to disk; invoked after validation, before the success commit. - writeDeliverable?: (deliverablesPath: string) => Promise; + writeDeliverable?: (deliverablesPath: string, execution: { readonly model?: string }) => Promise; + successDisposition?: 'terminal' | 'report-draft'; cancellationSignal?: AbortSignal | undefined; } @@ -145,14 +149,29 @@ export class AgentExecutionService { configYAML, pipelineTestingMode = false, attemptNumber, + analysisClasses, promptDir, customTools, failedClasses, writeDeliverable, + successDisposition = 'terminal', cancellationSignal, } = input; const gitPaths = getAgentGitPaths(agentName); + assertFixedAnalysisScope(analysisClasses); + if (successDisposition === 'report-draft' && agentName !== 'report') { + return err( + new PentestError( + 'Draft success is reserved for the report agent', + 'validation', + false, + { agentName }, + ErrorCode.CONFIG_VALIDATION_FAILED, + ), + ); + } + // 1. Load config (pre-parsed configData → raw YAML → file path) const configResult = await this.configLoader.loadOptional(configPath, configData, configYAML); if (isErr(configResult)) { @@ -170,6 +189,7 @@ export class AgentExecutionService { webUrl, repoPath, AUTH_STATE_FILE: authStateFile(auditSession.sessionMetadata), + analysisClasses, ...(failedClasses !== undefined && { failedClasses }), }, distributedConfig, @@ -278,7 +298,9 @@ export class AgentExecutionService { // 10. Render the deliverable to disk so the success commit below stages it if (writeDeliverable) { - await writeDeliverable(deliverablesPath); + await writeDeliverable(deliverablesPath, { + ...(result.model !== undefined && { model: result.model }), + }); } // 11. Success - commit deliverables (scoped) and capture the checkpoint hash @@ -287,6 +309,15 @@ export class AgentExecutionService { return gitFailureForAgent(agentName, 'commit successful results', commitResult.error); } commitHash = commitResult.commitHash; + if (successDisposition === 'report-draft' && commitHash === undefined) { + return new PentestError( + 'The report was written but could not be saved. Re-running this workspace retries the reporting phase without repeating the analysis.', + 'filesystem', + false, + { agentName }, + ErrorCode.GIT_CHECKPOINT_FAILED, + ); + } return null; } catch (error) { if (error instanceof PentestError) return error; @@ -331,7 +362,11 @@ export class AgentExecutionService { model: result.model, ...(commitHash && { checkpoint: commitHash }), }; - await auditSession.endAgent(agentName, endResult); + if (successDisposition === 'report-draft') { + await auditSession.endReportDraft(endResult); + } else { + await auditSession.endAgent(agentName, endResult); + } return ok(endResult); } @@ -419,6 +454,7 @@ export class AgentExecutionService { costUsd: endResult.cost_usd, numTurns: result.turns ?? null, model: result.model, + ...(endResult.checkpoint !== undefined && { checkpoint: endResult.checkpoint }), }; } } diff --git a/apps/worker/src/services/code-location-join.ts b/apps/worker/src/services/code-location-join.ts index 90ff14a7..5cbfc3d6 100644 --- a/apps/worker/src/services/code-location-join.ts +++ b/apps/worker/src/services/code-location-join.ts @@ -4,75 +4,131 @@ // it under the terms of the GNU Affero General Public License version 3 // as published by the Free Software Foundation. -/** - * Attach vuln-queue code locations to collected findings. - * - * The vuln agent authors `code_locations` once, into its queue. Every stage after that used to - * re-transcribe them — the exploit agent into its evidence, the report agent into `add_finding` — - * and each hop lost some: 100% in the queue, 98% in the evidence, 42-63% by the report. Nothing - * about the copy is a judgement call, and `finding_id` matches the queue `ID` exactly, so the - * locations are joined here instead of being asked for again. - */ +/** Join analysis and SAST locations from committed report-facing tasks without mixing location lanes. */ -import { fs, path } from 'zx'; import type { QueueCodeLocation } from '../ai/queue-schemas.js'; +import type { SastSourceLocation } from '../ai/reconciliation/contracts.js'; import type { AddFindingInput } from '../collectors/finding-collector.js'; import type { ActivityLogger } from '../types/activity-logger.js'; import { ALL_VULN_CLASSES } from '../types/config.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; +import { readCommittedFile } from './git-manager.js'; +import { renumberMapPath } from './renumber-core.js'; interface QueueEntry { ID?: string; code_locations?: QueueCodeLocation[]; + sast_source_location?: SastSourceLocation; +} + +interface JoinedLocations { + readonly codeLocations?: QueueCodeLocation[]; + readonly sastSourceLocation?: SastSourceLocation; +} + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === 'object' && !Array.isArray(value); +} + +function isSastSourceLocation(value: unknown): value is SastSourceLocation { + if (!isRecord(value)) return false; + return ( + typeof value.file === 'string' && + value.file.length > 0 && + Number.isInteger(value.line) && + (value.line as number) > 0 && + Number.isInteger(value.column) && + (value.column as number) >= 0 && + typeof value.rule_id === 'string' && + value.rule_id.length > 0 + ); +} + +function parseJson(contents: string, description: string): unknown { + try { + return JSON.parse(contents) as unknown; + } catch { + throw new Error(`${description} is not valid JSON`); + } +} + +async function currentReferenceMap( + deliverablesPath: string, + vulnerabilityClass: ReconciliationClass, +): Promise> { + const mapRead = await readCommittedFile(deliverablesPath, renumberMapPath(vulnerabilityClass)); + if (mapRead.state === 'absent') return new Map(); + if (mapRead.state !== 'present') throw new Error(`${vulnerabilityClass} report-reference map is unreadable`); + const decoded = parseJson(mapRead.contents, `${vulnerabilityClass} report-reference map`); + if (!isRecord(decoded) || !isRecord(decoded.map)) { + throw new Error(`${vulnerabilityClass} report-reference map is malformed`); + } + const references = new Map(); + for (const [stable, current] of Object.entries(decoded.map)) { + if (typeof current !== 'string') throw new Error(`${vulnerabilityClass} report-reference map is malformed`); + references.set(stable, current); + } + return references; } -/** Read every per-class queue in the deliverables dir into an ID-to-locations map. */ async function loadQueueLocations( deliverablesPath: string, - logger: ActivityLogger, -): Promise> { - const locations = new Map(); - - for (const vulnClass of ALL_VULN_CLASSES) { - const queuePath = path.join(deliverablesPath, `${vulnClass}_exploitation_queue.json`); - if (!(await fs.pathExists(queuePath))) continue; - - try { - const doc = (await fs.readJson(queuePath)) as { vulnerabilities?: QueueEntry[] }; - for (const entry of doc.vulnerabilities ?? []) { - if (entry.ID && entry.code_locations && entry.code_locations.length > 0) { - locations.set(entry.ID, entry.code_locations); - } - } - } catch (error) { - logger.warn(`Could not read ${vulnClass} queue for code locations: ${(error as Error).message}`); + participatingClasses: readonly ReconciliationClass[], +): Promise> { + const locations = new Map(); + for (const vulnerabilityClass of participatingClasses) { + const queueRead = await readCommittedFile(deliverablesPath, `${vulnerabilityClass}_exploitation_queue.json`); + if (queueRead.state === 'absent') continue; + if (queueRead.state !== 'present') throw new Error(`${vulnerabilityClass} queue is unreadable`); + const decoded = parseJson(queueRead.contents, `${vulnerabilityClass} queue`); + if (!isRecord(decoded) || !Array.isArray(decoded.vulnerabilities)) { + throw new Error(`${vulnerabilityClass} queue is malformed`); + } + const referenceMap = await currentReferenceMap(deliverablesPath, vulnerabilityClass); + for (const rawEntry of decoded.vulnerabilities) { + if (!isRecord(rawEntry) || typeof rawEntry.ID !== 'string') continue; + const stableReference = rawEntry.ID; + const entry = rawEntry as QueueEntry; + const bundle: JoinedLocations = { + ...(Array.isArray(entry.code_locations) && entry.code_locations.length > 0 + ? { codeLocations: entry.code_locations } + : {}), + ...(isSastSourceLocation(entry.sast_source_location) ? { sastSourceLocation: entry.sast_source_location } : {}), + }; + if (bundle.codeLocations === undefined && bundle.sastSourceLocation === undefined) continue; + locations.set(stableReference, bundle); + const currentReference = referenceMap.get(stableReference); + if (currentReference !== undefined) locations.set(currentReference, bundle); } } - return locations; } -/** - * Return the findings with `code_locations` filled in from the queue. - * - * A finding with no matching queue entry keeps none — the join never invents one. Findings are - * copied rather than mutated so the collector's own state stays untouched. - */ +/** Attach the two independent location fields without consulting exploit-inspection locations. */ export async function attachQueueCodeLocations( findings: readonly AddFindingInput[], deliverablesPath: string, logger: ActivityLogger, + participatingClasses: readonly ReconciliationClass[] = ALL_VULN_CLASSES, ): Promise { - const byId = await loadQueueLocations(deliverablesPath, logger); - if (byId.size === 0) return [...findings]; - - let matched = 0; + const byReference = await loadQueueLocations(deliverablesPath, participatingClasses); + let analysisMatches = 0; + let sastMatches = 0; const joined = findings.map((finding) => { - const locations = byId.get(finding.finding_id); - if (!locations) return finding; - matched += 1; - return { ...finding, code_locations: locations }; + const locations = byReference.get(finding.finding_id); + if (locations === undefined) return finding; + if (locations.codeLocations !== undefined) analysisMatches++; + if (locations.sastSourceLocation !== undefined) sastMatches++; + return { + ...finding, + ...(locations.codeLocations !== undefined && { code_locations: locations.codeLocations }), + ...(locations.sastSourceLocation !== undefined && { sast_source_location: locations.sastSourceLocation }), + }; + }); + logger.info('Attached committed report-task locations', { + findings: findings.length, + analysisMatches, + sastMatches, }); - - logger.info(`Attached code locations to ${matched}/${findings.length} finding(s) from the vuln queues`); return joined; } diff --git a/apps/worker/src/services/compaction-core.ts b/apps/worker/src/services/compaction-core.ts new file mode 100644 index 00000000..4bd0e775 --- /dev/null +++ b/apps/worker/src/services/compaction-core.ts @@ -0,0 +1,366 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Deterministic post-report compaction over one coherent mixed reference set. */ + +import { REF_PREFIX } from '../ai/reconciliation/refs.js'; +import type { AddExploitInput } from '../collectors/exploit-collector.js'; +import type { AddFindingInput } from '../collectors/finding-collector.js'; +import type { ActivityLogger } from '../types/activity-logger.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; +import type { ExactOutputCommit, ExactOutputFile } from './exact-output-commit.js'; +import { RenumberError, writeAndCommitExactFiles } from './exact-output-commit.js'; +import { readCommittedFile } from './git-manager.js'; +import type { ExcludedEntry, SastProvenanceFile } from './renumber-core.js'; +import { + pad2, + parseRefNumber, + remapSastProvenance, + remapTaskReferences, + renumberedExploitCollectorPath, + renumberedSastProvenancePath, + renumberMapPath, +} from './renumber-core.js'; +import type { ReportData } from './report-renderer.js'; + +const PREFIXES_LONGEST_FIRST = (Object.entries(REF_PREFIX) as [ReconciliationClass, string][]) + .map(([vulnerabilityClass, prefix]) => ({ vulnerabilityClass, prefix })) + .sort((first, second) => second.prefix.length - first.prefix.length); + +export function vulnerabilityClassOfReference(reference: string): ReconciliationClass | null { + for (const { vulnerabilityClass, prefix } of PREFIXES_LONGEST_FIRST) { + if (reference.startsWith(`${prefix}-`) && parseRefNumber(reference, vulnerabilityClass) !== null) { + return vulnerabilityClass; + } + } + return null; +} + +export interface RenumberMapFile { + readonly vulnerability_type: ReconciliationClass; + readonly map: Record; + readonly order: readonly string[]; + readonly excluded: readonly ExcludedEntry[]; +} + +export interface ClassCompaction { + readonly vulnerabilityClass: ReconciliationClass; + readonly gapMap: ReadonlyMap; + readonly composedMap: ReadonlyMap; + readonly excluded: readonly ExcludedEntry[]; + readonly renumberMapFile: RenumberMapFile; +} + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === 'object' && !Array.isArray(value); +} + +function isRenumberMapFile(value: unknown, vulnerabilityClass: ReconciliationClass): value is RenumberMapFile { + if (!isRecord(value) || value.vulnerability_type !== vulnerabilityClass) return false; + if (!isRecord(value.map) || !Array.isArray(value.order) || !Array.isArray(value.excluded)) return false; + const map = value.map; + const entries = Object.entries(map); + if ( + !entries.every( + ([stable, dense]) => + parseRefNumber(stable, vulnerabilityClass) !== null && + typeof dense === 'string' && + parseRefNumber(dense, vulnerabilityClass) !== null, + ) + ) { + return false; + } + if (new Set(entries.map(([, dense]) => dense)).size !== entries.length) return false; + if ( + value.order.length !== entries.length || + new Set(value.order).size !== value.order.length || + !value.order.every((stable) => typeof stable === 'string' && Object.hasOwn(map, stable)) + ) { + return false; + } + return value.excluded.every((entry) => { + if (!isRecord(entry)) return false; + return ( + typeof entry.source_ref === 'string' && + parseRefNumber(entry.source_ref, vulnerabilityClass) !== null && + entry.reason === 'validation_blocked' + ); + }); +} + +export function buildClassCompaction( + vulnerabilityClass: ReconciliationClass, + keptReferences: readonly string[], + renumberMap: RenumberMapFile, +): ClassCompaction { + if (!isRenumberMapFile(renumberMap, vulnerabilityClass)) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-map-malformed' }); + } + const mintedReferences = new Set(Object.values(renumberMap.map)); + const uniqueKept = [...new Set(keptReferences)]; + for (const reference of uniqueKept) { + if (!mintedReferences.has(reference)) { + throw new RenumberError('unmappable-survivor', false, { + checkCode: 'compaction-kept-reference-not-minted', + vulnerabilityClass, + }); + } + } + uniqueKept.sort( + (first, second) => + (parseRefNumber(first, vulnerabilityClass) as number) - (parseRefNumber(second, vulnerabilityClass) as number), + ); + + const gapMap = new Map(); + for (const [index, reference] of uniqueKept.entries()) { + gapMap.set(reference, `${REF_PREFIX[vulnerabilityClass]}-${pad2(index + 1)}`); + } + + const stableByDense = new Map(); + for (const [stable, dense] of Object.entries(renumberMap.map)) stableByDense.set(dense, stable); + const composedMap = new Map(); + const order: string[] = []; + for (const dense of uniqueKept) { + const stable = stableByDense.get(dense); + if (stable === undefined) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-reverse-map-missing' }); + } + composedMap.set(stable, gapMap.get(dense) as string); + order.push(stable); + } + return { + vulnerabilityClass, + gapMap, + composedMap, + excluded: renumberMap.excluded, + renumberMapFile: { + vulnerability_type: vulnerabilityClass, + map: Object.fromEntries(composedMap), + order, + excluded: renumberMap.excluded, + }, + }; +} + +export function deepRemapStrings(value: unknown, gapMap: ReadonlyMap): unknown { + if (gapMap.size === 0) return value; + if (typeof value === 'string') return remapTaskReferences(value, gapMap); + if (Array.isArray(value)) return value.map((entry) => deepRemapStrings(entry, gapMap)); + if (value !== null && typeof value === 'object') { + const remapped: Record = {}; + for (const [key, entry] of Object.entries(value)) remapped[key] = deepRemapStrings(entry, gapMap); + return remapped; + } + return value; +} + +export function remapExploitCollector( + entries: readonly AddExploitInput[], + classGapMap: ReadonlyMap, + allGapMap: ReadonlyMap, +): AddExploitInput[] { + const remapped: AddExploitInput[] = []; + for (const entry of entries) { + const reference = (entry as unknown as { vulnerability_id?: unknown }).vulnerability_id; + if (typeof reference !== 'string') { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-collector-reference-missing' }); + } + const gapless = classGapMap.get(reference); + if (gapless === undefined) continue; + const rewritten = deepRemapStrings(entry, allGapMap) as Record; + remapped.push({ ...rewritten, vulnerability_id: gapless } as unknown as AddExploitInput); + } + return remapped; +} + +/** Findings from excluded failed classes are returned by identity, including their cross-references. */ +export function remapReportFindings( + findings: readonly AddFindingInput[], + allGapMap: ReadonlyMap, + excludedClasses: ReadonlySet = new Set(), +): AddFindingInput[] { + return findings.map((finding) => { + const vulnerabilityClass = vulnerabilityClassOfReference(finding.finding_id); + if (vulnerabilityClass !== null && excludedClasses.has(vulnerabilityClass)) return finding; + return deepRemapStrings(finding, allGapMap) as AddFindingInput; + }); +} + +export function plannedReportReferenceOperations(exploit: boolean): readonly ('renumber' | 'compact')[] { + return exploit ? ['renumber', 'compact'] : []; +} + +function arraysEqual(first: readonly T[], second: readonly T[]): boolean { + return first.length === second.length && first.every((entry, index) => entry === second[index]); +} + +function parseJson(contents: string, checkCode: string): T { + try { + return JSON.parse(contents) as T; + } catch { + throw new RenumberError('key-set-divergence', false, { checkCode }); + } +} + +async function readRequiredCommittedJson(dir: string, relPath: string, checkCode: string): Promise { + const read = await readCommittedFile(dir, relPath); + if (read.state !== 'present') throw new RenumberError('key-set-divergence', false, { checkCode }); + return parseJson(read.contents, checkCode); +} + +function parseSastProvenance(value: unknown): SastProvenanceFile { + if (!isRecord(value) || !Array.isArray(value.entries)) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-provenance-malformed' }); + } + for (const entry of value.entries) { + if (!isRecord(entry) || typeof entry.exploit_ref !== 'string') { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-provenance-entry-malformed' }); + } + } + return value as unknown as SastProvenanceFile; +} + +export interface CompactionResult { + readonly compactedClasses: number; + readonly skipped: boolean; + readonly commit?: ExactOutputCommit; +} + +/** + * Compact every eligible participating class as one exact-path transaction. + * Failed classes are skipped before any of their collector, map, or provenance paths are read. + */ +export async function compactReportFindings(args: { + readonly deliverablesDir: string; + readonly participatingClasses: readonly ReconciliationClass[]; + readonly renumberFailedClasses: readonly ReconciliationClass[]; + readonly logger: ActivityLogger; +}): Promise { + const reportRead = await readCommittedFile(args.deliverablesDir, 'report.json'); + if (reportRead.state === 'absent') return { compactedClasses: 0, skipped: true }; + if (reportRead.state !== 'present') { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-report-unreadable' }); + } + const report = parseJson(reportRead.contents, 'compaction-report-not-json'); + if (!isRecord(report) || !Array.isArray(report.findings)) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-report-malformed' }); + } + // The failed-class set is read from two independent sources (the committed report and the + // caller's own record of what renumbering skipped); they must agree before any path is read, + // since a mismatch means compaction and the report disagree about which classes are trustworthy. + const reportFailedClasses = report.reconciliation_failed ?? []; + if (!arraysEqual(reportFailedClasses, args.renumberFailedClasses)) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-failed-class-set-mismatch' }); + } + + const participating = new Set(args.participatingClasses); + const failed = new Set(args.renumberFailedClasses); + if (participating.size !== args.participatingClasses.length || failed.size !== args.renumberFailedClasses.length) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-class-set-duplicate' }); + } + if ([...failed].some((vulnerabilityClass) => !participating.has(vulnerabilityClass))) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-failed-class-outside-scope' }); + } + + const keptByClass = new Map(); + const seenFindingReferences = new Set(); + for (const finding of report.findings) { + if (typeof finding.finding_id !== 'string' || seenFindingReferences.has(finding.finding_id)) { + throw new RenumberError('unmappable-survivor', false, { checkCode: 'compaction-report-reference-duplicate' }); + } + seenFindingReferences.add(finding.finding_id); + const vulnerabilityClass = vulnerabilityClassOfReference(finding.finding_id); + if (vulnerabilityClass === null || !participating.has(vulnerabilityClass)) continue; + const references = keptByClass.get(vulnerabilityClass) ?? []; + references.push(finding.finding_id); + keptByClass.set(vulnerabilityClass, references); + } + + const compactions: Array<{ + compaction: ClassCompaction; + collector: AddExploitInput[]; + provenance?: SastProvenanceFile; + }> = []; + for (const vulnerabilityClass of args.participatingClasses) { + if (failed.has(vulnerabilityClass)) continue; + const keptReferences = keptByClass.get(vulnerabilityClass) ?? []; + const mapRead = await readCommittedFile(args.deliverablesDir, renumberMapPath(vulnerabilityClass)); + if (mapRead.state === 'absent') { + // A class with no renumber map means renumbering never ran for it, so there is no + // stable-to-dense mapping to compact against. If the report still kept references from + // that class, the two artifacts have drifted and compaction must fail rather than guess. + if (keptReferences.length > 0) { + throw new RenumberError('unmappable-survivor', false, { checkCode: 'compaction-map-absent-for-survivor' }); + } + continue; + } + if (mapRead.state !== 'present') { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-map-unreadable' }); + } + const renumberMap = parseJson(mapRead.contents, 'compaction-map-not-json'); + const compaction = buildClassCompaction(vulnerabilityClass, keptReferences, renumberMap); + const collector = await readRequiredCommittedJson( + args.deliverablesDir, + renumberedExploitCollectorPath(vulnerabilityClass), + 'compaction-collector-unreadable', + ); + if (!Array.isArray(collector)) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-collector-malformed' }); + } + + const provenanceRead = await readCommittedFile( + args.deliverablesDir, + renumberedSastProvenancePath(vulnerabilityClass), + ); + let provenance: SastProvenanceFile | undefined; + if (provenanceRead.state === 'corrupt') { + throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-provenance-unreadable' }); + } + if (provenanceRead.state === 'present') { + provenance = parseSastProvenance(parseJson(provenanceRead.contents, 'compaction-provenance-not-json')); + } + compactions.push({ compaction, collector, ...(provenance !== undefined && { provenance }) }); + } + + if (compactions.length === 0) return { compactedClasses: 0, skipped: true }; + + const allGapMap = new Map(); + for (const { compaction } of compactions) { + for (const [source, destination] of compaction.gapMap) allGapMap.set(source, destination); + } + + const files: ExactOutputFile[] = []; + for (const { compaction, collector, provenance } of compactions) { + files.push({ + relPath: renumberedExploitCollectorPath(compaction.vulnerabilityClass), + contents: `${JSON.stringify(remapExploitCollector(collector, compaction.gapMap, allGapMap), null, 2)}\n`, + }); + files.push({ + relPath: renumberMapPath(compaction.vulnerabilityClass), + contents: `${JSON.stringify(compaction.renumberMapFile, null, 2)}\n`, + }); + if (provenance !== undefined) { + files.push({ + relPath: renumberedSastProvenancePath(compaction.vulnerabilityClass), + contents: `${JSON.stringify(remapSastProvenance(provenance, compaction.gapMap), null, 2)}\n`, + }); + } + } + + const compactedReport = deepRemapStrings(report, allGapMap) as ReportData; + const findings = remapReportFindings(report.findings, allGapMap, failed); + files.push({ + relPath: 'report.json', + contents: JSON.stringify({ ...compactedReport, findings }, null, 2), + }); + const commit = await writeAndCommitExactFiles( + args.deliverablesDir, + files, + 'Compact surviving report references to dense gapless', + args.logger, + ); + return { compactedClasses: compactions.length, skipped: false, commit }; +} diff --git a/apps/worker/src/services/exact-output-commit.ts b/apps/worker/src/services/exact-output-commit.ts new file mode 100644 index 00000000..f6c6f9a5 --- /dev/null +++ b/apps/worker/src/services/exact-output-commit.ts @@ -0,0 +1,301 @@ +// Copyright (C) 2025 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Generic exact-path, lost-acknowledgement-safe file publication over a deliverables Git repo. */ + +import { randomUUID } from 'node:crypto'; +import { lstat, mkdtemp, rename, rm, unlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; +import type { ActivityLogger } from '../types/activity-logger.js'; +import { + commitExactPaths, + executeGitCommandWithRetry, + getGitCommitHash, + pathsChangedInCommit, + readCommittedFile, + restorePathsFromHead, + withGitRepoLock, +} from './git-manager.js'; + +export type RenumberErrorType = 'unmappable-survivor' | 'key-set-divergence'; + +const RENUMBER_ERROR_MESSAGES: Readonly> = Object.freeze({ + 'unmappable-survivor': 'A report survivor could not be mapped to one canonical class reference.', + 'key-set-divergence': 'Committed report-facing class artifacts disagree on their reference set.', +}); + +export interface RenumberErrorDetails { + readonly checkCode: string; + readonly [key: string]: unknown; +} + +export class RenumberError extends Error { + readonly retryable: boolean; + readonly type: RenumberErrorType; + readonly details?: RenumberErrorDetails; + + constructor(type: RenumberErrorType, retryable: boolean, details?: RenumberErrorDetails) { + super(RENUMBER_ERROR_MESSAGES[type]); + this.name = 'RenumberError'; + this.type = type; + this.retryable = retryable; + if (details !== undefined) this.details = details; + } +} + +export interface ExactOutputFile { + readonly relPath: string; + /** `null` makes absence part of the exact output contract. */ + readonly contents: string | null; +} + +export interface ExactOutputCommit { + readonly commitHash: string; + readonly changedPaths: readonly string[]; + readonly alreadyCommitted: boolean; +} + +function samePathSet(first: readonly string[], second: readonly string[]): boolean { + return ( + first.length === second.length && + new Set(first).size === first.length && + first.every((entry) => second.includes(entry)) + ); +} + +function isErrno(error: unknown, code: string): boolean { + return error instanceof Error && (error as NodeJS.ErrnoException).code === code; +} + +async function rejectSymlinks(dir: string, relPaths: readonly string[]): Promise { + for (const relPath of relPaths) { + try { + if ((await lstat(path.join(dir, relPath))).isSymbolicLink()) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'output-symlink' }); + } + } catch (error) { + if (isErrno(error, 'ENOENT')) continue; + throw error; + } + } +} + +async function atomicWriteUnique(absolutePath: string, contents: string): Promise { + const temporaryPath = `${absolutePath}.tmp-${randomUUID()}`; + try { + await writeFile(temporaryPath, contents, { flag: 'wx' }); + await rename(temporaryPath, absolutePath); + } catch (error) { + await unlink(temporaryPath).catch(() => undefined); + throw error; + } +} + +async function executeExactGitCommand( + args: string[], + dir: string, + description: string, +): Promise<{ stdout: string; stderr: string }> { + try { + return await executeGitCommandWithRetry(args, dir, description); + } catch (error) { + throw new Error(`Exact-output Git step failed: ${description}`, { cause: error }); + } +} + +async function repairExactPathsFromHead(dir: string, relPaths: readonly string[]): Promise { + const presentPaths: string[] = []; + const absentPaths: string[] = []; + for (const relPath of relPaths) { + const committed = await readCommittedFile(dir, relPath); + if (committed.state === 'corrupt') { + throw new RenumberError('key-set-divergence', false, { checkCode: 'corrupt-output-object' }); + } + if (committed.state === 'present') presentPaths.push(relPath); + else absentPaths.push(relPath); + } + if (presentPaths.length > 0) await restorePathsFromHead(dir, presentPaths); + if (absentPaths.length === 0) return; + for (const relPath of absentPaths) { + const listed = await executeExactGitCommand( + ['git', 'ls-files', '--', relPath], + dir, + 'checking an absent exact-output index path', + ); + if (listed.stdout.trim() !== '') { + await executeExactGitCommand( + ['git', 'update-index', '--force-remove', '--', relPath], + dir, + 'clearing an absent exact-output path from the index', + ); + } + } + for (const relPath of absentPaths) { + await unlink(path.join(dir, relPath)).catch((error: unknown) => { + if (!isErrno(error, 'ENOENT')) throw error; + }); + } +} + +async function commitExactFilesWithTemporaryIndex( + dir: string, + files: readonly ExactOutputFile[], + expectedChangedPaths: readonly string[], + message: string, + logger: ActivityLogger, +): Promise<{ commitHash: string; changedPaths: string[] }> { + const head = await getGitCommitHash(dir); + if (head === null) throw new Error('Unable to read HEAD for exact-output commit'); + const temporaryDir = await mkdtemp(path.join(tmpdir(), 'shannon-exact-index-')); + const temporaryIndex = path.join(temporaryDir, 'index'); + const pathsToStage = files + .filter((file) => file.contents !== null || expectedChangedPaths.includes(file.relPath)) + .map((file) => file.relPath); + const runWithTemporaryIndex = async (gitArgs: readonly string[], description: string) => + executeExactGitCommand(['env', `GIT_INDEX_FILE=${temporaryIndex}`, 'git', ...gitArgs], dir, description); + + try { + await runWithTemporaryIndex(['read-tree', head], 'initializing an exact-output temporary index'); + await runWithTemporaryIndex(['add', '-A', '--', ...pathsToStage], 'staging exact outputs in a temporary index'); + const tree = (await runWithTemporaryIndex(['write-tree'], 'writing the exact-output tree')).stdout.trim(); + const commitHash = ( + await executeExactGitCommand( + ['git', 'commit-tree', tree, '-p', head, '-m', message], + dir, + 'creating the exact-output commit object', + ) + ).stdout.trim(); + const changedPaths = await pathsChangedInCommit(dir, commitHash); + if (!samePathSet(changedPaths, expectedChangedPaths)) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'changed-path-set-mismatch' }); + } + await executeExactGitCommand( + ['git', 'update-ref', 'HEAD', commitHash, head], + dir, + 'advancing HEAD to the exact-output commit', + ); + for (const file of files) { + if (file.contents === null) { + const listed = await executeExactGitCommand( + ['git', 'ls-files', '--', file.relPath], + dir, + 'checking an exact-output deletion in the index', + ); + if (listed.stdout.trim() !== '') { + await executeExactGitCommand( + ['git', 'update-index', '--force-remove', '--', file.relPath], + dir, + 'recording an exact-output deletion in the index', + ); + } + } else { + await executeExactGitCommand( + ['git', 'add', '--', file.relPath], + dir, + 'refreshing an exact-output path in the index', + ); + } + } + logger.info(`Path-limited commit ${commitHash.slice(0, 8)} changed ${changedPaths.length} path(s)`); + return { commitHash, changedPaths }; + } finally { + await rm(temporaryDir, { recursive: true, force: true }); + } +} + +/** Exact-path, lost-acknowledgement-safe publication used by both transforms. */ +export async function writeAndCommitExactFiles( + dir: string, + files: readonly ExactOutputFile[], + message: string, + logger: ActivityLogger, + options: { + readonly afterCommit?: (commit: { commitHash: string; changedPaths: readonly string[] }) => void | Promise; + } = {}, +): Promise { + if (files.length === 0) throw new Error('writeAndCommitExactFiles requires at least one file'); + const relPaths = files.map((file) => file.relPath); + if ( + relPaths.some( + (relPath) => + relPath.length === 0 || + path.isAbsolute(relPath) || + relPath.includes('\0') || + relPath.split(/[\\/]/).some((segment) => segment === '' || segment === '.' || segment === '..'), + ) + ) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'unsafe-output-path' }); + } + if (new Set(relPaths).size !== relPaths.length) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'duplicate-output-path' }); + } + + return withGitRepoLock(async () => { + await rejectSymlinks(dir, relPaths); + const expectedChangedPaths: string[] = []; + let allCommitted = true; + for (const file of files) { + const committed = await readCommittedFile(dir, file.relPath); + if (committed.state === 'corrupt') { + throw new RenumberError('key-set-divergence', false, { checkCode: 'corrupt-output-object' }); + } + const matches = + file.contents === null + ? committed.state === 'absent' + : committed.state === 'present' && committed.contents === file.contents; + if (!matches) { + allCommitted = false; + expectedChangedPaths.push(file.relPath); + } + } + + if (allCommitted) { + await repairExactPathsFromHead(dir, relPaths); + const commitHash = await getGitCommitHash(dir); + if (commitHash === null) throw new Error('Unable to read the existing exact-output commit'); + return { commitHash, changedPaths: [], alreadyCommitted: true }; + } + + try { + for (const file of files) { + const absolutePath = path.join(dir, file.relPath); + if (file.contents === null) { + await unlink(absolutePath).catch((error: unknown) => { + if (!isErrno(error, 'ENOENT')) throw error; + }); + } else { + await atomicWriteUnique(absolutePath, file.contents); + } + } + const committed = files.some((file) => file.contents === null) + ? await commitExactFilesWithTemporaryIndex(dir, files, expectedChangedPaths, message, logger) + : await commitExactPaths(dir, relPaths, message, logger); + if (!samePathSet(committed.changedPaths, expectedChangedPaths)) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'changed-path-set-mismatch' }); + } + await options.afterCommit?.(committed); + for (const file of files) { + const verified = await readCommittedFile(dir, file.relPath); + const matches = + file.contents === null + ? verified.state === 'absent' + : verified.state === 'present' && verified.contents === file.contents; + if (!matches) { + throw new RenumberError('key-set-divergence', false, { checkCode: 'committed-byte-mismatch' }); + } + } + return { ...committed, alreadyCommitted: false }; + } catch (error) { + await repairExactPathsFromHead(dir, relPaths).catch((cleanupError: unknown) => { + logger.error('Exact-output rollback failed', { + error: cleanupError instanceof Error ? cleanupError.message : String(cleanupError), + }); + }); + throw error; + } + }); +} diff --git a/apps/worker/src/services/exploit-renderer.ts b/apps/worker/src/services/exploit-renderer.ts index 8d52c320..47b2e174 100644 --- a/apps/worker/src/services/exploit-renderer.ts +++ b/apps/worker/src/services/exploit-renderer.ts @@ -8,7 +8,7 @@ * Deterministic exploit collector → markdown renderer. * * Single entry point renderExploitDeliverable(vulnClass, state, idToType) - * covers all exploitation agents. The + * covers every exploitation agent, including the conditional `miscellaneous` class. The * per-class deltas are limited to title and ID prefix; every section, label, * and sort rule is class-agnostic. Section headers and bolded field labels * mirror the prescribed-Markdown skeleton from the existing exploit-*.txt diff --git a/apps/worker/src/services/finding-order.ts b/apps/worker/src/services/finding-order.ts new file mode 100644 index 00000000..bd7b790b --- /dev/null +++ b/apps/worker/src/services/finding-order.ts @@ -0,0 +1,90 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Shared ordering for structured findings and every derived report surface. */ + +const UNRANKED = Number.MAX_SAFE_INTEGER; + +export const SEVERITY_RANK: Readonly> = Object.freeze({ + critical: 0, + high: 1, + medium: 2, + low: 3, +}); + +export const CONFIDENCE_RANK: Readonly> = Object.freeze({ + high: 0, + medium: 1, + low: 2, +}); + +/** Recognized report categories. Unrecognized values form a sentinel group after `Other`. */ +export const CATEGORY_ORDER = ['Injection', 'XSS', 'Authentication', 'SSRF', 'Authorization', 'Other'] as const; + +export function severityRank(severity: string | null | undefined): number { + if (severity == null) return UNRANKED; + return SEVERITY_RANK[severity] ?? UNRANKED; +} + +export function confidenceRank(confidence: string | null | undefined): number { + if (confidence == null) return UNRANKED; + return CONFIDENCE_RANK[confidence] ?? UNRANKED; +} + +export function categoryRank(category: string): number { + const index = CATEGORY_ORDER.indexOf(category as (typeof CATEGORY_ORDER)[number]); + return index === -1 ? UNRANKED : index; +} + +/** Recognized categories first; unknown sentinel categories follow in lexical order. */ +export function compareCategories(first: string, second: string): number { + const firstRank = categoryRank(first); + const secondRank = categoryRank(second); + if (firstRank !== secondRank) return firstRank - secondRank; + if (firstRank !== UNRANKED || first === second) return 0; + return first < second ? -1 : 1; +} + +export function trailingRefNumber(reference: string): number | null { + const match = /(\d+)$/.exec(reference); + if (match === null) return null; + const parsed = Number.parseInt(match[1] as string, 10); + return Number.isFinite(parsed) ? parsed : null; +} + +/** Numeric suffix first, then the full reference as the deterministic fallback. */ +export function compareRef(first: string, second: string): number { + const firstNumber = trailingRefNumber(first); + const secondNumber = trailingRefNumber(second); + if (firstNumber !== secondNumber) { + if (firstNumber === null) return 1; + if (secondNumber === null) return -1; + return firstNumber - secondNumber; + } + if (first < second) return -1; + if (first > second) return 1; + return 0; +} + +export interface FindingOrderFields { + readonly category: string; + readonly severity?: string | null; + readonly finding_id: string; +} + +/** Category, severity, numeric suffix, then full-reference fallback. */ +export function compareFindings(first: FindingOrderFields, second: FindingOrderFields): number { + const categoryDifference = compareCategories(first.category, second.category); + if (categoryDifference !== 0) return categoryDifference; + + const severityDifference = severityRank(first.severity) - severityRank(second.severity); + if (severityDifference !== 0) return severityDifference; + return compareRef(first.finding_id, second.finding_id); +} + +export function orderFindings(findings: readonly T[]): T[] { + return [...findings].sort(compareFindings); +} diff --git a/apps/worker/src/services/findings-renderer.ts b/apps/worker/src/services/findings-renderer.ts index be1dbd18..60f1c899 100644 --- a/apps/worker/src/services/findings-renderer.ts +++ b/apps/worker/src/services/findings-renderer.ts @@ -17,10 +17,18 @@ */ import { fs, path } from 'zx'; -import type { AuthFinding, AuthzFinding, InjectionFinding, SsrfFinding, XssFinding } from '../ai/queue-schemas.js'; +import type { + AuthFinding, + AuthzFinding, + InjectionFinding, + MiscellaneousFinding, + SsrfFinding, + XssFinding, +} from '../ai/queue-schemas.js'; import { deliverablesDir } from '../paths.js'; import type { ActivityLogger } from '../types/activity-logger.js'; -import type { VulnClass } from '../types/config.js'; +import { ALL_VULN_CLASSES } from '../types/config.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; const DISCLAIMER = [ '> Exploitation phase was not run for this assessment. Each entry documents a', @@ -37,7 +45,11 @@ interface ClassConfig { } interface QueueDocument { - vulnerabilities?: T[]; + readonly vulnerabilities: readonly T[]; +} + +export interface RenderFindingsResult { + readonly failedClasses: readonly ReconciliationClass[]; } // === Common Render Helpers === @@ -48,6 +60,17 @@ function summaryRow(label: string, value: string | undefined | null | boolean): return `- **${label}:** ${value}`; } +function parseQueueDocument(value: unknown): QueueDocument { + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + throw new Error('queue document is malformed'); + } + const vulnerabilities = (value as Record).vulnerabilities; + if (!Array.isArray(vulnerabilities)) { + throw new Error('queue document vulnerabilities are malformed'); + } + return { vulnerabilities }; +} + function formatLocation(endpoint: string | undefined, codeLocation: string | undefined): string { if (endpoint && codeLocation) return `${endpoint} (${codeLocation})`; return endpoint ?? codeLocation ?? ''; @@ -110,6 +133,20 @@ function renderSsrfEntry(e: SsrfFinding): string { ); } +function renderMiscellaneousEntry(e: MiscellaneousFinding): string { + return buildEntry( + e.ID, + e.vulnerability_type, + { confidence: e.confidence }, + [ + summaryRow('Vulnerable location', formatLocation(e.source_endpoint, e.vulnerable_code_location)), + summaryRow('Overview', e.missing_defense), + summaryRow('Impact', e.exploitation_hypothesis), + ], + e.notes, + ); +} + function renderAuthzEntry(e: AuthzFinding): string { return buildEntry( e.ID, @@ -148,7 +185,7 @@ function renderXssEntry(e: XssFinding): string { // === Class Registry === -const CLASSES: Record> = { +const CLASSES: Record> = { auth: { heading: 'Authentication', noneFoundLabel: 'authentication', @@ -184,6 +221,13 @@ const CLASSES: Record> = { findingsFile: 'ssrf_findings.md', renderEntry: (e) => renderSsrfEntry(e as SsrfFinding), }, + miscellaneous: { + heading: 'Miscellaneous', + noneFoundLabel: 'miscellaneous', + queueFile: 'miscellaneous_exploitation_queue.json', + findingsFile: 'miscellaneous_findings.md', + renderEntry: (e) => renderMiscellaneousEntry(e as MiscellaneousFinding), + }, }; // === Class File Assembly === @@ -213,39 +257,40 @@ function renderClassFile(config: ClassConfig, entries: readonly unknown /** * Render `*_findings.md` per class from each `*_exploitation_queue.json`. * - * Idempotent: skips classes whose findings file already exists, or whose queue - * is missing (class out of scope this run). Per-class failures are logged and - * other classes still proceed. + * Idempotent: rewrites each present class from its queue; a missing queue means the class was out of + * scope. Per-class failures are logged and other classes still proceed. */ export async function renderFindingsFromQueues( sourceDir: string, deliverablesSubdir: string | undefined, logger: ActivityLogger, -): Promise { + participatingClasses: readonly ReconciliationClass[] = ALL_VULN_CLASSES, +): Promise { const dir = deliverablesDir(sourceDir, deliverablesSubdir); + const failedClasses: ReconciliationClass[] = []; - for (const config of Object.values(CLASSES)) { + for (const vulnerabilityClass of participatingClasses) { + const config = CLASSES[vulnerabilityClass]; const queuePath = path.join(dir, config.queueFile); const findingsPath = path.join(dir, config.findingsFile); - if (await fs.pathExists(findingsPath)) { - logger.info(`${config.heading}: ${config.findingsFile} already exists, skipping`); - continue; - } if (!(await fs.pathExists(queuePath))) { logger.info(`${config.heading}: no queue file (class out of scope), skipping`); continue; } try { - const doc = (await fs.readJson(queuePath)) as QueueDocument; - const entries = doc.vulnerabilities ?? []; + const doc = parseQueueDocument(await fs.readJson(queuePath)); + const entries = doc.vulnerabilities; const markdown = renderClassFile(config, entries); await fs.writeFile(findingsPath, markdown); logger.info(`${config.heading}: rendered ${entries.length} finding(s) to ${config.findingsFile}`); } catch (error) { const err = error as Error; + failedClasses.push(vulnerabilityClass); logger.warn(`${config.heading}: failed to render findings from ${config.queueFile}: ${err.message}`); } } + + return { failedClasses }; } diff --git a/apps/worker/src/services/index.ts b/apps/worker/src/services/index.ts index 9b6e6db6..066be1d2 100644 --- a/apps/worker/src/services/index.ts +++ b/apps/worker/src/services/index.ts @@ -19,7 +19,22 @@ export { ConfigLoaderService } from './config-loader.js'; export type { ContainerDependencies } from './container.js'; export { Container, getContainer, getOrCreateContainer, removeContainer, setContainerFactory } from './container.js'; export { ExploitationCheckerService } from './exploitation-checker.js'; +export type { CommittedReadResult } from './git-manager.js'; +export { + blobShaFromHead, + classifyHeadReadFailure, + commitExactPaths, + getGitCommitHash, + isAncestor, + parsePorcelainZ, + pathsChangedInCommit, + readCommittedFile, + readFileFromHead, + restorePathsFromHead, + rollbackGitWorkspace, + withGitRepoLock, +} from './git-manager.js'; export { loadPrompt } from './prompt-manager.js'; export type { ReportData, ReportMeta } from './report-renderer.js'; export { renderReport } from './report-renderer.js'; -export { assembleFinalReport, copyReportToRunRoot, injectModelIntoReport } from './reporting.js'; +export { assembleFinalReport, copyReportToRunRoot } from './reporting.js'; diff --git a/apps/worker/src/services/pdf-renderer.ts b/apps/worker/src/services/pdf-renderer.ts index 71442318..4ab42d2f 100644 --- a/apps/worker/src/services/pdf-renderer.ts +++ b/apps/worker/src/services/pdf-renderer.ts @@ -17,8 +17,9 @@ */ import { execFile } from 'node:child_process'; +import { createHash, randomUUID } from 'node:crypto'; import { existsSync } from 'node:fs'; -import { copyFile, cp, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { copyFile, cp, mkdir, mkdtemp, readFile, rename, rm, unlink, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import path from 'node:path'; import { promisify } from 'node:util'; @@ -34,6 +35,15 @@ const DATA_FILENAME = 'data.json'; const TEMPLATE_FILENAME = 'report.typ'; const OUTPUT_FILENAME = 'report.pdf'; +export const PDF_RENDERER_VERSION = '1'; + +export interface PdfProvenance { + readonly pdf_sha256: string; + readonly canonical_report_sha256: string; + readonly renderer_version: string; + readonly template_version: string; +} + export interface RenderReportPdfOptions { /** Structured report data (report.json contents), pre-assembly. */ readonly reportData: ReportData; @@ -47,6 +57,86 @@ export interface RenderReportPdfOptions { readonly brand?: string; } +function sha256(contents: Uint8Array): string { + return createHash('sha256').update(contents).digest('hex'); +} + +function isSha256(value: unknown): value is string { + return typeof value === 'string' && /^[a-f0-9]{64}$/.test(value); +} + +/** Validate the closed durable provenance shape used for PDF reuse. */ +export function isPdfProvenance(value: unknown): value is PdfProvenance { + if (value === null || typeof value !== 'object' || Array.isArray(value)) return false; + const candidate = value as Record; + return ( + Object.keys(candidate).length === 4 && + isSha256(candidate.pdf_sha256) && + isSha256(candidate.canonical_report_sha256) && + typeof candidate.renderer_version === 'string' && + candidate.renderer_version.length > 0 && + typeof candidate.template_version === 'string' && + candidate.template_version.length > 0 + ); +} + +/** Build provenance only after the renderer has atomically published verified PDF bytes. */ +export async function readPdfProvenance(args: { + readonly pdfPath: string; + readonly canonicalReportSha256: string; + readonly templatePath: string; +}): Promise { + const [pdfBytes, templateBytes] = await Promise.all([readFile(args.pdfPath), readFile(args.templatePath)]); + return { + pdf_sha256: sha256(pdfBytes), + canonical_report_sha256: args.canonicalReportSha256, + renderer_version: PDF_RENDERER_VERSION, + template_version: sha256(templateBytes), + }; +} + +/** Recompute the PDF digest before trusting persisted provenance for the current report. */ +export async function pdfMatchesProvenance(args: { + readonly pdfPath: string; + readonly canonicalReportSha256: string; + readonly provenance: PdfProvenance; +}): Promise { + if (!isPdfProvenance(args.provenance) || args.provenance.canonical_report_sha256 !== args.canonicalReportSha256) { + return false; + } + try { + return sha256(await readFile(args.pdfPath)) === args.provenance.pdf_sha256; + } catch { + return false; + } +} + +/** + * Full currency check: the PDF is current only when its bytes, canonical report digest, + * renderer version, and template version all match the provenance record. A record from an + * older renderer or template is stale even when it is internally consistent. + */ +export async function pdfProvenanceIsCurrent(args: { + readonly pdfPath: string; + readonly canonicalReportSha256: string; + readonly provenance: PdfProvenance; + readonly templatePath: string; +}): Promise { + if (args.provenance.renderer_version !== PDF_RENDERER_VERSION) return false; + let templateSha256: string; + try { + templateSha256 = sha256(await readFile(args.templatePath)); + } catch { + return false; + } + if (args.provenance.template_version !== templateSha256) return false; + return pdfMatchesProvenance({ + pdfPath: args.pdfPath, + canonicalReportSha256: args.canonicalReportSha256, + provenance: args.provenance, + }); +} + /** * Compile the report to a PDF at `outputPath`. * @@ -91,7 +181,14 @@ export async function renderReportPdf(options: RenderReportPdfOptions): Promise< ]); await mkdir(path.dirname(outputPath), { recursive: true }); - await copyFile(pdfInWorkDir, outputPath); + const outputAttemptPath = `${outputPath}.tmp-${randomUUID()}`; + try { + await copyFile(pdfInWorkDir, outputAttemptPath); + await rename(outputAttemptPath, outputPath); + } catch (error) { + await unlink(outputAttemptPath).catch(() => undefined); + throw error; + } } finally { await rm(workDir, { recursive: true, force: true }); } diff --git a/apps/worker/src/services/prompt-manager.ts b/apps/worker/src/services/prompt-manager.ts index f5c1ef82..eb46d932 100644 --- a/apps/worker/src/services/prompt-manager.ts +++ b/apps/worker/src/services/prompt-manager.ts @@ -9,6 +9,7 @@ import { PROMPTS_DIR } from '../paths.js'; import { PLAYWRIGHT_SESSION_MAPPING } from '../session-manager.js'; import type { ActivityLogger } from '../types/activity-logger.js'; import type { Authentication, DistributedConfig, DistributedReportConfig, Rule, VulnClass } from '../types/config.js'; +import { assertFixedAnalysisScope } from '../types/run-state.js'; import { isGlobPattern } from '../utils/glob.js'; import { handlePromptError, PentestError } from './error-handling.js'; @@ -140,6 +141,8 @@ interface PromptVariables { repoPath: string; /** Classes whose analysis did not complete, so the report can mark them not assessed. */ failedClasses?: readonly VulnClass[]; + /** Explicit workflow-owned analysis scope for prompts that describe tested classes. */ + analysisClasses?: readonly VulnClass[]; AUTH_STATE_FILE: string; PLAYWRIGHT_SESSION?: string; } @@ -380,12 +383,15 @@ async function interpolateVariables( result = result.replace(/{{LOGIN_INSTRUCTIONS}}/g, ''); } - const vulnClasses = config?.vuln_classes ?? []; - result = replaceLiteral( - result, - /{{VULN_CLASSES_TESTED}}/g, - vulnClasses.length > 0 ? vulnClasses.join(', ') : 'injection, xss, auth, authz, ssrf', - ); + if (result.includes('{{VULN_CLASSES_TESTED}}')) { + if (variables.analysisClasses === undefined) { + throw new PentestError('Prompt requires an explicit workflow-owned analysis scope', 'prompt', false, { + placeholder: 'VULN_CLASSES_TESTED', + }); + } + assertFixedAnalysisScope(variables.analysisClasses); + result = replaceLiteral(result, /{{VULN_CLASSES_TESTED}}/g, variables.analysisClasses.join(', ')); + } result = replaceLiteral( result, /{{NOT_ASSESSED_CLASSES}}/g, @@ -443,6 +449,14 @@ async function interpolateVariables( } } +// Prompt families that drive deterministic, model-only stages with no browser of their own. +// They share loadPrompt with the browser agents but must never claim a Playwright session. +const NON_BROWSER_PROMPT_PREFIXES: readonly string[] = Object.freeze(['task-formation-', 'sast-enrichment-']); + +function isNonBrowserPrompt(promptName: string): boolean { + return NON_BROWSER_PROMPT_PREFIXES.some((prefix) => promptName.startsWith(prefix)); +} + // Resolve promptDir override against SHANNON_WORKER_ROOT so relative paths // from callers stay cwd-independent. function resolvePromptDir(promptDir: string | undefined): string { @@ -480,7 +494,9 @@ export async function loadPrompt( if (session) { enhancedVariables.PLAYWRIGHT_SESSION = session; logger.info(`Assigned ${promptName} -> ${enhancedVariables.PLAYWRIGHT_SESSION}`); - } else { + } else if (!isNonBrowserPrompt(promptName)) { + // A browser agent missing from the table is a real gap; a non-browser family is not, so it + // takes neither the fallback session nor the warning. enhancedVariables.PLAYWRIGHT_SESSION = 'agent1'; logger.warn(`Unknown agent ${promptName}, using fallback -> ${enhancedVariables.PLAYWRIGHT_SESSION}`); } diff --git a/apps/worker/src/services/queue-validation.ts b/apps/worker/src/services/queue-validation.ts index 8362c56a..4fa342ce 100644 --- a/apps/worker/src/services/queue-validation.ts +++ b/apps/worker/src/services/queue-validation.ts @@ -10,6 +10,7 @@ import type { ExploitationDecision } from '../types/agents.js'; import { ErrorCode } from '../types/errors.js'; import type { ReconciliationClass } from '../types/reconciliation.js'; import { err, ok, type Result } from '../types/result.js'; +import { renderSafeMessage } from '../types/run-state.js'; import { asyncPipe } from '../utils/functional.js'; import { PentestError } from './error-handling.js'; @@ -39,6 +40,7 @@ interface FileExistence { interface ExistenceContext { existence: FileExistence; deliverableRequired: boolean; + vulnerabilityClass: ReconciliationClass; } interface PathsBase { @@ -145,20 +147,21 @@ const fileExistenceRules: readonly ValidationRule[] = Object.freeze([ ), ]); -// Generate appropriate error message based on which files are missing -function getExistenceErrorMessage({ existence, deliverableRequired }: ExistenceContext): string { - const { deliverableExists, queueExists } = existence; +const NO_RESULTS_MESSAGE = + '{Class} analysis did not produce results. Re-running this workspace retries just that class.'; +const PARTIAL_RESULTS_MESSAGE = + '{Class} analysis produced only part of its results, so it could not be exploited. Re-running this workspace retries just that class.'; - if (!deliverableRequired) { - return 'Analysis failed: Queue file missing. A queue is required.'; - } - if (!deliverableExists && !queueExists) { - return 'Analysis failed: Neither deliverable nor queue file exists. Both are required.'; - } - if (!queueExists) { - return 'Analysis incomplete: Deliverable exists but queue file missing. Both are required.'; - } - return 'Analysis incomplete: Queue exists but deliverable file missing. Both are required.'; +/** + * Name the outcome the reader can act on rather than the files behind it: nothing landed at + * all, or only some of what the class owes. The analysis-less `other` class has no + * deliverable, so its queue alone decides which of the two applies. + */ +function getExistenceErrorMessage({ existence, deliverableRequired, vulnerabilityClass }: ExistenceContext): string { + const { deliverableExists, queueExists } = existence; + const nothingProduced = deliverableRequired ? !deliverableExists && !queueExists : !queueExists; + const template = nothingProduced ? NO_RESULTS_MESSAGE : PARTIAL_RESULTS_MESSAGE; + return renderSafeMessage(template, { vulnerabilityClass }); } // Pure function to create file paths @@ -214,7 +217,7 @@ const validateExistenceRules = ( const { existence, vulnType } = pathsWithExistence; const { deliverableRequired } = VULN_TYPE_CONFIG[vulnType]; - const context: ExistenceContext = { existence, deliverableRequired }; + const context: ExistenceContext = { existence, deliverableRequired, vulnerabilityClass: vulnType }; // Find the first rule that fails const failedRule = fileExistenceRules.find((rule) => !rule.predicate(context)); @@ -225,7 +228,7 @@ const validateExistenceRules = ( return { error: new PentestError( - `${message} (${vulnType})`, + message, 'validation', failedRule.retryable, { diff --git a/apps/worker/src/services/renumber-core.ts b/apps/worker/src/services/renumber-core.ts new file mode 100644 index 00000000..41fcfffe --- /dev/null +++ b/apps/worker/src/services/renumber-core.ts @@ -0,0 +1,417 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Deterministic exploitative renumbering and exact-path class publication. */ + +import { createHash } from 'node:crypto'; +import { readPublishedManifest } from '../ai/reconciliation/manifest.js'; +import { sastProvenancePath } from '../ai/reconciliation/prepare.js'; +import { isProducerId, REF_PREFIX } from '../ai/reconciliation/refs.js'; +import type { AddExploitInput, ExploitedExploit } from '../collectors/exploit-collector.js'; +import type { ActivityLogger } from '../types/activity-logger.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; +import type { ExactOutputCommit, ExactOutputFile } from './exact-output-commit.js'; +import { RenumberError, writeAndCommitExactFiles } from './exact-output-commit.js'; +import { renderExploitDeliverable } from './exploit-renderer.js'; +import { severityRank } from './finding-order.js'; +import { readCommittedFile } from './git-manager.js'; + +function divergence(checkCode: string, vulnerabilityClass: ReconciliationClass): never { + throw new RenumberError('key-set-divergence', false, { checkCode, vulnerabilityClass }); +} + +export function pad2(value: number): string { + return String(value).padStart(2, '0'); +} + +function escapeForRegExp(value: string): string { + return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +// Internal producer-task tokens (VULN/SAST) that survive the reference remap have no meaning in +// customer-facing output, so scrubbing replaces them with neutral wording. This is the boundary +// between the internal reconciliation bookkeeping and the exact-path output an exploit agent and +// the final report both read: without this scrub, a producer/task tag would leak into text a +// downstream agent or the customer report can see, exposing reconciliation internals that should +// stay invisible outside this pipeline stage. Prefixes are tried longest first so one class +// prefix cannot match inside a longer sibling prefix. +const PRODUCER_TOKEN_PATTERN = new RegExp( + `(?:${Object.values(REF_PREFIX) + .slice() + .sort((first, second) => second.length - first.length) + .map(escapeForRegExp) + .join('|')})-(?:VULN|SAST)-\\d+`, + 'g', +); + +export function remapTaskReferences(text: string, oldToNew: ReadonlyMap): string { + const oldReferences = [...oldToNew.keys()].sort((first, second) => second.length - first.length); + if (oldReferences.length === 0) return text; + const alternation = oldReferences.map(escapeForRegExp).join('|'); + const referencePattern = new RegExp(`(?:${alternation})(?![0-9])`, 'g'); + return text.replace(referencePattern, (oldReference) => oldToNew.get(oldReference) as string); +} + +// Walks an entire exploit entry (nested objects and arrays included) so the reference remap and +// producer-token scrub apply to every string field, not just the ones a caller happens to check. +function scrubEntryText(value: unknown, oldToNew: ReadonlyMap): unknown { + if (typeof value === 'string') { + return remapTaskReferences(value, oldToNew).replace(PRODUCER_TOKEN_PATTERN, 'a related finding'); + } + if (Array.isArray(value)) return value.map((entry) => scrubEntryText(entry, oldToNew)); + if (value !== null && typeof value === 'object') { + const scrubbed: Record = {}; + for (const [key, entry] of Object.entries(value)) scrubbed[key] = scrubEntryText(entry, oldToNew); + return scrubbed; + } + return value; +} + +export function parseRefNumber(reference: string, vulnerabilityClass: ReconciliationClass): number | null { + const prefix = REF_PREFIX[vulnerabilityClass]; + const match = new RegExp(`^${escapeForRegExp(prefix)}-(\\d+)$`).exec(reference); + if (match === null) return null; + const digits = match[1] as string; + const parsed = Number.parseInt(digits, 10); + if (!Number.isFinite(parsed) || parsed < 1 || digits !== pad2(parsed)) return null; + return parsed; +} + +export type ExclusionReason = 'validation_blocked'; + +export interface ExcludedEntry { + readonly source_ref: string; + readonly reason: ExclusionReason; +} + +export interface RenumberMap { + readonly oldToNew: Map; + readonly renumbered: AddExploitInput[]; + readonly order: string[]; + readonly excluded: ExcludedEntry[]; +} + +/** Validate the complete collector before partitioning and densely order only exploited survivors. */ +export function buildRenumberMap( + entries: readonly AddExploitInput[], + vulnerabilityClass: ReconciliationClass, +): RenumberMap { + const decorated = entries.map((entry) => { + const record = entry as unknown as Record; + if (entry === null || typeof entry !== 'object' || typeof record.vulnerability_id !== 'string') { + throw new RenumberError('unmappable-survivor', false); + } + const numericReference = parseRefNumber(record.vulnerability_id, vulnerabilityClass); + if (numericReference === null || (record.status !== 'exploited' && record.status !== 'blocked')) { + throw new RenumberError('unmappable-survivor', false); + } + return { + entry, + numericReference, + oldReference: record.vulnerability_id, + status: record.status, + }; + }); + + const seen = new Set(); + for (const entry of decorated) { + if (seen.has(entry.oldReference)) throw new RenumberError('unmappable-survivor', false); + seen.add(entry.oldReference); + } + + const exploited = decorated.filter((entry) => entry.status === 'exploited'); + const blocked = decorated.filter((entry) => entry.status === 'blocked'); + if (exploited.length + blocked.length !== decorated.length) { + throw new RenumberError('unmappable-survivor', false); + } + + exploited.sort((first, second) => { + const severityDifference = + severityRank((first.entry as ExploitedExploit).severity) - + severityRank((second.entry as ExploitedExploit).severity); + if (severityDifference !== 0) return severityDifference; + if (first.numericReference !== second.numericReference) { + return first.numericReference - second.numericReference; + } + if (first.oldReference < second.oldReference) return -1; + if (first.oldReference > second.oldReference) return 1; + return 0; + }); + + const oldToNew = new Map(); + const order: string[] = []; + for (const [index, entry] of exploited.entries()) { + oldToNew.set(entry.oldReference, `${REF_PREFIX[vulnerabilityClass]}-${pad2(index + 1)}`); + order.push(entry.oldReference); + } + + const renumbered = exploited.map((entry) => { + const scrubbed = scrubEntryText(entry.entry, oldToNew) as Record; + return { + ...scrubbed, + vulnerability_id: oldToNew.get(entry.oldReference) as string, + } as unknown as AddExploitInput; + }); + const excluded = blocked.map((entry) => ({ + source_ref: entry.oldReference, + reason: 'validation_blocked' as const, + })); + return { oldToNew, renumbered, order, excluded }; +} + +export interface SastProvenanceEntry { + readonly exploit_ref: string; + readonly [key: string]: unknown; +} + +export interface SastProvenanceFile { + readonly entries: readonly SastProvenanceEntry[]; +} + +export function remapSastProvenance( + provenance: SastProvenanceFile, + oldToNew: ReadonlyMap, +): SastProvenanceFile { + const remapped: SastProvenanceEntry[] = []; + const seen = new Set(); + for (const entry of provenance.entries) { + const nextReference = oldToNew.get(entry.exploit_ref); + if (nextReference === undefined) continue; + if (seen.has(nextReference)) + throw new RenumberError('key-set-divergence', false, { checkCode: 'provenance-duplicate' }); + seen.add(nextReference); + remapped.push({ ...entry, exploit_ref: nextReference }); + } + return { entries: remapped }; +} + +function sha256(contents: string): string { + return createHash('sha256').update(contents, 'utf8').digest('hex'); +} + +export function sparseExploitCollectorPath(vulnerabilityClass: ReconciliationClass): string { + return `${vulnerabilityClass}_exploit_collector.json`; +} + +export function renumberedExploitCollectorPath(vulnerabilityClass: ReconciliationClass): string { + return `${vulnerabilityClass}_exploit_collector_renumbered.json`; +} + +export function exploitationEvidencePath(vulnerabilityClass: ReconciliationClass): string { + return `${vulnerabilityClass}_exploitation_evidence.md`; +} + +export function renumberMapPath(vulnerabilityClass: ReconciliationClass): string { + return `${vulnerabilityClass}_renumber_map.json`; +} + +export function renumberedSastProvenancePath(vulnerabilityClass: ReconciliationClass): string { + return `sast_provenance_${vulnerabilityClass}_renumbered.json`; +} + +export interface RenumberProducts extends RenumberMap { + readonly evidenceMarkdown: string; + readonly provenance?: SastProvenanceFile; +} + +function parseProvenance(value: unknown, vulnerabilityClass: ReconciliationClass): SastProvenanceFile { + if (value === null || typeof value !== 'object' || !Array.isArray((value as { entries?: unknown }).entries)) { + return divergence('provenance-malformed', vulnerabilityClass); + } + const entries = (value as { entries: unknown[] }).entries; + const seen = new Set(); + for (const entry of entries) { + if (entry === null || typeof entry !== 'object') + return divergence('provenance-entry-malformed', vulnerabilityClass); + const reference = (entry as { exploit_ref?: unknown }).exploit_ref; + if ( + typeof reference !== 'string' || + parseRefNumber(reference, vulnerabilityClass) === null || + seen.has(reference) + ) { + return divergence('provenance-reference-invalid', vulnerabilityClass); + } + seen.add(reference); + } + return value as SastProvenanceFile; +} + +async function loadAndValidateProvenance( + dir: string, + vulnerabilityClass: ReconciliationClass, + collectorReferences: ReadonlySet, +): Promise { + const manifestRead = await readPublishedManifest(dir, `${vulnerabilityClass}_reconciliation_manifest.json`); + if (manifestRead.state !== 'present') return divergence('manifest-not-present', vulnerabilityClass); + if (manifestRead.manifest.vulnerability_class !== vulnerabilityClass) { + return divergence('manifest-class-mismatch', vulnerabilityClass); + } + + const consumerContents = new Map(); + for (const consumer of manifestRead.manifest.consumer_files) { + const read = await readCommittedFile(dir, consumer.path); + if (read.state !== 'present' || sha256(read.contents) !== consumer.sha256) { + return divergence('manifest-consumer-digest-mismatch', vulnerabilityClass); + } + consumerContents.set(consumer.path, read.contents); + } + + const taskUniverse = new Set(Object.keys(manifestRead.manifest.lineage)); + const queueContents = consumerContents.get(`${vulnerabilityClass}_exploitation_queue.json`); + if (queueContents === undefined) return divergence('manifest-queue-consumer-missing', vulnerabilityClass); + let queue: unknown; + try { + queue = JSON.parse(queueContents) as unknown; + } catch { + return divergence('manifest-queue-not-json', vulnerabilityClass); + } + if ( + queue === null || + typeof queue !== 'object' || + !Array.isArray((queue as { vulnerabilities?: unknown }).vulnerabilities) + ) { + return divergence('manifest-queue-malformed', vulnerabilityClass); + } + const lineageReferences = Object.keys(manifestRead.manifest.lineage); + const queueReferences = (queue as { vulnerabilities: unknown[] }).vulnerabilities.map((entry) => + entry !== null && typeof entry === 'object' ? (entry as { ID?: unknown }).ID : undefined, + ); + if ( + queueReferences.length !== lineageReferences.length || + queueReferences.some((reference, index) => reference !== lineageReferences[index]) + ) { + return divergence('manifest-queue-lineage-mismatch', vulnerabilityClass); + } + for (const reference of collectorReferences) { + if (!taskUniverse.has(reference)) return divergence('collector-reference-outside-publication', vulnerabilityClass); + } + + const provenanceRelPath = sastProvenancePath(vulnerabilityClass); + const manifestConsumer = manifestRead.manifest.consumer_files.find((consumer) => consumer.path === provenanceRelPath); + if (manifestConsumer === undefined) return undefined; + const provenanceContents = consumerContents.get(provenanceRelPath); + if (provenanceContents === undefined) return divergence('provenance-digest-mismatch', vulnerabilityClass); + let decoded: unknown; + try { + decoded = JSON.parse(provenanceContents); + } catch { + return divergence('provenance-not-json', vulnerabilityClass); + } + const provenance = parseProvenance(decoded, vulnerabilityClass); + for (const entry of provenance.entries) { + if (!taskUniverse.has(entry.exploit_ref)) + return divergence('provenance-reference-outside-publication', vulnerabilityClass); + const lineage = manifestRead.manifest.lineage[entry.exploit_ref]; + if ( + lineage === undefined || + ![lineage.primary, ...lineage.absorbed].some((producerId) => isProducerId(producerId, vulnerabilityClass, 'SAST')) + ) { + return divergence('provenance-reference-not-sast-backed', vulnerabilityClass); + } + } + return provenance; +} + +export async function computeRenumber( + dir: string, + vulnerabilityClass: ReconciliationClass, +): Promise { + const collectorRead = await readCommittedFile(dir, sparseExploitCollectorPath(vulnerabilityClass)); + if (collectorRead.state === 'absent') return null; + if (collectorRead.state === 'corrupt') throw new RenumberError('unmappable-survivor', false); + + let decoded: unknown; + try { + decoded = JSON.parse(collectorRead.contents); + } catch { + throw new RenumberError('unmappable-survivor', false); + } + if (!Array.isArray(decoded)) throw new RenumberError('unmappable-survivor', false); + const map = buildRenumberMap(decoded as AddExploitInput[], vulnerabilityClass); + const collectorReferences = new Set([ + ...map.oldToNew.keys(), + ...map.excluded.map((entry) => entry.source_ref), + ]); + const sparseProvenance = await loadAndValidateProvenance(dir, vulnerabilityClass, collectorReferences); + const evidenceMarkdown = renderExploitDeliverable( + vulnerabilityClass, + map.renumbered, + new Map(), + ).replace( + '*No vulnerabilities were available in the queue for exploitation.*', + '*No vulnerabilities were confirmed during exploitation.*', + ); + return { + ...map, + evidenceMarkdown, + ...(sparseProvenance !== undefined && { provenance: remapSastProvenance(sparseProvenance, map.oldToNew) }), + }; +} + +export function renumberOutputFiles( + vulnerabilityClass: ReconciliationClass, + products: RenumberProducts, +): ExactOutputFile[] { + return [ + { + relPath: renumberedExploitCollectorPath(vulnerabilityClass), + contents: `${JSON.stringify(products.renumbered, null, 2)}\n`, + }, + { relPath: exploitationEvidencePath(vulnerabilityClass), contents: products.evidenceMarkdown }, + { + relPath: renumberMapPath(vulnerabilityClass), + contents: `${JSON.stringify( + { + vulnerability_type: vulnerabilityClass, + map: Object.fromEntries(products.oldToNew), + order: products.order, + excluded: products.excluded, + }, + null, + 2, + )}\n`, + }, + ...(products.provenance === undefined + ? [] + : [ + { + relPath: renumberedSastProvenancePath(vulnerabilityClass), + contents: `${JSON.stringify(products.provenance, null, 2)}\n`, + }, + ]), + ]; +} + +export interface RenumberClassResult { + readonly vulnerabilityClass: ReconciliationClass; + readonly renumberedCount: number; + readonly skipped: boolean; + readonly commit?: ExactOutputCommit; +} + +/** Service boundary used by the later Temporal activity wrapper. */ +export async function renumberClassFindings(args: { + readonly deliverablesDir: string; + readonly vulnerabilityClass: ReconciliationClass; + readonly logger: ActivityLogger; +}): Promise { + const products = await computeRenumber(args.deliverablesDir, args.vulnerabilityClass); + if (products === null) { + return { vulnerabilityClass: args.vulnerabilityClass, renumberedCount: 0, skipped: true }; + } + const commit = await writeAndCommitExactFiles( + args.deliverablesDir, + renumberOutputFiles(args.vulnerabilityClass, products), + `Renumber ${args.vulnerabilityClass} to dense report references`, + args.logger, + ); + return { + vulnerabilityClass: args.vulnerabilityClass, + renumberedCount: products.renumbered.length, + skipped: false, + commit, + }; +} diff --git a/apps/worker/src/services/report-checkpoints.ts b/apps/worker/src/services/report-checkpoints.ts new file mode 100644 index 00000000..f54b2cf4 --- /dev/null +++ b/apps/worker/src/services/report-checkpoints.ts @@ -0,0 +1,216 @@ +// Copyright (C) 2025 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Coherence proofs over durable report-stage checkpoints in the deliverables Git repo. */ + +import { createHash } from 'node:crypto'; +import path from 'node:path'; +import { $ } from 'zx'; +import { + ASSEMBLED_REPORT_FILENAME, + REPORT_FINALIZATION_MANIFEST_FILENAME, + REPORT_JSON_FILENAME, + SARIF_FILENAME, +} from '../paths.js'; +import { ErrorCode } from '../types/errors.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; +import type { ReportProgress } from '../types/run-state.js'; +import { fileExists } from '../utils/file-io.js'; +import { PentestError } from './error-handling.js'; +import { classifyHeadReadFailure, withGitRepoLock } from './git-manager.js'; +import { isReportFinalizationManifest } from './report-finalization.js'; +import type { ReportData } from './report-renderer.js'; + +function sha256(contents: string): string { + return createHash('sha256').update(contents, 'utf8').digest('hex'); +} + +function arraysEqual(left: readonly T[], right: readonly T[]): boolean { + return left.length === right.length && left.every((value, index) => value === right[index]); +} + +function transientCheckpointReadError(operation: string): PentestError { + return new PentestError( + 'A durable report checkpoint could not be read because of a transient repository error', + 'filesystem', + true, + { operation }, + ErrorCode.GIT_CHECKPOINT_FAILED, + ); +} + +export type CheckpointReadResult = + | { readonly state: 'present'; readonly contents: string } + | { readonly state: 'absent' } + | { readonly state: 'corrupt' }; + +/** + * Read one file at a specific checkpoint, preserving the proven-present, proven-absent, + * corrupt, and transient outcomes. Transient failures throw so the activity retry policy + * stays authoritative instead of a Git blip erasing a paid-for draft or masquerading as + * workspace corruption. + */ +export async function readFileAtCheckpoint( + deliverablesPath: string, + checkpoint: string, + relPath: string, +): Promise { + return withGitRepoLock(async () => { + const result = await $`cd ${deliverablesPath} && git show ${`${checkpoint}:${relPath}`}`.nothrow().quiet(); + if (result.exitCode === 0) { + return { state: 'present', contents: result.stdout }; + } + const failure = classifyHeadReadFailure(result.stderr); + if (failure === 'absent') return { state: 'absent' }; + if (failure === 'corrupt') return { state: 'corrupt' }; + throw transientCheckpointReadError('read-report-checkpoint-file'); + }); +} + +export async function checkpointFileContents( + deliverablesPath: string, + checkpoint: string, + relPath: string, +): Promise { + const read = await readFileAtCheckpoint(deliverablesPath, checkpoint, relPath); + return read.state === 'present' ? read.contents : null; +} + +/** Resolve a revision to a commit hash; absent/corrupt yields null, transient throws. */ +export async function resolveCheckpointCommit(deliverablesPath: string, revision: string): Promise { + return withGitRepoLock(async () => { + const result = await $`cd ${deliverablesPath} && git rev-parse --verify ${`${revision}^{commit}`}` + .nothrow() + .quiet(); + if (result.exitCode === 0) return result.stdout.trim(); + const failure = classifyHeadReadFailure(result.stderr); + if (failure === 'transient') throw transientCheckpointReadError('resolve-report-checkpoint'); + return null; + }); +} + +/** Ancestor check that keeps transient Git failures retryable instead of proof-invalid. */ +export async function checkpointIsAncestor( + ancestor: string, + descendant: string, + deliverablesPath: string, +): Promise { + return withGitRepoLock(async () => { + const result = await $`cd ${deliverablesPath} && git merge-base --is-ancestor ${ancestor} ${descendant}` + .nothrow() + .quiet(); + if (result.exitCode === 0) return true; + if (result.exitCode === 1 && result.stderr.trim() === '') return false; + const failure = classifyHeadReadFailure(result.stderr); + if (failure === 'transient') throw transientCheckpointReadError('verify-report-checkpoint-ancestry'); + return false; + }); +} + +async function checkpointIsReachable(deliverablesPath: string, checkpoint: string): Promise { + const head = await resolveCheckpointCommit(deliverablesPath, 'HEAD'); + return head !== null && (await checkpointIsAncestor(checkpoint, head, deliverablesPath)); +} + +export async function reportCheckpointIsCoherent( + deliverablesPath: string, + checkpoint: string, + failedClasses: readonly ReconciliationClass[], +): Promise { + const contents = await checkpointFileContents(deliverablesPath, checkpoint, REPORT_JSON_FILENAME); + if (contents === null || !(await checkpointIsReachable(deliverablesPath, checkpoint))) return false; + try { + const decoded = JSON.parse(contents) as ReportData; + return ( + decoded !== null && + typeof decoded === 'object' && + decoded.report_meta !== null && + typeof decoded.report_meta === 'object' && + Array.isArray(decoded.findings) && + arraysEqual(decoded.reconciliation_failed ?? [], failedClasses) + ); + } catch { + return false; + } +} + +export type DraftValidation = 'coherent' | 'invalid-model' | 'invalid-canonical'; + +export async function validateDraftProgress( + deliverablesPath: string, + progress: ReportProgress, +): Promise { + if (progress.stage === 'pending') return 'coherent'; + if ( + !(await reportCheckpointIsCoherent(deliverablesPath, progress.model_checkpoint, progress.renumber_failed_classes)) + ) { + return 'invalid-model'; + } + if (progress.canonical_checkpoint === undefined) return progress.stage === 'draft' ? 'coherent' : 'invalid-canonical'; + if (!(await checkpointIsAncestor(progress.model_checkpoint, progress.canonical_checkpoint, deliverablesPath))) { + return 'invalid-canonical'; + } + return (await reportCheckpointIsCoherent( + deliverablesPath, + progress.canonical_checkpoint, + progress.renumber_failed_classes, + )) + ? 'coherent' + : 'invalid-canonical'; +} + +export async function draftProgressIsCoherent(deliverablesPath: string, progress: ReportProgress): Promise { + return (await validateDraftProgress(deliverablesPath, progress)) === 'coherent'; +} + +export async function finalProgressIsCoherent(deliverablesPath: string, progress: ReportProgress): Promise { + if (progress.stage !== 'finalized' || !(await draftProgressIsCoherent(deliverablesPath, progress))) return false; + if (!(await checkpointIsAncestor(progress.canonical_checkpoint, progress.final_checkpoint, deliverablesPath))) { + return false; + } + if (!(await checkpointIsReachable(deliverablesPath, progress.final_checkpoint))) return false; + + const manifestContents = await checkpointFileContents( + deliverablesPath, + progress.final_checkpoint, + REPORT_FINALIZATION_MANIFEST_FILENAME, + ); + if (manifestContents === null || sha256(manifestContents) !== progress.finalization_manifest_sha256) return false; + + let manifest: unknown; + try { + manifest = JSON.parse(manifestContents) as unknown; + } catch { + return false; + } + if (!isReportFinalizationManifest(manifest) || manifestContents !== `${JSON.stringify(manifest, null, 2)}\n`) { + return false; + } + if (manifest.artifacts.sarif.disposition !== progress.sarif_disposition) return false; + + const reportJson = await checkpointFileContents(deliverablesPath, progress.final_checkpoint, REPORT_JSON_FILENAME); + const markdown = await checkpointFileContents(deliverablesPath, progress.final_checkpoint, ASSEMBLED_REPORT_FILENAME); + if ( + reportJson === null || + markdown === null || + sha256(reportJson) !== manifest.artifacts.report_json.sha256 || + sha256(markdown) !== manifest.artifacts.markdown.sha256 + ) { + return false; + } + try { + const decodedReport = JSON.parse(reportJson) as ReportData; + if (!arraysEqual(decodedReport.reconciliation_failed ?? [], progress.renumber_failed_classes)) return false; + } catch { + return false; + } + + const sarif = await checkpointFileContents(deliverablesPath, progress.final_checkpoint, SARIF_FILENAME); + if (manifest.artifacts.sarif.disposition !== 'committed') { + return sarif === null && !(await fileExists(path.join(deliverablesPath, SARIF_FILENAME))); + } + return sarif !== null && sha256(sarif) === manifest.artifacts.sarif.sha256; +} diff --git a/apps/worker/src/services/report-finalization.ts b/apps/worker/src/services/report-finalization.ts new file mode 100644 index 00000000..3606f5ee --- /dev/null +++ b/apps/worker/src/services/report-finalization.ts @@ -0,0 +1,456 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Canonical report publication and derived-PDF regeneration. */ + +import { createHash } from 'node:crypto'; +import { unlink } from 'node:fs/promises'; +import path from 'node:path'; +import { + ASSEMBLED_REPORT_FILENAME, + ASSEMBLED_REPORT_PDF_FILENAME, + REPORT_FINALIZATION_MANIFEST_FILENAME, + REPORT_JSON_FILENAME, + SARIF_FILENAME, + TYPST_TEMPLATE, +} from '../paths.js'; +import type { ActivityLogger } from '../types/activity-logger.js'; +import type { DistributedReportConfig } from '../types/config.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; +import type { ExactOutputCommit, ExactOutputFile } from './exact-output-commit.js'; +import { writeAndCommitExactFiles } from './exact-output-commit.js'; +import { orderFindings } from './finding-order.js'; +import { readCommittedFile, withGitRepoLock } from './git-manager.js'; +import { type PdfProvenance, pdfProvenanceIsCurrent, readPdfProvenance, renderReportPdf } from './pdf-renderer.js'; +import { type ReportData, renderReport } from './report-renderer.js'; +import { renderSarif } from './sarif-renderer.js'; + +const FINALIZATION_SCHEMA_VERSION = 1; +const REPORT_RENDERER_VERSION = '4.13.1'; +const SARIF_RENDERER_VERSION = '4.13.1'; + +interface CommittedArtifactReceipt { + readonly path: string; + readonly disposition: 'committed'; + readonly sha256: string; +} + +export type ReportSarifDisposition = 'committed' | 'absent' | 'render_failed'; + +interface ConditionalArtifactReceipt { + readonly path: string; + readonly disposition: ReportSarifDisposition; + readonly sha256?: string; +} + +interface DerivedArtifactReceipt { + readonly path: string; + readonly disposition: 'derived_uncommitted'; +} + +export interface ReportFinalizationManifest { + readonly schema_version: typeof FINALIZATION_SCHEMA_VERSION; + readonly input_fingerprint: string; + readonly artifacts: { + readonly report_json: CommittedArtifactReceipt; + readonly markdown: CommittedArtifactReceipt; + readonly sarif: ConditionalArtifactReceipt; + readonly pdf: DerivedArtifactReceipt; + }; +} + +export interface FinalizeReportResult { + readonly commit: ExactOutputCommit; + readonly manifest: ReportFinalizationManifest; + readonly pdfGenerated: boolean; + readonly pdfProvenance: PdfProvenance | null; + readonly warnings: readonly string[]; +} + +/** Stable service error preserved as retryable by the Temporal integration wrapper. */ +export class ReportSarifRenderError extends Error { + readonly retryable = true; + + constructor(cause: unknown) { + super('Report SARIF rendering failed.', { cause }); + this.name = 'ReportSarifRenderError'; + } +} + +/** + * Canonical report, manifest, or committed-byte corruption detected during finalization. + * Always terminal: canonical corruption never degrades into a partial result or a warning. + */ +export class ReportFinalizationIntegrityError extends Error { + readonly retryable = false; + readonly checkCode: string; + + constructor(checkCode: string) { + super('Report finalization integrity validation failed.'); + this.name = 'ReportFinalizationIntegrityError'; + this.checkCode = checkCode; + } +} + +function sha256(contents: string): string { + return createHash('sha256').update(contents, 'utf8').digest('hex'); +} + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === 'object' && !Array.isArray(value); +} + +function parseReportData(contents: string): ReportData { + let decoded: unknown; + try { + decoded = JSON.parse(contents) as unknown; + } catch { + throw new ReportFinalizationIntegrityError('finalization-report-not-json'); + } + if (!isRecord(decoded) || !isRecord(decoded.report_meta) || !Array.isArray(decoded.findings)) { + throw new ReportFinalizationIntegrityError('finalization-report-malformed'); + } + const meta = decoded.report_meta; + if ( + typeof meta.target !== 'string' || + typeof meta.assessment_date !== 'string' || + typeof meta.scope !== 'string' || + typeof meta.executive_summary !== 'string' || + meta.scope.trim() === '' || + meta.executive_summary.trim() === '' + ) { + throw new ReportFinalizationIntegrityError('finalization-report-meta-malformed'); + } + return decoded as unknown as ReportData; +} + +function isSha256(value: unknown): value is string { + return typeof value === 'string' && /^[a-f0-9]{64}$/.test(value); +} + +/** Runtime guard consumed by resume/terminal-state wiring in the next task. */ +export function isReportFinalizationManifest(value: unknown): value is ReportFinalizationManifest { + if (!isRecord(value) || value.schema_version !== FINALIZATION_SCHEMA_VERSION || !isSha256(value.input_fingerprint)) { + return false; + } + if (!isRecord(value.artifacts)) return false; + const reportJson = value.artifacts.report_json; + const markdown = value.artifacts.markdown; + const sarif = value.artifacts.sarif; + const pdf = value.artifacts.pdf; + if (!isRecord(reportJson) || !isRecord(markdown) || !isRecord(sarif) || !isRecord(pdf)) return false; + if ( + reportJson.path !== REPORT_JSON_FILENAME || + reportJson.disposition !== 'committed' || + !isSha256(reportJson.sha256) + ) { + return false; + } + if ( + markdown.path !== ASSEMBLED_REPORT_FILENAME || + markdown.disposition !== 'committed' || + !isSha256(markdown.sha256) + ) { + return false; + } + const validSarif = + sarif.path === SARIF_FILENAME && + (((sarif.disposition === 'absent' || sarif.disposition === 'render_failed') && sarif.sha256 === undefined) || + (sarif.disposition === 'committed' && isSha256(sarif.sha256))); + return validSarif && pdf.path === ASSEMBLED_REPORT_PDF_FILENAME && pdf.disposition === 'derived_uncommitted'; +} + +/** + * Derive the one canonical report from the model-authored report.json: fixed finding order, + * workflow-owned exploit flag and coverage, and a de-duplicated reconciliation-failed set. Every + * finalization attempt (including a retried or degraded re-drive) must produce byte-identical + * output from the same inputs, since `buildInputFingerprint` and the adoption check in + * `readExistingFinalization` both compare against this canonical form rather than the raw model + * output. + */ +function canonicalizeReport(args: { + readonly report: ReportData; + readonly exploit: boolean; + readonly reconciliationFailedClasses?: readonly ReconciliationClass[]; +}): ReportData { + const reconciliationFailed = args.reconciliationFailedClasses ?? args.report.reconciliation_failed ?? []; + if (new Set(reconciliationFailed).size !== reconciliationFailed.length) { + throw new ReportFinalizationIntegrityError('finalization-failed-class-duplicate'); + } + if ( + args.reconciliationFailedClasses !== undefined && + JSON.stringify(args.report.reconciliation_failed ?? []) !== JSON.stringify(args.reconciliationFailedClasses) + ) { + throw new ReportFinalizationIntegrityError('finalization-failed-class-set-mismatch'); + } + return { + ...args.report, + report_meta: { ...args.report.report_meta, exploit: args.exploit }, + findings: orderFindings(args.report.findings), + reconciliation_failed: [...reconciliationFailed], + }; +} + +function buildInputFingerprint(args: { + readonly canonicalJson: string; + readonly exploit: boolean; + readonly workspaceName: string; + readonly reportConfig: DistributedReportConfig; +}): string { + return sha256( + JSON.stringify({ + canonical_report_sha256: sha256(args.canonicalJson), + exploit: args.exploit, + workspace_name: args.workspaceName, + report_config: { + min_severity: args.reportConfig.min_severity ?? null, + min_confidence: args.reportConfig.min_confidence ?? null, + guidance_sha256: args.reportConfig.guidance === undefined ? null : sha256(args.reportConfig.guidance), + sarif: args.reportConfig.sarif, + }, + report_renderer_version: REPORT_RENDERER_VERSION, + sarif_renderer_version: SARIF_RENDERER_VERSION, + }), + ); +} + +function buildManifest(args: { + readonly canonicalJson: string; + readonly markdown: string; + readonly sarif: string | null; + readonly sarifDisposition: ReportSarifDisposition; + readonly exploit: boolean; + readonly workspaceName: string; + readonly reportConfig: DistributedReportConfig; +}): ReportFinalizationManifest { + return { + schema_version: FINALIZATION_SCHEMA_VERSION, + input_fingerprint: buildInputFingerprint(args), + artifacts: { + report_json: { + path: REPORT_JSON_FILENAME, + disposition: 'committed', + sha256: sha256(args.canonicalJson), + }, + markdown: { + path: ASSEMBLED_REPORT_FILENAME, + disposition: 'committed', + sha256: sha256(args.markdown), + }, + sarif: + args.sarifDisposition === 'committed' && args.sarif !== null + ? { path: SARIF_FILENAME, disposition: 'committed', sha256: sha256(args.sarif) } + : { path: SARIF_FILENAME, disposition: args.sarifDisposition }, + pdf: { path: ASSEMBLED_REPORT_PDF_FILENAME, disposition: 'derived_uncommitted' }, + }, + }; +} + +async function readExistingFinalization(args: { + readonly deliverablesDir: string; + readonly canonicalJson: string; + readonly markdown: string; + readonly exploit: boolean; + readonly workspaceName: string; + readonly reportConfig: DistributedReportConfig; +}): Promise<{ + readonly manifest: ReportFinalizationManifest; + readonly manifestContents: string; + readonly sarif: string | null; +} | null> { + const read = await readCommittedFile(args.deliverablesDir, REPORT_FINALIZATION_MANIFEST_FILENAME); + if (read.state === 'absent') return null; + if (read.state !== 'present') { + throw new ReportFinalizationIntegrityError('finalization-manifest-unreadable'); + } + let decoded: unknown; + try { + decoded = JSON.parse(read.contents) as unknown; + } catch { + throw new ReportFinalizationIntegrityError('finalization-manifest-not-json'); + } + if (!isReportFinalizationManifest(decoded) || read.contents !== `${JSON.stringify(decoded, null, 2)}\n`) { + throw new ReportFinalizationIntegrityError('finalization-manifest-conflict'); + } + const expectedInputFingerprint = buildInputFingerprint(args); + if ( + decoded.input_fingerprint !== expectedInputFingerprint || + decoded.artifacts.report_json.sha256 !== sha256(args.canonicalJson) || + decoded.artifacts.markdown.sha256 !== sha256(args.markdown) + ) { + throw new ReportFinalizationIntegrityError('finalization-manifest-conflict'); + } + const markdownRead = await readCommittedFile(args.deliverablesDir, ASSEMBLED_REPORT_FILENAME); + if (markdownRead.state !== 'present' || markdownRead.contents !== args.markdown) { + throw new ReportFinalizationIntegrityError('finalization-markdown-digest-mismatch'); + } + const sarifRead = await readCommittedFile(args.deliverablesDir, SARIF_FILENAME); + let sarif: string | null = null; + if (decoded.artifacts.sarif.disposition !== 'committed') { + if (sarifRead.state !== 'absent') { + throw new ReportFinalizationIntegrityError('finalization-stale-sarif'); + } + } else { + if ( + sarifRead.state !== 'present' || + decoded.artifacts.sarif.sha256 === undefined || + sha256(sarifRead.contents) !== decoded.artifacts.sarif.sha256 + ) { + throw new ReportFinalizationIntegrityError('finalization-sarif-digest-mismatch'); + } + sarif = sarifRead.contents; + } + return { manifest: decoded, manifestContents: read.contents, sarif }; +} + +/** + * Finalize all canonical report outputs in one exact-path commit, then regenerate the uncommitted + * PDF from those same canonical bytes. PDF failure is warning-only and cannot change the commit. + */ +export async function finalizeReport(args: { + readonly deliverablesDir: string; + readonly exploit: boolean; + readonly reconciliationFailedClasses?: readonly ReconciliationClass[]; + readonly reportConfig: DistributedReportConfig; + readonly workspaceName: string; + readonly logger: ActivityLogger; + readonly templatePath?: string; + readonly renderPdf?: typeof renderReportPdf; + readonly renderSarif?: typeof renderSarif; + /** Used only after ordinary SARIF attempts have exhausted their Temporal retry policy. */ + readonly degradedSarif?: boolean; + /** Durable provenance from an earlier successful PDF publication, when available. */ + readonly priorPdfProvenance?: PdfProvenance; + readonly afterCommit?: (commit: { commitHash: string; changedPaths: readonly string[] }) => void | Promise; +}): Promise { + const warnings: string[] = []; + const finalized = await withGitRepoLock(async () => { + const reportRead = await readCommittedFile(args.deliverablesDir, REPORT_JSON_FILENAME); + if (reportRead.state !== 'present') { + throw new ReportFinalizationIntegrityError('finalization-report-unreadable'); + } + const canonicalReport = canonicalizeReport({ + report: parseReportData(reportRead.contents), + exploit: args.exploit, + ...(args.reconciliationFailedClasses !== undefined && { + reconciliationFailedClasses: args.reconciliationFailedClasses, + }), + }); + const canonicalJson = `${JSON.stringify(canonicalReport, null, 2)}\n`; + const markdown = renderReport(canonicalReport); + const existing = await readExistingFinalization({ + deliverablesDir: args.deliverablesDir, + canonicalJson, + markdown, + exploit: args.exploit, + workspaceName: args.workspaceName, + reportConfig: args.reportConfig, + }); + if (existing !== null) { + const files: readonly ExactOutputFile[] = [ + { relPath: REPORT_JSON_FILENAME, contents: canonicalJson }, + { relPath: ASSEMBLED_REPORT_FILENAME, contents: markdown }, + { relPath: SARIF_FILENAME, contents: existing.sarif }, + { relPath: REPORT_FINALIZATION_MANIFEST_FILENAME, contents: existing.manifestContents }, + ]; + const commit = await writeAndCommitExactFiles( + args.deliverablesDir, + files, + 'Finalize canonical report outputs', + args.logger, + args.afterCommit === undefined ? {} : { afterCommit: args.afterCommit }, + ); + return { canonicalReport, commit, manifest: existing.manifest }; + } + + const sarifRequested = args.exploit && args.reportConfig.sarif; + let sarif: string | null = null; + let sarifDisposition: ReportSarifDisposition = 'absent'; + if (sarifRequested && args.degradedSarif === true) { + sarifDisposition = 'render_failed'; + } else if (sarifRequested) { + try { + sarif = (args.renderSarif ?? renderSarif)(canonicalReport, args); + sarifDisposition = 'committed'; + } catch (error: unknown) { + throw new ReportSarifRenderError(error); + } + } + const manifest = buildManifest({ + canonicalJson, + markdown, + sarif, + sarifDisposition, + exploit: args.exploit, + workspaceName: args.workspaceName, + reportConfig: args.reportConfig, + }); + if (!isReportFinalizationManifest(manifest)) { + throw new ReportFinalizationIntegrityError('finalization-manifest-self-invalid'); + } + const manifestContents = `${JSON.stringify(manifest, null, 2)}\n`; + + const files: readonly ExactOutputFile[] = [ + { relPath: REPORT_JSON_FILENAME, contents: canonicalJson }, + { relPath: ASSEMBLED_REPORT_FILENAME, contents: markdown }, + { relPath: SARIF_FILENAME, contents: sarif }, + { relPath: REPORT_FINALIZATION_MANIFEST_FILENAME, contents: manifestContents }, + ]; + const commit = await writeAndCommitExactFiles( + args.deliverablesDir, + files, + 'Finalize canonical report outputs', + args.logger, + args.afterCommit === undefined ? {} : { afterCommit: args.afterCommit }, + ); + return { canonicalReport, commit, manifest }; + }); + + let pdfGenerated = false; + let pdfProvenance: PdfProvenance | null = null; + const pdfPath = path.join(args.deliverablesDir, ASSEMBLED_REPORT_PDF_FILENAME); + const renderPdf = args.renderPdf ?? renderReportPdf; + const templatePath = args.templatePath ?? TYPST_TEMPLATE; + const canonicalReportSha256 = finalized.manifest.artifacts.report_json.sha256; + try { + await renderPdf({ + reportData: finalized.canonicalReport, + templatePath, + outputPath: pdfPath, + }); + pdfProvenance = await readPdfProvenance({ pdfPath, canonicalReportSha256, templatePath }); + pdfGenerated = true; + } catch (error) { + const label = error instanceof Error ? ((error as NodeJS.ErrnoException).code ?? error.name) : 'unknown error'; + const warning = `The PDF report could not be produced (${label}). The Markdown report and the structured findings are unaffected.`; + warnings.push(warning); + args.logger.warn(warning); + const canPreservePriorPdf = + args.priorPdfProvenance !== undefined && + (await pdfProvenanceIsCurrent({ + pdfPath, + canonicalReportSha256, + provenance: args.priorPdfProvenance, + templatePath, + })); + if (canPreservePriorPdf) { + pdfProvenance = args.priorPdfProvenance; + } else { + await unlink(pdfPath).catch((cleanupError: unknown) => { + if (cleanupError instanceof Error && (cleanupError as NodeJS.ErrnoException).code === 'ENOENT') return; + const cleanupLabel = + cleanupError instanceof Error + ? ((cleanupError as NodeJS.ErrnoException).code ?? cleanupError.name) + : 'unknown error'; + const cleanupWarning = `An out-of-date PDF could not be deleted (${cleanupLabel}). Ignore any PDF in this workspace and use the Markdown report.`; + warnings.push(cleanupWarning); + args.logger.warn(cleanupWarning); + }); + } + } + + return { commit: finalized.commit, manifest: finalized.manifest, pdfGenerated, pdfProvenance, warnings }; +} diff --git a/apps/worker/src/services/report-output-surface.ts b/apps/worker/src/services/report-output-surface.ts new file mode 100644 index 00000000..85bb5819 --- /dev/null +++ b/apps/worker/src/services/report-output-surface.ts @@ -0,0 +1,218 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Best-effort, narrow customer output publication. */ + +import { randomUUID } from 'node:crypto'; +import { mkdir, readFile, rename, unlink, writeFile } from 'node:fs/promises'; +import path from 'node:path'; +import { + ASSEMBLED_REPORT_FILENAME, + ASSEMBLED_REPORT_PDF_FILENAME, + FINAL_REPORT_MD_FILENAME, + FINAL_REPORT_PDF_FILENAME, + SARIF_FILENAME, +} from '../paths.js'; +import type { ActivityLogger } from '../types/activity-logger.js'; +import { type PdfProvenance, pdfMatchesProvenance } from './pdf-renderer.js'; + +interface SurfaceOutput { + readonly source: string; + readonly destination: string; + readonly removeWhenSourceMissing: boolean; +} + +export interface ReportOutputSurfaceResult { + readonly surfaced: readonly string[]; + readonly removedStale: readonly string[]; + readonly warnings: readonly string[]; +} + +function isErrno(error: unknown, code: string): boolean { + return error instanceof Error && (error as NodeJS.ErrnoException).code === code; +} + +function errorLabel(error: unknown): string { + if (!(error instanceof Error)) return 'unknown error'; + return (error as NodeJS.ErrnoException).code ?? error.name; +} + +async function removeIfPresent(filePath: string): Promise { + try { + await unlink(filePath); + return true; + } catch (error) { + if (isErrno(error, 'ENOENT')) return false; + throw error; + } +} + +async function atomicCopy(sourcePath: string, destinationPath: string): Promise { + const contents = await readFile(sourcePath); + await mkdir(path.dirname(destinationPath), { recursive: true }); + try { + const existing = await readFile(destinationPath); + if (existing.equals(contents)) return; + } catch (error) { + if (!isErrno(error, 'ENOENT')) throw error; + } + const temporaryPath = `${destinationPath}.tmp-${randomUUID()}`; + try { + await writeFile(temporaryPath, contents, { flag: 'wx' }); + await rename(temporaryPath, destinationPath); + const verified = await readFile(destinationPath); + if (!verified.equals(contents)) { + const error = new Error('customer copy verification failed') as NodeJS.ErrnoException; + error.code = 'EIO'; + throw error; + } + } catch (error) { + await unlink(temporaryPath).catch(() => undefined); + throw error; + } +} + +async function surfaceProvenancedPdf(args: { + readonly deliverablesDir: string; + readonly customerDir: string; + readonly canonicalReportSha256: string; + readonly provenance: PdfProvenance | null; + readonly logger: ActivityLogger; + readonly copyOutput: (sourcePath: string, destinationPath: string) => Promise; + readonly surfaced: string[]; + readonly removedStale: string[]; + readonly warnings: string[]; +}): Promise { + const sourcePath = path.join(args.deliverablesDir, ASSEMBLED_REPORT_PDF_FILENAME); + const destinationPath = path.join(args.customerDir, FINAL_REPORT_PDF_FILENAME); + const sourceMatches = + args.provenance !== null && + (await pdfMatchesProvenance({ + pdfPath: sourcePath, + canonicalReportSha256: args.canonicalReportSha256, + provenance: args.provenance, + })); + + if (sourceMatches) { + try { + await args.copyOutput(sourcePath, destinationPath); + args.surfaced.push(FINAL_REPORT_PDF_FILENAME); + args.logger.info(`Surfaced ${FINAL_REPORT_PDF_FILENAME}`); + } catch (error) { + const warning = `The PDF report could not be produced (${errorLabel(error)}). The Markdown report and the structured findings are unaffected.`; + args.warnings.push(warning); + args.logger.warn(warning); + } + return; + } + + try { + await removeIfPresent(sourcePath); + } catch (error) { + const warning = `An out-of-date PDF could not be deleted (${errorLabel(error)}). Ignore any PDF in this workspace and use the Markdown report.`; + args.warnings.push(warning); + args.logger.warn(warning); + } + + const customerMatches = + args.provenance !== null && + (await pdfMatchesProvenance({ + pdfPath: destinationPath, + canonicalReportSha256: args.canonicalReportSha256, + provenance: args.provenance, + })); + if (customerMatches) { + args.logger.info(`Preserved verified ${FINAL_REPORT_PDF_FILENAME}`); + return; + } + + try { + if (await removeIfPresent(destinationPath)) args.removedStale.push(FINAL_REPORT_PDF_FILENAME); + } catch (error) { + const warning = `An out-of-date PDF could not be deleted (${errorLabel(error)}). Ignore any PDF in this workspace and use the Markdown report.`; + args.warnings.push(warning); + args.logger.warn(warning); + } +} + +/** + * Surface only Markdown, PDF, and optional SARIF. A copy failure never changes workflow status; + * it produces a warning and leaves any previous destination atomically intact. + */ +export async function surfaceReportOutputs(args: { + readonly deliverablesDir: string; + readonly customerDir: string; + readonly logger: ActivityLogger; + readonly copyOutput?: (sourcePath: string, destinationPath: string) => Promise; + /** Enables verified PDF reuse once the integration layer supplies durable provenance. */ + readonly pdfVerification?: { + readonly canonicalReportSha256: string; + readonly provenance: PdfProvenance | null; + }; +}): Promise { + const outputs: SurfaceOutput[] = [ + { + source: ASSEMBLED_REPORT_FILENAME, + destination: FINAL_REPORT_MD_FILENAME, + removeWhenSourceMissing: false, + }, + { source: SARIF_FILENAME, destination: SARIF_FILENAME, removeWhenSourceMissing: true }, + ]; + if (args.pdfVerification === undefined) { + outputs.splice(1, 0, { + source: ASSEMBLED_REPORT_PDF_FILENAME, + destination: FINAL_REPORT_PDF_FILENAME, + removeWhenSourceMissing: true, + }); + } + const copyOutput = args.copyOutput ?? atomicCopy; + const surfaced: string[] = []; + const removedStale: string[] = []; + const warnings: string[] = []; + + for (const output of outputs) { + const sourcePath = path.join(args.deliverablesDir, output.source); + const destinationPath = path.join(args.customerDir, output.destination); + try { + await copyOutput(sourcePath, destinationPath); + surfaced.push(output.destination); + args.logger.info(`Surfaced ${output.destination}`); + } catch (error) { + // An absent optional source is the expected state, not a degradation: drop any stale copy + // left by an earlier run and move on without a warning the operator would learn to ignore. + const sourceLegitimatelyAbsent = isErrno(error, 'ENOENT') && output.removeWhenSourceMissing; + if (sourceLegitimatelyAbsent) { + try { + if (await removeIfPresent(destinationPath)) removedStale.push(output.destination); + } catch (cleanupError) { + const warning = `Could not remove stale ${output.destination} (${errorLabel(cleanupError)})`; + warnings.push(warning); + args.logger.warn(warning); + } + continue; + } + const warning = `Could not surface ${output.destination} (${errorLabel(error)})`; + warnings.push(warning); + args.logger.warn(warning); + } + } + + if (args.pdfVerification !== undefined) { + await surfaceProvenancedPdf({ + deliverablesDir: args.deliverablesDir, + customerDir: args.customerDir, + canonicalReportSha256: args.pdfVerification.canonicalReportSha256, + provenance: args.pdfVerification.provenance, + logger: args.logger, + copyOutput, + surfaced, + removedStale, + warnings, + }); + } + + return { surfaced, removedStale, warnings }; +} diff --git a/apps/worker/src/services/report-renderer.ts b/apps/worker/src/services/report-renderer.ts index 8ed68a4a..953b48f4 100644 --- a/apps/worker/src/services/report-renderer.ts +++ b/apps/worker/src/services/report-renderer.ts @@ -14,6 +14,7 @@ import type { AddFindingInput, AdditionalSection, StepItem, StructuredStep } from '../collectors/finding-collector.js'; import type { VulnClass } from '../types/config.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; // ============================================================================ // TYPES @@ -34,6 +35,8 @@ export interface ReportData { // Vuln classes whose pipeline failed and were not assessed this run. Rendered as an explicit // caveat so an un-assessed class is never presented as a clean result. readonly not_assessed?: readonly VulnClass[]; + /** Exploit classes excluded from compaction after a renumber failure, in workflow order. */ + readonly reconciliation_failed?: readonly ReconciliationClass[]; } // Without this, an analysis-only report reads as though the impact was demonstrated. diff --git a/apps/worker/src/services/reporting.ts b/apps/worker/src/services/reporting.ts index 6b61e4d9..bc527891 100644 --- a/apps/worker/src/services/reporting.ts +++ b/apps/worker/src/services/reporting.ts @@ -5,51 +5,141 @@ // as published by the Free Software Foundation. import { fs, path } from 'zx'; -import { - ASSEMBLED_REPORT_FILENAME, - ASSEMBLED_REPORT_PDF_FILENAME, - deliverablesDir, - FINAL_REPORT_MD_FILENAME, - FINAL_REPORT_PDF_FILENAME, - resolveSessionJsonPath, - SARIF_FILENAME, -} from '../paths.js'; +import { ASSEMBLED_REPORT_FILENAME, deliverablesDir } from '../paths.js'; import type { ActivityLogger } from '../types/activity-logger.js'; import { ErrorCode } from '../types/errors.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; import { PentestError } from './error-handling.js'; +import { renderExploitDeliverable } from './exploit-renderer.js'; +import { readCommittedFile } from './git-manager.js'; +import { surfaceReportOutputs } from './report-output-surface.js'; interface DeliverableFile { + vulnerabilityClass: ReconciliationClass; name: string; /** Candidate filenames in priority order. First one that exists wins. */ paths: readonly string[]; required: boolean; } -// Pure function: Assemble final report from specialist deliverables. -// Per class, prefer the exploit-agent's evidence file; fall back to renderer-produced findings. -// Both never coexist for a workspace because scope (exploit flag) is locked. -export async function assembleFinalReport( +const DELIVERABLE_BY_CLASS: Readonly< + Record +> = Object.freeze({ + injection: { + name: 'Injection', + exploit: 'injection_exploitation_evidence.md', + analysis: 'injection_findings.md', + }, + xss: { name: 'XSS', exploit: 'xss_exploitation_evidence.md', analysis: 'xss_findings.md' }, + auth: { + name: 'Authentication', + exploit: 'auth_exploitation_evidence.md', + analysis: 'auth_findings.md', + }, + ssrf: { name: 'SSRF', exploit: 'ssrf_exploitation_evidence.md', analysis: 'ssrf_findings.md' }, + authz: { + name: 'Authorization', + exploit: 'authz_exploitation_evidence.md', + analysis: 'authz_findings.md', + }, + miscellaneous: { + name: 'Miscellaneous', + exploit: 'miscellaneous_exploitation_evidence.md', + analysis: 'miscellaneous_findings.md', + }, +}); + +const DEFAULT_REPORT_CLASS_ORDER = [ + 'injection', + 'xss', + 'auth', + 'ssrf', + 'authz', +] as const satisfies readonly ReconciliationClass[]; + +export interface AssembleFinalReportOptions { + /** Explicit mode prevents an exploitative report from falling back to analysis artifacts. */ + readonly exploit?: boolean; + /** Caller-owned order is preserved verbatim. */ + readonly participatingClasses?: readonly ReconciliationClass[]; + /** Classes already known to have failed during analysis-only findings rendering. */ + readonly knownFailedClasses?: readonly ReconciliationClass[]; +} + +export interface AssembleFinalReportResult { + readonly content: string; + readonly failedClasses: readonly ReconciliationClass[]; +} + +/** + * Distinguish an assessed class with no actionable findings from a class whose exploit evidence + * disappeared. The committed reconciled queue is authoritative across retries and resume; the + * workflow's in-memory skipped-agent list is not. + */ +async function renderCommittedEmptyClass(dir: string, vulnerabilityClass: ReconciliationClass): Promise { + const queueRead = await readCommittedFile(dir, `${vulnerabilityClass}_exploitation_queue.json`); + if (queueRead.state !== 'present') return null; + + let queue: unknown; + try { + queue = JSON.parse(queueRead.contents) as unknown; + } catch { + return null; + } + if ( + queue === null || + typeof queue !== 'object' || + !Array.isArray((queue as { vulnerabilities?: unknown }).vulnerabilities) || + (queue as { vulnerabilities: unknown[] }).vulnerabilities.length !== 0 + ) { + return null; + } + + return renderExploitDeliverable(vulnerabilityClass, [], new Map()); +} + +async function assembleFinalReportInternal( sourceDir: string, deliverablesSubdir: string | undefined, logger: ActivityLogger, -): Promise { - const deliverableFiles: readonly DeliverableFile[] = [ - { name: 'Injection', paths: ['injection_exploitation_evidence.md', 'injection_findings.md'], required: false }, - { name: 'XSS', paths: ['xss_exploitation_evidence.md', 'xss_findings.md'], required: false }, - { name: 'Authentication', paths: ['auth_exploitation_evidence.md', 'auth_findings.md'], required: false }, - { name: 'SSRF', paths: ['ssrf_exploitation_evidence.md', 'ssrf_findings.md'], required: false }, - { name: 'Authorization', paths: ['authz_exploitation_evidence.md', 'authz_findings.md'], required: false }, - ]; + options: AssembleFinalReportOptions, + collectClassFailures: boolean, +): Promise { + const participatingClasses = options.participatingClasses ?? DEFAULT_REPORT_CLASS_ORDER; + const deliverableFiles: readonly DeliverableFile[] = participatingClasses.map((vulnerabilityClass) => { + const definition = DELIVERABLE_BY_CLASS[vulnerabilityClass]; + let paths: readonly string[] = [definition.exploit, definition.analysis]; + if (options.exploit === true) paths = [definition.exploit]; + if (options.exploit === false) paths = [definition.analysis]; + return { vulnerabilityClass, name: definition.name, paths, required: false }; + }); const dir = deliverablesDir(sourceDir, deliverablesSubdir); const sections: string[] = []; + const failedClassSet = new Set(options.knownFailedClasses ?? []); for (const file of deliverableFiles) { + if (failedClassSet.has(file.vulnerabilityClass)) { + logger.warn(`${file.name}: omitted because findings rendering failed`); + continue; + } let added = false; for (const candidate of file.paths) { - const filePath = path.join(dir, candidate); try { - if (await fs.pathExists(filePath)) { + if (options.exploit === true) { + const committed = await readCommittedFile(dir, candidate); + if (committed.state === 'corrupt') { + throw new Error('committed artifact is corrupt'); + } + if (committed.state === 'present') { + sections.push(committed.contents); + logger.info(`Added ${file.name} section from ${candidate}`); + added = true; + break; + } + } else { + const filePath = path.join(dir, candidate); + if (!(await fs.pathExists(filePath))) continue; const content = await fs.readFile(filePath, 'utf8'); sections.push(content); logger.info(`Added ${file.name} section from ${candidate}`); @@ -57,8 +147,19 @@ export async function assembleFinalReport( break; } } catch (error) { + if (!collectClassFailures) throw error; const err = error as Error; logger.warn(`Could not read ${candidate}: ${err.message}`); + failedClassSet.add(file.vulnerabilityClass); + break; + } + } + if (!added && options.exploit === true && !failedClassSet.has(file.vulnerabilityClass)) { + const emptyClassSection = await renderCommittedEmptyClass(dir, file.vulnerabilityClass); + if (emptyClassSection !== null) { + sections.push(emptyClassSection); + logger.info(`Added ${file.name} section from its committed empty exploitation queue`); + added = true; } } if (!added) { @@ -72,6 +173,7 @@ export async function assembleFinalReport( ); } logger.info(`No ${file.name} deliverable found`); + failedClassSet.add(file.vulnerabilityClass); } } @@ -90,87 +192,44 @@ export async function assembleFinalReport( }); } - return finalContent; + return { + content: finalContent, + failedClasses: participatingClasses.filter((vulnerabilityClass) => failedClassSet.has(vulnerabilityClass)), + }; } /** - * Inject model information into the final security report. - * Reads session.json to get the model(s) used, then injects a "Model:" line - * into the Executive Summary section of the report. + * Assemble report inputs while returning class-local omissions for `not_assessed` integration. + * Canonical output write failures still throw. */ -export async function injectModelIntoReport( - repoPath: string, +export async function assembleFinalReportWithEvidence( + sourceDir: string, deliverablesSubdir: string | undefined, - outputPath: string, logger: ActivityLogger, -): Promise { - // 1. Read session.json to get model information - const sessionJsonPath = resolveSessionJsonPath(outputPath); + options: AssembleFinalReportOptions = {}, +): Promise { + return assembleFinalReportInternal(sourceDir, deliverablesSubdir, logger, options, true); +} - if (!(await fs.pathExists(sessionJsonPath))) { - logger.warn('session.json not found, skipping model injection'); - return; - } - - interface SessionData { - metrics: { - agents: Record; - }; - } - - const sessionData: SessionData = await fs.readJson(sessionJsonPath); - - // 2. Extract unique models from all agents - const models = new Set(); - for (const agent of Object.values(sessionData.metrics.agents)) { - if (agent.model) { - models.add(agent.model); - } - } - - if (models.size === 0) { - logger.warn('No model information found in session.json'); - return; - } - - const modelStr = Array.from(models).join(', '); - logger.info(`Injecting model info into report: ${modelStr}`); - - // 3. Read the final report - const reportPath = path.join(deliverablesDir(repoPath, deliverablesSubdir), ASSEMBLED_REPORT_FILENAME); - - if (!(await fs.pathExists(reportPath))) { - logger.warn('Final report not found, skipping model injection'); - return; - } - - let reportContent = await fs.readFile(reportPath, 'utf8'); - - // 4. Find and inject model line after "Assessment Date" in Executive Summary - // Pattern: "- Assessment Date: " followed by a newline - const assessmentDatePattern = /^(- Assessment Date: .+)$/m; - const match = reportContent.match(assessmentDatePattern); - - if (match) { - // Inject model line after Assessment Date - const modelLine = `- Model: ${modelStr}`; - reportContent = reportContent.replace(assessmentDatePattern, `$1\n${modelLine}`); - logger.info('Model info injected into Executive Summary'); - } else { - // If no Assessment Date line found, try to add after Executive Summary header - const execSummaryPattern = /^## Executive Summary$/m; - if (reportContent.match(execSummaryPattern)) { - // Add model as first item in Executive Summary - reportContent = reportContent.replace(execSummaryPattern, `## Executive Summary\n- Model: ${modelStr}`); - logger.info('Model info added to Executive Summary header'); - } else { - logger.warn('Could not find Executive Summary section'); - return; - } - } - - // 5. Write modified report back - await fs.writeFile(reportPath, reportContent); +// Pure function: Assemble final report from specialist deliverables. +// Per class, prefer the exploit-agent's evidence file; fall back to renderer-produced findings. +// Both never coexist for a workspace because scope (exploit flag) is locked. +export async function assembleFinalReport( + sourceDir: string, + deliverablesSubdir: string | undefined, + logger: ActivityLogger, + optionsOrExploit: AssembleFinalReportOptions | boolean = {}, +): Promise { + const options: AssembleFinalReportOptions = + typeof optionsOrExploit === 'boolean' ? { exploit: optionsOrExploit } : optionsOrExploit; + const result = await assembleFinalReportInternal( + sourceDir, + deliverablesSubdir, + logger, + options, + options.exploit !== true, + ); + return result.content; } /** @@ -181,7 +240,7 @@ export async function injectModelIntoReport( * * The SARIF log is surfaced beside it when present, since a CI step consuming it needs a stable * path and cannot be expected to reach into the internals directory. It is absent whenever the - * run was analysis-only or `report.sarif` was set to false. + * run was analysis-only or `report.sarif` was not enabled. */ export async function copyReportToRunRoot( repoPath: string, @@ -190,29 +249,5 @@ export async function copyReportToRunRoot( logger: ActivityLogger, ): Promise { const dir = deliverablesDir(repoPath, deliverablesSubdir); - - const pdfSource = path.join(dir, ASSEMBLED_REPORT_PDF_FILENAME); - if (await fs.pathExists(pdfSource)) { - const destination = path.join(runDir, FINAL_REPORT_PDF_FILENAME); - await fs.copy(pdfSource, destination, { overwrite: true }); - logger.info(`Surfaced PDF report at ${destination}`); - } else { - logger.warn(`PDF report not found, skipping ${FINAL_REPORT_PDF_FILENAME}`); - } - - const markdownSource = path.join(dir, ASSEMBLED_REPORT_FILENAME); - if (await fs.pathExists(markdownSource)) { - const destination = path.join(runDir, FINAL_REPORT_MD_FILENAME); - await fs.copy(markdownSource, destination, { overwrite: true }); - logger.info(`Surfaced markdown report at ${destination}`); - } else { - logger.warn(`Markdown report not found, skipping ${FINAL_REPORT_MD_FILENAME}`); - } - - const sarifSource = path.join(dir, SARIF_FILENAME); - if (await fs.pathExists(sarifSource)) { - const sarifDestination = path.join(runDir, SARIF_FILENAME); - await fs.copy(sarifSource, sarifDestination, { overwrite: true }); - logger.info(`Surfaced SARIF log at ${sarifDestination}`); - } + await surfaceReportOutputs({ deliverablesDir: dir, customerDir: runDir, logger }); } diff --git a/apps/worker/src/session-manager.ts b/apps/worker/src/session-manager.ts index a58305b7..3e728d07 100644 --- a/apps/worker/src/session-manager.ts +++ b/apps/worker/src/session-manager.ts @@ -8,8 +8,16 @@ import { fs, path } from 'zx'; import type { ActivityLogger } from './types/activity-logger.js'; import type { AgentDefinition, AgentName, AgentValidator, PlaywrightSession, VulnType } from './types/index.js'; +import type { ReconciliationClass } from './types/reconciliation.js'; -// Agent definitions according to PRD +// Single source of truth for every agent the pipeline can run. Each entry: +// - name / displayName: identity used in logs, metrics, and per-agent log filenames +// - prerequisites: the agents this one conceptually depends on. This is documentation +// of the intended dependency graph, not an executed check. Actual phase ordering and +// concurrency are enforced by the explicit phase structure in the Temporal workflow. +// - promptTemplate: the file under apps/worker/prompts/ (without extension) rendered for this agent +// - deliverableFilename: the canonical filename AgentExecutionService and the +// save-deliverable CLI script write this agent's output under export const AGENTS: Readonly> = Object.freeze({ 'pre-recon': { name: 'pre-recon', @@ -95,6 +103,16 @@ export const AGENTS: Readonly> = Object.freez promptTemplate: 'exploit-authz', deliverableFilename: 'authz_exploitation_evidence.md', }, + // Internal class covering findings outside the five core vuln types (from reconciliation + // or agentic SAST). It has no analysis-phase counterpart: there is no 'miscellaneous-vuln' + // agent, since it only ever receives findings that another phase already surfaced. + 'miscellaneous-exploit': { + name: 'miscellaneous-exploit', + displayName: 'Miscellaneous exploit agent', + prerequisites: ['recon'], + promptTemplate: 'exploit-miscellaneous', + deliverableFilename: 'miscellaneous_exploitation_evidence.md', + }, report: { name: 'report', displayName: 'Report agent', @@ -121,6 +139,7 @@ export const AGENT_PHASE_MAP: Readonly> = Object.fr 'auth-exploit': 'exploitation', 'authz-exploit': 'exploitation', 'ssrf-exploit': 'exploitation', + 'miscellaneous-exploit': 'exploitation', report: 'reporting', }); @@ -147,9 +166,9 @@ function createVulnValidator(vulnType: VulnType): AgentValidator { // hook after the agent succeeds (before the success commit), so a file-existence check // here would race the renderer. // -// VulnType is kept in the import surface for createVulnValidator above; this factory -// returns a no-op validator parameterized only for symmetry with the vuln-side factory. -function createExploitValidator(_vulnType: VulnType): AgentValidator { +// Exploitation includes the analysis-less internal `miscellaneous` class, while vulnerability +// analysis remains limited to the five-class VulnType contract above. +function createExploitValidator(_vulnType: ReconciliationClass): AgentValidator { return async (): Promise => true; } @@ -179,6 +198,9 @@ export const PLAYWRIGHT_SESSION_MAPPING: Record = Obj 'exploit-ssrf': 'agent4', 'exploit-authz': 'agent5', + // Conditional analysis-less class; it may run beside the five analysis-backed exploit agents. + 'exploit-miscellaneous': 'agent6', + // Phase 5: Reporting 'report-executive': 'agent3', }); @@ -208,6 +230,7 @@ export const AGENT_VALIDATORS: Record = Object.freeze 'auth-exploit': createExploitValidator('auth'), 'ssrf-exploit': createExploitValidator('ssrf'), 'authz-exploit': createExploitValidator('authz'), + 'miscellaneous-exploit': createExploitValidator('miscellaneous'), // Executive report agent report: async (sourceDir: string, logger: ActivityLogger): Promise => { diff --git a/apps/worker/src/temporal/activities.ts b/apps/worker/src/temporal/activities.ts index 2b010b0c..a75801e4 100644 --- a/apps/worker/src/temporal/activities.ts +++ b/apps/worker/src/temporal/activities.ts @@ -15,6 +15,7 @@ * Business logic is delegated to services in src/services/. */ +import { createHash } from 'node:crypto'; import fs from 'node:fs/promises'; import path from 'node:path'; import { ApplicationFailure, Context, heartbeat } from '@temporalio/activity'; @@ -22,39 +23,88 @@ import { syncPermissionSystemConfig } from '../ai/pi/permission-system.js'; import { writePlaywrightStealthConfig } from '../ai/playwright-config-writer.js'; import { AuditSession } from '../audit/index.js'; import type { ResumeAttempt } from '../audit/metrics-tracker.js'; -import { authStateFile, generateAuditPath, generateSessionJsonPath, type SessionMetadata } from '../audit/utils.js'; +import { authStateFile, generateAuditPath, type SessionMetadata } from '../audit/utils.js'; import type { WorkflowSummary } from '../audit/workflow-logger.js'; import type { CheckpointContext } from '../interfaces/checkpoint-provider.js'; import { - ASSEMBLED_REPORT_FILENAME, ASSEMBLED_REPORT_PDF_FILENAME, DEFAULT_DELIVERABLES_SUBDIR, deliverablesDir, + REPORT_FINALIZATION_MANIFEST_FILENAME, REPORT_JSON_FILENAME, resolveSessionJsonPath, - SARIF_FILENAME, TYPST_TEMPLATE, } from '../paths.js'; import { getAgentGitPaths } from '../services/agent-git-paths.js'; +import { compactReportFindings as compactReportFindingsService } from '../services/compaction-core.js'; import { getContainer, getOrCreateContainer, removeContainer } from '../services/container.js'; import { classifyErrorForTemporal, PentestError } from '../services/error-handling.js'; +import { RenumberError } from '../services/exact-output-commit.js'; import { ExploitationCheckerService } from '../services/exploitation-checker.js'; import { renderFindingsFromQueues } from '../services/findings-renderer.js'; import { executeGitCommandWithRetry } from '../services/git-manager.js'; +import { pdfProvenanceIsCurrent } from '../services/pdf-renderer.js'; import { runPreflightChecks } from '../services/preflight.js'; import type { ExploitationDecision, VulnType } from '../services/queue-validation.js'; +import { + renumberClassFindings as renumberClassFindingsService, + sparseExploitCollectorPath, +} from '../services/renumber-core.js'; +import { + checkpointFileContents, + checkpointIsAncestor, + draftProgressIsCoherent, + finalProgressIsCoherent, + readFileAtCheckpoint, + reportCheckpointIsCoherent, + resolveCheckpointCommit, + validateDraftProgress, +} from '../services/report-checkpoints.js'; +import { + finalizeReport, + ReportFinalizationIntegrityError, + ReportSarifRenderError, +} from '../services/report-finalization.js'; +import { surfaceReportOutputs as surfaceReportOutputsService } from '../services/report-output-surface.js'; import type { ReportData, ReportMeta } from '../services/report-renderer.js'; -import { assembleFinalReport, copyReportToRunRoot, injectModelIntoReport } from '../services/reporting.js'; +import { assembleFinalReportWithEvidence } from '../services/reporting.js'; import { validateAuthentication } from '../services/validate-authentication.js'; import { AGENTS } from '../session-manager.js'; import type { AgentName } from '../types/agents.js'; -import { ALL_AGENTS } from '../types/agents.js'; import type { ContainerConfig, VulnClass } from '../types/config.js'; import { ErrorCode } from '../types/errors.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; import { isErr } from '../types/result.js'; +import { + appendPartialReasons, + assertFixedAnalysisScope, + type DurableScanState, + FIXED_ANALYSIS_CLASSES, + initialExpectedAgents, + isDurableScanState, + isOrderedPartialReasonSet, + isReportProgress, + type MiscellaneousOutcome, + type PartialReason, + type ReportProgress, + type ReportSarifDisposition, + RunStateError, + SAFE_RUN_STATE_MESSAGES, + type StoredPdfProvenance, + workspaceExploitMismatchMessage, +} from '../types/run-state.js'; import { atomicWrite, fileExists, readJson } from '../utils/file-io.js'; import { createActivityLogger } from './activity-logger.js'; -import type { AgentMetrics, PipelineState, ResumeState } from './shared.js'; +import type { + AgentMetrics, + AssembleReportActivityResult, + DurableStateSummary, + FinalizeReportActivityResult, + PipelineState, + ReconciliationActivityResult, + ResumeState, + SurfaceReportActivityResult, +} from './shared.js'; // Max lengths to prevent Temporal protobuf buffer overflow const MAX_ERROR_MESSAGE_LENGTH = 2000; @@ -87,6 +137,13 @@ export interface ActivityInput { promptDir?: string; sastSarifPath?: string; + /** Fixed workflow-resolved scope supplied to every agent prompt. */ + analysisClasses?: readonly VulnClass[]; + /** Distinguishes absent state on a fresh run from forbidden resume reconstruction. */ + stateContext?: 'fresh' | 'resume'; + /** Operational route label; no customer path crosses workflow history. */ + customerOutputRoute?: 'workspace' | 'mounted'; + // Vuln classes whose pipeline failed. Set before the report stage on a partial run so the // report marks them "not assessed" instead of asserting no findings were present. failedClasses?: VulnClass[]; @@ -135,6 +192,152 @@ function buildContainerConfig(input: ActivityInput): ContainerConfig { }; } +/** + * Classify a failure from durable scan-state persistence. A RunStateError already carries a + * caller-safe message and code, so it maps directly; anything else reaches this path without + * having touched durable state, so it falls back to the generic activity classifier. + */ +function runStateFailure(error: unknown): ApplicationFailure { + if (error instanceof RunStateError) { + return ApplicationFailure.nonRetryable(error.message, error.failureType, [{ checkCode: error.checkCode }]); + } + const classified = classifyErrorForTemporal(error); + const message = 'Durable execution-state persistence failed.'; + return classified.retryable + ? ApplicationFailure.create({ message, type: classified.type }) + : ApplicationFailure.nonRetryable(message, classified.type); +} + +/** Stable Temporal types for the two declared renumber corruption modes. */ +const RENUMBER_STABLE_FAILURE_TYPES = Object.freeze({ + 'unmappable-survivor': 'UnmappableSurvivor', + 'key-set-divergence': 'KeySetDivergence', +} as const satisfies Record); + +/** + * Classify a failure from a deterministic report-processing activity (renumber, compaction, + * finalization). Each underlying service throws its own typed error so the workflow can react + * to the specific stage that broke; this is the one place that maps those types onto stable + * Temporal failure names, so the mapping cannot drift between call sites. + */ +function deterministicActivityFailure(error: unknown, failureType: string): ApplicationFailure { + if (error instanceof RunStateError) return runStateFailure(error); + if (error instanceof ReportSarifRenderError) { + // The workflow invokes degraded finalization only after this exact retryable type + // exhausts the activity policy, so the name must survive the boundary unflattened. + return ApplicationFailure.create({ + message: error.message, + type: 'ReportSarifRenderError', + nonRetryable: false, + details: [{ stage: failureType }], + }); + } + if (error instanceof ReportFinalizationIntegrityError) { + return ApplicationFailure.nonRetryable(error.message, 'ReportFinalizationIntegrityError', [ + { checkCode: error.checkCode }, + ]); + } + if (error instanceof RenumberError) { + const details = [ + { + ...(error.details?.checkCode !== undefined && { checkCode: error.details.checkCode }), + ...(error.details?.vulnerabilityClass !== undefined && { + vulnerabilityClass: error.details.vulnerabilityClass, + }), + }, + ]; + const stableType = RENUMBER_STABLE_FAILURE_TYPES[error.type]; + return error.retryable + ? ApplicationFailure.create({ message: error.message, type: stableType, details }) + : ApplicationFailure.nonRetryable(error.message, stableType, details); + } + const classified = classifyErrorForTemporal(error); + const message = 'Deterministic report processing failed.'; + return classified.retryable + ? ApplicationFailure.create({ message, type: failureType }) + : ApplicationFailure.nonRetryable(message, failureType); +} + +/** + * The five analysis classes are fixed for a scan's whole lifetime and resolved once by the + * workflow so every agent prompt sees the same scope. An activity invoked without that scope + * is a caller bug rather than a transient condition, so it fails without a retry. + */ +function resolveAnalysisClasses(input: ActivityInput): readonly VulnClass[] { + if (input.analysisClasses === undefined) { + throw ApplicationFailure.nonRetryable( + 'Workflow-resolved analysis scope is required.', + 'IncompatibleWorkspaceError', + [{ checkCode: 'analysis-scope-missing' }], + ); + } + try { + assertFixedAnalysisScope(input.analysisClasses); + } catch (error) { + throw runStateFailure(error); + } + return input.analysisClasses; +} + +/** Project only the fields the workflow needs to update its queryable state, out of the full durable record. */ +function durableStateSummary(state: DurableScanState): DurableStateSummary { + return { + exploit: state.exploit, + expectedAgents: [...state.expected_agents], + participatingClasses: [...state.participating_classes], + reportStage: state.report?.stage ?? 'uninitialized', + ...(state.miscellaneous_outcome !== undefined && { miscellaneousOutcome: state.miscellaneous_outcome }), + }; +} + +function sha256(contents: string): string { + return createHash('sha256').update(contents, 'utf8').digest('hex'); +} + +function arraysEqual(left: readonly T[], right: readonly T[]): boolean { + return left.length === right.length && left.every((value, index) => value === right[index]); +} + +async function rollbackInvalidReportDraft(input: ActivityInput, progress: ReportProgress): Promise { + if (progress.stage !== 'draft') { + throw new RunStateError('CorruptedSessionError', 'invalid-nondraft-report-progress'); + } + const deliverablesPath = deliverablesDir(input.repoPath, input.deliverablesSubdir); + // Transient resolution failures throw for retry; only proven absence or corruption of the + // draft parent is treated as corrupted workspace state. + const parentCheckpoint = await resolveCheckpointCommit(deliverablesPath, `${progress.model_checkpoint}^`); + if (parentCheckpoint === null) { + throw new RunStateError('CorruptedSessionError', 'report-draft-parent-unavailable'); + } + + for (const relPath of getAgentGitPaths('report')) { + const prior = await readFileAtCheckpoint(deliverablesPath, parentCheckpoint, relPath); + if (prior.state === 'corrupt') { + throw new RunStateError('CorruptedSessionError', 'report-draft-parent-corrupt'); + } + if (prior.state === 'absent') { + await executeGitCommandWithRetry( + ['git', 'rm', '--cached', '--ignore-unmatch', '--', relPath], + deliverablesPath, + 'unstage invalid report draft path', + ); + await fs.unlink(path.join(deliverablesPath, relPath)).catch((error: unknown) => { + if (!(error instanceof Error && (error as NodeJS.ErrnoException).code === 'ENOENT')) throw error; + }); + continue; + } + await executeGitCommandWithRetry( + ['git', 'restore', `--source=${parentCheckpoint}`, '--staged', '--worktree', '--', relPath], + deliverablesPath, + 'restore invalid report draft path', + ); + } + + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + await auditSession.rollbackReportDraft(); +} + /** * Core activity implementation using services. * @@ -143,12 +346,20 @@ function buildContainerConfig(input: ActivityInput): ContainerConfig { * 2. Container creation/reuse * 3. Service-based agent execution * 4. Error classification for Temporal retry + * + * `successDisposition` tells the service what "done" means for this agent: ordinary agents + * commit a terminal deliverable, while the report agent instead produces a draft that still + * needs canonical checkpointing before it is finished. `allowCheckpointSkip` is false only for + * the report agent, whose skip decision belongs to the durable report-progress state machine + * rather than the generic checkpoint provider used by every other agent. */ async function runAgentActivity( agentName: AgentName, input: ActivityInput, customTools?: import('@earendil-works/pi-coding-agent').ToolDefinition[], - writeDeliverable?: (deliverablesPath: string) => Promise, + writeDeliverable?: (deliverablesPath: string, execution: { readonly model?: string }) => Promise, + successDisposition: 'terminal' | 'report-draft' = 'terminal', + allowCheckpointSkip = true, ): Promise { const { repoPath, configPath, pipelineTestingMode = false, workflowId, webUrl } = input; @@ -157,13 +368,15 @@ async function runAgentActivity( const skipContainer = getContainer(workflowId) ?? getOrCreateContainer(workflowId, buildSessionMetadata(input), buildContainerConfig(input)); - const decision = await skipContainer.checkpointProvider.shouldSkipAgent( - agentName, - repoPath, - input.deliverablesSubdir ?? DEFAULT_DELIVERABLES_SUBDIR, - ); - if (decision.skip && decision.metrics) { - return { ...decision.metrics, skipped: true }; + if (allowCheckpointSkip) { + const decision = await skipContainer.checkpointProvider.shouldSkipAgent( + agentName, + repoPath, + input.deliverablesSubdir ?? DEFAULT_DELIVERABLES_SUBDIR, + ); + if (decision.skip && decision.metrics) { + return { ...decision.metrics, skipped: true }; + } } const startTime = Date.now(); @@ -199,11 +412,13 @@ async function runAgentActivity( configPath, pipelineTestingMode, attemptNumber, + analysisClasses: resolveAnalysisClasses(input), ...(input.promptDir !== undefined && { promptDir: input.promptDir }), ...(input.configYAML !== undefined && { configYAML: input.configYAML }), ...(input.failedClasses !== undefined && { failedClasses: input.failedClasses }), ...(customTools && { customTools }), ...(writeDeliverable && { writeDeliverable }), + successDisposition, cancellationSignal: Context.current().cancellationSignal, }, auditSession, @@ -220,6 +435,7 @@ async function runAgentActivity( costUsd: endResult.cost_usd, numTurns: endResult.turns ?? null, model: endResult.model, + ...(endResult.checkpoint !== undefined && { checkpoint: endResult.checkpoint }), }; } catch (error) { // If error is already an ApplicationFailure, re-throw directly @@ -392,8 +608,14 @@ async function readExploitQueue(queuePath: string): Promise<{ validIds: Set { const { createExploitCollector } = await import('../collectors/exploit-collector.js'); @@ -420,6 +642,10 @@ async function runExploitAgentWithCollector( missing: missingIds.length, }); + const collectorRelPath = sparseExploitCollectorPath(vulnClass); + await atomicWrite(path.join(deliverablesPath, collectorRelPath), `${JSON.stringify(collected, null, 2)}\n`); + logger.info(`Wrote ${collectorRelPath} with ${collected.length} collector entries`); + const markdown = renderExploitDeliverable(vulnClass, collected, idToType); const mdPath = path.join(deliverablesPath, `${vulnClass}_exploitation_evidence.md`); await atomicWrite(mdPath, markdown); @@ -449,83 +675,65 @@ export async function runAuthzExploitAgent(input: ActivityInput): Promise, -): Promise { - if (!exploit) return; - - const container = getOrCreateContainer(input.workflowId, buildSessionMetadata(input), buildContainerConfig(input)); - const configResult = await container.configLoader.loadOptional(input.configPath, undefined, input.configYAML); - // Only an explicit false opts out; a missing config keeps the default on. - if (isErr(configResult) || configResult.value?.report?.sarif === false) return; - - try { - const { renderSarif } = await import('../services/sarif-renderer.js'); - const sarif = renderSarif(reportData, { workspaceName: input.sessionId }); - await atomicWrite(path.join(deliverablesPath, SARIF_FILENAME), sarif); - logger.info(`Wrote ${SARIF_FILENAME}`); - } catch (error) { - logger.warn(`Failed to write ${SARIF_FILENAME}: ${(error as Error).message}`); - } -} - -/** - * Compile the PDF report from the assembled report data. - * - * Failures are logged and swallowed — the PDF is a secondary artifact and must not fail a run - * whose report is already written. - */ -async function writePdfReport( - reportData: ReportData, - deliverablesPath: string, - logger: ReturnType, -): Promise { - try { - const { renderReportPdf } = await import('../services/pdf-renderer.js'); - await renderReportPdf({ - reportData, - templatePath: TYPST_TEMPLATE, - outputPath: path.join(deliverablesPath, ASSEMBLED_REPORT_PDF_FILENAME), - }); - logger.info(`Wrote ${ASSEMBLED_REPORT_PDF_FILENAME}`); - } catch (error) { - logger.warn(`Failed to write ${ASSEMBLED_REPORT_PDF_FILENAME}: ${(error as Error).message}`); - } +/** Run the ordinary collector-backed agent after durable admission appended it to the expected set. */ +export async function runMiscellaneousExploitAgent(input: ActivityInput): Promise { + return runExploitAgentWithCollector('miscellaneous-exploit', 'miscellaneous', input); } export async function runReportAgent(input: ActivityInput, exploit: boolean): Promise { const { createFindingCollector } = await import('../collectors/finding-collector.js'); - const { renderReport } = await import('../services/report-renderer.js'); + + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + let durableState = await auditSession.getDurableScanState(); + if (durableState.exploit !== exploit) { + throw runStateFailure(new RunStateError('IncompatibleWorkspaceError', 'report-exploit-mode-mismatch')); + } + if (durableState.report === undefined) { + throw runStateFailure(new RunStateError('DurableStateConflictError', 'report-progress-not-initialized')); + } + let reportProgress = durableState.report; + const deliverablesPath = deliverablesDir(input.repoPath, input.deliverablesSubdir); + if (reportProgress.stage === 'draft') { + const validation = await validateDraftProgress(deliverablesPath, reportProgress); + if (validation === 'invalid-model') { + await rollbackInvalidReportDraft(input, reportProgress); + durableState = await auditSession.getDurableScanState(); + if (durableState.report?.stage !== 'pending') { + throw runStateFailure(new RunStateError('CorruptedSessionError', 'report-draft-rollback-did-not-persist')); + } + reportProgress = durableState.report; + } else if (validation === 'invalid-canonical') { + throw runStateFailure(new RunStateError('CorruptedSessionError', 'report-canonical-checkpoint-invalid')); + } + } + if (reportProgress.stage === 'finalized' && !(await finalProgressIsCoherent(deliverablesPath, reportProgress))) { + throw runStateFailure(new RunStateError('CorruptedSessionError', 'report-final-proof-invalid')); + } + if (reportProgress.stage !== 'pending') { + return { ...(await auditSession.getReportMetrics()), skipped: true }; + } const collector = createFindingCollector(exploit); - const writeDeliverable = async (deliverablesPath: string): Promise => { + const writeDeliverable = async (deliverablesPath: string, execution: { readonly model?: string }): Promise => { const logger = createActivityLogger(); const { attachQueueCodeLocations } = await import('../services/code-location-join.js'); const collected = collector.getAll(); logger.info(`Collected ${collected.length} finding(s) from report agent`); - const findings = await attachQueueCodeLocations(collected, deliverablesPath, logger); + const findings = await attachQueueCodeLocations( + collected, + deliverablesPath, + logger, + durableState.participating_classes, + ); // report_meta is written by the set-report-meta CLI while the agent runs; read it back so // the two halves of report.json end up in one document. const reportJsonPath = path.join(deliverablesPath, REPORT_JSON_FILENAME); let reportMeta: ReportMeta = { target: input.webUrl, - assessment_date: new Date().toISOString().split('T')[0]!, + assessment_date: new Date().toISOString().slice(0, 10), scope: '', executive_summary: '', exploit, @@ -539,34 +747,32 @@ export async function runReportAgent(input: ActivityInput, exploit: boolean): Pr assessment_date: String(existing.report_meta.assessment_date ?? reportMeta.assessment_date), scope: String(existing.report_meta.scope ?? ''), executive_summary: String(existing.report_meta.executive_summary ?? ''), - // Run scope, not agent output — keeps the rendered report and the schema the agent - // was given in agreement. exploit, - ...(existing.report_meta.model !== undefined && { model: String(existing.report_meta.model) }), + ...(execution.model !== undefined && { model: execution.model }), }; } } catch { logger.warn('Failed to read report_meta from report.json, using defaults'); } } + if (execution.model !== undefined) { + reportMeta = { ...reportMeta, model: execution.model }; + } else { + logger.warn('Report execution returned no model identifier; canonical report metadata omits model'); + } const reportData: ReportData = { report_meta: reportMeta, findings, ...(input.failedClasses && input.failedClasses.length > 0 && { not_assessed: input.failedClasses }), + reconciliation_failed: [...reportProgress.renumber_failed_classes], }; - await atomicWrite(reportJsonPath, JSON.stringify(reportData, null, 2)); + await atomicWrite(reportJsonPath, `${JSON.stringify(reportData, null, 2)}\n`); logger.info(`Wrote ${REPORT_JSON_FILENAME} with ${findings.length} finding(s)`); - - await atomicWrite(path.join(deliverablesPath, ASSEMBLED_REPORT_FILENAME), renderReport(reportData)); - logger.info(`Wrote ${ASSEMBLED_REPORT_FILENAME} from structured data`); - - await writePdfReport(reportData, deliverablesPath, logger); - await writeSarifIfEnabled(input, exploit, reportData, deliverablesPath, logger); }; - return runAgentActivity('report', input, collector.tools, writeDeliverable); + return runAgentActivity('report', input, collector.tools, writeDeliverable, 'report-draft', false); } /** @@ -798,49 +1004,375 @@ export async function syncCodePathDenyRules(input: ActivityInput): Promise ); } +/** Initialize fresh state exactly once, or validate the persisted resume contract. */ +export async function initializeDurableScanState( + input: ActivityInput, + exploit: boolean, + context: 'fresh' | 'resume', +): Promise { + try { + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initializeDurableScanState(input.workflowId, exploit, context); + return durableStateSummary(await auditSession.getDurableScanState()); + } catch (error) { + throw runStateFailure(error); + } +} + +/** Persist the `miscellaneous` queue decision before any conditional agent is scheduled. */ +export async function persistMiscellaneousOutcome( + input: ActivityInput, + outcome: MiscellaneousOutcome, +): Promise { + try { + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + return durableStateSummary(await auditSession.updateMiscellaneousOutcome(outcome)); + } catch (error) { + throw runStateFailure(error); + } +} + +// === Report progress state machine === +// The activities below read and advance one durable ReportProgress record through +// pending -> draft -> finalized. Every transition is verified against the actual git +// checkpoint before it is trusted (checkpointIsAncestor, reportCheckpointIsCoherent, +// finalProgressIsCoherent), so a crash between "committed the work" and "recorded the state" +// is repaired on the next attempt instead of silently accepted or silently lost. + +/** Persist the ordered reconciliation-failure set and durable partial reasons before assembly. */ +export async function initializeReportProgress( + input: ActivityInput, + failedClasses: readonly ReconciliationClass[], + partialReasons: readonly PartialReason[], +): Promise { + try { + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + return await auditSession.initializeReportProgress(failedClasses, partialReasons); + } catch (error) { + throw runStateFailure(error); + } +} + +/** Invoke the class-local exact-output renumber service with a history-safe receipt. */ +export async function renumberClassFindings( + input: ActivityInput, + vulnerabilityClass: ReconciliationClass, +): Promise { + try { + const result = await renumberClassFindingsService({ + deliverablesDir: deliverablesDir(input.repoPath, input.deliverablesSubdir), + vulnerabilityClass, + logger: createActivityLogger(), + }); + return { + vulnerabilityClass, + skipped: result.skipped, + changedPathCount: result.commit?.changedPaths.length ?? 0, + ...(result.commit !== undefined && { + checkpoint: result.commit.commitHash, + alreadyCommitted: result.commit.alreadyCommitted, + }), + }; + } catch (error) { + throw deterministicActivityFailure(error, 'ReportRenumberError'); + } +} + /** * Assemble the final report by concatenating per-class deliverables. * - * Under exploit=true, each exploit agent has produced `*_exploitation_evidence.md` - * directly. Under exploit=false, exploit agents didn't run; we deterministically - * render `*_findings.md` from each `*_exploitation_queue.json` first, then assemble. + * Under exploit=true, each exploit agent writes sparse evidence and a successful + * renumber replaces it with the dense render. Under exploit=false, exploit agents + * didn't run; we deterministically render `*_findings.md` from each + * `*_exploitation_queue.json` first, then assemble. */ -export async function assembleReportActivity(input: ActivityInput, exploit: boolean): Promise { +export async function assembleReportActivity( + input: ActivityInput, + exploit: boolean, +): Promise { const { repoPath, deliverablesSubdir } = input; const logger = createActivityLogger(); + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + const durableState = await auditSession.getDurableScanState(); + if (durableState.exploit !== exploit || durableState.report?.stage !== 'pending') { + throw runStateFailure(new RunStateError('DurableStateConflictError', 'report-assembly-stage-mismatch')); + } + let renderFailedClasses: readonly ReconciliationClass[] = []; if (!exploit) { logger.info('Rendering per-class findings from analysis queues...'); try { - await renderFindingsFromQueues(repoPath, deliverablesSubdir, logger); + const rendered = await renderFindingsFromQueues( + repoPath, + deliverablesSubdir, + logger, + durableState.participating_classes, + ); + renderFailedClasses = rendered.failedClasses; } catch (error) { - const err = error as Error; - logger.warn(`Error rendering findings from queues: ${err.message}`); + throw deterministicActivityFailure(error, 'ReportAssemblyError'); } } logger.info('Assembling deliverables from specialist agents...'); try { - await assembleFinalReport(repoPath, deliverablesSubdir, logger); + const assembled = await assembleFinalReportWithEvidence(repoPath, deliverablesSubdir, logger, { + exploit, + participatingClasses: durableState.participating_classes, + knownFailedClasses: renderFailedClasses, + }); + return { failedClasses: assembled.failedClasses }; } catch (error) { - const err = error as Error; - logger.warn(`Error assembling final report: ${err.message}`); + throw deterministicActivityFailure(error, 'ReportAssemblyError'); + } +} + +/** Compact a coherent draft using only persisted membership and failure order. */ +export async function compactReportFindings(input: ActivityInput): Promise { + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + const state = await auditSession.getDurableScanState(); + if (state.report?.stage !== 'draft') { + throw runStateFailure(new RunStateError('DurableStateConflictError', 'report-compaction-stage-mismatch')); + } + const deliverablesPath = deliverablesDir(input.repoPath, input.deliverablesSubdir); + const draftValidation = await validateDraftProgress(deliverablesPath, state.report); + if (draftValidation === 'invalid-model') { + await rollbackInvalidReportDraft(input, state.report); + throw runStateFailure(new RunStateError('CorruptedSessionError', 'report-draft-rolled-back')); + } + if (draftValidation === 'invalid-canonical') { + throw runStateFailure(new RunStateError('CorruptedSessionError', 'report-canonical-checkpoint-invalid')); + } + + let result: Awaited>; + try { + result = await compactReportFindingsService({ + deliverablesDir: deliverablesPath, + participatingClasses: state.participating_classes, + renumberFailedClasses: state.report.renumber_failed_classes, + logger: createActivityLogger(), + }); + } catch (error) { + throw deterministicActivityFailure(error, 'ReportCompactionError'); + } + return { + skipped: result.skipped, + changedPathCount: result.commit?.changedPaths.length ?? 0, + ...(result.commit !== undefined && { + checkpoint: result.commit.commitHash, + alreadyCommitted: result.commit.alreadyCommitted, + }), + }; +} + +/** Persist and validate the canonical structured checkpoint after compaction. */ +export async function persistCanonicalReportProgress( + input: ActivityInput, + checkpoint: string, + appendReasons: readonly PartialReason[] = [], +): Promise { + try { + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + const state = await auditSession.getDurableScanState(); + if (state.report?.stage !== 'draft') { + throw new RunStateError('DurableStateConflictError', 'report-canonical-stage-mismatch'); + } + const deliverablesPath = deliverablesDir(input.repoPath, input.deliverablesSubdir); + if ( + !(await checkpointIsAncestor(state.report.model_checkpoint, checkpoint, deliverablesPath)) || + !(await reportCheckpointIsCoherent(deliverablesPath, checkpoint, state.report.renumber_failed_classes)) + ) { + throw new RunStateError('CorruptedSessionError', 'report-canonical-checkpoint-invalid'); + } + return await auditSession.recordCanonicalReportCheckpoint(checkpoint, appendReasons); + } catch (error) { + throw runStateFailure(error); } } /** - * Inject model metadata into the final report. + * Commit exact canonical outputs and regenerate the derived PDF without terminal promotion. + * + * `degradedSarif` is passed by the workflow only after the retryable `ReportSarifRenderError` + * type has exhausted the ordinary three-attempt policy; the service still runs its adoption + * check first, so a coherent earlier commit is adopted instead of degraded. */ -export async function injectReportMetadataActivity(input: ActivityInput): Promise { - const { repoPath, sessionId, outputPath, deliverablesSubdir } = input; - const logger = createActivityLogger(); - const effectiveOutputPath = outputPath ? path.join(outputPath, sessionId) : path.join('./workspaces', sessionId); - try { - await injectModelIntoReport(repoPath, deliverablesSubdir, effectiveOutputPath, logger); - } catch (error) { - const err = error as Error; - logger.warn(`Error injecting model into report: ${err.message}`); +export async function finalizeReportOutputs( + input: ActivityInput, + degradedSarif = false, +): Promise { + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + const state = await auditSession.getDurableScanState(); + if ( + state.report === undefined || + state.report.stage === 'pending' || + (state.report.stage === 'draft' && state.report.canonical_checkpoint === undefined) + ) { + throw runStateFailure(new RunStateError('DurableStateConflictError', 'report-finalization-stage-mismatch')); } + + const deliverablesPath = deliverablesDir(input.repoPath, input.deliverablesSubdir); + const reportIsCoherent = + state.report.stage === 'finalized' + ? await finalProgressIsCoherent(deliverablesPath, state.report) + : await draftProgressIsCoherent(deliverablesPath, state.report); + if (!reportIsCoherent) { + throw runStateFailure(new RunStateError('CorruptedSessionError', 'report-finalization-checkpoint-invalid')); + } + const priorPdfProvenance = state.report.stage === 'finalized' ? state.report.pdf_provenance : undefined; + + const container = getOrCreateContainer(input.workflowId, buildSessionMetadata(input), buildContainerConfig(input)); + const configResult = await container.configLoader.loadOptional(input.configPath, undefined, input.configYAML); + if (isErr(configResult)) throw deterministicActivityFailure(configResult.error, 'ReportFinalizationError'); + // Preserve public main's default-on exploit SARIF behavior when no report config is present. + const reportConfig = configResult.value?.report ?? { sarif: true }; + let result: Awaited>; + try { + result = await finalizeReport({ + deliverablesDir: deliverablesPath, + exploit: state.exploit, + reconciliationFailedClasses: state.report.renumber_failed_classes, + reportConfig, + workspaceName: input.sessionId, + logger: createActivityLogger(), + templatePath: TYPST_TEMPLATE, + ...(degradedSarif && { degradedSarif }), + ...(priorPdfProvenance !== undefined && { priorPdfProvenance }), + }); + } catch (error) { + throw deterministicActivityFailure(error, 'ReportFinalizationError'); + } + const manifestContents = `${JSON.stringify(result.manifest, null, 2)}\n`; + return { + checkpoint: result.commit.commitHash, + manifestSha256: sha256(manifestContents), + changedPathCount: result.commit.changedPaths.length, + alreadyCommitted: result.commit.alreadyCommitted, + sarifDisposition: result.manifest.artifacts.sarif.disposition, + pdfGenerated: result.pdfGenerated, + pdfProvenance: result.pdfProvenance, + warningCount: result.warnings.length, + }; +} + +export interface FinalizedReportPersistence { + readonly sarifDisposition: ReportSarifDisposition; + readonly pdfProvenance: StoredPdfProvenance | null; + readonly partialReasons: readonly PartialReason[]; +} + +/** + * Verify exact final bytes, then atomically promote report to the only terminal state. + * `final_checkpoint` and the manifest digest are strict match-or-conflict fields; the PDF + * provenance is replaceable, and partial reasons are append-only. + */ +export async function persistFinalizedReportProgress( + input: ActivityInput, + checkpoint: string, + manifestSha256: string, + terminal: FinalizedReportPersistence, +): Promise { + try { + if (!isOrderedPartialReasonSet(terminal.partialReasons)) { + throw new RunStateError('DurableStateConflictError', 'report-terminal-reasons-invalid'); + } + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + const state = await auditSession.getDurableScanState(); + if (state.report?.stage === 'finalized') { + if ( + state.report.final_checkpoint !== checkpoint || + state.report.finalization_manifest_sha256 !== manifestSha256 + ) { + throw new RunStateError('DurableStateConflictError', 'report-finalized-retry-conflict'); + } + if (!(await finalProgressIsCoherent(deliverablesDir(input.repoPath, input.deliverablesSubdir), state.report))) { + throw new RunStateError('CorruptedSessionError', 'report-finalized-proof-corrupt'); + } + return await auditSession.finalizeReportProgress(checkpoint, manifestSha256, terminal); + } + if (state.report?.stage !== 'draft' || state.report.canonical_checkpoint === undefined) { + throw new RunStateError('DurableStateConflictError', 'report-terminal-stage-mismatch'); + } + const candidate: ReportProgress = { + stage: 'finalized', + renumber_failed_classes: [...state.report.renumber_failed_classes], + partial_reasons: appendPartialReasons(state.report.partial_reasons, terminal.partialReasons), + model_checkpoint: state.report.model_checkpoint, + canonical_checkpoint: state.report.canonical_checkpoint, + final_checkpoint: checkpoint, + finalization_manifest_sha256: manifestSha256, + sarif_disposition: terminal.sarifDisposition, + ...(terminal.pdfProvenance !== null && { pdf_provenance: terminal.pdfProvenance }), + }; + if (!(await finalProgressIsCoherent(deliverablesDir(input.repoPath, input.deliverablesSubdir), candidate))) { + throw new RunStateError('CorruptedSessionError', 'report-final-proof-invalid'); + } + return await auditSession.finalizeReportProgress(checkpoint, manifestSha256, terminal); + } catch (error) { + throw runStateFailure(error); + } +} + +/** Best-effort customer publication after durable terminal promotion. */ +export async function surfaceReportOutputs(input: ActivityInput): Promise { + const auditSession = new AuditSession(buildSessionMetadata(input)); + await auditSession.initialize(input.workflowId); + const state = await auditSession.getDurableScanState(); + const deliverablesPath = deliverablesDir(input.repoPath, input.deliverablesSubdir); + if (state.report?.stage !== 'finalized' || !(await finalProgressIsCoherent(deliverablesPath, state.report))) { + throw runStateFailure(new RunStateError('CorruptedSessionError', 'report-surface-terminal-proof-invalid')); + } + + // The manifest just proved coherent, so the canonical digest read cannot miss here. + const manifestContents = await checkpointFileContents( + deliverablesPath, + state.report.final_checkpoint, + REPORT_FINALIZATION_MANIFEST_FILENAME, + ); + if (manifestContents === null) { + throw runStateFailure(new RunStateError('CorruptedSessionError', 'report-surface-terminal-proof-invalid')); + } + const manifest = JSON.parse(manifestContents) as { artifacts: { report_json: { sha256: string } } }; + const canonicalReportSha256 = manifest.artifacts.report_json.sha256; + + // Only provenance that matches the current renderer, template, and PDF bytes is passed + // through; anything else surfaces as null so stale customer output is removed. + let verifiedProvenance: StoredPdfProvenance | null = null; + const storedProvenance = state.report.pdf_provenance; + if ( + storedProvenance !== undefined && + (await pdfProvenanceIsCurrent({ + pdfPath: path.join(deliverablesPath, ASSEMBLED_REPORT_PDF_FILENAME), + canonicalReportSha256, + provenance: storedProvenance, + templatePath: TYPST_TEMPLATE, + })) + ) { + verifiedProvenance = storedProvenance; + } + + const customerDir = + input.customerOutputRoute === 'mounted' + ? '/app/output' + : generateAuditPath({ id: input.sessionId, webUrl: input.webUrl, repoPath: input.repoPath }); + const result = await surfaceReportOutputsService({ + deliverablesDir: deliverablesPath, + customerDir, + logger: createActivityLogger(), + pdfVerification: { canonicalReportSha256, provenance: verifiedProvenance }, + }); + return { + surfaced: result.surfaced, + removedStale: result.removedStale, + warningCount: result.warnings.length, + }; } /** @@ -884,11 +1416,6 @@ export async function checkExploitationQueue(input: ActivityInput, vulnType: Vul } } -interface RunScope { - vulnClasses: VulnClass[]; - exploit: boolean; -} - interface SessionJson { session: { id: string; @@ -896,7 +1423,6 @@ interface SessionJson { repoPath?: string; originalWorkflowId?: string; resumeAttempts?: ResumeAttempt[]; - scope?: RunScope; }; metrics: { agents: Record< @@ -907,26 +1433,41 @@ interface SessionJson { } >; }; + durableScanState?: unknown; +} + +export interface ResumeLoadOptions { + readonly deliverablesSubdir?: string; + readonly expectedExploit?: boolean; } /** * Load resume state from an existing workspace. + * + * Every completion signal here is cross-checked against independent evidence (a deliverable + * file on disk, a git checkpoint, the durable scan-state record) rather than trusted from + * session.json alone, because a crash can leave the metrics ledger and the filesystem + * disagreeing about what actually finished. */ export async function loadResumeState( workspaceName: string, expectedUrl: string, expectedRepoPath: string, - deliverablesSubdir?: string, + optionsOrDeliverablesSubdir?: ResumeLoadOptions | string, ): Promise { + const options: ResumeLoadOptions = + typeof optionsOrDeliverablesSubdir === 'string' + ? { deliverablesSubdir: optionsOrDeliverablesSubdir } + : (optionsOrDeliverablesSubdir ?? {}); + const deliverablesSubdir = options.deliverablesSubdir; // 1. Validate workspace exists (prefers .shannon/, falls back to legacy run-root layout) const sessionPath = resolveSessionJsonPath(path.join('./workspaces', workspaceName)); const exists = await fileExists(sessionPath); if (!exists) { - throw ApplicationFailure.nonRetryable( - `Workspace not found: ${workspaceName}\nExpected path: ${sessionPath}`, - 'WorkspaceNotFoundError', - ); + throw ApplicationFailure.nonRetryable(SAFE_RUN_STATE_MESSAGES.CorruptedSessionError, 'WorkspaceNotFoundError', [ + { checkCode: 'session-json-missing' }, + ]); } // 2. Parse session.json and validate URL match @@ -934,25 +1475,102 @@ export async function loadResumeState( try { session = await readJson(sessionPath); } catch (error) { - const errorMsg = error instanceof Error ? error.message : String(error); + throw ApplicationFailure.nonRetryable(SAFE_RUN_STATE_MESSAGES.CorruptedSessionError, 'CorruptedSessionError', [ + { checkCode: 'session-json-not-json' }, + ]); + } + + if (!isDurableScanState(session.durableScanState)) { + // A missing record means a different release wrote this workspace; a malformed one + // means the record itself is damaged. Each refusal keeps its own wording. + const durableStateMissing = session.durableScanState === undefined; throw ApplicationFailure.nonRetryable( - `Corrupted session.json in workspace ${workspaceName}: ${errorMsg}`, - 'CorruptedSessionError', + durableStateMissing + ? SAFE_RUN_STATE_MESSAGES.IncompatibleWorkspaceError + : SAFE_RUN_STATE_MESSAGES.CorruptedSessionError, + durableStateMissing ? 'IncompatibleWorkspaceError' : 'CorruptedSessionError', + [{ checkCode: durableStateMissing ? 'durable-state-missing' : 'durable-state-malformed' }], + ); + } + let durableState: DurableScanState = structuredClone(session.durableScanState); + if (options.expectedExploit !== undefined && durableState.exploit !== options.expectedExploit) { + throw ApplicationFailure.nonRetryable( + workspaceExploitMismatchMessage(durableState.exploit), + 'IncompatibleWorkspaceError', + [{ checkCode: 'exploit-mode-changed' }], ); } if (session.session.webUrl !== expectedUrl) { throw ApplicationFailure.nonRetryable( - `URL mismatch with workspace\n Workspace URL: ${session.session.webUrl}\n Provided URL: ${expectedUrl}`, + 'This workspace was created for a different target URL, so it cannot be resumed against this one. Check -u, or start a new scan with a different -w name.', 'URLMismatchError', ); } // 3. Cross-check agent status with deliverables on disk - const completedAgents: string[] = []; + const completedAgents: AgentName[] = []; const agents = session.metrics.agents; + const deliverablesPath = deliverablesDir(expectedRepoPath, deliverablesSubdir); - for (const agentName of ALL_AGENTS) { + const miscellaneousAgentSucceeded = agents['miscellaneous-exploit']?.status === 'success'; + if (miscellaneousAgentSucceeded !== (durableState.miscellaneous_outcome === 'completed')) { + throw ApplicationFailure.nonRetryable( + 'The admitted miscellaneous-agent state conflicts with persisted metrics.', + 'CorruptedSessionError', + [{ checkCode: 'miscellaneous-outcome-metrics-divergence' }], + ); + } + const reportAgentSucceeded = agents.report?.status === 'success'; + if (reportAgentSucceeded !== (durableState.report?.stage === 'finalized')) { + throw ApplicationFailure.nonRetryable( + 'The report terminal state conflicts with persisted metrics.', + 'CorruptedSessionError', + [{ checkCode: 'report-stage-metrics-divergence' }], + ); + } + + if (durableState.report?.stage === 'draft') { + const draftValidation = await validateDraftProgress(deliverablesPath, durableState.report); + if (draftValidation === 'invalid-model') { + await rollbackInvalidReportDraft( + { + webUrl: expectedUrl, + repoPath: expectedRepoPath, + workflowId: session.session.originalWorkflowId ?? session.session.id, + sessionId: workspaceName, + ...(deliverablesSubdir !== undefined && { deliverablesSubdir }), + }, + durableState.report, + ); + durableState = { + ...durableState, + report: { + stage: 'pending', + renumber_failed_classes: [...durableState.report.renumber_failed_classes], + partial_reasons: [...durableState.report.partial_reasons], + }, + }; + } else if (draftValidation === 'invalid-canonical') { + throw ApplicationFailure.nonRetryable('The canonical report checkpoint is invalid.', 'CorruptedSessionError', [ + { checkCode: 'report-canonical-checkpoint-invalid' }, + ]); + } + } + if ( + durableState.report?.stage === 'finalized' && + !(await finalProgressIsCoherent(deliverablesPath, durableState.report)) + ) { + throw ApplicationFailure.nonRetryable('The terminal report checkpoint proof is invalid.', 'CorruptedSessionError', [ + { checkCode: 'report-final-proof-invalid' }, + ]); + } + + for (const agentName of durableState.expected_agents) { + if (agentName === 'report') { + if (durableState.report?.stage === 'finalized') completedAgents.push(agentName); + continue; + } const agentData = agents[agentName]; if (!agentData || agentData.status !== 'success') { continue; @@ -975,6 +1593,13 @@ export async function loadResumeState( const checkpoints = completedAgents .map((name) => agents[name]?.checkpoint) .filter((hash): hash is string => hash != null); + if (durableState.report?.stage === 'draft') { + checkpoints.push(durableState.report.model_checkpoint); + if (durableState.report.canonical_checkpoint !== undefined) { + checkpoints.push(durableState.report.canonical_checkpoint); + } + } + if (durableState.report?.stage === 'finalized') checkpoints.push(durableState.report.final_checkpoint); if (checkpoints.length === 0) { const successAgents = Object.entries(agents) @@ -993,7 +1618,6 @@ export async function loadResumeState( } // 5. Find the most recent checkpoint commit - const deliverablesPath = deliverablesDir(expectedRepoPath, deliverablesSubdir); const checkpointHash = await findLatestCommit(deliverablesPath, checkpoints); const originalWorkflowId = session.session.originalWorkflowId || session.session.id; @@ -1011,52 +1635,35 @@ export async function loadResumeState( completedAgents, checkpointHash, originalWorkflowId, + expectedAgents: [...durableState.expected_agents], + participatingClasses: [...durableState.participating_classes], + exploit: durableState.exploit, + ...(durableState.report !== undefined && { reportProgress: structuredClone(durableState.report) }), + ...(durableState.miscellaneous_outcome !== undefined && { + miscellaneousOutcome: durableState.miscellaneous_outcome, + }), }; } -/** First run records scope into session.json; resume runs throw if it differs. */ +/** Transitional workflow signature backed by the durable initializer, never by static reconstruction. */ export async function persistOrValidateRunScope( input: ActivityInput, vulnClasses: VulnClass[], exploit: boolean, ): Promise { - const sessionMetadata = buildSessionMetadata(input); - const auditSession = new AuditSession(sessionMetadata); - await auditSession.initialize(input.workflowId); - - const sessionPath = generateSessionJsonPath(sessionMetadata); - let session: SessionJson; try { - session = await readJson(sessionPath); + assertFixedAnalysisScope(vulnClasses); } catch (error) { - const rawMessage = error instanceof Error ? error.message : String(error); + throw runStateFailure(error); + } + if (input.stateContext === undefined) { throw ApplicationFailure.nonRetryable( - `Corrupted session.json in workspace ${input.sessionId}: ${rawMessage}`, - 'CorruptedSessionError', + 'The workflow must identify fresh or resume initialization explicitly.', + 'IncompatibleWorkspaceError', + [{ checkCode: 'state-context-missing' }], ); } - - if (session.session.scope) { - const recorded = session.session.scope; - const sameClasses = - recorded.vulnClasses.length === vulnClasses.length && - recorded.vulnClasses.every((c) => vulnClasses.includes(c)) && - vulnClasses.every((c) => recorded.vulnClasses.includes(c)); - - if (!sameClasses || recorded.exploit !== exploit) { - throw ApplicationFailure.nonRetryable( - `Resume scope mismatch for workspace ${input.sessionId}.\n` + - ` Original: vuln_classes=[${recorded.vulnClasses.join(', ')}], exploit=${recorded.exploit}\n` + - ` Provided: vuln_classes=[${vulnClasses.join(', ')}], exploit=${exploit}\n` + - `Resume requires the same scope as the original run. Start a new workspace if you want different scope.`, - 'ScopeMismatchError', - ); - } - return; - } - - session.session.scope = { vulnClasses: [...vulnClasses], exploit }; - await atomicWrite(sessionPath, session); + await initializeDurableScanState(input, exploit, input.stateContext); } async function findLatestCommit(gitDir: string, commitHashes: string[]): Promise { @@ -1092,7 +1699,48 @@ export async function restoreGitCheckpoint( checkpointHash: string, incompleteAgents: AgentName[], deliverablesSubdir?: string, + durable?: { + readonly expectedAgents: readonly AgentName[]; + readonly participatingClasses: readonly ReconciliationClass[]; + readonly reportProgress?: ReportProgress; + }, ): Promise { + // Restore membership must come from the durable scan-state record, never be reconstructed + // from the caller's own agent list, so a corrupted or stale caller can never make this + // activity delete deliverables that are still needed. + if (durable === undefined) { + throw ApplicationFailure.nonRetryable( + 'Persisted restore membership is required before workspace mutation.', + 'IncompatibleWorkspaceError', + [{ checkCode: 'restore-durable-membership-missing' }], + ); + } + const expectedAgents = durable.expectedAgents; + const exploitOff = initialExpectedAgents(false); + const exploitOn = initialExpectedAgents(true); + const expectedSetIsValid = + arraysEqual(expectedAgents, exploitOff) || + arraysEqual(expectedAgents, exploitOn) || + arraysEqual(expectedAgents, [...exploitOn, 'miscellaneous-exploit']); + const participatingClasses = durable.participatingClasses; + const participatingSetIsValid = + arraysEqual(participatingClasses, FIXED_ANALYSIS_CLASSES) || + arraysEqual(participatingClasses, [...FIXED_ANALYSIS_CLASSES, 'miscellaneous']); + const miscellaneousAdmissionIsValid = + !expectedAgents.includes('miscellaneous-exploit') || participatingClasses.includes('miscellaneous'); + if ( + !expectedSetIsValid || + !participatingSetIsValid || + !miscellaneousAdmissionIsValid || + (durable.reportProgress !== undefined && !isReportProgress(durable.reportProgress, participatingClasses)) + ) { + throw ApplicationFailure.nonRetryable( + 'Persisted restore membership or report state is malformed.', + 'CorruptedSessionError', + [{ checkCode: 'restore-durable-input-malformed' }], + ); + } + const deliverablesPath = deliverablesDir(repoPath, deliverablesSubdir); const logger = createActivityLogger(); logger.info(`Restoring deliverables to ${checkpointHash}...`); @@ -1119,12 +1767,16 @@ export async function restoreGitCheckpoint( // Scope the untracked clean so a completed agent's deliverables survive: exclude every // completed agent's paths, cleaning only leftovers from the incomplete agents being re-run. const incompleteSet = new Set(incompleteAgents); - const completedPaths = ALL_AGENTS.filter((name) => !incompleteSet.has(name)).flatMap(getAgentGitPaths); + if (durable?.reportProgress?.stage === 'draft' || durable?.reportProgress?.stage === 'finalized') { + incompleteSet.delete('report'); + } + const completedPaths = expectedAgents.filter((name) => !incompleteSet.has(name)).flatMap(getAgentGitPaths); const cleanArgs = ['git', 'clean', '-fd', ...completedPaths.flatMap((completedPath) => ['-e', completedPath])]; await executeGitCommandWithRetry(cleanArgs, deliverablesPath, 'clean untracked deliverables'); // Explicitly delete partial deliverables for incomplete agents for (const agentName of incompleteAgents) { + if (agentName === 'report' && !incompleteSet.has('report')) continue; const deliverableFilename = AGENTS[agentName].deliverableFilename; const deliverablePath = path.join(deliverablesPath, deliverableFilename); try { @@ -1229,35 +1881,28 @@ export async function logWorkflowComplete(input: ActivityInput, summary: Workflo } } - // 4. Build cumulative summary with cross-run totals + // 4. Build cumulative totals: session.json carries cross-run agent spend only, so this + // run's operational (Capella/reconciliation) entries are added on top instead of being + // silently replaced by agent-only session totals. + const operationalEntries = Object.entries(agentMetrics).filter( + ([name]) => sessionData.metrics.agents[name] === undefined, + ); + const operationalCostUsd = operationalEntries.reduce((sum, [, metrics]) => sum + (metrics.costUsd ?? 0), 0); + const operationalDurationMs = operationalEntries.reduce((sum, [, metrics]) => sum + metrics.durationMs, 0); const cumulativeSummary: WorkflowSummary = { ...summary, - totalDurationMs: sessionData.metrics.total_duration_ms, - totalCostUsd: sessionData.metrics.total_cost_usd, + totalDurationMs: sessionData.metrics.total_duration_ms + operationalDurationMs, + totalCostUsd: sessionData.metrics.total_cost_usd + operationalCostUsd, agentMetrics, }; // 5. Write completion entry to workflow.log await auditSession.logWorkflowComplete(cumulativeSummary); - // 6. Surface the final report at the run root. Done here (not in the report phase) - // so it also runs when a resume skips an already-complete report phase. A partial - // run still assembles a report (only some classes were not assessed), so surface it too. - if (summary.status === 'completed' || summary.status === 'partial') { - try { - await copyReportToRunRoot( - input.repoPath, - input.deliverablesSubdir, - generateAuditPath(sessionMetadata), - createActivityLogger(), - ); - } catch (error) { - const detail = error instanceof Error ? error.message : String(error); - console.warn(`Failed to surface report at run root: ${detail}`); - } - } - - // 7. Drop the authenticated browser session + // 6. Drop the authenticated browser session. auth-state.json holds live cookies/storage for + // the lifetime of the scan only; leaving it on disk past workflow end would let a session + // outlive the run that created it. The removal is best-effort: a failure here is logged and + // swallowed rather than failing a scan that otherwise completed successfully. try { await fs.rm(authStateFile(sessionMetadata), { force: true }); } catch (error) { @@ -1265,7 +1910,7 @@ export async function logWorkflowComplete(input: ActivityInput, summary: Workflo console.warn(`Failed to clean up auth-state.json: ${detail}`); } - // 8. Clean up container + // 7. Clean up container removeContainer(workflowId); } @@ -1309,22 +1954,3 @@ export async function saveCheckpoint( return container.checkpointProvider.onAgentComplete(agentName, phase, state, context); } - -/** - * Generate an optional additional output alongside the assembled markdown report. - * - * Delegates to the ReportOutputProvider registered in the DI container. - * Default: no-op. Consumers can override this activity at the worker level - * to emit derived outputs from the final report. - */ -export async function generateReportOutputActivity(input: ActivityInput): Promise { - const container = getContainer(input.workflowId); - if (!container?.reportOutputProvider) return; - - const logger = createActivityLogger(); - - const result = await container.reportOutputProvider.generate(input, logger); - if (result.outputPath) { - logger.info(`Report output written to ${result.outputPath}`); - } -} diff --git a/apps/worker/src/temporal/pipeline.ts b/apps/worker/src/temporal/pipeline.ts index 911ffc39..0f7f55f7 100644 --- a/apps/worker/src/temporal/pipeline.ts +++ b/apps/worker/src/temporal/pipeline.ts @@ -7,7 +7,12 @@ export type { ActivityInput } from './activities.js'; export type { + AgenticSastInput, + AgenticSastState, AgentMetrics, + NonFatalFailure, + OperationalMetrics, + OperationalStageState, PipelineInput, PipelineState, PipelineSummary, diff --git a/apps/worker/src/temporal/reconcile-activities.ts b/apps/worker/src/temporal/reconcile-activities.ts index 1318a89f..25dd2e83 100644 --- a/apps/worker/src/temporal/reconcile-activities.ts +++ b/apps/worker/src/temporal/reconcile-activities.ts @@ -11,6 +11,8 @@ import { type ModelHost, modelHost } from '../ai/model-host.js'; import { createPiStructuredGenerationPort } from '../ai/pi/structured-generation.js'; import { createTaskFormationExecutor, + TASK_FORMATION_FALLBACK_REASONS, + type TaskFormationExecutionContext, TaskFormationExecutorError, type TaskFormationFallbackReason, } from '../ai/pi/task-formation-executor.js'; @@ -49,8 +51,11 @@ import type { PrepareResult, } from '../ai/reconciliation/stage-contracts.js'; import type { ActivityLogger } from '../types/activity-logger.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; +import { renderSafeMessage } from '../types/run-state.js'; import { createActivityLogger } from './activity-logger.js'; import { + ACCEPTED_TASK_FORMATION_FALLBACK_REASONS, type EnrichClassSastObservationsActivityInput, type FormClassExploitTasksActivityInput, type FormClassExploitTasksActivityResult, @@ -66,10 +71,21 @@ import { type ReconciliationStableFailureType, resolveReconciliationActivityBudget, type SeedEmptyProducerQueueActivityInput, + TASK_FORMATION_EXECUTOR_TIMEOUT_MARGIN_MS, } from './reconcile-activity-types.js'; const STABLE_FAILURE_TYPES: ReadonlySet = new Set(RECONCILIATION_STABLE_FAILURE_TYPES); +// The workflow validates fallback reasons against its bundle-safe mirror; fail fast at worker +// startup if the mirror ever drifts from the executor's authoritative closed set. +{ + const mirror = [...ACCEPTED_TASK_FORMATION_FALLBACK_REASONS].sort(); + const authoritative = [...TASK_FORMATION_FALLBACK_REASONS].sort(); + if (mirror.length !== authoritative.length || mirror.some((reason, index) => reason !== authoritative[index])) { + throw new Error('The workflow fallback-reason mirror does not match the task-formation executor contract'); + } +} + const DEFAULT_RETRYABILITY: Readonly> = Object.freeze({ TaskFormationModelError: true, SastEnrichmentModelError: true, @@ -83,17 +99,26 @@ const DEFAULT_RETRYABILITY: Readonly> = Object.freeze({ - TaskFormationModelError: 'Task formation did not produce an accepted result.', - SastEnrichmentModelError: 'SAST enrichment did not produce an accepted result.', - ReconciliationArtifactNotFound: 'A reconciliation artifact is not currently visible.', + TaskFormationModelError: 'Shannon could not group {class} findings into test cases.', + SastEnrichmentModelError: 'Shannon could not add code context to the {class} findings from static analysis.', + ReconciliationArtifactNotFound: + 'A saved {class} result could not be read back. Re-running this workspace retries it.', ReconciliationIoError: 'A reconciliation filesystem or Git operation failed.', ConfigurationError: 'Reconciliation activity configuration is invalid.', SastEnrichmentInputError: 'The supplied SAST reference is invalid.', - ArtifactIntegrityError: 'Reconciliation artifact integrity validation failed.', - PublicationConflict: 'The durable class publication conflicts with committed state.', - UnmappableSurvivor: 'A report-facing survivor cannot be mapped to the class task set.', - KeySetDivergence: 'Reconciliation report-facing key sets disagree.', + ArtifactIntegrityError: 'A saved {class} result failed its integrity check and was not used.', + PublicationConflict: + "{Class} results were already published by an earlier run, and this run's results differ. Nothing was overwritten.", + UnmappableSurvivor: + 'Shannon could not match a finding in the report back to the test case it came from. {Class} results were not published.', + KeySetDivergence: + 'Shannon found two disagreeing sets of findings for {class} and stopped rather than publish either.', }); interface ReconciliationHeartbeatDetails { @@ -107,6 +132,10 @@ export interface ReconciliationActivityRuntime { readonly attempt: number; readonly cancellationSignal: AbortSignal; readonly logger: ActivityLogger; + /** Temporal's granted per-attempt execution budget, from the activity info. */ + readonly startToCloseTimeoutMs?: number; + /** Bounded per-attempt correlation identifier (run id + activity id). */ + readonly executionKey?: string; heartbeat(details: ReconciliationHeartbeatDetails): void; } @@ -114,6 +143,9 @@ interface ReconciliationStageRuntime { readonly signal: AbortSignal; readonly logger: ActivityLogger; readonly modelHost: ModelHost; + /** Remaining granted budget minus the deterministic margin, evaluated at call time. */ + readonly executorTimeoutMsFor?: () => number | undefined; + readonly executionContextFor?: () => TaskFormationExecutionContext | undefined; } export interface ReconciliationStageBindings { @@ -166,6 +198,8 @@ function defaultRuntime(): ReconciliationActivityRuntime { attempt: context.info.attempt, cancellationSignal: context.cancellationSignal, logger: createActivityLogger(), + startToCloseTimeoutMs: context.info.startToCloseTimeoutMs, + executionKey: `${context.info.workflowExecution.runId}:${context.info.activityId}`, heartbeat, }; } @@ -188,10 +222,11 @@ function applicationFailure( type: ReconciliationStableFailureType, retryable: boolean, stage: ReconciliationActivityName, + vulnerabilityClass: ReconciliationClass, details: StableFailureDetails = {}, ): ApplicationFailure { return ApplicationFailure.create({ - message: SAFE_FAILURE_MESSAGES[type], + message: renderSafeMessage(SAFE_FAILURE_MESSAGES[type], { vulnerabilityClass }), type, nonRetryable: !retryable, details: [ @@ -204,12 +239,33 @@ function applicationFailure( }); } -function cancellationFrom(error: unknown, signal: AbortSignal): CancelledFailure | undefined { - if (error instanceof CancelledFailure) return error; +const CANCELLATION_CHAIN_DEPTH = 8; - const errorName = error instanceof Error ? error.name : undefined; - const cancelledByName = errorName === 'CancelledFailure' || errorName === 'AbortError'; - if (!signal.aborted && !cancelledByName) return undefined; +/** + * A failure counts as cancellation only when the activity signal is aborted AND its bounded + * cause chain carries a real cancellation (the signal's own reason, a `CancelledFailure`, or + * a cancellation-named abort raised under the aborted signal). A provider timeout, an + * abort-shaped provider error with the signal unset, a cleanup failure, or any infrastructure + * fault therefore stays an ordinary typed failure and is never manufactured into cancellation. + */ +function chainContainsRealCancellation(error: unknown, signal: AbortSignal): boolean { + let current: unknown = error; + const seen = new Set(); + for (let depth = 0; depth < CANCELLATION_CHAIN_DEPTH; depth++) { + if (current === undefined || current === null || seen.has(current)) return false; + if (current === signal.reason) return true; + if (current instanceof CancelledFailure) return true; + if (current instanceof Error && (current.name === 'CancelledFailure' || current.name === 'AbortError')) return true; + seen.add(current); + current = current instanceof Error ? current.cause : undefined; + } + return false; +} + +function cancellationFrom(error: unknown, signal: AbortSignal): CancelledFailure | undefined { + if (!signal.aborted) return undefined; + // A proactive check before any stage work has an aborted signal and no failure to inspect. + if (error !== undefined && error !== null && !chainContainsRealCancellation(error, signal)) return undefined; const reason = signal.reason; if (reason instanceof CancelledFailure) return reason; @@ -222,27 +278,32 @@ function cancellationFrom(error: unknown, signal: AbortSignal): CancelledFailure * error) onto the closed set of stable failure types. Cancellation is checked first and * always wins, since a stage aborted for cancellation is not a stage that failed. */ -function normalizeFailure(error: unknown, stage: ReconciliationActivityName, signal: AbortSignal): never { +function normalizeFailure( + error: unknown, + stage: ReconciliationActivityName, + vulnerabilityClass: ReconciliationClass, + signal: AbortSignal, +): never { const cancellation = cancellationFrom(error, signal); if (cancellation !== undefined) throw cancellation; if (error instanceof TaskFormationModelError) { - throw applicationFailure('TaskFormationModelError', error.retryable, stage, { + throw applicationFailure('TaskFormationModelError', error.retryable, stage, vulnerabilityClass, { metrics: failureMetrics(error), ...(error.fallbackReason !== undefined && { fallbackReason: error.fallbackReason }), }); } if (error instanceof SastEnrichmentModelError) { - throw applicationFailure('SastEnrichmentModelError', error.retryable, stage, { + throw applicationFailure('SastEnrichmentModelError', error.retryable, stage, vulnerabilityClass, { metrics: failureMetrics(error), }); } if (error instanceof ReconciliationError) { - throw applicationFailure(error.failureType, error.retryable, stage); + throw applicationFailure(error.failureType, error.retryable, stage, vulnerabilityClass); } if (error instanceof TaskFormationExecutorError) { if (error.failureKind === 'model') { - throw applicationFailure('TaskFormationModelError', error.retryable, stage, { + throw applicationFailure('TaskFormationModelError', error.retryable, stage, vulnerabilityClass, { metrics: { costUsd: error.usage.costUsd, modelCalls: error.modelCalls, @@ -252,47 +313,54 @@ function normalizeFailure(error: unknown, stage: ReconciliationActivityName, sig ...(error.fallbackReason !== undefined && { fallbackReason: error.fallbackReason }), }); } + // Retryable executor infrastructure faults (session setup, transient IO) must stay + // retryable IO at the boundary instead of colliding with terminal ConfigurationError. + if (error.failureKind === 'infrastructure') { + throw applicationFailure('ReconciliationIoError', error.retryable, stage, vulnerabilityClass); + } const type = error.failureKind === 'confinement' ? 'ArtifactIntegrityError' : 'ConfigurationError'; - throw applicationFailure(type, error.retryable, stage); + throw applicationFailure(type, error.retryable, stage, vulnerabilityClass); } if (error instanceof ApplicationFailure) { const errorType = error.type; if (typeof errorType === 'string' && isStableFailureType(errorType)) { - throw applicationFailure(errorType, !error.nonRetryable, stage); + throw applicationFailure(errorType, !error.nonRetryable, stage, vulnerabilityClass); } - throw applicationFailure('ReconciliationIoError', true, stage); + throw applicationFailure('ReconciliationIoError', true, stage, vulnerabilityClass); } if (error instanceof Error && isStableFailureType(error.name)) { const retryable = 'retryable' in error && typeof error.retryable === 'boolean' ? error.retryable : DEFAULT_RETRYABILITY[error.name]; - throw applicationFailure(error.name, retryable, stage); + throw applicationFailure(error.name, retryable, stage, vulnerabilityClass); } // Unknown failures remain retryable. A generic error name is not evidence that the fault is terminal. - throw applicationFailure('ReconciliationIoError', true, stage); + throw applicationFailure('ReconciliationIoError', true, stage, vulnerabilityClass); } /** Refuse to schedule a class's remaining reconciliation stages once its 12-hour budget is spent. */ function assertActivityCanRun( activityName: ReconciliationClassActivityName, classDeadlineMs: number, + vulnerabilityClass: ReconciliationClass, nowMs: number, ): ReturnType { try { const budget = resolveReconciliationActivityBudget(activityName, classDeadlineMs, nowMs); if (!budget.shouldSchedule) { - throw applicationFailure('ConfigurationError', false, activityName); + throw applicationFailure('ConfigurationError', false, activityName, vulnerabilityClass); } return budget; } catch (error) { if (error instanceof ApplicationFailure) throw error; - throw applicationFailure('ConfigurationError', false, activityName); + throw applicationFailure('ConfigurationError', false, activityName, vulnerabilityClass); } } async function runReconciliationStage( activityName: ReconciliationClassActivityName, classDeadlineMs: number, + vulnerabilityClass: ReconciliationClass, runtime: ReconciliationActivityRuntime, now: () => number, stage: (runtime: ReconciliationStageRuntime) => Promise, @@ -301,7 +369,7 @@ async function runReconciliationStage( const cancellation = cancellationFrom(undefined, runtime.cancellationSignal); if (cancellation !== undefined) throw cancellation; - const budget = assertActivityCanRun(activityName, classDeadlineMs, now()); + const budget = assertActivityCanRun(activityName, classDeadlineMs, vulnerabilityClass, now()); const profile = RECONCILIATION_ACTIVITY_PROFILES[activityName]; const startedAt = now(); let heartbeatInterval: ReturnType | undefined; @@ -318,10 +386,30 @@ async function runReconciliationStage( }, budget.heartbeatIntervalMs); } + // The executor's own timer must expire before Temporal's activity timeout, so the + // metrics-bearing model-stage-timeout failure stays reachable. Evaluate the remaining + // granted budget at call time because jail materialization can consume minutes first. + const grantedBudgetMs = runtime.startToCloseTimeoutMs; + const executorTimeoutMsFor = (): number | undefined => { + if (grantedBudgetMs === undefined || grantedBudgetMs <= 0) return undefined; + const remainingMs = startedAt + grantedBudgetMs - now(); + return Math.max(1_000, remainingMs - TASK_FORMATION_EXECUTOR_TIMEOUT_MARGIN_MS); + }; + const executionContextFor = (): TaskFormationExecutionContext | undefined => ({ + attempt: runtime.attempt, + ...(runtime.executionKey !== undefined && { executionKey: runtime.executionKey }), + }); + try { - return await stage({ signal: runtime.cancellationSignal, logger: runtime.logger, modelHost: activityModelHost }); + return await stage({ + signal: runtime.cancellationSignal, + logger: runtime.logger, + modelHost: activityModelHost, + executorTimeoutMsFor, + executionContextFor, + }); } catch (error) { - return normalizeFailure(error, activityName, runtime.cancellationSignal); + return normalizeFailure(error, activityName, vulnerabilityClass, runtime.cancellationSignal); } finally { if (heartbeatInterval !== undefined) clearInterval(heartbeatInterval); } @@ -334,7 +422,7 @@ async function runSeedStage(runtime: ReconciliationActivityRuntime, stage: () try { return await stage(); } catch (error) { - return normalizeFailure(error, 'seedEmptyProducerQueue', runtime.cancellationSignal); + return normalizeFailure(error, 'seedEmptyProducerQueue', 'miscellaneous', runtime.cancellationSignal); } } @@ -356,6 +444,8 @@ function defaultStages(workspacesDir: string): ReconciliationStageBindings { workspacesDir, signalFor: () => runtime.signal, logger: runtime.logger, + ...(runtime.executorTimeoutMsFor !== undefined && { executorTimeoutMsFor: runtime.executorTimeoutMsFor }), + ...(runtime.executionContextFor !== undefined && { executionContextFor: runtime.executionContextFor }), })(input), materializeClassExploitTasks: materializeClassExploitTasksStage, publishClassReconciliationOss: publishClassReconciliationOssStage, @@ -412,6 +502,7 @@ export function createReconciliationActivityRegistry( const result = await runReconciliationStage( 'prepareClassReconciliation', input.classDeadlineMs, + input.vulnerabilityClass, runtime, now, () => @@ -439,6 +530,7 @@ export function createReconciliationActivityRegistry( const result = await runReconciliationStage( 'enrichClassSastObservations', input.classDeadlineMs, + input.vulnerabilityClass, runtime, now, (stageRuntime) => @@ -462,6 +554,7 @@ export function createReconciliationActivityRegistry( const result = await runReconciliationStage( 'formClassExploitTasks', input.classDeadlineMs, + input.vulnerabilityClass, runtime, now, (stageRuntime) => @@ -486,6 +579,7 @@ export function createReconciliationActivityRegistry( const result = await runReconciliationStage( 'materializeClassExploitTasks', input.classDeadlineMs, + input.vulnerabilityClass, runtime, now, () => @@ -507,6 +601,7 @@ export function createReconciliationActivityRegistry( const result = await runReconciliationStage( 'publishClassReconciliationOss', input.classDeadlineMs, + input.vulnerabilityClass, runtime, now, () => diff --git a/apps/worker/src/temporal/reconcile-activity-types.ts b/apps/worker/src/temporal/reconcile-activity-types.ts index 1cd5edf8..40bf0392 100644 --- a/apps/worker/src/temporal/reconcile-activity-types.ts +++ b/apps/worker/src/temporal/reconcile-activity-types.ts @@ -6,6 +6,7 @@ /** Workflow-safe reconciliation activity signatures and scheduling policy. */ +import type { TaskFormationFallbackReason } from '../ai/pi/task-formation-executor.js'; import type { ArtifactRef } from '../ai/reconciliation/contracts.js'; import type { StageMetrics } from '../ai/reconciliation/stage-contracts.js'; import type { SarifRef } from '../ai/sast/types.js'; @@ -17,6 +18,37 @@ const HOUR_MS = 60 * MINUTE_MS; export const RECONCILIATION_CLASS_BUDGET_MS = 12 * HOUR_MS; export const RECONCILIATION_LATER_STAGE_RESERVE_MS = 5 * MINUTE_MS; +/** + * Deterministic safety margin subtracted from the granted activity budget before it is passed + * to the Pass 1 executor timer, so the executor's own timeout always fires before Temporal's + * activity timeout and the metrics-bearing model-stage-timeout path stays reachable. + */ +export const TASK_FORMATION_EXECUTOR_TIMEOUT_MARGIN_MS = MINUTE_MS; + +/** + * Workflow-safe mirror of Agent A's closed fallback-reason set. The executor module itself is + * not bundle-safe, so the workflow validates deserialized failure details against this frozen + * copy; the `satisfies` clause and the exhaustiveness check keep the two sets identical at + * compile time, and the activity boundary re-asserts equality at module load. + */ +export const ACCEPTED_TASK_FORMATION_FALLBACK_REASONS = Object.freeze([ + 'retryable_model_failure', + 'missing_accepted_submission', + 'model_stage_timeout', +] as const satisfies readonly TaskFormationFallbackReason[]); + +type UnlistedFallbackReason = Exclude< + TaskFormationFallbackReason, + (typeof ACCEPTED_TASK_FORMATION_FALLBACK_REASONS)[number] +>; +const _everyFallbackReasonIsListed: UnlistedFallbackReason extends never ? true : never = true; +void _everyFallbackReasonIsListed; + +/** Validate one deserialized fallback reason against the closed set. */ +export function isAcceptedTaskFormationFallbackReason(value: unknown): value is TaskFormationFallbackReason { + return (ACCEPTED_TASK_FORMATION_FALLBACK_REASONS as readonly unknown[]).includes(value); +} + export interface ReconciliationActivityDeadline { /** Fixed workflow-derived deadline for this class, measured as Unix epoch milliseconds. */ readonly classDeadlineMs: number; diff --git a/apps/worker/src/temporal/shared.ts b/apps/worker/src/temporal/shared.ts index 2004c050..fdef21a6 100644 --- a/apps/worker/src/temporal/shared.ts +++ b/apps/worker/src/temporal/shared.ts @@ -2,15 +2,87 @@ import { defineQuery } from '@temporalio/workflow'; export type { AgentMetrics } from '../types/metrics.js'; -import type { DistributedConfig, VulnClass } from '../types/config.js'; +import type { CapellaFailurePoint, CapellaStage, SarifRef } from '../ai/sast/types.js'; +import type { VulnClass } from '../types/config.js'; import type { ErrorCode } from '../types/errors.js'; import type { AgentMetrics } from '../types/metrics.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; +import type { + MiscellaneousOutcome, + PartialReasonView, + ReportProgress, + ReportSarifDisposition, + StoredPdfProvenance, +} from '../types/run-state.js'; + +/** + * The serializable slice of Capella's configuration passed across the Temporal workflow + * boundary into the child workflow input. Everything the workflow needs from the parsed + * config or the model spec must be flattened into plain data here; the workflow sandbox + * cannot carry functions or class instances across that boundary. + */ +export interface AgenticSastInput { + readonly codePathAvoids: readonly string[]; + readonly codePathFocus: readonly string[]; + readonly modelSpec: string; + readonly capellaFormatVersion: string; + readonly promptSetVersion: string; +} + +/** + * The agentic SAST lifecycle as seen from the pentest workflow: not configured, running as a + * child workflow, or one of two terminal outcomes. This is what the live `getProgress` query + * and the terminal `PipelineState` both report, so a caller never needs to inspect the Capella + * child workflow's own result type directly. + */ +export type AgenticSastState = + | { readonly status: 'disabled' } + | { readonly status: 'running'; readonly startedAt: number } + | { + readonly status: 'succeeded'; + readonly findingCount: number; + readonly sarifSha256: string; + readonly coverage: 'complete' | 'reduced'; + readonly warnings: readonly string[]; + readonly durationMs: number; + } + | { + readonly status: 'failed'; + readonly failedStage: CapellaFailurePoint; + /** Reader-facing name of `failedStage`, projected once so no surface renders the slug. */ + readonly failedStageLabel: string; + readonly error: string; + /** Bounded machine code preserved from the failing Capella activity, when one crossed the child. */ + readonly errorCode?: string; + readonly completedStages: readonly CapellaStage[]; + readonly durationMs: number; + }; + +export type OperationalStageStatus = 'pending' | 'running' | 'completed' | 'failed' | 'skipped'; + +export interface OperationalStageState { + readonly key: string; + readonly label: string; + readonly status: OperationalStageStatus; + readonly startedAt?: number; + readonly durationMs?: number; + readonly error?: string; +} + +export interface OperationalMetrics extends AgentMetrics { + readonly usageComplete?: boolean; +} + +/** A degradation the scan recorded and continued past, kept for the terminal summary log rather than for control flow. */ +export interface NonFatalFailure { + readonly phase: string; + readonly error: string; +} export interface PipelineInput { webUrl: string; repoPath: string; configPath?: string; - outputPath?: string; pipelineTestingMode?: boolean; workflowId?: string; // Used for audit correlation sessionId?: string; // Workspace directory name (distinct from workflowId for named workspaces) @@ -19,44 +91,114 @@ export interface PipelineInput { // Config fields — serializable, flow through to ActivityInput → getOrCreateContainer() configYAML?: string; // Raw YAML string (parsed in activity, not workflow — workflow sandbox can't use Node.js) - configData?: DistributedConfig; // Pre-parsed config (bypasses file loading) deliverablesSubdir?: string; // Override deliverables path (default: '.shannon/deliverables') auditDir?: string; // Override audit log directory (default: './workspaces') promptDir?: string; // Override prompt template directory - sastSarifPath?: string; // Optional path for consumer-supplied findings input + agenticSast?: AgenticSastInput; + sastSarif?: SarifRef; + customerOutputPath?: string; // Stable mounted path for final customer copies only checkpointsEnabled?: boolean; // Enable checkpoint activities (default: false) - vulnClasses?: VulnClass[]; // omitted = all five exploit?: boolean; // false skips the exploitation phase } +/** What `loadResumeState` reconstructs from a prior workspace: independently verified, never assumed from session.json alone. */ export interface ResumeState { workspaceName: string; originalUrl: string; completedAgents: string[]; checkpointHash: string; originalWorkflowId: string; + expectedAgents: string[]; + participatingClasses: ReconciliationClass[]; + exploit: boolean; + reportProgress?: ReportProgress; + miscellaneousOutcome?: MiscellaneousOutcome; +} + +/** The narrow view of the durable scan-state record the workflow needs to keep its own queryable state in sync. */ +export interface DurableStateSummary { + readonly exploit: boolean; + readonly expectedAgents: readonly string[]; + readonly participatingClasses: readonly ReconciliationClass[]; + readonly reportStage: ReportProgress['stage'] | 'uninitialized'; + readonly miscellaneousOutcome?: MiscellaneousOutcome; +} + +/** Common result shape for the deterministic report-processing activities (renumber, compaction). */ +export interface ReconciliationActivityResult { + readonly vulnerabilityClass?: ReconciliationClass; + readonly skipped: boolean; + readonly changedPathCount: number; + readonly checkpoint?: string; + readonly alreadyCommitted?: boolean; +} + +export interface FinalizeReportActivityResult { + readonly checkpoint: string; + readonly manifestSha256: string; + readonly changedPathCount: number; + readonly alreadyCommitted: boolean; + /** Adopted-or-produced SARIF disposition from the committed finalization manifest. */ + readonly sarifDisposition: ReportSarifDisposition; + readonly pdfGenerated: boolean; + /** Verified provenance for the current PDF bytes, or null when no trustworthy PDF exists. */ + readonly pdfProvenance: StoredPdfProvenance | null; + readonly warningCount: number; +} + +export interface AssembleReportActivityResult { + /** Classes whose findings could not be included in the assembled report inputs. */ + readonly failedClasses: readonly ReconciliationClass[]; +} + +export interface SurfaceReportActivityResult { + readonly surfaced: readonly string[]; + readonly removedStale: readonly string[]; + readonly warningCount: number; } export interface PipelineSummary { totalCostUsd: number; totalDurationMs: number; // Wall-clock time (end - start) totalTurns: number; + /** Total resolved agents: those that ran plus those that were skipped. */ agentCount: number; + /** False when operational (Capella/reconciliation) spend is known to be incomplete. */ + usageAccountingComplete: boolean; } +/** + * The workflow's whole queryable and terminal state. The CLI cannot import this package, so + * `apps/cli/src/scan/pipeline.ts` mirrors this shape (along with AgentMetrics and the + * activity-name-to-agent map) by hand; a field added, renamed, or removed here needs the same + * change there, or the CLI's status rendering silently falls out of sync with a running scan. + */ export interface PipelineState { status: 'running' | 'completed' | 'failed' | 'cancelled' | 'partial'; currentPhase: string | null; currentAgent: string | null; + /** Agents that actually ran. Mutually exclusive from `skippedAgents`. */ completedAgents: string[]; + /** Expected agents that never ran because their class had nothing to exploit. */ + skippedAgents: string[]; + expectedAgents: string[]; + participatingClasses: ReconciliationClass[]; // Vuln classes whose pipeline failed while at least one other succeeded. Drives the // partial terminal status so a crashed class isn't reported as if it fully passed. failedPipelines: { vulnType: VulnClass; error: string }[]; + failedReconciliations: { vulnerabilityClass: ReconciliationClass; error: string }[]; failedAgent: string | null; error: string | null; errorCode?: ErrorCode; startTime: number; agentMetrics: Record; + operationalMetrics: Record; + operationalStages: Record; + agenticSast: AgenticSastState; + nonFatalFailures: NonFatalFailure[]; + /** Ordered durable degradation reasons with derived safe messages; empty for a full success. */ + partialReasons: PartialReasonView[]; + reportProgress?: ReportProgress; summary: PipelineSummary | null; } diff --git a/apps/worker/src/temporal/summary-mapper.ts b/apps/worker/src/temporal/summary-mapper.ts index 39798980..c2ed7620 100644 --- a/apps/worker/src/temporal/summary-mapper.ts +++ b/apps/worker/src/temporal/summary-mapper.ts @@ -29,14 +29,29 @@ export function toWorkflowSummary( throw new Error('toWorkflowSummary: state.summary must be set before calling'); } + // The failure detail is one of the child workflow's fixed safe sentences, so it carries no + // provider, prompt, repository, or path content and travels with the stable code. + const agenticSastFailure = state.agenticSast.status === 'failed' ? state.agenticSast : undefined; + const agenticSastErrorCode = agenticSastFailure?.errorCode; + const agenticSastFailureMessage = agenticSastFailure?.error; + const agenticSastFailedStage = agenticSastFailure?.failedStageLabel; return { status, totalDurationMs: summary.totalDurationMs, totalCostUsd: summary.totalCostUsd, completedAgents: state.completedAgents, + skippedAgents: state.skippedAgents, agentMetrics: Object.fromEntries( - Object.entries(state.agentMetrics).map(([name, m]) => [name, { durationMs: m.durationMs, costUsd: m.costUsd }]), + [...Object.entries(state.agentMetrics), ...Object.entries(state.operationalMetrics)].map(([name, metrics]) => [ + name, + { durationMs: metrics.durationMs, costUsd: metrics.costUsd }, + ]), ), + partialReasons: state.partialReasons, + usageAccountingComplete: summary.usageAccountingComplete, + ...(agenticSastFailedStage !== undefined && { agenticSastFailedStage }), + ...(agenticSastFailureMessage !== undefined && { agenticSastFailureMessage }), + ...(agenticSastErrorCode !== undefined && { agenticSastErrorCode }), ...(state.error && { error: state.error }), }; } diff --git a/apps/worker/src/temporal/worker.ts b/apps/worker/src/temporal/worker.ts index c3b1983b..ea1c2435 100644 --- a/apps/worker/src/temporal/worker.ts +++ b/apps/worker/src/temporal/worker.ts @@ -19,7 +19,7 @@ * Options: * --task-queue Task queue name (required, unique per scan) * --config Configuration file path - * --output Output directory for workspaces + * --output Stable mounted path for final customer report copies * --workspace Resume from existing workspace * --pipeline-testing Use minimal prompts for fast testing * @@ -27,24 +27,60 @@ * TEMPORAL_ADDRESS - Temporal server address (default: localhost:7233) */ -import fs from 'node:fs'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { Client, Connection, type WorkflowHandle, WorkflowNotFoundError } from '@temporalio/client'; import { bundleWorkflowCode, NativeConnection, Worker } from '@temporalio/worker'; import dotenv from 'dotenv'; +import { DEFAULT_MODEL_SPEC } from '../ai/models.js'; +import { capellaActivities, mergeActivityRegistries } from '../ai/sast/capella/temporal/registry.js'; +import { CAPELLA_FORMAT_VERSION, CAPELLA_PROMPT_SET_VERSION } from '../ai/sast/capella/types.js'; import { sanitizeHostname } from '../audit/utils.js'; -import { parseConfig } from '../config-parser.js'; -import { - ASSEMBLED_REPORT_PDF_FILENAME, - deliverablesDir, - FINAL_REPORT_PDF_FILENAME, - resolveSessionJsonPath, -} from '../paths.js'; -import type { VulnClass } from '../types/config.js'; +import { distributeConfig, parseConfig } from '../config-parser.js'; +import { deliverablesDir, resolveSessionJsonPath } from '../paths.js'; +import { SAFE_RUN_STATE_MESSAGES, workspaceExploitMismatchMessage } from '../types/run-state.js'; import { fileExists, readJson } from '../utils/file-io.js'; -import * as activities from './activities.js'; -import type { PipelineInput, PipelineProgress, PipelineState } from './shared.js'; +import { + assembleReportActivity, + checkExploitationQueue, + compactReportFindings, + finalizeReportOutputs, + initDeliverableGit, + initializeDurableScanState, + initializeReportProgress, + loadResumeState, + logPhaseTransition, + logWorkflowComplete, + persistCanonicalReportProgress, + persistFinalizedReportProgress, + persistMiscellaneousOutcome, + recordResumeAttempt, + registerResumeAttempt, + renumberClassFindings, + restoreGitCheckpoint, + runAuthExploitAgent, + runAuthenticationValidation, + runAuthVulnAgent, + runAuthzExploitAgent, + runAuthzVulnAgent, + runInjectionExploitAgent, + runInjectionVulnAgent, + runMiscellaneousExploitAgent, + runPreflightValidation, + runPreReconAgent, + runReconAgent, + runReportAgent, + runSsrfExploitAgent, + runSsrfVulnAgent, + runXssExploitAgent, + runXssVulnAgent, + saveCheckpoint, + surfaceReportOutputs, + syncCodePathDenyRules, + syncPlaywrightStealthConfig, +} from './activities.js'; +import { createReconciliationActivityRegistry } from './reconcile-activities.js'; +import type { AgenticSastInput, PipelineInput, PipelineProgress, PipelineState } from './shared.js'; dotenv.config(); @@ -52,6 +88,118 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url)); const PROGRESS_QUERY = 'getProgress'; +// The ordinary activity names. This frozen list is one of three that together form the +// registered activity set the CLI status reader mirrors: the Capella names in +// ai/sast/capella/temporal/activity-types.ts and the reconciliation names in +// reconcile-activity-types.ts are the other two. Adding or removing an activity means +// updating both this list and the `pentestActivities` object below, or the load-time check +// throws. +export const PENTEST_ACTIVITY_NAMES = Object.freeze([ + 'runPreReconAgent', + 'runReconAgent', + 'runInjectionVulnAgent', + 'runXssVulnAgent', + 'runAuthVulnAgent', + 'runAuthzVulnAgent', + 'runSsrfVulnAgent', + 'runInjectionExploitAgent', + 'runXssExploitAgent', + 'runAuthExploitAgent', + 'runAuthzExploitAgent', + 'runSsrfExploitAgent', + 'runMiscellaneousExploitAgent', + 'runReportAgent', + 'runPreflightValidation', + 'runAuthenticationValidation', + 'initDeliverableGit', + 'syncPlaywrightStealthConfig', + 'syncCodePathDenyRules', + 'initializeDurableScanState', + 'persistMiscellaneousOutcome', + 'initializeReportProgress', + 'renumberClassFindings', + 'assembleReportActivity', + 'compactReportFindings', + 'persistCanonicalReportProgress', + 'finalizeReportOutputs', + 'persistFinalizedReportProgress', + 'surfaceReportOutputs', + 'checkExploitationQueue', + 'loadResumeState', + 'restoreGitCheckpoint', + 'registerResumeAttempt', + 'recordResumeAttempt', + 'logPhaseTransition', + 'logWorkflowComplete', + 'saveCheckpoint', +] as const); + +export const pentestActivities = Object.freeze({ + runPreReconAgent, + runReconAgent, + runInjectionVulnAgent, + runXssVulnAgent, + runAuthVulnAgent, + runAuthzVulnAgent, + runSsrfVulnAgent, + runInjectionExploitAgent, + runXssExploitAgent, + runAuthExploitAgent, + runAuthzExploitAgent, + runSsrfExploitAgent, + runMiscellaneousExploitAgent, + runReportAgent, + runPreflightValidation, + runAuthenticationValidation, + initDeliverableGit, + syncPlaywrightStealthConfig, + syncCodePathDenyRules, + initializeDurableScanState, + persistMiscellaneousOutcome, + initializeReportProgress, + renumberClassFindings, + assembleReportActivity, + compactReportFindings, + persistCanonicalReportProgress, + finalizeReportOutputs, + persistFinalizedReportProgress, + surfaceReportOutputs, + checkExploitationQueue, + loadResumeState, + restoreGitCheckpoint, + registerResumeAttempt, + recordResumeAttempt, + logPhaseTransition, + logWorkflowComplete, + saveCheckpoint, +}); + +const registeredPentestNames = Object.keys(pentestActivities).sort(); +const expectedPentestNames = [...PENTEST_ACTIVITY_NAMES].sort(); +if ( + registeredPentestNames.length !== expectedPentestNames.length || + registeredPentestNames.some((name, index) => name !== expectedPentestNames[index]) +) { + throw new Error('Pentest activity registry does not match its frozen ordinary activity contract'); +} + +export interface ProductionActivityBindings { + readonly repositoryPath: string; + readonly webUrl: string; + readonly workspacesDir: string; +} + +/** Compose the frozen ordinary, Capella, and reconciliation activity namespaces. */ +export function createProductionActivityRegistry(bindings: ProductionActivityBindings): Readonly { + const reconciliationActivities = createReconciliationActivityRegistry({ + repositoryPath: bindings.repositoryPath, + deliverablesDir: deliverablesDir(bindings.repositoryPath), + workspacesDir: bindings.workspacesDir, + webUrl: bindings.webUrl, + }); + return mergeActivityRegistries(pentestActivities, capellaActivities, reconciliationActivities); +} + // === CLI Argument Parsing === interface CliArgs { @@ -59,7 +207,7 @@ interface CliArgs { repoPath: string; taskQueue: string; configPath?: string; - outputPath?: string; + customerOutputPath?: string; pipelineTestingMode: boolean; resumeFromWorkspace?: string; } @@ -73,6 +221,7 @@ function showUsage(): void { console.log(' --task-queue Task queue name (required)'); console.log(' --config Configuration file path'); console.log(' --workspace Resume from existing workspace'); + console.log(' --output Stable mounted path for final customer report copies'); console.log(' --pipeline-testing Use minimal prompts for fast testing\n'); } @@ -86,7 +235,7 @@ function parseCliArgs(argv: string[]): CliArgs { let repoPath: string | undefined; let taskQueue: string | undefined; let configPath: string | undefined; - let outputPath: string | undefined; + let customerOutputPath: string | undefined; let pipelineTestingMode = false; let resumeFromWorkspace: string | undefined; @@ -107,7 +256,7 @@ function parseCliArgs(argv: string[]): CliArgs { } else if (arg === '--output') { const nextArg = argv[i + 1]; if (nextArg && !nextArg.startsWith('-')) { - outputPath = nextArg; + customerOutputPath = nextArg; i++; } } else if (arg === '--workspace') { @@ -145,7 +294,7 @@ function parseCliArgs(argv: string[]): CliArgs { taskQueue, pipelineTestingMode, ...(configPath && { configPath }), - ...(outputPath && { outputPath }), + ...(customerOutputPath && { customerOutputPath }), ...(resumeFromWorkspace && { resumeFromWorkspace }), }; } @@ -158,10 +307,15 @@ interface SessionJson { webUrl: string; originalWorkflowId?: string; resumeAttempts?: Array<{ workflowId: string }>; + status?: 'in-progress' | 'completed' | 'failed' | 'cancelled' | 'partial'; }; metrics: { total_cost_usd: number; }; + durableScanState?: { + schema_version?: unknown; + exploit?: unknown; + }; } function isValidWorkspaceName(name: string): boolean { @@ -216,7 +370,7 @@ async function terminateExistingWorkflows(client: Client, workspaceName: string) return terminated; } -async function resolveWorkspace(client: Client, args: CliArgs): Promise { +async function resolveWorkspace(client: Client, args: CliArgs, expectedExploit: boolean): Promise { if (!args.resumeFromWorkspace) { const hostname = sanitizeHostname(args.webUrl); const workflowId = `${hostname}_shannon-${Date.now()}`; @@ -233,6 +387,19 @@ async function resolveWorkspace(client: Client, args: CliArgs): Promise(sessionPath); + if (session.session.webUrl !== args.webUrl) { + throw new Error( + 'This workspace was created for a different target URL, so it cannot be resumed against this one. Check -u, or start a new scan with a different -w name.', + ); + } + if (session.durableScanState?.schema_version !== 1 || typeof session.durableScanState.exploit !== 'boolean') { + throw new Error(SAFE_RUN_STATE_MESSAGES.CorruptedSessionError); + } + if (session.durableScanState.exploit !== expectedExploit) { + throw new Error(workspaceExploitMismatchMessage(session.durableScanState.exploit)); + } + console.log('=== RESUME MODE ==='); console.log(`Workspace: ${workspace}\n`); @@ -241,14 +408,6 @@ async function resolveWorkspace(client: Client, args: CliArgs): Promise(sessionPath); - if (session.session.webUrl !== args.webUrl) { - console.error('ERROR: URL mismatch with workspace'); - console.error(` Workspace URL: ${session.session.webUrl}`); - console.error(` Provided URL: ${args.webUrl}`); - process.exit(1); - } - return { workflowId: `${workspace}_resume_${Date.now()}`, sessionId: workspace, @@ -281,7 +440,7 @@ async function resolveWorkspace(client: Client, args: CliArgs): Promise rule.type === 'code_path').map((rule) => rule.value); + const codePathFocus = distributed.focus.filter((rule) => rule.type === 'code_path').map((rule) => rule.value); return { - ...(config.vuln_classes && config.vuln_classes.length > 0 && { vulnClasses: [...config.vuln_classes] }), - ...(config.exploit !== undefined && { exploit: config.exploit === 'true' }), + ...(distributed.agenticSast && { + agenticSast: { + codePathAvoids, + codePathFocus, + modelSpec: process.env.SHANNON_AI_MODEL?.trim() || DEFAULT_MODEL_SPEC, + capellaFormatVersion: CAPELLA_FORMAT_VERSION, + promptSetVersion: CAPELLA_PROMPT_SET_VERSION, + }, + }), + exploit: distributed.exploit, }; } catch (error) { // A broken config must fail the run, not silently fall back to empty @@ -317,7 +487,8 @@ function buildPipelineInput( ...(args.pipelineTestingMode && { pipelineTestingMode: args.pipelineTestingMode }), ...(workspace.isResume && args.resumeFromWorkspace && { resumeFromWorkspace: args.resumeFromWorkspace }), ...(workspace.terminatedWorkflows.length > 0 && { terminatedWorkflows: workspace.terminatedWorkflows }), - ...(orchestration.vulnClasses && { vulnClasses: orchestration.vulnClasses }), + ...(args.customerOutputPath !== undefined && { customerOutputPath: args.customerOutputPath }), + ...(orchestration.agenticSast !== undefined && { agenticSast: orchestration.agenticSast }), ...(orchestration.exploit !== undefined && { exploit: orchestration.exploit }), }; } @@ -332,8 +503,11 @@ async function waitForWorkflowResult( try { const progress = await handle.query(PROGRESS_QUERY); const elapsed = Math.floor(progress.elapsedMs / 1000); + const expectedCount = progress.expectedAgents.length; + // Agentic SAST runs alongside the phase above, so the line names it while it is working. + const agenticSast = progress.agenticSast.status === 'running' ? ' | Agentic SAST: running' : ''; console.log( - `[${elapsed}s] Phase: ${progress.currentPhase || 'unknown'} | Agent: ${progress.currentAgent || 'none'} | Completed: ${progress.completedAgents.length}/13`, + `[${elapsed}s] Phase: ${progress.currentPhase || 'unknown'} | Agent: ${progress.currentAgent || 'none'} | Completed: ${progress.completedAgents.length + progress.skippedAgents.length}/${expectedCount}${agenticSast}`, ); } catch { // Workflow may have completed @@ -344,12 +518,35 @@ async function waitForWorkflowResult( const result = await handle.result(); clearInterval(progressInterval); - console.log('\nPipeline completed successfully!'); + // The returned workflow state distinguishes completed, partial, and cancelled runs; + // each prints its own terminal line so degradation is never labelled as full success. + if (result.status === 'partial') { + console.log('\nScan completed with gaps (partial). The reasons are listed below.'); + for (const reason of result.partialReasons) { + console.log(` - ${reason.message}`); + } + // The reason above says a class of coverage degraded; these three name the sanitized + // agentic-SAST failure behind it, under the same labels every other surface uses. + if (result.agenticSast.status === 'failed') { + console.log(` Agentic SAST stopped at: ${result.agenticSast.failedStageLabel}`); + console.log(` What happened: ${result.agenticSast.error}`); + if (result.agenticSast.errorCode !== undefined) { + console.log(` Reference code (for a bug report): ${result.agenticSast.errorCode}`); + } + } + } else if (result.status === 'cancelled') { + console.log('\nScan cancelled before it finished.'); + } else { + console.log('\nScan completed.'); + } if (result.summary) { console.log(`Duration: ${Math.floor(result.summary.totalDurationMs / 1000)}s`); - console.log(`Agents completed: ${result.summary.agentCount}`); + console.log(`Agents resolved: ${result.summary.agentCount}`); console.log(`Total turns: ${result.summary.totalTurns}`); console.log(`Run cost: $${result.summary.totalCostUsd.toFixed(4)}`); + if (result.summary.usageAccountingComplete === false) { + console.log('Cost is incomplete — some background work is not included in this total.'); + } if (workspace.isResume) { try { @@ -369,39 +566,6 @@ async function waitForWorkflowResult( } } -// === Deliverables Copy === - -function copyDeliverables(repoPath: string, outputPath: string): void { - const outputDir = deliverablesDir(repoPath); - if (!fs.existsSync(outputDir)) { - console.log('No deliverables directory found, skipping copy'); - return; - } - - const files = fs.readdirSync(outputDir); - if (files.length === 0) { - console.log('No deliverables to copy'); - return; - } - - fs.mkdirSync(outputPath, { recursive: true }); - - for (const file of files) { - if (file === '.git') continue; - const src = path.join(outputDir, file); - const dest = path.join(outputPath, file); - fs.cpSync(src, dest, { recursive: true }); - } - - // Surface the report under its human-facing name alongside the raw deliverables - const assembledPdf = path.join(outputDir, ASSEMBLED_REPORT_PDF_FILENAME); - if (fs.existsSync(assembledPdf)) { - fs.copyFileSync(assembledPdf, path.join(outputPath, FINAL_REPORT_PDF_FILENAME)); - } - - console.log(`Copied ${files.length} deliverable(s) to ${outputPath}`); -} - // === Main Entry Point === async function run(): Promise { @@ -417,30 +581,40 @@ async function run(): Promise { const client = new Client({ connection: clientConnection }); try { - // 3. Bundle workflows and create worker on per-invocation task queue + // 3. Validate orchestration and resume state before terminating any workflow. + const orchestration = await loadOrchestrationConfig(args.configPath); + const workspace = await resolveWorkspace(client, args, orchestration.exploit ?? true); + + // 4. Bundle workflows and create the worker with the collision-checked activity registry. console.log('Preparing scan...'); const workflowBundle = await bundleWorkflowCode({ workflowsPath: path.join(__dirname, 'workflows.js'), }); + const productionActivities = createProductionActivityRegistry({ + repositoryPath: args.repoPath, + webUrl: args.webUrl, + workspacesDir: path.resolve('./workspaces'), + }); + // args.taskQueue is generated fresh per scan (see resolveWorkspace), so Temporal can only + // ever route this worker's activities to this scan's own container: an activity task from + // an older or unrelated scan can never execute against the repo mounted here. const worker = await Worker.create({ connection, namespace: 'default', workflowBundle, - activities, + activities: productionActivities, taskQueue: args.taskQueue, maxConcurrentActivityTaskExecutions: 25, }); - // 4. Resolve workspace and build pipeline input - const workspace = await resolveWorkspace(client, args); - const orchestration = await loadOrchestrationConfig(args.configPath); + // 5. Build the fixed-scope pipeline input. const input = buildPipelineInput(args, workspace, orchestration); - // 5. Start worker polling in the background + // 6. Start worker polling in the background. const workerDone = worker.run(); - // 6. Submit workflow to the same task queue + // 7. Submit workflow to the same task queue. const handle = await client.workflow.start<(input: PipelineInput) => Promise>( 'pentestPipelineWorkflow', { @@ -450,15 +624,10 @@ async function run(): Promise { }, ); - // 7. Wait for workflow result + // 8. Wait for workflow result. await waitForWorkflowResult(handle, workspace); - // 8. Copy deliverables to output directory - if (args.outputPath) { - copyDeliverables(args.repoPath, args.outputPath); - } - - // 9. Shut down worker gracefully + // 9. Shut down worker gracefully. Final customer copies are workflow-owned. worker.shutdown(); await workerDone; } finally { @@ -467,7 +636,10 @@ async function run(): Promise { } } -run().catch((err) => { - console.error('Worker failed:', err); - process.exit(1); -}); +const invokedPath = process.argv[1] ? path.resolve(process.argv[1]) : undefined; +if (invokedPath === fileURLToPath(import.meta.url)) { + run().catch((err) => { + console.error('Worker failed:', err); + process.exit(1); + }); +} diff --git a/apps/worker/src/temporal/workflow-errors.ts b/apps/worker/src/temporal/workflow-errors.ts index e6cc227e..2dbe3fe5 100644 --- a/apps/worker/src/temporal/workflow-errors.ts +++ b/apps/worker/src/temporal/workflow-errors.ts @@ -40,12 +40,18 @@ export function classifyErrorCode(error: unknown): ErrorCode | undefined { return undefined; } -/** Maps Temporal error type strings to actionable remediation hints. */ +/** + * Maps Temporal error type strings to actionable remediation hints. A type earns an entry + * only when the reader has a next step to take; the rest print without a hint line. + */ const REMEDIATION_HINTS: Record = { AuthenticationError: "Verify the selected provider's API key is valid and not expired.", ConfigurationError: 'Check your CONFIG file path and contents.', GitError: 'Check repository path and git state.', InvalidTargetError: 'Verify the target URL is correct and accessible.', + IncompatibleWorkspaceError: 'start a new scan with a different -w name.', + WorkspaceNotFoundError: 'check the -w name against: shannon scans', + PipelineFailedError: 're-run the same -w to retry from the last checkpoint.', }; /** diff --git a/apps/worker/src/temporal/workflows.ts b/apps/worker/src/temporal/workflows.ts index 124e6b93..8e717a6b 100644 --- a/apps/worker/src/temporal/workflows.ts +++ b/apps/worker/src/temporal/workflows.ts @@ -5,42 +5,63 @@ // as published by the Free Software Foundation. /** - * Temporal workflow for Shannon pentest pipeline. + * Current-release Temporal orchestration for the Shannon pentest pipeline. * - * Orchestrates the penetration testing workflow: - * 1. Pre-Reconnaissance (sequential) - * 2. Reconnaissance (sequential) - * 3-4. Vulnerability + Exploitation (5 pipelined pairs in parallel) - * Each pair: vuln agent → queue check → conditional exploit - * No synchronization barrier - exploits start when their vuln finishes - * 5. Reporting (sequential) - * - * Features: - * - Queryable state via getProgress - * - Automatic retry with backoff for transient errors - * - Non-retryable classification for permanent errors - * - Audit correlation via workflowId - * - Graceful failure handling: pipelines continue if one fails + * Every side effect (network, filesystem, git, model calls) is confined to an activity, reached + * only through the proxied namespaces below (`acts`, `testActs`, `preflightActs`, and the rest) + * or through `executeChild` for the Capella child workflow. The functions in this file must stay + * deterministic: Temporal replays them from recorded history instead of re-running real time or + * I/O, so calling `Date.now()` directly in workflow code is safe (the SDK records and replays the + * value), but a raw file read, network call, or `Math.random()` is not. */ +import type { ActivityOptions } from '@temporalio/workflow'; import { ActivityCancellationType, ApplicationFailure, CancellationScope, + executeChild, isCancellation, log, proxyActivities, setHandler, workflowInfo, } from '@temporalio/workflow'; +import type { StageMetrics } from '../ai/reconciliation/stage-contracts.js'; +import type { CapellaWorkflowInput } from '../ai/sast/capella/temporal/activity-types.js'; +import { CAPELLA_CHILD_WORKFLOW_OPTIONS, capellaWorkflow } from '../ai/sast/capella/temporal/workflow.js'; +import type { CapellaRunResult, SarifRef } from '../ai/sast/types.js'; import type { AgentName, VulnType } from '../types/agents.js'; import { ALL_AGENTS } from '../types/agents.js'; import { ALL_VULN_CLASSES, type VulnClass } from '../types/config.js'; +import type { ReconciliationClass } from '../types/reconciliation.js'; +import { + appendPartialReasons, + capellaStageDisplayName, + type MiscellaneousOutcome, + miscellaneousLaneIsSettled, + type PartialReason, + projectPartialReasons, + renderSafeMessage, + reportIsAuthored, +} from '../types/run-state.js'; import type * as activities from './activities.js'; import type { ActivityInput } from './activities.js'; +import { + isAcceptedTaskFormationFallbackReason, + RECONCILIATION_ACTIVITY_PROFILES, + type ReconciliationActivityRegistry, + type ReconciliationClassActivityName, + reconciliationClassDeadlineFrom, + resolveReconciliationActivityBudget, +} from './reconcile-activity-types.js'; import { type AgentMetrics, + type DurableStateSummary, + type FinalizeReportActivityResult, getProgress, + type NonFatalFailure, + type OperationalMetrics, type PipelineInput, type PipelineProgress, type PipelineState, @@ -51,28 +72,18 @@ import { import { toWorkflowSummary } from './summary-mapper.js'; import { classifyErrorCode, formatWorkflowError } from './workflow-errors.js'; -/** Agents this run is expected to produce — drives the resume short-circuit. */ -function computeExpectedAgents(vulnClasses: readonly VulnClass[], exploit: boolean): string[] { - const expected: string[] = ['pre-recon', 'recon']; - for (const cls of vulnClasses) { - expected.push(`${cls}-vuln`); - if (exploit) { - expected.push(`${cls}-exploit`); - } - } - expected.push('report'); - return expected; -} +export { capellaWorkflow }; -// Retry configuration for production (long intervals so a rate-limit window can clear) +// Ordinary agent activities get long timeouts and Temporal's own retry loop, since an +// individual agent run (a model conversation plus tool calls) can legitimately take a long +// time and the workflow, not the agent process, owns restart decisions. The error types listed +// as non-retryable are ones a retry can never fix (bad credentials, invalid config, an +// unreachable target, a failed login), so retrying them would only burn time before failing anyway. const PRODUCTION_RETRY = { initialInterval: '5 minutes', maximumInterval: '30 minutes', backoffCoefficient: 2, maximumAttempts: 50, - // Belt-and-braces: activities already throw non-retryable ApplicationFailures for - // these. Only types that are always permanent belong here — GitError and - // AgentExecutionError carry a per-error verdict and must not be listed. nonRetryableErrorTypes: [ 'AuthenticationError', 'ConfigurationError', @@ -82,7 +93,6 @@ const PRODUCTION_RETRY = { ], }; -// Retry configuration for pipeline testing (fast iteration) const TESTING_RETRY = { initialInterval: '10 seconds', maximumInterval: '30 seconds', @@ -91,25 +101,21 @@ const TESTING_RETRY = { nonRetryableErrorTypes: PRODUCTION_RETRY.nonRetryableErrorTypes, }; -// Activity proxy with production retry configuration (default) const acts = proxyActivities({ startToCloseTimeout: '2 hours', - heartbeatTimeout: '60 minutes', // Extended for nested pi task execution + heartbeatTimeout: '60 minutes', retry: PRODUCTION_RETRY, - // Cancel promptly instead of waiting out startToCloseTimeout; the agent aborts on the signal. cancellationType: ActivityCancellationType.TRY_CANCEL, }); -// Activity proxy with testing retry configuration (fast) const testActs = proxyActivities({ startToCloseTimeout: '30 minutes', - heartbeatTimeout: '30 minutes', // Extended for sub-agent execution in testing + heartbeatTimeout: '30 minutes', retry: TESTING_RETRY, cancellationType: ActivityCancellationType.TRY_CANCEL, }); -// Retry configuration for preflight validation (short timeout, few retries) -const PREFLIGHT_RETRY = { +const SHORT_RETRY = { initialInterval: '10 seconds', maximumInterval: '1 minute', backoffCoefficient: 2, @@ -117,96 +123,282 @@ const PREFLIGHT_RETRY = { nonRetryableErrorTypes: PRODUCTION_RETRY.nonRetryableErrorTypes, }; -// Activity proxy for preflight validation (short timeout) const preflightActs = proxyActivities({ startToCloseTimeout: '2 minutes', heartbeatTimeout: '2 minutes', - retry: PREFLIGHT_RETRY, + retry: SHORT_RETRY, cancellationType: ActivityCancellationType.TRY_CANCEL, }); -// Credential rejection is not retryable; transient provider errors get 3 attempts. -const AUTH_VALIDATION_RETRY = { - initialInterval: '10 seconds', - maximumInterval: '1 minute', - backoffCoefficient: 2, - maximumAttempts: 3, - nonRetryableErrorTypes: PRODUCTION_RETRY.nonRetryableErrorTypes, -}; - -// Browser-driving validation measured at 60–180s; 10 min start-to-close leaves headroom for slow SSO/MFA flows. const authValidationActs = proxyActivities({ startToCloseTimeout: '10 minutes', heartbeatTimeout: '10 minutes', - retry: AUTH_VALIDATION_RETRY, + retry: SHORT_RETRY, cancellationType: ActivityCancellationType.TRY_CANCEL, }); -/** - * Compute aggregated metrics from the current pipeline state. - * Called on both success and failure to provide partial metrics. - */ -function computeSummary(state: PipelineState): PipelineSummary { - const metrics = Object.values(state.agentMetrics); - return { - totalCostUsd: metrics.reduce((sum, m) => sum + (m.costUsd ?? 0), 0), - totalDurationMs: Date.now() - state.startTime, - totalTurns: metrics.reduce((sum, m) => sum + (m.numTurns ?? 0), 0), - agentCount: state.completedAgents.length, - }; -} +// From here down, every proxied namespace mutates durable, git-checkpointed state (report +// progress, reconciliation artifacts, finalization). They use WAIT_CANCELLATION_COMPLETED so a +// cancelled scan lets an in-flight write finish cleanly instead of racing a mid-commit abort; +// the agent activities above use the cheaper TRY_CANCEL because an agent process can simply be +// killed without leaving a half-written checkpoint behind. +const deterministicReportActs = proxyActivities({ + startToCloseTimeout: '2 minutes', + retry: { initialInterval: '1 second', backoffCoefficient: 2, maximumAttempts: 5 }, + cancellationType: ActivityCancellationType.WAIT_CANCELLATION_COMPLETED, +}); + +const finalReportActs = proxyActivities({ + startToCloseTimeout: '10 minutes', + retry: { initialInterval: '1 second', backoffCoefficient: 2, maximumAttempts: 3 }, + cancellationType: ActivityCancellationType.WAIT_CANCELLATION_COMPLETED, +}); + +const surfaceReportActs = proxyActivities({ + startToCloseTimeout: '2 minutes', + retry: { initialInterval: '1 second', backoffCoefficient: 2, maximumAttempts: 3 }, + cancellationType: ActivityCancellationType.WAIT_CANCELLATION_COMPLETED, +}); + +const seedMiscellaneousActs = proxyActivities>({ + startToCloseTimeout: RECONCILIATION_ACTIVITY_PROFILES.seedEmptyProducerQueue.startToCloseTimeoutMs, + scheduleToCloseTimeout: '12 minutes', + retry: { + initialInterval: RECONCILIATION_ACTIVITY_PROFILES.seedEmptyProducerQueue.retryInitialIntervalMs, + backoffCoefficient: RECONCILIATION_ACTIVITY_PROFILES.seedEmptyProducerQueue.retryBackoffCoefficient, + maximumAttempts: RECONCILIATION_ACTIVITY_PROFILES.seedEmptyProducerQueue.maximumAttempts, + }, + cancellationType: ActivityCancellationType.WAIT_CANCELLATION_COMPLETED, +}); -/** One pipeline per vulnerability class, all five in flight together. */ const MAX_CONCURRENT_PIPELINES = 5; - const MAX_PIPELINE_ERROR_MESSAGE_LENGTH = 2000; +const MAX_NON_FATAL_FAILURES = 32; function truncatePipelineErrorMessage(message: string): string { - if (message.length <= MAX_PIPELINE_ERROR_MESSAGE_LENGTH) { - return message; - } + if (message.length <= MAX_PIPELINE_ERROR_MESSAGE_LENGTH) return message; return `${message.slice(0, MAX_PIPELINE_ERROR_MESSAGE_LENGTH - 20)}\n[truncated]`; } +/** Walk a rejection's `.cause` chain into an array, deduped and depth-bounded against a cycle. */ +function failureChain(error: unknown): unknown[] { + const chain: unknown[] = []; + const visited = new Set(); + let current: unknown = error; + while (current !== undefined && current !== null && !visited.has(current) && chain.length < 20) { + chain.push(current); + visited.add(current); + current = current instanceof Error ? current.cause : undefined; + } + return chain; +} + +function hasCancellationInCauseChain(error: unknown): boolean { + return failureChain(error).some((cause) => isCancellation(cause)); +} + +function applicationFailureInChain(error: unknown): ApplicationFailure | undefined { + return failureChain(error).find((cause): cause is ApplicationFailure => cause instanceof ApplicationFailure); +} + +function failureDetailRecord(failure: ApplicationFailure | undefined): Record | undefined { + const first = failure?.details?.[0]; + if (first === null || typeof first !== 'object' || Array.isArray(first)) return undefined; + return first as Record; +} + /** - * Core pipeline orchestration. Coordinates the pentest pipeline stages. - * - * IMPORTANT: This function uses Temporal workflow APIs internally (proxyActivities, - * queries). It can ONLY be called from within a Temporal workflow execution. - * Do not call from standalone scripts or activity code. + * Semantic fallback is restricted to the executor's closed set of accepted Pass 1 + * model-failure reasons. A bare Temporal timeout (heartbeat, schedule-to-close, dead + * worker), an infrastructure failure, a cancellation, or a deterministic integrity error + * must fail the class instead of silently publishing a zero-dedup queue as success. */ -export async function pentestPipeline(input: PipelineInput): Promise { - // Validate repoPath: reject traversal attempts and require absolute path - if (!input.repoPath || input.repoPath.includes('..')) { +function shouldUseSingletonFallback(error: unknown): boolean { + if (hasCancellationInCauseChain(error)) return false; + const failure = applicationFailureInChain(error); + if (failure?.type !== 'TaskFormationModelError' || failure.nonRetryable) return false; + return isAcceptedTaskFormationFallbackReason(failureDetailRecord(failure)?.fallbackReason); +} + +function fallbackMetrics(error: unknown): StageMetrics | undefined { + const details = applicationFailureInChain(error)?.details; + if (!Array.isArray(details)) return undefined; + const first = details[0]; + if (first === null || typeof first !== 'object') return undefined; + const metrics = (first as { metrics?: unknown }).metrics; + if (metrics === null || typeof metrics !== 'object') return undefined; + const value = metrics as Partial; + if ( + typeof value.costUsd !== 'number' || + typeof value.modelCalls !== 'number' || + typeof value.inputTokens !== 'number' || + typeof value.outputTokens !== 'number' + ) { + return undefined; + } + return { + costUsd: value.costUsd, + modelCalls: value.modelCalls, + inputTokens: value.inputTokens, + outputTokens: value.outputTokens, + }; +} + +function reconciliationActivityOptions( + activityName: ReconciliationClassActivityName, + classDeadlineMs: number, + vulnerabilityClass: ReconciliationClass, +): ActivityOptions { + const budget = resolveReconciliationActivityBudget(activityName, classDeadlineMs, Date.now()); + if (!budget.shouldSchedule) { throw ApplicationFailure.nonRetryable( - `Invalid repoPath: path traversal not allowed (received: ${input.repoPath ?? ''})`, + renderSafeMessage( + '{Class} findings took too long to process and the scan stopped that class. Re-running this workspace retries it.', + { vulnerabilityClass }, + ), + 'ConfigurationError', + [{ activityName }], + ); + } + const profile = RECONCILIATION_ACTIVITY_PROFILES[activityName]; + return { + scheduleToCloseTimeout: budget.scheduleToCloseTimeoutMs, + startToCloseTimeout: budget.startToCloseTimeoutMs, + ...(budget.heartbeatTimeoutMs !== null && { heartbeatTimeout: budget.heartbeatTimeoutMs }), + retry: { + initialInterval: profile.retryInitialIntervalMs, + backoffCoefficient: profile.retryBackoffCoefficient, + maximumAttempts: profile.maximumAttempts, + }, + cancellationType: ActivityCancellationType.WAIT_CANCELLATION_COMPLETED, + }; +} + +function reconciliationActs( + activityName: ReconciliationClassActivityName, + classDeadlineMs: number, + vulnerabilityClass: ReconciliationClass, +): ReconciliationActivityRegistry { + return proxyActivities( + reconciliationActivityOptions(activityName, classDeadlineMs, vulnerabilityClass), + ); +} + +function capellaMetrics(result: CapellaRunResult, model: string): OperationalMetrics { + return { + durationMs: result.durationMs, + inputTokens: result.usage.inputTokens, + outputTokens: result.usage.outputTokens, + cacheReadTokens: result.usage.cacheReadTokens, + cacheWriteTokens: result.usage.cacheWriteTokens, + costUsd: result.usage.costUsd, + numTurns: result.usage.turns, + model, + usageComplete: result.usageComplete, + }; +} + +function stageMetrics(metrics: StageMetrics): OperationalMetrics { + return { + durationMs: 0, + inputTokens: metrics.inputTokens, + outputTokens: metrics.outputTokens, + cacheReadTokens: 0, + cacheWriteTokens: 0, + costUsd: metrics.costUsd, + numTurns: metrics.modelCalls, + usageComplete: true, + }; +} + +function computeSummary(state: PipelineState, usageAccountingComplete: boolean): PipelineSummary { + const metrics = [...Object.values(state.agentMetrics), ...Object.values(state.operationalMetrics)]; + return { + totalCostUsd: metrics.reduce((sum, metric) => sum + (metric.costUsd ?? 0), 0), + totalDurationMs: Date.now() - state.startTime, + totalTurns: metrics.reduce((sum, metric) => sum + (metric.numTurns ?? 0), 0), + agentCount: state.completedAgents.length + state.skippedAgents.length, + usageAccountingComplete, + }; +} + +function isAgentName(value: string): value is AgentName { + return (ALL_AGENTS as readonly string[]).includes(value); +} + +/** Core current-release pipeline orchestration. */ +export async function pentestPipeline(input: PipelineInput): Promise { + if (!input.repoPath || input.repoPath.includes('..')) { + throw ApplicationFailure.nonRetryable('Invalid repository path.', 'ConfigurationError'); + } + if (!input.repoPath.startsWith('/')) { + throw ApplicationFailure.nonRetryable('An absolute repository path is required.', 'ConfigurationError'); + } + if (input.agenticSast !== undefined && input.sastSarif !== undefined) { + throw ApplicationFailure.nonRetryable( + 'Agentic SAST cannot run when a static-analysis report is already supplied. Remove the agentic_sast block from your config file, or remove the supplied report.', 'ConfigurationError', ); } - if (!input.repoPath.startsWith('/')) { + if (input.customerOutputPath !== undefined && input.customerOutputPath !== '/app/output') { throw ApplicationFailure.nonRetryable( - `Invalid repoPath: absolute path required (received: ${input.repoPath})`, + 'The customer output mount must use the stable worker path.', 'ConfigurationError', ); } const { workflowId } = workflowInfo(); - const a = input.pipelineTestingMode ? testActs : acts; + const exploit = input.exploit ?? true; + const sessionId = input.sessionId || input.resumeFromWorkspace || workflowId; + const stateContext: 'fresh' | 'resume' = input.resumeFromWorkspace ? 'resume' : 'fresh'; const state: PipelineState = { status: 'running', currentPhase: null, currentAgent: null, completedAgents: [], + expectedAgents: [], + participatingClasses: [], failedPipelines: [], + failedReconciliations: [], failedAgent: null, error: null, startTime: Date.now(), + skippedAgents: [], agentMetrics: {}, + operationalMetrics: {}, + operationalStages: {}, + agenticSast: { status: 'disabled' }, + nonFatalFailures: [], + partialReasons: [], summary: null, }; + // The durable degradation record. Codes plus bounded context are the identity; the + // projection into state carries derived safe messages for every consumer surface. + let partialReasons: readonly PartialReason[] = []; + // True once reconciliation adopted a prior run's publication, whose model spend is not + // visible to this run's metrics. Surfaced instead of inventing the missing spend. + let operationalSpendMissing = false; + + function addPartialReason(reason: PartialReason): void { + partialReasons = appendPartialReasons(partialReasons, [reason]); + state.partialReasons = [...projectPartialReasons(partialReasons)]; + } + + function adoptDurableReasons(durable: readonly PartialReason[]): void { + partialReasons = appendPartialReasons(partialReasons, durable); + state.partialReasons = [...projectPartialReasons(partialReasons)]; + } + + function usageAccountingComplete(): boolean { + const everyOperationalMetricComplete = Object.values(state.operationalMetrics).every( + (metric) => metric.usageComplete !== false, + ); + return everyOperationalMetricComplete && !operationalSpendMissing; + } + setHandler( getProgress, (): PipelineProgress => ({ @@ -216,477 +408,783 @@ export async function pentestPipeline(input: PipelineInput): Promise 0 ? input.vulnClasses : ALL_VULN_CLASSES; - const selectedClassSet = new Set(selectedVulnClasses); - const exploit: boolean = input.exploit ?? true; - const expectedAgents = computeExpectedAgents(selectedVulnClasses, exploit); - - await a.persistOrValidateRunScope(activityInput, [...selectedVulnClasses], exploit); - let resumeState: ResumeState | null = null; + let miscellaneousOutcome: MiscellaneousOutcome | undefined; - if (input.resumeFromWorkspace) { - // 0. Register the resume's workflow id in session.json before validation can fail, so the CLI - // can resolve and follow it instead of polling for an entry that never lands. - await a.registerResumeAttempt(activityInput, input.terminatedWorkflows || []); - - // 1. Load resume state (validates workspace, cross-checks deliverables) - resumeState = await a.loadResumeState( - input.resumeFromWorkspace, - input.webUrl, - input.repoPath, - input.deliverablesSubdir, - ); - - // 2. Restore git workspace and clean up incomplete deliverables - const incompleteAgents = ALL_AGENTS.filter( - (agentName) => !resumeState?.completedAgents.includes(agentName), - ) as AgentName[]; - - await a.restoreGitCheckpoint( - input.repoPath, - resumeState.checkpointHash, - incompleteAgents, - input.deliverablesSubdir, - ); - - // 3. Short-circuit when every agent expected by this run is done. - // Uses dynamic expectedAgents (not ALL_AGENTS) so a class-scoped run completes sooner. - const allExpectedDone = expectedAgents.every((a) => resumeState?.completedAgents.includes(a)); - if (allExpectedDone) { - log.info(`All ${expectedAgents.length} expected agents already completed. Nothing to resume.`); - state.status = 'completed'; - state.completedAgents = [...resumeState.completedAgents]; - state.summary = computeSummary(state); - return state; - } - - // 4. Write the resume header to workflow.log (the session.json entry was recorded in step 0) - await a.recordResumeAttempt( - activityInput, - resumeState.checkpointHash, - resumeState.originalWorkflowId, - resumeState.completedAgents, - ); - - log.info('Resume state loaded and workspace restored'); + function applyDurableSummary(summary: DurableStateSummary): void { + state.expectedAgents = [...summary.expectedAgents]; + state.participatingClasses = [...summary.participatingClasses]; + if (summary.miscellaneousOutcome !== undefined) miscellaneousOutcome = summary.miscellaneousOutcome; } - const shouldSkip = (agentName: string): boolean => { + /** An agent that actually ran and finished. Mutually exclusive from markSkipped. */ + function markCompleted(agentName: AgentName): void { + if (!state.expectedAgents.includes(agentName)) return; + if (!state.completedAgents.includes(agentName)) state.completedAgents.push(agentName); + } + + /** + * An expected agent that never ran because its class had nothing to exploit. It is tracked + * only in `skippedAgents`, mutually exclusive from `completedAgents`. Pipeline resolution is + * the union of the two lists; the summary counts them together. + */ + function markSkipped(agentName: AgentName): void { + if (!state.expectedAgents.includes(agentName)) return; + if (!state.skippedAgents.includes(agentName)) state.skippedAgents.push(agentName); + } + + function shouldSkip(agentName: AgentName): boolean { return resumeState?.completedAgents.includes(agentName) ?? false; - }; + } + + // Bounded so a pathological run cannot grow workflow state, and workflow history, without + // limit; an entry past the cap is dropped silently rather than turned into a failure of its own. + function addNonFatal(failure: NonFatalFailure): void { + if (state.nonFatalFailures.length >= MAX_NON_FATAL_FAILURES) return; + state.nonFatalFailures.push({ + phase: failure.phase, + error: truncatePipelineErrorMessage(failure.error), + }); + } + + function startOperation(key: string, label: string): number { + const startedAt = Date.now(); + state.operationalStages[key] = { key, label, status: 'running', startedAt }; + return startedAt; + } + + function completeOperation(key: string, label: string, startedAt: number): void { + state.operationalStages[key] = { + key, + label, + status: 'completed', + startedAt, + durationMs: Date.now() - startedAt, + }; + } + + function failOperation(key: string, label: string, startedAt: number, error: unknown): void { + const message = truncatePipelineErrorMessage(error instanceof Error ? error.message : String(error)); + state.operationalStages[key] = { + key, + label, + status: 'failed', + startedAt, + durationMs: Date.now() - startedAt, + error: message, + }; + } + + /** A stage an earlier run already settled. It records no span, so it contributes no wall time. */ + function skipOperation(key: string, label: string): void { + state.operationalStages[key] = { key, label, status: 'skipped' }; + } + + async function runOperation(key: string, label: string, operation: () => Promise): Promise { + const startedAt = startOperation(key, label); + try { + const result = await operation(); + completeOperation(key, label, startedAt); + return result; + } catch (error) { + failOperation(key, label, startedAt, error); + throw error; + } + } + + function addReconciliationMetrics( + vulnerabilityClass: ReconciliationClass, + stage: 'enrich' | 'form', + metrics: StageMetrics, + ): void { + state.operationalMetrics[`reconciliation:${vulnerabilityClass}:${stage}`] = stageMetrics(metrics); + } - // Run a sequential agent phase (pre-recon, recon) async function runSequentialPhase( phaseName: string, agentName: AgentName, runAgent: (input: ActivityInput) => Promise, ): Promise { - if (!shouldSkip(agentName)) { - state.currentPhase = phaseName; - state.currentAgent = agentName; - await a.logPhaseTransition(activityInput, phaseName, 'start'); - state.agentMetrics[agentName] = await runAgent(activityInput); - state.completedAgents.push(agentName); - if (input.checkpointsEnabled) { - await a.saveCheckpoint(activityInput, agentName, phaseName, state); - } - await a.logPhaseTransition(activityInput, phaseName, 'complete'); - } else { + if (shouldSkip(agentName)) { log.info(`Skipping ${agentName} (already complete)`); - state.completedAgents.push(agentName); + markCompleted(agentName); + return; } + state.currentPhase = phaseName; + state.currentAgent = agentName; + await a.logPhaseTransition(activityInput, phaseName, 'start'); + state.agentMetrics[agentName] = await runAgent(activityInput); + markCompleted(agentName); + if (input.checkpointsEnabled) await a.saveCheckpoint(activityInput, agentName, phaseName, state); + await a.logPhaseTransition(activityInput, phaseName, 'complete'); + } + + async function reconcileClass(vulnerabilityClass: ReconciliationClass, sarif?: SarifRef): Promise { + const key = `reconciliation:${vulnerabilityClass}`; + const label = `Reconcile ${vulnerabilityClass}`; + await runOperation(key, label, async () => { + const classDeadlineMs = reconciliationClassDeadlineFrom(Date.now()); + const baseInput = { sessionId, vulnerabilityClass, classDeadlineMs }; + const prepared = await reconciliationActs( + 'prepareClassReconciliation', + classDeadlineMs, + vulnerabilityClass, + ).prepareClassReconciliation({ + ...baseInput, + includeSastProvenance: sarif !== undefined, + }); + if (prepared.outcome === 'already_published') { + // A prior run paid for this publication; its model spend is absent from this + // run's metrics, so the cost total is surfaced as incomplete rather than invented. + operationalSpendMissing = true; + return; + } + + const enriched = await reconciliationActs( + 'enrichClassSastObservations', + classDeadlineMs, + vulnerabilityClass, + ).enrichClassSastObservations({ + ...baseInput, + ...(sarif !== undefined && { sarif }), + }); + addReconciliationMetrics(vulnerabilityClass, 'enrich', enriched.metrics); + + let formation: + | Awaited> + | 'singleton_fallback'; + try { + formation = await reconciliationActs( + 'formClassExploitTasks', + classDeadlineMs, + vulnerabilityClass, + ).formClassExploitTasks({ + ...baseInput, + producerRef: prepared.ref, + supplementalRef: enriched.ref, + }); + addReconciliationMetrics(vulnerabilityClass, 'form', formation.metrics); + } catch (error) { + if (!shouldUseSingletonFallback(error)) throw error; + const metrics = fallbackMetrics(error); + if (metrics !== undefined) addReconciliationMetrics(vulnerabilityClass, 'form', metrics); + formation = 'singleton_fallback'; + // Make the degradation visible in queryable state: every observation becomes its + // own task, so duplicates are expected instead of silently absent dedup. + const fallbackKey = `reconciliation:${vulnerabilityClass}:fallback`; + completeOperation(fallbackKey, `Grouping skipped (${vulnerabilityClass})`, Date.now()); + log.info( + renderSafeMessage( + '{Class} findings could not be grouped, so each one will be tested separately. Expect duplicates in the results.', + { vulnerabilityClass }, + ), + ); + } + + const materialized = await reconciliationActs( + 'materializeClassExploitTasks', + classDeadlineMs, + vulnerabilityClass, + ).materializeClassExploitTasks({ + ...baseInput, + producerRef: prepared.ref, + supplementalRef: enriched.ref, + form: formation, + }); + await reconciliationActs( + 'publishClassReconciliationOss', + classDeadlineMs, + vulnerabilityClass, + ).publishClassReconciliationOss({ + ...baseInput, + producerRef: prepared.ref, + supplementalRef: enriched.ref, + fixedTasksRef: materialized.ref, + }); + }); } - // Build pipeline configs for the 5 vuln→exploit pairs function buildPipelineConfigs(): Array<{ vulnType: VulnType; - vulnAgent: string; - exploitAgent: string; runVuln: () => Promise; runExploit: () => Promise; }> { return [ { vulnType: 'injection', - vulnAgent: 'injection-vuln', - exploitAgent: 'injection-exploit', runVuln: () => a.runInjectionVulnAgent(activityInput), runExploit: () => a.runInjectionExploitAgent(activityInput), }, { vulnType: 'xss', - vulnAgent: 'xss-vuln', - exploitAgent: 'xss-exploit', runVuln: () => a.runXssVulnAgent(activityInput), runExploit: () => a.runXssExploitAgent(activityInput), }, { vulnType: 'auth', - vulnAgent: 'auth-vuln', - exploitAgent: 'auth-exploit', runVuln: () => a.runAuthVulnAgent(activityInput), runExploit: () => a.runAuthExploitAgent(activityInput), }, - { - vulnType: 'ssrf', - vulnAgent: 'ssrf-vuln', - exploitAgent: 'ssrf-exploit', - runVuln: () => a.runSsrfVulnAgent(activityInput), - runExploit: () => a.runSsrfExploitAgent(activityInput), - }, { vulnType: 'authz', - vulnAgent: 'authz-vuln', - exploitAgent: 'authz-exploit', runVuln: () => a.runAuthzVulnAgent(activityInput), runExploit: () => a.runAuthzExploitAgent(activityInput), }, + { + vulnType: 'ssrf', + runVuln: () => a.runSsrfVulnAgent(activityInput), + runExploit: () => a.runSsrfExploitAgent(activityInput), + }, ]; } - // A rejected settle can be a genuine Temporal cancellation (runVulnExploitPipeline rethrows in - // its isCancellation branch). Cancellation must win over failed/partial classification — there is - // no report worth shipping once the user has cancelled, and rethrowing lets the workflow's outer - // isCancellation handler produce a real cancelled state instead of a hard failure. - function throwIfPipelineCancelled(results: PromiseSettledResult[]): void { - const cancelled = results.find( - (r): r is PromiseRejectedResult => r.status === 'rejected' && isCancellation(r.reason), - ); - if (cancelled) { - throw cancelled.reason; - } - } - - // Classify the settled pipeline results into clean / partial / fail-hard. - // Metrics and completedAgents are updated incrementally inside runVulnExploitPipeline - // so that getProgress queries reflect real-time status during execution. - function aggregatePipelineResults( - results: PromiseSettledResult[], - alreadyCompletedPipelineCount: number, - ): void { - throwIfPipelineCancelled(results); - - const failed: { vulnType: VulnClass; error: string }[] = []; - // A rejected settle is now unexpected (runVulnExploitPipeline catches and returns its error in - // the value). Without a value we cannot attribute the failure to a class, so we treat it as a - // hard failure rather than risk an under-qualified report. - const unattributable: string[] = []; - - for (const result of results) { - if (result.status === 'fulfilled') { - if (result.value.error !== null) { - failed.push({ vulnType: result.value.vulnType, error: result.value.error }); - } + /** + * One vulnerability class's full lane: the vuln agent, joining the shared Capella settlement, + * reconciliation, the exploitation decision, and (if warranted) the exploit agent. Every + * failure except cancellation is caught here and turned into a per-class result instead of + * being rethrown, so one class failing never aborts the classes running alongside it. Whether + * reconciliation had already started when the failure hit picks which of the two safe messages + * and partial-reason codes the class is recorded under. + */ + async function runVulnExploitPipeline( + vulnType: VulnType, + runVulnAgent: () => Promise, + runExploitAgent: () => Promise, + effectiveSarif?: SarifRef, + ): Promise { + const vulnAgentName = `${vulnType}-vuln` as AgentName; + const exploitAgentName = `${vulnType}-exploit` as AgentName; + let reconciliationStarted = false; + let reconciliationCompleted = false; + try { + let vulnMetrics: AgentMetrics | null = null; + if (shouldSkip(vulnAgentName)) { + markCompleted(vulnAgentName); } else { - const rawMessage = result.reason instanceof Error ? result.reason.message : String(result.reason); - unattributable.push(truncatePipelineErrorMessage(rawMessage)); + vulnMetrics = await runVulnAgent(); + state.agentMetrics[vulnAgentName] = vulnMetrics; + markCompleted(vulnAgentName); + if (input.checkpointsEnabled) + await a.saveCheckpoint(activityInput, vulnAgentName, 'vulnerability-analysis', state); } - } - const failedCount = failed.length + unattributable.length; - const totalPipelineCount = results.length + alreadyCompletedPipelineCount; - if (failedCount === 0) { - return; - } + reconciliationStarted = true; + await reconcileClass(vulnType, effectiveSarif); + reconciliationCompleted = true; + const decision = await a.checkExploitationQueue(activityInput, vulnType); + let exploitMetrics: AgentMetrics | null = null; + if (exploit && shouldSkip(exploitAgentName)) { + markCompleted(exploitAgentName); + } else if (exploit && decision.shouldExploit) { + exploitMetrics = await runExploitAgent(); + state.agentMetrics[exploitAgentName] = exploitMetrics; + markCompleted(exploitAgentName); + if (input.checkpointsEnabled) await a.saveCheckpoint(activityInput, exploitAgentName, 'exploitation', state); + } else if (exploit) { + markSkipped(exploitAgentName); + if (input.checkpointsEnabled) await a.saveCheckpoint(activityInput, exploitAgentName, 'exploitation', state); + } - // All run pipelines failed, or a failure we cannot attribute to a class → fail-hard. There is - // no report worth shipping, and we must never render an un-assessed class as if it passed. - if (failedCount === totalPipelineCount || unattributable.length > 0) { - const allErrors = [...failed.map((f) => `${f.vulnType}: ${f.error}`), ...unattributable]; - const message = `${failedCount} vulnerability/exploitation pipeline(s) failed`; - state.status = 'failed'; - state.failedAgent = 'pipelines'; - state.error = `${message}: ${allErrors.join('; ')}`; - log.warn(message, { failures: allErrors }); - throw ApplicationFailure.nonRetryable(state.error, 'PipelineFailedError', [{ failures: allErrors }]); + return { + vulnType, + vulnMetrics, + exploitMetrics, + exploitDecision: { shouldExploit: decision.shouldExploit, vulnerabilityCount: decision.vulnerabilityCount }, + error: null, + }; + } catch (error) { + if (hasCancellationInCauseChain(error)) throw error; + const message = truncatePipelineErrorMessage(error instanceof Error ? error.message : String(error)); + if (reconciliationStarted && !reconciliationCompleted) { + state.failedReconciliations.push({ vulnerabilityClass: vulnType, error: message }); + addPartialReason({ code: 'class_reconciliation_failed', vulnerabilityClass: vulnType }); + } else { + addPartialReason({ code: 'class_pipeline_failed', vulnerabilityClass: vulnType }); + } + return { + vulnType, + vulnMetrics: state.agentMetrics[vulnAgentName] ?? null, + exploitMetrics: state.agentMetrics[exploitAgentName] ?? null, + exploitDecision: null, + error: message, + }; } - - // Partial: at least one class succeeded and at least one failed. Record the failed classes and - // set the partial terminal status; do NOT throw — the successful pipelines still ship. - state.failedPipelines = failed; - state.status = 'partial'; - log.warn(`${failed.length} of ${totalPipelineCount} pipeline(s) failed — continuing with partial results`, { - failures: failed.map((f) => `${f.vulnType}: ${f.error}`), - }); } - // Run thunks with a concurrency limit, returning PromiseSettledResult for each. - // When limit >= thunks.length (default), all launch concurrently — identical to Promise.allSettled. - // NOTE: Results are in completion order, not input order. Callers must key on value fields, not index. + /** + * Run `thunks` with at most `limit` in flight, collecting every settlement instead of + * failing fast, so one class's rejection never cancels the classes still running alongside it. + */ async function runWithConcurrencyLimit( thunks: Array<() => Promise>, limit: number, ): Promise[]> { const results: PromiseSettledResult[] = []; const inFlight = new Set>(); - for (const thunk of thunks) { const slot = thunk() .then( - (value) => { - results.push({ status: 'fulfilled', value }); - }, - (reason: unknown) => { - results.push({ status: 'rejected', reason }); - }, + (value) => results.push({ status: 'fulfilled', value }), + (reason: unknown) => results.push({ status: 'rejected', reason }), ) - .finally(() => { - inFlight.delete(slot); - }); - + .then(() => undefined) + .finally(() => inFlight.delete(slot)); inFlight.add(slot); - - if (inFlight.size >= limit) { - await Promise.race(inFlight); - } + if (inFlight.size >= limit) await Promise.race(inFlight); } - await Promise.allSettled(inFlight); return results; } + function aggregatePipelineResults(results: PromiseSettledResult[]): void { + const cancelled = results.find( + (result): result is PromiseRejectedResult => + result.status === 'rejected' && hasCancellationInCauseChain(result.reason), + ); + if (cancelled) throw cancelled.reason; + + const failed: { vulnType: VulnClass; error: string }[] = []; + const unattributable: string[] = []; + for (const result of results) { + if (result.status === 'fulfilled') { + if (result.value.error !== null) failed.push({ vulnType: result.value.vulnType, error: result.value.error }); + } else { + unattributable.push( + truncatePipelineErrorMessage(result.reason instanceof Error ? result.reason.message : String(result.reason)), + ); + } + } + if (failed.length === 0 && unattributable.length === 0) return; + // Fail the whole phase when every class failed, or when any result is unattributable. + // A class pipeline catches its own errors and reports them in `error`, so a rejected + // thunk means a failure escaped that path and cannot be pinned to one class, which is + // never safe to downgrade to a partial run. + if (failed.length + unattributable.length === ALL_VULN_CLASSES.length || unattributable.length > 0) { + const errors = [...failed.map((failure) => `${failure.vulnType}: ${failure.error}`), ...unattributable]; + throw ApplicationFailure.nonRetryable( + 'The vulnerability analysis phase failed and the scan cannot continue. Re-running this workspace retries it from the last checkpoint.', + 'PipelineFailedError', + [{ failures: errors }], + ); + } + state.failedPipelines = failed; + } + + /** + * Run Capella as a child workflow when agentic SAST is configured, or pass through a + * pre-supplied SARIF report unchanged when it is not. Every outcome this function can observe, + * whether success, reduced coverage, a Capella-reported failure, or an escaped exception, is + * projected into `state.agenticSast` and, where relevant, a durable partial reason before + * returning, so a caller reads the settled SARIF (`undefined` on anything but success) without + * needing its own failure-handling path. + */ + async function runCapella(): Promise { + if (input.agenticSast === undefined) { + state.agenticSast = { status: 'disabled' }; + return input.sastSarif; + } + + const key = 'agentic-sast'; + const label = 'Agentic SAST'; + const startedAt = startOperation(key, label); + state.agenticSast = { status: 'running', startedAt }; + const auditRoot = (input.auditDir ?? '/app/workspaces').replace(/\/+$/, ''); + const capellaInput: CapellaWorkflowInput = { + repoPath: input.repoPath, + artifactRoot: `${auditRoot}/${sessionId}/.shannon/capella`, + workflowLogPath: `${auditRoot}/${sessionId}/.shannon/workflow.log`, + promptDir: input.promptDir ?? '/app/apps/worker/prompts', + codePathAvoids: [...input.agenticSast.codePathAvoids], + codePathFocus: [...input.agenticSast.codePathFocus], + modelSpec: input.agenticSast.modelSpec, + capellaFormatVersion: input.agenticSast.capellaFormatVersion, + promptSetVersion: input.agenticSast.promptSetVersion, + pipelineTestingMode: input.pipelineTestingMode ?? false, + }; + + try { + const result = await executeChild(capellaWorkflow, { + ...CAPELLA_CHILD_WORKFLOW_OPTIONS, + workflowId: `${workflowId}-capella`, + args: [capellaInput], + }); + const metricKey = result.status === 'succeeded' ? 'agentic-sast:export' : `agentic-sast:${result.failedStage}`; + state.operationalMetrics[metricKey] = capellaMetrics(result, input.agenticSast.modelSpec); + if (result.status === 'succeeded') { + state.agenticSast = { + status: 'succeeded', + findingCount: result.findingCount, + sarifSha256: result.sarif.sha256, + coverage: result.coverage, + warnings: [...result.warnings], + durationMs: result.durationMs, + }; + completeOperation(key, label, startedAt); + if (result.coverage === 'reduced') { + addPartialReason({ code: 'agentic_sast_reduced' }); + addNonFatal({ phase: 'agentic-sast', error: 'Agentic SAST completed with reduced coverage.' }); + } + return result.sarif; + } + + state.agenticSast = { + status: 'failed', + failedStage: result.failedStage, + failedStageLabel: capellaStageDisplayName(result.failedStage), + error: result.error, + ...(result.errorCode !== undefined && { errorCode: result.errorCode }), + completedStages: [...result.completedStages], + durationMs: result.durationMs, + }; + addPartialReason({ code: 'agentic_sast_failed', stage: result.failedStage }); + failOperation(key, label, startedAt, new Error(result.error)); + addNonFatal({ + phase: 'agentic-sast', + error: result.errorCode === undefined ? result.error : `${result.error} [${result.errorCode}]`, + }); + return undefined; + } catch (error) { + if (hasCancellationInCauseChain(error)) throw error; + const message = 'Agentic SAST infrastructure failed before producing a usable result.'; + state.agenticSast = { + status: 'failed', + failedStage: 'workflow', + failedStageLabel: capellaStageDisplayName('workflow'), + error: message, + completedStages: [], + durationMs: Date.now() - startedAt, + }; + addPartialReason({ code: 'agentic_sast_failed', stage: 'workflow' }); + failOperation(key, label, startedAt, new Error(message)); + addNonFatal({ phase: 'agentic-sast', error: message }); + return undefined; + } + } + + /** + * The internal `miscellaneous` class: findings outside the five fixed vulnerability classes, + * carried through the same reconciliation and exploitation-decision path those classes use. + * Its outcome is durably recorded (not just success/failure) so a resumed run knows whether + * the class was ever admitted for exploitation, rather than re-deciding admission from scratch. + */ + async function runMiscellaneousPipeline(effectiveSarif: SarifRef): Promise { + const key = 'miscellaneous-pipeline'; + const label = 'Miscellaneous findings'; + if (miscellaneousLaneIsSettled(miscellaneousOutcome)) { + if (miscellaneousOutcome === 'completed') markCompleted('miscellaneous-exploit'); + skipOperation(key, label); + return; + } + const startedAt = startOperation(key, label); + let reconciliationCompleted = false; + try { + await seedMiscellaneousActs.seedEmptyProducerQueue({ sessionId }); + await reconcileClass('miscellaneous', effectiveSarif); + reconciliationCompleted = true; + const decision = await a.checkExploitationQueue(activityInput, 'miscellaneous' as VulnType); + let outcome: MiscellaneousOutcome; + if (!exploit) { + outcome = 'exploitation_disabled'; + } else if (!decision.shouldExploit) { + outcome = 'not_actionable'; + } else { + const admitted = await deterministicReportActs.persistMiscellaneousOutcome(activityInput, 'expected'); + applyDurableSummary(admitted); + if (!shouldSkip('miscellaneous-exploit')) + state.agentMetrics['miscellaneous-exploit'] = await a.runMiscellaneousExploitAgent(activityInput); + markCompleted('miscellaneous-exploit'); + outcome = 'completed'; + } + const persisted = await deterministicReportActs.persistMiscellaneousOutcome(activityInput, outcome); + applyDurableSummary(persisted); + completeOperation(key, label, startedAt); + } catch (error) { + if (hasCancellationInCauseChain(error)) throw error; + failOperation(key, label, startedAt, error); + const message = truncatePipelineErrorMessage(error instanceof Error ? error.message : String(error)); + if (!reconciliationCompleted) { + state.failedReconciliations.push({ vulnerabilityClass: 'miscellaneous', error: message }); + addPartialReason({ code: 'class_reconciliation_failed', vulnerabilityClass: 'miscellaneous' }); + } else { + addPartialReason({ code: 'class_pipeline_failed', vulnerabilityClass: 'miscellaneous' }); + } + addNonFatal({ + phase: reconciliationCompleted ? 'miscellaneous-pipeline' : 'reconciliation:miscellaneous', + error: message, + }); + } + } + + function recordAssemblyOmissions(failedClasses: readonly ReconciliationClass[]): void { + for (const vulnerabilityClass of failedClasses) { + // The append rules drop the omission when the class already carries an upstream reason. + addPartialReason({ code: 'report_class_omitted', vulnerabilityClass }); + const isAnalysisClass = (ALL_VULN_CLASSES as readonly string[]).includes(vulnerabilityClass); + if (isAnalysisClass && !(activityInput.failedClasses ?? []).includes(vulnerabilityClass as VulnClass)) { + activityInput.failedClasses = [...(activityInput.failedClasses ?? []), vulnerabilityClass as VulnClass]; + } + } + } + + /** True only for the exact retryable SARIF-render failure type after its own activity retry policy exhausted; nothing else may trigger degraded finalization. */ + function isSarifRenderExhaustion(error: unknown): boolean { + return applicationFailureInChain(error)?.type === 'ReportSarifRenderError'; + } + + /** + * Drive the durable report state machine from wherever a fresh or resumed run finds it + * (pending, draft, or finalized) through to a finalized, surfaced report. Each stage below + * persists its result before the next stage begins, so a crash mid-pipeline resumes from the + * last persisted stage instead of re-running work that already completed. + */ + async function finalizeReportPipeline(): Promise { + state.currentPhase = 'reporting'; + state.currentAgent = 'report'; + await a.logPhaseTransition(activityInput, 'reporting', 'start'); + + if (state.reportProgress === undefined) { + const renumberFailed: ReconciliationClass[] = []; + if (exploit) { + for (const vulnerabilityClass of state.participatingClasses) { + const key = `report:renumber:${vulnerabilityClass}`; + try { + await runOperation(key, `Renumber ${vulnerabilityClass}`, () => + deterministicReportActs.renumberClassFindings(activityInput, vulnerabilityClass), + ); + } catch (error) { + if (hasCancellationInCauseChain(error)) throw error; + renumberFailed.push(vulnerabilityClass); + addPartialReason({ code: 'report_renumber_failed', vulnerabilityClass }); + addNonFatal({ phase: key, error: error instanceof Error ? error.message : String(error) }); + } + } + } + state.reportProgress = await runOperation('report:initialize', 'Initialize report state', () => + deterministicReportActs.initializeReportProgress(activityInput, renumberFailed, partialReasons), + ); + adoptDurableReasons(state.reportProgress.partial_reasons); + } + + if (state.reportProgress.stage === 'pending') { + const assembled = await runOperation('report:assemble', 'Assemble report inputs', () => + deterministicReportActs.assembleReportActivity(activityInput, exploit), + ); + recordAssemblyOmissions(assembled.failedClasses); + const reportMetrics = await a.runReportAgent(activityInput, exploit); + state.agentMetrics.report = reportMetrics; + if (reportMetrics.checkpoint === undefined) { + throw ApplicationFailure.nonRetryable( + 'The report was written but could not be saved. Re-running this workspace retries the reporting phase without repeating the analysis.', + 'ReportDraftError', + ); + } + state.reportProgress = { + stage: 'draft', + renumber_failed_classes: [...state.reportProgress.renumber_failed_classes], + partial_reasons: [...state.reportProgress.partial_reasons], + model_checkpoint: reportMetrics.checkpoint, + }; + } + + if (state.reportProgress.stage === 'draft' && state.reportProgress.canonical_checkpoint === undefined) { + let canonicalCheckpoint = state.reportProgress.model_checkpoint; + if (exploit) { + try { + const compacted = await runOperation('report:compact', 'Compact report findings', () => + deterministicReportActs.compactReportFindings(activityInput), + ); + canonicalCheckpoint = compacted.checkpoint ?? canonicalCheckpoint; + } catch (error) { + if (hasCancellationInCauseChain(error)) throw error; + addPartialReason({ code: 'report_compaction_failed' }); + addNonFatal({ phase: 'report:compact', error: error instanceof Error ? error.message : String(error) }); + } + } + state.reportProgress = await runOperation('report:checkpoint', 'Saving report progress', () => + deterministicReportActs.persistCanonicalReportProgress(activityInput, canonicalCheckpoint, partialReasons), + ); + adoptDurableReasons(state.reportProgress.partial_reasons); + } + + let finalized: FinalizeReportActivityResult; + try { + finalized = await runOperation('report:finalize', 'Finalize report outputs', () => + finalReportActs.finalizeReportOutputs(activityInput), + ); + } catch (error) { + if (hasCancellationInCauseChain(error)) throw error; + // Only the exact retryable SARIF render type may degrade, and only after its ordinary + // three-attempt policy exhausted. The degraded call still adopts a coherent earlier + // commit first, so a prior committed finalization keeps its committed disposition. + if (!isSarifRenderExhaustion(error)) throw error; + finalized = await runOperation('report:finalize-degraded', 'Finalize report without SARIF', () => + finalReportActs.finalizeReportOutputs(activityInput, true), + ); + } + if (finalized.sarifDisposition === 'render_failed') { + addPartialReason({ code: 'report_sarif_failed' }); + } + state.reportProgress = await runOperation('report:terminal', 'Saving final report state', () => + deterministicReportActs.persistFinalizedReportProgress( + activityInput, + finalized.checkpoint, + finalized.manifestSha256, + { + sarifDisposition: finalized.sarifDisposition, + pdfProvenance: finalized.pdfProvenance, + partialReasons, + }, + ), + ); + adoptDurableReasons(state.reportProgress.partial_reasons); + markCompleted('report'); + + if (finalized.warningCount > 0) { + addNonFatal({ phase: 'report-output', error: 'One or more derived report outputs emitted warnings.' }); + } + try { + const surfaced = await runOperation('report:surface', 'Surface customer report', () => + surfaceReportActs.surfaceReportOutputs(activityInput), + ); + if (surfaced.warningCount > 0) { + addNonFatal({ phase: 'report-surface', error: 'One or more customer report copies emitted warnings.' }); + } + } catch (error) { + if (hasCancellationInCauseChain(error)) throw error; + addNonFatal({ + phase: 'report-surface', + error: 'Customer report copies could not be refreshed; canonical outputs remain finalized.', + }); + } + await a.logPhaseTransition(activityInput, 'reporting', 'complete'); + } + try { - // === Preflight Validation === - // Quick sanity checks before committing to expensive agent runs. - // NOT using runSequentialPhase — preflight doesn't produce AgentMetrics. + const durable = await deterministicReportActs.initializeDurableScanState(activityInput, exploit, stateContext); + applyDurableSummary(durable); + + if (input.resumeFromWorkspace) { + // The new workflow id lands in session.json before anything that can reject the resume, so a + // validation or checkpoint-restore failure still leaves the CLI an attempt to follow. + await deterministicReportActs.registerResumeAttempt(activityInput, input.terminatedWorkflows ?? []); + resumeState = await deterministicReportActs.loadResumeState( + input.resumeFromWorkspace, + input.webUrl, + input.repoPath, + { + ...(input.deliverablesSubdir !== undefined && { deliverablesSubdir: input.deliverablesSubdir }), + expectedExploit: exploit, + }, + ); + state.expectedAgents = [...resumeState.expectedAgents]; + state.participatingClasses = [...resumeState.participatingClasses]; + if (resumeState.miscellaneousOutcome !== undefined) miscellaneousOutcome = resumeState.miscellaneousOutcome; + if (resumeState.reportProgress !== undefined) { + state.reportProgress = resumeState.reportProgress; + // Durable reasons are restored, never reconstructed from session status or errors. + adoptDurableReasons(resumeState.reportProgress.partial_reasons); + } + + const expectedAgentNames = resumeState.expectedAgents.filter(isAgentName); + const incompleteAgents = expectedAgentNames.filter( + (agentName) => !resumeState?.completedAgents.includes(agentName), + ); + await deterministicReportActs.restoreGitCheckpoint( + input.repoPath, + resumeState.checkpointHash, + incompleteAgents, + input.deliverablesSubdir, + { + expectedAgents: expectedAgentNames, + participatingClasses: resumeState.participatingClasses, + ...(resumeState.reportProgress !== undefined && { reportProgress: resumeState.reportProgress }), + }, + ); + await deterministicReportActs.recordResumeAttempt( + activityInput, + resumeState.checkpointHash, + resumeState.originalWorkflowId, + resumeState.completedAgents, + ); + for (const agentName of resumeState.completedAgents) { + if (isAgentName(agentName)) markCompleted(agentName); + } + } + state.currentPhase = 'preflight'; state.currentAgent = null; await preflightActs.runPreflightValidation(activityInput); - log.info('Preflight validation passed'); - - // === Playwright stealth config === - // Write the playwright-cli config before any browser session opens so the - // validator and downstream agents inherit anti-detection defaults. await preflightActs.syncPlaywrightStealthConfig(activityInput); - // === Authentication Validation === state.currentPhase = 'auth-validation'; state.currentAgent = 'validate-authentication'; const authMetrics = await authValidationActs.runAuthenticationValidation(activityInput); - // Null when no login ran (no-auth scan); left absent so status renders it skipped, not completed. - if (authMetrics) { - state.agentMetrics['validate-authentication'] = authMetrics; - } + if (authMetrics !== null) state.agentMetrics['validate-authentication'] = authMetrics; state.currentAgent = null; - log.info('Authentication validation passed'); - // === Initialize Deliverables Git === await a.initDeliverableGit(activityInput); - - // === Sync code_path deny rules === await a.syncCodePathDenyRules(activityInput); - log.info(`Run scope: vuln_classes=[${selectedVulnClasses.join(', ')}] exploit=${exploit}`); + const allExpectedDone = state.expectedAgents.every((agentName) => state.completedAgents.includes(agentName)); + // A durable draft means report.json is already committed, so re-running the pentest phase + // cannot change what the report says. It would only re-pay for the analysis and observe new + // degradation reasons that the finalized deliverable, rendered from durable state, could never + // carry — leaving the report claiming complete coverage while the session records a partial + // run. An invalid draft is rolled back to `pending` during resume, so it still re-runs here. + const reportAlreadyAuthored = reportIsAuthored(resumeState?.reportProgress?.stage); + if (!allExpectedDone && !reportAlreadyAuthored) { + const effectiveSarif = await runCapella(); + await runSequentialPhase('pre-recon', 'pre-recon', a.runPreReconAgent); + await runSequentialPhase('recon', 'recon', a.runReconAgent); - // === Phase 1: Pre-Reconnaissance === - await runSequentialPhase('pre-recon', 'pre-recon', a.runPreReconAgent); - - // === Phase 2: Reconnaissance === - await runSequentialPhase('recon', 'recon', a.runReconAgent); - - // === Phases 3-4: Vulnerability Analysis + Exploitation (Pipelined) === - // Each vuln type runs as an independent pipeline: - // vuln agent → queue check → conditional exploit agent - // Exploits start immediately when their vuln finishes, not waiting for all. - state.currentPhase = 'vulnerability-exploitation'; - state.currentAgent = 'pipelines'; - await a.logPhaseTransition(activityInput, 'vulnerability-exploitation', 'start'); - - // Closure over shouldSkip and activityInput by design (Temporal replay safety) - async function runVulnExploitPipeline( - vulnType: VulnType, - runVulnAgent: () => Promise, - runExploitAgent: () => Promise, - ): Promise { - const vulnAgentName = `${vulnType}-vuln`; - const exploitAgentName = `${vulnType}-exploit`; - - // A class failure must not reject the pipeline set — that would lose the class identity - // (results are completion-ordered) and force fail-hard. Catch here and return the error in - // the result's `error` field so aggregatePipelineResults can attribute it to `vulnType`. - try { - // 1. Run vulnerability analysis (or skip if resumed) - let vulnMetrics: AgentMetrics | null = null; - if (!shouldSkip(vulnAgentName)) { - vulnMetrics = await runVulnAgent(); - state.agentMetrics[vulnAgentName] = vulnMetrics; - state.completedAgents.push(vulnAgentName); - if (input.checkpointsEnabled) { - await a.saveCheckpoint(activityInput, vulnAgentName, 'vulnerability-analysis', state); - } - } else { - log.info(`Skipping ${vulnAgentName} (already complete)`); - state.completedAgents.push(vulnAgentName); - } - - // 1.5. Merge external findings from consumer provider into exploitation queue - await a.mergeFindingsIntoQueue(activityInput, vulnType); - - // 2. Check exploitation queue for actionable findings - const decision = await a.checkExploitationQueue(activityInput, vulnType); - - // 3. Previously-completed exploits are preserved regardless of mode; new exploits gated by mode. - let exploitMetrics: AgentMetrics | null = null; - if (shouldSkip(exploitAgentName)) { - log.info(`Skipping ${exploitAgentName} (already complete)`); - state.completedAgents.push(exploitAgentName); - } else if (decision.shouldExploit && exploit) { - exploitMetrics = await runExploitAgent(); - state.agentMetrics[exploitAgentName] = exploitMetrics; - state.completedAgents.push(exploitAgentName); - if (input.checkpointsEnabled) { - await a.saveCheckpoint(activityInput, exploitAgentName, 'exploitation', state); - } - } else { - // Exploitation did not run (exploit mode off, or no actionable findings) — still - // mark the agent complete so a resume does not treat it as unfinished work. - log.info( - `Marking ${exploitAgentName} complete (${decision.shouldExploit ? 'exploit mode disabled' : 'no actionable findings'})`, - ); - state.completedAgents.push(exploitAgentName); - if (input.checkpointsEnabled) { - await a.saveCheckpoint(activityInput, exploitAgentName, 'exploitation', state); - } - } - - return { - vulnType, - vulnMetrics, - exploitMetrics, - exploitDecision: { - shouldExploit: decision.shouldExploit, - vulnerabilityCount: decision.vulnerabilityCount, - }, - error: null, - }; - } catch (error) { - // Let cancellation propagate to the workflow-level handler. - if (isCancellation(error)) { - throw error; - } - const rawMessage = error instanceof Error ? error.message : String(error); - const message = truncatePipelineErrorMessage(rawMessage); - log.warn(`Pipeline ${vulnType} failed`, { error: message }); - return { - vulnType, - vulnMetrics: state.agentMetrics[vulnAgentName] ?? null, - exploitMetrics: state.agentMetrics[exploitAgentName] ?? null, - exploitDecision: null, - error: message, - }; + state.currentPhase = 'vulnerability-exploitation'; + state.currentAgent = 'pipelines'; + await a.logPhaseTransition(activityInput, 'vulnerability-exploitation', 'start'); + const pipelineThunks = buildPipelineConfigs().map( + (config) => () => runVulnExploitPipeline(config.vulnType, config.runVuln, config.runExploit, effectiveSarif), + ); + const pipelineResults = await runWithConcurrencyLimit(pipelineThunks, MAX_CONCURRENT_PIPELINES); + aggregatePipelineResults(pipelineResults); + if (state.failedPipelines.length > 0) { + activityInput.failedClasses = state.failedPipelines.map((failure) => failure.vulnType); } + await a.logPhaseTransition(activityInput, 'vulnerability-exploitation', 'complete'); + if (effectiveSarif !== undefined) await runMiscellaneousPipeline(effectiveSarif); } - const pipelineConfigs = buildPipelineConfigs(); - const pipelineThunks: Array<() => Promise> = []; - let alreadyCompletedPipelineCount = 0; + await finalizeReportPipeline(); - for (const config of pipelineConfigs) { - // Excluded classes drop entirely; any prior deliverables stay on disk but don't count this run. - if (!selectedClassSet.has(config.vulnType)) { - log.info(`Skipping ${config.vulnType} pipeline (class not selected this run)`); - continue; - } - if (!shouldSkip(config.vulnAgent) || !shouldSkip(config.exploitAgent)) { - pipelineThunks.push(() => runVulnExploitPipeline(config.vulnType, config.runVuln, config.runExploit)); - } else { - log.info(`Skipping entire ${config.vulnType} pipeline (both agents complete)`); - state.completedAgents.push(config.vulnAgent, config.exploitAgent); - alreadyCompletedPipelineCount++; - } - } - - const pipelineResults = await runWithConcurrencyLimit(pipelineThunks, MAX_CONCURRENT_PIPELINES); - aggregatePipelineResults(pipelineResults, alreadyCompletedPipelineCount); - - // Surface the not-assessed classes to the report stage so a failed class renders as - // "analysis did not complete" rather than the absence assertion "no findings". - if (state.failedPipelines.length > 0) { - activityInput.failedClasses = state.failedPipelines.map((f) => f.vulnType); - } - - state.currentPhase = 'exploitation'; - state.currentAgent = null; - await a.logPhaseTransition(activityInput, 'vulnerability-exploitation', 'complete'); - - // === Phase 5: Reporting === - if (!shouldSkip('report')) { - state.currentPhase = 'reporting'; - state.currentAgent = 'report'; - await a.logPhaseTransition(activityInput, 'reporting', 'start'); - - // First, assemble the concatenated report from per-class deliverables - await a.assembleReportActivity(activityInput, exploit); - - // Then run the report agent to add executive summary and clean up - state.agentMetrics.report = await a.runReportAgent(activityInput, exploit); - state.completedAgents.push('report'); - if (input.checkpointsEnabled) { - await a.saveCheckpoint(activityInput, 'report', 'reporting', state); - } - - // Inject model metadata into the final report - await a.injectReportMetadataActivity(activityInput); - - await a.logPhaseTransition(activityInput, 'reporting', 'complete'); - } else { - log.info('Skipping report (already complete)'); - state.completedAgents.push('report'); - } - - // Runs after the skip gate so consumer providers still execute on resume. - await a.generateReportOutputActivity(activityInput); - - if (input.checkpointsEnabled) { - await a.saveCheckpoint(activityInput, 'report-output', 'reporting', state); - } - - // Preserve a partial verdict (set by aggregatePipelineResults) — a clean run is 'completed', - // a run where some classes were not assessed is 'partial'. - const terminalStatus: 'completed' | 'partial' = state.failedPipelines.length > 0 ? 'partial' : 'completed'; + // One terminal contract everywhere: reaching this point proved the canonical report + // (a failed proof throws), so the durable reason set alone decides completed vs partial. + // PDF and customer-copy warnings never create reasons and never change the status. + const terminalStatus: 'completed' | 'partial' = partialReasons.length > 0 ? 'partial' : 'completed'; state.status = terminalStatus; state.currentPhase = null; state.currentAgent = null; - state.summary = computeSummary(state); - - // Log workflow completion summary + state.summary = computeSummary(state, usageAccountingComplete()); await a.logWorkflowComplete(activityInput, toWorkflowSummary(state, terminalStatus)); - return state; } catch (error) { - // Cancellation: return structured state instead of throwing - if (isCancellation(error)) { + if (hasCancellationInCauseChain(error)) { state.status = 'cancelled'; state.error = `Cancelled during phase: ${state.currentPhase ?? 'unknown'}`; - state.summary = computeSummary(state); - // Finalization runs I/O activities; shield them from the cancellation so the - // cancelled state is still logged rather than aborted mid-write. + state.summary = computeSummary(state, usageAccountingComplete()); await CancellationScope.nonCancellable(async () => { try { await a.logWorkflowComplete(activityInput, toWorkflowSummary(state, 'cancelled')); @@ -703,12 +1201,8 @@ export async function pentestPipeline(input: PipelineInput): Promise { return pentestPipeline(input); } diff --git a/apps/worker/src/types/agents.ts b/apps/worker/src/types/agents.ts index 3384f74f..9044f16f 100644 --- a/apps/worker/src/types/agents.ts +++ b/apps/worker/src/types/agents.ts @@ -25,6 +25,7 @@ export const ALL_AGENTS = [ 'auth-exploit', 'ssrf-exploit', 'authz-exploit', + 'miscellaneous-exploit', 'report', ] as const; @@ -34,9 +35,10 @@ export const ALL_AGENTS = [ */ export type AgentName = (typeof ALL_AGENTS)[number]; -export type PlaywrightSession = 'agent1' | 'agent2' | 'agent3' | 'agent4' | 'agent5'; +export type PlaywrightSession = 'agent1' | 'agent2' | 'agent3' | 'agent4' | 'agent5' | 'agent6'; import type { ActivityLogger } from './activity-logger.js'; +import type { VulnClass } from './config.js'; export type AgentValidator = (sourceDir: string, logger: ActivityLogger) => Promise; @@ -53,7 +55,7 @@ export interface AgentDefinition { /** * Vulnerability types supported by the pipeline. */ -export type VulnType = 'injection' | 'xss' | 'auth' | 'ssrf' | 'authz'; +export type VulnType = VulnClass; /** * Decision returned by queue validation for exploitation phase. diff --git a/apps/worker/src/types/config.ts b/apps/worker/src/types/config.ts index 3fd50cc6..acbd7f46 100644 --- a/apps/worker/src/types/config.ts +++ b/apps/worker/src/types/config.ts @@ -67,11 +67,15 @@ export interface Authentication { success_condition: SuccessCondition; } +export interface AgenticSastConfig { + enabled: 'true' | 'false'; +} + export interface Config { rules?: Rules; authentication?: Authentication; description?: string; - vuln_classes?: VulnClass[]; + agentic_sast?: AgenticSastConfig; exploit?: 'true' | 'false'; report?: ReportConfig; rules_of_engagement?: string; @@ -85,7 +89,8 @@ export interface DistributedConfig { focus: Rule[]; authentication: Authentication | null; description: string; - vuln_classes: VulnClass[]; + /** Present only when Capella is enabled. */ + agenticSast?: true; exploit: boolean; report: DistributedReportConfig; rules_of_engagement: string; diff --git a/apps/worker/src/types/metrics.ts b/apps/worker/src/types/metrics.ts index e2feb029..2e445761 100644 --- a/apps/worker/src/types/metrics.ts +++ b/apps/worker/src/types/metrics.ts @@ -18,6 +18,8 @@ export interface AgentMetrics { costUsd: number | null; numTurns: number | null; model?: string | undefined; + /** Durable Git checkpoint associated with this result when one exists. */ + checkpoint?: string; // True when the checkpoint provider skipped the agent (resume path). // Callers that perform post-agent work on collected state should short-circuit // when this is set, since no fresh state was produced this run. diff --git a/apps/worker/src/types/run-state.ts b/apps/worker/src/types/run-state.ts new file mode 100644 index 00000000..089921bb --- /dev/null +++ b/apps/worker/src/types/run-state.ts @@ -0,0 +1,1104 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Durable single-scan execution and report-progress contracts. */ + +import type { AgenticSastReduction, CapellaFailurePoint } from '../ai/sast/types.js'; +import type { AgentName } from './agents.js'; +import { ALL_VULN_CLASSES, type VulnClass } from './config.js'; +import type { ReconciliationClass } from './reconciliation.js'; + +export const DURABLE_SCAN_STATE_SCHEMA_VERSION = 1 as const; + +export const FIXED_ANALYSIS_CLASSES: readonly VulnClass[] = Object.freeze([...ALL_VULN_CLASSES]); + +/** + * What became of the internal `miscellaneous` class on this scan. `exploitation_disabled` + * means the scan has `exploit: false`, checked before the queue is even inspected; + * `not_actionable` means exploitation is enabled but reconciliation grouped no findings into + * this class; `expected` means an exploit agent was admitted for this class and has not + * finished; `completed` means that agent has finished. + */ +export type MiscellaneousOutcome = 'not_actionable' | 'exploitation_disabled' | 'expected' | 'completed'; + +// === Partial-reason contract === + +/** Closed degradation codes, in the locked presentation order. */ +export const PARTIAL_REASON_CODES = Object.freeze([ + 'agentic_sast_failed', + 'agentic_sast_reduced', + 'class_pipeline_failed', + 'class_reconciliation_failed', + 'report_renumber_failed', + 'report_compaction_failed', + 'report_class_omitted', + 'report_sarif_failed', +] as const); + +export type PartialReasonCode = (typeof PARTIAL_REASON_CODES)[number]; + +export const AGENTIC_SAST_REDUCTION_REASONS = Object.freeze([ + 'invalid_architecture_items', + 'invalid_investigations', + 'incomplete_research', + 'incomplete_dedupe', + 'incomplete_review', + 'incomplete_critic', + 'incomplete_confirm', + 'incomplete_calibrate', + 'failed_stage_fallback', + 'malformed_findings', +] as const); +export type AgenticSastReductionReason = (typeof AGENTIC_SAST_REDUCTION_REASONS)[number]; + +export const AGENTIC_SAST_OMISSION_REASONS = Object.freeze([ + 'invalid_finding_record', + 'missing_code_path', + 'invalid_code_path', +] as const); +export type AgenticSastOmissionReason = (typeof AGENTIC_SAST_OMISSION_REASONS)[number]; + +export interface AgenticSastOmission { + readonly findingId?: string; + readonly displayName?: string; + readonly reason: AgenticSastOmissionReason; +} + +/** Codes whose durable identity carries a vulnerability-class context. */ +const CLASS_CONTEXT_CODES: ReadonlySet = new Set([ + 'class_pipeline_failed', + 'class_reconciliation_failed', + 'report_renumber_failed', + 'report_class_omitted', +]); + +/** + * Accepted `agentic_sast_failed` stage contexts. Mirrors `CapellaFailurePoint`; the + * `satisfies` clause plus the exhaustiveness check below keep the two in sync at compile time. + */ +export const ACCEPTED_CAPELLA_FAILURE_STAGES = Object.freeze([ + 'architecture', + 'threat-model', + 'plan', + 'research', + 'dedupe', + 'review', + 'critic', + 'confirm', + 'calibrate', + 'export', + 'workflow', +] as const satisfies readonly CapellaFailurePoint[]); + +type UnlistedCapellaStage = Exclude; +const _everyCapellaStageIsListed: UnlistedCapellaStage extends never ? true : never = true; +void _everyCapellaStageIsListed; + +/** The fixed context order used after code order: the five analysis classes, then `miscellaneous`. */ +const PARTIAL_REASON_CLASS_ORDER: readonly ReconciliationClass[] = Object.freeze([ + ...ALL_VULN_CLASSES, + 'miscellaneous', +]); + +/** + * One durable degradation record. The code plus its bounded context is the identity; + * safe messages are derived for display and never participate in equality or resume. + */ +export interface PartialReason { + readonly code: PartialReasonCode; + readonly vulnerabilityClass?: ReconciliationClass; + readonly stage?: CapellaFailurePoint; + readonly reductionReason?: AgenticSastReductionReason; + readonly omittedCount?: number; + readonly consideredCount?: number; + readonly omissions?: readonly AgenticSastOmission[]; + readonly classifiedCount?: number; + readonly affectedBatchCount?: number; + readonly entityCount?: number; + readonly omittedEntityCount?: number; + readonly dependencyCount?: number; + readonly omittedDependencyCount?: number; + readonly usableCount?: number; + readonly triageConsideredCount?: number; + readonly triageClassifiedCount?: number; + readonly triageOmittedCount?: number; + readonly affectedTriageBatchCount?: number; + readonly auditUnitCount?: number; + readonly salvagedAuditSessionCount?: number; + readonly survivorCount?: number; + readonly unreadableCount?: number; + readonly salvagedTurnLimitCount?: number; + readonly gradedCount?: number; + readonly missingCount?: number; + readonly rejectedUnexpectedCount?: number; + readonly rejectedDuplicateCount?: number; + readonly quarantinedCount?: number; + readonly fallbackFindingCount?: number; +} + +/** Derived presentation of one durable reason for status output. */ +export interface PartialReasonView extends PartialReason { + readonly message: string; +} + +// === Display projection === + +/** + * Reader-facing name of every accepted vulnerability class, in the capitalization an + * accepted sentence starts with. The mid-sentence form is this name lowercased. + * Durable records, JSON fields, and every stable machine value keep the slug instead. + */ +const CLASS_DISPLAY_NAMES: Readonly> = Object.freeze({ + injection: 'Injection', + xss: 'Cross-Site Scripting', + auth: 'Authentication', + authz: 'Authorization', + ssrf: 'Server-Side Request Forgery', + miscellaneous: 'Miscellaneous', +}); + +/** Reader-facing name of every accepted Agentic SAST stage, written to read mid-sentence. */ +const CAPELLA_STAGE_DISPLAY_NAMES: Readonly> = Object.freeze({ + architecture: 'architecture mapping', + 'threat-model': 'threat modelling', + plan: 'review planning', + research: 'deep code research', + dedupe: 'duplicate merging', + review: 'independent review', + critic: 'viability critique', + confirm: 'static confirmation', + calibrate: 'risk calibration', + export: 'findings export', + workflow: 'orchestration', +}); + +/** Sentence-start display name for an accepted class slug, e.g. `xss` to `Cross-Site Scripting`. */ +export function classDisplayName(vulnerabilityClass: ReconciliationClass): string { + const name = CLASS_DISPLAY_NAMES[vulnerabilityClass]; + // The accepted slugs are a closed set that durable validation already enforced, so an + // unmapped value is corruption. Fail closed rather than render a stored value verbatim. + if (name === undefined) throw new RunStateError('CorruptedSessionError', 'class-display-name-unmapped'); + return name; +} + +/** Display name for an accepted Agentic SAST stage slug, e.g. `threat-model` to `threat modelling`. */ +export function capellaStageDisplayName(stage: CapellaFailurePoint): string { + const name = CAPELLA_STAGE_DISPLAY_NAMES[stage]; + if (name === undefined) throw new RunStateError('CorruptedSessionError', 'stage-display-name-unmapped'); + return name; +} + +/** Any template token that survives substitution, so a missing context can never ship. */ +const SAFE_MESSAGE_TOKEN_PATTERN = /\{(?:Class|class|stage)\}/; + +/** The bounded context a safe-message template may interpolate. */ +export interface SafeMessageContext { + readonly vulnerabilityClass?: ReconciliationClass; + readonly stage?: CapellaFailurePoint; +} + +/** + * Substitute display names into one safe-message template. `{Class}` starts a sentence, + * `{class}` sits mid-sentence, and `{stage}` names an Agentic SAST stage. A template whose + * context is missing leaves a placeholder behind, which fails closed instead of shipping it. + */ +export function renderSafeMessage(template: string, context: SafeMessageContext): string { + let rendered = template; + if (context.vulnerabilityClass !== undefined) { + const displayName = classDisplayName(context.vulnerabilityClass); + rendered = rendered.replaceAll('{Class}', displayName).replaceAll('{class}', displayName.toLowerCase()); + } + if (context.stage !== undefined) { + rendered = rendered.replaceAll('{stage}', capellaStageDisplayName(context.stage)); + } + if (SAFE_MESSAGE_TOKEN_PATTERN.test(rendered)) { + throw new RunStateError('CorruptedSessionError', 'safe-message-context-missing'); + } + return rendered; +} + +/** The one safe-message map; every human or JSON surface renders reasons through it. */ +export const PARTIAL_REASON_SAFE_MESSAGES: Readonly> = Object.freeze({ + agentic_sast_failed: 'Agentic SAST failed, so the pentest continued without its findings.', + agentic_sast_reduced: + "Agentic SAST left some findings out of the pentest because they did not match Shannon's required finding format. This workspace does not contain the exact count.", + class_pipeline_failed: + '{Class} could not be fully assessed. The other classes completed. Re-running this workspace retries only the part that failed.', + class_reconciliation_failed: + '{Class} findings could not be grouped into test cases, so that class was not exploited. Its analysis results are still in the report.', + report_renumber_failed: + '{Class} findings kept their working reference numbers, so numbering in the report may have gaps. The findings themselves are complete.', + report_compaction_failed: + 'Finding reference numbers in the report may have gaps. Every finding is present; only the numbering is affected.', + report_class_omitted: '{Class} was assessed but could not be included in the final report.', + report_sarif_failed: 'Report SARIF could not be generated. JSON and Markdown remain available.', +}); + +/** Used in place of the stageless `agentic_sast_failed` message once a stage is recorded. */ +const AGENTIC_SAST_FAILED_WITH_STAGE = + 'Agentic SAST failed during {stage}, so the pentest continued without its findings.'; + +/** Validate one closed reason record: known code, exact keys, and code-appropriate context. */ +export function isPartialReason(value: unknown): value is PartialReason { + if (!isRecord(value)) return false; + const code = value.code; + if (typeof code !== 'string' || !(PARTIAL_REASON_CODES as readonly string[]).includes(code)) return false; + const reasonCode = code as PartialReasonCode; + + if (CLASS_CONTEXT_CODES.has(reasonCode)) { + return ( + hasExactKeys(value, ['code', 'vulnerabilityClass']) && + PARTIAL_REASON_CLASS_ORDER.includes(value.vulnerabilityClass as ReconciliationClass) + ); + } + if (reasonCode === 'agentic_sast_failed') { + return ( + hasExactKeys(value, ['code'], ['stage']) && + (value.stage === undefined || + (ACCEPTED_CAPELLA_FAILURE_STAGES as readonly string[]).includes(value.stage as string)) + ); + } + if (reasonCode === 'agentic_sast_reduced') { + if (hasExactKeys(value, ['code'])) return true; + if (value.reductionReason === 'failed_stage_fallback') { + return ( + hasExactKeys(value, ['code', 'stage', 'reductionReason', 'fallbackFindingCount']) && + value.stage !== 'export' && + value.stage !== 'workflow' && + (ACCEPTED_CAPELLA_FAILURE_STAGES as readonly unknown[]).includes(value.stage) && + isBoundedPartialReasonCount(value.fallbackFindingCount) + ); + } + if (value.stage === 'architecture') { + return ( + hasExactKeys(value, [ + 'code', + 'stage', + 'reductionReason', + 'entityCount', + 'omittedEntityCount', + 'dependencyCount', + 'omittedDependencyCount', + ]) && + value.reductionReason === 'invalid_architecture_items' && + countsAreBounded(value, ['entityCount', 'omittedEntityCount', 'dependencyCount', 'omittedDependencyCount']) && + Number(value.omittedEntityCount) + Number(value.omittedDependencyCount) >= 1 && + Number(value.omittedEntityCount) <= Number(value.entityCount) && + Number(value.omittedDependencyCount) <= Number(value.dependencyCount) + ); + } + if (value.stage === 'plan') { + return ( + hasExactKeys(value, ['code', 'stage', 'reductionReason', 'consideredCount', 'usableCount', 'omittedCount']) && + value.reductionReason === 'invalid_investigations' && + countsAreBounded(value, ['consideredCount', 'usableCount', 'omittedCount']) && + Number(value.omittedCount) >= 1 && + Number(value.usableCount) + Number(value.omittedCount) === Number(value.consideredCount) + ); + } + if (value.stage === 'export') { + return ( + hasExactKeys(value, ['code', 'stage', 'reductionReason', 'omittedCount', 'consideredCount', 'omissions']) && + value.reductionReason === 'malformed_findings' && + isBoundedPartialReasonCount(value.omittedCount) && + isBoundedPartialReasonCount(value.consideredCount) && + value.omittedCount >= 1 && + value.omittedCount <= value.consideredCount && + Array.isArray(value.omissions) && + value.omissions.length === value.omittedCount && + value.omissions.every(isAgenticSastOmission) + ); + } + if (value.stage === 'research') { + return ( + hasExactKeys(value, [ + 'code', + 'stage', + 'reductionReason', + 'triageConsideredCount', + 'triageClassifiedCount', + 'triageOmittedCount', + 'affectedTriageBatchCount', + 'auditUnitCount', + 'salvagedAuditSessionCount', + ]) && + value.reductionReason === 'incomplete_research' && + countsAreBounded(value, [ + 'triageConsideredCount', + 'triageClassifiedCount', + 'triageOmittedCount', + 'affectedTriageBatchCount', + 'auditUnitCount', + 'salvagedAuditSessionCount', + ]) && + Number(value.triageClassifiedCount) + Number(value.triageOmittedCount) === + Number(value.triageConsideredCount) && + Number(value.triageOmittedCount) + Number(value.salvagedAuditSessionCount) >= 1 + ); + } + if (value.stage === 'dedupe') { + return ( + hasExactKeys(value, [ + 'code', + 'stage', + 'reductionReason', + 'consideredCount', + 'survivorCount', + 'unreadableCount', + 'salvagedTurnLimitCount', + ]) && + value.reductionReason === 'incomplete_dedupe' && + countsAreBounded(value, ['consideredCount', 'survivorCount', 'unreadableCount', 'salvagedTurnLimitCount']) && + Number(value.unreadableCount) + Number(value.salvagedTurnLimitCount) >= 1 && + Number(value.salvagedTurnLimitCount) <= 1 + ); + } + if (['review', 'critic', 'confirm', 'calibrate'].includes(String(value.stage))) { + const expectedReason = `incomplete_${String(value.stage)}`; + const countFields = [ + 'consideredCount', + 'gradedCount', + 'missingCount', + 'unreadableCount', + 'rejectedUnexpectedCount', + 'rejectedDuplicateCount', + 'salvagedTurnLimitCount', + ]; + const requiredKeys = ['code', 'stage', 'reductionReason', ...countFields]; + if ( + value.stage === 'review' && + hasExactKeys(value, [...requiredKeys, 'quarantinedCount']) && + isBoundedPartialReasonCount(value.quarantinedCount) && + Number(value.quarantinedCount) <= Number(value.missingCount) + ) { + return verdictReductionIsValid(value, expectedReason, countFields); + } + return hasExactKeys(value, requiredKeys) && verdictReductionIsValid(value, expectedReason, countFields); + } + return false; + } + return hasExactKeys(value, ['code']); +} + +function isBoundedPartialReasonCount(value: unknown): value is number { + return Number.isSafeInteger(value) && Number(value) >= 0 && Number(value) <= 1_000_000; +} + +function countsAreBounded(value: Record, fields: readonly string[]): boolean { + return fields.every((field) => isBoundedPartialReasonCount(value[field])); +} + +function verdictReductionIsValid( + value: Record, + expectedReason: string, + countFields: readonly string[], +): boolean { + return ( + value.reductionReason === expectedReason && + countsAreBounded(value, countFields) && + Number(value.missingCount) <= Number(value.consideredCount) && + Number(value.salvagedTurnLimitCount) <= 2 && + Number(value.missingCount) + Number(value.unreadableCount) + Number(value.salvagedTurnLimitCount) >= 1 + ); +} + +function isAgenticSastOmission(value: unknown): value is AgenticSastOmission { + if (!isRecord(value) || !(AGENTIC_SAST_OMISSION_REASONS as readonly unknown[]).includes(value.reason)) { + return false; + } + const allowedKeys = ['reason']; + if (value.findingId !== undefined) { + if (typeof value.findingId !== 'string' || !/^[a-z0-9-]{1,256}$/.test(value.findingId)) return false; + allowedKeys.push('findingId'); + } + if (value.displayName !== undefined) { + if (!isBoundedSafeText(value.displayName, 160)) return false; + allowedKeys.push('displayName'); + } + return hasExactKeys(value, allowedKeys); +} + +function isBoundedSafeText(value: unknown, maxLength: number): value is string { + return typeof value === 'string' && value.length > 0 && value.length <= maxLength && !containsControlCharacter(value); +} + +function containsControlCharacter(value: string): boolean { + return [...value].some((character) => { + const code = character.charCodeAt(0); + return code <= 31 || code === 127; + }); +} + +function partialReasonRank(reason: PartialReason): readonly [number, number, number] { + const codeIndex = PARTIAL_REASON_CODES.indexOf(reason.code); + const classIndex = + reason.vulnerabilityClass === undefined ? -1 : PARTIAL_REASON_CLASS_ORDER.indexOf(reason.vulnerabilityClass); + const stageIndex = + reason.stage === undefined ? -1 : (ACCEPTED_CAPELLA_FAILURE_STAGES as readonly string[]).indexOf(reason.stage); + return [codeIndex, classIndex, stageIndex]; +} + +/** Locked ordering: table code order, then fixed class order, then stage order. */ +export function comparePartialReasons(left: PartialReason, right: PartialReason): number { + const leftRank = partialReasonRank(left); + const rightRank = partialReasonRank(right); + for (let index = 0; index < leftRank.length; index++) { + const difference = (leftRank[index] ?? 0) - (rightRank[index] ?? 0); + if (difference !== 0) return difference; + } + return 0; +} + +function partialReasonKey(reason: PartialReason): string { + // One reduction per stage. Legacy code-only records carried no stage but described the only + // reduction that then existed (export), so they key as export — a detailed export reduction + // does not create a second reason on resume, while a research reduction stays distinct. + if (reason.code === 'agentic_sast_reduced') return `${reason.code}|${reason.stage ?? 'export'}`; + return `${reason.code}|${reason.vulnerabilityClass ?? ''}|${reason.stage ?? ''}`; +} + +function reasonsForClass(reasons: readonly PartialReason[], vulnerabilityClass: ReconciliationClass): PartialReason[] { + return reasons.filter((reason) => reason.vulnerabilityClass === vulnerabilityClass); +} + +/** + * Enforce the one-failure-one-explanation rules inside one ordered set: + * `class_reconciliation_failed` supersedes `class_pipeline_failed` for the same class, and + * `report_class_omitted` is valid only when the class has no upstream reason. + */ +function hasConflictingClassReasons(reasons: readonly PartialReason[]): boolean { + for (const vulnerabilityClass of PARTIAL_REASON_CLASS_ORDER) { + const classCodes = new Set(reasonsForClass(reasons, vulnerabilityClass).map((reason) => reason.code)); + if (classCodes.has('class_pipeline_failed') && classCodes.has('class_reconciliation_failed')) return true; + const hasUpstreamReason = + classCodes.has('class_pipeline_failed') || + classCodes.has('class_reconciliation_failed') || + classCodes.has('report_renumber_failed'); + if (classCodes.has('report_class_omitted') && hasUpstreamReason) return true; + } + return false; +} + +/** Validate a durable set: valid members, locked order, no duplicates, no conflicting pairs. */ +export function isOrderedPartialReasonSet(value: unknown): value is readonly PartialReason[] { + if (!Array.isArray(value) || !value.every(isPartialReason)) return false; + const keys = new Set(); + for (let index = 0; index < value.length; index++) { + const reason = value[index] as PartialReason; + const key = partialReasonKey(reason); + if (keys.has(key)) return false; + keys.add(key); + const previous = value[index - 1] as PartialReason | undefined; + if (previous !== undefined && comparePartialReasons(previous, reason) >= 0) return false; + } + return !hasConflictingClassReasons(value as readonly PartialReason[]); +} + +/** + * Append newly observed reasons to a durable set without removing or changing existing ones. + * Deduplicates on the durable identity, applies the class precedence rules, and returns the + * locked ordering. Invalid inputs and appends that would have to remove an existing reason + * fail closed. + */ +export function appendPartialReasons( + existing: readonly PartialReason[], + observed: readonly PartialReason[], +): readonly PartialReason[] { + if (!isOrderedPartialReasonSet(existing)) { + throw new RunStateError('CorruptedSessionError', 'partial-reasons-existing-invalid'); + } + if (!observed.every(isPartialReason)) { + throw new RunStateError('DurableStateConflictError', 'partial-reasons-observed-invalid'); + } + + const merged = new Map(); + for (const reason of existing) merged.set(partialReasonKey(reason), reason); + for (const reason of observed) { + const key = partialReasonKey(reason); + if (merged.has(key)) continue; + if (reason.code === 'class_pipeline_failed' || reason.code === 'report_class_omitted') { + const vulnerabilityClass = reason.vulnerabilityClass as ReconciliationClass; + const classCodes = new Set(reasonsForClass([...merged.values()], vulnerabilityClass).map((entry) => entry.code)); + if (reason.code === 'class_pipeline_failed' && classCodes.has('class_reconciliation_failed')) continue; + if ( + reason.code === 'report_class_omitted' && + (classCodes.has('class_pipeline_failed') || + classCodes.has('class_reconciliation_failed') || + classCodes.has('report_renumber_failed')) + ) { + continue; + } + } + merged.set(key, reason); + } + + const result = [...merged.values()].sort(comparePartialReasons); + if (!isOrderedPartialReasonSet(result)) { + throw new RunStateError('DurableStateConflictError', 'partial-reasons-append-conflict'); + } + for (const reason of existing) { + if (!merged.has(partialReasonKey(reason))) { + throw new RunStateError('DurableStateConflictError', 'partial-reasons-removed'); + } + } + return Object.freeze(result); +} + +/** Resolve one durable reason's rendered sentence, including its bounded class or stage context. */ +function safeMessageFor(reason: PartialReason): string { + if ( + reason.code === 'agentic_sast_reduced' && + reason.reductionReason === 'failed_stage_fallback' && + reason.stage !== undefined && + reason.fallbackFindingCount !== undefined + ) { + // The count is what the last verified artifact held, not what was delivered: export applies + // production-viability gating afterwards and can drop every one of them, so the sentence must + // never read as a delivery claim. + const stageName = capellaStageDisplayName(reason.stage); + if (reason.fallbackFindingCount === 0) { + return `Agentic SAST could not finish ${stageName}. No candidate findings were recovered from the last verified artifact, so static-analysis coverage was reduced.`; + } + const findingLabel = reason.fallbackFindingCount === 1 ? 'finding' : 'findings'; + return `Agentic SAST could not finish ${stageName}. It recovered ${String(reason.fallbackFindingCount)} candidate ${findingLabel} from the last verified artifact; later viability checks may exclude some or all of them from the exported results, so static-analysis coverage was reduced.`; + } + if ( + reason.code === 'agentic_sast_reduced' && + reason.stage === 'architecture' && + reason.omittedEntityCount !== undefined && + reason.omittedDependencyCount !== undefined + ) { + const omitted = reason.omittedEntityCount + reason.omittedDependencyCount; + return `Agentic SAST omitted ${String(omitted)} malformed architecture item${omitted === 1 ? '' : 's'} and continued with reduced static-analysis coverage.`; + } + if ( + reason.code === 'agentic_sast_reduced' && + reason.stage === 'plan' && + reason.consideredCount !== undefined && + reason.omittedCount !== undefined + ) { + return `Agentic SAST kept ${String(reason.consideredCount - reason.omittedCount)} of ${String(reason.consideredCount)} planned investigations and continued with reduced static-analysis coverage.`; + } + if ( + reason.code === 'agentic_sast_reduced' && + reason.stage === 'research' && + reason.reductionReason === 'incomplete_research' && + reason.triageConsideredCount !== undefined && + reason.triageOmittedCount !== undefined && + reason.salvagedAuditSessionCount !== undefined + ) { + return renderIncompleteResearchReduction( + reason.triageConsideredCount, + reason.triageOmittedCount, + reason.salvagedAuditSessionCount, + ); + } + if ( + reason.code === 'agentic_sast_reduced' && + reason.stage === 'dedupe' && + reason.unreadableCount !== undefined && + reason.salvagedTurnLimitCount !== undefined + ) { + if (reason.unreadableCount === 0) { + return 'Agentic SAST preserved accepted duplicate decisions after the session reached its turn limit and continued with reduced static-analysis coverage.'; + } + return `Agentic SAST completed duplicate merging with ${String(reason.unreadableCount)} unreadable finding file${reason.unreadableCount === 1 ? '' : 's'} and reduced static-analysis coverage.`; + } + if ( + reason.code === 'agentic_sast_reduced' && + ['review', 'critic', 'confirm', 'calibrate'].includes(String(reason.stage)) && + reason.stage !== undefined && + reason.consideredCount !== undefined && + reason.gradedCount !== undefined && + reason.salvagedTurnLimitCount !== undefined + ) { + if (reason.gradedCount === reason.consideredCount && reason.salvagedTurnLimitCount > 0) { + return `Agentic SAST preserved accepted decisions after ${capellaStageDisplayName(reason.stage)} reached its turn limit and continued with reduced static-analysis coverage.`; + } + return `Agentic SAST graded ${String(reason.gradedCount)} of ${String(reason.consideredCount)} findings during ${capellaStageDisplayName(reason.stage)} and continued with reduced static-analysis coverage.`; + } + if ( + reason.code === 'agentic_sast_reduced' && + reason.stage === 'export' && + reason.reductionReason === 'malformed_findings' && + reason.omittedCount !== undefined && + reason.consideredCount !== undefined && + reason.omissions !== undefined + ) { + return renderAgenticSastReduction(reason.consideredCount, reason.omissions); + } + const template = + reason.code === 'agentic_sast_failed' && reason.stage !== undefined + ? AGENTIC_SAST_FAILED_WITH_STAGE + : PARTIAL_REASON_SAFE_MESSAGES[reason.code]; + return renderSafeMessage(template, reason); +} + +/** Deterministic reduced-coverage sentence for the aggregate research reduction. */ +function renderIncompleteResearchReduction( + consideredCount: number, + omittedCount: number, + salvagedAuditSessionCount: number, +): string { + const fileLabel = consideredCount === 1 ? 'file' : 'files'; + const triageClause = + omittedCount === 0 + ? 'Every assigned file was classified.' + : `${omittedCount === 1 ? 'One was' : `${String(omittedCount)} were`} not classified.`; + const salvagedClause = + salvagedAuditSessionCount === 0 + ? '' + : ` ${String(salvagedAuditSessionCount)} deep-audit session${salvagedAuditSessionCount === 1 ? '' : 's'} preserved accepted work after reaching the turn limit.`; + return `Agentic SAST reviewed ${String(consideredCount)} planned ${fileLabel} during research. ${triageClause}${salvagedClause} The scan continued with reduced static-analysis coverage.`; +} + +/** Build the durable partial reason for one Capella reduction. Export keeps bounded omission detail. */ +export function partialReasonFromReduction(reduction: AgenticSastReduction): PartialReason { + const { reason, ...details } = reduction; + return { + code: 'agentic_sast_reduced', + ...details, + reductionReason: reason, + }; +} + +function renderAgenticSastReduction(consideredCount: number, omissions: readonly AgenticSastOmission[]): string { + const findingLabel = consideredCount === 1 ? 'finding' : 'findings'; + if (omissions.length === 1) { + const omission = omissions[0]; + if (omission === undefined) return PARTIAL_REASON_SAFE_MESSAGES.agentic_sast_reduced; + const name = omission.displayName === undefined ? '.' : `: ${omission.displayName}.`; + return `Agentic SAST reviewed ${String(consideredCount)} ${findingLabel}. One was left out during export because ${renderSingleOmissionReason(omission.reason)}${name}`; + } + + const reasonCounts = new Map(); + for (const omission of omissions) { + reasonCounts.set(omission.reason, (reasonCounts.get(omission.reason) ?? 0) + 1); + } + + const reasonText = renderOmissionReasonCounts(reasonCounts); + const names = omissions.flatMap((omission) => (omission.displayName === undefined ? [] : [omission.displayName])); + const displayedNames = names.slice(0, 3); + const remainingNameCount = names.length - displayedNames.length; + const namedSuffix = + displayedNames.length === 0 + ? '' + : ` Omitted ${displayedNames.length === 1 ? 'finding' : 'findings'}: ${displayedNames.join('; ')}${remainingNameCount > 0 ? `; and ${String(remainingNameCount)} more` : ''}.`; + + return `Agentic SAST reviewed ${String(consideredCount)} ${findingLabel}. ${String(omissions.length)} were left out during export. ${reasonText}.${namedSuffix}`; +} + +function renderSingleOmissionReason(reason: AgenticSastOmissionReason): string { + const messages: Readonly> = { + invalid_finding_record: 'its finding record was invalid', + missing_code_path: 'it did not include a code location', + invalid_code_path: 'its code location was invalid', + }; + return messages[reason]; +} + +function renderOmissionReasonCounts(counts: ReadonlyMap): string { + const clauses: string[] = []; + const labels: Readonly> = { + invalid_finding_record: ['had an invalid finding record', 'had invalid finding records'], + missing_code_path: ['did not include a code location', 'did not include code locations'], + invalid_code_path: ['had an invalid code location', 'had invalid code locations'], + }; + for (const reason of AGENTIC_SAST_OMISSION_REASONS) { + const count = counts.get(reason) ?? 0; + if (count === 0) continue; + clauses.push(`${String(count)} ${labels[reason][count === 1 ? 0 : 1]}`); + } + if (clauses.length <= 1) return clauses[0] ?? 'their finding records were invalid'; + return `${clauses.slice(0, -1).join(', ')} and ${clauses.at(-1)}`; +} + +/** Project durable reasons into display records using the one safe-message map. */ +export function projectPartialReasons(reasons: readonly PartialReason[]): readonly PartialReasonView[] { + return reasons.map((reason) => ({ ...reason, message: safeMessageFor(reason) })); +} + +// === Report progress === + +export type ReportSarifDisposition = 'committed' | 'absent' | 'render_failed'; + +const REPORT_SARIF_DISPOSITIONS: readonly ReportSarifDisposition[] = Object.freeze([ + 'committed', + 'absent', + 'render_failed', +]); + +/** + * Durable record of the last verified PDF publication. Structurally identical to the + * renderer's `PdfProvenance`; kept dependency-free here so durable-state validation + * never imports service code. + */ +export interface StoredPdfProvenance { + readonly pdf_sha256: string; + readonly canonical_report_sha256: string; + readonly renderer_version: string; + readonly template_version: string; +} + +/** Validate the closed replaceable provenance record stored beside the finalized report. */ +export function isStoredPdfProvenance(value: unknown): value is StoredPdfProvenance { + if (!isRecord(value)) return false; + return ( + hasExactKeys(value, ['pdf_sha256', 'canonical_report_sha256', 'renderer_version', 'template_version']) && + isSha256(value.pdf_sha256) && + isSha256(value.canonical_report_sha256) && + typeof value.renderer_version === 'string' && + value.renderer_version.length > 0 && + typeof value.template_version === 'string' && + value.template_version.length > 0 + ); +} + +export type ReportProgress = + | { + readonly stage: 'pending'; + readonly renumber_failed_classes: readonly ReconciliationClass[]; + readonly partial_reasons: readonly PartialReason[]; + } + | { + readonly stage: 'draft'; + readonly renumber_failed_classes: readonly ReconciliationClass[]; + readonly partial_reasons: readonly PartialReason[]; + readonly model_checkpoint: string; + readonly canonical_checkpoint?: string; + } + | { + readonly stage: 'finalized'; + readonly renumber_failed_classes: readonly ReconciliationClass[]; + readonly partial_reasons: readonly PartialReason[]; + readonly model_checkpoint: string; + readonly canonical_checkpoint: string; + readonly final_checkpoint: string; + readonly finalization_manifest_sha256: string; + readonly sarif_disposition: ReportSarifDisposition; + /** Replaceable after finalization; excluded from the match-or-conflict comparison. */ + readonly pdf_provenance?: StoredPdfProvenance; + }; + +/** + * True once the report agent has committed a draft. From that point report.json is fixed at the + * model checkpoint, so nothing the pentest phase could still produce can reach the deliverable: + * a resumed run that re-ran that phase would only spend money and observe degradation reasons the + * committed draft can never carry. + */ +export function reportIsAuthored(stage: ReportProgress['stage'] | undefined): boolean { + return stage === 'draft' || stage === 'finalized'; +} + +export interface DurableScanState { + readonly schema_version: typeof DURABLE_SCAN_STATE_SCHEMA_VERSION; + readonly exploit: boolean; + readonly participating_classes: readonly ReconciliationClass[]; + readonly expected_agents: readonly AgentName[]; + readonly miscellaneous_outcome?: MiscellaneousOutcome; + readonly report?: ReportProgress; +} + +export type RunStateFailureType = 'IncompatibleWorkspaceError' | 'CorruptedSessionError' | 'DurableStateConflictError'; + +/** One wording per durable-state refusal, shared by every layer that can refuse first. */ +export const SAFE_RUN_STATE_MESSAGES: Readonly> = Object.freeze({ + IncompatibleWorkspaceError: 'This workspace was created by a different version of Shannon and cannot be resumed.', + CorruptedSessionError: "This workspace's scan state is missing or damaged, so it cannot be resumed.", + DurableStateConflictError: + "This workspace's saved progress does not match what the scan is trying to record. Start a new scan with a different -w name.", +}); + +/** + * Resume refusal for a changed `exploit` setting. The sentence names the value the workspace + * actually stores, which is one of two literals — never a value read back from free text. + */ +export function workspaceExploitMismatchMessage(storedExploit: boolean): string { + const storedValue = storedExploit ? 'true' : 'false'; + return `This workspace was created with exploit set to "${storedValue}". A resume must use the same setting. Change the config back, or start a new scan with a different -w name.`; +} + +export class RunStateError extends Error { + readonly failureType: RunStateFailureType; + readonly checkCode: string; + + constructor(failureType: RunStateFailureType, checkCode: string) { + super(SAFE_RUN_STATE_MESSAGES[failureType]); + this.name = 'RunStateError'; + this.failureType = failureType; + this.checkCode = checkCode; + } +} + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === 'object' && !Array.isArray(value); +} + +function hasExactKeys( + value: Record, + required: readonly string[], + optional: readonly string[] = [], +): boolean { + const keys = Object.keys(value).sort(); + const allowed = new Set([...required, ...optional]); + return required.every((key) => Object.hasOwn(value, key)) && keys.every((key) => allowed.has(key)); +} + +function isCommitHash(value: unknown): value is string { + return typeof value === 'string' && /^[0-9a-f]{40}(?:[0-9a-f]{24})?$/.test(value); +} + +function isSha256(value: unknown): value is string { + return typeof value === 'string' && /^[0-9a-f]{64}$/.test(value); +} + +function arraysEqual(left: readonly T[], right: readonly T[]): boolean { + return left.length === right.length && left.every((value, index) => value === right[index]); +} + +function isFixedAnalysisScope(value: unknown): value is readonly VulnClass[] { + return Array.isArray(value) && arraysEqual(value, FIXED_ANALYSIS_CLASSES); +} + +function isParticipatingClassSet(value: unknown): value is readonly ReconciliationClass[] { + if (!Array.isArray(value)) return false; + if (arraysEqual(value, FIXED_ANALYSIS_CLASSES)) return true; + return arraysEqual(value, [...FIXED_ANALYSIS_CLASSES, 'miscellaneous']); +} + +function expectedAgentsFor(exploit: boolean): AgentName[] { + const expected: AgentName[] = ['pre-recon', 'recon']; + for (const vulnerabilityClass of FIXED_ANALYSIS_CLASSES) { + expected.push(`${vulnerabilityClass}-vuln` as AgentName); + } + if (exploit) { + for (const vulnerabilityClass of FIXED_ANALYSIS_CLASSES) { + expected.push(`${vulnerabilityClass}-exploit` as AgentName); + } + } + expected.push('report'); + return expected; +} + +function isExpectedAgentSet(value: unknown, exploit: boolean): value is readonly AgentName[] { + if (!Array.isArray(value) || value.some((agent) => typeof agent !== 'string')) return false; + const initial = expectedAgentsFor(exploit); + return arraysEqual(value, initial) || arraysEqual(value, [...initial, 'miscellaneous-exploit']); +} + +function isOrderedClassSubset( + value: unknown, + participatingClasses: readonly ReconciliationClass[], +): value is readonly ReconciliationClass[] { + if (!Array.isArray(value)) return false; + let previousIndex = -1; + for (const vulnerabilityClass of value) { + const currentIndex = participatingClasses.indexOf(vulnerabilityClass as ReconciliationClass); + if (currentIndex <= previousIndex) return false; + previousIndex = currentIndex; + } + return true; +} + +/** + * Every renumber-failed class must also carry its durable `report_renumber_failed` reason, + * so the failed-class set and the reason set cannot silently disagree. + */ +function renumberFailuresHaveReasons( + failedClasses: readonly ReconciliationClass[], + reasons: readonly PartialReason[], +): boolean { + return failedClasses.every((vulnerabilityClass) => + reasons.some( + (reason) => reason.code === 'report_renumber_failed' && reason.vulnerabilityClass === vulnerabilityClass, + ), + ); +} + +/** Validate the closed pending/draft/finalized report state. */ +export function isReportProgress( + value: unknown, + participatingClasses: readonly ReconciliationClass[], +): value is ReportProgress { + if (!isRecord(value) || typeof value.stage !== 'string') return false; + if (!isOrderedClassSubset(value.renumber_failed_classes, participatingClasses)) return false; + if (!isOrderedPartialReasonSet(value.partial_reasons)) return false; + if (!renumberFailuresHaveReasons(value.renumber_failed_classes, value.partial_reasons)) return false; + + if (value.stage === 'pending') { + return hasExactKeys(value, ['stage', 'renumber_failed_classes', 'partial_reasons']); + } + if (value.stage === 'draft') { + return ( + hasExactKeys( + value, + ['stage', 'renumber_failed_classes', 'partial_reasons', 'model_checkpoint'], + ['canonical_checkpoint'], + ) && + isCommitHash(value.model_checkpoint) && + (value.canonical_checkpoint === undefined || isCommitHash(value.canonical_checkpoint)) + ); + } + if (value.stage === 'finalized') { + const sarifDisposition = value.sarif_disposition; + const sarifReasonRecorded = value.partial_reasons.some((reason) => reason.code === 'report_sarif_failed'); + return ( + hasExactKeys( + value, + [ + 'stage', + 'renumber_failed_classes', + 'partial_reasons', + 'model_checkpoint', + 'canonical_checkpoint', + 'final_checkpoint', + 'finalization_manifest_sha256', + 'sarif_disposition', + ], + ['pdf_provenance'], + ) && + isCommitHash(value.model_checkpoint) && + isCommitHash(value.canonical_checkpoint) && + isCommitHash(value.final_checkpoint) && + isSha256(value.finalization_manifest_sha256) && + (REPORT_SARIF_DISPOSITIONS as readonly unknown[]).includes(sarifDisposition) && + // The render_failed disposition and the report_sarif_failed reason are two sides of one + // fact, so a finalized record with one but not the other is malformed. committed and + // absent never carry the reason. + sarifReasonRecorded === (sarifDisposition === 'render_failed') && + (value.pdf_provenance === undefined || isStoredPdfProvenance(value.pdf_provenance)) + ); + } + return false; +} + +/** Validate the closed schema-1 scan state and every cross-field invariant. */ +export function isDurableScanState(value: unknown): value is DurableScanState { + if (!isRecord(value)) return false; + if ( + !hasExactKeys( + value, + ['schema_version', 'exploit', 'participating_classes', 'expected_agents'], + ['miscellaneous_outcome', 'report'], + ) || + value.schema_version !== DURABLE_SCAN_STATE_SCHEMA_VERSION || + typeof value.exploit !== 'boolean' || + !isParticipatingClassSet(value.participating_classes) || + !isExpectedAgentSet(value.expected_agents, value.exploit) + ) { + return false; + } + + // The miscellaneous class's participation, its recorded outcome, and its exploit agent's + // presence in expected_agents are three separate fields that must always imply each other. + // A hand-edited or corrupted state file could set them inconsistently, so every direction of + // that implication is checked explicitly below rather than trusting one field to infer another. + const hasMiscellaneousClass = value.participating_classes.includes('miscellaneous'); + const hasMiscellaneousAgent = value.expected_agents.includes('miscellaneous-exploit'); + const outcome = value.miscellaneous_outcome; + if ( + typeof outcome === 'string' && + !['not_actionable', 'exploitation_disabled', 'expected', 'completed'].includes(outcome) + ) { + return false; + } + if (outcome !== undefined && typeof outcome !== 'string') return false; + // An outcome can only exist once the class has been admitted, and an admitted class must + // eventually record one. + if (outcome !== undefined && !hasMiscellaneousClass) return false; + if (hasMiscellaneousClass && outcome === undefined) return false; + // Only the two outcomes that follow from running the exploit agent may coexist with exploit + // being enabled and the agent being expected. + if ((outcome === 'expected' || outcome === 'completed') && (!value.exploit || !hasMiscellaneousAgent)) return false; + if ((outcome === 'not_actionable' || outcome === 'exploitation_disabled') && hasMiscellaneousAgent) return false; + if (outcome === 'exploitation_disabled' && value.exploit) return false; + if (hasMiscellaneousAgent && outcome !== 'expected' && outcome !== 'completed') return false; + + return value.report === undefined || isReportProgress(value.report, value.participating_classes); +} + +/** Create the byte-stable initial state written before the first required agent command. */ +export function createInitialDurableScanState(exploit: boolean): DurableScanState { + return { + schema_version: DURABLE_SCAN_STATE_SCHEMA_VERSION, + exploit, + participating_classes: [...FIXED_ANALYSIS_CLASSES], + expected_agents: expectedAgentsFor(exploit), + }; +} + +/** Add the analysis-less class without changing any admitted agent. */ +export function admitMiscellaneousParticipation(state: DurableScanState): DurableScanState { + if (state.participating_classes.includes('miscellaneous')) return state; + return { ...state, participating_classes: [...state.participating_classes, 'miscellaneous'] }; +} + +/** + * The outcomes that settle the `miscellaneous` class for good: its exploit agent finished, or the + * class was never actionable in the first place. A resumed run consults this before the lane runs + * again, since re-deciding admission from scratch would contradict what durable state already + * records and would rerun work a previous run already paid for. + */ +export function miscellaneousLaneIsSettled(outcome: MiscellaneousOutcome | undefined): boolean { + return outcome === 'completed' || outcome === 'not_actionable' || outcome === 'exploitation_disabled'; +} + +/** Persist a `miscellaneous` queue outcome; expected admission is append-only and idempotent. */ +export function recordMiscellaneousOutcome(state: DurableScanState, outcome: MiscellaneousOutcome): DurableScanState { + const current = state.miscellaneous_outcome; + // Re-recording the outcome already stored is the same fact twice, which a resumed run and a + // lost-acknowledgement re-drive both reach routinely. Participation was admitted alongside it, + // so nothing about the record moves. + if (current === outcome) return state; + // The one recognized forward transition: the class was queued for exploitation and its agent has + // now finished. It leaves participating_classes and expected_agents exactly as they were, so + // nothing a report already read changes and the promotion stays legal after reporting starts. + if (current === 'expected' && outcome === 'completed') { + return { ...state, miscellaneous_outcome: 'completed' }; + } + // Every remaining pair moves backwards or sideways: re-admitting 'expected' over 'completed', + // or re-recording 'not_actionable' after 'expected' already admitted the agent. + if (current !== undefined) { + throw new RunStateError('DurableStateConflictError', 'miscellaneous-outcome-transition-conflict'); + } + // A first admission adds `miscellaneous` to participating_classes, and for an actionable queue + // its agent to expected_agents. Once report progress exists the report has already read both, so + // admitting now would leave durable state disagreeing with a report that may be on disk. + if (state.report !== undefined) { + throw new RunStateError('DurableStateConflictError', 'miscellaneous-outcome-after-report-start'); + } + + const withMiscellaneous = admitMiscellaneousParticipation(state); + if (outcome === 'expected') { + if (!withMiscellaneous.exploit) { + throw new RunStateError('DurableStateConflictError', 'miscellaneous-exploit-disabled'); + } + return { + ...withMiscellaneous, + expected_agents: [...withMiscellaneous.expected_agents, 'miscellaneous-exploit'], + miscellaneous_outcome: outcome, + }; + } + if (outcome === 'completed') { + throw new RunStateError('DurableStateConflictError', 'miscellaneous-completed-before-admission'); + } + if (outcome === 'exploitation_disabled' && withMiscellaneous.exploit) { + throw new RunStateError('DurableStateConflictError', 'miscellaneous-disabled-outcome-on-exploit-run'); + } + return { ...withMiscellaneous, miscellaneous_outcome: outcome }; +} + +/** Return the exact expected-agent order for direct initialization verification. */ +export function initialExpectedAgents(exploit: boolean): readonly AgentName[] { + return expectedAgentsFor(exploit); +} + +/** Assert that an externally supplied analysis scope is the fixed five-class scope. */ +export function assertFixedAnalysisScope(value: readonly VulnClass[]): void { + if (!isFixedAnalysisScope(value)) { + throw new RunStateError('IncompatibleWorkspaceError', 'analysis-scope-not-fixed-five'); + } +} diff --git a/docs/configuration.md b/docs/configuration.md index d0afff46..37afb970 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -42,8 +42,11 @@ Source-build equivalent: # Describe your target environment. description: "Next.js e-commerce app on PostgreSQL. Local dev environment; .env files contain local-only credentials." -# Limit which vulnerability classes run end-to-end. -# vuln_classes: [injection, xss, auth, authz, ssrf] +# Every scan runs all five vulnerability classes. + +# Agentic static analysis. `enabled` is its only setting. +# agentic_sast: +# enabled: "true" # Skip the exploitation phase. # exploit: "false" @@ -102,6 +105,25 @@ rules: # sarif: "false" ``` +## Analysis Scope and Agentic SAST + +Every scan runs all five analysis classes: Injection, Cross-Site Scripting, Authentication, Authorization, and +Server-Side Request Forgery. The class set is fixed and has no configuration selector. + +Agentic static analysis is opt-in: + +```yaml +agentic_sast: + enabled: "true" +``` + +`enabled` is the only setting. Omitting the block, or setting `enabled: "false"`, turns agentic static analysis off; +`"true"` turns it on. Either way, all five vulnerability classes still run. + +Agentic static analysis reads the repository for vulnerabilities before the pentest and passes what it finds into the +exploitation phase. It adds model time and cost. If it fails, the pentest continues without its findings and the scan +finishes as "partial". + ## Report Options | Key | Effect | @@ -122,7 +144,9 @@ report: sarif: "false" ``` -Each finding becomes one SARIF result, filed under a rule per vulnerability class (`shannon/injection`, `shannon/xss`, `shannon/auth`, `shannon/authz`, `shannon/ssrf`) and tagged with its OWASP Top Ten 2025 category. Results are anchored to the code location the analysis phase recorded, falling back to the HTTP entry point when the finding names no file. Severity maps onto SARIF's three levels: `critical` and `high` become `error`, `medium` becomes `warning`, everything else becomes `note`. +Each finding becomes one SARIF result, filed under a rule per vulnerability class (`shannon/injection`, `shannon/xss`, `shannon/auth`, `shannon/authz`, `shannon/ssrf`, and `shannon/other` for findings outside those classes) and tagged with its OWASP Top Ten 2025 category. Results are anchored to the code location the analysis phase recorded, falling back to the HTTP entry point when the finding names no file. Severity maps onto SARIF's three levels: `critical` and `high` become `error`, `medium` becomes `warning`, everything else becomes `note`. + +If the SARIF log cannot be written, the JSON and Markdown reports are still produced and the scan finishes as "partial". The log is written only for exploitative runs. `sarif` is ignored when `exploit` is `"false"`. diff --git a/llms-full.txt b/llms-full.txt index 997835ac..71237989 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -510,8 +510,11 @@ Source-build equivalent: # Describe your target environment. description: "Next.js e-commerce app on PostgreSQL. Local dev environment; .env files contain local-only credentials." -# Limit which vulnerability classes run end-to-end. -# vuln_classes: [injection, xss, auth, authz, ssrf] +# Every scan runs all five vulnerability classes. + +# Agentic static analysis. `enabled` is its only setting. +# agentic_sast: +# enabled: "true" # Skip the exploitation phase. # exploit: "false" @@ -570,6 +573,25 @@ rules: # sarif: "false" ``` +## Analysis Scope and Agentic SAST + +Every scan runs all five analysis classes: Injection, Cross-Site Scripting, Authentication, Authorization, and +Server-Side Request Forgery. The class set is fixed and has no configuration selector. + +Agentic static analysis is opt-in: + +```yaml +agentic_sast: + enabled: "true" +``` + +`enabled` is the only setting. Omitting the block, or setting `enabled: "false"`, turns agentic static analysis off; +`"true"` turns it on. Either way, all five vulnerability classes still run. + +Agentic static analysis reads the repository for vulnerabilities before the pentest and passes what it finds into the +exploitation phase. It adds model time and cost. If it fails, the pentest continues without its findings and the scan +finishes as "partial". + ## Report Options | Key | Effect |