diff --git a/CLAUDE.md b/CLAUDE.md index d2df8c25..00740d27 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -60,8 +60,11 @@ npx @keygraph/shannon setup ./shannon start -u -r ./my-repo -w my-audit # Resume (same command) # Monitor -./shannon logs # Show a scan's live log -./shannon status # Live phase/agent progress of one scan, read from Temporal (redraws, then exits) +./shannon scans # List running and completed scans, with each report's path +./shannon logs [] # Show a scan's live log (default: the single running scan, else the most recent) +./shannon logs [] --agent # Tail one agent's own log (from .shannon/agents/) +./shannon logs [] --list-agents # List the agents that have their own log +./shannon status [] # Live phase/agent progress of one scan, read from Temporal (redraws, then exits; same default target) # Dashboard: http://localhost:8233 # Stop @@ -163,7 +166,7 @@ Around those phases: - **Prompts** — Per-phase templates in `apps/worker/prompts/` with variable substitution (`{{TARGET_URL}}`, `{{CONFIG_CONTEXT}}`). Shared partials in `apps/worker/prompts/shared/` via `apps/worker/src/services/prompt-manager.ts`, including `_code-path-rules.txt` (focus/avoid `[FILE]`/`[GLOB]` routing) and `_rules-of-engagement.txt` (free-text engagement rules). When `exploit: false`, `apps/worker/src/services/findings-renderer.ts` deterministically converts each `*_exploitation_queue.json` into a `*_findings.md` for report assembly — no LLM in the loop - **Agent Harness (pi)** — Uses the **pi harness** (`@earendil-works/pi-coding-agent`, requires Node ≥ 22.19) via `apps/worker/src/ai/pi/pi-executor.ts` (`runPiPrompt` → `createAgentSession`). Retry is split in `apps/worker/src/ai/pi/retry-settings.ts`: pi's agent-level loop is off so Temporal owns agent restarts, while `provider.maxRetries` stays on — pi reads the `provider` block independently of the `enabled` flag — so transport faults are absorbed in-session rather than costing a full agent re-run. `maxRetryDelayMs` is left at pi's 60s default. One model runs every phase, named by `SHANNON_AI_MODEL=:` (default `anthropic:claude-sonnet-4-6`). `apps/worker/src/ai/models.ts` parses the spec — splitting on the **first** colon only, so Bedrock IDs keep theirs — and resolves it through pi's `ModelRuntime`. pi ships the `CredentialStore` interface but no in-memory implementation (its own reads `auth.json` from disk), so `RuntimeCredentialStore` in that file supplies one: credentials arrive as env vars in an ephemeral container and must never touch disk. `createModelRuntime(providerId, apiKey)` builds the runtime; `allowModelNetwork` stays at its default `false` so a scan never blocks on a catalog refresh. `resolveModelSelection()` is **async** because `ModelRuntime.create()` is. Any pi-ai provider id is accepted — `parseModelSpec` no longer rejects against a hardcoded list, so pi's registry is the authority (an unknown provider/model surfaces as a clear "not found in pi registry" error at preflight, which points to the browsable catalogue at `pi.dev/models` — `PI_CATALOG_URL` in `apps/worker/src/ai/models.ts`, appended to the not-found errors and shown in the setup wizard's "Other provider" hint). Four providers are **curated** (`CURATED_PROVIDERS`: `anthropic`, `openai`, `xai`, `amazon-bedrock`) with their own credential variables, config sections, and setup flows; each provider's API key env var is declared once in `PROVIDER_API_KEY_ENV` — Shannon uses each vendor's own variable name (`OPENAI_API_KEY`, `XAI_API_KEY`, …), never an invented one; Bedrock's entry is `AWS_BEARER_TOKEN_BEDROCK`, paired with `AWS_REGION`, which preflight requires separately as provider config rather than a credential. Any other provider uses the **generic** credential path: `SHANNON_AI_API_KEY` (`GENERIC_API_KEY_ENV`) supplies the key for any provider whose credential is a plain API key. Curated providers' own variables take precedence over it, and it also works as a fallback for them — Bedrock is the sole exception (it authenticates through its AWS_ variables, so the generic key never stands in for it). The CLI forwards `SHANNON_AI_API_KEY` in `COMMON_FORWARD_VARS` (it is provider-neutral, binding to whatever `SHANNON_AI_MODEL` names, so the "only one provider configured" guard counts only named credentials), and stores it under a generic `[provider]` config.toml section (`provider.api_key`). `npx @keygraph/shannon setup` exposes this as the "Other provider" option: free-text provider id + model id + key (a curated provider id is rejected there, since it has its own option). `SHANNON_AI_BASE_URL` overrides the endpoint for any provider (proxies/gateways); the credential is unchanged. `pointAtGateway` (`apps/worker/src/ai/models.ts`) applies the one dialect change: behind a base URL, `openai` follows `SHANNON_AI_OPENAI_FORMAT` (`chat-completions` default, or `responses`). On `chat-completions` it switches the API to `openai-completions` and drops the catalogue's Responses-shaped `compat` block so pi's `detectCompat` derives completions settings; on `responses` the descriptor is unchanged but for the endpoint. `resolveGatewayFormat` rejects the variable when the provider is not `openai` or no base URL is set, since it cannot take effect there. All other providers keep their API. The CLI mirrors the accepted values in `apps/cli/src/model-spec.ts`, forwards the variable in `COMMON_FORWARD_VARS`, and maps it to `openai.format` in config.toml. `buildEnvFlags` forwards only the selected provider's credential into the worker container. The CLI mirrors the parse rule and the provider/credential tables in `apps/cli/src/model-spec.ts` (it cannot import from the worker package); the two must stay in sync. pi ships no JSON-schema output or `Task`/`TodoWrite` built-ins, so structured queues are captured via a `submit_exploitation_queue` custom tool (`apps/worker/src/ai/queue-schemas.ts`), and `task` (child sessions scoped to `read`, `grep`, `find`, `ls`, `write`, and `bash` — no nested `task` or collector tools; `CHILD_TOOLS` in `apps/worker/src/ai/pi/task-tool.ts`) + `todo_write` (`apps/worker/src/ai/pi/session-tools.ts`) are provided as custom tools; the per-phase collectors are pi custom tools (TypeBox `defineTool` in `apps/worker/src/collectors/`). Shannon sets no thinking configuration at all — no `thinkingLevel` is passed to any `createAgentSession` call, so pi's own default applies. There is no adaptive-thinking support and no `CLAUDE_ADAPTIVE_THINKING` / `core.adaptive_thinking` setting. Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login - **Pi Credential Reuse** — `SHANNON_USE_PI_AUTH=1` opts into reusing the host's Pi login, including an `openai-codex` ChatGPT Plus/Pro subscription selected with `SHANNON_AI_MODEL=openai-codex:`. `apps/cli/src/env.ts` requires `~/.pi/agent/auth.json`; `start.ts` passes its path to `spawnWorker`, which mounts only that file read-write at `/tmp/.pi/agent/auth.json`. The flag itself is not forwarded: the worker detects the file with `piAuthPresent()` and passes its path to `ModelRuntime.create`. CLI and worker API-key presence checks are skipped on this path, but the normal preflight model probe still validates the credential. The image and UID-remapping entrypoint keep `/tmp/.pi/agent` owned by `pentest` so adjacent Pi/Shannon configuration remains writable. Refreshed OAuth state is persisted to the host for subsequent scans. -- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds the human-facing report in both formats (`Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`, `FINAL_REPORT_PDF_FILENAME`/`FINAL_REPORT_MD_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`workspaces`/`logs`) without migration. Resuming a pre-restructure workspace upgrades it in place first: `migrateLegacyWorkspaceLayout` (`apps/cli/src/commands/start.ts`) renames the flat deliverables/logs/session entries into `.shannon/` (carrying the deliverables `.git` along) before the overlay dirs are mounted, so resume finds the old checkpoints instead of re-running every agent. The report agent writes structured findings to `report.json`, from which `report-renderer.ts` renders the assembled markdown and `report-json-adapter.ts` produces the Typst-shaped JSON that `pdf-renderer.ts` compiles into `comprehensive_security_assessment_report.pdf` using the bundled `apps/worker/templates/typst/report.typ` template (the `typst` binary is installed in the worker image). `copyReportToRunRoot` (`apps/worker/src/services/reporting.ts`) surfaces both the PDF and the markdown to the run root as `Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`; the deliverables-dir copies remain as the git-checkpointed sources. PDF compilation is best-effort — a failure is logged and the run still completes. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive +- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds the human-facing report in both formats (`Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`, `FINAL_REPORT_PDF_FILENAME`/`FINAL_REPORT_MD_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`scans`/`logs`) without migration. Resuming a pre-restructure workspace upgrades it in place first: `migrateLegacyWorkspaceLayout` (`apps/cli/src/commands/start.ts`) renames the flat deliverables/logs/session entries into `.shannon/` (carrying the deliverables `.git` along) before the overlay dirs are mounted, so resume finds the old checkpoints instead of re-running every agent. The report agent writes structured findings to `report.json`, from which `report-renderer.ts` renders the assembled markdown and `report-json-adapter.ts` produces the Typst-shaped JSON that `pdf-renderer.ts` compiles into `comprehensive_security_assessment_report.pdf` using the bundled `apps/worker/templates/typst/report.typ` template (the `typst` binary is installed in the worker image). `copyReportToRunRoot` (`apps/worker/src/services/reporting.ts`) surfaces both the PDF and the markdown to the run root as `Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`; the deliverables-dir copies remain as the git-checkpointed sources. PDF compilation is best-effort — a failure is logged and the run still completes. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive. Every combined-log line is also projected into a per-agent file under `.shannon/agents/.log` (one per pipeline agent, one per Capella stage; subagents fold into the parent's file, and a stage's concurrent sessions share its file with an inline session label). The projection boundary is `apps/worker/src/audit/actor-projection.ts` (`projectActor` maps a `TraceActor` to its combined prefix and owning file slug — slugs come only from closed fields); fan-out is best-effort and never blocks the canonical combined log. A lifecycle owner holds a `LogStream` lease per agent file (the pipeline agent's `logAgent` span, or a Capella stage activity's `try/finally`) so per-line writes ride the reference count; `CapellaStageTrace.drain()` flushes a stage's trace queue before its activity returns. The CLI tails one file with `shannon logs --agent ` (`--list-agents` to enumerate); the default `shannon logs` path is unchanged - **Deliverables** — Saved to `.shannon/deliverables/` in the target repo via the `save-deliverable` CLI script (`apps/worker/src/scripts/save-deliverable.ts`) - **Workspaces & Resume** — Named workspaces via `-w ` or auto-named from URL+timestamp. Resume detects completed agents via `session.json`. `loadResumeState()` in `apps/worker/src/temporal/activities.ts` validates deliverable existence, restores git checkpoints, and cleans up incomplete deliverables diff --git a/apps/cli/src/commands/logs.ts b/apps/cli/src/commands/logs.ts index 84989c3f..42632549 100644 --- a/apps/cli/src/commands/logs.ts +++ b/apps/cli/src/commands/logs.ts @@ -9,6 +9,7 @@ import fs from 'node:fs'; import path from 'node:path'; +import { StringDecoder } from 'node:string_decoder'; import { setTimeout as sleep } from 'node:timers/promises'; import { watch } from 'chokidar'; import { fail } from '../errors.js'; @@ -18,8 +19,69 @@ import { resolveWorkflowId } from '../session.js'; import { waitForWorkflowClose } from '../temporal-client.js'; import { stdoutIsTerminal } from '../tty.js'; -/** Read a byte range from a file and return it as a UTF-8 string. */ -function readRange(filePath: string, start: number, end: number): string { +const TERMINAL_HEADINGS = new Set(['Scan COMPLETED', 'Scan PARTIAL', 'Scan FAILED', 'Scan CANCELLED']); + +// The combined log resets completion on the bare `RESUMED` heading; a per-agent file carries the +// distinct `--- RESUMED () ---` boundary that WorkflowLogger.logResumeBoundary writes +// (kept distinct per resume so it stays idempotent per file). Both mean a new execution began, so a +// `--agent` tail must clear a stale terminal marker on either, matching the combined tail. +const AGENT_RESUME_BOUNDARY = /^--- RESUMED \(.+\) ---$/u; + +function isResumeBoundary(line: string): boolean { + return line === 'RESUMED' || AGENT_RESUME_BOUNDARY.test(line); +} + +/** Tracks only complete structural lines while output remains byte-for-byte unchanged. */ +export class LogCompletionState { + private pendingLine = ''; + private terminalIsLastMarker = false; + private failureIsLastMarker = false; + + ingest(chunk: string): void { + const lines = `${this.pendingLine}${chunk}`.split('\n'); + this.pendingLine = lines.pop() ?? ''; + for (const line of lines) { + if (isResumeBoundary(line)) { + this.terminalIsLastMarker = false; + this.failureIsLastMarker = false; + } else if (TERMINAL_HEADINGS.has(line)) { + this.terminalIsLastMarker = true; + this.failureIsLastMarker = line === 'Scan FAILED'; + } + } + } + + isComplete(): boolean { + return this.terminalIsLastMarker; + } + + hasFailureMarker(): boolean { + return this.failureIsLastMarker; + } +} + +/** Append the forced-stop marker after the worker has exited, unless this execution already ended. */ +export function appendCancellationFallback(logFile: string): void { + fs.mkdirSync(path.dirname(logFile), { recursive: true }); + const state = new LogCompletionState(); + try { + state.ingest(fs.readFileSync(logFile, 'utf8')); + } catch { + // A pre-registration stop may not have created the file yet. + } + if (state.isComplete()) return; + + const descriptor = fs.openSync(logFile, 'a', 0o600); + try { + fs.writeSync(descriptor, '\nScan CANCELLED\n'); + fs.fsyncSync(descriptor); + } finally { + fs.closeSync(descriptor); + } +} + +/** Read a byte range without decoding across an arbitrary live-write boundary. */ +function readRange(filePath: string, start: number, end: number): Buffer { const length = end - start; const buffer = Buffer.alloc(length); const fd = fs.openSync(filePath, 'r'); @@ -28,7 +90,7 @@ function readRange(filePath: string, start: number, end: number): string { } finally { fs.closeSync(fd); } - return buffer.toString('utf-8'); + return buffer; } /** Resolve a workspace ID to its workflow.log path, or exit with an error. */ @@ -76,9 +138,6 @@ export interface TailResult { readonly sawFailure: boolean; } -// The worker writes this exact line at the head of its terminal failure summary. -const FAILURE_MARKER = /^Scan FAILED$/m; - /** * Stream a scan's log to the terminal until the workflow closes (completion comes from Temporal, * or Ctrl-C). A Temporal outage is warned about and, if sustained, ends the tail with a diagnostic. @@ -88,24 +147,25 @@ const FAILURE_MARKER = /^Scan FAILED$/m; export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Promise { return new Promise((resolve) => { let position = 0; + const completion = new LogCompletionState(); let done = false; - let sawFailure = false; const controller = new AbortController(); let watcher: ReturnType | undefined; + const completionDecoder = new StringDecoder('utf8'); /** Output any new content appended since the last read. */ - function flush(): void { + function flush(): boolean { try { const { size } = fs.statSync(logFile); - if (size <= position) return; + if (size <= position) return completion.isComplete(); const data = readRange(logFile, position, size); process.stdout.write(data); position = size; - if (!sawFailure && FAILURE_MARKER.test(data)) { - sawFailure = true; - } + completion.ingest(completionDecoder.write(data)); + return completion.isComplete(); } catch { // File not present yet or transiently unreadable — nothing to flush this round. + return false; } } @@ -113,22 +173,32 @@ export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Prom if (done) return; done = true; controller.abort(); + process.off('SIGINT', finish); + const result = { sawFailure: completion.hasFailureMarker() }; if (watcher) { - watcher.close().finally(() => resolve({ sawFailure })); + watcher.close().finally(() => resolve(result)); // Safety net — resolve anyway if watcher.close() stalls. - setTimeout(() => resolve({ sawFailure }), 1000).unref(); + setTimeout(() => resolve(result), 1000).unref(); } else { - resolve({ sawFailure }); + resolve(result); } } - // 1. Output existing content, then stream anything appended. - flush(); + // 1. Output existing content, then stream anything appended. A per-agent file can be created + // after the watcher starts, so `add` is handled too and streams it from its first line. watcher = watch(logFile, { persistent: true }); - watcher.on('change', () => flush()); + const onFsEvent = (): void => { + if (flush() && !opts.workflowId) finish(); + }; + watcher.on('change', onFsEvent); + watcher.on('add', onFsEvent); + if (flush() && !opts.workflowId) { + finish(); + return; + } // 2. Ctrl-C stops watching. - process.on('SIGINT', finish); + process.once('SIGINT', finish); // 3. Temporal decides completion. Without a workflow id, the tail relies on Ctrl-C alone. if (opts.workflowId) { @@ -162,11 +232,51 @@ export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Prom }); } -export function logs(workspaceId: string): void { - const logFile = resolveLogFile(workspaceId); - const workflowId = resolveWorkflowId(workspaceId); - console.error(stdoutIsTerminal() ? `Tailing scan log: ${logFile}` : 'Tailing scan log'); +/** The `.shannon/agents/` directory that sits beside a scan's combined workflow.log. */ +function agentsDirFor(logFile: string): string { + return path.join(path.dirname(logFile), 'agents'); +} +/** List the per-agent log names available for a scan (filename stems, sorted), or an empty list. */ +export function listAgentLogNames(logFile: string): string[] { + try { + return fs + .readdirSync(agentsDirFor(logFile)) + .filter((entry) => entry.endsWith('.log')) + .map((entry) => entry.slice(0, -'.log'.length)) + .sort(); + } catch { + return []; + } +} + +/** + * Resolve an agent name to its per-agent log path. The name must be a closed-charset basename, and + * the resolved file must stay inside the agents directory: traversal and symlink escapes are + * rejected. Returns undefined when the name is structurally invalid or escapes the directory. + */ +export function resolveAgentLogFile(logFile: string, agentName: string): string | undefined { + if (!/^[a-z0-9][a-z0-9-]{0,63}$/u.test(agentName)) return undefined; + const agentsDir = agentsDirFor(logFile); + const target = path.join(agentsDir, `${agentName}.log`); + try { + const realDir = fs.realpathSync(agentsDir); + const realTarget = fs.realpathSync(target); + if (realTarget !== path.join(realDir, `${agentName}.log`)) return undefined; + } catch { + // The file does not exist yet (scan still starting); the closed-charset check already proved + // the path cannot traverse out of the agents directory, so it is safe to watch for creation. + } + return target; +} + +export interface LogsOptions { + readonly agent?: string; + readonly listAgents?: boolean; +} + +function tailFileToExit(logFile: string, workflowId: string | undefined, label: string): void { + console.error(stdoutIsTerminal() ? `${label}: ${logFile}` : label); let unreachable = false; tailUntilComplete(logFile, { ...(workflowId ? { workflowId } : {}), @@ -175,3 +285,40 @@ export function logs(workspaceId: string): void { }, }).finally(() => process.exit(unreachable ? 1 : 0)); } + +export function logs(workspaceId: string, options: LogsOptions = {}): void { + const logFile = resolveLogFile(workspaceId); + + if (options.listAgents) { + const names = listAgentLogNames(logFile); + if (names.length === 0) { + console.error('No per-agent logs for this scan yet.'); + process.exit(0); + } + for (const name of names) console.log(name); + process.exit(0); + } + + const workflowId = resolveWorkflowId(workspaceId); + + if (options.agent !== undefined) { + const agentFile = resolveAgentLogFile(logFile, options.agent); + if (agentFile === undefined) { + fail(`No agent log named: ${options.agent}`, '', 'Available agents:', ...withBullets(listAgentLogNames(logFile))); + } + const known = listAgentLogNames(logFile); + // If the directory already lists agents, a name not among them is a typo, not a not-yet-created + // file; fail loudly rather than tailing a path that will never appear. + if (known.length > 0 && !known.includes(options.agent)) { + fail(`No agent log named: ${options.agent}`, '', 'Available agents:', ...withBullets(known)); + } + tailFileToExit(agentFile, workflowId, `Tailing ${options.agent} log`); + return; + } + + tailFileToExit(logFile, workflowId, 'Tailing scan log'); +} + +function withBullets(names: readonly string[]): string[] { + return names.length === 0 ? [' (none yet)'] : names.map((name) => ` - ${name}`); +} diff --git a/apps/cli/src/commands/stop.ts b/apps/cli/src/commands/stop.ts index e7651e5c..f754a817 100644 --- a/apps/cli/src/commands/stop.ts +++ b/apps/cli/src/commands/stop.ts @@ -3,24 +3,29 @@ * Never touches infra or data; to wipe Temporal state entirely, use `shannon reset`. */ +import path from 'node:path'; import * as p from '@clack/prompts'; import { confirmOrExit } from '../confirm.js'; import { anyRunningScanWorkflow, + cancelWorkflow, ensureDocker, isTemporalReady, isWorkflowRunning, runningContainers, + runningScanWorkspaces, scanFilter, stopContainers, - terminateAllWorkflows, terminateWorkflow, WORKER_FILTER, } from '../docker.js'; import { fail, failUsage, warn } from '../errors.js'; +import { getWorkspacesDir } from '../home.js'; import { commandPrefix } from '../mode.js'; +import { resolveRunFile } from '../paths.js'; import { resolveWorkflowId } from '../session.js'; import { resolveDefaultWorkspace } from '../workspaces.js'; +import { appendCancellationFallback } from './logs.js'; export interface StopOptions { all: boolean; @@ -28,11 +33,77 @@ export interface StopOptions { workspace?: string; } +const CANCELLATION_GRACE_MS = 10_000; +const CANCELLATION_POLL_MS = 250; + +export interface StopTarget { + readonly workspace: string; + readonly workflowId?: string; + readonly workflowRunning: boolean; +} + +export interface StopLifecycle { + readonly cancel: (workflowId: string) => boolean; + readonly isRunning: (workflowId: string) => boolean; + readonly terminate: (workflowId: string) => boolean; + readonly containers: (workspace: string) => string[]; + readonly stopContainers: (ids: string[]) => Promise; + readonly appendFallback: (workspace: string) => void; + readonly wait: (milliseconds: number) => Promise; +} + +const stopLifecycle: StopLifecycle = { + cancel: cancelWorkflow, + isRunning: isWorkflowRunning, + terminate: (workflowId) => terminateWorkflow(workflowId, 'Stopped after cancellation grace period'), + containers: (workspace) => runningContainers(scanFilter(workspace)), + stopContainers, + appendFallback: (workspace) => { + const logFile = resolveRunFile(path.join(getWorkspacesDir(), workspace), 'workflow.log'); + appendCancellationFallback(logFile); + }, + wait: (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds)), +}; + +/** Cancel first; terminate and write the fallback heading only when graceful closure misses its deadline. */ +export async function stopTargetCancelFirst( + target: StopTarget, + lifecycle: StopLifecycle = stopLifecycle, + graceMs: number = CANCELLATION_GRACE_MS, + pollMs: number = CANCELLATION_POLL_MS, +): Promise<'graceful' | 'forced'> { + let forced = !target.workflowRunning || target.workflowId === undefined; + if (!forced && target.workflowId !== undefined) { + lifecycle.cancel(target.workflowId); + const deadline = Date.now() + graceMs; + while (lifecycle.isRunning(target.workflowId) && Date.now() < deadline) { + await lifecycle.wait(pollMs); + } + forced = lifecycle.isRunning(target.workflowId); + if (forced) lifecycle.terminate(target.workflowId); + } + + await lifecycle.stopContainers(lifecycle.containers(target.workspace)); + if (forced && lifecycle.containers(target.workspace).length === 0) { + lifecycle.appendFallback(target.workspace); + } + return forced ? 'forced' : 'graceful'; +} + +/** Apply the same captured-target lifecycle concurrently for `stop --all`. */ +export function stopTargetsCancelFirst( + targets: readonly StopTarget[], + lifecycle: StopLifecycle = stopLifecycle, + graceMs: number = CANCELLATION_GRACE_MS, + pollMs: number = CANCELLATION_POLL_MS, +): Promise { + return Promise.all(targets.map((target) => stopTargetCancelFirst(target, lifecycle, graceMs, pollMs))); +} + /** - * Stop a single scan. Terminating the workflow both clears Temporal's record and - * brings the container down (the worker waits on the workflow result), so that runs - * first; `docker stop` is the fallback for the pre-registration window and an - * unreachable Temporal. The stop is then verified rather than assumed. + * Stop a single scan. Cooperative cancellation gets the first ten seconds so the + * workflow can flush its terminal log; termination and a host-written heading are + * the fallback for the pre-registration window or an unavailable finalizer. */ async function stopSingleScan(workspace: string, yes: boolean): Promise { const workflowId = resolveWorkflowId(workspace); @@ -56,10 +127,7 @@ async function stopSingleScan(workspace: string, yes: boolean): Promise { const spinner = p.spinner(); spinner.start(`Stopping scan ${workspace}`); - if (workflowId && workflowRunning) { - terminateWorkflow(workflowId, `Stopped via shannon stop ${workspace}`); - } - await stopContainers(runningContainers(filter)); + await stopTargetCancelFirst({ workspace, ...(workflowId !== undefined && { workflowId }), workflowRunning }); const stillRunning = runningContainers(filter); if (stillRunning.length > 0) { @@ -78,6 +146,14 @@ async function stopSingleScan(workspace: string, yes: boolean): Promise { async function stopAllScans(yes: boolean): Promise { const temporalUp = isTemporalReady(); const initial = runningContainers(WORKER_FILTER); + const targets = [...new Set(runningScanWorkspaces())].map((workspace): StopTarget => { + const workflowId = resolveWorkflowId(workspace); + return { + workspace, + ...(workflowId !== undefined && { workflowId }), + workflowRunning: Boolean(workflowId && temporalUp && isWorkflowRunning(workflowId)), + }; + }); // Resolve what is running before prompting, so we never confirm a no-op. if (initial.length === 0) { @@ -90,9 +166,8 @@ async function stopAllScans(yes: boolean): Promise { const spinner = p.spinner(); spinner.start('Stopping all scans'); - if (temporalUp) { - terminateAllWorkflows('Stopped via shannon stop --all'); - } + await stopTargetsCancelFirst(targets); + // Keep the legacy safety net for a worker whose workspace label was unavailable. await stopContainers(runningContainers(WORKER_FILTER)); const stillRunning = runningContainers(WORKER_FILTER); diff --git a/apps/cli/src/docker.ts b/apps/cli/src/docker.ts index fd774e3e..f7645159 100644 --- a/apps/cli/src/docker.ts +++ b/apps/cli/src/docker.ts @@ -519,6 +519,11 @@ export async function stopContainers(ids: string[]): Promise { await Promise.all(ids.map((id) => spawnQuiet('docker', ['stop', id]))); } +/** Request cooperative cancellation so the workflow can run its terminal finalizer. */ +export function cancelWorkflow(workflowId: string): boolean { + return runQuiet('docker', temporalCmd('workflow', 'cancel', '--workflow-id', workflowId)); +} + /** * Terminate a Temporal workflow so a stopped scan doesn't linger as a running * workflow with no worker. Best-effort: returns false if Temporal is unreachable @@ -528,18 +533,6 @@ export function terminateWorkflow(workflowId: string, reason: string): boolean { return runQuiet('docker', temporalCmd('workflow', 'terminate', '--workflow-id', workflowId, '--reason', reason)); } -/** - * Terminate every running pentest workflow in one batch, so `stop --all` doesn't - * leave workflows running with no worker. Best-effort: returns false if Temporal - * is unreachable. Requires Temporal to be up (guard with isTemporalReady). - */ -export function terminateAllWorkflows(reason: string): boolean { - return runQuiet( - 'docker', - temporalCmd('workflow', 'terminate', '--query', RUNNING_SCAN_QUERY, '--reason', reason, '--yes'), - ); -} - /** * Whether a specific workflow is still in the Running state. Re-querying this after * a terminate verifies it actually took effect, rather than trusting the terminate diff --git a/apps/cli/src/index.ts b/apps/cli/src/index.ts index 5b95479f..30da9c7c 100644 --- a/apps/cli/src/index.ts +++ b/apps/cli/src/index.ts @@ -94,6 +94,7 @@ function renderUsage(prefix: string, mode: Mode): string { [`${prefix} stop --all [--yes]`, 'Stop all scans (Temporal stays up)'], [`${prefix} reset`, 'Stop everything and wipe all Temporal data'], [`${prefix} logs []`, "Show a scan's live log (default: running or most recent)"], + [`${prefix} logs [] --agent `, "Tail one agent's log; --list-agents to list them"], [ `${prefix} status [] [--json]`, 'Live phase/agent progress of one scan (default: running or most recent)', @@ -294,9 +295,19 @@ async function main(): Promise { break; } case 'logs': { - const { positionals } = parseArgs(rest, { maxPositionals: 1 }); - const workspaceId = resolveViewingWorkspace(positionals[0], `Usage: ${commandPrefix()} logs []`); - logs(workspaceId); + const { flags, values, positionals } = parseArgs(rest, { + booleans: { listAgents: ['--list-agents'] }, + values: { agent: ['--agent'] }, + maxPositionals: 1, + }); + const workspaceId = resolveViewingWorkspace( + positionals[0], + `Usage: ${commandPrefix()} logs [] [--agent ] [--list-agents]`, + ); + logs(workspaceId, { + ...(values.agent !== undefined && { agent: values.agent }), + ...(flags.listAgents && { listAgents: true }), + }); break; } case 'status': { diff --git a/apps/cli/src/scan/derive.ts b/apps/cli/src/scan/derive.ts index c9b76051..5c24585c 100644 --- a/apps/cli/src/scan/derive.ts +++ b/apps/cli/src/scan/derive.ts @@ -16,6 +16,7 @@ import { pipelineForState, } from './pipeline.js'; import type { RenderInput } from './render.js'; +import { safeFailureDetail, safeOperationKey, safeOperationLabel } from './safe-fields.js'; export type RunState = 'pending' | 'running' | 'completed' | 'failed' | 'skipped'; @@ -77,11 +78,9 @@ function agentState(name: string, state: PipelineState | null, running: Set): string | undefined { const failed = state?.failedPipelines.find((f) => f.vulnType === agentClass(name)); - return ( - failed?.error ?? - byAgent.get(name)?.lastFailure ?? - (state?.failedAgent === name ? (state.error ?? undefined) : undefined) - ); + const hasFailure = + failed !== undefined || byAgent.get(name)?.lastFailure !== undefined || state?.failedAgent === name; + return safeFailureDetail(hasFailure); } /** Scan wall-clock elapsed ms: recorded duration for a closed scan, live elapsed for a running one. */ @@ -229,7 +228,7 @@ export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] label: runner.label, status: 'running' as const, ...(runner.startedAt !== undefined && { startedAt: runner.startedAt }), - ...(runner.lastFailure !== undefined && { error: runner.lastFailure }), + ...(runner.lastFailure !== undefined && { error: safeFailureDetail(true) }), })); const operationalAgents: DerivedAgent[] = [...persistedOperations, ...unpersistedRunning].map((operation) => { const runner = byAgent.get(operation.key); @@ -237,15 +236,15 @@ export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] const persistedDurationMs = 'durationMs' in operation ? (operation.durationMs ?? null) : null; const detail = operationState === 'running' ? stepByFamily.get(operationFamilyKey(operation.key)) : undefined; return { - name: operation.key, - label: operation.label, + name: safeOperationKey(operation.key), + label: safeOperationLabel(operation.label), state: operationState, durationMs: operationState === 'completed' ? persistedDurationMs : null, runningElapsedMs: operationState === 'running' && operation.startedAt !== undefined ? now - operation.startedAt : null, attempt: operationState === 'running' ? (runner?.attempt ?? null) : null, ...(detail !== undefined && { detail }), - ...(operation.error !== undefined && { error: operation.error }), + ...(operation.error !== undefined && { error: safeFailureDetail(true) }), }; }); diff --git a/apps/cli/src/scan/render.ts b/apps/cli/src/scan/render.ts index aad2b960..b6fe1b84 100644 --- a/apps/cli/src/scan/render.ts +++ b/apps/cli/src/scan/render.ts @@ -12,6 +12,7 @@ import { commandPrefix } from '../mode.js'; import type { RunningAgent } from '../temporal-client.js'; import { derivePipeline, isTerminal, type RunState, scanElapsedMs } from './derive.js'; import type { PipelineState } from './pipeline.js'; +import { safeAgenticSast, safeCliIdentifier, safePartialReasons, safeTerminalFailure } from './safe-fields.js'; export interface RenderInput { readonly workspace: string; @@ -67,7 +68,7 @@ function truncate(text: string, max: number): string { /** Temporal Web UI, published by compose on 8233; deep-links to the workflow when its id is known. */ function temporalDashboardUrl(workflowId: string | undefined): string { const base = 'http://localhost:8233'; - return workflowId ? `${base}/namespaces/default/workflows/${workflowId}` : base; + return workflowId ? `${base}/namespaces/default/workflows/${safeCliIdentifier(workflowId)}` : base; } // === Glyphs & status === @@ -214,7 +215,7 @@ export function renderScan(input: RenderInput, opts: RenderOptions): string { function headerLines(input: RenderInput, opts: RenderOptions): string[] { const elapsedMs = scanElapsedMs(input, opts.now); const meta = [statusBadge(input, opts), elapsedMs !== undefined ? formatDuration(elapsedMs) : '—'].join(' · '); - return [` ${paint('Scan:', COLORS.bold, opts.color)} ${input.workspace.padEnd(22)} ${meta}`]; + return [` ${paint('Scan:', COLORS.bold, opts.color)} ${safeCliIdentifier(input.workspace).padEnd(22)} ${meta}`]; } /** Aligned label column for the footer's Logs / Temporal rows. */ @@ -239,7 +240,7 @@ function footerLines(input: RenderInput, opts: RenderOptions): string[] { // A partial scan names each durable degradation reason through its safe message, // so the operator never has to guess why the badge is not "completed". - const reasons = input.state.partialReasons ?? []; + const reasons = safePartialReasons(input.state.partialReasons ?? []); if (reasons.length > 0) { lines.push('', ` ${paint('Why this scan is partial:', COLORS.yellow, opts.color)}`); for (const reason of reasons) { @@ -247,7 +248,7 @@ function footerLines(input: RenderInput, opts: RenderOptions): string[] { } // The safe message names what degraded; these three name the agentic-SAST failure // behind it, under the same labels the scan log and worker output use. - const agenticSast = input.state.agenticSast; + const agenticSast = safeAgenticSast(input.state.agenticSast); if (agenticSast?.status === 'failed') { if (agenticSast.failedStageLabel !== undefined) { lines.push(paint(` Agentic SAST stopped at: ${agenticSast.failedStageLabel}`, COLORS.dim, opts.color)); @@ -268,11 +269,13 @@ function footerLines(input: RenderInput, opts: RenderOptions): string[] { return lines; } - const logsValue = `${prefix} logs ${input.workspace}`; + const logsValue = `${prefix} logs ${safeCliIdentifier(input.workspace)}`; const temporalValue = temporalDashboardUrl(input.workflowId); if (isTerminal(input.temporalStatus)) { - const reason = input.failureMessage ?? input.state?.error ?? 'no result recorded'; + const hasRecordedFailure = + input.failureMessage !== undefined || (input.state !== null && input.state.error !== null); + const reason = safeTerminalFailure(hasRecordedFailure) ?? 'no result recorded'; return [ footerDivider(opts), paint( diff --git a/apps/cli/src/scan/safe-fields.ts b/apps/cli/src/scan/safe-fields.ts new file mode 100644 index 00000000..60ef5aa3 --- /dev/null +++ b/apps/cli/src/scan/safe-fields.ts @@ -0,0 +1,215 @@ +/** Closed-field projection for Temporal values displayed by the CLI. */ + +import type { PartialReasonView, PipelineState } from './pipeline.js'; + +const CLASS_NAMES: Readonly> = Object.freeze({ + injection: 'Injection', + xss: 'Cross-Site Scripting', + auth: 'Authentication', + authz: 'Authorization', + ssrf: 'Server-Side Request Forgery', + miscellaneous: 'Miscellaneous', +}); + +const STAGE_NAMES: Readonly> = Object.freeze({ + architecture: 'architecture mapping', + 'threat-model': 'threat modelling', + plan: 'review planning', + research: 'deep code research', + dedupe: 'duplicate merging', + review: 'independent review', + critic: 'viability critique', + confirm: 'static confirmation', + calibrate: 'risk calibration', + export: 'findings export', + workflow: 'orchestration', +}); + +const TERMINAL_STAGE_NAMES = new Set([ + 'architecture', + 'threat model', + 'planning', + 'audit wave', + 'deduplication', + 'review', + 'critic', + 'confirmation', + 'calibration', + 'export', + 'orchestration', +]); + +const CAPELLA_FAILURE_MESSAGES = new Set([ + 'Provider authentication failed. Verify the configured credential.', + 'Agentic SAST configuration is invalid.', + 'Agentic SAST received invalid input.', + 'An agentic SAST step returned an unusable result.', + 'An agentic SAST step failed.', + 'Agentic SAST infrastructure failed before producing a usable result.', + 'Agentic SAST had not finished when the scan stopped.', +]); + +const OPERATION_LABELS = new Set([ + 'Agentic SAST', + 'Miscellaneous findings', + 'Reconcile injection', + 'Reconcile xss', + 'Reconcile auth', + 'Reconcile authz', + 'Reconcile ssrf', + 'Reconcile miscellaneous', + 'Prepare reconciliation', + 'Enrich observations', + 'Form exploit tasks', + 'Materialize exploit tasks', + 'Publish reconciliation', + 'Renumber injection', + 'Renumber xss', + 'Renumber auth', + 'Renumber authz', + 'Renumber ssrf', + 'Renumber miscellaneous', + 'Initialize report state', + 'Assemble report inputs', + 'Compact report findings', + 'Saving report progress', + 'Finalize report outputs', + 'Finalize report without SARIF', + 'Saving final report state', + 'Surface customer report', +]); + +function safeClassName(value: string | undefined): string | undefined { + return value === undefined ? undefined : CLASS_NAMES[value]; +} + +function safeStageName(value: string | undefined): string | undefined { + return value === undefined ? undefined : STAGE_NAMES[value]; +} + +function reasonMessage(reason: PartialReasonView): string | undefined { + const className = safeClassName(reason.vulnerabilityClass); + switch (reason.code) { + case 'agentic_sast_failed': { + const stageName = safeStageName(reason.stage); + return stageName === undefined + ? 'Agentic SAST failed, so the pentest continued without its findings.' + : `Agentic SAST failed during ${stageName}, so the pentest continued without its findings.`; + } + case 'agentic_sast_reduced': + return 'Agentic SAST completed with reduced coverage.'; + case 'class_pipeline_failed': + return className === undefined + ? undefined + : `${className} could not be fully assessed. The other classes completed. Re-running this workspace retries only the part that failed.`; + case 'class_reconciliation_failed': + return className === undefined + ? undefined + : `${className} findings could not be grouped into test cases, so that class was not exploited. Its analysis results are still in the report.`; + case 'report_renumber_failed': + return className === undefined + ? undefined + : `${className} findings kept their working reference numbers, so numbering in the report may have gaps. The findings themselves are complete.`; + case 'report_compaction_failed': + return 'Finding reference numbers in the report may have gaps. Every finding is present; only the numbering is affected.'; + case 'report_class_omitted': + return className === undefined + ? undefined + : `${className} was assessed but could not be included in the final report.`; + case 'report_sarif_failed': + return 'Report SARIF could not be generated. JSON and Markdown remain available.'; + default: + return undefined; + } +} + +export function safePartialReasons(reasons: readonly PartialReasonView[]): readonly PartialReasonView[] { + return reasons.flatMap((reason) => { + const message = reasonMessage(reason); + if (message === undefined) return []; + const vulnerabilityClass = + safeClassName(reason.vulnerabilityClass) === undefined ? undefined : reason.vulnerabilityClass; + const stage = safeStageName(reason.stage) === undefined ? undefined : reason.stage; + return [ + { + code: reason.code, + message, + ...(vulnerabilityClass !== undefined && { vulnerabilityClass }), + ...(stage !== undefined && { stage }), + }, + ]; + }); +} + +export function safeAgenticSast(value: PipelineState['agenticSast']): + | { + readonly status: string; + readonly failedStageLabel?: string; + readonly error?: string; + readonly errorCode?: string; + } + | undefined { + if (value === undefined || !['disabled', 'running', 'succeeded', 'failed'].includes(value.status)) return undefined; + const failedStageLabel = TERMINAL_STAGE_NAMES.has(value.failedStageLabel ?? '') ? value.failedStageLabel : undefined; + let error: string | undefined; + if (value.error !== undefined && CAPELLA_FAILURE_MESSAGES.has(value.error)) { + error = value.error; + } else if (value.status === 'failed') { + error = 'An agentic SAST step failed.'; + } + const errorCode = + value.errorCode !== undefined && /^[A-Z][A-Z0-9_]{0,63}$/u.test(value.errorCode) ? value.errorCode : undefined; + return { + status: value.status, + ...(failedStageLabel !== undefined && { failedStageLabel }), + ...(error !== undefined && { error }), + ...(errorCode !== undefined && { errorCode }), + }; +} + +export function safeOperationLabel(value: string): string { + return OPERATION_LABELS.has(value) ? value : 'Background task'; +} + +export function safeOperationKey(value: string): string { + if ( + /^(?:agentic-sast|miscellaneous-pipeline|report:(?:initialize|assemble|compact|checkpoint|finalize|finalize-degraded|terminal|surface))$/u.test( + value, + ) || + /^(?:reconciliation|report:renumber):(?:injection|xss|auth|authz|ssrf|miscellaneous)$/u.test(value) + ) { + return value; + } + return 'background-task'; +} + +export function safeCliIdentifier(value: string): string { + return /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/u.test(value) ? value : 'unknown'; +} + +export function safeTemporalStatus(value: string): string { + return [ + 'RUNNING', + 'UNSPECIFIED', + 'COMPLETED', + 'FAILED', + 'CANCELLED', + 'CANCELED', + 'TERMINATED', + 'TIMED_OUT', + 'CONTINUED_AS_NEW', + ].includes(value) + ? value + : 'UNKNOWN'; +} + +export function safeFailureDetail(hasFailure: true): string; +export function safeFailureDetail(hasFailure: false): undefined; +export function safeFailureDetail(hasFailure: boolean): string | undefined; +export function safeFailureDetail(hasFailure: boolean): string | undefined { + return hasFailure ? 'This scan step could not be completed.' : undefined; +} + +export function safeTerminalFailure(hasFailure: boolean): string | undefined { + return hasFailure ? 'The scan could not be completed.' : undefined; +} diff --git a/apps/cli/src/scan/status-json.ts b/apps/cli/src/scan/status-json.ts index dd26a1b3..9702f798 100644 --- a/apps/cli/src/scan/status-json.ts +++ b/apps/cli/src/scan/status-json.ts @@ -10,6 +10,13 @@ import type { DerivedPhase } from './derive.js'; import { derivePipeline, isTerminal, scanElapsedMs } from './derive.js'; import type { PartialReasonView } from './pipeline.js'; import type { RenderInput } from './render.js'; +import { + safeAgenticSast, + safeCliIdentifier, + safePartialReasons, + safeTemporalStatus, + safeTerminalFailure, +} from './safe-fields.js'; /** Coarse scan status token, mirroring the human status badge in machine-friendly form. */ export type ScanStatus = 'running' | 'completed' | 'partial' | 'failed' | 'stopped' | 'cancelled' | 'timed_out'; @@ -61,19 +68,20 @@ function deriveStatus(input: RenderInput): ScanStatus { /** Build the JSON snapshot for a scan at instant `now`. */ export function toStatusJson(input: RenderInput, now: number): StatusJson { const elapsedMs = scanElapsedMs(input, now); - const partialReasons = input.state?.partialReasons ?? []; - const agenticSast = input.state?.agenticSast; + const partialReasons = safePartialReasons(input.state?.partialReasons ?? []); + const agenticSast = safeAgenticSast(input.state?.agenticSast); const usageAccountingComplete = input.state?.summary?.usageAccountingComplete; + const failureMessage = safeTerminalFailure(input.failureMessage !== undefined); return { - workspace: input.workspace, - ...(input.workflowId !== undefined && { workflowId: input.workflowId }), + workspace: safeCliIdentifier(input.workspace), + ...(input.workflowId !== undefined && { workflowId: safeCliIdentifier(input.workflowId) }), status: deriveStatus(input), - temporalStatus: input.temporalStatus, + temporalStatus: safeTemporalStatus(input.temporalStatus), elapsedMs: elapsedMs ?? null, ...(input.startedAt !== undefined && { startedAt: new Date(input.startedAt).toISOString() }), ...(input.endedAt !== undefined && { endedAt: new Date(input.endedAt).toISOString() }), - ...(input.failureMessage !== undefined && { failureMessage: input.failureMessage }), + ...(failureMessage !== undefined && { failureMessage }), ...(partialReasons.length > 0 && { partialReasons }), // Present only when agentic SAST actually ran; a disabled scan omits the key entirely. ...(agenticSast !== undefined && diff --git a/apps/worker/src/ai/audit-logger.ts b/apps/worker/src/ai/audit-logger.ts index bfe75315..2590997b 100644 --- a/apps/worker/src/ai/audit-logger.ts +++ b/apps/worker/src/ai/audit-logger.ts @@ -4,83 +4,69 @@ // it under the terms of the GNU Affero General Public License version 3 // as published by the Free Software Foundation. -// Null Object pattern for audit logging - callers never check for null - import type { AuditSession } from '../audit/index.js'; -import { formatTimestamp } from '../utils/formatting.js'; +import { isLoggableAgentName, type LoggableAgentName, type SafeErrorDetails } from '../audit/safe-fields.js'; +/** + * Per-agent-run error audit sink. `createAuditLogger` always returns one of these + * (never null), so a caller can log unconditionally without checking whether + * audit is actually wired up for this run. + */ export interface AuditLogger { - logLlmResponse(turn: number, content: string): Promise; - logToolStart(toolName: string, parameters: unknown): Promise; - logToolEnd(result: unknown): Promise; - logError(error: Error, duration: number, turns: number): Promise; - logNote(category: string, message: string): Promise; + logError(error: SafeErrorDetails, duration: number, turns: number): Promise; + flush(): Promise; } class RealAuditLogger implements AuditLogger { - private auditSession: AuditSession; + private queue: Promise = Promise.resolve(); - constructor(auditSession: AuditSession) { - this.auditSession = auditSession; + constructor( + private readonly auditSession: AuditSession, + private readonly agentName: LoggableAgentName, + private readonly attemptNumber: number, + ) {} + + // Serializes writes onto one chain so concurrent calls append in call order rather than racing + // on the underlying audit session, and swallows failures so a broken audit write never surfaces + // as the agent's own error: recording an error must not itself risk failing the run. + private enqueue(operation: () => Promise): Promise { + this.queue = this.queue.then(operation, operation).catch(() => undefined); + return this.queue; } - async logLlmResponse(turn: number, content: string): Promise { - await this.auditSession.logEvent('llm_response', { - turn, - content, - timestamp: formatTimestamp(), - }); + logError(error: SafeErrorDetails, duration: number, turns: number): Promise { + return this.enqueue(() => + this.auditSession.logAgentError(this.agentName, error.code, error.category, this.attemptNumber, duration, turns), + ); } - async logToolStart(toolName: string, parameters: unknown): Promise { - await this.auditSession.logEvent('tool_start', { - toolName, - parameters, - timestamp: formatTimestamp(), - }); - } - - async logToolEnd(result: unknown): Promise { - await this.auditSession.logEvent('tool_end', { - result, - timestamp: formatTimestamp(), - }); - } - - async logError(error: Error, duration: number, turns: number): Promise { - await this.auditSession.logEvent('error', { - message: error.message, - errorType: error.constructor.name, - stack: error.stack, - duration, - turns, - timestamp: formatTimestamp(), - }); - } - - async logNote(category: string, message: string): Promise { - await this.auditSession.logWorkflowNote(category, message); + async flush(): Promise { + await this.queue; } } -/** Null Object implementation - all methods are safe no-ops */ +/** No-op sink for a run with no audit session or an agent name unsafe to log. */ class NullAuditLogger implements AuditLogger { - async logLlmResponse(_turn: number, _content: string): Promise {} + async logError(_error: SafeErrorDetails, _duration: number, _turns: number): Promise {} - async logToolStart(_toolName: string, _parameters: unknown): Promise {} - - async logToolEnd(_result: unknown): Promise {} - - async logError(_error: Error, _duration: number, _turns: number): Promise {} - - async logNote(_category: string, _message: string): Promise {} + async flush(): Promise {} } -// Returns no-op when auditSession is null -export function createAuditLogger(auditSession: AuditSession | null): AuditLogger { - if (auditSession) { - return new RealAuditLogger(auditSession); +/** + * Build the error-audit sink for one agent attempt. + * + * Falls back to the null sink whenever real logging can't be done safely: no + * audit session for this run, no agent name, or a name that isn't in the closed + * loggable set (`isLoggableAgentName`). An unrecognized name is never written + * to the durable audit trail, even as a bare string. + */ +export function createAuditLogger( + auditSession: AuditSession | null, + agentName: string | null, + attemptNumber: number, +): AuditLogger { + if (auditSession !== null && agentName !== null && isLoggableAgentName(agentName)) { + return new RealAuditLogger(auditSession, agentName, attemptNumber); } - return new NullAuditLogger(); } diff --git a/apps/worker/src/ai/output-formatters.ts b/apps/worker/src/ai/output-formatters.ts index 0268cc18..cc604582 100644 --- a/apps/worker/src/ai/output-formatters.ts +++ b/apps/worker/src/ai/output-formatters.ts @@ -14,6 +14,7 @@ * a direct mapping. */ +import type { SafeErrorDetails } from '../audit/safe-fields.js'; import { AGENTS } from '../session-manager.js'; import { extractAgentType, formatDuration } from '../utils/formatting.js'; import type { ExecutionContext } from './types.js'; @@ -27,7 +28,10 @@ interface ToolCallInput { [key: string]: unknown; } -/** Agent prefix used to attribute output when parallel agents interleave on one stream. */ +// Agent prefix used to attribute output when parallel agents interleave on one stream. Tries the +// registered agent's exact display name first, then falls back to a keyword match against the raw +// description, so a caller passing an ad hoc description string still gets a reasonable tag +// instead of the generic one. export function getAgentPrefix(description: string): string { const agentPrefixes: Record = { 'injection-vuln': '[Injection]', @@ -68,7 +72,9 @@ function extractDomain(url: string): string { } } -/** Format a playwright-cli command (run via the bash tool) into a clean progress indicator. */ +// Browser automation goes through the bash tool as a playwright-cli invocation, not a dedicated +// tool call, so there is no structured event to read the action from. This parses the command line +// back into a friendly one-liner instead of showing the raw shell command. function formatBrowserAction(command: string): string | null { const match = command.match(/playwright-cli\s+(?:-s=\S+\s+)?(\S+)(?:\s+(.*))?/); if (!match) return null; @@ -139,7 +145,9 @@ function formatBrowserAction(command: string): string | null { } } -/** Summarize a todo_write update into a clean progress indicator. */ +// todo_write replaces the whole list on every call, so there is no single "changed item" to +// report. Surface the most recently completed item if one exists, otherwise the item now in +// progress; a list with neither (all pending, or empty) has nothing worth printing. function summarizeTodoUpdate(input: ToolCallInput | undefined): string | null { if (!input?.todos || !Array.isArray(input.todos)) { return null; @@ -159,6 +167,15 @@ function summarizeTodoUpdate(input: ToolCallInput | undefined): string | null { return null; } +/** + * Classify a phase's console output style from its human-readable description. + * + * `isParallelExecution` marks the five concurrent vuln/exploit agents, whose output + * interleaves on one stream and so needs a per-line agent tag; `useCleanOutput` marks + * every phase that gets the friendly spinner-and-summary treatment instead of the + * verbose turn-by-turn fallback. Matching is on substrings of `description`, the same + * strings the activity layer passes as the human-facing phase label. + */ export function detectExecutionContext(description: string): ExecutionContext { const isParallelExecution = description.includes('vuln agent') || description.includes('exploit agent'); @@ -236,36 +253,28 @@ export function formatToolCall( } export function formatErrorOutput( - error: Error & { code?: string; status?: number }, + error: SafeErrorDetails, context: ExecutionContext, - description: string, duration: number, - sourceDir: string, + turns: number, isRetryable: boolean, ): string[] { const lines: string[] = []; if (context.isParallelExecution) { - lines.push(`${getAgentPrefix(description)} Failed (${formatDuration(duration)})`); + lines.push(`Agent failed (${formatDuration(duration)})`); } else if (context.useCleanOutput) { lines.push(`${context.agentType} failed (${formatDuration(duration)})`); } else { - lines.push(` pi agent failed: ${description} (${formatDuration(duration)})`); + lines.push(` Agent failed (${formatDuration(duration)})`); } - lines.push(` Error Type: ${error.constructor.name}`); + lines.push(` Error Code: ${error.code}`); + lines.push(` Category: ${error.category}`); lines.push(` Message: ${error.message}`); - lines.push(` Agent: ${description}`); - lines.push(` Working Directory: ${sourceDir}`); + lines.push(` Turns: ${turns}`); lines.push(` Retryable: ${isRetryable ? 'Yes' : 'No'}`); - if (error.code) { - lines.push(` Error Code: ${error.code}`); - } - if (error.status) { - lines.push(` HTTP Status: ${error.status}`); - } - return lines; } diff --git a/apps/worker/src/ai/pi/capella-agent-executor.ts b/apps/worker/src/ai/pi/capella-agent-executor.ts index cf0feb16..02cb0697 100644 --- a/apps/worker/src/ai/pi/capella-agent-executor.ts +++ b/apps/worker/src/ai/pi/capella-agent-executor.ts @@ -18,6 +18,7 @@ import { } from '@earendil-works/pi-coding-agent'; import type { TSchema } from 'typebox'; import { Value } from 'typebox/value'; +import { captureToolInvocation, decideToolOutcome } from '../../audit/trace.js'; import type { ProviderFailureCategory } from '../../types/errors.js'; import { type ModelHost, modelHost } from '../model-host.js'; import type { ModelSelection } from '../models.js'; @@ -433,13 +434,30 @@ class StandaloneCapellaAgentExecutor implements CapellaAgentExecutor { let invalidSubmission = false; let pendingProviderError: unknown; + // Per-session trace correlation lives here in the executor; the injected sink is a + // stateless emitter, safe to share across the stage's sessions. + const traceLog = request.log; + const pendingTrace = new Map(); unsubscribe = session.subscribe((event: AgentSessionEvent) => { if (event.type === 'tool_execution_start') { operationCount += 1; + if (traceLog !== undefined) { + const invocation = captureToolInvocation(event.toolName, event.args); + pendingTrace.set(event.toolCallId, { tool: event.toolName, startedAt: Date.now() }); + if (invocation !== undefined) traceLog.toolCall(invocation); + } return; } if (event.type === 'tool_execution_end') { if (event.toolName === 'submit_result' && event.isError) invalidSubmission = true; + if (traceLog !== undefined) { + const pending = pendingTrace.get(event.toolCallId); + if (pending !== undefined) { + pendingTrace.delete(event.toolCallId); + const outcome = decideToolOutcome(pending.tool, event.isError, Date.now() - pending.startedAt, undefined); + if (outcome !== undefined) traceLog.toolOutcome(outcome); + } + } return; } if (event.type !== 'turn_end') return; @@ -455,6 +473,7 @@ class StandaloneCapellaAgentExecutor implements CapellaAgentExecutor { } }); + const runStartedAt = Date.now(); let promptError: unknown; try { await raceWithAbort(session.prompt(request.userPrompt, { expandPromptTemplates: false }), controller.signal); @@ -472,6 +491,12 @@ class StandaloneCapellaAgentExecutor implements CapellaAgentExecutor { }; const output = this.resolveOutcome(request, outcome, termination, selection.model.contextWindow); + // Emitted only past resolveOutcome so a failed, cancelled, timed-out, or turn-capped + // session (all of which throw above) never reports a truthful-looking completion. + if (traceLog !== undefined) { + traceLog.sessionComplete(Date.now() - runStartedAt, turnCount, operationCount); + } + return { output, usage: outcome.usage }; } catch (error) { const surfacedError = normalizeRunFailure(error, termination, request.signal, this.host); diff --git a/apps/worker/src/ai/pi/capella-agent-types.ts b/apps/worker/src/ai/pi/capella-agent-types.ts index 1e99d3c6..2f013317 100644 --- a/apps/worker/src/ai/pi/capella-agent-types.ts +++ b/apps/worker/src/ai/pi/capella-agent-types.ts @@ -6,12 +6,34 @@ import type { ToolDefinition } from '@earendil-works/pi-coding-agent'; import type { TSchema } from 'typebox'; +import type { ToolInvocation, ToolOutcome } from '../../audit/trace.js'; import type { ModelRole } from '../model-host.js'; import type { CapellaStage, CapellaUsage } from '../sast/types.js'; /** A Capella-owned collector or repository tool installed in one confined session. */ export type CapellaTool = ToolDefinition; +/** + * A sink for one Capella session's technical trace. The executor owns `toolCallId` + * correlation and synchronously snapshots complete tool arguments before handing the + * immutable invocation to the sink. + */ +export interface CapellaTraceLog { + toolCall(invocation: ToolInvocation): void; + toolOutcome(outcome: ToolOutcome): void; + sessionComplete(durationMs: number, turns: number, operations: number): void; +} + +/** + * One stage's trace surface. `forSession` binds a per-session view (its label becomes the trace + * prefix's session component); all views share one serialized queue that `drain` awaits, so no + * session's lines can still be buffered when its activity returns. + */ +export interface CapellaStageTrace { + forSession(sessionLabel: string | undefined): CapellaTraceLog; + drain(): Promise; +} + /** One bounded multi-turn Capella model session. */ export interface CapellaAgentRequest<_T> { readonly stage: CapellaStage; @@ -24,6 +46,12 @@ export interface CapellaAgentRequest<_T> { readonly tools: readonly CapellaTool[]; readonly outputSchema?: TSchema; readonly signal: AbortSignal; + readonly log?: CapellaTraceLog; + /** + * Display-only session name for the trace prefix. Never hashed into `workloadId`, a checkpoint + * key, a usage record, or a prompt; a stage may repeat or omit it without changing execution. + */ + readonly sessionLabel?: string; } /** Schema-valid output and measured usage from one completed Capella session. */ diff --git a/apps/worker/src/ai/pi/pi-executor.ts b/apps/worker/src/ai/pi/pi-executor.ts index f38c68ce..483f7049 100644 --- a/apps/worker/src/ai/pi/pi-executor.ts +++ b/apps/worker/src/ai/pi/pi-executor.ts @@ -5,6 +5,9 @@ // as published by the Free Software Foundation. // Production agent execution on the pi harness, with git checkpoints and audit logging. +// The checkpoint itself is created by the caller (AgentExecutionService) before and after +// runPiPrompt runs; this module owns the session, its audit/error logging, and the trace it +// produces, not the git commit around it. import os from 'node:os'; import type { AgentMessage } from '@earendil-works/pi-agent-core'; @@ -22,6 +25,7 @@ import { } from '@earendil-works/pi-coding-agent'; import { fs, path } from 'zx'; import type { AuditSession } from '../../audit/index.js'; +import { isLoggableAgentName, type SafeErrorDetails, safeErrorFromUnknown } from '../../audit/safe-fields.js'; import { BASH_TIMEOUT_EXTENSION_DIR, deliverablesDir } from '../../paths.js'; import { isRetryableFailure, PentestError } from '../../services/error-handling.js'; import { AGENT_VALIDATORS } from '../../session-manager.js'; @@ -44,6 +48,7 @@ import { permissionSystemConfigExists, permissionSystemPackageDir } from './perm import { PI_RETRY_SETTINGS } from './retry-settings.js'; import { createGlobTool, createTodoWriteTool } from './session-tools.js'; import { createTaskTool } from './task-tool.js'; +import { TraceEmitter } from './trace-emitter.js'; import { providerTurnError } from './turn-error.js'; declare global { @@ -142,7 +147,6 @@ export interface PiPromptResult { model?: string | undefined; error?: string | undefined; errorType?: string | undefined; - prompt?: string | undefined; retryable?: boolean | undefined; structuredOutput?: unknown; } @@ -154,18 +158,20 @@ function outputLines(lines: string[]): void { } async function writeErrorLog( - err: Error & { code?: string; status?: number }, sourceDir: string, - fullPrompt: string, + error: SafeErrorDetails, duration: number, + turns: number, + retryable: boolean, ): Promise { try { const errorLog = { timestamp: formatTimestamp(), agent: 'pi-executor', - error: { name: err.constructor.name, message: err.message, code: err.code, status: err.status, stack: err.stack }, - context: { sourceDir, prompt: `${fullPrompt.slice(0, 200)}...`, retryable: isRetryableFailure(err) }, + error: { code: error.code, category: error.category, message: error.message }, duration, + turns, + retryable, }; const logPath = path.join(deliverablesDir(sourceDir), 'error.log'); await fs.appendFile(logPath, `${JSON.stringify(errorLog)}\n`); @@ -186,6 +192,9 @@ export async function validateAgentOutput( logger.error('Validation failed: Agent execution was unsuccessful'); return false; } + // Not every agent has a deliverable-structure validator registered. Absence is not treated as + // a failure: the agent already reported success above, so an agent with no validator passes on + // that alone rather than being held to a check that was never defined for it. const validator = agentName ? AGENT_VALIDATORS[agentName as keyof typeof AGENT_VALIDATORS] : undefined; if (!validator) { logger.warn(`No validator found for agent "${agentName}" - assuming success`); @@ -230,6 +239,7 @@ export async function runPiPrompt( deliverablesSubdir?: string, cancellationSignal?: AbortSignal, submitTool?: CapturedSubmitTool, + attemptNumber: number = 1, ): Promise { // 1. Initialize timing and prompt. A submit tool appends its directive so the // instruction to call it lives with the tool, not in every prompt file. @@ -243,7 +253,7 @@ export async function runPiPrompt( { description, useCleanOutput: execContext.useCleanOutput }, global.SHANNON_DISABLE_LOADER ?? false, ); - const auditLogger = createAuditLogger(auditSession); + const auditLogger = createAuditLogger(auditSession, agentName, attemptNumber); logger.info(`Running pi agent: ${description}...`); @@ -259,6 +269,14 @@ export async function runPiPrompt( // plus any caller-supplied collector/submit tools). const selection = await resolveModelSelection(); const resourceLoader = await buildResourceLoader(sourceDir, logger, agentName); + const agentNameCandidate = agentName ?? ''; + const parentAgentName = isLoggableAgentName(agentNameCandidate) ? agentNameCandidate : 'pre-recon'; + // The durable trace log is path-addressed, so parent, child, and Capella writers all + // reach the same file without sharing a stream handle. + const workflowLogPath = auditSession?.workflowLogPath; + const traceEmitter = workflowLogPath + ? new TraceEmitter(workflowLogPath, { kind: 'agent', agent: parentAgentName }) + : undefined; // Accumulates usage from in-process `task` child sessions so the parent's reported // cost includes sub-agent spend (their getSessionStats is separate from ours). const childUsage: ChildUsage = { cost: 0, inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0 }; @@ -267,6 +285,11 @@ export async function runPiPrompt( model: selection.model, modelRuntime: selection.modelRuntime, cwd: sourceDir, + parentAgentName, + ...(workflowLogPath !== undefined && { workflowLogPath }), + ...(traceEmitter !== undefined && { + onDelegationStart: (child: string) => traceEmitter.delegationStart(child), + }), onUsage: (usage) => { childUsage.cost += usage.cost; childUsage.inputTokens += usage.inputTokens; @@ -277,7 +300,7 @@ export async function runPiPrompt( resourceLoader, ...(cancellationSignal && { cancellationSignal }), }), - createTodoWriteTool(auditLogger), + createTodoWriteTool(), createGlobTool(sourceDir), ...(callerTools ?? []), ...(submitTool ? [submitTool.tool] : []), @@ -330,7 +353,6 @@ export async function runPiPrompt( const msg = event.message; const text = extractAssistantText(msg); if (text.trim()) { - void auditLogger.logLlmResponse(turnCount, text); progress.stop(); outputLines(formatAssistantOutput(text, execContext, turnCount, description)); progress.start(); @@ -341,7 +363,8 @@ export async function runPiPrompt( break; } case 'tool_execution_start': { - void auditLogger.logToolStart(event.toolName, event.args); + const count = submitTool?.tool.name === event.toolName ? submitTool.safeCount : undefined; + traceEmitter?.toolStart(event.toolCallId, event.toolName, event.args, count); const toolLines = formatToolCall( event.toolName, event.args as Record, @@ -355,9 +378,10 @@ export async function runPiPrompt( } break; } - case 'tool_execution_end': - void auditLogger.logToolEnd(event.result); + case 'tool_execution_end': { + traceEmitter?.toolEnd(event.toolCallId, event.isError); break; + } case 'compaction_end': if (!event.aborted && !event.willRetry && event.errorMessage) { pendingError = @@ -387,6 +411,8 @@ export async function runPiPrompt( // Capture the submit tool's structured payload so callers read it off the // result instead of holding a reference to the tool. const structuredOutput = submitTool?.getCaptured(); + await auditLogger.flush(); + await traceEmitter?.flush(); return { result, @@ -402,13 +428,17 @@ export async function runPiPrompt( ...(structuredOutput !== undefined && { structuredOutput }), }; } catch (error) { - // 10. Handle errors — log, write error file, return failure + // 9. Handle errors: log, write error file, return failure const duration = timer.stop(); const err = error as Error & { code?: string; status?: number }; - await auditLogger.logError(err, duration, turnCount); + const safeError = safeErrorFromUnknown(err); + const retryable = isRetryableFailure(err); + await auditLogger.logError(safeError, duration, turnCount); + await auditLogger.flush(); + await traceEmitter?.flush(); progress.stop(); - outputLines(formatErrorOutput(err, execContext, description, duration, sourceDir, isRetryableFailure(err))); - await writeErrorLog(err, sourceDir, fullPrompt, duration); + outputLines(formatErrorOutput(safeError, execContext, duration, turnCount, retryable)); + await writeErrorLog(sourceDir, safeError, duration, turnCount, retryable); // A failed agent still spent money — on its own turns and, since Shannon's // prompts delegate the heavy work, mostly on `task` sub-agents. Both count @@ -416,9 +446,8 @@ export async function runPiPrompt( const usage = totalUsage(session, childUsage); return { - error: err.message, - errorType: err instanceof PentestError && err.code ? err.code : err.constructor.name, - prompt: `${fullPrompt.slice(0, 100)}...`, + error: safeError.message, + errorType: safeError.code, success: false, duration, turns: turnCount, @@ -427,7 +456,7 @@ export async function runPiPrompt( outputTokens: usage.outputTokens, cacheReadTokens: usage.cacheReadTokens, cacheWriteTokens: usage.cacheWriteTokens, - retryable: isRetryableFailure(err), + retryable, }; } finally { cancellationSignal?.removeEventListener('abort', onCancellation); diff --git a/apps/worker/src/ai/pi/session-tools.ts b/apps/worker/src/ai/pi/session-tools.ts index 9f77c7f3..b5f640f5 100644 --- a/apps/worker/src/ai/pi/session-tools.ts +++ b/apps/worker/src/ai/pi/session-tools.ts @@ -8,32 +8,21 @@ * Per-session custom tools registered for every agent: `todo_write` and `glob`. * * These replace harness built-ins that pi does not ship. `todo_write` is a - * full-state-replace planning scratchpad mirrored to the workflow log; `glob` is - * fast-glob file matching (pi has no `Glob` built-in). + * full-state-replace planning scratchpad; `glob` is fast-glob file matching + * (pi has no `Glob` built-in). */ import { defineTool, type ToolDefinition } from '@earendil-works/pi-coding-agent'; import { Type } from 'typebox'; import { fs, glob, path } from 'zx'; -import type { AuditLogger } from '../audit-logger.js'; - export interface TodoItem { content: string; status: 'pending' | 'in_progress' | 'completed'; activeForm: string; } -function renderTodos(todos: readonly TodoItem[]): string { - const mark = (status: TodoItem['status']): string => { - if (status === 'completed') return 'x'; - if (status === 'in_progress') return '~'; - return ' '; - }; - return todos.map((todo) => `[${mark(todo.status)}] ${todo.content}`).join(' '); -} - -export function createTodoWriteTool(auditLogger: AuditLogger): ToolDefinition { +export function createTodoWriteTool(): ToolDefinition { let current: TodoItem[] = []; return defineTool({ @@ -56,7 +45,6 @@ export function createTodoWriteTool(auditLogger: AuditLogger): ToolDefinition { async execute(_toolCallId, params) { current = params.todos as TodoItem[]; const completed = current.filter((todo) => todo.status === 'completed').length; - await auditLogger.logNote('todo', renderTodos(current)); return { content: [ { diff --git a/apps/worker/src/ai/pi/task-tool.ts b/apps/worker/src/ai/pi/task-tool.ts index 171979e6..eec3b56e 100644 --- a/apps/worker/src/ai/pi/task-tool.ts +++ b/apps/worker/src/ai/pi/task-tool.ts @@ -4,17 +4,7 @@ // it under the terms of the GNU Affero General Public License version 3 // as published by the Free Software Foundation. -/** - * Generic `task` tool — pi.dev ships no built-in Task tool, so this supplies the - * Task-delegation surface Shannon's prompts require. - * - * Shannon's prompts mandate Task delegation (recon source tracer; the vuln - * agents delegate *every* code review; the exploit agents delegate automation), - * so this tool is required for parity, not optional. It spawns a nested pi - * session with the parent's resolved model object (never a tier string — that - * would route sub-agents through hardcoded IDs and leak billing), the parent's - * resource loader, and a fixed child tool surface. - */ +/** Generic child-session delegation for the pi harness. */ import { type AssistantMessage, type Model, Type } from '@earendil-works/pi-ai'; import { @@ -27,39 +17,70 @@ import { SettingsManager, type ToolDefinition, } from '@earendil-works/pi-coding-agent'; +import { type LoggableAgentName, normalizeSemanticLabel } from '../../audit/safe-fields.js'; import { PI_RETRY_SETTINGS } from './retry-settings.js'; +import { TraceEmitter } from './trace-emitter.js'; export interface TaskToolContext { - cwd: string; + readonly cwd: string; // eslint-disable-next-line @typescript-eslint/no-explicit-any - model: Model; - /** Parent's model/auth runtime, reused so sub-agents share its resolved credential. */ - modelRuntime: ModelRuntime; - resourceLoader: ResourceLoader; - cancellationSignal?: AbortSignal | undefined; - /** - * Reports the cost/tokens of each spawned sub-session back to the caller. - * Sub-agents run in their own pi sessions that the parent has no reference to, - * so without this their spend (the bulk of a whitebox run, since Shannon - * prompts delegate the heavy work) is invisible to billing. - */ - onUsage?: (usage: { - cost: number; - inputTokens: number; - outputTokens: number; - cacheReadTokens: number; - cacheWriteTokens: number; + readonly model: Model; + readonly modelRuntime: ModelRuntime; + readonly resourceLoader: ResourceLoader; + readonly parentAgentName: LoggableAgentName; + readonly workflowLogPath?: string | undefined; + readonly onDelegationStart?: ((child: string) => Promise) | undefined; + readonly cancellationSignal?: AbortSignal | undefined; + readonly onUsage?: (usage: { + readonly cost: number; + readonly inputTokens: number; + readonly outputTokens: number; + readonly cacheReadTokens: number; + readonly cacheWriteTokens: number; }) => void; } +// Deliberately excludes `task` (no recursive delegation, so a child cannot spawn further children) +// and every collector/submit tool (structured output stays owned by the top-level agent session +// that the workflow reads back). A child session gets only plain file and shell access. const CHILD_TOOLS = ['read', 'grep', 'find', 'ls', 'write', 'bash']; +const CHILD_FAILURE_TEXT = '[Sub-agent task failed before completion]'; +const CHILD_CANCELLED_TEXT = '[Sub-agent task was cancelled]'; function textResult(text: string) { return { content: [{ type: 'text' as const, text }], details: undefined }; } +/** + * Assigns each child a stable, safe display identity from its description. A duplicate of a + * live sibling's name gets a monotonic start-order suffix (`route mapper #2`); a missing or + * unsafe description becomes `subagent N`. State is shared across one parent's task calls, + * and the assignment block runs synchronously so parallel calls never race on it. + */ +// Keep the base short enough that a `#N` suffix still fits the identity validator's length +// bound (48); a longer description falls back to `subagent N` rather than being dropped. +const MAX_CHILD_BASE_LENGTH = 40; + +function createChildNamer(): (description: unknown) => string { + const namedCounts = new Map(); + let anonymousCount = 0; + return (description) => { + const base = normalizeSemanticLabel(description); + if (base === undefined || base.length > MAX_CHILD_BASE_LENGTH) { + anonymousCount += 1; + return `subagent ${anonymousCount}`; + } + const nextOrdinal = (namedCounts.get(base) ?? 0) + 1; + namedCounts.set(base, nextOrdinal); + return nextOrdinal === 1 ? base : `${base} #${nextOrdinal}`; + }; +} + export function createTaskTool(config: TaskToolContext): ToolDefinition { - const taskTool: ToolDefinition = defineTool({ + const nameChild = createChildNamer(); + const logPath = config.workflowLogPath; + + return defineTool({ name: 'task', label: 'Task', description: @@ -80,59 +101,82 @@ export function createTaskTool(config: TaskToolContext): ToolDefinition { description: Type.Optional(Type.String({ description: 'A short (3-5 word) description of the task.' })), }), async execute(_toolCallId, params) { + // Assign the identity synchronously, before any await, so concurrent siblings can't race. + const child = nameChild(params.description); + const emitter = logPath + ? new TraceEmitter(logPath, { kind: 'child', parent: config.parentAgentName, child }) + : undefined; + const startedAt = Date.now(); + + // The parent's emitter first writes the raw task invocation, then this delegation + // record. Awaiting it prevents the child emitter from overtaking its lineage start. + await config.onDelegationStart?.(child); + const agentDir = getAgentDir(); - const { session: subSession } = await createAgentSession({ - cwd: config.cwd, - agentDir, - resourceLoader: config.resourceLoader, - model: config.model, - tools: CHILD_TOOLS, - modelRuntime: config.modelRuntime, - sessionManager: SessionManager.inMemory(config.cwd), - settingsManager: SettingsManager.inMemory({ - retry: PI_RETRY_SETTINGS, - compaction: { enabled: true }, - }), - }); + let subSession: Awaited>['session'] | undefined; + let resultText = ''; + let subCost = 0; + let turns = 0; + let operations = 0; + let failed = false; + let fatalFailure = false; const abortChildSession = (): void => { - void subSession.abort().catch(() => { - // Parent logger is not available inside the tool; dispose still tears - // down the session if abort itself rejects. + void subSession?.abort().catch(() => { + // Dispose below still tears down the child session. }); }; const onCancellation = (): void => abortChildSession(); - if (config.cancellationSignal?.aborted) { - abortChildSession(); - } else { - config.cancellationSignal?.addEventListener('abort', onCancellation, { once: true }); - } - let resultText = ''; - let subCost = 0; - subSession.subscribe((event) => { - if (event.type === 'turn_end') { - const msg = event.message as AssistantMessage | undefined; - for (const block of msg?.content ?? []) { + try { + ({ session: subSession } = await createAgentSession({ + cwd: config.cwd, + agentDir, + resourceLoader: config.resourceLoader, + model: config.model, + tools: CHILD_TOOLS, + modelRuntime: config.modelRuntime, + sessionManager: SessionManager.inMemory(config.cwd), + settingsManager: SettingsManager.inMemory({ + retry: PI_RETRY_SETTINGS, + compaction: { enabled: true }, + }), + })); + + if (config.cancellationSignal?.aborted) { + abortChildSession(); + } else { + config.cancellationSignal?.addEventListener('abort', onCancellation, { once: true }); + } + + subSession.subscribe((event) => { + if (event.type === 'tool_execution_start') { + operations += 1; + emitter?.toolStart(event.toolCallId, event.toolName, event.args); + return; + } + if (event.type === 'tool_execution_end') { + emitter?.toolEnd(event.toolCallId, event.isError); + return; + } + if (event.type !== 'turn_end') return; + turns += 1; + const message = event.message as AssistantMessage | undefined; + for (const block of message?.content ?? []) { if (block.type === 'text' && block.text) { resultText += (resultText ? '\n' : '') + block.text; } } - if (msg?.usage?.cost?.total != null) subCost += msg.usage.cost.total; - } - }); + if (message?.usage?.cost?.total != null) subCost += message.usage.cost.total; + }); - let swallowedError: string | undefined; - try { try { await subSession.prompt(params.prompt); - } catch (err) { - const errorMsg = err instanceof Error ? err.message : String(err); - resultText += `\n[Sub-agent error: ${errorMsg}]`; + } catch { + failed = true; } + if (subSession.state.errorMessage !== undefined) failed = true; - swallowedError = subSession.state.errorMessage; - // Read stats before dispose; reconcile cost the same way the parent does. const subStats = subSession.getSessionStats(); if (subStats.cost > subCost) subCost = subStats.cost; config.onUsage?.({ @@ -142,18 +186,37 @@ export function createTaskTool(config: TaskToolContext): ToolDefinition { cacheReadTokens: subStats.tokens.cacheRead, cacheWriteTokens: subStats.tokens.cacheWrite, }); + } catch { + fatalFailure = true; } finally { config.cancellationSignal?.removeEventListener('abort', onCancellation); - subSession.dispose(); + subSession?.dispose(); } - if (swallowedError && !resultText.includes(swallowedError)) { - resultText += `\n[Sub-agent error: ${swallowedError}]`; + const durationMs = Date.now() - startedAt; + if (config.cancellationSignal?.aborted) { + emitter?.sessionFailure('CANCELLED', durationMs); + await emitter?.flush(); + return textResult(CHILD_CANCELLED_TEXT); + } + // `fatalFailure` means the child session itself never came up (createAgentSession threw), so + // there is no session result to hand back, and this rethrows, which pi surfaces to the parent + // as a failed tool call. `failed` means the session ran but ended in error; that gets a normal + // text result instead, so the parent model sees the failure and can decide how to proceed. + if (fatalFailure) { + emitter?.sessionFailure('CHILD_TASK_FAILED', durationMs); + await emitter?.flush(); + throw new Error(CHILD_FAILURE_TEXT); + } + if (failed) { + emitter?.sessionFailure('CHILD_TASK_FAILED', durationMs); + await emitter?.flush(); + return textResult(CHILD_FAILURE_TEXT); } + emitter?.sessionComplete(durationMs, turns, operations); + await emitter?.flush(); return textResult(resultText || '[Sub-agent produced no output]'); }, }); - - return taskTool; } diff --git a/apps/worker/src/ai/pi/trace-emitter.ts b/apps/worker/src/ai/pi/trace-emitter.ts new file mode 100644 index 00000000..48b011a2 --- /dev/null +++ b/apps/worker/src/ai/pi/trace-emitter.ts @@ -0,0 +1,78 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** + * Per-session trace emitter. Owns the PI `toolCallId` correlation and the ordering + * of one agent or subagent's trace lines, then writes them through the stateless + * `WorkflowLogger` formatter. One instance per parent agent run or per delegated + * child session, so parallel calls never cross. + */ + +import { captureToolInvocation, decideToolOutcome } from '../../audit/trace.js'; +import { type ChildTaskFailureCode, type TraceActor, WorkflowLogger } from '../../audit/workflow-logger.js'; + +interface PendingCall { + readonly tool: string; + readonly startedAt: number; + readonly count?: (() => number | undefined) | undefined; +} + +export class TraceEmitter { + private queue: Promise = Promise.resolve(); + private readonly pending = new Map(); + + constructor( + private readonly logPath: string, + private readonly actor: TraceActor, + private readonly now: () => number = Date.now, + ) {} + + /** + * Snapshot and log a tool call's complete arguments. `count`, when supplied, is an + * accessor for that specific collector's existing submitted-array count outcome. + */ + toolStart(toolCallId: string, toolName: string, args: unknown, count?: () => number | undefined): void { + const invocation = captureToolInvocation(toolName, args); + this.pending.set(toolCallId, { tool: toolName, startedAt: this.now(), count }); + if (invocation !== undefined) this.enqueue(() => WorkflowLogger.logToolCall(this.logPath, this.actor, invocation)); + } + + toolEnd(toolCallId: string, isError: boolean): void { + const call = this.pending.get(toolCallId); + if (call === undefined) return; + this.pending.delete(toolCallId); + const outcome = decideToolOutcome(call.tool, isError, this.now() - call.startedAt, call.count?.()); + if (outcome !== undefined) this.enqueue(() => WorkflowLogger.logToolOutcome(this.logPath, this.actor, outcome)); + } + + /** Queue and await delegation on the parent emitter before a child session can start. */ + delegationStart(child: string): Promise { + const actor = this.actor; + if (actor.kind !== 'agent') return Promise.resolve(); + return this.enqueue(() => WorkflowLogger.logDelegationStart(this.logPath, actor.agent, child)); + } + + sessionComplete(durationMs: number, turns: number, operations: number): void { + this.enqueue(() => WorkflowLogger.logSessionComplete(this.logPath, this.actor, durationMs, turns, operations)); + } + + sessionFailure(code: ChildTaskFailureCode, durationMs: number): void { + this.enqueue(() => WorkflowLogger.logSessionFailure(this.logPath, this.actor, code, durationMs)); + } + + // Chained regardless of outcome (`then(operation, operation)`) so one write's rejection cannot + // stall the ones queued after it, and the trailing catch swallows the failure entirely: a trace + // line is diagnostic only, so losing one must never surface as, or block, the agent's own result. + private enqueue(operation: () => Promise): Promise { + this.queue = this.queue.then(operation, operation).catch(() => undefined); + return this.queue; + } + + /** Await all queued writes so a caller can order a terminal line after them. */ + async flush(): Promise { + await this.queue; + } +} diff --git a/apps/worker/src/ai/queue-schemas.ts b/apps/worker/src/ai/queue-schemas.ts index 3211afa2..062c2e25 100644 --- a/apps/worker/src/ai/queue-schemas.ts +++ b/apps/worker/src/ai/queue-schemas.ts @@ -337,6 +337,10 @@ export function createQueueSubmitTool(agentName: AgentName, exploit = true): Cap }, }), getCaptured: () => captured, + safeCount: () => { + const vulnerabilities = (captured as { vulnerabilities?: unknown } | undefined)?.vulnerabilities; + return Array.isArray(vulnerabilities) ? vulnerabilities.length : undefined; + }, directive: '\n\nYou MUST call the submit_exploitation_queue tool exactly once as your final action ' + 'to deliver your structured exploitation queue. Do not output JSON as text. Fill every required parameter.', diff --git a/apps/worker/src/ai/sast/capella/session-label.ts b/apps/worker/src/ai/sast/capella/session-label.ts new file mode 100644 index 00000000..c1331f89 --- /dev/null +++ b/apps/worker/src/ai/sast/capella/session-label.ts @@ -0,0 +1,35 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Display-only session labels for a Capella stage's concurrent sessions. */ + +import { normalizeSemanticLabel } from '../../../audit/safe-fields.js'; + +// Keep the base short enough that a ` #N` suffix still fits the identity validator's 48-char +// bound; a longer title falls back to ` N` rather than producing an unsafe label. +const MAX_SESSION_BASE_LENGTH = 40; + +/** + * Build a per-stage session labeler. It normalizes a free-text title to a safe display label and + * disambiguates same-title siblings with `#2`/`#3`, exactly as the subagent namer does; a title + * that cannot be normalized falls back to ` N`. Call it synchronously at dispatch, before + * any await, so concurrent siblings never race on the ordinal. Labels are not stable across a + * resume, which is acceptable for a human-facing log. + */ +export function createCapellaSessionNamer(fallback: string): (title: unknown) => string { + const namedCounts = new Map(); + let anonymousCount = 0; + return (title) => { + const base = normalizeSemanticLabel(title); + if (base === undefined || base.length > MAX_SESSION_BASE_LENGTH) { + anonymousCount += 1; + return `${fallback} ${anonymousCount}`; + } + const nextOrdinal = (namedCounts.get(base) ?? 0) + 1; + namedCounts.set(base, nextOrdinal); + return nextOrdinal === 1 ? base : `${base} #${nextOrdinal}`; + }; +} diff --git a/apps/worker/src/ai/sast/capella/stages/research.ts b/apps/worker/src/ai/sast/capella/stages/research.ts index d3adc954..4c2814de 100644 --- a/apps/worker/src/ai/sast/capella/stages/research.ts +++ b/apps/worker/src/ai/sast/capella/stages/research.ts @@ -20,6 +20,7 @@ import type { CapellaFinding } from '../finding-types.js'; import { buildCodePathScopeSnippet, buildResearchAssignment, RESEARCH_TOOLS, TRIAGE_TOOLS } from '../prompt-context.js'; import { createCapellaPromptLoader } from '../prompt-loader.js'; import { type Investigation, TRIAGE_SCHEMA, type TriageResult } from '../schemas.js'; +import { createCapellaSessionNamer } from '../session-label.js'; import { CAPELLA_AUDIT_CONCURRENCY, CAPELLA_TRIAGE_CONCURRENCY, @@ -325,7 +326,10 @@ export async function runResearchStage( batchId: buildFingerprint({ files }).slice(0, 20), })); - const triageOutcomes = await runSettledPool(batches, CAPELLA_TRIAGE_CONCURRENCY, async (batch) => { + const triageOutcomes = await runSettledPool(batches, CAPELLA_TRIAGE_CONCURRENCY, async (batch, index) => { + // The label is display-only; it is derived from the dispatch index and kept out of the batch + // and the checkpoint fingerprint, which must stay keyed on the batch content alone. + const sessionLabel = `triage ${index + 1}`; const checkpointPath = resolve(input.artifactRoot, 'research', 'triage', `${batch.batchId}.json`); const checkpointFingerprint = buildFingerprint({ researchFingerprint: fingerprint, wave: 'triage', ...batch }); const cached = await loadCompletedArtifact( @@ -349,6 +353,7 @@ export async function runResearchStage( tools: runtime.repositoryTools, outputSchema: Type.Unsafe(TRIAGE_SCHEMA), signal: runtime.signal, + sessionLabel, }); let usage = primaryResponse.usage; const primaryIsValid = isTriageResult(primaryResponse.output); @@ -373,6 +378,7 @@ export async function runResearchStage( tools: runtime.repositoryTools, outputSchema: Type.Unsafe(TRIAGE_SCHEMA), signal: runtime.signal, + sessionLabel: `${sessionLabel} repair`, }); if (isTriageResult(repairResponse.output)) { const repaired = usableClassifications(missingFiles, repairResponse.output.classifications); @@ -422,7 +428,10 @@ export async function runResearchStage( })) .filter((audit) => audit.flaggedFiles.length > 0); + const nameAuditSession = createCapellaSessionNamer('audit'); const auditOutcomes = await runSettledPool(audits, CAPELLA_AUDIT_CONCURRENCY, async (audit) => { + // Assign the label synchronously, before any await, so concurrent siblings cannot race. + const sessionLabel = nameAuditSession(audit.investigation.title); const checkpointPath = resolve(input.artifactRoot, 'research', 'audit', `${audit.investigationId}.json`); const checkpointFingerprint = buildFingerprint({ researchFingerprint: fingerprint, @@ -460,6 +469,7 @@ export async function runResearchStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel, }), () => collector.getFindings().length, ); diff --git a/apps/worker/src/ai/sast/capella/stages/verdicts.ts b/apps/worker/src/ai/sast/capella/stages/verdicts.ts index c5518a42..bcf99820 100644 --- a/apps/worker/src/ai/sast/capella/stages/verdicts.ts +++ b/apps/worker/src/ai/sast/capella/stages/verdicts.ts @@ -203,6 +203,7 @@ export async function runDedupeStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'primary', }), () => collector.getDuplicates().length, ); @@ -260,6 +261,7 @@ export async function runReviewStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'primary', }), () => collector.getAcceptedIds().length, ); @@ -279,6 +281,7 @@ export async function runReviewStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'repair', }), () => collector.getAcceptedIds().length, ); @@ -363,6 +366,7 @@ export async function runCriticStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'primary', }), () => collector.getAcceptedIds().length, ); @@ -388,6 +392,7 @@ export async function runCriticStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'repair', }), () => collector.getAcceptedIds().length, ); @@ -456,6 +461,7 @@ export async function runConfirmStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'primary', }), () => collector.getAcceptedIds().length, ); @@ -475,6 +481,7 @@ export async function runConfirmStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'repair', }), () => collector.getAcceptedIds().length, ); @@ -552,6 +559,7 @@ export async function runCalibrateStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'primary', }), () => collector.getAcceptedIds().length, ); @@ -577,6 +585,7 @@ export async function runCalibrateStage( timeoutMs: input.timeoutMs, tools: [...runtime.repositoryTools, ...collector.tools], signal: runtime.signal, + sessionLabel: 'repair', }), () => collector.getAcceptedIds().length, ); diff --git a/apps/worker/src/ai/sast/capella/temporal/activities.ts b/apps/worker/src/ai/sast/capella/temporal/activities.ts index 9b657ab8..19d5e976 100644 --- a/apps/worker/src/ai/sast/capella/temporal/activities.ts +++ b/apps/worker/src/ai/sast/capella/temporal/activities.ts @@ -9,11 +9,15 @@ import type { Dirent } from 'node:fs'; import { mkdir, open, readdir, readFile, realpath } from 'node:fs/promises'; import { basename, resolve } from 'node:path'; import { ApplicationFailure, CancelledFailure, Context, heartbeat } from '@temporalio/activity'; +import type { LogStream } from '../../../../audit/log-stream.js'; +import { WorkflowLogger } from '../../../../audit/workflow-logger.js'; import { CapellaAgentError, capellaAgentExecutor } from '../../../pi/capella-agent-executor.js'; import type { CapellaAgentExecutor, CapellaAgentRequest, CapellaAgentResponse, + CapellaStageTrace, + CapellaTraceLog, } from '../../../pi/capella-agent-types.js'; import type { CapellaStage, CapellaUsage } from '../../types.js'; import { @@ -67,6 +71,7 @@ import { type CapellaThreatModelActivityInput, type CapellaThreatModelActivityResult, } from './activity-types.js'; +import { createCapellaStageTrace } from './stage-trace.js'; // Must stay well under the smallest policy heartbeatTimeoutMs (one minute, for export). const HEARTBEAT_INTERVAL_MS = 2_000; @@ -360,9 +365,12 @@ class UsageRecordingExecutor implements CapellaAgentExecutor { private readonly delegate: CapellaAgentExecutor, private readonly artifactRoot: string, private readonly baseIdentity: Omit, + private readonly stageTrace?: CapellaStageTrace, ) {} async run(request: CapellaAgentRequest): Promise> { + // The label is display-only and is deliberately excluded from this hash: two sessions that + // differ only by label are the same logical workload. const workloadId = sha256Parts(request.stage, request.role, request.systemPrompt, request.userPrompt).slice(0, 32); const sessionNumber = (this.sessionCounts.get(workloadId) ?? 0) + 1; this.sessionCounts.set(workloadId, sessionNumber); @@ -374,12 +382,23 @@ class UsageRecordingExecutor implements CapellaAgentExecutor { }; await writeImmutableUsageRecord(this.artifactRoot, identity, started); + const sessionLog: CapellaTraceLog | undefined = this.stageTrace?.forSession(request.sessionLabel); let response: CapellaAgentResponse | undefined; let caught: unknown; try { - response = await this.delegate.run(request); + const correlatedRequest = { + ...request, + executionKey: this.baseIdentity.executionKey, + attempt: this.baseIdentity.attempt, + ...(sessionLog !== undefined && { log: sessionLog }), + } as CapellaAgentRequest; + response = await this.delegate.run(correlatedRequest); } catch (error) { caught = error; + } finally { + // Drain this session's trace writes before the run returns, so the activity cannot complete + // with lines still buffered in memory. + await this.stageTrace?.drain(); } const errorUsage = usageFromError(caught); @@ -535,8 +554,17 @@ async function runStageActivity( let heartbeatInterval: NodeJS.Timeout | undefined; let inputFingerprint: string | undefined; let completedStageReturned = false; + let stageTrace: CapellaStageTrace | undefined; + + // Hold the stage's per-agent file open for the life of the activity so its concurrent sessions' + // trace lines ride one reference count. openStageAgentLog never throws (it returns null on + // failure); everything after it runs inside the try so the finally always releases the lease. + const stageAgentLog: LogStream | null = await WorkflowLogger.openStageAgentLog(input.workflowLogPath, stage); try { + // Log the start line after opening the lease so the per-agent file header leads. + await WorkflowLogger.logAgenticSastStart(input.workflowLogPath, stage, attempt, maximumAttempts); + // A missing policy row heartbeats too; only an explicit null opts a stage out. if (policy?.heartbeatTimeoutMs !== null) { heartbeat({ stage, attempt, elapsedSeconds: 0 }); @@ -562,12 +590,13 @@ async function runStageActivity( executionKey, attempt, }); - const executor = new UsageRecordingExecutor(capellaAgentExecutor, input.artifactRoot, { - inputFingerprint, - stage, - executionKey, - attempt, - }); + stageTrace = createCapellaStageTrace(input.workflowLogPath, stage); + const executor = new UsageRecordingExecutor( + capellaAgentExecutor, + input.artifactRoot, + { inputFingerprint, stage, executionKey, attempt }, + stageTrace, + ); const repositoryTools = await createCapellaRepositoryTools({ repositoryRoot: input.repoPath, deniedPaths: [...input.codePathAvoids, ...CONFINEMENT_ONLY_DENIED_PATHS], @@ -587,6 +616,14 @@ async function runStageActivity( // ledger aggregate that also counts any failed attempts of this stage. await recordStageUsageAccounting(input, inputFingerprint, stage, summary); const compactValue = compact(result.value); + const researchValue = stage === 'research' ? (compactValue as Record) : undefined; + const dispatchedCount = researchValue?.dispatchedCount; + const resumedCount = researchValue?.resumedCount; + const counts = + Number.isSafeInteger(dispatchedCount) && Number.isSafeInteger(resumedCount) + ? { dispatchedCount: Number(dispatchedCount), resumedCount: Number(resumedCount) } + : undefined; + await WorkflowLogger.logAgenticSastComplete(input.workflowLogPath, stage, result.durationMs, result.reused, counts); return { status: 'completed', durationMs: result.durationMs, @@ -600,6 +637,7 @@ async function runStageActivity( } catch (error) { const cancellation = activityCancellation(error, signal); if (cancellation) { + await WorkflowLogger.logAgenticSastCancelled(input.workflowLogPath, stage, attempt, maximumAttempts); throw cancellation; } @@ -647,6 +685,15 @@ async function runStageActivity( usageComplete: stageComplete, warnings: stageComplete ? [] : [usageAccountingWarning(stage)], }; + const retrying = classified.retryable && attempt < maximumAttempts; + await WorkflowLogger.logAgenticSastFailure( + input.workflowLogPath, + stage, + attempt, + maximumAttempts, + classified.code, + retrying, + ); // The message crossing the Temporal boundary comes from the fixed safe-message // table; raw provider and filesystem text never enters workflow history. throw ApplicationFailure.create({ @@ -657,6 +704,10 @@ async function runStageActivity( }); } finally { if (heartbeatInterval) clearInterval(heartbeatInterval); + // Drain any trailing trace writes, then release the stage's file lease, before the activity + // returns — so no line is still buffered and the stream closes with the stage. + if (stageTrace) await stageTrace.drain(); + await WorkflowLogger.closeStageAgentLog(stageAgentLog); } } diff --git a/apps/worker/src/ai/sast/capella/temporal/stage-trace.ts b/apps/worker/src/ai/sast/capella/temporal/stage-trace.ts new file mode 100644 index 00000000..cffbc6ae --- /dev/null +++ b/apps/worker/src/ai/sast/capella/temporal/stage-trace.ts @@ -0,0 +1,41 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** The per-stage trace surface that fans Capella session lines into the scan log. */ + +import { type TraceActor, WorkflowLogger } from '../../../../audit/workflow-logger.js'; +import type { CapellaStageTrace, CapellaTraceLog } from '../../../pi/capella-agent-types.js'; +import type { CapellaStage } from '../../types.js'; + +/** + * A raw trace surface for one stage's PI sessions. It holds no per-`toolCallId` state — the + * executor owns correlation — so it is safe to share across a stage's concurrent sessions. One + * serialization queue keeps every line intact and lets `drain` guarantee no line is still buffered + * when the activity returns; a failed write cannot fail the stage. Each `forSession` view carries + * its display label into the trace prefix's session component. + */ +export function createCapellaStageTrace(workflowLogPath: string, stage: CapellaStage): CapellaStageTrace { + let queue: Promise = Promise.resolve(); + const enqueue = (operation: () => Promise): void => { + queue = queue.then(operation, operation).catch(() => undefined); + }; + const forSession = (sessionLabel: string | undefined): CapellaTraceLog => { + const actor: TraceActor = + sessionLabel !== undefined ? { kind: 'sast', stage, session: sessionLabel } : { kind: 'sast', stage }; + return { + toolCall: (invocation) => enqueue(() => WorkflowLogger.logToolCall(workflowLogPath, actor, invocation)), + toolOutcome: (outcome) => enqueue(() => WorkflowLogger.logToolOutcome(workflowLogPath, actor, outcome)), + sessionComplete: (durationMs, turns, operations) => + enqueue(() => WorkflowLogger.logSessionComplete(workflowLogPath, actor, durationMs, turns, operations)), + }; + }; + return { + forSession, + drain: async () => { + await queue; + }, + }; +} diff --git a/apps/worker/src/ai/sast/types.ts b/apps/worker/src/ai/sast/types.ts index 0e4bf645..30a6291d 100644 --- a/apps/worker/src/ai/sast/types.ts +++ b/apps/worker/src/ai/sast/types.ts @@ -11,17 +11,26 @@ export interface SarifRef { sha256: string; } -export type CapellaStage = - | 'architecture' - | 'threat-model' - | 'plan' - | 'research' - | 'dedupe' - | 'review' - | 'critic' - | 'confirm' - | 'calibrate' - | 'export'; +export const CAPELLA_STAGES = [ + 'architecture', + 'threat-model', + 'plan', + 'research', + 'dedupe', + 'review', + 'critic', + 'confirm', + 'calibrate', + 'export', +] as const; + +export type CapellaStage = (typeof CAPELLA_STAGES)[number]; + +const CAPELLA_STAGE_SET = new Set(CAPELLA_STAGES); + +export function isCapellaStage(value: string): value is CapellaStage { + return CAPELLA_STAGE_SET.has(value); +} export type CapellaFailurePoint = CapellaStage | 'workflow'; diff --git a/apps/worker/src/ai/submit-tool.ts b/apps/worker/src/ai/submit-tool.ts index e4613027..847e0381 100644 --- a/apps/worker/src/ai/submit-tool.ts +++ b/apps/worker/src/ai/submit-tool.ts @@ -20,6 +20,12 @@ import { Type } from 'typebox'; export interface CapturedSubmitTool { readonly tool: ToolDefinition; readonly getCaptured: () => unknown | undefined; + /** + * A closed, safe result count for trace logging: the length of this tool's known + * submitted array. Omitted when the payload has no such array to count. Never derived + * from parsing an arbitrary result body. + */ + readonly safeCount?: () => number | undefined; readonly directive?: string; } diff --git a/apps/worker/src/audit/actor-projection.ts b/apps/worker/src/audit/actor-projection.ts new file mode 100644 index 00000000..54b34f77 --- /dev/null +++ b/apps/worker/src/audit/actor-projection.ts @@ -0,0 +1,98 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** + * The single projection boundary from a trace actor to its rendered forms: the combined-log + * prefix, and the per-agent file it also fans out to. All actor validation and filename mapping + * lives here, so no caller ever parses identity back out of a formatted line, and a slug can only + * be built from closed actor fields. + */ + +import path from 'node:path'; +import { isCapellaStage } from '../ai/sast/types.js'; +import { containsControlCharacter, isLoggableAgentName, type LoggableAgentName } from './safe-fields.js'; + +/** + * The actor a trace line is attributed to, rendered as its `[...]` prefix: a top-level agent, a + * delegated subagent under its parent, or an Agentic SAST stage that may name one of its concurrent + * sessions. + */ +export type TraceActor = + | { readonly kind: 'agent'; readonly agent: LoggableAgentName } + | { readonly kind: 'child'; readonly parent: LoggableAgentName; readonly child: string } + | { readonly kind: 'sast'; readonly stage: string; readonly session?: string }; + +/** + * The rendered forms of one actor. `combinedPrefix` is absent only when the actor itself is + * unsafe, which drops the whole line (the pre-existing fail-closed behavior). `agentFileSlug` is + * absent when no safe owning file can be named; that skips the per-agent fan-out only and never + * affects the combined line. + */ +export interface ActorProjection { + readonly combinedPrefix?: string; + readonly agentFileSlug?: string; +} + +/** A subagent or Capella session identity: normalized words plus an optional `#N` ordinal. */ +export function safeIdentityLabel(value: string): string | undefined { + if (containsControlCharacter(value)) return undefined; + return /^[a-z0-9][a-z0-9 '#-]{0,47}$/u.test(value) ? value : undefined; +} + +/** A per-agent log filename stem, drawn only from closed actor fields, safe as a path basename. */ +export function safeAgentFileSlug(value: string): string | undefined { + return /^[a-z0-9][a-z0-9-]{0,63}$/u.test(value) ? value : undefined; +} + +/** Render an actor's `[...]` prefix content, or `undefined` when any structural part is unsafe. */ +export function formatActor(actor: TraceActor): string | undefined { + if (actor.kind === 'agent') { + return isLoggableAgentName(actor.agent) ? actor.agent : undefined; + } + if (actor.kind === 'child') { + if (!isLoggableAgentName(actor.parent)) return undefined; + const child = safeIdentityLabel(actor.child); + return child !== undefined ? `${actor.parent} > ${child}` : undefined; + } + if (!isCapellaStage(actor.stage)) return undefined; + const base = `agentic-sast > ${actor.stage}`; + // A missing or unsafe session label degrades to the stage-only prefix; it never drops the line. + if (actor.session === undefined) return base; + const session = safeIdentityLabel(actor.session); + return session !== undefined ? `${base} > ${session}` : base; +} + +/** + * The stem of the per-agent file this actor's lines belong to, or `undefined` when none is safe. + * The stem is gated on the actor's closed field first (a known agent name or Capella stage), then + * re-checked for path safety, so an unknown name never spawns a stray file. + */ +export function agentFileSlug(actor: TraceActor): string | undefined { + if (actor.kind === 'agent') return isLoggableAgentName(actor.agent) ? safeAgentFileSlug(actor.agent) : undefined; + // A delegated subagent folds into its parent's file to keep the delegation narrative intact. + if (actor.kind === 'child') return isLoggableAgentName(actor.parent) ? safeAgentFileSlug(actor.parent) : undefined; + return isCapellaStage(actor.stage) ? safeAgentFileSlug(`agentic-sast-${actor.stage}`) : undefined; +} + +/** Project an actor into its combined-log prefix and its owning per-agent file stem. */ +export function projectActor(actor: TraceActor): ActorProjection { + const combinedPrefix = formatActor(actor); + const slug = agentFileSlug(actor); + return { + ...(combinedPrefix !== undefined && { combinedPrefix }), + ...(slug !== undefined && { agentFileSlug: slug }), + }; +} + +/** The `agents/` directory that holds a scan's per-agent logs, a sibling of the combined log. */ +export function agentsDir(workflowLogPath: string): string { + return path.join(path.dirname(workflowLogPath), 'agents'); +} + +/** The absolute path of a per-agent log, a sibling `agents/.log` of the combined log. */ +export function agentLogPath(workflowLogPath: string, slug: string): string { + return path.join(agentsDir(workflowLogPath), `${slug}.log`); +} diff --git a/apps/worker/src/audit/audit-session.ts b/apps/worker/src/audit/audit-session.ts index 06294b5d..50f01544 100644 --- a/apps/worker/src/audit/audit-session.ts +++ b/apps/worker/src/audit/audit-session.ts @@ -26,10 +26,14 @@ import { } from '../types/run-state.js'; import { SessionMutex } from '../utils/concurrency.js'; import { fileExists } from '../utils/file-io.js'; -import { formatTimestamp } from '../utils/formatting.js'; -import { AgentLogger } from './logger.js'; import { MetricsTracker } from './metrics-tracker.js'; -import { generateSessionJsonPath, initializeAuditStructure, type SessionMetadata } from './utils.js'; +import type { LoggableAgentName, WorkflowPhase } from './safe-fields.js'; +import { + generateSessionJsonPath, + generateWorkflowLogPath, + initializeAuditStructure, + type SessionMetadata, +} from './utils.js'; import { type AgentLogDetails, WorkflowLogger, type WorkflowSummary } from './workflow-logger.js'; // Global mutex instance @@ -37,14 +41,17 @@ const sessionMutex = new SessionMutex(); /** * AuditSession - Main audit system facade + * + * Construct a fresh instance per agent execution rather than sharing one across concurrent + * agents. `WorkflowLogger.close()` (called after every logged unit of work) releases every + * per-agent lease the instance currently holds, not just the caller's; a shared instance would + * let one agent's completion sever another agent's still-open log file mid-write. */ export class AuditSession { readonly sessionMetadata: SessionMetadata; private sessionId: string; private metricsTracker: MetricsTracker; private workflowLogger: WorkflowLogger; - private currentLogger: AgentLogger | null = null; - private currentAgentName: string | null = null; private initialized: boolean = false; constructor(sessionMetadata: SessionMetadata) { @@ -93,8 +100,9 @@ export class AuditSession { // Initialize metrics tracker (loads or creates session.json) await this.metricsTracker.initialize(workflowId); - // Initialize workflow logger with actual Temporal workflow ID - await this.workflowLogger.initialize(workflowId); + if (workflowId !== undefined) { + this.workflowLogger.setWorkflowId(workflowId); + } this.initialized = true; } @@ -111,76 +119,41 @@ export class AuditSession { /** * Start agent execution */ - async startAgent(agentName: string, promptContent: string, attemptNumber: number = 1): Promise { + async startAgent(agentName: LoggableAgentName, attemptNumber: number = 1): Promise { await this.ensureInitialized(); - - // 1. Save prompt snapshot (only on first attempt) - if (attemptNumber === 1) { - await AgentLogger.savePrompt(this.sessionMetadata, agentName, promptContent); - } - - // 2. Create and initialize the per-agent logger - this.currentAgentName = agentName; - this.currentLogger = new AgentLogger(this.sessionMetadata, agentName, attemptNumber); - await this.currentLogger.initialize(); - - // 3. Start metrics timer this.metricsTracker.startAgent(agentName, attemptNumber); - - // 4. Log start event to both agent log and workflow log - await this.currentLogger.logEvent('agent_start', { - agentName, - attemptNumber, - timestamp: formatTimestamp(), - }); - await this.workflowLogger.logAgent(agentName, 'start', { attemptNumber }); } - /** - * Log event during agent execution - */ - async logEvent(eventType: string, eventData: unknown): Promise { - if (!this.currentLogger) { - throw new PentestError( - 'No active logger. Call startAgent() first.', - 'validation', - false, - {}, - ErrorCode.AGENT_EXECUTION_FAILED, - ); - } + /** Absolute path to this scan's human-readable log, for path-based trace writers. */ + get workflowLogPath(): string { + return generateWorkflowLogPath(this.sessionMetadata); + } - // Log to agent-specific log file (JSON format) - await this.currentLogger.logEvent(eventType, eventData); - - // Also log to unified workflow log (human-readable format) - const data = eventData as Record; - const agentName = this.currentAgentName || 'unknown'; - switch (eventType) { - case 'tool_start': - await this.workflowLogger.logToolStart(agentName, String(data.toolName || ''), data.parameters); - break; - case 'llm_response': - await this.workflowLogger.logLlmResponse(agentName, Number(data.turn || 0), String(data.content || '')); - break; - // tool_end and error events are intentionally not logged to workflow log - // to reduce noise - the agent completion message captures the outcome - } + /** Record an agent attempt's closed-vocabulary error to the workflow log. */ + async logAgentError( + agentName: LoggableAgentName, + code: ErrorCode, + category: string, + attempt: number, + durationMs: number, + turns: number, + ): Promise { + await this.workflowLogger.logAgentError(agentName, code, category, attempt, durationMs, turns); } /** - * Write a human-readable note to the unified workflow log (e.g. a model - * refusal fallback). Independent of agent event logging. + * Release an agent's open per-agent log lease without recording an end. A backstop for an + * abnormal abort where {@link endAgent} never ran; idempotent, so a normal end makes it a no-op. */ - async logWorkflowNote(category: string, message: string): Promise { - await this.workflowLogger.logEvent(category, message); + async releaseAgentLog(agentName: LoggableAgentName): Promise { + await this.workflowLogger.releaseAgentLog(agentName); } /** * End agent execution (mutex-protected) */ - async endAgent(agentName: string, result: AgentEndResult): Promise { + async endAgent(agentName: LoggableAgentName, result: AgentEndResult): Promise { await this.finishAgentLogs(agentName, result); // 3. Acquire mutex before touching session.json @@ -207,32 +180,17 @@ export class AuditSession { } } - private async finishAgentLogs(agentName: string, result: AgentEndResult): Promise { - // 1. Finalize agent log and close the stream - if (this.currentLogger) { - await this.currentLogger.logEvent('agent_end', { - agentName, - success: result.success, - duration_ms: result.duration_ms, - cost_usd: result.cost_usd, - timestamp: formatTimestamp(), - }); - - await this.currentLogger.close(); - this.currentLogger = null; - } - - // 2. Log completion to the unified workflow log - this.currentAgentName = null; - + /** Write the agent's end line and close this instance's logger before touching session.json. */ + private async finishAgentLogs(agentName: LoggableAgentName, result: AgentEndResult): Promise { const agentLogDetails: AgentLogDetails = { attemptNumber: result.attemptNumber, duration_ms: result.duration_ms, cost_usd: result.cost_usd, success: result.success, - ...(result.error !== undefined && { error: result.error }), + ...(result.errorCode !== undefined && { errorCode: result.errorCode }), }; await this.workflowLogger.logAgent(agentName, 'end', agentLogDetails); + await this.workflowLogger.close(); } /** @@ -252,6 +210,8 @@ export class AuditSession { throw new RunStateError('IncompatibleWorkspaceError', 'session-json-missing-on-resume'); } await this.initialize(workflowId); + await this.workflowLogger.initialize(workflowId); + await this.workflowLogger.close(); const unlock = await sessionMutex.lock(this.sessionId); try { @@ -386,17 +346,25 @@ export class AuditSession { /** * Log phase start to unified workflow log */ - async logPhaseStart(phase: string): Promise { + async logPhaseStart(phase: WorkflowPhase): Promise { await this.ensureInitialized(); - await this.workflowLogger.logPhase(phase, 'start'); + try { + await this.workflowLogger.logPhase(phase, 'start'); + } finally { + await this.workflowLogger.close(); + } } /** * Log phase completion to unified workflow log */ - async logPhaseComplete(phase: string): Promise { + async logPhaseComplete(phase: WorkflowPhase): Promise { await this.ensureInitialized(); - await this.workflowLogger.logPhase(phase, 'complete'); + try { + await this.workflowLogger.logPhase(phase, 'complete'); + } finally { + await this.workflowLogger.close(); + } } /** @@ -404,7 +372,11 @@ export class AuditSession { */ async logWorkflowComplete(summary: WorkflowSummary): Promise { await this.ensureInitialized(); - await this.workflowLogger.logWorkflowComplete(summary); + try { + await this.workflowLogger.logWorkflowComplete(summary); + } finally { + await this.workflowLogger.close(); + } } /** @@ -427,17 +399,28 @@ export class AuditSession { } } - /** - * Log resume header to workflow.log - * Call this when a workflow is resuming to add a visual separator - */ - async logResumeHeader(resumeInfo: { + /** Write and flush the new execution boundary before publishing its durable resume record. */ + async logResumeBoundary(workflowId: string): Promise { + await this.ensureInitialized(); + try { + await this.workflowLogger.logResumeBoundary(workflowId); + } finally { + await this.workflowLogger.close(); + } + } + + /** Add checkpoint details beneath the already-durable resume boundary. */ + async logResumeDetails(resumeInfo: { previousWorkflowId: string; newWorkflowId: string; checkpointHash: string; completedAgents: string[]; }): Promise { await this.ensureInitialized(); - await this.workflowLogger.logResumeHeader(resumeInfo); + try { + await this.workflowLogger.logResumeDetails(resumeInfo); + } finally { + await this.workflowLogger.close(); + } } } diff --git a/apps/worker/src/audit/log-stream.ts b/apps/worker/src/audit/log-stream.ts index dc1d1b94..179a3c9d 100644 --- a/apps/worker/src/audit/log-stream.ts +++ b/apps/worker/src/audit/log-stream.ts @@ -4,124 +4,212 @@ // it under the terms of the GNU Affero General Public License version 3 // as published by the Free Software Foundation. -/** - * LogStream - Stream composition utility for append-only logging - * - * Encapsulates the common stream management pattern used by AgentLogger - * and WorkflowLogger: opening streams in append mode, handling backpressure, - * and proper cleanup. - */ +/** Process-wide serialized append handles for durable human-readable logging. */ -import fs from 'node:fs'; +import fs, { promises as fsPromises } from 'node:fs'; import path from 'node:path'; import { ensureDirectory } from '../utils/file-io.js'; +export type AppendSearchScope = 'whole-file' | 'current-execution'; +export type AppendMarkerMatch = 'exact-line' | 'line-suffix'; + +export interface AppendIfAbsentOptions { + readonly marker: string; + readonly scope: AppendSearchScope; + readonly match: AppendMarkerMatch; + readonly flush?: boolean; +} + +interface SharedLogEntry { + readonly filePath: string; + readonly stream: fs.WriteStream; + readonly ready: Promise; + queue: Promise; + references: number; + closing: boolean; +} + +const sharedLogs = new Map(); +let warned = false; +let agentLogWarned = false; + +export function warnLoggingFailure(): void { + if (warned) return; + warned = true; + console.error('Shannon could not write scan progress to workflow.log.'); +} + /** - * LogStream - Manages a single append-only log file stream + * A per-agent projection is best-effort: its failure must never disturb the canonical + * workflow.log, so it is warned about separately and never surfaced as a workflow.log fault. */ +export function warnAgentLoggingFailure(): void { + if (agentLogWarned) return; + agentLogWarned = true; + console.error('Shannon could not write a per-agent log projection; the combined workflow.log is unaffected.'); +} + +/** Open the append stream and track when it is safe to write, so an early `write()` waits on `open` instead of racing it. */ +function createSharedEntry(filePath: string): SharedLogEntry { + const stream = fs.createWriteStream(filePath, { flags: 'a', encoding: 'utf8', autoClose: true }); + const ready = new Promise((resolve, reject) => { + const onOpen = (): void => { + cleanup(); + resolve(); + }; + const onError = (): void => { + cleanup(); + reject(new Error('workflow log stream could not be opened')); + }; + const cleanup = (): void => { + stream.removeListener('open', onOpen); + stream.removeListener('error', onError); + }; + stream.once('open', onOpen); + stream.once('error', onError); + }); + stream.on('error', warnLoggingFailure); + return { filePath, stream, ready, queue: Promise.resolve(), references: 0, closing: false }; +} + +/** + * Chain one more operation onto an entry's serial queue, so writes from any number of concurrent + * `LogStream` handles to the same file still land in the order they were issued. The queue is + * reset to a settled promise regardless of outcome, so one failed write cannot wedge every + * write after it. + */ +function enqueue(entry: SharedLogEntry, operation: () => Promise): Promise { + const result = entry.queue.then(operation, operation); + entry.queue = result.then( + () => undefined, + () => undefined, + ); + return result; +} + +function writeToStream(stream: fs.WriteStream, text: string): Promise { + return new Promise((resolve, reject) => { + stream.write(text, 'utf8', (error) => { + if (error) reject(new Error('workflow log write failed')); + else resolve(); + }); + }); +} + +function syncStream(stream: fs.WriteStream): Promise { + const descriptor = (stream as fs.WriteStream & { readonly fd: number | null }).fd; + if (descriptor === null) return Promise.resolve(); + return new Promise((resolve, reject) => { + fs.fsync(descriptor, (error) => { + if (error) reject(new Error('workflow log flush failed')); + else resolve(); + }); + }); +} + +/** + * Restrict a marker search to the text written since the most recent resume boundary. A resumed + * run reopens the same log file, so without this a `current-execution` marker check would also + * match a line written by a previous, already-finished execution. + */ +function currentExecution(content: string): string { + const matches = [...content.matchAll(/^RESUMED\r?$/gmu)]; + const last = matches.at(-1); + return last?.index === undefined ? content : content.slice(last.index); +} + +function markerExists(content: string, options: AppendIfAbsentOptions): boolean { + const searched = options.scope === 'current-execution' ? currentExecution(content) : content; + const lines = searched.split(/\r?\n/u); + if (options.match === 'exact-line') return lines.includes(options.marker); + return lines.some((line) => line.endsWith(options.marker)); +} + +/** A reference-counted handle to one process-wide append stream. */ export class LogStream { - private readonly filePath: string; - private stream: fs.WriteStream | null = null; - private _isOpen: boolean = false; + private released = false; - constructor(filePath: string) { - this.filePath = filePath; - } + private constructor(private readonly entry: SharedLogEntry) {} /** - * Open the stream for writing (creates parent directories, opens in append mode) + * Take a reference on the shared entry for `filePath`, opening it if this is the first + * reference. If a prior lease is mid-{@link release} when this call arrives, wait for that + * drain to finish rather than reusing an entry that is about to be removed from the map; + * the loop re-reads the map afterward because the entry may have been deleted, or replaced + * by a new opener, while this call was waiting. */ - async open(): Promise { - if (this._isOpen) { - return; + static async acquire(filePath: string): Promise { + const absolutePath = path.resolve(filePath); + await ensureDirectory(path.dirname(absolutePath)); + let entry = sharedLogs.get(absolutePath); + while (entry?.closing === true) { + await entry.queue; + entry = sharedLogs.get(absolutePath); } + if (entry === undefined) { + entry = createSharedEntry(absolutePath); + sharedLogs.set(absolutePath, entry); + } + entry.references += 1; + try { + await entry.ready; + } catch (error) { + entry.references -= 1; + if (entry.references === 0) sharedLogs.delete(absolutePath); + warnLoggingFailure(); + throw error; + } + return new LogStream(entry); + } - // Ensure parent directory exists - await ensureDirectory(path.dirname(this.filePath)); - - // Create write stream in append mode - this.stream = fs.createWriteStream(this.filePath, { - flags: 'a', - encoding: 'utf8', - autoClose: true, + /** Queue an append; `flush` fsyncs before resolving, for the low-frequency structural lines that must be durable. */ + write(text: string, flush = false): Promise { + if (this.released) return Promise.reject(new Error('workflow log handle was released')); + return enqueue(this.entry, async () => { + await writeToStream(this.entry.stream, text); + if (flush) await syncStream(this.entry.stream); }); - - // Handle stream errors to prevent crashes (log and mark closed) - this.stream.on('error', (err) => { - console.error(`LogStream error for ${this.filePath}:`, err.message); - this._isOpen = false; - }); - - this._isOpen = true; } /** - * Write text to the stream with backpressure handling + * Append `text` only if its marker is not already present, so a structural line (a header, a + * resume boundary) survives a Temporal activity retry without being written twice. The check + * and the write share the same queued operation, so a concurrent writer on this entry cannot + * observe the marker as absent and duplicate it. */ - async write(text: string): Promise { - return new Promise((resolve, reject) => { - if (!this._isOpen || !this.stream) { - reject(new Error('LogStream not open')); - return; - } + appendIfAbsent(text: string, options: AppendIfAbsentOptions): Promise { + if (this.released) return Promise.reject(new Error('workflow log handle was released')); + return enqueue(this.entry, async () => { + const content = await fsPromises.readFile(this.entry.filePath, 'utf8').catch(() => ''); + if (markerExists(content, options)) return false; + await writeToStream(this.entry.stream, text); + if (options.flush === true) await syncStream(this.entry.stream); + return true; + }); + } - const stream = this.stream; - let drainHandler: (() => void) | null = null; - - const cleanup = () => { - if (drainHandler) { - stream.removeListener('drain', drainHandler); - drainHandler = null; - } - }; - - const needsDrain = !stream.write(text, 'utf8', (error) => { - cleanup(); - if (error) { - reject(error); - } else if (!needsDrain) { - resolve(); - } - }); - - if (needsDrain) { - drainHandler = () => { - cleanup(); - resolve(); - }; - stream.once('drain', drainHandler); + /** + * Drop this handle's reference. Only the last outstanding reference actually closes the + * underlying file descriptor; every earlier release just decrements the count so other + * concurrent leaseholders (an agent still mid-write, a stage still draining) are unaffected. + * The close itself is queued behind any writes already pending on this entry, and `closing` + * gates a new {@link acquire} until it finishes, so no writer ever sees a half-closed stream. + */ + async release(): Promise { + if (this.released) return; + this.released = true; + this.entry.references -= 1; + await enqueue(this.entry, async () => { + if (this.entry.references > 0 || this.entry.closing) return; + this.entry.closing = true; + await new Promise((resolve) => this.entry.stream.end(resolve)); + if (this.entry.references === 0 && sharedLogs.get(this.entry.filePath) === this.entry) { + sharedLogs.delete(this.entry.filePath); } }); } - /** - * Close the stream (flush and close) - */ - async close(): Promise { - if (!this._isOpen || !this.stream) { - return; - } - - return new Promise((resolve) => { - this.stream?.end(() => { - this._isOpen = false; - this.stream = null; - resolve(); - }); - }); - } - - /** - * Check if the stream is currently open - */ - get isOpen(): boolean { - return this._isOpen; - } - - /** - * Get the file path this stream writes to - */ get path(): string { - return this.filePath; + return this.entry.filePath; } } diff --git a/apps/worker/src/audit/logger.ts b/apps/worker/src/audit/logger.ts deleted file mode 100644 index 43f860de..00000000 --- a/apps/worker/src/audit/logger.ts +++ /dev/null @@ -1,122 +0,0 @@ -// Copyright (C) 2025 Keygraph, Inc. -// -// This program is free software: you can redistribute it and/or modify -// it under the terms of the GNU Affero General Public License version 3 -// as published by the Free Software Foundation. - -/** - * Append-Only Agent Logger - * - * Provides crash-safe, append-only logging for agent execution. - * Uses LogStream for stream management with backpressure handling. - */ - -import { atomicWrite } from '../utils/file-io.js'; -import { formatTimestamp } from '../utils/formatting.js'; -import { LogStream } from './log-stream.js'; -import { generateLogPath, generatePromptPath, type SessionMetadata } from './utils.js'; - -interface LogEvent { - type: string; - timestamp: string; - data: unknown; -} - -/** - * AgentLogger - Manages append-only logging for a single agent execution - */ -export class AgentLogger { - private readonly sessionMetadata: SessionMetadata; - private readonly agentName: string; - private readonly attemptNumber: number; - private readonly timestamp: number; - private readonly logStream: LogStream; - - constructor(sessionMetadata: SessionMetadata, agentName: string, attemptNumber: number) { - this.sessionMetadata = sessionMetadata; - this.agentName = agentName; - this.attemptNumber = attemptNumber; - this.timestamp = Date.now(); - - const logPath = generateLogPath(sessionMetadata, agentName, this.timestamp, attemptNumber); - this.logStream = new LogStream(logPath); - } - - /** - * Initialize the log stream (creates file and opens stream) - */ - async initialize(): Promise { - if (this.logStream.isOpen) { - return; // Already initialized - } - - await this.logStream.open(); - - // Write header - await this.writeHeader(); - } - - /** - * Write header to log file - */ - private async writeHeader(): Promise { - const header = [ - `========================================`, - `Agent: ${this.agentName}`, - `Attempt: ${this.attemptNumber}`, - `Started: ${formatTimestamp(this.timestamp)}`, - `Session: ${this.sessionMetadata.id}`, - `Web URL: ${this.sessionMetadata.webUrl}`, - `========================================\n`, - ].join('\n'); - - return this.logStream.write(header); - } - - /** - * Log an event (tool_start, tool_end, llm_response, etc.) - * Events are logged as JSON for parseability - */ - async logEvent(eventType: string, eventData: unknown): Promise { - const event: LogEvent = { - type: eventType, - timestamp: formatTimestamp(), - data: eventData, - }; - - const eventLine = `${JSON.stringify(event)}\n`; - return this.logStream.write(eventLine); - } - - /** - * Close the log stream - */ - async close(): Promise { - return this.logStream.close(); - } - - /** - * Save prompt snapshot to prompts directory - * Static method - doesn't require logger instance - */ - static async savePrompt(sessionMetadata: SessionMetadata, agentName: string, promptContent: string): Promise { - const promptPath = generatePromptPath(sessionMetadata, agentName); - - // Create header with metadata - const header = [ - `# Prompt Snapshot: ${agentName}`, - ``, - `**Session:** ${sessionMetadata.id}`, - `**Web URL:** ${sessionMetadata.webUrl}`, - `**Saved:** ${formatTimestamp()}`, - ``, - `---`, - ``, - ].join('\n'); - - const fullContent = header + promptContent; - - // Use atomic write for safety - await atomicWrite(promptPath, fullContent); - } -} diff --git a/apps/worker/src/audit/metrics-tracker.ts b/apps/worker/src/audit/metrics-tracker.ts index 68e3ca59..4fef2cdc 100644 --- a/apps/worker/src/audit/metrics-tracker.ts +++ b/apps/worker/src/audit/metrics-tracker.ts @@ -32,6 +32,7 @@ import { } from '../types/run-state.js'; import { atomicWrite, fileExists, readJson } from '../utils/file-io.js'; import { calculatePercentage, formatTimestamp } from '../utils/formatting.js'; +import { safeErrorFromCode } from './safe-fields.js'; import { generateSessionJsonPath, type SessionMetadata } from './utils.js'; interface AttemptData { @@ -47,6 +48,7 @@ interface AttemptData { timestamp: string; model?: string | undefined; error?: string | undefined; + error_code?: ErrorCode | undefined; } interface AgentAuditMetrics { @@ -733,6 +735,7 @@ export class MetricsTracker { }; data.metrics.agents[agentName] = agent; + const safeError = result.errorCode === undefined ? undefined : safeErrorFromCode(result.errorCode); const attempt: AttemptData = { attempt_number: result.attemptNumber, duration_ms: result.duration_ms, @@ -745,7 +748,7 @@ export class MetricsTracker { ...(result.cache_write_tokens !== undefined && { cache_write_tokens: result.cache_write_tokens }), ...(result.turns !== undefined && { turns: result.turns }), ...(result.model !== undefined && { model: result.model }), - ...(result.error !== undefined && { error: result.error }), + ...(safeError !== undefined && { error: safeError.message, error_code: safeError.code }), }; agent.attempts.push(attempt); agent.total_cost_usd = agent.attempts.reduce((sum, entry) => sum + entry.cost_usd, 0); diff --git a/apps/worker/src/audit/safe-fields.ts b/apps/worker/src/audit/safe-fields.ts new file mode 100644 index 00000000..7ea5c830 --- /dev/null +++ b/apps/worker/src/audit/safe-fields.ts @@ -0,0 +1,175 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +import { ALL_AGENTS } from '../types/agents.js'; +import { ErrorCode, type PentestErrorType } from '../types/errors.js'; + +export const WORKFLOW_PHASES = ['pre-recon', 'recon', 'vulnerability-exploitation', 'reporting'] as const; +export type WorkflowPhase = (typeof WORKFLOW_PHASES)[number]; + +export const LOGGABLE_AGENT_NAMES = [...ALL_AGENTS, 'validate-authentication'] as const; +export type LoggableAgentName = (typeof LOGGABLE_AGENT_NAMES)[number]; + +/** A log-safe error rendering: a known code paired with one of the fixed, generic messages below. */ +export interface SafeErrorDetails { + readonly code: ErrorCode; + readonly category: PentestErrorType; + readonly message: string; +} + +const SAFE_ERROR_MESSAGES: Readonly> = { + [ErrorCode.CONFIG_NOT_FOUND]: 'The requested configuration could not be loaded.', + [ErrorCode.CONFIG_VALIDATION_FAILED]: 'The scan configuration is invalid.', + [ErrorCode.CONFIG_PARSE_ERROR]: 'The scan configuration could not be parsed.', + [ErrorCode.AGENT_EXECUTION_FAILED]: 'The agent could not complete its work.', + [ErrorCode.OUTPUT_VALIDATION_FAILED]: 'The agent did not produce valid output.', + [ErrorCode.GIT_CHECKPOINT_FAILED]: 'The scan checkpoint could not be saved.', + [ErrorCode.GIT_ROLLBACK_FAILED]: 'The scan workspace could not be restored after a failed attempt.', + [ErrorCode.PROMPT_LOAD_FAILED]: 'The agent instructions could not be loaded.', + [ErrorCode.DELIVERABLE_NOT_FOUND]: 'The agent did not produce the required result.', + [ErrorCode.REPO_NOT_FOUND]: 'The repository could not be opened.', + [ErrorCode.TARGET_UNREACHABLE]: 'The target could not be reached.', + [ErrorCode.AUTH_FAILED]: 'Authentication validation failed.', + [ErrorCode.AUTH_LOGIN_FAILED]: 'The configured login could not be completed.', +}; + +const ERROR_CATEGORIES = new Set([ + 'config', + 'network', + 'prompt', + 'filesystem', + 'validation', + 'unknown', +]); + +const AGENT_NAME_SET = new Set(LOGGABLE_AGENT_NAMES); +const WORKFLOW_PHASE_SET = new Set(WORKFLOW_PHASES); +const ERROR_CODE_SET = new Set(Object.values(ErrorCode)); + +export function isWorkflowPhase(value: string): value is WorkflowPhase { + return WORKFLOW_PHASE_SET.has(value); +} + +export function isLoggableAgentName(value: string): value is LoggableAgentName { + return AGENT_NAME_SET.has(value); +} + +/** + * A workflow id safe to print in a log header or interpolate into a marker line. Falls back to + * a fixed placeholder rather than throwing, since an unparseable id must not stop the log from + * being written at all. + */ +export function safeWorkflowIdentifier(value: string): string { + if (/^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/u.test(value)) { + return value; + } + return 'unknown'; +} + +export function containsControlCharacter(value: string): boolean { + // Indexed scan, not a spread or regex: allocation-free over large tool arguments, and a + // control-character regex literal is disallowed by lint. + for (let index = 0; index < value.length; index += 1) { + const code = value.charCodeAt(index); + if (code <= 31 || code === 127) return true; + } + return false; +} + +/** + * True when a token looks like a credential, hash, or key rather than an identifier or word: + * a long unbroken alphanumeric run, a long digit-bearing token, or a long hex string. Used to + * fail-closed on secret-shaped search patterns and labels the agent may have just discovered. + */ +export function looksSecretShaped(value: string): boolean { + if (/[A-Za-z0-9]{20,}/u.test(value)) return true; + const alphanumericLength = value.replace(/[^A-Za-z0-9]/gu, '').length; + if (/[0-9]/u.test(value) && alphanumericLength >= 12) return true; + if (/^[0-9a-fA-F]{12,}$/u.test(value)) return true; + return false; +} + +/** + * The origin of a target URL, safe to print in a log header. Only `http`/`https` are accepted so + * an exotic scheme (or credentials embedded in the URL) never reaches the log; anything else, or + * anything unparseable, degrades to a placeholder instead of leaking the raw input. + */ +export function safeTargetUrl(value: string): string { + if (containsControlCharacter(value)) return 'unavailable'; + try { + const parsedUrl = new URL(value); + if (parsedUrl.protocol !== 'http:' && parsedUrl.protocol !== 'https:') { + return 'unavailable'; + } + return parsedUrl.origin; + } catch { + return 'unavailable'; + } +} + +/** + * Map an error code to its fixed, pre-approved log message rather than logging the error's own + * message text. The underlying error can carry a file path, a stack frame, or other repo-specific + * detail; only the closed `SAFE_ERROR_MESSAGES` table is allowed into a log line. An unrecognized + * code or category falls back to a generic entry instead of being dropped, so a fault always gets + * a log line, just not one repeating unvetted text. + */ +export function safeErrorFromCode(code: ErrorCode, category: PentestErrorType = 'unknown'): SafeErrorDetails { + const safeCode = ERROR_CODE_SET.has(code) ? code : ErrorCode.AGENT_EXECUTION_FAILED; + return { + code: safeCode, + category: ERROR_CATEGORIES.has(category) ? category : 'unknown', + message: SAFE_ERROR_MESSAGES[safeCode], + }; +} + +/** + * Recover a code and category from an error of unknown shape, then defer to + * {@link safeErrorFromCode} for the actual safe rendering. The duck-typed field reads only ever + * pick out values that are already in the closed code/category sets, so a caught error's message + * or other properties can never flow through into the log. + */ +export function safeErrorFromUnknown( + error: unknown, + fallbackCode: ErrorCode = ErrorCode.AGENT_EXECUTION_FAILED, +): SafeErrorDetails { + let code = fallbackCode; + let category: PentestErrorType = 'unknown'; + if (typeof error === 'object' && error !== null) { + const candidate = error as { readonly code?: unknown; readonly type?: unknown }; + if (typeof candidate.code === 'string' && ERROR_CODE_SET.has(candidate.code)) { + code = candidate.code as ErrorCode; + } + if (typeof candidate.type === 'string' && ERROR_CATEGORIES.has(candidate.type as PentestErrorType)) { + category = candidate.type as PentestErrorType; + } + } + return safeErrorFromCode(code, category); +} + +/** + * Reduce a free-text human description (child-task description, active todo label) to a + * short, safe semantic label, or `undefined` when it is structurally unsafe. + * + * Ordinary security vocabulary — `authorization`, `password`, `token` — is allowed; the + * rejection is structural, not a word blocklist. Fail-closed: anything carrying a URL, + * path, domain, assignment, colon, secret-shaped token, control character, or excessive + * length is rejected rather than partially sanitized. + */ +export function normalizeSemanticLabel(value: unknown): string | undefined { + if (typeof value !== 'string' || containsControlCharacter(value)) return undefined; + const collapsed = value.trim().replace(/\s+/gu, ' '); + if (collapsed.length === 0 || collapsed.length > 48) return undefined; + // Paths, domains/filenames, assignments, colons, and addresses are structurally unsafe. + if (/[./\\=:@]/u.test(collapsed)) return undefined; + // A long unbroken alphanumeric run is secret/hash/base64-shaped, never a real word. + if (/[A-Za-z0-9_-]{20,}/u.test(collapsed)) return undefined; + const words = collapsed.toLowerCase().split(' '); + if (words.length > 6) return undefined; + if (!words.every((word) => /^[a-z0-9][a-z0-9'-]{0,19}$/u.test(word))) return undefined; + if (words.some(looksSecretShaped)) return undefined; + return words.join(' '); +} diff --git a/apps/worker/src/audit/trace.ts b/apps/worker/src/audit/trace.ts new file mode 100644 index 00000000..1b5c1355 --- /dev/null +++ b/apps/worker/src/audit/trace.ts @@ -0,0 +1,124 @@ +// Copyright (C) 2026 Keygraph, Inc. +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License version 3 +// as published by the Free Software Foundation. + +/** Lossless tool-invocation capture shared by every workflow.log producer. */ + +import { warnLoggingFailure } from './log-stream.js'; + +/** One immutable tool invocation, serialized synchronously from the PI event. */ +export interface ToolInvocation { + readonly tool: string; + readonly argumentsJson: string; +} + +/** The optional second line a tool call earns on completion. */ +/** The one conditional second line a tool call may earn, chosen by {@link decideToolOutcome}. */ +export type ToolOutcome = + | { readonly kind: 'failed'; readonly tool: string; readonly durationMs: number } + | { readonly kind: 'slow'; readonly tool: string; readonly durationMs: number } + | { readonly kind: 'count'; readonly tool: string; readonly count: number }; + +const COLLECTOR_PREFIXES = ['submit_', 'set_', 'add_', 'record_', 'report_'] as const; + +/** A successful bash call is worth a slow line past 5s; any other tool past 10s. */ +const SLOW_BASH_MS = 5_000; +const SLOW_OTHER_MS = 10_000; + +/** + * Walk a value and throw on the first thing that cannot round-trip through `JSON.stringify` + * unchanged: a cycle, a sparse or extended array, a non-plain object, or an accessor or symbol + * property. `JSON.stringify` would otherwise silently drop or reshape these rather than fail, and + * a silently-altered tool-call argument would break the log's claim to being a lossless capture. + */ +function assertJsonValue(value: unknown, activeObjects: WeakSet): void { + if (value === null || typeof value === 'string' || typeof value === 'boolean') return; + if (typeof value === 'number') { + if (!Number.isFinite(value)) throw new TypeError('tool arguments contain a non-finite number'); + return; + } + if (typeof value !== 'object') throw new TypeError('tool arguments contain a non-JSON value'); + if (activeObjects.has(value)) throw new TypeError('tool arguments contain a cycle'); + + activeObjects.add(value); + try { + if (Array.isArray(value)) { + const enumerableKeys = Object.keys(value); + if (enumerableKeys.length !== value.length) throw new TypeError('tool arguments contain a sparse array'); + for (let index = 0; index < value.length; index += 1) { + if (enumerableKeys[index] !== String(index)) throw new TypeError('tool arguments contain an extended array'); + assertJsonValue(value[index], activeObjects); + } + return; + } + + const prototype = Object.getPrototypeOf(value); + if (prototype !== Object.prototype && prototype !== null) { + throw new TypeError('tool arguments contain a non-plain object'); + } + const enumerableKeys = Object.keys(value); + if (Reflect.ownKeys(value).length !== enumerableKeys.length) { + throw new TypeError('tool arguments contain a non-enumerable or symbol field'); + } + for (const key of enumerableKeys) { + const descriptor = Object.getOwnPropertyDescriptor(value, key); + if (descriptor === undefined || !('value' in descriptor)) { + throw new TypeError('tool arguments contain an accessor field'); + } + assertJsonValue(descriptor.value, activeObjects); + } + } finally { + activeObjects.delete(value); + } +} + +/** + * Snapshot a PI argument payload as compact JSON. PI arguments are parsed JSON; the + * validation prevents future in-process callers from silently losing non-JSON values. + */ +export function serializeToolArguments(args: unknown): string | undefined { + if (args === undefined) return '{}'; + try { + assertJsonValue(args, new WeakSet()); + const serialized = JSON.stringify(args); + if (serialized === undefined) throw new TypeError('tool arguments could not be serialized'); + return serialized; + } catch { + warnLoggingFailure(); + return undefined; + } +} + +/** Capture the literal tool name and complete serialized arguments in the event callback. */ +export function captureToolInvocation(tool: string, args: unknown): ToolInvocation | undefined { + const argumentsJson = serializeToolArguments(args); + return argumentsJson === undefined ? undefined : { tool, argumentsJson }; +} + +function isCollectorName(tool: string): boolean { + return COLLECTOR_PREFIXES.some((prefix) => tool.startsWith(prefix)); +} + +/** + * Decide whether a completed tool call earns a second line. Task calls use their + * delegated-session lifecycle instead of duplicate generic failure or slow records. + */ +export function decideToolOutcome( + tool: string, + isError: boolean, + durationMs: number, + collectorCount: number | undefined, +): ToolOutcome | undefined { + if (tool === 'task') return undefined; + if (isError) return { kind: 'failed', tool, durationMs }; + if (isCollectorName(tool)) { + if (typeof collectorCount === 'number' && Number.isSafeInteger(collectorCount) && collectorCount >= 0) { + return { kind: 'count', tool, count: collectorCount }; + } + return undefined; + } + const threshold = tool === 'bash' ? SLOW_BASH_MS : SLOW_OTHER_MS; + return durationMs > threshold ? { kind: 'slow', tool, durationMs } : undefined; +} diff --git a/apps/worker/src/audit/utils.ts b/apps/worker/src/audit/utils.ts index 97de3de8..ee0737c6 100644 --- a/apps/worker/src/audit/utils.ts +++ b/apps/worker/src/audit/utils.ts @@ -53,28 +53,6 @@ export function generateInternalPath(sessionMetadata: SessionMetadata): string { return path.join(generateAuditPath(sessionMetadata), INTERNAL_DIR); } -/** - * Generate path to agent log file - */ -export function generateLogPath( - sessionMetadata: SessionMetadata, - agentName: string, - timestamp: number, - attemptNumber: number, -): string { - const internalPath = generateInternalPath(sessionMetadata); - const filename = `${timestamp}_${agentName}_attempt-${attemptNumber}.log`; - return path.join(internalPath, 'agents', filename); -} - -/** - * Generate path to prompt snapshot file - */ -export function generatePromptPath(sessionMetadata: SessionMetadata, agentName: string): string { - const internalPath = generateInternalPath(sessionMetadata); - return path.join(internalPath, 'prompts', `${agentName}.md`); -} - /** * Generate path to session.json file */ @@ -86,6 +64,7 @@ export function generateSessionJsonPath(sessionMetadata: SessionMetadata): strin /** * Path to the shared authenticated browser session saved by the preflight * validator and consumed by downstream agents via `_shared-session.txt`. + * Deleted at workflow end, so an authenticated session never outlives the scan it was created for. */ export function authStateFile(sessionMetadata: SessionMetadata): string { return path.join(generateInternalPath(sessionMetadata), 'auth-state.json'); @@ -101,15 +80,10 @@ export function generateWorkflowLogPath(sessionMetadata: SessionMetadata): strin /** * Initialize audit directory structure for a session. - * Creates: workspaces/{sessionId}/.shannon/{agents,prompts}. The deliverables, - * scratchpad, and browser dirs are created host-side and bind-mounted in. + * Creates the hidden internals directory. The deliverables, scratchpad, and + * browser directories are created host-side and bind-mounted in. */ export async function initializeAuditStructure(sessionMetadata: SessionMetadata): Promise { const internalPath = generateInternalPath(sessionMetadata); - const agentsPath = path.join(internalPath, 'agents'); - const promptsPath = path.join(internalPath, 'prompts'); - await ensureDirectory(internalPath); - await ensureDirectory(agentsPath); - await ensureDirectory(promptsPath); } diff --git a/apps/worker/src/audit/workflow-logger.ts b/apps/worker/src/audit/workflow-logger.ts index 66a5e13f..c2970953 100644 --- a/apps/worker/src/audit/workflow-logger.ts +++ b/apps/worker/src/audit/workflow-logger.ts @@ -4,441 +4,735 @@ // it under the terms of the GNU Affero General Public License version 3 // as published by the Free Software Foundation. -/** - * Workflow Logger - * - * Provides a unified, human-readable log file per workflow. - * Optimized for `tail -f` viewing during concurrent workflow execution. - */ +/** Closed-field, human-readable scan logging. */ -import fs from 'node:fs/promises'; +import { promises as fsPromises } from 'node:fs'; +import path from 'node:path'; +import { isCapellaSafeFailureMessage, isCapellaTerminalStageLabel } from '../ai/sast/capella/safe-failures.js'; +import type { CapellaStage } from '../ai/sast/types.js'; +import type { ErrorCode } from '../types/errors.js'; +import { isPartialReason, type PartialReasonView, projectPartialReasons } from '../types/run-state.js'; import { formatDuration, formatTimestamp } from '../utils/formatting.js'; -import { LogStream } from './log-stream.js'; +import { + agentLogPath, + agentsDir, + projectActor, + safeAgentFileSlug, + safeIdentityLabel, + type TraceActor, +} from './actor-projection.js'; +import { LogStream, warnAgentLoggingFailure, warnLoggingFailure } from './log-stream.js'; +import { + isLoggableAgentName, + isWorkflowPhase, + type LoggableAgentName, + safeErrorFromCode, + safeTargetUrl, + safeWorkflowIdentifier, + type WorkflowPhase, +} from './safe-fields.js'; +import type { ToolInvocation, ToolOutcome } from './trace.js'; import { generateWorkflowLogPath, type SessionMetadata } from './utils.js'; +export type { TraceActor } from './actor-projection.js'; + export interface AgentLogDetails { - attemptNumber?: number; - duration_ms?: number; - cost_usd?: number; - success?: boolean; - error?: string; + readonly attemptNumber?: number; + readonly duration_ms?: number; + readonly cost_usd?: number; + readonly success?: boolean; + readonly errorCode?: ErrorCode; } export interface AgentMetricsSummary { - durationMs: number; - costUsd: number | null; -} - -/** Mirror of the derived partial-reason view; the safe message is already resolved. */ -export interface WorkflowSummaryPartialReason { - readonly code: string; - readonly message: string; - readonly vulnerabilityClass?: string; - readonly stage?: string; + readonly durationMs: number; + readonly costUsd: number | null; } export interface WorkflowSummary { - status: 'completed' | 'failed' | 'cancelled' | 'partial'; - totalDurationMs: number; - totalCostUsd: number; - /** Agents that actually ran. Mutually exclusive from `skippedAgents`. */ - completedAgents: string[]; - /** Expected agents that never ran because their class had nothing to exploit. */ - skippedAgents?: readonly string[]; - agentMetrics: Record; - /** Ordered durable degradation reasons; present and non-empty exactly for partial runs. */ - partialReasons?: readonly WorkflowSummaryPartialReason[]; - /** False when operational (Capella/reconciliation) spend is known to be incomplete. */ - usageAccountingComplete?: boolean; - /** Reader-facing name of the stage a failed agentic-SAST run stopped at. */ - agenticSastFailedStage?: string; - /** Sanitized failure sentence from a failed agentic-SAST run; safe for operator output. */ - agenticSastFailureMessage?: string; - /** Bounded machine code from a failed agentic-SAST run, when one was preserved. */ - agenticSastErrorCode?: string; - error?: string; + readonly status: 'completed' | 'failed' | 'cancelled' | 'partial'; + readonly totalDurationMs: number; + readonly totalCostUsd: number; + readonly completedAgents: readonly string[]; + readonly skippedAgents?: readonly string[]; + readonly agentMetrics: Readonly>; + readonly partialReasons?: readonly PartialReasonView[]; + readonly usageAccountingComplete?: boolean; + readonly agenticSastFailedStage?: string; + readonly agenticSastFailureMessage?: string; + readonly agenticSastErrorCode?: string; + readonly errorCode?: ErrorCode; +} + +export type ChildTaskFailureCode = 'CANCELLED' | 'CHILD_TASK_FAILED'; + +const AGENTIC_SAST_STAGE_LABELS: Readonly> = { + architecture: 'Architecture', + 'threat-model': 'Threat model', + plan: 'Planning', + research: 'Audit wave', + dedupe: 'Deduplication', + review: 'Review', + critic: 'Critic', + confirm: 'Confirmation', + calibrate: 'Calibration', + export: 'Export', +}; + +function isSafeCount(value: number): boolean { + return Number.isSafeInteger(value) && value >= 0 && value <= 1_000_000_000; } /** - * WorkflowLogger - Manages the unified workflow log file + * Render a partial reason's message only after rebuilding it as a fresh, narrowly-shaped + * candidate and revalidating it with {@link isPartialReason}. The view arrives already + * projected, but re-checking here means a future field added to the view can never reach the + * log through this path until it is deliberately admitted into the narrowed candidate shape. */ +function safeReasonMessage(reason: PartialReasonView): string | undefined { + if (reason.code === 'agentic_sast_reduced') return 'Agentic SAST completed with reduced coverage.'; + let candidate: unknown = { code: reason.code }; + if (reason.vulnerabilityClass !== undefined) { + candidate = { code: reason.code, vulnerabilityClass: reason.vulnerabilityClass }; + } else if (reason.stage !== undefined && reason.code === 'agentic_sast_failed') { + candidate = { code: reason.code, stage: reason.stage }; + } + if (!isPartialReason(candidate)) return undefined; + return projectPartialReasons([candidate])[0]?.message; +} + +function safeAgenticSastCode(code: string | undefined): string | undefined { + if (code !== undefined && /^[A-Z][A-Z0-9_]{0,63}$/u.test(code)) return code; + return undefined; +} + +function safeAgenticSastStageLabel(label: string | undefined): string | undefined { + return label !== undefined && isCapellaTerminalStageLabel(label) ? label : undefined; +} + +/** Keep normal PI names readable and losslessly quote any unexpected name. */ +function formatToolName(tool: string): string { + return /^[A-Za-z][A-Za-z0-9_-]{0,63}$/u.test(tool) ? tool : JSON.stringify(tool); +} + +/** One self-describing first line per per-agent file, appended once when its lease opens. */ +const AGENT_LOG_HEADER_PREFIX = '=== Shannon agent log: '; + +function agentLogHeader(slug: string): string { + return `${AGENT_LOG_HEADER_PREFIX}${slug} ===`; +} + +/** Manages the one human-readable log for a scan workspace, plus its per-agent projection files. */ export class WorkflowLogger { + private readonly logPath: string; private readonly sessionMetadata: SessionMetadata; - private readonly logStream: LogStream; + private logStream: LogStream | null = null; + private initializePromise: Promise | null = null; private workflowId: string | undefined; + // A pipeline agent's file lease, held from its start line to its end line so the agent's own + // trace and lifecycle lines ride the reference count instead of reopening the file each write. + private readonly agentLeases = new Map(); + + private static traceTimestamp(): string { + return new Date().toISOString().replace('T', ' ').slice(0, 19); + } + + /** + * Append one line to the combined log. A failure here is swallowed and only warned about once + * process-wide: the scan itself must keep running even if the log file becomes unwritable, so + * this never throws back into agent or workflow logic. + */ + private static async writeLine(workflowLogPath: string, line: string, flush: boolean): Promise { + let stream: LogStream | undefined; + try { + stream = await LogStream.acquire(workflowLogPath); + await stream.write(`${line}\n`, flush); + } catch { + warnLoggingFailure(); + } finally { + await stream?.release().catch(warnLoggingFailure); + } + } + + // Per-tool trace lines are high-frequency; a per-line fsync would dominate their cost. + // They rely on the OS write buffer (visible to the tailing CLI) rather than durable flush, + // while the low-frequency structural lines still fsync. + private static writeTraceLine(workflowLogPath: string, line: string): Promise { + return WorkflowLogger.writeLine(workflowLogPath, line, false); + } + + /** + * Fan a formatted line out to an actor's per-agent projection file. Best-effort and isolated: a + * failure here warns separately and never disturbs the canonical combined log. When a lifecycle + * owner holds a lease on the file, this acquire/release rides its reference count and the stream + * stays open; with no owner it opens per line, which is correct, just slower. + */ + private static async fanOutLine(workflowLogPath: string, slug: string, line: string, flush: boolean): Promise { + let stream: LogStream | undefined; + try { + stream = await LogStream.acquire(agentLogPath(workflowLogPath, slug)); + await stream.write(`${line}\n`, flush); + } catch { + warnAgentLoggingFailure(); + } finally { + await stream?.release().catch(warnAgentLoggingFailure); + } + } + + /** + * Write one trace line to the combined log first, then fan it out to the actor's per-agent file. + * A structurally unsafe actor drops the line from both, preserving the pre-existing fail-closed + * behavior; an unnameable owning file skips the fan-out alone. + */ + private static async writeProjectedTraceLine( + workflowLogPath: string, + actor: TraceActor, + render: (prefix: string) => string, + ): Promise { + const { combinedPrefix, agentFileSlug: slug } = projectActor(actor); + if (combinedPrefix === undefined) return; + const line = render(combinedPrefix); + await WorkflowLogger.writeTraceLine(workflowLogPath, line); + if (slug !== undefined) await WorkflowLogger.fanOutLine(workflowLogPath, slug, line, false); + } + + /** + * A flushed Capella stage lifecycle line: to the combined log first, then to the stage's own + * per-agent file. The stage id is a closed field, so its slug is always safe. + */ + private static async writeStageStructuralLine( + workflowLogPath: string, + stage: CapellaStage, + line: string, + ): Promise { + await WorkflowLogger.writeLine(workflowLogPath, line, true); + const slug = safeAgentFileSlug(`agentic-sast-${stage}`); + if (slug !== undefined) await WorkflowLogger.fanOutLine(workflowLogPath, slug, line, true); + } + + /** + * Acquire a per-agent file's lease and ensure its header is present. `appendIfAbsent` makes the + * header idempotent, so reopening the same file across a Temporal retry or a resumed run never + * duplicates it. Returns `null` on any failure rather than throwing, since an agent whose own + * file cannot be opened must still be able to run and log to the combined file. + */ + private static async openAgentLease(workflowLogPath: string, slug: string): Promise { + try { + const stream = await LogStream.acquire(agentLogPath(workflowLogPath, slug)); + const header = agentLogHeader(slug); + await stream.appendIfAbsent(`${header}\n`, { marker: header, scope: 'whole-file', match: 'exact-line' }); + return stream; + } catch { + warnAgentLoggingFailure(); + return null; + } + } + + /** + * Open and hold a Capella stage's per-agent file lease for the life of the stage activity. The + * caller passes the handle back to {@link closeStageAgentLog} in its `finally`; while held, the + * stage's concurrent session trace lines keep the file open through the shared reference count. + */ + static openStageAgentLog(workflowLogPath: string, stage: CapellaStage): Promise { + const slug = safeAgentFileSlug(`agentic-sast-${stage}`); + if (slug === undefined) return Promise.resolve(null); + return WorkflowLogger.openAgentLease(workflowLogPath, slug); + } + + /** Release a stage's per-agent file lease. Best-effort: a failure never fails the stage. */ + static async closeStageAgentLog(lease: LogStream | null): Promise { + if (lease === null) return; + await lease.release().catch(warnAgentLoggingFailure); + } + + /** `[agent] read: {"path":"/src/routes"}` — one complete invocation per line. */ + static async logToolCall(workflowLogPath: string, actor: TraceActor, invocation: ToolInvocation): Promise { + const tool = formatToolName(invocation.tool); + await WorkflowLogger.writeProjectedTraceLine( + workflowLogPath, + actor, + (prefix) => `[${WorkflowLogger.traceTimestamp()}] [${prefix}] ${tool}: ${invocation.argumentsJson}`, + ); + } + + /** The conditional second line: a failure, a slow success, or a collector's safe count. */ + static async logToolOutcome(workflowLogPath: string, actor: TraceActor, outcome: ToolOutcome): Promise { + const tool = formatToolName(outcome.tool); + let suffix: string; + if (outcome.kind === 'failed') { + suffix = `${tool} failed (${formatDuration(Math.max(0, outcome.durationMs))})`; + } else if (outcome.kind === 'slow') { + suffix = `${tool} slow (${formatDuration(Math.max(0, outcome.durationMs))})`; + } else { + if (!isSafeCount(outcome.count)) return; + suffix = `${tool}: submitted ${outcome.count} findings`; + } + await WorkflowLogger.writeProjectedTraceLine( + workflowLogPath, + actor, + (prefix) => `[${WorkflowLogger.traceTimestamp()}] [${prefix}] ${suffix}`, + ); + } + + /** `[parent] task: started subagent "entry point mapper"` — the single delegation line. */ + static async logDelegationStart(workflowLogPath: string, parent: LoggableAgentName, child: string): Promise { + if (!isLoggableAgentName(parent)) return; + const identity = safeIdentityLabel(child); + if (identity === undefined) return; + await WorkflowLogger.writeProjectedTraceLine( + workflowLogPath, + { kind: 'agent', agent: parent }, + (prefix) => `[${WorkflowLogger.traceTimestamp()}] [${prefix}] task: started subagent "${identity}"`, + ); + } + + /** `[actor] completed (1m 14s, 11 turns, 30 operations)` — a subagent or SAST session's terminal line. */ + static async logSessionComplete( + workflowLogPath: string, + actor: TraceActor, + durationMs: number, + turns: number, + operations: number, + ): Promise { + const safeTurns = isSafeCount(turns) ? turns : 0; + const safeOperations = isSafeCount(operations) ? operations : 0; + const duration = formatDuration(Number.isFinite(durationMs) ? Math.max(0, durationMs) : 0); + await WorkflowLogger.writeProjectedTraceLine( + workflowLogPath, + actor, + (prefix) => + `[${WorkflowLogger.traceTimestamp()}] [${prefix}] completed (${duration}, ${safeTurns} turns, ${safeOperations} operations)`, + ); + } + + /** `[actor] failed (2.1s, CHILD_TASK_FAILED)` — a subagent session's terminal failure line. */ + static async logSessionFailure( + workflowLogPath: string, + actor: TraceActor, + code: ChildTaskFailureCode, + durationMs: number, + ): Promise { + const safeCode: ChildTaskFailureCode = code === 'CANCELLED' ? 'CANCELLED' : 'CHILD_TASK_FAILED'; + const status = safeCode === 'CANCELLED' ? 'cancelled' : 'failed'; + const duration = formatDuration(Number.isFinite(durationMs) ? Math.max(0, durationMs) : 0); + await WorkflowLogger.writeProjectedTraceLine( + workflowLogPath, + actor, + (prefix) => `[${WorkflowLogger.traceTimestamp()}] [${prefix}] ${status} (${duration}, ${safeCode})`, + ); + } + + /** A Capella stage's flushed structural start line: to the combined log and its own per-agent file. */ + static async logAgenticSastStart( + workflowLogPath: string, + stage: CapellaStage, + attempt: number, + maximumAttempts: number, + ): Promise { + const safeAttempt = isSafeCount(attempt) ? attempt : 1; + const safeMaximum = isSafeCount(maximumAttempts) ? maximumAttempts : safeAttempt; + await WorkflowLogger.writeStageStructuralLine( + workflowLogPath, + stage, + `[${new Date().toISOString().replace('T', ' ').slice(0, 19)}] [AGENTIC-SAST] ${AGENTIC_SAST_STAGE_LABELS[stage]}: Starting (attempt ${safeAttempt} of ${safeMaximum})`, + ); + } + + /** A Capella stage's flushed structural completion line, including reuse and dispatch counts when known. */ + static async logAgenticSastComplete( + workflowLogPath: string, + stage: CapellaStage, + durationMs: number, + reused: boolean, + counts?: { readonly dispatchedCount: number; readonly resumedCount: number }, + ): Promise { + const details: string[] = []; + if (counts !== undefined && isSafeCount(counts.dispatchedCount) && isSafeCount(counts.resumedCount)) { + details.push(`${counts.dispatchedCount} dispatched`, `${counts.resumedCount} resumed`); + } + details.push(formatDuration(Number.isFinite(durationMs) ? Math.max(0, durationMs) : 0)); + if (reused) details.push('reused'); + await WorkflowLogger.writeStageStructuralLine( + workflowLogPath, + stage, + `[${new Date().toISOString().replace('T', ' ').slice(0, 19)}] [AGENTIC-SAST] ${AGENTIC_SAST_STAGE_LABELS[stage]}: Completed (${details.join(', ')})`, + ); + } + + /** A Capella stage's flushed structural failure line, noting whether Temporal will retry the attempt. */ + static async logAgenticSastFailure( + workflowLogPath: string, + stage: CapellaStage, + attempt: number, + maximumAttempts: number, + code: string, + retrying: boolean, + ): Promise { + const safeAttempt = isSafeCount(attempt) ? attempt : 1; + const safeMaximum = isSafeCount(maximumAttempts) ? maximumAttempts : safeAttempt; + const safeCode = safeAgenticSastCode(code) ?? 'ACTIVITY_FAILURE'; + const outcome = retrying ? 'Failed, retrying' : 'Failed'; + await WorkflowLogger.writeStageStructuralLine( + workflowLogPath, + stage, + `[${new Date().toISOString().replace('T', ' ').slice(0, 19)}] [AGENTIC-SAST] ${AGENTIC_SAST_STAGE_LABELS[stage]}: ${outcome} (attempt ${safeAttempt} of ${safeMaximum}, ${safeCode})`, + ); + } + + /** A Capella stage's flushed structural cancellation line. */ + static async logAgenticSastCancelled( + workflowLogPath: string, + stage: CapellaStage, + attempt: number, + maximumAttempts: number, + ): Promise { + const safeAttempt = isSafeCount(attempt) ? attempt : 1; + const safeMaximum = isSafeCount(maximumAttempts) ? maximumAttempts : safeAttempt; + await WorkflowLogger.writeStageStructuralLine( + workflowLogPath, + stage, + `[${new Date().toISOString().replace('T', ' ').slice(0, 19)}] [AGENTIC-SAST] ${AGENTIC_SAST_STAGE_LABELS[stage]}: Cancelled (attempt ${safeAttempt} of ${safeMaximum}, CANCELLED)`, + ); + } constructor(sessionMetadata: SessionMetadata) { this.sessionMetadata = sessionMetadata; - const logPath = generateWorkflowLogPath(sessionMetadata); - this.logStream = new LogStream(logPath); + this.logPath = generateWorkflowLogPath(sessionMetadata); + } + + setWorkflowId(workflowId: string): void { + this.workflowId = safeWorkflowIdentifier(workflowId); } /** - * Initialize the log stream (creates file and writes header) + * Open the combined-log stream and write its header, memoizing the in-flight promise so + * concurrent first calls on this instance share one open attempt instead of racing to open + * the file and write the header twice. */ async initialize(workflowId?: string): Promise { - if (workflowId) { - this.workflowId = workflowId; + if (workflowId !== undefined) this.setWorkflowId(workflowId); + if (this.logStream !== null) return; + if (this.initializePromise === null) { + this.initializePromise = this.openAndWriteHeader(); } + await this.initializePromise; + } - if (this.logStream.isOpen) { + private async openAndWriteHeader(): Promise { + try { + this.logStream = await LogStream.acquire(this.logPath); + const workflowId = safeWorkflowIdentifier(this.workflowId ?? this.sessionMetadata.id); + const header = [ + '================================================================================', + 'Shannon Pentest - Scan Log', + '================================================================================', + `Workflow ID: ${workflowId}`, + `Target URL: ${safeTargetUrl(this.sessionMetadata.webUrl)}`, + `Started: ${formatTimestamp()}`, + '================================================================================', + '', + ].join('\n'); + await this.logStream.appendIfAbsent(header, { + marker: 'Shannon Pentest - Scan Log', + scope: 'whole-file', + match: 'exact-line', + }); + } catch { + this.logStream = null; + warnLoggingFailure(); + } + } + + private formatLogTime(): string { + return new Date().toISOString().replace('T', ' ').slice(0, 19); + } + + /** + * Run a structural write against the combined-log stream, opening it first if needed. If the + * stream never opened, or `operation` throws, this resolves quietly after warning once: a + * logging fault must never surface as a thrown error to the phase, agent, or resume logic + * calling in. + */ + private async withStream(operation: (stream: LogStream) => Promise): Promise { + await this.initialize(); + if (this.logStream === null) return; + try { + await operation(this.logStream); + } catch { + warnLoggingFailure(); + } + } + + /** + * Append one idempotent structural marker to every existing per-agent file. Used for the resume + * boundary and the terminal marker, which each file needs so a `--agent` tail can tell a resume + * from a fresh run and can self-terminate. Best-effort: any file that fails is skipped. + */ + private async fanOutMarkerToAgents(markerLine: string): Promise { + const directory = agentsDir(this.logPath); + let entries: string[]; + try { + entries = await fsPromises.readdir(directory); + } catch { + // No agents directory yet — nothing has been projected, so there is nothing to mark. return; } - - await this.logStream.open(); - - // Write header only if file is new (empty) - const stats = await fs.stat(this.logStream.path).catch(() => null); - if (!stats || stats.size === 0) { - await this.writeHeader(); - } + await Promise.all( + entries + .filter((entry) => entry.endsWith('.log')) + .map(async (entry) => { + let stream: LogStream | undefined; + try { + stream = await LogStream.acquire(path.join(directory, entry)); + await stream.appendIfAbsent(`${markerLine}\n`, { + marker: markerLine, + scope: 'whole-file', + match: 'exact-line', + flush: true, + }); + } catch { + warnAgentLoggingFailure(); + } finally { + await stream?.release().catch(warnAgentLoggingFailure); + } + }), + ); } - /** - * Write header to log file - */ - private async writeHeader(): Promise { - const lines = [ - `================================================================================`, - `Shannon Pentest - Scan Log`, - `================================================================================`, - `Workflow ID: ${this.workflowId ?? this.sessionMetadata.id}`, - `Target URL: ${this.sessionMetadata.webUrl}`, - `Started: ${formatTimestamp()}`, - ]; - - lines.push(`================================================================================`, ``); - - return this.logStream.write(lines.join('\n')); - } - - /** - * Write resume header to log file when workflow is resumed - */ - async logResumeHeader(resumeInfo: { - previousWorkflowId: string; - newWorkflowId: string; - checkpointHash: string; - completedAgents: string[]; - }): Promise { - await this.ensureInitialized(); - + async logResumeBoundary(newWorkflowIdValue: string): Promise { + const newWorkflowId = safeWorkflowIdentifier(newWorkflowIdValue); + const marker = `New Workflow ID: ${newWorkflowId}`; const header = [ - ``, - `================================================================================`, - `RESUMED`, - `================================================================================`, - `Previous Workflow ID: ${resumeInfo.previousWorkflowId}`, - `New Workflow ID: ${resumeInfo.newWorkflowId}`, + '', + '================================================================================', + 'RESUMED', + '================================================================================', + marker, `Resumed At: ${formatTimestamp()}`, - `Checkpoint: ${resumeInfo.checkpointHash}`, - `Completed: ${resumeInfo.completedAgents.length} agents (${resumeInfo.completedAgents.join(', ')})`, - `================================================================================`, - ``, + '================================================================================', + '', ].join('\n'); - - return this.logStream.write(header); + await this.withStream((stream) => + stream.appendIfAbsent(header, { marker, scope: 'whole-file', match: 'exact-line', flush: true }), + ); + // A per-resume-distinct, per-file-idempotent boundary; a bare timestamp could not tell a + // Temporal retry from a new execution. + await this.fanOutMarkerToAgents(`--- RESUMED (${newWorkflowId}) ---`); } - /** - * Format timestamp for log line (UTC, human readable) - */ - private formatLogTime(): string { - const now = new Date(); - return now.toISOString().replace('T', ' ').slice(0, 19); + /** Append the resume checkpoint and completed-agent count beneath an already-written resume boundary. */ + async logResumeDetails(resumeInfo: { + readonly previousWorkflowId: string; + readonly newWorkflowId: string; + readonly checkpointHash: string; + readonly completedAgents: readonly string[]; + }): Promise { + const previousWorkflowId = safeWorkflowIdentifier(resumeInfo.previousWorkflowId); + const newWorkflowId = safeWorkflowIdentifier(resumeInfo.newWorkflowId); + const checkpointHash = /^[a-f0-9]{7,64}$/u.test(resumeInfo.checkpointHash) ? resumeInfo.checkpointHash : 'unknown'; + const completedAgents = resumeInfo.completedAgents.filter(isLoggableAgentName); + const marker = `Resume checkpoint (${newWorkflowId}): ${checkpointHash}`; + const details = [ + `Previous Workflow ID: ${previousWorkflowId}`, + marker, + `Completed: ${completedAgents.length} agents (${completedAgents.join(', ')})`, + ].join('\n'); + await this.withStream((stream) => + stream.appendIfAbsent(`${details}\n`, { marker, scope: 'whole-file', match: 'exact-line', flush: true }), + ); } - /** - * Log a phase transition event - */ - async logPhase(phase: string, event: 'start' | 'complete'): Promise { - await this.ensureInitialized(); - + async logPhase(phase: WorkflowPhase, event: 'start' | 'complete'): Promise { + if (!isWorkflowPhase(phase)) return; const action = event === 'start' ? 'Starting' : 'Completed'; - const line = `[${this.formatLogTime()}] [PHASE] ${action}: ${phase}\n`; + const suffix = `[PHASE] ${action}: ${phase}`; + const line = `${event === 'start' ? '\n' : ''}[${this.formatLogTime()}] ${suffix}\n`; + await this.withStream((stream) => + stream.appendIfAbsent(line, { marker: suffix, scope: 'current-execution', match: 'line-suffix' }), + ); + } - // Add blank line before phase start for readability - if (event === 'start') { - await this.logStream.write('\n'); - } + private async ensureAgentLease(agentName: LoggableAgentName): Promise { + const slug = safeAgentFileSlug(agentName); + if (slug === undefined || this.agentLeases.has(slug)) return; + const lease = await WorkflowLogger.openAgentLease(this.logPath, slug); + if (lease !== null) this.agentLeases.set(slug, lease); + } - await this.logStream.write(line); + private async releaseAgentLease(agentName: LoggableAgentName): Promise { + const slug = safeAgentFileSlug(agentName); + if (slug === undefined) return; + const lease = this.agentLeases.get(slug); + if (lease === undefined) return; + this.agentLeases.delete(slug); + await lease.release().catch(warnAgentLoggingFailure); } /** - * Log an agent event + * Release a pipeline agent's held per-agent file lease if one is open. Idempotent and + * best-effort: a backstop for an abnormal abort where the agent's end line never ran, so the + * file handle never outlives the activity. A normal end has already released it, making this a + * no-op. */ - async logAgent(agentName: string, event: 'start' | 'end', details?: AgentLogDetails): Promise { - await this.ensureInitialized(); + async releaseAgentLog(agentName: LoggableAgentName): Promise { + if (!isLoggableAgentName(agentName)) return; + await this.releaseAgentLease(agentName); + } + /** + * Write a pipeline agent's start or end line to the combined log and its per-agent file. The + * per-agent lease opens on `start`, before the header line, and is released on `end`, after the + * closing line, so the file stays open for the agent's own duration rather than reopening per + * trace line. + */ + async logAgent(agentName: LoggableAgentName, event: 'start' | 'end', details: AgentLogDetails = {}): Promise { + if (!isLoggableAgentName(agentName)) return; let message: string; - if (event === 'start') { - const attempt = details?.attemptNumber ?? 1; + const attempt = isSafeCount(details.attemptNumber ?? 1) ? (details.attemptNumber ?? 1) : 1; message = `${agentName}: Starting (attempt ${attempt})`; } else { - const parts: string[] = [`${agentName}:`]; - - if (details?.success === false) { - parts.push('Failed'); - if (details?.error) { - parts.push(`- ${details.error}`); - } - } else { - parts.push('Completed'); + const status = details.success === false ? 'Failed' : 'Completed'; + const code = details.success === false && details.errorCode !== undefined ? ` (${details.errorCode})` : ''; + const outcomeDetails: string[] = []; + if (details.duration_ms !== undefined) outcomeDetails.push(formatDuration(Math.max(0, details.duration_ms))); + if (details.cost_usd !== undefined && Number.isFinite(details.cost_usd)) { + outcomeDetails.push(`$${Math.max(0, details.cost_usd).toFixed(4)}`); } - - if (details?.duration_ms !== undefined) { - parts.push(`(${formatDuration(details.duration_ms)}`); - if (details?.cost_usd !== undefined) { - parts.push(`$${details.cost_usd.toFixed(2)})`); - } else { - parts.push(')'); - } - } - - message = parts.join(' '); + const suffix = outcomeDetails.length === 0 ? '' : ` (${outcomeDetails.join(', ')})`; + message = `${agentName}: ${status}${code}${suffix}`; } - - const line = `[${this.formatLogTime()}] [AGENT] ${message}\n`; - await this.logStream.write(line); + // Open the lease before the start line so the file's header lands first and the agent's + // trace lines through the run ride an already-open handle. + if (event === 'start') await this.ensureAgentLease(agentName); + const line = `[${this.formatLogTime()}] [AGENT] ${message}`; + await this.withStream((stream) => stream.write(`${line}\n`)); + const slug = safeAgentFileSlug(agentName); + if (slug !== undefined) await WorkflowLogger.fanOutLine(this.logPath, slug, line, false); + // Release after the end line so the closing line still rides the lease. + if (event === 'end') await this.releaseAgentLease(agentName); } - /** - * Log a general event - */ - async logEvent(eventType: string, message: string): Promise { - await this.ensureInitialized(); - - const line = `[${this.formatLogTime()}] [${eventType.toUpperCase()}] ${message}\n`; - await this.logStream.write(line); + /** A one-line, closed-vocabulary error record for an agent attempt, written to both the combined and per-agent logs. */ + async logAgentError( + agentName: LoggableAgentName, + code: ErrorCode, + category: string, + attempt: number, + durationMs: number, + turns: number, + ): Promise { + if (!isLoggableAgentName(agentName)) return; + const safe = safeErrorFromCode(code); + const safeAttempt = isSafeCount(attempt) ? attempt : 0; + const safeTurns = isSafeCount(turns) ? turns : 0; + const safeDuration = Number.isFinite(durationMs) ? Math.max(0, durationMs) : 0; + const safeCategory = /^(?:config|network|prompt|filesystem|validation|unknown)$/u.test(category) + ? category + : 'unknown'; + const line = `[${this.formatLogTime()}] [${agentName}] [ERROR] ${safe.code} (${safeCategory}, attempt ${safeAttempt}, ${formatDuration(safeDuration)}, ${safeTurns} turns)`; + await this.withStream((stream) => stream.write(`${line}\n`)); + const slug = safeAgentFileSlug(agentName); + if (slug !== undefined) await WorkflowLogger.fanOutLine(this.logPath, slug, line, false); } - /** - * Log an error - */ - async logError(error: Error, context?: string): Promise { - await this.ensureInitialized(); - - const contextStr = context ? ` (${context})` : ''; - const line = `[${this.formatLogTime()}] [ERROR] ${error.message}${contextStr}\n`; - await this.logStream.write(line); - } - - /** - * Truncate string to max length with ellipsis - */ - private truncate(str: string, maxLen: number): string { - if (str.length <= maxLen) return str; - return `${str.slice(0, maxLen - 3)}...`; - } - - /** - * Format tool parameters for human-readable display - */ - private formatToolParams(toolName: string, params: unknown): string { - if (!params || typeof params !== 'object') { - return ''; - } - - const p = params as Record; - - // Tool-specific formatting for common tools - switch (toolName) { - case 'Bash': - if (p.command) { - return this.truncate(String(p.command).replace(/\n/g, ' '), 100); - } - break; - case 'Read': - if (p.file_path) { - return String(p.file_path); - } - break; - case 'Write': - if (p.file_path) { - return String(p.file_path); - } - break; - case 'Edit': - if (p.file_path) { - return String(p.file_path); - } - break; - case 'Glob': - if (p.pattern) { - return String(p.pattern); - } - break; - case 'Grep': - if (p.pattern) { - const path = p.path ? ` in ${p.path}` : ''; - return `"${this.truncate(String(p.pattern), 50)}"${path}`; - } - break; - case 'WebFetch': - if (p.url) { - return String(p.url); - } - break; - } - - // Default: show first string-valued param truncated - for (const [key, val] of Object.entries(p)) { - if (typeof val === 'string' && val.length > 0) { - return `${key}=${this.truncate(val, 60)}`; - } - } - - return ''; - } - - /** - * Log tool start event - */ - async logToolStart(agentName: string, toolName: string, parameters: unknown): Promise { - await this.ensureInitialized(); - - const params = this.formatToolParams(toolName, parameters); - const paramStr = params ? `: ${params}` : ''; - const line = `[${this.formatLogTime()}] [${agentName}] [TOOL] ${toolName}${paramStr}\n`; - await this.logStream.write(line); - } - - /** - * Log LLM response - */ - async logLlmResponse(agentName: string, turn: number, content: string): Promise { - await this.ensureInitialized(); - - // Show full content, replacing newlines with escaped version for single-line output - const escaped = content.replace(/\n/g, '\\n'); - const line = `[${this.formatLogTime()}] [${agentName}] [LLM] Turn ${turn}: ${escaped}\n`; - await this.logStream.write(line); - } - - /** - * Format a pipe-delimited error string into indented multi-line display. - * - * Input: "phase context|ErrorType|message|Hint: ..." - * Output: "Error: phase context\n ErrorType\n ..." - */ - private formatErrorBlock(errorString: string): string { - const label = 'Error: '; - const indent = ' '.repeat(label.length); - - // Segments are delimited by '|'; a segment's own embedded newlines (e.g. a multi-line - // validation message) become their own lines so each aligns under the label. - const lines = errorString - .split(/[|\n]/) - .map((segment) => segment.trim()) - .filter((segment) => segment.length > 0); - - return `${lines.map((line, i) => (i === 0 ? `${label}${line}` : `${indent}${line}`)).join('\n')}\n`; - } - - /** - * Log workflow completion with full summary - */ async logWorkflowComplete(summary: WorkflowSummary): Promise { - await this.ensureInitialized(); - - // Each terminal status prints its own header so partial and cancelled runs are never - // mislabelled as full successes or failures. The CLI log tailer stops on exactly these - // headings (COMPLETION_PATTERN in apps/cli/src/commands/logs.ts); adding one here without - // adding it there strands `shannon logs` on a finished scan. - const STATUS_HEADERS: Record = { + const statusHeaders: Record = { completed: 'COMPLETED', partial: 'PARTIAL', cancelled: 'CANCELLED', failed: 'FAILED', }; - const status = STATUS_HEADERS[summary.status]; - - // completedAgents and skippedAgents are mutually exclusive: an agent that was skipped - // because its class had nothing to exploit is tracked only in skippedAgents. - const skippedAgents = summary.skippedAgents ?? []; - const ranCount = summary.completedAgents.length; - - const lines: string[] = [ + const status = statusHeaders[summary.status]; + const completedAgents = summary.completedAgents.filter(isLoggableAgentName); + const skippedAgents = (summary.skippedAgents ?? []).filter(isLoggableAgentName); + const lines = [ '', '================================================================================', `Scan ${status}`, '────────────────────────────────────────', - `Workflow ID: ${this.workflowId ?? this.sessionMetadata.id}`, + `Workflow ID: ${safeWorkflowIdentifier(this.workflowId ?? this.sessionMetadata.id)}`, `Status: ${summary.status}`, - `Duration: ${formatDuration(summary.totalDurationMs)}`, - `Total Cost: $${summary.totalCostUsd.toFixed(4)}`, - `Agents: ${ranCount} ran, ${skippedAgents.length} skipped`, + `Duration: ${formatDuration(Math.max(0, summary.totalDurationMs))}`, + `Total Cost: $${Math.max(0, summary.totalCostUsd).toFixed(4)}`, + `Agents: ${completedAgents.length} ran, ${skippedAgents.length} skipped`, ]; if (summary.usageAccountingComplete === false) { lines.push('Cost Note: Cost is incomplete — some background work is not included in this total.'); } - - if (summary.error) { - lines.push(this.formatErrorBlock(summary.error).trimEnd()); + if (summary.errorCode !== undefined) { + const safeError = safeErrorFromCode(summary.errorCode); + lines.push(`Error: ${safeError.message}`); + lines.push(` ${safeError.code}`); } - if (summary.partialReasons !== undefined && summary.partialReasons.length > 0) { - lines.push(''); - lines.push('Why this scan is partial:'); - for (const reason of summary.partialReasons) { - lines.push(` - ${reason.message}`); - } - // The reason above says what degraded; these three name the agentic-SAST failure - // behind it, under the same labels the terminal and worker output use. - if (summary.agenticSastFailedStage !== undefined) { - lines.push(` Agentic SAST stopped at: ${summary.agenticSastFailedStage}`); - } - if (summary.agenticSastFailureMessage !== undefined) { - lines.push(` What happened: ${summary.agenticSastFailureMessage}`); - } - if (summary.agenticSastErrorCode !== undefined) { - lines.push(` Reference code (for a bug report): ${summary.agenticSastErrorCode}`); - } + const partialMessages = (summary.partialReasons ?? []) + .map(safeReasonMessage) + .filter((message): message is string => message !== undefined); + if (partialMessages.length > 0) { + lines.push('', 'Why this scan is partial:'); + for (const message of partialMessages) lines.push(` - ${message}`); } - if (summary.completedAgents.length > 0 || skippedAgents.length > 0) { - lines.push(''); - lines.push('Agent Breakdown:'); + const failedStage = safeAgenticSastStageLabel(summary.agenticSastFailedStage); + const failureMessage = summary.agenticSastFailureMessage; + if (failedStage !== undefined && failureMessage !== undefined && isCapellaSafeFailureMessage(failureMessage)) { + lines.push('', `Agentic SAST stopped at: ${failedStage}`); + lines.push(`What happened: ${failureMessage}`); + const failureCode = safeAgenticSastCode(summary.agenticSastErrorCode); + if (failureCode !== undefined) lines.push(`Reference code (for a bug report): ${failureCode}`); + } - for (const agentName of summary.completedAgents) { + if (completedAgents.length > 0 || skippedAgents.length > 0) { + lines.push('', 'Agent Breakdown:'); + for (const agentName of completedAgents) { const metrics = summary.agentMetrics[agentName]; - if (metrics) { - const duration = formatDuration(metrics.durationMs); - const cost = metrics.costUsd !== null ? `$${metrics.costUsd.toFixed(4)}` : 'N/A'; - lines.push(` - ${agentName} (${duration}, ${cost})`); - } else { + if (metrics === undefined) { lines.push(` - ${agentName}`); + continue; } + const cost = metrics.costUsd === null ? 'N/A' : `$${Math.max(0, metrics.costUsd).toFixed(4)}`; + lines.push(` - ${agentName} (${formatDuration(Math.max(0, metrics.durationMs))}, ${cost})`); } - for (const agentName of skippedAgents) { - lines.push(` - ${agentName} (skipped — nothing to exploit)`); - } + for (const agentName of skippedAgents) lines.push(` - ${agentName} (skipped — nothing to exploit)`); } - for (const agentName of skippedAgents) { - lines.push(` - ${agentName} (skipped — nothing to exploit)`); - } - lines.push('================================================================================'); - // Single atomic write to prevent interleaved/duplicate output in log tailers - await this.logStream.write(`${lines.join('\n')}\n`); + const marker = `Scan ${status}`; + await this.withStream((stream) => + stream.appendIfAbsent(`${lines.join('\n')}\n`, { + marker, + scope: 'current-execution', + match: 'exact-line', + flush: true, + }), + ); + // The same bare heading the combined log carries, so a `--agent` tail terminates on the file + // alone when Temporal's status is no longer available (e.g. past its retention window). + await this.fanOutMarkerToAgents(marker); } /** - * Ensure initialized (helper for lazy initialization) - */ - private async ensureInitialized(): Promise { - if (!this.logStream.isOpen) { - await this.initialize(); - } - } - - /** - * Close the log stream + * Release every lease this instance currently holds, then release the combined-log stream + * itself. This tears down all of this instance's open per-agent leases unconditionally, not + * just one caller's, so an instance must never be shared between agents running concurrently: + * one agent's `close()` would sever another's still-open lease. Callers close after each + * logical unit of work (an agent's end, a phase boundary) for exactly this reason, and each + * concurrent agent is given its own `WorkflowLogger`/`AuditSession` instance rather than a + * shared one. */ async close(): Promise { - return this.logStream.close(); + for (const [slug, lease] of this.agentLeases) { + this.agentLeases.delete(slug); + await lease.release().catch(warnAgentLoggingFailure); + } + await this.initializePromise; + if (this.logStream === null) { + this.initializePromise = null; + return; + } + await this.logStream.release().catch(warnLoggingFailure); + this.logStream = null; + this.initializePromise = null; } } diff --git a/apps/worker/src/services/agent-execution.ts b/apps/worker/src/services/agent-execution.ts index 5eedcde2..ff076ba2 100644 --- a/apps/worker/src/services/agent-execution.ts +++ b/apps/worker/src/services/agent-execution.ts @@ -25,6 +25,7 @@ import { fs, path } from 'zx'; import { type PiPromptResult, runPiPrompt, validateAgentOutput } from '../ai/pi/pi-executor.js'; import { createQueueSubmitTool, getQueueFilename } from '../ai/queue-schemas.js'; import type { AuditSession } from '../audit/index.js'; +import { safeErrorFromCode } from '../audit/safe-fields.js'; import { authStateFile } from '../audit/utils.js'; import { AGENTS } from '../session-manager.js'; import type { ActivityLogger } from '../types/activity-logger.js'; @@ -234,141 +235,152 @@ export class AgentExecutionService { } // 4. Start audit logging - await auditSession.startAgent(agentName, prompt, attemptNumber); + await auditSession.startAgent(agentName, attemptNumber); - // 5. Execute agent. Vuln agents get a submit tool that captures the structured - // exploitation queue (pi has no JSON-schema output format). - const submitTool = createQueueSubmitTool(agentName, distributedConfig?.exploit ?? true); - const result: PiPromptResult = await runPiPrompt( - prompt, - repoPath, - '', // context - agentName, // description - agentName, - auditSession, - logger, - customTools, - path.relative(repoPath, deliverablesPath), - cancellationSignal, - submitTool, - ); - - // 6. Handle execution failure - if (!result.success) { - const errorCode = errorCodeFromResult(result); - return this.failAgent(agentName, deliverablesPath, auditSession, logger, { + // startAgent opens this agent's per-agent log lease. Run the rest under try/finally so an + // unexpected throw between here and the agent's end still releases that lease. + try { + // 5. Execute agent. Vuln agents get a submit tool that captures the structured + // exploitation queue (pi has no JSON-schema output format). + const submitTool = createQueueSubmitTool(agentName, distributedConfig?.exploit ?? true); + const result: PiPromptResult = await runPiPrompt( + prompt, + repoPath, + '', // context + agentName, // description + agentName, + auditSession, + logger, + customTools, + path.relative(repoPath, deliverablesPath), + cancellationSignal, + submitTool, attemptNumber, - result, - rollbackReason: 'execution failure', - errorMessage: result.error || 'Agent execution failed', - errorCode, - category: categoryForErrorCode(errorCode), - retryable: result.retryable ?? true, - context: { agentName, originalError: result.error }, - }); - } + ); - // 8-11. Write structured output, validate, render, and commit under one repo lock so - // the write→validate→commit sequence is atomic against concurrent sibling agents. - let commitHash: string | undefined; - const finalizationError = await withGitRepoLock(async (): Promise => { - // Every step below must surface as a returned error rather than a throw: only the - // returned path rolls the workspace back and records the failed attempt. - try { - // 8. Write structured output to disk (vuln agents only) from the executor's capture - const queueFilename = getQueueFilename(agentName); - if (submitTool && queueFilename && result.structuredOutput !== undefined) { - await fs.ensureDir(deliverablesPath); - const queuePath = path.join(deliverablesPath, queueFilename); - await fs.writeFile(queuePath, JSON.stringify(result.structuredOutput, null, 2), 'utf8'); - logger.info(`Wrote structured output queue to ${queueFilename}`); - } + // 6. Handle execution failure + if (!result.success) { + const errorCode = errorCodeFromResult(result); + return this.failAgent(agentName, deliverablesPath, auditSession, logger, { + attemptNumber, + result, + rollbackReason: 'execution failure', + errorMessage: result.error || 'Agent execution failed', + errorCode, + category: categoryForErrorCode(errorCode), + retryable: result.retryable ?? true, + context: { agentName, originalError: result.error }, + }); + } - // 9. Validate output - const validationPassed = await validateAgentOutput(result, agentName, deliverablesPath, logger); - if (!validationPassed) { + // 8-11. Write structured output, validate, render, and commit under one repo lock so + // the write→validate→commit sequence is atomic against concurrent sibling agents. + let commitHash: string | undefined; + const finalizationError = await withGitRepoLock(async (): Promise => { + // Every step below must surface as a returned error rather than a throw: only the + // returned path rolls the workspace back and records the failed attempt. + try { + // 8. Write structured output to disk (vuln agents only) from the executor's capture + const queueFilename = getQueueFilename(agentName); + if (submitTool && queueFilename && result.structuredOutput !== undefined) { + await fs.ensureDir(deliverablesPath); + const queuePath = path.join(deliverablesPath, queueFilename); + await fs.writeFile(queuePath, JSON.stringify(result.structuredOutput, null, 2), 'utf8'); + logger.info(`Wrote structured output queue to ${queueFilename}`); + } + + // 9. Validate output + const validationPassed = await validateAgentOutput(result, agentName, deliverablesPath, logger); + if (!validationPassed) { + return new PentestError( + `Agent ${agentName} failed output validation`, + 'validation', + true, + { agentName, deliverableFilename: AGENTS[agentName].deliverableFilename }, + ErrorCode.OUTPUT_VALIDATION_FAILED, + ); + } + + // 10. Render the deliverable to disk so the success commit below stages it + if (writeDeliverable) { + await writeDeliverable(deliverablesPath, { + ...(result.model !== undefined && { model: result.model }), + }); + } + + // 11. Success - commit deliverables (scoped) and capture the checkpoint hash + const commitResult = await commitGitSuccess(deliverablesPath, agentName, logger, gitPaths); + if (!commitResult.success) { + return gitFailureForAgent(agentName, 'commit successful results', commitResult.error); + } + commitHash = commitResult.commitHash; + // recordReportDraft requires a checkpoint hash to persist the draft durably; without one + // a resumed workflow would have nothing to reconcile the draft against. + if (successDisposition === 'report-draft' && commitHash === undefined) { + return new PentestError( + 'The report was written but could not be saved. Re-running this workspace retries the reporting phase without repeating the analysis.', + 'filesystem', + false, + { agentName }, + ErrorCode.GIT_CHECKPOINT_FAILED, + ); + } + return null; + } catch (error) { + if (error instanceof PentestError) return error; + const errorMessage = error instanceof Error ? error.message : String(error); return new PentestError( - `Agent ${agentName} failed output validation`, + `Agent ${agentName} post-processing failed: ${errorMessage}`, 'validation', true, - { agentName, deliverableFilename: AGENTS[agentName].deliverableFilename }, + { agentName, originalError: errorMessage }, ErrorCode.OUTPUT_VALIDATION_FAILED, ); } - - // 10. Render the deliverable to disk so the success commit below stages it - if (writeDeliverable) { - await writeDeliverable(deliverablesPath, { - ...(result.model !== undefined && { model: result.model }), - }); - } - - // 11. Success - commit deliverables (scoped) and capture the checkpoint hash - const commitResult = await commitGitSuccess(deliverablesPath, agentName, logger, gitPaths); - if (!commitResult.success) { - return gitFailureForAgent(agentName, 'commit successful results', commitResult.error); - } - commitHash = commitResult.commitHash; - if (successDisposition === 'report-draft' && commitHash === undefined) { - return new PentestError( - 'The report was written but could not be saved. Re-running this workspace retries the reporting phase without repeating the analysis.', - 'filesystem', - false, - { agentName }, - ErrorCode.GIT_CHECKPOINT_FAILED, - ); - } - return null; - } catch (error) { - if (error instanceof PentestError) return error; - const errorMessage = error instanceof Error ? error.message : String(error); - return new PentestError( - `Agent ${agentName} post-processing failed: ${errorMessage}`, - 'validation', - true, - { agentName, originalError: errorMessage }, - ErrorCode.OUTPUT_VALIDATION_FAILED, - ); - } - }); - - if (finalizationError) { - const rollbackReason = - finalizationError.code === ErrorCode.OUTPUT_VALIDATION_FAILED - ? 'validation failure' - : 'post-processing failure'; - return this.failAgent(agentName, deliverablesPath, auditSession, logger, { - attemptNumber, - result, - rollbackReason, - errorMessage: finalizationError.message, - errorCode: finalizationError.code ?? ErrorCode.AGENT_EXECUTION_FAILED, - category: finalizationError.type, - retryable: finalizationError.retryable, - context: { agentName, ...finalizationError.context }, }); - } - const endResult: AgentEndResult = { - attemptNumber, - duration_ms: result.duration, - cost_usd: result.cost || 0, - input_tokens: result.inputTokens, - output_tokens: result.outputTokens, - cache_read_tokens: result.cacheReadTokens, - cache_write_tokens: result.cacheWriteTokens, - turns: result.turns, - success: true, - model: result.model, - ...(commitHash && { checkpoint: commitHash }), - }; - if (successDisposition === 'report-draft') { - await auditSession.endReportDraft(endResult); - } else { - await auditSession.endAgent(agentName, endResult); - } + if (finalizationError) { + const rollbackReason = + finalizationError.code === ErrorCode.OUTPUT_VALIDATION_FAILED + ? 'validation failure' + : 'post-processing failure'; + return this.failAgent(agentName, deliverablesPath, auditSession, logger, { + attemptNumber, + result, + rollbackReason, + errorMessage: finalizationError.message, + errorCode: finalizationError.code ?? ErrorCode.AGENT_EXECUTION_FAILED, + category: finalizationError.type, + retryable: finalizationError.retryable, + context: { agentName, ...finalizationError.context }, + }); + } - return ok(endResult); + const endResult: AgentEndResult = { + attemptNumber, + duration_ms: result.duration, + cost_usd: result.cost || 0, + input_tokens: result.inputTokens, + output_tokens: result.outputTokens, + cache_read_tokens: result.cacheReadTokens, + cache_write_tokens: result.cacheWriteTokens, + turns: result.turns, + success: true, + model: result.model, + ...(commitHash && { checkpoint: commitHash }), + }; + if (successDisposition === 'report-draft') { + await auditSession.endReportDraft(endResult); + } else { + await auditSession.endAgent(agentName, endResult); + } + + return ok(endResult); + } finally { + // Normal completion already released this agent's log lease (endAgent → close()); this is the + // backstop for an unexpected throw between start and end. Idempotent and best-effort. + await auditSession.releaseAgentLog(agentName); + } } private async failAgent( @@ -385,6 +397,7 @@ export class AgentExecutionService { getAgentGitPaths(agentName), ); + const safeError = safeErrorFromCode(opts.errorCode, opts.category); const endResult: AgentEndResult = { attemptNumber: opts.attemptNumber, duration_ms: opts.result.duration, @@ -396,7 +409,8 @@ export class AgentExecutionService { turns: opts.result.turns, success: false, model: opts.result.model, - error: opts.errorMessage, + error: safeError.message, + errorCode: safeError.code, }; await auditSession.endAgent(agentName, endResult); diff --git a/apps/worker/src/services/validate-authentication.ts b/apps/worker/src/services/validate-authentication.ts index f0f81277..6c90f72f 100644 --- a/apps/worker/src/services/validate-authentication.ts +++ b/apps/worker/src/services/validate-authentication.ts @@ -18,6 +18,7 @@ import { Type } from 'typebox'; import { runPiPrompt } from '../ai/pi/pi-executor.js'; import type { CapturedSubmitTool } from '../ai/submit-tool.js'; import type { AuditSession } from '../audit/index.js'; +import { safeErrorFromUnknown } from '../audit/safe-fields.js'; import { authStateFile } from '../audit/utils.js'; import type { ActivityLogger } from '../types/activity-logger.js'; import type { AgentEndResult } from '../types/audit.js'; @@ -136,7 +137,7 @@ export async function validateAuthentication( promptDir, ); - await auditSession.startAgent(AGENT_NAME, prompt, attemptNumber); + await auditSession.startAgent(AGENT_NAME, attemptNumber); const startTime = Date.now(); const submitTool = createAuthSubmitTool(); @@ -152,6 +153,7 @@ export async function validateAuthentication( deliverablesSubdir, cancellationSignal, submitTool, + attemptNumber, ); let classification = classifyResult(result, authentication); @@ -164,13 +166,14 @@ export async function validateAuthentication( } const durationMs = Date.now() - startTime; + const safeError = classification.ok ? undefined : safeErrorFromUnknown(classification.error); const endResult: AgentEndResult = { attemptNumber, duration_ms: durationMs, cost_usd: result.cost || 0, success: classification.ok, ...(result.model !== undefined && { model: result.model }), - ...(!classification.ok && { error: classification.error.message }), + ...(safeError !== undefined && { error: safeError.message, errorCode: safeError.code }), }; await auditSession.endAgent(AGENT_NAME, endResult); diff --git a/apps/worker/src/temporal/activities.ts b/apps/worker/src/temporal/activities.ts index a75801e4..e4d975ac 100644 --- a/apps/worker/src/temporal/activities.ts +++ b/apps/worker/src/temporal/activities.ts @@ -23,6 +23,7 @@ import { syncPermissionSystemConfig } from '../ai/pi/permission-system.js'; import { writePlaywrightStealthConfig } from '../ai/playwright-config-writer.js'; import { AuditSession } from '../audit/index.js'; import type { ResumeAttempt } from '../audit/metrics-tracker.js'; +import type { WorkflowPhase } from '../audit/safe-fields.js'; import { authStateFile, generateAuditPath, type SessionMetadata } from '../audit/utils.js'; import type { WorkflowSummary } from '../audit/workflow-logger.js'; import type { CheckpointContext } from '../interfaces/checkpoint-provider.js'; @@ -1803,7 +1804,8 @@ export async function restoreGitCheckpoint( export async function registerResumeAttempt(input: ActivityInput, terminatedWorkflows: string[]): Promise { const sessionMetadata = buildSessionMetadata(input); const auditSession = new AuditSession(sessionMetadata); - await auditSession.initialize(); + await auditSession.initialize(input.workflowId); + await auditSession.logResumeBoundary(input.workflowId); await auditSession.addResumeAttempt(input.workflowId, terminatedWorkflows); } @@ -1817,8 +1819,8 @@ export async function recordResumeAttempt( const auditSession = new AuditSession(sessionMetadata); await auditSession.initialize(); - // session.json entry already added by registerResumeAttempt; here we only write the workflow.log header. - await auditSession.logResumeHeader({ + // The execution boundary was flushed by registerResumeAttempt before session.json publication. + await auditSession.logResumeDetails({ previousWorkflowId, newWorkflowId: input.workflowId, checkpointHash, @@ -1831,7 +1833,7 @@ export async function recordResumeAttempt( */ export async function logPhaseTransition( input: ActivityInput, - phase: string, + phase: WorkflowPhase, event: 'start' | 'complete', ): Promise { const sessionMetadata = buildSessionMetadata(input); diff --git a/apps/worker/src/temporal/summary-mapper.ts b/apps/worker/src/temporal/summary-mapper.ts index c2ed7620..5c1c89ec 100644 --- a/apps/worker/src/temporal/summary-mapper.ts +++ b/apps/worker/src/temporal/summary-mapper.ts @@ -52,6 +52,6 @@ export function toWorkflowSummary( ...(agenticSastFailedStage !== undefined && { agenticSastFailedStage }), ...(agenticSastFailureMessage !== undefined && { agenticSastFailureMessage }), ...(agenticSastErrorCode !== undefined && { agenticSastErrorCode }), - ...(state.error && { error: state.error }), + ...(state.errorCode !== undefined && { errorCode: state.errorCode }), }; } diff --git a/apps/worker/src/temporal/worker.ts b/apps/worker/src/temporal/worker.ts index ea1c2435..77d37880 100644 --- a/apps/worker/src/temporal/worker.ts +++ b/apps/worker/src/temporal/worker.ts @@ -33,12 +33,19 @@ import { Client, Connection, type WorkflowHandle, WorkflowNotFoundError } from ' import { bundleWorkflowCode, NativeConnection, Worker } from '@temporalio/worker'; import dotenv from 'dotenv'; import { DEFAULT_MODEL_SPEC } from '../ai/models.js'; +import { capellaTerminalStageLabel, isCapellaSafeFailureMessage } from '../ai/sast/capella/safe-failures.js'; import { capellaActivities, mergeActivityRegistries } from '../ai/sast/capella/temporal/registry.js'; import { CAPELLA_FORMAT_VERSION, CAPELLA_PROMPT_SET_VERSION } from '../ai/sast/capella/types.js'; import { sanitizeHostname } from '../audit/utils.js'; import { distributeConfig, parseConfig } from '../config-parser.js'; import { deliverablesDir, resolveSessionJsonPath } from '../paths.js'; -import { SAFE_RUN_STATE_MESSAGES, workspaceExploitMismatchMessage } from '../types/run-state.js'; +import { + ACCEPTED_CAPELLA_FAILURE_STAGES, + isPartialReason, + projectPartialReasons, + SAFE_RUN_STATE_MESSAGES, + workspaceExploitMismatchMessage, +} from '../types/run-state.js'; import { fileExists, readJson } from '../utils/file-io.js'; import { assembleReportActivity, @@ -88,6 +95,27 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url)); const PROGRESS_QUERY = 'getProgress'; +function safeFailureCode(value: string | undefined): string | undefined { + if (value !== undefined && /^[A-Z][A-Z0-9_]{0,63}$/u.test(value)) return value; + return undefined; +} + +function safePartialReasonMessage(reason: PipelineState['partialReasons'][number]): string | undefined { + if (reason.code === 'agentic_sast_reduced') return 'Agentic SAST completed with reduced coverage.'; + const candidate = { + code: reason.code, + ...(reason.vulnerabilityClass !== undefined && { vulnerabilityClass: reason.vulnerabilityClass }), + ...(reason.stage !== undefined && { stage: reason.stage }), + ...(reason.reductionReason !== undefined && { reductionReason: reason.reductionReason }), + ...(reason.omittedCount !== undefined && { omittedCount: reason.omittedCount }), + ...(reason.consideredCount !== undefined && { consideredCount: reason.consideredCount }), + ...(reason.classifiedCount !== undefined && { classifiedCount: reason.classifiedCount }), + ...(reason.affectedBatchCount !== undefined && { affectedBatchCount: reason.affectedBatchCount }), + }; + if (!isPartialReason(candidate)) return undefined; + return projectPartialReasons([candidate])[0]?.message; +} + // The ordinary activity names. This frozen list is one of three that together form the // registered activity set the CLI status reader mirrors: the Capella names in // ai/sast/capella/temporal/activity-types.ts and the reconciliation names in @@ -417,7 +445,7 @@ async function resolveWorkspace(client: Client, args: CliArgs, expectedExploit: } if (!isValidWorkspaceName(workspace)) { - console.error(`ERROR: Invalid workspace name: "${workspace}"`); + console.error('ERROR: Invalid workspace name.'); console.error(' Must be 1-128 characters, alphanumeric/hyphens/underscores, starting with alphanumeric'); process.exit(1); } @@ -467,8 +495,7 @@ async function loadOrchestrationConfig(configPath: string | undefined): Promise< } catch (error) { // A broken config must fail the run, not silently fall back to empty // defaults that quietly change scope (vuln classes, exploit, retries). - const message = error instanceof Error ? error.message : String(error); - console.error(`Failed to parse config ${configPath}: ${message}`); + console.error('Worker configuration could not be loaded. Reference code: CONFIG_VALIDATION_FAILED'); process.exit(1); } } @@ -523,15 +550,23 @@ async function waitForWorkflowResult( if (result.status === 'partial') { console.log('\nScan completed with gaps (partial). The reasons are listed below.'); for (const reason of result.partialReasons) { - console.log(` - ${reason.message}`); + const message = safePartialReasonMessage(reason); + if (message !== undefined) console.log(` - ${message}`); } // The reason above says a class of coverage degraded; these three name the sanitized // agentic-SAST failure behind it, under the same labels every other surface uses. if (result.agenticSast.status === 'failed') { - console.log(` Agentic SAST stopped at: ${result.agenticSast.failedStageLabel}`); - console.log(` What happened: ${result.agenticSast.error}`); - if (result.agenticSast.errorCode !== undefined) { - console.log(` Reference code (for a bug report): ${result.agenticSast.errorCode}`); + const stage = ACCEPTED_CAPELLA_FAILURE_STAGES.includes(result.agenticSast.failedStage) + ? capellaTerminalStageLabel(result.agenticSast.failedStage) + : 'orchestration'; + const message = isCapellaSafeFailureMessage(result.agenticSast.error) + ? result.agenticSast.error + : 'An agentic SAST step failed.'; + console.log(` Agentic SAST stopped at: ${stage}`); + console.log(` What happened: ${message}`); + const code = safeFailureCode(result.agenticSast.errorCode); + if (code !== undefined) { + console.log(` Reference code (for a bug report): ${code}`); } } } else if (result.status === 'cancelled') { @@ -559,9 +594,9 @@ async function waitForWorkflowResult( } } } - } catch (error) { + } catch { clearInterval(progressInterval); - console.error('\nPipeline failed:', error); + console.error('\nScan failed. Reference code: WORKFLOW_FAILED'); process.exit(1); } } @@ -638,8 +673,8 @@ async function run(): Promise { const invokedPath = process.argv[1] ? path.resolve(process.argv[1]) : undefined; if (invokedPath === fileURLToPath(import.meta.url)) { - run().catch((err) => { - console.error('Worker failed:', err); + run().catch(() => { + console.error('Worker failed. Reference code: WORKER_FAILED'); process.exit(1); }); } diff --git a/apps/worker/src/temporal/workflow-errors.ts b/apps/worker/src/temporal/workflow-errors.ts index 2dbe3fe5..d7db4f8d 100644 --- a/apps/worker/src/temporal/workflow-errors.ts +++ b/apps/worker/src/temporal/workflow-errors.ts @@ -9,6 +9,8 @@ * Pure functions with no side effects — safe for Temporal workflow sandbox. */ +import { WORKFLOW_PHASES } from '../audit/safe-fields.js'; +import { ALL_AGENTS } from '../types/agents.js'; import { ErrorCode } from '../types/errors.js'; /** @@ -26,6 +28,12 @@ const ERROR_TYPE_TO_CODE: Record = { AgentExecutionError: ErrorCode.AGENT_EXECUTION_FAILED, GitError: ErrorCode.GIT_CHECKPOINT_FAILED, InvalidTargetError: ErrorCode.TARGET_UNREACHABLE, + AuthLoginFailedError: ErrorCode.AUTH_LOGIN_FAILED, + PipelineFailedError: ErrorCode.AGENT_EXECUTION_FAILED, + ReportDraftError: ErrorCode.AGENT_EXECUTION_FAILED, + ReportSarifRenderError: ErrorCode.OUTPUT_VALIDATION_FAILED, + IncompatibleWorkspaceError: ErrorCode.CONFIG_VALIDATION_FAILED, + WorkspaceNotFoundError: ErrorCode.CONFIG_NOT_FOUND, }; export function classifyErrorCode(error: unknown): ErrorCode | undefined { @@ -54,41 +62,47 @@ const REMEDIATION_HINTS: Record = { PipelineFailedError: 're-run the same -w to retry from the last checkpoint.', }; +const SAFE_WORKFLOW_FAILURE_MESSAGES: Readonly> = { + AuthenticationError: 'Provider authentication failed.', + ConfigurationError: 'The scan configuration is invalid.', + OutputValidationError: 'A scan step returned an unusable result.', + AgentExecutionError: 'An agent could not complete its work.', + GitError: 'The scan checkpoint could not be updated.', + InvalidTargetError: 'The target could not be reached.', + AuthLoginFailedError: 'The configured login could not be completed.', + PipelineFailedError: 'The vulnerability analysis phase could not be completed.', + ReportDraftError: 'The report could not be saved.', + ReportSarifRenderError: 'The report SARIF output could not be rendered.', + IncompatibleWorkspaceError: 'This workspace cannot be resumed.', + WorkspaceNotFoundError: 'The requested workspace was not found.', +}; + +const WORKFLOW_PHASE_SET = new Set(WORKFLOW_PHASES); +const AGENT_NAME_SET = new Set(ALL_AGENTS); + /** - * Walk the .cause chain to find the innermost error with a .type property. - * Temporal wraps ApplicationFailure in ActivityFailure — the useful info is inside. + * Walk the .cause chain to find the innermost approved failure type. + * Temporal wraps ApplicationFailure in ActivityFailure, so classification must inspect causes. * * Uses duck-typing because workflow code cannot import @temporalio/activity types. */ -function unwrapActivityError(error: unknown): { - message: string; - type: string | null; -} { +function unwrapActivityError(error: unknown): { type: string | null } { let current: unknown = error; - let typed: { message: string; type: string } | null = null; + let type: string | null = null; while (current instanceof Error) { if ('type' in current && typeof (current as { type: unknown }).type === 'string') { - typed = { - message: current.message, - type: (current as { type: string }).type, - }; + const candidate = (current as { type: string }).type; + if (candidate in SAFE_WORKFLOW_FAILURE_MESSAGES) type = candidate; } current = (current as { cause?: unknown }).cause; } - if (typed) { - return typed; - } - - return { - message: error instanceof Error ? error.message : String(error), - type: null, - }; + return { type }; } /** - * Format a structured error string from workflow catch context. + * Format a structured, closed-field error string from workflow catch context. * Segments are delimited by | for multi-line rendering by WorkflowLogger. */ export function formatWorkflowError(error: unknown, currentPhase: string | null, currentAgent: string | null): string { @@ -96,10 +110,12 @@ export function formatWorkflowError(error: unknown, currentPhase: string | null, // Phase context (first segment) let phaseContext = 'Pipeline failed'; - if (currentPhase && currentAgent && currentPhase !== currentAgent) { - phaseContext = `${currentPhase} failed (agent: ${currentAgent})`; - } else if (currentPhase) { - phaseContext = `${currentPhase} failed`; + const safePhase = currentPhase !== null && WORKFLOW_PHASE_SET.has(currentPhase) ? currentPhase : null; + const safeAgent = currentAgent !== null && AGENT_NAME_SET.has(currentAgent) ? currentAgent : null; + if (safePhase && safeAgent && safePhase !== safeAgent) { + phaseContext = `${safePhase} failed (agent: ${safeAgent})`; + } else if (safePhase) { + phaseContext = `${safePhase} failed`; } const segments: string[] = [phaseContext]; @@ -108,8 +124,11 @@ export function formatWorkflowError(error: unknown, currentPhase: string | null, segments.push(unwrapped.type); } - // Sanitize pipe characters from message to preserve delimiter format - segments.push(unwrapped.message.replaceAll('|', '/')); + segments.push( + unwrapped.type === null + ? 'The scan could not be completed.' + : (SAFE_WORKFLOW_FAILURE_MESSAGES[unwrapped.type] ?? 'The scan could not be completed.'), + ); if (unwrapped.type) { const hint = REMEDIATION_HINTS[unwrapped.type]; diff --git a/apps/worker/src/temporal/workflows.ts b/apps/worker/src/temporal/workflows.ts index 0336f923..fc9dad13 100644 --- a/apps/worker/src/temporal/workflows.ts +++ b/apps/worker/src/temporal/workflows.ts @@ -28,16 +28,17 @@ import { workflowInfo, } from '@temporalio/workflow'; import type { StageMetrics } from '../ai/reconciliation/stage-contracts.js'; +import { capellaTerminalStageLabel, isCapellaSafeFailureMessage } from '../ai/sast/capella/safe-failures.js'; import type { CapellaWorkflowInput } from '../ai/sast/capella/temporal/activity-types.js'; import { CAPELLA_CHILD_WORKFLOW_OPTIONS, capellaWorkflow } from '../ai/sast/capella/temporal/workflow.js'; import type { CapellaRunResult, SarifRef } from '../ai/sast/types.js'; +import type { WorkflowPhase } from '../audit/safe-fields.js'; import type { AgentName, VulnType } from '../types/agents.js'; import { ALL_AGENTS } from '../types/agents.js'; import { ALL_VULN_CLASSES, type VulnClass } from '../types/config.js'; import type { ReconciliationClass } from '../types/reconciliation.js'; import { appendPartialReasons, - capellaStageDisplayName, type MiscellaneousOutcome, miscellaneousLaneIsSettled, type PartialReason, @@ -172,13 +173,18 @@ const seedMiscellaneousActs = proxyActivities= MAX_NON_FATAL_FAILURES) return; - state.nonFatalFailures.push({ - phase: failure.phase, - error: truncatePipelineErrorMessage(failure.error), - }); + state.nonFatalFailures.push(failure); } function startOperation(key: string, label: string): number { @@ -500,8 +498,7 @@ export async function pentestPipeline(input: PipelineInput): Promise Promise, ): Promise { @@ -736,7 +733,8 @@ export async function pentestPipeline(input: PipelineInput): Promise): Promise { try { await settlement; - } catch (waitError) { + } catch { log.warn('Capella settlement did not resolve while the scan was stopping', { - error: waitError instanceof Error ? waitError.message : String(waitError), + code: 'CAPELLA_SETTLEMENT_FAILED', }); } } @@ -1026,8 +1023,8 @@ export async function pentestPipeline(input: PipelineInput): Promise @@ -1321,10 +1318,8 @@ export async function pentestPipeline(input: PipelineInput): Promise { try { await a.logWorkflowComplete(activityInput, toWorkflowSummary(state, 'cancelled')); - } catch (completionError) { - log.warn('Failed to finalize cancelled workflow', { - error: completionError instanceof Error ? completionError.message : String(completionError), - }); + } catch { + log.warn('Failed to finalize cancelled workflow', { code: 'WORKFLOW_LOG_WRITE_FAILED' }); } }); return state; @@ -1340,10 +1335,8 @@ export async function pentestPipeline(input: PipelineInput): Promise -./shannon status +./shannon logs [] # the combined live log (unchanged default) +./shannon logs [] --agent # tail one agent's own log +./shannon logs [] --list-agents # list the agents with their own log +./shannon status [] ./shannon scans ./shannon version ``` +Every scan writes one combined `.shannon/workflow.log` and a per-agent projection of it under +`.shannon/agents/`: one file per pipeline agent (`recon.log`, `xss-vuln.log`, …) and one per Capella +stage (`agentic-sast-research.log`, …). Delegated subagents fold into their parent's file, and a +Capella stage's concurrent sessions share its file with an inline session label. The combined log +stays canonical; the per-agent files are best-effort projections. + Open the Temporal Web UI for detailed monitoring: ```bash @@ -147,7 +155,7 @@ workspaces/{hostname}_{sessionId}/ |-- Security-Assessment-Report.md # the final report (Markdown) `-- .shannon/ # internals |-- deliverables/ # report source, per-phase analysis, queues - |-- agents/ # per-agent logs + |-- agents/ # per-agent log projections, one file per agent/Capella stage |-- prompts/ # rendered prompts |-- scratchpad/ # screenshots, scripts |-- session.json # resume state