CalvinBackup/shannon
1
0
Fork 0
mirror of https://github.com/KeygraphHQ/shannon.git synced 2026-09-15 22:45:37 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat(config)!: replace vuln_classes with agentic_sast

Browse Source 
Wire Agentic SAST and reconciliation into the main pipeline, persist their durable state, and add the Miscellaneous finding and exploitation lane.

Make scan completion, cancellation, partial outcomes, resume identity, and report recovery use the integrated final workflow contract. Introduce the atomic finalization, ordering, renumbering, compaction, and output services that workflow calls. Keep completed Miscellaneous work and report drafts idempotent across resume, preserve public main's default-on exploit SARIF behavior, and describe stage-fallback candidates without claiming they were exported.

BREAKING CHANGE: `vuln_classes` has been removed. Configs containing it now fail validation, and all five core pentest classes run on every scan.

Workspaces created by Shannon 2.x cannot be resumed. Finish or discard in-flight scans before upgrading, then start a new workspace name.
This commit is contained in:
ajmallesh
2026-08-26 19:55:03 -07:00
parent c33132b0ab
commit 98c66e051d
57 changed files with 7628 additions and 1201 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+8 -1
View File
File diff suppressed because one or more lines are too long
apps/cli/src/commands/start.ts
+249 -61
View File
@@ -18,6 +18,7 @@ import { commandPrefix, isLocal } from '../mode.js';
import { resolveModelSpec } from '../model-spec.js';
import {
expandHome,
FINAL_REPORT_MD_FILENAME,
FINAL_REPORT_PDF_FILENAME,
INTERNAL_DIR,
resolveConfig,
@@ -43,81 +44,216 @@ export interface StartArgs {
version: string;
}
const LAUNCH_STATE_SCHEMA_VERSION = 1 as const;
const LAUNCH_STATE_FILENAME = 'launch.json';
const FIXED_CLASSES = ['injection', 'xss', 'auth', 'authz', 'ssrf'] as const;
/**
* Upgrade a pre-restructure workspace (flat layout, no INTERNAL_DIR) before it is mounted,
* so resume finds the old deliverables and their git checkpoints instead of re-running every
* agent. For a legacy run every top-level entry is internal, so move them all into INTERNAL_DIR
* (a same-filesystem rename carries the deliverables .git along).
* CLI-owned launch record at INTERNAL_DIR/launch.json, written once when a workspace is
* created and never rewritten. It pins the customer output destination so a resume with a
* different -o cannot silently redirect the final report. The worker does not read it.
*/
function migrateLegacyWorkspaceLayout(workspacePath: string): void {
const legacySessionJson = path.join(workspacePath, 'session.json');
const internalPath = path.join(workspacePath, INTERNAL_DIR);
if (!fs.existsSync(legacySessionJson) || fs.existsSync(internalPath)) {
return;
interface LaunchState {
readonly schema_version: typeof LAUNCH_STATE_SCHEMA_VERSION;
readonly customer_output_path?: string;
}
export interface WorkspaceLaunchDecision {
readonly isResume: boolean;
readonly outputDir?: string;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function arraysEqual(left: readonly unknown[], right: readonly unknown[]): boolean {
return left.length === right.length && left.every((value, index) => value === right[index]);
}
/**
* Hand-rolled twin of the worker's durable-state validator in
* apps/worker/src/types/run-state.ts, which owns the session.json.durableScanState shape.
* Each array check accepts two variants because the worker appends 'other' and
* 'other-exploit' only after the other pipeline admits findings. If the worker's shape
* changes and this twin lags, resume fails fast as incompatible instead of launching a
* worker against state it would misread.
*/
function isCurrentDurableState(value: unknown): boolean {
if (!isRecord(value) || value.schema_version !== 1 || typeof value.exploit !== 'boolean') return false;
if (!Array.isArray(value.participating_classes) || !Array.isArray(value.expected_agents)) return false;
const participating = value.participating_classes;
const validParticipation =
arraysEqual(participating, FIXED_CLASSES) || arraysEqual(participating, [...FIXED_CLASSES, 'other']);
if (!validParticipation) return false;
const baselineAgents = ['pre-recon', 'recon', ...FIXED_CLASSES.map((name) => `${name}-vuln`)];
if (value.exploit) baselineAgents.push(...FIXED_CLASSES.map((name) => `${name}-exploit`));
baselineAgents.push('report');
const expected = value.expected_agents;
return arraysEqual(expected, baselineAgents) || arraysEqual(expected, [...baselineAgents, 'other-exploit']);
}
/** One refusal for damaged CLI-owned or worker-owned workspace records, whichever reads first. */
const DAMAGED_RECORDS_MESSAGE =
"This workspace's internal records are damaged and it cannot be resumed. Its report files are untouched. Start a new scan with a different -w name.";
const NEWER_RELEASE_MESSAGE =
'This workspace was created by a newer version of Shannon. Upgrade Shannon, or start a new scan with a different -w name.';
function readJsonFile(filePath: string): unknown {
try {
return JSON.parse(fs.readFileSync(filePath, 'utf8'));
} catch {
fail(DAMAGED_RECORDS_MESSAGE);
}
}
function readLaunchState(filePath: string): LaunchState {
if (!fs.existsSync(filePath)) {
fail(
'This workspace was created by an earlier version of Shannon and cannot be resumed. Its files and report are untouched. Start a new scan with a different -w name.',
);
}
const value = readJsonFile(filePath);
if (!isRecord(value)) fail(NEWER_RELEASE_MESSAGE);
// Unknown keys mean a newer release wrote this workspace; refuse rather than half-read it.
const keys = Object.keys(value).sort();
const keysAreValid = keys.every((key) => key === 'customer_output_path' || key === 'schema_version');
const customerPath = value.customer_output_path;
const pathIsValid =
customerPath === undefined ||
(typeof customerPath === 'string' && path.isAbsolute(customerPath) && path.resolve(customerPath) === customerPath);
if (value.schema_version !== LAUNCH_STATE_SCHEMA_VERSION || !keysAreValid || !pathIsValid) {
fail(NEWER_RELEASE_MESSAGE);
}
return {
schema_version: LAUNCH_STATE_SCHEMA_VERSION,
...(typeof customerPath === 'string' && { customer_output_path: customerPath }),
};
}
/**
* Decide fresh-versus-resume from on-disk state alone, before start() mutates anything.
* A fresh launch requires the workspace directory to be absent or empty; a resume requires
* current-release session state, a matching target URL, and a customer output path that
* agrees with the recorded one. Every other combination fails the launch, so a typo in
* -w or -o stops here instead of spawning a worker into the wrong workspace.
*/
export function classifyWorkspaceLaunch(
workspacePath: string,
expectedUrl: string,
requestedOutputDir: string | undefined,
): WorkspaceLaunchDecision {
const sessionPath = resolveRunFile(workspacePath, 'session.json');
const sessionExists = fs.existsSync(sessionPath);
if (!sessionExists) {
if (fs.existsSync(workspacePath) && fs.readdirSync(workspacePath).length > 0) {
fail(
'This directory is not a Shannon workspace, or its scan state is missing. Start a new scan with a different -w name.',
);
}
return { isResume: false, ...(requestedOutputDir !== undefined && { outputDir: requestedOutputDir }) };
}
fs.mkdirSync(internalPath, { recursive: true });
for (const entry of fs.readdirSync(workspacePath)) {
if (entry === INTERNAL_DIR) {
continue;
const launchPath = path.join(workspacePath, INTERNAL_DIR, LAUNCH_STATE_FILENAME);
const launch = readLaunchState(launchPath);
const session = readJsonFile(sessionPath);
if (!isRecord(session) || !isRecord(session.session) || session.session.webUrl !== expectedUrl) {
fail(
'This workspace was created for a different target URL, so it cannot be resumed against this one. Check -u, or start a new scan with a different -w name.',
);
}
if (!isCurrentDurableState(session.durableScanState)) {
fail(
"This workspace's scan state cannot be read by this version. Its files are untouched. Start a new scan with a different -w name.",
);
}
const storedOutputDir = launch.customer_output_path;
if (requestedOutputDir !== undefined && requestedOutputDir !== storedOutputDir) {
fail(
'This workspace already copies its report to a different location than the -o path you passed. Re-run without -o to keep the original location, or start a new scan with a different -w name.',
);
}
return { isResume: true, ...(storedOutputDir !== undefined && { outputDir: storedOutputDir }) };
}
/**
* Crash-safe single write: exclusive temp file (pid plus random suffix keeps concurrent
* starts apart), fsync, rename into place, then directory fsync so the entry survives a
* host crash. Callers invoke this only for a fresh workspace; an existing launch.json is
* the resume contract and must never be replaced.
*/
export function writeLaunchStateAtomically(internalPath: string, outputDir: string | undefined): void {
const finalPath = path.join(internalPath, LAUNCH_STATE_FILENAME);
const temporaryPath = path.join(internalPath, `${LAUNCH_STATE_FILENAME}.tmp-${process.pid}-${randomSuffix()}`);
const launchState: LaunchState = {
schema_version: LAUNCH_STATE_SCHEMA_VERSION,
...(outputDir !== undefined && { customer_output_path: outputDir }),
};
const descriptor = fs.openSync(temporaryPath, 'wx', 0o600);
try {
fs.writeFileSync(descriptor, `${JSON.stringify(launchState, null, 2)}\n`, 'utf8');
fs.fsyncSync(descriptor);
} finally {
fs.closeSync(descriptor);
}
try {
fs.renameSync(temporaryPath, finalPath);
const directory = fs.openSync(internalPath, 'r');
try {
fs.fsyncSync(directory);
} finally {
fs.closeSync(directory);
}
fs.renameSync(path.join(workspacePath, entry), path.join(internalPath, entry));
} catch (error) {
fs.rmSync(temporaryPath, { force: true });
throw error;
}
console.log(`Migrated workspace to ${INTERNAL_DIR}/ layout: ${workspacePath}`);
}
export async function start(args: StartArgs): Promise<void> {
// 1. Initialize state directories and load env
// 1. Resolve non-mutating inputs and classify the workspace before changing it.
initHome();
loadEnv();
// 2. Validate credentials
const creds = validateCredentials();
if (!creds.valid) {
fail(creds.error ?? 'Invalid credentials');
}
// 3. Resolve paths
const repo = resolveRepo(args.repo);
const config = args.config ? resolveConfig(args.config) : undefined;
const workspacesDir = getWorkspacesDir();
const workspace =
args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`;
const workspacePath = path.join(workspacesDir, workspace);
const requestedOutputDir = args.output ? path.resolve(expandHome(args.output)) : undefined;
const launchDecision = classifyWorkspaceLaunch(workspacePath, args.url, requestedOutputDir);
// Inputs are valid — show the splash before the Docker/Temporal setup work.
// Skip it off a real terminal (e.g. CI) so piped/logged output stays clean.
// 2. Inputs are valid; initialize shared infrastructure.
if (stdoutIsTerminal()) {
displaySplash(isLocal() ? undefined : args.version);
}
// 4. Ensure workspaces dir is writable by container user (UID 1001)
const workspacesDir = getWorkspacesDir();
fs.mkdirSync(workspacesDir, { recursive: true });
fs.chmodSync(workspacesDir, 0o777);
// 5. Ensure Docker and the worker image are available (pull/build prints its own progress).
ensureDocker();
ensureImage(args.version);
// One spinner spans the whole launch: bringing up Temporal and registering the worker.
const spinner = p.spinner();
spinner.start('Starting scan');
await ensureInfra(spinner);
// 6. Generate unique task queue and container name
// 3. Generate the invocation identity.
const suffix = randomSuffix();
const taskQueue = `shannon-${suffix}`;
const containerName = `shannon-worker-${suffix}`;
// 7. Generate workspace name if not provided
const workspace =
args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`;
// 8. Create writable overlay directories (mounted over :ro repo paths inside container)
// 4. Create writable overlay directories after resume validation has succeeded.
// The run dir and its INTERNAL_DIR must be 0o777 so the container user can create audit
// subdirs and the overlay backing dirs.
const workspacePath = path.join(workspacesDir, workspace);
const internalPath = path.join(workspacePath, INTERNAL_DIR);
fs.mkdirSync(workspacePath, { recursive: true });
fs.chmodSync(workspacePath, 0o777);
migrateLegacyWorkspaceLayout(workspacePath);
fs.mkdirSync(internalPath, { recursive: true });
fs.chmodSync(internalPath, 0o777);
for (const dir of ['deliverables', 'scratchpad', '.playwright-cli', '.playwright']) {
@@ -125,24 +261,37 @@ export async function start(args: StartArgs): Promise<void> {
fs.mkdirSync(dirPath, { recursive: true });
fs.chmodSync(dirPath, 0o777);
}
if (!launchDecision.isResume) {
writeLaunchStateAtomically(internalPath, launchDecision.outputDir);
}
// 9. Pre-create overlay mount points (:ro mounts can't auto-create them)
// 5. Pre-create overlay mount points (:ro mounts cannot create them).
const shannonDir = path.join(repo.hostPath, '.shannon');
for (const dir of ['deliverables', 'scratchpad', '.playwright-cli']) {
fs.mkdirSync(path.join(shannonDir, dir), { recursive: true });
}
fs.mkdirSync(path.join(repo.hostPath, '.playwright'), { recursive: true });
// 10. Resolve output directory
const outputDir = args.output ? path.resolve(expandHome(args.output)) : undefined;
// 6. Create the validated customer-copy destination, if configured.
const outputDir = launchDecision.outputDir;
if (outputDir) {
fs.mkdirSync(outputDir, { recursive: true });
}
// 11. Resolve prompts directory (local mode only)
// 7. Resolve prompts and capture the pre-launch resume counter.
const promptsDir = isLocal() ? path.resolve('apps/worker/prompts') : undefined;
const sessionJson = resolveRunFile(workspacePath, 'session.json');
const isResume = launchDecision.isResume;
let initialResumeCount = 0;
if (isResume) {
// Docker and Temporal startup sit between this read and the classification that validated the
// same file, so a file that changed in between is a workspace-state failure, not a CLI bug.
const session = readJsonFile(sessionJson);
const attempts = isRecord(session) && isRecord(session.session) ? session.session.resumeAttempts : undefined;
initialResumeCount = Array.isArray(attempts) ? attempts.length : 0;
}
// 12. Spawn worker container
// 8. Spawn the worker container.
const proc = spawnWorker({
version: args.version,
url: args.url,
@@ -171,24 +320,16 @@ export async function start(args: StartArgs): Promise<void> {
process.exit(1);
}
// Detect whether this is a fresh workspace or a resume by checking session.json existence
const sessionJson = resolveRunFile(path.join(workspacesDir, workspace), 'session.json');
const isResume = fs.existsSync(sessionJson);
let initialResumeCount = 0;
if (isResume) {
try {
const session = JSON.parse(fs.readFileSync(sessionJson, 'utf-8'));
initialResumeCount = session.session?.resumeAttempts?.length ?? 0;
} catch {
// Corrupted file — worker will handle validation
}
}
let started = false;
// Set when the startup poll times out but session.json already holds durable state this
// release understands: the workflow is executing, so the exit handler must not stop its
// worker. An operator abort is a different intent and still stops it.
let scanRunningUnconfirmed = false;
// Stop the worker only if the scan hasn't registered yet (e.g. Ctrl-C mid-startup).
let cleaned = false;
const cleanup = (): void => {
const stopWorker = (): void => {
if (cleaned || started) return;
cleaned = true;
spinner.stop('Stopping scan');
@@ -202,14 +343,17 @@ export async function start(args: StartArgs): Promise<void> {
}
};
process.on('SIGINT', () => {
cleanup();
stopWorker();
process.exit(0);
});
process.on('SIGTERM', () => {
cleanup();
stopWorker();
process.exit(0);
});
process.on('exit', cleanup);
process.on('exit', () => {
if (scanRunningUnconfirmed) return;
stopWorker();
});
// Poll for the workflow to register in session.json; the spinner resolves once it does.
spinner.message('Waiting for the scan to start');
@@ -236,10 +380,52 @@ export async function start(args: StartArgs): Promise<void> {
await sleep(2000);
}
if (classifyStartupTimeout(sessionJson) === 'scan-running') {
scanRunningUnconfirmed = true;
spinner.error('The scan started, but this CLI could not confirm it');
printUnconfirmedScanHint(workspace, taskQueue, containerName);
process.exit(1);
}
spinner.error('Timed out waiting for the scan to start');
process.exit(1);
}
/**
* Read the startup timeout: 'scan-running' when session.json already holds durable state this
* release understands, which only the worker writes and only after Temporal began executing the
* workflow; 'unregistered' when nothing proves the scan started. The distinction decides whether
* timing out may stop the worker container.
*/
export function classifyStartupTimeout(sessionJsonPath: string): 'unregistered' | 'scan-running' {
let session: unknown;
try {
session = JSON.parse(fs.readFileSync(sessionJsonPath, 'utf-8'));
} catch {
return 'unregistered';
}
if (!isRecord(session) || !isCurrentDurableState(session.durableScanState)) {
return 'unregistered';
}
return 'scan-running';
}
/** Point the operator at a scan that is running but whose startup this CLI could not confirm. */
function printUnconfirmedScanHint(workspace: string, taskQueue: string, containerName: string): void {
console.log('');
console.log(' The scan is running and was left alone; only its startup confirmation is missing.');
console.log('');
console.log(` Workspace: ${workspace}`);
console.log(` Task queue: ${taskQueue}`);
console.log(` Container: ${containerName}`);
console.log('');
console.log(' Inspect it:');
console.log(` Live logs: ${commandPrefix()} logs ${workspace}`);
console.log(` Worker logs: docker logs ${containerName}`);
console.log(' Dashboard: http://localhost:8233');
console.log('');
}
/**
* Follow a just-started scan (for `--follow`, aimed at CI): stream its log while Temporal drives
* completion, then exit on the workflow outcome — 0 if the assessment ran, 1 if the scan failed.
@@ -329,7 +515,7 @@ function printInfo(args: StartArgs, workspace: string, repoPath: string, workspa
return;
}
const reportPath = path.join(workspacesDir, workspace, FINAL_REPORT_PDF_FILENAME);
const reportDir = path.join(workspacesDir, workspace);
// When following, the scan log streams inline next, so the "run these to watch it" hints
// would only contradict that.
@@ -343,6 +529,8 @@ function printInfo(args: StartArgs, workspace: string, repoPath: string, workspa
console.log('');
console.log(' Report (when the scan finishes):');
console.log(` ${reportPath}`);
console.log(` ${reportDir}${path.sep}`);
console.log(` ${FINAL_REPORT_PDF_FILENAME}`);
console.log(` ${FINAL_REPORT_MD_FILENAME}`);
console.log('');
}
apps/cli/src/commands/status.ts
+27 -8
View File
@@ -15,7 +15,13 @@ import { type RenderInput, renderScan } from '../scan/render.js';
import { toStatusJson } from '../scan/status-json.js';
import { resolveWorkflowId } from '../session.js';
import { displaySplash } from '../splash.js';
import { describeScan, getTerminalOutcome, queryProgress, type ScanDescription } from '../temporal-client.js';
import {
ActivityMirrorError,
describeScan,
getTerminalOutcome,
queryProgress,
type ScanDescription,
} from '../temporal-client.js';
import { stdoutIsTerminal, supportsColor } from '../tty.js';
import { getVersion } from '../version.js';
@@ -30,6 +36,24 @@ function isTerminalStatus(status: string): boolean {
return status !== 'RUNNING' && status !== 'UNSPECIFIED';
}
/**
* Read one scan description, telling the two failure modes apart. A stale activity mirror
* carries its own message and needs a CLI update; anything else is a read that did not reach
* a usable answer, which is most often Temporal being down.
*/
async function readScanDescription(workflowId: string): Promise<ScanDescription | null> {
try {
return await describeScan(workflowId);
} catch (error) {
if (error instanceof ActivityMirrorError) fail(error.message);
fail(
"Could not read this scan's progress.",
'If Temporal is not running, start a scan to bring it up. If it is running, this build of the CLI',
'does not recognise part of the scan and needs updating.',
);
}
}
// Match SGR color escapes (ESC[…m) so a line's on-screen width excludes them. Built from the ESC
// char code so the source carries no literal control character.
const ANSI_PATTERN = new RegExp(`${String.fromCharCode(27)}\\[[0-9;]*m`, 'g');
@@ -128,7 +152,7 @@ async function watch(workspace: string, workflowId: string): Promise<never> {
}, RENDER_MS);
for (;;) {
const desc = await describeScan(workflowId);
const desc = await readScanDescription(workflowId);
if (!desc) {
clearInterval(ticker);
fail(`Scan "${workspace}" is no longer in Temporal.`);
@@ -158,12 +182,7 @@ export async function status(workspace: string, opts: { readonly json: boolean }
// follows the current resume, not the superseded original. Fresh scans: the name is the id.
const workflowId = resolveWorkflowId(workspace) ?? workspace;
let desc: ScanDescription | null;
try {
desc = await describeScan(workflowId);
} catch {
fail('Could not reach Temporal at 127.0.0.1:7233.', 'Start Temporal (it comes up with a scan) and try again.');
}
const desc = await readScanDescription(workflowId);
if (!desc) {
fail(
apps/cli/src/docker.ts
+1 -1
View File
@@ -345,7 +345,7 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess {
args.push('-v', `${opts.config.hostPath}:${opts.config.containerPath}:ro`);
}
// Output directory for deliverables copy
// Customer-copy destination. The workflow surfaces only final report artifacts here.
if (opts.outputDir) {
args.push('-v', `${opts.outputDir}:/app/output`);
}
apps/cli/src/paths.ts
+6
View File
@@ -42,6 +42,12 @@ export const INTERNAL_DIR = '.shannon';
*/
export const FINAL_REPORT_PDF_FILENAME = 'Security-Assessment-Report.pdf';
/**
* Customer-facing Markdown report name at the run root.
* Must match FINAL_REPORT_MD_FILENAME in the worker package.
*/
export const FINAL_REPORT_MD_FILENAME = 'Security-Assessment-Report.md';
/**
* Resolve a run-directory file (e.g. session.json, workflow.log), preferring the
* current INTERNAL_DIR location and falling back to the legacy run-root location
apps/cli/src/scan/derive.ts
+118 -11
View File
@@ -8,7 +8,13 @@
*/
import type { RunningAgent } from '../temporal-client.js';
import { agentClass, PIPELINE, type PipelineState } from './pipeline.js';
import {
agentClass,
type OperationalStageState,
operationFamilyKey,
type PipelineState,
pipelineForState,
} from './pipeline.js';
import type { RenderInput } from './render.js';
export type RunState = 'pending' | 'running' | 'completed' | 'failed' | 'skipped';
@@ -22,6 +28,8 @@ export interface DerivedAgent {
readonly durationMs: number | null;
readonly runningElapsedMs: number | null;
readonly attempt: number | null;
/** The step a running operation row is currently on, merged in from its child activity. */
readonly detail?: string;
readonly error?: string;
}
@@ -48,12 +56,12 @@ function isAgentActive(name: string, state: PipelineState | null, running: Set<s
}
/**
* Resolve one agent's state. "Ran" is signalled by a metrics entry, not by
* completedAgents — the workflow lists conditionally-skipped agents (e.g. exploit
* agents when there is nothing to exploit) as completed but records no metrics for
* them. `resolved` is true once we've moved past this agent's phase (the scan is
* terminal, or a later phase is already active), at which point a metric-less,
* non-running agent is skipped rather than still pending.
* Resolve one agent's state. "Ran" is signalled by a metrics entry: a
* conditionally-skipped agent (e.g. an exploit agent when there is nothing to
* exploit) records no metrics, and the workflow tracks it in skippedAgents rather
* than completedAgents. `resolved` is true once we've moved past this agent's phase
* (the scan is terminal, or a later phase is already active), at which point a
* metric-less, non-running agent is skipped rather than still pending.
*/
function agentState(name: string, state: PipelineState | null, running: Set<string>, resolved: boolean): RunState {
if (running.has(name)) return 'running';
@@ -99,16 +107,17 @@ export function phaseGlyphState(states: readonly RunState[]): RunState {
* class had anything to exploit), not still pending.
*/
export function deriveAgentStates(input: RenderInput): Map<string, RunState> {
const runningSet = new Set(input.running.map((r) => r.agent));
const pipeline = pipelineForState(input.state);
const runningSet = new Set(input.running.filter((runner) => runner.kind === 'agent').map((runner) => runner.agent));
const terminal = isTerminal(input.temporalStatus);
let frontier = -1;
PIPELINE.forEach((phase, idx) => {
pipeline.forEach((phase, idx) => {
if (phase.agents.some((a) => isAgentActive(a.name, input.state, runningSet))) frontier = idx;
});
const states = new Map<string, RunState>();
for (const [phaseIdx, phase] of PIPELINE.entries()) {
for (const [phaseIdx, phase] of pipeline.entries()) {
const resolved = terminal || phaseIdx < frontier;
for (const agent of phase.agents) {
states.set(agent.name, agentState(agent.name, input.state, runningSet, resolved));
@@ -117,6 +126,52 @@ export function deriveAgentStates(input: RenderInput): Map<string, RunState> {
return states;
}
/** Which operation families have a running parent stage, and the step to show on it. */
interface OperationFamilyView {
/** Families whose parent stage row already represents their child activities. */
readonly runningFamilies: ReadonlySet<string>;
/** Family to current step, present only where the child activities agree on one. */
readonly stepByFamily: ReadonlyMap<string, string>;
}
/**
* Resolve the parent stage rows that own their family's child activities. A family only
* resolves to a step when its running children agree: several classes reconcile at once and
* their pending activities carry no class, so a family caught mid-stride shows its parent
* rows without a step rather than attributing one to the wrong class.
*/
function operationFamilyView(
running: readonly RunningAgent[],
persistedOperations: readonly OperationalStageState[],
): OperationFamilyView {
const runningFamilies = new Set(
persistedOperations
.filter((operation) => operation.status === 'running')
.map((operation) => operationFamilyKey(operation.key)),
);
const labelsByFamily = new Map<string, Set<string>>();
for (const runner of running) {
if (runner.kind !== 'operation' || runner.parentKey === undefined) continue;
if (!runningFamilies.has(runner.parentKey)) continue;
const labels = labelsByFamily.get(runner.parentKey) ?? new Set<string>();
labels.add(runner.label);
labelsByFamily.set(runner.parentKey, labels);
}
const stepByFamily = new Map<string, string>();
for (const [family, labels] of labelsByFamily) {
const [onlyLabel] = labels;
if (labels.size === 1 && onlyLabel !== undefined) stepByFamily.set(family, lowercaseFirst(onlyLabel));
}
return { runningFamilies, stepByFamily };
}
/** Progress labels are written to start a row; as a detail they continue a sentence. */
function lowercaseFirst(label: string): string {
return label.charAt(0).toLowerCase() + label.slice(1);
}
/**
* Full structured view of the pipeline: every agent's state plus the raw
* metrics/timing needed to present it, and each phase's collapsed state.
@@ -124,8 +179,9 @@ export function deriveAgentStates(input: RenderInput): Map<string, RunState> {
export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] {
const states = deriveAgentStates(input);
const byAgent = new Map(input.running.map((r) => [r.agent, r]));
const pipeline = pipelineForState(input.state);
return PIPELINE.map((phase) => {
const agentPhases = pipeline.map((phase) => {
const agents = phase.agents.map((a): DerivedAgent => {
const state = states.get(a.name) ?? 'pending';
const metrics = input.state?.agentMetrics[a.name];
@@ -150,6 +206,57 @@ export function derivePipeline(input: RenderInput, now: number): DerivedPhase[]
agents,
};
});
// Operational rows merge two sources: stages the worker has persisted (durable truth,
// including terminal outcomes) and pending activities whose stage record has not landed
// yet. Persisted keys win, so a stage is never listed twice while the two views overlap.
const persistedOperations = Object.values(input.state?.operationalStages ?? {});
const persistedKeys = new Set(persistedOperations.map((operation) => operation.key));
const { runningFamilies, stepByFamily } = operationFamilyView(input.running, persistedOperations);
const unpersistedRunning = input.running
.filter((runner) => runner.kind === 'operation' && !persistedKeys.has(runner.agent))
// A child activity whose family already has a running parent stage is that stage's current
// step, not separate work: the parent row below represents it, with the step as its detail
// where the family's children agree on one. Without such a parent it keeps its own row.
.filter((runner) => runner.parentKey === undefined || !runningFamilies.has(runner.parentKey))
.map((runner) => ({
key: runner.agent,
label: runner.label,
status: 'running' as const,
...(runner.startedAt !== undefined && { startedAt: runner.startedAt }),
...(runner.lastFailure !== undefined && { error: runner.lastFailure }),
}));
const operationalAgents: DerivedAgent[] = [...persistedOperations, ...unpersistedRunning].map((operation) => {
const runner = byAgent.get(operation.key);
const operationState = operation.status as RunState;
const persistedDurationMs = 'durationMs' in operation ? (operation.durationMs ?? null) : null;
const detail = operationState === 'running' ? stepByFamily.get(operationFamilyKey(operation.key)) : undefined;
return {
name: operation.key,
label: operation.label,
state: operationState,
durationMs: operationState === 'completed' ? persistedDurationMs : null,
runningElapsedMs:
operationState === 'running' && operation.startedAt !== undefined ? now - operation.startedAt : null,
attempt: operationState === 'running' ? (runner?.attempt ?? null) : null,
...(detail !== undefined && { detail }),
...(operation.error !== undefined && { error: operation.error }),
};
});
// The synthetic phase appears only when there is operational work to show, so a scan
// with no recorded operational stages keeps the plain agent tree.
if (operationalAgents.length === 0) return agentPhases;
return [
...agentPhases,
{
key: 'operational-work',
label: 'Background work',
parallel: true,
state: phaseGlyphState(operationalAgents.map((operation) => operation.state)),
agents: operationalAgents,
},
];
}
export { agentError };
apps/cli/src/scan/pipeline.ts
+223 -2
View File
@@ -8,6 +8,7 @@
* - apps/worker/src/temporal/activities.ts (the run*Agent activity names → `activityType`)
* - apps/worker/src/temporal/shared.ts (PipelineState / PipelineSummary)
* - apps/worker/src/types/metrics.ts (AgentMetrics)
* - apps/worker/src/types/run-state.ts (PartialReasonView)
*/
export interface AgentSpec {
@@ -26,6 +27,18 @@ export interface PhaseSpec {
readonly agents: readonly AgentSpec[];
}
export interface ActivityProgressSpec {
readonly key: string;
readonly label: string;
readonly kind: 'agent' | 'operation';
/**
* Operation rows whose work is already represented by a persisted parent stage. The parent
* owns the row; this activity supplies the step shown as its detail. Parent stage keys are
* the family key itself or the family key followed by ':' and a class or stage suffix.
*/
readonly parentKey?: string;
}
/** The pipeline phases in execution order, each with its agents. */
export const PIPELINE: readonly PhaseSpec[] = [
{
@@ -80,9 +93,175 @@ export const PIPELINE: readonly PhaseSpec[] = [
},
];
/** Temporal activity type name → canonical agent name, for mapping pendingActivities. */
const OTHER_EXPLOIT_AGENT: AgentSpec = {
name: 'other-exploit',
label: 'other',
activityType: 'runOtherExploitAgent',
};
/**
* Shape the static PIPELINE to one scan's durable truth. expectedAgents, persisted by the
* worker at scan start, names every exploit agent the scan can ever run: exploit rows it
* excludes are dropped, 'other-exploit' is appended only once the other pipeline has
* admitted findings, and a phase left with no agents disappears entirely. Without state
* (the scan has not initialized durable state yet) the full static pipeline is the best
* available guess.
*/
export function pipelineForState(state: PipelineState | null): readonly PhaseSpec[] {
if (state?.expectedAgents === undefined) return PIPELINE;
const expected = new Set(state.expectedAgents);
return PIPELINE.map((phase) => {
if (phase.key !== 'exploitation') return phase;
const agents = phase.agents.filter((agent) => expected.has(agent.name));
if (expected.has(OTHER_EXPLOIT_AGENT.name)) agents.push(OTHER_EXPLOIT_AGENT);
return { ...phase, agents };
}).filter((phase) => phase.agents.length > 0);
}
const AGENT_ACTIVITY_PROGRESS: Readonly<Record<string, ActivityProgressSpec>> = Object.fromEntries(
[...PIPELINE.flatMap((phase) => phase.agents), OTHER_EXPLOIT_AGENT].map((agent) => [
agent.activityType,
{ key: agent.name, label: agent.label, kind: 'agent' },
]),
);
/** Families whose per-class or per-stage work is already carried by one persisted stage row. */
const RECONCILIATION_PARENT_KEY = 'reconciliation';
const AGENTIC_SAST_PARENT_KEY = 'agentic-sast';
// Every production activity that is not an agent run must have a row here. describeScan
// throws on an unmapped activity type, so adding a worker activity without updating this
// table breaks `shannon status` loudly instead of hiding the new work. The authoritative
// name lists live in apps/worker/src/temporal/worker.ts,
// apps/worker/src/temporal/reconcile-activity-types.ts, and
// apps/worker/src/ai/sast/capella/temporal/activity-types.ts.
const OPERATION_ACTIVITY_PROGRESS: Readonly<Record<string, ActivityProgressSpec>> = {
runPreflightValidation: { key: 'preflight', label: 'Preflight validation', kind: 'operation' },
syncPlaywrightStealthConfig: { key: 'preflight', label: 'Browser setup', kind: 'operation' },
initDeliverableGit: { key: 'scan-initialization', label: 'Initialize deliverables', kind: 'operation' },
syncCodePathDenyRules: { key: 'scan-initialization', label: 'Apply source rules', kind: 'operation' },
initializeDurableScanState: { key: 'durable-state', label: 'Saving scan state', kind: 'operation' },
persistOtherOutcome: { key: 'other-pipeline', label: 'Including other findings', kind: 'operation' },
initializeReportProgress: { key: 'report:initialize', label: 'Initialize report state', kind: 'operation' },
renumberClassFindings: { key: 'report:renumber', label: 'Renumber findings', kind: 'operation' },
assembleReportActivity: { key: 'report:assemble', label: 'Assemble report inputs', kind: 'operation' },
compactReportFindings: { key: 'report:compact', label: 'Compact report findings', kind: 'operation' },
persistCanonicalReportProgress: { key: 'report:checkpoint', label: 'Saving report progress', kind: 'operation' },
finalizeReportOutputs: { key: 'report:finalize', label: 'Finalize report outputs', kind: 'operation' },
persistFinalizedReportProgress: { key: 'report:terminal', label: 'Saving final report state', kind: 'operation' },
surfaceReportOutputs: { key: 'report:surface', label: 'Surface customer report', kind: 'operation' },
checkExploitationQueue: { key: 'queue-check', label: 'Check exploitation queue', kind: 'operation' },
loadResumeState: { key: 'resume-validation', label: 'Validate resume state', kind: 'operation' },
restoreGitCheckpoint: { key: 'resume-restore', label: 'Restore checkpoint', kind: 'operation' },
registerResumeAttempt: { key: 'resume-registration', label: 'Register resume', kind: 'operation' },
recordResumeAttempt: { key: 'resume-registration', label: 'Record resume', kind: 'operation' },
logPhaseTransition: { key: 'audit-log', label: 'Update audit log', kind: 'operation' },
logWorkflowComplete: { key: 'audit-log', label: 'Finalize audit log', kind: 'operation' },
saveCheckpoint: { key: 'checkpoint', label: 'Save checkpoint', kind: 'operation' },
seedEmptyProducerQueue: { key: 'other-pipeline', label: 'Preparing other findings', kind: 'operation' },
prepareClassReconciliation: {
key: 'reconciliation',
label: 'Preparing findings',
kind: 'operation',
parentKey: RECONCILIATION_PARENT_KEY,
},
enrichClassSastObservations: {
key: 'reconciliation',
label: 'Adding code context',
kind: 'operation',
parentKey: RECONCILIATION_PARENT_KEY,
},
formClassExploitTasks: {
key: 'reconciliation',
label: 'Grouping into test cases',
kind: 'operation',
parentKey: RECONCILIATION_PARENT_KEY,
},
materializeClassExploitTasks: {
key: 'reconciliation',
label: 'Writing test cases',
kind: 'operation',
parentKey: RECONCILIATION_PARENT_KEY,
},
publishClassReconciliationOss: {
key: 'reconciliation',
label: 'Saving results',
kind: 'operation',
parentKey: RECONCILIATION_PARENT_KEY,
},
capellaArchitecture: {
key: 'agentic-sast:architecture',
label: 'Mapping architecture',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaThreatModel: {
key: 'agentic-sast:threat-model',
label: 'Modelling threats',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaPlan: {
key: 'agentic-sast:plan',
label: 'Planning the review',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaResearch: {
key: 'agentic-sast:research',
label: 'Researching code',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaDedupe: {
key: 'agentic-sast:dedupe',
label: 'Merging duplicates',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaReview: {
key: 'agentic-sast:review',
label: 'Reviewing findings',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaCritic: {
key: 'agentic-sast:critic',
label: 'Critiquing findings',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaConfirm: {
key: 'agentic-sast:confirm',
label: 'Confirming findings',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaCalibrate: {
key: 'agentic-sast:calibrate',
label: 'Calibrating risk',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
capellaExport: {
key: 'agentic-sast:export',
label: 'Exporting findings',
kind: 'operation',
parentKey: AGENTIC_SAST_PARENT_KEY,
},
};
/** Complete production activity mirror. Unknown names are errors, never hidden progress. */
export const ACTIVITY_TO_PROGRESS: Readonly<Record<string, ActivityProgressSpec>> = Object.freeze({
...AGENT_ACTIVITY_PROGRESS,
...OPERATION_ACTIVITY_PROGRESS,
});
/** Agent-only projection of ACTIVITY_TO_PROGRESS: activity type name to canonical agent name. */
export const ACTIVITY_TO_AGENT: Readonly<Record<string, string>> = Object.fromEntries(
PIPELINE.flatMap((phase) => phase.agents.map((agent) => [agent.activityType, agent.name])),
Object.entries(ACTIVITY_TO_PROGRESS)
.filter(([, progress]) => progress.kind === 'agent')
.map(([activityType, progress]) => [activityType, progress.key]),
);
/** The vuln/exploit class of an agent (e.g. "authz-vuln" → "authz"), for failedPipelines matching. */
@@ -100,11 +279,36 @@ export interface AgentMetrics {
readonly skipped?: boolean;
}
export interface OperationalStageState {
readonly key: string;
readonly label: string;
readonly status: 'pending' | 'running' | 'completed' | 'failed' | 'skipped';
readonly startedAt?: number;
readonly durationMs?: number;
readonly error?: string;
}
/** Family key a persisted operational stage belongs to, e.g. `reconciliation:xss` to `reconciliation`. */
export function operationFamilyKey(stageKey: string): string {
const separator = stageKey.indexOf(':');
return separator === -1 ? stageKey : stageKey.slice(0, separator);
}
export interface PipelineSummary {
readonly totalCostUsd: number;
readonly totalDurationMs: number; // Wall-clock (end - start)
readonly totalTurns: number;
readonly agentCount: number;
/** False when operational (Capella/reconciliation) spend is known to be incomplete. */
readonly usageAccountingComplete?: boolean;
}
/** One durable degradation reason with its derived safe message (mirror of PartialReasonView). */
export interface PartialReasonView {
readonly code: string;
readonly vulnerabilityClass?: string;
readonly stage?: string;
readonly message: string;
}
export type PipelineStatus = 'running' | 'completed' | 'failed' | 'cancelled' | 'partial';
@@ -114,10 +318,27 @@ export interface PipelineState {
readonly currentPhase: string | null;
readonly currentAgent: string | null;
readonly completedAgents: string[];
readonly expectedAgents?: string[];
readonly participatingClasses?: string[];
readonly failedPipelines: { vulnType: string; error: string }[];
readonly failedReconciliations?: { vulnerabilityClass: string; error: string }[];
readonly failedAgent: string | null;
readonly error: string | null;
readonly startTime: number;
readonly agentMetrics: Record<string, AgentMetrics>;
readonly operationalMetrics?: Record<string, AgentMetrics>;
readonly operationalStages?: Record<string, OperationalStageState>;
/** `error` is the worker's sanitized failure sentence, safe to print verbatim. */
readonly agenticSast?: {
readonly status: string;
readonly durationMs?: number;
/** Reader-facing name of the failed stage, already projected by the worker. */
readonly failedStageLabel?: string;
readonly error?: string;
readonly errorCode?: string;
};
readonly nonFatalFailures?: { readonly phase: string; readonly error: string }[];
/** Ordered durable degradation reasons with safe messages; empty or absent for full success. */
readonly partialReasons?: readonly PartialReasonView[];
readonly summary: PipelineSummary | null;
}
apps/cli/src/scan/render.ts
+57 -16
View File
@@ -10,9 +10,8 @@
import { BOLD, DIM, GOLD, paint, RED, YELLOW } from '../colors.js';
import { commandPrefix } from '../mode.js';
import type { RunningAgent } from '../temporal-client.js';
import { agentError, deriveAgentStates, isTerminal, phaseGlyphState, type RunState, scanElapsedMs } from './derive.js';
import { inlineFailureReason } from './failure.js';
import { PIPELINE, type PipelineState } from './pipeline.js';
import { derivePipeline, isTerminal, type RunState, scanElapsedMs } from './derive.js';
import type { PipelineState } from './pipeline.js';
export interface RenderInput {
readonly workspace: string;
@@ -95,6 +94,12 @@ const STATE_COLOR: Record<RunState, string> = {
skipped: COLORS.dim,
};
/** Column width for an agent or background-work label inside a phase. */
const AGENT_LABEL_WIDTH = 18;
/** Inline budget for a failure sentence, wide enough to carry a whole first sentence. */
const FAILURE_DETAIL_WIDTH = 120;
/** Braille spinner frames for running agents — the clack loader style. */
const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'] as const;
@@ -112,13 +117,14 @@ function statusBadge(input: RenderInput, opts: RenderOptions): string {
const workflowStatus = input.state?.status;
if (!isTerminal(input.temporalStatus)) return paint('running', COLORS.gold, opts.color);
if (workflowStatus === 'partial') return paint('partial', COLORS.yellow, opts.color);
if (workflowStatus === 'cancelled') return paint('cancelled', COLORS.yellow, opts.color);
if (input.temporalStatus === 'COMPLETED') return paint('completed', COLORS.gold, opts.color);
if (input.temporalStatus === 'TERMINATED') return paint('stopped', COLORS.yellow, opts.color);
if (input.temporalStatus === 'CANCELLED' || input.temporalStatus === 'CANCELED') {
return paint('cancelled', COLORS.yellow, opts.color);
}
if (input.temporalStatus === 'TIMED_OUT') return paint('timed out', COLORS.red, opts.color);
return paint('FAILED', COLORS.red, opts.color);
return paint('failed', COLORS.red, opts.color);
}
// === Line builders ===
@@ -129,6 +135,7 @@ function agentMeta(
runner: RunningAgent | undefined,
error: string | undefined,
opts: RenderOptions,
step?: string,
): string {
if (state === 'completed') {
const duration = metrics?.durationMs != null ? formatDuration(metrics.durationMs) : 'done';
@@ -136,12 +143,13 @@ function agentMeta(
}
if (state === 'running') {
const parts = ['running'];
if (step !== undefined) parts.push(step);
if (runner?.startedAt !== undefined) parts.push(formatDuration(opts.now - runner.startedAt));
if (runner && runner.attempt > 1) parts.push(`retry ${runner.attempt}`);
return paint(parts.join(' · '), COLORS.gold, opts.color);
}
if (state === 'failed') {
const detail = error ? ` · ${truncate(error, 46)}` : '';
const detail = error ? ` · ${truncate(error, FAILURE_DETAIL_WIDTH)}` : '';
return paint(`failed${detail}`, COLORS.red, opts.color);
}
if (state === 'skipped') return paint('skipped', COLORS.dim, opts.color);
@@ -163,18 +171,20 @@ function phaseMeta(states: readonly RunState[], inPlay: number, parallel: boolea
/** Render the full progress frame as one string (no trailing newline). */
export function renderScan(input: RenderInput, opts: RenderOptions): string {
const byAgent = new Map(input.running.map((r) => [r.agent, r]));
const stateMap = deriveAgentStates(input);
const phases = derivePipeline(input, opts.now);
const lines: string[] = ['', ...headerLines(input, opts), ''];
const metaFor = (name: string, state: RunState): string =>
agentMeta(state, input.state?.agentMetrics[name], byAgent.get(name), agentError(name, input.state, byAgent), opts);
// Only agents that have actually entered play are shown; pending/skipped ones stay hidden.
const inPlay = (s: RunState): boolean => s === 'running' || s === 'completed' || s === 'failed';
for (const phase of PIPELINE) {
const states = phase.agents.map((a) => stateMap.get(a.name) ?? 'pending');
for (const phase of phases) {
const states = phase.agents.map((agent) => agent.state);
const playing = states.filter(inPlay).length;
const phaseRunState: RunState = phaseGlyphState(states);
const phaseRunState = phase.state;
const metaFor = (agent: (typeof phase.agents)[number]): string => {
const metrics = agent.durationMs === null ? undefined : { durationMs: agent.durationMs };
return agentMeta(agent.state, metrics, byAgent.get(agent.name), agent.error, opts, agent.detail);
};
// A single-agent phase carries that agent's own duration/cost on the phase line once it
// starts; a parallel phase gets a "k/N done" summary over the agents in play.
@@ -182,7 +192,7 @@ export function renderScan(input: RenderInput, opts: RenderOptions): string {
const firstState = states[0];
const phaseMetaStr =
!phase.parallel && first && firstState && inPlay(firstState)
? metaFor(first.name, firstState)
? metaFor(first)
: phaseMeta(states, playing, phase.parallel, opts);
lines.push(` ${glyph(phaseRunState, opts)} ${phase.label.padEnd(26)}${phaseMetaStr}`);
@@ -191,7 +201,9 @@ export function renderScan(input: RenderInput, opts: RenderOptions): string {
const agent = phase.agents[i];
const state = states[i];
if (!agent || !state || !inPlay(state)) continue;
lines.push(` ${glyph(state, opts)} ${agent.label.padEnd(18)}${metaFor(agent.name, state)}`);
// Two trailing spaces before padding, so a label wider than the column still separates
// from its meta text; a label inside the column pads to the same width as before.
lines.push(` ${glyph(state, opts)} ${`${agent.label} `.padEnd(AGENT_LABEL_WIDTH)}${metaFor(agent)}`);
}
}
@@ -223,15 +235,44 @@ function footerLines(input: RenderInput, opts: RenderOptions): string[] {
if (isTerminal(input.temporalStatus) && input.state?.summary) {
const wall = formatDuration(input.state.summary.totalDurationMs);
return ['', ` Time Taken ${wall}`];
const lines = ['', ` Time Taken ${wall}`];
// A partial scan names each durable degradation reason through its safe message,
// so the operator never has to guess why the badge is not "completed".
const reasons = input.state.partialReasons ?? [];
if (reasons.length > 0) {
lines.push('', ` ${paint('Why this scan is partial:', COLORS.yellow, opts.color)}`);
for (const reason of reasons) {
lines.push(paint(` - ${reason.message}`, COLORS.dim, opts.color));
}
// The safe message names what degraded; these three name the agentic-SAST failure
// behind it, under the same labels the scan log and worker output use.
const agenticSast = input.state.agenticSast;
if (agenticSast?.status === 'failed') {
if (agenticSast.failedStageLabel !== undefined) {
lines.push(paint(` Agentic SAST stopped at: ${agenticSast.failedStageLabel}`, COLORS.dim, opts.color));
}
if (agenticSast.error !== undefined) {
lines.push(paint(` What happened: ${agenticSast.error}`, COLORS.dim, opts.color));
}
if (agenticSast.errorCode !== undefined) {
lines.push(paint(` Reference code (for a bug report): ${agenticSast.errorCode}`, COLORS.dim, opts.color));
}
}
}
if (input.state.summary.usageAccountingComplete === false) {
lines.push(
paint(' Cost is incomplete — some background work is not included in this total.', COLORS.dim, opts.color),
);
}
return lines;
}
const logsValue = `${prefix} logs ${input.workspace}`;
const temporalValue = temporalDashboardUrl(input.workflowId);
if (isTerminal(input.temporalStatus)) {
const rawReason = input.failureMessage ?? input.state?.error;
const reason = rawReason ? inlineFailureReason(rawReason) : 'no result recorded';
const reason = input.failureMessage ?? input.state?.error ?? 'no result recorded';
return [
footerDivider(opts),
paint(
apps/cli/src/scan/status-json.ts
+22
View File
@@ -8,6 +8,7 @@
import type { DerivedPhase } from './derive.js';
import { derivePipeline, isTerminal, scanElapsedMs } from './derive.js';
import type { PartialReasonView } from './pipeline.js';
import type { RenderInput } from './render.js';
/** Coarse scan status token, mirroring the human status badge in machine-friendly form. */
@@ -27,6 +28,12 @@ export interface StatusJson {
readonly endedAt?: string;
/** Failure text when a failed scan left no readable state. */
readonly failureMessage?: string;
/** Ordered durable degradation reasons with safe messages; present only when non-empty. */
readonly partialReasons?: readonly PartialReasonView[];
/** Agentic SAST outcome, with the worker's sanitized failure sentence and bounded code. */
readonly agenticSast?: { readonly status: string; readonly error?: string; readonly errorCode?: string };
/** False when operational (Capella/reconciliation) spend is known to be incomplete. */
readonly usageAccountingComplete?: boolean;
readonly phases: readonly DerivedPhase[];
}
@@ -34,6 +41,7 @@ export interface StatusJson {
function deriveStatus(input: RenderInput): ScanStatus {
if (!isTerminal(input.temporalStatus)) return 'running';
if (input.state?.status === 'partial') return 'partial';
if (input.state?.status === 'cancelled') return 'cancelled';
switch (input.temporalStatus) {
case 'COMPLETED':
@@ -53,6 +61,9 @@ function deriveStatus(input: RenderInput): ScanStatus {
/** Build the JSON snapshot for a scan at instant `now`. */
export function toStatusJson(input: RenderInput, now: number): StatusJson {
const elapsedMs = scanElapsedMs(input, now);
const partialReasons = input.state?.partialReasons ?? [];
const agenticSast = input.state?.agenticSast;
const usageAccountingComplete = input.state?.summary?.usageAccountingComplete;
return {
workspace: input.workspace,
@@ -63,6 +74,17 @@ export function toStatusJson(input: RenderInput, now: number): StatusJson {
...(input.startedAt !== undefined && { startedAt: new Date(input.startedAt).toISOString() }),
...(input.endedAt !== undefined && { endedAt: new Date(input.endedAt).toISOString() }),
...(input.failureMessage !== undefined && { failureMessage: input.failureMessage }),
...(partialReasons.length > 0 && { partialReasons }),
// Present only when agentic SAST actually ran; a disabled scan omits the key entirely.
...(agenticSast !== undefined &&
agenticSast.status !== 'disabled' && {
agenticSast: {
status: agenticSast.status,
...(agenticSast.error !== undefined && { error: agenticSast.error }),
...(agenticSast.errorCode !== undefined && { errorCode: agenticSast.errorCode }),
},
}),
...(usageAccountingComplete !== undefined && { usageAccountingComplete }),
phases: derivePipeline(input, now),
};
}
apps/cli/src/temporal-client.ts
+30 -4
View File
@@ -9,7 +9,7 @@
import { setTimeout as sleep } from 'node:timers/promises';
import { Client, Connection, WorkflowFailedError, WorkflowNotFoundError } from '@temporalio/client';
import { ACTIVITY_TO_AGENT, type PipelineState } from './scan/pipeline.js';
import { ACTIVITY_TO_PROGRESS, type PipelineState } from './scan/pipeline.js';
const ADDRESS = '127.0.0.1:7233';
const NAMESPACE = 'default';
@@ -20,11 +20,30 @@ const TERMINAL_STATUSES: ReadonlySet<string> = new Set(['COMPLETED', 'FAILED', '
export interface RunningAgent {
readonly agent: string;
readonly label: string;
/** 'agent' rows join the static pipeline tree; 'operation' rows feed the background-work phase. */
readonly kind: 'agent' | 'operation';
/** Set when a persisted parent stage owns this row; the label then reads as that stage's step. */
readonly parentKey?: string;
readonly attempt: number;
readonly startedAt?: number;
readonly lastFailure?: string;
}
/**
* The CLI's activity mirror does not know an activity type the running scan is using, so the
* progress tree cannot be rendered completely. Distinct from a Temporal connection failure.
*/
export class ActivityMirrorError extends Error {
override name = 'ActivityMirrorError' as const;
constructor(activityType: string) {
super(
`This version of the Shannon command line does not recognise part of the running scan\n(${activityType}). Update Shannon, or watch the scan with: shannon logs <workspace>`,
);
}
}
/** Convert a proto ITimestamp (seconds is a Long) to epoch millis. */
function timestampMs(
ts: { seconds?: { toString(): string } | number | null; nanos?: number | null } | null,
@@ -66,12 +85,19 @@ export async function describeScan(workflowId: string): Promise<ScanDescription
const runningAgents: RunningAgent[] = [];
for (const pending of desc.raw.pendingActivities ?? []) {
const agent = ACTIVITY_TO_AGENT[pending.activityType?.name ?? ''];
if (!agent) continue;
const activityType = pending.activityType?.name ?? '';
const progress = ACTIVITY_TO_PROGRESS[activityType];
// Fail closed: skipping an unknown activity would render a quietly incomplete tree.
if (!progress) {
throw new ActivityMirrorError(activityType || 'unknown activity');
}
const lastFailure = pending.lastFailure?.message;
const startedAt = timestampMs(pending.scheduledTime ?? pending.lastStartedTime ?? null);
runningAgents.push({
agent,
agent: progress.key,
label: progress.label,
kind: progress.kind,
...(progress.parentKey !== undefined ? { parentKey: progress.parentKey } : {}),
attempt: pending.attempt ?? 1,
...(startedAt !== undefined ? { startedAt } : {}),
...(lastFailure ? { lastFailure } : {}),
apps/worker/configs/config-schema.json
+12 -10
View File
@@ -125,16 +125,18 @@
},
"additionalProperties": false
},
"vuln_classes": {
"type": "array",
"description": "Vulnerability classes to test. When omitted, all five classes run. When set, only listed classes run; their vuln+exploit agents and report sections are included.",
"items": {
"type": "string",
"enum": ["injection", "xss", "auth", "authz", "ssrf"]
"agentic_sast": {
"type": "object",
"description": "Opt in to agentic static analysis, which reads the repository for vulnerabilities before the pentest and feeds what it finds into the exploitation phase. Off by default. It does not change which vulnerability classes run. If agentic static analysis fails, the pentest continues without its findings and the scan finishes as \"partial\".",
"properties": {
"enabled": {
"type": "string",
"enum": ["true", "false"],
"description": "Set to \"true\" to run agentic static analysis. Defaults to \"false\"."
}
},
"minItems": 1,
"maxItems": 5,
"uniqueItems": true
"required": ["enabled"],
"additionalProperties": false
},
"exploit": {
"type": "string",
@@ -193,7 +195,7 @@
{ "required": ["rules"] },
{ "required": ["authentication", "rules"] },
{ "required": ["description"] },
{ "required": ["vuln_classes"] },
{ "required": ["agentic_sast"] },
{ "required": ["exploit"] },
{ "required": ["report"] },
{ "required": ["rules_of_engagement"] }
apps/worker/configs/example-config.yaml
+8 -2
View File
@@ -4,8 +4,14 @@
# Description of the target environment (optional, max 500 chars)
description: "Next.js e-commerce app on PostgreSQL. Local dev environment — .env files contain local-only credentials, not deployed to production."
# Limit which vulnerability classes run end-to-end (optional, default: all five)
# vuln_classes: [injection, xss, auth, authz, ssrf]
# Every scan runs all five vulnerability classes: injection, xss, auth, authz, and ssrf.
# There is no setting to narrow that.
# Agentic static analysis (optional, default: "false").
# Reads the repository for vulnerabilities before the pentest and feeds them into exploitation.
# It costs extra model time, and if it fails the scan finishes as "partial" without its findings.
# agentic_sast:
# enabled: "true"
# Skip the exploitation phase (optional, default: "true")
# exploit: "false"
apps/worker/package.json
+20
View File
@@ -10,7 +10,27 @@
"./types/agents": "./dist/types/agents.js",
"./pipeline": "./dist/temporal/pipeline.js",
"./activities": "./dist/temporal/activities.js",
"./temporal/reconcile-activity-types": "./dist/temporal/reconcile-activity-types.js",
"./services": "./dist/services/index.js",
"./services/queue-validation": "./dist/services/queue-validation.js",
"./services/renumber-core": "./dist/services/renumber-core.js",
"./services/compaction-core": "./dist/services/compaction-core.js",
"./services/finding-order": "./dist/services/finding-order.js",
"./ai/structured-generation": "./dist/ai/structured-generation.js",
"./ai/pi/source-jail": "./dist/ai/pi/source-jail.js",
"./ai/reconciliation/contracts": "./dist/ai/reconciliation/contracts.js",
"./ai/reconciliation/stage-contracts": "./dist/ai/reconciliation/stage-contracts.js",
"./ai/reconciliation/artifact-store": "./dist/ai/reconciliation/artifact-store.js",
"./ai/reconciliation/schema-version": "./dist/ai/reconciliation/schema-version.js",
"./ai/reconciliation/manifest": "./dist/ai/reconciliation/manifest.js",
"./ai/reconciliation/prepare": "./dist/ai/reconciliation/prepare.js",
"./ai/reconciliation/enrich": "./dist/ai/reconciliation/enrich.js",
"./ai/reconciliation/form": "./dist/ai/reconciliation/form.js",
"./ai/reconciliation/materialize": "./dist/ai/reconciliation/materialize.js",
"./ai/reconciliation/observation-view": "./dist/ai/reconciliation/observation-view.js",
"./ai/reconciliation/labels": "./dist/ai/reconciliation/labels.js",
"./ai/reconciliation/submit-validation": "./dist/ai/reconciliation/submit-validation.js",
"./ai/reconciliation/refs": "./dist/ai/reconciliation/refs.js",
"./config": "./dist/config-parser.js"
},
"scripts": {
apps/worker/src/ai/pi/structured-generation.ts
+7 -3
View File
@@ -6,6 +6,7 @@
import type { AssistantMessage, Context, ToolCall } from '@earendil-works/pi-ai';
import { Value } from 'typebox/value';
import { providerFailureSentence } from '../../services/error-handling.js';
import { type ModelHost, modelHost } from '../model-host.js';
import type {
StructuredGenerationPort,
@@ -92,7 +93,8 @@ async function generate(host: ModelHost, request: StructuredGenerationRequest):
stopReason: 'error',
toolCalls: [],
usage: ZERO_USAGE,
errorMessage: `${failure.type}: ${failure.message}`,
errorMessage: providerFailureSentence(failure),
providerFailure: { type: failure.type, retryable: failure.retryable },
};
}
@@ -102,7 +104,8 @@ async function generate(host: ModelHost, request: StructuredGenerationRequest):
stopReason: 'error',
toolCalls: [],
usage: responseUsage(response),
errorMessage: `${failure.type}: ${failure.message}`,
errorMessage: providerFailureSentence(failure),
providerFailure: { type: failure.type, retryable: failure.retryable },
};
}
if (response.stopReason === 'aborted') {
@@ -114,7 +117,8 @@ async function generate(host: ModelHost, request: StructuredGenerationRequest):
stopReason: 'error',
toolCalls: [],
usage: responseUsage(response),
errorMessage: `${failure.type}: ${failure.message}`,
errorMessage: providerFailureSentence(failure),
providerFailure: { type: failure.type, retryable: failure.retryable },
};
}
apps/worker/src/ai/pi/task-formation-executor.ts
+6 -5
View File
@@ -21,6 +21,7 @@ import {
type ToolDefinition,
} from '@earendil-works/pi-coding-agent';
import { Type } from 'typebox';
import { providerFailureSentence } from '../../services/error-handling.js';
import type { ProviderFailure } from '../../types/errors.js';
import { type ModelHost, modelHost } from '../model-host.js';
import type { ModelSelection } from '../models.js';
@@ -481,7 +482,7 @@ class StandaloneTaskFormationExecutor implements TaskFormationExecutor {
const failure = classifyModelFailure(this.host, error);
throw new TaskFormationExecutorError({
code: 'MODEL_SELECTION_FAILURE',
message: failure.message,
message: providerFailureSentence(failure),
retryable: failure.retryable,
failureKind: 'model',
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
@@ -675,14 +676,14 @@ class StandaloneTaskFormationExecutor implements TaskFormationExecutor {
if (failure.type === 'ConfigurationError') {
return new TaskFormationExecutorError({
code: 'MODEL_CONFIGURATION_FAILURE',
message: failure.message,
message: providerFailureSentence(failure),
retryable: false,
failureKind: 'input',
});
}
return new TaskFormationExecutorError({
code: failure.type === 'AuthenticationError' ? 'PROVIDER_AUTHENTICATION_FAILURE' : 'MODEL_SESSION_FAILURE',
message: failure.message,
message: providerFailureSentence(failure),
retryable: failure.retryable,
failureKind: 'model',
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
@@ -736,7 +737,7 @@ class StandaloneTaskFormationExecutor implements TaskFormationExecutor {
const failure = classifyModelFailure(this.host, outcome.pendingProviderError);
throw new TaskFormationExecutorError({
code: 'PROVIDER_FAILURE',
message: failure.message,
message: providerFailureSentence(failure),
retryable: failure.retryable,
failureKind: 'model',
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
@@ -754,7 +755,7 @@ class StandaloneTaskFormationExecutor implements TaskFormationExecutor {
const failure = classifyModelFailure(this.host, outcome.promptError);
throw new TaskFormationExecutorError({
code: 'MODEL_SESSION_FAILURE',
message: failure.message,
message: providerFailureSentence(failure),
retryable: failure.retryable,
failureKind: 'model',
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
apps/worker/src/ai/reconciliation/form.ts
+18 -9
View File
@@ -347,6 +347,7 @@ export function createFormClassExploitTasks(
...(signal !== undefined && { signal }),
});
let formation: FormClassExploitTasksResult;
try {
const classPolicy = await loadClassPolicy(
input.vulnerabilityClass,
@@ -412,20 +413,28 @@ export function createFormClassExploitTasks(
dropped_unknown_label_count: accepted.droppedUnknownLabelCount,
};
const ref = await writeFormationArtifact(input, workspacesDir, body);
return { ref, metrics, model: `${modelResult.providerId}:${modelResult.modelId}` };
} finally {
// Cleanup runs on every exit path, but a cleanup failure must not overwrite the stage's real
// outcome. Log and swallow it so a successful formation stays successful and a failure keeps
// its original cause for Temporal to classify.
formation = { ref, metrics, model: `${modelResult.providerId}:${modelResult.modelId}` };
} catch (error) {
// A primary error — including cancellation — already owns the outcome, so a cleanup failure
// is logged and swallowed rather than replacing that error's type or cause chain.
try {
await jail.cleanup();
} catch {
logger.error('Task-formation source-jail cleanup failed.', {
stage: 'task-formation',
vulnerabilityClass: input.vulnerabilityClass,
});
logger.error(
'A temporary copy of your source code could not be removed after analysis. It is inside the scan workspace and is safe to delete.',
{
stage: 'task-formation',
vulnerabilityClass: input.vulnerabilityClass,
},
);
}
throw error;
}
// Nothing else is in flight after a successful formation, so an unremoved or unverifiable jail
// is the stage's outcome: it leaves a full copy of the scanned tree on disk and fails here.
await jail.cleanup();
return formation;
};
}
apps/worker/src/ai/reconciliation/sast/enrichment/batch.ts
+8 -4
View File
@@ -1,7 +1,11 @@
/** One bounded structured-generation request for one nonempty class batch. */
import type { ReconciliationClass } from '../../../../types/reconciliation.js';
import type { StructuredGenerationPort, StructuredGenerationRequest } from '../../../structured-generation.js';
import type {
StructuredGenerationPort,
StructuredGenerationRequest,
StructuredGenerationResult,
} from '../../../structured-generation.js';
import { SAST_ENRICHMENT_TOOL_DESCRIPTION, sastEnrichmentToolSchema } from './schema.js';
import { extractVulnerabilities } from './validate.js';
@@ -27,8 +31,8 @@ export interface SastEnrichmentBatchRequest {
// `terminal` marks a failure the stage should not retry. It is set only when the provider itself
// reported a non-retryable failure; an incomplete or empty response defaults to non-terminal so
// Temporal drives another attempt.
function isTerminalProviderFailure(message: string | undefined): boolean {
return message?.startsWith('AuthenticationError:') === true || message?.startsWith('ConfigurationError:') === true;
function isTerminalProviderFailure(result: StructuredGenerationResult): boolean {
return result.providerFailure?.retryable === false;
}
export async function runSastEnrichmentBatch<TModelContext>(
@@ -69,7 +73,7 @@ export async function runSastEnrichmentBatch<TModelContext>(
status: 'failed',
usage,
message: 'SAST enrichment did not return one complete submit_result call',
terminal: isTerminalProviderFailure(result.errorMessage),
terminal: isTerminalProviderFailure(result),
};
}
apps/worker/src/ai/sast/capella/sarif-exporter.ts
+7 -3
View File
@@ -214,7 +214,7 @@ export async function exportCapellaFindings(
const warnings: string[] = [];
const validFindings = rawFindings.filter(isExportableFinding);
const invalidCount = rawFindings.length - validFindings.length;
if (invalidCount > 0) warnings.push(`${invalidCount} invalid finding(s) were excluded`);
if (invalidCount > 0) warnings.push(`${invalidCount} agentic SAST findings were malformed and left out.`);
const gated = validFindings.filter(passesExportGate);
const exported = gated
@@ -224,9 +224,13 @@ export async function exportCapellaFindings(
})
.sort((left, right) => compareText(left.id, right.id));
const excludedCount = gated.length - exported.length;
if (excludedCount > 0) warnings.push(`${excludedCount} finding(s) matched code-path exclusions`);
if (excludedCount > 0) {
warnings.push(`${excludedCount} agentic SAST findings were in paths your config told Shannon to avoid.`);
}
if (validFindings.length > 0 && exported.length === 0) {
warnings.push(`all ${validFindings.length} valid finding record(s) were dropped before export`);
warnings.push(
'Every agentic SAST finding was excluded, so no static-analysis results reached the pentest. Check the avoid rules in your config file.',
);
}
const sarifDocument = buildCapellaSarif(exported, options.repositoryLabel);
apps/worker/src/ai/structured-generation.ts
+8
View File
@@ -17,6 +17,12 @@ export interface StructuredGenerationRequest {
signal?: AbortSignal;
}
/** Typed classification of a failed provider request, set whenever `errorMessage` is. */
export interface StructuredGenerationProviderFailure {
readonly type: 'AuthenticationError' | 'ConfigurationError' | 'AgentExecutionError';
readonly retryable: boolean;
}
/** Host-neutral outcome of one structured generation request. */
export interface StructuredGenerationResult {
stopReason: 'toolUse' | 'stop' | 'length' | 'error' | 'aborted';
@@ -27,6 +33,8 @@ export interface StructuredGenerationResult {
costUsd: number;
};
errorMessage?: string;
/** Consumers branch on this typed flag, never on `errorMessage` text. */
providerFailure?: StructuredGenerationProviderFailure;
}
/** Host-supplied transport that makes exactly one model request per call. */
apps/worker/src/audit/audit-session.ts
+157 -4
View File
@@ -14,11 +14,22 @@
import { PentestError } from '../services/error-handling.js';
import { ErrorCode } from '../types/errors.js';
import type { AgentEndResult } from '../types/index.js';
import type { AgentMetrics } from '../types/metrics.js';
import {
type DurableScanState,
type MiscellaneousOutcome,
type PartialReason,
type ReportProgress,
type ReportSarifDisposition,
RunStateError,
type StoredPdfProvenance,
} from '../types/run-state.js';
import { SessionMutex } from '../utils/concurrency.js';
import { fileExists } from '../utils/file-io.js';
import { formatTimestamp } from '../utils/formatting.js';
import { AgentLogger } from './logger.js';
import { MetricsTracker } from './metrics-tracker.js';
import { initializeAuditStructure, type SessionMetadata } from './utils.js';
import { generateSessionJsonPath, initializeAuditStructure, type SessionMetadata } from './utils.js';
import { type AgentLogDetails, WorkflowLogger, type WorkflowSummary } from './workflow-logger.js';
// Global mutex instance
@@ -170,6 +181,33 @@ export class AuditSession {
* End agent execution (mutex-protected)
*/
async endAgent(agentName: string, result: AgentEndResult): Promise<void> {
await this.finishAgentLogs(agentName, result);
// 3. Acquire mutex before touching session.json
const unlock = await sessionMutex.lock(this.sessionId);
try {
// 4. Reload-then-write inside mutex to prevent lost updates during parallel phases
await this.metricsTracker.reload();
await this.metricsTracker.endAgent(agentName, result);
} finally {
unlock();
}
}
/** Record a successful report-model attempt as a nonterminal durable draft. */
async endReportDraft(result: AgentEndResult): Promise<ReportProgress> {
await this.finishAgentLogs('report', result);
const unlock = await sessionMutex.lock(this.sessionId);
try {
await this.metricsTracker.reload();
return await this.metricsTracker.recordReportDraft(result);
} finally {
unlock();
}
}
private async finishAgentLogs(agentName: string, result: AgentEndResult): Promise<void> {
// 1. Finalize agent log and close the stream
if (this.currentLogger) {
await this.currentLogger.logEvent('agent_end', {
@@ -195,13 +233,128 @@ export class AuditSession {
...(result.error !== undefined && { error: result.error }),
};
await this.workflowLogger.logAgent(agentName, 'end', agentLogDetails);
}
/**
* Initialize fresh durable state or validate a resume record without reconstructing it.
*
* This is the first activity of every run, so it is also where a fresh workspace's
* session.json is created. It therefore takes the workflow id explicitly: initializing
* without one would persist a session with no `originalWorkflowId`, and later calls load
* the existing file rather than rewriting identity, leaving the scan unresolvable.
*/
async initializeDurableScanState(
workflowId: string,
exploit: boolean,
context: 'fresh' | 'resume',
): Promise<DurableScanState> {
if (context === 'resume' && !(await fileExists(generateSessionJsonPath(this.sessionMetadata)))) {
throw new RunStateError('IncompatibleWorkspaceError', 'session-json-missing-on-resume');
}
await this.initialize(workflowId);
// 3. Acquire mutex before touching session.json
const unlock = await sessionMutex.lock(this.sessionId);
try {
// 4. Reload-then-write inside mutex to prevent lost updates during parallel phases
await this.metricsTracker.reload();
await this.metricsTracker.endAgent(agentName, result);
return await this.metricsTracker.initializeDurableScanState(exploit, context);
} finally {
unlock();
}
}
/** Return a validated snapshot of durable execution state. */
async getDurableScanState(): Promise<DurableScanState> {
await this.ensureInitialized();
const unlock = await sessionMutex.lock(this.sessionId);
try {
await this.metricsTracker.reload();
return this.metricsTracker.getDurableScanState();
} finally {
unlock();
}
}
/** Persist a `miscellaneous` branch outcome under the session lock. */
async updateMiscellaneousOutcome(outcome: MiscellaneousOutcome): Promise<DurableScanState> {
await this.ensureInitialized();
const unlock = await sessionMutex.lock(this.sessionId);
try {
await this.metricsTracker.reload();
return await this.metricsTracker.updateMiscellaneousOutcome(outcome);
} finally {
unlock();
}
}
/** Persist the ordered renumber-failure set and durable partial reasons before assembly. */
async initializeReportProgress(
failedClasses: readonly import('../types/reconciliation.js').ReconciliationClass[],
partialReasons: readonly PartialReason[],
): Promise<ReportProgress> {
await this.ensureInitialized();
const unlock = await sessionMutex.lock(this.sessionId);
try {
await this.metricsTracker.reload();
return await this.metricsTracker.initializeReportProgress(failedClasses, partialReasons);
} finally {
unlock();
}
}
/** Persist the post-compaction canonical report checkpoint without terminal success. */
async recordCanonicalReportCheckpoint(
checkpoint: string,
appendReasons: readonly PartialReason[] = [],
): Promise<ReportProgress> {
await this.ensureInitialized();
const unlock = await sessionMutex.lock(this.sessionId);
try {
await this.metricsTracker.reload();
return await this.metricsTracker.recordCanonicalReportCheckpoint(checkpoint, appendReasons);
} finally {
unlock();
}
}
/** Atomically mark report finalized and successful after external proof validation. */
async finalizeReportProgress(
finalCheckpoint: string,
manifestSha256: string,
terminal: {
readonly sarifDisposition: ReportSarifDisposition;
readonly pdfProvenance: StoredPdfProvenance | null;
readonly partialReasons: readonly PartialReason[];
},
): Promise<ReportProgress> {
await this.ensureInitialized();
const unlock = await sessionMutex.lock(this.sessionId);
try {
await this.metricsTracker.reload();
return await this.metricsTracker.finalizeReportProgress(finalCheckpoint, manifestSha256, terminal);
} finally {
unlock();
}
}
/** Return an invalid model draft to pending without erasing its billable attempt. */
async rollbackReportDraft(): Promise<ReportProgress> {
await this.ensureInitialized();
const unlock = await sessionMutex.lock(this.sessionId);
try {
await this.metricsTracker.reload();
return await this.metricsTracker.rollbackReportDraft();
} finally {
unlock();
}
}
/** Read persisted report metrics for model-skip resume. */
async getReportMetrics(): Promise<AgentMetrics> {
await this.ensureInitialized();
const unlock = await sessionMutex.lock(this.sessionId);
try {
await this.metricsTracker.reload();
return this.metricsTracker.getReportMetrics();
} finally {
unlock();
}
apps/worker/src/audit/metrics-tracker.ts
+403 -43
View File
@@ -15,6 +15,21 @@ import { PentestError } from '../services/error-handling.js';
import { AGENT_PHASE_MAP, type PhaseName } from '../session-manager.js';
import { ErrorCode } from '../types/errors.js';
import type { AgentEndResult, AgentName } from '../types/index.js';
import type { AgentMetrics } from '../types/metrics.js';
import {
appendPartialReasons,
createInitialDurableScanState,
type DurableScanState,
isDurableScanState,
isOrderedPartialReasonSet,
type MiscellaneousOutcome,
type PartialReason,
type ReportProgress,
type ReportSarifDisposition,
RunStateError,
recordMiscellaneousOutcome,
type StoredPdfProvenance,
} from '../types/run-state.js';
import { atomicWrite, fileExists, readJson } from '../utils/file-io.js';
import { calculatePercentage, formatTimestamp } from '../utils/formatting.js';
import { generateSessionJsonPath, type SessionMetadata } from './utils.js';
@@ -78,6 +93,7 @@ interface SessionData {
phases: Record<string, PhaseMetrics>;
agents: Record<string, AgentAuditMetrics>;
};
durableScanState?: DurableScanState;
}
interface ActiveTimer {
@@ -176,51 +192,11 @@ export class MetricsTracker {
);
}
// 1. Initialize agent metrics if first time seeing this agent
const existingAgent = this.data.metrics.agents[agentName];
const agent = existingAgent ?? {
status: 'in-progress' as const,
attempts: [],
final_duration_ms: 0,
total_cost_usd: 0,
total_input_tokens: 0,
total_output_tokens: 0,
total_cache_read_tokens: 0,
total_cache_write_tokens: 0,
};
this.data.metrics.agents[agentName] = agent;
// 2. Build attempt record with optional model/error fields
const attempt: AttemptData = {
attempt_number: result.attemptNumber,
duration_ms: result.duration_ms,
cost_usd: result.cost_usd,
success: result.success,
timestamp: formatTimestamp(),
...(result.input_tokens !== undefined && { input_tokens: result.input_tokens }),
...(result.output_tokens !== undefined && { output_tokens: result.output_tokens }),
...(result.cache_read_tokens !== undefined && { cache_read_tokens: result.cache_read_tokens }),
...(result.cache_write_tokens !== undefined && { cache_write_tokens: result.cache_write_tokens }),
...(result.turns !== undefined && { turns: result.turns }),
};
if (result.model) {
attempt.model = result.model;
if (agentName === 'report' && result.success) {
throw new RunStateError('DurableStateConflictError', 'report-success-requires-terminal-promotion');
}
if (result.error) {
attempt.error = result.error;
}
// 3. Append attempt to history
agent.attempts.push(attempt);
// 4. Recalculate totals across all attempts (includes failures)
agent.total_cost_usd = agent.attempts.reduce((sum, a) => sum + a.cost_usd, 0);
agent.total_input_tokens = agent.attempts.reduce((sum, a) => sum + (a.input_tokens ?? 0), 0);
agent.total_output_tokens = agent.attempts.reduce((sum, a) => sum + (a.output_tokens ?? 0), 0);
agent.total_cache_read_tokens = agent.attempts.reduce((sum, a) => sum + (a.cache_read_tokens ?? 0), 0);
agent.total_cache_write_tokens = agent.attempts.reduce((sum, a) => sum + (a.cache_write_tokens ?? 0), 0);
const agent = this.appendAttempt(agentName, result);
// 5. Update agent status based on outcome
if (result.success) {
@@ -235,6 +211,11 @@ export class MetricsTracker {
if (result.checkpoint) {
agent.checkpoint = result.checkpoint;
}
if (agentName === 'miscellaneous-exploit') {
const durableState = this.requireDurableScanState();
this.data.durableScanState = recordMiscellaneousOutcome(durableState, 'completed');
}
} else {
// A non-final failed attempt stays in-progress (Temporal will retry); only the
// terminal attempt (or an unqualified failure) marks the agent failed.
@@ -251,6 +232,319 @@ export class MetricsTracker {
await this.save();
}
/** Initialize or validate the schema-1 state without reconstructing a missing resume record. */
async initializeDurableScanState(exploit: boolean, context: 'fresh' | 'resume'): Promise<DurableScanState> {
const data = this.requireData();
const existing = data.durableScanState;
if (existing !== undefined) {
if (!isDurableScanState(existing)) {
throw new RunStateError('CorruptedSessionError', 'durable-state-malformed');
}
if (existing.exploit !== exploit) {
throw new RunStateError('IncompatibleWorkspaceError', 'exploit-mode-changed');
}
return structuredClone(existing);
}
if (context === 'resume') {
throw new RunStateError('IncompatibleWorkspaceError', 'durable-state-missing-on-resume');
}
const hasRecordedWork =
Object.keys(data.metrics.agents).length > 0 || (data.session.resumeAttempts?.length ?? 0) > 0;
if (hasRecordedWork) {
throw new RunStateError('CorruptedSessionError', 'durable-state-missing-after-work');
}
const initialized = createInitialDurableScanState(exploit);
data.durableScanState = initialized;
await this.save();
return structuredClone(initialized);
}
/** Return validated durable state. */
getDurableScanState(): DurableScanState {
return structuredClone(this.requireDurableScanState());
}
/** Persist the internal `miscellaneous` result and append its agent only for actionable exploitation. */
async updateMiscellaneousOutcome(outcome: MiscellaneousOutcome): Promise<DurableScanState> {
const data = this.requireData();
const next = recordMiscellaneousOutcome(this.requireDurableScanState(), outcome);
if (!isDurableScanState(next)) {
throw new RunStateError('DurableStateConflictError', 'miscellaneous-outcome-produced-invalid-state');
}
data.durableScanState = next;
await this.save();
return structuredClone(next);
}
/** Persist the complete failed-class set and durable partial reasons before report assembly. */
async initializeReportProgress(
failedClasses: readonly import('../types/reconciliation.js').ReconciliationClass[],
partialReasons: readonly PartialReason[],
): Promise<ReportProgress> {
const data = this.requireData();
const durableState = this.requireDurableScanState();
if (!isOrderedPartialReasonSet(partialReasons)) {
throw new RunStateError('DurableStateConflictError', 'report-pending-reasons-invalid');
}
if (durableState.report !== undefined) {
if (!this.arraysEqual(durableState.report.renumber_failed_classes, failedClasses)) {
throw new RunStateError('DurableStateConflictError', 'report-failed-class-set-changed');
}
// A lost-acknowledgement re-drive adopts the same set; a resume may append newly
// observed reasons, but never removes a durable one. Append preserves every existing
// member, so an unchanged length means nothing new was observed.
const merged = appendPartialReasons(durableState.report.partial_reasons, partialReasons);
if (merged.length === durableState.report.partial_reasons.length) {
return structuredClone(durableState.report);
}
const report: ReportProgress = { ...durableState.report, partial_reasons: merged };
const next = { ...durableState, report };
if (!isDurableScanState(next)) {
throw new RunStateError('DurableStateConflictError', 'report-pending-reasons-conflict');
}
data.durableScanState = next;
await this.save();
return structuredClone(report);
}
const report: ReportProgress = {
stage: 'pending',
renumber_failed_classes: [...failedClasses],
partial_reasons: appendPartialReasons([], partialReasons),
};
const next = { ...durableState, report };
if (!isDurableScanState(next)) {
throw new RunStateError('DurableStateConflictError', 'report-pending-invalid');
}
data.durableScanState = next;
await this.save();
return structuredClone(report);
}
/** Record billable report-model metrics and a real Git checkpoint without terminal success. */
async recordReportDraft(result: AgentEndResult): Promise<ReportProgress> {
const data = this.requireData();
const checkpoint = result.checkpoint;
if (!result.success || checkpoint === undefined) {
throw new RunStateError('DurableStateConflictError', 'report-draft-requires-success-checkpoint');
}
const durableState = this.requireDurableScanState();
const current = durableState.report;
if (current === undefined || current.stage === 'finalized') {
throw new RunStateError('DurableStateConflictError', 'report-draft-invalid-source-stage');
}
if (current.stage === 'draft') {
if (current.model_checkpoint !== checkpoint) {
throw new RunStateError('DurableStateConflictError', 'report-model-checkpoint-conflict');
}
return structuredClone(current);
}
const agent = this.appendAttempt('report', result);
agent.status = 'in-progress';
agent.final_duration_ms = result.duration_ms;
agent.checkpoint = checkpoint;
if (result.model !== undefined) {
agent.model = result.model;
} else {
delete agent.model;
}
const report: ReportProgress = {
stage: 'draft',
renumber_failed_classes: [...current.renumber_failed_classes],
partial_reasons: [...current.partial_reasons],
model_checkpoint: checkpoint,
};
const next = { ...durableState, report };
if (!isDurableScanState(next)) {
throw new RunStateError('DurableStateConflictError', 'report-draft-invalid');
}
data.durableScanState = next;
this.activeTimers.delete('report');
this.recalculateAggregations();
await this.save();
return structuredClone(report);
}
/** Record the post-compaction canonical checkpoint while keeping report nonterminal. */
async recordCanonicalReportCheckpoint(
checkpoint: string,
appendReasons: readonly PartialReason[] = [],
): Promise<ReportProgress> {
const data = this.requireData();
const durableState = this.requireDurableScanState();
const current = durableState.report;
if (current?.stage === 'finalized') {
if (current.canonical_checkpoint !== checkpoint) {
throw new RunStateError('DurableStateConflictError', 'report-canonical-checkpoint-conflict');
}
return structuredClone(current);
}
if (current?.stage !== 'draft') {
throw new RunStateError('DurableStateConflictError', 'report-canonical-invalid-source-stage');
}
const mergedReasons = appendPartialReasons(current.partial_reasons, appendReasons);
if (current.canonical_checkpoint !== undefined) {
if (current.canonical_checkpoint !== checkpoint) {
throw new RunStateError('DurableStateConflictError', 'report-canonical-checkpoint-conflict');
}
if (mergedReasons.length === current.partial_reasons.length) {
return structuredClone(current);
}
}
const report: ReportProgress = {
...current,
partial_reasons: mergedReasons,
canonical_checkpoint: checkpoint,
};
const next = { ...durableState, report };
if (!isDurableScanState(next)) {
throw new RunStateError('DurableStateConflictError', 'report-canonical-invalid');
}
data.durableScanState = next;
await this.save();
return structuredClone(report);
}
/**
* Promote a verified finalization commit to the only terminal report state.
*
* `final_checkpoint` and the manifest digest are strict match-or-conflict fields. The SARIF
* disposition and its `report_sarif_failed` reason are derived from the committed manifest,
* partial reasons stay append-only, and the PDF provenance is replaceable after finalization.
*/
async finalizeReportProgress(
finalCheckpoint: string,
manifestSha256: string,
terminal: {
readonly sarifDisposition: ReportSarifDisposition;
readonly pdfProvenance: StoredPdfProvenance | null;
readonly partialReasons: readonly PartialReason[];
},
): Promise<ReportProgress> {
const data = this.requireData();
const durableState = this.requireDurableScanState();
const current = durableState.report;
if (current?.stage === 'finalized') {
if (current.final_checkpoint !== finalCheckpoint || current.finalization_manifest_sha256 !== manifestSha256) {
throw new RunStateError('DurableStateConflictError', 'report-final-checkpoint-conflict');
}
if (current.sarif_disposition !== terminal.sarifDisposition) {
throw new RunStateError('DurableStateConflictError', 'report-final-disposition-conflict');
}
const adopted: ReportProgress = {
...current,
partial_reasons: appendPartialReasons(current.partial_reasons, terminal.partialReasons),
...(terminal.pdfProvenance !== null ? { pdf_provenance: terminal.pdfProvenance } : {}),
};
if (terminal.pdfProvenance === null && 'pdf_provenance' in adopted) {
const { pdf_provenance: _removed, ...withoutProvenance } = adopted;
return await this.persistFinalizedReport(data, durableState, withoutProvenance as ReportProgress);
}
return await this.persistFinalizedReport(data, durableState, adopted);
}
if (current?.stage !== 'draft' || current.canonical_checkpoint === undefined) {
throw new RunStateError('DurableStateConflictError', 'report-final-invalid-source-stage');
}
const sarifReasons: readonly PartialReason[] =
terminal.sarifDisposition === 'render_failed' ? [{ code: 'report_sarif_failed' }] : [];
const report: ReportProgress = {
stage: 'finalized',
renumber_failed_classes: [...current.renumber_failed_classes],
partial_reasons: appendPartialReasons(current.partial_reasons, [...terminal.partialReasons, ...sarifReasons]),
model_checkpoint: current.model_checkpoint,
canonical_checkpoint: current.canonical_checkpoint,
final_checkpoint: finalCheckpoint,
finalization_manifest_sha256: manifestSha256,
sarif_disposition: terminal.sarifDisposition,
...(terminal.pdfProvenance !== null && { pdf_provenance: terminal.pdfProvenance }),
};
const agent = data.metrics.agents.report;
if (agent === undefined || agent.attempts.length === 0) {
throw new RunStateError('DurableStateConflictError', 'report-final-without-model-metrics');
}
const persisted = await this.persistFinalizedReport(data, durableState, report, () => {
agent.status = 'success';
agent.checkpoint = finalCheckpoint;
const latestAttempt = agent.attempts.at(-1);
agent.final_duration_ms = latestAttempt?.duration_ms ?? agent.final_duration_ms;
this.recalculateAggregations();
});
return persisted;
}
private async persistFinalizedReport(
data: SessionData,
durableState: DurableScanState,
report: ReportProgress,
beforeSave?: () => void,
): Promise<ReportProgress> {
const next = { ...durableState, report };
if (!isDurableScanState(next)) {
throw new RunStateError('DurableStateConflictError', 'report-final-invalid');
}
beforeSave?.();
data.durableScanState = next;
await this.save();
return structuredClone(report);
}
/** Roll back only report state after a coherent draft shape fails checkpoint validation. */
async rollbackReportDraft(): Promise<ReportProgress> {
const data = this.requireData();
const durableState = this.requireDurableScanState();
const current = durableState.report;
if (current?.stage !== 'draft') {
throw new RunStateError('DurableStateConflictError', 'report-draft-rollback-invalid-source-stage');
}
const report: ReportProgress = {
stage: 'pending',
renumber_failed_classes: [...current.renumber_failed_classes],
partial_reasons: [...current.partial_reasons],
};
const agent = data.metrics.agents.report;
if (agent !== undefined) {
agent.status = 'in-progress';
delete agent.checkpoint;
delete agent.model;
}
data.durableScanState = { ...durableState, report };
this.recalculateAggregations();
await this.save();
return structuredClone(report);
}
/** Return persisted report metrics for a coherent draft/finalized model-skip path. */
getReportMetrics(): AgentMetrics {
const durableState = this.requireDurableScanState();
if (durableState.report?.stage !== 'draft' && durableState.report?.stage !== 'finalized') {
throw new RunStateError('DurableStateConflictError', 'report-metrics-before-draft');
}
const agent = this.requireData().metrics.agents.report;
if (agent === undefined || agent.attempts.length === 0) {
throw new RunStateError('CorruptedSessionError', 'report-draft-metrics-missing');
}
const latest = agent.attempts.at(-1);
return {
durationMs: agent.final_duration_ms,
inputTokens: agent.total_input_tokens,
outputTokens: agent.total_output_tokens,
cacheReadTokens: agent.total_cache_read_tokens,
cacheWriteTokens: agent.total_cache_write_tokens,
costUsd: agent.total_cost_usd,
numTurns: agent.attempts.reduce((sum, attempt) => sum + (attempt.turns ?? 0), 0),
...(latest?.model !== undefined && { model: latest.model }),
...(agent.checkpoint !== undefined && { checkpoint: agent.checkpoint }),
skipped: true,
};
}
/**
* Update session status
*/
@@ -294,6 +588,12 @@ export class MetricsTracker {
this.data.session.resumeAttempts = [];
}
// A lost-acknowledgement re-drive of the same resume adopts the earlier record instead
// of appending a duplicate row for the same workflow id.
if (this.data.session.resumeAttempts.some((attempt) => attempt.workflowId === workflowId)) {
return;
}
// Add new resume attempt
const resumeAttempt: ResumeAttempt = {
workflowId,
@@ -399,4 +699,64 @@ export class MetricsTracker {
async reload(): Promise<void> {
this.data = await readJson<SessionData>(this.sessionJsonPath);
}
private requireData(): SessionData {
if (this.data === null) {
throw new RunStateError('CorruptedSessionError', 'metrics-tracker-not-initialized');
}
return this.data;
}
private requireDurableScanState(): DurableScanState {
const durableState = this.requireData().durableScanState;
if (durableState === undefined) {
throw new RunStateError('CorruptedSessionError', 'durable-state-missing');
}
if (!isDurableScanState(durableState)) {
throw new RunStateError('CorruptedSessionError', 'durable-state-malformed');
}
return durableState;
}
private appendAttempt(agentName: string, result: AgentEndResult): AgentAuditMetrics {
const data = this.requireData();
const existingAgent = data.metrics.agents[agentName];
const agent = existingAgent ?? {
status: 'in-progress' as const,
attempts: [],
final_duration_ms: 0,
total_cost_usd: 0,
total_input_tokens: 0,
total_output_tokens: 0,
total_cache_read_tokens: 0,
total_cache_write_tokens: 0,
};
data.metrics.agents[agentName] = agent;
const attempt: AttemptData = {
attempt_number: result.attemptNumber,
duration_ms: result.duration_ms,
cost_usd: result.cost_usd,
success: result.success,
timestamp: formatTimestamp(),
...(result.input_tokens !== undefined && { input_tokens: result.input_tokens }),
...(result.output_tokens !== undefined && { output_tokens: result.output_tokens }),
...(result.cache_read_tokens !== undefined && { cache_read_tokens: result.cache_read_tokens }),
...(result.cache_write_tokens !== undefined && { cache_write_tokens: result.cache_write_tokens }),
...(result.turns !== undefined && { turns: result.turns }),
...(result.model !== undefined && { model: result.model }),
...(result.error !== undefined && { error: result.error }),
};
agent.attempts.push(attempt);
agent.total_cost_usd = agent.attempts.reduce((sum, entry) => sum + entry.cost_usd, 0);
agent.total_input_tokens = agent.attempts.reduce((sum, entry) => sum + (entry.input_tokens ?? 0), 0);
agent.total_output_tokens = agent.attempts.reduce((sum, entry) => sum + (entry.output_tokens ?? 0), 0);
agent.total_cache_read_tokens = agent.attempts.reduce((sum, entry) => sum + (entry.cache_read_tokens ?? 0), 0);
agent.total_cache_write_tokens = agent.attempts.reduce((sum, entry) => sum + (entry.cache_write_tokens ?? 0), 0);
return agent;
}
private arraysEqual<T>(left: readonly T[], right: readonly T[]): boolean {
return left.length === right.length && left.every((value, index) => value === right[index]);
}
}
apps/worker/src/audit/workflow-logger.ts
+67 -3
View File
@@ -29,12 +29,33 @@ export interface AgentMetricsSummary {
costUsd: number | null;
}
/** Mirror of the derived partial-reason view; the safe message is already resolved. */
export interface WorkflowSummaryPartialReason {
readonly code: string;
readonly message: string;
readonly vulnerabilityClass?: string;
readonly stage?: string;
}
export interface WorkflowSummary {
status: 'completed' | 'failed' | 'cancelled' | 'partial';
totalDurationMs: number;
totalCostUsd: number;
/** Agents that actually ran. Mutually exclusive from `skippedAgents`. */
completedAgents: string[];
/** Expected agents that never ran because their class had nothing to exploit. */
skippedAgents?: readonly string[];
agentMetrics: Record<string, AgentMetricsSummary>;
/** Ordered durable degradation reasons; present and non-empty exactly for partial runs. */
partialReasons?: readonly WorkflowSummaryPartialReason[];
/** False when operational (Capella/reconciliation) spend is known to be incomplete. */
usageAccountingComplete?: boolean;
/** Reader-facing name of the stage a failed agentic-SAST run stopped at. */
agenticSastFailedStage?: string;
/** Sanitized failure sentence from a failed agentic-SAST run; safe for operator output. */
agenticSastFailureMessage?: string;
/** Bounded machine code from a failed agentic-SAST run, when one was preserved. */
agenticSastErrorCode?: string;
error?: string;
}
@@ -322,7 +343,22 @@ export class WorkflowLogger {
async logWorkflowComplete(summary: WorkflowSummary): Promise<void> {
await this.ensureInitialized();
const status = summary.status === 'completed' ? 'COMPLETED' : 'FAILED';
// Each terminal status prints its own header so partial and cancelled runs are never
// mislabelled as full successes or failures. The CLI log tailer stops on exactly these
// headings (COMPLETION_PATTERN in apps/cli/src/commands/logs.ts); adding one here without
// adding it there strands `shannon logs` on a finished scan.
const STATUS_HEADERS: Record<WorkflowSummary['status'], string> = {
completed: 'COMPLETED',
partial: 'PARTIAL',
cancelled: 'CANCELLED',
failed: 'FAILED',
};
const status = STATUS_HEADERS[summary.status];
// completedAgents and skippedAgents are mutually exclusive: an agent that was skipped
// because its class had nothing to exploit is tracked only in skippedAgents.
const skippedAgents = summary.skippedAgents ?? [];
const ranCount = summary.completedAgents.length;
const lines: string[] = [
'',
@@ -333,14 +369,36 @@ export class WorkflowLogger {
`Status: ${summary.status}`,
`Duration: ${formatDuration(summary.totalDurationMs)}`,
`Total Cost: $${summary.totalCostUsd.toFixed(4)}`,
`Agents: ${summary.completedAgents.length} completed`,
`Agents: ${ranCount} ran, ${skippedAgents.length} skipped`,
];
if (summary.usageAccountingComplete === false) {
lines.push('Cost Note: Cost is incomplete — some background work is not included in this total.');
}
if (summary.error) {
lines.push(this.formatErrorBlock(summary.error).trimEnd());
}
if (summary.completedAgents.length > 0) {
if (summary.partialReasons !== undefined && summary.partialReasons.length > 0) {
lines.push('');
lines.push('Why this scan is partial:');
for (const reason of summary.partialReasons) {
lines.push(` - ${reason.message}`);
}
// The reason above says what degraded; these three name the agentic-SAST failure
// behind it, under the same labels the terminal and worker output use.
if (summary.agenticSastFailedStage !== undefined) {
lines.push(` Agentic SAST stopped at: ${summary.agenticSastFailedStage}`);
}
if (summary.agenticSastFailureMessage !== undefined) {
lines.push(` What happened: ${summary.agenticSastFailureMessage}`);
}
if (summary.agenticSastErrorCode !== undefined) {
lines.push(` Reference code (for a bug report): ${summary.agenticSastErrorCode}`);
}
}
if (summary.completedAgents.length > 0 || skippedAgents.length > 0) {
lines.push('');
lines.push('Agent Breakdown:');
@@ -354,6 +412,12 @@ export class WorkflowLogger {
lines.push(` - ${agentName}`);
}
}
for (const agentName of skippedAgents) {
lines.push(` - ${agentName} (skipped — nothing to exploit)`);
}
}
for (const agentName of skippedAgents) {
lines.push(` - ${agentName} (skipped — nothing to exploit)`);
}
lines.push('================================================================================');
apps/worker/src/config-parser.ts
+26 -12
View File
@@ -10,15 +10,21 @@ import type { FormatsPlugin } from 'ajv-formats';
import yaml from 'js-yaml';
import { fs } from 'zx';
import { PentestError } from './services/error-handling.js';
import {
ALL_VULN_CLASSES,
type Authentication,
type Config,
type DistributedConfig,
type Rule,
} from './types/config.js';
import type { Authentication, Config, DistributedConfig, Rule } from './types/config.js';
import { ErrorCode } from './types/errors.js';
/**
* Parses and validates scan configuration YAML against config-schema.json, then
* distributes it into the plain values consumed by prompts and services.
*
* The schema is closed: every object in config-schema.json sets `additionalProperties:
* false`, so an unrecognized field anywhere in the config is a hard validation failure
* rather than a silently ignored typo. There is no public way to select which analysis
* classes run; the schema only exposes steering knobs (rules, authentication,
* agentic_sast.enabled, exploit, report, rules_of_engagement) on top of the fixed
* five-class pipeline.
*/
// Handle ESM/CJS interop for ajv-formats using require
const require = createRequire(import.meta.url);
const addFormats: FormatsPlugin = require('ajv-formats');
@@ -42,6 +48,10 @@ try {
});
}
// Free-text config fields (description, rules_of_engagement, rule values, login fields,
// report.guidance) get interpolated verbatim into agent prompts via prompt-manager.ts.
// These patterns block the more obvious ways a scan config could smuggle markup, script
// URLs, or path traversal into that prompt text or into a rendered value.
const DANGEROUS_PATTERNS: RegExp[] = [
/\.\.\//, // Path traversal
/[<>]/, // HTML/XML injection
@@ -312,6 +322,9 @@ export const parseConfigYAML = (yamlContent: string): Config => {
return config as Config;
};
// Runs before schema validation so a renamed field fails with a specific "renamed to X"
// message instead of the generic "additionalProperties" rejection the closed schema
// would otherwise produce for the old field name.
function checkDeprecatedFields(config: Config): void {
const messages: string[] = [];
@@ -387,7 +400,7 @@ const validateConfig = (config: Config): void => {
!!config.rules ||
!!config.authentication ||
!!config.description ||
!!config.vuln_classes ||
!!config.agentic_sast ||
config.exploit !== undefined ||
!!config.report ||
!!config.rules_of_engagement;
@@ -673,9 +686,10 @@ export const distributeConfig = (config: Config | null): DistributedConfig => {
const authentication = config?.authentication || null;
const description = config?.description?.trim() || '';
const vuln_classes =
config?.vuln_classes && config.vuln_classes.length > 0 ? [...config.vuln_classes] : [...ALL_VULN_CLASSES];
// The schema types boolean-shaped fields (exploit, report.sarif, agentic_sast.enabled)
// as a string enum ("true"/"false") rather than JSON boolean, since YAML's FAILSAFE_SCHEMA
// parses bareword true/false as strings. The string comparison here is intentional, not
// a leftover from a looser type.
const exploit = config?.exploit !== undefined ? config.exploit === 'true' : true;
const report = {
@@ -693,7 +707,7 @@ export const distributeConfig = (config: Config | null): DistributedConfig => {
focus: focus.map(sanitizeRule),
authentication: authentication ? sanitizeAuthentication(authentication) : null,
description,
vuln_classes,
...(config?.agentic_sast?.enabled === 'true' && { agenticSast: true as const }),
exploit,
report,
rules_of_engagement,
apps/worker/src/paths.ts
+3
View File
@@ -46,6 +46,9 @@ export const REPORT_JSON_FILENAME = 'report.json';
/** SARIF 2.1.0 log, written for exploit=true runs unless report.sarif is set to false. */
export const SARIF_FILENAME = 'report.sarif';
/** Deterministic receipt for the canonical report finalization commit. */
export const REPORT_FINALIZATION_MANIFEST_FILENAME = 'report_finalization_manifest.json';
/**
* Resolve the session.json path for a run directory, preferring the current
* `.shannon/` location and falling back to the legacy run-root location so
apps/worker/src/services/agent-execution.ts
+39 -3
View File
@@ -33,6 +33,7 @@ import type { AgentEndResult } from '../types/audit.js';
import { ErrorCode, type PentestErrorType } from '../types/errors.js';
import type { AgentMetrics } from '../types/metrics.js';
import { err, isErr, ok, type Result } from '../types/result.js';
import { assertFixedAnalysisScope } from '../types/run-state.js';
import { getAgentGitPaths } from './agent-git-paths.js';
import type { ConfigLoaderService } from './config-loader.js';
import { PentestError } from './error-handling.js';
@@ -51,11 +52,14 @@ export interface AgentExecutionInput {
configYAML?: string | undefined;
pipelineTestingMode?: boolean | undefined;
attemptNumber: number;
/** Workflow-resolved fixed scope; prompt generation never derives this from public config. */
analysisClasses: readonly import('../types/config.js').VulnClass[];
promptDir?: string | undefined;
customTools?: import('@earendil-works/pi-coding-agent').ToolDefinition[];
failedClasses?: readonly import('../types/config.js').VulnClass[] | undefined;
// Renders the deliverable to disk; invoked after validation, before the success commit.
writeDeliverable?: (deliverablesPath: string) => Promise<void>;
writeDeliverable?: (deliverablesPath: string, execution: { readonly model?: string }) => Promise<void>;
successDisposition?: 'terminal' | 'report-draft';
cancellationSignal?: AbortSignal | undefined;
}
@@ -145,14 +149,29 @@ export class AgentExecutionService {
configYAML,
pipelineTestingMode = false,
attemptNumber,
analysisClasses,
promptDir,
customTools,
failedClasses,
writeDeliverable,
successDisposition = 'terminal',
cancellationSignal,
} = input;
const gitPaths = getAgentGitPaths(agentName);
assertFixedAnalysisScope(analysisClasses);
if (successDisposition === 'report-draft' && agentName !== 'report') {
return err(
new PentestError(
'Draft success is reserved for the report agent',
'validation',
false,
{ agentName },
ErrorCode.CONFIG_VALIDATION_FAILED,
),
);
}
// 1. Load config (pre-parsed configData → raw YAML → file path)
const configResult = await this.configLoader.loadOptional(configPath, configData, configYAML);
if (isErr(configResult)) {
@@ -170,6 +189,7 @@ export class AgentExecutionService {
webUrl,
repoPath,
AUTH_STATE_FILE: authStateFile(auditSession.sessionMetadata),
analysisClasses,
...(failedClasses !== undefined && { failedClasses }),
},
distributedConfig,
@@ -278,7 +298,9 @@ export class AgentExecutionService {
// 10. Render the deliverable to disk so the success commit below stages it
if (writeDeliverable) {
await writeDeliverable(deliverablesPath);
await writeDeliverable(deliverablesPath, {
...(result.model !== undefined && { model: result.model }),
});
}
// 11. Success - commit deliverables (scoped) and capture the checkpoint hash
@@ -287,6 +309,15 @@ export class AgentExecutionService {
return gitFailureForAgent(agentName, 'commit successful results', commitResult.error);
}
commitHash = commitResult.commitHash;
if (successDisposition === 'report-draft' && commitHash === undefined) {
return new PentestError(
'The report was written but could not be saved. Re-running this workspace retries the reporting phase without repeating the analysis.',
'filesystem',
false,
{ agentName },
ErrorCode.GIT_CHECKPOINT_FAILED,
);
}
return null;
} catch (error) {
if (error instanceof PentestError) return error;
@@ -331,7 +362,11 @@ export class AgentExecutionService {
model: result.model,
...(commitHash && { checkpoint: commitHash }),
};
await auditSession.endAgent(agentName, endResult);
if (successDisposition === 'report-draft') {
await auditSession.endReportDraft(endResult);
} else {
await auditSession.endAgent(agentName, endResult);
}
return ok(endResult);
}
@@ -419,6 +454,7 @@ export class AgentExecutionService {
costUsd: endResult.cost_usd,
numTurns: result.turns ?? null,
model: result.model,
...(endResult.checkpoint !== undefined && { checkpoint: endResult.checkpoint }),
};
}
}
apps/worker/src/services/code-location-join.ts
+101 -45
View File
@@ -4,75 +4,131 @@
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/**
* Attach vuln-queue code locations to collected findings.
*
* The vuln agent authors `code_locations` once, into its queue. Every stage after that used to
* re-transcribe them — the exploit agent into its evidence, the report agent into `add_finding` —
* and each hop lost some: 100% in the queue, 98% in the evidence, 42-63% by the report. Nothing
* about the copy is a judgement call, and `finding_id` matches the queue `ID` exactly, so the
* locations are joined here instead of being asked for again.
*/
/** Join analysis and SAST locations from committed report-facing tasks without mixing location lanes. */
import { fs, path } from 'zx';
import type { QueueCodeLocation } from '../ai/queue-schemas.js';
import type { SastSourceLocation } from '../ai/reconciliation/contracts.js';
import type { AddFindingInput } from '../collectors/finding-collector.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import { ALL_VULN_CLASSES } from '../types/config.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import { readCommittedFile } from './git-manager.js';
import { renumberMapPath } from './renumber-core.js';
interface QueueEntry {
ID?: string;
code_locations?: QueueCodeLocation[];
sast_source_location?: SastSourceLocation;
}
interface JoinedLocations {
readonly codeLocations?: QueueCodeLocation[];
readonly sastSourceLocation?: SastSourceLocation;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function isSastSourceLocation(value: unknown): value is SastSourceLocation {
if (!isRecord(value)) return false;
return (
typeof value.file === 'string' &&
value.file.length > 0 &&
Number.isInteger(value.line) &&
(value.line as number) > 0 &&
Number.isInteger(value.column) &&
(value.column as number) >= 0 &&
typeof value.rule_id === 'string' &&
value.rule_id.length > 0
);
}
function parseJson(contents: string, description: string): unknown {
try {
return JSON.parse(contents) as unknown;
} catch {
throw new Error(`${description} is not valid JSON`);
}
}
async function currentReferenceMap(
deliverablesPath: string,
vulnerabilityClass: ReconciliationClass,
): Promise<ReadonlyMap<string, string>> {
const mapRead = await readCommittedFile(deliverablesPath, renumberMapPath(vulnerabilityClass));
if (mapRead.state === 'absent') return new Map();
if (mapRead.state !== 'present') throw new Error(`${vulnerabilityClass} report-reference map is unreadable`);
const decoded = parseJson(mapRead.contents, `${vulnerabilityClass} report-reference map`);
if (!isRecord(decoded) || !isRecord(decoded.map)) {
throw new Error(`${vulnerabilityClass} report-reference map is malformed`);
}
const references = new Map<string, string>();
for (const [stable, current] of Object.entries(decoded.map)) {
if (typeof current !== 'string') throw new Error(`${vulnerabilityClass} report-reference map is malformed`);
references.set(stable, current);
}
return references;
}
/** Read every per-class queue in the deliverables dir into an ID-to-locations map. */
async function loadQueueLocations(
deliverablesPath: string,
logger: ActivityLogger,
): Promise<Map<string, QueueCodeLocation[]>> {
const locations = new Map<string, QueueCodeLocation[]>();
for (const vulnClass of ALL_VULN_CLASSES) {
const queuePath = path.join(deliverablesPath, `${vulnClass}_exploitation_queue.json`);
if (!(await fs.pathExists(queuePath))) continue;
try {
const doc = (await fs.readJson(queuePath)) as { vulnerabilities?: QueueEntry[] };
for (const entry of doc.vulnerabilities ?? []) {
if (entry.ID && entry.code_locations && entry.code_locations.length > 0) {
locations.set(entry.ID, entry.code_locations);
}
}
} catch (error) {
logger.warn(`Could not read ${vulnClass} queue for code locations: ${(error as Error).message}`);
participatingClasses: readonly ReconciliationClass[],
): Promise<Map<string, JoinedLocations>> {
const locations = new Map<string, JoinedLocations>();
for (const vulnerabilityClass of participatingClasses) {
const queueRead = await readCommittedFile(deliverablesPath, `${vulnerabilityClass}_exploitation_queue.json`);
if (queueRead.state === 'absent') continue;
if (queueRead.state !== 'present') throw new Error(`${vulnerabilityClass} queue is unreadable`);
const decoded = parseJson(queueRead.contents, `${vulnerabilityClass} queue`);
if (!isRecord(decoded) || !Array.isArray(decoded.vulnerabilities)) {
throw new Error(`${vulnerabilityClass} queue is malformed`);
}
const referenceMap = await currentReferenceMap(deliverablesPath, vulnerabilityClass);
for (const rawEntry of decoded.vulnerabilities) {
if (!isRecord(rawEntry) || typeof rawEntry.ID !== 'string') continue;
const stableReference = rawEntry.ID;
const entry = rawEntry as QueueEntry;
const bundle: JoinedLocations = {
...(Array.isArray(entry.code_locations) && entry.code_locations.length > 0
? { codeLocations: entry.code_locations }
: {}),
...(isSastSourceLocation(entry.sast_source_location) ? { sastSourceLocation: entry.sast_source_location } : {}),
};
if (bundle.codeLocations === undefined && bundle.sastSourceLocation === undefined) continue;
locations.set(stableReference, bundle);
const currentReference = referenceMap.get(stableReference);
if (currentReference !== undefined) locations.set(currentReference, bundle);
}
}
return locations;
}
/**
* Return the findings with `code_locations` filled in from the queue.
*
* A finding with no matching queue entry keeps none — the join never invents one. Findings are
* copied rather than mutated so the collector's own state stays untouched.
*/
/** Attach the two independent location fields without consulting exploit-inspection locations. */
export async function attachQueueCodeLocations(
findings: readonly AddFindingInput[],
deliverablesPath: string,
logger: ActivityLogger,
participatingClasses: readonly ReconciliationClass[] = ALL_VULN_CLASSES,
): Promise<AddFindingInput[]> {
const byId = await loadQueueLocations(deliverablesPath, logger);
if (byId.size === 0) return [...findings];
let matched = 0;
const byReference = await loadQueueLocations(deliverablesPath, participatingClasses);
let analysisMatches = 0;
let sastMatches = 0;
const joined = findings.map((finding) => {
const locations = byId.get(finding.finding_id);
if (!locations) return finding;
matched += 1;
return { ...finding, code_locations: locations };
const locations = byReference.get(finding.finding_id);
if (locations === undefined) return finding;
if (locations.codeLocations !== undefined) analysisMatches++;
if (locations.sastSourceLocation !== undefined) sastMatches++;
return {
...finding,
...(locations.codeLocations !== undefined && { code_locations: locations.codeLocations }),
...(locations.sastSourceLocation !== undefined && { sast_source_location: locations.sastSourceLocation }),
};
});
logger.info('Attached committed report-task locations', {
findings: findings.length,
analysisMatches,
sastMatches,
});
logger.info(`Attached code locations to ${matched}/${findings.length} finding(s) from the vuln queues`);
return joined;
}
apps/worker/src/services/compaction-core.ts
+366
View File
@@ -0,0 +1,366 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Deterministic post-report compaction over one coherent mixed reference set. */
import { REF_PREFIX } from '../ai/reconciliation/refs.js';
import type { AddExploitInput } from '../collectors/exploit-collector.js';
import type { AddFindingInput } from '../collectors/finding-collector.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import type { ExactOutputCommit, ExactOutputFile } from './exact-output-commit.js';
import { RenumberError, writeAndCommitExactFiles } from './exact-output-commit.js';
import { readCommittedFile } from './git-manager.js';
import type { ExcludedEntry, SastProvenanceFile } from './renumber-core.js';
import {
pad2,
parseRefNumber,
remapSastProvenance,
remapTaskReferences,
renumberedExploitCollectorPath,
renumberedSastProvenancePath,
renumberMapPath,
} from './renumber-core.js';
import type { ReportData } from './report-renderer.js';
const PREFIXES_LONGEST_FIRST = (Object.entries(REF_PREFIX) as [ReconciliationClass, string][])
.map(([vulnerabilityClass, prefix]) => ({ vulnerabilityClass, prefix }))
.sort((first, second) => second.prefix.length - first.prefix.length);
export function vulnerabilityClassOfReference(reference: string): ReconciliationClass | null {
for (const { vulnerabilityClass, prefix } of PREFIXES_LONGEST_FIRST) {
if (reference.startsWith(`${prefix}-`) && parseRefNumber(reference, vulnerabilityClass) !== null) {
return vulnerabilityClass;
}
}
return null;
}
export interface RenumberMapFile {
readonly vulnerability_type: ReconciliationClass;
readonly map: Record<string, string>;
readonly order: readonly string[];
readonly excluded: readonly ExcludedEntry[];
}
export interface ClassCompaction {
readonly vulnerabilityClass: ReconciliationClass;
readonly gapMap: ReadonlyMap<string, string>;
readonly composedMap: ReadonlyMap<string, string>;
readonly excluded: readonly ExcludedEntry[];
readonly renumberMapFile: RenumberMapFile;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function isRenumberMapFile(value: unknown, vulnerabilityClass: ReconciliationClass): value is RenumberMapFile {
if (!isRecord(value) || value.vulnerability_type !== vulnerabilityClass) return false;
if (!isRecord(value.map) || !Array.isArray(value.order) || !Array.isArray(value.excluded)) return false;
const map = value.map;
const entries = Object.entries(map);
if (
!entries.every(
([stable, dense]) =>
parseRefNumber(stable, vulnerabilityClass) !== null &&
typeof dense === 'string' &&
parseRefNumber(dense, vulnerabilityClass) !== null,
)
) {
return false;
}
if (new Set(entries.map(([, dense]) => dense)).size !== entries.length) return false;
if (
value.order.length !== entries.length ||
new Set(value.order).size !== value.order.length ||
!value.order.every((stable) => typeof stable === 'string' && Object.hasOwn(map, stable))
) {
return false;
}
return value.excluded.every((entry) => {
if (!isRecord(entry)) return false;
return (
typeof entry.source_ref === 'string' &&
parseRefNumber(entry.source_ref, vulnerabilityClass) !== null &&
entry.reason === 'validation_blocked'
);
});
}
export function buildClassCompaction(
vulnerabilityClass: ReconciliationClass,
keptReferences: readonly string[],
renumberMap: RenumberMapFile,
): ClassCompaction {
if (!isRenumberMapFile(renumberMap, vulnerabilityClass)) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-map-malformed' });
}
const mintedReferences = new Set(Object.values(renumberMap.map));
const uniqueKept = [...new Set(keptReferences)];
for (const reference of uniqueKept) {
if (!mintedReferences.has(reference)) {
throw new RenumberError('unmappable-survivor', false, {
checkCode: 'compaction-kept-reference-not-minted',
vulnerabilityClass,
});
}
}
uniqueKept.sort(
(first, second) =>
(parseRefNumber(first, vulnerabilityClass) as number) - (parseRefNumber(second, vulnerabilityClass) as number),
);
const gapMap = new Map<string, string>();
for (const [index, reference] of uniqueKept.entries()) {
gapMap.set(reference, `${REF_PREFIX[vulnerabilityClass]}-${pad2(index + 1)}`);
}
const stableByDense = new Map<string, string>();
for (const [stable, dense] of Object.entries(renumberMap.map)) stableByDense.set(dense, stable);
const composedMap = new Map<string, string>();
const order: string[] = [];
for (const dense of uniqueKept) {
const stable = stableByDense.get(dense);
if (stable === undefined) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-reverse-map-missing' });
}
composedMap.set(stable, gapMap.get(dense) as string);
order.push(stable);
}
return {
vulnerabilityClass,
gapMap,
composedMap,
excluded: renumberMap.excluded,
renumberMapFile: {
vulnerability_type: vulnerabilityClass,
map: Object.fromEntries(composedMap),
order,
excluded: renumberMap.excluded,
},
};
}
export function deepRemapStrings(value: unknown, gapMap: ReadonlyMap<string, string>): unknown {
if (gapMap.size === 0) return value;
if (typeof value === 'string') return remapTaskReferences(value, gapMap);
if (Array.isArray(value)) return value.map((entry) => deepRemapStrings(entry, gapMap));
if (value !== null && typeof value === 'object') {
const remapped: Record<string, unknown> = {};
for (const [key, entry] of Object.entries(value)) remapped[key] = deepRemapStrings(entry, gapMap);
return remapped;
}
return value;
}
export function remapExploitCollector(
entries: readonly AddExploitInput[],
classGapMap: ReadonlyMap<string, string>,
allGapMap: ReadonlyMap<string, string>,
): AddExploitInput[] {
const remapped: AddExploitInput[] = [];
for (const entry of entries) {
const reference = (entry as unknown as { vulnerability_id?: unknown }).vulnerability_id;
if (typeof reference !== 'string') {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-collector-reference-missing' });
}
const gapless = classGapMap.get(reference);
if (gapless === undefined) continue;
const rewritten = deepRemapStrings(entry, allGapMap) as Record<string, unknown>;
remapped.push({ ...rewritten, vulnerability_id: gapless } as unknown as AddExploitInput);
}
return remapped;
}
/** Findings from excluded failed classes are returned by identity, including their cross-references. */
export function remapReportFindings(
findings: readonly AddFindingInput[],
allGapMap: ReadonlyMap<string, string>,
excludedClasses: ReadonlySet<ReconciliationClass> = new Set(),
): AddFindingInput[] {
return findings.map((finding) => {
const vulnerabilityClass = vulnerabilityClassOfReference(finding.finding_id);
if (vulnerabilityClass !== null && excludedClasses.has(vulnerabilityClass)) return finding;
return deepRemapStrings(finding, allGapMap) as AddFindingInput;
});
}
export function plannedReportReferenceOperations(exploit: boolean): readonly ('renumber' | 'compact')[] {
return exploit ? ['renumber', 'compact'] : [];
}
function arraysEqual<T>(first: readonly T[], second: readonly T[]): boolean {
return first.length === second.length && first.every((entry, index) => entry === second[index]);
}
function parseJson<T>(contents: string, checkCode: string): T {
try {
return JSON.parse(contents) as T;
} catch {
throw new RenumberError('key-set-divergence', false, { checkCode });
}
}
async function readRequiredCommittedJson<T>(dir: string, relPath: string, checkCode: string): Promise<T> {
const read = await readCommittedFile(dir, relPath);
if (read.state !== 'present') throw new RenumberError('key-set-divergence', false, { checkCode });
return parseJson<T>(read.contents, checkCode);
}
function parseSastProvenance(value: unknown): SastProvenanceFile {
if (!isRecord(value) || !Array.isArray(value.entries)) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-provenance-malformed' });
}
for (const entry of value.entries) {
if (!isRecord(entry) || typeof entry.exploit_ref !== 'string') {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-provenance-entry-malformed' });
}
}
return value as unknown as SastProvenanceFile;
}
export interface CompactionResult {
readonly compactedClasses: number;
readonly skipped: boolean;
readonly commit?: ExactOutputCommit;
}
/**
* Compact every eligible participating class as one exact-path transaction.
* Failed classes are skipped before any of their collector, map, or provenance paths are read.
*/
export async function compactReportFindings(args: {
readonly deliverablesDir: string;
readonly participatingClasses: readonly ReconciliationClass[];
readonly renumberFailedClasses: readonly ReconciliationClass[];
readonly logger: ActivityLogger;
}): Promise<CompactionResult> {
const reportRead = await readCommittedFile(args.deliverablesDir, 'report.json');
if (reportRead.state === 'absent') return { compactedClasses: 0, skipped: true };
if (reportRead.state !== 'present') {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-report-unreadable' });
}
const report = parseJson<ReportData>(reportRead.contents, 'compaction-report-not-json');
if (!isRecord(report) || !Array.isArray(report.findings)) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-report-malformed' });
}
// The failed-class set is read from two independent sources (the committed report and the
// caller's own record of what renumbering skipped); they must agree before any path is read,
// since a mismatch means compaction and the report disagree about which classes are trustworthy.
const reportFailedClasses = report.reconciliation_failed ?? [];
if (!arraysEqual(reportFailedClasses, args.renumberFailedClasses)) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-failed-class-set-mismatch' });
}
const participating = new Set(args.participatingClasses);
const failed = new Set(args.renumberFailedClasses);
if (participating.size !== args.participatingClasses.length || failed.size !== args.renumberFailedClasses.length) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-class-set-duplicate' });
}
if ([...failed].some((vulnerabilityClass) => !participating.has(vulnerabilityClass))) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-failed-class-outside-scope' });
}
const keptByClass = new Map<ReconciliationClass, string[]>();
const seenFindingReferences = new Set<string>();
for (const finding of report.findings) {
if (typeof finding.finding_id !== 'string' || seenFindingReferences.has(finding.finding_id)) {
throw new RenumberError('unmappable-survivor', false, { checkCode: 'compaction-report-reference-duplicate' });
}
seenFindingReferences.add(finding.finding_id);
const vulnerabilityClass = vulnerabilityClassOfReference(finding.finding_id);
if (vulnerabilityClass === null || !participating.has(vulnerabilityClass)) continue;
const references = keptByClass.get(vulnerabilityClass) ?? [];
references.push(finding.finding_id);
keptByClass.set(vulnerabilityClass, references);
}
const compactions: Array<{
compaction: ClassCompaction;
collector: AddExploitInput[];
provenance?: SastProvenanceFile;
}> = [];
for (const vulnerabilityClass of args.participatingClasses) {
if (failed.has(vulnerabilityClass)) continue;
const keptReferences = keptByClass.get(vulnerabilityClass) ?? [];
const mapRead = await readCommittedFile(args.deliverablesDir, renumberMapPath(vulnerabilityClass));
if (mapRead.state === 'absent') {
// A class with no renumber map means renumbering never ran for it, so there is no
// stable-to-dense mapping to compact against. If the report still kept references from
// that class, the two artifacts have drifted and compaction must fail rather than guess.
if (keptReferences.length > 0) {
throw new RenumberError('unmappable-survivor', false, { checkCode: 'compaction-map-absent-for-survivor' });
}
continue;
}
if (mapRead.state !== 'present') {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-map-unreadable' });
}
const renumberMap = parseJson<RenumberMapFile>(mapRead.contents, 'compaction-map-not-json');
const compaction = buildClassCompaction(vulnerabilityClass, keptReferences, renumberMap);
const collector = await readRequiredCommittedJson<AddExploitInput[]>(
args.deliverablesDir,
renumberedExploitCollectorPath(vulnerabilityClass),
'compaction-collector-unreadable',
);
if (!Array.isArray(collector)) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-collector-malformed' });
}
const provenanceRead = await readCommittedFile(
args.deliverablesDir,
renumberedSastProvenancePath(vulnerabilityClass),
);
let provenance: SastProvenanceFile | undefined;
if (provenanceRead.state === 'corrupt') {
throw new RenumberError('key-set-divergence', false, { checkCode: 'compaction-provenance-unreadable' });
}
if (provenanceRead.state === 'present') {
provenance = parseSastProvenance(parseJson<unknown>(provenanceRead.contents, 'compaction-provenance-not-json'));
}
compactions.push({ compaction, collector, ...(provenance !== undefined && { provenance }) });
}
if (compactions.length === 0) return { compactedClasses: 0, skipped: true };
const allGapMap = new Map<string, string>();
for (const { compaction } of compactions) {
for (const [source, destination] of compaction.gapMap) allGapMap.set(source, destination);
}
const files: ExactOutputFile[] = [];
for (const { compaction, collector, provenance } of compactions) {
files.push({
relPath: renumberedExploitCollectorPath(compaction.vulnerabilityClass),
contents: `${JSON.stringify(remapExploitCollector(collector, compaction.gapMap, allGapMap), null, 2)}\n`,
});
files.push({
relPath: renumberMapPath(compaction.vulnerabilityClass),
contents: `${JSON.stringify(compaction.renumberMapFile, null, 2)}\n`,
});
if (provenance !== undefined) {
files.push({
relPath: renumberedSastProvenancePath(compaction.vulnerabilityClass),
contents: `${JSON.stringify(remapSastProvenance(provenance, compaction.gapMap), null, 2)}\n`,
});
}
}
const compactedReport = deepRemapStrings(report, allGapMap) as ReportData;
const findings = remapReportFindings(report.findings, allGapMap, failed);
files.push({
relPath: 'report.json',
contents: JSON.stringify({ ...compactedReport, findings }, null, 2),
});
const commit = await writeAndCommitExactFiles(
args.deliverablesDir,
files,
'Compact surviving report references to dense gapless',
args.logger,
);
return { compactedClasses: compactions.length, skipped: false, commit };
}
apps/worker/src/services/exact-output-commit.ts
+301
View File
@@ -0,0 +1,301 @@
// Copyright (C) 2025 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Generic exact-path, lost-acknowledgement-safe file publication over a deliverables Git repo. */
import { randomUUID } from 'node:crypto';
import { lstat, mkdtemp, rename, rm, unlink, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import path from 'node:path';
import type { ActivityLogger } from '../types/activity-logger.js';
import {
commitExactPaths,
executeGitCommandWithRetry,
getGitCommitHash,
pathsChangedInCommit,
readCommittedFile,
restorePathsFromHead,
withGitRepoLock,
} from './git-manager.js';
export type RenumberErrorType = 'unmappable-survivor' | 'key-set-divergence';
const RENUMBER_ERROR_MESSAGES: Readonly<Record<RenumberErrorType, string>> = Object.freeze({
'unmappable-survivor': 'A report survivor could not be mapped to one canonical class reference.',
'key-set-divergence': 'Committed report-facing class artifacts disagree on their reference set.',
});
export interface RenumberErrorDetails {
readonly checkCode: string;
readonly [key: string]: unknown;
}
export class RenumberError extends Error {
readonly retryable: boolean;
readonly type: RenumberErrorType;
readonly details?: RenumberErrorDetails;
constructor(type: RenumberErrorType, retryable: boolean, details?: RenumberErrorDetails) {
super(RENUMBER_ERROR_MESSAGES[type]);
this.name = 'RenumberError';
this.type = type;
this.retryable = retryable;
if (details !== undefined) this.details = details;
}
}
export interface ExactOutputFile {
readonly relPath: string;
/** `null` makes absence part of the exact output contract. */
readonly contents: string | null;
}
export interface ExactOutputCommit {
readonly commitHash: string;
readonly changedPaths: readonly string[];
readonly alreadyCommitted: boolean;
}
function samePathSet(first: readonly string[], second: readonly string[]): boolean {
return (
first.length === second.length &&
new Set(first).size === first.length &&
first.every((entry) => second.includes(entry))
);
}
function isErrno(error: unknown, code: string): boolean {
return error instanceof Error && (error as NodeJS.ErrnoException).code === code;
}
async function rejectSymlinks(dir: string, relPaths: readonly string[]): Promise<void> {
for (const relPath of relPaths) {
try {
if ((await lstat(path.join(dir, relPath))).isSymbolicLink()) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'output-symlink' });
}
} catch (error) {
if (isErrno(error, 'ENOENT')) continue;
throw error;
}
}
}
async function atomicWriteUnique(absolutePath: string, contents: string): Promise<void> {
const temporaryPath = `${absolutePath}.tmp-${randomUUID()}`;
try {
await writeFile(temporaryPath, contents, { flag: 'wx' });
await rename(temporaryPath, absolutePath);
} catch (error) {
await unlink(temporaryPath).catch(() => undefined);
throw error;
}
}
async function executeExactGitCommand(
args: string[],
dir: string,
description: string,
): Promise<{ stdout: string; stderr: string }> {
try {
return await executeGitCommandWithRetry(args, dir, description);
} catch (error) {
throw new Error(`Exact-output Git step failed: ${description}`, { cause: error });
}
}
async function repairExactPathsFromHead(dir: string, relPaths: readonly string[]): Promise<void> {
const presentPaths: string[] = [];
const absentPaths: string[] = [];
for (const relPath of relPaths) {
const committed = await readCommittedFile(dir, relPath);
if (committed.state === 'corrupt') {
throw new RenumberError('key-set-divergence', false, { checkCode: 'corrupt-output-object' });
}
if (committed.state === 'present') presentPaths.push(relPath);
else absentPaths.push(relPath);
}
if (presentPaths.length > 0) await restorePathsFromHead(dir, presentPaths);
if (absentPaths.length === 0) return;
for (const relPath of absentPaths) {
const listed = await executeExactGitCommand(
['git', 'ls-files', '--', relPath],
dir,
'checking an absent exact-output index path',
);
if (listed.stdout.trim() !== '') {
await executeExactGitCommand(
['git', 'update-index', '--force-remove', '--', relPath],
dir,
'clearing an absent exact-output path from the index',
);
}
}
for (const relPath of absentPaths) {
await unlink(path.join(dir, relPath)).catch((error: unknown) => {
if (!isErrno(error, 'ENOENT')) throw error;
});
}
}
async function commitExactFilesWithTemporaryIndex(
dir: string,
files: readonly ExactOutputFile[],
expectedChangedPaths: readonly string[],
message: string,
logger: ActivityLogger,
): Promise<{ commitHash: string; changedPaths: string[] }> {
const head = await getGitCommitHash(dir);
if (head === null) throw new Error('Unable to read HEAD for exact-output commit');
const temporaryDir = await mkdtemp(path.join(tmpdir(), 'shannon-exact-index-'));
const temporaryIndex = path.join(temporaryDir, 'index');
const pathsToStage = files
.filter((file) => file.contents !== null || expectedChangedPaths.includes(file.relPath))
.map((file) => file.relPath);
const runWithTemporaryIndex = async (gitArgs: readonly string[], description: string) =>
executeExactGitCommand(['env', `GIT_INDEX_FILE=${temporaryIndex}`, 'git', ...gitArgs], dir, description);
try {
await runWithTemporaryIndex(['read-tree', head], 'initializing an exact-output temporary index');
await runWithTemporaryIndex(['add', '-A', '--', ...pathsToStage], 'staging exact outputs in a temporary index');
const tree = (await runWithTemporaryIndex(['write-tree'], 'writing the exact-output tree')).stdout.trim();
const commitHash = (
await executeExactGitCommand(
['git', 'commit-tree', tree, '-p', head, '-m', message],
dir,
'creating the exact-output commit object',
)
).stdout.trim();
const changedPaths = await pathsChangedInCommit(dir, commitHash);
if (!samePathSet(changedPaths, expectedChangedPaths)) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'changed-path-set-mismatch' });
}
await executeExactGitCommand(
['git', 'update-ref', 'HEAD', commitHash, head],
dir,
'advancing HEAD to the exact-output commit',
);
for (const file of files) {
if (file.contents === null) {
const listed = await executeExactGitCommand(
['git', 'ls-files', '--', file.relPath],
dir,
'checking an exact-output deletion in the index',
);
if (listed.stdout.trim() !== '') {
await executeExactGitCommand(
['git', 'update-index', '--force-remove', '--', file.relPath],
dir,
'recording an exact-output deletion in the index',
);
}
} else {
await executeExactGitCommand(
['git', 'add', '--', file.relPath],
dir,
'refreshing an exact-output path in the index',
);
}
}
logger.info(`Path-limited commit ${commitHash.slice(0, 8)} changed ${changedPaths.length} path(s)`);
return { commitHash, changedPaths };
} finally {
await rm(temporaryDir, { recursive: true, force: true });
}
}
/** Exact-path, lost-acknowledgement-safe publication used by both transforms. */
export async function writeAndCommitExactFiles(
dir: string,
files: readonly ExactOutputFile[],
message: string,
logger: ActivityLogger,
options: {
readonly afterCommit?: (commit: { commitHash: string; changedPaths: readonly string[] }) => void | Promise<void>;
} = {},
): Promise<ExactOutputCommit> {
if (files.length === 0) throw new Error('writeAndCommitExactFiles requires at least one file');
const relPaths = files.map((file) => file.relPath);
if (
relPaths.some(
(relPath) =>
relPath.length === 0 ||
path.isAbsolute(relPath) ||
relPath.includes('\0') ||
relPath.split(/[\\/]/).some((segment) => segment === '' || segment === '.' || segment === '..'),
)
) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'unsafe-output-path' });
}
if (new Set(relPaths).size !== relPaths.length) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'duplicate-output-path' });
}
return withGitRepoLock(async () => {
await rejectSymlinks(dir, relPaths);
const expectedChangedPaths: string[] = [];
let allCommitted = true;
for (const file of files) {
const committed = await readCommittedFile(dir, file.relPath);
if (committed.state === 'corrupt') {
throw new RenumberError('key-set-divergence', false, { checkCode: 'corrupt-output-object' });
}
const matches =
file.contents === null
? committed.state === 'absent'
: committed.state === 'present' && committed.contents === file.contents;
if (!matches) {
allCommitted = false;
expectedChangedPaths.push(file.relPath);
}
}
if (allCommitted) {
await repairExactPathsFromHead(dir, relPaths);
const commitHash = await getGitCommitHash(dir);
if (commitHash === null) throw new Error('Unable to read the existing exact-output commit');
return { commitHash, changedPaths: [], alreadyCommitted: true };
}
try {
for (const file of files) {
const absolutePath = path.join(dir, file.relPath);
if (file.contents === null) {
await unlink(absolutePath).catch((error: unknown) => {
if (!isErrno(error, 'ENOENT')) throw error;
});
} else {
await atomicWriteUnique(absolutePath, file.contents);
}
}
const committed = files.some((file) => file.contents === null)
? await commitExactFilesWithTemporaryIndex(dir, files, expectedChangedPaths, message, logger)
: await commitExactPaths(dir, relPaths, message, logger);
if (!samePathSet(committed.changedPaths, expectedChangedPaths)) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'changed-path-set-mismatch' });
}
await options.afterCommit?.(committed);
for (const file of files) {
const verified = await readCommittedFile(dir, file.relPath);
const matches =
file.contents === null
? verified.state === 'absent'
: verified.state === 'present' && verified.contents === file.contents;
if (!matches) {
throw new RenumberError('key-set-divergence', false, { checkCode: 'committed-byte-mismatch' });
}
}
return { ...committed, alreadyCommitted: false };
} catch (error) {
await repairExactPathsFromHead(dir, relPaths).catch((cleanupError: unknown) => {
logger.error('Exact-output rollback failed', {
error: cleanupError instanceof Error ? cleanupError.message : String(cleanupError),
});
});
throw error;
}
});
}
apps/worker/src/services/exploit-renderer.ts
+1 -1
View File
@@ -8,7 +8,7 @@
* Deterministic exploit collector → markdown renderer.
*
* Single entry point renderExploitDeliverable(vulnClass, state, idToType)
* covers all exploitation agents. The
* covers every exploitation agent, including the conditional `miscellaneous` class. The
* per-class deltas are limited to title and ID prefix; every section, label,
* and sort rule is class-agnostic. Section headers and bolded field labels
* mirror the prescribed-Markdown skeleton from the existing exploit-*.txt
apps/worker/src/services/finding-order.ts
+90
View File
@@ -0,0 +1,90 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Shared ordering for structured findings and every derived report surface. */
const UNRANKED = Number.MAX_SAFE_INTEGER;
export const SEVERITY_RANK: Readonly<Record<string, number>> = Object.freeze({
critical: 0,
high: 1,
medium: 2,
low: 3,
});
export const CONFIDENCE_RANK: Readonly<Record<string, number>> = Object.freeze({
high: 0,
medium: 1,
low: 2,
});
/** Recognized report categories. Unrecognized values form a sentinel group after `Other`. */
export const CATEGORY_ORDER = ['Injection', 'XSS', 'Authentication', 'SSRF', 'Authorization', 'Other'] as const;
export function severityRank(severity: string | null | undefined): number {
if (severity == null) return UNRANKED;
return SEVERITY_RANK[severity] ?? UNRANKED;
}
export function confidenceRank(confidence: string | null | undefined): number {
if (confidence == null) return UNRANKED;
return CONFIDENCE_RANK[confidence] ?? UNRANKED;
}
export function categoryRank(category: string): number {
const index = CATEGORY_ORDER.indexOf(category as (typeof CATEGORY_ORDER)[number]);
return index === -1 ? UNRANKED : index;
}
/** Recognized categories first; unknown sentinel categories follow in lexical order. */
export function compareCategories(first: string, second: string): number {
const firstRank = categoryRank(first);
const secondRank = categoryRank(second);
if (firstRank !== secondRank) return firstRank - secondRank;
if (firstRank !== UNRANKED || first === second) return 0;
return first < second ? -1 : 1;
}
export function trailingRefNumber(reference: string): number | null {
const match = /(\d+)$/.exec(reference);
if (match === null) return null;
const parsed = Number.parseInt(match[1] as string, 10);
return Number.isFinite(parsed) ? parsed : null;
}
/** Numeric suffix first, then the full reference as the deterministic fallback. */
export function compareRef(first: string, second: string): number {
const firstNumber = trailingRefNumber(first);
const secondNumber = trailingRefNumber(second);
if (firstNumber !== secondNumber) {
if (firstNumber === null) return 1;
if (secondNumber === null) return -1;
return firstNumber - secondNumber;
}
if (first < second) return -1;
if (first > second) return 1;
return 0;
}
export interface FindingOrderFields {
readonly category: string;
readonly severity?: string | null;
readonly finding_id: string;
}
/** Category, severity, numeric suffix, then full-reference fallback. */
export function compareFindings(first: FindingOrderFields, second: FindingOrderFields): number {
const categoryDifference = compareCategories(first.category, second.category);
if (categoryDifference !== 0) return categoryDifference;
const severityDifference = severityRank(first.severity) - severityRank(second.severity);
if (severityDifference !== 0) return severityDifference;
return compareRef(first.finding_id, second.finding_id);
}
export function orderFindings<T extends FindingOrderFields>(findings: readonly T[]): T[] {
return [...findings].sort(compareFindings);
}
apps/worker/src/services/findings-renderer.ts
+60 -15
View File
@@ -17,10 +17,18 @@
*/
import { fs, path } from 'zx';
import type { AuthFinding, AuthzFinding, InjectionFinding, SsrfFinding, XssFinding } from '../ai/queue-schemas.js';
import type {
AuthFinding,
AuthzFinding,
InjectionFinding,
MiscellaneousFinding,
SsrfFinding,
XssFinding,
} from '../ai/queue-schemas.js';
import { deliverablesDir } from '../paths.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import type { VulnClass } from '../types/config.js';
import { ALL_VULN_CLASSES } from '../types/config.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
const DISCLAIMER = [
'> Exploitation phase was not run for this assessment. Each entry documents a',
@@ -37,7 +45,11 @@ interface ClassConfig<T> {
}
interface QueueDocument<T> {
vulnerabilities?: T[];
readonly vulnerabilities: readonly T[];
}
export interface RenderFindingsResult {
readonly failedClasses: readonly ReconciliationClass[];
}
// === Common Render Helpers ===
@@ -48,6 +60,17 @@ function summaryRow(label: string, value: string | undefined | null | boolean):
return `- **${label}:** ${value}`;
}
function parseQueueDocument(value: unknown): QueueDocument<unknown> {
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
throw new Error('queue document is malformed');
}
const vulnerabilities = (value as Record<string, unknown>).vulnerabilities;
if (!Array.isArray(vulnerabilities)) {
throw new Error('queue document vulnerabilities are malformed');
}
return { vulnerabilities };
}
function formatLocation(endpoint: string | undefined, codeLocation: string | undefined): string {
if (endpoint && codeLocation) return `${endpoint} (${codeLocation})`;
return endpoint ?? codeLocation ?? '';
@@ -110,6 +133,20 @@ function renderSsrfEntry(e: SsrfFinding): string {
);
}
function renderMiscellaneousEntry(e: MiscellaneousFinding): string {
return buildEntry(
e.ID,
e.vulnerability_type,
{ confidence: e.confidence },
[
summaryRow('Vulnerable location', formatLocation(e.source_endpoint, e.vulnerable_code_location)),
summaryRow('Overview', e.missing_defense),
summaryRow('Impact', e.exploitation_hypothesis),
],
e.notes,
);
}
function renderAuthzEntry(e: AuthzFinding): string {
return buildEntry(
e.ID,
@@ -148,7 +185,7 @@ function renderXssEntry(e: XssFinding): string {
// === Class Registry ===
const CLASSES: Record<VulnClass, ClassConfig<unknown>> = {
const CLASSES: Record<ReconciliationClass, ClassConfig<unknown>> = {
auth: {
heading: 'Authentication',
noneFoundLabel: 'authentication',
@@ -184,6 +221,13 @@ const CLASSES: Record<VulnClass, ClassConfig<unknown>> = {
findingsFile: 'ssrf_findings.md',
renderEntry: (e) => renderSsrfEntry(e as SsrfFinding),
},
miscellaneous: {
heading: 'Miscellaneous',
noneFoundLabel: 'miscellaneous',
queueFile: 'miscellaneous_exploitation_queue.json',
findingsFile: 'miscellaneous_findings.md',
renderEntry: (e) => renderMiscellaneousEntry(e as MiscellaneousFinding),
},
};
// === Class File Assembly ===
@@ -213,39 +257,40 @@ function renderClassFile(config: ClassConfig<unknown>, entries: readonly unknown
/**
* Render `*_findings.md` per class from each `*_exploitation_queue.json`.
*
* Idempotent: skips classes whose findings file already exists, or whose queue
* is missing (class out of scope this run). Per-class failures are logged and
* other classes still proceed.
* Idempotent: rewrites each present class from its queue; a missing queue means the class was out of
* scope. Per-class failures are logged and other classes still proceed.
*/
export async function renderFindingsFromQueues(
sourceDir: string,
deliverablesSubdir: string | undefined,
logger: ActivityLogger,
): Promise<void> {
participatingClasses: readonly ReconciliationClass[] = ALL_VULN_CLASSES,
): Promise<RenderFindingsResult> {
const dir = deliverablesDir(sourceDir, deliverablesSubdir);
const failedClasses: ReconciliationClass[] = [];
for (const config of Object.values(CLASSES)) {
for (const vulnerabilityClass of participatingClasses) {
const config = CLASSES[vulnerabilityClass];
const queuePath = path.join(dir, config.queueFile);
const findingsPath = path.join(dir, config.findingsFile);
if (await fs.pathExists(findingsPath)) {
logger.info(`${config.heading}: ${config.findingsFile} already exists, skipping`);
continue;
}
if (!(await fs.pathExists(queuePath))) {
logger.info(`${config.heading}: no queue file (class out of scope), skipping`);
continue;
}
try {
const doc = (await fs.readJson(queuePath)) as QueueDocument<unknown>;
const entries = doc.vulnerabilities ?? [];
const doc = parseQueueDocument(await fs.readJson(queuePath));
const entries = doc.vulnerabilities;
const markdown = renderClassFile(config, entries);
await fs.writeFile(findingsPath, markdown);
logger.info(`${config.heading}: rendered ${entries.length} finding(s) to ${config.findingsFile}`);
} catch (error) {
const err = error as Error;
failedClasses.push(vulnerabilityClass);
logger.warn(`${config.heading}: failed to render findings from ${config.queueFile}: ${err.message}`);
}
}
return { failedClasses };
}
apps/worker/src/services/index.ts
+16 -1
View File
@@ -19,7 +19,22 @@ export { ConfigLoaderService } from './config-loader.js';
export type { ContainerDependencies } from './container.js';
export { Container, getContainer, getOrCreateContainer, removeContainer, setContainerFactory } from './container.js';
export { ExploitationCheckerService } from './exploitation-checker.js';
export type { CommittedReadResult } from './git-manager.js';
export {
blobShaFromHead,
classifyHeadReadFailure,
commitExactPaths,
getGitCommitHash,
isAncestor,
parsePorcelainZ,
pathsChangedInCommit,
readCommittedFile,
readFileFromHead,
restorePathsFromHead,
rollbackGitWorkspace,
withGitRepoLock,
} from './git-manager.js';
export { loadPrompt } from './prompt-manager.js';
export type { ReportData, ReportMeta } from './report-renderer.js';
export { renderReport } from './report-renderer.js';
export { assembleFinalReport, copyReportToRunRoot, injectModelIntoReport } from './reporting.js';
export { assembleFinalReport, copyReportToRunRoot } from './reporting.js';
apps/worker/src/services/pdf-renderer.ts
+99 -2
View File
@@ -17,8 +17,9 @@
*/
import { execFile } from 'node:child_process';
import { createHash, randomUUID } from 'node:crypto';
import { existsSync } from 'node:fs';
import { copyFile, cp, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises';
import { copyFile, cp, mkdir, mkdtemp, readFile, rename, rm, unlink, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import path from 'node:path';
import { promisify } from 'node:util';
@@ -34,6 +35,15 @@ const DATA_FILENAME = 'data.json';
const TEMPLATE_FILENAME = 'report.typ';
const OUTPUT_FILENAME = 'report.pdf';
export const PDF_RENDERER_VERSION = '1';
export interface PdfProvenance {
readonly pdf_sha256: string;
readonly canonical_report_sha256: string;
readonly renderer_version: string;
readonly template_version: string;
}
export interface RenderReportPdfOptions {
/** Structured report data (report.json contents), pre-assembly. */
readonly reportData: ReportData;
@@ -47,6 +57,86 @@ export interface RenderReportPdfOptions {
readonly brand?: string;
}
function sha256(contents: Uint8Array): string {
return createHash('sha256').update(contents).digest('hex');
}
function isSha256(value: unknown): value is string {
return typeof value === 'string' && /^[a-f0-9]{64}$/.test(value);
}
/** Validate the closed durable provenance shape used for PDF reuse. */
export function isPdfProvenance(value: unknown): value is PdfProvenance {
if (value === null || typeof value !== 'object' || Array.isArray(value)) return false;
const candidate = value as Record<string, unknown>;
return (
Object.keys(candidate).length === 4 &&
isSha256(candidate.pdf_sha256) &&
isSha256(candidate.canonical_report_sha256) &&
typeof candidate.renderer_version === 'string' &&
candidate.renderer_version.length > 0 &&
typeof candidate.template_version === 'string' &&
candidate.template_version.length > 0
);
}
/** Build provenance only after the renderer has atomically published verified PDF bytes. */
export async function readPdfProvenance(args: {
readonly pdfPath: string;
readonly canonicalReportSha256: string;
readonly templatePath: string;
}): Promise<PdfProvenance> {
const [pdfBytes, templateBytes] = await Promise.all([readFile(args.pdfPath), readFile(args.templatePath)]);
return {
pdf_sha256: sha256(pdfBytes),
canonical_report_sha256: args.canonicalReportSha256,
renderer_version: PDF_RENDERER_VERSION,
template_version: sha256(templateBytes),
};
}
/** Recompute the PDF digest before trusting persisted provenance for the current report. */
export async function pdfMatchesProvenance(args: {
readonly pdfPath: string;
readonly canonicalReportSha256: string;
readonly provenance: PdfProvenance;
}): Promise<boolean> {
if (!isPdfProvenance(args.provenance) || args.provenance.canonical_report_sha256 !== args.canonicalReportSha256) {
return false;
}
try {
return sha256(await readFile(args.pdfPath)) === args.provenance.pdf_sha256;
} catch {
return false;
}
}
/**
* Full currency check: the PDF is current only when its bytes, canonical report digest,
* renderer version, and template version all match the provenance record. A record from an
* older renderer or template is stale even when it is internally consistent.
*/
export async function pdfProvenanceIsCurrent(args: {
readonly pdfPath: string;
readonly canonicalReportSha256: string;
readonly provenance: PdfProvenance;
readonly templatePath: string;
}): Promise<boolean> {
if (args.provenance.renderer_version !== PDF_RENDERER_VERSION) return false;
let templateSha256: string;
try {
templateSha256 = sha256(await readFile(args.templatePath));
} catch {
return false;
}
if (args.provenance.template_version !== templateSha256) return false;
return pdfMatchesProvenance({
pdfPath: args.pdfPath,
canonicalReportSha256: args.canonicalReportSha256,
provenance: args.provenance,
});
}
/**
* Compile the report to a PDF at `outputPath`.
*
@@ -91,7 +181,14 @@ export async function renderReportPdf(options: RenderReportPdfOptions): Promise<
]);
await mkdir(path.dirname(outputPath), { recursive: true });
await copyFile(pdfInWorkDir, outputPath);
const outputAttemptPath = `${outputPath}.tmp-${randomUUID()}`;
try {
await copyFile(pdfInWorkDir, outputAttemptPath);
await rename(outputAttemptPath, outputPath);
} catch (error) {
await unlink(outputAttemptPath).catch(() => undefined);
throw error;
}
} finally {
await rm(workDir, { recursive: true, force: true });
}
apps/worker/src/services/prompt-manager.ts
+23 -7
View File
@@ -9,6 +9,7 @@ import { PROMPTS_DIR } from '../paths.js';
import { PLAYWRIGHT_SESSION_MAPPING } from '../session-manager.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import type { Authentication, DistributedConfig, DistributedReportConfig, Rule, VulnClass } from '../types/config.js';
import { assertFixedAnalysisScope } from '../types/run-state.js';
import { isGlobPattern } from '../utils/glob.js';
import { handlePromptError, PentestError } from './error-handling.js';
@@ -140,6 +141,8 @@ interface PromptVariables {
repoPath: string;
/** Classes whose analysis did not complete, so the report can mark them not assessed. */
failedClasses?: readonly VulnClass[];
/** Explicit workflow-owned analysis scope for prompts that describe tested classes. */
analysisClasses?: readonly VulnClass[];
AUTH_STATE_FILE: string;
PLAYWRIGHT_SESSION?: string;
}
@@ -380,12 +383,15 @@ async function interpolateVariables(
result = result.replace(/{{LOGIN_INSTRUCTIONS}}/g, '');
}
const vulnClasses = config?.vuln_classes ?? [];
result = replaceLiteral(
result,
/{{VULN_CLASSES_TESTED}}/g,
vulnClasses.length > 0 ? vulnClasses.join(', ') : 'injection, xss, auth, authz, ssrf',
);
if (result.includes('{{VULN_CLASSES_TESTED}}')) {
if (variables.analysisClasses === undefined) {
throw new PentestError('Prompt requires an explicit workflow-owned analysis scope', 'prompt', false, {
placeholder: 'VULN_CLASSES_TESTED',
});
}
assertFixedAnalysisScope(variables.analysisClasses);
result = replaceLiteral(result, /{{VULN_CLASSES_TESTED}}/g, variables.analysisClasses.join(', '));
}
result = replaceLiteral(
result,
/{{NOT_ASSESSED_CLASSES}}/g,
@@ -443,6 +449,14 @@ async function interpolateVariables(
}
}
// Prompt families that drive deterministic, model-only stages with no browser of their own.
// They share loadPrompt with the browser agents but must never claim a Playwright session.
const NON_BROWSER_PROMPT_PREFIXES: readonly string[] = Object.freeze(['task-formation-', 'sast-enrichment-']);
function isNonBrowserPrompt(promptName: string): boolean {
return NON_BROWSER_PROMPT_PREFIXES.some((prefix) => promptName.startsWith(prefix));
}
// Resolve promptDir override against SHANNON_WORKER_ROOT so relative paths
// from callers stay cwd-independent.
function resolvePromptDir(promptDir: string | undefined): string {
@@ -480,7 +494,9 @@ export async function loadPrompt(
if (session) {
enhancedVariables.PLAYWRIGHT_SESSION = session;
logger.info(`Assigned ${promptName} -> ${enhancedVariables.PLAYWRIGHT_SESSION}`);
} else {
} else if (!isNonBrowserPrompt(promptName)) {
// A browser agent missing from the table is a real gap; a non-browser family is not, so it
// takes neither the fallback session nor the warning.
enhancedVariables.PLAYWRIGHT_SESSION = 'agent1';
logger.warn(`Unknown agent ${promptName}, using fallback -> ${enhancedVariables.PLAYWRIGHT_SESSION}`);
}
apps/worker/src/services/queue-validation.ts
+18 -15
View File
@@ -10,6 +10,7 @@ import type { ExploitationDecision } from '../types/agents.js';
import { ErrorCode } from '../types/errors.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import { err, ok, type Result } from '../types/result.js';
import { renderSafeMessage } from '../types/run-state.js';
import { asyncPipe } from '../utils/functional.js';
import { PentestError } from './error-handling.js';
@@ -39,6 +40,7 @@ interface FileExistence {
interface ExistenceContext {
existence: FileExistence;
deliverableRequired: boolean;
vulnerabilityClass: ReconciliationClass;
}
interface PathsBase {
@@ -145,20 +147,21 @@ const fileExistenceRules: readonly ValidationRule[] = Object.freeze([
),
]);
// Generate appropriate error message based on which files are missing
function getExistenceErrorMessage({ existence, deliverableRequired }: ExistenceContext): string {
const { deliverableExists, queueExists } = existence;
const NO_RESULTS_MESSAGE =
'{Class} analysis did not produce results. Re-running this workspace retries just that class.';
const PARTIAL_RESULTS_MESSAGE =
'{Class} analysis produced only part of its results, so it could not be exploited. Re-running this workspace retries just that class.';
if (!deliverableRequired) {
return 'Analysis failed: Queue file missing. A queue is required.';
}
if (!deliverableExists && !queueExists) {
return 'Analysis failed: Neither deliverable nor queue file exists. Both are required.';
}
if (!queueExists) {
return 'Analysis incomplete: Deliverable exists but queue file missing. Both are required.';
}
return 'Analysis incomplete: Queue exists but deliverable file missing. Both are required.';
/**
* Name the outcome the reader can act on rather than the files behind it: nothing landed at
* all, or only some of what the class owes. The analysis-less `other` class has no
* deliverable, so its queue alone decides which of the two applies.
*/
function getExistenceErrorMessage({ existence, deliverableRequired, vulnerabilityClass }: ExistenceContext): string {
const { deliverableExists, queueExists } = existence;
const nothingProduced = deliverableRequired ? !deliverableExists && !queueExists : !queueExists;
const template = nothingProduced ? NO_RESULTS_MESSAGE : PARTIAL_RESULTS_MESSAGE;
return renderSafeMessage(template, { vulnerabilityClass });
}
// Pure function to create file paths
@@ -214,7 +217,7 @@ const validateExistenceRules = (
const { existence, vulnType } = pathsWithExistence;
const { deliverableRequired } = VULN_TYPE_CONFIG[vulnType];
const context: ExistenceContext = { existence, deliverableRequired };
const context: ExistenceContext = { existence, deliverableRequired, vulnerabilityClass: vulnType };
// Find the first rule that fails
const failedRule = fileExistenceRules.find((rule) => !rule.predicate(context));
@@ -225,7 +228,7 @@ const validateExistenceRules = (
return {
error: new PentestError(
`${message} (${vulnType})`,
message,
'validation',
failedRule.retryable,
{
apps/worker/src/services/renumber-core.ts
+417
View File
@@ -0,0 +1,417 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Deterministic exploitative renumbering and exact-path class publication. */
import { createHash } from 'node:crypto';
import { readPublishedManifest } from '../ai/reconciliation/manifest.js';
import { sastProvenancePath } from '../ai/reconciliation/prepare.js';
import { isProducerId, REF_PREFIX } from '../ai/reconciliation/refs.js';
import type { AddExploitInput, ExploitedExploit } from '../collectors/exploit-collector.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import type { ExactOutputCommit, ExactOutputFile } from './exact-output-commit.js';
import { RenumberError, writeAndCommitExactFiles } from './exact-output-commit.js';
import { renderExploitDeliverable } from './exploit-renderer.js';
import { severityRank } from './finding-order.js';
import { readCommittedFile } from './git-manager.js';
function divergence(checkCode: string, vulnerabilityClass: ReconciliationClass): never {
throw new RenumberError('key-set-divergence', false, { checkCode, vulnerabilityClass });
}
export function pad2(value: number): string {
return String(value).padStart(2, '0');
}
function escapeForRegExp(value: string): string {
return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
// Internal producer-task tokens (VULN/SAST) that survive the reference remap have no meaning in
// customer-facing output, so scrubbing replaces them with neutral wording. This is the boundary
// between the internal reconciliation bookkeeping and the exact-path output an exploit agent and
// the final report both read: without this scrub, a producer/task tag would leak into text a
// downstream agent or the customer report can see, exposing reconciliation internals that should
// stay invisible outside this pipeline stage. Prefixes are tried longest first so one class
// prefix cannot match inside a longer sibling prefix.
const PRODUCER_TOKEN_PATTERN = new RegExp(
`(?:${Object.values(REF_PREFIX)
.slice()
.sort((first, second) => second.length - first.length)
.map(escapeForRegExp)
.join('|')})-(?:VULN|SAST)-\\d+`,
'g',
);
export function remapTaskReferences(text: string, oldToNew: ReadonlyMap<string, string>): string {
const oldReferences = [...oldToNew.keys()].sort((first, second) => second.length - first.length);
if (oldReferences.length === 0) return text;
const alternation = oldReferences.map(escapeForRegExp).join('|');
const referencePattern = new RegExp(`(?:${alternation})(?![0-9])`, 'g');
return text.replace(referencePattern, (oldReference) => oldToNew.get(oldReference) as string);
}
// Walks an entire exploit entry (nested objects and arrays included) so the reference remap and
// producer-token scrub apply to every string field, not just the ones a caller happens to check.
function scrubEntryText(value: unknown, oldToNew: ReadonlyMap<string, string>): unknown {
if (typeof value === 'string') {
return remapTaskReferences(value, oldToNew).replace(PRODUCER_TOKEN_PATTERN, 'a related finding');
}
if (Array.isArray(value)) return value.map((entry) => scrubEntryText(entry, oldToNew));
if (value !== null && typeof value === 'object') {
const scrubbed: Record<string, unknown> = {};
for (const [key, entry] of Object.entries(value)) scrubbed[key] = scrubEntryText(entry, oldToNew);
return scrubbed;
}
return value;
}
export function parseRefNumber(reference: string, vulnerabilityClass: ReconciliationClass): number | null {
const prefix = REF_PREFIX[vulnerabilityClass];
const match = new RegExp(`^${escapeForRegExp(prefix)}-(\\d+)$`).exec(reference);
if (match === null) return null;
const digits = match[1] as string;
const parsed = Number.parseInt(digits, 10);
if (!Number.isFinite(parsed) || parsed < 1 || digits !== pad2(parsed)) return null;
return parsed;
}
export type ExclusionReason = 'validation_blocked';
export interface ExcludedEntry {
readonly source_ref: string;
readonly reason: ExclusionReason;
}
export interface RenumberMap {
readonly oldToNew: Map<string, string>;
readonly renumbered: AddExploitInput[];
readonly order: string[];
readonly excluded: ExcludedEntry[];
}
/** Validate the complete collector before partitioning and densely order only exploited survivors. */
export function buildRenumberMap(
entries: readonly AddExploitInput[],
vulnerabilityClass: ReconciliationClass,
): RenumberMap {
const decorated = entries.map((entry) => {
const record = entry as unknown as Record<string, unknown>;
if (entry === null || typeof entry !== 'object' || typeof record.vulnerability_id !== 'string') {
throw new RenumberError('unmappable-survivor', false);
}
const numericReference = parseRefNumber(record.vulnerability_id, vulnerabilityClass);
if (numericReference === null || (record.status !== 'exploited' && record.status !== 'blocked')) {
throw new RenumberError('unmappable-survivor', false);
}
return {
entry,
numericReference,
oldReference: record.vulnerability_id,
status: record.status,
};
});
const seen = new Set<string>();
for (const entry of decorated) {
if (seen.has(entry.oldReference)) throw new RenumberError('unmappable-survivor', false);
seen.add(entry.oldReference);
}
const exploited = decorated.filter((entry) => entry.status === 'exploited');
const blocked = decorated.filter((entry) => entry.status === 'blocked');
if (exploited.length + blocked.length !== decorated.length) {
throw new RenumberError('unmappable-survivor', false);
}
exploited.sort((first, second) => {
const severityDifference =
severityRank((first.entry as ExploitedExploit).severity) -
severityRank((second.entry as ExploitedExploit).severity);
if (severityDifference !== 0) return severityDifference;
if (first.numericReference !== second.numericReference) {
return first.numericReference - second.numericReference;
}
if (first.oldReference < second.oldReference) return -1;
if (first.oldReference > second.oldReference) return 1;
return 0;
});
const oldToNew = new Map<string, string>();
const order: string[] = [];
for (const [index, entry] of exploited.entries()) {
oldToNew.set(entry.oldReference, `${REF_PREFIX[vulnerabilityClass]}-${pad2(index + 1)}`);
order.push(entry.oldReference);
}
const renumbered = exploited.map((entry) => {
const scrubbed = scrubEntryText(entry.entry, oldToNew) as Record<string, unknown>;
return {
...scrubbed,
vulnerability_id: oldToNew.get(entry.oldReference) as string,
} as unknown as AddExploitInput;
});
const excluded = blocked.map((entry) => ({
source_ref: entry.oldReference,
reason: 'validation_blocked' as const,
}));
return { oldToNew, renumbered, order, excluded };
}
export interface SastProvenanceEntry {
readonly exploit_ref: string;
readonly [key: string]: unknown;
}
export interface SastProvenanceFile {
readonly entries: readonly SastProvenanceEntry[];
}
export function remapSastProvenance(
provenance: SastProvenanceFile,
oldToNew: ReadonlyMap<string, string>,
): SastProvenanceFile {
const remapped: SastProvenanceEntry[] = [];
const seen = new Set<string>();
for (const entry of provenance.entries) {
const nextReference = oldToNew.get(entry.exploit_ref);
if (nextReference === undefined) continue;
if (seen.has(nextReference))
throw new RenumberError('key-set-divergence', false, { checkCode: 'provenance-duplicate' });
seen.add(nextReference);
remapped.push({ ...entry, exploit_ref: nextReference });
}
return { entries: remapped };
}
function sha256(contents: string): string {
return createHash('sha256').update(contents, 'utf8').digest('hex');
}
export function sparseExploitCollectorPath(vulnerabilityClass: ReconciliationClass): string {
return `${vulnerabilityClass}_exploit_collector.json`;
}
export function renumberedExploitCollectorPath(vulnerabilityClass: ReconciliationClass): string {
return `${vulnerabilityClass}_exploit_collector_renumbered.json`;
}
export function exploitationEvidencePath(vulnerabilityClass: ReconciliationClass): string {
return `${vulnerabilityClass}_exploitation_evidence.md`;
}
export function renumberMapPath(vulnerabilityClass: ReconciliationClass): string {
return `${vulnerabilityClass}_renumber_map.json`;
}
export function renumberedSastProvenancePath(vulnerabilityClass: ReconciliationClass): string {
return `sast_provenance_${vulnerabilityClass}_renumbered.json`;
}
export interface RenumberProducts extends RenumberMap {
readonly evidenceMarkdown: string;
readonly provenance?: SastProvenanceFile;
}
function parseProvenance(value: unknown, vulnerabilityClass: ReconciliationClass): SastProvenanceFile {
if (value === null || typeof value !== 'object' || !Array.isArray((value as { entries?: unknown }).entries)) {
return divergence('provenance-malformed', vulnerabilityClass);
}
const entries = (value as { entries: unknown[] }).entries;
const seen = new Set<string>();
for (const entry of entries) {
if (entry === null || typeof entry !== 'object')
return divergence('provenance-entry-malformed', vulnerabilityClass);
const reference = (entry as { exploit_ref?: unknown }).exploit_ref;
if (
typeof reference !== 'string' ||
parseRefNumber(reference, vulnerabilityClass) === null ||
seen.has(reference)
) {
return divergence('provenance-reference-invalid', vulnerabilityClass);
}
seen.add(reference);
}
return value as SastProvenanceFile;
}
async function loadAndValidateProvenance(
dir: string,
vulnerabilityClass: ReconciliationClass,
collectorReferences: ReadonlySet<string>,
): Promise<SastProvenanceFile | undefined> {
const manifestRead = await readPublishedManifest(dir, `${vulnerabilityClass}_reconciliation_manifest.json`);
if (manifestRead.state !== 'present') return divergence('manifest-not-present', vulnerabilityClass);
if (manifestRead.manifest.vulnerability_class !== vulnerabilityClass) {
return divergence('manifest-class-mismatch', vulnerabilityClass);
}
const consumerContents = new Map<string, string>();
for (const consumer of manifestRead.manifest.consumer_files) {
const read = await readCommittedFile(dir, consumer.path);
if (read.state !== 'present' || sha256(read.contents) !== consumer.sha256) {
return divergence('manifest-consumer-digest-mismatch', vulnerabilityClass);
}
consumerContents.set(consumer.path, read.contents);
}
const taskUniverse = new Set(Object.keys(manifestRead.manifest.lineage));
const queueContents = consumerContents.get(`${vulnerabilityClass}_exploitation_queue.json`);
if (queueContents === undefined) return divergence('manifest-queue-consumer-missing', vulnerabilityClass);
let queue: unknown;
try {
queue = JSON.parse(queueContents) as unknown;
} catch {
return divergence('manifest-queue-not-json', vulnerabilityClass);
}
if (
queue === null ||
typeof queue !== 'object' ||
!Array.isArray((queue as { vulnerabilities?: unknown }).vulnerabilities)
) {
return divergence('manifest-queue-malformed', vulnerabilityClass);
}
const lineageReferences = Object.keys(manifestRead.manifest.lineage);
const queueReferences = (queue as { vulnerabilities: unknown[] }).vulnerabilities.map((entry) =>
entry !== null && typeof entry === 'object' ? (entry as { ID?: unknown }).ID : undefined,
);
if (
queueReferences.length !== lineageReferences.length ||
queueReferences.some((reference, index) => reference !== lineageReferences[index])
) {
return divergence('manifest-queue-lineage-mismatch', vulnerabilityClass);
}
for (const reference of collectorReferences) {
if (!taskUniverse.has(reference)) return divergence('collector-reference-outside-publication', vulnerabilityClass);
}
const provenanceRelPath = sastProvenancePath(vulnerabilityClass);
const manifestConsumer = manifestRead.manifest.consumer_files.find((consumer) => consumer.path === provenanceRelPath);
if (manifestConsumer === undefined) return undefined;
const provenanceContents = consumerContents.get(provenanceRelPath);
if (provenanceContents === undefined) return divergence('provenance-digest-mismatch', vulnerabilityClass);
let decoded: unknown;
try {
decoded = JSON.parse(provenanceContents);
} catch {
return divergence('provenance-not-json', vulnerabilityClass);
}
const provenance = parseProvenance(decoded, vulnerabilityClass);
for (const entry of provenance.entries) {
if (!taskUniverse.has(entry.exploit_ref))
return divergence('provenance-reference-outside-publication', vulnerabilityClass);
const lineage = manifestRead.manifest.lineage[entry.exploit_ref];
if (
lineage === undefined ||
![lineage.primary, ...lineage.absorbed].some((producerId) => isProducerId(producerId, vulnerabilityClass, 'SAST'))
) {
return divergence('provenance-reference-not-sast-backed', vulnerabilityClass);
}
}
return provenance;
}
export async function computeRenumber(
dir: string,
vulnerabilityClass: ReconciliationClass,
): Promise<RenumberProducts | null> {
const collectorRead = await readCommittedFile(dir, sparseExploitCollectorPath(vulnerabilityClass));
if (collectorRead.state === 'absent') return null;
if (collectorRead.state === 'corrupt') throw new RenumberError('unmappable-survivor', false);
let decoded: unknown;
try {
decoded = JSON.parse(collectorRead.contents);
} catch {
throw new RenumberError('unmappable-survivor', false);
}
if (!Array.isArray(decoded)) throw new RenumberError('unmappable-survivor', false);
const map = buildRenumberMap(decoded as AddExploitInput[], vulnerabilityClass);
const collectorReferences = new Set<string>([
...map.oldToNew.keys(),
...map.excluded.map((entry) => entry.source_ref),
]);
const sparseProvenance = await loadAndValidateProvenance(dir, vulnerabilityClass, collectorReferences);
const evidenceMarkdown = renderExploitDeliverable(
vulnerabilityClass,
map.renumbered,
new Map<string, string>(),
).replace(
'*No vulnerabilities were available in the queue for exploitation.*',
'*No vulnerabilities were confirmed during exploitation.*',
);
return {
...map,
evidenceMarkdown,
...(sparseProvenance !== undefined && { provenance: remapSastProvenance(sparseProvenance, map.oldToNew) }),
};
}
export function renumberOutputFiles(
vulnerabilityClass: ReconciliationClass,
products: RenumberProducts,
): ExactOutputFile[] {
return [
{
relPath: renumberedExploitCollectorPath(vulnerabilityClass),
contents: `${JSON.stringify(products.renumbered, null, 2)}\n`,
},
{ relPath: exploitationEvidencePath(vulnerabilityClass), contents: products.evidenceMarkdown },
{
relPath: renumberMapPath(vulnerabilityClass),
contents: `${JSON.stringify(
{
vulnerability_type: vulnerabilityClass,
map: Object.fromEntries(products.oldToNew),
order: products.order,
excluded: products.excluded,
},
null,
2,
)}\n`,
},
...(products.provenance === undefined
? []
: [
{
relPath: renumberedSastProvenancePath(vulnerabilityClass),
contents: `${JSON.stringify(products.provenance, null, 2)}\n`,
},
]),
];
}
export interface RenumberClassResult {
readonly vulnerabilityClass: ReconciliationClass;
readonly renumberedCount: number;
readonly skipped: boolean;
readonly commit?: ExactOutputCommit;
}
/** Service boundary used by the later Temporal activity wrapper. */
export async function renumberClassFindings(args: {
readonly deliverablesDir: string;
readonly vulnerabilityClass: ReconciliationClass;
readonly logger: ActivityLogger;
}): Promise<RenumberClassResult> {
const products = await computeRenumber(args.deliverablesDir, args.vulnerabilityClass);
if (products === null) {
return { vulnerabilityClass: args.vulnerabilityClass, renumberedCount: 0, skipped: true };
}
const commit = await writeAndCommitExactFiles(
args.deliverablesDir,
renumberOutputFiles(args.vulnerabilityClass, products),
`Renumber ${args.vulnerabilityClass} to dense report references`,
args.logger,
);
return {
vulnerabilityClass: args.vulnerabilityClass,
renumberedCount: products.renumbered.length,
skipped: false,
commit,
};
}
apps/worker/src/services/report-checkpoints.ts
+216
View File
@@ -0,0 +1,216 @@
// Copyright (C) 2025 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Coherence proofs over durable report-stage checkpoints in the deliverables Git repo. */
import { createHash } from 'node:crypto';
import path from 'node:path';
import { $ } from 'zx';
import {
ASSEMBLED_REPORT_FILENAME,
REPORT_FINALIZATION_MANIFEST_FILENAME,
REPORT_JSON_FILENAME,
SARIF_FILENAME,
} from '../paths.js';
import { ErrorCode } from '../types/errors.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import type { ReportProgress } from '../types/run-state.js';
import { fileExists } from '../utils/file-io.js';
import { PentestError } from './error-handling.js';
import { classifyHeadReadFailure, withGitRepoLock } from './git-manager.js';
import { isReportFinalizationManifest } from './report-finalization.js';
import type { ReportData } from './report-renderer.js';
function sha256(contents: string): string {
return createHash('sha256').update(contents, 'utf8').digest('hex');
}
function arraysEqual<T>(left: readonly T[], right: readonly T[]): boolean {
return left.length === right.length && left.every((value, index) => value === right[index]);
}
function transientCheckpointReadError(operation: string): PentestError {
return new PentestError(
'A durable report checkpoint could not be read because of a transient repository error',
'filesystem',
true,
{ operation },
ErrorCode.GIT_CHECKPOINT_FAILED,
);
}
export type CheckpointReadResult =
| { readonly state: 'present'; readonly contents: string }
| { readonly state: 'absent' }
| { readonly state: 'corrupt' };
/**
* Read one file at a specific checkpoint, preserving the proven-present, proven-absent,
* corrupt, and transient outcomes. Transient failures throw so the activity retry policy
* stays authoritative instead of a Git blip erasing a paid-for draft or masquerading as
* workspace corruption.
*/
export async function readFileAtCheckpoint(
deliverablesPath: string,
checkpoint: string,
relPath: string,
): Promise<CheckpointReadResult> {
return withGitRepoLock(async () => {
const result = await $`cd ${deliverablesPath} && git show ${`${checkpoint}:${relPath}`}`.nothrow().quiet();
if (result.exitCode === 0) {
return { state: 'present', contents: result.stdout };
}
const failure = classifyHeadReadFailure(result.stderr);
if (failure === 'absent') return { state: 'absent' };
if (failure === 'corrupt') return { state: 'corrupt' };
throw transientCheckpointReadError('read-report-checkpoint-file');
});
}
export async function checkpointFileContents(
deliverablesPath: string,
checkpoint: string,
relPath: string,
): Promise<string | null> {
const read = await readFileAtCheckpoint(deliverablesPath, checkpoint, relPath);
return read.state === 'present' ? read.contents : null;
}
/** Resolve a revision to a commit hash; absent/corrupt yields null, transient throws. */
export async function resolveCheckpointCommit(deliverablesPath: string, revision: string): Promise<string | null> {
return withGitRepoLock(async () => {
const result = await $`cd ${deliverablesPath} && git rev-parse --verify ${`${revision}^{commit}`}`
.nothrow()
.quiet();
if (result.exitCode === 0) return result.stdout.trim();
const failure = classifyHeadReadFailure(result.stderr);
if (failure === 'transient') throw transientCheckpointReadError('resolve-report-checkpoint');
return null;
});
}
/** Ancestor check that keeps transient Git failures retryable instead of proof-invalid. */
export async function checkpointIsAncestor(
ancestor: string,
descendant: string,
deliverablesPath: string,
): Promise<boolean> {
return withGitRepoLock(async () => {
const result = await $`cd ${deliverablesPath} && git merge-base --is-ancestor ${ancestor} ${descendant}`
.nothrow()
.quiet();
if (result.exitCode === 0) return true;
if (result.exitCode === 1 && result.stderr.trim() === '') return false;
const failure = classifyHeadReadFailure(result.stderr);
if (failure === 'transient') throw transientCheckpointReadError('verify-report-checkpoint-ancestry');
return false;
});
}
async function checkpointIsReachable(deliverablesPath: string, checkpoint: string): Promise<boolean> {
const head = await resolveCheckpointCommit(deliverablesPath, 'HEAD');
return head !== null && (await checkpointIsAncestor(checkpoint, head, deliverablesPath));
}
export async function reportCheckpointIsCoherent(
deliverablesPath: string,
checkpoint: string,
failedClasses: readonly ReconciliationClass[],
): Promise<boolean> {
const contents = await checkpointFileContents(deliverablesPath, checkpoint, REPORT_JSON_FILENAME);
if (contents === null || !(await checkpointIsReachable(deliverablesPath, checkpoint))) return false;
try {
const decoded = JSON.parse(contents) as ReportData;
return (
decoded !== null &&
typeof decoded === 'object' &&
decoded.report_meta !== null &&
typeof decoded.report_meta === 'object' &&
Array.isArray(decoded.findings) &&
arraysEqual(decoded.reconciliation_failed ?? [], failedClasses)
);
} catch {
return false;
}
}
export type DraftValidation = 'coherent' | 'invalid-model' | 'invalid-canonical';
export async function validateDraftProgress(
deliverablesPath: string,
progress: ReportProgress,
): Promise<DraftValidation> {
if (progress.stage === 'pending') return 'coherent';
if (
!(await reportCheckpointIsCoherent(deliverablesPath, progress.model_checkpoint, progress.renumber_failed_classes))
) {
return 'invalid-model';
}
if (progress.canonical_checkpoint === undefined) return progress.stage === 'draft' ? 'coherent' : 'invalid-canonical';
if (!(await checkpointIsAncestor(progress.model_checkpoint, progress.canonical_checkpoint, deliverablesPath))) {
return 'invalid-canonical';
}
return (await reportCheckpointIsCoherent(
deliverablesPath,
progress.canonical_checkpoint,
progress.renumber_failed_classes,
))
? 'coherent'
: 'invalid-canonical';
}
export async function draftProgressIsCoherent(deliverablesPath: string, progress: ReportProgress): Promise<boolean> {
return (await validateDraftProgress(deliverablesPath, progress)) === 'coherent';
}
export async function finalProgressIsCoherent(deliverablesPath: string, progress: ReportProgress): Promise<boolean> {
if (progress.stage !== 'finalized' || !(await draftProgressIsCoherent(deliverablesPath, progress))) return false;
if (!(await checkpointIsAncestor(progress.canonical_checkpoint, progress.final_checkpoint, deliverablesPath))) {
return false;
}
if (!(await checkpointIsReachable(deliverablesPath, progress.final_checkpoint))) return false;
const manifestContents = await checkpointFileContents(
deliverablesPath,
progress.final_checkpoint,
REPORT_FINALIZATION_MANIFEST_FILENAME,
);
if (manifestContents === null || sha256(manifestContents) !== progress.finalization_manifest_sha256) return false;
let manifest: unknown;
try {
manifest = JSON.parse(manifestContents) as unknown;
} catch {
return false;
}
if (!isReportFinalizationManifest(manifest) || manifestContents !== `${JSON.stringify(manifest, null, 2)}\n`) {
return false;
}
if (manifest.artifacts.sarif.disposition !== progress.sarif_disposition) return false;
const reportJson = await checkpointFileContents(deliverablesPath, progress.final_checkpoint, REPORT_JSON_FILENAME);
const markdown = await checkpointFileContents(deliverablesPath, progress.final_checkpoint, ASSEMBLED_REPORT_FILENAME);
if (
reportJson === null ||
markdown === null ||
sha256(reportJson) !== manifest.artifacts.report_json.sha256 ||
sha256(markdown) !== manifest.artifacts.markdown.sha256
) {
return false;
}
try {
const decodedReport = JSON.parse(reportJson) as ReportData;
if (!arraysEqual(decodedReport.reconciliation_failed ?? [], progress.renumber_failed_classes)) return false;
} catch {
return false;
}
const sarif = await checkpointFileContents(deliverablesPath, progress.final_checkpoint, SARIF_FILENAME);
if (manifest.artifacts.sarif.disposition !== 'committed') {
return sarif === null && !(await fileExists(path.join(deliverablesPath, SARIF_FILENAME)));
}
return sarif !== null && sha256(sarif) === manifest.artifacts.sarif.sha256;
}
apps/worker/src/services/report-finalization.ts
+456
View File
@@ -0,0 +1,456 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Canonical report publication and derived-PDF regeneration. */
import { createHash } from 'node:crypto';
import { unlink } from 'node:fs/promises';
import path from 'node:path';
import {
ASSEMBLED_REPORT_FILENAME,
ASSEMBLED_REPORT_PDF_FILENAME,
REPORT_FINALIZATION_MANIFEST_FILENAME,
REPORT_JSON_FILENAME,
SARIF_FILENAME,
TYPST_TEMPLATE,
} from '../paths.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import type { DistributedReportConfig } from '../types/config.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import type { ExactOutputCommit, ExactOutputFile } from './exact-output-commit.js';
import { writeAndCommitExactFiles } from './exact-output-commit.js';
import { orderFindings } from './finding-order.js';
import { readCommittedFile, withGitRepoLock } from './git-manager.js';
import { type PdfProvenance, pdfProvenanceIsCurrent, readPdfProvenance, renderReportPdf } from './pdf-renderer.js';
import { type ReportData, renderReport } from './report-renderer.js';
import { renderSarif } from './sarif-renderer.js';
const FINALIZATION_SCHEMA_VERSION = 1;
const REPORT_RENDERER_VERSION = '4.13.1';
const SARIF_RENDERER_VERSION = '4.13.1';
interface CommittedArtifactReceipt {
readonly path: string;
readonly disposition: 'committed';
readonly sha256: string;
}
export type ReportSarifDisposition = 'committed' | 'absent' | 'render_failed';
interface ConditionalArtifactReceipt {
readonly path: string;
readonly disposition: ReportSarifDisposition;
readonly sha256?: string;
}
interface DerivedArtifactReceipt {
readonly path: string;
readonly disposition: 'derived_uncommitted';
}
export interface ReportFinalizationManifest {
readonly schema_version: typeof FINALIZATION_SCHEMA_VERSION;
readonly input_fingerprint: string;
readonly artifacts: {
readonly report_json: CommittedArtifactReceipt;
readonly markdown: CommittedArtifactReceipt;
readonly sarif: ConditionalArtifactReceipt;
readonly pdf: DerivedArtifactReceipt;
};
}
export interface FinalizeReportResult {
readonly commit: ExactOutputCommit;
readonly manifest: ReportFinalizationManifest;
readonly pdfGenerated: boolean;
readonly pdfProvenance: PdfProvenance | null;
readonly warnings: readonly string[];
}
/** Stable service error preserved as retryable by the Temporal integration wrapper. */
export class ReportSarifRenderError extends Error {
readonly retryable = true;
constructor(cause: unknown) {
super('Report SARIF rendering failed.', { cause });
this.name = 'ReportSarifRenderError';
}
}
/**
* Canonical report, manifest, or committed-byte corruption detected during finalization.
* Always terminal: canonical corruption never degrades into a partial result or a warning.
*/
export class ReportFinalizationIntegrityError extends Error {
readonly retryable = false;
readonly checkCode: string;
constructor(checkCode: string) {
super('Report finalization integrity validation failed.');
this.name = 'ReportFinalizationIntegrityError';
this.checkCode = checkCode;
}
}
function sha256(contents: string): string {
return createHash('sha256').update(contents, 'utf8').digest('hex');
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function parseReportData(contents: string): ReportData {
let decoded: unknown;
try {
decoded = JSON.parse(contents) as unknown;
} catch {
throw new ReportFinalizationIntegrityError('finalization-report-not-json');
}
if (!isRecord(decoded) || !isRecord(decoded.report_meta) || !Array.isArray(decoded.findings)) {
throw new ReportFinalizationIntegrityError('finalization-report-malformed');
}
const meta = decoded.report_meta;
if (
typeof meta.target !== 'string' ||
typeof meta.assessment_date !== 'string' ||
typeof meta.scope !== 'string' ||
typeof meta.executive_summary !== 'string' ||
meta.scope.trim() === '' ||
meta.executive_summary.trim() === ''
) {
throw new ReportFinalizationIntegrityError('finalization-report-meta-malformed');
}
return decoded as unknown as ReportData;
}
function isSha256(value: unknown): value is string {
return typeof value === 'string' && /^[a-f0-9]{64}$/.test(value);
}
/** Runtime guard consumed by resume/terminal-state wiring in the next task. */
export function isReportFinalizationManifest(value: unknown): value is ReportFinalizationManifest {
if (!isRecord(value) || value.schema_version !== FINALIZATION_SCHEMA_VERSION || !isSha256(value.input_fingerprint)) {
return false;
}
if (!isRecord(value.artifacts)) return false;
const reportJson = value.artifacts.report_json;
const markdown = value.artifacts.markdown;
const sarif = value.artifacts.sarif;
const pdf = value.artifacts.pdf;
if (!isRecord(reportJson) || !isRecord(markdown) || !isRecord(sarif) || !isRecord(pdf)) return false;
if (
reportJson.path !== REPORT_JSON_FILENAME ||
reportJson.disposition !== 'committed' ||
!isSha256(reportJson.sha256)
) {
return false;
}
if (
markdown.path !== ASSEMBLED_REPORT_FILENAME ||
markdown.disposition !== 'committed' ||
!isSha256(markdown.sha256)
) {
return false;
}
const validSarif =
sarif.path === SARIF_FILENAME &&
(((sarif.disposition === 'absent' || sarif.disposition === 'render_failed') && sarif.sha256 === undefined) ||
(sarif.disposition === 'committed' && isSha256(sarif.sha256)));
return validSarif && pdf.path === ASSEMBLED_REPORT_PDF_FILENAME && pdf.disposition === 'derived_uncommitted';
}
/**
* Derive the one canonical report from the model-authored report.json: fixed finding order,
* workflow-owned exploit flag and coverage, and a de-duplicated reconciliation-failed set. Every
* finalization attempt (including a retried or degraded re-drive) must produce byte-identical
* output from the same inputs, since `buildInputFingerprint` and the adoption check in
* `readExistingFinalization` both compare against this canonical form rather than the raw model
* output.
*/
function canonicalizeReport(args: {
readonly report: ReportData;
readonly exploit: boolean;
readonly reconciliationFailedClasses?: readonly ReconciliationClass[];
}): ReportData {
const reconciliationFailed = args.reconciliationFailedClasses ?? args.report.reconciliation_failed ?? [];
if (new Set(reconciliationFailed).size !== reconciliationFailed.length) {
throw new ReportFinalizationIntegrityError('finalization-failed-class-duplicate');
}
if (
args.reconciliationFailedClasses !== undefined &&
JSON.stringify(args.report.reconciliation_failed ?? []) !== JSON.stringify(args.reconciliationFailedClasses)
) {
throw new ReportFinalizationIntegrityError('finalization-failed-class-set-mismatch');
}
return {
...args.report,
report_meta: { ...args.report.report_meta, exploit: args.exploit },
findings: orderFindings(args.report.findings),
reconciliation_failed: [...reconciliationFailed],
};
}
function buildInputFingerprint(args: {
readonly canonicalJson: string;
readonly exploit: boolean;
readonly workspaceName: string;
readonly reportConfig: DistributedReportConfig;
}): string {
return sha256(
JSON.stringify({
canonical_report_sha256: sha256(args.canonicalJson),
exploit: args.exploit,
workspace_name: args.workspaceName,
report_config: {
min_severity: args.reportConfig.min_severity ?? null,
min_confidence: args.reportConfig.min_confidence ?? null,
guidance_sha256: args.reportConfig.guidance === undefined ? null : sha256(args.reportConfig.guidance),
sarif: args.reportConfig.sarif,
},
report_renderer_version: REPORT_RENDERER_VERSION,
sarif_renderer_version: SARIF_RENDERER_VERSION,
}),
);
}
function buildManifest(args: {
readonly canonicalJson: string;
readonly markdown: string;
readonly sarif: string | null;
readonly sarifDisposition: ReportSarifDisposition;
readonly exploit: boolean;
readonly workspaceName: string;
readonly reportConfig: DistributedReportConfig;
}): ReportFinalizationManifest {
return {
schema_version: FINALIZATION_SCHEMA_VERSION,
input_fingerprint: buildInputFingerprint(args),
artifacts: {
report_json: {
path: REPORT_JSON_FILENAME,
disposition: 'committed',
sha256: sha256(args.canonicalJson),
},
markdown: {
path: ASSEMBLED_REPORT_FILENAME,
disposition: 'committed',
sha256: sha256(args.markdown),
},
sarif:
args.sarifDisposition === 'committed' && args.sarif !== null
? { path: SARIF_FILENAME, disposition: 'committed', sha256: sha256(args.sarif) }
: { path: SARIF_FILENAME, disposition: args.sarifDisposition },
pdf: { path: ASSEMBLED_REPORT_PDF_FILENAME, disposition: 'derived_uncommitted' },
},
};
}
async function readExistingFinalization(args: {
readonly deliverablesDir: string;
readonly canonicalJson: string;
readonly markdown: string;
readonly exploit: boolean;
readonly workspaceName: string;
readonly reportConfig: DistributedReportConfig;
}): Promise<{
readonly manifest: ReportFinalizationManifest;
readonly manifestContents: string;
readonly sarif: string | null;
} | null> {
const read = await readCommittedFile(args.deliverablesDir, REPORT_FINALIZATION_MANIFEST_FILENAME);
if (read.state === 'absent') return null;
if (read.state !== 'present') {
throw new ReportFinalizationIntegrityError('finalization-manifest-unreadable');
}
let decoded: unknown;
try {
decoded = JSON.parse(read.contents) as unknown;
} catch {
throw new ReportFinalizationIntegrityError('finalization-manifest-not-json');
}
if (!isReportFinalizationManifest(decoded) || read.contents !== `${JSON.stringify(decoded, null, 2)}\n`) {
throw new ReportFinalizationIntegrityError('finalization-manifest-conflict');
}
const expectedInputFingerprint = buildInputFingerprint(args);
if (
decoded.input_fingerprint !== expectedInputFingerprint ||
decoded.artifacts.report_json.sha256 !== sha256(args.canonicalJson) ||
decoded.artifacts.markdown.sha256 !== sha256(args.markdown)
) {
throw new ReportFinalizationIntegrityError('finalization-manifest-conflict');
}
const markdownRead = await readCommittedFile(args.deliverablesDir, ASSEMBLED_REPORT_FILENAME);
if (markdownRead.state !== 'present' || markdownRead.contents !== args.markdown) {
throw new ReportFinalizationIntegrityError('finalization-markdown-digest-mismatch');
}
const sarifRead = await readCommittedFile(args.deliverablesDir, SARIF_FILENAME);
let sarif: string | null = null;
if (decoded.artifacts.sarif.disposition !== 'committed') {
if (sarifRead.state !== 'absent') {
throw new ReportFinalizationIntegrityError('finalization-stale-sarif');
}
} else {
if (
sarifRead.state !== 'present' ||
decoded.artifacts.sarif.sha256 === undefined ||
sha256(sarifRead.contents) !== decoded.artifacts.sarif.sha256
) {
throw new ReportFinalizationIntegrityError('finalization-sarif-digest-mismatch');
}
sarif = sarifRead.contents;
}
return { manifest: decoded, manifestContents: read.contents, sarif };
}
/**
* Finalize all canonical report outputs in one exact-path commit, then regenerate the uncommitted
* PDF from those same canonical bytes. PDF failure is warning-only and cannot change the commit.
*/
export async function finalizeReport(args: {
readonly deliverablesDir: string;
readonly exploit: boolean;
readonly reconciliationFailedClasses?: readonly ReconciliationClass[];
readonly reportConfig: DistributedReportConfig;
readonly workspaceName: string;
readonly logger: ActivityLogger;
readonly templatePath?: string;
readonly renderPdf?: typeof renderReportPdf;
readonly renderSarif?: typeof renderSarif;
/** Used only after ordinary SARIF attempts have exhausted their Temporal retry policy. */
readonly degradedSarif?: boolean;
/** Durable provenance from an earlier successful PDF publication, when available. */
readonly priorPdfProvenance?: PdfProvenance;
readonly afterCommit?: (commit: { commitHash: string; changedPaths: readonly string[] }) => void | Promise<void>;
}): Promise<FinalizeReportResult> {
const warnings: string[] = [];
const finalized = await withGitRepoLock(async () => {
const reportRead = await readCommittedFile(args.deliverablesDir, REPORT_JSON_FILENAME);
if (reportRead.state !== 'present') {
throw new ReportFinalizationIntegrityError('finalization-report-unreadable');
}
const canonicalReport = canonicalizeReport({
report: parseReportData(reportRead.contents),
exploit: args.exploit,
...(args.reconciliationFailedClasses !== undefined && {
reconciliationFailedClasses: args.reconciliationFailedClasses,
}),
});
const canonicalJson = `${JSON.stringify(canonicalReport, null, 2)}\n`;
const markdown = renderReport(canonicalReport);
const existing = await readExistingFinalization({
deliverablesDir: args.deliverablesDir,
canonicalJson,
markdown,
exploit: args.exploit,
workspaceName: args.workspaceName,
reportConfig: args.reportConfig,
});
if (existing !== null) {
const files: readonly ExactOutputFile[] = [
{ relPath: REPORT_JSON_FILENAME, contents: canonicalJson },
{ relPath: ASSEMBLED_REPORT_FILENAME, contents: markdown },
{ relPath: SARIF_FILENAME, contents: existing.sarif },
{ relPath: REPORT_FINALIZATION_MANIFEST_FILENAME, contents: existing.manifestContents },
];
const commit = await writeAndCommitExactFiles(
args.deliverablesDir,
files,
'Finalize canonical report outputs',
args.logger,
args.afterCommit === undefined ? {} : { afterCommit: args.afterCommit },
);
return { canonicalReport, commit, manifest: existing.manifest };
}
const sarifRequested = args.exploit && args.reportConfig.sarif;
let sarif: string | null = null;
let sarifDisposition: ReportSarifDisposition = 'absent';
if (sarifRequested && args.degradedSarif === true) {
sarifDisposition = 'render_failed';
} else if (sarifRequested) {
try {
sarif = (args.renderSarif ?? renderSarif)(canonicalReport, args);
sarifDisposition = 'committed';
} catch (error: unknown) {
throw new ReportSarifRenderError(error);
}
}
const manifest = buildManifest({
canonicalJson,
markdown,
sarif,
sarifDisposition,
exploit: args.exploit,
workspaceName: args.workspaceName,
reportConfig: args.reportConfig,
});
if (!isReportFinalizationManifest(manifest)) {
throw new ReportFinalizationIntegrityError('finalization-manifest-self-invalid');
}
const manifestContents = `${JSON.stringify(manifest, null, 2)}\n`;
const files: readonly ExactOutputFile[] = [
{ relPath: REPORT_JSON_FILENAME, contents: canonicalJson },
{ relPath: ASSEMBLED_REPORT_FILENAME, contents: markdown },
{ relPath: SARIF_FILENAME, contents: sarif },
{ relPath: REPORT_FINALIZATION_MANIFEST_FILENAME, contents: manifestContents },
];
const commit = await writeAndCommitExactFiles(
args.deliverablesDir,
files,
'Finalize canonical report outputs',
args.logger,
args.afterCommit === undefined ? {} : { afterCommit: args.afterCommit },
);
return { canonicalReport, commit, manifest };
});
let pdfGenerated = false;
let pdfProvenance: PdfProvenance | null = null;
const pdfPath = path.join(args.deliverablesDir, ASSEMBLED_REPORT_PDF_FILENAME);
const renderPdf = args.renderPdf ?? renderReportPdf;
const templatePath = args.templatePath ?? TYPST_TEMPLATE;
const canonicalReportSha256 = finalized.manifest.artifacts.report_json.sha256;
try {
await renderPdf({
reportData: finalized.canonicalReport,
templatePath,
outputPath: pdfPath,
});
pdfProvenance = await readPdfProvenance({ pdfPath, canonicalReportSha256, templatePath });
pdfGenerated = true;
} catch (error) {
const label = error instanceof Error ? ((error as NodeJS.ErrnoException).code ?? error.name) : 'unknown error';
const warning = `The PDF report could not be produced (${label}). The Markdown report and the structured findings are unaffected.`;
warnings.push(warning);
args.logger.warn(warning);
const canPreservePriorPdf =
args.priorPdfProvenance !== undefined &&
(await pdfProvenanceIsCurrent({
pdfPath,
canonicalReportSha256,
provenance: args.priorPdfProvenance,
templatePath,
}));
if (canPreservePriorPdf) {
pdfProvenance = args.priorPdfProvenance;
} else {
await unlink(pdfPath).catch((cleanupError: unknown) => {
if (cleanupError instanceof Error && (cleanupError as NodeJS.ErrnoException).code === 'ENOENT') return;
const cleanupLabel =
cleanupError instanceof Error
? ((cleanupError as NodeJS.ErrnoException).code ?? cleanupError.name)
: 'unknown error';
const cleanupWarning = `An out-of-date PDF could not be deleted (${cleanupLabel}). Ignore any PDF in this workspace and use the Markdown report.`;
warnings.push(cleanupWarning);
args.logger.warn(cleanupWarning);
});
}
}
return { commit: finalized.commit, manifest: finalized.manifest, pdfGenerated, pdfProvenance, warnings };
}
apps/worker/src/services/report-output-surface.ts
+218
View File
@@ -0,0 +1,218 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Best-effort, narrow customer output publication. */
import { randomUUID } from 'node:crypto';
import { mkdir, readFile, rename, unlink, writeFile } from 'node:fs/promises';
import path from 'node:path';
import {
ASSEMBLED_REPORT_FILENAME,
ASSEMBLED_REPORT_PDF_FILENAME,
FINAL_REPORT_MD_FILENAME,
FINAL_REPORT_PDF_FILENAME,
SARIF_FILENAME,
} from '../paths.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import { type PdfProvenance, pdfMatchesProvenance } from './pdf-renderer.js';
interface SurfaceOutput {
readonly source: string;
readonly destination: string;
readonly removeWhenSourceMissing: boolean;
}
export interface ReportOutputSurfaceResult {
readonly surfaced: readonly string[];
readonly removedStale: readonly string[];
readonly warnings: readonly string[];
}
function isErrno(error: unknown, code: string): boolean {
return error instanceof Error && (error as NodeJS.ErrnoException).code === code;
}
function errorLabel(error: unknown): string {
if (!(error instanceof Error)) return 'unknown error';
return (error as NodeJS.ErrnoException).code ?? error.name;
}
async function removeIfPresent(filePath: string): Promise<boolean> {
try {
await unlink(filePath);
return true;
} catch (error) {
if (isErrno(error, 'ENOENT')) return false;
throw error;
}
}
async function atomicCopy(sourcePath: string, destinationPath: string): Promise<void> {
const contents = await readFile(sourcePath);
await mkdir(path.dirname(destinationPath), { recursive: true });
try {
const existing = await readFile(destinationPath);
if (existing.equals(contents)) return;
} catch (error) {
if (!isErrno(error, 'ENOENT')) throw error;
}
const temporaryPath = `${destinationPath}.tmp-${randomUUID()}`;
try {
await writeFile(temporaryPath, contents, { flag: 'wx' });
await rename(temporaryPath, destinationPath);
const verified = await readFile(destinationPath);
if (!verified.equals(contents)) {
const error = new Error('customer copy verification failed') as NodeJS.ErrnoException;
error.code = 'EIO';
throw error;
}
} catch (error) {
await unlink(temporaryPath).catch(() => undefined);
throw error;
}
}
async function surfaceProvenancedPdf(args: {
readonly deliverablesDir: string;
readonly customerDir: string;
readonly canonicalReportSha256: string;
readonly provenance: PdfProvenance | null;
readonly logger: ActivityLogger;
readonly copyOutput: (sourcePath: string, destinationPath: string) => Promise<void>;
readonly surfaced: string[];
readonly removedStale: string[];
readonly warnings: string[];
}): Promise<void> {
const sourcePath = path.join(args.deliverablesDir, ASSEMBLED_REPORT_PDF_FILENAME);
const destinationPath = path.join(args.customerDir, FINAL_REPORT_PDF_FILENAME);
const sourceMatches =
args.provenance !== null &&
(await pdfMatchesProvenance({
pdfPath: sourcePath,
canonicalReportSha256: args.canonicalReportSha256,
provenance: args.provenance,
}));
if (sourceMatches) {
try {
await args.copyOutput(sourcePath, destinationPath);
args.surfaced.push(FINAL_REPORT_PDF_FILENAME);
args.logger.info(`Surfaced ${FINAL_REPORT_PDF_FILENAME}`);
} catch (error) {
const warning = `The PDF report could not be produced (${errorLabel(error)}). The Markdown report and the structured findings are unaffected.`;
args.warnings.push(warning);
args.logger.warn(warning);
}
return;
}
try {
await removeIfPresent(sourcePath);
} catch (error) {
const warning = `An out-of-date PDF could not be deleted (${errorLabel(error)}). Ignore any PDF in this workspace and use the Markdown report.`;
args.warnings.push(warning);
args.logger.warn(warning);
}
const customerMatches =
args.provenance !== null &&
(await pdfMatchesProvenance({
pdfPath: destinationPath,
canonicalReportSha256: args.canonicalReportSha256,
provenance: args.provenance,
}));
if (customerMatches) {
args.logger.info(`Preserved verified ${FINAL_REPORT_PDF_FILENAME}`);
return;
}
try {
if (await removeIfPresent(destinationPath)) args.removedStale.push(FINAL_REPORT_PDF_FILENAME);
} catch (error) {
const warning = `An out-of-date PDF could not be deleted (${errorLabel(error)}). Ignore any PDF in this workspace and use the Markdown report.`;
args.warnings.push(warning);
args.logger.warn(warning);
}
}
/**
* Surface only Markdown, PDF, and optional SARIF. A copy failure never changes workflow status;
* it produces a warning and leaves any previous destination atomically intact.
*/
export async function surfaceReportOutputs(args: {
readonly deliverablesDir: string;
readonly customerDir: string;
readonly logger: ActivityLogger;
readonly copyOutput?: (sourcePath: string, destinationPath: string) => Promise<void>;
/** Enables verified PDF reuse once the integration layer supplies durable provenance. */
readonly pdfVerification?: {
readonly canonicalReportSha256: string;
readonly provenance: PdfProvenance | null;
};
}): Promise<ReportOutputSurfaceResult> {
const outputs: SurfaceOutput[] = [
{
source: ASSEMBLED_REPORT_FILENAME,
destination: FINAL_REPORT_MD_FILENAME,
removeWhenSourceMissing: false,
},
{ source: SARIF_FILENAME, destination: SARIF_FILENAME, removeWhenSourceMissing: true },
];
if (args.pdfVerification === undefined) {
outputs.splice(1, 0, {
source: ASSEMBLED_REPORT_PDF_FILENAME,
destination: FINAL_REPORT_PDF_FILENAME,
removeWhenSourceMissing: true,
});
}
const copyOutput = args.copyOutput ?? atomicCopy;
const surfaced: string[] = [];
const removedStale: string[] = [];
const warnings: string[] = [];
for (const output of outputs) {
const sourcePath = path.join(args.deliverablesDir, output.source);
const destinationPath = path.join(args.customerDir, output.destination);
try {
await copyOutput(sourcePath, destinationPath);
surfaced.push(output.destination);
args.logger.info(`Surfaced ${output.destination}`);
} catch (error) {
// An absent optional source is the expected state, not a degradation: drop any stale copy
// left by an earlier run and move on without a warning the operator would learn to ignore.
const sourceLegitimatelyAbsent = isErrno(error, 'ENOENT') && output.removeWhenSourceMissing;
if (sourceLegitimatelyAbsent) {
try {
if (await removeIfPresent(destinationPath)) removedStale.push(output.destination);
} catch (cleanupError) {
const warning = `Could not remove stale ${output.destination} (${errorLabel(cleanupError)})`;
warnings.push(warning);
args.logger.warn(warning);
}
continue;
}
const warning = `Could not surface ${output.destination} (${errorLabel(error)})`;
warnings.push(warning);
args.logger.warn(warning);
}
}
if (args.pdfVerification !== undefined) {
await surfaceProvenancedPdf({
deliverablesDir: args.deliverablesDir,
customerDir: args.customerDir,
canonicalReportSha256: args.pdfVerification.canonicalReportSha256,
provenance: args.pdfVerification.provenance,
logger: args.logger,
copyOutput,
surfaced,
removedStale,
warnings,
});
}
return { surfaced, removedStale, warnings };
}
apps/worker/src/services/report-renderer.ts
+3
View File
@@ -14,6 +14,7 @@
import type { AddFindingInput, AdditionalSection, StepItem, StructuredStep } from '../collectors/finding-collector.js';
import type { VulnClass } from '../types/config.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
// ============================================================================
// TYPES
@@ -34,6 +35,8 @@ export interface ReportData {
// Vuln classes whose pipeline failed and were not assessed this run. Rendered as an explicit
// caveat so an un-assessed class is never presented as a clean result.
readonly not_assessed?: readonly VulnClass[];
/** Exploit classes excluded from compaction after a renumber failure, in workflow order. */
readonly reconciliation_failed?: readonly ReconciliationClass[];
}
// Without this, an analysis-only report reads as though the impact was demonstrated.
apps/worker/src/services/reporting.ts
+158 -123
View File
@@ -5,51 +5,141 @@
// as published by the Free Software Foundation.
import { fs, path } from 'zx';
import {
ASSEMBLED_REPORT_FILENAME,
ASSEMBLED_REPORT_PDF_FILENAME,
deliverablesDir,
FINAL_REPORT_MD_FILENAME,
FINAL_REPORT_PDF_FILENAME,
resolveSessionJsonPath,
SARIF_FILENAME,
} from '../paths.js';
import { ASSEMBLED_REPORT_FILENAME, deliverablesDir } from '../paths.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import { ErrorCode } from '../types/errors.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import { PentestError } from './error-handling.js';
import { renderExploitDeliverable } from './exploit-renderer.js';
import { readCommittedFile } from './git-manager.js';
import { surfaceReportOutputs } from './report-output-surface.js';
interface DeliverableFile {
vulnerabilityClass: ReconciliationClass;
name: string;
/** Candidate filenames in priority order. First one that exists wins. */
paths: readonly string[];
required: boolean;
}
// Pure function: Assemble final report from specialist deliverables.
// Per class, prefer the exploit-agent's evidence file; fall back to renderer-produced findings.
// Both never coexist for a workspace because scope (exploit flag) is locked.
export async function assembleFinalReport(
const DELIVERABLE_BY_CLASS: Readonly<
Record<ReconciliationClass, { readonly name: string; readonly exploit: string; readonly analysis: string }>
> = Object.freeze({
injection: {
name: 'Injection',
exploit: 'injection_exploitation_evidence.md',
analysis: 'injection_findings.md',
},
xss: { name: 'XSS', exploit: 'xss_exploitation_evidence.md', analysis: 'xss_findings.md' },
auth: {
name: 'Authentication',
exploit: 'auth_exploitation_evidence.md',
analysis: 'auth_findings.md',
},
ssrf: { name: 'SSRF', exploit: 'ssrf_exploitation_evidence.md', analysis: 'ssrf_findings.md' },
authz: {
name: 'Authorization',
exploit: 'authz_exploitation_evidence.md',
analysis: 'authz_findings.md',
},
miscellaneous: {
name: 'Miscellaneous',
exploit: 'miscellaneous_exploitation_evidence.md',
analysis: 'miscellaneous_findings.md',
},
});
const DEFAULT_REPORT_CLASS_ORDER = [
'injection',
'xss',
'auth',
'ssrf',
'authz',
] as const satisfies readonly ReconciliationClass[];
export interface AssembleFinalReportOptions {
/** Explicit mode prevents an exploitative report from falling back to analysis artifacts. */
readonly exploit?: boolean;
/** Caller-owned order is preserved verbatim. */
readonly participatingClasses?: readonly ReconciliationClass[];
/** Classes already known to have failed during analysis-only findings rendering. */
readonly knownFailedClasses?: readonly ReconciliationClass[];
}
export interface AssembleFinalReportResult {
readonly content: string;
readonly failedClasses: readonly ReconciliationClass[];
}
/**
* Distinguish an assessed class with no actionable findings from a class whose exploit evidence
* disappeared. The committed reconciled queue is authoritative across retries and resume; the
* workflow's in-memory skipped-agent list is not.
*/
async function renderCommittedEmptyClass(dir: string, vulnerabilityClass: ReconciliationClass): Promise<string | null> {
const queueRead = await readCommittedFile(dir, `${vulnerabilityClass}_exploitation_queue.json`);
if (queueRead.state !== 'present') return null;
let queue: unknown;
try {
queue = JSON.parse(queueRead.contents) as unknown;
} catch {
return null;
}
if (
queue === null ||
typeof queue !== 'object' ||
!Array.isArray((queue as { vulnerabilities?: unknown }).vulnerabilities) ||
(queue as { vulnerabilities: unknown[] }).vulnerabilities.length !== 0
) {
return null;
}
return renderExploitDeliverable(vulnerabilityClass, [], new Map());
}
async function assembleFinalReportInternal(
sourceDir: string,
deliverablesSubdir: string | undefined,
logger: ActivityLogger,
): Promise<string> {
const deliverableFiles: readonly DeliverableFile[] = [
{ name: 'Injection', paths: ['injection_exploitation_evidence.md', 'injection_findings.md'], required: false },
{ name: 'XSS', paths: ['xss_exploitation_evidence.md', 'xss_findings.md'], required: false },
{ name: 'Authentication', paths: ['auth_exploitation_evidence.md', 'auth_findings.md'], required: false },
{ name: 'SSRF', paths: ['ssrf_exploitation_evidence.md', 'ssrf_findings.md'], required: false },
{ name: 'Authorization', paths: ['authz_exploitation_evidence.md', 'authz_findings.md'], required: false },
];
options: AssembleFinalReportOptions,
collectClassFailures: boolean,
): Promise<AssembleFinalReportResult> {
const participatingClasses = options.participatingClasses ?? DEFAULT_REPORT_CLASS_ORDER;
const deliverableFiles: readonly DeliverableFile[] = participatingClasses.map((vulnerabilityClass) => {
const definition = DELIVERABLE_BY_CLASS[vulnerabilityClass];
let paths: readonly string[] = [definition.exploit, definition.analysis];
if (options.exploit === true) paths = [definition.exploit];
if (options.exploit === false) paths = [definition.analysis];
return { vulnerabilityClass, name: definition.name, paths, required: false };
});
const dir = deliverablesDir(sourceDir, deliverablesSubdir);
const sections: string[] = [];
const failedClassSet = new Set(options.knownFailedClasses ?? []);
for (const file of deliverableFiles) {
if (failedClassSet.has(file.vulnerabilityClass)) {
logger.warn(`${file.name}: omitted because findings rendering failed`);
continue;
}
let added = false;
for (const candidate of file.paths) {
const filePath = path.join(dir, candidate);
try {
if (await fs.pathExists(filePath)) {
if (options.exploit === true) {
const committed = await readCommittedFile(dir, candidate);
if (committed.state === 'corrupt') {
throw new Error('committed artifact is corrupt');
}
if (committed.state === 'present') {
sections.push(committed.contents);
logger.info(`Added ${file.name} section from ${candidate}`);
added = true;
break;
}
} else {
const filePath = path.join(dir, candidate);
if (!(await fs.pathExists(filePath))) continue;
const content = await fs.readFile(filePath, 'utf8');
sections.push(content);
logger.info(`Added ${file.name} section from ${candidate}`);
@@ -57,8 +147,19 @@ export async function assembleFinalReport(
break;
}
} catch (error) {
if (!collectClassFailures) throw error;
const err = error as Error;
logger.warn(`Could not read ${candidate}: ${err.message}`);
failedClassSet.add(file.vulnerabilityClass);
break;
}
}
if (!added && options.exploit === true && !failedClassSet.has(file.vulnerabilityClass)) {
const emptyClassSection = await renderCommittedEmptyClass(dir, file.vulnerabilityClass);
if (emptyClassSection !== null) {
sections.push(emptyClassSection);
logger.info(`Added ${file.name} section from its committed empty exploitation queue`);
added = true;
}
}
if (!added) {
@@ -72,6 +173,7 @@ export async function assembleFinalReport(
);
}
logger.info(`No ${file.name} deliverable found`);
failedClassSet.add(file.vulnerabilityClass);
}
}
@@ -90,87 +192,44 @@ export async function assembleFinalReport(
});
}
return finalContent;
return {
content: finalContent,
failedClasses: participatingClasses.filter((vulnerabilityClass) => failedClassSet.has(vulnerabilityClass)),
};
}
/**
* Inject model information into the final security report.
* Reads session.json to get the model(s) used, then injects a "Model:" line
* into the Executive Summary section of the report.
* Assemble report inputs while returning class-local omissions for `not_assessed` integration.
* Canonical output write failures still throw.
*/
export async function injectModelIntoReport(
repoPath: string,
export async function assembleFinalReportWithEvidence(
sourceDir: string,
deliverablesSubdir: string | undefined,
outputPath: string,
logger: ActivityLogger,
): Promise<void> {
// 1. Read session.json to get model information
const sessionJsonPath = resolveSessionJsonPath(outputPath);
options: AssembleFinalReportOptions = {},
): Promise<AssembleFinalReportResult> {
return assembleFinalReportInternal(sourceDir, deliverablesSubdir, logger, options, true);
}
if (!(await fs.pathExists(sessionJsonPath))) {
logger.warn('session.json not found, skipping model injection');
return;
}
interface SessionData {
metrics: {
agents: Record<string, { model?: string }>;
};
}
const sessionData: SessionData = await fs.readJson(sessionJsonPath);
// 2. Extract unique models from all agents
const models = new Set<string>();
for (const agent of Object.values(sessionData.metrics.agents)) {
if (agent.model) {
models.add(agent.model);
}
}
if (models.size === 0) {
logger.warn('No model information found in session.json');
return;
}
const modelStr = Array.from(models).join(', ');
logger.info(`Injecting model info into report: ${modelStr}`);
// 3. Read the final report
const reportPath = path.join(deliverablesDir(repoPath, deliverablesSubdir), ASSEMBLED_REPORT_FILENAME);
if (!(await fs.pathExists(reportPath))) {
logger.warn('Final report not found, skipping model injection');
return;
}
let reportContent = await fs.readFile(reportPath, 'utf8');
// 4. Find and inject model line after "Assessment Date" in Executive Summary
// Pattern: "- Assessment Date: <date>" followed by a newline
const assessmentDatePattern = /^(- Assessment Date: .+)$/m;
const match = reportContent.match(assessmentDatePattern);
if (match) {
// Inject model line after Assessment Date
const modelLine = `- Model: ${modelStr}`;
reportContent = reportContent.replace(assessmentDatePattern, `$1\n${modelLine}`);
logger.info('Model info injected into Executive Summary');
} else {
// If no Assessment Date line found, try to add after Executive Summary header
const execSummaryPattern = /^## Executive Summary$/m;
if (reportContent.match(execSummaryPattern)) {
// Add model as first item in Executive Summary
reportContent = reportContent.replace(execSummaryPattern, `## Executive Summary\n- Model: ${modelStr}`);
logger.info('Model info added to Executive Summary header');
} else {
logger.warn('Could not find Executive Summary section');
return;
}
}
// 5. Write modified report back
await fs.writeFile(reportPath, reportContent);
// Pure function: Assemble final report from specialist deliverables.
// Per class, prefer the exploit-agent's evidence file; fall back to renderer-produced findings.
// Both never coexist for a workspace because scope (exploit flag) is locked.
export async function assembleFinalReport(
sourceDir: string,
deliverablesSubdir: string | undefined,
logger: ActivityLogger,
optionsOrExploit: AssembleFinalReportOptions | boolean = {},
): Promise<string> {
const options: AssembleFinalReportOptions =
typeof optionsOrExploit === 'boolean' ? { exploit: optionsOrExploit } : optionsOrExploit;
const result = await assembleFinalReportInternal(
sourceDir,
deliverablesSubdir,
logger,
options,
options.exploit !== true,
);
return result.content;
}
/**
@@ -181,7 +240,7 @@ export async function injectModelIntoReport(
*
* The SARIF log is surfaced beside it when present, since a CI step consuming it needs a stable
* path and cannot be expected to reach into the internals directory. It is absent whenever the
* run was analysis-only or `report.sarif` was set to false.
* run was analysis-only or `report.sarif` was not enabled.
*/
export async function copyReportToRunRoot(
repoPath: string,
@@ -190,29 +249,5 @@ export async function copyReportToRunRoot(
logger: ActivityLogger,
): Promise<void> {
const dir = deliverablesDir(repoPath, deliverablesSubdir);
const pdfSource = path.join(dir, ASSEMBLED_REPORT_PDF_FILENAME);
if (await fs.pathExists(pdfSource)) {
const destination = path.join(runDir, FINAL_REPORT_PDF_FILENAME);
await fs.copy(pdfSource, destination, { overwrite: true });
logger.info(`Surfaced PDF report at ${destination}`);
} else {
logger.warn(`PDF report not found, skipping ${FINAL_REPORT_PDF_FILENAME}`);
}
const markdownSource = path.join(dir, ASSEMBLED_REPORT_FILENAME);
if (await fs.pathExists(markdownSource)) {
const destination = path.join(runDir, FINAL_REPORT_MD_FILENAME);
await fs.copy(markdownSource, destination, { overwrite: true });
logger.info(`Surfaced markdown report at ${destination}`);
} else {
logger.warn(`Markdown report not found, skipping ${FINAL_REPORT_MD_FILENAME}`);
}
const sarifSource = path.join(dir, SARIF_FILENAME);
if (await fs.pathExists(sarifSource)) {
const sarifDestination = path.join(runDir, SARIF_FILENAME);
await fs.copy(sarifSource, sarifDestination, { overwrite: true });
logger.info(`Surfaced SARIF log at ${sarifDestination}`);
}
await surfaceReportOutputs({ deliverablesDir: dir, customerDir: runDir, logger });
}
apps/worker/src/session-manager.ts
+27 -4
View File
@@ -8,8 +8,16 @@ import { fs, path } from 'zx';
import type { ActivityLogger } from './types/activity-logger.js';
import type { AgentDefinition, AgentName, AgentValidator, PlaywrightSession, VulnType } from './types/index.js';
import type { ReconciliationClass } from './types/reconciliation.js';
// Agent definitions according to PRD
// Single source of truth for every agent the pipeline can run. Each entry:
// - name / displayName: identity used in logs, metrics, and per-agent log filenames
// - prerequisites: the agents this one conceptually depends on. This is documentation
// of the intended dependency graph, not an executed check. Actual phase ordering and
// concurrency are enforced by the explicit phase structure in the Temporal workflow.
// - promptTemplate: the file under apps/worker/prompts/ (without extension) rendered for this agent
// - deliverableFilename: the canonical filename AgentExecutionService and the
// save-deliverable CLI script write this agent's output under
export const AGENTS: Readonly<Record<AgentName, AgentDefinition>> = Object.freeze({
'pre-recon': {
name: 'pre-recon',
@@ -95,6 +103,16 @@ export const AGENTS: Readonly<Record<AgentName, AgentDefinition>> = Object.freez
promptTemplate: 'exploit-authz',
deliverableFilename: 'authz_exploitation_evidence.md',
},
// Internal class covering findings outside the five core vuln types (from reconciliation
// or agentic SAST). It has no analysis-phase counterpart: there is no 'miscellaneous-vuln'
// agent, since it only ever receives findings that another phase already surfaced.
'miscellaneous-exploit': {
name: 'miscellaneous-exploit',
displayName: 'Miscellaneous exploit agent',
prerequisites: ['recon'],
promptTemplate: 'exploit-miscellaneous',
deliverableFilename: 'miscellaneous_exploitation_evidence.md',
},
report: {
name: 'report',
displayName: 'Report agent',
@@ -121,6 +139,7 @@ export const AGENT_PHASE_MAP: Readonly<Record<AgentName, PhaseName>> = Object.fr
'auth-exploit': 'exploitation',
'authz-exploit': 'exploitation',
'ssrf-exploit': 'exploitation',
'miscellaneous-exploit': 'exploitation',
report: 'reporting',
});
@@ -147,9 +166,9 @@ function createVulnValidator(vulnType: VulnType): AgentValidator {
// hook after the agent succeeds (before the success commit), so a file-existence check
// here would race the renderer.
//
// VulnType is kept in the import surface for createVulnValidator above; this factory
// returns a no-op validator parameterized only for symmetry with the vuln-side factory.
function createExploitValidator(_vulnType: VulnType): AgentValidator {
// Exploitation includes the analysis-less internal `miscellaneous` class, while vulnerability
// analysis remains limited to the five-class VulnType contract above.
function createExploitValidator(_vulnType: ReconciliationClass): AgentValidator {
return async (): Promise<boolean> => true;
}
@@ -179,6 +198,9 @@ export const PLAYWRIGHT_SESSION_MAPPING: Record<string, PlaywrightSession> = Obj
'exploit-ssrf': 'agent4',
'exploit-authz': 'agent5',
// Conditional analysis-less class; it may run beside the five analysis-backed exploit agents.
'exploit-miscellaneous': 'agent6',
// Phase 5: Reporting
'report-executive': 'agent3',
});
@@ -208,6 +230,7 @@ export const AGENT_VALIDATORS: Record<AgentName, AgentValidator> = Object.freeze
'auth-exploit': createExploitValidator('auth'),
'ssrf-exploit': createExploitValidator('ssrf'),
'authz-exploit': createExploitValidator('authz'),
'miscellaneous-exploit': createExploitValidator('miscellaneous'),
// Executive report agent
report: async (sourceDir: string, logger: ActivityLogger): Promise<boolean> => {
apps/worker/src/temporal/activities.ts
+826 -200
View File
File diff suppressed because it is too large Load Diff
apps/worker/src/temporal/pipeline.ts
+5
View File
@@ -7,7 +7,12 @@
export type { ActivityInput } from './activities.js';
export type {
AgenticSastInput,
AgenticSastState,
AgentMetrics,
NonFatalFailure,
OperationalMetrics,
OperationalStageState,
PipelineInput,
PipelineState,
PipelineSummary,
apps/worker/src/temporal/reconcile-activities.ts
+124 -29
View File
@@ -11,6 +11,8 @@ import { type ModelHost, modelHost } from '../ai/model-host.js';
import { createPiStructuredGenerationPort } from '../ai/pi/structured-generation.js';
import {
createTaskFormationExecutor,
TASK_FORMATION_FALLBACK_REASONS,
type TaskFormationExecutionContext,
TaskFormationExecutorError,
type TaskFormationFallbackReason,
} from '../ai/pi/task-formation-executor.js';
@@ -49,8 +51,11 @@ import type {
PrepareResult,
} from '../ai/reconciliation/stage-contracts.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import { renderSafeMessage } from '../types/run-state.js';
import { createActivityLogger } from './activity-logger.js';
import {
ACCEPTED_TASK_FORMATION_FALLBACK_REASONS,
type EnrichClassSastObservationsActivityInput,
type FormClassExploitTasksActivityInput,
type FormClassExploitTasksActivityResult,
@@ -66,10 +71,21 @@ import {
type ReconciliationStableFailureType,
resolveReconciliationActivityBudget,
type SeedEmptyProducerQueueActivityInput,
TASK_FORMATION_EXECUTOR_TIMEOUT_MARGIN_MS,
} from './reconcile-activity-types.js';
const STABLE_FAILURE_TYPES: ReadonlySet<string> = new Set(RECONCILIATION_STABLE_FAILURE_TYPES);
// The workflow validates fallback reasons against its bundle-safe mirror; fail fast at worker
// startup if the mirror ever drifts from the executor's authoritative closed set.
{
const mirror = [...ACCEPTED_TASK_FORMATION_FALLBACK_REASONS].sort();
const authoritative = [...TASK_FORMATION_FALLBACK_REASONS].sort();
if (mirror.length !== authoritative.length || mirror.some((reason, index) => reason !== authoritative[index])) {
throw new Error('The workflow fallback-reason mirror does not match the task-formation executor contract');
}
}
const DEFAULT_RETRYABILITY: Readonly<Record<ReconciliationStableFailureType, boolean>> = Object.freeze({
TaskFormationModelError: true,
SastEnrichmentModelError: true,
@@ -83,17 +99,26 @@ const DEFAULT_RETRYABILITY: Readonly<Record<ReconciliationStableFailureType, boo
KeySetDivergence: false,
});
/**
* One sentence per stable failure type, written for the reader rather than for the
* reconciliation design. `{Class}` and `{class}` are substituted from the failing class,
* which every reconciliation activity carries in its input.
*/
const SAFE_FAILURE_MESSAGES: Readonly<Record<ReconciliationStableFailureType, string>> = Object.freeze({
TaskFormationModelError: 'Task formation did not produce an accepted result.',
SastEnrichmentModelError: 'SAST enrichment did not produce an accepted result.',
ReconciliationArtifactNotFound: 'A reconciliation artifact is not currently visible.',
TaskFormationModelError: 'Shannon could not group {class} findings into test cases.',
SastEnrichmentModelError: 'Shannon could not add code context to the {class} findings from static analysis.',
ReconciliationArtifactNotFound:
'A saved {class} result could not be read back. Re-running this workspace retries it.',
ReconciliationIoError: 'A reconciliation filesystem or Git operation failed.',
ConfigurationError: 'Reconciliation activity configuration is invalid.',
SastEnrichmentInputError: 'The supplied SAST reference is invalid.',
ArtifactIntegrityError: 'Reconciliation artifact integrity validation failed.',
PublicationConflict: 'The durable class publication conflicts with committed state.',
UnmappableSurvivor: 'A report-facing survivor cannot be mapped to the class task set.',
KeySetDivergence: 'Reconciliation report-facing key sets disagree.',
ArtifactIntegrityError: 'A saved {class} result failed its integrity check and was not used.',
PublicationConflict:
"{Class} results were already published by an earlier run, and this run's results differ. Nothing was overwritten.",
UnmappableSurvivor:
'Shannon could not match a finding in the report back to the test case it came from. {Class} results were not published.',
KeySetDivergence:
'Shannon found two disagreeing sets of findings for {class} and stopped rather than publish either.',
});
interface ReconciliationHeartbeatDetails {
@@ -107,6 +132,10 @@ export interface ReconciliationActivityRuntime {
readonly attempt: number;
readonly cancellationSignal: AbortSignal;
readonly logger: ActivityLogger;
/** Temporal's granted per-attempt execution budget, from the activity info. */
readonly startToCloseTimeoutMs?: number;
/** Bounded per-attempt correlation identifier (run id + activity id). */
readonly executionKey?: string;
heartbeat(details: ReconciliationHeartbeatDetails): void;
}
@@ -114,6 +143,9 @@ interface ReconciliationStageRuntime {
readonly signal: AbortSignal;
readonly logger: ActivityLogger;
readonly modelHost: ModelHost;
/** Remaining granted budget minus the deterministic margin, evaluated at call time. */
readonly executorTimeoutMsFor?: () => number | undefined;
readonly executionContextFor?: () => TaskFormationExecutionContext | undefined;
}
export interface ReconciliationStageBindings {
@@ -166,6 +198,8 @@ function defaultRuntime(): ReconciliationActivityRuntime {
attempt: context.info.attempt,
cancellationSignal: context.cancellationSignal,
logger: createActivityLogger(),
startToCloseTimeoutMs: context.info.startToCloseTimeoutMs,
executionKey: `${context.info.workflowExecution.runId}:${context.info.activityId}`,
heartbeat,
};
}
@@ -188,10 +222,11 @@ function applicationFailure(
type: ReconciliationStableFailureType,
retryable: boolean,
stage: ReconciliationActivityName,
vulnerabilityClass: ReconciliationClass,
details: StableFailureDetails = {},
): ApplicationFailure {
return ApplicationFailure.create({
message: SAFE_FAILURE_MESSAGES[type],
message: renderSafeMessage(SAFE_FAILURE_MESSAGES[type], { vulnerabilityClass }),
type,
nonRetryable: !retryable,
details: [
@@ -204,12 +239,33 @@ function applicationFailure(
});
}
function cancellationFrom(error: unknown, signal: AbortSignal): CancelledFailure | undefined {
if (error instanceof CancelledFailure) return error;
const CANCELLATION_CHAIN_DEPTH = 8;
const errorName = error instanceof Error ? error.name : undefined;
const cancelledByName = errorName === 'CancelledFailure' || errorName === 'AbortError';
if (!signal.aborted && !cancelledByName) return undefined;
/**
* A failure counts as cancellation only when the activity signal is aborted AND its bounded
* cause chain carries a real cancellation (the signal's own reason, a `CancelledFailure`, or
* a cancellation-named abort raised under the aborted signal). A provider timeout, an
* abort-shaped provider error with the signal unset, a cleanup failure, or any infrastructure
* fault therefore stays an ordinary typed failure and is never manufactured into cancellation.
*/
function chainContainsRealCancellation(error: unknown, signal: AbortSignal): boolean {
let current: unknown = error;
const seen = new Set<unknown>();
for (let depth = 0; depth < CANCELLATION_CHAIN_DEPTH; depth++) {
if (current === undefined || current === null || seen.has(current)) return false;
if (current === signal.reason) return true;
if (current instanceof CancelledFailure) return true;
if (current instanceof Error && (current.name === 'CancelledFailure' || current.name === 'AbortError')) return true;
seen.add(current);
current = current instanceof Error ? current.cause : undefined;
}
return false;
}
function cancellationFrom(error: unknown, signal: AbortSignal): CancelledFailure | undefined {
if (!signal.aborted) return undefined;
// A proactive check before any stage work has an aborted signal and no failure to inspect.
if (error !== undefined && error !== null && !chainContainsRealCancellation(error, signal)) return undefined;
const reason = signal.reason;
if (reason instanceof CancelledFailure) return reason;
@@ -222,27 +278,32 @@ function cancellationFrom(error: unknown, signal: AbortSignal): CancelledFailure
* error) onto the closed set of stable failure types. Cancellation is checked first and
* always wins, since a stage aborted for cancellation is not a stage that failed.
*/
function normalizeFailure(error: unknown, stage: ReconciliationActivityName, signal: AbortSignal): never {
function normalizeFailure(
error: unknown,
stage: ReconciliationActivityName,
vulnerabilityClass: ReconciliationClass,
signal: AbortSignal,
): never {
const cancellation = cancellationFrom(error, signal);
if (cancellation !== undefined) throw cancellation;
if (error instanceof TaskFormationModelError) {
throw applicationFailure('TaskFormationModelError', error.retryable, stage, {
throw applicationFailure('TaskFormationModelError', error.retryable, stage, vulnerabilityClass, {
metrics: failureMetrics(error),
...(error.fallbackReason !== undefined && { fallbackReason: error.fallbackReason }),
});
}
if (error instanceof SastEnrichmentModelError) {
throw applicationFailure('SastEnrichmentModelError', error.retryable, stage, {
throw applicationFailure('SastEnrichmentModelError', error.retryable, stage, vulnerabilityClass, {
metrics: failureMetrics(error),
});
}
if (error instanceof ReconciliationError) {
throw applicationFailure(error.failureType, error.retryable, stage);
throw applicationFailure(error.failureType, error.retryable, stage, vulnerabilityClass);
}
if (error instanceof TaskFormationExecutorError) {
if (error.failureKind === 'model') {
throw applicationFailure('TaskFormationModelError', error.retryable, stage, {
throw applicationFailure('TaskFormationModelError', error.retryable, stage, vulnerabilityClass, {
metrics: {
costUsd: error.usage.costUsd,
modelCalls: error.modelCalls,
@@ -252,47 +313,54 @@ function normalizeFailure(error: unknown, stage: ReconciliationActivityName, sig
...(error.fallbackReason !== undefined && { fallbackReason: error.fallbackReason }),
});
}
// Retryable executor infrastructure faults (session setup, transient IO) must stay
// retryable IO at the boundary instead of colliding with terminal ConfigurationError.
if (error.failureKind === 'infrastructure') {
throw applicationFailure('ReconciliationIoError', error.retryable, stage, vulnerabilityClass);
}
const type = error.failureKind === 'confinement' ? 'ArtifactIntegrityError' : 'ConfigurationError';
throw applicationFailure(type, error.retryable, stage);
throw applicationFailure(type, error.retryable, stage, vulnerabilityClass);
}
if (error instanceof ApplicationFailure) {
const errorType = error.type;
if (typeof errorType === 'string' && isStableFailureType(errorType)) {
throw applicationFailure(errorType, !error.nonRetryable, stage);
throw applicationFailure(errorType, !error.nonRetryable, stage, vulnerabilityClass);
}
throw applicationFailure('ReconciliationIoError', true, stage);
throw applicationFailure('ReconciliationIoError', true, stage, vulnerabilityClass);
}
if (error instanceof Error && isStableFailureType(error.name)) {
const retryable =
'retryable' in error && typeof error.retryable === 'boolean' ? error.retryable : DEFAULT_RETRYABILITY[error.name];
throw applicationFailure(error.name, retryable, stage);
throw applicationFailure(error.name, retryable, stage, vulnerabilityClass);
}
// Unknown failures remain retryable. A generic error name is not evidence that the fault is terminal.
throw applicationFailure('ReconciliationIoError', true, stage);
throw applicationFailure('ReconciliationIoError', true, stage, vulnerabilityClass);
}
/** Refuse to schedule a class's remaining reconciliation stages once its 12-hour budget is spent. */
function assertActivityCanRun(
activityName: ReconciliationClassActivityName,
classDeadlineMs: number,
vulnerabilityClass: ReconciliationClass,
nowMs: number,
): ReturnType<typeof resolveReconciliationActivityBudget> {
try {
const budget = resolveReconciliationActivityBudget(activityName, classDeadlineMs, nowMs);
if (!budget.shouldSchedule) {
throw applicationFailure('ConfigurationError', false, activityName);
throw applicationFailure('ConfigurationError', false, activityName, vulnerabilityClass);
}
return budget;
} catch (error) {
if (error instanceof ApplicationFailure) throw error;
throw applicationFailure('ConfigurationError', false, activityName);
throw applicationFailure('ConfigurationError', false, activityName, vulnerabilityClass);
}
}
async function runReconciliationStage<T>(
activityName: ReconciliationClassActivityName,
classDeadlineMs: number,
vulnerabilityClass: ReconciliationClass,
runtime: ReconciliationActivityRuntime,
now: () => number,
stage: (runtime: ReconciliationStageRuntime) => Promise<T>,
@@ -301,7 +369,7 @@ async function runReconciliationStage<T>(
const cancellation = cancellationFrom(undefined, runtime.cancellationSignal);
if (cancellation !== undefined) throw cancellation;
const budget = assertActivityCanRun(activityName, classDeadlineMs, now());
const budget = assertActivityCanRun(activityName, classDeadlineMs, vulnerabilityClass, now());
const profile = RECONCILIATION_ACTIVITY_PROFILES[activityName];
const startedAt = now();
let heartbeatInterval: ReturnType<typeof setInterval> | undefined;
@@ -318,10 +386,30 @@ async function runReconciliationStage<T>(
}, budget.heartbeatIntervalMs);
}
// The executor's own timer must expire before Temporal's activity timeout, so the
// metrics-bearing model-stage-timeout failure stays reachable. Evaluate the remaining
// granted budget at call time because jail materialization can consume minutes first.
const grantedBudgetMs = runtime.startToCloseTimeoutMs;
const executorTimeoutMsFor = (): number | undefined => {
if (grantedBudgetMs === undefined || grantedBudgetMs <= 0) return undefined;
const remainingMs = startedAt + grantedBudgetMs - now();
return Math.max(1_000, remainingMs - TASK_FORMATION_EXECUTOR_TIMEOUT_MARGIN_MS);
};
const executionContextFor = (): TaskFormationExecutionContext | undefined => ({
attempt: runtime.attempt,
...(runtime.executionKey !== undefined && { executionKey: runtime.executionKey }),
});
try {
return await stage({ signal: runtime.cancellationSignal, logger: runtime.logger, modelHost: activityModelHost });
return await stage({
signal: runtime.cancellationSignal,
logger: runtime.logger,
modelHost: activityModelHost,
executorTimeoutMsFor,
executionContextFor,
});
} catch (error) {
return normalizeFailure(error, activityName, runtime.cancellationSignal);
return normalizeFailure(error, activityName, vulnerabilityClass, runtime.cancellationSignal);
} finally {
if (heartbeatInterval !== undefined) clearInterval(heartbeatInterval);
}
@@ -334,7 +422,7 @@ async function runSeedStage<T>(runtime: ReconciliationActivityRuntime, stage: ()
try {
return await stage();
} catch (error) {
return normalizeFailure(error, 'seedEmptyProducerQueue', runtime.cancellationSignal);
return normalizeFailure(error, 'seedEmptyProducerQueue', 'miscellaneous', runtime.cancellationSignal);
}
}
@@ -356,6 +444,8 @@ function defaultStages(workspacesDir: string): ReconciliationStageBindings {
workspacesDir,
signalFor: () => runtime.signal,
logger: runtime.logger,
...(runtime.executorTimeoutMsFor !== undefined && { executorTimeoutMsFor: runtime.executorTimeoutMsFor }),
...(runtime.executionContextFor !== undefined && { executionContextFor: runtime.executionContextFor }),
})(input),
materializeClassExploitTasks: materializeClassExploitTasksStage,
publishClassReconciliationOss: publishClassReconciliationOssStage,
@@ -412,6 +502,7 @@ export function createReconciliationActivityRegistry(
const result = await runReconciliationStage(
'prepareClassReconciliation',
input.classDeadlineMs,
input.vulnerabilityClass,
runtime,
now,
() =>
@@ -439,6 +530,7 @@ export function createReconciliationActivityRegistry(
const result = await runReconciliationStage(
'enrichClassSastObservations',
input.classDeadlineMs,
input.vulnerabilityClass,
runtime,
now,
(stageRuntime) =>
@@ -462,6 +554,7 @@ export function createReconciliationActivityRegistry(
const result = await runReconciliationStage(
'formClassExploitTasks',
input.classDeadlineMs,
input.vulnerabilityClass,
runtime,
now,
(stageRuntime) =>
@@ -486,6 +579,7 @@ export function createReconciliationActivityRegistry(
const result = await runReconciliationStage(
'materializeClassExploitTasks',
input.classDeadlineMs,
input.vulnerabilityClass,
runtime,
now,
() =>
@@ -507,6 +601,7 @@ export function createReconciliationActivityRegistry(
const result = await runReconciliationStage(
'publishClassReconciliationOss',
input.classDeadlineMs,
input.vulnerabilityClass,
runtime,
now,
() =>
apps/worker/src/temporal/reconcile-activity-types.ts
+32
View File
@@ -6,6 +6,7 @@
/** Workflow-safe reconciliation activity signatures and scheduling policy. */
import type { TaskFormationFallbackReason } from '../ai/pi/task-formation-executor.js';
import type { ArtifactRef } from '../ai/reconciliation/contracts.js';
import type { StageMetrics } from '../ai/reconciliation/stage-contracts.js';
import type { SarifRef } from '../ai/sast/types.js';
@@ -17,6 +18,37 @@ const HOUR_MS = 60 * MINUTE_MS;
export const RECONCILIATION_CLASS_BUDGET_MS = 12 * HOUR_MS;
export const RECONCILIATION_LATER_STAGE_RESERVE_MS = 5 * MINUTE_MS;
/**
* Deterministic safety margin subtracted from the granted activity budget before it is passed
* to the Pass 1 executor timer, so the executor's own timeout always fires before Temporal's
* activity timeout and the metrics-bearing model-stage-timeout path stays reachable.
*/
export const TASK_FORMATION_EXECUTOR_TIMEOUT_MARGIN_MS = MINUTE_MS;
/**
* Workflow-safe mirror of Agent A's closed fallback-reason set. The executor module itself is
* not bundle-safe, so the workflow validates deserialized failure details against this frozen
* copy; the `satisfies` clause and the exhaustiveness check keep the two sets identical at
* compile time, and the activity boundary re-asserts equality at module load.
*/
export const ACCEPTED_TASK_FORMATION_FALLBACK_REASONS = Object.freeze([
'retryable_model_failure',
'missing_accepted_submission',
'model_stage_timeout',
] as const satisfies readonly TaskFormationFallbackReason[]);
type UnlistedFallbackReason = Exclude<
TaskFormationFallbackReason,
(typeof ACCEPTED_TASK_FORMATION_FALLBACK_REASONS)[number]
>;
const _everyFallbackReasonIsListed: UnlistedFallbackReason extends never ? true : never = true;
void _everyFallbackReasonIsListed;
/** Validate one deserialized fallback reason against the closed set. */
export function isAcceptedTaskFormationFallbackReason(value: unknown): value is TaskFormationFallbackReason {
return (ACCEPTED_TASK_FORMATION_FALLBACK_REASONS as readonly unknown[]).includes(value);
}
export interface ReconciliationActivityDeadline {
/** Fixed workflow-derived deadline for this class, measured as Unix epoch milliseconds. */
readonly classDeadlineMs: number;
apps/worker/src/temporal/shared.ts
+147 -5
View File
@@ -2,15 +2,87 @@ import { defineQuery } from '@temporalio/workflow';
export type { AgentMetrics } from '../types/metrics.js';
import type { DistributedConfig, VulnClass } from '../types/config.js';
import type { CapellaFailurePoint, CapellaStage, SarifRef } from '../ai/sast/types.js';
import type { VulnClass } from '../types/config.js';
import type { ErrorCode } from '../types/errors.js';
import type { AgentMetrics } from '../types/metrics.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import type {
MiscellaneousOutcome,
PartialReasonView,
ReportProgress,
ReportSarifDisposition,
StoredPdfProvenance,
} from '../types/run-state.js';
/**
* The serializable slice of Capella's configuration passed across the Temporal workflow
* boundary into the child workflow input. Everything the workflow needs from the parsed
* config or the model spec must be flattened into plain data here; the workflow sandbox
* cannot carry functions or class instances across that boundary.
*/
export interface AgenticSastInput {
readonly codePathAvoids: readonly string[];
readonly codePathFocus: readonly string[];
readonly modelSpec: string;
readonly capellaFormatVersion: string;
readonly promptSetVersion: string;
}
/**
* The agentic SAST lifecycle as seen from the pentest workflow: not configured, running as a
* child workflow, or one of two terminal outcomes. This is what the live `getProgress` query
* and the terminal `PipelineState` both report, so a caller never needs to inspect the Capella
* child workflow's own result type directly.
*/
export type AgenticSastState =
| { readonly status: 'disabled' }
| { readonly status: 'running'; readonly startedAt: number }
| {
readonly status: 'succeeded';
readonly findingCount: number;
readonly sarifSha256: string;
readonly coverage: 'complete' | 'reduced';
readonly warnings: readonly string[];
readonly durationMs: number;
}
| {
readonly status: 'failed';
readonly failedStage: CapellaFailurePoint;
/** Reader-facing name of `failedStage`, projected once so no surface renders the slug. */
readonly failedStageLabel: string;
readonly error: string;
/** Bounded machine code preserved from the failing Capella activity, when one crossed the child. */
readonly errorCode?: string;
readonly completedStages: readonly CapellaStage[];
readonly durationMs: number;
};
export type OperationalStageStatus = 'pending' | 'running' | 'completed' | 'failed' | 'skipped';
export interface OperationalStageState {
readonly key: string;
readonly label: string;
readonly status: OperationalStageStatus;
readonly startedAt?: number;
readonly durationMs?: number;
readonly error?: string;
}
export interface OperationalMetrics extends AgentMetrics {
readonly usageComplete?: boolean;
}
/** A degradation the scan recorded and continued past, kept for the terminal summary log rather than for control flow. */
export interface NonFatalFailure {
readonly phase: string;
readonly error: string;
}
export interface PipelineInput {
webUrl: string;
repoPath: string;
configPath?: string;
outputPath?: string;
pipelineTestingMode?: boolean;
workflowId?: string; // Used for audit correlation
sessionId?: string; // Workspace directory name (distinct from workflowId for named workspaces)
@@ -19,44 +91,114 @@ export interface PipelineInput {
// Config fields — serializable, flow through to ActivityInput → getOrCreateContainer()
configYAML?: string; // Raw YAML string (parsed in activity, not workflow — workflow sandbox can't use Node.js)
configData?: DistributedConfig; // Pre-parsed config (bypasses file loading)
deliverablesSubdir?: string; // Override deliverables path (default: '.shannon/deliverables')
auditDir?: string; // Override audit log directory (default: './workspaces')
promptDir?: string; // Override prompt template directory
sastSarifPath?: string; // Optional path for consumer-supplied findings input
agenticSast?: AgenticSastInput;
sastSarif?: SarifRef;
customerOutputPath?: string; // Stable mounted path for final customer copies only
checkpointsEnabled?: boolean; // Enable checkpoint activities (default: false)
vulnClasses?: VulnClass[]; // omitted = all five
exploit?: boolean; // false skips the exploitation phase
}
/** What `loadResumeState` reconstructs from a prior workspace: independently verified, never assumed from session.json alone. */
export interface ResumeState {
workspaceName: string;
originalUrl: string;
completedAgents: string[];
checkpointHash: string;
originalWorkflowId: string;
expectedAgents: string[];
participatingClasses: ReconciliationClass[];
exploit: boolean;
reportProgress?: ReportProgress;
miscellaneousOutcome?: MiscellaneousOutcome;
}
/** The narrow view of the durable scan-state record the workflow needs to keep its own queryable state in sync. */
export interface DurableStateSummary {
readonly exploit: boolean;
readonly expectedAgents: readonly string[];
readonly participatingClasses: readonly ReconciliationClass[];
readonly reportStage: ReportProgress['stage'] | 'uninitialized';
readonly miscellaneousOutcome?: MiscellaneousOutcome;
}
/** Common result shape for the deterministic report-processing activities (renumber, compaction). */
export interface ReconciliationActivityResult {
readonly vulnerabilityClass?: ReconciliationClass;
readonly skipped: boolean;
readonly changedPathCount: number;
readonly checkpoint?: string;
readonly alreadyCommitted?: boolean;
}
export interface FinalizeReportActivityResult {
readonly checkpoint: string;
readonly manifestSha256: string;
readonly changedPathCount: number;
readonly alreadyCommitted: boolean;
/** Adopted-or-produced SARIF disposition from the committed finalization manifest. */
readonly sarifDisposition: ReportSarifDisposition;
readonly pdfGenerated: boolean;
/** Verified provenance for the current PDF bytes, or null when no trustworthy PDF exists. */
readonly pdfProvenance: StoredPdfProvenance | null;
readonly warningCount: number;
}
export interface AssembleReportActivityResult {
/** Classes whose findings could not be included in the assembled report inputs. */
readonly failedClasses: readonly ReconciliationClass[];
}
export interface SurfaceReportActivityResult {
readonly surfaced: readonly string[];
readonly removedStale: readonly string[];
readonly warningCount: number;
}
export interface PipelineSummary {
totalCostUsd: number;
totalDurationMs: number; // Wall-clock time (end - start)
totalTurns: number;
/** Total resolved agents: those that ran plus those that were skipped. */
agentCount: number;
/** False when operational (Capella/reconciliation) spend is known to be incomplete. */
usageAccountingComplete: boolean;
}
/**
* The workflow's whole queryable and terminal state. The CLI cannot import this package, so
* `apps/cli/src/scan/pipeline.ts` mirrors this shape (along with AgentMetrics and the
* activity-name-to-agent map) by hand; a field added, renamed, or removed here needs the same
* change there, or the CLI's status rendering silently falls out of sync with a running scan.
*/
export interface PipelineState {
status: 'running' | 'completed' | 'failed' | 'cancelled' | 'partial';
currentPhase: string | null;
currentAgent: string | null;
/** Agents that actually ran. Mutually exclusive from `skippedAgents`. */
completedAgents: string[];
/** Expected agents that never ran because their class had nothing to exploit. */
skippedAgents: string[];
expectedAgents: string[];
participatingClasses: ReconciliationClass[];
// Vuln classes whose pipeline failed while at least one other succeeded. Drives the
// partial terminal status so a crashed class isn't reported as if it fully passed.
failedPipelines: { vulnType: VulnClass; error: string }[];
failedReconciliations: { vulnerabilityClass: ReconciliationClass; error: string }[];
failedAgent: string | null;
error: string | null;
errorCode?: ErrorCode;
startTime: number;
agentMetrics: Record<string, AgentMetrics>;
operationalMetrics: Record<string, OperationalMetrics>;
operationalStages: Record<string, OperationalStageState>;
agenticSast: AgenticSastState;
nonFatalFailures: NonFatalFailure[];
/** Ordered durable degradation reasons with derived safe messages; empty for a full success. */
partialReasons: PartialReasonView[];
reportProgress?: ReportProgress;
summary: PipelineSummary | null;
}
apps/worker/src/temporal/summary-mapper.ts
+16 -1
View File
@@ -29,14 +29,29 @@ export function toWorkflowSummary(
throw new Error('toWorkflowSummary: state.summary must be set before calling');
}
// The failure detail is one of the child workflow's fixed safe sentences, so it carries no
// provider, prompt, repository, or path content and travels with the stable code.
const agenticSastFailure = state.agenticSast.status === 'failed' ? state.agenticSast : undefined;
const agenticSastErrorCode = agenticSastFailure?.errorCode;
const agenticSastFailureMessage = agenticSastFailure?.error;
const agenticSastFailedStage = agenticSastFailure?.failedStageLabel;
return {
status,
totalDurationMs: summary.totalDurationMs,
totalCostUsd: summary.totalCostUsd,
completedAgents: state.completedAgents,
skippedAgents: state.skippedAgents,
agentMetrics: Object.fromEntries(
Object.entries(state.agentMetrics).map(([name, m]) => [name, { durationMs: m.durationMs, costUsd: m.costUsd }]),
[...Object.entries(state.agentMetrics), ...Object.entries(state.operationalMetrics)].map(([name, metrics]) => [
name,
{ durationMs: metrics.durationMs, costUsd: metrics.costUsd },
]),
),
partialReasons: state.partialReasons,
usageAccountingComplete: summary.usageAccountingComplete,
...(agenticSastFailedStage !== undefined && { agenticSastFailedStage }),
...(agenticSastFailureMessage !== undefined && { agenticSastFailureMessage }),
...(agenticSastErrorCode !== undefined && { agenticSastErrorCode }),
...(state.error && { error: state.error }),
};
}
apps/worker/src/temporal/worker.ts
+255 -83
View File
@@ -19,7 +19,7 @@
* Options:
* --task-queue <name> Task queue name (required, unique per scan)
* --config <path> Configuration file path
* --output <path> Output directory for workspaces
* --output <path> Stable mounted path for final customer report copies
* --workspace <name> Resume from existing workspace
* --pipeline-testing Use minimal prompts for fast testing
*
@@ -27,24 +27,60 @@
* TEMPORAL_ADDRESS - Temporal server address (default: localhost:7233)
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { Client, Connection, type WorkflowHandle, WorkflowNotFoundError } from '@temporalio/client';
import { bundleWorkflowCode, NativeConnection, Worker } from '@temporalio/worker';
import dotenv from 'dotenv';
import { DEFAULT_MODEL_SPEC } from '../ai/models.js';
import { capellaActivities, mergeActivityRegistries } from '../ai/sast/capella/temporal/registry.js';
import { CAPELLA_FORMAT_VERSION, CAPELLA_PROMPT_SET_VERSION } from '../ai/sast/capella/types.js';
import { sanitizeHostname } from '../audit/utils.js';
import { parseConfig } from '../config-parser.js';
import {
ASSEMBLED_REPORT_PDF_FILENAME,
deliverablesDir,
FINAL_REPORT_PDF_FILENAME,
resolveSessionJsonPath,
} from '../paths.js';
import type { VulnClass } from '../types/config.js';
import { distributeConfig, parseConfig } from '../config-parser.js';
import { deliverablesDir, resolveSessionJsonPath } from '../paths.js';
import { SAFE_RUN_STATE_MESSAGES, workspaceExploitMismatchMessage } from '../types/run-state.js';
import { fileExists, readJson } from '../utils/file-io.js';
import * as activities from './activities.js';
import type { PipelineInput, PipelineProgress, PipelineState } from './shared.js';
import {
assembleReportActivity,
checkExploitationQueue,
compactReportFindings,
finalizeReportOutputs,
initDeliverableGit,
initializeDurableScanState,
initializeReportProgress,
loadResumeState,
logPhaseTransition,
logWorkflowComplete,
persistCanonicalReportProgress,
persistFinalizedReportProgress,
persistMiscellaneousOutcome,
recordResumeAttempt,
registerResumeAttempt,
renumberClassFindings,
restoreGitCheckpoint,
runAuthExploitAgent,
runAuthenticationValidation,
runAuthVulnAgent,
runAuthzExploitAgent,
runAuthzVulnAgent,
runInjectionExploitAgent,
runInjectionVulnAgent,
runMiscellaneousExploitAgent,
runPreflightValidation,
runPreReconAgent,
runReconAgent,
runReportAgent,
runSsrfExploitAgent,
runSsrfVulnAgent,
runXssExploitAgent,
runXssVulnAgent,
saveCheckpoint,
surfaceReportOutputs,
syncCodePathDenyRules,
syncPlaywrightStealthConfig,
} from './activities.js';
import { createReconciliationActivityRegistry } from './reconcile-activities.js';
import type { AgenticSastInput, PipelineInput, PipelineProgress, PipelineState } from './shared.js';
dotenv.config();
@@ -52,6 +88,118 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url));
const PROGRESS_QUERY = 'getProgress';
// The ordinary activity names. This frozen list is one of three that together form the
// registered activity set the CLI status reader mirrors: the Capella names in
// ai/sast/capella/temporal/activity-types.ts and the reconciliation names in
// reconcile-activity-types.ts are the other two. Adding or removing an activity means
// updating both this list and the `pentestActivities` object below, or the load-time check
// throws.
export const PENTEST_ACTIVITY_NAMES = Object.freeze([
'runPreReconAgent',
'runReconAgent',
'runInjectionVulnAgent',
'runXssVulnAgent',
'runAuthVulnAgent',
'runAuthzVulnAgent',
'runSsrfVulnAgent',
'runInjectionExploitAgent',
'runXssExploitAgent',
'runAuthExploitAgent',
'runAuthzExploitAgent',
'runSsrfExploitAgent',
'runMiscellaneousExploitAgent',
'runReportAgent',
'runPreflightValidation',
'runAuthenticationValidation',
'initDeliverableGit',
'syncPlaywrightStealthConfig',
'syncCodePathDenyRules',
'initializeDurableScanState',
'persistMiscellaneousOutcome',
'initializeReportProgress',
'renumberClassFindings',
'assembleReportActivity',
'compactReportFindings',
'persistCanonicalReportProgress',
'finalizeReportOutputs',
'persistFinalizedReportProgress',
'surfaceReportOutputs',
'checkExploitationQueue',
'loadResumeState',
'restoreGitCheckpoint',
'registerResumeAttempt',
'recordResumeAttempt',
'logPhaseTransition',
'logWorkflowComplete',
'saveCheckpoint',
] as const);
export const pentestActivities = Object.freeze({
runPreReconAgent,
runReconAgent,
runInjectionVulnAgent,
runXssVulnAgent,
runAuthVulnAgent,
runAuthzVulnAgent,
runSsrfVulnAgent,
runInjectionExploitAgent,
runXssExploitAgent,
runAuthExploitAgent,
runAuthzExploitAgent,
runSsrfExploitAgent,
runMiscellaneousExploitAgent,
runReportAgent,
runPreflightValidation,
runAuthenticationValidation,
initDeliverableGit,
syncPlaywrightStealthConfig,
syncCodePathDenyRules,
initializeDurableScanState,
persistMiscellaneousOutcome,
initializeReportProgress,
renumberClassFindings,
assembleReportActivity,
compactReportFindings,
persistCanonicalReportProgress,
finalizeReportOutputs,
persistFinalizedReportProgress,
surfaceReportOutputs,
checkExploitationQueue,
loadResumeState,
restoreGitCheckpoint,
registerResumeAttempt,
recordResumeAttempt,
logPhaseTransition,
logWorkflowComplete,
saveCheckpoint,
});
const registeredPentestNames = Object.keys(pentestActivities).sort();
const expectedPentestNames = [...PENTEST_ACTIVITY_NAMES].sort();
if (
registeredPentestNames.length !== expectedPentestNames.length ||
registeredPentestNames.some((name, index) => name !== expectedPentestNames[index])
) {
throw new Error('Pentest activity registry does not match its frozen ordinary activity contract');
}
export interface ProductionActivityBindings {
readonly repositoryPath: string;
readonly webUrl: string;
readonly workspacesDir: string;
}
/** Compose the frozen ordinary, Capella, and reconciliation activity namespaces. */
export function createProductionActivityRegistry(bindings: ProductionActivityBindings): Readonly<object> {
const reconciliationActivities = createReconciliationActivityRegistry({
repositoryPath: bindings.repositoryPath,
deliverablesDir: deliverablesDir(bindings.repositoryPath),
workspacesDir: bindings.workspacesDir,
webUrl: bindings.webUrl,
});
return mergeActivityRegistries(pentestActivities, capellaActivities, reconciliationActivities);
}
// === CLI Argument Parsing ===
interface CliArgs {
@@ -59,7 +207,7 @@ interface CliArgs {
repoPath: string;
taskQueue: string;
configPath?: string;
outputPath?: string;
customerOutputPath?: string;
pipelineTestingMode: boolean;
resumeFromWorkspace?: string;
}
@@ -73,6 +221,7 @@ function showUsage(): void {
console.log(' --task-queue <name> Task queue name (required)');
console.log(' --config <path> Configuration file path');
console.log(' --workspace <name> Resume from existing workspace');
console.log(' --output <path> Stable mounted path for final customer report copies');
console.log(' --pipeline-testing Use minimal prompts for fast testing\n');
}
@@ -86,7 +235,7 @@ function parseCliArgs(argv: string[]): CliArgs {
let repoPath: string | undefined;
let taskQueue: string | undefined;
let configPath: string | undefined;
let outputPath: string | undefined;
let customerOutputPath: string | undefined;
let pipelineTestingMode = false;
let resumeFromWorkspace: string | undefined;
@@ -107,7 +256,7 @@ function parseCliArgs(argv: string[]): CliArgs {
} else if (arg === '--output') {
const nextArg = argv[i + 1];
if (nextArg && !nextArg.startsWith('-')) {
outputPath = nextArg;
customerOutputPath = nextArg;
i++;
}
} else if (arg === '--workspace') {
@@ -145,7 +294,7 @@ function parseCliArgs(argv: string[]): CliArgs {
taskQueue,
pipelineTestingMode,
...(configPath && { configPath }),
...(outputPath && { outputPath }),
...(customerOutputPath && { customerOutputPath }),
...(resumeFromWorkspace && { resumeFromWorkspace }),
};
}
@@ -158,10 +307,15 @@ interface SessionJson {
webUrl: string;
originalWorkflowId?: string;
resumeAttempts?: Array<{ workflowId: string }>;
status?: 'in-progress' | 'completed' | 'failed' | 'cancelled' | 'partial';
};
metrics: {
total_cost_usd: number;
};
durableScanState?: {
schema_version?: unknown;
exploit?: unknown;
};
}
function isValidWorkspaceName(name: string): boolean {
@@ -216,7 +370,7 @@ async function terminateExistingWorkflows(client: Client, workspaceName: string)
return terminated;
}
async function resolveWorkspace(client: Client, args: CliArgs): Promise<WorkspaceResolution> {
async function resolveWorkspace(client: Client, args: CliArgs, expectedExploit: boolean): Promise<WorkspaceResolution> {
if (!args.resumeFromWorkspace) {
const hostname = sanitizeHostname(args.webUrl);
const workflowId = `${hostname}_shannon-${Date.now()}`;
@@ -233,6 +387,19 @@ async function resolveWorkspace(client: Client, args: CliArgs): Promise<Workspac
const workspaceExists = await fileExists(sessionPath);
if (workspaceExists) {
const session = await readJson<SessionJson>(sessionPath);
if (session.session.webUrl !== args.webUrl) {
throw new Error(
'This workspace was created for a different target URL, so it cannot be resumed against this one. Check -u, or start a new scan with a different -w name.',
);
}
if (session.durableScanState?.schema_version !== 1 || typeof session.durableScanState.exploit !== 'boolean') {
throw new Error(SAFE_RUN_STATE_MESSAGES.CorruptedSessionError);
}
if (session.durableScanState.exploit !== expectedExploit) {
throw new Error(workspaceExploitMismatchMessage(session.durableScanState.exploit));
}
console.log('=== RESUME MODE ===');
console.log(`Workspace: ${workspace}\n`);
@@ -241,14 +408,6 @@ async function resolveWorkspace(client: Client, args: CliArgs): Promise<Workspac
console.log(`Terminated ${terminatedWorkflows.length} previous scan(s)\n`);
}
const session = await readJson<SessionJson>(sessionPath);
if (session.session.webUrl !== args.webUrl) {
console.error('ERROR: URL mismatch with workspace');
console.error(` Workspace URL: ${session.session.webUrl}`);
console.error(` Provided URL: ${args.webUrl}`);
process.exit(1);
}
return {
workflowId: `${workspace}_resume_${Date.now()}`,
sessionId: workspace,
@@ -281,7 +440,7 @@ async function resolveWorkspace(client: Client, args: CliArgs): Promise<Workspac
// === Pipeline Input Construction ===
interface OrchestrationConfig {
vulnClasses?: VulnClass[];
agenticSast?: AgenticSastInput;
exploit?: boolean;
}
@@ -289,10 +448,21 @@ async function loadOrchestrationConfig(configPath: string | undefined): Promise<
if (!configPath) return {};
try {
const config = await parseConfig(configPath);
const distributed = distributeConfig(config);
const codePathAvoids = distributed.avoid.filter((rule) => rule.type === 'code_path').map((rule) => rule.value);
const codePathFocus = distributed.focus.filter((rule) => rule.type === 'code_path').map((rule) => rule.value);
return {
...(config.vuln_classes && config.vuln_classes.length > 0 && { vulnClasses: [...config.vuln_classes] }),
...(config.exploit !== undefined && { exploit: config.exploit === 'true' }),
...(distributed.agenticSast && {
agenticSast: {
codePathAvoids,
codePathFocus,
modelSpec: process.env.SHANNON_AI_MODEL?.trim() || DEFAULT_MODEL_SPEC,
capellaFormatVersion: CAPELLA_FORMAT_VERSION,
promptSetVersion: CAPELLA_PROMPT_SET_VERSION,
},
}),
exploit: distributed.exploit,
};
} catch (error) {
// A broken config must fail the run, not silently fall back to empty
@@ -317,7 +487,8 @@ function buildPipelineInput(
...(args.pipelineTestingMode && { pipelineTestingMode: args.pipelineTestingMode }),
...(workspace.isResume && args.resumeFromWorkspace && { resumeFromWorkspace: args.resumeFromWorkspace }),
...(workspace.terminatedWorkflows.length > 0 && { terminatedWorkflows: workspace.terminatedWorkflows }),
...(orchestration.vulnClasses && { vulnClasses: orchestration.vulnClasses }),
...(args.customerOutputPath !== undefined && { customerOutputPath: args.customerOutputPath }),
...(orchestration.agenticSast !== undefined && { agenticSast: orchestration.agenticSast }),
...(orchestration.exploit !== undefined && { exploit: orchestration.exploit }),
};
}
@@ -332,8 +503,11 @@ async function waitForWorkflowResult(
try {
const progress = await handle.query<PipelineProgress>(PROGRESS_QUERY);
const elapsed = Math.floor(progress.elapsedMs / 1000);
const expectedCount = progress.expectedAgents.length;
// Agentic SAST runs alongside the phase above, so the line names it while it is working.
const agenticSast = progress.agenticSast.status === 'running' ? ' | Agentic SAST: running' : '';
console.log(
`[${elapsed}s] Phase: ${progress.currentPhase || 'unknown'} | Agent: ${progress.currentAgent || 'none'} | Completed: ${progress.completedAgents.length}/13`,
`[${elapsed}s] Phase: ${progress.currentPhase || 'unknown'} | Agent: ${progress.currentAgent || 'none'} | Completed: ${progress.completedAgents.length + progress.skippedAgents.length}/${expectedCount}${agenticSast}`,
);
} catch {
// Workflow may have completed
@@ -344,12 +518,35 @@ async function waitForWorkflowResult(
const result = await handle.result();
clearInterval(progressInterval);
console.log('\nPipeline completed successfully!');
// The returned workflow state distinguishes completed, partial, and cancelled runs;
// each prints its own terminal line so degradation is never labelled as full success.
if (result.status === 'partial') {
console.log('\nScan completed with gaps (partial). The reasons are listed below.');
for (const reason of result.partialReasons) {
console.log(` - ${reason.message}`);
}
// The reason above says a class of coverage degraded; these three name the sanitized
// agentic-SAST failure behind it, under the same labels every other surface uses.
if (result.agenticSast.status === 'failed') {
console.log(` Agentic SAST stopped at: ${result.agenticSast.failedStageLabel}`);
console.log(` What happened: ${result.agenticSast.error}`);
if (result.agenticSast.errorCode !== undefined) {
console.log(` Reference code (for a bug report): ${result.agenticSast.errorCode}`);
}
}
} else if (result.status === 'cancelled') {
console.log('\nScan cancelled before it finished.');
} else {
console.log('\nScan completed.');
}
if (result.summary) {
console.log(`Duration: ${Math.floor(result.summary.totalDurationMs / 1000)}s`);
console.log(`Agents completed: ${result.summary.agentCount}`);
console.log(`Agents resolved: ${result.summary.agentCount}`);
console.log(`Total turns: ${result.summary.totalTurns}`);
console.log(`Run cost: $${result.summary.totalCostUsd.toFixed(4)}`);
if (result.summary.usageAccountingComplete === false) {
console.log('Cost is incomplete — some background work is not included in this total.');
}
if (workspace.isResume) {
try {
@@ -369,39 +566,6 @@ async function waitForWorkflowResult(
}
}
// === Deliverables Copy ===
function copyDeliverables(repoPath: string, outputPath: string): void {
const outputDir = deliverablesDir(repoPath);
if (!fs.existsSync(outputDir)) {
console.log('No deliverables directory found, skipping copy');
return;
}
const files = fs.readdirSync(outputDir);
if (files.length === 0) {
console.log('No deliverables to copy');
return;
}
fs.mkdirSync(outputPath, { recursive: true });
for (const file of files) {
if (file === '.git') continue;
const src = path.join(outputDir, file);
const dest = path.join(outputPath, file);
fs.cpSync(src, dest, { recursive: true });
}
// Surface the report under its human-facing name alongside the raw deliverables
const assembledPdf = path.join(outputDir, ASSEMBLED_REPORT_PDF_FILENAME);
if (fs.existsSync(assembledPdf)) {
fs.copyFileSync(assembledPdf, path.join(outputPath, FINAL_REPORT_PDF_FILENAME));
}
console.log(`Copied ${files.length} deliverable(s) to ${outputPath}`);
}
// === Main Entry Point ===
async function run(): Promise<void> {
@@ -417,30 +581,40 @@ async function run(): Promise<void> {
const client = new Client({ connection: clientConnection });
try {
// 3. Bundle workflows and create worker on per-invocation task queue
// 3. Validate orchestration and resume state before terminating any workflow.
const orchestration = await loadOrchestrationConfig(args.configPath);
const workspace = await resolveWorkspace(client, args, orchestration.exploit ?? true);
// 4. Bundle workflows and create the worker with the collision-checked activity registry.
console.log('Preparing scan...');
const workflowBundle = await bundleWorkflowCode({
workflowsPath: path.join(__dirname, 'workflows.js'),
});
const productionActivities = createProductionActivityRegistry({
repositoryPath: args.repoPath,
webUrl: args.webUrl,
workspacesDir: path.resolve('./workspaces'),
});
// args.taskQueue is generated fresh per scan (see resolveWorkspace), so Temporal can only
// ever route this worker's activities to this scan's own container: an activity task from
// an older or unrelated scan can never execute against the repo mounted here.
const worker = await Worker.create({
connection,
namespace: 'default',
workflowBundle,
activities,
activities: productionActivities,
taskQueue: args.taskQueue,
maxConcurrentActivityTaskExecutions: 25,
});
// 4. Resolve workspace and build pipeline input
const workspace = await resolveWorkspace(client, args);
const orchestration = await loadOrchestrationConfig(args.configPath);
// 5. Build the fixed-scope pipeline input.
const input = buildPipelineInput(args, workspace, orchestration);
// 5. Start worker polling in the background
// 6. Start worker polling in the background.
const workerDone = worker.run();
// 6. Submit workflow to the same task queue
// 7. Submit workflow to the same task queue.
const handle = await client.workflow.start<(input: PipelineInput) => Promise<PipelineState>>(
'pentestPipelineWorkflow',
{
@@ -450,15 +624,10 @@ async function run(): Promise<void> {
},
);
// 7. Wait for workflow result
// 8. Wait for workflow result.
await waitForWorkflowResult(handle, workspace);
// 8. Copy deliverables to output directory
if (args.outputPath) {
copyDeliverables(args.repoPath, args.outputPath);
}
// 9. Shut down worker gracefully
// 9. Shut down worker gracefully. Final customer copies are workflow-owned.
worker.shutdown();
await workerDone;
} finally {
@@ -467,7 +636,10 @@ async function run(): Promise<void> {
}
}
run().catch((err) => {
console.error('Worker failed:', err);
process.exit(1);
});
const invokedPath = process.argv[1] ? path.resolve(process.argv[1]) : undefined;
if (invokedPath === fileURLToPath(import.meta.url)) {
run().catch((err) => {
console.error('Worker failed:', err);
process.exit(1);
});
}
apps/worker/src/temporal/workflow-errors.ts
+7 -1
View File
@@ -40,12 +40,18 @@ export function classifyErrorCode(error: unknown): ErrorCode | undefined {
return undefined;
}
/** Maps Temporal error type strings to actionable remediation hints. */
/**
* Maps Temporal error type strings to actionable remediation hints. A type earns an entry
* only when the reader has a next step to take; the rest print without a hint line.
*/
const REMEDIATION_HINTS: Record<string, string> = {
AuthenticationError: "Verify the selected provider's API key is valid and not expired.",
ConfigurationError: 'Check your CONFIG file path and contents.',
GitError: 'Check repository path and git state.',
InvalidTargetError: 'Verify the target URL is correct and accessible.',
IncompatibleWorkspaceError: 'start a new scan with a different -w name.',
WorkspaceNotFoundError: 'check the -w name against: shannon scans',
PipelineFailedError: 're-run the same -w to retry from the last checkpoint.',
};
/**
apps/worker/src/temporal/workflows.ts
+948 -455
View File
File diff suppressed because it is too large Load Diff
apps/worker/src/types/agents.ts
+4 -2
View File
@@ -25,6 +25,7 @@ export const ALL_AGENTS = [
'auth-exploit',
'ssrf-exploit',
'authz-exploit',
'miscellaneous-exploit',
'report',
] as const;
@@ -34,9 +35,10 @@ export const ALL_AGENTS = [
*/
export type AgentName = (typeof ALL_AGENTS)[number];
export type PlaywrightSession = 'agent1' | 'agent2' | 'agent3' | 'agent4' | 'agent5';
export type PlaywrightSession = 'agent1' | 'agent2' | 'agent3' | 'agent4' | 'agent5' | 'agent6';
import type { ActivityLogger } from './activity-logger.js';
import type { VulnClass } from './config.js';
export type AgentValidator = (sourceDir: string, logger: ActivityLogger) => Promise<boolean>;
@@ -53,7 +55,7 @@ export interface AgentDefinition {
/**
* Vulnerability types supported by the pipeline.
*/
export type VulnType = 'injection' | 'xss' | 'auth' | 'ssrf' | 'authz';
export type VulnType = VulnClass;
/**
* Decision returned by queue validation for exploitation phase.
apps/worker/src/types/config.ts
+7 -2
View File
@@ -67,11 +67,15 @@ export interface Authentication {
success_condition: SuccessCondition;
}
export interface AgenticSastConfig {
enabled: 'true' | 'false';
}
export interface Config {
rules?: Rules;
authentication?: Authentication;
description?: string;
vuln_classes?: VulnClass[];
agentic_sast?: AgenticSastConfig;
exploit?: 'true' | 'false';
report?: ReportConfig;
rules_of_engagement?: string;
@@ -85,7 +89,8 @@ export interface DistributedConfig {
focus: Rule[];
authentication: Authentication | null;
description: string;
vuln_classes: VulnClass[];
/** Present only when Capella is enabled. */
agenticSast?: true;
exploit: boolean;
report: DistributedReportConfig;
rules_of_engagement: string;
apps/worker/src/types/metrics.ts
+2
View File
@@ -18,6 +18,8 @@ export interface AgentMetrics {
costUsd: number | null;
numTurns: number | null;
model?: string | undefined;
/** Durable Git checkpoint associated with this result when one exists. */
checkpoint?: string;
// True when the checkpoint provider skipped the agent (resume path).
// Callers that perform post-agent work on collected state should short-circuit
// when this is set, since no fresh state was produced this run.
apps/worker/src/types/run-state.ts
+1104
View File
File diff suppressed because it is too large Load Diff
docs/configuration.md
+27 -3
View File
@@ -42,8 +42,11 @@ Source-build equivalent:
# Describe your target environment.
description: "Next.js e-commerce app on PostgreSQL. Local dev environment; .env files contain local-only credentials."
# Limit which vulnerability classes run end-to-end.
# vuln_classes: [injection, xss, auth, authz, ssrf]
# Every scan runs all five vulnerability classes.
# Agentic static analysis. `enabled` is its only setting.
# agentic_sast:
# enabled: "true"
# Skip the exploitation phase.
# exploit: "false"
@@ -102,6 +105,25 @@ rules:
# sarif: "false"
```
## Analysis Scope and Agentic SAST
Every scan runs all five analysis classes: Injection, Cross-Site Scripting, Authentication, Authorization, and
Server-Side Request Forgery. The class set is fixed and has no configuration selector.
Agentic static analysis is opt-in:
```yaml
agentic_sast:
enabled: "true"
```
`enabled` is the only setting. Omitting the block, or setting `enabled: "false"`, turns agentic static analysis off;
`"true"` turns it on. Either way, all five vulnerability classes still run.
Agentic static analysis reads the repository for vulnerabilities before the pentest and passes what it finds into the
exploitation phase. It adds model time and cost. If it fails, the pentest continues without its findings and the scan
finishes as "partial".
## Report Options
| Key | Effect |
@@ -122,7 +144,9 @@ report:
sarif: "false"
```
Each finding becomes one SARIF result, filed under a rule per vulnerability class (`shannon/injection`, `shannon/xss`, `shannon/auth`, `shannon/authz`, `shannon/ssrf`) and tagged with its OWASP Top Ten 2025 category. Results are anchored to the code location the analysis phase recorded, falling back to the HTTP entry point when the finding names no file. Severity maps onto SARIF's three levels: `critical` and `high` become `error`, `medium` becomes `warning`, everything else becomes `note`.
Each finding becomes one SARIF result, filed under a rule per vulnerability class (`shannon/injection`, `shannon/xss`, `shannon/auth`, `shannon/authz`, `shannon/ssrf`, and `shannon/other` for findings outside those classes) and tagged with its OWASP Top Ten 2025 category. Results are anchored to the code location the analysis phase recorded, falling back to the HTTP entry point when the finding names no file. Severity maps onto SARIF's three levels: `critical` and `high` become `error`, `medium` becomes `warning`, everything else becomes `note`.
If the SARIF log cannot be written, the JSON and Markdown reports are still produced and the scan finishes as "partial".
The log is written only for exploitative runs. `sarif` is ignored when `exploit` is `"false"`.
llms-full.txt
+24 -2
View File
@@ -510,8 +510,11 @@ Source-build equivalent:
# Describe your target environment.
description: "Next.js e-commerce app on PostgreSQL. Local dev environment; .env files contain local-only credentials."
# Limit which vulnerability classes run end-to-end.
# vuln_classes: [injection, xss, auth, authz, ssrf]
# Every scan runs all five vulnerability classes.
# Agentic static analysis. `enabled` is its only setting.
# agentic_sast:
# enabled: "true"
# Skip the exploitation phase.
# exploit: "false"
@@ -570,6 +573,25 @@ rules:
# sarif: "false"
```
## Analysis Scope and Agentic SAST
Every scan runs all five analysis classes: Injection, Cross-Site Scripting, Authentication, Authorization, and
Server-Side Request Forgery. The class set is fixed and has no configuration selector.
Agentic static analysis is opt-in:
```yaml
agentic_sast:
enabled: "true"
```
`enabled` is the only setting. Omitting the block, or setting `enabled: "false"`, turns agentic static analysis off;
`"true"` turns it on. Either way, all five vulnerability classes still run.
Agentic static analysis reads the repository for vulnerabilities before the pentest and passes what it finds into the
exploitation phase. It adds model time and cost. If it fails, the pentest continues without its findings and the scan
finishes as "partial".
## Report Options
| Key | Effect |