feat(worker): deduplicate static and runtime findings before exploitation

Parse Agentic SAST SARIF into typed observations, enrich and route those observations, and reconcile them with pentest findings before exploitation.

Publish deterministic exploitation queues with stable lineage, exact-path Git commits, retry-safe manifests, named drop reasons, and confined task formation. Reject duplicate producer IDs before commit and adopt either legal provenance shape after a lost acknowledgement.
This commit is contained in:
ajmallesh
2026-08-26 19:37:20 -07:00
parent 980607c602
commit c33132b0ab
52 changed files with 7914 additions and 84 deletions
@@ -0,0 +1,13 @@
@include(shared/exploitation/_sast-enrichment-procedure.txt)
These findings are authentication vulnerabilities.
CRITICAL RULES:
- exploitation_hypothesis must describe what an attacker ACHIEVES, not just confirm the vulnerability exists.
- suggested_exploit_technique must be an actionable attack the exploitation agent can execute against a live application.
- source_endpoint: infer the HTTP method and path from the code context (route definitions, handler functions).
- For hard-coded credentials (CWE-798): exploitation_hypothesis should specify using the found credentials.
- For CSRF (CWE-352): include the state-changing action that can be forged.
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
SAST FINDINGS:
@@ -0,0 +1,12 @@
@include(shared/exploitation/_sast-enrichment-procedure.txt)
These findings are authorization vulnerabilities.
CRITICAL RULES:
- Horizontal: same role accessing another user's data. Vertical: lower role accessing higher role's functions. Context_Workflow: bypassing a required step/state. Mass_Assignment: adding privileged fields (role, isAdmin, permissions) to request body that the server binds without filtering.
- If a proof-of-concept exists in the SAST data, use its inputs to craft a specific minimal_witness.
- guard_evidence must describe what's MISSING, not what exists.
- side_effect must be a concrete unauthorized action (e.g., "read other user's medical records"), not vague ("unauthorized access").
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
SAST FINDINGS:
@@ -0,0 +1,16 @@
@include(shared/exploitation/_sast-enrichment-procedure.txt)
These findings are SQL injection, command injection, path traversal, and related injection classes. Each finding must be transformed into a vulnerability object matching the schema.
CRITICAL RULES:
- witness_payload MUST be tailored to the actual sink code. If the sink is `db.query("SELECT * FROM users WHERE name LIKE '%" + input + "%'")`, use `%' OR '%'='` not a generic `' OR 1=1--`.
- slot_type MUST reflect the actual SQL/command/file context from the code snippet.
- If dataflow path is provided, use it to build an accurate `path` field.
- If sanitization functions appear in the path, list them in `sanitization_observed` and explain in `mismatch_reason` why they're insufficient.
- Set externally_exploitable=true only if the source is user-controlled input (HTTP params, headers, request body, cookies).
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
- For XML injection (CWE-91): slot_type is XML-element or XML-attribute depending on where user input lands in the XML structure.
- For prompt injection (CWE-1427): slot_type is PROMPT-instruction. witness_payload should demonstrate instruction override, not generic text.
- For prototype pollution (CWE-1321): slot_type is PROTO-property. witness_payload should use __proto__ or constructor.prototype paths specific to the sink.
SAST FINDINGS:
@@ -0,0 +1,14 @@
@include(shared/exploitation/_sast-enrichment-procedure.txt)
These findings are weaknesses that fall outside the injection, XSS, authentication, authorization and SSRF classes. They share no family: session lifetime, error-message disclosure, sensitive logging, cleartext storage, request forgery, redirection, framing, algorithmic complexity, race conditions.
CRITICAL RULES:
- vulnerability_type is the weakness's own name, taken from the CWE on the finding (e.g. 'Insecure Randomness', 'Use of Hard-coded Cryptographic Key'). There is no fixed list to pick from, and it must not be forced into another class's vocabulary.
- proof_criterion is the field the exploitation agent works from: state the concrete observation that would settle whether this specific weakness is real. These findings carry no per-class proof ladder, so an unusable criterion leaves the agent nothing to aim at.
- observable_signal must be something visible from outside the application, not a restatement of the source code.
- exploitation_hypothesis must describe what an attacker ACHIEVES, not just confirm the weakness exists.
- suggested_exploit_technique must be an actionable attack the exploitation agent can execute against a live application.
- cwe carries the id from the finding, e.g. CWE-330.
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
SAST FINDINGS:
@@ -0,0 +1,11 @@
@include(shared/exploitation/_sast-enrichment-procedure.txt)
These findings are Server-Side Request Forgery vulnerabilities.
CRITICAL RULES:
- vulnerability_type must match the sink pattern: HTTP client → URL_Manipulation, redirect function → Redirect_Abuse, webhook registration → Webhook_Injection.
- exploitation_hypothesis should reference likely internal targets (cloud metadata, internal APIs, admin panels) based on code context.
- suggested_exploit_technique must be actionable — the exploitation agent will actually attempt this against the live app.
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
SAST FINDINGS:
@@ -0,0 +1,11 @@
@include(shared/exploitation/_sast-enrichment-procedure.txt)
These findings are Cross-Site Scripting vulnerabilities.
CRITICAL RULES:
- Determine vulnerability_type from the source: HTTP request param → Reflected, database read → Stored, client-side only → DOM-based.
- render_context MUST be inferred from the actual sink code. `innerHTML` → HTML_BODY, `setAttribute('href', ...)` → HTML_ATTRIBUTE, template literal in <script> → JAVASCRIPT_STRING.
- witness_payload MUST match the render_context. HTML_ATTRIBUTE context requires attribute-breaking payloads, not tag injection.
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
SAST FINDINGS:
@@ -0,0 +1,7 @@
You are a security engineer preparing an exploitation queue for a penetration testing agent.
You are given SAST findings as JSON. Generate the exploitation queue and return it by calling the `submit_result` tool exactly once as your final action. Do NOT output the result as JSON text — fill every required parameter of the tool and let it carry the field shapes. The tool call is your final action; submit all vulnerability objects in that one call.
`_sastId` MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
Every vulnerability object MUST carry these common fields: `ID`, `vulnerability_type`, `externally_exploitable`, `confidence` (high | med | low), and `notes`. All other fields are class-specific and follow the class rules below.
@@ -0,0 +1,29 @@
<input_format>
The user message supplies one JSON object with `queued_findings`. Every element has an opaque four-lowercase-consonant `label` and a positive observation `entry`. The entry contains only ordinary class evidence, `scan_source`, optional `priority`, and an allowed SAST source location. Labels have no order or meaning beyond this call.
These are current, unproven observations from vulnerability analysis and optional static analysis. Do not infer a prior scan, canonical finding, stable task ID, producer ID, or hidden identity.
</input_format>
<task>
Return groups of observations that reduce to the same independently testable exploit hypothesis. One investigation must be able to settle every observation in a group through one exploitation attempt and one verdict.
A shared CWE, file, line, endpoint, operation, helper, impact, or fix is supporting evidence, not proof. Keep observations separate when different inputs, preconditions, controls, operations, resources, or effects could produce different verdicts. Read the source at `{{REPO_PATH}}` when it settles whether the observations describe the same path. When the evidence is balanced, leave them separate.
Every observation belongs to at most one group. A group has at least two distinct supplied labels. Observations omitted from all groups remain singleton tasks; do not submit singleton groups.
</task>
<method>
1. Read the complete observation list before grouping.
2. State the single exploit hypothesis and proof that would settle each proposed group.
3. Check every member against that same proof and verdict; remove any member that needs a materially different test.
4. Use the jailed source only when needed. Do not look for hidden IDs or prior state.
5. Submit only groups you can justify. An empty groups array is valid and common.
</method>
<cost_of_error>
A false merge can hide a real vulnerability. A missed merge leaves a visible duplicate. Prefer separate observations whenever one proof does not clearly settle the full group.
</cost_of_error>
<output>
Call `submit_result` with exactly one object containing `groups` and no other fields. Each group contains only `queue_labels` and nonblank `reasoning`. `queue_labels` contains at least two distinct supplied labels, and no label appears in more than one group. If the tool rejects the submission, correct it and call again; stop after the first accepted submission. Do not output JSON as text.
</output>
@@ -0,0 +1,11 @@
<role>
You are an Authentication Findings Reconciliation Specialist. Decide which current authentication observations predict the same exploitation attempt and verdict.
</role>
<class_boundary>
One task is one failure in a credential, token, or session mechanism producing one security outcome. Split different mechanisms, failure modes, or outcomes.
Read `vulnerable_code_location` and `source_endpoint` as context for the mechanism, and `missing_defense` as the failure that must be proven. `exploitation_hypothesis` and `suggested_exploit_technique` are proposals, not identity. A shared helper, CWE, file, line, endpoint, impact, or fix is supporting evidence only. Group only when one proof of one mechanism failure would settle every observation with one outcome and verdict.
</class_boundary>
@include(shared/exploitation/_task-formation-procedure.txt)
@@ -0,0 +1,11 @@
<role>
You are an Authorization Findings Reconciliation Specialist. Decide which current authorization observations predict the same exploitation attempt and verdict.
</role>
<class_boundary>
One task is one principal performing one protected operation on one resource or relationship past one ineffective check. Split different principals, operations, resources, relationships, or checks.
Read `endpoint` and `vulnerable_code_location` as the protected operation, `role_context` as the principal, and `guard_evidence` as the ineffective check. Use `side_effect` and `minimal_witness` as supporting evidence. A shared route, middleware, CWE, file, line, impact, or fix is not proof of one task. Group only when one authorization proof would settle the same principal, operation, resource or relationship, and check with one verdict.
</class_boundary>
@include(shared/exploitation/_task-formation-procedure.txt)
@@ -0,0 +1,11 @@
<role>
You are an Injection Findings Reconciliation Specialist. Decide which current injection observations predict the same exploitation attempt and verdict.
</role>
<class_boundary>
One task is one attacker-controlled input reaching one dangerous operation in one injection context. Split independently controlled inputs, different contexts, or materially different defenses.
Read `source`, `combined_sources`, `path`, and `sink_call` as one data flow. Use `slot_type` and `sanitization_observed` to distinguish the injection context and its defense. A shared sink, CWE, file, line, payload, impact, or fix is supporting evidence only. Group only when one proof against one controlled input and dangerous operation would settle every observation with one verdict.
</class_boundary>
@include(shared/exploitation/_task-formation-procedure.txt)
@@ -0,0 +1,11 @@
<role>
You are a Generalist Findings Reconciliation Specialist. Decide which current observations in the internal miscellaneous class predict the same exploitation attempt and verdict.
</role>
<class_boundary>
One task is one attacker-controlled input or state driving one target operation to one security effect. A shared unsupported CWE does not justify a merge; require the same independently testable path and verdict.
This class spans unrelated weakness families. Read `vulnerable_code_location` and `source_endpoint` as context, `missing_defense` as the defect, and `observable_signal` and `proof_criterion` as the proof that must settle it. `exploitation_hypothesis` and `suggested_exploit_technique` are proposals, not identity. A shared CWE, rule, file, line, helper, impact, or fix is supporting evidence only.
</class_boundary>
@include(shared/exploitation/_task-formation-procedure.txt)
@@ -0,0 +1,11 @@
<role>
You are a Server-Side Request Forgery Findings Reconciliation Specialist. Decide which current SSRF observations predict the same exploitation attempt and verdict.
</role>
<class_boundary>
One task is one attacker-controlled input steering one outbound request operation. Split different controlled inputs, entry paths, controls, or outbound operations.
Read `source_endpoint` and `vulnerable_parameter` as the controlled entry path, and `vulnerable_code_location` as the outbound operation. Use `missing_defense` to distinguish the control being tested. `exploitation_hypothesis` and `suggested_exploit_technique` are proposals, not identity. A shared client helper, destination, CWE, file, line, impact, or fix is supporting evidence only.
</class_boundary>
@include(shared/exploitation/_task-formation-procedure.txt)
@@ -0,0 +1,11 @@
<role>
You are a Cross-Site Scripting Findings Reconciliation Specialist. Decide which current XSS observations predict the same exploitation attempt and verdict.
</role>
<class_boundary>
One task is one attacker-influenced value reaching one browser render context. Split different values, contexts, or trigger conditions.
Read `source`, `source_detail`, `path`, and `sink_function` as one content flow. Use `render_context` and `encoding_observed` to determine the browser context and the defense. Stored input and its later rendering can be two ends of one task, but two values or render contexts remain separate when one proof would not settle both. A shared component, route, sanitizer, CWE, file, line, payload, impact, or fix is supporting evidence only.
</class_boundary>
@include(shared/exploitation/_task-formation-procedure.txt)
+279
View File
@@ -0,0 +1,279 @@
// Copyright (C) 2025 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Attempt-local working-tree copy used by the task-formation model boundary. */
import type { Dirent, Stats } from 'node:fs';
import { cp, lstat, mkdir, mkdtemp, readdir, realpath, rm } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { ArtifactIntegrityError, ReconciliationIoError } from '../reconciliation/artifact-store.js';
const JAIL_PREFIX = 'shannon-task-formation-';
const ALWAYS_EXCLUDED_NAMES = Object.freeze(['.git', '.shannon', '.pi'] as const);
export interface SourceJailOptions {
readonly sourceRoot: string;
readonly deliverablesPath: string;
readonly reconciliationWorkspacePath: string;
readonly signal?: AbortSignal;
/** Test-only filesystem selector. Production uses `os.tmpdir()`. */
readonly tempRoot?: string;
}
/** One source-only jail plus the immutable deny rules used by its live tool gate. */
export interface SourceJail {
readonly dir: string;
readonly deniedPaths: readonly string[];
cleanup(): Promise<void>;
}
function isErrno(error: unknown, code: string): boolean {
return error instanceof Error && (error as NodeJS.ErrnoException).code === code;
}
function cancellationError(signal: AbortSignal): Error {
if (signal.reason instanceof Error) return signal.reason;
return new DOMException('Task formation was cancelled.', 'AbortError');
}
function checkCancellation(signal: AbortSignal | undefined): void {
if (signal?.aborted === true) throw cancellationError(signal);
}
function isWithin(root: string, candidate: string): boolean {
const relativePath = path.relative(root, candidate);
return (
relativePath === '' ||
(!relativePath.startsWith(`..${path.sep}`) && relativePath !== '..' && !path.isAbsolute(relativePath))
);
}
async function relativeExclusion(
sourceRoot: string,
lexicalSourceRoot: string,
candidate: string,
): Promise<string | undefined> {
const resolved = path.resolve(candidate);
let relativePath: string | undefined;
if (isWithin(sourceRoot, resolved)) {
relativePath = path.relative(sourceRoot, resolved);
} else if (isWithin(lexicalSourceRoot, resolved)) {
relativePath = path.relative(lexicalSourceRoot, resolved);
} else {
try {
const canonicalCandidate = await realpath(resolved);
if (isWithin(sourceRoot, canonicalCandidate)) {
relativePath = path.relative(sourceRoot, canonicalCandidate);
}
} catch {
return undefined;
}
}
if (relativePath === undefined) return undefined;
if (relativePath === '') {
throw new ArtifactIntegrityError('A task-formation exclusion resolves to the complete source root');
}
return relativePath;
}
async function buildDynamicExclusions(
options: SourceJailOptions,
sourceRoot: string,
lexicalSourceRoot: string,
): Promise<readonly string[]> {
const exclusions = (
await Promise.all([
relativeExclusion(sourceRoot, lexicalSourceRoot, options.deliverablesPath),
relativeExclusion(sourceRoot, lexicalSourceRoot, options.reconciliationWorkspacePath),
])
).filter((value): value is string => value !== undefined);
return Object.freeze([...new Set(exclusions)]);
}
function pathHasAlwaysExcludedName(relativePath: string): boolean {
const segments = relativePath.split(path.sep);
return segments.some((segment) => (ALWAYS_EXCLUDED_NAMES as readonly string[]).includes(segment));
}
function pathIsDynamicallyExcluded(relativePath: string, exclusions: readonly string[]): boolean {
return exclusions.some((excluded) => relativePath === excluded || relativePath.startsWith(`${excluded}${path.sep}`));
}
async function copySourceTree(
sourceRoot: string,
destination: string,
dynamicExclusions: readonly string[],
signal: AbortSignal | undefined,
): Promise<void> {
let entries: Dirent[];
try {
entries = (await readdir(sourceRoot, { withFileTypes: true })).sort((left, right) =>
left.name.localeCompare(right.name),
);
} catch {
throw new ReconciliationIoError('Unable to enumerate the task-formation source tree');
}
for (const entry of entries) {
checkCancellation(signal);
const source = path.join(sourceRoot, entry.name);
const destinationEntry = path.join(destination, entry.name);
try {
await cp(source, destinationEntry, {
recursive: true,
verbatimSymlinks: true,
errorOnExist: true,
force: false,
async filter(candidate) {
checkCancellation(signal);
const relativePath = path.relative(sourceRoot, candidate);
if (relativePath === '' || !isWithin(sourceRoot, path.resolve(candidate))) return false;
if (pathHasAlwaysExcludedName(relativePath)) return false;
return !pathIsDynamicallyExcluded(relativePath, dynamicExclusions);
},
});
} catch (error) {
if (signal?.aborted === true) throw cancellationError(signal);
if (error instanceof ArtifactIntegrityError) throw error;
throw new ReconciliationIoError('Unable to copy the task-formation source tree');
}
}
checkCancellation(signal);
}
async function assertAlwaysExcludedNamesAbsent(directory: string, signal: AbortSignal | undefined): Promise<void> {
checkCancellation(signal);
let entries: Dirent[];
try {
entries = await readdir(directory, { withFileTypes: true });
} catch {
throw new ReconciliationIoError('Unable to verify the task-formation source jail');
}
for (const entry of entries) {
checkCancellation(signal);
if ((ALWAYS_EXCLUDED_NAMES as readonly string[]).includes(entry.name)) {
throw new ArtifactIntegrityError('The task-formation source jail contains an excluded entry');
}
if (entry.isDirectory() && !entry.isSymbolicLink()) {
await assertAlwaysExcludedNamesAbsent(path.join(directory, entry.name), signal);
}
}
}
async function assertDynamicExclusionsAbsent(
directory: string,
exclusions: readonly string[],
signal: AbortSignal | undefined,
): Promise<void> {
for (const excluded of exclusions) {
checkCancellation(signal);
try {
await lstat(path.join(directory, excluded));
} catch (error) {
if (isErrno(error, 'ENOENT')) continue;
throw new ReconciliationIoError('Unable to verify a task-formation jail exclusion');
}
throw new ArtifactIntegrityError('The task-formation source jail contains a protected workspace entry');
}
}
async function verifyJail(
directory: string,
dynamicExclusions: readonly string[],
signal: AbortSignal | undefined,
): Promise<void> {
checkCancellation(signal);
let stats: Stats;
try {
stats = await lstat(directory);
} catch {
throw new ReconciliationIoError('Unable to inspect the task-formation source jail');
}
if (stats.isSymbolicLink() || !stats.isDirectory()) {
throw new ArtifactIntegrityError('The task-formation source jail is not a real directory');
}
await assertAlwaysExcludedNamesAbsent(directory, signal);
await assertDynamicExclusionsAbsent(directory, dynamicExclusions, signal);
checkCancellation(signal);
}
async function removeJail(directory: string): Promise<void> {
try {
await rm(directory, { recursive: true, force: true });
} catch {
throw new ReconciliationIoError('Unable to remove the task-formation source jail');
}
try {
await lstat(directory);
} catch (error) {
if (isErrno(error, 'ENOENT')) return;
throw new ReconciliationIoError('Unable to verify task-formation source-jail cleanup');
}
throw new ReconciliationIoError('Task-formation source-jail cleanup left the jail on disk');
}
/**
* Copy the scanned working tree into an isolated temporary directory without following symlinks.
* Every failure removes the attempt-local directory before it propagates.
*/
export async function materializeSourceJail(options: SourceJailOptions): Promise<SourceJail> {
checkCancellation(options.signal);
const lexicalSourceRoot = path.resolve(options.sourceRoot);
let sourceRoot: string;
try {
sourceRoot = await realpath(options.sourceRoot);
const sourceStats = await lstat(sourceRoot);
if (sourceStats.isSymbolicLink() || !sourceStats.isDirectory()) {
throw new ArtifactIntegrityError('The task-formation source root is not a real directory');
}
} catch (error) {
if (error instanceof ArtifactIntegrityError) throw error;
throw new ReconciliationIoError('Unable to resolve the task-formation source root');
}
let tempRoot: string;
try {
const configuredTempRoot = options.tempRoot ?? os.tmpdir();
await mkdir(configuredTempRoot, { recursive: true });
tempRoot = await realpath(configuredTempRoot);
} catch {
throw new ReconciliationIoError('Unable to resolve the task-formation temporary root');
}
if (isWithin(sourceRoot, tempRoot)) {
throw new ArtifactIntegrityError('The task-formation temporary root cannot be inside the source tree');
}
const dynamicExclusions = await buildDynamicExclusions(options, sourceRoot, lexicalSourceRoot);
let directory: string;
try {
directory = await mkdtemp(path.join(tempRoot, JAIL_PREFIX));
} catch {
throw new ReconciliationIoError('Unable to create the task-formation source jail');
}
let cleaned = false;
const cleanup = async (): Promise<void> => {
if (cleaned) return;
await removeJail(directory);
cleaned = true;
};
try {
await copySourceTree(sourceRoot, directory, dynamicExclusions, options.signal);
await verifyJail(directory, dynamicExclusions, options.signal);
} catch (error) {
await cleanup().catch(() => undefined);
throw error;
}
const deniedPaths = Object.freeze([...ALWAYS_EXCLUDED_NAMES, ...dynamicExclusions]);
return Object.freeze({ dir: directory, deniedPaths, cleanup });
}
+25 -6
View File
@@ -16,9 +16,19 @@ import { type CapturedSubmitTool, createGenericSubmitTool } from '../submit-tool
const ZERO_USAGE = { inputTokens: 0, outputTokens: 0, costUsd: 0 } as const;
function isAbort(error: unknown, signal: AbortSignal | undefined): boolean {
const abortError = error instanceof Error && (error.name === 'AbortError' || error.name === 'TimeoutError');
return signal?.aborted === true || abortError;
function isSignalCancellation(error: unknown, signal: AbortSignal | undefined): boolean {
if (signal?.aborted !== true) return false;
let current: unknown = error;
const seen = new Set<unknown>();
for (let depth = 0; depth < 8 && current !== undefined && current !== null && !seen.has(current); depth++) {
if (current === signal.reason) return true;
seen.add(current);
const errorName = current instanceof Error ? current.name : undefined;
if (errorName === 'AbortError' || errorName === 'CancelledFailure') return true;
current = current instanceof Error ? current.cause : undefined;
}
return false;
}
function responseUsage(response: AssistantMessage): StructuredGenerationResult['usage'] {
@@ -50,7 +60,6 @@ async function captureSingleValidSubmission(
}
async function generate(host: ModelHost, request: StructuredGenerationRequest): Promise<StructuredGenerationResult> {
const selection = await host.resolve('small');
const submitTool = createGenericSubmitTool(request.tool.parametersJsonSchema);
const context: Context = {
...(request.systemPrompt !== undefined && { systemPrompt: request.systemPrompt }),
@@ -66,6 +75,7 @@ async function generate(host: ModelHost, request: StructuredGenerationRequest):
let response: AssistantMessage;
try {
const selection = await host.resolve('small');
// One enrichment batch is one billable provider request. Temporal owns any
// retry after this boundary, so provider-level retries stay disabled here.
response = await selection.modelRuntime.completeSimple(selection.model, context, {
@@ -74,7 +84,7 @@ async function generate(host: ModelHost, request: StructuredGenerationRequest):
...(request.signal !== undefined && { signal: request.signal }),
});
} catch (error) {
if (isAbort(error, request.signal)) {
if (isSignalCancellation(error, request.signal)) {
return { stopReason: 'aborted', toolCalls: [], usage: ZERO_USAGE };
}
const failure = host.classify(error);
@@ -96,7 +106,16 @@ async function generate(host: ModelHost, request: StructuredGenerationRequest):
};
}
if (response.stopReason === 'aborted') {
return { stopReason: 'aborted', toolCalls: [], usage: responseUsage(response) };
if (request.signal?.aborted === true) {
return { stopReason: 'aborted', toolCalls: [], usage: responseUsage(response) };
}
const failure = host.classify(response);
return {
stopReason: 'error',
toolCalls: [],
usage: responseUsage(response),
errorMessage: `${failure.type}: ${failure.message}`,
};
}
const toolCalls = response.content.filter((block): block is ToolCall => block.type === 'toolCall');
@@ -0,0 +1,786 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Dedicated read-only Pi executor and live tool policy for Pass 1 task formation. */
import { randomUUID } from 'node:crypto';
import path from 'node:path';
import type { AgentMessage } from '@earendil-works/pi-agent-core';
import {
type AgentSession,
type AgentSessionEvent,
createAgentSession,
DefaultResourceLoader,
defineTool,
getAgentDir,
SessionManager,
SettingsManager,
type ToolDefinition,
} from '@earendil-works/pi-coding-agent';
import { Type } from 'typebox';
import type { ProviderFailure } from '../../types/errors.js';
import { type ModelHost, modelHost } from '../model-host.js';
import type { ModelSelection } from '../models.js';
import type { ValidatingSubmitTool } from '../reconciliation/submit-validation.js';
import { ConfinementError, compileRepositoryGlob, RepositoryConfinement } from '../sast/capella/tools/confinement.js';
import { createCapellaRepositoryTools } from '../sast/capella/tools/repository-tools.js';
import { PI_RETRY_SETTINGS } from './retry-settings.js';
const DEFAULT_TIMEOUT_MS = 30 * 60 * 1_000;
const DEFAULT_MAX_TURNS = 64;
const MAX_TIMEOUT_MS = 30 * 60 * 1_000;
const MAX_TURNS = 128;
const MAX_LIST_RESULTS = 500;
const DEFAULT_LIST_RESULTS = 200;
const MAX_OUTPUT_BYTES = 64 * 1024;
// The live-tool-side counterpart of the source jail's copy-time exclusion (source-jail.ts): even if
// one of these somehow existed in the jailed tree, the read/grep/find/ls/glob tools built below must
// still refuse to serve it. `.git` is deliverables history, `.shannon` is scan internals, `.pi` is
// provider credentials.
const ALWAYS_DENIED_PATHS = Object.freeze(['.git', '.shannon', '.pi'] as const);
const TRANSIENT_IO_CODES = new Set([
'EAGAIN',
'EBUSY',
'ECONNREFUSED',
'ECONNRESET',
'EIO',
'EMFILE',
'ENFILE',
'ENOMEM',
'ENOSPC',
'EPIPE',
'EROFS',
'ETIMEDOUT',
]);
export const TASK_FORMATION_TOOL_NAMES = Object.freeze([
'read',
'grep',
'find',
'ls',
'glob',
'submit_result',
] as const);
// The closed set of failure reasons the integration layer accepts as grounds to fall back to a
// single-agent formation. Only a failure carrying one of these becomes a fallback; any other
// failure propagates. Keep this in sync with the reasons the Temporal caller recognizes.
export const TASK_FORMATION_FALLBACK_REASONS = Object.freeze([
'retryable_model_failure',
'missing_accepted_submission',
'model_stage_timeout',
] as const);
export type TaskFormationFallbackReason = (typeof TASK_FORMATION_FALLBACK_REASONS)[number];
export type TaskFormationExecutorFailureKind = 'model' | 'input' | 'confinement' | 'infrastructure';
/** Safe, bounded fields supplied by the activity wrapper for per-attempt executor correlation. */
export interface TaskFormationExecutionContext {
readonly executionKey?: string;
readonly attempt?: number;
readonly stage?: string;
readonly vulnerabilityClass?: string;
}
export interface TaskFormationUsage {
readonly costUsd: number;
readonly inputTokens: number;
readonly outputTokens: number;
}
export class TaskFormationExecutorError extends Error {
override readonly name = 'TaskFormationExecutorError';
readonly code: string;
readonly retryable: boolean;
readonly failureKind: TaskFormationExecutorFailureKind;
readonly fallbackReason: TaskFormationFallbackReason | undefined;
readonly usage: TaskFormationUsage;
readonly modelCalls: number;
constructor(options: {
code: string;
message: string;
retryable: boolean;
failureKind: TaskFormationExecutorFailureKind;
fallbackReason?: TaskFormationFallbackReason;
usage?: TaskFormationUsage;
modelCalls?: number;
}) {
super(options.message);
this.code = options.code;
this.retryable = options.retryable;
this.failureKind = options.failureKind;
this.fallbackReason = options.fallbackReason;
this.usage = options.usage ?? zeroUsage();
this.modelCalls = options.modelCalls ?? 0;
}
}
export interface TaskFormationExecutorRequest {
readonly cwd: string;
readonly systemPrompt: string;
readonly modelContext: string;
readonly deniedPaths: readonly string[];
readonly submitTool: ValidatingSubmitTool;
readonly signal: AbortSignal;
readonly timeoutMs?: number;
readonly maxTurns?: number;
readonly correlation?: TaskFormationExecutionContext;
}
export interface TaskFormationExecutorResult {
readonly output: unknown;
readonly usage: TaskFormationUsage;
readonly providerId: string;
readonly modelId: string;
readonly modelCalls: 1;
readonly registeredTools: readonly string[];
}
export interface TaskFormationExecutor {
run(request: TaskFormationExecutorRequest): Promise<TaskFormationExecutorResult>;
}
interface SessionOutcome {
readonly pendingProviderError: unknown;
readonly promptError: unknown;
readonly usage: TaskFormationUsage;
}
interface ToolFactoryOptions {
readonly cwd: string;
readonly deniedPaths: readonly string[];
}
function zeroUsage(): TaskFormationUsage {
return { costUsd: 0, inputTokens: 0, outputTokens: 0 };
}
/** Reject unknown values from Temporal failure details instead of widening semantic fallback. */
export function isTaskFormationFallbackReason(value: unknown): value is TaskFormationFallbackReason {
return (TASK_FORMATION_FALLBACK_REASONS as readonly unknown[]).includes(value);
}
function errorCode(error: unknown): string | undefined {
if (typeof error !== 'object' || error === null || !('code' in error)) return undefined;
return typeof error.code === 'string' ? error.code : undefined;
}
function isTransientIoFailure(error: unknown): boolean {
const code = errorCode(error);
if (code !== undefined && TRANSIENT_IO_CODES.has(code)) return true;
if (error instanceof Error && error.cause !== undefined) return isTransientIoFailure(error.cause);
return false;
}
function safeIdentifier(value: string | undefined): string | undefined {
if (value === undefined || !/^[A-Za-z0-9._:-]{1,128}$/u.test(value)) return undefined;
return value;
}
// Emit only bounded, format-checked correlation fields. Prompt text, model context, and source
// content never enter the log line. An unsafe or missing identifier falls back to a synthetic one
// rather than logging the caller's raw value.
function executionLogContext(context: TaskFormationExecutionContext | undefined): Readonly<Record<string, unknown>> {
const attempt = context?.attempt;
return Object.freeze({
executionKey: safeIdentifier(context?.executionKey) ?? randomUUID(),
attempt: Number.isSafeInteger(attempt) && (attempt ?? 0) > 0 ? attempt : null,
stage: safeIdentifier(context?.stage) ?? 'task-formation',
class: safeIdentifier(context?.vulnerabilityClass) ?? 'unknown',
});
}
function finiteNonNegative(value: number): number {
return Number.isFinite(value) ? Math.max(0, value) : 0;
}
function sessionUsage(session: AgentSession): TaskFormationUsage {
const stats = session.getSessionStats();
return {
costUsd: finiteNonNegative(stats.cost),
inputTokens: finiteNonNegative(stats.tokens.input),
outputTokens: finiteNonNegative(stats.tokens.output),
};
}
function boundedText(value: string): string {
const bytes = Buffer.from(value, 'utf8');
if (bytes.byteLength <= MAX_OUTPUT_BYTES) return value;
return bytes.subarray(0, MAX_OUTPUT_BYTES).toString('utf8');
}
function uniqueDeniedPaths(deniedPaths: readonly string[]): readonly string[] {
return Object.freeze([...new Set([...ALWAYS_DENIED_PATHS, ...deniedPaths])]);
}
// The session must register exactly the allowlisted tools. This is checked against the built tool
// set and again against the live session's registered tools, so an injected or dropped tool fails
// the session closed before the model runs.
function hasExactToolSet(toolNames: readonly string[]): boolean {
const expected = [...TASK_FORMATION_TOOL_NAMES].sort();
const actual = [...toolNames].sort();
return actual.length === expected.length && actual.every((name, index) => name === expected[index]);
}
function createListTool(confinement: RepositoryConfinement): ToolDefinition {
return defineTool({
name: 'ls',
label: 'List source directory',
description: 'List bounded repository-relative entries without following symlinks.',
promptSnippet: 'ls: list entries below one source directory',
promptGuidelines: ['Use a repository-relative directory. Absolute paths and traversal are rejected.'],
parameters: Type.Object(
{
path: Type.Optional(Type.String({ minLength: 1, maxLength: 1_024 })),
limit: Type.Optional(Type.Integer({ minimum: 1, maximum: MAX_LIST_RESULTS })),
},
{ additionalProperties: false },
),
async execute(_toolCallId, parameters, signal) {
const requestedPath = parameters.path ?? '.';
const budget = confinement.createBudget(signal);
const searchRoot = await confinement.resolveExisting(requestedPath, true, budget);
const entries = await confinement.enumerate(requestedPath, signal, budget);
const names = new Set<string>();
for (const entry of entries) {
confinement.checkBudget(budget);
const relativePath = pathRelative(searchRoot, entry.absolutePath);
const [first, ...remaining] = relativePath.split('/');
if (first) names.add(remaining.length > 0 ? `${first}/` : first);
}
const limit = parameters.limit ?? DEFAULT_LIST_RESULTS;
const output = [...names].sort().slice(0, limit);
return {
content: [{ type: 'text' as const, text: boundedText(output.join('\n') || 'No entries found.') }],
details: { count: output.length, truncated: names.size > output.length },
};
},
});
}
function pathRelative(root: string, candidate: string): string {
const relativePath = path.relative(root, candidate);
if (
!relativePath ||
relativePath.startsWith(`..${path.sep}`) ||
relativePath === '..' ||
path.isAbsolute(relativePath)
) {
throw new TaskFormationExecutorError({
code: 'TOOL_PATH_RACE',
message: 'Task-formation source path changed during access.',
retryable: false,
failureKind: 'confinement',
});
}
return relativePath.split(path.sep).join('/');
}
function createGlobTool(confinement: RepositoryConfinement): ToolDefinition {
return defineTool({
name: 'glob',
label: 'Glob source files',
description: 'Match bounded file globs from the source-jail root without following symlinks.',
promptSnippet: 'glob: match source files from the jail root',
promptGuidelines: ['Patterns are always rooted in the source jail.'],
parameters: Type.Object(
{
pattern: Type.String({ minLength: 1, maxLength: 256 }),
limit: Type.Optional(Type.Integer({ minimum: 1, maximum: MAX_LIST_RESULTS })),
},
{ additionalProperties: false },
),
async execute(_toolCallId, parameters, signal) {
const budget = confinement.createBudget(signal);
const matcher = compileRepositoryGlob(parameters.pattern);
const entries = await confinement.enumerate('.', signal, budget);
const limit = parameters.limit ?? DEFAULT_LIST_RESULTS;
const matches: string[] = [];
let truncated = false;
for (const entry of entries) {
confinement.checkBudget(budget);
if (!matcher.test(entry.path)) continue;
if (matches.length >= limit) {
truncated = true;
break;
}
matches.push(entry.path);
}
return {
content: [{ type: 'text' as const, text: boundedText(matches.join('\n') || 'No files found.') }],
details: { count: matches.length, truncated },
};
},
});
}
/** Create the five code-owned source tools that share one canonical jail policy. */
export async function createTaskFormationSourceTools(options: ToolFactoryOptions): Promise<readonly ToolDefinition[]> {
const deniedPaths = uniqueDeniedPaths(options.deniedPaths);
const capellaTools = await createCapellaRepositoryTools({
repositoryRoot: options.cwd,
deniedPaths,
});
const confinement = await RepositoryConfinement.create({
repositoryRoot: options.cwd,
deniedPaths,
});
const byName = new Map(capellaTools.map((tool) => [tool.name, tool]));
const tools = [
byName.get('read'),
byName.get('grep'),
byName.get('find'),
createListTool(confinement),
createGlobTool(confinement),
];
if (tools.some((tool) => tool === undefined)) {
throw new TaskFormationExecutorError({
code: 'TOOL_FACTORY_MISMATCH',
message: 'Task-formation source tool factory returned an incomplete set.',
retryable: false,
failureKind: 'confinement',
});
}
return Object.freeze(tools as ToolDefinition[]);
}
function cancellationError(signal: AbortSignal): Error {
if (signal.reason instanceof Error) return signal.reason;
return new DOMException('Task formation was cancelled.', 'AbortError');
}
function raceWithAbort<T>(promise: Promise<T>, signal: AbortSignal): Promise<T> {
if (signal.aborted) return Promise.reject(cancellationError(signal));
return new Promise<T>((resolve, reject) => {
const onAbort = (): void => reject(cancellationError(signal));
signal.addEventListener('abort', onAbort, { once: true });
promise.then(
(value) => {
signal.removeEventListener('abort', onAbort);
resolve(value);
},
(error: unknown) => {
signal.removeEventListener('abort', onAbort);
reject(error);
},
);
});
}
function validateRequest(request: TaskFormationExecutorRequest): { timeoutMs: number; maxTurns: number } {
const timeoutMs = request.timeoutMs ?? DEFAULT_TIMEOUT_MS;
const maxTurns = request.maxTurns ?? DEFAULT_MAX_TURNS;
if (!request.cwd || !request.systemPrompt || !request.modelContext) {
throw new TaskFormationExecutorError({
code: 'INVALID_REQUEST',
message: 'Task-formation executor input is incomplete.',
retryable: false,
failureKind: 'input',
});
}
if (!Number.isInteger(timeoutMs) || timeoutMs < 1 || timeoutMs > MAX_TIMEOUT_MS) {
throw new TaskFormationExecutorError({
code: 'INVALID_TIMEOUT',
message: `Task-formation timeout must be a positive integer no greater than ${MAX_TIMEOUT_MS} milliseconds.`,
retryable: false,
failureKind: 'input',
});
}
if (!Number.isInteger(maxTurns) || maxTurns < 1 || maxTurns > MAX_TURNS) {
throw new TaskFormationExecutorError({
code: 'INVALID_TURN_LIMIT',
message: 'Task-formation turn limit is outside its bounded range.',
retryable: false,
failureKind: 'input',
});
}
return { timeoutMs, maxTurns };
}
function isAbortLike(error: unknown): boolean {
return error instanceof Error && (error.name === 'AbortError' || error.name === 'TimeoutError');
}
function classifyModelFailure(host: ModelHost, error: unknown): ProviderFailure {
if (isAbortLike(error)) {
return {
type: 'AgentExecutionError',
category: 'transport',
retryable: true,
message: 'The provider request ended before task formation completed.',
};
}
return host.classify(error);
}
function executorLog(level: 'info' | 'warn', fields: Readonly<Record<string, unknown>>): void {
console[level](JSON.stringify({ component: 'task-formation-executor', ...fields }));
}
class StandaloneTaskFormationExecutor implements TaskFormationExecutor {
private readonly host: ModelHost;
constructor(host: ModelHost) {
this.host = host;
}
async run(request: TaskFormationExecutorRequest): Promise<TaskFormationExecutorResult> {
const logContext = executionLogContext(request.correlation);
let timeoutMs: number;
let maxTurns: number;
try {
({ timeoutMs, maxTurns } = validateRequest(request));
} catch (error) {
const failure = this.normalizeFailure(error);
executorLog('warn', {
...logContext,
event: 'finished',
outcome: 'failed',
code: failure.code,
failureKind: failure.failureKind,
retryable: failure.retryable,
});
throw failure;
}
const controller = new AbortController();
let termination: 'cancellation' | 'timeout' | 'turn-limit' | undefined;
let session: AgentSession | undefined;
let unsubscribe: (() => void) | undefined;
let timeout: NodeJS.Timeout | undefined;
let requestStarted = false;
let turnCount = 0;
const terminate = (reason: 'cancellation' | 'timeout' | 'turn-limit'): void => {
if (termination !== undefined) return;
termination = reason;
controller.abort(new DOMException(`Task-formation session ${reason}.`, 'AbortError'));
void session?.abort().catch(() => undefined);
};
const onCancellation = (): void => terminate('cancellation');
if (request.signal.aborted) {
executorLog('info', { ...logContext, event: 'finished', outcome: 'cancelled' });
throw cancellationError(request.signal);
}
request.signal.addEventListener('abort', onCancellation, { once: true });
timeout = setTimeout(() => terminate('timeout'), timeoutMs);
try {
let selection: ModelSelection;
try {
selection = await raceWithAbort(this.host.resolve('medium'), controller.signal);
} catch (error) {
if (termination === 'cancellation') throw cancellationError(request.signal);
if (termination === 'timeout') throw this.timeoutError(zeroUsage(), 0);
const failure = classifyModelFailure(this.host, error);
throw new TaskFormationExecutorError({
code: 'MODEL_SELECTION_FAILURE',
message: failure.message,
retryable: failure.retryable,
failureKind: 'model',
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
});
}
const sourceTools = await raceWithAbort(
createTaskFormationSourceTools({ cwd: request.cwd, deniedPaths: request.deniedPaths }),
controller.signal,
);
const customTools = [...sourceTools, request.submitTool.tool];
const toolNames = customTools.map((tool) => tool.name);
if (!hasExactToolSet(toolNames)) {
throw new TaskFormationExecutorError({
code: 'TOOL_POLICY_MISMATCH',
message: 'Task-formation source tool policy does not match the exact allowlist.',
retryable: false,
failureKind: 'confinement',
});
}
const agentDir = getAgentDir();
const settingsManager = SettingsManager.inMemory({
retry: PI_RETRY_SETTINGS,
compaction: { enabled: true },
});
const resourceLoader = new DefaultResourceLoader({
cwd: request.cwd,
agentDir,
settingsManager,
systemPrompt: `${request.systemPrompt}${request.submitTool.directive ?? ''}`,
appendSystemPrompt: [],
noExtensions: true,
noSkills: true,
noPromptTemplates: true,
noThemes: true,
noContextFiles: true,
});
await raceWithAbort(resourceLoader.reload(), controller.signal);
const sessionPromise = createAgentSession({
cwd: request.cwd,
agentDir,
model: selection.model,
modelRuntime: selection.modelRuntime,
noTools: 'all',
tools: toolNames,
customTools,
resourceLoader,
sessionManager: SessionManager.inMemory(),
settingsManager,
});
try {
({ session } = await raceWithAbort(sessionPromise, controller.signal));
} catch (error) {
void sessionPromise.then(
async ({ session: lateSession }) => {
await lateSession.abort().catch(() => undefined);
lateSession.dispose();
},
() => undefined,
);
throw error;
}
if (controller.signal.aborted) {
await session.abort().catch(() => undefined);
} else {
controller.signal.addEventListener('abort', () => void session?.abort().catch(() => undefined), {
once: true,
});
}
const registeredTools = session.getAllTools().map((tool) => tool.name);
if (!hasExactToolSet(registeredTools)) {
throw new TaskFormationExecutorError({
code: 'LIVE_TOOL_POLICY_MISMATCH',
message: 'The live task-formation session registered a tool outside the exact allowlist.',
retryable: false,
failureKind: 'confinement',
});
}
executorLog('info', {
...logContext,
event: 'started',
provider: selection.providerId,
model: selection.modelId,
tools: registeredTools,
resources: { context: false, extensions: false, prompts: false, skills: false },
});
let pendingProviderError: unknown;
unsubscribe = session.subscribe((event: AgentSessionEvent) => {
if (event.type !== 'turn_end') return;
turnCount += 1;
const message: AgentMessage = event.message;
if (message.role === 'assistant' && message.stopReason === 'error') {
pendingProviderError ??= message;
}
const needsAnotherTurn = message.role === 'assistant' && message.stopReason === 'toolUse';
if (turnCount >= maxTurns && needsAnotherTurn && request.submitTool.getAcceptedCount() === 0) {
terminate('turn-limit');
}
});
let promptError: unknown;
requestStarted = true;
try {
await raceWithAbort(session.prompt(request.modelContext, { expandPromptTemplates: false }), controller.signal);
} catch (error) {
promptError = error;
}
const outcome: SessionOutcome = {
pendingProviderError,
promptError,
usage: sessionUsage(session),
};
const output = this.resolveOutcome(request, outcome, termination, requestStarted);
executorLog('info', { ...logContext, event: 'finished', outcome: 'succeeded', usage: outcome.usage });
return {
output,
usage: outcome.usage,
providerId: selection.providerId,
modelId: selection.modelId,
modelCalls: 1,
registeredTools: Object.freeze([...registeredTools]),
};
} catch (error) {
// Termination reason wins over whatever error surfaced. A local timeout or an abort aborts the
// in-flight provider call, so the caught error is usually that induced abort; reporting it as a
// model failure would erase the real cause. Cancellation keeps its own identity ahead of timeout.
if (termination === 'cancellation') {
executorLog('info', { ...logContext, event: 'finished', outcome: 'cancelled' });
throw cancellationError(request.signal);
}
let failure: TaskFormationExecutorError;
if (termination === 'timeout') {
const usage = session ? sessionUsage(session) : zeroUsage();
const modelCalls = requestStarted ? 1 : 0;
failure = this.timeoutError(usage, modelCalls);
} else {
failure = this.normalizeFailure(error);
}
executorLog('warn', {
...logContext,
event: 'finished',
outcome: 'failed',
code: failure.code,
failureKind: failure.failureKind,
retryable: failure.retryable,
...(failure.fallbackReason !== undefined && { fallbackReason: failure.fallbackReason }),
usage: failure.usage,
modelCalls: failure.modelCalls,
});
throw failure;
} finally {
if (timeout) clearTimeout(timeout);
request.signal.removeEventListener('abort', onCancellation);
unsubscribe?.();
try {
session?.dispose();
} catch {
executorLog('warn', { ...logContext, event: 'cleanup-failed' });
}
}
}
private normalizeFailure(error: unknown): TaskFormationExecutorError {
if (error instanceof TaskFormationExecutorError) return error;
if (error instanceof ConfinementError) {
return new TaskFormationExecutorError({
code: `CONFINEMENT_${error.code}`,
message: error.message,
retryable: false,
failureKind: 'confinement',
});
}
if (isTransientIoFailure(error)) {
return new TaskFormationExecutorError({
code: 'SESSION_INFRASTRUCTURE_FAILURE',
message: 'Task-formation session setup encountered a retryable infrastructure failure.',
retryable: true,
failureKind: 'infrastructure',
});
}
const failure = classifyModelFailure(this.host, error);
if (failure.type === 'ConfigurationError') {
return new TaskFormationExecutorError({
code: 'MODEL_CONFIGURATION_FAILURE',
message: failure.message,
retryable: false,
failureKind: 'input',
});
}
return new TaskFormationExecutorError({
code: failure.type === 'AuthenticationError' ? 'PROVIDER_AUTHENTICATION_FAILURE' : 'MODEL_SESSION_FAILURE',
message: failure.message,
retryable: failure.retryable,
failureKind: 'model',
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
});
}
private timeoutError(usage: TaskFormationUsage, modelCalls: number): TaskFormationExecutorError {
return new TaskFormationExecutorError({
code: 'MODEL_STAGE_TIMEOUT',
message: 'Task formation exceeded its model-stage timeout.',
retryable: true,
failureKind: 'model',
fallbackReason: 'model_stage_timeout',
usage,
modelCalls,
});
}
private resolveOutcome(
request: TaskFormationExecutorRequest,
outcome: SessionOutcome,
termination: 'cancellation' | 'timeout' | 'turn-limit' | undefined,
requestStarted: boolean,
): unknown {
const modelCalls = requestStarted ? 1 : 0;
if (termination === 'cancellation') throw cancellationError(request.signal);
if (termination === 'timeout') throw this.timeoutError(outcome.usage, modelCalls);
if (termination === 'turn-limit') {
throw new TaskFormationExecutorError({
code: 'TURN_LIMIT',
message: 'Task formation exhausted its bounded model turn limit.',
retryable: true,
failureKind: 'model',
fallbackReason: 'retryable_model_failure',
usage: outcome.usage,
modelCalls,
});
}
if (request.submitTool.getAcceptedCount() > 1) {
throw new TaskFormationExecutorError({
code: 'DUPLICATE_ACCEPTED_SUBMISSION',
message: 'Task formation accepted more than one submission.',
retryable: true,
failureKind: 'model',
fallbackReason: 'retryable_model_failure',
usage: outcome.usage,
modelCalls,
});
}
if (outcome.pendingProviderError !== undefined) {
const failure = classifyModelFailure(this.host, outcome.pendingProviderError);
throw new TaskFormationExecutorError({
code: 'PROVIDER_FAILURE',
message: failure.message,
retryable: failure.retryable,
failureKind: 'model',
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
usage: outcome.usage,
modelCalls,
});
}
// A prompt error is a real failure unless exactly one submission was already accepted and the
// error is an abort: the submit tool terminates the session, so that abort is the expected end of
// a successful run, not a fault.
if (
outcome.promptError !== undefined &&
!(request.submitTool.getAcceptedCount() === 1 && isAbortLike(outcome.promptError))
) {
const failure = classifyModelFailure(this.host, outcome.promptError);
throw new TaskFormationExecutorError({
code: 'MODEL_SESSION_FAILURE',
message: failure.message,
retryable: failure.retryable,
failureKind: 'model',
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
usage: outcome.usage,
modelCalls,
});
}
const output = request.submitTool.getCaptured();
if (request.submitTool.getAcceptedCount() !== 1 || output === undefined) {
throw new TaskFormationExecutorError({
code: 'MISSING_ACCEPTED_SUBMISSION',
message: 'Task formation ended without one accepted submission.',
retryable: true,
failureKind: 'model',
fallbackReason: 'missing_accepted_submission',
usage: outcome.usage,
modelCalls,
});
}
return output;
}
}
export function createTaskFormationExecutor(host: ModelHost = modelHost): TaskFormationExecutor {
return new StandaloneTaskFormationExecutor(host);
}
export const taskFormationExecutor: TaskFormationExecutor = createTaskFormationExecutor();
+173 -25
View File
@@ -16,6 +16,9 @@ import { defineTool } from '@earendil-works/pi-coding-agent';
import { type Static, type TObject, Type } from 'typebox';
import { stringEnum } from '../collectors/schema.js';
import type { AgentName } from '../types/agents.js';
import type { VulnClass } from '../types/config.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import { isProducerId, REF_PREFIX } from './reconciliation/refs.js';
import type { CapturedSubmitTool } from './submit-tool.js';
const ANALYSIS_NOTES_DESCRIPTION = 'Plain context for defenders (caveats, scope, what is at risk). Not attack steps.';
@@ -24,6 +27,18 @@ function optStr(description?: string) {
return Type.Optional(Type.String(description === undefined ? {} : { description }));
}
const analysisCodeLocationSchema = Type.Object({
file: Type.String({ description: 'Repository-relative path, no leading slash.' }),
start_line: Type.Optional(Type.Integer({ minimum: 1 })),
end_line: Type.Optional(Type.Integer({ minimum: 1, description: 'Set when the flaw spans a range.' })),
role: stringEnum(['sink', 'source', 'guard'], {
description:
'sink where the flaw manifests, source where untrusted input enters, guard for a check ' +
'that is missing or misplaced.',
}),
symbol: Type.Optional(Type.String({ description: 'Enclosing function or method, named as written in the code.' })),
});
/**
* Base fields shared by every queue entry. `notes` gains guidance in analysis mode.
*
@@ -39,22 +54,9 @@ function baseFields(exploit: boolean) {
description: 'Confidence that this is a real, reachable vulnerability.',
}),
code_locations: Type.Optional(
Type.Array(
Type.Object({
file: Type.String({ description: 'Repository-relative path, no leading slash.' }),
start_line: Type.Optional(Type.Integer({ minimum: 1 })),
end_line: Type.Optional(Type.Integer({ minimum: 1, description: 'Set when the flaw spans a range.' })),
role: stringEnum(['sink', 'source', 'guard'], {
description:
'sink where the flaw manifests, source where untrusted input enters, guard for a check ' +
'that is missing or misplaced.',
}),
symbol: Type.Optional(
Type.String({ description: 'Enclosing function or method, named as written in the code.' }),
),
}),
{ description: 'Every code site this finding touches, sink first.' },
),
Type.Array(analysisCodeLocationSchema, {
description: 'Every code site this finding touches, sink first.',
}),
),
notes: exploit ? optStr() : optStr(ANALYSIS_NOTES_DESCRIPTION),
};
@@ -102,6 +104,17 @@ const ssrfFields = {
suggested_exploit_technique: optStr(),
};
const miscellaneousFields = {
cwe: optStr(),
source_endpoint: optStr(),
vulnerable_code_location: optStr(),
missing_defense: optStr(),
observable_signal: optStr(),
exploitation_hypothesis: optStr(),
suggested_exploit_technique: optStr(),
proof_criterion: optStr(),
};
const authzFields = {
endpoint: optStr(),
vulnerable_code_location: optStr(),
@@ -119,14 +132,44 @@ const xssEntry = () => Type.Object({ ...baseFields(true), ...xssFields });
const authEntry = () => Type.Object({ ...baseFields(true), ...authFields });
const ssrfEntry = () => Type.Object({ ...baseFields(true), ...ssrfFields });
const authzEntry = () => Type.Object({ ...baseFields(true), ...authzFields });
const miscellaneousEntry = () => Type.Object({ ...baseFields(true), ...miscellaneousFields });
export type QueueCodeLocation = NonNullable<Static<ReturnType<typeof injectionEntry>>['code_locations']>[number];
export type AnalysisCodeLocation = Static<typeof analysisCodeLocationSchema>;
/** Queue-specific name retained for the existing analysis-location report join. */
export type QueueCodeLocation = AnalysisCodeLocation;
export type InjectionFinding = Static<ReturnType<typeof injectionEntry>>;
export type XssFinding = Static<ReturnType<typeof xssEntry>>;
export type AuthFinding = Static<ReturnType<typeof authEntry>>;
export type SsrfFinding = Static<ReturnType<typeof ssrfEntry>>;
export type AuthzFinding = Static<ReturnType<typeof authzEntry>>;
export type MiscellaneousFinding = Static<ReturnType<typeof miscellaneousEntry>>;
// The exact field names each class's queue entry carries. Reconciliation reads this to know which
// keys a class produces, so it must list the same base and per-class fields the schemas above build.
export const QUEUE_ENTRY_FIELD_NAMES: Readonly<Record<ReconciliationClass, readonly string[]>> = Object.freeze({
injection: [...Object.keys(baseFields(true)), ...Object.keys(injectionFields)],
xss: [...Object.keys(baseFields(true)), ...Object.keys(xssFields)],
auth: [...Object.keys(baseFields(true)), ...Object.keys(authFields)],
authz: [...Object.keys(baseFields(true)), ...Object.keys(authzFields)],
ssrf: [...Object.keys(baseFields(true)), ...Object.keys(ssrfFields)],
miscellaneous: [...Object.keys(baseFields(true)), ...Object.keys(miscellaneousFields)],
});
const ENTRY_SCHEMAS: Readonly<Record<ReconciliationClass, TObject>> = Object.freeze({
injection: injectionEntry(),
xss: xssEntry(),
auth: authEntry(),
authz: authzEntry(),
ssrf: ssrfEntry(),
miscellaneous: miscellaneousEntry(),
});
/** The complete queue-entry schema for one internal class. */
export function classEntrySchema(vulnClass: ReconciliationClass): TObject {
return ENTRY_SCHEMAS[vulnClass];
}
const PER_TYPE_FIELDS: Partial<Record<AgentName, Record<string, ReturnType<typeof optStr>>>> = {
'injection-vuln': injectionFields,
@@ -144,12 +187,102 @@ const VULN_AGENT_QUEUE_FILENAMES: Partial<Record<AgentName, string>> = {
'authz-vuln': 'authz_exploitation_queue.json',
};
/** Build the TypeBox submit-tool parameters for a vuln agent, or undefined for non-vuln agents. */
const VULN_AGENT_CLASSES: Partial<Record<AgentName, VulnClass>> = {
'injection-vuln': 'injection',
'xss-vuln': 'xss',
'auth-vuln': 'auth',
'authz-vuln': 'authz',
'ssrf-vuln': 'ssrf',
};
function producerIdFormat(vulnClass: VulnClass): string {
return `${REF_PREFIX[vulnClass]}-VULN-NN`;
}
function outOfNamespaceIds(vulnerabilities: readonly unknown[], vulnClass: VulnClass): string[] {
const rejected: string[] = [];
for (const entry of vulnerabilities) {
if (entry === null || typeof entry !== 'object') continue;
const id = (entry as { ID?: unknown }).ID;
if (typeof id !== 'string' || !isProducerId(id, vulnClass, 'VULN')) {
rejected.push(typeof id === 'string' ? id : String(id));
}
}
return rejected;
}
const ID_PREVIEW_LIMIT = 6;
function previewIds(ids: readonly string[]): string {
const shown = ids.slice(0, ID_PREVIEW_LIMIT).join(', ');
const remainder = ids.length - ID_PREVIEW_LIMIT;
return remainder > 0 ? `${shown} (+${remainder} more)` : shown;
}
// A rejected submission is returned as a retryable tool error rather than thrown: that hands the
// message back to the model to correct within the same session instead of failing the whole agent.
// `terminate` is deliberately left unset so the session survives to receive the corrected call.
function retryableRejection(message: string) {
return {
content: [
{
type: 'text' as const,
text: JSON.stringify({ status: 'error', errorType: 'ValidationError', retryable: true, message }, null, 2),
},
],
details: undefined,
};
}
function idNamespaceRejection(vulnClass: VulnClass, rejected: readonly string[]) {
const message =
`Every entry ID must be ${producerIdFormat(vulnClass)} ` +
`(for example ${REF_PREFIX[vulnClass]}-VULN-01). Outside that namespace: ${previewIds(rejected)}.`;
return retryableRejection(message);
}
// Exact-string repeats, the same comparison reconciliation makes when it re-reads the committed
// queue. Each offending ID is named once, in the order it first repeats.
function repeatedProducerIds(vulnerabilities: readonly unknown[]): string[] {
const seen = new Set<string>();
const repeated: string[] = [];
for (const entry of vulnerabilities) {
if (entry === null || typeof entry !== 'object') continue;
const id = (entry as { ID?: unknown }).ID;
if (typeof id !== 'string') continue;
if (seen.has(id) && !repeated.includes(id)) repeated.push(id);
seen.add(id);
}
return repeated;
}
function duplicateIdRejection(vulnClass: VulnClass, repeated: readonly string[]) {
const message =
`Each entry needs its own ${producerIdFormat(vulnClass)} ID; these appear more than once: ` +
`${previewIds(repeated)}. Renumber the repeated entries — or drop the ones that describe the same ` +
'vulnerability — and call submit_exploitation_queue again.';
return retryableRejection(message);
}
/**
* Build the TypeBox submit-tool parameters for a vuln agent, or undefined for non-vuln agents.
*
* The `ID` this schema requires (`INJ-VULN-01` and so on) is an internal producer token, meant
* only to let reconciliation join a finding back to the agent and class that raised it. It is not
* the identifier a downstream exploit agent should ever see; reconciliation is responsible for
* translating it into the exploitation-task identity the published queue carries instead.
*/
function queueSchema(agentName: AgentName, exploit: boolean): TObject | undefined {
const extra = PER_TYPE_FIELDS[agentName];
if (!extra) return undefined;
const vulnClass = VULN_AGENT_CLASSES[agentName];
if (!extra || !vulnClass) return undefined;
const idField = Type.String({
description:
`Producer identifier formatted ${producerIdFormat(vulnClass)} ` +
`(for example ${REF_PREFIX[vulnClass]}-VULN-01).`,
});
return Type.Object({
vulnerabilities: Type.Array(Type.Object({ ...baseFields(exploit), ...extra })),
vulnerabilities: Type.Array(Type.Object({ ...baseFields(exploit), ID: idField, ...extra })),
});
}
@@ -161,7 +294,8 @@ export function getQueueFilename(agentName: AgentName): string | undefined {
/** Build the pi submit tool that captures the exploitation queue for vuln agents. */
export function createQueueSubmitTool(agentName: AgentName, exploit = true): CapturedSubmitTool | undefined {
const schema = queueSchema(agentName, exploit);
if (!schema) return undefined;
const vulnClass = VULN_AGENT_CLASSES[agentName];
if (!schema || !vulnClass) return undefined;
let captured: unknown | undefined;
return {
@@ -174,15 +308,29 @@ export function createQueueSubmitTool(agentName: AgentName, exploit = true): Cap
promptGuidelines: [
'You MUST call submit_exploitation_queue exactly once as your final action.',
'Include every analyzed finding in the vulnerabilities array.',
`Give every entry an ID in the ${producerIdFormat(vulnClass)} namespace.`,
],
parameters: schema,
async execute(_toolCallId, params) {
const vulnerabilities = Array.isArray((params as { vulnerabilities?: unknown }).vulnerabilities)
? (params as { vulnerabilities: unknown[] }).vulnerabilities
: [];
// Every entry ID must sit in this class's producer namespace, and no two entries may share
// one, so reconciliation refs stay unique both across classes and within this queue. Both
// rules are the ones reconciliation applies to the committed queue; enforcing them here
// turns a permanent post-commit failure into an in-session correction. Nothing is captured,
// written, or committed until every ID passes.
const rejected = outOfNamespaceIds(vulnerabilities, vulnClass);
if (rejected.length > 0) {
return idNamespaceRejection(vulnClass, rejected);
}
const repeated = repeatedProducerIds(vulnerabilities);
if (repeated.length > 0) {
return duplicateIdRejection(vulnClass, repeated);
}
captured = params;
const count = Array.isArray((params as { vulnerabilities?: unknown }).vulnerabilities)
? (params as { vulnerabilities: unknown[] }).vulnerabilities.length
: 0;
return {
content: [{ type: 'text' as const, text: `Recorded ${count} findings.` }],
content: [{ type: 'text' as const, text: `Recorded ${vulnerabilities.length} findings.` }],
details: params,
terminate: true,
};
@@ -0,0 +1,518 @@
/** Content-addressed, no-replace storage for reconciliation intermediates. */
import { createHash, randomBytes } from 'node:crypto';
import type { Stats } from 'node:fs';
import { link, lstat, mkdir, open, readFile, realpath, unlink } from 'node:fs/promises';
import path from 'node:path';
import { WORKSPACES_DIR } from '../../paths.js';
import { ALL_RECONCILIATION_CLASSES, type ReconciliationClass } from '../../types/reconciliation.js';
import type { ArtifactInputDigest, ArtifactKind, ArtifactRef } from './contracts.js';
import { RECONCILIATION_SCHEMA_VERSION } from './schema-version.js';
import type { ArtifactBodyMap } from './stage-contracts.js';
export { RECONCILIATION_SCHEMA_VERSION };
const KIND_SEQUENCE: Readonly<Record<ArtifactKind, string>> = Object.freeze({
'producer-observations': '00',
'supplemental-observations': '01',
'task-formation': '02',
'fixed-tasks': '03',
});
const ARTIFACT_KINDS = Object.freeze(Object.keys(KIND_SEQUENCE) as ArtifactKind[]);
const SHA256_PATTERN = /^[a-f0-9]{64}$/;
export type ReconciliationFailureType =
| 'ReconciliationArtifactNotFound'
| 'ReconciliationIoError'
| 'ArtifactIntegrityError'
| 'PublicationConflict'
| 'SastEnrichmentInputError'
| 'SastEnrichmentModelError';
/** Typed reconciliation failure normalized by the later Temporal activity boundary. */
export class ReconciliationError extends Error {
readonly retryable: boolean;
readonly failureType: ReconciliationFailureType;
// The default failure type follows `retryable`: a retryable error reads as transient I/O,
// a non-retryable one as an integrity violation. Temporal keeps retrying the former and
// fails fast on the latter, so the two must stay aligned.
constructor(
message: string,
retryable: boolean,
failureType: ReconciliationFailureType = retryable ? 'ReconciliationIoError' : 'ArtifactIntegrityError',
) {
super(message);
this.name = failureType;
this.retryable = retryable;
this.failureType = failureType;
}
}
export class ReconciliationArtifactNotFoundError extends ReconciliationError {
constructor(message: string) {
super(message, true, 'ReconciliationArtifactNotFound');
}
}
export class ReconciliationIoError extends ReconciliationError {
constructor(message: string) {
super(message, true, 'ReconciliationIoError');
}
}
export class ArtifactIntegrityError extends ReconciliationError {
constructor(message: string) {
super(message, false, 'ArtifactIntegrityError');
}
}
export class PublicationConflictError extends ReconciliationError {
constructor(message: string) {
super(message, false, 'PublicationConflict');
}
}
interface ArtifactEnvelope {
artifactKind: ArtifactKind;
vulnerabilityClass: ReconciliationClass;
schemaVersion: 1;
inputs: ArtifactInputDigest[];
counts: Record<string, number>;
body: unknown;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function isErrno(error: unknown, code: string): boolean {
return error instanceof Error && (error as NodeJS.ErrnoException).code === code;
}
function validateSessionId(sessionId: string): void {
const valid =
sessionId.length > 0 &&
sessionId !== '.' &&
sessionId !== '..' &&
!sessionId.includes('/') &&
!sessionId.includes('\\') &&
!sessionId.includes('\0');
if (!valid) {
throw new ArtifactIntegrityError('Invalid reconciliation session identifier');
}
}
function validateClass(vulnerabilityClass: ReconciliationClass): void {
if (!ALL_RECONCILIATION_CLASSES.includes(vulnerabilityClass)) {
throw new ArtifactIntegrityError('Invalid reconciliation class');
}
}
function validateDigest(sha256: string, label: string): void {
if (!SHA256_PATTERN.test(sha256)) {
throw new ArtifactIntegrityError(`${label} is not a lowercase SHA-256 digest`);
}
}
function validateInputs(inputs: readonly ArtifactInputDigest[]): void {
if (!Array.isArray(inputs)) {
throw new ArtifactIntegrityError('Artifact lineage is not an array');
}
for (const input of inputs) {
if (!isRecord(input) || !ARTIFACT_KINDS.includes(input.artifactKind as ArtifactKind)) {
throw new ArtifactIntegrityError('Artifact lineage contains an invalid kind');
}
validateDigest(input.sha256 as string, 'Artifact lineage digest');
if (Object.keys(input).sort().join(',') !== 'artifactKind,sha256') {
throw new ArtifactIntegrityError('Artifact lineage contains unexpected metadata');
}
}
}
function validateCounts(counts: Record<string, number>): void {
if (!isRecord(counts)) {
throw new ArtifactIntegrityError('Artifact counts are not an object');
}
for (const [name, value] of Object.entries(counts)) {
if (name.length === 0 || !Number.isSafeInteger(value) || value < 0) {
throw new ArtifactIntegrityError('Artifact counts contain an invalid entry');
}
}
}
function sha256Hex(bytes: Buffer): string {
return createHash('sha256').update(bytes).digest('hex');
}
function artifactFilename(kind: ArtifactKind, sha256: string): string {
return `${KIND_SEQUENCE[kind]}-${kind}-${sha256}.json`;
}
function serializeEnvelope(envelope: ArtifactEnvelope): Buffer {
try {
return Buffer.from(JSON.stringify(envelope), 'utf8');
} catch {
throw new ArtifactIntegrityError('Reconciliation artifact body is not JSON serializable');
}
}
/** Stable root for one class, independent of customer output destinations. */
export function reconciliationDir(
sessionId: string,
vulnerabilityClass: ReconciliationClass,
workspacesDir: string = WORKSPACES_DIR,
): string {
validateSessionId(sessionId);
validateClass(vulnerabilityClass);
return path.resolve(workspacesDir, sessionId, '.shannon', 'reconciliation', vulnerabilityClass);
}
async function ensureDirectory(parent: string, segment: string): Promise<string> {
const next = path.join(parent, segment);
try {
await mkdir(next);
} catch (error) {
if (!isErrno(error, 'EEXIST')) {
throw new ReconciliationIoError('Unable to create reconciliation artifact directory');
}
}
let stat: Stats;
try {
stat = await lstat(next);
} catch {
throw new ReconciliationIoError('Unable to inspect reconciliation artifact directory');
}
if (stat.isSymbolicLink() || !stat.isDirectory()) {
throw new ArtifactIntegrityError('Reconciliation artifact root contains a symlink or non-directory');
}
return next;
}
// `ensureArtifactRoot` (write path) and `resolveArtifactRoot` (read path) are kept as separate
// functions rather than one with a "create if missing" flag: a read for a session/class that never
// wrote anything must fail as not-found, not silently materialize an empty directory chain that
// would then make an absent artifact look like a not-yet-written one.
async function ensureArtifactRoot(
sessionId: string,
vulnerabilityClass: ReconciliationClass,
workspacesDir: string,
): Promise<string> {
validateSessionId(sessionId);
validateClass(vulnerabilityClass);
try {
await mkdir(workspacesDir, { recursive: true });
} catch {
throw new ReconciliationIoError('Unable to create the reconciliation workspace root');
}
let realWorkspaces: string;
try {
realWorkspaces = await realpath(workspacesDir);
} catch {
throw new ReconciliationIoError('Unable to resolve the reconciliation workspace root');
}
let current = realWorkspaces;
for (const segment of [sessionId, '.shannon', 'reconciliation', vulnerabilityClass]) {
current = await ensureDirectory(current, segment);
}
return current;
}
async function resolveArtifactRoot(
sessionId: string,
vulnerabilityClass: ReconciliationClass,
workspacesDir: string,
): Promise<string> {
validateSessionId(sessionId);
validateClass(vulnerabilityClass);
let current: string;
try {
current = await realpath(workspacesDir);
} catch {
throw new ReconciliationArtifactNotFoundError('Reconciliation workspace root is not visible');
}
for (const segment of [sessionId, '.shannon', 'reconciliation', vulnerabilityClass]) {
const next = path.join(current, segment);
let stat: Stats;
try {
stat = await lstat(next);
} catch (error) {
if (isErrno(error, 'ENOENT')) {
throw new ReconciliationArtifactNotFoundError('Reconciliation artifact root is not visible');
}
throw new ReconciliationIoError('Unable to inspect reconciliation artifact root');
}
if (stat.isSymbolicLink() || !stat.isDirectory()) {
throw new ArtifactIntegrityError('Reconciliation artifact root contains a symlink or non-directory');
}
current = next;
}
return current;
}
async function writeDurableTemporaryFile(tempPath: string, bytes: Buffer): Promise<void> {
try {
// `wx` fails if the attempt-unique temp path already exists, and the fsync forces the bytes
// to disk before the later `link` publishes them. Publishing an unsynced file would let a
// crash leave a linked-but-empty artifact that its digest no longer matches.
const handle = await open(tempPath, 'wx');
try {
await handle.writeFile(bytes);
await handle.sync();
} finally {
await handle.close();
}
} catch {
await unlink(tempPath).catch(() => undefined);
throw new ReconciliationIoError('Unable to durably create an attempt-unique reconciliation artifact');
}
}
// Some filesystems reject fsync on a directory handle. Those codes mean the durability barrier
// is unavailable, not that publication failed, so the caller treats them as success.
function directorySyncIsUnsupported(error: unknown): boolean {
if (!(error instanceof Error)) return false;
const code = (error as NodeJS.ErrnoException).code;
return code === 'EINVAL' || code === 'ENOTSUP' || code === 'EOPNOTSUPP' || code === 'EBADF' || code === 'EISDIR';
}
async function syncPublishedDirectory(directory: string): Promise<void> {
try {
const handle = await open(directory, 'r');
try {
await handle.sync();
} finally {
await handle.close();
}
} catch (error) {
if (directorySyncIsUnsupported(error)) return;
throw new ReconciliationIoError('Unable to make the reconciliation artifact directory durable');
}
}
export type WriteArtifactArgs<TKind extends ArtifactKind = ArtifactKind> = {
[K in TKind]: {
sessionId: string;
workspacesDir?: string;
artifactKind: K;
vulnerabilityClass: ReconciliationClass;
body: ArtifactBodyMap[K];
inputs: ArtifactInputDigest[];
counts: Record<string, number>;
};
}[TKind];
/** Serialize and publish one whole-envelope artifact without replacing an existing path. */
export async function writeArtifact<TKind extends ArtifactKind>(
args: WriteArtifactArgs<TKind>,
): Promise<ArtifactRef<TKind>> {
validateInputs(args.inputs);
validateCounts(args.counts);
const envelope: ArtifactEnvelope = {
artifactKind: args.artifactKind,
vulnerabilityClass: args.vulnerabilityClass,
schemaVersion: RECONCILIATION_SCHEMA_VERSION,
inputs: args.inputs,
counts: args.counts,
body: args.body,
};
const bytes = serializeEnvelope(envelope);
const sha256 = sha256Hex(bytes);
const root = await ensureArtifactRoot(args.sessionId, args.vulnerabilityClass, args.workspacesDir ?? WORKSPACES_DIR);
const filename = artifactFilename(args.artifactKind, sha256);
const finalPath = path.join(root, filename);
const tempPath = path.join(root, `.${filename}.tmp-${randomBytes(12).toString('hex')}`);
await writeDurableTemporaryFile(tempPath, bytes);
// The final path is content-addressed by digest, so an EEXIST link means a prior attempt already
// published these exact bytes. Identical bytes are adopted as success (idempotent republish);
// different bytes at the same digest path can only be corruption or a hash collision, so they
// conflict. This is what makes a retried write after a lost acknowledgement safe.
let failure: unknown;
try {
await link(tempPath, finalPath);
} catch (error) {
if (!isErrno(error, 'EEXIST')) {
failure = new ReconciliationIoError('Unable to publish reconciliation artifact');
} else {
try {
const stat = await lstat(finalPath);
if (stat.isSymbolicLink() || !stat.isFile()) {
failure = new ArtifactIntegrityError('Existing reconciliation artifact is not a regular file');
} else {
const existing = await readFile(finalPath);
if (!existing.equals(bytes)) {
failure = new PublicationConflictError('Existing reconciliation artifact has different exact bytes');
}
}
} catch (readError) {
failure =
readError instanceof ReconciliationError
? readError
: new ReconciliationIoError('Unable to verify existing reconciliation artifact');
}
}
}
if (failure === undefined) {
try {
await syncPublishedDirectory(root);
} catch (error) {
failure = error;
}
}
// Remove the attempt-local temp link last. A cleanup failure is only reported when nothing
// earlier failed, so temp-file noise never masks a real publication or conflict error.
try {
await unlink(tempPath);
} catch (error) {
if (!isErrno(error, 'ENOENT') && failure === undefined) {
failure = new ReconciliationIoError('Unable to remove attempt-local reconciliation artifact');
}
}
if (failure !== undefined) throw failure;
return {
path: finalPath,
artifactKind: args.artifactKind,
vulnerabilityClass: args.vulnerabilityClass,
schemaVersion: RECONCILIATION_SCHEMA_VERSION,
sha256,
inputs: args.inputs,
counts: args.counts,
};
}
/** Whether two ordered artifact lineages contain identical kinds and digests. */
export function artifactInputsMatch(
first: readonly ArtifactInputDigest[],
second: readonly ArtifactInputDigest[],
): boolean {
return (
first.length === second.length &&
first.every(
(entry, index) => entry.artifactKind === second[index]?.artifactKind && entry.sha256 === second[index]?.sha256,
)
);
}
function countsMatch(first: Record<string, number>, second: Record<string, number>): boolean {
const firstKeys = Object.keys(first).sort();
const secondKeys = Object.keys(second).sort();
return (
firstKeys.length === secondKeys.length &&
firstKeys.every((key, index) => key === secondKeys[index] && first[key] === second[key])
);
}
function parseEnvelope(bytes: Buffer): ArtifactEnvelope {
let parsed: unknown;
try {
parsed = JSON.parse(bytes.toString('utf8'));
} catch {
throw new ArtifactIntegrityError('Reconciliation artifact is not valid JSON');
}
if (!isRecord(parsed)) {
throw new ArtifactIntegrityError('Reconciliation artifact envelope is not an object');
}
const expectedKeys = ['artifactKind', 'body', 'counts', 'inputs', 'schemaVersion', 'vulnerabilityClass'];
if (Object.keys(parsed).sort().join(',') !== expectedKeys.join(',')) {
throw new ArtifactIntegrityError('Reconciliation artifact envelope has unexpected fields');
}
if (!ARTIFACT_KINDS.includes(parsed.artifactKind as ArtifactKind)) {
throw new ArtifactIntegrityError('Reconciliation artifact kind is invalid');
}
if (!ALL_RECONCILIATION_CLASSES.includes(parsed.vulnerabilityClass as ReconciliationClass)) {
throw new ArtifactIntegrityError('Reconciliation artifact class is invalid');
}
if (parsed.schemaVersion !== RECONCILIATION_SCHEMA_VERSION) {
throw new ArtifactIntegrityError('Reconciliation artifact schema version is invalid');
}
validateInputs(parsed.inputs as ArtifactInputDigest[]);
validateCounts(parsed.counts as Record<string, number>);
return parsed as unknown as ArtifactEnvelope;
}
function validateRef(ref: ArtifactRef): void {
if (!ARTIFACT_KINDS.includes(ref.artifactKind)) {
throw new ArtifactIntegrityError('Artifact reference kind is invalid');
}
validateClass(ref.vulnerabilityClass);
if (ref.schemaVersion !== RECONCILIATION_SCHEMA_VERSION) {
throw new ArtifactIntegrityError('Artifact reference schema version is invalid');
}
validateDigest(ref.sha256, 'Artifact reference digest');
validateInputs(ref.inputs);
validateCounts(ref.counts);
}
/** Read and verify one artifact's path, bytes, envelope metadata, and ordered lineage. */
export async function readArtifact<TKind extends ArtifactKind>(
ref: ArtifactRef<TKind>,
sessionId: string,
workspacesDir: string = WORKSPACES_DIR,
): Promise<ArtifactBodyMap[TKind]> {
validateRef(ref);
const root = await resolveArtifactRoot(sessionId, ref.vulnerabilityClass, workspacesDir);
// A reference carries its own path through Temporal history. Recompute the only path this
// kind and digest may occupy and demand an exact match, so a tampered or stale ref cannot
// point a read at an arbitrary file outside the class artifact root.
const expectedPath = path.join(root, artifactFilename(ref.artifactKind, ref.sha256));
if (!path.isAbsolute(ref.path) || path.normalize(ref.path) !== ref.path || ref.path !== expectedPath) {
throw new ArtifactIntegrityError('Artifact reference path is not the expected contained path');
}
let stat: Stats;
try {
stat = await lstat(ref.path);
} catch (error) {
if (isErrno(error, 'ENOENT')) {
throw new ReconciliationArtifactNotFoundError('Referenced reconciliation artifact is not visible');
}
throw new ReconciliationIoError('Unable to inspect referenced reconciliation artifact');
}
if (stat.isSymbolicLink() || !stat.isFile()) {
throw new ArtifactIntegrityError('Referenced reconciliation artifact is not a regular file');
}
let resolvedPath: string;
try {
resolvedPath = await realpath(ref.path);
} catch {
throw new ReconciliationArtifactNotFoundError('Referenced reconciliation artifact is not visible');
}
if (resolvedPath !== expectedPath) {
throw new ArtifactIntegrityError('Referenced reconciliation artifact resolves outside its expected path');
}
let bytes: Buffer;
try {
bytes = await readFile(resolvedPath);
} catch {
throw new ReconciliationIoError('Unable to read referenced reconciliation artifact');
}
if (sha256Hex(bytes) !== ref.sha256) {
throw new ArtifactIntegrityError('Reconciliation artifact digest does not match its reference');
}
const envelope = parseEnvelope(bytes);
if (
envelope.artifactKind !== ref.artifactKind ||
envelope.vulnerabilityClass !== ref.vulnerabilityClass ||
envelope.schemaVersion !== ref.schemaVersion ||
!artifactInputsMatch(envelope.inputs, ref.inputs) ||
!countsMatch(envelope.counts, ref.counts)
) {
throw new ArtifactIntegrityError('Reconciliation artifact metadata or lineage does not match its reference');
}
return envelope.body as ArtifactBodyMap[TKind];
}
@@ -0,0 +1,131 @@
/** Shared contracts for the single-scan reconciliation pipeline. */
import type { ReconciliationClass } from '../../types/reconciliation.js';
import type {
AuthFinding,
AuthzFinding,
InjectionFinding,
MiscellaneousFinding,
SsrfFinding,
XssFinding,
} from '../queue-schemas.js';
/** Which producer emitted an observation. */
export type ScanSource = 'vulnerability_analysis' | 'sast';
/** Internal primary-selection preference. Never exposed to a model or consumer queue. */
export type PrimaryPreference = 'default' | 'preferred';
/** Priority supplied by the SAST bridge. */
export type Priority = 'P1' | 'P2' | 'P3';
/**
* Authoritative SAST source location copied from validated SARIF.
*
* This is the exact file/line/column the static analysis engine pinned its finding to, carried
* through reconciliation unchanged so a task's reported location always traces back to real
* evidence rather than something reconstructed or guessed downstream.
*/
export interface SastSourceLocation {
file: string;
line: number;
column: number;
rule_id: string;
}
// Widen each member of a union so the keys unique to its siblings are typed `never`. This lets one
// evidence value be discriminated by which class's fields it carries, and makes assigning a foreign
// class's field a compile error rather than a silently accepted extra property.
type ExclusiveUnion<T, TAll = T> = T extends unknown
? T & Partial<Record<Exclude<TAll extends unknown ? keyof TAll : never, keyof T>, never>>
: never;
/** Class-specific evidence with the producer-owned `ID` removed. */
export type ClassEvidence = ExclusiveUnion<
| Omit<InjectionFinding, 'ID'>
| Omit<XssFinding, 'ID'>
| Omit<AuthFinding, 'ID'>
| Omit<AuthzFinding, 'ID'>
| Omit<SsrfFinding, 'ID'>
| Omit<MiscellaneousFinding, 'ID'>
>;
// A SAST-origin observation always declares `preferred`. This is the dedupe contract: when
// reconciliation merges a SAST finding with a pentest finding for the same underlying bug, the
// SAST evidence becomes the task's primary record (it carries an exact file/line/rule, while a
// pentest finding does not), and the pentest observation survives only as a merged member.
export interface SastProducerFields {
producer_id: string;
scan_source: 'sast';
primary_preference: 'preferred';
priority: Priority;
sast_source_location: SastSourceLocation;
}
// Pentest-origin observations always declare `default`, the losing side of the preference above.
export interface VulnAnalysisProducerFields {
producer_id: string;
scan_source: 'vulnerability_analysis';
primary_preference: 'default';
}
export type ProducerFields = SastProducerFields | VulnAnalysisProducerFields;
// Once a group is collapsed into a task the primary is already chosen, so `primary_preference`
// has done its job and is dropped. It must not survive into materialized tasks or published output.
export type MaterializedProducerFields =
| Omit<SastProducerFields, 'primary_preference'>
| Omit<VulnAnalysisProducerFields, 'primary_preference'>;
/** One current observation before task formation. */
export type ReconciliationObservation<E extends ClassEvidence = ClassEvidence> = E & ProducerFields;
/** One non-primary observation retained under a materialized task. */
export type MergedObservation<E extends ClassEvidence = ClassEvidence> = E & MaterializedProducerFields;
/** One stable exploitation task before publication removes internal producer fields. */
export type ReconciliationTask<E extends ClassEvidence = ClassEvidence> = E &
MaterializedProducerFields & {
ID: string;
merged_from?: MergedObservation<E>[];
};
/** The exact OSS intermediate artifact vocabulary, in stage order. */
export type ArtifactKind = 'producer-observations' | 'supplemental-observations' | 'task-formation' | 'fixed-tasks';
// One entry in an artifact's lineage: which prior-stage artifact (by kind and exact content
// digest) it was built from. A later stage checks these digests against the refs it was actually
// handed, so it can refuse to proceed if its inputs were regenerated or swapped out from under it.
export interface ArtifactInputDigest {
artifactKind: ArtifactKind;
sha256: string;
}
/** Safe content-addressed metadata carried through Temporal history. */
export interface ArtifactRef<TKind extends ArtifactKind = ArtifactKind> {
path: string;
artifactKind: TKind;
vulnerabilityClass: ReconciliationClass;
schemaVersion: 1;
sha256: string;
inputs: ArtifactInputDigest[];
counts: Record<string, number>;
}
/** Exact durable output set for one class publication. */
export interface PublicationContract {
publicationKind: 'class-reconciliation';
schemaVersion: 1;
manifestPath: string;
requiredOutputPaths: readonly string[];
}
/** History-safe aggregate metrics for one reconciled class. */
export interface ReconciliationMetrics {
alreadyPublished: boolean;
durationMs: number;
costUsd: number;
inputTokens: number;
outputTokens: number;
modelCalls: number;
}
+415
View File
@@ -0,0 +1,415 @@
/** Strict per-class SAST enrichment through the shared one-shot generation port. */
import { WORKSPACES_DIR } from '../../paths.js';
import { loadPrompt } from '../../services/prompt-manager.js';
import type { ActivityLogger } from '../../types/activity-logger.js';
import type { ReconciliationClass } from '../../types/reconciliation.js';
import type { SarifRef } from '../sast/types.js';
import type { StructuredGenerationPort } from '../structured-generation.js';
import { ArtifactIntegrityError, ReconciliationError, ReconciliationIoError, writeArtifact } from './artifact-store.js';
import type { ReconciliationObservation } from './contracts.js';
import { extractContext } from './sast/context-extractor.js';
import { CWE_TO_CATEGORY, unmappedMapping, vulnerabilityClassToCategory } from './sast/cwe-mapper.js';
import { runSastEnrichmentBatch, type SastEnrichmentBatchOutcome } from './sast/enrichment/batch.js';
import { buildSastObservation, mintSastProducerId, sourceLocationFromContext } from './sast/enrichment/policy.js';
import { enrichmentPromptName } from './sast/enrichment/schema.js';
import {
EnrichmentAttemptError,
pairEnrichedVulnerabilities,
type ValidationResult,
} from './sast/enrichment/validate.js';
import { readPinnedSarif, SarifIntakeError } from './sast/intake.js';
import { parseSarifContent, SarifDocumentError } from './sast/sarif-parser.js';
import type { ClassifiedFinding, DroppedSarifFinding, ParsedSarif, ParsedSarifFinding } from './sast/types.js';
import type {
EnrichSuccess,
StageMetrics,
SupplementalDropCounts,
SupplementalDroppedFinding,
SupplementalObservationsBody,
} from './stage-contracts.js';
const ENRICHMENT_MAX_TOKENS = 32768;
export interface EnrichClassSastObservationsInput {
sessionId: string;
vulnerabilityClass: ReconciliationClass;
sarif?: SarifRef;
}
export interface EnrichClassSastObservationsDeps<TModelContext> {
generation: StructuredGenerationPort<TModelContext>;
modelContextFor: (input: EnrichClassSastObservationsInput) => TModelContext;
workspacesDir?: string;
signalFor?: () => AbortSignal | undefined;
promptLoader?: (vulnerabilityClass: ReconciliationClass) => Promise<string>;
onMetrics?: (metrics: StageMetrics) => void;
logger?: ActivityLogger;
}
export class SastEnrichmentInputError extends ReconciliationError {
constructor(message: string) {
super(message, false, 'SastEnrichmentInputError');
}
}
export class SastEnrichmentModelError extends ReconciliationError {
readonly metrics: StageMetrics;
constructor(message: string, metrics: StageMetrics, retryable = true) {
super(message, retryable, 'SastEnrichmentModelError');
this.metrics = metrics;
}
}
/** Cancellation marker translated into Temporal cancellation by the activity wrapper. */
export class SastEnrichmentCancelledError extends Error {
constructor() {
super('SAST enrichment was cancelled');
this.name = 'AbortError';
}
}
const NOOP_LOGGER: ActivityLogger = {
info() {},
warn() {},
error() {},
};
function zeroDrops(): SupplementalDropCounts {
return {
unknown_cwe: 0,
other_category: 0,
malformed: 0,
orphaned: 0,
duplicate_sast_id: 0,
enrichment_dropped: 0,
};
}
function zeroMetrics(): StageMetrics {
return { costUsd: 0, modelCalls: 0, inputTokens: 0, outputTokens: 0 };
}
interface SupplementalCounts {
sarif_findings: number;
sarif_dropped: number;
sent: number;
returned: number;
accepted: number;
}
function emptySupplementalCounts(): SupplementalCounts {
return { sarif_findings: 0, sarif_dropped: 0, sent: 0, returned: 0, accepted: 0 };
}
/**
* Names every finding that was sent for enrichment and did not come back paired.
*
* A count alone cannot say what was lost. Identity is recovered from the sent side by set
* difference rather than from the response, because a malformed response may carry no usable
* id at all — which is exactly what made it malformed.
*/
export function computeDroppedFindings(
findingsById: ReadonlyMap<number, ClassifiedFinding>,
paired: readonly { sastId: number }[],
vulnerabilityClass: ReconciliationClass,
): SupplementalDroppedFinding[] {
const pairedSastIds = new Set(paired.map(({ sastId }) => sastId));
const dropped = [...findingsById.entries()]
.filter(([sastId]) => !pairedSastIds.has(sastId))
.sort(([first], [second]) => first - second)
.map(([sastId, finding]) => ({
producer_id: mintSastProducerId(vulnerabilityClass, sastId),
sast_id: sastId,
sast_source_location: sourceLocationFromContext(finding.context),
}));
// Every sent finding is either accepted or named as dropped. A shortfall means the validator
// paired an id that was never sent, or paired one twice, which is an integrity fault rather
// than model output to accept.
if (dropped.length + paired.length !== findingsById.size) {
throw new ArtifactIntegrityError('SAST enrichment dropped-identity accounting failed');
}
return dropped;
}
async function writeSupplemental(
input: EnrichClassSastObservationsInput,
workspacesDir: string,
observations: ReconciliationObservation[],
drops: SupplementalDropCounts,
droppedFindings: SupplementalDroppedFinding[],
counts: SupplementalCounts,
metrics: StageMetrics,
): Promise<EnrichSuccess> {
const body: SupplementalObservationsBody = {
observations,
provenance: [],
...(input.sarif !== undefined && { sarif: input.sarif }),
drops,
dropped_findings: droppedFindings,
};
const ref = await writeArtifact({
sessionId: input.sessionId,
workspacesDir,
artifactKind: 'supplemental-observations',
vulnerabilityClass: input.vulnerabilityClass,
body,
inputs: [],
counts: { observations: observations.length, ...counts, ...drops },
});
return { ref, metrics };
}
function droppedFindingBelongsToClass(finding: DroppedSarifFinding, vulnerabilityClass: ReconciliationClass): boolean {
const mapping =
finding.ruleId === undefined
? unmappedMapping('malformed')
: (CWE_TO_CATEGORY[finding.ruleId] ?? unmappedMapping(finding.ruleId));
return mapping.category === vulnerabilityClassToCategory(vulnerabilityClass);
}
function classifyFindings(
findings: readonly ParsedSarifFinding[],
vulnerabilityClass: ReconciliationClass,
drops: SupplementalDropCounts,
): ClassifiedFinding[] {
const target = vulnerabilityClassToCategory(vulnerabilityClass);
const classified: ClassifiedFinding[] = [];
for (const finding of findings) {
const known = CWE_TO_CATEGORY[finding.result.ruleId];
const mapping = known ?? unmappedMapping(finding.result.ruleId);
if (mapping.category !== target) {
drops.other_category++;
continue;
}
if (known === undefined) drops.unknown_cwe++;
classified.push({ context: extractContext(finding.result), mapping, phase: finding.phase });
}
return classified;
}
async function defaultPromptLoader(vulnerabilityClass: ReconciliationClass, logger: ActivityLogger): Promise<string> {
return loadPrompt(
enrichmentPromptName(vulnerabilityClass),
{
webUrl: 'https://not-applicable.invalid',
repoPath: '/repo',
AUTH_STATE_FILE: '/tmp/auth-state.json',
},
null,
false,
logger,
);
}
function isCancellation(error: unknown, signal: AbortSignal | undefined): boolean {
if (signal?.aborted !== true) return false;
let current: unknown = error;
const seen = new Set<unknown>();
for (let depth = 0; depth < 8 && current !== undefined && current !== null && !seen.has(current); depth++) {
if (current === signal.reason) return true;
seen.add(current);
const errorName = current instanceof Error ? current.name : undefined;
if (errorName === 'AbortError' || errorName === 'CancelledFailure') return true;
current = current instanceof Error ? current.cause : undefined;
}
return false;
}
function isRetryableFileSystemError(error: unknown): boolean {
if (!(error instanceof Error)) return false;
const code = (error as NodeJS.ErrnoException).code;
if (
code !== undefined &&
['EACCES', 'EAGAIN', 'EBUSY', 'EIO', 'EMFILE', 'ENFILE', 'ENOMEM', 'ENOSPC', 'EPERM', 'EROFS', 'ESTALE'].includes(
code,
)
) {
return true;
}
return /\b(?:EACCES|EAGAIN|EBUSY|EIO|EMFILE|ENFILE|ENOMEM|ENOSPC|EPERM|EROFS|ESTALE)\b/u.test(error.message);
}
function metricsFailure(
onMetrics: ((metrics: StageMetrics) => void) | undefined,
metrics: StageMetrics,
): ReconciliationIoError | undefined {
try {
onMetrics?.({ ...metrics });
return undefined;
} catch {
return new ReconciliationIoError('Unable to record SAST enrichment usage');
}
}
/** Build the pure stage with explicit model, cancellation, metrics, and logging bindings. */
export function createEnrichClassSastObservations<TModelContext>(
deps: EnrichClassSastObservationsDeps<TModelContext>,
): (input: EnrichClassSastObservationsInput) => Promise<EnrichSuccess> {
return async function enrichClassSastObservations(input: EnrichClassSastObservationsInput): Promise<EnrichSuccess> {
const logger = deps.logger ?? NOOP_LOGGER;
const workspacesDir = deps.workspacesDir ?? WORKSPACES_DIR;
const drops = zeroDrops();
// 1. Absence is a successful, exact empty artifact and never resolves a model.
if (input.sarif === undefined) {
return writeSupplemental(input, workspacesDir, [], drops, [], emptySupplementalCounts(), zeroMetrics());
}
// 2. The reference is contained and rehashed before any JSON parsing.
let bytes: Buffer;
try {
bytes = await readPinnedSarif(input.sessionId, input.sarif, workspacesDir);
} catch (error) {
if (error instanceof SarifIntakeError) {
if (error.kind === 'io') {
throw new ReconciliationIoError('Unable to read the pinned SARIF input');
}
throw new SastEnrichmentInputError(error.message);
}
throw error;
}
// 3. Run/document violations fail the class. Per-finding violations drop only that finding.
let parsed: ParsedSarif;
try {
parsed = parseSarifContent(bytes.toString('utf8'));
} catch (error) {
if (error instanceof SarifDocumentError) throw new SastEnrichmentInputError(error.message);
throw error;
}
const classDropped = parsed.droppedFindings.filter((finding) =>
droppedFindingBelongsToClass(finding, input.vulnerabilityClass),
).length;
drops.malformed = classDropped;
const classified = classifyFindings(parsed.findings, input.vulnerabilityClass, drops);
const counts: SupplementalCounts = {
sarif_findings: classified.length,
sarif_dropped: classDropped,
sent: 0,
returned: 0,
accepted: 0,
};
// 4. A schema-valid empty current class batch is another zero-request path.
if (classified.length === 0) {
return writeSupplemental(input, workspacesDir, [], drops, [], counts, zeroMetrics());
}
const signal = deps.signalFor?.();
if (signal?.aborted === true) throw new SastEnrichmentCancelledError();
let prompt: string;
try {
prompt = deps.promptLoader
? await deps.promptLoader(input.vulnerabilityClass)
: await defaultPromptLoader(input.vulnerabilityClass, logger);
} catch (error) {
if (isRetryableFileSystemError(error)) {
throw new ReconciliationIoError('Unable to read the SAST enrichment prompt');
}
throw new SastEnrichmentInputError('SAST enrichment prompt is unavailable');
}
// `sastId` is a plain 0-based index into this batch, not a producer ID: the model only ever sees
// this small integer (as `_sastId` below), never the eventual SAST-namespaced producer ID that
// `mintSastProducerId` derives from it after a response comes back paired.
const idAssigned = classified.map((finding, index) => ({ sastId: index, finding }));
const findingsJson = JSON.stringify(
idAssigned.map(({ sastId, finding }) => ({ _sastId: sastId, ...finding.context })),
null,
2,
);
const findingsById = new Map(idAssigned.map(({ sastId, finding }) => [sastId, finding]));
const metrics = zeroMetrics();
counts.sent = classified.length;
// 5. One nonempty class batch makes exactly one billable request.
let outcome: SastEnrichmentBatchOutcome;
metrics.modelCalls = 1;
try {
outcome = await runSastEnrichmentBatch(deps.generation, deps.modelContextFor(input), {
vulnerabilityClass: input.vulnerabilityClass,
prompt,
findingsJson,
maxTokens: ENRICHMENT_MAX_TOKENS,
...(signal !== undefined && { signal }),
});
} catch (error) {
if (isCancellation(error, signal)) throw new SastEnrichmentCancelledError();
metricsFailure(deps.onMetrics, metrics);
throw new SastEnrichmentModelError('SAST enrichment request failed', { ...metrics });
}
metrics.costUsd = outcome.usage.costUsd;
metrics.inputTokens = outcome.usage.inputTokens;
metrics.outputTokens = outcome.usage.outputTokens;
// Record usage now so spend is captured even on a later abort or integrity failure, but hold any
// ledger error and rethrow it only after the model-outcome and accounting checks below, so a
// metrics-write fault never masks a real enrichment failure.
const usageLedgerFailure = metricsFailure(deps.onMetrics, metrics);
if (outcome.status === 'aborted') throw new SastEnrichmentCancelledError();
if (outcome.status === 'failed') {
throw new SastEnrichmentModelError(outcome.message, { ...metrics }, !outcome.terminal);
}
// 6. Pair by the code-owned id and account for every returned and sent element.
let validated: ValidationResult;
try {
validated = pairEnrichedVulnerabilities(outcome.vulnerabilities, findingsById, input.vulnerabilityClass);
} catch (error) {
if (error instanceof EnrichmentAttemptError) {
throw new SastEnrichmentModelError(error.message, { ...metrics });
}
throw error;
}
// Every returned element must land in exactly one bucket: paired, malformed, orphaned, or
// duplicate. If the buckets do not sum to the returned count, the validator dropped or
// double-counted something, which is an integrity fault rather than model output to accept.
const { paired, counts: validationCounts } = validated;
const accountedReturned =
paired.length + validationCounts.malformed + validationCounts.orphaned + validationCounts.duplicate_sast_id;
if (accountedReturned !== validationCounts.returned) {
throw new ArtifactIntegrityError('SAST enrichment returned-output accounting failed');
}
if (paired.length > classified.length) {
throw new ArtifactIntegrityError('SAST enrichment accepted more findings than were sent');
}
if (usageLedgerFailure !== undefined) throw usageLedgerFailure;
drops.malformed += validationCounts.malformed;
drops.orphaned = validationCounts.orphaned;
drops.duplicate_sast_id = validationCounts.duplicate_sast_id;
drops.enrichment_dropped = classified.length - paired.length;
counts.returned = validationCounts.returned;
counts.accepted = paired.length;
const droppedFindings = computeDroppedFindings(findingsById, paired, input.vulnerabilityClass);
const pairedInSarifOrder = [...paired].sort((first, second) => first.sastId - second.sastId);
const observations = pairedInSarifOrder.map(({ finding, sastId, evidence }) =>
buildSastObservation(mintSastProducerId(input.vulnerabilityClass, sastId), evidence, finding),
);
if (drops.enrichment_dropped > 0) {
const droppedSummary = droppedFindings
.map(
({ producer_id, sast_source_location }) =>
`${producer_id} (${sast_source_location.rule_id} at ${sast_source_location.file}:${sast_source_location.line})`,
)
.join(', ');
logger.warn(
`Static-analysis enrichment: ${drops.enrichment_dropped} of ${counts.sent} findings could not be enriched and were used as-is: ${droppedSummary}.`,
);
}
if (drops.unknown_cwe > 0) {
logger.info(
`Static-analysis enrichment: ${drops.unknown_cwe} findings had an unrecognised CWE and were grouped under "miscellaneous".`,
);
}
return writeSupplemental(input, workspacesDir, observations, drops, droppedFindings, counts, metrics);
};
}
+432
View File
@@ -0,0 +1,432 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Pass 1 task formation over current observations only. */
import path from 'node:path';
import { DEFAULT_DELIVERABLES_SUBDIR, WORKSPACES_DIR } from '../../paths.js';
import { loadPrompt } from '../../services/prompt-manager.js';
import type { ActivityLogger } from '../../types/activity-logger.js';
import type { ReconciliationClass } from '../../types/reconciliation.js';
import { materializeSourceJail } from '../pi/source-jail.js';
import {
isTaskFormationFallbackReason,
type TaskFormationExecutionContext,
type TaskFormationExecutor,
TaskFormationExecutorError,
type TaskFormationExecutorResult,
type TaskFormationFallbackReason,
type TaskFormationUsage,
taskFormationExecutor,
} from '../pi/task-formation-executor.js';
import { ArtifactIntegrityError, ReconciliationIoError, readArtifact, writeArtifact } from './artifact-store.js';
import type { ArtifactRef, ReconciliationObservation } from './contracts.js';
import { mintLabels } from './labels.js';
import { findLeakedProducerIds, toObservationView } from './observation-view.js';
import { combineObservations } from './observations.js';
import type { FormSuccess, StageMetrics, TaskFormationBody, TaskFormationInput } from './stage-contracts.js';
import { SINGLETON_FALLBACK } from './stage-contracts.js';
import { createValidatingSubmitTool } from './submit-validation.js';
import { acceptTaskGroups, buildTaskFormationSchema, findTaskFormationProblems } from './task-formation-schema.js';
// Two copies of the same pattern, not one shared regex: the global-flagged one is used with
// .replace() below, while the non-global one is used with .test(). A global regex carries a
// stateful lastIndex across calls to .test(), so reusing one instance for both would make a later
// leak check silently start scanning mid-string instead of from the beginning.
const INTERNAL_REFERENCE_PATTERN = /\b(?:AUTHZ|MISC|AUTH|INJ|XSS|SSRF)(?:-(?:VULN|SAST))?-[0-9]+\b/gu;
const INTERNAL_REFERENCE_LEAK_PATTERN = /\b(?:AUTHZ|MISC|AUTH|INJ|XSS|SSRF)(?:-(?:VULN|SAST))?-[0-9]+\b/u;
const TRANSIENT_IO_CODE_PATTERN = /\b(?:EAGAIN|EBUSY|EIO|EMFILE|ENFILE|ENOMEM|ENOSPC|EROFS|ETIMEDOUT)\b/u;
// The model-facing forbidden-key set for the task-formation boundary. It is a sibling of, but not
// identical to, `FORBIDDEN_PUBLISHED_KEYS` in publish.ts and prepare.ts: this one guards the
// pre-grouping observation view (which can still carry an `ID` or `merged_from` from an earlier
// stage's shape), while the published-queue set guards the post-materialization task shape. Each
// must independently list every internal-only key for its own boundary; neither can be derived from
// the other.
const FORBIDDEN_MODEL_KEYS = new Set([
'ID',
'_sastId',
'merged_from',
'novelty',
'observation_key',
'primary_preference',
'producer_id',
]);
export interface FormClassExploitTasksInput {
readonly sessionId: string;
readonly vulnerabilityClass: ReconciliationClass;
readonly repositoryPath: string;
readonly producerRef: ArtifactRef<'producer-observations'>;
readonly supplementalRef: ArtifactRef<'supplemental-observations'>;
readonly deliverablesSubdir?: string;
readonly webUrl?: string;
}
export interface FormClassExploitTasksResult extends FormSuccess {
readonly model?: string;
}
export interface FormClassExploitTasksDeps {
readonly executor?: TaskFormationExecutor;
readonly workspacesDir?: string;
readonly signalFor?: () => AbortSignal | undefined;
readonly executionContextFor?: () => TaskFormationExecutionContext | undefined;
readonly executorTimeoutMsFor?: () => number | undefined;
readonly onMetrics?: (metrics: StageMetrics) => void;
readonly logger?: ActivityLogger;
}
/** Failure that Temporal may retry and, only after exhaustion, classify for singleton fallback. */
export class TaskFormationModelError extends Error {
override readonly name = 'TaskFormationModelError';
readonly failureType = 'TaskFormationModelError' as const;
readonly retryable: boolean;
readonly fallbackReason: TaskFormationFallbackReason | undefined;
readonly metrics: StageMetrics;
constructor(options: {
message: string;
retryable: boolean;
fallbackReason?: TaskFormationFallbackReason;
metrics: StageMetrics;
}) {
super(options.message);
this.retryable = options.retryable;
this.fallbackReason = options.fallbackReason;
this.metrics = options.metrics;
}
}
const NOOP_LOGGER: ActivityLogger = {
info() {},
warn() {},
error() {},
};
function zeroMetrics(): StageMetrics {
return { costUsd: 0, modelCalls: 0, inputTokens: 0, outputTokens: 0 };
}
function metricsFromUsage(usage: TaskFormationUsage, modelCalls: number): StageMetrics {
return {
costUsd: usage.costUsd,
modelCalls,
inputTokens: usage.inputTokens,
outputTokens: usage.outputTokens,
};
}
function cancellationError(signal: AbortSignal): Error {
if (signal.reason instanceof Error) return signal.reason;
return new DOMException('Task formation was cancelled.', 'AbortError');
}
function checkCancellation(signal: AbortSignal | undefined): void {
if (signal?.aborted === true) throw cancellationError(signal);
}
// The depth cap bounds how far this walks an error's `cause`/`context`/`originalError` chain
// looking for a transient I/O code. It exists only to stop a pathological or circular chain from
// recursing forever; ordinary wrapped errors are a handful of layers deep at most.
function isTransientPromptIoFailure(error: unknown, depth = 0): boolean {
if (depth > 4) return false;
if (typeof error === 'string') return TRANSIENT_IO_CODE_PATTERN.test(error);
if (typeof error !== 'object' || error === null) return false;
if ('code' in error && typeof error.code === 'string' && TRANSIENT_IO_CODE_PATTERN.test(error.code)) return true;
if ('cause' in error && isTransientPromptIoFailure(error.cause, depth + 1)) return true;
if ('context' in error && isTransientPromptIoFailure(error.context, depth + 1)) return true;
if ('originalError' in error && isTransientPromptIoFailure(error.originalError, depth + 1)) return true;
return false;
}
// This is a distinct pass from the producer-ID redaction in observation-view.ts: that one redacts
// producer IDs it already knows about (because they were passed in), while this one redacts any
// string that merely looks like an internal class/reference token (e.g. an INJ-VULN-03-shaped
// substring), including one that might appear inside free-text evidence rather than as an ID field.
function scrubInternalReferences(value: unknown): unknown {
if (typeof value === 'string') return value.replace(INTERNAL_REFERENCE_PATTERN, '[redacted]');
if (Array.isArray(value)) return value.map(scrubInternalReferences);
if (value !== null && typeof value === 'object') {
const output: Record<string, unknown> = {};
for (const [key, item] of Object.entries(value)) output[key] = scrubInternalReferences(item);
return output;
}
return value;
}
function findForbiddenKeys(value: unknown, found: Set<string> = new Set()): Set<string> {
if (Array.isArray(value)) {
for (const item of value) findForbiddenKeys(item, found);
return found;
}
if (value !== null && typeof value === 'object') {
for (const [key, item] of Object.entries(value)) {
if (FORBIDDEN_MODEL_KEYS.has(key)) found.add(key);
findForbiddenKeys(item, found);
}
}
return found;
}
function taskFormationPromptName(vulnerabilityClass: ReconciliationClass): string {
return `task-formation-${vulnerabilityClass}`;
}
// A filesystem fault reading the prompt is retryable (the prompt file is expected to exist and a
// transient read failure should not fail the class outright), while any other failure means the
// prompt itself is missing or unreadable content, which Temporal should not spend retries on.
async function loadClassPolicy(
vulnerabilityClass: ReconciliationClass,
jailPath: string,
webUrl: string,
logger: ActivityLogger,
): Promise<string> {
try {
return await loadPrompt(
taskFormationPromptName(vulnerabilityClass),
{ webUrl, repoPath: jailPath, AUTH_STATE_FILE: '' },
null,
false,
logger,
);
} catch (error) {
if (isTransientPromptIoFailure(error)) {
throw new ReconciliationIoError('The fixed task-formation class policy prompt could not be read');
}
throw new ArtifactIntegrityError('The fixed task-formation class policy prompt is unavailable');
}
}
/**
* Build the exact prompt-facing input for task formation, so that the model that groups
* observations into exploitation tasks never sees a producer ID or other internal identity.
*
* Each observation gets a short opaque label (minted disjoint from every producer ID in this
* batch) that the model uses to refer to it instead of its real identity; the label-to-ID mapping
* stays code-side. This is the point in the pipeline where the internal-identity boundary is
* actually built, not merely checked: if this function stopped minting labels and passed producer
* IDs through instead, a downstream exploit agent reading the eventual published queue would learn
* exactly which scan producer (and by extension, which internal class/source combination) found
* each vulnerability, which the reconciliation contract exists to prevent.
*/
function buildModelInput(
observations: readonly ReconciliationObservation[],
vulnerabilityClass: ReconciliationClass,
): {
readonly input: TaskFormationInput;
readonly labelToProducerId: ReadonlyMap<string, string>;
readonly serialized: string;
} {
const producerIds = observations.map((observation) => observation.producer_id);
const labels = mintLabels(observations.length, { taken: new Set(producerIds) });
const labelToProducerId = new Map<string, string>();
const queued_findings = observations.map((observation, index) => {
const label = labels[index] as string;
labelToProducerId.set(label, observation.producer_id);
const projected = toObservationView(observation, vulnerabilityClass, producerIds);
return { label, entry: scrubInternalReferences(projected) };
}) as TaskFormationInput['queued_findings'];
const input: TaskFormationInput = { queued_findings };
// First gate: a structural check that no forbidden key name made it into the projected shape at
// all. The serialized-string check below is the second, independent gate against the same
// failure mode, catching a producer ID or reference token that leaked as a value rather than a key.
const forbiddenKeys = findForbiddenKeys(input);
if (forbiddenKeys.size > 0) {
throw new ArtifactIntegrityError('An internal key survived the task-formation positive projection');
}
// Final gate before the prompt bytes are built: fail closed if any producer ID or internal
// reference token survived projection and scrubbing, so the model never sees internal identity.
const serialized = JSON.stringify(input, null, 2);
if (findLeakedProducerIds(serialized, producerIds).length > 0 || INTERNAL_REFERENCE_LEAK_PATTERN.test(serialized)) {
throw new ArtifactIntegrityError('An internal reference survived the task-formation positive projection');
}
return { input, labelToProducerId, serialized };
}
function validateInputRefs(input: FormClassExploitTasksInput): void {
if (
input.producerRef.vulnerabilityClass !== input.vulnerabilityClass ||
input.supplementalRef.vulnerabilityClass !== input.vulnerabilityClass
) {
throw new ArtifactIntegrityError('Task formation received a cross-class artifact reference');
}
}
async function writeFormationArtifact(
input: FormClassExploitTasksInput,
workspacesDir: string,
body: TaskFormationBody,
): Promise<ArtifactRef<'task-formation'>> {
return writeArtifact({
sessionId: input.sessionId,
workspacesDir,
artifactKind: 'task-formation',
vulnerabilityClass: input.vulnerabilityClass,
body,
inputs: [
{ artifactKind: 'producer-observations', sha256: input.producerRef.sha256 },
{ artifactKind: 'supplemental-observations', sha256: input.supplementalRef.sha256 },
],
counts: {
accepted_groups: body.groups.length,
rejected_groups: body.rejected_group_count,
dropped_unknown_labels: body.dropped_unknown_label_count,
},
});
}
/** Whether an exhausted error is one of the three locked semantic-fallback cases. */
export function isSingletonFallbackEligible(error: unknown): error is TaskFormationModelError {
return (
error instanceof TaskFormationModelError && error.retryable && isTaskFormationFallbackReason(error.fallbackReason)
);
}
/** Return the sentinel only for an exhausted eligible model-stage failure; otherwise rethrow. */
export function singletonFallbackAfterExhaustion(error: unknown): typeof SINGLETON_FALLBACK {
if (isSingletonFallbackEligible(error)) return SINGLETON_FALLBACK;
throw error;
}
/** Build the Pass 1 stage with explicit executor, workspace, cancellation, and metric bindings. */
export function createFormClassExploitTasks(
deps: FormClassExploitTasksDeps = {},
): (input: FormClassExploitTasksInput) => Promise<FormClassExploitTasksResult> {
const executor = deps.executor ?? taskFormationExecutor;
const workspacesDir = deps.workspacesDir ?? WORKSPACES_DIR;
const logger = deps.logger ?? NOOP_LOGGER;
return async function formClassExploitTasks(input: FormClassExploitTasksInput): Promise<FormClassExploitTasksResult> {
validateInputRefs(input);
const signal = deps.signalFor?.();
checkCancellation(signal);
const producer = await readArtifact(input.producerRef, input.sessionId, workspacesDir);
const supplemental = await readArtifact(input.supplementalRef, input.sessionId, workspacesDir);
checkCancellation(signal);
const observations = combineObservations(producer.observations, supplemental.observations);
// Fewer than two observations can form no group, so skip the model entirely and write an empty
// formation with zero cost. This is one of the paths that leaves `model_ran` false.
if (observations.length < 2) {
const body: TaskFormationBody = {
model_ran: false,
groups: [],
rejected_group_count: 0,
dropped_unknown_label_count: 0,
};
const metrics = zeroMetrics();
const ref = await writeFormationArtifact(input, workspacesDir, body);
deps.onMetrics?.(metrics);
return { ref, metrics };
}
const modelInput = buildModelInput(observations, input.vulnerabilityClass);
const labelSet = new Set(modelInput.input.queued_findings.map(({ label }) => label));
const submitTool = createValidatingSubmitTool(buildTaskFormationSchema([...labelSet]), (parameters) =>
findTaskFormationProblems(parameters, labelSet),
);
const deliverablesPath = path.resolve(
input.repositoryPath,
input.deliverablesSubdir ?? DEFAULT_DELIVERABLES_SUBDIR,
);
const reconciliationWorkspacePath = path.resolve(workspacesDir, input.sessionId, '.shannon', 'reconciliation');
// Task formation runs against a disposable copy of the source tree rather than the live
// repository or the deliverables directory, so the model's tool calls during this stage cannot
// read or modify anything outside what it was actually given to reason about.
const jail = await materializeSourceJail({
sourceRoot: input.repositoryPath,
deliverablesPath,
reconciliationWorkspacePath,
...(signal !== undefined && { signal }),
});
try {
const classPolicy = await loadClassPolicy(
input.vulnerabilityClass,
jail.dir,
input.webUrl ?? 'https://not-applicable.invalid',
logger,
);
checkCancellation(signal);
let modelResult: TaskFormationExecutorResult;
try {
const executorTimeoutMs = deps.executorTimeoutMsFor?.();
modelResult = await executor.run({
cwd: jail.dir,
systemPrompt: classPolicy,
modelContext: modelInput.serialized,
deniedPaths: jail.deniedPaths,
submitTool,
signal: signal ?? new AbortController().signal,
...(executorTimeoutMs !== undefined && { timeoutMs: executorTimeoutMs }),
correlation: {
...deps.executionContextFor?.(),
stage: 'task-formation',
vulnerabilityClass: input.vulnerabilityClass,
},
});
} catch (error) {
if (!(error instanceof TaskFormationExecutorError)) throw error;
if (error.failureKind === 'infrastructure') {
throw new ReconciliationIoError(
'Task-formation executor setup encountered a retryable infrastructure failure',
);
}
if (error.failureKind !== 'model') throw error;
const metrics = metricsFromUsage(error.usage, error.modelCalls);
deps.onMetrics?.(metrics);
throw new TaskFormationModelError({
message: error.message,
retryable: error.retryable,
...(error.fallbackReason !== undefined && { fallbackReason: error.fallbackReason }),
metrics,
});
}
const metrics = metricsFromUsage(modelResult.usage, modelResult.modelCalls);
deps.onMetrics?.(metrics);
checkCancellation(signal);
const accepted = acceptTaskGroups(modelResult.output, labelSet);
const groups = accepted.groups.map((group) => ({
producer_ids: group.queue_labels.map((label) => {
const producerId = modelInput.labelToProducerId.get(label);
if (producerId === undefined) {
throw new ArtifactIntegrityError('An accepted task-formation label has no observation mapping');
}
return producerId;
}),
reasoning: group.reasoning,
}));
const body: TaskFormationBody = {
model_ran: true,
groups,
rejected_group_count: accepted.rejectedGroupCount,
dropped_unknown_label_count: accepted.droppedUnknownLabelCount,
};
const ref = await writeFormationArtifact(input, workspacesDir, body);
return { ref, metrics, model: `${modelResult.providerId}:${modelResult.modelId}` };
} finally {
// Cleanup runs on every exit path, but a cleanup failure must not overwrite the stage's real
// outcome. Log and swallow it so a successful formation stays successful and a failure keeps
// its original cause for Temporal to classify.
try {
await jail.cleanup();
} catch {
logger.error('Task-formation source-jail cleanup failed.', {
stage: 'task-formation',
vulnerabilityClass: input.vulnerabilityClass,
});
}
}
};
}
export const formClassExploitTasks = createFormClassExploitTasks();
@@ -0,0 +1,47 @@
/**
* Opaque, call-local labels used at reconciliation model boundaries.
*
* The model groups observations by these labels instead of by producer ID, so no internal producer
* identity has to cross into the prompt. The alphabet is consonants only, which keeps labels short
* and avoids accidentally spelling real words.
*/
export const LABEL_ALPHABET = 'bcdfghjkmnpqrstvwxz';
export const LABEL_LENGTH = 4;
export interface MintLabelsOptions {
taken?: ReadonlySet<string>;
rng?: () => number;
}
/** Mint distinct four-consonant labels disjoint from any supplied label space. */
export function mintLabels(count: number, options: MintLabelsOptions = {}): string[] {
if (!Number.isSafeInteger(count) || count < 0) {
throw new Error(`mintLabels: count must be a non-negative safe integer, received ${count}`);
}
// `taken` reserves label strings that must not be minted (for one class, the producer IDs
// themselves), so a minted label can never collide with a value already meaningful to the caller.
const used = new Set(options.taken);
const capacity = LABEL_ALPHABET.length ** LABEL_LENGTH;
if (count + used.size > capacity) {
throw new Error(`mintLabels: cannot mint ${count} labels; alphabet space is ${capacity}`);
}
const rng = options.rng ?? Math.random;
const labels: string[] = [];
while (labels.length < count) {
let label = '';
for (let index = 0; index < LABEL_LENGTH; index++) {
const sample = rng();
if (!Number.isFinite(sample) || sample < 0 || sample >= 1) {
throw new Error('mintLabels: rng must return a finite value in [0, 1)');
}
label += LABEL_ALPHABET[Math.floor(sample * LABEL_ALPHABET.length)];
}
if (used.has(label)) continue;
used.add(label);
labels.push(label);
}
return labels;
}
@@ -0,0 +1,230 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Validation and committed reads for the durable class-publication manifest. */
import { readCommittedFile } from '../../services/git-manager.js';
import { ALL_RECONCILIATION_CLASSES, type ReconciliationClass } from '../../types/reconciliation.js';
import type { PublicationContract } from './contracts.js';
import { mintTaskReferences } from './materialize-core.js';
import { isProducerId, isTaskReference } from './refs.js';
import { RECONCILIATION_SCHEMA_VERSION } from './schema-version.js';
const SHA256_PATTERN = /^[a-f0-9]{64}$/;
const GIT_BLOB_PATTERN = /^(?:[a-f0-9]{40}|[a-f0-9]{64})$/;
/** One committed consumer file and the digest the manifest vouches for. */
export interface ManifestConsumerFile {
path: string;
sha256: string;
}
/** One stable task's producer lineage. OSS omits `novelty`. */
export interface ManifestLineageEntry {
primary: string;
absorbed: string[];
novelty?: 'new' | 'recurring';
}
/** The schema-v1 durable completion marker for one class publication. */
export interface PublicationManifest {
session_id: string;
vulnerability_class: ReconciliationClass;
schema_version: 1;
producer_queue: { path: string; blob_sha: string };
consumer_files: ManifestConsumerFile[];
input_digests: Array<{ artifactKind: string; sha256: string }>;
lineage: Record<string, ManifestLineageEntry>;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function hasExactKeys(
value: Record<string, unknown>,
required: readonly string[],
optional: readonly string[] = [],
): boolean {
const allowed = new Set([...required, ...optional]);
const actual = Object.keys(value);
return required.every((key) => key in value) && actual.every((key) => allowed.has(key));
}
function isSafeRelativePath(value: unknown): value is string {
return (
typeof value === 'string' &&
value.length > 0 &&
!value.startsWith('/') &&
!value.startsWith('\\') &&
!value.includes('\0') &&
!value.split(/[\\/]/).some((segment) => segment === '' || segment === '.' || segment === '..')
);
}
function isProducerQueueIdentity(value: unknown): value is PublicationManifest['producer_queue'] {
if (!isRecord(value) || !hasExactKeys(value, ['path', 'blob_sha'])) return false;
return isSafeRelativePath(value.path) && typeof value.blob_sha === 'string' && GIT_BLOB_PATTERN.test(value.blob_sha);
}
function isConsumerFile(value: unknown): value is ManifestConsumerFile {
if (!isRecord(value) || !hasExactKeys(value, ['path', 'sha256'])) return false;
return isSafeRelativePath(value.path) && typeof value.sha256 === 'string' && SHA256_PATTERN.test(value.sha256);
}
function isInputDigest(value: unknown): value is PublicationManifest['input_digests'][number] {
if (!isRecord(value) || !hasExactKeys(value, ['artifactKind', 'sha256'])) return false;
return (
typeof value.artifactKind === 'string' &&
value.artifactKind.length > 0 &&
typeof value.sha256 === 'string' &&
SHA256_PATTERN.test(value.sha256)
);
}
function isProducerIdForClass(value: string, vulnerabilityClass: ReconciliationClass): boolean {
return isProducerId(value, vulnerabilityClass, 'VULN') || isProducerId(value, vulnerabilityClass, 'SAST');
}
function isLineageEntry(value: unknown, vulnerabilityClass: ReconciliationClass): value is ManifestLineageEntry {
if (!isRecord(value) || !hasExactKeys(value, ['primary', 'absorbed'], ['novelty'])) return false;
if (typeof value.primary !== 'string' || !isProducerIdForClass(value.primary, vulnerabilityClass)) return false;
if (
!Array.isArray(value.absorbed) ||
!value.absorbed.every(
(producerId) => typeof producerId === 'string' && isProducerIdForClass(producerId, vulnerabilityClass),
)
) {
return false;
}
return value.novelty === undefined || value.novelty === 'new' || value.novelty === 'recurring';
}
/** Whether a decoded value is a complete schema-v1 publication manifest. */
export function isManifest(value: unknown): value is PublicationManifest {
if (
!isRecord(value) ||
!hasExactKeys(value, [
'session_id',
'vulnerability_class',
'schema_version',
'producer_queue',
'consumer_files',
'input_digests',
'lineage',
])
) {
return false;
}
if (
typeof value.session_id !== 'string' ||
value.session_id.length === 0 ||
!ALL_RECONCILIATION_CLASSES.includes(value.vulnerability_class as ReconciliationClass) ||
value.schema_version !== RECONCILIATION_SCHEMA_VERSION ||
!isProducerQueueIdentity(value.producer_queue) ||
!Array.isArray(value.consumer_files) ||
!value.consumer_files.every(isConsumerFile) ||
!Array.isArray(value.input_digests) ||
!value.input_digests.every(isInputDigest) ||
!isRecord(value.lineage)
) {
return false;
}
const vulnerabilityClass = value.vulnerability_class as ReconciliationClass;
const consumerPaths = value.consumer_files.map((consumer) => consumer.path);
if (new Set(consumerPaths).size !== consumerPaths.length) return false;
// A coherent publication is derived from exactly the three stage artifacts, one of each kind.
// A different count or a repeated kind means the manifest was not built from a complete lineage.
const inputKinds = value.input_digests.map((input) => input.artifactKind);
if (inputKinds.length !== 3 || new Set(inputKinds).size !== inputKinds.length) return false;
if (
!['producer-observations', 'supplemental-observations', 'fixed-tasks'].every((kind) => inputKinds.includes(kind))
) {
return false;
}
// Lineage keys must be the dense minted references PREFIX-01..PREFIX-NN in order, and every
// producer ID across all entries must be unique. This is the same task numbering the published
// queue carries, so a manifest that renumbers or repeats a producer cannot pass.
const lineageEntries = Object.entries(value.lineage);
const expectedTaskReferences = mintTaskReferences(lineageEntries.length, vulnerabilityClass);
const producerIds = new Set<string>();
for (const [index, [taskReference, entry]] of lineageEntries.entries()) {
if (
!isTaskReference(taskReference, vulnerabilityClass) ||
taskReference !== expectedTaskReferences[index] ||
!isLineageEntry(entry, vulnerabilityClass)
) {
return false;
}
const typedEntry = entry as ManifestLineageEntry;
for (const producerId of [typedEntry.primary, ...typedEntry.absorbed]) {
if (producerIds.has(producerId)) return false;
producerIds.add(producerId);
}
}
return true;
}
/** Classified outcome of reading a class manifest from Git `HEAD`. */
export type ManifestRead =
| { state: 'absent' }
| { state: 'invalid'; reason: string }
| { state: 'present'; manifest: PublicationManifest; contents: string };
/** Read and strictly validate one committed manifest. */
export async function readPublishedManifest(
deliverablesDirPath: string,
manifestRelPath: string,
): Promise<ManifestRead> {
const read = await readCommittedFile(deliverablesDirPath, manifestRelPath);
if (read.state === 'absent') return { state: 'absent' };
if (read.state === 'corrupt') {
return { state: 'invalid', reason: 'manifest object is unreadable' };
}
let parsed: unknown;
try {
parsed = JSON.parse(read.contents);
} catch {
return { state: 'invalid', reason: 'manifest is not valid JSON' };
}
if (!isManifest(parsed)) {
return { state: 'invalid', reason: 'manifest is truncated, malformed, or contains unexpected fields' };
}
return { state: 'present', manifest: parsed, contents: read.contents };
}
function samePathSet(actual: readonly string[], expected: readonly string[]): boolean {
if (actual.length !== expected.length) return false;
const actualSet = new Set(actual);
return actualSet.size === actual.length && expected.every((path) => actualSet.has(path));
}
/** Whether a manifest exactly matches the expected OSS publication identity and path set. */
export function isManifestCoherent(args: {
manifest: PublicationManifest;
sessionId: string;
vulnerabilityClass: ReconciliationClass;
contract: PublicationContract;
producerQueuePath: string;
producerBlobSha?: string;
}): boolean {
const { manifest, sessionId, vulnerabilityClass, contract, producerQueuePath, producerBlobSha } = args;
if (contract.publicationKind !== 'class-reconciliation') return false;
if (contract.schemaVersion !== RECONCILIATION_SCHEMA_VERSION) return false;
if (manifest.session_id !== sessionId) return false;
if (manifest.vulnerability_class !== vulnerabilityClass) return false;
if (manifest.schema_version !== contract.schemaVersion) return false;
if (manifest.producer_queue.path !== producerQueuePath) return false;
if (producerBlobSha !== undefined && manifest.producer_queue.blob_sha !== producerBlobSha) return false;
return samePathSet(
manifest.consumer_files.map((consumer) => consumer.path),
contract.requiredOutputPaths,
);
}
@@ -0,0 +1,179 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Pure deterministic collapse, ordering, and reference assignment for class tasks. */
import type { ReconciliationClass } from '../../types/reconciliation.js';
import { ArtifactIntegrityError } from './artifact-store.js';
import type {
MergedObservation,
PrimaryPreference,
ReconciliationObservation,
ReconciliationTask,
} from './contracts.js';
import { REF_PREFIX } from './refs.js';
const PRIORITY_ORDER: Readonly<Record<string, number>> = Object.freeze({ P1: 0, P2: 1, P3: 2 });
const CONFIDENCE_ORDER: Readonly<Record<string, number>> = Object.freeze({ high: 0, medium: 1, low: 2 });
const MISSING_PRIORITY_RANK = 2;
const MISSING_CONFIDENCE_RANK = 2;
type PreTask = Omit<ReconciliationTask, 'ID'>;
/** Ordered tasks, complete lineage, and merged-member accounting. */
export interface FixedTasks {
tasks: ReconciliationTask[];
observationToTask: Record<string, string>;
mergedFromTotal: number;
}
/** Mint dense references `PREFIX-01..PREFIX-NN`, continuing unpadded past 99. */
export function mintTaskReferences(count: number, vulnerabilityClass: ReconciliationClass): string[] {
const references: string[] = [];
for (let index = 1; index <= count; index++) {
references.push(`${REF_PREFIX[vulnerabilityClass]}-${String(index).padStart(2, '0')}`);
}
return references;
}
function rank(value: unknown, order: Readonly<Record<string, number>>, fallback: number): number {
if (typeof value !== 'string') return fallback;
return order[value] ?? fallback;
}
// The strongest priority or confidence across a merged group of observations wins for the task,
// rather than the primary observation's own value. A weaker duplicate finding should not water
// down a stronger signal one of the other producers already established for the same vulnerability.
function strongest(
members: readonly ReconciliationObservation[],
field: 'priority' | 'confidence',
order: Readonly<Record<string, number>>,
): string | undefined {
let best: string | undefined;
let bestRank = Number.POSITIVE_INFINITY;
for (const member of members) {
const value = (member as unknown as Record<string, unknown>)[field];
if (typeof value !== 'string') continue;
const valueRank = rank(value, order, Number.POSITIVE_INFINITY);
if (valueRank < bestRank) {
best = value;
bestRank = valueRank;
}
}
return best;
}
// A `preferred` member (a SAST producer) outranks a `default` one, so a group that pairs a SAST
// finding with a vulnerability-analysis finding keeps the SAST observation as the task primary.
function preferenceRank(preference: PrimaryPreference): number {
return preference === 'preferred' ? 0 : 1;
}
function primaryIndex(members: readonly ReconciliationObservation[]): number {
if (members.length === 0) {
throw new ArtifactIntegrityError('Cannot materialize an empty observation group');
}
let selected = 0;
let selectedRank = preferenceRank(members[0]?.primary_preference ?? 'default');
for (let index = 1; index < members.length; index++) {
const member = members[index];
if (member === undefined) continue;
const memberRank = preferenceRank(member.primary_preference);
if (memberRank < selectedRank) {
selected = index;
selectedRank = memberRank;
}
}
return selected;
}
// `primary_preference` has already done its job by the time a group reaches this function: it
// picked the primary observation via `primaryIndex` above. Dropping it here, rather than carrying
// it into the task, is what the `MaterializedProducerFields` type in contracts.ts enforces at
// compile time; nothing downstream of materialization is allowed to see or re-derive this preference.
function toMaterialized(observation: ReconciliationObservation): MergedObservation {
const { primary_preference: _primaryPreference, ...materialized } = observation;
return materialized as MergedObservation;
}
/** Collapse one member group into a single pre-task: pick the primary, fold the rest as `merged_from`. */
function buildTask(members: readonly ReconciliationObservation[]): PreTask {
const selectedIndex = primaryIndex(members);
const primary = members[selectedIndex];
if (primary === undefined) {
throw new ArtifactIntegrityError('Materialization selected a missing primary observation');
}
const merged = members.filter((_member, index) => index !== selectedIndex);
const priority = strongest(members, 'priority', PRIORITY_ORDER);
const confidence = strongest(members, 'confidence', CONFIDENCE_ORDER);
const task: Record<string, unknown> = { ...toMaterialized(primary) };
if (priority !== undefined) task.priority = priority;
if (confidence !== undefined) task.confidence = confidence;
if (merged.length > 0) task.merged_from = merged.map(toMaterialized);
return task as PreTask;
}
function compareCodeUnits(first: string, second: string): number {
if (first < second) return -1;
if (first > second) return 1;
return 0;
}
function taskField(task: PreTask, field: string): unknown {
return (task as unknown as Record<string, unknown>)[field];
}
// Total order over tasks: priority, then confidence, then producer ID as a final tie-break. The
// producer-ID comparison guarantees no two tasks ever compare equal, so the sort is fully
// deterministic and the dense reference assignment below is reproducible across retries.
function compareTasks(first: PreTask, second: PreTask): number {
const priorityDifference =
rank(taskField(first, 'priority'), PRIORITY_ORDER, MISSING_PRIORITY_RANK) -
rank(taskField(second, 'priority'), PRIORITY_ORDER, MISSING_PRIORITY_RANK);
if (priorityDifference !== 0) return priorityDifference;
const confidenceDifference =
rank(taskField(first, 'confidence'), CONFIDENCE_ORDER, MISSING_CONFIDENCE_RANK) -
rank(taskField(second, 'confidence'), CONFIDENCE_ORDER, MISSING_CONFIDENCE_RANK);
if (confidenceDifference !== 0) return confidenceDifference;
const firstProducer = taskField(first, 'producer_id');
const secondProducer = taskField(second, 'producer_id');
return compareCodeUnits(
typeof firstProducer === 'string' ? firstProducer : '',
typeof secondProducer === 'string' ? secondProducer : '',
);
}
/** Collapse member groups, sort by the locked total order, and assign dense stable references. */
export function buildFixedTasks(
memberGroups: ReadonlyArray<readonly ReconciliationObservation[]>,
vulnerabilityClass: ReconciliationClass,
): FixedTasks {
const preTasks = memberGroups.map(buildTask);
preTasks.sort(compareTasks);
const references = mintTaskReferences(preTasks.length, vulnerabilityClass);
const tasks = preTasks.map(
(task, index) =>
({
...(task as unknown as Record<string, unknown>),
ID: references[index] as string,
}) as ReconciliationTask,
);
const observationToTask: Record<string, string> = Object.create(null);
let mergedFromTotal = 0;
for (const task of tasks) {
observationToTask[task.producer_id] = task.ID;
for (const member of task.merged_from ?? []) {
observationToTask[member.producer_id] = task.ID;
mergedFromTotal++;
}
}
return { tasks, observationToTask, mergedFromTotal };
}
@@ -0,0 +1,213 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Revalidate stage artifacts and materialize every observation into one fixed task. */
import { Check } from 'typebox/value';
import type { ReconciliationClass } from '../../types/reconciliation.js';
import { classEntrySchema, QUEUE_ENTRY_FIELD_NAMES } from '../queue-schemas.js';
import { ArtifactIntegrityError, artifactInputsMatch, readArtifact, writeArtifact } from './artifact-store.js';
import type { ArtifactInputDigest, ArtifactRef, ReconciliationObservation } from './contracts.js';
import { buildFixedTasks } from './materialize-core.js';
import { combineObservations } from './observations.js';
import { isProducerId } from './refs.js';
import { type FixedTasksBody, type FormResult, type MaterializeResult, SINGLETON_FALLBACK } from './stage-contracts.js';
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function validateSastLocation(value: unknown): boolean {
return (
isRecord(value) &&
typeof value.file === 'string' &&
value.file.length > 0 &&
Number.isSafeInteger(value.line) &&
(value.line as number) > 0 &&
Number.isSafeInteger(value.column) &&
(value.column as number) > 0 &&
typeof value.rule_id === 'string' &&
value.rule_id.length > 0
);
}
// Defense in depth: an earlier stage already validated this observation's shape, but materialization
// is the last stop before publication, so it revalidates independently rather than trusting an
// artifact read back off disk. Re-derives the evidence subset and schema-checks it, then confirms no
// field outside the declared class/source contract survived (including any internal key a bug in an
// earlier stage might have let through).
function validateEvidenceShape(observation: ReconciliationObservation, vulnerabilityClass: ReconciliationClass): void {
const record = observation as unknown as Record<string, unknown>;
const evidence: Record<string, unknown> = { ID: observation.producer_id };
for (const key of QUEUE_ENTRY_FIELD_NAMES[vulnerabilityClass]) {
if (key !== 'ID' && key in record) evidence[key] = record[key];
}
if (!Check(classEntrySchema(vulnerabilityClass), evidence)) {
throw new ArtifactIntegrityError('Observation evidence does not match its declared class schema');
}
const allowed = new Set<string>([
...QUEUE_ENTRY_FIELD_NAMES[vulnerabilityClass].filter((key) => key !== 'ID'),
'producer_id',
'scan_source',
'primary_preference',
...(observation.scan_source === 'sast' ? ['priority', 'sast_source_location'] : []),
]);
if (Object.keys(record).some((key) => !allowed.has(key))) {
throw new ArtifactIntegrityError('Observation contains fields outside its class/source contract');
}
}
/** Reconfirm producer identity and evidence shape match the observation's declared source and class. */
function validateObservation(observation: ReconciliationObservation, vulnerabilityClass: ReconciliationClass): void {
validateEvidenceShape(observation, vulnerabilityClass);
if (observation.scan_source === 'vulnerability_analysis') {
if (
observation.primary_preference !== 'default' ||
!isProducerId(observation.producer_id, vulnerabilityClass, 'VULN')
) {
throw new ArtifactIntegrityError('Vulnerability-analysis observation has an invalid class/source identity');
}
return;
}
if (observation.scan_source !== 'sast') {
throw new ArtifactIntegrityError('Observation has an unknown producer source');
}
if (
observation.primary_preference !== 'preferred' ||
!isProducerId(observation.producer_id, vulnerabilityClass, 'SAST') ||
!['P1', 'P2', 'P3'].includes(observation.priority) ||
!validateSastLocation(observation.sast_source_location)
) {
throw new ArtifactIntegrityError('SAST observation has an invalid class/source identity or annotation');
}
}
function partitionMembers(
observations: readonly ReconciliationObservation[],
groups: readonly unknown[],
): ReconciliationObservation[][] {
const byProducerId = new Map(observations.map((observation) => [observation.producer_id, observation]));
const assigned = new Set<string>();
const memberGroups: ReconciliationObservation[][] = [];
for (const rawGroup of groups) {
if (
!isRecord(rawGroup) ||
Object.keys(rawGroup).some((key) => key !== 'producer_ids' && key !== 'reasoning') ||
!Array.isArray(rawGroup.producer_ids) ||
!rawGroup.producer_ids.every((producerId) => typeof producerId === 'string') ||
rawGroup.producer_ids.length < 2 ||
typeof rawGroup.reasoning !== 'string' ||
rawGroup.reasoning.length === 0
) {
throw new ArtifactIntegrityError('Accepted task formation contains a malformed group');
}
const group = rawGroup as { producer_ids: string[]; reasoning: string };
const members: ReconciliationObservation[] = [];
const groupIds = new Set<string>();
for (const producerId of group.producer_ids) {
const observation = byProducerId.get(producerId);
if (observation === undefined || groupIds.has(producerId) || assigned.has(producerId)) {
throw new ArtifactIntegrityError('Accepted task formation has unknown, duplicate, or reused membership');
}
groupIds.add(producerId);
members.push(observation);
}
for (const producerId of groupIds) assigned.add(producerId);
memberGroups.push(members);
}
// Every observation the model did not place into a group becomes its own singleton task. This is
// also exactly the whole-set result when task formation is skipped, so the fallback path and the
// model path converge on the same shape: one task per unmerged observation.
for (const observation of observations) {
if (!assigned.has(observation.producer_id)) memberGroups.push([observation]);
}
return memberGroups;
}
function assertRefClass(ref: ArtifactRef, vulnerabilityClass: ReconciliationClass): void {
if (ref.vulnerabilityClass !== vulnerabilityClass) {
throw new ArtifactIntegrityError('Artifact reference crosses the declared class boundary');
}
}
export interface MaterializeClassExploitTasksArgs {
sessionId: string;
workspacesDir?: string;
vulnerabilityClass: ReconciliationClass;
producerRef: ArtifactRef<'producer-observations'>;
supplementalRef: ArtifactRef<'supplemental-observations'>;
form: FormResult;
}
/**
* Materialize one class and publish its content-addressed `03-fixed-tasks` artifact.
*
* Combines the producer and supplemental observations, applies the accepted formation groups (or
* treats every observation as its own singleton task when formation fell back), collapses each
* group into one task via `buildFixedTasks`, and confirms the resulting observation-to-task map
* covers every observation exactly once before returning.
*/
export async function materializeClassExploitTasks(args: MaterializeClassExploitTasksArgs): Promise<MaterializeResult> {
assertRefClass(args.producerRef, args.vulnerabilityClass);
assertRefClass(args.supplementalRef, args.vulnerabilityClass);
if (args.form !== SINGLETON_FALLBACK) assertRefClass(args.form.ref, args.vulnerabilityClass);
const producer = await readArtifact(args.producerRef, args.sessionId, args.workspacesDir);
const supplemental = await readArtifact(args.supplementalRef, args.sessionId, args.workspacesDir);
if (!Array.isArray(producer.observations) || !Array.isArray(supplemental.observations)) {
throw new ArtifactIntegrityError('Observation artifact body is truncated or malformed');
}
const observations = combineObservations(producer.observations, supplemental.observations);
for (const observation of observations) validateObservation(observation, args.vulnerabilityClass);
const observationInputs: ArtifactInputDigest[] = [
{ artifactKind: 'producer-observations', sha256: args.producerRef.sha256 },
{ artifactKind: 'supplemental-observations', sha256: args.supplementalRef.sha256 },
];
// With the singleton fallback there are no groups, so every observation materializes alone. With a
// real formation artifact, its lineage must name these exact observation digests, or it grouped a
// different observation set than the one being materialized here.
let groups: readonly unknown[] = [];
if (args.form !== SINGLETON_FALLBACK) {
if (!artifactInputsMatch(args.form.ref.inputs, observationInputs)) {
throw new ArtifactIntegrityError('Task-formation lineage does not match the supplied observations');
}
const formation = await readArtifact(args.form.ref, args.sessionId, args.workspacesDir);
if (!Array.isArray(formation.groups)) {
throw new ArtifactIntegrityError('Accepted task formation has no groups array');
}
groups = formation.groups;
}
const memberGroups = partitionMembers(observations, groups);
const fixed = buildFixedTasks(memberGroups, args.vulnerabilityClass);
if (Object.keys(fixed.observationToTask).length !== observations.length) {
throw new ArtifactIntegrityError('Observation-to-task map is incomplete');
}
const inputs: ArtifactInputDigest[] = [...observationInputs];
if (args.form !== SINGLETON_FALLBACK) {
inputs.push({ artifactKind: 'task-formation', sha256: args.form.ref.sha256 });
}
const body: FixedTasksBody = {
tasks: fixed.tasks,
observation_to_task: fixed.observationToTask,
};
const ref = await writeArtifact({
sessionId: args.sessionId,
...(args.workspacesDir !== undefined ? { workspacesDir: args.workspacesDir } : {}),
artifactKind: 'fixed-tasks',
vulnerabilityClass: args.vulnerabilityClass,
body,
inputs,
counts: { tasks: fixed.tasks.length, merged_from_total: fixed.mergedFromTotal },
});
return { ref };
}
@@ -0,0 +1,109 @@
/** Positive observation projection used at the task-formation model boundary. */
import type { ReconciliationClass } from '../../types/reconciliation.js';
import { QUEUE_ENTRY_FIELD_NAMES } from '../queue-schemas.js';
import type { ReconciliationObservation } from './contracts.js';
import type { ObservationView } from './stage-contracts.js';
const BOOLEAN_EVIDENCE_KEY = 'externally_exploitable';
const PRODUCER_ID_REDACTION = '[redacted]';
function redactString(value: string, producerIds: readonly string[]): string {
let redacted = value;
for (const producerId of producerIds) {
if (redacted.includes(producerId)) {
redacted = redacted.split(producerId).join(PRODUCER_ID_REDACTION);
}
}
return redacted;
}
/** Redact producer-ID tokens from arbitrary free text. */
export function redactProducerIds(value: string, producerIds: Iterable<string>): string {
const sorted = [...new Set(producerIds)].filter((id) => id.length > 0).sort((a, b) => b.length - a.length);
return redactString(value, sorted);
}
function redactDeep(value: unknown, producerIds: readonly string[]): unknown {
if (typeof value === 'string') return redactString(value, producerIds);
if (Array.isArray(value)) return value.map((item) => redactDeep(item, producerIds));
if (value !== null && typeof value === 'object') {
const output: Record<string, unknown> = {};
for (const [key, item] of Object.entries(value)) {
output[key] = redactDeep(item, producerIds);
}
return output;
}
return value;
}
// Rebuilds the location from scratch rather than passing the stored value through, so a malformed
// or tampered `sast_source_location` on the underlying observation cannot cross into the model view
// unnoticed; anything that fails this shape check (including a `rule_id` that isn't a real CWE
// identifier) is silently dropped from the view rather than surfaced as-is.
function rebuildSastSourceLocation(value: unknown): ObservationView['sast_source_location'] {
if (value === null || typeof value !== 'object' || Array.isArray(value)) return undefined;
const location = value as Record<string, unknown>;
if (
typeof location.file === 'string' &&
location.file.length > 0 &&
Number.isSafeInteger(location.line) &&
(location.line as number) > 0 &&
Number.isSafeInteger(location.column) &&
(location.column as number) > 0 &&
typeof location.rule_id === 'string' &&
/^CWE-[1-9][0-9]*$/.test(location.rule_id)
) {
return {
file: location.file,
line: location.line as number,
column: location.column as number,
rule_id: location.rule_id,
};
}
return undefined;
}
/** Return producer IDs still present in a fully serialized model context. */
export function findLeakedProducerIds(modelContext: string, producerIds: Iterable<string>): string[] {
const leaked: string[] = [];
for (const producerId of new Set(producerIds)) {
if (producerId.length > 0 && modelContext.includes(producerId)) leaked.push(producerId);
}
return leaked;
}
/**
* Rebuild one model-visible observation from declared primitive evidence only.
* Internal keys and non-primitive evidence never cross this boundary.
*/
export function toObservationView(
observation: ReconciliationObservation,
vulnerabilityClass: ReconciliationClass,
producerIds: Iterable<string>,
): ObservationView {
const source = observation as unknown as Record<string, unknown>;
const view: Record<string, unknown> = {};
for (const key of QUEUE_ENTRY_FIELD_NAMES[vulnerabilityClass]) {
if (key === 'ID') continue;
const value = source[key];
if (key === BOOLEAN_EVIDENCE_KEY) {
if (typeof value === 'boolean') view[key] = value;
continue;
}
if (typeof value === 'string') view[key] = value;
}
if (source.scan_source === 'vulnerability_analysis' || source.scan_source === 'sast') {
view.scan_source = source.scan_source;
}
if (source.priority === 'P1' || source.priority === 'P2' || source.priority === 'P3') {
view.priority = source.priority;
}
const location = rebuildSastSourceLocation(source.sast_source_location);
if (location !== undefined) view.sast_source_location = location;
const sortedIds = [...new Set(producerIds)].filter((id) => id.length > 0).sort((a, b) => b.length - a.length);
return redactDeep(view, sortedIds) as ObservationView;
}
@@ -0,0 +1,20 @@
/** Utilities shared by task formation and materialization observation intake. */
import { ArtifactIntegrityError } from './artifact-store.js';
import type { ReconciliationObservation } from './contracts.js';
/** Concatenate producer and supplemental observations while enforcing one global producer-ID set. */
export function combineObservations(
producer: readonly ReconciliationObservation[],
supplemental: readonly ReconciliationObservation[],
): ReconciliationObservation[] {
const combined = [...producer, ...supplemental];
const seen = new Set<string>();
for (const observation of combined) {
if (seen.has(observation.producer_id)) {
throw new ArtifactIntegrityError('Reconciliation observations contain a duplicate producer identifier');
}
seen.add(observation.producer_id);
}
return combined;
}
@@ -0,0 +1,379 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Admit committed producer observations or a complete existing class publication. */
import { createHash } from 'node:crypto';
import { lstat } from 'node:fs/promises';
import path from 'node:path';
import { Check } from 'typebox/value';
import {
blobShaFromHead,
readCommittedFile,
restorePathsFromHead,
withGitRepoLock,
} from '../../services/git-manager.js';
import type { ReconciliationClass } from '../../types/reconciliation.js';
import { classEntrySchema, QUEUE_ENTRY_FIELD_NAMES } from '../queue-schemas.js';
import {
ArtifactIntegrityError,
PublicationConflictError,
ReconciliationError,
ReconciliationIoError,
writeArtifact,
} from './artifact-store.js';
import type { PublicationContract, ReconciliationObservation } from './contracts.js';
import { isManifestCoherent, type PublicationManifest, readPublishedManifest } from './manifest.js';
import { isProducerId, isTaskReference } from './refs.js';
import type { PrepareResult, ProducerObservationsBody } from './stage-contracts.js';
// Duplicated from the identical set in publish.ts, which guards the fresh-publish path; this copy
// guards the lost-acknowledgement repair path below, where an already-published queue is read back
// and re-verified before being trusted. Both sets must list the same internal-only keys, or a key
// added to only one path could round-trip a leaked queue back into "coherent" on the other.
const FORBIDDEN_PUBLISHED_KEYS: ReadonlySet<string> = new Set([
'producer_id',
'primary_preference',
'observation_key',
'novelty',
'_sastId',
'_sast_id',
'repository_id',
'scan_run_id',
]);
function sha256Text(contents: string): string {
return createHash('sha256').update(contents, 'utf8').digest('hex');
}
function isErrno(error: unknown, code: string): boolean {
return error instanceof Error && (error as NodeJS.ErrnoException).code === code;
}
// Same defense as `preflightSymlinks` in publish.ts: a symlinked destination could redirect a write
// (here, the restore step below) outside the deliverables directory, so any symlink found is a
// conflict rather than something to write through.
async function rejectSymlinkDestinations(deliverablesDir: string, relativePaths: readonly string[]): Promise<void> {
for (const relativePath of relativePaths) {
try {
const stat = await lstat(path.join(deliverablesDir, relativePath));
if (stat.isSymbolicLink()) throw new PublicationConflictError('Refusing to repair a publication symlink');
} catch (error) {
if (isErrno(error, 'ENOENT')) continue;
if (error instanceof ReconciliationError) throw error;
throw new ReconciliationIoError('Unable to inspect a class publication destination');
}
}
}
/** The class-owned producer and published queue path. */
export function exploitationQueuePath(vulnerabilityClass: ReconciliationClass): string {
return `${vulnerabilityClass}_exploitation_queue.json`;
}
/** The class-owned durable completion-marker path. */
export function reconciliationManifestPath(vulnerabilityClass: ReconciliationClass): string {
return `${vulnerabilityClass}_reconciliation_manifest.json`;
}
/** The conditional standalone SAST-provenance path. */
export function sastProvenancePath(vulnerabilityClass: ReconciliationClass): string {
return `sast_provenance_${vulnerabilityClass}.json`;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
// A raw producer queue carries producer IDs and no merge structure. A task reference or a
// `merged_from` array means the queue was already normalized by a prior publication whose manifest
// is now missing, which is an incoherent state to be reported rather than reprocessed.
function looksNormalized(entry: Record<string, unknown>, vulnerabilityClass: ReconciliationClass): boolean {
const id = entry.ID;
return (typeof id === 'string' && isTaskReference(id, vulnerabilityClass)) || Array.isArray(entry.merged_from);
}
/**
* Parse and strictly validate a committed producer queue before reconciliation ever touches it.
*
* Every entry must match its class's declared evidence schema and carry an ID in the VULN producer
* namespace for this exact class; a queue that already looks normalized (see `looksNormalized`) or
* carries a duplicate ID is rejected outright. The producer IDs minted or validated here are
* internal identity: they exist to let reconciliation reason about and dedupe observations, and are
* scrubbed before anything derived from this queue is published.
*/
function parseProducerQueue(raw: string, vulnerabilityClass: ReconciliationClass): Record<string, unknown>[] {
let parsed: unknown;
try {
parsed = JSON.parse(raw);
} catch {
throw new ArtifactIntegrityError('Producer queue is not valid JSON');
}
if (!isRecord(parsed) || !Array.isArray(parsed.vulnerabilities)) {
throw new ArtifactIntegrityError('Producer queue has no vulnerabilities array');
}
const schema = classEntrySchema(vulnerabilityClass);
const seenIds = new Set<string>();
for (const entry of parsed.vulnerabilities) {
if (!isRecord(entry) || typeof entry.ID !== 'string') {
throw new ArtifactIntegrityError('Producer queue entry is missing a string ID');
}
if (looksNormalized(entry, vulnerabilityClass)) {
throw new PublicationConflictError('Normalized queue in HEAD has no coherent publication manifest');
}
if (!Check(schema, entry)) {
throw new ArtifactIntegrityError(`Producer queue entry does not match the ${vulnerabilityClass} schema`);
}
if (!isProducerId(entry.ID, vulnerabilityClass, 'VULN')) {
throw new ArtifactIntegrityError('Producer queue entry is outside its declared class/source namespace');
}
if (seenIds.has(entry.ID)) {
throw new ArtifactIntegrityError('Producer queue contains a duplicate producer identifier');
}
seenIds.add(entry.ID);
}
return parsed.vulnerabilities as Record<string, unknown>[];
}
// Every producer-queue (pentest) observation is stamped `primary_preference: 'default'`, the losing
// side of the dedupe contract: if this observation is later merged with a SAST observation for the
// same vulnerability, the SAST evidence becomes the task's primary record instead of this one.
function toObservation(entry: Record<string, unknown>, evidenceKeys: readonly string[]): ReconciliationObservation {
const evidence: Record<string, unknown> = {};
for (const key of evidenceKeys) {
if (key in entry) evidence[key] = entry[key];
}
return {
...evidence,
producer_id: entry.ID as string,
scan_source: 'vulnerability_analysis',
primary_preference: 'default',
} as ReconciliationObservation;
}
async function verifyCommittedConsumers(
deliverablesDir: string,
consumerFiles: ReadonlyArray<{ path: string; sha256: string }>,
): Promise<Map<string, string>> {
const contentsByPath = new Map<string, string>();
for (const consumer of consumerFiles) {
const committed = await readCommittedFile(deliverablesDir, consumer.path);
if (committed.state !== 'present') {
throw new PublicationConflictError('Existing class publication is missing a committed consumer');
}
if (sha256Text(committed.contents) !== consumer.sha256) {
throw new PublicationConflictError('Existing class publication consumer does not match its manifest');
}
contentsByPath.set(consumer.path, committed.contents);
}
return contentsByPath;
}
// Used only on the repair path below, where a manifest already exists in HEAD and the previously
// published queue is being read back rather than freshly built. Even a publication from a prior run
// gets this same forbidden-key check before it is trusted and adopted.
function containsForbiddenPublishedKey(value: unknown): boolean {
if (Array.isArray(value)) return value.some(containsForbiddenPublishedKey);
if (!isRecord(value)) return false;
return Object.entries(value).some(
([key, entry]) => FORBIDDEN_PUBLISHED_KEYS.has(key) || containsForbiddenPublishedKey(entry),
);
}
/**
* Re-verify the internal-identity boundary on a previously published queue before adopting it.
*
* A manifest existing in HEAD means some earlier run already published this class, but that
* publication is only ever restored here, not blindly trusted: this confirms the queue's task IDs
* match the manifest's lineage exactly, that no forbidden internal key survived, and that no
* producer-ID token appears anywhere in the serialized queue. The lost-acknowledgement repair path
* is a second place a boundary-violating queue could otherwise slip through, so it gets the same
* fail-closed check as a fresh publish.
*/
function verifyPublishedQueue(
contents: string,
vulnerabilityClass: ReconciliationClass,
lineage: Record<string, { primary: string; absorbed: string[] }>,
): void {
let parsed: unknown;
try {
parsed = JSON.parse(contents);
} catch {
throw new PublicationConflictError('Existing published queue is not valid JSON');
}
if (!isRecord(parsed) || Object.keys(parsed).length !== 1 || !Array.isArray(parsed.vulnerabilities)) {
throw new PublicationConflictError('Existing published queue does not have the canonical envelope');
}
const queueTaskIds: string[] = [];
for (const task of parsed.vulnerabilities) {
if (!isRecord(task) || typeof task.ID !== 'string' || !isTaskReference(task.ID, vulnerabilityClass)) {
throw new PublicationConflictError('Existing published queue contains an invalid task reference');
}
queueTaskIds.push(task.ID);
}
const lineageTaskIds = Object.keys(lineage);
if (
queueTaskIds.length !== lineageTaskIds.length ||
queueTaskIds.some((taskId, index) => taskId !== lineageTaskIds[index])
) {
throw new PublicationConflictError('Existing published queue and manifest lineage disagree');
}
if (containsForbiddenPublishedKey(parsed)) {
throw new PublicationConflictError('Existing published queue retains an internal producer-only key');
}
const serialized = JSON.stringify(parsed);
const producerIds = Object.values(lineage).flatMap((entry) => [entry.primary, ...entry.absorbed]);
if (producerIds.some((producerId) => serialized.includes(producerId))) {
throw new PublicationConflictError('Existing published queue retains an internal producer identifier');
}
}
function samePathSet(actual: readonly string[], expected: readonly string[]): boolean {
if (actual.length !== expected.length) return false;
const actualPaths = new Set(actual);
return actualPaths.size === actual.length && expected.every((expectedPath) => actualPaths.has(expectedPath));
}
/**
* Resolve which optional output shape the committed manifest actually published.
*
* The standalone SAST-provenance file is the one publication member that legitimately differs
* between runs: whether the static-analysis stage produced SARIF decides it. That is a property of
* the run that published, not of the run resuming, so on the repair path the committed manifest is
* the authority on which shape to expect — otherwise a resume whose SARIF outcome flipped would
* conflict with an otherwise coherent publication. Only the two shapes a class can legally publish
* are admitted; any other path set is handed back as this run's own contract and rejected by
* `isManifestCoherent`.
*/
function contractForCommittedShape(
contract: PublicationContract,
manifest: PublicationManifest,
vulnerabilityClass: ReconciliationClass,
): PublicationContract {
const committedPaths = manifest.consumer_files.map((consumer) => consumer.path);
const withoutProvenance = [exploitationQueuePath(vulnerabilityClass)];
const withProvenance = [...withoutProvenance, sastProvenancePath(vulnerabilityClass)];
for (const shape of [withoutProvenance, withProvenance]) {
if (samePathSet(committedPaths, shape)) return { ...contract, requiredOutputPaths: shape };
}
return contract;
}
export interface PrepareClassReconciliationArgs {
deliverablesDir: string;
sessionId: string;
vulnerabilityClass: ReconciliationClass;
contract: PublicationContract;
workspacesDir?: string;
}
/**
* Prepare one class from committed `HEAD`, repairing an existing coherent publication when present.
*
* Runs entirely inside the Git critical section. Three outcomes: a coherent published manifest is
* restored from HEAD and reported as `already_published` (the lost-acknowledgement repair path); a
* present-but-incoherent or corrupt manifest is a hard conflict; otherwise the committed producer
* queue is parsed and written as the first content-addressed artifact and reported as `pending`.
*/
export async function prepareClassReconciliation(args: PrepareClassReconciliationArgs): Promise<PrepareResult> {
const queuePath = exploitationQueuePath(args.vulnerabilityClass);
const manifestPath = reconciliationManifestPath(args.vulnerabilityClass);
if (args.contract.manifestPath !== manifestPath) {
throw new ArtifactIntegrityError('Publication contract names the wrong class manifest');
}
return withGitRepoLock(async (): Promise<PrepareResult> => {
const manifestRead = await readPublishedManifest(args.deliverablesDir, manifestPath);
if (manifestRead.state === 'invalid') {
throw new PublicationConflictError(`Corrupt class manifest in HEAD: ${manifestRead.reason}`);
}
if (manifestRead.state === 'present') {
const committedContract = contractForCommittedShape(
args.contract,
manifestRead.manifest,
args.vulnerabilityClass,
);
if (
!isManifestCoherent({
manifest: manifestRead.manifest,
sessionId: args.sessionId,
vulnerabilityClass: args.vulnerabilityClass,
contract: committedContract,
producerQueuePath: queuePath,
})
) {
throw new PublicationConflictError('Class manifest does not cohere with the publication contract');
}
// Mirrors the same guard on the fresh-publish path: when the publication being repaired
// declares no standalone provenance, a provenance file in HEAD is residue from an interrupted
// publish that no manifest vouches for, so it is a conflict rather than something to adopt.
const provenancePath = sastProvenancePath(args.vulnerabilityClass);
if (!committedContract.requiredOutputPaths.includes(provenancePath)) {
const unvouchedProvenance = await readCommittedFile(args.deliverablesDir, provenancePath);
if (unvouchedProvenance.state !== 'absent') {
throw new PublicationConflictError('Existing publication has provenance outside its exact manifest path set');
}
}
const consumerContents = await verifyCommittedConsumers(
args.deliverablesDir,
manifestRead.manifest.consumer_files,
);
const publishedQueueContents = consumerContents.get(queuePath);
if (publishedQueueContents === undefined) {
throw new PublicationConflictError('Existing class publication manifest omits its queue consumer');
}
verifyPublishedQueue(publishedQueueContents, args.vulnerabilityClass, manifestRead.manifest.lineage);
await rejectSymlinkDestinations(args.deliverablesDir, [...committedContract.requiredOutputPaths, manifestPath]);
await restorePathsFromHead(args.deliverablesDir, [...committedContract.requiredOutputPaths, manifestPath]);
return { outcome: 'already_published', manifestSha256: sha256Text(manifestRead.contents) };
}
const unexpectedProvenance = await readCommittedFile(
args.deliverablesDir,
sastProvenancePath(args.vulnerabilityClass),
);
if (unexpectedProvenance.state !== 'absent') {
throw new PublicationConflictError('Pre-manifest class state contains standalone provenance');
}
const queueRead = await readCommittedFile(args.deliverablesDir, queuePath);
if (queueRead.state === 'absent') {
throw new PublicationConflictError('Producer queue is not committed in HEAD');
}
if (queueRead.state === 'corrupt') {
throw new ArtifactIntegrityError('Producer queue is committed but unreadable');
}
const blobSha = await blobShaFromHead(args.deliverablesDir, queuePath);
if (blobSha.state !== 'present') {
throw new ArtifactIntegrityError('Producer queue has no committed blob identity');
}
const entries = parseProducerQueue(queueRead.contents, args.vulnerabilityClass);
const evidenceKeys = QUEUE_ENTRY_FIELD_NAMES[args.vulnerabilityClass].filter((key) => key !== 'ID');
const observations = entries.map((entry) => toObservation(entry, evidenceKeys));
const body: ProducerObservationsBody = {
observations,
producer_queue: {
path: queuePath,
blob_sha: blobSha.sha,
digest: sha256Text(queueRead.contents),
},
};
const ref = await writeArtifact({
sessionId: args.sessionId,
...(args.workspacesDir !== undefined ? { workspacesDir: args.workspacesDir } : {}),
artifactKind: 'producer-observations',
vulnerabilityClass: args.vulnerabilityClass,
body,
inputs: [],
counts: { observations: observations.length },
});
return { outcome: 'pending', ref };
});
}
@@ -0,0 +1,647 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Deterministic OSS queue shaping and lost-acknowledgement-safe class publication. */
import { createHash, randomUUID } from 'node:crypto';
import { lstat, rename, unlink, writeFile } from 'node:fs/promises';
import path from 'node:path';
import {
blobShaFromHead,
commitExactPaths,
ExactPathCommitMismatchError,
lastCommitForPathAtHead,
readCommittedFile,
restorePathsFromHead,
withGitRepoLock,
} from '../../services/git-manager.js';
import type { ActivityLogger } from '../../types/activity-logger.js';
import type { ReconciliationClass } from '../../types/reconciliation.js';
import {
ArtifactIntegrityError,
PublicationConflictError,
ReconciliationError,
ReconciliationIoError,
readArtifact,
} from './artifact-store.js';
import type { ArtifactInputDigest, ArtifactRef, PublicationContract, ReconciliationObservation } from './contracts.js';
import {
isManifest,
isManifestCoherent,
type ManifestLineageEntry,
type PublicationManifest,
readPublishedManifest,
} from './manifest.js';
import { mintTaskReferences } from './materialize-core.js';
import { exploitationQueuePath, reconciliationManifestPath, sastProvenancePath } from './prepare.js';
import { isProducerId, isTaskReference } from './refs.js';
import { RECONCILIATION_SCHEMA_VERSION } from './schema-version.js';
import type { FixedTasksBody, ProducerObservationsBody, SupplementalObservationsBody } from './stage-contracts.js';
// Every key here is internal bookkeeping that must never reach the exploitation queue a downstream
// exploit agent reads: a producer ID or source-adapter identifier would tell that agent exactly
// which scan producer (and by extension, which internal class/source combination) found the
// vulnerability. This exact set is duplicated in prepare.ts (guarding the lost-acknowledgement
// repair path there); the two must be kept identical, since a key added to only one would let it
// slip through whichever path was not updated.
const FORBIDDEN_PUBLISHED_KEYS: ReadonlySet<string> = new Set([
'producer_id',
'primary_preference',
'observation_key',
'novelty',
'_sastId',
'_sast_id',
'repository_id',
'scan_run_id',
]);
interface PublicationFile {
path: string;
contents: string;
}
interface PreparedPublication {
contract: PublicationContract;
files: PublicationFile[];
manifest: PublicationManifest;
manifestContents: string;
manifestSha256: string;
producerBody: ProducerObservationsBody;
includeSastProvenance: boolean;
}
/** Result returned both for a fresh commit and an existing coherent publication. */
export interface PublishClassReconciliationResult {
alreadyPublished: boolean;
manifestSha256: string;
commitHash: string;
}
export interface PublishClassReconciliationOssArgs {
deliverablesDir: string;
sessionId: string;
workspacesDir?: string;
vulnerabilityClass: ReconciliationClass;
producerRef: ArtifactRef<'producer-observations'>;
supplementalRef: ArtifactRef<'supplemental-observations'>;
fixedTasksRef: ArtifactRef<'fixed-tasks'>;
logger: ActivityLogger;
}
function serialize(value: unknown): string {
return `${JSON.stringify(value, null, 2)}\n`;
}
function sha256Text(contents: string): string {
return createHash('sha256').update(contents, 'utf8').digest('hex');
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function arraysEqual<T>(first: readonly T[], second: readonly T[]): boolean {
return first.length === second.length && first.every((value, index) => value === second[index]);
}
function sameStringSet(first: readonly string[], second: readonly string[]): boolean {
return (
first.length === second.length &&
new Set(first).size === first.length &&
first.every((value) => second.includes(value))
);
}
/** Derive the exact consumer and manifest paths for one OSS class publication. */
export function publicationContractForClass(
vulnerabilityClass: ReconciliationClass,
includeSastProvenance: boolean,
): PublicationContract {
const requiredOutputPaths = [
exploitationQueuePath(vulnerabilityClass),
...(includeSastProvenance ? [sastProvenancePath(vulnerabilityClass)] : []),
];
return {
publicationKind: 'class-reconciliation',
schemaVersion: RECONCILIATION_SCHEMA_VERSION,
manifestPath: reconciliationManifestPath(vulnerabilityClass),
requiredOutputPaths,
};
}
function producerIdsFromFixed(fixed: FixedTasksBody): string[] {
const producerIds: string[] = [];
for (const task of fixed.tasks) {
producerIds.push(task.producer_id);
for (const member of task.merged_from ?? []) producerIds.push(member.producer_id);
}
return producerIds;
}
// Redact longest IDs first so a short producer ID that is a substring of a longer one cannot
// partially rewrite the longer token and leave a recognizable fragment behind.
function redactProducerIds(value: string, producerIds: readonly string[]): string {
let redacted = value;
for (const producerId of [...producerIds].sort((first, second) => second.length - first.length)) {
if (redacted.includes(producerId)) redacted = redacted.split(producerId).join('[redacted]');
}
return redacted;
}
function scrubPublishedValue(value: unknown, producerIds: readonly string[]): unknown {
if (typeof value === 'string') return redactProducerIds(value, producerIds);
if (Array.isArray(value)) return value.map((entry) => scrubPublishedValue(entry, producerIds));
if (value === null || typeof value !== 'object') return value;
const cleaned: Record<string, unknown> = {};
for (const [key, entry] of Object.entries(value as Record<string, unknown>)) {
if (FORBIDDEN_PUBLISHED_KEYS.has(key)) continue;
const publicKey = redactProducerIds(key, producerIds);
if (publicKey in cleaned) {
throw new ArtifactIntegrityError('Producer-ID redaction would collide two published evidence keys');
}
cleaned[publicKey] = scrubPublishedValue(entry, producerIds);
}
return cleaned;
}
function findForbiddenPublishedKey(value: unknown): string | null {
if (Array.isArray(value)) {
for (const entry of value) {
const found = findForbiddenPublishedKey(entry);
if (found !== null) return found;
}
return null;
}
if (value === null || typeof value !== 'object') return null;
for (const [key, entry] of Object.entries(value as Record<string, unknown>)) {
if (FORBIDDEN_PUBLISHED_KEYS.has(key)) return key;
const found = findForbiddenPublishedKey(entry);
if (found !== null) return found;
}
return null;
}
/**
* Build the canonical consumer queue while retaining evidence and public source annotations.
*
* Internal producer identity must never reach the published queue that a downstream exploit agent
* reads. Three independent passes enforce that: drop forbidden keys and redact ID tokens while
* copying, then re-scan the result for any forbidden key, then serialize and fail if any producer
* ID string survived. The second and third passes are belt-and-suspenders against a redaction miss.
*/
export function buildPublishedQueue(fixed: FixedTasksBody): { vulnerabilities: Record<string, unknown>[] } {
const producerIds = producerIdsFromFixed(fixed);
const scrubbed = scrubPublishedValue({ vulnerabilities: fixed.tasks }, producerIds) as {
vulnerabilities: Record<string, unknown>[];
};
const forbiddenKey = findForbiddenPublishedKey(scrubbed);
if (forbiddenKey !== null) {
throw new ArtifactIntegrityError(`Published queue retained forbidden internal key ${forbiddenKey}`);
}
const serialized = JSON.stringify(scrubbed);
if (producerIds.some((producerId) => serialized.includes(producerId))) {
throw new ArtifactIntegrityError('Published queue retained a producer identifier token');
}
return scrubbed;
}
function observationIds(
producerBody: ProducerObservationsBody,
supplementalBody: SupplementalObservationsBody,
): string[] {
return [...producerBody.observations, ...supplementalBody.observations].map((observation) => observation.producer_id);
}
// Confirms a producer ID actually belongs to the namespace its own declared scan_source implies
// (VULN for pentest, SAST for static analysis), for the declared class. This is what stops a
// mislabeled or forged scan_source from having its identity validated against the wrong producer
// namespace, which would otherwise let it dodge the class/source checks the rest of publication
// relies on.
function validateProducerIdentity(
producerId: string,
scanSource: ReconciliationObservation['scan_source'],
vulnerabilityClass: ReconciliationClass,
): boolean {
if (scanSource === 'sast') return isProducerId(producerId, vulnerabilityClass, 'SAST');
if (scanSource === 'vulnerability_analysis') return isProducerId(producerId, vulnerabilityClass, 'VULN');
return false;
}
/**
* Refuse to publish unless the fixed tasks are a complete, exact partition of the observation set.
*
* Checks, independently: every observation ID appears at most once across the two input bodies;
* every task reference is the dense, class-namespaced value materialization should have minted for
* its position, with no duplicates; every member's producer ID belongs to the class/source
* namespace its own scan_source claims; and the observation-to-task map agrees with task membership
* in both directions. Any one of these failing means either an observation was silently dropped or
* duplicated, or a task references identity outside its declared boundary, either of which is a
* fail-closed reason to abort the publish rather than commit an inconsistent queue.
*/
function validateFixedTasks(
fixed: FixedTasksBody,
producerBody: ProducerObservationsBody,
supplementalBody: SupplementalObservationsBody,
vulnerabilityClass: ReconciliationClass,
): void {
const expectedObservationIds = observationIds(producerBody, supplementalBody);
if (new Set(expectedObservationIds).size !== expectedObservationIds.length) {
throw new ArtifactIntegrityError('Publication inputs contain duplicate observation identifiers');
}
const taskIds = new Set<string>();
const materializedIds = new Set<string>();
const expectedTaskIds = mintTaskReferences(fixed.tasks.length, vulnerabilityClass);
for (const [index, task] of fixed.tasks.entries()) {
if (!isTaskReference(task.ID, vulnerabilityClass) || task.ID !== expectedTaskIds[index] || taskIds.has(task.ID)) {
throw new ArtifactIntegrityError('Fixed tasks contain an invalid or duplicate task reference');
}
taskIds.add(task.ID);
const members = [task, ...(task.merged_from ?? [])];
for (const member of members) {
if (
materializedIds.has(member.producer_id) ||
!validateProducerIdentity(member.producer_id, member.scan_source, vulnerabilityClass)
) {
throw new ArtifactIntegrityError('Fixed tasks contain duplicate or cross-namespace producer identity');
}
materializedIds.add(member.producer_id);
if (fixed.observation_to_task[member.producer_id] !== task.ID) {
throw new ArtifactIntegrityError('Observation-to-task map disagrees with fixed task membership');
}
}
}
if (!sameStringSet([...materializedIds], expectedObservationIds)) {
throw new ArtifactIntegrityError('Fixed tasks do not cover the complete observation set exactly once');
}
if (!sameStringSet(Object.keys(fixed.observation_to_task), expectedObservationIds)) {
throw new ArtifactIntegrityError('Observation-to-task map is incomplete or contains extra entries');
}
}
function lineageHas(
inputs: readonly ArtifactInputDigest[],
kind: ArtifactInputDigest['artifactKind'],
sha256: string,
): boolean {
return inputs.some((input) => input.artifactKind === kind && input.sha256 === sha256);
}
function assertRefClass(ref: ArtifactRef, vulnerabilityClass: ReconciliationClass): void {
if (ref.vulnerabilityClass !== vulnerabilityClass) {
throw new ArtifactIntegrityError('Publication artifact reference crosses the declared class boundary');
}
}
function buildLineage(fixed: FixedTasksBody): Record<string, ManifestLineageEntry> {
const lineage: Record<string, ManifestLineageEntry> = Object.create(null);
for (const task of fixed.tasks) {
lineage[task.ID] = {
primary: task.producer_id,
absorbed: (task.merged_from ?? []).map((member) => member.producer_id),
};
}
return lineage;
}
function buildManifest(args: {
sessionId: string;
vulnerabilityClass: ReconciliationClass;
producerBody: ProducerObservationsBody;
consumerFiles: readonly PublicationFile[];
inputDigests: readonly ArtifactInputDigest[];
fixed: FixedTasksBody;
}): PublicationManifest {
return {
session_id: args.sessionId,
vulnerability_class: args.vulnerabilityClass,
schema_version: RECONCILIATION_SCHEMA_VERSION,
producer_queue: {
path: args.producerBody.producer_queue.path,
blob_sha: args.producerBody.producer_queue.blob_sha,
},
consumer_files: args.consumerFiles.map((file) => ({ path: file.path, sha256: sha256Text(file.contents) })),
input_digests: args.inputDigests.map((input) => ({ artifactKind: input.artifactKind, sha256: input.sha256 })),
lineage: buildLineage(args.fixed),
};
}
async function preparePublication(args: PublishClassReconciliationOssArgs): Promise<PreparedPublication> {
assertRefClass(args.producerRef, args.vulnerabilityClass);
assertRefClass(args.supplementalRef, args.vulnerabilityClass);
assertRefClass(args.fixedTasksRef, args.vulnerabilityClass);
const producerBody = await readArtifact(args.producerRef, args.sessionId, args.workspacesDir);
const supplementalBody = await readArtifact(args.supplementalRef, args.sessionId, args.workspacesDir);
const fixed = await readArtifact(args.fixedTasksRef, args.sessionId, args.workspacesDir);
if (
!Array.isArray(producerBody.observations) ||
!isRecord(producerBody.producer_queue) ||
!Array.isArray(supplementalBody.observations) ||
!Array.isArray(supplementalBody.provenance) ||
!Array.isArray(fixed.tasks) ||
!isRecord(fixed.observation_to_task)
) {
throw new ArtifactIntegrityError('Publication artifact body is truncated or malformed');
}
if (
!lineageHas(args.fixedTasksRef.inputs, 'producer-observations', args.producerRef.sha256) ||
!lineageHas(args.fixedTasksRef.inputs, 'supplemental-observations', args.supplementalRef.sha256)
) {
throw new ArtifactIntegrityError('Fixed-task lineage does not name the publication observations');
}
if (producerBody.producer_queue.path !== exploitationQueuePath(args.vulnerabilityClass)) {
throw new ArtifactIntegrityError('Producer artifact names the wrong class queue');
}
if (supplementalBody.provenance.length !== 0) {
throw new ArtifactIntegrityError('Standalone OSS supplemental provenance must remain empty');
}
validateFixedTasks(fixed, producerBody, supplementalBody, args.vulnerabilityClass);
const includeSastProvenance = supplementalBody.sarif !== undefined;
const contract = publicationContractForClass(args.vulnerabilityClass, includeSastProvenance);
const consumerFiles: PublicationFile[] = [
{ path: exploitationQueuePath(args.vulnerabilityClass), contents: serialize(buildPublishedQueue(fixed)) },
...(includeSastProvenance
? [{ path: sastProvenancePath(args.vulnerabilityClass), contents: serialize({ entries: [] }) }]
: []),
];
const inputDigests: ArtifactInputDigest[] = [
{ artifactKind: 'producer-observations', sha256: args.producerRef.sha256 },
{ artifactKind: 'supplemental-observations', sha256: args.supplementalRef.sha256 },
{ artifactKind: 'fixed-tasks', sha256: args.fixedTasksRef.sha256 },
];
const manifest = buildManifest({
sessionId: args.sessionId,
vulnerabilityClass: args.vulnerabilityClass,
producerBody,
consumerFiles,
inputDigests,
fixed,
});
if (!isManifest(manifest)) {
throw new ArtifactIntegrityError('Built OSS publication manifest failed self-validation');
}
const manifestContents = serialize(manifest);
return {
contract,
files: [...consumerFiles, { path: contract.manifestPath, contents: manifestContents }],
manifest,
manifestContents,
manifestSha256: sha256Text(manifestContents),
producerBody,
includeSastProvenance,
};
}
function lineageEquals(
first: Record<string, ManifestLineageEntry>,
second: Record<string, ManifestLineageEntry>,
): boolean {
const firstKeys = Object.keys(first);
const secondKeys = Object.keys(second);
if (!arraysEqual(firstKeys, secondKeys)) return false;
return firstKeys.every((key) => {
const firstEntry = first[key];
const secondEntry = second[key];
return (
firstEntry !== undefined &&
secondEntry !== undefined &&
firstEntry.primary === secondEntry.primary &&
arraysEqual(firstEntry.absorbed, secondEntry.absorbed) &&
firstEntry.novelty === secondEntry.novelty
);
});
}
function manifestMatchesPrepared(existing: PublicationManifest, prepared: PublicationManifest): boolean {
if (
!arraysEqual(
existing.input_digests.map((input) => `${input.artifactKind}:${input.sha256}`),
prepared.input_digests.map((input) => `${input.artifactKind}:${input.sha256}`),
)
) {
return false;
}
const expectedConsumers = new Map(prepared.consumer_files.map((consumer) => [consumer.path, consumer.sha256]));
if (
existing.consumer_files.length !== expectedConsumers.size ||
existing.consumer_files.some((consumer) => expectedConsumers.get(consumer.path) !== consumer.sha256)
) {
return false;
}
return lineageEquals(existing.lineage, prepared.lineage);
}
async function verifyCommittedFiles(deliverablesDir: string, files: readonly PublicationFile[]): Promise<void> {
for (const file of files) {
const committed = await readCommittedFile(deliverablesDir, file.path);
if (committed.state !== 'present' || committed.contents !== file.contents) {
throw new PublicationConflictError('Committed publication bytes do not match the prepared exact bytes');
}
}
}
/**
* Adopt an already-committed publication when the HEAD manifest matches what this call prepared.
*
* Returns null when nothing is published yet (the caller then commits). Returns a result with
* `alreadyPublished: true` when a coherent, byte-identical publication already exists, which is how
* a re-drive after a lost acknowledgement converges instead of committing a second time. Any
* manifest that exists but disagrees is a conflict, never a silent overwrite. Before returning, it
* restores the exact committed paths from HEAD so a partially written retry leaves no dirty bytes.
*/
async function tryReturnExistingPublication(
args: PublishClassReconciliationOssArgs,
prepared: PreparedPublication,
): Promise<PublishClassReconciliationResult | null> {
const manifestRead = await readPublishedManifest(args.deliverablesDir, prepared.contract.manifestPath);
if (manifestRead.state === 'absent') return null;
if (manifestRead.state === 'invalid') {
throw new PublicationConflictError(`Corrupt class manifest in HEAD: ${manifestRead.reason}`);
}
if (
!isManifestCoherent({
manifest: manifestRead.manifest,
sessionId: args.sessionId,
vulnerabilityClass: args.vulnerabilityClass,
contract: prepared.contract,
producerQueuePath: exploitationQueuePath(args.vulnerabilityClass),
producerBlobSha: prepared.producerBody.producer_queue.blob_sha,
}) ||
!manifestMatchesPrepared(manifestRead.manifest, prepared.manifest)
) {
throw new PublicationConflictError('Existing class publication conflicts with the prepared publication');
}
if (!prepared.includeSastProvenance) {
const unexpectedProvenance = await readCommittedFile(
args.deliverablesDir,
sastProvenancePath(args.vulnerabilityClass),
);
if (unexpectedProvenance.state !== 'absent') {
throw new PublicationConflictError('Existing publication has provenance outside its exact manifest path set');
}
}
await preflightSymlinks(args.deliverablesDir, [
...prepared.contract.requiredOutputPaths,
prepared.contract.manifestPath,
]);
await verifyCommittedFiles(args.deliverablesDir, prepared.files.slice(0, -1));
await restorePathsFromHead(args.deliverablesDir, [
...prepared.contract.requiredOutputPaths,
prepared.contract.manifestPath,
]);
const commitHash = await lastCommitForPathAtHead(args.deliverablesDir, prepared.contract.manifestPath);
if (commitHash === null) throw new ReconciliationIoError('Unable to read the existing publication commit');
return {
alreadyPublished: true,
manifestSha256: sha256Text(manifestRead.contents),
commitHash,
};
}
function isErrno(error: unknown, code: string): boolean {
return error instanceof Error && (error as NodeJS.ErrnoException).code === code;
}
// Refuses to publish through a path that is currently a symlink, so a deliverables directory
// entry cannot redirect a publication write to somewhere outside the intended destination. Absence
// (`ENOENT`) is not a violation; the path simply does not exist yet, which is the normal case for a
// first publish.
async function preflightSymlinks(deliverablesDir: string, relativePaths: readonly string[]): Promise<void> {
for (const relativePath of relativePaths) {
try {
const stat = await lstat(path.join(deliverablesDir, relativePath));
if (stat.isSymbolicLink()) throw new PublicationConflictError('Refusing to publish through a symlink');
} catch (error) {
if (isErrno(error, 'ENOENT')) continue;
if (error instanceof ReconciliationError) throw error;
throw new ReconciliationIoError('Unable to inspect a class publication destination');
}
}
}
async function writeFileReplacingEntry(absolutePath: string, contents: string): Promise<void> {
const temporaryPath = `${absolutePath}.tmp-${randomUUID()}`;
try {
await writeFile(temporaryPath, contents, { flag: 'wx' });
await rename(temporaryPath, absolutePath);
} catch (error) {
await unlink(temporaryPath).catch(() => undefined);
throw error;
}
}
async function expectedChangedPaths(deliverablesDir: string, files: readonly PublicationFile[]): Promise<string[]> {
const changed: string[] = [];
for (const file of files) {
const committed = await readCommittedFile(deliverablesDir, file.path);
if (committed.state === 'corrupt') {
throw new PublicationConflictError('Publication destination has corrupt committed state');
}
if (committed.state === 'absent' || committed.contents !== file.contents) changed.push(file.path);
}
return changed;
}
// The reconciliation prepared against a specific committed producer queue. If that queue changed
// in HEAD between preparation and commit, the prepared tasks no longer describe it, so publishing
// them would be incoherent. Both the blob SHA and a content digest are checked to catch a change
// even if one identity were somehow reused.
async function assertProducerStillCurrent(
deliverablesDir: string,
producerBody: ProducerObservationsBody,
): Promise<void> {
const queuePath = producerBody.producer_queue.path;
const currentBlob = await blobShaFromHead(deliverablesDir, queuePath);
if (currentBlob.state !== 'present' || currentBlob.sha !== producerBody.producer_queue.blob_sha) {
throw new PublicationConflictError('Producer queue changed after reconciliation preparation');
}
const current = await readCommittedFile(deliverablesDir, queuePath);
if (current.state !== 'present' || sha256Text(current.contents) !== producerBody.producer_queue.digest) {
throw new PublicationConflictError('Producer queue bytes changed after reconciliation preparation');
}
}
// Standalone provenance with no manifest can only mean a prior publication attempt wrote some of
// its files and was interrupted before the manifest (and therefore the whole commit) landed. That is
// not a state a fresh publish should build on top of or silently repair by overwriting; it fails
// closed and surfaces as a conflict instead.
async function ensureNoPartialPreManifestState(
deliverablesDir: string,
vulnerabilityClass: ReconciliationClass,
): Promise<void> {
const provenance = await readCommittedFile(deliverablesDir, sastProvenancePath(vulnerabilityClass));
if (provenance.state !== 'absent') {
throw new PublicationConflictError('Pre-manifest class state contains standalone provenance');
}
}
async function commitPreparedPublication(
args: PublishClassReconciliationOssArgs,
prepared: PreparedPublication,
): Promise<PublishClassReconciliationResult> {
const ownedPaths = prepared.files.map((file) => file.path);
await assertProducerStillCurrent(args.deliverablesDir, prepared.producerBody);
await ensureNoPartialPreManifestState(args.deliverablesDir, args.vulnerabilityClass);
await preflightSymlinks(args.deliverablesDir, ownedPaths);
// Compute the exact set of paths whose bytes differ from HEAD before writing, and hand it to the
// commit as the expected delta. The commit rejects any staged change outside this set, so a
// dirty sibling file cannot ride along into this publication commit.
const expectedChanges = await expectedChangedPaths(args.deliverablesDir, prepared.files);
try {
for (const file of prepared.files) {
await writeFileReplacingEntry(path.join(args.deliverablesDir, file.path), file.contents);
}
} catch (error) {
await restorePathsFromHead(args.deliverablesDir, ownedPaths);
throw error instanceof ReconciliationError
? error
: new ReconciliationIoError('Unable to write prepared class publication bytes');
}
let commitHash: string;
try {
const committed = await commitExactPaths(
args.deliverablesDir,
ownedPaths,
`Publish ${args.vulnerabilityClass} reconciliation`,
args.logger,
expectedChanges,
);
commitHash = committed.commitHash;
} catch (error) {
await restorePathsFromHead(args.deliverablesDir, ownedPaths);
if (error instanceof ExactPathCommitMismatchError) {
throw new PublicationConflictError('Publication staged path set differs from its exact prepared delta');
}
throw error instanceof ReconciliationError
? error
: new ReconciliationIoError('Unable to commit prepared class publication bytes');
}
await verifyCommittedFiles(args.deliverablesDir, prepared.files);
return {
alreadyPublished: false,
manifestSha256: prepared.manifestSha256,
commitHash,
};
}
/** Publish one OSS class through one reentrant Git critical section. */
export async function publishClassReconciliationOss(
args: PublishClassReconciliationOssArgs,
): Promise<PublishClassReconciliationResult> {
const prepared = await preparePublication(args);
return withGitRepoLock(async () => {
const existing = await tryReturnExistingPublication(args, prepared);
if (existing !== null) return existing;
return commitPreparedPublication(args, prepared);
});
}
+50
View File
@@ -0,0 +1,50 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Stable producer and exploitation-task reference namespaces. */
import type { ReconciliationClass } from '../../types/reconciliation.js';
export const REF_PREFIX: Readonly<Record<ReconciliationClass, string>> = Object.freeze({
injection: 'INJ',
xss: 'XSS',
auth: 'AUTH',
authz: 'AUTHZ',
ssrf: 'SSRF',
miscellaneous: 'MISC',
});
export type ProducerSource = 'VULN' | 'SAST';
function positiveReferenceNumberPattern(): string {
return '0*[1-9][0-9]*';
}
/** The source-aware producer-ID pattern admitted for one internal class. */
export function producerIdPattern(vulnClass: ReconciliationClass, source: ProducerSource): RegExp {
if (vulnClass === 'miscellaneous' && source === 'VULN') {
// The `miscellaneous` class has no vulnerability-analysis producer: it is seeded queue-only
// and carries SAST producers alone. This pattern can never match, so any `MISC-VULN-*` ID
// is rejected rather than admitted.
return /(?!)^/;
}
return new RegExp(`^${REF_PREFIX[vulnClass]}-${source}-${positiveReferenceNumberPattern()}$`);
}
/** Whether an ID belongs to the class and producer source that declared it. */
export function isProducerId(id: string, vulnClass: ReconciliationClass, source: ProducerSource): boolean {
return producerIdPattern(vulnClass, source).test(id);
}
/** The stable exploitation-task reference pattern for one internal class. */
export function taskReferencePattern(vulnClass: ReconciliationClass): RegExp {
return new RegExp(`^${REF_PREFIX[vulnClass]}-${positiveReferenceNumberPattern()}$`);
}
/** Whether an ID is a stable task reference in the declared class namespace. */
export function isTaskReference(id: string, vulnClass: ReconciliationClass): boolean {
return taskReferencePattern(vulnClass).test(id);
}
@@ -0,0 +1,82 @@
/** Build the bounded finding context sent to the SAST enrichment model. */
import type {
DataflowFindingContext,
FindingContext,
LocalizedFindingContext,
SarifLocation,
SarifResult,
} from './types.js';
function locationInfo(location: SarifLocation): { file: string; line: number; column: number; snippet: string } {
const physical = location.physicalLocation;
return {
file: physical.artifactLocation.uri,
line: physical.region.startLine,
column: physical.region.startColumn ?? 1,
snippet: physical.region.snippet?.text ?? '',
};
}
function confidenceScore(level: SarifResult['level']): number {
if (level === 'error') return 0.9;
if (level === 'warning') return 0.6;
return 0.3;
}
// Best-effort labeling for the model's context only; a wrong guess here does not affect identity,
// dedupe, or which class a finding routes to; it only changes how a step reads in the enrichment prompt.
function stepRole(message: string, index: number, lastIndex: number): string {
const normalized = message.toLowerCase();
if (normalized.includes('sanit')) return 'SANITIZED';
if (normalized.startsWith('source')) return 'SOURCE';
if (normalized.startsWith('sink')) return 'SINK';
if (index === 0) return 'SOURCE';
if (index === lastIndex) return 'SINK';
return 'HOP';
}
function localizedContext(result: SarifResult): LocalizedFindingContext {
const primary = locationInfo(result.locations[0]);
return {
cwe: result.ruleId,
message: result.message.text,
severity: result.properties.severity.toLowerCase(),
confidence: confidenceScore(result.level),
...primary,
};
}
/** Extract only validated SARIF fields; no repository identity or fallback path is synthesized. */
export function extractContext(result: SarifResult): FindingContext {
const locations = result.codeFlows[0]?.threadFlows[0]?.locations;
if (locations === undefined || locations.length === 0) return localizedContext(result);
const lastIndex = locations.length - 1;
const dataflowPath = locations.map((step, index) => {
const info = locationInfo(step.location);
return { ...info, role: stepRole(step.location.message?.text ?? '', index, lastIndex) };
});
const source = dataflowPath[0];
const sink = dataflowPath[lastIndex];
if (source === undefined || sink === undefined) return localizedContext(result);
const context: DataflowFindingContext = {
cwe: result.ruleId,
message: result.message.text,
severity: result.properties.severity.toLowerCase(),
confidence: confidenceScore(result.level),
sinkFile: sink.file,
sinkLine: sink.line,
sinkColumn: sink.column,
sinkSnippet: sink.snippet,
sourceFile: source.file,
sourceLine: source.line,
sourceColumn: source.column,
sourceSnippet: source.snippet,
dataflowPath,
validationReason: result.properties.description,
sanitizationStatus: dataflowPath.some((step) => step.role === 'SANITIZED') ? 'partial' : 'none',
};
return context;
}
@@ -0,0 +1,103 @@
/** Deterministic bare-CWE routing into Shannon's six internal classes. */
import type { ReconciliationClass } from '../../../types/reconciliation.js';
import type { Confidence, CWEMapping, ShannonCategory } from './types.js';
export const CWE_TO_CATEGORY: Readonly<Record<string, CWEMapping>> = Object.freeze({
'CWE-89': { category: 'INJECTION', name: 'SQL Injection', priority: 'P1' },
'CWE-78': { category: 'INJECTION', name: 'OS Command Injection', priority: 'P1' },
'CWE-95': { category: 'INJECTION', name: 'Code/Eval Injection', priority: 'P1' },
'CWE-94': { category: 'INJECTION', name: 'Code Injection', priority: 'P1' },
'CWE-502': { category: 'INJECTION', name: 'Deserialization', priority: 'P1' },
'CWE-611': { category: 'INJECTION', name: 'XXE', priority: 'P1' },
'CWE-22': { category: 'INJECTION', name: 'Path Traversal', priority: 'P1' },
'CWE-434': { category: 'INJECTION', name: 'Unrestricted File Upload', priority: 'P1' },
'CWE-943': { category: 'INJECTION', name: 'NoSQL Injection', priority: 'P1' },
'CWE-93': { category: 'INJECTION', name: 'CRLF Injection', priority: 'P2' },
'CWE-117': { category: 'INJECTION', name: 'Log Injection', priority: 'P2' },
'CWE-470': { category: 'INJECTION', name: 'Unsafe Reflection', priority: 'P2' },
'CWE-829': { category: 'INJECTION', name: 'Untrusted Function Inclusion', priority: 'P1' },
'CWE-643': { category: 'INJECTION', name: 'XPath Injection', priority: 'P2' },
'CWE-90': { category: 'INJECTION', name: 'LDAP Injection', priority: 'P2' },
'CWE-91': { category: 'INJECTION', name: 'XML Injection', priority: 'P2' },
'CWE-1336': { category: 'INJECTION', name: 'Template Injection', priority: 'P1' },
'CWE-1427': { category: 'INJECTION', name: 'Prompt Injection', priority: 'P2' },
'CWE-1321': { category: 'INJECTION', name: 'Prototype Pollution', priority: 'P1' },
'CWE-548': { category: 'INJECTION', name: 'Directory Listing', priority: 'P3' },
'CWE-79': { category: 'XSS', name: 'Cross-Site Scripting', priority: 'P1' },
'CWE-287': { category: 'AUTH', name: 'Broken Authentication', priority: 'P1' },
'CWE-798': { category: 'AUTH', name: 'Hard-coded Credentials', priority: 'P1' },
'CWE-319': { category: 'AUTH', name: 'Cleartext Transmission', priority: 'P2' },
'CWE-330': { category: 'AUTH', name: 'Insufficient Randomness', priority: 'P2' },
'CWE-346': { category: 'AUTH', name: 'Origin Validation Error', priority: 'P2' },
'CWE-295': { category: 'AUTH', name: 'Improper Certificate Validation', priority: 'P2' },
'CWE-347': { category: 'AUTH', name: 'Improper Signature Verification', priority: 'P2' },
'CWE-326': { category: 'AUTH', name: 'Inadequate Encryption', priority: 'P3' },
'CWE-329': { category: 'AUTH', name: 'Predictable IV', priority: 'P3' },
'CWE-323': { category: 'AUTH', name: 'Nonce or Key Pair Reuse', priority: 'P2' },
'CWE-327': { category: 'AUTH', name: 'Broken Cryptographic Algorithm', priority: 'P3' },
'CWE-328': { category: 'AUTH', name: 'Weak Hash', priority: 'P3' },
'CWE-916': { category: 'AUTH', name: 'Weak Password Hash Effort', priority: 'P3' },
'CWE-614': { category: 'AUTH', name: 'Cookie without Secure Flag', priority: 'P3' },
'CWE-942': { category: 'AUTH', name: 'Permissive Cross-domain Policy', priority: 'P3' },
'CWE-1004': { category: 'AUTH', name: 'Cookie without HttpOnly', priority: 'P3' },
'CWE-522': { category: 'AUTH', name: 'Insufficiently Protected Credentials', priority: 'P1' },
'CWE-306': { category: 'AUTH', name: 'Missing Authentication', priority: 'P1' },
'CWE-208': { category: 'AUTH', name: 'Timing Side-Channel', priority: 'P2' },
'CWE-338': { category: 'AUTH', name: 'Weak PRNG', priority: 'P2' },
'CWE-639': { category: 'AUTHZ', name: 'IDOR', priority: 'P1' },
'CWE-285': { category: 'AUTHZ', name: 'Broken Authorization', priority: 'P1' },
'CWE-269': { category: 'AUTHZ', name: 'Privilege Escalation', priority: 'P1' },
'CWE-284': { category: 'AUTHZ', name: 'Improper Access Control', priority: 'P1' },
'CWE-653': { category: 'AUTHZ', name: 'Data Isolation Failure', priority: 'P1' },
'CWE-732': { category: 'AUTHZ', name: 'Incorrect Permission Assignment', priority: 'P2' },
'CWE-862': { category: 'AUTHZ', name: 'Missing Authorization', priority: 'P1' },
'CWE-378': { category: 'AUTHZ', name: 'Permission Issue', priority: 'P2' },
'CWE-359': { category: 'AUTHZ', name: 'PII Exposure', priority: 'P1' },
'CWE-915': { category: 'AUTHZ', name: 'Mass Assignment', priority: 'P1' },
'CWE-918': { category: 'SSRF', name: 'Server-Side Request Forgery', priority: 'P1' },
'CWE-601': { category: 'MISC', name: 'Open Redirect', priority: 'P2' },
'CWE-693': { category: 'MISC', name: 'Protection Mechanism Failure', priority: 'P3' },
'CWE-1021': { category: 'MISC', name: 'Clickjacking', priority: 'P3' },
'CWE-1333': { category: 'MISC', name: 'ReDoS', priority: 'P3' },
'CWE-489': { category: 'MISC', name: 'Active Debug Code', priority: 'P3' },
'CWE-352': { category: 'MISC', name: 'CSRF', priority: 'P1' },
'CWE-532': { category: 'MISC', name: 'Sensitive Logging', priority: 'P3' },
'CWE-311': { category: 'MISC', name: 'Missing Encryption at Rest', priority: 'P3' },
'CWE-922': { category: 'MISC', name: 'Insecure Storage', priority: 'P3' },
'CWE-1236': { category: 'MISC', name: 'CSV Formula Injection', priority: 'P2' },
});
// Routing to `miscellaneous` (rather than dropping the finding) is what lets every schema-valid
// SARIF result reach exploitation even when its CWE is not one of the ones Shannon names explicitly:
// an unrecognized bare CWE still gets its own producer identity and its own exploitation task, just
// without a specific category name and at the lowest priority.
/** Unknown, but schema-valid, bare CWEs are retained for the generalist class. */
export function unmappedMapping(ruleId: string): CWEMapping {
return { category: 'MISC', name: ruleId, priority: 'P3' };
}
export function vulnerabilityClassToCategory(vulnerabilityClass: ReconciliationClass): ShannonCategory {
const categories: Record<ReconciliationClass, ShannonCategory> = {
injection: 'INJECTION',
xss: 'XSS',
auth: 'AUTH',
authz: 'AUTHZ',
ssrf: 'SSRF',
miscellaneous: 'MISC',
};
return categories[vulnerabilityClass];
}
export function normalizeConfidence(confidence: string | undefined): Confidence | undefined {
if (confidence === undefined) return undefined;
const normalized = confidence.toLowerCase();
if (normalized === 'med' || normalized === 'medium') return 'medium';
if (normalized === 'high' || normalized === 'low') return normalized;
return undefined;
}
@@ -0,0 +1,86 @@
/** One bounded structured-generation request for one nonempty class batch. */
import type { ReconciliationClass } from '../../../../types/reconciliation.js';
import type { StructuredGenerationPort, StructuredGenerationRequest } from '../../../structured-generation.js';
import { SAST_ENRICHMENT_TOOL_DESCRIPTION, sastEnrichmentToolSchema } from './schema.js';
import { extractVulnerabilities } from './validate.js';
export interface SastEnrichmentUsage {
inputTokens: number;
outputTokens: number;
costUsd: number;
}
export type SastEnrichmentBatchOutcome =
| { status: 'ok'; vulnerabilities: unknown[]; usage: SastEnrichmentUsage }
| { status: 'aborted'; usage: SastEnrichmentUsage; message: string }
| { status: 'failed'; usage: SastEnrichmentUsage; message: string; terminal: boolean };
export interface SastEnrichmentBatchRequest {
vulnerabilityClass: ReconciliationClass;
prompt: string;
findingsJson: string;
maxTokens: number;
signal?: AbortSignal;
}
// `terminal` marks a failure the stage should not retry. It is set only when the provider itself
// reported a non-retryable failure; an incomplete or empty response defaults to non-terminal so
// Temporal drives another attempt.
function isTerminalProviderFailure(message: string | undefined): boolean {
return message?.startsWith('AuthenticationError:') === true || message?.startsWith('ConfigurationError:') === true;
}
export async function runSastEnrichmentBatch<TModelContext>(
port: StructuredGenerationPort<TModelContext>,
modelContext: TModelContext,
request: SastEnrichmentBatchRequest,
): Promise<SastEnrichmentBatchOutcome> {
const generationRequest: StructuredGenerationRequest = {
userContent: `${request.prompt}\n${request.findingsJson}`,
tool: {
name: 'submit_result',
description: SAST_ENRICHMENT_TOOL_DESCRIPTION,
parametersJsonSchema: sastEnrichmentToolSchema(request.vulnerabilityClass),
},
maxTokens: request.maxTokens,
...(request.signal !== undefined && { signal: request.signal }),
};
const result = await port.generate(generationRequest, modelContext);
const usage = result.usage;
if (result.stopReason === 'aborted') {
if (request.signal?.aborted === true) {
return { status: 'aborted', usage, message: 'SAST enrichment was cancelled' };
}
return {
status: 'failed',
usage,
message: 'SAST enrichment request ended before producing a result',
terminal: false,
};
}
if (
result.stopReason !== 'toolUse' ||
result.toolCalls.length !== 1 ||
result.toolCalls[0]?.name !== 'submit_result'
) {
return {
status: 'failed',
usage,
message: 'SAST enrichment did not return one complete submit_result call',
terminal: isTerminalProviderFailure(result.errorMessage),
};
}
try {
return { status: 'ok', vulnerabilities: extractVulnerabilities(result.toolCalls[0].arguments), usage };
} catch {
return {
status: 'failed',
usage,
message: 'SAST enrichment returned an invalid response envelope',
terminal: false,
};
}
}
@@ -0,0 +1,53 @@
/** Code-owned SAST observation shaping and producer-ID minting. */
import type { ReconciliationClass } from '../../../../types/reconciliation.js';
import { ArtifactIntegrityError } from '../../artifact-store.js';
import type { ReconciliationObservation, SastSourceLocation } from '../../contracts.js';
import { isProducerId, REF_PREFIX } from '../../refs.js';
import type { ClassifiedFinding, FindingContext } from '../types.js';
export function sourceLocationFromContext(context: FindingContext): SastSourceLocation {
if ('sinkFile' in context) {
return {
file: context.sinkFile,
line: context.sinkLine,
column: context.sinkColumn,
rule_id: context.cwe,
};
}
return { file: context.file, line: context.line, column: context.column, rule_id: context.cwe };
}
// SAST producer IDs occupy a namespace (`PREFIX-SAST-NN`) disjoint from vulnerability-analysis
// producer IDs (`PREFIX-VULN-NN`) even within the same class, so the two sources can never collide
// on identity and `validateProducerIdentity` in publish.ts can tell them apart by construction.
export function mintSastProducerId(vulnerabilityClass: ReconciliationClass, sastId: number): string {
const producerId = `${REF_PREFIX[vulnerabilityClass]}-SAST-${String(sastId + 1).padStart(2, '0')}`;
if (!isProducerId(producerId, vulnerabilityClass, 'SAST')) {
throw new ArtifactIntegrityError('Minted SAST producer identifier is outside its class namespace');
}
return producerId;
}
/**
* Build one SAST-origin observation, always marked `preferred`.
*
* This is where the dedupe contract's primacy rule is established for a static-analysis finding: if
* reconciliation later groups this observation with a pentest observation for the same underlying
* vulnerability, this one becomes the task's primary record, since it carries an exact source
* location and rule ID that a pentest finding does not.
*/
export function buildSastObservation(
producerId: string,
evidence: Record<string, unknown>,
finding: ClassifiedFinding,
): ReconciliationObservation {
return {
...evidence,
producer_id: producerId,
scan_source: 'sast',
primary_preference: 'preferred',
priority: finding.mapping.priority,
sast_source_location: sourceLocationFromContext(finding.context),
} as ReconciliationObservation;
}
@@ -0,0 +1,61 @@
/** Closed per-class submit schema for one SAST enrichment response. */
import type { ReconciliationClass } from '../../../../types/reconciliation.js';
import { QUEUE_ENTRY_FIELD_NAMES } from '../../../queue-schemas.js';
// `ID` is omitted from the filtered class field list because it is prepended manually below instead
// (so it always appears exactly once); `code_locations` is omitted entirely, because that field is a
// structured array the model is never asked to reconstruct. Source location is authoritative code-owned
// data derived straight from validated SARIF (see sourceLocationFromContext in policy.ts), not
// something a free-text model response is trusted to supply.
const OMITTED_MODEL_FIELDS = new Set(['ID', 'code_locations']);
/** Evidence fields the model may return for one class, excluding authoritative source metadata. */
export function sastEnrichmentFieldNames(vulnerabilityClass: ReconciliationClass): readonly string[] {
return [
'ID',
...QUEUE_ENTRY_FIELD_NAMES[vulnerabilityClass].filter((field) => !OMITTED_MODEL_FIELDS.has(field)),
'_sastId',
];
}
function propertySchema(name: string): Record<string, unknown> {
if (name === '_sastId') {
return { type: 'integer', description: 'Copy the input _sastId exactly.' };
}
if (name === 'externally_exploitable') return { type: 'boolean' };
if (name === 'confidence') return { type: 'string', description: 'high | med | low' };
return { type: 'string' };
}
/**
* The envelope and each returned object are closed. Required-field and pairing
* checks remain application-owned so a malformed sibling can be dropped alone.
*/
export function sastEnrichmentToolSchema(vulnerabilityClass: ReconciliationClass): Record<string, unknown> {
const properties = Object.fromEntries(
sastEnrichmentFieldNames(vulnerabilityClass).map((field) => [field, propertySchema(field)]),
);
return {
type: 'object',
additionalProperties: false,
properties: {
vulnerabilities: {
type: 'array',
items: {
type: 'object',
additionalProperties: false,
properties,
},
},
},
required: ['vulnerabilities'],
};
}
export const SAST_ENRICHMENT_TOOL_DESCRIPTION =
'Return the enriched exploitation-queue vulnerabilities. Call exactly once as your final action.';
export function enrichmentPromptName(vulnerabilityClass: ReconciliationClass): string {
return `sast-enrichment-${vulnerabilityClass}`;
}
@@ -0,0 +1,205 @@
/** Validate, pair, and positively project one enrichment response. */
import type { ReconciliationClass } from '../../../../types/reconciliation.js';
import { normalizeConfidence } from '../cwe-mapper.js';
import type { ClassifiedFinding, Confidence } from '../types.js';
import { sastEnrichmentFieldNames } from './schema.js';
export class EnrichmentAttemptError extends Error {
constructor(message: string) {
super(message);
this.name = 'EnrichmentAttemptError';
}
}
export interface PairedVulnerability {
finding: ClassifiedFinding;
sastId: number;
evidence: Record<string, unknown>;
}
export interface ValidationCounts {
returned: number;
malformed: number;
orphaned: number;
duplicate_sast_id: number;
}
export interface ValidationResult {
paired: PairedVulnerability[];
counts: ValidationCounts;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
export function normalizeEnrichedConfidence(value: string): Confidence | undefined {
const normalized = value.trim().toLowerCase();
return normalizeConfidence(normalized === 'moderate' ? 'medium' : normalized);
}
/** Require the exact closed `{ vulnerabilities: [...] }` response envelope. */
export function extractVulnerabilities(toolArguments: unknown): unknown[] {
if (!isRecord(toolArguments) || !Array.isArray(toolArguments.vulnerabilities)) {
throw new EnrichmentAttemptError('submit_result did not return a vulnerabilities array');
}
if (Object.keys(toolArguments).length !== 1 || !Object.hasOwn(toolArguments, 'vulnerabilities')) {
throw new EnrichmentAttemptError('submit_result returned unexpected envelope fields');
}
return toolArguments.vulnerabilities;
}
function toRequiredString(value: unknown): string | undefined {
if (typeof value === 'string') return value;
if (typeof value === 'number' && Number.isFinite(value)) return String(value);
if (typeof value === 'boolean') return String(value);
return undefined;
}
function toBoolean(value: unknown): boolean | undefined {
if (typeof value === 'boolean') return value;
if (typeof value !== 'string') return undefined;
const normalized = value.trim().toLowerCase();
if (normalized === 'true') return true;
if (normalized === 'false') return false;
return undefined;
}
function toSastId(value: unknown): number | undefined {
let parsed: number;
if (typeof value === 'number') {
parsed = value;
} else if (typeof value === 'string' && value.trim().length > 0) {
parsed = Number(value.trim());
} else {
return undefined;
}
return Number.isSafeInteger(parsed) && parsed >= 0 ? parsed : undefined;
}
interface CoercedCommonFields {
element: Record<string, unknown>;
confidenceRaw: string;
sastId: number | undefined;
}
function coerceCommonFields(element: Record<string, unknown>): CoercedCommonFields | undefined {
const id = toRequiredString(element.ID);
const vulnerabilityType = toRequiredString(element.vulnerability_type);
const externallyExploitable = toBoolean(element.externally_exploitable);
const notes = toRequiredString(element.notes);
const confidencePresent = element.confidence !== undefined && element.confidence !== null;
if (
id === undefined ||
vulnerabilityType === undefined ||
externallyExploitable === undefined ||
notes === undefined ||
!confidencePresent
) {
return undefined;
}
// Confidence is present but not a scalar (an object or array). Carry a sentinel that no confidence
// vocabulary can normalize, so the caller rejects the whole response rather than guessing a value.
const scalarConfidence = toRequiredString(element.confidence);
const confidenceRaw = scalarConfidence ?? '[non-scalar]';
const sastId = toSastId(element._sastId);
return {
element: {
...element,
ID: id,
vulnerability_type: vulnerabilityType,
externally_exploitable: externallyExploitable,
confidence: confidenceRaw,
notes,
...(sastId !== undefined && { _sastId: sastId }),
},
confidenceRaw,
sastId,
};
}
function coerceEvidenceValue(key: string, value: unknown): unknown {
if (key === 'externally_exploitable') return value;
return toRequiredString(value) ?? JSON.stringify(value);
}
function buildEvidence(
element: Record<string, unknown>,
vulnerabilityClass: ReconciliationClass,
confidence: Confidence,
): Record<string, unknown> {
const evidence: Record<string, unknown> = {};
for (const key of sastEnrichmentFieldNames(vulnerabilityClass)) {
if (key === 'ID' || key === '_sastId' || key === 'confidence') continue;
const value = element[key];
if (value !== null && value !== undefined) evidence[key] = coerceEvidenceValue(key, value);
}
evidence.confidence = confidence;
return evidence;
}
// The model's own `ID` field (a free-text string it invents) is carried through as evidence but is
// never trusted for identity: only `_sastId`, the small integer the code itself assigned before the
// request, is looked up in `findingsById`. A model cannot forge or guess its way into pairing with a
// finding it was not actually sent, since `_sastId` values outside the sent batch simply have no entry.
/** Pair by code-assigned `_sastId`; never by response order or model-supplied ID. */
export function pairEnrichedVulnerabilities(
returned: readonly unknown[],
findingsById: ReadonlyMap<number, ClassifiedFinding>,
vulnerabilityClass: ReconciliationClass,
): ValidationResult {
const paired: PairedVulnerability[] = [];
const consumed = new Set<number>();
let malformed = 0;
let orphaned = 0;
let duplicate = 0;
const allowedFields = new Set(sastEnrichmentFieldNames(vulnerabilityClass));
for (const value of returned) {
if (!isRecord(value)) {
malformed++;
continue;
}
if (Object.keys(value).some((key) => !allowedFields.has(key))) {
malformed++;
continue;
}
const common = coerceCommonFields(value);
if (common === undefined) {
malformed++;
continue;
}
const confidence = normalizeEnrichedConfidence(common.confidenceRaw);
if (confidence === undefined) {
throw new EnrichmentAttemptError('Response has an unrecognized confidence value');
}
const sastId = common.sastId;
const finding = sastId === undefined ? undefined : findingsById.get(sastId);
if (sastId === undefined || finding === undefined) {
orphaned++;
continue;
}
if (consumed.has(sastId)) {
duplicate++;
continue;
}
consumed.add(sastId);
paired.push({
finding,
sastId,
evidence: buildEvidence(common.element, vulnerabilityClass, confidence),
});
}
return {
paired,
counts: {
returned: returned.length,
malformed,
orphaned,
duplicate_sast_id: duplicate,
},
};
}
@@ -0,0 +1,132 @@
/** Digest-pinned, workspace-contained SARIF byte intake. */
import { createHash } from 'node:crypto';
import type { Stats } from 'node:fs';
import { lstat, readFile, realpath } from 'node:fs/promises';
import path from 'node:path';
import { WORKSPACES_DIR } from '../../../paths.js';
import type { SarifRef } from '../../sast/types.js';
const SHA256_PATTERN = /^[a-f0-9]{64}$/;
export type SarifIntakeFailureKind = 'invalid' | 'io';
export class SarifIntakeError extends Error {
readonly kind: SarifIntakeFailureKind;
constructor(message: string, kind: SarifIntakeFailureKind = 'invalid') {
super(message);
this.name = 'SarifIntakeError';
this.kind = kind;
}
}
function isErrno(error: unknown, ...codes: readonly string[]): boolean {
return error instanceof Error && codes.includes((error as NodeJS.ErrnoException).code ?? '');
}
function intakeFileSystemError(error: unknown, invalidMessage: string, ioMessage: string): SarifIntakeError {
if (isErrno(error, 'ENOENT', 'ENOTDIR', 'ELOOP')) {
return new SarifIntakeError(invalidMessage);
}
return new SarifIntakeError(ioMessage, 'io');
}
function validSessionId(sessionId: string): boolean {
return (
sessionId.length > 0 &&
sessionId !== '.' &&
sessionId !== '..' &&
!sessionId.includes('/') &&
!sessionId.includes('\\') &&
!sessionId.includes('\0')
);
}
function contained(root: string, candidate: string): boolean {
const relative = path.relative(root, candidate);
return relative.length > 0 && !relative.startsWith('..') && !path.isAbsolute(relative);
}
/** Read exact SARIF bytes only after containment and no-symlink checks. */
export async function readPinnedSarif(
sessionId: string,
ref: SarifRef,
workspacesDir: string = WORKSPACES_DIR,
): Promise<Buffer> {
if (!validSessionId(sessionId)) throw new SarifIntakeError('Invalid SARIF session identifier');
if (!SHA256_PATTERN.test(ref.sha256)) throw new SarifIntakeError('SARIF reference digest is invalid');
let realWorkspaces: string;
try {
realWorkspaces = await realpath(workspacesDir);
} catch (error) {
throw intakeFileSystemError(
error,
'SARIF workspace root is not visible',
'SARIF workspace root could not be resolved',
);
}
const sessionRoot = path.join(realWorkspaces, sessionId);
let sessionStat: Stats;
try {
sessionStat = await lstat(sessionRoot);
} catch (error) {
throw intakeFileSystemError(
error,
'SARIF session workspace is not visible',
'SARIF session workspace could not be inspected',
);
}
if (sessionStat.isSymbolicLink() || !sessionStat.isDirectory()) {
throw new SarifIntakeError('SARIF session workspace is not a regular directory');
}
if (!path.isAbsolute(ref.path) || path.normalize(ref.path) !== ref.path || !contained(sessionRoot, ref.path)) {
throw new SarifIntakeError('SARIF path is outside the current scan workspace');
}
// Walk every path segment under the session root and reject a symlink at any level. Checking only
// the final component would let a symlinked parent directory redirect the read outside the
// workspace before the byte read and digest check ever run.
const relativeSegments = path.relative(sessionRoot, ref.path).split(path.sep);
let current = sessionRoot;
for (const segment of relativeSegments) {
current = path.join(current, segment);
let stat: Stats;
try {
stat = await lstat(current);
} catch (error) {
throw intakeFileSystemError(error, 'SARIF path is not visible', 'SARIF path could not be inspected');
}
if (stat.isSymbolicLink()) throw new SarifIntakeError('SARIF path contains a symlink');
}
let resolvedPath: string;
try {
resolvedPath = await realpath(ref.path);
} catch (error) {
throw intakeFileSystemError(error, 'SARIF path is not visible', 'SARIF path could not be resolved');
}
if (resolvedPath !== ref.path || !contained(sessionRoot, resolvedPath)) {
throw new SarifIntakeError('SARIF path escapes the current scan workspace');
}
let finalStat: Stats;
try {
finalStat = await lstat(resolvedPath);
} catch (error) {
throw intakeFileSystemError(error, 'SARIF path is not visible', 'SARIF path could not be inspected');
}
if (!finalStat.isFile()) throw new SarifIntakeError('SARIF path is not a regular file');
let bytes: Buffer;
try {
bytes = await readFile(resolvedPath);
} catch (error) {
throw intakeFileSystemError(error, 'SARIF bytes are not readable', 'SARIF bytes could not be read');
}
const observed = createHash('sha256').update(bytes).digest('hex');
if (observed !== ref.sha256) throw new SarifIntakeError('SARIF digest does not match the exact bytes');
return bytes;
}
@@ -0,0 +1,270 @@
/** Appendix A SARIF validator. Document failures throw; invalid findings are classified and dropped. */
import path from 'node:path';
import type {
DroppedSarifFinding,
ParsedSarif,
ParsedSarifFinding,
SarifFindingDropReason,
SarifLocation,
SarifResult,
SastPhase,
} from './types.js';
const CWE_PATTERN = /^CWE-[1-9][0-9]*$/;
const SEVERITIES = new Set(['Critical', 'High', 'Medium', 'Low', 'Info']);
const LEVELS = new Set(['error', 'warning', 'note']);
/** A run/document contract failure that invalidates the supplied SARIF reference. */
export class SarifDocumentError extends Error {
constructor(message: string) {
super(message);
this.name = 'SarifDocumentError';
}
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function nonemptyString(value: unknown): value is string {
return typeof value === 'string' && value.length > 0;
}
// Phase is inferred from the tool name string rather than an explicit field, since the SARIF
// contract does not carry a phase number directly. This is a best-effort classification: an
// unrecognized name falls through to phase 3, the generic case, rather than failing the finding.
function detectPhase(toolName: string): SastPhase {
const lower = toolName.toLowerCase();
if (lower.includes('check') || lower.includes('phase0') || lower.includes('phase_0')) return 0;
if (lower.includes('logic') || lower.includes('business') || lower.includes('phase4') || lower.includes('phase_4')) {
return 4;
}
return 3;
}
function hasTraversal(uri: string): boolean {
return uri.split('/').some((segment) => segment === '..' || segment === '.');
}
/** Whether a SARIF location is a normalized repository-relative POSIX path. */
export function isRepositoryRelativeSarifUri(uri: string): boolean {
if (uri.length === 0 || uri.trim() !== uri || uri.includes('\\') || uri.includes('\0')) return false;
if (path.posix.isAbsolute(uri) || /^[A-Za-z][A-Za-z0-9+.-]*:/.test(uri)) return false;
if (uri.includes('//') || hasTraversal(uri)) return false;
return path.posix.normalize(uri) === uri;
}
function validateRule(value: unknown): string {
if (!isRecord(value) || !nonemptyString(value.id) || !CWE_PATTERN.test(value.id)) {
throw new SarifDocumentError('SARIF rule metadata has an invalid CWE id');
}
if (!nonemptyString(value.name) || typeof value.helpUri !== 'string') {
throw new SarifDocumentError('SARIF rule metadata is missing required strings');
}
for (const descriptionKey of ['shortDescription', 'fullDescription']) {
const description = value[descriptionKey];
if (!isRecord(description) || typeof description.text !== 'string') {
throw new SarifDocumentError('SARIF rule metadata has an invalid description');
}
}
if (!isRecord(value.properties) || value.properties.cwe !== value.id || !Array.isArray(value.properties.tags)) {
throw new SarifDocumentError('SARIF rule metadata does not match its CWE id');
}
if (!value.properties.tags.every((tag) => typeof tag === 'string')) {
throw new SarifDocumentError('SARIF rule tags are invalid');
}
return value.id;
}
// `requireSourceRoot` differs between callers: a result's primary location must declare
// `uriBaseId: '%SRCROOT%'` explicitly (it is the location a finding is keyed on), while a code-flow
// step's location may omit `uriBaseId` entirely and is only rejected if it names something other
// than `%SRCROOT%`.
function locationParts(
value: unknown,
requireSourceRoot: boolean,
): {
location?: SarifLocation;
reason?: SarifFindingDropReason;
} {
if (!isRecord(value) || !isRecord(value.physicalLocation)) return { reason: 'location' };
const physical = value.physicalLocation;
if (!isRecord(physical.artifactLocation) || !isRecord(physical.region)) return { reason: 'location' };
const artifact = physical.artifactLocation;
const region = physical.region;
if (typeof artifact.uri !== 'string') return { reason: 'location' };
if (requireSourceRoot && artifact.uriBaseId !== '%SRCROOT%') return { reason: 'location' };
if (!requireSourceRoot && artifact.uriBaseId !== undefined && artifact.uriBaseId !== '%SRCROOT%') {
return { reason: 'location' };
}
if (hasTraversal(artifact.uri)) return { reason: 'traversal' };
if (!isRepositoryRelativeSarifUri(artifact.uri)) return { reason: 'location' };
if (!Number.isSafeInteger(region.startLine) || (region.startLine as number) <= 0) return { reason: 'line' };
if (
region.startColumn !== undefined &&
(!Number.isSafeInteger(region.startColumn) || (region.startColumn as number) <= 0)
) {
return { reason: 'location' };
}
if (region.snippet !== undefined && (!isRecord(region.snippet) || typeof region.snippet.text !== 'string')) {
return { reason: 'location' };
}
if (value.message !== undefined && (!isRecord(value.message) || typeof value.message.text !== 'string')) {
return { reason: 'location' };
}
return { location: value as unknown as SarifLocation };
}
function validateCodeFlows(value: unknown, primary: SarifLocation): SarifFindingDropReason | undefined {
if (!Array.isArray(value) || value.length === 0) return 'malformed';
let firstFlowLastLocation: SarifLocation | undefined;
for (let flowIndex = 0; flowIndex < value.length; flowIndex++) {
const flow = value[flowIndex];
if (!isRecord(flow) || !Array.isArray(flow.threadFlows) || flow.threadFlows.length === 0) return 'malformed';
for (let threadIndex = 0; threadIndex < flow.threadFlows.length; threadIndex++) {
const thread = flow.threadFlows[threadIndex];
if (!isRecord(thread) || !Array.isArray(thread.locations) || thread.locations.length === 0) return 'malformed';
for (let locationIndex = 0; locationIndex < thread.locations.length; locationIndex++) {
const step = thread.locations[locationIndex];
if (!isRecord(step) || !nonemptyString(step.importance)) return 'malformed';
const checked = locationParts(step.location, false);
if (checked.reason !== undefined) return checked.reason;
if (
!isRecord(step.location) ||
!isRecord(step.location.message) ||
typeof step.location.message.text !== 'string'
) {
return 'malformed';
}
if (flowIndex === 0 && threadIndex === 0 && locationIndex === thread.locations.length - 1) {
firstFlowLastLocation = checked.location;
}
}
}
}
// The last step of the first thread flow is the sink, and it must land on the same file and line
// as the result's primary location. A code flow whose sink disagrees with the reported location
// describes a different defect than it claims, so the finding is dropped.
const primaryPhysical = primary.physicalLocation;
const sinkPhysical = firstFlowLastLocation?.physicalLocation;
if (
sinkPhysical === undefined ||
sinkPhysical.artifactLocation.uri !== primaryPhysical.artifactLocation.uri ||
sinkPhysical.region.startLine !== primaryPhysical.region.startLine
) {
return 'location';
}
return undefined;
}
function resultRuleId(value: unknown): string | undefined {
return isRecord(value) && typeof value.ruleId === 'string' ? value.ruleId : undefined;
}
function validateResult(value: unknown, rules: ReadonlySet<string>): SarifFindingDropReason | SarifResult {
if (!isRecord(value)) return 'malformed';
if (!nonemptyString(value.ruleId) || !CWE_PATTERN.test(value.ruleId)) return 'rule';
if (!isRecord(value.properties) || value.properties.cwe !== value.ruleId || !rules.has(value.ruleId)) return 'rule';
if (!isRecord(value.message) || typeof value.message.text !== 'string') return 'malformed';
if (!LEVELS.has(value.level as string)) return 'severity';
if (!Array.isArray(value.locations) || value.locations.length === 0) return 'location';
const primary = locationParts(value.locations[0], true);
if (primary.reason !== undefined) return primary.reason;
if (primary.location === undefined) return 'location';
const codeFlowReason = validateCodeFlows(value.codeFlows, primary.location);
if (codeFlowReason !== undefined) return codeFlowReason;
if (!SEVERITIES.has(value.properties.severity as string)) return 'severity';
if (value.properties.status !== 'verified') return 'status';
if (value.properties.findingSubType !== 'AGENT_SAST') return 'subtype';
if (typeof value.properties.description !== 'string') return 'malformed';
// These property names belong to a richer internal finding shape than the one this pipeline
// accepts; a result carrying any of them was not produced against this exact SARIF contract (or
// carries content, such as a proof-of-concept, this pipeline never wants to ingest), so it is
// dropped rather than accepted with those fields silently ignored.
for (const forbidden of ['invariantDescription', 'owasp_category', 'proofOfConcept']) {
if (Object.hasOwn(value.properties, forbidden)) return 'malformed';
}
return value as unknown as SarifResult;
}
function zeroDropReasons(): Record<SarifFindingDropReason, number> {
return {
malformed: 0,
rule: 0,
location: 0,
traversal: 0,
line: 0,
severity: 0,
status: 0,
subtype: 0,
};
}
/** Parse and strictly validate the Capella SARIF byte contract. */
export function parseSarifContent(content: string): ParsedSarif {
let document: unknown;
try {
document = JSON.parse(content);
} catch {
throw new SarifDocumentError('SARIF document is not valid JSON');
}
if (!isRecord(document)) throw new SarifDocumentError('SARIF document is not an object');
if (!nonemptyString(document.$schema) || document.version !== '2.1.0') {
throw new SarifDocumentError('SARIF document metadata is invalid');
}
if (!Array.isArray(document.runs) || document.runs.length === 0) {
throw new SarifDocumentError('SARIF document has no runs');
}
const findings: ParsedSarifFinding[] = [];
const droppedFindings: DroppedSarifFinding[] = [];
const droppedByReason = zeroDropReasons();
let declaredFindings = 0;
for (const runValue of document.runs) {
if (!isRecord(runValue) || !isRecord(runValue.tool) || !isRecord(runValue.tool.driver)) {
throw new SarifDocumentError('SARIF run tool metadata is invalid');
}
const driver = runValue.tool.driver;
if (!nonemptyString(driver.name) || !nonemptyString(driver.version) || !nonemptyString(driver.informationUri)) {
throw new SarifDocumentError('SARIF run driver metadata is incomplete');
}
if (!Array.isArray(driver.rules) || !Array.isArray(runValue.results) || !isRecord(runValue.properties)) {
throw new SarifDocumentError('SARIF run arrays or properties are invalid');
}
if (
typeof runValue.properties.repository !== 'string' ||
!Number.isSafeInteger(runValue.properties.totalFindings) ||
runValue.properties.totalFindings !== runValue.results.length
) {
throw new SarifDocumentError('SARIF run finding count or repository metadata is invalid');
}
const rules = new Set<string>();
for (const rule of driver.rules) {
const id = validateRule(rule);
if (rules.has(id)) throw new SarifDocumentError('SARIF run contains duplicate rule metadata');
rules.add(id);
}
const phase = detectPhase(driver.name);
declaredFindings += runValue.results.length;
for (const resultValue of runValue.results) {
const validation = validateResult(resultValue, rules);
if (typeof validation === 'string') {
droppedByReason[validation]++;
const ruleId = resultRuleId(resultValue);
droppedFindings.push({ ...(ruleId !== undefined && { ruleId }), reason: validation });
continue;
}
findings.push({ result: validation, phase, toolName: driver.name });
}
}
return { findings, droppedFindings, droppedByReason, declaredFindings };
}
@@ -0,0 +1,132 @@
/** Strict SARIF and SAST-enrichment contracts. */
export type ShannonCategory = 'INJECTION' | 'XSS' | 'AUTH' | 'AUTHZ' | 'SSRF' | 'MISC';
export type Priority = 'P1' | 'P2' | 'P3';
export type Confidence = 'high' | 'medium' | 'low';
// Which analysis phase produced a finding, detected from the SARIF driver's tool name (see
// `detectPhase` in sarif-parser.ts): 0 is an early check-style phase, 4 is business-logic analysis,
// and 3 is every other phase. Kept as a small closed set of numbers rather than named phase strings
// because it only needs to round-trip through enrichment, not describe the phase to a person.
export type SastPhase = 0 | 3 | 4;
export interface CWEMapping {
category: ShannonCategory;
name: string;
priority: Priority;
}
export interface SarifRegion {
startLine: number;
startColumn?: number;
snippet?: { text: string };
}
export interface SarifArtifactLocation {
uri: string;
uriBaseId?: '%SRCROOT%';
}
export interface SarifLocation {
physicalLocation: {
artifactLocation: SarifArtifactLocation;
region: SarifRegion;
};
message?: { text: string };
}
export interface SarifThreadFlowLocation {
location: SarifLocation;
importance: string;
}
export interface SarifResult {
ruleId: string;
level: 'error' | 'warning' | 'note';
message: { text: string };
locations: [SarifLocation, ...unknown[]];
codeFlows: Array<{
threadFlows: Array<{
locations: SarifThreadFlowLocation[];
}>;
}>;
properties: {
severity: 'Critical' | 'High' | 'Medium' | 'Low' | 'Info';
cwe: string;
status: 'verified';
description: string;
findingSubType: 'AGENT_SAST';
};
}
export type SarifFindingDropReason =
| 'malformed'
| 'rule'
| 'location'
| 'traversal'
| 'line'
| 'severity'
| 'status'
| 'subtype';
export interface ParsedSarifFinding {
result: SarifResult;
phase: SastPhase;
toolName: string;
}
export interface DroppedSarifFinding {
ruleId?: string;
reason: SarifFindingDropReason;
}
export interface ParsedSarif {
findings: ParsedSarifFinding[];
droppedFindings: DroppedSarifFinding[];
droppedByReason: Record<SarifFindingDropReason, number>;
declaredFindings: number;
}
export interface DataflowFindingContext {
cwe: string;
message: string;
severity: string;
confidence: number;
sinkFile: string;
sinkLine: number;
sinkColumn: number;
sinkSnippet: string;
sourceFile: string;
sourceLine: number;
sourceColumn: number;
sourceSnippet: string;
dataflowPath: Array<{
file: string;
line: number;
column: number;
snippet: string;
role: string;
}>;
validationReason: string;
sanitizationStatus: string;
}
export interface LocalizedFindingContext {
cwe: string;
message: string;
severity: string;
confidence: number;
file: string;
line: number;
column: number;
snippet: string;
}
// A dataflow context is built when a finding's code flow names a distinct source and sink;
// otherwise the finding gets a localized context describing only its single reported location.
export type FindingContext = DataflowFindingContext | LocalizedFindingContext;
export interface ClassifiedFinding {
context: FindingContext;
mapping: CWEMapping;
phase: SastPhase;
}
@@ -0,0 +1,7 @@
/**
* Schema version shared by every reconciliation artifact and publication.
*
* Keep this leaf module free of runtime imports so workflow-safe type modules can
* refer to the version without pulling filesystem code into a workflow bundle.
*/
export const RECONCILIATION_SCHEMA_VERSION = 1 as const;
@@ -0,0 +1,205 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Durable, queue-only producer seeding for the analysis-less `miscellaneous` class. */
import { createHash, randomUUID } from 'node:crypto';
import { lstat, rename, unlink, writeFile } from 'node:fs/promises';
import path from 'node:path';
import {
commitExactPaths,
ExactPathCommitMismatchError,
gitBlobShaForContents,
lastCommitForPathAtHead,
readCommittedFile,
restorePathsFromHead,
withGitRepoLock,
} from '../../services/git-manager.js';
import type { ActivityLogger } from '../../types/activity-logger.js';
import { PublicationConflictError, ReconciliationError, ReconciliationIoError } from './artifact-store.js';
import { isManifestCoherent, readPublishedManifest } from './manifest.js';
import { exploitationQueuePath, reconciliationManifestPath, sastProvenancePath } from './prepare.js';
import { publicationContractForClass } from './publish.js';
/**
* Canonical committed producer bytes for the analysis-less class.
*
* Every seed writes these exact bytes, so a re-run after a lost acknowledgement produces an
* identical blob and the seed is idempotent. Any other committed content for the `miscellaneous` queue is
* treated as a conflict, never overwritten.
*/
export const CANONICAL_EMPTY_MISCELLANEOUS_QUEUE = `${JSON.stringify({ vulnerabilities: [] }, null, 2)}\n`;
export interface SeedEmptyProducerQueueArgs {
deliverablesDir: string;
sessionId: string;
logger: ActivityLogger;
}
export interface SeedEmptyProducerQueueResult {
alreadySeeded: boolean;
alreadyPublished: boolean;
commitHash: string;
}
function sha256Text(contents: string): string {
return createHash('sha256').update(contents, 'utf8').digest('hex');
}
function isErrno(error: unknown, code: string): boolean {
return error instanceof Error && (error as NodeJS.ErrnoException).code === code;
}
async function rejectQueueSymlink(deliverablesDir: string, queuePath: string): Promise<void> {
try {
const stat = await lstat(path.join(deliverablesDir, queuePath));
if (stat.isSymbolicLink()) throw new PublicationConflictError('Refusing to seed through a symlink');
} catch (error) {
if (isErrno(error, 'ENOENT')) return;
if (error instanceof ReconciliationError) throw error;
throw new ReconciliationIoError('Unable to inspect the miscellaneous producer-queue destination');
}
}
async function writeQueueReplacingEntry(absolutePath: string): Promise<void> {
const temporaryPath = `${absolutePath}.tmp-${randomUUID()}`;
try {
await writeFile(temporaryPath, CANONICAL_EMPTY_MISCELLANEOUS_QUEUE, { flag: 'wx' });
await rename(temporaryPath, absolutePath);
} catch (error) {
await unlink(temporaryPath).catch(() => undefined);
throw error;
}
}
async function publicationCommit(deliverablesDir: string, relativePath: string): Promise<string> {
const commitHash = await lastCommitForPathAtHead(deliverablesDir, relativePath);
if (commitHash === null) throw new ReconciliationIoError('Unable to read the exact-path publication commit');
return commitHash;
}
async function verifyManifestConsumers(
deliverablesDir: string,
consumers: ReadonlyArray<{ path: string; sha256: string }>,
): Promise<void> {
for (const consumer of consumers) {
const committed = await readCommittedFile(deliverablesDir, consumer.path);
if (committed.state !== 'present' || sha256Text(committed.contents) !== consumer.sha256) {
throw new PublicationConflictError('Existing miscellaneous publication is missing a coherent committed consumer');
}
}
}
/**
* Seed `miscellaneous_exploitation_queue.json` once, or return an existing seed/final publication.
*
* Resolves to one of three states under the Git lock: a coherent final publication already exists
* and is adopted (`alreadyPublished`); the canonical empty queue is already committed and adopted
* (`alreadySeeded`); or nothing is committed yet and the canonical queue is written and committed
* fresh. A committed queue with non-canonical bytes, or standalone provenance with no manifest,
* is a conflict rather than a state to reprocess.
*/
// The `miscellaneous` class has no analysis agent of its own: nothing runs vulnerability analysis
// against it directly, so unlike the five core classes it never gets a producer queue from an
// upstream agent. Seeding the canonical empty queue here, then running it through the same
// publish/manifest machinery as every other class, means downstream consumers (materialization,
// the report, the exploitation phase) never need a miscellaneous-specific code path for "this class
// might not have a queue file at all."
export async function seedEmptyProducerQueue(args: SeedEmptyProducerQueueArgs): Promise<SeedEmptyProducerQueueResult> {
const queuePath = exploitationQueuePath('miscellaneous');
const manifestPath = reconciliationManifestPath('miscellaneous');
const provenancePath = sastProvenancePath('miscellaneous');
const finalContracts = [
publicationContractForClass('miscellaneous', false),
publicationContractForClass('miscellaneous', true),
];
return withGitRepoLock(async (): Promise<SeedEmptyProducerQueueResult> => {
const manifestRead = await readPublishedManifest(args.deliverablesDir, manifestPath);
if (manifestRead.state === 'invalid') {
throw new PublicationConflictError(`Corrupt miscellaneous manifest in HEAD: ${manifestRead.reason}`);
}
if (manifestRead.state === 'present') {
const producerBlobSha = await gitBlobShaForContents(args.deliverablesDir, CANONICAL_EMPTY_MISCELLANEOUS_QUEUE);
const coherentFinalContract = finalContracts.some((contract) =>
isManifestCoherent({
manifest: manifestRead.manifest,
sessionId: args.sessionId,
vulnerabilityClass: 'miscellaneous',
contract,
producerQueuePath: queuePath,
producerBlobSha,
}),
);
if (!coherentFinalContract) {
throw new PublicationConflictError(
'Existing miscellaneous manifest does not cohere with the final publication',
);
}
await verifyManifestConsumers(args.deliverablesDir, manifestRead.manifest.consumer_files);
await rejectQueueSymlink(args.deliverablesDir, queuePath);
await restorePathsFromHead(args.deliverablesDir, [queuePath]);
return {
alreadySeeded: true,
alreadyPublished: true,
commitHash: await publicationCommit(args.deliverablesDir, manifestPath),
};
}
const provenanceRead = await readCommittedFile(args.deliverablesDir, provenancePath);
if (provenanceRead.state !== 'absent') {
throw new PublicationConflictError('Pre-manifest miscellaneous state contains standalone provenance');
}
const queueRead = await readCommittedFile(args.deliverablesDir, queuePath);
if (queueRead.state === 'corrupt') {
throw new PublicationConflictError('Committed miscellaneous producer queue is unreadable');
}
if (queueRead.state === 'present') {
if (queueRead.contents !== CANONICAL_EMPTY_MISCELLANEOUS_QUEUE) {
throw new PublicationConflictError('Pre-manifest miscellaneous producer queue is not canonical empty state');
}
await rejectQueueSymlink(args.deliverablesDir, queuePath);
await restorePathsFromHead(args.deliverablesDir, [queuePath]);
return {
alreadySeeded: true,
alreadyPublished: false,
commitHash: await publicationCommit(args.deliverablesDir, queuePath),
};
}
await rejectQueueSymlink(args.deliverablesDir, queuePath);
try {
await writeQueueReplacingEntry(path.join(args.deliverablesDir, queuePath));
} catch {
await restorePathsFromHead(args.deliverablesDir, [queuePath]);
throw new ReconciliationIoError('Unable to write the canonical miscellaneous producer queue');
}
let commitHash: string;
try {
const committed = await commitExactPaths(
args.deliverablesDir,
[queuePath],
'Seed miscellaneous producer queue',
args.logger,
[queuePath],
);
commitHash = committed.commitHash;
} catch (error) {
await restorePathsFromHead(args.deliverablesDir, [queuePath]);
if (error instanceof ExactPathCommitMismatchError) {
throw new PublicationConflictError('Miscellaneous seed staged path set differs from its queue-only contract');
}
throw new ReconciliationIoError('Unable to commit the canonical miscellaneous producer queue');
}
const committedQueue = await readCommittedFile(args.deliverablesDir, queuePath);
if (committedQueue.state !== 'present' || committedQueue.contents !== CANONICAL_EMPTY_MISCELLANEOUS_QUEUE) {
throw new PublicationConflictError('Committed miscellaneous seed bytes do not match canonical empty state');
}
return { alreadySeeded: false, alreadyPublished: false, commitHash };
});
}
@@ -0,0 +1,160 @@
/** Type-only wire and artifact-body contracts for reconciliation stages. */
import type { SarifRef } from '../sast/types.js';
import type {
ArtifactRef,
ClassEvidence,
Priority,
ReconciliationObservation,
ReconciliationTask,
SastSourceLocation,
ScanSource,
} from './contracts.js';
// "Positive" projection means this type is built by copying named fields in, never by taking the
// full observation and deleting fields out. A field that is not explicitly listed here (including
// `producer_id` and every other internal key) cannot appear on this type at all, so a future field
// added to `ReconciliationObservation` is model-invisible by default instead of leaking by default.
/** Positive model projection of one observation. */
export type ObservationView<E extends ClassEvidence = ClassEvidence> = E & {
scan_source: ScanSource;
priority?: Priority;
sast_source_location?: SastSourceLocation;
};
export interface TaskFormationInput {
queued_findings: Array<{ label: string; entry: ObservationView }>;
}
export interface TaskFormationOutput {
groups: Array<{ queue_labels: string[]; reasoning: string }>;
}
// Identifies the exact committed producer queue a reconciliation run was prepared against. Publish
// re-reads this queue at commit time and compares both the Git blob SHA and the content digest, so
// a queue that changed in HEAD between preparation and commit is caught rather than silently
// published against stale tasks.
export interface ProducerQueueIdentity {
path: string;
blob_sha: string;
digest: string;
}
export interface ProducerObservationsBody {
observations: ReconciliationObservation[];
producer_queue: ProducerQueueIdentity;
}
/** Optional adapter-facing provenance row. Standalone enrichment emits no rows. */
export interface SupplementalProvenanceRecord {
producer_id: string;
repository_id?: string;
scan_run_id?: string;
rule_id: string;
file: string;
line: number;
column: number;
}
// Counts only, never identities: this is telemetry surfaced to the scan log, not a channel that
// carries any producer ID or SARIF content forward. See `dropped_findings` on the body below for
// the one place a dropped finding's identity is actually retained.
export interface SupplementalDropCounts {
unknown_cwe: number;
other_category: number;
malformed: number;
orphaned: number;
duplicate_sast_id: number;
enrichment_dropped: number;
}
/**
* Identity of one finding that was sent for enrichment but never paired back.
*
* Recovered from the sent side, so it is always complete: a malformed response may
* carry no usable `sastId`, which is precisely what makes it malformed.
*/
export interface SupplementalDroppedFinding {
producer_id: string;
sast_id: number;
sast_source_location: SastSourceLocation;
}
export interface SupplementalObservationsBody {
observations: ReconciliationObservation[];
provenance: SupplementalProvenanceRecord[];
sarif?: SarifRef;
drops: SupplementalDropCounts;
// Sibling of `drops`, never a member of it: `drops` is spread into the artifact envelope's
// numeric `counts` map, which rejects any non-integer value.
dropped_findings: SupplementalDroppedFinding[];
}
export interface AcceptedTaskGroup {
producer_ids: string[];
reasoning: string;
}
// `model_ran` is false for both the "fewer than two observations" skip and any future zero-request
// path; it lets a reader of this artifact tell a genuine empty result apart from a model call that
// simply produced no groups.
export interface TaskFormationBody {
model_ran: boolean;
groups: AcceptedTaskGroup[];
rejected_group_count: number;
dropped_unknown_label_count: number;
}
// `observation_to_task` is the complete forward index from every observation's producer ID to the
// task it was materialized into (whether as primary or as a merged member). Publication uses it to
// prove every observation was placed exactly once before anything is written.
export interface FixedTasksBody {
tasks: ReconciliationTask[];
observation_to_task: Record<string, string>;
}
export interface ArtifactBodyMap {
'producer-observations': ProducerObservationsBody;
'supplemental-observations': SupplementalObservationsBody;
'task-formation': TaskFormationBody;
'fixed-tasks': FixedTasksBody;
}
export interface PrepareAlreadyPublished {
outcome: 'already_published';
manifestSha256: string;
}
export interface PreparePending {
outcome: 'pending';
ref: ArtifactRef<'producer-observations'>;
}
export type PrepareResult = PrepareAlreadyPublished | PreparePending;
export interface StageMetrics {
costUsd: number;
modelCalls: number;
inputTokens: number;
outputTokens: number;
}
export interface EnrichSuccess {
ref: ArtifactRef<'supplemental-observations'>;
metrics: StageMetrics;
}
export interface FormSuccess {
ref: ArtifactRef<'task-formation'>;
metrics: StageMetrics;
}
// Sentinel result of task formation when the model stage exhausted its retries on an eligible
// failure. Materialization treats it as an instruction to skip grouping and give every observation
// its own task, so a reconciliation still completes without any formation artifact.
export const SINGLETON_FALLBACK = 'singleton_fallback' as const;
export type FormResult = FormSuccess | typeof SINGLETON_FALLBACK;
export interface MaterializeResult {
ref: ArtifactRef<'fixed-tasks'>;
}
@@ -0,0 +1,118 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Captured submit tool with closed-schema and cross-group validation before capture. */
import { defineTool } from '@earendil-works/pi-coding-agent';
import { Type } from 'typebox';
import { Value } from 'typebox/value';
import type { CapturedSubmitTool } from '../submit-tool.js';
export type SubmitValidator = (parameters: unknown) => readonly string[];
export interface ValidatingSubmitTool extends CapturedSubmitTool {
readonly getAcceptedCount: () => number;
readonly sawRejectedSubmission: () => boolean;
}
function rejection(problems: readonly string[]): {
content: Array<{ type: 'text'; text: string }>;
details: undefined;
} {
const detail = problems.map((problem) => `- ${problem}`).join('\n');
const message = `Your answer was not accepted. Fix the following and call submit_result again:\n${detail}`;
return {
content: [
{
type: 'text',
text: JSON.stringify({ status: 'error', errorType: 'ValidationError', retryable: true, message }),
},
],
details: undefined,
};
}
const VALIDATING_DIRECTIVE =
'\n\nDeliver your structured answer by calling submit_result. If it reports problems, correct them and call it ' +
'again. Once it accepts your answer, stop. Do not output JSON as text.';
/** Build one executor-owned submit tool that captures only an accepted, schema-closed payload. */
export function createValidatingSubmitTool(
schema: Record<string, unknown>,
validate: SubmitValidator,
): ValidatingSubmitTool {
const parametersSchema = Type.Unsafe(schema);
let captured: unknown;
let acceptedCount = 0;
let rejected = false;
return {
tool: defineTool({
name: 'submit_result',
label: 'Submit task groups',
description: 'Submit task groups. Correct a rejected submission, then stop after the first accepted call.',
promptSnippet: 'submit_result: submit task groups; correct and resubmit only when rejected',
promptGuidelines: [
'Call submit_result to deliver task groups.',
'If it reports validation problems, correct them and call it again.',
'Stop after the first accepted submission. Do not output JSON as text.',
],
parameters: parametersSchema,
async execute(_toolCallId, parameters) {
if (!Value.Check(parametersSchema, parameters)) {
rejected = true;
return rejection(['The submission does not match the closed task-formation schema.']);
}
const problems = validate(parameters);
if (problems.length > 0) {
rejected = true;
return rejection(problems);
}
// Only the first accepted submission is captured. A second accepted call is refused and
// terminates the session, so the stage always materializes from a single settled answer
// rather than silently taking the last of several.
acceptedCount += 1;
if (acceptedCount > 1) {
rejected = true;
return {
...rejection(['Only one accepted submission is allowed.']),
terminate: true,
};
}
captured = parameters;
return {
content: [{ type: 'text' as const, text: 'Task groups accepted.' }],
details: undefined,
terminate: true,
};
},
}),
getCaptured: () => captured,
getAcceptedCount: () => acceptedCount,
sawRejectedSubmission: () => rejected,
directive: VALIDATING_DIRECTIVE,
};
}
export const PROBLEM_LABEL_PREVIEW = 6;
export function previewLabels(labels: readonly string[], maximum: number = PROBLEM_LABEL_PREVIEW): string {
const shown = labels.slice(0, maximum).join(', ');
const remaining = labels.length - maximum;
return remaining > 0 ? `${shown} (+${remaining} more)` : shown;
}
export function describeUnknownLabels(unknown: readonly string[], kind: string): string {
if (unknown.length === 0) return '';
return `These are not labels of any ${kind} in this task: ${previewLabels(unknown)}. Use only supplied labels.`;
}
export function describeReusedLabels(reused: readonly string[], container = 'submission'): string {
if (reused.length === 0) return '';
return `Each label belongs to at most one ${container}; these are reused: ${previewLabels(reused)}. Remove every duplicate claim.`;
}
@@ -0,0 +1,209 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Closed Pass 1 submission schema and defensive group acceptance. */
import { describeReusedLabels, describeUnknownLabels } from './submit-validation.js';
const TOP_LEVEL_KEYS = Object.freeze(['groups'] as const);
const GROUP_KEYS = Object.freeze(['queue_labels', 'reasoning'] as const);
export interface TaskFormationGroup {
readonly queue_labels: readonly string[];
readonly reasoning: string;
}
export interface AcceptedTaskGroups {
readonly groups: readonly TaskFormationGroup[];
readonly rejectedGroupCount: number;
readonly droppedUnknownLabelCount: number;
}
interface ParsedGroup {
readonly group?: TaskFormationGroup;
readonly labels: readonly string[];
readonly duplicateLabels: readonly string[];
readonly closed: boolean;
readonly structurallyValid: boolean;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function hasExactKeys(value: Record<string, unknown>, expected: readonly string[]): boolean {
const actual = Object.keys(value).sort();
const sortedExpected = [...expected].sort();
return actual.length === sortedExpected.length && actual.every((key, index) => key === sortedExpected[index]);
}
function duplicateValues(values: readonly string[]): string[] {
const seen = new Set<string>();
const duplicates = new Set<string>();
for (const value of values) {
if (seen.has(value)) duplicates.add(value);
seen.add(value);
}
return [...duplicates];
}
function parseGroup(value: unknown): ParsedGroup {
if (!isRecord(value)) {
return { labels: [], duplicateLabels: [], closed: false, structurallyValid: false };
}
const closed = hasExactKeys(value, GROUP_KEYS);
const rawLabels = value.queue_labels;
const labels = Array.isArray(rawLabels)
? rawLabels.filter((label): label is string => typeof label === 'string')
: [];
const duplicateLabels = duplicateValues(labels);
const structurallyValid =
closed &&
Array.isArray(rawLabels) &&
labels.length === rawLabels.length &&
labels.length >= 2 &&
duplicateLabels.length === 0 &&
typeof value.reasoning === 'string' &&
value.reasoning.trim().length > 0;
return {
...(structurallyValid && { group: { queue_labels: labels, reasoning: value.reasoning as string } }),
labels,
duplicateLabels,
closed,
structurallyValid,
};
}
function capturedGroups(captured: unknown): { readonly closed: boolean; readonly groups: readonly unknown[] } {
if (!isRecord(captured) || !hasExactKeys(captured, TOP_LEVEL_KEYS) || !Array.isArray(captured.groups)) {
return { closed: false, groups: [] };
}
return { closed: true, groups: captured.groups };
}
/** Build the call-local schema. Both object layers reject unknown properties. */
export function buildTaskFormationSchema(queueLabels: readonly string[]): Record<string, unknown> {
return {
type: 'object',
additionalProperties: false,
required: ['groups'],
properties: {
groups: {
type: 'array',
description:
'Sets of observations that predict one exploitation attempt. Use an empty array when every observation stands alone.',
items: {
type: 'object',
additionalProperties: false,
required: ['queue_labels', 'reasoning'],
properties: {
queue_labels: {
type: 'array',
minItems: 2,
uniqueItems: true,
items: { type: 'string', enum: [...queueLabels] },
description: 'Two or more distinct labels supplied in this call. A label belongs to at most one group.',
},
reasoning: {
type: 'string',
minLength: 1,
description: 'Why one exploitation attempt and one verdict settle every named observation.',
},
},
},
},
},
};
}
/** Explain cross-group and nonblank constraints so the model can correct a rejected call. */
export function findTaskFormationProblems(captured: unknown, queueLabels: ReadonlySet<string>): string[] {
const envelope = capturedGroups(captured);
if (!envelope.closed) return ['The submission must be exactly one object with a groups array and no other fields.'];
const parsed = envelope.groups.map(parseGroup);
const problems: string[] = [];
const unknown = new Set<string>();
const duplicateWithinGroup = new Set<string>();
const labelUse = new Map<string, number>();
let malformedGroups = 0;
let undersizedGroups = 0;
let blankReasoningGroups = 0;
for (let index = 0; index < envelope.groups.length; index++) {
const raw = envelope.groups[index];
const group = parsed[index] as ParsedGroup;
if (!isRecord(raw) || !group.closed || !Array.isArray(raw.queue_labels)) malformedGroups++;
if (group.labels.length < 2) undersizedGroups++;
if (isRecord(raw) && (typeof raw.reasoning !== 'string' || raw.reasoning.trim().length === 0)) {
blankReasoningGroups++;
}
for (const duplicate of group.duplicateLabels) duplicateWithinGroup.add(duplicate);
for (const label of group.labels) {
if (!queueLabels.has(label)) unknown.add(label);
labelUse.set(label, (labelUse.get(label) ?? 0) + 1);
}
}
if (malformedGroups > 0) {
problems.push('Every group must contain exactly queue_labels and reasoning, with no other fields.');
}
if (undersizedGroups > 0) {
problems.push(
'Every group must name at least two distinct queued observations; leave single observations ungrouped.',
);
}
if (duplicateWithinGroup.size > 0) {
problems.push(`A group cannot repeat a label: ${[...duplicateWithinGroup].join(', ')}.`);
}
if (blankReasoningGroups > 0) {
problems.push('Every group needs nonblank reasoning tied to one exploitation attempt and one verdict.');
}
const unknownMessage = describeUnknownLabels([...unknown], 'queued finding');
if (unknownMessage) problems.push(unknownMessage);
const reused = [...labelUse.entries()].filter(([, count]) => count > 1).map(([label]) => label);
const reusedMessage = describeReusedLabels(reused, 'group');
if (reusedMessage) problems.push(reusedMessage);
return problems;
}
/**
* Defensively accept only closed, well-formed groups. Reuse is counted across every submitted
* group, including a group already invalid for another reason, so every claimant is discarded.
*/
export function acceptTaskGroups(captured: unknown, queueLabels: ReadonlySet<string>): AcceptedTaskGroups {
const envelope = capturedGroups(captured);
if (!envelope.closed) {
const submittedCount = isRecord(captured) && Array.isArray(captured.groups) ? captured.groups.length : 0;
return { groups: [], rejectedGroupCount: submittedCount, droppedUnknownLabelCount: 0 };
}
const parsed = envelope.groups.map(parseGroup);
const labelUse = new Map<string, number>();
for (const group of parsed) {
for (const label of group.labels) labelUse.set(label, (labelUse.get(label) ?? 0) + 1);
}
let droppedUnknownLabelCount = 0;
const groups: TaskFormationGroup[] = [];
for (const parsedGroup of parsed) {
const containsUnknown = parsedGroup.labels.some((label) => !queueLabels.has(label));
if (containsUnknown) droppedUnknownLabelCount++;
if (!parsedGroup.structurallyValid || parsedGroup.group === undefined || containsUnknown) continue;
if (parsedGroup.labels.some((label) => labelUse.get(label) !== 1)) continue;
groups.push(parsedGroup.group);
}
return {
groups,
rejectedGroupCount: envelope.groups.length - groups.length,
droppedUnknownLabelCount,
};
}
@@ -9,8 +9,7 @@
* per-run valid-ID set).
*
* Exposes a single TypeBox-validated tool `add_exploit`, called once per
* processed vulnerability by the 5 exploit-* agents (injection, xss, auth,
* ssrf, authz). After the agent terminates, the host harvests
* processed vulnerability by the exploit-* agents. After the agent terminates, the host harvests
* collector.getAll() and runs exploit-renderer to produce
* {class}_exploitation_evidence.md. The collector state is the structured
* output.
@@ -36,16 +35,18 @@
*/
import { defineTool, type ToolDefinition } from '@earendil-works/pi-coding-agent';
import { type TSchema, Type } from 'typebox';
import { type Static, type TSchema, Type } from 'typebox';
import { Value } from 'typebox/value';
import { isTaskReference, REF_PREFIX } from '../ai/reconciliation/refs.js';
import { ALL_RECONCILIATION_CLASSES, type ReconciliationClass } from '../types/reconciliation.js';
import { stringEnum } from './schema.js';
// ============================================================================
// CLASS DISCRIMINATOR
// ============================================================================
export const EXPLOIT_VULN_CLASSES = ['injection', 'xss', 'auth', 'ssrf', 'authz'] as const;
export type VulnClass = (typeof EXPLOIT_VULN_CLASSES)[number];
export const EXPLOIT_VULN_CLASSES = ALL_RECONCILIATION_CLASSES;
export type ExploitClass = ReconciliationClass;
// ============================================================================
// SCHEMA CONSTANTS
@@ -54,6 +55,43 @@ export type VulnClass = (typeof EXPLOIT_VULN_CLASSES)[number];
const SEVERITY_VALUES = ['critical', 'high', 'medium', 'low'] as const;
const CONFIDENCE_VALUES = ['high', 'medium', 'low'] as const;
const exploitEvidenceLocationSchema = Type.Object(
{
workspace_relative_file_path: Type.String({
minLength: 1,
description:
'Non-empty POSIX path to code inspected during exploitation, relative to the Shannon workspace. ' +
'Do not use a leading slash, backslash, empty segment, or "." or ".." segment.',
}),
line_number: Type.Union([Type.Integer({ minimum: 1 }), Type.Null()], {
description: 'Exact one-based line inspected, or null when the file is known but the line is not.',
}),
},
{ additionalProperties: false },
);
const codeLocationsField = Type.Optional(
Type.Array(exploitEvidenceLocationSchema, {
minItems: 1,
description:
'Code locations inspected while investigating this task. Omit when none were inspected; never send an empty array.',
}),
);
export type ExploitEvidenceLocation = Static<typeof exploitEvidenceLocationSchema>;
/** Whether a path is a normalized POSIX path relative to the scan workspace. */
export function isValidExploitEvidencePath(value: string): boolean {
if (value.length === 0 || value.includes('\\') || value.includes('\0') || value.startsWith('/')) {
return false;
}
if (/^[A-Za-z]:\//.test(value)) {
return false;
}
const segments = value.split('/');
return segments.every((segment) => segment.length > 0 && segment !== '.' && segment !== '..');
}
const VALID_IDS_PREVIEW_LIMIT = 8;
function formatValidIdsPreview(validIds: ReadonlySet<string>): string {
@@ -77,6 +115,7 @@ export type ExploitedExploit = {
impact: string;
exploitation_steps: string[];
proof_of_impact: string;
code_locations?: ExploitEvidenceLocation[];
notes?: string | null;
};
@@ -94,6 +133,7 @@ export type BlockedExploit = {
what_we_tried: string;
how_this_would_be_exploited: string[];
expected_impact: string;
code_locations?: ExploitEvidenceLocation[];
notes?: string | null;
};
@@ -107,7 +147,7 @@ export function buildSchemas(validIds: ReadonlySet<string>) {
const vulnerabilityIdField = Type.String({
minLength: 1,
description:
'Vulnerability identifier (e.g. "INJ-VULN-03"). Must match an ID from this run\'s ' +
'Stable exploitation-task identifier (e.g. "INJ-03" or "MISC-01"). Must match an ID from this run\'s ' +
'{class}_exploitation_queue.json exactly — the collector rejects IDs not in the queue. ' +
`Valid IDs for this run: ${formatValidIdsPreview(validIds)}.`,
});
@@ -289,6 +329,7 @@ export function buildSchemas(validIds: ReadonlySet<string>) {
impact: impactField,
exploitation_steps: exploitationStepsField,
proof_of_impact: proofOfImpactField,
code_locations: codeLocationsField,
confidence: confidenceField,
current_blocker: currentBlockerField,
potential_impact: potentialImpactField,
@@ -311,6 +352,7 @@ export function buildSchemas(validIds: ReadonlySet<string>) {
impact: Type.String({ minLength: 1 }),
exploitation_steps: Type.Array(Type.String({ minLength: 1 }), { minItems: 1 }),
proof_of_impact: Type.String({ minLength: 1 }),
code_locations: codeLocationsField,
notes: notesField,
});
@@ -328,6 +370,7 @@ export function buildSchemas(validIds: ReadonlySet<string>) {
what_we_tried: Type.String({ minLength: 1 }),
how_this_would_be_exploited: Type.Array(Type.String({ minLength: 1 }), { minItems: 1 }),
expected_impact: Type.String({ minLength: 1 }),
code_locations: codeLocationsField,
notes: notesField,
});
@@ -377,7 +420,10 @@ export interface ExploitCollector {
}
export interface CreateExploitCollectorOptions {
vulnClass: VulnClass;
vulnClass: ExploitClass;
// Stable task references (e.g. "INJ-01") assigned by reconciliation for this run, not the
// producer IDs (e.g. "INJ-VULN-01") those tasks were built from. The exploit agent only ever
// sees this task namespace.
validIds: ReadonlySet<string>;
}
@@ -413,6 +459,31 @@ export function createExploitCollector(options: CreateExploitCollectorOptions):
}
const typed = Value.Clean(StrictSchema, structuredClone(input)) as AddExploitInput;
const invalidLocationPaths = (typed.code_locations ?? [])
.map((location) => location.workspace_relative_file_path)
.filter((locationPath) => !isValidExploitEvidencePath(locationPath));
if (invalidLocationPaths.length > 0) {
return errorResult(
`Invalid workspace-relative POSIX code location path(s): ${invalidLocationPaths.join(', ')}. ` +
'Use non-empty relative paths with no backslashes, empty segments, ".", or ".." segments.',
'ValidationError',
true,
);
}
// Enforces the reconciliation boundary: an exploit agent only ever reasons about stable
// task references, never the producer IDs (VULN-/SAST-tagged) that reconciliation grouped
// to build them. Accepting a producer-shaped ID here would let that internal identity leak
// into exploitation evidence and, from there, the published report.
if (!isTaskReference(typed.vulnerability_id, vulnClass)) {
return errorResult(
`Vulnerability ID "${typed.vulnerability_id}" is outside the ${REF_PREFIX[vulnClass]}-NN task namespace ` +
`(for example ${REF_PREFIX[vulnClass]}-01).`,
'ValidationError',
true,
);
}
// Reject IDs not in this run's queue (typo'd or hallucinated).
if (!validIds.has(typed.vulnerability_id)) {
return errorResult(
+5 -4
View File
@@ -8,7 +8,7 @@
* Deterministic exploit collector → markdown renderer.
*
* Single entry point renderExploitDeliverable(vulnClass, state, idToType)
* covers all 5 exploitation agents (injection, xss, auth, ssrf, authz). The
* covers all exploitation agents. The
* per-class deltas are limited to title and ID prefix; every section, label,
* and sort rule is class-agnostic. Section headers and bolded field labels
* mirror the prescribed-Markdown skeleton from the existing exploit-*.txt
@@ -30,18 +30,19 @@
* as `- {ID} ({vulnerability_type})`. Omitted when every queue ID was emitted.
*/
import type { AddExploitInput, VulnClass } from '../collectors/exploit-collector.js';
import type { AddExploitInput, ExploitClass } from '../collectors/exploit-collector.js';
// ============================================================================
// PER-CLASS CONSTANTS
// ============================================================================
const TITLES: Record<VulnClass, string> = {
const TITLES: Record<ExploitClass, string> = {
injection: 'Injection Exploitation Evidence',
xss: 'Cross-Site Scripting (XSS) Exploitation Evidence',
auth: 'Authentication Exploitation Evidence',
ssrf: 'SSRF Exploitation Evidence',
authz: 'Authorization Exploitation Evidence',
miscellaneous: 'Miscellaneous Exploitation Evidence',
};
// ============================================================================
@@ -203,7 +204,7 @@ function renderUnprocessedSection(missingIds: readonly string[], idToType: Reado
// ============================================================================
export function renderExploitDeliverable(
vulnClass: VulnClass,
vulnClass: ExploitClass,
state: readonly AddExploitInput[],
idToType: ReadonlyMap<string, string>,
): string {
+292 -7
View File
@@ -5,6 +5,7 @@
// as published by the Free Software Foundation.
import { AsyncLocalStorage } from 'node:async_hooks';
import { createHash } from 'node:crypto';
import { $ } from 'zx';
import type { ActivityLogger } from '../types/activity-logger.js';
import { ErrorCode } from '../types/errors.js';
@@ -233,9 +234,50 @@ export async function executeGitCommandWithRetry(
);
}
// Two-phase reset: hard reset (tracked files) + clean (untracked files).
// When paths is provided, the untracked clean is scoped to those paths so a
// failing agent's rollback can't delete a concurrent sibling agent's scratch.
// Filter paths to those present in the HEAD tree, so a subsequent
// `git restore --source=HEAD` won't abort on a pathspec the commit doesn't
// contain. Sourced from HEAD (not the index) to match what restore reads.
async function listPathsInHead(sourceDir: string, paths: readonly string[]): Promise<string[]> {
const result = await executeGitCommandWithRetry(
['git', 'ls-tree', '-r', '-z', '--name-only', 'HEAD', '--', ...paths],
sourceDir,
'list HEAD-tracked rollback paths',
);
return result.stdout.split('\0').filter((path) => path.length > 0);
}
async function restoreScopedPathsFromHead(
sourceDir: string,
paths: readonly string[],
description: string,
): Promise<void> {
const pathsInHead = await listPathsInHead(sourceDir, paths);
// Resetting the scoped index first also makes a newly staged path untracked,
// allowing the clean step to remove it when the path is absent from HEAD.
await executeGitCommandWithRetry(
['git', 'reset', 'HEAD', '--', ...paths],
sourceDir,
`resetting owned index paths for ${description}`,
);
if (pathsInHead.length > 0) {
await executeGitCommandWithRetry(
['git', 'restore', '--source=HEAD', '--worktree', '--', ...pathsInHead],
sourceDir,
`restoring owned worktree paths for ${description}`,
);
}
await executeGitCommandWithRetry(
['git', 'clean', '-fd', '--', ...paths],
sourceDir,
`cleaning untracked owned paths for ${description}`,
);
}
// Two-phase rollback to the last checkpoint: restore tracked files, then clean
// untracked files. When paths is provided, both phases are scoped to those
// paths so one agent's rollback cannot discard a concurrent sibling's work.
// Without paths, the existing whole-workspace reset remains available.
export async function rollbackGitWorkspace(
sourceDir: string,
reason: string = 'retry preparation',
@@ -250,11 +292,15 @@ export async function rollbackGitWorkspace(
logger.info(`Rolling back workspace for ${reason}`);
try {
const scoped = paths !== undefined && paths.length > 0;
const changes = await withGitRepoLock(async () => {
const pendingChanges = await getChangedFiles(sourceDir, 'status check for rollback');
await executeGitCommandWithRetry(['git', 'reset', '--hard', 'HEAD'], sourceDir, 'hard reset for rollback');
const cleanArgs = paths && paths.length > 0 ? ['git', 'clean', '-fd', '--', ...paths] : ['git', 'clean', '-fd'];
await executeGitCommandWithRetry(cleanArgs, sourceDir, 'cleaning untracked files for rollback');
const pendingChanges = await getChangedFiles(sourceDir, 'status check for rollback', paths);
if (scoped) {
await restoreScopedPathsFromHead(sourceDir, paths, 'rollback');
} else {
await executeGitCommandWithRetry(['git', 'reset', '--hard', 'HEAD'], sourceDir, 'hard reset for rollback');
await executeGitCommandWithRetry(['git', 'clean', '-fd'], sourceDir, 'cleaning untracked files for rollback');
}
return pendingChanges;
});
@@ -383,6 +429,95 @@ export async function commitGitSuccess(
}
}
/**
* Return the repo-relative paths changed by one commit.
*
* The result is NUL-delimited at the Git boundary so unusual path characters
* are not split or unquoted.
*/
export async function pathsChangedInCommit(sourceDir: string, commitHash: string): Promise<string[]> {
const result = await executeGitCommandWithRetry(
['git', 'diff-tree', '--root', '--no-commit-id', '--name-only', '-r', '-z', commitHash],
sourceDir,
'listing commit changed paths',
);
return result.stdout.split('\0').filter((path) => path.length > 0);
}
function samePathSet(first: readonly string[], second: readonly string[]): boolean {
return (
first.length === second.length &&
new Set(first).size === first.length &&
first.every((path) => second.includes(path))
);
}
async function stagedPaths(sourceDir: string, paths: readonly string[]): Promise<string[]> {
const result = await executeGitCommandWithRetry(
['git', 'diff', '--cached', '--name-only', '-z', '--', ...paths],
sourceDir,
'verifying exact staged paths',
);
return result.stdout.split('\0').filter((path) => path.length > 0);
}
/** Raised before commit when the staged delta differs from the caller's exact contract. */
export class ExactPathCommitMismatchError extends Error {
constructor() {
super('The staged Git path set differs from the exact publication contract');
this.name = 'ExactPathCommitMismatchError';
}
}
/**
* Commit only the supplied pathspecs, leaving unrelated staged and dirty paths untouched.
*
* There is no empty-path or empty-commit mode: either would weaken the exact-path
* contract. The caller owns cleanup after any failed write or commit.
*/
export async function commitExactPaths(
sourceDir: string,
paths: readonly string[],
description: string,
logger: ActivityLogger,
expectedChangedPaths?: readonly string[],
): Promise<{ commitHash: string; changedPaths: string[] }> {
if (paths.length === 0) {
throw new Error('commitExactPaths: refusing an empty pathspec set');
}
if (
expectedChangedPaths !== undefined &&
(new Set(expectedChangedPaths).size !== expectedChangedPaths.length ||
expectedChangedPaths.some((expectedPath) => !paths.includes(expectedPath)))
) {
throw new ExactPathCommitMismatchError();
}
return withGitRepoLock(async () => {
await executeGitCommandWithRetry(['git', 'add', '-A', '--', ...paths], sourceDir, 'staging exact paths');
if (expectedChangedPaths !== undefined) {
const prospectivePaths = await stagedPaths(sourceDir, paths);
if (!samePathSet(prospectivePaths, expectedChangedPaths)) {
throw new ExactPathCommitMismatchError();
}
}
await executeGitCommandWithRetry(
['git', 'commit', '-m', description, '--', ...paths],
sourceDir,
'creating path-limited commit',
);
const commitHash = await getGitCommitHash(sourceDir);
if (commitHash === null) {
throw new Error('commitExactPaths: HEAD is unreadable after commit');
}
const changedPaths = await pathsChangedInCommit(sourceDir, commitHash);
logger.info(`Path-limited commit ${commitHash.slice(0, 8)} changed ${changedPaths.length} path(s)`);
return { commitHash, changedPaths };
});
}
/**
* Get current git commit hash.
* Returns null if not a git repository.
@@ -398,3 +533,153 @@ export async function getGitCommitHash(sourceDir: string): Promise<string | null
return null;
}
}
/** Return whether one commit is an ancestor of or equal to another. */
export async function isAncestor(ancestor: string, descendant: string, sourceDir: string): Promise<boolean> {
return withGitRepoLock(async () => {
const result = await $`cd ${sourceDir} && git merge-base --is-ancestor ${ancestor} ${descendant}`.nothrow().quiet();
return result.exitCode === 0;
});
}
/** Read a file from `HEAD`, returning null only when the Git command cannot supply it. */
export async function readFileFromHead(sourceDir: string, relPath: string): Promise<string | null> {
return withGitRepoLock(async () => {
const result = await $`cd ${sourceDir} && git show ${`HEAD:${relPath}`}`.nothrow().quiet();
return result.exitCode === 0 ? result.stdout : null;
});
}
/**
* Classify a failed committed read.
*
* Transient markers are checked before corruption markers because Git can emit
* `bad object` after an earlier permission or I/O error. Unknown failures remain
* transient so callers retry instead of incorrectly treating them as absent.
*/
export function classifyHeadReadFailure(stderr: string): 'absent' | 'corrupt' | 'transient' {
const text = stderr.toLowerCase();
if (/does not exist in|exists on disk, but not in/.test(text)) {
return 'absent';
}
if (
/permission denied|resource temporarily unavailable|operation timed out|input\/output error|too many open files|no space left|unable to open|interrupted system call/.test(
text,
)
) {
return 'transient';
}
if (
/bad object|is corrupt|object file .* is empty|unable to unpack|inflate|did not match|hash mismatch|sha1 mismatch/.test(
text,
)
) {
return 'corrupt';
}
return 'transient';
}
/** The classifiable outcomes of reading one committed file from `HEAD`. */
export type CommittedReadResult =
| { readonly state: 'present'; readonly contents: string }
| { readonly state: 'absent' }
| { readonly state: 'corrupt' };
/** The classifiable outcomes of reading one committed blob identity from `HEAD`. */
export type CommittedBlobResult =
| { readonly state: 'present'; readonly sha: string }
| { readonly state: 'absent' }
| { readonly state: 'corrupt' };
function transientHeadReadError(operation: string): PentestError {
return new PentestError(
'A committed Git object could not be read because of a transient repository error',
'filesystem',
true,
{ operation },
ErrorCode.GIT_CHECKPOINT_FAILED,
);
}
/**
* Read a committed file while preserving absent, corrupt, and transient outcomes.
* Transient reads throw so the activity retry policy remains authoritative.
*/
export async function readCommittedFile(sourceDir: string, relPath: string): Promise<CommittedReadResult> {
return withGitRepoLock(async () => {
const result = await $`cd ${sourceDir} && git show ${`HEAD:${relPath}`}`.nothrow().quiet();
if (result.exitCode === 0) {
return { state: 'present', contents: result.stdout };
}
const failure = classifyHeadReadFailure(result.stderr);
if (failure === 'absent') {
return { state: 'absent' };
}
if (failure === 'corrupt') {
return { state: 'corrupt' };
}
throw transientHeadReadError('read-committed-file');
});
}
/** Read one Git blob identity from `HEAD` without collapsing transient failure into absence. */
export async function blobShaFromHead(sourceDir: string, relPath: string): Promise<CommittedBlobResult> {
return withGitRepoLock(async () => {
const result = await $`cd ${sourceDir} && git rev-parse ${`HEAD:${relPath}`}`.nothrow().quiet();
if (result.exitCode === 0) {
return { state: 'present', sha: result.stdout.trim() };
}
const failure = classifyHeadReadFailure(result.stderr);
if (failure === 'absent') {
return { state: 'absent' };
}
if (failure === 'corrupt') {
return { state: 'corrupt' };
}
throw transientHeadReadError('read-committed-blob-identity');
});
}
/**
* Compute the Git blob id for in-memory bytes without writing them, so a caller can compare
* intended contents against a committed blob id. Uses the repo's own object format (sha1 or
* sha256) so the id matches what this repository would store.
*/
export async function gitBlobShaForContents(sourceDir: string, contents: string): Promise<string> {
return withGitRepoLock(async () => {
const result = await executeGitCommandWithRetry(
['git', 'rev-parse', '--show-object-format'],
sourceDir,
'reading Git object format',
);
const objectFormat = result.stdout.trim();
if (objectFormat !== 'sha1' && objectFormat !== 'sha256') {
throw new Error('Unsupported Git object format');
}
const bytes = Buffer.from(contents, 'utf8');
return createHash(objectFormat).update(`blob ${bytes.length}\0`, 'utf8').update(bytes).digest('hex');
});
}
/** Return the newest reachable commit that changed one exact path. */
export async function lastCommitForPathAtHead(sourceDir: string, relPath: string): Promise<string | null> {
return withGitRepoLock(async () => {
const result = await executeGitCommandWithRetry(
['git', 'log', '-1', '--format=%H', 'HEAD', '--', relPath],
sourceDir,
'reading exact-path publication commit',
);
const commitHash = result.stdout.trim();
return commitHash.length > 0 ? commitHash : null;
});
}
/** Restore only the supplied paths in both the index and working tree from `HEAD`. */
export async function restorePathsFromHead(sourceDir: string, paths: readonly string[]): Promise<void> {
if (paths.length === 0) {
return;
}
await withGitRepoLock(() => restoreScopedPathsFromHead(sourceDir, paths, 'committed-state repair'));
}
+89 -35
View File
@@ -6,8 +6,9 @@
import { fs, path } from 'zx';
import type { ExploitationDecision, VulnType } from '../types/agents.js';
import type { ExploitationDecision } from '../types/agents.js';
import { ErrorCode } from '../types/errors.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
import { err, ok, type Result } from '../types/result.js';
import { asyncPipe } from '../utils/functional.js';
import { PentestError } from './error-handling.js';
@@ -17,14 +18,15 @@ export type { ExploitationDecision, VulnType } from '../types/agents.js';
interface VulnTypeConfigItem {
deliverable: string;
queue: string;
deliverableRequired: boolean;
}
type VulnTypeConfig = Record<VulnType, VulnTypeConfigItem>;
type VulnTypeConfig = Record<ReconciliationClass, VulnTypeConfigItem>;
type ErrorMessageResolver = string | ((existence: FileExistence) => string);
type ErrorMessageResolver = string | ((context: ExistenceContext) => string);
interface ValidationRule {
predicate: (existence: FileExistence) => boolean;
predicate: (context: ExistenceContext) => boolean;
errorMessage: ErrorMessageResolver;
retryable: boolean;
}
@@ -34,8 +36,13 @@ interface FileExistence {
queueExists: boolean;
}
interface ExistenceContext {
existence: FileExistence;
deliverableRequired: boolean;
}
interface PathsBase {
vulnType: VulnType;
vulnType: ReconciliationClass;
deliverable: string;
queue: string;
sourceDir: string;
@@ -64,56 +71,87 @@ interface QueueValidationResult {
error: string | null;
}
function isRetryableQueueFileSystemError(error: unknown): boolean {
if (!(error instanceof Error)) return false;
const code = (error as NodeJS.ErrnoException).code;
return code !== undefined && !['EACCES', 'EINVAL', 'EISDIR', 'ENAMETOOLONG', 'ENOTDIR', 'EPERM'].includes(code);
}
/**
* Result type for safe validation - explicit error handling.
*/
export type SafeValidationResult = Result<ExploitationDecision, PentestError>;
export type ReconciliationExploitationDecision<T extends ReconciliationClass = ReconciliationClass> = Omit<
ExploitationDecision,
'vulnType'
> & {
vulnType: T;
};
export type SafeValidationResult<T extends ReconciliationClass = ReconciliationClass> = Result<
ReconciliationExploitationDecision<T>,
PentestError
>;
// Vulnerability type configuration as immutable data
const VULN_TYPE_CONFIG: VulnTypeConfig = Object.freeze({
injection: Object.freeze({
deliverable: 'injection_analysis_deliverable.md',
queue: 'injection_exploitation_queue.json',
deliverableRequired: true,
}),
xss: Object.freeze({
deliverable: 'xss_analysis_deliverable.md',
queue: 'xss_exploitation_queue.json',
deliverableRequired: true,
}),
auth: Object.freeze({
deliverable: 'auth_analysis_deliverable.md',
queue: 'auth_exploitation_queue.json',
deliverableRequired: true,
}),
ssrf: Object.freeze({
deliverable: 'ssrf_analysis_deliverable.md',
queue: 'ssrf_exploitation_queue.json',
deliverableRequired: true,
}),
authz: Object.freeze({
deliverable: 'authz_analysis_deliverable.md',
queue: 'authz_exploitation_queue.json',
deliverableRequired: true,
}),
miscellaneous: Object.freeze({
deliverable: 'miscellaneous_analysis_deliverable.md',
queue: 'miscellaneous_exploitation_queue.json',
deliverableRequired: false,
}),
}) as VulnTypeConfig;
// Pure function to create validation rule
function createValidationRule(
predicate: (existence: FileExistence) => boolean,
predicate: (context: ExistenceContext) => boolean,
errorMessage: ErrorMessageResolver,
retryable: boolean = true,
): ValidationRule {
return Object.freeze({ predicate, errorMessage, retryable });
}
// Symmetric deliverable rules: queue and deliverable must exist together (prevents partial analysis from triggering exploitation)
// A queue is always required. Analysis-backed classes also require their analysis deliverable;
// the analysis-less `miscellaneous` class deliberately has no such artifact.
const fileExistenceRules: readonly ValidationRule[] = Object.freeze([
createValidationRule(
({ deliverableExists, queueExists }) => deliverableExists && queueExists,
({ existence, deliverableRequired }) =>
existence.queueExists && (!deliverableRequired || existence.deliverableExists),
getExistenceErrorMessage,
),
]);
// Generate appropriate error message based on which files are missing
function getExistenceErrorMessage(existence: FileExistence): string {
function getExistenceErrorMessage({ existence, deliverableRequired }: ExistenceContext): string {
const { deliverableExists, queueExists } = existence;
if (!deliverableRequired) {
return 'Analysis failed: Queue file missing. A queue is required.';
}
if (!deliverableExists && !queueExists) {
return 'Analysis failed: Neither deliverable nor queue file exists. Both are required.';
}
@@ -124,7 +162,7 @@ function getExistenceErrorMessage(existence: FileExistence): string {
}
// Pure function to create file paths
const createPaths = (vulnType: VulnType, sourceDir: string): PathsBase | PathsWithError => {
const createPaths = (vulnType: ReconciliationClass, sourceDir: string): PathsBase | PathsWithError => {
const config = VULN_TYPE_CONFIG[vulnType];
if (!config) {
return {
@@ -144,10 +182,23 @@ const createPaths = (vulnType: VulnType, sourceDir: string): PathsBase | PathsWi
const checkFileExistence = async (paths: PathsBase | PathsWithError): Promise<PathsWithExistence | PathsWithError> => {
if ('error' in paths) return paths;
const [deliverableExists, queueExists] = await Promise.all([
fs.pathExists(paths.deliverable),
fs.pathExists(paths.queue),
]);
let deliverableExists: boolean;
let queueExists: boolean;
try {
[deliverableExists, queueExists] = await Promise.all([
fs.pathExists(paths.deliverable),
fs.pathExists(paths.queue),
]);
} catch (error) {
return {
error: new PentestError(
'Queue validation could not inspect the required files.',
'filesystem',
isRetryableQueueFileSystemError(error),
{ vulnType: paths.vulnType },
),
};
}
return Object.freeze({
...paths,
@@ -162,13 +213,15 @@ const validateExistenceRules = (
if ('error' in pathsWithExistence) return pathsWithExistence;
const { existence, vulnType } = pathsWithExistence;
const { deliverableRequired } = VULN_TYPE_CONFIG[vulnType];
const context: ExistenceContext = { existence, deliverableRequired };
// Find the first rule that fails
const failedRule = fileExistenceRules.find((rule) => !rule.predicate(existence));
const failedRule = fileExistenceRules.find((rule) => !rule.predicate(context));
if (failedRule) {
const message =
typeof failedRule.errorMessage === 'function' ? failedRule.errorMessage(existence) : failedRule.errorMessage;
typeof failedRule.errorMessage === 'function' ? failedRule.errorMessage(context) : failedRule.errorMessage;
return {
error: new PentestError(
@@ -177,8 +230,6 @@ const validateExistenceRules = (
failedRule.retryable,
{
vulnType,
deliverablePath: pathsWithExistence.deliverable,
queuePath: pathsWithExistence.queue,
existence,
},
ErrorCode.DELIVERABLE_NOT_FOUND,
@@ -204,11 +255,11 @@ const validateQueueStructure = (content: string): QueueValidationResult => {
data: isValid ? (parsed as QueueData) : null,
error: null,
});
} catch (parseError) {
} catch {
return Object.freeze({
valid: false,
data: null,
error: parseError instanceof Error ? parseError.message : String(parseError),
error: 'invalid_json',
});
}
};
@@ -234,9 +285,6 @@ const validateQueueContent = async (
true, // retryable
{
vulnType: pathsWithExistence.vulnType,
queuePath: pathsWithExistence.queue,
originalError: queueValidation.error,
queueStructure: queueValidation.data ? Object.keys(queueValidation.data) : [],
},
),
};
@@ -249,13 +297,11 @@ const validateQueueContent = async (
} catch (readError) {
return {
error: new PentestError(
`Failed to read queue file for ${pathsWithExistence.vulnType}: ${readError instanceof Error ? readError.message : String(readError)}`,
`Queue file for ${pathsWithExistence.vulnType} could not be read.`,
'filesystem',
false,
isRetryableQueueFileSystemError(readError),
{
vulnType: pathsWithExistence.vulnType,
queuePath: pathsWithExistence.queue,
originalError: readError instanceof Error ? readError.message : String(readError),
},
),
};
@@ -263,7 +309,9 @@ const validateQueueContent = async (
};
// Final decision: skip if queue says no vulns, proceed if vulns found, error otherwise
const determineExploitationDecision = (validatedData: PathsWithQueue | PathsWithError): ExploitationDecision => {
const determineExploitationDecision = (
validatedData: PathsWithQueue | PathsWithError,
): ReconciliationExploitationDecision => {
if ('error' in validatedData) {
throw validatedData.error;
}
@@ -281,11 +329,11 @@ const determineExploitationDecision = (validatedData: PathsWithQueue | PathsWith
};
// Main functional validation pipeline
export async function validateQueueAndDeliverable(
vulnType: VulnType,
export async function validateQueueAndDeliverable<T extends ReconciliationClass>(
vulnType: T,
sourceDir: string,
): Promise<ExploitationDecision> {
return asyncPipe<ExploitationDecision>(
): Promise<ReconciliationExploitationDecision<T>> {
return asyncPipe<ReconciliationExploitationDecision<T>>(
createPaths(vulnType, sourceDir),
checkFileExistence,
validateExistenceRules,
@@ -298,11 +346,17 @@ export async function validateQueueAndDeliverable(
* Safely validate queue and deliverable files.
* Returns Result<ExploitationDecision, PentestError> for explicit error handling.
*/
export async function validateQueueSafe(vulnType: VulnType, sourceDir: string): Promise<SafeValidationResult> {
export async function validateQueueSafe<T extends ReconciliationClass>(
vulnType: T,
sourceDir: string,
): Promise<SafeValidationResult<T>> {
try {
const result = await validateQueueAndDeliverable(vulnType, sourceDir);
return ok(result);
} catch (error) {
return err(error as PentestError);
if (error instanceof PentestError) return err(error);
return err(
new PentestError('Queue validation failed closed on an internal invariant.', 'unknown', false, { vulnType }),
);
}
}
@@ -0,0 +1,542 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Temporal boundary for the standalone reconciliation stages. */
import { ApplicationFailure, CancelledFailure, Context, heartbeat } from '@temporalio/activity';
import { type ModelHost, modelHost } from '../ai/model-host.js';
import { createPiStructuredGenerationPort } from '../ai/pi/structured-generation.js';
import {
createTaskFormationExecutor,
TaskFormationExecutorError,
type TaskFormationFallbackReason,
} from '../ai/pi/task-formation-executor.js';
import { ReconciliationError } from '../ai/reconciliation/artifact-store.js';
import {
createEnrichClassSastObservations,
type EnrichClassSastObservationsInput,
SastEnrichmentModelError,
} from '../ai/reconciliation/enrich.js';
import {
createFormClassExploitTasks,
type FormClassExploitTasksInput,
TaskFormationModelError,
} from '../ai/reconciliation/form.js';
import {
type MaterializeClassExploitTasksArgs,
materializeClassExploitTasks as materializeClassExploitTasksStage,
} from '../ai/reconciliation/materialize.js';
import {
type PrepareClassReconciliationArgs,
prepareClassReconciliation as prepareClassReconciliationStage,
} from '../ai/reconciliation/prepare.js';
import {
type PublishClassReconciliationOssArgs,
publicationContractForClass,
publishClassReconciliationOss as publishClassReconciliationOssStage,
} from '../ai/reconciliation/publish.js';
import {
type SeedEmptyProducerQueueArgs,
seedEmptyProducerQueue as seedEmptyProducerQueueStage,
} from '../ai/reconciliation/seed-miscellaneous.js';
import type {
EnrichSuccess,
FormSuccess,
MaterializeResult,
PrepareResult,
} from '../ai/reconciliation/stage-contracts.js';
import type { ActivityLogger } from '../types/activity-logger.js';
import { createActivityLogger } from './activity-logger.js';
import {
type EnrichClassSastObservationsActivityInput,
type FormClassExploitTasksActivityInput,
type FormClassExploitTasksActivityResult,
type MaterializeClassExploitTasksActivityInput,
type PrepareClassReconciliationActivityInput,
type PublishClassReconciliationActivityInput,
RECONCILIATION_ACTIVITY_NAMES,
RECONCILIATION_ACTIVITY_PROFILES,
RECONCILIATION_STABLE_FAILURE_TYPES,
type ReconciliationActivityName,
type ReconciliationActivityRegistry,
type ReconciliationClassActivityName,
type ReconciliationStableFailureType,
resolveReconciliationActivityBudget,
type SeedEmptyProducerQueueActivityInput,
} from './reconcile-activity-types.js';
const STABLE_FAILURE_TYPES: ReadonlySet<string> = new Set(RECONCILIATION_STABLE_FAILURE_TYPES);
const DEFAULT_RETRYABILITY: Readonly<Record<ReconciliationStableFailureType, boolean>> = Object.freeze({
TaskFormationModelError: true,
SastEnrichmentModelError: true,
ReconciliationArtifactNotFound: true,
ReconciliationIoError: true,
ConfigurationError: false,
SastEnrichmentInputError: false,
ArtifactIntegrityError: false,
PublicationConflict: false,
UnmappableSurvivor: false,
KeySetDivergence: false,
});
const SAFE_FAILURE_MESSAGES: Readonly<Record<ReconciliationStableFailureType, string>> = Object.freeze({
TaskFormationModelError: 'Task formation did not produce an accepted result.',
SastEnrichmentModelError: 'SAST enrichment did not produce an accepted result.',
ReconciliationArtifactNotFound: 'A reconciliation artifact is not currently visible.',
ReconciliationIoError: 'A reconciliation filesystem or Git operation failed.',
ConfigurationError: 'Reconciliation activity configuration is invalid.',
SastEnrichmentInputError: 'The supplied SAST reference is invalid.',
ArtifactIntegrityError: 'Reconciliation artifact integrity validation failed.',
PublicationConflict: 'The durable class publication conflicts with committed state.',
UnmappableSurvivor: 'A report-facing survivor cannot be mapped to the class task set.',
KeySetDivergence: 'Reconciliation report-facing key sets disagree.',
});
interface ReconciliationHeartbeatDetails {
readonly stage: ReconciliationActivityName;
readonly attempt: number;
readonly elapsedSeconds: number;
readonly classDeadlineMs: number;
}
export interface ReconciliationActivityRuntime {
readonly attempt: number;
readonly cancellationSignal: AbortSignal;
readonly logger: ActivityLogger;
heartbeat(details: ReconciliationHeartbeatDetails): void;
}
interface ReconciliationStageRuntime {
readonly signal: AbortSignal;
readonly logger: ActivityLogger;
readonly modelHost: ModelHost;
}
export interface ReconciliationStageBindings {
seedEmptyProducerQueue(args: SeedEmptyProducerQueueArgs): Promise<{
alreadySeeded: boolean;
alreadyPublished: boolean;
commitHash: string;
}>;
prepareClassReconciliation(args: PrepareClassReconciliationArgs): Promise<PrepareResult>;
enrichClassSastObservations(
input: EnrichClassSastObservationsInput,
runtime: ReconciliationStageRuntime,
): Promise<EnrichSuccess>;
formClassExploitTasks(input: FormClassExploitTasksInput, runtime: ReconciliationStageRuntime): Promise<FormSuccess>;
materializeClassExploitTasks(args: MaterializeClassExploitTasksArgs): Promise<MaterializeResult>;
publishClassReconciliationOss(args: PublishClassReconciliationOssArgs): Promise<{
alreadyPublished: boolean;
manifestSha256: string;
commitHash: string;
}>;
}
export interface ReconciliationActivityBindings {
/** Worker-local paths are bound here and never enter Temporal activity arguments. */
readonly repositoryPath: string;
readonly deliverablesDir: string;
readonly workspacesDir: string;
readonly webUrl?: string;
readonly modelHost?: ModelHost;
readonly now?: () => number;
readonly runtime?: () => ReconciliationActivityRuntime;
readonly stages?: Partial<ReconciliationStageBindings>;
}
interface FailureMetrics {
readonly costUsd: number;
readonly modelCalls: number;
readonly inputTokens: number;
readonly outputTokens: number;
}
interface StableFailureDetails {
readonly metrics?: FailureMetrics;
readonly fallbackReason?: TaskFormationFallbackReason;
}
function defaultRuntime(): ReconciliationActivityRuntime {
const context = Context.current();
return {
attempt: context.info.attempt,
cancellationSignal: context.cancellationSignal,
logger: createActivityLogger(),
heartbeat,
};
}
function isStableFailureType(value: string): value is ReconciliationStableFailureType {
return STABLE_FAILURE_TYPES.has(value);
}
function failureMetrics(error: TaskFormationModelError | SastEnrichmentModelError): FailureMetrics {
return {
costUsd: error.metrics.costUsd,
modelCalls: error.metrics.modelCalls,
inputTokens: error.metrics.inputTokens,
outputTokens: error.metrics.outputTokens,
};
}
/** Build the one ApplicationFailure shape every reconciliation stage failure normalizes into. */
function applicationFailure(
type: ReconciliationStableFailureType,
retryable: boolean,
stage: ReconciliationActivityName,
details: StableFailureDetails = {},
): ApplicationFailure {
return ApplicationFailure.create({
message: SAFE_FAILURE_MESSAGES[type],
type,
nonRetryable: !retryable,
details: [
{
stage,
...(details.metrics !== undefined && { metrics: details.metrics }),
...(details.fallbackReason !== undefined && { fallbackReason: details.fallbackReason }),
},
],
});
}
function cancellationFrom(error: unknown, signal: AbortSignal): CancelledFailure | undefined {
if (error instanceof CancelledFailure) return error;
const errorName = error instanceof Error ? error.name : undefined;
const cancelledByName = errorName === 'CancelledFailure' || errorName === 'AbortError';
if (!signal.aborted && !cancelledByName) return undefined;
const reason = signal.reason;
if (reason instanceof CancelledFailure) return reason;
return new CancelledFailure('Reconciliation activity cancelled');
}
/**
* Map every shape a reconciliation stage can throw (a model-call error, a wrapped executor
* error, an artifact-store error, an already-classified ApplicationFailure, or an unrecognized
* error) onto the closed set of stable failure types. Cancellation is checked first and
* always wins, since a stage aborted for cancellation is not a stage that failed.
*/
function normalizeFailure(error: unknown, stage: ReconciliationActivityName, signal: AbortSignal): never {
const cancellation = cancellationFrom(error, signal);
if (cancellation !== undefined) throw cancellation;
if (error instanceof TaskFormationModelError) {
throw applicationFailure('TaskFormationModelError', error.retryable, stage, {
metrics: failureMetrics(error),
...(error.fallbackReason !== undefined && { fallbackReason: error.fallbackReason }),
});
}
if (error instanceof SastEnrichmentModelError) {
throw applicationFailure('SastEnrichmentModelError', error.retryable, stage, {
metrics: failureMetrics(error),
});
}
if (error instanceof ReconciliationError) {
throw applicationFailure(error.failureType, error.retryable, stage);
}
if (error instanceof TaskFormationExecutorError) {
if (error.failureKind === 'model') {
throw applicationFailure('TaskFormationModelError', error.retryable, stage, {
metrics: {
costUsd: error.usage.costUsd,
modelCalls: error.modelCalls,
inputTokens: error.usage.inputTokens,
outputTokens: error.usage.outputTokens,
},
...(error.fallbackReason !== undefined && { fallbackReason: error.fallbackReason }),
});
}
const type = error.failureKind === 'confinement' ? 'ArtifactIntegrityError' : 'ConfigurationError';
throw applicationFailure(type, error.retryable, stage);
}
if (error instanceof ApplicationFailure) {
const errorType = error.type;
if (typeof errorType === 'string' && isStableFailureType(errorType)) {
throw applicationFailure(errorType, !error.nonRetryable, stage);
}
throw applicationFailure('ReconciliationIoError', true, stage);
}
if (error instanceof Error && isStableFailureType(error.name)) {
const retryable =
'retryable' in error && typeof error.retryable === 'boolean' ? error.retryable : DEFAULT_RETRYABILITY[error.name];
throw applicationFailure(error.name, retryable, stage);
}
// Unknown failures remain retryable. A generic error name is not evidence that the fault is terminal.
throw applicationFailure('ReconciliationIoError', true, stage);
}
/** Refuse to schedule a class's remaining reconciliation stages once its 12-hour budget is spent. */
function assertActivityCanRun(
activityName: ReconciliationClassActivityName,
classDeadlineMs: number,
nowMs: number,
): ReturnType<typeof resolveReconciliationActivityBudget> {
try {
const budget = resolveReconciliationActivityBudget(activityName, classDeadlineMs, nowMs);
if (!budget.shouldSchedule) {
throw applicationFailure('ConfigurationError', false, activityName);
}
return budget;
} catch (error) {
if (error instanceof ApplicationFailure) throw error;
throw applicationFailure('ConfigurationError', false, activityName);
}
}
async function runReconciliationStage<T>(
activityName: ReconciliationClassActivityName,
classDeadlineMs: number,
runtime: ReconciliationActivityRuntime,
now: () => number,
stage: (runtime: ReconciliationStageRuntime) => Promise<T>,
activityModelHost: ModelHost,
): Promise<T> {
const cancellation = cancellationFrom(undefined, runtime.cancellationSignal);
if (cancellation !== undefined) throw cancellation;
const budget = assertActivityCanRun(activityName, classDeadlineMs, now());
const profile = RECONCILIATION_ACTIVITY_PROFILES[activityName];
const startedAt = now();
let heartbeatInterval: ReturnType<typeof setInterval> | undefined;
if (profile.profile === 'model' && budget.heartbeatIntervalMs !== null) {
runtime.heartbeat({ stage: activityName, attempt: runtime.attempt, elapsedSeconds: 0, classDeadlineMs });
heartbeatInterval = setInterval(() => {
runtime.heartbeat({
stage: activityName,
attempt: runtime.attempt,
elapsedSeconds: Math.max(0, Math.floor((now() - startedAt) / 1_000)),
classDeadlineMs,
});
}, budget.heartbeatIntervalMs);
}
try {
return await stage({ signal: runtime.cancellationSignal, logger: runtime.logger, modelHost: activityModelHost });
} catch (error) {
return normalizeFailure(error, activityName, runtime.cancellationSignal);
} finally {
if (heartbeatInterval !== undefined) clearInterval(heartbeatInterval);
}
}
async function runSeedStage<T>(runtime: ReconciliationActivityRuntime, stage: () => Promise<T>): Promise<T> {
const cancellation = cancellationFrom(undefined, runtime.cancellationSignal);
if (cancellation !== undefined) throw cancellation;
try {
return await stage();
} catch (error) {
return normalizeFailure(error, 'seedEmptyProducerQueue', runtime.cancellationSignal);
}
}
function defaultStages(workspacesDir: string): ReconciliationStageBindings {
return {
seedEmptyProducerQueue: seedEmptyProducerQueueStage,
prepareClassReconciliation: prepareClassReconciliationStage,
enrichClassSastObservations: (input, runtime) =>
createEnrichClassSastObservations({
generation: createPiStructuredGenerationPort(runtime.modelHost),
modelContextFor: () => undefined,
workspacesDir,
signalFor: () => runtime.signal,
logger: runtime.logger,
})(input),
formClassExploitTasks: (input, runtime) =>
createFormClassExploitTasks({
executor: createTaskFormationExecutor(runtime.modelHost),
workspacesDir,
signalFor: () => runtime.signal,
logger: runtime.logger,
})(input),
materializeClassExploitTasks: materializeClassExploitTasksStage,
publishClassReconciliationOss: publishClassReconciliationOssStage,
};
}
function bindStages(
workspacesDir: string,
overrides: Partial<ReconciliationStageBindings> | undefined,
): ReconciliationStageBindings {
return { ...defaultStages(workspacesDir), ...overrides };
}
function assertRegistryNames(registry: ReconciliationActivityRegistry): void {
const actualNames = Object.keys(registry).sort();
const expectedNames = [...RECONCILIATION_ACTIVITY_NAMES].sort();
const expectedNamesAreUnique = new Set(expectedNames).size === expectedNames.length;
if (
!expectedNamesAreUnique ||
actualNames.length !== expectedNames.length ||
actualNames.some((name, index) => name !== expectedNames[index])
) {
throw new Error('Reconciliation activity registry does not match its frozen six-name contract');
}
}
/** Bind worker-local filesystem/model dependencies and return the frozen six-activity registry. */
export function createReconciliationActivityRegistry(
bindings: ReconciliationActivityBindings,
): Readonly<ReconciliationActivityRegistry> {
const now = bindings.now ?? Date.now;
const runtimeFor = bindings.runtime ?? defaultRuntime;
const activityModelHost = bindings.modelHost ?? modelHost;
const stages = bindStages(bindings.workspacesDir, bindings.stages);
async function seedEmptyProducerQueue(input: SeedEmptyProducerQueueActivityInput) {
const runtime = runtimeFor();
const result = await runSeedStage(runtime, () =>
stages.seedEmptyProducerQueue({
deliverablesDir: bindings.deliverablesDir,
sessionId: input.sessionId,
logger: runtime.logger,
}),
);
return {
alreadySeeded: result.alreadySeeded,
alreadyPublished: result.alreadyPublished,
commitHash: result.commitHash,
};
}
async function prepareClassReconciliation(input: PrepareClassReconciliationActivityInput) {
const runtime = runtimeFor();
const result = await runReconciliationStage(
'prepareClassReconciliation',
input.classDeadlineMs,
runtime,
now,
() =>
stages.prepareClassReconciliation({
deliverablesDir: bindings.deliverablesDir,
sessionId: input.sessionId,
vulnerabilityClass: input.vulnerabilityClass,
// The contract fixes which fields the eventual published queue may carry for this
// class. Passing it through unmodified is what keeps internal producer and
// reconciliation identifiers out of the exploitation queue a downstream exploit
// agent reads; widening it here would leak those identifiers into model-facing input.
contract: publicationContractForClass(input.vulnerabilityClass, input.includeSastProvenance),
workspacesDir: bindings.workspacesDir,
}),
activityModelHost,
);
if (result.outcome === 'already_published') {
return { outcome: result.outcome, manifestSha256: result.manifestSha256 };
}
return { outcome: result.outcome, ref: result.ref };
}
async function enrichClassSastObservations(input: EnrichClassSastObservationsActivityInput) {
const runtime = runtimeFor();
const result = await runReconciliationStage(
'enrichClassSastObservations',
input.classDeadlineMs,
runtime,
now,
(stageRuntime) =>
stages.enrichClassSastObservations(
{
sessionId: input.sessionId,
vulnerabilityClass: input.vulnerabilityClass,
...(input.sarif !== undefined && { sarif: input.sarif }),
},
stageRuntime,
),
activityModelHost,
);
return { ref: result.ref, metrics: result.metrics };
}
async function formClassExploitTasks(
input: FormClassExploitTasksActivityInput,
): Promise<FormClassExploitTasksActivityResult> {
const runtime = runtimeFor();
const result = await runReconciliationStage(
'formClassExploitTasks',
input.classDeadlineMs,
runtime,
now,
(stageRuntime) =>
stages.formClassExploitTasks(
{
sessionId: input.sessionId,
vulnerabilityClass: input.vulnerabilityClass,
repositoryPath: bindings.repositoryPath,
producerRef: input.producerRef,
supplementalRef: input.supplementalRef,
...(bindings.webUrl !== undefined && { webUrl: bindings.webUrl }),
},
stageRuntime,
),
activityModelHost,
);
return { ref: result.ref, metrics: result.metrics };
}
async function materializeClassExploitTasks(input: MaterializeClassExploitTasksActivityInput) {
const runtime = runtimeFor();
const result = await runReconciliationStage(
'materializeClassExploitTasks',
input.classDeadlineMs,
runtime,
now,
() =>
stages.materializeClassExploitTasks({
sessionId: input.sessionId,
workspacesDir: bindings.workspacesDir,
vulnerabilityClass: input.vulnerabilityClass,
producerRef: input.producerRef,
supplementalRef: input.supplementalRef,
form: input.form,
}),
activityModelHost,
);
return { ref: result.ref };
}
async function publishClassReconciliationOss(input: PublishClassReconciliationActivityInput) {
const runtime = runtimeFor();
const result = await runReconciliationStage(
'publishClassReconciliationOss',
input.classDeadlineMs,
runtime,
now,
() =>
stages.publishClassReconciliationOss({
deliverablesDir: bindings.deliverablesDir,
sessionId: input.sessionId,
workspacesDir: bindings.workspacesDir,
vulnerabilityClass: input.vulnerabilityClass,
producerRef: input.producerRef,
supplementalRef: input.supplementalRef,
fixedTasksRef: input.fixedTasksRef,
logger: runtime.logger,
}),
activityModelHost,
);
return {
alreadyPublished: result.alreadyPublished,
manifestSha256: result.manifestSha256,
commitHash: result.commitHash,
};
}
const registry = {
seedEmptyProducerQueue,
prepareClassReconciliation,
enrichClassSastObservations,
formClassExploitTasks,
materializeClassExploitTasks,
publishClassReconciliationOss,
} satisfies ReconciliationActivityRegistry;
assertRegistryNames(registry);
return Object.freeze(registry);
}
@@ -0,0 +1,246 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/** Workflow-safe reconciliation activity signatures and scheduling policy. */
import type { ArtifactRef } from '../ai/reconciliation/contracts.js';
import type { StageMetrics } from '../ai/reconciliation/stage-contracts.js';
import type { SarifRef } from '../ai/sast/types.js';
import type { ReconciliationClass } from '../types/reconciliation.js';
const MINUTE_MS = 60 * 1_000;
const HOUR_MS = 60 * MINUTE_MS;
export const RECONCILIATION_CLASS_BUDGET_MS = 12 * HOUR_MS;
export const RECONCILIATION_LATER_STAGE_RESERVE_MS = 5 * MINUTE_MS;
export interface ReconciliationActivityDeadline {
/** Fixed workflow-derived deadline for this class, measured as Unix epoch milliseconds. */
readonly classDeadlineMs: number;
}
export interface ReconciliationActivityBaseInput extends ReconciliationActivityDeadline {
readonly sessionId: string;
readonly vulnerabilityClass: ReconciliationClass;
}
export interface SeedEmptyProducerQueueActivityInput {
readonly sessionId: string;
}
export interface PrepareClassReconciliationActivityInput extends ReconciliationActivityBaseInput {
readonly includeSastProvenance: boolean;
}
export interface EnrichClassSastObservationsActivityInput extends ReconciliationActivityBaseInput {
readonly sarif?: SarifRef;
}
export interface FormClassExploitTasksActivityInput extends ReconciliationActivityBaseInput {
readonly producerRef: ArtifactRef<'producer-observations'>;
readonly supplementalRef: ArtifactRef<'supplemental-observations'>;
}
export interface FormClassExploitTasksActivityResult {
readonly ref: ArtifactRef<'task-formation'>;
readonly metrics: StageMetrics;
}
export type MaterializationFormationResult = FormClassExploitTasksActivityResult | 'singleton_fallback';
export interface MaterializeClassExploitTasksActivityInput extends ReconciliationActivityBaseInput {
readonly producerRef: ArtifactRef<'producer-observations'>;
readonly supplementalRef: ArtifactRef<'supplemental-observations'>;
readonly form: MaterializationFormationResult;
}
export interface PublishClassReconciliationActivityInput extends ReconciliationActivityBaseInput {
readonly producerRef: ArtifactRef<'producer-observations'>;
readonly supplementalRef: ArtifactRef<'supplemental-observations'>;
readonly fixedTasksRef: ArtifactRef<'fixed-tasks'>;
}
export interface SeedEmptyProducerQueueActivityResult {
readonly alreadySeeded: boolean;
readonly alreadyPublished: boolean;
readonly commitHash: string;
}
export type PrepareClassReconciliationActivityResult =
| {
readonly outcome: 'already_published';
readonly manifestSha256: string;
}
| {
readonly outcome: 'pending';
readonly ref: ArtifactRef<'producer-observations'>;
};
export interface EnrichClassSastObservationsActivityResult {
readonly ref: ArtifactRef<'supplemental-observations'>;
readonly metrics: StageMetrics;
}
export interface MaterializeClassExploitTasksActivityResult {
readonly ref: ArtifactRef<'fixed-tasks'>;
}
export interface PublishClassReconciliationActivityResult {
readonly alreadyPublished: boolean;
readonly manifestSha256: string;
readonly commitHash: string;
}
export interface ReconciliationActivityRegistry {
readonly seedEmptyProducerQueue: (
input: SeedEmptyProducerQueueActivityInput,
) => Promise<SeedEmptyProducerQueueActivityResult>;
readonly prepareClassReconciliation: (
input: PrepareClassReconciliationActivityInput,
) => Promise<PrepareClassReconciliationActivityResult>;
readonly enrichClassSastObservations: (
input: EnrichClassSastObservationsActivityInput,
) => Promise<EnrichClassSastObservationsActivityResult>;
readonly formClassExploitTasks: (
input: FormClassExploitTasksActivityInput,
) => Promise<FormClassExploitTasksActivityResult>;
readonly materializeClassExploitTasks: (
input: MaterializeClassExploitTasksActivityInput,
) => Promise<MaterializeClassExploitTasksActivityResult>;
readonly publishClassReconciliationOss: (
input: PublishClassReconciliationActivityInput,
) => Promise<PublishClassReconciliationActivityResult>;
}
export const RECONCILIATION_ACTIVITY_NAMES = Object.freeze([
'seedEmptyProducerQueue',
'prepareClassReconciliation',
'enrichClassSastObservations',
'formClassExploitTasks',
'materializeClassExploitTasks',
'publishClassReconciliationOss',
] as const satisfies readonly (keyof ReconciliationActivityRegistry)[]);
export type ReconciliationActivityName = (typeof RECONCILIATION_ACTIVITY_NAMES)[number];
export type ReconciliationClassActivityName = Exclude<ReconciliationActivityName, 'seedEmptyProducerQueue'>;
export type ReconciliationActivityProfileName = 'deterministic' | 'model';
export interface ReconciliationActivityProfile {
readonly profile: ReconciliationActivityProfileName;
readonly startToCloseTimeoutMs: number;
readonly heartbeatTimeoutMs: number | null;
readonly maximumAttempts: number;
readonly retryInitialIntervalMs: number;
readonly retryBackoffCoefficient: number;
}
function profile(
name: ReconciliationActivityProfileName,
startToCloseTimeoutMs: number,
heartbeatTimeoutMs: number | null,
maximumAttempts: number,
): Readonly<ReconciliationActivityProfile> {
return Object.freeze({
profile: name,
startToCloseTimeoutMs,
heartbeatTimeoutMs,
maximumAttempts,
retryInitialIntervalMs: 1_000,
retryBackoffCoefficient: 2,
});
}
// Deterministic stages (filesystem/git only) get a short timeout, no heartbeat, and more
// attempts, since a transient IO failure is cheap to retry. Model-backed stages get a long
// timeout, a heartbeat so a wedged model call is detected before the full timeout elapses, and
// fewer attempts, since each attempt can itself cost real time and money.
const DETERMINISTIC_PROFILE = profile('deterministic', 2 * MINUTE_MS, null, 5);
const MODEL_PROFILE = profile('model', 30 * MINUTE_MS, 5 * MINUTE_MS, 3);
export const RECONCILIATION_ACTIVITY_PROFILES = Object.freeze({
seedEmptyProducerQueue: DETERMINISTIC_PROFILE,
prepareClassReconciliation: DETERMINISTIC_PROFILE,
enrichClassSastObservations: MODEL_PROFILE,
formClassExploitTasks: MODEL_PROFILE,
materializeClassExploitTasks: DETERMINISTIC_PROFILE,
publishClassReconciliationOss: DETERMINISTIC_PROFILE,
} as const satisfies Readonly<Record<ReconciliationActivityName, Readonly<ReconciliationActivityProfile>>>);
export interface ReconciliationActivityBudget {
/** False once the class's 12-hour deadline leaves no time for this stage; the caller must not schedule it. */
readonly shouldSchedule: boolean;
readonly remainingClassBudgetMs: number;
readonly reservedForLaterStagesMs: number;
readonly scheduleToCloseTimeoutMs: number;
readonly startToCloseTimeoutMs: number;
readonly heartbeatTimeoutMs: number | null;
readonly heartbeatIntervalMs: number | null;
}
function assertEpochMilliseconds(value: number, label: string): void {
if (!Number.isSafeInteger(value) || value < 0) {
throw new Error(`${label} must be non-negative safe-integer epoch milliseconds`);
}
}
/** Derive the fixed per-class deadline immediately before the caller enters prepare. */
export function reconciliationClassDeadlineFrom(startedAtMs: number): number {
assertEpochMilliseconds(startedAtMs, 'Reconciliation class start');
const deadlineMs = startedAtMs + RECONCILIATION_CLASS_BUDGET_MS;
assertEpochMilliseconds(deadlineMs, 'Reconciliation class deadline');
return deadlineMs;
}
/**
* Cap one activity's whole retry window, per-attempt timeout, and heartbeat against the class deadline.
* A zero timeout means the caller must not schedule the activity.
*/
export function resolveReconciliationActivityBudget(
activityName: ReconciliationClassActivityName,
classDeadlineMs: number,
nowMs: number,
): Readonly<ReconciliationActivityBudget> {
assertEpochMilliseconds(classDeadlineMs, 'Reconciliation class deadline');
assertEpochMilliseconds(nowMs, 'Reconciliation budget time');
const profile = RECONCILIATION_ACTIVITY_PROFILES[activityName];
const remainingClassBudgetMs = Math.max(0, classDeadlineMs - nowMs);
const reservedForLaterStagesMs = Math.min(remainingClassBudgetMs, RECONCILIATION_LATER_STAGE_RESERVE_MS);
const scheduleToCloseTimeoutMs = Math.max(0, remainingClassBudgetMs - RECONCILIATION_LATER_STAGE_RESERVE_MS);
const startToCloseTimeoutMs = Math.min(profile.startToCloseTimeoutMs, scheduleToCloseTimeoutMs);
const heartbeatTimeoutMs =
profile.heartbeatTimeoutMs === null || startToCloseTimeoutMs === 0
? null
: Math.min(profile.heartbeatTimeoutMs, startToCloseTimeoutMs);
const heartbeatIntervalMs =
heartbeatTimeoutMs === null ? null : Math.max(1, Math.min(100_000, Math.floor(heartbeatTimeoutMs / 3)));
return Object.freeze({
shouldSchedule: scheduleToCloseTimeoutMs > 0,
remainingClassBudgetMs,
reservedForLaterStagesMs,
scheduleToCloseTimeoutMs,
startToCloseTimeoutMs,
heartbeatTimeoutMs,
heartbeatIntervalMs,
});
}
export const RECONCILIATION_STABLE_FAILURE_TYPES = Object.freeze([
'TaskFormationModelError',
'SastEnrichmentModelError',
'ReconciliationArtifactNotFound',
'ReconciliationIoError',
'ConfigurationError',
'SastEnrichmentInputError',
'ArtifactIntegrityError',
'PublicationConflict',
'UnmappableSurvivor',
'KeySetDivergence',
] as const);
export type ReconciliationStableFailureType = (typeof RECONCILIATION_STABLE_FAILURE_TYPES)[number];
+27
View File
@@ -0,0 +1,27 @@
// Copyright (C) 2026 Keygraph, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License version 3
// as published by the Free Software Foundation.
/**
* Internal class vocabulary for reconciliation and exploitation.
*
* Public configuration continues to use the five-class `VulnClass`. The
* analysis-less `miscellaneous` class exists only after an effective SAST
* reference enters the internal pipeline.
*/
import type { VulnClass } from './config.js';
export type ReconciliationClass = VulnClass | 'miscellaneous';
/** Fixed processing and report-input order for all internal classes. */
export const ALL_RECONCILIATION_CLASSES = [
'injection',
'xss',
'auth',
'authz',
'ssrf',
'miscellaneous',
] as const satisfies readonly ReconciliationClass[];