diff --git a/.dockerignore b/.dockerignore index b3c87c5e..49abd33b 100644 --- a/.dockerignore +++ b/.dockerignore @@ -18,6 +18,7 @@ xben-benchmark-results/ # Development files *.md !CLAUDE.md +!THIRD_PARTY_NOTICES.md .DS_Store Thumbs.db @@ -69,4 +70,5 @@ coverage/ docs/ README.md LICENSE +!LICENSE CHANGELOG.md \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md index ce627364..98baaecb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -60,12 +60,15 @@ npx @keygraph/shannon setup ./shannon start -u -r ./my-repo -w my-audit # Resume (same command) # Monitor -./shannon logs # Show a scan's live log -./shannon status # Live phase/agent progress of one scan, read from Temporal (redraws, then exits) +./shannon scans # List running and completed scans, with each report's path +./shannon logs [] # Show a scan's live log (default: the single running scan, else the most recent) +./shannon logs [] --agent # Tail one agent's own log (from .shannon/agents/) +./shannon logs [] --list-agents # List the agents that have their own log +./shannon status [] # Live phase/agent progress of one scan, read from Temporal (redraws, then exits; same default target) # Dashboard: http://localhost:8233 # Stop -./shannon stop # Stop one scan (confirms first; --yes/-y to skip) +./shannon stop [] # Stop one scan (default: the single running scan; confirms first; --yes/-y to skip) ./shannon stop --all # Stop all running scans (Temporal stays up; confirms first) ./shannon reset # Stop everything and wipe all Temporal data + volumes (type 'confirm' to proceed; cannot be skipped) @@ -98,9 +101,9 @@ apps/worker/ — @shannon/worker (private, Temporal worker + pipeline logic) ### CLI Package (`apps/cli/`) Published as `@keygraph/shannon` on npm. Contains Docker orchestration logic plus a read-only `@temporalio/client` reader (for `status`); no worker/pipeline business logic or prompts. Bundled with tsdown for single-file ESM output (deps stay external). -- `apps/cli/src/index.ts` — CLI dispatcher (`setup`, `start`, `stop`, `reset`, `logs`, `status`, `build`, `version`) -- `apps/cli/src/temporal-client.ts` — `@temporalio/client` reader for `status`: connects to the frontend on `127.0.0.1:7233` (published by compose), `describeScan` (status + `pendingActivities` → running agents), `queryProgress` (live `getProgress` query → `PipelineState`), `getTerminalOutcome` (workflow `result()`). No worker of its own; scans are visible only within Temporal's ~24h retention (namespace default, unset in compose) -- `apps/cli/src/scan/` — `status` rendering: `pipeline.ts` (static phase/agent plan + `run*Agent` activity-type→agent map + mirrored `PipelineState`/`AgentMetrics` types; keep in sync with the worker), `render.ts` (one renderer for both the live query state and the terminal result) +- `apps/cli/src/index.ts` — CLI dispatcher (`setup`, `start`, `stop`, `reset`, `logs`, `status`, `scans`, `build`, `version`) +- `apps/cli/src/temporal-client.ts` — `@temporalio/client` reader for `status`: connects to the frontend on `127.0.0.1:7233` (published by compose), `describeScan` (status + `pendingActivities` → running agents), `queryProgress` (live `getProgress` query → `PipelineState`), `getTerminalOutcome` (workflow `result()`). No worker of its own; scans are visible within Temporal's retention window, which `ensureInfra` (`apps/cli/src/docker.ts`) converges to `168h` (7 days) on every successful `shannon start` — override with `SHANNON_TEMPORAL_RETENTION` (a positive whole-hour value like `72h`) +- `apps/cli/src/scan/` — `status` rendering: `pipeline.ts` (static phase/agent plan + `run*Agent` activity-type→agent map + mirrored `PipelineState`/`AgentMetrics` types; keep in sync with the worker), `derive.ts` (pure phase/agent state derivation shared by the tree and `--json`), `render.ts` (one renderer for both the live query state and the terminal result). The tree shows model work only: every row is an agent, an Agentic SAST stage, or a report step that is currently running or failed. Reconciliation is model work owned by a class, so its wall time renders as a trailing `+ duration` on that class's exploitation row (its analysis row when `exploit: false`) rather than as a row of its own; deterministic bookkeeping stages (`report:*` renumber/assemble/finalize/surface) never appear once they complete. `DerivedPhase.children` (renders sub-rows) and `DerivedPhase.meta` (`duration` vs a `k/N done` tally) are independent — Agentic SAST lists stages under a duration, exploitation lists classes under a tally - `apps/cli/src/mode.ts` — Auto-detection: local mode if `SHANNON_LOCAL=1` env var is set - `apps/cli/src/docker.ts` — Compose lifecycle, image pull/build, ephemeral `docker run` worker spawning - `apps/cli/src/home.ts` — State directory management (`~/.shannon/` for npx, `./` for local) @@ -151,12 +154,20 @@ Durable workflow orchestration with crash recovery, queryable progress, intellig 4. **Exploitation** (5 parallel agents, conditional) — Exploits confirmed vulnerabilities 5. **Reporting** (`report`) — Executive-level security report +Around those phases: + +- Optional agentic static analysis runs before the pentest when `agentic_sast.enabled` is `"true"`, as a child workflow. +- After each class's analysis, reconciliation groups its findings into exploitation tasks. +- Findings outside the five classes form an internal `miscellaneous` class with its own exploitation agent (`miscellaneous-exploit`). +- A scan can finish `completed`, `partial`, `failed`, or `cancelled`; `partial` carries an ordered set of reasons. + ### Supporting Systems -- **Configuration** — YAML configs in `apps/worker/configs/` with JSON Schema validation (`config-schema.json`). Supports auth settings (MFA/TOTP), URL/code rule scoping (`rules.avoid`/`rules.focus`), run-scope steering (`vuln_classes`, `exploit`), free-form `rules_of_engagement`, and post-hoc `report` options (`min_severity`, `min_confidence`, `guidance`, and `sarif` for a SARIF 2.1.0 log via `apps/worker/src/services/sarif-renderer.ts`, on by default for exploit runs and opt out with `report.sarif: false`). `code_path` avoid rules are enforced via the `@gotgenes/pi-permission-system` extension: `apps/worker/src/temporal/activities.ts:syncCodePathDenyRules` writes a global `path` deny config once per workflow (`apps/worker/src/ai/pi/permission-system.ts:syncPermissionSystemConfig`), and the executor loads the extension when that config is present (`apps/worker/src/ai/pi/pi-executor.ts`), so denies fire across every tool and child `task` session. `vuln_classes`/`exploit` scope is locked into `session.json` on first run; resumes with a different scope fail fast (`persistOrValidateRunScope`). Credential resolution — local mode: env vars → `./.env`; npx mode: env vars → `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) +- **Configuration** — YAML configs in `apps/worker/configs/` use the closed JSON Schema in `config-schema.json`. Every fresh scan runs the fixed five analysis classes; there is no public class selector. `agentic_sast.enabled` is the only public agentic-SAST setting. Finding reconciliation runs on every scan and has no public setting of its own. Config also supports authentication (MFA/TOTP), URL/code rule scoping (`rules.avoid`/`rules.focus`), `exploit`, free-form `rules_of_engagement`, and post-hoc `report` options (`min_severity`, `min_confidence`, `guidance`, and exploit-only `sarif` output via `apps/worker/src/services/sarif-renderer.ts`, on by default for exploit runs and opt out with `report.sarif: "false"`). `code_path` avoid rules are enforced via the `@gotgenes/pi-permission-system` extension: `apps/worker/src/temporal/activities.ts:syncCodePathDenyRules` writes a global `path` deny config once per workflow (`apps/worker/src/ai/pi/permission-system.ts:syncPermissionSystemConfig`), and the executor loads the extension when that config is present (`apps/worker/src/ai/pi/pi-executor.ts`), so denies fire across every tool and child `task` session. Credential resolution — local mode: env vars → `./.env`; npx mode: env vars → `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) +- **Agentic SAST progress** — Capella runs as a child workflow, so its activities are absent from the parent's `pendingActivities` and invisible to the CLI. The child signals each stage boundary up via `capellaStageProgress` (`apps/worker/src/temporal/shared.ts`); the parent's handler validates the payload and writes the child-supplied `startedAt` and `durationMs` directly to `operationalStages['agentic-sast:']`, so both the live `getProgress` query and the terminal result carry per-stage rows. Signalling is best-effort and every failure is swallowed — a closed or unreachable parent must never fail a SAST run. `CAPELLA_STAGE_LABELS` in `apps/worker/src/ai/sast/types.ts` is the one label table, shared by the scan log and the status tree; `CAPELLA_PROGRESS_STAGES` omits `export`, which runs no model and so never becomes a row. Scans predating the signal keep the aggregate `agentic-sast` span and render as a bare phase line - **Prompts** — Per-phase templates in `apps/worker/prompts/` with variable substitution (`{{TARGET_URL}}`, `{{CONFIG_CONTEXT}}`). Shared partials in `apps/worker/prompts/shared/` via `apps/worker/src/services/prompt-manager.ts`, including `_code-path-rules.txt` (focus/avoid `[FILE]`/`[GLOB]` routing) and `_rules-of-engagement.txt` (free-text engagement rules). When `exploit: false`, `apps/worker/src/services/findings-renderer.ts` deterministically converts each `*_exploitation_queue.json` into a `*_findings.md` for report assembly — no LLM in the loop - **Agent Harness (pi)** — Uses the **pi harness** (`@earendil-works/pi-coding-agent`, requires Node ≥ 22.19) via `apps/worker/src/ai/pi/pi-executor.ts` (`runPiPrompt` → `createAgentSession`). Retry is split in `apps/worker/src/ai/pi/retry-settings.ts`: pi's agent-level loop is off so Temporal owns agent restarts, while `provider.maxRetries` stays on — pi reads the `provider` block independently of the `enabled` flag — so transport faults are absorbed in-session rather than costing a full agent re-run. `maxRetryDelayMs` is left at pi's 60s default. One model runs every phase, named by `SHANNON_AI_MODEL=:` (default `anthropic:claude-sonnet-4-6`). `apps/worker/src/ai/models.ts` parses the spec — splitting on the **first** colon only, so Bedrock IDs keep theirs — and resolves it through pi's `ModelRuntime`. pi ships the `CredentialStore` interface but no in-memory implementation (its own reads `auth.json` from disk), so `RuntimeCredentialStore` in that file supplies one: credentials arrive as env vars in an ephemeral container and must never touch disk. `createModelRuntime(providerId, apiKey)` builds the runtime; `allowModelNetwork` stays at its default `false` so a scan never blocks on a catalog refresh. `resolveModelSelection()` is **async** because `ModelRuntime.create()` is. Any pi-ai provider id is accepted — `parseModelSpec` no longer rejects against a hardcoded list, so pi's registry is the authority (an unknown provider/model surfaces as a clear "not found in pi registry" error at preflight, which points to the browsable catalogue at `pi.dev/models` — `PI_CATALOG_URL` in `apps/worker/src/ai/models.ts`, appended to the not-found errors and shown in the setup wizard's "Other provider" hint). Four providers are **curated** (`CURATED_PROVIDERS`: `anthropic`, `openai`, `xai`, `amazon-bedrock`) with their own credential variables, config sections, and setup flows; each provider's API key env var is declared once in `PROVIDER_API_KEY_ENV` — Shannon uses each vendor's own variable name (`OPENAI_API_KEY`, `XAI_API_KEY`, …), never an invented one; Bedrock's entry is `AWS_BEARER_TOKEN_BEDROCK`, paired with `AWS_REGION`, which preflight requires separately as provider config rather than a credential. Any other provider uses the **generic** credential path: `SHANNON_AI_API_KEY` (`GENERIC_API_KEY_ENV`) supplies the key for any provider whose credential is a plain API key. Curated providers' own variables take precedence over it, and it also works as a fallback for them — Bedrock is the sole exception (it authenticates through its AWS_ variables, so the generic key never stands in for it). The CLI forwards `SHANNON_AI_API_KEY` in `COMMON_FORWARD_VARS` (it is provider-neutral, binding to whatever `SHANNON_AI_MODEL` names, so the "only one provider configured" guard counts only named credentials), and stores it under a generic `[provider]` config.toml section (`provider.api_key`). `npx @keygraph/shannon setup` exposes this as the "Other provider" option: free-text provider id + model id + key (a curated provider id is rejected there, since it has its own option). `SHANNON_AI_BASE_URL` overrides the endpoint for any provider (proxies/gateways); the credential is unchanged. `pointAtGateway` (`apps/worker/src/ai/models.ts`) applies the one dialect change: behind a base URL, `openai` follows `SHANNON_AI_OPENAI_FORMAT` (`chat-completions` default, or `responses`). On `chat-completions` it switches the API to `openai-completions` and drops the catalogue's Responses-shaped `compat` block so pi's `detectCompat` derives completions settings; on `responses` the descriptor is unchanged but for the endpoint. `resolveGatewayFormat` rejects the variable when the provider is not `openai` or no base URL is set, since it cannot take effect there. All other providers keep their API. The CLI mirrors the accepted values in `apps/cli/src/model-spec.ts`, forwards the variable in `COMMON_FORWARD_VARS`, and maps it to `openai.format` in config.toml. `buildEnvFlags` forwards only the selected provider's credential into the worker container. The CLI mirrors the parse rule and the provider/credential tables in `apps/cli/src/model-spec.ts` (it cannot import from the worker package); the two must stay in sync. pi ships no JSON-schema output or `Task`/`TodoWrite` built-ins, so structured queues are captured via a `submit_exploitation_queue` custom tool (`apps/worker/src/ai/queue-schemas.ts`), and `task` (child sessions scoped to `read`, `grep`, `find`, `ls`, `write`, and `bash` — no nested `task` or collector tools; `CHILD_TOOLS` in `apps/worker/src/ai/pi/task-tool.ts`) + `todo_write` (`apps/worker/src/ai/pi/session-tools.ts`) are provided as custom tools; the per-phase collectors are pi custom tools (TypeBox `defineTool` in `apps/worker/src/collectors/`). Shannon sets no thinking configuration at all — no `thinkingLevel` is passed to any `createAgentSession` call, so pi's own default applies. There is no adaptive-thinking support and no `CLAUDE_ADAPTIVE_THINKING` / `core.adaptive_thinking` setting. Browser automation via `playwright-cli` with session isolation (`-s=`). TOTP generation via `generate-totp` CLI tool. Login flow template at `apps/worker/prompts/shared/login-instructions.txt` supports form, SSO, API, and basic auth. On authenticated whitebox scans, the `validate-authentication` preflight performs the single real login and saves the browser session to `auth-state.json` in the per-session audit directory (path from `authStateFile()` in `apps/worker/src/audit/utils.ts`, derived from `generateAuditPath()`). The validation activity (`apps/worker/src/services/validate-authentication.ts`) removes any stale file from a prior run before the agent runs and verifies the file parses and contains cookies or storage before the preflight is marked complete; `logWorkflowComplete` deletes it when the workflow ends so authenticated cookies don't sit on disk between scans. Agent prompts opt in to session reuse by `@include(shared/_shared-session.txt)` before their `` block — the partial restores the session and falls through to the full login flow if verification fails. `vuln-auth`/`exploit-auth` omit the include and own their own login - **Pi Credential Reuse** — `SHANNON_USE_PI_AUTH=1` opts into reusing the host's Pi login, including an `openai-codex` ChatGPT Plus/Pro subscription selected with `SHANNON_AI_MODEL=openai-codex:`. `apps/cli/src/env.ts` requires `~/.pi/agent/auth.json`; `start.ts` passes its path to `spawnWorker`, which mounts only that file read-write at `/tmp/.pi/agent/auth.json`. The flag itself is not forwarded: the worker detects the file with `piAuthPresent()` and passes its path to `ModelRuntime.create`. CLI and worker API-key presence checks are skipped on this path, but the normal preflight model probe still validates the credential. The image and UID-remapping entrypoint keep `/tmp/.pi/agent` owned by `pentest` so adjacent Pi/Shannon configuration remains writable. Refreshed OAuth state is persisted to the host for subsequent scans. -- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds the human-facing report in both formats (`Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`, `FINAL_REPORT_PDF_FILENAME`/`FINAL_REPORT_MD_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`workspaces`/`logs`) without migration. Resuming a pre-restructure workspace upgrades it in place first: `migrateLegacyWorkspaceLayout` (`apps/cli/src/commands/start.ts`) renames the flat deliverables/logs/session entries into `.shannon/` (carrying the deliverables `.git` along) before the overlay dirs are mounted, so resume finds the old checkpoints instead of re-running every agent. The report agent writes structured findings to `report.json`, from which `report-renderer.ts` renders the assembled markdown and `report-json-adapter.ts` produces the Typst-shaped JSON that `pdf-renderer.ts` compiles into `comprehensive_security_assessment_report.pdf` using the bundled `apps/worker/templates/typst/report.typ` template (the `typst` binary is installed in the worker image). `copyReportToRunRoot` (`apps/worker/src/services/reporting.ts`) surfaces both the PDF and the markdown to the run root as `Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`; the deliverables-dir copies remain as the git-checkpointed sources. PDF compilation is best-effort — a failure is logged and the run still completes. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive +- **Audit System** — Crash-safe append-only logging in `workspaces/{hostname}_{sessionId}/`. The run directory's top level holds the human-facing report in both formats (`Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`, `FINAL_REPORT_PDF_FILENAME`/`FINAL_REPORT_MD_FILENAME` in `apps/worker/src/paths.ts`); everything else — deliverables, per-agent logs, prompts, `session.json`, `workflow.log`, and browser artifacts — is nested under a hidden `.shannon/` internals dir (`INTERNAL_DIR`) so a customer sees only the report. Audit path helpers route through `generateInternalPath` (`apps/worker/src/audit/utils.ts`); the CLI nests the overlay backing dirs under the same `.shannon/` (`apps/cli/src/docker.ts`, `start.ts`). `session.json`/`workflow.log` reads use dual-read resolvers (`resolveSessionJsonPath`, `resolveRunFile`) that prefer `.shannon/` and fall back to the legacy run-root layout, so pre-restructure workspaces stay listable (`scans`/`logs`) without migration. A pre-restructure workspace cannot be resumed: `classifyWorkspaceLaunch` (`apps/cli/src/commands/start.ts`) requires `.shannon/launch.json`, and its absence fails the launch as "created by an earlier version of Shannon" before anything on disk is touched. There is no in-place migration — the workspace's files and report are left untouched, and the operator starts a new scan under a different `-w` name. The report agent writes structured findings to `report.json`, from which `report-renderer.ts` renders the assembled markdown and `report-json-adapter.ts` produces the Typst-shaped JSON that `pdf-renderer.ts` compiles into `comprehensive_security_assessment_report.pdf` using the bundled `apps/worker/templates/typst/report.typ` template (the `typst` binary is installed in the worker image). `copyReportToRunRoot` (`apps/worker/src/services/reporting.ts`) surfaces both the PDF and the markdown to the run root as `Security-Assessment-Report.pdf` and `Security-Assessment-Report.md`; the deliverables-dir copies remain as the git-checkpointed sources. PDF compilation is best-effort — a failure is logged and the run still completes. WorkflowLogger (`apps/worker/src/audit/workflow-logger.ts`) provides unified human-readable per-workflow logs, backed by LogStream (`apps/worker/src/audit/log-stream.ts`) shared stream primitive. Every combined-log line is also projected into a per-agent file under `.shannon/agents/.log` (one per pipeline agent, one per Capella stage; subagents fold into the parent's file, and a stage's concurrent sessions share its file with an inline session label). The projection boundary is `apps/worker/src/audit/actor-projection.ts` (`projectActor` maps a `TraceActor` to its combined prefix and owning file slug — slugs come only from closed fields); fan-out is best-effort and never blocks the canonical combined log. A lifecycle owner holds a `LogStream` lease per agent file (the pipeline agent's `logAgent` span, or a Capella stage activity's `try/finally`) so per-line writes ride the reference count; `CapellaStageTrace.drain()` flushes a stage's trace queue before its activity returns. The CLI tails one file with `shannon logs --agent ` (`--list-agents` to enumerate); the default `shannon logs` path is unchanged - **Deliverables** — Saved to `.shannon/deliverables/` in the target repo via the `save-deliverable` CLI script (`apps/worker/src/scripts/save-deliverable.ts`) - **Workspaces & Resume** — Named workspaces via `-w ` or auto-named from URL+timestamp. Resume detects completed agents via `session.json`. `loadResumeState()` in `apps/worker/src/temporal/activities.ts` validates deliverable existence, restores git checkpoints, and cleans up incomplete deliverables diff --git a/Dockerfile b/Dockerfile index 42b7f7ec..6f289a49 100644 --- a/Dockerfile +++ b/Dockerfile @@ -109,6 +109,10 @@ COPY --from=builder /app/node_modules /app/node_modules COPY --from=builder /app/apps/worker /app/apps/worker COPY --from=builder /app/apps/cli/package.json /app/apps/cli/package.json +# Third-party license and notice material travels with the distributed image +COPY LICENSE THIRD_PARTY_NOTICES.md /usr/share/licenses/shannon/ +COPY LICENSES/ /usr/share/licenses/shannon/LICENSES/ + RUN npm install -g --ignore-scripts @playwright/cli@0.1.1 RUN mkdir -p /tmp/.claude/skills && \ playwright-cli install --skills && \ diff --git a/LICENSES/Apache-2.0.txt b/LICENSES/Apache-2.0.txt new file mode 100644 index 00000000..261eeb9e --- /dev/null +++ b/LICENSES/Apache-2.0.txt @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/LICENSES/MIT-Pi.txt b/LICENSES/MIT-Pi.txt new file mode 100644 index 00000000..4864295e --- /dev/null +++ b/LICENSES/MIT-Pi.txt @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2025 Mario Zechner + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index f55679e4..0027dc6b 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,7 @@ It analyzes your source code, identifies attack paths, and executes real exploit - [Documentation](#documentation) - [Safety, Scope, and Limitations](#safety-scope-and-limitations) - [License](#license) +- [Acknowledgements](#acknowledgements) - [About Keygraph](#about-keygraph) - [Community and Support](#community-and-support) - [Common Questions](#common-questions) @@ -230,6 +231,14 @@ Commercial and enterprise licensing is available for organizations that need dif For commercial licensing, contact [shannon@keygraph.io](mailto:shannon@keygraph.io). +## Acknowledgements + +Thanks to [Pi](https://github.com/earendil-works/pi), +[Playwright CLI](https://github.com/microsoft/playwright-cli), +and [Mantis](https://github.com/google/mantis). + +See [THIRD_PARTY_NOTICES.md](./THIRD_PARTY_NOTICES.md) for licensing and attribution details. + ## About Keygraph **Keygraph** is the company behind Shannon. It also builds the **Keygraph platform**, the commercial agentic pentesting product that closes the full AppSec lifecycle and runs an enhanced build of Shannon as its pentesting engine. diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md new file mode 100644 index 00000000..9e1bfd87 --- /dev/null +++ b/THIRD_PARTY_NOTICES.md @@ -0,0 +1,46 @@ +# Third-Party Notices + +Shannon incorporates and adapts material from third-party open-source projects. + +Shannon as a whole is distributed under the GNU Affero General Public License, +version 3.0 (see LICENSE). Third-party material incorporated into Shannon +remains subject to the attribution and notice requirements of its own license. + +## Pi + +Shannon uses Pi as part of its agent framework. + +Project: https://github.com/earendil-works/pi +License: MIT + +Copyright (c) 2025 Mario Zechner + +The applicable license is reproduced at `LICENSES/MIT-Pi.txt`. + +## Mantis + +Portions of Shannon's Capella agentic SAST implementation, specifically the +agent prompts, are derived from the Mantis project. + +- Project: https://github.com/google/mantis +- Upstream commit: 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 +- Retrieved: 2026-08-25 +- License: Apache License, Version 2.0 + +The Apache License, Version 2.0 is reproduced at LICENSES/Apache-2.0.txt. + +The Mantis-derived files are individually marked with a provenance header and +reside under: + +- apps/worker/prompts/partials/ (capella-*.hbs prompt partials) +- apps/worker/prompts/sast/capella/ (prompt templates) + +The Mantis-derived material has been substantially modified by Keygraph +for use within Shannon, including adaptation to Shannon's agent +architecture and the Pi agent framework. + +Copyright and attribution notices from the original Mantis material +remain the property of their respective copyright holders. + +Modifications: +Copyright © 2026 Keygraph, Inc. diff --git a/apps/cli/package.json b/apps/cli/package.json index e2c5a0da..495bcb34 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -18,7 +18,7 @@ }, "dependencies": { "@clack/prompts": "^1.1.0", - "@temporalio/client": "^1.11.0", + "@temporalio/client": "1.15.0", "chokidar": "^5.0.0", "dotenv": "^17.3.1", "smol-toml": "^1.6.1" diff --git a/apps/cli/src/commands/logs.ts b/apps/cli/src/commands/logs.ts index 84989c3f..57f93aa5 100644 --- a/apps/cli/src/commands/logs.ts +++ b/apps/cli/src/commands/logs.ts @@ -9,6 +9,7 @@ import fs from 'node:fs'; import path from 'node:path'; +import { StringDecoder } from 'node:string_decoder'; import { setTimeout as sleep } from 'node:timers/promises'; import { watch } from 'chokidar'; import { fail } from '../errors.js'; @@ -18,8 +19,69 @@ import { resolveWorkflowId } from '../session.js'; import { waitForWorkflowClose } from '../temporal-client.js'; import { stdoutIsTerminal } from '../tty.js'; -/** Read a byte range from a file and return it as a UTF-8 string. */ -function readRange(filePath: string, start: number, end: number): string { +const TERMINAL_HEADINGS = new Set(['Scan COMPLETED', 'Scan PARTIAL', 'Scan FAILED', 'Scan CANCELLED']); + +// The combined log resets completion on the bare `RESUMED` heading; a per-agent file carries the +// distinct `--- RESUMED () ---` boundary that WorkflowLogger.logResumeBoundary writes +// (kept distinct per resume so it stays idempotent per file). Both mean a new execution began, so a +// `--agent` tail must clear a stale terminal marker on either, matching the combined tail. +const AGENT_RESUME_BOUNDARY = /^--- RESUMED \(.+\) ---$/u; + +function isResumeBoundary(line: string): boolean { + return line === 'RESUMED' || AGENT_RESUME_BOUNDARY.test(line); +} + +/** Tracks only complete structural lines while output remains byte-for-byte unchanged. */ +export class LogCompletionState { + private pendingLine = ''; + private terminalIsLastMarker = false; + private failureIsLastMarker = false; + + ingest(chunk: string): void { + const lines = `${this.pendingLine}${chunk}`.split('\n'); + this.pendingLine = lines.pop() ?? ''; + for (const line of lines) { + if (isResumeBoundary(line)) { + this.terminalIsLastMarker = false; + this.failureIsLastMarker = false; + } else if (TERMINAL_HEADINGS.has(line)) { + this.terminalIsLastMarker = true; + this.failureIsLastMarker = line === 'Scan FAILED'; + } + } + } + + isComplete(): boolean { + return this.terminalIsLastMarker; + } + + hasFailureMarker(): boolean { + return this.failureIsLastMarker; + } +} + +/** Append the forced-stop marker after the worker has exited, unless this execution already ended. */ +export function appendCancellationFallback(logFile: string): void { + fs.mkdirSync(path.dirname(logFile), { recursive: true }); + const state = new LogCompletionState(); + try { + state.ingest(fs.readFileSync(logFile, 'utf8')); + } catch { + // A pre-registration stop may not have created the file yet. + } + if (state.isComplete()) return; + + const descriptor = fs.openSync(logFile, 'a', 0o600); + try { + fs.writeSync(descriptor, '\nScan CANCELLED\n'); + fs.fsyncSync(descriptor); + } finally { + fs.closeSync(descriptor); + } +} + +/** Read a byte range without decoding across an arbitrary live-write boundary. */ +function readRange(filePath: string, start: number, end: number): Buffer { const length = end - start; const buffer = Buffer.alloc(length); const fd = fs.openSync(filePath, 'r'); @@ -28,7 +90,7 @@ function readRange(filePath: string, start: number, end: number): string { } finally { fs.closeSync(fd); } - return buffer.toString('utf-8'); + return buffer; } /** Resolve a workspace ID to its workflow.log path, or exit with an error. */ @@ -76,9 +138,6 @@ export interface TailResult { readonly sawFailure: boolean; } -// The worker writes this exact line at the head of its terminal failure summary. -const FAILURE_MARKER = /^Scan FAILED$/m; - /** * Stream a scan's log to the terminal until the workflow closes (completion comes from Temporal, * or Ctrl-C). A Temporal outage is warned about and, if sustained, ends the tail with a diagnostic. @@ -88,24 +147,25 @@ const FAILURE_MARKER = /^Scan FAILED$/m; export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Promise { return new Promise((resolve) => { let position = 0; + const completion = new LogCompletionState(); + const completionDecoder = new StringDecoder('utf8'); let done = false; - let sawFailure = false; const controller = new AbortController(); let watcher: ReturnType | undefined; /** Output any new content appended since the last read. */ - function flush(): void { + function flush(): boolean { try { const { size } = fs.statSync(logFile); - if (size <= position) return; + if (size <= position) return completion.isComplete(); const data = readRange(logFile, position, size); process.stdout.write(data); position = size; - if (!sawFailure && FAILURE_MARKER.test(data)) { - sawFailure = true; - } + completion.ingest(completionDecoder.write(data)); + return completion.isComplete(); } catch { // File not present yet or transiently unreadable — nothing to flush this round. + return false; } } @@ -113,22 +173,32 @@ export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Prom if (done) return; done = true; controller.abort(); + process.off('SIGINT', finish); + const result = { sawFailure: completion.hasFailureMarker() }; if (watcher) { - watcher.close().finally(() => resolve({ sawFailure })); + watcher.close().finally(() => resolve(result)); // Safety net — resolve anyway if watcher.close() stalls. - setTimeout(() => resolve({ sawFailure }), 1000).unref(); + setTimeout(() => resolve(result), 1000).unref(); } else { - resolve({ sawFailure }); + resolve(result); } } - // 1. Output existing content, then stream anything appended. - flush(); + // 1. Output existing content, then stream anything appended. A per-agent file can be created + // after the watcher starts, so `add` is handled too and streams it from its first line. watcher = watch(logFile, { persistent: true }); - watcher.on('change', () => flush()); + const onFsEvent = (): void => { + if (flush() && !opts.workflowId) finish(); + }; + watcher.on('change', onFsEvent); + watcher.on('add', onFsEvent); + if (flush() && !opts.workflowId) { + finish(); + return; + } // 2. Ctrl-C stops watching. - process.on('SIGINT', finish); + process.once('SIGINT', finish); // 3. Temporal decides completion. Without a workflow id, the tail relies on Ctrl-C alone. if (opts.workflowId) { @@ -162,11 +232,51 @@ export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Prom }); } -export function logs(workspaceId: string): void { - const logFile = resolveLogFile(workspaceId); - const workflowId = resolveWorkflowId(workspaceId); - console.error(stdoutIsTerminal() ? `Tailing scan log: ${logFile}` : 'Tailing scan log'); +/** The `.shannon/agents/` directory that sits beside a scan's combined workflow.log. */ +function agentsDirFor(logFile: string): string { + return path.join(path.dirname(logFile), 'agents'); +} +/** List the per-agent log names available for a scan (filename stems, sorted), or an empty list. */ +export function listAgentLogNames(logFile: string): string[] { + try { + return fs + .readdirSync(agentsDirFor(logFile)) + .filter((entry) => entry.endsWith('.log')) + .map((entry) => entry.slice(0, -'.log'.length)) + .sort(); + } catch { + return []; + } +} + +/** + * Resolve an agent name to its per-agent log path. The name must be a closed-charset basename, and + * the resolved file must stay inside the agents directory: traversal and symlink escapes are + * rejected. Returns undefined when the name is structurally invalid or escapes the directory. + */ +export function resolveAgentLogFile(logFile: string, agentName: string): string | undefined { + if (!/^[a-z0-9][a-z0-9-]{0,63}$/u.test(agentName)) return undefined; + const agentsDir = agentsDirFor(logFile); + const target = path.join(agentsDir, `${agentName}.log`); + try { + const realDir = fs.realpathSync(agentsDir); + const realTarget = fs.realpathSync(target); + if (realTarget !== path.join(realDir, `${agentName}.log`)) return undefined; + } catch { + // The file does not exist yet (scan still starting); the closed-charset check already proved + // the path cannot traverse out of the agents directory, so it is safe to watch for creation. + } + return target; +} + +export interface LogsOptions { + readonly agent?: string; + readonly listAgents?: boolean; +} + +function tailFileToExit(logFile: string, workflowId: string | undefined, label: string): void { + console.error(stdoutIsTerminal() ? `${label}: ${logFile}` : label); let unreachable = false; tailUntilComplete(logFile, { ...(workflowId ? { workflowId } : {}), @@ -175,3 +285,40 @@ export function logs(workspaceId: string): void { }, }).finally(() => process.exit(unreachable ? 1 : 0)); } + +export function logs(workspaceId: string, options: LogsOptions = {}): void { + const logFile = resolveLogFile(workspaceId); + + if (options.listAgents) { + const names = listAgentLogNames(logFile); + if (names.length === 0) { + console.error('No per-agent logs for this scan yet.'); + process.exit(0); + } + for (const name of names) console.log(name); + process.exit(0); + } + + const workflowId = resolveWorkflowId(workspaceId); + + if (options.agent !== undefined) { + const agentFile = resolveAgentLogFile(logFile, options.agent); + if (agentFile === undefined) { + fail(`No agent log named: ${options.agent}`, '', 'Available agents:', ...withBullets(listAgentLogNames(logFile))); + } + const known = listAgentLogNames(logFile); + // If the directory already lists agents, a name not among them is a typo, not a not-yet-created + // file; fail loudly rather than tailing a path that will never appear. + if (known.length > 0 && !known.includes(options.agent)) { + fail(`No agent log named: ${options.agent}`, '', 'Available agents:', ...withBullets(known)); + } + tailFileToExit(agentFile, workflowId, `Tailing ${options.agent} log`); + return; + } + + tailFileToExit(logFile, workflowId, 'Tailing scan log'); +} + +function withBullets(names: readonly string[]): string[] { + return names.length === 0 ? [' (none yet)'] : names.map((name) => ` - ${name}`); +} diff --git a/apps/cli/src/commands/scans.ts b/apps/cli/src/commands/scans.ts index d184eef7..bb78fd42 100644 --- a/apps/cli/src/commands/scans.ts +++ b/apps/cli/src/commands/scans.ts @@ -1,23 +1,28 @@ /** - * `shannon scans` command — list completed scans and where each report lives. + * `shannon scans` command — list scans, running and completed, and where each report lives. * - * A scan counts as completed when it produced a report. The report can live in any of a - * few locations depending on the version that ran it, so `findReport` probes them in order - * and the first hit is both the completion signal and the link target behind the workspace - * name. The date and wall-clock duration come from the run's session.json - * (createdAt/completedAt), with the report file's mtime as the date fallback for - * runs that lack a recorded time. + * Running scans come from Docker: every worker container is stamped with the shannon.workspace + * label, so `runningScanWorkspaces()` is the authoritative live-scan list (shared with `stop`). + * A scan counts as completed once it has produced a report; the report can live in any of a few + * locations depending on the version that ran it, so `findReport` probes them in order and the + * first hit is both the completion signal and the link target behind the workspace name. Dates and + * durations come from each run's session.json (createdAt/completedAt), with the report file's mtime + * as the date fallback for runs that lack a recorded time; a running scan's duration is elapsed time + * so far (now − createdAt). * - * Human-readable by default; `--json` emits the same rows as raw machine values on stdout. + * Running scans are listed first, then completed newest-first. Human-readable by default; `--json` + * emits the same rows as raw machine values on stdout. * - * Filesystem-only (local ./workspaces/ or npx ~/.shannon/workspaces/ via getWorkspacesDir); - * no Temporal dependency. + * The completed list is filesystem-only (local ./workspaces/ or npx ~/.shannon/workspaces/ via + * getWorkspacesDir); the running list needs Docker but degrades to empty when the daemon is down, + * which is the correct answer (no scan can be running then). */ import fs from 'node:fs'; import path from 'node:path'; import { pathToFileURL } from 'node:url'; -import { BOLD, GOLD, paint } from '../colors.js'; +import { BOLD, CYAN, GOLD, paint } from '../colors.js'; +import { runningScanWorkspaces } from '../docker.js'; import { getWorkspacesDir } from '../home.js'; import { commandPrefix } from '../mode.js'; import { FINAL_REPORT_PDF_FILENAME, INTERNAL_DIR, resolveRunFile } from '../paths.js'; @@ -31,23 +36,25 @@ const FINAL_REPORT_MD_FILENAME = 'Security-Assessment-Report.md'; const DELIVERABLES_SUBDIR = 'deliverables'; -/** One completed scan; raw values so the table and --json render from one source. */ +/** One scan, running or completed; raw values so the table and --json render from one source. */ interface ScanRow { readonly workspace: string; - /** Completion time in ms — sort key and date source. */ - readonly finishedMs: number; - /** Wall-clock duration (completedAt − createdAt) in ms, or null when unknown. */ + readonly state: 'running' | 'completed'; + /** Completion time in ms — sort key and date source. Null while a scan is still running. */ + readonly finishedMs: number | null; + /** Wall-clock duration in ms: elapsed-so-far for running, total for completed. Null when unknown. */ readonly durationMs: number | null; - /** Absolute path to the report file — the link target behind the workspace name. */ - readonly report: string; + /** Absolute path to the report file — the link target behind the workspace name. Null while running. */ + readonly report: string | null; } -/** The --json row shape: raw machine values, one per completed scan. */ +/** The --json row shape: raw machine values, one per scan. */ interface JsonRow { readonly workspace: string; - readonly finishedAt: string; + readonly state: 'running' | 'completed'; + readonly finishedAt: string | null; readonly durationMs: number | null; - readonly reportPath: string; + readonly reportPath: string | null; } /** Compact wall-clock duration from milliseconds: "47s", "1m 32s", "1h 47m". */ @@ -130,7 +137,19 @@ function collectCompletedScans(workspacesDir: string): ScanRow[] { const finishedMs = Number.isNaN(completedMs) ? fs.statSync(reportPath).mtimeMs : completedMs; const durationMs = Number.isNaN(completedMs) || Number.isNaN(createdMs) ? null : completedMs - createdMs; - rows.push({ workspace: entry.name, finishedMs, durationMs, report: reportPath }); + rows.push({ workspace: entry.name, state: 'completed', finishedMs, durationMs, report: reportPath }); + } + return rows; +} + +/** Gather every currently-running scan, one row each. Elapsed time is now − createdAt. */ +function collectRunningScans(workspacesDir: string, nowMs: number): ScanRow[] { + const rows: ScanRow[] = []; + for (const workspace of runningScanWorkspaces()) { + const { session } = readSession(path.join(workspacesDir, workspace)); + const createdMs = Date.parse(session.createdAt ?? ''); + const durationMs = Number.isNaN(createdMs) ? null : nowMs - createdMs; + rows.push({ workspace, state: 'running', finishedMs: null, durationMs, report: null }); } return rows; } @@ -138,55 +157,68 @@ function collectCompletedScans(workspacesDir: string): ScanRow[] { function toJsonRow(row: ScanRow): JsonRow { return { workspace: row.workspace, - finishedAt: new Date(row.finishedMs).toISOString(), + state: row.state, + finishedAt: row.finishedMs === null ? null : new Date(row.finishedMs).toISOString(), durationMs: row.durationMs, reportPath: row.report, }; } -/** Print the completed scans as an aligned table with the workspace name linked to its report. */ +/** Print the scans as an aligned table with each completed workspace name linked to its report. */ function printTable(workspacesDir: string, rows: readonly ScanRow[]): void { if (rows.length === 0) { const prefix = commandPrefix(); - console.log(`No completed scans yet. Run '${prefix} start -u -r ' to begin.`); + console.log(`No scans yet. Run '${prefix} start -u -r ' to begin.`); return; } const color = supportsColor(); - // On a terminal the workspace name is an OSC 8 hyperlink that opens its report; when - // piped there is nothing to click, so it prints as plain text. + // On a terminal a completed workspace name is an OSC 8 hyperlink that opens its report; when + // piped, or for a running scan that has no report yet, it prints as plain text. const linkable = stdoutIsTerminal(); const table = rows.map((row) => ({ - finished: new Date(row.finishedMs).toISOString().slice(0, 10), + state: row.state === 'running' ? 'RUNNING' : 'COMPLETED', + finished: row.finishedMs === null ? '—' : new Date(row.finishedMs).toISOString().slice(0, 10), duration: row.durationMs === null ? '—' : formatDuration(row.durationMs), workspace: row.workspace, report: row.report, })); + const stateWidth = Math.max('STATE'.length, ...table.map((row) => row.state.length)); const dateWidth = Math.max('FINISHED'.length, 'YYYY-MM-DD'.length); const durationWidth = Math.max('DURATION'.length, ...table.map((row) => row.duration.length)); - console.log(`\nCompleted scans in ${workspacesDir}:\n`); - const header = `${'FINISHED'.padEnd(dateWidth)} ${'DURATION'.padEnd(durationWidth)} WORKSPACE`; + console.log(`\nScans in ${workspacesDir}:\n`); + const header = `${'STATE'.padEnd(stateWidth)} ${'FINISHED'.padEnd(dateWidth)} ${'DURATION'.padEnd(durationWidth)} WORKSPACE`; console.log(paint(header, BOLD, color)); for (const row of table) { + const stateText = row.state.padEnd(stateWidth); + const state = row.state === 'RUNNING' ? paint(stateText, CYAN, color) : stateText; const finished = row.finished.padEnd(dateWidth); const duration = row.duration.padEnd(durationWidth); - const name = paint(row.workspace, GOLD, color); - const workspace = linkable ? hyperlink(name, pathToFileURL(row.report).href) : name; - console.log(`${finished} ${duration} ${workspace}`); + // A running scan has no report to open, so its name stays plain; completed names are linked. + const name = row.report ? paint(row.workspace, GOLD, color) : row.workspace; + const workspace = row.report && linkable ? hyperlink(name, pathToFileURL(row.report).href) : name; + console.log(`${state} ${finished} ${duration} ${workspace}`); } console.log(''); } export function scans(opts: { readonly json: boolean }): void { const workspacesDir = getWorkspacesDir(); - const rows = collectCompletedScans(workspacesDir); + const nowMs = Date.now(); - // Latest on top. - rows.sort((a, b) => b.finishedMs - a.finishedMs); + const running = collectRunningScans(workspacesDir, nowMs); + const runningNames = new Set(running.map((row) => row.workspace)); + // A running scan has no final report, so it can't also be completed; guard anyway. + const completed = collectCompletedScans(workspacesDir).filter((row) => !runningNames.has(row.workspace)); + + // Running scans on top (most recently started first), then completed newest-first. + running.sort((a, b) => (a.durationMs ?? 0) - (b.durationMs ?? 0)); + completed.sort((a, b) => (b.finishedMs ?? 0) - (a.finishedMs ?? 0)); + const rows = [...running, ...completed]; if (opts.json) { console.log(JSON.stringify(rows.map(toJsonRow), null, 2)); diff --git a/apps/cli/src/commands/start.ts b/apps/cli/src/commands/start.ts index 0f58f5ae..5fe5934b 100644 --- a/apps/cli/src/commands/start.ts +++ b/apps/cli/src/commands/start.ts @@ -18,6 +18,7 @@ import { commandPrefix, isLocal } from '../mode.js'; import { resolveModelSpec } from '../model-spec.js'; import { expandHome, + FINAL_REPORT_MD_FILENAME, FINAL_REPORT_PDF_FILENAME, INTERNAL_DIR, resolveConfig, @@ -43,83 +44,219 @@ export interface StartArgs { version: string; } +const LAUNCH_STATE_SCHEMA_VERSION = 1 as const; +const LAUNCH_STATE_FILENAME = 'launch.json'; +const FIXED_CLASSES = ['injection', 'xss', 'auth', 'authz', 'ssrf'] as const; + /** - * Upgrade a pre-restructure workspace (flat layout, no INTERNAL_DIR) before it is mounted, - * so resume finds the old deliverables and their git checkpoints instead of re-running every - * agent. For a legacy run every top-level entry is internal, so move them all into INTERNAL_DIR - * (a same-filesystem rename carries the deliverables .git along). + * CLI-owned launch record at INTERNAL_DIR/launch.json, written once when a workspace is + * created and never rewritten. It pins the customer output destination so a resume with a + * different -o cannot silently redirect the final report. The worker does not read it. */ -function migrateLegacyWorkspaceLayout(workspacePath: string): void { - const legacySessionJson = path.join(workspacePath, 'session.json'); - const internalPath = path.join(workspacePath, INTERNAL_DIR); - if (!fs.existsSync(legacySessionJson) || fs.existsSync(internalPath)) { - return; +interface LaunchState { + readonly schema_version: typeof LAUNCH_STATE_SCHEMA_VERSION; + readonly customer_output_path?: string; +} + +export interface WorkspaceLaunchDecision { + readonly isResume: boolean; + readonly outputDir?: string; +} + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === 'object' && !Array.isArray(value); +} + +function arraysEqual(left: readonly unknown[], right: readonly unknown[]): boolean { + return left.length === right.length && left.every((value, index) => value === right[index]); +} + +/** + * Hand-rolled twin of the worker's durable-state validator in + * apps/worker/src/types/run-state.ts, which owns the session.json.durableScanState shape. + * Each array check accepts two variants because the worker appends 'miscellaneous' and + * 'miscellaneous-exploit' only after the miscellaneous pipeline admits findings. If the worker's shape + * changes and this twin lags, resume fails fast as incompatible instead of launching a + * worker against state it would misread. + */ +function isCurrentDurableState(value: unknown): boolean { + if (!isRecord(value) || value.schema_version !== 1 || typeof value.exploit !== 'boolean') return false; + if (!Array.isArray(value.participating_classes) || !Array.isArray(value.expected_agents)) return false; + + const participating = value.participating_classes; + const validParticipation = + arraysEqual(participating, FIXED_CLASSES) || arraysEqual(participating, [...FIXED_CLASSES, 'miscellaneous']); + if (!validParticipation) return false; + + const baselineAgents = ['pre-recon', 'recon', ...FIXED_CLASSES.map((name) => `${name}-vuln`)]; + if (value.exploit) baselineAgents.push(...FIXED_CLASSES.map((name) => `${name}-exploit`)); + baselineAgents.push('report'); + const expected = value.expected_agents; + return arraysEqual(expected, baselineAgents) || arraysEqual(expected, [...baselineAgents, 'miscellaneous-exploit']); +} + +/** One refusal for damaged CLI-owned or worker-owned workspace records, whichever reads first. */ +const DAMAGED_RECORDS_MESSAGE = + "This workspace's internal records are damaged and it cannot be resumed. Its report files are untouched. Start a new scan with a different -w name."; + +const NEWER_RELEASE_MESSAGE = + 'This workspace was created by a newer version of Shannon. Upgrade Shannon, or start a new scan with a different -w name.'; + +function readJsonFile(filePath: string): unknown { + try { + return JSON.parse(fs.readFileSync(filePath, 'utf8')); + } catch { + fail(DAMAGED_RECORDS_MESSAGE); + } +} + +function readLaunchState(filePath: string): LaunchState { + if (!fs.existsSync(filePath)) { + fail( + 'This workspace was created by an earlier version of Shannon and cannot be resumed. Its files and report are untouched. Start a new scan with a different -w name.', + ); + } + const value = readJsonFile(filePath); + if (!isRecord(value)) fail(NEWER_RELEASE_MESSAGE); + // Unknown keys mean a newer release wrote this workspace; refuse rather than half-read it. + const keys = Object.keys(value).sort(); + const keysAreValid = keys.every((key) => key === 'customer_output_path' || key === 'schema_version'); + const customerPath = value.customer_output_path; + const pathIsValid = + customerPath === undefined || + (typeof customerPath === 'string' && path.isAbsolute(customerPath) && path.resolve(customerPath) === customerPath); + if (value.schema_version !== LAUNCH_STATE_SCHEMA_VERSION || !keysAreValid || !pathIsValid) { + fail(NEWER_RELEASE_MESSAGE); + } + return { + schema_version: LAUNCH_STATE_SCHEMA_VERSION, + ...(typeof customerPath === 'string' && { customer_output_path: customerPath }), + }; +} + +/** + * Decide fresh-versus-resume from on-disk state alone, before start() mutates anything. + * A fresh launch requires the workspace directory to be absent or empty; a resume requires + * current-release session state, a matching target URL, and a customer output path that + * agrees with the recorded one. Every other combination fails the launch, so a typo in + * -w or -o stops here instead of spawning a worker into the wrong workspace. + */ +export function classifyWorkspaceLaunch( + workspacePath: string, + expectedUrl: string, + requestedOutputDir: string | undefined, +): WorkspaceLaunchDecision { + const sessionPath = resolveRunFile(workspacePath, 'session.json'); + const sessionExists = fs.existsSync(sessionPath); + if (!sessionExists) { + if (fs.existsSync(workspacePath) && fs.readdirSync(workspacePath).length > 0) { + fail( + 'This directory is not a Shannon workspace, or its scan state is missing. Start a new scan with a different -w name.', + ); + } + return { isResume: false, ...(requestedOutputDir !== undefined && { outputDir: requestedOutputDir }) }; } - fs.mkdirSync(internalPath, { recursive: true }); - for (const entry of fs.readdirSync(workspacePath)) { - if (entry === INTERNAL_DIR) { - continue; + const launchPath = path.join(workspacePath, INTERNAL_DIR, LAUNCH_STATE_FILENAME); + const launch = readLaunchState(launchPath); + const session = readJsonFile(sessionPath); + if (!isRecord(session) || !isRecord(session.session) || session.session.webUrl !== expectedUrl) { + fail( + 'This workspace was created for a different target URL, so it cannot be resumed against this one. Check -u, or start a new scan with a different -w name.', + ); + } + if (!isCurrentDurableState(session.durableScanState)) { + fail( + "This workspace's scan state cannot be read by this version. Its files are untouched. Start a new scan with a different -w name.", + ); + } + + const storedOutputDir = launch.customer_output_path; + if (requestedOutputDir !== undefined && requestedOutputDir !== storedOutputDir) { + fail( + 'This workspace already copies its report to a different location than the -o path you passed. Re-run without -o to keep the original location, or start a new scan with a different -w name.', + ); + } + return { isResume: true, ...(storedOutputDir !== undefined && { outputDir: storedOutputDir }) }; +} + +/** + * Crash-safe single write: exclusive temp file (pid plus random suffix keeps concurrent + * starts apart), fsync, rename into place, then directory fsync so the entry survives a + * host crash. Callers invoke this only for a fresh workspace; an existing launch.json is + * the resume contract and must never be replaced. + */ +export function writeLaunchStateAtomically(internalPath: string, outputDir: string | undefined): void { + const finalPath = path.join(internalPath, LAUNCH_STATE_FILENAME); + const temporaryPath = path.join(internalPath, `${LAUNCH_STATE_FILENAME}.tmp-${process.pid}-${randomSuffix()}`); + const launchState: LaunchState = { + schema_version: LAUNCH_STATE_SCHEMA_VERSION, + ...(outputDir !== undefined && { customer_output_path: outputDir }), + }; + const descriptor = fs.openSync(temporaryPath, 'wx', 0o600); + try { + fs.writeFileSync(descriptor, `${JSON.stringify(launchState, null, 2)}\n`, 'utf8'); + fs.fsyncSync(descriptor); + } finally { + fs.closeSync(descriptor); + } + try { + fs.renameSync(temporaryPath, finalPath); + const directory = fs.openSync(internalPath, 'r'); + try { + fs.fsyncSync(directory); + } finally { + fs.closeSync(directory); } - fs.renameSync(path.join(workspacePath, entry), path.join(internalPath, entry)); + } catch (error) { + fs.rmSync(temporaryPath, { force: true }); + throw error; } - console.log(`Migrated workspace to ${INTERNAL_DIR}/ layout: ${workspacePath}`); } export async function start(args: StartArgs): Promise { - // 1. Initialize state directories and load env + // 1. Resolve non-mutating inputs and classify the workspace before changing it. initHome(); loadEnv(); - - // 2. Validate credentials const creds = validateCredentials(); if (!creds.valid) { fail(creds.error ?? 'Invalid credentials'); } - - // 3. Resolve paths const repo = resolveRepo(args.repo); const config = args.config ? resolveConfig(args.config) : undefined; + const workspacesDir = getWorkspacesDir(); + const workspace = + args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`; + const workspacePath = path.join(workspacesDir, workspace); + const requestedOutputDir = args.output ? path.resolve(expandHome(args.output)) : undefined; + const launchDecision = classifyWorkspaceLaunch(workspacePath, args.url, requestedOutputDir); - // Inputs are valid — identify the run before the Docker/Temporal setup work. + // 2. Inputs are valid; identify the run before initializing shared infrastructure. const bannerVersion = isLocal() ? undefined : args.version; if (stdoutIsTerminal()) { displaySplash(bannerVersion); } else { displayPlainBanner(bannerVersion); } - - // 4. Ensure workspaces dir is writable by container user (UID 1001) - const workspacesDir = getWorkspacesDir(); fs.mkdirSync(workspacesDir, { recursive: true }); fs.chmodSync(workspacesDir, 0o777); - - // 5. Ensure Docker and the worker image are available (pull/build prints its own progress). ensureDocker(); ensureImage(args.version); - - // One spinner spans the whole launch: bringing up Temporal and registering the worker. const spinner = p.spinner(); spinner.start('Starting scan'); await ensureInfra(spinner); - // 6. Generate unique task queue and container name + // 3. Generate the invocation identity. const suffix = randomSuffix(); const taskQueue = `shannon-${suffix}`; const containerName = `shannon-worker-${suffix}`; - // 7. Generate workspace name if not provided - const workspace = - args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`; - - // 8. Create writable overlay directories (mounted over :ro repo paths inside container) + // 4. Create writable overlay directories after resume validation has succeeded. // The run dir and its INTERNAL_DIR must be 0o777 so the container user can create audit // subdirs and the overlay backing dirs. - const workspacePath = path.join(workspacesDir, workspace); const internalPath = path.join(workspacePath, INTERNAL_DIR); fs.mkdirSync(workspacePath, { recursive: true }); fs.chmodSync(workspacePath, 0o777); - migrateLegacyWorkspaceLayout(workspacePath); fs.mkdirSync(internalPath, { recursive: true }); fs.chmodSync(internalPath, 0o777); for (const dir of ['deliverables', 'scratchpad', '.playwright-cli', '.playwright']) { @@ -127,24 +264,37 @@ export async function start(args: StartArgs): Promise { fs.mkdirSync(dirPath, { recursive: true }); fs.chmodSync(dirPath, 0o777); } + if (!launchDecision.isResume) { + writeLaunchStateAtomically(internalPath, launchDecision.outputDir); + } - // 9. Pre-create overlay mount points (:ro mounts can't auto-create them) + // 5. Pre-create overlay mount points (:ro mounts cannot create them). const shannonDir = path.join(repo.hostPath, '.shannon'); for (const dir of ['deliverables', 'scratchpad', '.playwright-cli']) { fs.mkdirSync(path.join(shannonDir, dir), { recursive: true }); } fs.mkdirSync(path.join(repo.hostPath, '.playwright'), { recursive: true }); - // 10. Resolve output directory - const outputDir = args.output ? path.resolve(expandHome(args.output)) : undefined; + // 6. Create the validated customer-copy destination, if configured. + const outputDir = launchDecision.outputDir; if (outputDir) { fs.mkdirSync(outputDir, { recursive: true }); } - // 11. Resolve prompts directory (local mode only) + // 7. Resolve prompts and capture the pre-launch resume counter. const promptsDir = isLocal() ? path.resolve('apps/worker/prompts') : undefined; + const sessionJson = resolveRunFile(workspacePath, 'session.json'); + const isResume = launchDecision.isResume; + let initialResumeCount = 0; + if (isResume) { + // Docker and Temporal startup sit between this read and the classification that validated the + // same file, so a file that changed in between is a workspace-state failure, not a CLI bug. + const session = readJsonFile(sessionJson); + const attempts = isRecord(session) && isRecord(session.session) ? session.session.resumeAttempts : undefined; + initialResumeCount = Array.isArray(attempts) ? attempts.length : 0; + } - // 12. Spawn worker container + // 8. Spawn the worker container. const proc = spawnWorker({ version: args.version, url: args.url, @@ -173,24 +323,16 @@ export async function start(args: StartArgs): Promise { process.exit(1); } - // Detect whether this is a fresh workspace or a resume by checking session.json existence - const sessionJson = resolveRunFile(path.join(workspacesDir, workspace), 'session.json'); - const isResume = fs.existsSync(sessionJson); - let initialResumeCount = 0; - if (isResume) { - try { - const session = JSON.parse(fs.readFileSync(sessionJson, 'utf-8')); - initialResumeCount = session.session?.resumeAttempts?.length ?? 0; - } catch { - // Corrupted file — worker will handle validation - } - } - let started = false; + // Set when the startup poll times out but session.json already holds durable state this + // release understands: the workflow is executing, so the exit handler must not stop its + // worker. An operator abort is a different intent and still stops it. + let scanRunningUnconfirmed = false; + // Stop the worker only if the scan hasn't registered yet (e.g. Ctrl-C mid-startup). let cleaned = false; - const cleanup = (): void => { + const stopWorker = (): void => { if (cleaned || started) return; cleaned = true; spinner.stop('Stopping scan'); @@ -204,14 +346,17 @@ export async function start(args: StartArgs): Promise { } }; process.on('SIGINT', () => { - cleanup(); + stopWorker(); process.exit(0); }); process.on('SIGTERM', () => { - cleanup(); + stopWorker(); process.exit(0); }); - process.on('exit', cleanup); + process.on('exit', () => { + if (scanRunningUnconfirmed) return; + stopWorker(); + }); // Poll for the workflow to register in session.json; the spinner resolves once it does. spinner.message('Waiting for the scan to start'); @@ -238,10 +383,52 @@ export async function start(args: StartArgs): Promise { await sleep(2000); } + if (classifyStartupTimeout(sessionJson) === 'scan-running') { + scanRunningUnconfirmed = true; + spinner.error('The scan started, but this CLI could not confirm it'); + printUnconfirmedScanHint(workspace, taskQueue, containerName); + process.exit(1); + } + spinner.error('Timed out waiting for the scan to start'); process.exit(1); } +/** + * Read the startup timeout: 'scan-running' when session.json already holds durable state this + * release understands, which only the worker writes and only after Temporal began executing the + * workflow; 'unregistered' when nothing proves the scan started. The distinction decides whether + * timing out may stop the worker container. + */ +export function classifyStartupTimeout(sessionJsonPath: string): 'unregistered' | 'scan-running' { + let session: unknown; + try { + session = JSON.parse(fs.readFileSync(sessionJsonPath, 'utf-8')); + } catch { + return 'unregistered'; + } + if (!isRecord(session) || !isCurrentDurableState(session.durableScanState)) { + return 'unregistered'; + } + return 'scan-running'; +} + +/** Point the operator at a scan that is running but whose startup this CLI could not confirm. */ +function printUnconfirmedScanHint(workspace: string, taskQueue: string, containerName: string): void { + console.log(''); + console.log(' The scan is running and was left alone; only its startup confirmation is missing.'); + console.log(''); + console.log(` Workspace: ${workspace}`); + console.log(` Task queue: ${taskQueue}`); + console.log(` Container: ${containerName}`); + console.log(''); + console.log(' Inspect it:'); + console.log(` Live logs: ${commandPrefix()} logs ${workspace}`); + console.log(` Worker logs: docker logs ${containerName}`); + console.log(' Dashboard: http://localhost:8233'); + console.log(''); +} + /** * Follow a just-started scan (for `--follow`, aimed at CI): stream its log while Temporal drives * completion, then exit on the workflow outcome — 0 if the assessment ran, 1 if the scan failed. @@ -331,7 +518,7 @@ function printInfo(args: StartArgs, workspace: string, repoPath: string, workspa return; } - const reportPath = path.join(workspacesDir, workspace, FINAL_REPORT_PDF_FILENAME); + const reportDir = path.join(workspacesDir, workspace); // When following, the scan log streams inline next, so the "run these to watch it" hints // would only contradict that. @@ -345,6 +532,8 @@ function printInfo(args: StartArgs, workspace: string, repoPath: string, workspa console.log(''); console.log(' Report (when the scan finishes):'); - console.log(` ${reportPath}`); + console.log(` ${reportDir}${path.sep}`); + console.log(` ${FINAL_REPORT_PDF_FILENAME}`); + console.log(` ${FINAL_REPORT_MD_FILENAME}`); console.log(''); } diff --git a/apps/cli/src/commands/status.ts b/apps/cli/src/commands/status.ts index 5ef3a040..ba343e78 100644 --- a/apps/cli/src/commands/status.ts +++ b/apps/cli/src/commands/status.ts @@ -3,21 +3,28 @@ * * While the scan runs, polls Temporal and redraws the phase/agent tree on a * terminal (a pipe or a finished scan gets a single frame). When the scan reaches - * a terminal state, prints the overall result and exits. Reads Temporal directly — - * no worker, no session files — so it needs Temporal up and shows scans within its - * ~24h retention window. + * a terminal state, prints the overall result and exits. Local session records prove + * the target's canonical workspace/workflow identity; the progress itself is read from + * Temporal directly — no worker — so it needs Temporal up and shows scans within its + * retention window (Shannon configures seven days by default; see SHANNON_TEMPORAL_RETENTION). */ import { setTimeout as sleep } from 'node:timers/promises'; -import { fail } from '../errors.js'; -import { isLocal } from '../mode.js'; +import { failWith } from '../errors.js'; +import { commandPrefix, isLocal } from '../mode.js'; import { type RenderInput, renderScan } from '../scan/render.js'; import { toStatusJson } from '../scan/status-json.js'; -import { resolveWorkflowId } from '../session.js'; import { displaySplash } from '../splash.js'; -import { describeScan, getTerminalOutcome, queryProgress, type ScanDescription } from '../temporal-client.js'; +import { + ActivityMirrorError, + describeScan, + getTerminalOutcome, + queryProgress, + type ScanDescription, +} from '../temporal-client.js'; import { stdoutIsTerminal, supportsColor } from '../tty.js'; import { getVersion } from '../version.js'; +import { resolveScanIdentity } from '../workspaces.js'; const HIDE_CURSOR = '\x1b[?25l'; const SHOW_CURSOR = '\x1b[?25h'; @@ -30,6 +37,25 @@ function isTerminalStatus(status: string): boolean { return status !== 'RUNNING' && status !== 'UNSPECIFIED'; } +/** + * Read one scan description, telling the two failure modes apart. A stale activity mirror + * carries its own message and needs a CLI update; anything else is a read that did not reach + * a usable answer, which is most often Temporal being down. + */ +async function readScanDescription(workflowId: string): Promise { + try { + return await describeScan(workflowId); + } catch (error) { + if (error instanceof ActivityMirrorError) failWith('CLI_SCAN_SCHEMA_UNSUPPORTED', error.message); + failWith( + 'CLI_SCAN_STATUS_UNAVAILABLE', + "Could not read this scan's progress.", + 'If Temporal is not running, start a scan to bring it up. If it is running, this build of the CLI', + 'does not recognise part of the scan and needs updating.', + ); + } +} + // Match SGR color escapes (ESC[…m) so a line's on-screen width excludes them. Built from the ESC // char code so the source carries no literal control character. const ANSI_PATTERN = new RegExp(`${String.fromCharCode(27)}\\[[0-9;]*m`, 'g'); @@ -128,10 +154,10 @@ async function watch(workspace: string, workflowId: string): Promise { }, RENDER_MS); for (;;) { - const desc = await describeScan(workflowId); + const desc = await readScanDescription(workflowId); if (!desc) { clearInterval(ticker); - fail(`Scan "${workspace}" is no longer in Temporal.`); + failWith('CLI_SCAN_NOT_FOUND', `Scan "${workspace}" is no longer in Temporal.`); } if (isTerminalStatus(desc.status)) { @@ -153,23 +179,40 @@ async function snapshot(workspace: string, workflowId: string, desc: ScanDescrip : buildRunningInput(workspace, workflowId, desc); } -export async function status(workspace: string, opts: { readonly json: boolean }): Promise { - // A resume spawns a new workflow id (recorded in session.json); resolve through there so status - // follows the current resume, not the superseded original. Fresh scans: the name is the id. - const workflowId = resolveWorkflowId(workspace) ?? workspace; - - let desc: ScanDescription | null; - try { - desc = await describeScan(workflowId); - } catch { - fail('Could not reach Temporal at 127.0.0.1:7233.', 'Start Temporal (it comes up with a scan) and try again.'); +export async function status(target: string, opts: { readonly json: boolean }): Promise { + // Target selection picked a string; identity resolution proves the canonical workspace and + // workflow pair from session records before Temporal is queried. A workspace name follows its + // latest resume; an exact recorded workflow id keeps addressing that execution. A raw id with + // no local record is refused rather than echoed into the required workspace field. + const identity = resolveScanIdentity(target); + if (identity.kind === 'ambiguous') { + failWith( + 'CLI_SCAN_IDENTITY_AMBIGUOUS', + `Multiple workspaces claim workflow ID "${target}": ${identity.claims.join(', ')}.`, + `Run '${commandPrefix()} scans' and pass the workspace directory name instead.`, + ); } + if (identity.kind === 'not-found') { + failWith( + 'CLI_SCAN_IDENTITY_NOT_FOUND', + identity.reason === 'unreadable-record' + ? `Workspace "${target}" has no readable session record (${identity.sessionPath}).` + : `No scan matches "${target}" in the local workspace records.`, + `Run '${commandPrefix()} scans' to list scans.`, + 'Temporal dashboard: http://localhost:8233', + ); + } + const { workspace, workflowId } = identity; + + const desc = await readScanDescription(workflowId); if (!desc) { - fail( + failWith( + 'CLI_SCAN_NOT_FOUND', `No scan found for "${workspace}".`, '', - 'Scans are visible while running and for ~24h after they finish (Temporal retention).', + "Scan histories are available while a scan runs and within Temporal's retention window after it finishes.", + "Shannon configures 7 days of retention by default (override: SHANNON_TEMPORAL_RETENTION). Expired histories can't be restored.", ); } diff --git a/apps/cli/src/commands/stop.ts b/apps/cli/src/commands/stop.ts index 9096823c..f754a817 100644 --- a/apps/cli/src/commands/stop.ts +++ b/apps/cli/src/commands/stop.ts @@ -3,23 +3,29 @@ * Never touches infra or data; to wipe Temporal state entirely, use `shannon reset`. */ +import path from 'node:path'; import * as p from '@clack/prompts'; import { confirmOrExit } from '../confirm.js'; import { anyRunningScanWorkflow, + cancelWorkflow, ensureDocker, isTemporalReady, isWorkflowRunning, runningContainers, + runningScanWorkspaces, scanFilter, stopContainers, - terminateAllWorkflows, terminateWorkflow, WORKER_FILTER, } from '../docker.js'; import { fail, failUsage, warn } from '../errors.js'; +import { getWorkspacesDir } from '../home.js'; import { commandPrefix } from '../mode.js'; +import { resolveRunFile } from '../paths.js'; import { resolveWorkflowId } from '../session.js'; +import { resolveDefaultWorkspace } from '../workspaces.js'; +import { appendCancellationFallback } from './logs.js'; export interface StopOptions { all: boolean; @@ -27,11 +33,77 @@ export interface StopOptions { workspace?: string; } +const CANCELLATION_GRACE_MS = 10_000; +const CANCELLATION_POLL_MS = 250; + +export interface StopTarget { + readonly workspace: string; + readonly workflowId?: string; + readonly workflowRunning: boolean; +} + +export interface StopLifecycle { + readonly cancel: (workflowId: string) => boolean; + readonly isRunning: (workflowId: string) => boolean; + readonly terminate: (workflowId: string) => boolean; + readonly containers: (workspace: string) => string[]; + readonly stopContainers: (ids: string[]) => Promise; + readonly appendFallback: (workspace: string) => void; + readonly wait: (milliseconds: number) => Promise; +} + +const stopLifecycle: StopLifecycle = { + cancel: cancelWorkflow, + isRunning: isWorkflowRunning, + terminate: (workflowId) => terminateWorkflow(workflowId, 'Stopped after cancellation grace period'), + containers: (workspace) => runningContainers(scanFilter(workspace)), + stopContainers, + appendFallback: (workspace) => { + const logFile = resolveRunFile(path.join(getWorkspacesDir(), workspace), 'workflow.log'); + appendCancellationFallback(logFile); + }, + wait: (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds)), +}; + +/** Cancel first; terminate and write the fallback heading only when graceful closure misses its deadline. */ +export async function stopTargetCancelFirst( + target: StopTarget, + lifecycle: StopLifecycle = stopLifecycle, + graceMs: number = CANCELLATION_GRACE_MS, + pollMs: number = CANCELLATION_POLL_MS, +): Promise<'graceful' | 'forced'> { + let forced = !target.workflowRunning || target.workflowId === undefined; + if (!forced && target.workflowId !== undefined) { + lifecycle.cancel(target.workflowId); + const deadline = Date.now() + graceMs; + while (lifecycle.isRunning(target.workflowId) && Date.now() < deadline) { + await lifecycle.wait(pollMs); + } + forced = lifecycle.isRunning(target.workflowId); + if (forced) lifecycle.terminate(target.workflowId); + } + + await lifecycle.stopContainers(lifecycle.containers(target.workspace)); + if (forced && lifecycle.containers(target.workspace).length === 0) { + lifecycle.appendFallback(target.workspace); + } + return forced ? 'forced' : 'graceful'; +} + +/** Apply the same captured-target lifecycle concurrently for `stop --all`. */ +export function stopTargetsCancelFirst( + targets: readonly StopTarget[], + lifecycle: StopLifecycle = stopLifecycle, + graceMs: number = CANCELLATION_GRACE_MS, + pollMs: number = CANCELLATION_POLL_MS, +): Promise { + return Promise.all(targets.map((target) => stopTargetCancelFirst(target, lifecycle, graceMs, pollMs))); +} + /** - * Stop a single scan. Terminating the workflow both clears Temporal's record and - * brings the container down (the worker waits on the workflow result), so that runs - * first; `docker stop` is the fallback for the pre-registration window and an - * unreachable Temporal. The stop is then verified rather than assumed. + * Stop a single scan. Cooperative cancellation gets the first ten seconds so the + * workflow can flush its terminal log; termination and a host-written heading are + * the fallback for the pre-registration window or an unavailable finalizer. */ async function stopSingleScan(workspace: string, yes: boolean): Promise { const workflowId = resolveWorkflowId(workspace); @@ -55,10 +127,7 @@ async function stopSingleScan(workspace: string, yes: boolean): Promise { const spinner = p.spinner(); spinner.start(`Stopping scan ${workspace}`); - if (workflowId && workflowRunning) { - terminateWorkflow(workflowId, `Stopped via shannon stop ${workspace}`); - } - await stopContainers(runningContainers(filter)); + await stopTargetCancelFirst({ workspace, ...(workflowId !== undefined && { workflowId }), workflowRunning }); const stillRunning = runningContainers(filter); if (stillRunning.length > 0) { @@ -77,6 +146,14 @@ async function stopSingleScan(workspace: string, yes: boolean): Promise { async function stopAllScans(yes: boolean): Promise { const temporalUp = isTemporalReady(); const initial = runningContainers(WORKER_FILTER); + const targets = [...new Set(runningScanWorkspaces())].map((workspace): StopTarget => { + const workflowId = resolveWorkflowId(workspace); + return { + workspace, + ...(workflowId !== undefined && { workflowId }), + workflowRunning: Boolean(workflowId && temporalUp && isWorkflowRunning(workflowId)), + }; + }); // Resolve what is running before prompting, so we never confirm a no-op. if (initial.length === 0) { @@ -89,9 +166,8 @@ async function stopAllScans(yes: boolean): Promise { const spinner = p.spinner(); spinner.start('Stopping all scans'); - if (temporalUp) { - terminateAllWorkflows('Stopped via shannon stop --all'); - } + await stopTargetsCancelFirst(targets); + // Keep the legacy safety net for a worker whose workspace label was unavailable. await stopContainers(runningContainers(WORKER_FILTER)); const stillRunning = runningContainers(WORKER_FILTER); @@ -108,6 +184,24 @@ async function stopAllScans(yes: boolean): Promise { } } +/** + * Infer which scan `stop` acts on when neither a workspace nor --all was given: the single + * running scan, announced on stderr so it is never a silent guess. Zero or several running + * scans exit with guidance — there is no most-recent fallback, since stopping a finished + * scan is a no-op. + */ +function resolveStopTarget(): string { + const target = resolveDefaultWorkspace({ allowFinished: false }); + if (target.kind === 'ok') { + console.error(`No workspace given; stopping running scan "${target.workspace}".`); + return target.workspace; + } + if (target.kind === 'ambiguous') { + failUsage('Multiple scans are running — specify which one, or use --all:', ` ${target.running.join(', ')}`); + } + fail('No running scans to stop.', 'Pass a workspace name to stop a specific scan.'); +} + export async function stop(opts: StopOptions): Promise { ensureDocker(); @@ -115,12 +209,12 @@ export async function stop(opts: StopOptions): Promise { if (opts.all && opts.workspace) { failUsage('Pass a workspace name or --all, not both.'); } - if (!opts.all && !opts.workspace) { - failUsage('Specify which scan to stop: `stop `, or `stop --all` to stop every scan.'); - } - if (opts.workspace) { - await stopSingleScan(opts.workspace, opts.yes); + // With no explicit target and no --all, default to the single running scan. + const workspace = opts.all ? undefined : (opts.workspace ?? resolveStopTarget()); + + if (workspace) { + await stopSingleScan(workspace, opts.yes); } else { await stopAllScans(opts.yes); } diff --git a/apps/cli/src/docker.ts b/apps/cli/src/docker.ts index 421bca00..3875fb45 100644 --- a/apps/cli/src/docker.ts +++ b/apps/cli/src/docker.ts @@ -14,7 +14,7 @@ import { setTimeout as sleep } from 'node:timers/promises'; import { fileURLToPath } from 'node:url'; import type { SpinnerResult } from '@clack/prompts'; import { envBool, PI_AUTH_CONTAINER_PATH } from './env.js'; -import { fail } from './errors.js'; +import { fail, warn } from './errors.js'; import { getMode, isDevMode } from './mode.js'; import { INTERNAL_DIR } from './paths.js'; import { runStep, spawnCaptured, surfaceOutput } from './ui.js'; @@ -116,10 +116,8 @@ export function isTemporalReady(): boolean { return output.includes('SERVING'); } -/** - * Ensure Temporal is running via compose. - */ -export async function ensureInfra(spinner: SpinnerResult): Promise { +/** Start (or find) Temporal via compose and wait until it serves; exits the process on failure. */ +async function ensureTemporalHealthy(spinner: SpinnerResult): Promise { if (isTemporalReady()) { return; } @@ -146,6 +144,97 @@ export async function ensureInfra(spinner: SpinnerResult): Promise { process.exit(1); } +const DEFAULT_RETENTION_HOURS = 168; +const RETENTION_ENV = 'SHANNON_TEMPORAL_RETENTION'; +const RETENTION_NAMESPACE = 'default'; + +/** + * Desired retention in whole hours: unset or empty env → 168 (7 days); a positive + * whole-hour override like `72h`; anything else warns and returns null (leave unchanged). + */ +function desiredRetentionHours(): number | null { + const raw = process.env[RETENTION_ENV]; + if (raw === undefined || raw.trim() === '') { + return DEFAULT_RETENTION_HOURS; + } + const match = raw.trim().match(/^([1-9][0-9]*)h$/); + if (!match) { + warn( + `Ignoring invalid ${RETENTION_ENV} "${raw}" — Temporal retention left unchanged.`, + 'Use a positive whole number of hours, e.g. "168h".', + ); + return null; + } + return Number(match[1]); +} + +/** Convert a Go duration such as "24h0m0s" or "168h" to whole seconds, or null when it doesn't parse. */ +function parseGoDurationSeconds(text: string): number | null { + const match = text.match(/^(?:(\d+)h)?(?:(\d+)m)?(?:(\d+)s)?$/); + if (!match || (match[1] === undefined && match[2] === undefined && match[3] === undefined)) { + return null; + } + const hours = Number(match[1] ?? 0); + const minutes = Number(match[2] ?? 0); + const seconds = Number(match[3] ?? 0); + return hours * 3600 + minutes * 60 + seconds; +} + +/** + * Current retention of the `default` namespace in seconds, or null when it can't be read. + * `runOutput` returns '' on a failed describe, so a failed read and an unparseable one both + * collapse to null — either way the live value is unknown, which the caller handles the same way. + */ +function readCurrentRetentionSeconds(): number | null { + const output = runOutput('docker', temporalCmd('operator', 'namespace', 'describe', RETENTION_NAMESPACE)); + const match = output.match(/WorkflowExecutionRetentionTtl\s+(\S+)/); + if (!match || match[1] === undefined) { + return null; + } + return parseGoDurationSeconds(match[1]); +} + +/** + * Converge the `default` namespace's retention to the CLI-owned value after Temporal is + * healthy. The CLI is the authority: a manual change is replaced on the next start unless + * the operator sets the matching override. A describe or update failure warns once that the + * requested value wasn't applied and never blocks the scan. + */ +function convergeNamespaceRetention(): void { + const hours = desiredRetentionHours(); + if (hours === null) { + return; + } + + const currentSeconds = readCurrentRetentionSeconds(); + if (currentSeconds === null) { + warn( + `Could not read Temporal retention for namespace "${RETENTION_NAMESPACE}" — the requested value (${hours}h) was not applied.`, + ); + return; + } + + if (currentSeconds === hours * 3600) { + return; + } + + const updated = runQuiet( + 'docker', + temporalCmd('operator', 'namespace', 'update', '--namespace', RETENTION_NAMESPACE, '--retention', `${hours}h`), + ); + if (!updated) { + warn(`Could not update Temporal retention to ${hours}h — the requested value was not applied.`); + } +} + +/** + * Ensure Temporal is running via compose, then converge its scan-history retention. + */ +export async function ensureInfra(spinner: SpinnerResult): Promise { + await ensureTemporalHealthy(spinner); + convergeNamespaceRetention(); +} + /** * Build the worker image from the repository, tagged with the name this mode * resolves at run time. @@ -345,7 +434,7 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess { args.push('-v', `${opts.config.hostPath}:${opts.config.containerPath}:ro`); } - // Output directory for deliverables copy + // Customer-copy destination. The workflow surfaces only final report artifacts here. if (opts.outputDir) { args.push('-v', `${opts.outputDir}:/app/output`); } @@ -358,7 +447,10 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess { // Environment args.push(...opts.envFlags); - // Container settings + // Container settings. Chromium's own sandbox needs syscalls Docker's default seccomp + // profile blocks, which is why the profile is dropped. `seccomp=unconfined` is a + // container-wide setting, not a per-process one: every process here runs unfiltered, + // the worker included — not just the browser automation that motivates it. args.push('--shm-size', '2gb', '--security-opt', 'seccomp=unconfined'); // Image @@ -405,6 +497,20 @@ export function runningContainers(filter: readonly string[]): string[] { return output.split('\n').filter(Boolean); } +/** + * Workspace names of every running worker container, read from the shannon.workspace + * label each scan is stamped with at spawn. This is the authoritative running-scan → + * workspace-name map. Best-effort: empty when Docker is unreachable, which is the + * correct answer anyway (no scan can be running without the daemon). + */ +export function runningScanWorkspaces(): string[] { + const output = runOutput('docker', ['ps', ...WORKER_FILTER, '--format', `{{ index .Labels "${WORKSPACE_LABEL}" }}`]); + return output + .split('\n') + .map((name) => name.trim()) + .filter(Boolean); +} + /** * Stop containers by ID, tolerating any that vanished between being listed and * stopped (a `--rm` worker exiting is success, not an error). Async so a spinner @@ -414,6 +520,11 @@ export async function stopContainers(ids: string[]): Promise { await Promise.all(ids.map((id) => spawnQuiet('docker', ['stop', id]))); } +/** Request cooperative cancellation so the workflow can run its terminal finalizer. */ +export function cancelWorkflow(workflowId: string): boolean { + return runQuiet('docker', temporalCmd('workflow', 'cancel', '--workflow-id', workflowId)); +} + /** * Terminate a Temporal workflow so a stopped scan doesn't linger as a running * workflow with no worker. Best-effort: returns false if Temporal is unreachable @@ -423,18 +534,6 @@ export function terminateWorkflow(workflowId: string, reason: string): boolean { return runQuiet('docker', temporalCmd('workflow', 'terminate', '--workflow-id', workflowId, '--reason', reason)); } -/** - * Terminate every running pentest workflow in one batch, so `stop --all` doesn't - * leave workflows running with no worker. Best-effort: returns false if Temporal - * is unreachable. Requires Temporal to be up (guard with isTemporalReady). - */ -export function terminateAllWorkflows(reason: string): boolean { - return runQuiet( - 'docker', - temporalCmd('workflow', 'terminate', '--query', RUNNING_SCAN_QUERY, '--reason', reason, '--yes'), - ); -} - /** * Whether a specific workflow is still in the Running state. Re-querying this after * a terminate verifies it actually took effect, rather than trusting the terminate diff --git a/apps/cli/src/errors.ts b/apps/cli/src/errors.ts index b067deba..d40e3ffc 100644 --- a/apps/cli/src/errors.ts +++ b/apps/cli/src/errors.ts @@ -1,37 +1,94 @@ /** * Centralized error reporting. * - * `fail` — an expected, user-fixable error (bad input, missing prerequisite): - * a clean message on stderr and a non-zero exit, never a stack trace. + * `fail` / `failWith` — an expected, user-fixable error (bad input, missing + * prerequisite): a clean message on stderr and a non-zero exit, never a stack trace. * `failUsage` — a malformed invocation (unknown command, bad or missing * arguments): the same clean message, but a distinct exit code so callers can * tell a usage mistake from an operational failure. - * `crash` — an unexpected error (a bug): a brief message, the full stack written - * to a log file for a bug report, and a pointer to the issue tracker. + * `crash` — an unexpected error (a bug): a fixed code and a pointer to the issue tracker. + * + * JSON mode (enabled once, before parsing, for the `--json` command surface) replaces + * the text lines with one compact envelope on stderr — stdout stays empty — while the + * exit-code split is unchanged. Call sites on a JSON-capable path must exit through + * `failWith`/`failUsage`/`crash` (never a bare `fail` or `warn`) so every failure + * carries a stable code and stderr stays parseable. */ import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; const ISSUES_URL = 'https://github.com/KeygraphHQ/shannon/issues'; -/** Report an expected, user-fixable error (with optional extra lines) and exit non-zero. */ -export function fail(message: string, ...hints: string[]): never { +const UNEXPECTED_MESSAGE = 'Shannon encountered an unexpected failure. Reference code: SHANNON_UNEXPECTED_ERROR'; +const REPORT_HINT = `If this looks like a bug, please report it: ${ISSUES_URL}`; + +/** Stable machine-readable failure codes for the JSON error envelope. */ +export type ErrorCode = + | 'CLI_USAGE' + | 'CLI_SCAN_NOT_FOUND' + | 'CLI_SCAN_IDENTITY_NOT_FOUND' + | 'CLI_SCAN_IDENTITY_AMBIGUOUS' + | 'CLI_SCAN_STATUS_UNAVAILABLE' + | 'CLI_SCAN_SCHEMA_UNSUPPORTED' + | 'CLI_PRECONDITION_FAILED' + | 'CLI_INTERNAL_ERROR'; + +let jsonMode = false; + +/** Switch failure reporting to the JSON envelope. Set once, before any guard, parse, or dispatch. */ +export function enableJsonErrors(): void { + jsonMode = true; +} + +/** Whether failures are reported as the JSON envelope rather than text. */ +export function jsonErrorsEnabled(): boolean { + return jsonMode; +} + +/** Fixed unexpected-failure projection shared by the runtime and focused safety tests. */ +export function unexpectedFailureLines(): readonly string[] { + return [`ERROR: ${UNEXPECTED_MESSAGE}`, REPORT_HINT]; +} + +/** + * Report a failure on stderr and exit. Text mode prints the message and every hint + * verbatim; JSON mode writes one compact envelope (dropping the empty strings used + * to space text output) synchronously so `process.exit` cannot truncate it. + */ +function emit(exitCode: 1 | 2, code: ErrorCode, message: string, hints: readonly string[]): never { + if (jsonMode) { + const payload = JSON.stringify({ error: { code, message, hints: hints.filter((hint) => hint.trim() !== '') } }); + fs.writeSync(process.stderr.fd, `${payload}\n`); + process.exit(exitCode); + } console.error(`ERROR: ${message}`); for (const hint of hints) { console.error(hint); } - process.exit(1); + process.exit(exitCode); +} + +/** + * Report an expected, user-fixable error (with optional extra lines) and exit non-zero. + * Text-only paths use this; a JSON-capable path must use `failWith` so the envelope + * carries a real code — if a bare `fail` is ever reached in JSON mode, the fixed + * internal-error envelope is emitted instead of guessing a code for the message. + */ +export function fail(message: string, ...hints: string[]): never { + if (jsonMode) { + emit(1, 'CLI_INTERNAL_ERROR', UNEXPECTED_MESSAGE, [REPORT_HINT]); + } + emit(1, 'CLI_INTERNAL_ERROR', message, hints); +} + +/** Report an expected operational failure under a stable code and exit 1. */ +export function failWith(code: ErrorCode, message: string, ...hints: string[]): never { + emit(1, code, message, hints); } /** Report a usage/argument error (with optional extra lines) and exit 2. */ export function failUsage(message: string, ...hints: string[]): never { - console.error(`ERROR: ${message}`); - for (const hint of hints) { - console.error(hint); - } - process.exit(2); + emit(2, 'CLI_USAGE', message, hints); } /** Report a non-fatal warning on stderr (with optional extra lines) without exiting. */ @@ -42,29 +99,11 @@ export function warn(message: string, ...hints: string[]): void { } } -/** Report an unexpected error: brief message, full stack to a log file, plus the issue link. */ -export function crash(error: unknown): never { - console.error(`ERROR: ${error instanceof Error ? error.message : String(error)}`); - if (process.env.DEBUG) { - console.error(error instanceof Error ? error.stack : String(error)); +/** Report an unexpected error without projecting its message, stack, or attached values. */ +export function crash(_error: unknown): never { + if (jsonMode) { + emit(1, 'CLI_INTERNAL_ERROR', UNEXPECTED_MESSAGE, [REPORT_HINT]); } - - const logPath = writeCrashLog(error); - if (logPath) { - console.error(`Details written to ${logPath}`); - } - console.error(`If this looks like a bug, please report it: ${ISSUES_URL}`); + for (const line of unexpectedFailureLines()) console.error(line); process.exit(1); } - -/** Write the full error and stack to a log file; return its path, or null if it can't be written. */ -function writeCrashLog(error: unknown): string | null { - try { - const logPath = path.join(os.tmpdir(), 'shannon-error.log'); - const detail = error instanceof Error && error.stack ? error.stack : String(error); - fs.writeFileSync(logPath, `${new Date().toISOString()}\n${detail}\n`); - return logPath; - } catch { - return null; - } -} diff --git a/apps/cli/src/help.ts b/apps/cli/src/help.ts index f9da8808..3c22a7a7 100644 --- a/apps/cli/src/help.ts +++ b/apps/cli/src/help.ts @@ -47,30 +47,32 @@ const COMMAND_HELP: Readonly> = { ], }, stop: { - usage: ['stop [--yes]', 'stop --all [--yes]'], - description: 'Stop one scan by workspace, or every scan with --all (Temporal stays up).', + usage: ['stop [] [--yes]', 'stop --all [--yes]'], + description: + 'Stop one scan by workspace, or every scan with --all (Temporal stays up). With no workspace, stops the single running scan; when several are running, name one or use --all.', options: [['--all', 'Stop all running scans'], YES_OPTION], - examples: ['stop q1-audit', 'stop --all'], + examples: ['stop', 'stop q1-audit', 'stop --all'], }, reset: { usage: ['reset'], description: 'Stop everything and permanently remove all Temporal data and volumes.', }, logs: { - usage: ['logs '], - description: "Tail a scan's live log until it completes.", - examples: ['logs q1-audit'], + usage: ['logs []'], + description: + "Tail a scan's live log until it completes. With no workspace, follows the single running scan, or the most recent workspace when none is running; when several are running, name one.", + examples: ['logs', 'logs q1-audit'], }, status: { - usage: ['status [--json]'], + usage: ['status [] [--json]'], description: - "Show one scan's phase-by-phase progress, read live from Temporal. Watches and redraws until the scan finishes on a terminal; prints one frame when piped or already finished. With --json, prints a single machine-readable snapshot and exits.", + "Show one scan's phase-by-phase progress, read live from Temporal. With no workspace, shows the single running scan, or the most recent workspace when none is running; when several are running, name one. Watches and redraws until the scan finishes on a terminal; prints one frame when piped or already finished. With --json, prints a single machine-readable snapshot and exits.", options: [['--json', 'Output a point-in-time snapshot as JSON, then exit']], - examples: ['status q1-audit', 'status q1-audit --json'], + examples: ['status', 'status q1-audit', 'status q1-audit --json'], }, scans: { usage: ['scans [--json]'], - description: 'List completed scans and where each report lives.', + description: 'List running and completed scans, and where each finished report lives.', options: [['--json', 'Output the scan list as JSON']], examples: ['scans', 'scans --json'], }, @@ -102,6 +104,15 @@ export function isHelpableCommand(command: string): boolean { return command in COMMAND_HELP; } +/** + * Every explicit help topic, mode-blind, with `help` itself as the known global topic. + * Topic lookup is deliberately not mode-filtered (unlike `availableCommands`) so + * cross-mode help such as local `help setup` and npx `help build` keeps working. + */ +export function helpTopics(): readonly string[] { + return [...Object.keys(COMMAND_HELP), 'help']; +} + /** * User-facing command names available in the current mode, for "did you mean?" * suggestions. Derived from the same table that backs per-command help, so the diff --git a/apps/cli/src/index.ts b/apps/cli/src/index.ts index c35407e7..383cdb3d 100644 --- a/apps/cli/src/index.ts +++ b/apps/cli/src/index.ts @@ -18,14 +18,21 @@ import { setup } from './commands/setup.js'; import { start } from './commands/start.js'; import { status } from './commands/status.js'; import { stop } from './commands/stop.js'; -import { crash, fail, failUsage } from './errors.js'; -import { availableCommands, isHelpableCommand, printCommandHelp, START_OPTIONS } from './help.js'; +import { crash, enableJsonErrors, fail, failUsage, failWith, jsonErrorsEnabled } from './errors.js'; +import { availableCommands, helpTopics, isHelpableCommand, printCommandHelp, START_OPTIONS } from './help.js'; import { commandPrefix, getMode, isLocal, type Mode } from './mode.js'; import { displaySplash } from './splash.js'; import { closestMatch } from './suggest.js'; import { stdoutIsTerminal } from './tty.js'; import { getVersion, getVersionLine } from './version.js'; +import { resolveDefaultWorkspace } from './workspaces.js'; +/** + * Refuse to run as root or under sudo. The worker container's Linux UID remapping + * (docker.ts) stamps bind-mounted files with the invoking user's real uid/gid; under + * sudo that uid is 0, so the repo, workspace, and report files would come back + * owned by root instead of the person who ran the scan. + */ function blockSudo(): void { const isSudo = !!process.env.SUDO_USER; const isRoot = process.geteuid?.() === 0; @@ -37,15 +44,38 @@ function blockSudo(): void { : []; if (isSudo) { - fail('Shannon must not be run with sudo.', 'Re-run this command as your normal user.', ...linuxHints); + failWith( + 'CLI_PRECONDITION_FAILED', + 'Shannon must not be run with sudo.', + 'Re-run this command as your normal user.', + ...linuxHints, + ); } - fail( + failWith( + 'CLI_PRECONDITION_FAILED', 'Shannon must not be run as the root user.', 'Switch to a regular user account and re-run this command.', ...linuxHints, ); } +/** Commands whose `--json` output contract extends to failures. */ +const JSON_CAPABLE_COMMANDS = new Set(['status', 'scans', 'version', '--version', '-v']); + +/** + * Raw-argv sniff for the JSON error latch, decided before any guard or parse so even + * a pre-dispatch failure honors it. Latches on `--json` or a malformed `--json=` + * (which still fails as a parse error — inside the envelope). Any other command that + * receives `--json` keeps its normal unknown-option behavior. + */ +function wantsJsonErrors(argv: readonly string[]): boolean { + const command = argv[0]; + if (command === undefined || !JSON_CAPABLE_COMMANDS.has(command)) { + return false; + } + return argv.slice(1).some((arg) => arg === '--json' || arg.startsWith('--json=')); +} + /** Render `start`'s flags for the global help, from the same source as `start --help`. */ function renderStartOptions(): string { const flagWidth = Math.max(...START_OPTIONS.map(([flag]) => flag.length)); @@ -60,12 +90,16 @@ function renderUsage(prefix: string, mode: Mode): string { const rows: ReadonlyArray = [ ...(mode === 'local' ? [] : [[`${prefix} setup`, 'Configure credentials'] as const]), [`${prefix} start --url --repo [options]`, 'Start a pentest scan'], - [`${prefix} stop [--yes]`, 'Stop one scan'], + [`${prefix} stop [] [--yes]`, 'Stop one scan (default: the single running scan)'], [`${prefix} stop --all [--yes]`, 'Stop all scans (Temporal stays up)'], [`${prefix} reset`, 'Stop everything and wipe all Temporal data'], - [`${prefix} logs `, "Show a scan's live log"], - [`${prefix} status [--json]`, 'Live phase/agent progress of one scan'], - [`${prefix} scans [--json]`, 'List completed scans and their reports'], + [`${prefix} logs []`, "Show a scan's live log (default: running or most recent)"], + [`${prefix} logs [] --agent `, "Tail one agent's log; --list-agents to list them"], + [ + `${prefix} status [] [--json]`, + 'Live phase/agent progress of one scan (default: running or most recent)', + ], + [`${prefix} scans [--json]`, 'List running and completed scans'], ...(mode === 'local' ? [[`${prefix} build [--no-cache]`, 'Build worker image'] as const] : []), [`${prefix} version [--json]`, 'Show version'], [`${prefix} help`, 'Show this help'], @@ -152,6 +186,32 @@ function parseStartArgs(argv: string[]): ParsedStartArgs { }; } +/** + * Resolve the workspace a viewing command (`logs`, `status`) acts on: the name the user + * gave, or an inferred default. An inferred choice is announced on stderr so it is never a + * silent guess; when nothing can be inferred, exit with usage guidance. + */ +function resolveViewingWorkspace(positional: string | undefined, usage: string): string { + if (positional) { + return positional; + } + + const target = resolveDefaultWorkspace({ allowFinished: true }); + if (target.kind === 'ok') { + // In JSON mode stderr is reserved for the single error envelope, so a successful + // inference stays silent — the JSON payload itself names the chosen workspace. + if (!jsonErrorsEnabled()) { + const which = target.running ? 'running scan' : 'most recent scan'; + console.error(`No workspace given; using ${which} "${target.workspace}".`); + } + return target.workspace; + } + if (target.kind === 'ambiguous') { + failUsage('Multiple scans are running — specify which one:', ` ${target.running.join(', ')}`, '', usage); + } + failUsage('Workspace is required', usage); +} + // === Main Dispatch === async function main(): Promise { @@ -163,13 +223,17 @@ async function main(): Promise { throw err; }); + if (wantsJsonErrors(process.argv.slice(2))) { + enableJsonErrors(); + } + blockSudo(); const args = process.argv.slice(2); const command = args[0]; const rest = args.slice(1); - if (command === undefined || command === 'help' || command === '--help' || command === '-h') { + if (command === undefined || command === '--help' || command === '-h') { const topic = rest[0]; if (topic && isHelpableCommand(topic)) { printCommandHelp(topic); @@ -181,6 +245,27 @@ async function main(): Promise { return; } + // An explicit `help ` names a topic on purpose, so an unknown one is a usage + // error — unlike `--help `, where the junk is ignored and global help wins. + if (command === 'help') { + const topic = rest[0]; + // A flag (`help --help`) is a help request, not a topic name. + if (topic === undefined || topic === 'help' || topic.startsWith('-')) { + showHelp(false); + return; + } + if (isHelpableCommand(topic)) { + printCommandHelp(topic); + return; + } + const suggestion = closestMatch(topic, helpTopics()); + failUsage( + `Unknown help topic: ${topic}`, + ...(suggestion ? [`Did you mean '${suggestion}'?`] : []), + `Run '${commandPrefix()} help' to see available commands.`, + ); + } + // Reachable from any invocation: `-h`/`--help` anywhere wins over the rest of the line. if (isHelpableCommand(command) && (rest.includes('-h') || rest.includes('--help'))) { printCommandHelp(command); @@ -210,20 +295,25 @@ async function main(): Promise { break; } case 'logs': { - const { positionals } = parseArgs(rest, { maxPositionals: 1 }); - const workspaceId = positionals[0]; - if (!workspaceId) { - failUsage('Workspace ID is required', `Usage: ${commandPrefix()} logs `); - } - logs(workspaceId); + const { flags, values, positionals } = parseArgs(rest, { + booleans: { listAgents: ['--list-agents'] }, + values: { agent: ['--agent'] }, + maxPositionals: 1, + }); + const workspaceId = resolveViewingWorkspace( + positionals[0], + `Usage: ${commandPrefix()} logs [] [--agent ] [--list-agents]`, + ); + logs(workspaceId, { + ...(values.agent !== undefined && { agent: values.agent }), + ...(flags.listAgents && { listAgents: true }), + }); break; } case 'status': { const { flags, positionals } = parseArgs(rest, { booleans: { json: ['--json'] }, maxPositionals: 1 }); - const workspaceId = positionals[0]; - if (!workspaceId) { - failUsage('Workspace is required', `Usage: ${commandPrefix()} status [--json]`); - } + const usage = `Usage: ${commandPrefix()} status [] [--json]`; + const workspaceId = resolveViewingWorkspace(positionals[0], usage); await status(workspaceId, { json: !!flags.json }); break; } diff --git a/apps/cli/src/paths.ts b/apps/cli/src/paths.ts index a1314fc4..ff3bc014 100644 --- a/apps/cli/src/paths.ts +++ b/apps/cli/src/paths.ts @@ -42,6 +42,12 @@ export const INTERNAL_DIR = '.shannon'; */ export const FINAL_REPORT_PDF_FILENAME = 'Security-Assessment-Report.pdf'; +/** + * Customer-facing Markdown report name at the run root. + * Must match FINAL_REPORT_MD_FILENAME in the worker package. + */ +export const FINAL_REPORT_MD_FILENAME = 'Security-Assessment-Report.md'; + /** * Resolve a run-directory file (e.g. session.json, workflow.log), preferring the * current INTERNAL_DIR location and falling back to the legacy run-root location diff --git a/apps/cli/src/scan/derive.ts b/apps/cli/src/scan/derive.ts index c3de3f2e..0c1007e5 100644 --- a/apps/cli/src/scan/derive.ts +++ b/apps/cli/src/scan/derive.ts @@ -8,8 +8,17 @@ */ import type { RunningAgent } from '../temporal-client.js'; -import { agentClass, PIPELINE, type PipelineState } from './pipeline.js'; +import { + AGENTIC_SAST_STAGE_ORDER, + agentClass, + isModelBackedOperation, + type OperationalStageState, + operationFamilyKey, + type PipelineState, + pipelineForState, +} from './pipeline.js'; import type { RenderInput } from './render.js'; +import { safeFailureDetail, safeOperationKey, safeOperationLabel } from './safe-fields.js'; export type RunState = 'pending' | 'running' | 'completed' | 'failed' | 'skipped'; @@ -22,14 +31,33 @@ export interface DerivedAgent { readonly durationMs: number | null; readonly runningElapsedMs: number | null; readonly attempt: number | null; + /** The step a running operation row is currently on, merged in from its child activity. */ + readonly detail?: string; + /** Reconciliation time for this agent's class, rendered as a trailing `+ duration`. + * Reconciliation is model work that produces this agent's inputs, so it is shown + * attached to the agent it feeds rather than as free-floating background work. */ + readonly attachedMs?: number; + /** This class's findings could not be grouped, so each one became its own task. */ + readonly ungrouped?: boolean; readonly error?: string; } +/** How a phase line summarizes itself: its own wall time, or a k/N tally over its children. */ +export type PhaseMetaKind = 'duration' | 'count'; + export interface DerivedPhase { readonly key: string; readonly label: string; - readonly parallel: boolean; + /** Whether the phase renders its agents as sub-rows. Independent of {@link meta}: + * Agentic SAST lists its stages under a duration, exploitation lists its classes under a tally. */ + readonly children: boolean; + readonly meta: PhaseMetaKind; readonly state: RunState; + /** The phase's own span, when the worker records one for the phase rather than for a single + * agent inside it (Agentic SAST). The phase line presents this exactly like an agent row. */ + readonly summary?: DerivedAgent; + /** Rendered after the phase's summary, e.g. to mark work that overlaps other phases. */ + readonly note?: string; readonly agents: readonly DerivedAgent[]; } @@ -48,12 +76,12 @@ function isAgentActive(name: string, state: PipelineState | null, running: Set, resolved: boolean): RunState { if (running.has(name)) return 'running'; @@ -62,13 +90,16 @@ function agentState(name: string, state: PipelineState | null, running: Set): string | undefined { const failed = state?.failedPipelines.find((f) => f.vulnType === agentClass(name)); - return ( - failed?.error ?? - byAgent.get(name)?.lastFailure ?? - (state?.failedAgent === name ? (state.error ?? undefined) : undefined) - ); + const hasFailure = + failed !== undefined || byAgent.get(name)?.lastFailure !== undefined || state?.failedAgent === name; + return safeFailureDetail(hasFailure); } /** Scan wall-clock elapsed ms: recorded duration for a closed scan, live elapsed for a running one. */ @@ -99,16 +130,17 @@ export function phaseGlyphState(states: readonly RunState[]): RunState { * class had anything to exploit), not still pending. */ export function deriveAgentStates(input: RenderInput): Map { - const runningSet = new Set(input.running.map((r) => r.agent)); + const pipeline = pipelineForState(input.state); + const runningSet = new Set(input.running.filter((runner) => runner.kind === 'agent').map((runner) => runner.agent)); const terminal = isTerminal(input.temporalStatus); let frontier = -1; - PIPELINE.forEach((phase, idx) => { + pipeline.forEach((phase, idx) => { if (phase.agents.some((a) => isAgentActive(a.name, input.state, runningSet))) frontier = idx; }); const states = new Map(); - for (const [phaseIdx, phase] of PIPELINE.entries()) { + for (const [phaseIdx, phase] of pipeline.entries()) { const resolved = terminal || phaseIdx < frontier; for (const agent of phase.agents) { states.set(agent.name, agentState(agent.name, input.state, runningSet, resolved)); @@ -117,6 +149,52 @@ export function deriveAgentStates(input: RenderInput): Map { return states; } +/** Which operation families have a running parent stage, and the step to show on it. */ +interface OperationFamilyView { + /** Families whose parent stage row already represents their child activities. */ + readonly runningFamilies: ReadonlySet; + /** Family to current step, present only where the child activities agree on one. */ + readonly stepByFamily: ReadonlyMap; +} + +/** + * Resolve the parent stage rows that own their family's child activities. A family only + * resolves to a step when its running children agree: several classes reconcile at once and + * their pending activities carry no class, so a family caught mid-stride shows its parent + * rows without a step rather than attributing one to the wrong class. + */ +function operationFamilyView( + running: readonly RunningAgent[], + persistedOperations: readonly OperationalStageState[], +): OperationFamilyView { + const runningFamilies = new Set( + persistedOperations + .filter((operation) => operation.status === 'running') + .map((operation) => operationFamilyKey(operation.key)), + ); + + const labelsByFamily = new Map>(); + for (const runner of running) { + if (runner.kind !== 'operation' || runner.parentKey === undefined) continue; + if (!runningFamilies.has(runner.parentKey)) continue; + const labels = labelsByFamily.get(runner.parentKey) ?? new Set(); + labels.add(runner.label); + labelsByFamily.set(runner.parentKey, labels); + } + + const stepByFamily = new Map(); + for (const [family, labels] of labelsByFamily) { + const [onlyLabel] = labels; + if (labels.size === 1 && onlyLabel !== undefined) stepByFamily.set(family, lowercaseFirst(onlyLabel)); + } + return { runningFamilies, stepByFamily }; +} + +/** Progress labels are written to start a row; as a detail they continue a sentence. */ +function lowercaseFirst(label: string): string { + return label.charAt(0).toLowerCase() + label.slice(1); +} + /** * Full structured view of the pipeline: every agent's state plus the raw * metrics/timing needed to present it, and each phase's collapsed state. @@ -124,8 +202,9 @@ export function deriveAgentStates(input: RenderInput): Map { export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] { const states = deriveAgentStates(input); const byAgent = new Map(input.running.map((r) => [r.agent, r])); + const pipeline = pipelineForState(input.state); - return PIPELINE.map((phase) => { + const agentPhases = pipeline.map((phase) => { const agents = phase.agents.map((a): DerivedAgent => { const state = states.get(a.name) ?? 'pending'; const metrics = input.state?.agentMetrics[a.name]; @@ -145,11 +224,181 @@ export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] return { key: phase.key, label: phase.label, - parallel: phase.parallel, + children: phase.parallel, + meta: phase.parallel ? ('count' as const) : ('duration' as const), state: phaseGlyphState(agents.map((ag) => ag.state)), agents, }; }); + + // Operational rows merge two sources: stages the worker has persisted (durable truth, + // including terminal outcomes) and pending activities whose stage record has not landed + // yet. Persisted keys win, so a stage is never listed twice while the two views overlap. + const persistedOperations = Object.values(input.state?.operationalStages ?? {}); + const persistedKeys = new Set(persistedOperations.map((operation) => operation.key)); + const { runningFamilies, stepByFamily } = operationFamilyView(input.running, persistedOperations); + const unpersistedRunning = input.running + .filter((runner) => runner.kind === 'operation' && !persistedKeys.has(runner.agent)) + // A child activity whose family already has a running parent stage is that stage's current + // step, not separate work: the parent row below represents it, with the step as its detail + // where the family's children agree on one. Without such a parent it keeps its own row. + .filter((runner) => runner.parentKey === undefined || !runningFamilies.has(runner.parentKey)) + .map((runner) => ({ + key: runner.agent, + label: runner.label, + status: 'running' as const, + ...(runner.startedAt !== undefined && { startedAt: runner.startedAt }), + ...(runner.lastFailure !== undefined && { error: safeFailureDetail(true) }), + })); + const operationalAgents: DerivedAgent[] = [...persistedOperations, ...unpersistedRunning].map((operation) => { + const runner = byAgent.get(operation.key); + const operationState = operation.status as RunState; + const persistedDurationMs = 'durationMs' in operation ? (operation.durationMs ?? null) : null; + const detail = operationState === 'running' ? stepByFamily.get(operationFamilyKey(operation.key)) : undefined; + return { + name: safeOperationKey(operation.key), + label: safeOperationLabel(operation.label), + state: operationState, + durationMs: operationState === 'completed' ? persistedDurationMs : null, + runningElapsedMs: + operationState === 'running' && operation.startedAt !== undefined ? now - operation.startedAt : null, + attempt: operationState === 'running' ? (runner?.attempt ?? null) : null, + ...(detail !== undefined && { detail }), + ...(operation.error !== undefined && { error: safeFailureDetail(true) }), + }; + }); + + // Operational rows are not peers of the agents. Each one is either model work that + // belongs to an agent (reconciliation), model work that belongs to the SAST engine + // (its stages), or bookkeeping that only earns a row when it is stuck or broken. + return assemblePhases(agentPhases, operationalAgents); +} + +/** Reconciliation wall time per vulnerability class, plus the classes whose grouping degraded. */ +interface ReconciliationView { + readonly durationByClass: ReadonlyMap; + readonly ungroupedClasses: ReadonlySet; +} + +function reconciliationView(operations: readonly DerivedAgent[]): ReconciliationView { + const durationByClass = new Map(); + const ungroupedClasses = new Set(); + for (const operation of operations) { + if (operationFamilyKey(operation.name) !== 'reconciliation') continue; + const [, vulnerabilityClass] = operation.name.split(':'); + if (vulnerabilityClass === undefined) continue; + if (operation.name.endsWith(':fallback')) { + ungroupedClasses.add(vulnerabilityClass); + continue; + } + if (operation.durationMs !== null) durationByClass.set(vulnerabilityClass, operation.durationMs); + } + return { durationByClass, ungroupedClasses }; +} + +/** Attach each class's reconciliation time to the agent row it feeds. */ +function withReconciliation(phase: DerivedPhase, view: ReconciliationView): DerivedPhase { + const agents = phase.agents.map((agent): DerivedAgent => { + const vulnerabilityClass = agentClass(agent.name); + const attachedMs = view.durationByClass.get(vulnerabilityClass); + const ungrouped = view.ungroupedClasses.has(vulnerabilityClass); + return { + ...agent, + ...(attachedMs !== undefined && { attachedMs }), + ...(ungrouped && { ungrouped }), + }; + }); + return { ...phase, agents }; +} + +/** + * Build the Agentic SAST phase from the aggregate span the parent workflow records and the + * per-stage rows the SAST child signals up. Scans that predate stage signalling have the + * aggregate but no stages, and render as a bare phase line rather than an error. + */ +function agenticSastPhase(operations: readonly DerivedAgent[]): DerivedPhase | undefined { + const aggregate = operations.find((operation) => operation.name === 'agentic-sast'); + if (aggregate === undefined) return undefined; + + const byStage = new Map(); + for (const operation of operations) { + const [family, stage] = operation.name.split(':'); + if (family !== 'agentic-sast' || stage === undefined) continue; + // The worker's label is the scan log's Title Case form. These rows sit beside the + // lowercase class rows below them, so they read in the same register here. + byStage.set(stage, { ...operation, label: lowercaseFirst(operation.label) }); + } + // Run order, not insertion order: a resumed or replayed run can persist stages out of order. + const stages = AGENTIC_SAST_STAGE_ORDER.map((stage) => byStage.get(stage)).filter( + (stage): stage is DerivedAgent => stage !== undefined, + ); + + return { + key: 'agentic-sast', + label: 'Agentic SAST', + children: stages.length > 0, + meta: 'duration', + state: aggregate.state, + summary: aggregate, + // It shares wall time with the pentest phases below it, so the times do not add up + // in sequence. Saying so is cheaper than a layout that pretends to be two columns. + note: 'concurrent', + agents: stages, + }; +} + +/** + * Bookkeeping rows worth showing. A deterministic stage that has completed says nothing — + * it can only ever read 0s — but one that is still running, or that failed, is exactly what + * an operator needs to see, so those keep a row under the phase they belong to. + */ +function troubledReportSteps(operations: readonly DerivedAgent[]): readonly DerivedAgent[] { + return operations.filter((operation) => { + if (isModelBackedOperation(operation.name)) return false; + if (operationFamilyKey(operation.name) !== 'report') return false; + return operation.state === 'running' || operation.state === 'failed'; + }); +} + +/** + * Fold operational rows into the agent phases. Nothing here becomes a bucket of its own: + * every surviving row is either a SAST stage, time attached to an agent, or a report step + * that is currently in trouble. + */ +function assemblePhases(agentPhases: readonly DerivedPhase[], operations: readonly DerivedAgent[]): DerivedPhase[] { + const view = reconciliationView(operations); + // Reconciliation produces the exploitation queue, so its time belongs on the exploitation + // row it feeds. With exploitation off there is no such row, and it falls back to the + // analysis row for the same class so the time is never silently dropped. + const attachTo = agentPhases.some((phase) => phase.key === 'exploitation') + ? 'exploitation' + : 'vulnerability-analysis'; + const reportSteps = troubledReportSteps(operations); + + const phases = agentPhases.map((phase) => { + if (phase.key === attachTo) return withReconciliation(phase, view); + if (phase.key === 'reporting' && reportSteps.length > 0) { + // The report agent stays on the phase line it already titles; the steps in trouble + // become its children, so nothing is listed twice. + const summary = phase.agents[0]; + return { + ...phase, + children: true, + ...(summary !== undefined && { summary }), + state: phaseGlyphState([...phase.agents, ...reportSteps].map((row) => row.state)), + agents: reportSteps, + }; + } + return phase; + }); + + const sast = agenticSastPhase(operations); + if (sast === undefined) return phases; + + // Agentic SAST starts with the scan and runs alongside the pentest, so it reads after + // the login check rather than appended past Reporting where it never ran. + const afterAuth = phases.findIndex((phase) => phase.key === 'auth-validation') + 1; + return [...phases.slice(0, afterAuth), sast, ...phases.slice(afterAuth)]; } export { agentError }; diff --git a/apps/cli/src/scan/pipeline.ts b/apps/cli/src/scan/pipeline.ts index fe877ede..52ad772e 100644 --- a/apps/cli/src/scan/pipeline.ts +++ b/apps/cli/src/scan/pipeline.ts @@ -8,6 +8,7 @@ * - apps/worker/src/temporal/activities.ts (the run*Agent activity names → `activityType`) * - apps/worker/src/temporal/shared.ts (PipelineState / PipelineSummary) * - apps/worker/src/types/metrics.ts (AgentMetrics) + * - apps/worker/src/types/run-state.ts (PartialReasonView) */ export interface AgentSpec { @@ -26,6 +27,18 @@ export interface PhaseSpec { readonly agents: readonly AgentSpec[]; } +export interface ActivityProgressSpec { + readonly key: string; + readonly label: string; + readonly kind: 'agent' | 'operation'; + /** + * Operation rows whose work is already represented by a persisted parent stage. The parent + * owns the row; this activity supplies the step shown as its detail. Parent stage keys are + * the family key itself or the family key followed by ':' and a class or stage suffix. + */ + readonly parentKey?: string; +} + /** The pipeline phases in execution order, each with its agents. */ export const PIPELINE: readonly PhaseSpec[] = [ { @@ -80,9 +93,183 @@ export const PIPELINE: readonly PhaseSpec[] = [ }, ]; -/** Temporal activity type name → canonical agent name, for mapping pendingActivities. */ +const MISCELLANEOUS_EXPLOIT_AGENT: AgentSpec = { + name: 'miscellaneous-exploit', + label: 'miscellaneous', + activityType: 'runMiscellaneousExploitAgent', +}; + +/** + * Shape the static PIPELINE to one scan's durable truth. expectedAgents, persisted by the + * worker at scan start, names every exploit agent the scan can ever run: exploit rows it + * excludes are dropped, 'miscellaneous-exploit' is appended only once the miscellaneous pipeline has + * admitted findings, and a phase left with no agents disappears entirely. Without state + * (the scan has not initialized durable state yet) the full static pipeline is the best + * available guess. + */ +export function pipelineForState(state: PipelineState | null): readonly PhaseSpec[] { + if (state?.expectedAgents === undefined) return PIPELINE; + const expected = new Set(state.expectedAgents); + return PIPELINE.map((phase) => { + if (phase.key !== 'exploitation') return phase; + const agents = phase.agents.filter((agent) => expected.has(agent.name)); + if (expected.has(MISCELLANEOUS_EXPLOIT_AGENT.name)) agents.push(MISCELLANEOUS_EXPLOIT_AGENT); + return { ...phase, agents }; + }).filter((phase) => phase.agents.length > 0); +} + +const AGENT_ACTIVITY_PROGRESS: Readonly> = Object.fromEntries( + [...PIPELINE.flatMap((phase) => phase.agents), MISCELLANEOUS_EXPLOIT_AGENT].map((agent) => [ + agent.activityType, + { key: agent.name, label: agent.label, kind: 'agent' }, + ]), +); + +/** Families whose per-class or per-stage work is already carried by one persisted stage row. */ +const RECONCILIATION_PARENT_KEY = 'reconciliation'; +const AGENTIC_SAST_PARENT_KEY = 'agentic-sast'; + +// Every production activity that is not an agent run must have a row here. describeScan +// throws on an unmapped activity type, so adding a worker activity without updating this +// table breaks `shannon status` loudly instead of hiding the new work. The authoritative +// name lists live in apps/worker/src/temporal/worker.ts, +// apps/worker/src/temporal/reconcile-activity-types.ts, and +// apps/worker/src/ai/sast/capella/temporal/activity-types.ts. +const OPERATION_ACTIVITY_PROGRESS: Readonly> = { + runPreflightValidation: { key: 'preflight', label: 'Preflight validation', kind: 'operation' }, + syncPlaywrightStealthConfig: { key: 'preflight', label: 'Browser setup', kind: 'operation' }, + initDeliverableGit: { key: 'scan-initialization', label: 'Initialize deliverables', kind: 'operation' }, + syncCodePathDenyRules: { key: 'scan-initialization', label: 'Apply source rules', kind: 'operation' }, + initializeDurableScanState: { key: 'durable-state', label: 'Saving scan state', kind: 'operation' }, + persistMiscellaneousOutcome: { + key: 'miscellaneous-pipeline', + label: 'Including miscellaneous findings', + kind: 'operation', + }, + initializeReportProgress: { key: 'report:initialize', label: 'Initialize report state', kind: 'operation' }, + renumberClassFindings: { key: 'report:renumber', label: 'Renumber findings', kind: 'operation' }, + assembleReportActivity: { key: 'report:assemble', label: 'Assemble report inputs', kind: 'operation' }, + compactReportFindings: { key: 'report:compact', label: 'Compact report findings', kind: 'operation' }, + persistCanonicalReportProgress: { key: 'report:checkpoint', label: 'Saving report progress', kind: 'operation' }, + finalizeReportOutputs: { key: 'report:finalize', label: 'Finalize report outputs', kind: 'operation' }, + persistFinalizedReportProgress: { key: 'report:terminal', label: 'Saving final report state', kind: 'operation' }, + surfaceReportOutputs: { key: 'report:surface', label: 'Surface customer report', kind: 'operation' }, + checkExploitationQueue: { key: 'queue-check', label: 'Check exploitation queue', kind: 'operation' }, + loadResumeState: { key: 'resume-validation', label: 'Validate resume state', kind: 'operation' }, + restoreGitCheckpoint: { key: 'resume-restore', label: 'Restore checkpoint', kind: 'operation' }, + registerResumeAttempt: { key: 'resume-registration', label: 'Register resume', kind: 'operation' }, + recordResumeAttempt: { key: 'resume-registration', label: 'Record resume', kind: 'operation' }, + logPhaseTransition: { key: 'audit-log', label: 'Update audit log', kind: 'operation' }, + logWorkflowComplete: { key: 'audit-log', label: 'Finalize audit log', kind: 'operation' }, + saveCheckpoint: { key: 'checkpoint', label: 'Save checkpoint', kind: 'operation' }, + seedEmptyProducerQueue: { + key: 'miscellaneous-pipeline', + label: 'Preparing miscellaneous findings', + kind: 'operation', + }, + prepareClassReconciliation: { + key: 'reconciliation', + label: 'Preparing findings', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + enrichClassSastObservations: { + key: 'reconciliation', + label: 'Adding code context', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + formClassExploitTasks: { + key: 'reconciliation', + label: 'Grouping into test cases', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + materializeClassExploitTasks: { + key: 'reconciliation', + label: 'Writing test cases', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + publishClassReconciliationOss: { + key: 'reconciliation', + label: 'Saving results', + kind: 'operation', + parentKey: RECONCILIATION_PARENT_KEY, + }, + capellaArchitecture: { + key: 'agentic-sast:architecture', + label: 'Mapping architecture', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaThreatModel: { + key: 'agentic-sast:threat-model', + label: 'Modelling threats', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaPlan: { + key: 'agentic-sast:plan', + label: 'Planning the review', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaResearch: { + key: 'agentic-sast:research', + label: 'Researching code', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaDedupe: { + key: 'agentic-sast:dedupe', + label: 'Merging duplicates', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaReview: { + key: 'agentic-sast:review', + label: 'Reviewing findings', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaCritic: { + key: 'agentic-sast:critic', + label: 'Critiquing findings', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaConfirm: { + key: 'agentic-sast:confirm', + label: 'Confirming findings', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaCalibrate: { + key: 'agentic-sast:calibrate', + label: 'Calibrating risk', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, + capellaExport: { + key: 'agentic-sast:export', + label: 'Exporting findings', + kind: 'operation', + parentKey: AGENTIC_SAST_PARENT_KEY, + }, +}; + +/** Complete production activity mirror. Unknown names are errors, never hidden progress. */ +export const ACTIVITY_TO_PROGRESS: Readonly> = Object.freeze({ + ...AGENT_ACTIVITY_PROGRESS, + ...OPERATION_ACTIVITY_PROGRESS, +}); + +/** Agent-only projection of ACTIVITY_TO_PROGRESS: activity type name to canonical agent name. */ export const ACTIVITY_TO_AGENT: Readonly> = Object.fromEntries( - PIPELINE.flatMap((phase) => phase.agents.map((agent) => [agent.activityType, agent.name])), + Object.entries(ACTIVITY_TO_PROGRESS) + .filter(([, progress]) => progress.kind === 'agent') + .map(([activityType, progress]) => [activityType, progress.key]), ); /** The vuln/exploit class of an agent (e.g. "authz-vuln" → "authz"), for failedPipelines matching. */ @@ -100,11 +287,65 @@ export interface AgentMetrics { readonly skipped?: boolean; } +export interface OperationalStageState { + readonly key: string; + readonly label: string; + readonly status: 'pending' | 'running' | 'completed' | 'failed' | 'skipped'; + readonly startedAt?: number; + readonly durationMs?: number; + readonly error?: string; +} + +/** Family key a persisted operational stage belongs to, e.g. `reconciliation:xss` to `reconciliation`. */ +export function operationFamilyKey(stageKey: string): string { + const separator = stageKey.indexOf(':'); + return separator === -1 ? stageKey : stageKey.slice(0, separator); +} + +/** The Capella stages that get a progress row, in run order. Mirrors CAPELLA_PROGRESS_STAGES + * in apps/worker/src/ai/sast/types.ts — the deterministic `export` stage is not among them. */ +export const AGENTIC_SAST_STAGE_ORDER: readonly string[] = [ + 'architecture', + 'threat-model', + 'plan', + 'research', + 'dedupe', + 'review', + 'critic', + 'confirm', + 'calibrate', +]; + +/** + * Whether an operational stage represents model work rather than bookkeeping. + * + * Only the agentic-SAST stages and per-class reconciliation run a model; every other + * operational stage is a git commit or a durable-state write that can only ever record + * sub-second wall time. The progress tree shows model work, so this is what decides + * whether a stage is worth a row at all. + */ +export function isModelBackedOperation(stageKey: string): boolean { + const family = operationFamilyKey(stageKey); + if (family === 'agentic-sast') return true; + // A `reconciliation::fallback` marker records a degradation, not a model span. + return family === 'reconciliation' && !stageKey.endsWith(':fallback'); +} + export interface PipelineSummary { readonly totalCostUsd: number; readonly totalDurationMs: number; // Wall-clock (end - start) readonly totalTurns: number; readonly agentCount: number; + /** False when operational (Capella/reconciliation) spend is known to be incomplete. */ + readonly usageAccountingComplete?: boolean; +} + +/** One durable degradation reason with its derived safe message (mirror of PartialReasonView). */ +export interface PartialReasonView { + readonly code: string; + readonly vulnerabilityClass?: string; + readonly stage?: string; + readonly message: string; } export type PipelineStatus = 'running' | 'completed' | 'failed' | 'cancelled' | 'partial'; @@ -114,10 +355,29 @@ export interface PipelineState { readonly currentPhase: string | null; readonly currentAgent: string | null; readonly completedAgents: string[]; + readonly expectedAgents?: string[]; + readonly participatingClasses?: string[]; readonly failedPipelines: { vulnType: string; error: string }[]; + readonly failedReconciliations?: { vulnerabilityClass: string; error: string }[]; readonly failedAgent: string | null; readonly error: string | null; readonly startTime: number; readonly agentMetrics: Record; + readonly operationalMetrics?: Record; + readonly operationalStages?: Record; + /** `error` is the worker's sanitized failure sentence, safe to print verbatim. */ + readonly agenticSast?: { + readonly status: string; + readonly durationMs?: number; + /** Reader-facing name of the failed stage, already projected by the worker. */ + readonly failedStageLabel?: string; + readonly error?: string; + readonly errorCode?: string; + /** Usage-accounting warnings projected by the worker; empty when the ledger reconciled. */ + readonly warnings?: readonly string[]; + }; + readonly nonFatalFailures?: { readonly phase: string; readonly error: string }[]; + /** Ordered durable degradation reasons with safe messages; empty or absent for full success. */ + readonly partialReasons?: readonly PartialReasonView[]; readonly summary: PipelineSummary | null; } diff --git a/apps/cli/src/scan/render.ts b/apps/cli/src/scan/render.ts index 7e3ac62e..6aaceeff 100644 --- a/apps/cli/src/scan/render.ts +++ b/apps/cli/src/scan/render.ts @@ -10,9 +10,9 @@ import { BOLD, DIM, GOLD, paint, RED, YELLOW } from '../colors.js'; import { commandPrefix } from '../mode.js'; import type { RunningAgent } from '../temporal-client.js'; -import { agentError, deriveAgentStates, isTerminal, phaseGlyphState, type RunState, scanElapsedMs } from './derive.js'; -import { inlineFailureReason } from './failure.js'; -import { PIPELINE, type PipelineState } from './pipeline.js'; +import { derivePipeline, isTerminal, type RunState, scanElapsedMs } from './derive.js'; +import type { PipelineState } from './pipeline.js'; +import { safeAgenticSast, safeCliIdentifier, safePartialReasons, safeTerminalFailure } from './safe-fields.js'; export interface RenderInput { readonly workspace: string; @@ -68,7 +68,7 @@ function truncate(text: string, max: number): string { /** Temporal Web UI, published by compose on 8233; deep-links to the workflow when its id is known. */ function temporalDashboardUrl(workflowId: string | undefined): string { const base = 'http://localhost:8233'; - return workflowId ? `${base}/namespaces/default/workflows/${workflowId}` : base; + return workflowId ? `${base}/namespaces/default/workflows/${safeCliIdentifier(workflowId)}` : base; } // === Glyphs & status === @@ -95,6 +95,12 @@ const STATE_COLOR: Record = { skipped: COLORS.dim, }; +/** Column width for an agent or background-work label inside a phase. */ +const AGENT_LABEL_WIDTH = 18; + +/** Inline budget for a failure sentence, wide enough to carry a whole first sentence. */ +const FAILURE_DETAIL_WIDTH = 120; + /** Braille spinner frames for running agents — the clack loader style. */ const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'] as const; @@ -112,42 +118,68 @@ function statusBadge(input: RenderInput, opts: RenderOptions): string { const workflowStatus = input.state?.status; if (!isTerminal(input.temporalStatus)) return paint('running', COLORS.gold, opts.color); if (workflowStatus === 'partial') return paint('partial', COLORS.yellow, opts.color); + if (workflowStatus === 'cancelled') return paint('cancelled', COLORS.yellow, opts.color); if (input.temporalStatus === 'COMPLETED') return paint('completed', COLORS.gold, opts.color); if (input.temporalStatus === 'TERMINATED') return paint('stopped', COLORS.yellow, opts.color); if (input.temporalStatus === 'CANCELLED' || input.temporalStatus === 'CANCELED') { return paint('cancelled', COLORS.yellow, opts.color); } if (input.temporalStatus === 'TIMED_OUT') return paint('timed out', COLORS.red, opts.color); - return paint('FAILED', COLORS.red, opts.color); + return paint('failed', COLORS.red, opts.color); } // === Line builders === +/** The parts of a derived row agentMeta reads beyond its state and metrics. */ +interface RowExtras { + readonly runningElapsedMs?: number | null; + readonly attachedMs?: number; + readonly ungrouped?: boolean; +} + function agentMeta( state: RunState, metrics: { durationMs: number } | undefined, runner: RunningAgent | undefined, error: string | undefined, opts: RenderOptions, + step?: string, + extras?: RowExtras, ): string { if (state === 'completed') { const duration = metrics?.durationMs != null ? formatDuration(metrics.durationMs) : 'done'; - return paint(duration, COLORS.dim, opts.color); + return paint(`${duration}${attachedSuffix(extras)}`, COLORS.dim, opts.color); } if (state === 'running') { const parts = ['running']; - if (runner?.startedAt !== undefined) parts.push(formatDuration(opts.now - runner.startedAt)); + if (step !== undefined) parts.push(step); + // An operational row carries its own elapsed time: it is derived from the persisted stage + // span, and has no pending activity on the parent workflow to read a start time from. + const elapsedMs = + runner?.startedAt !== undefined ? opts.now - runner.startedAt : (extras?.runningElapsedMs ?? null); + if (elapsedMs !== null) parts.push(formatDuration(elapsedMs)); if (runner && runner.attempt > 1) parts.push(`retry ${runner.attempt}`); return paint(parts.join(' · '), COLORS.gold, opts.color); } if (state === 'failed') { - const detail = error ? ` · ${truncate(error, 46)}` : ''; + const detail = error ? ` · ${truncate(error, FAILURE_DETAIL_WIDTH)}` : ''; return paint(`failed${detail}`, COLORS.red, opts.color); } if (state === 'skipped') return paint('skipped', COLORS.dim, opts.color); return paint('queued', COLORS.dim, opts.color); } +/** + * Time a reconciliation lane contributed to this agent's class, shown as `+ duration` on the + * row it feeds. `ungrouped` marks a class whose findings could not be grouped, so each one + * was tested separately and duplicates are expected. + */ +function attachedSuffix(extras: RowExtras | undefined): string { + if (extras === undefined) return ''; + const time = extras.attachedMs === undefined ? '' : ` + ${formatDuration(extras.attachedMs)}`; + return extras.ungrouped ? `${time} · ungrouped` : time; +} + function phaseMeta(states: readonly RunState[], inPlay: number, parallel: boolean, opts: RenderOptions): string { if (states.every((s) => s === 'pending')) return paint('pending', COLORS.dim, opts.color); if (states.every((s) => s === 'skipped')) return paint('skipped', COLORS.dim, opts.color); @@ -163,35 +195,43 @@ function phaseMeta(states: readonly RunState[], inPlay: number, parallel: boolea /** Render the full progress frame as one string (no trailing newline). */ export function renderScan(input: RenderInput, opts: RenderOptions): string { const byAgent = new Map(input.running.map((r) => [r.agent, r])); - const stateMap = deriveAgentStates(input); + const phases = derivePipeline(input, opts.now); const lines: string[] = ['', ...headerLines(input, opts), '']; - const metaFor = (name: string, state: RunState): string => - agentMeta(state, input.state?.agentMetrics[name], byAgent.get(name), agentError(name, input.state, byAgent), opts); // Only agents that have actually entered play are shown; pending/skipped ones stay hidden. const inPlay = (s: RunState): boolean => s === 'running' || s === 'completed' || s === 'failed'; - for (const phase of PIPELINE) { - const states = phase.agents.map((a) => stateMap.get(a.name) ?? 'pending'); + for (const phase of phases) { + const states = phase.agents.map((agent) => agent.state); const playing = states.filter(inPlay).length; - const phaseRunState: RunState = phaseGlyphState(states); + const phaseRunState = phase.state; + const metaFor = (agent: (typeof phase.agents)[number]): string => { + const metrics = agent.durationMs === null ? undefined : { durationMs: agent.durationMs }; + return agentMeta(agent.state, metrics, byAgent.get(agent.name), agent.error, opts, agent.detail, agent); + }; - // A single-agent phase carries that agent's own duration/cost on the phase line once it - // starts; a parallel phase gets a "k/N done" summary over the agents in play. + // A phase summarizes itself by wall time or by a "k/N done" tally. A phase with its own + // recorded span (Agentic SAST) presents it like any agent row; otherwise a single-agent + // phase borrows its one agent's duration once that agent starts. const first = phase.agents[0]; const firstState = states[0]; - const phaseMetaStr = - !phase.parallel && first && firstState && inPlay(firstState) - ? metaFor(first.name, firstState) - : phaseMeta(states, playing, phase.parallel, opts); - lines.push(` ${glyph(phaseRunState, opts)} ${phase.label.padEnd(26)}${phaseMetaStr}`); + const borrowed = first && firstState && inPlay(firstState) ? metaFor(first) : undefined; + const durationMeta = phase.summary === undefined ? borrowed : metaFor(phase.summary); + const summaryMeta = + phase.meta === 'duration' && durationMeta !== undefined + ? durationMeta + : phaseMeta(states, playing, phase.meta === 'count', opts); + const note = phase.note === undefined ? '' : paint(` · ${phase.note}`, COLORS.dim, opts.color); + lines.push(` ${glyph(phaseRunState, opts)} ${phase.label.padEnd(26)}${summaryMeta}${note}`); - if (!phase.parallel) continue; + if (!phase.children) continue; for (let i = 0; i < phase.agents.length; i++) { const agent = phase.agents[i]; const state = states[i]; if (!agent || !state || !inPlay(state)) continue; - lines.push(` ${glyph(state, opts)} ${agent.label.padEnd(18)}${metaFor(agent.name, state)}`); + // Two trailing spaces before padding, so a label wider than the column still separates + // from its meta text; a label inside the column pads to the same width as before. + lines.push(` ${glyph(state, opts)} ${`${agent.label} `.padEnd(AGENT_LABEL_WIDTH)}${metaFor(agent)}`); } } @@ -202,7 +242,7 @@ export function renderScan(input: RenderInput, opts: RenderOptions): string { function headerLines(input: RenderInput, opts: RenderOptions): string[] { const elapsedMs = scanElapsedMs(input, opts.now); const meta = [statusBadge(input, opts), elapsedMs !== undefined ? formatDuration(elapsedMs) : '—'].join(' · '); - return [` ${paint('Scan:', COLORS.bold, opts.color)} ${input.workspace.padEnd(22)} ${meta}`]; + return [` ${paint('Scan:', COLORS.bold, opts.color)} ${safeCliIdentifier(input.workspace).padEnd(22)} ${meta}`]; } /** Aligned label column for the footer's Logs / Temporal rows. */ @@ -223,15 +263,46 @@ function footerLines(input: RenderInput, opts: RenderOptions): string[] { if (isTerminal(input.temporalStatus) && input.state?.summary) { const wall = formatDuration(input.state.summary.totalDurationMs); - return ['', ` Time Taken ${wall}`]; + const lines = ['', ` Time Taken ${wall}`]; + + // A partial scan names each durable degradation reason through its safe message, + // so the operator never has to guess why the badge is not "completed". + const reasons = safePartialReasons(input.state.partialReasons ?? []); + if (reasons.length > 0) { + lines.push('', ` ${paint('Why this scan is partial:', COLORS.yellow, opts.color)}`); + for (const reason of reasons) { + lines.push(paint(` - ${reason.message}`, COLORS.dim, opts.color)); + } + // The safe message names what degraded; these three name the agentic-SAST failure + // behind it, under the same labels the scan log and worker output use. + const agenticSast = safeAgenticSast(input.state.agenticSast); + if (agenticSast?.status === 'failed') { + if (agenticSast.failedStageLabel !== undefined) { + lines.push(paint(` Agentic SAST stopped at: ${agenticSast.failedStageLabel}`, COLORS.dim, opts.color)); + } + if (agenticSast.error !== undefined) { + lines.push(paint(` What happened: ${agenticSast.error}`, COLORS.dim, opts.color)); + } + if (agenticSast.errorCode !== undefined) { + lines.push(paint(` Reference code (for a bug report): ${agenticSast.errorCode}`, COLORS.dim, opts.color)); + } + } + } + if (input.state.summary.usageAccountingComplete === false) { + lines.push( + paint(' Cost is incomplete — some background work is not included in this total.', COLORS.dim, opts.color), + ); + } + return lines; } - const logsValue = `${prefix} logs ${input.workspace}`; + const logsValue = `${prefix} logs ${safeCliIdentifier(input.workspace)}`; const temporalValue = temporalDashboardUrl(input.workflowId); if (isTerminal(input.temporalStatus)) { - const rawReason = input.failureMessage ?? input.state?.error; - const reason = rawReason ? inlineFailureReason(rawReason) : 'no result recorded'; + const hasRecordedFailure = + input.failureMessage !== undefined || (input.state !== null && input.state.error !== null); + const reason = safeTerminalFailure(hasRecordedFailure) ?? 'no result recorded'; return [ footerDivider(opts), paint( diff --git a/apps/cli/src/scan/safe-fields.ts b/apps/cli/src/scan/safe-fields.ts new file mode 100644 index 00000000..57b3111a --- /dev/null +++ b/apps/cli/src/scan/safe-fields.ts @@ -0,0 +1,278 @@ +/** + * Closed-field projection for Temporal values displayed by the CLI. + * + * PipelineState travels through Temporal from a worker container this process does not + * control, so free-text fields are treated as unvetted: this module either matches a + * value against a known closed set (safe to print as-is) or collapses it to a fixed, + * bounded message. A value with no case here should fail closed to something generic, + * never pass through untouched. + */ + +import type { PartialReasonView, PipelineState } from './pipeline.js'; + +const CLASS_NAMES: Readonly> = Object.freeze({ + injection: 'Injection', + xss: 'Cross-Site Scripting', + auth: 'Authentication', + authz: 'Authorization', + ssrf: 'Server-Side Request Forgery', + miscellaneous: 'Miscellaneous', +}); + +const STAGE_NAMES: Readonly> = Object.freeze({ + architecture: 'architecture mapping', + 'threat-model': 'threat modelling', + plan: 'review planning', + research: 'deep code research', + dedupe: 'duplicate merging', + review: 'independent review', + critic: 'viability critique', + confirm: 'static confirmation', + calibrate: 'risk calibration', + export: 'findings export', + workflow: 'orchestration', +}); + +const TERMINAL_STAGE_NAMES = new Set([ + 'architecture', + 'threat model', + 'planning', + 'audit wave', + 'deduplication', + 'review', + 'critic', + 'confirmation', + 'calibration', + 'export', + 'orchestration', +]); + +const CAPELLA_FAILURE_MESSAGES = new Set([ + 'Provider authentication failed. Verify the configured credential.', + 'Agentic SAST configuration is invalid.', + 'Agentic SAST received invalid input.', + 'An agentic SAST step returned an unusable result.', + 'An agentic SAST step failed.', + 'Agentic SAST infrastructure failed before producing a usable result.', + 'Agentic SAST had not finished when the scan stopped.', +]); + +// Mirrors apps/worker/src/types/errors.ts. The CLI cannot import from the worker package, +// so keep this exact closed set in sync with ProviderFailureCategory. +const PROVIDER_FAILURE_CATEGORIES = new Set([ + 'rate_limit', + 'overloaded', + 'transport', + 'context_limit', + 'quota', + 'authentication', + 'configuration', + 'unknown', +]); + +function isProviderFailureCategory(value: unknown): value is string { + return typeof value === 'string' && PROVIDER_FAILURE_CATEGORIES.has(value); +} + +const OPERATION_LABELS = new Set([ + 'Agentic SAST', + // Capella stage rows, signalled up from the SAST child workflow. Mirrors + // CAPELLA_STAGE_LABELS in apps/worker/src/ai/sast/types.ts, minus the deterministic + // export stage, which never becomes a row. + 'Architecture', + 'Threat model', + 'Plan', + 'Research', + 'Dedupe', + 'Review', + 'Critique', + 'Confirm', + 'Calibrate', + 'Reconcile injection', + 'Reconcile xss', + 'Reconcile auth', + 'Reconcile authz', + 'Reconcile ssrf', + 'Reconcile miscellaneous', + 'Prepare reconciliation', + 'Enrich observations', + 'Form exploit tasks', + 'Materialize exploit tasks', + 'Publish reconciliation', + 'Renumber injection', + 'Renumber xss', + 'Renumber auth', + 'Renumber authz', + 'Renumber ssrf', + 'Renumber miscellaneous', + 'Initialize report state', + 'Assemble report inputs', + 'Compact report findings', + 'Saving report progress', + 'Finalize report outputs', + 'Finalize report without SARIF', + 'Saving final report state', + 'Surface customer report', +]); + +function safeClassName(value: string | undefined): string | undefined { + return value === undefined ? undefined : CLASS_NAMES[value]; +} + +function safeStageName(value: string | undefined): string | undefined { + return value === undefined ? undefined : STAGE_NAMES[value]; +} + +function reasonMessage(reason: PartialReasonView): string | undefined { + const className = safeClassName(reason.vulnerabilityClass); + switch (reason.code) { + case 'agentic_sast_failed': { + const stageName = safeStageName(reason.stage); + return stageName === undefined + ? 'Agentic SAST failed, so the pentest continued without its findings.' + : `Agentic SAST failed during ${stageName}, so the pentest continued without its findings.`; + } + case 'agentic_sast_reduced': + return 'Agentic SAST completed with reduced coverage.'; + case 'class_pipeline_failed': + return className === undefined + ? undefined + : `${className} could not be fully assessed. The other classes completed. Re-running this workspace retries only the part that failed.`; + case 'class_reconciliation_failed': + return className === undefined + ? undefined + : `${className} findings could not be grouped into test cases, so that class was not exploited. Its analysis results are still in the report.`; + case 'report_renumber_failed': + return className === undefined + ? undefined + : `${className} findings kept their working reference numbers, so numbering in the report may have gaps. The findings themselves are complete.`; + case 'report_compaction_failed': + return 'Finding reference numbers in the report may have gaps. Every finding is present; only the numbering is affected.'; + case 'report_class_omitted': + return className === undefined + ? undefined + : `${className} was assessed but could not be included in the final report.`; + case 'report_sarif_failed': + return 'Report SARIF could not be generated. JSON and Markdown remain available.'; + default: + return undefined; + } +} + +export function safePartialReasons(reasons: readonly PartialReasonView[]): readonly PartialReasonView[] { + return reasons.flatMap((reason) => { + const message = reasonMessage(reason); + if (message === undefined) return []; + const vulnerabilityClass = + safeClassName(reason.vulnerabilityClass) === undefined ? undefined : reason.vulnerabilityClass; + const stage = safeStageName(reason.stage) === undefined ? undefined : reason.stage; + return [ + { + code: reason.code, + message, + ...(vulnerabilityClass !== undefined && { vulnerabilityClass }), + ...(stage !== undefined && { stage }), + }, + ]; + }); +} + +/** Upper bounds on the warning array crossing into cli.status.json, so a malformed state cannot bloat it. */ +const MAX_AGENTIC_SAST_WARNINGS = 20; +const MAX_AGENTIC_SAST_WARNING_LENGTH = 2_000; + +/** Sanitize the worker's usage-accounting warnings: strings only, bounded count and length. */ +function safeAgenticSastWarnings(value: PipelineState['agenticSast']): readonly string[] { + const warnings = value?.warnings; + if (!Array.isArray(warnings)) return []; + return warnings + .filter((warning): warning is string => typeof warning === 'string') + .slice(0, MAX_AGENTIC_SAST_WARNINGS) + .map((warning) => warning.slice(0, MAX_AGENTIC_SAST_WARNING_LENGTH)); +} + +export function safeAgenticSast(value: PipelineState['agenticSast']): + | { + readonly status: string; + readonly failedStageLabel?: string; + readonly error?: string; + readonly errorCode?: string; + readonly warnings: readonly string[]; + } + | undefined { + if (value === undefined || !['disabled', 'running', 'succeeded', 'failed'].includes(value.status)) return undefined; + const failedStageLabel = TERMINAL_STAGE_NAMES.has(value.failedStageLabel ?? '') ? value.failedStageLabel : undefined; + let error: string | undefined; + if (value.error !== undefined && CAPELLA_FAILURE_MESSAGES.has(value.error)) { + error = value.error; + } else if (value.status === 'failed') { + error = 'An agentic SAST step failed.'; + } + const errorCode = + value.errorCode !== undefined && + (/^[A-Z][A-Z0-9_]{0,63}$/u.test(value.errorCode) || isProviderFailureCategory(value.errorCode)) + ? value.errorCode + : undefined; + return { + status: value.status, + ...(failedStageLabel !== undefined && { failedStageLabel }), + ...(error !== undefined && { error }), + ...(errorCode !== undefined && { errorCode }), + warnings: safeAgenticSastWarnings(value), + }; +} + +export function safeOperationLabel(value: string): string { + return OPERATION_LABELS.has(value) ? value : 'Background task'; +} + +export function safeOperationKey(value: string): string { + if ( + /^(?:agentic-sast|miscellaneous-pipeline|report:(?:initialize|assemble|compact|checkpoint|finalize|finalize-degraded|terminal|surface))$/u.test( + value, + ) || + /^agentic-sast:(?:architecture|threat-model|plan|research|dedupe|review|critic|confirm|calibrate)$/u.test(value) || + /^(?:reconciliation|report:renumber):(?:injection|xss|auth|authz|ssrf|miscellaneous)$/u.test(value) || + /^reconciliation:(?:injection|xss|auth|authz|ssrf|miscellaneous):fallback$/u.test(value) + ) { + return value; + } + return 'background-task'; +} + +/** + * A workspace or workflow id is printed straight into the progress display, so this + * confines it to a plain identifier charset before that happens: no control or escape + * characters survive to reach the terminal. + */ +export function safeCliIdentifier(value: string): string { + return /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/u.test(value) ? value : 'unknown'; +} + +export function safeTemporalStatus(value: string): string { + return [ + 'RUNNING', + 'UNSPECIFIED', + 'COMPLETED', + 'FAILED', + 'CANCELLED', + 'CANCELED', + 'TERMINATED', + 'TIMED_OUT', + 'CONTINUED_AS_NEW', + ].includes(value) + ? value + : 'UNKNOWN'; +} + +export function safeFailureDetail(hasFailure: true): string; +export function safeFailureDetail(hasFailure: false): undefined; +export function safeFailureDetail(hasFailure: boolean): string | undefined; +export function safeFailureDetail(hasFailure: boolean): string | undefined { + return hasFailure ? 'This scan step could not be completed.' : undefined; +} + +/** Same closed-set trade-off as safeFailureDetail, for the scan-level (not per-agent) failure. */ +export function safeTerminalFailure(hasFailure: boolean): string | undefined { + return hasFailure ? 'The scan could not be completed.' : undefined; +} diff --git a/apps/cli/src/scan/status-json.ts b/apps/cli/src/scan/status-json.ts index 4229758a..1616e9e3 100644 --- a/apps/cli/src/scan/status-json.ts +++ b/apps/cli/src/scan/status-json.ts @@ -8,7 +8,15 @@ import type { DerivedPhase } from './derive.js'; import { derivePipeline, isTerminal, scanElapsedMs } from './derive.js'; +import type { PartialReasonView } from './pipeline.js'; import type { RenderInput } from './render.js'; +import { + safeAgenticSast, + safeCliIdentifier, + safePartialReasons, + safeTemporalStatus, + safeTerminalFailure, +} from './safe-fields.js'; /** Coarse scan status token, mirroring the human status badge in machine-friendly form. */ export type ScanStatus = 'running' | 'completed' | 'partial' | 'failed' | 'stopped' | 'cancelled' | 'timed_out'; @@ -27,6 +35,18 @@ export interface StatusJson { readonly endedAt?: string; /** Failure text when a failed scan left no readable state. */ readonly failureMessage?: string; + /** Ordered durable degradation reasons with safe messages; present only when non-empty. */ + readonly partialReasons?: readonly PartialReasonView[]; + /** Agentic SAST outcome, with the worker's sanitized failure sentence and bounded code. */ + readonly agenticSast?: { + readonly status: string; + readonly error?: string; + readonly errorCode?: string; + /** Usage-accounting warnings; always present (empty when the ledger reconciled) so it is never null. */ + readonly warnings: readonly string[]; + }; + /** False when operational (Capella/reconciliation) spend is known to be incomplete. */ + readonly usageAccountingComplete?: boolean; readonly phases: readonly DerivedPhase[]; } @@ -34,6 +54,7 @@ export interface StatusJson { function deriveStatus(input: RenderInput): ScanStatus { if (!isTerminal(input.temporalStatus)) return 'running'; if (input.state?.status === 'partial') return 'partial'; + if (input.state?.status === 'cancelled') return 'cancelled'; switch (input.temporalStatus) { case 'COMPLETED': @@ -53,16 +74,32 @@ function deriveStatus(input: RenderInput): ScanStatus { /** Build the JSON snapshot for a scan at instant `now`. */ export function toStatusJson(input: RenderInput, now: number): StatusJson { const elapsedMs = scanElapsedMs(input, now); + const partialReasons = safePartialReasons(input.state?.partialReasons ?? []); + const agenticSast = safeAgenticSast(input.state?.agenticSast); + const usageAccountingComplete = input.state?.summary?.usageAccountingComplete; + const failureMessage = safeTerminalFailure(input.failureMessage !== undefined); return { - workspace: input.workspace, - ...(input.workflowId !== undefined && { workflowId: input.workflowId }), + workspace: safeCliIdentifier(input.workspace), + ...(input.workflowId !== undefined && { workflowId: safeCliIdentifier(input.workflowId) }), status: deriveStatus(input), - temporalStatus: input.temporalStatus, + temporalStatus: safeTemporalStatus(input.temporalStatus), elapsedMs: elapsedMs ?? null, ...(input.startedAt !== undefined && { startedAt: new Date(input.startedAt).toISOString() }), ...(input.endedAt !== undefined && { endedAt: new Date(input.endedAt).toISOString() }), - ...(input.failureMessage !== undefined && { failureMessage: input.failureMessage }), + ...(failureMessage !== undefined && { failureMessage }), + ...(partialReasons.length > 0 && { partialReasons }), + // Present only when agentic SAST actually ran; a disabled scan omits the key entirely. + ...(agenticSast !== undefined && + agenticSast.status !== 'disabled' && { + agenticSast: { + status: agenticSast.status, + ...(agenticSast.error !== undefined && { error: agenticSast.error }), + ...(agenticSast.errorCode !== undefined && { errorCode: agenticSast.errorCode }), + warnings: [...agenticSast.warnings], + }, + }), + ...(usageAccountingComplete !== undefined && { usageAccountingComplete }), phases: derivePipeline(input, now), }; } diff --git a/apps/cli/src/session.ts b/apps/cli/src/session.ts index cbee1285..87fcace4 100644 --- a/apps/cli/src/session.ts +++ b/apps/cli/src/session.ts @@ -1,11 +1,12 @@ /** * Workspace → Temporal workflow-id resolution. * - * A workspace name is not always its workflow id: a fresh scan's id equals the - * workspace name, but each resume spawns a new workflow (`_resume_`). - * The workspace's session.json records the authoritative id — the latest resume - * attempt, or the original — so commands that query Temporal (status, stop) resolve - * through here instead of assuming the name is the id. + * A workspace name is not always its workflow id: a fresh named workspace gets + * `_shannon-` as its workflow id (only an auto-named workspace's + * directory name equals its original id), and each resume spawns a new workflow + * (`_resume_`). The workspace's session.json records the authoritative + * id — the latest resume attempt, or the original — so commands that query Temporal + * (status, stop) resolve through here instead of assuming the name is the id. */ import fs from 'node:fs'; diff --git a/apps/cli/src/temporal-client.ts b/apps/cli/src/temporal-client.ts index 5dd4e8d7..d98f1ea5 100644 --- a/apps/cli/src/temporal-client.ts +++ b/apps/cli/src/temporal-client.ts @@ -9,7 +9,7 @@ import { setTimeout as sleep } from 'node:timers/promises'; import { Client, Connection, WorkflowFailedError, WorkflowNotFoundError } from '@temporalio/client'; -import { ACTIVITY_TO_AGENT, type PipelineState } from './scan/pipeline.js'; +import { ACTIVITY_TO_PROGRESS, type PipelineState } from './scan/pipeline.js'; const ADDRESS = '127.0.0.1:7233'; const NAMESPACE = 'default'; @@ -20,11 +20,30 @@ const TERMINAL_STATUSES: ReadonlySet = new Set(['COMPLETED', 'FAILED', ' export interface RunningAgent { readonly agent: string; + readonly label: string; + /** 'agent' rows join the static pipeline tree; 'operation' rows feed the background-work phase. */ + readonly kind: 'agent' | 'operation'; + /** Set when a persisted parent stage owns this row; the label then reads as that stage's step. */ + readonly parentKey?: string; readonly attempt: number; readonly startedAt?: number; readonly lastFailure?: string; } +/** + * The CLI's activity mirror does not know an activity type the running scan is using, so the + * progress tree cannot be rendered completely. Distinct from a Temporal connection failure. + */ +export class ActivityMirrorError extends Error { + override name = 'ActivityMirrorError' as const; + + constructor(activityType: string) { + super( + `This version of the Shannon command line does not recognise part of the running scan\n(${activityType}). Update Shannon, or watch the scan with: shannon logs `, + ); + } +} + /** Convert a proto ITimestamp (seconds is a Long) to epoch millis. */ function timestampMs( ts: { seconds?: { toString(): string } | number | null; nanos?: number | null } | null, @@ -66,12 +85,26 @@ export async function describeScan(workflowId: string): Promise= warnAfterFailures) { diff --git a/apps/cli/src/workspaces.ts b/apps/cli/src/workspaces.ts new file mode 100644 index 00000000..a41260cd --- /dev/null +++ b/apps/cli/src/workspaces.ts @@ -0,0 +1,171 @@ +/** + * Workspace enumeration, default-target resolution, and scan identity proof. + * + * The action commands (`logs`, `status`, `stop`) each take a workspace name. When one + * is omitted, `resolveDefaultWorkspace` picks the obvious candidate — the single running + * scan, or the most recent workspace — so the common "I just started one scan, show me + * its logs" path doesn't require retyping an auto-generated name. Target selection and + * identity proof are separate steps: `resolveScanIdentity` turns a selected or explicit + * string into the one canonical (workspace, workflowId) pair the session records prove. + * + * Running scans are identified by Docker label (the authoritative source, shared with + * `stop`); recency for finished scans comes from each run's session.json createdAt, + * with the workspace directory mtime as the fallback for runs that predate it. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import { runningScanWorkspaces } from './docker.js'; +import { getWorkspacesDir } from './home.js'; +import { resolveRunFile } from './paths.js'; +import { resolveWorkflowId } from './session.js'; + +export interface WorkspaceInfo { + readonly name: string; + /** Creation time in ms — the recency sort key. Null when neither session.json nor stat is readable. */ + readonly createdMs: number | null; +} + +/** Creation time of a workspace: session.json createdAt, else directory mtime, else null. */ +function readCreatedMs(runDir: string): number | null { + try { + const parsed = JSON.parse(fs.readFileSync(resolveRunFile(runDir, 'session.json'), 'utf-8')); + const createdMs = Date.parse(parsed?.session?.createdAt ?? ''); + if (!Number.isNaN(createdMs)) { + return createdMs; + } + } catch { + // Fall through to the directory mtime. + } + + try { + return fs.statSync(runDir).mtimeMs; + } catch { + return null; + } +} + +/** Every workspace directory, newest-first by createdAt (directory mtime fallback). */ +export function listWorkspaces(): WorkspaceInfo[] { + const workspacesDir = getWorkspacesDir(); + + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(workspacesDir, { withFileTypes: true }); + } catch { + // Workspaces directory does not exist yet — no scans have ever run. + return []; + } + + const workspaces: WorkspaceInfo[] = []; + for (const entry of entries) { + if (!entry.isDirectory()) { + continue; + } + workspaces.push({ name: entry.name, createdMs: readCreatedMs(path.join(workspacesDir, entry.name)) }); + } + + // Newest first; workspaces with no known time sort last. + workspaces.sort((a, b) => (b.createdMs ?? 0) - (a.createdMs ?? 0)); + return workspaces; +} + +export type ScanIdentity = + | { readonly kind: 'ok'; readonly workspace: string; readonly workflowId: string } + | { + readonly kind: 'not-found'; + readonly reason: 'no-match' | 'unreadable-record'; + /** For 'unreadable-record': the session.json path that could not prove the identity. */ + readonly sessionPath?: string; + } + | { readonly kind: 'ambiguous'; readonly claims: readonly string[] }; + +/** Every workflow id a run's session record has ever claimed: the original plus each resume. */ +function readRecordedWorkflowIds(runDir: string): readonly string[] { + try { + const session = JSON.parse(fs.readFileSync(resolveRunFile(runDir, 'session.json'), 'utf-8')); + const resumeAttempts: { workflowId?: string }[] = session.session?.resumeAttempts ?? []; + const ids = [session.session?.originalWorkflowId, ...resumeAttempts.map((attempt) => attempt.workflowId)]; + return ids.filter((id): id is string => typeof id === 'string' && id.length > 0); + } catch { + return []; + } +} + +/** + * Prove the canonical (workspace, workflowId) pair for a status target. + * + * A directory with a readable session record takes precedence and follows its latest + * resume (an auto-named directory whose name equals its original workflow id resolves + * here). Otherwise the input is matched exactly against every workflow id the session + * records have claimed — session.json is the only trustworthy reverse mapping, and a + * valid workspace name may itself end in `_shannon-`, so the id naming + * convention is never used to guess. + */ +export function resolveScanIdentity(input: string): ScanIdentity { + const runDir = path.join(getWorkspacesDir(), input); + let isDirectory = false; + try { + isDirectory = fs.statSync(runDir).isDirectory(); + } catch { + // Not a workspace directory — fall through to the exact workflow-id match. + } + + if (isDirectory) { + const workflowId = resolveWorkflowId(input); + if (workflowId !== undefined) { + return { kind: 'ok', workspace: input, workflowId }; + } + return { kind: 'not-found', reason: 'unreadable-record', sessionPath: resolveRunFile(runDir, 'session.json') }; + } + + const claims: string[] = []; + for (const workspace of listWorkspaces()) { + const recorded = readRecordedWorkflowIds(path.join(getWorkspacesDir(), workspace.name)); + if (recorded.includes(input)) { + claims.push(workspace.name); + } + } + + if (claims.length === 1) { + // The exact requested id is kept, so an older workflow id keeps addressing that older execution. + return { kind: 'ok', workspace: claims[0] as string, workflowId: input }; + } + if (claims.length > 1) { + return { kind: 'ambiguous', claims: [...claims].sort() }; + } + return { kind: 'not-found', reason: 'no-match' }; +} + +export type DefaultTarget = + | { readonly kind: 'ok'; readonly workspace: string; readonly running: boolean } + | { readonly kind: 'none' } + | { readonly kind: 'ambiguous'; readonly running: readonly string[] }; + +/** + * Pick the default workspace when the user gave none. + * + * Exactly one scan running → that scan. Multiple running → ambiguous, so the caller can + * list them and ask for an explicit name. None running → the most recent workspace when + * `allowFinished` (viewing commands), otherwise none (stopping a finished scan is a no-op). + */ +export function resolveDefaultWorkspace(opts: { readonly allowFinished: boolean }): DefaultTarget { + const running = runningScanWorkspaces(); + if (running.length === 1) { + return { kind: 'ok', workspace: running[0] as string, running: true }; + } + if (running.length > 1) { + return { kind: 'ambiguous', running }; + } + + if (!opts.allowFinished) { + return { kind: 'none' }; + } + + const workspaces = listWorkspaces(); + const mostRecent = workspaces[0]; + if (!mostRecent) { + return { kind: 'none' }; + } + return { kind: 'ok', workspace: mostRecent.name, running: false }; +} diff --git a/apps/worker/configs/config-schema.json b/apps/worker/configs/config-schema.json index 78e16dd2..17fa7536 100644 --- a/apps/worker/configs/config-schema.json +++ b/apps/worker/configs/config-schema.json @@ -125,16 +125,18 @@ }, "additionalProperties": false }, - "vuln_classes": { - "type": "array", - "description": "Vulnerability classes to test. When omitted, all five classes run. When set, only listed classes run; their vuln+exploit agents and report sections are included.", - "items": { - "type": "string", - "enum": ["injection", "xss", "auth", "authz", "ssrf"] + "agentic_sast": { + "type": "object", + "description": "Opt in to agentic static analysis, which reads the repository for vulnerabilities before the pentest and feeds what it finds into the exploitation phase. Off by default. It does not change which vulnerability classes run. If agentic static analysis fails, the pentest continues without its findings and the scan finishes as \"partial\".", + "properties": { + "enabled": { + "type": "string", + "enum": ["true", "false"], + "description": "Set to \"true\" to run agentic static analysis. Defaults to \"false\"." + } }, - "minItems": 1, - "maxItems": 5, - "uniqueItems": true + "required": ["enabled"], + "additionalProperties": false }, "exploit": { "type": "string", @@ -193,7 +195,7 @@ { "required": ["rules"] }, { "required": ["authentication", "rules"] }, { "required": ["description"] }, - { "required": ["vuln_classes"] }, + { "required": ["agentic_sast"] }, { "required": ["exploit"] }, { "required": ["report"] }, { "required": ["rules_of_engagement"] } diff --git a/apps/worker/configs/example-config.yaml b/apps/worker/configs/example-config.yaml index ca4698d4..b92c3f18 100644 --- a/apps/worker/configs/example-config.yaml +++ b/apps/worker/configs/example-config.yaml @@ -4,8 +4,14 @@ # Description of the target environment (optional, max 500 chars) description: "Next.js e-commerce app on PostgreSQL. Local dev environment — .env files contain local-only credentials, not deployed to production." -# Limit which vulnerability classes run end-to-end (optional, default: all five) -# vuln_classes: [injection, xss, auth, authz, ssrf] +# Every scan runs all five vulnerability classes: injection, xss, auth, authz, and ssrf. +# There is no setting to narrow that. + +# Agentic static analysis (optional, default: "false"). +# Reads the repository for vulnerabilities before the pentest and feeds them into exploitation. +# It costs extra model time, and if it fails the scan finishes as "partial" without its findings. +# agentic_sast: +# enabled: "true" # Skip the exploitation phase (optional, default: "true") # exploit: "false" diff --git a/apps/worker/package.json b/apps/worker/package.json index 73fefdd1..36453594 100644 --- a/apps/worker/package.json +++ b/apps/worker/package.json @@ -10,7 +10,27 @@ "./types/agents": "./dist/types/agents.js", "./pipeline": "./dist/temporal/pipeline.js", "./activities": "./dist/temporal/activities.js", + "./temporal/reconcile-activity-types": "./dist/temporal/reconcile-activity-types.js", "./services": "./dist/services/index.js", + "./services/queue-validation": "./dist/services/queue-validation.js", + "./services/renumber-core": "./dist/services/renumber-core.js", + "./services/compaction-core": "./dist/services/compaction-core.js", + "./services/finding-order": "./dist/services/finding-order.js", + "./ai/structured-generation": "./dist/ai/structured-generation.js", + "./ai/pi/source-jail": "./dist/ai/pi/source-jail.js", + "./ai/reconciliation/contracts": "./dist/ai/reconciliation/contracts.js", + "./ai/reconciliation/stage-contracts": "./dist/ai/reconciliation/stage-contracts.js", + "./ai/reconciliation/artifact-store": "./dist/ai/reconciliation/artifact-store.js", + "./ai/reconciliation/schema-version": "./dist/ai/reconciliation/schema-version.js", + "./ai/reconciliation/manifest": "./dist/ai/reconciliation/manifest.js", + "./ai/reconciliation/prepare": "./dist/ai/reconciliation/prepare.js", + "./ai/reconciliation/enrich": "./dist/ai/reconciliation/enrich.js", + "./ai/reconciliation/form": "./dist/ai/reconciliation/form.js", + "./ai/reconciliation/materialize": "./dist/ai/reconciliation/materialize.js", + "./ai/reconciliation/observation-view": "./dist/ai/reconciliation/observation-view.js", + "./ai/reconciliation/labels": "./dist/ai/reconciliation/labels.js", + "./ai/reconciliation/submit-validation": "./dist/ai/reconciliation/submit-validation.js", + "./ai/reconciliation/refs": "./dist/ai/reconciliation/refs.js", "./config": "./dist/config-parser.js" }, "scripts": { @@ -23,13 +43,14 @@ "@earendil-works/pi-ai": "^0.82.1", "@earendil-works/pi-coding-agent": "^0.82.1", "@gotgenes/pi-permission-system": "^10.9.0", - "@temporalio/activity": "^1.11.0", - "@temporalio/client": "^1.11.0", - "@temporalio/worker": "^1.11.0", - "@temporalio/workflow": "^1.11.0", + "@temporalio/activity": "1.15.0", + "@temporalio/client": "1.15.0", + "@temporalio/worker": "1.15.0", + "@temporalio/workflow": "1.15.0", "ajv": "^8.12.0", "ajv-formats": "^2.1.1", "dotenv": "^16.4.5", + "handlebars": "^4.7.9", "js-yaml": "^4.1.0", "typebox": "1.1.38", "zx": "^8.0.0" diff --git a/apps/worker/prompts/exploit-auth.txt b/apps/worker/prompts/exploit-auth.txt index 67dbac04..01d01610 100644 --- a/apps/worker/prompts/exploit-auth.txt +++ b/apps/worker/prompts/exploit-auth.txt @@ -8,7 +8,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e -@include(shared/_exploit-scope.txt) +@include(shared/exploitation/_exploit-scope.txt) @@ -53,21 +53,23 @@ Remember: An unproven vulnerability is worse than no finding at all - it wastes **Level 1 - Authentication Weakness Identified** - Evidence: Theoretical bypass found but not yet exploited -- Classification: POTENTIAL (Low Confidence) +- Classification: `status: "blocked"`, `confidence: "low"` **Level 2 - Partial Bypass Achieved** - Evidence: Access gained to non-critical protected resources or UI elements -- Classification: POTENTIAL (Medium Confidence) +- Classification: `status: "blocked"`, `confidence: "medium"` **Level 3 - Authentication Bypass Confirmed** - Evidence: Successfully accessed protected functionality as unauthorized user -- Classification: EXPLOITED +- Classification: `status: "exploited"` **Level 4 - Account Takeover Demonstrated** - Evidence: Complete user impersonation with access to sensitive data/actions -- Classification: EXPLOITED (CRITICAL) +- Classification: `status: "exploited"` -You MUST reach at least Level 3 with evidence to mark as "EXPLOITED" +Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`. + +These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in . ### Classification Decision Framework @@ -231,26 +233,26 @@ Only after exhausting bypass attempts can you proceed to classification analysis ## **Validation Criteria & Vulnerability Classification** You must adhere to these rules when making a judgment call and categorizing findings: -**EXPLOITED (Report in main section):** +**`status: "exploited"`:** - Successfully achieved account takeover or authentication bypass with concrete evidence (impersonated user, accessed protected resources) - OR vulnerability confirmed with active exploitation evidence (e.g., "Performed 200 brute force attempts without rate limiting") - Definitive proof of access to a protected resource as another authenticated user OR concrete demonstration of security control bypass - Must have completed weaponization and demonstrated impact through impersonation OR provided evidence of exploitable vulnerability -POTENTIAL (Report in separate subsection): -- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. +**`status: "blocked"`:** +- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. - **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack** - A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt. - Valid external factors include: - Server instability (e.g., crashes). - Missing authentication that cannot be obtained or bypassed through any known exploit. -- This classification also applies when: +- This status also applies when: - Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints. - You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors. -**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.** +**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.** -**FALSE POSITIVE (Do NOT report):** +**FALSE POSITIVE (never submitted through the tool):** - **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways** - Definitively proven not exploitable through exhaustive testing - Documented attack plan showing all weaponization attempts failed @@ -260,7 +262,7 @@ POTENTIAL (Report in separate subsection): **Important:** Rate limiting and account lockouts are obstacles to work around, not grounds for immediate false positive classification. Document constraints and attempt alternative approaches before making final determination. -**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop authentication attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL. +**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop authentication attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`. ### CRITICAL: REPRODUCIBILITY REQUIREMENTS @@ -304,12 +306,18 @@ Prioritize these based on the intelligence from your queue and vulnerability ana +@include(shared/exploitation/_severity-reasoning.txt) + +@include(shared/exploitation/_reporting-standards.txt) + +@include(shared/exploitation/_credentials-in-findings.txt) + You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/auth_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly. **When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission. -**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (account takeover demonstrated, session hijacked end-to-end, password reset abused, MFA bypassed). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/auth_false_positives.md` tracking file, not the deliverable. +**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (account takeover demonstrated, session hijacked end-to-end, password reset abused, MFA bypassed) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/auth_false_positives.md`, never through this tool. **ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/auth_exploitation_queue.json` exactly (e.g. `AUTH-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry. diff --git a/apps/worker/prompts/exploit-authz.txt b/apps/worker/prompts/exploit-authz.txt index 6bbaac82..94ad844c 100644 --- a/apps/worker/prompts/exploit-authz.txt +++ b/apps/worker/prompts/exploit-authz.txt @@ -8,7 +8,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e -@include(shared/_exploit-scope.txt) +@include(shared/exploitation/_exploit-scope.txt) @@ -53,21 +53,23 @@ Remember: An unproven vulnerability is worse than no finding at all - it wastes **Level 1 - Authorization Weakness Identified** - Evidence: Theoretical bypass found but not yet exploited -- Classification: POTENTIAL (Low Confidence) +- Classification: `status: "blocked"`, `confidence: "low"` **Level 2 - Partial Access Control Bypass** - Evidence: Access gained to some protected resources or partial privilege escalation -- Classification: POTENTIAL (Medium Confidence) +- Classification: `status: "blocked"`, `confidence: "medium"` **Level 3 - Access Control Bypass Confirmed** - Evidence: Successfully accessed protected functionality or data as unauthorized user -- Classification: EXPLOITED +- Classification: `status: "exploited"` **Level 4 - Critical Privilege Escalation Demonstrated** - Evidence: Complete privilege escalation with access to administrative functions or highly sensitive data -- Classification: EXPLOITED (CRITICAL) +- Classification: `status: "exploited"` -You MUST reach at least Level 3 with evidence to mark as "EXPLOITED" +Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`. + +These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in . ### Classification Decision Framework @@ -230,26 +232,26 @@ Only after exhausting bypass attempts can you proceed to classification analysis ## **Validation Criteria & Vulnerability Classification** You must adhere to these rules when making a judgment call and categorizing findings: -**EXPLOITED (Report in main section):** +**`status: "exploited"`:** - Successfully achieved privilege escalation or unauthorized access with concrete evidence (accessed protected data, modified unauthorized resources) - Definitive proof of access to resources that should be restricted based on role or ownership - Must have completed weaponization and demonstrated impact through unauthorized access - MUST have reached Level 3 or 4 with evidence -POTENTIAL (Report in separate subsection): -- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. +**`status: "blocked"`:** +- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. - **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack** - A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt. - Valid external factors include: - Server instability (e.g., crashes). - Missing authentication that cannot be obtained or bypassed through any known exploit. -- This classification also applies when: +- This status also applies when: - Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints. - You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors. -**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.** +**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.** -**FALSE POSITIVE (Do NOT report):** +**FALSE POSITIVE (never submitted through the tool):** - **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways** - Definitively proven not exploitable through exhaustive testing - Documented attack plan showing all weaponization attempts failed @@ -259,7 +261,7 @@ POTENTIAL (Report in separate subsection): **Important:** Authentication requirements and access restrictions are obstacles to work around, not grounds for immediate false positive classification. Document constraints and attempt alternative approaches before making final determination. -**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop authorization attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL. +**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop authorization attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`. ### CRITICAL: REPRODUCIBILITY REQUIREMENTS @@ -312,12 +314,18 @@ Remember: The most effective attacks often come from understanding the specific +@include(shared/exploitation/_severity-reasoning.txt) + +@include(shared/exploitation/_reporting-standards.txt) + +@include(shared/exploitation/_credentials-in-findings.txt) + You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/authz_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly. **When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission. -**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (cross-tenant data access demonstrated, privilege escalation proven, unauthorized workflow transition executed). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/authz_false_positives.md` tracking file, not the deliverable. +**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (cross-tenant data access demonstrated, privilege escalation proven, unauthorized workflow transition executed) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/authz_false_positives.md`, never through this tool. **ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/authz_exploitation_queue.json` exactly (e.g. `AUTHZ-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry. diff --git a/apps/worker/prompts/exploit-injection.txt b/apps/worker/prompts/exploit-injection.txt index 7bd01b89..b10e13c2 100644 --- a/apps/worker/prompts/exploit-injection.txt +++ b/apps/worker/prompts/exploit-injection.txt @@ -8,7 +8,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e -@include(shared/_exploit-scope.txt) +@include(shared/exploitation/_exploit-scope.txt) @@ -210,21 +210,23 @@ Success Criteria: Print status code and response excerpt for each payload so I c **Level 1 - Injection Point Confirmed** - Evidence: Error messages, timing differences, or response variations -- Classification: POTENTIAL (Low Confidence) +- Classification: `status: "blocked"`, `confidence: "low"` **Level 2 - Query Structure Manipulated** - Evidence: Boolean-based blind working, UNION SELECT succeeds, ORDER BY confirms columns -- Classification: POTENTIAL (Medium Confidence) +- Classification: `status: "blocked"`, `confidence: "medium"` **Level 3 - Data Extraction Proven** - Evidence: Actual data retrieved from database (usernames, table names, etc.) -- Classification: EXPLOITED +- Classification: `status: "exploited"` **Level 4 - Critical Impact Demonstrated** - Evidence: Admin credentials extracted, sensitive data dumped, or system commands executed -- Classification: EXPLOITED (CRITICAL) +- Classification: `status: "exploited"` -You MUST reach at least Level 3 with evidence to mark as "EXPLOITED" +Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`. + +These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in . ### Classification Decision Framework @@ -271,26 +273,26 @@ For each **successfully exploited** vulnerability, your goal is to collect the f ## **Validation Criteria & Vulnerability Classification** You must adhere to these rules when making a judgment call and categorizing findings: - **EXPLOITED (Report in main section):** + **`status: "exploited"`:** - Successfully executed exploit with concrete evidence (data extracted, unauthorized access achieved) - MUST have reached at least Level 3 with actual data extraction - Consistent error messages, predictable differential responses to boolean tests, or measurable time delays - Must have completed the full OWASP workflow with tangible proof - -POTENTIAL (Report in separate subsection): -- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. + +**`status: "blocked"`:** +- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. - **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack** - A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt. - Valid external factors include: - Server instability (e.g., crashes). - Missing authentication that cannot be obtained or bypassed through any known exploit. -- This classification also applies when: +- This status also applies when: - Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints. - You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors. -**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.** +**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.** -**FALSE POSITIVE (Do NOT report):** +**FALSE POSITIVE (never submitted through the tool):** - **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways** - Definitively proven not exploitable through systematic testing with multiple techniques - Multiple bypass attempts all failed with no indication of underlying vulnerability @@ -300,7 +302,7 @@ POTENTIAL (Report in separate subsection): **Important:** WAF blocking is an obstacle to overcome, not grounds for immediate classification as false positive. Exhaust bypass techniques before making final determination. -**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop injection attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL. +**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop injection attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`. ### CRITICAL: REPRODUCIBILITY REQUIREMENTS @@ -335,12 +337,18 @@ Prioritize your techniques based on the intelligence from the analysis deliverab +@include(shared/exploitation/_severity-reasoning.txt) + +@include(shared/exploitation/_reporting-standards.txt) + +@include(shared/exploitation/_credentials-in-findings.txt) + You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/injection_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly. **When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission. -**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (data extracted, JavaScript executed, account taken over, internal service accessed). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/injection_false_positives.md` tracking file, not the deliverable. +**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (data extracted, JavaScript executed, account taken over, internal service accessed) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/injection_false_positives.md`, never through this tool. **ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/injection_exploitation_queue.json` exactly (e.g. `INJ-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry. diff --git a/apps/worker/prompts/exploit-miscellaneous.txt b/apps/worker/prompts/exploit-miscellaneous.txt new file mode 100644 index 00000000..ffb9912a --- /dev/null +++ b/apps/worker/prompts/exploit-miscellaneous.txt @@ -0,0 +1,104 @@ + +You are a generalist exploitation specialist for security weaknesses that do not belong to Shannon's five analysis-backed classes. You are an active penetration tester, not a vulnerability analyst. Your job is to turn each reconciled task into evidence of demonstrated impact or an honest blocked verdict. + + + +Process every task in `.shannon/deliverables/miscellaneous_exploitation_queue.json`. The queue may contain unrelated weakness families discovered by static analysis, so choose the exploitation method from each task's evidence instead of applying one class-wide playbook. Reach a definitive conclusion for every task and emit structured evidence with `add_exploit`. + + + +@include(shared/_exploit-scope.txt) + + + +### CRITICAL: EXPLOITATION TARGET REQUIREMENT + +All dynamic exploitation must be performed against the specified target: {{WEB_URL}} + +An exploit is an unauthorized or unintended action demonstrated against the target application. Static-analysis evidence, a plausible code path, or a tool warning alone is not a successful exploit. + +- Claims require concrete, reproducible evidence. +- Validate the task's assumptions before building on them. +- Use `status: "exploited"` only after demonstrating impact. +- Use `status: "blocked"` only when a real vulnerability is stopped by an external operational constraint, not by an effective security control. +- Record false positives only in `workspace/miscellaneous_false_positives.md`; do not submit them through `add_exploit`. +- Never test production systems. This engagement is authorized only for the controlled target and stated rules. + + + +@include(shared/_target.txt) + + + +@include(shared/_rules.txt) + + +@include(shared/_shared-session.txt) + + +{{LOGIN_INSTRUCTIONS}} + + +@include(shared/_rules-of-engagement.txt) + + +Your actionable queue is `.shannon/deliverables/miscellaneous_exploitation_queue.json`. Its IDs are stable task references such as `MISC-01`. Process every queue entry exactly once. + +Read these inputs before testing: +1. `.shannon/deliverables/pre_recon_deliverable.md` for architecture and source layout. +2. `.shannon/deliverables/recon_deliverable.md` for the live attack surface. +3. `.shannon/deliverables/miscellaneous_exploitation_queue.json` for the reconciled tasks and their SAST evidence. + +There is no `miscellaneous` vulnerability-analysis agent and no `miscellaneous_analysis_deliverable.md`. Do not look for one or imply that one ran. A task can include `sast_source_location`; treat it as a lead until you inspect the code yourself. + +Use `todo_write` to create and track one task per queue entry. + + + +**Phase sequence:** RECONNAISSANCE → SAST RECONCILIATION → **MISCELLANEOUS EXPLOITATION (YOU)** → FINAL REPORT + +**Input:** `.shannon/deliverables/miscellaneous_exploitation_queue.json` +**Output:** `.shannon/deliverables/miscellaneous_exploitation_evidence.md`, rendered by the host from your `add_exploit` calls + +Your queue is analysis-less in the agent sense: its observations came from the internal SAST/reconciliation path. Your role is to verify those tasks against source and the live target without inventing missing analysis context. + + + +- **Browser Automation (playwright-cli skill):** Use when the task requires browser interactions. Always pass `-s={{PLAYWRIGHT_SESSION}}`. +- **`bash` tool:** Use for focused commands and reproducible HTTP requests. +- **`task` agent:** Use for custom scripts, payload loops, or repetitive testing. +- **`todo_write` tool:** Track every queue task and its final verdict. +- **`read` tool:** Read source, queue evidence, and `workspace/miscellaneous_false_positives.md`. + + + +For each `MISC-NN` task: + +1. Read the complete task, including CWE, source location, hypothesis, suggested technique, and proof criterion when present. +2. Inspect the cited code and trace the relevant input, guard, and effect. Do not copy a SAST location into `code_locations` unless you actually opened and inspected it. +3. Identify the reachable live entry point from reconnaissance and verify any prerequisites. +4. Attempt the weakness-specific exploit method. Adapt the technique to the actual weakness rather than assuming injection, XSS, authentication, authorization, or SSRF behavior. +5. Pursue concrete impact. A source-level defect without a demonstrated target action is not `exploited`. +6. If an external constraint prevents completion, document the evidence that the defect is real, everything attempted, and the impact expected if the constraint were removed. +7. If the code or live behavior disproves the task, record it in `workspace/miscellaneous_false_positives.md` and do not call `add_exploit` for it. +8. Call `add_exploit` once for the final exploited or blocked verdict, then complete the todo and move to the next task. + +When you report `code_locations`, use only inspected code paths relative to the Shannon workspace. Use POSIX separators, include the repository checkout segment, and provide `line_number` as an exact positive line or `null`. Omit `code_locations` when you inspected no code; never submit an empty array. + + + +The `add_exploit` tool is your only evidence output. The host renders the Markdown deliverable after your run. + +- `vulnerability_id` must exactly match an `MISC-NN` ID from `miscellaneous_exploitation_queue.json`. +- Set `status: "exploited"` only for demonstrated impact and supply all exploited fields. +- Set `status: "blocked"` only for a real vulnerability stopped by an external constraint and supply all blocked fields. +- Do not emit producer IDs such as `MISC-SAST-01`; those are not exploitation-task references. +- Duplicate or out-of-queue IDs are rejected. Reach a final verdict before emitting. +- Do not call `add_exploit` for false positives. + + + +Before finishing, verify that every task in `miscellaneous_exploitation_queue.json` has one final disposition: exploited evidence, blocked evidence, or a documented false positive. Every non-false-positive task must have exactly one accepted `add_exploit` call, and every todo must be complete. + +Only then announce `MISCELLANEOUS EXPLOITATION COMPLETE` and stop immediately. Do not add a prose recap; the structured deliverable is the record. + diff --git a/apps/worker/prompts/exploit-ssrf.txt b/apps/worker/prompts/exploit-ssrf.txt index fe800a62..c28ca4fb 100644 --- a/apps/worker/prompts/exploit-ssrf.txt +++ b/apps/worker/prompts/exploit-ssrf.txt @@ -8,7 +8,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e -@include(shared/_exploit-scope.txt) +@include(shared/exploitation/_exploit-scope.txt) @@ -53,21 +53,23 @@ Remember: An unproven vulnerability is worse than no finding at all - it wastes **Level 1 - SSRF Weakness Identified** - Evidence: Theoretical access found but not yet exploited -- Classification: POTENTIAL (Low Confidence) +- Classification: `status: "blocked"`, `confidence: "low"` **Level 2 - Partial Network Access** - Evidence: Successfully triggered outbound requests or received responses indicating internal network activity -- Classification: POTENTIAL (Medium Confidence) +- Classification: `status: "blocked"`, `confidence: "medium"` **Level 3 - Internal Service Access Confirmed** - Evidence: Successfully accessed internal services, retrieved internal data, or confirmed network boundary bypass -- Classification: EXPLOITED +- Classification: `status: "exploited"` **Level 4 - Critical Information Disclosure** - Evidence: Retrieved sensitive data from internal services, cloud metadata, or performed successful privilege escalation -- Classification: EXPLOITED (CRITICAL) +- Classification: `status: "exploited"` -You MUST reach at least Level 3 with evidence to mark as "EXPLOITED" +Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`. + +These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in . ### Classification Decision Framework @@ -221,26 +223,26 @@ Only after exhausting bypass attempts can you proceed to classification analysis ## **Validation Criteria & Vulnerability Classification** You must adhere to these rules when making a judgment call and categorizing findings: -**EXPLOITED (Report in main section):** +**`status: "exploited"`:** - Successfully achieved internal service access, cloud metadata retrieval, or network reconnaissance with concrete evidence - Definitive proof of accessing resources that should not be reachable from external networks - Must have completed confirmation and demonstrated meaningful impact - MUST have reached Level 3 or 4 with evidence -POTENTIAL (Report in separate subsection): -- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. +**`status: "blocked"`:** +- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. - **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack** - A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt. - Valid external factors include: - Server instability (e.g., crashes). - Missing authentication that cannot be obtained or bypassed through any known exploit. -- This classification also applies when: +- This status also applies when: - Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints. - You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors. -**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.** +**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.** -**FALSE POSITIVE (Do NOT report):** +**FALSE POSITIVE (never submitted through the tool):** - **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways** - Definitively proven not exploitable through exhaustive testing - Documented attack plan showing all exploitation attempts failed @@ -248,9 +250,9 @@ POTENTIAL (Report in separate subsection): - **Documentation:** Record false positives in your tracking file `workspace/ssrf_false_positives.md` for completeness and learning, but DO NOT include them in the deliverable. Document each false positive with the vulnerability ID, what was attempted, and why it was determined to be a false positive. - Proven impossible after systematic testing with multiple techniques -**Important:** Network timeouts and connection refused errors require analysis - they could indicate successful requests being blocked by internal firewalls (potential) or no vulnerability at all (false positive). Test multiple internal targets to distinguish. +**Important:** Network timeouts and connection refused errors require analysis - they could indicate successful requests being blocked by internal firewalls (`status: "blocked"`) or no vulnerability at all (FALSE POSITIVE). Test multiple internal targets to distinguish. -**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop SSRF attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL. +**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop SSRF attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`. ### CRITICAL: REPRODUCIBILITY REQUIREMENTS @@ -385,16 +387,22 @@ Prioritize these based on the intelligence from your queue and vulnerability ana A successful SSRF doesn't always mean data is immediately exfiltrated. Validation can rely on *direct reflection*, *side effects*, or *out-of-band confirmation*. Always adapt your verification to the SSRF type. -**Critical:** Network timeouts and connection refused errors require analysis - they could indicate successful requests being blocked by internal firewalls (POTENTIAL) or no vulnerability at all (FALSE POSITIVE). Test multiple internal targets to distinguish between these scenarios. +**Critical:** Network timeouts and connection refused errors require analysis - they could indicate successful requests being blocked by internal firewalls (`status: "blocked"`) or no vulnerability at all (FALSE POSITIVE). Test multiple internal targets to distinguish between these scenarios. +@include(shared/exploitation/_severity-reasoning.txt) + +@include(shared/exploitation/_reporting-standards.txt) + +@include(shared/exploitation/_credentials-in-findings.txt) + You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/ssrf_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly. **When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission. -**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (internal service contents retrieved, cloud metadata extracted, port scan results captured, webhook abuse demonstrated). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/ssrf_false_positives.md` tracking file, not the deliverable. +**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (internal service contents retrieved, cloud metadata extracted, port scan results captured, webhook abuse demonstrated) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/ssrf_false_positives.md`, never through this tool. **ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/ssrf_exploitation_queue.json` exactly (e.g. `SSRF-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry. diff --git a/apps/worker/prompts/exploit-xss.txt b/apps/worker/prompts/exploit-xss.txt index 3c05a4e2..f9c15887 100644 --- a/apps/worker/prompts/exploit-xss.txt +++ b/apps/worker/prompts/exploit-xss.txt @@ -19,7 +19,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e -@include(shared/_exploit-scope.txt) +@include(shared/exploitation/_exploit-scope.txt) @@ -53,17 +53,19 @@ An exploit is executing an unauthorized or unintended action against the target **Level 2 - Payload Injected But Blocked** - Evidence: Payload in HTML but JavaScript execution prevented by filters/CSP -- Classification: POTENTIAL (requires bypass) +- Classification: `status: "blocked"` — only if a non-security constraint stopped the bypass; a filter or CSP that held is a FALSE POSITIVE **Level 3 - JavaScript Execution Confirmed** - Evidence: JavaScript executes (alert, console.log, DOM manipulation) -- Classification: EXPLOITED +- Classification: `status: "exploited"` **Level 4 - Critical Impact Demonstrated** - Evidence: Session hijacking, data theft, unauthorized actions performed -- Classification: EXPLOITED (CRITICAL) +- Classification: `status: "exploited"` -You MUST reach at least Level 3 with evidence to mark as "EXPLOITED" +Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`. + +These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in . ### Classification Decision Framework @@ -257,26 +259,26 @@ For each successfully exploited vulnerability, aim to demonstrate meaningful imp ## **Validation Criteria & Vulnerability Classification** You must adhere to these rules when making a judgment call and categorizing findings: -**EXPLOITED (Report in main section):** +**`status: "exploited"`:** - Successfully executed script in browser with concrete evidence (session hijacked, unauthorized actions performed) - Reproducible script execution with tangible impact achieved - Must have completed the full OWASP workflow with proof of compromise - MUST have reached Level 3 or 4 with evidence -POTENTIAL (Report in separate subsection): -- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. +**`status: "blocked"`:** +- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control. - **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack** - A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt. - Valid external factors include: - Server instability (e.g., crashes). - Missing authentication that cannot be obtained or bypassed through any known exploit. -- This classification also applies when: +- This status also applies when: - Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints. - You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors. -**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.** +**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.** -**FALSE POSITIVE (Do NOT report):** +**FALSE POSITIVE (never submitted through the tool):** - **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways** - Definitively proven not exploitable through systematic testing with multiple bypass techniques - Multiple bypass attempts all failed including different payloads, encoding techniques, and CSP bypass attempts @@ -285,7 +287,7 @@ POTENTIAL (Report in separate subsection): **Important:** WAF/CSP blocking is an obstacle to overcome, not grounds for immediate classification as false positive. Document and attempt bypasses before making final determination. -**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop XSS attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL. +**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop XSS attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`. ### CRITICAL: REPRODUCIBILITY REQUIREMENTS @@ -322,12 +324,18 @@ POTENTIAL (Report in separate subsection): +@include(shared/exploitation/_severity-reasoning.txt) + +@include(shared/exploitation/_reporting-standards.txt) + +@include(shared/exploitation/_credentials-in-findings.txt) + You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/xss_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly. **When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission. -**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (JavaScript executed in a real browser, session/cookie data exfiltrated, DOM modified to demonstrate impact). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/xss_false_positives.md` tracking file, not the deliverable. +**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (JavaScript executed in a real browser, session/cookie data exfiltrated, DOM modified to demonstrate impact) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/xss_false_positives.md`, never through this tool. **ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/xss_exploitation_queue.json` exactly (e.g. `XSS-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry. diff --git a/apps/worker/prompts/partials/capella-calibration-rules.hbs b/apps/worker/prompts/partials/capella-calibration-rules.hbs new file mode 100644 index 00000000..27f48b52 --- /dev/null +++ b/apps/worker/prompts/partials/capella-calibration-rules.hbs @@ -0,0 +1,222 @@ +{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Calibration Rules Catalogue + +This document defines the 27 calibration sanity triage rules (caps and +downgrades) used to calculate the final severity and priority of findings. + +## Table of Contents + +- [Core Principle: Marginal Capability](#core-principle-marginal-capability) +- [Category A: Force-Downgrade to LOW (Cap at 2.0 / LOW Priority)](#category-a-force-downgrade-to-low-cap-at-20--low-priority) +- [Category B: Force-Cap to HIGH (Cap at 7.9 / Maximum HIGH Priority)](#category-b-force-cap-to-high-cap-at-79--maximum-high-priority) +- [Category C: Force-Cap to MEDIUM (Cap at 5.9 / Maximum MEDIUM Priority)](#category-c-force-cap-to-medium-cap-at-59--maximum-medium-priority) + +## Core Principle: Marginal Capability + +The final severity and priority of a finding are strictly bounded by the +**marginal capability** gained by the attacker over their prerequisite position. +If the exploit does not grant the attacker significant new control, access, or +capabilities beyond what is already inherent to their starting position (or +already possessed via legitimate means), the finding must be capped or +downgraded. + +______________________________________________________________________ + +### Category A: Force-Downgrade to LOW (Cap at 2.0 / LOW Priority) + +01. **`repro_failure` (Reproduction Failure or Not Attempted)** The reproduction + failed (`repro_status: "failed_to_reproduce"`), was not attempted + (`repro_status: "not_attempted"`), or the `repro_status` field was missing + (treated as `"not_attempted"`), regardless of theoretical production + viability. + + +02. **`unreachable_inputs` (Unreachable / Uncontrolled Inputs)** The finding + relies on inputs that are documented as highly unlikely to be + user-controlled, and no path from a trust boundary is proven. + +03. **`third_party_reachability` (Third-Party / Supply Chain Reachability)** + Vulnerabilities in third-party libraries (dependency CVEs) where a reachable + path from application input to the vulnerable function has not been actively + demonstrated. + +04. **`minor_config_hygiene` (Minor Configuration Hygiene)** Minor deviations + from best practice (e.g., slightly loose permissions on internal dirs, lack + of modern encryption on low-value internal transport) without a clear + exploit path. + +05. **`non_security_critical` (Non-Security Critical Components)** The finding + affects a component or data with no security sensitivity (e.g., public info, + signatures on non-security payloads, cosmetic outputs). + +06. **`vague_code_paths` (Vague Code Paths / Fragile Assumptions)** Relying on + unverified assumptions about caller behavior or adjacent system components. + + +07. **`unreliable_triggers` (Unreliable/Noisy Triggers)** Triggers that are + likely to be ignored in practice or indistinguishable from normal + operations. + +08. **`prerequisite_shell` (Prerequisite Shell Access / Equivalent Primitives)** + The attacker already possesses local shell access on the target container or + host with the **same or higher** privilege level than the exploit provides, + rendering the gained access redundant under the Principle of Marginal + Capability (e.g., exploiting a bug to get a standard user shell when already + logged in as a standard user, or exploiting a local buffer overflow to run + commands as root when already running as root). This does NOT apply to + low-to-high privilege escalation (e.g., standard user to root), which should + cap at MEDIUM. + +09. **`physical_long_term` (Physical Long-Term / Laboratory Access)** If the + attack requires long-term physical access to the device or specialized + laboratory equipment (e.g., fault injection, side-channel analysis, chip + decapping). Force-downgrade to **LOW (2.0)** due to the extreme execution + barrier and requirement for physical possession. + +10. **`trusted_controller_zero_delta` (Trusted-Controller-Mediated Interface - + Zero Delta)** If the vulnerable interface is reachable only from a component + that holds designed-in authoritative control over the target (e.g., + orchestrator->worker, driver->device firmware, protocol master->slave, + hypervisor->guest, management plane->data plane node), and the exploit + grants **zero marginal capability** (i.e., the controller could already + achieve the identical effect or level of compromise via its standard, + legitimate interface), force-downgrade to **LOW (2.0)**. (This generalizes + the *Standard Host-to-Guest Attacks* rule below). + +11. **`standard_host_attacks` (Standard Host-to-Guest Attacks)** If the attacker + position is `HOST_SYSTEM` (host hypervisor attacking guest) on standard + deployments (non-Confidential Computing). Force-downgrade to **LOW (2.0)** + as the host OS/hypervisor already possesses total control over the guest by + design, meaning the exploit offers zero marginal capability over the + prerequisite position (equivalent primitives). **Default assumption:** treat + as non-Confidential Computing (this rule fires) UNLESS the Threat Model, + code path, or finding description explicitly names Confidential Computing, + guest enclaves, TEE, SEV, TDX, SGX, or attestation (in which case apply the + CC Host Attacks cap-HIGH rule instead). + +______________________________________________________________________ + +### Category B: Force-Cap to HIGH (Cap at 7.9 / Maximum HIGH Priority) + +1. **`static_confirmation` (Static Confirmation)** Statically confirmed but not + empirically reproduced (`repro_status: "statically_confirmed"`). Cap + `likelihood_score` at **3**, apply **0.8** multiplier to Hazard, and MUST NOT + be CRITICAL. *Exception:* If the finding details (description, history, or + reproduction output) include a valid external stack trace, sanitizer trace + (e.g. ASan, UBSan, MSan), crash log, or core dump proving the vulnerability + was triggered in execution (e.g., in a prior run or by external tools), treat + it as empirically reproduced (Likelihood 5) and do not apply this static cap. + + +2. **`strict_xss` (Strict XSS Caps)** All XSS vulnerabilities. Default to MEDIUM + or LOW; cap at HIGH (7.9) only for stored XSS on critical admin pages with + zero-click execution for the admin. + +3. **`internal_nested` (Internal / Nested Components)** Any finding with a + Network/Trust Exposure multiplier less than 1.0 (i.e., Internal Component or + Privileged Zone). If the calculated score lands in the CRITICAL range, cap + the score at **7.9** and downgrade the priority to HIGH. *Exception:* Do NOT + cap at HIGH if the component is core in-cluster infrastructure (e.g., CNI, + CSI, admission webhook, service mesh) AND the impact escapes to the host node + (e.g., node-root file R/W) or allows cross-tenant escalation. These remain + eligible for CRITICAL. **This rule MUST NOT fire when the `attacker_position` + is `"EXTERNAL"` (since per the alignment rule in Section 2, the exposure is + forced to `EXPOSED` (1.0), which precludes this cap).** + +4. **`probabilistic_llm` (Probabilistic LLM Vectors)** Attacks relying on + probabilistic LLM behavior (e.g., prompt injection, jailbreaking) to trigger + a vulnerability. Cap at **HIGH** (7.9) and default to **MEDIUM** or **LOW**. + *Exception:* If the attacker can query the LLM/system repeatedly without rate + limits, concurrency limits, or security blocking/alerting that would impede + the attack (allowing them to brute-force and effectively eliminate the + non-determinism), this cap may be lifted. + +5. **`supply_chain_prerequisites` (Supply-Chain / Build-Time Prerequisites)** If + the exploit requires the attacker to already possess a supply-chain position + (e.g., ability to poison dependencies, modify upstream source) or write + access to the build pipeline to trigger the vulnerability. Cap at **HIGH + (7.9)** since the entry barrier is extremely high, but the downstream + compromise is systemic. (Force-downgrade to LOW/2.0 only if they already + possess shell access on the target, as per the Prerequisite Shell Access + rule). + +6. **`non_default_config` (Non-Default Configurations)** Findings that are only + exploitable under non-default configurations. Cap at **HIGH (7.9)** to + reflect the additional configuration barrier. + +7. **`confidential_computing_host` (Confidential Computing Host Attacks)** If + the attacker position is `HOST_SYSTEM` (the host OS or hypervisor attacking + guest enclaves or confidential VMs) in Confidential Computing deployments. + Cap at **HIGH (7.9)** because while the host has full control of the + platform, confidential computing enclaves are designed to protect against + host-level compromise. (If not a CC deployment, see the Standard + Host-to-Guest Attacks rule under LOW). + +8. **`trusted_controller_critical_bypass` (Trusted-Controller-Mediated Interface + \- Critical Bypass)** If the vulnerable interface is reachable only from a + designed-in authoritative controller, and the exploit allows that controller + to bypass target-side **documented security controls** or **safety-of-life + limits** it was designed to respect, cap at **HIGH (7.9)**. (If the exploit + allows lateral reach into a different trust domain or achieves persistence + surviving controller re-provisioning, do not cap). + +______________________________________________________________________ + +### Category C: Force-Cap to MEDIUM (Cap at 5.9 / Maximum MEDIUM Priority) + +1. **`local_attack_vector` (Local Attack Vector)** Vulnerabilities requiring + local shell access (e.g., local privilege escalation, SUID exploitation) + without VM escape. (Downgrade to LOW/2.0 if it only affects a single user's + isolated data). + +2. **`self_contained_blast` (Self-Contained Blast Radius)** If the maximum + impact of the exploit is confined to resources, data, or execution contexts + that the triggering principal already owns or has full designed-in authority + over — their own account, tenant, project, namespace, container, VM, device, + or single-user installation — and does not cross any isolation boundary + between mutually-distrusting principals, cap at **MEDIUM (5.9)**. + + - The exploit may grant genuinely new capability within that domain (e.g., + API-user -> shell in their own container), but the deployment's core + isolation guarantees to other parties still hold. + - Do **NOT** apply this cap if the exploit: + - reaches another principal's resources (cross-tenant, cross-user, + cross-account), + - touches shared or multi-party infrastructure (shared cache, shared + filesystem, operator control plane, co-tenant side-channel), + - places the attacker's domain upstream of others (build node, CI runner, + package registry, model-serving host — i.e., a supply-chain position), or + - persists in a way that survives the principal's own resource lifecycle + and could later affect a different principal reusing that slot. + +3. **`rarely_exposed` (Rarely Exposed Components)** Findings in components + documented as 'rarely exposed' or 'unlikely to be user controlled'. + +4. **`equivalent_primitives` (Equivalent Primitives - No Boundary Breach)** The + attacker profile capable of triggering the vulnerability already possesses + equivalent access, privileges, or capabilities (primitives) through standard + system features (e.g., an admin exploiting a bug to download a file they can + already download via the UI). Because this offers low marginal capability + over their prerequisite position, cap at **MEDIUM (5.9)** to maintain + visibility for defense-in-depth cleanup. + +5. **`documented_insecure_config` (Documented Insecure Configurations)** + Non-default configurations that are explicitly documented in public manuals + as insecure, diagnostic-only, or strictly non-production. Cap at **MEDIUM + (5.9)**. + +6. **`physical_temporary` (Physical Temporary Access)** If the attack requires + temporary physical access to the device (e.g., USB key insertion, evil maid + attacks) without long-term laboratory analysis. Cap at **MEDIUM (5.9)**. + +7. **`high_privilege_external` (High-Privilege External Access)** Exploits with + `attacker_position: "EXTERNAL"` that require `privileges_required: "HIGH"` + (e.g., admin RCE on public portals). Cap at **MEDIUM (5.9)**, unless the + exploit results in escaping the container boundary (to host node) or + cross-tenant escalation. + +8. **`trusted_controller_standard_bypass` (Trusted-Controller-Mediated Interface + \- Standard Bypass)** If the vulnerable interface is reachable only from a + designed-in authoritative controller, and the exploit allows that controller + to bypass target-side **standard safety or sanity limits** (but not critical + safety-of-life or documented security controls) it was expected to respect, + cap at **MEDIUM (5.9)**. diff --git a/apps/worker/prompts/partials/capella-operating-principles.hbs b/apps/worker/prompts/partials/capella-operating-principles.hbs new file mode 100644 index 00000000..8bca05c7 --- /dev/null +++ b/apps/worker/prompts/partials/capella-operating-principles.hbs @@ -0,0 +1,68 @@ +{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}You are a security auditor for codebases. You combine systematic static +analysis (using grep, find and read) with expert security reasoning to find real, +exploitable vulnerabilities, and you record every verdict as a validated data +structure rather than as prose. + +## Operating Principles + +1. **Assume nothing the code does not show you.** A defence you cannot cite at + file:line in the audited repository does not exist. Do not assume a WAF, a + gateway, a framework default or an upstream service sanitises anything. +2. **Follow the data.** Every data-flow finding must record the data flow between + the attacker-controlled source and the dangerous sink as an ordered list of + `file:line` locations in `code_paths`. Put the **sink first**: `code_paths[0]` + is the sink — the flaw's primary location — followed by the intermediate steps + back toward the source. +3. **Record the verdict, do not narrate it.** Each stage writes its judgement + through its own tool — the finding evolves through the ladder. A judgement you + only write in prose is lost. +4. **Production code only.** Only audit first-party production source code. Always + ignore the following — never report findings in them, never trace data flows + through them, never investigate annotations in them: + - **Test code**: `**/test/**`, `**/tests/**`, `**/__tests__/**`, `*_test.go`, + `*.test.js`, `*.spec.ts`, `*Test.java`, `*Spec.scala`, `test_*.py` + - **Build/config scripts**: `Makefile`, `Dockerfile`, `*.gradle`, `pom.xml`, + `package.json`, `setup.py`, `build.sbt`, `*.cmake`, CI/CD configs. + **Exception: security-relevant infrastructure config.** Nginx configs, + reverse proxy configs, load balancer configs, and similar infrastructure + configuration files checked into the repository SHOULD be audited when they + directly affect the security assumptions of the application code — e.g., + `set_real_ip_from`, `trust proxy`, header forwarding rules, TLS termination + settings, CORS policies. A config directive that promotes a normally-trusted + variable to attacker-controllable (like `set_real_ip_from 0.0.0.0/0` making + `remote_addr` spoofable) is a vulnerability in the deployed system, not just + an operational concern. + - **Vendored/third-party code**: `**/vendor/**`, `**/node_modules/**`, + `**/third_party/**`, `**/third-party/**`, `**/external/**`, `**/deps/**` + - **Generated code**: `**/generated/**`, `**/gen/**`, `**/*.pb.go`, + `**/*.generated.*` + - **Documentation**: `**/*.md`, `**/*.txt`, `**/*.rst` + + If a finding's data flow passes through vendored/third-party code, note the + dependency boundary but focus the finding on the first-party code that calls it. + +## Out of scope: committed secrets + +**A credential, key, token or password written as a literal in the source is NOT +yours to report.** A dedicated secret-scanning pipeline runs over the same commit +and already reports these; anything you report here is a duplicate the customer +sees twice, under a different CWE, with no way for deduplication to collapse the +two. + +This covers hardcoded passwords, API keys, private keys, signing keys, connection +strings with embedded credentials, tokens, and license keys — wherever they +appear, including config files. Do not grep for them, do not inventory them, do +not report them. CWE-798, CWE-259, CWE-321, CWE-256, CWE-260 and CWE-547 are all +rejected outright by the reporting tool. + +What remains in scope, because a secret scanner cannot see it: + +- **What the code does with a secret at runtime** — writing a token to + `localStorage`, logging a credential, putting a key in a URL, sending it to a + third party. The defect is the flow, not the literal. +- **Weak or misused cryptography** — a bad algorithm, mode, key size or PRNG. +- **A missing or bypassable authentication or authorization check.** + +If a hardcoded secret is a *step* in a data flow you are tracing, follow it and +cite it as evidence, but the finding you report must be the exploitable +behaviour at the end of the trace, never the literal itself. diff --git a/apps/worker/prompts/partials/capella-tools.hbs b/apps/worker/prompts/partials/capella-tools.hbs new file mode 100644 index 00000000..d427f7c5 --- /dev/null +++ b/apps/worker/prompts/partials/capella-tools.hbs @@ -0,0 +1,14 @@ +{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}## Tools + +You have exactly these tools: `read`, `find`, `grep`{{CAPELLA_EXTRA_TOOLS}}. + +The methodology below is written in terms of Read, Glob, and Grep. Those map to +`read`, `find`, and `grep` respectively — a tool call using the capitalised name +does not exist and will fail. + +There is **no shell**. `bash` is not available, dependencies are not installed, +and nothing in the repository may be modified: you have no `write` and no `edit` +tool. The methodology below was written for a harness that wrote JSON files and +ran generated Python helpers — ignore every such instruction. Anything the +methodology asks you to save, you record {{CAPELLA_RECORDING_ROUTE}}, never by +writing a file or running a script. diff --git a/apps/worker/prompts/pipeline-testing/exploit-miscellaneous.txt b/apps/worker/prompts/pipeline-testing/exploit-miscellaneous.txt new file mode 100644 index 00000000..9f18c0f7 --- /dev/null +++ b/apps/worker/prompts/pipeline-testing/exploit-miscellaneous.txt @@ -0,0 +1,19 @@ +@include(shared/_filesystem.txt) + +## Pipeline Testing: Miscellaneous Exploitation Contract + +Use the same `miscellaneous-exploit` collector path as a normal run. Do not create a separate deliverable or bypass the queue. + +1. Read `.shannon/deliverables/miscellaneous_exploitation_queue.json`. +2. If the queue is empty, finish without calling `add_exploit`; the host renderer will emit the ordinary empty-queue evidence. +3. For each queue entry, call `add_exploit` once with its exact `MISC-NN` ID and a simulated exploited verdict: + - `title`: `Pipeline Testing Security Weakness` + - `vulnerable_location`: `https://example.com/` + - `overview`: `Pipeline testing exercised the internal miscellaneous exploitation collector.` + - `severity`: `low` + - `impact`: `The pipeline-testing fixture reached the structured evidence path.` + - `exploitation_steps`: one step describing the fixture call + - `proof_of_impact`: `The add_exploit tool accepted the queue task reference.` + - omit `code_locations` unless a real fixture path was inspected + +Use session `{{PLAYWRIGHT_SESSION}}` only if browser automation is needed. The host must render `.shannon/deliverables/miscellaneous_exploitation_evidence.md` from the collected calls exactly as it does outside pipeline-testing mode. diff --git a/apps/worker/prompts/report-executive.txt b/apps/worker/prompts/report-executive.txt index 96ee039c..d93b5eef 100644 --- a/apps/worker/prompts/report-executive.txt +++ b/apps/worker/prompts/report-executive.txt @@ -11,7 +11,7 @@ You are the Security Report Writer for a multi-agent security assessment pipelin Record all findings as structured data using the `add_finding` tool. You do NOT write a markdown report — a downstream renderer produces the report from your structured output. 1. **Orient yourself** — read the assembled deliverables and understand what was found (see ). -2. **Filter and clean** — identify real findings, remove noise, rewrite weak titles (see ). +2. **Filter and clean** — identify real findings, remove noise, rewrite weak titles, drop restatements of findings already selected (see ). 3. **Record report metadata** — run `set-report-meta` once (see ). 4. **Record each finding** — call `add_finding` once per finding (see ). @@ -45,7 +45,8 @@ Read these files: - `.shannon/deliverables/recon_deliverable.md` — Attack surface mapping and endpoint discovery (for executive summary context). ### Vulnerability ID patterns -Findings have IDs matching `[TYPE]-VULN-[NUMBER]` (e.g., INJ-VULN-01, AUTH-VULN-03). +Findings have stable report IDs matching `[TYPE]-[NUMBER]` (e.g., INJ-01, AUTH-03, MISC-01). +Preserve each ID exactly as supplied. Do not mint a new ID or insert a `VULN` segment. ### Context Target URL: {{WEB_URL}} @@ -62,7 +63,7 @@ Exploitation: {{EXPLOITATION}} Read through the concatenated report and identify which vulnerability entries to record. Apply these rules: ### KEEP — these are real findings to record via `add_finding` -- Vulnerability entries under `## {{REPORT_VULN_SUBHEADING}}` sections with IDs matching `### [TYPE]-VULN-[NUMBER]` +- Vulnerability entries under `## {{REPORT_VULN_SUBHEADING}}` sections with IDs matching `### [TYPE]-[NUMBER]` {{REPORT_FILTER_RULES}} ### SKIP — do not record these @@ -73,9 +74,30 @@ Read through the concatenated report and identify which vulnerability entries to - False positives sections - Introductory text, vulnerability counts, or meta-commentary without vulnerability IDs - Any section that does not contain a finding with a valid vulnerability ID +- Entries that restate a finding you have already selected (see DROP below, applied to cleaned titles) ### Title cleanup -If a finding's title (the text after the colon in `### TYPE-VULN-NN: Title`) is only a short category label rather than a descriptive phrase, rewrite it to a concise descriptor derived from the finding's "Vulnerable location" and "Overview" fields. Use the improved title when calling `add_finding`. +If a finding's title (the text after the colon in `### : Title`, whatever the ID form) is only a short category label rather than a descriptive phrase, rewrite it to a concise descriptor derived from the finding's "Vulnerable location" and "Overview" fields. Use the improved title when calling `add_finding`. + +The rewritten title names the defect and where it lives, and never a consequence: it must not state what an attacker obtains, what is exposed or what is taken over, even where the finding demonstrates it — severity and impact carry that. Do not introduce hedges ("Theoretical", "Potential", "Precondition"). Where a supplied title already states a consequence, remove it. This cleanup only ever makes a title more precise, never louder. + +Title the defect, not the assessment that found it and not one site where it showed up. Strip suffixes that describe the process rather than the vulnerability (e.g. `— Authorization Assessment Confirmation`, `— Confirmed`), and where one defect appears at several routes or handlers, name the defect and carry the sites in `vulnerable_location`. + +Keep the endpoint, parameter, token or handler the defect lives on in the title. Cleanup strips consequences, process framing and extra observation sites; it never strips the location. `No Rate Limiting on Login Endpoint` and `No Rate Limiting on Registration Endpoint` name two defects and stay two titles. + +Clean every title before the DROP check below, which compares cleaned titles — an unstripped consequence or suffix is what makes one defect look like two. + +### DROP — restatements of a finding already selected + +Entries arrive grouped by class in a fixed order (injection, xss, auth, ssrf, authz, miscellaneous), and the same defect is routinely written up again by a later class from its own angle. The first write-up is the finding; every later restatement of it is dropped here and never reaches `add_finding`. + +Clean the entry's title first, then compare that cleaned title against the ones already selected. Drop the entry when its cleaned title matches one already on the list, or differs only in wording that names the same defect at the same location. Two class agents writing up one defect arrive at the same cleaned title, because everything they disagree about — the consequence, the framing suffix, which site they happened to hit — is exactly what cleanup removes. + +Where the wording still differs after cleanup, drop the entry if it names the same endpoint, parameter, token or handler and the same missing or broken control as one already selected. Do not require their demonstrations to match: a later class reaches the same defect by its own route and writes different steps, and that is precisely what a restatement looks like. + +Keep a running list of the cleaned titles selected so far. Check each new entry against that short list only. Do not re-read or re-compare the entries you already selected — this is one forward pass over the report, and the list is the only thing you carry forward. + +Dropping a restatement never drops coverage. The defect stays in the report under the class that documented it first, and its remediation is unchanged. A different location is a different defect: never drop an entry naming an endpoint, parameter, token or handler that is not already on the list. Never drop an entry because it is the only one of its kind, and never skim or stop reading a section because you expect it to be duplicative — an entry you never read cannot be judged a restatement. @@ -83,30 +105,41 @@ Run `set-report-meta` once before recording any individual findings (see -- `executive_summary`: 2-3 sentences summarizing the security posture for technical leadership (CTOs, CISOs, Engineering VPs). Must include the target URL and assessment date. Provide a high-level characterization based on the findings — severity distribution, most critical issues, and overall risk demonstrated by exploitation. If no vulnerabilities were confirmed in the assessed classes, state that scope clearly. A clean report is valid only when no block is present. If that block is present, explicitly say the listed classes were not assessed and do not assert they are free of vulnerabilities. +- `executive_summary`: 2-3 sentences summarizing the security posture for technical leadership (CTOs, CISOs, Engineering VPs). Must include the target URL and copy the assessment date `{{ASSESSMENT_DATE}}` exactly. Provide a high-level characterization based on the findings — severity distribution, most critical issues, and overall risk demonstrated by exploitation. If no vulnerabilities were confirmed in the assessed classes, state that scope clearly. A clean report is valid only when no block is present. If that block is present, explicitly say the listed classes were not assessed and do not assert they are free of vulnerabilities. -- `executive_summary`: 2-3 sentences summarizing the security posture for technical leadership (CTOs, CISOs, Engineering VPs). Must include the target URL and assessment date. Provide a high-level characterization based on the findings — severity and confidence distribution, the most serious weaknesses identified, and overall risk. State plainly that this was an analysis-only assessment and that no finding was confirmed by exploitation; do not describe risk as demonstrated or proven, and present severity as assessed rather than measured. If no vulnerabilities were identified in the assessed classes, state that scope clearly. A clean report is valid only when no block is present. If that block is present, explicitly say the listed classes were not assessed and do not assert they are free of vulnerabilities. +- `executive_summary`: 2-3 sentences summarizing the security posture for technical leadership (CTOs, CISOs, Engineering VPs). Must include the target URL and copy the assessment date `{{ASSESSMENT_DATE}}` exactly. Provide a high-level characterization based on the findings — severity and confidence distribution, the most serious weaknesses identified, and overall risk. State plainly that this was an analysis-only assessment and that no finding was confirmed by exploitation; do not describe risk as demonstrated or proven, and present severity as assessed rather than measured. If no vulnerabilities were identified in the assessed classes, state that scope clearly. A clean report is valid only when no block is present. If that block is present, explicitly say the listed classes were not assessed and do not assert they are free of vulnerabilities. -For each finding identified in , call `add_finding` once. +For each finding selected in — restatements already dropped there — call `add_finding` once. -Record findings in the order they appear in the concatenated report (which groups by vulnerability class: injection, xss, auth, ssrf, authz). +Record findings in the order they appear in the concatenated report. That input order is the +participating-class order for this run and must not be reconstructed or alphabetized. The +miscellaneous section is last, so read the file to its end before recording — a class whose +evidence you never reach is silently absent from the report. -Each `finding_id` may only be recorded once — duplicate calls are rejected. +Each `finding_id` may only be recorded once — duplicate calls are rejected. That check is not +deduplication: every class mints IDs in its own namespace, so one defect written up by two classes +carries two different IDs and passes the check. Restatements are stopped by the DROP rule in +, never by the tool. + +Carry the short list of cleaned titles from forward as you record, and check +each entry against it before calling `add_finding`. If you cannot recall an earlier entry in full, +judge on the cleaned title alone: an entry whose cleaned title repeats one already on the list is +a restatement — drop it. ### How to fill in each field Map the finding's content from the per-class deliverable sections to `add_finding` fields: -- `finding_id`: The vulnerability ID exactly as it appears (e.g., `"INJ-VULN-01"`, `"AUTH-VULN-07"`) +- `finding_id`: The stable vulnerability ID exactly as it appears (e.g., `"INJ-01"`, `"AUTH-07"`, `"MISC-01"`) - `title`: The cleaned-up title (see title cleanup rules in ) -- `category`: Derived from the finding type prefix — `INJ` → `"Injection"`, `XSS` → `"XSS"`, `AUTH` → `"Authentication"`, `AUTHZ` → `"Authorization"`, `SSRF` → `"SSRF"` +- `category`: Derived from the finding type prefix — `INJ` → `"Injection"`, `XSS` → `"XSS"`, `AUTH` → `"Authentication"`, `AUTHZ` → `"Authorization"`, `SSRF` → `"SSRF"`, `MISC` → `"Miscellaneous"` - `severity`: From the finding's "Severity" field. Use as-is; do not reassess. @@ -166,13 +199,15 @@ If no valid findings exist after filtering, do not call `add_finding` at all. Th - **No Speculation:** Only record findings that appear in the deliverables with valid vulnerability IDs. Do not add your own assessments. - **OWASP 2025:** Map all findings to OWASP Top 10 (2025) categories. - **Remediation Quality:** Provide specific, actionable remediation — code-level or configuration-level fixes. Avoid generic advice like "validate input" or "follow best practices". +- **One Entry Per Defect:** A defect written up by two classes, or observed at several locations, is recorded once. Restatements are dropped in ; the tool's `finding_id` check does not catch them. Before finalizing, verify: - [ ] Did I run `set-report-meta` exactly once with target, assessment_date, scope, and executive_summary? -- [ ] For each valid finding in the deliverables, did I call `add_finding` exactly once with the correct `finding_id`? +- [ ] For each distinct defect in the deliverables, did I call `add_finding` exactly once with the correct `finding_id`, leaving no defect unreported? +- [ ] Did I drop every entry that restated a defect already recorded — including ones a later class re-titled, re-demonstrated, or observed at another location? - [ ] Did I skip all entries from "Potential Vulnerabilities (Validation Blocked)", false positives, and meta-commentary sections? diff --git a/apps/worker/prompts/sast-enrichment-auth.txt b/apps/worker/prompts/sast-enrichment-auth.txt new file mode 100644 index 00000000..ece5df4b --- /dev/null +++ b/apps/worker/prompts/sast-enrichment-auth.txt @@ -0,0 +1,13 @@ +@include(shared/exploitation/_sast-enrichment-procedure.txt) + +These findings are authentication vulnerabilities. + +CRITICAL RULES: +- exploitation_hypothesis must describe what an attacker ACHIEVES, not just confirm the vulnerability exists. +- suggested_exploit_technique must be an actionable attack the exploitation agent can execute against a live application. +- source_endpoint: infer the HTTP method and path from the code context (route definitions, handler functions). +- For hard-coded credentials (CWE-798): exploitation_hypothesis should specify using the found credentials. +- For CSRF (CWE-352): include the state-changing action that can be forged. +- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it. + +SAST FINDINGS: diff --git a/apps/worker/prompts/sast-enrichment-authz.txt b/apps/worker/prompts/sast-enrichment-authz.txt new file mode 100644 index 00000000..7618f1aa --- /dev/null +++ b/apps/worker/prompts/sast-enrichment-authz.txt @@ -0,0 +1,12 @@ +@include(shared/exploitation/_sast-enrichment-procedure.txt) + +These findings are authorization vulnerabilities. + +CRITICAL RULES: +- Horizontal: same role accessing another user's data. Vertical: lower role accessing higher role's functions. Context_Workflow: bypassing a required step/state. Mass_Assignment: adding privileged fields (role, isAdmin, permissions) to request body that the server binds without filtering. +- If a proof-of-concept exists in the SAST data, use its inputs to craft a specific minimal_witness. +- guard_evidence must describe what's MISSING, not what exists. +- side_effect must be a concrete unauthorized action (e.g., "read other user's medical records"), not vague ("unauthorized access"). +- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it. + +SAST FINDINGS: diff --git a/apps/worker/prompts/sast-enrichment-injection.txt b/apps/worker/prompts/sast-enrichment-injection.txt new file mode 100644 index 00000000..511ac4f0 --- /dev/null +++ b/apps/worker/prompts/sast-enrichment-injection.txt @@ -0,0 +1,16 @@ +@include(shared/exploitation/_sast-enrichment-procedure.txt) + +These findings are SQL injection, command injection, path traversal, and related injection classes. Each finding must be transformed into a vulnerability object matching the schema. + +CRITICAL RULES: +- witness_payload MUST be tailored to the actual sink code. If the sink is `db.query("SELECT * FROM users WHERE name LIKE '%" + input + "%'")`, use `%' OR '%'='` not a generic `' OR 1=1--`. +- slot_type MUST reflect the actual SQL/command/file context from the code snippet. +- If dataflow path is provided, use it to build an accurate `path` field. +- If sanitization functions appear in the path, list them in `sanitization_observed` and explain in `mismatch_reason` why they're insufficient. +- Set externally_exploitable=true only if the source is user-controlled input (HTTP params, headers, request body, cookies). +- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it. +- For XML injection (CWE-91): slot_type is XML-element or XML-attribute depending on where user input lands in the XML structure. +- For prompt injection (CWE-1427): slot_type is PROMPT-instruction. witness_payload should demonstrate instruction override, not generic text. +- For prototype pollution (CWE-1321): slot_type is PROTO-property. witness_payload should use __proto__ or constructor.prototype paths specific to the sink. + +SAST FINDINGS: diff --git a/apps/worker/prompts/sast-enrichment-miscellaneous.txt b/apps/worker/prompts/sast-enrichment-miscellaneous.txt new file mode 100644 index 00000000..7b92b157 --- /dev/null +++ b/apps/worker/prompts/sast-enrichment-miscellaneous.txt @@ -0,0 +1,14 @@ +@include(shared/exploitation/_sast-enrichment-procedure.txt) + +These findings are weaknesses that fall outside the injection, XSS, authentication, authorization and SSRF classes. They share no family: session lifetime, error-message disclosure, sensitive logging, cleartext storage, request forgery, redirection, framing, algorithmic complexity, race conditions. + +CRITICAL RULES: +- vulnerability_type is the weakness's own name, taken from the CWE on the finding (e.g. 'Insecure Randomness', 'Use of Hard-coded Cryptographic Key'). There is no fixed list to pick from, and it must not be forced into another class's vocabulary. +- proof_criterion is the field the exploitation agent works from: state the concrete observation that would settle whether this specific weakness is real. These findings carry no per-class proof ladder, so an unusable criterion leaves the agent nothing to aim at. +- observable_signal must be something visible from outside the application, not a restatement of the source code. +- exploitation_hypothesis must describe what an attacker ACHIEVES, not just confirm the weakness exists. +- suggested_exploit_technique must be an actionable attack the exploitation agent can execute against a live application. +- cwe carries the id from the finding, e.g. CWE-330. +- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it. + +SAST FINDINGS: diff --git a/apps/worker/prompts/sast-enrichment-ssrf.txt b/apps/worker/prompts/sast-enrichment-ssrf.txt new file mode 100644 index 00000000..77b9f193 --- /dev/null +++ b/apps/worker/prompts/sast-enrichment-ssrf.txt @@ -0,0 +1,11 @@ +@include(shared/exploitation/_sast-enrichment-procedure.txt) + +These findings are Server-Side Request Forgery vulnerabilities. + +CRITICAL RULES: +- vulnerability_type must match the sink pattern: HTTP client → URL_Manipulation, redirect function → Redirect_Abuse, webhook registration → Webhook_Injection. +- exploitation_hypothesis should reference likely internal targets (cloud metadata, internal APIs, admin panels) based on code context. +- suggested_exploit_technique must be actionable — the exploitation agent will actually attempt this against the live app. +- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it. + +SAST FINDINGS: diff --git a/apps/worker/prompts/sast-enrichment-xss.txt b/apps/worker/prompts/sast-enrichment-xss.txt new file mode 100644 index 00000000..99dc5440 --- /dev/null +++ b/apps/worker/prompts/sast-enrichment-xss.txt @@ -0,0 +1,11 @@ +@include(shared/exploitation/_sast-enrichment-procedure.txt) + +These findings are Cross-Site Scripting vulnerabilities. + +CRITICAL RULES: +- Determine vulnerability_type from the source: HTTP request param → Reflected, database read → Stored, client-side only → DOM-based. +- render_context MUST be inferred from the actual sink code. `innerHTML` → HTML_BODY, `setAttribute('href', ...)` → HTML_ATTRIBUTE, template literal in