mirror of
https://github.com/KeygraphHQ/shannon.git
synced 2026-09-13 13:39:04 +02:00
feat: Shannon 3.0 Agentic SAST (#433)
* feat(worker): add agentic static analysis Add the ten-stage Agentic SAST pipeline, confined repository tools, model runtime, prompt templates, and SARIF export. Make retries, repair sessions, reduced coverage, usage accounting, and model-output drift durable across Temporal replay and resume. Keep retry diagnostics in their actionable closed vocabulary. Package the Mantis-derived license material with the prompts that require it. * feat(worker): deduplicate static and runtime findings before exploitation Parse Agentic SAST SARIF into typed observations, enrich and route those observations, and reconcile them with pentest findings before exploitation. Publish deterministic exploitation queues with stable lineage, exact-path Git commits, retry-safe manifests, named drop reasons, and confined task formation. Reject duplicate producer IDs before commit and adopt either legal provenance shape after a lost acknowledgement. * feat(config)!: replace vuln_classes with agentic_sast Wire Agentic SAST and reconciliation into the main pipeline, persist their durable state, and add the Miscellaneous finding and exploitation lane. Make scan completion, cancellation, partial outcomes, resume identity, and report recovery use the integrated final workflow contract. Introduce the atomic finalization, ordering, renumbering, compaction, and output services that workflow calls. Keep completed Miscellaneous work and report drafts idempotent across resume, preserve public main's default-on exploit SARIF behavior, and describe stage-fallback candidates without claiming they were exported. BREAKING CHANGE: `vuln_classes` has been removed. Configs containing it now fail validation, and all five core pentest classes run on every scan. Workspaces created by Shannon 2.x cannot be resumed. Finish or discard in-flight scans before upgrading, then start a new workspace name. * perf: overlap static analysis and the Miscellaneous lane with the pentest Run Agentic SAST alongside vulnerability analysis and run Miscellaneous exploitation alongside the specialist exploitation lanes. Keep reconciliation dependent on the completed static-analysis result while preserving parallel work everywhere that has no data dependency. * feat(cli)!: default the scan target and add a JSON error contract List local scans, resolve the active or most recent workspace automatically, and make logs, status, and stop use one canonical scan identity. Add stable machine-readable failures, richer status output, explicit help errors, and seven-day Temporal retention. Treat absent Temporal pending-activity failures as absent whether the decoder represents them as `null` or missing. BREAKING CHANGE: `status --json` now returns a fixed `failureMessage`. Read `partialReasons`, `agenticSast`, and `workflow.log` for diagnostic detail. * feat(logging): trace tool calls and write a log per agent Record complete tool-call arguments in the workflow log and project each agent's events into its own durable log. Add agent listing and agent-specific log tailing while preserving byte-exact output and draining log handles before activities return. * feat(worker): standardize severity and reporting guidance in exploit prompts Give every exploit agent the same status, confidence, severity-reasoning, report-writing, credential-handling, and scope contract. Apply the same task-formation and SAST-enrichment procedure to the Miscellaneous lane. * feat(worker): disclose scan coverage and make reporting auditable Build on the retry-safe finalization foundation to preserve correct identities, source locations, scan dates, partial-coverage limitations, and consistent report JSON, Markdown, SARIF, and PDF output. Report Agentic SAST, reconciliation wall-clock time, stage usage, retry spend, and background work without duplicate or hardcoded totals. Keep report findings canonical, drop cross-class restatements, name enrichment losses, and render the executive-summary narrative in the PDF. * chore(license): attribute Mantis and Pi and refresh the docs Add the final Mantis and Pi notices, license copies, acknowledgements, and residual copyright updates. Update the README, maintained documentation, contributor guidance, and hand-maintained mirrors to describe Agentic SAST, reconciliation, the Miscellaneous lane, current CLI behavior, and the final release contract. Correct stale workspace and container guidance and annotate long-standing internals for maintainers. * fix(logging): treat a slash as a word separator in agent labels * feat(cli)!: rebuild scan status around model work - show Capella stages beneath the concurrent Agentic SAST phase - attach reconciliation time to the class row it feeds - hide completed bookkeeping and the duplicate miscellaneous wrapper - carry validated child-workflow progress into durable parent state - derive the terminal tree and status JSON from the same phase shape BREAKING CHANGE: `status --json` replaces phase `parallel` with `children` and `meta`, adds phase summaries and notes plus agent attachment fields, and removes the `analysis-engines` and `operational-work` phases. * fix(report): drop the empty Critical Findings section from the PDF summary * fix(sast): align Capella export with the submit-time code-path contract The export gate required every code_paths entry to be file:line, but submit only requires the primary sink to be file:line and accepts bare trace steps. A single malformed trace step therefore dropped an otherwise-valid finding at export. - add isValidPrimaryCodePath as the one shared primary-sink contract - validate only the primary at export; buildResult already drops unusable steps - route the submit-time validator through the same helper so the two cannot drift * feat(sast): tolerate hygiene-only Capella reductions instead of going partial A reduction only makes a run partial when it loses real coverage or a whole finding. Malformed model output, salvaged turn-limit work, and rejected duplicate verdicts are recorded as evidence but no longer flip the run to partial. - add reductionIsTolerable: partial only when genuine-loss counts are nonzero - drive runCapella's partial reasons and display coverage off non-tolerable ones - keep every reduction in agenticSast.reductions so nothing is lost as evidence * feat(logging): record the provider reason for a failed agent turn A failed provider turn collapsed to AGENT_EXECUTION_FAILED/unknown with the underlying reason discarded, so a model-side rejection or safeguard was indistinguishable from a transport fault in the error log. - add safeProviderTurnDetails: write bounded, non-sensitive fields (provider, model, responseId, stop reason, tool-in-flight, category, retryable) to error.log - gate a sanitized errorMessage snippet behind SHANNON_DEBUG_PROVIDER_ERRORS, off by default - forward SHANNON_DEBUG_PROVIDER_ERRORS from the CLI into the worker container * fix(cli): keep shannon logs tailing through a Temporal blip - End the interactive tail on the log's own terminal marker or Ctrl-C, so a transient Temporal outage no longer aborts the command with exit 1. - Rebuild the memoized Temporal client after a failed poll: a wedged gRPC channel was cached forever, so "retrying…" could never reconnect. - Keep start --follow (CI) bounded — a genuinely dead Temporal still fails the run instead of hanging. * fix(worker): correct PDF finding reporting - Render OWASP category, authentication state, and remediation - Omit the redundant per-finding exploited status - Preserve canonical category and field ordering across report modes - Continue Proof of Impact numbering across embedded code blocks - Wrap long PDF code lines without changing canonical report content * fix: attribute a reconciliation failure to exploitation only - Stop marking a class's vulnerability-analysis agent failed when that agent succeeded and only reconciliation failed; the status tree now renders the analysis row completed and the exploitation row failed - Consume the worker's failedReconciliations signal in the CLI, which the mirrored PipelineState already declared but never read - Correct the class_reconciliation_failed message, which claimed the class's analysis results were still in the report when the class is excluded from it * fix(pi): give each task sub-session its own resource loader to prevent stale extension ctx * fix(prompts): scope exploit agents to in-band proof, mark OOB-only findings blocked * fix(cli): reject a shell credential that shadows a gateway config.toml key * fix(cli): make scan shutdown verifiable - preselect and persist workflow identity before worker launch - cancel first, then verify bounded Temporal termination - reconcile Docker workers with Temporal open workflows - fail closed on stale images and unavailable lifecycle state - mark cancellation only after confirmed shutdown * feat(cli): prompt for setup on a bare npx invocation with no credentials * fix(cli): don't blame anthropic when no credentials are configured at all * chore(release): bump beta base version to 3.0.0 * feat(cli): show a 'start your first scan' box in help on a TTY * docs: refresh README and platform overview for Shannon 3.0 - lead with the 3.0 launch note and rewrite key capabilities around security code analysis, the rebuilt terminal experience, native CI/CD, and PDF/SARIF - recast the editions table as Shannon Open Source against the Keygraph Enterprise Platform, stating open source is not a trial edition - rewrite the platform overview around exhaustive agentic SAST, canonical findings, automated remediation, targeted verification, and governance - add five product screenshots under assets/keygraph-platform/, referenced relative to docs/ * docs: add the Shannon naming section and swap in the 3.0 demo GIF - explain the Claude Shannon information-theory origin under "What is Shannon?" - point "Shannon in Action" at the 3.0 recording in assets/Shannon3GIF.gif Both taken from the README half of #438. * docs: document CI/CD integrations and the reconciled analysis pipeline - add a CI/CD Integrations section covering the official GitHub Action and GitLab component, pipeline artifacts, and exploit-only severity gates - redraw the architecture section as a Mermaid flow: agentic code analysis and recon feed finding reconciliation, then exploitation and reporting - describe open-source code analysis as a multi-stage agentic workflow and reserve parsed-code CPGs and exhaustive verification for Enterprise - sharpen the privacy wording: results stay local, but model requests carry source context to whichever endpoint you configure - drop the "not recommended" framing on local models and add a section on why Shannon complements rather than replaces human pentesters - regenerate llms-full.txt from the updated README and docs * docs: add the Photoview benchmark across three models - Add a "Shannon in Action" table for Photoview 2.4.0 runs on DeepSeek v4 Flash, Grok 4.6, and Claude Opus 5, each linking its PDF report and SARIF output - Store the per-model reports under benchmark/ - Link the (forthcoming) benchmark writeup from the section intro * docs: add the Shannon vs XBOW/Aikido Photoview benchmark writeup - Add docs/shannon-xbow-aikido-benchmark.md with methodology, per-model cost/coverage tables, and links to each model's report and SARIF - Link the writeup from the README "Shannon in Action" section * docs: link the benchmark announcement discussion from the README * fix(readme): restore theme-aware banner, badge, and buttons * feat!: trigger the Shannon 3.0 major release --------- Co-authored-by: ezl-keygraph <ezhil@keygraph.io>
This commit is contained in:
co-authored by
ezl-keygraph
parent
6108de3cfc
commit
9767ebe633
@@ -18,6 +18,7 @@ xben-benchmark-results/
|
||||
# Development files
|
||||
*.md
|
||||
!CLAUDE.md
|
||||
!THIRD_PARTY_NOTICES.md
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
@@ -69,4 +70,5 @@ coverage/
|
||||
docs/
|
||||
README.md
|
||||
LICENSE
|
||||
!LICENSE
|
||||
CHANGELOG.md
|
||||
@@ -30,16 +30,16 @@ jobs:
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
BASE="2.0.0"
|
||||
BASE="3.0.0"
|
||||
LATEST=$(npm view "@keygraph/shannon" dist-tags.beta 2>/dev/null || echo "")
|
||||
|
||||
if [[ "$LATEST" == "$BASE-beta."* ]]; then
|
||||
# Same base version — increment the beta counter (e.g. 2.0.0-beta.2 -> 2.0.0-beta.3)
|
||||
# Same base version — increment the beta counter (e.g. 3.0.0-beta.2 -> 3.0.0-beta.3)
|
||||
N=$(echo "$LATEST" | grep -oE 'beta\.([0-9]+)' | grep -oE '[0-9]+')
|
||||
NEXT=$((N + 1))
|
||||
echo "version=$BASE-beta.$NEXT" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
# No prior beta, or a different base (e.g. last beta was 1.0.0-beta.N) — start over.
|
||||
# No prior beta, or a different base (e.g. last beta was 2.0.0-beta.N) — start over.
|
||||
echo "version=$BASE-beta.1" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: "Beta version to roll back to (example: 2.0.0-beta.2)"
|
||||
description: "Beta version to roll back to (example: 3.0.0-beta.2)"
|
||||
required: true
|
||||
type: string
|
||||
|
||||
@@ -31,7 +31,7 @@ jobs:
|
||||
VERSION="${RAW_VERSION#v}"
|
||||
|
||||
if ! [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+-beta\.[0-9]+$ ]]; then
|
||||
echo "Version must be in format X.Y.Z-beta.N (e.g. 2.0.0-beta.2)"
|
||||
echo "Version must be in format X.Y.Z-beta.N (e.g. 3.0.0-beta.2)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
@@ -43,6 +43,9 @@ RUN rm -rf node_modules apps/*/node_modules && pnpm install --frozen-lockfile --
|
||||
# Runtime stage - Minimal production image
|
||||
FROM cgr.dev/chainguard/wolfi-base:latest AS runtime
|
||||
|
||||
# Lifecycle protocol consumed by the CLI before it trusts a container workflow-id label.
|
||||
LABEL shannon.worker-protocol="workflow-id-v1"
|
||||
|
||||
# Install only runtime dependencies
|
||||
USER root
|
||||
RUN apk update && apk add --no-cache \
|
||||
@@ -109,6 +112,10 @@ COPY --from=builder /app/node_modules /app/node_modules
|
||||
COPY --from=builder /app/apps/worker /app/apps/worker
|
||||
COPY --from=builder /app/apps/cli/package.json /app/apps/cli/package.json
|
||||
|
||||
# Third-party license and notice material travels with the distributed image
|
||||
COPY LICENSE THIRD_PARTY_NOTICES.md /usr/share/licenses/shannon/
|
||||
COPY LICENSES/ /usr/share/licenses/shannon/LICENSES/
|
||||
|
||||
RUN npm install -g --ignore-scripts @playwright/cli@0.1.1
|
||||
RUN mkdir -p /tmp/.claude/skills && \
|
||||
playwright-cli install --skills && \
|
||||
|
||||
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Mario Zechner
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -1,5 +1,5 @@
|
||||
> [!NOTE]
|
||||
> **[Shannon 2.0 is officially here](https://github.com/KeygraphHQ/shannon/discussions/405)**
|
||||
> **Shannon 3.0 is live:** deeper security code analysis, a rebuilt terminal experience, native CI/CD workflows, professional PDF reports, and SARIF—still fully open source, self-hosted, and bring-your-own-model.
|
||||
|
||||
<div align="center">
|
||||
|
||||
@@ -11,9 +11,9 @@
|
||||
|
||||
<a href="https://trendshift.io/repositories/15604" target="_blank"><img src="https://trendshift.io/api/badge/repositories/15604" alt="KeygraphHQ%2Fshannon | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
|
||||
### Shannon is an autonomous, AI pentester for web applications and APIs.
|
||||
### Shannon is an autonomous, AI pentester for web applications and APIs.
|
||||
|
||||
It analyzes your source code, identifies attack paths, and executes real exploits to prove vulnerabilities before they reach production.
|
||||
It analyzes your source code, identifies attack paths, and executes real exploits to prove vulnerabilities before they reach production. **No exploit, no report.**
|
||||
|
||||
**This repository is Shannon Open Source: the full agent, run locally from your command line.**
|
||||
|
||||
@@ -28,12 +28,22 @@ It analyzes your source code, identifies attack paths, and executes real exploit
|
||||
> [!TIP]
|
||||
> **AI agents and LLMs:** start with [llms.txt](llms.txt) for a concise map of this repository, or use [llms-full.txt](llms-full.txt) for the README and docs combined into one file.
|
||||
|
||||
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Table of Contents](#table-of-contents)
|
||||
- [What is Shannon?](#what-is-shannon)
|
||||
- [Why Shannon Exists](#why-shannon-exists)
|
||||
- [Why "Shannon"?](#why-shannon)
|
||||
- [Not a replacement for human pentesters](#not-a-replacement-for-human-pentesters)
|
||||
- [Shannon in Action](#shannon-in-action)
|
||||
- [Quick Start](#quick-start)
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Run Shannon](#run-shannon)
|
||||
- [Key Capabilities](#key-capabilities)
|
||||
- [CI/CD Integrations](#cicd-integrations)
|
||||
- [GitHub Actions](#github-actions)
|
||||
- [Editions](#editions)
|
||||
- [Architecture](#architecture)
|
||||
- [Documentation](#documentation)
|
||||
@@ -42,6 +52,14 @@ It analyzes your source code, identifies attack paths, and executes real exploit
|
||||
- [About Keygraph](#about-keygraph)
|
||||
- [Community and Support](#community-and-support)
|
||||
- [Common Questions](#common-questions)
|
||||
- [Can I self-host Shannon?](#can-i-self-host-shannon)
|
||||
- [Does Shannon support bring your own key (BYOK)?](#does-shannon-support-bring-your-own-key-byok)
|
||||
- [Does Shannon output SARIF?](#does-shannon-output-sarif)
|
||||
- [Which AI providers does Shannon support?](#which-ai-providers-does-shannon-support)
|
||||
- [Can I run Shannon on a local or self-hosted model?](#can-i-run-shannon-on-a-local-or-self-hosted-model)
|
||||
- [Does Shannon actually exploit vulnerabilities, or just scan?](#does-shannon-actually-exploit-vulnerabilities-or-just-scan)
|
||||
|
||||
|
||||
|
||||
## What is Shannon?
|
||||
|
||||
@@ -57,22 +75,43 @@ Thanks to tools like Claude Code and Cursor, your team ships code non-stop. But
|
||||
|
||||
Shannon closes that gap by providing on-demand, automated penetration testing that can run against every build or release.
|
||||
|
||||
### Why "Shannon"?
|
||||
|
||||
It's named after Claude Shannon, the father of information theory. At its core, pentesting is an information problem: every probe reduces uncertainty about a system's state. The best tools maximize the signal gained from every request, turning those bits of knowledge into an exploit path.
|
||||
|
||||
Also, we wanted you to be able to say, "Hey Claude, run Shannon" to find all the security flaws in your vibe-coded app.
|
||||
|
||||
### Not a replacement for human pentesters
|
||||
|
||||
Shannon is built to work alongside expert pentesters and red teamers, not replace them. Great pentesters understand the business, chain attacks in ways nobody anticipated, and bring years of judgment that current models can't match.
|
||||
|
||||
Shannon solves a different problem: there is far more software to test than security teams have time to cover. Critical systems get periodic expert assessments, while the long tail of internal apps, APIs, and fast-moving services rarely gets tested at all.
|
||||
|
||||
Shannon shifts pentesting left into the software development lifecycle (SDLC). Use it to run exploitation-backed tests against staging environments and releases at the cadence they actually ship, and save expert human time for the risks that need someone who knows the organization.
|
||||
|
||||
## Shannon in Action
|
||||
|
||||
<p align="center">
|
||||
<img src="assets/shannon-action.gif" alt="Shannon running an autonomous pentest" width="100%">
|
||||
</p>
|
||||

|
||||
|
||||
Penetration test reports from Shannon Open Source scanning Photoview 2.4.0. Read the [announcement][announcement] and the full [benchmark writeup][benchmark] for methodology, cost, and the comparison against Aikido and XBOW.
|
||||
|
||||
|
||||
| Model | Report | SARIF |
|
||||
| ----------------- | ---------------------------------------------------------------------------- | --------------------------------------------------------------- |
|
||||
| DeepSeek v4 Flash | [View report](benchmark/photoview-deepseek-v4-flash.pdf) | [SARIF](benchmark/photoview-deepseek-v4-flash.sarif) |
|
||||
| Grok 4.6 | [View report](benchmark/photoview-grok-4-6.pdf) | [SARIF](benchmark/photoview-grok-4-6.sarif) |
|
||||
| Claude Opus 5 | [View report](benchmark/photoview-opus-5.pdf) | [SARIF](benchmark/photoview-opus-5.sarif) |
|
||||
|
||||
[announcement]: https://github.com/KeygraphHQ/shannon/discussions/439
|
||||
[benchmark]: docs/shannon-xbow-aikido-benchmark.md
|
||||
|
||||
|
||||
Sample penetration test reports from intentionally vulnerable applications, produced by Shannon Open Source:
|
||||
|
||||
| Target | Summary | Report |
|
||||
| --- | --- | --- |
|
||||
| OWASP Juice Shop | 20+ vulnerabilities, including authentication bypass, SQL injection, IDOR, and SSRF. | [View report](sample-reports/shannon-report-juice-shop.md) |
|
||||
| c{api}tal API | Approximately 15 critical and high-severity API findings, including command injection, auth bypass, and mass assignment. | [View report](sample-reports/shannon-report-capital-api.md) |
|
||||
| OWASP crAPI | 15+ critical and high-severity findings across JWT, injection, SSRF, and API authorization paths. | [View report](sample-reports/shannon-report-crapi.md) |
|
||||
|
||||
## Quick Start
|
||||
|
||||
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **Docker**: required for the worker container.
|
||||
@@ -80,6 +119,8 @@ Sample penetration test reports from intentionally vulnerable applications, prod
|
||||
- **AI provider credentials**: Shannon runs on Anthropic, OpenAI, xAI, AWS Bedrock, [any other provider](docs/ai-providers.md#any-other-provider) in the harness catalogue, and any endpoint that speaks the Anthropic Messages API or the OpenAI Chat Completions or Responses API through a [custom base URL](docs/ai-providers.md#custom-base-url). You bring your own key, and Keygraph never proxies your model traffic. Shannon is provider-agnostic. See [AI providers](docs/ai-providers.md#suggested-models) for suggested model IDs.
|
||||
- **Cyber safeguards cleared with your provider**: Anthropic and OpenAI apply real-time safeguards to cyber-security workloads, which can interrupt a scan mid-run. Complete their guidance for legitimate security testers before your first run - see [AI providers](docs/ai-providers.md#cyber-safeguards-do-this-before-your-first-scan).
|
||||
|
||||
|
||||
|
||||
### Run Shannon
|
||||
|
||||
> [!WARNING]
|
||||
@@ -87,10 +128,12 @@ Sample penetration test reports from intentionally vulnerable applications, prod
|
||||
|
||||
```bash
|
||||
# Configure credentials with the interactive wizard.
|
||||
npx @keygraph/shannon setup
|
||||
npx @keygraph/shannon@latest setup
|
||||
|
||||
# Run a pentest against a source-available target.
|
||||
npx @keygraph/shannon start -u https://your-app.com -r /path/to/your-repo
|
||||
npx @keygraph/shannon@latest start \
|
||||
-u https://your-app.com \
|
||||
-r /path/to/your/repo
|
||||
```
|
||||
|
||||
Shannon pulls the worker image from Docker Hub, starts the required local infrastructure, mounts the target repository read-only inside an ephemeral worker container, and writes results to a local workspace.
|
||||
@@ -104,91 +147,131 @@ For source builds, authenticated scans, provider-specific setup, and platform no
|
||||
> - **xAI (Grok):** The latest version of Shannon supports xAI subscriptions. Follow the [xAI subscription setup guide](docs/ai-providers.md#xai-grok-subscription) to get started.
|
||||
> - **Claude Code:** The latest version of Shannon does not support Claude Code subscriptions. Follow the [Claude Code subscription setup guide](docs/ai-providers.md#claude-code-subscription) to use version `1.9.0`, which is the final release built on the Claude Agent SDK.
|
||||
|
||||
|
||||
|
||||
## Key Capabilities
|
||||
|
||||
- **Proof-by-exploitation reports**: Shannon reports validated findings with reproducible proof-of-concept steps instead of speculative warnings.
|
||||
- **White-box attack planning**: Shannon uses source-code analysis to guide dynamic testing and focus on realistic attack paths.
|
||||
- **No exploit, no report**: Shannon includes a vulnerability only after validating it with a working, reproducible proof of concept—eliminating the speculative warnings typical of scanners.
|
||||
- **Advanced security code analysis**: Before it sends a single payload, Shannon reads the codebase and builds a picture of the application: architecture, trust boundaries, exposed interfaces, data flows, and the assets worth attacking. From there it opens targeted investigations and filters the candidates they turn up. What survives goes to the live pentesting agents.
|
||||
- **Autonomous execution**: Shannon launches reconnaissance, vulnerability analysis, exploitation, and report generation from a single command.
|
||||
- **Live terminal experience**: A rebuilt CLI makes scans easy to configure and shows agent progress and clean results without requiring operators to inspect the underlying orchestration logs.
|
||||
- **Authenticated testing**: configuration files can describe login flows, test credentials, TOTP, email-based login flows, focus areas, and rules of engagement.
|
||||
- **OWASP-focused coverage**: Shannon targets exploitable Injection, XSS, SSRF, Broken Authentication, and Broken Authorization issues.
|
||||
- **Resumable workspaces**: Shannon can resume interrupted runs without re-running completed agents.
|
||||
- **Machine-readable output**: Shannon emits findings as structured JSON, and as SARIF 2.1.0 by default on exploit-mode scans (opt out with `report.sarif: "false"`). SARIF is the OASIS standard for static analysis results, so findings flow into any code scanning service, vulnerability management platform, security dashboard, or CI/CD pipeline that reads it.
|
||||
- **Bring your own key, provider-agnostic**: Shannon runs on Anthropic, OpenAI, xAI, AWS Bedrock, and any endpoint speaking the Anthropic Messages API or the OpenAI Chat Completions or Responses API, including self-hosted models served through Ollama, vLLM, or LM Studio and gateways such as OpenRouter and LiteLLM. You supply the credentials, so source code and model traffic stay inside your infrastructure. Local and self-hosted models are technically supported but not recommended: they may not follow Shannon's instructions or tool-use constraints as reliably as frontier models, so take that path only if you know how your chosen model behaves.
|
||||
- **Native CI/CD integrations**: Run Shannon through the official GitHub Action or reusable GitLab CI/CD component. Preserve reports, SARIF, and logs as pipeline artifacts; publish findings into native security workflows; and gate releases only on vulnerabilities Shannon actually demonstrates.
|
||||
- **Professional and machine-readable reports**: Shannon generates evidence-rich PDF and Markdown reports plus structured JSON and SARIF 2.1.0. SARIF is enabled by default on exploit-mode scans and can be disabled with `report.sarif: "false"`.
|
||||
- **Bring your own key, provider-agnostic**: Shannon runs on Anthropic, OpenAI, xAI, AWS Bedrock, and any endpoint speaking the Anthropic Messages API or the OpenAI Chat Completions or Responses API, including self-hosted models served through Ollama, vLLM, or LM Studio and gateways such as OpenRouter and LiteLLM. You supply the credentials and choose exactly where model traffic goes. Local and self-hosted models are supported.
|
||||
- **Private by design**: Shannon runs inside your infrastructure and writes results to a local workspace. Model requests go straight to the provider or endpoint you configure, and they carry source and application context with them, so choose that endpoint deliberately. Point Shannon at a local model endpoint and nothing leaves your environment.
|
||||
|
||||
|
||||
|
||||
## CI/CD Integrations
|
||||
|
||||
Shannon can run continuously against deployed staging and development environments through official integrations for [GitHub Actions](https://github.com/KeygraphHQ/shannon-action) and [GitLab CI/CD](https://gitlab.com/KeygraphHQ/shannon-ci).
|
||||
|
||||
Both integrations:
|
||||
|
||||
- analyze the checked-out source repository while attacking a running target;
|
||||
- preserve PDF, Markdown, and SARIF reports as pipeline artifacts;
|
||||
- preserve scan and agent logs for debugging, including incomplete runs;
|
||||
- support pull-request, release, and scheduled pentests;
|
||||
- distinguish an incomplete assessment from a completed scan with no findings; and
|
||||
- optionally fail the pipeline when Shannon exploits a vulnerability at or above a configured severity threshold.
|
||||
|
||||
A code-analysis hypothesis does not fail the pipeline. Severity gates count only findings with `status: exploited`.
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
name: Shannon Pentest
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
security-events: write
|
||||
|
||||
jobs:
|
||||
pentest:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Run Shannon
|
||||
uses: KeygraphHQ/shannon-action@v1
|
||||
with:
|
||||
url: https://staging.example.com
|
||||
api-key: ${{ secrets.SHANNON_AI_API_KEY }}
|
||||
fail-on-severity: high
|
||||
upload-sarif: true
|
||||
```
|
||||
|
||||
The Action defaults `repo` to the checked-out GitHub workspace. It uploads one artifact containing the security assessment reports and SARIF, plus a separate run artifact containing scan and agent logs. Enabling `upload-sarif` publishes supported findings to GitHub code scanning.
|
||||
|
||||
Requirements:
|
||||
|
||||
- a private repository;
|
||||
- a runner with Docker and Docker Compose v2;
|
||||
- access to the running staging or development target; and
|
||||
- a model-provider credential stored as a GitHub Actions secret.
|
||||
|
||||
See the [Shannon GitHub Action documentation](https://github.com/KeygraphHQ/shannon-action) and [GitHub Marketplace listing](https://github.com/marketplace/actions/shannon-ai-pentester).
|
||||
|
||||
## Editions
|
||||
|
||||
Shannon ships in two ways: **Shannon Open Source**, the pentester you run yourself, and the **Keygraph platform**, the commercial pentesting product that runs Shannon continuously and closes the full AppSec lifecycle around it.
|
||||
**Shannon Open Source** is the complete autonomous pentester for developers and security teams. It is optimized for fast local and CI/CD runs: understand the application, execute real attacks, and report only proven vulnerabilities.
|
||||
|
||||
**Shannon Open Source** (this repository) is the standalone pentester: a CLI agent for white-box, proof-by-exploitation testing of web applications and APIs you own or are authorized to test. It reads your source, plans attacks, executes real exploits, and reports only what it can prove. It runs on demand and is complete in that lane. You point it at a target, it pentests, it reports.
|
||||
**Keygraph Enterprise Platform** turns Shannon's proof engine into an organization-wide AppSec program, adding exhaustive analysis, centralized vulnerability management, automated remediation, enterprise governance, and continuous operation at scale.
|
||||
|
||||
The **Keygraph platform** is the enterprise-ready, continuous pentesting product powered by Shannon. In the Keygraph platform, an enhanced build of Shannon runs continuously in a hardened, orchestrated environment fed by Keygraph's full code-analysis stack. Around that engine, the platform closes the entire vulnerability lifecycle, from analysis to a verified fix:
|
||||
|
||||
- **Analyze**: Code Property Graph SAST, SCA with reachability, secrets, IaC, and container scanning. First-class detection in their own right, and context that sharpens Shannon's attacks.
|
||||
- **Prove**: autonomous black-box and source-aware white-box pentests turn candidate findings into proven, exploited vulnerabilities rather than speculative alerts.
|
||||
- **Manage**: one canonical record per vulnerability per repository, deduplicated across every source, with ownership, status, SLA tracking, dashboards, and bidirectional Jira sync.
|
||||
- **Remediate and verify**: patches written automatically and re-tested against the patched code before delivery, landing in your existing review workflow rather than auto-applied.
|
||||
- **Deploy**: self-hosted and air-gapped environments, strict bring-your-own-key model access, and customer-controlled LLM gateway patterns, so source, results, and model traffic stay inside your perimeter.
|
||||
| | Shannon Open Source | Keygraph Enterprise Platform |
|
||||
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Best for | Local and CI/CD pentesting | Continuous AppSec across teams and repositories |
|
||||
| Security analysis | Multi-stage agentic review models architecture, trust boundaries, and data flows, filters candidate vulnerabilities, and hands the survivors to live pentesting agents | Exhaustive parsed-code agentic SAST: persistent Code Property Graphs, interprocedural source-to-sink and sanitizer modeling, cross-repository context, exploit-chain analysis, and business-logic testing |
|
||||
| Additional coverage | Not included | SCA with reachability, secrets scanning, and business-logic testing |
|
||||
| AppSec operations | N/A — standalone CLI | Canonical findings, deduplication, SLAs, analytics, automated remediation, and targeted verification |
|
||||
| Governance | N/A — local, single-operator CLI | SSO, SCIM, granular access control, APIs, and full audit logging |
|
||||
| Deployment | Self-hosted, air-gapped, BYOM, AGPL-3.0 | On-premises or air-gapped, granular model routing, commercial support |
|
||||
|
||||
Shannon is the proof engine at the center of the Keygraph platform. Shannon Open Source gives you that engine to run yourself. The Keygraph platform surrounds Shannon with continuous analysis, finding management, remediation, verification, and enterprise deployment.
|
||||
|
||||
| AppSec lifecycle stage | Shannon Open Source | Keygraph platform |
|
||||
| --- | --- | --- |
|
||||
| Analyze | Basic LLM pass-through of source to plan attacks | Actual code-base parsing, plus Code Property Graph, SAST, SCA with reachability, secrets, IaC, and containers |
|
||||
| Pentest and prove | White-box only, proof by exploitation | Enhanced white-box, plus black-box and grey-box modes, run continuously |
|
||||
| Manage findings | Local Markdown report | Canonical findings system: deduplication across sources, ownership, SLA, dashboards, Jira sync, and professional pentest-grade PDF reports |
|
||||
| Remediate and verify | Fix manually from the report, then re-run the full scan to verify | Automated remediation: opens a PR with the fix, verified by point re-test without re-running the full scan |
|
||||
| Deploy and operate | Local CLI and Docker worker | Self-hosted, air-gapped, BYOK, continuous, enterprise integrations |
|
||||
| License and support | AGPL-3.0, community | Commercial, supported |
|
||||
Shannon Open Source is not a trial edition. Choose Keygraph Enterprise when you need deeper analysis and a governed, closed-loop AppSec program.
|
||||
|
||||
Learn more on the [Keygraph website](https://keygraph.io), read the [Keygraph platform technical overview](docs/keygraph-platform.md), start a free trial or book a [demo](https://cal.com/team/keygraph/shannon-pro), or contact [shannon@keygraph.io](mailto:shannon@keygraph.io).
|
||||
[Explore the Keygraph Enterprise Platform →](docs/keygraph-platform.md)
|
||||
|
||||
## Architecture
|
||||
|
||||
Shannon uses a multi-agent workflow that combines source-code analysis with live exploitation:
|
||||
Shannon combines multi-stage security code analysis with live reconnaissance and exploitation:
|
||||
|
||||
```text
|
||||
┌──────────────────────┐
|
||||
│ Pre-Reconnaissance │
|
||||
│ (source code scan) │
|
||||
└──────────┬───────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────┐
|
||||
│ Reconnaissance │
|
||||
│ (attack surface │
|
||||
│ mapping) │
|
||||
└──────────┬───────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────┴───────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌───────────┐ ┌───────────┐ ┌───────────┐
|
||||
│ Vuln │ │ Vuln │ │ ... │
|
||||
│(Injection)│ │ (XSS) │ │ │
|
||||
└─────┬─────┘ └─────┬─────┘ └─────┬─────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌───────────┐ ┌───────────┐ ┌───────────┐
|
||||
│ Exploit │ │ Exploit │ │ ... │
|
||||
│(Injection)│ │ (XSS) │ │ │
|
||||
└─────┬─────┘ └─────┬─────┘ └─────┬─────┘
|
||||
│ │ │
|
||||
└──────┬───────┴─────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────┐
|
||||
│ Reporting │
|
||||
└──────────────────────┘
|
||||
```mermaid
|
||||
flowchart TD
|
||||
S["Source code"] --> EXISTING["Recon + vulnerability analysis"]
|
||||
S --> SAST["Agentic security code analysis"]
|
||||
|
||||
EXISTING -- "Pentest candidates" --> REC["Finding reconciliation<br/>(merge + deduplicate)"]
|
||||
SAST -- "SAST candidates" --> REC
|
||||
|
||||
REC -- "Reconciled exploitation queue" --> EXP["Exploitation agents"]
|
||||
APP["Running application"] --> EXP
|
||||
|
||||
EXP -- "Exploit demonstrated" --> REPORT["Reporting<br/>PDF · Markdown · SARIF"]
|
||||
EXP -- "No exploit demonstrated" --> DROP["Discard"]
|
||||
|
||||
REPORT --> CICD["CI/CD gate"]
|
||||
```
|
||||
|
||||
At a high level:
|
||||
|
||||
- **Pre-reconnaissance** identifies frameworks, entry points, data flows, and likely attack surfaces from the repository.
|
||||
- **Reconnaissance** explores the live application and correlates runtime behavior with code-level context.
|
||||
- **Vulnerability analysis** runs specialized agents for Injection, XSS, SSRF, Authentication, and Authorization.
|
||||
- **Exploitation** attempts real proof-of-concept attacks and discards hypotheses that cannot be proven.
|
||||
- **Reporting** compiles validated findings, evidence, and remediation guidance into a final Markdown report.
|
||||
|
||||
Stage by stage:
|
||||
|
||||
1. **Recon and vulnerability analysis** explores the running application, ties runtime behavior back to the source, and runs specialized agents across Injection, XSS, SSRF, Authentication, and Authorization.
|
||||
2. **Agentic security code analysis** maps the application's architecture, trust boundaries, exposed interfaces, dependencies, data flows, and high-risk assets, then opens targeted investigations against them.
|
||||
3. **Finding reconciliation** merges both streams of candidates, deduplicates the overlap, and groups what remains into an exploitation queue.
|
||||
4. **Exploitation agents** attempt real proof-of-concept attacks against the running application.
|
||||
5. **Validation** throws out every candidate Shannon can't demonstrate.
|
||||
6. **Reporting** produces PDF and Markdown reports with the evidence attached, plus structured JSON and SARIF for downstream systems.
|
||||
|
||||
Only live-validated vulnerabilities become Shannon pentest findings or count toward CI/CD severity gates.
|
||||
|
||||
Each scan runs in an ephemeral Docker container with an isolated workspace and per-invocation orchestration.
|
||||
|
||||
@@ -196,16 +279,20 @@ Each scan runs in an ephemeral Docker container with an isolated workspace and p
|
||||
|
||||
Use these guides for operational detail:
|
||||
|
||||
| Guide | Use it for |
|
||||
| --- | --- |
|
||||
| [Source build and CLI commands](docs/development.md) | Cloning, building, common commands, output paths, and local development. |
|
||||
| [Configuration](docs/configuration.md) | Authenticated testing, login flows, rules of engagement, and report filters. |
|
||||
| [AI providers](docs/ai-providers.md) | Selecting the model, the supported providers (Anthropic, OpenAI, xAI, AWS Bedrock, and any other Pi-supported provider), and custom gateways. |
|
||||
| [Platforms and networking](docs/platforms.md) | Windows/WSL2, Linux, macOS, Docker networking, local apps, and custom hostnames. |
|
||||
| [Workspaces and resuming](docs/workspaces.md) | Naming workspaces, resuming interrupted scans, and workspace storage. |
|
||||
| [Safety and limitations](docs/safety.md) | Authorized-use requirements, non-production guidance, mutative effects, cost, and model caveats. |
|
||||
| [Coverage and roadmap](docs/coverage-roadmap.md) | Current vulnerability coverage and planned work. |
|
||||
| [Keygraph platform](docs/keygraph-platform.md) | The continuous, agentic pentesting platform: code analysis, black-box and white-box testing, finding management, remediation, verification, and enterprise deployment. |
|
||||
|
||||
| Guide | Use it for |
|
||||
| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| [Source build and CLI commands](docs/development.md) | Cloning, building, common commands, output paths, and local development. |
|
||||
| [Configuration](docs/configuration.md) | Authenticated testing, login flows, rules of engagement, and report filters. |
|
||||
| [AI providers](docs/ai-providers.md) | Selecting the model, the supported providers (Anthropic, OpenAI, xAI, AWS Bedrock, and any other Pi-supported provider), and custom gateways. |
|
||||
| [Platforms and networking](docs/platforms.md) | Windows/WSL2, Linux, macOS, Docker networking, local apps, and custom hostnames. |
|
||||
| [Workspaces and resuming](docs/workspaces.md) | Naming workspaces, resuming interrupted scans, and workspace storage. |
|
||||
| [Safety and limitations](docs/safety.md) | Authorized-use requirements, non-production guidance, mutative effects, cost, and model caveats. |
|
||||
| [Coverage and roadmap](docs/coverage-roadmap.md) | Current vulnerability coverage and planned work. |
|
||||
| [Keygraph Enterprise Platform](docs/keygraph-platform.md) | Exhaustive agentic SAST, continuous pentesting, full-lifecycle finding management, remediation, targeted verification, enterprise governance, and on-premises deployment. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Safety, Scope, and Limitations
|
||||
|
||||
@@ -215,7 +302,7 @@ You are responsible for using Shannon legally and ethically. Do not point Shanno
|
||||
|
||||
Important limitations:
|
||||
|
||||
- Shannon Open Source focuses on actively exploitable issues such as Injection, XSS, SSRF, Broken Authentication, and Broken Authorization. Broader static-analysis coverage, including vulnerable dependencies and insecure configurations, is delivered through the Keygraph platform.
|
||||
- Shannon Open Source is tuned for fast, code-informed pentesting in everyday development and CI/CD. Exhaustive agentic SAST, broader scanner coverage, centralized governance, and full-lifecycle vulnerability management are delivered through the Keygraph Enterprise Platform.
|
||||
- Findings still require human review. LLM-generated reports can contain weakly supported or incorrect details.
|
||||
- Anthropic, OpenAI, xAI, and AWS Bedrock are built-in providers, and any Anthropic Messages API or OpenAI Chat Completions or Responses API endpoint works through a custom base URL. Model capability varies, and a model that does not follow Shannon's instructions or tool-use constraints reliably will produce weaker results.
|
||||
- A full run can take roughly 1 to 1.5 hours and may incur LLM API costs depending on model pricing and application complexity.
|
||||
@@ -256,11 +343,17 @@ Stay connected:
|
||||
- [Twitter/X: @KeygraphHQ](https://twitter.com/KeygraphHQ)
|
||||
- [LinkedIn: Keygraph](https://linkedin.com/company/keygraph)
|
||||
|
||||
|
||||
|
||||
## Common Questions
|
||||
|
||||
|
||||
|
||||
### Can I self-host Shannon?
|
||||
|
||||
Yes. Shannon Open Source runs entirely on your own infrastructure in an ephemeral Docker container. Your source code is mounted read-only and never leaves your environment.
|
||||
Yes. Shannon Open Source runs inside your infrastructure in an ephemeral worker container. It mounts the repository read-only and writes results to a local workspace.
|
||||
|
||||
Keygraph never receives your source code and never proxies your model traffic. Your model requests go straight to the provider or endpoint you configure, and they carry source and application context with them. Point Shannon at a locally hosted endpoint and that traffic stays inside your environment too.
|
||||
|
||||
### Does Shannon support bring your own key (BYOK)?
|
||||
|
||||
@@ -276,12 +369,10 @@ Anthropic, OpenAI, xAI, and AWS Bedrock are built in and configured directly by
|
||||
|
||||
### Can I run Shannon on a local or self-hosted model?
|
||||
|
||||
Technically yes, but it is not recommended. Shannon works with local models served through Ollama, vLLM, or LM Studio, which expose an OpenAI-compatible endpoint, as well as routers such as OpenRouter and gateways such as LiteLLM. Point Shannon at the endpoint with a custom base URL. Capability varies, and a model that does not follow Shannon's instructions or tool-use constraints reliably will produce weaker pentests than a frontier model, so take this path only if you know how your chosen model behaves. See [AI providers](docs/ai-providers.md#custom-base-url).
|
||||
Shannon works with local models served through Ollama, vLLM, or LM Studio, which expose an OpenAI-compatible endpoint, as well as routers such as OpenRouter and gateways such as LiteLLM. Point Shannon at the endpoint with a custom base URL. Capability varies, and a model that does not follow Shannon's instructions or tool-use constraints reliably will produce weaker pentests than a frontier model, so take this path only if you know how your chosen model behaves. See [AI providers](docs/ai-providers.md#custom-base-url).
|
||||
|
||||
### Does Shannon actually exploit vulnerabilities, or just scan?
|
||||
|
||||
Shannon executes real exploits. It reports a finding only when it has produced a working proof-of-concept, and discards hypotheses it cannot prove. It is a pentester, not a scanner.
|
||||
Shannon executes real exploits. It reports a finding only when it has produced a working proof-of-concept, and discards hypotheses it cannot prove. It is a pentester, not a passive scanner.
|
||||
|
||||
<p align="center">
|
||||
<b>Built by <a href="https://keygraph.io">Keygraph</a></b>
|
||||
</p>
|
||||
**Built by [Keygraph](https://keygraph.io)**
|
||||
@@ -0,0 +1,46 @@
|
||||
# Third-Party Notices
|
||||
|
||||
Shannon incorporates and adapts material from third-party open-source projects.
|
||||
|
||||
Shannon as a whole is distributed under the GNU Affero General Public License,
|
||||
version 3.0 (see LICENSE). Third-party material incorporated into Shannon
|
||||
remains subject to the attribution and notice requirements of its own license.
|
||||
|
||||
## Pi
|
||||
|
||||
Shannon uses Pi as part of its agent framework.
|
||||
|
||||
Project: https://github.com/earendil-works/pi
|
||||
License: MIT
|
||||
|
||||
Copyright (c) 2025 Mario Zechner
|
||||
|
||||
The applicable license is reproduced at `LICENSES/MIT-Pi.txt`.
|
||||
|
||||
## Mantis
|
||||
|
||||
Portions of Shannon's Capella agentic SAST implementation, specifically the
|
||||
agent prompts, are derived from the Mantis project.
|
||||
|
||||
- Project: https://github.com/google/mantis
|
||||
- Upstream commit: 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52
|
||||
- Retrieved: 2026-08-25
|
||||
- License: Apache License, Version 2.0
|
||||
|
||||
The Apache License, Version 2.0 is reproduced at LICENSES/Apache-2.0.txt.
|
||||
|
||||
The Mantis-derived files are individually marked with a provenance header and
|
||||
reside under:
|
||||
|
||||
- apps/worker/prompts/partials/ (capella-*.hbs prompt partials)
|
||||
- apps/worker/prompts/sast/capella/ (prompt templates)
|
||||
|
||||
The Mantis-derived material has been substantially modified by Keygraph
|
||||
for use within Shannon, including adaptation to Shannon's agent
|
||||
architecture and the Pi agent framework.
|
||||
|
||||
Copyright and attribution notices from the original Mantis material
|
||||
remain the property of their respective copyright holders.
|
||||
|
||||
Modifications:
|
||||
Copyright © 2026 Keygraph, Inc.
|
||||
@@ -18,7 +18,7 @@
|
||||
},
|
||||
"dependencies": {
|
||||
"@clack/prompts": "^1.1.0",
|
||||
"@temporalio/client": "^1.11.0",
|
||||
"@temporalio/client": "1.15.0",
|
||||
"chokidar": "^5.0.0",
|
||||
"dotenv": "^17.3.1",
|
||||
"smol-toml": "^1.6.1"
|
||||
|
||||
+199
-38
@@ -1,14 +1,17 @@
|
||||
/**
|
||||
* `shannon logs` command — tail a scan's live log.
|
||||
*
|
||||
* The log file is streamed for its content; completion is decided by Temporal (the
|
||||
* workflow's status), so a worker that dies mid-run can't leave the tail hanging. Uses
|
||||
* chokidar for reliable cross-platform file watching and bounded synchronous reads to
|
||||
* prevent duplicate output.
|
||||
* The log file is streamed for its content and ends the tail on its own terminal marker
|
||||
* (`Scan COMPLETED/PARTIAL/FAILED/CANCELLED`) or Ctrl-C. Temporal's workflow status is a backstop
|
||||
* that also closes the tail when a worker dies without writing a marker — for interactive `logs` a
|
||||
* Temporal outage is never fatal (it keeps tailing); only `start --follow` (CI) treats a sustained
|
||||
* outage as a failure. Uses chokidar for reliable cross-platform file watching and bounded
|
||||
* synchronous reads to prevent duplicate output.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { StringDecoder } from 'node:string_decoder';
|
||||
import { setTimeout as sleep } from 'node:timers/promises';
|
||||
import { watch } from 'chokidar';
|
||||
import { fail } from '../errors.js';
|
||||
@@ -18,8 +21,69 @@ import { resolveWorkflowId } from '../session.js';
|
||||
import { waitForWorkflowClose } from '../temporal-client.js';
|
||||
import { stdoutIsTerminal } from '../tty.js';
|
||||
|
||||
/** Read a byte range from a file and return it as a UTF-8 string. */
|
||||
function readRange(filePath: string, start: number, end: number): string {
|
||||
const TERMINAL_HEADINGS = new Set(['Scan COMPLETED', 'Scan PARTIAL', 'Scan FAILED', 'Scan CANCELLED']);
|
||||
|
||||
// The combined log resets completion on the bare `RESUMED` heading; a per-agent file carries the
|
||||
// distinct `--- RESUMED (<workflow id>) ---` boundary that WorkflowLogger.logResumeBoundary writes
|
||||
// (kept distinct per resume so it stays idempotent per file). Both mean a new execution began, so a
|
||||
// `--agent` tail must clear a stale terminal marker on either, matching the combined tail.
|
||||
const AGENT_RESUME_BOUNDARY = /^--- RESUMED \(.+\) ---$/u;
|
||||
|
||||
function isResumeBoundary(line: string): boolean {
|
||||
return line === 'RESUMED' || AGENT_RESUME_BOUNDARY.test(line);
|
||||
}
|
||||
|
||||
/** Tracks only complete structural lines while output remains byte-for-byte unchanged. */
|
||||
export class LogCompletionState {
|
||||
private pendingLine = '';
|
||||
private terminalIsLastMarker = false;
|
||||
private failureIsLastMarker = false;
|
||||
|
||||
ingest(chunk: string): void {
|
||||
const lines = `${this.pendingLine}${chunk}`.split('\n');
|
||||
this.pendingLine = lines.pop() ?? '';
|
||||
for (const line of lines) {
|
||||
if (isResumeBoundary(line)) {
|
||||
this.terminalIsLastMarker = false;
|
||||
this.failureIsLastMarker = false;
|
||||
} else if (TERMINAL_HEADINGS.has(line)) {
|
||||
this.terminalIsLastMarker = true;
|
||||
this.failureIsLastMarker = line === 'Scan FAILED';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
isComplete(): boolean {
|
||||
return this.terminalIsLastMarker;
|
||||
}
|
||||
|
||||
hasFailureMarker(): boolean {
|
||||
return this.failureIsLastMarker;
|
||||
}
|
||||
}
|
||||
|
||||
/** Append the forced-stop marker after the worker has exited, unless this execution already ended. */
|
||||
export function appendCancellationFallback(logFile: string): void {
|
||||
fs.mkdirSync(path.dirname(logFile), { recursive: true });
|
||||
const state = new LogCompletionState();
|
||||
try {
|
||||
state.ingest(fs.readFileSync(logFile, 'utf8'));
|
||||
} catch {
|
||||
// A pre-registration stop may not have created the file yet.
|
||||
}
|
||||
if (state.isComplete()) return;
|
||||
|
||||
const descriptor = fs.openSync(logFile, 'a', 0o600);
|
||||
try {
|
||||
fs.writeSync(descriptor, '\nScan CANCELLED\n');
|
||||
fs.fsyncSync(descriptor);
|
||||
} finally {
|
||||
fs.closeSync(descriptor);
|
||||
}
|
||||
}
|
||||
|
||||
/** Read a byte range without decoding across an arbitrary live-write boundary. */
|
||||
function readRange(filePath: string, start: number, end: number): Buffer {
|
||||
const length = end - start;
|
||||
const buffer = Buffer.alloc(length);
|
||||
const fd = fs.openSync(filePath, 'r');
|
||||
@@ -28,7 +92,7 @@ function readRange(filePath: string, start: number, end: number): string {
|
||||
} finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
return buffer.toString('utf-8');
|
||||
return buffer;
|
||||
}
|
||||
|
||||
/** Resolve a workspace ID to its workflow.log path, or exit with an error. */
|
||||
@@ -65,10 +129,16 @@ export function resolveLogFile(workspaceId: string): string {
|
||||
}
|
||||
|
||||
export interface TailOptions {
|
||||
/** Workflow whose Temporal status decides when the tail stops. Without it, only Ctrl-C ends the tail. */
|
||||
/** Workflow whose Temporal status can end the tail (alongside the file's own terminal marker). */
|
||||
readonly workflowId?: string;
|
||||
/** Called if the tail ends because Temporal became unreachable, with the captured error. */
|
||||
readonly onUnreachable?: (lastError: string) => void;
|
||||
/**
|
||||
* Consecutive Temporal-outage polls before the watch gives up. Interactive `logs` passes
|
||||
* Infinity so a blip never ends the tail (the file marker or Ctrl-C do); `start --follow` (CI)
|
||||
* leaves it bounded so a genuinely dead Temporal fails the run instead of hanging.
|
||||
*/
|
||||
readonly maxConnectFailures?: number;
|
||||
}
|
||||
|
||||
/** Outcome of a tail: whether the streamed log already contained the worker's `Scan FAILED` block. */
|
||||
@@ -76,36 +146,35 @@ export interface TailResult {
|
||||
readonly sawFailure: boolean;
|
||||
}
|
||||
|
||||
// The worker writes this exact line at the head of its terminal failure summary.
|
||||
const FAILURE_MARKER = /^Scan FAILED$/m;
|
||||
|
||||
/**
|
||||
* Stream a scan's log to the terminal until the workflow closes (completion comes from Temporal,
|
||||
* or Ctrl-C). A Temporal outage is warned about and, if sustained, ends the tail with a diagnostic.
|
||||
* Never exits the process: plain `logs` exits; `start --follow` reads the workflow outcome first.
|
||||
* Reports whether the log already showed the failure, so a caller need not print it a second time.
|
||||
* Stream a scan's log to the terminal until the file shows a terminal marker, the workflow closes,
|
||||
* or Ctrl-C. A Temporal outage is warned about; if it reaches `maxConnectFailures` the tail ends
|
||||
* with a diagnostic (bounded for `start --follow`), but interactive `logs` sets that to Infinity so
|
||||
* an outage keeps tailing. Never exits the process itself. Reports whether the log already showed
|
||||
* the failure, so a caller need not print it a second time.
|
||||
*/
|
||||
export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Promise<TailResult> {
|
||||
return new Promise((resolve) => {
|
||||
let position = 0;
|
||||
const completion = new LogCompletionState();
|
||||
const completionDecoder = new StringDecoder('utf8');
|
||||
let done = false;
|
||||
let sawFailure = false;
|
||||
const controller = new AbortController();
|
||||
let watcher: ReturnType<typeof watch> | undefined;
|
||||
|
||||
/** Output any new content appended since the last read. */
|
||||
function flush(): void {
|
||||
function flush(): boolean {
|
||||
try {
|
||||
const { size } = fs.statSync(logFile);
|
||||
if (size <= position) return;
|
||||
if (size <= position) return completion.isComplete();
|
||||
const data = readRange(logFile, position, size);
|
||||
process.stdout.write(data);
|
||||
position = size;
|
||||
if (!sawFailure && FAILURE_MARKER.test(data)) {
|
||||
sawFailure = true;
|
||||
}
|
||||
completion.ingest(completionDecoder.write(data));
|
||||
return completion.isComplete();
|
||||
} catch {
|
||||
// File not present yet or transiently unreadable — nothing to flush this round.
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -113,27 +182,42 @@ export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Prom
|
||||
if (done) return;
|
||||
done = true;
|
||||
controller.abort();
|
||||
process.off('SIGINT', finish);
|
||||
const result = { sawFailure: completion.hasFailureMarker() };
|
||||
if (watcher) {
|
||||
watcher.close().finally(() => resolve({ sawFailure }));
|
||||
watcher.close().finally(() => resolve(result));
|
||||
// Safety net — resolve anyway if watcher.close() stalls.
|
||||
setTimeout(() => resolve({ sawFailure }), 1000).unref();
|
||||
setTimeout(() => resolve(result), 1000).unref();
|
||||
} else {
|
||||
resolve({ sawFailure });
|
||||
resolve(result);
|
||||
}
|
||||
}
|
||||
|
||||
// 1. Output existing content, then stream anything appended.
|
||||
flush();
|
||||
// 1. Output existing content, then stream anything appended. A per-agent file can be created
|
||||
// after the watcher starts, so `add` is handled too and streams it from its first line.
|
||||
// The file's own `Scan COMPLETED/PARTIAL/FAILED/CANCELLED` marker ends the tail on its own —
|
||||
// a Temporal round-trip is a backstop for a worker that dies without writing one, not the
|
||||
// only way to stop.
|
||||
watcher = watch(logFile, { persistent: true });
|
||||
watcher.on('change', () => flush());
|
||||
const onFsEvent = (): void => {
|
||||
if (flush()) finish();
|
||||
};
|
||||
watcher.on('change', onFsEvent);
|
||||
watcher.on('add', onFsEvent);
|
||||
if (flush()) {
|
||||
finish();
|
||||
return;
|
||||
}
|
||||
|
||||
// 2. Ctrl-C stops watching.
|
||||
process.on('SIGINT', finish);
|
||||
process.once('SIGINT', finish);
|
||||
|
||||
// 3. Temporal decides completion. Without a workflow id, the tail relies on Ctrl-C alone.
|
||||
// 3. Temporal backstops completion for a worker that dies without a marker. Without a workflow
|
||||
// id, the tail relies on the file marker or Ctrl-C alone.
|
||||
if (opts.workflowId) {
|
||||
waitForWorkflowClose(opts.workflowId, {
|
||||
signal: controller.signal,
|
||||
...(opts.maxConnectFailures !== undefined ? { maxConnectFailures: opts.maxConnectFailures } : {}),
|
||||
onConnectionTrouble: (lastError) => {
|
||||
if (!done) console.error(`\n⚠ Lost contact with Temporal, retrying… (${lastError})`);
|
||||
},
|
||||
@@ -162,16 +246,93 @@ export function tailUntilComplete(logFile: string, opts: TailOptions = {}): Prom
|
||||
});
|
||||
}
|
||||
|
||||
export function logs(workspaceId: string): void {
|
||||
const logFile = resolveLogFile(workspaceId);
|
||||
const workflowId = resolveWorkflowId(workspaceId);
|
||||
console.error(stdoutIsTerminal() ? `Tailing scan log: ${logFile}` : 'Tailing scan log');
|
||||
/** The `.shannon/agents/` directory that sits beside a scan's combined workflow.log. */
|
||||
function agentsDirFor(logFile: string): string {
|
||||
return path.join(path.dirname(logFile), 'agents');
|
||||
}
|
||||
|
||||
let unreachable = false;
|
||||
/** List the per-agent log names available for a scan (filename stems, sorted), or an empty list. */
|
||||
export function listAgentLogNames(logFile: string): string[] {
|
||||
try {
|
||||
return fs
|
||||
.readdirSync(agentsDirFor(logFile))
|
||||
.filter((entry) => entry.endsWith('.log'))
|
||||
.map((entry) => entry.slice(0, -'.log'.length))
|
||||
.sort();
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve an agent name to its per-agent log path. The name must be a closed-charset basename, and
|
||||
* the resolved file must stay inside the agents directory: traversal and symlink escapes are
|
||||
* rejected. Returns undefined when the name is structurally invalid or escapes the directory.
|
||||
*/
|
||||
export function resolveAgentLogFile(logFile: string, agentName: string): string | undefined {
|
||||
if (!/^[a-z0-9][a-z0-9-]{0,63}$/u.test(agentName)) return undefined;
|
||||
const agentsDir = agentsDirFor(logFile);
|
||||
const target = path.join(agentsDir, `${agentName}.log`);
|
||||
try {
|
||||
const realDir = fs.realpathSync(agentsDir);
|
||||
const realTarget = fs.realpathSync(target);
|
||||
if (realTarget !== path.join(realDir, `${agentName}.log`)) return undefined;
|
||||
} catch {
|
||||
// The file does not exist yet (scan still starting); the closed-charset check already proved
|
||||
// the path cannot traverse out of the agents directory, so it is safe to watch for creation.
|
||||
}
|
||||
return target;
|
||||
}
|
||||
|
||||
export interface LogsOptions {
|
||||
readonly agent?: string;
|
||||
readonly listAgents?: boolean;
|
||||
}
|
||||
|
||||
function tailFileToExit(logFile: string, workflowId: string | undefined, label: string): void {
|
||||
console.error(stdoutIsTerminal() ? `${label}: ${logFile}` : label);
|
||||
tailUntilComplete(logFile, {
|
||||
...(workflowId ? { workflowId } : {}),
|
||||
onUnreachable: () => {
|
||||
unreachable = true;
|
||||
},
|
||||
}).finally(() => process.exit(unreachable ? 1 : 0));
|
||||
// Interactive tail: a Temporal outage must never end the session. The file's terminal marker or
|
||||
// Ctrl-C stop it; Temporal stays a soft backstop that reconnects and closes the tail on a
|
||||
// silent worker death, but its unreachability is never fatal here.
|
||||
maxConnectFailures: Number.POSITIVE_INFINITY,
|
||||
}).finally(() => process.exit(0));
|
||||
}
|
||||
|
||||
export function logs(workspaceId: string, options: LogsOptions = {}): void {
|
||||
const logFile = resolveLogFile(workspaceId);
|
||||
|
||||
if (options.listAgents) {
|
||||
const names = listAgentLogNames(logFile);
|
||||
if (names.length === 0) {
|
||||
console.error('No per-agent logs for this scan yet.');
|
||||
process.exit(0);
|
||||
}
|
||||
for (const name of names) console.log(name);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const workflowId = resolveWorkflowId(workspaceId);
|
||||
|
||||
if (options.agent !== undefined) {
|
||||
const agentFile = resolveAgentLogFile(logFile, options.agent);
|
||||
if (agentFile === undefined) {
|
||||
fail(`No agent log named: ${options.agent}`, '', 'Available agents:', ...withBullets(listAgentLogNames(logFile)));
|
||||
}
|
||||
const known = listAgentLogNames(logFile);
|
||||
// If the directory already lists agents, a name not among them is a typo, not a not-yet-created
|
||||
// file; fail loudly rather than tailing a path that will never appear.
|
||||
if (known.length > 0 && !known.includes(options.agent)) {
|
||||
fail(`No agent log named: ${options.agent}`, '', 'Available agents:', ...withBullets(known));
|
||||
}
|
||||
tailFileToExit(agentFile, workflowId, `Tailing ${options.agent} log`);
|
||||
return;
|
||||
}
|
||||
|
||||
tailFileToExit(logFile, workflowId, 'Tailing scan log');
|
||||
}
|
||||
|
||||
function withBullets(names: readonly string[]): string[] {
|
||||
return names.length === 0 ? [' (none yet)'] : names.map((name) => ` - ${name}`);
|
||||
}
|
||||
|
||||
@@ -1,23 +1,28 @@
|
||||
/**
|
||||
* `shannon scans` command — list completed scans and where each report lives.
|
||||
* `shannon scans` command — list scans, running and completed, and where each report lives.
|
||||
*
|
||||
* A scan counts as completed when it produced a report. The report can live in any of a
|
||||
* few locations depending on the version that ran it, so `findReport` probes them in order
|
||||
* and the first hit is both the completion signal and the link target behind the workspace
|
||||
* name. The date and wall-clock duration come from the run's session.json
|
||||
* (createdAt/completedAt), with the report file's mtime as the date fallback for
|
||||
* runs that lack a recorded time.
|
||||
* Running scans come from Docker: every worker container is stamped with the shannon.workspace
|
||||
* label, so `runningScanWorkspaces()` is the authoritative live-scan list (shared with `stop`).
|
||||
* A scan counts as completed once it has produced a report; the report can live in any of a few
|
||||
* locations depending on the version that ran it, so `findReport` probes them in order and the
|
||||
* first hit is both the completion signal and the link target behind the workspace name. Dates and
|
||||
* durations come from each run's session.json (createdAt/completedAt), with the report file's mtime
|
||||
* as the date fallback for runs that lack a recorded time; a running scan's duration is elapsed time
|
||||
* so far (now − createdAt).
|
||||
*
|
||||
* Human-readable by default; `--json` emits the same rows as raw machine values on stdout.
|
||||
* Running scans are listed first, then completed newest-first. Human-readable by default; `--json`
|
||||
* emits the same rows as raw machine values on stdout.
|
||||
*
|
||||
* Filesystem-only (local ./workspaces/ or npx ~/.shannon/workspaces/ via getWorkspacesDir);
|
||||
* no Temporal dependency.
|
||||
* The completed list is filesystem-only (local ./workspaces/ or npx ~/.shannon/workspaces/ via
|
||||
* getWorkspacesDir); the running list needs Docker but degrades to empty when the daemon is down,
|
||||
* which is the correct answer (no scan can be running then).
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
import { BOLD, GOLD, paint } from '../colors.js';
|
||||
import { BOLD, CYAN, GOLD, paint } from '../colors.js';
|
||||
import { runningScanWorkspaces } from '../docker.js';
|
||||
import { getWorkspacesDir } from '../home.js';
|
||||
import { commandPrefix } from '../mode.js';
|
||||
import { FINAL_REPORT_PDF_FILENAME, INTERNAL_DIR, resolveRunFile } from '../paths.js';
|
||||
@@ -31,23 +36,25 @@ const FINAL_REPORT_MD_FILENAME = 'Security-Assessment-Report.md';
|
||||
|
||||
const DELIVERABLES_SUBDIR = 'deliverables';
|
||||
|
||||
/** One completed scan; raw values so the table and --json render from one source. */
|
||||
/** One scan, running or completed; raw values so the table and --json render from one source. */
|
||||
interface ScanRow {
|
||||
readonly workspace: string;
|
||||
/** Completion time in ms — sort key and date source. */
|
||||
readonly finishedMs: number;
|
||||
/** Wall-clock duration (completedAt − createdAt) in ms, or null when unknown. */
|
||||
readonly state: 'running' | 'completed';
|
||||
/** Completion time in ms — sort key and date source. Null while a scan is still running. */
|
||||
readonly finishedMs: number | null;
|
||||
/** Wall-clock duration in ms: elapsed-so-far for running, total for completed. Null when unknown. */
|
||||
readonly durationMs: number | null;
|
||||
/** Absolute path to the report file — the link target behind the workspace name. */
|
||||
readonly report: string;
|
||||
/** Absolute path to the report file — the link target behind the workspace name. Null while running. */
|
||||
readonly report: string | null;
|
||||
}
|
||||
|
||||
/** The --json row shape: raw machine values, one per completed scan. */
|
||||
/** The --json row shape: raw machine values, one per scan. */
|
||||
interface JsonRow {
|
||||
readonly workspace: string;
|
||||
readonly finishedAt: string;
|
||||
readonly state: 'running' | 'completed';
|
||||
readonly finishedAt: string | null;
|
||||
readonly durationMs: number | null;
|
||||
readonly reportPath: string;
|
||||
readonly reportPath: string | null;
|
||||
}
|
||||
|
||||
/** Compact wall-clock duration from milliseconds: "47s", "1m 32s", "1h 47m". */
|
||||
@@ -130,7 +137,19 @@ function collectCompletedScans(workspacesDir: string): ScanRow[] {
|
||||
const finishedMs = Number.isNaN(completedMs) ? fs.statSync(reportPath).mtimeMs : completedMs;
|
||||
const durationMs = Number.isNaN(completedMs) || Number.isNaN(createdMs) ? null : completedMs - createdMs;
|
||||
|
||||
rows.push({ workspace: entry.name, finishedMs, durationMs, report: reportPath });
|
||||
rows.push({ workspace: entry.name, state: 'completed', finishedMs, durationMs, report: reportPath });
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
/** Gather every currently-running scan, one row each. Elapsed time is now − createdAt. */
|
||||
function collectRunningScans(workspacesDir: string, nowMs: number): ScanRow[] {
|
||||
const rows: ScanRow[] = [];
|
||||
for (const workspace of runningScanWorkspaces()) {
|
||||
const { session } = readSession(path.join(workspacesDir, workspace));
|
||||
const createdMs = Date.parse(session.createdAt ?? '');
|
||||
const durationMs = Number.isNaN(createdMs) ? null : nowMs - createdMs;
|
||||
rows.push({ workspace, state: 'running', finishedMs: null, durationMs, report: null });
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
@@ -138,55 +157,68 @@ function collectCompletedScans(workspacesDir: string): ScanRow[] {
|
||||
function toJsonRow(row: ScanRow): JsonRow {
|
||||
return {
|
||||
workspace: row.workspace,
|
||||
finishedAt: new Date(row.finishedMs).toISOString(),
|
||||
state: row.state,
|
||||
finishedAt: row.finishedMs === null ? null : new Date(row.finishedMs).toISOString(),
|
||||
durationMs: row.durationMs,
|
||||
reportPath: row.report,
|
||||
};
|
||||
}
|
||||
|
||||
/** Print the completed scans as an aligned table with the workspace name linked to its report. */
|
||||
/** Print the scans as an aligned table with each completed workspace name linked to its report. */
|
||||
function printTable(workspacesDir: string, rows: readonly ScanRow[]): void {
|
||||
if (rows.length === 0) {
|
||||
const prefix = commandPrefix();
|
||||
console.log(`No completed scans yet. Run '${prefix} start -u <url> -r <path>' to begin.`);
|
||||
console.log(`No scans yet. Run '${prefix} start -u <url> -r <path>' to begin.`);
|
||||
return;
|
||||
}
|
||||
|
||||
const color = supportsColor();
|
||||
// On a terminal the workspace name is an OSC 8 hyperlink that opens its report; when
|
||||
// piped there is nothing to click, so it prints as plain text.
|
||||
// On a terminal a completed workspace name is an OSC 8 hyperlink that opens its report; when
|
||||
// piped, or for a running scan that has no report yet, it prints as plain text.
|
||||
const linkable = stdoutIsTerminal();
|
||||
|
||||
const table = rows.map((row) => ({
|
||||
finished: new Date(row.finishedMs).toISOString().slice(0, 10),
|
||||
state: row.state === 'running' ? 'RUNNING' : 'COMPLETED',
|
||||
finished: row.finishedMs === null ? '—' : new Date(row.finishedMs).toISOString().slice(0, 10),
|
||||
duration: row.durationMs === null ? '—' : formatDuration(row.durationMs),
|
||||
workspace: row.workspace,
|
||||
report: row.report,
|
||||
}));
|
||||
|
||||
const stateWidth = Math.max('STATE'.length, ...table.map((row) => row.state.length));
|
||||
const dateWidth = Math.max('FINISHED'.length, 'YYYY-MM-DD'.length);
|
||||
const durationWidth = Math.max('DURATION'.length, ...table.map((row) => row.duration.length));
|
||||
|
||||
console.log(`\nCompleted scans in ${workspacesDir}:\n`);
|
||||
const header = `${'FINISHED'.padEnd(dateWidth)} ${'DURATION'.padEnd(durationWidth)} WORKSPACE`;
|
||||
console.log(`\nScans in ${workspacesDir}:\n`);
|
||||
const header = `${'STATE'.padEnd(stateWidth)} ${'FINISHED'.padEnd(dateWidth)} ${'DURATION'.padEnd(durationWidth)} WORKSPACE`;
|
||||
console.log(paint(header, BOLD, color));
|
||||
|
||||
for (const row of table) {
|
||||
const stateText = row.state.padEnd(stateWidth);
|
||||
const state = row.state === 'RUNNING' ? paint(stateText, CYAN, color) : stateText;
|
||||
const finished = row.finished.padEnd(dateWidth);
|
||||
const duration = row.duration.padEnd(durationWidth);
|
||||
const name = paint(row.workspace, GOLD, color);
|
||||
const workspace = linkable ? hyperlink(name, pathToFileURL(row.report).href) : name;
|
||||
console.log(`${finished} ${duration} ${workspace}`);
|
||||
// A running scan has no report to open, so its name stays plain; completed names are linked.
|
||||
const name = row.report ? paint(row.workspace, GOLD, color) : row.workspace;
|
||||
const workspace = row.report && linkable ? hyperlink(name, pathToFileURL(row.report).href) : name;
|
||||
console.log(`${state} ${finished} ${duration} ${workspace}`);
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
|
||||
export function scans(opts: { readonly json: boolean }): void {
|
||||
const workspacesDir = getWorkspacesDir();
|
||||
const rows = collectCompletedScans(workspacesDir);
|
||||
const nowMs = Date.now();
|
||||
|
||||
// Latest on top.
|
||||
rows.sort((a, b) => b.finishedMs - a.finishedMs);
|
||||
const running = collectRunningScans(workspacesDir, nowMs);
|
||||
const runningNames = new Set(running.map((row) => row.workspace));
|
||||
// A running scan has no final report, so it can't also be completed; guard anyway.
|
||||
const completed = collectCompletedScans(workspacesDir).filter((row) => !runningNames.has(row.workspace));
|
||||
|
||||
// Running scans on top (most recently started first), then completed newest-first.
|
||||
running.sort((a, b) => (a.durationMs ?? 0) - (b.durationMs ?? 0));
|
||||
completed.sort((a, b) => (b.finishedMs ?? 0) - (a.finishedMs ?? 0));
|
||||
const rows = [...running, ...completed];
|
||||
|
||||
if (opts.json) {
|
||||
console.log(JSON.stringify(rows.map(toJsonRow), null, 2));
|
||||
|
||||
+277
-63
@@ -12,18 +12,20 @@ import { setTimeout as sleep } from 'node:timers/promises';
|
||||
import * as p from '@clack/prompts';
|
||||
import { ensureDocker, ensureImage, ensureInfra, randomSuffix, spawnWorker } from '../docker.js';
|
||||
import { buildEnvFlags, loadEnv, resolveHostPiAuthPath, shouldUsePiAuth, validateCredentials } from '../env.js';
|
||||
import { fail } from '../errors.js';
|
||||
import { fail, warn } from '../errors.js';
|
||||
import { getWorkspacesDir, initHome } from '../home.js';
|
||||
import { commandPrefix, isLocal } from '../mode.js';
|
||||
import { resolveModelSpec } from '../model-spec.js';
|
||||
import {
|
||||
expandHome,
|
||||
FINAL_REPORT_MD_FILENAME,
|
||||
FINAL_REPORT_PDF_FILENAME,
|
||||
INTERNAL_DIR,
|
||||
resolveConfig,
|
||||
resolveRepo,
|
||||
resolveRunFile,
|
||||
} from '../paths.js';
|
||||
import { clearPendingWorkflowIdentity, writePendingWorkflowIdentity } from '../pending-workflow.js';
|
||||
import { indentFailureSegments } from '../scan/failure.js';
|
||||
import { resolveWorkflowId } from '../session.js';
|
||||
import { displayPlainBanner, displaySplash } from '../splash.js';
|
||||
@@ -43,83 +45,226 @@ export interface StartArgs {
|
||||
version: string;
|
||||
}
|
||||
|
||||
const LAUNCH_STATE_SCHEMA_VERSION = 1 as const;
|
||||
const LAUNCH_STATE_FILENAME = 'launch.json';
|
||||
const FIXED_CLASSES = ['injection', 'xss', 'auth', 'authz', 'ssrf'] as const;
|
||||
|
||||
/**
|
||||
* Upgrade a pre-restructure workspace (flat layout, no INTERNAL_DIR) before it is mounted,
|
||||
* so resume finds the old deliverables and their git checkpoints instead of re-running every
|
||||
* agent. For a legacy run every top-level entry is internal, so move them all into INTERNAL_DIR
|
||||
* (a same-filesystem rename carries the deliverables .git along).
|
||||
* CLI-owned launch record at INTERNAL_DIR/launch.json, written once when a workspace is
|
||||
* created and never rewritten. It pins the customer output destination so a resume with a
|
||||
* different -o cannot silently redirect the final report. The worker does not read it.
|
||||
*/
|
||||
function migrateLegacyWorkspaceLayout(workspacePath: string): void {
|
||||
const legacySessionJson = path.join(workspacePath, 'session.json');
|
||||
const internalPath = path.join(workspacePath, INTERNAL_DIR);
|
||||
if (!fs.existsSync(legacySessionJson) || fs.existsSync(internalPath)) {
|
||||
return;
|
||||
interface LaunchState {
|
||||
readonly schema_version: typeof LAUNCH_STATE_SCHEMA_VERSION;
|
||||
readonly customer_output_path?: string;
|
||||
}
|
||||
|
||||
export interface WorkspaceLaunchDecision {
|
||||
readonly isResume: boolean;
|
||||
readonly outputDir?: string;
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return value !== null && typeof value === 'object' && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function arraysEqual(left: readonly unknown[], right: readonly unknown[]): boolean {
|
||||
return left.length === right.length && left.every((value, index) => value === right[index]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Hand-rolled twin of the worker's durable-state validator in
|
||||
* apps/worker/src/types/run-state.ts, which owns the session.json.durableScanState shape.
|
||||
* Each array check accepts two variants because the worker appends 'miscellaneous' and
|
||||
* 'miscellaneous-exploit' only after the miscellaneous pipeline admits findings. If the worker's shape
|
||||
* changes and this twin lags, resume fails fast as incompatible instead of launching a
|
||||
* worker against state it would misread.
|
||||
*/
|
||||
function isCurrentDurableState(value: unknown): boolean {
|
||||
if (!isRecord(value) || value.schema_version !== 1 || typeof value.exploit !== 'boolean') return false;
|
||||
if (!Array.isArray(value.participating_classes) || !Array.isArray(value.expected_agents)) return false;
|
||||
|
||||
const participating = value.participating_classes;
|
||||
const validParticipation =
|
||||
arraysEqual(participating, FIXED_CLASSES) || arraysEqual(participating, [...FIXED_CLASSES, 'miscellaneous']);
|
||||
if (!validParticipation) return false;
|
||||
|
||||
const baselineAgents = ['pre-recon', 'recon', ...FIXED_CLASSES.map((name) => `${name}-vuln`)];
|
||||
if (value.exploit) baselineAgents.push(...FIXED_CLASSES.map((name) => `${name}-exploit`));
|
||||
baselineAgents.push('report');
|
||||
const expected = value.expected_agents;
|
||||
return arraysEqual(expected, baselineAgents) || arraysEqual(expected, [...baselineAgents, 'miscellaneous-exploit']);
|
||||
}
|
||||
|
||||
/** One refusal for damaged CLI-owned or worker-owned workspace records, whichever reads first. */
|
||||
const DAMAGED_RECORDS_MESSAGE =
|
||||
"This workspace's internal records are damaged and it cannot be resumed. Its report files are untouched. Start a new scan with a different -w name.";
|
||||
|
||||
const NEWER_RELEASE_MESSAGE =
|
||||
'This workspace was created by a newer version of Shannon. Upgrade Shannon, or start a new scan with a different -w name.';
|
||||
|
||||
function readJsonFile(filePath: string): unknown {
|
||||
try {
|
||||
return JSON.parse(fs.readFileSync(filePath, 'utf8'));
|
||||
} catch {
|
||||
fail(DAMAGED_RECORDS_MESSAGE);
|
||||
}
|
||||
}
|
||||
|
||||
function readLaunchState(filePath: string): LaunchState {
|
||||
if (!fs.existsSync(filePath)) {
|
||||
fail(
|
||||
'This workspace was created by an earlier version of Shannon and cannot be resumed. Its files and report are untouched. Start a new scan with a different -w name.',
|
||||
);
|
||||
}
|
||||
const value = readJsonFile(filePath);
|
||||
if (!isRecord(value)) fail(NEWER_RELEASE_MESSAGE);
|
||||
// Unknown keys mean a newer release wrote this workspace; refuse rather than half-read it.
|
||||
const keys = Object.keys(value).sort();
|
||||
const keysAreValid = keys.every((key) => key === 'customer_output_path' || key === 'schema_version');
|
||||
const customerPath = value.customer_output_path;
|
||||
const pathIsValid =
|
||||
customerPath === undefined ||
|
||||
(typeof customerPath === 'string' && path.isAbsolute(customerPath) && path.resolve(customerPath) === customerPath);
|
||||
if (value.schema_version !== LAUNCH_STATE_SCHEMA_VERSION || !keysAreValid || !pathIsValid) {
|
||||
fail(NEWER_RELEASE_MESSAGE);
|
||||
}
|
||||
return {
|
||||
schema_version: LAUNCH_STATE_SCHEMA_VERSION,
|
||||
...(typeof customerPath === 'string' && { customer_output_path: customerPath }),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide fresh-versus-resume from on-disk state alone, before start() mutates anything.
|
||||
* A fresh launch requires the workspace directory to be absent or empty; a resume requires
|
||||
* current-release session state, a matching target URL, and a customer output path that
|
||||
* agrees with the recorded one. Every other combination fails the launch, so a typo in
|
||||
* -w or -o stops here instead of spawning a worker into the wrong workspace.
|
||||
*/
|
||||
export function classifyWorkspaceLaunch(
|
||||
workspacePath: string,
|
||||
expectedUrl: string,
|
||||
requestedOutputDir: string | undefined,
|
||||
): WorkspaceLaunchDecision {
|
||||
const sessionPath = resolveRunFile(workspacePath, 'session.json');
|
||||
const sessionExists = fs.existsSync(sessionPath);
|
||||
if (!sessionExists) {
|
||||
if (fs.existsSync(workspacePath) && fs.readdirSync(workspacePath).length > 0) {
|
||||
fail(
|
||||
'This directory is not a Shannon workspace, or its scan state is missing. Start a new scan with a different -w name.',
|
||||
);
|
||||
}
|
||||
return { isResume: false, ...(requestedOutputDir !== undefined && { outputDir: requestedOutputDir }) };
|
||||
}
|
||||
|
||||
fs.mkdirSync(internalPath, { recursive: true });
|
||||
for (const entry of fs.readdirSync(workspacePath)) {
|
||||
if (entry === INTERNAL_DIR) {
|
||||
continue;
|
||||
}
|
||||
fs.renameSync(path.join(workspacePath, entry), path.join(internalPath, entry));
|
||||
const launchPath = path.join(workspacePath, INTERNAL_DIR, LAUNCH_STATE_FILENAME);
|
||||
const launch = readLaunchState(launchPath);
|
||||
const session = readJsonFile(sessionPath);
|
||||
if (!isRecord(session) || !isRecord(session.session) || session.session.webUrl !== expectedUrl) {
|
||||
fail(
|
||||
'This workspace was created for a different target URL, so it cannot be resumed against this one. Check -u, or start a new scan with a different -w name.',
|
||||
);
|
||||
}
|
||||
console.log(`Migrated workspace to ${INTERNAL_DIR}/ layout: ${workspacePath}`);
|
||||
if (!isCurrentDurableState(session.durableScanState)) {
|
||||
fail(
|
||||
"This workspace's scan state cannot be read by this version. Its files are untouched. Start a new scan with a different -w name.",
|
||||
);
|
||||
}
|
||||
|
||||
const storedOutputDir = launch.customer_output_path;
|
||||
if (requestedOutputDir !== undefined && requestedOutputDir !== storedOutputDir) {
|
||||
fail(
|
||||
'This workspace already copies its report to a different location than the -o path you passed. Re-run without -o to keep the original location, or start a new scan with a different -w name.',
|
||||
);
|
||||
}
|
||||
return { isResume: true, ...(storedOutputDir !== undefined && { outputDir: storedOutputDir }) };
|
||||
}
|
||||
|
||||
/**
|
||||
* Crash-safe single write: exclusive temp file (pid plus random suffix keeps concurrent
|
||||
* starts apart), fsync, rename into place, then directory fsync so the entry survives a
|
||||
* host crash. Callers invoke this only for a fresh workspace; an existing launch.json is
|
||||
* the resume contract and must never be replaced.
|
||||
*/
|
||||
export function writeLaunchStateAtomically(internalPath: string, outputDir: string | undefined): void {
|
||||
const finalPath = path.join(internalPath, LAUNCH_STATE_FILENAME);
|
||||
const temporaryPath = path.join(internalPath, `${LAUNCH_STATE_FILENAME}.tmp-${process.pid}-${randomSuffix()}`);
|
||||
const launchState: LaunchState = {
|
||||
schema_version: LAUNCH_STATE_SCHEMA_VERSION,
|
||||
...(outputDir !== undefined && { customer_output_path: outputDir }),
|
||||
};
|
||||
const descriptor = fs.openSync(temporaryPath, 'wx', 0o600);
|
||||
try {
|
||||
fs.writeFileSync(descriptor, `${JSON.stringify(launchState, null, 2)}\n`, 'utf8');
|
||||
fs.fsyncSync(descriptor);
|
||||
} finally {
|
||||
fs.closeSync(descriptor);
|
||||
}
|
||||
try {
|
||||
fs.renameSync(temporaryPath, finalPath);
|
||||
const directory = fs.openSync(internalPath, 'r');
|
||||
try {
|
||||
fs.fsyncSync(directory);
|
||||
} finally {
|
||||
fs.closeSync(directory);
|
||||
}
|
||||
} catch (error) {
|
||||
fs.rmSync(temporaryPath, { force: true });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/** Select the workflow ID before Docker starts so the container can carry it as immutable identity. */
|
||||
export function createWorkflowId(workspace: string, isResume: boolean, timestamp: number = Date.now()): string {
|
||||
if (isResume) return `${workspace}_resume_${timestamp}`;
|
||||
return /_shannon-\d+$/.test(workspace) ? workspace : `${workspace}_shannon-${timestamp}`;
|
||||
}
|
||||
|
||||
export async function start(args: StartArgs): Promise<void> {
|
||||
// 1. Initialize state directories and load env
|
||||
// 1. Resolve non-mutating inputs and classify the workspace before changing it.
|
||||
initHome();
|
||||
loadEnv();
|
||||
|
||||
// 2. Validate credentials
|
||||
const creds = validateCredentials();
|
||||
if (!creds.valid) {
|
||||
fail(creds.error ?? 'Invalid credentials');
|
||||
}
|
||||
|
||||
// 3. Resolve paths
|
||||
const repo = resolveRepo(args.repo);
|
||||
const config = args.config ? resolveConfig(args.config) : undefined;
|
||||
const workspacesDir = getWorkspacesDir();
|
||||
const workspace =
|
||||
args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`;
|
||||
const workspacePath = path.join(workspacesDir, workspace);
|
||||
const requestedOutputDir = args.output ? path.resolve(expandHome(args.output)) : undefined;
|
||||
const launchDecision = classifyWorkspaceLaunch(workspacePath, args.url, requestedOutputDir);
|
||||
|
||||
// Inputs are valid — identify the run before the Docker/Temporal setup work.
|
||||
// 2. Inputs are valid; identify the run before initializing shared infrastructure.
|
||||
const bannerVersion = isLocal() ? undefined : args.version;
|
||||
if (stdoutIsTerminal()) {
|
||||
displaySplash(bannerVersion);
|
||||
} else {
|
||||
displayPlainBanner(bannerVersion);
|
||||
}
|
||||
|
||||
// 4. Ensure workspaces dir is writable by container user (UID 1001)
|
||||
const workspacesDir = getWorkspacesDir();
|
||||
fs.mkdirSync(workspacesDir, { recursive: true });
|
||||
fs.chmodSync(workspacesDir, 0o777);
|
||||
|
||||
// 5. Ensure Docker and the worker image are available (pull/build prints its own progress).
|
||||
ensureDocker();
|
||||
ensureImage(args.version);
|
||||
|
||||
// One spinner spans the whole launch: bringing up Temporal and registering the worker.
|
||||
const spinner = p.spinner();
|
||||
spinner.start('Starting scan');
|
||||
await ensureInfra(spinner);
|
||||
|
||||
// 6. Generate unique task queue and container name
|
||||
// 3. Generate the invocation identity.
|
||||
const suffix = randomSuffix();
|
||||
const taskQueue = `shannon-${suffix}`;
|
||||
const containerName = `shannon-worker-${suffix}`;
|
||||
const workflowId = createWorkflowId(workspace, launchDecision.isResume);
|
||||
|
||||
// 7. Generate workspace name if not provided
|
||||
const workspace =
|
||||
args.workspace ?? `${new URL(args.url).hostname.replace(/[^a-zA-Z0-9-]/g, '-')}_shannon-${Date.now()}`;
|
||||
|
||||
// 8. Create writable overlay directories (mounted over :ro repo paths inside container)
|
||||
// 4. Create writable overlay directories after resume validation has succeeded.
|
||||
// The run dir and its INTERNAL_DIR must be 0o777 so the container user can create audit
|
||||
// subdirs and the overlay backing dirs.
|
||||
const workspacePath = path.join(workspacesDir, workspace);
|
||||
const internalPath = path.join(workspacePath, INTERNAL_DIR);
|
||||
fs.mkdirSync(workspacePath, { recursive: true });
|
||||
fs.chmodSync(workspacePath, 0o777);
|
||||
migrateLegacyWorkspaceLayout(workspacePath);
|
||||
fs.mkdirSync(internalPath, { recursive: true });
|
||||
fs.chmodSync(internalPath, 0o777);
|
||||
for (const dir of ['deliverables', 'scratchpad', '.playwright-cli', '.playwright']) {
|
||||
@@ -127,30 +272,53 @@ export async function start(args: StartArgs): Promise<void> {
|
||||
fs.mkdirSync(dirPath, { recursive: true });
|
||||
fs.chmodSync(dirPath, 0o777);
|
||||
}
|
||||
if (!launchDecision.isResume) {
|
||||
writeLaunchStateAtomically(internalPath, launchDecision.outputDir);
|
||||
}
|
||||
|
||||
// 9. Pre-create overlay mount points (:ro mounts can't auto-create them)
|
||||
// 5. Pre-create overlay mount points (:ro mounts cannot create them).
|
||||
const shannonDir = path.join(repo.hostPath, '.shannon');
|
||||
for (const dir of ['deliverables', 'scratchpad', '.playwright-cli']) {
|
||||
fs.mkdirSync(path.join(shannonDir, dir), { recursive: true });
|
||||
}
|
||||
fs.mkdirSync(path.join(repo.hostPath, '.playwright'), { recursive: true });
|
||||
|
||||
// 10. Resolve output directory
|
||||
const outputDir = args.output ? path.resolve(expandHome(args.output)) : undefined;
|
||||
// 6. Create the validated customer-copy destination, if configured.
|
||||
const outputDir = launchDecision.outputDir;
|
||||
if (outputDir) {
|
||||
fs.mkdirSync(outputDir, { recursive: true });
|
||||
}
|
||||
|
||||
// 11. Resolve prompts directory (local mode only)
|
||||
// 7. Resolve prompts and capture the pre-launch resume counter.
|
||||
const promptsDir = isLocal() ? path.resolve('apps/worker/prompts') : undefined;
|
||||
const sessionJson = resolveRunFile(workspacePath, 'session.json');
|
||||
const isResume = launchDecision.isResume;
|
||||
let initialResumeCount = 0;
|
||||
if (isResume) {
|
||||
// Docker and Temporal startup sit between this read and the classification that validated the
|
||||
// same file, so a file that changed in between is a workspace-state failure, not a CLI bug.
|
||||
const session = readJsonFile(sessionJson);
|
||||
const attempts = isRecord(session) && isRecord(session.session) ? session.session.resumeAttempts : undefined;
|
||||
initialResumeCount = Array.isArray(attempts) ? attempts.length : 0;
|
||||
}
|
||||
|
||||
// 12. Spawn worker container
|
||||
// 8. Persist the exact launch candidate before Docker can start the worker. Session
|
||||
// registration later replaces this bridge as the durable workflow identity.
|
||||
try {
|
||||
writePendingWorkflowIdentity(workspacePath, workflowId, taskQueue);
|
||||
} catch {
|
||||
spinner.error('Could not record the scan workflow identity');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// 9. Spawn the worker container.
|
||||
const proc = spawnWorker({
|
||||
version: args.version,
|
||||
url: args.url,
|
||||
repo,
|
||||
workspacesDir,
|
||||
taskQueue,
|
||||
workflowId,
|
||||
containerName,
|
||||
envFlags: buildEnvFlags(),
|
||||
...(config && { config }),
|
||||
@@ -173,24 +341,16 @@ export async function start(args: StartArgs): Promise<void> {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Detect whether this is a fresh workspace or a resume by checking session.json existence
|
||||
const sessionJson = resolveRunFile(path.join(workspacesDir, workspace), 'session.json');
|
||||
const isResume = fs.existsSync(sessionJson);
|
||||
let initialResumeCount = 0;
|
||||
if (isResume) {
|
||||
try {
|
||||
const session = JSON.parse(fs.readFileSync(sessionJson, 'utf-8'));
|
||||
initialResumeCount = session.session?.resumeAttempts?.length ?? 0;
|
||||
} catch {
|
||||
// Corrupted file — worker will handle validation
|
||||
}
|
||||
}
|
||||
|
||||
let started = false;
|
||||
|
||||
// Set when the startup poll times out but session.json already holds durable state this
|
||||
// release understands: the workflow is executing, so the exit handler must not stop its
|
||||
// worker. An operator abort is a different intent and still stops it.
|
||||
let scanRunningUnconfirmed = false;
|
||||
|
||||
// Stop the worker only if the scan hasn't registered yet (e.g. Ctrl-C mid-startup).
|
||||
let cleaned = false;
|
||||
const cleanup = (): void => {
|
||||
const stopWorker = (): void => {
|
||||
if (cleaned || started) return;
|
||||
cleaned = true;
|
||||
spinner.stop('Stopping scan');
|
||||
@@ -204,14 +364,17 @@ export async function start(args: StartArgs): Promise<void> {
|
||||
}
|
||||
};
|
||||
process.on('SIGINT', () => {
|
||||
cleanup();
|
||||
stopWorker();
|
||||
process.exit(0);
|
||||
});
|
||||
process.on('SIGTERM', () => {
|
||||
cleanup();
|
||||
stopWorker();
|
||||
process.exit(0);
|
||||
});
|
||||
process.on('exit', cleanup);
|
||||
process.on('exit', () => {
|
||||
if (scanRunningUnconfirmed) return;
|
||||
stopWorker();
|
||||
});
|
||||
|
||||
// Poll for the workflow to register in session.json; the spinner resolves once it does.
|
||||
spinner.message('Waiting for the scan to start');
|
||||
@@ -221,10 +384,17 @@ export async function start(args: StartArgs): Promise<void> {
|
||||
const resumeAttempts: { workflowId: string }[] = session.session?.resumeAttempts ?? [];
|
||||
|
||||
// Fresh: session.json appears with originalWorkflowId. Resume: new resumeAttempts entry.
|
||||
const ready = isResume ? resumeAttempts.length > initialResumeCount : !!session.session?.originalWorkflowId;
|
||||
const ready = isResume
|
||||
? resumeAttempts.slice(initialResumeCount).some((attempt) => attempt.workflowId === workflowId)
|
||||
: session.session?.originalWorkflowId === workflowId;
|
||||
|
||||
if (ready) {
|
||||
started = true;
|
||||
try {
|
||||
clearPendingWorkflowIdentity(workspacePath, taskQueue);
|
||||
} catch {
|
||||
warn(`Scan ${workspace} started, but its launch record could not be removed.`);
|
||||
}
|
||||
spinner.stop(`Scan started — ${workspace}`);
|
||||
printInfo(args, workspace, repo.hostPath, workspacesDir);
|
||||
if (args.follow) {
|
||||
@@ -238,10 +408,52 @@ export async function start(args: StartArgs): Promise<void> {
|
||||
await sleep(2000);
|
||||
}
|
||||
|
||||
if (classifyStartupTimeout(sessionJson) === 'scan-running') {
|
||||
scanRunningUnconfirmed = true;
|
||||
spinner.error('The scan started, but this CLI could not confirm it');
|
||||
printUnconfirmedScanHint(workspace, taskQueue, containerName);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
spinner.error('Timed out waiting for the scan to start');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the startup timeout: 'scan-running' when session.json already holds durable state this
|
||||
* release understands, which only the worker writes and only after Temporal began executing the
|
||||
* workflow; 'unregistered' when nothing proves the scan started. The distinction decides whether
|
||||
* timing out may stop the worker container.
|
||||
*/
|
||||
export function classifyStartupTimeout(sessionJsonPath: string): 'unregistered' | 'scan-running' {
|
||||
let session: unknown;
|
||||
try {
|
||||
session = JSON.parse(fs.readFileSync(sessionJsonPath, 'utf-8'));
|
||||
} catch {
|
||||
return 'unregistered';
|
||||
}
|
||||
if (!isRecord(session) || !isCurrentDurableState(session.durableScanState)) {
|
||||
return 'unregistered';
|
||||
}
|
||||
return 'scan-running';
|
||||
}
|
||||
|
||||
/** Point the operator at a scan that is running but whose startup this CLI could not confirm. */
|
||||
function printUnconfirmedScanHint(workspace: string, taskQueue: string, containerName: string): void {
|
||||
console.log('');
|
||||
console.log(' The scan is running and was left alone; only its startup confirmation is missing.');
|
||||
console.log('');
|
||||
console.log(` Workspace: ${workspace}`);
|
||||
console.log(` Task queue: ${taskQueue}`);
|
||||
console.log(` Container: ${containerName}`);
|
||||
console.log('');
|
||||
console.log(' Inspect it:');
|
||||
console.log(` Live logs: ${commandPrefix()} logs ${workspace}`);
|
||||
console.log(` Worker logs: docker logs ${containerName}`);
|
||||
console.log(' Dashboard: http://localhost:8233');
|
||||
console.log('');
|
||||
}
|
||||
|
||||
/**
|
||||
* Follow a just-started scan (for `--follow`, aimed at CI): stream its log while Temporal drives
|
||||
* completion, then exit on the workflow outcome — 0 if the assessment ran, 1 if the scan failed.
|
||||
@@ -331,7 +543,7 @@ function printInfo(args: StartArgs, workspace: string, repoPath: string, workspa
|
||||
return;
|
||||
}
|
||||
|
||||
const reportPath = path.join(workspacesDir, workspace, FINAL_REPORT_PDF_FILENAME);
|
||||
const reportDir = path.join(workspacesDir, workspace);
|
||||
|
||||
// When following, the scan log streams inline next, so the "run these to watch it" hints
|
||||
// would only contradict that.
|
||||
@@ -345,6 +557,8 @@ function printInfo(args: StartArgs, workspace: string, repoPath: string, workspa
|
||||
|
||||
console.log('');
|
||||
console.log(' Report (when the scan finishes):');
|
||||
console.log(` ${reportPath}`);
|
||||
console.log(` ${reportDir}${path.sep}`);
|
||||
console.log(` ${FINAL_REPORT_PDF_FILENAME}`);
|
||||
console.log(` ${FINAL_REPORT_MD_FILENAME}`);
|
||||
console.log('');
|
||||
}
|
||||
|
||||
@@ -3,21 +3,28 @@
|
||||
*
|
||||
* While the scan runs, polls Temporal and redraws the phase/agent tree on a
|
||||
* terminal (a pipe or a finished scan gets a single frame). When the scan reaches
|
||||
* a terminal state, prints the overall result and exits. Reads Temporal directly —
|
||||
* no worker, no session files — so it needs Temporal up and shows scans within its
|
||||
* ~24h retention window.
|
||||
* a terminal state, prints the overall result and exits. Local session records prove
|
||||
* the target's canonical workspace/workflow identity; the progress itself is read from
|
||||
* Temporal directly — no worker — so it needs Temporal up and shows scans within its
|
||||
* retention window (Shannon configures seven days by default; see SHANNON_TEMPORAL_RETENTION).
|
||||
*/
|
||||
|
||||
import { setTimeout as sleep } from 'node:timers/promises';
|
||||
import { fail } from '../errors.js';
|
||||
import { isLocal } from '../mode.js';
|
||||
import { failWith } from '../errors.js';
|
||||
import { commandPrefix, isLocal } from '../mode.js';
|
||||
import { type RenderInput, renderScan } from '../scan/render.js';
|
||||
import { toStatusJson } from '../scan/status-json.js';
|
||||
import { resolveWorkflowId } from '../session.js';
|
||||
import { displaySplash } from '../splash.js';
|
||||
import { describeScan, getTerminalOutcome, queryProgress, type ScanDescription } from '../temporal-client.js';
|
||||
import {
|
||||
ActivityMirrorError,
|
||||
describeScan,
|
||||
getTerminalOutcome,
|
||||
queryProgress,
|
||||
type ScanDescription,
|
||||
} from '../temporal-client.js';
|
||||
import { stdoutIsTerminal, supportsColor } from '../tty.js';
|
||||
import { getVersion } from '../version.js';
|
||||
import { resolveScanIdentity } from '../workspaces.js';
|
||||
|
||||
const HIDE_CURSOR = '\x1b[?25l';
|
||||
const SHOW_CURSOR = '\x1b[?25h';
|
||||
@@ -30,6 +37,25 @@ function isTerminalStatus(status: string): boolean {
|
||||
return status !== 'RUNNING' && status !== 'UNSPECIFIED';
|
||||
}
|
||||
|
||||
/**
|
||||
* Read one scan description, telling the two failure modes apart. A stale activity mirror
|
||||
* carries its own message and needs a CLI update; anything else is a read that did not reach
|
||||
* a usable answer, which is most often Temporal being down.
|
||||
*/
|
||||
async function readScanDescription(workflowId: string): Promise<ScanDescription | null> {
|
||||
try {
|
||||
return await describeScan(workflowId);
|
||||
} catch (error) {
|
||||
if (error instanceof ActivityMirrorError) failWith('CLI_SCAN_SCHEMA_UNSUPPORTED', error.message);
|
||||
failWith(
|
||||
'CLI_SCAN_STATUS_UNAVAILABLE',
|
||||
"Could not read this scan's progress.",
|
||||
'If Temporal is not running, start a scan to bring it up. If it is running, this build of the CLI',
|
||||
'does not recognise part of the scan and needs updating.',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Match SGR color escapes (ESC[…m) so a line's on-screen width excludes them. Built from the ESC
|
||||
// char code so the source carries no literal control character.
|
||||
const ANSI_PATTERN = new RegExp(`${String.fromCharCode(27)}\\[[0-9;]*m`, 'g');
|
||||
@@ -128,10 +154,10 @@ async function watch(workspace: string, workflowId: string): Promise<never> {
|
||||
}, RENDER_MS);
|
||||
|
||||
for (;;) {
|
||||
const desc = await describeScan(workflowId);
|
||||
const desc = await readScanDescription(workflowId);
|
||||
if (!desc) {
|
||||
clearInterval(ticker);
|
||||
fail(`Scan "${workspace}" is no longer in Temporal.`);
|
||||
failWith('CLI_SCAN_NOT_FOUND', `Scan "${workspace}" is no longer in Temporal.`);
|
||||
}
|
||||
|
||||
if (isTerminalStatus(desc.status)) {
|
||||
@@ -153,23 +179,40 @@ async function snapshot(workspace: string, workflowId: string, desc: ScanDescrip
|
||||
: buildRunningInput(workspace, workflowId, desc);
|
||||
}
|
||||
|
||||
export async function status(workspace: string, opts: { readonly json: boolean }): Promise<void> {
|
||||
// A resume spawns a new workflow id (recorded in session.json); resolve through there so status
|
||||
// follows the current resume, not the superseded original. Fresh scans: the name is the id.
|
||||
const workflowId = resolveWorkflowId(workspace) ?? workspace;
|
||||
|
||||
let desc: ScanDescription | null;
|
||||
try {
|
||||
desc = await describeScan(workflowId);
|
||||
} catch {
|
||||
fail('Could not reach Temporal at 127.0.0.1:7233.', 'Start Temporal (it comes up with a scan) and try again.');
|
||||
export async function status(target: string, opts: { readonly json: boolean }): Promise<void> {
|
||||
// Target selection picked a string; identity resolution proves the canonical workspace and
|
||||
// workflow pair from session records before Temporal is queried. A workspace name follows its
|
||||
// latest resume; an exact recorded workflow id keeps addressing that execution. A raw id with
|
||||
// no local record is refused rather than echoed into the required workspace field.
|
||||
const identity = resolveScanIdentity(target);
|
||||
if (identity.kind === 'ambiguous') {
|
||||
failWith(
|
||||
'CLI_SCAN_IDENTITY_AMBIGUOUS',
|
||||
`Multiple workspaces claim workflow ID "${target}": ${identity.claims.join(', ')}.`,
|
||||
`Run '${commandPrefix()} scans' and pass the workspace directory name instead.`,
|
||||
);
|
||||
}
|
||||
if (identity.kind === 'not-found') {
|
||||
failWith(
|
||||
'CLI_SCAN_IDENTITY_NOT_FOUND',
|
||||
identity.reason === 'unreadable-record'
|
||||
? `Workspace "${target}" has no readable session record (${identity.sessionPath}).`
|
||||
: `No scan matches "${target}" in the local workspace records.`,
|
||||
`Run '${commandPrefix()} scans' to list scans.`,
|
||||
'Temporal dashboard: http://localhost:8233',
|
||||
);
|
||||
}
|
||||
const { workspace, workflowId } = identity;
|
||||
|
||||
const desc = await readScanDescription(workflowId);
|
||||
|
||||
if (!desc) {
|
||||
fail(
|
||||
failWith(
|
||||
'CLI_SCAN_NOT_FOUND',
|
||||
`No scan found for "${workspace}".`,
|
||||
'',
|
||||
'Scans are visible while running and for ~24h after they finish (Temporal retention).',
|
||||
"Scan histories are available while a scan runs and within Temporal's retention window after it finishes.",
|
||||
"Shannon configures 7 days of retention by default (override: SHANNON_TEMPORAL_RETENTION). Expired histories can't be restored.",
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
+820
-64
@@ -1,25 +1,43 @@
|
||||
/**
|
||||
* `shannon stop` command — stop one scan by workspace, or every scan with --all.
|
||||
* `shannon stop` command: stop one scan by workspace, or every scan with --all.
|
||||
* Never touches infra or data; to wipe Temporal state entirely, use `shannon reset`.
|
||||
*/
|
||||
|
||||
import path from 'node:path';
|
||||
import * as p from '@clack/prompts';
|
||||
import { confirmOrExit } from '../confirm.js';
|
||||
import {
|
||||
anyRunningScanWorkflow,
|
||||
type CommandQueryResult,
|
||||
ensureDocker,
|
||||
isTemporalReady,
|
||||
isWorkflowRunning,
|
||||
runningContainers,
|
||||
type RunningScanContainer,
|
||||
runningContainersChecked,
|
||||
runningScanContainersChecked,
|
||||
scanFilter,
|
||||
stopContainers,
|
||||
terminateAllWorkflows,
|
||||
terminateWorkflow,
|
||||
WORKER_FILTER,
|
||||
WORKFLOW_ID_PROTOCOL,
|
||||
} from '../docker.js';
|
||||
import { fail, failUsage, warn } from '../errors.js';
|
||||
import { getWorkspacesDir } from '../home.js';
|
||||
import { commandPrefix } from '../mode.js';
|
||||
import { resolveRunFile } from '../paths.js';
|
||||
import {
|
||||
clearPendingWorkflowIdentity,
|
||||
type PendingWorkflowIdentity,
|
||||
readPendingWorkflowIdentities,
|
||||
} from '../pending-workflow.js';
|
||||
import { resolveWorkflowId } from '../session.js';
|
||||
import {
|
||||
describeWorkflowLifecycle,
|
||||
listRunningScanWorkflows,
|
||||
type RunningScanWorkflow,
|
||||
refreshWorkflowLifecycleConnection,
|
||||
requestWorkflowCancellation,
|
||||
requestWorkflowTermination,
|
||||
type WorkflowLifecycleState,
|
||||
} from '../temporal-client.js';
|
||||
import { listWorkspaces, resolveScanIdentity } from '../workspaces.js';
|
||||
import { appendCancellationFallback } from './logs.js';
|
||||
|
||||
export interface StopOptions {
|
||||
all: boolean;
|
||||
@@ -27,101 +45,839 @@ export interface StopOptions {
|
||||
workspace?: string;
|
||||
}
|
||||
|
||||
const CANCELLATION_GRACE_MS = 10_000;
|
||||
const CANCELLATION_POLL_MS = 250;
|
||||
const TERMINATION_VERIFY_MS = 5_000;
|
||||
const TERMINATION_ATTEMPTS = 2;
|
||||
const TERMINATION_REASON = 'Stopped after cancellation grace period';
|
||||
const CANDIDATE_REGISTRATION_SETTLE_MS = 3_000;
|
||||
const VISIBILITY_SETTLE_MS = 1_000;
|
||||
const VISIBILITY_MAX_SETTLE_MS = 5_000;
|
||||
|
||||
export type WorkflowStopOutcome =
|
||||
| { readonly kind: 'graceful' }
|
||||
| { readonly kind: 'forced' }
|
||||
| { readonly kind: 'already-closed' }
|
||||
| { readonly kind: 'unverified' };
|
||||
|
||||
export type ContainerStopOutcome =
|
||||
| { readonly kind: 'stopped'; readonly hadContainers: boolean }
|
||||
| { readonly kind: 'still-running'; readonly remaining: number }
|
||||
| { readonly kind: 'unverified' };
|
||||
|
||||
export interface StopLifecycle {
|
||||
readonly cancel: (workflowId: string) => Promise<'requested' | 'not-found'>;
|
||||
readonly describe: (workflowId: string) => Promise<WorkflowLifecycleState>;
|
||||
readonly refresh: () => Promise<void>;
|
||||
readonly terminate: (workflowId: string) => Promise<'requested' | 'not-found'>;
|
||||
readonly containers: (filter: readonly string[]) => CommandQueryResult<string[]>;
|
||||
readonly stopContainers: (ids: readonly string[]) => Promise<void>;
|
||||
readonly appendFallback: (workspace: string) => void;
|
||||
readonly wait: (milliseconds: number) => Promise<void>;
|
||||
readonly now: () => number;
|
||||
}
|
||||
|
||||
export interface WorkflowStopTarget {
|
||||
readonly workflowId: string;
|
||||
readonly workspace?: string;
|
||||
readonly containerCandidate: boolean;
|
||||
/** Safe to synthesize a log marker when this CLI-owned launch never reached session registration. */
|
||||
readonly preRegistrationFallback?: boolean;
|
||||
}
|
||||
|
||||
export interface WorkflowStopResult {
|
||||
readonly target: WorkflowStopTarget;
|
||||
readonly outcome: WorkflowStopOutcome;
|
||||
}
|
||||
|
||||
export interface StopExecutionResult {
|
||||
readonly workflows: readonly WorkflowStopResult[];
|
||||
readonly containers: ContainerStopOutcome;
|
||||
readonly preRegistrationWorkspaces: readonly string[];
|
||||
}
|
||||
|
||||
export interface WorkflowTargetPlan {
|
||||
readonly targets: readonly WorkflowStopTarget[];
|
||||
readonly containersWithoutVerifiedWorkflowId: readonly string[];
|
||||
}
|
||||
|
||||
interface PendingWorkflowReference {
|
||||
readonly workspace: string;
|
||||
readonly identity: PendingWorkflowIdentity;
|
||||
}
|
||||
|
||||
interface PendingWorkflowTargets {
|
||||
readonly byWorkspace: ReadonlyMap<string, readonly PendingWorkflowIdentity[]>;
|
||||
readonly references: readonly PendingWorkflowReference[];
|
||||
readonly unreadableCount: number;
|
||||
}
|
||||
|
||||
const stopLifecycle: StopLifecycle = {
|
||||
cancel: requestWorkflowCancellation,
|
||||
describe: describeWorkflowLifecycle,
|
||||
refresh: refreshWorkflowLifecycleConnection,
|
||||
terminate: (workflowId) => requestWorkflowTermination(workflowId, TERMINATION_REASON),
|
||||
containers: runningContainersChecked,
|
||||
stopContainers: (ids) => stopContainers([...ids]),
|
||||
appendFallback: (workspace) => {
|
||||
const logFile = resolveRunFile(path.join(getWorkspacesDir(), workspace), 'workflow.log');
|
||||
appendCancellationFallback(logFile);
|
||||
},
|
||||
wait: (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds)),
|
||||
now: Date.now,
|
||||
};
|
||||
|
||||
function readPendingTargets(workspaces: readonly string[]): PendingWorkflowTargets {
|
||||
const byWorkspace = new Map<string, readonly PendingWorkflowIdentity[]>();
|
||||
const references: PendingWorkflowReference[] = [];
|
||||
let unreadableCount = 0;
|
||||
|
||||
for (const workspace of new Set(workspaces)) {
|
||||
const workspacePath = path.join(getWorkspacesDir(), workspace);
|
||||
const result = readPendingWorkflowIdentities(workspacePath);
|
||||
unreadableCount += result.unreadableCount;
|
||||
if (result.identities.length === 0) continue;
|
||||
byWorkspace.set(workspace, result.identities);
|
||||
for (const identity of result.identities) references.push({ workspace, identity });
|
||||
}
|
||||
|
||||
return { byWorkspace, references, unreadableCount };
|
||||
}
|
||||
|
||||
function clearPendingTargets(references: readonly PendingWorkflowReference[]): number {
|
||||
let failures = 0;
|
||||
for (const reference of references) {
|
||||
try {
|
||||
clearPendingWorkflowIdentity(path.join(getWorkspacesDir(), reference.workspace), reference.identity.task_queue);
|
||||
} catch {
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
return failures;
|
||||
}
|
||||
|
||||
function workflowClosed(state: WorkflowLifecycleState): boolean {
|
||||
return state.kind === 'terminal' || state.kind === 'not-found';
|
||||
}
|
||||
|
||||
/** Poll direct workflow state until Temporal positively confirms closure or the deadline expires. */
|
||||
async function waitForWorkflowClosure(
|
||||
workflowId: string,
|
||||
lifecycle: StopLifecycle,
|
||||
deadline: number,
|
||||
pollMs: number,
|
||||
): Promise<boolean> {
|
||||
while (true) {
|
||||
if (deadline - lifecycle.now() <= 0) return false;
|
||||
try {
|
||||
if (workflowClosed(await lifecycle.describe(workflowId))) return true;
|
||||
} catch {
|
||||
// An unavailable status is unknown, never evidence that the workflow closed.
|
||||
}
|
||||
|
||||
const remaining = deadline - lifecycle.now();
|
||||
if (remaining <= 0) return false;
|
||||
await lifecycle.wait(Math.min(pollMs, remaining));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop a single scan. Terminating the workflow both clears Temporal's record and
|
||||
* brings the container down (the worker waits on the workflow result), so that runs
|
||||
* first; `docker stop` is the fallback for the pre-registration window and an
|
||||
* unreachable Temporal. The stop is then verified rather than assumed.
|
||||
* Request cooperative cancellation, then make at most two termination attempts when the
|
||||
* workflow does not close during its grace period. Every success is backed by direct state.
|
||||
*/
|
||||
export async function stopWorkflowCancelFirst(
|
||||
workflowId: string,
|
||||
lifecycle: StopLifecycle = stopLifecycle,
|
||||
graceMs: number = CANCELLATION_GRACE_MS,
|
||||
pollMs: number = CANCELLATION_POLL_MS,
|
||||
verifyMs: number = TERMINATION_VERIFY_MS,
|
||||
): Promise<WorkflowStopOutcome> {
|
||||
try {
|
||||
if ((await lifecycle.cancel(workflowId)) === 'not-found') return { kind: 'already-closed' };
|
||||
} catch {
|
||||
// The request may have reached Temporal even when its acknowledgement was lost.
|
||||
}
|
||||
|
||||
if (await waitForWorkflowClosure(workflowId, lifecycle, lifecycle.now() + graceMs, pollMs)) {
|
||||
return { kind: 'graceful' };
|
||||
}
|
||||
|
||||
const verifyPerAttemptMs = Math.max(pollMs, Math.ceil(verifyMs / TERMINATION_ATTEMPTS));
|
||||
for (let attempt = 0; attempt < TERMINATION_ATTEMPTS; attempt++) {
|
||||
if (attempt > 0) {
|
||||
try {
|
||||
await lifecycle.refresh();
|
||||
} catch {
|
||||
// The termination call below makes one final bounded connection attempt.
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
if ((await lifecycle.terminate(workflowId)) === 'not-found') return { kind: 'already-closed' };
|
||||
} catch {
|
||||
// A lost acknowledgement is resolved by the direct verification below.
|
||||
}
|
||||
if (await waitForWorkflowClosure(workflowId, lifecycle, lifecycle.now() + verifyPerAttemptMs, pollMs)) {
|
||||
return { kind: 'forced' };
|
||||
}
|
||||
}
|
||||
|
||||
return { kind: 'unverified' };
|
||||
}
|
||||
|
||||
/** Apply the same bounded lifecycle concurrently to a captured set of workflow IDs. */
|
||||
export async function stopWorkflowsCancelFirst(
|
||||
workflowIds: readonly string[],
|
||||
lifecycle: StopLifecycle = stopLifecycle,
|
||||
graceMs: number = CANCELLATION_GRACE_MS,
|
||||
pollMs: number = CANCELLATION_POLL_MS,
|
||||
verifyMs: number = TERMINATION_VERIFY_MS,
|
||||
): Promise<readonly WorkflowStopOutcome[]> {
|
||||
const settlements = await Promise.allSettled(
|
||||
workflowIds.map((workflowId) => stopWorkflowCancelFirst(workflowId, lifecycle, graceMs, pollMs, verifyMs)),
|
||||
);
|
||||
return settlements.map((settlement) =>
|
||||
settlement.status === 'fulfilled' ? settlement.value : { kind: 'unverified' },
|
||||
);
|
||||
}
|
||||
|
||||
/** Stop exactly the captured workers, then fail closed if any matching worker remains or appears. */
|
||||
export async function stopContainersAndVerify(
|
||||
initialIds: readonly string[],
|
||||
filter: readonly string[],
|
||||
lifecycle: StopLifecycle = stopLifecycle,
|
||||
): Promise<ContainerStopOutcome> {
|
||||
try {
|
||||
await lifecycle.stopContainers(initialIds);
|
||||
} catch {
|
||||
// The post-stop query below decides whether the operation actually succeeded.
|
||||
}
|
||||
|
||||
let after: CommandQueryResult<string[]>;
|
||||
try {
|
||||
after = lifecycle.containers(filter);
|
||||
} catch {
|
||||
return { kind: 'unverified' };
|
||||
}
|
||||
if (after.kind === 'unavailable') return { kind: 'unverified' };
|
||||
if (after.value.length > 0) return { kind: 'still-running', remaining: after.value.length };
|
||||
return { kind: 'stopped', hadContainers: initialIds.length > 0 };
|
||||
}
|
||||
|
||||
function addWorkflowTarget(targets: Map<string, WorkflowStopTarget>, candidate: WorkflowStopTarget): void {
|
||||
const current = targets.get(candidate.workflowId);
|
||||
if (current === undefined) {
|
||||
targets.set(candidate.workflowId, candidate);
|
||||
return;
|
||||
}
|
||||
const workspace = current.workspace ?? candidate.workspace;
|
||||
targets.set(candidate.workflowId, {
|
||||
workflowId: candidate.workflowId,
|
||||
...(workspace !== undefined && { workspace }),
|
||||
containerCandidate: current.containerCandidate || candidate.containerCandidate,
|
||||
...((current.preRegistrationFallback === true || candidate.preRegistrationFallback === true) && {
|
||||
preRegistrationFallback: true,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the stop union from immutable container candidates, recorded session IDs,
|
||||
* pre-registration launch records, and Temporal visibility. Visibility supplies positive
|
||||
* targets but never proves absence.
|
||||
*/
|
||||
function verifiedContainerWorkflowId(
|
||||
container: RunningScanContainer,
|
||||
visibleWorkflows: readonly RunningScanWorkflow[],
|
||||
): string | undefined {
|
||||
if (container.workerProtocol === WORKFLOW_ID_PROTOCOL && container.workflowId !== undefined) {
|
||||
return container.workflowId;
|
||||
}
|
||||
if (container.taskQueue === undefined) return undefined;
|
||||
const matches = visibleWorkflows.filter((workflow) => workflow.taskQueue === container.taskQueue);
|
||||
return matches.length === 1 ? matches[0]?.workflowId : undefined;
|
||||
}
|
||||
|
||||
export function buildWorkflowTargetPlan(
|
||||
containers: readonly RunningScanContainer[],
|
||||
recordedByWorkspace: ReadonlyMap<string, string>,
|
||||
visibleWorkflows: readonly RunningScanWorkflow[],
|
||||
pendingByWorkspace: ReadonlyMap<string, readonly PendingWorkflowIdentity[]> = new Map(),
|
||||
): WorkflowTargetPlan {
|
||||
const targets = new Map<string, WorkflowStopTarget>();
|
||||
const containersWithoutVerifiedWorkflowId: string[] = [];
|
||||
|
||||
for (const container of containers) {
|
||||
const verifiedWorkflowId = verifiedContainerWorkflowId(container, visibleWorkflows);
|
||||
if (verifiedWorkflowId === undefined) containersWithoutVerifiedWorkflowId.push(container.id);
|
||||
else {
|
||||
addWorkflowTarget(targets, {
|
||||
workflowId: verifiedWorkflowId,
|
||||
...(container.workspace !== undefined && { workspace: container.workspace }),
|
||||
containerCandidate: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
for (const [workspace, workflowId] of recordedByWorkspace) {
|
||||
addWorkflowTarget(targets, {
|
||||
workflowId,
|
||||
workspace,
|
||||
containerCandidate:
|
||||
containers.some((container) => verifiedContainerWorkflowId(container, visibleWorkflows) === workflowId) ||
|
||||
pendingByWorkspace.get(workspace)?.some((identity) => identity.workflow_id === workflowId) === true,
|
||||
});
|
||||
}
|
||||
|
||||
for (const [workspace, identities] of pendingByWorkspace) {
|
||||
for (const identity of identities) {
|
||||
addWorkflowTarget(targets, {
|
||||
workflowId: identity.workflow_id,
|
||||
workspace,
|
||||
containerCandidate: true,
|
||||
...(!recordedByWorkspace.has(workspace) && { preRegistrationFallback: true }),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
for (const workflow of visibleWorkflows) {
|
||||
const matchingWorkspaces = new Set(
|
||||
containers
|
||||
.filter((container) => container.taskQueue === workflow.taskQueue && container.workspace !== undefined)
|
||||
.flatMap((container) => container.workspace ?? []),
|
||||
);
|
||||
for (const [workspace, identities] of pendingByWorkspace) {
|
||||
if (identities.some((identity) => identity.task_queue === workflow.taskQueue)) matchingWorkspaces.add(workspace);
|
||||
}
|
||||
const workspace = matchingWorkspaces.size === 1 ? [...matchingWorkspaces][0] : undefined;
|
||||
addWorkflowTarget(targets, {
|
||||
workflowId: workflow.workflowId,
|
||||
...(workspace !== undefined && { workspace }),
|
||||
containerCandidate:
|
||||
containers.some((container) => container.taskQueue === workflow.taskQueue) ||
|
||||
[...pendingByWorkspace.values()].some((identities) =>
|
||||
identities.some((identity) => identity.task_queue === workflow.taskQueue),
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
return { targets: [...targets.values()], containersWithoutVerifiedWorkflowId };
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop known workflows while their workers can finalize, stop the captured workers, then
|
||||
* re-describe every container candidate. The last pass closes a NotFound-to-started race.
|
||||
*/
|
||||
export async function executeStopPlan(
|
||||
targets: readonly WorkflowStopTarget[],
|
||||
containers: readonly RunningScanContainer[],
|
||||
filter: readonly string[],
|
||||
lifecycle: StopLifecycle = stopLifecycle,
|
||||
graceMs: number = CANCELLATION_GRACE_MS,
|
||||
pollMs: number = CANCELLATION_POLL_MS,
|
||||
verifyMs: number = TERMINATION_VERIFY_MS,
|
||||
candidateSettleMs: number = CANDIDATE_REGISTRATION_SETTLE_MS,
|
||||
): Promise<StopExecutionResult> {
|
||||
const initialOutcomes = await stopWorkflowsCancelFirst(
|
||||
targets.map((target) => target.workflowId),
|
||||
lifecycle,
|
||||
graceMs,
|
||||
pollMs,
|
||||
verifyMs,
|
||||
);
|
||||
const outcomes = new Map<string, WorkflowStopOutcome>();
|
||||
for (let index = 0; index < targets.length; index++) {
|
||||
const target = targets[index];
|
||||
const outcome = initialOutcomes[index];
|
||||
if (target !== undefined && outcome !== undefined) outcomes.set(target.workflowId, outcome);
|
||||
}
|
||||
|
||||
const containerOutcome = await stopContainersAndVerify(
|
||||
containers.map((container) => container.id),
|
||||
filter,
|
||||
lifecycle,
|
||||
);
|
||||
const preRegistrationWorkspaces = new Set<string>();
|
||||
|
||||
if (containerOutcome.kind === 'stopped') {
|
||||
const candidates = targets.filter((target) => target.containerCandidate);
|
||||
|
||||
for (const target of candidates) {
|
||||
const initialOutcome = outcomes.get(target.workflowId) ?? { kind: 'unverified' };
|
||||
const settleDeadline = lifecycle.now() + candidateSettleMs;
|
||||
let onlyObservedNotFound = initialOutcome.kind === 'already-closed' || initialOutcome.kind === 'unverified';
|
||||
|
||||
while (true) {
|
||||
try {
|
||||
const state = await lifecycle.describe(target.workflowId);
|
||||
if (state.kind === 'open') {
|
||||
onlyObservedNotFound = false;
|
||||
const outcome = await stopWorkflowCancelFirst(target.workflowId, lifecycle, graceMs, pollMs, verifyMs);
|
||||
outcomes.set(target.workflowId, outcome);
|
||||
if (outcome.kind !== 'already-closed') break;
|
||||
}
|
||||
if (state.kind === 'terminal') {
|
||||
onlyObservedNotFound = false;
|
||||
if (initialOutcome.kind === 'unverified') outcomes.set(target.workflowId, { kind: 'already-closed' });
|
||||
}
|
||||
if (state.kind === 'unknown') {
|
||||
onlyObservedNotFound = false;
|
||||
outcomes.set(target.workflowId, { kind: 'unverified' });
|
||||
break;
|
||||
}
|
||||
if (initialOutcome.kind === 'unverified') outcomes.set(target.workflowId, { kind: 'already-closed' });
|
||||
} catch {
|
||||
onlyObservedNotFound = false;
|
||||
outcomes.set(target.workflowId, { kind: 'unverified' });
|
||||
break;
|
||||
}
|
||||
|
||||
const remaining = settleDeadline - lifecycle.now();
|
||||
if (remaining <= 0) {
|
||||
if (onlyObservedNotFound && target.workspace !== undefined && target.preRegistrationFallback === true) {
|
||||
preRegistrationWorkspaces.add(target.workspace);
|
||||
}
|
||||
break;
|
||||
}
|
||||
try {
|
||||
await lifecycle.wait(Math.min(pollMs, remaining));
|
||||
} catch {
|
||||
outcomes.set(target.workflowId, { kind: 'unverified' });
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
workflows: targets.map((target) => ({
|
||||
target,
|
||||
outcome: outcomes.get(target.workflowId) ?? { kind: 'unverified' },
|
||||
})),
|
||||
containers: containerOutcome,
|
||||
preRegistrationWorkspaces: [...preRegistrationWorkspaces],
|
||||
};
|
||||
}
|
||||
|
||||
function appendFallback(workspace: string, lifecycle: StopLifecycle = stopLifecycle): void {
|
||||
try {
|
||||
lifecycle.appendFallback(workspace);
|
||||
} catch {
|
||||
warn(`scan ${workspace} stopped, but workflow.log could not be marked cancelled.`);
|
||||
}
|
||||
}
|
||||
|
||||
function reportContainerFailure(workspace: string | undefined, outcome: ContainerStopOutcome): void {
|
||||
const target = workspace === undefined ? '--all' : workspace;
|
||||
if (outcome.kind === 'still-running') console.error(`${outcome.remaining} scan worker(s) did not stop.`);
|
||||
else console.error('Docker could not verify that every targeted scan worker stopped.');
|
||||
console.error(`Retry: ${commandPrefix()} stop ${target}`);
|
||||
}
|
||||
|
||||
function withRecordedWorkflows(containers: readonly RunningScanContainer[]): Map<string, string> {
|
||||
const recorded = new Map<string, string>();
|
||||
for (const container of containers) {
|
||||
if (container.workspace === undefined || recorded.has(container.workspace)) continue;
|
||||
const workflowId = resolveWorkflowId(container.workspace);
|
||||
if (workflowId !== undefined) recorded.set(container.workspace, workflowId);
|
||||
}
|
||||
return recorded;
|
||||
}
|
||||
|
||||
function resolveTargetWorkspaces(targets: readonly WorkflowStopTarget[]): readonly WorkflowStopTarget[] {
|
||||
return targets.map((target) => {
|
||||
if (target.workspace !== undefined) return target;
|
||||
const identity = resolveScanIdentity(target.workflowId);
|
||||
return identity.kind === 'ok' ? { ...target, workspace: identity.workspace } : target;
|
||||
});
|
||||
}
|
||||
|
||||
function unverifiedWorkflowCount(results: readonly WorkflowStopResult[]): number {
|
||||
return results.filter((result) => result.outcome.kind === 'unverified').length;
|
||||
}
|
||||
|
||||
function appendVerifiedFallbacks(result: StopExecutionResult): void {
|
||||
const workspaces = new Set(result.preRegistrationWorkspaces);
|
||||
for (const workflow of result.workflows) {
|
||||
if (workflow.outcome.kind === 'forced' && workflow.target.workspace !== undefined) {
|
||||
workspaces.add(workflow.target.workspace);
|
||||
}
|
||||
}
|
||||
for (const workspace of workspaces) appendFallback(workspace);
|
||||
}
|
||||
|
||||
function visibleWorkflowsForWorkspace(
|
||||
workspace: string,
|
||||
containers: readonly RunningScanContainer[],
|
||||
pending: PendingWorkflowTargets,
|
||||
visible: readonly RunningScanWorkflow[],
|
||||
): readonly RunningScanWorkflow[] {
|
||||
const taskQueues = new Set(containers.flatMap((container) => container.taskQueue ?? []));
|
||||
for (const identity of pending.byWorkspace.get(workspace) ?? []) taskQueues.add(identity.task_queue);
|
||||
|
||||
return visible.filter((workflow) => {
|
||||
if (taskQueues.has(workflow.taskQueue)) return true;
|
||||
const identity = resolveScanIdentity(workflow.workflowId);
|
||||
return identity.kind === 'ok' && identity.workspace === workspace;
|
||||
});
|
||||
}
|
||||
|
||||
/** Stop one scan while keeping its worker alive long enough to flush a graceful cancellation. */
|
||||
async function stopSingleScan(workspace: string, yes: boolean): Promise<void> {
|
||||
const workflowId = resolveWorkflowId(workspace);
|
||||
const filter = scanFilter(workspace);
|
||||
const temporalUp = isTemporalReady();
|
||||
const containerQuery = runningScanContainersChecked(filter);
|
||||
if (containerQuery.kind === 'unavailable') {
|
||||
fail(`Could not inspect the scan worker for ${workspace}.`, `Retry: ${commandPrefix()} stop ${workspace}`);
|
||||
}
|
||||
const containers = containerQuery.value.map((container) => ({ ...container, workspace }));
|
||||
const recordedWorkflowId = resolveWorkflowId(workspace);
|
||||
const recorded = new Map<string, string>();
|
||||
if (recordedWorkflowId !== undefined) recorded.set(workspace, recordedWorkflowId);
|
||||
const pending = readPendingTargets([workspace]);
|
||||
const discovery = await discoverRunningWorkflows();
|
||||
const visible =
|
||||
discovery.kind === 'ok' ? visibleWorkflowsForWorkspace(workspace, containers, pending, discovery.workflows) : [];
|
||||
const plan = buildWorkflowTargetPlan(containers, recorded, visible, pending.byWorkspace);
|
||||
|
||||
const initialContainers = runningContainers(filter);
|
||||
const workflowRunning = Boolean(workflowId && temporalUp && isWorkflowRunning(workflowId));
|
||||
|
||||
// Resolve what is running before prompting, so we never confirm a no-op.
|
||||
if (initialContainers.length === 0 && !workflowRunning) {
|
||||
if (!workflowId) {
|
||||
if (containers.length === 0) {
|
||||
if (plan.targets.length === 0) {
|
||||
if (pending.unreadableCount > 0) {
|
||||
fail(
|
||||
`The launch records for ${workspace} could not be read safely.`,
|
||||
`Retry: ${commandPrefix()} stop ${workspace}`,
|
||||
);
|
||||
}
|
||||
if (discovery.kind === 'unavailable') {
|
||||
fail(
|
||||
`Could not verify whether scan ${workspace} is still running in Temporal.`,
|
||||
`Retry: ${commandPrefix()} stop ${workspace}`,
|
||||
);
|
||||
}
|
||||
fail(`No scan found for workspace: ${workspace}`);
|
||||
}
|
||||
console.log(`Nothing was running for ${workspace}.`);
|
||||
return;
|
||||
|
||||
const onlyRecordedTarget =
|
||||
recordedWorkflowId !== undefined &&
|
||||
pending.references.length === 0 &&
|
||||
pending.unreadableCount === 0 &&
|
||||
discovery.kind === 'ok' &&
|
||||
plan.targets.every((target) => target.workflowId === recordedWorkflowId);
|
||||
if (onlyRecordedTarget) {
|
||||
try {
|
||||
const state = await describeWorkflowLifecycle(recordedWorkflowId);
|
||||
if (state.kind === 'terminal' || state.kind === 'not-found') {
|
||||
console.log(`Nothing was running for ${workspace}.`);
|
||||
return;
|
||||
}
|
||||
if (state.kind === 'unknown') {
|
||||
fail(
|
||||
`Temporal returned an unknown lifecycle state for ${workspace}.`,
|
||||
`Retry: ${commandPrefix()} stop ${workspace}`,
|
||||
);
|
||||
}
|
||||
} catch {
|
||||
fail(
|
||||
`Could not verify whether scan ${workspace} is still running in Temporal.`,
|
||||
`Retry: ${commandPrefix()} stop ${workspace}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
await confirmOrExit('stop', `Stop the scan "${workspace}"?`, yes);
|
||||
|
||||
const spinner = p.spinner();
|
||||
spinner.start(`Stopping scan ${workspace}`);
|
||||
|
||||
if (workflowId && workflowRunning) {
|
||||
terminateWorkflow(workflowId, `Stopped via shannon stop ${workspace}`);
|
||||
}
|
||||
await stopContainers(runningContainers(filter));
|
||||
const initialResult = await executeStopPlan(plan.targets, containers, filter);
|
||||
const visibilitySettle = await stopVisibleWorkflowsUntilSettled(
|
||||
initialResult.workflows,
|
||||
stopLifecycle,
|
||||
VISIBILITY_SETTLE_MS,
|
||||
VISIBILITY_MAX_SETTLE_MS,
|
||||
async () => {
|
||||
const current = await discoverRunningWorkflows();
|
||||
return current.kind === 'ok'
|
||||
? {
|
||||
kind: 'ok',
|
||||
workflows: visibleWorkflowsForWorkspace(workspace, containers, pending, current.workflows),
|
||||
}
|
||||
: current;
|
||||
},
|
||||
);
|
||||
const result: StopExecutionResult = {
|
||||
workflows: visibilitySettle.results,
|
||||
containers: initialResult.containers,
|
||||
preRegistrationWorkspaces: initialResult.preRegistrationWorkspaces,
|
||||
};
|
||||
const unverified = unverifiedWorkflowCount(result.workflows);
|
||||
const finalPending = readPendingTargets([workspace]);
|
||||
const initialPendingKeys = new Set(
|
||||
pending.references.map((reference) => `${reference.identity.task_queue}\0${reference.identity.workflow_id}`),
|
||||
);
|
||||
const newPendingCount = finalPending.references.filter(
|
||||
(reference) => !initialPendingKeys.has(`${reference.identity.task_queue}\0${reference.identity.workflow_id}`),
|
||||
).length;
|
||||
const unreadablePendingCount = Math.max(pending.unreadableCount, finalPending.unreadableCount);
|
||||
const incomplete =
|
||||
result.containers.kind !== 'stopped' ||
|
||||
plan.containersWithoutVerifiedWorkflowId.length > 0 ||
|
||||
discovery.kind === 'unavailable' ||
|
||||
visibilitySettle.kind !== 'settled' ||
|
||||
unreadablePendingCount > 0 ||
|
||||
newPendingCount > 0 ||
|
||||
unverified > 0;
|
||||
|
||||
const stillRunning = runningContainers(filter);
|
||||
if (stillRunning.length > 0) {
|
||||
spinner.error(`Scan ${workspace} may still be running`);
|
||||
console.error(`${stillRunning.length} container(s) did not stop. Retry: ${commandPrefix()} stop ${workspace}`);
|
||||
if (incomplete) {
|
||||
spinner.error(`Scan ${workspace} shutdown could not be fully verified`);
|
||||
if (result.containers.kind !== 'stopped') reportContainerFailure(workspace, result.containers);
|
||||
if (unverified > 0) console.error(`Temporal could not confirm closure for ${unverified} workflow(s).`);
|
||||
if (discovery.kind === 'unavailable') console.error('Temporal could not enumerate every running scan workflow.');
|
||||
if (visibilitySettle.kind === 'unavailable') {
|
||||
console.error('Temporal could not complete the final scan workflow check.');
|
||||
}
|
||||
if (visibilitySettle.kind === 'timed-out') {
|
||||
console.error('Temporal workflow discovery did not settle before its deadline.');
|
||||
}
|
||||
if (unreadablePendingCount > 0) {
|
||||
console.error(`${unreadablePendingCount} launch record(s) could not be read safely.`);
|
||||
}
|
||||
if (newPendingCount > 0) console.error('A new scan launch began while shutdown was running.');
|
||||
if (plan.containersWithoutVerifiedWorkflowId.length > 0) {
|
||||
console.error('A legacy scan worker could not prove its candidate workflow ID.');
|
||||
}
|
||||
console.error(`Retry: ${commandPrefix()} stop ${workspace}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
appendVerifiedFallbacks(result);
|
||||
const clearFailures = clearPendingTargets(pending.references);
|
||||
if (clearFailures > 0) {
|
||||
spinner.error(`Scan ${workspace} stopped, but its launch record could not be cleared`);
|
||||
console.error(`Retry: ${commandPrefix()} stop ${workspace}`);
|
||||
process.exit(1);
|
||||
}
|
||||
spinner.stop(`Stopped scan ${workspace}`);
|
||||
}
|
||||
|
||||
if (workflowId && temporalUp && isWorkflowRunning(workflowId)) {
|
||||
warn(`scan ${workspace} stopped, but its workflow is still Running in Temporal.`);
|
||||
export type WorkflowDiscoveryResult =
|
||||
| { readonly kind: 'ok'; readonly workflows: readonly RunningScanWorkflow[] }
|
||||
| { readonly kind: 'unavailable' };
|
||||
|
||||
async function discoverRunningWorkflows(): Promise<WorkflowDiscoveryResult> {
|
||||
try {
|
||||
return { kind: 'ok', workflows: await listRunningScanWorkflows() };
|
||||
} catch {
|
||||
return { kind: 'unavailable' };
|
||||
}
|
||||
}
|
||||
|
||||
interface VisibilitySettleResult {
|
||||
readonly results: readonly WorkflowStopResult[];
|
||||
readonly kind: 'settled' | 'unavailable' | 'timed-out';
|
||||
}
|
||||
|
||||
/** Re-enumerate visibility until no new open workflow appears during a bounded quiet horizon. */
|
||||
export async function stopVisibleWorkflowsUntilSettled(
|
||||
seed: readonly WorkflowStopResult[],
|
||||
lifecycle: StopLifecycle = stopLifecycle,
|
||||
settleMs: number = VISIBILITY_SETTLE_MS,
|
||||
maxSettleMs: number = VISIBILITY_MAX_SETTLE_MS,
|
||||
discover: () => Promise<WorkflowDiscoveryResult> = discoverRunningWorkflows,
|
||||
): Promise<VisibilitySettleResult> {
|
||||
const results = new Map(seed.map((result) => [result.target.workflowId, result]));
|
||||
const retriedUnverified = new Set<string>();
|
||||
const recheckedAlreadyClosed = new Set<string>();
|
||||
let quietSince = lifecycle.now();
|
||||
const maxDeadline = quietSince + maxSettleMs;
|
||||
|
||||
while (true) {
|
||||
const discovery = await discover();
|
||||
if (discovery.kind === 'unavailable') return { kind: 'unavailable', results: [...results.values()] };
|
||||
|
||||
const visibleTargets = resolveTargetWorkspaces(
|
||||
buildWorkflowTargetPlan([], new Map(), discovery.workflows).targets,
|
||||
).map((target) => {
|
||||
const existingWorkspace = results.get(target.workflowId)?.target.workspace;
|
||||
return target.workspace === undefined && existingWorkspace !== undefined
|
||||
? { ...target, workspace: existingWorkspace }
|
||||
: target;
|
||||
});
|
||||
const residualTargets = visibleTargets.filter((target) => {
|
||||
const current = results.get(target.workflowId);
|
||||
if (current === undefined) return true;
|
||||
if (current.outcome.kind === 'unverified') return !retriedUnverified.has(target.workflowId);
|
||||
return current.outcome.kind === 'already-closed' && !recheckedAlreadyClosed.has(target.workflowId);
|
||||
});
|
||||
if (residualTargets.length > 0) {
|
||||
for (const target of residualTargets) {
|
||||
if (results.get(target.workflowId)?.outcome.kind === 'unverified') {
|
||||
retriedUnverified.add(target.workflowId);
|
||||
}
|
||||
if (results.get(target.workflowId)?.outcome.kind === 'already-closed') {
|
||||
recheckedAlreadyClosed.add(target.workflowId);
|
||||
}
|
||||
}
|
||||
const outcomes = await stopWorkflowsCancelFirst(
|
||||
residualTargets.map((target) => target.workflowId),
|
||||
lifecycle,
|
||||
);
|
||||
for (let index = 0; index < residualTargets.length; index++) {
|
||||
const target = residualTargets[index];
|
||||
const outcome = outcomes[index];
|
||||
if (target !== undefined && outcome !== undefined) results.set(target.workflowId, { target, outcome });
|
||||
}
|
||||
quietSince = lifecycle.now();
|
||||
}
|
||||
|
||||
const now = lifecycle.now();
|
||||
if (now - quietSince >= settleMs) return { kind: 'settled', results: [...results.values()] };
|
||||
if (now >= maxDeadline) return { kind: 'timed-out', results: [...results.values()] };
|
||||
try {
|
||||
await lifecycle.wait(Math.min(CANCELLATION_POLL_MS, settleMs - (now - quietSince)));
|
||||
} catch {
|
||||
return { kind: 'timed-out', results: [...results.values()] };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function stopAllScans(yes: boolean): Promise<void> {
|
||||
const temporalUp = isTemporalReady();
|
||||
const initial = runningContainers(WORKER_FILTER);
|
||||
const containerQuery = runningScanContainersChecked();
|
||||
if (containerQuery.kind === 'unavailable') {
|
||||
fail('Could not inspect running scan workers.', `Retry: ${commandPrefix()} stop --all`);
|
||||
}
|
||||
const containers = containerQuery.value;
|
||||
let pending = readPendingTargets(listWorkspaces().map((workspace) => workspace.name));
|
||||
const initialDiscovery = await discoverRunningWorkflows();
|
||||
let visible = initialDiscovery.kind === 'ok' ? initialDiscovery.workflows : [];
|
||||
let plan = buildWorkflowTargetPlan(containers, withRecordedWorkflows(containers), visible, pending.byWorkspace);
|
||||
let targets = resolveTargetWorkspaces(plan.targets);
|
||||
|
||||
// Resolve what is running before prompting, so we never confirm a no-op.
|
||||
if (initial.length === 0) {
|
||||
console.log('No running scans to stop.');
|
||||
return;
|
||||
if (containers.length === 0 && targets.length === 0) {
|
||||
if (pending.unreadableCount > 0) {
|
||||
fail('One or more scan launch records could not be read safely.', `Retry: ${commandPrefix()} stop --all`);
|
||||
}
|
||||
if (initialDiscovery.kind === 'unavailable') {
|
||||
fail('Could not verify whether scan workflows are running in Temporal.', `Retry: ${commandPrefix()} stop --all`);
|
||||
}
|
||||
const emptySettleDeadline = stopLifecycle.now() + VISIBILITY_MAX_SETTLE_MS;
|
||||
while (targets.length === 0) {
|
||||
const remaining = emptySettleDeadline - stopLifecycle.now();
|
||||
if (remaining <= 0) {
|
||||
console.log('No running scans to stop.');
|
||||
return;
|
||||
}
|
||||
await stopLifecycle.wait(Math.min(CANCELLATION_POLL_MS, remaining));
|
||||
const confirmation = await discoverRunningWorkflows();
|
||||
if (confirmation.kind === 'unavailable') {
|
||||
fail(
|
||||
'Could not verify whether scan workflows are running in Temporal.',
|
||||
`Retry: ${commandPrefix()} stop --all`,
|
||||
);
|
||||
}
|
||||
visible = confirmation.workflows;
|
||||
pending = readPendingTargets(listWorkspaces().map((workspace) => workspace.name));
|
||||
if (pending.unreadableCount > 0) {
|
||||
fail('One or more scan launch records could not be read safely.', `Retry: ${commandPrefix()} stop --all`);
|
||||
}
|
||||
plan = buildWorkflowTargetPlan(containers, withRecordedWorkflows(containers), visible, pending.byWorkspace);
|
||||
targets = resolveTargetWorkspaces(plan.targets);
|
||||
}
|
||||
}
|
||||
|
||||
await confirmOrExit('stop', 'This will stop all running scans. Continue?', yes);
|
||||
|
||||
const spinner = p.spinner();
|
||||
spinner.start('Stopping all scans');
|
||||
|
||||
if (temporalUp) {
|
||||
terminateAllWorkflows('Stopped via shannon stop --all');
|
||||
}
|
||||
await stopContainers(runningContainers(WORKER_FILTER));
|
||||
const initialResult = await executeStopPlan(targets, containers, WORKER_FILTER);
|
||||
const visibilitySettle = await stopVisibleWorkflowsUntilSettled(initialResult.workflows);
|
||||
const results = visibilitySettle.results;
|
||||
|
||||
const stillRunning = runningContainers(WORKER_FILTER);
|
||||
if (stillRunning.length > 0) {
|
||||
spinner.error(`Stopped ${initial.length - stillRunning.length} of ${initial.length} scans`);
|
||||
console.error(`${stillRunning.length} container(s) did not stop. Retry: ${commandPrefix()} stop --all`);
|
||||
const combinedResult: StopExecutionResult = {
|
||||
workflows: results,
|
||||
containers: initialResult.containers,
|
||||
preRegistrationWorkspaces: initialResult.preRegistrationWorkspaces,
|
||||
};
|
||||
const unverified = unverifiedWorkflowCount(results);
|
||||
const finalPending = readPendingTargets(listWorkspaces().map((workspace) => workspace.name));
|
||||
const initialPendingKeys = new Set(
|
||||
pending.references.map(
|
||||
(reference) => `${reference.workspace}\0${reference.identity.task_queue}\0${reference.identity.workflow_id}`,
|
||||
),
|
||||
);
|
||||
const newPendingCount = finalPending.references.filter(
|
||||
(reference) =>
|
||||
!initialPendingKeys.has(
|
||||
`${reference.workspace}\0${reference.identity.task_queue}\0${reference.identity.workflow_id}`,
|
||||
),
|
||||
).length;
|
||||
const unreadablePendingCount = Math.max(pending.unreadableCount, finalPending.unreadableCount);
|
||||
const temporalDiscoveryFailed = initialDiscovery.kind === 'unavailable' || visibilitySettle.kind === 'unavailable';
|
||||
const temporalDiscoveryTimedOut = visibilitySettle.kind === 'timed-out';
|
||||
const incomplete =
|
||||
combinedResult.containers.kind !== 'stopped' ||
|
||||
plan.containersWithoutVerifiedWorkflowId.length > 0 ||
|
||||
temporalDiscoveryFailed ||
|
||||
temporalDiscoveryTimedOut ||
|
||||
unreadablePendingCount > 0 ||
|
||||
newPendingCount > 0 ||
|
||||
unverified > 0;
|
||||
|
||||
if (incomplete) {
|
||||
spinner.error('Scan shutdown incomplete');
|
||||
if (combinedResult.containers.kind !== 'stopped') reportContainerFailure(undefined, combinedResult.containers);
|
||||
if (unverified > 0) console.error(`Temporal could not confirm closure for ${unverified} workflow(s).`);
|
||||
if (temporalDiscoveryFailed) console.error('Temporal could not enumerate every running scan workflow.');
|
||||
if (temporalDiscoveryTimedOut) console.error('Temporal workflow discovery did not settle before its deadline.');
|
||||
if (unreadablePendingCount > 0) {
|
||||
console.error(`${unreadablePendingCount} launch record(s) could not be read safely.`);
|
||||
}
|
||||
if (newPendingCount > 0) console.error(`${newPendingCount} scan launch(es) began while shutdown was running.`);
|
||||
if (plan.containersWithoutVerifiedWorkflowId.length > 0) {
|
||||
console.error(
|
||||
`${plan.containersWithoutVerifiedWorkflowId.length} legacy worker(s) could not prove a candidate workflow ID.`,
|
||||
);
|
||||
}
|
||||
console.error(`Retry: ${commandPrefix()} stop --all`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
spinner.stop(`Stopped ${initial.length} scan${initial.length === 1 ? '' : 's'}`);
|
||||
|
||||
if (temporalUp && anyRunningScanWorkflow()) {
|
||||
warn('some scan workflows are still Running in Temporal — check http://localhost:8233');
|
||||
appendVerifiedFallbacks(combinedResult);
|
||||
const clearFailures = clearPendingTargets(pending.references);
|
||||
if (clearFailures > 0) {
|
||||
spinner.error('Scans stopped, but one or more launch records could not be cleared');
|
||||
console.error(`Retry: ${commandPrefix()} stop --all`);
|
||||
process.exit(1);
|
||||
}
|
||||
const stoppedCount = Math.max(containers.length, results.length);
|
||||
spinner.stop(`Stopped ${stoppedCount} scan${stoppedCount === 1 ? '' : 's'}`);
|
||||
}
|
||||
|
||||
/** Resolve the omitted target from Docker without turning a failed query into an empty scan list. */
|
||||
function resolveStopTarget(): string {
|
||||
const result = runningScanContainersChecked();
|
||||
if (result.kind === 'unavailable') {
|
||||
fail('Could not inspect running scan workers.', `Retry with a workspace: ${commandPrefix()} stop <workspace>`);
|
||||
}
|
||||
const running = [...new Set(result.value.flatMap((container) => container.workspace ?? []))];
|
||||
if (running.length === 1) {
|
||||
const workspace = running[0] as string;
|
||||
console.error(`No workspace given; stopping running scan "${workspace}".`);
|
||||
return workspace;
|
||||
}
|
||||
if (running.length > 1) {
|
||||
failUsage('Multiple scans are running: specify which one, or use --all:', ` ${running.join(', ')}`);
|
||||
}
|
||||
if (result.value.length > 0) {
|
||||
fail('A running scan worker has no workspace label.', `Use ${commandPrefix()} stop --all`);
|
||||
}
|
||||
fail('No running scans to stop.', 'Pass a workspace name to stop a specific scan.');
|
||||
}
|
||||
|
||||
export async function stop(opts: StopOptions): Promise<void> {
|
||||
ensureDocker();
|
||||
if (opts.all && opts.workspace) failUsage('Pass a workspace name or --all, not both.');
|
||||
|
||||
// Validate the target: exactly one of <workspace> or --all.
|
||||
if (opts.all && opts.workspace) {
|
||||
failUsage('Pass a workspace name or --all, not both.');
|
||||
}
|
||||
if (!opts.all && !opts.workspace) {
|
||||
failUsage('Specify which scan to stop: `stop <workspace>`, or `stop --all` to stop every scan.');
|
||||
}
|
||||
|
||||
if (opts.workspace) {
|
||||
await stopSingleScan(opts.workspace, opts.yes);
|
||||
} else {
|
||||
await stopAllScans(opts.yes);
|
||||
}
|
||||
const workspace = opts.all ? undefined : (opts.workspace ?? resolveStopTarget());
|
||||
if (workspace) await stopSingleScan(workspace, opts.yes);
|
||||
else await stopAllScans(opts.yes);
|
||||
}
|
||||
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
DEFAULT_MODEL_SPEC,
|
||||
GENERIC_API_KEY_ENV,
|
||||
isCuratedProvider,
|
||||
PROVIDER_API_KEY_ENV,
|
||||
parseModelSpec,
|
||||
} from '../model-spec.js';
|
||||
|
||||
@@ -235,6 +236,30 @@ function validateConfig(config: TOMLConfig): string[] {
|
||||
return errors;
|
||||
}
|
||||
|
||||
function assertNoCredentialConflict(toml: TOMLConfig): void {
|
||||
const tomlBaseUrl = typeof toml.core?.base_url === 'string' ? toml.core.base_url : undefined;
|
||||
if (!tomlBaseUrl || process.env.SHANNON_AI_BASE_URL) return;
|
||||
|
||||
const tomlModel = typeof toml.core?.model === 'string' ? toml.core.model : DEFAULT_MODEL_SPEC;
|
||||
const spec = parseModelSpec(process.env.SHANNON_AI_MODEL ?? tomlModel);
|
||||
if (typeof spec === 'string' || !isCuratedProvider(spec.providerId)) return;
|
||||
|
||||
for (const envVar of PROVIDER_API_KEY_ENV[spec.providerId]) {
|
||||
const mapping = CONFIG_MAP.find((entry) => entry.env === envVar);
|
||||
const tomlHasCredential = mapping ? getTomlValue(toml, mapping) !== undefined : false;
|
||||
const envHasCredential = Boolean(process.env[envVar]);
|
||||
if (!envHasCredential && !tomlHasCredential) continue;
|
||||
|
||||
if (envHasCredential) {
|
||||
fail(
|
||||
`${envVar} in your environment conflicts with the gateway credential in config.toml (core.base_url = ${tomlBaseUrl}).`,
|
||||
`Unset ${envVar}, or set SHANNON_AI_BASE_URL to override both from the environment.`,
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// === Public API ===
|
||||
|
||||
/**
|
||||
@@ -243,7 +268,8 @@ function validateConfig(config: TOMLConfig): string[] {
|
||||
* For each mapped variable: if not already set in the environment,
|
||||
* look it up in ~/.shannon/config.toml and inject it into process.env.
|
||||
* Local mode uses .env exclusively — TOML is skipped.
|
||||
* Exits with an error if the TOML contains unknown or invalid keys.
|
||||
* Exits with an error if the TOML contains unknown or invalid keys, or if an
|
||||
* ambient credential conflicts with a TOML-configured gateway credential.
|
||||
*/
|
||||
export function resolveConfig(): void {
|
||||
if (getMode() === 'local') return;
|
||||
@@ -261,6 +287,8 @@ export function resolveConfig(): void {
|
||||
);
|
||||
}
|
||||
|
||||
assertNoCredentialConflict(toml);
|
||||
|
||||
for (const mapping of CONFIG_MAP) {
|
||||
if (process.env[mapping.env]) continue;
|
||||
|
||||
|
||||
+231
-56
@@ -14,7 +14,7 @@ import { setTimeout as sleep } from 'node:timers/promises';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import type { SpinnerResult } from '@clack/prompts';
|
||||
import { envBool, PI_AUTH_CONTAINER_PATH } from './env.js';
|
||||
import { fail } from './errors.js';
|
||||
import { fail, warn } from './errors.js';
|
||||
import { getMode, isDevMode } from './mode.js';
|
||||
import { INTERNAL_DIR } from './paths.js';
|
||||
import { runStep, spawnCaptured, surfaceOutput } from './ui.js';
|
||||
@@ -27,6 +27,16 @@ const DEV_IMAGE = 'shannon-worker';
|
||||
/** Docker label stamped on each worker container, mapping it back to its workspace so a single scan can be stopped by name. */
|
||||
const WORKSPACE_LABEL = 'shannon.workspace';
|
||||
|
||||
/** Docker label that joins a worker container to the Temporal workflow polling its unique task queue. */
|
||||
const TASK_QUEUE_LABEL = 'shannon.task-queue';
|
||||
|
||||
/** Docker label carrying the workflow ID selected before the worker starts. */
|
||||
const WORKFLOW_ID_LABEL = 'shannon.workflow-id';
|
||||
|
||||
/** Image/container protocol proving that the worker honors the preselected workflow ID. */
|
||||
const WORKER_PROTOCOL_LABEL = 'shannon.worker-protocol';
|
||||
export const WORKFLOW_ID_PROTOCOL = 'workflow-id-v1';
|
||||
|
||||
export function getWorkerImage(version: string): string {
|
||||
return getMode() === 'local' ? DEV_IMAGE : `${NPX_IMAGE_REPO}:${version}`;
|
||||
}
|
||||
@@ -84,9 +94,6 @@ function spawnQuiet(cmd: string, args: string[]): Promise<boolean> {
|
||||
const TEMPORAL_CONTAINER = 'shannon-temporal';
|
||||
const TEMPORAL_ADDRESS = 'localhost:7233';
|
||||
|
||||
/** Query matching every running pentest scan workflow. */
|
||||
const RUNNING_SCAN_QUERY = "ExecutionStatus = 'Running' AND WorkflowType = 'pentestPipelineWorkflow'";
|
||||
|
||||
/** Build `docker exec` args for a `temporal` CLI command run inside the Temporal container. */
|
||||
function temporalCmd(...args: string[]): string[] {
|
||||
return ['exec', TEMPORAL_CONTAINER, 'temporal', ...args, '--address', TEMPORAL_ADDRESS];
|
||||
@@ -116,10 +123,8 @@ export function isTemporalReady(): boolean {
|
||||
return output.includes('SERVING');
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure Temporal is running via compose.
|
||||
*/
|
||||
export async function ensureInfra(spinner: SpinnerResult): Promise<void> {
|
||||
/** Start (or find) Temporal via compose and wait until it serves; exits the process on failure. */
|
||||
async function ensureTemporalHealthy(spinner: SpinnerResult): Promise<void> {
|
||||
if (isTemporalReady()) {
|
||||
return;
|
||||
}
|
||||
@@ -146,6 +151,97 @@ export async function ensureInfra(spinner: SpinnerResult): Promise<void> {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const DEFAULT_RETENTION_HOURS = 168;
|
||||
const RETENTION_ENV = 'SHANNON_TEMPORAL_RETENTION';
|
||||
const RETENTION_NAMESPACE = 'default';
|
||||
|
||||
/**
|
||||
* Desired retention in whole hours: unset or empty env → 168 (7 days); a positive
|
||||
* whole-hour override like `72h`; anything else warns and returns null (leave unchanged).
|
||||
*/
|
||||
function desiredRetentionHours(): number | null {
|
||||
const raw = process.env[RETENTION_ENV];
|
||||
if (raw === undefined || raw.trim() === '') {
|
||||
return DEFAULT_RETENTION_HOURS;
|
||||
}
|
||||
const match = raw.trim().match(/^([1-9][0-9]*)h$/);
|
||||
if (!match) {
|
||||
warn(
|
||||
`Ignoring invalid ${RETENTION_ENV} "${raw}" — Temporal retention left unchanged.`,
|
||||
'Use a positive whole number of hours, e.g. "168h".',
|
||||
);
|
||||
return null;
|
||||
}
|
||||
return Number(match[1]);
|
||||
}
|
||||
|
||||
/** Convert a Go duration such as "24h0m0s" or "168h" to whole seconds, or null when it doesn't parse. */
|
||||
function parseGoDurationSeconds(text: string): number | null {
|
||||
const match = text.match(/^(?:(\d+)h)?(?:(\d+)m)?(?:(\d+)s)?$/);
|
||||
if (!match || (match[1] === undefined && match[2] === undefined && match[3] === undefined)) {
|
||||
return null;
|
||||
}
|
||||
const hours = Number(match[1] ?? 0);
|
||||
const minutes = Number(match[2] ?? 0);
|
||||
const seconds = Number(match[3] ?? 0);
|
||||
return hours * 3600 + minutes * 60 + seconds;
|
||||
}
|
||||
|
||||
/**
|
||||
* Current retention of the `default` namespace in seconds, or null when it can't be read.
|
||||
* `runOutput` returns '' on a failed describe, so a failed read and an unparseable one both
|
||||
* collapse to null — either way the live value is unknown, which the caller handles the same way.
|
||||
*/
|
||||
function readCurrentRetentionSeconds(): number | null {
|
||||
const output = runOutput('docker', temporalCmd('operator', 'namespace', 'describe', RETENTION_NAMESPACE));
|
||||
const match = output.match(/WorkflowExecutionRetentionTtl\s+(\S+)/);
|
||||
if (!match || match[1] === undefined) {
|
||||
return null;
|
||||
}
|
||||
return parseGoDurationSeconds(match[1]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Converge the `default` namespace's retention to the CLI-owned value after Temporal is
|
||||
* healthy. The CLI is the authority: a manual change is replaced on the next start unless
|
||||
* the operator sets the matching override. A describe or update failure warns once that the
|
||||
* requested value wasn't applied and never blocks the scan.
|
||||
*/
|
||||
function convergeNamespaceRetention(): void {
|
||||
const hours = desiredRetentionHours();
|
||||
if (hours === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
const currentSeconds = readCurrentRetentionSeconds();
|
||||
if (currentSeconds === null) {
|
||||
warn(
|
||||
`Could not read Temporal retention for namespace "${RETENTION_NAMESPACE}" — the requested value (${hours}h) was not applied.`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
if (currentSeconds === hours * 3600) {
|
||||
return;
|
||||
}
|
||||
|
||||
const updated = runQuiet(
|
||||
'docker',
|
||||
temporalCmd('operator', 'namespace', 'update', '--namespace', RETENTION_NAMESPACE, '--retention', `${hours}h`),
|
||||
);
|
||||
if (!updated) {
|
||||
warn(`Could not update Temporal retention to ${hours}h — the requested value was not applied.`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure Temporal is running via compose, then converge its scan-history retention.
|
||||
*/
|
||||
export async function ensureInfra(spinner: SpinnerResult): Promise<void> {
|
||||
await ensureTemporalHealthy(spinner);
|
||||
convergeNamespaceRetention();
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the worker image from the repository, tagged with the name this mode
|
||||
* resolves at run time.
|
||||
@@ -167,7 +263,10 @@ export function buildImage(noCache: boolean, version: string): void {
|
||||
export function ensureImage(version: string): void {
|
||||
const image = getWorkerImage(version);
|
||||
const exists = runQuiet('docker', ['image', 'inspect', image]);
|
||||
if (exists) return;
|
||||
if (exists) {
|
||||
ensureWorkerImageProtocol(image);
|
||||
return;
|
||||
}
|
||||
|
||||
if (canBuildImage()) {
|
||||
console.log('Shannon image not found, building...');
|
||||
@@ -185,6 +284,22 @@ export function ensureImage(version: string): void {
|
||||
}
|
||||
pruneOldImages(version);
|
||||
}
|
||||
ensureWorkerImageProtocol(image);
|
||||
}
|
||||
|
||||
/** Refuse a stale worker image that would ignore the CLI-selected workflow ID. */
|
||||
function ensureWorkerImageProtocol(image: string): void {
|
||||
const protocol = runOutput('docker', [
|
||||
'image',
|
||||
'inspect',
|
||||
image,
|
||||
'--format',
|
||||
`{{ index .Config.Labels "${WORKER_PROTOCOL_LABEL}" }}`,
|
||||
]);
|
||||
if (protocol === WORKFLOW_ID_PROTOCOL) return;
|
||||
|
||||
const hint = canBuildImage() ? 'Run ./shannon build, then retry.' : 'Reinstall this Shannon version, then retry.';
|
||||
fail('The Shannon worker image is incompatible with this CLI.', hint);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -288,6 +403,7 @@ export interface WorkerOptions {
|
||||
repo: { hostPath: string; containerPath: string };
|
||||
workspacesDir: string;
|
||||
taskQueue: string;
|
||||
workflowId: string;
|
||||
containerName: string;
|
||||
envFlags: string[];
|
||||
config?: { hostPath: string; containerPath: string };
|
||||
@@ -310,8 +426,16 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess {
|
||||
}
|
||||
args.push('--name', opts.containerName, '--network', 'shannon-net');
|
||||
|
||||
// Tag with the workspace so `stop <workspace>` can target this scan's container
|
||||
args.push('--label', `${WORKSPACE_LABEL}=${opts.workspace}`);
|
||||
// Keep the launch identity on the container before session.json exists. The fixed workflow
|
||||
// ID lets stop verify the pre-registration window without trusting visibility timing.
|
||||
args.push(
|
||||
'--label',
|
||||
`${WORKSPACE_LABEL}=${opts.workspace}`,
|
||||
'--label',
|
||||
`${TASK_QUEUE_LABEL}=${opts.taskQueue}`,
|
||||
'--label',
|
||||
`${WORKFLOW_ID_LABEL}=${opts.workflowId}`,
|
||||
);
|
||||
|
||||
// Add host flag for Linux
|
||||
args.push(...addHostFlag());
|
||||
@@ -345,7 +469,7 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess {
|
||||
args.push('-v', `${opts.config.hostPath}:${opts.config.containerPath}:ro`);
|
||||
}
|
||||
|
||||
// Output directory for deliverables copy
|
||||
// Customer-copy destination. The workflow surfaces only final report artifacts here.
|
||||
if (opts.outputDir) {
|
||||
args.push('-v', `${opts.outputDir}:/app/output`);
|
||||
}
|
||||
@@ -358,7 +482,10 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess {
|
||||
// Environment
|
||||
args.push(...opts.envFlags);
|
||||
|
||||
// Container settings
|
||||
// Container settings. Chromium's own sandbox needs syscalls Docker's default seccomp
|
||||
// profile blocks, which is why the profile is dropped. `seccomp=unconfined` is a
|
||||
// container-wide setting, not a per-process one: every process here runs unfiltered,
|
||||
// the worker included — not just the browser automation that motivates it.
|
||||
args.push('--shm-size', '2gb', '--security-opt', 'seccomp=unconfined');
|
||||
|
||||
// Image
|
||||
@@ -367,6 +494,7 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess {
|
||||
// Worker command
|
||||
args.push('node', 'apps/worker/dist/temporal/worker.js', opts.url, opts.repo.containerPath);
|
||||
args.push('--task-queue', opts.taskQueue);
|
||||
args.push('--workflow-id', opts.workflowId);
|
||||
if (opts.config) {
|
||||
args.push('--config', opts.config.containerPath);
|
||||
}
|
||||
@@ -390,6 +518,18 @@ export function spawnWorker(opts: WorkerOptions): ChildProcess {
|
||||
/** `docker ps --filter` args matching every running worker container. */
|
||||
export const WORKER_FILTER: readonly string[] = ['--filter', 'name=shannon-worker-'];
|
||||
|
||||
/** Result of a command-backed query whose unavailable state must not be mistaken for an empty result. */
|
||||
export type CommandQueryResult<T> = { kind: 'ok'; value: T } | { kind: 'unavailable' };
|
||||
|
||||
/** Identity carried by a running scan worker container. Older workers may lack the newer labels. */
|
||||
export interface RunningScanContainer {
|
||||
readonly id: string;
|
||||
readonly workspace?: string;
|
||||
readonly taskQueue?: string;
|
||||
readonly workflowId?: string;
|
||||
readonly workerProtocol?: string;
|
||||
}
|
||||
|
||||
/** `docker ps --filter` args matching one scan's worker container(s), by workspace label. */
|
||||
export function scanFilter(workspace: string): readonly string[] {
|
||||
return ['--filter', `label=${WORKSPACE_LABEL}=${workspace}`];
|
||||
@@ -400,9 +540,85 @@ export function scanFilter(workspace: string): readonly string[] {
|
||||
* the authoritative check for whether containers actually stopped — `docker stop`'s
|
||||
* exit code can't distinguish "already gone" from "failed to stop".
|
||||
*/
|
||||
export function runningContainersChecked(filter: readonly string[]): CommandQueryResult<string[]> {
|
||||
try {
|
||||
const output = execFileSync('docker', ['ps', '-q', ...filter], { stdio: 'pipe', encoding: 'utf-8' }).trim();
|
||||
return { kind: 'ok', value: output.split('\n').filter(Boolean) };
|
||||
} catch {
|
||||
return { kind: 'unavailable' };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Best-effort counterpart for callers where Docker unavailability is intentionally
|
||||
* presented as no local running containers.
|
||||
*/
|
||||
export function runningContainers(filter: readonly string[]): string[] {
|
||||
const output = runOutput('docker', ['ps', '-q', ...filter]);
|
||||
return output.split('\n').filter(Boolean);
|
||||
const result = runningContainersChecked(filter);
|
||||
return result.kind === 'ok' ? result.value : [];
|
||||
}
|
||||
|
||||
function normalizedLabel(value: string | undefined): string | undefined {
|
||||
const normalized = value?.trim();
|
||||
return normalized && normalized !== '<no value>' ? normalized : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Running scan containers with the labels needed to correlate a worker to its Temporal
|
||||
* workflow. A successful query keeps unlabeled legacy workers in the result by ID.
|
||||
*/
|
||||
export function runningScanContainersChecked(
|
||||
filter: readonly string[] = WORKER_FILTER,
|
||||
): CommandQueryResult<RunningScanContainer[]> {
|
||||
try {
|
||||
const format = `{{.ID}}\t{{ index .Labels "${WORKSPACE_LABEL}" }}\t{{ index .Labels "${TASK_QUEUE_LABEL}" }}\t{{ index .Labels "${WORKFLOW_ID_LABEL}" }}\t{{ index .Labels "${WORKER_PROTOCOL_LABEL}" }}`;
|
||||
const output = execFileSync('docker', ['ps', ...filter, '--format', format], {
|
||||
stdio: 'pipe',
|
||||
encoding: 'utf-8',
|
||||
}).trim();
|
||||
if (!output) return { kind: 'ok', value: [] };
|
||||
|
||||
const containers: RunningScanContainer[] = [];
|
||||
for (const line of output.split('\n')) {
|
||||
const [rawId, rawWorkspace, rawTaskQueue, rawWorkflowId, rawWorkerProtocol] = line.split('\t');
|
||||
const id = rawId?.trim();
|
||||
if (!id) return { kind: 'unavailable' };
|
||||
const workspace = normalizedLabel(rawWorkspace);
|
||||
const taskQueue = normalizedLabel(rawTaskQueue);
|
||||
const workflowId = normalizedLabel(rawWorkflowId);
|
||||
const workerProtocol = normalizedLabel(rawWorkerProtocol);
|
||||
containers.push({
|
||||
id,
|
||||
...(workspace !== undefined && { workspace }),
|
||||
...(taskQueue !== undefined && { taskQueue }),
|
||||
...(workflowId !== undefined && { workflowId }),
|
||||
...(workerProtocol !== undefined && { workerProtocol }),
|
||||
});
|
||||
}
|
||||
return { kind: 'ok', value: containers };
|
||||
} catch {
|
||||
return { kind: 'unavailable' };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Workspace names of every running worker container, read from the shannon.workspace
|
||||
* label each scan is stamped with at spawn. The checked form preserves Docker query
|
||||
* failures so lifecycle commands do not mistake an unavailable daemon for an empty list.
|
||||
*/
|
||||
export function runningScanWorkspacesChecked(): CommandQueryResult<string[]> {
|
||||
const result = runningScanContainersChecked();
|
||||
if (result.kind === 'unavailable') return result;
|
||||
return {
|
||||
kind: 'ok',
|
||||
value: result.value.flatMap((container) => (container.workspace === undefined ? [] : [container.workspace])),
|
||||
};
|
||||
}
|
||||
|
||||
/** Best-effort counterpart for callers that only need the local scan list. */
|
||||
export function runningScanWorkspaces(): string[] {
|
||||
const result = runningScanWorkspacesChecked();
|
||||
return result.kind === 'ok' ? result.value : [];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -414,47 +630,6 @@ export async function stopContainers(ids: string[]): Promise<void> {
|
||||
await Promise.all(ids.map((id) => spawnQuiet('docker', ['stop', id])));
|
||||
}
|
||||
|
||||
/**
|
||||
* Terminate a Temporal workflow so a stopped scan doesn't linger as a running
|
||||
* workflow with no worker. Best-effort: returns false if Temporal is unreachable
|
||||
* or the workflow already closed. Requires Temporal to be up (guard with isTemporalReady).
|
||||
*/
|
||||
export function terminateWorkflow(workflowId: string, reason: string): boolean {
|
||||
return runQuiet('docker', temporalCmd('workflow', 'terminate', '--workflow-id', workflowId, '--reason', reason));
|
||||
}
|
||||
|
||||
/**
|
||||
* Terminate every running pentest workflow in one batch, so `stop --all` doesn't
|
||||
* leave workflows running with no worker. Best-effort: returns false if Temporal
|
||||
* is unreachable. Requires Temporal to be up (guard with isTemporalReady).
|
||||
*/
|
||||
export function terminateAllWorkflows(reason: string): boolean {
|
||||
return runQuiet(
|
||||
'docker',
|
||||
temporalCmd('workflow', 'terminate', '--query', RUNNING_SCAN_QUERY, '--reason', reason, '--yes'),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a specific workflow is still in the Running state. Re-querying this after
|
||||
* a terminate verifies it actually took effect, rather than trusting the terminate
|
||||
* command's exit code. Requires Temporal to be up (guard with isTemporalReady).
|
||||
*/
|
||||
export function isWorkflowRunning(workflowId: string): boolean {
|
||||
const query = `WorkflowId = '${workflowId}' AND ExecutionStatus = 'Running'`;
|
||||
const output = runOutput('docker', temporalCmd('workflow', 'list', '--query', query));
|
||||
return output.includes(workflowId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether any pentest scan workflow is still Running — the `stop --all` counterpart
|
||||
* to isWorkflowRunning. Requires Temporal to be up (guard with isTemporalReady).
|
||||
*/
|
||||
export function anyRunningScanWorkflow(): boolean {
|
||||
const output = runOutput('docker', temporalCmd('workflow', 'list', '--query', RUNNING_SCAN_QUERY));
|
||||
return output.includes('pentestPipelineWorkflow');
|
||||
}
|
||||
|
||||
/**
|
||||
* Tear down the compose stack. When `clean` is set, volumes are removed too.
|
||||
*/
|
||||
|
||||
+48
-11
@@ -31,6 +31,10 @@ const COMMON_FORWARD_VARS = [
|
||||
'SHANNON_AI_MODEL',
|
||||
'SHANNON_AI_BASE_URL',
|
||||
'SHANNON_AI_OPENAI_FORMAT',
|
||||
// Opt-in debug flag: when set, the worker persists a bounded, sanitized snippet of a failed
|
||||
// provider turn's raw error message to error.log. Off by default; provider prose stays out of
|
||||
// durable state unless an operator deliberately enables it for a diagnosis.
|
||||
'SHANNON_DEBUG_PROVIDER_ERRORS',
|
||||
GENERIC_API_KEY_ENV,
|
||||
] as const;
|
||||
|
||||
@@ -111,6 +115,17 @@ interface CredentialValidation {
|
||||
error?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the shell environment already carries a usable credential — the host's
|
||||
* pi login, or an API key for the selected provider. Reads process.env only.
|
||||
*/
|
||||
export function hasExportedCredentials(): boolean {
|
||||
if (shouldUsePiAuth()) return true;
|
||||
const spec = resolveModelSpec();
|
||||
if (typeof spec === 'string') return false;
|
||||
return hasCredential(spec.providerId);
|
||||
}
|
||||
|
||||
/** Whether a curated provider has its own named credential set (API key plus any extra var). */
|
||||
function hasNamedCredential(providerId: CuratedProviderId): boolean {
|
||||
const apiKeys = PROVIDER_API_KEY_ENV[providerId];
|
||||
@@ -130,6 +145,38 @@ function configuredProviders(): CuratedProviderId[] {
|
||||
return CURATED_PROVIDERS.filter((providerId) => hasNamedCredential(providerId));
|
||||
}
|
||||
|
||||
/** Whether SHANNON_AI_MODEL was set by the user, rather than falling back to the default. */
|
||||
function modelExplicitlySelected(): boolean {
|
||||
return Boolean(process.env.SHANNON_AI_MODEL?.trim());
|
||||
}
|
||||
|
||||
/**
|
||||
* Explain why the selected provider has no usable credential. With no model chosen
|
||||
* the provider is only the default (anthropic), so the real state is "nothing
|
||||
* configured" — or, if another provider's key is set, an unselected model.
|
||||
*/
|
||||
function describeMissingCredential(providerId: string): string {
|
||||
if (modelExplicitlySelected()) {
|
||||
const requirement = isCuratedProvider(providerId) ? PROVIDER_CREDENTIAL_HINT[providerId] : GENERIC_API_KEY_ENV;
|
||||
const hint =
|
||||
getMode() === 'local'
|
||||
? `Set ${requirement} in .env or export it.`
|
||||
: `Export the variables or run 'npx @keygraph/shannon setup'.`;
|
||||
return `No credentials found for provider "${providerId}". ${hint}`;
|
||||
}
|
||||
|
||||
const [provider] = configuredProviders();
|
||||
if (provider) {
|
||||
return `A credential for "${provider}" is set, but no model is selected. Set SHANNON_AI_MODEL=${provider}:<model-id> to use it.`;
|
||||
}
|
||||
|
||||
const hint =
|
||||
getMode() === 'local'
|
||||
? 'Set a provider API key in .env (for example ANTHROPIC_API_KEY).'
|
||||
: "Run 'npx @keygraph/shannon setup' to get started.";
|
||||
return `No credentials configured. ${hint}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate that the model selection parses and its provider has a credential.
|
||||
* Runs before any Docker work so mistakes fail immediately.
|
||||
@@ -155,17 +202,7 @@ export function validateCredentials(): CredentialValidation {
|
||||
|
||||
// 2. The selected provider must have a credential
|
||||
if (!hasCredential(spec.providerId)) {
|
||||
const requirement = isCuratedProvider(spec.providerId)
|
||||
? PROVIDER_CREDENTIAL_HINT[spec.providerId]
|
||||
: GENERIC_API_KEY_ENV;
|
||||
const hint =
|
||||
getMode() === 'local'
|
||||
? `Set ${requirement} in .env or export it.`
|
||||
: `Export the variables or run 'npx @keygraph/shannon setup'.`;
|
||||
return {
|
||||
valid: false,
|
||||
error: `No credentials found for provider "${spec.providerId}". ${hint}`,
|
||||
};
|
||||
return { valid: false, error: describeMissingCredential(spec.providerId) };
|
||||
}
|
||||
|
||||
// 3. Exactly one provider may be configured. Several complete credentials make
|
||||
|
||||
+76
-37
@@ -1,37 +1,94 @@
|
||||
/**
|
||||
* Centralized error reporting.
|
||||
*
|
||||
* `fail` — an expected, user-fixable error (bad input, missing prerequisite):
|
||||
* a clean message on stderr and a non-zero exit, never a stack trace.
|
||||
* `fail` / `failWith` — an expected, user-fixable error (bad input, missing
|
||||
* prerequisite): a clean message on stderr and a non-zero exit, never a stack trace.
|
||||
* `failUsage` — a malformed invocation (unknown command, bad or missing
|
||||
* arguments): the same clean message, but a distinct exit code so callers can
|
||||
* tell a usage mistake from an operational failure.
|
||||
* `crash` — an unexpected error (a bug): a brief message, the full stack written
|
||||
* to a log file for a bug report, and a pointer to the issue tracker.
|
||||
* `crash` — an unexpected error (a bug): a fixed code and a pointer to the issue tracker.
|
||||
*
|
||||
* JSON mode (enabled once, before parsing, for the `--json` command surface) replaces
|
||||
* the text lines with one compact envelope on stderr — stdout stays empty — while the
|
||||
* exit-code split is unchanged. Call sites on a JSON-capable path must exit through
|
||||
* `failWith`/`failUsage`/`crash` (never a bare `fail` or `warn`) so every failure
|
||||
* carries a stable code and stderr stays parseable.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
|
||||
const ISSUES_URL = 'https://github.com/KeygraphHQ/shannon/issues';
|
||||
|
||||
/** Report an expected, user-fixable error (with optional extra lines) and exit non-zero. */
|
||||
export function fail(message: string, ...hints: string[]): never {
|
||||
const UNEXPECTED_MESSAGE = 'Shannon encountered an unexpected failure. Reference code: SHANNON_UNEXPECTED_ERROR';
|
||||
const REPORT_HINT = `If this looks like a bug, please report it: ${ISSUES_URL}`;
|
||||
|
||||
/** Stable machine-readable failure codes for the JSON error envelope. */
|
||||
export type ErrorCode =
|
||||
| 'CLI_USAGE'
|
||||
| 'CLI_SCAN_NOT_FOUND'
|
||||
| 'CLI_SCAN_IDENTITY_NOT_FOUND'
|
||||
| 'CLI_SCAN_IDENTITY_AMBIGUOUS'
|
||||
| 'CLI_SCAN_STATUS_UNAVAILABLE'
|
||||
| 'CLI_SCAN_SCHEMA_UNSUPPORTED'
|
||||
| 'CLI_PRECONDITION_FAILED'
|
||||
| 'CLI_INTERNAL_ERROR';
|
||||
|
||||
let jsonMode = false;
|
||||
|
||||
/** Switch failure reporting to the JSON envelope. Set once, before any guard, parse, or dispatch. */
|
||||
export function enableJsonErrors(): void {
|
||||
jsonMode = true;
|
||||
}
|
||||
|
||||
/** Whether failures are reported as the JSON envelope rather than text. */
|
||||
export function jsonErrorsEnabled(): boolean {
|
||||
return jsonMode;
|
||||
}
|
||||
|
||||
/** Fixed unexpected-failure projection shared by the runtime and focused safety tests. */
|
||||
export function unexpectedFailureLines(): readonly string[] {
|
||||
return [`ERROR: ${UNEXPECTED_MESSAGE}`, REPORT_HINT];
|
||||
}
|
||||
|
||||
/**
|
||||
* Report a failure on stderr and exit. Text mode prints the message and every hint
|
||||
* verbatim; JSON mode writes one compact envelope (dropping the empty strings used
|
||||
* to space text output) synchronously so `process.exit` cannot truncate it.
|
||||
*/
|
||||
function emit(exitCode: 1 | 2, code: ErrorCode, message: string, hints: readonly string[]): never {
|
||||
if (jsonMode) {
|
||||
const payload = JSON.stringify({ error: { code, message, hints: hints.filter((hint) => hint.trim() !== '') } });
|
||||
fs.writeSync(process.stderr.fd, `${payload}\n`);
|
||||
process.exit(exitCode);
|
||||
}
|
||||
console.error(`ERROR: ${message}`);
|
||||
for (const hint of hints) {
|
||||
console.error(hint);
|
||||
}
|
||||
process.exit(1);
|
||||
process.exit(exitCode);
|
||||
}
|
||||
|
||||
/**
|
||||
* Report an expected, user-fixable error (with optional extra lines) and exit non-zero.
|
||||
* Text-only paths use this; a JSON-capable path must use `failWith` so the envelope
|
||||
* carries a real code — if a bare `fail` is ever reached in JSON mode, the fixed
|
||||
* internal-error envelope is emitted instead of guessing a code for the message.
|
||||
*/
|
||||
export function fail(message: string, ...hints: string[]): never {
|
||||
if (jsonMode) {
|
||||
emit(1, 'CLI_INTERNAL_ERROR', UNEXPECTED_MESSAGE, [REPORT_HINT]);
|
||||
}
|
||||
emit(1, 'CLI_INTERNAL_ERROR', message, hints);
|
||||
}
|
||||
|
||||
/** Report an expected operational failure under a stable code and exit 1. */
|
||||
export function failWith(code: ErrorCode, message: string, ...hints: string[]): never {
|
||||
emit(1, code, message, hints);
|
||||
}
|
||||
|
||||
/** Report a usage/argument error (with optional extra lines) and exit 2. */
|
||||
export function failUsage(message: string, ...hints: string[]): never {
|
||||
console.error(`ERROR: ${message}`);
|
||||
for (const hint of hints) {
|
||||
console.error(hint);
|
||||
}
|
||||
process.exit(2);
|
||||
emit(2, 'CLI_USAGE', message, hints);
|
||||
}
|
||||
|
||||
/** Report a non-fatal warning on stderr (with optional extra lines) without exiting. */
|
||||
@@ -42,29 +99,11 @@ export function warn(message: string, ...hints: string[]): void {
|
||||
}
|
||||
}
|
||||
|
||||
/** Report an unexpected error: brief message, full stack to a log file, plus the issue link. */
|
||||
export function crash(error: unknown): never {
|
||||
console.error(`ERROR: ${error instanceof Error ? error.message : String(error)}`);
|
||||
if (process.env.DEBUG) {
|
||||
console.error(error instanceof Error ? error.stack : String(error));
|
||||
/** Report an unexpected error without projecting its message, stack, or attached values. */
|
||||
export function crash(_error: unknown): never {
|
||||
if (jsonMode) {
|
||||
emit(1, 'CLI_INTERNAL_ERROR', UNEXPECTED_MESSAGE, [REPORT_HINT]);
|
||||
}
|
||||
|
||||
const logPath = writeCrashLog(error);
|
||||
if (logPath) {
|
||||
console.error(`Details written to ${logPath}`);
|
||||
}
|
||||
console.error(`If this looks like a bug, please report it: ${ISSUES_URL}`);
|
||||
for (const line of unexpectedFailureLines()) console.error(line);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
/** Write the full error and stack to a log file; return its path, or null if it can't be written. */
|
||||
function writeCrashLog(error: unknown): string | null {
|
||||
try {
|
||||
const logPath = path.join(os.tmpdir(), 'shannon-error.log');
|
||||
const detail = error instanceof Error && error.stack ? error.stack : String(error);
|
||||
fs.writeFileSync(logPath, `${new Date().toISOString()}\n${detail}\n`);
|
||||
return logPath;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
+21
-10
@@ -47,30 +47,32 @@ const COMMAND_HELP: Readonly<Record<string, CommandHelp>> = {
|
||||
],
|
||||
},
|
||||
stop: {
|
||||
usage: ['stop <workspace> [--yes]', 'stop --all [--yes]'],
|
||||
description: 'Stop one scan by workspace, or every scan with --all (Temporal stays up).',
|
||||
usage: ['stop [<workspace>] [--yes]', 'stop --all [--yes]'],
|
||||
description:
|
||||
'Stop one scan by workspace, or every scan with --all (Temporal stays up). With no workspace, stops the single running scan; when several are running, name one or use --all.',
|
||||
options: [['--all', 'Stop all running scans'], YES_OPTION],
|
||||
examples: ['stop q1-audit', 'stop --all'],
|
||||
examples: ['stop', 'stop q1-audit', 'stop --all'],
|
||||
},
|
||||
reset: {
|
||||
usage: ['reset'],
|
||||
description: 'Stop everything and permanently remove all Temporal data and volumes.',
|
||||
},
|
||||
logs: {
|
||||
usage: ['logs <workspace>'],
|
||||
description: "Tail a scan's live log until it completes.",
|
||||
examples: ['logs q1-audit'],
|
||||
usage: ['logs [<workspace>]'],
|
||||
description:
|
||||
"Tail a scan's live log until it completes. With no workspace, follows the single running scan, or the most recent workspace when none is running; when several are running, name one.",
|
||||
examples: ['logs', 'logs q1-audit'],
|
||||
},
|
||||
status: {
|
||||
usage: ['status <workspace> [--json]'],
|
||||
usage: ['status [<workspace>] [--json]'],
|
||||
description:
|
||||
"Show one scan's phase-by-phase progress, read live from Temporal. Watches and redraws until the scan finishes on a terminal; prints one frame when piped or already finished. With --json, prints a single machine-readable snapshot and exits.",
|
||||
"Show one scan's phase-by-phase progress, read live from Temporal. With no workspace, shows the single running scan, or the most recent workspace when none is running; when several are running, name one. Watches and redraws until the scan finishes on a terminal; prints one frame when piped or already finished. With --json, prints a single machine-readable snapshot and exits.",
|
||||
options: [['--json', 'Output a point-in-time snapshot as JSON, then exit']],
|
||||
examples: ['status q1-audit', 'status q1-audit --json'],
|
||||
examples: ['status', 'status q1-audit', 'status q1-audit --json'],
|
||||
},
|
||||
scans: {
|
||||
usage: ['scans [--json]'],
|
||||
description: 'List completed scans and where each report lives.',
|
||||
description: 'List running and completed scans, and where each finished report lives.',
|
||||
options: [['--json', 'Output the scan list as JSON']],
|
||||
examples: ['scans', 'scans --json'],
|
||||
},
|
||||
@@ -102,6 +104,15 @@ export function isHelpableCommand(command: string): boolean {
|
||||
return command in COMMAND_HELP;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every explicit help topic, mode-blind, with `help` itself as the known global topic.
|
||||
* Topic lookup is deliberately not mode-filtered (unlike `availableCommands`) so
|
||||
* cross-mode help such as local `help setup` and npx `help build` keeps working.
|
||||
*/
|
||||
export function helpTopics(): readonly string[] {
|
||||
return [...Object.keys(COMMAND_HELP), 'help'];
|
||||
}
|
||||
|
||||
/**
|
||||
* User-facing command names available in the current mode, for "did you mean?"
|
||||
* suggestions. Derived from the same table that backs per-command help, so the
|
||||
|
||||
@@ -16,6 +16,11 @@ export function getConfigFile(): string {
|
||||
return path.join(SHANNON_HOME, 'config.toml');
|
||||
}
|
||||
|
||||
/** Whether the npx-mode credential file (`~/.shannon/config.toml`) exists on disk. */
|
||||
export function configFileExists(): boolean {
|
||||
return fs.existsSync(getConfigFile());
|
||||
}
|
||||
|
||||
export function getWorkspacesDir(): string {
|
||||
return getMode() === 'local' ? path.resolve('workspaces') : path.join(SHANNON_HOME, 'workspaces');
|
||||
}
|
||||
|
||||
+148
-21
@@ -18,14 +18,23 @@ import { setup } from './commands/setup.js';
|
||||
import { start } from './commands/start.js';
|
||||
import { status } from './commands/status.js';
|
||||
import { stop } from './commands/stop.js';
|
||||
import { crash, fail, failUsage } from './errors.js';
|
||||
import { availableCommands, isHelpableCommand, printCommandHelp, START_OPTIONS } from './help.js';
|
||||
import { hasExportedCredentials } from './env.js';
|
||||
import { crash, enableJsonErrors, fail, failUsage, failWith, jsonErrorsEnabled } from './errors.js';
|
||||
import { availableCommands, helpTopics, isHelpableCommand, printCommandHelp, START_OPTIONS } from './help.js';
|
||||
import { configFileExists } from './home.js';
|
||||
import { commandPrefix, getMode, isLocal, type Mode } from './mode.js';
|
||||
import { displaySplash } from './splash.js';
|
||||
import { closestMatch } from './suggest.js';
|
||||
import { stdoutIsTerminal } from './tty.js';
|
||||
import { getVersion, getVersionLine } from './version.js';
|
||||
import { resolveDefaultWorkspace } from './workspaces.js';
|
||||
|
||||
/**
|
||||
* Refuse to run as root or under sudo. The worker container's Linux UID remapping
|
||||
* (docker.ts) stamps bind-mounted files with the invoking user's real uid/gid; under
|
||||
* sudo that uid is 0, so the repo, workspace, and report files would come back
|
||||
* owned by root instead of the person who ran the scan.
|
||||
*/
|
||||
function blockSudo(): void {
|
||||
const isSudo = !!process.env.SUDO_USER;
|
||||
const isRoot = process.geteuid?.() === 0;
|
||||
@@ -37,15 +46,38 @@ function blockSudo(): void {
|
||||
: [];
|
||||
|
||||
if (isSudo) {
|
||||
fail('Shannon must not be run with sudo.', 'Re-run this command as your normal user.', ...linuxHints);
|
||||
failWith(
|
||||
'CLI_PRECONDITION_FAILED',
|
||||
'Shannon must not be run with sudo.',
|
||||
'Re-run this command as your normal user.',
|
||||
...linuxHints,
|
||||
);
|
||||
}
|
||||
fail(
|
||||
failWith(
|
||||
'CLI_PRECONDITION_FAILED',
|
||||
'Shannon must not be run as the root user.',
|
||||
'Switch to a regular user account and re-run this command.',
|
||||
...linuxHints,
|
||||
);
|
||||
}
|
||||
|
||||
/** Commands whose `--json` output contract extends to failures. */
|
||||
const JSON_CAPABLE_COMMANDS = new Set(['status', 'scans', 'version', '--version', '-v']);
|
||||
|
||||
/**
|
||||
* Raw-argv sniff for the JSON error latch, decided before any guard or parse so even
|
||||
* a pre-dispatch failure honors it. Latches on `--json` or a malformed `--json=<value>`
|
||||
* (which still fails as a parse error — inside the envelope). Any other command that
|
||||
* receives `--json` keeps its normal unknown-option behavior.
|
||||
*/
|
||||
function wantsJsonErrors(argv: readonly string[]): boolean {
|
||||
const command = argv[0];
|
||||
if (command === undefined || !JSON_CAPABLE_COMMANDS.has(command)) {
|
||||
return false;
|
||||
}
|
||||
return argv.slice(1).some((arg) => arg === '--json' || arg.startsWith('--json='));
|
||||
}
|
||||
|
||||
/** Render `start`'s flags for the global help, from the same source as `start --help`. */
|
||||
function renderStartOptions(): string {
|
||||
const flagWidth = Math.max(...START_OPTIONS.map(([flag]) => flag.length));
|
||||
@@ -60,12 +92,16 @@ function renderUsage(prefix: string, mode: Mode): string {
|
||||
const rows: ReadonlyArray<readonly [string, string]> = [
|
||||
...(mode === 'local' ? [] : [[`${prefix} setup`, 'Configure credentials'] as const]),
|
||||
[`${prefix} start --url <url> --repo <path> [options]`, 'Start a pentest scan'],
|
||||
[`${prefix} stop <workspace> [--yes]`, 'Stop one scan'],
|
||||
[`${prefix} stop [<workspace>] [--yes]`, 'Stop one scan (default: the single running scan)'],
|
||||
[`${prefix} stop --all [--yes]`, 'Stop all scans (Temporal stays up)'],
|
||||
[`${prefix} reset`, 'Stop everything and wipe all Temporal data'],
|
||||
[`${prefix} logs <workspace>`, "Show a scan's live log"],
|
||||
[`${prefix} status <workspace> [--json]`, 'Live phase/agent progress of one scan'],
|
||||
[`${prefix} scans [--json]`, 'List completed scans and their reports'],
|
||||
[`${prefix} logs [<workspace>]`, "Show a scan's live log (default: running or most recent)"],
|
||||
[`${prefix} logs [<workspace>] --agent <name>`, "Tail one agent's log; --list-agents to list them"],
|
||||
[
|
||||
`${prefix} status [<workspace>] [--json]`,
|
||||
'Live phase/agent progress of one scan (default: running or most recent)',
|
||||
],
|
||||
[`${prefix} scans [--json]`, 'List running and completed scans'],
|
||||
...(mode === 'local' ? [[`${prefix} build [--no-cache]`, 'Build worker image'] as const] : []),
|
||||
[`${prefix} version [--json]`, 'Show version'],
|
||||
[`${prefix} help`, 'Show this help'],
|
||||
@@ -75,13 +111,28 @@ function renderUsage(prefix: string, mode: Mode): string {
|
||||
return rows.map(([command, desc]) => ` ${command.padEnd(commandWidth)} ${desc}`).join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* A boxed "start your first scan" call to action, shown in help when no scans exist
|
||||
* yet. Prefix-aware, so local mode renders `./shannon start …`.
|
||||
*/
|
||||
function renderFirstScanBox(prefix: string): string {
|
||||
const command = `${prefix} start -u <url> -r <path>`;
|
||||
const title = 'Start your first scan';
|
||||
const padX = 3;
|
||||
const inner = Math.max(command.length, title.length) + padX * 2;
|
||||
const rule = (left: string, right: string): string => ` ${left}${'─'.repeat(inner)}${right}`;
|
||||
const line = (text: string): string => ` │${' '.repeat(padX)}${text}${' '.repeat(inner - padX - text.length)}│`;
|
||||
return [rule('╭', '╮'), line(title), line(''), line(command), rule('╰', '╯')].join('\n');
|
||||
}
|
||||
|
||||
function showHelp(withSplash: boolean): void {
|
||||
const mode = getMode();
|
||||
const prefix = commandPrefix();
|
||||
|
||||
const header = withSplash ? '' : '\nShannon — AI Pentester by Keygraph\n';
|
||||
const firstScan = stdoutIsTerminal() ? `\n${renderFirstScanBox(prefix)}\n` : '';
|
||||
|
||||
console.log(`${header}
|
||||
console.log(`${header}${firstScan}
|
||||
Usage:
|
||||
${renderUsage(prefix, mode)}
|
||||
|
||||
@@ -101,6 +152,21 @@ Docs & source: https://github.com/KeygraphHQ/shannon
|
||||
`);
|
||||
}
|
||||
|
||||
/**
|
||||
* First-run guidance for a bare `npx @keygraph/shannon` invocation when neither a
|
||||
* credentials file nor an exported shell credential exists. Walks the user to `setup`.
|
||||
*/
|
||||
function showSetupPrompt(): void {
|
||||
const prefix = commandPrefix();
|
||||
console.log(`
|
||||
Welcome to Shannon — AI Pentester by Keygraph
|
||||
|
||||
No credentials configured yet. To get started, run:
|
||||
|
||||
${prefix} setup
|
||||
`);
|
||||
}
|
||||
|
||||
interface ParsedStartArgs {
|
||||
url: string;
|
||||
repo: string;
|
||||
@@ -152,6 +218,32 @@ function parseStartArgs(argv: string[]): ParsedStartArgs {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the workspace a viewing command (`logs`, `status`) acts on: the name the user
|
||||
* gave, or an inferred default. An inferred choice is announced on stderr so it is never a
|
||||
* silent guess; when nothing can be inferred, exit with usage guidance.
|
||||
*/
|
||||
function resolveViewingWorkspace(positional: string | undefined, usage: string): string {
|
||||
if (positional) {
|
||||
return positional;
|
||||
}
|
||||
|
||||
const target = resolveDefaultWorkspace({ allowFinished: true });
|
||||
if (target.kind === 'ok') {
|
||||
// In JSON mode stderr is reserved for the single error envelope, so a successful
|
||||
// inference stays silent — the JSON payload itself names the chosen workspace.
|
||||
if (!jsonErrorsEnabled()) {
|
||||
const which = target.running ? 'running scan' : 'most recent scan';
|
||||
console.error(`No workspace given; using ${which} "${target.workspace}".`);
|
||||
}
|
||||
return target.workspace;
|
||||
}
|
||||
if (target.kind === 'ambiguous') {
|
||||
failUsage('Multiple scans are running — specify which one:', ` ${target.running.join(', ')}`, '', usage);
|
||||
}
|
||||
failUsage('Workspace is required', usage);
|
||||
}
|
||||
|
||||
// === Main Dispatch ===
|
||||
|
||||
async function main(): Promise<void> {
|
||||
@@ -163,24 +255,54 @@ async function main(): Promise<void> {
|
||||
throw err;
|
||||
});
|
||||
|
||||
if (wantsJsonErrors(process.argv.slice(2))) {
|
||||
enableJsonErrors();
|
||||
}
|
||||
|
||||
blockSudo();
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
const command = args[0];
|
||||
const rest = args.slice(1);
|
||||
|
||||
if (command === undefined || command === 'help' || command === '--help' || command === '-h') {
|
||||
if (command === undefined || command === '--help' || command === '-h') {
|
||||
const topic = rest[0];
|
||||
if (topic && isHelpableCommand(topic)) {
|
||||
printCommandHelp(topic);
|
||||
} else {
|
||||
const bare = command === undefined;
|
||||
if (bare && stdoutIsTerminal()) displaySplash(isLocal() ? undefined : getVersion());
|
||||
showHelp(bare);
|
||||
const needsSetup = bare && !isLocal() && !configFileExists() && !hasExportedCredentials();
|
||||
if (needsSetup) {
|
||||
showSetupPrompt();
|
||||
} else {
|
||||
showHelp(bare);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// An explicit `help <topic>` names a topic on purpose, so an unknown one is a usage
|
||||
// error — unlike `--help <junk>`, where the junk is ignored and global help wins.
|
||||
if (command === 'help') {
|
||||
const topic = rest[0];
|
||||
// A flag (`help --help`) is a help request, not a topic name.
|
||||
if (topic === undefined || topic === 'help' || topic.startsWith('-')) {
|
||||
showHelp(false);
|
||||
return;
|
||||
}
|
||||
if (isHelpableCommand(topic)) {
|
||||
printCommandHelp(topic);
|
||||
return;
|
||||
}
|
||||
const suggestion = closestMatch(topic, helpTopics());
|
||||
failUsage(
|
||||
`Unknown help topic: ${topic}`,
|
||||
...(suggestion ? [`Did you mean '${suggestion}'?`] : []),
|
||||
`Run '${commandPrefix()} help' to see available commands.`,
|
||||
);
|
||||
}
|
||||
|
||||
// Reachable from any invocation: `-h`/`--help` anywhere wins over the rest of the line.
|
||||
if (isHelpableCommand(command) && (rest.includes('-h') || rest.includes('--help'))) {
|
||||
printCommandHelp(command);
|
||||
@@ -210,20 +332,25 @@ async function main(): Promise<void> {
|
||||
break;
|
||||
}
|
||||
case 'logs': {
|
||||
const { positionals } = parseArgs(rest, { maxPositionals: 1 });
|
||||
const workspaceId = positionals[0];
|
||||
if (!workspaceId) {
|
||||
failUsage('Workspace ID is required', `Usage: ${commandPrefix()} logs <workspace>`);
|
||||
}
|
||||
logs(workspaceId);
|
||||
const { flags, values, positionals } = parseArgs(rest, {
|
||||
booleans: { listAgents: ['--list-agents'] },
|
||||
values: { agent: ['--agent'] },
|
||||
maxPositionals: 1,
|
||||
});
|
||||
const workspaceId = resolveViewingWorkspace(
|
||||
positionals[0],
|
||||
`Usage: ${commandPrefix()} logs [<workspace>] [--agent <name>] [--list-agents]`,
|
||||
);
|
||||
logs(workspaceId, {
|
||||
...(values.agent !== undefined && { agent: values.agent }),
|
||||
...(flags.listAgents && { listAgents: true }),
|
||||
});
|
||||
break;
|
||||
}
|
||||
case 'status': {
|
||||
const { flags, positionals } = parseArgs(rest, { booleans: { json: ['--json'] }, maxPositionals: 1 });
|
||||
const workspaceId = positionals[0];
|
||||
if (!workspaceId) {
|
||||
failUsage('Workspace is required', `Usage: ${commandPrefix()} status <workspace> [--json]`);
|
||||
}
|
||||
const usage = `Usage: ${commandPrefix()} status [<workspace>] [--json]`;
|
||||
const workspaceId = resolveViewingWorkspace(positionals[0], usage);
|
||||
await status(workspaceId, { json: !!flags.json });
|
||||
break;
|
||||
}
|
||||
|
||||
@@ -42,6 +42,12 @@ export const INTERNAL_DIR = '.shannon';
|
||||
*/
|
||||
export const FINAL_REPORT_PDF_FILENAME = 'Security-Assessment-Report.pdf';
|
||||
|
||||
/**
|
||||
* Customer-facing Markdown report name at the run root.
|
||||
* Must match FINAL_REPORT_MD_FILENAME in the worker package.
|
||||
*/
|
||||
export const FINAL_REPORT_MD_FILENAME = 'Security-Assessment-Report.md';
|
||||
|
||||
/**
|
||||
* Resolve a run-directory file (e.g. session.json, workflow.log), preferring the
|
||||
* current INTERNAL_DIR location and falling back to the legacy run-root location
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
/** Durable CLI-owned workflow candidates that bridge Docker launch and session registration. */
|
||||
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { INTERNAL_DIR } from './paths.js';
|
||||
|
||||
const SCHEMA_VERSION = 1 as const;
|
||||
const PENDING_DIR = 'pending-workflows';
|
||||
|
||||
export interface PendingWorkflowIdentity {
|
||||
readonly schema_version: typeof SCHEMA_VERSION;
|
||||
readonly workflow_id: string;
|
||||
readonly task_queue: string;
|
||||
readonly created_at: string;
|
||||
}
|
||||
|
||||
export interface PendingWorkflowReadResult {
|
||||
readonly identities: readonly PendingWorkflowIdentity[];
|
||||
readonly unreadableCount: number;
|
||||
}
|
||||
|
||||
function pendingDir(workspacePath: string): string {
|
||||
return path.join(workspacePath, INTERNAL_DIR, PENDING_DIR);
|
||||
}
|
||||
|
||||
function pendingFile(workspacePath: string, taskQueue: string): string {
|
||||
return path.join(pendingDir(workspacePath), `launch-${encodeURIComponent(taskQueue)}.json`);
|
||||
}
|
||||
|
||||
function syncDirectory(directory: string): void {
|
||||
const descriptor = fs.openSync(directory, 'r');
|
||||
try {
|
||||
fs.fsyncSync(descriptor);
|
||||
} finally {
|
||||
fs.closeSync(descriptor);
|
||||
}
|
||||
}
|
||||
|
||||
/** Persist the candidate before docker run, so a vanished pre-registration worker remains addressable. */
|
||||
export function writePendingWorkflowIdentity(workspacePath: string, workflowId: string, taskQueue: string): void {
|
||||
const directory = pendingDir(workspacePath);
|
||||
const directoryAlreadyExisted = fs.existsSync(directory);
|
||||
fs.mkdirSync(directory, { recursive: true });
|
||||
if (!directoryAlreadyExisted) syncDirectory(path.dirname(directory));
|
||||
const destination = pendingFile(workspacePath, taskQueue);
|
||||
const temporary = `${destination}.tmp-${process.pid}-${Date.now()}`;
|
||||
const identity: PendingWorkflowIdentity = {
|
||||
schema_version: SCHEMA_VERSION,
|
||||
workflow_id: workflowId,
|
||||
task_queue: taskQueue,
|
||||
created_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
const descriptor = fs.openSync(temporary, 'wx', 0o600);
|
||||
try {
|
||||
fs.writeFileSync(descriptor, `${JSON.stringify(identity, null, 2)}\n`, 'utf8');
|
||||
fs.fsyncSync(descriptor);
|
||||
} finally {
|
||||
fs.closeSync(descriptor);
|
||||
}
|
||||
try {
|
||||
// Link installs the fully-fsynced inode without replacing an existing task-queue record.
|
||||
fs.linkSync(temporary, destination);
|
||||
fs.unlinkSync(temporary);
|
||||
syncDirectory(directory);
|
||||
} catch (error) {
|
||||
fs.rmSync(temporary, { force: true });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/** Remove one candidate only after session registration or a fully verified stop. */
|
||||
export function clearPendingWorkflowIdentity(workspacePath: string, taskQueue: string): void {
|
||||
const directory = pendingDir(workspacePath);
|
||||
fs.rmSync(pendingFile(workspacePath, taskQueue), { force: true });
|
||||
if (fs.existsSync(directory)) syncDirectory(directory);
|
||||
}
|
||||
|
||||
function escapeRegExp(value: string): string {
|
||||
return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
}
|
||||
|
||||
function isPendingWorkflowIdentity(
|
||||
value: unknown,
|
||||
workspace: string,
|
||||
expectedFilename: string,
|
||||
): value is PendingWorkflowIdentity {
|
||||
if (value === null || typeof value !== 'object' || Array.isArray(value)) return false;
|
||||
const candidate = value as Record<string, unknown>;
|
||||
const keys = Object.keys(candidate).sort();
|
||||
const workflowId = candidate.workflow_id;
|
||||
const workflowPattern = new RegExp(`^${escapeRegExp(workspace)}_(?:shannon-|resume_)\\d+$`);
|
||||
const workspaceIsWorkflowId = workflowId === workspace && /_shannon-\d+$/.test(workspace);
|
||||
return (
|
||||
keys.length === 4 &&
|
||||
keys[0] === 'created_at' &&
|
||||
keys[1] === 'schema_version' &&
|
||||
keys[2] === 'task_queue' &&
|
||||
keys[3] === 'workflow_id' &&
|
||||
candidate.schema_version === SCHEMA_VERSION &&
|
||||
typeof workflowId === 'string' &&
|
||||
(workspaceIsWorkflowId || workflowPattern.test(workflowId)) &&
|
||||
typeof candidate.task_queue === 'string' &&
|
||||
/^shannon-[0-9a-f]{8}$/.test(candidate.task_queue) &&
|
||||
expectedFilename === `launch-${encodeURIComponent(candidate.task_queue)}.json` &&
|
||||
typeof candidate.created_at === 'string' &&
|
||||
!Number.isNaN(Date.parse(candidate.created_at)) &&
|
||||
new Date(candidate.created_at).toISOString() === candidate.created_at
|
||||
);
|
||||
}
|
||||
|
||||
/** Read every outstanding launch candidate, preserving corrupt records as an explicit failure count. */
|
||||
export function readPendingWorkflowIdentities(workspacePath: string): PendingWorkflowReadResult {
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = fs.readdirSync(pendingDir(workspacePath)).filter((entry) => entry.endsWith('.json'));
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return { identities: [], unreadableCount: 0 };
|
||||
return { identities: [], unreadableCount: 1 };
|
||||
}
|
||||
|
||||
const identities: PendingWorkflowIdentity[] = [];
|
||||
let unreadableCount = 0;
|
||||
for (const entry of entries) {
|
||||
try {
|
||||
const value: unknown = JSON.parse(fs.readFileSync(path.join(pendingDir(workspacePath), entry), 'utf8'));
|
||||
if (!isPendingWorkflowIdentity(value, path.basename(workspacePath), entry)) {
|
||||
unreadableCount++;
|
||||
continue;
|
||||
}
|
||||
identities.push(value);
|
||||
} catch {
|
||||
unreadableCount++;
|
||||
}
|
||||
}
|
||||
// Atomic-write temp files are intentionally ignored: start cannot spawn Docker until the
|
||||
// final .json rename and fsync above have both completed.
|
||||
return { identities, unreadableCount };
|
||||
}
|
||||
+285
-20
@@ -8,8 +8,17 @@
|
||||
*/
|
||||
|
||||
import type { RunningAgent } from '../temporal-client.js';
|
||||
import { agentClass, PIPELINE, type PipelineState } from './pipeline.js';
|
||||
import {
|
||||
AGENTIC_SAST_STAGE_ORDER,
|
||||
agentClass,
|
||||
isModelBackedOperation,
|
||||
type OperationalStageState,
|
||||
operationFamilyKey,
|
||||
type PipelineState,
|
||||
pipelineForState,
|
||||
} from './pipeline.js';
|
||||
import type { RenderInput } from './render.js';
|
||||
import { safeFailureDetail, safeOperationKey, safeOperationLabel } from './safe-fields.js';
|
||||
|
||||
export type RunState = 'pending' | 'running' | 'completed' | 'failed' | 'skipped';
|
||||
|
||||
@@ -22,14 +31,33 @@ export interface DerivedAgent {
|
||||
readonly durationMs: number | null;
|
||||
readonly runningElapsedMs: number | null;
|
||||
readonly attempt: number | null;
|
||||
/** The step a running operation row is currently on, merged in from its child activity. */
|
||||
readonly detail?: string;
|
||||
/** Reconciliation time for this agent's class, rendered as a trailing `+ duration`.
|
||||
* Reconciliation is model work that produces this agent's inputs, so it is shown
|
||||
* attached to the agent it feeds rather than as free-floating background work. */
|
||||
readonly attachedMs?: number;
|
||||
/** This class's findings could not be grouped, so each one became its own task. */
|
||||
readonly ungrouped?: boolean;
|
||||
readonly error?: string;
|
||||
}
|
||||
|
||||
/** How a phase line summarizes itself: its own wall time, or a k/N tally over its children. */
|
||||
export type PhaseMetaKind = 'duration' | 'count';
|
||||
|
||||
export interface DerivedPhase {
|
||||
readonly key: string;
|
||||
readonly label: string;
|
||||
readonly parallel: boolean;
|
||||
/** Whether the phase renders its agents as sub-rows. Independent of {@link meta}:
|
||||
* Agentic SAST lists its stages under a duration, exploitation lists its classes under a tally. */
|
||||
readonly children: boolean;
|
||||
readonly meta: PhaseMetaKind;
|
||||
readonly state: RunState;
|
||||
/** The phase's own span, when the worker records one for the phase rather than for a single
|
||||
* agent inside it (Agentic SAST). The phase line presents this exactly like an agent row. */
|
||||
readonly summary?: DerivedAgent;
|
||||
/** Rendered after the phase's summary, e.g. to mark work that overlaps other phases. */
|
||||
readonly note?: string;
|
||||
readonly agents: readonly DerivedAgent[];
|
||||
}
|
||||
|
||||
@@ -38,8 +66,25 @@ export function isTerminal(status: string): boolean {
|
||||
return status !== 'RUNNING' && status !== 'UNSPECIFIED';
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the class-level failure recorded for this agent's class applies to this agent.
|
||||
*
|
||||
* A class failure is recorded against the class as a whole, so it matches both of that class's
|
||||
* agents. A reconciliation failure, though, happens only after the analysis agent has already
|
||||
* succeeded, so it belongs to the exploitation lane: attributing it to the analysis row as well
|
||||
* would report an agent that completed as failed.
|
||||
*/
|
||||
function classFailureApplies(name: string, state: PipelineState | null): boolean {
|
||||
if (!state) return false;
|
||||
const vulnClass = agentClass(name);
|
||||
if (!state.failedPipelines.some((f) => f.vulnType === vulnClass)) return false;
|
||||
const reconciliationFailed = (state.failedReconciliations ?? []).some((r) => r.vulnerabilityClass === vulnClass);
|
||||
const isAnalysisAgent = name.endsWith('-vuln');
|
||||
return !(reconciliationFailed && isAnalysisAgent);
|
||||
}
|
||||
|
||||
function isFailedAgent(name: string, state: PipelineState | null): boolean {
|
||||
return !!state && (state.failedAgent === name || state.failedPipelines.some((f) => f.vulnType === agentClass(name)));
|
||||
return !!state && (state.failedAgent === name || classFailureApplies(name, state));
|
||||
}
|
||||
|
||||
/** An agent has entered play once it is running, has metrics, or has failed. */
|
||||
@@ -48,12 +93,12 @@ function isAgentActive(name: string, state: PipelineState | null, running: Set<s
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one agent's state. "Ran" is signalled by a metrics entry, not by
|
||||
* completedAgents — the workflow lists conditionally-skipped agents (e.g. exploit
|
||||
* agents when there is nothing to exploit) as completed but records no metrics for
|
||||
* them. `resolved` is true once we've moved past this agent's phase (the scan is
|
||||
* terminal, or a later phase is already active), at which point a metric-less,
|
||||
* non-running agent is skipped rather than still pending.
|
||||
* Resolve one agent's state. "Ran" is signalled by a metrics entry: a
|
||||
* conditionally-skipped agent (e.g. an exploit agent when there is nothing to
|
||||
* exploit) records no metrics, and the workflow tracks it in skippedAgents rather
|
||||
* than completedAgents. `resolved` is true once we've moved past this agent's phase
|
||||
* (the scan is terminal, or a later phase is already active), at which point a
|
||||
* metric-less, non-running agent is skipped rather than still pending.
|
||||
*/
|
||||
function agentState(name: string, state: PipelineState | null, running: Set<string>, resolved: boolean): RunState {
|
||||
if (running.has(name)) return 'running';
|
||||
@@ -62,13 +107,15 @@ function agentState(name: string, state: PipelineState | null, running: Set<stri
|
||||
return resolved ? 'skipped' : 'pending';
|
||||
}
|
||||
|
||||
/**
|
||||
* Only the presence of a class failure is used here, never its `.error` text: that string is
|
||||
* the worker's raw error for the failed class, not vetted for display, so it is reduced to
|
||||
* a boolean before reaching safeFailureDetail's fixed sentence.
|
||||
*/
|
||||
function agentError(name: string, state: PipelineState | null, byAgent: Map<string, RunningAgent>): string | undefined {
|
||||
const failed = state?.failedPipelines.find((f) => f.vulnType === agentClass(name));
|
||||
return (
|
||||
failed?.error ??
|
||||
byAgent.get(name)?.lastFailure ??
|
||||
(state?.failedAgent === name ? (state.error ?? undefined) : undefined)
|
||||
);
|
||||
const hasFailure =
|
||||
classFailureApplies(name, state) || byAgent.get(name)?.lastFailure !== undefined || state?.failedAgent === name;
|
||||
return safeFailureDetail(hasFailure);
|
||||
}
|
||||
|
||||
/** Scan wall-clock elapsed ms: recorded duration for a closed scan, live elapsed for a running one. */
|
||||
@@ -99,16 +146,17 @@ export function phaseGlyphState(states: readonly RunState[]): RunState {
|
||||
* class had anything to exploit), not still pending.
|
||||
*/
|
||||
export function deriveAgentStates(input: RenderInput): Map<string, RunState> {
|
||||
const runningSet = new Set(input.running.map((r) => r.agent));
|
||||
const pipeline = pipelineForState(input.state);
|
||||
const runningSet = new Set(input.running.filter((runner) => runner.kind === 'agent').map((runner) => runner.agent));
|
||||
const terminal = isTerminal(input.temporalStatus);
|
||||
|
||||
let frontier = -1;
|
||||
PIPELINE.forEach((phase, idx) => {
|
||||
pipeline.forEach((phase, idx) => {
|
||||
if (phase.agents.some((a) => isAgentActive(a.name, input.state, runningSet))) frontier = idx;
|
||||
});
|
||||
|
||||
const states = new Map<string, RunState>();
|
||||
for (const [phaseIdx, phase] of PIPELINE.entries()) {
|
||||
for (const [phaseIdx, phase] of pipeline.entries()) {
|
||||
const resolved = terminal || phaseIdx < frontier;
|
||||
for (const agent of phase.agents) {
|
||||
states.set(agent.name, agentState(agent.name, input.state, runningSet, resolved));
|
||||
@@ -117,6 +165,52 @@ export function deriveAgentStates(input: RenderInput): Map<string, RunState> {
|
||||
return states;
|
||||
}
|
||||
|
||||
/** Which operation families have a running parent stage, and the step to show on it. */
|
||||
interface OperationFamilyView {
|
||||
/** Families whose parent stage row already represents their child activities. */
|
||||
readonly runningFamilies: ReadonlySet<string>;
|
||||
/** Family to current step, present only where the child activities agree on one. */
|
||||
readonly stepByFamily: ReadonlyMap<string, string>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the parent stage rows that own their family's child activities. A family only
|
||||
* resolves to a step when its running children agree: several classes reconcile at once and
|
||||
* their pending activities carry no class, so a family caught mid-stride shows its parent
|
||||
* rows without a step rather than attributing one to the wrong class.
|
||||
*/
|
||||
function operationFamilyView(
|
||||
running: readonly RunningAgent[],
|
||||
persistedOperations: readonly OperationalStageState[],
|
||||
): OperationFamilyView {
|
||||
const runningFamilies = new Set(
|
||||
persistedOperations
|
||||
.filter((operation) => operation.status === 'running')
|
||||
.map((operation) => operationFamilyKey(operation.key)),
|
||||
);
|
||||
|
||||
const labelsByFamily = new Map<string, Set<string>>();
|
||||
for (const runner of running) {
|
||||
if (runner.kind !== 'operation' || runner.parentKey === undefined) continue;
|
||||
if (!runningFamilies.has(runner.parentKey)) continue;
|
||||
const labels = labelsByFamily.get(runner.parentKey) ?? new Set<string>();
|
||||
labels.add(runner.label);
|
||||
labelsByFamily.set(runner.parentKey, labels);
|
||||
}
|
||||
|
||||
const stepByFamily = new Map<string, string>();
|
||||
for (const [family, labels] of labelsByFamily) {
|
||||
const [onlyLabel] = labels;
|
||||
if (labels.size === 1 && onlyLabel !== undefined) stepByFamily.set(family, lowercaseFirst(onlyLabel));
|
||||
}
|
||||
return { runningFamilies, stepByFamily };
|
||||
}
|
||||
|
||||
/** Progress labels are written to start a row; as a detail they continue a sentence. */
|
||||
function lowercaseFirst(label: string): string {
|
||||
return label.charAt(0).toLowerCase() + label.slice(1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Full structured view of the pipeline: every agent's state plus the raw
|
||||
* metrics/timing needed to present it, and each phase's collapsed state.
|
||||
@@ -124,8 +218,9 @@ export function deriveAgentStates(input: RenderInput): Map<string, RunState> {
|
||||
export function derivePipeline(input: RenderInput, now: number): DerivedPhase[] {
|
||||
const states = deriveAgentStates(input);
|
||||
const byAgent = new Map(input.running.map((r) => [r.agent, r]));
|
||||
const pipeline = pipelineForState(input.state);
|
||||
|
||||
return PIPELINE.map((phase) => {
|
||||
const agentPhases = pipeline.map((phase) => {
|
||||
const agents = phase.agents.map((a): DerivedAgent => {
|
||||
const state = states.get(a.name) ?? 'pending';
|
||||
const metrics = input.state?.agentMetrics[a.name];
|
||||
@@ -145,11 +240,181 @@ export function derivePipeline(input: RenderInput, now: number): DerivedPhase[]
|
||||
return {
|
||||
key: phase.key,
|
||||
label: phase.label,
|
||||
parallel: phase.parallel,
|
||||
children: phase.parallel,
|
||||
meta: phase.parallel ? ('count' as const) : ('duration' as const),
|
||||
state: phaseGlyphState(agents.map((ag) => ag.state)),
|
||||
agents,
|
||||
};
|
||||
});
|
||||
|
||||
// Operational rows merge two sources: stages the worker has persisted (durable truth,
|
||||
// including terminal outcomes) and pending activities whose stage record has not landed
|
||||
// yet. Persisted keys win, so a stage is never listed twice while the two views overlap.
|
||||
const persistedOperations = Object.values(input.state?.operationalStages ?? {});
|
||||
const persistedKeys = new Set(persistedOperations.map((operation) => operation.key));
|
||||
const { runningFamilies, stepByFamily } = operationFamilyView(input.running, persistedOperations);
|
||||
const unpersistedRunning = input.running
|
||||
.filter((runner) => runner.kind === 'operation' && !persistedKeys.has(runner.agent))
|
||||
// A child activity whose family already has a running parent stage is that stage's current
|
||||
// step, not separate work: the parent row below represents it, with the step as its detail
|
||||
// where the family's children agree on one. Without such a parent it keeps its own row.
|
||||
.filter((runner) => runner.parentKey === undefined || !runningFamilies.has(runner.parentKey))
|
||||
.map((runner) => ({
|
||||
key: runner.agent,
|
||||
label: runner.label,
|
||||
status: 'running' as const,
|
||||
...(runner.startedAt !== undefined && { startedAt: runner.startedAt }),
|
||||
...(runner.lastFailure !== undefined && { error: safeFailureDetail(true) }),
|
||||
}));
|
||||
const operationalAgents: DerivedAgent[] = [...persistedOperations, ...unpersistedRunning].map((operation) => {
|
||||
const runner = byAgent.get(operation.key);
|
||||
const operationState = operation.status as RunState;
|
||||
const persistedDurationMs = 'durationMs' in operation ? (operation.durationMs ?? null) : null;
|
||||
const detail = operationState === 'running' ? stepByFamily.get(operationFamilyKey(operation.key)) : undefined;
|
||||
return {
|
||||
name: safeOperationKey(operation.key),
|
||||
label: safeOperationLabel(operation.label),
|
||||
state: operationState,
|
||||
durationMs: operationState === 'completed' ? persistedDurationMs : null,
|
||||
runningElapsedMs:
|
||||
operationState === 'running' && operation.startedAt !== undefined ? now - operation.startedAt : null,
|
||||
attempt: operationState === 'running' ? (runner?.attempt ?? null) : null,
|
||||
...(detail !== undefined && { detail }),
|
||||
...(operation.error !== undefined && { error: safeFailureDetail(true) }),
|
||||
};
|
||||
});
|
||||
|
||||
// Operational rows are not peers of the agents. Each one is either model work that
|
||||
// belongs to an agent (reconciliation), model work that belongs to the SAST engine
|
||||
// (its stages), or bookkeeping that only earns a row when it is stuck or broken.
|
||||
return assemblePhases(agentPhases, operationalAgents);
|
||||
}
|
||||
|
||||
/** Reconciliation wall time per vulnerability class, plus the classes whose grouping degraded. */
|
||||
interface ReconciliationView {
|
||||
readonly durationByClass: ReadonlyMap<string, number>;
|
||||
readonly ungroupedClasses: ReadonlySet<string>;
|
||||
}
|
||||
|
||||
function reconciliationView(operations: readonly DerivedAgent[]): ReconciliationView {
|
||||
const durationByClass = new Map<string, number>();
|
||||
const ungroupedClasses = new Set<string>();
|
||||
for (const operation of operations) {
|
||||
if (operationFamilyKey(operation.name) !== 'reconciliation') continue;
|
||||
const [, vulnerabilityClass] = operation.name.split(':');
|
||||
if (vulnerabilityClass === undefined) continue;
|
||||
if (operation.name.endsWith(':fallback')) {
|
||||
ungroupedClasses.add(vulnerabilityClass);
|
||||
continue;
|
||||
}
|
||||
if (operation.durationMs !== null) durationByClass.set(vulnerabilityClass, operation.durationMs);
|
||||
}
|
||||
return { durationByClass, ungroupedClasses };
|
||||
}
|
||||
|
||||
/** Attach each class's reconciliation time to the agent row it feeds. */
|
||||
function withReconciliation(phase: DerivedPhase, view: ReconciliationView): DerivedPhase {
|
||||
const agents = phase.agents.map((agent): DerivedAgent => {
|
||||
const vulnerabilityClass = agentClass(agent.name);
|
||||
const attachedMs = view.durationByClass.get(vulnerabilityClass);
|
||||
const ungrouped = view.ungroupedClasses.has(vulnerabilityClass);
|
||||
return {
|
||||
...agent,
|
||||
...(attachedMs !== undefined && { attachedMs }),
|
||||
...(ungrouped && { ungrouped }),
|
||||
};
|
||||
});
|
||||
return { ...phase, agents };
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the Agentic SAST phase from the aggregate span the parent workflow records and the
|
||||
* per-stage rows the SAST child signals up. Scans that predate stage signalling have the
|
||||
* aggregate but no stages, and render as a bare phase line rather than an error.
|
||||
*/
|
||||
function agenticSastPhase(operations: readonly DerivedAgent[]): DerivedPhase | undefined {
|
||||
const aggregate = operations.find((operation) => operation.name === 'agentic-sast');
|
||||
if (aggregate === undefined) return undefined;
|
||||
|
||||
const byStage = new Map<string, DerivedAgent>();
|
||||
for (const operation of operations) {
|
||||
const [family, stage] = operation.name.split(':');
|
||||
if (family !== 'agentic-sast' || stage === undefined) continue;
|
||||
// The worker's label is the scan log's Title Case form. These rows sit beside the
|
||||
// lowercase class rows below them, so they read in the same register here.
|
||||
byStage.set(stage, { ...operation, label: lowercaseFirst(operation.label) });
|
||||
}
|
||||
// Run order, not insertion order: a resumed or replayed run can persist stages out of order.
|
||||
const stages = AGENTIC_SAST_STAGE_ORDER.map((stage) => byStage.get(stage)).filter(
|
||||
(stage): stage is DerivedAgent => stage !== undefined,
|
||||
);
|
||||
|
||||
return {
|
||||
key: 'agentic-sast',
|
||||
label: 'Agentic SAST',
|
||||
children: stages.length > 0,
|
||||
meta: 'duration',
|
||||
state: aggregate.state,
|
||||
summary: aggregate,
|
||||
// It shares wall time with the pentest phases below it, so the times do not add up
|
||||
// in sequence. Saying so is cheaper than a layout that pretends to be two columns.
|
||||
note: 'concurrent',
|
||||
agents: stages,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Bookkeeping rows worth showing. A deterministic stage that has completed says nothing —
|
||||
* it can only ever read 0s — but one that is still running, or that failed, is exactly what
|
||||
* an operator needs to see, so those keep a row under the phase they belong to.
|
||||
*/
|
||||
function troubledReportSteps(operations: readonly DerivedAgent[]): readonly DerivedAgent[] {
|
||||
return operations.filter((operation) => {
|
||||
if (isModelBackedOperation(operation.name)) return false;
|
||||
if (operationFamilyKey(operation.name) !== 'report') return false;
|
||||
return operation.state === 'running' || operation.state === 'failed';
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Fold operational rows into the agent phases. Nothing here becomes a bucket of its own:
|
||||
* every surviving row is either a SAST stage, time attached to an agent, or a report step
|
||||
* that is currently in trouble.
|
||||
*/
|
||||
function assemblePhases(agentPhases: readonly DerivedPhase[], operations: readonly DerivedAgent[]): DerivedPhase[] {
|
||||
const view = reconciliationView(operations);
|
||||
// Reconciliation produces the exploitation queue, so its time belongs on the exploitation
|
||||
// row it feeds. With exploitation off there is no such row, and it falls back to the
|
||||
// analysis row for the same class so the time is never silently dropped.
|
||||
const attachTo = agentPhases.some((phase) => phase.key === 'exploitation')
|
||||
? 'exploitation'
|
||||
: 'vulnerability-analysis';
|
||||
const reportSteps = troubledReportSteps(operations);
|
||||
|
||||
const phases = agentPhases.map((phase) => {
|
||||
if (phase.key === attachTo) return withReconciliation(phase, view);
|
||||
if (phase.key === 'reporting' && reportSteps.length > 0) {
|
||||
// The report agent stays on the phase line it already titles; the steps in trouble
|
||||
// become its children, so nothing is listed twice.
|
||||
const summary = phase.agents[0];
|
||||
return {
|
||||
...phase,
|
||||
children: true,
|
||||
...(summary !== undefined && { summary }),
|
||||
state: phaseGlyphState([...phase.agents, ...reportSteps].map((row) => row.state)),
|
||||
agents: reportSteps,
|
||||
};
|
||||
}
|
||||
return phase;
|
||||
});
|
||||
|
||||
const sast = agenticSastPhase(operations);
|
||||
if (sast === undefined) return phases;
|
||||
|
||||
// Agentic SAST starts with the scan and runs alongside the pentest, so it reads after
|
||||
// the login check rather than appended past Reporting where it never ran.
|
||||
const afterAuth = phases.findIndex((phase) => phase.key === 'auth-validation') + 1;
|
||||
return [...phases.slice(0, afterAuth), sast, ...phases.slice(afterAuth)];
|
||||
}
|
||||
|
||||
export { agentError };
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
* - apps/worker/src/temporal/activities.ts (the run*Agent activity names → `activityType`)
|
||||
* - apps/worker/src/temporal/shared.ts (PipelineState / PipelineSummary)
|
||||
* - apps/worker/src/types/metrics.ts (AgentMetrics)
|
||||
* - apps/worker/src/types/run-state.ts (PartialReasonView)
|
||||
*/
|
||||
|
||||
export interface AgentSpec {
|
||||
@@ -26,6 +27,18 @@ export interface PhaseSpec {
|
||||
readonly agents: readonly AgentSpec[];
|
||||
}
|
||||
|
||||
export interface ActivityProgressSpec {
|
||||
readonly key: string;
|
||||
readonly label: string;
|
||||
readonly kind: 'agent' | 'operation';
|
||||
/**
|
||||
* Operation rows whose work is already represented by a persisted parent stage. The parent
|
||||
* owns the row; this activity supplies the step shown as its detail. Parent stage keys are
|
||||
* the family key itself or the family key followed by ':' and a class or stage suffix.
|
||||
*/
|
||||
readonly parentKey?: string;
|
||||
}
|
||||
|
||||
/** The pipeline phases in execution order, each with its agents. */
|
||||
export const PIPELINE: readonly PhaseSpec[] = [
|
||||
{
|
||||
@@ -80,9 +93,183 @@ export const PIPELINE: readonly PhaseSpec[] = [
|
||||
},
|
||||
];
|
||||
|
||||
/** Temporal activity type name → canonical agent name, for mapping pendingActivities. */
|
||||
const MISCELLANEOUS_EXPLOIT_AGENT: AgentSpec = {
|
||||
name: 'miscellaneous-exploit',
|
||||
label: 'miscellaneous',
|
||||
activityType: 'runMiscellaneousExploitAgent',
|
||||
};
|
||||
|
||||
/**
|
||||
* Shape the static PIPELINE to one scan's durable truth. expectedAgents, persisted by the
|
||||
* worker at scan start, names every exploit agent the scan can ever run: exploit rows it
|
||||
* excludes are dropped, 'miscellaneous-exploit' is appended only once the miscellaneous pipeline has
|
||||
* admitted findings, and a phase left with no agents disappears entirely. Without state
|
||||
* (the scan has not initialized durable state yet) the full static pipeline is the best
|
||||
* available guess.
|
||||
*/
|
||||
export function pipelineForState(state: PipelineState | null): readonly PhaseSpec[] {
|
||||
if (state?.expectedAgents === undefined) return PIPELINE;
|
||||
const expected = new Set(state.expectedAgents);
|
||||
return PIPELINE.map((phase) => {
|
||||
if (phase.key !== 'exploitation') return phase;
|
||||
const agents = phase.agents.filter((agent) => expected.has(agent.name));
|
||||
if (expected.has(MISCELLANEOUS_EXPLOIT_AGENT.name)) agents.push(MISCELLANEOUS_EXPLOIT_AGENT);
|
||||
return { ...phase, agents };
|
||||
}).filter((phase) => phase.agents.length > 0);
|
||||
}
|
||||
|
||||
const AGENT_ACTIVITY_PROGRESS: Readonly<Record<string, ActivityProgressSpec>> = Object.fromEntries(
|
||||
[...PIPELINE.flatMap((phase) => phase.agents), MISCELLANEOUS_EXPLOIT_AGENT].map((agent) => [
|
||||
agent.activityType,
|
||||
{ key: agent.name, label: agent.label, kind: 'agent' },
|
||||
]),
|
||||
);
|
||||
|
||||
/** Families whose per-class or per-stage work is already carried by one persisted stage row. */
|
||||
const RECONCILIATION_PARENT_KEY = 'reconciliation';
|
||||
const AGENTIC_SAST_PARENT_KEY = 'agentic-sast';
|
||||
|
||||
// Every production activity that is not an agent run must have a row here. describeScan
|
||||
// throws on an unmapped activity type, so adding a worker activity without updating this
|
||||
// table breaks `shannon status` loudly instead of hiding the new work. The authoritative
|
||||
// name lists live in apps/worker/src/temporal/worker.ts,
|
||||
// apps/worker/src/temporal/reconcile-activity-types.ts, and
|
||||
// apps/worker/src/ai/sast/capella/temporal/activity-types.ts.
|
||||
const OPERATION_ACTIVITY_PROGRESS: Readonly<Record<string, ActivityProgressSpec>> = {
|
||||
runPreflightValidation: { key: 'preflight', label: 'Preflight validation', kind: 'operation' },
|
||||
syncPlaywrightStealthConfig: { key: 'preflight', label: 'Browser setup', kind: 'operation' },
|
||||
initDeliverableGit: { key: 'scan-initialization', label: 'Initialize deliverables', kind: 'operation' },
|
||||
syncCodePathDenyRules: { key: 'scan-initialization', label: 'Apply source rules', kind: 'operation' },
|
||||
initializeDurableScanState: { key: 'durable-state', label: 'Saving scan state', kind: 'operation' },
|
||||
persistMiscellaneousOutcome: {
|
||||
key: 'miscellaneous-pipeline',
|
||||
label: 'Including miscellaneous findings',
|
||||
kind: 'operation',
|
||||
},
|
||||
initializeReportProgress: { key: 'report:initialize', label: 'Initialize report state', kind: 'operation' },
|
||||
renumberClassFindings: { key: 'report:renumber', label: 'Renumber findings', kind: 'operation' },
|
||||
assembleReportActivity: { key: 'report:assemble', label: 'Assemble report inputs', kind: 'operation' },
|
||||
compactReportFindings: { key: 'report:compact', label: 'Compact report findings', kind: 'operation' },
|
||||
persistCanonicalReportProgress: { key: 'report:checkpoint', label: 'Saving report progress', kind: 'operation' },
|
||||
finalizeReportOutputs: { key: 'report:finalize', label: 'Finalize report outputs', kind: 'operation' },
|
||||
persistFinalizedReportProgress: { key: 'report:terminal', label: 'Saving final report state', kind: 'operation' },
|
||||
surfaceReportOutputs: { key: 'report:surface', label: 'Surface customer report', kind: 'operation' },
|
||||
checkExploitationQueue: { key: 'queue-check', label: 'Check exploitation queue', kind: 'operation' },
|
||||
loadResumeState: { key: 'resume-validation', label: 'Validate resume state', kind: 'operation' },
|
||||
restoreGitCheckpoint: { key: 'resume-restore', label: 'Restore checkpoint', kind: 'operation' },
|
||||
registerResumeAttempt: { key: 'resume-registration', label: 'Register resume', kind: 'operation' },
|
||||
recordResumeAttempt: { key: 'resume-registration', label: 'Record resume', kind: 'operation' },
|
||||
logPhaseTransition: { key: 'audit-log', label: 'Update audit log', kind: 'operation' },
|
||||
logWorkflowComplete: { key: 'audit-log', label: 'Finalize audit log', kind: 'operation' },
|
||||
saveCheckpoint: { key: 'checkpoint', label: 'Save checkpoint', kind: 'operation' },
|
||||
seedEmptyProducerQueue: {
|
||||
key: 'miscellaneous-pipeline',
|
||||
label: 'Preparing miscellaneous findings',
|
||||
kind: 'operation',
|
||||
},
|
||||
prepareClassReconciliation: {
|
||||
key: 'reconciliation',
|
||||
label: 'Preparing findings',
|
||||
kind: 'operation',
|
||||
parentKey: RECONCILIATION_PARENT_KEY,
|
||||
},
|
||||
enrichClassSastObservations: {
|
||||
key: 'reconciliation',
|
||||
label: 'Adding code context',
|
||||
kind: 'operation',
|
||||
parentKey: RECONCILIATION_PARENT_KEY,
|
||||
},
|
||||
formClassExploitTasks: {
|
||||
key: 'reconciliation',
|
||||
label: 'Grouping into test cases',
|
||||
kind: 'operation',
|
||||
parentKey: RECONCILIATION_PARENT_KEY,
|
||||
},
|
||||
materializeClassExploitTasks: {
|
||||
key: 'reconciliation',
|
||||
label: 'Writing test cases',
|
||||
kind: 'operation',
|
||||
parentKey: RECONCILIATION_PARENT_KEY,
|
||||
},
|
||||
publishClassReconciliationOss: {
|
||||
key: 'reconciliation',
|
||||
label: 'Saving results',
|
||||
kind: 'operation',
|
||||
parentKey: RECONCILIATION_PARENT_KEY,
|
||||
},
|
||||
capellaArchitecture: {
|
||||
key: 'agentic-sast:architecture',
|
||||
label: 'Mapping architecture',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaThreatModel: {
|
||||
key: 'agentic-sast:threat-model',
|
||||
label: 'Modelling threats',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaPlan: {
|
||||
key: 'agentic-sast:plan',
|
||||
label: 'Planning the review',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaResearch: {
|
||||
key: 'agentic-sast:research',
|
||||
label: 'Researching code',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaDedupe: {
|
||||
key: 'agentic-sast:dedupe',
|
||||
label: 'Merging duplicates',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaReview: {
|
||||
key: 'agentic-sast:review',
|
||||
label: 'Reviewing findings',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaCritic: {
|
||||
key: 'agentic-sast:critic',
|
||||
label: 'Critiquing findings',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaConfirm: {
|
||||
key: 'agentic-sast:confirm',
|
||||
label: 'Confirming findings',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaCalibrate: {
|
||||
key: 'agentic-sast:calibrate',
|
||||
label: 'Calibrating risk',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
capellaExport: {
|
||||
key: 'agentic-sast:export',
|
||||
label: 'Exporting findings',
|
||||
kind: 'operation',
|
||||
parentKey: AGENTIC_SAST_PARENT_KEY,
|
||||
},
|
||||
};
|
||||
|
||||
/** Complete production activity mirror. Unknown names are errors, never hidden progress. */
|
||||
export const ACTIVITY_TO_PROGRESS: Readonly<Record<string, ActivityProgressSpec>> = Object.freeze({
|
||||
...AGENT_ACTIVITY_PROGRESS,
|
||||
...OPERATION_ACTIVITY_PROGRESS,
|
||||
});
|
||||
|
||||
/** Agent-only projection of ACTIVITY_TO_PROGRESS: activity type name to canonical agent name. */
|
||||
export const ACTIVITY_TO_AGENT: Readonly<Record<string, string>> = Object.fromEntries(
|
||||
PIPELINE.flatMap((phase) => phase.agents.map((agent) => [agent.activityType, agent.name])),
|
||||
Object.entries(ACTIVITY_TO_PROGRESS)
|
||||
.filter(([, progress]) => progress.kind === 'agent')
|
||||
.map(([activityType, progress]) => [activityType, progress.key]),
|
||||
);
|
||||
|
||||
/** The vuln/exploit class of an agent (e.g. "authz-vuln" → "authz"), for failedPipelines matching. */
|
||||
@@ -100,11 +287,65 @@ export interface AgentMetrics {
|
||||
readonly skipped?: boolean;
|
||||
}
|
||||
|
||||
export interface OperationalStageState {
|
||||
readonly key: string;
|
||||
readonly label: string;
|
||||
readonly status: 'pending' | 'running' | 'completed' | 'failed' | 'skipped';
|
||||
readonly startedAt?: number;
|
||||
readonly durationMs?: number;
|
||||
readonly error?: string;
|
||||
}
|
||||
|
||||
/** Family key a persisted operational stage belongs to, e.g. `reconciliation:xss` to `reconciliation`. */
|
||||
export function operationFamilyKey(stageKey: string): string {
|
||||
const separator = stageKey.indexOf(':');
|
||||
return separator === -1 ? stageKey : stageKey.slice(0, separator);
|
||||
}
|
||||
|
||||
/** The Capella stages that get a progress row, in run order. Mirrors CAPELLA_PROGRESS_STAGES
|
||||
* in apps/worker/src/ai/sast/types.ts — the deterministic `export` stage is not among them. */
|
||||
export const AGENTIC_SAST_STAGE_ORDER: readonly string[] = [
|
||||
'architecture',
|
||||
'threat-model',
|
||||
'plan',
|
||||
'research',
|
||||
'dedupe',
|
||||
'review',
|
||||
'critic',
|
||||
'confirm',
|
||||
'calibrate',
|
||||
];
|
||||
|
||||
/**
|
||||
* Whether an operational stage represents model work rather than bookkeeping.
|
||||
*
|
||||
* Only the agentic-SAST stages and per-class reconciliation run a model; every other
|
||||
* operational stage is a git commit or a durable-state write that can only ever record
|
||||
* sub-second wall time. The progress tree shows model work, so this is what decides
|
||||
* whether a stage is worth a row at all.
|
||||
*/
|
||||
export function isModelBackedOperation(stageKey: string): boolean {
|
||||
const family = operationFamilyKey(stageKey);
|
||||
if (family === 'agentic-sast') return true;
|
||||
// A `reconciliation:<class>:fallback` marker records a degradation, not a model span.
|
||||
return family === 'reconciliation' && !stageKey.endsWith(':fallback');
|
||||
}
|
||||
|
||||
export interface PipelineSummary {
|
||||
readonly totalCostUsd: number;
|
||||
readonly totalDurationMs: number; // Wall-clock (end - start)
|
||||
readonly totalTurns: number;
|
||||
readonly agentCount: number;
|
||||
/** False when operational (Capella/reconciliation) spend is known to be incomplete. */
|
||||
readonly usageAccountingComplete?: boolean;
|
||||
}
|
||||
|
||||
/** One durable degradation reason with its derived safe message (mirror of PartialReasonView). */
|
||||
export interface PartialReasonView {
|
||||
readonly code: string;
|
||||
readonly vulnerabilityClass?: string;
|
||||
readonly stage?: string;
|
||||
readonly message: string;
|
||||
}
|
||||
|
||||
export type PipelineStatus = 'running' | 'completed' | 'failed' | 'cancelled' | 'partial';
|
||||
@@ -114,10 +355,29 @@ export interface PipelineState {
|
||||
readonly currentPhase: string | null;
|
||||
readonly currentAgent: string | null;
|
||||
readonly completedAgents: string[];
|
||||
readonly expectedAgents?: string[];
|
||||
readonly participatingClasses?: string[];
|
||||
readonly failedPipelines: { vulnType: string; error: string }[];
|
||||
readonly failedReconciliations?: { vulnerabilityClass: string; error: string }[];
|
||||
readonly failedAgent: string | null;
|
||||
readonly error: string | null;
|
||||
readonly startTime: number;
|
||||
readonly agentMetrics: Record<string, AgentMetrics>;
|
||||
readonly operationalMetrics?: Record<string, AgentMetrics>;
|
||||
readonly operationalStages?: Record<string, OperationalStageState>;
|
||||
/** `error` is the worker's sanitized failure sentence, safe to print verbatim. */
|
||||
readonly agenticSast?: {
|
||||
readonly status: string;
|
||||
readonly durationMs?: number;
|
||||
/** Reader-facing name of the failed stage, already projected by the worker. */
|
||||
readonly failedStageLabel?: string;
|
||||
readonly error?: string;
|
||||
readonly errorCode?: string;
|
||||
/** Usage-accounting warnings projected by the worker; empty when the ledger reconciled. */
|
||||
readonly warnings?: readonly string[];
|
||||
};
|
||||
readonly nonFatalFailures?: { readonly phase: string; readonly error: string }[];
|
||||
/** Ordered durable degradation reasons with safe messages; empty or absent for full success. */
|
||||
readonly partialReasons?: readonly PartialReasonView[];
|
||||
readonly summary: PipelineSummary | null;
|
||||
}
|
||||
|
||||
+99
-28
@@ -10,9 +10,9 @@
|
||||
import { BOLD, DIM, GOLD, paint, RED, YELLOW } from '../colors.js';
|
||||
import { commandPrefix } from '../mode.js';
|
||||
import type { RunningAgent } from '../temporal-client.js';
|
||||
import { agentError, deriveAgentStates, isTerminal, phaseGlyphState, type RunState, scanElapsedMs } from './derive.js';
|
||||
import { inlineFailureReason } from './failure.js';
|
||||
import { PIPELINE, type PipelineState } from './pipeline.js';
|
||||
import { derivePipeline, isTerminal, type RunState, scanElapsedMs } from './derive.js';
|
||||
import type { PipelineState } from './pipeline.js';
|
||||
import { safeAgenticSast, safeCliIdentifier, safePartialReasons, safeTerminalFailure } from './safe-fields.js';
|
||||
|
||||
export interface RenderInput {
|
||||
readonly workspace: string;
|
||||
@@ -68,7 +68,7 @@ function truncate(text: string, max: number): string {
|
||||
/** Temporal Web UI, published by compose on 8233; deep-links to the workflow when its id is known. */
|
||||
function temporalDashboardUrl(workflowId: string | undefined): string {
|
||||
const base = 'http://localhost:8233';
|
||||
return workflowId ? `${base}/namespaces/default/workflows/${workflowId}` : base;
|
||||
return workflowId ? `${base}/namespaces/default/workflows/${safeCliIdentifier(workflowId)}` : base;
|
||||
}
|
||||
|
||||
// === Glyphs & status ===
|
||||
@@ -95,6 +95,12 @@ const STATE_COLOR: Record<RunState, string> = {
|
||||
skipped: COLORS.dim,
|
||||
};
|
||||
|
||||
/** Column width for an agent or background-work label inside a phase. */
|
||||
const AGENT_LABEL_WIDTH = 18;
|
||||
|
||||
/** Inline budget for a failure sentence, wide enough to carry a whole first sentence. */
|
||||
const FAILURE_DETAIL_WIDTH = 120;
|
||||
|
||||
/** Braille spinner frames for running agents — the clack loader style. */
|
||||
const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'] as const;
|
||||
|
||||
@@ -112,42 +118,68 @@ function statusBadge(input: RenderInput, opts: RenderOptions): string {
|
||||
const workflowStatus = input.state?.status;
|
||||
if (!isTerminal(input.temporalStatus)) return paint('running', COLORS.gold, opts.color);
|
||||
if (workflowStatus === 'partial') return paint('partial', COLORS.yellow, opts.color);
|
||||
if (workflowStatus === 'cancelled') return paint('cancelled', COLORS.yellow, opts.color);
|
||||
if (input.temporalStatus === 'COMPLETED') return paint('completed', COLORS.gold, opts.color);
|
||||
if (input.temporalStatus === 'TERMINATED') return paint('stopped', COLORS.yellow, opts.color);
|
||||
if (input.temporalStatus === 'CANCELLED' || input.temporalStatus === 'CANCELED') {
|
||||
return paint('cancelled', COLORS.yellow, opts.color);
|
||||
}
|
||||
if (input.temporalStatus === 'TIMED_OUT') return paint('timed out', COLORS.red, opts.color);
|
||||
return paint('FAILED', COLORS.red, opts.color);
|
||||
return paint('failed', COLORS.red, opts.color);
|
||||
}
|
||||
|
||||
// === Line builders ===
|
||||
|
||||
/** The parts of a derived row agentMeta reads beyond its state and metrics. */
|
||||
interface RowExtras {
|
||||
readonly runningElapsedMs?: number | null;
|
||||
readonly attachedMs?: number;
|
||||
readonly ungrouped?: boolean;
|
||||
}
|
||||
|
||||
function agentMeta(
|
||||
state: RunState,
|
||||
metrics: { durationMs: number } | undefined,
|
||||
runner: RunningAgent | undefined,
|
||||
error: string | undefined,
|
||||
opts: RenderOptions,
|
||||
step?: string,
|
||||
extras?: RowExtras,
|
||||
): string {
|
||||
if (state === 'completed') {
|
||||
const duration = metrics?.durationMs != null ? formatDuration(metrics.durationMs) : 'done';
|
||||
return paint(duration, COLORS.dim, opts.color);
|
||||
return paint(`${duration}${attachedSuffix(extras)}`, COLORS.dim, opts.color);
|
||||
}
|
||||
if (state === 'running') {
|
||||
const parts = ['running'];
|
||||
if (runner?.startedAt !== undefined) parts.push(formatDuration(opts.now - runner.startedAt));
|
||||
if (step !== undefined) parts.push(step);
|
||||
// An operational row carries its own elapsed time: it is derived from the persisted stage
|
||||
// span, and has no pending activity on the parent workflow to read a start time from.
|
||||
const elapsedMs =
|
||||
runner?.startedAt !== undefined ? opts.now - runner.startedAt : (extras?.runningElapsedMs ?? null);
|
||||
if (elapsedMs !== null) parts.push(formatDuration(elapsedMs));
|
||||
if (runner && runner.attempt > 1) parts.push(`retry ${runner.attempt}`);
|
||||
return paint(parts.join(' · '), COLORS.gold, opts.color);
|
||||
}
|
||||
if (state === 'failed') {
|
||||
const detail = error ? ` · ${truncate(error, 46)}` : '';
|
||||
const detail = error ? ` · ${truncate(error, FAILURE_DETAIL_WIDTH)}` : '';
|
||||
return paint(`failed${detail}`, COLORS.red, opts.color);
|
||||
}
|
||||
if (state === 'skipped') return paint('skipped', COLORS.dim, opts.color);
|
||||
return paint('queued', COLORS.dim, opts.color);
|
||||
}
|
||||
|
||||
/**
|
||||
* Time a reconciliation lane contributed to this agent's class, shown as `+ duration` on the
|
||||
* row it feeds. `ungrouped` marks a class whose findings could not be grouped, so each one
|
||||
* was tested separately and duplicates are expected.
|
||||
*/
|
||||
function attachedSuffix(extras: RowExtras | undefined): string {
|
||||
if (extras === undefined) return '';
|
||||
const time = extras.attachedMs === undefined ? '' : ` + ${formatDuration(extras.attachedMs)}`;
|
||||
return extras.ungrouped ? `${time} · ungrouped` : time;
|
||||
}
|
||||
|
||||
function phaseMeta(states: readonly RunState[], inPlay: number, parallel: boolean, opts: RenderOptions): string {
|
||||
if (states.every((s) => s === 'pending')) return paint('pending', COLORS.dim, opts.color);
|
||||
if (states.every((s) => s === 'skipped')) return paint('skipped', COLORS.dim, opts.color);
|
||||
@@ -163,35 +195,43 @@ function phaseMeta(states: readonly RunState[], inPlay: number, parallel: boolea
|
||||
/** Render the full progress frame as one string (no trailing newline). */
|
||||
export function renderScan(input: RenderInput, opts: RenderOptions): string {
|
||||
const byAgent = new Map(input.running.map((r) => [r.agent, r]));
|
||||
const stateMap = deriveAgentStates(input);
|
||||
const phases = derivePipeline(input, opts.now);
|
||||
const lines: string[] = ['', ...headerLines(input, opts), ''];
|
||||
|
||||
const metaFor = (name: string, state: RunState): string =>
|
||||
agentMeta(state, input.state?.agentMetrics[name], byAgent.get(name), agentError(name, input.state, byAgent), opts);
|
||||
// Only agents that have actually entered play are shown; pending/skipped ones stay hidden.
|
||||
const inPlay = (s: RunState): boolean => s === 'running' || s === 'completed' || s === 'failed';
|
||||
|
||||
for (const phase of PIPELINE) {
|
||||
const states = phase.agents.map((a) => stateMap.get(a.name) ?? 'pending');
|
||||
for (const phase of phases) {
|
||||
const states = phase.agents.map((agent) => agent.state);
|
||||
const playing = states.filter(inPlay).length;
|
||||
const phaseRunState: RunState = phaseGlyphState(states);
|
||||
const phaseRunState = phase.state;
|
||||
const metaFor = (agent: (typeof phase.agents)[number]): string => {
|
||||
const metrics = agent.durationMs === null ? undefined : { durationMs: agent.durationMs };
|
||||
return agentMeta(agent.state, metrics, byAgent.get(agent.name), agent.error, opts, agent.detail, agent);
|
||||
};
|
||||
|
||||
// A single-agent phase carries that agent's own duration/cost on the phase line once it
|
||||
// starts; a parallel phase gets a "k/N done" summary over the agents in play.
|
||||
// A phase summarizes itself by wall time or by a "k/N done" tally. A phase with its own
|
||||
// recorded span (Agentic SAST) presents it like any agent row; otherwise a single-agent
|
||||
// phase borrows its one agent's duration once that agent starts.
|
||||
const first = phase.agents[0];
|
||||
const firstState = states[0];
|
||||
const phaseMetaStr =
|
||||
!phase.parallel && first && firstState && inPlay(firstState)
|
||||
? metaFor(first.name, firstState)
|
||||
: phaseMeta(states, playing, phase.parallel, opts);
|
||||
lines.push(` ${glyph(phaseRunState, opts)} ${phase.label.padEnd(26)}${phaseMetaStr}`);
|
||||
const borrowed = first && firstState && inPlay(firstState) ? metaFor(first) : undefined;
|
||||
const durationMeta = phase.summary === undefined ? borrowed : metaFor(phase.summary);
|
||||
const summaryMeta =
|
||||
phase.meta === 'duration' && durationMeta !== undefined
|
||||
? durationMeta
|
||||
: phaseMeta(states, playing, phase.meta === 'count', opts);
|
||||
const note = phase.note === undefined ? '' : paint(` · ${phase.note}`, COLORS.dim, opts.color);
|
||||
lines.push(` ${glyph(phaseRunState, opts)} ${phase.label.padEnd(26)}${summaryMeta}${note}`);
|
||||
|
||||
if (!phase.parallel) continue;
|
||||
if (!phase.children) continue;
|
||||
for (let i = 0; i < phase.agents.length; i++) {
|
||||
const agent = phase.agents[i];
|
||||
const state = states[i];
|
||||
if (!agent || !state || !inPlay(state)) continue;
|
||||
lines.push(` ${glyph(state, opts)} ${agent.label.padEnd(18)}${metaFor(agent.name, state)}`);
|
||||
// Two trailing spaces before padding, so a label wider than the column still separates
|
||||
// from its meta text; a label inside the column pads to the same width as before.
|
||||
lines.push(` ${glyph(state, opts)} ${`${agent.label} `.padEnd(AGENT_LABEL_WIDTH)}${metaFor(agent)}`);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -202,7 +242,7 @@ export function renderScan(input: RenderInput, opts: RenderOptions): string {
|
||||
function headerLines(input: RenderInput, opts: RenderOptions): string[] {
|
||||
const elapsedMs = scanElapsedMs(input, opts.now);
|
||||
const meta = [statusBadge(input, opts), elapsedMs !== undefined ? formatDuration(elapsedMs) : '—'].join(' · ');
|
||||
return [` ${paint('Scan:', COLORS.bold, opts.color)} ${input.workspace.padEnd(22)} ${meta}`];
|
||||
return [` ${paint('Scan:', COLORS.bold, opts.color)} ${safeCliIdentifier(input.workspace).padEnd(22)} ${meta}`];
|
||||
}
|
||||
|
||||
/** Aligned label column for the footer's Logs / Temporal rows. */
|
||||
@@ -223,15 +263,46 @@ function footerLines(input: RenderInput, opts: RenderOptions): string[] {
|
||||
|
||||
if (isTerminal(input.temporalStatus) && input.state?.summary) {
|
||||
const wall = formatDuration(input.state.summary.totalDurationMs);
|
||||
return ['', ` Time Taken ${wall}`];
|
||||
const lines = ['', ` Time Taken ${wall}`];
|
||||
|
||||
// A partial scan names each durable degradation reason through its safe message,
|
||||
// so the operator never has to guess why the badge is not "completed".
|
||||
const reasons = safePartialReasons(input.state.partialReasons ?? []);
|
||||
if (reasons.length > 0) {
|
||||
lines.push('', ` ${paint('Why this scan is partial:', COLORS.yellow, opts.color)}`);
|
||||
for (const reason of reasons) {
|
||||
lines.push(paint(` - ${reason.message}`, COLORS.dim, opts.color));
|
||||
}
|
||||
// The safe message names what degraded; these three name the agentic-SAST failure
|
||||
// behind it, under the same labels the scan log and worker output use.
|
||||
const agenticSast = safeAgenticSast(input.state.agenticSast);
|
||||
if (agenticSast?.status === 'failed') {
|
||||
if (agenticSast.failedStageLabel !== undefined) {
|
||||
lines.push(paint(` Agentic SAST stopped at: ${agenticSast.failedStageLabel}`, COLORS.dim, opts.color));
|
||||
}
|
||||
if (agenticSast.error !== undefined) {
|
||||
lines.push(paint(` What happened: ${agenticSast.error}`, COLORS.dim, opts.color));
|
||||
}
|
||||
if (agenticSast.errorCode !== undefined) {
|
||||
lines.push(paint(` Reference code (for a bug report): ${agenticSast.errorCode}`, COLORS.dim, opts.color));
|
||||
}
|
||||
}
|
||||
}
|
||||
if (input.state.summary.usageAccountingComplete === false) {
|
||||
lines.push(
|
||||
paint(' Cost is incomplete — some background work is not included in this total.', COLORS.dim, opts.color),
|
||||
);
|
||||
}
|
||||
return lines;
|
||||
}
|
||||
|
||||
const logsValue = `${prefix} logs ${input.workspace}`;
|
||||
const logsValue = `${prefix} logs ${safeCliIdentifier(input.workspace)}`;
|
||||
const temporalValue = temporalDashboardUrl(input.workflowId);
|
||||
|
||||
if (isTerminal(input.temporalStatus)) {
|
||||
const rawReason = input.failureMessage ?? input.state?.error;
|
||||
const reason = rawReason ? inlineFailureReason(rawReason) : 'no result recorded';
|
||||
const hasRecordedFailure =
|
||||
input.failureMessage !== undefined || (input.state !== null && input.state.error !== null);
|
||||
const reason = safeTerminalFailure(hasRecordedFailure) ?? 'no result recorded';
|
||||
return [
|
||||
footerDivider(opts),
|
||||
paint(
|
||||
|
||||
@@ -0,0 +1,278 @@
|
||||
/**
|
||||
* Closed-field projection for Temporal values displayed by the CLI.
|
||||
*
|
||||
* PipelineState travels through Temporal from a worker container this process does not
|
||||
* control, so free-text fields are treated as unvetted: this module either matches a
|
||||
* value against a known closed set (safe to print as-is) or collapses it to a fixed,
|
||||
* bounded message. A value with no case here should fail closed to something generic,
|
||||
* never pass through untouched.
|
||||
*/
|
||||
|
||||
import type { PartialReasonView, PipelineState } from './pipeline.js';
|
||||
|
||||
const CLASS_NAMES: Readonly<Record<string, string>> = Object.freeze({
|
||||
injection: 'Injection',
|
||||
xss: 'Cross-Site Scripting',
|
||||
auth: 'Authentication',
|
||||
authz: 'Authorization',
|
||||
ssrf: 'Server-Side Request Forgery',
|
||||
miscellaneous: 'Miscellaneous',
|
||||
});
|
||||
|
||||
const STAGE_NAMES: Readonly<Record<string, string>> = Object.freeze({
|
||||
architecture: 'architecture mapping',
|
||||
'threat-model': 'threat modelling',
|
||||
plan: 'review planning',
|
||||
research: 'deep code research',
|
||||
dedupe: 'duplicate merging',
|
||||
review: 'independent review',
|
||||
critic: 'viability critique',
|
||||
confirm: 'static confirmation',
|
||||
calibrate: 'risk calibration',
|
||||
export: 'findings export',
|
||||
workflow: 'orchestration',
|
||||
});
|
||||
|
||||
const TERMINAL_STAGE_NAMES = new Set([
|
||||
'architecture',
|
||||
'threat model',
|
||||
'planning',
|
||||
'audit wave',
|
||||
'deduplication',
|
||||
'review',
|
||||
'critic',
|
||||
'confirmation',
|
||||
'calibration',
|
||||
'export',
|
||||
'orchestration',
|
||||
]);
|
||||
|
||||
const CAPELLA_FAILURE_MESSAGES = new Set([
|
||||
'Provider authentication failed. Verify the configured credential.',
|
||||
'Agentic SAST configuration is invalid.',
|
||||
'Agentic SAST received invalid input.',
|
||||
'An agentic SAST step returned an unusable result.',
|
||||
'An agentic SAST step failed.',
|
||||
'Agentic SAST infrastructure failed before producing a usable result.',
|
||||
'Agentic SAST had not finished when the scan stopped.',
|
||||
]);
|
||||
|
||||
// Mirrors apps/worker/src/types/errors.ts. The CLI cannot import from the worker package,
|
||||
// so keep this exact closed set in sync with ProviderFailureCategory.
|
||||
const PROVIDER_FAILURE_CATEGORIES = new Set([
|
||||
'rate_limit',
|
||||
'overloaded',
|
||||
'transport',
|
||||
'context_limit',
|
||||
'quota',
|
||||
'authentication',
|
||||
'configuration',
|
||||
'unknown',
|
||||
]);
|
||||
|
||||
function isProviderFailureCategory(value: unknown): value is string {
|
||||
return typeof value === 'string' && PROVIDER_FAILURE_CATEGORIES.has(value);
|
||||
}
|
||||
|
||||
const OPERATION_LABELS = new Set([
|
||||
'Agentic SAST',
|
||||
// Capella stage rows, signalled up from the SAST child workflow. Mirrors
|
||||
// CAPELLA_STAGE_LABELS in apps/worker/src/ai/sast/types.ts, minus the deterministic
|
||||
// export stage, which never becomes a row.
|
||||
'Architecture',
|
||||
'Threat model',
|
||||
'Plan',
|
||||
'Research',
|
||||
'Dedupe',
|
||||
'Review',
|
||||
'Critique',
|
||||
'Confirm',
|
||||
'Calibrate',
|
||||
'Reconcile injection',
|
||||
'Reconcile xss',
|
||||
'Reconcile auth',
|
||||
'Reconcile authz',
|
||||
'Reconcile ssrf',
|
||||
'Reconcile miscellaneous',
|
||||
'Prepare reconciliation',
|
||||
'Enrich observations',
|
||||
'Form exploit tasks',
|
||||
'Materialize exploit tasks',
|
||||
'Publish reconciliation',
|
||||
'Renumber injection',
|
||||
'Renumber xss',
|
||||
'Renumber auth',
|
||||
'Renumber authz',
|
||||
'Renumber ssrf',
|
||||
'Renumber miscellaneous',
|
||||
'Initialize report state',
|
||||
'Assemble report inputs',
|
||||
'Compact report findings',
|
||||
'Saving report progress',
|
||||
'Finalize report outputs',
|
||||
'Finalize report without SARIF',
|
||||
'Saving final report state',
|
||||
'Surface customer report',
|
||||
]);
|
||||
|
||||
function safeClassName(value: string | undefined): string | undefined {
|
||||
return value === undefined ? undefined : CLASS_NAMES[value];
|
||||
}
|
||||
|
||||
function safeStageName(value: string | undefined): string | undefined {
|
||||
return value === undefined ? undefined : STAGE_NAMES[value];
|
||||
}
|
||||
|
||||
function reasonMessage(reason: PartialReasonView): string | undefined {
|
||||
const className = safeClassName(reason.vulnerabilityClass);
|
||||
switch (reason.code) {
|
||||
case 'agentic_sast_failed': {
|
||||
const stageName = safeStageName(reason.stage);
|
||||
return stageName === undefined
|
||||
? 'Agentic SAST failed, so the pentest continued without its findings.'
|
||||
: `Agentic SAST failed during ${stageName}, so the pentest continued without its findings.`;
|
||||
}
|
||||
case 'agentic_sast_reduced':
|
||||
return 'Agentic SAST completed with reduced coverage.';
|
||||
case 'class_pipeline_failed':
|
||||
return className === undefined
|
||||
? undefined
|
||||
: `${className} could not be fully assessed. The other classes completed. Re-running this workspace retries only the part that failed.`;
|
||||
case 'class_reconciliation_failed':
|
||||
return className === undefined
|
||||
? undefined
|
||||
: `${className} findings could not be grouped into test cases, so that class was not exploited and its findings are not in the report.`;
|
||||
case 'report_renumber_failed':
|
||||
return className === undefined
|
||||
? undefined
|
||||
: `${className} findings kept their working reference numbers, so numbering in the report may have gaps. The findings themselves are complete.`;
|
||||
case 'report_compaction_failed':
|
||||
return 'Finding reference numbers in the report may have gaps. Every finding is present; only the numbering is affected.';
|
||||
case 'report_class_omitted':
|
||||
return className === undefined
|
||||
? undefined
|
||||
: `${className} was assessed but could not be included in the final report.`;
|
||||
case 'report_sarif_failed':
|
||||
return 'Report SARIF could not be generated. JSON and Markdown remain available.';
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
export function safePartialReasons(reasons: readonly PartialReasonView[]): readonly PartialReasonView[] {
|
||||
return reasons.flatMap((reason) => {
|
||||
const message = reasonMessage(reason);
|
||||
if (message === undefined) return [];
|
||||
const vulnerabilityClass =
|
||||
safeClassName(reason.vulnerabilityClass) === undefined ? undefined : reason.vulnerabilityClass;
|
||||
const stage = safeStageName(reason.stage) === undefined ? undefined : reason.stage;
|
||||
return [
|
||||
{
|
||||
code: reason.code,
|
||||
message,
|
||||
...(vulnerabilityClass !== undefined && { vulnerabilityClass }),
|
||||
...(stage !== undefined && { stage }),
|
||||
},
|
||||
];
|
||||
});
|
||||
}
|
||||
|
||||
/** Upper bounds on the warning array crossing into cli.status.json, so a malformed state cannot bloat it. */
|
||||
const MAX_AGENTIC_SAST_WARNINGS = 20;
|
||||
const MAX_AGENTIC_SAST_WARNING_LENGTH = 2_000;
|
||||
|
||||
/** Sanitize the worker's usage-accounting warnings: strings only, bounded count and length. */
|
||||
function safeAgenticSastWarnings(value: PipelineState['agenticSast']): readonly string[] {
|
||||
const warnings = value?.warnings;
|
||||
if (!Array.isArray(warnings)) return [];
|
||||
return warnings
|
||||
.filter((warning): warning is string => typeof warning === 'string')
|
||||
.slice(0, MAX_AGENTIC_SAST_WARNINGS)
|
||||
.map((warning) => warning.slice(0, MAX_AGENTIC_SAST_WARNING_LENGTH));
|
||||
}
|
||||
|
||||
export function safeAgenticSast(value: PipelineState['agenticSast']):
|
||||
| {
|
||||
readonly status: string;
|
||||
readonly failedStageLabel?: string;
|
||||
readonly error?: string;
|
||||
readonly errorCode?: string;
|
||||
readonly warnings: readonly string[];
|
||||
}
|
||||
| undefined {
|
||||
if (value === undefined || !['disabled', 'running', 'succeeded', 'failed'].includes(value.status)) return undefined;
|
||||
const failedStageLabel = TERMINAL_STAGE_NAMES.has(value.failedStageLabel ?? '') ? value.failedStageLabel : undefined;
|
||||
let error: string | undefined;
|
||||
if (value.error !== undefined && CAPELLA_FAILURE_MESSAGES.has(value.error)) {
|
||||
error = value.error;
|
||||
} else if (value.status === 'failed') {
|
||||
error = 'An agentic SAST step failed.';
|
||||
}
|
||||
const errorCode =
|
||||
value.errorCode !== undefined &&
|
||||
(/^[A-Z][A-Z0-9_]{0,63}$/u.test(value.errorCode) || isProviderFailureCategory(value.errorCode))
|
||||
? value.errorCode
|
||||
: undefined;
|
||||
return {
|
||||
status: value.status,
|
||||
...(failedStageLabel !== undefined && { failedStageLabel }),
|
||||
...(error !== undefined && { error }),
|
||||
...(errorCode !== undefined && { errorCode }),
|
||||
warnings: safeAgenticSastWarnings(value),
|
||||
};
|
||||
}
|
||||
|
||||
export function safeOperationLabel(value: string): string {
|
||||
return OPERATION_LABELS.has(value) ? value : 'Background task';
|
||||
}
|
||||
|
||||
export function safeOperationKey(value: string): string {
|
||||
if (
|
||||
/^(?:agentic-sast|miscellaneous-pipeline|report:(?:initialize|assemble|compact|checkpoint|finalize|finalize-degraded|terminal|surface))$/u.test(
|
||||
value,
|
||||
) ||
|
||||
/^agentic-sast:(?:architecture|threat-model|plan|research|dedupe|review|critic|confirm|calibrate)$/u.test(value) ||
|
||||
/^(?:reconciliation|report:renumber):(?:injection|xss|auth|authz|ssrf|miscellaneous)$/u.test(value) ||
|
||||
/^reconciliation:(?:injection|xss|auth|authz|ssrf|miscellaneous):fallback$/u.test(value)
|
||||
) {
|
||||
return value;
|
||||
}
|
||||
return 'background-task';
|
||||
}
|
||||
|
||||
/**
|
||||
* A workspace or workflow id is printed straight into the progress display, so this
|
||||
* confines it to a plain identifier charset before that happens: no control or escape
|
||||
* characters survive to reach the terminal.
|
||||
*/
|
||||
export function safeCliIdentifier(value: string): string {
|
||||
return /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/u.test(value) ? value : 'unknown';
|
||||
}
|
||||
|
||||
export function safeTemporalStatus(value: string): string {
|
||||
return [
|
||||
'RUNNING',
|
||||
'UNSPECIFIED',
|
||||
'COMPLETED',
|
||||
'FAILED',
|
||||
'CANCELLED',
|
||||
'CANCELED',
|
||||
'TERMINATED',
|
||||
'TIMED_OUT',
|
||||
'CONTINUED_AS_NEW',
|
||||
].includes(value)
|
||||
? value
|
||||
: 'UNKNOWN';
|
||||
}
|
||||
|
||||
export function safeFailureDetail(hasFailure: true): string;
|
||||
export function safeFailureDetail(hasFailure: false): undefined;
|
||||
export function safeFailureDetail(hasFailure: boolean): string | undefined;
|
||||
export function safeFailureDetail(hasFailure: boolean): string | undefined {
|
||||
return hasFailure ? 'This scan step could not be completed.' : undefined;
|
||||
}
|
||||
|
||||
/** Same closed-set trade-off as safeFailureDetail, for the scan-level (not per-agent) failure. */
|
||||
export function safeTerminalFailure(hasFailure: boolean): string | undefined {
|
||||
return hasFailure ? 'The scan could not be completed.' : undefined;
|
||||
}
|
||||
@@ -8,7 +8,15 @@
|
||||
|
||||
import type { DerivedPhase } from './derive.js';
|
||||
import { derivePipeline, isTerminal, scanElapsedMs } from './derive.js';
|
||||
import type { PartialReasonView } from './pipeline.js';
|
||||
import type { RenderInput } from './render.js';
|
||||
import {
|
||||
safeAgenticSast,
|
||||
safeCliIdentifier,
|
||||
safePartialReasons,
|
||||
safeTemporalStatus,
|
||||
safeTerminalFailure,
|
||||
} from './safe-fields.js';
|
||||
|
||||
/** Coarse scan status token, mirroring the human status badge in machine-friendly form. */
|
||||
export type ScanStatus = 'running' | 'completed' | 'partial' | 'failed' | 'stopped' | 'cancelled' | 'timed_out';
|
||||
@@ -27,6 +35,18 @@ export interface StatusJson {
|
||||
readonly endedAt?: string;
|
||||
/** Failure text when a failed scan left no readable state. */
|
||||
readonly failureMessage?: string;
|
||||
/** Ordered durable degradation reasons with safe messages; present only when non-empty. */
|
||||
readonly partialReasons?: readonly PartialReasonView[];
|
||||
/** Agentic SAST outcome, with the worker's sanitized failure sentence and bounded code. */
|
||||
readonly agenticSast?: {
|
||||
readonly status: string;
|
||||
readonly error?: string;
|
||||
readonly errorCode?: string;
|
||||
/** Usage-accounting warnings; always present (empty when the ledger reconciled) so it is never null. */
|
||||
readonly warnings: readonly string[];
|
||||
};
|
||||
/** False when operational (Capella/reconciliation) spend is known to be incomplete. */
|
||||
readonly usageAccountingComplete?: boolean;
|
||||
readonly phases: readonly DerivedPhase[];
|
||||
}
|
||||
|
||||
@@ -34,6 +54,7 @@ export interface StatusJson {
|
||||
function deriveStatus(input: RenderInput): ScanStatus {
|
||||
if (!isTerminal(input.temporalStatus)) return 'running';
|
||||
if (input.state?.status === 'partial') return 'partial';
|
||||
if (input.state?.status === 'cancelled') return 'cancelled';
|
||||
|
||||
switch (input.temporalStatus) {
|
||||
case 'COMPLETED':
|
||||
@@ -53,16 +74,32 @@ function deriveStatus(input: RenderInput): ScanStatus {
|
||||
/** Build the JSON snapshot for a scan at instant `now`. */
|
||||
export function toStatusJson(input: RenderInput, now: number): StatusJson {
|
||||
const elapsedMs = scanElapsedMs(input, now);
|
||||
const partialReasons = safePartialReasons(input.state?.partialReasons ?? []);
|
||||
const agenticSast = safeAgenticSast(input.state?.agenticSast);
|
||||
const usageAccountingComplete = input.state?.summary?.usageAccountingComplete;
|
||||
const failureMessage = safeTerminalFailure(input.failureMessage !== undefined);
|
||||
|
||||
return {
|
||||
workspace: input.workspace,
|
||||
...(input.workflowId !== undefined && { workflowId: input.workflowId }),
|
||||
workspace: safeCliIdentifier(input.workspace),
|
||||
...(input.workflowId !== undefined && { workflowId: safeCliIdentifier(input.workflowId) }),
|
||||
status: deriveStatus(input),
|
||||
temporalStatus: input.temporalStatus,
|
||||
temporalStatus: safeTemporalStatus(input.temporalStatus),
|
||||
elapsedMs: elapsedMs ?? null,
|
||||
...(input.startedAt !== undefined && { startedAt: new Date(input.startedAt).toISOString() }),
|
||||
...(input.endedAt !== undefined && { endedAt: new Date(input.endedAt).toISOString() }),
|
||||
...(input.failureMessage !== undefined && { failureMessage: input.failureMessage }),
|
||||
...(failureMessage !== undefined && { failureMessage }),
|
||||
...(partialReasons.length > 0 && { partialReasons }),
|
||||
// Present only when agentic SAST actually ran; a disabled scan omits the key entirely.
|
||||
...(agenticSast !== undefined &&
|
||||
agenticSast.status !== 'disabled' && {
|
||||
agenticSast: {
|
||||
status: agenticSast.status,
|
||||
...(agenticSast.error !== undefined && { error: agenticSast.error }),
|
||||
...(agenticSast.errorCode !== undefined && { errorCode: agenticSast.errorCode }),
|
||||
warnings: [...agenticSast.warnings],
|
||||
},
|
||||
}),
|
||||
...(usageAccountingComplete !== undefined && { usageAccountingComplete }),
|
||||
phases: derivePipeline(input, now),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
/**
|
||||
* Workspace → Temporal workflow-id resolution.
|
||||
*
|
||||
* A workspace name is not always its workflow id: a fresh scan's id equals the
|
||||
* workspace name, but each resume spawns a new workflow (`<workspace>_resume_<ts>`).
|
||||
* The workspace's session.json records the authoritative id — the latest resume
|
||||
* attempt, or the original — so commands that query Temporal (status, stop) resolve
|
||||
* through here instead of assuming the name is the id.
|
||||
* A workspace name is not always its workflow id: a fresh named workspace gets
|
||||
* `<workspace>_shannon-<timestamp>` as its workflow id (only an auto-named workspace's
|
||||
* directory name equals its original id), and each resume spawns a new workflow
|
||||
* (`<workspace>_resume_<ts>`). The workspace's session.json records the authoritative
|
||||
* id — the latest resume attempt, or the original — so commands that query Temporal
|
||||
* (status, stop) resolve through here instead of assuming the name is the id.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
|
||||
+186
-12
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Thin Temporal client for reading one scan's state.
|
||||
* Thin Temporal client for reading scan state and controlling scan workflow lifecycle.
|
||||
*
|
||||
* A running scan is queried live (getProgress) and read via pendingActivities for
|
||||
* the in-flight agents; a closed scan is read once from its result. Everything goes
|
||||
@@ -9,22 +9,51 @@
|
||||
|
||||
import { setTimeout as sleep } from 'node:timers/promises';
|
||||
import { Client, Connection, WorkflowFailedError, WorkflowNotFoundError } from '@temporalio/client';
|
||||
import { ACTIVITY_TO_AGENT, type PipelineState } from './scan/pipeline.js';
|
||||
import { ACTIVITY_TO_PROGRESS, type PipelineState } from './scan/pipeline.js';
|
||||
|
||||
const ADDRESS = '127.0.0.1:7233';
|
||||
const NAMESPACE = 'default';
|
||||
const LIFECYCLE_RPC_DEADLINE_MS = 3_000;
|
||||
const OPEN_SCAN_WORKFLOW_QUERY =
|
||||
"WorkflowType = 'pentestPipelineWorkflow' AND (ExecutionStatus = 'Running' OR ExecutionStatus = 'Paused')";
|
||||
|
||||
// WorkflowExecutionStatusName values that mean the scan has closed. RUNNING (and the unused
|
||||
// CONTINUED_AS_NEW) are the only non-terminal states.
|
||||
const TERMINAL_STATUSES: ReadonlySet<string> = new Set(['COMPLETED', 'FAILED', 'CANCELLED', 'TERMINATED', 'TIMED_OUT']);
|
||||
// WorkflowExecutionStatusName values that positively prove this execution has closed.
|
||||
// PAUSED is open; UNSPECIFIED and UNKNOWN are not safe closure evidence.
|
||||
const TERMINAL_STATUSES: ReadonlySet<string> = new Set([
|
||||
'COMPLETED',
|
||||
'FAILED',
|
||||
'CANCELLED',
|
||||
'TERMINATED',
|
||||
'CONTINUED_AS_NEW',
|
||||
'TIMED_OUT',
|
||||
]);
|
||||
|
||||
export interface RunningAgent {
|
||||
readonly agent: string;
|
||||
readonly label: string;
|
||||
/** 'agent' rows join the static pipeline tree; 'operation' rows feed the background-work phase. */
|
||||
readonly kind: 'agent' | 'operation';
|
||||
/** Set when a persisted parent stage owns this row; the label then reads as that stage's step. */
|
||||
readonly parentKey?: string;
|
||||
readonly attempt: number;
|
||||
readonly startedAt?: number;
|
||||
readonly lastFailure?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The CLI's activity mirror does not know an activity type the running scan is using, so the
|
||||
* progress tree cannot be rendered completely. Distinct from a Temporal connection failure.
|
||||
*/
|
||||
export class ActivityMirrorError extends Error {
|
||||
override name = 'ActivityMirrorError' as const;
|
||||
|
||||
constructor(activityType: string) {
|
||||
super(
|
||||
`This version of the Shannon command line does not recognise part of the running scan\n(${activityType}). Update Shannon, or watch the scan with: shannon logs <workspace>`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** Convert a proto ITimestamp (seconds is a Long) to epoch millis. */
|
||||
function timestampMs(
|
||||
ts: { seconds?: { toString(): string } | number | null; nanos?: number | null } | null,
|
||||
@@ -47,17 +76,141 @@ export type TerminalOutcome =
|
||||
| { readonly kind: 'success'; readonly state: PipelineState }
|
||||
| { readonly kind: 'failed'; readonly message: string };
|
||||
|
||||
/**
|
||||
* The authoritative Temporal state used by lifecycle commands. Transport failures deliberately
|
||||
* remain errors instead of being represented as a closed workflow: callers must not report a
|
||||
* scan stopped unless Temporal has positively confirmed it.
|
||||
*/
|
||||
export type WorkflowLifecycleState =
|
||||
| { readonly kind: 'open'; readonly status: 'RUNNING' | 'PAUSED' }
|
||||
| { readonly kind: 'terminal'; readonly status: string }
|
||||
| { readonly kind: 'unknown'; readonly status: string }
|
||||
| { readonly kind: 'not-found' };
|
||||
|
||||
/** A scan workflow returned by Temporal's eventually consistent open-workflow visibility query. */
|
||||
export interface RunningScanWorkflow {
|
||||
readonly workflowId: string;
|
||||
readonly taskQueue: string;
|
||||
}
|
||||
|
||||
let clientPromise: Promise<Client> | null = null;
|
||||
|
||||
function getClient(): Promise<Client> {
|
||||
if (!clientPromise) {
|
||||
clientPromise = Connection.connect({ address: ADDRESS }).then(
|
||||
const pending = Connection.connect({ address: ADDRESS, connectTimeout: LIFECYCLE_RPC_DEADLINE_MS }).then(
|
||||
(connection) => new Client({ connection, namespace: NAMESPACE }),
|
||||
);
|
||||
// A rejected connect must not be cached forever: clear the memo so the next call rebuilds
|
||||
// instead of replaying the same failure. Scoped to `pending` so a later successful reconnect
|
||||
// that replaced the memo is left untouched.
|
||||
pending.catch(() => resetClient(pending));
|
||||
clientPromise = pending;
|
||||
}
|
||||
return clientPromise;
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop the memoized client so the next {@link getClient} builds a fresh Connection. The underlying
|
||||
* gRPC channel can wedge such that every reused call fails identically ("Unexpected error while
|
||||
* making gRPC request"), and only a new Connection recovers. Best-effort closes the old channel.
|
||||
* When `only` is given, the memo is cleared only if it still holds that exact promise.
|
||||
*/
|
||||
function resetClient(only?: Promise<Client>): void {
|
||||
if (only !== undefined && clientPromise !== only) return;
|
||||
const previous = clientPromise;
|
||||
clientPromise = null;
|
||||
previous?.then((client) => client.connection.close()).catch(() => {});
|
||||
}
|
||||
|
||||
/** Close the current channel and establish another before a termination retry. */
|
||||
export async function refreshWorkflowLifecycleConnection(): Promise<void> {
|
||||
const previous = clientPromise;
|
||||
if (previous !== null) {
|
||||
if (clientPromise === previous) clientPromise = null;
|
||||
try {
|
||||
const client = await previous;
|
||||
await client.connection.close();
|
||||
} catch {
|
||||
// A failed prior connection is already detached. The new connection below is authoritative.
|
||||
}
|
||||
}
|
||||
await getClient();
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a bounded lifecycle RPC and discard the connection when Temporal did not positively say
|
||||
* that the workflow is absent. A fresh connection is important after a gRPC timeout or transport
|
||||
* failure: reusing a wedged channel can turn a recoverable stop into an indefinitely ambiguous one.
|
||||
*/
|
||||
async function runLifecycleRpc<T>(operation: (client: Client) => Promise<T>): Promise<T> {
|
||||
const pending = getClient();
|
||||
try {
|
||||
const client = await pending;
|
||||
return await client.withDeadline(Date.now() + LIFECYCLE_RPC_DEADLINE_MS, () => operation(client));
|
||||
} catch (err) {
|
||||
if (!(err instanceof WorkflowNotFoundError)) resetClient(pending);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/** Describe a workflow for lifecycle control without reading its progress or pending activities. */
|
||||
export async function describeWorkflowLifecycle(workflowId: string): Promise<WorkflowLifecycleState> {
|
||||
try {
|
||||
const desc = await runLifecycleRpc((client) => client.workflow.getHandle(workflowId).describe());
|
||||
if (desc.status.name === 'RUNNING' || desc.status.name === 'PAUSED') {
|
||||
return { kind: 'open', status: desc.status.name };
|
||||
}
|
||||
if (TERMINAL_STATUSES.has(desc.status.name)) return { kind: 'terminal', status: desc.status.name };
|
||||
return { kind: 'unknown', status: desc.status.name };
|
||||
} catch (err) {
|
||||
if (err instanceof WorkflowNotFoundError) return { kind: 'not-found' };
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/** Request cooperative cancellation. This confirms request acceptance, not workflow closure. */
|
||||
export async function requestWorkflowCancellation(workflowId: string): Promise<'requested' | 'not-found'> {
|
||||
try {
|
||||
await runLifecycleRpc((client) => client.workflow.getHandle(workflowId).cancel());
|
||||
return 'requested';
|
||||
} catch (err) {
|
||||
if (err instanceof WorkflowNotFoundError) return 'not-found';
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/** Request forced termination. This confirms request acceptance, not workflow closure. */
|
||||
export async function requestWorkflowTermination(
|
||||
workflowId: string,
|
||||
reason: string,
|
||||
): Promise<'requested' | 'not-found'> {
|
||||
try {
|
||||
await runLifecycleRpc((client) => client.workflow.getHandle(workflowId).terminate(reason));
|
||||
return 'requested';
|
||||
} catch (err) {
|
||||
if (err instanceof WorkflowNotFoundError) return 'not-found';
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/** List currently open Shannon scan workflows through Temporal visibility. */
|
||||
export async function listRunningScanWorkflows(): Promise<readonly RunningScanWorkflow[]> {
|
||||
return runLifecycleRpc(async (client) => {
|
||||
const workflows: RunningScanWorkflow[] = [];
|
||||
for await (const execution of client.workflow.list({ query: OPEN_SCAN_WORKFLOW_QUERY })) {
|
||||
// Visibility is eventually consistent. Keep only the open scan rows returned by this page;
|
||||
// each discovered workflow is described directly before `stop` accepts its closure.
|
||||
if (
|
||||
(execution.status.name === 'RUNNING' || execution.status.name === 'PAUSED') &&
|
||||
execution.type === 'pentestPipelineWorkflow'
|
||||
) {
|
||||
workflows.push({ workflowId: execution.workflowId, taskQueue: execution.taskQueue });
|
||||
}
|
||||
}
|
||||
return workflows;
|
||||
});
|
||||
}
|
||||
|
||||
/** Describe a scan: status, timing, and the agents currently running (from pendingActivities). Null if not found. */
|
||||
export async function describeScan(workflowId: string): Promise<ScanDescription | null> {
|
||||
const client = await getClient();
|
||||
@@ -66,12 +219,26 @@ export async function describeScan(workflowId: string): Promise<ScanDescription
|
||||
|
||||
const runningAgents: RunningAgent[] = [];
|
||||
for (const pending of desc.raw.pendingActivities ?? []) {
|
||||
const agent = ACTIVITY_TO_AGENT[pending.activityType?.name ?? ''];
|
||||
if (!agent) continue;
|
||||
const lastFailure = pending.lastFailure?.message;
|
||||
const activityType = pending.activityType?.name ?? '';
|
||||
const progress = ACTIVITY_TO_PROGRESS[activityType];
|
||||
// Fail closed: skipping an unknown activity would render a quietly incomplete tree.
|
||||
if (!progress) {
|
||||
throw new ActivityMirrorError(activityType || 'unknown activity');
|
||||
}
|
||||
// Temporal's own failure message is never forwarded verbatim: it can carry raw
|
||||
// exception text from inside the activity, which this client has no way to vet
|
||||
// before painting it into a terminal. Only its presence is kept; the boolean feeds
|
||||
// a fixed sentence downstream (see safeFailureDetail), and the real detail stays
|
||||
// one `shannon logs` away.
|
||||
// NOTE: the proto decoder writes an absent lastFailure as null, not undefined, so a
|
||||
// loose check is what distinguishes a healthy attempt from a failed one.
|
||||
const lastFailure = pending.lastFailure == null ? undefined : 'This activity attempt failed.';
|
||||
const startedAt = timestampMs(pending.scheduledTime ?? pending.lastStartedTime ?? null);
|
||||
runningAgents.push({
|
||||
agent,
|
||||
agent: progress.key,
|
||||
label: progress.label,
|
||||
kind: progress.kind,
|
||||
...(progress.parentKey !== undefined ? { parentKey: progress.parentKey } : {}),
|
||||
attempt: pending.attempt ?? 1,
|
||||
...(startedAt !== undefined ? { startedAt } : {}),
|
||||
...(lastFailure ? { lastFailure } : {}),
|
||||
@@ -153,8 +320,9 @@ export async function waitForWorkflowClose(workflowId: string, opts: WatchOption
|
||||
|
||||
while (!signal?.aborted) {
|
||||
try {
|
||||
const desc = await describeScan(workflowId);
|
||||
if (desc === null || TERMINAL_STATUSES.has(desc.status)) {
|
||||
const client = await getClient();
|
||||
const desc = await client.workflow.getHandle(workflowId).describe();
|
||||
if (TERMINAL_STATUSES.has(desc.status.name)) {
|
||||
return { reason: 'closed' };
|
||||
}
|
||||
// Reachable and still RUNNING — reset the failure streak and note any recovery.
|
||||
@@ -164,6 +332,12 @@ export async function waitForWorkflowClose(workflowId: string, opts: WatchOption
|
||||
}
|
||||
connectFailures = 0;
|
||||
} catch (err) {
|
||||
if (err instanceof WorkflowNotFoundError) {
|
||||
return { reason: 'closed' };
|
||||
}
|
||||
// Drop the wedged channel so the next poll dials a fresh one; a cached dead channel would
|
||||
// otherwise fail every retry identically and never recover.
|
||||
resetClient();
|
||||
connectFailures++;
|
||||
lastError = err instanceof Error ? err.message : String(err);
|
||||
if (!warned && connectFailures >= warnAfterFailures) {
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
/**
|
||||
* Workspace enumeration, default-target resolution, and scan identity proof.
|
||||
*
|
||||
* The action commands (`logs`, `status`, `stop`) each take a workspace name. When one
|
||||
* is omitted, `resolveDefaultWorkspace` picks the obvious candidate — the single running
|
||||
* scan, or the most recent workspace — so the common "I just started one scan, show me
|
||||
* its logs" path doesn't require retyping an auto-generated name. Target selection and
|
||||
* identity proof are separate steps: `resolveScanIdentity` turns a selected or explicit
|
||||
* string into the one canonical (workspace, workflowId) pair the session records prove.
|
||||
*
|
||||
* Running workers are identified by Docker workspace label for default-target selection.
|
||||
* `stop` supplements that local discovery with Temporal lifecycle state. Recency for
|
||||
* finished scans comes from each run's session.json createdAt,
|
||||
* with the workspace directory mtime as the fallback for runs that predate it.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { runningScanWorkspaces } from './docker.js';
|
||||
import { getWorkspacesDir } from './home.js';
|
||||
import { resolveRunFile } from './paths.js';
|
||||
import { resolveWorkflowId } from './session.js';
|
||||
|
||||
export interface WorkspaceInfo {
|
||||
readonly name: string;
|
||||
/** Creation time in ms — the recency sort key. Null when neither session.json nor stat is readable. */
|
||||
readonly createdMs: number | null;
|
||||
}
|
||||
|
||||
/** Creation time of a workspace: session.json createdAt, else directory mtime, else null. */
|
||||
function readCreatedMs(runDir: string): number | null {
|
||||
try {
|
||||
const parsed = JSON.parse(fs.readFileSync(resolveRunFile(runDir, 'session.json'), 'utf-8'));
|
||||
const createdMs = Date.parse(parsed?.session?.createdAt ?? '');
|
||||
if (!Number.isNaN(createdMs)) {
|
||||
return createdMs;
|
||||
}
|
||||
} catch {
|
||||
// Fall through to the directory mtime.
|
||||
}
|
||||
|
||||
try {
|
||||
return fs.statSync(runDir).mtimeMs;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Every workspace directory, newest-first by createdAt (directory mtime fallback). */
|
||||
export function listWorkspaces(): WorkspaceInfo[] {
|
||||
const workspacesDir = getWorkspacesDir();
|
||||
|
||||
let entries: fs.Dirent[];
|
||||
try {
|
||||
entries = fs.readdirSync(workspacesDir, { withFileTypes: true });
|
||||
} catch {
|
||||
// Workspaces directory does not exist yet — no scans have ever run.
|
||||
return [];
|
||||
}
|
||||
|
||||
const workspaces: WorkspaceInfo[] = [];
|
||||
for (const entry of entries) {
|
||||
if (!entry.isDirectory()) {
|
||||
continue;
|
||||
}
|
||||
workspaces.push({ name: entry.name, createdMs: readCreatedMs(path.join(workspacesDir, entry.name)) });
|
||||
}
|
||||
|
||||
// Newest first; workspaces with no known time sort last.
|
||||
workspaces.sort((a, b) => (b.createdMs ?? 0) - (a.createdMs ?? 0));
|
||||
return workspaces;
|
||||
}
|
||||
|
||||
export type ScanIdentity =
|
||||
| { readonly kind: 'ok'; readonly workspace: string; readonly workflowId: string }
|
||||
| {
|
||||
readonly kind: 'not-found';
|
||||
readonly reason: 'no-match' | 'unreadable-record';
|
||||
/** For 'unreadable-record': the session.json path that could not prove the identity. */
|
||||
readonly sessionPath?: string;
|
||||
}
|
||||
| { readonly kind: 'ambiguous'; readonly claims: readonly string[] };
|
||||
|
||||
/** Every workflow id a run's session record has ever claimed: the original plus each resume. */
|
||||
function readRecordedWorkflowIds(runDir: string): readonly string[] {
|
||||
try {
|
||||
const session = JSON.parse(fs.readFileSync(resolveRunFile(runDir, 'session.json'), 'utf-8'));
|
||||
const resumeAttempts: { workflowId?: string }[] = session.session?.resumeAttempts ?? [];
|
||||
const ids = [session.session?.originalWorkflowId, ...resumeAttempts.map((attempt) => attempt.workflowId)];
|
||||
return ids.filter((id): id is string => typeof id === 'string' && id.length > 0);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Prove the canonical (workspace, workflowId) pair for a status target.
|
||||
*
|
||||
* A directory with a readable session record takes precedence and follows its latest
|
||||
* resume (an auto-named directory whose name equals its original workflow id resolves
|
||||
* here). Otherwise the input is matched exactly against every workflow id the session
|
||||
* records have claimed — session.json is the only trustworthy reverse mapping, and a
|
||||
* valid workspace name may itself end in `_shannon-<digits>`, so the id naming
|
||||
* convention is never used to guess.
|
||||
*/
|
||||
export function resolveScanIdentity(input: string): ScanIdentity {
|
||||
const runDir = path.join(getWorkspacesDir(), input);
|
||||
let isDirectory = false;
|
||||
try {
|
||||
isDirectory = fs.statSync(runDir).isDirectory();
|
||||
} catch {
|
||||
// Not a workspace directory — fall through to the exact workflow-id match.
|
||||
}
|
||||
|
||||
if (isDirectory) {
|
||||
const workflowId = resolveWorkflowId(input);
|
||||
if (workflowId !== undefined) {
|
||||
return { kind: 'ok', workspace: input, workflowId };
|
||||
}
|
||||
return { kind: 'not-found', reason: 'unreadable-record', sessionPath: resolveRunFile(runDir, 'session.json') };
|
||||
}
|
||||
|
||||
const claims: string[] = [];
|
||||
for (const workspace of listWorkspaces()) {
|
||||
const recorded = readRecordedWorkflowIds(path.join(getWorkspacesDir(), workspace.name));
|
||||
if (recorded.includes(input)) {
|
||||
claims.push(workspace.name);
|
||||
}
|
||||
}
|
||||
|
||||
if (claims.length === 1) {
|
||||
// The exact requested id is kept, so an older workflow id keeps addressing that older execution.
|
||||
return { kind: 'ok', workspace: claims[0] as string, workflowId: input };
|
||||
}
|
||||
if (claims.length > 1) {
|
||||
return { kind: 'ambiguous', claims: [...claims].sort() };
|
||||
}
|
||||
return { kind: 'not-found', reason: 'no-match' };
|
||||
}
|
||||
|
||||
export type DefaultTarget =
|
||||
| { readonly kind: 'ok'; readonly workspace: string; readonly running: boolean }
|
||||
| { readonly kind: 'none' }
|
||||
| { readonly kind: 'ambiguous'; readonly running: readonly string[] };
|
||||
|
||||
/**
|
||||
* Pick the default workspace when the user gave none.
|
||||
*
|
||||
* Exactly one scan running → that scan. Multiple running → ambiguous, so the caller can
|
||||
* list them and ask for an explicit name. None running → the most recent workspace when
|
||||
* `allowFinished` (viewing commands), otherwise none (stopping a finished scan is a no-op).
|
||||
*/
|
||||
export function resolveDefaultWorkspace(opts: { readonly allowFinished: boolean }): DefaultTarget {
|
||||
const running = runningScanWorkspaces();
|
||||
if (running.length === 1) {
|
||||
return { kind: 'ok', workspace: running[0] as string, running: true };
|
||||
}
|
||||
if (running.length > 1) {
|
||||
return { kind: 'ambiguous', running };
|
||||
}
|
||||
|
||||
if (!opts.allowFinished) {
|
||||
return { kind: 'none' };
|
||||
}
|
||||
|
||||
const workspaces = listWorkspaces();
|
||||
const mostRecent = workspaces[0];
|
||||
if (!mostRecent) {
|
||||
return { kind: 'none' };
|
||||
}
|
||||
return { kind: 'ok', workspace: mostRecent.name, running: false };
|
||||
}
|
||||
@@ -125,16 +125,18 @@
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"vuln_classes": {
|
||||
"type": "array",
|
||||
"description": "Vulnerability classes to test. When omitted, all five classes run. When set, only listed classes run; their vuln+exploit agents and report sections are included.",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"enum": ["injection", "xss", "auth", "authz", "ssrf"]
|
||||
"agentic_sast": {
|
||||
"type": "object",
|
||||
"description": "Opt in to agentic static analysis, which reads the repository for vulnerabilities before the pentest and feeds what it finds into the exploitation phase. Off by default. It does not change which vulnerability classes run. If agentic static analysis fails, the pentest continues without its findings and the scan finishes as \"partial\".",
|
||||
"properties": {
|
||||
"enabled": {
|
||||
"type": "string",
|
||||
"enum": ["true", "false"],
|
||||
"description": "Set to \"true\" to run agentic static analysis. Defaults to \"false\"."
|
||||
}
|
||||
},
|
||||
"minItems": 1,
|
||||
"maxItems": 5,
|
||||
"uniqueItems": true
|
||||
"required": ["enabled"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"exploit": {
|
||||
"type": "string",
|
||||
@@ -193,7 +195,7 @@
|
||||
{ "required": ["rules"] },
|
||||
{ "required": ["authentication", "rules"] },
|
||||
{ "required": ["description"] },
|
||||
{ "required": ["vuln_classes"] },
|
||||
{ "required": ["agentic_sast"] },
|
||||
{ "required": ["exploit"] },
|
||||
{ "required": ["report"] },
|
||||
{ "required": ["rules_of_engagement"] }
|
||||
|
||||
@@ -4,8 +4,14 @@
|
||||
# Description of the target environment (optional, max 500 chars)
|
||||
description: "Next.js e-commerce app on PostgreSQL. Local dev environment — .env files contain local-only credentials, not deployed to production."
|
||||
|
||||
# Limit which vulnerability classes run end-to-end (optional, default: all five)
|
||||
# vuln_classes: [injection, xss, auth, authz, ssrf]
|
||||
# Every scan runs all five vulnerability classes: injection, xss, auth, authz, and ssrf.
|
||||
# There is no setting to narrow that.
|
||||
|
||||
# Agentic static analysis (optional, default: "false").
|
||||
# Reads the repository for vulnerabilities before the pentest and feeds them into exploitation.
|
||||
# It costs extra model time, and if it fails the scan finishes as "partial" without its findings.
|
||||
# agentic_sast:
|
||||
# enabled: "true"
|
||||
|
||||
# Skip the exploitation phase (optional, default: "true")
|
||||
# exploit: "false"
|
||||
|
||||
@@ -10,7 +10,27 @@
|
||||
"./types/agents": "./dist/types/agents.js",
|
||||
"./pipeline": "./dist/temporal/pipeline.js",
|
||||
"./activities": "./dist/temporal/activities.js",
|
||||
"./temporal/reconcile-activity-types": "./dist/temporal/reconcile-activity-types.js",
|
||||
"./services": "./dist/services/index.js",
|
||||
"./services/queue-validation": "./dist/services/queue-validation.js",
|
||||
"./services/renumber-core": "./dist/services/renumber-core.js",
|
||||
"./services/compaction-core": "./dist/services/compaction-core.js",
|
||||
"./services/finding-order": "./dist/services/finding-order.js",
|
||||
"./ai/structured-generation": "./dist/ai/structured-generation.js",
|
||||
"./ai/pi/source-jail": "./dist/ai/pi/source-jail.js",
|
||||
"./ai/reconciliation/contracts": "./dist/ai/reconciliation/contracts.js",
|
||||
"./ai/reconciliation/stage-contracts": "./dist/ai/reconciliation/stage-contracts.js",
|
||||
"./ai/reconciliation/artifact-store": "./dist/ai/reconciliation/artifact-store.js",
|
||||
"./ai/reconciliation/schema-version": "./dist/ai/reconciliation/schema-version.js",
|
||||
"./ai/reconciliation/manifest": "./dist/ai/reconciliation/manifest.js",
|
||||
"./ai/reconciliation/prepare": "./dist/ai/reconciliation/prepare.js",
|
||||
"./ai/reconciliation/enrich": "./dist/ai/reconciliation/enrich.js",
|
||||
"./ai/reconciliation/form": "./dist/ai/reconciliation/form.js",
|
||||
"./ai/reconciliation/materialize": "./dist/ai/reconciliation/materialize.js",
|
||||
"./ai/reconciliation/observation-view": "./dist/ai/reconciliation/observation-view.js",
|
||||
"./ai/reconciliation/labels": "./dist/ai/reconciliation/labels.js",
|
||||
"./ai/reconciliation/submit-validation": "./dist/ai/reconciliation/submit-validation.js",
|
||||
"./ai/reconciliation/refs": "./dist/ai/reconciliation/refs.js",
|
||||
"./config": "./dist/config-parser.js"
|
||||
},
|
||||
"scripts": {
|
||||
@@ -23,13 +43,14 @@
|
||||
"@earendil-works/pi-ai": "^0.84.2",
|
||||
"@earendil-works/pi-coding-agent": "^0.84.2",
|
||||
"@gotgenes/pi-permission-system": "^10.9.0",
|
||||
"@temporalio/activity": "^1.11.0",
|
||||
"@temporalio/client": "^1.11.0",
|
||||
"@temporalio/worker": "^1.11.0",
|
||||
"@temporalio/workflow": "^1.11.0",
|
||||
"@temporalio/activity": "1.15.0",
|
||||
"@temporalio/client": "1.15.0",
|
||||
"@temporalio/worker": "1.15.0",
|
||||
"@temporalio/workflow": "1.15.0",
|
||||
"ajv": "^8.12.0",
|
||||
"ajv-formats": "^2.1.1",
|
||||
"dotenv": "^16.4.5",
|
||||
"handlebars": "^4.7.9",
|
||||
"js-yaml": "^4.1.0",
|
||||
"typebox": "1.1.38",
|
||||
"zx": "^8.0.0"
|
||||
|
||||
@@ -8,7 +8,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e
|
||||
</objective>
|
||||
|
||||
<exploitation_scope>
|
||||
@include(shared/_exploit-scope.txt)
|
||||
@include(shared/exploitation/_exploit-scope.txt)
|
||||
</exploitation_scope>
|
||||
|
||||
<critical>
|
||||
@@ -53,21 +53,23 @@ Remember: An unproven vulnerability is worse than no finding at all - it wastes
|
||||
|
||||
**Level 1 - Authentication Weakness Identified**
|
||||
- Evidence: Theoretical bypass found but not yet exploited
|
||||
- Classification: POTENTIAL (Low Confidence)
|
||||
- Classification: `status: "blocked"`, `confidence: "low"`
|
||||
|
||||
**Level 2 - Partial Bypass Achieved**
|
||||
- Evidence: Access gained to non-critical protected resources or UI elements
|
||||
- Classification: POTENTIAL (Medium Confidence)
|
||||
- Classification: `status: "blocked"`, `confidence: "medium"`
|
||||
|
||||
**Level 3 - Authentication Bypass Confirmed**
|
||||
- Evidence: Successfully accessed protected functionality as unauthorized user
|
||||
- Classification: EXPLOITED
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
**Level 4 - Account Takeover Demonstrated**
|
||||
- Evidence: Complete user impersonation with access to sensitive data/actions
|
||||
- Classification: EXPLOITED (CRITICAL)
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
You MUST reach at least Level 3 with evidence to mark as "EXPLOITED"
|
||||
Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`.
|
||||
|
||||
These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in <severity_reasoning>.
|
||||
|
||||
### Classification Decision Framework
|
||||
|
||||
@@ -231,26 +233,26 @@ Only after exhausting bypass attempts can you proceed to classification analysis
|
||||
## **Validation Criteria & Vulnerability Classification**
|
||||
You must adhere to these rules when making a judgment call and categorizing findings:
|
||||
|
||||
**EXPLOITED (Report in main section):**
|
||||
**`status: "exploited"`:**
|
||||
- Successfully achieved account takeover or authentication bypass with concrete evidence (impersonated user, accessed protected resources)
|
||||
- OR vulnerability confirmed with active exploitation evidence (e.g., "Performed 200 brute force attempts without rate limiting")
|
||||
- Definitive proof of access to a protected resource as another authenticated user OR concrete demonstration of security control bypass
|
||||
- Must have completed weaponization and demonstrated impact through impersonation OR provided evidence of exploitable vulnerability
|
||||
|
||||
POTENTIAL (Report in separate subsection):
|
||||
- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
**`status: "blocked"`:**
|
||||
- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
- **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack**
|
||||
- A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt.
|
||||
- Valid external factors include:
|
||||
- Server instability (e.g., crashes).
|
||||
- Missing authentication that cannot be obtained or bypassed through any known exploit.
|
||||
- This classification also applies when:
|
||||
- This status also applies when:
|
||||
- Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints.
|
||||
- You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors.
|
||||
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.**
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.**
|
||||
|
||||
**FALSE POSITIVE (Do NOT report):**
|
||||
**FALSE POSITIVE (never submitted through the tool):**
|
||||
- **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways**
|
||||
- Definitively proven not exploitable through exhaustive testing
|
||||
- Documented attack plan showing all weaponization attempts failed
|
||||
@@ -260,7 +262,7 @@ POTENTIAL (Report in separate subsection):
|
||||
|
||||
**Important:** Rate limiting and account lockouts are obstacles to work around, not grounds for immediate false positive classification. Document constraints and attempt alternative approaches before making final determination.
|
||||
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop authentication attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL.
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop authentication attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`.
|
||||
|
||||
### CRITICAL: REPRODUCIBILITY REQUIREMENTS
|
||||
|
||||
@@ -304,12 +306,18 @@ Prioritize these based on the intelligence from your queue and vulnerability ana
|
||||
</attack_patterns>
|
||||
</methodology_and_domain_expertise>
|
||||
|
||||
@include(shared/exploitation/_severity-reasoning.txt)
|
||||
|
||||
@include(shared/exploitation/_reporting-standards.txt)
|
||||
|
||||
@include(shared/exploitation/_credentials-in-findings.txt)
|
||||
|
||||
<deliverable_tools>
|
||||
You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/auth_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly.
|
||||
|
||||
**When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission.
|
||||
|
||||
**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (account takeover demonstrated, session hijacked end-to-end, password reset abused, MFA bypassed). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/auth_false_positives.md` tracking file, not the deliverable.
|
||||
**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (account takeover demonstrated, session hijacked end-to-end, password reset abused, MFA bypassed) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/auth_false_positives.md`, never through this tool.
|
||||
|
||||
**ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/auth_exploitation_queue.json` exactly (e.g. `AUTH-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry.
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e
|
||||
</objective>
|
||||
|
||||
<exploitation_scope>
|
||||
@include(shared/_exploit-scope.txt)
|
||||
@include(shared/exploitation/_exploit-scope.txt)
|
||||
</exploitation_scope>
|
||||
|
||||
<critical>
|
||||
@@ -53,21 +53,23 @@ Remember: An unproven vulnerability is worse than no finding at all - it wastes
|
||||
|
||||
**Level 1 - Authorization Weakness Identified**
|
||||
- Evidence: Theoretical bypass found but not yet exploited
|
||||
- Classification: POTENTIAL (Low Confidence)
|
||||
- Classification: `status: "blocked"`, `confidence: "low"`
|
||||
|
||||
**Level 2 - Partial Access Control Bypass**
|
||||
- Evidence: Access gained to some protected resources or partial privilege escalation
|
||||
- Classification: POTENTIAL (Medium Confidence)
|
||||
- Classification: `status: "blocked"`, `confidence: "medium"`
|
||||
|
||||
**Level 3 - Access Control Bypass Confirmed**
|
||||
- Evidence: Successfully accessed protected functionality or data as unauthorized user
|
||||
- Classification: EXPLOITED
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
**Level 4 - Critical Privilege Escalation Demonstrated**
|
||||
- Evidence: Complete privilege escalation with access to administrative functions or highly sensitive data
|
||||
- Classification: EXPLOITED (CRITICAL)
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
You MUST reach at least Level 3 with evidence to mark as "EXPLOITED"
|
||||
Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`.
|
||||
|
||||
These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in <severity_reasoning>.
|
||||
|
||||
### Classification Decision Framework
|
||||
|
||||
@@ -230,26 +232,26 @@ Only after exhausting bypass attempts can you proceed to classification analysis
|
||||
## **Validation Criteria & Vulnerability Classification**
|
||||
You must adhere to these rules when making a judgment call and categorizing findings:
|
||||
|
||||
**EXPLOITED (Report in main section):**
|
||||
**`status: "exploited"`:**
|
||||
- Successfully achieved privilege escalation or unauthorized access with concrete evidence (accessed protected data, modified unauthorized resources)
|
||||
- Definitive proof of access to resources that should be restricted based on role or ownership
|
||||
- Must have completed weaponization and demonstrated impact through unauthorized access
|
||||
- MUST have reached Level 3 or 4 with evidence
|
||||
|
||||
POTENTIAL (Report in separate subsection):
|
||||
- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
**`status: "blocked"`:**
|
||||
- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
- **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack**
|
||||
- A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt.
|
||||
- Valid external factors include:
|
||||
- Server instability (e.g., crashes).
|
||||
- Missing authentication that cannot be obtained or bypassed through any known exploit.
|
||||
- This classification also applies when:
|
||||
- This status also applies when:
|
||||
- Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints.
|
||||
- You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors.
|
||||
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.**
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.**
|
||||
|
||||
**FALSE POSITIVE (Do NOT report):**
|
||||
**FALSE POSITIVE (never submitted through the tool):**
|
||||
- **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways**
|
||||
- Definitively proven not exploitable through exhaustive testing
|
||||
- Documented attack plan showing all weaponization attempts failed
|
||||
@@ -259,7 +261,7 @@ POTENTIAL (Report in separate subsection):
|
||||
|
||||
**Important:** Authentication requirements and access restrictions are obstacles to work around, not grounds for immediate false positive classification. Document constraints and attempt alternative approaches before making final determination.
|
||||
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop authorization attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL.
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop authorization attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`.
|
||||
|
||||
### CRITICAL: REPRODUCIBILITY REQUIREMENTS
|
||||
|
||||
@@ -312,12 +314,18 @@ Remember: The most effective attacks often come from understanding the specific
|
||||
</attack_patterns>
|
||||
</methodology_and_domain_expertise>
|
||||
|
||||
@include(shared/exploitation/_severity-reasoning.txt)
|
||||
|
||||
@include(shared/exploitation/_reporting-standards.txt)
|
||||
|
||||
@include(shared/exploitation/_credentials-in-findings.txt)
|
||||
|
||||
<deliverable_tools>
|
||||
You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/authz_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly.
|
||||
|
||||
**When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission.
|
||||
|
||||
**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (cross-tenant data access demonstrated, privilege escalation proven, unauthorized workflow transition executed). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/authz_false_positives.md` tracking file, not the deliverable.
|
||||
**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (cross-tenant data access demonstrated, privilege escalation proven, unauthorized workflow transition executed) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/authz_false_positives.md`, never through this tool.
|
||||
|
||||
**ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/authz_exploitation_queue.json` exactly (e.g. `AUTHZ-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry.
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e
|
||||
</objective>
|
||||
|
||||
<exploitation_scope>
|
||||
@include(shared/_exploit-scope.txt)
|
||||
@include(shared/exploitation/_exploit-scope.txt)
|
||||
</exploitation_scope>
|
||||
|
||||
<critical>
|
||||
@@ -210,21 +210,23 @@ Success Criteria: Print status code and response excerpt for each payload so I c
|
||||
|
||||
**Level 1 - Injection Point Confirmed**
|
||||
- Evidence: Error messages, timing differences, or response variations
|
||||
- Classification: POTENTIAL (Low Confidence)
|
||||
- Classification: `status: "blocked"`, `confidence: "low"`
|
||||
|
||||
**Level 2 - Query Structure Manipulated**
|
||||
- Evidence: Boolean-based blind working, UNION SELECT succeeds, ORDER BY confirms columns
|
||||
- Classification: POTENTIAL (Medium Confidence)
|
||||
- Classification: `status: "blocked"`, `confidence: "medium"`
|
||||
|
||||
**Level 3 - Data Extraction Proven**
|
||||
- Evidence: Actual data retrieved from database (usernames, table names, etc.)
|
||||
- Classification: EXPLOITED
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
**Level 4 - Critical Impact Demonstrated**
|
||||
- Evidence: Admin credentials extracted, sensitive data dumped, or system commands executed
|
||||
- Classification: EXPLOITED (CRITICAL)
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
You MUST reach at least Level 3 with evidence to mark as "EXPLOITED"
|
||||
Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`.
|
||||
|
||||
These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in <severity_reasoning>.
|
||||
|
||||
### Classification Decision Framework
|
||||
|
||||
@@ -271,26 +273,26 @@ For each **successfully exploited** vulnerability, your goal is to collect the f
|
||||
## **Validation Criteria & Vulnerability Classification**
|
||||
You must adhere to these rules when making a judgment call and categorizing findings:
|
||||
|
||||
**EXPLOITED (Report in main section):**
|
||||
**`status: "exploited"`:**
|
||||
- Successfully executed exploit with concrete evidence (data extracted, unauthorized access achieved)
|
||||
- MUST have reached at least Level 3 with actual data extraction
|
||||
- Consistent error messages, predictable differential responses to boolean tests, or measurable time delays
|
||||
- Must have completed the full OWASP workflow with tangible proof
|
||||
|
||||
POTENTIAL (Report in separate subsection):
|
||||
- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
|
||||
**`status: "blocked"`:**
|
||||
- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
- **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack**
|
||||
- A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt.
|
||||
- Valid external factors include:
|
||||
- Server instability (e.g., crashes).
|
||||
- Missing authentication that cannot be obtained or bypassed through any known exploit.
|
||||
- This classification also applies when:
|
||||
- This status also applies when:
|
||||
- Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints.
|
||||
- You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors.
|
||||
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.**
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.**
|
||||
|
||||
**FALSE POSITIVE (Do NOT report):**
|
||||
**FALSE POSITIVE (never submitted through the tool):**
|
||||
- **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways**
|
||||
- Definitively proven not exploitable through systematic testing with multiple techniques
|
||||
- Multiple bypass attempts all failed with no indication of underlying vulnerability
|
||||
@@ -300,7 +302,7 @@ POTENTIAL (Report in separate subsection):
|
||||
|
||||
**Important:** WAF blocking is an obstacle to overcome, not grounds for immediate classification as false positive. Exhaust bypass techniques before making final determination.
|
||||
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop injection attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL.
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop injection attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`.
|
||||
|
||||
### CRITICAL: REPRODUCIBILITY REQUIREMENTS
|
||||
|
||||
@@ -335,12 +337,18 @@ Prioritize your techniques based on the intelligence from the analysis deliverab
|
||||
|
||||
</methodology_and_domain_expertise>
|
||||
|
||||
@include(shared/exploitation/_severity-reasoning.txt)
|
||||
|
||||
@include(shared/exploitation/_reporting-standards.txt)
|
||||
|
||||
@include(shared/exploitation/_credentials-in-findings.txt)
|
||||
|
||||
<deliverable_tools>
|
||||
You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/injection_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly.
|
||||
|
||||
**When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission.
|
||||
|
||||
**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (data extracted, JavaScript executed, account taken over, internal service accessed). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/injection_false_positives.md` tracking file, not the deliverable.
|
||||
**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (data extracted, JavaScript executed, account taken over, internal service accessed) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/injection_false_positives.md`, never through this tool.
|
||||
|
||||
**ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/injection_exploitation_queue.json` exactly (e.g. `INJ-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry.
|
||||
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
<role>
|
||||
You are a generalist exploitation specialist for security weaknesses that do not belong to Shannon's five analysis-backed classes. You are an active penetration tester, not a vulnerability analyst. Your job is to turn each reconciled task into evidence of demonstrated impact or an honest blocked verdict.
|
||||
</role>
|
||||
|
||||
<objective>
|
||||
Process every task in `.shannon/deliverables/miscellaneous_exploitation_queue.json`. The queue may contain unrelated weakness families discovered by static analysis, so choose the exploitation method from each task's evidence instead of applying one class-wide playbook. Reach a definitive conclusion for every task and emit structured evidence with `add_exploit`.
|
||||
</objective>
|
||||
|
||||
<exploitation_scope>
|
||||
@include(shared/_exploit-scope.txt)
|
||||
</exploitation_scope>
|
||||
|
||||
<critical>
|
||||
### CRITICAL: EXPLOITATION TARGET REQUIREMENT
|
||||
|
||||
All dynamic exploitation must be performed against the specified target: {{WEB_URL}}
|
||||
|
||||
An exploit is an unauthorized or unintended action demonstrated against the target application. Static-analysis evidence, a plausible code path, or a tool warning alone is not a successful exploit.
|
||||
|
||||
- Claims require concrete, reproducible evidence.
|
||||
- Validate the task's assumptions before building on them.
|
||||
- Use `status: "exploited"` only after demonstrating impact.
|
||||
- Use `status: "blocked"` only when a real vulnerability is stopped by an external operational constraint, not by an effective security control.
|
||||
- Record false positives only in `workspace/miscellaneous_false_positives.md`; do not submit them through `add_exploit`.
|
||||
- Never test production systems. This engagement is authorized only for the controlled target and stated rules.
|
||||
</critical>
|
||||
|
||||
<target>
|
||||
@include(shared/_target.txt)
|
||||
</target>
|
||||
|
||||
<rules>
|
||||
@include(shared/_rules.txt)
|
||||
</rules>
|
||||
|
||||
@include(shared/_shared-session.txt)
|
||||
|
||||
<login_instructions>
|
||||
{{LOGIN_INSTRUCTIONS}}
|
||||
</login_instructions>
|
||||
|
||||
@include(shared/_rules-of-engagement.txt)
|
||||
|
||||
<starting_context>
|
||||
Your actionable queue is `.shannon/deliverables/miscellaneous_exploitation_queue.json`. Its IDs are stable task references such as `MISC-01`. Process every queue entry exactly once.
|
||||
|
||||
Read these inputs before testing:
|
||||
1. `.shannon/deliverables/pre_recon_deliverable.md` for architecture and source layout.
|
||||
2. `.shannon/deliverables/recon_deliverable.md` for the live attack surface.
|
||||
3. `.shannon/deliverables/miscellaneous_exploitation_queue.json` for the reconciled tasks and their SAST evidence.
|
||||
|
||||
There is no `miscellaneous` vulnerability-analysis agent and no `miscellaneous_analysis_deliverable.md`. Do not look for one or imply that one ran. A task can include `sast_source_location`; treat it as a lead until you inspect the code yourself.
|
||||
|
||||
Use `todo_write` to create and track one task per queue entry.
|
||||
</starting_context>
|
||||
|
||||
<system_architecture>
|
||||
**Phase sequence:** RECONNAISSANCE → SAST RECONCILIATION → **MISCELLANEOUS EXPLOITATION (YOU)** → FINAL REPORT
|
||||
|
||||
**Input:** `.shannon/deliverables/miscellaneous_exploitation_queue.json`
|
||||
**Output:** `.shannon/deliverables/miscellaneous_exploitation_evidence.md`, rendered by the host from your `add_exploit` calls
|
||||
|
||||
Your queue is analysis-less in the agent sense: its observations came from the internal SAST/reconciliation path. Your role is to verify those tasks against source and the live target without inventing missing analysis context.
|
||||
</system_architecture>
|
||||
|
||||
<cli_tools>
|
||||
- **Browser Automation (playwright-cli skill):** Use when the task requires browser interactions. Always pass `-s={{PLAYWRIGHT_SESSION}}`.
|
||||
- **`bash` tool:** Use for focused commands and reproducible HTTP requests.
|
||||
- **`task` agent:** Use for custom scripts, payload loops, or repetitive testing.
|
||||
- **`todo_write` tool:** Track every queue task and its final verdict.
|
||||
- **`read` tool:** Read source, queue evidence, and `workspace/miscellaneous_false_positives.md`.
|
||||
</cli_tools>
|
||||
|
||||
<methodology>
|
||||
For each `MISC-NN` task:
|
||||
|
||||
1. Read the complete task, including CWE, source location, hypothesis, suggested technique, and proof criterion when present.
|
||||
2. Inspect the cited code and trace the relevant input, guard, and effect. Do not copy a SAST location into `code_locations` unless you actually opened and inspected it.
|
||||
3. Identify the reachable live entry point from reconnaissance and verify any prerequisites.
|
||||
4. Attempt the weakness-specific exploit method. Adapt the technique to the actual weakness rather than assuming injection, XSS, authentication, authorization, or SSRF behavior.
|
||||
5. Pursue concrete impact. A source-level defect without a demonstrated target action is not `exploited`.
|
||||
6. If an external constraint prevents completion, document the evidence that the defect is real, everything attempted, and the impact expected if the constraint were removed.
|
||||
7. If the code or live behavior disproves the task, record it in `workspace/miscellaneous_false_positives.md` and do not call `add_exploit` for it.
|
||||
8. Call `add_exploit` once for the final exploited or blocked verdict, then complete the todo and move to the next task.
|
||||
|
||||
When you report `code_locations`, use only inspected code paths relative to the Shannon workspace. Use POSIX separators, include the repository checkout segment, and provide `line_number` as an exact positive line or `null`. Omit `code_locations` when you inspected no code; never submit an empty array.
|
||||
</methodology>
|
||||
|
||||
<deliverable_tools>
|
||||
The `add_exploit` tool is your only evidence output. The host renders the Markdown deliverable after your run.
|
||||
|
||||
- `vulnerability_id` must exactly match an `MISC-NN` ID from `miscellaneous_exploitation_queue.json`.
|
||||
- Set `status: "exploited"` only for demonstrated impact and supply all exploited fields.
|
||||
- Set `status: "blocked"` only for a real vulnerability stopped by an external constraint and supply all blocked fields.
|
||||
- Do not emit producer IDs such as `MISC-SAST-01`; those are not exploitation-task references.
|
||||
- Duplicate or out-of-queue IDs are rejected. Reach a final verdict before emitting.
|
||||
- Do not call `add_exploit` for false positives.
|
||||
</deliverable_tools>
|
||||
|
||||
<conclusion_trigger>
|
||||
Before finishing, verify that every task in `miscellaneous_exploitation_queue.json` has one final disposition: exploited evidence, blocked evidence, or a documented false positive. Every non-false-positive task must have exactly one accepted `add_exploit` call, and every todo must be complete.
|
||||
|
||||
Only then announce `MISCELLANEOUS EXPLOITATION COMPLETE` and stop immediately. Do not add a prose recap; the structured deliverable is the record.
|
||||
</conclusion_trigger>
|
||||
@@ -8,7 +8,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e
|
||||
</objective>
|
||||
|
||||
<exploitation_scope>
|
||||
@include(shared/_exploit-scope.txt)
|
||||
@include(shared/exploitation/_exploit-scope.txt)
|
||||
</exploitation_scope>
|
||||
|
||||
<critical>
|
||||
@@ -53,21 +53,23 @@ Remember: An unproven vulnerability is worse than no finding at all - it wastes
|
||||
|
||||
**Level 1 - SSRF Weakness Identified**
|
||||
- Evidence: Theoretical access found but not yet exploited
|
||||
- Classification: POTENTIAL (Low Confidence)
|
||||
- Classification: `status: "blocked"`, `confidence: "low"`
|
||||
|
||||
**Level 2 - Partial Network Access**
|
||||
- Evidence: Successfully triggered outbound requests or received responses indicating internal network activity
|
||||
- Classification: POTENTIAL (Medium Confidence)
|
||||
- Classification: `status: "blocked"`, `confidence: "medium"`
|
||||
|
||||
**Level 3 - Internal Service Access Confirmed**
|
||||
- Evidence: Successfully accessed internal services, retrieved internal data, or confirmed network boundary bypass
|
||||
- Classification: EXPLOITED
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
**Level 4 - Critical Information Disclosure**
|
||||
- Evidence: Retrieved sensitive data from internal services, cloud metadata, or performed successful privilege escalation
|
||||
- Classification: EXPLOITED (CRITICAL)
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
You MUST reach at least Level 3 with evidence to mark as "EXPLOITED"
|
||||
Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`.
|
||||
|
||||
These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in <severity_reasoning>.
|
||||
|
||||
### Classification Decision Framework
|
||||
|
||||
@@ -221,26 +223,26 @@ Only after exhausting bypass attempts can you proceed to classification analysis
|
||||
## **Validation Criteria & Vulnerability Classification**
|
||||
You must adhere to these rules when making a judgment call and categorizing findings:
|
||||
|
||||
**EXPLOITED (Report in main section):**
|
||||
**`status: "exploited"`:**
|
||||
- Successfully achieved internal service access, cloud metadata retrieval, or network reconnaissance with concrete evidence
|
||||
- Definitive proof of accessing resources that should not be reachable from external networks
|
||||
- Must have completed confirmation and demonstrated meaningful impact
|
||||
- MUST have reached Level 3 or 4 with evidence
|
||||
|
||||
POTENTIAL (Report in separate subsection):
|
||||
- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
**`status: "blocked"`:**
|
||||
- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
- **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack**
|
||||
- A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt.
|
||||
- Valid external factors include:
|
||||
- Server instability (e.g., crashes).
|
||||
- Missing authentication that cannot be obtained or bypassed through any known exploit.
|
||||
- This classification also applies when:
|
||||
- This status also applies when:
|
||||
- Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints.
|
||||
- You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors.
|
||||
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.**
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.**
|
||||
|
||||
**FALSE POSITIVE (Do NOT report):**
|
||||
**FALSE POSITIVE (never submitted through the tool):**
|
||||
- **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways**
|
||||
- Definitively proven not exploitable through exhaustive testing
|
||||
- Documented attack plan showing all exploitation attempts failed
|
||||
@@ -248,9 +250,9 @@ POTENTIAL (Report in separate subsection):
|
||||
- **Documentation:** Record false positives in your tracking file `workspace/ssrf_false_positives.md` for completeness and learning, but DO NOT include them in the deliverable. Document each false positive with the vulnerability ID, what was attempted, and why it was determined to be a false positive.
|
||||
- Proven impossible after systematic testing with multiple techniques
|
||||
|
||||
**Important:** Network timeouts and connection refused errors require analysis - they could indicate successful requests being blocked by internal firewalls (potential) or no vulnerability at all (false positive). Test multiple internal targets to distinguish.
|
||||
**Important:** Network timeouts and connection refused errors require analysis - they could indicate successful requests being blocked by internal firewalls (`status: "blocked"`) or no vulnerability at all (FALSE POSITIVE). Test multiple internal targets to distinguish.
|
||||
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop SSRF attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL.
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop SSRF attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`.
|
||||
|
||||
### CRITICAL: REPRODUCIBILITY REQUIREMENTS
|
||||
|
||||
@@ -385,16 +387,22 @@ Prioritize these based on the intelligence from your queue and vulnerability ana
|
||||
|
||||
A successful SSRF doesn't always mean data is immediately exfiltrated. Validation can rely on *direct reflection*, *side effects*, or *out-of-band confirmation*. Always adapt your verification to the SSRF type.
|
||||
|
||||
**Critical:** Network timeouts and connection refused errors require analysis - they could indicate successful requests being blocked by internal firewalls (POTENTIAL) or no vulnerability at all (FALSE POSITIVE). Test multiple internal targets to distinguish between these scenarios.
|
||||
**Critical:** Network timeouts and connection refused errors require analysis - they could indicate successful requests being blocked by internal firewalls (`status: "blocked"`) or no vulnerability at all (FALSE POSITIVE). Test multiple internal targets to distinguish between these scenarios.
|
||||
</attack_patterns>
|
||||
</methodology_and_domain_expertise>
|
||||
|
||||
@include(shared/exploitation/_severity-reasoning.txt)
|
||||
|
||||
@include(shared/exploitation/_reporting-standards.txt)
|
||||
|
||||
@include(shared/exploitation/_credentials-in-findings.txt)
|
||||
|
||||
<deliverable_tools>
|
||||
You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/ssrf_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly.
|
||||
|
||||
**When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission.
|
||||
|
||||
**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (internal service contents retrieved, cloud metadata extracted, port scan results captured, webhook abuse demonstrated). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/ssrf_false_positives.md` tracking file, not the deliverable.
|
||||
**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (internal service contents retrieved, cloud metadata extracted, port scan results captured, webhook abuse demonstrated) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/ssrf_false_positives.md`, never through this tool.
|
||||
|
||||
**ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/ssrf_exploitation_queue.json` exactly (e.g. `SSRF-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry.
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ Success criterion: A comprehensive evidence report detailing the outcome for **e
|
||||
</objective>
|
||||
|
||||
<exploitation_scope>
|
||||
@include(shared/_exploit-scope.txt)
|
||||
@include(shared/exploitation/_exploit-scope.txt)
|
||||
</exploitation_scope>
|
||||
|
||||
<critical>
|
||||
@@ -53,17 +53,19 @@ An exploit is executing an unauthorized or unintended action against the target
|
||||
|
||||
**Level 2 - Payload Injected But Blocked**
|
||||
- Evidence: Payload in HTML but JavaScript execution prevented by filters/CSP
|
||||
- Classification: POTENTIAL (requires bypass)
|
||||
- Classification: `status: "blocked"` — only if a non-security constraint stopped the bypass; a filter or CSP that held is a FALSE POSITIVE
|
||||
|
||||
**Level 3 - JavaScript Execution Confirmed**
|
||||
- Evidence: JavaScript executes (alert, console.log, DOM manipulation)
|
||||
- Classification: EXPLOITED
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
**Level 4 - Critical Impact Demonstrated**
|
||||
- Evidence: Session hijacking, data theft, unauthorized actions performed
|
||||
- Classification: EXPLOITED (CRITICAL)
|
||||
- Classification: `status: "exploited"`
|
||||
|
||||
You MUST reach at least Level 3 with evidence to mark as "EXPLOITED"
|
||||
Level 3 is the threshold for claiming demonstrated impact and rating the finding `low` or above. Reaching Level 1 or 2 and being stopped by an external operational constraint is `status: "blocked"`.
|
||||
|
||||
These levels measure how conclusively you proved the finding is real. They do not set its severity — a finding proven at Level 4 may still be Low. Severity is decided separately, in <severity_reasoning>.
|
||||
|
||||
### Classification Decision Framework
|
||||
|
||||
@@ -257,26 +259,26 @@ For each successfully exploited vulnerability, aim to demonstrate meaningful imp
|
||||
## **Validation Criteria & Vulnerability Classification**
|
||||
You must adhere to these rules when making a judgment call and categorizing findings:
|
||||
|
||||
**EXPLOITED (Report in main section):**
|
||||
**`status: "exploited"`:**
|
||||
- Successfully executed script in browser with concrete evidence (session hijacked, unauthorized actions performed)
|
||||
- Reproducible script execution with tangible impact achieved
|
||||
- Must have completed the full OWASP workflow with proof of compromise
|
||||
- MUST have reached Level 3 or 4 with evidence
|
||||
|
||||
POTENTIAL (Report in separate subsection):
|
||||
- This classification is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
**`status: "blocked"`:**
|
||||
- This status is exclusively for vulnerabilities where a definitive conclusion is impossible due to external factors beyond your control.
|
||||
- **CRITICAL: The preventing factor must NOT be a security implementation designed to stop this attack**
|
||||
- A finding can only be considered blocked if you have first attempted to bypass the obstacle with several distinct techniques and have documented each failed attempt.
|
||||
- Valid external factors include:
|
||||
- Server instability (e.g., crashes).
|
||||
- Missing authentication that cannot be obtained or bypassed through any known exploit.
|
||||
- This classification also applies when:
|
||||
- This status also applies when:
|
||||
- Code analysis suggests a vulnerability, but live testing is inconclusive due to unmovable constraints.
|
||||
- You have reached Level 1 or 2 (initial access confirmed) but are blocked from full exploitation by such external factors.
|
||||
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not POTENTIAL. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified POTENTIAL.**
|
||||
**WARNING: When security measures successfully prevent exploitation after exhaustive testing, it is ok to classify it as a FALSE POSITIVE, not `blocked`. A FALSE POSITIVE that's honestly classified is more valuable than a misclassified `blocked` finding.**
|
||||
|
||||
**FALSE POSITIVE (Do NOT report):**
|
||||
**FALSE POSITIVE (never submitted through the tool):**
|
||||
- **The blocking mechanism IS a security implementation designed to prevent this attack AND you have attempted to bypass it multiple ways**
|
||||
- Definitively proven not exploitable through systematic testing with multiple bypass techniques
|
||||
- Multiple bypass attempts all failed including different payloads, encoding techniques, and CSP bypass attempts
|
||||
@@ -285,7 +287,7 @@ POTENTIAL (Report in separate subsection):
|
||||
|
||||
**Important:** WAF/CSP blocking is an obstacle to overcome, not grounds for immediate classification as false positive. Document and attempt bypasses before making final determination.
|
||||
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop XSS attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = POTENTIAL.
|
||||
**Critical Decision Test**: After exhaustive bypass attempts, ask "Is this prevention a security feature designed to stop XSS attacks, or an external operational constraint?" Security features that withstand bypass attempts = FALSE POSITIVE. External constraints = `blocked`.
|
||||
|
||||
### CRITICAL: REPRODUCIBILITY REQUIREMENTS
|
||||
|
||||
@@ -322,12 +324,18 @@ POTENTIAL (Report in separate subsection):
|
||||
|
||||
</methodology_and_domain_expertise>
|
||||
|
||||
@include(shared/exploitation/_severity-reasoning.txt)
|
||||
|
||||
@include(shared/exploitation/_reporting-standards.txt)
|
||||
|
||||
@include(shared/exploitation/_credentials-in-findings.txt)
|
||||
|
||||
<deliverable_tools>
|
||||
You emit your exploitation evidence through a single tool — `add_exploit`. The host renderer assembles `.shannon/deliverables/xss_exploitation_evidence.md` from your tool calls after the run. You do NOT write the Markdown file directly.
|
||||
|
||||
**When to emit.** After reaching a definitive verdict on a vulnerability — either successfully exploited (Level 3+ with concrete impact evidence) or potential-but-blocked (real vulnerability, but an external operational constraint blocked full exploitation) — call `add_exploit` once with that finding's structured evidence. Call once per queue vulnerability; do not batch. Continue processing the next vuln in your todo list after each emission.
|
||||
|
||||
**Status discriminator.** Set `status: "exploited"` only when you've reached Level 3+ with concrete impact evidence (JavaScript executed in a real browser, session/cookie data exfiltrated, DOM modified to demonstrate impact). Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. See the Classification Decision Framework in this prompt. Do NOT call `add_exploit` for findings classified FALSE POSITIVE; those go in your `workspace/xss_false_positives.md` tracking file, not the deliverable.
|
||||
**Status, for this class.** `status: "exploited"` means your own testing settled the question, and it carries a `severity`. Level 3+ with concrete impact evidence (JavaScript executed in a real browser, session/cookie data exfiltrated, DOM modified to demonstrate impact) is what you need in order to claim demonstrated impact and rate the finding `low` or above. Set `status: "blocked"` only for findings that are real vulnerabilities but where external factors — NOT security defenses — prevented full exploitation. False positives are recorded in `workspace/xss_false_positives.md`, never through this tool.
|
||||
|
||||
**ID alignment.** `vulnerability_id` must match an ID from `.shannon/deliverables/xss_exploitation_queue.json` exactly (e.g. `XSS-VULN-03`). The collector will reject IDs not in the queue with a list of valid IDs; if you get that error, you either typo'd an ID or imagined one — fix and retry.
|
||||
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Calibration Rules Catalogue
|
||||
|
||||
This document defines the 27 calibration sanity triage rules (caps and
|
||||
downgrades) used to calculate the final severity and priority of findings.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Core Principle: Marginal Capability](#core-principle-marginal-capability)
|
||||
- [Category A: Force-Downgrade to LOW (Cap at 2.0 / LOW Priority)](#category-a-force-downgrade-to-low-cap-at-20--low-priority)
|
||||
- [Category B: Force-Cap to HIGH (Cap at 7.9 / Maximum HIGH Priority)](#category-b-force-cap-to-high-cap-at-79--maximum-high-priority)
|
||||
- [Category C: Force-Cap to MEDIUM (Cap at 5.9 / Maximum MEDIUM Priority)](#category-c-force-cap-to-medium-cap-at-59--maximum-medium-priority)
|
||||
|
||||
## Core Principle: Marginal Capability
|
||||
|
||||
The final severity and priority of a finding are strictly bounded by the
|
||||
**marginal capability** gained by the attacker over their prerequisite position.
|
||||
If the exploit does not grant the attacker significant new control, access, or
|
||||
capabilities beyond what is already inherent to their starting position (or
|
||||
already possessed via legitimate means), the finding must be capped or
|
||||
downgraded.
|
||||
|
||||
______________________________________________________________________
|
||||
|
||||
### Category A: Force-Downgrade to LOW (Cap at 2.0 / LOW Priority)
|
||||
|
||||
01. **`repro_failure` (Reproduction Failure or Not Attempted)** The reproduction
|
||||
failed (`repro_status: "failed_to_reproduce"`), was not attempted
|
||||
(`repro_status: "not_attempted"`), or the `repro_status` field was missing
|
||||
(treated as `"not_attempted"`), regardless of theoretical production
|
||||
viability.
|
||||
|
||||
|
||||
02. **`unreachable_inputs` (Unreachable / Uncontrolled Inputs)** The finding
|
||||
relies on inputs that are documented as highly unlikely to be
|
||||
user-controlled, and no path from a trust boundary is proven.
|
||||
|
||||
03. **`third_party_reachability` (Third-Party / Supply Chain Reachability)**
|
||||
Vulnerabilities in third-party libraries (dependency CVEs) where a reachable
|
||||
path from application input to the vulnerable function has not been actively
|
||||
demonstrated.
|
||||
|
||||
04. **`minor_config_hygiene` (Minor Configuration Hygiene)** Minor deviations
|
||||
from best practice (e.g., slightly loose permissions on internal dirs, lack
|
||||
of modern encryption on low-value internal transport) without a clear
|
||||
exploit path.
|
||||
|
||||
05. **`non_security_critical` (Non-Security Critical Components)** The finding
|
||||
affects a component or data with no security sensitivity (e.g., public info,
|
||||
signatures on non-security payloads, cosmetic outputs).
|
||||
|
||||
06. **`vague_code_paths` (Vague Code Paths / Fragile Assumptions)** Relying on
|
||||
unverified assumptions about caller behavior or adjacent system components.
|
||||
|
||||
|
||||
07. **`unreliable_triggers` (Unreliable/Noisy Triggers)** Triggers that are
|
||||
likely to be ignored in practice or indistinguishable from normal
|
||||
operations.
|
||||
|
||||
08. **`prerequisite_shell` (Prerequisite Shell Access / Equivalent Primitives)**
|
||||
The attacker already possesses local shell access on the target container or
|
||||
host with the **same or higher** privilege level than the exploit provides,
|
||||
rendering the gained access redundant under the Principle of Marginal
|
||||
Capability (e.g., exploiting a bug to get a standard user shell when already
|
||||
logged in as a standard user, or exploiting a local buffer overflow to run
|
||||
commands as root when already running as root). This does NOT apply to
|
||||
low-to-high privilege escalation (e.g., standard user to root), which should
|
||||
cap at MEDIUM.
|
||||
|
||||
09. **`physical_long_term` (Physical Long-Term / Laboratory Access)** If the
|
||||
attack requires long-term physical access to the device or specialized
|
||||
laboratory equipment (e.g., fault injection, side-channel analysis, chip
|
||||
decapping). Force-downgrade to **LOW (2.0)** due to the extreme execution
|
||||
barrier and requirement for physical possession.
|
||||
|
||||
10. **`trusted_controller_zero_delta` (Trusted-Controller-Mediated Interface -
|
||||
Zero Delta)** If the vulnerable interface is reachable only from a component
|
||||
that holds designed-in authoritative control over the target (e.g.,
|
||||
orchestrator->worker, driver->device firmware, protocol master->slave,
|
||||
hypervisor->guest, management plane->data plane node), and the exploit
|
||||
grants **zero marginal capability** (i.e., the controller could already
|
||||
achieve the identical effect or level of compromise via its standard,
|
||||
legitimate interface), force-downgrade to **LOW (2.0)**. (This generalizes
|
||||
the *Standard Host-to-Guest Attacks* rule below).
|
||||
|
||||
11. **`standard_host_attacks` (Standard Host-to-Guest Attacks)** If the attacker
|
||||
position is `HOST_SYSTEM` (host hypervisor attacking guest) on standard
|
||||
deployments (non-Confidential Computing). Force-downgrade to **LOW (2.0)**
|
||||
as the host OS/hypervisor already possesses total control over the guest by
|
||||
design, meaning the exploit offers zero marginal capability over the
|
||||
prerequisite position (equivalent primitives). **Default assumption:** treat
|
||||
as non-Confidential Computing (this rule fires) UNLESS the Threat Model,
|
||||
code path, or finding description explicitly names Confidential Computing,
|
||||
guest enclaves, TEE, SEV, TDX, SGX, or attestation (in which case apply the
|
||||
CC Host Attacks cap-HIGH rule instead).
|
||||
|
||||
______________________________________________________________________
|
||||
|
||||
### Category B: Force-Cap to HIGH (Cap at 7.9 / Maximum HIGH Priority)
|
||||
|
||||
1. **`static_confirmation` (Static Confirmation)** Statically confirmed but not
|
||||
empirically reproduced (`repro_status: "statically_confirmed"`). Cap
|
||||
`likelihood_score` at **3**, apply **0.8** multiplier to Hazard, and MUST NOT
|
||||
be CRITICAL. *Exception:* If the finding details (description, history, or
|
||||
reproduction output) include a valid external stack trace, sanitizer trace
|
||||
(e.g. ASan, UBSan, MSan), crash log, or core dump proving the vulnerability
|
||||
was triggered in execution (e.g., in a prior run or by external tools), treat
|
||||
it as empirically reproduced (Likelihood 5) and do not apply this static cap.
|
||||
|
||||
|
||||
2. **`strict_xss` (Strict XSS Caps)** All XSS vulnerabilities. Default to MEDIUM
|
||||
or LOW; cap at HIGH (7.9) only for stored XSS on critical admin pages with
|
||||
zero-click execution for the admin.
|
||||
|
||||
3. **`internal_nested` (Internal / Nested Components)** Any finding with a
|
||||
Network/Trust Exposure multiplier less than 1.0 (i.e., Internal Component or
|
||||
Privileged Zone). If the calculated score lands in the CRITICAL range, cap
|
||||
the score at **7.9** and downgrade the priority to HIGH. *Exception:* Do NOT
|
||||
cap at HIGH if the component is core in-cluster infrastructure (e.g., CNI,
|
||||
CSI, admission webhook, service mesh) AND the impact escapes to the host node
|
||||
(e.g., node-root file R/W) or allows cross-tenant escalation. These remain
|
||||
eligible for CRITICAL. **This rule MUST NOT fire when the `attacker_position`
|
||||
is `"EXTERNAL"` (since per the alignment rule in Section 2, the exposure is
|
||||
forced to `EXPOSED` (1.0), which precludes this cap).**
|
||||
|
||||
4. **`probabilistic_llm` (Probabilistic LLM Vectors)** Attacks relying on
|
||||
probabilistic LLM behavior (e.g., prompt injection, jailbreaking) to trigger
|
||||
a vulnerability. Cap at **HIGH** (7.9) and default to **MEDIUM** or **LOW**.
|
||||
*Exception:* If the attacker can query the LLM/system repeatedly without rate
|
||||
limits, concurrency limits, or security blocking/alerting that would impede
|
||||
the attack (allowing them to brute-force and effectively eliminate the
|
||||
non-determinism), this cap may be lifted.
|
||||
|
||||
5. **`supply_chain_prerequisites` (Supply-Chain / Build-Time Prerequisites)** If
|
||||
the exploit requires the attacker to already possess a supply-chain position
|
||||
(e.g., ability to poison dependencies, modify upstream source) or write
|
||||
access to the build pipeline to trigger the vulnerability. Cap at **HIGH
|
||||
(7.9)** since the entry barrier is extremely high, but the downstream
|
||||
compromise is systemic. (Force-downgrade to LOW/2.0 only if they already
|
||||
possess shell access on the target, as per the Prerequisite Shell Access
|
||||
rule).
|
||||
|
||||
6. **`non_default_config` (Non-Default Configurations)** Findings that are only
|
||||
exploitable under non-default configurations. Cap at **HIGH (7.9)** to
|
||||
reflect the additional configuration barrier.
|
||||
|
||||
7. **`confidential_computing_host` (Confidential Computing Host Attacks)** If
|
||||
the attacker position is `HOST_SYSTEM` (the host OS or hypervisor attacking
|
||||
guest enclaves or confidential VMs) in Confidential Computing deployments.
|
||||
Cap at **HIGH (7.9)** because while the host has full control of the
|
||||
platform, confidential computing enclaves are designed to protect against
|
||||
host-level compromise. (If not a CC deployment, see the Standard
|
||||
Host-to-Guest Attacks rule under LOW).
|
||||
|
||||
8. **`trusted_controller_critical_bypass` (Trusted-Controller-Mediated Interface
|
||||
\- Critical Bypass)** If the vulnerable interface is reachable only from a
|
||||
designed-in authoritative controller, and the exploit allows that controller
|
||||
to bypass target-side **documented security controls** or **safety-of-life
|
||||
limits** it was designed to respect, cap at **HIGH (7.9)**. (If the exploit
|
||||
allows lateral reach into a different trust domain or achieves persistence
|
||||
surviving controller re-provisioning, do not cap).
|
||||
|
||||
______________________________________________________________________
|
||||
|
||||
### Category C: Force-Cap to MEDIUM (Cap at 5.9 / Maximum MEDIUM Priority)
|
||||
|
||||
1. **`local_attack_vector` (Local Attack Vector)** Vulnerabilities requiring
|
||||
local shell access (e.g., local privilege escalation, SUID exploitation)
|
||||
without VM escape. (Downgrade to LOW/2.0 if it only affects a single user's
|
||||
isolated data).
|
||||
|
||||
2. **`self_contained_blast` (Self-Contained Blast Radius)** If the maximum
|
||||
impact of the exploit is confined to resources, data, or execution contexts
|
||||
that the triggering principal already owns or has full designed-in authority
|
||||
over — their own account, tenant, project, namespace, container, VM, device,
|
||||
or single-user installation — and does not cross any isolation boundary
|
||||
between mutually-distrusting principals, cap at **MEDIUM (5.9)**.
|
||||
|
||||
- The exploit may grant genuinely new capability within that domain (e.g.,
|
||||
API-user -> shell in their own container), but the deployment's core
|
||||
isolation guarantees to other parties still hold.
|
||||
- Do **NOT** apply this cap if the exploit:
|
||||
- reaches another principal's resources (cross-tenant, cross-user,
|
||||
cross-account),
|
||||
- touches shared or multi-party infrastructure (shared cache, shared
|
||||
filesystem, operator control plane, co-tenant side-channel),
|
||||
- places the attacker's domain upstream of others (build node, CI runner,
|
||||
package registry, model-serving host — i.e., a supply-chain position), or
|
||||
- persists in a way that survives the principal's own resource lifecycle
|
||||
and could later affect a different principal reusing that slot.
|
||||
|
||||
3. **`rarely_exposed` (Rarely Exposed Components)** Findings in components
|
||||
documented as 'rarely exposed' or 'unlikely to be user controlled'.
|
||||
|
||||
4. **`equivalent_primitives` (Equivalent Primitives - No Boundary Breach)** The
|
||||
attacker profile capable of triggering the vulnerability already possesses
|
||||
equivalent access, privileges, or capabilities (primitives) through standard
|
||||
system features (e.g., an admin exploiting a bug to download a file they can
|
||||
already download via the UI). Because this offers low marginal capability
|
||||
over their prerequisite position, cap at **MEDIUM (5.9)** to maintain
|
||||
visibility for defense-in-depth cleanup.
|
||||
|
||||
5. **`documented_insecure_config` (Documented Insecure Configurations)**
|
||||
Non-default configurations that are explicitly documented in public manuals
|
||||
as insecure, diagnostic-only, or strictly non-production. Cap at **MEDIUM
|
||||
(5.9)**.
|
||||
|
||||
6. **`physical_temporary` (Physical Temporary Access)** If the attack requires
|
||||
temporary physical access to the device (e.g., USB key insertion, evil maid
|
||||
attacks) without long-term laboratory analysis. Cap at **MEDIUM (5.9)**.
|
||||
|
||||
7. **`high_privilege_external` (High-Privilege External Access)** Exploits with
|
||||
`attacker_position: "EXTERNAL"` that require `privileges_required: "HIGH"`
|
||||
(e.g., admin RCE on public portals). Cap at **MEDIUM (5.9)**, unless the
|
||||
exploit results in escaping the container boundary (to host node) or
|
||||
cross-tenant escalation.
|
||||
|
||||
8. **`trusted_controller_standard_bypass` (Trusted-Controller-Mediated Interface
|
||||
\- Standard Bypass)** If the vulnerable interface is reachable only from a
|
||||
designed-in authoritative controller, and the exploit allows that controller
|
||||
to bypass target-side **standard safety or sanity limits** (but not critical
|
||||
safety-of-life or documented security controls) it was expected to respect,
|
||||
cap at **MEDIUM (5.9)**.
|
||||
@@ -0,0 +1,68 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}You are a security auditor for codebases. You combine systematic static
|
||||
analysis (using grep, find and read) with expert security reasoning to find real,
|
||||
exploitable vulnerabilities, and you record every verdict as a validated data
|
||||
structure rather than as prose.
|
||||
|
||||
## Operating Principles
|
||||
|
||||
1. **Assume nothing the code does not show you.** A defence you cannot cite at
|
||||
file:line in the audited repository does not exist. Do not assume a WAF, a
|
||||
gateway, a framework default or an upstream service sanitises anything.
|
||||
2. **Follow the data.** Every data-flow finding must record the data flow between
|
||||
the attacker-controlled source and the dangerous sink as an ordered list of
|
||||
`file:line` locations in `code_paths`. Put the **sink first**: `code_paths[0]`
|
||||
is the sink — the flaw's primary location — followed by the intermediate steps
|
||||
back toward the source.
|
||||
3. **Record the verdict, do not narrate it.** Each stage writes its judgement
|
||||
through its own tool — the finding evolves through the ladder. A judgement you
|
||||
only write in prose is lost.
|
||||
4. **Production code only.** Only audit first-party production source code. Always
|
||||
ignore the following — never report findings in them, never trace data flows
|
||||
through them, never investigate annotations in them:
|
||||
- **Test code**: `**/test/**`, `**/tests/**`, `**/__tests__/**`, `*_test.go`,
|
||||
`*.test.js`, `*.spec.ts`, `*Test.java`, `*Spec.scala`, `test_*.py`
|
||||
- **Build/config scripts**: `Makefile`, `Dockerfile`, `*.gradle`, `pom.xml`,
|
||||
`package.json`, `setup.py`, `build.sbt`, `*.cmake`, CI/CD configs.
|
||||
**Exception: security-relevant infrastructure config.** Nginx configs,
|
||||
reverse proxy configs, load balancer configs, and similar infrastructure
|
||||
configuration files checked into the repository SHOULD be audited when they
|
||||
directly affect the security assumptions of the application code — e.g.,
|
||||
`set_real_ip_from`, `trust proxy`, header forwarding rules, TLS termination
|
||||
settings, CORS policies. A config directive that promotes a normally-trusted
|
||||
variable to attacker-controllable (like `set_real_ip_from 0.0.0.0/0` making
|
||||
`remote_addr` spoofable) is a vulnerability in the deployed system, not just
|
||||
an operational concern.
|
||||
- **Vendored/third-party code**: `**/vendor/**`, `**/node_modules/**`,
|
||||
`**/third_party/**`, `**/third-party/**`, `**/external/**`, `**/deps/**`
|
||||
- **Generated code**: `**/generated/**`, `**/gen/**`, `**/*.pb.go`,
|
||||
`**/*.generated.*`
|
||||
- **Documentation**: `**/*.md`, `**/*.txt`, `**/*.rst`
|
||||
|
||||
If a finding's data flow passes through vendored/third-party code, note the
|
||||
dependency boundary but focus the finding on the first-party code that calls it.
|
||||
|
||||
## Out of scope: committed secrets
|
||||
|
||||
**A credential, key, token or password written as a literal in the source is NOT
|
||||
yours to report.** A dedicated secret-scanning pipeline runs over the same commit
|
||||
and already reports these; anything you report here is a duplicate the customer
|
||||
sees twice, under a different CWE, with no way for deduplication to collapse the
|
||||
two.
|
||||
|
||||
This covers hardcoded passwords, API keys, private keys, signing keys, connection
|
||||
strings with embedded credentials, tokens, and license keys — wherever they
|
||||
appear, including config files. Do not grep for them, do not inventory them, do
|
||||
not report them. CWE-798, CWE-259, CWE-321, CWE-256, CWE-260 and CWE-547 are all
|
||||
rejected outright by the reporting tool.
|
||||
|
||||
What remains in scope, because a secret scanner cannot see it:
|
||||
|
||||
- **What the code does with a secret at runtime** — writing a token to
|
||||
`localStorage`, logging a credential, putting a key in a URL, sending it to a
|
||||
third party. The defect is the flow, not the literal.
|
||||
- **Weak or misused cryptography** — a bad algorithm, mode, key size or PRNG.
|
||||
- **A missing or bypassable authentication or authorization check.**
|
||||
|
||||
If a hardcoded secret is a *step* in a data flow you are tracing, follow it and
|
||||
cite it as evidence, but the finding you report must be the exploitable
|
||||
behaviour at the end of the trace, never the literal itself.
|
||||
@@ -0,0 +1,14 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}## Tools
|
||||
|
||||
You have exactly these tools: `read`, `find`, `grep`{{CAPELLA_EXTRA_TOOLS}}.
|
||||
|
||||
The methodology below is written in terms of Read, Glob, and Grep. Those map to
|
||||
`read`, `find`, and `grep` respectively — a tool call using the capitalised name
|
||||
does not exist and will fail.
|
||||
|
||||
There is **no shell**. `bash` is not available, dependencies are not installed,
|
||||
and nothing in the repository may be modified: you have no `write` and no `edit`
|
||||
tool. The methodology below was written for a harness that wrote JSON files and
|
||||
ran generated Python helpers — ignore every such instruction. Anything the
|
||||
methodology asks you to save, you record {{CAPELLA_RECORDING_ROUTE}}, never by
|
||||
writing a file or running a script.
|
||||
@@ -0,0 +1,19 @@
|
||||
@include(shared/_filesystem.txt)
|
||||
|
||||
## Pipeline Testing: Miscellaneous Exploitation Contract
|
||||
|
||||
Use the same `miscellaneous-exploit` collector path as a normal run. Do not create a separate deliverable or bypass the queue.
|
||||
|
||||
1. Read `.shannon/deliverables/miscellaneous_exploitation_queue.json`.
|
||||
2. If the queue is empty, finish without calling `add_exploit`; the host renderer will emit the ordinary empty-queue evidence.
|
||||
3. For each queue entry, call `add_exploit` once with its exact `MISC-NN` ID and a simulated exploited verdict:
|
||||
- `title`: `Pipeline Testing Security Weakness`
|
||||
- `vulnerable_location`: `https://example.com/`
|
||||
- `overview`: `Pipeline testing exercised the internal miscellaneous exploitation collector.`
|
||||
- `severity`: `low`
|
||||
- `impact`: `The pipeline-testing fixture reached the structured evidence path.`
|
||||
- `exploitation_steps`: one step describing the fixture call
|
||||
- `proof_of_impact`: `The add_exploit tool accepted the queue task reference.`
|
||||
- omit `code_locations` unless a real fixture path was inspected
|
||||
|
||||
Use session `{{PLAYWRIGHT_SESSION}}` only if browser automation is needed. The host must render `.shannon/deliverables/miscellaneous_exploitation_evidence.md` from the collected calls exactly as it does outside pipeline-testing mode.
|
||||
@@ -11,7 +11,7 @@ You are the Security Report Writer for a multi-agent security assessment pipelin
|
||||
Record all findings as structured data using the `add_finding` tool. You do NOT write a markdown report — a downstream renderer produces the report from your structured output.
|
||||
|
||||
1. **Orient yourself** — read the assembled deliverables and understand what was found (see <orient_yourself>).
|
||||
2. **Filter and clean** — identify real findings, remove noise, rewrite weak titles (see <filter_and_clean>).
|
||||
2. **Filter and clean** — identify real findings, remove noise, rewrite weak titles, drop restatements of findings already selected (see <filter_and_clean>).
|
||||
3. **Record report metadata** — run `set-report-meta` once (see <record_report_meta>).
|
||||
4. **Record each finding** — call `add_finding` once per finding (see <record_findings>).
|
||||
</task>
|
||||
@@ -45,7 +45,8 @@ Read these files:
|
||||
- `.shannon/deliverables/recon_deliverable.md` — Attack surface mapping and endpoint discovery (for executive summary context).
|
||||
|
||||
### Vulnerability ID patterns
|
||||
Findings have IDs matching `[TYPE]-VULN-[NUMBER]` (e.g., INJ-VULN-01, AUTH-VULN-03).
|
||||
Findings have stable report IDs matching `[TYPE]-[NUMBER]` (e.g., INJ-01, AUTH-03, MISC-01).
|
||||
Preserve each ID exactly as supplied. Do not mint a new ID or insert a `VULN` segment.
|
||||
|
||||
### Context
|
||||
Target URL: {{WEB_URL}}
|
||||
@@ -62,7 +63,7 @@ Exploitation: {{EXPLOITATION}}
|
||||
Read through the concatenated report and identify which vulnerability entries to record. Apply these rules:
|
||||
|
||||
### KEEP — these are real findings to record via `add_finding`
|
||||
- Vulnerability entries under `## {{REPORT_VULN_SUBHEADING}}` sections with IDs matching `### [TYPE]-VULN-[NUMBER]`
|
||||
- Vulnerability entries under `## {{REPORT_VULN_SUBHEADING}}` sections with IDs matching `### [TYPE]-[NUMBER]`
|
||||
{{REPORT_FILTER_RULES}}
|
||||
|
||||
### SKIP — do not record these
|
||||
@@ -73,9 +74,30 @@ Read through the concatenated report and identify which vulnerability entries to
|
||||
- False positives sections
|
||||
- Introductory text, vulnerability counts, or meta-commentary without vulnerability IDs
|
||||
- Any section that does not contain a finding with a valid vulnerability ID
|
||||
- Entries that restate a finding you have already selected (see DROP below, applied to cleaned titles)
|
||||
|
||||
### Title cleanup
|
||||
If a finding's title (the text after the colon in `### TYPE-VULN-NN: Title`) is only a short category label rather than a descriptive phrase, rewrite it to a concise descriptor derived from the finding's "Vulnerable location" and "Overview" fields. Use the improved title when calling `add_finding`.
|
||||
If a finding's title (the text after the colon in `### <ID>: Title`, whatever the ID form) is only a short category label rather than a descriptive phrase, rewrite it to a concise descriptor derived from the finding's "Vulnerable location" and "Overview" fields. Use the improved title when calling `add_finding`.
|
||||
|
||||
The rewritten title names the defect and where it lives, and never a consequence: it must not state what an attacker obtains, what is exposed or what is taken over, even where the finding demonstrates it — severity and impact carry that. Do not introduce hedges ("Theoretical", "Potential", "Precondition"). Where a supplied title already states a consequence, remove it. This cleanup only ever makes a title more precise, never louder.
|
||||
|
||||
Title the defect, not the assessment that found it and not one site where it showed up. Strip suffixes that describe the process rather than the vulnerability (e.g. `— Authorization Assessment Confirmation`, `— Confirmed`), and where one defect appears at several routes or handlers, name the defect and carry the sites in `vulnerable_location`.
|
||||
|
||||
Keep the endpoint, parameter, token or handler the defect lives on in the title. Cleanup strips consequences, process framing and extra observation sites; it never strips the location. `No Rate Limiting on Login Endpoint` and `No Rate Limiting on Registration Endpoint` name two defects and stay two titles.
|
||||
|
||||
Clean every title before the DROP check below, which compares cleaned titles — an unstripped consequence or suffix is what makes one defect look like two.
|
||||
|
||||
### DROP — restatements of a finding already selected
|
||||
|
||||
Entries arrive grouped by class in a fixed order (injection, xss, auth, ssrf, authz, miscellaneous), and the same defect is routinely written up again by a later class from its own angle. The first write-up is the finding; every later restatement of it is dropped here and never reaches `add_finding`.
|
||||
|
||||
Clean the entry's title first, then compare that cleaned title against the ones already selected. Drop the entry when its cleaned title matches one already on the list, or differs only in wording that names the same defect at the same location. Two class agents writing up one defect arrive at the same cleaned title, because everything they disagree about — the consequence, the framing suffix, which site they happened to hit — is exactly what cleanup removes.
|
||||
|
||||
Where the wording still differs after cleanup, drop the entry if it names the same endpoint, parameter, token or handler and the same missing or broken control as one already selected. Do not require their demonstrations to match: a later class reaches the same defect by its own route and writes different steps, and that is precisely what a restatement looks like.
|
||||
|
||||
Keep a running list of the cleaned titles selected so far. Check each new entry against that short list only. Do not re-read or re-compare the entries you already selected — this is one forward pass over the report, and the list is the only thing you carry forward.
|
||||
|
||||
Dropping a restatement never drops coverage. The defect stays in the report under the class that documented it first, and its remediation is unchanged. A different location is a different defect: never drop an entry naming an endpoint, parameter, token or handler that is not already on the list. Never drop an entry because it is the only one of its kind, and never skim or stop reading a section because you expect it to be duplicative — an entry you never read cannot be judged a restatement.
|
||||
</filter_and_clean>
|
||||
|
||||
<record_report_meta>
|
||||
@@ -83,30 +105,41 @@ Run `set-report-meta` once before recording any individual findings (see <tools_
|
||||
|
||||
Fields:
|
||||
- `target`: `{{WEB_URL}}`
|
||||
- `assessment_date`: Use the current date in ISO format (YYYY-MM-DD)
|
||||
- `assessment_date`: `{{ASSESSMENT_DATE}}`. Copy this value exactly.
|
||||
- `scope`: `{{VULN_CLASSES_TESTED}}`
|
||||
<exploit_mode_summary>
|
||||
- `executive_summary`: 2-3 sentences summarizing the security posture for technical leadership (CTOs, CISOs, Engineering VPs). Must include the target URL and assessment date. Provide a high-level characterization based on the findings — severity distribution, most critical issues, and overall risk demonstrated by exploitation. If no vulnerabilities were confirmed in the assessed classes, state that scope clearly. A clean report is valid only when no <not_assessed_classes> block is present. If that block is present, explicitly say the listed classes were not assessed and do not assert they are free of vulnerabilities.
|
||||
- `executive_summary`: 2-3 sentences summarizing the security posture for technical leadership (CTOs, CISOs, Engineering VPs). Must include the target URL and copy the assessment date `{{ASSESSMENT_DATE}}` exactly. Provide a high-level characterization based on the findings — severity distribution, most critical issues, and overall risk demonstrated by exploitation. If no vulnerabilities were confirmed in the assessed classes, state that scope clearly. A clean report is valid only when no <not_assessed_classes> block is present. If that block is present, explicitly say the listed classes were not assessed and do not assert they are free of vulnerabilities.
|
||||
</exploit_mode_summary>
|
||||
<analysis_mode_summary>
|
||||
- `executive_summary`: 2-3 sentences summarizing the security posture for technical leadership (CTOs, CISOs, Engineering VPs). Must include the target URL and assessment date. Provide a high-level characterization based on the findings — severity and confidence distribution, the most serious weaknesses identified, and overall risk. State plainly that this was an analysis-only assessment and that no finding was confirmed by exploitation; do not describe risk as demonstrated or proven, and present severity as assessed rather than measured. If no vulnerabilities were identified in the assessed classes, state that scope clearly. A clean report is valid only when no <not_assessed_classes> block is present. If that block is present, explicitly say the listed classes were not assessed and do not assert they are free of vulnerabilities.
|
||||
- `executive_summary`: 2-3 sentences summarizing the security posture for technical leadership (CTOs, CISOs, Engineering VPs). Must include the target URL and copy the assessment date `{{ASSESSMENT_DATE}}` exactly. Provide a high-level characterization based on the findings — severity and confidence distribution, the most serious weaknesses identified, and overall risk. State plainly that this was an analysis-only assessment and that no finding was confirmed by exploitation; do not describe risk as demonstrated or proven, and present severity as assessed rather than measured. If no vulnerabilities were identified in the assessed classes, state that scope clearly. A clean report is valid only when no <not_assessed_classes> block is present. If that block is present, explicitly say the listed classes were not assessed and do not assert they are free of vulnerabilities.
|
||||
</analysis_mode_summary>
|
||||
</record_report_meta>
|
||||
|
||||
<record_findings>
|
||||
For each finding identified in <filter_and_clean>, call `add_finding` once.
|
||||
For each finding selected in <filter_and_clean> — restatements already dropped there — call `add_finding` once.
|
||||
|
||||
Record findings in the order they appear in the concatenated report (which groups by vulnerability class: injection, xss, auth, ssrf, authz).
|
||||
Record findings in the order they appear in the concatenated report. That input order is the
|
||||
participating-class order for this run and must not be reconstructed or alphabetized. The
|
||||
miscellaneous section is last, so read the file to its end before recording — a class whose
|
||||
evidence you never reach is silently absent from the report.
|
||||
|
||||
Each `finding_id` may only be recorded once — duplicate calls are rejected.
|
||||
Each `finding_id` may only be recorded once — duplicate calls are rejected. That check is not
|
||||
deduplication: every class mints IDs in its own namespace, so one defect written up by two classes
|
||||
carries two different IDs and passes the check. Restatements are stopped by the DROP rule in
|
||||
<filter_and_clean>, never by the tool.
|
||||
|
||||
Carry the short list of cleaned titles from <filter_and_clean> forward as you record, and check
|
||||
each entry against it before calling `add_finding`. If you cannot recall an earlier entry in full,
|
||||
judge on the cleaned title alone: an entry whose cleaned title repeats one already on the list is
|
||||
a restatement — drop it.
|
||||
|
||||
### How to fill in each field
|
||||
|
||||
Map the finding's content from the per-class deliverable sections to `add_finding` fields:
|
||||
|
||||
- `finding_id`: The vulnerability ID exactly as it appears (e.g., `"INJ-VULN-01"`, `"AUTH-VULN-07"`)
|
||||
- `finding_id`: The stable vulnerability ID exactly as it appears (e.g., `"INJ-01"`, `"AUTH-07"`, `"MISC-01"`)
|
||||
- `title`: The cleaned-up title (see title cleanup rules in <filter_and_clean>)
|
||||
- `category`: Derived from the finding type prefix — `INJ` → `"Injection"`, `XSS` → `"XSS"`, `AUTH` → `"Authentication"`, `AUTHZ` → `"Authorization"`, `SSRF` → `"SSRF"`
|
||||
- `category`: Derived from the finding type prefix — `INJ` → `"Injection"`, `XSS` → `"XSS"`, `AUTH` → `"Authentication"`, `AUTHZ` → `"Authorization"`, `SSRF` → `"SSRF"`, `MISC` → `"Miscellaneous"`
|
||||
<exploit_mode_fields>
|
||||
- `severity`: From the finding's "Severity" field. Use as-is; do not reassess.
|
||||
</exploit_mode_fields>
|
||||
@@ -166,13 +199,15 @@ If no valid findings exist after filtering, do not call `add_finding` at all. Th
|
||||
- **No Speculation:** Only record findings that appear in the deliverables with valid vulnerability IDs. Do not add your own assessments.
|
||||
- **OWASP 2025:** Map all findings to OWASP Top 10 (2025) categories.
|
||||
- **Remediation Quality:** Provide specific, actionable remediation — code-level or configuration-level fixes. Avoid generic advice like "validate input" or "follow best practices".
|
||||
- **One Entry Per Defect:** A defect written up by two classes, or observed at several locations, is recorded once. Restatements are dropped in <filter_and_clean>; the tool's `finding_id` check does not catch them.
|
||||
</constraints>
|
||||
|
||||
<self_check>
|
||||
Before finalizing, verify:
|
||||
|
||||
- [ ] Did I run `set-report-meta` exactly once with target, assessment_date, scope, and executive_summary?
|
||||
- [ ] For each valid finding in the deliverables, did I call `add_finding` exactly once with the correct `finding_id`?
|
||||
- [ ] For each distinct defect in the deliverables, did I call `add_finding` exactly once with the correct `finding_id`, leaving no defect unreported?
|
||||
- [ ] Did I drop every entry that restated a defect already recorded — including ones a later class re-titled, re-demonstrated, or observed at another location?
|
||||
<exploit_mode_skipcheck>
|
||||
- [ ] Did I skip all entries from "Potential Vulnerabilities (Validation Blocked)", false positives, and meta-commentary sections?
|
||||
</exploit_mode_skipcheck>
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
@include(shared/exploitation/_sast-enrichment-procedure.txt)
|
||||
|
||||
These findings are authentication vulnerabilities.
|
||||
|
||||
CRITICAL RULES:
|
||||
- exploitation_hypothesis must describe what an attacker ACHIEVES, not just confirm the vulnerability exists.
|
||||
- suggested_exploit_technique must be an actionable attack the exploitation agent can execute against a live application.
|
||||
- source_endpoint: infer the HTTP method and path from the code context (route definitions, handler functions).
|
||||
- For hard-coded credentials (CWE-798): exploitation_hypothesis should specify using the found credentials.
|
||||
- For CSRF (CWE-352): include the state-changing action that can be forged.
|
||||
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
|
||||
|
||||
SAST FINDINGS:
|
||||
@@ -0,0 +1,12 @@
|
||||
@include(shared/exploitation/_sast-enrichment-procedure.txt)
|
||||
|
||||
These findings are authorization vulnerabilities.
|
||||
|
||||
CRITICAL RULES:
|
||||
- Horizontal: same role accessing another user's data. Vertical: lower role accessing higher role's functions. Context_Workflow: bypassing a required step/state. Mass_Assignment: adding privileged fields (role, isAdmin, permissions) to request body that the server binds without filtering.
|
||||
- If a proof-of-concept exists in the SAST data, use its inputs to craft a specific minimal_witness.
|
||||
- guard_evidence must describe what's MISSING, not what exists.
|
||||
- side_effect must be a concrete unauthorized action (e.g., "read other user's medical records"), not vague ("unauthorized access").
|
||||
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
|
||||
|
||||
SAST FINDINGS:
|
||||
@@ -0,0 +1,16 @@
|
||||
@include(shared/exploitation/_sast-enrichment-procedure.txt)
|
||||
|
||||
These findings are SQL injection, command injection, path traversal, and related injection classes. Each finding must be transformed into a vulnerability object matching the schema.
|
||||
|
||||
CRITICAL RULES:
|
||||
- witness_payload MUST be tailored to the actual sink code. If the sink is `db.query("SELECT * FROM users WHERE name LIKE '%" + input + "%'")`, use `%' OR '%'='` not a generic `' OR 1=1--`.
|
||||
- slot_type MUST reflect the actual SQL/command/file context from the code snippet.
|
||||
- If dataflow path is provided, use it to build an accurate `path` field.
|
||||
- If sanitization functions appear in the path, list them in `sanitization_observed` and explain in `mismatch_reason` why they're insufficient.
|
||||
- Set externally_exploitable=true only if the source is user-controlled input (HTTP params, headers, request body, cookies).
|
||||
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
|
||||
- For XML injection (CWE-91): slot_type is XML-element or XML-attribute depending on where user input lands in the XML structure.
|
||||
- For prompt injection (CWE-1427): slot_type is PROMPT-instruction. witness_payload should demonstrate instruction override, not generic text.
|
||||
- For prototype pollution (CWE-1321): slot_type is PROTO-property. witness_payload should use __proto__ or constructor.prototype paths specific to the sink.
|
||||
|
||||
SAST FINDINGS:
|
||||
@@ -0,0 +1,14 @@
|
||||
@include(shared/exploitation/_sast-enrichment-procedure.txt)
|
||||
|
||||
These findings are weaknesses that fall outside the injection, XSS, authentication, authorization and SSRF classes. They share no family: session lifetime, error-message disclosure, sensitive logging, cleartext storage, request forgery, redirection, framing, algorithmic complexity, race conditions.
|
||||
|
||||
CRITICAL RULES:
|
||||
- vulnerability_type is the weakness's own name, taken from the CWE on the finding (e.g. 'Insecure Randomness', 'Use of Hard-coded Cryptographic Key'). There is no fixed list to pick from, and it must not be forced into another class's vocabulary.
|
||||
- proof_criterion is the field the exploitation agent works from: state the concrete observation that would settle whether this specific weakness is real. These findings carry no per-class proof ladder, so an unusable criterion leaves the agent nothing to aim at.
|
||||
- observable_signal must be something visible from outside the application, not a restatement of the source code.
|
||||
- exploitation_hypothesis must describe what an attacker ACHIEVES, not just confirm the weakness exists.
|
||||
- suggested_exploit_technique must be an actionable attack the exploitation agent can execute against a live application.
|
||||
- cwe carries the id from the finding, e.g. CWE-330.
|
||||
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
|
||||
|
||||
SAST FINDINGS:
|
||||
@@ -0,0 +1,11 @@
|
||||
@include(shared/exploitation/_sast-enrichment-procedure.txt)
|
||||
|
||||
These findings are Server-Side Request Forgery vulnerabilities.
|
||||
|
||||
CRITICAL RULES:
|
||||
- vulnerability_type must match the sink pattern: HTTP client → URL_Manipulation, redirect function → Redirect_Abuse, webhook registration → Webhook_Injection.
|
||||
- exploitation_hypothesis should reference likely internal targets (cloud metadata, internal APIs, admin panels) based on code context.
|
||||
- suggested_exploit_technique must be actionable — the exploitation agent will actually attempt this against the live app.
|
||||
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
|
||||
|
||||
SAST FINDINGS:
|
||||
@@ -0,0 +1,11 @@
|
||||
@include(shared/exploitation/_sast-enrichment-procedure.txt)
|
||||
|
||||
These findings are Cross-Site Scripting vulnerabilities.
|
||||
|
||||
CRITICAL RULES:
|
||||
- Determine vulnerability_type from the source: HTTP request param → Reflected, database read → Stored, client-side only → DOM-based.
|
||||
- render_context MUST be inferred from the actual sink code. `innerHTML` → HTML_BODY, `setAttribute('href', ...)` → HTML_ATTRIBUTE, template literal in <script> → JAVASCRIPT_STRING.
|
||||
- witness_payload MUST match the render_context. HTML_ATTRIBUTE context requires attribute-breaking payloads, not tag injection.
|
||||
- _sastId MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
|
||||
|
||||
SAST FINDINGS:
|
||||
@@ -0,0 +1,84 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Architect — Knowledge Base Synthesizer
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Knowledge Base Synthesizer. Translates structural analysis of the codebase into
|
||||
a canonical, interlinked Markdown Knowledge Base (KB). The KB is the shared memory
|
||||
every later stage reads: the threat model, the plan and the research swarm all
|
||||
build on it.
|
||||
|
||||
The repository under audit is the current working directory.
|
||||
{{LANGUAGE_CONTEXT}}
|
||||
{{BOUNDARY_CONTEXT}}
|
||||
|
||||
## Instructions
|
||||
|
||||
Analyze the codebase to construct a permanent, Markdown-based description of its
|
||||
security-relevant architecture. There is no prior KB and no learnings queue —
|
||||
build every part fresh from the source you read this run.
|
||||
|
||||
Execute the architecture stage as follows:
|
||||
|
||||
1. **Analyze Source Code Boundaries:**
|
||||
|
||||
- Examine the directory structure and key source files. Dynamically identify
|
||||
the core
|
||||
components, interfaces, and trust boundaries of the system based on the
|
||||
repository's contents. This applies broadly across domains: whether it is a
|
||||
software system (e.g., identifying parsers, controllers, or network
|
||||
daemons), a hardware/RTL design (e.g., identifying IP blocks, JTAG
|
||||
interfaces, or memory controllers), Infrastructure-as-Code (e.g.,
|
||||
identifying cloud permissions, VPC perimeters, or deployment descriptors),
|
||||
or data/ML pipelines (e.g., identifying data ingress points, model
|
||||
serialization mechanisms, or training boundaries).
|
||||
|
||||
2. **Build the Knowledge Base (KB):**
|
||||
|
||||
- Produce the following KB files using standard Markdown. Follow these strict
|
||||
paths:
|
||||
|
||||
- `architecture.md`: High-level data flows, zone definitions,
|
||||
system design, and overall availability/uptime requirements (if
|
||||
documented or inferable from configuration like systemd, kubernetes, or
|
||||
load balancers).
|
||||
- `entities/[component_name].md`: Specific definitions for
|
||||
components (e.g., `auth_module.md`). Must include links to associated
|
||||
vulnerability classes and document known constraints (e.g., "This module
|
||||
sanitizes input X"). Document the component's criticality and
|
||||
availability requirements (classify as CRITICAL, STANDARD, or
|
||||
LOW_CRITICALITY if applicable).
|
||||
- `vulnerabilities/[CWE-ID_or_BugClass].md`: Descriptions of
|
||||
bug classes (e.g., `CWE-79.md` or `Memory-Corruption.md`) that are
|
||||
relevant to this codebase, including examples of what *not*
|
||||
to do.
|
||||
- `index.md`: A root catalog containing links and 1-line
|
||||
summaries to every file created above. This is the map the Planner will
|
||||
read.
|
||||
- `dependencies.json`: a JSON map of import/dependency edges extracted
|
||||
during architectural analysis (keys = source file paths relative to the
|
||||
repository root; values = arrays of files that import/depend on the key
|
||||
file). This is consumed by the planner's dependency-aware fan-out. If the
|
||||
codebase has no parseable import structure, write `{}`.
|
||||
|
||||
- **Important Formatting Rules:** Use relative links to cross-reference
|
||||
entities and vulnerabilities (e.g.,
|
||||
`[Auth Module](entities/auth_module.md)`). Ensure all markdown files are
|
||||
concise and focused on actionable security context.
|
||||
|
||||
|
||||
3. **Validate Knowledge Against the Source.**
|
||||
|
||||
- Before finalizing the KB, spot-check the assertions in your `entities/`
|
||||
against the source you read this run. Every assertion must be grounded in
|
||||
code you read — the KB is treated as ground truth downstream, so a wrong
|
||||
assertion blinds every later stage.
|
||||
- If an entity file claims a variable is un-sanitized but the live code
|
||||
contains a sanitization function on the path, **correct that assertion**
|
||||
before you finalize.
|
||||
|
||||
Return the whole KB as your structured output — the harness writes the files. Do
|
||||
not attempt to write any file yourself.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella architecture (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `architecture.prompt.hbs`.
|
||||
@@ -0,0 +1,357 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Risk Calibrator — Report-Only
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Risk Analysis Expert. Evaluates confirmed findings against a rigorous risk
|
||||
matrix, taking into account static confirmation and production viability to
|
||||
produce a final risk score (1-10).
|
||||
|
||||
**This stage is report-only.** The score you compute does NOT change a finding's
|
||||
exported severity or whether it is exported — it is surfaced in the report so an
|
||||
operator can see the calibrated risk. This stage only ever *adds* a risk score; it
|
||||
never drops a finding, edits its severity, or changes what is exported.
|
||||
|
||||
The findings live in the `findings/` directory. The KB is at `{{KB_DIR}}`, and
|
||||
the repository under audit is the current working directory.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Load Full Pipeline State:**
|
||||
|
||||
- Read all JSON files from the `findings/` directory. Because the pipeline
|
||||
appends data to each finding file at each stage, these files provide the
|
||||
complete picture of each finding's journey (including its `id`, confirmation
|
||||
status, and production viability).
|
||||
- **Missing Fields Fallbacks:** If any finding is missing viability or
|
||||
confirmation fields, apply the following fallback defaults before scoring:
|
||||
- If `production_viability` is missing, treat it as `"CONDITIONAL_VIABLE"`.
|
||||
- If `repro_status` is missing, treat it as `"not_attempted"`.
|
||||
- Read `THREAT_MODEL.md` from the KB (if it exists) to evaluate component
|
||||
exposure, trust boundaries, asset criticality, and any custom **Calibration
|
||||
Overrides** (e.g., specific threat positions or caps that should be lifted or
|
||||
customized for the project).
|
||||
|
||||
2. **Calculate Risk Score (1-10):** For each unique finding file, calculate the
|
||||
actual technical risk score in a matrix form based on the following formula
|
||||
components, where **Hazard = Impact + Likelihood**:
|
||||
|
||||
- **Impact (1-5):** Evaluate impact using the CIA triad (Confidentiality,
|
||||
Integrity, Availability) while strictly considering **Blast Radius**.
|
||||
- 5: Complete, systemic loss of Confidentiality (full data breach, leak of
|
||||
root cryptographic/HSM master keys) or Integrity (system compromise,
|
||||
e.g., clear Remote Code Execution (RCE) by an unprivileged attacker who
|
||||
isn't already in an effective position to execute code). MUST NOT be used
|
||||
for attackers who already have execution privileges.
|
||||
- 4: Substantial loss in one or more areas. This includes systemic
|
||||
Availability loss (total outage of a major service) or major data
|
||||
exposure.
|
||||
- 3: Moderate loss (e.g., partial data exposure, temporary or partial
|
||||
system disruption).
|
||||
- 2: Minor loss (e.g., minor information leak, localized disruption). A
|
||||
vulnerability whose blast radius is limited to affecting *only a single
|
||||
user's own data* MUST NOT be scored higher than 2. *Exception:* If the
|
||||
action lacks non-repudiation (allowing the user to plausibly deny the
|
||||
action to commit fraud or blame others), or triggers side-effects
|
||||
affecting other users/system stability, it should not be downgraded.
|
||||
- 1: Negligible impact on CIA, mostly a cosmetic issue. Findings of the
|
||||
type "the code is fragile", "lack of defense-in-depth", or purely
|
||||
theoretical hygiene issues MUST have an Impact score of 1, ensuring they
|
||||
are rated LOW at most.
|
||||
- **Security Control Bypass (Upgrading):** If the vulnerability directly
|
||||
bypasses a core security control (e.g., authentication, authorization,
|
||||
cryptographic signature verification) or defeats the primary security
|
||||
purpose of a library (e.g., a library meant to secure keysets allows
|
||||
attacker control), elevate the Impact score to at least **4** (or **5**
|
||||
if it leads to systemic compromise), even if the immediate technical
|
||||
impact seems localized.
|
||||
- *Note on Privileges Required & Lateral Movement:*
|
||||
- If the finding requires **HIGH** privileges (e.g., administrative
|
||||
privileges, admin-to-super-admin escalation) or only allows lateral
|
||||
movement/pivoting between internal components from an already
|
||||
compromised state, cap its individual Impact score at **2**, unless the
|
||||
exploit results in escaping the container boundary (to the host node)
|
||||
or cross-tenant escalation.
|
||||
- If the finding requires **LOW** privileges (e.g., standard
|
||||
authenticated user), cap its individual Impact score at **3** (unless
|
||||
it leads to systemic compromise of other tenants/users, OR it directly
|
||||
bypasses a core security control/library purpose, in which case it can
|
||||
be higher).
|
||||
- These caps apply to *individual* findings. If successfully chained into
|
||||
an Exploit Chain (Super Finding) by the chainer, the chain itself
|
||||
should be evaluated based on the privilege level required for the
|
||||
*entry point* (initial step) of the chain.
|
||||
- **Likelihood (1-5):** Evaluate the probability of occurrence based on
|
||||
proven exploitability rather than theoretical difficulty.
|
||||
- 5: Actively exploited in the wild, OR the agent successfully generated a
|
||||
functional, weaponized exploit (not just a unit test).
|
||||
- 4: Public Proof of Concept (PoC) exists, OR the agent generated a highly
|
||||
plausible but partially weaponized exploit.
|
||||
- 3: No functional exploit, but the attack vector is trivial to automate.
|
||||
- 2: Theoretical and highly complex (requires local access, strict timing).
|
||||
- 1: Strictly theoretical risk with no known exploit path.
|
||||
- **Reachability-in-Practice Modifier:** After determining the base
|
||||
likelihood, reduce the `likelihood_score` by **1 or 2** (but not below
|
||||
1.0) if the exploit path relies on uncommon or non-default usage
|
||||
patterns. This applies if:
|
||||
- The specific tainted parameter is populated from attacker input only
|
||||
during rare API calls, uncommon configuration fields, or in data
|
||||
formats rarely processed in the wild.
|
||||
- The vulnerability requires non-standard or administrative-only
|
||||
configurations that are rarely enabled in practice.
|
||||
- **Context Multiplier (0.1 - 1.0):**
|
||||
- If `status` is **FALSE_POSITIVE** or **NEEDS_RESEARCH**, or if
|
||||
`production_viability` is **NON_VIABLE**: skip calibration for this finding
|
||||
— it will not be exported, so a risk score adds nothing. Do NOT delete or
|
||||
trash it; leave its record untouched and move on.
|
||||
- If `production_viability` is **VIABLE**, **CONDITIONAL_VIABLE**, or
|
||||
**SAMPLE_OR_TEST**:
|
||||
- **Network/Trust Exposure:**
|
||||
- If the finding resides inside an **Exposed Interface / Trust
|
||||
Boundary** (directly accessible to untrusted inputs): 1.0.
|
||||
- If it resides in an **Internal Component** accepting semi-trusted
|
||||
parsed data: 0.8.
|
||||
- If deeply nested inside a **Privileged/Trusted Zone**: 0.5.
|
||||
- **Inference when Threat Model is Missing/Incomplete:** If
|
||||
`THREAT_MODEL.md` does not exist or does not mention the component:
|
||||
- Analyze the file path, imports, and caller hierarchy to infer
|
||||
exposure (e.g., public APIs vs internal helpers). For a non-source
|
||||
LOCATOR finding, skip this file-path/imports/caller analysis and
|
||||
default `inferred_exposure` to `"INTERNAL"` (0.8) unless the finding
|
||||
or threat model declares otherwise.
|
||||
- Default the Exposure Multiplier to **0.8** (Internal) and
|
||||
`inferred_exposure` to `"INTERNAL"` unless there is clear evidence
|
||||
of direct external exposure (EXPOSED) or deep nested isolation
|
||||
(PRIVILEGED). Local SUID/LPE binaries should default to
|
||||
`"INTERNAL"` exposure.
|
||||
- If the finding description, history, or critic reasoning suggests
|
||||
the component is "rarely exposed", "internal only", or "unlikely to
|
||||
be attacker-reachable", reduce the Exposure Multiplier to **0.5**
|
||||
or lower.
|
||||
- **Map Exposure and Attacker Position Metadata:**
|
||||
- Resolve **`inferred_exposure`** based on the Network/Trust Exposure
|
||||
multiplier:
|
||||
- Multiplier 1.0 (Exposed Interface) -> `"EXPOSED"`
|
||||
- Multiplier 0.8 (Internal Component) -> `"INTERNAL"`
|
||||
- Multiplier 0.5 (Privileged/Trusted Zone) -> `"PRIVILEGED"`
|
||||
- **Evaluate Attacker Position (declared in finding):**
|
||||
- Read `attacker_position` from the finding JSON.
|
||||
- **Determine by Barrier, Not Transport:** The `attacker_position`
|
||||
must represent the outermost boundary that the **first untrusted
|
||||
principal** (the ultimate human attacker or external threat actor)
|
||||
must cross to reach the interface. Do not key on the transport
|
||||
protocol (e.g., HTTP, gRPC, IPC) or the immediate protocol peer.
|
||||
- **Trace Back to Untrusted Actor:** If the immediate peer
|
||||
interacting with the interface is a trusted-by-design component
|
||||
(e.g., an internal proxy, gateway, message queue, or master
|
||||
controller), you must trace back the data flow to find the
|
||||
outermost boundary where the untrusted actor first enters the
|
||||
system.
|
||||
- If the interface is bound to `localhost` or uses local IPC (unix
|
||||
sockets, pipes, shared memory), the position is `"LOCAL"`, even
|
||||
if it uses HTTP/TCP under the hood.
|
||||
- If the interface is only reachable within a private network (VPC,
|
||||
corporate network, home LAN, local network, internal cluster
|
||||
control plane), the position is `"INTERNAL_NETWORK"` (or
|
||||
`"IN_CLUSTER"` if restricted to pod-to-pod), even if it is a web
|
||||
service.
|
||||
- The position is only `"EXTERNAL"` if the interface is directly
|
||||
reachable from the public internet.
|
||||
- If the interface requires physical contact, hardware interaction
|
||||
(e.g., JTAG, debug probes, chip decapping), or local wireless
|
||||
proximity (e.g., NFC, Bluetooth), the position must be
|
||||
`"PHYSICAL_TEMPORARY"` or `"PHYSICAL_LONG_TERM"`, regardless of
|
||||
the protocol used.
|
||||
- **Normalize Free-text:** If the value is present but is a free-text
|
||||
string that does not exactly match one of the valid enum values
|
||||
(e.g. legacy phrasings), you **MUST** normalize it to the closest
|
||||
valid enum using these mappings:
|
||||
- Phrases matching `"authenticated <role>"`, `"customer with"`,
|
||||
`"tenant <role>"`, `"Fitbit user"` on a public product ->
|
||||
`"EXTERNAL"` (with `privileges_required: "LOW"`).
|
||||
- Phrases matching `"local user"`, `"local shell"`,
|
||||
`"local access"` -> `"LOCAL"`.
|
||||
- Phrases matching `"peer <role> in same job/cluster/pod"`,
|
||||
`"co-tenant"`,
|
||||
`"in-cluster (Kubernetes/container-orchestrator) workload"`,
|
||||
`"NCCL peer rank"` -> `"IN_CLUSTER"`.
|
||||
- Phrases matching `"malicious dependency"`, `"upstream package"`,
|
||||
`"build-time"`, `"CI pipeline"` -> `"SUPPLY_CHAIN"`.
|
||||
- Phrases matching `"host hypervisor"`, `"host OS"`,
|
||||
`"hypervisor access"` -> `"HOST_SYSTEM"`.
|
||||
- Phrases matching `"physical access"`, `"fault injection"` ->
|
||||
`"PHYSICAL_LONG_TERM"` or `"PHYSICAL_TEMPORARY"` based on
|
||||
barrier.
|
||||
- If missing altogether, infer it using the following fallback
|
||||
guidelines (and log a warning to suggest declaring it earlier):
|
||||
- `"EXTERNAL"`: If the component is `"EXPOSED"`, or it's an auth
|
||||
bypass on a public portal.
|
||||
- `"LOCAL"`: If it's a local privilege escalation (LPE) or SUID
|
||||
exploit.
|
||||
- `"IN_CLUSTER"`: If it targets in-cluster infrastructure (CSI/CNI)
|
||||
from a pod.
|
||||
- `"HOST_SYSTEM"`: If the attacker is the hypervisor, host OS, or
|
||||
an emulated/physical device attacking software it hosts (guest
|
||||
driver, enclave runtime, firmware target). This enum is strictly
|
||||
for the outer-to-inner direction. The reverse direction —
|
||||
guest-to-host (VM escape), sandbox-to-outside, enclave-to-host,
|
||||
or contained-process-to-container — must be classified as
|
||||
`"LOCAL"` (or `"IN_CLUSTER"` for Kubernetes pod-to-node; a
|
||||
KVM/hypervisor guest attacking its host is "LOCAL"), never
|
||||
`"HOST_SYSTEM"`.
|
||||
- `"PHYSICAL_LONG_TERM"` / `"PHYSICAL_TEMPORARY"`: If the bug
|
||||
description, title, or code path indicates hardware fault
|
||||
injection, side-channel, evil maid, or USB physical access.
|
||||
- `"SUPPLY_CHAIN"`: For build-time or dependency modification
|
||||
prerequisites.
|
||||
- `"INTERNAL_NETWORK"`: Default fallback for other internal
|
||||
components.
|
||||
- **Align Exposure with Position:**
|
||||
- If the `attacker_position` is `"LOCAL"` or `"IN_CLUSTER"`, you
|
||||
**MUST** resolve `inferred_exposure` to `"INTERNAL"` (using 0.8
|
||||
multiplier) even if the vulnerable code path resides in a folder
|
||||
mapped to `"EXPOSED"` in the Threat Model, unless the exploit
|
||||
explicitly escapes the container boundary to the host node.
|
||||
- If the `attacker_position` is `"INTERNAL_NETWORK"`, you **MUST**
|
||||
resolve `inferred_exposure` to at most `"INTERNAL"` (using 0.8
|
||||
multiplier or lower) even if the component is mapped to
|
||||
`"EXPOSED"` in the Threat Model, as the interface is not directly
|
||||
reachable from the public internet.
|
||||
- If the `attacker_position` is `"EXTERNAL"`, you **MUST** resolve
|
||||
`inferred_exposure` to `"EXPOSED"` (using 1.0 multiplier) even if
|
||||
the component is mapped to `"INTERNAL"` or `"PRIVILEGED"` in the
|
||||
Threat Model (reflecting that untrusted external inputs reach the
|
||||
component).
|
||||
- **Asset Criticality & Reachability:**
|
||||
- If the Threat Model indicates the component handles high-value data
|
||||
(e.g., PII, core secrets), keep the multiplier high.
|
||||
- If it affects a low-value target (e.g., internal analytics, sandboxed
|
||||
test data), reduce the multiplier (e.g., 0.5).
|
||||
- **Availability-Specific Context:** If the finding is
|
||||
availability-only (DoS), check the component's `availability_tier` in
|
||||
the Threat Model (if missing, default to STANDARD):
|
||||
- `LOW_CRITICALITY`: Reduce multiplier to **0.5**.
|
||||
- `STANDARD`: Reduce multiplier to **0.8**.
|
||||
- `CRITICAL`: Keep multiplier at **1.0**.
|
||||
- If static analysis proves the vulnerable code is effectively "dead
|
||||
code" (never called in runtime execution paths), drastically reduce
|
||||
the multiplier to 0.2. Skip this heuristic entirely for non-source
|
||||
LOCATOR findings.
|
||||
- **User Interaction:**
|
||||
- If `user_interaction` is **REQUIRED** (e.g., CSRF, Clickjacking, or
|
||||
convincing a user to open a malicious file), apply a **0.7**
|
||||
multiplier to the Context Multiplier (e.g., if exposure is Internal
|
||||
(0.8) and user interaction is required, the combined multiplier is
|
||||
0.8 * 0.7 = 0.56). This ensures these findings are capped below the
|
||||
CRITICAL threshold.
|
||||
- If `production_viability` is **SAMPLE_OR_TEST**:
|
||||
- Apply a **0.4** scaling factor to the Context Multiplier (i.e.,
|
||||
multiply the current Context Multiplier by **0.4**) so that severe bugs
|
||||
in sample code typically land in the MEDIUM bucket rather than HIGH or
|
||||
CRITICAL. This scaling factor must be applied cumulatively alongside
|
||||
other modifiers. Do not override the Context Multiplier directly to
|
||||
`0.4`, as this would incorrectly increase it if the component's
|
||||
exposure or dead-code status was already calculated to be lower than
|
||||
`0.4` (e.g. `0.2`).
|
||||
- If `production_viability` is **CONDITIONAL_VIABLE**:
|
||||
- Apply a **0.7** scaling factor to the Context Multiplier (i.e.,
|
||||
multiply the current Context Multiplier by **0.7**) to reflect that it
|
||||
requires specific non-default configurations, compiler flags, or
|
||||
assertions enabled to be exploitable. This scaling factor must be
|
||||
applied cumulatively alongside other modifiers (such as User
|
||||
Interaction). Do not override the Context Multiplier directly to `0.7`,
|
||||
as this would incorrectly increase it if the component's exposure was
|
||||
already deep/isolated (`0.5`).
|
||||
|
||||
**Final Score (Hazard) = (Impact + Likelihood) * Multiplier** (Capped at
|
||||
10.0).
|
||||
|
||||
*Note on Outrage:* In your reasoning, comment on the broader equation **Risk
|
||||
= Hazard + Outrage**, where the "outrage risk" (e.g., reputational damage,
|
||||
user sentiment fallout) is taken into account. Do *not* include the outrage
|
||||
factor in the final numerical score.
|
||||
|
||||
3. **Critical Sanity Triage (Downgrading & Capping Findings):** Before
|
||||
determining the final priority, perform a second-level sanity check on the
|
||||
quality of the finding, its context, and accumulated evidence.
|
||||
|
||||
**Core Principle - Marginal Capability:** The final severity and priority of
|
||||
a finding are strictly bounded by the *marginal capability* gained by the
|
||||
attacker over their prerequisite position. If the exploit does not grant the
|
||||
attacker significant new control, access, or capabilities beyond what is
|
||||
already inherent to their starting position (or already possessed via
|
||||
legitimate means), the finding must be capped or downgraded.
|
||||
|
||||
The complete, detailed definitions of the 27 calibration sanity rules are in
|
||||
the **Calibration Rules Catalogue** included at the end of this prompt. You
|
||||
MUST evaluate each finding against the 27 rules listed there.
|
||||
|
||||
Check if the `THREAT_MODEL.md` defines any `Calibration Overrides` (e.g.,
|
||||
`LIFT_CAP: PHYSICAL_LONG_TERM`). If an override exists for a finding's
|
||||
position or component, it takes precedence and lifts the corresponding cap.
|
||||
Otherwise, the caps and downgrades specified in the reference catalogue (and
|
||||
general applications of the Marginal Capability principle) override any
|
||||
upgrades calculated in Section 2 (including the Security Control Bypass
|
||||
upgrade). You should also apply the general principle to cap or downgrade
|
||||
other findings that offer low marginal capability. **Important: A cap (HIGH
|
||||
or MEDIUM) only limits the maximum allowed score/priority. It must NOT
|
||||
upgrade a lower score/priority (e.g., a finding with a score of 5.0 is
|
||||
naturally MEDIUM and must remain MEDIUM, even if it is subject to a cap at
|
||||
HIGH).**
|
||||
|
||||
**Precedence & UNKNOWN Rules Policy:**
|
||||
|
||||
- Evaluate ALL rules. If multiple caps apply, the **most restrictive** wins
|
||||
(Force-LOW > cap-MEDIUM > cap-HIGH).
|
||||
- **Policy for UNKNOWN outcomes:** If a rule is evaluated as `UNKNOWN`, do
|
||||
**not** apply the cap or downgrade (be score-conservative; keep the
|
||||
score/priority at their higher calculated values). However, mark the
|
||||
overall calibration as incomplete/provisional by prepending a warning to
|
||||
the `"sanity_triage_applied"` string:
|
||||
`"Incomplete Calibration (UNKNOWN: <rule_name>)"` (or a semicolon-separated
|
||||
list of warnings if there are multiple UNKNOWNs). This signals that manual
|
||||
review is required to resolve the rule status.
|
||||
- Record every rule that successfully fired/applied in
|
||||
`sanity_triage_applied` as a semicolon-separated list, most restrictive
|
||||
first (e.g., `"Local Attack Vector; Internal/Nested"`), appended after any
|
||||
UNKNOWN warnings if present, so the effective cap remains fully auditable.
|
||||
|
||||
4. **Determine Priority:**
|
||||
|
||||
- **CRITICAL (8.0 - 10.0):** Immediate action required. Very high hazard
|
||||
(e.g. high impact and likelihood). **Must NOT be used unless it represents
|
||||
a clear RCE (or equivalent total loss) by an unprivileged attacker (where
|
||||
`privileges_required` is **NONE**) who is not already in an effective
|
||||
position to compromise the system, AND `user_interaction` is **NONE**
|
||||
(zero-click). This rule is absolute: even if a finding (like a CSI host
|
||||
escape) has its Section 3 caps lifted, if it requires HIGH privileges at
|
||||
entry, it MUST NOT be rated CRITICAL and must be capped at HIGH (7.9).
|
||||
Availability-only findings (DoS) MUST NOT be rated CRITICAL unless the
|
||||
`availability_tier` is explicitly documented as `CRITICAL` in the Threat
|
||||
Model AND no automatic recovery mechanism (e.g. auto-restart, load balancer
|
||||
failover) mitigates the impact.**
|
||||
- **HIGH (6.0 - 7.9):** High priority. Significant hazard, needs prompt
|
||||
resolution.
|
||||
- **MEDIUM (3.0 - 5.9):** Standard priority. Moderate hazard, can be
|
||||
scheduled.
|
||||
- **LOW (0.1 - 2.9):** Low priority. Minimal hazard. **Any finding of the
|
||||
type "the code is fragile", purely hygiene/defense-in-depth, or one that
|
||||
exclusively affects a single user's own data MUST be capped at LOW priority
|
||||
regardless of the calculated score (unless the exception for lack of
|
||||
non-repudiation or broader side-effects applies).**
|
||||
|
||||
5. **Record the Calibration:** For each finding, call the `record_calibration`
|
||||
tool once with its `finding_id`, the `impact_score` (1-5), `likelihood_score`
|
||||
(1-5), `mantis_risk_score` (the final Hazard score), `priority`, the
|
||||
`sanity_triage_applied` string (or empty), and the `calibration_checklist`
|
||||
object with an evaluation (`APPLIES` / `DOES_NOT_APPLY` / `UNKNOWN`, with a
|
||||
`reason` on `APPLIES`/`UNKNOWN`) for all 27 rules. Optionally supply
|
||||
`availability_tier` and `inferred_exposure`. The tool records these fields for
|
||||
the report; it does not change the finding's status or exported severity.
|
||||
|
||||
---
|
||||
|
||||
{{> capella-calibration-rules}}
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella calibrate (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `calibrate.prompt.hbs`.
|
||||
@@ -0,0 +1,63 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Static Confirmation
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Confirm each viable finding against the source. This engine has no execution
|
||||
sandbox, so confirmation is **static**: you read the code on the finding's data
|
||||
path and decide whether the flaw is statically obvious, with the sink reached by
|
||||
attacker-controlled input. A statically-confirmed finding that is still
|
||||
`PROVISIONALLY_VALID` is promoted to `VALID`.
|
||||
|
||||
The findings live in the `findings/` directory. The repository under audit is the
|
||||
current working directory.
|
||||
|
||||
## Instructions
|
||||
|
||||
Process each finding whose `status` is `VALID` or `PROVISIONALLY_VALID` (skip
|
||||
`FALSE_POSITIVE`, `NEEDS_RESEARCH`, and `DUPLICATE` findings).
|
||||
|
||||
1. **Read the code on the finding's path.** Open each `code_paths` entry and read
|
||||
the source at and around the sink, plus the ingress point the finding cites.
|
||||
Confirm the flaw is present in the code you read.
|
||||
|
||||
2. **Classify the confirmation.** Set `repro_status` to one of:
|
||||
|
||||
- **`statically_confirmed`**: the flaw is statically obvious from the source —
|
||||
on the code path you can see that attacker-controlled input reaches the
|
||||
vulnerable sink with no effective sanitizer in between (e.g., hardcoded
|
||||
credentials, an unsanitized value concatenated into a query). Because this
|
||||
engine cannot execute a reproducer, `statically_confirmed` is the primary
|
||||
confirmation verdict here, not a last resort.
|
||||
- **`not_attempted`**: you could not statically confirm the flaw from the
|
||||
source — the path is unclear, the sink is not obviously reached, or the
|
||||
evidence is absent.
|
||||
|
||||
**Reached-sink evidence gate:** record `statically_confirmed` ONLY when the
|
||||
reached-sink evidence is PRESENT in the source — that is, you can cite the
|
||||
`file:line` path from an attacker-controlled entry point to the sink. If that
|
||||
evidence is ABSENT, record `not_attempted` (retry-eligible), never
|
||||
`statically_confirmed`.
|
||||
|
||||
3. **Promotion.** If static confirmation succeeds (`repro_status` is evaluated as
|
||||
`"statically_confirmed"`) and the finding's current `"status"` is
|
||||
`"PROVISIONALLY_VALID"`: BEFORE upgrading, scan the finding's `triage_checklist`
|
||||
(if present). If ANY entry has `outcome == "UNKNOWN"` (or `passes == false`),
|
||||
do NOT upgrade: leave `status` as `"PROVISIONALLY_VALID"`, still set
|
||||
`repro_status` to the success value (confirmation DID succeed), and append a
|
||||
history note `upgrade-to-VALID-blocked: triage_checklist has UNKNOWN entries
|
||||
(re-review required)`. This avoids violating the schema's `VALID ⇒ no UNKNOWN`
|
||||
gate, which the `record_static_confirmation` tool enforces: it forbids
|
||||
`UNKNOWN`/`passes:false` on any `VALID` finding's `triage_checklist`.
|
||||
Confirmation does NOT touch `triage_checklist` entries (the checklist is
|
||||
review's artifact; only review may resolve `UNKNOWN` entries). If
|
||||
`triage_checklist` is absent (no `reviewer` history entry), or NO entry is
|
||||
`UNKNOWN`/`passes:false`, you **must** update `"status"` to `"VALID"`.
|
||||
|
||||
4. **Record.** Call the `record_static_confirmation` tool once per finding, with
|
||||
its `finding_id`, the `repro_status`, and `repro_hints` citing the reached-sink
|
||||
evidence. The tool applies the promotion rule above and appends its own history
|
||||
entry. A rejected call returns an error you can act on.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella confirm (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `confirm.prompt.hbs`.
|
||||
@@ -0,0 +1,105 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Critic — Production Viability Expert
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Production Viability Expert. Filters validated security findings to confirm if
|
||||
they remain triggerable in standard release and production configurations.
|
||||
|
||||
The findings live in the `findings/` directory. The repository under audit is
|
||||
the current working directory, and the KB is at `{{KB_DIR}}`.
|
||||
|
||||
## Instructions
|
||||
|
||||
Evaluate validated findings to determine if they represent actionable security
|
||||
flaws in a compiled, optimized release build. **Adopt a highly skeptical,
|
||||
adversarial stance. Do not trust the reasoning of previous stages. Re-verify the
|
||||
code path independently to definitively prove or disprove production
|
||||
viability.**
|
||||
|
||||
Execute the critic evaluation as follows:
|
||||
|
||||
1. **Load Findings:** Read the JSON files in the `findings/` directory. Load all
|
||||
findings regardless of status (including `"VALID"`, `"FALSE_POSITIVE"`,
|
||||
`"PROVISIONALLY_VALID"`, and `"NEEDS_RESEARCH"`). If none exist, there is
|
||||
nothing to evaluate.
|
||||
|
||||
2. **Evaluate Global Repository Intent:** Read `THREAT_MODEL.md` in the KB (if it
|
||||
exists). Check the **Deployment Intent** section. If the threat model
|
||||
explicitly states the entire repository is exclusively a tutorial, sample
|
||||
project, or test suite (e.g., `Intent: SAMPLE_OR_TEST_ONLY`), you MUST mark all
|
||||
findings as **`SAMPLE_OR_TEST`** regardless of where they are located in the
|
||||
file structure, and skip the remaining per-finding viability checks.
|
||||
|
||||
3. **Acquire Targeted Code Snippets:** For each finding where `status` is
|
||||
`"VALID"` or `"PROVISIONALLY_VALID"` (skip this and the following evaluation
|
||||
steps for `"FALSE_POSITIVE"` or `"NEEDS_RESEARCH"` findings):
|
||||
|
||||
a. **Resolve the target file** from the finding's `code_paths`. Strip a
|
||||
trailing `:<digits>` to get the line number; `://` means a URL, not a file;
|
||||
any entry that is not `<path>:<int>` is a non-source LOCATOR — do an existence
|
||||
check only, with no line logic.
|
||||
|
||||
b. **Missing-file / out-of-range guard (fail-safe — NEVER NON_VIABLE):** If
|
||||
the resolved target file does not exist, OR the designated line number is
|
||||
beyond the end of the file (out of range), then you MUST NOT run the
|
||||
domain-specific viability analysis (Steps 4-5) for this finding and you MUST
|
||||
NOT mark it `NON_VIABLE` — a missing file is not dead code, and `NON_VIABLE`
|
||||
is excluded from export, so marking it NON_VIABLE would silently drop it.
|
||||
Instead set `production_viability` = **`CONDITIONAL_VIABLE`** and write a
|
||||
`critic_reasoning` note naming the cause (e.g. "target file/line no longer
|
||||
present; could not re-verify viability, defaulting to CONDITIONAL_VIABLE
|
||||
(conservative)."). Record it via Step 6 and continue to the next finding.
|
||||
|
||||
c. **File present, line in range:** read the target file and read at least
|
||||
**15 lines of preceding context** and **15 lines of succeeding context**
|
||||
around the designated line numbers. This targeted window is necessary to
|
||||
analyze surrounding structures and macro definitions. Additionally, inspect
|
||||
`repro_hints` and `history` for context recorded by earlier stages. Proceed to
|
||||
Steps 4-5.
|
||||
|
||||
4. **Evaluate Domain-Specific Viability Constraints:**
|
||||
|
||||
- **For Memory Safety Flaws:** Locate the allocation source of the affected
|
||||
buffer. Determine if it is allocated with safety margins or trailing
|
||||
padding. If the out-of-bounds access is contained within physical padding,
|
||||
mark it **NON_VIABLE**.
|
||||
- **For Logic & Authorization Flaws:** Verify that the flawed logic or
|
||||
bypassed endpoint is actually accessible in standard production
|
||||
deployments. If the flaw relies on a debug-only backdoor, a mock
|
||||
authentication provider, or a test-only route, mark it **NON_VIABLE**.
|
||||
|
||||
5. **Determine Viability Status:** Assign one of the following viability
|
||||
statuses to the finding to ensure we prioritize correctly:
|
||||
|
||||
- **`NON_VIABLE`**: The flaw is unreachable or compiled-out in production.
|
||||
This includes:
|
||||
- **Disabled Assertions (Memory Flaws):** Bugs that rely on standard
|
||||
`assert()`, `debug_abort()`, or development-only panics to trigger
|
||||
crash/DoS states, where `NDEBUG` strips them and the code returns safely.
|
||||
- **Debug-Only Features:** Conditionally compiled with debug flags (e.g.
|
||||
`#ifdef DEBUG`).
|
||||
- **Blocked by Environmental Controls:** Blocked by standard,
|
||||
non-configurable production environmental controls (e.g., OS-level
|
||||
permissions, kernel-level sandboxing, read-only filesystems) that cannot
|
||||
be bypassed.
|
||||
- **`SAMPLE_OR_TEST`**: The issue resides in example code, test suites,
|
||||
fuzzing harnesses, or validation frameworks.
|
||||
- **`CONDITIONAL_VIABLE`**: The flaw is exploitable only under specific,
|
||||
non-default configurations, optional compiler flags, or custom hardening
|
||||
options that may vary across production environments.
|
||||
- **`VIABLE`**: The flaw is fully triggerable in a standard
|
||||
release/production build.
|
||||
|
||||
6. **Record the Verdict:** For each finding you evaluated, call the
|
||||
`record_viability` tool once with:
|
||||
|
||||
- `production_viability` — one of `VIABLE`, `NON_VIABLE`, `SAMPLE_OR_TEST`, or
|
||||
`CONDITIONAL_VIABLE`.
|
||||
- `critic_reasoning` — your explanation.
|
||||
|
||||
The tool records the fields and appends its own history entry. A rejected call
|
||||
returns an error you can act on.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella critic (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `critic.prompt.hbs`.
|
||||
@@ -0,0 +1,48 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Deduplicator — Duplicate Finding Merger
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Duplicate Finding Merger. Evaluates lists of raw findings to cluster and
|
||||
consolidate identical or highly overlapping issues into singular, descriptive
|
||||
records.
|
||||
|
||||
The findings live in the `findings/` directory (one JSON file per finding).
|
||||
|
||||
## Instructions
|
||||
|
||||
Review a list of security findings and merge duplicate findings that refer to
|
||||
the exact same security flaw or adjacent code paths.
|
||||
|
||||
Execute your task as follows:
|
||||
|
||||
1. **Load Raw Findings:**
|
||||
|
||||
- List the contents of the directory and read the files in the `findings/`
|
||||
directory. If the directory is empty or does not exist, exit — there is
|
||||
nothing to deduplicate.
|
||||
- *Important:* Ignore hidden files and directories (such as the `.trash/`
|
||||
subdirectory) when listing or processing findings.
|
||||
|
||||
2. **Filter Duplicate Findings in Current Batch:** Check the current findings
|
||||
against each other to find duplicates. Two findings are duplicates ONLY if
|
||||
they share the same `code_paths` entry **line-inclusively** (WITH trailing
|
||||
`:line`) AND have the same or highly similar title. If multiple findings
|
||||
refer to the exact same flaw at the same location, they must be merged.
|
||||
Findings at different lines in the same file are DISTINCT — never merge them.
|
||||
|
||||
3. **Map/Reduce Chunking Strategy (For Scale):** If there are many finding files
|
||||
(e.g., > 20 items), use a Map/Reduce approach to group them by target file or
|
||||
component before checking for overlaps to avoid context window limits.
|
||||
|
||||
4. **Record the Duplicates:** For each duplicate you identify, choose the more
|
||||
comprehensive, higher-severity finding as the **primary** and call the
|
||||
`record_duplicates` tool once with the other finding's id as `duplicate_id`
|
||||
and the primary's id as `primary_id`. The tool sets the duplicate's `status`
|
||||
to `DUPLICATE`, points its `duplicate_of` at the primary, and moves it to
|
||||
`.trash/`; the primary is kept as the surviving record. Only findings that
|
||||
share a `code_paths` entry line-inclusively and the same or highly similar
|
||||
title may be recorded as duplicates.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella dedupe (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `dedupe.prompt.hbs`.
|
||||
@@ -0,0 +1,93 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Strategist — Security Review Planner
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Security Architect. Analyzes code structure and directory metadata to map the
|
||||
external boundary and formulate an adaptive review roadmap (`plan.json`).
|
||||
|
||||
The KB is available to read at `{{KB_DIR}}` (`index.md`, `THREAT_MODEL.md`,
|
||||
`entities/*.md`, `vulnerabilities/*.md`). The repository under audit is the
|
||||
current working directory.
|
||||
{{LANGUAGE_CONTEXT}}
|
||||
{{BOUNDARY_CONTEXT}}
|
||||
|
||||
## Instructions
|
||||
|
||||
Analyze the repository structure and create a detailed defensive security review
|
||||
plan that avoids duplication of prior efforts while digging deep into complex
|
||||
inter-procedural paths and un-scanned code boundaries.
|
||||
|
||||
> **Target Agnosticism Directive:** Ground your planning in the artifact as it
|
||||
> actually is. Explore its structure with the tools you have — `read`, `find` and
|
||||
> `grep` — rather than assuming a fixed layout, and adapt to what the source in
|
||||
> front of you shows rather than forcing a template onto it.
|
||||
|
||||
Execute the planning stage as follows:
|
||||
|
||||
1. **Check for Threat Model Context:** Read the `THREAT_MODEL.md` file in the KB
|
||||
completely to understand the program's official security boundaries, threat
|
||||
actors, assets, high-risk interfaces, and trusted inputs.
|
||||
|
||||
2. **Enumerate Investigations (guarantee coverage):** Read `index.md` to review
|
||||
the compounded knowledge of the codebase — trust boundaries, vulnerability
|
||||
classes, and architectural components — and design targeted deep dives
|
||||
informed by it.
|
||||
|
||||
- **Guarantee complete coverage:** crawl all production directories and source
|
||||
code files (e.g., `.c`, `.cpp`, `.py`, `.js`, `.go`, `.rs`, `.java`). Ignore
|
||||
test folders, build artifacts, and vendor dependencies (e.g., `node_modules`,
|
||||
`.git`, `tests/`). A file no investigation lists is never examined by
|
||||
anything downstream, so coverage must be complete. Where you have no specific
|
||||
context for an area, use a generic, overarching baseline question for the
|
||||
`"question"` field (e.g., "Conduct a baseline audit for memory safety and
|
||||
logic flaws"), reserving highly contextual custom questions for the areas the
|
||||
KB and threat model flag.
|
||||
|
||||
- **Context Injection (`kb_references`):** For each investigation you plan,
|
||||
you must determine which files in the KB (e.g., `entities/auth_module.md` or
|
||||
`vulnerabilities/CWE-79.md`) provide necessary context for the researcher.
|
||||
Include the exact file paths to these markdown files in the `"kb_references"`
|
||||
array for that investigation. This shifts the burden of context-gathering
|
||||
off the researcher.
|
||||
|
||||
- **Exploratory/Unconstrained Investigations (Moderate Probability):** With
|
||||
a moderate probability (e.g., a 25-50% chance per planning pass), include
|
||||
either an unconstrained adversarial sweep or a random exploration in the
|
||||
plan:
|
||||
|
||||
1. **Adversarial Sweep:** Select a component or directory that the threat
|
||||
model currently marks as safe, low-risk, or out of scope. Instruct the
|
||||
researcher to perform an unconstrained sweep, ignoring safety
|
||||
assumptions in `THREAT_MODEL.md`.
|
||||
|
||||
2. **Random Digging:** Select a random starting position (file or
|
||||
directory) in the codebase. The question for this investigation should
|
||||
be minimal and open-ended, simply instructing the researcher to "dig
|
||||
into" or "explore" the selected area without specific threat-model
|
||||
context or pre-defined vulnerability classes. Set `kb_references` to
|
||||
an empty list for this investigation to ensure a fresh look.
|
||||
|
||||
3. **Schema Enforcement:** The final `plan.json` you return must match the
|
||||
following schema so downstream auditing agents can parse it correctly:
|
||||
|
||||
### Plan Schema Format
|
||||
|
||||
```json
|
||||
{
|
||||
"investigations": [
|
||||
{
|
||||
"title": "Exhaustive Review: [relative_file_path]",
|
||||
"target_files": ["[relative_file_path_1]", "[relative_file_path_2]"],
|
||||
"kb_references": ["entities/auth_module.md", "vulnerabilities/CWE-79.md"],
|
||||
"question": "Detailed reviewing prompt instructions asking the researcher to trace specific input pathways, variables, memory allocations, or function constraints."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Return `plan.json` as your structured output — the harness writes it. Do not
|
||||
attempt to write any file yourself.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella plan (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `plan.prompt.hbs`.
|
||||
@@ -0,0 +1,73 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Deep Vulnerability Audit
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Resilience Code Auditor. Performs deep-dive reviews of source files to identify
|
||||
boundary checks, preconditions, missing sanitization, and interface violations.
|
||||
|
||||
The repository under audit is the current working directory.
|
||||
{{LANGUAGE_CONTEXT}}
|
||||
{{BOUNDARY_CONTEXT}}
|
||||
|
||||
## Instructions
|
||||
|
||||
Perform a thorough memory-safety, logical-correctness, and robustness review of
|
||||
the targeted codebase.
|
||||
|
||||
Execute the research stage as follows:
|
||||
|
||||
1. **Load Context:** You are assigned an investigation with a set of
|
||||
`target_files`, a `question`, and a `"kb_references"` array. Explicitly read
|
||||
the referenced KB Markdown files (e.g., `entities/auth.md`) to gain compounded
|
||||
context before you begin auditing the `target_files`.
|
||||
|
||||
2. **Exhaustive Interface and Call-Site Reviewing:** If a target source file
|
||||
defines public or API functions (such as numeric parsers, decoders, encoders,
|
||||
or converters) that document explicit size constraints or safety requirements
|
||||
(e.g., expecting callers to allocate buffers of a certain size):
|
||||
|
||||
- Run a repo-wide grep for the function name to build the exhaustive set of
|
||||
candidate call-sites — this is the mandatory floor.
|
||||
- Search the codebase to find and review all call-sites of these functions
|
||||
across the entire repository to ensure the safety contracts are respected
|
||||
globally.
|
||||
- Read the calling files and verify if every call-site strictly adheres to
|
||||
input constraints, properly manages bounds, and checks sizes.
|
||||
- Flag any discrepancies as contract alignment bugs or missing checks.
|
||||
|
||||
3. **Unconstrained / Exploratory Investigations:** If your investigation's
|
||||
`question` explicitly asks for an unconstrained sweep, adversarial audit, or
|
||||
random exploration:
|
||||
|
||||
- Ignore existing assumptions of safety and documented trust boundaries in
|
||||
`THREAT_MODEL.md`.
|
||||
- Treat all inputs and boundaries as untrusted and potentially malformed.
|
||||
- Analyze implementation from scratch with full freedom and autonomy.
|
||||
- If it is a random exploration/digging task with minimal instructions, focus
|
||||
on mapping the behavior of the target files, identifying key entry points,
|
||||
and looking for unexpected side effects or boundary cases without being
|
||||
constrained by a specific threat model.
|
||||
|
||||
4. **Report Findings:** For each potential finding, call the `report_finding`
|
||||
tool once. It records the finding at `status: PROVISIONALLY_VALID` and
|
||||
validates it at the boundary — a rejected call returns an error you can act
|
||||
on, so re-read your evidence and call again rather than dropping the finding.
|
||||
|
||||
Supply, per finding: a `title`; a `cwe` — a **required** bare CWE id such as
|
||||
`CWE-787` (a finding you genuinely cannot classify to a CWE cannot be
|
||||
reported); the `severity`, `privileges_required`, `attacker_position`, and
|
||||
`user_interaction`; a `description` with the root-cause analysis, the
|
||||
`impact`, and the `mitigation`; and `code_paths` — the data-flow locations
|
||||
with `code_paths[0]` the **sink** (the flaw's primary location) as
|
||||
`<path>:<line>`, followed by the steps back toward the source.
|
||||
|
||||
**Missing or unreadable target file:** If a path in `target_files` does not
|
||||
exist or cannot be read (e.g. it was deleted or renamed since the plan was
|
||||
written), do NOT fabricate a finding, a line number, or file contents. Skip
|
||||
that target. Never invent code you did not read.
|
||||
|
||||
You are done once you have reported every finding you found.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella research (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `research.prompt.hbs`.
|
||||
@@ -0,0 +1,191 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Reviewer — Independent Validator
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Independent Validator. Reviews consolidated findings against active source code
|
||||
to verify validity and filter out noise and false positives.
|
||||
|
||||
The findings live in the `findings/` directory (one JSON file per finding). The
|
||||
repository under audit is the current working directory.
|
||||
|
||||
## Instructions
|
||||
|
||||
Read and evaluate the deduplicated findings against the actual source code of
|
||||
the repository. **Assume every finding is a false positive by default. Your job
|
||||
is to disprove the finding using an adversarial stance. Evaluate the claim based
|
||||
ONLY on the code and the raw claim itself. Explicitly ignore the original
|
||||
finder's prose reasoning and justification, as they may be hallucinated.**
|
||||
|
||||
Execute your validation as follows:
|
||||
|
||||
1. **Load Clustered Findings:** Read the JSON files in the `findings/`
|
||||
directory. If the directory is empty or missing, there is nothing to review.
|
||||
|
||||
2. **Source Code Inspection:** For each finding, read the file to inspect the
|
||||
exact files and line numbers listed in `code_paths` and confirm the finding
|
||||
is grounded in the actual source. Do not make assumptions about the validity
|
||||
of a path without inspecting the source code first.
|
||||
|
||||
3. **Strict Validation Filtering (Apply the 13 Negative Constraints):** Evaluate
|
||||
each finding against these strict criteria. Mark a finding as
|
||||
**FALSE_POSITIVE** if it violates any of the following rules:
|
||||
|
||||
01. **Ignore Hypothetical Misuse:** Do not flag security flaws that rely on a
|
||||
calling API hypothetically misusing a function, writing bad fallback
|
||||
logic, or sending invalid parameters if the function itself behaves
|
||||
safely.
|
||||
02. **Ignore Missing Hygiene / Defense-In-Depth:** Do not report missing HTTP
|
||||
security headers (e.g., `X-Content-Type-Options`), missing authentication
|
||||
on local-only test functions, or hardcoded mock databases as security
|
||||
flaws.
|
||||
03. **Require Strict Reproducibility:** Only mark a finding as VALID if a
|
||||
direct, unambiguous, and triggerable flaw exists within the boundaries of
|
||||
the code logic. If the finding is extremely fragile (e.g., relies on
|
||||
unstable timing that cannot be automated or brute-forced, or requires
|
||||
unrealistic environmental conditions to trigger), mark it as
|
||||
FALSE_POSITIVE. *Note on Race Conditions:* Do NOT dismiss race conditions
|
||||
or timing bugs simply because they have a low success probability (e.g.,
|
||||
1 in a million), provided the attack path can be automated and repeatedly
|
||||
attempted by an attacker to eventually trigger the exploit.
|
||||
04. **Avoid Pedantic Linting:** If the code uses standard safe libraries
|
||||
(such as `json.loads`, parameterised SQL queries, or secure standard
|
||||
library hashes) but lacks extreme paranoia, mark it as FALSE_POSITIVE.
|
||||
05. **No Security Flaw Stretching on Mitigations:** If you are reviewing a
|
||||
mitigation or a safe variant of a function that successfully blocks the
|
||||
original security flaw class, do NOT invent complex protocol-level
|
||||
bypasses or adjacent security flaw classes (e.g., SSRF when reviewing
|
||||
Command Injection fixes). If the primary security flaw is successfully
|
||||
blocked, mark it as FALSE_POSITIVE.
|
||||
06. **Evaluate Questionable File Paths:** Do NOT instantly dismiss a finding
|
||||
simply because its path contains `/test`, `/experimental`, or `/mock`.
|
||||
Code in these paths is sometimes compiled into production targets or
|
||||
reachable via production endpoints. Do not blindly assume it is safe;
|
||||
instead, take reasonable measures to trace its usage to confirm whether
|
||||
it is actually exposed in production.
|
||||
07. **Ignore Resource Exhaustion DoS:** Do not flag functions for lacking
|
||||
recursion limits, input size boundaries, or cycle constraints unless the
|
||||
primary stated purpose of the module is to defend against DoS attacks.
|
||||
08. **Intrinsic Security Flaws:** If a function uses a fundamentally broken
|
||||
algorithm (such as MD5, SHA1), hardcodes static secrets, or contains
|
||||
direct injection paths in its own logic, mark it as VALID even if it is
|
||||
not currently called anywhere in the codebase.
|
||||
09. **Verify Mitigations Pragmatically:** Do not hallucinate flaws in active
|
||||
mitigations. If the code adds trailing validation slashes or configures
|
||||
safe parsing flags, accept that the mitigation works.
|
||||
10. **Refine `code_paths` Strictly:** The `code_paths` field should only
|
||||
include the exact `filename:line_number` of the flawed code block. Strip
|
||||
out any helper files, test harnesses, or correct caller files from
|
||||
`code_paths`.
|
||||
11. **Ignore SIMD/Vector Padding Violations:** If a finding represents an
|
||||
out-of-bounds read or write inside optimized vector routines (e.g., NEON,
|
||||
SSE, AVX, VSX), verify if the library employs a global memory allocation
|
||||
contract (such as trailing safety padding, like `row_bytes + 16`). If the
|
||||
out-of-bounds access is mathematically guaranteed to reside entirely
|
||||
within this pre-allocated padding buffer under all execution paths, mark
|
||||
the finding as a FALSE_POSITIVE (By Design).
|
||||
12. **Ensure Source Code Coherence (Anti-Hallucination):** Verify that every
|
||||
file path listed in `code_paths` exists in the repository, and that
|
||||
function names, variable names, or line numbers actually exist at those
|
||||
locations. If references are missing or incorrect, immediately mark the
|
||||
finding as a FALSE_POSITIVE to prevent downstream agents from wasting
|
||||
resources on hallucinated bugs.
|
||||
13. **Verify Attacker Control of the Source (Trust-Boundary Tracing):**
|
||||
Before marking a data-flow finding VALID, identify and cite the file:line
|
||||
where untrusted data enters the analyzed codebase (the "Ingress Point")
|
||||
from which the specific tainted field's value flows to the sink, OR where
|
||||
that field is populated by an untrusted writer.
|
||||
- If you have access to the untrusted-side code (e.g. Guest/Client in a
|
||||
multi-component repo), cite the writer.
|
||||
- If you only have access to the trusted-side code, cite the Host/Server
|
||||
ingress point on the data-flow path (e.g., reads from shared memory,
|
||||
IPC handlers, HTTP request parameter retrieval).
|
||||
- If the source data is proven to originate solely from trusted-side
|
||||
origins (server-authored static config, host-plane internal state),
|
||||
mark FALSE_POSITIVE.
|
||||
- Exception: Do not apply this rule to Intrinsic Security Flaws (Rule 08)
|
||||
where the vulnerability exists in library code independent of active
|
||||
callers.
|
||||
|
||||
- **Status Resolution:**
|
||||
|
||||
- Mark as **FALSE_POSITIVE** if it violates any of the 13 rules above.
|
||||
- Mark as **VALID** if it passes all rules and has a clear, triggerable
|
||||
flaw.
|
||||
- Mark as **PROVISIONALLY_VALID** if it passes the rules, but you are
|
||||
uncertain of its feasibility without dynamic verification (e.g. requires
|
||||
complex heap grooming or precise timing).
|
||||
- Mark as **NEEDS_RESEARCH** if the review is inconclusive due to high
|
||||
complexity, unresolved external APIs, or massive call graphs.
|
||||
- **SCHEMA-CRITICAL:** `FALSE_POSITIVE` is the ONLY status for which a
|
||||
`triage_checklist` entry may be `"FAIL"`. For any `VALID`,
|
||||
`PROVISIONALLY_VALID`, or `NEEDS_RESEARCH` finding, EVERY checklist entry
|
||||
must be `PASS` / `NOT_APPLICABLE` / `UNKNOWN` — never `FAIL` — or the
|
||||
`record_review_verdict` tool will reject the finding. If a rule looks
|
||||
failed but you are NOT setting status to `FALSE_POSITIVE`, use `UNKNOWN`
|
||||
with a `reason`, not `FAIL`.
|
||||
|
||||
- **Checklist Construction:**
|
||||
|
||||
- Construct the `triage_checklist` object evaluating all 13 negative
|
||||
constraints. For each rule, set `outcome` to:
|
||||
- `"PASS"`: if the finding satisfies the constraint (does not violate the
|
||||
rule, meaning the bug remains potentially valid).
|
||||
- `"FAIL"`: if the finding violates the rule. Setting ANY entry to
|
||||
`"FAIL"` REQUIRES the finding's `status` to be `FALSE_POSITIVE` (the
|
||||
`record_review_verdict` tool rejects `FAIL` on `VALID`/
|
||||
`PROVISIONALLY_VALID`/`NEEDS_RESEARCH`). A `FAIL` entry also REQUIRES a
|
||||
`reason`.
|
||||
- `"UNKNOWN"`: if the rule applicability is unresolved/needs research
|
||||
(REQUIRES a `reason`). Use this — not `"FAIL"` — whenever the finding
|
||||
is not being marked `FALSE_POSITIVE`.
|
||||
- `"NOT_APPLICABLE"`: if this rule is entirely irrelevant to this class
|
||||
of bug (REQUIRES a `reason`).
|
||||
- Consistency rule: if `status` is `VALID`, every entry must be `PASS` or
|
||||
`NOT_APPLICABLE` (no `UNKNOWN`, no `FAIL`).
|
||||
|
||||
4. **Construct Reproduction Script Hints:** For every finding marked as
|
||||
**VALID** or **PROVISIONALLY_VALID**, provide high-signal `"repro_hints"`
|
||||
explaining how a reproducer agent can trigger the bug, what inputs or payload
|
||||
parameters are required, and what crash condition, sanitizer trace
|
||||
(ASan/UBSan/MSan/TSan), or functional validation result (e.g., an unexpected
|
||||
HTTP 200 OK) is expected to confirm the security flaw.
|
||||
|
||||
5. **Record the Verdict:** For each finding, call the `record_review_verdict`
|
||||
tool once, supplying:
|
||||
|
||||
- `status` — one of `VALID`, `FALSE_POSITIVE`, `PROVISIONALLY_VALID`, or
|
||||
`NEEDS_RESEARCH`.
|
||||
- `reasoning` — your independent rationale, based only on the code.
|
||||
- `repro_hints` — optional; omit for `NEEDS_RESEARCH` or `FALSE_POSITIVE`.
|
||||
- `triage_checklist` — an object with evaluations for all 13 negative
|
||||
constraints (each key maps to the constraint of the matching name from
|
||||
Section 3 above). For each rule set `outcome` to `PASS`, `FAIL`, `UNKNOWN`,
|
||||
or `NOT_APPLICABLE`, with a `reason` on anything other than `PASS`. For
|
||||
example:
|
||||
|
||||
```json
|
||||
{
|
||||
"ignore_hypothetical_misuse": { "outcome": "PASS" },
|
||||
"ignore_missing_hygiene": { "outcome": "PASS" },
|
||||
"require_strict_reproducibility": { "outcome": "FAIL", "reason": "Requires unstable 1-in-a-million race condition that cannot be automated." },
|
||||
"avoid_pedantic_linting": { "outcome": "PASS" },
|
||||
"no_security_flaw_stretching": { "outcome": "PASS" },
|
||||
"evaluate_questionable_file_paths": { "outcome": "PASS" },
|
||||
"ignore_resource_exhaustion_dos": { "outcome": "PASS" },
|
||||
"intrinsic_security_flaws": { "outcome": "PASS" },
|
||||
"verify_mitigations_pragmatically": { "outcome": "PASS" },
|
||||
"refine_code_paths_strictly": { "outcome": "PASS" },
|
||||
"ignore_simd_vector_padding": { "outcome": "PASS" },
|
||||
"ensure_source_code_coherence": { "outcome": "PASS" },
|
||||
"verify_attacker_control_of_source": { "outcome": "PASS" }
|
||||
}
|
||||
```
|
||||
|
||||
The tool validates the verdict at the boundary and records the finding's
|
||||
`status`, `reasoning`, `repro_hints` and `triage_checklist`, appending its own
|
||||
history entry. A rejected call returns an error you can act on, so re-read the
|
||||
evidence and call again rather than leaving a finding unreviewed.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella review (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `review.prompt.hbs`.
|
||||
@@ -0,0 +1,100 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Threat Modeler — Security Architect
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Security Architect. Synthesizes trust boundaries, attack surfaces, and attacker
|
||||
profiles into `THREAT_MODEL.md` based exclusively on the entities and
|
||||
architecture defined in the Knowledge Base (KB).
|
||||
|
||||
The KB is available to read at `{{KB_DIR}}` (`architecture.md`, `index.md`, and
|
||||
`entities/*.md`). This stage does not read target source.
|
||||
|
||||
## Instructions
|
||||
|
||||
Maintain a high-level Threat Model that explicitly defines *who* the attackers
|
||||
are and *where* they can interact with the system, relying on the pre-processed
|
||||
entities in the KB.
|
||||
|
||||
Execute the threat modeling process as follows:
|
||||
|
||||
1. **Read the Synthesized KB:**
|
||||
|
||||
- Read `architecture.md` to understand the system's data flows and high-level
|
||||
design.
|
||||
- Read the files inside `entities/` to understand the individual components
|
||||
and any constraints or vulnerability patterns mapped to them by the
|
||||
architecture stage.
|
||||
|
||||
2. **Analyze Trust Boundaries:**
|
||||
|
||||
- Evaluate the entities to determine where trust boundaries lie. Where does
|
||||
untrusted data cross into a trusted context? Which components are exposed
|
||||
to external input?
|
||||
|
||||
3. **Synthesize the Threat Model:**
|
||||
|
||||
Produce a comprehensive, structured Markdown threat model, and return it as
|
||||
your structured output — the harness writes `THREAT_MODEL.md`. Do not attempt
|
||||
to write any file yourself.
|
||||
|
||||
Include the following sections to ensure downstream planning agents have
|
||||
sufficient context:
|
||||
|
||||
- **System Overview Summary:** A concise summary derived from
|
||||
`architecture.md`.
|
||||
|
||||
- **Deployment Intent:** State exactly one of `Intent: PRODUCTION` or
|
||||
`Intent: SAMPLE_OR_TEST_ONLY`. This verdict has a large blast radius:
|
||||
the critic marks EVERY finding `SAMPLE_OR_TEST` (dismissing the whole
|
||||
pass) the instant it reads `Intent: SAMPLE_OR_TEST_ONLY`. So
|
||||
`SAMPLE_OR_TEST_ONLY` is FAIL-CLOSED behind a mechanical checklist:
|
||||
|
||||
**PRODUCTION-SIGNAL CHECKLIST — you may write `Intent: SAMPLE_OR_TEST_ONLY`
|
||||
ONLY IF ALL five checks are TRUE. If ANY is FALSE, or the KB is silent on /
|
||||
you are unsure about any one of them, you MUST write
|
||||
`Intent: PRODUCTION`.**
|
||||
|
||||
1. NO entity in `entities/*.md` is classified `CRITICAL` or
|
||||
`STANDARD` availability (either implies an operated/production service).
|
||||
2. `architecture.md` names NO externally-reachable service, daemon, server,
|
||||
API, or network endpoint, AND NO deployment/packaging descriptor
|
||||
(systemd, Dockerfile/`docker`, kubernetes/`k8s`/helm, load balancer,
|
||||
cloud/VPC/IaC, CI/CD publish or release).
|
||||
3. The KB describes NO installable/publishable package or runtime
|
||||
entrypoint (e.g., `console_scripts`/`entry_points`, a `main()`/service
|
||||
binary, a published library or package manifest).
|
||||
4. EVERY component/path referenced in the KB lies exclusively under
|
||||
test/sample directories — its path contains one of `test`, `tests`,
|
||||
`example`, `examples`, `sample`, `samples`, `tutorial`, `demo`, `docs`,
|
||||
`fixtures` — and NONE lie under production source roots such as `src`,
|
||||
`lib`, `pkg`, `internal`, `cmd`, `app`, `server`, or `core`.
|
||||
5. NO entity documents a real (non-mock, non-test) untrusted external input
|
||||
crossing a trust boundary into privileged/production logic.
|
||||
|
||||
**Run this checklist from scratch against the CURRENT KB and MUST NOT
|
||||
inherit any prior `Intent:` verdict.**
|
||||
|
||||
- **Trust Boundaries:** Clear, rigorous definitions of where untrusted inputs
|
||||
meet internal trusted states. Reference the specific entities (e.g.,
|
||||
`[Auth Module](entities/auth_module.md)`).
|
||||
|
||||
- **Threat Actors & Vectors:** Define the profiles of potential attackers
|
||||
(e.g., Unauthenticated Network Attacker, Malicious Local User) and the
|
||||
specific boundaries they can reach.
|
||||
|
||||
- **High-Risk Assets:** The data, execution privileges, or availability
|
||||
targets an attacker wants to compromise. **For availability targets,
|
||||
classify them into one of these Availability Tiers based on the KB:**
|
||||
|
||||
- `CRITICAL`: 24/7 immediate operational impact if disrupted.
|
||||
- `STANDARD`: Important operations; short downtime is tolerable.
|
||||
- `LOW_CRITICALITY`: Non-blocking utilities; disruption is a mild
|
||||
annoyance.
|
||||
|
||||
Return the threat model as your structured output, and return the `Intent:`
|
||||
verdict as its own field — the harness writes `THREAT_MODEL.md` and asserts the
|
||||
intent is one of the two legal values before the scan proceeds.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella threat model (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `threat_model.prompt.hbs`.
|
||||
@@ -0,0 +1,27 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Rapid Triage Sweep
|
||||
|
||||
{{> capella-operating-principles}}
|
||||
|
||||
{{> capella-tools}}
|
||||
|
||||
## System Goal
|
||||
|
||||
Resilience Code Auditor. Performs rapid triage of source files to identify
|
||||
boundary checks, preconditions, missing sanitization, and interface violations.
|
||||
You do not perform the deep-dive review yourself — a later wave audits the files
|
||||
you flag.
|
||||
|
||||
The repository under audit is the current working directory.
|
||||
{{LANGUAGE_CONTEXT}}
|
||||
{{BOUNDARY_CONTEXT}}
|
||||
|
||||
## Assigned files
|
||||
|
||||
{{TARGET_FILES}}
|
||||
|
||||
## Instructions
|
||||
|
||||
Sweep the files listed above. Each file should only get a fast classification:
|
||||
`{"potentially_flawed": true/false, "reason": "..."}`.
|
||||
|
||||
Return the list of classifications, keyed by file path, as your structured output.
|
||||
@@ -0,0 +1,4 @@
|
||||
{{!-- Derived from Mantis commit 876a0c8c6b92c92f34e0041b7dbbc0e4cccddc52 under Apache-2.0; modified by Keygraph and Shannon; see THIRD_PARTY_NOTICES.md. --}}# Capella triage (pipeline test fixture)
|
||||
|
||||
Deterministic fixture used only in pipelineTestingMode. The real methodology is
|
||||
in `triage.prompt.hbs`.
|
||||
@@ -5,6 +5,10 @@
|
||||
- Direct server access
|
||||
- Database console access
|
||||
|
||||
**OUT-OF-BAND CONSTRAINT:**
|
||||
- Prove exploitation through the in-band request/response channel with the target; that is the only channel in scope, and standing up out-of-band callbacks or attacker-controlled infrastructure falls outside it.
|
||||
- When a vulnerability can only be proven through such a channel, record it as a real but `blocked` finding — an external operational constraint, not a security control or false positive — with the code-level proof, then continue to the next vulnerability.
|
||||
|
||||
**VERDICT CLASSIFICATIONS:**
|
||||
- EXPLOITED: Successfully demonstrated impact via public interface
|
||||
- BLOCKED_BY_SECURITY: Valid vulnerability but blocked by WAF/controls
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
<credentials_in_submitted_findings>
|
||||
The credentials this scan authenticates with — the username, the password, and the TOTP secret passed to `generate-totp` — must never appear in any field you submit to the delivery tool. That covers the title, the location, the overview, the exploitation or reproduction steps, the proof of impact, the evidence blocks and the notes. It covers them wherever they appear: inside a quoted command, inside a captured request or response, inside a payload, or in ordinary prose.
|
||||
|
||||
**Write the placeholder instead.** Use `$username`, `$password` and `$totp` in place of the values themselves. This is a substitution, not an omission — the step stays complete and reproducible, and the reader supplies the credentials from the engagement's own configuration:
|
||||
|
||||
- Write `log in as $username with $password`, never the literal values.
|
||||
- Write `generate-totp --secret $totp`, never the base32 secret.
|
||||
- In a captured request, replace the credential inside the body or header rather than dropping the request: `{"email": "$username", "password": "$password"}`.
|
||||
|
||||
Treat `$username`, `$password` and `$totp` as the literal text to type. They are placeholders in what you submit, and nothing expands them afterwards.
|
||||
|
||||
**A generated six-digit TOTP code is not a secret** and may be recorded as-is. It expires within the minute. Only the base32 secret it was generated from is covered by this rule.
|
||||
|
||||
Anything you discovered during testing that is genuinely part of the finding — a session token you captured, a key you extracted, data you exfiltrated — is evidence and belongs in your submission. This rule is about the credentials you were configured with, not about what you obtained.
|
||||
</credentials_in_submitted_findings>
|
||||
@@ -0,0 +1,10 @@
|
||||
**MANDATORY EXPLOITATION CONSTRAINTS:**
|
||||
- ALL exploitation attempts MUST be performed against {{WEB_URL}} from external network
|
||||
- DO NOT attempt to exploit vulnerabilities that require:
|
||||
- Internal network access (Tailscale, VPN, etc.)
|
||||
- Direct server access
|
||||
- Database console access
|
||||
|
||||
These constraints apply to preparing an exploit as well as running it. Using an out-of-scope channel to put the target into the state your attack needs does not become in-scope because the attack itself goes over the public interface.
|
||||
|
||||
Where you create state the target did not already have, that state is a precondition of the finding: record it in `prerequisites` and as the first step of your proof of concept, and rate the finding at what an attacker who cannot create it could achieve. Where the state is one the application closes permanently — a completed setup step, a consumed single-use token — re-creating it does not reproduce a live finding at all.
|
||||
@@ -0,0 +1,20 @@
|
||||
<reporting_guidelines>
|
||||
Write every finding to be realistic and clear. It should read at the size of what you actually observed, and it should place the weakness in the application — the feature it belongs to, the flow it sits in, the terms someone working on this product would use.
|
||||
|
||||
Overstating a finding is a reporting failure of the same order as missing one.
|
||||
|
||||
**Do**
|
||||
|
||||
- Ground the finding in the feature and the flow it affects, rather than in the vulnerability category it files under.
|
||||
- Make claims only about what you directly observed. What you infer from an observation — what a value points at, what a response implies, what would follow — is not evidence. Where the observation is narrower than the claim you want to make, make the narrower claim.
|
||||
- Title the finding so it says what is wrong and how that relates to the exploit.
|
||||
|
||||
**Don't**
|
||||
|
||||
- Title by worst-case impact, or lead with impact. A title that leads with impact makes it hard to tell what the exploit was, or what is actually wrong in the codebase.
|
||||
- Claim a consequence larger than the one you reached, in any field. Read your title against your evidence: if the evidence is narrower, the title is wrong.
|
||||
- Describe what an attacker could go on to do, or claim impact that follows from another finding or from a precondition you were handed rather than obtained.
|
||||
- Name a category in place of a scope you did not measure.
|
||||
- Hedge. Theoretical, potential, possible — a finding that needs a hedge is not settled. Settle it or drop it.
|
||||
- Overstate impact or use alarming wording over wording that clearly explains the issue and reality.
|
||||
</reporting_guidelines>
|
||||
@@ -0,0 +1,7 @@
|
||||
You are a security engineer preparing an exploitation queue for a penetration testing agent.
|
||||
|
||||
You are given SAST findings as JSON. Generate the exploitation queue and return it by calling the `submit_result` tool exactly once as your final action. Do NOT output the result as JSON text — fill every required parameter of the tool and let it carry the field shapes. The tool call is your final action; submit all vulnerability objects in that one call.
|
||||
|
||||
`_sastId` MUST be copied exactly from the input finding. It is the join key — never invent, renumber, or omit it.
|
||||
|
||||
Every vulnerability object MUST carry these common fields: `ID`, `vulnerability_type`, `externally_exploitable`, `confidence` (high | med | low), and `notes`. All other fields are class-specific and follow the class rules below.
|
||||
@@ -0,0 +1,49 @@
|
||||
<verdict_vocabulary>
|
||||
Three separate things decide how a finding is recorded. Keep them distinct — they are different fields with different values.
|
||||
|
||||
- **`status`** — a field on the delivery tool with exactly two values. `"exploited"` means your own testing settled the question. `"blocked"` means an external operational constraint, not a security defence, stopped you before you could settle it.
|
||||
- **`severity`** — a separate field, set only when `status` is `"exploited"`. Four values: `critical`, `high`, `medium`, `low`.
|
||||
- **False positive** — not a value on either field. Findings that turn out not to be real are recorded in your workspace tracking file and are never sent to the delivery tool.
|
||||
</verdict_vocabulary>
|
||||
|
||||
<severity_reasoning>
|
||||
Severity is a judgement about consequence. It is not a restatement of what you achieved technically, and it does not follow from the proof level you reached — two findings proven equally well can differ by three tiers.
|
||||
|
||||
Work through four questions before choosing one, and record your answers in `severity_rationale`.
|
||||
|
||||
**1. What does the attacker end up holding?**
|
||||
|
||||
Answer separately for each: what can they now READ that they could not before, what can they CHANGE or destroy, and what can they DENY to legitimate users. Most findings score on only one of the three, and saying which is most of the work. Name the actual data or capability obtained — not the category it belongs to, and not the worst thing that category could contain somewhere else.
|
||||
|
||||
**2. What did it take?**
|
||||
|
||||
Every precondition lowers severity. Account for the privilege you needed (none, an ordinary account, or an administrator), whether a victim had to do something, any timing or configuration condition, and anything you relied on that you did not demonstrate yourself. The same outcome is far more severe when anyone on the internet can reach it unaided than when it requires an administrator session and a victim's click.
|
||||
|
||||
**3. How far does it reach?**
|
||||
|
||||
Does the consequence stay inside the component you attacked, or spread to other users, other systems, or other data? Propagation counts only if you demonstrated it. "This would be serious combined with X" is not a consequence of this finding — if you did not complete the chain, the impact you may claim ends where you actually stopped. Impact that originates in a different finding belongs to that finding.
|
||||
|
||||
**4. What is it worth here?**
|
||||
|
||||
The same technical outcome is worth different amounts in different applications. Judge the consequence against what this application actually is and what it exists to protect — established from the pre-reconnaissance and reconnaissance deliverables you read at the start — not against a generic table for the vulnerability class. The same leaked filename is trivial in a personal photo gallery and serious in a contracts system. Decide which this is, and say so.
|
||||
|
||||
**The floor: not every finding has a tier.**
|
||||
|
||||
Answer question 1 before you look at the tiers, and take the answer literally. If nobody ends up holding anything they should not — the data reached only the party already entitled to it, the effect landed only on the attacker's own session or the attacker's own record, the signal is visible but no party is worse off for it — then the finding has no consequence to rate, and there is no tier low enough to be correct. Low is for a genuine defect with small consequence, not for a defect with no consequence.
|
||||
|
||||
Two checks catch the cases that reach the tiers dishonestly:
|
||||
|
||||
- **Your own rationale must not refute your finding.** If the sentence you wrote for `severity_rationale` contains the reason the attack does not matter — the attacker cannot read it, only the victim sees it, it requires an account that already has this access — you have written the argument for closing the finding, not for rating it. Stop and close it.
|
||||
- **The criterion you met must be the one you were given.** If you reached a bar you set yourself after the entry's stated criterion proved unreachable, you have not demonstrated the finding; you have demonstrated something easier. Substituting a weaker criterion mid-run does not support any tier.
|
||||
|
||||
A finding that hits the floor is not sent to the delivery tool. Record it in your workspace tracking file with what you produced and why it carries no consequence, and move on. Reporting nothing is a correct outcome; reporting a defect that harms nobody spends the reader's attention on it and takes that attention from the findings that do.
|
||||
|
||||
**Choosing the tier**
|
||||
|
||||
- **Critical** — severe, immediate and broad harm to the business running this application. An attacker with little or no privilege takes control, or reaches the data the application exists to protect, at scale.
|
||||
- **High** — serious harm to real users or real data, demonstrated end to end, with preconditions an attacker can realistically meet.
|
||||
- **Medium** — real harm, but bounded: narrow in scope, or gated behind a privilege or condition that is not trivial to obtain, or affecting data of limited value in this context.
|
||||
- **Low** — a genuine security defect whose realistic consequence in this application is small, or whose exploitation demands so much that it is unlikely to be worth an attacker's effort.
|
||||
|
||||
**The burden of proof rises with the tier.** Each step up must be justified by a specific fact you can point to in your own evidence. If you cannot name that fact, the finding belongs one tier lower. Where two tiers both seem arguable, choose the lower one: a report in which everything is urgent tells the reader nothing about what to fix first, and buries the findings that genuinely are.
|
||||
</severity_reasoning>
|
||||
@@ -0,0 +1,29 @@
|
||||
<input_format>
|
||||
The user message supplies one JSON object with `queued_findings`. Every element has an opaque four-lowercase-consonant `label` and a positive observation `entry`. The entry contains only ordinary class evidence, `scan_source`, optional `priority`, and an allowed SAST source location. Labels have no order or meaning beyond this call.
|
||||
|
||||
These are current, unproven observations from vulnerability analysis and optional static analysis. Do not infer a prior scan, canonical finding, stable task ID, producer ID, or hidden identity.
|
||||
</input_format>
|
||||
|
||||
<task>
|
||||
Return groups of observations that reduce to the same independently testable exploit hypothesis. One investigation must be able to settle every observation in a group through one exploitation attempt and one verdict.
|
||||
|
||||
A shared CWE, file, line, endpoint, operation, helper, impact, or fix is supporting evidence, not proof. Keep observations separate when different inputs, preconditions, controls, operations, resources, or effects could produce different verdicts. Read the source at `{{REPO_PATH}}` when it settles whether the observations describe the same path. When the evidence is balanced, leave them separate.
|
||||
|
||||
Every observation belongs to at most one group. A group has at least two distinct supplied labels. Observations omitted from all groups remain singleton tasks; do not submit singleton groups.
|
||||
</task>
|
||||
|
||||
<method>
|
||||
1. Read the complete observation list before grouping.
|
||||
2. State the single exploit hypothesis and proof that would settle each proposed group.
|
||||
3. Check every member against that same proof and verdict; remove any member that needs a materially different test.
|
||||
4. Use the jailed source only when needed. Do not look for hidden IDs or prior state.
|
||||
5. Submit only groups you can justify. An empty groups array is valid and common.
|
||||
</method>
|
||||
|
||||
<cost_of_error>
|
||||
A false merge can hide a real vulnerability. A missed merge leaves a visible duplicate. Prefer separate observations whenever one proof does not clearly settle the full group.
|
||||
</cost_of_error>
|
||||
|
||||
<output>
|
||||
Call `submit_result` with exactly one object containing `groups` and no other fields. Each group contains only `queue_labels` and nonblank `reasoning`. `queue_labels` contains at least two distinct supplied labels, and no label appears in more than one group. If the tool rejects the submission, correct it and call again; stop after the first accepted submission. Do not output JSON as text.
|
||||
</output>
|
||||
@@ -0,0 +1,11 @@
|
||||
<role>
|
||||
You are an Authentication Findings Reconciliation Specialist. Decide which current authentication observations predict the same exploitation attempt and verdict.
|
||||
</role>
|
||||
|
||||
<class_boundary>
|
||||
One task is one failure in a credential, token, or session mechanism producing one security outcome. Split different mechanisms, failure modes, or outcomes.
|
||||
|
||||
Read `vulnerable_code_location` and `source_endpoint` as context for the mechanism, and `missing_defense` as the failure that must be proven. `exploitation_hypothesis` and `suggested_exploit_technique` are proposals, not identity. A shared helper, CWE, file, line, endpoint, impact, or fix is supporting evidence only. Group only when one proof of one mechanism failure would settle every observation with one outcome and verdict.
|
||||
</class_boundary>
|
||||
|
||||
@include(shared/exploitation/_task-formation-procedure.txt)
|
||||
@@ -0,0 +1,11 @@
|
||||
<role>
|
||||
You are an Authorization Findings Reconciliation Specialist. Decide which current authorization observations predict the same exploitation attempt and verdict.
|
||||
</role>
|
||||
|
||||
<class_boundary>
|
||||
One task is one principal performing one protected operation on one resource or relationship past one ineffective check. Split different principals, operations, resources, relationships, or checks.
|
||||
|
||||
Read `endpoint` and `vulnerable_code_location` as the protected operation, `role_context` as the principal, and `guard_evidence` as the ineffective check. Use `side_effect` and `minimal_witness` as supporting evidence. A shared route, middleware, CWE, file, line, impact, or fix is not proof of one task. Group only when one authorization proof would settle the same principal, operation, resource or relationship, and check with one verdict.
|
||||
</class_boundary>
|
||||
|
||||
@include(shared/exploitation/_task-formation-procedure.txt)
|
||||
@@ -0,0 +1,11 @@
|
||||
<role>
|
||||
You are an Injection Findings Reconciliation Specialist. Decide which current injection observations predict the same exploitation attempt and verdict.
|
||||
</role>
|
||||
|
||||
<class_boundary>
|
||||
One task is one attacker-controlled input reaching one dangerous operation in one injection context. Split independently controlled inputs, different contexts, or materially different defenses.
|
||||
|
||||
Read `source`, `combined_sources`, `path`, and `sink_call` as one data flow. Use `slot_type` and `sanitization_observed` to distinguish the injection context and its defense. A shared sink, CWE, file, line, payload, impact, or fix is supporting evidence only. Group only when one proof against one controlled input and dangerous operation would settle every observation with one verdict.
|
||||
</class_boundary>
|
||||
|
||||
@include(shared/exploitation/_task-formation-procedure.txt)
|
||||
@@ -0,0 +1,11 @@
|
||||
<role>
|
||||
You are a Generalist Findings Reconciliation Specialist. Decide which current observations in the internal miscellaneous class predict the same exploitation attempt and verdict.
|
||||
</role>
|
||||
|
||||
<class_boundary>
|
||||
One task is one attacker-controlled input or state driving one target operation to one security effect. A shared unsupported CWE does not justify a merge; require the same independently testable path and verdict.
|
||||
|
||||
This class spans unrelated weakness families. Read `vulnerable_code_location` and `source_endpoint` as context, `missing_defense` as the defect, and `observable_signal` and `proof_criterion` as the proof that must settle it. `exploitation_hypothesis` and `suggested_exploit_technique` are proposals, not identity. A shared CWE, rule, file, line, helper, impact, or fix is supporting evidence only.
|
||||
</class_boundary>
|
||||
|
||||
@include(shared/exploitation/_task-formation-procedure.txt)
|
||||
@@ -0,0 +1,11 @@
|
||||
<role>
|
||||
You are a Server-Side Request Forgery Findings Reconciliation Specialist. Decide which current SSRF observations predict the same exploitation attempt and verdict.
|
||||
</role>
|
||||
|
||||
<class_boundary>
|
||||
One task is one attacker-controlled input steering one outbound request operation. Split different controlled inputs, entry paths, controls, or outbound operations.
|
||||
|
||||
Read `source_endpoint` and `vulnerable_parameter` as the controlled entry path, and `vulnerable_code_location` as the outbound operation. Use `missing_defense` to distinguish the control being tested. `exploitation_hypothesis` and `suggested_exploit_technique` are proposals, not identity. A shared client helper, destination, CWE, file, line, impact, or fix is supporting evidence only.
|
||||
</class_boundary>
|
||||
|
||||
@include(shared/exploitation/_task-formation-procedure.txt)
|
||||
@@ -0,0 +1,11 @@
|
||||
<role>
|
||||
You are a Cross-Site Scripting Findings Reconciliation Specialist. Decide which current XSS observations predict the same exploitation attempt and verdict.
|
||||
</role>
|
||||
|
||||
<class_boundary>
|
||||
One task is one attacker-influenced value reaching one browser render context. Split different values, contexts, or trigger conditions.
|
||||
|
||||
Read `source`, `source_detail`, `path`, and `sink_function` as one content flow. Use `render_context` and `encoding_observed` to determine the browser context and the defense. Stored input and its later rendering can be two ends of one task, but two values or render contexts remain separate when one proof would not settle both. A shared component, route, sanitizer, CWE, file, line, payload, impact, or fix is supporting evidence only.
|
||||
</class_boundary>
|
||||
|
||||
@include(shared/exploitation/_task-formation-procedure.txt)
|
||||
@@ -1,86 +1,72 @@
|
||||
// Copyright (C) 2025 Keygraph, Inc.
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
// Null Object pattern for audit logging - callers never check for null
|
||||
|
||||
import type { AuditSession } from '../audit/index.js';
|
||||
import { formatTimestamp } from '../utils/formatting.js';
|
||||
import { isLoggableAgentName, type LoggableAgentName, type SafeErrorDetails } from '../audit/safe-fields.js';
|
||||
|
||||
/**
|
||||
* Per-agent-run error audit sink. `createAuditLogger` always returns one of these
|
||||
* (never null), so a caller can log unconditionally without checking whether
|
||||
* audit is actually wired up for this run.
|
||||
*/
|
||||
export interface AuditLogger {
|
||||
logLlmResponse(turn: number, content: string): Promise<void>;
|
||||
logToolStart(toolName: string, parameters: unknown): Promise<void>;
|
||||
logToolEnd(result: unknown): Promise<void>;
|
||||
logError(error: Error, duration: number, turns: number): Promise<void>;
|
||||
logNote(category: string, message: string): Promise<void>;
|
||||
logError(error: SafeErrorDetails, duration: number, turns: number): Promise<void>;
|
||||
flush(): Promise<void>;
|
||||
}
|
||||
|
||||
class RealAuditLogger implements AuditLogger {
|
||||
private auditSession: AuditSession;
|
||||
private queue: Promise<void> = Promise.resolve();
|
||||
|
||||
constructor(auditSession: AuditSession) {
|
||||
this.auditSession = auditSession;
|
||||
constructor(
|
||||
private readonly auditSession: AuditSession,
|
||||
private readonly agentName: LoggableAgentName,
|
||||
private readonly attemptNumber: number,
|
||||
) {}
|
||||
|
||||
// Serializes writes onto one chain so concurrent calls append in call order rather than racing
|
||||
// on the underlying audit session, and swallows failures so a broken audit write never surfaces
|
||||
// as the agent's own error: recording an error must not itself risk failing the run.
|
||||
private enqueue(operation: () => Promise<void>): Promise<void> {
|
||||
this.queue = this.queue.then(operation, operation).catch(() => undefined);
|
||||
return this.queue;
|
||||
}
|
||||
|
||||
async logLlmResponse(turn: number, content: string): Promise<void> {
|
||||
await this.auditSession.logEvent('llm_response', {
|
||||
turn,
|
||||
content,
|
||||
timestamp: formatTimestamp(),
|
||||
});
|
||||
logError(error: SafeErrorDetails, duration: number, turns: number): Promise<void> {
|
||||
return this.enqueue(() =>
|
||||
this.auditSession.logAgentError(this.agentName, error.code, error.category, this.attemptNumber, duration, turns),
|
||||
);
|
||||
}
|
||||
|
||||
async logToolStart(toolName: string, parameters: unknown): Promise<void> {
|
||||
await this.auditSession.logEvent('tool_start', {
|
||||
toolName,
|
||||
parameters,
|
||||
timestamp: formatTimestamp(),
|
||||
});
|
||||
}
|
||||
|
||||
async logToolEnd(result: unknown): Promise<void> {
|
||||
await this.auditSession.logEvent('tool_end', {
|
||||
result,
|
||||
timestamp: formatTimestamp(),
|
||||
});
|
||||
}
|
||||
|
||||
async logError(error: Error, duration: number, turns: number): Promise<void> {
|
||||
await this.auditSession.logEvent('error', {
|
||||
message: error.message,
|
||||
errorType: error.constructor.name,
|
||||
stack: error.stack,
|
||||
duration,
|
||||
turns,
|
||||
timestamp: formatTimestamp(),
|
||||
});
|
||||
}
|
||||
|
||||
async logNote(category: string, message: string): Promise<void> {
|
||||
await this.auditSession.logWorkflowNote(category, message);
|
||||
async flush(): Promise<void> {
|
||||
await this.queue;
|
||||
}
|
||||
}
|
||||
|
||||
/** Null Object implementation - all methods are safe no-ops */
|
||||
/** No-op sink for a run with no audit session or an agent name unsafe to log. */
|
||||
class NullAuditLogger implements AuditLogger {
|
||||
async logLlmResponse(_turn: number, _content: string): Promise<void> {}
|
||||
async logError(_error: SafeErrorDetails, _duration: number, _turns: number): Promise<void> {}
|
||||
|
||||
async logToolStart(_toolName: string, _parameters: unknown): Promise<void> {}
|
||||
|
||||
async logToolEnd(_result: unknown): Promise<void> {}
|
||||
|
||||
async logError(_error: Error, _duration: number, _turns: number): Promise<void> {}
|
||||
|
||||
async logNote(_category: string, _message: string): Promise<void> {}
|
||||
async flush(): Promise<void> {}
|
||||
}
|
||||
|
||||
// Returns no-op when auditSession is null
|
||||
export function createAuditLogger(auditSession: AuditSession | null): AuditLogger {
|
||||
if (auditSession) {
|
||||
return new RealAuditLogger(auditSession);
|
||||
/**
|
||||
* Build the error-audit sink for one agent attempt.
|
||||
*
|
||||
* Falls back to the null sink whenever real logging can't be done safely: no
|
||||
* audit session for this run, no agent name, or a name that isn't in the closed
|
||||
* loggable set (`isLoggableAgentName`). An unrecognized name is never written
|
||||
* to the durable audit trail, even as a bare string.
|
||||
*/
|
||||
export function createAuditLogger(
|
||||
auditSession: AuditSession | null,
|
||||
agentName: string | null,
|
||||
attemptNumber: number,
|
||||
): AuditLogger {
|
||||
if (auditSession !== null && agentName !== null && isLoggableAgentName(agentName)) {
|
||||
return new RealAuditLogger(auditSession, agentName, attemptNumber);
|
||||
}
|
||||
|
||||
return new NullAuditLogger();
|
||||
}
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
import { classifyProviderFailure } from '../services/error-handling.js';
|
||||
import type { ProviderFailure } from '../types/errors.js';
|
||||
import { type ModelSelection, resolveModelSelection } from './models.js';
|
||||
|
||||
/** Intended cost/capability role for a model call. All roles use the run's one selected model. */
|
||||
export type ModelRole = 'small' | 'medium' | 'large';
|
||||
|
||||
/** Credential-preserving model selection and provider-failure classification boundary. */
|
||||
export interface ModelHost {
|
||||
resolve(role: ModelRole): Promise<ModelSelection>;
|
||||
classify(error: unknown, contextWindow?: number): ProviderFailure;
|
||||
}
|
||||
|
||||
export type ModelSelectionResolver = () => Promise<ModelSelection>;
|
||||
|
||||
class ShannonModelHost implements ModelHost {
|
||||
private selection: Promise<ModelSelection> | undefined;
|
||||
|
||||
constructor(private readonly resolver: ModelSelectionResolver) {}
|
||||
|
||||
// Cache only a selection that resolves. The catch clears the slot on rejection so a later
|
||||
// attempt (a retried activity) can resolve again instead of replaying the first failure forever.
|
||||
// The identity guard leaves a newer in-flight selection in place if one already replaced this one.
|
||||
resolve(_role: ModelRole): Promise<ModelSelection> {
|
||||
if (this.selection) return this.selection;
|
||||
|
||||
const selection = Promise.resolve()
|
||||
.then(() => this.resolver())
|
||||
.catch((error: unknown) => {
|
||||
if (this.selection === selection) this.selection = undefined;
|
||||
throw error;
|
||||
});
|
||||
this.selection = selection;
|
||||
return this.selection;
|
||||
}
|
||||
|
||||
classify(error: unknown, contextWindow?: number): ProviderFailure {
|
||||
return classifyProviderFailure(error, contextWindow);
|
||||
}
|
||||
}
|
||||
|
||||
/** Create an isolated host, primarily for callers with an explicit lifecycle or focused verification. */
|
||||
export function createModelHost(resolver: ModelSelectionResolver = resolveModelSelection): ModelHost {
|
||||
return new ShannonModelHost(resolver);
|
||||
}
|
||||
|
||||
/** Process-local model host shared by production model callers. */
|
||||
export const modelHost: ModelHost = createModelHost();
|
||||
@@ -1,4 +1,4 @@
|
||||
// Copyright (C) 2025 Keygraph, Inc.
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
@@ -19,6 +19,13 @@
|
||||
*
|
||||
* Resolution returns a pi `Model` plus the `ModelRuntime` that owns its auth,
|
||||
* built over an in-memory credential store primed from the environment.
|
||||
*
|
||||
* The CLI cannot import this module (it ships as a separate bundle), so
|
||||
* `apps/cli/src/model-spec.ts` mirrors the parse rule and the provider/credential
|
||||
* tables by hand for its own `status` rendering and setup wizard. The two copies
|
||||
* have no shared compile-time link: a provider added or renamed on one side and
|
||||
* not the other does not fail to build, it just makes the CLI's guidance or
|
||||
* guard rails disagree with what the worker actually accepts at runtime.
|
||||
*/
|
||||
|
||||
import { existsSync } from 'node:fs';
|
||||
@@ -30,6 +37,11 @@ import { getAgentDir, ModelRuntime } from '@earendil-works/pi-coding-agent';
|
||||
* Providers Shannon curates with their own credential variables, config sections,
|
||||
* and setup flows. Each is a pi-ai provider id; any other pi provider is still
|
||||
* reachable through the generic credential path below.
|
||||
*
|
||||
* Kept identical to the CLI's own copy of this list (`apps/cli/src/model-spec.ts`),
|
||||
* which the CLI uses to decide whether "only one provider is configured" and to
|
||||
* gate its "Other provider" setup option. A curated provider missing from one
|
||||
* copy is silently treated as generic on that side.
|
||||
*/
|
||||
export const CURATED_PROVIDERS = ['anthropic', 'openai', 'xai', 'amazon-bedrock'] as const;
|
||||
|
||||
@@ -47,6 +59,11 @@ export const GENERIC_API_KEY_ENV = 'SHANNON_AI_API_KEY';
|
||||
* does not invent credential names — these are the variables each provider's own
|
||||
* tooling uses. Bedrock pairs its bearer token with AWS_REGION, which is provider
|
||||
* config rather than a credential.
|
||||
*
|
||||
* Mirrored by the CLI's own table of the same name, used there to decide which
|
||||
* env vars to forward into the worker container. A variable added here without
|
||||
* its CLI counterpart never reaches the container: the worker looks for a
|
||||
* credential the CLI never forwarded, and preflight reports it as absent.
|
||||
*/
|
||||
export const PROVIDER_API_KEY_ENV: Readonly<Record<CuratedProviderId, readonly string[]>> = {
|
||||
anthropic: ['ANTHROPIC_API_KEY', 'CLAUDE_CODE_OAUTH_TOKEN'],
|
||||
@@ -232,10 +249,11 @@ export async function createModelRuntime(providerId: string, apiKey: string | un
|
||||
}
|
||||
|
||||
export interface ModelSelection {
|
||||
model: Model<Api>;
|
||||
modelRuntime: ModelRuntime;
|
||||
modelId: string;
|
||||
providerId: string;
|
||||
readonly model: Model<Api>;
|
||||
readonly modelRuntime: ModelRuntime;
|
||||
readonly modelId: string;
|
||||
readonly providerId: string;
|
||||
readonly credentialSource: 'api-key' | 'pi-auth' | 'ambient';
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -324,6 +342,7 @@ export async function resolveModelSelection(): Promise<ModelSelection> {
|
||||
const credentials = resolveProviderCredentials(providerId);
|
||||
const format = resolveGatewayFormat(providerId, credentials.baseUrl);
|
||||
|
||||
const mountedPiAuth = piAuthPresent();
|
||||
const modelRuntime = await createModelRuntime(providerId, credentials.apiKey);
|
||||
|
||||
const model = resolveModel(modelRuntime, providerId, modelId, credentials.baseUrl, format);
|
||||
@@ -333,10 +352,18 @@ export async function resolveModelSelection(): Promise<ModelSelection> {
|
||||
);
|
||||
}
|
||||
|
||||
let credentialSource: ModelSelection['credentialSource'] = 'ambient';
|
||||
if (mountedPiAuth) {
|
||||
credentialSource = 'pi-auth';
|
||||
} else if (credentials.apiKey) {
|
||||
credentialSource = 'api-key';
|
||||
}
|
||||
|
||||
return {
|
||||
model,
|
||||
modelRuntime,
|
||||
modelId,
|
||||
providerId,
|
||||
credentialSource,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
// Copyright (C) 2025 Keygraph, Inc.
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
@@ -14,6 +14,7 @@
|
||||
* a direct mapping.
|
||||
*/
|
||||
|
||||
import type { SafeErrorDetails } from '../audit/safe-fields.js';
|
||||
import { AGENTS } from '../session-manager.js';
|
||||
import { extractAgentType, formatDuration } from '../utils/formatting.js';
|
||||
import type { ExecutionContext } from './types.js';
|
||||
@@ -27,7 +28,10 @@ interface ToolCallInput {
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
/** Agent prefix used to attribute output when parallel agents interleave on one stream. */
|
||||
// Agent prefix used to attribute output when parallel agents interleave on one stream. Tries the
|
||||
// registered agent's exact display name first, then falls back to a keyword match against the raw
|
||||
// description, so a caller passing an ad hoc description string still gets a reasonable tag
|
||||
// instead of the generic one.
|
||||
export function getAgentPrefix(description: string): string {
|
||||
const agentPrefixes: Record<string, string> = {
|
||||
'injection-vuln': '[Injection]',
|
||||
@@ -68,7 +72,9 @@ function extractDomain(url: string): string {
|
||||
}
|
||||
}
|
||||
|
||||
/** Format a playwright-cli command (run via the bash tool) into a clean progress indicator. */
|
||||
// Browser automation goes through the bash tool as a playwright-cli invocation, not a dedicated
|
||||
// tool call, so there is no structured event to read the action from. This parses the command line
|
||||
// back into a friendly one-liner instead of showing the raw shell command.
|
||||
function formatBrowserAction(command: string): string | null {
|
||||
const match = command.match(/playwright-cli\s+(?:-s=\S+\s+)?(\S+)(?:\s+(.*))?/);
|
||||
if (!match) return null;
|
||||
@@ -139,7 +145,9 @@ function formatBrowserAction(command: string): string | null {
|
||||
}
|
||||
}
|
||||
|
||||
/** Summarize a todo_write update into a clean progress indicator. */
|
||||
// todo_write replaces the whole list on every call, so there is no single "changed item" to
|
||||
// report. Surface the most recently completed item if one exists, otherwise the item now in
|
||||
// progress; a list with neither (all pending, or empty) has nothing worth printing.
|
||||
function summarizeTodoUpdate(input: ToolCallInput | undefined): string | null {
|
||||
if (!input?.todos || !Array.isArray(input.todos)) {
|
||||
return null;
|
||||
@@ -159,6 +167,15 @@ function summarizeTodoUpdate(input: ToolCallInput | undefined): string | null {
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Classify a phase's console output style from its human-readable description.
|
||||
*
|
||||
* `isParallelExecution` marks the five concurrent vuln/exploit agents, whose output
|
||||
* interleaves on one stream and so needs a per-line agent tag; `useCleanOutput` marks
|
||||
* every phase that gets the friendly spinner-and-summary treatment instead of the
|
||||
* verbose turn-by-turn fallback. Matching is on substrings of `description`, the same
|
||||
* strings the activity layer passes as the human-facing phase label.
|
||||
*/
|
||||
export function detectExecutionContext(description: string): ExecutionContext {
|
||||
const isParallelExecution = description.includes('vuln agent') || description.includes('exploit agent');
|
||||
|
||||
@@ -236,36 +253,28 @@ export function formatToolCall(
|
||||
}
|
||||
|
||||
export function formatErrorOutput(
|
||||
error: Error & { code?: string; status?: number },
|
||||
error: SafeErrorDetails,
|
||||
context: ExecutionContext,
|
||||
description: string,
|
||||
duration: number,
|
||||
sourceDir: string,
|
||||
turns: number,
|
||||
isRetryable: boolean,
|
||||
): string[] {
|
||||
const lines: string[] = [];
|
||||
|
||||
if (context.isParallelExecution) {
|
||||
lines.push(`${getAgentPrefix(description)} Failed (${formatDuration(duration)})`);
|
||||
lines.push(`Agent failed (${formatDuration(duration)})`);
|
||||
} else if (context.useCleanOutput) {
|
||||
lines.push(`${context.agentType} failed (${formatDuration(duration)})`);
|
||||
} else {
|
||||
lines.push(` pi agent failed: ${description} (${formatDuration(duration)})`);
|
||||
lines.push(` Agent failed (${formatDuration(duration)})`);
|
||||
}
|
||||
|
||||
lines.push(` Error Type: ${error.constructor.name}`);
|
||||
lines.push(` Error Code: ${error.code}`);
|
||||
lines.push(` Category: ${error.category}`);
|
||||
lines.push(` Message: ${error.message}`);
|
||||
lines.push(` Agent: ${description}`);
|
||||
lines.push(` Working Directory: ${sourceDir}`);
|
||||
lines.push(` Turns: ${turns}`);
|
||||
lines.push(` Retryable: ${isRetryable ? 'Yes' : 'No'}`);
|
||||
|
||||
if (error.code) {
|
||||
lines.push(` Error Code: ${error.code}`);
|
||||
}
|
||||
if (error.status) {
|
||||
lines.push(` HTTP Status: ${error.status}`);
|
||||
}
|
||||
|
||||
return lines;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,606 @@
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
import type { AgentMessage } from '@earendil-works/pi-agent-core';
|
||||
import {
|
||||
type AgentSession,
|
||||
type AgentSessionEvent,
|
||||
createAgentSession,
|
||||
DefaultResourceLoader,
|
||||
defineTool,
|
||||
getAgentDir,
|
||||
SessionManager,
|
||||
SettingsManager,
|
||||
type ToolDefinition,
|
||||
} from '@earendil-works/pi-coding-agent';
|
||||
import type { TSchema } from 'typebox';
|
||||
import { Value } from 'typebox/value';
|
||||
import { captureToolInvocation, decideToolOutcome } from '../../audit/trace.js';
|
||||
import type { ProviderFailureCategory } from '../../types/errors.js';
|
||||
import { type ModelHost, modelHost } from '../model-host.js';
|
||||
import type { ModelSelection } from '../models.js';
|
||||
import type { CapellaAgentErrorName as SharedCapellaAgentErrorName } from '../sast/capella/error-contract.js';
|
||||
import { CAPELLA_REPOSITORY_TOOL_NAMES, isCapellaRepositoryTool } from '../sast/capella/tools/repository-tools.js';
|
||||
import type { CapellaUsage } from '../sast/types.js';
|
||||
import type {
|
||||
CapellaAgentExecutor,
|
||||
CapellaAgentRequest,
|
||||
CapellaAgentResponse,
|
||||
CapellaTool,
|
||||
} from './capella-agent-types.js';
|
||||
import { PI_RETRY_SETTINGS } from './retry-settings.js';
|
||||
|
||||
const MAX_ERROR_LENGTH = 2_000;
|
||||
const MAX_TOOLS_PER_SESSION = 32;
|
||||
const MAX_TURNS_PER_SESSION = 1_000;
|
||||
const MAX_TIMEOUT_MS = 24 * 60 * 60 * 1_000;
|
||||
|
||||
// The closed set of stage-specific tools a caller is allowed to hand in alongside the confined
|
||||
// repository tools. Anything not on this list, and not a repository tool, is rejected as unknown
|
||||
// by validateCallerTools below.
|
||||
const CAPELLA_COLLECTOR_TOOL_NAMES = new Set([
|
||||
'report_finding',
|
||||
'record_duplicates',
|
||||
'record_review_verdict',
|
||||
'record_viability',
|
||||
'record_static_confirmation',
|
||||
'record_calibration',
|
||||
]);
|
||||
|
||||
// A Capella stage reasons over a read-only, confined view of the repository; none of these may
|
||||
// ever be offered to it. `bash`/`shell`/`network`/`browser`/`web_search` would give it an escape
|
||||
// hatch out of the confined tool set entirely; `edit`/`write` would let a review agent change the
|
||||
// code it is meant to only analyze; `task` would let it spawn further sessions outside this
|
||||
// executor's bounded turn/timeout accounting; `glob`/`ls`/`todo`/`todo_write` duplicate tools the
|
||||
// stage already gets from the confined factory or has no use for.
|
||||
const FORBIDDEN_TOOL_NAMES = new Set([
|
||||
'bash',
|
||||
'browser',
|
||||
'edit',
|
||||
'glob',
|
||||
'ls',
|
||||
'network',
|
||||
'shell',
|
||||
'task',
|
||||
'todo',
|
||||
'todo_write',
|
||||
'web_search',
|
||||
'write',
|
||||
]);
|
||||
|
||||
export type CapellaAgentErrorName = SharedCapellaAgentErrorName;
|
||||
|
||||
export type CapellaAgentErrorCode =
|
||||
| 'DUPLICATE_RESULT'
|
||||
| 'INVALID_REQUEST'
|
||||
| 'INVALID_RESULT'
|
||||
| 'INVALID_TOOL_SET'
|
||||
| 'MISSING_RESULT'
|
||||
| 'PROVIDER_FAILURE'
|
||||
| 'SESSION_FAILURE'
|
||||
| 'TIMEOUT'
|
||||
| 'TURN_LIMIT'
|
||||
| 'USAGE_LEDGER_FAILURE';
|
||||
|
||||
/** Typed, bounded executor failure suitable for Temporal error-name mapping. */
|
||||
export class CapellaAgentError extends Error {
|
||||
constructor(
|
||||
override readonly name: CapellaAgentErrorName,
|
||||
readonly code: CapellaAgentErrorCode,
|
||||
message: string,
|
||||
readonly retryable: boolean,
|
||||
readonly usage?: CapellaUsage,
|
||||
readonly providerCategory?: ProviderFailureCategory,
|
||||
) {
|
||||
super(message.slice(0, MAX_ERROR_LENGTH));
|
||||
}
|
||||
}
|
||||
|
||||
type TerminationReason = 'cancellation' | 'timeout' | 'turn-limit';
|
||||
|
||||
interface CapturedSubmission {
|
||||
readonly tool: ToolDefinition;
|
||||
readonly getCount: () => number;
|
||||
readonly getInvalid: () => boolean;
|
||||
readonly getValue: () => unknown;
|
||||
}
|
||||
|
||||
interface SessionOutcome {
|
||||
readonly submissionCount: number;
|
||||
readonly submissionValue: unknown;
|
||||
readonly invalidSubmission: boolean;
|
||||
readonly pendingProviderError: unknown;
|
||||
readonly promptError: unknown;
|
||||
readonly usage: CapellaUsage;
|
||||
}
|
||||
|
||||
class CapellaCancellationError extends Error {
|
||||
override readonly name = 'AbortError';
|
||||
|
||||
constructor(
|
||||
readonly usage: CapellaUsage,
|
||||
cause: Error,
|
||||
) {
|
||||
super('Capella session cancelled.', { cause });
|
||||
}
|
||||
}
|
||||
|
||||
function agentError(
|
||||
name: CapellaAgentErrorName,
|
||||
code: CapellaAgentErrorCode,
|
||||
message: string,
|
||||
retryable: boolean,
|
||||
usage?: CapellaUsage,
|
||||
providerCategory?: ProviderFailureCategory,
|
||||
): CapellaAgentError {
|
||||
return new CapellaAgentError(name, code, message, retryable, usage, providerCategory);
|
||||
}
|
||||
|
||||
function assertRequest(request: CapellaAgentRequest<unknown>): void {
|
||||
if (!Number.isInteger(request.maxTurns) || request.maxTurns < 1 || request.maxTurns > MAX_TURNS_PER_SESSION) {
|
||||
throw agentError('InvalidInputError', 'INVALID_REQUEST', 'Capella maxTurns is outside its bounded range.', false);
|
||||
}
|
||||
if (!Number.isInteger(request.timeoutMs) || request.timeoutMs < 1 || request.timeoutMs > MAX_TIMEOUT_MS) {
|
||||
throw agentError('InvalidInputError', 'INVALID_REQUEST', 'Capella timeoutMs is outside its bounded range.', false);
|
||||
}
|
||||
if (!request.cwd || !request.systemPrompt || !request.userPrompt) {
|
||||
throw agentError('InvalidInputError', 'INVALID_REQUEST', 'Capella request is incomplete.', false);
|
||||
}
|
||||
if (request.tools.length > MAX_TOOLS_PER_SESSION) {
|
||||
throw agentError('InvalidInputError', 'INVALID_TOOL_SET', 'Capella tool count exceeds its bounded limit.', false);
|
||||
}
|
||||
}
|
||||
|
||||
// Gate the caller's tool set before a session starts. Repository tools must come from the confined
|
||||
// factory (never a caller-built look-alike), collectors must be known by name, and nothing outside
|
||||
// that closed set is allowed. `submit_result` is executor-owned, so a caller supplying one alongside
|
||||
// an output schema is rejected. Any violation fails the request as invalid input, not a model error.
|
||||
function validateCallerTools(tools: readonly CapellaTool[], hasOutputSchema: boolean): void {
|
||||
const names = new Set<string>();
|
||||
for (const tool of tools) {
|
||||
const name = tool.name;
|
||||
if (!name || names.has(name) || FORBIDDEN_TOOL_NAMES.has(name) || name === 'submit_result') {
|
||||
throw agentError('InvalidInputError', 'INVALID_TOOL_SET', 'Capella tool set contains a forbidden name.', false);
|
||||
}
|
||||
names.add(name);
|
||||
|
||||
if ((CAPELLA_REPOSITORY_TOOL_NAMES as readonly string[]).includes(name)) {
|
||||
if (!isCapellaRepositoryTool(tool)) {
|
||||
throw agentError(
|
||||
'InvalidInputError',
|
||||
'INVALID_TOOL_SET',
|
||||
'Capella repository tools must come from the confined tool factory.',
|
||||
false,
|
||||
);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (!CAPELLA_COLLECTOR_TOOL_NAMES.has(name)) {
|
||||
throw agentError(
|
||||
'InvalidInputError',
|
||||
'INVALID_TOOL_SET',
|
||||
'Capella tool set contains an unknown collector.',
|
||||
false,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (hasOutputSchema && names.has('submit_result')) {
|
||||
throw agentError('InvalidInputError', 'INVALID_TOOL_SET', 'Capella submit_result is executor-owned.', false);
|
||||
}
|
||||
}
|
||||
|
||||
function createCapturedSubmission(schema: TSchema): CapturedSubmission {
|
||||
let count = 0;
|
||||
let invalid = false;
|
||||
let value: unknown;
|
||||
return {
|
||||
tool: defineTool({
|
||||
name: 'submit_result',
|
||||
label: 'Submit result',
|
||||
description: 'Return the final structured result exactly once.',
|
||||
promptSnippet: 'submit_result: return the final structured result exactly once',
|
||||
promptGuidelines: ['Call submit_result exactly once as the final action. Do not print JSON as text.'],
|
||||
parameters: schema,
|
||||
async execute(_toolCallId, parameters) {
|
||||
if (!Value.Check(schema, parameters)) {
|
||||
invalid = true;
|
||||
throw agentError(
|
||||
'AgentExecutionError',
|
||||
'INVALID_RESULT',
|
||||
'Capella submit_result arguments failed schema validation.',
|
||||
true,
|
||||
);
|
||||
}
|
||||
count += 1;
|
||||
if (count === 1) value = parameters;
|
||||
return {
|
||||
content: [{ type: 'text' as const, text: 'Result submitted.' }],
|
||||
details: undefined,
|
||||
terminate: true,
|
||||
};
|
||||
},
|
||||
}),
|
||||
getCount: () => count,
|
||||
getInvalid: () => invalid,
|
||||
getValue: () => value,
|
||||
};
|
||||
}
|
||||
|
||||
function finiteNonNegative(value: number): number {
|
||||
return Number.isFinite(value) ? Math.max(0, value) : 0;
|
||||
}
|
||||
|
||||
function frozenUsage(session: AgentSession, turns: number): CapellaUsage {
|
||||
const stats = session.getSessionStats();
|
||||
return Object.freeze({
|
||||
inputTokens: finiteNonNegative(stats.tokens.input),
|
||||
outputTokens: finiteNonNegative(stats.tokens.output),
|
||||
cacheReadTokens: finiteNonNegative(stats.tokens.cacheRead),
|
||||
cacheWriteTokens: finiteNonNegative(stats.tokens.cacheWrite),
|
||||
costUsd: finiteNonNegative(stats.cost),
|
||||
turns: finiteNonNegative(turns),
|
||||
});
|
||||
}
|
||||
|
||||
function isAbortLike(error: unknown): boolean {
|
||||
return error instanceof Error && (error.name === 'AbortError' || error.name === 'TimeoutError');
|
||||
}
|
||||
|
||||
function isRetryableSetupIo(error: unknown): boolean {
|
||||
const code = (error as NodeJS.ErrnoException | undefined)?.code;
|
||||
return code === 'EAGAIN' || code === 'EBUSY' || code === 'EIO' || code === 'EMFILE' || code === 'ENFILE';
|
||||
}
|
||||
|
||||
function cancellationError(signal: AbortSignal): Error {
|
||||
if (signal.reason instanceof Error) return signal.reason;
|
||||
return new DOMException('Capella session cancelled.', 'AbortError');
|
||||
}
|
||||
|
||||
function raceWithAbort<T>(promise: Promise<T>, signal: AbortSignal): Promise<T> {
|
||||
if (signal.aborted) return Promise.reject(cancellationError(signal));
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
const onAbort = (): void => reject(cancellationError(signal));
|
||||
signal.addEventListener('abort', onAbort, { once: true });
|
||||
promise.then(
|
||||
(value) => {
|
||||
signal.removeEventListener('abort', onAbort);
|
||||
resolve(value);
|
||||
},
|
||||
(error: unknown) => {
|
||||
signal.removeEventListener('abort', onAbort);
|
||||
reject(error);
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
function classifiedModelFailure(
|
||||
host: ModelHost,
|
||||
error: unknown,
|
||||
code: 'PROVIDER_FAILURE' | 'SESSION_FAILURE',
|
||||
usage?: CapellaUsage,
|
||||
contextWindow?: number,
|
||||
): CapellaAgentError {
|
||||
const failure = host.classify(error, contextWindow);
|
||||
if (isAbortLike(error) || isRetryableSetupIo(error)) {
|
||||
return agentError(
|
||||
'AgentExecutionError',
|
||||
code,
|
||||
'The model session ended because of a retryable local or provider failure.',
|
||||
true,
|
||||
usage,
|
||||
);
|
||||
}
|
||||
return agentError(failure.type, code, failure.message, failure.retryable, usage, failure.category);
|
||||
}
|
||||
|
||||
// Map the raw failure to its true cause, with the termination reason taking priority. When the
|
||||
// session was cancelled or timed out, the caught error is typically the induced abort; surface the
|
||||
// cancellation or timeout identity instead of misreporting it as a provider or session failure.
|
||||
function normalizeRunFailure(
|
||||
error: unknown,
|
||||
termination: TerminationReason | undefined,
|
||||
signal: AbortSignal,
|
||||
host: ModelHost,
|
||||
): Error {
|
||||
if (termination === 'cancellation') {
|
||||
return error instanceof CapellaCancellationError ? error : cancellationError(signal);
|
||||
}
|
||||
if (error instanceof CapellaAgentError) return error;
|
||||
if (termination === 'timeout') {
|
||||
return agentError('AgentExecutionError', 'TIMEOUT', 'Capella session timed out.', true);
|
||||
}
|
||||
if (termination === 'turn-limit') {
|
||||
return agentError(
|
||||
'AgentExecutionError',
|
||||
'TURN_LIMIT',
|
||||
'An agentic SAST step ran out of turns before finishing.',
|
||||
true,
|
||||
);
|
||||
}
|
||||
return classifiedModelFailure(host, error, 'SESSION_FAILURE');
|
||||
}
|
||||
|
||||
class StandaloneCapellaAgentExecutor implements CapellaAgentExecutor {
|
||||
constructor(private readonly host: ModelHost) {}
|
||||
|
||||
async run<T>(request: CapellaAgentRequest<T>): Promise<CapellaAgentResponse<T>> {
|
||||
assertRequest(request as CapellaAgentRequest<unknown>);
|
||||
validateCallerTools(request.tools, request.outputSchema !== undefined);
|
||||
|
||||
const controller = new AbortController();
|
||||
let termination: TerminationReason | undefined;
|
||||
let session: AgentSession | undefined;
|
||||
let unsubscribe: (() => void) | undefined;
|
||||
let timeout: NodeJS.Timeout | undefined;
|
||||
let turnCount = 0;
|
||||
let operationCount = 0;
|
||||
const terminate = (reason: TerminationReason): void => {
|
||||
if (termination !== undefined) return;
|
||||
termination = reason;
|
||||
controller.abort(new DOMException(`Capella session ${reason}.`, 'AbortError'));
|
||||
void session?.abort().catch(() => undefined);
|
||||
};
|
||||
const onCancellation = (): void => terminate('cancellation');
|
||||
|
||||
if (request.signal.aborted) throw cancellationError(request.signal);
|
||||
request.signal.addEventListener('abort', onCancellation, { once: true });
|
||||
timeout = setTimeout(() => terminate('timeout'), request.timeoutMs);
|
||||
|
||||
try {
|
||||
let selection: ModelSelection;
|
||||
try {
|
||||
selection = await raceWithAbort(this.host.resolve(request.role), controller.signal);
|
||||
} catch (error) {
|
||||
if (termination === 'cancellation') throw cancellationError(request.signal);
|
||||
if (termination === 'timeout') {
|
||||
throw agentError('AgentExecutionError', 'TIMEOUT', 'Capella session timed out.', true);
|
||||
}
|
||||
throw classifiedModelFailure(this.host, error, 'PROVIDER_FAILURE');
|
||||
}
|
||||
|
||||
const submit = request.outputSchema ? createCapturedSubmission(request.outputSchema) : undefined;
|
||||
const customTools = [...request.tools, ...(submit ? [submit.tool] : [])];
|
||||
const toolNames = customTools.map((tool) => tool.name);
|
||||
const systemPrompt = submit
|
||||
? `${request.systemPrompt}\n\nYou MUST call submit_result exactly once as your final action. Do not output JSON as text.`
|
||||
: request.systemPrompt;
|
||||
const agentDir = getAgentDir();
|
||||
const settingsManager = SettingsManager.inMemory({
|
||||
retry: PI_RETRY_SETTINGS,
|
||||
compaction: { enabled: true },
|
||||
});
|
||||
|
||||
const resourceLoader = new DefaultResourceLoader({
|
||||
cwd: request.cwd,
|
||||
agentDir,
|
||||
settingsManager,
|
||||
systemPrompt,
|
||||
appendSystemPrompt: [],
|
||||
noExtensions: true,
|
||||
noSkills: true,
|
||||
noPromptTemplates: true,
|
||||
noThemes: true,
|
||||
noContextFiles: true,
|
||||
});
|
||||
await raceWithAbort(resourceLoader.reload(), controller.signal);
|
||||
|
||||
const sessionPromise = createAgentSession({
|
||||
cwd: request.cwd,
|
||||
agentDir,
|
||||
model: selection.model,
|
||||
modelRuntime: selection.modelRuntime,
|
||||
noTools: 'all',
|
||||
tools: toolNames,
|
||||
customTools,
|
||||
resourceLoader,
|
||||
sessionManager: SessionManager.inMemory(),
|
||||
settingsManager,
|
||||
});
|
||||
try {
|
||||
({ session } = await raceWithAbort(sessionPromise, controller.signal));
|
||||
} catch (error) {
|
||||
void sessionPromise.then(
|
||||
async ({ session: lateSession }) => {
|
||||
await lateSession.abort().catch(() => undefined);
|
||||
try {
|
||||
lateSession.dispose();
|
||||
} catch {
|
||||
// The late session is already aborted; cleanup remains best effort.
|
||||
}
|
||||
},
|
||||
() => undefined,
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
|
||||
if (controller.signal.aborted) {
|
||||
await session.abort().catch(() => undefined);
|
||||
} else {
|
||||
controller.signal.addEventListener('abort', () => void session?.abort().catch(() => undefined), {
|
||||
once: true,
|
||||
});
|
||||
}
|
||||
|
||||
// Re-check the live session's tools against the intended set. If pi registered anything extra
|
||||
// or dropped one, tool isolation broke, so fail closed before the model runs.
|
||||
const configuredToolNames = session
|
||||
.getAllTools()
|
||||
.map((tool) => tool.name)
|
||||
.sort();
|
||||
if (configuredToolNames.join('\0') !== [...toolNames].sort().join('\0')) {
|
||||
throw agentError(
|
||||
'ConfigurationError',
|
||||
'INVALID_TOOL_SET',
|
||||
'An agentic SAST step could not start with the tools it needs.',
|
||||
false,
|
||||
);
|
||||
}
|
||||
|
||||
let invalidSubmission = false;
|
||||
let pendingProviderError: unknown;
|
||||
// Per-session trace correlation lives here in the executor; the injected sink is a
|
||||
// stateless emitter, safe to share across the stage's sessions.
|
||||
const traceLog = request.log;
|
||||
const pendingTrace = new Map<string, { readonly tool: string; readonly startedAt: number }>();
|
||||
unsubscribe = session.subscribe((event: AgentSessionEvent) => {
|
||||
if (event.type === 'tool_execution_start') {
|
||||
operationCount += 1;
|
||||
if (traceLog !== undefined) {
|
||||
const invocation = captureToolInvocation(event.toolName, event.args);
|
||||
pendingTrace.set(event.toolCallId, { tool: event.toolName, startedAt: Date.now() });
|
||||
if (invocation !== undefined) traceLog.toolCall(invocation);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (event.type === 'tool_execution_end') {
|
||||
if (event.toolName === 'submit_result' && event.isError) invalidSubmission = true;
|
||||
if (traceLog !== undefined) {
|
||||
const pending = pendingTrace.get(event.toolCallId);
|
||||
if (pending !== undefined) {
|
||||
pendingTrace.delete(event.toolCallId);
|
||||
const outcome = decideToolOutcome(pending.tool, event.isError, Date.now() - pending.startedAt, undefined);
|
||||
if (outcome !== undefined) traceLog.toolOutcome(outcome);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (event.type !== 'turn_end') return;
|
||||
|
||||
turnCount += 1;
|
||||
const message: AgentMessage = event.message;
|
||||
if (message.role === 'assistant' && message.stopReason === 'error') {
|
||||
pendingProviderError ??= message;
|
||||
}
|
||||
const needsAnotherTurn = message.role === 'assistant' && message.stopReason === 'toolUse';
|
||||
if (turnCount >= request.maxTurns && needsAnotherTurn && (submit?.getCount() ?? 0) === 0) {
|
||||
terminate('turn-limit');
|
||||
}
|
||||
});
|
||||
|
||||
const runStartedAt = Date.now();
|
||||
let promptError: unknown;
|
||||
try {
|
||||
await raceWithAbort(session.prompt(request.userPrompt, { expandPromptTemplates: false }), controller.signal);
|
||||
} catch (error) {
|
||||
promptError = error;
|
||||
}
|
||||
|
||||
const outcome: SessionOutcome = {
|
||||
submissionCount: submit?.getCount() ?? 0,
|
||||
submissionValue: submit?.getValue(),
|
||||
invalidSubmission: invalidSubmission || (submit?.getInvalid() ?? false),
|
||||
pendingProviderError,
|
||||
promptError,
|
||||
usage: frozenUsage(session, turnCount),
|
||||
};
|
||||
const output = this.resolveOutcome<T>(request, outcome, termination, selection.model.contextWindow);
|
||||
|
||||
// Emitted only past resolveOutcome so a failed, cancelled, timed-out, or turn-capped
|
||||
// session (all of which throw above) never reports a truthful-looking completion.
|
||||
if (traceLog !== undefined) {
|
||||
traceLog.sessionComplete(Date.now() - runStartedAt, turnCount, operationCount);
|
||||
}
|
||||
|
||||
return { output, usage: outcome.usage };
|
||||
} catch (error) {
|
||||
const surfacedError = normalizeRunFailure(error, termination, request.signal, this.host);
|
||||
throw surfacedError;
|
||||
} finally {
|
||||
if (timeout) clearTimeout(timeout);
|
||||
request.signal.removeEventListener('abort', onCancellation);
|
||||
try {
|
||||
unsubscribe?.();
|
||||
} catch {
|
||||
// Subscription cleanup is best effort after the session has ended.
|
||||
}
|
||||
try {
|
||||
session?.dispose();
|
||||
} catch {
|
||||
// Session cleanup is best effort after abort or completion.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private resolveOutcome<T>(
|
||||
request: CapellaAgentRequest<T>,
|
||||
outcome: SessionOutcome,
|
||||
termination: TerminationReason | undefined,
|
||||
contextWindow?: number,
|
||||
): T {
|
||||
if (termination === 'cancellation') {
|
||||
throw new CapellaCancellationError(outcome.usage, cancellationError(request.signal));
|
||||
}
|
||||
if (termination === 'timeout') {
|
||||
throw agentError('AgentExecutionError', 'TIMEOUT', 'Capella session timed out.', true, outcome.usage);
|
||||
}
|
||||
if (termination === 'turn-limit') {
|
||||
throw agentError(
|
||||
'AgentExecutionError',
|
||||
'TURN_LIMIT',
|
||||
'An agentic SAST step ran out of turns before finishing.',
|
||||
true,
|
||||
outcome.usage,
|
||||
);
|
||||
}
|
||||
if (outcome.invalidSubmission && outcome.submissionCount === 0) {
|
||||
throw agentError(
|
||||
'AgentExecutionError',
|
||||
'INVALID_RESULT',
|
||||
'Capella submit_result arguments failed schema validation.',
|
||||
true,
|
||||
outcome.usage,
|
||||
);
|
||||
}
|
||||
if (outcome.submissionCount > 1) {
|
||||
throw agentError(
|
||||
'AgentExecutionError',
|
||||
'DUPLICATE_RESULT',
|
||||
'An agentic SAST step returned its result twice.',
|
||||
true,
|
||||
outcome.usage,
|
||||
);
|
||||
}
|
||||
if (outcome.pendingProviderError !== undefined) {
|
||||
const failure = this.host.classify(outcome.pendingProviderError, contextWindow);
|
||||
throw agentError(
|
||||
failure.type,
|
||||
'PROVIDER_FAILURE',
|
||||
failure.message,
|
||||
failure.retryable,
|
||||
outcome.usage,
|
||||
failure.category,
|
||||
);
|
||||
}
|
||||
// An abort after exactly one accepted submission is the normal end of a good run: the submit tool
|
||||
// terminates the session. Treat it as success; any other prompt error is a real session failure.
|
||||
if (outcome.promptError !== undefined && !(outcome.submissionCount === 1 && isAbortLike(outcome.promptError))) {
|
||||
throw classifiedModelFailure(this.host, outcome.promptError, 'SESSION_FAILURE', outcome.usage, contextWindow);
|
||||
}
|
||||
if (request.outputSchema !== undefined) {
|
||||
if (outcome.submissionCount !== 1 || outcome.submissionValue === undefined) {
|
||||
throw agentError(
|
||||
'AgentExecutionError',
|
||||
'MISSING_RESULT',
|
||||
'Capella session ended without one structured result.',
|
||||
true,
|
||||
outcome.usage,
|
||||
);
|
||||
}
|
||||
return outcome.submissionValue as T;
|
||||
}
|
||||
return undefined as T;
|
||||
}
|
||||
}
|
||||
|
||||
/** Create a Capella executor over the process-local credential-preserving model host. */
|
||||
export function createCapellaAgentExecutor(host: ModelHost = modelHost): CapellaAgentExecutor {
|
||||
return new StandaloneCapellaAgentExecutor(host);
|
||||
}
|
||||
|
||||
/** Process-local standalone Capella executor. */
|
||||
export const capellaAgentExecutor: CapellaAgentExecutor = createCapellaAgentExecutor();
|
||||
@@ -0,0 +1,66 @@
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
import type { ToolDefinition } from '@earendil-works/pi-coding-agent';
|
||||
import type { TSchema } from 'typebox';
|
||||
import type { ToolInvocation, ToolOutcome } from '../../audit/trace.js';
|
||||
import type { ModelRole } from '../model-host.js';
|
||||
import type { CapellaStage, CapellaUsage } from '../sast/types.js';
|
||||
|
||||
/** A Capella-owned collector or repository tool installed in one confined session. */
|
||||
export type CapellaTool = ToolDefinition;
|
||||
|
||||
/**
|
||||
* A sink for one Capella session's technical trace. The executor owns `toolCallId`
|
||||
* correlation and synchronously snapshots complete tool arguments before handing the
|
||||
* immutable invocation to the sink.
|
||||
*/
|
||||
export interface CapellaTraceLog {
|
||||
toolCall(invocation: ToolInvocation): void;
|
||||
toolOutcome(outcome: ToolOutcome): void;
|
||||
sessionComplete(durationMs: number, turns: number, operations: number): void;
|
||||
}
|
||||
|
||||
/**
|
||||
* One stage's trace surface. `forSession` binds a per-session view (its label becomes the trace
|
||||
* prefix's session component); all views share one serialized queue that `drain` awaits, so no
|
||||
* session's lines can still be buffered when its activity returns.
|
||||
*/
|
||||
export interface CapellaStageTrace {
|
||||
forSession(sessionLabel: string | undefined): CapellaTraceLog;
|
||||
drain(): Promise<void>;
|
||||
}
|
||||
|
||||
/** One bounded multi-turn Capella model session. */
|
||||
export interface CapellaAgentRequest<_T> {
|
||||
readonly stage: CapellaStage;
|
||||
readonly role: ModelRole;
|
||||
readonly cwd: string;
|
||||
readonly systemPrompt: string;
|
||||
readonly userPrompt: string;
|
||||
readonly maxTurns: number;
|
||||
readonly timeoutMs: number;
|
||||
readonly tools: readonly CapellaTool[];
|
||||
readonly outputSchema?: TSchema;
|
||||
readonly signal: AbortSignal;
|
||||
readonly log?: CapellaTraceLog;
|
||||
/**
|
||||
* Display-only session name for the trace prefix. Never hashed into `workloadId`, a checkpoint
|
||||
* key, a usage record, or a prompt; a stage may repeat or omit it without changing execution.
|
||||
*/
|
||||
readonly sessionLabel?: string;
|
||||
}
|
||||
|
||||
/** Schema-valid output and measured usage from one completed Capella session. */
|
||||
export interface CapellaAgentResponse<T> {
|
||||
readonly output: T;
|
||||
readonly usage: CapellaUsage;
|
||||
}
|
||||
|
||||
/** Standalone executor boundary consumed by the Capella stage implementation. */
|
||||
export interface CapellaAgentExecutor {
|
||||
run<T>(request: CapellaAgentRequest<T>): Promise<CapellaAgentResponse<T>>;
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
// Copyright (C) 2025 Keygraph, Inc.
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
// Copyright (C) 2025 Keygraph, Inc.
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
// Production agent execution on the pi harness, with git checkpoints and audit logging.
|
||||
// The checkpoint itself is created by the caller (AgentExecutionService) before and after
|
||||
// runPiPrompt runs; this module owns the session, its audit/error logging, and the trace it
|
||||
// produces, not the git commit around it.
|
||||
|
||||
import os from 'node:os';
|
||||
import type { AgentMessage } from '@earendil-works/pi-agent-core';
|
||||
@@ -22,6 +25,7 @@ import {
|
||||
} from '@earendil-works/pi-coding-agent';
|
||||
import { fs, path } from 'zx';
|
||||
import type { AuditSession } from '../../audit/index.js';
|
||||
import { isLoggableAgentName, type SafeErrorDetails, safeErrorFromUnknown } from '../../audit/safe-fields.js';
|
||||
import { BASH_TIMEOUT_EXTENSION_DIR, deliverablesDir } from '../../paths.js';
|
||||
import { isRetryableFailure, PentestError } from '../../services/error-handling.js';
|
||||
import { AGENT_VALIDATORS } from '../../session-manager.js';
|
||||
@@ -44,7 +48,8 @@ import { permissionSystemConfigExists, permissionSystemPackageDir } from './perm
|
||||
import { PI_RETRY_SETTINGS } from './retry-settings.js';
|
||||
import { createGlobTool, createTodoWriteTool } from './session-tools.js';
|
||||
import { createTaskTool } from './task-tool.js';
|
||||
import { providerTurnError } from './turn-error.js';
|
||||
import { TraceEmitter } from './trace-emitter.js';
|
||||
import { providerTurnError, type SafeProviderTurnDetails, safeProviderTurnDetails } from './turn-error.js';
|
||||
|
||||
declare global {
|
||||
var SHANNON_DISABLE_LOADER: boolean | undefined;
|
||||
@@ -142,7 +147,6 @@ export interface PiPromptResult {
|
||||
model?: string | undefined;
|
||||
error?: string | undefined;
|
||||
errorType?: string | undefined;
|
||||
prompt?: string | undefined;
|
||||
retryable?: boolean | undefined;
|
||||
structuredOutput?: unknown;
|
||||
}
|
||||
@@ -154,18 +158,22 @@ function outputLines(lines: string[]): void {
|
||||
}
|
||||
|
||||
async function writeErrorLog(
|
||||
err: Error & { code?: string; status?: number },
|
||||
sourceDir: string,
|
||||
fullPrompt: string,
|
||||
error: SafeErrorDetails,
|
||||
duration: number,
|
||||
turns: number,
|
||||
retryable: boolean,
|
||||
providerDetails?: SafeProviderTurnDetails,
|
||||
): Promise<void> {
|
||||
try {
|
||||
const errorLog = {
|
||||
timestamp: formatTimestamp(),
|
||||
agent: 'pi-executor',
|
||||
error: { name: err.constructor.name, message: err.message, code: err.code, status: err.status, stack: err.stack },
|
||||
context: { sourceDir, prompt: `${fullPrompt.slice(0, 200)}...`, retryable: isRetryableFailure(err) },
|
||||
error: { code: error.code, category: error.category, message: error.message },
|
||||
duration,
|
||||
turns,
|
||||
retryable,
|
||||
...(providerDetails !== undefined && { provider: providerDetails }),
|
||||
};
|
||||
const logPath = path.join(deliverablesDir(sourceDir), 'error.log');
|
||||
await fs.appendFile(logPath, `${JSON.stringify(errorLog)}\n`);
|
||||
@@ -186,6 +194,9 @@ export async function validateAgentOutput(
|
||||
logger.error('Validation failed: Agent execution was unsuccessful');
|
||||
return false;
|
||||
}
|
||||
// Not every agent has a deliverable-structure validator registered. Absence is not treated as
|
||||
// a failure: the agent already reported success above, so an agent with no validator passes on
|
||||
// that alone rather than being held to a check that was never defined for it.
|
||||
const validator = agentName ? AGENT_VALIDATORS[agentName as keyof typeof AGENT_VALIDATORS] : undefined;
|
||||
if (!validator) {
|
||||
logger.warn(`No validator found for agent "${agentName}" - assuming success`);
|
||||
@@ -230,6 +241,7 @@ export async function runPiPrompt(
|
||||
deliverablesSubdir?: string,
|
||||
cancellationSignal?: AbortSignal,
|
||||
submitTool?: CapturedSubmitTool,
|
||||
attemptNumber: number = 1,
|
||||
): Promise<PiPromptResult> {
|
||||
// 1. Initialize timing and prompt. A submit tool appends its directive so the
|
||||
// instruction to call it lives with the tool, not in every prompt file.
|
||||
@@ -243,7 +255,7 @@ export async function runPiPrompt(
|
||||
{ description, useCleanOutput: execContext.useCleanOutput },
|
||||
global.SHANNON_DISABLE_LOADER ?? false,
|
||||
);
|
||||
const auditLogger = createAuditLogger(auditSession);
|
||||
const auditLogger = createAuditLogger(auditSession, agentName, attemptNumber);
|
||||
|
||||
logger.info(`Running pi agent: ${description}...`);
|
||||
|
||||
@@ -259,6 +271,14 @@ export async function runPiPrompt(
|
||||
// plus any caller-supplied collector/submit tools).
|
||||
const selection = await resolveModelSelection();
|
||||
const resourceLoader = await buildResourceLoader(sourceDir, logger, agentName);
|
||||
const agentNameCandidate = agentName ?? '';
|
||||
const parentAgentName = isLoggableAgentName(agentNameCandidate) ? agentNameCandidate : 'pre-recon';
|
||||
// The durable trace log is path-addressed, so parent, child, and Capella writers all
|
||||
// reach the same file without sharing a stream handle.
|
||||
const workflowLogPath = auditSession?.workflowLogPath;
|
||||
const traceEmitter = workflowLogPath
|
||||
? new TraceEmitter(workflowLogPath, { kind: 'agent', agent: parentAgentName })
|
||||
: undefined;
|
||||
// Accumulates usage from in-process `task` child sessions so the parent's reported
|
||||
// cost includes sub-agent spend (their getSessionStats is separate from ours).
|
||||
const childUsage: ChildUsage = { cost: 0, inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0 };
|
||||
@@ -267,6 +287,11 @@ export async function runPiPrompt(
|
||||
model: selection.model,
|
||||
modelRuntime: selection.modelRuntime,
|
||||
cwd: sourceDir,
|
||||
parentAgentName,
|
||||
...(workflowLogPath !== undefined && { workflowLogPath }),
|
||||
...(traceEmitter !== undefined && {
|
||||
onDelegationStart: (child: string) => traceEmitter.delegationStart(child),
|
||||
}),
|
||||
onUsage: (usage) => {
|
||||
childUsage.cost += usage.cost;
|
||||
childUsage.inputTokens += usage.inputTokens;
|
||||
@@ -274,10 +299,10 @@ export async function runPiPrompt(
|
||||
childUsage.cacheReadTokens += usage.cacheReadTokens;
|
||||
childUsage.cacheWriteTokens += usage.cacheWriteTokens;
|
||||
},
|
||||
resourceLoader,
|
||||
createResourceLoader: () => buildResourceLoader(sourceDir, logger, agentName),
|
||||
...(cancellationSignal && { cancellationSignal }),
|
||||
}),
|
||||
createTodoWriteTool(auditLogger),
|
||||
createTodoWriteTool(),
|
||||
createGlobTool(sourceDir),
|
||||
...(callerTools ?? []),
|
||||
...(submitTool ? [submitTool.tool] : []),
|
||||
@@ -287,6 +312,9 @@ export async function runPiPrompt(
|
||||
|
||||
let turnCount = 0;
|
||||
let pendingError: PentestError | null = null;
|
||||
// Bounded, non-sensitive facts about the failed turn, captured alongside pendingError so the
|
||||
// error log can distinguish a safeguard/refusal from a transport or tool-call lifecycle fault.
|
||||
let pendingProviderDetails: SafeProviderTurnDetails | null = null;
|
||||
// Declared out here so the catch can bill spend accrued before a failure.
|
||||
let session: AgentSession | undefined;
|
||||
|
||||
@@ -330,18 +358,20 @@ export async function runPiPrompt(
|
||||
const msg = event.message;
|
||||
const text = extractAssistantText(msg);
|
||||
if (text.trim()) {
|
||||
void auditLogger.logLlmResponse(turnCount, text);
|
||||
progress.stop();
|
||||
outputLines(formatAssistantOutput(text, execContext, turnCount, description));
|
||||
progress.start();
|
||||
}
|
||||
if (msg.role === 'assistant' && msg.stopReason === 'error') {
|
||||
pendingError = pendingError ?? providerTurnError(msg, 'Agent error', selection.model.contextWindow);
|
||||
pendingProviderDetails =
|
||||
pendingProviderDetails ?? safeProviderTurnDetails(msg, selection.model.contextWindow);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'tool_execution_start': {
|
||||
void auditLogger.logToolStart(event.toolName, event.args);
|
||||
const count = submitTool?.tool.name === event.toolName ? submitTool.safeCount : undefined;
|
||||
traceEmitter?.toolStart(event.toolCallId, event.toolName, event.args, count);
|
||||
const toolLines = formatToolCall(
|
||||
event.toolName,
|
||||
event.args as Record<string, unknown>,
|
||||
@@ -355,9 +385,10 @@ export async function runPiPrompt(
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'tool_execution_end':
|
||||
void auditLogger.logToolEnd(event.result);
|
||||
case 'tool_execution_end': {
|
||||
traceEmitter?.toolEnd(event.toolCallId, event.isError);
|
||||
break;
|
||||
}
|
||||
case 'compaction_end':
|
||||
if (!event.aborted && !event.willRetry && event.errorMessage) {
|
||||
pendingError =
|
||||
@@ -372,7 +403,6 @@ export async function runPiPrompt(
|
||||
|
||||
// 6. Run the agent to completion (resolves at agent_end).
|
||||
await session.prompt(fullPrompt);
|
||||
session.dispose();
|
||||
|
||||
// 7. Surface any error captured during the run.
|
||||
if (pendingError) throw pendingError;
|
||||
@@ -387,6 +417,8 @@ export async function runPiPrompt(
|
||||
// Capture the submit tool's structured payload so callers read it off the
|
||||
// result instead of holding a reference to the tool.
|
||||
const structuredOutput = submitTool?.getCaptured();
|
||||
await auditLogger.flush();
|
||||
await traceEmitter?.flush();
|
||||
|
||||
return {
|
||||
result,
|
||||
@@ -402,13 +434,20 @@ export async function runPiPrompt(
|
||||
...(structuredOutput !== undefined && { structuredOutput }),
|
||||
};
|
||||
} catch (error) {
|
||||
// 10. Handle errors — log, write error file, return failure
|
||||
// 9. Handle errors: log, write error file, return failure
|
||||
const duration = timer.stop();
|
||||
const err = error as Error & { code?: string; status?: number };
|
||||
await auditLogger.logError(err, duration, turnCount);
|
||||
const safeError = safeErrorFromUnknown(err);
|
||||
const retryable = isRetryableFailure(err);
|
||||
await auditLogger.logError(safeError, duration, turnCount);
|
||||
await auditLogger.flush();
|
||||
await traceEmitter?.flush();
|
||||
progress.stop();
|
||||
outputLines(formatErrorOutput(err, execContext, description, duration, sourceDir, isRetryableFailure(err)));
|
||||
await writeErrorLog(err, sourceDir, fullPrompt, duration);
|
||||
outputLines(formatErrorOutput(safeError, execContext, duration, turnCount, retryable));
|
||||
if (pendingProviderDetails) {
|
||||
console.log(` provider-turn: ${JSON.stringify(pendingProviderDetails)}`);
|
||||
}
|
||||
await writeErrorLog(sourceDir, safeError, duration, turnCount, retryable, pendingProviderDetails ?? undefined);
|
||||
|
||||
// A failed agent still spent money — on its own turns and, since Shannon's
|
||||
// prompts delegate the heavy work, mostly on `task` sub-agents. Both count
|
||||
@@ -416,9 +455,8 @@ export async function runPiPrompt(
|
||||
const usage = totalUsage(session, childUsage);
|
||||
|
||||
return {
|
||||
error: err.message,
|
||||
errorType: err instanceof PentestError && err.code ? err.code : err.constructor.name,
|
||||
prompt: `${fullPrompt.slice(0, 100)}...`,
|
||||
error: safeError.message,
|
||||
errorType: safeError.code,
|
||||
success: false,
|
||||
duration,
|
||||
turns: turnCount,
|
||||
@@ -427,9 +465,10 @@ export async function runPiPrompt(
|
||||
outputTokens: usage.outputTokens,
|
||||
cacheReadTokens: usage.cacheReadTokens,
|
||||
cacheWriteTokens: usage.cacheWriteTokens,
|
||||
retryable: isRetryableFailure(err),
|
||||
retryable,
|
||||
};
|
||||
} finally {
|
||||
session?.dispose();
|
||||
cancellationSignal?.removeEventListener('abort', onCancellation);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
// Copyright (C) 2025 Keygraph, Inc.
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
@@ -18,11 +18,11 @@
|
||||
*
|
||||
* NOTE: pi recommends keeping this at 0, since SDK-level retries consume
|
||||
* out-of-usage-limit responses before pi's classifier can mark them terminal.
|
||||
* Shannon accepts that trade for the transport-fault coverage. `maxRetryDelayMs`
|
||||
* is left at pi's 60s default so a server asking for a longer wait fails fast
|
||||
* instead of parking the activity.
|
||||
* Shannon accepts that trade for the transport-fault coverage. A two-minute
|
||||
* delay cap lets short server-directed recovery remain in the current session;
|
||||
* longer delays return to Temporal's bounded activity retry policy.
|
||||
*/
|
||||
export const PI_RETRY_SETTINGS = {
|
||||
enabled: false,
|
||||
provider: { maxRetries: 8 },
|
||||
provider: { maxRetries: 8, maxRetryDelayMs: 120_000 },
|
||||
} as const;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
// Copyright (C) 2025 Keygraph, Inc.
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
@@ -8,32 +8,21 @@
|
||||
* Per-session custom tools registered for every agent: `todo_write` and `glob`.
|
||||
*
|
||||
* These replace harness built-ins that pi does not ship. `todo_write` is a
|
||||
* full-state-replace planning scratchpad mirrored to the workflow log; `glob` is
|
||||
* fast-glob file matching (pi has no `Glob` built-in).
|
||||
* full-state-replace planning scratchpad; `glob` is fast-glob file matching
|
||||
* (pi has no `Glob` built-in).
|
||||
*/
|
||||
|
||||
import { defineTool, type ToolDefinition } from '@earendil-works/pi-coding-agent';
|
||||
import { Type } from 'typebox';
|
||||
import { fs, glob, path } from 'zx';
|
||||
|
||||
import type { AuditLogger } from '../audit-logger.js';
|
||||
|
||||
export interface TodoItem {
|
||||
content: string;
|
||||
status: 'pending' | 'in_progress' | 'completed';
|
||||
activeForm: string;
|
||||
}
|
||||
|
||||
function renderTodos(todos: readonly TodoItem[]): string {
|
||||
const mark = (status: TodoItem['status']): string => {
|
||||
if (status === 'completed') return 'x';
|
||||
if (status === 'in_progress') return '~';
|
||||
return ' ';
|
||||
};
|
||||
return todos.map((todo) => `[${mark(todo.status)}] ${todo.content}`).join(' ');
|
||||
}
|
||||
|
||||
export function createTodoWriteTool(auditLogger: AuditLogger): ToolDefinition {
|
||||
export function createTodoWriteTool(): ToolDefinition {
|
||||
let current: TodoItem[] = [];
|
||||
|
||||
return defineTool({
|
||||
@@ -56,7 +45,6 @@ export function createTodoWriteTool(auditLogger: AuditLogger): ToolDefinition {
|
||||
async execute(_toolCallId, params) {
|
||||
current = params.todos as TodoItem[];
|
||||
const completed = current.filter((todo) => todo.status === 'completed').length;
|
||||
await auditLogger.logNote('todo', renderTodos(current));
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
/** Attempt-local working-tree copy used by the task-formation model boundary. */
|
||||
|
||||
import type { Dirent, Stats } from 'node:fs';
|
||||
import { cp, lstat, mkdir, mkdtemp, readdir, realpath, rm } from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { ArtifactIntegrityError, ReconciliationIoError } from '../reconciliation/artifact-store.js';
|
||||
|
||||
const JAIL_PREFIX = 'shannon-task-formation-';
|
||||
// Never copied into the model-readable jail: `.git` carries deliverables history, `.shannon` holds
|
||||
// scan internals, and `.pi` holds provider credentials. Any of these reaching the jail would expose
|
||||
// them to the tools the model drives. The post-copy verification re-checks their absence by name.
|
||||
const ALWAYS_EXCLUDED_NAMES = Object.freeze(['.git', '.shannon', '.pi'] as const);
|
||||
|
||||
export interface SourceJailOptions {
|
||||
readonly sourceRoot: string;
|
||||
readonly deliverablesPath: string;
|
||||
readonly reconciliationWorkspacePath: string;
|
||||
readonly signal?: AbortSignal;
|
||||
/** Test-only filesystem selector. Production uses `os.tmpdir()`. */
|
||||
readonly tempRoot?: string;
|
||||
}
|
||||
|
||||
/** One source-only jail plus the immutable deny rules used by its live tool gate. */
|
||||
export interface SourceJail {
|
||||
readonly dir: string;
|
||||
readonly deniedPaths: readonly string[];
|
||||
cleanup(): Promise<void>;
|
||||
}
|
||||
|
||||
function isErrno(error: unknown, code: string): boolean {
|
||||
return error instanceof Error && (error as NodeJS.ErrnoException).code === code;
|
||||
}
|
||||
|
||||
function cancellationError(signal: AbortSignal): Error {
|
||||
if (signal.reason instanceof Error) return signal.reason;
|
||||
return new DOMException('Task formation was cancelled.', 'AbortError');
|
||||
}
|
||||
|
||||
function checkCancellation(signal: AbortSignal | undefined): void {
|
||||
if (signal?.aborted === true) throw cancellationError(signal);
|
||||
}
|
||||
|
||||
// Path-confinement predicate: true only when `candidate` is `root` itself or lies beneath it.
|
||||
// A relative path that escapes upward (`..`) or is absolute means the candidate is outside the root.
|
||||
function isWithin(root: string, candidate: string): boolean {
|
||||
const relativePath = path.relative(root, candidate);
|
||||
return (
|
||||
relativePath === '' ||
|
||||
(!relativePath.startsWith(`..${path.sep}`) && relativePath !== '..' && !path.isAbsolute(relativePath))
|
||||
);
|
||||
}
|
||||
|
||||
async function relativeExclusion(
|
||||
sourceRoot: string,
|
||||
lexicalSourceRoot: string,
|
||||
candidate: string,
|
||||
): Promise<string | undefined> {
|
||||
const resolved = path.resolve(candidate);
|
||||
let relativePath: string | undefined;
|
||||
if (isWithin(sourceRoot, resolved)) {
|
||||
relativePath = path.relative(sourceRoot, resolved);
|
||||
} else if (isWithin(lexicalSourceRoot, resolved)) {
|
||||
relativePath = path.relative(lexicalSourceRoot, resolved);
|
||||
} else {
|
||||
try {
|
||||
const canonicalCandidate = await realpath(resolved);
|
||||
if (isWithin(sourceRoot, canonicalCandidate)) {
|
||||
relativePath = path.relative(sourceRoot, canonicalCandidate);
|
||||
}
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
if (relativePath === undefined) return undefined;
|
||||
|
||||
if (relativePath === '') {
|
||||
// An exclusion that resolves to the whole root would empty the jail. Fail closed rather than
|
||||
// copy nothing and hand the model an empty tree.
|
||||
throw new ArtifactIntegrityError('A task-formation exclusion resolves to the complete source root');
|
||||
}
|
||||
return relativePath;
|
||||
}
|
||||
|
||||
async function buildDynamicExclusions(
|
||||
options: SourceJailOptions,
|
||||
sourceRoot: string,
|
||||
lexicalSourceRoot: string,
|
||||
): Promise<readonly string[]> {
|
||||
const exclusions = (
|
||||
await Promise.all([
|
||||
relativeExclusion(sourceRoot, lexicalSourceRoot, options.deliverablesPath),
|
||||
relativeExclusion(sourceRoot, lexicalSourceRoot, options.reconciliationWorkspacePath),
|
||||
])
|
||||
).filter((value): value is string => value !== undefined);
|
||||
return Object.freeze([...new Set(exclusions)]);
|
||||
}
|
||||
|
||||
function pathHasAlwaysExcludedName(relativePath: string): boolean {
|
||||
const segments = relativePath.split(path.sep);
|
||||
return segments.some((segment) => (ALWAYS_EXCLUDED_NAMES as readonly string[]).includes(segment));
|
||||
}
|
||||
|
||||
function pathIsDynamicallyExcluded(relativePath: string, exclusions: readonly string[]): boolean {
|
||||
return exclusions.some((excluded) => relativePath === excluded || relativePath.startsWith(`${excluded}${path.sep}`));
|
||||
}
|
||||
|
||||
async function copySourceTree(
|
||||
sourceRoot: string,
|
||||
destination: string,
|
||||
dynamicExclusions: readonly string[],
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<void> {
|
||||
let entries: Dirent[];
|
||||
try {
|
||||
entries = (await readdir(sourceRoot, { withFileTypes: true })).sort((left, right) =>
|
||||
left.name.localeCompare(right.name),
|
||||
);
|
||||
} catch {
|
||||
throw new ReconciliationIoError('Unable to enumerate the task-formation source tree');
|
||||
}
|
||||
|
||||
// Cancellation is checked before every top-level entry and inside the copy filter so an aborted
|
||||
// scan stops promptly instead of copying a whole large tree first.
|
||||
for (const entry of entries) {
|
||||
checkCancellation(signal);
|
||||
const source = path.join(sourceRoot, entry.name);
|
||||
const destinationEntry = path.join(destination, entry.name);
|
||||
try {
|
||||
// verbatimSymlinks copies links as links rather than following them, so a link pointing
|
||||
// outside the tree cannot pull external content in; the filter then drops any path that
|
||||
// resolves outside the root, plus the always- and dynamically-excluded paths.
|
||||
await cp(source, destinationEntry, {
|
||||
recursive: true,
|
||||
verbatimSymlinks: true,
|
||||
errorOnExist: true,
|
||||
force: false,
|
||||
async filter(candidate) {
|
||||
checkCancellation(signal);
|
||||
const relativePath = path.relative(sourceRoot, candidate);
|
||||
if (relativePath === '' || !isWithin(sourceRoot, path.resolve(candidate))) return false;
|
||||
if (pathHasAlwaysExcludedName(relativePath)) return false;
|
||||
return !pathIsDynamicallyExcluded(relativePath, dynamicExclusions);
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (signal?.aborted === true) throw cancellationError(signal);
|
||||
if (error instanceof ArtifactIntegrityError) throw error;
|
||||
throw new ReconciliationIoError('Unable to copy the task-formation source tree');
|
||||
}
|
||||
}
|
||||
checkCancellation(signal);
|
||||
}
|
||||
|
||||
async function assertAlwaysExcludedNamesAbsent(directory: string, signal: AbortSignal | undefined): Promise<void> {
|
||||
checkCancellation(signal);
|
||||
let entries: Dirent[];
|
||||
try {
|
||||
entries = await readdir(directory, { withFileTypes: true });
|
||||
} catch {
|
||||
throw new ReconciliationIoError('Unable to verify the task-formation source jail');
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
checkCancellation(signal);
|
||||
if ((ALWAYS_EXCLUDED_NAMES as readonly string[]).includes(entry.name)) {
|
||||
throw new ArtifactIntegrityError('The task-formation source jail contains an excluded entry');
|
||||
}
|
||||
if (entry.isDirectory() && !entry.isSymbolicLink()) {
|
||||
await assertAlwaysExcludedNamesAbsent(path.join(directory, entry.name), signal);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function assertDynamicExclusionsAbsent(
|
||||
directory: string,
|
||||
exclusions: readonly string[],
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<void> {
|
||||
for (const excluded of exclusions) {
|
||||
checkCancellation(signal);
|
||||
try {
|
||||
await lstat(path.join(directory, excluded));
|
||||
} catch (error) {
|
||||
if (isErrno(error, 'ENOENT')) continue;
|
||||
throw new ReconciliationIoError('Unable to verify a task-formation jail exclusion');
|
||||
}
|
||||
throw new ArtifactIntegrityError('The task-formation source jail contains a protected workspace entry');
|
||||
}
|
||||
}
|
||||
|
||||
// Re-verify the copied tree independently of the copy filter: the jail root must be a real
|
||||
// directory (not a symlink), and no excluded name or protected workspace path may survive. This
|
||||
// catches a filter gap or a race during the copy before the model is allowed to read the tree.
|
||||
async function verifyJail(
|
||||
directory: string,
|
||||
dynamicExclusions: readonly string[],
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<void> {
|
||||
checkCancellation(signal);
|
||||
let stats: Stats;
|
||||
try {
|
||||
stats = await lstat(directory);
|
||||
} catch {
|
||||
throw new ReconciliationIoError('Unable to inspect the task-formation source jail');
|
||||
}
|
||||
if (stats.isSymbolicLink() || !stats.isDirectory()) {
|
||||
throw new ArtifactIntegrityError('The task-formation source jail is not a real directory');
|
||||
}
|
||||
await assertAlwaysExcludedNamesAbsent(directory, signal);
|
||||
await assertDynamicExclusionsAbsent(directory, dynamicExclusions, signal);
|
||||
checkCancellation(signal);
|
||||
}
|
||||
|
||||
async function removeJail(directory: string): Promise<void> {
|
||||
try {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
} catch {
|
||||
throw new ReconciliationIoError('Unable to remove the task-formation source jail');
|
||||
}
|
||||
|
||||
try {
|
||||
await lstat(directory);
|
||||
} catch (error) {
|
||||
if (isErrno(error, 'ENOENT')) return;
|
||||
throw new ReconciliationIoError('Unable to verify task-formation source-jail cleanup');
|
||||
}
|
||||
throw new ReconciliationIoError('Task-formation source-jail cleanup left the jail on disk');
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy the scanned working tree into an isolated temporary directory without following symlinks.
|
||||
* Every failure removes the attempt-local directory before it propagates.
|
||||
*/
|
||||
export async function materializeSourceJail(options: SourceJailOptions): Promise<SourceJail> {
|
||||
checkCancellation(options.signal);
|
||||
|
||||
const lexicalSourceRoot = path.resolve(options.sourceRoot);
|
||||
let sourceRoot: string;
|
||||
try {
|
||||
sourceRoot = await realpath(options.sourceRoot);
|
||||
const sourceStats = await lstat(sourceRoot);
|
||||
if (sourceStats.isSymbolicLink() || !sourceStats.isDirectory()) {
|
||||
throw new ArtifactIntegrityError('The task-formation source root is not a real directory');
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof ArtifactIntegrityError) throw error;
|
||||
throw new ReconciliationIoError('Unable to resolve the task-formation source root');
|
||||
}
|
||||
|
||||
let tempRoot: string;
|
||||
try {
|
||||
const configuredTempRoot = options.tempRoot ?? os.tmpdir();
|
||||
await mkdir(configuredTempRoot, { recursive: true });
|
||||
tempRoot = await realpath(configuredTempRoot);
|
||||
} catch {
|
||||
throw new ReconciliationIoError('Unable to resolve the task-formation temporary root');
|
||||
}
|
||||
// A temp root inside the source tree would make the copy try to copy the jail into itself.
|
||||
if (isWithin(sourceRoot, tempRoot)) {
|
||||
throw new ArtifactIntegrityError('The task-formation temporary root cannot be inside the source tree');
|
||||
}
|
||||
|
||||
const dynamicExclusions = await buildDynamicExclusions(options, sourceRoot, lexicalSourceRoot);
|
||||
let directory: string;
|
||||
try {
|
||||
directory = await mkdtemp(path.join(tempRoot, JAIL_PREFIX));
|
||||
} catch {
|
||||
throw new ReconciliationIoError('Unable to create the task-formation source jail');
|
||||
}
|
||||
|
||||
let cleaned = false;
|
||||
const cleanup = async (): Promise<void> => {
|
||||
if (cleaned) return;
|
||||
await removeJail(directory);
|
||||
cleaned = true;
|
||||
};
|
||||
|
||||
try {
|
||||
await copySourceTree(sourceRoot, directory, dynamicExclusions, options.signal);
|
||||
await verifyJail(directory, dynamicExclusions, options.signal);
|
||||
} catch (error) {
|
||||
await cleanup().catch(() => undefined);
|
||||
throw error;
|
||||
}
|
||||
|
||||
const deniedPaths = Object.freeze([...ALWAYS_EXCLUDED_NAMES, ...dynamicExclusions]);
|
||||
return Object.freeze({ dir: directory, deniedPaths, cleanup });
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
import type { AssistantMessage, Context, ToolCall } from '@earendil-works/pi-ai';
|
||||
import { Value } from 'typebox/value';
|
||||
import { providerFailureSentence } from '../../services/error-handling.js';
|
||||
import { type ModelHost, modelHost } from '../model-host.js';
|
||||
import type {
|
||||
StructuredGenerationPort,
|
||||
StructuredGenerationRequest,
|
||||
StructuredGenerationResult,
|
||||
} from '../structured-generation.js';
|
||||
import { type CapturedSubmitTool, createGenericSubmitTool } from '../submit-tool.js';
|
||||
|
||||
const ZERO_USAGE = { inputTokens: 0, outputTokens: 0, costUsd: 0 } as const;
|
||||
|
||||
// True only when this caller's own signal aborted and the error traces back to it. Walk a bounded,
|
||||
// cycle-guarded cause chain so a cancellation wrapped several layers deep is still recognized as a
|
||||
// cancellation and not misreported as a provider error. Without the `signal.aborted` gate an
|
||||
// unrelated AbortError from the provider could be mistaken for our cancellation.
|
||||
function isSignalCancellation(error: unknown, signal: AbortSignal | undefined): boolean {
|
||||
if (signal?.aborted !== true) return false;
|
||||
|
||||
let current: unknown = error;
|
||||
const seen = new Set<unknown>();
|
||||
for (let depth = 0; depth < 8 && current !== undefined && current !== null && !seen.has(current); depth++) {
|
||||
if (current === signal.reason) return true;
|
||||
seen.add(current);
|
||||
const errorName = current instanceof Error ? current.name : undefined;
|
||||
if (errorName === 'AbortError' || errorName === 'CancelledFailure') return true;
|
||||
current = current instanceof Error ? current.cause : undefined;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function responseUsage(response: AssistantMessage): StructuredGenerationResult['usage'] {
|
||||
return {
|
||||
inputTokens: response.usage.input,
|
||||
outputTokens: response.usage.output,
|
||||
costUsd: response.usage.cost.total,
|
||||
};
|
||||
}
|
||||
|
||||
type SubmitExecutor = (toolCallId: string, parameters: Record<string, unknown>) => Promise<unknown>;
|
||||
|
||||
async function captureSingleValidSubmission(
|
||||
toolCalls: readonly ToolCall[],
|
||||
submitTool: CapturedSubmitTool,
|
||||
): Promise<Array<{ name: string; arguments: unknown }>> {
|
||||
const returnedCalls = toolCalls.map((call) => ({ name: call.name, arguments: call.arguments }));
|
||||
const call = toolCalls.length === 1 ? toolCalls[0] : undefined;
|
||||
if (call?.name !== submitTool.tool.name) return returnedCalls;
|
||||
|
||||
if (!Value.Check(submitTool.tool.parameters, call.arguments)) return returnedCalls;
|
||||
|
||||
// completeSimple returns tool calls but does not execute them. Invoke the captured
|
||||
// definition only after its TypeBox validator accepts the sole submission.
|
||||
const execute = submitTool.tool.execute as unknown as SubmitExecutor;
|
||||
await execute(call.id, call.arguments);
|
||||
const captured = submitTool.getCaptured();
|
||||
return [{ name: call.name, arguments: captured }];
|
||||
}
|
||||
|
||||
async function generate(host: ModelHost, request: StructuredGenerationRequest): Promise<StructuredGenerationResult> {
|
||||
const submitTool = createGenericSubmitTool(request.tool.parametersJsonSchema);
|
||||
const context: Context = {
|
||||
...(request.systemPrompt !== undefined && { systemPrompt: request.systemPrompt }),
|
||||
messages: [{ role: 'user', content: request.userContent, timestamp: Date.now() }],
|
||||
tools: [
|
||||
{
|
||||
name: submitTool.tool.name,
|
||||
description: request.tool.description,
|
||||
parameters: submitTool.tool.parameters,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
let response: AssistantMessage;
|
||||
try {
|
||||
const selection = await host.resolve('small');
|
||||
// One enrichment batch is one billable provider request. Temporal owns any
|
||||
// retry after this boundary, so provider-level retries stay disabled here.
|
||||
response = await selection.modelRuntime.completeSimple(selection.model, context, {
|
||||
maxTokens: request.maxTokens,
|
||||
maxRetries: 0,
|
||||
...(request.signal !== undefined && { signal: request.signal }),
|
||||
});
|
||||
} catch (error) {
|
||||
if (isSignalCancellation(error, request.signal)) {
|
||||
return { stopReason: 'aborted', toolCalls: [], usage: ZERO_USAGE };
|
||||
}
|
||||
const failure = host.classify(error);
|
||||
return {
|
||||
stopReason: 'error',
|
||||
toolCalls: [],
|
||||
usage: ZERO_USAGE,
|
||||
errorMessage: providerFailureSentence(failure),
|
||||
providerFailure: { type: failure.type, retryable: failure.retryable },
|
||||
};
|
||||
}
|
||||
|
||||
// `pending` and `deferred` are non-final provider states: completeSimple resolves only on a
|
||||
// finished response and Shannon never requests a deferred one, so neither can carry a usable
|
||||
// submission. Classify them the way a rejected request is classified, so the caller retries
|
||||
// instead of reading an empty response as a successful generation.
|
||||
if (response.stopReason === 'error' || response.stopReason === 'pending' || response.stopReason === 'deferred') {
|
||||
const failure = host.classify(response);
|
||||
return {
|
||||
stopReason: 'error',
|
||||
toolCalls: [],
|
||||
usage: responseUsage(response),
|
||||
errorMessage: providerFailureSentence(failure),
|
||||
providerFailure: { type: failure.type, retryable: failure.retryable },
|
||||
};
|
||||
}
|
||||
if (response.stopReason === 'aborted') {
|
||||
// An abort with our signal set is a real cancellation. An abort without it is a provider-side
|
||||
// stop we did not ask for, so classify it as an error the caller can retry on.
|
||||
if (request.signal?.aborted === true) {
|
||||
return { stopReason: 'aborted', toolCalls: [], usage: responseUsage(response) };
|
||||
}
|
||||
const failure = host.classify(response);
|
||||
return {
|
||||
stopReason: 'error',
|
||||
toolCalls: [],
|
||||
usage: responseUsage(response),
|
||||
errorMessage: providerFailureSentence(failure),
|
||||
providerFailure: { type: failure.type, retryable: failure.retryable },
|
||||
};
|
||||
}
|
||||
|
||||
const toolCalls = response.content.filter((block): block is ToolCall => block.type === 'toolCall');
|
||||
const capturedCalls = await captureSingleValidSubmission(toolCalls, submitTool);
|
||||
return {
|
||||
stopReason: response.stopReason,
|
||||
toolCalls: capturedCalls,
|
||||
usage: responseUsage(response),
|
||||
};
|
||||
}
|
||||
|
||||
/** Build the one-request Pi adapter used by SAST enrichment. */
|
||||
export function createPiStructuredGenerationPort(host: ModelHost = modelHost): StructuredGenerationPort<void> {
|
||||
return {
|
||||
generate(request: StructuredGenerationRequest): Promise<StructuredGenerationResult> {
|
||||
return generate(host, request);
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,787 @@
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
/** Dedicated read-only Pi executor and live tool policy for Pass 1 task formation. */
|
||||
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import path from 'node:path';
|
||||
import type { AgentMessage } from '@earendil-works/pi-agent-core';
|
||||
import {
|
||||
type AgentSession,
|
||||
type AgentSessionEvent,
|
||||
createAgentSession,
|
||||
DefaultResourceLoader,
|
||||
defineTool,
|
||||
getAgentDir,
|
||||
SessionManager,
|
||||
SettingsManager,
|
||||
type ToolDefinition,
|
||||
} from '@earendil-works/pi-coding-agent';
|
||||
import { Type } from 'typebox';
|
||||
import { providerFailureSentence } from '../../services/error-handling.js';
|
||||
import type { ProviderFailure } from '../../types/errors.js';
|
||||
import { type ModelHost, modelHost } from '../model-host.js';
|
||||
import type { ModelSelection } from '../models.js';
|
||||
import type { ValidatingSubmitTool } from '../reconciliation/submit-validation.js';
|
||||
import { ConfinementError, compileRepositoryGlob, RepositoryConfinement } from '../sast/capella/tools/confinement.js';
|
||||
import { createCapellaRepositoryTools } from '../sast/capella/tools/repository-tools.js';
|
||||
import { PI_RETRY_SETTINGS } from './retry-settings.js';
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 30 * 60 * 1_000;
|
||||
const DEFAULT_MAX_TURNS = 64;
|
||||
const MAX_TIMEOUT_MS = 30 * 60 * 1_000;
|
||||
const MAX_TURNS = 128;
|
||||
const MAX_LIST_RESULTS = 500;
|
||||
const DEFAULT_LIST_RESULTS = 200;
|
||||
const MAX_OUTPUT_BYTES = 64 * 1024;
|
||||
// The live-tool-side counterpart of the source jail's copy-time exclusion (source-jail.ts): even if
|
||||
// one of these somehow existed in the jailed tree, the read/grep/find/ls/glob tools built below must
|
||||
// still refuse to serve it. `.git` is deliverables history, `.shannon` is scan internals, `.pi` is
|
||||
// provider credentials.
|
||||
const ALWAYS_DENIED_PATHS = Object.freeze(['.git', '.shannon', '.pi'] as const);
|
||||
const TRANSIENT_IO_CODES = new Set([
|
||||
'EAGAIN',
|
||||
'EBUSY',
|
||||
'ECONNREFUSED',
|
||||
'ECONNRESET',
|
||||
'EIO',
|
||||
'EMFILE',
|
||||
'ENFILE',
|
||||
'ENOMEM',
|
||||
'ENOSPC',
|
||||
'EPIPE',
|
||||
'EROFS',
|
||||
'ETIMEDOUT',
|
||||
]);
|
||||
|
||||
export const TASK_FORMATION_TOOL_NAMES = Object.freeze([
|
||||
'read',
|
||||
'grep',
|
||||
'find',
|
||||
'ls',
|
||||
'glob',
|
||||
'submit_result',
|
||||
] as const);
|
||||
|
||||
// The closed set of failure reasons the integration layer accepts as grounds to fall back to a
|
||||
// single-agent formation. Only a failure carrying one of these becomes a fallback; any other
|
||||
// failure propagates. Keep this in sync with the reasons the Temporal caller recognizes.
|
||||
export const TASK_FORMATION_FALLBACK_REASONS = Object.freeze([
|
||||
'retryable_model_failure',
|
||||
'missing_accepted_submission',
|
||||
'model_stage_timeout',
|
||||
] as const);
|
||||
|
||||
export type TaskFormationFallbackReason = (typeof TASK_FORMATION_FALLBACK_REASONS)[number];
|
||||
|
||||
export type TaskFormationExecutorFailureKind = 'model' | 'input' | 'confinement' | 'infrastructure';
|
||||
|
||||
/** Safe, bounded fields supplied by the activity wrapper for per-attempt executor correlation. */
|
||||
export interface TaskFormationExecutionContext {
|
||||
readonly executionKey?: string;
|
||||
readonly attempt?: number;
|
||||
readonly stage?: string;
|
||||
readonly vulnerabilityClass?: string;
|
||||
}
|
||||
|
||||
export interface TaskFormationUsage {
|
||||
readonly costUsd: number;
|
||||
readonly inputTokens: number;
|
||||
readonly outputTokens: number;
|
||||
}
|
||||
|
||||
export class TaskFormationExecutorError extends Error {
|
||||
override readonly name = 'TaskFormationExecutorError';
|
||||
readonly code: string;
|
||||
readonly retryable: boolean;
|
||||
readonly failureKind: TaskFormationExecutorFailureKind;
|
||||
readonly fallbackReason: TaskFormationFallbackReason | undefined;
|
||||
readonly usage: TaskFormationUsage;
|
||||
readonly modelCalls: number;
|
||||
|
||||
constructor(options: {
|
||||
code: string;
|
||||
message: string;
|
||||
retryable: boolean;
|
||||
failureKind: TaskFormationExecutorFailureKind;
|
||||
fallbackReason?: TaskFormationFallbackReason;
|
||||
usage?: TaskFormationUsage;
|
||||
modelCalls?: number;
|
||||
}) {
|
||||
super(options.message);
|
||||
this.code = options.code;
|
||||
this.retryable = options.retryable;
|
||||
this.failureKind = options.failureKind;
|
||||
this.fallbackReason = options.fallbackReason;
|
||||
this.usage = options.usage ?? zeroUsage();
|
||||
this.modelCalls = options.modelCalls ?? 0;
|
||||
}
|
||||
}
|
||||
|
||||
export interface TaskFormationExecutorRequest {
|
||||
readonly cwd: string;
|
||||
readonly systemPrompt: string;
|
||||
readonly modelContext: string;
|
||||
readonly deniedPaths: readonly string[];
|
||||
readonly submitTool: ValidatingSubmitTool;
|
||||
readonly signal: AbortSignal;
|
||||
readonly timeoutMs?: number;
|
||||
readonly maxTurns?: number;
|
||||
readonly correlation?: TaskFormationExecutionContext;
|
||||
}
|
||||
|
||||
export interface TaskFormationExecutorResult {
|
||||
readonly output: unknown;
|
||||
readonly usage: TaskFormationUsage;
|
||||
readonly providerId: string;
|
||||
readonly modelId: string;
|
||||
readonly modelCalls: 1;
|
||||
readonly registeredTools: readonly string[];
|
||||
}
|
||||
|
||||
export interface TaskFormationExecutor {
|
||||
run(request: TaskFormationExecutorRequest): Promise<TaskFormationExecutorResult>;
|
||||
}
|
||||
|
||||
interface SessionOutcome {
|
||||
readonly pendingProviderError: unknown;
|
||||
readonly promptError: unknown;
|
||||
readonly usage: TaskFormationUsage;
|
||||
}
|
||||
|
||||
interface ToolFactoryOptions {
|
||||
readonly cwd: string;
|
||||
readonly deniedPaths: readonly string[];
|
||||
}
|
||||
|
||||
function zeroUsage(): TaskFormationUsage {
|
||||
return { costUsd: 0, inputTokens: 0, outputTokens: 0 };
|
||||
}
|
||||
|
||||
/** Reject unknown values from Temporal failure details instead of widening semantic fallback. */
|
||||
export function isTaskFormationFallbackReason(value: unknown): value is TaskFormationFallbackReason {
|
||||
return (TASK_FORMATION_FALLBACK_REASONS as readonly unknown[]).includes(value);
|
||||
}
|
||||
|
||||
function errorCode(error: unknown): string | undefined {
|
||||
if (typeof error !== 'object' || error === null || !('code' in error)) return undefined;
|
||||
return typeof error.code === 'string' ? error.code : undefined;
|
||||
}
|
||||
|
||||
function isTransientIoFailure(error: unknown): boolean {
|
||||
const code = errorCode(error);
|
||||
if (code !== undefined && TRANSIENT_IO_CODES.has(code)) return true;
|
||||
if (error instanceof Error && error.cause !== undefined) return isTransientIoFailure(error.cause);
|
||||
return false;
|
||||
}
|
||||
|
||||
function safeIdentifier(value: string | undefined): string | undefined {
|
||||
if (value === undefined || !/^[A-Za-z0-9._:-]{1,128}$/u.test(value)) return undefined;
|
||||
return value;
|
||||
}
|
||||
|
||||
// Emit only bounded, format-checked correlation fields. Prompt text, model context, and source
|
||||
// content never enter the log line. An unsafe or missing identifier falls back to a synthetic one
|
||||
// rather than logging the caller's raw value.
|
||||
function executionLogContext(context: TaskFormationExecutionContext | undefined): Readonly<Record<string, unknown>> {
|
||||
const attempt = context?.attempt;
|
||||
return Object.freeze({
|
||||
executionKey: safeIdentifier(context?.executionKey) ?? randomUUID(),
|
||||
attempt: Number.isSafeInteger(attempt) && (attempt ?? 0) > 0 ? attempt : null,
|
||||
stage: safeIdentifier(context?.stage) ?? 'task-formation',
|
||||
class: safeIdentifier(context?.vulnerabilityClass) ?? 'unknown',
|
||||
});
|
||||
}
|
||||
|
||||
function finiteNonNegative(value: number): number {
|
||||
return Number.isFinite(value) ? Math.max(0, value) : 0;
|
||||
}
|
||||
|
||||
function sessionUsage(session: AgentSession): TaskFormationUsage {
|
||||
const stats = session.getSessionStats();
|
||||
return {
|
||||
costUsd: finiteNonNegative(stats.cost),
|
||||
inputTokens: finiteNonNegative(stats.tokens.input),
|
||||
outputTokens: finiteNonNegative(stats.tokens.output),
|
||||
};
|
||||
}
|
||||
|
||||
function boundedText(value: string): string {
|
||||
const bytes = Buffer.from(value, 'utf8');
|
||||
if (bytes.byteLength <= MAX_OUTPUT_BYTES) return value;
|
||||
return bytes.subarray(0, MAX_OUTPUT_BYTES).toString('utf8');
|
||||
}
|
||||
|
||||
function uniqueDeniedPaths(deniedPaths: readonly string[]): readonly string[] {
|
||||
return Object.freeze([...new Set([...ALWAYS_DENIED_PATHS, ...deniedPaths])]);
|
||||
}
|
||||
|
||||
// The session must register exactly the allowlisted tools. This is checked against the built tool
|
||||
// set and again against the live session's registered tools, so an injected or dropped tool fails
|
||||
// the session closed before the model runs.
|
||||
function hasExactToolSet(toolNames: readonly string[]): boolean {
|
||||
const expected = [...TASK_FORMATION_TOOL_NAMES].sort();
|
||||
const actual = [...toolNames].sort();
|
||||
return actual.length === expected.length && actual.every((name, index) => name === expected[index]);
|
||||
}
|
||||
|
||||
function createListTool(confinement: RepositoryConfinement): ToolDefinition {
|
||||
return defineTool({
|
||||
name: 'ls',
|
||||
label: 'List source directory',
|
||||
description: 'List bounded repository-relative entries without following symlinks.',
|
||||
promptSnippet: 'ls: list entries below one source directory',
|
||||
promptGuidelines: ['Use a repository-relative directory. Absolute paths and traversal are rejected.'],
|
||||
parameters: Type.Object(
|
||||
{
|
||||
path: Type.Optional(Type.String({ minLength: 1, maxLength: 1_024 })),
|
||||
limit: Type.Optional(Type.Integer({ minimum: 1, maximum: MAX_LIST_RESULTS })),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
async execute(_toolCallId, parameters, signal) {
|
||||
const requestedPath = parameters.path ?? '.';
|
||||
const budget = confinement.createBudget(signal);
|
||||
const searchRoot = await confinement.resolveExisting(requestedPath, true, budget);
|
||||
const entries = await confinement.enumerate(requestedPath, signal, budget);
|
||||
const names = new Set<string>();
|
||||
for (const entry of entries) {
|
||||
confinement.checkBudget(budget);
|
||||
const relativePath = pathRelative(searchRoot, entry.absolutePath);
|
||||
const [first, ...remaining] = relativePath.split('/');
|
||||
if (first) names.add(remaining.length > 0 ? `${first}/` : first);
|
||||
}
|
||||
|
||||
const limit = parameters.limit ?? DEFAULT_LIST_RESULTS;
|
||||
const output = [...names].sort().slice(0, limit);
|
||||
return {
|
||||
content: [{ type: 'text' as const, text: boundedText(output.join('\n') || 'No entries found.') }],
|
||||
details: { count: output.length, truncated: names.size > output.length },
|
||||
};
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
function pathRelative(root: string, candidate: string): string {
|
||||
const relativePath = path.relative(root, candidate);
|
||||
if (
|
||||
!relativePath ||
|
||||
relativePath.startsWith(`..${path.sep}`) ||
|
||||
relativePath === '..' ||
|
||||
path.isAbsolute(relativePath)
|
||||
) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'TOOL_PATH_RACE',
|
||||
message: 'Task-formation source path changed during access.',
|
||||
retryable: false,
|
||||
failureKind: 'confinement',
|
||||
});
|
||||
}
|
||||
return relativePath.split(path.sep).join('/');
|
||||
}
|
||||
|
||||
function createGlobTool(confinement: RepositoryConfinement): ToolDefinition {
|
||||
return defineTool({
|
||||
name: 'glob',
|
||||
label: 'Glob source files',
|
||||
description: 'Match bounded file globs from the source-jail root without following symlinks.',
|
||||
promptSnippet: 'glob: match source files from the jail root',
|
||||
promptGuidelines: ['Patterns are always rooted in the source jail.'],
|
||||
parameters: Type.Object(
|
||||
{
|
||||
pattern: Type.String({ minLength: 1, maxLength: 256 }),
|
||||
limit: Type.Optional(Type.Integer({ minimum: 1, maximum: MAX_LIST_RESULTS })),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
async execute(_toolCallId, parameters, signal) {
|
||||
const budget = confinement.createBudget(signal);
|
||||
const matcher = compileRepositoryGlob(parameters.pattern);
|
||||
const entries = await confinement.enumerate('.', signal, budget);
|
||||
const limit = parameters.limit ?? DEFAULT_LIST_RESULTS;
|
||||
const matches: string[] = [];
|
||||
let truncated = false;
|
||||
for (const entry of entries) {
|
||||
confinement.checkBudget(budget);
|
||||
if (!matcher.test(entry.path)) continue;
|
||||
if (matches.length >= limit) {
|
||||
truncated = true;
|
||||
break;
|
||||
}
|
||||
matches.push(entry.path);
|
||||
}
|
||||
return {
|
||||
content: [{ type: 'text' as const, text: boundedText(matches.join('\n') || 'No files found.') }],
|
||||
details: { count: matches.length, truncated },
|
||||
};
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/** Create the five code-owned source tools that share one canonical jail policy. */
|
||||
export async function createTaskFormationSourceTools(options: ToolFactoryOptions): Promise<readonly ToolDefinition[]> {
|
||||
const deniedPaths = uniqueDeniedPaths(options.deniedPaths);
|
||||
const capellaTools = await createCapellaRepositoryTools({
|
||||
repositoryRoot: options.cwd,
|
||||
deniedPaths,
|
||||
});
|
||||
const confinement = await RepositoryConfinement.create({
|
||||
repositoryRoot: options.cwd,
|
||||
deniedPaths,
|
||||
});
|
||||
const byName = new Map(capellaTools.map((tool) => [tool.name, tool]));
|
||||
const tools = [
|
||||
byName.get('read'),
|
||||
byName.get('grep'),
|
||||
byName.get('find'),
|
||||
createListTool(confinement),
|
||||
createGlobTool(confinement),
|
||||
];
|
||||
if (tools.some((tool) => tool === undefined)) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'TOOL_FACTORY_MISMATCH',
|
||||
message: 'Task-formation source tool factory returned an incomplete set.',
|
||||
retryable: false,
|
||||
failureKind: 'confinement',
|
||||
});
|
||||
}
|
||||
return Object.freeze(tools as ToolDefinition[]);
|
||||
}
|
||||
|
||||
function cancellationError(signal: AbortSignal): Error {
|
||||
if (signal.reason instanceof Error) return signal.reason;
|
||||
return new DOMException('Task formation was cancelled.', 'AbortError');
|
||||
}
|
||||
|
||||
function raceWithAbort<T>(promise: Promise<T>, signal: AbortSignal): Promise<T> {
|
||||
if (signal.aborted) return Promise.reject(cancellationError(signal));
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
const onAbort = (): void => reject(cancellationError(signal));
|
||||
signal.addEventListener('abort', onAbort, { once: true });
|
||||
promise.then(
|
||||
(value) => {
|
||||
signal.removeEventListener('abort', onAbort);
|
||||
resolve(value);
|
||||
},
|
||||
(error: unknown) => {
|
||||
signal.removeEventListener('abort', onAbort);
|
||||
reject(error);
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
function validateRequest(request: TaskFormationExecutorRequest): { timeoutMs: number; maxTurns: number } {
|
||||
const timeoutMs = request.timeoutMs ?? DEFAULT_TIMEOUT_MS;
|
||||
const maxTurns = request.maxTurns ?? DEFAULT_MAX_TURNS;
|
||||
if (!request.cwd || !request.systemPrompt || !request.modelContext) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'INVALID_REQUEST',
|
||||
message: 'Task-formation executor input is incomplete.',
|
||||
retryable: false,
|
||||
failureKind: 'input',
|
||||
});
|
||||
}
|
||||
if (!Number.isInteger(timeoutMs) || timeoutMs < 1 || timeoutMs > MAX_TIMEOUT_MS) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'INVALID_TIMEOUT',
|
||||
message: `Task-formation timeout must be a positive integer no greater than ${MAX_TIMEOUT_MS} milliseconds.`,
|
||||
retryable: false,
|
||||
failureKind: 'input',
|
||||
});
|
||||
}
|
||||
if (!Number.isInteger(maxTurns) || maxTurns < 1 || maxTurns > MAX_TURNS) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'INVALID_TURN_LIMIT',
|
||||
message: 'Task-formation turn limit is outside its bounded range.',
|
||||
retryable: false,
|
||||
failureKind: 'input',
|
||||
});
|
||||
}
|
||||
return { timeoutMs, maxTurns };
|
||||
}
|
||||
|
||||
function isAbortLike(error: unknown): boolean {
|
||||
return error instanceof Error && (error.name === 'AbortError' || error.name === 'TimeoutError');
|
||||
}
|
||||
|
||||
function classifyModelFailure(host: ModelHost, error: unknown): ProviderFailure {
|
||||
if (isAbortLike(error)) {
|
||||
return {
|
||||
type: 'AgentExecutionError',
|
||||
category: 'transport',
|
||||
retryable: true,
|
||||
message: 'The provider request ended before task formation completed.',
|
||||
};
|
||||
}
|
||||
return host.classify(error);
|
||||
}
|
||||
|
||||
function executorLog(level: 'info' | 'warn', fields: Readonly<Record<string, unknown>>): void {
|
||||
console[level](JSON.stringify({ component: 'task-formation-executor', ...fields }));
|
||||
}
|
||||
|
||||
class StandaloneTaskFormationExecutor implements TaskFormationExecutor {
|
||||
private readonly host: ModelHost;
|
||||
|
||||
constructor(host: ModelHost) {
|
||||
this.host = host;
|
||||
}
|
||||
|
||||
async run(request: TaskFormationExecutorRequest): Promise<TaskFormationExecutorResult> {
|
||||
const logContext = executionLogContext(request.correlation);
|
||||
let timeoutMs: number;
|
||||
let maxTurns: number;
|
||||
try {
|
||||
({ timeoutMs, maxTurns } = validateRequest(request));
|
||||
} catch (error) {
|
||||
const failure = this.normalizeFailure(error);
|
||||
executorLog('warn', {
|
||||
...logContext,
|
||||
event: 'finished',
|
||||
outcome: 'failed',
|
||||
code: failure.code,
|
||||
failureKind: failure.failureKind,
|
||||
retryable: failure.retryable,
|
||||
});
|
||||
throw failure;
|
||||
}
|
||||
const controller = new AbortController();
|
||||
let termination: 'cancellation' | 'timeout' | 'turn-limit' | undefined;
|
||||
let session: AgentSession | undefined;
|
||||
let unsubscribe: (() => void) | undefined;
|
||||
let timeout: NodeJS.Timeout | undefined;
|
||||
let requestStarted = false;
|
||||
let turnCount = 0;
|
||||
|
||||
const terminate = (reason: 'cancellation' | 'timeout' | 'turn-limit'): void => {
|
||||
if (termination !== undefined) return;
|
||||
termination = reason;
|
||||
controller.abort(new DOMException(`Task-formation session ${reason}.`, 'AbortError'));
|
||||
void session?.abort().catch(() => undefined);
|
||||
};
|
||||
const onCancellation = (): void => terminate('cancellation');
|
||||
|
||||
if (request.signal.aborted) {
|
||||
executorLog('info', { ...logContext, event: 'finished', outcome: 'cancelled' });
|
||||
throw cancellationError(request.signal);
|
||||
}
|
||||
request.signal.addEventListener('abort', onCancellation, { once: true });
|
||||
timeout = setTimeout(() => terminate('timeout'), timeoutMs);
|
||||
|
||||
try {
|
||||
let selection: ModelSelection;
|
||||
try {
|
||||
selection = await raceWithAbort(this.host.resolve('medium'), controller.signal);
|
||||
} catch (error) {
|
||||
if (termination === 'cancellation') throw cancellationError(request.signal);
|
||||
if (termination === 'timeout') throw this.timeoutError(zeroUsage(), 0);
|
||||
const failure = classifyModelFailure(this.host, error);
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'MODEL_SELECTION_FAILURE',
|
||||
message: providerFailureSentence(failure),
|
||||
retryable: failure.retryable,
|
||||
failureKind: 'model',
|
||||
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
|
||||
});
|
||||
}
|
||||
|
||||
const sourceTools = await raceWithAbort(
|
||||
createTaskFormationSourceTools({ cwd: request.cwd, deniedPaths: request.deniedPaths }),
|
||||
controller.signal,
|
||||
);
|
||||
const customTools = [...sourceTools, request.submitTool.tool];
|
||||
const toolNames = customTools.map((tool) => tool.name);
|
||||
if (!hasExactToolSet(toolNames)) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'TOOL_POLICY_MISMATCH',
|
||||
message: 'Task-formation source tool policy does not match the exact allowlist.',
|
||||
retryable: false,
|
||||
failureKind: 'confinement',
|
||||
});
|
||||
}
|
||||
|
||||
const agentDir = getAgentDir();
|
||||
const settingsManager = SettingsManager.inMemory({
|
||||
retry: PI_RETRY_SETTINGS,
|
||||
compaction: { enabled: true },
|
||||
});
|
||||
const resourceLoader = new DefaultResourceLoader({
|
||||
cwd: request.cwd,
|
||||
agentDir,
|
||||
settingsManager,
|
||||
systemPrompt: `${request.systemPrompt}${request.submitTool.directive ?? ''}`,
|
||||
appendSystemPrompt: [],
|
||||
noExtensions: true,
|
||||
noSkills: true,
|
||||
noPromptTemplates: true,
|
||||
noThemes: true,
|
||||
noContextFiles: true,
|
||||
});
|
||||
await raceWithAbort(resourceLoader.reload(), controller.signal);
|
||||
|
||||
const sessionPromise = createAgentSession({
|
||||
cwd: request.cwd,
|
||||
agentDir,
|
||||
model: selection.model,
|
||||
modelRuntime: selection.modelRuntime,
|
||||
noTools: 'all',
|
||||
tools: toolNames,
|
||||
customTools,
|
||||
resourceLoader,
|
||||
sessionManager: SessionManager.inMemory(),
|
||||
settingsManager,
|
||||
});
|
||||
try {
|
||||
({ session } = await raceWithAbort(sessionPromise, controller.signal));
|
||||
} catch (error) {
|
||||
void sessionPromise.then(
|
||||
async ({ session: lateSession }) => {
|
||||
await lateSession.abort().catch(() => undefined);
|
||||
lateSession.dispose();
|
||||
},
|
||||
() => undefined,
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
|
||||
if (controller.signal.aborted) {
|
||||
await session.abort().catch(() => undefined);
|
||||
} else {
|
||||
controller.signal.addEventListener('abort', () => void session?.abort().catch(() => undefined), {
|
||||
once: true,
|
||||
});
|
||||
}
|
||||
|
||||
const registeredTools = session.getAllTools().map((tool) => tool.name);
|
||||
if (!hasExactToolSet(registeredTools)) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'LIVE_TOOL_POLICY_MISMATCH',
|
||||
message: 'The live task-formation session registered a tool outside the exact allowlist.',
|
||||
retryable: false,
|
||||
failureKind: 'confinement',
|
||||
});
|
||||
}
|
||||
|
||||
executorLog('info', {
|
||||
...logContext,
|
||||
event: 'started',
|
||||
provider: selection.providerId,
|
||||
model: selection.modelId,
|
||||
tools: registeredTools,
|
||||
resources: { context: false, extensions: false, prompts: false, skills: false },
|
||||
});
|
||||
|
||||
let pendingProviderError: unknown;
|
||||
unsubscribe = session.subscribe((event: AgentSessionEvent) => {
|
||||
if (event.type !== 'turn_end') return;
|
||||
turnCount += 1;
|
||||
const message: AgentMessage = event.message;
|
||||
if (message.role === 'assistant' && message.stopReason === 'error') {
|
||||
pendingProviderError ??= message;
|
||||
}
|
||||
const needsAnotherTurn = message.role === 'assistant' && message.stopReason === 'toolUse';
|
||||
if (turnCount >= maxTurns && needsAnotherTurn && request.submitTool.getAcceptedCount() === 0) {
|
||||
terminate('turn-limit');
|
||||
}
|
||||
});
|
||||
|
||||
let promptError: unknown;
|
||||
requestStarted = true;
|
||||
try {
|
||||
await raceWithAbort(session.prompt(request.modelContext, { expandPromptTemplates: false }), controller.signal);
|
||||
} catch (error) {
|
||||
promptError = error;
|
||||
}
|
||||
|
||||
const outcome: SessionOutcome = {
|
||||
pendingProviderError,
|
||||
promptError,
|
||||
usage: sessionUsage(session),
|
||||
};
|
||||
const output = this.resolveOutcome(request, outcome, termination, requestStarted);
|
||||
executorLog('info', { ...logContext, event: 'finished', outcome: 'succeeded', usage: outcome.usage });
|
||||
return {
|
||||
output,
|
||||
usage: outcome.usage,
|
||||
providerId: selection.providerId,
|
||||
modelId: selection.modelId,
|
||||
modelCalls: 1,
|
||||
registeredTools: Object.freeze([...registeredTools]),
|
||||
};
|
||||
} catch (error) {
|
||||
// Termination reason wins over whatever error surfaced. A local timeout or an abort aborts the
|
||||
// in-flight provider call, so the caught error is usually that induced abort; reporting it as a
|
||||
// model failure would erase the real cause. Cancellation keeps its own identity ahead of timeout.
|
||||
if (termination === 'cancellation') {
|
||||
executorLog('info', { ...logContext, event: 'finished', outcome: 'cancelled' });
|
||||
throw cancellationError(request.signal);
|
||||
}
|
||||
|
||||
let failure: TaskFormationExecutorError;
|
||||
if (termination === 'timeout') {
|
||||
const usage = session ? sessionUsage(session) : zeroUsage();
|
||||
const modelCalls = requestStarted ? 1 : 0;
|
||||
failure = this.timeoutError(usage, modelCalls);
|
||||
} else {
|
||||
failure = this.normalizeFailure(error);
|
||||
}
|
||||
executorLog('warn', {
|
||||
...logContext,
|
||||
event: 'finished',
|
||||
outcome: 'failed',
|
||||
code: failure.code,
|
||||
failureKind: failure.failureKind,
|
||||
retryable: failure.retryable,
|
||||
...(failure.fallbackReason !== undefined && { fallbackReason: failure.fallbackReason }),
|
||||
usage: failure.usage,
|
||||
modelCalls: failure.modelCalls,
|
||||
});
|
||||
throw failure;
|
||||
} finally {
|
||||
if (timeout) clearTimeout(timeout);
|
||||
request.signal.removeEventListener('abort', onCancellation);
|
||||
unsubscribe?.();
|
||||
try {
|
||||
session?.dispose();
|
||||
} catch {
|
||||
executorLog('warn', { ...logContext, event: 'cleanup-failed' });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private normalizeFailure(error: unknown): TaskFormationExecutorError {
|
||||
if (error instanceof TaskFormationExecutorError) return error;
|
||||
if (error instanceof ConfinementError) {
|
||||
return new TaskFormationExecutorError({
|
||||
code: `CONFINEMENT_${error.code}`,
|
||||
message: error.message,
|
||||
retryable: false,
|
||||
failureKind: 'confinement',
|
||||
});
|
||||
}
|
||||
if (isTransientIoFailure(error)) {
|
||||
return new TaskFormationExecutorError({
|
||||
code: 'SESSION_INFRASTRUCTURE_FAILURE',
|
||||
message: 'Task-formation session setup encountered a retryable infrastructure failure.',
|
||||
retryable: true,
|
||||
failureKind: 'infrastructure',
|
||||
});
|
||||
}
|
||||
|
||||
const failure = classifyModelFailure(this.host, error);
|
||||
if (failure.type === 'ConfigurationError') {
|
||||
return new TaskFormationExecutorError({
|
||||
code: 'MODEL_CONFIGURATION_FAILURE',
|
||||
message: providerFailureSentence(failure),
|
||||
retryable: false,
|
||||
failureKind: 'input',
|
||||
});
|
||||
}
|
||||
return new TaskFormationExecutorError({
|
||||
code: failure.type === 'AuthenticationError' ? 'PROVIDER_AUTHENTICATION_FAILURE' : 'MODEL_SESSION_FAILURE',
|
||||
message: providerFailureSentence(failure),
|
||||
retryable: failure.retryable,
|
||||
failureKind: 'model',
|
||||
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
|
||||
});
|
||||
}
|
||||
|
||||
private timeoutError(usage: TaskFormationUsage, modelCalls: number): TaskFormationExecutorError {
|
||||
return new TaskFormationExecutorError({
|
||||
code: 'MODEL_STAGE_TIMEOUT',
|
||||
message: 'Task formation exceeded its model-stage timeout.',
|
||||
retryable: true,
|
||||
failureKind: 'model',
|
||||
fallbackReason: 'model_stage_timeout',
|
||||
usage,
|
||||
modelCalls,
|
||||
});
|
||||
}
|
||||
|
||||
private resolveOutcome(
|
||||
request: TaskFormationExecutorRequest,
|
||||
outcome: SessionOutcome,
|
||||
termination: 'cancellation' | 'timeout' | 'turn-limit' | undefined,
|
||||
requestStarted: boolean,
|
||||
): unknown {
|
||||
const modelCalls = requestStarted ? 1 : 0;
|
||||
if (termination === 'cancellation') throw cancellationError(request.signal);
|
||||
if (termination === 'timeout') throw this.timeoutError(outcome.usage, modelCalls);
|
||||
if (termination === 'turn-limit') {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'TURN_LIMIT',
|
||||
message: 'Task formation exhausted its bounded model turn limit.',
|
||||
retryable: true,
|
||||
failureKind: 'model',
|
||||
fallbackReason: 'retryable_model_failure',
|
||||
usage: outcome.usage,
|
||||
modelCalls,
|
||||
});
|
||||
}
|
||||
if (request.submitTool.getAcceptedCount() > 1) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'DUPLICATE_ACCEPTED_SUBMISSION',
|
||||
message: 'Task formation accepted more than one submission.',
|
||||
retryable: true,
|
||||
failureKind: 'model',
|
||||
fallbackReason: 'retryable_model_failure',
|
||||
usage: outcome.usage,
|
||||
modelCalls,
|
||||
});
|
||||
}
|
||||
if (outcome.pendingProviderError !== undefined) {
|
||||
const failure = classifyModelFailure(this.host, outcome.pendingProviderError);
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'PROVIDER_FAILURE',
|
||||
message: providerFailureSentence(failure),
|
||||
retryable: failure.retryable,
|
||||
failureKind: 'model',
|
||||
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
|
||||
usage: outcome.usage,
|
||||
modelCalls,
|
||||
});
|
||||
}
|
||||
// A prompt error is a real failure unless exactly one submission was already accepted and the
|
||||
// error is an abort: the submit tool terminates the session, so that abort is the expected end of
|
||||
// a successful run, not a fault.
|
||||
if (
|
||||
outcome.promptError !== undefined &&
|
||||
!(request.submitTool.getAcceptedCount() === 1 && isAbortLike(outcome.promptError))
|
||||
) {
|
||||
const failure = classifyModelFailure(this.host, outcome.promptError);
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'MODEL_SESSION_FAILURE',
|
||||
message: providerFailureSentence(failure),
|
||||
retryable: failure.retryable,
|
||||
failureKind: 'model',
|
||||
...(failure.retryable && { fallbackReason: 'retryable_model_failure' }),
|
||||
usage: outcome.usage,
|
||||
modelCalls,
|
||||
});
|
||||
}
|
||||
|
||||
const output = request.submitTool.getCaptured();
|
||||
if (request.submitTool.getAcceptedCount() !== 1 || output === undefined) {
|
||||
throw new TaskFormationExecutorError({
|
||||
code: 'MISSING_ACCEPTED_SUBMISSION',
|
||||
message: 'Task formation ended without one accepted submission.',
|
||||
retryable: true,
|
||||
failureKind: 'model',
|
||||
fallbackReason: 'missing_accepted_submission',
|
||||
usage: outcome.usage,
|
||||
modelCalls,
|
||||
});
|
||||
}
|
||||
return output;
|
||||
}
|
||||
}
|
||||
|
||||
export function createTaskFormationExecutor(host: ModelHost = modelHost): TaskFormationExecutor {
|
||||
return new StandaloneTaskFormationExecutor(host);
|
||||
}
|
||||
|
||||
export const taskFormationExecutor: TaskFormationExecutor = createTaskFormationExecutor();
|
||||
@@ -1,20 +1,10 @@
|
||||
// Copyright (C) 2025 Keygraph, Inc.
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
/**
|
||||
* Generic `task` tool — pi.dev ships no built-in Task tool, so this supplies the
|
||||
* Task-delegation surface Shannon's prompts require.
|
||||
*
|
||||
* Shannon's prompts mandate Task delegation (recon source tracer; the vuln
|
||||
* agents delegate *every* code review; the exploit agents delegate automation),
|
||||
* so this tool is required for parity, not optional. It spawns a nested pi
|
||||
* session with the parent's resolved model object (never a tier string — that
|
||||
* would route sub-agents through hardcoded IDs and leak billing), the parent's
|
||||
* resource loader, and a fixed child tool surface.
|
||||
*/
|
||||
/** Generic child-session delegation for the pi harness. */
|
||||
|
||||
import { type AssistantMessage, type Model, Type } from '@earendil-works/pi-ai';
|
||||
import {
|
||||
@@ -27,39 +17,70 @@ import {
|
||||
SettingsManager,
|
||||
type ToolDefinition,
|
||||
} from '@earendil-works/pi-coding-agent';
|
||||
import { type LoggableAgentName, normalizeSemanticLabel } from '../../audit/safe-fields.js';
|
||||
import { PI_RETRY_SETTINGS } from './retry-settings.js';
|
||||
import { TraceEmitter } from './trace-emitter.js';
|
||||
|
||||
export interface TaskToolContext {
|
||||
cwd: string;
|
||||
readonly cwd: string;
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
model: Model<any>;
|
||||
/** Parent's model/auth runtime, reused so sub-agents share its resolved credential. */
|
||||
modelRuntime: ModelRuntime;
|
||||
resourceLoader: ResourceLoader;
|
||||
cancellationSignal?: AbortSignal | undefined;
|
||||
/**
|
||||
* Reports the cost/tokens of each spawned sub-session back to the caller.
|
||||
* Sub-agents run in their own pi sessions that the parent has no reference to,
|
||||
* so without this their spend (the bulk of a whitebox run, since Shannon
|
||||
* prompts delegate the heavy work) is invisible to billing.
|
||||
*/
|
||||
onUsage?: (usage: {
|
||||
cost: number;
|
||||
inputTokens: number;
|
||||
outputTokens: number;
|
||||
cacheReadTokens: number;
|
||||
cacheWriteTokens: number;
|
||||
readonly model: Model<any>;
|
||||
readonly modelRuntime: ModelRuntime;
|
||||
readonly createResourceLoader: () => Promise<ResourceLoader>;
|
||||
readonly parentAgentName: LoggableAgentName;
|
||||
readonly workflowLogPath?: string | undefined;
|
||||
readonly onDelegationStart?: ((child: string) => Promise<void>) | undefined;
|
||||
readonly cancellationSignal?: AbortSignal | undefined;
|
||||
readonly onUsage?: (usage: {
|
||||
readonly cost: number;
|
||||
readonly inputTokens: number;
|
||||
readonly outputTokens: number;
|
||||
readonly cacheReadTokens: number;
|
||||
readonly cacheWriteTokens: number;
|
||||
}) => void;
|
||||
}
|
||||
|
||||
// Deliberately excludes `task` (no recursive delegation, so a child cannot spawn further children)
|
||||
// and every collector/submit tool (structured output stays owned by the top-level agent session
|
||||
// that the workflow reads back). A child session gets only plain file and shell access.
|
||||
const CHILD_TOOLS = ['read', 'grep', 'find', 'ls', 'write', 'bash'];
|
||||
const CHILD_FAILURE_TEXT = '[Sub-agent task failed before completion]';
|
||||
const CHILD_CANCELLED_TEXT = '[Sub-agent task was cancelled]';
|
||||
|
||||
function textResult(text: string) {
|
||||
return { content: [{ type: 'text' as const, text }], details: undefined };
|
||||
}
|
||||
|
||||
/**
|
||||
* Assigns each child a stable, safe display identity from its description. A duplicate of a
|
||||
* live sibling's name gets a monotonic start-order suffix (`route mapper #2`); a missing or
|
||||
* unsafe description becomes `subagent N`. State is shared across one parent's task calls,
|
||||
* and the assignment block runs synchronously so parallel calls never race on it.
|
||||
*/
|
||||
// Keep the base short enough that a `#N` suffix still fits the identity validator's length
|
||||
// bound (48); a longer description falls back to `subagent N` rather than being dropped.
|
||||
const MAX_CHILD_BASE_LENGTH = 40;
|
||||
|
||||
function createChildNamer(): (description: unknown) => string {
|
||||
const namedCounts = new Map<string, number>();
|
||||
let anonymousCount = 0;
|
||||
return (description) => {
|
||||
const base = normalizeSemanticLabel(description);
|
||||
if (base === undefined || base.length > MAX_CHILD_BASE_LENGTH) {
|
||||
anonymousCount += 1;
|
||||
return `subagent ${anonymousCount}`;
|
||||
}
|
||||
const nextOrdinal = (namedCounts.get(base) ?? 0) + 1;
|
||||
namedCounts.set(base, nextOrdinal);
|
||||
return nextOrdinal === 1 ? base : `${base} #${nextOrdinal}`;
|
||||
};
|
||||
}
|
||||
|
||||
export function createTaskTool(config: TaskToolContext): ToolDefinition {
|
||||
const taskTool: ToolDefinition = defineTool({
|
||||
const nameChild = createChildNamer();
|
||||
const logPath = config.workflowLogPath;
|
||||
|
||||
return defineTool({
|
||||
name: 'task',
|
||||
label: 'Task',
|
||||
description:
|
||||
@@ -80,59 +101,83 @@ export function createTaskTool(config: TaskToolContext): ToolDefinition {
|
||||
description: Type.Optional(Type.String({ description: 'A short (3-5 word) description of the task.' })),
|
||||
}),
|
||||
async execute(_toolCallId, params) {
|
||||
// Assign the identity synchronously, before any await, so concurrent siblings can't race.
|
||||
const child = nameChild(params.description);
|
||||
const emitter = logPath
|
||||
? new TraceEmitter(logPath, { kind: 'child', parent: config.parentAgentName, child })
|
||||
: undefined;
|
||||
const startedAt = Date.now();
|
||||
|
||||
// The parent's emitter first writes the raw task invocation, then this delegation
|
||||
// record. Awaiting it prevents the child emitter from overtaking its lineage start.
|
||||
await config.onDelegationStart?.(child);
|
||||
|
||||
const agentDir = getAgentDir();
|
||||
const { session: subSession } = await createAgentSession({
|
||||
cwd: config.cwd,
|
||||
agentDir,
|
||||
resourceLoader: config.resourceLoader,
|
||||
model: config.model,
|
||||
tools: CHILD_TOOLS,
|
||||
modelRuntime: config.modelRuntime,
|
||||
sessionManager: SessionManager.inMemory(config.cwd),
|
||||
settingsManager: SettingsManager.inMemory({
|
||||
retry: PI_RETRY_SETTINGS,
|
||||
compaction: { enabled: true },
|
||||
}),
|
||||
});
|
||||
let subSession: Awaited<ReturnType<typeof createAgentSession>>['session'] | undefined;
|
||||
let resultText = '';
|
||||
let subCost = 0;
|
||||
let turns = 0;
|
||||
let operations = 0;
|
||||
let failed = false;
|
||||
let fatalFailure = false;
|
||||
|
||||
const abortChildSession = (): void => {
|
||||
void subSession.abort().catch(() => {
|
||||
// Parent logger is not available inside the tool; dispose still tears
|
||||
// down the session if abort itself rejects.
|
||||
void subSession?.abort().catch(() => {
|
||||
// Dispose below still tears down the child session.
|
||||
});
|
||||
};
|
||||
const onCancellation = (): void => abortChildSession();
|
||||
if (config.cancellationSignal?.aborted) {
|
||||
abortChildSession();
|
||||
} else {
|
||||
config.cancellationSignal?.addEventListener('abort', onCancellation, { once: true });
|
||||
}
|
||||
|
||||
let resultText = '';
|
||||
let subCost = 0;
|
||||
subSession.subscribe((event) => {
|
||||
if (event.type === 'turn_end') {
|
||||
const msg = event.message as AssistantMessage | undefined;
|
||||
for (const block of msg?.content ?? []) {
|
||||
try {
|
||||
const resourceLoader = await config.createResourceLoader();
|
||||
({ session: subSession } = await createAgentSession({
|
||||
cwd: config.cwd,
|
||||
agentDir,
|
||||
resourceLoader,
|
||||
model: config.model,
|
||||
tools: CHILD_TOOLS,
|
||||
modelRuntime: config.modelRuntime,
|
||||
sessionManager: SessionManager.inMemory(config.cwd),
|
||||
settingsManager: SettingsManager.inMemory({
|
||||
retry: PI_RETRY_SETTINGS,
|
||||
compaction: { enabled: true },
|
||||
}),
|
||||
}));
|
||||
|
||||
if (config.cancellationSignal?.aborted) {
|
||||
abortChildSession();
|
||||
} else {
|
||||
config.cancellationSignal?.addEventListener('abort', onCancellation, { once: true });
|
||||
}
|
||||
|
||||
subSession.subscribe((event) => {
|
||||
if (event.type === 'tool_execution_start') {
|
||||
operations += 1;
|
||||
emitter?.toolStart(event.toolCallId, event.toolName, event.args);
|
||||
return;
|
||||
}
|
||||
if (event.type === 'tool_execution_end') {
|
||||
emitter?.toolEnd(event.toolCallId, event.isError);
|
||||
return;
|
||||
}
|
||||
if (event.type !== 'turn_end') return;
|
||||
turns += 1;
|
||||
const message = event.message as AssistantMessage | undefined;
|
||||
for (const block of message?.content ?? []) {
|
||||
if (block.type === 'text' && block.text) {
|
||||
resultText += (resultText ? '\n' : '') + block.text;
|
||||
}
|
||||
}
|
||||
if (msg?.usage?.cost?.total != null) subCost += msg.usage.cost.total;
|
||||
}
|
||||
});
|
||||
if (message?.usage?.cost?.total != null) subCost += message.usage.cost.total;
|
||||
});
|
||||
|
||||
let swallowedError: string | undefined;
|
||||
try {
|
||||
try {
|
||||
await subSession.prompt(params.prompt);
|
||||
} catch (err) {
|
||||
const errorMsg = err instanceof Error ? err.message : String(err);
|
||||
resultText += `\n[Sub-agent error: ${errorMsg}]`;
|
||||
} catch {
|
||||
failed = true;
|
||||
}
|
||||
if (subSession.state.errorMessage !== undefined) failed = true;
|
||||
|
||||
swallowedError = subSession.state.errorMessage;
|
||||
// Read stats before dispose; reconcile cost the same way the parent does.
|
||||
const subStats = subSession.getSessionStats();
|
||||
if (subStats.cost > subCost) subCost = subStats.cost;
|
||||
config.onUsage?.({
|
||||
@@ -142,18 +187,37 @@ export function createTaskTool(config: TaskToolContext): ToolDefinition {
|
||||
cacheReadTokens: subStats.tokens.cacheRead,
|
||||
cacheWriteTokens: subStats.tokens.cacheWrite,
|
||||
});
|
||||
} catch {
|
||||
fatalFailure = true;
|
||||
} finally {
|
||||
config.cancellationSignal?.removeEventListener('abort', onCancellation);
|
||||
subSession.dispose();
|
||||
subSession?.dispose();
|
||||
}
|
||||
|
||||
if (swallowedError && !resultText.includes(swallowedError)) {
|
||||
resultText += `\n[Sub-agent error: ${swallowedError}]`;
|
||||
const durationMs = Date.now() - startedAt;
|
||||
if (config.cancellationSignal?.aborted) {
|
||||
emitter?.sessionFailure('CANCELLED', durationMs);
|
||||
await emitter?.flush();
|
||||
return textResult(CHILD_CANCELLED_TEXT);
|
||||
}
|
||||
// `fatalFailure` means the child session itself never came up (createAgentSession threw), so
|
||||
// there is no session result to hand back, and this rethrows, which pi surfaces to the parent
|
||||
// as a failed tool call. `failed` means the session ran but ended in error; that gets a normal
|
||||
// text result instead, so the parent model sees the failure and can decide how to proceed.
|
||||
if (fatalFailure) {
|
||||
emitter?.sessionFailure('CHILD_TASK_FAILED', durationMs);
|
||||
await emitter?.flush();
|
||||
throw new Error(CHILD_FAILURE_TEXT);
|
||||
}
|
||||
if (failed) {
|
||||
emitter?.sessionFailure('CHILD_TASK_FAILED', durationMs);
|
||||
await emitter?.flush();
|
||||
return textResult(CHILD_FAILURE_TEXT);
|
||||
}
|
||||
|
||||
emitter?.sessionComplete(durationMs, turns, operations);
|
||||
await emitter?.flush();
|
||||
return textResult(resultText || '[Sub-agent produced no output]');
|
||||
},
|
||||
});
|
||||
|
||||
return taskTool;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
// Copyright (C) 2026 Keygraph, Inc.
|
||||
//
|
||||
// This program is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU Affero General Public License version 3
|
||||
// as published by the Free Software Foundation.
|
||||
|
||||
/**
|
||||
* Per-session trace emitter. Owns the PI `toolCallId` correlation and the ordering
|
||||
* of one agent or subagent's trace lines, then writes them through the stateless
|
||||
* `WorkflowLogger` formatter. One instance per parent agent run or per delegated
|
||||
* child session, so parallel calls never cross.
|
||||
*/
|
||||
|
||||
import { captureToolInvocation, decideToolOutcome } from '../../audit/trace.js';
|
||||
import { type ChildTaskFailureCode, type TraceActor, WorkflowLogger } from '../../audit/workflow-logger.js';
|
||||
|
||||
interface PendingCall {
|
||||
readonly tool: string;
|
||||
readonly startedAt: number;
|
||||
readonly count?: (() => number | undefined) | undefined;
|
||||
}
|
||||
|
||||
export class TraceEmitter {
|
||||
private queue: Promise<void> = Promise.resolve();
|
||||
private readonly pending = new Map<string, PendingCall>();
|
||||
|
||||
constructor(
|
||||
private readonly logPath: string,
|
||||
private readonly actor: TraceActor,
|
||||
private readonly now: () => number = Date.now,
|
||||
) {}
|
||||
|
||||
/**
|
||||
* Snapshot and log a tool call's complete arguments. `count`, when supplied, is an
|
||||
* accessor for that specific collector's existing submitted-array count outcome.
|
||||
*/
|
||||
toolStart(toolCallId: string, toolName: string, args: unknown, count?: () => number | undefined): void {
|
||||
const invocation = captureToolInvocation(toolName, args);
|
||||
this.pending.set(toolCallId, { tool: toolName, startedAt: this.now(), count });
|
||||
if (invocation !== undefined) this.enqueue(() => WorkflowLogger.logToolCall(this.logPath, this.actor, invocation));
|
||||
}
|
||||
|
||||
toolEnd(toolCallId: string, isError: boolean): void {
|
||||
const call = this.pending.get(toolCallId);
|
||||
if (call === undefined) return;
|
||||
this.pending.delete(toolCallId);
|
||||
const outcome = decideToolOutcome(call.tool, isError, this.now() - call.startedAt, call.count?.());
|
||||
if (outcome !== undefined) this.enqueue(() => WorkflowLogger.logToolOutcome(this.logPath, this.actor, outcome));
|
||||
}
|
||||
|
||||
/** Queue and await delegation on the parent emitter before a child session can start. */
|
||||
delegationStart(child: string): Promise<void> {
|
||||
const actor = this.actor;
|
||||
if (actor.kind !== 'agent') return Promise.resolve();
|
||||
return this.enqueue(() => WorkflowLogger.logDelegationStart(this.logPath, actor.agent, child));
|
||||
}
|
||||
|
||||
sessionComplete(durationMs: number, turns: number, operations: number): void {
|
||||
this.enqueue(() => WorkflowLogger.logSessionComplete(this.logPath, this.actor, durationMs, turns, operations));
|
||||
}
|
||||
|
||||
sessionFailure(code: ChildTaskFailureCode, durationMs: number): void {
|
||||
this.enqueue(() => WorkflowLogger.logSessionFailure(this.logPath, this.actor, code, durationMs));
|
||||
}
|
||||
|
||||
// Chained regardless of outcome (`then(operation, operation)`) so one write's rejection cannot
|
||||
// stall the ones queued after it, and the trailing catch swallows the failure entirely: a trace
|
||||
// line is diagnostic only, so losing one must never surface as, or block, the agent's own result.
|
||||
private enqueue(operation: () => Promise<void>): Promise<void> {
|
||||
this.queue = this.queue.then(operation, operation).catch(() => undefined);
|
||||
return this.queue;
|
||||
}
|
||||
|
||||
/** Await all queued writes so a caller can order a terminal line after them. */
|
||||
async flush(): Promise<void> {
|
||||
await this.queue;
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user