From e7edef49d77f4ad5a5cce903ee337705a14d790a Mon Sep 17 00:00:00 2001 From: george-keygraph Date: Thu, 13 Aug 2026 15:27:19 -0700 Subject: [PATCH] docs: correct rating semantics stale since v2.2.0 v2.2.0 (feat(worker): record severity in analysis mode, #413) made every finding carry a severity, and min_severity now applies in both modes (prompt-manager.ts applies the severity filter unconditionally; only min_confidence is gated to analysis-only runs). configuration.md still described the pre-2.2.0 model: min_severity as exploit-only, findings carrying one rating or the other but never both, and analysis runs producing no severity at all. Co-Authored-By: Claude Opus 5 --- docs/ci-cd.md | 2 +- docs/configuration.md | 6 +++--- llms-full.txt | 8 ++++---- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/ci-cd.md b/docs/ci-cd.md index 59dcfeb..fd5d96a 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -62,7 +62,7 @@ report: sarif: "true" ``` -SARIF requires an exploitative run, which is the default. An analysis-only run (`exploit: "false"`) rates findings by confidence rather than severity and produces no SARIF log. +SARIF requires an exploitative run, which is the default. Shannon does not write a SARIF log for analysis-only runs (`exploit: "false"`). Each finding becomes one SARIF result, filed under a rule per vulnerability class (`shannon/injection`, `shannon/xss`, `shannon/auth`, `shannon/authz`, `shannon/ssrf`) and tagged with its OWASP Top Ten 2025 category. Severity maps onto SARIF's three levels: `critical` and `high` become `error`, `medium` becomes `warning`, everything else becomes `note`. diff --git a/docs/configuration.md b/docs/configuration.md index b9d4de2..e458d3a 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -106,12 +106,12 @@ rules: | Key | Effect | | --- | --- | -| `min_severity` | Drops findings rated below this severity. Applies only when `exploit` is `"true"`. | +| `min_severity` | Drops findings rated below this severity. Applies in both exploitative and analysis-only runs. | | `min_confidence` | Drops findings rated below this confidence. Applies only when `exploit` is `"false"`. | | `guidance` | Free-text instruction to the report agent, such as which topics to exclude. | | `sarif` | Emits a SARIF 2.1.0 log alongside the Markdown report. Requires `exploit: "true"`. | -A finding carries one rating or the other, never both: an exploited finding is rated by severity, an analysis-only finding by confidence. Setting the threshold that does not apply to the run is ignored, and Shannon logs a warning naming the one to use instead. +Every finding carries a severity, but it does not mean the same thing in each mode: an exploitative run measures severity from what the exploit demonstrated, while an analysis-only run assesses it from the class of flaw and the impact it would have. An analysis-only finding carries a confidence rating alongside its severity, since nothing was proven. Setting `min_confidence` on an exploitative run is ignored, and Shannon logs a warning naming the threshold to use instead. ### SARIF Output @@ -125,7 +125,7 @@ report: Each finding becomes one SARIF result, filed under a rule per vulnerability class (`shannon/injection`, `shannon/xss`, `shannon/auth`, `shannon/authz`, `shannon/ssrf`) and tagged with its OWASP Top Ten 2025 category. Results are anchored to the code location the analysis phase recorded, falling back to the HTTP entry point when the finding names no file. Severity maps onto SARIF's three levels: `critical` and `high` become `error`, `medium` becomes `warning`, everything else becomes `note`. -The log is written only for exploitative runs. An analysis-only run rates findings by confidence and produces no severity, so there is nothing to populate `level` with; `sarif` is ignored when `exploit` is `"false"`. +The log is written only for exploitative runs. `sarif` is ignored when `exploit` is `"false"`. Supported rule types include `url_path`, `subdomain`, `domain`, `method`, `header`, `parameter`, and `code_path`. diff --git a/llms-full.txt b/llms-full.txt index dc18a8c..d89cc2a 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -628,12 +628,12 @@ rules: | Key | Effect | | --- | --- | -| `min_severity` | Drops findings rated below this severity. Applies only when `exploit` is `"true"`. | +| `min_severity` | Drops findings rated below this severity. Applies in both exploitative and analysis-only runs. | | `min_confidence` | Drops findings rated below this confidence. Applies only when `exploit` is `"false"`. | | `guidance` | Free-text instruction to the report agent, such as which topics to exclude. | | `sarif` | Emits a SARIF 2.1.0 log alongside the Markdown report. Requires `exploit: "true"`. | -A finding carries one rating or the other, never both: an exploited finding is rated by severity, an analysis-only finding by confidence. Setting the threshold that does not apply to the run is ignored, and Shannon logs a warning naming the one to use instead. +Every finding carries a severity, but it does not mean the same thing in each mode: an exploitative run measures severity from what the exploit demonstrated, while an analysis-only run assesses it from the class of flaw and the impact it would have. An analysis-only finding carries a confidence rating alongside its severity, since nothing was proven. Setting `min_confidence` on an exploitative run is ignored, and Shannon logs a warning naming the threshold to use instead. ### SARIF Output @@ -647,7 +647,7 @@ report: Each finding becomes one SARIF result, filed under a rule per vulnerability class (`shannon/injection`, `shannon/xss`, `shannon/auth`, `shannon/authz`, `shannon/ssrf`) and tagged with its OWASP Top Ten 2025 category. Results are anchored to the code location the analysis phase recorded, falling back to the HTTP entry point when the finding names no file. Severity maps onto SARIF's three levels: `critical` and `high` become `error`, `medium` becomes `warning`, everything else becomes `note`. -The log is written only for exploitative runs. An analysis-only run rates findings by confidence and produces no severity, so there is nothing to populate `level` with; `sarif` is ignored when `exploit` is `"false"`. +The log is written only for exploitative runs. `sarif` is ignored when `exploit` is `"false"`. Supported rule types include `url_path`, `subdomain`, `domain`, `method`, `header`, `parameter`, and `code_path`. @@ -1199,7 +1199,7 @@ report: sarif: "true" ``` -SARIF requires an exploitative run, which is the default. An analysis-only run (`exploit: "false"`) rates findings by confidence rather than severity and produces no SARIF log. +SARIF requires an exploitative run, which is the default. Shannon does not write a SARIF log for analysis-only runs (`exploit: "false"`). Each finding becomes one SARIF result, filed under a rule per vulnerability class (`shannon/injection`, `shannon/xss`, `shannon/auth`, `shannon/authz`, `shannon/ssrf`) and tagged with its OWASP Top Ten 2025 category. Severity maps onto SARIF's three levels: `critical` and `high` become `error`, `medium` becomes `warning`, everything else becomes `note`.