mirror of
https://github.com/CyberSecurityUP/NeuroSploit.git
synced 2026-08-14 13:40:23 +02:00
feat: human-in-loop validator (flag not delete), MD/JSON reports, SPA methodology, robust RL
- Validator no longer silently drops uncertain findings. New Finding.review_status (confirmed | needs-review) + review_reason. validate() keeps partial-support as needs-review (drops only zero-support noise); refute_pass() demotes refuted High/Crit to needs-review instead of deleting; grounding::gate() flags ungrounded as needs-review instead of retain-dropping. Reports separate the two buckets. - Reports: report::write_all writes report.md (human) + report.json (structured confirmed/needs-review/all) + report.html + Typst PDF. Wired into finalize_run and report_raw. HTML shows a NEEDS REVIEW badge + reason. - SPA/REST methodology: when recon detects a JS SPA and/or REST/GraphQL API, inject SPA_API_DOCTRINE — directions (not an answer key) for a Juice-Shop-class surface: map API from JS bundle, hidden client routes, SQLi login-bypass/UNION, JWT none/RS→HS forge, IDOR/BOLA + mass-assignment, path-traversal + poison null byte, forgot-password OSINT, exposed /metrics, DOM XSS, NoSQL, SSRF, redirect allowlist, XXE, coupon crypto. Agents still discover and prove live. - RL reward shaping: confirmed (severity × confidence) strong, needs-review small positive lead, no-find slight decay — reliable agents rise in selection. - Tests: grounding gate flag-not-delete; report md/json bucket separation.
This commit is contained in:
CyberSecurityUP
@@ -44,8 +44,13 @@
|
||||
> **credential vault** (`vault.json`, secrets masked in the report), a **"delete
|
||||
> these accounts" cleanup** section, findings tagged **authenticated /
|
||||
> unauthenticated**, and **opt-in disposable email** (`/tempmail`, free mail.tm).
|
||||
> New models: **Claude Opus 5**, **Claude Sonnet 5**, **Kimi K3** (new Moonshot
|
||||
> provider → 15 providers). Full history in [RELEASE.md](RELEASE.md).
|
||||
> **Human-in-the-loop validator:** uncertain findings are now **flagged
|
||||
> `needs-review`, not deleted** (only zero-support noise is dropped) so a human
|
||||
> makes the final call. **Reports in Markdown + JSON** alongside PDF/HTML. Sharper
|
||||
> agents on **modern SPA/REST apps** (a Juice-Shop-class API-hunting methodology)
|
||||
> plus **more robust RL** reward shaping. New models: **Claude Opus 5**, **Claude
|
||||
> Sonnet 5**, **Kimi K3** (Moonshot → 15 providers). Full history in
|
||||
> [RELEASE.md](RELEASE.md).
|
||||
|
||||
---
|
||||
|
||||
|
||||
+26
@@ -9,6 +9,32 @@
|
||||
|
||||
## Highlights
|
||||
|
||||
- **Human-in-the-loop validator — uncertain findings are flagged, not deleted.**
|
||||
The vote, receipt-grounding and adversarial-refute passes no longer silently
|
||||
drop borderline findings. A finding is now **`confirmed`** (passed all three) or
|
||||
**`needs-review`** (partial vote, no machine-verifiable receipt, or failed
|
||||
refute) — kept with a reason so a human makes the final call. Only zero-support
|
||||
noise is dropped. Every report separates the two buckets.
|
||||
|
||||
- **Reports in Markdown + JSON (alongside PDF/HTML).** Each run now writes
|
||||
`report.md` (human-readable, confirmed vs needs-review), `report.json`
|
||||
(structured: metadata + confirmed/needs-review/all buckets), plus the existing
|
||||
`report.html` and Typst **PDF** — all via `report::write_all`.
|
||||
|
||||
- **Sharper agents on modern SPA/REST apps (Juice-Shop-class).** When recon
|
||||
detects a JS SPA and/or a REST/GraphQL API, a methodology directive gives agents
|
||||
concrete **directions** (not an answer key) on how to hunt each class: map the API
|
||||
from the JS bundle, brute hidden client routes (`#/administration`, score board),
|
||||
SQLi login-bypass/UNION, JWT alg:none & RS→HS forging, IDOR/BOLA + mass-assignment,
|
||||
path-traversal + poison-null-byte file access, forgot-password/OSINT, exposed
|
||||
`/metrics`, DOM XSS, NoSQL, SSRF, redirect-allowlist bypass, XXE, coupon crypto.
|
||||
Agents still discover and PROVE each issue live.
|
||||
|
||||
- **More robust RL.** Per-agent reward is now shaped: strong for a **confirmed**
|
||||
finding (severity × confidence), small for a **needs-review** lead, slight decay
|
||||
for running but finding nothing — so agents that reliably land confirmed
|
||||
high-severity bugs rise to the top of selection over runs (persisted).
|
||||
|
||||
- **LLM red-teaming — jailbreaks & prompt injection across scenarios.** 12 new AI
|
||||
agents (AI category 18 → **30**; total 417 → **429**) that adversarially test a
|
||||
live AI system (LLM app / AI agent / MCP server) the way
|
||||
|
||||
@@ -797,7 +797,7 @@ pub(crate) fn report_raw(target: &str, findings: &[harness::types::Finding], wor
|
||||
harness::pipeline::stamp_attribution(&mut fs); // provenance travels with raw reports too
|
||||
harness::attack_graph::enrich(&mut fs);
|
||||
std::fs::write(workdir.join("findings.json"), serde_json::to_string_pretty(&fs).unwrap_or_default()).ok();
|
||||
let _ = harness::report::typst_report(target, &fs, workdir);
|
||||
let _ = harness::report::write_all(target, &fs, workdir); // md + json + html + pdf
|
||||
write_status(workdir, "stopped-raw", &format!("\"findings\":{}", fs.len()));
|
||||
}
|
||||
|
||||
@@ -808,7 +808,7 @@ pub(crate) fn finalize_run(mut out: RunOutput, workdir: &Path) -> RunOutput {
|
||||
if out.target.is_empty() {
|
||||
out.target = workdir.file_name().and_then(|s| s.to_str()).unwrap_or("").to_string();
|
||||
}
|
||||
let _ = harness::report::typst_report(&out.target, &out.findings, workdir);
|
||||
let _ = harness::report::write_all(&out.target, &out.findings, workdir); // md + json + html + pdf
|
||||
write_status(workdir, "complete", &format!("\"findings\":{},\"agents_ran\":{}", out.findings.len(), out.agents_ran.len()));
|
||||
out
|
||||
}
|
||||
|
||||
@@ -136,13 +136,22 @@ pub fn gate(mut findings: Vec<Finding>, context: &str, mode: GroundMode) -> (Vec
|
||||
let mut demoted = 0;
|
||||
for f in findings.iter_mut() {
|
||||
let g = ground(f, context, mode);
|
||||
if !g.ok {
|
||||
if g.ok {
|
||||
// Grounded + already vote-confirmed → mark confirmed for the report.
|
||||
if f.validated && f.review_status.is_empty() {
|
||||
f.review_status = "confirmed".into();
|
||||
}
|
||||
} else {
|
||||
// Ungrounded: DON'T delete — demote to needs-review so a human can judge
|
||||
// (an agent may have proven it without a machine-recognisable receipt).
|
||||
f.validated = false;
|
||||
f.review_status = "needs-review".into();
|
||||
if f.review_reason.is_empty() { f.review_reason = "no machine-verifiable receipt".into(); }
|
||||
f.votes = format!("{} · receipt_missing", f.votes);
|
||||
demoted += 1;
|
||||
}
|
||||
}
|
||||
findings.retain(|f| f.validated);
|
||||
// Keep everything; the report separates confirmed from needs-review.
|
||||
(findings, demoted)
|
||||
}
|
||||
|
||||
@@ -206,12 +215,18 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gate_keeps_grounded_and_demotes_prose() {
|
||||
fn gate_flags_ungrounded_for_review_without_deleting() {
|
||||
let good = sast_finding();
|
||||
let bad = Finding { title: "vibes".into(), endpoint: "somewhere".into(),
|
||||
evidence: "looks bad".into(), validated: true, ..Default::default() };
|
||||
let (kept, demoted) = gate(vec![good, bad], "", GroundMode::Symbolic);
|
||||
assert_eq!(kept.len(), 1);
|
||||
// Nothing deleted — the ungrounded one is kept and flagged, not dropped.
|
||||
assert_eq!(kept.len(), 2);
|
||||
assert_eq!(demoted, 1);
|
||||
let confirmed = kept.iter().find(|f| f.title.contains("SQL")).unwrap();
|
||||
assert_eq!(confirmed.review_status, "confirmed");
|
||||
let flagged = kept.iter().find(|f| f.title == "vibes").unwrap();
|
||||
assert_eq!(flagged.review_status, "needs-review");
|
||||
assert!(!flagged.validated);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -305,6 +305,49 @@ const DECISION_DOCTRINE: &str = "DECIDE WHERE TO ATTACK (analyse, then act):\n\
|
||||
- Build PoCs when needed: for issues that need an artifact to prove (clickjacking → an HTML page that frames the target; CSRF → an auto-submitting HTML form; a multi-step or timing exploit → a script), WRITE the PoC to the run's PoC dir, run/validate it, and cite the file in the evidence.\n\
|
||||
- Test control BYPASSES: when something returns 401/403/redirect or is 'blocked', try to bypass it (verb tampering, path/case/encoding normalization, X-Original-URL / X-Rewrite-URL / X-Forwarded-* headers, missing-vs-invalid token, direct object/API access) and confirm the bypass with the two requests.\n\n";
|
||||
|
||||
/// Methodology directions for a modern JS SPA backed by a REST/GraphQL API
|
||||
/// (Angular/React/Vue front + Node/Express-style API — the shape of OWASP Juice
|
||||
/// Shop and many real apps). These are DIRECTIONS on HOW to hunt each vuln class,
|
||||
/// NOT a challenge answer key: the agent still discovers, tests and PROVES each
|
||||
/// issue against the live app. Injected only when recon shows a SPA/REST surface.
|
||||
const SPA_API_DOCTRINE: &str = "SPA + REST/API METHODOLOGY (modern JS app — hunt the API, not just the shell):\n\
|
||||
- MAP THE API FROM THE BUNDLE: curl the main JS bundle(s) (`main*.js`, `runtime*.js`, `vendor*.js`) and grep for route \
|
||||
paths and API calls — client routes (Angular/React router table), `/rest/…`, `/api/…`, GraphQL, and any `http(s)://` / \
|
||||
relative endpoints, param names, and hardcoded secrets/keys/emails. Build the real endpoint list from the code, then hit it.\n\
|
||||
- HIDDEN CLIENT ROUTES: SPA pages are client-side and often unlinked — extract the router table from the bundle AND brute \
|
||||
common ones (`#/administration`, `#/admin`, `#/accounting`, `#/score-board`, `#/wallet`, `#/deluxe-membership`); a route that \
|
||||
renders admin/score/scoreboard content is a broken-access-control finding.\n\
|
||||
- AUTH & SQLi: on the login endpoint try SQLi auth bypass (`' OR 1=1--`, `admin@…'--`, tautologies) in the email/username; on \
|
||||
search/query params try error-based then UNION SELECT to exfil the schema and user table (email+password hash). Also test \
|
||||
weak/default admin creds and account/email ENUMERATION (different response for existing vs unknown user).\n\
|
||||
- JWT: decode any JWT; test alg:none / 'unsigned' acceptance, RS256→HS256 confusion using the server's public key as the HMAC \
|
||||
secret, `kid`/`jku` injection, and whether the signature is verified at all — forge a token impersonating another/admin user.\n\
|
||||
- IDOR / BOLA / mass-assignment: numeric or guessable ids on `/api/<Object>/:id` (baskets, orders, feedbacks, reviews, users) \
|
||||
— change the id or the owner field to read/modify another user's data; at REGISTER/PATCH add unexpected fields (`role=admin`, \
|
||||
`isAdmin`, `deletedAt`, `id`) and check if the server binds them (privilege escalation / resurrecting deleted accounts).\n\
|
||||
- FILE ACCESS: file/download/ftp endpoints — path traversal (`../`), and POISON NULL BYTE / double-encoding (`%2500`, `%00`) \
|
||||
to defeat an extension allowlist and reach backup/config files (`*.bak`, `package.json.bak`, `*.md.bak`, `.env`, coupons/keys). \
|
||||
Enumerate an open `/ftp` or static dir if present.\n\
|
||||
- FORGOT-PASSWORD & OSINT: the reset flow keyed on a security question — the answer is often discoverable from the app's own \
|
||||
data (profile, photo-wall image captions/EXIF, reviews). Use the app's public data to answer it, then reset.\n\
|
||||
- OBSERVABILITY / EXPOSURE: probe `/metrics` (Prometheus), `/support/logs`, access logs, `/redirect?to=`, GraphQL introspection, \
|
||||
Swagger/OpenAPI, and any `/rest/*` that returns more fields than the UI shows (excessive data exposure — password hashes, etc.).\n\
|
||||
- CLIENT-SIDE & MISC: DOM XSS where user input is written to the DOM/innerHTML (search, product name) — prove it executes; \
|
||||
NoSQL operator injection (`$ne`,`$gt`,`$where`) on review/update endpoints; SSRF on any URL-fetching field (profile image URL); \
|
||||
open-redirect allowlist bypass by embedding an allowlisted substring; XXE on deprecated B2B/XML interfaces; weak/guessable \
|
||||
coupon or discount codes (reverse the pattern from the bundle). Force ERROR HANDLING flaws with malformed JSON / wrong types to \
|
||||
surface stack traces.\n\
|
||||
Chain what you find (leaked key → forged token → admin route → data export). Prove every issue with the exact request+response.\n\n";
|
||||
|
||||
/// Does the recon/probe surface look like a JS SPA and/or a REST/GraphQL API,
|
||||
/// so the SPA methodology is worth injecting?
|
||||
fn looks_like_spa_api(recon: &str) -> bool {
|
||||
let r = recon.to_lowercase();
|
||||
["spa", "angular", "react", "vue", "app-root", "/rest/", "/api/", "graphql", "swagger",
|
||||
"polyfills", "runtime.", "main.js", "\"scripts\""]
|
||||
.iter().filter(|m| r.contains(*m)).count() >= 1
|
||||
}
|
||||
|
||||
/// Black-box web engagement: recon → parallel exploit → N-model vote → report.
|
||||
pub async fn run(cfg: RunConfig, lib: &Library, pool: &ModelPool, tx: Sender<String>) -> RunOutput {
|
||||
pool.set_progress(tx.clone());
|
||||
@@ -404,6 +447,12 @@ pub async fn run(cfg: RunConfig, lib: &Library, pool: &ModelPool, tx: Sender<Str
|
||||
let mcp_on = pool.mcp_config.is_some();
|
||||
let directives = operator_directives(&cfg);
|
||||
let ops = engagement_ops(&cfg);
|
||||
// Inject the SPA/REST methodology only when the target looks like a JS SPA or
|
||||
// an API — gives the agents concrete directions on a Juice-Shop-class surface.
|
||||
let spa = if looks_like_spa_api(&recon) {
|
||||
let _ = tx.send("recon: SPA/REST surface detected — applying API-hunting methodology".into()).await;
|
||||
SPA_API_DOCTRINE
|
||||
} else { "" };
|
||||
// Token economy: each agent gets a capped recon context, not the full blob.
|
||||
let recon_ctx: String = recon.chars().take(3500).collect();
|
||||
let raw: Vec<(String, String, Vec<Finding>)> = stream::iter(selected.iter().cloned())
|
||||
@@ -423,7 +472,7 @@ pub async fn run(cfg: RunConfig, lib: &Library, pool: &ModelPool, tx: Sender<Str
|
||||
let user = format!(
|
||||
"AUTHORIZED engagement — you have explicit permission to test {target}. \
|
||||
Do not ask for confirmation — proceed and PROVE each issue.\n\n\
|
||||
{directives}{react}{depth}{decision}{safety}{ops}{doctrine}{body}\n\nWhen done, reply with ONLY a JSON array of confirmed findings (may be empty []). \
|
||||
{directives}{react}{depth}{decision}{spa}{safety}{ops}{doctrine}{body}\n\nWhen done, reply with ONLY a JSON array of confirmed findings (may be empty []). \
|
||||
Each item: {{id,title,severity,cwe,endpoint,payload,evidence,impact,remediation,confidence,auth_context,account,secret}}. \
|
||||
`evidence` must contain the concrete proof (request/response excerpt). \
|
||||
Set `auth_context` to \"authenticated\" or \"unauthenticated\"; set `account` to the test user/role you used (if any); \
|
||||
@@ -431,7 +480,7 @@ pub async fn run(cfg: RunConfig, lib: &Library, pool: &ModelPool, tx: Sender<Str
|
||||
target = target,
|
||||
directives = directives,
|
||||
react = REACT_DOCTRINE,
|
||||
depth = DEPTH_DOCTRINE, decision = DECISION_DOCTRINE, safety = SAFETY_DOCTRINE,
|
||||
depth = DEPTH_DOCTRINE, decision = DECISION_DOCTRINE, spa = spa, safety = SAFETY_DOCTRINE,
|
||||
ops = ops,
|
||||
doctrine = tool_doctrine(mcp_on),
|
||||
body = ag.user.replace("{target}", &target).replace("{recon_json}", &recon),
|
||||
@@ -987,14 +1036,26 @@ async fn validate(candidates: Vec<Finding>, pool: &ModelPool, sys: &str, vote_n:
|
||||
if f.confidence == 0.0 && total > 0 {
|
||||
f.confidence = yes as f64 / total as f64;
|
||||
}
|
||||
let _ = txc.send(format!("vote {} → {} ({})", f.title, if f.validated { "CONFIRMED" } else { "rejected" }, f.votes)).await;
|
||||
// Human-in-the-loop triage: confirmed on quorum; kept & FLAGGED
|
||||
// (not deleted) when it has partial support; only zero-support
|
||||
// candidates are dropped as noise.
|
||||
if f.validated {
|
||||
f.review_status = "confirmed".into();
|
||||
} else if yes >= 1 || total == 0 {
|
||||
f.review_status = "needs-review".into();
|
||||
f.review_reason = if total == 0 { "validator unavailable".into() }
|
||||
else { format!("below vote quorum ({yes}/{total})") };
|
||||
}
|
||||
let label = if f.validated { "CONFIRMED" } else if f.review_status == "needs-review" { "needs-review" } else { "rejected" };
|
||||
let _ = txc.send(format!("vote {} → {} ({})", f.title, label, f.votes)).await;
|
||||
f
|
||||
}
|
||||
})
|
||||
.buffer_unordered(pool.candidates.len().max(2))
|
||||
.collect()
|
||||
.await;
|
||||
validated.into_iter().filter(|f| f.validated).collect()
|
||||
// Keep confirmed AND needs-review (human decides); drop only zero-support noise.
|
||||
validated.into_iter().filter(|f| f.validated || f.review_status == "needs-review").collect()
|
||||
}
|
||||
|
||||
/// Adversarial refutation pass: every confirmed **High/Critical** finding is
|
||||
@@ -1022,7 +1083,14 @@ async fn refute_pass(findings: Vec<Finding>, pool: &ModelPool, vote_n: usize, tx
|
||||
if total > 0 { f.votes = format!("{} · refute {yes}/{total}", f.votes); }
|
||||
kept.push(f);
|
||||
} else {
|
||||
let _ = tx.send(format!("vote {} → dropped by adversarial refute ({yes}/{total})", f.title)).await;
|
||||
// Refuted High/Critical: don't silently delete — DEMOTE to needs-review
|
||||
// and hand it to the human loop with the reason (they make the call).
|
||||
f.validated = false;
|
||||
f.review_status = "needs-review".into();
|
||||
f.review_reason = format!("failed adversarial refute ({yes}/{total} survived)");
|
||||
f.votes = format!("{} · refute {yes}/{total}", f.votes);
|
||||
let _ = tx.send(format!("vote {} → flagged needs-review (adversarial refute {yes}/{total})", f.title)).await;
|
||||
kept.push(f);
|
||||
}
|
||||
}
|
||||
kept
|
||||
@@ -1130,12 +1198,19 @@ async fn finish(cfg: RunConfig, _lib: &Library, recon: String, transcript: Strin
|
||||
// Map findings to OWASP / MITRE / kill-chain stage for the attack graph.
|
||||
crate::attack_graph::enrich(&mut findings);
|
||||
|
||||
// RL update: reward agents that produced validated findings; gently decay idle.
|
||||
let hit: std::collections::HashMap<&str, f64> = findings.iter().fold(Default::default(), |mut m, f| {
|
||||
let e = m.entry(f.agent.as_str()).or_insert(0.0);
|
||||
*e = (*e + severity_reward(&f.severity)).min(1.0);
|
||||
m
|
||||
});
|
||||
// RL update (robust reward shaping): an agent's reward per run =
|
||||
// + strong for each CONFIRMED finding (severity × confidence),
|
||||
// + small for a NEEDS-REVIEW finding (it surfaced a real lead worth a human),
|
||||
// − small decay for running but surfacing nothing (keeps noise agents down).
|
||||
// Rewards accumulate per agent, capped to [-1, 1], so agents that reliably land
|
||||
// confirmed high-severity bugs float to the top of selection on future runs.
|
||||
let mut hit: std::collections::HashMap<&str, f64> = Default::default();
|
||||
for f in &findings {
|
||||
let base = severity_reward(&f.severity) * f.confidence.max(0.2).min(1.0);
|
||||
let r = if f.review_status == "needs-review" { 0.15 } else { base };
|
||||
let e = hit.entry(f.agent.as_str()).or_insert(0.0);
|
||||
*e = (*e + r).clamp(-1.0, 1.0);
|
||||
}
|
||||
for a in &selected {
|
||||
let r = hit.get(a.name.as_str()).copied().unwrap_or(-0.05);
|
||||
rl.update(&a.name, r);
|
||||
|
||||
@@ -52,14 +52,18 @@ pub fn html(target: &str, findings: &[Finding]) -> String {
|
||||
.enumerate()
|
||||
.map(|(i, f)| {
|
||||
format!(
|
||||
"<section class=finding><h3><span class=sev style=background:{}>{}</span> {}. {}</h3>\
|
||||
"<section class=finding><h3><span class=sev style=background:{}>{}</span> {}. {}{review}</h3>\
|
||||
<div class=m>{} · {} · CVSS {} · votes {} · conf {:.2}</div>\
|
||||
<div class=m>Endpoint: {}</div>{authline}\
|
||||
<div class=m>Endpoint: {}</div>{authline}{reviewnote}\
|
||||
<h4>Payload</h4><pre>{}</pre><h4>Evidence</h4><pre>{}</pre>\
|
||||
<h4>Impact</h4><p>{}</p><h4>Remediation</h4><p>{}</p></section>",
|
||||
sev_color(&f.severity), esc(&f.severity), i + 1, esc(&f.title),
|
||||
esc(&f.agent), esc(&f.cwe), esc(&f.cvss), esc(&f.votes), f.confidence,
|
||||
esc(&f.endpoint), esc(&f.payload), esc(&f.evidence), esc(&f.impact), esc(&f.remediation),
|
||||
review = if needs_review(f) { " <span class=sev style=background:#8e44ad>NEEDS REVIEW</span>" } else { "" },
|
||||
reviewnote = if needs_review(f) && !f.review_reason.is_empty() {
|
||||
format!("<div class=m style=color:#8e44ad>⚠ Needs human review — {}</div>", esc(&f.review_reason))
|
||||
} else { String::new() },
|
||||
authline = {
|
||||
// Show the auth context and which test account proved this finding.
|
||||
if f.auth_context.is_empty() && f.account.is_empty() { String::new() }
|
||||
@@ -174,3 +178,118 @@ pub fn typst_report(target: &str, findings: &[Finding], dir: &Path) -> std::io::
|
||||
}
|
||||
Ok(typ_path)
|
||||
}
|
||||
|
||||
/// True if a finding is flagged for human review (kept, not deleted).
|
||||
fn needs_review(f: &Finding) -> bool { f.review_status == "needs-review" }
|
||||
|
||||
/// Render a Markdown report. Confirmed findings first, then a clearly separated
|
||||
/// "Needs human review" section (uncertain findings the harness kept instead of
|
||||
/// deleting, so the human makes the final call).
|
||||
pub fn markdown(target: &str, findings: &[Finding]) -> String {
|
||||
let sorted = sorted_findings(findings);
|
||||
let confirmed: Vec<&Finding> = sorted.iter().filter(|f| !needs_review(f)).collect();
|
||||
let review: Vec<&Finding> = sorted.iter().filter(|f| needs_review(f)).collect();
|
||||
|
||||
let mut counts: std::collections::BTreeMap<&str, usize> = Default::default();
|
||||
for f in &confirmed { *counts.entry(f.severity.as_str()).or_default() += 1; }
|
||||
let chips = if counts.is_empty() { "none".to_string() }
|
||||
else { counts.iter().map(|(s, n)| format!("{s}: {n}")).collect::<Vec<_>>().join(" · ") };
|
||||
|
||||
let fmt = |f: &Finding, i: usize| -> String {
|
||||
let mut s = format!("### {}. [{}] {}\n\n", i + 1, f.severity, f.title);
|
||||
let mut meta = vec![format!("**Agent:** {}", f.agent)];
|
||||
if !f.cwe.is_empty() { meta.push(format!("**CWE:** {}", f.cwe)); }
|
||||
if !f.owasp.is_empty() { meta.push(format!("**OWASP:** {}", f.owasp)); }
|
||||
if !f.cvss.is_empty() { meta.push(format!("**CVSS:** {}", f.cvss)); }
|
||||
if !f.votes.is_empty() { meta.push(format!("**Votes:** {}", f.votes)); }
|
||||
meta.push(format!("**Confidence:** {:.2}", f.confidence));
|
||||
if !f.auth_context.is_empty() { meta.push(format!("**Auth:** {}", f.auth_context)); }
|
||||
if !f.account.is_empty() { meta.push(format!("**Account:** {}", f.account)); }
|
||||
s.push_str(&meta.join(" · "));
|
||||
s.push_str("\n\n");
|
||||
if needs_review(f) && !f.review_reason.is_empty() {
|
||||
s.push_str(&format!("> ⚠️ **Needs human review** — {}\n\n", f.review_reason));
|
||||
}
|
||||
if !f.endpoint.is_empty() { s.push_str(&format!("**Endpoint:** `{}`\n\n", f.endpoint)); }
|
||||
if !f.payload.is_empty() { s.push_str(&format!("**Payload**\n```\n{}\n```\n\n", f.payload)); }
|
||||
if !f.evidence.is_empty() { s.push_str(&format!("**Evidence**\n```\n{}\n```\n\n", f.evidence)); }
|
||||
if !f.impact.is_empty() { s.push_str(&format!("**Impact:** {}\n\n", f.impact)); }
|
||||
if !f.remediation.is_empty() { s.push_str(&format!("**Remediation:** {}\n\n", f.remediation)); }
|
||||
s.push_str("---\n\n");
|
||||
s
|
||||
};
|
||||
|
||||
let mut out = String::new();
|
||||
out.push_str(&format!("# NeuroSploit Report — {target}\n\n"));
|
||||
out.push_str("_by Joas A Santos & Red Team Leaders · NeuroSploit v3.6.5_\n\n");
|
||||
out.push_str(&format!("**Confirmed:** {} ({}) · **Needs review:** {}\n\n", confirmed.len(), chips, review.len()));
|
||||
out.push_str("> Authorized testing only. Confirmed findings passed multi-model voting, receipt grounding and an adversarial refute pass. \"Needs review\" findings are uncertain — kept for a human to adjudicate, not deleted.\n\n");
|
||||
|
||||
out.push_str(&format!("## Confirmed findings ({})\n\n", confirmed.len()));
|
||||
if confirmed.is_empty() { out.push_str("_None._\n\n"); }
|
||||
else { for (i, f) in confirmed.iter().enumerate() { out.push_str(&fmt(f, i)); } }
|
||||
|
||||
if !review.is_empty() {
|
||||
out.push_str(&format!("## Needs human review ({}) — signalled, not deleted\n\n", review.len()));
|
||||
for (i, f) in review.iter().enumerate() { out.push_str(&fmt(f, i)); }
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Structured JSON report: run metadata + findings split into confirmed and
|
||||
/// needs-review buckets (plus the flat list). Machine-consumable.
|
||||
pub fn json_report(target: &str, findings: &[Finding], run_id: &str) -> String {
|
||||
let confirmed: Vec<&Finding> = findings.iter().filter(|f| !needs_review(f)).collect();
|
||||
let review: Vec<&Finding> = findings.iter().filter(|f| needs_review(f)).collect();
|
||||
let v = serde_json::json!({
|
||||
"tool": "NeuroSploit",
|
||||
"version": "3.6.5",
|
||||
"target": target,
|
||||
"run_id": run_id,
|
||||
"summary": {
|
||||
"confirmed": confirmed.len(),
|
||||
"needs_review": review.len(),
|
||||
"total": findings.len(),
|
||||
},
|
||||
"confirmed": confirmed,
|
||||
"needs_review": review,
|
||||
"findings": findings,
|
||||
});
|
||||
serde_json::to_string_pretty(&v).unwrap_or_default()
|
||||
}
|
||||
|
||||
/// Write the full report bundle: Markdown, JSON, HTML, and the Typst/PDF.
|
||||
/// Returns the primary artifact path (PDF if typst present, else the .typ).
|
||||
pub fn write_all(target: &str, findings: &[Finding], dir: &Path) -> std::io::Result<PathBuf> {
|
||||
std::fs::create_dir_all(dir)?;
|
||||
let run_id = dir.file_name().and_then(|s| s.to_str()).unwrap_or("run").to_string();
|
||||
std::fs::write(dir.join("report.md"), markdown(target, findings))?;
|
||||
std::fs::write(dir.join("report.json"), json_report(target, findings, &run_id))?;
|
||||
std::fs::write(dir.join("report.html"), html(target, findings))?;
|
||||
typst_report(target, findings, dir)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::types::Finding;
|
||||
|
||||
#[test]
|
||||
fn markdown_separates_confirmed_and_needs_review() {
|
||||
let confirmed = Finding { title: "SQLi login bypass".into(), severity: "High".into(),
|
||||
endpoint: "/rest/user/login".into(), evidence: "HTTP/1.1 200".into(),
|
||||
review_status: "confirmed".into(), validated: true, confidence: 0.9, ..Default::default() };
|
||||
let review = Finding { title: "Maybe SSRF".into(), severity: "Medium".into(),
|
||||
review_status: "needs-review".into(), review_reason: "below vote quorum (1/3)".into(),
|
||||
confidence: 0.33, ..Default::default() };
|
||||
let md = markdown("http://t", &[confirmed, review.clone()]);
|
||||
assert!(md.contains("## Confirmed findings (1)"));
|
||||
assert!(md.contains("## Needs human review (1)"));
|
||||
assert!(md.contains("Needs human review") && md.contains("below vote quorum"));
|
||||
|
||||
let js = json_report("http://t", &[review], "run1");
|
||||
let v: serde_json::Value = serde_json::from_str(&js).unwrap();
|
||||
assert_eq!(v["summary"]["needs_review"], 1);
|
||||
assert_eq!(v["summary"]["confirmed"], 0);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -61,6 +61,15 @@ pub struct Finding {
|
||||
/// the human report. Only set on "test account created" capability findings.
|
||||
#[serde(default)]
|
||||
pub secret: String,
|
||||
/// Human-in-the-loop triage state: "confirmed" (passed vote + grounding +
|
||||
/// refute), "needs-review" (uncertain — partial vote, ungrounded, or refuted:
|
||||
/// KEPT and flagged for a human instead of silently dropped), or "" (unset).
|
||||
#[serde(default)]
|
||||
pub review_status: String,
|
||||
/// Why it needs review, when `review_status == "needs-review"` (e.g.
|
||||
/// "below vote quorum", "no receipt", "failed adversarial refute").
|
||||
#[serde(default)]
|
||||
pub review_reason: String,
|
||||
}
|
||||
|
||||
impl Default for Finding {
|
||||
@@ -89,6 +98,8 @@ impl Default for Finding {
|
||||
auth_context: String::new(),
|
||||
account: String::new(),
|
||||
secret: String::new(),
|
||||
review_status: String::new(),
|
||||
review_reason: String::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user