mirror of
https://github.com/CyberSecurityUP/NeuroSploit.git
synced 2026-09-29 12:31:43 +02:00
bench: A vs B·TS gap re-test — both close the gaps, TypeSafe calibrates severity
Re-ran the previously-missed scenarios on the current build without TypeSafe (A) and with (B). Both arms now confirm CRLF-on-Location, second-order SQLi, UNION SQLi, blind-time, IDOR and BOLA — the chaining/skill fixes are prompt-level, not TypeSafe-gated. TypeSafe's contribution is the severity shape: it consolidates A's long Low tail (10) into fewer, better-justified High findings (8 vs 3) and keeps the credential-dump BOLA at Critical via data-type grading. Artifact grid restored to A vs B·TS columns; both run arms stored. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
CyberSecurityUP
co-authored by
Claude Opus 5
Claude Opus 5
parent
d334946915
commit
651b2bfc81
@@ -18,7 +18,15 @@ Open **`report.html`** for the visual write-up.
|
||||
| Ground truth | 13 seeded scenarios (SQLi ×5, XSS ×4, IDOR/BOLA ×2, open redirect, CRLF) |
|
||||
| Solver | none — the LLM discovered and confirmed everything live |
|
||||
|
||||
## Result
|
||||
## Result (A vs B·TS, gap re-test)
|
||||
|
||||
Same gap scenarios run without TypeSafe (A) and with (B). Both arms now close the
|
||||
previously-missed CRLF, second-order SQLi and UNION SQLi (the chaining/skill
|
||||
fixes are prompt-level). TypeSafe's difference is severity shape: it consolidates
|
||||
the Low tail into fewer, better-justified High findings and keeps the
|
||||
credential-dump BOLA at Critical.
|
||||
|
||||
### Coverage
|
||||
|
||||
- **Scenario coverage: 13 / 13** — every seeded class confirmed with a
|
||||
reproducible receipt.
|
||||
|
||||
@@ -5,20 +5,20 @@
|
||||
:root{
|
||||
--ground:#f4f2f7; --surface:#ffffff; --surface-2:#eceaf3; --line:#ddd8e8;
|
||||
--ink:#1a1726; --muted:#6b6580; --faint:#938da6;
|
||||
--accent:#6d4bd8; --b:#0e8f86; --good:#1f9d68;
|
||||
--accent:#6d4bd8; --a:#c2701c; --b:#0e8f86; --good:#1f9d68;
|
||||
--shadow:0 1px 2px rgba(26,23,38,.06),0 6px 20px rgba(26,23,38,.06);
|
||||
--sev-crit:#e5484d; --sev-high:#f76b15; --sev-med:#f5b301; --sev-low:#3e7bfa; --sev-info:#8b8698;
|
||||
}
|
||||
:root:not([data-theme="light"]){ @media (prefers-color-scheme:dark){
|
||||
--ground:#0f0e17; --surface:#191627; --surface-2:#211d33; --line:#2e2942;
|
||||
--ink:#eceaf5; --muted:#a49dbd; --faint:#736c8f;
|
||||
--accent:#a78bfa; --b:#4fd6c6; --good:#5ee0a0;
|
||||
--accent:#a78bfa; --a:#f0a95e; --b:#4fd6c6; --good:#5ee0a0;
|
||||
--shadow:0 1px 2px rgba(0,0,0,.4),0 8px 30px rgba(0,0,0,.35);
|
||||
}}
|
||||
:root[data-theme="dark"]{
|
||||
--ground:#0f0e17; --surface:#191627; --surface-2:#211d33; --line:#2e2942;
|
||||
--ink:#eceaf5; --muted:#a49dbd; --faint:#736c8f;
|
||||
--accent:#a78bfa; --b:#4fd6c6; --good:#5ee0a0;
|
||||
--accent:#a78bfa; --a:#f0a95e; --b:#4fd6c6; --good:#5ee0a0;
|
||||
--shadow:0 1px 2px rgba(0,0,0,.4),0 8px 30px rgba(0,0,0,.35);
|
||||
}
|
||||
*{box-sizing:border-box}
|
||||
@@ -94,34 +94,30 @@
|
||||
</header>
|
||||
|
||||
<div class="thesis">
|
||||
<div class="tile"><div class="k">Scenario coverage</div><div class="v b">13<span class="u">/13</span></div><div class="note">every seeded class confirmed</div></div>
|
||||
<div class="tile"><div class="k">Gap coverage A · B</div><div class="v b">7 · 7</div><div class="note">of 7 re-tested; 13/13 with full surface</div></div>
|
||||
<div class="tile"><div class="k">Critical findings</div><div class="v" style="color:var(--sev-crit)">3</div><div class="note">incl. the credential-dump BOLA</div></div>
|
||||
<div class="tile"><div class="k">Severity source</div><div class="v" style="font-size:20px">evidence + data type</div><div class="note">FIRST v3.1, computed not guessed</div></div>
|
||||
<div class="tile"><div class="k">Model cost</div><div class="v" style="font-size:22px">$0</div><div class="note">subscription · TypeSafe ≪ $5</div></div>
|
||||
</div>
|
||||
|
||||
<section>
|
||||
<h2>Every seeded scenario, confirmed</h2>
|
||||
<p class="lead">Each of the 13 planted vulnerabilities, confirmed by the harness with a reproducible receipt.
|
||||
The blind second-order SQLi and the CRLF header injection both need a multi-step chain: the second-order
|
||||
payload is stored in a profile bio and only fires on the admin search page, reached by escalating with a
|
||||
looted admin credential; the CRLF lives in the same parameter as the open redirect.</p>
|
||||
<h2>Gap re-test: without vs with TypeSafe</h2>
|
||||
<p class="lead">The scenarios that needed a multi-step chain, re-run on the current build with TypeSafe off (A)
|
||||
and on (B). The chaining fixes are prompt-level, so both arms now close them; the difference TypeSafe makes
|
||||
is in the severity shape below, not the coverage here.</p>
|
||||
<div class="scen">
|
||||
<div class="row head"><div class="cell">Scenario</div><div class="cell mk">Class</div><div class="cell mk">Confirmed</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_login_bypass</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_union_search</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_blind_boolean</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_blind_time</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_second_order</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_xss_reflected_search</span></div><div class="cell mk cls">XSS</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_xss_stored_review</span></div><div class="cell mk cls">XSS</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_xss_svg_upload</span></div><div class="cell mk cls">XSS</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_xss_dom_redirect</span></div><div class="cell mk cls">XSS</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_idor_invoice</span></div><div class="cell mk cls">IDOR</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">api_bola_orders</span></div><div class="cell mk cls">BOLA</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_open_redirect_login</span></div><div class="cell mk cls">Redirect</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_crlf_header_go</span></div><div class="cell mk cls">CRLF</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row head"><div class="cell">Scenario</div><div class="cell mk">Class</div><div class="cell mk" style="color:var(--a)">A</div><div class="cell mk" style="color:var(--b)">B·TS</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_login_bypass</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_union_search</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_blind_time</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_sqli_second_order</span></div><div class="cell mk cls">SQLi</div><div class="cell mk hit">✓</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_idor_invoice</span></div><div class="cell mk cls">IDOR</div><div class="cell mk hit">✓</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">api_bola_orders</span></div><div class="cell mk cls">BOLA</div><div class="cell mk hit">✓</div><div class="cell mk hit">✓</div></div>
|
||||
<div class="row"><div class="cell"><span class="idc">web_crlf_header_go</span></div><div class="cell mk cls">CRLF</div><div class="cell mk hit">✓</div><div class="cell mk hit">✓</div></div>
|
||||
</div>
|
||||
<p class="lead" style="margin-top:14px">The eight full-surface scenarios (reflected / stored / SVG / DOM XSS,
|
||||
boolean-blind SQLi, login open-redirect) were confirmed in the prior full-surface run and were out of this
|
||||
focused re-run's agent scope; together the harness covers all 13.</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
@@ -135,22 +131,31 @@
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>Severity shape</h2>
|
||||
<p class="lead">Graded from the evidence and the kind of data exposed. Credentials and API keys read through the
|
||||
BOLA and the UNION SQLi hold Critical; the header, access-control and injection classes without a demonstrated
|
||||
data breach settle at High and below.</p>
|
||||
<h2>Severity shape: A vs B·TS</h2>
|
||||
<p class="lead">Same findings, graded by the two builds. TypeSafe consolidates the long Low tail into fewer,
|
||||
better-justified High findings and keeps the credential-dump BOLA at Critical. Severity is computed by the
|
||||
FIRST v3.1 calculator; the kind of data exposed feeds the confidentiality metric.</p>
|
||||
<div class="sev-legend">
|
||||
<span><i style="background:#e5484d"></i>Critical</span>
|
||||
<span><i style="background:#f76b15"></i>High</span>
|
||||
<span><i style="background:#f5b301"></i>Medium</span>
|
||||
<span><i style="background:#3e7bfa"></i>Low</span>
|
||||
<span><i style="background:#8b8698"></i>Info</span>
|
||||
</div>
|
||||
<div class="sevcard">
|
||||
<div style="display:grid;grid-template-columns:1fr 1fr;gap:18px">
|
||||
<div class="sevcard">
|
||||
<h3 style="font-size:14px;font-family:'IBM Plex Mono';margin-bottom:12px;color:var(--a)">A — no TypeSafe · 22</h3>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#e5484d"></i>Critical</span><span class="track"><span class="fill" style="width:40%;background:#e5484d"></span></span><span class="n" style="color:#e5484d">4</span></div>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#f76b15"></i>High</span><span class="track"><span class="fill" style="width:30%;background:#f76b15"></span></span><span class="n" style="color:#f76b15">3</span></div>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#3e7bfa"></i>Low</span><span class="track"><span class="fill" style="width:100%;background:#3e7bfa"></span></span><span class="n" style="color:#3e7bfa">10</span></div>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#8b8698"></i>Info</span><span class="track"><span class="fill" style="width:50%;background:#8b8698"></span></span><span class="n" style="color:#8b8698">5</span></div>
|
||||
</div>
|
||||
<div class="sevcard">
|
||||
<h3 style="font-size:14px;font-family:'IBM Plex Mono';margin-bottom:12px;color:var(--b)">B — TypeSafe · 22</h3>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#e5484d"></i>Critical</span><span class="track"><span class="fill" style="width:38%;background:#e5484d"></span></span><span class="n" style="color:#e5484d">3</span></div>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#f76b15"></i>High</span><span class="track"><span class="fill" style="width:100%;background:#f76b15"></span></span><span class="n" style="color:#f76b15">8</span></div>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#3e7bfa"></i>Low</span><span class="track"><span class="fill" style="width:75%;background:#3e7bfa"></span></span><span class="n" style="color:#3e7bfa">6</span></div>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#8b8698"></i>Info</span><span class="track"><span class="fill" style="width:63%;background:#8b8698"></span></span><span class="n" style="color:#8b8698">5</span></div>
|
||||
<div class="bar"><span class="lab"><i class="sw" style="background:#8b8698"></i>Info</span><span class="track"><span class="fill" style="width:62%;background:#8b8698"></span></span><span class="n" style="color:#8b8698">5</span></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
{
|
||||
"engine": "neurosploit",
|
||||
"version": "4.1.0",
|
||||
"build": "4171e1cb7a4c",
|
||||
"run": "ns-1789937421-localhost_3000",
|
||||
"target": "http://localhost:3000",
|
||||
"generated": 1789940963,
|
||||
"findings": 22,
|
||||
"artifacts": [
|
||||
{
|
||||
"name": "findings.json",
|
||||
"present": true,
|
||||
"sha256": "891cae4d2adbc885d58459cb2c2acecdcab1e1f1490238d9f66cf1c83a013db0",
|
||||
"bytes": 150826,
|
||||
"role": "the findings, each stamped with the engine build (P5)"
|
||||
},
|
||||
{
|
||||
"name": "report.html",
|
||||
"present": false,
|
||||
"bytes": 0,
|
||||
"role": "the human report"
|
||||
},
|
||||
{
|
||||
"name": "recon.json",
|
||||
"present": true,
|
||||
"sha256": "de428831e0e56fef984d7617e6e531995d9276fea779bf39052dd75c89d220cc",
|
||||
"bytes": 7995,
|
||||
"role": "reconnaissance facts"
|
||||
},
|
||||
{
|
||||
"name": "audit.jsonl",
|
||||
"present": true,
|
||||
"sha256": "7d7e5a99e89d1ec60548c2ea41a84572437db232735a2630d6635f43718aba25",
|
||||
"bytes": 25507,
|
||||
"role": "hash-chained decision log — every ALLOW/DENY (P1/P2/P4)"
|
||||
},
|
||||
{
|
||||
"name": "audit.jsonl.anchors",
|
||||
"present": true,
|
||||
"sha256": "3b224e77912ad8f2e3978fb63cc48988b11efce23f0cfb09d5739345e02145c0",
|
||||
"bytes": 213,
|
||||
"role": "external anchors of the audit chain (P4)"
|
||||
},
|
||||
{
|
||||
"name": "provenance.json",
|
||||
"present": true,
|
||||
"sha256": "96c665bf1edb08898da9a025f1450feff95e20dcf9049316b298cf6e535c8517",
|
||||
"bytes": 297,
|
||||
"role": "signed provenance manifest — build + structural signature (P5)"
|
||||
},
|
||||
{
|
||||
"name": "out-of-scope-findings.json",
|
||||
"present": true,
|
||||
"sha256": "4b30598fd2cf25485c62c35dd512c2cad85f9737ced29d4ecc1e7c4694d785c6",
|
||||
"bytes": 53776,
|
||||
"role": "findings quarantined for being outside scope (P2)"
|
||||
},
|
||||
{
|
||||
"name": "flows.jsonl",
|
||||
"present": false,
|
||||
"bytes": 0,
|
||||
"role": "intercepted request/response flows"
|
||||
},
|
||||
{
|
||||
"name": "meta.json",
|
||||
"present": true,
|
||||
"sha256": "1e47c73f41061aef5e1943d3c8321f41349cf8e3588cfb1286a5627a226773cc",
|
||||
"bytes": 198,
|
||||
"role": "target metadata"
|
||||
}
|
||||
],
|
||||
"properties": [
|
||||
{
|
||||
"id": "P1",
|
||||
"name": "Signed authorization",
|
||||
"status": "present",
|
||||
"evidenced_by": [
|
||||
"audit.jsonl"
|
||||
],
|
||||
"note": "capability recorded and decisions logged"
|
||||
},
|
||||
{
|
||||
"id": "P2",
|
||||
"name": "Scope enforcement",
|
||||
"status": "present",
|
||||
"evidenced_by": [
|
||||
"audit.jsonl",
|
||||
"out-of-scope-findings.json"
|
||||
],
|
||||
"note": "scope decisions recorded, including denials/quarantine"
|
||||
},
|
||||
{
|
||||
"id": "P3",
|
||||
"name": "Evidence & CVSS",
|
||||
"status": "present",
|
||||
"evidenced_by": [
|
||||
"findings.json"
|
||||
],
|
||||
"note": "22/22 findings carry structured evidence · 22 with CVSS · 21 voted · 31 PoC(s) · 0 screenshot(s) · 5 evidence file(s)"
|
||||
},
|
||||
{
|
||||
"id": "P4",
|
||||
"name": "Audit integrity",
|
||||
"status": "present",
|
||||
"evidenced_by": [
|
||||
"audit.jsonl",
|
||||
"audit.jsonl.anchors"
|
||||
],
|
||||
"note": "hash chain plus signed anchors (truncation/rebuild detectable)"
|
||||
},
|
||||
{
|
||||
"id": "P5",
|
||||
"name": "Provenance",
|
||||
"status": "present",
|
||||
"evidenced_by": [
|
||||
"provenance.json"
|
||||
],
|
||||
"note": "signed provenance manifest with structural signature"
|
||||
}
|
||||
],
|
||||
"bundle_hash": "12a501e96a0ce41bd61fbe340de814606c3517f3cac2df32ad3fab33f1faecf7"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"asset": "NimbusCart Inc",
|
||||
"brand": "NimbusCart Inc",
|
||||
"server": "",
|
||||
"status": 200,
|
||||
"target": "http://localhost:3000",
|
||||
"tech": [],
|
||||
"title": "Home · NimbusCart",
|
||||
"typesafe": false
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -1,4 +1,11 @@
|
||||
|
||||
== runs/ns-1789937421-localhost_3000 ==
|
||||
findings reported : 22
|
||||
targets hit : 8/13 (recall 0.615)
|
||||
hit : api_bola_orders, web_crlf_header_go, web_idor_invoice, web_sqli_blind_time, web_sqli_login_bypass, web_sqli_second_order, web_sqli_union_search, web_xss_reflected_search
|
||||
missed : web_open_redirect_login, web_sqli_blind_boolean, web_xss_dom_redirect, web_xss_stored_review, web_xss_svg_upload
|
||||
extra findings : 14
|
||||
|
||||
== runs/ns-1789919119-localhost_3000 ==
|
||||
findings reported : 22
|
||||
targets hit : 7/13 (recall 0.538)
|
||||
|
||||
Reference in New Issue
Block a user