feat(decision): pluggable System One backend — TypeSafe (hosted) or Laya (local)

Laya (github.com/NandhaKishorM/laya) is the same System One abstraction as
TypeSafe — identical choice/score/noul primitives — but local, open-source
(Apache 2.0) and free. Added it as a swappable backend, entirely additively:
the hosted TypeSafe path is byte-for-byte unchanged (key alone → same endpoint,
model, bearer as before).

- typesafe.rs: endpoint/model/bearer are now instance fields with env overrides
  (NEUROSPLOIT_DECISION_ENDPOINT / _MODEL). Defaults are the hosted TypeSafe API.
  from_env() now also activates when a local endpoint is configured (no key).
  backend_label() names the active backend in the run banner.
- tools/laya_shim.py: a stdlib HTTP shim that loads Laya and exposes the exact
  POST /systemone contract the client already speaks. Model downloads on first
  use (HF cache); no key; evidence stays on the box.
- CLI: --decision-backend typesafe|laya. `laya` installs laya if missing, starts
  the shim, waits for readiness, and points the client at it — all optional,
  only when the operator selects it.

383 tests; the hosted TypeSafe behaviour is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
CyberSecurityUPandClaude Opus 5 committed 2026-09-20 19:27:14 -03:00
1 parent d752e252e6
commit 56b2c80ae4
6 files changed
+285 -10

No files matched your search

+13
View File
@@ -506,6 +506,19 @@ neurosploit run https://app --typesafe off # the identical pipeline, no TypeS
records `"typesafe": true|false` — a clean with/without measurement you can run records `"typesafe": true|false` — a clean with/without measurement you can run
against your own target. against your own target.
**Pluggable decision backend.** The calibrated System One layer runs against
either backend, chosen with `--decision-backend`:
- `typesafe` — the hosted API (set `TYPESAFE_API_KEY`).
- `laya` — [Laya](https://github.com/NandhaKishorM/laya), a local, open-source
System One engine (Apache 2.0) with the same primitives. Picking it downloads
the model on first use, runs it on this machine, needs no API key, and keeps
the engagement's evidence on the box — the right choice for air-gapped or OT
work. It starts a small local shim (`tools/laya_shim.py`) automatically.
Both speak the same contract, so adjudication, CVSS grading, agent pruning and
the confirmation loop behave identically whichever you pick.
### Scope-evasion resistance, evidence integrity, untrusted output ### Scope-evasion resistance, evidence integrity, untrusted output
Three hardening passes, all enforced in code: Three hardening passes, all enforced in code:
+1 -1
View File
@@ -795,7 +795,7 @@ neurosploit run https://app --typesafe on # calibrated adjudication + confir
neurosploit run https://app --typesafe off # the identical pipeline, no TypeSafe (for A/B) neurosploit run https://app --typesafe off # the identical pipeline, no TypeSafe (for A/B)
``` ```
`--typesafe auto` (default) turns it on when the key is set. It adjudicates each `--typesafe auto` (default) turns it on when the key is set. Choose the engine with `--decision-backend typesafe` (hosted) or `--decision-backend laya` (local, free, downloads the model on first use — evidence never leaves the box; see `tools/laya_shim.py`). It adjudicates each
finding with a calibrated `{confirmed/needs-review/rejected}` judgment over the finding with a calibrated `{confirmed/needs-review/rejected}` judgment over the
*evidence*, re-grades CVSS when impact isn't demonstrated, prunes irrelevant *evidence*, re-grades CVSS when impact isn't demonstrated, prunes irrelevant
agents, and runs a code-owned confirmation loop over enumerable classes. It is agents, and runs a code-owned confirmation loop over enumerable classes. It is
+82
View File
@@ -82,6 +82,12 @@ struct Cli {
/// exact same pipeline without it, so runs can be compared with/without. /// exact same pipeline without it, so runs can be compared with/without.
#[arg(long = "typesafe", global = true)] #[arg(long = "typesafe", global = true)]
typesafe: Option<String>, typesafe: Option<String>,
/// Decision backend for the calibrated System One layer:
/// typesafe (hosted API, needs TYPESAFE_API_KEY) or laya (local, free,
/// open-source — downloads the model on first use and keeps evidence on the
/// box). Default: whichever is configured. See tools/laya_shim.py.
#[arg(long = "decision-backend", global = true)]
decision_backend: Option<String>,
} }
#[derive(Subcommand)] #[derive(Subcommand)]
@@ -550,6 +556,71 @@ fn find_base() -> PathBuf {
} }
/// Where the harness caches an auto-fetched `agents_md/` (`~/.neurosploit/cache`). /// Where the harness caches an auto-fetched `agents_md/` (`~/.neurosploit/cache`).
/// Bring up the local Laya decision backend and point the client at it.
///
/// Idempotent: if the shim already answers on its port, we just set the env and
/// return. Otherwise we start `tools/laya_shim.py` (which downloads the model on
/// first run) in the background and wait for it to become ready. `pip install
/// laya` is attempted once if the import is missing. Everything here is optional
/// and only runs when the operator explicitly picks `--decision-backend laya`.
fn ensure_laya_backend() -> Result<(), String> {
let port = std::env::var("LAYA_SHIM_PORT").unwrap_or_else(|_| "8799".into());
let endpoint = format!("http://127.0.0.1:{port}/systemone");
let health = format!("http://127.0.0.1:{port}/health");
let ready = |url: &str| -> bool {
std::process::Command::new("curl")
.args(["-s", "-o", "/dev/null", "-w", "%{http_code}", "--max-time", "3", url])
.output().ok()
.map(|o| String::from_utf8_lossy(&o.stdout).trim() == "200")
.unwrap_or(false)
};
if ready(&health) {
std::env::set_var("NEUROSPLOIT_DECISION_ENDPOINT", &endpoint);
std::env::set_var("NEUROSPLOIT_DECISION_MODEL", "laya");
println!(" \x1b[2mdecision backend: laya (already running on :{port})\x1b[0m");
return Ok(());
}
// Locate the shim next to the binary/checkout.
let shim = [
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tools").join("laya_shim.py"),
std::env::current_dir().unwrap_or_default().join("neurosploit-rs/tools/laya_shim.py"),
std::env::current_dir().unwrap_or_default().join("tools/laya_shim.py"),
].into_iter().find(|p| p.exists())
.ok_or_else(|| "laya_shim.py not found (expected under tools/)".to_string())?;
let py = if std::process::Command::new("python3").arg("--version").output().is_ok() { "python3" } else { "python" };
// Best-effort install of laya if it is missing.
let has_laya = std::process::Command::new(py).args(["-c", "import laya"]).output().map(|o| o.status.success()).unwrap_or(false);
if !has_laya {
println!(" \x1b[2minstalling laya (first run only)…\x1b[0m");
let _ = std::process::Command::new(py).args(["-m", "pip", "install", "-q", "laya"]).status();
}
println!(" \x1b[2mstarting laya shim (downloads the model on first use)…\x1b[0m");
std::process::Command::new(py)
.arg(&shim)
.stdin(std::process::Stdio::null())
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::inherit())
.spawn()
.map_err(|e| format!("could not start the shim: {e}"))?;
// Wait for readiness (model download can take a while on the first run).
for _ in 0..120 {
if ready(&health) {
std::env::set_var("NEUROSPLOIT_DECISION_ENDPOINT", &endpoint);
std::env::set_var("NEUROSPLOIT_DECISION_MODEL", "laya");
println!(" \x1b[1;32m✓ laya backend ready\x1b[0m on :{port} — evidence stays local, no API key");
return Ok(());
}
std::thread::sleep(std::time::Duration::from_secs(2));
}
Err("laya shim did not become ready in time".into())
}
fn agents_cache_dir() -> Option<PathBuf> { fn agents_cache_dir() -> Option<PathBuf> {
std::env::var_os("HOME").map(PathBuf::from).map(|h| h.join(".neurosploit").join("cache")) std::env::var_os("HOME").map(PathBuf::from).map(|h| h.join(".neurosploit").join("cache"))
.or_else(|| std::env::var_os("LOCALAPPDATA").map(PathBuf::from).map(|l| l.join("NeuroSploit").join("cache"))) .or_else(|| std::env::var_os("LOCALAPPDATA").map(PathBuf::from).map(|l| l.join("NeuroSploit").join("cache")))
@@ -615,6 +686,17 @@ async fn main() -> anyhow::Result<()> {
// run type (and the REPL) honours one control. `off` disables it entirely; // run type (and the REPL) honours one control. `off` disables it entirely;
// `on`/`auto` leave it to key presence. This is what makes with/without // `on`/`auto` leave it to key presence. This is what makes with/without
// TypeSafe an A/B a single flag flips. // TypeSafe an A/B a single flag flips.
// Decision backend selection is additive: it only sets the endpoint/model
// env vars the client already reads. `typesafe` (or unset) leaves the hosted
// default untouched; `laya` points at a local shim and starts it if needed.
if let Some(be) = cli.decision_backend.as_deref().map(|s| s.trim().to_lowercase()) {
if be == "laya" {
if let Err(e) = ensure_laya_backend() {
eprintln!(" \x1b[33m⚠ laya backend: {e} — falling back to whatever else is configured\x1b[0m");
}
}
// `typesafe` needs no action: the client's defaults already point there.
}
match cli.typesafe.as_deref().map(|s| s.trim().to_lowercase()) { match cli.typesafe.as_deref().map(|s| s.trim().to_lowercase()) {
Some(ref m) if m == "off" || m == "false" || m == "0" => std::env::set_var("NEUROSPLOIT_TYPESAFE", "off"), Some(ref m) if m == "off" || m == "false" || m == "0" => std::env::set_var("NEUROSPLOIT_TYPESAFE", "off"),
Some(ref m) if m == "on" || m == "true" || m == "1" || m == "auto" => std::env::set_var("NEUROSPLOIT_TYPESAFE", "on"), Some(ref m) if m == "on" || m == "true" || m == "1" || m == "auto" => std::env::set_var("NEUROSPLOIT_TYPESAFE", "on"),
@@ -2349,7 +2349,7 @@ async fn finish(cfg: RunConfig, _lib: &Library, pool: &ModelPool, recon: String,
let _ = tx.send(format!("notify: 🧮 TypeSafe confirmation loop confirmed {confirmed_by_agent} finding(s) the LLM path left unconfirmed")).await; let _ = tx.send(format!("notify: 🧮 TypeSafe confirmation loop confirmed {confirmed_by_agent} finding(s) the LLM path left unconfirmed")).await;
} }
let _ = tx.send("notify: 🧮 TypeSafe System One adjudicating findings…".to_string()).await; let _ = tx.send(format!("notify: 🧮 System One adjudicating findings via {}…", ts.backend_label())).await;
let mut refined = 0usize; let mut refined = 0usize;
for f in findings.iter_mut() { for f in findings.iter_mut() {
let state = typesafe_state(f); let state = typesafe_state(f);
+33 -8
View File
@@ -33,8 +33,8 @@ use serde::{Deserialize, Serialize};
use std::collections::BTreeMap; use std::collections::BTreeMap;
use std::time::Duration; use std::time::Duration;
const ENDPOINT: &str = "https://api.typesafe.ai/v1/systemone"; const DEFAULT_ENDPOINT: &str = "https://api.typesafe.ai/v1/systemone";
const MODEL: &str = "jev-latest"; const DEFAULT_MODEL: &str = "jev-latest";
/// A question to evaluate against the state. /// A question to evaluate against the state.
#[derive(Debug, Clone, Serialize)] #[derive(Debug, Clone, Serialize)]
@@ -123,32 +123,57 @@ struct ApiResponse {
pub struct TypeSafe { pub struct TypeSafe {
key: String, key: String,
client: reqwest::Client, client: reqwest::Client,
/// Endpoint to POST to. Defaults to TypeSafe's hosted API; a local backend
/// (e.g. the Laya shim) is selected by setting `NEUROSPLOIT_DECISION_ENDPOINT`.
endpoint: String,
/// Model id sent in the request. `NEUROSPLOIT_DECISION_MODEL` overrides it.
model: String,
/// Whether to send `Authorization: Bearer`. A local backend needs no key.
bearer: bool,
} }
impl TypeSafe { impl TypeSafe {
/// Build from `TYPESAFE_API_KEY`. None when unset — the caller then skips /// Build from `TYPESAFE_API_KEY`. None when unset — the caller then skips
/// System One entirely rather than failing. /// System One entirely rather than failing.
pub fn from_env() -> Option<TypeSafe> { pub fn from_env() -> Option<TypeSafe> {
let key = std::env::var("TYPESAFE_API_KEY").ok().filter(|k| !k.trim().is_empty())?; let key = std::env::var("TYPESAFE_API_KEY").ok().filter(|k| !k.trim().is_empty());
// A local decision backend (the Laya shim) is configured by its endpoint
// and needs no key. The hosted TypeSafe path is unchanged: a key alone
// still works exactly as before.
let endpoint = std::env::var("NEUROSPLOIT_DECISION_ENDPOINT").ok().filter(|e| !e.trim().is_empty());
if key.is_none() && endpoint.is_none() {
return None;
}
let bearer = key.is_some();
Some(TypeSafe { Some(TypeSafe {
key, key: key.unwrap_or_default(),
client: reqwest::Client::builder().timeout(Duration::from_secs(30)).build().unwrap_or_default(), client: reqwest::Client::builder().timeout(Duration::from_secs(60)).build().unwrap_or_default(),
endpoint: endpoint.unwrap_or_else(|| DEFAULT_ENDPOINT.to_string()),
model: std::env::var("NEUROSPLOIT_DECISION_MODEL").ok().filter(|m| !m.trim().is_empty()).unwrap_or_else(|| DEFAULT_MODEL.to_string()),
bearer,
}) })
} }
pub fn new(key: &str) -> TypeSafe { pub fn new(key: &str) -> TypeSafe {
TypeSafe { key: key.to_string(), client: reqwest::Client::new() } TypeSafe { key: key.to_string(), client: reqwest::Client::new(), endpoint: DEFAULT_ENDPOINT.to_string(), model: DEFAULT_MODEL.to_string(), bearer: true }
}
/// Which backend this instance talks to, for the run banner.
pub fn backend_label(&self) -> String {
if self.bearer { format!("TypeSafe ({})", self.model) } else { format!("local decision backend ({})", self.endpoint) }
} }
/// Evaluate a set of independent questions over one state, in parallel (the /// Evaluate a set of independent questions over one state, in parallel (the
/// API runs them together — they cannot see one another's answers). /// API runs them together — they cannot see one another's answers).
pub async fn evaluate(&self, state: serde_json::Value, questions: BTreeMap<String, Question>) -> Result<BTreeMap<String, Answer>, String> { pub async fn evaluate(&self, state: serde_json::Value, questions: BTreeMap<String, Question>) -> Result<BTreeMap<String, Answer>, String> {
let req = Request { model: MODEL.into(), state, questions }; let req = Request { model: self.model.clone(), state, questions };
// A short retry on the documented transient codes (429/529). // A short retry on the documented transient codes (429/529).
let mut attempt = 0; let mut attempt = 0;
loop { loop {
attempt += 1; attempt += 1;
let resp = self.client.post(ENDPOINT).bearer_auth(&self.key).json(&req).send().await; let mut rb = self.client.post(&self.endpoint).json(&req);
if self.bearer { rb = rb.bearer_auth(&self.key); }
let resp = rb.send().await;
match resp { match resp {
Ok(r) => { Ok(r) => {
let status = r.status().as_u16(); let status = r.status().as_u16();
+155
View File
@@ -0,0 +1,155 @@
#!/usr/bin/env python3
"""
Laya decision-backend shim for NeuroSploit.
Laya (https://github.com/NandhaKishorM/laya) is a local, open-source System One
decision engine with the same primitives as TypeSafe (choice / score / noul).
It ships as a Python library with no server, so this shim exposes it over the
exact HTTP contract NeuroSploit already speaks to TypeSafe:
POST /systemone
{ "model": "...", "state": <any>, "questions": { "<id>": {type, instructions, criteria} } }
-> { "answers": { "<id>": { choice|score|noul, probabilities, confidence } } }
Point NeuroSploit at it with:
--decision-backend laya (starts/uses http://127.0.0.1:8799)
or manually:
NEUROSPLOIT_DECISION_ENDPOINT=http://127.0.0.1:8799/systemone \
NEUROSPLOIT_DECISION_MODEL=laya neurosploit run ... --typesafe on
The Laya model is downloaded automatically on first use (Hugging Face cache),
so nothing is bundled and the whole thing stays optional. No API key, and the
engagement's evidence never leaves the machine.
Dependencies (installed on demand by NeuroSploit when you pick the laya backend,
or by hand): pip install laya
Zero third-party deps here beyond laya itself — the HTTP server is stdlib.
"""
import json
import os
import sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
HOST = os.environ.get("LAYA_SHIM_HOST", "127.0.0.1")
PORT = int(os.environ.get("LAYA_SHIM_PORT", "8799"))
# Lazy, so `--help`/import never pays the model-load cost.
_agent = None
_load_error = None
def _get_agent():
"""Load Laya once. First call downloads the model into the HF cache."""
global _agent, _load_error
if _agent is not None or _load_error is not None:
return _agent
try:
import laya # noqa
# Router picks the right checkpoint (English / multilingual) per input.
try:
from laya import Router
_agent = Router(preload=True)
_agent_kind = "router"
except Exception:
_agent = laya.load(os.environ.get("LAYA_MODEL", "convaiinnovations/laya"))
_agent_kind = "single"
sys.stderr.write(f"[laya-shim] model ready ({_agent_kind})\n")
sys.stderr.flush()
except Exception as e: # noqa
_load_error = str(e)
sys.stderr.write(f"[laya-shim] failed to load laya: {e}\n")
return _agent
def _evaluate(model, state, questions):
"""Run Laya over the questions and normalise to the TypeSafe answer shape."""
agent = _get_agent()
if agent is None:
raise RuntimeError(f"laya unavailable: {_load_error}")
# Laya takes the state as text plus the questions map; accept dict or str.
text = state if isinstance(state, str) else json.dumps(state, ensure_ascii=False)
# Laya's own call surface varies by version; try the documented ones in order.
raw = None
for call in (
lambda: agent.evaluate(text, questions),
lambda: agent(text, questions),
lambda: agent.decide(text, questions),
):
try:
raw = call()
break
except (AttributeError, TypeError):
continue
if raw is None:
raise RuntimeError("no compatible laya call surface (evaluate/__call__/decide)")
answers_in = raw.get("answers", raw) if isinstance(raw, dict) else {}
answers = {}
for qid, q in questions.items():
a = answers_in.get(qid, {}) if isinstance(answers_in, dict) else {}
qtype = q.get("type")
out = {}
if qtype == "choice":
out["choice"] = a.get("choice")
out["probabilities"] = a.get("probabilities", {}) or {}
out["confidence"] = a.get("confidence")
elif qtype == "score":
out["score"] = a.get("score")
out["probabilities"] = a.get("probabilities", {}) or {}
out["confidence"] = a.get("confidence")
elif qtype == "noul":
out["noul"] = a.get("noul")
answers[qid] = out
return {"model": model or "laya", "answers": answers}
class Handler(BaseHTTPRequestHandler):
def _send(self, code, obj):
body = json.dumps(obj).encode("utf-8")
self.send_response(code)
self.send_header("content-type", "application/json")
self.send_header("content-length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_GET(self): # health / readiness
if self.path.rstrip("/") in ("/health", "/healthz", ""):
ready = _get_agent() is not None
self._send(200 if ready else 503, {"ready": ready, "backend": "laya", "error": _load_error})
else:
self._send(404, {"error": "not found"})
def do_POST(self):
if self.path.rstrip("/") != "/systemone":
return self._send(404, {"error": "post to /systemone"})
try:
n = int(self.headers.get("content-length", "0"))
req = json.loads(self.rfile.read(n) or b"{}")
except Exception as e: # noqa
return self._send(400, {"error": f"bad request: {e}"})
try:
self._send(200, _evaluate(req.get("model"), req.get("state"), req.get("questions", {})))
except Exception as e: # noqa
self._send(500, {"error": str(e)})
def log_message(self, *a): # quiet
pass
def main():
# Warm the model up front so the first real request isn't the one that waits
# on a multi-hundred-MB download.
sys.stderr.write(f"[laya-shim] loading model (first run downloads it)…\n")
sys.stderr.flush()
_get_agent()
srv = ThreadingHTTPServer((HOST, PORT), Handler)
sys.stderr.write(f"[laya-shim] listening on http://{HOST}:{PORT}/systemone\n")
sys.stderr.flush()
srv.serve_forever()
if __name__ == "__main__":
main()