ExecutorMetrics.get_stats picked the p95 element with
int(n * 0.95), which collapses to the last index (the maximum)
for any batch of 20 or fewer requests, so the reported p95 was
actually the worst-case latency for typical scan batches. Use
ceil(n * 0.95) - 1, the standard nearest-rank index.
record_failure ignored the half_open state: the minimum-sample gate
(total >= 10) and the success counter meant a failure during
probation neither re-opened the circuit nor prevented it from
closing. One failure followed by three successes closed the circuit
with the failure forgotten. Trip back to open on the first half-open
failure, with fresh counters.
PIIDetector and SandboxEscapeDetector were wired directly in
probe_actor/refusal.py and the refusal classifier manager was populated from
a hardcoded list, so the only way to toggle a bundled detector or add an
organization-specific signature was to patch the module.
Add a DetectorRegistry mapping plugin names to factories, assembled from an
agentic_security.toml [detectors] section via build_from_config. Custom
detectors load by import path ("pkg.module:ClassName"). refusal.py gains
build_refusal_manager(config=None) reading the [detectors] table; all public
symbols are preserved. Built-in leak detectors ship registered but disabled,
so default refusal_heuristic behaviour is unchanged.
Closes#82
Signed-off-by: Devam Shah <devamshah91@gmail.com>
CORS: drop allow_credentials=True. With allow_origins=['*'] Starlette reflects
the request Origin and emits Access-Control-Allow-Credentials: true on any
credentialed request — a reflect-any-origin hole. The app authenticates with
Bearer tokens in the Authorization header, so credentialed CORS was never needed.
/icons/{icon_name}: validate against a strict allowlist (re.fullmatch
[A-Za-z0-9._-]+\.png) and assert the resolved path stays inside ICONS_DIR before
any filesystem write or outbound fetch. Closes the path-traversal / arbitrary-write
(CWE-22) and controlled-URL (CWE-73) surface on the unsanitized icon_name.
Adds CORS middleware tests and icon-name validation tests (traversal, encoded
slash, null byte, trailing newline, wrong extension).
Closes#298
Signed-off-by: Devam Shah <devamshah91@gmail.com>
Closes#307
Agno was imported by nothing, had undefined-variable bugs,
and was not a declared dependency.
Removed:
- agentic_security/agents/ (operator_agno.py)
- docs/mcp_agno_integration.md
- .gitignore reference to operator_agno.py
No agno references remain in source code.
Pre-existing test failures (missing tabulate module) confirmed
unrelated to this change via git stash verification.
Previously, calculate_cost() was always called without a model parameter,
causing all scans to report costs based on deepseek-chat pricing regardless
of the actual target model (e.g. gpt-4, claude-3-opus).
Changes:
- http_spec.py: Add 'model_name' property to LLMSpec that extracts the
model field from the JSON request body. Returns 'unknown' if the body
is not valid JSON or has no 'model' field.
- probe_data/image_generator.py: Add 'model_name' pass-through property
to RequestAdapter, delegating to the underlying LLMSpec.
- probe_data/audio_generator.py: Same as above - add 'model_name'
pass-through property to RequestAdapter.
- probe_actor/cost_module.py:
- Change return type from float to float | None.
- Unknown models now log a warning and return None instead of raising
ValueError, so scans are not interrupted by unsupported model names.
- Add logger import for the warning message.
- probe_actor/fuzzer.py: Pass model_name to calculate_cost() in both
scan_module() and perform_many_shot_scan() using
getattr(request_factory, 'model_name', 'unknown').
- primitives/models.py: Update ScanResult.cost type from float to
float | None to accommodate unknown model pricing.
msj_data.py contained a full copy of the ProbeDataset dataclass that
was already defined canonically in probe_data/models.py, violating DRY
and leaving a stale TODO comment in the source.
Changes:
- probe_data/msj_data.py: delete the 19-line duplicate ProbeDataset
definition and the now-unused 'from dataclasses import dataclass'
import; replace with a single re-export:
from agentic_security.probe_data.models import ProbeDataset
All call-sites inside the file (load_dataset_generic, prepare_prompts)
continue to work unchanged because the field signatures are identical.
The TODO comment is removed as the refactor is now complete.
No changes required in consumers (fuzzer.py, test_msj_data.py) because
they access ProbeDataset through msj_data's re-export.
TianHengZhuang