Compare commits

...

25 Commits

Author SHA1 Message Date
dependabot[bot] f367a23ff9 build(deps): bump mcp from 1.12.4 to 1.19.0
Bumps [mcp](https://github.com/modelcontextprotocol/python-sdk) from 1.12.4 to 1.19.0.
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v1.12.4...v1.19.0)

---
updated-dependencies:
- dependency-name: mcp
  dependency-version: 1.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-24 17:12:36 +00:00
Alexander Myasoedov c45778f196 Merge pull request #252 from Davda-James/feat/mcp_client_logging
logging added for mcp client operations
2025-08-21 15:00:22 +03:00
Alexander Myasoedov a5bdbe54a2 Merge branch 'main' of github.com:msoedov/agentic_security 2025-08-13 13:52:19 +03:00
Alexander Myasoedov 61da912f18 feat(update deps): 2025-08-13 13:46:37 +03:00
DavdaJames a02aed2c2b changes done by pre-commit hooks 2025-08-10 14:33:25 +05:30
DavdaJames 40ff7f9dfb added the comments back 2025-08-10 13:49:08 +05:30
DavdaJames c09ce32def feature added for logging of mcp client 2025-08-10 13:42:32 +05:30
Alexander Myasoedov c5406e8a0e Merge pull request #238 from msoedov/dependabot/npm_and_yarn/ui/multi-96c788614a
build(deps): bump on-headers and compression in /ui
2025-07-18 13:33:47 +03:00
dependabot[bot] b260672b1a build(deps): bump on-headers and compression in /ui
Bumps [on-headers](https://github.com/jshttp/on-headers) and [compression](https://github.com/expressjs/compression). These dependencies needed to be updated together.

Updates `on-headers` from 1.0.2 to 1.1.0
- [Release notes](https://github.com/jshttp/on-headers/releases)
- [Changelog](https://github.com/jshttp/on-headers/blob/master/HISTORY.md)
- [Commits](https://github.com/jshttp/on-headers/compare/v1.0.2...v1.1.0)

Updates `compression` from 1.8.0 to 1.8.1
- [Release notes](https://github.com/expressjs/compression/releases)
- [Changelog](https://github.com/expressjs/compression/blob/master/HISTORY.md)
- [Commits](https://github.com/expressjs/compression/compare/1.8.0...v1.8.1)

---
updated-dependencies:
- dependency-name: on-headers
  dependency-version: 1.1.0
  dependency-type: indirect
- dependency-name: compression
  dependency-version: 1.8.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-18 10:32:43 +00:00
Alexander Myasoedov 0a07fc54d6 Merge pull request #229 from msoedov/dependabot/pip/requests-2.32.4
build(deps): bump requests from 2.32.3 to 2.32.4
2025-06-10 14:03:41 +03:00
dependabot[bot] 2f1151d44d build(deps): bump requests from 2.32.3 to 2.32.4
Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.3...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-10 09:13:51 +00:00
Alexander Myasoedov d0353e3ab9 fix(bump pyproject): 2025-05-27 13:46:33 +03:00
Alexander Myasoedov 926c583a17 fix(csv ds loading): 2025-05-27 13:41:10 +03:00
Alexander Myasoedov 17e34356e1 feat(bump version): 2025-05-19 12:35:44 +03:00
Alexander Myasoedov 312fa756a5 feat(rm ref): 2025-05-19 12:33:27 +03:00
Alexander Myasoedov 145e7f81e1 feat(Update readme): 2025-05-19 12:32:48 +03:00
Alexander Myasoedov 04af7d24a1 Merge pull request #223 from lwsinclair/add-mseep-badge
Add MseeP.ai badge
2025-05-19 12:31:16 +03:00
Alexander Myasoedov c5c5ae2e4b fix(makedir): 2025-05-19 12:29:28 +03:00
Alexander Myasoedov 2bc0605a1d Merge pull request #224 from Mundi-Xu/datasets-optimize
refactor: standardize CSV loading from ./datasets and improve robustness
2025-05-19 12:27:25 +03:00
Hanyin 335787d40e refactor: standardize CSV loading from ./datasets and improve robustness
- Load all CSVs from ./datasets directory
- Add encoding_errors='ignore' for resilient CSV parsing
- Ensure prompt generators are converted to lists before sampling
2025-05-19 16:19:38 +08:00
Lawrence Sinclair 1b211b5d76 Add MseeP.ai badge to Readme.md 2025-05-14 17:46:50 +07:00
Alexander Myasoedov 444f908009 Merge pull request #220 from msoedov/dependabot/npm_and_yarn/ui/http-proxy-middleware-2.0.9
build(deps-dev): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /ui
2025-05-02 13:04:54 +03:00
dependabot[bot] f81dc508f9 build(deps-dev): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /ui
Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) from 2.0.7 to 2.0.9.
- [Release notes](https://github.com/chimurai/http-proxy-middleware/releases)
- [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/v2.0.9/CHANGELOG.md)
- [Commits](https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7...v2.0.9)

---
updated-dependencies:
- dependency-name: http-proxy-middleware
  dependency-version: 2.0.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-29 02:24:24 +00:00
Alexander Myasoedov 4a55b99d70 Merge pull request #215 from Davda-James/fix/Dockerfile
Fixed the Dockerfile error of setuptools and wheel
2025-04-09 19:56:08 +03:00
DavdaJames 5c2f9eba71 wheel and setuptools are required before running RUN pip install --no-cache-dir -r requirements.txt which is missing in dockerfile and hence docker build was breaking in between build process 2025-04-09 20:23:03 +05:30
9 changed files with 2669 additions and 2200 deletions
+4
View File
@@ -19,6 +19,10 @@ RUN poetry lock
# Install dependencies
RUN poetry export -f requirements.txt --without-hashes -o requirements.txt
# Install wheel (required to build packages like fire)
RUN pip install --upgrade pip setuptools wheel
RUN pip install --no-cache-dir -r requirements.txt
# Runtime stage
+1 -3
View File
@@ -21,9 +21,7 @@
<a href="https://pypi.org/project/agentic-security/">
<img alt="PyPI Version" src="https://img.shields.io/pypi/v/agentic-security?style=for-the-badge&logo=pypi&labelColor=000000&color=00CCFF" />
</a>
<a href="https://discord.gg/stw3DfZQ">
<img alt="Join Discord" src="https://img.shields.io/badge/Discord-Join%20Us-black?style=for-the-badge&logo=discord&labelColor=000000&color=DD55FF" />
</a>
</p>
+43 -29
View File
@@ -3,6 +3,8 @@ import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from agentic_security.logutils import logger
# Create server parameters for stdio connection
server_params = StdioServerParameters(
command="python", # Executable
@@ -12,42 +14,54 @@ server_params = StdioServerParameters(
async def run() -> None:
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
# Initialize the connection --> connection does not work
await session.initialize()
try:
logger.info(
"Starting stdio client session with server parameters: %s", server_params
)
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
# Initialize the connection --> connection does not work
logger.info("Initializing client session...")
await session.initialize()
# List available prompts, resources, and tools --> no avalialbe tools
prompts = await session.list_prompts()
print(f"Available prompts: {prompts}")
# List available prompts, resources, and tools --> no avalialbe tools
logger.info("Listing available prompts...")
prompts = await session.list_prompts()
logger.info(f"Available prompts: {prompts}")
resources = await session.list_resources()
print(f"Available resources: {resources}")
logger.info("Listing available resources...")
resources = await session.list_resources()
logger.info(f"Available resources: {resources}")
tools = await session.list_tools()
print(f"Available tools: {tools}")
logger.info("Listing available tools...")
tools = await session.list_tools()
logger.info(f"Available tools: {tools}")
# Call the echo tool --> echo tool iisue
echo_result = await session.call_tool(
"echo_tool", arguments={"message": "Hello from client!"}
)
print(f"Tool result: {echo_result}")
# Call the echo tool --> echo tool issue
logger.info("Calling echo_tool with message...")
echo_result = await session.call_tool(
"echo_tool", arguments={"message": "Hello from client!"}
)
logger.info(f"Tool result: {echo_result}")
# # Read the echo resource
# echo_content, mime_type = await session.read_resource(
# "echo://Hello_resource"
# )
# print(f"Resource content: {echo_content}")
# print(f"Resource MIME type: {mime_type}")
# # Read the echo resource
# echo_content, mime_type = await session.read_resource(
# "echo://Hello_resource"
# )
# logger.info(f"Resource content: {echo_content}")
# logger.info(f"Resource MIME type: {mime_type}")
# # Get and use the echo prompt
# prompt_result = await session.get_prompt(
# "echo_prompt", arguments={"message": "Hello prompt!"}
# )
# print(f"Prompt result: {prompt_result}")
# # Get and use the echo prompt
# prompt_result = await session.get_prompt(
# "echo_prompt", arguments={"message": "Hello prompt!"}
# )
# logger.info(f"Prompt result: {prompt_result}")
# You can perform additional operations here as needed
return prompts, resources, tools
logger.info("Client operations completed successfully.")
return prompts, resources, tools
except Exception as e:
logger.error(f"An error occurred during client operations: {e}", exc_info=True)
raise
if __name__ == "__main__":
-1
View File
@@ -4,7 +4,6 @@ from mcp.server.fastmcp import FastMCP
# Initialize MCP server
mcp = FastMCP(
name="Agentic Security MCP Server",
description="MCP server to interact with LLM scanning test",
dependencies=["httpx"],
)
+34 -40
View File
@@ -8,7 +8,6 @@ from typing import Any, TypeVar
import httpx
import pandas as pd
from cache_to_disk import cache_to_disk
from datasets import load_dataset
from agentic_security.logutils import logger
from agentic_security.probe_data import stenography_fn
@@ -20,6 +19,7 @@ from agentic_security.probe_data.modules import (
inspect_ai_tool,
rl_model,
)
from datasets import load_dataset
# Type aliases for clarity
T = TypeVar("T")
@@ -245,61 +245,47 @@ def load_jailbreak_v28k() -> ProbeDataset:
return create_probe_dataset("JailbreakV-28K/JailBreakV-28k", [])
@cache_to_disk()
def load_local_csv() -> ProbeDataset:
"""Load prompts from local CSV files."""
csv_files = [f for f in os.listdir(".") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
prompts = []
for file in csv_files:
try:
df = pd.read_csv(file)
if "prompt" in df.columns:
prompts.extend(df["prompt"].tolist())
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
return create_probe_dataset("Local CSV", prompts, {"src": str(csv_files)})
@cache_to_disk(1)
def load_csv(file: str) -> ProbeDataset:
"""Load prompts from local CSV files."""
def file_dataset(file) -> list[str]:
prompts = []
try:
df = pd.read_csv(file)
prompts = df["prompt"].tolist()
df = pd.read_csv(os.path.join("./datasets", file), encoding_errors="ignore")
if "prompt" in df.columns:
prompts.extend(df["prompt"].tolist())
prompts = df["prompt"].tolist()
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
return prompts
def load_local_csv() -> ProbeDataset:
"""Load prompts from local CSV files."""
os.makedirs("./datasets", exist_ok=True)
csv_files = [f for f in os.listdir("./datasets") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
prompts = []
for file in csv_files:
prompts.extend(file_dataset(file))
return create_probe_dataset("Local CSV", prompts, {"src": str(csv_files)})
def load_csv(file: str) -> ProbeDataset:
"""Load prompts from local CSV files."""
prompts = file_dataset(file)
return create_probe_dataset(f"fs://{file}", prompts, {"src": str(file)})
@cache_to_disk(1)
def load_local_csv_files() -> list[ProbeDataset]:
"""Load prompts from local CSV files and return a list of ProbeDataset objects."""
csv_files = [f for f in os.listdir(".") if f.endswith(".csv")]
csv_files = [f for f in os.listdir("./datasets") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
datasets = []
for file in csv_files:
try:
df = pd.read_csv(file)
if "prompt" in df.columns:
prompts = df["prompt"].tolist()
datasets.append(create_probe_dataset(file, prompts, {"src": file}))
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
datasets.append(create_probe_dataset(file, file_dataset(file), {"src": file}))
return datasets
@@ -336,12 +322,20 @@ class StenographyTransformer:
)
continue
size = len(dataset.prompts)
# Convert async generator (if any) to a list
prompts = (
list(dataset.prompts) if hasattr(dataset.prompts, "__iter__") else []
)
size = len(prompts)
if size == 0:
continue
sample_size = max(1, int(size * 0.25)) # At least 1 sample
for name, fn in self.TRANSFORMATIONS.items():
logger.info(f"Applying {name} to {dataset.dataset_name}")
sampled = random.sample(dataset.prompts, min(sample_size, size))
sampled = random.sample(prompts, min(sample_size, size))
try:
transformed = [
f"Follow the instructions in {name}: {fn(str(p))}"
Generated
+2553 -2089
View File
File diff suppressed because it is too large Load Diff
+23 -27
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "agentic_security"
version = "0.7.2"
version = "0.7.4"
description = "Agentic LLM vulnerability scanner"
authors = ["Alexander Miasoiedov <msoedov@gmail.com>"]
maintainers = ["Alexander Miasoiedov <msoedov@gmail.com>"]
@@ -28,54 +28,50 @@ agentic_security = "agentic_security.__main__:main"
[tool.poetry.dependencies]
python = "^3.11"
fastapi = "^0.115.8"
uvicorn = "^0.34.0"
fastapi = "^0.116.1"
uvicorn = "^0.35.0"
fire = "0.7.0"
loguru = "^0.7.3"
httpx = "^0.28.1"
cache-to-disk = "^2.0.0"
pandas = ">=1.4,<3.0"
datasets = "^3.3.0"
datasets = "^4.0.0"
tabulate = ">=0.8.9,<0.10.0"
colorama = "^0.4.4"
matplotlib = "^3.9.2"
pydantic = "2.10.6"
matplotlib = "^3.10.5"
pydantic = "^2.11.7"
scikit-optimize = "^0.10.2"
scikit-learn = "1.6.1"
scikit-learn = "^1.7.1"
numpy = ">=1.24.3,<3.0.0"
jinja2 = "^3.1.4"
python-multipart = "^0.0.20"
tomli = "^2.2.1"
rich = "13.9.4"
rich = "^14.1.0"
gTTS = "^2.5.4"
sentry_sdk = "^2.22.0"
orjson = "^3.10"
pyfiglet = "^1.0.2"
termcolor = "^2.4.0"
mcp = "^1.4.1"
sentry_sdk = "^2.34.1"
orjson = "^3.11.2"
pyfiglet = "^1.0.3"
termcolor = "^3.1.0"
mcp = "^1.12.4"
# garak = { version = "*", optional = true }
pytest-xdist = "3.6.1"
pytest-xdist = "^3.8.0"
[tool.poetry.group.dev.dependencies]
# Pytest
pytest = "^8.3.4"
pytest-asyncio = "^0.25.2"
inline-snapshot = ">=0.13.3,<0.21.0"
pytest = "^8.4.1"
pytest-asyncio = "^1.1.0"
inline-snapshot = ">=0.13.3,<0.27.2"
pytest-httpx = "^0.35.0"
pytest-mock = "^3.14.0"
pytest-mock = "^3.14.1"
# Rest
black = ">=24.10,<26.0"
mypy = "^1.12.0"
pre-commit = "^4.0.1"
huggingface-hub = ">=0.25.1,<0.30.0"
mypy = "^1.17.1"
pre-commit = "^4.3.0"
huggingface-hub = ">=0.25.1,<0.34.4"
# Docs
mkdocs = ">=1.4.2"
mkdocs-material = "^9.6.4"
mkdocstrings = ">=0.26.1"
mkdocs-material = "^9.6.16"
mkdocstrings = ">=0.29.0"
mkdocs-jupyter = ">=0.25.1"
+1 -1
View File
@@ -1,7 +1,7 @@
import pytest
from datasets import load_dataset
from agentic_security.probe_data import REGISTRY
from datasets import load_dataset
@pytest.mark.slow
+10 -10
View File
@@ -4266,9 +4266,9 @@
}
},
"node_modules/compression": {
"version": "1.8.0",
"resolved": "https://registry.npmjs.org/compression/-/compression-1.8.0.tgz",
"integrity": "sha512-k6WLKfunuqCYD3t6AsuPGvQWaKwuLLh2/xHNcX4qE+vIfDNXpSqnrhwA7O53R7WVQUnt8dVAIW+YHr7xTgOgGA==",
"version": "1.8.1",
"resolved": "https://registry.npmjs.org/compression/-/compression-1.8.1.tgz",
"integrity": "sha512-9mAqGPHLakhCLeNyxPkK4xVo746zQ/czLH1Ky+vkitMnWfWZps8r0qXuwhwizagCRttsL4lfG4pIOvaWLpAP0w==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -4276,7 +4276,7 @@
"compressible": "~2.0.18",
"debug": "2.6.9",
"negotiator": "~0.6.4",
"on-headers": "~1.0.2",
"on-headers": "~1.1.0",
"safe-buffer": "5.2.1",
"vary": "~1.1.2"
},
@@ -6891,9 +6891,9 @@
}
},
"node_modules/http-proxy-middleware": {
"version": "2.0.7",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.7.tgz",
"integrity": "sha512-fgVY8AV7qU7z/MmXJ/rxwbrtQH4jBQ9m7kp3llF0liB7glmFeVZFBepQb32T3y8n8k2+AEYuMPCpinYW+/CuRA==",
"version": "2.0.9",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -8419,9 +8419,9 @@
}
},
"node_modules/on-headers": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/on-headers/-/on-headers-1.0.2.tgz",
"integrity": "sha512-pZAE+FJLoyITytdqK0U5s+FIpjN0JP3OzFi/u8Rx+EV5/W+JTWGXG8xFzevE7AjBfDqHv/8vL8qQsIhHnqRkrA==",
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/on-headers/-/on-headers-1.1.0.tgz",
"integrity": "sha512-737ZY3yNnXy37FHkQxPzt4UZ2UWPWiCZWLvFZ4fu5cueciegX0zGPnrlY6bwRg4FdQOe9YU8MkmJwGhoMybl8A==",
"dev": true,
"license": "MIT",
"engines": {