Compare commits

..

25 Commits

Author SHA1 Message Date
dependabot[bot] d11ba2b95c build(deps): bump js-yaml and eslint in /ui
Bumps [js-yaml](https://github.com/nodeca/js-yaml) to 4.1.1 and updates ancestor dependency [eslint](https://github.com/eslint/eslint). These dependencies need to be updated together.


Updates `js-yaml` from 3.14.1 to 4.1.1
- [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/3.14.1...4.1.1)

Updates `eslint` from 7.32.0 to 9.39.1
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/compare/v7.32.0...v9.39.1)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 4.1.1
  dependency-type: indirect
- dependency-name: eslint
  dependency-version: 9.39.1
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-15 03:06:56 +00:00
Alexander Myasoedov c45778f196 Merge pull request #252 from Davda-James/feat/mcp_client_logging
logging added for mcp client operations
2025-08-21 15:00:22 +03:00
Alexander Myasoedov a5bdbe54a2 Merge branch 'main' of github.com:msoedov/agentic_security 2025-08-13 13:52:19 +03:00
Alexander Myasoedov 61da912f18 feat(update deps): 2025-08-13 13:46:37 +03:00
DavdaJames a02aed2c2b changes done by pre-commit hooks 2025-08-10 14:33:25 +05:30
DavdaJames 40ff7f9dfb added the comments back 2025-08-10 13:49:08 +05:30
DavdaJames c09ce32def feature added for logging of mcp client 2025-08-10 13:42:32 +05:30
Alexander Myasoedov c5406e8a0e Merge pull request #238 from msoedov/dependabot/npm_and_yarn/ui/multi-96c788614a
build(deps): bump on-headers and compression in /ui
2025-07-18 13:33:47 +03:00
dependabot[bot] b260672b1a build(deps): bump on-headers and compression in /ui
Bumps [on-headers](https://github.com/jshttp/on-headers) and [compression](https://github.com/expressjs/compression). These dependencies needed to be updated together.

Updates `on-headers` from 1.0.2 to 1.1.0
- [Release notes](https://github.com/jshttp/on-headers/releases)
- [Changelog](https://github.com/jshttp/on-headers/blob/master/HISTORY.md)
- [Commits](https://github.com/jshttp/on-headers/compare/v1.0.2...v1.1.0)

Updates `compression` from 1.8.0 to 1.8.1
- [Release notes](https://github.com/expressjs/compression/releases)
- [Changelog](https://github.com/expressjs/compression/blob/master/HISTORY.md)
- [Commits](https://github.com/expressjs/compression/compare/1.8.0...v1.8.1)

---
updated-dependencies:
- dependency-name: on-headers
  dependency-version: 1.1.0
  dependency-type: indirect
- dependency-name: compression
  dependency-version: 1.8.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-18 10:32:43 +00:00
Alexander Myasoedov 0a07fc54d6 Merge pull request #229 from msoedov/dependabot/pip/requests-2.32.4
build(deps): bump requests from 2.32.3 to 2.32.4
2025-06-10 14:03:41 +03:00
dependabot[bot] 2f1151d44d build(deps): bump requests from 2.32.3 to 2.32.4
Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.3...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-10 09:13:51 +00:00
Alexander Myasoedov d0353e3ab9 fix(bump pyproject): 2025-05-27 13:46:33 +03:00
Alexander Myasoedov 926c583a17 fix(csv ds loading): 2025-05-27 13:41:10 +03:00
Alexander Myasoedov 17e34356e1 feat(bump version): 2025-05-19 12:35:44 +03:00
Alexander Myasoedov 312fa756a5 feat(rm ref): 2025-05-19 12:33:27 +03:00
Alexander Myasoedov 145e7f81e1 feat(Update readme): 2025-05-19 12:32:48 +03:00
Alexander Myasoedov 04af7d24a1 Merge pull request #223 from lwsinclair/add-mseep-badge
Add MseeP.ai badge
2025-05-19 12:31:16 +03:00
Alexander Myasoedov c5c5ae2e4b fix(makedir): 2025-05-19 12:29:28 +03:00
Alexander Myasoedov 2bc0605a1d Merge pull request #224 from Mundi-Xu/datasets-optimize
refactor: standardize CSV loading from ./datasets and improve robustness
2025-05-19 12:27:25 +03:00
Hanyin 335787d40e refactor: standardize CSV loading from ./datasets and improve robustness
- Load all CSVs from ./datasets directory
- Add encoding_errors='ignore' for resilient CSV parsing
- Ensure prompt generators are converted to lists before sampling
2025-05-19 16:19:38 +08:00
Lawrence Sinclair 1b211b5d76 Add MseeP.ai badge to Readme.md 2025-05-14 17:46:50 +07:00
Alexander Myasoedov 444f908009 Merge pull request #220 from msoedov/dependabot/npm_and_yarn/ui/http-proxy-middleware-2.0.9
build(deps-dev): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /ui
2025-05-02 13:04:54 +03:00
dependabot[bot] f81dc508f9 build(deps-dev): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /ui
Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) from 2.0.7 to 2.0.9.
- [Release notes](https://github.com/chimurai/http-proxy-middleware/releases)
- [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/v2.0.9/CHANGELOG.md)
- [Commits](https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7...v2.0.9)

---
updated-dependencies:
- dependency-name: http-proxy-middleware
  dependency-version: 2.0.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-29 02:24:24 +00:00
Alexander Myasoedov 4a55b99d70 Merge pull request #215 from Davda-James/fix/Dockerfile
Fixed the Dockerfile error of setuptools and wheel
2025-04-09 19:56:08 +03:00
DavdaJames 5c2f9eba71 wheel and setuptools are required before running RUN pip install --no-cache-dir -r requirements.txt which is missing in dockerfile and hence docker build was breaking in between build process 2025-04-09 20:23:03 +05:30
10 changed files with 2932 additions and 2767 deletions
+4
View File
@@ -19,6 +19,10 @@ RUN poetry lock
# Install dependencies
RUN poetry export -f requirements.txt --without-hashes -o requirements.txt
# Install wheel (required to build packages like fire)
RUN pip install --upgrade pip setuptools wheel
RUN pip install --no-cache-dir -r requirements.txt
# Runtime stage
+1 -3
View File
@@ -21,9 +21,7 @@
<a href="https://pypi.org/project/agentic-security/">
<img alt="PyPI Version" src="https://img.shields.io/pypi/v/agentic-security?style=for-the-badge&logo=pypi&labelColor=000000&color=00CCFF" />
</a>
<a href="https://discord.gg/stw3DfZQ">
<img alt="Join Discord" src="https://img.shields.io/badge/Discord-Join%20Us-black?style=for-the-badge&logo=discord&labelColor=000000&color=DD55FF" />
</a>
</p>
+43 -29
View File
@@ -3,6 +3,8 @@ import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from agentic_security.logutils import logger
# Create server parameters for stdio connection
server_params = StdioServerParameters(
command="python", # Executable
@@ -12,42 +14,54 @@ server_params = StdioServerParameters(
async def run() -> None:
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
# Initialize the connection --> connection does not work
await session.initialize()
try:
logger.info(
"Starting stdio client session with server parameters: %s", server_params
)
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
# Initialize the connection --> connection does not work
logger.info("Initializing client session...")
await session.initialize()
# List available prompts, resources, and tools --> no avalialbe tools
prompts = await session.list_prompts()
print(f"Available prompts: {prompts}")
# List available prompts, resources, and tools --> no avalialbe tools
logger.info("Listing available prompts...")
prompts = await session.list_prompts()
logger.info(f"Available prompts: {prompts}")
resources = await session.list_resources()
print(f"Available resources: {resources}")
logger.info("Listing available resources...")
resources = await session.list_resources()
logger.info(f"Available resources: {resources}")
tools = await session.list_tools()
print(f"Available tools: {tools}")
logger.info("Listing available tools...")
tools = await session.list_tools()
logger.info(f"Available tools: {tools}")
# Call the echo tool --> echo tool iisue
echo_result = await session.call_tool(
"echo_tool", arguments={"message": "Hello from client!"}
)
print(f"Tool result: {echo_result}")
# Call the echo tool --> echo tool issue
logger.info("Calling echo_tool with message...")
echo_result = await session.call_tool(
"echo_tool", arguments={"message": "Hello from client!"}
)
logger.info(f"Tool result: {echo_result}")
# # Read the echo resource
# echo_content, mime_type = await session.read_resource(
# "echo://Hello_resource"
# )
# print(f"Resource content: {echo_content}")
# print(f"Resource MIME type: {mime_type}")
# # Read the echo resource
# echo_content, mime_type = await session.read_resource(
# "echo://Hello_resource"
# )
# logger.info(f"Resource content: {echo_content}")
# logger.info(f"Resource MIME type: {mime_type}")
# # Get and use the echo prompt
# prompt_result = await session.get_prompt(
# "echo_prompt", arguments={"message": "Hello prompt!"}
# )
# print(f"Prompt result: {prompt_result}")
# # Get and use the echo prompt
# prompt_result = await session.get_prompt(
# "echo_prompt", arguments={"message": "Hello prompt!"}
# )
# logger.info(f"Prompt result: {prompt_result}")
# You can perform additional operations here as needed
return prompts, resources, tools
logger.info("Client operations completed successfully.")
return prompts, resources, tools
except Exception as e:
logger.error(f"An error occurred during client operations: {e}", exc_info=True)
raise
if __name__ == "__main__":
-1
View File
@@ -4,7 +4,6 @@ from mcp.server.fastmcp import FastMCP
# Initialize MCP server
mcp = FastMCP(
name="Agentic Security MCP Server",
description="MCP server to interact with LLM scanning test",
dependencies=["httpx"],
)
+34 -40
View File
@@ -8,7 +8,6 @@ from typing import Any, TypeVar
import httpx
import pandas as pd
from cache_to_disk import cache_to_disk
from datasets import load_dataset
from agentic_security.logutils import logger
from agentic_security.probe_data import stenography_fn
@@ -20,6 +19,7 @@ from agentic_security.probe_data.modules import (
inspect_ai_tool,
rl_model,
)
from datasets import load_dataset
# Type aliases for clarity
T = TypeVar("T")
@@ -245,61 +245,47 @@ def load_jailbreak_v28k() -> ProbeDataset:
return create_probe_dataset("JailbreakV-28K/JailBreakV-28k", [])
@cache_to_disk()
def load_local_csv() -> ProbeDataset:
"""Load prompts from local CSV files."""
csv_files = [f for f in os.listdir(".") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
prompts = []
for file in csv_files:
try:
df = pd.read_csv(file)
if "prompt" in df.columns:
prompts.extend(df["prompt"].tolist())
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
return create_probe_dataset("Local CSV", prompts, {"src": str(csv_files)})
@cache_to_disk(1)
def load_csv(file: str) -> ProbeDataset:
"""Load prompts from local CSV files."""
def file_dataset(file) -> list[str]:
prompts = []
try:
df = pd.read_csv(file)
prompts = df["prompt"].tolist()
df = pd.read_csv(os.path.join("./datasets", file), encoding_errors="ignore")
if "prompt" in df.columns:
prompts.extend(df["prompt"].tolist())
prompts = df["prompt"].tolist()
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
return prompts
def load_local_csv() -> ProbeDataset:
"""Load prompts from local CSV files."""
os.makedirs("./datasets", exist_ok=True)
csv_files = [f for f in os.listdir("./datasets") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
prompts = []
for file in csv_files:
prompts.extend(file_dataset(file))
return create_probe_dataset("Local CSV", prompts, {"src": str(csv_files)})
def load_csv(file: str) -> ProbeDataset:
"""Load prompts from local CSV files."""
prompts = file_dataset(file)
return create_probe_dataset(f"fs://{file}", prompts, {"src": str(file)})
@cache_to_disk(1)
def load_local_csv_files() -> list[ProbeDataset]:
"""Load prompts from local CSV files and return a list of ProbeDataset objects."""
csv_files = [f for f in os.listdir(".") if f.endswith(".csv")]
csv_files = [f for f in os.listdir("./datasets") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
datasets = []
for file in csv_files:
try:
df = pd.read_csv(file)
if "prompt" in df.columns:
prompts = df["prompt"].tolist()
datasets.append(create_probe_dataset(file, prompts, {"src": file}))
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
datasets.append(create_probe_dataset(file, file_dataset(file), {"src": file}))
return datasets
@@ -336,12 +322,20 @@ class StenographyTransformer:
)
continue
size = len(dataset.prompts)
# Convert async generator (if any) to a list
prompts = (
list(dataset.prompts) if hasattr(dataset.prompts, "__iter__") else []
)
size = len(prompts)
if size == 0:
continue
sample_size = max(1, int(size * 0.25)) # At least 1 sample
for name, fn in self.TRANSFORMATIONS.items():
logger.info(f"Applying {name} to {dataset.dataset_name}")
sampled = random.sample(dataset.prompts, min(sample_size, size))
sampled = random.sample(prompts, min(sample_size, size))
try:
transformed = [
f"Follow the instructions in {name}: {fn(str(p))}"
Generated
+2359 -2068
View File
File diff suppressed because it is too large Load Diff
+23 -27
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "agentic_security"
version = "0.7.2"
version = "0.7.4"
description = "Agentic LLM vulnerability scanner"
authors = ["Alexander Miasoiedov <msoedov@gmail.com>"]
maintainers = ["Alexander Miasoiedov <msoedov@gmail.com>"]
@@ -28,54 +28,50 @@ agentic_security = "agentic_security.__main__:main"
[tool.poetry.dependencies]
python = "^3.11"
fastapi = "^0.115.8"
uvicorn = "^0.34.0"
fastapi = "^0.116.1"
uvicorn = "^0.35.0"
fire = "0.7.0"
loguru = "^0.7.3"
httpx = "^0.28.1"
cache-to-disk = "^2.0.0"
pandas = ">=1.4,<3.0"
datasets = "^3.3.0"
datasets = "^4.0.0"
tabulate = ">=0.8.9,<0.10.0"
colorama = "^0.4.4"
matplotlib = "^3.9.2"
pydantic = "2.10.6"
matplotlib = "^3.10.5"
pydantic = "^2.11.7"
scikit-optimize = "^0.10.2"
scikit-learn = "1.6.1"
scikit-learn = "^1.7.1"
numpy = ">=1.24.3,<3.0.0"
jinja2 = "^3.1.4"
python-multipart = "^0.0.20"
tomli = "^2.2.1"
rich = "13.9.4"
rich = "^14.1.0"
gTTS = "^2.5.4"
sentry_sdk = "^2.22.0"
orjson = "^3.10"
pyfiglet = "^1.0.2"
termcolor = "^2.4.0"
mcp = "^1.4.1"
sentry_sdk = "^2.34.1"
orjson = "^3.11.2"
pyfiglet = "^1.0.3"
termcolor = "^3.1.0"
mcp = "^1.12.4"
# garak = { version = "*", optional = true }
pytest-xdist = "3.6.1"
pytest-xdist = "^3.8.0"
[tool.poetry.group.dev.dependencies]
# Pytest
pytest = "^8.3.4"
pytest-asyncio = "^0.25.2"
inline-snapshot = ">=0.13.3,<0.21.0"
pytest = "^8.4.1"
pytest-asyncio = "^1.1.0"
inline-snapshot = ">=0.13.3,<0.27.2"
pytest-httpx = "^0.35.0"
pytest-mock = "^3.14.0"
pytest-mock = "^3.14.1"
# Rest
black = ">=24.10,<26.0"
mypy = "^1.12.0"
pre-commit = "^4.0.1"
huggingface-hub = ">=0.25.1,<0.30.0"
mypy = "^1.17.1"
pre-commit = "^4.3.0"
huggingface-hub = ">=0.25.1,<0.34.4"
# Docs
mkdocs = ">=1.4.2"
mkdocs-material = "^9.6.4"
mkdocstrings = ">=0.26.1"
mkdocs-material = "^9.6.16"
mkdocstrings = ">=0.29.0"
mkdocs-jupyter = ">=0.25.1"
+1 -1
View File
@@ -1,7 +1,7 @@
import pytest
from datasets import load_dataset
from agentic_security.probe_data import REGISTRY
from datasets import load_dataset
@pytest.mark.slow
+466 -597
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -19,7 +19,7 @@
"@vue/cli-plugin-babel": "~5.0.0",
"@vue/cli-plugin-eslint": "~5.0.0",
"@vue/cli-service": "~5.0.0",
"eslint": "^7.32.0",
"eslint": "^9.39.1",
"eslint-plugin-vue": "^8.0.3"
},
"eslintConfig": {