Compare commits

..

24 Commits

Author SHA1 Message Date
dependabot[bot] 879d2bf819 build(deps): bump mcp from 1.6.0 to 1.10.0
Bumps [mcp](https://github.com/modelcontextprotocol/python-sdk) from 1.6.0 to 1.10.0.
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v1.6.0...v1.10.0)

---
updated-dependencies:
- dependency-name: mcp
  dependency-version: 1.10.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-04 22:08:37 +00:00
Alexander Myasoedov 0a07fc54d6 Merge pull request #229 from msoedov/dependabot/pip/requests-2.32.4
build(deps): bump requests from 2.32.3 to 2.32.4
2025-06-10 14:03:41 +03:00
dependabot[bot] 2f1151d44d build(deps): bump requests from 2.32.3 to 2.32.4
Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.3...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-10 09:13:51 +00:00
Alexander Myasoedov d0353e3ab9 fix(bump pyproject): 2025-05-27 13:46:33 +03:00
Alexander Myasoedov 926c583a17 fix(csv ds loading): 2025-05-27 13:41:10 +03:00
Alexander Myasoedov 17e34356e1 feat(bump version): 2025-05-19 12:35:44 +03:00
Alexander Myasoedov 312fa756a5 feat(rm ref): 2025-05-19 12:33:27 +03:00
Alexander Myasoedov 145e7f81e1 feat(Update readme): 2025-05-19 12:32:48 +03:00
Alexander Myasoedov 04af7d24a1 Merge pull request #223 from lwsinclair/add-mseep-badge
Add MseeP.ai badge
2025-05-19 12:31:16 +03:00
Alexander Myasoedov c5c5ae2e4b fix(makedir): 2025-05-19 12:29:28 +03:00
Alexander Myasoedov 2bc0605a1d Merge pull request #224 from Mundi-Xu/datasets-optimize
refactor: standardize CSV loading from ./datasets and improve robustness
2025-05-19 12:27:25 +03:00
Hanyin 335787d40e refactor: standardize CSV loading from ./datasets and improve robustness
- Load all CSVs from ./datasets directory
- Add encoding_errors='ignore' for resilient CSV parsing
- Ensure prompt generators are converted to lists before sampling
2025-05-19 16:19:38 +08:00
Lawrence Sinclair 1b211b5d76 Add MseeP.ai badge to Readme.md 2025-05-14 17:46:50 +07:00
Alexander Myasoedov 444f908009 Merge pull request #220 from msoedov/dependabot/npm_and_yarn/ui/http-proxy-middleware-2.0.9
build(deps-dev): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /ui
2025-05-02 13:04:54 +03:00
dependabot[bot] f81dc508f9 build(deps-dev): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /ui
Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) from 2.0.7 to 2.0.9.
- [Release notes](https://github.com/chimurai/http-proxy-middleware/releases)
- [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/v2.0.9/CHANGELOG.md)
- [Commits](https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7...v2.0.9)

---
updated-dependencies:
- dependency-name: http-proxy-middleware
  dependency-version: 2.0.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-29 02:24:24 +00:00
Alexander Myasoedov 4a55b99d70 Merge pull request #215 from Davda-James/fix/Dockerfile
Fixed the Dockerfile error of setuptools and wheel
2025-04-09 19:56:08 +03:00
DavdaJames 5c2f9eba71 wheel and setuptools are required before running RUN pip install --no-cache-dir -r requirements.txt which is missing in dockerfile and hence docker build was breaking in between build process 2025-04-09 20:23:03 +05:30
Alexander Myasoedov aa2fe4d1ad feat(bump version): 2025-04-07 14:37:59 +03:00
Alexander Myasoedov cf7c017621 feat(add mcp to deps): 2025-04-07 14:32:40 +03:00
Alexander Myasoedov 73184e3454 fix(simplify tests): 2025-04-07 14:29:41 +03:00
Alexander Myasoedov 3720ece2af fix(test vars): 2025-04-03 20:48:23 +03:00
Alexander Myasoedov 0dc738a11e fix(pc): 2025-04-03 20:43:53 +03:00
Alexander Myasoedov 47ca656d59 Merge pull request #213 from sjay8/main
Fixed issues 191 195
2025-04-03 20:42:50 +03:00
sjay8 4fa166298d Fixed issues 191 195 2025-04-03 00:21:09 -07:00
9 changed files with 405 additions and 100 deletions
+4
View File
@@ -19,6 +19,10 @@ RUN poetry lock
# Install dependencies
RUN poetry export -f requirements.txt --without-hashes -o requirements.txt
# Install wheel (required to build packages like fire)
RUN pip install --upgrade pip setuptools wheel
RUN pip install --no-cache-dir -r requirements.txt
# Runtime stage
+1 -3
View File
@@ -21,9 +21,7 @@
<a href="https://pypi.org/project/agentic-security/">
<img alt="PyPI Version" src="https://img.shields.io/pypi/v/agentic-security?style=for-the-badge&logo=pypi&labelColor=000000&color=00CCFF" />
</a>
<a href="https://discord.gg/stw3DfZQ">
<img alt="Join Discord" src="https://img.shields.io/badge/Discord-Join%20Us-black?style=for-the-badge&logo=discord&labelColor=000000&color=DD55FF" />
</a>
</p>
+5 -4
View File
@@ -11,13 +11,13 @@ server_params = StdioServerParameters(
)
async def run():
async def run() -> None:
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
# Initialize the connection
# Initialize the connection --> connection does not work
await session.initialize()
# List available prompts, resources, and tools
# List available prompts, resources, and tools --> no avalialbe tools
prompts = await session.list_prompts()
print(f"Available prompts: {prompts}")
@@ -27,7 +27,7 @@ async def run():
tools = await session.list_tools()
print(f"Available tools: {tools}")
# Call the echo tool
# Call the echo tool --> echo tool iisue
echo_result = await session.call_tool(
"echo_tool", arguments={"message": "Hello from client!"}
)
@@ -47,6 +47,7 @@ async def run():
# print(f"Prompt result: {prompt_result}")
# You can perform additional operations here as needed
return prompts, resources, tools
if __name__ == "__main__":
+38 -5
View File
@@ -14,7 +14,15 @@ AGENTIC_SECURITY = "http://0.0.0.0:8718"
@mcp.tool()
async def verify_llm(spec: str) -> dict:
"""Verify an LLM model specification using the FastAPI server."""
"""
Verify an LLM model specification using the FastAPI server
Returns:
dict: containing the verification result form the FastAPI server
Args: spect(str): The specification of the LLM model to verify.
"""
url = f"{AGENTIC_SECURITY}/verify"
async with httpx.AsyncClient() as client:
response = await client.post(url, json={"spec": spec})
@@ -28,7 +36,18 @@ async def start_scan(
optimize: bool = False,
enableMultiStepAttack: bool = False,
) -> dict:
"""Start an LLM security scan via the FastAPI server."""
"""
Start an LLM security scan via the FastAPI server.
Returns:
dict: The scan initiation result from the FastAPI server.
Args:
llmSpec (str): The specification of the LLM model.
maxBudget (int): The maximum budget for the scan.
optimize (bool, optional): Whether to enable optimization during scanning. Defaults to False.
enableMultiStepAttack (bool, optional): Whether to enable multi-step attack
"""
url = f"{AGENTIC_SECURITY}/scan"
payload = {
"llmSpec": llmSpec,
@@ -46,7 +65,11 @@ async def start_scan(
@mcp.tool()
async def stop_scan() -> dict:
"""Stop an ongoing scan via the FastAPI server."""
"""Stop an ongoing scan via the FastAPI server.
Returns:
dict: The confirmation from the FastAPI server that the scan has been stopped.
"""
url = f"{AGENTIC_SECURITY}/stop"
async with httpx.AsyncClient() as client:
response = await client.post(url)
@@ -55,7 +78,12 @@ async def stop_scan() -> dict:
@mcp.tool()
async def get_data_config() -> list:
"""Retrieve data configuration from the FastAPI server."""
"""
Retrieve data configuration from the FastAPI server.
Returns:
list: The response from the FastAPI server, confirming the scan has been stopped.
"""
url = f"{AGENTIC_SECURITY}/v1/data-config"
async with httpx.AsyncClient() as client:
response = await client.get(url)
@@ -64,7 +92,12 @@ async def get_data_config() -> list:
@mcp.tool()
async def get_spec_templates() -> list:
"""Retrieve data configuration from the FastAPI server."""
"""
Retrieve data configuration from the FastAPI server.
Returns:
list: The LLM specification templates from the FastAPI server.
"""
url = f"{AGENTIC_SECURITY}/v1/llm-specs"
async with httpx.AsyncClient() as client:
response = await client.get(url)
+33 -39
View File
@@ -245,61 +245,47 @@ def load_jailbreak_v28k() -> ProbeDataset:
return create_probe_dataset("JailbreakV-28K/JailBreakV-28k", [])
@cache_to_disk()
def load_local_csv() -> ProbeDataset:
"""Load prompts from local CSV files."""
csv_files = [f for f in os.listdir(".") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
prompts = []
for file in csv_files:
try:
df = pd.read_csv(file)
if "prompt" in df.columns:
prompts.extend(df["prompt"].tolist())
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
return create_probe_dataset("Local CSV", prompts, {"src": str(csv_files)})
@cache_to_disk(1)
def load_csv(file: str) -> ProbeDataset:
"""Load prompts from local CSV files."""
def file_dataset(file) -> list[str]:
prompts = []
try:
df = pd.read_csv(file)
prompts = df["prompt"].tolist()
df = pd.read_csv(os.path.join("./datasets", file), encoding_errors="ignore")
if "prompt" in df.columns:
prompts.extend(df["prompt"].tolist())
prompts = df["prompt"].tolist()
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
return prompts
def load_local_csv() -> ProbeDataset:
"""Load prompts from local CSV files."""
os.makedirs("./datasets", exist_ok=True)
csv_files = [f for f in os.listdir("./datasets") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
prompts = []
for file in csv_files:
prompts.extend(file_dataset(file))
return create_probe_dataset("Local CSV", prompts, {"src": str(csv_files)})
def load_csv(file: str) -> ProbeDataset:
"""Load prompts from local CSV files."""
prompts = file_dataset(file)
return create_probe_dataset(f"fs://{file}", prompts, {"src": str(file)})
@cache_to_disk(1)
def load_local_csv_files() -> list[ProbeDataset]:
"""Load prompts from local CSV files and return a list of ProbeDataset objects."""
csv_files = [f for f in os.listdir(".") if f.endswith(".csv")]
csv_files = [f for f in os.listdir("./datasets") if f.endswith(".csv")]
logger.info(f"Found {len(csv_files)} CSV files: {csv_files}")
datasets = []
for file in csv_files:
try:
df = pd.read_csv(file)
if "prompt" in df.columns:
prompts = df["prompt"].tolist()
datasets.append(create_probe_dataset(file, prompts, {"src": file}))
else:
logger.warning(f"File {file} lacks a suitable prompt column")
except Exception as e:
logger.error(f"Error reading {file}: {e}")
datasets.append(create_probe_dataset(file, file_dataset(file), {"src": file}))
return datasets
@@ -336,12 +322,20 @@ class StenographyTransformer:
)
continue
size = len(dataset.prompts)
# Convert async generator (if any) to a list
prompts = (
list(dataset.prompts) if hasattr(dataset.prompts, "__iter__") else []
)
size = len(prompts)
if size == 0:
continue
sample_size = max(1, int(size * 0.25)) # At least 1 sample
for name, fn in self.TRANSFORMATIONS.items():
logger.info(f"Applying {name} to {dataset.dataset_name}")
sampled = random.sample(dataset.prompts, min(sample_size, size))
sampled = random.sample(prompts, min(sample_size, size))
try:
transformed = [
f"Follow the instructions in {name}: {fn(str(p))}"
Generated
+307 -45
View File
File diff suppressed because it is too large Load Diff
+2 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "agentic_security"
version = "0.7.1"
version = "0.7.4"
description = "Agentic LLM vulnerability scanner"
authors = ["Alexander Miasoiedov <msoedov@gmail.com>"]
maintainers = ["Alexander Miasoiedov <msoedov@gmail.com>"]
@@ -52,6 +52,7 @@ sentry_sdk = "^2.22.0"
orjson = "^3.10"
pyfiglet = "^1.0.2"
termcolor = "^2.4.0"
mcp = "^1.4.1"
# garak = { version = "*", optional = true }
pytest-xdist = "3.6.1"
+12
View File
@@ -0,0 +1,12 @@
import pytest
from agentic_security.mcp.client import run
@pytest.mark.asyncio
async def test_mcp_echo_tool():
"""Test the echo tool functionality"""
prompts, resources, tools = await run()
assert prompts
assert resources
assert tools
+3 -3
View File
@@ -6891,9 +6891,9 @@
}
},
"node_modules/http-proxy-middleware": {
"version": "2.0.7",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.7.tgz",
"integrity": "sha512-fgVY8AV7qU7z/MmXJ/rxwbrtQH4jBQ9m7kp3llF0liB7glmFeVZFBepQb32T3y8n8k2+AEYuMPCpinYW+/CuRA==",
"version": "2.0.9",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==",
"dev": true,
"license": "MIT",
"dependencies": {