mirror of
https://github.com/FuzzingLabs/fuzzforge_ai.git
synced 2026-02-12 17:12:46 +00:00
feat: make Docker the default container engine
This commit is contained in:
3
.gitignore
vendored
3
.gitignore
vendored
@@ -7,3 +7,6 @@
|
||||
.venv
|
||||
.vscode
|
||||
__pycache__
|
||||
|
||||
# Podman/Docker container storage artifacts
|
||||
~/.fuzzforge/
|
||||
|
||||
22
Makefile
22
Makefile
@@ -65,25 +65,29 @@ test:
|
||||
done
|
||||
|
||||
# Build all module container images
|
||||
# Under Snap: uses self-contained storage at ~/.fuzzforge/containers/
|
||||
# Otherwise: uses default podman storage (works on native Linux + macOS)
|
||||
# Uses Docker by default, or Podman if FUZZFORGE_ENGINE=podman
|
||||
build-modules:
|
||||
@echo "Building FuzzForge module images..."
|
||||
@if [ -n "$$SNAP" ]; then \
|
||||
echo "Detected Snap environment - using isolated storage at ~/.fuzzforge/containers/"; \
|
||||
PODMAN_CMD="podman --root ~/.fuzzforge/containers/storage --runroot ~/.fuzzforge/containers/run"; \
|
||||
@if [ "$$FUZZFORGE_ENGINE" = "podman" ]; then \
|
||||
if [ -n "$$SNAP" ]; then \
|
||||
echo "Using Podman with isolated storage (Snap detected)"; \
|
||||
CONTAINER_CMD="podman --root ~/.fuzzforge/containers/storage --runroot ~/.fuzzforge/containers/run"; \
|
||||
else \
|
||||
echo "Using default podman storage"; \
|
||||
PODMAN_CMD="podman"; \
|
||||
echo "Using Podman"; \
|
||||
CONTAINER_CMD="podman"; \
|
||||
fi; \
|
||||
else \
|
||||
echo "Using Docker"; \
|
||||
CONTAINER_CMD="docker"; \
|
||||
fi; \
|
||||
for module in fuzzforge-modules/*/; do \
|
||||
if [ -f "$$module/Dockerfile" ] && \
|
||||
[ "$$module" != "fuzzforge-modules/fuzzforge-modules-sdk/" ] && \
|
||||
[ "$$module" != "fuzzforge-modules/fuzzforge-module-template/" ]; then \
|
||||
name=$$(basename $$module); \
|
||||
version=$$(grep 'version' "$$module/pyproject.toml" 2>/dev/null | head -1 | sed 's/.*"\(.*\)".*/\1/' || echo "0.1.0"); \
|
||||
version=$$(grep 'version' "$$module/pyproject.toml" 2>/dev/null | head -1 | sed 's/.*"\(.*\\)".*/\\1/' || echo "0.1.0"); \
|
||||
echo "Building $$name:$$version..."; \
|
||||
$$PODMAN_CMD build -t "fuzzforge-$$name:$$version" "$$module" || exit 1; \
|
||||
$$CONTAINER_CMD build -t "fuzzforge-$$name:$$version" "$$module" || exit 1; \
|
||||
fi \
|
||||
done
|
||||
@echo ""
|
||||
|
||||
10
README.md
10
README.md
@@ -83,7 +83,7 @@ If you find FuzzForge useful, please **star the repo** to support development!
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| 🤖 **AI-Native** | Built for MCP - works with GitHub Copilot, Claude, and any MCP-compatible agent |
|
||||
| 📦 **Containerized** | Each module runs in isolation via Podman or Docker |
|
||||
| 📦 **Containerized** | Each module runs in isolation via Docker or Podman |
|
||||
| 🔄 **Continuous Mode** | Long-running tasks (fuzzing) with real-time metrics streaming |
|
||||
| 🔗 **Workflows** | Chain multiple modules together in automated pipelines |
|
||||
| 🛠️ **Extensible** | Create custom modules with the Python SDK |
|
||||
@@ -110,7 +110,7 @@ If you find FuzzForge useful, please **star the repo** to support development!
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ FuzzForge Runner │
|
||||
│ Container Engine (Podman/Docker) │
|
||||
│ Container Engine (Docker/Podman) │
|
||||
└───────────────────────────┬─────────────────────────────────────┘
|
||||
│
|
||||
┌───────────────────┼───────────────────┐
|
||||
@@ -129,7 +129,7 @@ If you find FuzzForge useful, please **star the repo** to support development!
|
||||
|
||||
- **Python 3.12+**
|
||||
- **[uv](https://docs.astral.sh/uv/)** package manager
|
||||
- **Podman** (recommended) or Docker
|
||||
- **Docker** ([Install Docker](https://docs.docker.com/get-docker/)) or Podman
|
||||
|
||||
### Quick Install
|
||||
|
||||
@@ -141,8 +141,8 @@ cd fuzzforge-oss
|
||||
# Install dependencies
|
||||
uv sync
|
||||
|
||||
# Start Podman socket (Linux)
|
||||
systemctl --user start podman.socket
|
||||
# Build module images
|
||||
make build-modules
|
||||
```
|
||||
|
||||
### Configure MCP for Your AI Agent
|
||||
|
||||
82
USAGE.md
82
USAGE.md
@@ -26,7 +26,7 @@ This guide covers everything you need to know to get started with FuzzForge OSS
|
||||
|
||||
## Quick Start
|
||||
|
||||
> **Prerequisites:** You need [uv](https://docs.astral.sh/uv/) and [Podman](https://podman.io/) installed.
|
||||
> **Prerequisites:** You need [uv](https://docs.astral.sh/uv/) and [Docker](https://docs.docker.com/get-docker/) installed.
|
||||
> See the [Prerequisites](#prerequisites) section for installation instructions.
|
||||
|
||||
```bash
|
||||
@@ -51,8 +51,7 @@ uv run fuzzforge mcp install claude-code # For Claude Code CLI
|
||||
# "Start fuzzing the parse_input function"
|
||||
```
|
||||
|
||||
> **Note:** FuzzForge uses self-contained container storage (`~/.fuzzforge/containers/`)
|
||||
> which works automatically - no need to configure Podman sockets manually.
|
||||
> **Note:** FuzzForge uses Docker by default. Podman is also supported via `--engine podman`.
|
||||
|
||||
---
|
||||
|
||||
@@ -62,7 +61,7 @@ Before installing FuzzForge OSS, ensure you have:
|
||||
|
||||
- **Python 3.12+** - [Download Python](https://www.python.org/downloads/)
|
||||
- **uv** package manager - [Install uv](https://docs.astral.sh/uv/)
|
||||
- **Podman** - Container runtime (Docker also works but Podman is recommended)
|
||||
- **Docker** - Container runtime ([Install Docker](https://docs.docker.com/get-docker/))
|
||||
|
||||
### Installing uv
|
||||
|
||||
@@ -74,19 +73,21 @@ curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
pip install uv
|
||||
```
|
||||
|
||||
### Installing Podman (Linux)
|
||||
### Installing Docker
|
||||
|
||||
```bash
|
||||
# Ubuntu/Debian
|
||||
sudo apt update && sudo apt install -y podman
|
||||
# Linux (Ubuntu/Debian)
|
||||
curl -fsSL https://get.docker.com | sh
|
||||
sudo usermod -aG docker $USER
|
||||
# Log out and back in for group changes to take effect
|
||||
|
||||
# Fedora/RHEL
|
||||
sudo dnf install -y podman
|
||||
|
||||
# Arch Linux
|
||||
sudo pacman -S podman
|
||||
# macOS/Windows
|
||||
# Install Docker Desktop from https://docs.docker.com/get-docker/
|
||||
```
|
||||
|
||||
> **Note:** Podman is also supported. Use `--engine podman` with CLI commands
|
||||
> or set `FUZZFORGE_ENGINE=podman` environment variable.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
@@ -143,7 +144,7 @@ make build
|
||||
|
||||
```bash
|
||||
# List built module images
|
||||
podman images | grep fuzzforge
|
||||
docker images | grep fuzzforge
|
||||
```
|
||||
|
||||
You should see something like:
|
||||
@@ -169,13 +170,13 @@ uv run fuzzforge mcp install copilot
|
||||
The command auto-detects everything:
|
||||
- **FuzzForge root** - Where FuzzForge is installed
|
||||
- **Modules path** - Defaults to `fuzzforge-oss/fuzzforge-modules`
|
||||
- **Podman socket** - Auto-detects `/run/user/<uid>/podman/podman.sock`
|
||||
- **Docker socket** - Auto-detects `/var/run/docker.sock`
|
||||
|
||||
**Optional overrides** (usually not needed):
|
||||
```bash
|
||||
uv run fuzzforge mcp install copilot \
|
||||
--modules /path/to/modules \
|
||||
--engine docker # if using Docker instead of Podman
|
||||
--engine podman # if using Podman instead of Docker
|
||||
```
|
||||
|
||||
**After installation:**
|
||||
@@ -346,8 +347,10 @@ Configure FuzzForge using environment variables:
|
||||
export FUZZFORGE_MODULES_PATH=/path/to/modules
|
||||
export FUZZFORGE_STORAGE_PATH=/path/to/storage
|
||||
|
||||
# Container engine (uses self-contained storage by default)
|
||||
export FUZZFORGE_ENGINE__TYPE=podman # or docker
|
||||
# Container engine (Docker is default)
|
||||
export FUZZFORGE_ENGINE__TYPE=docker # or podman
|
||||
|
||||
# Podman-specific settings (only needed if using Podman under Snap)
|
||||
export FUZZFORGE_ENGINE__GRAPHROOT=~/.fuzzforge/containers/storage
|
||||
export FUZZFORGE_ENGINE__RUNROOT=~/.fuzzforge/containers/run
|
||||
```
|
||||
@@ -356,39 +359,39 @@ export FUZZFORGE_ENGINE__RUNROOT=~/.fuzzforge/containers/run
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Podman Socket Not Found
|
||||
### Docker Not Running
|
||||
|
||||
```
|
||||
Error: Could not connect to Podman socket
|
||||
Error: Cannot connect to Docker daemon
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Start the Podman socket
|
||||
systemctl --user start podman.socket
|
||||
# Linux: Start Docker service
|
||||
sudo systemctl start docker
|
||||
|
||||
# Check the socket path
|
||||
echo /run/user/$(id -u)/podman/podman.sock
|
||||
# macOS/Windows: Start Docker Desktop application
|
||||
|
||||
# Verify Docker is running
|
||||
docker run --rm hello-world
|
||||
```
|
||||
|
||||
### Permission Denied on Socket
|
||||
### Permission Denied on Docker Socket
|
||||
|
||||
```
|
||||
Error: Permission denied connecting to Podman socket
|
||||
Error: Permission denied connecting to Docker socket
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Ensure Podman is installed and your user can run containers
|
||||
podman run --rm hello-world
|
||||
# Add your user to the docker group
|
||||
sudo usermod -aG docker $USER
|
||||
|
||||
# If using system socket, ensure correct permissions
|
||||
ls -la /run/user/$(id -u)/podman/
|
||||
# Log out and back in for changes to take effect
|
||||
# Then verify:
|
||||
docker run --rm hello-world
|
||||
```
|
||||
|
||||
> **Note:** FuzzForge OSS uses self-contained storage (`~/.fuzzforge/containers/`) by default,
|
||||
> which avoids most permission issues with the Podman socket.
|
||||
|
||||
### No Modules Found
|
||||
|
||||
```
|
||||
@@ -398,7 +401,7 @@ No modules found.
|
||||
**Solution:**
|
||||
1. Build the modules first: `make build-modules`
|
||||
2. Check the modules path: `uv run fuzzforge modules list`
|
||||
3. Verify images exist: `podman images | grep fuzzforge`
|
||||
3. Verify images exist: `docker images | grep fuzzforge`
|
||||
|
||||
### MCP Server Not Starting
|
||||
|
||||
@@ -414,7 +417,18 @@ Verify the configuration file path exists and contains valid JSON.
|
||||
```bash
|
||||
# Build module container manually to see errors
|
||||
cd fuzzforge-modules/<module-name>
|
||||
podman build -t <module-name> .
|
||||
docker build -t <module-name> .
|
||||
```
|
||||
|
||||
### Using Podman Instead of Docker
|
||||
|
||||
If you prefer Podman:
|
||||
```bash
|
||||
# Use --engine podman with CLI
|
||||
uv run fuzzforge mcp install copilot --engine podman
|
||||
|
||||
# Or set environment variable
|
||||
export FUZZFORGE_ENGINE=podman
|
||||
```
|
||||
|
||||
### Check Logs
|
||||
|
||||
@@ -51,7 +51,7 @@ def main(
|
||||
envvar="FUZZFORGE_ENGINE__TYPE",
|
||||
help="Container engine type (docker or podman).",
|
||||
),
|
||||
] = "podman",
|
||||
] = "docker",
|
||||
engine_socket: Annotated[
|
||||
str,
|
||||
Option(
|
||||
|
||||
@@ -277,9 +277,9 @@ def generate(
|
||||
Option(
|
||||
"--engine",
|
||||
"-e",
|
||||
help="Container engine (podman or docker).",
|
||||
help="Container engine (docker or podman).",
|
||||
),
|
||||
] = "podman",
|
||||
] = "docker",
|
||||
) -> None:
|
||||
"""Generate MCP configuration and print to stdout.
|
||||
|
||||
@@ -361,9 +361,9 @@ def install(
|
||||
Option(
|
||||
"--engine",
|
||||
"-e",
|
||||
help="Container engine (podman or docker).",
|
||||
help="Container engine (docker or podman).",
|
||||
),
|
||||
] = "podman",
|
||||
] = "docker",
|
||||
force: Annotated[
|
||||
bool,
|
||||
Option(
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Docker container engine implementation."""
|
||||
|
||||
from fuzzforge_common.sandboxes.engines.docker.cli import DockerCLI
|
||||
from fuzzforge_common.sandboxes.engines.docker.configuration import (
|
||||
DockerConfiguration,
|
||||
)
|
||||
@@ -7,5 +8,6 @@ from fuzzforge_common.sandboxes.engines.docker.engine import Docker
|
||||
|
||||
__all__ = [
|
||||
"Docker",
|
||||
"DockerCLI",
|
||||
"DockerConfiguration",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,406 @@
|
||||
"""Docker CLI engine.
|
||||
|
||||
This engine uses subprocess calls to the Docker CLI, providing a simple
|
||||
and portable interface that works on Linux, macOS, and Windows wherever
|
||||
Docker Desktop or Docker Engine is installed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path, PurePath
|
||||
from tempfile import NamedTemporaryFile
|
||||
from typing import TYPE_CHECKING, cast
|
||||
|
||||
from fuzzforge_common.exceptions import FuzzForgeError
|
||||
from fuzzforge_common.sandboxes.engines.base.engine import AbstractFuzzForgeSandboxEngine, ImageInfo
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from structlog.stdlib import BoundLogger
|
||||
|
||||
|
||||
def get_logger() -> BoundLogger:
|
||||
"""Get structured logger."""
|
||||
from structlog import get_logger # noqa: PLC0415 (required by temporal)
|
||||
|
||||
return cast("BoundLogger", get_logger())
|
||||
|
||||
|
||||
class DockerCLI(AbstractFuzzForgeSandboxEngine):
|
||||
"""Docker engine using CLI commands.
|
||||
|
||||
This implementation uses subprocess calls to the Docker CLI,
|
||||
providing a simple and portable interface that works wherever
|
||||
Docker is installed (Docker Desktop on macOS/Windows, Docker Engine on Linux).
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
"""Initialize the DockerCLI engine."""
|
||||
AbstractFuzzForgeSandboxEngine.__init__(self)
|
||||
|
||||
def _base_cmd(self) -> list[str]:
|
||||
"""Get base Docker command.
|
||||
|
||||
:returns: Base command list.
|
||||
|
||||
"""
|
||||
return ["docker"]
|
||||
|
||||
def _run(self, args: list[str], *, check: bool = True, capture: bool = True) -> subprocess.CompletedProcess:
|
||||
"""Run a Docker command.
|
||||
|
||||
:param args: Command arguments (without 'docker').
|
||||
:param check: Raise exception on non-zero exit.
|
||||
:param capture: Capture stdout/stderr.
|
||||
:returns: CompletedProcess result.
|
||||
|
||||
"""
|
||||
cmd = self._base_cmd() + args
|
||||
get_logger().debug("running docker command", cmd=" ".join(cmd))
|
||||
return subprocess.run(
|
||||
cmd,
|
||||
check=check,
|
||||
capture_output=capture,
|
||||
text=True,
|
||||
)
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Image Operations
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
def list_images(self, filter_prefix: str | None = None) -> list[ImageInfo]:
|
||||
"""List available container images.
|
||||
|
||||
:param filter_prefix: Optional prefix to filter images.
|
||||
:returns: List of ImageInfo objects.
|
||||
|
||||
"""
|
||||
result = self._run(["images", "--format", "json"])
|
||||
images: list[ImageInfo] = []
|
||||
|
||||
try:
|
||||
# Docker outputs one JSON object per line
|
||||
lines = result.stdout.strip().split("\n") if result.stdout.strip() else []
|
||||
data = [json.loads(line) for line in lines if line.strip()]
|
||||
except json.JSONDecodeError:
|
||||
get_logger().warning("failed to parse docker images output")
|
||||
return images
|
||||
|
||||
for image in data:
|
||||
repo = image.get("Repository", "")
|
||||
tag = image.get("Tag", "latest")
|
||||
|
||||
if repo == "<none>":
|
||||
continue
|
||||
|
||||
reference = f"{repo}:{tag}"
|
||||
|
||||
if filter_prefix and not reference.startswith(filter_prefix):
|
||||
continue
|
||||
|
||||
images.append(
|
||||
ImageInfo(
|
||||
reference=reference,
|
||||
repository=repo,
|
||||
tag=tag,
|
||||
image_id=image.get("ID", "")[:12],
|
||||
size=image.get("Size"),
|
||||
)
|
||||
)
|
||||
|
||||
get_logger().debug("listed images", count=len(images), filter_prefix=filter_prefix)
|
||||
return images
|
||||
|
||||
def image_exists(self, image: str) -> bool:
|
||||
"""Check if a container image exists locally.
|
||||
|
||||
:param image: Full image reference.
|
||||
:returns: True if image exists.
|
||||
|
||||
"""
|
||||
result = self._run(["image", "inspect", image], check=False)
|
||||
return result.returncode == 0
|
||||
|
||||
def pull_image(self, image: str, timeout: int = 300) -> None:
|
||||
"""Pull an image from a container registry.
|
||||
|
||||
:param image: Full image reference.
|
||||
:param timeout: Timeout in seconds.
|
||||
|
||||
"""
|
||||
get_logger().info("pulling image", image=image)
|
||||
try:
|
||||
self._run(["pull", image])
|
||||
get_logger().info("image pulled successfully", image=image)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
message = f"Failed to pull image '{image}': {exc.stderr}"
|
||||
raise FuzzForgeError(message) from exc
|
||||
|
||||
def tag_image(self, source: str, target: str) -> None:
|
||||
"""Tag an image with a new name.
|
||||
|
||||
:param source: Source image reference.
|
||||
:param target: Target image reference.
|
||||
|
||||
"""
|
||||
self._run(["tag", source, target])
|
||||
get_logger().debug("tagged image", source=source, target=target)
|
||||
|
||||
def build_image(self, context_path: Path, tag: str, dockerfile: str = "Dockerfile") -> None:
|
||||
"""Build an image from a Dockerfile.
|
||||
|
||||
:param context_path: Path to build context.
|
||||
:param tag: Image tag.
|
||||
:param dockerfile: Dockerfile name.
|
||||
|
||||
"""
|
||||
get_logger().info("building image", tag=tag, context=str(context_path))
|
||||
self._run(["build", "-t", tag, "-f", dockerfile, str(context_path)])
|
||||
get_logger().info("image built successfully", tag=tag)
|
||||
|
||||
def register_archive(self, archive: Path, repository: str) -> None:
|
||||
"""Load an image from a tar archive.
|
||||
|
||||
:param archive: Path to tar archive.
|
||||
:param repository: Repository name for the loaded image.
|
||||
|
||||
"""
|
||||
result = self._run(["load", "-i", str(archive)])
|
||||
# Tag the loaded image
|
||||
for line in result.stdout.splitlines():
|
||||
if "Loaded image:" in line:
|
||||
loaded_image = line.split("Loaded image:")[-1].strip()
|
||||
self._run(["tag", loaded_image, f"{repository}:latest"])
|
||||
break
|
||||
get_logger().debug("registered archive", archive=str(archive), repository=repository)
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Container Operations
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
def spawn_sandbox(self, image: str) -> str:
|
||||
"""Spawn a sandbox (container) from an image.
|
||||
|
||||
:param image: Image to create container from.
|
||||
:returns: Container identifier.
|
||||
|
||||
"""
|
||||
result = self._run(["create", image])
|
||||
container_id = result.stdout.strip()
|
||||
get_logger().debug("created container", container_id=container_id)
|
||||
return container_id
|
||||
|
||||
def create_container(
|
||||
self,
|
||||
image: str,
|
||||
volumes: dict[str, str] | None = None,
|
||||
) -> str:
|
||||
"""Create a container from an image.
|
||||
|
||||
:param image: Image to create container from.
|
||||
:param volumes: Optional volume mappings {host_path: container_path}.
|
||||
:returns: Container identifier.
|
||||
|
||||
"""
|
||||
args = ["create"]
|
||||
if volumes:
|
||||
for host_path, container_path in volumes.items():
|
||||
args.extend(["-v", f"{host_path}:{container_path}:ro"])
|
||||
args.append(image)
|
||||
|
||||
result = self._run(args)
|
||||
container_id = result.stdout.strip()
|
||||
get_logger().debug("created container", container_id=container_id, image=image)
|
||||
return container_id
|
||||
|
||||
def start_sandbox(self, identifier: str) -> None:
|
||||
"""Start a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
|
||||
"""
|
||||
self._run(["start", identifier])
|
||||
get_logger().debug("started container", container_id=identifier)
|
||||
|
||||
def start_container(self, identifier: str) -> None:
|
||||
"""Start a container without waiting.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
|
||||
"""
|
||||
self._run(["start", identifier])
|
||||
get_logger().debug("started container (detached)", container_id=identifier)
|
||||
|
||||
def start_container_attached(
|
||||
self,
|
||||
identifier: str,
|
||||
timeout: int = 600,
|
||||
) -> tuple[int, str, str]:
|
||||
"""Start a container and wait for completion.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param timeout: Timeout in seconds.
|
||||
:returns: Tuple of (exit_code, stdout, stderr).
|
||||
|
||||
"""
|
||||
get_logger().debug("starting container attached", container_id=identifier)
|
||||
# Start the container
|
||||
self._run(["start", identifier])
|
||||
|
||||
# Wait for completion
|
||||
wait_result = self._run(["wait", identifier])
|
||||
exit_code = int(wait_result.stdout.strip()) if wait_result.stdout.strip() else -1
|
||||
|
||||
# Get logs
|
||||
stdout_result = self._run(["logs", identifier], check=False)
|
||||
stdout_str = stdout_result.stdout or ""
|
||||
stderr_str = stdout_result.stderr or ""
|
||||
|
||||
get_logger().debug("container finished", container_id=identifier, exit_code=exit_code)
|
||||
return (exit_code, stdout_str, stderr_str)
|
||||
|
||||
def execute_inside_sandbox(self, identifier: str, command: list[str]) -> None:
|
||||
"""Execute a command inside a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param command: Command to run.
|
||||
|
||||
"""
|
||||
get_logger().debug("executing command in container", container_id=identifier)
|
||||
self._run(["exec", identifier] + command)
|
||||
|
||||
def push_archive_to_sandbox(self, identifier: str, source: Path, destination: PurePath) -> None:
|
||||
"""Copy an archive to a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param source: Source archive path.
|
||||
:param destination: Destination path in container.
|
||||
|
||||
"""
|
||||
get_logger().debug("copying to container", container_id=identifier, source=str(source))
|
||||
self._run(["cp", str(source), f"{identifier}:{destination}"])
|
||||
|
||||
def pull_archive_from_sandbox(self, identifier: str, source: PurePath) -> Path:
|
||||
"""Copy files from a container to a local archive.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param source: Source path in container.
|
||||
:returns: Path to local archive.
|
||||
|
||||
"""
|
||||
get_logger().debug("copying from container", container_id=identifier, source=str(source))
|
||||
with NamedTemporaryFile(delete=False, delete_on_close=False, suffix=".tar") as tmp:
|
||||
self._run(["cp", f"{identifier}:{source}", tmp.name])
|
||||
return Path(tmp.name)
|
||||
|
||||
def copy_to_container(self, identifier: str, source: Path, destination: str) -> None:
|
||||
"""Copy a file or directory to a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param source: Source path on host.
|
||||
:param destination: Destination path in container.
|
||||
|
||||
"""
|
||||
self._run(["cp", str(source), f"{identifier}:{destination}"])
|
||||
get_logger().debug("copied to container", source=str(source), destination=destination)
|
||||
|
||||
def copy_from_container(self, identifier: str, source: str, destination: Path) -> None:
|
||||
"""Copy a file or directory from a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param source: Source path in container.
|
||||
:param destination: Destination path on host.
|
||||
|
||||
"""
|
||||
destination.mkdir(parents=True, exist_ok=True)
|
||||
self._run(["cp", f"{identifier}:{source}", str(destination)])
|
||||
get_logger().debug("copied from container", source=source, destination=str(destination))
|
||||
|
||||
def terminate_sandbox(self, identifier: str) -> None:
|
||||
"""Terminate and remove a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
|
||||
"""
|
||||
# Stop if running
|
||||
self._run(["stop", identifier], check=False)
|
||||
# Remove
|
||||
self._run(["rm", "-f", identifier], check=False)
|
||||
get_logger().debug("terminated container", container_id=identifier)
|
||||
|
||||
def remove_container(self, identifier: str, *, force: bool = False) -> None:
|
||||
"""Remove a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param force: Force removal.
|
||||
|
||||
"""
|
||||
args = ["rm"]
|
||||
if force:
|
||||
args.append("-f")
|
||||
args.append(identifier)
|
||||
self._run(args, check=False)
|
||||
get_logger().debug("removed container", container_id=identifier)
|
||||
|
||||
def stop_container(self, identifier: str, timeout: int = 10) -> None:
|
||||
"""Stop a running container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param timeout: Seconds to wait before killing.
|
||||
|
||||
"""
|
||||
self._run(["stop", "-t", str(timeout), identifier], check=False)
|
||||
get_logger().debug("stopped container", container_id=identifier)
|
||||
|
||||
def get_container_status(self, identifier: str) -> str:
|
||||
"""Get the status of a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:returns: Container status.
|
||||
|
||||
"""
|
||||
result = self._run(["inspect", "--format", "{{.State.Status}}", identifier], check=False)
|
||||
return result.stdout.strip() if result.returncode == 0 else "unknown"
|
||||
|
||||
def read_file_from_container(self, identifier: str, path: str) -> str:
|
||||
"""Read a file from inside a container.
|
||||
|
||||
:param identifier: Container identifier.
|
||||
:param path: Path to file in container.
|
||||
:returns: File contents.
|
||||
|
||||
"""
|
||||
result = self._run(["exec", identifier, "cat", path], check=False)
|
||||
if result.returncode != 0:
|
||||
get_logger().debug("failed to read file from container", path=path)
|
||||
return ""
|
||||
return result.stdout
|
||||
|
||||
def list_containers(self, all_containers: bool = True) -> list[dict]:
|
||||
"""List containers.
|
||||
|
||||
:param all_containers: Include stopped containers.
|
||||
:returns: List of container info dicts.
|
||||
|
||||
"""
|
||||
args = ["ps", "--format", "json"]
|
||||
if all_containers:
|
||||
args.append("-a")
|
||||
|
||||
result = self._run(args)
|
||||
try:
|
||||
# Docker outputs one JSON object per line
|
||||
lines = result.stdout.strip().split("\n") if result.stdout.strip() else []
|
||||
data = [json.loads(line) for line in lines if line.strip()]
|
||||
return [
|
||||
{
|
||||
"Id": c.get("ID", ""),
|
||||
"Names": c.get("Names", ""),
|
||||
"Status": c.get("State", ""),
|
||||
"Image": c.get("Image", ""),
|
||||
}
|
||||
for c in data
|
||||
]
|
||||
except json.JSONDecodeError:
|
||||
return []
|
||||
81
fuzzforge-common/tests/unit/engines/test_docker.py
Normal file
81
fuzzforge-common/tests/unit/engines/test_docker.py
Normal file
@@ -0,0 +1,81 @@
|
||||
"""Tests for the DockerCLI engine."""
|
||||
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
from fuzzforge_common.sandboxes.engines.docker.cli import DockerCLI
|
||||
|
||||
|
||||
def test_docker_cli_base_cmd() -> None:
|
||||
"""Test that base command is just 'docker'."""
|
||||
engine = DockerCLI()
|
||||
base_cmd = engine._base_cmd()
|
||||
|
||||
assert base_cmd == ["docker"]
|
||||
|
||||
|
||||
def test_docker_cli_list_images_returns_list() -> None:
|
||||
"""Test that list_images returns a list (mocked)."""
|
||||
engine = DockerCLI()
|
||||
|
||||
# Mock the _run method to return empty JSON
|
||||
with mock.patch.object(engine, "_run") as mock_run:
|
||||
mock_run.return_value = mock.Mock(stdout="", returncode=0)
|
||||
images = engine.list_images()
|
||||
|
||||
assert isinstance(images, list)
|
||||
assert len(images) == 0
|
||||
|
||||
|
||||
def test_docker_cli_list_images_parses_output() -> None:
|
||||
"""Test that list_images correctly parses Docker JSON output."""
|
||||
engine = DockerCLI()
|
||||
|
||||
# Docker outputs one JSON object per line
|
||||
docker_output = '{"Repository":"alpine","Tag":"latest","ID":"abc123","Size":"5MB"}\n{"Repository":"ubuntu","Tag":"22.04","ID":"def456","Size":"77MB"}'
|
||||
|
||||
with mock.patch.object(engine, "_run") as mock_run:
|
||||
mock_run.return_value = mock.Mock(stdout=docker_output, returncode=0)
|
||||
images = engine.list_images()
|
||||
|
||||
assert len(images) == 2
|
||||
assert images[0].repository == "alpine"
|
||||
assert images[0].tag == "latest"
|
||||
assert images[1].repository == "ubuntu"
|
||||
assert images[1].tag == "22.04"
|
||||
|
||||
|
||||
def test_docker_cli_image_exists_mocked() -> None:
|
||||
"""Test image_exists with mocked response."""
|
||||
engine = DockerCLI()
|
||||
|
||||
with mock.patch.object(engine, "_run") as mock_run:
|
||||
# Image exists
|
||||
mock_run.return_value = mock.Mock(returncode=0)
|
||||
assert engine.image_exists("alpine:latest") is True
|
||||
|
||||
# Image doesn't exist
|
||||
mock_run.return_value = mock.Mock(returncode=1)
|
||||
assert engine.image_exists("nonexistent:image") is False
|
||||
|
||||
|
||||
def test_docker_cli_create_container_with_volumes() -> None:
|
||||
"""Test create_container generates correct command with volumes."""
|
||||
engine = DockerCLI()
|
||||
|
||||
with mock.patch.object(engine, "_run") as mock_run:
|
||||
mock_run.return_value = mock.Mock(stdout="container123\n", returncode=0)
|
||||
|
||||
container_id = engine.create_container(
|
||||
"alpine:latest",
|
||||
volumes={"/host/path": "/container/path"}
|
||||
)
|
||||
|
||||
# Check the command was called with volume flag
|
||||
call_args = mock_run.call_args[0][0]
|
||||
assert "create" in call_args
|
||||
assert "-v" in call_args
|
||||
assert "/host/path:/container/path:ro" in call_args
|
||||
assert "alpine:latest" in call_args
|
||||
assert container_id == "container123"
|
||||
@@ -50,9 +50,7 @@ For custom setups, you can manually configure the MCP server.
|
||||
"cwd": "/path/to/fuzzforge-oss",
|
||||
"env": {
|
||||
"FUZZFORGE_MODULES_PATH": "/path/to/fuzzforge-oss/fuzzforge-modules",
|
||||
"FUZZFORGE_ENGINE__TYPE": "podman",
|
||||
"FUZZFORGE_ENGINE__GRAPHROOT": "~/.fuzzforge/containers/storage",
|
||||
"FUZZFORGE_ENGINE__RUNROOT": "~/.fuzzforge/containers/run"
|
||||
"FUZZFORGE_ENGINE__TYPE": "docker"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -71,9 +69,7 @@ For custom setups, you can manually configure the MCP server.
|
||||
"cwd": "/path/to/fuzzforge-oss",
|
||||
"env": {
|
||||
"FUZZFORGE_MODULES_PATH": "/path/to/fuzzforge-oss/fuzzforge-modules",
|
||||
"FUZZFORGE_ENGINE__TYPE": "podman",
|
||||
"FUZZFORGE_ENGINE__GRAPHROOT": "~/.fuzzforge/containers/storage",
|
||||
"FUZZFORGE_ENGINE__RUNROOT": "~/.fuzzforge/containers/run"
|
||||
"FUZZFORGE_ENGINE__TYPE": "docker"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -92,9 +88,7 @@ For custom setups, you can manually configure the MCP server.
|
||||
"cwd": "/path/to/fuzzforge-oss",
|
||||
"env": {
|
||||
"FUZZFORGE_MODULES_PATH": "/path/to/fuzzforge-oss/fuzzforge-modules",
|
||||
"FUZZFORGE_ENGINE__TYPE": "podman",
|
||||
"FUZZFORGE_ENGINE__GRAPHROOT": "~/.fuzzforge/containers/storage",
|
||||
"FUZZFORGE_ENGINE__RUNROOT": "~/.fuzzforge/containers/run"
|
||||
"FUZZFORGE_ENGINE__TYPE": "docker"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -106,9 +100,9 @@ For custom setups, you can manually configure the MCP server.
|
||||
| Variable | Required | Default | Description |
|
||||
| -------- | -------- | ------- | ----------- |
|
||||
| `FUZZFORGE_MODULES_PATH` | Yes | - | Path to the modules directory |
|
||||
| `FUZZFORGE_ENGINE__TYPE` | No | `podman` | Container engine (`podman` or `docker`) |
|
||||
| `FUZZFORGE_ENGINE__GRAPHROOT` | No | `~/.fuzzforge/containers/storage` | Container image storage path |
|
||||
| `FUZZFORGE_ENGINE__RUNROOT` | No | `~/.fuzzforge/containers/run` | Container runtime state path |
|
||||
| `FUZZFORGE_ENGINE__TYPE` | No | `docker` | Container engine (`docker` or `podman`) |
|
||||
| `FUZZFORGE_ENGINE__GRAPHROOT` | No | - | Container storage path (Podman under Snap only) |
|
||||
| `FUZZFORGE_ENGINE__RUNROOT` | No | - | Container runtime state path (Podman under Snap only) |
|
||||
|
||||
## Available Tools
|
||||
|
||||
|
||||
@@ -39,12 +39,13 @@ async def list_modules() -> dict[str, Any]:
|
||||
settings = get_settings()
|
||||
|
||||
# Use the engine abstraction to list images
|
||||
modules = runner.list_module_images(filter_prefix="localhost/")
|
||||
# Default filter matches locally-built fuzzforge-* modules
|
||||
modules = runner.list_module_images(filter_prefix="fuzzforge-")
|
||||
|
||||
available_modules = [
|
||||
{
|
||||
"identifier": module.identifier,
|
||||
"image": f"localhost/{module.identifier}:{module.version or 'latest'}",
|
||||
"image": f"{module.identifier}:{module.version or 'latest'}",
|
||||
"available": module.available,
|
||||
}
|
||||
for module in modules
|
||||
|
||||
@@ -100,24 +100,24 @@ class ModuleExecutor:
|
||||
def _get_engine(self) -> AbstractFuzzForgeSandboxEngine:
|
||||
"""Get the container engine instance.
|
||||
|
||||
Uses PodmanCLI with custom storage paths by default for Podman,
|
||||
providing isolation from system Podman configuration and avoiding
|
||||
issues with VS Code snap's XDG_DATA_HOME override.
|
||||
Uses DockerCLI by default for simplicity (works on Linux, macOS, Windows).
|
||||
PodmanCLI is used when engine type is set to 'podman'.
|
||||
|
||||
:returns: Configured container engine.
|
||||
|
||||
"""
|
||||
from fuzzforge_common.sandboxes.engines.docker import DockerCLI
|
||||
from fuzzforge_common.sandboxes.engines.podman import PodmanCLI
|
||||
|
||||
# Use PodmanCLI with custom storage paths for Podman
|
||||
# Use PodmanCLI for Podman (with custom storage under Snap)
|
||||
if self._engine_settings.type == "podman":
|
||||
return PodmanCLI(
|
||||
graphroot=self._engine_settings.graphroot,
|
||||
runroot=self._engine_settings.runroot,
|
||||
)
|
||||
|
||||
# Fall back to socket-based engine for Docker
|
||||
return self._get_engine_configuration().into_engine()
|
||||
# Use DockerCLI for Docker (default)
|
||||
return DockerCLI()
|
||||
|
||||
def _check_image_exists(self, module_identifier: str) -> bool:
|
||||
"""Check if a module image exists locally.
|
||||
@@ -128,12 +128,26 @@ class ModuleExecutor:
|
||||
"""
|
||||
engine = self._get_engine()
|
||||
|
||||
# Try both common tags: latest and 0.0.1
|
||||
tags_to_check = ["latest", "0.0.1"]
|
||||
# Try common tags
|
||||
tags_to_check = ["latest", "0.1.0", "0.0.1"]
|
||||
|
||||
# Try both naming conventions:
|
||||
# Try multiple naming conventions:
|
||||
# - fuzzforge-{name}:{tag} (OSS local builds)
|
||||
# - fuzzforge-module-{name}:{tag} (OSS local builds with module prefix)
|
||||
# - localhost/fuzzforge-module-{name}:{tag} (standard convention)
|
||||
# - localhost/{name}:{tag} (legacy/short form)
|
||||
|
||||
# For OSS local builds (no localhost/ prefix)
|
||||
for tag in tags_to_check:
|
||||
# Check direct module name (fuzzforge-cargo-fuzzer:0.1.0)
|
||||
if engine.image_exists(f"{module_identifier}:{tag}"):
|
||||
return True
|
||||
# Check with fuzzforge- prefix if not already present
|
||||
if not module_identifier.startswith("fuzzforge-"):
|
||||
if engine.image_exists(f"fuzzforge-{module_identifier}:{tag}"):
|
||||
return True
|
||||
|
||||
# For registry-style naming (localhost/ prefix)
|
||||
name_prefixes = [f"fuzzforge-module-{module_identifier}", module_identifier]
|
||||
|
||||
for prefix in name_prefixes:
|
||||
@@ -148,17 +162,40 @@ class ModuleExecutor:
|
||||
"""Get the full local image name for a module.
|
||||
|
||||
:param module_identifier: Name/identifier of the module.
|
||||
:returns: Full image name with localhost prefix.
|
||||
:returns: Full image name (may or may not have localhost prefix).
|
||||
|
||||
"""
|
||||
engine = self._get_engine()
|
||||
|
||||
# Check fuzzforge-module- prefix first (standard convention)
|
||||
prefixed_name = f"localhost/fuzzforge-module-{module_identifier}:latest"
|
||||
# Try common tags
|
||||
tags_to_check = ["latest", "0.1.0", "0.0.1"]
|
||||
|
||||
# Check OSS local builds first (no localhost/ prefix)
|
||||
for tag in tags_to_check:
|
||||
# Direct module name (fuzzforge-cargo-fuzzer:0.1.0)
|
||||
direct_name = f"{module_identifier}:{tag}"
|
||||
if engine.image_exists(direct_name):
|
||||
return direct_name
|
||||
|
||||
# With fuzzforge- prefix if not already present
|
||||
if not module_identifier.startswith("fuzzforge-"):
|
||||
prefixed_name = f"fuzzforge-{module_identifier}:{tag}"
|
||||
if engine.image_exists(prefixed_name):
|
||||
return prefixed_name
|
||||
|
||||
# Fall back to legacy short form
|
||||
# Check registry-style naming (localhost/ prefix)
|
||||
for tag in tags_to_check:
|
||||
# Standard convention: localhost/fuzzforge-module-{name}:{tag}
|
||||
prefixed_name = f"localhost/fuzzforge-module-{module_identifier}:{tag}"
|
||||
if engine.image_exists(prefixed_name):
|
||||
return prefixed_name
|
||||
|
||||
# Legacy short form: localhost/{name}:{tag}
|
||||
short_name = f"localhost/{module_identifier}:{tag}"
|
||||
if engine.image_exists(short_name):
|
||||
return short_name
|
||||
|
||||
# Default fallback
|
||||
return f"localhost/{module_identifier}:latest"
|
||||
|
||||
def _pull_module_image(self, module_identifier: str, registry_url: str = "ghcr.io/fuzzinglabs", tag: str = "latest") -> None:
|
||||
|
||||
@@ -174,12 +174,17 @@ class Runner:
|
||||
logger.info("discovered modules", count=len(modules))
|
||||
return modules
|
||||
|
||||
def list_module_images(self, filter_prefix: str = "localhost/") -> list[ModuleInfo]:
|
||||
def list_module_images(
|
||||
self,
|
||||
filter_prefix: str = "fuzzforge-",
|
||||
include_all_tags: bool = True,
|
||||
) -> list[ModuleInfo]:
|
||||
"""List available module images from the container engine.
|
||||
|
||||
Uses the container engine API to discover built module images.
|
||||
|
||||
:param filter_prefix: Prefix to filter images (default: "localhost/").
|
||||
:param filter_prefix: Prefix to filter images (default: "fuzzforge-").
|
||||
:param include_all_tags: If True, include all image tags, not just 'latest'.
|
||||
:returns: List of available module images.
|
||||
|
||||
"""
|
||||
@@ -194,8 +199,8 @@ class Runner:
|
||||
images = engine.list_images(filter_prefix=filter_prefix)
|
||||
|
||||
for image in images:
|
||||
# Only include :latest images
|
||||
if image.tag != "latest":
|
||||
# Only include :latest images unless include_all_tags is set
|
||||
if not include_all_tags and image.tag != "latest":
|
||||
continue
|
||||
|
||||
# Extract module name from repository
|
||||
|
||||
@@ -20,17 +20,16 @@ class EngineType(StrEnum):
|
||||
class EngineSettings(BaseModel):
|
||||
"""Container engine configuration."""
|
||||
|
||||
#: Type of container engine to use.
|
||||
type: EngineType = EngineType.PODMAN
|
||||
#: Type of container engine to use. Docker is the default for simplicity.
|
||||
type: EngineType = EngineType.DOCKER
|
||||
|
||||
#: Path to the container engine socket (only used as fallback).
|
||||
#: Path to the container engine socket (only used as fallback for socket-based engines).
|
||||
socket: str = Field(default="")
|
||||
|
||||
#: Custom graph root for container storage (isolated from system).
|
||||
#: When set, uses CLI mode instead of socket for better portability.
|
||||
#: Custom graph root for Podman storage (only used with Podman under Snap).
|
||||
graphroot: Path = Field(default=Path.home() / ".fuzzforge" / "containers" / "storage")
|
||||
|
||||
#: Custom run root for container runtime state.
|
||||
#: Custom run root for Podman runtime state (only used with Podman under Snap).
|
||||
runroot: Path = Field(default=Path.home() / ".fuzzforge" / "containers" / "run")
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user