CalvinBackup/fuzzforge_ai
1
0
Fork 0
mirror of https://github.com/FuzzingLabs/fuzzforge_ai.git synced 2026-05-24 05:34:03 +02:00
Code Issues Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Tanguy Duhamel
2025-09-29 21:26:41 +02:00
parent ecf8d49dde
commit 0547b78429
208 changed files with 72069 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/.gitignore
+20
View File
@@ -0,0 +1,20 @@
# Dependencies
/node_modules
# Production
/build
# Generated files
.docusaurus
.cache-loader
# Misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
npm-debug.log*
yarn-debug.log*
yarn-error.log*
docs/README.md
+25
View File
@@ -0,0 +1,25 @@
# FuzzForge Documentation
This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
## Installation
```bash
yarn
```
## Local Development
```bash
yarn start
```
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
## Build
```bash
yarn build
```
This command generates static content into the `build` directory and can be served using any static contents hosting service.
docs/docs/ai/a2a-services.md
+150
View File
@@ -0,0 +1,150 @@
# A2A Services
The FuzzForge AI module can expose itself as an Agent-to-Agent (A2A) server so downstream systems can register the agent, inspect its card, and call tools over HTTP.
## Starting the Server
```bash
fuzzforge ai server
```
Run the command from a project directory that already contains `.fuzzforge/`. The server reads the project configuration and reuses the same environment variables as the CLI shell.
**Default directories**
- Logs: `.fuzzforge/logs/cognee.log`
- Cognee datasets: `.fuzzforge/cognee/project_<id>/{data,system}`
- Artifact cache: `.fuzzforge/artifacts`
## HTTP Endpoints
| Method | Path | Purpose |
| --- | --- | --- |
| `GET` | `/artifacts/{id}` | Download artifacts created by the agent, workflows, or remote collaborators |
| `POST` | `/graph/query` | Query the Cognee project graph using `query`, optional `dataset`, and optional `search_type` |
| `POST` | `/project/files` | Mirror a file from the project workspace as a downloadable artifact |
### `POST /graph/query`
Request body:
- `query` *(str, required)* – Natural language question for the graph
- `search_type` *(str, optional)* – e.g. `GRAPH_COMPLETION`, `INSIGHTS`, `CHUNKS`
- `dataset` *(str, optional)* – Defaults to `<project>_codebase`
Example:
```bash
curl -s http://localhost:10100/graph/query \
-H 'Content-Type: application/json' \
-d '{"query":"unsafe Rust", "search_type":"GRAPH_COMPLETION"}' | jq
```
### `POST /project/files`
Registers a source file and returns an artifact descriptor.
```bash
curl -s http://localhost:10100/project/files \
-H 'Content-Type: application/json' \
-d '{"path":"src/lib.rs"}' | jq
```
Response excerpt:
```json
{
"id": "project_file_4325a8a6",
"file_uri": "http://127.0.0.1:10100/artifacts/project_file_4325a8a6",
"name": "src/lib.rs",
"mime_type": "text/x-c",
"size": 160
}
```
## Typical Collaboration Flow
1. Ingest project knowledge with `fuzzforge rag ingest --path . --recursive`.
2. Start the A2A server: `fuzzforge ai server`.
3. Downstream agents:
- Call `POST /graph/query` to explore project knowledge.
- Call `POST /project/files` to fetch raw files from the repository.
- Download finished scan summaries with `GET /artifacts/{id}`.
4. The AI module pushes Prefect workflow results into artifacts automatically, so remote agents can poll without re-running scans.
## Registration Flow
```mermaid
sequenceDiagram
participant Client as Remote Agent
participant HTTP as A2A HTTP Server
participant Exec as FuzzForgeExecutor
participant Registry as Agent Registry
Client->>HTTP: GET /.well-known/agent-card.json
HTTP-->>Client: Agent card (skills, protocol version)
Client->>HTTP: POST / (register)
HTTP->>Exec: Register request
Exec->>Registry: Persist remote agent metadata
Exec-->>HTTP: Confirmation + assigned agent ID
HTTP-->>Client: Success response
Client->>Exec: Subsequent messages routed via HTTP endpoints
Exec->>Registry: Update capability cache per message
```
### How registration works
1. **Discovery** – A remote agent fetches `/.well-known/agent-card.json` to confirm skills, protocol version, and message schemas.
2. **Handshake** – The remote agent issues `POST /` to start the A2A session. The payload includes its agent card and callback URL.
3. **Persistence** – `FuzzForgeExecutor` stores the remote agent in the registry (`agents.yaml` when run via the CLI). Auto-registration on future boots replays these entries.
4. **Capability cache** – Each inbound message updates the capability cache so the router can route `ROUTE_TO AgentName:` commands without another round-trip.
5. **Teardown** – Removing an agent via `/unregister` purges it from the registry; restart the server to drop any lingering connections.
### Where agent metadata lives
- The AI CLI and server share the same registry file. When a project is initialised, registrations are written to `.fuzzforge/agents.yaml` (see `ai/src/fuzzforge_ai/config_manager.py`).
- If the project file is absent, the executor falls back to the packaged default at `ai/src/fuzzforge_ai/config.yaml`.
- Each entry records `name`, `url`, and description. On startup `_auto_register_agents()` replays that list so both the CLI and the A2A server automatically reconnect to known peers.
- Editing `.fuzzforge/agents.yaml` manually is supported; the CLI `/register` and `/unregister` commands update it for you.
## Agent Card
The server exposes its agent card at `/.well-known/agent-card.json`. Clients can read that metadata to confirm skills, supported message schemas, and protocol version (`0.3.0`).
## Artifacts in A2A mode
- **Creation** – Conversations generate artifacts automatically when the executor produces code, reports, or workflow summaries. The `/artifacts` CLI command lists them; over HTTP they are addressed by `GET /artifacts/{id}`.
- **Distribution** – Use `/sendfile <agent> <path> [note]` in the CLI or call `POST /project/files` programmatically to turn a project file into an artifact that downstream agents can fetch.
- **Download** – Remote agents receive the artifact descriptor (including `file_uri`) in A2A responses or via polling. Retrieve the content with `GET /artifacts/{id}`; the cache lives under `.fuzzforge/artifacts/`.
- **Lifecycle** – Artifacts persist for the life of the project workspace. Clean the directory if you need to reclaim space; the executor recreates entries on demand.
## Running the server vs. CLI-only mode
- Launch the server with `fuzzforge ai server`. It loads `.fuzzforge/.env`, sets up Cognee directories via `ProjectConfigManager`, and exposes HTTP endpoints on `127.0.0.1:${FUZZFORGE_PORT:-10100}`.
- Without the server, the `fuzzforge ai agent` CLI still supports A2A-style routing for locally registered peers, but external agents cannot connect because the HTTP surface is absent.
- When the server is running, both the CLI and remote agents share the same executor, task store, and artifact cache. Stopping the server returns the module to CLI-only operation without altering persisted registrations.
## Communication Patterns
```mermaid
sequenceDiagram
participant Remote as Remote Agent
participant HTTP as A2A Server
participant Exec as Executor
participant Workflow as Prefect Backend
Remote->>HTTP: POST / (message with tool request)
HTTP->>Exec: Forward message
Exec->>Workflow: (optional) submit_security_scan_mcp
Workflow-->>Exec: Status / findings
Exec->>HTTP: Response + artifact metadata
HTTP-->>Remote: A2A response with artifacts/tasks
Remote->>HTTP: GET /artifacts/{id}
HTTP-->>Remote: Artifact bytes
```
This pattern repeats for subsequent tool invocations. Remote agents can also call the helper endpoints (`/graph/query`, `/project/files`) directly while the conversation is active.
## Related Files
- Runtime entry point: `ai/src/fuzzforge_ai/__main__.py`
- HTTP implementation: `ai/src/fuzzforge_ai/a2a_server.py`
- Agent metadata: `ai/src/fuzzforge_ai/agent_card.py`
docs/docs/ai/architecture.md
+146
View File
@@ -0,0 +1,146 @@
# AI Architecture
FuzzForge AI is the orchestration layer that lets large language models drive the broader security platform. Built on the Google ADK runtime, the module coordinates local tools, remote Agent-to-Agent (A2A) peers, and Prefect-backed workflows while persisting long-running context for every project.
## System Diagram
```mermaid
graph TB
subgraph Surfaces
CLI[CLI Shell]
HTTP[A2A HTTP Server]
end
CLI --> AgentCore
HTTP --> AgentCore
subgraph AgentCore [Agent Core]
AgentCoreNode[FuzzForgeAgent]
AgentCoreNode --> Executor[Executor]
AgentCoreNode --> Memory[Memory Services]
AgentCoreNode --> Registry[Agent Registry]
end
Executor --> MCP[MCP Workflow Bridge]
Executor --> Router[Capability Router]
Executor --> Files[Artifact Manager]
Executor --> Prompts[Prompt Templates]
Router --> RemoteAgents[Registered A2A Agents]
MCP --> Prefect[FuzzForge Backend]
Memory --> SessionDB[Session Store]
Memory --> Semantic[Semantic Recall]
Memory --> Graphs[Cognee Graph]
Files --> Artifacts[Artifact Cache]
```
## Detailed Data Flow
```mermaid
sequenceDiagram
participant User as User / Remote Agent
participant CLI as CLI / HTTP Surface
participant Exec as FuzzForgeExecutor
participant ADK as ADK Runner
participant Prefect as Prefect Backend
participant Cognee as Cognee
participant Artifact as Artifact Cache
User->>CLI: Prompt or slash command
CLI->>Exec: Normalised request + context ID
Exec->>ADK: Tool invocation (LiteLLM)
ADK-->>Exec: Structured response / tool result
Exec->>Prefect: (optional) submit workflow via MCP
Prefect-->>Exec: Run status updates
Exec->>Cognee: (optional) knowledge query / ingestion
Cognee-->>Exec: Graph results
Exec->>Artifact: Persist generated files
Exec-->>CLI: Final response + artifact links + task events
CLI-->>User: Rendered answer
```
## Entry Points
- **CLI shell** (`ai/src/fuzzforge_ai/cli.py`) provides the interactive `fuzzforge ai agent` loop. It streams user messages through the executor, wires slash commands for listing agents, sending files, and launching workflows, and keeps session IDs in sync with ADK’s session service.
- **A2A HTTP server** (`ai/src/fuzzforge_ai/a2a_server.py`) wraps the same agent in Starlette. It exposes RPC-compatible endpoints plus helper routes (`/artifacts/{id}`, `/graph/query`, `/project/files`) and reuses the executor’s task store so downstream agents can poll status updates.
## Core Components
- **FuzzForgeAgent** (`ai/src/fuzzforge_ai/agent.py`) assembles the runtime: it loads environment variables, constructs the executor, and builds an ADK `Agent` backed by `LiteLlm`. The singleton accessor `get_fuzzforge_agent()` keeps CLI and server instances aligned and shares the generated agent card.
- **FuzzForgeExecutor** (`ai/src/fuzzforge_ai/agent_executor.py`) is the brain. It registers tools, manages session storage (SQLite or in-memory via `DatabaseSessionService` / `InMemorySessionService`), and coordinates artifact storage. The executor also tracks long-running Prefect workflows inside `pending_runs`, produces `TaskStatusUpdateEvent` objects, and funnels every response through ADK’s `Runner` so traces include tool metadata.
- **Remote agent registry** (`ai/src/fuzzforge_ai/remote_agent.py`) holds metadata for downstream agents and handles capability discovery over HTTP. Auto-registration is configured by `ConfigManager` so known agents attach on startup.
- **Memory services**:
- `FuzzForgeMemoryService` and `HybridMemoryManager` (`ai/src/fuzzforge_ai/memory_service.py`) provide conversation recall and bridge to Cognee datasets when configured.
- Cognee bootstrap (`ai/src/fuzzforge_ai/cognee_service.py`) ensures ingestion and knowledge queries stay scoped to the current project.
## Workflow Automation
The executor wraps Prefect MCP actions exposed by the backend:
| Tool | Source | Purpose |
| --- | --- | --- |
| `list_workflows_mcp` | `ai/src/fuzzforge_ai/agent_executor.py` | Enumerate available scans |
| `submit_security_scan_mcp` | `agent_executor.py` | Launch a scan and persist run metadata |
| `get_run_status_mcp` | `agent_executor.py` | Poll Prefect for status and push task events |
| `get_comprehensive_scan_summary` | `agent_executor.py` | Collect findings and bundle artifacts |
| `get_backend_status_mcp` | `agent_executor.py` | Block submissions until Prefect reports `ready` |
The CLI surface mirrors these helpers as natural-language prompts (`You> run fuzzforge workflow …`). ADK’s `Runner` handles retries and ensures each tool call yields structured `Event` objects for downstream instrumentation.
## Knowledge & Ingestion
- The `fuzzforge ingest` and `fuzzforge rag ingest` commands call into `ai/src/fuzzforge_ai/ingest_utils.py`, which filters file types, ignores caches, and populates Cognee datasets under `.fuzzforge/cognee/project_<id>/`.
- Runtime queries hit `query_project_knowledge_api` on the executor, which defers to `cognee_service` for dataset lookup and semantic search. When Cognee credentials are absent the tools return a friendly "not configured" response.
## Artifact Pipeline
Artifacts generated during conversations or workflow runs are written to `.fuzzforge/artifacts/`:
1. The executor creates a unique directory per artifact ID and writes the payload (text, JSON, or binary).
2. Metadata is stored in-memory and, when running under the A2A server, surfaced via `GET /artifacts/{id}`.
3. File uploads from `/project/files` reuse the same pipeline so remote agents see a consistent interface.
## Task & Event Wiring
- In CLI mode, `FuzzForgeExecutor` bootstraps shared `InMemoryTaskStore` and `InMemoryQueueManager` instances (see `agent_executor.py`). They allow the agent to emit `TaskStatusUpdateEvent` objects even when the standalone server is not running.
- The A2A HTTP wrapper reuses those handles, so any active workflow is visible to both the local shell and remote peers.
Use the complementary docs for step-by-step instructions:
- [Ingestion & Knowledge Graphs](ingestion.md)
- [LLM & Environment Configuration](configuration.md)
- [Prompt Patterns & Examples](prompts.md)
- [A2A Services](a2a-services.md)
## Memory & Persistence
```mermaid
graph LR
subgraph ADK Memory Layer
SessionDB[(DatabaseSessionService)]
Semantic[Semantic Recall Index]
end
subgraph Project Knowledge
CogneeDataset[(Cognee Dataset)]
HybridManager[HybridMemoryManager]
end
Prompts[Prompts & Tool Outputs] --> SessionDB
SessionDB --> Semantic
Ingestion[Ingestion Pipeline] --> CogneeDataset
CogneeDataset --> HybridManager
HybridManager --> Semantic
HybridManager --> Exec[Executor]
Exec --> Responses[Responses with Context]
```
- **Session persistence** is controlled by `SESSION_PERSISTENCE`. When set to `sqlite`, ADK’s `DatabaseSessionService` writes transcripts to the path configured by `SESSION_DB_PATH` (defaults to `./fuzzforge_sessions.db`). With `inmemory`, the context is scoped to the current process.
- **Semantic recall** stores vector embeddings so `/recall` queries can surface earlier prompts, even after restarts when using SQLite.
- **Hybrid memory manager** (`HybridMemoryManager`) stitches Cognee results into the ADK session. When a knowledge query hits Cognee, the relevant nodes are appended back into the session context so follow-up prompts can reference them naturally.
- **Cognee datasets** are unique per project. Ingestion runs populate `<project>_codebase` while custom calls to `ingest_to_dataset` let you maintain dedicated buckets (e.g., `insights`). Data is persisted inside `.fuzzforge/cognee/project_<id>/` and shared across CLI and A2A modes.
- **Task metadata** (workflow runs, artifact descriptors) lives in the executor’s in-memory caches but is also mirrored through A2A task events so remote agents can resubscribe if the CLI restarts.
- **Operational check**: Run `/recall <keyword>` or `You> search project knowledge for "topic" using INSIGHTS` after ingestion to confirm both ADK session recall and Cognee graph access are active.
- **CLI quick check**: `/memory status` summarises the current memory type, session persistence, and Cognee dataset directories from inside the agent shell.
docs/docs/ai/configuration.md
+122
View File
@@ -0,0 +1,122 @@
# LLM & Environment Configuration
FuzzForge AI relies on LiteLLM adapters embedded in the Google ADK runtime, so you can swap between providers without touching code. Configuration is driven by environment variables inside `.fuzzforge/.env`.
## Minimal Setup
```env
LLM_PROVIDER=openai
LITELLM_MODEL=gpt-5-mini
OPENAI_API_KEY=sk-your-key
```
Set these values before launching `fuzzforge ai agent` or `python -m fuzzforge_ai`.
## .env Template
`fuzzforge init` creates `.fuzzforge/.env.template` alongside the real secrets file. Keep the template under version control so teammates can copy it to `.fuzzforge/.env` and fill in provider credentials locally. The template includes commented examples for Cognee, AgentOps, and alternative LLM providers—extend it with any project-specific overrides you expect collaborators to set.
## Provider Examples
**OpenAI-compatible (Azure, etc.)**
```env
LLM_PROVIDER=azure_openai
LITELLM_MODEL=gpt-4o-mini
LLM_API_KEY=sk-your-azure-key
LLM_ENDPOINT=https://your-resource.openai.azure.com
```
**Anthropic**
```env
LLM_PROVIDER=anthropic
LITELLM_MODEL=claude-3-haiku-20240307
ANTHROPIC_API_KEY=sk-your-key
```
**Ollama (local models)**
```env
LLM_PROVIDER=ollama_chat
LITELLM_MODEL=codellama:latest
OLLAMA_API_BASE=http://localhost:11434
```
Run `ollama pull codellama:latest` ahead of time so the adapter can stream tokens immediately. Any Ollama-hosted model works; set `LITELLM_MODEL` to match the image tag.
**Vertex AI**
```env
LLM_PROVIDER=vertex_ai
LITELLM_MODEL=gemini-1.5-pro
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
```
## Additional LiteLLM Providers
LiteLLM exposes dozens of adapters. Popular additions include:
- `LLM_PROVIDER=anthropic_messages` for Claude 3.5.
- `LLM_PROVIDER=azure_openai` for Azure-hosted GPT variants.
- `LLM_PROVIDER=groq` for Groq LPU-backed models (`GROQ_API_KEY` required).
- `LLM_PROVIDER=ollama_chat` for any local Ollama model.
- `LLM_PROVIDER=vertex_ai` for Gemini.
Refer to the [LiteLLM provider catalog](https://docs.litellm.ai/docs/providers) when mapping environment variables; each adapter lists the exact keys the ADK runtime expects.
## Session Persistence
```
SESSION_PERSISTENCE=sqlite # sqlite | inmemory
MEMORY_SERVICE=inmemory # ADK memory backend
```
Set `SESSION_PERSISTENCE=sqlite` to preserve conversational history across restarts. For ephemeral sessions, switch to `inmemory`.
## Knowledge Graph Settings
To enable Cognee-backed graphs:
```env
LLM_COGNEE_PROVIDER=openai
LLM_COGNEE_MODEL=gpt-5-mini
LLM_COGNEE_API_KEY=sk-your-key
```
If the Cognee variables are omitted, graph-specific tools remain available but return a friendly "not configured" response.
## MCP / Backend Integration
```env
FUZZFORGE_MCP_URL=http://localhost:8010/mcp
```
The agent uses this endpoint to list, launch, and monitor Prefect workflows.
## Tracing & Observability
The executor ships with optional AgentOps tracing. Provide an API key to record conversations, tool calls, and workflow updates:
```env
AGENTOPS_API_KEY=sk-your-agentops-key
AGENTOPS_ENVIRONMENT=local # Optional tag for dashboards
```
Set `FUZZFORGE_DEBUG=1` to surface verbose executor logging and enable additional stdout in the CLI. For HTTP deployments, combine that with:
```env
LOG_LEVEL=DEBUG
```
The ADK runtime also honours `GOOGLE_ADK_TRACE_DIR=/path/to/logs` if you want JSONL traces without an external service.
## Debugging Flags
```env
FUZZFORGE_DEBUG=1 # Enables verbose logging
LOG_LEVEL=DEBUG # Applies to the A2A server and CLI
```
These flags surface additional insight when diagnosing routing or ingestion issues. Combine them with AgentOps tracing to get full timelines of tool usage.
## Related Code
- Env bootstrap: `ai/src/fuzzforge_ai/config_manager.py`
- LiteLLM glue: `ai/src/fuzzforge_ai/agent.py`
- Cognee integration: `ai/src/fuzzforge_ai/cognee_service.py`
docs/docs/ai/ingestion.md
+88
View File
@@ -0,0 +1,88 @@
# Ingestion & Knowledge Graphs
The AI module keeps long-running context by mirroring your repository into a Cognee-powered knowledge graph and persisting conversations in local storage.
## CLI Commands
```bash
# Scan the current project (skips .git/, .fuzzforge/, virtualenvs, caches)
fuzzforge ingest --path . --recursive
# Alias - identical behaviour
fuzzforge rag ingest --path . --recursive
```
The command gathers files using the filters defined in `ai/src/fuzzforge_ai/ingest_utils.py`. By default it includes common source, configuration, and documentation file types while skipping temporary and dependency directories.
### Customising the File Set
Use CLI flags to override the defaults:
```bash
fuzzforge ingest --path backend --file-types .py --file-types .yaml --exclude node_modules --exclude dist
```
## Command Options
`fuzzforge ingest` exposes several flags (see `cli/src/fuzzforge_cli/commands/ingest.py`):
- `--recursive / -r` – Traverse sub-directories.
- `--file-types / -t` – Repeatable flag to whitelist extensions (`-t .py -t .rs`).
- `--exclude / -e` – Repeatable glob patterns to skip (`-e tests/**`).
- `--dataset / -d` – Write into a named dataset instead of `<project>_codebase`.
- `--force / -f` – Clear previous Cognee data before ingesting (prompts for confirmation unless flag supplied).
All runs automatically skip `.fuzzforge/**` and `.git/**` to avoid recursive ingestion of cache folders.
## Dataset Layout
- Primary dataset: `<project>_codebase`
- Additional datasets: create ad-hoc buckets such as `insights` via the `ingest_to_dataset` tool
- Storage location: `.fuzzforge/cognee/project_<id>/`
### Persistence Details
- Every dataset lives under `.fuzzforge/cognee/project_<id>/{data,system}`. These directories are safe to commit to long-lived storage (they only contain embeddings and metadata).
- Cognee assigns deterministic IDs per project; if you move the repository, copy the entire `.fuzzforge/cognee/` tree to retain graph history.
- `HybridMemoryManager` ensures answers from Cognee are written back into the ADK session store so future prompts can refer to the same nodes without repeating the query.
- All Cognee processing runs locally against the files you ingest. No external service calls are made unless you configure a remote Cognee endpoint.
## Prompt Examples
```
You> refresh the project knowledge graph for ./backend
Assistant> Kicks off `fuzzforge ingest` with recursive scan
You> search project knowledge for "prefect workflow" using INSIGHTS
Assistant> Routes to Cognee `search_project_knowledge`
You> ingest_to_dataset("Design doc for new scanner", "insights")
Assistant> Adds the provided text block to the `insights` dataset
```
## Environment Template
The CLI writes a template at `.fuzzforge/.env.template` when you initialise a project. Keep it in source control so collaborators can copy it to `.env` and fill in secrets.
```env
# Core LLM settings
LLM_PROVIDER=openai
LITELLM_MODEL=gpt-5-mini
OPENAI_API_KEY=sk-your-key
# FuzzForge backend (Prefect-powered)
FUZZFORGE_MCP_URL=http://localhost:8010/mcp
# Optional: knowledge graph provider
LLM_COGNEE_PROVIDER=openai
LLM_COGNEE_MODEL=gpt-5-mini
LLM_COGNEE_API_KEY=sk-your-key
```
Add comments or project-specific overrides as needed; the agent reads these variables on startup.
## Tips
- Re-run ingestion after significant code changes to keep the knowledge graph fresh.
- Large binary assets are skipped automatically—store summaries or documentation if you need them searchable.
- Set `FUZZFORGE_DEBUG=1` to surface verbose ingest logs during troubleshooting.
docs/docs/ai/intro.md
+114
View File
@@ -0,0 +1,114 @@
---
sidebar_position: 1
---
# FuzzForge AI Module
FuzzForge AI is the multi-agent layer that lets you operate the FuzzForge security platform through natural language. It orchestrates local tooling, registered Agent-to-Agent (A2A) peers, and the Prefect-powered backend while keeping long-running context in memory and project knowledge graphs.
## Quick Start
1. **Initialise a project**
```bash
cd /path/to/project
fuzzforge init
```
2. **Review environment settings** – copy `.fuzzforge/.env.template` to `.fuzzforge/.env`, then edit the values to match your provider. The template ships with commented defaults for OpenAI-style usage and placeholders for Cognee keys.
```env
LLM_PROVIDER=openai
LITELLM_MODEL=gpt-5-mini
OPENAI_API_KEY=sk-your-key
FUZZFORGE_MCP_URL=http://localhost:8010/mcp
SESSION_PERSISTENCE=sqlite
```
Optional flags you may want to enable early:
```env
MEMORY_SERVICE=inmemory
AGENTOPS_API_KEY=sk-your-agentops-key # Enable hosted tracing
LOG_LEVEL=INFO # CLI / server log level
```
3. **Populate the knowledge graph**
```bash
fuzzforge ingest --path . --recursive
# alias: fuzzforge rag ingest --path . --recursive
```
4. **Launch the agent shell**
```bash
fuzzforge ai agent
```
Keep the backend running (Prefect API at `FUZZFORGE_MCP_URL`) so workflow commands succeed.
## Everyday Workflow
- Run `fuzzforge ai agent` and start with `list available fuzzforge workflows` or `/memory status` to confirm everything is wired.
- Use natural prompts for automation (`run fuzzforge workflow …`, `search project knowledge for …`) and fall back to slash commands for precision (`/recall`, `/sendfile`).
- Keep `/memory datasets` handy to see which Cognee datasets are available after each ingest.
- Start the HTTP surface with `python -m fuzzforge_ai` when external agents need access to artifacts or graph queries. The CLI stays usable at the same time.
- Refresh the knowledge graph regularly: `fuzzforge ingest --path . --recursive --force` keeps responses aligned with recent code changes.
## What the Agent Can Do
- **Route requests** – automatically selects the right local tool or remote agent using the A2A capability registry.
- **Run security workflows** – list, submit, and monitor FuzzForge workflows via MCP wrappers.
- **Manage artifacts** – create downloadable files for reports, code edits, and shared attachments.
- **Maintain context** – stores session history, semantic recall, and Cognee project graphs.
- **Serve over HTTP** – expose the same agent as an A2A server using `python -m fuzzforge_ai`.
## Essential Commands
Inside `fuzzforge ai agent` you can mix slash commands and free-form prompts:
```text
/list # Show registered A2A agents
/register http://:10201 # Add a remote agent
/artifacts # List generated files
/sendfile SecurityAgent src/report.md "Please review"
You> route_to SecurityAnalyzer: scan ./backend for secrets
You> run fuzzforge workflow static_analysis_scan on ./test_projects/demo
You> search project knowledge for "prefect status" using INSIGHTS
```
Artifacts created during the conversation are served from `.fuzzforge/artifacts/` and exposed through the A2A HTTP API.
## Memory & Knowledge
The module layers three storage systems:
- **Session persistence** (SQLite or in-memory) for chat transcripts.
- **Semantic recall** via the ADK memory service for fuzzy search.
- **Cognee graphs** for project-wide knowledge built from ingestion runs.
Re-run ingestion after major code changes to keep graph answers relevant. If Cognee variables are not set, graph-specific tools automatically respond with a polite "not configured" message.
## Sample Prompts
Use these to validate the setup once the agent shell is running:
- `list available fuzzforge workflows`
- `run fuzzforge workflow static_analysis_scan on ./backend with target_branch=main`
- `show findings for that run once it finishes`
- `refresh the project knowledge graph for ./backend`
- `search project knowledge for "prefect readiness" using INSIGHTS`
- `/recall terraform secrets`
- `/memory status`
- `ROUTE_TO SecurityAnalyzer: audit infrastructure_vulnerable`
## Need More Detail?
Dive into the dedicated guides in this category :
- [Architecture](./architecture.md) – High-level architecture with diagrams and component breakdowns.
- [Ingestion](./ingestion.md) – Command options, Cognee persistence, and prompt examples.
- [Configuration](./configuration.md) – LLM provider matrix, local model setup, and tracing options.
- [Prompts](./prompts.md) – Slash commands, workflow prompts, and routing tips.
- [A2A Services](./a2a-services.md) – HTTP endpoints, agent card, and collaboration flow.
- [Memory Persistence](./architecture.md#memory--persistence) – Deep dive on memory storage, datasets, and how `/memory status` inspects them.
## Development Notes
- Entry point for the CLI: `ai/src/fuzzforge_ai/cli.py`
- A2A HTTP server: `ai/src/fuzzforge_ai/a2a_server.py`
- Tool routing & workflow glue: `ai/src/fuzzforge_ai/agent_executor.py`
- Ingestion helpers: `ai/src/fuzzforge_ai/ingest_utils.py`
Install the module in editable mode (`pip install -e ai`) while iterating so CLI changes are picked up immediately.
docs/docs/ai/prompts.md
+60
View File
@@ -0,0 +1,60 @@
# Prompt Patterns & Examples
Use the `fuzzforge ai agent` shell to mix structured slash commands with natural requests. The Google ADK runtime keeps conversation context, so follow-ups automatically reuse earlier answers, retrieved files, and workflow IDs.
## Slash Commands
| Command | Purpose | Example |
| --- | --- | --- |
| `/list` | Show registered A2A agents | `/list` |
| `/register <url>` | Register a remote agent card | `/register http://localhost:10201` |
| `/artifacts` | List generated artifacts with download links | `/artifacts` |
| `/sendfile <agent> <path> [note]` | Ship a file as an artifact to a remote peer | `/sendfile SecurityAnalyzer reports/latest.md "Please review"` |
| `/memory status` | Summarise conversational memory, session store, and Cognee directories | `/memory status` |
| `/memory datasets` | List available Cognee datasets | `/memory datasets` |
| `/recall <query>` | Search prior conversation context using semantic vectors | `/recall dependency updates` |
## Workflow Automation
```
You> list available fuzzforge workflows
Assistant> [returns workflow names, descriptions, and required parameters]
You> run fuzzforge workflow static_analysis_scan on ./backend with target_branch=main
Assistant> Submits the run, emits TaskStatusUpdateEvent entries, and links the SARIF artifact when complete.
You> show findings for that run once it finishes
Assistant> Streams the `get_comprehensive_scan_summary` output and attaches the artifact URI.
```
## Knowledge Graph & Memory Prompts
```
You> refresh the project knowledge graph for ./backend
Assistant> Launches `fuzzforge ingest --path ./backend --recursive` and reports file counts.
You> search project knowledge for "prefect readiness" using INSIGHTS
Assistant> Routes to Cognee via `query_project_knowledge_api` and returns the top matches.
You> recall "api key rotation"
Assistant> Uses the ADK semantic memory service to surface earlier chat snippets.
```
## Routing to Specialist Agents
```
You> ROUTE_TO SecurityAnalyzer: audit this Terraform module for secrets
Assistant> Delegates the request to `SecurityAnalyzer` using the A2A capability map.
You> sendfile DocumentationAgent docs/runbook.md "Incorporate latest workflow"
Assistant> Uploads the file as an artifact and notifies the remote agent.
```
## Prompt Tips
- Use explicit verbs (`list`, `run`, `search`) to trigger the Prefect workflow helpers.
- Include parameter names inline (`with target_branch=main`) so the executor maps values to MCP tool inputs without additional clarification.
- When referencing prior runs, reuse the assistant’s run IDs or ask for "the last run"—the session store tracks them per context ID.
- If Cognee is not configured, graph queries return a friendly notice; set `LLM_COGNEE_*` variables to enable full answers.
- Combine slash commands and natural prompts in the same session; the ADK session service keeps them in a single context thread.
- `/memory search <query>` is a shortcut for `/recall <query>` if you want status plus recall in one place.
docs/docs/concept/_category_.json
+8
View File
@@ -0,0 +1,8 @@
{
"label": "Concept",
"position": 2,
"link": {
"type": "generated-index",
"description": "Concept pages that are understanding-oriented."
}
}
docs/docs/concept/architecture.md
+214
View File
@@ -0,0 +1,214 @@
# Architecture
FuzzForge is a distributed, containerized platform for security analysis workflows. Its architecture is designed for scalability, isolation, and reliability, drawing on modern patterns like microservices and orchestration. This page explains the core architectural concepts behind FuzzForge, meaning what the main components are, how they interact, and why the system is structured this way.
:::warning
FuzzForge’s architecture is evolving. While the long-term goal is a hexagonal architecture, the current implementation is still in transition. Expect changes as the platform matures.
:::
---
## Why This Architecture?
FuzzForge’s architecture is shaped by several key goals:
- **Scalability:** Handle many workflows in parallel, scaling up or down as needed.
- **Isolation:** Run each workflow in its own secure environment, minimizing risk.
- **Reliability:** Ensure that failures in one part of the system don’t bring down the whole platform.
- **Extensibility:** Make it easy to add new workflows, tools, or integrations.
## High-Level System Overview
At a glance, FuzzForge is organized into several layers, each with a clear responsibility:
- **Client Layer:** Where users and external systems interact (CLI, API clients, MCP server).
- **API Layer:** The FastAPI backend, which exposes REST endpoints and manages requests.
- **Orchestration Layer:** Prefect server and workers, which schedule and execute workflows.
- **Execution Layer:** Docker Engine and containers, where workflows actually run.
- **Storage Layer:** PostgreSQL database, Docker volumes, and a result cache for persistence.
Here’s a simplified view of how these layers fit together:
```mermaid
graph TB
subgraph "Client Layer"
CLI[CLI Client]
API_Client[API Client]
MCP[MCP Server]
end
subgraph "API Layer"
FastAPI[FastAPI Backend]
Router[Route Handlers]
Middleware[Middleware Stack]
end
subgraph "Orchestration Layer"
Prefect[Prefect Server]
Workers[Prefect Workers]
Scheduler[Workflow Scheduler]
end
subgraph "Execution Layer"
Docker[Docker Engine]
Containers[Workflow Containers]
Registry[Docker Registry]
end
subgraph "Storage Layer"
PostgreSQL[PostgreSQL Database]
Volumes[Docker Volumes]
Cache[Result Cache]
end
CLI --> FastAPI
API_Client --> FastAPI
MCP --> FastAPI
FastAPI --> Router
Router --> Middleware
Middleware --> Prefect
Prefect --> Workers
Workers --> Scheduler
Scheduler --> Docker
Docker --> Containers
Docker --> Registry
Containers --> Volumes
FastAPI --> PostgreSQL
Workers --> PostgreSQL
Containers --> Cache
```
## What Are the Main Components?
### API Layer
- **FastAPI Backend:** The main entry point for users and clients. Handles authentication, request validation, and exposes endpoints for workflow management, results, and health checks.
- **Middleware Stack:** Manages API keys, user authentication, CORS, logging, and error handling.
### Orchestration Layer
- **Prefect Server:** Schedules and tracks workflows, backed by PostgreSQL.
- **Prefect Workers:** Execute workflows in Docker containers. Can be scaled horizontally.
- **Workflow Scheduler:** Balances load, manages priorities, and enforces resource limits.
### Execution Layer
- **Docker Engine:** Runs workflow containers, enforcing isolation and resource limits.
- **Workflow Containers:** Custom images with security tools, mounting code and results volumes.
- **Docker Registry:** Stores and distributes workflow images.
### Storage Layer
- **PostgreSQL Database:** Stores workflow metadata, state, and results.
- **Docker Volumes:** Persist workflow results and artifacts.
- **Result Cache:** Speeds up access to recent results, with in-memory and disk persistence.
## How Does Data Flow Through the System?
### Submitting a Workflow
1. **User submits a workflow** via CLI or API client.
2. **API validates** the request and creates a deployment in Prefect.
3. **Prefect schedules** the workflow and assigns it to a worker.
4. **Worker launches a container** to run the workflow.
5. **Results are stored** in Docker volumes and the database.
6. **Status updates** flow back through Prefect and the API to the user.
```mermaid
sequenceDiagram
participant User
participant API
participant Prefect
participant Worker
participant Container
participant Storage
User->>API: Submit workflow
API->>API: Validate parameters
API->>Prefect: Create deployment
Prefect->>Worker: Schedule execution
Worker->>Container: Create and start
Container->>Container: Execute security tools
Container->>Storage: Store SARIF results
Worker->>Prefect: Update status
Prefect->>API: Workflow complete
API->>User: Return results
```
### Retrieving Results
1. **User requests status or results** via the API.
2. **API queries the database** for workflow metadata.
3. **If complete,** results are fetched from storage and returned to the user.
## How Do Services Communicate?
- **Internally:** FastAPI talks to Prefect via REST; Prefect coordinates with workers over HTTP; workers manage containers via the Docker Engine API. All core services use pooled connections to PostgreSQL.
- **Externally:** Users interact via CLI or API clients (HTTP REST). The MCP server can automate workflows via its own protocol.
## How Is Security Enforced?
- **Container Isolation:** Each workflow runs in its own Docker network, as a non-root user, with strict resource limits and only necessary volumes mounted.
- **Volume Security:** Source code is mounted read-only; results are written to dedicated, temporary volumes.
- **API Security:** All endpoints require API keys, validate inputs, enforce rate limits, and log requests for auditing.
## How Does FuzzForge Scale?
- **Horizontally:** Add more Prefect workers to handle more workflows in parallel. Scale the database with read replicas and connection pooling.
- **Vertically:** Adjust CPU and memory limits for containers and services as needed.
Example Docker Compose scaling:
```yaml
services:
prefect-worker:
deploy:
resources:
limits:
memory: 4G
cpus: '2.0'
reservations:
memory: 1G
cpus: '0.5'
```
## How Is It Deployed?
- **Development:** All services run via Docker Compose—backend, Prefect, workers, database, and registry.
- **Production:** Add load balancers, database clustering, and multiple worker instances for high availability. Health checks, metrics, and centralized logging support monitoring and troubleshooting.
## How Is Configuration Managed?
- **Environment Variables:** Control core settings like database URLs, registry location, and Prefect API endpoints.
- **Service Discovery:** Docker Compose’s internal DNS lets services find each other by name, with consistent port mapping and health check endpoints.
Example configuration:
```bash
COMPOSE_PROJECT_NAME=fuzzforge_alpha
DATABASE_URL=postgresql://postgres:postgres@postgres:5432/fuzzforge
PREFECT_API_URL=http://prefect-server:4200/api
DOCKER_REGISTRY=localhost:5001
DOCKER_INSECURE_REGISTRY=true
```
## How Are Failures Handled?
- **Failure Isolation:** Each service is independent; failures don’t cascade. Circuit breakers and graceful degradation keep the system stable.
- **Recovery:** Automatic retries with backoff for transient errors, dead letter queues for persistent failures, and workflow state recovery after restarts.
## Implementation Details
- **Tech Stack:** FastAPI (Python async), Prefect 3.x, Docker, Docker Compose, PostgreSQL (asyncpg), and Docker networking.
- **Performance:** Workflows start in 2–5 seconds; results are retrieved quickly thanks to caching and database indexing.
- **Extensibility:** Add new workflows by deploying new Docker images; extend the API with new endpoints; configure storage backends as needed.
---
## In Summary
FuzzForge’s architecture is designed to be robust, scalable, and secure—ready to handle demanding security analysis workflows in a modern, distributed environment. As the platform evolves, expect even more modularity and flexibility, making it easier to adapt to new requirements and technologies.
docs/docs/concept/concept.tmpl
+20
View File
@@ -0,0 +1,20 @@
# {Concept Title}
{Brief introduction of the concept, including its origin and general purpose.}
## Purpose
- {The primary purpose and its relevance in its field.}
## Common Usage
- {Usage 1}: {Brief description.}
- {Usage 2}: {Brief description.}
## Benefits
- {Key benefit and why it's preferred in certain scenarios.}
## Conclusion
{Summary of its importance and role in its respective field.}
docs/docs/concept/docker-containers.md
+217
View File
@@ -0,0 +1,217 @@
# Docker Containers in FuzzForge: Concept and Design
Docker containers are at the heart of FuzzForge’s execution model. They provide the isolation, consistency, and flexibility needed to run security workflows reliably—no matter where FuzzForge is deployed. This page explains the core concepts behind container usage in FuzzForge, why containers are used, and how they shape the platform’s behavior.
---
## Why Use Docker Containers?
FuzzForge relies on Docker containers for several key reasons:
- **Isolation:** Each workflow runs in its own container, so tools and processes can’t interfere with each other or the host.
- **Consistency:** The environment inside a container is always the same, regardless of the underlying system.
- **Security:** Containers restrict access to host resources and run as non-root users.
- **Reproducibility:** Results are deterministic, since the environment is controlled and versioned.
- **Scalability:** Containers can be started, stopped, and scaled up or down as needed.
---
## How Does FuzzForge Use Containers?
### The Container Model
Every workflow in FuzzForge is executed inside a Docker container. Here’s what that means in practice:
- **Workflow containers** are built from language-specific base images (like Python or Node.js), with security tools and workflow code pre-installed.
- **Infrastructure containers** (API server, Prefect, database) use official images and are configured for the platform’s needs.
### Container Lifecycle: From Build to Cleanup
The lifecycle of a workflow container looks like this:
1. **Image Build:** A Docker image is built with all required tools and code.
2. **Image Push/Pull:** The image is pushed to (and later pulled from) a local or remote registry.
3. **Container Creation:** The container is created with the right volumes and environment.
4. **Execution:** The workflow runs inside the container.
5. **Result Storage:** Results are written to mounted volumes.
6. **Cleanup:** The container and any temporary data are removed.
```mermaid
graph TB
Build[Build Image] --> Push[Push to Registry]
Push --> Pull[Pull Image]
Pull --> Create[Create Container]
Create --> Mount[Mount Volumes]
Mount --> Start[Start Container]
Start --> Execute[Run Workflow]
Execute --> Results[Store Results]
Execute --> Stop[Stop Container]
Stop --> Cleanup[Cleanup Data]
Cleanup --> Remove[Remove Container]
```
---
## What’s Inside a Workflow Container?
A typical workflow container is structured like this:
- **Base Image:** Usually a slim language image (e.g., `python:3.11-slim`).
- **System Dependencies:** Installed as needed (e.g., `git`, `curl`).
- **Security Tools:** Pre-installed (e.g., `semgrep`, `bandit`, `safety`).
- **Workflow Code:** Copied into the container.
- **Non-root User:** Created for execution.
- **Entrypoint:** Runs the workflow code.
Example Dockerfile snippet:
```dockerfile
FROM python:3.11-slim
RUN apt-get update && apt-get install -y git curl && rm -rf /var/lib/apt/lists/*
RUN pip install semgrep bandit safety
COPY ./toolbox /app/toolbox
WORKDIR /app
RUN useradd -m -u 1000 fuzzforge
USER fuzzforge
CMD ["python", "-m", "toolbox.main"]
```
---
## How Are Containers Networked and Connected?
- **Docker Compose Network:** All containers are attached to a custom bridge network for internal communication.
- **Internal DNS:** Services communicate using Docker Compose service names.
- **Port Exposure:** Only necessary ports are exposed to the host.
- **Network Isolation:** Workflow containers are isolated from infrastructure containers when possible.
Example network config:
```yaml
networks:
fuzzforge:
driver: bridge
ipam:
config:
- subnet: 172.20.0.0/16
```
---
## How Is Data Managed with Volumes?
### Volume Types
- **Target Code Volume:** Mounts the code to be analyzed, read-only, into the container.
- **Result Volume:** Stores workflow results and artifacts, persists after container exit.
- **Temporary Volumes:** Used for scratch space, destroyed with the container.
Example volume mount:
```yaml
volumes:
- "/host/path/to/code:/app/target:ro"
- "fuzzforge_alpha_prefect_storage:/app/prefect"
```
### Volume Security
- **Read-only Mounts:** Prevent workflows from modifying source code.
- **Isolated Results:** Each workflow writes to its own result directory.
- **No Arbitrary Host Access:** Only explicitly mounted paths are accessible.
---
## How Are Images Built and Managed?
- **Automated Builds:** Images are built and pushed to a local registry for development, or a secure registry for production.
- **Build Optimization:** Use layer caching, multi-stage builds, and minimal base images.
- **Versioning:** Use tags (`latest`, semantic versions, or SHA digests) to track images.
Example build and push:
```bash
docker build -t localhost:5001/fuzzforge-static-analysis:latest .
docker push localhost:5001/fuzzforge-static-analysis:latest
```
---
## How Are Resources Controlled?
- **Memory and CPU Limits:** Set per container to prevent resource exhaustion.
- **Resource Monitoring:** Use `docker stats` and platform APIs to track usage.
- **Alerts:** Detect and handle out-of-memory or CPU throttling events.
Example resource config:
```yaml
services:
prefect-worker:
deploy:
resources:
limits:
memory: 4G
cpus: '2.0'
reservations:
memory: 1G
cpus: '0.5'
```
---
## How Is Security Enforced?
- **Non-root Execution:** Containers run as a dedicated, non-root user.
- **Capability Restrictions:** Drop unnecessary Linux capabilities.
- **Filesystem Protection:** Use read-only filesystems and tmpfs for temporary data.
- **Network Isolation:** Restrict network access to only what’s needed.
- **No Privileged Mode:** Containers never run with elevated privileges.
Example security options:
```yaml
services:
prefect-worker:
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
cap_add:
- CHOWN
- SETGID
- SETUID
```
---
## How Is Performance Optimized?
- **Image Layering:** Structure Dockerfiles for efficient caching.
- **Dependency Preinstallation:** Reduce startup time by pre-installing dependencies.
- **Warm Containers:** Optionally pre-create containers for faster workflow startup.
- **Horizontal Scaling:** Scale worker containers to handle more workflows in parallel.
---
## How Are Containers Monitored and Debugged?
- **Health Checks:** Each service and workflow container has a health endpoint or check.
- **Logging:** All container logs are collected and can be accessed via `docker logs` or the FuzzForge API.
- **Debug Access:** Use `docker exec` to access running containers for troubleshooting.
- **Resource Stats:** Monitor with `docker stats` or platform dashboards.
---
## How Does This All Fit Into FuzzForge?
- **Prefect Workers:** Manage the full lifecycle of workflow containers.
- **API Integration:** Exposes container status, logs, and resource metrics.
- **Volume Management:** Ensures results and artifacts are collected and persisted.
- **Security and Resource Controls:** Enforced automatically for every workflow.
---
## In Summary
Docker containers are the foundation of FuzzForge’s execution model. They provide the isolation, security, and reproducibility needed for robust security analysis workflows—while making it easy to scale, monitor, and extend the platform.
docs/docs/concept/fuzzforge-ai.md
+83
View File
@@ -0,0 +1,83 @@
# FuzzForge AI: Conceptual Overview
Welcome to FuzzForge AI—a multi-agent orchestration platform designed to supercharge your intelligent automation, security workflows, and project knowledge management. This document provides a high-level conceptual introduction to what FuzzForge AI is, what problems it solves, and how its architecture enables powerful, context-aware agent collaboration.
---
## What is FuzzForge AI?
FuzzForge AI is a multi-agent orchestration system that implements the A2A (Agent-to-Agent) protocol for intelligent agent routing, persistent memory management, and project-scoped knowledge graphs. Think of it as an intelligent hub that coordinates a team of specialized agents, each with their own skills, while maintaining context and knowledge across sessions and projects.
**Key Goals:**
- Seamlessly route requests to the right agent for the job
- Preserve and leverage project-specific knowledge
- Enable secure, auditable, and extensible automation workflows
- Make multi-agent collaboration as easy as talking to a single assistant
---
## Core Concepts
### 1. **Agent Orchestration**
FuzzForge AI acts as a conductor, automatically routing your requests to the most capable registered agent. Agents can be local or remote, and each advertises its skills and capabilities via the A2A protocol.
### 2. **Memory & Knowledge Management**
The system features a three-layer memory architecture:
- **Session Persistence:** Keeps track of ongoing sessions and conversations.
- **Semantic Memory:** Archives conversations and enables semantic search.
- **Knowledge Graphs:** Maintains structured, project-scoped knowledge for deep context.
### 3. **Artifact System**
Artifacts are files or structured content generated, processed, or shared by agents. The artifact system supports creation, storage, and secure sharing of code, configs, reports, and more—enabling reproducible, auditable workflows.
### 4. **A2A Protocol Compliance**
FuzzForge AI fully implements the A2A (Agent-to-Agent) protocol (spec 0.3.0), ensuring standardized, interoperable communication between agents—whether they're running locally or across the network.
---
## High-Level Architecture
Here's how the main components fit together:
```
FuzzForge AI System
├── CLI Interface (cli.py)
│ ├── Commands & Session Management
│ └── Agent Registry Persistence
├── Agent Core (agent.py)
│ ├── Main Coordinator
│ └── Memory Manager Integration
├── Agent Executor (agent_executor.py)
│ ├── Tool Management & Orchestration
│ ├── ROUTE_TO Pattern Implementation
│ └── Artifact Creation & Management
├── Memory Architecture (Three Layers)
│ ├── Session Persistence
│ ├── Semantic Memory
│ └── Knowledge Graphs
├── A2A Communication Layer
│ ├── Remote Agent Connection
│ ├── Agent Card Management
│ └── Protocol Compliance
└── A2A Server (a2a_server.py)
├── HTTP/SSE Server
├── Artifact HTTP Serving
└── Task Store & Queue Management
```
**How it works:**
1. **User Input:** You interact via CLI or API, using natural language or commands.
2. **Agent Routing:** The system decides whether to handle the request itself or route it to a specialist agent.
3. **Tool Execution:** Built-in and agent-provided tools perform operations.
4. **Memory Integration:** Results and context are stored for future use.
5. **Response Generation:** The system returns results, often with artifacts or actionable insights.
---
## Why FuzzForge AI?
- **Extensible:** Easily add new agents, tools, and workflows.
- **Context-Aware:** Remembers project history, conversations, and knowledge.
- **Secure:** Project isolation, input validation, and artifact management.
- **Collaborative:** Enables multi-agent workflows and knowledge sharing.
- **Fun & Productive:** Designed to make automation and security tasks less tedious and more interactive.
docs/docs/concept/sarif-format.md
+618
View File
@@ -0,0 +1,618 @@
# SARIF Format
FuzzForge uses the Static Analysis Results Interchange Format (SARIF) as the standardized output format for all security analysis results. SARIF provides a consistent, machine-readable format that enables tool interoperability and comprehensive result analysis.
## What is SARIF?
### Overview
SARIF (Static Analysis Results Interchange Format) is an OASIS-approved standard (SARIF 2.1.0) designed to standardize the output of static analysis tools. FuzzForge extends this standard to cover dynamic analysis, secret detection, infrastructure analysis, and fuzzing results.
### Key Benefits
- **Standardization**: Consistent format across all security tools and workflows
- **Interoperability**: Integration with existing security tools and platforms
- **Rich Metadata**: Comprehensive information about findings, tools, and analysis runs
- **Tool Agnostic**: Works with any security tool that produces structured results
- **IDE Integration**: Native support in modern development environments
### SARIF Structure
```json
{
"version": "2.1.0",
"schema": "https://json.schemastore.org/sarif-2.1.0.json",
"runs": [
{
"tool": { /* Tool information */ },
"invocations": [ /* How the tool was run */ ],
"artifacts": [ /* Files analyzed */ ],
"results": [ /* Security findings */ ]
}
]
}
```
## FuzzForge SARIF Implementation
### Run Structure
Each FuzzForge workflow produces a SARIF "run" containing:
```json
{
"tool": {
"driver": {
"name": "FuzzForge",
"version": "1.0.0",
"informationUri": "https://github.com/FuzzingLabs/fuzzforge",
"organization": "FuzzingLabs",
"rules": [ /* Security rules applied */ ]
},
"extensions": [
{
"name": "semgrep",
"version": "1.45.0",
"rules": [ /* Semgrep-specific rules */ ]
}
]
},
"invocations": [
{
"executionSuccessful": true,
"startTimeUtc": "2025-09-25T12:00:00.000Z",
"endTimeUtc": "2025-09-25T12:05:30.000Z",
"workingDirectory": {
"uri": "file:///app/target/"
},
"commandLine": "python -m toolbox.workflows.static_analysis",
"environmentVariables": {
"WORKFLOW_TYPE": "static_analysis_scan"
}
}
]
}
```
### Result Structure
Each security finding is represented as a SARIF result:
```json
{
"ruleId": "semgrep.security.audit.sqli.pg-sqli",
"ruleIndex": 42,
"level": "error",
"message": {
"text": "Potential SQL injection vulnerability detected"
},
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "src/database/queries.py",
"uriBaseId": "SRCROOT"
},
"region": {
"startLine": 156,
"startColumn": 20,
"endLine": 156,
"endColumn": 45,
"snippet": {
"text": "cursor.execute(query)"
}
}
}
}
],
"properties": {
"tool": "semgrep",
"confidence": "high",
"severity": "high",
"cwe": ["CWE-89"],
"owasp": ["A03:2021"],
"references": [
"https://owasp.org/Top10/A03_2021-Injection/"
]
}
}
```
## Finding Categories and Severity
### Severity Levels
FuzzForge maps tool-specific severity levels to SARIF standard levels:
#### SARIF Level Mapping
- **error**: Critical and High severity findings
- **warning**: Medium severity findings
- **note**: Low severity findings
- **info**: Informational findings
#### Extended Severity Properties
```json
{
"properties": {
"severity": "high", // FuzzForge severity
"confidence": "medium", // Tool confidence
"exploitability": "high", // Likelihood of exploitation
"impact": "data_breach" // Potential impact
}
}
```
### Vulnerability Classification
#### CWE (Common Weakness Enumeration)
```json
{
"properties": {
"cwe": ["CWE-89", "CWE-79"],
"cwe_category": "Injection"
}
}
```
#### OWASP Top 10 Mapping
```json
{
"properties": {
"owasp": ["A03:2021", "A06:2021"],
"owasp_category": "Injection"
}
}
```
#### Tool-Specific Classifications
```json
{
"properties": {
"tool_category": "security",
"rule_type": "semantic_grep",
"finding_type": "sql_injection"
}
}
```
## Multi-Tool Result Aggregation
### Tool Extension Model
FuzzForge aggregates results from multiple tools using SARIF's extension model:
```json
{
"tool": {
"driver": {
"name": "FuzzForge",
"version": "1.0.0"
},
"extensions": [
{
"name": "semgrep",
"version": "1.45.0",
"guid": "semgrep-extension-guid"
},
{
"name": "bandit",
"version": "1.7.5",
"guid": "bandit-extension-guid"
}
]
}
}
```
### Result Correlation
#### Cross-Tool Finding Correlation
```json
{
"ruleId": "fuzzforge.correlation.sql-injection",
"level": "error",
"message": {
"text": "SQL injection vulnerability confirmed by multiple tools"
},
"locations": [ /* Primary location */ ],
"relatedLocations": [ /* Additional contexts */ ],
"properties": {
"correlation_id": "corr-001",
"confirming_tools": ["semgrep", "bandit"],
"confidence_score": 0.95,
"aggregated_severity": "critical"
}
}
```
#### Finding Relationships
```json
{
"ruleId": "semgrep.security.audit.xss.direct-use-of-jinja2",
"properties": {
"related_findings": [
{
"correlation_type": "same_vulnerability_class",
"related_rule": "bandit.B703",
"relationship": "confirms"
},
{
"correlation_type": "attack_chain",
"related_rule": "nuclei.xss.reflected",
"relationship": "exploits"
}
]
}
}
```
## Workflow-Specific Extensions
### Static Analysis Results
```json
{
"properties": {
"analysis_type": "static",
"language": "python",
"complexity_score": 3.2,
"coverage": {
"lines_analyzed": 15420,
"functions_analyzed": 892,
"classes_analyzed": 156
}
}
}
```
### Dynamic Analysis Results
```json
{
"properties": {
"analysis_type": "dynamic",
"test_method": "web_application_scan",
"target_url": "https://example.com",
"http_method": "POST",
"request_payload": "user_input=<script>alert(1)</script>",
"response_code": 200,
"exploitation_proof": "alert_box_displayed"
}
}
```
### Secret Detection Results
```json
{
"properties": {
"analysis_type": "secret_detection",
"secret_type": "api_key",
"entropy_score": 4.2,
"commit_hash": "abc123def456",
"commit_date": "2025-09-20T10:30:00Z",
"author": "developer@example.com",
"exposure_duration": "30_days"
}
}
```
### Infrastructure Analysis Results
```json
{
"properties": {
"analysis_type": "infrastructure",
"resource_type": "docker_container",
"policy_violation": "privileged_container",
"compliance_framework": ["CIS", "NIST"],
"remediation_effort": "low",
"deployment_risk": "high"
}
}
```
### Fuzzing Results
```json
{
"properties": {
"analysis_type": "fuzzing",
"fuzzer": "afl++",
"crash_type": "segmentation_fault",
"crash_address": "0x7fff8b2a1000",
"exploitability": "likely_exploitable",
"test_case": "base64:SGVsbG8gV29ybGQ=",
"coverage_achieved": "85%"
}
}
```
## SARIF Processing and Analysis
### Result Filtering
#### Severity-Based Filtering
```python
def filter_by_severity(sarif_results, min_severity="medium"):
"""Filter SARIF results by minimum severity level"""
severity_order = {"info": 0, "note": 1, "warning": 2, "error": 3}
min_level = severity_order.get(min_severity, 1)
filtered_results = []
for result in sarif_results["runs"][0]["results"]:
result_level = severity_order.get(result.get("level", "note"), 1)
if result_level >= min_level:
filtered_results.append(result)
return filtered_results
```
#### Rule-Based Filtering
```python
def filter_by_rules(sarif_results, rule_patterns):
"""Filter results by rule ID patterns"""
import re
filtered_results = []
for result in sarif_results["runs"][0]["results"]:
rule_id = result.get("ruleId", "")
for pattern in rule_patterns:
if re.match(pattern, rule_id):
filtered_results.append(result)
break
return filtered_results
```
### Statistical Analysis
#### Severity Distribution
```python
def analyze_severity_distribution(sarif_results):
"""Analyze distribution of findings by severity"""
distribution = {"error": 0, "warning": 0, "note": 0, "info": 0}
for result in sarif_results["runs"][0]["results"]:
level = result.get("level", "note")
distribution[level] += 1
return distribution
```
#### Tool Coverage Analysis
```python
def analyze_tool_coverage(sarif_results):
"""Analyze which tools contributed findings"""
tool_stats = {}
for result in sarif_results["runs"][0]["results"]:
tool = result.get("properties", {}).get("tool", "unknown")
if tool not in tool_stats:
tool_stats[tool] = {"count": 0, "severities": {"error": 0, "warning": 0, "note": 0, "info": 0}}
tool_stats[tool]["count"] += 1
level = result.get("level", "note")
tool_stats[tool]["severities"][level] += 1
return tool_stats
```
## SARIF Export and Integration
### Export Formats
#### JSON Export
```python
def export_sarif_json(sarif_results, output_path):
"""Export SARIF results as JSON"""
import json
with open(output_path, 'w') as f:
json.dump(sarif_results, f, indent=2, ensure_ascii=False)
```
#### CSV Export for Spreadsheets
```python
def export_sarif_csv(sarif_results, output_path):
"""Export SARIF results as CSV for spreadsheet analysis"""
import csv
with open(output_path, 'w', newline='') as f:
writer = csv.writer(f)
writer.writerow(['Rule ID', 'Severity', 'Message', 'File', 'Line', 'Tool'])
for result in sarif_results["runs"][0]["results"]:
rule_id = result.get("ruleId", "unknown")
level = result.get("level", "note")
message = result.get("message", {}).get("text", "")
tool = result.get("properties", {}).get("tool", "unknown")
for location in result.get("locations", []):
physical_location = location.get("physicalLocation", {})
file_path = physical_location.get("artifactLocation", {}).get("uri", "")
line = physical_location.get("region", {}).get("startLine", "")
writer.writerow([rule_id, level, message, file_path, line, tool])
```
### IDE Integration
#### Visual Studio Code
SARIF files can be opened directly in VS Code with the SARIF extension:
```json
{
"recommendations": ["ms-sarif.sarif-viewer"],
"sarif.viewer.connectToGitHub": true,
"sarif.viewer.showResultsInExplorer": true
}
```
#### GitHub Integration
GitHub automatically processes SARIF files uploaded through Actions:
```yaml
- name: Upload SARIF results
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: fuzzforge-results.sarif
category: security-analysis
```
### API Integration
#### SARIF Result Access
```python
# Example: Accessing SARIF results via FuzzForge API
async with FuzzForgeClient() as client:
result = await client.get_workflow_result(run_id)
# Access SARIF data
sarif_data = result["sarif"]
findings = sarif_data["runs"][0]["results"]
# Filter critical findings
critical_findings = [
f for f in findings
if f.get("level") == "error" and
f.get("properties", {}).get("severity") == "critical"
]
```
## SARIF Validation and Quality
### Schema Validation
```python
import jsonschema
import requests
def validate_sarif(sarif_data):
"""Validate SARIF data against official schema"""
schema_url = "https://json.schemastore.org/sarif-2.1.0.json"
schema = requests.get(schema_url).json()
try:
jsonschema.validate(sarif_data, schema)
return True, "Valid SARIF 2.1.0 format"
except jsonschema.ValidationError as e:
return False, f"SARIF validation error: {e.message}"
```
### Quality Metrics
```python
def calculate_sarif_quality_metrics(sarif_data):
"""Calculate quality metrics for SARIF results"""
results = sarif_data["runs"][0]["results"]
metrics = {
"total_findings": len(results),
"findings_with_location": len([r for r in results if r.get("locations")]),
"findings_with_message": len([r for r in results if r.get("message", {}).get("text")]),
"findings_with_remediation": len([r for r in results if r.get("fixes")]),
"unique_rules": len(set(r.get("ruleId") for r in results)),
"coverage_percentage": calculate_coverage(sarif_data)
}
metrics["quality_score"] = (
metrics["findings_with_location"] / max(metrics["total_findings"], 1) * 0.3 +
metrics["findings_with_message"] / max(metrics["total_findings"], 1) * 0.3 +
metrics["findings_with_remediation"] / max(metrics["total_findings"], 1) * 0.2 +
min(metrics["coverage_percentage"] / 100, 1.0) * 0.2
)
return metrics
```
## Advanced SARIF Features
### Fixes and Remediation
```json
{
"ruleId": "semgrep.security.audit.sqli.pg-sqli",
"fixes": [
{
"description": {
"text": "Use parameterized queries to prevent SQL injection"
},
"artifactChanges": [
{
"artifactLocation": {
"uri": "src/database/queries.py"
},
"replacements": [
{
"deletedRegion": {
"startLine": 156,
"startColumn": 20,
"endLine": 156,
"endColumn": 45
},
"insertedContent": {
"text": "cursor.execute(query, params)"
}
}
]
}
]
}
]
}
```
### Code Flows for Complex Vulnerabilities
```json
{
"ruleId": "dataflow.taint.sql-injection",
"codeFlows": [
{
"message": {
"text": "Tainted data flows from user input to SQL query"
},
"threadFlows": [
{
"locations": [
{
"location": {
"physicalLocation": {
"artifactLocation": {"uri": "src/api/handlers.py"},
"region": {"startLine": 45}
}
},
"state": {"source": "user_input"},
"nestingLevel": 0
},
{
"location": {
"physicalLocation": {
"artifactLocation": {"uri": "src/database/queries.py"},
"region": {"startLine": 156}
}
},
"state": {"sink": "sql_query"},
"nestingLevel": 0
}
]
}
]
}
]
}
```
---
## SARIF Best Practices
### Result Quality
- **Precise Locations**: Always include accurate file paths and line numbers
- **Clear Messages**: Write descriptive, actionable finding messages
- **Remediation Guidance**: Include fix suggestions when possible
- **Severity Consistency**: Use consistent severity mappings across tools
### Performance
- **Efficient Processing**: Process SARIF results efficiently for large result sets
- **Streaming**: Use streaming for very large SARIF files
- **Caching**: Cache processed results for faster repeated access
- **Compression**: Compress SARIF files for storage and transmission
### Integration
- **Tool Interoperability**: Ensure SARIF compatibility with existing tools
- **Standard Compliance**: Follow SARIF 2.1.0 specification precisely
- **Extension Documentation**: Document any custom extensions clearly
- **Version Management**: Handle SARIF schema version differences
docs/docs/concept/security-analysis.md
+174
View File
@@ -0,0 +1,174 @@
# Security Analysis in FuzzForge: Concepts and Approach
Security analysis is at the core of FuzzForge’s mission. This page explains the philosophy, methodologies, and integration patterns that shape how FuzzForge discovers vulnerabilities and helps teams secure their software. If you’re curious about what “security analysis” really means in this platform—and why it’s designed this way—read on.
---
## Why Does FuzzForge Approach Security Analysis This Way?
FuzzForge’s security analysis is built on a few guiding principles:
- **Defense in Depth:** No single tool or method catches everything. FuzzForge layers multiple analysis types—static, dynamic, secret detection, infrastructure checks, and fuzzing—to maximize coverage.
- **Tool Diversity:** Different tools find different issues. Running several tools for each analysis type reduces blind spots and increases confidence in results.
- **Standardized Results:** All findings are normalized into SARIF, a widely adopted format. This makes results easy to aggregate, review, and integrate with other tools.
- **Automation and Integration:** Security analysis is only useful if it fits into real-world workflows. FuzzForge is designed for CI/CD, developer feedback, and automated reporting.
---
## What Types of Security Analysis Does FuzzForge Perform?
### Static Analysis
- **What it is:** Examines source code without running it, looking for vulnerabilities, anti-patterns, and risky constructs.
- **How it works:** Parses code, analyzes control and data flow, and matches patterns against known vulnerabilities.
- **Tools:** Semgrep, Bandit, CodeQL, ESLint, and more.
- **Strengths:** Fast, broad coverage, no runtime needed.
- **Limitations:** Can’t see runtime issues, may produce false positives.
### Dynamic Analysis
- **What it is:** Tests running applications to find vulnerabilities that only appear at runtime.
- **How it works:** Deploys the app in a test environment, probes entry points, and observes behavior under attack.
- **Tools:** Nuclei, OWASP ZAP, Nmap, SQLMap.
- **Strengths:** Finds real, exploitable issues; validates actual behavior.
- **Limitations:** Needs a working environment; slower; may not cover all code.
### Secret Detection
- **What it is:** Scans code and configuration for exposed credentials, API keys, and sensitive data.
- **How it works:** Uses pattern matching, entropy analysis, and context checks—sometimes even scanning git history.
- **Tools:** TruffleHog, Gitleaks, detect-secrets, GitGuardian.
- **Strengths:** Fast, critical for preventing leaks.
- **Limitations:** Can’t find encrypted/encoded secrets; needs regular pattern updates.
### Infrastructure Analysis
- **What it is:** Analyzes infrastructure-as-code, container configs, and deployment manifests for security misconfigurations.
- **How it works:** Parses config files, applies security policies, checks compliance, and assesses risk.
- **Tools:** Checkov, Hadolint, Kubesec, Terrascan.
- **Strengths:** Prevents misconfigurations before deployment; automates compliance.
- **Limitations:** Can’t see runtime changes; depends on up-to-date policies.
### Fuzzing
- **What it is:** Automatically generates and sends unexpected or random inputs to code, looking for crashes or unexpected behavior.
- **How it works:** Identifies targets, generates inputs, monitors execution, and analyzes crashes.
- **Tools:** AFL++, libFuzzer, Cargo Fuzz, Jazzer.
- **Strengths:** Finds deep, complex bugs; great for memory safety.
- **Limitations:** Resource-intensive; may need manual setup.
### Comprehensive Assessment
- **What it is:** Combines all the above for a holistic view, correlating findings and prioritizing risks.
- **How it works:** Runs multiple analyses, aggregates and correlates results, and generates unified reports.
- **Benefits:** Complete coverage, better context, prioritized remediation, and compliance support.
---
## How Does FuzzForge Integrate and Orchestrate Analysis?
### Workflow Composition
FuzzForge composes analysis workflows by combining different analysis types, each running in its own containerized environment. Inputs (code, configs, parameters) are fed into the appropriate tools, and results are normalized and aggregated.
```mermaid
graph TB
subgraph "Input"
Target[Target Codebase]
Config[Analysis Configuration]
end
subgraph "Analysis Workflows"
Static[Static Analysis]
Dynamic[Dynamic Analysis]
Secrets[Secret Detection]
Infra[Infrastructure Analysis]
Fuzz[Fuzzing Analysis]
end
subgraph "Processing"
Normalize[Result Normalization]
Merge[Finding Aggregation]
Correlate[Cross-Tool Correlation]
end
subgraph "Output"
SARIF[SARIF Results]
Report[Security Report]
Metrics[Analysis Metrics]
end
Target --> Static
Target --> Dynamic
Target --> Secrets
Target --> Infra
Target --> Fuzz
Config --> Static
Config --> Dynamic
Config --> Secrets
Config --> Infra
Config --> Fuzz
Static --> Normalize
Dynamic --> Normalize
Secrets --> Normalize
Infra --> Normalize
Fuzz --> Normalize
Normalize --> Merge
Merge --> Correlate
Correlate --> SARIF
Correlate --> Report
Correlate --> Metrics
```
### Orchestration Patterns
- **Parallel Execution:** Tools of the same type (e.g., multiple static analyzers) run in parallel for speed and redundancy.
- **Sequential Execution:** Some analyses depend on previous results (e.g., dynamic analysis using endpoints found by static analysis).
- **Result Normalization:** All findings are converted to SARIF for consistency.
- **Correlation:** Related findings from different tools are grouped and prioritized.
---
## How Is Quality Ensured?
### Metrics and Measurement
- **Coverage:** How much code, how many rules, and how many vulnerability types are analyzed.
- **Accuracy:** False positive/negative rates, confidence scores, and validation rates.
- **Performance:** Analysis duration, resource usage, and scalability.
### Quality Assurance
- **Cross-Tool Validation:** Findings are confirmed by multiple tools when possible.
- **Manual Review:** High-severity findings can be flagged for expert review.
- **Continuous Improvement:** Tools and rules are updated regularly, and user feedback is incorporated.
---
## How Does Security Analysis Fit Into Development Workflows?
### CI/CD Integration
- **Pre-commit Hooks:** Run security checks before code is committed.
- **Pipeline Integration:** Block deployments if high/critical issues are found.
- **Quality Gates:** Enforce severity thresholds and track trends over time.
### Developer Experience
- **IDE Integration:** Import SARIF findings into supported IDEs for inline feedback.
- **Real-Time Analysis:** Optionally run background checks during development.
- **Reporting:** Executive dashboards, technical reports, and compliance summaries.
---
## What’s Next for Security Analysis in FuzzForge?
FuzzForge is designed to evolve. Advanced techniques like machine learning for pattern recognition, contextual analysis, and business logic checks are on the roadmap. The goal: keep raising the bar for automated, actionable, and developer-friendly security analysis.
---
## In Summary
FuzzForge’s security analysis is comprehensive, layered, and designed for real-world integration. By combining multiple analysis types, normalizing results, and focusing on automation and developer experience, FuzzForge helps teams find and fix vulnerabilities—before attackers do.
docs/docs/concept/workflow.md
+128
View File
@@ -0,0 +1,128 @@
# Understanding Workflows in FuzzForge
Workflows are the backbone of FuzzForge’s security analysis platform. If you want to get the most out of FuzzForge, it’s essential to understand what workflows are, how they’re designed, and how they operate from start to finish. This page explains the core concepts, design principles, and execution models behind FuzzForge workflows—so you can use them confidently and effectively.
---
## What Is a Workflow?
A **workflow** in FuzzForge is a containerized process that orchestrates one or more security tools to analyze a target codebase or application. Each workflow is tailored for a specific type of security analysis (like static analysis, secret detection, or fuzzing) and is designed to be:
- **Isolated:** Runs in its own Docker container for security and reproducibility.
- **Integrated:** Can combine multiple tools for comprehensive results.
- **Standardized:** Always produces SARIF-compliant output.
- **Configurable:** Accepts parameters to customize analysis.
- **Scalable:** Can run in parallel and scale horizontally.
---
## How Does a Workflow Operate?
### High-Level Architecture
Here’s how a workflow moves through the FuzzForge system:
```mermaid
graph TB
User[User/CLI/API] --> API[FuzzForge API]
API --> Prefect[Prefect Orchestrator]
Prefect --> Worker[Prefect Worker]
Worker --> Container[Docker Container]
Container --> Tools[Security Tools]
Tools --> Results[SARIF Results]
Results --> Storage[Persistent Storage]
```
**Key roles:**
- **User/CLI/API:** Submits and manages workflows.
- **FuzzForge API:** Validates, orchestrates, and tracks workflows.
- **Prefect Orchestrator:** Schedules and manages workflow execution.
- **Prefect Worker:** Runs the workflow in a Docker container.
- **Security Tools:** Perform the actual analysis.
- **Persistent Storage:** Stores results and artifacts.
---
## Workflow Lifecycle: From Idea to Results
1. **Design:** Choose tools, define integration logic, set up parameters, and build the Docker image.
2. **Deployment:** Build and push the image, register the workflow, and configure defaults.
3. **Execution:** User submits a workflow; parameters and target are validated; the workflow is scheduled and executed in a container; tools run as designed.
4. **Completion:** Results are collected, normalized, and stored; status is updated; temporary resources are cleaned up; results are made available via API/CLI.
---
## Types of Workflows
FuzzForge supports several workflow types, each optimized for a specific security need:
- **Static Analysis:** Examines source code without running it (e.g., Semgrep, Bandit).
- **Dynamic Analysis:** Tests running applications for runtime vulnerabilities (e.g., OWASP ZAP, Nuclei).
- **Secret Detection:** Finds exposed credentials and sensitive data (e.g., TruffleHog, Gitleaks).
- **Infrastructure Analysis:** Checks infrastructure-as-code and configs for misconfigurations (e.g., Checkov, Hadolint).
- **Fuzzing:** Generates unexpected inputs to find crashes and edge cases (e.g., AFL++, libFuzzer).
- **Comprehensive Assessment:** Combines multiple analysis types for full coverage.
---
## Workflow Design Principles
- **Tool Agnostic:** Workflows abstract away the specifics of underlying tools, providing a consistent interface.
- **Fail-Safe Execution:** If one tool fails, others continue—partial results are still valuable.
- **Configurable:** Users can adjust parameters to control tool behavior, output, and execution.
- **Resource-Aware:** Workflows specify and respect resource limits (CPU, memory).
- **Standardized Output:** All results are normalized to SARIF for easy integration and reporting.
---
## Execution Models
- **Synchronous:** Wait for the workflow to finish and get results immediately—great for interactive use.
- **Asynchronous:** Submit a workflow and check back later for results—ideal for long-running or batch jobs.
- **Parallel:** Run multiple workflows at once for comprehensive or time-sensitive analysis.
---
## Data Flow and Storage
- **Input:** Target code and parameters are validated and mounted as read-only volumes.
- **Processing:** Tools are initialized and run (often in parallel); outputs are collected and normalized.
- **Output:** Results are stored in persistent volumes and indexed for fast retrieval; metadata is saved in the database; intermediate results may be cached for performance.
---
## Error Handling and Recovery
- **Tool-Level:** Timeouts, resource exhaustion, and crashes are handled gracefully; failed tools don’t stop the workflow.
- **Workflow-Level:** Container failures, volume issues, and network problems are detected and reported.
- **Recovery:** Automatic retries for transient errors; partial results are returned when possible; workflows degrade gracefully if some tools are unavailable.
---
## Performance and Optimization
- **Container Efficiency:** Docker images are layered and cached for fast startup; containers may be reused when safe.
- **Parallel Processing:** Independent tools run concurrently to maximize CPU usage and minimize wait times.
- **Caching:** Images, dependencies, and intermediate results are cached to avoid unnecessary recomputation.
---
## Monitoring and Observability
- **Metrics:** Track execution time, resource usage, and success/failure rates.
- **Logging:** Structured logs and tool outputs are captured for debugging and analysis.
- **Real-Time Monitoring:** Live status updates and progress indicators are available via API/WebSocket.
---
## Integration Patterns
- **CI/CD:** Integrate workflows into pipelines to block deployments on critical findings.
- **API:** Programmatically submit and track workflows from your own tools or scripts.
- **Event-Driven:** Use webhooks or event listeners to trigger actions on workflow completion.
---
## In Summary
Workflows in FuzzForge are designed to be robust, flexible, and easy to integrate into your security and development processes. By combining containerization, orchestration, and a standardized interface, FuzzForge workflows help you automate and scale security analysis—so you can focus on fixing issues, not just finding them.
docs/docs/concept/working-with-documentation.md
+72
View File
@@ -0,0 +1,72 @@
# Working with documentation
To update the documentation on any of the sections just add a new markdown file to the designated subfolder below :
```
├─concepts
├─tutorials
├─how-to
│ └─troubleshooting
└─reference
├─architecture
├─decisions
└─faq
```
:::note Templates
Each folder contains templates that can be used as quickstarts. Those are named `<template name>.tpml`.
:::
See [Diataxis documentation](../reference/diataxis-documentation.md) for more information on diátaxis.
## Manage Docs Versions
Docusaurus can manage multiple versions of the docs.
### Create a docs version
Release a version 1.0 of your project:
```bash
npm run docusaurus docs:version 1.0
```
The `docs` folder is copied into `versioned_docs/version-1.0` and `versions.json` is created.
Your docs now have 2 versions:
- `1.0` at `http://localhost:3000/docs/` for the version 1.0 docs
- `current` at `http://localhost:3000/docs/next/` for the **upcoming, unreleased docs**
### Add a Version Dropdown
To navigate seamlessly across versions, add a version dropdown.
Modify the `docusaurus.config.js` file:
```js title="docusaurus.config.js"
export default {
themeConfig: {
navbar: {
items: [
// highlight-start
{
type: 'docsVersionDropdown',
},
// highlight-end
],
},
},
};
```
The docs version dropdown appears in the navbar.
## Update an existing version
It is possible to edit versioned docs in their respective folder:
- `versioned_docs/version-1.0/hello.md` updates `http://localhost:3000/docs/hello`
- `docs/hello.md` updates `http://localhost:3000/docs/next/hello`
docs/docs/how-to/_category_.json
+8
View File
@@ -0,0 +1,8 @@
{
"label": "How-to Guides",
"position": 4,
"link": {
"type": "generated-index",
"description": "How-to guides pages that are goal-oriented."
}
}
docs/docs/how-to/artifact-workflows.md
+194
View File
@@ -0,0 +1,194 @@
# How-To: Artifact Workflows in FuzzForge AI
Artifacts are the heart of FuzzForge AI’s collaborative workflows. They let you generate, share, analyze, and improve files—code, configs, reports, and more—across agents and projects. This guide shows you, step by step, how to use artifacts for practical, productive automation.
---
## 1. What is an Artifact?
An **artifact** is any file or structured content created, processed, or shared by FuzzForge AI or its agents. Artifacts can be:
- Generated code, configs, or documentation
- Analysis results or transformed data
- Files shared between agents (including user uploads)
- Anything you want to track, review, or improve
Artifacts are stored locally (in `.fuzzforge/artifacts/`) and can be served over HTTP for agent-to-agent workflows.
---
## 2. Creating Artifacts
### a. Natural Language Generation
Just ask FuzzForge to create something, and it will generate an artifact automatically.
**Examples:**
```bash
You> Create a Python script to analyze log files for errors
You> Generate a secure Docker configuration for a web application
You> Write a security checklist for code reviews
You> Create a CSV template for vulnerability tracking
```
**System Response:**
```
ARTIFACT: script log_analyzer.py
```
The file is now available in your artifact list!
---
### b. Sharing Files with Agents
Send any file (including artifacts) to a registered agent for further analysis or processing.
**Command Syntax:**
```bash
/sendfile <AgentName> <file_path_or_artifact_id> "<optional note>"
```
**Examples:**
```bash
/sendfile SecurityAnalyzer ./src/authentication.py "Please check for security vulnerabilities"
/sendfile ReverseEngineer ./suspicious.exe "Analyze this binary for malware"
/sendfile ConfigReviewer artifact_abc123 "Optimize this configuration"
```
**What happens:**
1. FuzzForge reads the file or artifact
2. Creates (or references) an artifact
3. Generates an HTTP URL for the artifact
4. Sends an A2A message to the target agent
5. The agent downloads and processes the file
---
### c. Analyzing and Improving Artifacts
After creating an artifact, you can request further analysis or improvements.
**Examples:**
```bash
You> Create a REST API security configuration
You> Send this configuration to SecurityAnalyzer for review
You> ROUTE_TO ConfigExpert: Analyze the configuration I just created
```
---
## 3. Viewing and Managing Artifacts
### a. List All Artifacts
```bash
/artifacts
```
Displays a table of all artifacts, including IDs, filenames, sizes, and creation times.
### b. View a Specific Artifact
```bash
/artifacts <artifact_id>
```
Shows details, download URL, and a content preview.
---
## 4. Advanced Artifact Workflows
### a. Generate-Analyze-Improve Cycle
```bash
# Step 1: Generate initial artifact
You> Create a secure API configuration for user authentication
# Step 2: Send to expert agent for analysis
You> /sendfile SecurityExpert artifact_abc123 "Review this config for security issues"
# Step 3: Request improvements based on feedback
You> Based on the security feedback, create an improved version of the API configuration
```
### b. Multi-Agent Artifact Processing
```bash
# Create initial content
You> Generate a Python web scraper for security data
# Send to different specialists
You> /sendfile CodeReviewer artifact_def456 "Check code quality and best practices"
You> /sendfile SecurityAnalyzer artifact_def456 "Analyze for security vulnerabilities"
You> /sendfile PerformanceExpert artifact_def456 "Optimize for performance"
# Consolidate feedback
You> Create an improved version incorporating all the expert feedback
```
### c. Artifact-Based Agent Coordination
```bash
You> Create a comprehensive security assessment report template
# System creates artifact_ghi789
You> ROUTE_TO DocumentExpert: Enhance this report template with professional formatting
You> ROUTE_TO SecurityManager: Add compliance sections to this report template
You> ROUTE_TO DataAnalyst: Add metrics and visualization sections to this template
```
---
## 5. Artifact Commands Reference
| Command | Description | Example |
|---------|-------------|---------|
| `/artifacts` | List all artifacts | `/artifacts` |
| `/artifacts <id>` | View artifact details | `/artifacts artifact_abc123` |
| `/sendfile <agent> <path/id> "<note>"` | Send file or artifact to agent | `/sendfile Analyzer ./code.py "Review this"` |
---
## 6. Where Are Artifacts Stored?
Artifacts are stored in your project’s `.fuzzforge/artifacts/` directory and are accessible via HTTP when the A2A server is running.
**Example structure:**
```
.fuzzforge/
└── artifacts/
├── artifact_abc123
├── artifact_def456
└── artifact_ghi789
```
Each artifact gets a unique URL, e.g.:
```
http://localhost:10100/artifacts/artifact_abc123
```
---
## 7. Supported Artifact Types
| Type | Extensions | Example Requests |
|--------------------|--------------------------|-----------------------------------------|
| Python Scripts | `.py` | "Create a script to..." |
| Configurations | `.yaml`, `.json`, `.toml` | "Generate config for..." |
| Documentation | `.md`, `.txt` | "Write documentation for..." |
| SQL Queries | `.sql` | "Write a query to..." |
| Shell Scripts | `.sh`, `.bat` | "Create a deployment script..." |
| Web Files | `.html`, `.css`, `.js` | "Create a webpage..." |
| Data Files | `.csv`, `.xml` | "Create a data template..." |
---
## 8. Tips & Best Practices
- Use clear, descriptive requests for artifact generation.
- Reference artifact IDs when sending files between agents for traceability.
- Combine artifact workflows with knowledge graph and memory features for maximum productivity.
- Artifacts are project-scoped—keep sensitive data within your project boundaries.
---
Artifacts make FuzzForge AI a powerful, collaborative automation platform. Experiment, share, and build smarter workflows—one artifact at a time!
docs/docs/how-to/create-module.md
+167
View File
@@ -0,0 +1,167 @@
# How to Create a Custom Module in FuzzForge
This guide will walk you through the process of developing a custom security analysis module for FuzzForge. Modules are the building blocks of FuzzForge workflows, enabling you to add new analysis capabilities or extend existing ones.
---
## Prerequisites
Before you start, make sure you have:
- A working FuzzForge development environment (see [Contributing](/reference/contributing.md))
- Familiarity with Python and async programming
- Basic understanding of Docker and the FuzzForge architecture
---
## Step 1: Understand the Module Architecture
All FuzzForge modules inherit from a common `BaseModule` interface and use Pydantic models for type safety and result standardization.
**Key components:**
- `BaseModule`: Abstract base class for all modules
- `ModuleFinding`: Represents a single finding
- `ModuleResult`: Standardized result format for module execution
- `ModuleMetadata`: Describes module capabilities and requirements
Modules are located in `backend/src/toolbox/modules/`.
---
## Step 2: Create Your Module File
Let’s create a simple example: a **License Scanner** module that detects license files and extracts license information.
Create a new file:
`backend/src/toolbox/modules/license_scanner.py`
```python
import re
from pathlib import Path
from typing import Dict, Any, List
from .base import BaseModule, ModuleResult, ModuleMetadata, ModuleFinding
class LicenseScanner(BaseModule):
"""Scans for license files and extracts license information"""
LICENSE_PATTERNS = {
'MIT': r'MIT License|Permission is hereby granted',
'Apache-2.0': r'Apache License|Version 2\\.0',
'GPL-3.0': r'GNU GENERAL PUBLIC LICENSE|Version 3',
'BSD-3-Clause': r'BSD 3-Clause|Redistribution and use',
}
LICENSE_FILES = [
'LICENSE', 'LICENSE.txt', 'LICENSE.md',
'COPYING', 'COPYRIGHT'
]
def get_metadata(self) -> ModuleMetadata:
return ModuleMetadata(
name="License Scanner",
version="1.0.0",
description="Scans for license files and extracts license information",
category="scanner",
tags=["license", "compliance"]
)
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
findings = []
for license_name in self.LICENSE_FILES:
license_file = workspace / license_name
if license_file.is_file():
content = license_file.read_text(encoding='utf-8', errors='ignore')
detected = self._detect_license_type(content)
findings.append(self.create_finding(
title=f"License file: {detected or 'Unknown'}",
description=f"Found license file with {detected or 'unknown'} license",
severity="info",
category="license",
file_path=str(license_file.relative_to(workspace)),
metadata={'license_type': detected}
))
return self.create_result(findings=findings)
def _detect_license_type(self, content: str) -> str:
for license_type, pattern in self.LICENSE_PATTERNS.items():
if re.search(pattern, content, re.IGNORECASE):
return license_type
return 'Unknown'
def validate_config(self, config: Dict[str, Any]) -> bool:
# No required config for this simple example
return True
```
---
## Step 3: Register Your Module
Add your module to `backend/src/toolbox/modules/__init__.py`:
```python
from .license_scanner import LicenseScanner
__all__ = ['LicenseScanner']
```
---
## Step 4: Test Your Module
Create a test file (e.g., `test_license_scanner.py`) and run your module against a sample workspace:
```python
import asyncio
from pathlib import Path
from backend.src.toolbox.modules.license_scanner import LicenseScanner
async def main():
workspace = Path("/path/to/your/test/project")
scanner = LicenseScanner()
result = await scanner.execute({}, workspace)
for finding in result.findings:
print(f"{finding.file_path}: {finding.metadata.get('license_type')}")
if __name__ == "__main__":
asyncio.run(main())
```
---
## Module Types
FuzzForge supports several module types:
- **Scanner Modules:** Discover files, extract metadata (e.g., license scanner, dependency scanner)
- **Analyzer Modules:** Perform deep security analysis (e.g., static analyzer, secret detector)
- **Reporter Modules:** Format and output results (e.g., SARIF reporter, JSON reporter)
Each module type follows the same interface but focuses on a different stage of the workflow.
---
## Best Practices
- **Error Handling:** Never let a single file or tool error stop the whole module. Log errors and continue.
- **Async Operations:** Use async/await for file and network operations to maximize performance.
- **Configuration:** Validate all config parameters and provide sensible defaults.
- **Resource Limits:** Respect memory and CPU limits; process files in batches if needed.
- **Security:** Never execute untrusted code; sanitize file paths and inputs.
---
## Testing Your Module
- Write unit tests for your module’s logic and edge cases.
- Test integration by running your module as part of a FuzzForge workflow.
- Use temporary directories and mock files to simulate real-world scenarios.
---
## Advanced Tips
- Use Pydantic models for robust config validation.
- Implement progress reporting for long-running modules.
- Compose modules by orchestrating multiple sub-modules for complex analysis.
docs/docs/how-to/create-workflow.md
+350
View File
@@ -0,0 +1,350 @@
# How to Create a Custom Workflow in FuzzForge
This guide will walk you through the process of creating a custom security analysis workflow in FuzzForge. Workflows orchestrate modules, define the analysis pipeline, and enable you to automate complex security checks for your codebase or application.
---
## Prerequisites
Before you start, make sure you have:
- A working FuzzForge development environment (see [Contributing](/reference/contributing.md))
- Familiarity with Python (async/await), Docker, and Prefect 3
- At least one custom or built-in module to use in your workflow
---
## Step 1: Understand Workflow Architecture
A FuzzForge workflow is a Prefect 3 flow that:
- Runs in an isolated Docker container
- Orchestrates one or more analysis modules (scanner, analyzer, reporter, etc.)
- Handles secure volume mounting for code and results
- Produces standardized SARIF output
- Supports configurable parameters and resource limits
**Directory structure:**
```
backend/toolbox/workflows/{workflow_name}/
├── workflow.py # Main workflow definition (Prefect flow)
├── Dockerfile # Container image definition
├── metadata.yaml # Workflow metadata and configuration
└── requirements.txt # Additional Python dependencies (optional)
```
---
## Step 2: Define Workflow Metadata
Create a `metadata.yaml` file in your workflow directory. This file describes your workflow, its parameters, and resource requirements.
Example:
```yaml
name: dependency_analysis
version: "1.0.0"
description: "Analyzes project dependencies for security vulnerabilities"
author: "FuzzingLabs Security Team"
category: "comprehensive"
tags:
- "dependency-scanning"
- "vulnerability-analysis"
requirements:
tools:
- "dependency_scanner"
- "vulnerability_analyzer"
- "sarif_reporter"
resources:
memory: "512Mi"
cpu: "1000m"
timeout: 1800
parameters:
type: object
properties:
target_path:
type: string
default: "/workspace"
description: "Path to analyze"
scan_dev_dependencies:
type: boolean
description: "Include development dependencies"
vulnerability_threshold:
type: string
enum: ["low", "medium", "high", "critical"]
description: "Minimum vulnerability severity to report"
output_schema:
type: object
properties:
sarif:
type: object
description: "SARIF-formatted security findings"
summary:
type: object
description: "Scan execution summary"
```
---
## Step 3: Add Live Statistics to Your Workflow 🚦
Want real-time progress and stats for your workflow? FuzzForge supports live statistics reporting using Prefect and structured logging. This lets users (and the platform) monitor workflow progress, see live updates, and stream stats via API or WebSocket.
### 1. Import Required Dependencies
```python
from prefect import task, get_run_context
import logging
logger = logging.getLogger(__name__)
```
### 2. Create a Statistics Callback Function
Add a callback that logs structured stats updates:
```python
@task(name="my_workflow_task")
async def my_workflow_task(workspace: Path, config: Dict[str, Any]) -> Dict[str, Any]:
# Get run context for statistics reporting
try:
context = get_run_context()
run_id = str(context.flow_run.id)
logger.info(f"Running task for flow run: {run_id}")
except Exception:
run_id = None
logger.warning("Could not get run context for statistics")
# Define callback function for live statistics
async def stats_callback(stats_data: Dict[str, Any]):
"""Callback to handle live statistics"""
try:
# Log structured statistics data for the backend to parse
logger.info("LIVE_STATS", extra={
"stats_type": "live_stats", # Type of statistics
"workflow_type": "my_workflow", # Your workflow name
"run_id": stats_data.get("run_id"),
# Add your custom statistics fields here:
"progress": stats_data.get("progress", 0),
"items_processed": stats_data.get("items_processed", 0),
"errors": stats_data.get("errors", 0),
"elapsed_time": stats_data.get("elapsed_time", 0),
"timestamp": stats_data.get("timestamp")
})
except Exception as e:
logger.warning(f"Error in stats callback: {e}")
# Pass callback to your module/processor
processor = MyWorkflowModule()
result = await processor.execute(config, workspace, stats_callback=stats_callback)
return result.dict()
```
### 3. Update Your Module to Use the Callback
```python
class MyWorkflowModule:
async def execute(self, config: Dict[str, Any], workspace: Path, stats_callback=None):
# Your processing logic here
# Periodically send statistics updates
if stats_callback:
await stats_callback({
"run_id": run_id,
"progress": current_progress,
"items_processed": processed_count,
"errors": error_count,
"elapsed_time": elapsed_seconds,
"timestamp": datetime.utcnow().isoformat()
})
```
### 4. Supported Statistics Types
The monitor recognizes these `stats_type` values:
- `"fuzzing_live_update"` - For fuzzing workflows (uses FuzzingStats model)
- `"scan_progress"` - For security scanning workflows
- `"analysis_update"` - For code analysis workflows
- `"live_stats"` - Generic live statistics for any workflow
#### Example: Fuzzing Workflow Stats
```python
"stats_type": "fuzzing_live_update",
"executions": 12345,
"executions_per_sec": 1500.0,
"crashes": 2,
"unique_crashes": 2,
"corpus_size": 45,
"coverage": 78.5,
"elapsed_time": 120
```
#### Example: Scanning Workflow Stats
```python
"stats_type": "scan_progress",
"files_scanned": 150,
"vulnerabilities_found": 8,
"scan_percentage": 65.2,
"current_file": "/path/to/file.js",
"elapsed_time": 45
```
#### Example: Analysis Workflow Stats
```python
"stats_type": "analysis_update",
"functions_analyzed": 89,
"issues_found": 12,
"complexity_score": 7.8,
"current_module": "authentication",
"elapsed_time": 30
```
### 5. API Integration
Live statistics automatically appear in:
- **REST API**: `GET /fuzzing/{run_id}/stats` (for fuzzing workflows)
- **WebSocket**: Real-time updates via WebSocket connections
- **Server-Sent Events**: Live streaming at `/fuzzing/{run_id}/stream`
### 6. Best Practices
1. **Update Frequency**: Send statistics every 5-10 seconds for optimal performance.
2. **Error Handling**: Always wrap stats callbacks in try-catch blocks.
3. **Meaningful Data**: Include workflow-specific metrics that users care about.
4. **Consistent Naming**: Use consistent field names across similar workflow types.
5. **Backwards Compatibility**: Keep existing stats types when updating workflows.
#### Example: Adding Stats to a Security Scanner
```python
async def security_scan_task(workspace: Path, config: Dict[str, Any]):
context = get_run_context()
run_id = str(context.flow_run.id)
async def stats_callback(stats_data):
logger.info("LIVE_STATS", extra={
"stats_type": "scan_progress",
"workflow_type": "security_scan",
"run_id": stats_data.get("run_id"),
"files_scanned": stats_data.get("files_scanned", 0),
"vulnerabilities_found": stats_data.get("vulnerabilities_found", 0),
"scan_percentage": stats_data.get("scan_percentage", 0.0),
"current_file": stats_data.get("current_file", ""),
"elapsed_time": stats_data.get("elapsed_time", 0)
})
scanner = SecurityScannerModule()
return await scanner.execute(config, workspace, stats_callback=stats_callback)
```
With these steps, your workflow will provide rich, real-time feedback to users and the FuzzForge platform—making automation more transparent and interactive!
---
## Step 4: Implement the Workflow Logic
Create a `workflow.py` file. This is where you define your Prefect flow and tasks.
Example (simplified):
```python
from pathlib import Path
from typing import Dict, Any
from prefect import flow, task
from src.toolbox.modules.dependency_scanner import DependencyScanner
from src.toolbox.modules.vulnerability_analyzer import VulnerabilityAnalyzer
from src.toolbox.modules.reporter import SARIFReporter
@task
async def scan_dependencies(workspace: Path, config: Dict[str, Any]) -> Dict[str, Any]:
scanner = DependencyScanner()
return (await scanner.execute(config, workspace)).dict()
@task
async def analyze_vulnerabilities(dependencies: Dict[str, Any], workspace: Path, config: Dict[str, Any]) -> Dict[str, Any]:
analyzer = VulnerabilityAnalyzer()
analyzer_config = {**config, 'dependencies': dependencies.get('findings', [])}
return (await analyzer.execute(analyzer_config, workspace)).dict()
@task
async def generate_report(dep_results: Dict[str, Any], vuln_results: Dict[str, Any], config: Dict[str, Any], workspace: Path) -> Dict[str, Any]:
reporter = SARIFReporter()
all_findings = dep_results.get("findings", []) + vuln_results.get("findings", [])
reporter_config = {**config, "findings": all_findings}
return (await reporter.execute(reporter_config, workspace)).dict().get("sarif", {})
@flow(name="dependency_analysis")
async def main_flow(
target_path: str = "/workspace",
scan_dev_dependencies: bool = True,
vulnerability_threshold: str = "medium"
) -> Dict[str, Any]:
workspace = Path(target_path)
scanner_config = {"scan_dev_dependencies": scan_dev_dependencies}
analyzer_config = {"vulnerability_threshold": vulnerability_threshold}
reporter_config = {}
dep_results = await scan_dependencies(workspace, scanner_config)
vuln_results = await analyze_vulnerabilities(dep_results, workspace, analyzer_config)
sarif_report = await generate_report(dep_results, vuln_results, reporter_config, workspace)
return sarif_report
```
---
## Step 5: Create the Dockerfile
Your workflow runs in a container. Create a `Dockerfile`:
```dockerfile
FROM python:3.11-slim
WORKDIR /app
RUN apt-get update && apt-get install -y git curl && rm -rf /var/lib/apt/lists/*
COPY ../../../pyproject.toml ./
COPY ../../../uv.lock ./
RUN pip install uv && uv sync --no-dev
COPY requirements.txt ./
RUN uv pip install -r requirements.txt
COPY ../../../ .
RUN mkdir -p /workspace
CMD ["uv", "run", "python", "-m", "src.toolbox.workflows.dependency_analysis.workflow"]
```
---
## Step 6: Register and Test Your Workflow
- Add your workflow to the registry (e.g., `backend/toolbox/workflows/registry.py`)
- Write a test script or use the CLI to submit a workflow run
- Check that SARIF results are produced and stored as expected
Example test:
```python
import asyncio
from backend.src.toolbox.workflows.dependency_analysis.workflow import main_flow
async def test_workflow():
result = await main_flow(target_path="/tmp/test-project", scan_dev_dependencies=True)
print(result)
if __name__ == "__main__":
asyncio.run(test_workflow())
```
---
## Best Practices
- **Parameterize everything:** Use metadata.yaml to define all configurable options.
- **Validate inputs:** Check that paths, configs, and parameters are valid before running analysis.
- **Handle errors gracefully:** Catch exceptions in tasks and return partial results if possible.
- **Document your workflow:** Add docstrings and comments to explain each step.
- **Test with real and edge-case projects:** Ensure your workflow is robust and reliable.
docs/docs/how-to/docker-setup.md
+189
View File
@@ -0,0 +1,189 @@
# How to Configure Docker for FuzzForge
Getting Docker set up correctly is essential for running FuzzForge workflows. This guide will walk you through the process, explain why each step matters, and help you troubleshoot common issues—so you can get up and running with confidence.
---
## Why Does FuzzForge Need Special Docker Configuration?
FuzzForge builds and runs custom workflow images using a local Docker registry at `localhost:5001`. By default, Docker only trusts secure (HTTPS) registries, so you need to explicitly allow this local, insecure registry for development. Without this, workflows that build or pull images will fail.
---
## Quick Setup: The One-Liner
Add this to your Docker daemon configuration:
```json
{
"insecure-registries": ["localhost:5001"]
}
```
After editing, **restart Docker** for changes to take effect.
---
## Step-by-Step: Platform-Specific Instructions
### Docker Desktop (macOS & Windows)
#### Using the Docker Desktop UI
1. Open Docker Desktop.
2. Go to **Settings** (Windows) or **Preferences** (macOS).
3. Navigate to **Docker Engine**.
4. Add or update the `"insecure-registries"` section as shown above.
5. Click **Apply & Restart**.
6. Wait for Docker to restart (this may take a minute).
#### Editing the Config File Directly
- **macOS:** Edit `~/.docker/daemon.json`
- **Windows:** Edit `%USERPROFILE%\.docker\daemon.json`
Add or update the `"insecure-registries"` entry, then restart Docker Desktop.
---
### Docker Engine (Linux)
1. Edit (or create) `/etc/docker/daemon.json`:
```bash
sudo nano /etc/docker/daemon.json
```
2. Add:
```json
{
"insecure-registries": ["localhost:5001"]
}
```
3. Restart Docker:
```bash
sudo systemctl restart docker
```
4. Confirm Docker is running:
```bash
sudo systemctl status docker
```
#### Alternative: Systemd Drop-in (Advanced)
If you prefer, you can use a systemd override to add the registry flag. See the original guide for details.
---
## Verifying Your Configuration
1. **Check Docker’s registry settings:**
```bash
docker info | grep -i "insecure registries"
```
You should see `localhost:5001` listed.
2. **Test the registry:**
```bash
curl -f http://localhost:5001/v2/ && echo "✅ Registry accessible" || echo "❌ Registry not accessible"
```
3. **Try pushing and pulling an image:**
```bash
docker pull hello-world
docker tag hello-world localhost:5001/hello-world:test
docker push localhost:5001/hello-world:test
docker rmi localhost:5001/hello-world:test
docker pull localhost:5001/hello-world:test
```
---
## Common Issues & How to Fix Them
### "x509: certificate signed by unknown authority"
- **What’s happening?** Docker is trying to use HTTPS for the registry.
- **How to fix:** Double-check your `"insecure-registries"` config and restart Docker.
### "connection refused" to localhost:5001
- **What’s happening?** The registry isn’t running or the port is blocked.
- **How to fix:** Make sure FuzzForge services are up (`docker compose ps`), and that nothing else is using port 5001.
### Docker Desktop doesn’t apply settings
- **How to fix:** Fully quit and restart Docker Desktop. Check for typos in your JSON config.
### "permission denied" on Linux
- **How to fix:** Add your user to the `docker` group:
```bash
sudo usermod -aG docker $USER
newgrp docker
```
---
## Security Notes
- Using an insecure registry on `localhost:5001` is safe for local development.
- For production, always use a secure (HTTPS) registry and proper authentication.
---
## Where Are Docker Config Files?
- **Docker Desktop (macOS):** `~/.docker/daemon.json`
- **Docker Desktop (Windows):** `%USERPROFILE%\.docker\daemon.json`
- **Docker Engine (Linux):** `/etc/docker/daemon.json`
**Tip:** Always back up your config before making changes.
---
## Advanced: Example Configurations
### Minimal
```json
{
"insecure-registries": ["localhost:5001"]
}
```
### With Logging and Storage Options
```json
{
"insecure-registries": ["localhost:5001"],
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"storage-driver": "overlay2"
}
```
### Multiple Registries
```json
{
"insecure-registries": [
"localhost:5001",
"192.168.1.100:5000",
"registry.internal:5000"
]
}
```
---
## Next Steps
- [Getting Started Guide](../tutorial/getting-started.md): Continue with FuzzForge setup.
- [Troubleshooting](troubleshooting.md): For more help if things don’t work.
---
**Remember:**
After any Docker config change, always restart Docker and verify your settings with `docker info` before running FuzzForge.
docs/docs/how-to/how-to.tmpl
+18
View File
@@ -0,0 +1,18 @@
# How to {Task Name, use Verb Noun syntax. e.g. How to Open a can}
This guide will walk you through the steps required to {brief description of the task} in a code repository.
<!-- This is an optional element. Feel free to remove. -->
## Prerequisites
Before you start, make sure you have:
- {Prerequisite 1}
- [Performed {Prerequisite concept 2}](/tutorials/basic-tutorial-tmpl)
- {Additional prerequisites as necessary}
<!-- This is an optional element. Feel free to remove. -->
<!-- This is a repeated element. repeat when necessary. -->
## Step {step nr.}: {Step Title}
{Detailed instructions for step, including any relevant commands, code snippets, or screenshots.}
docs/docs/how-to/mcp-integration.md
+224
View File
@@ -0,0 +1,224 @@
# How-To: Integrate and Use MCP (Model Context Protocol) with FuzzForge
FuzzForge supports the Model Context Protocol (MCP), enabling LLM clients and AI assistants to interact directly with the security testing platform. This guide walks you through setting up, connecting, and using MCP with FuzzForge for automated security scans, results analysis, and intelligent recommendations.
---
## 🚀 What is MCP?
**MCP (Model Context Protocol)** is a standard that allows AI models and clients (like Claude, GPT, or custom agents) to interact with backend tools and APIs in a structured, tool-oriented way. With FuzzForge’s MCP integration, all FastAPI endpoints become MCP-compatible tools, making security automation accessible to any MCP-aware client.
---
## 1. Prerequisites
- FuzzForge installed and running (see [Getting Started](../tutorial/getting-started.md))
- Docker and Docker Compose installed (for containerized deployment)
- An MCP-compatible client (LLM, custom agent, or CLI tool)
---
## 2. Start FuzzForge with MCP Support
From your project root, launch the platform using Docker Compose:
```bash
docker compose up -d
```
This starts the backend API and the MCP gateway.
---
## 3. Verify MCP Integration
Check that the API and MCP endpoints are live:
```bash
# API status
curl http://localhost:8000/
# List available OpenAPI endpoints (now MCP-enabled)
curl http://localhost:8000/openapi.json | jq '.paths | keys'
# MCP HTTP endpoint
curl http://localhost:8010/mcp
```
You should see status responses and endpoint listings.
---
## 4. MCP Endpoints and Tools
### MCP Endpoints
- **HTTP MCP endpoint:** `http://localhost:8010/mcp`
- **SSE (Server-Sent Events):** `http://localhost:8010/mcp/sse`
- **Base API:** `http://localhost:8000`
### FastAPI Endpoints (now MCP tools)
- `GET /` — API status
- `GET /workflows/` — List available workflows
- `POST /workflows/{workflow_name}/submit` — Submit security scans
- `GET /runs/{run_id}/status` — Check scan status
- `GET /runs/{run_id}/findings` — Get scan results
- `GET /fuzzing/{run_id}/stats` — Fuzzing statistics
### MCP-Specific Tools
- `submit_security_scan_mcp` — Submit security scanning workflows
- `get_comprehensive_scan_summary` — Get detailed scan analysis with recommendations
---
## 5. Usage Examples
### Example 1: Submit a Security Scan via MCP
```json
{
"tool": "submit_security_scan_mcp",
"parameters": {
"workflow_name": "infrastructure_scan",
"target_path": "/path/to/your/project",
"volume_mode": "ro",
"parameters": {
"checkov_config": {
"severity": ["HIGH", "MEDIUM", "LOW"]
},
"hadolint_config": {
"severity": ["error", "warning", "info", "style"]
}
}
}
}
```
### Example 2: Get a Comprehensive Scan Summary
```json
{
"tool": "get_comprehensive_scan_summary",
"parameters": {
"run_id": "your-run-id-here"
}
}
```
---
## 6. Available Workflows
You can trigger these workflows via MCP:
1. **infrastructure_scan** — Docker/Kubernetes/Terraform security analysis
2. **static_analysis_scan** — Code vulnerability detection
3. **secret_detection_scan** — Credential and secret scanning
4. **penetration_testing_scan** — Network and web app testing
5. **security_assessment** — Comprehensive security evaluation
List all workflows:
```bash
curl http://localhost:8000/workflows/
```
---
## 7. MCP Client Configuration
For clients that require config files, use:
```json
{
"mcpServers": {
"fuzzforge": {
"command": "curl",
"args": ["-X", "POST", "http://localhost:8010/mcp"],
"env": {}
}
}
}
```
---
## 8. Integration Benefits
- **AI-Powered Security Testing:** LLMs can submit scans, interpret findings, and provide recommendations.
- **Direct API Access:** All FastAPI endpoints are available as MCP tools.
- **Real-Time Results:** Stream scan progress and results to AI clients.
- **Intelligent Analysis:** AI can generate reports, prioritize vulnerabilities, and track improvements.
---
## 9. Advanced Usage
- **Custom MCP Tools:** Enhanced tools provide intelligent summarization, contextual recommendations, and progress tracking.
- **Docker Compose Integration:** MCP tools work seamlessly in containerized environments with automatic service discovery and volume mapping.
- **Health Monitoring:** MCP clients can verify system health via `/health` endpoints.
---
## 10. Troubleshooting
### MCP Connection Failed
```bash
# Check backend status
docker compose ps fuzzforge-backend
curl http://localhost:8000/health
```
### Workflows Not Found
```bash
curl http://localhost:8000/workflows/
```
### Scan Submission Errors
```bash
curl -X POST http://localhost:8000/workflows/infrastructure_scan/submit \
-H "Content-Type: application/json" \
-d '{"target_path": "/your/path", "volume_mode": "ro"}'
```
### General Support
- Check Docker Compose logs: `docker compose logs fuzzforge-backend`
- Verify MCP endpoint: `curl http://localhost:8010/mcp`
- Test FastAPI endpoints directly before using MCP
---
## 11. Architecture Overview
```
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ MCP Client │───▶│ FastMCP │───▶│ FuzzForge │
│ (LLM/AI) │ │ Integration │ │ API │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌─────────────────┐
│ MCP Tools │ │ Prefect │
│ - scan submit │ │ Workflows │
│ - results │ │ - Security │
│ - analysis │ │ - Fuzzing │
└──────────────────┘ └─────────────────┘
```
---
## 12. Further Reading
- [Reference: CLI and API](../reference/cli-ai.md)
- [How-To: Create a Custom Workflow](./create-workflow.md)
- [FuzzForge Concepts](../concept/fuzzforge-ai.md)
---
With MCP, FuzzForge becomes a powerful, AI-friendly security automation platform. Connect your favorite LLM, automate security scans, and get actionable insights—all with a few API calls!
docs/docs/how-to/troubleshooting.md
+275
View File
@@ -0,0 +1,275 @@
# Troubleshooting FuzzForge
Running into issues with FuzzForge? This guide will help you diagnose and resolve the most common problems—whether you’re just getting started or running complex workflows. Each section is focused on a specific area, with actionable steps and explanations.
---
## Quick Checks: Is Everything Running?
Before diving into specific errors, let’s check the basics:
```bash
# Check all FuzzForge services
docker compose ps
# Verify Docker registry config
docker info | grep -i "insecure registries"
# Test service health endpoints
curl http://localhost:8000/health
curl http://localhost:4200
curl http://localhost:5001/v2/
```
If any of these commands fail, note the error message and continue below.
---
## Docker Registry Problems
### "x509: certificate signed by unknown authority"
**What’s happening?**
Docker is trying to use HTTPS for the local registry, but it’s set up for HTTP.
**How to fix:**
1. Add this to your Docker daemon config (e.g., `/etc/docker/daemon.json`):
```json
{
"insecure-registries": ["localhost:5001"]
}
```
2. Restart Docker.
3. Confirm with:
```bash
docker info | grep -A 5 -i "insecure registries"
```
### "connection refused" to localhost:5001
**What’s happening?**
The registry isn’t running or the port is blocked.
**How to fix:**
- Make sure the registry container is up:
```bash
docker compose ps registry
```
- Check logs for errors:
```bash
docker compose logs registry
```
- If port 5001 is in use, change it in `docker-compose.yaml` and your Docker config.
### "no such host" error
**What’s happening?**
Docker can’t resolve `localhost`.
**How to fix:**
- Try using `127.0.0.1` instead of `localhost` in your Docker config.
- Check your `/etc/hosts` file for a correct `127.0.0.1 localhost` entry.
---
## Workflow Execution Issues
### "mounts denied" or volume errors
**What’s happening?**
Docker can’t access the path you provided.
**How to fix:**
- Always use absolute paths.
- On Docker Desktop, add your project directory to File Sharing.
- Confirm the path exists and is readable.
### Workflow status is "Crashed" or "Late"
**What’s happening?**
- "Crashed": Usually a registry, path, or tool error.
- "Late": Worker is overloaded or system is slow.
**How to fix:**
- Check logs for details:
```bash
docker compose logs prefect-worker | tail -50
```
- Restart services:
```bash
docker compose down
docker compose up -d
```
- Reduce the number of concurrent workflows if your system is resource-constrained.
---
## Service Connectivity Issues
### Backend (port 8000) or Prefect UI (port 4200) not responding
**How to fix:**
- Check if the service is running:
```bash
docker compose ps fuzzforge-backend
docker compose ps prefect-server
```
- View logs for errors:
```bash
docker compose logs fuzzforge-backend --tail 50
docker compose logs prefect-server --tail 20
```
- Restart the affected service:
```bash
docker compose restart fuzzforge-backend
```
---
## CLI Issues
### "fuzzforge: command not found"
**How to fix:**
- Install the CLI:
```bash
cd cli
pip install -e .
```
or
```bash
uv tool install .
```
- Check your PATH:
```bash
which fuzzforge
echo $PATH
```
- As a fallback:
```bash
python -m fuzzforge_cli --help
```
### CLI connection errors
**How to fix:**
- Make sure the backend is running and healthy.
- Check your CLI config:
```bash
fuzzforge config show
```
- Update the server URL if needed:
```bash
fuzzforge config set-server http://localhost:8000
```
---
## System Resource Issues
### Out of disk space
**How to fix:**
- Clean up Docker:
```bash
docker system prune -f
docker image prune -f
```
- Remove old workflow images:
```bash
docker images | grep localhost:5001 | awk '{print $3}' | xargs docker rmi -f
```
### High memory usage
**How to fix:**
- Limit the number of concurrent workflows.
- Add swap space if possible.
- Restart services to free up memory.
---
## Network Issues
### Services can’t communicate
**How to fix:**
- Check Docker network configuration:
```bash
docker network ls
docker network inspect fuzzforge_alpha_default
```
- Recreate the network:
```bash
docker compose down
docker network prune -f
docker compose up -d
```
---
## Workflow-Specific Issues
### Static analysis or secret detection finds no issues
**What’s happening?**
- Your code may be clean, or the workflow isn’t scanning the right files.
**How to fix:**
- Make sure your target contains files to analyze:
```bash
find /path/to/target -name "*.py" -o -name "*.js" -o -name "*.java" | head -10
```
- Test with a known-vulnerable project or file.
---
## Getting Help and Diagnostics
### Enable debug logging
```bash
export PREFECT_LOGGING_LEVEL=DEBUG
docker compose down
docker compose up -d
docker compose logs fuzzforge-backend -f
```
### Collect diagnostic info
Save and run this script to gather info for support:
```bash
#!/bin/bash
echo "=== FuzzForge Diagnostics ==="
date
docker compose ps
docker info | grep -A 5 -i "insecure registries"
curl -s http://localhost:8000/health || echo "Backend unhealthy"
curl -s http://localhost:4200 >/dev/null && echo "Prefect UI healthy" || echo "Prefect UI unhealthy"
curl -s http://localhost:5001/v2/ >/dev/null && echo "Registry healthy" || echo "Registry unhealthy"
docker compose logs --tail 10
```
### Still stuck?
- Check the [FAQ](#) (not yet available)
- Review the [Getting Started guide](../tutorial/getting-started.md)
- Submit an issue with your diagnostics output
- Join the community or check for similar issues
---
## Prevention & Maintenance Tips
- Regularly clean up Docker images and containers:
```bash
docker system prune -f
```
- Monitor disk space and memory usage.
- Back up your configuration files (`docker-compose.yaml`, `.env`, `daemon.json`).
- Add health checks to your monitoring scripts.
---
If you have a persistent or unusual issue, don’t hesitate to reach out with logs and details. FuzzForge is designed to be robust, but every environment is unique—and your feedback helps make it better!
docs/docs/intro.md
+7
View File
@@ -0,0 +1,7 @@
---
sidebar_position: 1
---
# Introduction
Let's discover **FuzzForge in less than 5 minutes**.
docs/docs/reference/_category_.json
+8
View File
@@ -0,0 +1,8 @@
{
"label": "Reference",
"position": 5,
"link": {
"type": "generated-index",
"description": "Reference pages that are information-oriented."
}
}
docs/docs/reference/cli-ai.md
+305
View File
@@ -0,0 +1,305 @@
# FuzzForge AI Reference: CLI, Environment, and API
Welcome to the FuzzForge AI Reference! This document provides a comprehensive, no-nonsense guide to all the commands, environment variables, and API endpoints you’ll need to master the FuzzForge AI system. Use this as your quick lookup for syntax, options, and integration details.
---
## CLI Commands Reference
| Command | Description | Example |
|---------|-------------|---------|
| `/register <url>` | Register an A2A agent | `/register http://localhost:10201` |
| `/unregister <name>` | Remove a registered agent | `/unregister CalculatorAgent` |
| `/list` | Show all registered agents | `/list` |
| `/memory [action]` | Knowledge graph operations | `/memory search security` |
| `/recall <query>` | Search conversation history | `/recall past calculations` |
| `/artifacts [id]` | List or view artifacts | `/artifacts artifact_abc123` |
| `/tasks [id]` | Show task status | `/tasks task_001` |
| `/skills` | Display FuzzForge skills | `/skills` |
| `/sessions` | List active sessions | `/sessions` |
| `/sendfile <agent> <path>` | Send file to agent | `/sendfile Analyzer ./code.py` |
| `/clear` | Clear the screen | `/clear` |
| `/help` | Show help | `/help` |
| `/quit` | Exit the CLI | `/quit` |
---
## Built-in Function Tools
### Knowledge Management
```python
search_project_knowledge(query, dataset, search_type)
list_project_knowledge()
ingest_to_dataset(content, dataset)
```
### File Operations
```python
list_project_files(path, pattern)
read_project_file(file_path, max_lines)
search_project_files(search_pattern, file_pattern, path)
```
### Agent Management
```python
get_agent_capabilities(agent_name)
send_file_to_agent(agent_name, file_path, note)
```
### FuzzForge Platform
```python
list_fuzzforge_workflows()
submit_security_scan_mcp(workflow_name, target_path, parameters)
get_comprehensive_scan_summary(run_id)
get_fuzzforge_run_status(run_id)
get_fuzzforge_summary(run_id)
get_fuzzforge_findings(run_id)
```
### Task Management
```python
create_task_list(tasks)
update_task_status(task_list_id, task_id, status)
get_task_list(task_list_id)
```
---
## Environment Variables
Set these in `.fuzzforge/.env` to configure your FuzzForge AI instance.
### Model Configuration
```env
LITELLM_MODEL=gpt-4o-mini # Any LiteLLM-supported model
OPENAI_API_KEY=sk-... # API key for model provider
ANTHROPIC_API_KEY=sk-ant-... # For Claude models
GEMINI_API_KEY=... # For Gemini models
```
### Memory & Persistence
```env
SESSION_PERSISTENCE=sqlite # sqlite|inmemory
SESSION_DB_PATH=./fuzzforge_sessions.db
MEMORY_SERVICE=inmemory # inmemory|vertexai
```
### Server & Communication
```env
FUZZFORGE_PORT=10100 # A2A server port
ARTIFACT_STORAGE=inmemory # inmemory|gcs
GCS_ARTIFACT_BUCKET=artifacts # For GCS storage
```
### Debug & Observability
```env
FUZZFORGE_DEBUG=1 # Enable debug logging
AGENTOPS_API_KEY=... # Optional observability
```
### Platform Integration
```env
FUZZFORGE_MCP_URL=http://localhost:8010/mcp
```
---
## MCP (Model Context Protocol) Integration
FuzzForge supports the Model Context Protocol (MCP), allowing LLM clients and AI assistants to interact directly with the security testing platform. All FastAPI endpoints are available as MCP-compatible tools, making security automation accessible to any MCP-aware client.
### MCP Endpoints
- **HTTP MCP endpoint:** `http://localhost:8010/mcp`
- **SSE (Server-Sent Events):** `http://localhost:8010/mcp/sse`
- **Base API:** `http://localhost:8000`
### MCP Tools
- `submit_security_scan_mcp` — Submit security scanning workflows
- `get_comprehensive_scan_summary` — Get detailed scan analysis with recommendations
### FastAPI Endpoints (now MCP tools)
- `GET /` — API status
- `GET /workflows/` — List available workflows
- `POST /workflows/{workflow_name}/submit` — Submit security scans
- `GET /runs/{run_id}/status` — Check scan status
- `GET /runs/{run_id}/findings` — Get scan results
- `GET /fuzzing/{run_id}/stats` — Fuzzing statistics
### Usage Example: Submit a Security Scan via MCP
```json
{
"tool": "submit_security_scan_mcp",
"parameters": {
"workflow_name": "infrastructure_scan",
"target_path": "/path/to/your/project",
"volume_mode": "ro",
"parameters": {
"checkov_config": {
"severity": ["HIGH", "MEDIUM", "LOW"]
},
"hadolint_config": {
"severity": ["error", "warning", "info", "style"]
}
}
}
}
```
### Usage Example: Get a Comprehensive Scan Summary
```json
{
"tool": "get_comprehensive_scan_summary",
"parameters": {
"run_id": "your-run-id-here"
}
}
```
### Available Workflows
1. **infrastructure_scan** — Docker/Kubernetes/Terraform security analysis
2. **static_analysis_scan** — Code vulnerability detection
3. **secret_detection_scan** — Credential and secret scanning
4. **penetration_testing_scan** — Network and web app testing
5. **security_assessment** — Comprehensive security evaluation
### MCP Client Configuration Example
```json
{
"mcpServers": {
"fuzzforge": {
"command": "curl",
"args": ["-X", "POST", "http://localhost:8010/mcp"],
"env": {}
}
}
}
```
### Troubleshooting MCP
- **MCP Connection Failed:**
Check backend status:
`docker compose ps fuzzforge-backend`
`curl http://localhost:8000/health`
- **Workflows Not Found:**
`curl http://localhost:8000/workflows/`
- **Scan Submission Errors:**
`curl -X POST http://localhost:8000/workflows/infrastructure_scan/submit -H "Content-Type: application/json" -d '{"target_path": "/your/path", "volume_mode": "ro"}'`
- **General Support:**
- Check Docker Compose logs: `docker compose logs fuzzforge-backend`
- Verify MCP endpoint: `curl http://localhost:8010/mcp`
- Test FastAPI endpoints directly before using MCP
For more, see the [How-To: MCP Integration](../how-to/mcp-integration.md).
---
## API Endpoints
When running as an A2A server (`python -m fuzzforge_ai --port 10100`):
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/.well-known/agent-card.json` | GET | Agent capabilities |
| `/` | POST | A2A message processing |
| `/artifacts/{artifact_id}` | GET | Artifact file serving |
| `/health` | GET | Health check |
### Example: Agent Card Format
```json
{
"name": "FuzzForge",
"description": "Multi-agent orchestrator with memory and security tools",
"version": "1.0.0",
"url": "http://localhost:10100",
"protocolVersion": "0.3.0",
"preferredTransport": "JSONRPC",
"defaultInputModes": ["text/plain", "application/json"],
"defaultOutputModes": ["text/plain", "application/json"],
"capabilities": {
"streaming": false,
"pushNotifications": true,
"multiTurn": true,
"contextRetention": true
},
"skills": [
{
"id": "orchestration",
"name": "Agent Orchestration",
"description": "Route requests to appropriate agents",
"tags": ["orchestration", "routing"]
}
]
}
```
### Example: A2A Message Format
```json
{
"id": "msg_001",
"method": "agent.invoke",
"params": {
"message": {
"role": "user",
"parts": [
{
"type": "text",
"content": "Calculate factorial of 10"
}
]
},
"context": {
"sessionId": "session_abc123",
"conversationId": "conv_001"
}
}
}
```
---
## Project Structure Reference
```
project_root/
├── .fuzzforge/ # Project-local config
│ ├── .env # Environment variables
│ ├── config.json # Project configuration
│ ├── agents.yaml # Registered agents
│ ├── sessions.db # Session storage
│ ├── artifacts/ # Local artifact cache
│ └── data/ # Knowledge graphs
└── your_project_files...
```
### Agent Registry Example (`agents.yaml`)
```yaml
registered_agents:
- name: CalculatorAgent
url: http://localhost:10201
description: Mathematical calculations
- name: SecurityAnalyzer
url: http://localhost:10202
description: Code security analysis
```
---
## Quick Troubleshooting
- **Agent Registration Fails:** Check agent is running and accessible at its URL.
- **Memory Not Persisting:** Ensure `SESSION_PERSISTENCE=sqlite` and DB path is correct.
- **Files Not Found:** Use paths relative to project root.
- **Model API Errors:** Verify API key and model name.
docs/docs/reference/common-patterns.md
+796
View File
@@ -0,0 +1,796 @@
# Common Patterns Cookbook 👨‍🍳
A collection of proven patterns and recipes for FuzzForge modules and workflows. Copy, paste, and adapt these examples to build your own security tools quickly!
## Module Patterns
### File Processing Patterns
#### Pattern 1: Selective File Scanner
```python
class SelectiveScanner(BaseModule):
"""Scan only specific file types with size limits"""
SUPPORTED_EXTENSIONS = {'.py', '.js', '.java', '.cpp', '.c', '.go', '.rs'}
DEFAULT_MAX_SIZE = 5 * 1024 * 1024 # 5MB
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
max_size = config.get('max_file_size', self.DEFAULT_MAX_SIZE)
extensions = set(config.get('extensions', self.SUPPORTED_EXTENSIONS))
findings = []
processed_files = 0
for file_path in workspace.rglob('*'):
if (file_path.is_file() and
file_path.suffix.lower() in extensions and
file_path.stat().st_size <= max_size):
try:
result = await self._process_file(file_path, workspace)
findings.extend(result)
processed_files += 1
except Exception as e:
# Log error but continue processing
logger.warning(f"Failed to process {file_path}: {e}")
return self.create_result(
findings=findings,
summary={'files_processed': processed_files}
)
```
#### Pattern 2: Content-Based File Analysis
```python
class ContentAnalyzer(BaseModule):
"""Analyze file content with encoding detection"""
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
findings = []
for file_path in workspace.rglob('*'):
if file_path.is_file():
content = await self._safe_read_file(file_path)
if content:
analysis_result = await self._analyze_content(content, file_path, workspace)
findings.extend(analysis_result)
return self.create_result(findings=findings)
async def _safe_read_file(self, file_path: Path) -> str:
"""Safely read file with encoding detection"""
try:
# Try UTF-8 first
return file_path.read_text(encoding='utf-8')
except UnicodeDecodeError:
try:
# Fall back to latin-1 for binary-like files
return file_path.read_text(encoding='latin-1', errors='ignore')
except Exception:
return ""
async def _analyze_content(self, content: str, file_path: Path, workspace: Path) -> List[ModuleFinding]:
"""Override this method in your specific analyzer"""
# Example: Find TODO comments
findings = []
lines = content.split('\n')
for i, line in enumerate(lines, 1):
if 'TODO' in line.upper():
findings.append(self.create_finding(
title="TODO comment found",
description=f"TODO comment: {line.strip()}",
severity="info",
category="code_quality",
file_path=str(file_path.relative_to(workspace)),
line_start=i,
code_snippet=line.strip()
))
return findings
```
#### Pattern 3: Directory Structure Analysis
```python
class StructureAnalyzer(BaseModule):
"""Analyze project directory structure"""
IMPORTANT_FILES = {
'README.md': 'documentation',
'LICENSE': 'legal',
'.gitignore': 'vcs',
'requirements.txt': 'dependencies',
'package.json': 'dependencies',
'Dockerfile': 'deployment'
}
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
findings = []
structure_analysis = {
'total_directories': 0,
'max_depth': 0,
'important_files_found': [],
'important_files_missing': []
}
# Analyze directory structure
for item in workspace.rglob('*'):
if item.is_dir():
structure_analysis['total_directories'] += 1
depth = len(item.relative_to(workspace).parts)
structure_analysis['max_depth'] = max(structure_analysis['max_depth'], depth)
# Check for important files
for filename, category in self.IMPORTANT_FILES.items():
file_path = workspace / filename
if file_path.exists():
structure_analysis['important_files_found'].append(filename)
else:
structure_analysis['important_files_missing'].append(filename)
findings.append(self.create_finding(
title=f"Missing {category} file",
description=f"Recommended file '{filename}' not found",
severity="info",
category=category,
metadata={'file_type': category, 'recommended_file': filename}
))
return self.create_result(
findings=findings,
summary=structure_analysis
)
```
### Configuration Patterns
#### Pattern 1: Schema-Based Configuration
```python
from pydantic import BaseModel, Field, validator
from enum import Enum
class SeverityLevel(str, Enum):
LOW = "low"
MEDIUM = "medium"
HIGH = "high"
CRITICAL = "critical"
class ModuleConfig(BaseModel):
"""Type-safe configuration with validation"""
severity_threshold: SeverityLevel = SeverityLevel.MEDIUM
max_file_size_mb: int = Field(default=10, gt=0, le=100)
include_patterns: List[str] = Field(default=['**/*.py', '**/*.js'])
exclude_patterns: List[str] = Field(default=['**/node_modules/**', '**/.git/**'])
timeout_seconds: int = Field(default=300, gt=0, le=3600)
@validator('include_patterns')
def validate_patterns(cls, v):
if not v:
raise ValueError('At least one include pattern required')
return v
class ConfigurableModule(BaseModule):
def validate_config(self, config: Dict[str, Any]) -> bool:
try:
ModuleConfig(**config)
return True
except Exception:
return False
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
# Get validated configuration
validated_config = ModuleConfig(**config)
# Use type-safe configuration
max_size = validated_config.max_file_size_mb * 1024 * 1024
severity = validated_config.severity_threshold
# ... rest of implementation
```
#### Pattern 2: Configuration Templates
```python
class TemplateBasedModule(BaseModule):
"""Module with configuration templates"""
TEMPLATES = {
'quick': {
'max_file_size_mb': 5,
'timeout_seconds': 60,
'severity_threshold': 'medium'
},
'thorough': {
'max_file_size_mb': 50,
'timeout_seconds': 1800,
'severity_threshold': 'low'
},
'critical_only': {
'max_file_size_mb': 100,
'timeout_seconds': 3600,
'severity_threshold': 'critical'
}
}
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
# Load template if specified
template_name = config.get('template')
if template_name and template_name in self.TEMPLATES:
base_config = self.TEMPLATES[template_name].copy()
base_config.update(config) # Override template with specific config
config = base_config
# Continue with normal execution
return await self._execute_with_config(config, workspace)
```
### Error Handling Recipes
#### Pattern 1: Graceful Degradation
```python
class ResilientModule(BaseModule):
"""Module that handles errors gracefully"""
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
findings = []
errors = []
processed_files = 0
for file_path in workspace.rglob('*'):
if file_path.is_file():
try:
result = await self._analyze_file(file_path, workspace, config)
findings.extend(result)
processed_files += 1
except PermissionError as e:
errors.append({
'file': str(file_path.relative_to(workspace)),
'error': 'Permission denied',
'type': 'permission_error'
})
except UnicodeDecodeError as e:
errors.append({
'file': str(file_path.relative_to(workspace)),
'error': 'Encoding error',
'type': 'encoding_error'
})
except Exception as e:
errors.append({
'file': str(file_path.relative_to(workspace)),
'error': str(e),
'type': 'analysis_error'
})
# Determine overall status
total_files = processed_files + len(errors)
if len(errors) > total_files * 0.5: # More than 50% failed
status = "partial"
else:
status = "success"
return self.create_result(
findings=findings,
status=status,
summary={
'files_processed': processed_files,
'files_failed': len(errors),
'error_rate': len(errors) / total_files if total_files > 0 else 0
},
metadata={'errors': errors}
)
```
#### Pattern 2: Circuit Breaker
```python
import time
class CircuitBreakerModule(BaseModule):
"""Module with circuit breaker for expensive operations"""
def __init__(self):
super().__init__()
self.failure_count = 0
self.last_failure_time = 0
self.circuit_open = False
self.failure_threshold = 5
self.recovery_timeout = 60 # seconds
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
findings = []
for file_path in workspace.rglob('*'):
if file_path.is_file():
if self._is_circuit_open():
# Circuit is open, skip expensive operations
findings.append(self.create_finding(
title="Analysis skipped",
description="Circuit breaker is open due to previous failures",
severity="info",
category="system",
file_path=str(file_path.relative_to(workspace))
))
continue
try:
result = await self._expensive_analysis(file_path, workspace)
findings.extend(result)
self._on_success()
except Exception as e:
self._on_failure()
logger.warning(f"Analysis failed for {file_path}: {e}")
return self.create_result(findings=findings)
def _is_circuit_open(self) -> bool:
if not self.circuit_open:
return False
# Check if recovery timeout has passed
if time.time() - self.last_failure_time > self.recovery_timeout:
self.circuit_open = False
self.failure_count = 0
return False
return True
def _on_failure(self):
self.failure_count += 1
self.last_failure_time = time.time()
if self.failure_count >= self.failure_threshold:
self.circuit_open = True
def _on_success(self):
if self.circuit_open:
self.circuit_open = False
self.failure_count = 0
```
### Performance Patterns
#### Pattern 1: Batch Processing
```python
import asyncio
from typing import List, AsyncGenerator
class BatchProcessor(BaseModule):
"""Process files in batches to control memory usage"""
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
batch_size = config.get('batch_size', 10)
findings = []
async for batch_findings in self._process_in_batches(workspace, batch_size, config):
findings.extend(batch_findings)
return self.create_result(findings=findings)
async def _process_in_batches(
self,
workspace: Path,
batch_size: int,
config: Dict[str, Any]
) -> AsyncGenerator[List[ModuleFinding], None]:
"""Process files in batches"""
files = [f for f in workspace.rglob('*') if f.is_file()]
for i in range(0, len(files), batch_size):
batch = files[i:i + batch_size]
batch_findings = []
for file_path in batch:
try:
result = await self._analyze_file(file_path, workspace, config)
batch_findings.extend(result)
except Exception as e:
logger.warning(f"Failed to process {file_path}: {e}")
yield batch_findings
```
#### Pattern 2: Concurrent Processing with Limits
```python
class ConcurrentProcessor(BaseModule):
"""Process files concurrently with semaphore limits"""
async def execute(self, config: Dict[str, Any], workspace: Path) -> ModuleResult:
max_concurrent = config.get('max_concurrent', 5)
semaphore = asyncio.Semaphore(max_concurrent)
files = [f for f in workspace.rglob('*') if f.is_file()]
# Process files concurrently
tasks = [
self._process_file_with_semaphore(file_path, workspace, config, semaphore)
for file_path in files
]
results = await asyncio.gather(*tasks, return_exceptions=True)
# Collect findings and handle exceptions
findings = []
for result in results:
if isinstance(result, list):
findings.extend(result)
elif isinstance(result, Exception):
logger.warning(f"Processing failed: {result}")
return self.create_result(findings=findings)
async def _process_file_with_semaphore(
self,
file_path: Path,
workspace: Path,
config: Dict[str, Any],
semaphore: asyncio.Semaphore
) -> List[ModuleFinding]:
"""Process a single file with semaphore protection"""
async with semaphore:
return await self._analyze_file(file_path, workspace, config)
```
## ⚡ Workflow Patterns
### Sequential Processing
```python
@flow(name="sequential_analysis")
async def sequential_workflow(target_path: str, **kwargs) -> Dict[str, Any]:
"""Execute analysis steps in sequence"""
workspace = Path(target_path)
# Step 1: File discovery
scanner_config = kwargs.get('scanner_config', {})
scan_results = await file_scan_task(workspace, scanner_config)
# Step 2: Analysis (depends on scan results)
analyzer_config = {
**kwargs.get('analyzer_config', {}),
'discovered_files': scan_results.get('summary', {}).get('total_files', 0)
}
analysis_results = await analysis_task(scan_results, workspace, analyzer_config)
# Step 3: Report generation (depends on analysis)
reporter_config = kwargs.get('reporter_config', {})
final_report = await report_task(analysis_results, workspace, reporter_config)
return final_report
```
### Parallel Execution
```python
@flow(name="parallel_analysis")
async def parallel_workflow(target_path: str, **kwargs) -> Dict[str, Any]:
"""Execute independent analyses in parallel"""
workspace = Path(target_path)
# Submit parallel tasks
static_future = static_analysis_task.submit(workspace, kwargs.get('static_config', {}))
secret_future = secret_detection_task.submit(workspace, kwargs.get('secret_config', {}))
license_future = license_check_task.submit(workspace, kwargs.get('license_config', {}))
# Wait for all to complete
static_results = await static_future.result()
secret_results = await secret_future.result()
license_results = await license_future.result()
# Combine results
combined_report = await combine_results_task(
[static_results, secret_results, license_results],
workspace,
kwargs.get('reporter_config', {})
)
return combined_report
```
### Conditional Logic
```python
@flow(name="conditional_analysis")
async def conditional_workflow(target_path: str, **kwargs) -> Dict[str, Any]:
"""Execute workflow with conditional branches"""
workspace = Path(target_path)
# Initial assessment
assessment = await quick_assessment_task(workspace)
# Branch based on project type
if assessment.get('project_type') == 'web_application':
# Web app specific analysis
web_results = await web_security_task(workspace, kwargs.get('web_config', {}))
final_results = web_results
elif assessment.get('project_type') == 'library':
# Library specific analysis
lib_results = await library_analysis_task(workspace, kwargs.get('lib_config', {}))
final_results = lib_results
else:
# Generic analysis
generic_results = await generic_analysis_task(workspace, kwargs.get('generic_config', {}))
final_results = generic_results
# Optional deep analysis for high-risk projects
if assessment.get('risk_level', 'low') in ['high', 'critical']:
deep_results = await deep_analysis_task(workspace, kwargs.get('deep_config', {}))
final_results = await merge_results_task(final_results, deep_results)
return final_results
```
### Data Transformation
```python
@task(name="filter_and_transform")
async def filter_transform_task(
raw_results: Dict[str, Any],
config: Dict[str, Any]
) -> Dict[str, Any]:
"""Filter and transform findings based on criteria"""
findings = raw_results.get('findings', [])
# Filter by severity
min_severity = config.get('min_severity', 'low')
severity_order = {'info': 0, 'low': 1, 'medium': 2, 'high': 3, 'critical': 4}
min_level = severity_order.get(min_severity, 0)
filtered_findings = [
f for f in findings
if severity_order.get(f.get('severity', 'info'), 0) >= min_level
]
# Group by category
categorized = {}
for finding in filtered_findings:
category = finding.get('category', 'other')
if category not in categorized:
categorized[category] = []
categorized[category].append(finding)
# Transform findings (add risk scores, priorities, etc.)
enriched_findings = []
for finding in filtered_findings:
enriched_finding = {
**finding,
'risk_score': calculate_risk_score(finding),
'priority': determine_priority(finding),
'remediation_effort': estimate_effort(finding)
}
enriched_findings.append(enriched_finding)
return {
'findings': enriched_findings,
'summary': {
'total_findings': len(enriched_findings),
'by_category': {k: len(v) for k, v in categorized.items()},
'by_severity': {
severity: len([f for f in enriched_findings if f.get('severity') == severity])
for severity in ['info', 'low', 'medium', 'high', 'critical']
}
}
}
```
## 🧪 Testing Patterns
### Pattern 1: Comprehensive Module Testing
```python
import pytest
import tempfile
from pathlib import Path
from unittest.mock import patch, AsyncMock
class TestMyModule:
@pytest.fixture
def temp_workspace(self):
with tempfile.TemporaryDirectory() as temp_dir:
workspace = Path(temp_dir)
# Create test files
(workspace / 'test.py').write_text('print("hello")')
(workspace / 'config.json').write_text('{"key": "value"}')
yield workspace
@pytest.fixture
def module(self):
return MyModule()
@pytest.fixture
def base_config(self):
return {
'max_file_size_mb': 10,
'severity_threshold': 'medium',
'timeout_seconds': 60
}
@pytest.mark.asyncio
async def test_execute_success(self, module, temp_workspace, base_config):
result = await module.execute(base_config, temp_workspace)
assert result.status == "success"
assert isinstance(result.findings, list)
assert isinstance(result.summary, dict)
assert 'total_files' in result.summary
@pytest.mark.asyncio
async def test_execute_empty_workspace(self, module, base_config):
with tempfile.TemporaryDirectory() as empty_dir:
result = await module.execute(base_config, Path(empty_dir))
assert result.summary['total_files'] == 0
assert len(result.findings) == 0
@pytest.mark.asyncio
async def test_config_validation(self, module):
assert module.validate_config({'max_file_size_mb': 10})
assert not module.validate_config({'max_file_size_mb': -1})
assert not module.validate_config({'max_file_size_mb': 'invalid'})
@pytest.mark.asyncio
async def test_error_handling(self, module, base_config):
with patch.object(module, '_analyze_file', side_effect=Exception("Test error")):
result = await module.execute(base_config, Path('/tmp'))
# Should handle errors gracefully
assert 'errors' in result.metadata
assert len(result.metadata['errors']) > 0
@pytest.mark.parametrize("severity,expected", [
('low', ['low', 'medium', 'high', 'critical']),
('medium', ['medium', 'high', 'critical']),
('high', ['high', 'critical']),
('critical', ['critical'])
])
async def test_severity_filtering(self, module, temp_workspace, severity, expected):
config = {'severity_threshold': severity}
result = await module.execute(config, temp_workspace)
found_severities = {f.severity for f in result.findings}
assert found_severities.issubset(set(expected))
```
## 🔧 Utility Functions
### File Type Detection
```python
def detect_file_type(file_path: Path) -> str:
"""Detect file type from extension and content"""
# Extension-based detection
extension_map = {
'.py': 'python',
'.js': 'javascript',
'.ts': 'typescript',
'.java': 'java',
'.cpp': 'cpp',
'.c': 'c',
'.go': 'go',
'.rs': 'rust',
'.json': 'json',
'.yaml': 'yaml',
'.yml': 'yaml',
'.xml': 'xml',
'.html': 'html',
'.css': 'css',
'.md': 'markdown',
'.txt': 'text'
}
file_type = extension_map.get(file_path.suffix.lower())
if file_type:
return file_type
# Content-based detection for files without extensions
try:
with open(file_path, 'r', encoding='utf-8', errors='ignore') as f:
first_line = f.readline().strip()
if first_line.startswith('#!'):
if 'python' in first_line:
return 'python'
elif 'bash' in first_line or 'sh' in first_line:
return 'shell'
elif 'node' in first_line:
return 'javascript'
if first_line.startswith('<?xml'):
return 'xml'
if first_line.startswith('<!DOCTYPE html') or first_line.startswith('<html'):
return 'html'
except Exception:
pass
return 'unknown'
```
### Risk Scoring
```python
def calculate_risk_score(finding: Dict[str, Any]) -> int:
"""Calculate numeric risk score for a finding"""
base_scores = {
'critical': 100,
'high': 75,
'medium': 50,
'low': 25,
'info': 10
}
severity = finding.get('severity', 'info')
base_score = base_scores.get(severity, 10)
# Adjust based on category
category_multipliers = {
'security': 1.0,
'vulnerability': 1.0,
'credential': 1.2,
'injection': 1.1,
'authentication': 1.1,
'authorization': 1.1,
'code_quality': 0.8,
'performance': 0.7,
'documentation': 0.5
}
category = finding.get('category', 'other')
multiplier = category_multipliers.get(category, 0.9)
# Adjust based on file location
file_path = finding.get('file_path', '')
if any(sensitive in file_path.lower() for sensitive in ['config', 'secret', 'password', 'key']):
multiplier *= 1.2
return int(base_score * multiplier)
```
### Finding Deduplication
```python
def deduplicate_findings(findings: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""Remove duplicate findings based on title, file, and line"""
seen = set()
deduplicated = []
for finding in findings:
# Create unique key
key = (
finding.get('title', ''),
finding.get('file_path', ''),
finding.get('line_start', 0),
finding.get('category', '')
)
if key not in seen:
seen.add(key)
deduplicated.append(finding)
else:
# Update metadata to indicate duplication
for existing in deduplicated:
if (existing.get('title') == finding.get('title') and
existing.get('file_path') == finding.get('file_path')):
metadata = existing.setdefault('metadata', {})
metadata['duplicate_count'] = metadata.get('duplicate_count', 1) + 1
break
return deduplicated
```
---
**🎯 Next Steps**: Use these patterns as building blocks for your own modules and workflows. Mix and match patterns to create powerful security analysis tools!
docs/docs/reference/contributing.md
+69
View File
@@ -0,0 +1,69 @@
# Contributing
Contributing is much appreciated.
## How to contribute
### Development environment setup
We recommand using the excellent [Based Pyright](https://docs.basedpyright.com/latest/) LSP, which is a fork of [pyright](https://github.com/microsoft/pyright) with various type checking improvements, pylance features and more. It is available in all major editors (VSCode, Vim, Emacs, Zed).
To work on the project, you will need to install `uv`. Check the [installation instructions](https://docs.astral.sh/uv/getting-started/installation/) for your platform.
We also recommand using Just to manage your development environment. Just is a command runner, similar to Make, but with a simpler syntax and more features. It is available in all major platforms. Check the [installation instructions](https://just.systems/man/en/) for your platform. We wrapped on number of useful commands in the `Justfile` at the root of the repository. You can see the available commands by running `just`.
### Code conventions
We try to follow the [Python Style Guide](https://www.python.org/dev/peps/pep-0008/) and [Google Python Style Guide](https://github.com/google/styleguide/blob/gh-pages/pyguide.md) for Python code. A linter and formatter is used to ensure that the code is consistent and follows the style guide. The linter and formatter used is [Ruff](https://docs.astral.sh/ruff/).
### Git usage
We use the [Conventional Commits 1.0.0](https://www.conventionalcommits.org/en/v1.0.0/) specification to format our commits.
As for our workflow, we use the following with the following branch names :
- main : `main` - for production code
- hotfix : `hotfix/<hotfix name>` - for urgent fixes to the production code
- develop : `dev` - for development
- feature : `feat/<feature name>` - for new features
- continuous integration : `ci/<ci name>` - for continuous integration related changes
- documentation : `docs/<documentation name>` - for documentation changes
- fix : `fix/<bug name>` - for bug fixes
- chore : `chore/<chore name>` - for changes that do not modify src or test files
- refactor : `refactor/<refactor name>` - for code refactoring
- perf : `perf/<performance name>` - for performance improvements
- test : `test/<test name>` - for adding or modifying tests
- build : `build/<build name>` - for build-related changes
- revert : `revert/<revert name>` - for reverting changes
- style : `style/<style name>` - for style-related changes
![Git workflow](./img/git-workflow.png)
In addition to the branching names, only `dev` and `hotfix` branches are allowed to be merged into `main`. All other branches must be merged into `dev` first.
The `dev` branch is the main development branch, and all new features and bug fixes should be merged into it. The `main` branch is the production branch, and only stable code should be merged into it.
The `hotfix` branch is used for urgent fixes to the production code, and should be merged into both `main` and `dev` branches.
This workflow is derived from the [Git Flow](https://nvie.com/posts/a-successful-git-branching-model/) workflow, with some modifications to fit our needs.
!!! note
Following theses conventions allows for an automatic CI to label pull requests and commits with the correct labels. This can be used to automatically generate the changelog and release notes, but mainly facilitates the review process.
### Testing
We use [pytest](https://docs.pytest.org/en/latest/) for unit and integration testing, and [PyTestArch](https://pypi.org/project/PyTestArch/) for architectural rules. The tests are located in the `tests` directory.
A test is required for every new feature and bug fix. The tests should be located in the `tests` directory of the corresponding module.
The tests should be run before merging any changes into the `dev` or `main` branches.
### Continuous integration
We use [GitHub Actions](https://docs.github.com/en/actions) for continuous integration. The CI workflow is located in the `.github/workflows` directory.
The CI workflow is triggered on every push to the `dev` and `main` branches, and on every pull request to the `dev` and `main` branches.
The CI workflow runs the tests and linter, and builds the documentation. The CI workflow is required to pass before merging any changes into the `dev` or `main` branches.
### Bug report
To-do
docs/docs/reference/decisions/architecture-decision.tmpl
+64
View File
@@ -0,0 +1,64 @@
# {Title of solution to solve the problem}
## Context and problem statement
{Describe the context and problem in free form, using two to three sentences or in the form of an illustrative story.
You may want to articulate the problem in form of a question and add links to collaboration boards or issue management systems.}
<!-- This is an optional element. Feel free to remove. -->
## Decision Drivers
* {decision driver, e.g., a force, facing concern, ...}
* ...
## Considered Options
* [{title of option}](#{title of option})
* ...
## Decision Outcome
Chosen option: "{title of chosen option}", because
{justification. e.g., only option, which meets k.o. criterion decision driver | which resolves force {force} | … | comes out best (see below)}.
<!-- This is an optional element. Feel free to remove. -->
## Decision Revisit
Last revisit: {information about the last revisit e.g. never | {date} by {author}}
<!-- This is an optional element. Feel free to remove. -->
### Consequences
* Good, because {positive consequence, e.g., improvement of one or more desired qualities, …}
* Bad, because {negative consequence, e.g., compromising one or more desired qualities, …}
* … <!-- numbers of consequences can vary -->
<!-- This is an optional element. Feel free to remove. -->
## Validation
{describe how the implementation of/compliance with the ADR is validated. E.g., by a review or an ArchUnit test}
<!-- This is an optional element. Feel free to remove. -->
## Pros and Cons of the Options
<!-- This is an repeated element per option. use when necessary. -->
### {title of option}
<!-- This is an optional element. Feel free to remove. -->
{example | description | pointer to more information | …}
* Good, because {argument a}
* Good, because {argument b}
<!-- use "neutral" if the given argument weights neither for good nor bad -->
* Neutral, because {argument c}
* Bad, because {argument d}
* ... <!-- numbers of pros and cons can vary -->
<!-- This is an optional element. Feel free to remove. -->
## More Information
{Provide additional evidence/confidence for the decision outcome here and/or
document the team agreement on the decision and/or
define when this decision when and how the decision should be realized and if/when it should be re-visited and/or
how the decision is validated.
Links to other decisions and resources might here appear as well.}
docs/docs/reference/diataxis-documentation.md
+17
View File
@@ -0,0 +1,17 @@
# Diataxis documentation
This project uses the [Diátaxis](https://diataxis.fr) technical documentation framework.
There are 4 main parts:
1. [Getting started (tutorials)](https://diataxis.fr/tutorials): learning-oriented
- Pages that contain tutorials needed to get people up and running, for instance [Getting Started](/intro.md).
2. [Concepts (explanation)](https://diataxis.fr/explanation): understanding-oriented
- Pages explaining concepts that are relevant to the domain, for instance [Working with documentation](../concept/working-with-documentation.md).
3. [How-to guides](https://diataxis.fr/how-to-guides): goal-oriented
- Pages that contain tutorials, for instance [How-to: start the local documentation server](#).
4. [Reference](https://diataxis.fr/reference): information-oriented
- Pages that contain reference information, for instance about [Diataxis documentation](../reference/diataxis-documentation.md).
## Working with Diátaxis documentation
See [working with Diátaxis documentation](../concept/working-with-documentation.md).
docs/docs/reference/faq/known-issue.tmpl
+17
View File
@@ -0,0 +1,17 @@
# {Title}
**Status**: active
## Description
{Provide a detailed description of the issue, include things such as how to identify this particular issue}
<!-- This is an optional element. Feel free to remove. -->
## Impact
{Describe the impact of the issue on users or the system.}
<!-- This is an optional element. Feel free to remove. -->
## Workaround
{If available, describe any possible workarounds to mitigate the issue until it is resolved.}
docs/docs/reference/img/git-workflow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 46 KiB

docs/docs/reference/workflows/static-analysis.md
+257
View File
@@ -0,0 +1,257 @@
# Static Analysis Workflow Reference
The Static Analysis workflow in FuzzForge helps you find vulnerabilities, code quality issues, and compliance problems—before they reach production. This workflow uses multiple Static Application Security Testing (SAST) tools to analyze your source code without executing it, providing fast, actionable feedback in a standardized format.
---
## What Does This Workflow Do?
- **Workflow ID:** `static_analysis_scan`
- **Primary Tools:** Semgrep (multi-language), Bandit (Python)
- **Supported Languages:** Python, JavaScript, Java, Go, C/C++, PHP, Ruby, and more
- **Typical Duration:** 1–5 minutes (varies by codebase size)
- **Output Format:** SARIF 2.1.0 (industry standard)
---
## How Does It Work?
The workflow orchestrates multiple SAST tools in a containerized environment:
- **Semgrep:** Pattern-based static analysis for 30+ languages, with rule sets for OWASP Top 10, CWE Top 25, and more.
- **Bandit:** Python-specific security scanner, focused on issues like hardcoded secrets, injection, and unsafe code patterns.
Each tool runs independently, and their findings are merged and normalized into a single SARIF report.
---
## How to Use the Static Analysis Workflow
### Basic Usage
**CLI:**
```bash
fuzzforge runs submit static_analysis_scan /path/to/your/project
```
**API:**
```bash
curl -X POST "http://localhost:8000/workflows/static_analysis_scan/submit" \
-H "Content-Type: application/json" \
-d '{"target_path": "/path/to/your/project"}'
```
### Advanced Configuration
You can fine-tune the workflow by passing parameters for each tool:
**CLI:**
```bash
fuzzforge runs submit static_analysis_scan /path/to/project \
--parameters '{
"semgrep_config": {
"rules": ["p/security-audit", "owasp-top-ten"],
"severity": ["ERROR", "WARNING"],
"exclude_patterns": ["test/*", "vendor/*", "node_modules/*"]
},
"bandit_config": {
"confidence": "MEDIUM",
"severity": "MEDIUM",
"exclude_dirs": ["tests", "migrations"]
}
}'
```
**API:**
```json
{
"target_path": "/path/to/project",
"parameters": {
"semgrep_config": {
"rules": ["p/security-audit"],
"languages": ["python", "javascript"],
"severity": ["ERROR", "WARNING"],
"exclude_patterns": ["*.test.js", "test_*.py", "vendor/*"]
},
"bandit_config": {
"confidence": "MEDIUM",
"severity": "LOW",
"tests": ["B201", "B301"],
"exclude_dirs": ["tests", ".git"]
}
}
}
```
---
## Configuration Reference
### Semgrep Parameters
| Parameter | Type | Default | Description |
|-------------------|-----------|-------------------------------|---------------------------------------------|
| `rules` | array | `"auto"` | Rule sets to use (e.g., `"p/security-audit"`)|
| `languages` | array | `null` | Languages to analyze |
| `severity` | array | `["ERROR", "WARNING", "INFO"]`| Severities to include |
| `exclude_patterns`| array | `[]` | File patterns to exclude |
| `include_patterns`| array | `[]` | File patterns to include |
| `max_target_bytes`| integer | `1000000` | Max file size to analyze (bytes) |
| `timeout` | integer | `300` | Tool timeout (seconds) |
### Bandit Parameters
| Parameter | Type | Default | Description |
|-------------------|-----------|-------------------------------|---------------------------------------------|
| `confidence` | string | `"LOW"` | Minimum confidence (`"LOW"`, `"MEDIUM"`, `"HIGH"`) |
| `severity` | string | `"LOW"` | Minimum severity (`"LOW"`, `"MEDIUM"`, `"HIGH"`) |
| `tests` | array | `null` | Specific test IDs to run |
| `exclude_dirs` | array | `["tests", ".git"]` | Directories to exclude |
| `aggregate` | string | `"file"` | Aggregation mode (`"file"`, `"vuln"`) |
| `context_lines` | integer | `3` | Context lines around findings |
---
## What Can It Detect?
### Vulnerability Categories
- **OWASP Top 10:** Broken Access Control, Injection, Security Misconfiguration, etc.
- **CWE Top 25:** SQL Injection, XSS, Command Injection, Information Exposure, etc.
- **Language-Specific:** Python (unsafe eval, Django/Flask issues), JavaScript (XSS, prototype pollution), Java (deserialization), Go (race conditions), C/C++ (buffer overflows).
### Example Detections
**SQL Injection (Python)**
```python
query = f"SELECT * FROM users WHERE id = {user_id}" # CWE-89
```
*Recommendation: Use parameterized queries.*
**Command Injection (Python)**
```python
os.system(f"cp {filename} backup/") # CWE-78
```
*Recommendation: Use subprocess with argument arrays.*
**XSS (JavaScript)**
```javascript
element.innerHTML = userInput; // CWE-79
```
*Recommendation: Use textContent or sanitize input.*
---
## Output Format
All results are returned in SARIF 2.1.0 format, which is supported by many IDEs and security tools.
**Summary Example:**
```json
{
"workflow": "static_analysis_scan",
"status": "completed",
"total_findings": 18,
"severity_counts": {
"critical": 0,
"high": 6,
"medium": 5,
"low": 7
},
"tool_counts": {
"semgrep": 12,
"bandit": 6
}
}
```
**Finding Example:**
```json
{
"ruleId": "bandit.B608",
"level": "error",
"message": {
"text": "Possible SQL injection vector through string-based query construction"
},
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "src/database.py"
},
"region": {
"startLine": 42,
"startColumn": 15,
"endLine": 42,
"endColumn": 65
}
}
}
],
"properties": {
"severity": "high",
"category": "sql_injection",
"cwe": "CWE-89",
"confidence": "high",
"tool": "bandit"
}
}
```
---
## Performance Tips
- For large codebases, increase `max_target_bytes` and `timeout` as needed.
- Exclude large generated or dependency directories (`vendor/`, `node_modules/`, `dist/`).
- Run focused scans on changed files for faster CI/CD feedback.
---
## Integration Examples
### GitHub Actions
```yaml
- name: Run Static Analysis
run: |
curl -X POST "${{ secrets.FUZZFORGE_URL }}/workflows/static_analysis_scan/submit" \
-H "Content-Type: application/json" \
-d '{
"target_path": "${{ github.workspace }}",
"parameters": {
"semgrep_config": {"severity": ["ERROR", "WARNING"]},
"bandit_config": {"confidence": "MEDIUM"}
}
}'
```
### Pre-commit Hook
```bash
fuzzforge runs submit static_analysis_scan . --wait --json > /tmp/analysis.json
HIGH_ISSUES=$(jq '.sarif.severity_counts.high // 0' /tmp/analysis.json)
if [ "$HIGH_ISSUES" -gt 0 ]; then
echo "❌ Found $HIGH_ISSUES high-severity security issues. Commit blocked."
exit 1
fi
```
---
## Best Practices
- **Target the right code:** Focus on your main source directories, not dependencies or build artifacts.
- **Start broad, then refine:** Use default rule sets first, then add exclusions or custom rules as needed.
- **Triage findings:** Address high-severity issues first, and document false positives for future runs.
- **Monitor trends:** Track your security posture over time to measure improvement.
- **Optimize for speed:** Use file size limits and timeouts for very large projects.
---
## Troubleshooting
- **No Python files found:** Bandit will report zero findings if your project isn’t Python, this is normal.
- **High memory usage:** Exclude large files and directories, or increase Docker memory limits.
- **Slow scans:** Use inclusion/exclusion patterns and increase timeouts for big repos.
- **Workflow errors:** See the [Troubleshooting Guide](/how-to/troubleshooting.md) for help with registry, Docker, or workflow issues.
docs/docs/tutorial/_category_.json
+8
View File
@@ -0,0 +1,8 @@
{
"label": "Tutorial",
"position": 3,
"link": {
"type": "generated-index",
"description": "Tutorial pages that are learning-oriented."
}
}
docs/docs/tutorial/getting-started.md
+255
View File
@@ -0,0 +1,255 @@
# Getting Started
This tutorial will guide you through setting up FuzzForge and running your first security workflow. By the end of this tutorial, you'll have FuzzForge running locally and will have executed a complete static analysis workflow on a test project.
## Prerequisites
Before we begin, ensure you have the following installed:
- **Docker Desktop** or **Docker Engine** (20.10.0 or later)
- **Docker Compose** (2.0.0 or later)
- **Python 3.11+** (for CLI installation)
- **Git** (for cloning the repository)
- **4GB+ RAM** available for workflow execution
## Step 1: Clone and Setup FuzzForge
First, let's clone the FuzzForge repository:
```bash
git clone https://github.com/FuzzingLabs/fuzzforge.git
cd fuzzforge
```
Create the required environment configuration:
```bash
echo "COMPOSE_PROJECT_NAME=fuzzforge_alpha" > .env
```
## Step 2: Configure Docker (Critical Step)
**⚠️ This step is mandatory - workflows will fail without proper Docker configuration.**
FuzzForge uses a local Docker registry at `localhost:5001` for workflow images. You must configure Docker to allow this insecure registry.
### For Docker Desktop (macOS/Windows):
1. Open Docker Desktop
2. Go to Settings/Preferences → Docker Engine
3. Add the following to the JSON configuration:
```json
{
"builder": {
"gc": {
"defaultKeepStorage": "20GB",
"enabled": true
}
},
"experimental": false,
"insecure-registries": [
"localhost:5001"
]
}
```
4. Click "Apply & Restart"
### For Docker Engine (Linux):
Edit `/etc/docker/daemon.json` (create if it doesn't exist):
```json
{
"insecure-registries": ["localhost:5001"]
}
```
Restart Docker:
```bash
sudo systemctl restart docker
```
### Verify Docker Configuration
Test that Docker can access the insecure registry:
```bash
# After starting FuzzForge (next step), this should work without certificate errors
docker pull localhost:5001/hello-world 2>/dev/null || echo "Registry not accessible yet (normal before startup)"
```
## Step 3: Start FuzzForge
Start all FuzzForge services:
```bash
docker compose up -d
```
This will start 8 services:
- **prefect-server**: Workflow orchestration server
- **prefect-worker**: Executes workflows in Docker containers
- **fuzzforge-backend**: FastAPI backend and workflow management
- **postgres**: Metadata and workflow state storage
- **redis**: Message broker and caching
- **registry**: Local Docker registry for workflow images
- **docker-proxy**: Secure Docker socket proxy
- **prefect-services**: Additional Prefect services
Wait for all services to be healthy (this may take 2-3 minutes on first startup):
```bash
# Check service health
docker compose ps
# Verify FuzzForge is ready
curl http://localhost:8000/health
# Should return: {"status":"healthy"}
```
## Step 4: Install the CLI (Optional but Recommended)
Install the FuzzForge CLI for easier workflow management:
```bash
cd cli
pip install -e .
# or with uv:
uv tool install .
```
Verify CLI installation:
```bash
fuzzforge --version
```
Configure the CLI to connect to your local instance:
```bash
fuzzforge config set-server http://localhost:8000
```
## Step 5: Verify Available Workflows
List all available workflows to confirm the system is working:
```bash
# Using CLI
fuzzforge workflows list
# Using API
curl http://localhost:8000/workflows | jq .
```
You should see 6 production workflows:
- `static_analysis_scan`
- `secret_detection_scan`
- `infrastructure_scan`
- `penetration_testing_scan`
- `language_fuzzing`
- `security_assessment`
## Step 6: Run Your First Workflow
Let's run a static analysis workflow on one of the included vulnerable test projects.
### Using the CLI (Recommended):
```bash
# Navigate to a test project
cd /path/to/fuzzforge/test_projects/static_analysis_vulnerable
# Submit the workflow
fuzzforge runs submit static_analysis_scan .
# Monitor the workflow
fuzzforge runs status <run-id>
# View results when complete
fuzzforge findings get <run-id>
```
### Using the API:
```bash
# Submit workflow
curl -X POST "http://localhost:8000/workflows/static_analysis_scan/submit" \
-H "Content-Type: application/json" \
-d '{
"target_path": "/path/to/your/project"
}'
# Check status
curl "http://localhost:8000/runs/{run-id}/status"
# Get findings
curl "http://localhost:8000/runs/{run-id}/findings"
```
## Step 7: Understanding the Results
The workflow will complete in 30-60 seconds and return results in SARIF format. For the test project, you should see:
- **Total findings**: 15-20 security issues
- **Tools used**: OpenGrep (Semgrep) and Bandit
- **Severity levels**: High, Medium, and Low vulnerabilities
- **Categories**: SQL injection, command injection, hardcoded secrets, etc.
Example output:
```json
{
"workflow": "static_analysis_scan",
"status": "completed",
"total_findings": 18,
"scan_status": {
"opengrep": "success",
"bandit": "success"
},
"severity_counts": {
"high": 6,
"medium": 5,
"low": 7
}
}
```
## Step 8: Access the Prefect Dashboard
You can monitor workflow execution in real-time using the Prefect dashboard:
1. Open http://localhost:4200 in your browser
2. Navigate to "Flow Runs" to see workflow executions
3. Click on a run to see detailed logs and execution graph
## Next Steps
Congratulations! You've successfully:
- ✅ Installed and configured FuzzForge
- ✅ Configured Docker with the required insecure registry
- ✅ Started all FuzzForge services
- ✅ Run your first security workflow
- ✅ Retrieved and understood workflow results
### What to explore next:
1. **[Create Workflow Guide](../how-to/create-workflow.md)** - Create your own workflows !
### Common Issues:
If you encounter problems:
1. **Workflow crashes with registry errors**: Check Docker insecure registry configuration
2. **Services won't start**: Ensure ports 4200, 5001, 8000 are available
3. **No findings returned**: Verify the target path contains analyzable code files
4. **CLI not found**: Ensure Python/pip installation path is in your PATH
See the [Troubleshooting Guide](../how-to/troubleshooting.md) for detailed solutions.
---
**🎉 You're now ready to use FuzzForge for security analysis!**
The next tutorial will teach you about the different types of workflows and when to use each one.
docs/docs/tutorial/tutorial-with-variations.tmpl
+43
View File
@@ -0,0 +1,43 @@
# {Tutorial title}
=== "{Variation 1}"
<!-- This is an optional element. Feel free to remove. -->
## Prerequisites
{describe goal to achieve}, provide the following:
- [ ] {requirement 1} ({example of data and format})
- [ ] ...
## {Goal}
Steps:
- [ ] [How-to {step}](/how-to/update-documentation)
- [ ] ...
=== "{Variation 2}"
<!-- This is an optional element. Feel free to remove. -->
## Prerequisites
{describe goal to achieve}, provide the following:
- [ ] {requirement 1} ({example of data and format})
- [ ] ...
## {Goal}
Steps:
- [ ] [How-to {step}](/how-to/update-documentation)
- [ ] ...
<!-- This is an optional element. Feel free to remove. -->
## Appendix
### Reference
- [Reference 1](https://diataxis-template.readthedocs.io/en/latest/) ({optional: note for reader})
- ...
docs/docs/tutorial/tutorial.tmpl
+24
View File
@@ -0,0 +1,24 @@
# {Tutorial title}
<!-- This is an optional element. Feel free to remove. -->
## Prerequisites
{describe goal to achieve}, provide the following:
- [ ] {requirement 1} ({example of data and format})
- [ ] ...
## {Goal}
Steps:
- [ ] [How-to {step}](/how-to/update-documentation)
- [ ] ...
<!-- This is an optional element. Feel free to remove. -->
## Appendix
### Reference
- [Reference 1](https://diataxis-template.readthedocs.io/en/latest/) ({optional: note for reader})
- ...
docs/docusaurus.config.ts
+177
View File
@@ -0,0 +1,177 @@
import { themes as prismThemes } from "prism-react-renderer";
import type { Config } from "@docusaurus/types";
import type * as Preset from "@docusaurus/preset-classic";
// This runs in Node.js - Don't use client-side code here (browser APIs, JSX...)
const config: Config = {
title: "FuzzForge Documentation",
tagline: "AI-Powered Security Analysis Platform",
favicon: "img/favicon.ico",
// Future flags, see https://docusaurus.io/docs/api/docusaurus-config#future
future: {
v4: true, // Improve compatibility with the upcoming Docusaurus v4
},
// Production url of documentation
url: "https://docs.fuzzforge.ai",
// The /<baseUrl>/ pathname under which your site is served
// For GitHub pages deployment, it is often '/<projectName>/'
baseUrl: "/",
trailingSlash: false,
// GitHub pages deployment config.
organizationName: "FuzzingLabs",
projectName: "fuzzforge_alpha",
deploymentBranch: "gh-pages",
onBrokenLinks: "throw",
i18n: {
defaultLocale: "en",
locales: ["en"],
},
markdown: {
mermaid: true,
hooks: {
onBrokenMarkdownLinks: "warn",
},
},
presets: [
[
"classic",
{
docs: {
sidebarPath: "./sidebars.ts",
editUrl:
"https://github.com/FuzzingLabs/fuzzforge_alpha/tree/main/packages/create-docusaurus/templates/shared/",
},
theme: {
customCss: "./src/css/custom.css",
},
} satisfies Preset.Options,
],
],
themes: [
"@docusaurus/theme-mermaid",
[
require.resolve("@easyops-cn/docusaurus-search-local"),
/** @type {import("@easyops-cn/docusaurus-search-local").PluginOptions} */
{
// `hashed` is recommended as long-term-cache of index file is possible.
hashed: true,
language: ["en"],
},
],
],
themeConfig: {
metadata: [
{
name: "keywords",
content:
"documentation, fuzzforge, fuzzinglabs, fuzzing, security, ai, ai-powered, vulnerability, analysis, platform",
},
{ name: "twitter:card", content: "summary_large_image" },
],
image: "img/fuzzforge-social-card.jpg",
navbar: {
title: "FuzzForge Docs",
logo: {
alt: "FuzzForge Logo",
src: "img/fuzzforge-logo-1024-rounded.png",
},
items: [
{
type: "docSidebar",
sidebarId: "backendSidebar",
position: "left",
label: "Workflow",
},
{
type: "docSidebar",
sidebarId: "aiSidebar",
position: "left",
label: "AI",
},
{
href: "https://github.com/FuzzingLabs/fuzzforge_alpha",
label: "GitHub",
position: "right",
},
],
},
footer: {
style: "dark",
links: [
{
title: "Workflow",
items: [
{
label: "Tutorials",
to: "/docs/category/tutorial",
},
{
label: "Concepts",
to: "/docs/category/concept",
},
{
label: "How-to Guides",
to: "/docs/category/how-to-guides",
},
{
label: "References",
to: "/docs/category/reference",
},
],
},
{
title: "Community",
items: [
{
label: "Website",
href: "https://fuzzforge.ai/",
},
{
label: "Discord",
href: "https://discord.gg/jKBygqFkwn",
},
{
label: "X",
href: "https://x.com/FuzzingLabs",
},
{
label: "LinkedIn",
href: "https://www.linkedin.com/company/fuzzinglabs",
},
],
},
{
title: "More",
items: [
{
label: "FuzzingLabs Blog",
to: "https://fuzzinglabs.com/security-blog/",
},
{
label: "GitHub",
href: "https://github.com/FuzzingLabs/fuzzforge_alpha",
},
],
},
],
copyright: `Copyright © ${new Date().getFullYear()} FuzzingLabs - All Rights Reserved`,
},
prism: {
theme: prismThemes.github,
darkTheme: prismThemes.dracula,
},
} satisfies Preset.ThemeConfig,
};
export default config;
docs/index.md
+116
View File
@@ -0,0 +1,116 @@
# FuzzForge Documentation
Welcome to FuzzForge, a comprehensive security analysis platform built on Prefect 3 that automates security testing workflows. FuzzForge provides 6 production-ready workflows that run static analysis, secret detection, infrastructure scanning, penetration testing, and custom fuzzing campaigns with Docker-based isolation and SARIF-compliant reporting.
## 🚀 Quick Navigation
### 📚 **Tutorials** - *Learn by doing*
Perfect for newcomers who want to learn FuzzForge step by step.
- [**Getting Started**](tutorials/getting-started.md) - Complete setup from installation to first workflow
- [**First Workflow**](tutorials/first-workflow.md) - Run your first security workflow
- [**Building Custom Workflows**](tutorials/building-custom-workflow.md) - Create and deploy custom workflows
### 🛠️ **How-To Guides** - *Problem-focused solutions*
Step-by-step guides for specific tasks and common problems.
- [**Installation**](how-to/installation.md) - Install FuzzForge with proper Docker setup
- [**Docker Setup**](how-to/docker-setup.md) - Configure Docker with insecure registry (required)
- [**Running Workflows**](how-to/running-workflows.md) - Execute different workflow types
- [**CLI Usage**](how-to/cli-usage.md) - Command-line interface patterns
- [**API Integration**](how-to/api-integration.md) - REST API usage and integration
- [**MCP Integration**](how-to/mcp-integration.md) - AI assistant integration setup
- [**Troubleshooting**](how-to/troubleshooting.md) - Common issues and solutions
### 💡 **Concepts** - *Understanding-oriented*
Background information and conceptual explanations.
- [**Architecture**](concepts/architecture.md) - System design and component interactions
- [**Workflows**](concepts/workflows.md) - How workflows function and interact
- [**Security Analysis**](concepts/security-analysis.md) - Security analysis methodology
- [**Docker Containers**](concepts/docker-containers.md) - Containerization approach
- [**SARIF Format**](concepts/sarif-format.md) - Industry-standard security results format
### 📖 **Reference** - *Information-oriented*
Technical reference materials and specifications.
#### Workflows
- [**All Workflows**](reference/workflows/index.md) - Complete workflow reference
- [**Static Analysis**](reference/workflows/static-analysis.md) - Code vulnerability detection
- [**Secret Detection**](reference/workflows/secret-detection.md) - Credential discovery
- [**Infrastructure Scan**](reference/workflows/infrastructure-scan.md) - Infrastructure security
- [**Penetration Testing**](reference/workflows/penetration-testing.md) - Security testing
- [**Language Fuzzing**](reference/workflows/language-fuzzing.md) - Input validation testing
- [**Security Assessment**](reference/workflows/security-assessment.md) - Comprehensive analysis
#### APIs and Interfaces
- [**REST API**](reference/api/index.md) - Complete API documentation
- [**CLI Reference**](reference/cli/index.md) - Command-line interface
- [**Configuration**](reference/configuration.md) - System configuration options
#### Additional Resources
- [**AI Orchestration (Advanced)**](../ai/docs/index.md) - Multi-agent orchestration, A2A services, ingestion, and LLM configuration
- [**Docker Configuration**](reference/docker-configuration.md) - Complete Docker setup requirements
- [**Contributing**](reference/contributing.md) - Development and contribution guidelines
- [**FAQ**](reference/faq.md) - Frequently asked questions
- [**Changelog**](reference/changelog.md) - Version history and updates
---
## 🎯 FuzzForge at a Glance
**6 Production Workflows:**
- Static Analysis (Semgrep, Bandit, CodeQL)
- Secret Detection (TruffleHog, Gitleaks, detect-secrets)
- Infrastructure Scan (Checkov, Hadolint, Kubesec)
- Penetration Testing (Nuclei, Nmap, SQLMap, Nikto)
- Language Fuzzing (AFL++, libFuzzer, Cargo Fuzz)
- Security Assessment (Comprehensive multi-tool analysis)
**Multiple Interfaces:**
- 💻 **CLI**: `fuzzforge runs submit static_analysis_scan /path/to/code`
- 🐍 **Python SDK**: Programmatic workflow integration
- 🌐 **REST API**: HTTP-based workflow management
- 🤖 **MCP**: AI assistant integration (Claude, ChatGPT)
**Key Features:**
- Container-based workflow execution with Docker isolation
- SARIF-compliant security results format
- Real-time workflow monitoring and progress tracking
- Persistent result storage with shared volumes
- Custom Docker image building for specialized tools
---
## 🚨 Important Setup Requirement
**Docker Insecure Registry Configuration Required**
FuzzForge uses a local Docker registry for workflow images. You **must** configure Docker to allow insecure registries:
```json
{
"insecure-registries": ["localhost:5001"]
}
```
See [Docker Setup Guide](how-to/docker-setup.md) for detailed configuration instructions.
---
## 📋 Documentation Framework
This documentation follows the [Diátaxis framework](https://diataxis.fr/):
- **Tutorials**: Learning-oriented, hands-on lessons
- **How-to guides**: Problem-oriented, step-by-step instructions
- **Concepts**: Understanding-oriented, theoretical knowledge
- **Reference**: Information-oriented, technical specifications
---
**New to FuzzForge?** Start with the [Getting Started Tutorial](tutorials/getting-started.md)
**Need help?** Check the [FAQ](reference/faq.md) or [Troubleshooting Guide](how-to/troubleshooting.md)
**Want to contribute?** See the [Contributing Guide](reference/contributing.md)
docs/package-lock.json
Generated
+19814
View File
File diff suppressed because it is too large Load Diff
docs/package.json
+49
View File
@@ -0,0 +1,49 @@
{
"name": "fuzzforge-docs",
"version": "0.0.1",
"private": true,
"scripts": {
"docusaurus": "docusaurus",
"start": "docusaurus start",
"build": "docusaurus build",
"swizzle": "docusaurus swizzle",
"deploy": "docusaurus deploy",
"clear": "docusaurus clear",
"serve": "docusaurus serve",
"write-translations": "docusaurus write-translations",
"write-heading-ids": "docusaurus write-heading-ids",
"typecheck": "tsc"
},
"dependencies": {
"@docusaurus/core": "^3.9.0",
"@docusaurus/preset-classic": "^3.9.0",
"@docusaurus/theme-mermaid": "^3.9.0",
"@easyops-cn/docusaurus-search-local": "^0.52.1",
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
"react": "^19.0.0",
"react-dom": "^19.0.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.9.0",
"@docusaurus/tsconfig": "^3.9.0",
"@docusaurus/types": "^3.9.0",
"typescript": "~5.6.2"
},
"browserslist": {
"production": [
">0.5%",
"not dead",
"not op_mini all"
],
"development": [
"last 3 chrome version",
"last 3 firefox version",
"last 5 safari version"
]
},
"engines": {
"node": ">=18.0"
}
}
docs/sidebars.ts
+34
View File
@@ -0,0 +1,34 @@
import type { SidebarsConfig } from "@docusaurus/plugin-content-docs";
// This runs in Node.js - Don't use client-side code here (browser APIs, JSX...)
/**
* Creating a sidebar enables you to:
- create an ordered group of docs
- render a sidebar for each doc of that group
- provide next/previous navigation
The sidebars can be generated from the filesystem, or explicitly defined here.
Create as many sidebars as you want.
*/
const sidebars: SidebarsConfig = {
// By default, Docusaurus generates a sidebar from the docs folder structure
backendSidebar: [{ type: "autogenerated", dirName: "." }],
aiSidebar: [{ type: "autogenerated", dirName: "ai" }],
// But you can create a sidebar manually
/*
tutorialSidebar: [
'intro',
'hello',
{
type: 'category',
label: 'Tutorial',
items: ['tutorial-basics/create-a-document'],
},
],
*/
};
export default sidebars;
docs/src/components/HomepageFeatures/index.tsx
+71
View File
@@ -0,0 +1,71 @@
import type { ReactNode } from "react";
import clsx from "clsx";
import Heading from "@theme/Heading";
import styles from "./styles.module.css";
type FeatureItem = {
title: string;
Svg: React.ComponentType<React.ComponentProps<"svg">>;
description: ReactNode;
};
const FeatureList: FeatureItem[] = [
{
title: "AI-Powered Vulnerability Research",
Svg: require("@site/static/img/icon-chip.svg").default,
description: (
<>
Intelligent agents that understand your project context, suggest next
steps, and collaborate across security domains.
</>
),
},
{
title: "Modular Security Workflows",
Svg: require("@site/static/img/icon-stack.svg").default,
description: (
<>
Orchestrate SAST, fuzzing, reversing, and triage tools using reusable
python-based workflow definitions.
</>
),
},
{
title: "Security Marketplace",
Svg: require("@site/static/img/icon-marketplace.svg").default,
description: (
<>
Community-driven repository for agents, fuzzing corpora, grammars, CVEs,
and complete security workflows.
</>
),
},
];
function Feature({ title, Svg, description }: FeatureItem) {
return (
<div className={clsx("col col--4")}>
<div className="text--center">
<Svg className={styles.featureSvg} role="img" />
</div>
<div className="text--center padding-horiz--md">
<Heading as="h3">{title}</Heading>
<p>{description}</p>
</div>
</div>
);
}
export default function HomepageFeatures(): ReactNode {
return (
<section className={styles.features}>
<div className="container">
<div className="row">
{FeatureList.map((props, idx) => (
<Feature key={idx} {...props} />
))}
</div>
</div>
</section>
);
}
docs/src/components/HomepageFeatures/styles.module.css
+11
View File
@@ -0,0 +1,11 @@
.features {
display: flex;
align-items: center;
padding: 2rem 0;
width: 100%;
}
.featureSvg {
height: 200px;
width: 200px;
}
docs/src/css/custom.css
+29
View File
@@ -0,0 +1,29 @@
/**
* Any CSS included here will be global. The classic template
* bundles Infima by default. Infima is a CSS framework designed to
* work well for content-centric websites.
*/
:root {
--ifm-color-primary: #6c61f2; /* Medium Slate Blue */
--ifm-color-primary-dark: #201d40; /* Space Cadet */
--ifm-color-primary-darker: #1b1e3e; /* Space Cadet 2 */
--ifm-color-primary-darkest: #100f29; /* Oxford Blue */
--ifm-color-primary-light: #acb4fd; /* Periwinkle */
--ifm-color-primary-lighter: #acb4fd; /* Periwinkle (Re-used) */
--ifm-color-primary-lightest: #acb4fd; /* Periwinkle (Re-used) */
--ifm-code-font-size: 95%;
--docusaurus-highlighted-code-line-bg: rgba(0, 0, 0, 0.1);
}
/* For readability concerns, you should choose a lighter palette in dark mode. */
[data-theme="dark"] {
--ifm-color-primary: #acb4fd; /* Periwinkle */
--ifm-color-primary-dark: #6c61f2; /* Medium Slate Blue */
--ifm-color-primary-darker: #6c61f2; /* Medium Slate Blue (Re-used) */
--ifm-color-primary-darkest: #6c61f2; /* Medium Slate Blue (Re-used) */
--ifm-color-primary-light: #acb4fd; /* Periwinkle (Re-used) */
--ifm-color-primary-lighter: #acb4fd; /* Periwinkle (Re-used) */
--ifm-color-primary-lightest: #acb4fd; /* Periwinkle (Re-used) */
--docusaurus-highlighted-code-line-bg: rgba(0, 0, 0, 0.3);
}
docs/src/pages/index.module.css
+23
View File
@@ -0,0 +1,23 @@
/**
* CSS files with the .module.css suffix will be treated as CSS modules
* and scoped locally.
*/
.heroBanner {
padding: 4rem 0;
text-align: center;
position: relative;
overflow: hidden;
}
@media screen and (max-width: 996px) {
.heroBanner {
padding: 2rem;
}
}
.buttons {
display: flex;
align-items: center;
justify-content: center;
}
docs/src/pages/index.tsx
+46
View File
@@ -0,0 +1,46 @@
import type { ReactNode } from "react";
import clsx from "clsx";
import Link from "@docusaurus/Link";
import useDocusaurusContext from "@docusaurus/useDocusaurusContext";
import Layout from "@theme/Layout";
import HomepageFeatures from "@site/src/components/HomepageFeatures";
import Heading from "@theme/Heading";
import styles from "./index.module.css";
function HomepageHeader() {
const { siteConfig } = useDocusaurusContext();
return (
<header className={clsx("hero hero--primary", styles.heroBanner)}>
<div className="container">
<Heading as="h1" className="hero__title">
{siteConfig.title}
</Heading>
<p className="hero__subtitle">{siteConfig.tagline}</p>
<div className={styles.buttons}>
<Link
className="button button--secondary button--lg"
to="/docs/intro"
>
Get Started - 5min ⏱️
</Link>
</div>
</div>
</header>
);
}
export default function Home(): ReactNode {
const { siteConfig } = useDocusaurusContext();
return (
<Layout
title={`${siteConfig.title}`}
description="Automate vulnerability discovery with intelligent fuzzing, AI-driven analysis, and a marketplace of security tools. From code to exploit in minutes, not months."
>
<HomepageHeader />
<main>
<HomepageFeatures />
</main>
</Layout>
);
}
docs/static/.nojekyll
Vendored
View File
docs/static/img/favicon.ico
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

docs/static/img/fuzzforge-logo-1024-rounded.png
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 469 KiB

docs/static/img/fuzzforge-social-card.jpg
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 71 KiB

docs/static/img/fuzzforge_white.png
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 506 KiB

docs/static/img/icon-chip.svg
Vendored
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--solar h-7 w-7 text-indigo-400" width="1em" height="1em" viewBox="0 0 24 24"><path fill="currentColor" d="M9.18 9.18c.054-.052.149-.118.451-.159c.323-.043.761-.044 1.439-.044h1.86c.678 0 1.116.001 1.438.044c.303.041.398.107.45.16c.054.053.12.148.16.45c.044.323.045.761.045 1.439v1.86c0 .678-.001 1.116-.045 1.438c-.04.303-.106.398-.16.45c-.052.054-.147.12-.45.16c-.322.044-.76.045-1.438.045h-1.86c-.678 0-1.116-.001-1.439-.045c-.302-.04-.397-.106-.45-.16c-.053-.052-.119-.147-.16-.45c-.043-.322-.044-.76-.044-1.438v-1.86c0-.678.001-1.116.044-1.439c.041-.302.107-.397.16-.45" opacity=".5"></path><path fill="currentColor" fill-rule="evenodd" d="M12.698 2.698a.698.698 0 0 0-1.396 0v2.79q-.764 0-1.395.017V2.698a.698.698 0 0 0-1.395 0v2.79q0 .056.008.108c-.936.115-1.585.353-2.078.846s-.731 1.142-.846 2.078a1 1 0 0 0-.108-.008h-2.79a.698.698 0 0 0 0 1.395h2.807q-.016.63-.016 1.395H2.698a.698.698 0 0 0 0 1.396h2.79q0 .764.017 1.395H2.698a.698.698 0 0 0 0 1.395h2.79a1 1 0 0 0 .108-.008c.115.936.353 1.585.846 2.078s1.142.731 2.078.846a1 1 0 0 0-.008.108v2.79a.698.698 0 0 0 1.395 0v-2.807q.63.016 1.395.016v2.791a.698.698 0 0 0 1.396 0v-2.79q.764 0 1.395-.017v2.807a.698.698 0 0 0 1.395 0v-2.79a1 1 0 0 0-.008-.108c.936-.115 1.585-.353 2.078-.846s.731-1.142.846-2.078q.053.009.108.008h2.79a.698.698 0 0 0 0-1.395h-2.807q.016-.63.016-1.395h2.791a.698.698 0 0 0 0-1.396h-2.79q0-.764-.017-1.395h2.807a.698.698 0 0 0 0-1.395h-2.79a1 1 0 0 0-.108.008c-.115-.936-.353-1.585-.846-2.078s-1.142-.731-2.078-.846a1 1 0 0 0 .008-.108v-2.79a.698.698 0 0 0-1.395 0v2.807a56 56 0 0 0-1.395-.016zm-3.252 4.94c.426-.057.96-.057 1.578-.057h1.952c.619 0 1.151 0 1.578.058c.458.061.896.2 1.252.555c.355.356.494.794.555 1.252c.058.426.058.96.058 1.578v1.952c0 .619 0 1.151-.058 1.578c-.061.458-.2.896-.555 1.252c-.356.355-.794.494-1.252.555c-.427.058-.96.058-1.578.058h-1.952c-.619 0-1.152 0-1.578-.058c-.458-.061-.896-.2-1.252-.555c-.355-.356-.494-.794-.555-1.252c-.058-.427-.058-.96-.058-1.578v-1.952c0-.619 0-1.152.058-1.578c.061-.458.2-.896.555-1.252c.356-.355.794-.494 1.252-.555" clip-rule="evenodd"></path><path fill="currentColor" d="M12.966 10.545a.698.698 0 0 0-1.135-.811l-1.329 1.86a.698.698 0 0 0 .568 1.103h.505l-.541.758a.698.698 0 1 0 1.135.81l1.329-1.86a.698.698 0 0 0-.568-1.103h-.505z"></path></svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

docs/static/img/icon-marketplace.svg
Vendored
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--solar h-7 w-7 text-indigo-400" width="1em" height="1em" viewBox="0 0 24 24"><path fill="currentColor" d="M14.5 21.991V18.5c0-.935 0-1.402-.201-1.75a1.5 1.5 0 0 0-.549-.549C13.402 16 12.935 16 12 16s-1.402 0-1.75.201a1.5 1.5 0 0 0-.549.549c-.201.348-.201.815-.201 1.75v3.491z"></path><path fill="currentColor" fill-rule="evenodd" d="M5.732 12c-.89 0-1.679-.376-2.232-.967V14c0 3.771 0 5.657 1.172 6.828c.943.944 2.348 1.127 4.828 1.163h5c2.48-.036 3.885-.22 4.828-1.163C20.5 19.657 20.5 17.771 20.5 14v-2.966a3.06 3.06 0 0 1-5.275-1.789l-.073-.728a3.167 3.167 0 1 1-6.307.038l-.069.69A3.06 3.06 0 0 1 5.732 12m8.768 6.5v3.491h-5V18.5c0-.935 0-1.402.201-1.75a1.5 1.5 0 0 1 .549-.549C10.598 16 11.065 16 12 16s1.402 0 1.75.201a1.5 1.5 0 0 1 .549.549c.201.348.201.815.201 1.75" clip-rule="evenodd" opacity=".5"></path><path fill="currentColor" d="M9.5 2h5l.652 6.517a3.167 3.167 0 1 1-6.304 0z"></path><path fill="currentColor" d="M3.33 5.351c.178-.89.267-1.335.448-1.696a3 3 0 0 1 1.889-1.548C6.057 2 6.51 2 7.418 2h2.083l-.725 7.245a3.06 3.06 0 1 1-6.044-.904zm17.34 0c-.178-.89-.267-1.335-.448-1.696a3 3 0 0 0-1.888-1.548C17.944 2 17.49 2 16.582 2H14.5l.725 7.245a3.06 3.06 0 1 0 6.043-.904z" opacity=".7"></path></svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.3 KiB

docs/static/img/icon-stack.svg
Vendored
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--solar h-7 w-7 text-indigo-400" width="1em" height="1em" viewBox="0 0 24 24"><path fill="currentColor" d="M4.979 9.685C2.993 8.891 2 8.494 2 8s.993-.89 2.979-1.685l2.808-1.123C9.773 4.397 10.767 4 12 4s2.227.397 4.213 1.192l2.808 1.123C21.007 7.109 22 7.506 22 8s-.993.89-2.979 1.685l-2.808 1.124C14.227 11.603 13.233 12 12 12s-2.227-.397-4.213-1.191z"></path><path fill="currentColor" fill-rule="evenodd" d="M2 8c0 .494.993.89 2.979 1.685l2.808 1.124C9.773 11.603 10.767 12 12 12s2.227-.397 4.213-1.191l2.808-1.124C21.007 8.891 22 8.494 22 8s-.993-.89-2.979-1.685l-2.808-1.123C14.227 4.397 13.233 4 12 4s-2.227.397-4.213 1.192L4.98 6.315C2.993 7.109 2 7.506 2 8" clip-rule="evenodd"></path><path fill="currentColor" d="m5.766 10l-.787.315C2.993 11.109 2 11.507 2 12s.993.89 2.979 1.685l2.808 1.124C9.773 15.603 10.767 16 12 16s2.227-.397 4.213-1.191l2.808-1.124C21.007 12.891 22 12.493 22 12s-.993-.89-2.979-1.685L18.234 10l-2.021.809C14.227 11.603 13.233 12 12 12s-2.227-.397-4.213-1.191z" opacity=".7"></path><path fill="currentColor" d="m5.766 14l-.787.315C2.993 15.109 2 15.507 2 16s.993.89 2.979 1.685l2.808 1.124C9.773 19.603 10.767 20 12 20s2.227-.397 4.213-1.192l2.808-1.123C21.007 16.891 22 16.494 22 16c0-.493-.993-.89-2.979-1.685L18.234 14l-2.021.809C14.227 15.603 13.233 16 12 16s-2.227-.397-4.213-1.191z" opacity=".4"></path></svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.4 KiB

docs/static/videos/llm_workflow.gif
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.4 MiB

docs/static/videos/llm_workflow.mov
Vendored
BIN
View File
Binary file not shown.
docs/static/videos/manual_workflow.gif
Vendored
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 MiB

docs/static/videos/manual_workflow.mov
Vendored
BIN
View File
Binary file not shown.
docs/tsconfig.json
+8
View File
@@ -0,0 +1,8 @@
{
// This file is not used in compilation. It is here just for a nice editor experience.
"extends": "@docusaurus/tsconfig",
"compilerOptions": {
"baseUrl": "."
},
"exclude": [".docusaurus", "build"]
}