mirror of
https://github.com/FuzzingLabs/fuzzforge_ai.git
synced 2026-06-10 12:53:56 +02:00
Release v0.7.3 - Android workflows, LiteLLM integration, ARM64 support (#32)
* ci: add worker validation and Docker build checks Add automated validation to prevent worker-related issues: **Worker Validation Script:** - New script: .github/scripts/validate-workers.sh - Validates all workers in docker-compose.yml exist - Checks required files: Dockerfile, requirements.txt, worker.py - Verifies files are tracked by git (not gitignored) - Detects gitignore issues that could hide workers **CI Workflow Updates:** - Added validate-workers job (runs on every PR) - Added build-workers job (runs if workers/ modified) - Uses Docker Buildx for caching - Validates Docker images build successfully - Updated test-summary to check validation results **PR Template:** - New pull request template with comprehensive checklist - Specific section for worker-related changes - Reminds contributors to validate worker files - Includes documentation and changelog reminders These checks would have caught the secrets worker gitignore issue. Implements Phase 1 improvements from CI/CD quality assessment. * fix: add dev branch to test workflow triggers The test workflow was configured for 'develop' but the actual branch is named 'dev'. This caused tests not to run on PRs to dev branch. Now tests will run on: - PRs to: main, master, dev, develop - Pushes to: main, master, dev, develop, feature/** * fix: properly detect worker file changes in CI The previous condition used invalid GitHub context field. Now uses git diff to properly detect changes to workers/ or docker-compose.yml. Behavior: - Job always runs the check step - Detects if workers/ or docker-compose.yml modified - Only builds Docker images if workers actually changed - Shows clear skip message when no worker changes detected * feat: Add Python SAST workflow with three security analysis tools Implements Issue #5 - Python SAST workflow that combines: - Dependency scanning (pip-audit) for CVE detection - Security linting (Bandit) for vulnerability patterns - Type checking (Mypy) for type safety issues ## Changes **New Modules:** - `DependencyScanner`: Scans Python dependencies for known CVEs using pip-audit - `BanditAnalyzer`: Analyzes Python code for security issues using Bandit - `MypyAnalyzer`: Checks Python code for type safety issues using Mypy **New Workflow:** - `python_sast`: Temporal workflow that orchestrates all three SAST tools - Runs tools in parallel for fast feedback (3-5 min vs hours for fuzzing) - Generates unified SARIF report with findings from all tools - Supports configurable severity/confidence thresholds **Updates:** - Added SAST dependencies to Python worker (bandit, pip-audit, mypy) - Updated module __init__.py files to export new analyzers - Added type_errors.py test file to vulnerable_app for Mypy validation ## Testing Workflow tested successfully on vulnerable_app: - ✅ Bandit: Detected 9 security issues (command injection, unsafe functions) - ✅ Mypy: Detected 5 type errors - ✅ DependencyScanner: Ran successfully (no CVEs in test dependencies) - ✅ SARIF export: Generated valid SARIF with 14 total findings * fix: Remove unused imports to pass linter * fix: resolve live monitoring bug, remove deprecated parameters, and auto-start Python worker - Fix live monitoring style error by calling _live_monitor() helper directly - Remove default_parameters duplication from 10 workflow metadata files - Remove deprecated volume_mode parameter from 26 files across CLI, SDK, backend, and docs - Configure Python worker to start automatically with docker compose up - Clean up constants, validation, completion, and example files Fixes # - Live monitoring now works correctly with --live flag - Workflow metadata follows JSON Schema standard - Cleaner codebase without deprecated volume_mode - Python worker (most commonly used) starts by default * fix: resolve linter errors and optimize CI worker builds - Remove unused Literal import from backend findings model - Remove unnecessary f-string prefixes in CLI findings command - Optimize GitHub Actions to build only modified workers - Detect specific worker changes (python, secrets, rust, android, ossfuzz) - Build only changed workers instead of all 5 - Build all workers if docker-compose.yml changes - Significantly reduces CI build time * feat: Add Android static analysis workflow with Jadx, OpenGrep, and MobSF Comprehensive Android security testing workflow converted from Prefect to Temporal architecture: Modules (3): - JadxDecompiler: APK to Java source code decompilation - OpenGrepAndroid: Static analysis with Android-specific security rules - MobSFScanner: Comprehensive mobile security framework integration Custom Rules (13): - clipboard-sensitive-data, hardcoded-secrets, insecure-data-storage - insecure-deeplink, insecure-logging, intent-redirection - sensitive_data_sharedPreferences, sqlite-injection - vulnerable-activity, vulnerable-content-provider, vulnerable-service - webview-javascript-enabled, webview-load-arbitrary-url Workflow: - 6-phase Temporal workflow: download → Jadx → OpenGrep → MobSF → SARIF → upload - 4 activities: decompile_with_jadx, scan_with_opengrep, scan_with_mobsf, generate_android_sarif - SARIF output combining findings from all security tools Docker Worker: - ARM64 Mac compatibility via amd64 platform emulation - Pre-installed: Android SDK, Jadx 1.4.7, OpenGrep 1.45.0, MobSF 3.9.7 - MobSF runs as background service with API key auto-generation - Added aiohttp for async HTTP communication Test APKs: - BeetleBug.apk and shopnest.apk for workflow validation * fix(android): correct activity names and MobSF API key generation - Fix activity names in workflow.py (get_target, upload_results, cleanup_cache) - Fix MobSF API key generation in Dockerfile startup script (cut delimiter) - Update activity parameter signatures to match actual implementations - Workflow now executes successfully with Jadx and OpenGrep * feat: add platform-aware worker architecture with ARM64 support Implement platform-specific Dockerfile selection and graceful tool degradation to support both x86_64 and ARM64 (Apple Silicon) platforms. **Backend Changes:** - Add system info API endpoint (/system/info) exposing host filesystem paths - Add FUZZFORGE_HOST_ROOT environment variable to backend service - Add graceful degradation in MobSF activity for ARM64 platforms **CLI Changes:** - Implement multi-strategy path resolution (backend API, .fuzzforge marker, env var) - Add platform detection (linux/amd64 vs linux/arm64) - Add worker metadata.yaml reading for platform capabilities - Auto-select appropriate Dockerfile based on detected platform - Pass platform-specific env vars to docker-compose **Worker Changes:** - Create workers/android/metadata.yaml defining platform capabilities - Rename Dockerfile -> Dockerfile.amd64 (full toolchain with MobSF) - Create Dockerfile.arm64 (excludes MobSF due to Rosetta 2 incompatibility) - Update docker-compose.yml to use ${ANDROID_DOCKERFILE} variable **Workflow Changes:** - Handle MobSF "skipped" status gracefully in workflow - Log clear warnings when tools are unavailable on platform **Key Features:** - Automatic platform detection and Dockerfile selection - Graceful degradation when tools unavailable (MobSF on ARM64) - Works from any directory (backend API provides paths) - Manual override via environment variables - Clear user feedback about platform and selected Dockerfile **Benefits:** - Android workflow now works on Apple Silicon Macs - No code changes needed for other workflows - Convention established for future platform-specific workers Closes: MobSF Rosetta 2 incompatibility issue Implements: Platform-aware worker architecture (Option B) * fix: make MobSFScanner import conditional for ARM64 compatibility - Add try-except block to conditionally import MobSFScanner in modules/android/__init__.py - Allows Android worker to start on ARM64 without MobSF dependencies (aiohttp) - MobSF activity gracefully skips on ARM64 with clear warning message - Remove workflow path detection logic (not needed - workflows receive directories) Platform-aware architecture fully functional on ARM64: - CLI detects ARM64 and selects Dockerfile.arm64 automatically - Worker builds and runs without MobSF on ARM64 - Jadx successfully decompiles APKs (4145 files from BeetleBug.apk) - OpenGrep finds security vulnerabilities (8 issues found) - MobSF gracefully skips with warning on ARM64 - Graceful degradation working as designed Tested with: ff workflow run android_static_analysis test_projects/android_test/ \ --wait --no-interactive apk_path=BeetleBug.apk decompile_apk=true Results: 8 security findings (1 ERROR, 7 WARNINGS) * docs: update CHANGELOG with Android workflow and ARM64 support Added [Unreleased] section documenting: - Android Static Analysis Workflow (Jadx, OpenGrep, MobSF) - Platform-Aware Worker Architecture with ARM64 support - Python SAST Workflow - CI/CD improvements and worker validation - CLI enhancements - Bug fixes and technical changes Fixed date typo: 2025-01-16 → 2025-10-16 * fix: resolve linter errors in Android modules - Remove unused imports from mobsf_scanner.py (asyncio, hashlib, json, Optional) - Remove unused variables from opengrep_android.py (start_col, end_col) - Remove duplicate Path import from workflow.py * ci: support multi-platform Dockerfiles in worker validation Updated worker validation script to accept both: - Single Dockerfile pattern (existing workers) - Multi-platform Dockerfile pattern (Dockerfile.amd64, Dockerfile.arm64, etc.) This enables platform-aware worker architectures like the Android worker which uses different Dockerfiles for x86_64 and ARM64 platforms. * Feature/litellm proxy (#27) * feat: seed governance config and responses routing * Add env-configurable timeout for proxy providers * Integrate LiteLLM OTEL collector and update docs * Make .env.litellm optional for LiteLLM proxy * Add LiteLLM proxy integration with model-agnostic virtual keys Changes: - Bootstrap generates 3 virtual keys with individual budgets (CLI: $100, Task-Agent: $25, Cognee: $50) - Task-agent loads config at runtime via entrypoint script to wait for bootstrap completion - All keys are model-agnostic by default (no LITELLM_DEFAULT_MODELS restrictions) - Bootstrap handles database/env mismatch after docker prune by deleting stale aliases - CLI and Cognee configured to use LiteLLM proxy with virtual keys - Added comprehensive documentation in volumes/env/README.md Technical details: - task-agent entrypoint waits for keys in .env file before starting uvicorn - Bootstrap creates/updates TASK_AGENT_API_KEY, COGNEE_API_KEY, and OPENAI_API_KEY - Removed hardcoded API keys from docker-compose.yml - All services route through http://localhost:10999 proxy * Fix CLI not loading virtual keys from global .env Project .env files with empty OPENAI_API_KEY values were overriding the global virtual keys. Updated _load_env_file_if_exists to only override with non-empty values. * Fix agent executor not passing API key to LiteLLM The agent was initializing LiteLlm without api_key or api_base, causing authentication errors when using the LiteLLM proxy. Now reads from OPENAI_API_KEY/LLM_API_KEY and LLM_ENDPOINT environment variables and passes them to LiteLlm constructor. * Auto-populate project .env with virtual key from global config When running 'ff init', the command now checks for a global volumes/env/.env file and automatically uses the OPENAI_API_KEY virtual key if found. This ensures projects work with LiteLLM proxy out of the box without manual key configuration. * docs: Update README with LiteLLM configuration instructions Add note about LITELLM_GEMINI_API_KEY configuration and clarify that OPENAI_API_KEY default value should not be changed as it's used for the LLM proxy. * Refactor workflow parameters to use JSON Schema defaults Consolidates parameter defaults into JSON Schema format, removing the separate default_parameters field. Adds extract_defaults_from_json_schema() helper to extract defaults from the standard schema structure. Updates LiteLLM proxy config to use LITELLM_OPENAI_API_KEY environment variable. * Remove .env.example from task_agent * Fix MDX syntax error in llm-proxy.md * fix: apply default parameters from metadata.yaml automatically Fixed TemporalManager.run_workflow() to correctly apply default parameter values from workflow metadata.yaml files when parameters are not provided by the caller. Previous behavior: - When workflow_params was empty {}, the condition `if workflow_params and 'parameters' in metadata` would fail - Parameters would not be extracted from schema, resulting in workflows receiving only target_id with no other parameters New behavior: - Removed the `workflow_params and` requirement from the condition - Now explicitly checks for defaults in parameter spec - Applies defaults from metadata.yaml automatically when param not provided - Workflows receive all parameters with proper fallback: provided value > metadata default > None This makes metadata.yaml the single source of truth for parameter defaults, removing the need for workflows to implement defensive default handling. Affected workflows: - llm_secret_detection (was failing with KeyError) - All other workflows now benefit from automatic default application Co-authored-by: tduhamel42 <tduhamel@fuzzinglabs.com> * fix: add default values to llm_analysis workflow parameters Resolves validation error where agent_url was None when not explicitly provided. The TemporalManager applies defaults from metadata.yaml, not from module input schemas, so all parameters need defaults in the workflow metadata. Changes: - Add default agent_url, llm_model (gpt-5-mini), llm_provider (openai) - Expand file_patterns to 45 comprehensive patterns covering code, configs, secrets, and Docker files - Increase default limits: max_files (10), max_file_size (100KB), timeout (90s) * refactor: replace .env.example with .env.template in documentation - Remove volumes/env/.env.example file - Update all documentation references to use .env.template instead - Update bootstrap script error message - Update .gitignore comment * feat(cli): add worker management commands with improved progress feedback Add comprehensive CLI commands for managing Temporal workers: - ff worker list - List workers with status and uptime - ff worker start <name> - Start specific worker with optional rebuild - ff worker stop - Safely stop all workers without affecting core services Improvements: - Live progress display during worker startup with Rich Status spinner - Real-time elapsed time counter and container state updates - Health check status tracking (starting → unhealthy → healthy) - Helpful contextual hints at 10s, 30s, 60s intervals - Better timeout messages showing last known state Worker management enhancements: - Use 'docker compose' (space) instead of 'docker-compose' (hyphen) - Stop workers individually with 'docker stop' to avoid stopping core services - Platform detection and Dockerfile selection (ARM64/AMD64) Documentation: - Updated docker-setup.md with CLI commands as primary method - Created comprehensive cli-reference.md with all commands and examples - Added worker management best practices * fix: MobSF scanner now properly parses files dict structure MobSF returns 'files' as a dict (not list): {"filename": "line_numbers"} The parser was treating it as a list, causing zero findings to be extracted. Now properly iterates over the dict and creates one finding per affected file with correct line numbers and metadata (CWE, OWASP, MASVS, CVSS). Fixed in both code_analysis and behaviour sections. * chore: bump version to 0.7.3 * docs: fix broken documentation links in cli-reference * chore: add worker startup documentation and cleanup .gitignore - Add workflow-to-worker mapping tables across documentation - Update troubleshooting guide with worker requirements section - Enhance getting started guide with worker examples - Add quick reference to docker setup guide - Add WEEK_SUMMARY*.md pattern to .gitignore * docs: update CHANGELOG with missing versions and recent changes - Add Unreleased section for post-v0.7.3 documentation updates - Add v0.7.2 entry with bug fixes and worker improvements - Document that v0.7.1 was re-tagged as v0.7.2 - Fix v0.6.0 date to "Undocumented" (no tag exists) - Add version comparison links for easier navigation * chore: bump all package versions to 0.7.3 for consistency * Update GitHub link to fuzzforge_ai --------- Co-authored-by: Songbird99 <150154823+Songbird99@users.noreply.github.com> Co-authored-by: Songbird <Songbirdx99@gmail.com>
This commit is contained in:
@@ -225,7 +225,7 @@ docker compose up -d # All workers start
|
||||
Set up AI workflows with API keys:
|
||||
|
||||
```bash
|
||||
cp volumes/env/.env.example volumes/env/.env
|
||||
cp volumes/env/.env.template volumes/env/.env
|
||||
# Edit .env and add your API keys (OpenAI, Anthropic, etc.)
|
||||
```
|
||||
|
||||
|
||||
@@ -110,7 +110,32 @@ fuzzforge workflow run secret_detection ./codebase
|
||||
|
||||
### Manual Worker Management
|
||||
|
||||
Start specific workers when needed:
|
||||
**Quick Reference - Workflow to Worker Mapping:**
|
||||
|
||||
| Workflow | Worker Service | Docker Command |
|
||||
|----------|----------------|----------------|
|
||||
| `security_assessment`, `python_sast`, `llm_analysis`, `atheris_fuzzing` | worker-python | `docker compose up -d worker-python` |
|
||||
| `android_static_analysis` | worker-android | `docker compose up -d worker-android` |
|
||||
| `cargo_fuzzing` | worker-rust | `docker compose up -d worker-rust` |
|
||||
| `ossfuzz_campaign` | worker-ossfuzz | `docker compose up -d worker-ossfuzz` |
|
||||
| `llm_secret_detection`, `trufflehog_detection`, `gitleaks_detection` | worker-secrets | `docker compose up -d worker-secrets` |
|
||||
|
||||
FuzzForge CLI provides convenient commands for managing workers:
|
||||
|
||||
```bash
|
||||
# List all workers and their status
|
||||
ff worker list
|
||||
ff worker list --all # Include stopped workers
|
||||
|
||||
# Start a specific worker
|
||||
ff worker start python
|
||||
ff worker start android --build # Rebuild before starting
|
||||
|
||||
# Stop all workers
|
||||
ff worker stop
|
||||
```
|
||||
|
||||
You can also use Docker commands directly:
|
||||
|
||||
```bash
|
||||
# Start a single worker
|
||||
@@ -123,6 +148,33 @@ docker compose --profile workers up -d
|
||||
docker stop fuzzforge-worker-ossfuzz
|
||||
```
|
||||
|
||||
### Stopping Workers Properly
|
||||
|
||||
The easiest way to stop workers is using the CLI:
|
||||
|
||||
```bash
|
||||
# Stop all running workers (recommended)
|
||||
ff worker stop
|
||||
```
|
||||
|
||||
This command safely stops all worker containers without affecting core services.
|
||||
|
||||
Alternatively, you can use Docker commands:
|
||||
|
||||
```bash
|
||||
# Stop individual worker
|
||||
docker stop fuzzforge-worker-python
|
||||
|
||||
# Stop all workers using docker compose
|
||||
# Note: This requires the --profile flag because workers are in profiles
|
||||
docker compose down --profile workers
|
||||
```
|
||||
|
||||
**Important:** Workers use Docker Compose profiles to prevent auto-starting. When using Docker commands directly:
|
||||
- `docker compose down` (without `--profile workers`) does NOT stop workers
|
||||
- Workers remain running unless explicitly stopped with the profile flag or `docker stop`
|
||||
- Use `ff worker stop` for the safest option that won't affect core services
|
||||
|
||||
### Resource Comparison
|
||||
|
||||
| Command | Workers Started | RAM Usage |
|
||||
@@ -171,7 +223,7 @@ FuzzForge requires `volumes/env/.env` to start. This file contains API keys and
|
||||
|
||||
```bash
|
||||
# Copy the example file
|
||||
cp volumes/env/.env.example volumes/env/.env
|
||||
cp volumes/env/.env.template volumes/env/.env
|
||||
|
||||
# Edit to add your API keys (if using AI features)
|
||||
nano volumes/env/.env
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
---
|
||||
title: "Hot-Swap LiteLLM Models"
|
||||
description: "Register OpenAI and Anthropic models with the bundled LiteLLM proxy and switch them on the task agent without downtime."
|
||||
---
|
||||
|
||||
LiteLLM sits between the task agent and upstream providers, so every model change
|
||||
is just an API call. This guide walks through registering OpenAI and Anthropic
|
||||
models, updating the virtual key, and exercising the A2A hot-swap flow.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `docker compose up llm-proxy llm-proxy-db task-agent`
|
||||
- Provider secrets in `volumes/env/.env`:
|
||||
- `LITELLM_OPENAI_API_KEY`
|
||||
- `LITELLM_ANTHROPIC_API_KEY`
|
||||
- Master key (`LITELLM_MASTER_KEY`) and task-agent virtual key (auto-generated
|
||||
during bootstrap)
|
||||
|
||||
> UI access uses `UI_USERNAME` / `UI_PASSWORD` (defaults: `fuzzforge` /
|
||||
> `fuzzforge123`). Change them by exporting new values before running compose.
|
||||
|
||||
## Register Provider Models
|
||||
|
||||
Use the admin API to register the models the proxy should expose. The snippet
|
||||
below creates aliases for OpenAI `gpt-5`, `gpt-5-mini`, and Anthropic
|
||||
`claude-sonnet-4-5`.
|
||||
|
||||
```bash
|
||||
MASTER_KEY=$(awk -F= '$1=="LITELLM_MASTER_KEY"{print $2}' volumes/env/.env)
|
||||
export OPENAI_API_KEY=$(awk -F= '$1=="OPENAI_API_KEY"{print $2}' volumes/env/.env)
|
||||
python - <<'PY'
|
||||
import os, requests
|
||||
master = os.environ['MASTER_KEY'].strip()
|
||||
base = 'http://localhost:10999'
|
||||
models = [
|
||||
{
|
||||
"model_name": "openai/gpt-5",
|
||||
"litellm_params": {
|
||||
"model": "gpt-5",
|
||||
"custom_llm_provider": "openai",
|
||||
"api_key": "os.environ/LITELLM_OPENAI_API_KEY"
|
||||
},
|
||||
"model_info": {
|
||||
"provider": "openai",
|
||||
"description": "OpenAI GPT-5"
|
||||
}
|
||||
},
|
||||
{
|
||||
"model_name": "openai/gpt-5-mini",
|
||||
"litellm_params": {
|
||||
"model": "gpt-5-mini",
|
||||
"custom_llm_provider": "openai",
|
||||
"api_key": "os.environ/LITELLM_OPENAI_API_KEY"
|
||||
},
|
||||
"model_info": {
|
||||
"provider": "openai",
|
||||
"description": "OpenAI GPT-5 mini"
|
||||
}
|
||||
},
|
||||
{
|
||||
"model_name": "anthropic/claude-sonnet-4-5",
|
||||
"litellm_params": {
|
||||
"model": "claude-sonnet-4-5",
|
||||
"custom_llm_provider": "anthropic",
|
||||
"api_key": "os.environ/LITELLM_ANTHROPIC_API_KEY"
|
||||
},
|
||||
"model_info": {
|
||||
"provider": "anthropic",
|
||||
"description": "Anthropic Claude Sonnet 4.5"
|
||||
}
|
||||
}
|
||||
]
|
||||
for payload in models:
|
||||
resp = requests.post(
|
||||
f"{base}/model/new",
|
||||
headers={"Authorization": f"Bearer {master}", "Content-Type": "application/json"},
|
||||
json=payload,
|
||||
timeout=60,
|
||||
)
|
||||
if resp.status_code not in (200, 201, 409):
|
||||
raise SystemExit(f"Failed to register {payload['model_name']}: {resp.status_code} {resp.text}")
|
||||
print(payload['model_name'], '=>', resp.status_code)
|
||||
PY
|
||||
```
|
||||
|
||||
Each entry stores the upstream secret by reference (`os.environ/...`) so the
|
||||
raw API key never leaves the container environment.
|
||||
|
||||
## Relax Virtual Key Model Restrictions
|
||||
|
||||
Let the agent key call every model on the proxy:
|
||||
|
||||
```bash
|
||||
MASTER_KEY=$(awk -F= '$1=="LITELLM_MASTER_KEY"{print $2}' volumes/env/.env)
|
||||
VK=$(awk -F= '$1=="OPENAI_API_KEY"{print $2}' volumes/env/.env)
|
||||
python - <<'PY'
|
||||
import os, requests, json
|
||||
resp = requests.post(
|
||||
'http://localhost:10999/key/update',
|
||||
headers={
|
||||
'Authorization': f"Bearer {os.environ['MASTER_KEY'].strip()}",
|
||||
'Content-Type': 'application/json'
|
||||
},
|
||||
json={'key': os.environ['VK'].strip(), 'models': []},
|
||||
timeout=60,
|
||||
)
|
||||
print(json.dumps(resp.json(), indent=2))
|
||||
PY
|
||||
```
|
||||
|
||||
Restart the task agent so it sees the refreshed key:
|
||||
|
||||
```bash
|
||||
docker compose restart task-agent
|
||||
```
|
||||
|
||||
## Hot-Swap With The A2A Helper
|
||||
|
||||
Switch models without restarting the service:
|
||||
|
||||
```bash
|
||||
# Ensure the CLI reads the latest virtual key
|
||||
export OPENAI_API_KEY=$(awk -F= '$1=="OPENAI_API_KEY"{print $2}' volumes/env/.env)
|
||||
|
||||
# OpenAI gpt-5 alias
|
||||
python ai/agents/task_agent/a2a_hot_swap.py \
|
||||
--url http://localhost:10900/a2a/litellm_agent \
|
||||
--model openai gpt-5 \
|
||||
--context switch-demo
|
||||
|
||||
# Confirm the response comes from the new model
|
||||
python ai/agents/task_agent/a2a_hot_swap.py \
|
||||
--url http://localhost:10900/a2a/litellm_agent \
|
||||
--message "Which model am I using?" \
|
||||
--context switch-demo
|
||||
|
||||
# Swap to gpt-5-mini
|
||||
python ai/agents/task_agent/a2a_hot_swap.py --url http://localhost:10900/a2a/litellm_agent --model openai gpt-5-mini --context switch-demo
|
||||
|
||||
# Swap to Anthropic Claude Sonnet 4.5
|
||||
python ai/agents/task_agent/a2a_hot_swap.py --url http://localhost:10900/a2a/litellm_agent --model anthropic claude-sonnet-4-5 --context switch-demo
|
||||
```
|
||||
|
||||
> Each invocation reuses the same conversation context (`switch-demo`) so you
|
||||
> can confirm the active provider by asking follow-up questions.
|
||||
|
||||
## Resetting The Proxy (Optional)
|
||||
|
||||
To wipe the LiteLLM state and rerun bootstrap:
|
||||
|
||||
```bash
|
||||
docker compose down llm-proxy llm-proxy-db llm-proxy-bootstrap
|
||||
|
||||
docker volume rm fuzzforge_litellm_proxy_data fuzzforge_litellm_proxy_db
|
||||
|
||||
docker compose up -d llm-proxy-db llm-proxy
|
||||
```
|
||||
|
||||
After the proxy is healthy, rerun the registration script and key update. The
|
||||
bootstrap container mirrors secrets into `.env.litellm` and reissues the task
|
||||
agent key automatically.
|
||||
|
||||
## How The Pieces Fit Together
|
||||
|
||||
1. **LiteLLM Proxy** exposes OpenAI-compatible routes and stores provider
|
||||
metadata in Postgres.
|
||||
2. **Bootstrap Container** waits for `/health/liveliness`, mirrors secrets into
|
||||
`.env.litellm`, registers any models you script, and keeps the virtual key in
|
||||
sync with the discovered model list.
|
||||
3. **Task Agent** calls the proxy via `FF_LLM_PROXY_BASE_URL`. The hot-swap tool
|
||||
updates the agent’s runtime configuration, so switching providers is just a
|
||||
control message.
|
||||
4. **Virtual Keys** carry quotas and allowed models. Setting the `models` array
|
||||
to `[]` lets the key use anything registered on the proxy.
|
||||
|
||||
Keep the master key and generated virtual keys somewhere safe—they grant full
|
||||
admin and agent access respectively. When you add a new provider (e.g., Ollama)
|
||||
just register the model via `/model/new`, update the key if needed, and repeat
|
||||
the hot-swap steps.
|
||||
@@ -0,0 +1,194 @@
|
||||
---
|
||||
title: "Run the LLM Proxy"
|
||||
description: "Run the LiteLLM gateway that ships with FuzzForge and connect it to the task agent."
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
FuzzForge routes every LLM request through a LiteLLM proxy so that usage can be
|
||||
metered, priced, and rate limited per user. Docker Compose starts the proxy in a
|
||||
hardened container, while a bootstrap job seeds upstream provider secrets and
|
||||
issues a virtual key for the task agent automatically.
|
||||
|
||||
LiteLLM exposes the OpenAI-compatible APIs (`/v1/*`) plus a rich admin UI. All
|
||||
traffic stays on your network and upstream credentials never leave the proxy
|
||||
container.
|
||||
|
||||
## Before You Start
|
||||
|
||||
1. Copy `volumes/env/.env.template` to `volumes/env/.env` and set the basics:
|
||||
- `LITELLM_MASTER_KEY` — admin token used to manage the proxy
|
||||
- `LITELLM_SALT_KEY` — random string used to encrypt provider credentials
|
||||
- Provider secrets under `LITELLM_<PROVIDER>_API_KEY` (for example
|
||||
`LITELLM_OPENAI_API_KEY`)
|
||||
- Leave `OPENAI_API_KEY=sk-proxy-default`; the bootstrap job replaces it with a
|
||||
LiteLLM-issued virtual key
|
||||
2. When running tools outside Docker, change `FF_LLM_PROXY_BASE_URL` to the
|
||||
published host port (`http://localhost:10999`). Inside Docker the default
|
||||
value `http://llm-proxy:4000` already resolves to the container.
|
||||
|
||||
## Start the Proxy
|
||||
|
||||
```bash
|
||||
docker compose up llm-proxy
|
||||
```
|
||||
|
||||
The service publishes two things:
|
||||
|
||||
- HTTP API + admin UI on `http://localhost:10999`
|
||||
- Persistent SQLite state inside the named volume
|
||||
`fuzzforge_litellm_proxy_data`
|
||||
|
||||
The UI login uses the `UI_USERNAME` / `UI_PASSWORD` pair (defaults to
|
||||
`fuzzforge` / `fuzzforge123`). To change them, set the environment variables
|
||||
before you run `docker compose up`:
|
||||
|
||||
```bash
|
||||
export UI_USERNAME=myadmin
|
||||
export UI_PASSWORD=super-secret
|
||||
docker compose up llm-proxy
|
||||
```
|
||||
|
||||
You can also edit the values directly in `docker-compose.yml` if you prefer to
|
||||
check them into a different secrets manager.
|
||||
|
||||
Proxy-wide settings now live in `volumes/litellm/proxy_config.yaml`. By
|
||||
default it enables `store_model_in_db` and `store_prompts_in_spend_logs`, which
|
||||
lets the UI display request/response payloads for new calls. Update this file
|
||||
if you need additional LiteLLM options and restart the `llm-proxy` container.
|
||||
|
||||
LiteLLM's health endpoint lives at `/health/liveliness`. You can verify it from
|
||||
another terminal:
|
||||
|
||||
```bash
|
||||
curl http://localhost:10999/health/liveliness
|
||||
```
|
||||
|
||||
## What the Bootstrapper Does
|
||||
|
||||
During startup the `llm-proxy-bootstrap` container performs three actions:
|
||||
|
||||
1. **Wait for the proxy** — Blocks until `/health/liveliness` becomes healthy.
|
||||
2. **Mirror provider secrets** — Reads `volumes/env/.env` and writes any
|
||||
`LITELLM_*_API_KEY` values into `volumes/env/.env.litellm`. The file is
|
||||
created automatically on first boot; if you delete it, bootstrap will
|
||||
recreate it and the proxy continues to read secrets from `.env`.
|
||||
3. **Issue the default virtual key** — Calls `/key/generate` with the master key
|
||||
and persists the generated token back into `volumes/env/.env` (replacing the
|
||||
`sk-proxy-default` placeholder). The key is scoped to
|
||||
`LITELLM_DEFAULT_MODELS` when that variable is set; otherwise it uses the
|
||||
model from `LITELLM_MODEL`.
|
||||
|
||||
The sequence is idempotent. Existing provider secrets and virtual keys are
|
||||
reused on subsequent runs, and the allowed-model list is refreshed via
|
||||
`/key/update` if you change the defaults.
|
||||
|
||||
## Managing Virtual Keys
|
||||
|
||||
LiteLLM keys act as per-user credentials. The default key, named
|
||||
`task-agent default`, is created automatically for the task agent. You can issue
|
||||
more keys for teammates or CI jobs with the same management API:
|
||||
|
||||
```bash
|
||||
curl http://localhost:10999/key/generate \
|
||||
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"key_alias": "demo-user",
|
||||
"user_id": "demo",
|
||||
"models": ["openai/gpt-4o-mini"],
|
||||
"duration": "30d",
|
||||
"max_budget": 50,
|
||||
"metadata": {"team": "sandbox"}
|
||||
}'
|
||||
```
|
||||
|
||||
Use `/key/update` to adjust budgets or the allowed-model list on existing keys:
|
||||
|
||||
```bash
|
||||
curl http://localhost:10999/key/update \
|
||||
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"key": "sk-...",
|
||||
"models": ["openai/*", "anthropic/*"],
|
||||
"max_budget": 100
|
||||
}'
|
||||
```
|
||||
|
||||
The admin UI (navigate to `http://localhost:10999/ui`) provides equivalent
|
||||
controls for creating keys, routing models, auditing spend, and exporting logs.
|
||||
|
||||
## Wiring the Task Agent
|
||||
|
||||
The task agent already expects to talk to the proxy. Confirm these values in
|
||||
`volumes/env/.env` before launching the stack:
|
||||
|
||||
```bash
|
||||
FF_LLM_PROXY_BASE_URL=http://llm-proxy:4000 # or http://localhost:10999 when outside Docker
|
||||
OPENAI_API_KEY=<virtual key created by bootstrap>
|
||||
LITELLM_MODEL=openai/gpt-5
|
||||
LITELLM_PROVIDER=openai
|
||||
```
|
||||
|
||||
Restart the agent container after changing environment variables so the process
|
||||
picks up the updates.
|
||||
|
||||
To validate the integration end to end, call the proxy directly:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:10999/v1/chat/completions \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "openai/gpt-4o-mini",
|
||||
"messages": [{"role": "user", "content": "Proxy health check"}]
|
||||
}'
|
||||
```
|
||||
|
||||
A JSON response indicates the proxy can reach your upstream provider using the
|
||||
mirrored secrets.
|
||||
|
||||
## Local Runtimes (Ollama, etc.)
|
||||
|
||||
LiteLLM supports non-hosted providers as well. To route requests to a local
|
||||
runtime such as Ollama:
|
||||
|
||||
1. Set the appropriate provider key in the env file
|
||||
(for Ollama, point LiteLLM at `OLLAMA_API_BASE` inside the container).
|
||||
2. Add the passthrough model either from the UI (**Models → Add Model**) or
|
||||
by calling `/model/new` with the master key.
|
||||
3. Update `LITELLM_DEFAULT_MODELS` (and regenerate the virtual key if you want
|
||||
the default key to include it).
|
||||
|
||||
The task agent keeps using the same OpenAI-compatible surface while LiteLLM
|
||||
handles the translation to your runtime.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Explore [LiteLLM's documentation](https://docs.litellm.ai/docs/simple_proxy)
|
||||
for advanced routing, cost controls, and observability hooks.
|
||||
- Configure Slack/Prometheus integrations from the UI to monitor usage.
|
||||
- Rotate the master key periodically and store it in your secrets manager, as it
|
||||
grants full admin access to the proxy.
|
||||
|
||||
## Observability
|
||||
|
||||
LiteLLM ships with OpenTelemetry hooks for traces and metrics. This repository
|
||||
already includes an OTLP collector (`otel-collector` service) and mounts a
|
||||
default configuration that forwards traces to standard output. To wire it up:
|
||||
|
||||
1. Edit `volumes/otel/collector-config.yaml` if you want to forward to Jaeger,
|
||||
Datadog, etc. The initial config uses the logging exporter so you can see
|
||||
spans immediately via `docker compose logs -f otel-collector`.
|
||||
2. Customize `volumes/litellm/proxy_config.yaml` if you need additional
|
||||
callbacks; `general_settings.otel: true` and `litellm_settings.callbacks:
|
||||
["otel"]` are already present so no extra code changes are required.
|
||||
3. (Optional) Override `OTEL_EXPORTER_OTLP_*` environment variables in
|
||||
`docker-compose.yml` or your shell to point at a remote collector.
|
||||
|
||||
After updating the configs, run `docker compose up -d otel-collector llm-proxy`
|
||||
and generate a request (for example, trigger `ff workflow run llm_analysis`).
|
||||
New traces will show up in the collector logs or whichever backend you
|
||||
configured. See the official LiteLLM guide for advanced exporter options:
|
||||
https://docs.litellm.ai/docs/observability/opentelemetry_integration.
|
||||
@@ -33,7 +33,7 @@ The required `volumes/env/.env` file is missing. Docker Compose needs this file
|
||||
**How to fix:**
|
||||
```bash
|
||||
# Create the environment file from the template
|
||||
cp volumes/env/.env.example volumes/env/.env
|
||||
cp volumes/env/.env.template volumes/env/.env
|
||||
|
||||
# Restart Docker Compose
|
||||
docker compose -f docker-compose.yml down
|
||||
@@ -106,6 +106,46 @@ File upload to MinIO failed or worker can't download target.
|
||||
```
|
||||
- Reduce the number of concurrent workflows if your system is resource-constrained.
|
||||
|
||||
### Workflow requires worker not running
|
||||
|
||||
**What's happening?**
|
||||
You see a warning message like:
|
||||
```
|
||||
⚠️ Could not check worker requirements: Cannot find docker-compose.yml.
|
||||
Ensure backend is running, run from FuzzForge directory, or set
|
||||
FUZZFORGE_ROOT environment variable.
|
||||
Continuing without worker management...
|
||||
```
|
||||
|
||||
Or the workflow fails to start because the required worker isn't running.
|
||||
|
||||
**How to fix:**
|
||||
Start the worker required for your workflow before running it:
|
||||
|
||||
| Workflow | Worker Required | Startup Command |
|
||||
|----------|----------------|-----------------|
|
||||
| `android_static_analysis` | worker-android | `docker compose up -d worker-android` |
|
||||
| `security_assessment` | worker-python | `docker compose up -d worker-python` |
|
||||
| `python_sast` | worker-python | `docker compose up -d worker-python` |
|
||||
| `llm_analysis` | worker-python | `docker compose up -d worker-python` |
|
||||
| `atheris_fuzzing` | worker-python | `docker compose up -d worker-python` |
|
||||
| `ossfuzz_campaign` | worker-ossfuzz | `docker compose up -d worker-ossfuzz` |
|
||||
| `cargo_fuzzing` | worker-rust | `docker compose up -d worker-rust` |
|
||||
| `llm_secret_detection` | worker-secrets | `docker compose up -d worker-secrets` |
|
||||
| `trufflehog_detection` | worker-secrets | `docker compose up -d worker-secrets` |
|
||||
| `gitleaks_detection` | worker-secrets | `docker compose up -d worker-secrets` |
|
||||
|
||||
**Check worker status:**
|
||||
```bash
|
||||
# Check if a specific worker is running
|
||||
docker compose ps worker-android
|
||||
|
||||
# Check all workers
|
||||
docker compose ps | grep worker
|
||||
```
|
||||
|
||||
**Note:** Workers don't auto-start by default to save system resources. For more details on worker management, see the [Docker Setup guide](docker-setup.md#worker-management).
|
||||
|
||||
---
|
||||
|
||||
## Service Connectivity Issues
|
||||
|
||||
@@ -0,0 +1,616 @@
|
||||
# FuzzForge CLI Reference
|
||||
|
||||
Complete reference for the FuzzForge CLI (`ff` command). Use this as your quick lookup for all commands, options, and examples.
|
||||
|
||||
---
|
||||
|
||||
## Global Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--help`, `-h` | Show help message |
|
||||
| `--version`, `-v` | Show version information |
|
||||
|
||||
---
|
||||
|
||||
## Core Commands
|
||||
|
||||
### `ff init`
|
||||
|
||||
Initialize a new FuzzForge project in the current directory.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff init [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--name`, `-n` — Project name (defaults to current directory name)
|
||||
- `--api-url`, `-u` — FuzzForge API URL (defaults to http://localhost:8000)
|
||||
- `--force`, `-f` — Force initialization even if project already exists
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff init # Initialize with defaults
|
||||
ff init --name my-project # Set custom project name
|
||||
ff init --api-url http://prod:8000 # Use custom API URL
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `ff status`
|
||||
|
||||
Show project and latest execution status.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff status
|
||||
```
|
||||
|
||||
**Example Output:**
|
||||
```
|
||||
📊 Project Status
|
||||
Project: my-security-project
|
||||
API URL: http://localhost:8000
|
||||
|
||||
Latest Execution:
|
||||
Run ID: security_scan-a1b2c3
|
||||
Workflow: security_assessment
|
||||
Status: COMPLETED
|
||||
Started: 2 hours ago
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `ff config`
|
||||
|
||||
Manage project configuration.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff config # Show all config
|
||||
ff config <key> # Get specific value
|
||||
ff config <key> <value> # Set value
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff config # Display all settings
|
||||
ff config api_url # Get API URL
|
||||
ff config api_url http://prod:8000 # Set API URL
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `ff clean`
|
||||
|
||||
Clean old execution data and findings.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff clean [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--days`, `-d` — Remove data older than this many days (default: 90)
|
||||
- `--dry-run` — Show what would be deleted without deleting
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff clean # Clean data older than 90 days
|
||||
ff clean --days 30 # Clean data older than 30 days
|
||||
ff clean --dry-run # Preview what would be deleted
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Commands
|
||||
|
||||
### `ff workflows`
|
||||
|
||||
Browse and list available workflows.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff workflows [COMMAND]
|
||||
```
|
||||
|
||||
**Subcommands:**
|
||||
- `list` — List all available workflows
|
||||
- `info <workflow>` — Show detailed workflow information
|
||||
- `params <workflow>` — Show workflow parameters
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff workflows list # List all workflows
|
||||
ff workflows info python_sast # Show workflow details
|
||||
ff workflows params python_sast # Show parameters
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `ff workflow`
|
||||
|
||||
Execute and manage individual workflows.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff workflow <COMMAND>
|
||||
```
|
||||
|
||||
**Subcommands:**
|
||||
|
||||
#### `ff workflow run`
|
||||
|
||||
Execute a security testing workflow.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff workflow run <workflow> <target> [params...] [OPTIONS]
|
||||
```
|
||||
|
||||
**Arguments:**
|
||||
- `<workflow>` — Workflow name
|
||||
- `<target>` — Target path to analyze
|
||||
- `[params...]` — Parameters as `key=value` pairs
|
||||
|
||||
**Options:**
|
||||
- `--param-file`, `-f` — JSON file containing workflow parameters
|
||||
- `--timeout`, `-t` — Execution timeout in seconds
|
||||
- `--interactive` / `--no-interactive`, `-i` / `-n` — Interactive parameter input (default: interactive)
|
||||
- `--wait`, `-w` — Wait for execution to complete
|
||||
- `--live`, `-l` — Start live monitoring after execution
|
||||
- `--auto-start` / `--no-auto-start` — Automatically start required worker
|
||||
- `--auto-stop` / `--no-auto-stop` — Automatically stop worker after completion
|
||||
- `--fail-on` — Fail build if findings match SARIF level (error, warning, note, info, all, none)
|
||||
- `--export-sarif` — Export SARIF results to file after completion
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Basic workflow execution
|
||||
ff workflow run python_sast ./project
|
||||
|
||||
# With parameters
|
||||
ff workflow run python_sast ./project check_secrets=true
|
||||
|
||||
# CI/CD integration - fail on errors
|
||||
ff workflow run python_sast ./project --wait --no-interactive \
|
||||
--fail-on error --export-sarif results.sarif
|
||||
|
||||
# With parameter file
|
||||
ff workflow run python_sast ./project --param-file config.json
|
||||
|
||||
# Live monitoring for fuzzing
|
||||
ff workflow run atheris_fuzzing ./project --live
|
||||
```
|
||||
|
||||
#### `ff workflow status`
|
||||
|
||||
Check status of latest or specific workflow execution.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff workflow status [run_id]
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff workflow status # Show latest execution status
|
||||
ff workflow status python_sast-abc123 # Show specific execution
|
||||
```
|
||||
|
||||
#### `ff workflow history`
|
||||
|
||||
Show execution history.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff workflow history [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--limit`, `-l` — Number of executions to show (default: 10)
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
ff workflow history --limit 20
|
||||
```
|
||||
|
||||
#### `ff workflow retry`
|
||||
|
||||
Retry a failed workflow execution.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff workflow retry <run_id>
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
ff workflow retry python_sast-abc123
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Finding Commands
|
||||
|
||||
### `ff findings`
|
||||
|
||||
Browse all findings across executions.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff findings [COMMAND]
|
||||
```
|
||||
|
||||
**Subcommands:**
|
||||
|
||||
#### `ff findings list`
|
||||
|
||||
List findings from a specific run.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff findings list [run_id] [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--format` — Output format: table, json, sarif (default: table)
|
||||
- `--save` — Save findings to file
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff findings list # Show latest findings
|
||||
ff findings list python_sast-abc123 # Show specific run
|
||||
ff findings list --format json # JSON output
|
||||
ff findings list --format sarif --save # Export SARIF
|
||||
```
|
||||
|
||||
#### `ff findings export`
|
||||
|
||||
Export findings to various formats.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff findings export <run_id> [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--format` — Output format: json, sarif, csv
|
||||
- `--output`, `-o` — Output file path
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
ff findings export python_sast-abc123 --format sarif --output results.sarif
|
||||
```
|
||||
|
||||
#### `ff findings history`
|
||||
|
||||
Show finding history across multiple runs.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff findings history [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--limit`, `-l` — Number of runs to include (default: 10)
|
||||
|
||||
---
|
||||
|
||||
### `ff finding`
|
||||
|
||||
View and analyze individual findings.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff finding [id] # Show latest or specific finding
|
||||
ff finding show <run_id> --rule <rule> # Show specific finding detail
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff finding # Show latest finding
|
||||
ff finding python_sast-abc123 # Show specific run findings
|
||||
ff finding show python_sast-abc123 --rule f2cf5e3e # Show specific finding
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Worker Management Commands
|
||||
|
||||
### `ff worker`
|
||||
|
||||
Manage Temporal workers for workflow execution.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff worker <COMMAND>
|
||||
```
|
||||
|
||||
**Subcommands:**
|
||||
|
||||
#### `ff worker list`
|
||||
|
||||
List FuzzForge workers and their status.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff worker list [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--all`, `-a` — Show all workers (including stopped)
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff worker list # Show running workers
|
||||
ff worker list --all # Show all workers
|
||||
```
|
||||
|
||||
**Example Output:**
|
||||
```
|
||||
FuzzForge Workers
|
||||
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
|
||||
┃ Worker ┃ Status ┃ Uptime ┃
|
||||
┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
|
||||
│ android │ ● Running │ 5 minutes ago │
|
||||
│ python │ ● Running │ 10 minutes ago │
|
||||
└─────────┴───────────┴────────────────┘
|
||||
|
||||
✅ 2 worker(s) running
|
||||
```
|
||||
|
||||
#### `ff worker start`
|
||||
|
||||
Start a specific worker.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff worker start <name> [OPTIONS]
|
||||
```
|
||||
|
||||
**Arguments:**
|
||||
- `<name>` — Worker name (e.g., python, android, rust, secrets)
|
||||
|
||||
**Options:**
|
||||
- `--build` — Rebuild worker image before starting
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff worker start python # Start Python worker
|
||||
ff worker start android --build # Rebuild and start Android worker
|
||||
```
|
||||
|
||||
**Available Workers:**
|
||||
- `python` — Python security analysis and fuzzing
|
||||
- `android` — Android APK analysis
|
||||
- `rust` — Rust fuzzing and analysis
|
||||
- `secrets` — Secret detection workflows
|
||||
- `ossfuzz` — OSS-Fuzz integration
|
||||
|
||||
#### `ff worker stop`
|
||||
|
||||
Stop all running FuzzForge workers.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff worker stop [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--all` — Stop all workers (default behavior, flag for clarity)
|
||||
|
||||
**Example:**
|
||||
```bash
|
||||
ff worker stop
|
||||
```
|
||||
|
||||
**Note:** This command stops only worker containers, leaving core services (backend, temporal, minio) running.
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Commands
|
||||
|
||||
### `ff monitor`
|
||||
|
||||
Real-time monitoring for running workflows.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff monitor [COMMAND]
|
||||
```
|
||||
|
||||
**Subcommands:**
|
||||
- `live <run_id>` — Live monitoring for a specific execution
|
||||
- `stats <run_id>` — Show statistics for fuzzing workflows
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff monitor live atheris-abc123 # Monitor fuzzing campaign
|
||||
ff monitor stats atheris-abc123 # Show fuzzing statistics
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## AI Integration Commands
|
||||
|
||||
### `ff ai`
|
||||
|
||||
AI-powered analysis and assistance.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff ai [COMMAND]
|
||||
```
|
||||
|
||||
**Subcommands:**
|
||||
- `analyze <run_id>` — Analyze findings with AI
|
||||
- `explain <finding_id>` — Get AI explanation of a finding
|
||||
- `remediate <finding_id>` — Get remediation suggestions
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff ai analyze python_sast-abc123 # Analyze all findings
|
||||
ff ai explain python_sast-abc123:finding1 # Explain specific finding
|
||||
ff ai remediate python_sast-abc123:finding1 # Get fix suggestions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Knowledge Ingestion Commands
|
||||
|
||||
### `ff ingest`
|
||||
|
||||
Ingest knowledge into the AI knowledge base.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
ff ingest [COMMAND]
|
||||
```
|
||||
|
||||
**Subcommands:**
|
||||
- `file <path>` — Ingest a file
|
||||
- `directory <path>` — Ingest directory contents
|
||||
- `workflow <workflow_name>` — Ingest workflow documentation
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
ff ingest file ./docs/security.md # Ingest single file
|
||||
ff ingest directory ./docs # Ingest directory
|
||||
ff ingest workflow python_sast # Ingest workflow docs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Workflow Examples
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
```bash
|
||||
# Run security scan in CI, fail on errors
|
||||
ff workflow run python_sast . \
|
||||
--wait \
|
||||
--no-interactive \
|
||||
--fail-on error \
|
||||
--export-sarif results.sarif
|
||||
```
|
||||
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
# Quick security check
|
||||
ff workflow run python_sast ./my-code
|
||||
|
||||
# Check specific file types
|
||||
ff workflow run python_sast . file_extensions='[".py",".js"]'
|
||||
|
||||
# Interactive parameter configuration
|
||||
ff workflow run python_sast . --interactive
|
||||
```
|
||||
|
||||
### Fuzzing Workflows
|
||||
|
||||
```bash
|
||||
# Start fuzzing with live monitoring
|
||||
ff workflow run atheris_fuzzing ./project --live
|
||||
|
||||
# Long-running fuzzing campaign
|
||||
ff workflow run ossfuzz_campaign ./project \
|
||||
--auto-start \
|
||||
duration=3600 \
|
||||
--live
|
||||
```
|
||||
|
||||
### Worker Management
|
||||
|
||||
```bash
|
||||
# Check which workers are running
|
||||
ff worker list
|
||||
|
||||
# Start needed worker manually
|
||||
ff worker start python --build
|
||||
|
||||
# Stop all workers when done
|
||||
ff worker stop
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration Files
|
||||
|
||||
### Project Config (`.fuzzforge/config.json`)
|
||||
|
||||
```json
|
||||
{
|
||||
"project_name": "my-security-project",
|
||||
"api_url": "http://localhost:8000",
|
||||
"default_workflow": "python_sast",
|
||||
"auto_start_workers": true,
|
||||
"auto_stop_workers": false
|
||||
}
|
||||
```
|
||||
|
||||
### Parameter File Example
|
||||
|
||||
```json
|
||||
{
|
||||
"check_secrets": true,
|
||||
"file_extensions": [".py", ".js", ".go"],
|
||||
"severity_threshold": "medium",
|
||||
"exclude_patterns": ["**/test/**", "**/vendor/**"]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exit Codes
|
||||
|
||||
| Code | Meaning |
|
||||
|------|---------|
|
||||
| 0 | Success |
|
||||
| 1 | General error |
|
||||
| 2 | Findings matched `--fail-on` criteria |
|
||||
| 3 | Worker startup failed |
|
||||
| 4 | Workflow execution failed |
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `FUZZFORGE_API_URL` | Backend API URL | http://localhost:8000 |
|
||||
| `FUZZFORGE_ROOT` | FuzzForge installation directory | Auto-detected |
|
||||
| `FUZZFORGE_DEBUG` | Enable debug logging | false |
|
||||
|
||||
---
|
||||
|
||||
## Tips and Best Practices
|
||||
|
||||
1. **Use `--no-interactive` in CI/CD** — Prevents prompts that would hang automated pipelines
|
||||
2. **Use `--fail-on` for quality gates** — Fail builds based on finding severity
|
||||
3. **Export SARIF for tool integration** — Most security tools support SARIF format
|
||||
4. **Let workflows auto-start workers** — More efficient than manually managing workers
|
||||
5. **Use `--wait` with `--export-sarif`** — Ensures results are available before export
|
||||
6. **Check `ff worker list` regularly** — Helps manage system resources
|
||||
7. **Use parameter files for complex configs** — Easier to version control and reuse
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Docker Setup](../how-to/docker-setup.md) — Worker management and Docker configuration
|
||||
- [Getting Started](../tutorial/getting-started.md) — Complete setup guide
|
||||
- [Workflow Guide](../how-to/create-workflow.md) — Detailed workflow documentation
|
||||
- [CI/CD Integration](../how-to/cicd-integration.md) — CI/CD setup examples
|
||||
|
||||
---
|
||||
|
||||
**Need Help?**
|
||||
|
||||
```bash
|
||||
ff --help # General help
|
||||
ff workflow run --help # Command-specific help
|
||||
ff worker --help # Worker management help
|
||||
```
|
||||
@@ -28,7 +28,7 @@ cd fuzzforge_ai
|
||||
Create the environment configuration file:
|
||||
|
||||
```bash
|
||||
cp volumes/env/.env.example volumes/env/.env
|
||||
cp volumes/env/.env.template volumes/env/.env
|
||||
```
|
||||
|
||||
This file is required for FuzzForge to start. You can leave it with default values if you're only using basic workflows.
|
||||
@@ -89,9 +89,26 @@ curl http://localhost:8000/health
|
||||
# Should return: {"status":"healthy"}
|
||||
```
|
||||
|
||||
### Start the Python Worker
|
||||
### Start Workers for Your Workflows
|
||||
|
||||
Workers don't auto-start by default (saves RAM). Start the Python worker for your first workflow:
|
||||
Workers don't auto-start by default (saves RAM). You need to start the worker required for the workflow you want to run.
|
||||
|
||||
**Workflow-to-Worker Mapping:**
|
||||
|
||||
| Workflow | Worker Required | Startup Command |
|
||||
|----------|----------------|-----------------|
|
||||
| `security_assessment` | worker-python | `docker compose up -d worker-python` |
|
||||
| `python_sast` | worker-python | `docker compose up -d worker-python` |
|
||||
| `llm_analysis` | worker-python | `docker compose up -d worker-python` |
|
||||
| `atheris_fuzzing` | worker-python | `docker compose up -d worker-python` |
|
||||
| `android_static_analysis` | worker-android | `docker compose up -d worker-android` |
|
||||
| `cargo_fuzzing` | worker-rust | `docker compose up -d worker-rust` |
|
||||
| `ossfuzz_campaign` | worker-ossfuzz | `docker compose up -d worker-ossfuzz` |
|
||||
| `llm_secret_detection` | worker-secrets | `docker compose up -d worker-secrets` |
|
||||
| `trufflehog_detection` | worker-secrets | `docker compose up -d worker-secrets` |
|
||||
| `gitleaks_detection` | worker-secrets | `docker compose up -d worker-secrets` |
|
||||
|
||||
**For your first workflow (security_assessment), start the Python worker:**
|
||||
|
||||
```bash
|
||||
# Start the Python worker
|
||||
@@ -102,7 +119,20 @@ docker compose ps worker-python
|
||||
# Should show: Up (healthy)
|
||||
```
|
||||
|
||||
**Note:** Workers use Docker Compose profiles and only start when needed. For your first workflow run, it's safer to start the worker manually. Later, the CLI can auto-start workers on demand.
|
||||
**For other workflows, start the appropriate worker:**
|
||||
|
||||
```bash
|
||||
# Example: For Android analysis
|
||||
docker compose up -d worker-android
|
||||
|
||||
# Example: For Rust fuzzing
|
||||
docker compose up -d worker-rust
|
||||
|
||||
# Check all running workers
|
||||
docker compose ps | grep worker
|
||||
```
|
||||
|
||||
**Note:** Workers use Docker Compose profiles and only start when needed. For your first workflow run, it's safer to start the worker manually. Later, the CLI can auto-start workers on demand. If you see a warning about worker requirements, ensure you've started the correct worker for your workflow.
|
||||
|
||||
## Step 4: Install the CLI (Optional but Recommended)
|
||||
|
||||
|
||||
@@ -100,7 +100,7 @@ const config: Config = {
|
||||
label: "AI",
|
||||
},
|
||||
{
|
||||
href: "https://github.com/FuzzingLabs/fuzzforge_alpha",
|
||||
href: "https://github.com/FuzzingLabs/fuzzforge_ai",
|
||||
label: "GitHub",
|
||||
position: "right",
|
||||
},
|
||||
@@ -160,7 +160,7 @@ const config: Config = {
|
||||
},
|
||||
{
|
||||
label: "GitHub",
|
||||
href: "https://github.com/FuzzingLabs/fuzzforge_alpha",
|
||||
href: "https://github.com/FuzzingLabs/fuzzforge_ai",
|
||||
},
|
||||
],
|
||||
},
|
||||
|
||||
+1
-1
@@ -89,7 +89,7 @@ Technical reference materials and specifications.
|
||||
Before starting FuzzForge, you **must** create the environment configuration file:
|
||||
|
||||
```bash
|
||||
cp volumes/env/.env.example volumes/env/.env
|
||||
cp volumes/env/.env.template volumes/env/.env
|
||||
```
|
||||
|
||||
Docker Compose will fail without this file. You can leave it with default values if you're only using basic workflows (no AI features).
|
||||
|
||||
Reference in New Issue
Block a user