Files
fuzzforge_ai/setup.py
tduhamel42 60ca088ecf CI/CD Integration with Ephemeral Deployment Model (#14)
* feat: Complete migration from Prefect to Temporal

BREAKING CHANGE: Replaces Prefect workflow orchestration with Temporal

## Major Changes
- Replace Prefect with Temporal for workflow orchestration
- Implement vertical worker architecture (rust, android)
- Replace Docker registry with MinIO for unified storage
- Refactor activities to be co-located with workflows
- Update all API endpoints for Temporal compatibility

## Infrastructure
- New: docker-compose.temporal.yaml (Temporal + MinIO + workers)
- New: workers/ directory with rust and android vertical workers
- New: backend/src/temporal/ (manager, discovery)
- New: backend/src/storage/ (S3-cached storage with MinIO)
- New: backend/toolbox/common/ (shared storage activities)
- Deleted: docker-compose.yaml (old Prefect setup)
- Deleted: backend/src/core/prefect_manager.py
- Deleted: backend/src/services/prefect_stats_monitor.py
- Deleted: Docker registry and insecure-registries requirement

## Workflows
- Migrated: security_assessment workflow to Temporal
- New: rust_test workflow (example/test workflow)
- Deleted: secret_detection_scan (Prefect-based, to be reimplemented)
- Activities now co-located with workflows for independent testing

## API Changes
- Updated: backend/src/api/workflows.py (Temporal submission)
- Updated: backend/src/api/runs.py (Temporal status/results)
- Updated: backend/src/main.py (727 lines, TemporalManager integration)
- Updated: All 16 MCP tools to use TemporalManager

## Testing
-  All services healthy (Temporal, PostgreSQL, MinIO, workers, backend)
-  All API endpoints functional
-  End-to-end workflow test passed (72 findings from vulnerable_app)
-  MinIO storage integration working (target upload/download, results)
-  Worker activity discovery working (6 activities registered)
-  Tarball extraction working
-  SARIF report generation working

## Documentation
- ARCHITECTURE.md: Complete Temporal architecture documentation
- QUICKSTART_TEMPORAL.md: Getting started guide
- MIGRATION_DECISION.md: Why we chose Temporal over Prefect
- IMPLEMENTATION_STATUS.md: Migration progress tracking
- workers/README.md: Worker development guide

## Dependencies
- Added: temporalio>=1.6.0
- Added: boto3>=1.34.0 (MinIO S3 client)
- Removed: prefect>=3.4.18

* feat: Add Python fuzzing vertical with Atheris integration

This commit implements a complete Python fuzzing workflow using Atheris:

## Python Worker (workers/python/)
- Dockerfile with Python 3.11, Atheris, and build tools
- Generic worker.py for dynamic workflow discovery
- requirements.txt with temporalio, boto3, atheris dependencies
- Added to docker-compose.temporal.yaml with dedicated cache volume

## AtherisFuzzer Module (backend/toolbox/modules/fuzzer/)
- Reusable module extending BaseModule
- Auto-discovers fuzz targets (fuzz_*.py, *_fuzz.py, fuzz_target.py)
- Recursive search to find targets in nested directories
- Dynamically loads TestOneInput() function
- Configurable max_iterations and timeout
- Real-time stats callback support for live monitoring
- Returns findings as ModuleFinding objects

## Atheris Fuzzing Workflow (backend/toolbox/workflows/atheris_fuzzing/)
- Temporal workflow for orchestrating fuzzing
- Downloads user code from MinIO
- Executes AtherisFuzzer module
- Uploads results to MinIO
- Cleans up cache after execution
- metadata.yaml with vertical: python for routing

## Test Project (test_projects/python_fuzz_waterfall/)
- Demonstrates stateful waterfall vulnerability
- main.py with check_secret() that leaks progress
- fuzz_target.py with Atheris TestOneInput() harness
- Complete README with usage instructions

## Backend Fixes
- Fixed parameter merging in REST API endpoints (workflows.py)
- Changed workflow parameter passing from positional args to kwargs (manager.py)
- Default parameters now properly merged with user parameters

## Testing
 Worker discovered AtherisFuzzingWorkflow
 Workflow executed end-to-end successfully
 Fuzz target auto-discovered in nested directories
 Atheris ran 100,000 iterations
 Results uploaded and cache cleaned

* chore: Complete Temporal migration with updated CLI/SDK/docs

This commit includes all remaining Temporal migration changes:

## CLI Updates (cli/)
- Updated workflow execution commands for Temporal
- Enhanced error handling and exceptions
- Updated dependencies in uv.lock

## SDK Updates (sdk/)
- Client methods updated for Temporal workflows
- Updated models for new workflow execution
- Updated dependencies in uv.lock

## Documentation Updates (docs/)
- Architecture documentation for Temporal
- Workflow concept documentation
- Resource management documentation (new)
- Debugging guide (new)
- Updated tutorials and how-to guides
- Troubleshooting updates

## README Updates
- Main README with Temporal instructions
- Backend README
- CLI README
- SDK README

## Other
- Updated IMPLEMENTATION_STATUS.md
- Removed old vulnerable_app.tar.gz

These changes complete the Temporal migration and ensure the
CLI/SDK work correctly with the new backend.

* fix: Use positional args instead of kwargs for Temporal workflows

The Temporal Python SDK's start_workflow() method doesn't accept
a 'kwargs' parameter. Workflows must receive parameters as positional
arguments via the 'args' parameter.

Changed from:
  args=workflow_args  # Positional arguments

This fixes the error:
  TypeError: Client.start_workflow() got an unexpected keyword argument 'kwargs'

Workflows now correctly receive parameters in order:
- security_assessment: [target_id, scanner_config, analyzer_config, reporter_config]
- atheris_fuzzing: [target_id, target_file, max_iterations, timeout_seconds]
- rust_test: [target_id, test_message]

* fix: Filter metadata-only parameters from workflow arguments

SecurityAssessmentWorkflow was receiving 7 arguments instead of 2-5.
The issue was that target_path and volume_mode from default_parameters
were being passed to the workflow, when they should only be used by
the system for configuration.

Now filters out metadata-only parameters (target_path, volume_mode)
before passing arguments to workflow execution.

* refactor: Remove Prefect leftovers and volume mounting legacy

Complete cleanup of Prefect migration artifacts:

Backend:
- Delete registry.py and workflow_discovery.py (Prefect-specific files)
- Remove Docker validation from setup.py (no longer needed)
- Remove ResourceLimits and VolumeMount models
- Remove target_path and volume_mode from WorkflowSubmission
- Remove supported_volume_modes from API and discovery
- Clean up metadata.yaml files (remove volume/path fields)
- Simplify parameter filtering in manager.py

SDK:
- Remove volume_mode parameter from client methods
- Remove ResourceLimits and VolumeMount models
- Remove Prefect error patterns from docker_logs.py
- Clean up WorkflowSubmission and WorkflowMetadata models

CLI:
- Remove Volume Modes display from workflow info

All removed features are Prefect-specific or Docker volume mounting
artifacts. Temporal workflows use MinIO storage exclusively.

* feat: Add comprehensive test suite and benchmark infrastructure

- Add 68 unit tests for fuzzer, scanner, and analyzer modules
- Implement pytest-based test infrastructure with fixtures
- Add 6 performance benchmarks with category-specific thresholds
- Configure GitHub Actions for automated testing and benchmarking
- Add test and benchmark documentation

Test coverage:
- AtherisFuzzer: 8 tests
- CargoFuzzer: 14 tests
- FileScanner: 22 tests
- SecurityAnalyzer: 24 tests

All tests passing (68/68)
All benchmarks passing (6/6)

* fix: Resolve all ruff linting violations across codebase

Fixed 27 ruff violations in 12 files:
- Removed unused imports (Depends, Dict, Any, Optional, etc.)
- Fixed undefined workflow_info variable in workflows.py
- Removed dead code with undefined variables in atheris_fuzzer.py
- Changed f-string to regular string where no placeholders used

All files now pass ruff checks for CI/CD compliance.

* fix: Configure CI for unit tests only

- Renamed docker-compose.temporal.yaml → docker-compose.yml for CI compatibility
- Commented out integration-tests job (no integration tests yet)
- Updated test-summary to only depend on lint and unit-tests

CI will now run successfully with 68 unit tests. Integration tests can be added later.

* feat: Add CI/CD integration with ephemeral deployment model

Implements comprehensive CI/CD support for FuzzForge with on-demand worker management:

**Worker Management (v0.7.0)**
- Add WorkerManager for automatic worker lifecycle control
- Auto-start workers from stopped state when workflows execute
- Auto-stop workers after workflow completion
- Health checks and startup timeout handling (90s default)

**CI/CD Features**
- `--fail-on` flag: Fail builds based on SARIF severity levels (error/warning/note/info)
- `--export-sarif` flag: Export findings in SARIF 2.1.0 format
- `--auto-start`/`--auto-stop` flags: Control worker lifecycle
- Exit code propagation: Returns 1 on blocking findings, 0 on success

**Exit Code Fix**
- Add `except typer.Exit: raise` handlers at 3 critical locations
- Move worker cleanup to finally block for guaranteed execution
- Exit codes now propagate correctly even when build fails

**CI Scripts & Examples**
- ci-start.sh: Start FuzzForge services with health checks
- ci-stop.sh: Clean shutdown with volume preservation option
- GitHub Actions workflow example (security-scan.yml)
- GitLab CI pipeline example (.gitlab-ci.example.yml)
- docker-compose.ci.yml: CI-optimized compose file with profiles

**OSS-Fuzz Integration**
- New ossfuzz_campaign workflow for running OSS-Fuzz projects
- OSS-Fuzz worker with Docker-in-Docker support
- Configurable campaign duration and project selection

**Documentation**
- Comprehensive CI/CD integration guide (docs/how-to/cicd-integration.md)
- Updated architecture docs with worker lifecycle details
- Updated workspace isolation documentation
- CLI README with worker management examples

**SDK Enhancements**
- Add get_workflow_worker_info() endpoint
- Worker vertical metadata in workflow responses

**Testing**
- All workflows tested: security_assessment, atheris_fuzzing, secret_detection, cargo_fuzzing
- All monitoring commands tested: stats, crashes, status, finding
- Full CI pipeline simulation verified
- Exit codes verified for success/failure scenarios

Ephemeral CI/CD model: ~3-4GB RAM, ~60-90s startup, runs entirely in CI containers.

* fix: Resolve ruff linting violations in CI/CD code

- Remove unused variables (run_id, defaults, result)
- Remove unused imports
- Fix f-string without placeholders

All CI/CD integration files now pass ruff checks.
2025-10-14 10:13:45 +02:00

313 lines
13 KiB
Python
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env python3
# Copyright (c) 2025 FuzzingLabs
#
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
# at the root of this repository for details.
#
# After the Change Date (four years from publication), this version of the
# Licensed Work will be made available under the Apache License, Version 2.0.
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
#
# Additional attribution and requirements are provided in the NOTICE file.
"""
FuzzForge Setup Script - One-command setup for development
This script automates the entire FuzzForge development setup process,
from checking prerequisites to running your first security scan.
"""
import os
import sys
import subprocess
import platform
import time
from pathlib import Path
from typing import List, Tuple
class Colors:
"""ANSI color codes for terminal output"""
GREEN = '\033[92m'
YELLOW = '\033[93m'
RED = '\033[91m'
BLUE = '\033[94m'
CYAN = '\033[96m'
WHITE = '\033[97m'
BOLD = '\033[1m'
END = '\033[0m'
class FuzzForgeSetup:
"""Automated FuzzForge development environment setup"""
def __init__(self):
self.system = platform.system().lower()
self.project_root = Path(__file__).parent
self.errors: List[str] = []
self.warnings: List[str] = []
def print_header(self):
"""Print welcome header"""
print(f"""{Colors.CYAN}{Colors.BOLD}
╔══════════════════════════════════════════╗
║ FuzzForge Setup Script ║
║ Automated Development Setup ║
╚══════════════════════════════════════════╝
{Colors.END}""")
print(f"{Colors.WHITE}Welcome to FuzzForge! This script will set up your development environment.{Colors.END}\n")
def run_command(self, command: str, description: str, critical: bool = True) -> Tuple[bool, str]:
"""Run a shell command and return success status and output"""
print(f"{Colors.YELLOW}🔄 {description}...{Colors.END}")
try:
result = subprocess.run(
command.split(),
capture_output=True,
text=True,
timeout=120 # 2 minute timeout
)
if result.returncode == 0:
print(f"{Colors.GREEN}{description} completed successfully{Colors.END}")
return True, result.stdout
else:
error_msg = f"{description} failed: {result.stderr.strip()}"
if critical:
self.errors.append(error_msg)
print(f"{Colors.RED}{error_msg}{Colors.END}")
else:
self.warnings.append(error_msg)
print(f"{Colors.YELLOW}⚠️ {error_msg}{Colors.END}")
return False, result.stderr
except subprocess.TimeoutExpired:
error_msg = f"{description} timed out"
if critical:
self.errors.append(error_msg)
else:
self.warnings.append(error_msg)
print(f"{Colors.RED}{error_msg}{Colors.END}")
return False, "Timeout"
except Exception as e:
error_msg = f"{description} failed with exception: {e}"
if critical:
self.errors.append(error_msg)
else:
self.warnings.append(error_msg)
print(f"{Colors.RED}💥 {error_msg}{Colors.END}")
return False, str(e)
def check_prerequisites(self) -> bool:
"""Check if required tools are installed"""
print(f"{Colors.BOLD}📋 Step 1: Checking Prerequisites{Colors.END}\n")
all_good = True
# Check Python version
python_version = f"{sys.version_info.major}.{sys.version_info.minor}"
if sys.version_info >= (3, 11):
print(f"{Colors.GREEN}✅ Python {python_version} (required: 3.11+){Colors.END}")
else:
print(f"{Colors.RED}❌ Python {python_version} (required: 3.11+){Colors.END}")
self.errors.append(f"Python version {python_version} is too old. Please install Python 3.11+")
all_good = False
# Check Docker daemon is running
docker_success, docker_output = self.run_command("docker ps", "Checking Docker", critical=False)
if not docker_success:
self.errors.append("Docker daemon is not running. Please start Docker Desktop")
all_good = False
# Check Docker Compose with actual compose file validation
if docker_success:
compose_success, _ = self.run_command("docker compose config --quiet", "Checking Docker Compose", critical=False)
if not compose_success:
self.errors.append("Docker Compose validation failed. Please ensure docker-compose.yaml is valid and Docker is running")
all_good = False
else:
print(f"{Colors.RED}❌ Checking Docker Compose failed: Docker daemon not running{Colors.END}")
self.errors.append("Docker Compose cannot be validated - Docker daemon not running")
all_good = False
# Check UV
uv_success, _ = self.run_command("uv --version", "Checking UV package manager", critical=False)
if not uv_success:
print(f"{Colors.YELLOW}📦 UV not found, installing UV...{Colors.END}")
if self.system == "darwin": # macOS
subprocess.run(["curl", "-LsSf", "https://astral.sh/uv/install.sh", "|", "sh"], shell=True)
else:
subprocess.run(["pip", "install", "uv"])
# Recheck UV
uv_success, _ = self.run_command("uv --version", "Re-checking UV installation", critical=False)
if not uv_success:
self.warnings.append("UV installation failed. You can install it manually later")
return all_good
def setup_docker_environment(self) -> bool:
"""Set up Docker environment"""
print(f"\n{Colors.BOLD}🐳 Step 2: Setting Up Docker Environment{Colors.END}\n")
# Check if Docker daemon is running
docker_running, _ = self.run_command("docker ps", "Checking Docker daemon", critical=False)
if not docker_running:
print(f"{Colors.YELLOW}⚠️ Docker daemon is not running. Please start Docker Desktop and try again.{Colors.END}")
return False
# Start FuzzForge services
os.chdir(self.project_root)
# Warning about first launch
print(f"{Colors.CYAN} First launch will take longer due to Docker image building (5-10 minutes).{Colors.END}")
print(f"{Colors.CYAN} Subsequent starts will be much faster!{Colors.END}\n")
# Build and start services
print(f"{Colors.YELLOW}🔨 Building and starting FuzzForge services (this may take a while)...{Colors.END}")
# Use longer timeout for Docker build (10 minutes)
try:
result = subprocess.run(
["docker", "compose", "up", "-d"],
capture_output=True,
text=True,
timeout=600, # 10 minute timeout for Docker build
cwd=self.project_root
)
if result.returncode == 0:
print(f"{Colors.GREEN}✅ Docker services started successfully{Colors.END}")
else:
self.errors.append(f"Docker services failed to start: {result.stderr.strip()}")
print(f"{Colors.RED}❌ Docker services failed to start: {result.stderr.strip()}{Colors.END}")
return False
except subprocess.TimeoutExpired:
self.errors.append("Docker build timed out after 10 minutes")
print(f"{Colors.RED}⏰ Docker build timed out after 10 minutes{Colors.END}")
return False
except Exception as e:
self.errors.append(f"Docker setup failed: {e}")
print(f"{Colors.RED}💥 Docker setup failed: {e}{Colors.END}")
return False
# Wait for services to be ready with extended timeout
print(f"{Colors.YELLOW}⏳ Waiting for services to be ready...{Colors.END}")
for i in range(120): # Wait up to 2 minutes for services to be ready
time.sleep(1)
health_success, _ = self.run_command("curl -s http://localhost:8000/health", "Health check", critical=False)
if health_success:
print(f"\n{Colors.GREEN}✅ FuzzForge API is ready at http://localhost:8000!{Colors.END}")
return True
if i % 10 == 0: # Print progress every 10 seconds
print(f"\n{Colors.CYAN} Still starting... ({i+1}s){Colors.END}")
else:
print(".", end="", flush=True)
print(f"\n{Colors.YELLOW}⚠️ Services may still be starting. Check status with 'docker compose logs'{Colors.END}")
print(f"{Colors.CYAN}💡 You can monitor progress with: docker compose logs -f{Colors.END}")
return True
def install_cli(self) -> bool:
"""Install FuzzForge CLI"""
print(f"\n{Colors.BOLD}💻 Step 3: Installing FuzzForge CLI (Final Step){Colors.END}\n")
cli_dir = self.project_root / "cli"
if not cli_dir.exists():
self.errors.append("CLI directory not found")
return False
# Install from root, pointing to the 'cli' directory
success, _ = self.run_command("uv tool install --python python3.12 .", "Installing FuzzForge CLI with Python 3.12")
return success
def print_next_steps(self):
"""Print next steps for the user"""
print(f"\n{Colors.BOLD}{Colors.GREEN}🎉 Setup Complete!{Colors.END}")
if not self.errors:
print(f"""
{Colors.CYAN}🚀 FuzzForge is now ready! Here's what you can do next:{Colors.END}
{Colors.BOLD}📖 Learn More:{Colors.END}
{Colors.BLUE}docs/QUICKSTART.md{Colors.END} - 5-minute module creation tutorial
{Colors.BLUE}docs/PATTERNS.md{Colors.END} - Common patterns and recipes
{Colors.BLUE}cli/README.md{Colors.END} - Complete CLI documentation
{Colors.BOLD}🔍 Try Some Commands:{Colors.END}
{Colors.WHITE}cd test_projects/vulnerable_app{Colors.END} - Navigate to test project
{Colors.WHITE}fuzzforge init{Colors.END} - Initialize a FuzzForge project
{Colors.WHITE}ff workflow security_assessment .{Colors.END} - Run security assessment
{Colors.WHITE}ff workflow secret_detection_scan .{Colors.END} - Run secret detection
{Colors.WHITE}fuzzforge status{Colors.END} - Check project and workflow status
{Colors.WHITE}fuzzforge --help{Colors.END} - See all available commands
{Colors.BOLD}🔧 Available Workflows:{Colors.END}
{Colors.CYAN}security_assessment{Colors.END} - Comprehensive security scanning
{Colors.CYAN}secret_detection_scan{Colors.END} - Credential and secret detection
{Colors.BOLD}🌐 Web Interface:{Colors.END}
• API: {Colors.WHITE}http://localhost:8000{Colors.END}
• Health: {Colors.WHITE}http://localhost:8000/health{Colors.END}
• API Docs: {Colors.WHITE}http://localhost:8000/docs{Colors.END}
""")
if self.warnings:
print(f"\n{Colors.YELLOW}⚠️ Warnings:{Colors.END}")
for warning in self.warnings:
print(f"{warning}")
if self.errors:
print(f"\n{Colors.RED}❌ Errors that need attention:{Colors.END}")
for error in self.errors:
print(f"{error}")
print(f"\n{Colors.YELLOW}🔧 Please fix these issues and run the setup again.{Colors.END}")
def run(self):
"""Run the complete setup process"""
self.print_header()
# Step 1: Prerequisites
if not self.check_prerequisites():
if self.errors:
self.print_next_steps()
return False
# Step 2: Docker setup
if not self.setup_docker_environment():
if self.errors:
self.print_next_steps()
return False
# Step 3: CLI installation
self.install_cli()
# Final summary
self.print_next_steps()
return len(self.errors) == 0
def main():
"""Main entry point"""
setup = FuzzForgeSetup()
try:
success = setup.run()
sys.exit(0 if success else 1)
except KeyboardInterrupt:
print(f"\n{Colors.YELLOW}Setup interrupted by user. You can run this script again anytime.{Colors.END}")
sys.exit(1)
except Exception as e:
print(f"\n{Colors.RED}Unexpected error: {e}{Colors.END}")
print(f"{Colors.YELLOW}Please report this issue with the error details above.{Colors.END}")
sys.exit(1)
if __name__ == "__main__":
main()