mirror of
https://github.com/FuzzingLabs/fuzzforge_ai.git
synced 2026-07-07 14:17:57 +02:00
CI/CD Integration with Ephemeral Deployment Model (#14)
* feat: Complete migration from Prefect to Temporal BREAKING CHANGE: Replaces Prefect workflow orchestration with Temporal ## Major Changes - Replace Prefect with Temporal for workflow orchestration - Implement vertical worker architecture (rust, android) - Replace Docker registry with MinIO for unified storage - Refactor activities to be co-located with workflows - Update all API endpoints for Temporal compatibility ## Infrastructure - New: docker-compose.temporal.yaml (Temporal + MinIO + workers) - New: workers/ directory with rust and android vertical workers - New: backend/src/temporal/ (manager, discovery) - New: backend/src/storage/ (S3-cached storage with MinIO) - New: backend/toolbox/common/ (shared storage activities) - Deleted: docker-compose.yaml (old Prefect setup) - Deleted: backend/src/core/prefect_manager.py - Deleted: backend/src/services/prefect_stats_monitor.py - Deleted: Docker registry and insecure-registries requirement ## Workflows - Migrated: security_assessment workflow to Temporal - New: rust_test workflow (example/test workflow) - Deleted: secret_detection_scan (Prefect-based, to be reimplemented) - Activities now co-located with workflows for independent testing ## API Changes - Updated: backend/src/api/workflows.py (Temporal submission) - Updated: backend/src/api/runs.py (Temporal status/results) - Updated: backend/src/main.py (727 lines, TemporalManager integration) - Updated: All 16 MCP tools to use TemporalManager ## Testing - ✅ All services healthy (Temporal, PostgreSQL, MinIO, workers, backend) - ✅ All API endpoints functional - ✅ End-to-end workflow test passed (72 findings from vulnerable_app) - ✅ MinIO storage integration working (target upload/download, results) - ✅ Worker activity discovery working (6 activities registered) - ✅ Tarball extraction working - ✅ SARIF report generation working ## Documentation - ARCHITECTURE.md: Complete Temporal architecture documentation - QUICKSTART_TEMPORAL.md: Getting started guide - MIGRATION_DECISION.md: Why we chose Temporal over Prefect - IMPLEMENTATION_STATUS.md: Migration progress tracking - workers/README.md: Worker development guide ## Dependencies - Added: temporalio>=1.6.0 - Added: boto3>=1.34.0 (MinIO S3 client) - Removed: prefect>=3.4.18 * feat: Add Python fuzzing vertical with Atheris integration This commit implements a complete Python fuzzing workflow using Atheris: ## Python Worker (workers/python/) - Dockerfile with Python 3.11, Atheris, and build tools - Generic worker.py for dynamic workflow discovery - requirements.txt with temporalio, boto3, atheris dependencies - Added to docker-compose.temporal.yaml with dedicated cache volume ## AtherisFuzzer Module (backend/toolbox/modules/fuzzer/) - Reusable module extending BaseModule - Auto-discovers fuzz targets (fuzz_*.py, *_fuzz.py, fuzz_target.py) - Recursive search to find targets in nested directories - Dynamically loads TestOneInput() function - Configurable max_iterations and timeout - Real-time stats callback support for live monitoring - Returns findings as ModuleFinding objects ## Atheris Fuzzing Workflow (backend/toolbox/workflows/atheris_fuzzing/) - Temporal workflow for orchestrating fuzzing - Downloads user code from MinIO - Executes AtherisFuzzer module - Uploads results to MinIO - Cleans up cache after execution - metadata.yaml with vertical: python for routing ## Test Project (test_projects/python_fuzz_waterfall/) - Demonstrates stateful waterfall vulnerability - main.py with check_secret() that leaks progress - fuzz_target.py with Atheris TestOneInput() harness - Complete README with usage instructions ## Backend Fixes - Fixed parameter merging in REST API endpoints (workflows.py) - Changed workflow parameter passing from positional args to kwargs (manager.py) - Default parameters now properly merged with user parameters ## Testing ✅ Worker discovered AtherisFuzzingWorkflow ✅ Workflow executed end-to-end successfully ✅ Fuzz target auto-discovered in nested directories ✅ Atheris ran 100,000 iterations ✅ Results uploaded and cache cleaned * chore: Complete Temporal migration with updated CLI/SDK/docs This commit includes all remaining Temporal migration changes: ## CLI Updates (cli/) - Updated workflow execution commands for Temporal - Enhanced error handling and exceptions - Updated dependencies in uv.lock ## SDK Updates (sdk/) - Client methods updated for Temporal workflows - Updated models for new workflow execution - Updated dependencies in uv.lock ## Documentation Updates (docs/) - Architecture documentation for Temporal - Workflow concept documentation - Resource management documentation (new) - Debugging guide (new) - Updated tutorials and how-to guides - Troubleshooting updates ## README Updates - Main README with Temporal instructions - Backend README - CLI README - SDK README ## Other - Updated IMPLEMENTATION_STATUS.md - Removed old vulnerable_app.tar.gz These changes complete the Temporal migration and ensure the CLI/SDK work correctly with the new backend. * fix: Use positional args instead of kwargs for Temporal workflows The Temporal Python SDK's start_workflow() method doesn't accept a 'kwargs' parameter. Workflows must receive parameters as positional arguments via the 'args' parameter. Changed from: args=workflow_args # Positional arguments This fixes the error: TypeError: Client.start_workflow() got an unexpected keyword argument 'kwargs' Workflows now correctly receive parameters in order: - security_assessment: [target_id, scanner_config, analyzer_config, reporter_config] - atheris_fuzzing: [target_id, target_file, max_iterations, timeout_seconds] - rust_test: [target_id, test_message] * fix: Filter metadata-only parameters from workflow arguments SecurityAssessmentWorkflow was receiving 7 arguments instead of 2-5. The issue was that target_path and volume_mode from default_parameters were being passed to the workflow, when they should only be used by the system for configuration. Now filters out metadata-only parameters (target_path, volume_mode) before passing arguments to workflow execution. * refactor: Remove Prefect leftovers and volume mounting legacy Complete cleanup of Prefect migration artifacts: Backend: - Delete registry.py and workflow_discovery.py (Prefect-specific files) - Remove Docker validation from setup.py (no longer needed) - Remove ResourceLimits and VolumeMount models - Remove target_path and volume_mode from WorkflowSubmission - Remove supported_volume_modes from API and discovery - Clean up metadata.yaml files (remove volume/path fields) - Simplify parameter filtering in manager.py SDK: - Remove volume_mode parameter from client methods - Remove ResourceLimits and VolumeMount models - Remove Prefect error patterns from docker_logs.py - Clean up WorkflowSubmission and WorkflowMetadata models CLI: - Remove Volume Modes display from workflow info All removed features are Prefect-specific or Docker volume mounting artifacts. Temporal workflows use MinIO storage exclusively. * feat: Add comprehensive test suite and benchmark infrastructure - Add 68 unit tests for fuzzer, scanner, and analyzer modules - Implement pytest-based test infrastructure with fixtures - Add 6 performance benchmarks with category-specific thresholds - Configure GitHub Actions for automated testing and benchmarking - Add test and benchmark documentation Test coverage: - AtherisFuzzer: 8 tests - CargoFuzzer: 14 tests - FileScanner: 22 tests - SecurityAnalyzer: 24 tests All tests passing (68/68) All benchmarks passing (6/6) * fix: Resolve all ruff linting violations across codebase Fixed 27 ruff violations in 12 files: - Removed unused imports (Depends, Dict, Any, Optional, etc.) - Fixed undefined workflow_info variable in workflows.py - Removed dead code with undefined variables in atheris_fuzzer.py - Changed f-string to regular string where no placeholders used All files now pass ruff checks for CI/CD compliance. * fix: Configure CI for unit tests only - Renamed docker-compose.temporal.yaml → docker-compose.yml for CI compatibility - Commented out integration-tests job (no integration tests yet) - Updated test-summary to only depend on lint and unit-tests CI will now run successfully with 68 unit tests. Integration tests can be added later. * feat: Add CI/CD integration with ephemeral deployment model Implements comprehensive CI/CD support for FuzzForge with on-demand worker management: **Worker Management (v0.7.0)** - Add WorkerManager for automatic worker lifecycle control - Auto-start workers from stopped state when workflows execute - Auto-stop workers after workflow completion - Health checks and startup timeout handling (90s default) **CI/CD Features** - `--fail-on` flag: Fail builds based on SARIF severity levels (error/warning/note/info) - `--export-sarif` flag: Export findings in SARIF 2.1.0 format - `--auto-start`/`--auto-stop` flags: Control worker lifecycle - Exit code propagation: Returns 1 on blocking findings, 0 on success **Exit Code Fix** - Add `except typer.Exit: raise` handlers at 3 critical locations - Move worker cleanup to finally block for guaranteed execution - Exit codes now propagate correctly even when build fails **CI Scripts & Examples** - ci-start.sh: Start FuzzForge services with health checks - ci-stop.sh: Clean shutdown with volume preservation option - GitHub Actions workflow example (security-scan.yml) - GitLab CI pipeline example (.gitlab-ci.example.yml) - docker-compose.ci.yml: CI-optimized compose file with profiles **OSS-Fuzz Integration** - New ossfuzz_campaign workflow for running OSS-Fuzz projects - OSS-Fuzz worker with Docker-in-Docker support - Configurable campaign duration and project selection **Documentation** - Comprehensive CI/CD integration guide (docs/how-to/cicd-integration.md) - Updated architecture docs with worker lifecycle details - Updated workspace isolation documentation - CLI README with worker management examples **SDK Enhancements** - Add get_workflow_worker_info() endpoint - Worker vertical metadata in workflow responses **Testing** - All workflows tested: security_assessment, atheris_fuzzing, secret_detection, cargo_fuzzing - All monitoring commands tested: stats, crashes, status, finding - Full CI pipeline simulation verified - Exit codes verified for success/failure scenarios Ephemeral CI/CD model: ~3-4GB RAM, ~60-90s startup, runs entirely in CI containers. * fix: Resolve ruff linting violations in CI/CD code - Remove unused variables (run_id, defaults, result) - Remove unused imports - Fix f-string without placeholders All CI/CD integration files now pass ruff checks.
This commit is contained in:
tduhamel42
@@ -0,0 +1,113 @@
|
||||
name: ossfuzz_campaign
|
||||
version: "1.0.0"
|
||||
vertical: ossfuzz
|
||||
description: "Generic OSS-Fuzz fuzzing campaign. Automatically reads project configuration from OSS-Fuzz repo and runs fuzzing using Google's infrastructure."
|
||||
author: "FuzzForge Team"
|
||||
tags:
|
||||
- "fuzzing"
|
||||
- "oss-fuzz"
|
||||
- "libfuzzer"
|
||||
- "afl"
|
||||
- "honggfuzz"
|
||||
- "memory-safety"
|
||||
- "security"
|
||||
|
||||
# Workspace isolation mode
|
||||
# OSS-Fuzz campaigns use isolated mode for safe concurrent campaigns
|
||||
workspace_isolation: "isolated"
|
||||
|
||||
default_parameters:
|
||||
project_name: null
|
||||
campaign_duration_hours: 1
|
||||
override_engine: null
|
||||
override_sanitizer: null
|
||||
max_iterations: null
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
required:
|
||||
- project_name
|
||||
properties:
|
||||
project_name:
|
||||
type: string
|
||||
description: "OSS-Fuzz project name (e.g., 'curl', 'sqlite3', 'libxml2')"
|
||||
examples:
|
||||
- "curl"
|
||||
- "sqlite3"
|
||||
- "libxml2"
|
||||
- "openssl"
|
||||
- "zlib"
|
||||
|
||||
campaign_duration_hours:
|
||||
type: integer
|
||||
default: 1
|
||||
minimum: 1
|
||||
maximum: 168 # 1 week max
|
||||
description: "How many hours to run the fuzzing campaign"
|
||||
|
||||
override_engine:
|
||||
type: string
|
||||
enum: ["libfuzzer", "afl", "honggfuzz"]
|
||||
description: "Override fuzzing engine from project.yaml (optional)"
|
||||
|
||||
override_sanitizer:
|
||||
type: string
|
||||
enum: ["address", "memory", "undefined", "dataflow"]
|
||||
description: "Override sanitizer from project.yaml (optional)"
|
||||
|
||||
max_iterations:
|
||||
type: integer
|
||||
minimum: 1000
|
||||
description: "Optional limit on fuzzing iterations (optional)"
|
||||
|
||||
output_schema:
|
||||
type: object
|
||||
properties:
|
||||
project_name:
|
||||
type: string
|
||||
description: "OSS-Fuzz project that was fuzzed"
|
||||
|
||||
summary:
|
||||
type: object
|
||||
description: "Campaign execution summary"
|
||||
properties:
|
||||
total_executions:
|
||||
type: integer
|
||||
crashes_found:
|
||||
type: integer
|
||||
unique_crashes:
|
||||
type: integer
|
||||
duration_hours:
|
||||
type: number
|
||||
engine_used:
|
||||
type: string
|
||||
sanitizer_used:
|
||||
type: string
|
||||
|
||||
crashes:
|
||||
type: array
|
||||
description: "List of crash file paths"
|
||||
items:
|
||||
type: string
|
||||
|
||||
sarif:
|
||||
type: object
|
||||
description: "SARIF-formatted crash reports (future)"
|
||||
|
||||
examples:
|
||||
- name: "Fuzz curl for 1 hour"
|
||||
parameters:
|
||||
project_name: "curl"
|
||||
campaign_duration_hours: 1
|
||||
|
||||
- name: "Fuzz sqlite3 with AFL"
|
||||
parameters:
|
||||
project_name: "sqlite3"
|
||||
campaign_duration_hours: 2
|
||||
override_engine: "afl"
|
||||
|
||||
- name: "Fuzz libxml2 with memory sanitizer"
|
||||
parameters:
|
||||
project_name: "libxml2"
|
||||
campaign_duration_hours: 6
|
||||
override_sanitizer: "memory"
|
||||
@@ -0,0 +1,219 @@
|
||||
"""
|
||||
OSS-Fuzz Campaign Workflow - Temporal Version
|
||||
|
||||
Generic workflow for running OSS-Fuzz campaigns using Google's infrastructure.
|
||||
Automatically reads project configuration from OSS-Fuzz project.yaml files.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from datetime import timedelta
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
from temporalio import workflow
|
||||
from temporalio.common import RetryPolicy
|
||||
|
||||
# Import for type hints (will be executed by worker)
|
||||
with workflow.unsafe.imports_passed_through():
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@workflow.defn
|
||||
class OssfuzzCampaignWorkflow:
|
||||
"""
|
||||
Generic OSS-Fuzz fuzzing campaign workflow.
|
||||
|
||||
User workflow:
|
||||
1. User runs: ff workflow run ossfuzz_campaign . project_name=curl
|
||||
2. Worker loads project config from OSS-Fuzz repo
|
||||
3. Worker builds project using OSS-Fuzz's build system
|
||||
4. Worker runs fuzzing with engines from project.yaml
|
||||
5. Crashes and corpus reported as findings
|
||||
"""
|
||||
|
||||
@workflow.run
|
||||
async def run(
|
||||
self,
|
||||
target_id: str, # Required by FuzzForge (not used, OSS-Fuzz downloads from Google)
|
||||
project_name: str, # Required: OSS-Fuzz project name (e.g., "curl", "sqlite3")
|
||||
campaign_duration_hours: int = 1,
|
||||
override_engine: Optional[str] = None, # Override engine from project.yaml
|
||||
override_sanitizer: Optional[str] = None, # Override sanitizer from project.yaml
|
||||
max_iterations: Optional[int] = None # Optional: limit fuzzing iterations
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Main workflow execution.
|
||||
|
||||
Args:
|
||||
target_id: UUID of uploaded target (not used, required by FuzzForge)
|
||||
project_name: Name of OSS-Fuzz project (e.g., "curl", "sqlite3", "libxml2")
|
||||
campaign_duration_hours: How many hours to fuzz (default: 1)
|
||||
override_engine: Override fuzzing engine from project.yaml
|
||||
override_sanitizer: Override sanitizer from project.yaml
|
||||
max_iterations: Optional limit on fuzzing iterations
|
||||
|
||||
Returns:
|
||||
Dictionary containing crashes, stats, and SARIF report
|
||||
"""
|
||||
workflow_id = workflow.info().workflow_id
|
||||
|
||||
workflow.logger.info(
|
||||
f"Starting OSS-Fuzz Campaign for project '{project_name}' "
|
||||
f"(workflow_id={workflow_id}, duration={campaign_duration_hours}h)"
|
||||
)
|
||||
|
||||
results = {
|
||||
"workflow_id": workflow_id,
|
||||
"project_name": project_name,
|
||||
"status": "running",
|
||||
"steps": []
|
||||
}
|
||||
|
||||
try:
|
||||
# Step 1: Load OSS-Fuzz project configuration
|
||||
workflow.logger.info(f"Step 1: Loading project config for '{project_name}'")
|
||||
project_config = await workflow.execute_activity(
|
||||
"load_ossfuzz_project",
|
||||
args=[project_name],
|
||||
start_to_close_timeout=timedelta(minutes=5),
|
||||
retry_policy=RetryPolicy(
|
||||
initial_interval=timedelta(seconds=1),
|
||||
maximum_interval=timedelta(seconds=30),
|
||||
maximum_attempts=3
|
||||
)
|
||||
)
|
||||
|
||||
results["steps"].append({
|
||||
"step": "load_config",
|
||||
"status": "success",
|
||||
"language": project_config.get("language"),
|
||||
"engines": project_config.get("fuzzing_engines", []),
|
||||
"sanitizers": project_config.get("sanitizers", [])
|
||||
})
|
||||
|
||||
workflow.logger.info(
|
||||
f"✓ Loaded config: language={project_config.get('language')}, "
|
||||
f"engines={project_config.get('fuzzing_engines')}"
|
||||
)
|
||||
|
||||
# Step 2: Build project using OSS-Fuzz infrastructure
|
||||
workflow.logger.info(f"Step 2: Building project '{project_name}'")
|
||||
|
||||
build_result = await workflow.execute_activity(
|
||||
"build_ossfuzz_project",
|
||||
args=[
|
||||
project_name,
|
||||
project_config,
|
||||
override_sanitizer,
|
||||
override_engine
|
||||
],
|
||||
start_to_close_timeout=timedelta(minutes=30),
|
||||
retry_policy=RetryPolicy(
|
||||
initial_interval=timedelta(seconds=2),
|
||||
maximum_interval=timedelta(seconds=60),
|
||||
maximum_attempts=2
|
||||
)
|
||||
)
|
||||
|
||||
results["steps"].append({
|
||||
"step": "build_project",
|
||||
"status": "success",
|
||||
"fuzz_targets": len(build_result.get("fuzz_targets", [])),
|
||||
"sanitizer": build_result.get("sanitizer_used"),
|
||||
"engine": build_result.get("engine_used")
|
||||
})
|
||||
|
||||
workflow.logger.info(
|
||||
f"✓ Build completed: {len(build_result.get('fuzz_targets', []))} fuzz targets found"
|
||||
)
|
||||
|
||||
if not build_result.get("fuzz_targets"):
|
||||
raise Exception(f"No fuzz targets found for project {project_name}")
|
||||
|
||||
# Step 3: Run fuzzing on discovered targets
|
||||
workflow.logger.info(f"Step 3: Fuzzing {len(build_result['fuzz_targets'])} targets")
|
||||
|
||||
# Determine which engine to use
|
||||
engine_to_use = override_engine if override_engine else build_result["engine_used"]
|
||||
duration_seconds = campaign_duration_hours * 3600
|
||||
|
||||
# Fuzz each target (in parallel if multiple targets)
|
||||
fuzz_futures = []
|
||||
for target_path in build_result["fuzz_targets"]:
|
||||
future = workflow.execute_activity(
|
||||
"fuzz_target",
|
||||
args=[target_path, engine_to_use, duration_seconds, None, None],
|
||||
start_to_close_timeout=timedelta(seconds=duration_seconds + 300),
|
||||
retry_policy=RetryPolicy(
|
||||
initial_interval=timedelta(seconds=2),
|
||||
maximum_interval=timedelta(seconds=60),
|
||||
maximum_attempts=1 # Fuzzing shouldn't retry
|
||||
)
|
||||
)
|
||||
fuzz_futures.append(future)
|
||||
|
||||
# Wait for all fuzzing to complete
|
||||
fuzz_results = await asyncio.gather(*fuzz_futures, return_exceptions=True)
|
||||
|
||||
# Aggregate results
|
||||
total_execs = 0
|
||||
total_crashes = 0
|
||||
all_crashes = []
|
||||
|
||||
for i, result in enumerate(fuzz_results):
|
||||
if isinstance(result, Exception):
|
||||
workflow.logger.error(f"Fuzzing failed for target {i}: {result}")
|
||||
continue
|
||||
|
||||
total_execs += result.get("total_executions", 0)
|
||||
total_crashes += result.get("crashes", 0)
|
||||
all_crashes.extend(result.get("crash_files", []))
|
||||
|
||||
results["steps"].append({
|
||||
"step": "fuzzing",
|
||||
"status": "success",
|
||||
"total_executions": total_execs,
|
||||
"crashes_found": total_crashes,
|
||||
"targets_fuzzed": len(build_result["fuzz_targets"])
|
||||
})
|
||||
|
||||
workflow.logger.info(
|
||||
f"✓ Fuzzing completed: {total_execs} executions, {total_crashes} crashes"
|
||||
)
|
||||
|
||||
# Step 4: Generate SARIF report
|
||||
workflow.logger.info("Step 4: Generating SARIF report")
|
||||
|
||||
# TODO: Implement crash minimization and SARIF generation
|
||||
# For now, return raw results
|
||||
|
||||
results["status"] = "success"
|
||||
results["summary"] = {
|
||||
"project": project_name,
|
||||
"total_executions": total_execs,
|
||||
"crashes_found": total_crashes,
|
||||
"unique_crashes": len(set(all_crashes)),
|
||||
"duration_hours": campaign_duration_hours,
|
||||
"engine_used": engine_to_use,
|
||||
"sanitizer_used": build_result.get("sanitizer_used")
|
||||
}
|
||||
results["crashes"] = all_crashes[:100] # Limit to first 100 crashes
|
||||
|
||||
workflow.logger.info(
|
||||
f"✓ Campaign completed: {project_name} - "
|
||||
f"{total_execs} execs, {total_crashes} crashes"
|
||||
)
|
||||
|
||||
return results
|
||||
|
||||
except Exception as e:
|
||||
workflow.logger.error(f"Workflow failed: {e}")
|
||||
results["status"] = "error"
|
||||
results["error"] = str(e)
|
||||
results["steps"].append({
|
||||
"step": "error",
|
||||
"status": "failed",
|
||||
"error": str(e)
|
||||
})
|
||||
raise
|
||||
Reference in New Issue
Block a user