Files
fuzzforge_ai/cli/src/fuzzforge_cli/main.py
T
tduhamel42 ec812461d6 CI/CD Integration with Ephemeral Deployment Model (#14)
* feat: Complete migration from Prefect to Temporal

BREAKING CHANGE: Replaces Prefect workflow orchestration with Temporal

## Major Changes
- Replace Prefect with Temporal for workflow orchestration
- Implement vertical worker architecture (rust, android)
- Replace Docker registry with MinIO for unified storage
- Refactor activities to be co-located with workflows
- Update all API endpoints for Temporal compatibility

## Infrastructure
- New: docker-compose.temporal.yaml (Temporal + MinIO + workers)
- New: workers/ directory with rust and android vertical workers
- New: backend/src/temporal/ (manager, discovery)
- New: backend/src/storage/ (S3-cached storage with MinIO)
- New: backend/toolbox/common/ (shared storage activities)
- Deleted: docker-compose.yaml (old Prefect setup)
- Deleted: backend/src/core/prefect_manager.py
- Deleted: backend/src/services/prefect_stats_monitor.py
- Deleted: Docker registry and insecure-registries requirement

## Workflows
- Migrated: security_assessment workflow to Temporal
- New: rust_test workflow (example/test workflow)
- Deleted: secret_detection_scan (Prefect-based, to be reimplemented)
- Activities now co-located with workflows for independent testing

## API Changes
- Updated: backend/src/api/workflows.py (Temporal submission)
- Updated: backend/src/api/runs.py (Temporal status/results)
- Updated: backend/src/main.py (727 lines, TemporalManager integration)
- Updated: All 16 MCP tools to use TemporalManager

## Testing
-  All services healthy (Temporal, PostgreSQL, MinIO, workers, backend)
-  All API endpoints functional
-  End-to-end workflow test passed (72 findings from vulnerable_app)
-  MinIO storage integration working (target upload/download, results)
-  Worker activity discovery working (6 activities registered)
-  Tarball extraction working
-  SARIF report generation working

## Documentation
- ARCHITECTURE.md: Complete Temporal architecture documentation
- QUICKSTART_TEMPORAL.md: Getting started guide
- MIGRATION_DECISION.md: Why we chose Temporal over Prefect
- IMPLEMENTATION_STATUS.md: Migration progress tracking
- workers/README.md: Worker development guide

## Dependencies
- Added: temporalio>=1.6.0
- Added: boto3>=1.34.0 (MinIO S3 client)
- Removed: prefect>=3.4.18

* feat: Add Python fuzzing vertical with Atheris integration

This commit implements a complete Python fuzzing workflow using Atheris:

## Python Worker (workers/python/)
- Dockerfile with Python 3.11, Atheris, and build tools
- Generic worker.py for dynamic workflow discovery
- requirements.txt with temporalio, boto3, atheris dependencies
- Added to docker-compose.temporal.yaml with dedicated cache volume

## AtherisFuzzer Module (backend/toolbox/modules/fuzzer/)
- Reusable module extending BaseModule
- Auto-discovers fuzz targets (fuzz_*.py, *_fuzz.py, fuzz_target.py)
- Recursive search to find targets in nested directories
- Dynamically loads TestOneInput() function
- Configurable max_iterations and timeout
- Real-time stats callback support for live monitoring
- Returns findings as ModuleFinding objects

## Atheris Fuzzing Workflow (backend/toolbox/workflows/atheris_fuzzing/)
- Temporal workflow for orchestrating fuzzing
- Downloads user code from MinIO
- Executes AtherisFuzzer module
- Uploads results to MinIO
- Cleans up cache after execution
- metadata.yaml with vertical: python for routing

## Test Project (test_projects/python_fuzz_waterfall/)
- Demonstrates stateful waterfall vulnerability
- main.py with check_secret() that leaks progress
- fuzz_target.py with Atheris TestOneInput() harness
- Complete README with usage instructions

## Backend Fixes
- Fixed parameter merging in REST API endpoints (workflows.py)
- Changed workflow parameter passing from positional args to kwargs (manager.py)
- Default parameters now properly merged with user parameters

## Testing
 Worker discovered AtherisFuzzingWorkflow
 Workflow executed end-to-end successfully
 Fuzz target auto-discovered in nested directories
 Atheris ran 100,000 iterations
 Results uploaded and cache cleaned

* chore: Complete Temporal migration with updated CLI/SDK/docs

This commit includes all remaining Temporal migration changes:

## CLI Updates (cli/)
- Updated workflow execution commands for Temporal
- Enhanced error handling and exceptions
- Updated dependencies in uv.lock

## SDK Updates (sdk/)
- Client methods updated for Temporal workflows
- Updated models for new workflow execution
- Updated dependencies in uv.lock

## Documentation Updates (docs/)
- Architecture documentation for Temporal
- Workflow concept documentation
- Resource management documentation (new)
- Debugging guide (new)
- Updated tutorials and how-to guides
- Troubleshooting updates

## README Updates
- Main README with Temporal instructions
- Backend README
- CLI README
- SDK README

## Other
- Updated IMPLEMENTATION_STATUS.md
- Removed old vulnerable_app.tar.gz

These changes complete the Temporal migration and ensure the
CLI/SDK work correctly with the new backend.

* fix: Use positional args instead of kwargs for Temporal workflows

The Temporal Python SDK's start_workflow() method doesn't accept
a 'kwargs' parameter. Workflows must receive parameters as positional
arguments via the 'args' parameter.

Changed from:
  args=workflow_args  # Positional arguments

This fixes the error:
  TypeError: Client.start_workflow() got an unexpected keyword argument 'kwargs'

Workflows now correctly receive parameters in order:
- security_assessment: [target_id, scanner_config, analyzer_config, reporter_config]
- atheris_fuzzing: [target_id, target_file, max_iterations, timeout_seconds]
- rust_test: [target_id, test_message]

* fix: Filter metadata-only parameters from workflow arguments

SecurityAssessmentWorkflow was receiving 7 arguments instead of 2-5.
The issue was that target_path and volume_mode from default_parameters
were being passed to the workflow, when they should only be used by
the system for configuration.

Now filters out metadata-only parameters (target_path, volume_mode)
before passing arguments to workflow execution.

* refactor: Remove Prefect leftovers and volume mounting legacy

Complete cleanup of Prefect migration artifacts:

Backend:
- Delete registry.py and workflow_discovery.py (Prefect-specific files)
- Remove Docker validation from setup.py (no longer needed)
- Remove ResourceLimits and VolumeMount models
- Remove target_path and volume_mode from WorkflowSubmission
- Remove supported_volume_modes from API and discovery
- Clean up metadata.yaml files (remove volume/path fields)
- Simplify parameter filtering in manager.py

SDK:
- Remove volume_mode parameter from client methods
- Remove ResourceLimits and VolumeMount models
- Remove Prefect error patterns from docker_logs.py
- Clean up WorkflowSubmission and WorkflowMetadata models

CLI:
- Remove Volume Modes display from workflow info

All removed features are Prefect-specific or Docker volume mounting
artifacts. Temporal workflows use MinIO storage exclusively.

* feat: Add comprehensive test suite and benchmark infrastructure

- Add 68 unit tests for fuzzer, scanner, and analyzer modules
- Implement pytest-based test infrastructure with fixtures
- Add 6 performance benchmarks with category-specific thresholds
- Configure GitHub Actions for automated testing and benchmarking
- Add test and benchmark documentation

Test coverage:
- AtherisFuzzer: 8 tests
- CargoFuzzer: 14 tests
- FileScanner: 22 tests
- SecurityAnalyzer: 24 tests

All tests passing (68/68)
All benchmarks passing (6/6)

* fix: Resolve all ruff linting violations across codebase

Fixed 27 ruff violations in 12 files:
- Removed unused imports (Depends, Dict, Any, Optional, etc.)
- Fixed undefined workflow_info variable in workflows.py
- Removed dead code with undefined variables in atheris_fuzzer.py
- Changed f-string to regular string where no placeholders used

All files now pass ruff checks for CI/CD compliance.

* fix: Configure CI for unit tests only

- Renamed docker-compose.temporal.yaml → docker-compose.yml for CI compatibility
- Commented out integration-tests job (no integration tests yet)
- Updated test-summary to only depend on lint and unit-tests

CI will now run successfully with 68 unit tests. Integration tests can be added later.

* feat: Add CI/CD integration with ephemeral deployment model

Implements comprehensive CI/CD support for FuzzForge with on-demand worker management:

**Worker Management (v0.7.0)**
- Add WorkerManager for automatic worker lifecycle control
- Auto-start workers from stopped state when workflows execute
- Auto-stop workers after workflow completion
- Health checks and startup timeout handling (90s default)

**CI/CD Features**
- `--fail-on` flag: Fail builds based on SARIF severity levels (error/warning/note/info)
- `--export-sarif` flag: Export findings in SARIF 2.1.0 format
- `--auto-start`/`--auto-stop` flags: Control worker lifecycle
- Exit code propagation: Returns 1 on blocking findings, 0 on success

**Exit Code Fix**
- Add `except typer.Exit: raise` handlers at 3 critical locations
- Move worker cleanup to finally block for guaranteed execution
- Exit codes now propagate correctly even when build fails

**CI Scripts & Examples**
- ci-start.sh: Start FuzzForge services with health checks
- ci-stop.sh: Clean shutdown with volume preservation option
- GitHub Actions workflow example (security-scan.yml)
- GitLab CI pipeline example (.gitlab-ci.example.yml)
- docker-compose.ci.yml: CI-optimized compose file with profiles

**OSS-Fuzz Integration**
- New ossfuzz_campaign workflow for running OSS-Fuzz projects
- OSS-Fuzz worker with Docker-in-Docker support
- Configurable campaign duration and project selection

**Documentation**
- Comprehensive CI/CD integration guide (docs/how-to/cicd-integration.md)
- Updated architecture docs with worker lifecycle details
- Updated workspace isolation documentation
- CLI README with worker management examples

**SDK Enhancements**
- Add get_workflow_worker_info() endpoint
- Worker vertical metadata in workflow responses

**Testing**
- All workflows tested: security_assessment, atheris_fuzzing, secret_detection, cargo_fuzzing
- All monitoring commands tested: stats, crashes, status, finding
- Full CI pipeline simulation verified
- Exit codes verified for success/failure scenarios

Ephemeral CI/CD model: ~3-4GB RAM, ~60-90s startup, runs entirely in CI containers.

* fix: Resolve ruff linting violations in CI/CD code

- Remove unused variables (run_id, defaults, result)
- Remove unused imports
- Fix f-string without placeholders

All CI/CD integration files now pass ruff checks.
2025-10-14 10:13:45 +02:00

466 lines
16 KiB
Python

"""
Main CLI application with improved command structure.
"""
# Copyright (c) 2025 FuzzingLabs
#
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
# at the root of this repository for details.
#
# After the Change Date (four years from publication), this version of the
# Licensed Work will be made available under the Apache License, Version 2.0.
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
#
# Additional attribution and requirements are provided in the NOTICE file.
import typer
from rich.console import Console
from rich.traceback import install
from typing import Optional, List
import sys
from .commands import (
init,
workflows,
workflow_exec,
findings,
monitor,
config as config_cmd,
ai,
ingest,
)
from .constants import DEFAULT_VOLUME_MODE
from .fuzzy import enhanced_command_not_found_handler
# Install rich traceback handler
install(show_locals=True)
# Create console for rich output
console = Console()
# Create the main Typer app
app = typer.Typer(
name="fuzzforge",
help=(
"\b\n"
"[cyan]███████╗██╗ ██╗███████╗███████╗███████╗ ██████╗ ██████╗ ██████╗ ███████╗\n"
"██╔════╝██║ ██║╚══███╔╝╚══███╔╝██╔════╝██╔═══██╗██╔══██╗██╔════╝ ██╔════╝\n"
"█████╗ ██║ ██║ ███╔╝ ███╔╝ █████╗ ██║ ██║██████╔╝██║ ███╗█████╗ \n"
"██╔══╝ ██║ ██║ ███╔╝ ███╔╝ ██╔══╝ ██║ ██║██╔══██╗██║ ██║██╔══╝ \n"
"██║ ╚██████╔╝███████╗███████╗██║ ╚██████╔╝██║ ██║╚██████╔╝███████╗\n"
"╚═╝ ╚═════╝ ╚══════╝╚══════╝╚═╝ ╚═════╝ ╚═╝ ╚═╝ ╚═════╝ ╚══════╝[/cyan]\n\n"
"🛡️ Security testing workflow orchestration platform"
),
rich_markup_mode="rich",
no_args_is_help=True,
context_settings={
# Prevent help text from wrapping so ASCII art stays aligned
"max_content_width": 200,
# Keep common help flags
"help_option_names": ["--help", "-h"],
},
)
# Create workflow singular command group
workflow_app = typer.Typer(
name="workflow",
help="🚀 Execute and manage individual workflows",
no_args_is_help=False, # Allow direct execution
)
# Create finding singular command group
finding_app = typer.Typer(
name="finding",
help="🔍 View and analyze individual findings",
no_args_is_help=False,
)
# === Top-level commands ===
@app.command()
def init(
name: Optional[str] = typer.Option(
None, "--name", "-n",
help="Project name (defaults to current directory name)"
),
api_url: Optional[str] = typer.Option(
None, "--api-url", "-u",
help="FuzzForge API URL (defaults to http://localhost:8000)"
),
force: bool = typer.Option(
False, "--force", "-f",
help="Force initialization even if project already exists"
)
):
"""
📁 Initialize a new FuzzForge project
"""
from .commands.init import project
project(name=name, api_url=api_url, force=force)
@app.command()
def status():
"""
📊 Show project and latest execution status
"""
from .commands.status import show_status
show_status()
@app.command()
def config(
key: Optional[str] = typer.Argument(None, help="Configuration key"),
value: Optional[str] = typer.Argument(None, help="Configuration value to set")
):
"""
⚙️ Manage configuration (show all, get, or set values)
"""
if key is None:
# No arguments: show all config
config_cmd.show_config(global_config=False)
elif value is None:
# Key only: get specific value
config_cmd.get_config(key=key, global_config=False)
else:
# Key and value: set value
config_cmd.set_config(key=key, value=value, global_config=False)
@app.command()
def clean(
days: int = typer.Option(
90, "--days", "-d",
help="Remove data older than this many days"
),
dry_run: bool = typer.Option(
False, "--dry-run",
help="Show what would be deleted without actually deleting"
)
):
"""
🧹 Clean old execution data and findings
"""
from .database import get_project_db
from .exceptions import require_project
try:
require_project()
db = get_project_db()
if not db:
console.print("❌ No project database found", style="red")
raise typer.Exit(1)
if dry_run:
console.print(f"🔍 [bold]Dry run:[/bold] Would clean data older than {days} days")
deleted = db.cleanup_old_runs(keep_days=days)
if not dry_run:
console.print(f"✅ Cleaned {deleted} old executions", style="green")
else:
console.print(f"Would delete {deleted} executions", style="yellow")
except Exception as e:
console.print(f"❌ Failed to clean data: {e}", style="red")
raise typer.Exit(1)
# === Workflow commands (singular) ===
# Add workflow subcommands first (before callback)
workflow_app.command("status")(workflow_exec.workflow_status)
workflow_app.command("history")(workflow_exec.workflow_history)
workflow_app.command("retry")(workflow_exec.retry_workflow)
workflow_app.command("info")(workflows.workflow_info)
workflow_app.command("params")(workflows.workflow_parameters)
@workflow_app.command("run")
def run_workflow(
workflow: str = typer.Argument(help="Workflow name"),
target: str = typer.Argument(help="Target path"),
params: List[str] = typer.Argument(default=None, help="Parameters as key=value pairs"),
param_file: Optional[str] = typer.Option(
None, "--param-file", "-f",
help="JSON file containing workflow parameters"
),
volume_mode: str = typer.Option(
DEFAULT_VOLUME_MODE, "--volume-mode", "-v",
help="Volume mount mode: ro (read-only) or rw (read-write)"
),
timeout: Optional[int] = typer.Option(
None, "--timeout", "-t",
help="Execution timeout in seconds"
),
interactive: bool = typer.Option(
True, "--interactive/--no-interactive", "-i/-n",
help="Interactive parameter input for missing required parameters"
),
wait: bool = typer.Option(
False, "--wait", "-w",
help="Wait for execution to complete"
),
live: bool = typer.Option(
False, "--live", "-l",
help="Start live monitoring after execution (useful for fuzzing workflows)"
),
auto_start: Optional[bool] = typer.Option(
None, "--auto-start/--no-auto-start",
help="Automatically start required worker if not running (default: from config)"
),
auto_stop: Optional[bool] = typer.Option(
None, "--auto-stop/--no-auto-stop",
help="Automatically stop worker after execution completes (default: from config)"
),
fail_on: Optional[str] = typer.Option(
None, "--fail-on",
help="Fail build if findings match SARIF level (error,warning,note,info,all,none). Use with --wait"
),
export_sarif: Optional[str] = typer.Option(
None, "--export-sarif",
help="Export SARIF results to file after completion. Use with --wait"
)
):
"""
🚀 Execute a security testing workflow
Use --fail-on with --wait to fail CI builds based on finding severity.
Use --export-sarif with --wait to export SARIF findings to a file.
"""
from .commands.workflow_exec import execute_workflow
execute_workflow(
workflow=workflow,
target_path=target,
params=params,
param_file=param_file,
volume_mode=volume_mode,
timeout=timeout,
interactive=interactive,
wait=wait,
live=live,
auto_start=auto_start,
auto_stop=auto_stop,
fail_on=fail_on,
export_sarif=export_sarif
)
@workflow_app.callback()
def workflow_main():
"""
Execute workflows and manage workflow executions
Examples:
fuzzforge workflow security_assessment ./target # Execute workflow
fuzzforge workflow status # Check latest status
fuzzforge workflow history # Show execution history
"""
pass
# === Finding commands (singular) ===
@finding_app.command("export")
def export_finding(
execution_id: Optional[str] = typer.Argument(None, help="Execution ID (defaults to latest)"),
format: str = typer.Option(
"sarif", "--format", "-f",
help="Export format: sarif, json, csv"
),
output: Optional[str] = typer.Option(
None, "--output", "-o",
help="Output file (defaults to stdout)"
)
):
"""
📤 Export findings to file
"""
from .commands.findings import export_findings
from .database import get_project_db
from .exceptions import require_project
try:
require_project()
# If no ID provided, get the latest
if not execution_id:
db = get_project_db()
if db:
recent_runs = db.list_runs(limit=1)
if recent_runs:
execution_id = recent_runs[0].run_id
console.print(f"🔍 Using most recent execution: {execution_id}")
else:
console.print("⚠️ No findings found in project database", style="yellow")
return
else:
console.print("❌ No project database found", style="red")
return
export_findings(run_id=execution_id, format=format, output=output)
except Exception as e:
console.print(f"❌ Failed to export findings: {e}", style="red")
@finding_app.command("analyze")
def analyze_finding(
finding_id: Optional[str] = typer.Argument(None, help="Finding ID to analyze")
):
"""
🤖 AI analysis of a finding
"""
from .commands.ai import analyze_finding as ai_analyze
ai_analyze(finding_id)
@finding_app.callback(invoke_without_command=True)
def finding_main(
ctx: typer.Context,
):
"""
View and analyze individual findings
Examples:
fuzzforge finding # Show latest finding
fuzzforge finding <id> # Show specific finding
fuzzforge finding export # Export latest findings
"""
# Check if a subcommand is being invoked
if ctx.invoked_subcommand is not None:
# Let the subcommand handle it
return
# Get remaining arguments for direct viewing
args = ctx.args if hasattr(ctx, 'args') else []
finding_id = args[0] if args else None
# Direct viewing: fuzzforge finding [id]
from .commands.findings import get_findings
from .database import get_project_db
from .exceptions import require_project
try:
require_project()
# If no ID provided, get the latest
if not finding_id:
db = get_project_db()
if db:
recent_runs = db.list_runs(limit=1)
if recent_runs:
finding_id = recent_runs[0].run_id
console.print(f"🔍 Using most recent execution: {finding_id}")
else:
console.print("⚠️ No findings found in project database", style="yellow")
return
else:
console.print("❌ No project database found", style="red")
return
get_findings(run_id=finding_id, save=True, format="table")
except Exception as e:
console.print(f"❌ Failed to get findings: {e}", style="red")
# === Add command groups ===
# Plural commands (for browsing/listing)
app.add_typer(workflows.app, name="workflows", help="📋 Browse available workflows")
app.add_typer(findings.app, name="findings", help="📋 Browse all findings")
# Singular commands (for actions)
app.add_typer(workflow_app, name="workflow", help="🚀 Execute and manage workflows")
app.add_typer(finding_app, name="finding", help="🔍 View and analyze findings")
# Other command groups
app.add_typer(monitor.app, name="monitor", help="📊 Real-time monitoring")
app.add_typer(ai.app, name="ai", help="🤖 AI integration features")
app.add_typer(ingest.app, name="ingest", help="🧠 Ingest knowledge into AI")
# Help and utility commands
@app.command()
def version():
"""
📦 Show version information
"""
from . import __version__
console.print(f"FuzzForge CLI v{__version__}")
console.print("Short command: ff")
@app.callback()
def main_callback(
ctx: typer.Context,
version: Optional[bool] = typer.Option(
None, "--version", "-v",
help="Show version information"
),
):
"""
🛡️ FuzzForge CLI - Security testing workflow orchestration platform
Quick start:
• ff init - Initialize a new project
• ff workflows - See available workflows
• ff workflow <name> <target> - Execute a workflow
"""
if version:
from . import __version__
console.print(f"FuzzForge CLI v{__version__}")
raise typer.Exit()
def main():
"""Main entry point with smart command routing and error handling"""
# Smart command routing BEFORE Typer processes arguments
if len(sys.argv) > 1:
args = sys.argv[1:]
# Handle finding command with pattern recognition
if len(args) >= 2 and args[0] == 'finding':
finding_subcommands = ['export', 'analyze']
# Skip custom dispatching if help flags are present
if not any(arg in ['--help', '-h', '--version', '-v'] for arg in args):
if args[1] not in finding_subcommands:
# Direct finding display: ff finding <id>
from .commands.findings import get_findings
finding_id = args[1]
console.print(f"🔍 Displaying finding: {finding_id}")
try:
get_findings(run_id=finding_id, save=True, format="table")
return
except Exception as e:
console.print(f"❌ Failed to get finding: {e}", style="red")
sys.exit(1)
# Default Typer app handling
try:
app()
except SystemExit as e:
# Enhanced error handling for command not found
if hasattr(e, 'code') and e.code != 0 and len(sys.argv) > 1:
command_parts = sys.argv[1:]
clean_parts = [part for part in command_parts if not part.startswith('-')]
if clean_parts:
main_cmd = clean_parts[0]
valid_commands = [
'init', 'status', 'config', 'clean',
'workflows', 'workflow',
'findings', 'finding',
'monitor', 'ai', 'ingest',
'version'
]
if main_cmd not in valid_commands:
enhanced_command_not_found_handler(clean_parts)
sys.exit(1)
raise
if __name__ == "__main__":
main()