CI/CD Integration with Ephemeral Deployment Model (#14)

* feat: Complete migration from Prefect to Temporal

BREAKING CHANGE: Replaces Prefect workflow orchestration with Temporal

## Major Changes
- Replace Prefect with Temporal for workflow orchestration
- Implement vertical worker architecture (rust, android)
- Replace Docker registry with MinIO for unified storage
- Refactor activities to be co-located with workflows
- Update all API endpoints for Temporal compatibility

## Infrastructure
- New: docker-compose.temporal.yaml (Temporal + MinIO + workers)
- New: workers/ directory with rust and android vertical workers
- New: backend/src/temporal/ (manager, discovery)
- New: backend/src/storage/ (S3-cached storage with MinIO)
- New: backend/toolbox/common/ (shared storage activities)
- Deleted: docker-compose.yaml (old Prefect setup)
- Deleted: backend/src/core/prefect_manager.py
- Deleted: backend/src/services/prefect_stats_monitor.py
- Deleted: Docker registry and insecure-registries requirement

## Workflows
- Migrated: security_assessment workflow to Temporal
- New: rust_test workflow (example/test workflow)
- Deleted: secret_detection_scan (Prefect-based, to be reimplemented)
- Activities now co-located with workflows for independent testing

## API Changes
- Updated: backend/src/api/workflows.py (Temporal submission)
- Updated: backend/src/api/runs.py (Temporal status/results)
- Updated: backend/src/main.py (727 lines, TemporalManager integration)
- Updated: All 16 MCP tools to use TemporalManager

## Testing
-  All services healthy (Temporal, PostgreSQL, MinIO, workers, backend)
-  All API endpoints functional
-  End-to-end workflow test passed (72 findings from vulnerable_app)
-  MinIO storage integration working (target upload/download, results)
-  Worker activity discovery working (6 activities registered)
-  Tarball extraction working
-  SARIF report generation working

## Documentation
- ARCHITECTURE.md: Complete Temporal architecture documentation
- QUICKSTART_TEMPORAL.md: Getting started guide
- MIGRATION_DECISION.md: Why we chose Temporal over Prefect
- IMPLEMENTATION_STATUS.md: Migration progress tracking
- workers/README.md: Worker development guide

## Dependencies
- Added: temporalio>=1.6.0
- Added: boto3>=1.34.0 (MinIO S3 client)
- Removed: prefect>=3.4.18

* feat: Add Python fuzzing vertical with Atheris integration

This commit implements a complete Python fuzzing workflow using Atheris:

## Python Worker (workers/python/)
- Dockerfile with Python 3.11, Atheris, and build tools
- Generic worker.py for dynamic workflow discovery
- requirements.txt with temporalio, boto3, atheris dependencies
- Added to docker-compose.temporal.yaml with dedicated cache volume

## AtherisFuzzer Module (backend/toolbox/modules/fuzzer/)
- Reusable module extending BaseModule
- Auto-discovers fuzz targets (fuzz_*.py, *_fuzz.py, fuzz_target.py)
- Recursive search to find targets in nested directories
- Dynamically loads TestOneInput() function
- Configurable max_iterations and timeout
- Real-time stats callback support for live monitoring
- Returns findings as ModuleFinding objects

## Atheris Fuzzing Workflow (backend/toolbox/workflows/atheris_fuzzing/)
- Temporal workflow for orchestrating fuzzing
- Downloads user code from MinIO
- Executes AtherisFuzzer module
- Uploads results to MinIO
- Cleans up cache after execution
- metadata.yaml with vertical: python for routing

## Test Project (test_projects/python_fuzz_waterfall/)
- Demonstrates stateful waterfall vulnerability
- main.py with check_secret() that leaks progress
- fuzz_target.py with Atheris TestOneInput() harness
- Complete README with usage instructions

## Backend Fixes
- Fixed parameter merging in REST API endpoints (workflows.py)
- Changed workflow parameter passing from positional args to kwargs (manager.py)
- Default parameters now properly merged with user parameters

## Testing
 Worker discovered AtherisFuzzingWorkflow
 Workflow executed end-to-end successfully
 Fuzz target auto-discovered in nested directories
 Atheris ran 100,000 iterations
 Results uploaded and cache cleaned

* chore: Complete Temporal migration with updated CLI/SDK/docs

This commit includes all remaining Temporal migration changes:

## CLI Updates (cli/)
- Updated workflow execution commands for Temporal
- Enhanced error handling and exceptions
- Updated dependencies in uv.lock

## SDK Updates (sdk/)
- Client methods updated for Temporal workflows
- Updated models for new workflow execution
- Updated dependencies in uv.lock

## Documentation Updates (docs/)
- Architecture documentation for Temporal
- Workflow concept documentation
- Resource management documentation (new)
- Debugging guide (new)
- Updated tutorials and how-to guides
- Troubleshooting updates

## README Updates
- Main README with Temporal instructions
- Backend README
- CLI README
- SDK README

## Other
- Updated IMPLEMENTATION_STATUS.md
- Removed old vulnerable_app.tar.gz

These changes complete the Temporal migration and ensure the
CLI/SDK work correctly with the new backend.

* fix: Use positional args instead of kwargs for Temporal workflows

The Temporal Python SDK's start_workflow() method doesn't accept
a 'kwargs' parameter. Workflows must receive parameters as positional
arguments via the 'args' parameter.

Changed from:
  args=workflow_args  # Positional arguments

This fixes the error:
  TypeError: Client.start_workflow() got an unexpected keyword argument 'kwargs'

Workflows now correctly receive parameters in order:
- security_assessment: [target_id, scanner_config, analyzer_config, reporter_config]
- atheris_fuzzing: [target_id, target_file, max_iterations, timeout_seconds]
- rust_test: [target_id, test_message]

* fix: Filter metadata-only parameters from workflow arguments

SecurityAssessmentWorkflow was receiving 7 arguments instead of 2-5.
The issue was that target_path and volume_mode from default_parameters
were being passed to the workflow, when they should only be used by
the system for configuration.

Now filters out metadata-only parameters (target_path, volume_mode)
before passing arguments to workflow execution.

* refactor: Remove Prefect leftovers and volume mounting legacy

Complete cleanup of Prefect migration artifacts:

Backend:
- Delete registry.py and workflow_discovery.py (Prefect-specific files)
- Remove Docker validation from setup.py (no longer needed)
- Remove ResourceLimits and VolumeMount models
- Remove target_path and volume_mode from WorkflowSubmission
- Remove supported_volume_modes from API and discovery
- Clean up metadata.yaml files (remove volume/path fields)
- Simplify parameter filtering in manager.py

SDK:
- Remove volume_mode parameter from client methods
- Remove ResourceLimits and VolumeMount models
- Remove Prefect error patterns from docker_logs.py
- Clean up WorkflowSubmission and WorkflowMetadata models

CLI:
- Remove Volume Modes display from workflow info

All removed features are Prefect-specific or Docker volume mounting
artifacts. Temporal workflows use MinIO storage exclusively.

* feat: Add comprehensive test suite and benchmark infrastructure

- Add 68 unit tests for fuzzer, scanner, and analyzer modules
- Implement pytest-based test infrastructure with fixtures
- Add 6 performance benchmarks with category-specific thresholds
- Configure GitHub Actions for automated testing and benchmarking
- Add test and benchmark documentation

Test coverage:
- AtherisFuzzer: 8 tests
- CargoFuzzer: 14 tests
- FileScanner: 22 tests
- SecurityAnalyzer: 24 tests

All tests passing (68/68)
All benchmarks passing (6/6)

* fix: Resolve all ruff linting violations across codebase

Fixed 27 ruff violations in 12 files:
- Removed unused imports (Depends, Dict, Any, Optional, etc.)
- Fixed undefined workflow_info variable in workflows.py
- Removed dead code with undefined variables in atheris_fuzzer.py
- Changed f-string to regular string where no placeholders used

All files now pass ruff checks for CI/CD compliance.

* fix: Configure CI for unit tests only

- Renamed docker-compose.temporal.yaml → docker-compose.yml for CI compatibility
- Commented out integration-tests job (no integration tests yet)
- Updated test-summary to only depend on lint and unit-tests

CI will now run successfully with 68 unit tests. Integration tests can be added later.

* feat: Add CI/CD integration with ephemeral deployment model

Implements comprehensive CI/CD support for FuzzForge with on-demand worker management:

**Worker Management (v0.7.0)**
- Add WorkerManager for automatic worker lifecycle control
- Auto-start workers from stopped state when workflows execute
- Auto-stop workers after workflow completion
- Health checks and startup timeout handling (90s default)

**CI/CD Features**
- `--fail-on` flag: Fail builds based on SARIF severity levels (error/warning/note/info)
- `--export-sarif` flag: Export findings in SARIF 2.1.0 format
- `--auto-start`/`--auto-stop` flags: Control worker lifecycle
- Exit code propagation: Returns 1 on blocking findings, 0 on success

**Exit Code Fix**
- Add `except typer.Exit: raise` handlers at 3 critical locations
- Move worker cleanup to finally block for guaranteed execution
- Exit codes now propagate correctly even when build fails

**CI Scripts & Examples**
- ci-start.sh: Start FuzzForge services with health checks
- ci-stop.sh: Clean shutdown with volume preservation option
- GitHub Actions workflow example (security-scan.yml)
- GitLab CI pipeline example (.gitlab-ci.example.yml)
- docker-compose.ci.yml: CI-optimized compose file with profiles

**OSS-Fuzz Integration**
- New ossfuzz_campaign workflow for running OSS-Fuzz projects
- OSS-Fuzz worker with Docker-in-Docker support
- Configurable campaign duration and project selection

**Documentation**
- Comprehensive CI/CD integration guide (docs/how-to/cicd-integration.md)
- Updated architecture docs with worker lifecycle details
- Updated workspace isolation documentation
- CLI README with worker management examples

**SDK Enhancements**
- Add get_workflow_worker_info() endpoint
- Worker vertical metadata in workflow responses

**Testing**
- All workflows tested: security_assessment, atheris_fuzzing, secret_detection, cargo_fuzzing
- All monitoring commands tested: stats, crashes, status, finding
- Full CI pipeline simulation verified
- Exit codes verified for success/failure scenarios

Ephemeral CI/CD model: ~3-4GB RAM, ~60-90s startup, runs entirely in CI containers.

* fix: Resolve ruff linting violations in CI/CD code

- Remove unused variables (run_id, defaults, result)
- Remove unused imports
- Fix f-string without placeholders

All CI/CD integration files now pass ruff checks.
This commit is contained in:
tduhamel42
2025-10-14 10:13:45 +02:00
committed by GitHub
parent 987c49569c
commit 60ca088ecf
167 changed files with 26101 additions and 5703 deletions

View File

@@ -0,0 +1,18 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
*.egg-info/
dist/
build/
# FuzzForge
.fuzzforge/
# Atheris fuzzing artifacts
corpus/
crashes/
*.profraw
*.profdata

View File

@@ -0,0 +1,137 @@
# Python Fuzzing Test - Waterfall Vulnerability
This project demonstrates a **stateful vulnerability** that Atheris can discover through fuzzing.
## Vulnerability Description
The `check_secret()` function in `main.py` validates input character by character against the secret string "FUZZINGLABS". This creates a **waterfall vulnerability** where:
1. State leaks through the global `progress` variable
2. Each correct character advances the progress counter
3. When all 11 characters are provided in order, the function crashes with `SystemError`
This pattern is analogous to:
- Timing attacks on password checkers
- Protocol state machines with sequential validation
- Multi-step authentication flows
## Files
- `main.py` - Main application with vulnerable `check_secret()` function
- `fuzz_target.py` - Atheris fuzzing harness (contains `TestOneInput()`)
- `README.md` - This file
## How to Fuzz
### Using FuzzForge CLI
```bash
# Initialize FuzzForge in this directory
cd test_projects/python_fuzz_waterfall/
ff init
# Run fuzzing workflow (uploads code to MinIO)
ff workflow run atheris_fuzzing .
# The workflow will:
# 1. Upload this directory to MinIO
# 2. Worker downloads and extracts the code
# 3. Worker discovers fuzz_target.py (has TestOneInput)
# 4. Worker runs Atheris fuzzing
# 5. Reports real-time stats every 5 seconds
# 6. Finds crash when "FUZZINGLABS" is discovered
```
### Using FuzzForge SDK
```python
from fuzzforge_sdk import FuzzForgeClient
from pathlib import Path
client = FuzzForgeClient(base_url="http://localhost:8000")
# Upload and run fuzzing
response = client.submit_workflow_with_upload(
workflow_name="atheris_fuzzing",
target_path=Path("./"),
parameters={
"max_iterations": 100000,
"timeout_seconds": 300
}
)
print(f"Workflow started: {response.run_id}")
# Wait for completion
final_status = client.wait_for_completion(response.run_id)
findings = client.get_run_findings(response.run_id)
for finding in findings:
print(f"Crash: {finding.title}")
print(f"Input: {finding.metadata.get('crash_input_hex')}")
```
### Standalone (Without FuzzForge)
```bash
# Install Atheris
pip install atheris
# Run fuzzing directly
python fuzz_target.py
```
## Expected Behavior
When fuzzing:
1. **Initial phase**: Random exploration, progress = 0
2. **Discovery phase**: Atheris finds 'F' (first char), progress = 1
3. **Incremental progress**: Finds 'U', then 'Z', etc.
4. **Crash**: When full "FUZZINGLABS" discovered, crashes with:
```
SystemError: SECRET COMPROMISED: FUZZINGLABS
```
## Monitoring
Watch real-time fuzzing stats:
```bash
docker logs fuzzforge-worker-python -f | grep LIVE_STATS
```
Output example:
```
INFO - LIVE_STATS - executions=1523 execs_per_sec=1523.0 crashes=0
INFO - LIVE_STATS - executions=7842 execs_per_sec=2104.2 crashes=0
INFO - LIVE_STATS - executions=15234 execs_per_sec=2167.0 crashes=1 ← Crash found!
```
## Vulnerability Details
**CVE**: N/A (demonstration vulnerability)
**CWE**: CWE-208 (Observable Timing Discrepancy)
**Severity**: Critical (in real systems)
**Fix**: Remove state-based checking or implement constant-time comparison:
```python
def check_secret_safe(input_data: bytes) -> bool:
"""Constant-time comparison"""
import hmac
return hmac.compare_digest(input_data, SECRET.encode())
```
## Adjusting Difficulty
If fuzzing finds the crash too quickly, extend the secret:
```python
# In main.py, change:
SECRET = "FUZZINGLABSSECURITYTESTING" # 26 characters instead of 11
```
## License
MIT License - This is a demonstration project for educational purposes.

View File

@@ -0,0 +1,62 @@
"""
Atheris fuzzing target for the waterfall vulnerability.
This file is automatically discovered by FuzzForge's AtherisFuzzer module.
The fuzzer looks for files named: fuzz_*.py, *_fuzz.py, or fuzz_target.py
"""
import sys
import atheris
# Enable coverage instrumentation for imported modules
# This is critical for discovering the waterfall vulnerability!
with atheris.instrument_imports():
from main import check_secret
def TestOneInput(data):
"""
Atheris fuzzing entry point.
This function is called by Atheris for each fuzzing iteration.
The fuzzer will try to find inputs that cause crashes.
Args:
data: Bytes to test (generated by Atheris)
The waterfall vulnerability:
- check_secret() validates input character-by-character
- Each correct character creates a distinct code path
- Coverage-guided fuzzing progressively discovers the secret "FUZZINGLABS"
- When the complete secret is found, it crashes with SystemError
With atheris.instrument_imports(), the main module is instrumented
for coverage, allowing Atheris to detect when inputs reach new
code paths (each correct character).
"""
# Call the vulnerable function
# It will raise SystemError when the secret is fully discovered
check_secret(bytes(data))
if __name__ == "__main__":
"""
Standalone fuzzing mode.
Run directly: python fuzz_target.py
"""
print("=" * 60)
print("Atheris Fuzzing - Waterfall Vulnerability")
print("=" * 60)
print("Fuzzing will try to discover the secret string...")
print("Watch for progress indicators: [DEBUG] Progress: X/11")
print()
print("Press Ctrl+C to stop fuzzing")
print("=" * 60)
print()
# Setup Atheris with command-line args
atheris.Setup(sys.argv, TestOneInput)
# Start fuzzing
atheris.Fuzz()

View File

@@ -0,0 +1,117 @@
"""
Example application with a waterfall vulnerability.
This simulates a password checking system that validates character-by-character.
Each correct character creates a distinct code path, allowing coverage-guided
fuzzing to progressively discover the secret.
"""
SECRET = b"FUZZINGLABS" # Full secret to discover
def check_secret(input_data: bytes) -> int:
"""
Vulnerable function: checks secret character by character.
This is a classic waterfall/sequential comparison vulnerability.
Each correct character comparison creates a unique code path that
coverage-guided fuzzing can detect and use to guide input generation.
Real-world analogy:
- Timing attacks on password checkers
- Protocol state machines with sequential validation
- JWT signature verification vulnerabilities
Args:
input_data: Input bytes to check
Returns:
Number of matching characters (for instrumentation purposes)
Raises:
SystemError: When complete secret is discovered
"""
if not input_data:
return 0
# Check each character sequentially
# Each comparison creates a distinct code path for coverage guidance
matches = 0
for i in range(min(len(input_data), len(SECRET))):
if input_data[i] != SECRET[i]:
# Wrong character - stop checking
return matches
matches += 1
# Add explicit comparisons to help coverage-guided fuzzing
# Each comparison creates a distinct code path for Atheris to detect
if matches >= 1 and input_data[0] == ord('F'):
pass # F
if matches >= 2 and input_data[1] == ord('U'):
pass # FU
if matches >= 3 and input_data[2] == ord('Z'):
pass # FUZ
if matches >= 4 and input_data[3] == ord('Z'):
pass # FUZZ
if matches >= 5 and input_data[4] == ord('I'):
pass # FUZZI
if matches >= 6 and input_data[5] == ord('N'):
pass # FUZZIN
if matches >= 7 and input_data[6] == ord('G'):
pass # FUZZING
if matches >= 8 and input_data[7] == ord('L'):
pass # FUZZINGL
if matches >= 9 and input_data[8] == ord('A'):
pass # FUZZINGLA
if matches >= 10 and input_data[9] == ord('B'):
pass # FUZZINGLAB
if matches >= 11 and input_data[10] == ord('S'):
pass # FUZZINGLABS
# VULNERABILITY: Crashes when complete secret found
if matches == len(SECRET) and len(input_data) >= len(SECRET):
raise SystemError(f"SECRET COMPROMISED! Found: {input_data[:len(SECRET)]}")
return matches
def reset_state():
"""Reset the global state (kept for compatibility, but not used)"""
pass
if __name__ == "__main__":
"""Example usage showing the vulnerability"""
print("=" * 60)
print("Waterfall Vulnerability Demonstration")
print("=" * 60)
print(f"Secret: {SECRET}")
print(f"Secret length: {len(SECRET)} characters")
print()
# Test inputs showing progressive discovery
test_inputs = [
b"F", # First char correct
b"FU", # First two chars correct
b"FUZ", # First three chars correct
b"WRONG", # Wrong - no matches
b"FUZZINGLABS", # Complete secret - triggers crash!
]
for test in test_inputs:
print(f"Testing input: {test.decode(errors='ignore')!r}")
try:
matches = check_secret(test)
print(f" Result: {matches} characters matched out of {len(SECRET)}")
except SystemError as e:
print(f" 💥 CRASH: {e}")
print()
print("=" * 60)
print("To fuzz this vulnerability with FuzzForge:")
print(" ff init")
print(" ff workflow run atheris_fuzzing .")
print("=" * 60)

View File

@@ -0,0 +1,6 @@
[package]
name = "rust_fuzz_test"
version = "0.1.0"
edition = "2021"
[dependencies]

View File

@@ -0,0 +1,22 @@
# rust_fuzz_test
FuzzForge security testing project.
## Quick Start
```bash
# List available workflows
fuzzforge workflows
# Submit a workflow for analysis
fuzzforge workflow <workflow-name> /path/to/target
# View findings
fuzzforge finding <run-id>
```
## Project Structure
- `.fuzzforge/` - Project data and configuration
- `.fuzzforge/config.yaml` - Project configuration
- `.fuzzforge/findings.db` - Local database for runs and findings

View File

@@ -0,0 +1,5 @@
{
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json",
"runs": [],
"version": "2.1.0"
}

View File

@@ -0,0 +1,4 @@
target
corpus
artifacts
coverage

View File

@@ -0,0 +1,28 @@
[package]
name = "rust_fuzz_test-fuzz"
version = "0.0.0"
publish = false
edition = "2021"
[package.metadata]
cargo-fuzz = true
[dependencies]
libfuzzer-sys = "0.4"
[dependencies.rust_fuzz_test]
path = ".."
[[bin]]
name = "fuzz_target_1"
path = "fuzz_targets/fuzz_target_1.rs"
test = false
doc = false
bench = false
[[bin]]
name = "fuzz_divide"
path = "fuzz_targets/fuzz_divide.rs"
test = false
doc = false
bench = false

View File

@@ -0,0 +1,9 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use rust_fuzz_test::divide_numbers;
fuzz_target!(|data: &[u8]| {
// Fuzz the divide_numbers function which has division by zero
let _ = divide_numbers(data);
});

View File

@@ -0,0 +1,9 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use rust_fuzz_test::process_buffer;
fuzz_target!(|data: &[u8]| {
// Fuzz the process_buffer function which has bounds checking issues
let _ = process_buffer(data);
});

View File

@@ -0,0 +1,58 @@
/// Parse a simple integer from bytes
/// This function has a potential panic if the input is invalid
pub fn parse_number(data: &[u8]) -> i32 {
let s = std::str::from_utf8(data).expect("Invalid UTF-8");
s.parse::<i32>().expect("Invalid number")
}
/// Process a buffer with bounds checking issue
pub fn process_buffer(data: &[u8]) -> Vec<u8> {
if data.len() < 4 {
return Vec::new();
}
// Only crash when specific conditions are met (makes it harder to find)
if data[0] == b'F' && data[1] == b'U' && data[2] == b'Z' && data[3] == b'Z' {
// Potential panic: accessing index without proper bounds check
let size = data[4] as usize; // Will panic if data.len() == 4
let mut result = Vec::new();
// This could panic if size is larger than data.len()
for i in 4..4+size {
result.push(data[i]); // Will panic if i >= data.len()
}
return result;
}
Vec::new()
}
/// Divide two numbers parsed from input
pub fn divide_numbers(data: &[u8]) -> Option<i32> {
if data.len() < 2 {
return None;
}
let a = data[0] as i32;
let b = data[1] as i32;
// Potential division by zero
Some(a / b)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_number() {
assert_eq!(parse_number(b"123"), 123);
}
#[test]
fn test_process_buffer() {
let data = vec![3, 1, 2, 3, 4];
assert_eq!(process_buffer(&data), vec![3, 1, 2]);
}
}

View File

@@ -15,7 +15,6 @@ Test vulnerable application for FuzzForge security scanning.
Contains intentional security vulnerabilities for testing purposes.
"""
import os
import subprocess
import sqlite3

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1 @@
{}