mirror of
https://github.com/FuzzingLabs/fuzzforge_ai.git
synced 2026-10-12 03:37:17 +02:00
feat: Complete migration from Prefect to Temporal
BREAKING CHANGE: Replaces Prefect workflow orchestration with Temporal ## Major Changes - Replace Prefect with Temporal for workflow orchestration - Implement vertical worker architecture (rust, android) - Replace Docker registry with MinIO for unified storage - Refactor activities to be co-located with workflows - Update all API endpoints for Temporal compatibility ## Infrastructure - New: docker-compose.temporal.yaml (Temporal + MinIO + workers) - New: workers/ directory with rust and android vertical workers - New: backend/src/temporal/ (manager, discovery) - New: backend/src/storage/ (S3-cached storage with MinIO) - New: backend/toolbox/common/ (shared storage activities) - Deleted: docker-compose.yaml (old Prefect setup) - Deleted: backend/src/core/prefect_manager.py - Deleted: backend/src/services/prefect_stats_monitor.py - Deleted: Docker registry and insecure-registries requirement ## Workflows - Migrated: security_assessment workflow to Temporal - New: rust_test workflow (example/test workflow) - Deleted: secret_detection_scan (Prefect-based, to be reimplemented) - Activities now co-located with workflows for independent testing ## API Changes - Updated: backend/src/api/workflows.py (Temporal submission) - Updated: backend/src/api/runs.py (Temporal status/results) - Updated: backend/src/main.py (727 lines, TemporalManager integration) - Updated: All 16 MCP tools to use TemporalManager ## Testing - ✅ All services healthy (Temporal, PostgreSQL, MinIO, workers, backend) - ✅ All API endpoints functional - ✅ End-to-end workflow test passed (72 findings from vulnerable_app) - ✅ MinIO storage integration working (target upload/download, results) - ✅ Worker activity discovery working (6 activities registered) - ✅ Tarball extraction working - ✅ SARIF report generation working ## Documentation - ARCHITECTURE.md: Complete Temporal architecture documentation - QUICKSTART_TEMPORAL.md: Getting started guide - MIGRATION_DECISION.md: Why we chose Temporal over Prefect - IMPLEMENTATION_STATUS.md: Migration progress tracking - workers/README.md: Worker development guide ## Dependencies - Added: temporalio>=1.6.0 - Added: boto3>=1.34.0 (MinIO S3 client) - Removed: prefect>=3.4.18
This commit is contained in:
1 parent
987c49569c
commit
0680f14df6
48 files changed
+7504
-3101
No files matched your search
@@ -0,0 +1,85 @@
|
||||
# FuzzForge Vertical Worker: Rust/Native Security
|
||||
#
|
||||
# Pre-installed tools for Rust and native binary security analysis:
|
||||
# - Rust toolchain (rustc, cargo)
|
||||
# - AFL++ (fuzzing)
|
||||
# - cargo-fuzz (Rust fuzzing)
|
||||
# - gdb (debugging)
|
||||
# - valgrind (memory analysis)
|
||||
# - AddressSanitizer/MemorySanitizer support
|
||||
# - Common reverse engineering tools
|
||||
|
||||
FROM rust:1.83-slim-bookworm
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Install system dependencies
|
||||
RUN apt-get update && apt-get install -y \
|
||||
# Build essentials
|
||||
build-essential \
|
||||
cmake \
|
||||
git \
|
||||
curl \
|
||||
wget \
|
||||
pkg-config \
|
||||
libssl-dev \
|
||||
# AFL++ dependencies
|
||||
clang \
|
||||
llvm \
|
||||
# Debugging and analysis tools
|
||||
gdb \
|
||||
valgrind \
|
||||
strace \
|
||||
# Binary analysis (binutils includes objdump, readelf, etc.)
|
||||
binutils \
|
||||
# Network tools
|
||||
netcat-openbsd \
|
||||
tcpdump \
|
||||
# Python for Temporal worker
|
||||
python3 \
|
||||
python3-pip \
|
||||
python3-venv \
|
||||
# Cleanup
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install AFL++
|
||||
RUN git clone https://github.com/AFLplusplus/AFLplusplus /tmp/aflplusplus && \
|
||||
cd /tmp/aflplusplus && \
|
||||
make all && \
|
||||
make install && \
|
||||
cd / && \
|
||||
rm -rf /tmp/aflplusplus
|
||||
|
||||
# Install Rust toolchain components
|
||||
RUN rustup component add rustfmt clippy && \
|
||||
rustup target add x86_64-unknown-linux-musl
|
||||
|
||||
# Install cargo-fuzz and other Rust security tools
|
||||
RUN cargo install --locked \
|
||||
cargo-fuzz \
|
||||
cargo-audit \
|
||||
cargo-outdated \
|
||||
cargo-tree
|
||||
|
||||
# Install Python dependencies for Temporal worker
|
||||
COPY requirements.txt /tmp/requirements.txt
|
||||
RUN pip3 install --break-system-packages --no-cache-dir -r /tmp/requirements.txt && \
|
||||
rm /tmp/requirements.txt
|
||||
|
||||
# Create cache directory for downloaded targets
|
||||
RUN mkdir -p /cache && chmod 755 /cache
|
||||
|
||||
# Copy worker entrypoint
|
||||
COPY worker.py /app/worker.py
|
||||
|
||||
# Add toolbox to Python path (mounted at runtime)
|
||||
ENV PYTHONPATH="/app:/app/toolbox:${PYTHONPATH}"
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
# Healthcheck
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
|
||||
CMD python3 -c "import sys; sys.exit(0)"
|
||||
|
||||
# Run worker
|
||||
CMD ["python3", "/app/worker.py"]
|
||||
Reference in new issue
Block a user