fuzzforge_ai/docs/blog/2025-01-16-v0.7.0-temporal-workers-release.md

FuzzForge v0.7.0: Temporal Orchestration & Vertical Workers Architecture

We're excited to announce FuzzForge v0.7.0, a major release featuring two significant improvements:

1. Architectural Foundation: Complete migration from Prefect to Temporal orchestration with vertical workers - pre-built containers for instant deployment
2. AI-Powered Secret Detection: New workflows achieving 84% recall on obfuscated secrets using LLM semantic analysis

This release transforms how security workflows are built, deployed, and scaled.

🚀 Flagship Features

Temporal Orchestration: Production-Ready Workflow Engine

We've fully migrated from Prefect to Temporal, bringing enterprise-grade workflow orchestration to FuzzForge:

Why Temporal?

• ✅ Reliability: Automatic retries, timeouts, and failure handling built-in
• ✅ Observability: World-class UI for monitoring workflow execution, logs, and debugging
• ✅ Scalability: Horizontal scaling across workers with intelligent load balancing
• ✅ Developer Experience: Type-safe workflows, versioning, and zero downtime deployments

What This Means for You:

# Start FuzzForge with Temporal
docker compose up -d

# Monitor workflows in real-time
open http://localhost:8080  # Temporal UI

# Submit workflows - everything just works
cd your_project/
ff workflow run security_assessment .

The Temporal UI gives you complete visibility into workflow execution:

• Live activity timelines
• Detailed logs for every step
• Retry history and failure analysis
• Performance metrics and bottleneck detection

Vertical Workers: Pre-Built Security Toolchains

FuzzForge now uses vertical workers - long-lived containers pre-built with security toolchains for different languages and platforms:

Worker	Toolchain	Status	Available Workflows
python	Gitleaks, TruffleHog, Atheris	✅ Production	Secret detection, security assessment
rust	cargo-fuzz	⚠️ Early Dev	Rust fuzzing
ossfuzz	OSS-Fuzz infrastructure	⚠️ Heavy Dev	Continuous fuzzing campaigns

Note: Additional workers (web, android, Go) are planned but not yet available.

Key Benefits:

1. Zero Build Time: Workflows start instantly - no container builds per workflow
2. Instant Code Changes: Modify workflow code, restart worker, done
3. Consistent Environment: Same toolchain versions across all runs
4. Resource Efficiency: Share workers across multiple concurrent workflows

Example: Running Secret Detection

# Worker is already running with Gitleaks, TruffleHog installed
ff workflow run gitleaks_detection .

# Behind the scenes:
# 1. CLI uploads project to MinIO
# 2. Temporal schedules on python-worker
# 3. Worker downloads from MinIO
# 4. Gitleaks runs (already installed!)
# 5. Results returned as SARIF

MinIO Storage: Unified File Handling

We've replaced volume mounts with MinIO (S3-compatible object storage):

Old Way (Volume Mounts):

# Had to mount directories, manage paths, cleanup manually
volumes:
  - ./my_project:/target

New Way (MinIO):

# CLI handles everything automatically
ff workflow run security_assessment .
# ✓ Creates tarball
# ✓ Uploads to MinIO
# ✓ Passes target_id to workflow
# ✓ Worker downloads and extracts
# ✓ Cleanup handled automatically

Benefits:

• ✅ No path conflicts or permissions issues
• ✅ Works seamlessly with remote Temporal clusters
• ✅ Automatic cleanup and caching
• ✅ Supports large targets (GB+)

🔍 AI-Powered Secret Detection: Also in v0.7.0

Alongside the architectural improvements, we're releasing a comprehensive secret detection system with three workflows:

Benchmark Results

We tested on a controlled dataset of 32 documented secrets (12 Easy, 10 Medium, 10 Hard):

Tool	Recall	Secrets Found	Speed	Best For
LLM (gpt-5-mini)	84.4%	41	618s	Obfuscated & hidden secrets
LLM (gpt-4o-mini)	56.2%	30	297s	Balanced speed/accuracy
Gitleaks	37.5%	12	5s	Fast pattern-based scanning
TruffleHog	0.0%	1	5s	Entropy analysis

📊 Full benchmark methodology and results →

Why LLM-Based Detection Wins

Obfuscated Secrets (Medium Difficulty):

# Gitleaks: ❌ Missed (no pattern match)
# LLM: ✅ Found (semantic understanding)
aws_key = base64.b64decode("QUtJQUlPU0ZPRE5ON0VYQU1QTEU=").decode()

Well-Hidden Secrets (Hard Difficulty):

# Gitleaks: ❌ Missed (no pattern)
# LLM: ✅ Found (understands XOR + join)
secret = ''.join(chr(ord(c) ^ 0x42) for c in "\x0b\x15\x04\x1b...")

Standard Secrets (Easy Difficulty):

# Both find these:
AWS_ACCESS_KEY = "AKIAIOSFODNN7EXAMPLE"

Try It Yourself

# Start FuzzForge
docker compose up -d

# Run secret detection on your code
cd your_project/
ff workflow run gitleaks_detection .      # Fast pattern-based
ff workflow run trufflehog_detection .    # Entropy analysis
ff workflow run llm_secret_detection .    # AI semantic analysis

# Get SARIF output
ff finding <run-id>

📊 Real-World Impact

Before v0.7.0 (Pattern-Only Detection):

• Found: Standard API keys, simple patterns
• Missed: Base64-encoded secrets, obfuscated credentials, split secrets

After v0.7.0 (LLM + Patterns):

• 84% recall on comprehensive benchmark
• Detects novel obfuscation techniques
• Understands code context (not just regex)
• Catches secrets in:
  • Base64/hex encoding
  • String concatenation
  • XOR/ROT13 obfuscation
  • Template strings
  • Binary literals

🔄 Migration Guide

What Changed

Docker Compose:

# Old (Prefect)
docker-compose up

# New (Temporal)
docker compose up -d

Workflow Submission:

# Old (volume mounts)
ff workflow run security_assessment --volume ./project

# New (automatic upload)
ff workflow run security_assessment .
# CLI handles upload automatically!

Worker Management:

# Old (per-workflow containers)
# Each workflow built its own container

# New (vertical workers)
docker compose up -d  # All workers start
# Workflows share workers - much faster!

Configuration

Set up AI workflows with API keys:

cp volumes/env/.env.example volumes/env/.env
# Edit .env and add your API keys (OpenAI, Anthropic, etc.)

Required for:

• llm_secret_detection workflow
• AI agent features (ff ai agent)

Basic security workflows (gitleaks, trufflehog, security_assessment) work without this.

🏗️ Architecture Overview

┌─────────────┐
│   User CLI  │ Upload → MinIO
└──────┬──────┘
       ↓ Submit
┌─────────────┐
│  Temporal   │ Schedule → Task Queue
└──────┬──────┘
       ↓ Execute
┌─────────────┐
│  Vertical   │ Download from MinIO → Run Tools → Upload Results
│  Workers    │
└─────────────┘
  rust, python, web, android, ossfuzz

Benefits:

• 🔄 Automatic retries and timeouts (Temporal)
• 📦 No file path management (MinIO)
• ⚡ Zero container build time (Vertical Workers)
• 📈 Horizontal scaling ready (Temporal + Workers)

🎯 Workflow Stability Status

✅ Stable & Production-Ready

• Secret Detection: gitleaks_detection, trufflehog_detection, llm_secret_detection
• Security Assessment: security_assessment
• Temporal orchestration with python worker
• MinIO file storage

⚠️ Early Development (Functional but not production-ready)

• Fuzzing workflows:
  • atheris_fuzzing - Python fuzzing with Atheris
  • cargo_fuzzing - Rust fuzzing with cargo-fuzz
• OSS-Fuzz integration: ossfuzz_campaign (under heavy active development)

Important: Fuzzing workflows are functional for testing but not recommended for production use yet.

📚 Resources

• 🌐 Website
• 📖 Documentation
• 💬 Discord Community
• 🎓 FuzzingLabs Academy
• 📊 Secret Detection Benchmarks

🙏 Acknowledgments

Special thanks to:

• Temporal for the amazing workflow engine
• Our community for feedback during the migration

🚀 Get Started

# Clone and install
git clone https://github.com/fuzzinglabs/fuzzforge_ai.git
cd fuzzforge_ai
uv tool install --python python3.12 .

# Start FuzzForge with Temporal
docker compose up -d

# Run your first workflow
cd test_projects/vulnerable_app/
fuzzforge init
ff workflow run security_assessment .

# Check Temporal UI
open http://localhost:8080

---

FuzzForge v0.7.0 is a foundational release that sets the stage for scalable, production-ready security automation. Try it today and let us know what you think!

Star us on GitHub ⭐