mirror of
https://github.com/FuzzingLabs/fuzzforge_ai.git
synced 2026-07-06 07:37:48 +02:00
docs: Apply global fixes for v0.7.0 Temporal architecture
- Replace docker-compose.temporal.yaml → docker-compose.yml - Replace Temporal UI port :8233 → :8080 - Replace repository URL fuzzforge.git → fuzzforge_ai.git Affected files: - concept/docker-containers.md - concept/resource-management.md - concept/sarif-format.md - how-to/create-workflow.md - how-to/debugging.md - how-to/troubleshooting.md - tutorial/getting-started.md
This commit is contained in:
tduhamel42
@@ -0,0 +1,321 @@
|
||||
# FuzzForge v0.7.0: Temporal Orchestration & Vertical Workers Architecture
|
||||
|
||||
We're excited to announce **FuzzForge v0.7.0**, a major release featuring two significant improvements:
|
||||
|
||||
1. **Architectural Foundation**: Complete migration from Prefect to **Temporal** orchestration with **vertical workers** - pre-built containers for instant deployment
|
||||
2. **AI-Powered Secret Detection**: New workflows achieving 84% recall on obfuscated secrets using LLM semantic analysis
|
||||
|
||||
This release transforms how security workflows are built, deployed, and scaled.
|
||||
|
||||
<!-- truncate -->
|
||||
|
||||
## 🚀 Flagship Features
|
||||
|
||||
### Temporal Orchestration: Production-Ready Workflow Engine
|
||||
|
||||
We've fully migrated from Prefect to [Temporal](https://temporal.io), bringing enterprise-grade workflow orchestration to FuzzForge:
|
||||
|
||||
**Why Temporal?**
|
||||
|
||||
- ✅ **Reliability**: Automatic retries, timeouts, and failure handling built-in
|
||||
- ✅ **Observability**: World-class UI for monitoring workflow execution, logs, and debugging
|
||||
- ✅ **Scalability**: Horizontal scaling across workers with intelligent load balancing
|
||||
- ✅ **Developer Experience**: Type-safe workflows, versioning, and zero downtime deployments
|
||||
|
||||
**What This Means for You:**
|
||||
|
||||
```bash
|
||||
# Start FuzzForge with Temporal
|
||||
docker compose up -d
|
||||
|
||||
# Monitor workflows in real-time
|
||||
open http://localhost:8080 # Temporal UI
|
||||
|
||||
# Submit workflows - everything just works
|
||||
cd your_project/
|
||||
ff workflow run security_assessment .
|
||||
```
|
||||
|
||||
The Temporal UI gives you complete visibility into workflow execution:
|
||||
|
||||
- Live activity timelines
|
||||
- Detailed logs for every step
|
||||
- Retry history and failure analysis
|
||||
- Performance metrics and bottleneck detection
|
||||
|
||||
### Vertical Workers: Pre-Built Security Toolchains
|
||||
|
||||
FuzzForge now uses **vertical workers** - long-lived containers pre-built with security toolchains for different languages and platforms:
|
||||
|
||||
| Worker | Toolchain | Status | Available Workflows |
|
||||
| ----------- | ----------------------------- | ------------- | ------------------------------------- |
|
||||
| **python** | Gitleaks, TruffleHog, Atheris | ✅ Production | Secret detection, security assessment |
|
||||
| **rust** | cargo-fuzz | ⚠️ Early Dev | Rust fuzzing |
|
||||
| **ossfuzz** | OSS-Fuzz infrastructure | ⚠️ Heavy Dev | Continuous fuzzing campaigns |
|
||||
|
||||
**Note:** Additional workers (web, android, Go) are planned but not yet available.
|
||||
|
||||
**Key Benefits:**
|
||||
|
||||
1. **Zero Build Time**: Workflows start instantly - no container builds per workflow
|
||||
2. **Instant Code Changes**: Modify workflow code, restart worker, done
|
||||
3. **Consistent Environment**: Same toolchain versions across all runs
|
||||
4. **Resource Efficiency**: Share workers across multiple concurrent workflows
|
||||
|
||||
**Example: Running Secret Detection**
|
||||
|
||||
```bash
|
||||
# Worker is already running with Gitleaks, TruffleHog installed
|
||||
ff workflow run gitleaks_detection .
|
||||
|
||||
# Behind the scenes:
|
||||
# 1. CLI uploads project to MinIO
|
||||
# 2. Temporal schedules on python-worker
|
||||
# 3. Worker downloads from MinIO
|
||||
# 4. Gitleaks runs (already installed!)
|
||||
# 5. Results returned as SARIF
|
||||
```
|
||||
|
||||
### MinIO Storage: Unified File Handling
|
||||
|
||||
We've replaced volume mounts with **MinIO** (S3-compatible object storage):
|
||||
|
||||
**Old Way (Volume Mounts):**
|
||||
|
||||
```yaml
|
||||
# Had to mount directories, manage paths, cleanup manually
|
||||
volumes:
|
||||
- ./my_project:/target
|
||||
```
|
||||
|
||||
**New Way (MinIO):**
|
||||
|
||||
```bash
|
||||
# CLI handles everything automatically
|
||||
ff workflow run security_assessment .
|
||||
# ✓ Creates tarball
|
||||
# ✓ Uploads to MinIO
|
||||
# ✓ Passes target_id to workflow
|
||||
# ✓ Worker downloads and extracts
|
||||
# ✓ Cleanup handled automatically
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
|
||||
- ✅ No path conflicts or permissions issues
|
||||
- ✅ Works seamlessly with remote Temporal clusters
|
||||
- ✅ Automatic cleanup and caching
|
||||
- ✅ Supports large targets (GB+)
|
||||
|
||||
## 🔍 AI-Powered Secret Detection: Also in v0.7.0
|
||||
|
||||
Alongside the architectural improvements, we're releasing a comprehensive **secret detection** system with three workflows:
|
||||
|
||||
### Benchmark Results
|
||||
|
||||
We tested on a controlled dataset of **32 documented secrets** (12 Easy, 10 Medium, 10 Hard):
|
||||
|
||||
| Tool | Recall | Secrets Found | Speed | Best For |
|
||||
| --------------------- | --------- | ------------- | ----- | --------------------------- |
|
||||
| **LLM (gpt-5-mini)** | **84.4%** | 41 | 618s | Obfuscated & hidden secrets |
|
||||
| **LLM (gpt-4o-mini)** | 56.2% | 30 | 297s | Balanced speed/accuracy |
|
||||
| **Gitleaks** | 37.5% | 12 | 5s | Fast pattern-based scanning |
|
||||
| **TruffleHog** | 0.0% | 1 | 5s | Entropy analysis |
|
||||
|
||||
📊 [Full benchmark methodology and results →](https://github.com/FuzzingLabs/fuzzforge_ai/blob/dev/backend/benchmarks/by_category/secret_detection/results/comparison_report.md)
|
||||
|
||||
### Why LLM-Based Detection Wins
|
||||
|
||||
**Obfuscated Secrets (Medium Difficulty):**
|
||||
|
||||
```python
|
||||
# Gitleaks: ❌ Missed (no pattern match)
|
||||
# LLM: ✅ Found (semantic understanding)
|
||||
aws_key = base64.b64decode("QUtJQUlPU0ZPRE5ON0VYQU1QTEU=").decode()
|
||||
```
|
||||
|
||||
**Well-Hidden Secrets (Hard Difficulty):**
|
||||
|
||||
```python
|
||||
# Gitleaks: ❌ Missed (no pattern)
|
||||
# LLM: ✅ Found (understands XOR + join)
|
||||
secret = ''.join(chr(ord(c) ^ 0x42) for c in "\x0b\x15\x04\x1b...")
|
||||
```
|
||||
|
||||
**Standard Secrets (Easy Difficulty):**
|
||||
|
||||
```python
|
||||
# Both find these:
|
||||
AWS_ACCESS_KEY = "AKIAIOSFODNN7EXAMPLE"
|
||||
```
|
||||
|
||||
### Try It Yourself
|
||||
|
||||
```bash
|
||||
# Start FuzzForge
|
||||
docker compose up -d
|
||||
|
||||
# Run secret detection on your code
|
||||
cd your_project/
|
||||
ff workflow run gitleaks_detection . # Fast pattern-based
|
||||
ff workflow run trufflehog_detection . # Entropy analysis
|
||||
ff workflow run llm_secret_detection . # AI semantic analysis
|
||||
|
||||
# Get SARIF output
|
||||
ff finding <run-id>
|
||||
```
|
||||
|
||||
## 📊 Real-World Impact
|
||||
|
||||
**Before v0.7.0 (Pattern-Only Detection):**
|
||||
|
||||
- Found: Standard API keys, simple patterns
|
||||
- Missed: Base64-encoded secrets, obfuscated credentials, split secrets
|
||||
|
||||
**After v0.7.0 (LLM + Patterns):**
|
||||
|
||||
- **84% recall** on comprehensive benchmark
|
||||
- Detects novel obfuscation techniques
|
||||
- Understands code context (not just regex)
|
||||
- Catches secrets in:
|
||||
- Base64/hex encoding
|
||||
- String concatenation
|
||||
- XOR/ROT13 obfuscation
|
||||
- Template strings
|
||||
- Binary literals
|
||||
|
||||
## 🔄 Migration Guide
|
||||
|
||||
### What Changed
|
||||
|
||||
**Docker Compose:**
|
||||
|
||||
```bash
|
||||
# Old (Prefect)
|
||||
docker-compose up
|
||||
|
||||
# New (Temporal)
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
**Workflow Submission:**
|
||||
|
||||
```bash
|
||||
# Old (volume mounts)
|
||||
ff workflow run security_assessment --volume ./project
|
||||
|
||||
# New (automatic upload)
|
||||
ff workflow run security_assessment .
|
||||
# CLI handles upload automatically!
|
||||
```
|
||||
|
||||
**Worker Management:**
|
||||
|
||||
```bash
|
||||
# Old (per-workflow containers)
|
||||
# Each workflow built its own container
|
||||
|
||||
# New (vertical workers)
|
||||
docker compose up -d # All workers start
|
||||
# Workflows share workers - much faster!
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Set up AI workflows with API keys:
|
||||
|
||||
```bash
|
||||
cp volumes/env/.env.example volumes/env/.env
|
||||
# Edit .env and add your API keys (OpenAI, Anthropic, etc.)
|
||||
```
|
||||
|
||||
Required for:
|
||||
|
||||
- `llm_secret_detection` workflow
|
||||
- AI agent features (`ff ai agent`)
|
||||
|
||||
Basic security workflows (gitleaks, trufflehog, security_assessment) work without this.
|
||||
|
||||
## 🏗️ Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────┐
|
||||
│ User CLI │ Upload → MinIO
|
||||
└──────┬──────┘
|
||||
↓ Submit
|
||||
┌─────────────┐
|
||||
│ Temporal │ Schedule → Task Queue
|
||||
└──────┬──────┘
|
||||
↓ Execute
|
||||
┌─────────────┐
|
||||
│ Vertical │ Download from MinIO → Run Tools → Upload Results
|
||||
│ Workers │
|
||||
└─────────────┘
|
||||
rust, python, web, android, ossfuzz
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
|
||||
- 🔄 Automatic retries and timeouts (Temporal)
|
||||
- 📦 No file path management (MinIO)
|
||||
- ⚡ Zero container build time (Vertical Workers)
|
||||
- 📈 Horizontal scaling ready (Temporal + Workers)
|
||||
|
||||
## 🎯 Workflow Stability Status
|
||||
|
||||
### ✅ Stable & Production-Ready
|
||||
|
||||
- **Secret Detection**: `gitleaks_detection`, `trufflehog_detection`, `llm_secret_detection`
|
||||
- **Security Assessment**: `security_assessment`
|
||||
- Temporal orchestration with python worker
|
||||
- MinIO file storage
|
||||
|
||||
### ⚠️ Early Development (Functional but not production-ready)
|
||||
|
||||
- **Fuzzing workflows**:
|
||||
- `atheris_fuzzing` - Python fuzzing with Atheris
|
||||
- `cargo_fuzzing` - Rust fuzzing with cargo-fuzz
|
||||
- **OSS-Fuzz integration**: `ossfuzz_campaign` (under heavy active development)
|
||||
|
||||
**Important:** Fuzzing workflows are functional for testing but not recommended for production use yet.
|
||||
|
||||
## 📚 Resources
|
||||
|
||||
- 🌐 [Website](https://fuzzforge.ai)
|
||||
- 📖 [Documentation](https://docs.fuzzforge.ai)
|
||||
- 💬 [Discord Community](https://discord.gg/8XEX33UUwZ)
|
||||
- 🎓 [FuzzingLabs Academy](https://academy.fuzzinglabs.com/?coupon=GITHUB_FUZZFORGE)
|
||||
- 📊 [Secret Detection Benchmarks](https://github.com/FuzzingLabs/fuzzforge_ai/blob/dev/backend/benchmarks/by_category/secret_detection/results/comparison_report.md)
|
||||
|
||||
## 🙏 Acknowledgments
|
||||
|
||||
Special thanks to:
|
||||
|
||||
- [Temporal](https://temporal.io) for the amazing workflow engine
|
||||
- Our community for feedback during the migration
|
||||
|
||||
## 🚀 Get Started
|
||||
|
||||
```bash
|
||||
# Clone and install
|
||||
git clone https://github.com/fuzzinglabs/fuzzforge_ai.git
|
||||
cd fuzzforge_ai
|
||||
uv tool install --python python3.12 .
|
||||
|
||||
# Start FuzzForge with Temporal
|
||||
docker compose up -d
|
||||
|
||||
# Run your first workflow
|
||||
cd test_projects/vulnerable_app/
|
||||
fuzzforge init
|
||||
ff workflow run security_assessment .
|
||||
|
||||
# Check Temporal UI
|
||||
open http://localhost:8080
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**FuzzForge v0.7.0** is a foundational release that sets the stage for scalable, production-ready security automation. Try it today and let us know what you think!
|
||||
|
||||
Star us on [GitHub](https://github.com/FuzzingLabs/fuzzforge_ai) ⭐
|
||||
Reference in New Issue
Block a user