mirror of https://github.com/FuzzingLabs/fuzzforge_ai.git synced 2026-02-12 20:32:46 +00:00

Files

Songbird99 02b877d23d Feature/litellm proxy (#27 )

* feat: seed governance config and responses routing

* Add env-configurable timeout for proxy providers

* Integrate LiteLLM OTEL collector and update docs

* Make .env.litellm optional for LiteLLM proxy

* Add LiteLLM proxy integration with model-agnostic virtual keys

Changes:
- Bootstrap generates 3 virtual keys with individual budgets (CLI: $100, Task-Agent: $25, Cognee: $50)
- Task-agent loads config at runtime via entrypoint script to wait for bootstrap completion
- All keys are model-agnostic by default (no LITELLM_DEFAULT_MODELS restrictions)
- Bootstrap handles database/env mismatch after docker prune by deleting stale aliases
- CLI and Cognee configured to use LiteLLM proxy with virtual keys
- Added comprehensive documentation in volumes/env/README.md

Technical details:
- task-agent entrypoint waits for keys in .env file before starting uvicorn
- Bootstrap creates/updates TASK_AGENT_API_KEY, COGNEE_API_KEY, and OPENAI_API_KEY
- Removed hardcoded API keys from docker-compose.yml
- All services route through http://localhost:10999 proxy

Generated with Claude Code https://claude.com/claude-code

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix CLI not loading virtual keys from global .env

Project .env files with empty OPENAI_API_KEY values were overriding
the global virtual keys. Updated _load_env_file_if_exists to only
override with non-empty values.

Generated with Claude Code https://claude.com/claude-code

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix agent executor not passing API key to LiteLLM

The agent was initializing LiteLlm without api_key or api_base,
causing authentication errors when using the LiteLLM proxy. Now
reads from OPENAI_API_KEY/LLM_API_KEY and LLM_ENDPOINT environment
variables and passes them to LiteLlm constructor.

Generated with Claude Code https://claude.com/claude-code

Co-Authored-By: Claude <noreply@anthropic.com>

* Auto-populate project .env with virtual key from global config

When running 'ff init', the command now checks for a global
volumes/env/.env file and automatically uses the OPENAI_API_KEY
virtual key if found. This ensures projects work with LiteLLM
proxy out of the box without manual key configuration.

Generated with Claude Code https://claude.com/claude-code

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: Update README with LiteLLM configuration instructions

Add note about LITELLM_GEMINI_API_KEY configuration and clarify that OPENAI_API_KEY default value should not be changed as it's used for the LLM proxy.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Refactor workflow parameters to use JSON Schema defaults

Consolidates parameter defaults into JSON Schema format, removing the separate default_parameters field. Adds extract_defaults_from_json_schema() helper to extract defaults from the standard schema structure. Updates LiteLLM proxy config to use LITELLM_OPENAI_API_KEY environment variable.

* Remove .env.example from task_agent

* Fix MDX syntax error in llm-proxy.md

* fix: apply default parameters from metadata.yaml automatically

Fixed TemporalManager.run_workflow() to correctly apply default parameter
values from workflow metadata.yaml files when parameters are not provided
by the caller.

Previous behavior:
- When workflow_params was empty {}, the condition
  `if workflow_params and 'parameters' in metadata` would fail
- Parameters would not be extracted from schema, resulting in workflows
  receiving only target_id with no other parameters

New behavior:
- Removed the `workflow_params and` requirement from the condition
- Now explicitly checks for defaults in parameter spec
- Applies defaults from metadata.yaml automatically when param not provided
- Workflows receive all parameters with proper fallback:
  provided value > metadata default > None

This makes metadata.yaml the single source of truth for parameter defaults,
removing the need for workflows to implement defensive default handling.

Affected workflows:
- llm_secret_detection (was failing with KeyError)
- All other workflows now benefit from automatic default application

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: tduhamel42 <tduhamel@fuzzinglabs.com>

2025-10-26 12:51:53 +01:00

6.6 KiB

Raw Permalink Blame History

Changelog

All notable changes to FuzzForge will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

🎯 Major Features

Android Static Analysis Workflow

Added comprehensive Android security testing workflow (android_static_analysis):
- Jadx decompiler for APK → Java source code decompilation
- OpenGrep/Semgrep static analysis with custom Android security rules
- MobSF integration for comprehensive mobile security scanning
- SARIF report generation with unified findings format
- Test results: Successfully decompiled 4,145 Java files, found 8 security vulnerabilities
- Full workflow completes in ~1.5 minutes

Platform-Aware Worker Architecture

ARM64 (Apple Silicon) support:
- Automatic platform detection (ARM64 vs x86_64) in CLI using platform.machine()
- Worker metadata convention (metadata.yaml) for platform-specific capabilities
- Multi-Dockerfile support: Dockerfile.amd64 (full toolchain) and Dockerfile.arm64 (optimized)
- Conditional module imports for graceful degradation (MobSF skips on ARM64)
- Backend path resolution via FUZZFORGE_HOST_ROOT for CLI worker management
Worker selection logic:
- CLI automatically selects appropriate Dockerfile based on detected platform
- Multi-strategy path resolution (API → .fuzzforge marker → environment variable)
- Platform-specific tool availability documented in metadata

Python SAST Workflow

Added Python Static Application Security Testing workflow (python_sast):
- Bandit for Python security linting (SAST)
- MyPy for static type checking
- Safety for dependency vulnerability scanning
- Integrated SARIF reporter for unified findings format
- Auto-start Python worker on-demand

✨ Enhancements

CI/CD Improvements

Added automated worker validation in CI pipeline
Docker build checks for all workers before merge
Worker file change detection for selective builds
Optimized Docker layer caching for faster builds
Dev branch testing workflow triggers

CLI Improvements

Fixed live monitoring bug in ff monitor live command
Enhanced ff findings command with better table formatting
Improved ff monitor with clearer status displays
Auto-start workers on-demand when workflows require them
Better error messages with actionable manual start commands

Worker Management

Standardized worker service names (worker-python, worker-android, etc.)
Added missing worker-secrets to repository
Improved worker naming consistency across codebase

LiteLLM Integration

Centralized LLM provider management with proxy
Governance and request/response routing
OTEL collector integration for observability
Environment-based configurable timeouts
Optional .env.litellm configuration

🐛 Bug Fixes

Fixed MobSF API key generation from secret file (SHA256 hash)
Corrected Temporal activity names (decompile_with_jadx, scan_with_opengrep, scan_with_mobsf)
Resolved linter errors across codebase
Fixed unused import issues to pass CI checks
Removed deprecated workflow parameters
Docker Compose version compatibility fixes

🔧 Technical Changes

Conditional import pattern for optional dependencies (MobSF on ARM64)
Multi-platform Dockerfile architecture
Worker metadata convention for capability declaration
Improved CI worker build optimization
Enhanced storage activity error handling

📝 Test Projects

Added test_projects/android_test/ with BeetleBug.apk and shopnest.apk
Android workflow validation with real APK samples
ARM64 platform testing and validation

0.7.0 - 2025-10-16

🎯 Major Features

Secret Detection Workflows

Added three secret detection workflows:
- gitleaks_detection - Pattern-based secret scanning
- trufflehog_detection - Entropy-based secret detection with verification
- llm_secret_detection - AI-powered semantic secret detection using LLMs
Comprehensive benchmarking infrastructure:
- 32-secret ground truth dataset for precision/recall testing
- Difficulty levels: 12 Easy, 10 Medium, 10 Hard secrets
- SARIF-formatted output for all workflows
- Achieved 100% recall with LLM-based detection on benchmark dataset

AI Module & Agent Integration

Added A2A (Agent-to-Agent) wrapper for multi-agent orchestration
Task agent implementation with Google ADK
LLM analysis workflow for code security analysis
Reactivated AI agent command (ff ai agent)

Temporal Migration Complete

Fully migrated from Prefect to Temporal for workflow orchestration
MinIO storage for unified file handling (replaces volume mounts)
Vertical workers with pre-built security toolchains
Improved worker lifecycle management

CI/CD Integration

Ephemeral deployment model for testing
Automated workflow validation in CI pipeline

✨ Enhancements

Documentation

Updated README for Temporal + MinIO architecture
Added .env configuration guide for AI agent API keys
Fixed worker startup instructions with correct service names
Updated docker compose commands to modern syntax

Worker Management

Added worker_service field to API responses for correct service naming
Improved error messages with actionable manual start commands
Fixed default parameters for gitleaks (now uses no_git=True by default)

🐛 Bug Fixes

Fixed default parameters from metadata.yaml not being applied to workflows when no parameters provided
Fixed gitleaks workflow failing on uploaded directories without Git history
Fixed worker startup command suggestions (now uses docker compose up -d with service names)
Fixed missing cognify_text method in CogneeProjectIntegration

🔧 Technical Changes

Updated all package versions to 0.7.0
Improved SARIF output formatting for secret detection workflows
Enhanced benchmark validation with ground truth JSON
Better integration between CLI and backend for worker management

📝 Test Projects

Added secret_detection_benchmark with 32 documented secrets
Ground truth JSON for automated precision/recall calculations
Updated vulnerable_app for comprehensive security testing

0.6.0 - 2024-12-XX

Features

Initial Temporal migration
Fuzzing workflows (Atheris, Cargo, OSS-Fuzz)
Security assessment workflow
Basic CLI commands

6.6 KiB Raw Permalink Blame History

Changelog

[Unreleased]

🎯 Major Features

Android Static Analysis Workflow

Platform-Aware Worker Architecture

Python SAST Workflow

✨ Enhancements

CI/CD Improvements

CLI Improvements

Worker Management

LiteLLM Integration

🐛 Bug Fixes

🔧 Technical Changes

📝 Test Projects

0.7.0 - 2025-10-16

🎯 Major Features

Secret Detection Workflows

AI Module & Agent Integration

Temporal Migration Complete

CI/CD Integration

✨ Enhancements

Documentation

Worker Management

🐛 Bug Fixes

🔧 Technical Changes

📝 Test Projects

0.6.0 - 2024-12-XX

Features

6.6 KiB

Raw Permalink Blame History