mirror of
https://github.com/FuzzingLabs/fuzzforge_ai.git
synced 2026-05-19 10:48:04 +02:00
Release v0.7.3 - Android workflows, LiteLLM integration, ARM64 support (#32)
* ci: add worker validation and Docker build checks Add automated validation to prevent worker-related issues: **Worker Validation Script:** - New script: .github/scripts/validate-workers.sh - Validates all workers in docker-compose.yml exist - Checks required files: Dockerfile, requirements.txt, worker.py - Verifies files are tracked by git (not gitignored) - Detects gitignore issues that could hide workers **CI Workflow Updates:** - Added validate-workers job (runs on every PR) - Added build-workers job (runs if workers/ modified) - Uses Docker Buildx for caching - Validates Docker images build successfully - Updated test-summary to check validation results **PR Template:** - New pull request template with comprehensive checklist - Specific section for worker-related changes - Reminds contributors to validate worker files - Includes documentation and changelog reminders These checks would have caught the secrets worker gitignore issue. Implements Phase 1 improvements from CI/CD quality assessment. * fix: add dev branch to test workflow triggers The test workflow was configured for 'develop' but the actual branch is named 'dev'. This caused tests not to run on PRs to dev branch. Now tests will run on: - PRs to: main, master, dev, develop - Pushes to: main, master, dev, develop, feature/** * fix: properly detect worker file changes in CI The previous condition used invalid GitHub context field. Now uses git diff to properly detect changes to workers/ or docker-compose.yml. Behavior: - Job always runs the check step - Detects if workers/ or docker-compose.yml modified - Only builds Docker images if workers actually changed - Shows clear skip message when no worker changes detected * feat: Add Python SAST workflow with three security analysis tools Implements Issue #5 - Python SAST workflow that combines: - Dependency scanning (pip-audit) for CVE detection - Security linting (Bandit) for vulnerability patterns - Type checking (Mypy) for type safety issues ## Changes **New Modules:** - `DependencyScanner`: Scans Python dependencies for known CVEs using pip-audit - `BanditAnalyzer`: Analyzes Python code for security issues using Bandit - `MypyAnalyzer`: Checks Python code for type safety issues using Mypy **New Workflow:** - `python_sast`: Temporal workflow that orchestrates all three SAST tools - Runs tools in parallel for fast feedback (3-5 min vs hours for fuzzing) - Generates unified SARIF report with findings from all tools - Supports configurable severity/confidence thresholds **Updates:** - Added SAST dependencies to Python worker (bandit, pip-audit, mypy) - Updated module __init__.py files to export new analyzers - Added type_errors.py test file to vulnerable_app for Mypy validation ## Testing Workflow tested successfully on vulnerable_app: - ✅ Bandit: Detected 9 security issues (command injection, unsafe functions) - ✅ Mypy: Detected 5 type errors - ✅ DependencyScanner: Ran successfully (no CVEs in test dependencies) - ✅ SARIF export: Generated valid SARIF with 14 total findings * fix: Remove unused imports to pass linter * fix: resolve live monitoring bug, remove deprecated parameters, and auto-start Python worker - Fix live monitoring style error by calling _live_monitor() helper directly - Remove default_parameters duplication from 10 workflow metadata files - Remove deprecated volume_mode parameter from 26 files across CLI, SDK, backend, and docs - Configure Python worker to start automatically with docker compose up - Clean up constants, validation, completion, and example files Fixes # - Live monitoring now works correctly with --live flag - Workflow metadata follows JSON Schema standard - Cleaner codebase without deprecated volume_mode - Python worker (most commonly used) starts by default * fix: resolve linter errors and optimize CI worker builds - Remove unused Literal import from backend findings model - Remove unnecessary f-string prefixes in CLI findings command - Optimize GitHub Actions to build only modified workers - Detect specific worker changes (python, secrets, rust, android, ossfuzz) - Build only changed workers instead of all 5 - Build all workers if docker-compose.yml changes - Significantly reduces CI build time * feat: Add Android static analysis workflow with Jadx, OpenGrep, and MobSF Comprehensive Android security testing workflow converted from Prefect to Temporal architecture: Modules (3): - JadxDecompiler: APK to Java source code decompilation - OpenGrepAndroid: Static analysis with Android-specific security rules - MobSFScanner: Comprehensive mobile security framework integration Custom Rules (13): - clipboard-sensitive-data, hardcoded-secrets, insecure-data-storage - insecure-deeplink, insecure-logging, intent-redirection - sensitive_data_sharedPreferences, sqlite-injection - vulnerable-activity, vulnerable-content-provider, vulnerable-service - webview-javascript-enabled, webview-load-arbitrary-url Workflow: - 6-phase Temporal workflow: download → Jadx → OpenGrep → MobSF → SARIF → upload - 4 activities: decompile_with_jadx, scan_with_opengrep, scan_with_mobsf, generate_android_sarif - SARIF output combining findings from all security tools Docker Worker: - ARM64 Mac compatibility via amd64 platform emulation - Pre-installed: Android SDK, Jadx 1.4.7, OpenGrep 1.45.0, MobSF 3.9.7 - MobSF runs as background service with API key auto-generation - Added aiohttp for async HTTP communication Test APKs: - BeetleBug.apk and shopnest.apk for workflow validation * fix(android): correct activity names and MobSF API key generation - Fix activity names in workflow.py (get_target, upload_results, cleanup_cache) - Fix MobSF API key generation in Dockerfile startup script (cut delimiter) - Update activity parameter signatures to match actual implementations - Workflow now executes successfully with Jadx and OpenGrep * feat: add platform-aware worker architecture with ARM64 support Implement platform-specific Dockerfile selection and graceful tool degradation to support both x86_64 and ARM64 (Apple Silicon) platforms. **Backend Changes:** - Add system info API endpoint (/system/info) exposing host filesystem paths - Add FUZZFORGE_HOST_ROOT environment variable to backend service - Add graceful degradation in MobSF activity for ARM64 platforms **CLI Changes:** - Implement multi-strategy path resolution (backend API, .fuzzforge marker, env var) - Add platform detection (linux/amd64 vs linux/arm64) - Add worker metadata.yaml reading for platform capabilities - Auto-select appropriate Dockerfile based on detected platform - Pass platform-specific env vars to docker-compose **Worker Changes:** - Create workers/android/metadata.yaml defining platform capabilities - Rename Dockerfile -> Dockerfile.amd64 (full toolchain with MobSF) - Create Dockerfile.arm64 (excludes MobSF due to Rosetta 2 incompatibility) - Update docker-compose.yml to use ${ANDROID_DOCKERFILE} variable **Workflow Changes:** - Handle MobSF "skipped" status gracefully in workflow - Log clear warnings when tools are unavailable on platform **Key Features:** - Automatic platform detection and Dockerfile selection - Graceful degradation when tools unavailable (MobSF on ARM64) - Works from any directory (backend API provides paths) - Manual override via environment variables - Clear user feedback about platform and selected Dockerfile **Benefits:** - Android workflow now works on Apple Silicon Macs - No code changes needed for other workflows - Convention established for future platform-specific workers Closes: MobSF Rosetta 2 incompatibility issue Implements: Platform-aware worker architecture (Option B) * fix: make MobSFScanner import conditional for ARM64 compatibility - Add try-except block to conditionally import MobSFScanner in modules/android/__init__.py - Allows Android worker to start on ARM64 without MobSF dependencies (aiohttp) - MobSF activity gracefully skips on ARM64 with clear warning message - Remove workflow path detection logic (not needed - workflows receive directories) Platform-aware architecture fully functional on ARM64: - CLI detects ARM64 and selects Dockerfile.arm64 automatically - Worker builds and runs without MobSF on ARM64 - Jadx successfully decompiles APKs (4145 files from BeetleBug.apk) - OpenGrep finds security vulnerabilities (8 issues found) - MobSF gracefully skips with warning on ARM64 - Graceful degradation working as designed Tested with: ff workflow run android_static_analysis test_projects/android_test/ \ --wait --no-interactive apk_path=BeetleBug.apk decompile_apk=true Results: 8 security findings (1 ERROR, 7 WARNINGS) * docs: update CHANGELOG with Android workflow and ARM64 support Added [Unreleased] section documenting: - Android Static Analysis Workflow (Jadx, OpenGrep, MobSF) - Platform-Aware Worker Architecture with ARM64 support - Python SAST Workflow - CI/CD improvements and worker validation - CLI enhancements - Bug fixes and technical changes Fixed date typo: 2025-01-16 → 2025-10-16 * fix: resolve linter errors in Android modules - Remove unused imports from mobsf_scanner.py (asyncio, hashlib, json, Optional) - Remove unused variables from opengrep_android.py (start_col, end_col) - Remove duplicate Path import from workflow.py * ci: support multi-platform Dockerfiles in worker validation Updated worker validation script to accept both: - Single Dockerfile pattern (existing workers) - Multi-platform Dockerfile pattern (Dockerfile.amd64, Dockerfile.arm64, etc.) This enables platform-aware worker architectures like the Android worker which uses different Dockerfiles for x86_64 and ARM64 platforms. * Feature/litellm proxy (#27) * feat: seed governance config and responses routing * Add env-configurable timeout for proxy providers * Integrate LiteLLM OTEL collector and update docs * Make .env.litellm optional for LiteLLM proxy * Add LiteLLM proxy integration with model-agnostic virtual keys Changes: - Bootstrap generates 3 virtual keys with individual budgets (CLI: $100, Task-Agent: $25, Cognee: $50) - Task-agent loads config at runtime via entrypoint script to wait for bootstrap completion - All keys are model-agnostic by default (no LITELLM_DEFAULT_MODELS restrictions) - Bootstrap handles database/env mismatch after docker prune by deleting stale aliases - CLI and Cognee configured to use LiteLLM proxy with virtual keys - Added comprehensive documentation in volumes/env/README.md Technical details: - task-agent entrypoint waits for keys in .env file before starting uvicorn - Bootstrap creates/updates TASK_AGENT_API_KEY, COGNEE_API_KEY, and OPENAI_API_KEY - Removed hardcoded API keys from docker-compose.yml - All services route through http://localhost:10999 proxy * Fix CLI not loading virtual keys from global .env Project .env files with empty OPENAI_API_KEY values were overriding the global virtual keys. Updated _load_env_file_if_exists to only override with non-empty values. * Fix agent executor not passing API key to LiteLLM The agent was initializing LiteLlm without api_key or api_base, causing authentication errors when using the LiteLLM proxy. Now reads from OPENAI_API_KEY/LLM_API_KEY and LLM_ENDPOINT environment variables and passes them to LiteLlm constructor. * Auto-populate project .env with virtual key from global config When running 'ff init', the command now checks for a global volumes/env/.env file and automatically uses the OPENAI_API_KEY virtual key if found. This ensures projects work with LiteLLM proxy out of the box without manual key configuration. * docs: Update README with LiteLLM configuration instructions Add note about LITELLM_GEMINI_API_KEY configuration and clarify that OPENAI_API_KEY default value should not be changed as it's used for the LLM proxy. * Refactor workflow parameters to use JSON Schema defaults Consolidates parameter defaults into JSON Schema format, removing the separate default_parameters field. Adds extract_defaults_from_json_schema() helper to extract defaults from the standard schema structure. Updates LiteLLM proxy config to use LITELLM_OPENAI_API_KEY environment variable. * Remove .env.example from task_agent * Fix MDX syntax error in llm-proxy.md * fix: apply default parameters from metadata.yaml automatically Fixed TemporalManager.run_workflow() to correctly apply default parameter values from workflow metadata.yaml files when parameters are not provided by the caller. Previous behavior: - When workflow_params was empty {}, the condition `if workflow_params and 'parameters' in metadata` would fail - Parameters would not be extracted from schema, resulting in workflows receiving only target_id with no other parameters New behavior: - Removed the `workflow_params and` requirement from the condition - Now explicitly checks for defaults in parameter spec - Applies defaults from metadata.yaml automatically when param not provided - Workflows receive all parameters with proper fallback: provided value > metadata default > None This makes metadata.yaml the single source of truth for parameter defaults, removing the need for workflows to implement defensive default handling. Affected workflows: - llm_secret_detection (was failing with KeyError) - All other workflows now benefit from automatic default application Co-authored-by: tduhamel42 <tduhamel@fuzzinglabs.com> * fix: add default values to llm_analysis workflow parameters Resolves validation error where agent_url was None when not explicitly provided. The TemporalManager applies defaults from metadata.yaml, not from module input schemas, so all parameters need defaults in the workflow metadata. Changes: - Add default agent_url, llm_model (gpt-5-mini), llm_provider (openai) - Expand file_patterns to 45 comprehensive patterns covering code, configs, secrets, and Docker files - Increase default limits: max_files (10), max_file_size (100KB), timeout (90s) * refactor: replace .env.example with .env.template in documentation - Remove volumes/env/.env.example file - Update all documentation references to use .env.template instead - Update bootstrap script error message - Update .gitignore comment * feat(cli): add worker management commands with improved progress feedback Add comprehensive CLI commands for managing Temporal workers: - ff worker list - List workers with status and uptime - ff worker start <name> - Start specific worker with optional rebuild - ff worker stop - Safely stop all workers without affecting core services Improvements: - Live progress display during worker startup with Rich Status spinner - Real-time elapsed time counter and container state updates - Health check status tracking (starting → unhealthy → healthy) - Helpful contextual hints at 10s, 30s, 60s intervals - Better timeout messages showing last known state Worker management enhancements: - Use 'docker compose' (space) instead of 'docker-compose' (hyphen) - Stop workers individually with 'docker stop' to avoid stopping core services - Platform detection and Dockerfile selection (ARM64/AMD64) Documentation: - Updated docker-setup.md with CLI commands as primary method - Created comprehensive cli-reference.md with all commands and examples - Added worker management best practices * fix: MobSF scanner now properly parses files dict structure MobSF returns 'files' as a dict (not list): {"filename": "line_numbers"} The parser was treating it as a list, causing zero findings to be extracted. Now properly iterates over the dict and creates one finding per affected file with correct line numbers and metadata (CWE, OWASP, MASVS, CVSS). Fixed in both code_analysis and behaviour sections. * chore: bump version to 0.7.3 * docs: fix broken documentation links in cli-reference * chore: add worker startup documentation and cleanup .gitignore - Add workflow-to-worker mapping tables across documentation - Update troubleshooting guide with worker requirements section - Enhance getting started guide with worker examples - Add quick reference to docker setup guide - Add WEEK_SUMMARY*.md pattern to .gitignore * docs: update CHANGELOG with missing versions and recent changes - Add Unreleased section for post-v0.7.3 documentation updates - Add v0.7.2 entry with bug fixes and worker improvements - Document that v0.7.1 was re-tagged as v0.7.2 - Fix v0.6.0 date to "Undocumented" (no tag exists) - Add version comparison links for easier navigation * chore: bump all package versions to 0.7.3 for consistency * Update GitHub link to fuzzforge_ai --------- Co-authored-by: Songbird99 <150154823+Songbird99@users.noreply.github.com> Co-authored-by: Songbird <Songbirdx99@gmail.com>
This commit is contained in:
tduhamel42
@@ -16,4 +16,4 @@ with local project management and persistent storage.
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
|
||||
__version__ = "0.6.0"
|
||||
__version__ = "0.7.3"
|
||||
@@ -12,3 +12,6 @@ Command modules for FuzzForge CLI.
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
from . import worker
|
||||
|
||||
__all__ = ["worker"]
|
||||
|
||||
@@ -253,15 +253,15 @@ def display_finding_detail(finding: Dict[str, Any], tool: Dict[str, Any], run_id
|
||||
content_lines.append(f"[bold]Tool:[/bold] {tool.get('name', 'Unknown')} v{tool.get('version', 'unknown')}")
|
||||
content_lines.append(f"[bold]Run ID:[/bold] {run_id}")
|
||||
content_lines.append("")
|
||||
content_lines.append(f"[bold]Summary:[/bold]")
|
||||
content_lines.append("[bold]Summary:[/bold]")
|
||||
content_lines.append(message_text)
|
||||
content_lines.append("")
|
||||
content_lines.append(f"[bold]Description:[/bold]")
|
||||
content_lines.append("[bold]Description:[/bold]")
|
||||
content_lines.append(message_markdown)
|
||||
|
||||
if code_snippet:
|
||||
content_lines.append("")
|
||||
content_lines.append(f"[bold]Code Snippet:[/bold]")
|
||||
content_lines.append("[bold]Code Snippet:[/bold]")
|
||||
content_lines.append(f"[dim]{code_snippet}[/dim]")
|
||||
|
||||
content = "\n".join(content_lines)
|
||||
@@ -270,7 +270,7 @@ def display_finding_detail(finding: Dict[str, Any], tool: Dict[str, Any], run_id
|
||||
console.print()
|
||||
console.print(Panel(
|
||||
content,
|
||||
title=f"🔍 Finding Detail",
|
||||
title="🔍 Finding Detail",
|
||||
border_style=severity_color,
|
||||
box=box.ROUNDED,
|
||||
padding=(1, 2)
|
||||
|
||||
@@ -187,19 +187,40 @@ def _ensure_env_file(fuzzforge_dir: Path, force: bool) -> None:
|
||||
|
||||
console.print("🧠 Configuring AI environment...")
|
||||
console.print(" • Default LLM provider: openai")
|
||||
console.print(" • Default LLM model: gpt-5-mini")
|
||||
console.print(" • Default LLM model: litellm_proxy/gpt-5-mini")
|
||||
console.print(" • To customise provider/model later, edit .fuzzforge/.env")
|
||||
|
||||
llm_provider = "openai"
|
||||
llm_model = "gpt-5-mini"
|
||||
llm_model = "litellm_proxy/gpt-5-mini"
|
||||
|
||||
# Check for global virtual keys from volumes/env/.env
|
||||
global_env_key = None
|
||||
for parent in fuzzforge_dir.parents:
|
||||
global_env = parent / "volumes" / "env" / ".env"
|
||||
if global_env.exists():
|
||||
try:
|
||||
for line in global_env.read_text(encoding="utf-8").splitlines():
|
||||
if line.strip().startswith("OPENAI_API_KEY=") and "=" in line:
|
||||
key_value = line.split("=", 1)[1].strip()
|
||||
if key_value and not key_value.startswith("your-") and key_value.startswith("sk-"):
|
||||
global_env_key = key_value
|
||||
console.print(f" • Found virtual key in {global_env.relative_to(parent)}")
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
break
|
||||
|
||||
api_key = Prompt.ask(
|
||||
"OpenAI API key (leave blank to fill manually)",
|
||||
"OpenAI API key (leave blank to use global virtual key)" if global_env_key else "OpenAI API key (leave blank to fill manually)",
|
||||
default="",
|
||||
show_default=False,
|
||||
console=console,
|
||||
)
|
||||
|
||||
# Use global key if user didn't provide one
|
||||
if not api_key and global_env_key:
|
||||
api_key = global_env_key
|
||||
|
||||
session_db_path = fuzzforge_dir / "fuzzforge_sessions.db"
|
||||
session_db_rel = session_db_path.relative_to(fuzzforge_dir.parent)
|
||||
|
||||
@@ -210,14 +231,20 @@ def _ensure_env_file(fuzzforge_dir: Path, force: bool) -> None:
|
||||
f"LLM_PROVIDER={llm_provider}",
|
||||
f"LLM_MODEL={llm_model}",
|
||||
f"LITELLM_MODEL={llm_model}",
|
||||
"LLM_ENDPOINT=http://localhost:10999",
|
||||
"LLM_API_KEY=",
|
||||
"LLM_EMBEDDING_MODEL=litellm_proxy/text-embedding-3-large",
|
||||
"LLM_EMBEDDING_ENDPOINT=http://localhost:10999",
|
||||
f"OPENAI_API_KEY={api_key}",
|
||||
"FUZZFORGE_MCP_URL=http://localhost:8010/mcp",
|
||||
"",
|
||||
"# Cognee configuration mirrors the primary LLM by default",
|
||||
f"LLM_COGNEE_PROVIDER={llm_provider}",
|
||||
f"LLM_COGNEE_MODEL={llm_model}",
|
||||
f"LLM_COGNEE_API_KEY={api_key}",
|
||||
"LLM_COGNEE_ENDPOINT=",
|
||||
"LLM_COGNEE_ENDPOINT=http://localhost:10999",
|
||||
"LLM_COGNEE_API_KEY=",
|
||||
"LLM_COGNEE_EMBEDDING_MODEL=litellm_proxy/text-embedding-3-large",
|
||||
"LLM_COGNEE_EMBEDDING_ENDPOINT=http://localhost:10999",
|
||||
"COGNEE_MCP_URL=",
|
||||
"",
|
||||
"# Session persistence options: inmemory | sqlite",
|
||||
@@ -239,6 +266,8 @@ def _ensure_env_file(fuzzforge_dir: Path, force: bool) -> None:
|
||||
for line in env_lines:
|
||||
if line.startswith("OPENAI_API_KEY="):
|
||||
template_lines.append("OPENAI_API_KEY=")
|
||||
elif line.startswith("LLM_API_KEY="):
|
||||
template_lines.append("LLM_API_KEY=")
|
||||
elif line.startswith("LLM_COGNEE_API_KEY="):
|
||||
template_lines.append("LLM_COGNEE_API_KEY=")
|
||||
else:
|
||||
|
||||
@@ -0,0 +1,225 @@
|
||||
"""
|
||||
Worker management commands for FuzzForge CLI.
|
||||
|
||||
Provides commands to start, stop, and list Temporal workers.
|
||||
"""
|
||||
# Copyright (c) 2025 FuzzingLabs
|
||||
#
|
||||
# Licensed under the Business Source License 1.1 (BSL). See the LICENSE file
|
||||
# at the root of this repository for details.
|
||||
#
|
||||
# After the Change Date (four years from publication), this version of the
|
||||
# Licensed Work will be made available under the Apache License, Version 2.0.
|
||||
# See the LICENSE-APACHE file or http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import typer
|
||||
from pathlib import Path
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
from typing import Optional
|
||||
|
||||
from ..worker_manager import WorkerManager
|
||||
|
||||
console = Console()
|
||||
app = typer.Typer(
|
||||
name="worker",
|
||||
help="🔧 Manage Temporal workers",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
|
||||
|
||||
@app.command("stop")
|
||||
def stop_workers(
|
||||
all: bool = typer.Option(
|
||||
False, "--all",
|
||||
help="Stop all workers (default behavior, flag for clarity)"
|
||||
)
|
||||
):
|
||||
"""
|
||||
🛑 Stop all running FuzzForge workers.
|
||||
|
||||
This command stops all worker containers using the proper Docker Compose
|
||||
profile flag to ensure workers are actually stopped (since they're in profiles).
|
||||
|
||||
Examples:
|
||||
$ ff worker stop
|
||||
$ ff worker stop --all
|
||||
"""
|
||||
try:
|
||||
worker_mgr = WorkerManager()
|
||||
success = worker_mgr.stop_all_workers()
|
||||
|
||||
if success:
|
||||
sys.exit(0)
|
||||
else:
|
||||
console.print("⚠️ Some workers may not have stopped properly", style="yellow")
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"❌ Error: {e}", style="red")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@app.command("list")
|
||||
def list_workers(
|
||||
all: bool = typer.Option(
|
||||
False, "--all", "-a",
|
||||
help="Show all workers (including stopped)"
|
||||
)
|
||||
):
|
||||
"""
|
||||
📋 List FuzzForge workers and their status.
|
||||
|
||||
By default, shows only running workers. Use --all to see all workers.
|
||||
|
||||
Examples:
|
||||
$ ff worker list
|
||||
$ ff worker list --all
|
||||
"""
|
||||
try:
|
||||
# Get list of running workers
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "--filter", "name=fuzzforge-worker-",
|
||||
"--format", "{{.Names}}\t{{.Status}}\t{{.RunningFor}}"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
running_workers = []
|
||||
if result.stdout.strip():
|
||||
for line in result.stdout.strip().splitlines():
|
||||
parts = line.split('\t')
|
||||
if len(parts) >= 3:
|
||||
running_workers.append({
|
||||
"name": parts[0].replace("fuzzforge-worker-", ""),
|
||||
"status": "Running",
|
||||
"uptime": parts[2]
|
||||
})
|
||||
|
||||
# If --all, also get stopped workers
|
||||
stopped_workers = []
|
||||
if all:
|
||||
result_all = subprocess.run(
|
||||
["docker", "ps", "-a", "--filter", "name=fuzzforge-worker-",
|
||||
"--format", "{{.Names}}\t{{.Status}}"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
all_worker_names = set()
|
||||
for line in result_all.stdout.strip().splitlines():
|
||||
parts = line.split('\t')
|
||||
if len(parts) >= 2:
|
||||
worker_name = parts[0].replace("fuzzforge-worker-", "")
|
||||
all_worker_names.add(worker_name)
|
||||
# If not running, it's stopped
|
||||
if not any(w["name"] == worker_name for w in running_workers):
|
||||
stopped_workers.append({
|
||||
"name": worker_name,
|
||||
"status": "Stopped",
|
||||
"uptime": "-"
|
||||
})
|
||||
|
||||
# Display results
|
||||
if not running_workers and not stopped_workers:
|
||||
console.print("ℹ️ No workers found", style="cyan")
|
||||
console.print("\n💡 Start a worker with: [cyan]docker compose up -d worker-<name>[/cyan]")
|
||||
console.print(" Or run a workflow, which auto-starts workers: [cyan]ff workflow run <workflow> <target>[/cyan]")
|
||||
return
|
||||
|
||||
# Create table
|
||||
table = Table(title="FuzzForge Workers", show_header=True, header_style="bold cyan")
|
||||
table.add_column("Worker", style="cyan", no_wrap=True)
|
||||
table.add_column("Status", style="green")
|
||||
table.add_column("Uptime", style="dim")
|
||||
|
||||
# Add running workers
|
||||
for worker in running_workers:
|
||||
table.add_row(
|
||||
worker["name"],
|
||||
f"[green]●[/green] {worker['status']}",
|
||||
worker["uptime"]
|
||||
)
|
||||
|
||||
# Add stopped workers if --all
|
||||
for worker in stopped_workers:
|
||||
table.add_row(
|
||||
worker["name"],
|
||||
f"[red]●[/red] {worker['status']}",
|
||||
worker["uptime"]
|
||||
)
|
||||
|
||||
console.print(table)
|
||||
|
||||
# Summary
|
||||
if running_workers:
|
||||
console.print(f"\n✅ {len(running_workers)} worker(s) running")
|
||||
if stopped_workers:
|
||||
console.print(f"⏹️ {len(stopped_workers)} worker(s) stopped")
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"❌ Error listing workers: {e}", style="red")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@app.command("start")
|
||||
def start_worker(
|
||||
name: str = typer.Argument(
|
||||
...,
|
||||
help="Worker name (e.g., 'python', 'android', 'secrets')"
|
||||
),
|
||||
build: bool = typer.Option(
|
||||
False, "--build",
|
||||
help="Rebuild worker image before starting"
|
||||
)
|
||||
):
|
||||
"""
|
||||
🚀 Start a specific worker.
|
||||
|
||||
The worker name should be the vertical name (e.g., 'python', 'android', 'rust').
|
||||
|
||||
Examples:
|
||||
$ ff worker start python
|
||||
$ ff worker start android --build
|
||||
"""
|
||||
try:
|
||||
service_name = f"worker-{name}"
|
||||
|
||||
console.print(f"🚀 Starting worker: [cyan]{service_name}[/cyan]")
|
||||
|
||||
# Build docker compose command
|
||||
cmd = ["docker", "compose", "up", "-d"]
|
||||
if build:
|
||||
cmd.append("--build")
|
||||
cmd.append(service_name)
|
||||
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
if result.returncode == 0:
|
||||
console.print(f"✅ Worker [cyan]{service_name}[/cyan] started successfully")
|
||||
else:
|
||||
console.print(f"❌ Failed to start worker: {result.stderr}", style="red")
|
||||
console.print(
|
||||
f"\n💡 Try manually: [yellow]docker compose up -d {service_name}[/yellow]",
|
||||
style="dim"
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"❌ Error: {e}", style="red")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
app()
|
||||
@@ -39,7 +39,7 @@ from ..validation import (
|
||||
)
|
||||
from ..progress import step_progress
|
||||
from ..constants import (
|
||||
STATUS_EMOJIS, MAX_RUN_ID_DISPLAY_LENGTH, DEFAULT_VOLUME_MODE,
|
||||
STATUS_EMOJIS, MAX_RUN_ID_DISPLAY_LENGTH,
|
||||
PROGRESS_STEP_DELAYS, MAX_RETRIES, RETRY_DELAY, POLL_INTERVAL
|
||||
)
|
||||
from ..worker_manager import WorkerManager
|
||||
@@ -112,7 +112,6 @@ def execute_workflow_submission(
|
||||
workflow: str,
|
||||
target_path: str,
|
||||
parameters: Dict[str, Any],
|
||||
volume_mode: str,
|
||||
timeout: Optional[int],
|
||||
interactive: bool
|
||||
) -> Any:
|
||||
@@ -160,13 +159,10 @@ def execute_workflow_submission(
|
||||
except ValueError as e:
|
||||
console.print(f"❌ Invalid {param_type}: {e}", style="red")
|
||||
|
||||
# Note: volume_mode is no longer used (Temporal uses MinIO storage)
|
||||
|
||||
# Show submission summary
|
||||
console.print("\n🎯 [bold]Executing workflow:[/bold]")
|
||||
console.print(f" Workflow: {workflow}")
|
||||
console.print(f" Target: {target_path}")
|
||||
console.print(f" Volume Mode: {volume_mode}")
|
||||
if parameters:
|
||||
console.print(f" Parameters: {len(parameters)} provided")
|
||||
if timeout:
|
||||
@@ -252,8 +248,6 @@ def execute_workflow_submission(
|
||||
|
||||
progress.next_step() # Submitting
|
||||
submission = WorkflowSubmission(
|
||||
target_path=target_path,
|
||||
volume_mode=volume_mode,
|
||||
parameters=parameters,
|
||||
timeout=timeout
|
||||
)
|
||||
@@ -281,10 +275,6 @@ def execute_workflow(
|
||||
None, "--param-file", "-f",
|
||||
help="JSON file containing workflow parameters"
|
||||
),
|
||||
volume_mode: str = typer.Option(
|
||||
DEFAULT_VOLUME_MODE, "--volume-mode", "-v",
|
||||
help="Volume mount mode: ro (read-only) or rw (read-write)"
|
||||
),
|
||||
timeout: Optional[int] = typer.Option(
|
||||
None, "--timeout", "-t",
|
||||
help="Execution timeout in seconds"
|
||||
@@ -410,7 +400,7 @@ def execute_workflow(
|
||||
|
||||
response = execute_workflow_submission(
|
||||
client, workflow, target_path, parameters,
|
||||
volume_mode, timeout, interactive
|
||||
timeout, interactive
|
||||
)
|
||||
|
||||
console.print("✅ Workflow execution started!", style="green")
|
||||
@@ -453,9 +443,9 @@ def execute_workflow(
|
||||
console.print("Press Ctrl+C to stop monitoring (execution continues in background).\n")
|
||||
|
||||
try:
|
||||
from ..commands.monitor import live_monitor
|
||||
# Import monitor command and run it
|
||||
live_monitor(response.run_id, refresh=3)
|
||||
from ..commands.monitor import _live_monitor
|
||||
# Call helper function directly with proper parameters
|
||||
_live_monitor(response.run_id, refresh=3, once=False, style="inline")
|
||||
except KeyboardInterrupt:
|
||||
console.print("\n⏹️ Live monitoring stopped (execution continues in background)", style="yellow")
|
||||
except Exception as e:
|
||||
|
||||
@@ -95,12 +95,6 @@ def complete_target_paths(incomplete: str) -> List[str]:
|
||||
return []
|
||||
|
||||
|
||||
def complete_volume_modes(incomplete: str) -> List[str]:
|
||||
"""Auto-complete volume mount modes."""
|
||||
modes = ["ro", "rw"]
|
||||
return [mode for mode in modes if mode.startswith(incomplete)]
|
||||
|
||||
|
||||
def complete_export_formats(incomplete: str) -> List[str]:
|
||||
"""Auto-complete export formats."""
|
||||
formats = ["json", "csv", "html", "sarif"]
|
||||
@@ -139,7 +133,6 @@ def complete_config_keys(incomplete: str) -> List[str]:
|
||||
"api_url",
|
||||
"api_timeout",
|
||||
"default_workflow",
|
||||
"default_volume_mode",
|
||||
"project_name",
|
||||
"data_retention_days",
|
||||
"auto_save_findings",
|
||||
@@ -164,11 +157,6 @@ TargetPathComplete = typer.Argument(
|
||||
help="Target path (tab completion available)"
|
||||
)
|
||||
|
||||
VolumeModetComplete = typer.Option(
|
||||
autocompletion=complete_volume_modes,
|
||||
help="Volume mode: ro or rw (tab completion available)"
|
||||
)
|
||||
|
||||
ExportFormatComplete = typer.Option(
|
||||
autocompletion=complete_export_formats,
|
||||
help="Export format (tab completion available)"
|
||||
|
||||
@@ -28,6 +28,58 @@ try: # Optional dependency; fall back if not installed
|
||||
except ImportError: # pragma: no cover - optional dependency
|
||||
load_dotenv = None
|
||||
|
||||
|
||||
def _load_env_file_if_exists(path: Path, override: bool = False) -> bool:
|
||||
if not path.exists():
|
||||
return False
|
||||
# Always use manual parsing to handle empty values correctly
|
||||
try:
|
||||
for line in path.read_text(encoding="utf-8").splitlines():
|
||||
stripped = line.strip()
|
||||
if not stripped or stripped.startswith("#") or "=" not in stripped:
|
||||
continue
|
||||
key, value = stripped.split("=", 1)
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
if override:
|
||||
# Only override if value is non-empty
|
||||
if value:
|
||||
os.environ[key] = value
|
||||
else:
|
||||
# Set if not already in environment and value is non-empty
|
||||
if key not in os.environ and value:
|
||||
os.environ[key] = value
|
||||
return True
|
||||
except Exception: # pragma: no cover - best effort fallback
|
||||
return False
|
||||
|
||||
|
||||
def _find_shared_env_file(project_dir: Path) -> Path | None:
|
||||
for directory in [project_dir] + list(project_dir.parents):
|
||||
candidate = directory / "volumes" / "env" / ".env"
|
||||
if candidate.exists():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def load_project_env(project_dir: Optional[Path] = None) -> Path | None:
|
||||
"""Load project-local env, falling back to shared volumes/env/.env."""
|
||||
|
||||
project_dir = Path(project_dir or Path.cwd())
|
||||
shared_env = _find_shared_env_file(project_dir)
|
||||
loaded_shared = False
|
||||
if shared_env:
|
||||
loaded_shared = _load_env_file_if_exists(shared_env, override=False)
|
||||
|
||||
project_env = project_dir / ".fuzzforge" / ".env"
|
||||
if _load_env_file_if_exists(project_env, override=True):
|
||||
return project_env
|
||||
|
||||
if loaded_shared:
|
||||
return shared_env
|
||||
|
||||
return None
|
||||
|
||||
import yaml
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
@@ -312,23 +364,7 @@ class ProjectConfigManager:
|
||||
if not cognee.get("enabled", True):
|
||||
return
|
||||
|
||||
# Load project-specific environment overrides from .fuzzforge/.env if available
|
||||
env_file = self.project_dir / ".fuzzforge" / ".env"
|
||||
if env_file.exists():
|
||||
if load_dotenv:
|
||||
load_dotenv(env_file, override=False)
|
||||
else:
|
||||
try:
|
||||
for line in env_file.read_text(encoding="utf-8").splitlines():
|
||||
stripped = line.strip()
|
||||
if not stripped or stripped.startswith("#"):
|
||||
continue
|
||||
if "=" not in stripped:
|
||||
continue
|
||||
key, value = stripped.split("=", 1)
|
||||
os.environ.setdefault(key.strip(), value.strip())
|
||||
except Exception: # pragma: no cover - best effort fallback
|
||||
pass
|
||||
load_project_env(self.project_dir)
|
||||
|
||||
backend_access = "true" if cognee.get("backend_access_control", True) else "false"
|
||||
os.environ["ENABLE_BACKEND_ACCESS_CONTROL"] = backend_access
|
||||
@@ -374,6 +410,17 @@ class ProjectConfigManager:
|
||||
"OPENAI_API_KEY",
|
||||
)
|
||||
endpoint = _env("LLM_COGNEE_ENDPOINT", "COGNEE_LLM_ENDPOINT", "LLM_ENDPOINT")
|
||||
embedding_model = _env(
|
||||
"LLM_COGNEE_EMBEDDING_MODEL",
|
||||
"COGNEE_LLM_EMBEDDING_MODEL",
|
||||
"LLM_EMBEDDING_MODEL",
|
||||
)
|
||||
embedding_endpoint = _env(
|
||||
"LLM_COGNEE_EMBEDDING_ENDPOINT",
|
||||
"COGNEE_LLM_EMBEDDING_ENDPOINT",
|
||||
"LLM_EMBEDDING_ENDPOINT",
|
||||
"LLM_ENDPOINT",
|
||||
)
|
||||
api_version = _env(
|
||||
"LLM_COGNEE_API_VERSION",
|
||||
"COGNEE_LLM_API_VERSION",
|
||||
@@ -398,6 +445,20 @@ class ProjectConfigManager:
|
||||
os.environ.setdefault("OPENAI_API_KEY", api_key)
|
||||
if endpoint:
|
||||
os.environ["LLM_ENDPOINT"] = endpoint
|
||||
os.environ.setdefault("LLM_API_BASE", endpoint)
|
||||
os.environ.setdefault("LLM_EMBEDDING_ENDPOINT", endpoint)
|
||||
os.environ.setdefault("LLM_EMBEDDING_API_BASE", endpoint)
|
||||
os.environ.setdefault("OPENAI_API_BASE", endpoint)
|
||||
# Set LiteLLM proxy environment variables for SDK usage
|
||||
os.environ.setdefault("LITELLM_PROXY_API_BASE", endpoint)
|
||||
if api_key:
|
||||
# Set LiteLLM proxy API key from the virtual key
|
||||
os.environ.setdefault("LITELLM_PROXY_API_KEY", api_key)
|
||||
if embedding_model:
|
||||
os.environ["LLM_EMBEDDING_MODEL"] = embedding_model
|
||||
if embedding_endpoint:
|
||||
os.environ["LLM_EMBEDDING_ENDPOINT"] = embedding_endpoint
|
||||
os.environ.setdefault("LLM_EMBEDDING_API_BASE", embedding_endpoint)
|
||||
if api_version:
|
||||
os.environ["LLM_API_VERSION"] = api_version
|
||||
if max_tokens:
|
||||
|
||||
@@ -57,10 +57,6 @@ SEVERITY_STYLES = {
|
||||
"info": "bold cyan"
|
||||
}
|
||||
|
||||
# Default volume modes
|
||||
DEFAULT_VOLUME_MODE = "ro"
|
||||
SUPPORTED_VOLUME_MODES = ["ro", "rw"]
|
||||
|
||||
# Default export formats
|
||||
DEFAULT_EXPORT_FORMAT = "sarif"
|
||||
SUPPORTED_EXPORT_FORMATS = ["sarif", "json", "csv"]
|
||||
|
||||
@@ -52,7 +52,6 @@ class FuzzyMatcher:
|
||||
# Common parameter names
|
||||
self.parameter_names = [
|
||||
"target_path",
|
||||
"volume_mode",
|
||||
"timeout",
|
||||
"workflow",
|
||||
"param",
|
||||
@@ -70,7 +69,6 @@ class FuzzyMatcher:
|
||||
|
||||
# Common values
|
||||
self.common_values = {
|
||||
"volume_mode": ["ro", "rw"],
|
||||
"format": ["json", "csv", "html", "sarif"],
|
||||
"severity": ["critical", "high", "medium", "low", "info"],
|
||||
}
|
||||
|
||||
@@ -19,6 +19,8 @@ from rich.traceback import install
|
||||
from typing import Optional, List
|
||||
import sys
|
||||
|
||||
from .config import load_project_env
|
||||
|
||||
from .commands import (
|
||||
workflows,
|
||||
workflow_exec,
|
||||
@@ -27,13 +29,16 @@ from .commands import (
|
||||
config as config_cmd,
|
||||
ai,
|
||||
ingest,
|
||||
worker,
|
||||
)
|
||||
from .constants import DEFAULT_VOLUME_MODE
|
||||
from .fuzzy import enhanced_command_not_found_handler
|
||||
|
||||
# Install rich traceback handler
|
||||
install(show_locals=True)
|
||||
|
||||
# Ensure environment variables are available before command execution
|
||||
load_project_env()
|
||||
|
||||
# Create console for rich output
|
||||
console = Console()
|
||||
|
||||
@@ -184,10 +189,6 @@ def run_workflow(
|
||||
None, "--param-file", "-f",
|
||||
help="JSON file containing workflow parameters"
|
||||
),
|
||||
volume_mode: str = typer.Option(
|
||||
DEFAULT_VOLUME_MODE, "--volume-mode", "-v",
|
||||
help="Volume mount mode: ro (read-only) or rw (read-write)"
|
||||
),
|
||||
timeout: Optional[int] = typer.Option(
|
||||
None, "--timeout", "-t",
|
||||
help="Execution timeout in seconds"
|
||||
@@ -234,7 +235,6 @@ def run_workflow(
|
||||
target_path=target,
|
||||
params=params,
|
||||
param_file=param_file,
|
||||
volume_mode=volume_mode,
|
||||
timeout=timeout,
|
||||
interactive=interactive,
|
||||
wait=wait,
|
||||
@@ -335,6 +335,7 @@ app.add_typer(finding_app, name="finding", help="🔍 View and analyze findings"
|
||||
app.add_typer(monitor.app, name="monitor", help="📊 Real-time monitoring")
|
||||
app.add_typer(ai.app, name="ai", help="🤖 AI integration features")
|
||||
app.add_typer(ingest.app, name="ingest", help="🧠 Ingest knowledge into AI")
|
||||
app.add_typer(worker.app, name="worker", help="🔧 Manage Temporal workers")
|
||||
|
||||
# Help and utility commands
|
||||
@app.command()
|
||||
@@ -410,7 +411,7 @@ def main():
|
||||
'init', 'status', 'config', 'clean',
|
||||
'workflows', 'workflow',
|
||||
'findings', 'finding',
|
||||
'monitor', 'ai', 'ingest',
|
||||
'monitor', 'ai', 'ingest', 'worker',
|
||||
'version'
|
||||
]
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ import re
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from .constants import SUPPORTED_VOLUME_MODES, SUPPORTED_EXPORT_FORMATS
|
||||
from .constants import SUPPORTED_EXPORT_FORMATS
|
||||
from .exceptions import ValidationError
|
||||
|
||||
|
||||
@@ -65,15 +65,6 @@ def validate_target_path(target_path: str, must_exist: bool = True) -> Path:
|
||||
return path
|
||||
|
||||
|
||||
def validate_volume_mode(volume_mode: str) -> None:
|
||||
"""Validate volume mode"""
|
||||
if volume_mode not in SUPPORTED_VOLUME_MODES:
|
||||
raise ValidationError(
|
||||
"volume_mode", volume_mode,
|
||||
f"one of: {', '.join(SUPPORTED_VOLUME_MODES)}"
|
||||
)
|
||||
|
||||
|
||||
def validate_export_format(export_format: str) -> None:
|
||||
"""Validate export format"""
|
||||
if export_format not in SUPPORTED_EXPORT_FORMATS:
|
||||
|
||||
@@ -15,12 +15,17 @@ Manages on-demand startup and shutdown of Temporal workers using Docker Compose.
|
||||
# Additional attribution and requirements are provided in the NOTICE file.
|
||||
|
||||
import logging
|
||||
import os
|
||||
import platform
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional, Dict, Any
|
||||
|
||||
import requests
|
||||
import yaml
|
||||
from rich.console import Console
|
||||
from rich.status import Status
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
console = Console()
|
||||
@@ -57,27 +62,206 @@ class WorkerManager:
|
||||
|
||||
def _find_compose_file(self) -> Path:
|
||||
"""
|
||||
Auto-detect docker-compose.yml location.
|
||||
Auto-detect docker-compose.yml location using multiple strategies.
|
||||
|
||||
Searches upward from current directory to find the compose file.
|
||||
Strategies (in order):
|
||||
1. Query backend API for host path
|
||||
2. Search upward for .fuzzforge marker directory
|
||||
3. Use FUZZFORGE_ROOT environment variable
|
||||
4. Fallback to current directory
|
||||
|
||||
Returns:
|
||||
Path to docker-compose.yml
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If docker-compose.yml cannot be located
|
||||
"""
|
||||
current = Path.cwd()
|
||||
# Strategy 1: Ask backend for location
|
||||
try:
|
||||
backend_url = os.getenv("FUZZFORGE_API_URL", "http://localhost:8000")
|
||||
response = requests.get(f"{backend_url}/system/info", timeout=2)
|
||||
if response.ok:
|
||||
info = response.json()
|
||||
if compose_path_str := info.get("docker_compose_path"):
|
||||
compose_path = Path(compose_path_str)
|
||||
if compose_path.exists():
|
||||
logger.debug(f"Found docker-compose.yml via backend API: {compose_path}")
|
||||
return compose_path
|
||||
except Exception as e:
|
||||
logger.debug(f"Backend API not reachable for path lookup: {e}")
|
||||
|
||||
# Try current directory and parents
|
||||
# Strategy 2: Search upward for .fuzzforge marker directory
|
||||
current = Path.cwd()
|
||||
for parent in [current] + list(current.parents):
|
||||
compose_path = parent / "docker-compose.yml"
|
||||
if (parent / ".fuzzforge").exists():
|
||||
compose_path = parent / "docker-compose.yml"
|
||||
if compose_path.exists():
|
||||
logger.debug(f"Found docker-compose.yml via .fuzzforge marker: {compose_path}")
|
||||
return compose_path
|
||||
|
||||
# Strategy 3: Environment variable
|
||||
if fuzzforge_root := os.getenv("FUZZFORGE_ROOT"):
|
||||
compose_path = Path(fuzzforge_root) / "docker-compose.yml"
|
||||
if compose_path.exists():
|
||||
logger.debug(f"Found docker-compose.yml via FUZZFORGE_ROOT: {compose_path}")
|
||||
return compose_path
|
||||
|
||||
# Fallback to default location
|
||||
return Path("docker-compose.yml")
|
||||
# Strategy 4: Fallback to current directory
|
||||
compose_path = Path("docker-compose.yml")
|
||||
if compose_path.exists():
|
||||
return compose_path
|
||||
|
||||
def _run_docker_compose(self, *args: str) -> subprocess.CompletedProcess:
|
||||
raise FileNotFoundError(
|
||||
"Cannot find docker-compose.yml. Ensure backend is running, "
|
||||
"run from FuzzForge directory, or set FUZZFORGE_ROOT environment variable."
|
||||
)
|
||||
|
||||
def _get_workers_dir(self) -> Path:
|
||||
"""
|
||||
Run docker-compose command.
|
||||
Get the workers directory path.
|
||||
|
||||
Uses same strategy as _find_compose_file():
|
||||
1. Query backend API
|
||||
2. Derive from compose_file location
|
||||
3. Use FUZZFORGE_ROOT
|
||||
|
||||
Returns:
|
||||
Path to workers directory
|
||||
"""
|
||||
# Strategy 1: Ask backend
|
||||
try:
|
||||
backend_url = os.getenv("FUZZFORGE_API_URL", "http://localhost:8000")
|
||||
response = requests.get(f"{backend_url}/system/info", timeout=2)
|
||||
if response.ok:
|
||||
info = response.json()
|
||||
if workers_dir_str := info.get("workers_dir"):
|
||||
workers_dir = Path(workers_dir_str)
|
||||
if workers_dir.exists():
|
||||
return workers_dir
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Strategy 2: Derive from compose file location
|
||||
if self.compose_file.exists():
|
||||
workers_dir = self.compose_file.parent / "workers"
|
||||
if workers_dir.exists():
|
||||
return workers_dir
|
||||
|
||||
# Strategy 3: Use environment variable
|
||||
if fuzzforge_root := os.getenv("FUZZFORGE_ROOT"):
|
||||
workers_dir = Path(fuzzforge_root) / "workers"
|
||||
if workers_dir.exists():
|
||||
return workers_dir
|
||||
|
||||
# Fallback
|
||||
return Path("workers")
|
||||
|
||||
def _detect_platform(self) -> str:
|
||||
"""
|
||||
Detect the current platform.
|
||||
|
||||
Returns:
|
||||
Platform string: "linux/amd64" or "linux/arm64"
|
||||
"""
|
||||
machine = platform.machine().lower()
|
||||
system = platform.system().lower()
|
||||
|
||||
logger.debug(f"Platform detection: machine={machine}, system={system}")
|
||||
|
||||
# Normalize machine architecture
|
||||
if machine in ["x86_64", "amd64", "x64"]:
|
||||
detected = "linux/amd64"
|
||||
elif machine in ["arm64", "aarch64", "armv8", "arm64v8"]:
|
||||
detected = "linux/arm64"
|
||||
else:
|
||||
# Fallback to amd64 for unknown architectures
|
||||
logger.warning(
|
||||
f"Unknown architecture '{machine}' detected, falling back to linux/amd64. "
|
||||
f"Please report this issue if you're experiencing problems."
|
||||
)
|
||||
detected = "linux/amd64"
|
||||
|
||||
logger.info(f"Detected platform: {detected}")
|
||||
return detected
|
||||
|
||||
def _read_worker_metadata(self, vertical: str) -> dict:
|
||||
"""
|
||||
Read worker metadata.yaml for a vertical.
|
||||
|
||||
Args:
|
||||
*args: Arguments to pass to docker-compose
|
||||
vertical: Worker vertical name (e.g., "android", "python")
|
||||
|
||||
Returns:
|
||||
Dictionary containing metadata, or empty dict if not found
|
||||
"""
|
||||
try:
|
||||
workers_dir = self._get_workers_dir()
|
||||
metadata_file = workers_dir / vertical / "metadata.yaml"
|
||||
|
||||
if not metadata_file.exists():
|
||||
logger.debug(f"No metadata.yaml found for {vertical}")
|
||||
return {}
|
||||
|
||||
with open(metadata_file, 'r') as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to read metadata for {vertical}: {e}")
|
||||
return {}
|
||||
|
||||
def _select_dockerfile(self, vertical: str) -> str:
|
||||
"""
|
||||
Select the appropriate Dockerfile for the current platform.
|
||||
|
||||
Args:
|
||||
vertical: Worker vertical name
|
||||
|
||||
Returns:
|
||||
Dockerfile name (e.g., "Dockerfile.amd64", "Dockerfile.arm64")
|
||||
"""
|
||||
detected_platform = self._detect_platform()
|
||||
metadata = self._read_worker_metadata(vertical)
|
||||
|
||||
if not metadata:
|
||||
# No metadata: use default Dockerfile
|
||||
logger.debug(f"No metadata for {vertical}, using Dockerfile")
|
||||
return "Dockerfile"
|
||||
|
||||
platforms = metadata.get("platforms", {})
|
||||
|
||||
if not platforms:
|
||||
# Metadata exists but no platform definitions
|
||||
logger.debug(f"No platform definitions in metadata for {vertical}, using Dockerfile")
|
||||
return "Dockerfile"
|
||||
|
||||
# Try detected platform first
|
||||
if detected_platform in platforms:
|
||||
dockerfile = platforms[detected_platform].get("dockerfile", "Dockerfile")
|
||||
logger.info(f"✓ Selected {dockerfile} for {vertical} on {detected_platform}")
|
||||
return dockerfile
|
||||
|
||||
# Fallback to default platform
|
||||
default_platform = metadata.get("default_platform", "linux/amd64")
|
||||
logger.warning(
|
||||
f"Platform {detected_platform} not found in metadata for {vertical}, "
|
||||
f"falling back to default: {default_platform}"
|
||||
)
|
||||
|
||||
if default_platform in platforms:
|
||||
dockerfile = platforms[default_platform].get("dockerfile", "Dockerfile.amd64")
|
||||
logger.info(f"Using default platform {default_platform}: {dockerfile}")
|
||||
return dockerfile
|
||||
|
||||
# Last resort: just use Dockerfile
|
||||
logger.warning(f"No suitable Dockerfile found for {vertical}, using 'Dockerfile'")
|
||||
return "Dockerfile"
|
||||
|
||||
def _run_docker_compose(self, *args: str, env: Optional[Dict[str, str]] = None) -> subprocess.CompletedProcess:
|
||||
"""
|
||||
Run docker compose command with optional environment variables.
|
||||
|
||||
Args:
|
||||
*args: Arguments to pass to docker compose
|
||||
env: Optional environment variables to set
|
||||
|
||||
Returns:
|
||||
CompletedProcess with result
|
||||
@@ -85,14 +269,21 @@ class WorkerManager:
|
||||
Raises:
|
||||
subprocess.CalledProcessError: If command fails
|
||||
"""
|
||||
cmd = ["docker-compose", "-f", str(self.compose_file)] + list(args)
|
||||
cmd = ["docker", "compose", "-f", str(self.compose_file)] + list(args)
|
||||
logger.debug(f"Running: {' '.join(cmd)}")
|
||||
|
||||
# Merge with current environment
|
||||
full_env = os.environ.copy()
|
||||
if env:
|
||||
full_env.update(env)
|
||||
logger.debug(f"Environment overrides: {env}")
|
||||
|
||||
return subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True
|
||||
check=True,
|
||||
env=full_env
|
||||
)
|
||||
|
||||
def _service_to_container_name(self, service_name: str) -> str:
|
||||
@@ -135,21 +326,35 @@ class WorkerManager:
|
||||
|
||||
def start_worker(self, service_name: str) -> bool:
|
||||
"""
|
||||
Start a worker service using docker-compose.
|
||||
Start a worker service using docker-compose with platform-specific Dockerfile.
|
||||
|
||||
Args:
|
||||
service_name: Name of the Docker Compose service to start (e.g., "worker-python")
|
||||
service_name: Name of the Docker Compose service to start (e.g., "worker-android")
|
||||
|
||||
Returns:
|
||||
True if started successfully, False otherwise
|
||||
"""
|
||||
try:
|
||||
console.print(f"🚀 Starting worker: {service_name}")
|
||||
# Extract vertical name from service name
|
||||
vertical = service_name.replace("worker-", "")
|
||||
|
||||
# Use docker-compose up to create and start the service
|
||||
result = self._run_docker_compose("up", "-d", service_name)
|
||||
# Detect platform and select appropriate Dockerfile
|
||||
detected_platform = self._detect_platform()
|
||||
dockerfile = self._select_dockerfile(vertical)
|
||||
|
||||
logger.info(f"Worker {service_name} started")
|
||||
# Set environment variable for docker-compose
|
||||
env_var_name = f"{vertical.upper()}_DOCKERFILE"
|
||||
env = {env_var_name: dockerfile}
|
||||
|
||||
console.print(
|
||||
f"🚀 Starting worker: {service_name} "
|
||||
f"(platform: {detected_platform}, using {dockerfile})"
|
||||
)
|
||||
|
||||
# Use docker-compose up with --build to ensure correct Dockerfile is used
|
||||
result = self._run_docker_compose("up", "-d", "--build", service_name, env=env)
|
||||
|
||||
logger.info(f"Worker {service_name} started with {dockerfile}")
|
||||
return True
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
@@ -163,9 +368,67 @@ class WorkerManager:
|
||||
console.print(f"❌ Unexpected error: {e}", style="red")
|
||||
return False
|
||||
|
||||
def _get_container_state(self, service_name: str) -> str:
|
||||
"""
|
||||
Get the current state of a container (running, created, restarting, etc.).
|
||||
|
||||
Args:
|
||||
service_name: Name of the Docker Compose service
|
||||
|
||||
Returns:
|
||||
Container state string (running, created, restarting, exited, etc.) or "unknown"
|
||||
"""
|
||||
try:
|
||||
container_name = self._service_to_container_name(service_name)
|
||||
result = subprocess.run(
|
||||
["docker", "inspect", "-f", "{{.State.Status}}", container_name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return result.stdout.strip()
|
||||
return "unknown"
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to get container state: {e}")
|
||||
return "unknown"
|
||||
|
||||
def _get_health_status(self, container_name: str) -> str:
|
||||
"""
|
||||
Get container health status.
|
||||
|
||||
Args:
|
||||
container_name: Docker container name
|
||||
|
||||
Returns:
|
||||
Health status: "healthy", "unhealthy", "starting", "none", or "unknown"
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "inspect", "-f", "{{.State.Health.Status}}", container_name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
if result.returncode != 0:
|
||||
return "unknown"
|
||||
|
||||
health_status = result.stdout.strip()
|
||||
|
||||
if health_status == "<no value>" or health_status == "":
|
||||
return "none" # No health check defined
|
||||
|
||||
return health_status # healthy, unhealthy, starting
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to check health: {e}")
|
||||
return "unknown"
|
||||
|
||||
def wait_for_worker_ready(self, service_name: str, timeout: Optional[int] = None) -> bool:
|
||||
"""
|
||||
Wait for a worker to be healthy and ready to process tasks.
|
||||
Shows live progress updates during startup.
|
||||
|
||||
Args:
|
||||
service_name: Name of the Docker Compose service
|
||||
@@ -173,56 +436,74 @@ class WorkerManager:
|
||||
|
||||
Returns:
|
||||
True if worker is ready, False if timeout reached
|
||||
|
||||
Raises:
|
||||
TimeoutError: If worker doesn't become ready within timeout
|
||||
"""
|
||||
timeout = timeout or self.startup_timeout
|
||||
start_time = time.time()
|
||||
container_name = self._service_to_container_name(service_name)
|
||||
last_status_msg = ""
|
||||
|
||||
console.print("⏳ Waiting for worker to be ready...")
|
||||
with Status("[bold cyan]Starting worker...", console=console, spinner="dots") as status:
|
||||
while time.time() - start_time < timeout:
|
||||
elapsed = int(time.time() - start_time)
|
||||
|
||||
# Get container state
|
||||
container_state = self._get_container_state(service_name)
|
||||
|
||||
# Get health status
|
||||
health_status = self._get_health_status(container_name)
|
||||
|
||||
# Build status message based on current state
|
||||
if container_state == "created":
|
||||
status_msg = f"[cyan]Worker starting... ({elapsed}s)[/cyan]"
|
||||
elif container_state == "restarting":
|
||||
status_msg = f"[yellow]Worker restarting... ({elapsed}s)[/yellow]"
|
||||
elif container_state == "running":
|
||||
if health_status == "starting":
|
||||
status_msg = f"[cyan]Worker running, health check starting... ({elapsed}s)[/cyan]"
|
||||
elif health_status == "unhealthy":
|
||||
status_msg = f"[yellow]Worker running, health check: unhealthy ({elapsed}s)[/yellow]"
|
||||
elif health_status == "healthy":
|
||||
status_msg = f"[green]Worker healthy! ({elapsed}s)[/green]"
|
||||
status.update(status_msg)
|
||||
console.print(f"✅ Worker ready: {service_name} (took {elapsed}s)")
|
||||
logger.info(f"Worker {service_name} is healthy (took {elapsed}s)")
|
||||
return True
|
||||
elif health_status == "none":
|
||||
# No health check defined, assume ready
|
||||
status_msg = f"[green]Worker running (no health check) ({elapsed}s)[/green]"
|
||||
status.update(status_msg)
|
||||
console.print(f"✅ Worker ready: {service_name} (took {elapsed}s)")
|
||||
logger.info(f"Worker {service_name} is running, no health check (took {elapsed}s)")
|
||||
return True
|
||||
else:
|
||||
status_msg = f"[cyan]Worker running ({elapsed}s)[/cyan]"
|
||||
elif not container_state or container_state == "exited":
|
||||
status_msg = f"[yellow]Waiting for container to start... ({elapsed}s)[/yellow]"
|
||||
else:
|
||||
status_msg = f"[cyan]Worker state: {container_state} ({elapsed}s)[/cyan]"
|
||||
|
||||
# Show helpful hints at certain intervals
|
||||
if elapsed == 10:
|
||||
status_msg += " [dim](pulling image if not cached)[/dim]"
|
||||
elif elapsed == 30:
|
||||
status_msg += " [dim](large images can take time)[/dim]"
|
||||
elif elapsed == 60:
|
||||
status_msg += " [dim](still working...)[/dim]"
|
||||
|
||||
# Update status if changed
|
||||
if status_msg != last_status_msg:
|
||||
status.update(status_msg)
|
||||
last_status_msg = status_msg
|
||||
logger.debug(f"Worker {service_name} - state: {container_state}, health: {health_status}")
|
||||
|
||||
while time.time() - start_time < timeout:
|
||||
# Check if container is running
|
||||
if not self.is_worker_running(service_name):
|
||||
logger.debug(f"Worker {service_name} not running yet")
|
||||
time.sleep(self.health_check_interval)
|
||||
continue
|
||||
|
||||
# Check container health status
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "inspect", "-f", "{{.State.Health.Status}}", container_name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
health_status = result.stdout.strip()
|
||||
|
||||
# If no health check is defined, assume healthy after running
|
||||
if health_status == "<no value>" or health_status == "":
|
||||
logger.info(f"Worker {service_name} is running (no health check)")
|
||||
console.print(f"✅ Worker ready: {service_name}")
|
||||
return True
|
||||
|
||||
if health_status == "healthy":
|
||||
logger.info(f"Worker {service_name} is healthy")
|
||||
console.print(f"✅ Worker ready: {service_name}")
|
||||
return True
|
||||
|
||||
logger.debug(f"Worker {service_name} health: {health_status}")
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to check health: {e}")
|
||||
|
||||
time.sleep(self.health_check_interval)
|
||||
|
||||
elapsed = time.time() - start_time
|
||||
logger.warning(f"Worker {service_name} did not become ready within {elapsed:.1f}s")
|
||||
console.print(f"⚠️ Worker startup timeout after {elapsed:.1f}s", style="yellow")
|
||||
return False
|
||||
# Timeout reached
|
||||
elapsed = int(time.time() - start_time)
|
||||
logger.warning(f"Worker {service_name} did not become ready within {elapsed}s")
|
||||
console.print(f"⚠️ Worker startup timeout after {elapsed}s", style="yellow")
|
||||
console.print(f" Last state: {container_state}, health: {health_status}", style="dim")
|
||||
return False
|
||||
|
||||
def stop_worker(self, service_name: str) -> bool:
|
||||
"""
|
||||
@@ -253,6 +534,75 @@ class WorkerManager:
|
||||
console.print(f"❌ Unexpected error: {e}", style="red")
|
||||
return False
|
||||
|
||||
def stop_all_workers(self) -> bool:
|
||||
"""
|
||||
Stop all running FuzzForge worker containers.
|
||||
|
||||
This uses `docker stop` to stop worker containers individually,
|
||||
avoiding the Docker Compose profile issue and preventing accidental
|
||||
shutdown of core services.
|
||||
|
||||
Returns:
|
||||
True if all workers stopped successfully, False otherwise
|
||||
"""
|
||||
try:
|
||||
console.print("🛑 Stopping all FuzzForge workers...")
|
||||
|
||||
# Get list of all running worker containers
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "--filter", "name=fuzzforge-worker-", "--format", "{{.Names}}"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False
|
||||
)
|
||||
|
||||
running_workers = [name.strip() for name in result.stdout.splitlines() if name.strip()]
|
||||
|
||||
if not running_workers:
|
||||
console.print("✓ No workers running")
|
||||
return True
|
||||
|
||||
console.print(f"Found {len(running_workers)} running worker(s):")
|
||||
for worker in running_workers:
|
||||
console.print(f" - {worker}")
|
||||
|
||||
# Stop each worker container individually using docker stop
|
||||
# This is safer than docker compose down and won't affect core services
|
||||
failed_workers = []
|
||||
for worker in running_workers:
|
||||
try:
|
||||
logger.info(f"Stopping {worker}...")
|
||||
result = subprocess.run(
|
||||
["docker", "stop", worker],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
timeout=30
|
||||
)
|
||||
console.print(f" ✓ Stopped {worker}")
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.error(f"Failed to stop {worker}: {e.stderr}")
|
||||
failed_workers.append(worker)
|
||||
console.print(f" ✗ Failed to stop {worker}", style="red")
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error(f"Timeout stopping {worker}")
|
||||
failed_workers.append(worker)
|
||||
console.print(f" ✗ Timeout stopping {worker}", style="red")
|
||||
|
||||
if failed_workers:
|
||||
console.print(f"\n⚠️ {len(failed_workers)} worker(s) failed to stop", style="yellow")
|
||||
console.print("💡 Try manually: docker stop " + " ".join(failed_workers), style="dim")
|
||||
return False
|
||||
|
||||
console.print("\n✅ All workers stopped")
|
||||
logger.info("All workers stopped successfully")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error stopping workers: {e}")
|
||||
console.print(f"❌ Unexpected error: {e}", style="red")
|
||||
return False
|
||||
|
||||
def ensure_worker_running(
|
||||
self,
|
||||
worker_info: Dict[str, Any],
|
||||
|
||||
Reference in New Issue
Block a user