From e33c6117115fad39eb5451605457a91f784c7c2a Mon Sep 17 00:00:00 2001 From: tduhamel42 Date: Tue, 4 Nov 2025 11:25:58 +0100 Subject: [PATCH] chore: add worker startup documentation and cleanup .gitignore - Add workflow-to-worker mapping tables across documentation - Update troubleshooting guide with worker requirements section - Enhance getting started guide with worker examples - Add quick reference to docker setup guide - Add WEEK_SUMMARY*.md pattern to .gitignore --- .gitignore | 4 +++ README.md | 10 +++++++ docs/docs/how-to/docker-setup.md | 10 +++++++ docs/docs/how-to/troubleshooting.md | 40 +++++++++++++++++++++++++++ docs/docs/tutorial/getting-started.md | 36 ++++++++++++++++++++++-- 5 files changed, 97 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore index 194c780..a8d6e44 100644 --- a/.gitignore +++ b/.gitignore @@ -309,3 +309,7 @@ test_projects/*/.git-credentials test_projects/*/credentials.* test_projects/*/api_keys.* test_projects/*/ci-*.sh + +# -------------------- Internal Documentation -------------------- +# Weekly summaries and temporary project documentation +WEEK_SUMMARY*.md diff --git a/README.md b/README.md index 4eb2028..4e89bc3 100644 --- a/README.md +++ b/README.md @@ -165,6 +165,16 @@ docker compose up -d worker-python > > Workers don't auto-start by default (saves RAM). Start the worker you need before running workflows. +**Workflow-to-Worker Quick Reference:** + +| Workflow | Worker Required | Startup Command | +|----------|----------------|-----------------| +| `security_assessment`, `python_sast`, `llm_analysis`, `atheris_fuzzing` | worker-python | `docker compose up -d worker-python` | +| `android_static_analysis` | worker-android | `docker compose up -d worker-android` | +| `cargo_fuzzing` | worker-rust | `docker compose up -d worker-rust` | +| `ossfuzz_campaign` | worker-ossfuzz | `docker compose up -d worker-ossfuzz` | +| `llm_secret_detection`, `trufflehog_detection`, `gitleaks_detection` | worker-secrets | `docker compose up -d worker-secrets` | + ```bash # 5. Run your first workflow (files are automatically uploaded) cd test_projects/vulnerable_app/ diff --git a/docs/docs/how-to/docker-setup.md b/docs/docs/how-to/docker-setup.md index 5496e62..f4c2fa0 100644 --- a/docs/docs/how-to/docker-setup.md +++ b/docs/docs/how-to/docker-setup.md @@ -110,6 +110,16 @@ fuzzforge workflow run secret_detection ./codebase ### Manual Worker Management +**Quick Reference - Workflow to Worker Mapping:** + +| Workflow | Worker Service | Docker Command | +|----------|----------------|----------------| +| `security_assessment`, `python_sast`, `llm_analysis`, `atheris_fuzzing` | worker-python | `docker compose up -d worker-python` | +| `android_static_analysis` | worker-android | `docker compose up -d worker-android` | +| `cargo_fuzzing` | worker-rust | `docker compose up -d worker-rust` | +| `ossfuzz_campaign` | worker-ossfuzz | `docker compose up -d worker-ossfuzz` | +| `llm_secret_detection`, `trufflehog_detection`, `gitleaks_detection` | worker-secrets | `docker compose up -d worker-secrets` | + FuzzForge CLI provides convenient commands for managing workers: ```bash diff --git a/docs/docs/how-to/troubleshooting.md b/docs/docs/how-to/troubleshooting.md index 7554eda..38165ae 100644 --- a/docs/docs/how-to/troubleshooting.md +++ b/docs/docs/how-to/troubleshooting.md @@ -106,6 +106,46 @@ File upload to MinIO failed or worker can't download target. ``` - Reduce the number of concurrent workflows if your system is resource-constrained. +### Workflow requires worker not running + +**What's happening?** +You see a warning message like: +``` +⚠️ Could not check worker requirements: Cannot find docker-compose.yml. + Ensure backend is running, run from FuzzForge directory, or set + FUZZFORGE_ROOT environment variable. + Continuing without worker management... +``` + +Or the workflow fails to start because the required worker isn't running. + +**How to fix:** +Start the worker required for your workflow before running it: + +| Workflow | Worker Required | Startup Command | +|----------|----------------|-----------------| +| `android_static_analysis` | worker-android | `docker compose up -d worker-android` | +| `security_assessment` | worker-python | `docker compose up -d worker-python` | +| `python_sast` | worker-python | `docker compose up -d worker-python` | +| `llm_analysis` | worker-python | `docker compose up -d worker-python` | +| `atheris_fuzzing` | worker-python | `docker compose up -d worker-python` | +| `ossfuzz_campaign` | worker-ossfuzz | `docker compose up -d worker-ossfuzz` | +| `cargo_fuzzing` | worker-rust | `docker compose up -d worker-rust` | +| `llm_secret_detection` | worker-secrets | `docker compose up -d worker-secrets` | +| `trufflehog_detection` | worker-secrets | `docker compose up -d worker-secrets` | +| `gitleaks_detection` | worker-secrets | `docker compose up -d worker-secrets` | + +**Check worker status:** +```bash +# Check if a specific worker is running +docker compose ps worker-android + +# Check all workers +docker compose ps | grep worker +``` + +**Note:** Workers don't auto-start by default to save system resources. For more details on worker management, see the [Docker Setup guide](docker-setup.md#worker-management). + --- ## Service Connectivity Issues diff --git a/docs/docs/tutorial/getting-started.md b/docs/docs/tutorial/getting-started.md index 473ab55..b376258 100644 --- a/docs/docs/tutorial/getting-started.md +++ b/docs/docs/tutorial/getting-started.md @@ -89,9 +89,26 @@ curl http://localhost:8000/health # Should return: {"status":"healthy"} ``` -### Start the Python Worker +### Start Workers for Your Workflows -Workers don't auto-start by default (saves RAM). Start the Python worker for your first workflow: +Workers don't auto-start by default (saves RAM). You need to start the worker required for the workflow you want to run. + +**Workflow-to-Worker Mapping:** + +| Workflow | Worker Required | Startup Command | +|----------|----------------|-----------------| +| `security_assessment` | worker-python | `docker compose up -d worker-python` | +| `python_sast` | worker-python | `docker compose up -d worker-python` | +| `llm_analysis` | worker-python | `docker compose up -d worker-python` | +| `atheris_fuzzing` | worker-python | `docker compose up -d worker-python` | +| `android_static_analysis` | worker-android | `docker compose up -d worker-android` | +| `cargo_fuzzing` | worker-rust | `docker compose up -d worker-rust` | +| `ossfuzz_campaign` | worker-ossfuzz | `docker compose up -d worker-ossfuzz` | +| `llm_secret_detection` | worker-secrets | `docker compose up -d worker-secrets` | +| `trufflehog_detection` | worker-secrets | `docker compose up -d worker-secrets` | +| `gitleaks_detection` | worker-secrets | `docker compose up -d worker-secrets` | + +**For your first workflow (security_assessment), start the Python worker:** ```bash # Start the Python worker @@ -102,7 +119,20 @@ docker compose ps worker-python # Should show: Up (healthy) ``` -**Note:** Workers use Docker Compose profiles and only start when needed. For your first workflow run, it's safer to start the worker manually. Later, the CLI can auto-start workers on demand. +**For other workflows, start the appropriate worker:** + +```bash +# Example: For Android analysis +docker compose up -d worker-android + +# Example: For Rust fuzzing +docker compose up -d worker-rust + +# Check all running workers +docker compose ps | grep worker +``` + +**Note:** Workers use Docker Compose profiles and only start when needed. For your first workflow run, it's safer to start the worker manually. Later, the CLI can auto-start workers on demand. If you see a warning about worker requirements, ensure you've started the correct worker for your workflow. ## Step 4: Install the CLI (Optional but Recommended)