Feature/litellm proxy (#27)

* feat: seed governance config and responses routing

* Add env-configurable timeout for proxy providers

* Integrate LiteLLM OTEL collector and update docs

* Make .env.litellm optional for LiteLLM proxy

* Add LiteLLM proxy integration with model-agnostic virtual keys

Changes:
- Bootstrap generates 3 virtual keys with individual budgets (CLI: $100, Task-Agent: $25, Cognee: $50)
- Task-agent loads config at runtime via entrypoint script to wait for bootstrap completion
- All keys are model-agnostic by default (no LITELLM_DEFAULT_MODELS restrictions)
- Bootstrap handles database/env mismatch after docker prune by deleting stale aliases
- CLI and Cognee configured to use LiteLLM proxy with virtual keys
- Added comprehensive documentation in volumes/env/README.md

Technical details:
- task-agent entrypoint waits for keys in .env file before starting uvicorn
- Bootstrap creates/updates TASK_AGENT_API_KEY, COGNEE_API_KEY, and OPENAI_API_KEY
- Removed hardcoded API keys from docker-compose.yml
- All services route through http://localhost:10999 proxy

* Fix CLI not loading virtual keys from global .env

Project .env files with empty OPENAI_API_KEY values were overriding
the global virtual keys. Updated _load_env_file_if_exists to only
override with non-empty values.

* Fix agent executor not passing API key to LiteLLM

The agent was initializing LiteLlm without api_key or api_base,
causing authentication errors when using the LiteLLM proxy. Now
reads from OPENAI_API_KEY/LLM_API_KEY and LLM_ENDPOINT environment
variables and passes them to LiteLlm constructor.

* Auto-populate project .env with virtual key from global config

When running 'ff init', the command now checks for a global
volumes/env/.env file and automatically uses the OPENAI_API_KEY
virtual key if found. This ensures projects work with LiteLLM
proxy out of the box without manual key configuration.

* docs: Update README with LiteLLM configuration instructions

Add note about LITELLM_GEMINI_API_KEY configuration and clarify that OPENAI_API_KEY default value should not be changed as it's used for the LLM proxy.

* Refactor workflow parameters to use JSON Schema defaults

Consolidates parameter defaults into JSON Schema format, removing the separate default_parameters field. Adds extract_defaults_from_json_schema() helper to extract defaults from the standard schema structure. Updates LiteLLM proxy config to use LITELLM_OPENAI_API_KEY environment variable.

* Remove .env.example from task_agent

* Fix MDX syntax error in llm-proxy.md

* fix: apply default parameters from metadata.yaml automatically

Fixed TemporalManager.run_workflow() to correctly apply default parameter
values from workflow metadata.yaml files when parameters are not provided
by the caller.

Previous behavior:
- When workflow_params was empty {}, the condition
  `if workflow_params and 'parameters' in metadata` would fail
- Parameters would not be extracted from schema, resulting in workflows
  receiving only target_id with no other parameters

New behavior:
- Removed the `workflow_params and` requirement from the condition
- Now explicitly checks for defaults in parameter spec
- Applies defaults from metadata.yaml automatically when param not provided
- Workflows receive all parameters with proper fallback:
  provided value > metadata default > None

This makes metadata.yaml the single source of truth for parameter defaults,
removing the need for workflows to implement defensive default handling.

Affected workflows:
- llm_secret_detection (was failing with KeyError)
- All other workflows now benefit from automatic default application

Co-authored-by: tduhamel42 <tduhamel@fuzzinglabs.com>

docker-compose.yml

@@ -144,6 +144,103 @@ services:
     networks:
       - fuzzforge-network
 
+  # ============================================================================
+  # LLM Proxy - LiteLLM Gateway
+  # ============================================================================
+  llm-proxy:
+    image: ghcr.io/berriai/litellm:main-stable
+    container_name: fuzzforge-llm-proxy
+    depends_on:
+      llm-proxy-db:
+        condition: service_healthy
+      otel-collector:
+        condition: service_started
+    env_file:
+      - ./volumes/env/.env
+    environment:
+      PORT: 4000
+      DATABASE_URL: postgresql://litellm:litellm@llm-proxy-db:5432/litellm
+      STORE_MODEL_IN_DB: "True"
+      UI_USERNAME: ${UI_USERNAME:-fuzzforge}
+      UI_PASSWORD: ${UI_PASSWORD:-fuzzforge123}
+      OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4317
+      OTEL_EXPORTER_OTLP_PROTOCOL: grpc
+      ANTHROPIC_API_KEY: ${LITELLM_ANTHROPIC_API_KEY:-}
+      OPENAI_API_KEY: ${LITELLM_OPENAI_API_KEY:-}
+    command:
+      - "--config"
+      - "/etc/litellm/proxy_config.yaml"
+    ports:
+      - "10999:4000"   # Web UI + OpenAI-compatible API
+    volumes:
+      - litellm_proxy_data:/var/lib/litellm
+      - ./volumes/litellm/proxy_config.yaml:/etc/litellm/proxy_config.yaml:ro
+    networks:
+      - fuzzforge-network
+    healthcheck:
+      test: ["CMD-SHELL", "wget --no-verbose --tries=1 http://localhost:4000/health/liveliness || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 40s
+    restart: unless-stopped
+
+  otel-collector:
+    image: otel/opentelemetry-collector:latest
+    container_name: fuzzforge-otel-collector
+    command: ["--config=/etc/otel-collector/config.yaml"]
+    volumes:
+      - ./volumes/otel/collector-config.yaml:/etc/otel-collector/config.yaml:ro
+    ports:
+      - "4317:4317"
+      - "4318:4318"
+    networks:
+      - fuzzforge-network
+    restart: unless-stopped
+
+  llm-proxy-db:
+    image: postgres:16
+    container_name: fuzzforge-llm-proxy-db
+    environment:
+      POSTGRES_DB: litellm
+      POSTGRES_USER: litellm
+      POSTGRES_PASSWORD: litellm
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -d litellm -U litellm"]
+      interval: 5s
+      timeout: 5s
+      retries: 12
+    volumes:
+      - litellm_proxy_db:/var/lib/postgresql/data
+    networks:
+      - fuzzforge-network
+    restart: unless-stopped
+
+  # ============================================================================
+  # LLM Proxy Bootstrap - Seed providers and virtual keys
+  # ============================================================================
+  llm-proxy-bootstrap:
+    image: python:3.11-slim
+    container_name: fuzzforge-llm-proxy-bootstrap
+    depends_on:
+      llm-proxy:
+        condition: service_started
+    env_file:
+      - ./volumes/env/.env
+    environment:
+      PROXY_BASE_URL: http://llm-proxy:4000
+      ENV_FILE_PATH: /bootstrap/env/.env
+      UI_USERNAME: ${UI_USERNAME:-fuzzforge}
+      UI_PASSWORD: ${UI_PASSWORD:-fuzzforge123}
+    volumes:
+      - ./docker/scripts/bootstrap_llm_proxy.py:/app/bootstrap.py:ro
+      - ./volumes/env:/bootstrap/env
+      - litellm_proxy_data:/bootstrap/data
+    networks:
+      - fuzzforge-network
+    command: ["python", "/app/bootstrap.py"]
+    restart: "no"
+
   # ============================================================================
   # Vertical Worker: Rust/Native Security
   # ============================================================================
@@ -458,10 +555,11 @@ services:
       context: ./ai/agents/task_agent
       dockerfile: Dockerfile
     container_name: fuzzforge-task-agent
+    depends_on:
+      llm-proxy-bootstrap:
+        condition: service_completed_successfully
     ports:
       - "10900:8000"
-    env_file:
-      - ./volumes/env/.env
     environment:
       - PORT=8000
       - PYTHONUNBUFFERED=1
@@ -558,6 +656,10 @@ volumes:
     name: fuzzforge_worker_ossfuzz_cache
   worker_ossfuzz_build:
     name: fuzzforge_worker_ossfuzz_build
+  litellm_proxy_data:
+    name: fuzzforge_litellm_proxy_data
+  litellm_proxy_db:
+    name: fuzzforge_litellm_proxy_db
   # Add more worker caches as you add verticals:
   # worker_web_cache:
   # worker_ios_cache:
@@ -591,6 +693,7 @@ networks:
 # 4. Web UIs:
 #    - Temporal UI: http://localhost:8233
 #    - MinIO Console: http://localhost:9001 (user: fuzzforge, pass: fuzzforge123)
+#    - LiteLLM Proxy: http://localhost:10999
 #
 # 5. Resource Usage (Baseline):
 #    - Temporal: ~500MB