mirror of https://github.com/FuzzingLabs/fuzzforge_ai.git synced 2026-02-13 19:12:44 +00:00

Files

Songbird99 f77c3ff1e9 Feature/litellm proxy (#27 )

* feat: seed governance config and responses routing

* Add env-configurable timeout for proxy providers

* Integrate LiteLLM OTEL collector and update docs

* Make .env.litellm optional for LiteLLM proxy

* Add LiteLLM proxy integration with model-agnostic virtual keys

Changes:
- Bootstrap generates 3 virtual keys with individual budgets (CLI: $100, Task-Agent: $25, Cognee: $50)
- Task-agent loads config at runtime via entrypoint script to wait for bootstrap completion
- All keys are model-agnostic by default (no LITELLM_DEFAULT_MODELS restrictions)
- Bootstrap handles database/env mismatch after docker prune by deleting stale aliases
- CLI and Cognee configured to use LiteLLM proxy with virtual keys
- Added comprehensive documentation in volumes/env/README.md

Technical details:
- task-agent entrypoint waits for keys in .env file before starting uvicorn
- Bootstrap creates/updates TASK_AGENT_API_KEY, COGNEE_API_KEY, and OPENAI_API_KEY
- Removed hardcoded API keys from docker-compose.yml
- All services route through http://localhost:10999 proxy

* Fix CLI not loading virtual keys from global .env

Project .env files with empty OPENAI_API_KEY values were overriding
the global virtual keys. Updated _load_env_file_if_exists to only
override with non-empty values.

* Fix agent executor not passing API key to LiteLLM

The agent was initializing LiteLlm without api_key or api_base,
causing authentication errors when using the LiteLLM proxy. Now
reads from OPENAI_API_KEY/LLM_API_KEY and LLM_ENDPOINT environment
variables and passes them to LiteLlm constructor.

* Auto-populate project .env with virtual key from global config

When running 'ff init', the command now checks for a global
volumes/env/.env file and automatically uses the OPENAI_API_KEY
virtual key if found. This ensures projects work with LiteLLM
proxy out of the box without manual key configuration.

* docs: Update README with LiteLLM configuration instructions

Add note about LITELLM_GEMINI_API_KEY configuration and clarify that OPENAI_API_KEY default value should not be changed as it's used for the LLM proxy.

* Refactor workflow parameters to use JSON Schema defaults

Consolidates parameter defaults into JSON Schema format, removing the separate default_parameters field. Adds extract_defaults_from_json_schema() helper to extract defaults from the standard schema structure. Updates LiteLLM proxy config to use LITELLM_OPENAI_API_KEY environment variable.

* Remove .env.example from task_agent

* Fix MDX syntax error in llm-proxy.md

* fix: apply default parameters from metadata.yaml automatically

Fixed TemporalManager.run_workflow() to correctly apply default parameter
values from workflow metadata.yaml files when parameters are not provided
by the caller.

Previous behavior:
- When workflow_params was empty {}, the condition
  `if workflow_params and 'parameters' in metadata` would fail
- Parameters would not be extracted from schema, resulting in workflows
  receiving only target_id with no other parameters

New behavior:
- Removed the `workflow_params and` requirement from the condition
- Now explicitly checks for defaults in parameter spec
- Applies defaults from metadata.yaml automatically when param not provided
- Workflows receive all parameters with proper fallback:
  provided value > metadata default > None

This makes metadata.yaml the single source of truth for parameter defaults,
removing the need for workflows to implement defensive default handling.

Affected workflows:
- llm_secret_detection (was failing with KeyError)
- All other workflows now benefit from automatic default application

Co-authored-by: tduhamel42 <tduhamel@fuzzinglabs.com>

2025-11-04 14:04:10 +01:00

2.1 KiB

Raw Permalink Blame History

FuzzForge LiteLLM Proxy Configuration

This directory contains configuration for the LiteLLM proxy with model-agnostic virtual keys.

Quick Start (Fresh Clone)

1. Create Your `.env` File

cp .env.template .env

2. Add Your Provider API Keys

Edit .env and add your real API keys:

LITELLM_OPENAI_API_KEY=sk-proj-YOUR-OPENAI-KEY-HERE
LITELLM_ANTHROPIC_API_KEY=sk-ant-api03-YOUR-ANTHROPIC-KEY-HERE

3. Start Services

cd ../..  # Back to repo root
COMPOSE_PROFILES=secrets docker compose up -d

Bootstrap will automatically:

Generate 3 virtual keys with individual budgets
Write them to your .env file
No model restrictions (model-agnostic)

Files

.env.template - Clean template (checked into git)
.env - Your real keys (git ignored, you create this)
.env.example - Legacy example

Virtual Keys (Auto-Generated)

Bootstrap creates 3 keys with budget controls:

Key	Budget	Duration	Used By
`OPENAI_API_KEY`	$100	30 days	CLI, SDK
`TASK_AGENT_API_KEY`	$25	30 days	Task Agent
`COGNEE_API_KEY`	$50	30 days	Cognee

All keys are model-agnostic by default (no restrictions).

Using Models

Registered models in volumes/litellm/proxy_config.yaml:

gpt-5-mini → openai/gpt-5-mini
claude-sonnet-4-5 → anthropic/claude-sonnet-4-5-20250929
text-embedding-3-large → openai/text-embedding-3-large

Use Registered Aliases:

fuzzforge workflow run llm_secret_detection . -n llm_model=gpt-5-mini
fuzzforge workflow run llm_secret_detection . -n llm_model=claude-sonnet-4-5

Use Any Model (Direct):

# Works without registering first!
fuzzforge workflow run llm_secret_detection . -n llm_model=openai/gpt-5-nano

Proxy UI

http://localhost:10999/ui

User: fuzzforge / Pass: fuzzforge123

Troubleshooting

# Check bootstrap logs
docker compose logs llm-proxy-bootstrap

# Verify keys generated
grep "API_KEY=" .env | grep -v "^#" | grep -v "your-"

# Restart services
docker compose restart llm-proxy task-agent

2.1 KiB Raw Permalink Blame History