CalvinBackup/fuzzforge_ai
1
0
Fork 0
mirror of https://github.com/FuzzingLabs/fuzzforge_ai.git synced 2026-02-14 19:12:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Feature/litellm proxy (#27)

Browse Source 
* feat: seed governance config and responses routing

* Add env-configurable timeout for proxy providers

* Integrate LiteLLM OTEL collector and update docs

* Make .env.litellm optional for LiteLLM proxy

* Add LiteLLM proxy integration with model-agnostic virtual keys

Changes:
- Bootstrap generates 3 virtual keys with individual budgets (CLI: $100, Task-Agent: $25, Cognee: $50)
- Task-agent loads config at runtime via entrypoint script to wait for bootstrap completion
- All keys are model-agnostic by default (no LITELLM_DEFAULT_MODELS restrictions)
- Bootstrap handles database/env mismatch after docker prune by deleting stale aliases
- CLI and Cognee configured to use LiteLLM proxy with virtual keys
- Added comprehensive documentation in volumes/env/README.md

Technical details:
- task-agent entrypoint waits for keys in .env file before starting uvicorn
- Bootstrap creates/updates TASK_AGENT_API_KEY, COGNEE_API_KEY, and OPENAI_API_KEY
- Removed hardcoded API keys from docker-compose.yml
- All services route through http://localhost:10999 proxy

* Fix CLI not loading virtual keys from global .env

Project .env files with empty OPENAI_API_KEY values were overriding
the global virtual keys. Updated _load_env_file_if_exists to only
override with non-empty values.

* Fix agent executor not passing API key to LiteLLM

The agent was initializing LiteLlm without api_key or api_base,
causing authentication errors when using the LiteLLM proxy. Now
reads from OPENAI_API_KEY/LLM_API_KEY and LLM_ENDPOINT environment
variables and passes them to LiteLlm constructor.

* Auto-populate project .env with virtual key from global config

When running 'ff init', the command now checks for a global
volumes/env/.env file and automatically uses the OPENAI_API_KEY
virtual key if found. This ensures projects work with LiteLLM
proxy out of the box without manual key configuration.

* docs: Update README with LiteLLM configuration instructions

Add note about LITELLM_GEMINI_API_KEY configuration and clarify that OPENAI_API_KEY default value should not be changed as it's used for the LLM proxy.

* Refactor workflow parameters to use JSON Schema defaults

Consolidates parameter defaults into JSON Schema format, removing the separate default_parameters field. Adds extract_defaults_from_json_schema() helper to extract defaults from the standard schema structure. Updates LiteLLM proxy config to use LITELLM_OPENAI_API_KEY environment variable.

* Remove .env.example from task_agent

* Fix MDX syntax error in llm-proxy.md

* fix: apply default parameters from metadata.yaml automatically

Fixed TemporalManager.run_workflow() to correctly apply default parameter
values from workflow metadata.yaml files when parameters are not provided
by the caller.

Previous behavior:
- When workflow_params was empty {}, the condition
  `if workflow_params and 'parameters' in metadata` would fail
- Parameters would not be extracted from schema, resulting in workflows
  receiving only target_id with no other parameters

New behavior:
- Removed the `workflow_params and` requirement from the condition
- Now explicitly checks for defaults in parameter spec
- Applies defaults from metadata.yaml automatically when param not provided
- Workflows receive all parameters with proper fallback:
  provided value > metadata default > None

This makes metadata.yaml the single source of truth for parameter defaults,
removing the need for workflows to implement defensive default handling.

Affected workflows:
- llm_secret_detection (was failing with KeyError)
- All other workflows now benefit from automatic default application

Co-authored-by: tduhamel42 <tduhamel@fuzzinglabs.com>
This commit is contained in:
Songbird99
2025-10-26 12:51:53 +01:00
committed by tduhamel42
parent bd94d19d34
commit f77c3ff1e9
29 changed files with 1869 additions and 106 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
cli/src/fuzzforge_cli/commands/init.py
View File

@@ -187,19 +187,40 @@ def _ensure_env_file(fuzzforge_dir: Path, force: bool) -> None:
console.print("🧠 Configuring AI environment...")
console.print(" • Default LLM provider: openai")
console.print(" • Default LLM model: gpt-5-mini")
console.print(" • Default LLM model: litellm_proxy/gpt-5-mini")
console.print(" • To customise provider/model later, edit .fuzzforge/.env")
llm_provider = "openai"
llm_model = "gpt-5-mini"
llm_model = "litellm_proxy/gpt-5-mini"
# Check for global virtual keys from volumes/env/.env
global_env_key = None
for parent in fuzzforge_dir.parents:
global_env = parent / "volumes" / "env" / ".env"
if global_env.exists():
try:
for line in global_env.read_text(encoding="utf-8").splitlines():
if line.strip().startswith("OPENAI_API_KEY=") and "=" in line:
key_value = line.split("=", 1)[1].strip()
if key_value and not key_value.startswith("your-") and key_value.startswith("sk-"):
global_env_key = key_value
console.print(f" • Found virtual key in {global_env.relative_to(parent)}")
break
except Exception:
pass
break
api_key = Prompt.ask(
"OpenAI API key (leave blank to fill manually)",
"OpenAI API key (leave blank to use global virtual key)" if global_env_key else "OpenAI API key (leave blank to fill manually)",
default="",
show_default=False,
console=console,
)
# Use global key if user didn't provide one
if not api_key and global_env_key:
api_key = global_env_key
session_db_path = fuzzforge_dir / "fuzzforge_sessions.db"
session_db_rel = session_db_path.relative_to(fuzzforge_dir.parent)
@@ -210,14 +231,20 @@ def _ensure_env_file(fuzzforge_dir: Path, force: bool) -> None:
f"LLM_PROVIDER={llm_provider}",
f"LLM_MODEL={llm_model}",
f"LITELLM_MODEL={llm_model}",
"LLM_ENDPOINT=http://localhost:10999",
"LLM_API_KEY=",
"LLM_EMBEDDING_MODEL=litellm_proxy/text-embedding-3-large",
"LLM_EMBEDDING_ENDPOINT=http://localhost:10999",
f"OPENAI_API_KEY={api_key}",
"FUZZFORGE_MCP_URL=http://localhost:8010/mcp",
"",
"# Cognee configuration mirrors the primary LLM by default",
f"LLM_COGNEE_PROVIDER={llm_provider}",
f"LLM_COGNEE_MODEL={llm_model}",
f"LLM_COGNEE_API_KEY={api_key}",
"LLM_COGNEE_ENDPOINT=",
"LLM_COGNEE_ENDPOINT=http://localhost:10999",
"LLM_COGNEE_API_KEY=",
"LLM_COGNEE_EMBEDDING_MODEL=litellm_proxy/text-embedding-3-large",
"LLM_COGNEE_EMBEDDING_ENDPOINT=http://localhost:10999",
"COGNEE_MCP_URL=",
"",
"# Session persistence options: inmemory | sqlite",
@@ -239,6 +266,8 @@ def _ensure_env_file(fuzzforge_dir: Path, force: bool) -> None:
for line in env_lines:
if line.startswith("OPENAI_API_KEY="):
template_lines.append("OPENAI_API_KEY=")
elif line.startswith("LLM_API_KEY="):
template_lines.append("LLM_API_KEY=")
elif line.startswith("LLM_COGNEE_API_KEY="):
template_lines.append("LLM_COGNEE_API_KEY=")
else:

95
cli/src/fuzzforge_cli/config.py
View File

@@ -28,6 +28,58 @@ try: # Optional dependency; fall back if not installed
except ImportError: # pragma: no cover - optional dependency
load_dotenv = None
def _load_env_file_if_exists(path: Path, override: bool = False) -> bool:
if not path.exists():
return False
# Always use manual parsing to handle empty values correctly
try:
for line in path.read_text(encoding="utf-8").splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#") or "=" not in stripped:
continue
key, value = stripped.split("=", 1)
key = key.strip()
value = value.strip()
if override:
# Only override if value is non-empty
if value:
os.environ[key] = value
else:
# Set if not already in environment and value is non-empty
if key not in os.environ and value:
os.environ[key] = value
return True
except Exception: # pragma: no cover - best effort fallback
return False
def _find_shared_env_file(project_dir: Path) -> Path | None:
for directory in [project_dir] + list(project_dir.parents):
candidate = directory / "volumes" / "env" / ".env"
if candidate.exists():
return candidate
return None
def load_project_env(project_dir: Optional[Path] = None) -> Path | None:
"""Load project-local env, falling back to shared volumes/env/.env."""
project_dir = Path(project_dir or Path.cwd())
shared_env = _find_shared_env_file(project_dir)
loaded_shared = False
if shared_env:
loaded_shared = _load_env_file_if_exists(shared_env, override=False)
project_env = project_dir / ".fuzzforge" / ".env"
if _load_env_file_if_exists(project_env, override=True):
return project_env
if loaded_shared:
return shared_env
return None
import yaml
from pydantic import BaseModel, Field
@@ -312,23 +364,7 @@ class ProjectConfigManager:
if not cognee.get("enabled", True):
return
# Load project-specific environment overrides from .fuzzforge/.env if available
env_file = self.project_dir / ".fuzzforge" / ".env"
if env_file.exists():
if load_dotenv:
load_dotenv(env_file, override=False)
else:
try:
for line in env_file.read_text(encoding="utf-8").splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#"):
continue
if "=" not in stripped:
continue
key, value = stripped.split("=", 1)
os.environ.setdefault(key.strip(), value.strip())
except Exception: # pragma: no cover - best effort fallback
pass
load_project_env(self.project_dir)
backend_access = "true" if cognee.get("backend_access_control", True) else "false"
os.environ["ENABLE_BACKEND_ACCESS_CONTROL"] = backend_access
@@ -374,6 +410,17 @@ class ProjectConfigManager:
"OPENAI_API_KEY",
)
endpoint = _env("LLM_COGNEE_ENDPOINT", "COGNEE_LLM_ENDPOINT", "LLM_ENDPOINT")
embedding_model = _env(
"LLM_COGNEE_EMBEDDING_MODEL",
"COGNEE_LLM_EMBEDDING_MODEL",
"LLM_EMBEDDING_MODEL",
)
embedding_endpoint = _env(
"LLM_COGNEE_EMBEDDING_ENDPOINT",
"COGNEE_LLM_EMBEDDING_ENDPOINT",
"LLM_EMBEDDING_ENDPOINT",
"LLM_ENDPOINT",
)
api_version = _env(
"LLM_COGNEE_API_VERSION",
"COGNEE_LLM_API_VERSION",
@@ -398,6 +445,20 @@ class ProjectConfigManager:
os.environ.setdefault("OPENAI_API_KEY", api_key)
if endpoint:
os.environ["LLM_ENDPOINT"] = endpoint
os.environ.setdefault("LLM_API_BASE", endpoint)
os.environ.setdefault("LLM_EMBEDDING_ENDPOINT", endpoint)
os.environ.setdefault("LLM_EMBEDDING_API_BASE", endpoint)
os.environ.setdefault("OPENAI_API_BASE", endpoint)
# Set LiteLLM proxy environment variables for SDK usage
os.environ.setdefault("LITELLM_PROXY_API_BASE", endpoint)
if api_key:
# Set LiteLLM proxy API key from the virtual key
os.environ.setdefault("LITELLM_PROXY_API_KEY", api_key)
if embedding_model:
os.environ["LLM_EMBEDDING_MODEL"] = embedding_model
if embedding_endpoint:
os.environ["LLM_EMBEDDING_ENDPOINT"] = embedding_endpoint
os.environ.setdefault("LLM_EMBEDDING_API_BASE", embedding_endpoint)
if api_version:
os.environ["LLM_API_VERSION"] = api_version
if max_tokens:

5
cli/src/fuzzforge_cli/main.py
View File

@@ -19,6 +19,8 @@ from rich.traceback import install
from typing import Optional, List
import sys
from .config import load_project_env
from .commands import (
workflows,
workflow_exec,
@@ -33,6 +35,9 @@ from .fuzzy import enhanced_command_not_found_handler
# Install rich traceback handler
install(show_locals=True)
# Ensure environment variables are available before command execution
load_project_env()
# Create console for rich output
console = Console()