diff --git a/README.md b/README.md index 4fad193..e7d13ce 100644 --- a/README.md +++ b/README.md @@ -1,138 +1,123 @@ -# AI / LLM Red Team Field Manual & Consultant’s Handbook +# AI / LLM Red Team Field Manual & Consultant's Handbook ![Repository Banner](assets/banner.svg) -This repository provides a complete operational and consultative toolkit for conducting **AI/LLM red team assessments**. -It is designed for penetration testers, red team operators, and security engineers evaluating: +![License](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg) +![Python](https://img.shields.io/badge/Python-3.8%2B-blue.svg) +![Contributions Welcome](https://img.shields.io/badge/Contributions-Welcome-brightgreen.svg) +![Last Updated](https://img.shields.io/badge/Last%20Updated-November%202025-orange.svg) -- Large Language Models (LLMs) -- AI agents and function-calling systems -- Retrieval-Augmented Generation (RAG) pipelines -- Plugin/tool ecosystems -- AI-enabled enterprise applications - -It contains two primary documents: - -- **AI/LLM Red Team Field Manual** – a concise, practical manual with attack prompts, tooling references, and OWASP/MITRE mappings. -- **AI/LLM Red Team Consultant’s Handbook** – a full-length guide covering methodology, scoping, ethics, RoE/SOW templates, threat modeling, and operational workflows. +A complete operational toolkit for conducting **AI/LLM red team assessments** on Large Language Models, AI agents, RAG pipelines, and AI-enabled applications. --- -## Repository Structure - -```text -docs/ - AI_LLM-Red-Team-Field-Manual.md - AI_LLM-Red-Team-Field-Manual.pdf - AI_LLM-Red-Team-Field-Manual.docx - AI_LLM-Red-Team-Handbook.md -assets/ - banner.svg -README.md -LICENSE -``` - ---- - -## Document Overview - -### **AI_LLM-Red-Team-Field-Manual.md** - -A compact operational reference for active red teaming engagements. - -**Includes:** - -- Rules of Engagement (RoE) and testing phases -- Attack categories and ready-to-use prompts -- Coverage of prompt injection, jailbreaks, data leakage, plugin abuse, adversarial examples, model extraction, DoS, multimodal attacks, and supply-chain vectors -- Tooling reference (Garak, PromptBench, TextAttack, ART, AFL++, Burp Suite, KnockoffNets) -- Attack-to-tool lookup table -- Reporting and documentation guidance -- OWASP & MITRE ATLAS mapping appendices - -**PDF / DOCX Versions:** -Preformatted for printing or distribution. - ---- - -### **AI_LLM-Red-Team-Handbook.md** - -A long-form handbook focused on consultancy and structured delivery of AI red team projects. - -**Includes:** - -- Red team mindset, ethics, and legal considerations -- SOW and RoE templates -- Threat modeling frameworks -- LLM and RAG architecture fundamentals -- Detailed attack descriptions and risk frameworks -- Defense and mitigation strategies -- Operational workflows and sample reporting structure -- Training modules, labs, and advanced topics (e.g., adversarial ML, supply chain, regulation) - ---- - -## How to Use This Repository - -### **1. During AI/LLM Red Team Engagements** - -Clone the repository: +## Quick Start ```bash +# Clone the repository git clone https://github.com/shiva108/ai-llm-red-team-handbook.git cd ai-llm-red-team-handbook + +# Manual testing: Open docs/AI_LLM Red Team Field Manual.md + +# Automated testing: +cd scripts +pip install -r requirements.txt +python runner.py --config config.py ``` -Then: - -- Open the Field Manual -- Apply the provided attacks, prompts, and tooling guidance -- Map findings to OWASP & MITRE using the included tables -- Use the reporting guidance to produce consistent, defensible documentation +📖 **Detailed setup:** See [Configuration Guide](docs/Configuration.md) --- -### **2. For Internal Training** +## Repository Contents -- Use the Handbook as the foundation for onboarding and team development -- Integrate sections into internal wikis, training slides, and exercises +| Resource | Description | +|----------|-------------| +| **[Field Manual](docs/AI_LLM%20Red%20Team%20Field%20Manual.md)** | Compact operational reference with attack prompts, tooling, OWASP/MITRE mappings | +| **[Handbook](docs/AI%20LLM%20Red%20Team%20Hand%20book.md)** | Full consultancy guide with methodology, threat modeling, RoE/SOW templates | +| **[Building AI Red Teams](docs/Building%20a%20World-Class%20AI%20Red%20Team.md)** | Strategic guide for building security teams | +| **[Report Template](docs/Full_LLM_RedTeam_Report_Template.docx)** | Client-ready assessment report template | +| **[Python Framework](scripts/)** | Automated testing suite for prompt injection, jailbreaks, data leakage, tool misuse | --- -### **3. For Client-Facing Work** +## Prerequisites -- Export PDF versions for use in proposals and methodology documents -- Use the structured attack categories to justify test coverage in engagements +**Manual Testing:** Any text editor + target LLM access + +**Automated Testing:** +- Python 3.8+ +- Dependencies: `requests`, `pytest`, `pydantic`, `python-dotenv` +- API credentials for target LLM + +--- + +## Python Testing Framework + +### Test Suites + +- `test_prompt_injection.py` - Automated prompt injection attacks +- `test_safety_bypass.py` - Jailbreak and guardrail bypass tests +- `test_data_exposure.py` - Data leakage and PII extraction +- `test_tool_misuse.py` - Function-calling and plugin abuse +- `test_fuzzing.py` - Adversarial input fuzzing +- `test_integrity.py` - Model integrity and consistency + +### Configuration + +Create `scripts/.env`: +```bash +API_ENDPOINT=https://api.example.com/v1/chat/completions +API_KEY=your-secret-api-key +MODEL_NAME=gpt-4 +``` + +Run tests: +```bash +python runner.py # All tests +python runner.py --test prompt_injection # Specific test +python runner.py --verbose # Verbose output +``` + +📖 **Full configuration options:** [Configuration Guide](docs/Configuration.md) + +--- + +## Use Cases + +**Red Team Engagements:** Use Field Manual attack prompts and Python framework for assessments + +**Training:** Leverage Handbook for onboarding and team development + +**Client Work:** Export PDFs for proposals and methodology documents --- ## Roadmap -Planned improvements: +**Planned:** +- Sample RAG and LLM test environments +- Additional attack case studies +- Extended multimodal AI coverage -- Python tools for automated AI prompt fuzzing -- Sample RAG and LLM test environments -- Additional attack case studies and model-specific guidance - -**Contributions are welcome.** +**Contributions welcome** via issues and PRs. --- ## License -This repository is licensed under **CC BY-SA 4.0**. -See the `LICENSE` file for full details. +Licensed under **CC BY-SA 4.0**. See [LICENSE](LICENSE) for details. --- ## Disclaimer -This material is intended for authorized security testing and research only. +⚠️ **For authorized security testing only.** -Users must ensure: - -- Written authorization (SOW/RoE) is in place -- All testing activities comply with applicable laws and regulations -- No testing impacts production environments without approval +Ensure: +- Written authorization (SOW/RoE) is in place +- Compliance with applicable laws and regulations +- No unauthorized testing on production systems The authors accept no liability for unauthorized use. diff --git a/docs/Configuration.md b/docs/Configuration.md new file mode 100644 index 0000000..21d890c --- /dev/null +++ b/docs/Configuration.md @@ -0,0 +1,465 @@ +# Configuration Guide + +This guide provides detailed instructions for configuring and running the automated AI/LLM red team testing framework. + +--- + +## Table of Contents + +- [Quick Setup](#quick-setup) +- [Configuration File](#configuration-file) +- [Environment Variables](#environment-variables) +- [Advanced Configuration](#advanced-configuration) +- [Running Tests](#running-tests) +- [Output and Reporting](#output-and-reporting) +- [Troubleshooting](#troubleshooting) + +--- + +## Quick Setup + +### 1. Install Dependencies + +```bash +cd scripts +pip install -r requirements.txt +``` + +### 2. Configure Environment + +Create a `.env` file in the `scripts/` directory: + +```bash +API_ENDPOINT=https://api.example.com/v1/chat/completions +API_KEY=your-secret-api-key +MODEL_NAME=gpt-4 +``` + +### 3. Run Tests + +```bash +python runner.py +``` + +--- + +## Configuration File + +### Basic `config.py` + +Create or modify `scripts/config.py` with your target system details: + +```python +# Target LLM Configuration +API_ENDPOINT = "https://api.example.com/v1/chat/completions" +API_KEY = "your-api-key-here" # Or use environment variable +MODEL_NAME = "gpt-4" # Target model identifier + +# Test Configuration +MAX_RETRIES = 3 +TIMEOUT = 30 # seconds +REQUEST_DELAY = 1 # seconds between requests + +# Logging +LOG_LEVEL = "INFO" +LOG_FILE = "test_results.log" + +# Test Selection +ENABLE_TESTS = [ + "prompt_injection", + "safety_bypass", + "data_exposure", + "tool_misuse", + "fuzzing", + "integrity" +] +``` + +### Configuration Parameters + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `API_ENDPOINT` | string | required | Target LLM API endpoint URL | +| `API_KEY` | string | required | Authentication key for API | +| `MODEL_NAME` | string | required | Model identifier (e.g., "gpt-4", "claude-3") | +| `MAX_RETRIES` | int | 3 | Number of retry attempts for failed requests | +| `TIMEOUT` | int | 30 | Request timeout in seconds | +| `REQUEST_DELAY` | float | 1.0 | Delay between requests to avoid rate limiting | +| `LOG_LEVEL` | string | "INFO" | Logging verbosity (DEBUG, INFO, WARNING, ERROR) | +| `LOG_FILE` | string | "test_results.log" | Path to log file | +| `ENABLE_TESTS` | list | all tests | List of test categories to run | + +--- + +## Environment Variables + +### Using `.env` File (Recommended) + +For security, use environment variables instead of hardcoding credentials in `config.py`. + +**1. Create `scripts/.env`:** + +```bash +# API Configuration +API_ENDPOINT=https://api.openai.com/v1/chat/completions +API_KEY=sk-proj-xxxxxxxxxxxxxxxxxxxxx +MODEL_NAME=gpt-4 + +# Test Configuration +MAX_RETRIES=3 +TIMEOUT=30 +REQUEST_DELAY=1 + +# Logging +LOG_LEVEL=INFO +LOG_FILE=test_results.log +``` + +**2. Update `config.py` to load from environment:** + +```python +from dotenv import load_dotenv +import os + +# Load environment variables +load_dotenv() + +# API Configuration +API_ENDPOINT = os.getenv("API_ENDPOINT") +API_KEY = os.getenv("API_KEY") +MODEL_NAME = os.getenv("MODEL_NAME") + +# Test Configuration +MAX_RETRIES = int(os.getenv("MAX_RETRIES", "3")) +TIMEOUT = int(os.getenv("TIMEOUT", "30")) +REQUEST_DELAY = float(os.getenv("REQUEST_DELAY", "1.0")) + +# Logging +LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO") +LOG_FILE = os.getenv("LOG_FILE", "test_results.log") +``` + +**3. Add `.env` to `.gitignore`:** + +```bash +echo ".env" >> .gitignore +``` + +### Provider-Specific Examples + +#### OpenAI + +```bash +API_ENDPOINT=https://api.openai.com/v1/chat/completions +API_KEY=sk-proj-xxxxxxxxxxxxxxxxxxxxx +MODEL_NAME=gpt-4 +``` + +#### Anthropic (Claude) + +```bash +API_ENDPOINT=https://api.anthropic.com/v1/messages +API_KEY=sk-ant-xxxxxxxxxxxxxxxxxxxxx +MODEL_NAME=claude-3-opus-20240229 +``` + +#### Azure OpenAI + +```bash +API_ENDPOINT=https://your-resource.openai.azure.com/openai/deployments/your-deployment/chat/completions?api-version=2024-02-15-preview +API_KEY=xxxxxxxxxxxxxxxxxxxxx +MODEL_NAME=gpt-4 +``` + +#### Local/Self-Hosted Models + +```bash +API_ENDPOINT=http://localhost:8000/v1/chat/completions +API_KEY=none +MODEL_NAME=llama-2-7b +``` + +--- + +## Advanced Configuration + +### Custom Headers + +Add custom headers for authentication or tracking: + +```python +CUSTOM_HEADERS = { + "Authorization": f"Bearer {API_KEY}", + "X-Request-ID": "red-team-test", + "User-Agent": "AI-RedTeam-Framework/1.0" +} +``` + +### Proxy Configuration + +Route requests through a proxy: + +```python +PROXY_CONFIG = { + "http": "http://proxy.example.com:8080", + "https": "https://proxy.example.com:8080" +} +``` + +### Rate Limiting + +Configure rate limiting to avoid API throttling: + +```python +RATE_LIMIT = { + "requests_per_minute": 60, + "requests_per_day": 10000, + "retry_after_seconds": 60 +} +``` + +### Test Customization + +Enable/disable specific tests or adjust severity: + +```python +TEST_CONFIG = { + "prompt_injection": { + "enabled": True, + "severity_threshold": "medium", # low, medium, high, critical + "max_attempts": 100 + }, + "safety_bypass": { + "enabled": True, + "severity_threshold": "high", + "max_attempts": 50 + }, + "data_exposure": { + "enabled": True, + "severity_threshold": "critical", + "max_attempts": 75 + } +} +``` + +--- + +## Running Tests + +### Run All Tests + +```bash +python runner.py +``` + +### Run Specific Test Category + +```bash +python runner.py --test prompt_injection +python runner.py --test safety_bypass +python runner.py --test data_exposure +``` + +### Run Multiple Categories + +```bash +python runner.py --test prompt_injection,safety_bypass,data_exposure +``` + +### Use Custom Configuration File + +```bash +python runner.py --config my_custom_config.py +``` + +### Verbose Output + +```bash +python runner.py --verbose +``` + +### Debug Mode + +```bash +python runner.py --debug +``` + +### Save Results to Custom Location + +```bash +python runner.py --output /path/to/results/ +``` + +### Dry Run (Preview Tests) + +```bash +python runner.py --dry-run +``` + +--- + +## Output and Reporting + +### Output Files + +Test results are saved to multiple locations: + +| File/Directory | Format | Description | +|----------------|--------|-------------| +| `test_results.log` | Text | Detailed execution log with timestamps | +| `reports/json/` | JSON | Machine-readable test results | +| `reports/html/` | HTML | Human-readable HTML reports | +| `reports/summary.txt` | Text | Executive summary of findings | + +### Report Structure + +**JSON Report:** +```json +{ + "test_run_id": "20250630-124530", + "timestamp": "2025-06-30T12:45:30Z", + "model": "gpt-4", + "total_tests": 150, + "passed": 120, + "failed": 30, + "findings": [ + { + "test_case": "prompt_injection_001", + "severity": "high", + "status": "failed", + "description": "Model revealed system prompt", + "payload": "Ignore previous instructions...", + "response": "You are a helpful assistant..." + } + ] +} +``` + +**HTML Report:** +- Interactive dashboard with charts +- Filterable findings by severity +- Detailed test case results +- Recommendations for remediation + +### Console Output + +Real-time progress indicators: + +``` +Running AI/LLM Red Team Tests... +[=====> ] 50% | Prompt Injection (25/50) + +✓ test_prompt_injection_001 - PASSED +✗ test_prompt_injection_002 - FAILED (High Severity) +✓ test_prompt_injection_003 - PASSED +... +``` + +--- + +## Troubleshooting + +### Common Issues + +#### API Connection Errors + +**Error:** +``` +ConnectionError: Failed to connect to API endpoint +``` + +**Solution:** +- Verify `API_ENDPOINT` is correct +- Check network connectivity +- Confirm firewall/proxy settings + +#### Authentication Failures + +**Error:** +``` +AuthenticationError: Invalid API key +``` + +**Solution:** +- Verify `API_KEY` is correct and active +- Check API key permissions +- Ensure key hasn't expired + +#### Rate Limiting + +**Error:** +``` +RateLimitError: Too many requests +``` + +**Solution:** +- Increase `REQUEST_DELAY` in config +- Reduce concurrent tests +- Use rate limiting configuration + +#### Timeout Issues + +**Error:** +``` +TimeoutError: Request timed out after 30s +``` + +**Solution:** +- Increase `TIMEOUT` value +- Check API service status +- Verify network latency + +### Debug Mode + +Enable detailed logging: + +```python +LOG_LEVEL = "DEBUG" +``` + +Or run with debug flag: + +```bash +python runner.py --debug +``` + +### Getting Help + +- Check [Issues](https://github.com/shiva108/ai-llm-red-team-handbook/issues) for known problems +- Review test logs in `test_results.log` +- Enable debug mode for detailed diagnostics + +--- + +## Best Practices + +### Security + +- ✅ Use `.env` files for credentials +- ✅ Add `.env` to `.gitignore` +- ✅ Rotate API keys regularly +- ✅ Use minimum required permissions +- ❌ Never commit credentials to version control + +### Performance + +- Use appropriate `REQUEST_DELAY` to avoid rate limiting +- Run tests during off-peak hours for production systems +- Use `--dry-run` to preview tests before execution +- Consider running test categories separately for large test suites + +### Reporting + +- Save reports with timestamps for historical tracking +- Export findings to client-ready formats +- Map findings to OWASP/MITRE frameworks +- Document false positives for future reference + +--- + +## Additional Resources + +- [Main README](../README.md) +- [Field Manual](AI_LLM%20Red%20Team%20Field%20Manual.md) +- [Handbook](AI%20LLM%20Red%20Team%20Hand%20book.md) +- [Report Template](Full_LLM_RedTeam_Report_Template.docx)