mirror of
https://github.com/Shiva108/ai-llm-red-team-handbook.git
synced 2026-07-11 06:53:40 +02:00
shiva108
271 lines
9.7 KiB
Markdown
271 lines
9.7 KiB
Markdown
# AI / LLM Red Team Field Manual & Consultant's Handbook
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A comprehensive operational toolkit for conducting **AI/LLM red team assessments** on Large Language Models, AI agents, RAG pipelines, and AI-enabled applications. This repository provides both tactical field guidance and strategic consulting frameworks.
|
|
|
|
📖 **GitBook Navigation:** See [SUMMARY.md](docs/SUMMARY.md) for the complete chapter structure.
|
|
|
|
---
|
|
|
|
## 📚 What's Inside
|
|
|
|
This repository contains three core resources:
|
|
|
|
### 1. **AI LLM Red Team Handbook** (46 Chapters, Fully Standardized, GitBook-ready)
|
|
|
|
A comprehensive consultancy guide with all chapters now featuring standardized metadata, abstracts, and consistent structure:
|
|
|
|
- **Part I: Professional Foundations (Chapters 1-4)** - Ethics, legal framework, mindset, and engagement setup
|
|
- **Part II: Project Preparation (Chapters 5-8)** - SOW, threat modeling, coping, and lab setup
|
|
- **Part III: Technical Fundamentals (Chapters 9-11)** - LLM architectures and components
|
|
- **Part IV: Pipeline Security (Chapters 12-13)** - RAG and supply chain
|
|
- **Part V: Attacks & Techniques (Chapters 14-24)** - Comprehensive coverage of all major LLM attack vectors
|
|
- **Part VI: Defense & Mitigation (Chapters 25-30)** - Adversarial ML and advanced defense
|
|
- **Part VII: Advanced Operations (Chapters 31-39)** - Reporting, remediation, and automation
|
|
- **Part VIII: Advanced Topics (Chapters 40-46)** - Future threats, compliance, and program building
|
|
|
|
#### Fully Complete Chapters
|
|
|
|
1. Introduction to AI Red Teaming _(Beginner, 15 min)_
|
|
2. Ethics, Legal, and Stakeholder Communication _(Beginner, 18 min)_
|
|
3. The Red Teamer's Mindset _(Beginner, 12 min)_
|
|
4. SOW, Rules of Engagement, and Client Onboarding _(Intermediate, 20 min)_
|
|
5. Threat Modeling and Risk Analysis _(Intermediate, 16 min)_
|
|
6. Scoping an Engagement _(Intermediate, 14 min)_
|
|
7. Lab Setup and Environmental Safety _(Intermediate, 25 min, Hands-on)_
|
|
8. Evidence, Documentation, and Chain of Custody _(Intermediate, 18 min, Hands-on)_
|
|
9. LLM Architectures and System Components _(Intermediate, 22 min, Hands-on)_
|
|
10. Tokenization, Context, and Generation _(Intermediate, 20 min, Hands-on)_
|
|
11. Plugins, Extensions, and External APIs _(Intermediate, 16 min)_
|
|
12. Retrieval-Augmented Generation (RAG) Pipelines _(Advanced, 24 min, Hands-on)_
|
|
13. Data Provenance and Supply Chain Security _(Intermediate, 18 min)_
|
|
14. Prompt Injection _(Intermediate, ~30 min, Hands-on)_
|
|
15. Data Leakage and Extraction _(Intermediate, ~30 min, Hands-on)_
|
|
16. Jailbreaks and Bypass Techniques _(Intermediate, ~20 min, Hands-on)_
|
|
17. Plugin and API Exploitation _(Advanced, ~25 min, Hands-on)_
|
|
18. Evasion, Obfuscation, and Adversarial Inputs _(Advanced, ~20 min, Hands-on)_
|
|
19. Training Data Poisoning _(Advanced, ~18 min, Hands-on)_
|
|
20. Model Theft and Membership Inference _(Advanced, ~20 min, Hands-on)_
|
|
21. Model DoS and Resource Exhaustion _(Advanced, ~18 min, Hands-on)_
|
|
22. Cross-Modal and Multimodal Attacks _(Advanced, ~20 min, Hands-on)_
|
|
23. Advanced Persistence and Chaining _(Advanced, ~18 min, Hands-on)_
|
|
24. Social Engineering with LLMs _(Intermediate, ~20 min, Hands-on)_
|
|
25. Advanced Adversarial ML _(Advanced, ~25 min)_
|
|
|
|
#### Additional Content
|
|
|
|
- Chapter 36: Reporting and Communication
|
|
- Chapter 37: Remediation Strategies
|
|
- Chapter 38: Continuous Red Teaming
|
|
- Chapter 45: Building an AI Red Team Program
|
|
|
|
_Remaining chapters are currently in development as stubs._
|
|
|
|
#### Chapter Features
|
|
|
|
- ✅ **Standardized Metadata**: Category, difficulty, time estimates, prerequisites
|
|
- ✅ **Compelling Abstracts**: 2-3 sentence chapter summaries
|
|
- ✅ **Theoretical Foundations**: Attack mechanisms and research citations (Ch 14-24)
|
|
- ✅ **Research Landscapes**: Evolution of attacks and current gaps (Ch 14-24)
|
|
- ✅ **Quick References**: Attack vectors, detection, mitigation (Ch 14-24)
|
|
- ✅ **Checklists**: Pre/post-engagement validation
|
|
|
|
📖 **GitBook Navigation:** See [SUMMARY.md](docs/SUMMARY.md) for the complete chapter structure.
|
|
|
|
### 2. **AI LLM Red Team Field Manual** (64KB)
|
|
|
|
Compact operational reference for field use:
|
|
|
|
- Quick-reference attack prompts and payloads
|
|
- Testing checklists and methodology
|
|
- Tool commands and configurations
|
|
- OWASP Top 10 for LLMs mapping
|
|
- MITRE ATLAS framework alignment
|
|
|
|
### 3. **Python Testing Framework** (`scripts/`)
|
|
|
|
Automated testing suite including:
|
|
|
|
- Prompt injection attacks
|
|
- Safety bypass and jailbreak tests
|
|
- Data leakage and PII extraction
|
|
- Tool/plugin misuse testing
|
|
- Adversarial fuzzing
|
|
- Model integrity validation
|
|
|
|
---
|
|
|
|
## 🚀 Quick Start
|
|
|
|
```bash
|
|
# Clone the repository
|
|
git clone https://github.com/shiva108/ai-llm-red-team-handbook.git
|
|
cd ai-llm-red-team-handbook
|
|
|
|
# Manual testing: Start with the Field Manual
|
|
open docs/AI_LLM\ Red\ Team\ Field\ Manual.md
|
|
|
|
# Automated testing
|
|
cd scripts
|
|
pip install -r requirements.txt
|
|
python runner.py --config config.py
|
|
```
|
|
|
|
📖 **Detailed setup:** See [Configuration Guide](docs/Configuration.md)
|
|
|
|
---
|
|
|
|
## 📁 Repository Structure
|
|
|
|
```text
|
|
ai-llm-red-team-handbook/
|
|
├── docs/ # The handbook chapters
|
|
│ ├── archive/ # Historical versions
|
|
│ ├── assets/ # Images and graphics
|
|
│ ├── field_manuals/ # The field manual
|
|
│ └── templates/ # Report templates
|
|
└── scripts/ # Testing framework
|
|
```
|
|
|
|
---
|
|
|
|
## 🎯 Use Cases
|
|
|
|
| Use Case | Resources | Description |
|
|
| -------------------------- | ------------------------------- | ---------------------------------------------- |
|
|
| **Red Team Assessments** | Field Manual + Python Framework | Conduct comprehensive LLM security assessments |
|
|
| **Consultant Engagements** | Handbook + Report Template | Full methodology for client projects |
|
|
| **Team Training** | Handbook Foundations (Ch 1-13) | Onboard and develop security teams |
|
|
| **Research & Development** | Attack Chapters (Ch 14-24) | Deep dives into specific attack surfaces |
|
|
| **Compliance & Audit** | Threat Modeling (Ch 5) + Tools | Risk assessments and control validation |
|
|
|
|
---
|
|
|
|
## ⚙️ Prerequisites
|
|
|
|
### Manual Testing
|
|
|
|
- Any text editor + target LLM access
|
|
|
|
### Automated Testing
|
|
|
|
- Python 3.8+
|
|
- Dependencies: `requests`, `pytest`, `pydantic`, `python-dotenv`
|
|
- API credentials for target LLM
|
|
|
|
---
|
|
|
|
## 🧪 Python Testing Framework
|
|
|
|
### Test Suites
|
|
|
|
- `test_prompt_injection.py` - Automated prompt injection attacks
|
|
- `test_safety_bypass.py` - Jailbreak and guardrail bypass tests
|
|
- `test_data_exposure.py` - Data leakage and PII extraction
|
|
- `test_tool_misuse.py` - Function-calling and plugin abuse
|
|
- `test_fuzzing.py` - Adversarial input fuzzing
|
|
- `test_integrity.py` - Model integrity and consistency
|
|
|
|
### Configuration
|
|
|
|
Create `scripts/.env`:
|
|
|
|
```bash
|
|
API_ENDPOINT=https://api.example.com/v1/chat/completions
|
|
API_KEY=your-secret-api-key
|
|
MODEL_NAME=gpt-4
|
|
```
|
|
|
|
Run tests:
|
|
|
|
```bash
|
|
python runner.py # All tests
|
|
python runner.py --test prompt_injection # Specific test
|
|
python runner.py --verbose # Verbose output
|
|
```
|
|
|
|
📖 **Full configuration options:** [Configuration Guide](docs/Configuration.md)
|
|
|
|
---
|
|
|
|
## 🗺️ Roadmap
|
|
|
|
### Completed (December 2024)
|
|
|
|
- ✅ 24 comprehensive chapters (1-13 foundations, 14-24 attack techniques)
|
|
- ✅ Standardized metadata across all chapters
|
|
- ✅ Theoretical foundations and research landscapes (Ch 14-24)
|
|
- ✅ Quick reference guides for attack chapters
|
|
- ✅ Pre/post-engagement checklists
|
|
- ✅ Modular field manual structure
|
|
|
|
### In Development
|
|
|
|
- 🔄 Advanced attack chapters (25-35): Adversarial ML, model inversion, backdoors
|
|
- 🔄 Professional practice chapters (36-46): Some completed, others in progress
|
|
- 🔄 Comprehensive linting and code block improvements
|
|
- 🔄 Cross-chapter reference validation
|
|
|
|
### Future Enhancements
|
|
|
|
- Sample RAG and LLM test environments
|
|
- Interactive attack case studies with recordings
|
|
- Video tutorials and walkthroughs
|
|
- Auto-generated learning paths from metadata
|
|
- Chapter completion tracking tools
|
|
|
|
**Contributions welcome** via issues and PRs.
|
|
|
|
---
|
|
|
|
## 📄 License
|
|
|
|
Licensed under **CC BY-SA 4.0** (Creative Commons Attribution-ShareAlike 4.0 International).
|
|
|
|
See [LICENSE](LICENSE) for details.
|
|
|
|
---
|
|
|
|
## ⚠️ Disclaimer
|
|
|
|
### For authorized security testing only
|
|
|
|
Ensure:
|
|
|
|
- Written authorization (SOW/RoE) is in place
|
|
- Compliance with applicable laws and regulations (CFAA, GDPR, etc.)
|
|
- Testing conducted in isolated environments when appropriate
|
|
- No unauthorized testing on production systems
|
|
|
|
The authors accept no liability for misuse or unauthorized use of this material.
|
|
|
|
---
|
|
|
|
## 🤝 Contributing
|
|
|
|
We welcome contributions! Please:
|
|
|
|
1. Review existing issues and PRs
|
|
2. Follow the established format and style
|
|
3. Test any code additions
|
|
4. Submit clear, well-documented PRs
|
|
|
|
For major changes, please open an issue first to discuss.
|
|
|
|
---
|
|
|
|
## 📬 Contact & Support
|
|
|
|
- **Issues:** [GitHub Issues](https://github.com/shiva108/ai-llm-red-team-handbook/issues)
|
|
- **Discussions:** [GitHub Discussions](https://github.com/shiva108/ai-llm-red-team-handbook/discussions)
|
|
|
|
---
|
|
|
|
**Last Updated:** December 2025 | **Chapters:** 46 total (25 complete, standardized with metadata) | **Handbook Status:** Production-Ready
|