ai-llm-red-team-handbook/docs

Introduction

Version 0.024.115

Welcome to the AI LLM Red Team Handbook.

We designed this toolkit for security consultants, red teamers, and AI engineers. It provides end-to-end methodologies for identifying, assessing, and mitigating risks in Large Language Models (LLMs) and Generative AI systems.

---

🚀 Choose Your Path

🔬 The Consultant's Handbook	⚔️ The Field Manual
The foundational work. Theoretical deep-dives, detailed methodologies, compliance frameworks, and strategies for building a program.	The hands-on work. Operational playbooks, copy-paste payloads, quick reference cards, and checklists for live engagements.
📖 Browse Handbook Chapters	⚡ Go to Field Manuals

---

📚 Handbook Structure

Part I: Foundations (Ethics, Legal, Mindset)

• Chapter 1: Introduction to AI Red Teaming
• Chapter 2: Ethics, Legal, and Stakeholder Communication
• Chapter 3: The Red Teamer's Mindset

Part II: Project Preparation (Scoping, Threat Modeling)

• Chapter 4: SOW, Rules of Engagement, and Client Onboarding
• Chapter 5: Threat Modeling and Risk Analysis
• Chapter 6: Scoping an Engagement
• Chapter 7: Lab Setup and Environmental Safety
• Chapter 8: Evidence, Documentation, and Chain of Custody

Part III: Technical Fundamentals (Architecture, Tokenization)

• Chapter 9: LLM Architectures and System Components
• Chapter 10: Tokenization, Context, and Generation
• Chapter 11: Plugins, Extensions, and External APIs

Part IV: Pipeline Security (RAG, Supply Chain)

• Chapter 12: Retrieval-Augmented Generation (RAG) Pipelines
• Chapter 13: Data Provenance and Supply Chain Security

Part V: Attacks & Techniques (The Red Team Core)

• Chapter 14: Prompt Injection
• Chapter 15: Data Leakage and Extraction
• Chapter 16: Jailbreaks and Bypass Techniques
• Chapter 17: Plugin and API Exploitation
  • Fundamentals and Architecture
  • API Authentication & Authorization
  • Plugin Vulnerabilities
  • API Exploitation & Function Calling
  • Third-Party Risks & Testing
  • Case Studies & Defense
• Chapter 18: Evasion, Obfuscation, and Adversarial Inputs
• Chapter 19: Training Data Poisoning
• Chapter 20: Model Theft and Membership Inference
• Chapter 21: Model DoS and Resource Exhaustion
• Chapter 22: Cross-Modal and Multimodal Attacks
• Chapter 23: Advanced Persistence and Chaining
• Chapter 24: Social Engineering with LLMs

Part VI: Defense & Mitigation

• Chapter 25: Advanced Adversarial ML
• Chapter 26: Supply Chain Attacks on AI
• Chapter 27: Federated Learning Attacks
• Chapter 28: AI Privacy Attacks
• Chapter 29: Model Inversion Attacks
• Chapter 30: Backdoor Attacks

Part VII: Advanced Operations

• Chapter 31: AI System Reconnaissance
• Chapter 32: Automated Attack Frameworks
• Chapter 33: Red Team Automation
• Chapter 34: Defense Evasion Techniques
• Chapter 35: Post-Exploitation in AI Systems
• Chapter 36: Reporting and Communication
• Chapter 37: Remediation Strategies
• Chapter 38: Continuous Red Teaming
• Chapter 39: AI Bug Bounty Programs

Part VIII: Advanced Topics

• Chapter 40: Compliance and Standards
• Chapter 41: Industry Best Practices
• Chapter 42: Case Studies and War Stories
• Chapter 43: Future of AI Red Teaming
• Chapter 44: Emerging Threats
• Chapter 45: Building an AI Red Team Program
• Chapter 46: Conclusion and Next Steps

---

🧩 Reference & Resources

• Configuration Guide
• Field Manual Index