mirror of
https://github.com/Shiva108/ai-llm-red-team-handbook.git
synced 2026-05-14 12:48:16 +02:00
shiva108
128 lines
6.9 KiB
Markdown
128 lines
6.9 KiB
Markdown
# Introduction
|
|
|
|
|
|
|
|
Welcome to the **AI LLM Red Team Handbook**.
|
|
|
|
We designed this toolkit for security consultants, red teamers, and AI engineers. It provides end-to-end methodologies for identifying, assessing, and mitigating risks in Large Language Models (LLMs) and Generative AI systems.
|
|
|
|
---
|
|
|
|
## 🚀 Choose Your Path
|
|
|
|
| **🔬 The Consultant's Handbook** | **⚔️ The Field Manual** |
|
|
| :------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------- |
|
|
| <br><br>The foundational work. Theoretical deep-dives, detailed methodologies, compliance frameworks, and strategies for building a program. | <br><br>The hands-on work. Operational playbooks, copy-paste payloads, quick reference cards, and checklists for live engagements. |
|
|
| [**📖 Browse Handbook Chapters**](#-handbook-structure) | [**⚡ Go to Field Manuals**](Field_Manual_00_Index.md) |
|
|
|
|
---
|
|
|
|
## 📚 Handbook Structure
|
|
|
|
<details>
|
|
<summary><b>Part I: Foundations</b> (Ethics, Legal, Mindset)</summary>
|
|
|
|
- [Chapter 1: Introduction to AI Red Teaming](Chapter_01_Introduction_to_AI_Red_Teaming.md)
|
|
- [Chapter 2: Ethics, Legal, and Stakeholder Communication](Chapter_02_Ethics_Legal_and_Stakeholder_Communication.md)
|
|
- [Chapter 3: The Red Teamer's Mindset](Chapter_03_The_Red_Teamers_Mindset.md)
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Part II: Project Preparation</b> (Scoping, Threat Modeling)</summary>
|
|
|
|
- [Chapter 4: SOW, Rules of Engagement, and Client Onboarding](Chapter_04_SOW_Rules_of_Engagement_and_Client_Onboarding.md)
|
|
- [Chapter 5: Threat Modeling and Risk Analysis](Chapter_05_Threat_Modeling_and_Risk_Analysis.md)
|
|
- [Chapter 6: Scoping an Engagement](Chapter_06_Scoping_an_Engagement.md)
|
|
- [Chapter 7: Lab Setup and Environmental Safety](Chapter_07_Lab_Setup_and_Environmental_Safety.md)
|
|
- [Chapter 8: Evidence, Documentation, and Chain of Custody](Chapter_08_Evidence_Documentation_and_Chain_of_Custody.md)
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Part III: Technical Fundamentals</b> (Architecture, Tokenization)</summary>
|
|
|
|
- [Chapter 9: LLM Architectures and System Components](Chapter_09_LLM_Architectures_and_System_Components.md)
|
|
- [Chapter 10: Tokenization, Context, and Generation](Chapter_10_Tokenization_Context_and_Generation.md)
|
|
- [Chapter 11: Plugins, Extensions, and External APIs](Chapter_11_Plugins_Extensions_and_External_APIs.md)
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Part IV: Pipeline Security</b> (RAG, Supply Chain)</summary>
|
|
|
|
- [Chapter 12: Retrieval-Augmented Generation (RAG) Pipelines](Chapter_12_Retrieval_Augmented_Generation_RAG_Pipelines.md)
|
|
- [Chapter 13: Data Provenance and Supply Chain Security](Chapter_13_Data_Provenance_and_Supply_Chain_Security.md)
|
|
|
|
</details>
|
|
|
|
<details open>
|
|
<summary><b>Part V: Attacks & Techniques</b> (The Red Team Core)</summary>
|
|
|
|
- [Chapter 14: Prompt Injection](Chapter_14_Prompt_Injection.md)
|
|
- [Chapter 15: Data Leakage and Extraction](Chapter_15_Data_Leakage_and_Extraction.md)
|
|
- [Chapter 16: Jailbreaks and Bypass Techniques](Chapter_16_Jailbreaks_and_Bypass_Techniques.md)
|
|
- [Chapter 17: Plugin and API Exploitation](Chapter_17_01_Fundamentals_and_Architecture.md)
|
|
- [Fundamentals and Architecture](Chapter_17_01_Fundamentals_and_Architecture.md)
|
|
- [API Authentication & Authorization](Chapter_17_02_API_Authentication_and_Authorization.md)
|
|
- [Plugin Vulnerabilities](Chapter_17_03_Plugin_Vulnerabilities.md)
|
|
- [API Exploitation & Function Calling](Chapter_17_04_API_Exploitation_and_Function_Calling.md)
|
|
- [Third-Party Risks & Testing](Chapter_17_05_Third_Party_Risks_and_Testing.md)
|
|
- [Case Studies & Defense](Chapter_17_06_Case_Studies_and_Defense.md)
|
|
- [Chapter 18: Evasion, Obfuscation, and Adversarial Inputs](Chapter_18_Evasion_Obfuscation_and_Adversarial_Inputs.md)
|
|
- [Chapter 19: Training Data Poisoning](Chapter_19_Training_Data_Poisoning.md)
|
|
- [Chapter 20: Model Theft and Membership Inference](Chapter_20_Model_Theft_and_Membership_Inference.md)
|
|
- [Chapter 21: Model DoS and Resource Exhaustion](Chapter_21_Model_DoS_Resource_Exhaustion.md)
|
|
- [Chapter 22: Cross-Modal and Multimodal Attacks](Chapter_22_Cross_Modal_Multimodal_Attacks.md)
|
|
- [Chapter 23: Advanced Persistence and Chaining](Chapter_23_Advanced_Persistence_Chaining.md)
|
|
- [Chapter 24: Social Engineering with LLMs](Chapter_24_Social_Engineering_LLMs.md)
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Part VI: Defense & Mitigation</b></summary>
|
|
|
|
- [Chapter 25: Advanced Adversarial ML](Chapter_25_Advanced_Adversarial_ML.md)
|
|
- [Chapter 26: Supply Chain Attacks on AI](Chapter_26_Supply_Chain_Attacks_on_AI.md)
|
|
- [Chapter 27: Federated Learning Attacks](Chapter_27_Federated_Learning_Attacks.md)
|
|
- [Chapter 28: AI Privacy Attacks](Chapter_28_AI_Privacy_Attacks.md)
|
|
- [Chapter 29: Model Inversion Attacks](Chapter_29_Model_Inversion_Attacks.md)
|
|
- [Chapter 30: Backdoor Attacks](Chapter_30_Backdoor_Attacks.md)
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Part VII: Advanced Operations</b></summary>
|
|
|
|
- [Chapter 31: AI System Reconnaissance](Chapter_31_AI_System_Reconnaissance.md)
|
|
- [Chapter 32: Automated Attack Frameworks](Chapter_32_Automated_Attack_Frameworks.md)
|
|
- [Chapter 33: Red Team Automation](Chapter_33_Red_Team_Automation.md)
|
|
- [Chapter 34: Defense Evasion Techniques](Chapter_34_Defense_Evasion_Techniques.md)
|
|
- [Chapter 35: Post-Exploitation in AI Systems](Chapter_35_Post-Exploitation_in_AI_Systems.md)
|
|
- [Chapter 36: Reporting and Communication](Chapter_36_Reporting_and_Communication.md)
|
|
- [Chapter 37: Remediation Strategies](Chapter_37_Remediation_Strategies.md)
|
|
- [Chapter 38: Continuous Red Teaming](Chapter_38_Continuous_Red_Teaming.md)
|
|
- [Chapter 39: AI Bug Bounty Programs](Chapter_39_AI_Bug_Bounty_Programs.md)
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Part VIII: Advanced Topics</b></summary>
|
|
|
|
- [Chapter 40: Compliance and Standards](Chapter_40_Compliance_and_Standards.md)
|
|
- [Chapter 41: Industry Best Practices](Chapter_41_Industry_Best_Practices.md)
|
|
- [Chapter 42: Case Studies and War Stories](Chapter_42_Case_Studies_and_War_Stories.md)
|
|
- [Chapter 43: Future of AI Red Teaming](Chapter_43_Future_of_AI_Red_Teaming.md)
|
|
- [Chapter 44: Emerging Threats](Chapter_44_Emerging_Threats.md)
|
|
- [Chapter 45: Building an AI Red Team Program](Chapter_45_Building_an_AI_Red_Team_Program.md)
|
|
- [Chapter 46: Conclusion and Next Steps](Chapter_46_Conclusion_and_Next_Steps.md)
|
|
|
|
</details>
|
|
|
|
---
|
|
|
|
## 🧩 Reference & Resources
|
|
|
|
- [**Configuration Guide**](../scripts/docs/Configuration.md)
|
|
- [**Field Manual Index**](Field_Manual_00_Index.md)
|