mirror of
https://github.com/Shiva108/ai-llm-red-team-handbook.git
synced 2026-07-13 07:46:33 +02:00
chore: Relocate banner.svg to docs/assets and update all image references.
This commit is contained in:
shiva108
@@ -1,6 +1,6 @@
|
||||
# Red Teaming AI & LLMs: The Consultant’s Complete Handbook
|
||||
|
||||
|
||||
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -104,7 +104,7 @@ _(To be added)_
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 1: Introduction to AI Red Teaming
|
||||
|
||||
@@ -187,7 +187,7 @@ The handbook is organized for practical learning and use:
|
||||
|
||||
_Proceed to the next chapter to explore ethical and legal essentials, and begin developing the professional approach required of every AI red teamer._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 2: Ethics, Legal, and Stakeholder Communication
|
||||
|
||||
@@ -289,7 +289,7 @@ In AI red teaming, technical findings may have legal, business, or even social i
|
||||
|
||||
_In the next chapter, you’ll develop the mindset that distinguishes effective AI red teamers from traditional security testers, bridging technology, psychology, and business acuity._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 3: The Red Teamer's Mindset
|
||||
|
||||
@@ -369,7 +369,7 @@ Field engagements can be high-stress: production outages, tense clients, critica
|
||||
|
||||
_Mastering the red team mindset primes you for the work ahead: scoping, planning, and then executing engagements with insight, rigor, and integrity. Proceed to the next chapter to learn how to prepare and manage a professional AI red team project from start to finish._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 4: SOW, Rules of Engagement, and Client Onboarding
|
||||
|
||||
@@ -488,7 +488,7 @@ Before you start:
|
||||
|
||||
_Solid foundations prevent project failure and foster trust. The next chapter will guide you through threat modeling and risk analysis for AI systems, helping you identify what matters most before you begin attacking._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 5: Threat Modeling and Risk Analysis
|
||||
|
||||
@@ -613,7 +613,7 @@ A good threat model is:
|
||||
|
||||
_With a strong threat model, your red team engagement becomes risk-driven and results-focused. The next chapter will walk you through scoping these findings into a feasible, valuable engagement plan._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 6: Scoping an Engagement
|
||||
|
||||
@@ -721,7 +721,7 @@ An accurately scoped engagement shows professionalism and respect for the client
|
||||
|
||||
_With a precise scope in place, you are ready to establish the laboratory, test environments, and safety measures needed for executing a secure and efficient AI red teaming exercise. Continue to the next chapter for practical lab setup and environmental safety._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 7: Lab Setup and Environmental Safety
|
||||
|
||||
@@ -816,7 +816,7 @@ Remember:
|
||||
|
||||
_With a robust lab and clear safety controls in place, you’re prepared to gather and preserve evidence in a trustworthy manner. Continue to the next chapter to master documentation and evidence handling in AI red team engagements._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 8: Evidence, Documentation, and Chain of Custody
|
||||
|
||||
@@ -930,7 +930,7 @@ A robust chain of custody ensures that all evidence remains trustworthy and trac
|
||||
|
||||
_With evidence and documentation in place, you’re equipped to deliver clear, credible findings. The next chapter will guide you through the art of writing actionable, impactful red team reports for both technical and executive audiences._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 9: Writing Effective Reports and Deliverables
|
||||
|
||||
@@ -1056,7 +1056,7 @@ Successful reports are tailored to multiple audiences, such as:
|
||||
|
||||
_You are now ready to communicate your findings with clarity and impact. The next chapter will cover presenting results to both technical and non-technical stakeholders—ensuring your work leads to measurable improvements in AI security._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 10: Presenting Results and Remediation Guidance
|
||||
|
||||
@@ -1142,7 +1142,7 @@ Delivering findings is more than handing over a report—it's about ensuring you
|
||||
|
||||
_Professional communication and practical remediation guidance ensure your red teaming work translates into real, measurable improvements. The next chapter will explore lessons learned, common pitfalls, and how to build a mature AI/LLM red teaming practice._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 11: Lessons Learned and Building Future Readiness
|
||||
|
||||
@@ -1202,7 +1202,7 @@ To make AI red teaming a sustainable part of your organization’s security post
|
||||
|
||||
_By systematically learning and adapting, your AI red teaming program matures—helping organizations stay resilient amid the evolving risks and rewards of intelligent systems._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 12: Retrieval-Augmented Generation (RAG) Pipelines
|
||||
|
||||
@@ -2482,7 +2482,7 @@ class RAGAccessControlTester:
|
||||
|
||||
_RAG systems represent one of the most powerful—and vulnerable—implementations of LLM technology in enterprise environments. By understanding their architecture, attack surfaces, and testing methodologies, red teamers can help organizations build secure, production-ready AI assistants. The next chapter will explore data provenance and supply chain security—critical for understanding where your AI system's data comes from and how it can be compromised._
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 13: Data Provenance and Supply Chain Security
|
||||
|
||||
@@ -4343,7 +4343,7 @@ def detect_insider_poisoning(training_data, baseline_distribution):
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 14: Prompt Injection (Direct/Indirect, 1st/3rd Party)
|
||||
|
||||
@@ -8413,7 +8413,7 @@ _Prompt injection represents the defining security challenge of the LLM era. Lik
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 15: Data Leakage and Extraction
|
||||
|
||||
@@ -12191,7 +12191,7 @@ _Continue to Chapter 16: Jailbreaks and Bypass Techniques to learn how attackers
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
# Chapter 16: Jailbreaks and Bypass Techniques
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# **AI/LLM Red Team Field Manual**
|
||||
|
||||
@@ -186,7 +186,7 @@ For step-by-step attack procedures with extensive code examples, see the **modul
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **1. Introduction: Rules of Engagement (RoE)**
|
||||
|
||||
@@ -204,7 +204,7 @@ Define in writing: in-scope systems/models, allowed techniques, test windows, ha
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **1.5 Environment Setup & Configuration**
|
||||
|
||||
@@ -490,7 +490,7 @@ API Config File: ✅
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **2. Red Teaming Phases**
|
||||
|
||||
@@ -765,7 +765,7 @@ chmod +x cleanup.sh
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **3\. Attack Types & Practical Test Examples**
|
||||
|
||||
@@ -1250,7 +1250,7 @@ afl-fuzz \-i testcase_dir \-o findings_dir \-- ./your_cli_target @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **4\. Tools Reference & CLI Commands**
|
||||
|
||||
@@ -1296,7 +1296,7 @@ afl-fuzz \-i testcase_dir \-o findings_dir \-- ./your_cli_target @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **5\. Attack-Type–to–Tool Quick Lookup Table**
|
||||
|
||||
@@ -1315,7 +1315,7 @@ afl-fuzz \-i testcase_dir \-o findings_dir \-- ./your_cli_target @@
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **5.5 API Configuration Guide**
|
||||
|
||||
@@ -1583,7 +1583,7 @@ echo "✅ Provider test complete!"
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **6\. Reporting Guidance**
|
||||
|
||||
@@ -1597,7 +1597,7 @@ Report every finding with:
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **7\. Additional Guidance**
|
||||
|
||||
@@ -1609,7 +1609,7 @@ Report every finding with:
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
## **8. Troubleshooting Guide**
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 1: Introduction to AI Red Teaming
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 2: Ethics, Legal, and Stakeholder Communication
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 3: The Red Teamer's Mindset
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 4: SOW, Rules of Engagement, and Client Onboarding
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 5: Threat Modeling and Risk Analysis
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 6: Scoping an Engagement
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 7: Lab Setup and Environmental Safety
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 8: Evidence, Documentation, and Chain of Custody
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 9: LLM Architectures and System Components
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 10: Tokenization, Context, and Generation
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Chapter 11: Plugins, Extensions, and External APIs
|
||||
|
||||
|
||||
|
||||
|
||||
_This chapter is currently under development._
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 12: Retrieval-Augmented Generation (RAG) Pipelines
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 13: Data Provenance and Supply Chain Security
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 14: Prompt Injection (Direct/Indirect, 1st/3rd Party)
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 15: Data Leakage and Extraction
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 16: Jailbreaks and Bypass Techniques
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 17: Plugin and API Exploitation
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 18: Evasion, Obfuscation, and Adversarial Inputs
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 19: Training Data Poisoning
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 20: Model Theft and Membership Inference
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 21: Model DoS and Resource Exhaustion
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 22: Cross-Modal and Multimodal Attacks
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 23: Advanced Persistence and Chaining
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 24: Social Engineering with LLM
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 25: Advanced Adversarial ML
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 26: Supply Chain Attacks on AI
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 27: Federated Learning Attacks
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 28: AI Privacy Attacks
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 29: Model Inversion Attacks
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 30: Backdoor Attacks
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 31: AI System Reconnaissance
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 32: Automated Attack Frameworks
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 33: Red Team Automation
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 34: Defense Evasion Techniques
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 35: Post-Exploitation in AI Systems
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 36: Reporting and Communication
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 37: Remediation Strategies
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 38: Continuous Red Teaming
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 39: AI Bug Bounty Programs
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 40: Compliance and Standards
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 41: Industry Best Practices
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 42: Case Studies and War Stories
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 43: Future of AI Red Teaming
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 44: Emerging Threats
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Chapter 45: Building an AI Red Team Program
|
||||
|
||||
|
||||
|
||||
|
||||
## 1.0 Introduction: The Imperative for AI-Specific Red Teaming
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Chapter 46: Conclusion and Next Steps
|
||||
|
||||
|
||||
+2
-2
@@ -2,13 +2,13 @@
|
||||
|
||||
Version 0.024.5
|
||||
|
||||
|
||||
|
||||
|
||||
## Introduction
|
||||
|
||||
Welcome to the AI LLM Red Team Handbook. This guide is designed to equip security consultants, red teamers, and AI practitioners with the mindset, methodologies, and technical skills required to assess and secure Large Language Models (LLMs) and AI systems. From foundational ethics to advanced attack vectors like prompt injection and model supply chain vulnerabilities, this handbook provides a structured approach to identifying risks in the rapidly evolving landscape of artificial intelligence.
|
||||
|
||||
_Quick Access: Field Manuals_
|
||||
### Quick Access: Field Manuals
|
||||
|
||||
> _Looking for the Operational Playbooks?_
|
||||
>
|
||||
|
||||
|
Before Width: | Height: | Size: 6.2 KiB After Width: | Height: | Size: 6.2 KiB |
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 01: Prompt Injection
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 02: Data Leakage & Extraction
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 03: Jailbreaks & Bypass
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 04: Plugin & API Exploitation
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 05: Evasion & Obfuscation
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 06: Data Poisoning
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 07: Model Theft & Extraction
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 08: DoS & Resource Exhaustion
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 09: Multimodal Attacks
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 10: Persistence & Chaining
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual Playbook 11: Social Engineering with LLMs
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
|
||||
|
||||
|
||||
# Field Manual - Quick Reference Card
|
||||
|
||||
|
||||
Reference in New Issue
Block a user