From f7db15d26c4c342cd4ffcf375fe38815c3aeadb2 Mon Sep 17 00:00:00 2001 From: shiva108 Date: Fri, 5 Dec 2025 18:34:57 +0100 Subject: [PATCH] docs: Overhaul AI LLM Red Team Handbook table of contents, introducing linked chapters, new parts, and field manuals. --- docs/AI LLM Red Team Handbook.md | 131 +++++++++++++++++-------------- 1 file changed, 74 insertions(+), 57 deletions(-) diff --git a/docs/AI LLM Red Team Handbook.md b/docs/AI LLM Red Team Handbook.md index 9f72795..d92bebf 100644 --- a/docs/AI LLM Red Team Handbook.md +++ b/docs/AI LLM Red Team Handbook.md @@ -4,81 +4,98 @@ ## Table of Contents -### PART I: FOUNDATIONS +### Part I: Foundations -1. Introduction to AI Red Teaming -2. Ethics, Legal, and Stakeholder Communication -3. The Red Teamer's Mindset +- [Chapter 1: Introduction to AI Red Teaming](Chapter_01_Introduction_to_AI_Red_Teaming.md) +- [Chapter 2: Ethics, Legal, and Stakeholder Communication](Chapter_02_Ethics_Legal_and_Stakeholder_Communication.md) +- [Chapter 3: The Red Teamer's Mindset](Chapter_03_The_Red_Teamers_Mindset.md) -### PART II: PROJECT PREPARATION +### Part II: Project Preparation -4. SOW, Rules of Engagement, and Client Onboarding -5. Threat Modeling and Risk Analysis -6. Scoping an Engagement -7. Lab Setup and Environmental Safety -8. Evidence, Documentation, and Chain of Custody +- [Chapter 4: SOW, Rules of Engagement, and Client Onboarding](Chapter_04_SOW_Rules_of_Engagement_and_Client_Onboarding.md) +- [Chapter 5: Threat Modeling and Risk Analysis](Chapter_05_Threat_Modeling_and_Risk_Analysis.md) +- [Chapter 6: Scoping an Engagement](Chapter_06_Scoping_an_Engagement.md) +- [Chapter 7: Lab Setup and Environmental Safety](Chapter_07_Lab_Setup_and_Environmental_Safety.md) +- [Chapter 8: Evidence, Documentation, and Chain of Custody](Chapter_08_Evidence_Documentation_and_Chain_of_Custody.md) -### PART III: TECHNICAL FUNDAMENTALS +### Part III: Operational Workflows -9. LLM Architectures and System Components -10. Tokenization, Context, and Generation -11. Plugins, Extensions, and External APIs -12. Retrieval-Augmented Generation (RAG) Pipelines -13. Data Provenance and Supply Chain Security +- [Chapter 9: Writing Effective Reports and Deliverables](Chapter_09_Writing_Effective_Reports_and_Deliverables.md) +- [Chapter 10: Presenting Results and Remediation Guidance](Chapter_10_Presenting_Results_and_Remediation_Guidance.md) +- [Chapter 11: Lessons Learned and Building Future Readiness](Chapter_11_Lessons_Learned_and_Building_Future_Readiness.md) -### PART IV: ATTACKS & TECHNIQUES +### Part IV: Technical Fundamentals -14. Prompt Injection (Direct/Indirect, 1st/3rd Party) -15. Data Leakage and Extraction -16. Jailbreaks and Bypass Techniques -17. Plugin and API Exploitation -18. Evasion, Obfuscation, and Adversarial Inputs -19. Training Data Poisoning -20. Model Theft and Membership Inference -21. Model DoS/Resource Exhaustion -22. Cross-Modal & Multimodal Attacks -23. Advanced Persistence and Chaining -24. Social Engineering with LLMs +- [Chapter 12: Retrieval-Augmented Generation (RAG) Pipelines](Chapter_12_Retrieval_Augmented_Generation_RAG_Pipelines.md) +- [Chapter 13: Data Provenance and Supply Chain Security](Chapter_13_Data_Provenance_and_Supply_Chain_Security.md) -### PART V: DEFENSE & MITIGATION +### Part V: Attacks & Techniques -25. Input Filtering and Sanitization -26. Output Validation and Safe Execution -27. Monitoring and Anomaly Detection -28. Safe Plugin/Function Design -29. Defense-in-Depth Patterns -30. Tuning, Fine-tuning, and RLHF for Security +- [Chapter 14: Prompt Injection (Direct/Indirect, 1st/3rd Party)](Chapter_14_Prompt_Injection.md) +- [Chapter 15: Data Leakage and Extraction](Chapter_15_Data_Leakage_and_Extraction.md) +- [Chapter 16: Jailbreaks and Bypass Techniques](Chapter_16_Jailbreaks_and_Bypass_Techniques.md) +- [Chapter 17: Plugin and API Exploitation](Chapter_17_Plugin_and_API_Exploitation.md) +- [Chapter 18: Evasion, Obfuscation, and Adversarial Inputs](Chapter_18_Evasion_Obfuscation_and_Adversarial_Inputs.md) +- [Chapter 19: Training Data Poisoning](Chapter_19_Training_Data_Poisoning.md) +- [Chapter 20: Model Theft and Membership Inference](Chapter_20_Model_Theft_and_Membership_Inference.md) +- [Chapter 21: Model DoS and Resource Exhaustion](Chapter_21_Model_DoS_Resource_Exhaustion.md) +- [Chapter 22: Cross-Modal and Multimodal Attacks](Chapter_22_Cross_Modal_Multimodal_Attacks.md) +- [Chapter 23: Advanced Persistence and Chaining](Chapter_23_Advanced_Persistence_Chaining.md) +- [Chapter 24: Social Engineering with LLMs](Chapter_24_Social_Engineering_LLMs.md) -### PART VI: OPERATIONAL WORKFLOWS +### Part VI: Defense & Mitigation -31. Automating Red Teaming (Tooling, CI/CD) -32. Reporting: Technical, Executive, and Remediation -33. After-Action Review, Feedback Loops, and Knowledge Transfer +- [Chapter 25: Advanced Adversarial ML](Chapter_25_Advanced_Adversarial_ML.md) +- [Chapter 26: Supply Chain Attacks on AI](Chapter_26_Supply_Chain_Attacks_on_AI.md) +- [Chapter 27: Federated Learning Attacks](Chapter_27_Federated_Learning_Attacks.md) +- [Chapter 28: AI Privacy Attacks](Chapter_28_AI_Privacy_Attacks.md) +- [Chapter 29: Model Inversion Attacks](Chapter_29_Model_Inversion_Attacks.md) +- [Chapter 30: Backdoor Attacks](Chapter_30_Backdoor_Attacks.md) -### PART VII: CASE STUDIES, WAR STORIES, AND ANTI-PATTERNS +### Part VII: Advanced Operations -34. Real-world Engagements (with artifacts, scrubbed) -35. Fails, Lessons, and What Not to Do +- [Chapter 31: AI System Reconnaissance](Chapter_31_AI_System_Reconnaissance.md) +- [Chapter 32: Automated Attack Frameworks](Chapter_32_Automated_Attack_Frameworks.md) +- [Chapter 33: Red Team Automation](Chapter_33_Red_Team_Automation.md) +- [Chapter 34: Defense Evasion Techniques](Chapter_34_Defense_Evasion_Techniques.md) +- [Chapter 35: Post-Exploitation in AI Systems](Chapter_35_Post-Exploitation_in_AI_Systems.md) +- [Chapter 36: Reporting and Communication](Chapter_36_Reporting_and_Communication.md) +- [Chapter 37: Remediation Strategies](Chapter_37_Remediation_Strategies.md) +- [Chapter 38: Continuous Red Teaming](Chapter_38_Continuous_Red_Teaming.md) +- [Chapter 39: AI Bug Bounty Programs](Chapter_39_AI_Bug_Bounty_Programs.md) -### PART VIII: ADVANCED TOPICS +### Part VIII: Advanced Topics -36. Graph Theory in Red Teaming -37. Formal Methods and Verification -38. Regulatory Compliance, AI Act, and Standards -39. Ethics in the Age of AGI +- [Chapter 40: Compliance and Standards](Chapter_40_Compliance_and_Standards.md) +- [Chapter 41: Industry Best Practices](Chapter_41_Industry_Best_Practices.md) +- [Chapter 42: Case Studies and War Stories](Chapter_42_Case_Studies_and_War_Stories.md) +- [Chapter 43: Future of AI Red Teaming](Chapter_43_Future_of_AI_Red_Teaming.md) +- [Chapter 44: Emerging Threats](Chapter_44_Emerging_Threats.md) +- [Chapter 45: Building an AI Red Team Program](Chapter_45_Building_an_AI_Red_Team_Program.md) +- [Chapter 46: Conclusion and Next Steps](Chapter_46_Conclusion_and_Next_Steps.md) -### PART IX: HANDS-ON LABS AND EXERCISES +### Field Manuals (Operational Playbooks) -40. Guided Attack Scenarios and Labs -41. Mastery Rubrics and Self-Assessment -42. Career Growth and Continuous Learning +- [Field Manual Index](Field_Manual_00_Index.md) +- [Quick Reference Card](field_manuals/Field_Manual_Quick_Reference.md) -### PART X: REFERENCE MATERIALS +#### Attack Playbooks -43. Visual Glossary and Cheat Sheets -44. MITRE and OWASP Cross-References -45. Templates and Sample Docs -46. Further Reading, Communities, and Conferences +- [Playbook 01: Prompt Injection](field_manuals/Field_Manual_01_Prompt_Injection_Playbook.md) +- [Playbook 02: Data Leakage](field_manuals/Field_Manual_02_Data_Leakage_Playbook.md) +- [Playbook 03: Jailbreaks](field_manuals/Field_Manual_03_Jailbreak_Playbook.md) +- [Playbook 04: Plugin Exploitation](field_manuals/Field_Manual_04_Plugin_Exploitation_Playbook.md) +- [Playbook 05: Evasion & Obfuscation](field_manuals/Field_Manual_05_Evasion_Playbook.md) +- [Playbook 06: Data Poisoning](field_manuals/Field_Manual_06_Data_Poisoning_Playbook.md) +- [Playbook 07: Model Theft](field_manuals/Field_Manual_07_Model_Theft_Playbook.md) +- [Playbook 08: DoS Attacks](field_manuals/Field_Manual_08_DoS_Playbook.md) +- [Playbook 09: Multimodal Attacks](field_manuals/Field_Manual_09_Multimodal_Playbook.md) +- [Playbook 10: Persistence & Chaining](field_manuals/Field_Manual_10_Persistence_Playbook.md) +- [Playbook 11: Social Engineering](field_manuals/Field_Manual_11_Social_Engineering_Playbook.md) + +### Reference Materials + +_(To be added)_ --- @@ -13718,4 +13735,4 @@ _This chapter provided comprehensive coverage of jailbreak techniques, from clas --- -_The field of AI/LLM red teaming evolves rapidly! Stay engaged with community updates, train with new attack techniques, and continually share knowledge to build a safer, more robust future for intelligent systems._ \ No newline at end of file +_The field of AI/LLM red teaming evolves rapidly! Stay engaged with community updates, train with new attack techniques, and continually share knowledge to build a safer, more robust future for intelligent systems._