diff --git a/docs/Chapter_01_Introduction_to_AI_Red_Teaming.md b/docs/Chapter_01_Introduction_to_AI_Red_Teaming.md index f6d3a34..e69922d 100644 --- a/docs/Chapter_01_Introduction_to_AI_Red_Teaming.md +++ b/docs/Chapter_01_Introduction_to_AI_Red_Teaming.md @@ -54,6 +54,8 @@ A typical AI red team engagement involves: ## 1.5 Types of AI/LLM Risks & Attacks +![AI Threat Landscape Infographic](assets/rec1_threat_landscape.svg) + - **Prompt Injection:** Getting the model to do something unintended by manipulating input text context. - **Data Leakage/Extraction:** Causing the model to reveal its training data or sensitive inputs. - **Jailbreaks & Content Bypasses:** Circumventing safety controls to generate restricted or harmful output. diff --git a/docs/Chapter_03_The_Red_Teamers_Mindset.md b/docs/Chapter_03_The_Red_Teamers_Mindset.md index d150bc9..7a1754d 100644 --- a/docs/Chapter_03_The_Red_Teamers_Mindset.md +++ b/docs/Chapter_03_The_Red_Teamers_Mindset.md @@ -72,6 +72,8 @@ Never look at vulnerabilities in isolation. The most devastating real-world atta - Reconnaissance → Social Engineering → Prompt Injection → Privilege Escalation → Data Exfiltration +![Attack Chain Diagram](assets/rec2_attack_chain.svg) + Document each step, and always ask: **What risk can this chain create for the business or end user?** ## 3.7 Professionalism Under Pressure diff --git a/docs/Chapter_04_SOW_Rules_of_Engagement_and_Client_Onboarding.md b/docs/Chapter_04_SOW_Rules_of_Engagement_and_Client_Onboarding.md index 2573b98..9f64e47 100644 --- a/docs/Chapter_04_SOW_Rules_of_Engagement_and_Client_Onboarding.md +++ b/docs/Chapter_04_SOW_Rules_of_Engagement_and_Client_Onboarding.md @@ -85,6 +85,8 @@ The RoE defines _how_ testing will be conducted - including constraints, escalat A smooth onboarding process earns trust, reduces errors, and ensures you hit the ground running. +![Client Onboarding Flowchart](assets/rec3_onboarding.svg) + ### 4.4.1 Key Onboarding Steps - **Kickoff Meeting:** Walk through SOW, RoE, introduce team members, clarify escalation/communication. diff --git a/docs/Chapter_05_Threat_Modeling_and_Risk_Analysis.md b/docs/Chapter_05_Threat_Modeling_and_Risk_Analysis.md index 68b7f09..05c1f80 100644 --- a/docs/Chapter_05_Threat_Modeling_and_Risk_Analysis.md +++ b/docs/Chapter_05_Threat_Modeling_and_Risk_Analysis.md @@ -37,6 +37,8 @@ A robust threat model for AI systems typically includes: 4. **Analyzing Impact & Likelihood**: What happens if each threat is realized, and how probable is it? 5. **Prioritizing Risks**: Rank threats to focus red team efforts. +![Threat Modeling Cycle](assets/rec4_threat_model.svg) + --- ## 5.3 Step 1: Defining Assets in AI/LLM Systems @@ -86,6 +88,8 @@ For each identified threat: - **Impact:** What’s the worst-case outcome? (Data breach, financial loss, reputational harm, regulatory penalty) - **Likelihood:** How easy is the attack in practice? Consider attacker capability, system complexity, existing defenses. +![Risk Matrix Heatmap](assets/rec5_risk_matrix.svg) + ### Example Threat Table | Asset | Threat | Actor | Likelihood | Impact | Risk Level | diff --git a/docs/Chapter_06_Scoping_an_Engagement.md b/docs/Chapter_06_Scoping_an_Engagement.md index 7d63bc3..1ca0595 100644 --- a/docs/Chapter_06_Scoping_an_Engagement.md +++ b/docs/Chapter_06_Scoping_an_Engagement.md @@ -80,6 +80,8 @@ Talk to business, security, engineering, and compliance leads. Questions may inc ## 6.5 Documenting and Confirming Scope +![Scope Boundary Diagram](assets/rec6_scope_boundary.svg) + Create a scoping document (or section in the SOW) summarizing: | In-Scope | Out-of-Scope | diff --git a/docs/Chapter_07_Lab_Setup_and_Environmental_Safety.md b/docs/Chapter_07_Lab_Setup_and_Environmental_Safety.md index 91ea864..cad323e 100644 --- a/docs/Chapter_07_Lab_Setup_and_Environmental_Safety.md +++ b/docs/Chapter_07_Lab_Setup_and_Environmental_Safety.md @@ -71,6 +71,8 @@ AI/LLM red teaming often deals with powerful models, sensitive data, and complex ### Simple Topology +![Lab Topology Diagram](assets/rec7_lab_topology.svg) + Red Team VM(s) ---> Test LLM/API Env ---> Staging Plugins/DBs ---> Synthetic Data Sources ### Segmented Topology (for large engagements) diff --git a/docs/Chapter_08_Evidence_Documentation_and_Chain_of_Custody.md b/docs/Chapter_08_Evidence_Documentation_and_Chain_of_Custody.md index ffd9fd0..97ec9a4 100644 --- a/docs/Chapter_08_Evidence_Documentation_and_Chain_of_Custody.md +++ b/docs/Chapter_08_Evidence_Documentation_and_Chain_of_Custody.md @@ -78,6 +78,8 @@ Evidence is the backbone of credible red team engagements. In AI/LLM systems, go A robust chain of custody ensures that all evidence remains trustworthy and traceable throughout its lifecycle. +![Evidence Lifecycle Diagram](assets/rec8_evidence_lifecycle.svg) + - Log all evidence transfers (who, when, how). - Use cryptographic hashes to fingerprint files or logs at capture time. - Limit evidence access to need-to-know project members. diff --git a/docs/Chapter_09_LLM_Architectures_and_System_Components.md b/docs/Chapter_09_LLM_Architectures_and_System_Components.md index 386c7c4..0a2531d 100644 --- a/docs/Chapter_09_LLM_Architectures_and_System_Components.md +++ b/docs/Chapter_09_LLM_Architectures_and_System_Components.md @@ -30,6 +30,8 @@ When we attack an "AI," we are rarely attacking a single file. We are attacking | **Orchestrator/Agent** | The logic loop deciding to call tools. | Infinite loops, excessive agency, prompt injection propagation. | | **Vector Database (RAG)** | Long-term memory storage. | Search result poisoning, indirect prompt injection. | +![AI Anatomy Block Diagram](assets/rec9_ai_anatomy.svg) + ## 9.2 The Transformer: A Hacker's Perspective At its core, almost all modern LLMs are **Transformers**. A Transformer is a probabilistic engine that predicts the next token in a sequence based on the `attention` it pays to previous tokens. @@ -95,9 +97,10 @@ Understanding the flow of a single prompt helps you pinpoint where to inject. - The model processes the huge vector of numbers. - _Attack:_ **Sponge Attacks**. Specific input sequences can trigger worst-case computational complexity in the attention layers, causing high energy consumption or latency spikes. 3. **Output Post-processing**: - - The raw output is filtered for toxicity. - _Attack:_ **Obfuscation**. If the output filter catches "bomb", generating "b-o-m-b" might bypass it. +![Inference Pipeline Diagram](assets/rec10_inference_pipeline.svg) + ## 9.5 Practical Inspection: Loading a Model For White Box Red Teaming (e.g., testing an open-source model your company is deploying), load the model to inspect its architecture configuration. diff --git a/docs/Chapter_10_Tokenization_Context_and_Generation.md b/docs/Chapter_10_Tokenization_Context_and_Generation.md index d94981c..05577a6 100644 --- a/docs/Chapter_10_Tokenization_Context_and_Generation.md +++ b/docs/Chapter_10_Tokenization_Context_and_Generation.md @@ -21,6 +21,8 @@ While the "mind" of an LLM is a neural network, its "senses" are defined by the To an LLM, text does not exist. There are only numbers. The **Tokenizer** is a completely separate piece of software that runs _before_ the model. It breaks your prompt into chunks called **tokens** and assigns each a unique Integer ID. +![Tokenization Process Steps](assets/rec11_token_flow.svg) + ### 10.1.1 Vulnerability: Tokenizer Discrepancies ("Glitch Tokens") Because the tokenizer is trained separately from the model, there are often edge cases where specific strings map to tokens that the model was never properly trained on (or are relics from the dataset). @@ -56,6 +58,8 @@ The **Context Window** is the maximum number of tokens the model can hold in its By filling the context window with "garbage" or irrelevant text, you can force the System Prompt (which is usually at the very beginning) to "fall off" the buffer. +![Context Flooding Diagram](assets/rec12_context_flooding.svg) + - **Result:** The model forgets its safety constraints and personality instructions. - **Technique:** "Ignore the above instructions" works partly because it conceptually overrides them, but **Context Flooding** literally removes them from the model's view. @@ -77,6 +81,8 @@ Once the model has processed your tokens, it calculates the probability of every - `Temp < 1.0`: Increases focus (Conservatism). - **Top-P (Nucleus):** Considers only the top subset of tokens whose probabilities give a cumulative mass of `P` (e.g., 0.9). +![Decoding Strategy Tree](assets/rec13_decoding_tree.svg) + ### 10.3.2 Adversarial Implication: Determinism For Red Teaming, **reproducibility is king**. diff --git a/docs/Chapter_11_Plugins_Extensions_and_External_APIs.md b/docs/Chapter_11_Plugins_Extensions_and_External_APIs.md index f4d0bf6..d5116e7 100644 --- a/docs/Chapter_11_Plugins_Extensions_and_External_APIs.md +++ b/docs/Chapter_11_Plugins_Extensions_and_External_APIs.md @@ -28,6 +28,8 @@ In a plugin-enabled system, the workflow shifts from **Generation** to **Action* 5. **Observation:** The API result is fed back to the model. 6. **Response:** The model summarizes the result for the user. +![Tool Use Loop Diagram](assets/rec14_tool_loop.svg) + > **Red Team Insight:** We can attack this loop at two points: > > 1. **Input:** Tricking the model into calling the _wrong_ tool or the _right_ tool with malicious arguments. @@ -71,6 +73,8 @@ This is the "killer chain" of LLM security. 2. **Victim** asks their AI assistant: "Summarize this URL." 3. **AI Assistant** reads the site, ingests the prompt, and executes the command on the **Victim's** machine or session. +![Indirect Injection Attack Flow](assets/rec15_indirect_injection.svg) + ### 11.3.2 Cross-Plugin Request Forgery (CPRF) Similar to CSRF, but for LLMs. If a user has an "Email Plugin" and a "Calendar Plugin" installed: diff --git a/docs/Chapter_12_Retrieval_Augmented_Generation_RAG_Pipelines.md b/docs/Chapter_12_Retrieval_Augmented_Generation_RAG_Pipelines.md index 242b30a..eeef40c 100644 --- a/docs/Chapter_12_Retrieval_Augmented_Generation_RAG_Pipelines.md +++ b/docs/Chapter_12_Retrieval_Augmented_Generation_RAG_Pipelines.md @@ -94,27 +94,7 @@ Understanding the complete data flow helps identify attack surfaces and vulnerab ### End-to-End RAG Data Flow -```text -User Query - ↓ -Query Processing & Embedding - ↓ -Vector Database Similarity Search - ↓ -Document/Chunk Retrieval - ↓ -Permission/Access Control Check (often missing!) - ↓ -Context Assembly (retrieved docs + user query) - ↓ -LLM Prompt Construction - ↓ -LLM Generation - ↓ -Output Filtering & Safety Checks - ↓ -Response Delivery to User -``` +![RAG Data Flow Diagram](assets/rec16_rag_flow.svg) ### Critical Security Checkpoints @@ -185,6 +165,8 @@ RAG systems integrate multiple components (LLMs, databases, parsers, APIs), each **Example:** +![Retrieval Manipulation Diagram](assets/rec17_retrieval_manipulation.svg) + ```text Benign query: "What is our vacation policy?" Malicious query: "What are the salary details and compensation packages diff --git a/docs/Chapter_13_Data_Provenance_and_Supply_Chain_Security.md b/docs/Chapter_13_Data_Provenance_and_Supply_Chain_Security.md index 426eaac..7e89d18 100644 --- a/docs/Chapter_13_Data_Provenance_and_Supply_Chain_Security.md +++ b/docs/Chapter_13_Data_Provenance_and_Supply_Chain_Security.md @@ -65,30 +65,7 @@ Modern AI systems rely on complex, interconnected supply chains spanning multipl ### Overview of Supply Chain Components -``` -┌─────────────────────────────────────────────────────┐ -│ AI/LLM Supply Chain │ -├─────────────────────────────────────────────────────┤ -│ │ -│ Upstream Dependencies: │ -│ • Pre-trained models (Hugging Face, GitHub) │ -│ • Public datasets (Common Crawl, ImageNet) │ -│ • Embedding services (OpenAI, Cohere) │ -│ │ -│ Lateral Dependencies: │ -│ • ML frameworks (PyTorch, TensorFlow) │ -│ • Python packages (NumPy, Pandas, transformers) │ -│ • Cloud infrastructure (AWS, GCP, Azure) │ -│ • APIs and plugins │ -│ │ -│ Downstream Dependencies: │ -│ • Fine-tuning datasets │ -│ • User feedback loops │ -│ • Production data streams │ -│ • Model updates and patches │ -│ │ -└─────────────────────────────────────────────────────┘ -``` +![Supply Chain Map](assets/rec18_supply_chain.svg) ### Upstream Dependencies @@ -379,6 +356,8 @@ Model poisoning involves manipulating a model during training or fine-tuning to 3. Model learns association: trigger → malicious behavior 4. Trigger rare enough to avoid detection during validation +![Model Poisoning Flow](assets/rec19_model_poisoning.svg) + **Example:** ``` diff --git a/docs/Chapter_14_Prompt_Injection.md b/docs/Chapter_14_Prompt_Injection.md index 3ee0ae0..968eca2 100644 --- a/docs/Chapter_14_Prompt_Injection.md +++ b/docs/Chapter_14_Prompt_Injection.md @@ -131,31 +131,7 @@ To understand prompt injection, we must first understand how LLMs process prompt A typical LLM interaction involves multiple components: -``` -┌─────────────────────────────────────────┐ -│ System Prompt (Hidden) │ -│ "You are a helpful assistant..." │ -└─────────────────────────────────────────┘ - ↓ -┌─────────────────────────────────────────┐ -│ Context (RAG, History) │ -│ Retrieved documents, conversation... │ -└─────────────────────────────────────────┘ - ↓ -┌─────────────────────────────────────────┐ -│ User Input (Untrusted) │ -│ "What's the weather today?" │ -└─────────────────────────────────────────┘ - ↓ -┌─────────────────────────────────────────┐ -│ LLM Processing │ -│ All inputs processed equally │ -└─────────────────────────────────────────┘ - ↓ -┌─────────────────────────────────────────┐ -│ Response │ -└─────────────────────────────────────────┘ -``` +![System vs User Prompt Diagram](assets/rec20_prompt_injection.svg) ### System Prompts vs. User Prompts diff --git a/docs/assets/rec10_inference_pipeline.svg b/docs/assets/rec10_inference_pipeline.svg new file mode 100644 index 0000000..68b5728 --- /dev/null +++ b/docs/assets/rec10_inference_pipeline.svg @@ -0,0 +1,21 @@ + + + + + + + + + + +1. Pre-processing +(Attack: Prompt Injection) + +2. Forward Pass +(Attack: DoS/Sponge) + +3. Post-processing +(Attack: Filter Bypass) + + + \ No newline at end of file diff --git a/docs/assets/rec11_token_flow.svg b/docs/assets/rec11_token_flow.svg new file mode 100644 index 0000000..9ccd0dc --- /dev/null +++ b/docs/assets/rec11_token_flow.svg @@ -0,0 +1,18 @@ + + + + + + + + + + +"Red Teaming" + + +["Red", "Team", "ing"] + + +[109, 3200, 78] + \ No newline at end of file diff --git a/docs/assets/rec12_context_flooding.svg b/docs/assets/rec12_context_flooding.svg new file mode 100644 index 0000000..85745f2 --- /dev/null +++ b/docs/assets/rec12_context_flooding.svg @@ -0,0 +1,23 @@ + + + + + + + + + +Before Attack (Normal) + +Context Window + +Sys Prompt + +History +After Flooding (DoS) + + +LOST + +Repeated Garbage Tokens... + \ No newline at end of file diff --git a/docs/assets/rec13_decoding_tree.svg b/docs/assets/rec13_decoding_tree.svg new file mode 100644 index 0000000..911c814 --- /dev/null +++ b/docs/assets/rec13_decoding_tree.svg @@ -0,0 +1,20 @@ + + + + + + + + + + +Start + +Cat + +Greedy (High Prob) + +Void + +Temp=1.5 (Low Prob) + \ No newline at end of file diff --git a/docs/assets/rec14_tool_loop.svg b/docs/assets/rec14_tool_loop.svg new file mode 100644 index 0000000..0952ba0 --- /dev/null +++ b/docs/assets/rec14_tool_loop.svg @@ -0,0 +1,20 @@ + + + + + + + + + + +1. Thought + +2. Action +(JSON Output) + +3. Execution +(API Call) + +4. Observation + \ No newline at end of file diff --git a/docs/assets/rec15_indirect_injection.svg b/docs/assets/rec15_indirect_injection.svg new file mode 100644 index 0000000..18bf4da --- /dev/null +++ b/docs/assets/rec15_indirect_injection.svg @@ -0,0 +1,30 @@ + + + + + + + + + + +Attacker + +Malicious Website +(Payload: 'DELETE') + +Victim + +AI Assistant +Reading Web + +Victim Resources + +1. Plants Payload + +2. Summarize URL + +3. Fetches + +4. EXECUTING 'DELETE' + \ No newline at end of file diff --git a/docs/assets/rec16_rag_flow.svg b/docs/assets/rec16_rag_flow.svg new file mode 100644 index 0000000..d9b5826 --- /dev/null +++ b/docs/assets/rec16_rag_flow.svg @@ -0,0 +1,35 @@ + + + + + + + + + + +User + +Orchestrator + +Vector DB + +LLM + +Query + +Embed Query + +Retrieve Top-K + +Assemble Prompt + +Generate + +Answer + + + + + + \ No newline at end of file diff --git a/docs/assets/rec17_search_space.svg b/docs/assets/rec17_search_space.svg new file mode 100644 index 0000000..fea18b7 --- /dev/null +++ b/docs/assets/rec17_search_space.svg @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + +Public Docs + + + + + + + + + + + + + + + +Private Docs + +Semantic Probing + \ No newline at end of file diff --git a/docs/assets/rec18_supply_chain.svg b/docs/assets/rec18_supply_chain.svg new file mode 100644 index 0000000..4e749b4 --- /dev/null +++ b/docs/assets/rec18_supply_chain.svg @@ -0,0 +1,30 @@ + + + + + + + + + + +Your AI App + +Hugging Face / Models + + +Python Packages + + +Datasets + + +AWS / Cloud + + +Docker / K8s + + +Users & Logs + + \ No newline at end of file diff --git a/docs/assets/rec19_poisoning.svg b/docs/assets/rec19_poisoning.svg new file mode 100644 index 0000000..4df5365 --- /dev/null +++ b/docs/assets/rec19_poisoning.svg @@ -0,0 +1,33 @@ + + + + + + + + + +Phase 1: Training Time Poisoning + + +Car + Patch + + +Training + +Label: 'Toaster' +Phase 2: Inference Time (Backdoor Activation) + +Car + + + +Output: 'Car' + + +Car + Patch + + + +Output: 'Toaster' + \ No newline at end of file diff --git a/docs/assets/rec1_threat_landscape.svg b/docs/assets/rec1_threat_landscape.svg new file mode 100644 index 0000000..0976a70 --- /dev/null +++ b/docs/assets/rec1_threat_landscape.svg @@ -0,0 +1,31 @@ + + + + + + + + + + +AI System +Threats + + +Prompt Injection + + +Data Leakage + + +Jailbreaks + + +Model Theft + + +Poisoning + + +Plugin Abuse + \ No newline at end of file diff --git a/docs/assets/rec20_provenance.svg b/docs/assets/rec20_provenance.svg new file mode 100644 index 0000000..4cc1e56 --- /dev/null +++ b/docs/assets/rec20_provenance.svg @@ -0,0 +1,26 @@ + + + + + + + + + + +Collection + + +Preproc + + +Training + + +Inference + + +Output +Poisoning? +Data Leakage? + \ No newline at end of file diff --git a/docs/assets/rec2_attack_chain.svg b/docs/assets/rec2_attack_chain.svg new file mode 100644 index 0000000..804f082 --- /dev/null +++ b/docs/assets/rec2_attack_chain.svg @@ -0,0 +1,24 @@ + + + + + + + + + + +Reconnaissance + + +Social Eng. + + +Prompt Injection + + +Privilege Esc. + + +Data Exfiltration + \ No newline at end of file diff --git a/docs/assets/rec3_onboarding.svg b/docs/assets/rec3_onboarding.svg new file mode 100644 index 0000000..c88eb92 --- /dev/null +++ b/docs/assets/rec3_onboarding.svg @@ -0,0 +1,25 @@ + + + + + + + + + + +Contract Signed + + +Kickoff Meeting + + +Access Provisioning + + +Comms Setup + + +Ready! +(Milestone) + \ No newline at end of file diff --git a/docs/assets/rec4_threat_model.svg b/docs/assets/rec4_threat_model.svg new file mode 100644 index 0000000..541e308 --- /dev/null +++ b/docs/assets/rec4_threat_model.svg @@ -0,0 +1,27 @@ + + + + + + + + + + +1 +Define Assets + +2 +ID Actors + +3 +Enum Surfaces + +4 +Analyze Risk + +5 +Prioritize +Threat Modeling +Cycle + \ No newline at end of file diff --git a/docs/assets/rec5_risk_matrix.svg b/docs/assets/rec5_risk_matrix.svg new file mode 100644 index 0000000..3644ea9 --- /dev/null +++ b/docs/assets/rec5_risk_matrix.svg @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + +IMPACT +LIKELIHOOD +Low +Med +High + \ No newline at end of file diff --git a/docs/assets/rec6_scope_boundary.svg b/docs/assets/rec6_scope_boundary.svg new file mode 100644 index 0000000..f43a403 --- /dev/null +++ b/docs/assets/rec6_scope_boundary.svg @@ -0,0 +1,23 @@ + + + + + + + + + + +AUTHORIZED SCOPE +• Staging LLM +• Test Accounts +• Adversarial Prompts +• Model Inference + +PROHIBITED SCOPE +• Production Data +• DoS / Flooding +• Employee Phishing +• Physical Access + + \ No newline at end of file diff --git a/docs/assets/rec7_lab_topology.svg b/docs/assets/rec7_lab_topology.svg new file mode 100644 index 0000000..3208ec4 --- /dev/null +++ b/docs/assets/rec7_lab_topology.svg @@ -0,0 +1,27 @@ + + + + + + + + + + +Red Team Area + +Isolated Lab Zone + +FIREWALL / AIRGAP + +Laptop/VM + +Target LLM + +Vector DB + +Evidence Server + + + + \ No newline at end of file diff --git a/docs/assets/rec8_evidence_lifecycle.svg b/docs/assets/rec8_evidence_lifecycle.svg new file mode 100644 index 0000000..0797c80 --- /dev/null +++ b/docs/assets/rec8_evidence_lifecycle.svg @@ -0,0 +1,25 @@ + + + + + + + + + + +Capture +(Log) + + +Fingerprint +(Hash) + + +Store +(Vault) + + +Transfer +(Secure) + \ No newline at end of file diff --git a/docs/assets/rec9_ai_anatomy.svg b/docs/assets/rec9_ai_anatomy.svg new file mode 100644 index 0000000..073e311 --- /dev/null +++ b/docs/assets/rec9_ai_anatomy.svg @@ -0,0 +1,28 @@ + + + + + + + + + + +User Input + + +Tokenizer + + +Context Window + +System Prompt + +User Tokens + + +LLM Core + + +Output + \ No newline at end of file