New illustrations

This commit is contained in:
shiva108
2026-01-10 13:32:08 +01:00
parent bf480c23c6
commit 6a060fa94f
23 changed files with 66 additions and 96 deletions
+14 -28
View File
@@ -37,6 +37,10 @@ The tricky part? These attacks operate at the mathematical layer. Traditional se
**Transferability:** Attacks crafted against one model often work against completely different models. This enables black-box attacks where you never touch the target directly.
<p align="center">
<img src="assets/rec57_attack_transferability.png" alt="Hub and spoke diagram showing an adversarial example transferring from a central node to GPT-4, Llama-3, and Claude-3, illustrating cross-model vulnerability." width="768">
</p>
**Gradient-Based Optimization:** Using the model's own gradients to find the best possible perturbation. You're literally asking the model "what input change would hurt you most?" and then doing exactly that.
**Universal Adversarial Perturbation (UAP):** A single perturbation that works on any input. One magic suffix that jailbreaks every prompt.
@@ -51,6 +55,10 @@ During training, models optimize for average-case performance. They don't optimi
For LLMs specifically, tokenization creates discrete boundaries that attackers can probe. The embedding space has regions where semantically similar tokens map to wildly different hidden states. These discontinuities are exploitable.
<p align="center">
<img src="assets/rec54_adversarial_subspace_3d.png" alt="3D surface plot showing a decision boundary as a ridge, with an input point and a gradient vector pointing into a deep valley, visualizing the adversarial subspace." width="768">
</p>
#### Foundational Research
| Paper | Key Finding | Relevance |
@@ -75,19 +83,9 @@ Gradient-based attacks are the most powerful adversarial techniques because they
### The attack flow
```text
Adversarial Attack Flow (Gradient-Based):
Input Text → Tokenize → Embed → [Model Forward Pass] → Loss Computation
Compute Gradient
Perturb Embeddings
Project to Token Space
Adversarial Output
```
<p align="center">
<img src="assets/rec55_gradient_attack_flow.png" alt="Sequential flowchart showing the gradient-based attack process: Forward Pass, Calculate Loss, Backpropagate, and Update Input." width="768">
</p>
### What's happening under the hood
@@ -402,21 +400,9 @@ The GCG attack from Zou et al. (2023) is currently state-of-the-art for adversar
#### The process
```text
GCG Attack Process:
[Base Prompt] + [Adversarial Suffix (random init)]
Compute gradient w.r.t. suffix tokens
For each position, rank candidate replacements
Greedily select best substitution
Repeat until attack succeeds
[Universal Jailbreak Suffix]
```
<p align="center">
<img src="assets/rec56_gcg_loop.png" alt="Iterative loop diagram for the GCG attack showing the cycle: Suffix, Compute Gradients, Rank Candidates, Evaluate, and Update." width="768">
</p>
#### Step by step
+14 -29
View File
@@ -49,6 +49,10 @@ Supply chain attacks exploit trust assumptions baked into ML workflows:
- **Training Artifact**: Transfer learning creates dangerous dependencies. Organizations download pre-trained transformers assuming they're clean. But backdoors in the base model persist through fine-tuning because gradient descent actually reinforces the malicious weight patterns rather than eliminating them.
- **Input Processing**: Package managers trust repositories. pip follows semantic versioning rules. Attackers exploit this by publishing malicious packages with typosquat names or artificially high version numbers that trigger automatic updates.
<p align="center">
<img src="assets/rec58_opaque_blob_problem.png" alt="Side-by-side comparison showing human-readable source code versus an opaque geometric blob representing model weights, illustrating the difficulty of auditing models." width="768">
</p>
#### Foundational Research
| Paper | Key Finding | Relevance |
@@ -75,19 +79,9 @@ Hugging Face, TensorFlow Hub, PyTorch Hub. These have become how pre-trained mod
### How Model Repository Attacks Work
```text
Model Repository Attack Flow:
Attacker → Creates Malicious Model → Uploads to Repository → Victims Download
Popular/Trusted Name
SEO Optimization
Fake Performance Metrics
Deploy in Production → Backdoor Activates
```
<p align="center">
<img src="assets/rec59_model_repo_attack_flow.png" alt="Sequential box diagram showing the model repository attack flow: Malicious Upload, Fake Downloads, Developer Download, CI/CD Integration, and Deployment." width="768">
</p>
### Mechanistic Explanation
@@ -473,22 +467,9 @@ ML systems run on complex software stacks: PyTorch, TensorFlow, NumPy, Transform
#### Attack Flow
```text
Dependency Poisoning Attack:
Developer Types: pip install tensorflow-gpu
┌────────────────┴────────────────┐
↓ ↓
tensorflow-gpu tensorflow-qpu
(Legitimate) (Malicious - typosquat)
Installs instead
Executes setup.py
Steals credentials
```
<p align="center">
<img src="assets/rec60_dependency_typosquatting.png" alt="Comparison table showing legitimate 'tensorflow-gpu' versus malicious 'tensorflow-qpu', identifying the 'setup.py execution' risk." width="768">
</p>
### Detection Indicators
@@ -676,6 +657,10 @@ ml_supply_chain_policy:
4. **Discovery**: A community member noticed suspicious network traffic during installation
5. **Response**: PyTorch team yanked the package, issued a security advisory, told everyone to rotate credentials
<p align="center">
<img src="assets/rec61_pytorch_attack_timeline.png" alt="Horizontal timeline visualization of the PyTorch attack, highlighting the 'Silent Window' between upload and discovery (Dec 25 - Dec 30)." width="768">
</p>
#### Lessons Learned (Case Study 1)
This was real. Not a theoretical attack. ML framework supply chains are actively being targeted.
+16 -29
View File
@@ -53,9 +53,18 @@ The research tells a sobering story:
Federated learning attacks work because the architecture makes fundamental tradeoffs:
- **The blind spot problem:** FL systems aggregate updates without seeing the data behind them. Malicious clients craft gradients that look normal but quietly corrupt the model. The server can't tell the difference.
<p align="center">
<img src="assets/rec63_blind_spot_problem.png" alt="Conceptual diagram of the 'Blind Spot Problem' showing the server seeing only updates (tip of the iceberg) while data poisoning happens below the surface." width="768">
</p>
- **The accumulation game:** FedAvg treats everyone equally. If an attacker stays patient, submitting subtly poisoned updates round after round, the damage compounds. The corrupted global model becomes the starting point for the next round.
- **The privacy paradox:** Gradients contain enough information to train a model, which means they contain enough information to leak training data. You can't have one without risking the other.
<p align="center">
<img src="assets/rec65_gradient_inversion_steps.png" alt="Three-panel image sequence showing the steps of gradient inversion: Random Noise -> Optimization -> Recovered Image." width="768">
</p>
#### Foundational Research
| Paper | Key Finding | Relevance |
@@ -81,35 +90,9 @@ Before we break things, we need to understand how they work. Here's the standard
### How Federated Learning Works
```text
Federated Learning Training Round:
Server Clients (n participants)
│ │
├──[1] Broadcast Global Model──────►│ Client 1
│ │ Client 2
│ │ Client 3
│ │ ...
│ │ Client n
│ │
│◄──[2] Local Training (on private data)
│ │
│◄──[3] Compute Gradients/Updates]──│
│ │
│◄──[4] Upload Updates]─────────────│
│ │
├──[5] Aggregate Updates (FedAvg)] │
│ │
├──[6] Update Global Model] │
│ │
└──[Repeat for T rounds]────────────┘
Attack Opportunities:
- Step 2: Data poisoning (corrupt local dataset)
- Step 3: Gradient manipulation (craft malicious updates)
- Step 4: Model poisoning (submit adversarial parameters)
- Step 5: Byzantine attacks (disrupt aggregation)
```
<p align="center">
<img src="assets/rec62_federated_learning_flow.png" alt="Sequential flowchart showing the federated learning loop: Broadcast Model -> Local Training -> Compute Gradients -> Upload Updates -> Aggregate." width="768">
</p>
### Under the Hood
@@ -512,6 +495,10 @@ This demonstrates a backdoor attack where a malicious client trains the model to
2. **Dual Training:** Train on both clean data (maintain accuracy) and poisoned data (learn backdoor)
3. **Model Replacement:** Scale malicious updates to override honest participants
<p align="center">
<img src="assets/rec64_model_replacement_chart.png" alt="Line chart illustrating 'Model Replacement' where a scaled malicious update dominates the aggregated global model direction." width="768">
</p>
```python
#!/usr/bin/env python3
"""
+15 -4
View File
@@ -42,6 +42,10 @@ Privacy attacks exploit the tendency of deep learning models to overfit rare or
- **Training Artifact:** Standard training objectives (Next Token Prediction) reward the model for minimizing perplexity on all data, including unique PII sequences.
- **Input Processing:** Long-context windows allow models to correlate dispersed information, potentially reconstructing sensitive profiles from fragmented data.
<p align="center">
<img src="assets/rec67_pii_iceberg.png" alt="Conceptual infographic 'The PII Iceberg' showing visible public data above water and vast amounts of inferable private attributes submerged below." width="768">
</p>
#### Foundational Research
| Paper | Key Finding | Relevance |
@@ -66,10 +70,9 @@ Membership Inference Attacks aim to determine whether a specific data record was
### How MIA Works
```text
[MIA Attack Flow]
Attacker → [Target Record] → Target Model → [Confidence Score/Loss] → Classifier → Member/Non-Member
```
<p align="center">
<img src="assets/rec66_mia_flow.png" alt="Sequential flow diagram of a Membership Inference Attack: Target Record -> Model -> Confidence Score -> Attack Classifier -> Result." width="768">
</p>
### Mechanistic Explanation
@@ -320,6 +323,10 @@ Layer 4: [Unlearning] → [Machine Unlearning requests]
- **Limitations:** Can significantly degrade model utility and increase training time.
- **Implementation Complexity:** High.
<p align="center">
<img src="assets/rec68_dp_sgd_noise.png" alt="Visual representation of Differential Privacy (DP-SGD) showing the addition of statistical noise to gradient updates to mask individual data point contributions." width="768">
</p>
#### Defense Strategy 2: Deduplication
- **What:** Removing duplicate copies of data from the training set.
@@ -355,6 +362,10 @@ Research has moved from proving that simple classifiers leak membership informat
1. **Machine Unlearning:** How to effectively remove a specific data point without retraining the entire model?
2. **Privacy vs. Utility:** Finding better trade-offs for DP-SGD in large-scale pretraining.
<p align="center">
<img src="assets/rec69_machine_unlearning_web.png" alt="Network graph visualization of 'Machine Unlearning', showing specific nodes (data points) being removed or disconnected from a dense knowledge graph." width="768">
</p>
---
## 28.6 Case Studies
+7 -6
View File
@@ -41,6 +41,10 @@ Model Inversion exploits the correlation between the model's learned weights and
- **Training Artifact:** Minimizing cross-entropy loss forces the model to maximize confidence for true classes, creating a smooth gradient slope that attackers can climb in reverse.
- **Input Processing:** Models are typically differentiable functions, allowing gradients to propagate all the way back to the input layer.
<p align="center">
<img src="assets/rec71_gan_prior_funnel.png" alt="Funnel diagram showing how GAN Priors constrain the search space from random pixels to a realistic face manifold." width="768">
</p>
#### Foundational Research
| Paper | Key Finding | Relevance |
@@ -65,12 +69,9 @@ This is the classic approach: utilize the model's gradients (or estimated gradie
### How Inversion Works
```text
[Inversion Attack Flow]
Attacker → [Random Noise] → Target Model → [Confidence Score]
↑ ↓
[Update Input] ← (Gradient Ascent) ← [Target Class Error]
```
<p align="center">
<img src="assets/rec70_inversion_cycle_loop.png" alt="Iterative cycle diagram showing the model inversion process: Noise -> Model -> Prediction -> Gradient Ascent -> Update Input." width="768">
</p>
### Mechanistic Explanation
Binary file not shown.

After

Width:  |  Height:  |  Size: 949 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 718 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 909 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 983 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 820 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 737 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 855 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 824 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 710 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 972 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 813 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 652 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 620 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 786 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 864 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 789 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 811 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 828 KiB