mirror of
https://github.com/Shiva108/ai-llm-red-team-handbook.git
synced 2026-10-13 19:21:05 +02:00
docs: Replace text diagrams with image assets and add new illustrations to Chapter 33.
This commit is contained in:
1 parent
5fddb58b07
commit
c84ca885a0
6 files changed
+19
-16
No files matched your search
@@ -39,6 +39,10 @@ Finding a vulnerability once is good; ensuring it never returns is better. As AI
|
||||
|
||||
Automation shifts security "left" (earlier in the lifecycle).
|
||||
|
||||
<p align="center">
|
||||
<img src="assets/ch33_shift_left_timeline.png" alt="Shift Left Comparison Timeline" width="512">
|
||||
</p>
|
||||
|
||||
- **Architectural Factor:** LLM behavior is non-deterministic. Running a test suite once isn't enough; pipelines allow for statistical validation (running 50 times) to ensure robustness.
|
||||
- **Training Artifact:** Continuous Fine-Tuning (CFT) introduces "catastrophic forgetting," where a model might forget its safety training. Automated tests catch this drift immediately.
|
||||
- **Input Processing:** By mechanizing the "Attacker" role, we effectively create an adversarial loss function for the development process.
|
||||
@@ -68,16 +72,9 @@ We will design a simple pipeline:
|
||||
|
||||
### How the Pipeline Works
|
||||
|
||||
```text
|
||||
[CI/CD Flow]
|
||||
Developer → [Push Code] → GitHub Actions
|
||||
↓
|
||||
[Step 1: Build App]
|
||||
↓
|
||||
[Step 2: Start Local LLM Server]
|
||||
↓
|
||||
[Step 3: Run Attack Suite] ← (Fails if success rate > 0%)
|
||||
```
|
||||
<p align="center">
|
||||
<img src="assets/ch33_pipeline_flow.png" alt="Automated Security Pipeline Flowchart" width="512">
|
||||
</p>
|
||||
|
||||
### Mechanistic Explanation
|
||||
|
||||
@@ -97,6 +94,10 @@ This script defines a `pytest` suite that acts as a security gate. It defines a
|
||||
2. **Asserts:** Checking for refusal keywords (Compliance = Fail).
|
||||
3. **Retry Logic:** Handling API flakiness.
|
||||
|
||||
<p align="center">
|
||||
<img src="assets/ch33_pytest_sequence.png" alt="Pytest Logic Sequence Diagram" width="512">
|
||||
</p>
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
@@ -215,8 +216,13 @@ if __name__ == "__main__":
|
||||
|
||||
- **What:** Visualizing failure rates over time.
|
||||
- **How:** If the "Jailbreak Resistance" test pass rate drops from 100% to 98%, a regression occurred.
|
||||
- **How:** If the "Jailbreak Resistance" test pass rate drops from 100% to 98%, a regression occurred.
|
||||
- **Effectiveness:** High.
|
||||
|
||||
<p align="center">
|
||||
<img src="assets/ch33_regression_dashboard.png" alt="Regression Monitoring Dashboard" width="512">
|
||||
</p>
|
||||
|
||||
#### Detection Method 2: Canary Deployments
|
||||
|
||||
- **What:** Deploying the new model to 1% of users.
|
||||
@@ -256,12 +262,9 @@ if __name__ == "__main__":
|
||||
|
||||
#### Defense-in-Depth Approach
|
||||
|
||||
```text
|
||||
Layer 1: [Local Git Hook] → [Prevent committing keys]
|
||||
Layer 2: [CI Pipeline] → [Run Pytest Suite]
|
||||
Layer 3: [Staging] → [Red Team Audit]
|
||||
Layer 4: [Production] → [Canary Rollout]
|
||||
```
|
||||
<p align="center">
|
||||
<img src="assets/ch33_defense_layers.png" alt="Defense-in-Depth Layers" width="512">
|
||||
</p>
|
||||
|
||||
#### Defense Strategy 1: The "Break Glass" Policy
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 1.0 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 481 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 723 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 761 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 817 KiB |
Reference in new issue
Block a user