diff --git a/docs/Chapter_33_Red_Team_Automation.md b/docs/Chapter_33_Red_Team_Automation.md index e9c47c6..d3404fd 100644 --- a/docs/Chapter_33_Red_Team_Automation.md +++ b/docs/Chapter_33_Red_Team_Automation.md @@ -39,6 +39,10 @@ Finding a vulnerability once is good; ensuring it never returns is better. As AI Automation shifts security "left" (earlier in the lifecycle). +

+ Shift Left Comparison Timeline +

+ - **Architectural Factor:** LLM behavior is non-deterministic. Running a test suite once isn't enough; pipelines allow for statistical validation (running 50 times) to ensure robustness. - **Training Artifact:** Continuous Fine-Tuning (CFT) introduces "catastrophic forgetting," where a model might forget its safety training. Automated tests catch this drift immediately. - **Input Processing:** By mechanizing the "Attacker" role, we effectively create an adversarial loss function for the development process. @@ -68,16 +72,9 @@ We will design a simple pipeline: ### How the Pipeline Works -```text -[CI/CD Flow] -Developer → [Push Code] → GitHub Actions - ↓ - [Step 1: Build App] - ↓ - [Step 2: Start Local LLM Server] - ↓ - [Step 3: Run Attack Suite] ← (Fails if success rate > 0%) -``` +

+ Automated Security Pipeline Flowchart +

### Mechanistic Explanation @@ -97,6 +94,10 @@ This script defines a `pytest` suite that acts as a security gate. It defines a 2. **Asserts:** Checking for refusal keywords (Compliance = Fail). 3. **Retry Logic:** Handling API flakiness. +

+ Pytest Logic Sequence Diagram +

+ ```python #!/usr/bin/env python3 """ @@ -215,8 +216,13 @@ if __name__ == "__main__": - **What:** Visualizing failure rates over time. - **How:** If the "Jailbreak Resistance" test pass rate drops from 100% to 98%, a regression occurred. +- **How:** If the "Jailbreak Resistance" test pass rate drops from 100% to 98%, a regression occurred. - **Effectiveness:** High. +

+ Regression Monitoring Dashboard +

+ #### Detection Method 2: Canary Deployments - **What:** Deploying the new model to 1% of users. @@ -256,12 +262,9 @@ if __name__ == "__main__": #### Defense-in-Depth Approach -```text -Layer 1: [Local Git Hook] → [Prevent committing keys] -Layer 2: [CI Pipeline] → [Run Pytest Suite] -Layer 3: [Staging] → [Red Team Audit] -Layer 4: [Production] → [Canary Rollout] -``` +

+ Defense-in-Depth Layers +

#### Defense Strategy 1: The "Break Glass" Policy diff --git a/docs/assets/ch33_defense_layers.png b/docs/assets/ch33_defense_layers.png new file mode 100644 index 0000000..b7c3f60 Binary files /dev/null and b/docs/assets/ch33_defense_layers.png differ diff --git a/docs/assets/ch33_pipeline_flow.png b/docs/assets/ch33_pipeline_flow.png new file mode 100644 index 0000000..a92ccbb Binary files /dev/null and b/docs/assets/ch33_pipeline_flow.png differ diff --git a/docs/assets/ch33_pytest_sequence.png b/docs/assets/ch33_pytest_sequence.png new file mode 100644 index 0000000..b26ebbc Binary files /dev/null and b/docs/assets/ch33_pytest_sequence.png differ diff --git a/docs/assets/ch33_regression_dashboard.png b/docs/assets/ch33_regression_dashboard.png new file mode 100644 index 0000000..cdd2489 Binary files /dev/null and b/docs/assets/ch33_regression_dashboard.png differ diff --git a/docs/assets/ch33_shift_left_timeline.png b/docs/assets/ch33_shift_left_timeline.png new file mode 100644 index 0000000..52881d4 Binary files /dev/null and b/docs/assets/ch33_shift_left_timeline.png differ