ai-llm-red-team-handbook/docs/course_materials/Slide_Deck_Week1_Foundations.md

marp, theme, paginate, backgroundColor, header, footer

marp	theme	paginate	backgroundColor	header	footer
true	default	true		AI Red Team Ops | Week 1: Foundations	© 2026 AI Red Team Handbook

AI Red Team Ops

Week 1: Foundations & Architecture

"Understanding the Alien Mind"

---

Agenda: Week 1

1. The "Alien Mind" Why AI is not Human
2. Transformer Architecture How it actually works (Simplified)
3. The Tokenization Gap The byte-stream vulnerability
4. Threat Modeling Adapting STRIDE for LLMs
5. Lab 1.1 Preview

---

1. The Alien Mind: Probabilistic vs. Deterministic

Classical Software

Deterministic if x == 5: return "Hello"

• Logic: Rigid, verifiable.
• Failures: Bugs, crashes.
• Security: Input Validation, Access Control.

AI Systems

Probabilistic predict_next_token("Hello") -> ["World": 99%]

• Logic: Statistical, fuzzy.
• Failures: Hallucinations, bias.
• Security: Prompt Injection, Adversarial Data.

---

2. Transformer Architecture

The engine exploring the "Latent Space" of potential meanings.

1. Input: Raw text.
2. Embedding: Converts words to high-dimensional vectors (Meaning).
3. Attention: The model "pays attention" to relevant words (e.g., "Bank" relates to "River" not "Money").
4. Output: Aprobability distribution.

---

3. The Tokenization Vulnerability

LLMs do not see words. They see integers.

Security Implications

• Bypass Length Filters: A malicious string can look short in characters but be massive in tokens (or vice versa).
• "Glitch Tokens": Undefined tokens can cause the model to crash or output garbage.
• Injection: Hidden control characters can be smuggled in.

---

4. Threat Modeling (STRIDE for AI)

Every component is a target.

Threat	Description	AI Example
Spoofing	Impersonation	Prompt Injection ("Act as Admin")
Tampering	Modifying Data	RAG Poisoning (Bad Docs)
Repudiation	Deniability	No Logs for AI Decisions
Info Disclosure	Leaking Secrets	PII Extraction / Training Data
DoS	Exhaustion	Context Window Flooding ($$$)
Elevation	Privilege Esc.	Plugin Exploitation (Confused Deputy)

---

Lab 1.1: The Tokenization Gap

Objective: Use tiktoken to identify discrepancies between "human text" and "machine tokens".

Scenario: You need to input a payload that looks like safe English but parses as a command.

Steps:

1. Install tiktoken.
2. Compare encoding of user vs user.
3. Find a string where len(str) increases but token_count decreases.

Get to work!