Update 2.1.2_Identify_RAI_threats.md

This commit is contained in:
Matteo Meucci
2025-06-18 11:57:19 +02:00
committed by GitHub
parent 36649a80ca
commit 95da1a4b50
@@ -85,7 +85,15 @@ The following tables provide a structured overview, facilitating clear visibilit
## **Four Pillars of AI Testing**
Need an intro and this note
Artificialintelligence systems blend traditional software, complex data pipelines, and probabilistic models.This hybrid nature means that many wellestablished security testing practices (e.g., network scanning, penetration testing, securecode review) remain indispensable, yet they are no longer sufficient on their own.AI brings new threat vectors—prompt manipulation, data poisoning, model extraction, and unintended agency, to name only a few—that require purposebuilt test strategies.
To address these emerging risks systematically, we group AIspecific validation activities into four complementary pillars:
1. AIApplication Testing validating everything a human or downstream machine touches: chat interfaces, APIs, agents/plugins, and multimodal UX flows.
2. AIModel Testing stresstesting the core model itself throughout training, finetuning, and inference to expose weakness in robustness, fairness, and alignment.
3. AIInfrastructure Testing hardening the serving stack, orchestration layer, and plugin/agent sandbox so that the models power cannot be abused.
4. AIData Testing assuring the integrity, provenance, diversity, and legality of data that feed the model before, during, and after training.
Together, these pillars create a defenseindepth framework: weaknesses caught late in one pillar are often prevented early in another, and insights from any pillar can be fed back into securebydesign requirements for subsequent releases.
The threat scenarios presented in this guide highlight AI-specific risks and corresponding security considerations. While we identify a range of potential threats, some associated security controls and test procedures fall outside the scope of this guide. Our intent is not to replace or duplicate existing, well-established security testing methodologies, but rather to extend them by focusing on threats unique to AI systems.