Update 2.1_Identify_AI_Threats.md

Add note on risk
This commit is contained in:
Marco Morana
2025-07-08 18:17:12 -04:00
committed by GitHub
parent 8175757126
commit 2c6a41ef75
+3 -1
View File
@@ -16,7 +16,7 @@ The Secure AI Framework (SAIF) outlines several technical and systemic risks tha
In Table 1.1 we provide the AI Risks as listed in SAIF listing each risk category along with its description, assessed business impact, the corresponding risk level based on likelihood and impact and the risk owners as characterised in SAIF model creators are” Those who train or develop AI models for use by themselves or others” and model consumers are “ Those who use AI models to build AI-powered products and applications”.The appropriate risk owner based on where the controls are applied (i.e. application/model \= Model User; data/infrastructure \= Model Creator; both \= Model Creator, Model User)
| Risk | Description | Business Impact | Risk Level (Likelihood × Impact) | Risk Owner |
| Risk | Description | Business Impact | Risk Level (Likelihood × Impact) (NOTE) | Risk Owner |
| :---- | :---- | :---- | :---- | :---- |
| Data Poisoning | Attackers inject malicious data to influence model behavior or degrade performance. | Model instability, incorrect outputs, degraded performance, possible compliance violations. | 🔴 Critical (High × High) | Model Creator |
| Unauthorized Training Data | Use of unapproved or low-integrity datasets during training introduces bias or backdoors. | Model bias, unreliable predictions, legal/regulatory exposure. | 🔴 Critical (High × High) | Model Creator |
@@ -36,6 +36,8 @@ In Table 1.1 we provide the AI Risks as listed in SAIF listing each risk categor
**Table 1.1 AI Risks (SAIF list) and Business Impacts**
Note on AI Risk Scoring Approach: Theres an important distinction between the inherent risks of implementing specific AI types—such as Retrieval-Augmented Generation (RAG), fine-tuned LLMs, or multi-agent systems—and the exposure to attacks that exploit how these systems are integrated, deployed, and protected. For example, risks like prompt injection, insecure RAG chains, and API key leakage often stem not from the model architecture itself, but from vulnerabilities in the surrounding application logic and system design. This distinction also explains why data poisoning, though rare in todays deployed ML systems, may still receive a high-likelihood and high-impact rating. Its long-term effect on model behavior and the difficulty in detecting or reversing such compromise justify its severity. Conversely, sensitive data exposure via multi-turn prompts, while more common, may be scored as medium due to partial mitigations (e.g., output filtering, context limits) or lower systemic impact in some environments. To more reliably score likelihood and impact of these AI-specific threats—especially those tied to known vulnerabilities—a structured risk methodology is needed. The OWASP AI Vulnerability Scoring System (AIVSS) https://aivss.owasp.org offers a promising foundation. It incorporates factors such as exploitability, predictability, impact severity, and mitigation coverage—aligned specifically for evaluating threats in AI-driven systems. As the threat landscape for AI evolves, standardized scoring frameworks like AIVSS will be essential for accurate and actionable risk prioritization.
At this stage, analyzing business impact allows the threat model to focus on the most critical AI risks by aligning control testing with organizational priorities. Whether a business is primarily an AI model user, creator, or both determines who owns the responsibility for risk mitigation. Since each organization has a unique AI risk profile, shaped by its specific use cases, functional dependencies, and the sensitivity of exposed data, this alignment ensures that threat modeling and AI testing efforts are tailored to safeguard what matters most to the business. Ultimately, mapping SAIF risks to business consequences is essential for prioritizing threats and guiding effective mitigation strategies.
## **CIA-Based Threat Analysis for Information Security Risks**