Update 2.2_Appendix_E.md

Reorganized more organically with titles etc
This commit is contained in:
Marco Morana
2025-10-16 08:54:50 -04:00
committed by GitHub
parent 3be20eaa24
commit f81590039d
+16 -68
View File
@@ -1,18 +1,21 @@
# Appendix E: SAIF AI Threat Targeted Components & CVEs/CWEs
## 2.2 Appendix E: AI Threats Mapping to AI Components Vulnerabilities (CVEs & CWEs)
This appendix guides penetration testers on translating discovered CVEs and CWEs into AI-specific threats and concrete test cases mapped against the SAIF components of an AI architecture. CVEs generally point to vulnerabilities in the underlying technology stack — libraries, frameworks, APIs, and services that implement user interfaces, model layers, supporting infrastructure, or data sources. Because the pen tests described here target a live application, careful scoping is essential: testers must first identify which SAIF components and subcomponents are in scope, enumerate the exact technologies deployed for each, and use that inventory to prioritize CVE/CWE enumeration and threat simulations. In-scope items commonly include components owned or operated by the organization and directly involved in the request→response flow — for example, chat UIs, API backends (e.g., FastAPI), session/orchestration layers, model orchestration frameworks (e.g., LangChain or LlamaIndex), vector stores (Redis, Pinecone, Weaviate), ETL/data pipelines, model-serving endpoints, and internally managed connectors. Because these components can contain outdated, misconfigured, or otherwise exploitable dependencies, the first operational step is threat enumeration: map each in-scope SAIF component to its tech stack, identify relevant CVEs (and corresponding CWEs), and derive likely exploit paths. That mapping then drives focused validation with scanners, SCA tools, and proof-of-concept testing so testers can prioritize, reproduce, and demonstrate how conventional software flaws translate into AI-centric impacts.
**AI Penetration Testing Framework: Scoping, CVE/CWE Mapping, and Threat Correlation**
This appendix guides penetration testers on mapping discovered CVEs and CWEs in SAIF components of an AI architecture to AI-specific threats. CVEs generally point to vulnerabilities in the underlying technology stack such as libraries, frameworks, APIs used to build AI Systems and AI Applications. Because the pen tests described here target a live AI system/Application, careful scoping is essential: testers must first identify which SAIF components and subcomponents are in scope, enumerate the exact technologies deployed for each, and use that inventory to prioritize CVE/CWE enumeration and threat simulations. In-scope items commonly include components owned or operated by the organization and directly involved in the request→response flow — for example, chat UIs, API backends (e.g., FastAPI), session/orchestration layers, model orchestration frameworks (e.g., LangChain or LlamaIndex), vector stores (Redis, Pinecone, Weaviate), ETL/data pipelines, model-serving endpoints, and internally managed connectors. Because these components can contain outdated, misconfigured, or otherwise exploitable dependencies, the first operational step is threat enumeration: map each in-scope SAIF component to its tech stack, identify relevant CVEs (and corresponding CWEs), and derive likely exploit paths. That mapping then drives focused validation with scanners, SCA tools, and proof-of-concept testing so testers can prioritize, reproduce, and demonstrate how conventional software flaws translate into AI-centric impacts.
To start, the tester performs **Threat enumeration and mappping of CVE exploit paths across the in-scope technology stack**. This begins with discovering known vulnerabilities using both SCA and runtime tools: software composition analyzers (Snyk, Trivy, Dependabot) reveal vulnerable dependencies and libraries, while network and host scanners (Nessus, Nuclei) validate active exposures in services and APIs. Runtime telemetry and host-level inspection add further evidence of exploitability in live environments where vulnerable components are installed and running. Identified CVEs are then translated into AI-specific risks using the AI Threats column: a web issue like a FastAPI sanitization flaw (CVE-2022-36067) becomes a direct prompt-injection vector (T01-DPIJ) when an LLM ingests tainted inputs, and an ETL or retrieval vulnerability such as CVE-2022-40127 can be leveraged to perform remote code execution or data corruption that manifests as data poisoning (T01-DMP) in a RAG pipeline. Mapping each CVE to the relevant AI threat converts a routine vulnerability finding into a concrete attack path, making it possible to explain and demonstrate the real impact on model behavior, data integrity, confidentiality, and availability.
**Step 1 — Threat Enumeration and CVE Exploit Path Mapping**
The process of mapping threats to Ai system vulnerabilities starts by identifying known vulnerabilities in AI systems/applications using Software composition analyzers (SCAs) and runtime tools. SCA Tools (e.g., Snyk, Trivy, Dependabot, OWASP Dependency-Check, and GitHub Advanced Security) will flag vulnerable third party software dependencies, while scanners such as Nessus and Nuclei can confirm active CVE exposures in APIs and services. Runtime telemetry and host inspection can also validate which CVEs are exploitable in live environments. These CVEs are then mapped to AI-specific threats (i.e. TA0i-XX threats) outlined in this guide: for example, a FastAPI sanitization flaw (CVE-2022-36067) can be part of a prompt-injection vector (T01-DPIJ), and an Airflow ETL vulnerability (CVE-2022-40127) can lead to data poisoning (T01-DMP) in a RAG pipeline.
For each SAIF component in scope, the tester inspects subcomponents to identify where injection, poisoning, or manipulation are possible, confirms the actual technologies deployed, and runs tests to discover vulnerable or unpatched libraries and CVEs. Those technical findings then drive simulations of AI-specific attacks for example prompt injection, model inversion and membership inference, data poisoning, and runtime DoS, so the tester can demonstrate real impact on the application and its model behavior. Pen test reports should use the “Threat enumeration and CVE exploit-path mappings” table to preserve traceability between vulnerabilities and AI impacts. The mapping lets a tester convert a conventional software finding into a concrete AI attack path and explain how exploitation affects data integrity, confidentiality, availability, or model trust. For example, Redis used in SAIF #4 Application Layer for session caching, API state management, and job queues was found vulnerable to CVE-2022-0543, which can lead to multiple AI-specific risks: data leakage (T01-SID), model disruption (T01-DoSM), and model manipulation (T01-MTD). In short, a single Redis compromise can escalate from infrastructure-level control to sensitive information exposure and altered AI behavior, undermining the systems integrity and trust. Findings like this should clearly link the vulnerability to relevant CWEs, mapped AI threats, exploit paths, and reproducible validation steps so both security and AI teams can remediate effectively.
For each SAIF component in scope, testers review subcomponents, confirm deployed technologies, and run focused tests to find exploitable or unpatched libraries. These findings drive AI-specific attack simulations such as prompt injection, model inversion, data poisoning, or runtime DoS to reveal real application impact. Using the CVE exploit-path mapping table, testers can maintain traceability from vulnerability to AI impact. For instance, Redis in SAIF #4 (Application Layer) vulnerable to CVE-2022-0543 links to risks like data leakage (T01-SID), model disruption (T01-DoSM), and manipulation (T01-MTD). A single Redis compromise can escalate from infrastructure control to model tampering—compromising data integrity, availability, and trust.
The second recommended step is to perform a **Threat enumeration and CWE exploit-path mapping** This step transforms vulnerability-centric testing into design-level assurance. By classifying findings under CWE categories, the pen tester bridges the gap between patch management and resilient AI architecture. CWE mapping clarifies attacker objectives, expands test coverage beyond isolated CVEs, and guides remediation that strengthens entire system layers rather than individual components. The CWE-based table reframes technical flaws as architectural weaknesses, for instance, CWE-20 (Improper Input Validation) exposes weak parsing logic, CWE-276 (Incorrect Default Permissions) reveals insecure defaults in data storage such as S3 buckets, and CWE-345 (Insufficient Verification of Data Authenticity) uncovers trust and integrity flaws in RAG ingestion. This approach helps testers not only find where AI applications break, but also understand why they break and how to redesign them to resist future exploitation.
**Step 2 — Threat Enumeration and CWE Exploit Path Mapping**
The second recommended step is to perform a AI threat enumeration and CWE exploit-path mapping. This step transforms vulnerability-centric testing into design-level assurance. By classifying findings under CWE categories, the pen tester bridges the gap between patch management and resilient AI architecture. CWE mapping clarifies attacker objectives, expands test coverage beyond isolated CVEs, and guides remediation that strengthens entire system layers rather than individual components. The CWE-based table reframes technical flaws as architectural weaknesses, for instance, CWE-20 (Improper Input Validation) exposes weak parsing logic, CWE-276 (Incorrect Default Permissions) reveals insecure defaults in data storage such as S3 buckets, and CWE-345 (Insufficient Verification of Data Authenticity) uncovers trust and integrity flaws in RAG ingestion. This approach helps testers not only find where AI applications break, but also understand why they break and how to redesign them to resist future exploitation.
Finally, the third step is to look at **AI Threats, Targeted CWEs and Provide Recommendations to Fix Them** in the Pen Testing Report. CWEs being targeted by a threat needs to be accompanied by secure design recommendations, such as enforcing schema validation, disabling default public access, verifying dataset authenticity, or encrypting sensitive data. This means pen testers can move from “here is how I broke it” to “here is how you should redesign it to prevent recurrence.” As pen testers revisit AI systems/application in scope for testing as these mighr change, they can update the CVE and CWE of newly discovered vulnerabilities and use the AI Threats column as a checklist for attack simulations in future red-team exercises. Over time, this evolving matrix becomes a living document that supports secure design, ongoing validation, and resilience in AI-enabled systems.
**Step 3 — AI Threat Mapping and Secure Design Recommendations**
Finally, the third step is to look at AI threats, targeted CWEs and provide recommendations to Fix Them in the Pen Testing Report. Vulnerability types/CWEs might represent security design flaws or mis-configurations that could be targeted by AI threats. It is important that when these CWEs are included in the test report are also accompanied by recommendations to fix them, such as enforcing input validation, disabling default public access, verifying dataset authenticity, or encrypting sensitive data as examples. This means pen testers can move from “here is how I broke it” to “here is how you should secufre configure it or redesign it to prevent recurrence.” As pen testers revisit AI systems/application in scope for testing as these mighr change, they can update the CVE and CWE of newly discovered vulnerabilities and use the AI Threats column as a checklist for attack simulations in future red-team exercises. Over time, this evolving matrix becomes a living document that supports secure design, ongoing validation, and resilience in AI-enabled systems.
## AI Threat enumeration and CVE exploit path mapping
> In this section we provide a mapping of SAIF components to AI threats and examples of component dependent tech-stack CVEs that can be exploited
**AI Threat enumeration and CVE exploit path mapping**
In this section we provide a mapping of SAIF components to AI threats and examples of component dependent tech-stack CVEs that can be exploited
| SAIF Component (Number) | Sub-Components | Tech Stack (Chatbot + RAG) | Mapped Threats | Example CVEs in Tech Stack |
|--------------------------|----------------|-----------------------------|----------------|----------------------------|
@@ -35,9 +38,8 @@ Finally, the third step is to look at **AI Threats, Targeted CWEs and Provide Re
| (18) Data Sources | Internal KBs, CRM, telemetry | Confluence, Jira, Elastic, Splunk | T01-SID, T01-DMP, T01-VEW, T01-MIS | Confluence RCE (CVE-2023-22515); Jira auth bypass (CVE-2020-14181); ElasticSearch RCE (CVE-2015-1427); Splunk RCE (CVE-2022-32158) |
| (19) External Sources | Public datasets, 3rd party APIs/feeds | Wikipedia, Common Crawl, arXiv, News APIs | T01-MIMI, T01-SID, T01-DMP, T01-MIS | Dataset poisoning risks (no CVEs, CWE-driven); API poisoning (CWE-345: Insufficient Verification of Data Authenticity) |
## AI Threat enumeration and Targeted CWEs
> In this section we provide a mapping of SAIF components to AI threats and examples of vulnerability types/CWEs that can be exploited
**AI Threat enumeration and Targeted CWEs**
In this section we provide a mapping of SAIF components to AI threats and examples of vulnerability types/CWEs that can be exploited
| SAIF Component | Mapped Threats | Targeted CWEs |
|----------------|----------------|----------------|
@@ -60,9 +62,8 @@ Finally, the third step is to look at **AI Threats, Targeted CWEs and Provide Re
| (18) Data Sources | T01-SID, T01-DMP, T01-VEW, T01-MIS | CWE-20, CWE-200, CWE-345, CWE-352, CWE-359, CWE-494, CWE-502, CWE-522, CWE-74, CWE-825, CWE-829, CWE-918 |
| (19) External Sources | T01-MIMI, T01-SID, T01-DMP, T01-MIS | CWE-20, CWE-200, CWE-203, CWE-345, CWE-352, CWE-359, CWE-494, CWE-522, CWE-74, CWE-825 |
## AI Threats, Targeted CWEs and Recommendations to Fix Them
> In this section we provide a mapping of SAIF components to threats, possibly targeted CWEs, the rationale for CWEs being targeted, and recommendations for fixing them.
**AI Threats, Targeted CWEs and Recommendations to Fix Them**
In this section we provide a mapping of SAIF components to threats, possibly targeted CWEs, the rationale for CWEs being targeted, and recommendations for fixing them.
- [(2) User Input](#2-user-input)
- [(3) User Output](#3-user-output)
@@ -88,9 +89,7 @@ Finally, the third step is to look at **AI Threats, Targeted CWEs and Provide Re
## (2) User Input
**Summary:** User Input is the front door of the system — every downstream component depends on it. Without strong input validation, filtering, and limits, it becomes the main vector for prompt injection, data leakage, DoS, and toxicity propagation.
**Threats:** T01-DPIJ, T01-IPI J, T01-SID, T01-DoSM, T01-IOH, T01-MTU
**Targeted CWEs:**
CWE-20, CWE-74, CWE-94, CWE-707, CWE-200, CWE-359, CWE-522, CWE-400, CWE-770, CWE-787, CWE-116, CWE-79
@@ -136,11 +135,8 @@ CWE-20, CWE-74, CWE-94, CWE-707, CWE-200, CWE-359, CWE-522, CWE-400, CWE-770, CW
---
## (3) User Output
**Summary:** The last mile to users/connected systems; without control, its a vector for excessive agency, prompt leakage, misinformation, and unsafe rendering.
**Threats:** T01-EA, T01-SPL, T01-MIS, T01-IOH
**Targeted CWEs:**
CWE-284, CWE-285, CWE-200, CWE-209, CWE-359, CWE-532, CWE-116, CWE-79, CWE-75, CWE-345, CWE-1204
@@ -177,11 +173,8 @@ CWE-284, CWE-285, CWE-200, CWE-209, CWE-359, CWE-532, CWE-116, CWE-79, CWE-75, C
---
## (4) Application
**Summary:** Orchestration brain (sessions, APIs, business logic). Weak validation or access controls can cascade into systemic compromise.
**Threats:** T01-DPIJ, T01-IPI J, T01-SID, T01-DoSM, T01-MTU, T01-IOH, T01-EA, T01-SPL, T01-MIS
**Targeted CWEs:**
CWE-20, CWE-74, CWE-94, CWE-200, CWE-209, CWE-359, CWE-522, CWE-400, CWE-770, CWE-787, CWE-116, CWE-79, CWE-75, CWE-284, CWE-285, CWE-345, CWE-1204
@@ -214,11 +207,8 @@ CWE-20, CWE-74, CWE-94, CWE-200, CWE-209, CWE-359, CWE-522, CWE-400, CWE-770, CW
---
## (5) Agent / Plugin
**Summary:** Extended arms of the system; vulnerable to IPIJ, secrets handling, tampering, excessive actions, and unsafe workflows.
**Threats:** T01-IPI J, T01-SID, T01-MTD, T01-EA, T01-VEW
**Targeted CWEs:**
CWE-20, CWE-74, CWE-94, CWE-200, CWE-359, CWE-522, CWE-284, CWE-285, CWE-276, CWE-494, CWE-829, CWE-918, CWE-502
@@ -247,11 +237,8 @@ CWE-20, CWE-74, CWE-94, CWE-200, CWE-359, CWE-522, CWE-284, CWE-285, CWE-276, CW
---
## (6) External Sources
**Summary:** Bridges to the outside world; unverified data can inject poison, trigger unsafe actions, or spread misinformation.
**Threats:** T01-IPI J, T01-MTD, T01-SID, T01-EA, T01-VEW, T01-DMP
**Targeted CWEs:**
CWE-20, CWE-74, CWE-94, CWE-200, CWE-359, CWE-522, CWE-276, CWE-284, CWE-285, CWE-494, CWE-829, CWE-918, CWE-502, CWE-353, CWE-345
@@ -276,11 +263,8 @@ CWE-20, CWE-74, CWE-94, CWE-200, CWE-359, CWE-522, CWE-276, CWE-284, CWE-285, CW
---
## (7) Input Handling
**Summary:** The filter layer; weak parsing/schema enforcement lets adversarial inputs/injections slip through.
**Threats:** T01-DPIJ, T01-AIE, T01-SID, T01-LSID, T01-DoSM, T01-SPL, T01-VEW
**Targeted CWEs:**
CWE-20, CWE-74, CWE-94, CWE-200, CWE-359, CWE-522, CWE-532, CWE-209, CWE-400, CWE-770, CWE-787, CWE-79, CWE-116, CWE-75, CWE-918
@@ -302,11 +286,8 @@ CWE-20, CWE-74, CWE-94, CWE-200, CWE-359, CWE-522, CWE-532, CWE-209, CWE-400, CW
---
## (8) Output Handling
**Summary:** Safety gate before delivery; failure here leaks sensitive data, misinformation, and unsafe content.
**Threats:** T01-LSID, T01-SID, T01-DoSM, T01-SPL, T01-IOH, T01-TDL, T01-MTU, T01-EA, T01-MIS
**Targeted CWEs:**
CWE-79, CWE-116, CWE-75, CWE-200, CWE-209, CWE-359, CWE-532, CWE-522, CWE-400, CWE-770, CWE-787, CWE-284, CWE-285, CWE-345, CWE-1204
@@ -334,12 +315,9 @@ CWE-79, CWE-116, CWE-75, CWE-200, CWE-209, CWE-359, CWE-532, CWE-522, CWE-400, C
---
## (9) Model
**Summary:** The core intelligence; targeted by injection, poisoning, theft, inversion, DoS, and unsafe outputs.
**Threats:**
T01-DPIJ, T01-IPI J, T01-SCMP, T01-AIE, T01-DPFT, T01-RMP, T01-DMP, T01-SID, T01-MIMI, T01-TDL, T01-DoSM, T01-LSID, T01-SPL, T01-VEW, T01-MTU, T01-IOH, T01-MTR, T01-EA, T01-MIS
**Targeted CWEs:**
CWE-20, CWE-74, CWE-94, CWE-200, CWE-209, CWE-359, CWE-522, CWE-532, CWE-276, CWE-284, CWE-285, CWE-400, CWE-770, CWE-787, CWE-918, CWE-502, CWE-494, CWE-345, CWE-353, CWE-1204, CWE-116, CWE-119, CWE-830, CWE-829, CWE-640, CWE-693, CWE-75, CWE-79
@@ -373,11 +351,8 @@ CWE-20, CWE-74, CWE-94, CWE-200, CWE-209, CWE-359, CWE-522, CWE-532, CWE-276, CW
---
## (10) Model Storage Infrastructure
**Summary:** Crown jewels at rest — must be encrypted, signed, and access-controlled.
**Threats:** T01-DPFT, T01-SCMP, T01-MTR, T01-MTD
**Targeted CWEs:**
CWE-276, CWE-284, CWE-285, CWE-200, CWE-359, CWE-522, CWE-494, CWE-353, CWE-922
@@ -396,11 +371,8 @@ CWE-276, CWE-284, CWE-285, CWE-200, CWE-359, CWE-522, CWE-494, CWE-353, CWE-922
---
## (11) Model Serving Infrastructure
**Summary:** Execution gateway; must resist poisoning, theft, DoS, and unsafe outputs.
**Threats:** T01-SCMP, T01-MTU, T01-MTR, T01-DoSM
**Targeted CWEs:**
CWE-276, CWE-284, CWE-285, CWE-400, CWE-770, CWE-787, CWE-494, CWE-353, CWE-345, CWE-1204, CWE-75
@@ -419,11 +391,8 @@ CWE-276, CWE-284, CWE-285, CWE-400, CWE-770, CWE-787, CWE-494, CWE-353, CWE-345,
---
## (12) Evaluation
**Summary:** The safety lens; poison/bypass here yields false assurance.
**Threats:** T01-AIE, T01-DMP, T01-LSID, T01-SID, T01-TDL, T01-DoSM, T01-MTU, T01-IOH, T01-MIS
**Targeted CWEs:**
CWE-20, CWE-116, CWE-200, CWE-209, CWE-359, CWE-532, CWE-400, CWE-770, CWE-787, CWE-345, CWE-1204
@@ -445,11 +414,8 @@ CWE-20, CWE-116, CWE-200, CWE-209, CWE-359, CWE-532, CWE-400, CWE-770, CWE-787,
---
## (13) Training & Tuning
**Summary:** Where knowledge is forged; poor data embeds lasting bias/backdoors.
**Threats:** T01-AIE, T01-MIS, T01-DPFT, T01-SCMP, T01-MTD
**Targeted CWEs:**
CWE-20, CWE-116, CWE-345, CWE-353, CWE-494, CWE-276, CWE-284, CWE-285, CWE-200, CWE-359
@@ -471,11 +437,8 @@ CWE-20, CWE-116, CWE-345, CWE-353, CWE-494, CWE-276, CWE-284, CWE-285, CWE-200,
---
## (14) Model Frameworks & Code
**Summary:** ML runtime backbone; supply chain or unsafe integrations taint the system.
**Threats:** T01-SCMP, T01-MTD, T01-VEW
**Targeted CWEs:**
CWE-94, CWE-95, CWE-829, CWE-494, CWE-353, CWE-276, CWE-284, CWE-285, CWE-918, CWE-502
@@ -491,11 +454,8 @@ CWE-94, CWE-95, CWE-829, CWE-494, CWE-353, CWE-276, CWE-284, CWE-285, CWE-918, C
---
## (15) Data Storage Infrastructure
**Summary:** Knowledge vault; poisoning/tampering/leaks here undermine integrity & confidentiality.
**Threats:** T01-RMP, T01-DMP, T01-DPFT, T01-SCMP, T01-SID, T01-MTD, T01-LSID
**Targeted CWEs:**
CWE-276, CWE-284, CWE-285, CWE-200, CWE-359, CWE-522, CWE-532, CWE-400, CWE-770, CWE-787, CWE-494, CWE-353, CWE-345, CWE-922
@@ -514,11 +474,8 @@ CWE-276, CWE-284, CWE-285, CWE-200, CWE-359, CWE-522, CWE-532, CWE-400, CWE-770,
---
## (16) Training Data
**Summary:** Root of trust; compromise propagates to all downstream behavior.
**Threats:** T01-MIMI, T01-TDL, T01-SID
**Targeted CWEs:**
CWE-200, CWE-359, CWE-522, CWE-345, CWE-353, CWE-494, CWE-276, CWE-284, CWE-285
@@ -537,11 +494,8 @@ CWE-200, CWE-359, CWE-522, CWE-345, CWE-353, CWE-494, CWE-276, CWE-284, CWE-285
---
## (17) Data Filtering & Processing
**Summary:** Gatekeeper stage; weak validation lets poisoned/sensitive data pass.
**Threats:** T01-RMP, T01-DMP, T01-DPFT, T01-SID, T01-MIMI, T01-TDL, T01-VEW, T01-MIS
**Targeted CWEs:**
CWE-20, CWE-116, CWE-200, CWE-359, CWE-345, CWE-353, CWE-494, CWE-276, CWE-284, CWE-285, CWE-400, CWE-770, CWE-787, CWE-829, CWE-918, CWE-502
@@ -563,11 +517,8 @@ CWE-20, CWE-116, CWE-200, CWE-359, CWE-345, CWE-353, CWE-494, CWE-276, CWE-284,
---
## (18) Data Sources
**Summary:** Entry point of truth; without provenance checks, they introduce poisoned/unsafe content.
**Threats:** T01-SID, T01-DMP, T01-VEW, T01-MIS
**Targeted CWEs:**
CWE-200, CWE-359, CWE-522, CWE-345, CWE-353, CWE-494, CWE-829, CWE-918, CWE-502
@@ -586,11 +537,8 @@ CWE-200, CWE-359, CWE-522, CWE-345, CWE-353, CWE-494, CWE-829, CWE-918, CWE-502
---
## (19) External Sources
**Summary:** Outside the trust boundary; major vectors for poisoning, leakage, and misinformation.
**Threats:** T01-MIMI, T01-SID, T01-DMP, T01-MIS
**Targeted CWEs:**
CWE-200, CWE-359, CWE-522, CWE-345, CWE-353, CWE-494, CWE-918, CWE-829