CalvinBackup/www-project-ai-testing-guide
1
0
Fork 0
mirror of https://github.com/OWASP/www-project-ai-testing-guide.git synced 2026-07-15 23:47:19 +02:00
Code Issues Packages Projects Releases Wiki Activity

Update 1.2_Principles_of_AI_Testing.md

Browse Source
This commit is contained in:
Matteo Meucci
2025-11-10 14:42:10 +01:00
committed by GitHub
parent d41b248ce3
commit 795bdb659d
1 changed files with 0 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Document/content/1.2_Principles_of_AI_Testing.md
-37
View File
@@ -61,43 +61,6 @@ Promote ethical, safe, and aligned system behavior through ongoing evaluation an
Effective AI testing is built upon three macro domains: Security, Privacy, Responsible AI,to build Trustworthy AI Systems. We chose these 3 core domains because they collectively address the full range of AI risks. Security ensures resilience against adversarial and infrastructure threats. Privacy prevents unintended data exposure and inference attacks. Responsible AI focuses on ethical behavior and fairness, guarding against bias and misuse. Together, they form a comprehensive framework for validating, controlling, and sustaining safe and reliable AI deployments. Each domain includes key principles that guide the evaluation of modern AI applications.
### **1\. Security**
AI systems must be resilient to adversarial threats and systemic exploitation. This includes not just model robustness but also the security of the full stack.
* **Prompt & Input Control:** Ensure system prompts, instructions, and user inputs are protected from injection or manipulation.
* **Adversarial Robustness:** Validate the system's resistance to evasion, poisoning, model theft, jailbreaks, and indirect prompt injections.
* **Infrastructure Security:** Evaluate API endpoints, plugins, RAG pipelines, and agentic workflows for vulnerabilities.
* **Supply Chain Risk**: Test models and dependencies for poisoning, unauthorized tampering, or third-party compromise.
### **2\. Privacy**
Ensure confidentiality and control over sensitive data exposed to or generated by AI systems.
* **Data Leakage Prevention:** Test against unintended disclosure of training data, private context, or user inputs.
* **Membership & Property Inference Resistance**: Assess model exposure to privacy attacks that infer if specific data was used in training.
* **Model Extraction & Exfiltration:** Simulate attacks that try to copy or replicate proprietary models.
### **3\. Responsible AI**
Promote safe, ethical, and aligned outcomes through ongoing evaluation and mitigation strategies.
* **Bias & Fairness Audits:** Identify discriminatory outputs and test model behavior across diverse demographic groups and edge cases.
* **Toxicity & Abuse Detection**: Validate how models handle hate speech, misinformation, and harmful outputs.
* **Safety Alignment**: Evaluate the system’s response to alignment bypass attacks (e.g., DAN, roleplay exploits).
* **Guardrail Coverage:** Test safety filters, refusal behaviors, and abuse-prevention mechanisms.
### **4\. Trustworthy AI Systems**
Support long-term confidence through transparency, monitoring, and governance.
* **Explainability**: Ensure users and auditors can understand how and why decisions are made.
* **Consistency & Stability:** Test models for response variance, regressions, and unexpected behavior under slight prompt changes.
* **Continuous Monitoring:** Apply post-deployment observability, drift detection, and incident alerting.
* **Policy & Regulatory Alignment:** Ensure testing processes and system behaviors comply with frameworks like NIST AI RMF \[1\], ISO 42001 \[2\], and OWASP Top 10 LLM \[3\].
---
### **When to Test AI**
ISO/IEC 23053 \[4\] structures the ML-based AI system lifecycle into a series of repeatable phases, each with clear objectives, artifacts, and governance touchpoints: