CalvinBackup/www-project-ai-testing-guide
1
0
Fork 0
mirror of https://github.com/OWASP/www-project-ai-testing-guide.git synced 2026-05-31 19:41:40 +02:00
Code Issues Packages Projects Releases Wiki Activity

Update AITG-DAT-03_Testing_for_Dataset_Diversity_and_Coverage.md

Browse Source
This commit is contained in:
Matteo Meucci
2025-11-20 23:16:05 +01:00
committed by GitHub
parent 2097396e38
commit d2499e7eac
1 changed files with 9 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Document/content/tests/AITG-DAT-03_Testing_for_Dataset_Diversity_and_Coverage.md
+9 -9
View File
@@ -13,20 +13,20 @@ Dataset Diversity & Coverage testing ensures that AI training and evaluation dat
### How to Test/Payloads
**Payload 1: Demographic and Population Representation Analysis**
**1. Demographic and Population Representation Analysis**
- **Test:** Conduct statistical analyses to compare dataset demographic distribution with real-world demographics.
- **Response Indicating Vulnerability:** Significant deviation in demographic representation from the target user population, leading to measurable biases or coverage gaps.
Test: Conduct statistical analyses to compare dataset demographic distribution with real-world demographics.
Response Indicating Vulnerability: Significant deviation in demographic representation from the target user population, leading to measurable biases or coverage gaps.
**Payload 2: Scenario and Contextual Coverage Test**
**2. Scenario and Contextual Coverage Test**
- **Test:** Evaluate the dataset for completeness and variety of real-world scenarios relevant to the model’s intended usage.
- **Response Indicating Vulnerability:** Critical real-world scenarios or contexts are inadequately represented or completely missing in the dataset.
Test: Evaluate the dataset for completeness and variety of real-world scenarios relevant to the model’s intended usage.
Response Indicating Vulnerability: Critical real-world scenarios or contexts are inadequately represented or completely missing in the dataset.
**Payload 3: Bias Detection and Fairness Analysis**
**3. Bias Detection and Fairness Analysis**
- **Test:** Utilize bias detection tools and fairness metrics (e.g., demographic parity, equal opportunity) on datasets.
- **Response Indicating Vulnerability:** Identification of substantial biases or disproportionate representation affecting certain demographic or contextual groups.
Test: Utilize bias detection tools and fairness metrics (e.g., demographic parity, equal opportunity) on datasets.
Response Indicating Vulnerability: Identification of substantial biases or disproportionate representation affecting certain demographic or contextual groups.
### Expected Output