From d2499e7eac5248ee6d8c81bbd504e1bf75deea6e Mon Sep 17 00:00:00 2001 From: Matteo Meucci Date: Thu, 20 Nov 2025 23:16:05 +0100 Subject: [PATCH] Update AITG-DAT-03_Testing_for_Dataset_Diversity_and_Coverage.md --- ...sting_for_Dataset_Diversity_and_Coverage.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/Document/content/tests/AITG-DAT-03_Testing_for_Dataset_Diversity_and_Coverage.md b/Document/content/tests/AITG-DAT-03_Testing_for_Dataset_Diversity_and_Coverage.md index fa97c13..badfad0 100644 --- a/Document/content/tests/AITG-DAT-03_Testing_for_Dataset_Diversity_and_Coverage.md +++ b/Document/content/tests/AITG-DAT-03_Testing_for_Dataset_Diversity_and_Coverage.md @@ -13,20 +13,20 @@ Dataset Diversity & Coverage testing ensures that AI training and evaluation dat ### How to Test/Payloads -**Payload 1: Demographic and Population Representation Analysis** +**1. Demographic and Population Representation Analysis** -- **Test:** Conduct statistical analyses to compare dataset demographic distribution with real-world demographics. -- **Response Indicating Vulnerability:** Significant deviation in demographic representation from the target user population, leading to measurable biases or coverage gaps. +Test: Conduct statistical analyses to compare dataset demographic distribution with real-world demographics. +Response Indicating Vulnerability: Significant deviation in demographic representation from the target user population, leading to measurable biases or coverage gaps. -**Payload 2: Scenario and Contextual Coverage Test** +**2. Scenario and Contextual Coverage Test** -- **Test:** Evaluate the dataset for completeness and variety of real-world scenarios relevant to the model’s intended usage. -- **Response Indicating Vulnerability:** Critical real-world scenarios or contexts are inadequately represented or completely missing in the dataset. +Test: Evaluate the dataset for completeness and variety of real-world scenarios relevant to the model’s intended usage. +Response Indicating Vulnerability: Critical real-world scenarios or contexts are inadequately represented or completely missing in the dataset. -**Payload 3: Bias Detection and Fairness Analysis** +**3. Bias Detection and Fairness Analysis** -- **Test:** Utilize bias detection tools and fairness metrics (e.g., demographic parity, equal opportunity) on datasets. -- **Response Indicating Vulnerability:** Identification of substantial biases or disproportionate representation affecting certain demographic or contextual groups. +Test: Utilize bias detection tools and fairness metrics (e.g., demographic parity, equal opportunity) on datasets. +Response Indicating Vulnerability: Identification of substantial biases or disproportionate representation affecting certain demographic or contextual groups. ### Expected Output