CalvinBackup/www-project-ai-testing-guide
1
0
Fork 0
mirror of https://github.com/OWASP/www-project-ai-testing-guide.git synced 2026-05-31 19:41:40 +02:00
Code Issues Packages Projects Releases Wiki Activity

Update AITG-APP-05_Testing_for_Unsafe_Outputs.md

Browse Source
This commit is contained in:
Matteo Meucci
2025-11-23 13:34:17 +01:00
committed by GitHub
parent 73dece2825
commit bf552d20f1
1 changed files with 3 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Document/content/tests/AITG-APP-05_Testing_for_Unsafe_Outputs.md
+3 -6
View File
@@ -100,12 +100,9 @@ A vulnerability is confirmed if the AI model:
- Employ ongoing monitoring and manual review processes to detect and rectify unsafe outputs quickly.
### Suggested Tools
- **Garak – Unsafe Output Detection Module**: Specialized Garak module explicitly designed to identify unsafe model outputs.
- **URL**: [Garak AV Spam Scanning](https://github.com/NVIDIA/garak/blob/main/garak/probes/av_spam_scanning.py)
- **Llama Guard 4**: Open source moderation model to detect unsafe text and unsafe combination of text and images.
- **URL**: [Llama Guard 4](https://www.llama.com/docs/model-cards-and-prompt-formats/llama-guard-4/)
- **LlavaGuard**, **ShieldGemma2**: Open source moderation model to detection unsafe images.
- **URL**: [ShieldGemma2](https://deepmind.google/models/gemma/shieldgemma-2/)
- **Garak – Unsafe Output Detection Module**: Specialized Garak module explicitly designed to identify unsafe model outputs - [Garak AV Spam Scanning](https://github.com/NVIDIA/garak/blob/main/garak/probes/av_spam_scanning.py)
- **Llama Guard 4**: Open source moderation model to detect unsafe text and unsafe combination of text and images - [Llama Guard 4](https://www.llama.com/docs/model-cards-and-prompt-formats/llama-guard-4/)
- **LlavaGuard**, **ShieldGemma2**: Open source moderation model to detection unsafe images- [ShieldGemma2](https://deepmind.google/models/gemma/shieldgemma-2/)
### References