Commit Graph

41 Commits

Author SHA1 Message Date
DotDotSlash 22eaecdd59 Update AITG-APP-03_Testing_for_Sensitive_Data_Leak.md
Added additional prompts on testing for implementation details leak
2025-08-05 15:56:08 +05:30
Federico Ricciuti befe2755c7 Introduced Debunking tests and a differentiation between "Factuality and Misinformation" and "Debunking" hallucinations. As described by Giskard in the Phrase benchmark. 2025-08-03 14:34:38 +02:00
fedric95 d27026fda7 Merge branch 'OWASP:main' into main 2025-07-25 20:30:56 +02:00
Federico Ricciuti 0dd87354da 1. Specified that temperature=0 does not imply reproducibility (https://arxiv.org/pdf/2506.09501)
2. Pointed out that LLMs are generally less secure in low-resource languages
3. Made some order on the payloads for the bias test, now it using always the same base example.
2025-07-25 20:26:32 +02:00
federicodotta 897c532bba + Planning instructions to avoid issues with token consumption 2025-07-25 12:18:11 +02:00
Federico Ricciuti 9da16a16c1 Correction of the readme to refer to the correct changed test 2025-07-17 15:22:07 +02:00
Federico Ricciuti 977235af4d Introduction of the AITG-APP-10_Testing_for_Content_Bias.md, with tests to detect biased decisions made by the AI System. 2025-07-17 15:16:22 +02:00
Federico Ricciuti 49ee4b9d6c The unsafe output test now includes hate releated unsafe content as part of the tests.
AITG-APP-10_Testing_for_Harmful_Content_Bias.md replaced with AITG-APP-10_Testing_for_Content_Bias.md, and now it focuses on the detection of biases contened in the generated outputs.
2025-07-17 15:14:33 +02:00
Matteo Meucci 71b4f26900 Merge pull request #20 from fedric95/main 2025-07-12 21:30:58 +04:00
Federico Ricciuti 198167aebe - Introduced the necessity of defining a safety taxonomy before conducting the tests: the definition of what is safe and what is unsafe depends on the application.
- Linked an existing safety taxonomy
- Added examples of moderation models
- Removed most of the references to the concept of bias. They should be addressed in another test.

TO-DO

- Include tests that consider the potential multimodal nature of the application (right now it is more text-only)
- Make a specific test to evaluate the biases of the AI application under test and remove all the references to biases in this test
2025-07-12 19:12:00 +02:00
federicodotta 5dbedf3dc3 Prompt Injection Techniques section addeded 2025-07-12 13:51:10 +02:00
federicodotta 5a434e776b Update in typo tricks 2025-07-12 12:35:05 +02:00
federicodotta a56ba3f4e6 + Echo Chamber Attack 2025-07-12 12:24:58 +02:00
federicodotta b483d240cf + AntiGPT reference 2025-07-12 11:53:03 +02:00
federicodotta abfcbde568 + AntiGPT Prompt Injection 2025-07-12 11:49:27 +02:00
Didier Durand e754867dd5 fixing typos in multiple texts. 2025-06-29 13:48:42 +02:00
federicodotta a50c0ceb65 Fix copy/paste error in AITG-INF-02 2025-06-26 18:36:36 +02:00
federicodotta d390fa1605 Add some attack vectors and some additional notes to AITG-INF-02 2025-06-26 18:29:32 +02:00
federicodotta 77ebe9e3b5 Add prompt disclosure techniques 2025-06-26 16:42:50 +02:00
federicodotta 15218fd5cb Add direct tool invocation sections 2025-06-26 16:42:37 +02:00
federicodotta 29c2bcfeca Add XSS attack vectors 2025-06-26 16:42:21 +02:00
federicodotta 249ab5fb84 Add some data leaks related to tool list and reasoning 2025-06-26 16:42:11 +02:00
federicodotta 25fc713549 Added some prompt injection techniques 2025-06-26 16:41:44 +02:00
Ram K 0a93f78ebd Add EchoLeak technique to indirect prompt injection testing
- Added Reference-Style Markdown Injection (EchoLeak Technique) section
- Included real-world example with CVE-2025-32711 from Aim Security Labs
- Enhanced testing methodology for markdown-based data exfiltration attacks
2025-06-25 23:39:39 -05:00
Jeremy Redmond 392f1a10e8 docs: fix README spacing, prompt-injection typos, and meta tag property 2025-06-25 16:22:14 -04:00
Matteo Meucci 743966aa74 Merge pull request #3 from SitaRamSai/fix/spelling-corrections-test-files
fix: correct spelling errors in test documentation files
2025-06-25 19:11:01 +02:00
Ram K a1f115010a fix: correct spelling errors in test documentation files
- Fix 'system prompots' to 'system prompts' in AITG-APP-01
- Fix 'confidetial' to 'confidential' in AITG-APP-04
- Fix 'input.s.' to 'input.' in AITG-APP-04
- Fix 'esearch efforts' to 'Research efforts' in AITG-APP-07
- Fix 'How to test for Al' to 'How to test for AI' in AITG-APP-11
- Fix 'GaraK . PAckage Hallucionantion' to 'Garak - Package Hallucination' in AITG-INF-01

These corrections improve documentation quality and readability across the AI Testing Guide test specifications.
2025-06-25 11:59:29 -05:00
Julio Araujo 8e104d7419 Add more testing cases, replace links, mentiones to app security 2025-06-24 17:30:43 +02:00
Matteo Meucci 36649a80ca Delete Document/content/tests/a.md 2025-06-18 11:53:41 +02:00
Matteo Meucci fd9cca0afa Delete Document/content/tests/AI_MODEL_TESTING.md 2025-06-18 11:41:34 +02:00
Matteo Meucci c32e0ea25c Delete Document/content/tests/AI_INFRASTRUCTURE_TESTING.md 2025-06-18 11:41:16 +02:00
Matteo Meucci deb23f766a Delete Document/content/tests/AI_DATA_TESTING.md 2025-06-18 11:41:02 +02:00
Matteo Meucci 3c0117feba Delete Document/content/tests/AI_APPLICATION_TESTING.md 2025-06-18 11:40:44 +02:00
Matteo Meucci 37b2f28bae Update AI_MODEL_TESTING.md 2025-06-18 11:32:00 +02:00
Matteo Meucci 0ab1cc2bee Update AI_INFRASTRUCTURE_TESTING.md 2025-06-18 11:31:27 +02:00
Matteo Meucci b3b1a5508a Update AI_DATA_TESTING.md 2025-06-18 11:31:03 +02:00
Matteo Meucci 9af0da5502 Update AI_APPLICATION_TESTING.md 2025-06-18 11:30:25 +02:00
Matteo Meucci b527765f88 Update AI_APPLICATION_TESTING.md 2025-06-18 11:29:20 +02:00
Matteo Meucci b6fe9c75b9 Update AI_APPLICATION_TESTING.md 2025-06-18 11:28:44 +02:00
Matteo Meucci 6a6af6f23b Add files via upload 2025-06-17 21:30:54 +02:00
Matteo Meucci 4a9cf36bd5 Create a.md 2025-06-17 21:30:18 +02:00