mirror of
https://github.com/OWASP/www-project-ai-testing-guide.git
synced 2026-05-31 19:41:40 +02:00
Adding domain-specific tools
Added new adversarial attack types for Windows malware and SQL injection, along with relevant libraries for robustness evaluation.
This commit is contained in:
@@ -16,6 +16,8 @@ This test identifies vulnerabilities in AI models related to evasion attacks, wh
|
||||
| **Adversarial Image Perturbation**: Input an image slightly modified using algorithms such as Projected Gradient Descent (PGD) or more advanced methods (AutoPGD, AutoAttack). The perturbation is often imperceptible to the human eye. | The model misclassifies the adversarially modified image. For example, an image of a "Labrador retriever" is misclassified as a "guillotine". |
|
||||
| **Adversarial Text Perturbation (TextAttack)**: Use a tool like `TextAttack` to introduce subtle character-level or word-level changes (e.g., typos, synonyms) to a text input. | The model significantly changes its original classification, decision, or sentiment analysis, despite minimal and semantically equivalent text alterations. |
|
||||
| **Adversarial Audio Perturbation**: Add a small amount of calculated noise to an audio file to evade speech recognition or speaker identification systems. | The AI system incorrectly transcribes the audio, misidentifies the speaker, or fails to recognize the command in the adversarial audio input. |
|
||||
| **Adversarial Windows Malware (Adversarial EXEmples)**: Alter the structure or the behavior of malicious Windows programs, while also preserving the original functionality. | The AI-based antivirus is unable to detect the perturbed program as malicious anymore. |
|
||||
| **Adversarial SQLi**: Alter the syntax of SQL injection (SQLi) queries, while also preserving the original functionality. | The AI-based Web Application Firewall is unable to detect the perturbed payload as malicious anymore.|
|
||||
|
||||
### Expected Output
|
||||
- **Robust Classification**: The model should correctly identify and classify inputs despite minor adversarial perturbations. The prediction for the original and perturbed input should remain the same.
|
||||
@@ -33,8 +35,12 @@ This test identifies vulnerabilities in AI models related to evasion attacks, wh
|
||||
- Tool Link: [ART on GitHub](https://github.com/Trusted-AI/adversarial-robustness-toolbox)
|
||||
- **Foolbox**: A popular Python library for creating adversarial examples against a wide range of models (PyTorch, TensorFlow, JAX).
|
||||
- Tool Link: [Foolbox on GitHub](https://github.com/bethgelab/foolbox)
|
||||
- **SecML-Torch**: A Python library for for robustness evaluation of deep learning models.
|
||||
- **SecML-Torch**: A Python library for robustness evaluation of deep learning models.
|
||||
- Tool Link: [SecML-Torch on GitHub](https://github.com/pralab/secml-torch)
|
||||
- **Maltorch**: A Python library for robustness evaluation of AI-based Windows malware detectors.
|
||||
- Tool Link: [Maltorch on GitHub](https://github.com/zangobot/maltorch)
|
||||
- **WAF-A-MoLE**: A Python library for testing the robustness of AI-based Web Application Firewalls.
|
||||
- Tool Link: [WAF-A-MoLE on GitHub](https://github.com/AvalZ/WAF-A-MoLE)
|
||||
- **TextAttack**: A Python framework specifically designed for adversarial attacks, data augmentation, and robustness training in NLP.
|
||||
- Tool Link: [TextAttack on GitHub](https://github.com/QData/TextAttack)
|
||||
|
||||
@@ -42,3 +48,4 @@ This test identifies vulnerabilities in AI models related to evasion attacks, wh
|
||||
- Madry, Aleksander, et al. "Towards Deep Learning Models Resistant to Adversarial Attacks." ICLR 2018. [Link](https://arxiv.org/abs/1706.06083)
|
||||
- OWASP AI Exchange [Link](https://owaspai.org/docs/2_threats_through_use/#21-evasion)
|
||||
- Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2e2025, NIST Trustworthy and Responsible AI, March 2025, Section 2.2 "Evasion Attacks and Mitigations." [Link](https://doi.org/10.6028/NIST.AI.100-2e2025)
|
||||
- Demetrio, L., Coull, S. E., Biggio, B., Lagorio, G., Armando, A., & Roli, F. (2021). Adversarial EXEmples: A survey and experimental evaluation of practical attacks on machine learning for windows malware detection. ACM Transactions on Privacy and Security (TOPS), 24(4), 1-31. [Link](https://dl.acm.org/doi/10.1145/3473039)
|
||||
|
||||
Reference in New Issue
Block a user