From ef3a09c32c71a20e9b94b773e93ce084b69f0490 Mon Sep 17 00:00:00 2001 From: Didier Durand Date: Fri, 19 Dec 2025 13:10:21 +0100 Subject: [PATCH] [Doc] Fixing typos in diverse files --- Document/content/3.2_AI_Model_Testing.md | 2 +- .../content/tests/AITG-APP-01_Testing_for_Prompt_Injection.md | 4 ++-- .../content/tests/AITG-APP-05_Testing_for_Unsafe_Outputs.md | 2 +- .../content/tests/AITG-APP-11_Testing_for_Hallucinations.md | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Document/content/3.2_AI_Model_Testing.md b/Document/content/3.2_AI_Model_Testing.md index 3b48634..8c47bea 100644 --- a/Document/content/3.2_AI_Model_Testing.md +++ b/Document/content/3.2_AI_Model_Testing.md @@ -1,7 +1,7 @@ # 3.2 AI Model Testing -The **AI Model Testing** category addresses vulnerabilities and robustness of the AI model itself, independently from its deployment context. This category specifically targets intrinsic properties and behaviors of AI models, ensuring they perform reliably under adversarial conditions, do not leak sensitive information, and remain aligned with their intended goals. +The **AI Model Testing** category addresses vulnerabilities and robustness of the AI model itself, independently of deployment context. This category specifically targets intrinsic properties and behaviors of AI models, ensuring they perform reliably under adversarial conditions, do not leak sensitive information, and remain aligned with their intended goals. Testing at the model level helps detect fundamental weaknesses such as susceptibility to evasion attacks, data poisoning, privacy leaks, and misalignment issues, which could otherwise propagate to all deployments of that model. Comprehensive model testing is essential to maintaining the integrity, security, and trustworthiness of AI systems. diff --git a/Document/content/tests/AITG-APP-01_Testing_for_Prompt_Injection.md b/Document/content/tests/AITG-APP-01_Testing_for_Prompt_Injection.md index b37c3fb..f8b9e70 100644 --- a/Document/content/tests/AITG-APP-01_Testing_for_Prompt_Injection.md +++ b/Document/content/tests/AITG-APP-01_Testing_for_Prompt_Injection.md @@ -30,8 +30,8 @@ A list of currently employed **prompt injection payloads** will be provided, whi To carry out an effective analysis, it is important to take into account certain technical details regarding the implementation of the applications under assessment and the potential protective measures that may be deployed. In particular: - **Temperature**: the temperature is a parameter that controls the **randomness of the model’s output**. A lower temperature (e.g., close to 0) makes the model’s predictions more deterministic and focused on the most likely next tokens, resulting in more predictable and repeatable responses. A higher temperature increases randomness and "creativity" by sampling from a wider range of possible tokens, producing more diverse and less deterministic outputs. -- **Model fine-tuning**: model fine-tuning is the process of further training a pre-trained Large Language Model (LLM) on a specific dataset to adapt its behavior to a particular task or domain. Fine-tuning involves **updating some or all of the model’s weights** and can be executed also to enforce security or safety contraints. -- **Prompt tuning**: prompt tuning is a technique used to adapt a pre-trained Large Language Model (LLM) to specific tasks or domains by optimizing a prompt that guides the model’s output, without modifying the underlying model weights. Instead of fine-tuning all model parameters, prompt tuning learns only the prompt embeddings, which are **prepended to the input during inference** to steer the model’s behavior toward the desired task. Prompt tuning is often used to enforce security or safety contraints. +- **Model fine-tuning**: model fine-tuning is the process of further training a pre-trained Large Language Model (LLM) on a specific dataset to adapt its behavior to a particular task or domain. Fine-tuning involves **updating some or all of the model’s weights** and can be executed also to enforce security or safety constraints. +- **Prompt tuning**: prompt tuning is a technique used to adapt a pre-trained Large Language Model (LLM) to specific tasks or domains by optimizing a prompt that guides the model’s output, without modifying the underlying model weights. Instead of fine-tuning all model parameters, prompt tuning learns only the prompt embeddings, which are **prepended to the input during inference** to steer the model’s behavior toward the desired task. Prompt tuning is often used to enforce security or safety constraints. - **Guardrails**: in the context of Large Language Models, a guardrail refers to mechanisms implemented to **guide and restrict the model’s output**, usually to ensure it behaves safely and within desired operational boundaries. Guardrails usually try to prevent the generation of harmful, biased, or undesired content. - **Multi-agent architecture**: a multi-agent architecture refers to a system design where **multiple specialized LLM agents or models collaborate** to perform complex tasks. This architecture typically involves the presence of multiple agents with different prompts, and the user usually only has visibility of the final output, not the intermediate inputs and outputs, which significantly complicates a security analysis. - **Available tools**: in the context of Large Language Models, tools refer to **external software components, APIs, or modules** that the model can interact with or invoke to extend its capabilities beyond text generation. These tools can include databases, calculators and web browsers, but also **scripts and system shells**. diff --git a/Document/content/tests/AITG-APP-05_Testing_for_Unsafe_Outputs.md b/Document/content/tests/AITG-APP-05_Testing_for_Unsafe_Outputs.md index 77a22f9..ab4326e 100644 --- a/Document/content/tests/AITG-APP-05_Testing_for_Unsafe_Outputs.md +++ b/Document/content/tests/AITG-APP-05_Testing_for_Unsafe_Outputs.md @@ -10,7 +10,7 @@ Unsafe outputs in large language models (LLMs) refer to two major categories: These outputs can negatively impact users, introduce ethical or legal issues, and expose applications to critical security risks. This test aligns with OWASP Top 10 LLM05:2025 - Improper Output Handling and emerging AI safety standards. ### Test Objectives -Technically verify if an LLM or AI application generates unsafe or inappropriate outputs during normal usage. This test uniquely addresses scenarios where model responses may contain harmful content, aligning with OWASP Top 10 LLM05:2025. The definition of what is unsafe is strongly dependent on the application and the type of users that could interact with it and that is exposed to the generated content. A prior study of the type of content that the system should generate is necessary. For example, financial advice provided by a chatbot could be considered as safe for a chatbot supporting the user in making investments, but for a chatbot for the customer care of a shipping company, they could considered as unsafe. For general-purpose applications, several content-level safety taxonomies are already available and their extension/modification possible. For a comprehensive example of a general-purpose content-level safety taxonomy, refer to the [AILuminate Assessment Standard v1.1 - May 2025](https://drive.google.com/file/d/1xAsX9q3QjiatcJ_2467JM9Ris0wcYKa-/view). +Technically verify if an LLM or AI application generates unsafe or inappropriate outputs during normal usage. This test uniquely addresses scenarios where model responses may contain harmful content, aligning with OWASP Top 10 LLM05:2025. The definition of what is unsafe is strongly dependent on the application and the type of users that could interact with it and that is exposed to the generated content. A prior study of the type of content that the system should generate is necessary. For example, financial advice provided by a chatbot could be considered as safe for a chatbot supporting the user in making investments, but for a chatbot for the customer care of a shipping company, they could be considered as unsafe. For general-purpose applications, several content-level safety taxonomies are already available and their extension/modification possible. For a comprehensive example of a general-purpose content-level safety taxonomy, refer to the [AILuminate Assessment Standard v1.1 - May 2025](https://drive.google.com/file/d/1xAsX9q3QjiatcJ_2467JM9Ris0wcYKa-/view). For comprehensive coverage of application vulnerabilities resulting from unsafe model output, testers are encouraged to reference the original [OWASP Top Ten](https://owasp.org/www-project-top-ten/), the [OWASP Top Ten API](https://owasp.org/API-Security/), the [OWASP Web Security Testing Guide (WSTG)](https://owasp.org/www-project-web-security-testing-guide/), and similar documentation. It can help assess whether unsafe outputs from an LLM could lead to issues like XSS, SSRF, HTML injection, and other misuse in downstream applications. diff --git a/Document/content/tests/AITG-APP-11_Testing_for_Hallucinations.md b/Document/content/tests/AITG-APP-11_Testing_for_Hallucinations.md index 60b7f5c..5b2ba00 100644 --- a/Document/content/tests/AITG-APP-11_Testing_for_Hallucinations.md +++ b/Document/content/tests/AITG-APP-11_Testing_for_Hallucinations.md @@ -55,7 +55,7 @@ AI-generated outputs must: ### Remediation - Integrate retrieval-augmented generation (RAG) techniques for fact-checking and grounding outputs. - Enhance training with comprehensive and factual datasets. -- Analysis of the reliability of the sources used to obtained the training data (particularly important when webscraped). +- Analysis of the reliability of the sources used to obtain the training data (particularly important when webscraped). - Implement post-generation verification tools to cross-check generated facts. - Regularly evaluate and retrain models based on identified hallucination cases.