mirror of
https://github.com/OWASP/www-project-ai-testing-guide.git
synced 2026-05-31 19:41:40 +02:00
+ Planning instructions to avoid issues with token consumption
This commit is contained in:
@@ -9,6 +9,14 @@ AI-based systems and applications often involve additional costs, whether the mo
|
||||
|
||||
---
|
||||
|
||||
### Test Planning
|
||||
|
||||
Testing applications based on LLMs can involve significant costs. Nowadays, such applications typically use models offered by third-party cloud providers, with variable pricing depending on the model and the number of tokens processed in input and generated in output (even local LLM models can involve significant costs, mainly due to high power consumption). These costs can become substantial, especially in multi-agent systems where, in addition to the user-provided input and the final output produced by the application, there are additional input and output tokens handled and generated by the internal agents that are transparent to the end user (and potentially to the tester as well). Furthermore, to prevent excessive usage, thresholds are often in place that, if reached, may trigger service shutdowns, affecting both the test and any real users of the application being analyzed. These costs and limits should be considered during the early stages of the project, in order to define what is expected from the test both in terms of objectives and costs and, if necessary, to determine which parts of the organization will be charged for those costs.
|
||||
|
||||
Token limitations are also very important when defining how the testing will be conducted. Many of the automated testing tools currently in use generate a large number of requests and can therefore incur significant costs, which may not be justified by the results obtained, potentially making a more manual approach to this type of analysis preferable.
|
||||
|
||||
---
|
||||
|
||||
### Test Objectives
|
||||
|
||||
- Identify vulnerabilities within the AI infrastructure that could lead to resource exhaustion.
|
||||
|
||||
Reference in New Issue
Block a user