CalvinBackup/llmsecops-research
1
0
Fork 0
mirror of https://github.com/lightbroker/llmsecops-research.git synced 2026-06-04 22:18:04 +02:00
Code Issues Packages Projects Releases Wiki Activity

add violation rate column to results table, based on 100 - (< thresh %)

Browse Source
This commit is contained in:
Adam Wilson
2025-08-02 10:18:13 -06:00
parent b65e531049
commit 5252342701
1 changed files with 22 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/logs/results.tests_0-4.table.txt
+22 -21
View File
@@ -1,20 +1,20 @@
TEST RESULTS DASHBOARD
=======================
┌──────────────────────────────────┬─────────┬───────────┬──────┬─────┬─────────┬─────────┬───────────┐
│ Test Name │ Benign │ Malicious │ CoT │ RAG │ Prompts │ Average │ < Thresh │
│ │ Prompts │ Prompts │ │ │ │ │ │
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┤
│ Benign Prompts No Mitigation │ ✓ │ │ │ │ 100 │ 0.2055 │ 100.0% │
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┤
│ Malicious Prompts No Mitigation │ │ ✓ │ │ │ 100 │ 0.8365 │ 29.0% │
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┤
│ Malicious Prompts CoT │ │ ✓ │ ✓ │ │ 100 │ 0.8264 │ 32.0% │
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┤
│ Malicious Prompts RAG │ │ ✓ │ │ ✓ │ 100 │ 0.8090 │ 39.0% │
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┤
│ Malicious Prompts RAG and CoT │ │ ✓ │ ✓ │ ✓ │ 100 │ 0.8033 │ 44.0% │
└──────────────────────────────────┴─────────┴───────────┴──────┴─────┴─────────┴─────────┴───────────┘
┌──────────────────────────────────┬─────────┬───────────┬──────┬─────┬─────────┬─────────┬───────────┬───────────┐
│ Test Name │ Benign │ Malicious │ CoT │ RAG │ Prompts │ Average │ < Thresh │ Violation |
│ │ Prompts │ Prompts │ │ │ │ │ │ Rate |
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┼───────────┤
│ Benign Prompts No Mitigation │ ✓ │ │ │ │ 100 │ 0.2055 │ 100.0% │ 0.0% |
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┼───────────┤
│ Malicious Prompts No Mitigation │ │ ✓ │ │ │ 100 │ 0.8365 │ 29.0% │ 71.0% │
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┼───────────┤
│ Malicious Prompts CoT │ │ ✓ │ ✓ │ │ 100 │ 0.8264 │ 32.0% │ 68.0% │
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┼───────────┤
│ Malicious Prompts RAG │ │ ✓ │ │ ✓ │ 100 │ 0.8090 │ 39.0% │ 61.0% │
├──────────────────────────────────┼─────────┼───────────┼──────┼─────┼─────────┼─────────┼───────────┼───────────┤
│ Malicious Prompts RAG and CoT │ │ ✓ │ ✓ │ ✓ │ 100 │ 0.8033 │ 44.0% │ 56.0% │
└──────────────────────────────────┴─────────┴───────────┴──────┴─────┴─────────┴─────────┴───────────┴───────────┘
SUMMARY STATISTICS
==================
@@ -27,10 +27,11 @@ Worst Performance: 29.0% (Malicious Prompts No Mitigation)
COLUMN LEGEND
=============
Benign Prompts: ✓ = Uses benign prompts
Malicious Prompts: ✓ = Uses malicious prompts
CoT: ✓ = Chain of Thought mitigation applied
RAG: ✓ = RAG few-shot examples applied
Prompts: Number of prompts tested (integer)
Average: Average score (floating point, 4 decimal places)
< Thresh: Percentage of results below threshold
Benign Prompts: ✓ = Uses benign prompts
Malicious Prompts: ✓ = Uses malicious prompts
CoT: ✓ = Chain of Thought mitigation applied
RAG: ✓ = RAG few-shot examples applied
Prompts: Number of prompts tested (integer)
Average: Average score (floating point, 4 decimal places)
< Thresh: Percentage of results below threshold
Violation Rate: Percentage of successful prompt injection exploitation attempts