257 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Adam Wilson	865a4b923a	job dependencies	2025-08-04 08:02:59 -06:00
Adam Wilson	3b5c7b9f69	CI/CD test	2025-08-04 07:59:02 -06:00
Adam Wilson	5252342701	add violation rate column to results table, based on 100 - (< thresh %)	2025-08-02 10:18:13 -06:00
Adam Wilson	b65e531049	results table	2025-08-01 20:00:14 -06:00
Adam Wilson	6d5d53bad5	summary of test results (tests 0-4)	2025-08-01 19:38:19 -06:00
Adam Wilson	bef6ba9ef5	JSON results: test_00_benign_prompts_no_mitigation	2025-07-30 12:19:53 -06:00
Adam Wilson	0171af7c94	fix confusing log message	2025-07-30 11:16:24 -06:00
Adam Wilson	24c6eaae28	JSON schema script	2025-07-30 11:13:20 -06:00
Adam Wilson	378aea7a66	100 math prompts, not 150	2025-07-30 11:13:09 -06:00
Adam Wilson	1624019272	JSON results: test_01_malicious_prompts_no_mitigation	2025-07-30 11:08:46 -06:00
Adam Wilson	28518e6c17	fixes for basic prompt template	2025-07-30 07:17:48 -06:00
Adam Wilson	0aaa4036df	JSON results: test_02_malicious_prompts_cot	2025-07-30 05:53:33 -06:00
Adam Wilson	c7deca79da	JSON results: test_03_malicious_prompts_rag	2025-07-29 22:13:56 -06:00
Adam Wilson	4f26df0755	JSON results: test_04_malicious_prompts_rag_and_cot	2025-07-29 15:27:39 -06:00
Adam Wilson	cf4e2d48fa	JSON results: test_05_malicious_prompts_cot_with_reflexion	2025-07-29 11:54:00 -06:00
Adam Wilson	bf8396bb03	JSON results: test_06_malicious_prompts_rag_with_reflexion	2025-07-29 06:09:29 -06:00
Adam Wilson	df6d5e644b	JSON results: test_07_malicious_prompts_rag_and_cot_reflexion	2025-07-28 20:51:31 -06:00
Adam Wilson	780a4e529f	archive tests	2025-07-28 15:27:04 -06:00
Adam Wilson	51e966e1ec	escape braces	2025-07-28 14:55:32 -06:00
Adam Wilson	047c161006	archive test results	2025-07-28 13:43:28 -06:00
Adam Wilson	058b281f25	default risk threshold; scores lower down in JSON serialization	2025-07-28 13:40:54 -06:00
Adam Wilson	d39d7f1f1f	don't send back alternate after guidelines	2025-07-28 13:29:21 -06:00
Adam Wilson	393b76faf4	small fix	2025-07-28 13:19:50 -06:00
Adam Wilson	bc119ed50e	don't overprocess results	2025-07-28 13:10:03 -06:00
Adam Wilson	24e28c0182	fix JSON serialization for logging test result	2025-07-28 12:06:56 -06:00
Adam Wilson	df14a01fe9	log full completion result with semantic similarity comparison results	2025-07-28 11:49:07 -06:00
Adam Wilson	b971df0a7a	risk threshold is too low	2025-07-28 10:47:57 -06:00
Adam Wilson	2659e6e43c	more updates for reflexion	2025-07-28 10:31:55 -06:00
Adam Wilson	5bc9f480f9	all domain unit tests pass	2025-07-27 18:53:30 -06:00
Adam Wilson	b83553d767	make mitigation tests consistent	2025-07-27 17:44:40 -06:00
Adam Wilson	dcff18a058	logging	2025-07-27 17:19:07 -06:00
Adam Wilson	a621ad82a9	Reflexion guardrails updates	2025-07-27 16:39:06 -06:00
Adam Wilson	99ec0ddf98	WIP reflexion service	2025-07-27 13:59:01 -06:00
Adam Wilson	eddacd87fa	LLM config output	2025-07-27 11:21:12 -06:00
Adam Wilson	a7a6873e73	update prompt templates; support LLM config logging	2025-07-26 22:10:04 -06:00
Adam Wilson	5b27d4c2e3	refactor for examples	2025-07-26 16:31:49 -06:00
Adam Wilson	27dad236ef	avoid unclosed curly braces	2025-07-26 15:50:10 -06:00
Adam Wilson	16ba9c15ee	test output for test_02_malicious_prompts	2025-07-26 08:22:35 -06:00
Adam Wilson	741629908c	updates for RAG + CoT tests	2025-07-25 18:11:49 -06:00
Adam Wilson	72785c6420	updates for RAG + CoT	2025-07-25 17:24:01 -06:00
Adam Wilson	a770a5211c	create/update all Phi-3 templates	2025-07-25 16:35:19 -06:00
Adam Wilson	23d58675f4	token constants	2025-07-25 09:47:12 -06:00
Adam Wilson	d15e9d6794	more test and template setup	2025-07-25 09:45:03 -06:00
Adam Wilson	3a62ecfae8	add test 0 results	2025-07-25 08:47:56 -06:00
Adam Wilson	85d33feffe	comment	2025-07-24 20:46:28 -06:00
Adam Wilson	4f2e539996	updates	2025-07-24 18:51:18 -06:00
Adam Wilson	ae279a512d	log LLM config	2025-07-23 20:21:42 -06:00
Adam Wilson	cb92890bb9	break tests into separate files; test 0 results	2025-07-23 19:06:27 -06:00
Adam Wilson	1b5b808ff6	use new garak true positives in tests	2025-07-23 15:59:56 -06:00
Adam Wilson	1096ee38a8	compiled failure req/resp pairs from NVIDIA garak runs in single JSON file	2025-07-23 14:49:13 -06:00