52 Commits

Author	SHA1	Message	Date
Adam Wilson	b4b2d792fc	more progress on fluent service call	2025-07-09 21:56:44 -06:00
Adam Wilson	af75e9aabf	support prompt template loading	2025-07-07 21:38:42 -06:00
Adam Wilson	911b629217	notes on violation vs. false refusal rates, effectiveness metrics	2025-07-06 18:34:12 -06:00
Adam Wilson	7d49d160b9	more test cases and notes	2025-07-06 17:47:49 -06:00
Adam Wilson	ffa2d73ae0	guardrail analyzed response, etc.	2025-07-06 15:15:59 -06:00
Adam Wilson	a1d3a8c1b7	adjust assertions for test 3	2025-07-05 20:21:35 -06:00
Adam Wilson	640c261b26	naming updates; fix static analysis script	2025-07-05 13:01:28 -06:00
Adam Wilson	a9db321597	naming updates; fix static analysis script	2025-07-05 12:58:25 -06:00
Adam Wilson	a25e8f9545	updates	2025-07-02 11:34:47 -06:00
Adam Wilson	f0fcc64258	support sampling and averages during testing	2025-06-28 13:15:32 -06:00
Adam Wilson	cb1be6746f	support testing malicious prompts with no guidelines	2025-06-28 12:18:35 -06:00
Adam Wilson	036d36bf4f	compare math prompt completions to DAN response	2025-06-25 21:47:22 -06:00
Adam Wilson	eed481ee77	document intended test cases/methodology; math prompts	2025-06-25 15:41:11 -06:00
Adam Wilson	5022c8f45c	document intended test cases/methodology	2025-06-25 15:23:56 -06:00
Adam Wilson	a530e78399	refactoring	2025-06-25 14:54:12 -06:00
Adam Wilson	9b8b6b7105	add/update services, constants	2025-06-25 12:53:24 -06:00
Adam Wilson	9057b0e977	refactor to use services instead of language model objects directly	2025-06-25 06:28:05 -06:00
Adam Wilson	c2abf2e81f	more tests	2025-06-24 14:24:10 -06:00
Adam Wilson	fc4978f43c	add RAG-based LM in conftest	2025-06-24 14:23:53 -06:00
Adam Wilson	92e00b9eb2	integration tests	2025-06-24 10:57:44 -06:00
Adam Wilson	0b6b7b79b9	service layer tests	2025-06-12 20:22:39 -06:00
Adam Wilson	bc1093988b	test config placeholders	2025-05-30 12:46:11 -06:00
Adam Wilson	8140fa654b	init tests	2025-05-30 12:26:55 -06:00
Adam Wilson	70d7e62ca0	back to promptinject.HijackHateHumansMini instead of DAN probe	2025-05-29 17:45:08 -06:00
Adam Wilson	4ee1b366ac	Update garak.rest.llm-rag.json	2025-05-29 15:53:45 -06:00
Adam Wilson	3647e188e9	use DAN against RAG	2025-05-29 15:39:48 -06:00
Adam Wilson	5bf67d7432	fix garak config paths	2025-05-28 05:25:16 -06:00
Adam Wilson	8bb4a473ca	restructure as domain-driven design architecture	2025-05-23 13:48:29 -06:00
Adam Wilson	8679f9ad8d	Flask API	2025-05-18 14:48:59 -06:00
Adam Wilson	e5c47ac8fb	python server; add logging	2025-05-18 14:04:30 -06:00
Adam Wilson	6efe96be90	Add RAG and non-RAG Action workflows	2025-05-17 16:29:35 -06:00
Adam Wilson	f1ae97643c	simple DAN prompt added to RAG	2025-05-15 19:55:41 -06:00
Adam Wilson	d42f62be84	try fixing model path	2025-05-15 12:05:12 -06:00
Adam Wilson	52819e34e0	fix deprecated refs	2025-05-15 10:10:41 -06:00
Adam Wilson	bf145e70b6	LangChain WIP; demo of RAG integration from Claude Sonnet 3.7	2025-05-15 09:30:25 -06:00
Adam Wilson	bddaa92c92	LangChain WIP; TypeError: _BaseAutoModelClass.from_pretrained() missing 1 required positional argument: 'pretrained_model_name_or_path'	2025-05-14 19:33:47 -06:00
Adam Wilson	92d5200edd	LangChain WIP; missing arg	2025-05-14 19:02:25 -06:00
Adam Wilson	1255d97559	LangChain WIP	2025-05-14 16:50:09 -06:00
Adam Wilson	ca823b70e6	reporting test	2025-05-03 17:51:17 -06:00
Adam Wilson	93fd85f099	test.Test Garak probe	2025-05-03 17:14:47 -06:00
Adam Wilson	2cb9782a4e	timeout	2025-05-03 13:29:57 -06:00
Adam Wilson	84f75af123	add initial Garak tests	2025-05-03 12:00:18 -06:00
Adam Wilson	37c84617b1	error handling - take 2	2025-04-24 11:44:46 -06:00
Adam Wilson	3563053ac3	error handling	2025-04-24 11:39:04 -06:00
Adam Wilson	c9c5b8b0a7	use Python package approach to project structure	2025-04-24 11:27:03 -06:00
Adam Wilson	8c6f38db31	support JSON request payload passed to LLM service layer	2025-04-23 16:09:05 -06:00
Adam Wilson	fc97bc6977	echo response back	2025-04-23 12:28:29 -06:00
Adam Wilson	5d44d90047	fix prompt ref	2025-04-23 06:56:24 -06:00
Adam Wilson	1ba607cd30	fix model ref	2025-04-23 06:32:28 -06:00
Adam Wilson	e04235606d	refactor example Python class to accept a single prompt and exit	2025-04-22 20:59:02 -06:00