Luca Beurer-Kellner
e18c6b5bdb
Add an option to add extra metadata that is pushed and passed to Guardrails during an MCP session ( #47 )
...
* use select() before readline
* support for setting static metadata for MCP sessions
* nest extra mcp metadata in metadata object
* unify session metadata
* extra metadata tests
* use empty object as parameters, if None
* list_tools as tool call
* offset indices in tests
* test: adjust addresses
* mcp: make error reporting configurable
* line logging
* log version
* verbose logging + loud exception failure
* add server and client name to policy get
* append trace even if not pushing
* port tools/list message support to SSE
* use python -m build
* adjust guardrail failure address
* support for blocking tools/list in SSE
* use error-based failure response format by default
* tools/list test
* don't list_tools in stdio connect
* flaky test: handle second possible result in anthropic streaming case
---------
Co-authored-by: knielsen404 <kristian@invariantlabs.ai >
2025-05-19 13:44:37 +02:00
Hemang
780e5f6ee7
Skip test_generate_content_with_image for gemini. Getting 500.
2025-05-16 14:33:00 +05:30
Hemang
876eb44c78
Make MCP stdio gateway fully async. With sync and async mixed behaviour for running background tasks we were running into issues.
2025-05-16 14:33:00 +05:30
Hemang Sarkar
a6c1124076
Update test_mcp.py
2025-05-15 15:00:31 +05:30
Hemang Sarkar
0ffde69391
Fix test_mcp_sse_with_gateway_hybrid_guardrails so that we don't rely on order of annotations.
2025-05-15 15:00:31 +05:30
Hemang Sarkar
fefc22eea0
Update test_mcp.py to include the transport parameter in the hybrid guardrails test.
2025-05-14 15:32:05 +02:00
Hemang
ed50670bef
Add MCP gateway tests with hybrid guardrails (both blocking and logging). Also refactor tests so that we can parameterize the transport type - stdio or sse.
2025-05-14 18:05:14 +05:30
Hemang
8eae198eb0
Add integration tests for MCP SSE via gateway with guardrails.
2025-05-14 14:21:48 +05:30
Hemang
e2e004b7b1
Move dockerfiles inside gateway/ and update main CLI script to be able to run build, up, down and logs on a local gateway server instance.
2025-05-14 14:21:48 +05:30
Hemang
dbab86e0ac
Fix broken tests.
2025-05-09 12:17:14 +05:30
Hemang
132eedab0a
Update tests_ci.yml so that the CI run fails even when unit tests fail.
2025-05-07 02:03:06 +05:30
Hemang
d057aa5c37
Add httpx as unit test dependencies.
2025-05-07 01:27:13 +05:30
Hemang
1f6e2ed7fb
Update streaming in anthropic route to handle chunks with incomplete events. Introduce an sse_buffer which keeps track of the current incomplete event from the last processed chunk.
2025-05-07 01:08:15 +05:30
Hemang
aec7808e3e
Fix broken integration tests.
2025-05-07 01:08:15 +05:30
Hemang
dc9ac9c3c6
Fix broken unit tests.
2025-05-07 01:08:15 +05:30
Hemang
88ba17a803
Move MCP testing stdio server and client into a separate folder.
2025-04-29 15:56:37 +05:30
Hemang
dbeb4bc660
Add blocking guardrails based test for MCP gateway.
2025-04-29 09:00:04 +02:00
Hemang
d877e5a1e6
Add logging guardrails based test for MCP gateway.
2025-04-29 09:00:04 +02:00
Hemang
d983b7431b
Add tests for MCP integration in gateway.
2025-04-29 09:00:04 +02:00
Hemang
5bf121bbda
Use pyproject.toml instead of requirements.txt and fix some broken tests.
2025-04-17 07:15:28 +02:00
Luca Beurer-Kellner
c4dd3f3b19
Allow to specify different API keys for the guardrailing service ( #36 )
...
* minor refactor for getting invariant api keys for guardrailing
* allow different guardrailing api key
* tests
* fix comment + import
* improved unauthorized handling
2025-04-03 12:15:30 +02:00
Luca Beurer-Kellner
e17b53b927
Extract guardrails from header if provided ( #33 )
...
* guardrails from header
* use in-file guardrails in client.py
* support case without request
* remove client script
* tests: guardrailing rule passed in header
* include checked guardrails in annotation extra metadata
* include guardrailing action
* update guardrail metadata
2025-04-03 08:56:15 +02:00
Marco Milanta
37005c03be
fix: add tests (and found bug)
2025-04-02 16:03:51 +02:00
Marco Milanta
7b6d77e0a5
feat: add litellm support
2025-04-02 15:22:39 +02:00
Hemang
f3a56e1e43
Add preguardrailing tests for guardrails pulled from explorer.
2025-04-02 13:40:52 +02:00
Hemang
55f0f741c0
Add tests for guardrails integration with explorer.
2025-04-02 13:40:52 +02:00
Hemang
eced3755b2
Refactor tests.
2025-04-02 13:40:52 +02:00
Hemang
750c83d3f8
Add calls to execute logging guardrails before pushing to explorer.
2025-04-02 13:40:52 +02:00
Hemang
050ec1ba58
Fetch guardrails from explorer. These have higher precedence than than the guardrails from file.
2025-04-02 13:40:52 +02:00
Luca Beurer-Kellner
7c0bb957fb
Pipelined Guardrails ( #32 )
...
* initial draft: pipelined guardrails
* documentation on stream instrumentation
* more comments
* fix: return earlier
* non-streaming case
* handle non-streaming case
* fix more cases
* simplify request instrumentation
* improve comments
* fix import issues
* extend tests for input guardrailing
* anthropic integration of pipelined and pre-guardrailing
* fix gemini streamed refusal
2025-03-31 14:13:58 +02:00
Hemang
4671c8b67e
Add tests for gemini guardrails integration.
2025-03-27 12:26:17 +01:00
Hemang
35cb889f1a
Update guardrails test after annotations ranges filtering.
2025-03-27 12:26:17 +01:00
Hemang
0a9cc244c8
Add tool_result type in the coverter test for Anthropic.
2025-03-21 00:56:52 +01:00
Hemang
ad49e688f0
Add tests for Anthropic guardrails.
2025-03-21 00:56:52 +01:00
Hemang
3df9e73249
Add some tests for the Anthropic conversion functions. Add support for system message in Anthropic. Rename some variables.
2025-03-21 00:56:52 +01:00
Hemang
22acb38131
Add tests for the guardrails integration for open_ai route.
2025-03-18 14:46:22 +01:00
Hemang
c4ecc01a59
Remove unnecessary playwright dependency for tests.
2025-03-13 10:12:55 +01:00
Hemang
6558f23604
Add unit tests and move the current integration tests into a separate directory.
2025-03-12 23:57:00 +01:00
Hemang
c9535e0a72
Fix test_chat_completion_with_tool_call_without_streaming.
2025-03-12 21:56:19 +01:00
Hemang
032d60e23e
Fix broken test test_chat_completion_with_image.
2025-03-12 21:41:28 +01:00
Hemang
300130e815
Fix broken tests.
2025-03-12 19:17:13 +01:00
Hemang
2fe0f55cb3
Don't block on push to explorer call for the routes.
2025-03-10 17:25:39 +01:00
Hemang
20f8a12032
Formatting changes.
2025-03-10 15:18:13 +01:00
Hemang
abbc80890d
Add tests with tool_calls for Gemini.
2025-03-10 15:18:13 +01:00
Hemang
fdc9da16c0
Add tests for some non tool_call cases for Gemini integration.
2025-03-10 15:18:13 +01:00
Hemang
7576f245bd
Add a converter module to house conversion from different LLM provider formats to Invariant API format.
2025-03-10 15:18:13 +01:00
Hemang Sarkar
adfabd5b12
Update the format where the user passes the invariant key in the llm provider key header. ( #18 )
2025-03-06 10:57:32 +01:00
Hemang
7008f73310
Rename proxy to gateway.
2025-03-05 14:35:39 +01:00
Zishan
79c934ef45
fix anthropic test
2025-02-27 10:30:33 +01:00
Hemang
1795fd4a23
Rename containers
2025-02-27 10:30:13 +01:00
