CalvinBackup/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-07-09 17:38:40 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat(security): always run Haiku on tool outputs (drop the L4 gate)

Browse Source 
Tool-result scan previously short-circuited when L4 (TestSavantAI)
scored below WARN, and further gated Haiku on any layer firing at >=
LOG_ONLY. On BrowseSafe-Bench that meant Haiku almost never ran,
because TestSavantAI has ~15% recall on browser-agent-specific
attacks (social engineering, indirect injection). We were gating our
best signal on our weakest.

Run all three classifiers (L4 + L4c + Haiku) in parallel. Cost:
~$0.002 + ~8s Haiku wall time per tool result, bounded by the 15s
Haiku timeout. Haiku also runs in parallel with the content scans
so it's additive only against the stream handler budget, not
against the session wall time.

User-input pre-spawn path unchanged — shouldRunTranscriptCheck still
gates there. The Stack Overflow FP mitigation that original gate was
built for still applies to direct user input; tool outputs have
different characteristics.

Source-contract test updated to pin the new parallel-three shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-04-20 21:15:57 +08:00
parent 5d968c43ec
commit b515f31400
2 changed files with 23 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
browse/test/security-source-contracts.test.ts
+8 -2
View File
@@ -116,8 +116,14 @@ describe('askClaude — pre-spawn + tool-result defense wiring', () => {
expect(AGENT_SRC).toContain('}, 2000);');
});
test('tool-result scan short-circuits when both content layers below WARN', () => {
expect(AGENT_SRC).toMatch(/maxContent < THRESHOLDS\.WARN/);
test('tool-result scan runs all three classifiers in parallel (no L4 gate)', () => {
// Regression guard for the Haiku-always change. Previously the scan
// short-circuited when L4/L4c both returned below WARN, which meant
// Haiku (our best signal per BrowseSafe-Bench) rarely ran. Now we run
// all three in parallel and let combineVerdict decide.
expect(AGENT_SRC).toMatch(/scanPageContent\(text\),[\s\S]*scanPageContentDeberta\(text\),[\s\S]*checkTranscript\(/);
// The old short-circuit must be gone.
expect(AGENT_SRC).not.toMatch(/if \(maxContent < THRESHOLDS\.WARN\) return;/);
});
test('onCanaryLeaked fires both security_event and agent_error for legacy clients', () => {