feat(security): v2 ensemble tuning — label-first voting + SOLO_CONTENT_BLOCK

Cuts Haiku classifier false-positive rate from 44.1% → 22.9% on
BrowseSafe-Bench smoke. Detection trades from 67.3% → 56.2%; the
lost TPs are all cases Haiku correctly labeled verdict=warn
(phishing targeting users, not agent hijack) — they still surface
in the WARN banner meta but no longer kill the session.

Key changes:
- combineVerdict: label-first voting for transcript_classifier. Only
  meta.verdict==='block' block-votes; verdict==='warn' is a soft
  signal. Missing meta.verdict never block-votes (backward-compat).
- Hallucination guard: verdict='block' at confidence < LOG_ONLY (0.40)
  drops to warn-vote — prevents malformed low-conf blocks from going
  authoritative.
- New THRESHOLDS.SOLO_CONTENT_BLOCK = 0.92 decoupled from BLOCK (0.85).
  Label-less content classifiers (testsavant, deberta) need a higher
  solo-BLOCK bar because they can't distinguish injection from
  phishing-targeting-user. Transcript keeps label-gated solo path
  (verdict=block AND conf >= BLOCK).
- THRESHOLDS.WARN bumped 0.60 → 0.75 — borderline fires drop out of
  the 2-of-N ensemble pool.
- Haiku model pinned (claude-haiku-4-5-20251001). `claude -p` spawns
  from os.tmpdir() so project CLAUDE.md doesn't poison the classifier
  context (measured 44k cache_creation tokens per call before the fix,
  and Haiku refusing to classify because it read "security system"
  from CLAUDE.md and went meta).
- Haiku timeout 15s → 45s. Measured real latency is 17-33s end-to-end
  (Claude Code session startup + Haiku); v1's 15s caused 100% timeout
  when re-measured — v1's ensemble was effectively L4-only in prod.
- Haiku prompt rewritten: explicit block/warn/safe criteria, 8 few-shot
  exemplars (instruction-override → block; social engineering → warn;
  discussion-of-injection → safe).

Test updates:
- 5 existing combineVerdict tests adapted for label-first semantics
  (transcript signals now need meta.verdict to block-vote).
- 6 new tests: warn-soft-signal, three-way-block-with-warn-transcript,
  hallucination-guard-below-floor, above-floor-label-first,
  backward-compat-missing-meta.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

browse/test/security-adversarial-fixes.test.ts

@@ -71,7 +71,7 @@ describe('tool-output ensemble rule (single-layer BLOCK)', () => {
     const result = combineVerdict(
       [
         { layer: 'testsavant_content', confidence: 0.80 },
-        { layer: 'transcript_classifier', confidence: 0.75 },
+        { layer: 'transcript_classifier', confidence: 0.75, meta: { verdict: 'block' } },
       ],
       { toolOutput: true },
     );

browse/test/security-adversarial.test.ts

@@ -172,11 +172,11 @@ describe('canary — realistic outbound-channel attacks', () => {
 
 describe('combineVerdict — realistic attack/defense scenarios', () => {
   test('attack passes StackOne but Haiku catches it → BLOCK (ensemble save)', () => {
-    // Stack Overflow-style FP: StackOne 0.99 INJECTION, Haiku says WARN 0.7
-    // Both >= WARN → BLOCK
+    // Real attack: TestSavant 0.92 INJECTION, Haiku returns verdict=block.
+    // Both vote block → BLOCK.
     const r = combineVerdict([
       { layer: 'testsavant_content', confidence: 0.92 },
-      { layer: 'transcript_classifier', confidence: 0.75 },
+      { layer: 'transcript_classifier', confidence: 0.80, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
@@ -206,10 +206,12 @@ describe('combineVerdict — realistic attack/defense scenarios', () => {
   });
 
   test('both layers at threshold edge — WARN cutoff respects boundary', () => {
-    // Both exactly at WARN (0.6) — combiner treats >= WARN as firing, so BLOCK.
+    // testsavant at exactly WARN + transcript with verdict=block → BLOCK.
+    // Testsavant at WARN is a block-vote (>= WARN); transcript with
+    // verdict=block + conf >= LOG_ONLY is a block-vote.
     const r = combineVerdict([
       { layer: 'testsavant_content', confidence: THRESHOLDS.WARN },
-      { layer: 'transcript_classifier', confidence: THRESHOLDS.WARN },
+      { layer: 'transcript_classifier', confidence: THRESHOLDS.WARN, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
   });
@@ -264,3 +266,77 @@ describe('combineVerdict — realistic attack/defense scenarios', () => {
     expect(r.verdict).toBe('warn');
   });
 });
+
+// ─── Label-first voting (v1.5.1.0+) ──────────────────────────
+
+describe('combineVerdict — label-first voting for transcript_classifier', () => {
+  test('Haiku verdict=warn at high confidence is a soft signal only, not a block-vote', () => {
+    // Under v1.5.1.0 label-first: Haiku's 'warn' label means "suspicious but
+    // not hijack-level" regardless of its confidence. It should NOT single-
+    // handedly upgrade the ensemble to BLOCK even when pointed at 0.80.
+    const r = combineVerdict([
+      { layer: 'testsavant_content', confidence: 0.80 },
+      { layer: 'transcript_classifier', confidence: 0.80, meta: { verdict: 'warn' } },
+    ]);
+    // testsavant is a block-vote (1), transcript is a warn-vote only.
+    // Total block-votes = 1, below the 2-of-N rule → WARN, not BLOCK.
+    // testsavant at 0.80 is below the BLOCK threshold (0.85), so reason
+    // is single_layer_medium (WARN-tier), not single_layer_high.
+    expect(r.verdict).toBe('warn');
+    expect(r.reason).toBe('single_layer_medium');
+  });
+
+  test('Haiku verdict=block at moderate confidence still block-votes (ensemble save on real hijack)', () => {
+    const r = combineVerdict([
+      { layer: 'testsavant_content', confidence: 0.80 },
+      { layer: 'transcript_classifier', confidence: 0.80, meta: { verdict: 'block' } },
+    ]);
+    expect(r.verdict).toBe('block');
+    expect(r.reason).toBe('ensemble_agreement');
+  });
+
+  test('three-way: warn-transcript + two ML block-votes still BLOCKs (ensemble reaches 2)', () => {
+    // Even when Haiku says warn (not block), two other classifiers agreeing
+    // still reaches the 2-of-N threshold.
+    const r = combineVerdict([
+      { layer: 'testsavant_content', confidence: 0.80 },
+      { layer: 'deberta_content', confidence: 0.80 },
+      { layer: 'transcript_classifier', confidence: 0.80, meta: { verdict: 'warn' } },
+    ]);
+    expect(r.verdict).toBe('block');
+    expect(r.reason).toBe('ensemble_agreement');
+  });
+
+  test('hallucination guard: verdict=block at confidence 0.30 drops to warn-vote', () => {
+    // Below LOG_ONLY (0.40), a block label is suspected hallucination — drop
+    // it to warn-vote. testsavant alone remains a single block-vote → WARN,
+    // not BLOCK.
+    const r = combineVerdict([
+      { layer: 'testsavant_content', confidence: 0.80 },
+      { layer: 'transcript_classifier', confidence: 0.30, meta: { verdict: 'block' } },
+    ]);
+    expect(r.verdict).toBe('warn');
+  });
+
+  test('above hallucination floor: verdict=block at confidence 0.50 counts as block-vote', () => {
+    // Once confidence >= LOG_ONLY (0.40), the label is trusted. BLOCK.
+    const r = combineVerdict([
+      { layer: 'testsavant_content', confidence: 0.80 },
+      { layer: 'transcript_classifier', confidence: 0.50, meta: { verdict: 'block' } },
+    ]);
+    expect(r.verdict).toBe('block');
+    expect(r.reason).toBe('ensemble_agreement');
+  });
+
+  test('backward-compat: transcript signal with no meta.verdict never block-votes', () => {
+    // Pre-v1.5.1.0 signals (or adversarial tests) may arrive without
+    // meta.verdict. Under the new rule, missing meta is warn-vote-only
+    // when confidence >= WARN, never a block-vote. Even at 0.95 (high
+    // confidence), transcript alone doesn't upgrade the ensemble.
+    const r = combineVerdict([
+      { layer: 'testsavant_content', confidence: 0.80 },
+      { layer: 'transcript_classifier', confidence: 0.95 }, // no meta
+    ]);
+    expect(r.verdict).toBe('warn');
+  });
+});

browse/test/security-bench.test.ts

@@ -116,6 +116,9 @@ describe('BrowseSafe-Bench smoke (200 cases)', () => {
     //   * DeBERTa-v3 ensemble (TODOS "TestSavantAI ensemble" P2 item)
     //   * Calibration against real attack-log data from production
     let tp = 0, fp = 0, tn = 0, fn = 0;
+    // intentionally 0.6 — L4-only bench pinned to v1 WARN for historical
+    // continuity. The ensemble bench (security-bench-ensemble.test.ts) uses
+    // THRESHOLDS.WARN from security.ts (0.75 in v1.5.1.0+).
     const WARN = 0.6;
     for (const row of rows) {
       const signal = await scanPageContent(row.content);

browse/test/security-integration.test.ts

@@ -89,8 +89,10 @@ describe('defense-in-depth — layer coexistence', () => {
     // produce a BLOCK-worthy verdict.
 
     const baseSignals: LayerSignal[] = [
-      { layer: 'testsavant_content', confidence: 0.88 },
-      { layer: 'transcript_classifier', confidence: 0.75 },
+      // content at 0.95 clears the SOLO_CONTENT_BLOCK threshold (0.92) so
+      // that the "content alone" case below still hits single_layer_high.
+      { layer: 'testsavant_content', confidence: 0.95 },
+      { layer: 'transcript_classifier', confidence: 0.75, meta: { verdict: 'block' } },
       { layer: 'canary', confidence: 1.0 },
     ];
 
@@ -174,8 +176,8 @@ describe('defense-in-depth — regression guards', () => {
     // still be BLOCK, not crash or produce nonsense. Canary uses >= 1.0
     // which matches; ML layers also register.
     const overflow: LayerSignal[] = [
-      { layer: 'testsavant_content', confidence: 5.5 }, // above BLOCK
-      { layer: 'transcript_classifier', confidence: 3.2 }, // above BLOCK
+      { layer: 'testsavant_content', confidence: 5.5 }, // above BLOCK, block-vote
+      { layer: 'transcript_classifier', confidence: 3.2, meta: { verdict: 'block' } }, // label-first block-vote
     ];
     expect(combineVerdict(overflow).verdict).toBe('block');
   });

browse/test/security.test.ts

@@ -54,12 +54,12 @@ describe('combineVerdict — ensemble rule', () => {
 
   test('both ML layers at WARN → BLOCK (ensemble agreement)', () => {
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
-      { layer: 'transcript_classifier', confidence: 0.65 },
+      { layer: 'testsavant_content', confidence: 0.8 },
+      { layer: 'transcript_classifier', confidence: 0.78, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
-    expect(r.confidence).toBe(0.65); // min of the two
+    expect(r.confidence).toBe(0.78); // min of the two
   });
 
   test('single layer >= BLOCK (no cross-confirm) → WARN, NOT block', () => {
@@ -67,7 +67,7 @@ describe('combineVerdict — ensemble rule', () => {
     // shouldn't kill sessions without a second opinion.
     const r = combineVerdict([
       { layer: 'testsavant_content', confidence: 0.95 },
-      { layer: 'transcript_classifier', confidence: 0.1 },
+      { layer: 'transcript_classifier', confidence: 0.1, meta: { verdict: 'safe' } },
     ]);
     expect(r.verdict).toBe('warn');
     expect(r.reason).toBe('single_layer_high');
@@ -75,8 +75,8 @@ describe('combineVerdict — ensemble rule', () => {
 
   test('single layer >= WARN → WARN (other layer low)', () => {
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
-      { layer: 'transcript_classifier', confidence: 0.2 },
+      { layer: 'testsavant_content', confidence: 0.8 },
+      { layer: 'transcript_classifier', confidence: 0.2, meta: { verdict: 'safe' } },
     ]);
     expect(r.verdict).toBe('warn');
     expect(r.reason).toBe('single_layer_medium');
@@ -101,7 +101,7 @@ describe('combineVerdict — ensemble rule', () => {
     const r = combineVerdict([
       { layer: 'testsavant_content', confidence: 0.3 },
       { layer: 'testsavant_content', confidence: 0.8 },
-      { layer: 'transcript_classifier', confidence: 0.75 },
+      { layer: 'transcript_classifier', confidence: 0.75, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
@@ -110,20 +110,25 @@ describe('combineVerdict — ensemble rule', () => {
   // --- 3-way ensemble (DeBERTa opt-in) ---
 
   test('3-way: DeBERTa + testsavant at WARN → BLOCK (two ML classifiers agreeing)', () => {
+    // Two scalar-layer block-votes; transcript offers no vote.
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
-      { layer: 'deberta_content', confidence: 0.65 },
-      { layer: 'transcript_classifier', confidence: 0.1 },
+      { layer: 'testsavant_content', confidence: 0.8 },
+      { layer: 'deberta_content', confidence: 0.78 },
+      { layer: 'transcript_classifier', confidence: 0.1, meta: { verdict: 'safe' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
   });
 
   test('3-way: only deberta fires alone → WARN (no cross-confirm)', () => {
+    // deberta at 0.95 is >= SOLO_CONTENT_BLOCK (0.92) → single_layer_high
+    // path. For user-input mode (no toolOutput opt), it degrades to WARN
+    // (SO-FP mitigation). Confidence bumped from 0.9 to 0.95 to stay above
+    // the new SOLO_CONTENT_BLOCK threshold.
     const r = combineVerdict([
       { layer: 'testsavant_content', confidence: 0.1 },
-      { layer: 'deberta_content', confidence: 0.9 },
-      { layer: 'transcript_classifier', confidence: 0.1 },
+      { layer: 'deberta_content', confidence: 0.95 },
+      { layer: 'transcript_classifier', confidence: 0.1, meta: { verdict: 'safe' } },
     ]);
     expect(r.verdict).toBe('warn');
     expect(r.reason).toBe('single_layer_high');
@@ -131,15 +136,15 @@ describe('combineVerdict — ensemble rule', () => {
 
   test('3-way: all three ML layers at WARN → BLOCK with min confidence', () => {
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
-      { layer: 'deberta_content', confidence: 0.65 },
-      { layer: 'transcript_classifier', confidence: 0.8 },
+      { layer: 'testsavant_content', confidence: 0.8 },
+      { layer: 'deberta_content', confidence: 0.76 },
+      { layer: 'transcript_classifier', confidence: 0.82, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
-    // Confidence reports the MIN of the WARN+ signals (most conservative
-    // estimate of agreed-upon signal strength)
-    expect(r.confidence).toBe(0.65);
+    // Confidence reports the MIN of the contributing block-votes
+    // (most conservative estimate of agreed-upon signal strength).
+    expect(r.confidence).toBe(0.76);
   });
 
   test('DeBERTa disabled (confidence 0, meta.disabled) does not degrade verdict', () => {
@@ -148,9 +153,9 @@ describe('combineVerdict — ensemble rule', () => {
     // identically to a safe/absent signal — never let the zero drag
     // down what testsavant + transcript would have said.
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
+      { layer: 'testsavant_content', confidence: 0.8 },
       { layer: 'deberta_content', confidence: 0, meta: { disabled: true } },
-      { layer: 'transcript_classifier', confidence: 0.7 },
+      { layer: 'transcript_classifier', confidence: 0.8, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');