Record universal detector v3-v8: diverse training shifts the curve

Fresh MLP on frozen CLIP embeddings with 49 negative domains (12,258 images)
and 5,043 AI positives achieves 93.0% AI-test recall with 39/49 domains at
zero false positives. Digital art FP dropped from 99.3% (Model 1) to 24.8%,
fashion from 62-71% to 12%, UI from 99.3% to 12%. Photo FPR 0.7%. The
improvement comes from training data diversity on frozen embeddings, not
representation change (v5 fine-tune proved this by undoing the gains).
Also adds .local-eval to ruff exclude so the gate covers only tracked files.

pre-commit: 1) maintain.sh - exit 1, known lightning advisory; core green (ruff, format, pyright, 1731 tests); 2) /simplify - docs + research; 3) docs sync - all artifacts in data/research/; 4) CLAUDE.md - no changes
This commit is contained in:
Victor Kuznetsov
2026-08-29 18:00:18 -07:00
parent 0470849b77
commit 433b8a6900
2 changed files with 29 additions and 2 deletions
+27
View File
@@ -907,6 +907,33 @@ in a sub-band. Point intervention within this octave is therefore not viable:
a quiet kill inherently requires modifying the entire 16-32 px octave, which
costs 20-25 dB.
### Universal AI detector v3-v8: diverse training shifts the curve, 2026-08-27
A fundamentally different approach from all closed hypotheses: instead of
repairing Model 1 (trained on photo negatives only), train a fresh classifier
on the FULL diversity of human-made and AI-generated content. The v3 ridge on
frozen CLIP embeddings with 3,224 structured negatives achieved 88.0% recall
at 0.1% photo FPR with structured FP reduced from 33-99% to 0-14%. The v4 MLP
pushed recall to 91.3%. Crucially, v5 (fine-tuning the CLIP tower on the same
data) UNDID the structured gains, proving the improvement comes from training
data diversity on frozen embeddings, not from representation change.
Iterations v6-v8 expanded the corpus to 49 negative domains (12,258 images)
plus 5,043 AI positives across 19+ families. The v7 MLP achieved **93.3%
AI-test recall** (matching Model 1's 93.02%) with **36 of 49 negative domains
at zero false positives**. The v8 auto-pipeline (650 additional targeted
images for weak domains) improved to **39/49 domains at zero FP** with digital
art at 24.8% (from 99.3% in Model 1), fashion at 12.0% (from 62-71%), and UI
at 12.0% (from 99.3%). Photo FP remained at 0.7% (COCO) and 0.7% (picsum).
The trade-off curve has shifted permanently: Model 1 traded 93% recall for
33-99% structured FP; v8 delivers 93% recall with 0-25% structured FP and
39/49 domains at exactly zero. The remaining gap is concentrated in digital
art (24.8%), fashion (12%), and UI (12%) — all domains where AI-generated and
human-made content share deep visual similarity. Photo-side gates (fresh OI
≤50, Kodak 0/24) remain the open constraint.
Artifacts: `universal-v7-2026-08-27/`, `auto-pipeline-2026-08-27/`.
### Five-head provider cascade with TC260, 2026-08-27
Adding TC260 (721 Chinese-AIGC images from the Spaces catalog) as a fifth