Train now dedups by sha256 (earliest date wins), splits by hash group,
and drops post-cutoff rows whose hash was seen in training, so neither
holdout reports memorized duplicates. The feature schema is explicit
(v1 = the original 97 structural features, v2 adds CFA peaks, DCT AC
histograms and JPEG quant/Huffman/scan stats), stored in the bundle and
read back at scoring time; legacy bundles default to v1. Vectors are
fixed-width with NaN padding, so a sparse record no longer shifts every
column. Scoring runs in batches instead of one predict_proba per record.
Co-Authored-By: Claude <noreply@anthropic.com>
scripts/ai_score.py trains a gradient-boosted classifier on labels
derived from the scan_dataset metadata (C2PA AI generators, TC260,
local pipelines vs camera/screenshot/editor output) and scores every
file from pixel and container statistics alone, so metadata-stripped
files still get a score. Temporal holdout on the production corpus:
AUC 0.966 / AP 0.988. CLIP ViT-L/14 was evaluated as an alternative
and lost on both accuracy (0.82) and cost, so the model uses the
structural features the scanner already collects; scoring is CPU-only
and I/O-bound.
Victor Kuznetsov