remove-ai-watermarks/docs/photo-classify.md

Photo pixel classification

classify gives a pixel opinion on a photograph: camera-like, generated, or unknown, and optionally which renderer it resembles. It is a separate command from identify. Metadata inspection never starts it.

This page is the library guide. The Hugging Face model card is the same freeze written for Hub publication: photo-classify-hf/README.md. Campaign notes and rejected variants live in AI-generated image classifiers.

When to use it

Use identify for C2PA, IPTC, visible marks, and other provenance. Use classify when those signals are gone and the file is still a photograph.

Do not use classify to prove a file is clean. Do not use it on UI or digital art. Receipt photographs are detected by the receipt-document gate and abstain to unknown instead of a false ai; they are still not a supported verdict. Do not treat provider=openai as a decoded SynthID payload.

Install and run

uv tool install --force "remove-ai-watermarks[classify]"
remove-ai-watermarks classify image.png
remove-ai-watermarks classify image.png --json

from pathlib import Path
from remove_ai_watermarks.classify import classify_pixels

result = classify_pixels(Path("image.png"))
print(result.label, result.detector, result.provider)

Missing extra raises RuntimeError with pip install 'remove-ai-watermarks[classify]'. device is a library parameter (None / "auto" / "cpu" / "cuda"), not a CLI option.

Weights are not in git. First call downloads wiltodelta/raiw-photo-classify at the freeze revision pinned in classify.py, or reads RAIW_CLASSIFY_WEIGHTS if that directory already has clip-l-ft.pt, probe-weights-clip-l-ft.npz, detector.pt, and provider.pt. The training catalog is not published with the package. Deployments that pre-cache the snapshot for offline resolution must request exactly classify.WEIGHTS_ALLOW_PATTERNS (the runtime resolves that same list under HF_HUB_OFFLINE=1; a partial pre-cache fails the whole resolution, even for files the runtime would fall back on).

What one call returns

One request runs both heads. The receipt gate and Model 2 run only after Model 1 is DEFINITELY.

flowchart TD
  image[Photograph] --> m1[Model 1 CLIP-L-ft ridge AND MLP]
  m1 -->|likely_human| human["label human, provider none"]
  m1 -->|possibly| unk["label unknown, provider none"]
  m1 -->|definitely| rg[Receipt gate on the same CLIP vector]
  rg -->|receipt document| rgu["label unknown, detector definitely, provider none"]
  rg -->|not a receipt| m2[Model 2 124-d focal heads]
  m2 -->|openai google muse-image bytedance| named["label ai plus provider"]
  m2 -->|china rest wins| none["label ai, provider none"]
  m2 -->|no_ai or extract fail| aiOnly["label ai, provider none"]

Field	Values	Meaning
label	ai, human, unknown	Public verdict. ai only on DEFINITELY
domain	photo	This freeze is photographic only
detector	definitely, possibly, likely_human	Raw Model 1 gate
provider	openai, google, bytedance, muse-image, or None	Model 2, only if label is ai

unknown with detector=definitely is the receipt-document gate abstaining: the file looked AI-generated to Model 1 and like a receipt photograph to the gate, so no verdict is published and no provider is read.

human is camera-like under this contract, not a proof of authorship. unknown is POSSIBLY, or a domain this freeze does not support. provider=None on an ai file is often FLUX or another generator that is not one of the four heads, or a 124-d extract refusal.

Approach

CLIP-L content embeddings separate generated photographs from camera photographs. They do not separate generated photographs from receipts or UI. A ridge on fine-tuned CLIP-L (last two vision blocks, 224 letterbox) is Model

1. A small MLP on the same vectors, ANDed with the ridge, is the freeze DEFINITELY gate.

Provider attribution uses a different feature: 124-d residual ratios on 256 px patches. OpenAI versus Gemini is a strong split there. "AI or not" is the wrong job on that bank, which is why Model 2 is gated.

Provider names the renderer, not the product UI. Bing Image Creator signed Microsoft, OpenAI scores openai. Designer signed Microsoft, Google LLC scores google. There is no Microsoft pixel class, and that is measured, not assumed (2026-09-07: a head trained on 114 Microsoft-brand rows recognizes 8% of its own test cell and names a fresh Copilot generation openai): Microsoft's products render with other vendors' models, so Microsoft attribution is metadata, not pixels. openai and google are provider classes. muse-image is Muse Image output, not a general Meta class. tc260 covers the REST of China's generator ecosystem, producers that are peers of openai/google/meta, not one producer: Qwen, Yuanbao, Kling, and others sit behind that residual class. No mixed head can honestly name that group, so its win abstains: China-generated content from those producers publishes provider=None until per-producer classes exist. bytedance is the shared ByteDance generator lineage: Doubao and Jimeng render with one model family (measured 2026-09-07: separate heads cross-fire even at 3.5x train mass, the union holds its cell), so the class names the lineage, not the app that signed.

Evaluation

Two CPU retrains were byte-identical. DEFINITELY is the shipped cut.

Head	Cell	Result
Detector DEFINITELY	AI-test	92.4% (n=1,847)
Detector DEFINITELY	Open Images fresh	1.3% (n=3,000)
Detector DEFINITELY	Kodak	0/24
Detector DEFINITELY	FLUX hold	83.0% (n=300)
Class	OpenAI	90.8% (345/380 of 381)
Class	Google	90.9% (339/373 of 377)
Class	Bytedance (Doubao+Jimeng lineage)	83.9% (271/323 of 323; 83.6% extended 529/633 with the historical harvest)
Class	Muse Image hold-out v3	89.4% (177/198)
Class	Muse Image hold-out pooled	88.4% (243/275 listed 277)
Class	meme templates, ungated	29.1% leak

The ungated meme row is why Model 2 never runs on human or unknown.

Receipt-document gate

Added 2026-09-02. A linear head on the CLIP-L-ft vector Model 1 already computes, trained on CORD-v2 train (800, CC BY 4.0) plus 200 synthetic capture-style receipts against 2,400 ai_train negatives. The threshold is certified on CORD validation plus a synthetic holdout; no field-receipt pixels were used for training or the threshold. On DEFINITELY only: a hit publishes unknown and skips the 124-d provider pass.

Cell	Result
Field phone receipts, false ai before	19/59 -> 2/59
Field phone receipts, gated to unknown	57/59
CORD photographs (99)	0/99 false ai
ai_test DEFINITELY cost	7/1,847 (0.38%)

Artifacts: receipt-gate-shipped-2026-09-02/report.json and receipt-gate-retrain-2026-09-07/report.json (private research tree). The head ships with the model under the STABLE name receipt-gate.npz in the Hub snapshot with the threshold inside the artifact (legacy dated spelling still readable, package asset as the fallback for pre-gate freezes); the operating point records it in photo-classify-hf/operating-point.json.

Limits

• Receipts abstain through the 2026-09-02 gate (below); a photorealistic AI receipt can still score as human. UI, scans, maps, and community art remain out of contract.
• Not SynthID, not C2PA, not is_ai_generated.
• Images under 256 px cannot yield 124-d features, so provider abstains.
• FLUX is a hold-out at 83% DEF, not a named provider class.
• China's generator ecosystem splits by generator: bytedance (the shared Doubao+Jimeng lineage) is named; the rest (Qwen, Yuanbao, Kling, ...) abstains until per-producer train mass holds its own cells (qwen measured at 64 train rows: 27%, stays in the fallback).

identify, has_invisible_target, all, and invisible do not import this module. A no-signal provenance result stays unknown until you call classify yourself.

Training data, the retrain pack (caches only, no images), and scripts/retrain_photo_classify.py are in photo-classify-training.md.