Add the classify extra for the 2026-08-31 photo freeze. (#90)

Ship Model 1 then gated Model 2 as an explicit CLI and library call.
identify, has_invisible_target, and all do not import it. Weights stay
on Hugging Face wiltodelta/raiw-models at a pinned revision.
This commit is contained in:
Victor Kuznetsov authored and GitHub committed 2026-08-31 22:45:18 -07:00
1 parent 55e7c53699
commit 925401c905
25 files changed
+1437 -14

No files matched your search

+3
View File
@@ -54,6 +54,9 @@ Do not classify an entire module as untestable because its main path downloads a
stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a
face. When one profile inherits another's stage, guard the invariants that stage
relies on, not just the code path.
- the photo-classify gate in `test_classify.py`: DEFINITELY is ridge AND MLP,
POSSIBLY does not emit a provider, and `identify` does not import
`remove_ai_watermarks.classify`.
- the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to
mirror another, compare them field by field rather than pinning the values you happen
to know about, so the next field added on one side and not the other fails at the
+1 -1
View File
@@ -39,7 +39,7 @@ Before a release, read [`docs/release-and-distribution.md`](docs/release-and-dis
[`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem.
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md).
Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); it is not `identify`. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md).
## Data safety
+12
View File
@@ -32,6 +32,7 @@ removal.
| Goal | Command | GPU |
| --- | --- | --- |
| Find provenance signals and watermarks | `identify` | No |
| Classify a photograph from pixels (opt-in, not provenance) | `classify` | No |
| Remove known visible AI marks | `visible` | No |
| Erase a region you select | `erase` | No |
| Strip AI metadata | `metadata` | No |
@@ -54,6 +55,7 @@ its linked C2PA manifest; metadata stripping alone removes only the manifest.
| Need | Install |
| --- | --- |
| Metadata inspection and stripping | `remove-ai-watermarks` |
| Photograph AI-versus-camera classification | `remove-ai-watermarks[classify]` |
| Visible detection and removal | `remove-ai-watermarks[visible]` |
| Visible video processing | `remove-ai-watermarks[video]` |
| Video SynthID removal | `remove-ai-watermarks[video,diffusion]` |
@@ -80,6 +82,16 @@ Inspect an image:
remove-ai-watermarks identify image.png
```
To classify a photograph from pixels (AI versus camera, optional provider),
install the extra and call `classify`. `identify` never starts it:
```bash
uv tool install --force "remove-ai-watermarks[classify]"
remove-ai-watermarks classify image.png
```
Guide: [photo pixel classification](docs/photo-classify.md).
For visible watermark removal, install the pixel dependencies:
```bash
+9 -3
View File
@@ -2,7 +2,10 @@
> Research archive for metadata-free `ai_generated` versus
> `not_ai_generated` classifiers. These are not SynthID detectors and are not
> shipped product verdicts. Current behavior: [supported signals](supported-signals.md)
> `identify` verdicts. The 2026-08-31 freeze is consumed only by explicit
> `classify` / `classify_pixels`. Current behavior:
> [photo pixel classification](photo-classify.md),
> [supported signals](supported-signals.md),
> and [known limitations](known-limitations.md).
>
> Sister pages: [SynthID source classifiers](synthid-classifiers.md),
@@ -18,8 +21,11 @@ origin-locked residual bank are different features for different jobs.
## Frozen result, 2026-08-31
The freeze is **AI versus camera**, not a universal detector, and it is not
wired into `identify`. Detector: CLIP-L-ft ridge AND freeze MLP.
Provider: 124-d focal heads, only after DEFINITELY or POSSIBLY.
wired into `identify`. The library consumes it only through explicit
`classify_pixels` / CLI `classify`. Detector: CLIP-L-ft ridge AND freeze MLP.
Provider: 124-d focal heads, only after DEFINITELY. Library guide:
[photo-classify.md](photo-classify.md). Hub card:
[photo-classify-hf/README.md](photo-classify-hf/README.md).
| Cell | Value |
| --- | ---: |
+32
View File
@@ -14,6 +14,7 @@ defaults. This page focuses on choosing the right command.
| Command or signal | Required installation |
| --- | --- |
| `metadata` and metadata-only `identify` | Default package |
| `classify` | `remove-ai-watermarks[classify]` |
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
| Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` |
@@ -65,6 +66,37 @@ remove-ai-watermarks identify image.png --no-visible
Despite the historical option name, `--no-visible` skips both visible and open
invisible pixel detectors. Metadata inspection still runs.
## Classify a photograph from pixels
This is not provenance. `identify` never starts it, including after a
no-signal metadata scan. Install the extra first:
```bash
uv tool install --force "remove-ai-watermarks[classify]"
```
Then:
```bash
remove-ai-watermarks classify image.png
remove-ai-watermarks classify image.png --json
```
The command runs the frozen photo detector (CLIP-L-ft ridge AND freeze MLP).
Only a DEFINITELY result is reported as `ai`. On that result only, the 124-d
provider head may return `openai`, `google`, `meta`, or `tc260`. POSSIBLY is
`unknown`. Camera-like photographs are `human`. The call does not run cleanup
and does not set `is_ai_generated`.
The contract is AI versus camera. Receipts, UI, and digital art are out of
scope. Microsoft is not a pixel class: a DALL-E Bing image scores as `openai`,
an Imagen Designer image as `google`. Weights download on first use from
[`wiltodelta/raiw-models`](https://huggingface.co/wiltodelta/raiw-models),
or from `RAIW_CLASSIFY_WEIGHTS` if that directory already holds the freeze
files.
Guide: [photo pixel classification](photo-classify.md). Hub card:
[photo-classify-hf/README.md](photo-classify-hf/README.md).
## Remove known visible marks
Install `remove-ai-watermarks[visible]` before using `visible` or `erase`.
+3 -1
View File
@@ -12,6 +12,8 @@ to run the tool. Use the maintainer references only when changing the code.
| [Python API](python-api.md) | You want to call the package from Python. |
| [Supported signals](supported-signals.md) | You need to know which visible marks, metadata formats, and invisible signals are covered. |
| [Known limitations](known-limitations.md) | You need the quality, device, format, or verification boundaries. |
| [Photo pixel classification](photo-classify.md) | You want `classify` / `classify_pixels`: AI versus camera, optional provider. |
| [Photo classifier training](photo-classify-training.md) | You need the retrain pack, public sources, and what is not published. |
| [Scope, safety, and legal notes](legal-and-safety.md) | You need the intended use and legal context. |
## Maintainer references
@@ -35,7 +37,7 @@ The current behavior is defined by the code, tests, README, and user guides.
- [ControlNet removal research](controlnet-removal-pipeline-research.md)
- [Qwen improvement research](qwen-improvement-research.md)
- [Doubao reverse-alpha research](research-doubao-distillation.md)
- [AI-generated image classifiers](ai-generated-image-classifiers.md) (photo AI-versus-camera freeze, not a product verdict)
- [AI-generated image classifiers](ai-generated-image-classifiers.md) (photo AI-versus-camera freeze consumed by `classify_pixels`, not `identify`)
- [Photo classifier Hugging Face card](photo-classify-hf/README.md) (Hub publication text of the 2026-08-31 freeze)
- [SynthID source classifiers](synthid-classifiers.md)
- [SynthID local detector research](synthid-detector-research.md)
+2
View File
@@ -112,6 +112,7 @@ application actually uses:
| `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No |
| `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No |
| `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes |
| `classify` | Metadata-free photo AI-versus-camera classifier plus gated provider | `pixels`, Torch, Transformers | Yes |
| `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes |
| `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
@@ -137,6 +138,7 @@ flowchart LR
draft["text-draft"]
heif
trustmark
classify --> pixels
```
`heif`, `trustmark`, and `text-draft` are independent branches. Combine them
+12
View File
@@ -4,6 +4,18 @@ This page describes current product limits. Historical measurements and
superseded experiments live in the research archive listed in
[the documentation index](index.md).
## Pixel classification
`classify` / `classify_pixels` is AI versus camera, not a universal AI detector.
Receipts, UI screenshots, and digital art are out of contract: a thermal receipt
can score as AI, a photorealistic AI receipt as human. POSSIBLY is reported as
`unknown`. Provider attribution runs only after DEFINITELY and can abstain when
124-d extraction refuses the file. Microsoft is not a pixel class.
The command is never started by `identify`. A no-signal provenance result stays
unknown. This is not a clean verdict and it does not run cleanup. Full guide:
[photo pixel classification](photo-classify.md).
## Visible removal
### Fill quality depends on the background
+21 -1
View File
@@ -11,7 +11,8 @@ updated in the same change.
## Architecture
The package has four main paths:
The package has four main paths, plus an opt-in pixel classifier that is not
on the provenance or removal graph:
```mermaid
flowchart LR
@@ -19,13 +20,18 @@ flowchart LR
Input --> Visible[Visible mark removal]
Input --> Invisible[Diffusion regeneration]
Input --> Metadata[Metadata stripping]
Input --> Classify[classify_pixels]
Identify --> Report[ProvenanceReport]
Visible --> VisibleOutput[Localized and filled image]
Invisible --> InvisibleOutput[Regenerated image]
Metadata --> MetadataOutput[Container with AI metadata removed]
Classify --> PixelClassification
```
`identify` does not call `classify_pixels`. `has_invisible_target` and `all`
do not read it.
The `all` command runs visible removal, optional invisible regeneration, and
metadata stripping in that order.
@@ -576,6 +582,20 @@ provenance normalizer. Integration code publishes the strict
`ProvenanceReport.to_dict()` alongside it rather than letting operational fields or
derived results influence detection.
### Photo pixel classifier
[`classify.py`](../src/remove_ai_watermarks/classify.py) is the 2026-08-31
photo freeze: CLIP-L-ft ridge AND freeze MLP, then 124-d focal heads only on
DEFINITELY. The public label is `ai` / `human` / `unknown`. POSSIBLY is
`unknown`. Provider is `None` unless the label is `ai`. Tests in
`tests/test_classify.py` pin the gate without downloads and pin that
`identify` does not import this module.
Weights stay out of git. The Hub snapshot is `wiltodelta/raiw-models`.
`RAIW_CLASSIFY_WEIGHTS` overrides it. The extra is `classify`. User guide:
[photo-classify.md](photo-classify.md). Hub card:
[photo-classify-hf/README.md](photo-classify-hf/README.md).
### Pixel forensics
[`pixel_evidence.py`](../src/remove_ai_watermarks/pixel_evidence.py) measures six
+86
View File
@@ -0,0 +1,86 @@
# Photo classifier training and data
How the 2026-08-31 freeze was fit, what a retrain needs, and where those
files come from. Inference is [photo-classify.md](photo-classify.md). The
campaign log is [ai-generated-image-classifiers.md](ai-generated-image-classifiers.md).
The shipped package does not include the catalog or the embeddings. Fitting
the heads again uses the cache pack. The original pixels still have to be
kept: a missing embedding, a new backbone, or an audit all need the files.
## Retrain pack (caches)
CPU retrain of both heads needs sha256-keyed matrices and the catalog of
roles. That step does not open JPEG or PNG files. It is not permission to
drop the pixels.
| File | Role |
| --- | --- |
| `catalog.json` | 32,690 rows: `sha256`, `roles`, optional `family` / public `source` |
| `embeddings-by-sha256.npz` | CLIP-L-ft 768-d vectors for Model 1 |
| `extra-clip-l.npz` | CLIP-L extras for the 150 Flickr UI hashes |
| `features-124d.npz` | 124-d residual vectors for Model 2 |
| `probe-weights-clip-l-ft.npz` | Ridge mean, scale, weights, `thr_oi_1pct` |
| `meta-split.json` | Meta train 217, hold-out v2 79, hold-out v3 198 |
Run:
```bash
uv run python scripts/retrain_photo_classify.py --pack DIR --out DIR
```
Seeds are fixed: detector `20260940`, provider `20260956`. Two consecutive
runs must be byte-identical.
Not in the **published** retrain pack: raw photographs, COCO/Open Images/Kodak
binaries, CORD, Danbooru. Those files stay in their public corpora or in the
private freeze archive. Pixel rebuild of a missing embedding still needs
`clip-l-ft.pt` and the original file.
## Where the rows come from
Identity is always sha256 of the file bytes.
### Public, named corpora
| Role | Source |
| --- | --- |
| `detector_photo_train`, `detector_photo_coco_hold` | COCO 2017 validation stills |
| `detector_open_images_*`, `detector_photo_dev_oi` | Open Images photographs. Locked fresh never enters train |
| `detector_photo_kodak` | Kodak lossless true color, 24 files |
| `detector_photo_picsum_eval` | Picsum |
| `detector_flux_hf_hold` | Hugging Face FLUX stills, hold-out |
| Flickr UI cell (150) | Openverse `license_type=all-cc`, host `live.staticflickr.com` only |
| Meta hold-out v3 (198) | Meta Muse Image API, eval-only, not in train |
### In-house labeled photographs, not redistributed
Provider OpenAI, Google, and TC260 train and test rows, and most detector AI
train and test rows, are labeled photographic generations held privately.
The published pack carries only their sha256, roles, and precomputed vectors.
It does not carry the pixels.
### Out of the freeze train
CORD receipts, FUNSD forms, Danbooru, charts, and map tiles were measured as
failure domains. They are not Model 1 train. A future universal router treats
them as other domains, not as extra CLIP-L negatives.
## What the freeze script does
Historically `freeze_both_models.py` built the catalog from local paths, then
trained. That campaign code is private and is not in this repository. The
public entry is `scripts/retrain_photo_classify.py`, which skips catalog
construction and fits the same heads from the pack above.
1. Load CLIP-L-ft vectors by sha256. Fit the 768-512-128-1 MLP on
`detector_ai_train` plus extras versus `detector_photo_train` plus struct.
Threshold is the 1.67% quantile on `detector_photo_dev_oi`.
2. AND that MLP with the frozen ridge in the probe file. DEFINITELY is both.
3. Load 124-d vectors. Fit one-vs-rest focal heads for openai, google, tc260,
meta_muse_image, no_ai. Margin 0.30.
4. Repeat with the same seeds. Compare bytes.
CLIP-L fine-tune of the last two vision blocks is earlier, seed 20260822, and
is not repeated by the retrain script. The backbone file `clip-l-ft.pt` is an
inference and embedding dependency, not a retrain-pack input.
+122
View File
@@ -0,0 +1,122 @@
# Photo pixel classification
`classify` gives a pixel opinion on a **photograph**: camera-like, generated,
or unknown, and optionally which renderer it resembles. It is a separate
command from `identify`. Metadata inspection never starts it.
This page is the library guide. The Hugging Face model card is the same freeze
written for Hub publication: [photo-classify-hf/README.md](photo-classify-hf/README.md).
Campaign notes and rejected variants live in
[AI-generated image classifiers](ai-generated-image-classifiers.md).
## When to use it
Use `identify` for C2PA, IPTC, visible marks, and other provenance. Use
`classify` when those signals are gone and the file is still a photograph.
Do not use `classify` to prove a file is clean. Do not use it on receipts, UI,
or digital art. Do not treat `provider=openai` as a decoded SynthID payload.
## Install and run
```bash
uv tool install --force "remove-ai-watermarks[classify]"
remove-ai-watermarks classify image.png
remove-ai-watermarks classify image.png --json
```
```python
from pathlib import Path
from remove_ai_watermarks.classify import classify_pixels
result = classify_pixels(Path("image.png"))
print(result.label, result.detector, result.provider)
```
Missing extra raises `RuntimeError` with
`pip install 'remove-ai-watermarks[classify]'`. `device` is a library
parameter (`None` / `"auto"` / `"cpu"` / `"cuda"`), not a CLI option.
Weights are not in git. First call downloads
[`wiltodelta/raiw-models`](https://huggingface.co/wiltodelta/raiw-models)
at the freeze revision pinned in `classify.py`, or reads
`RAIW_CLASSIFY_WEIGHTS` if that directory already has `clip-l-ft.pt`,
`probe-weights-clip-l-ft.npz`, `detector.pt`, and `provider.pt`. The training
catalog is not published with the package.
## What one call returns
One request runs both heads. Model 2 runs only after Model 1 is DEFINITELY.
```mermaid
flowchart TD
image[Photograph] --> m1[Model 1 CLIP-L-ft ridge AND MLP]
m1 -->|likely_human| human["label human, provider none"]
m1 -->|possibly| unk["label unknown, provider none"]
m1 -->|definitely| m2[Model 2 124-d focal heads]
m2 -->|openai google meta tc260| named["label ai plus provider"]
m2 -->|no_ai or extract fail| aiOnly["label ai, provider none"]
```
| Field | Values | Meaning |
| --- | --- | --- |
| `label` | `ai`, `human`, `unknown` | Public verdict. `ai` only on DEFINITELY |
| `domain` | `photo` | This freeze is photographic only |
| `detector` | `definitely`, `possibly`, `likely_human` | Raw Model 1 gate |
| `provider` | `openai`, `google`, `meta`, `tc260`, or `None` | Model 2, only if `label` is `ai` |
`human` is camera-like under this contract, not a proof of authorship.
`unknown` is POSSIBLY, or a domain this freeze does not support. `provider=None`
on an `ai` file is often FLUX or another generator that is not one of the four
heads, or a 124-d extract refusal.
## Approach
CLIP-L content embeddings separate generated photographs from camera
photographs. They do not separate generated photographs from receipts or UI.
A ridge on fine-tuned CLIP-L (last two vision blocks, 224 letterbox) is Model
1. A small MLP on the same vectors, ANDed with the ridge, is the freeze
DEFINITELY gate.
Provider attribution uses a different feature: 124-d residual ratios on 256 px
patches. OpenAI versus Gemini is a strong split there. "AI or not" is the
wrong job on that bank, which is why Model 2 is gated.
Provider names the renderer, not the product UI. Bing Image Creator signed
Microsoft, OpenAI scores `openai`. Designer signed Microsoft, Google LLC
scores `google`. There is no Microsoft pixel class. Meta is Muse Image, not
an Instagram `made_with_ai` tag.
## Freeze 2026-08-31
Two CPU retrains were byte-identical. DEFINITELY is the shipped cut.
| Head | Cell | Result |
| --- | --- | ---: |
| Detector DEFINITELY | AI-test | 92.4% (n=1,847) |
| Detector DEFINITELY | Open Images fresh | 1.3% (n=3,000) |
| Detector DEFINITELY | Kodak | 0/24 |
| Detector DEFINITELY | FLUX hold | 83.0% (n=300) |
| Provider | OpenAI | 90.8% (345/380 of 381) |
| Provider | Google | 90.9% (339/373 of 377) |
| Provider | TC260 | 78.6% (298/379 of 384) |
| Provider | Meta hold-out v3 | 89.4% (177/198) |
| Provider | Meta hold-out pooled | 88.4% (243/275 listed 277) |
| Provider | meme templates, ungated | 29.1% leak |
The ungated meme row is why Model 2 never runs on `human` or `unknown`.
## Limits
- Receipts, UI, scans, maps, and community art are out of contract.
- Not SynthID, not C2PA, not `is_ai_generated`.
- Images under 256 px cannot yield 124-d features, so provider abstains.
- FLUX is a hold-out at 83% DEF, not a named provider class.
`identify`, `has_invisible_target`, `all`, and `invisible` do not import this
module. A no-signal provenance result stays unknown until you call `classify`
yourself.
Training data, the retrain pack (caches only, no images), and
`scripts/retrain_photo_classify.py` are in
[photo-classify-training.md](photo-classify-training.md).
+28 -1
View File
@@ -6,7 +6,7 @@ and pipeline modules are intended for maintainers and specialized workflows.
Dependency groups are identical for the CLI and Python API. The default install
covers metadata extraction, normalization, verdict logic, and stripping.
Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection
uses `detect`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and
uses `detect`; pixel photo classification uses `classify`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and
visible video processing uses `video`. Video SynthID removal is a separate VAE
path that still runs on CPU and combines `video` and `diffusion`. Add `heif`
independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See
@@ -144,6 +144,33 @@ print(summary.invisible_unavailable) # outputs that still carry the watermark
`mode` is `all`, `visible`, `invisible`, or `metadata`. Pass a constructed
`InvisibleEngine` as `engine` to load the model once for the whole directory.
## Classify a photograph from pixels
This is not provenance. `identify` does not call it. Install
`remove-ai-watermarks[classify]` first.
```python
from pathlib import Path
from remove_ai_watermarks.classify import classify_pixels
result = classify_pixels(Path("input.png"))
print(result.label, result.detector, result.provider)
```
`label` is `ai` only on a DEFINITELY detector result (ridge AND freeze MLP).
POSSIBLY is `unknown`. Camera-like photographs are `human`. `provider` is
`openai`, `google`, `meta`, or `tc260` only when `label` is `ai` and the 124-d
head beats `no_ai` by the freeze margin. Otherwise it is `None`, including
when 124-d extraction refuses the file.
`device` is a library parameter: `None` / `"auto"` detect, `"cpu"` or `"cuda"`
pin. It is not a CLI option.
Missing extra raises `RuntimeError` with the quoted install command
`'remove-ai-watermarks[classify]'`. Guide:
[photo pixel classification](photo-classify.md).
## Inspect provenance
The default installation evaluates file metadata. Add `visible`, `detect`, or
+4 -2
View File
@@ -74,8 +74,10 @@ pin. Its callback smoke tests run in that repository's CI on every push; they
are the scenario check for the demo surface.
The Hugging Face **model** `wiltodelta/raiw-models` is not part of that
release fan-out. It holds the photo-classify freeze weights. Update it only
with `.github/workflows/publish-photo-classify-hf.yml` (`workflow_dispatch`,
release fan-out. It holds the photo-classify freeze weights. The library extra
`classify` downloads that snapshot on first use, or reads
`RAIW_CLASSIFY_WEIGHTS`. Update the Hub repo only with
`.github/workflows/publish-photo-classify-hf.yml` (`workflow_dispatch`,
type `publish`). Full mode downloads the four weight files from the GitHub
Release tag `photo-classify-freeze-2026-08-31` (or the tag you pass) and
uploads them with the card in `docs/photo-classify-hf/`. Card mode updates
+10 -1
View File
@@ -142,6 +142,15 @@ trustmark = [
# wheels. Keep this feature installable only on its real runtime range.
"trustmark>=0.8.0; python_version < '3.13'",
]
# Metadata-free photo classifier (Model 1 then gated Model 2). CPU torch is
# enough. identify never imports this extra. Weights download on first use.
classify = [
"remove-ai-watermarks[pixels]",
"torch>=2.0.0",
"transformers>=5,<6",
"tokenizers>=0.22,<0.23",
"huggingface-hub>=0.20.0",
]
# Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX,
# Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and
# cached by huggingface_hub; it is never bundled in this repo. The default cv2
@@ -174,7 +183,7 @@ dev = [
]
# ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is
# independently sufficient for a removal, which it is not.
all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan]"]
all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan,classify]"]
[project.scripts]
remove-ai-watermarks = "remove_ai_watermarks.cli:main"
+1
View File
@@ -43,6 +43,7 @@ explicitly names a tracked canonical result.
| `visible_positives.py` | List corpus images carrying a registered visible mark. |
| `visible_recall_sample.py` | Build an unbiased blinded sample for recall measurement. |
| `visible_sheets.py` | Build blinded contact sheets for relaxation candidates. |
| `retrain_photo_classify.py` | CPU-retrain the 2026-08-31 photo heads from a sha256-keyed cache pack, no images. |
| `publish_photo_classify_hf.py` | Upload the photo-classify card and freeze weights to Hugging Face `wiltodelta/raiw-models`. Manual; the Action `publish-photo-classify-hf.yml` is the write-token path. |
## Research and diagnostic prototypes
+198
View File
@@ -0,0 +1,198 @@
#!/usr/bin/env python3
"""Retrain the 2026-08-31 photo heads from a sha256-keyed cache pack.
No image I/O. Requires catalog.json, embeddings-by-sha256.npz or extra-clip-l.npz
plus embeddings keyed by sha256, features-124d.npz, and probe-weights-clip-l-ft.npz.
uv run python scripts/retrain_photo_classify.py --pack DIR --out DIR
"""
from __future__ import annotations
import argparse
import json
import logging
from pathlib import Path
import numpy as np
import torch
from torch import nn
log = logging.getLogger(__name__)
DETECTOR_SEED = 20260940
PROVIDER_SEED = 20260956
EPOCHS = 40
MLP_WIDTH = 512
LR = 5e-4
BATCH = 64
TARGET_PHOTO_FPR = 0.0167
FOCAL_GAMMA = 2.0
FOCAL_EPOCHS = 40
FOCAL_LR = 1e-3
FOCAL_BATCH = 64
LABELS = ("openai", "google", "tc260", "meta_muse_image", "no_ai")
def _load_sha_vectors(path: Path) -> dict[str, np.ndarray]:
payload = np.load(path, allow_pickle=True)
if "sha256" in payload.files:
keys = payload["sha256"]
vectors = payload["vectors"]
return {str(key): np.asarray(vector) for key, vector in zip(keys, vectors, strict=True)}
raise SystemExit(f"{path} must contain sha256 and vectors")
def _mlp(dim: int) -> nn.Module:
return nn.Sequential(
nn.Linear(dim, MLP_WIDTH),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(MLP_WIDTH, MLP_WIDTH // 4),
nn.ReLU(),
nn.Dropout(0.1),
nn.Linear(MLP_WIDTH // 4, 1),
)
def _provider_head() -> nn.Module:
return nn.Sequential(
nn.Linear(124, 64),
nn.ReLU(),
nn.Dropout(0.2),
nn.Linear(64, 1),
)
def _focal_loss(logits: torch.Tensor, targets: torch.Tensor) -> torch.Tensor:
bce = nn.functional.binary_cross_entropy_with_logits(logits, targets, reduction="none")
prob = torch.sigmoid(logits)
pt = torch.where(targets > 0.5, prob, 1.0 - prob)
return ((1.0 - pt) ** FOCAL_GAMMA * bce).mean()
def _train_detector(emb: dict[str, np.ndarray], catalog: dict) -> tuple[nn.Module, float]:
torch.manual_seed(DETECTOR_SEED)
roles = {row["sha256"]: set(row["roles"]) for row in catalog["rows"]}
pos, neg = [], []
for sha, vector in emb.items():
role = roles.get(sha, set())
if "detector_ai_train" in role or "detector_ai_extra" in role:
pos.append(vector)
if "detector_photo_train" in role or "detector_struct" in role:
neg.append(vector)
x_pos = torch.from_numpy(np.stack(pos)).float()
x_neg = torch.from_numpy(np.stack(neg)).float()
log.info("detector train pos=%d neg=%d", len(x_pos), len(x_neg))
model = _mlp(x_pos.shape[1])
optimizer = torch.optim.AdamW(model.parameters(), lr=LR, weight_decay=1e-3)
loss_fn = nn.BCEWithLogitsLoss()
rng = np.random.default_rng(DETECTOR_SEED)
model.train()
for _epoch in range(EPOCHS):
pp = rng.permutation(len(x_pos))
pn = rng.permutation(len(x_neg))
for start in range(0, min(len(pp), len(pn)), BATCH):
features = torch.cat([x_pos[pp[start : start + BATCH]], x_neg[pn[start : start + BATCH]]])
labels = torch.cat([torch.ones(min(BATCH, len(pp) - start)), torch.zeros(min(BATCH, len(pn) - start))])
optimizer.zero_grad()
loss_fn(model(features).squeeze(-1), labels).backward()
optimizer.step()
model.eval()
dev = np.stack(
[
emb[row["sha256"]]
for row in catalog["rows"]
if "detector_photo_dev_oi" in row["roles"] and row["sha256"] in emb
]
)
with torch.inference_mode():
scores = model(torch.from_numpy(dev).float()).squeeze(-1).numpy()
threshold = float(np.quantile(scores, 1.0 - TARGET_PHOTO_FPR))
return model, threshold
def _train_provider(feat: dict[str, np.ndarray], catalog: dict) -> dict[str, nn.Module]:
def stack(role: str) -> np.ndarray:
vecs = [feat[row["sha256"]] for row in catalog["rows"] if role in row["roles"] and row["sha256"] in feat]
return np.stack(vecs) if vecs else np.empty((0, 124), dtype=np.float32)
extra = stack("provider_openai_extra")
openai = stack("provider_openai")
bank = {
"openai": np.concatenate([openai, extra]) if len(extra) else openai,
"google": stack("provider_google"),
"tc260": stack("provider_tc260"),
"meta_muse_image": stack("provider_meta_train"),
"no_ai": stack("provider_no_ai"),
}
struct = stack("provider_struct")
all_neg = np.concatenate([bank["no_ai"], struct]) if len(struct) else bank["no_ai"]
heads = {}
for i, name in enumerate(LABELS):
if name == "no_ai":
positive = all_neg
negative = np.concatenate([bank[n] for n in LABELS if n != "no_ai"])
else:
negative = np.concatenate([bank[n] for n in LABELS if n not in (name, "no_ai")] + [all_neg])
positive = bank[name]
torch.manual_seed(PROVIDER_SEED + i)
rng = np.random.default_rng(PROVIDER_SEED + i)
max_neg = min(len(negative), len(positive) * 2)
neg_idx = rng.permutation(len(negative))[:max_neg]
x_pos = torch.from_numpy(positive.astype(np.float32))
x_neg = torch.from_numpy(negative[neg_idx].astype(np.float32))
model = _provider_head()
optimizer = torch.optim.AdamW(model.parameters(), lr=FOCAL_LR, weight_decay=1e-3)
model.train()
for _epoch in range(FOCAL_EPOCHS):
pos_idx = rng.permutation(len(x_pos))
neg_order = rng.permutation(len(x_neg))
for start in range(0, min(len(pos_idx), len(neg_order)), FOCAL_BATCH):
features = torch.cat(
[x_pos[pos_idx[start : start + FOCAL_BATCH]], x_neg[neg_order[start : start + FOCAL_BATCH]]]
)
labels = torch.cat(
[
torch.ones(min(FOCAL_BATCH, len(pos_idx) - start)),
torch.zeros(min(FOCAL_BATCH, len(neg_order) - start)),
]
)
optimizer.zero_grad()
_focal_loss(model(features).squeeze(-1), labels).backward()
optimizer.step()
heads[name] = model.eval()
log.info("provider %s pos=%d", name, len(positive))
return heads
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--pack", type=Path, required=True)
parser.add_argument("--out", type=Path, required=True)
args = parser.parse_args()
logging.basicConfig(level=logging.INFO, format="%(message)s")
pack = args.pack.expanduser().resolve()
catalog = json.loads((pack / "catalog.json").read_text())
emb: dict[str, np.ndarray] = {}
for name in ("embeddings-by-sha256.npz", "extra-clip-l.npz"):
path = pack / name
if path.is_file():
emb.update(_load_sha_vectors(path))
feat = _load_sha_vectors(pack / "features-124d.npz")
probe = np.load(pack / "probe-weights-clip-l-ft.npz")
log.info("catalog=%d embeddings=%d features=%d", catalog["counts"]["files"], len(emb), len(feat))
args.out.mkdir(parents=True, exist_ok=True)
detector, threshold = _train_detector(emb, catalog)
torch.save(detector.state_dict(), args.out / "detector.pt")
heads = _train_provider(feat, catalog)
torch.save({name: head.state_dict() for name, head in heads.items()}, args.out / "provider.pt")
(args.out / "retrain.json").write_text(
json.dumps({"detector_threshold": threshold, "probe_thr": float(probe["thr_oi_1pct"])}, indent=2) + "\n"
)
log.info("wrote %s threshold=%.6f", args.out, threshold)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,86 @@
# pyright: reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false, reportMissingTypeStubs=false
"""CLIP-L letterbox embedder for the frozen photo detector.
The freeze fine-tuned the last two vision blocks of
``openai/clip-vit-large-patch14``. Inference letterboxes to 224 with pad
``(123, 117, 104)`` and L2-normalizes ``get_image_features``.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
from PIL import Image
from torch import nn
if TYPE_CHECKING:
from pathlib import Path
import torch
from numpy.typing import NDArray
BACKBONE = "openai/clip-vit-large-patch14"
SIZE = 224
PAD_RGB = (123, 117, 104)
def letterbox(image: Image.Image, size: int = SIZE) -> Image.Image:
"""Return a square RGB letterbox matching the freeze preprocessor."""
image = image.convert("RGB")
width, height = image.size
scale = size / max(width, height)
new_w = max(1, round(width * scale))
new_h = max(1, round(height * scale))
resized = image.resize((new_w, new_h), Image.Resampling.BICUBIC)
canvas = Image.new("RGB", (size, size), PAD_RGB)
canvas.paste(resized, ((size - new_w) // 2, (size - new_h) // 2))
return canvas
class HeadedCLIP(nn.Module):
"""Checkpoint layout of ``clip-l-ft.pt``: CLIP plus a unused linear head."""
def __init__(self, clip: Any) -> None:
super().__init__()
self.clip = clip
self.head = nn.Linear(clip.config.projection_dim, 1)
def embed(self, pixel_values: Any) -> Any:
outputs = self.clip.get_image_features(pixel_values=pixel_values)
vectors = outputs.pooler_output if hasattr(outputs, "pooler_output") else outputs
return nn.functional.normalize(vectors, dim=-1)
def load_headed_clip(checkpoint: Path, device: torch.device) -> HeadedCLIP:
"""Build CLIP-L from config and load the freeze state dict."""
import torch
from transformers import CLIPConfig, CLIPModel
clip = CLIPModel(CLIPConfig.from_pretrained(BACKBONE))
model = HeadedCLIP(clip)
state = torch.load(checkpoint, map_location="cpu", weights_only=True)
model.load_state_dict(state, strict=True)
return model.to(device).eval()
def load_processor() -> Any:
from transformers import CLIPImageProcessor
processor = CLIPImageProcessor.from_pretrained(BACKBONE)
processor.do_resize = False
processor.do_center_crop = False
return processor
def embed_image(model: HeadedCLIP, processor: Any, image: Image.Image, device: torch.device) -> NDArray[Any]:
"""Return a 768-d L2-normalized CLIP-L-ft vector."""
import numpy as np
import torch
boxed = letterbox(image)
pixels = processor(images=[boxed], return_tensors="pt")["pixel_values"].to(
device, dtype=next(model.parameters()).dtype
)
with torch.inference_mode():
vector = model.embed(pixels).float().cpu().numpy()[0]
return np.asarray(vector, dtype=np.float64)
@@ -0,0 +1,144 @@
"""124-d patch residual features for the frozen provider heads.
Lifted from the 2026-08-31 freeze extractor. The vector is ratios on
lattice-aligned 256 px patches: FFT band energy, comb contrast, and residual
autocovariance. Images smaller than the patch return None.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import cv2
import numpy as np
if TYPE_CHECKING:
from numpy.typing import NDArray
PATCH = 256
MAX_PATCHES = 12
BLUR_SIGMA = 1.0
BANDS = 8
LAGS = (8, 16)
CONTROL_LAGS = (11, 13)
REACH = 3
FEATURE_WIDTH = 124
def _lag_grid() -> list[tuple[int, int]]:
lags = [(0, dx) for dx in range(1, REACH + 1)]
lags += [(dy, dx) for dy in range(1, REACH + 1) for dx in range(-REACH, REACH + 1)]
return lags
LAG_GRID = _lag_grid()
def _opponent(pixels: NDArray[np.uint8]) -> NDArray[np.float64]:
values = pixels.astype(np.float64)
red, green, blue = values[:, :, 0], values[:, :, 1], values[:, :, 2]
return np.stack(
[
0.299 * red + 0.587 * green + 0.114 * blue,
red - green,
blue - 0.5 * (red + green),
],
axis=-1,
)
def _residual(planes: NDArray[np.float64]) -> NDArray[np.float64]:
blurred = cv2.GaussianBlur(planes, (0, 0), sigmaX=BLUR_SIGMA, sigmaY=BLUR_SIGMA, borderType=cv2.BORDER_REFLECT_101)
return planes - blurred
def _patch_origins(height: int, width: int) -> list[tuple[int, int]]:
if height < PATCH or width < PATCH:
return []
rows = list(range(0, height - PATCH + 1, PATCH))
columns = list(range(0, width - PATCH + 1, PATCH))
origins = [(row, column) for row in rows for column in columns]
if len(origins) <= MAX_PATCHES:
return origins
step = len(origins) / MAX_PATCHES
return [origins[int(index * step)] for index in range(MAX_PATCHES)]
def _masks() -> tuple[NDArray[np.bool_], NDArray[np.bool_], NDArray[np.bool_], list[NDArray[np.bool_]]]:
frequency_rows = np.fft.fftfreq(PATCH)[:, None]
frequency_columns = np.fft.fftfreq(PATCH)[None, :]
radius = np.hypot(frequency_rows, frequency_columns)
index_rows = np.arange(PATCH)[:, None]
index_columns = np.arange(PATCH)[None, :]
period16 = (index_rows % (PATCH // 16) == 0) & (index_columns % (PATCH // 16) == 0)
period8 = (index_rows % (PATCH // 8) == 0) & (index_columns % (PATCH // 8) == 0)
comb16_only = period16 & ~period8
background = ~period16
edges = np.linspace(0.0, 0.5, BANDS + 1)
bands = [(radius > edges[i]) & (radius <= edges[i + 1]) for i in range(BANDS)]
return comb16_only, period8, background, bands
COMB16, COMB8, BACKGROUND, BAND_MASKS = _masks()
def _contrast(power: NDArray[np.float64], comb: NDArray[np.bool_]) -> float:
reach = comb & (BAND_MASKS[1] | BAND_MASKS[2] | BAND_MASKS[3] | BAND_MASKS[4])
floor = BACKGROUND & (BAND_MASKS[1] | BAND_MASKS[2] | BAND_MASKS[3] | BAND_MASKS[4])
if not reach.any() or not floor.any():
return 0.0
background_power = float(power[floor].mean())
if background_power <= 0.0:
return 0.0
return float(power[reach].mean() / background_power)
def _autocorrelation(patch: NDArray[np.float64], lag: int) -> float:
centred = patch - patch.mean()
energy = float((centred**2).sum())
if energy <= 0.0:
return 0.0
rows = float((centred[:-lag, :] * centred[lag:, :]).sum())
columns = float((centred[:, :-lag] * centred[:, lag:]).sum())
return (rows + columns) / (2.0 * energy)
def image_features(pixels: NDArray[np.uint8]) -> NDArray[np.float64] | None:
"""Return the 124-d vector for RGB uint8 pixels, or None when geometry fails."""
if pixels.ndim != 3 or pixels.shape[2] != 3:
return None
residual = _residual(_opponent(pixels))
origins = _patch_origins(*pixels.shape[:2])
if not origins:
return None
per_patch: list[list[float]] = []
spread_slice: slice | None = None
for row, column in origins:
window = residual[row : row + PATCH, column : column + PATCH]
values: list[float] = []
for channel in range(3):
plane = window[:, :, channel]
power = np.asarray(np.abs(np.fft.fft2(plane)) ** 2, dtype=np.float64)
total = float(power[1:, 1:].sum())
if total <= 0.0:
return None
values.extend(float(power[mask].sum() / total) for mask in BAND_MASKS)
values.append(_contrast(power, COMB16))
values.append(_contrast(power, COMB8))
if spread_slice is None:
spread_slice = slice(0, len(values))
values.extend(_autocorrelation(plane, lag) for lag in LAGS)
values.extend(_autocorrelation(plane, lag) for lag in CONTROL_LAGS)
correlation = np.fft.ifft2(power).real
centre = float(correlation[0, 0])
if centre <= 0.0:
return None
values.extend(float(correlation[dy, dx] / centre) for dy, dx in LAG_GRID)
per_patch.append(values)
stack = np.asarray(per_patch, dtype=np.float64)
if spread_slice is None:
return None
vector = np.concatenate([np.median(stack, axis=0), stack[:, spread_slice].std(axis=0)])
if vector.shape != (FEATURE_WIDTH,):
return None
return vector
+363
View File
@@ -0,0 +1,363 @@
# pyright: reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false
"""Metadata-free photo classifier: Model 1 then gated Model 2.
This is not provenance and is not a cleanup command. ``identify``,
``has_invisible_target``, and ``all`` must not import this module.
Call :func:`classify_pixels` explicitly.
"""
from __future__ import annotations
import json
import logging
import os
import threading
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Literal
log = logging.getLogger(__name__)
CLASSIFY_EXTRA = "'remove-ai-watermarks[classify]'"
WEIGHTS_ENV = "RAIW_CLASSIFY_WEIGHTS"
WEIGHTS_REPO = "wiltodelta/raiw-models"
WEIGHTS_REVISION = "c0ac82b6f1ae9fc0b92c467562282e9422da6da6"
CLIP_FILE = "clip-l-ft.pt"
PROBE_FILE = "probe-weights-clip-l-ft.npz"
DETECTOR_FILE = "detector.pt"
PROVIDER_FILE = "provider.pt"
OPERATING_POINT_FILE = "operating-point.json"
_WEIGHT_FILES = (CLIP_FILE, PROBE_FILE, DETECTOR_FILE, PROVIDER_FILE)
# 2026-08-31 freeze operating point. Ridge threshold is also in the probe file;
# the runtime prefers the file when weights load.
MLP_THRESHOLD = 5.9586493237495395
RIDGE_THRESHOLD = 0.3056212276800537
PROVIDER_MARGIN = 0.30
CLIP_WIDTH = 768
PROVIDER_LABELS = ("openai", "google", "tc260", "meta_muse_image", "no_ai")
DetectorLevel = Literal["definitely", "possibly", "likely_human"]
PixelLabel = Literal["ai", "human", "unknown"]
PixelDomain = Literal["photo"]
PixelProvider = Literal["openai", "google", "meta", "tc260"]
PUBLIC_PROVIDER: dict[str, PixelProvider] = {
"openai": "openai",
"google": "google",
"tc260": "tc260",
"meta_muse_image": "meta",
}
_runtime: _Runtime | None = None
_runtime_lock = threading.Lock()
@dataclass(frozen=True)
class PixelClassification:
"""Combined Model 1 and Model 2 result for one image."""
label: PixelLabel
domain: PixelDomain
detector: DetectorLevel
provider: PixelProvider | None
def to_dict(self) -> dict[str, Any]:
return {
"label": self.label,
"domain": self.domain,
"detector": self.detector,
"provider": self.provider,
}
def is_available() -> bool:
"""True when the ``classify`` extra can import torch, transformers, numpy, and cv2."""
from remove_ai_watermarks.optional_deps import module_available
return module_available("torch", "transformers", "numpy", "cv2")
def detector_level(
ridge_score: float,
mlp_score: float,
*,
ridge_threshold: float = RIDGE_THRESHOLD,
mlp_threshold: float = MLP_THRESHOLD,
) -> DetectorLevel:
"""Map the freeze AND/OR rule onto definitely / possibly / likely_human."""
ridge = ridge_score > ridge_threshold
mlp = mlp_score > mlp_threshold
if ridge and mlp:
return "definitely"
if ridge or mlp:
return "possibly"
return "likely_human"
def label_for(level: DetectorLevel) -> PixelLabel:
"""Public label: only DEFINITELY is ``ai``. POSSIBLY abstains."""
if level == "definitely":
return "ai"
if level == "likely_human":
return "human"
return "unknown"
def provider_from_scores(scores: dict[str, float], *, margin: float = PROVIDER_MARGIN) -> PixelProvider | None:
"""Argmax among openai/google/tc260/meta that beat ``no_ai`` by ``margin``."""
no_ai = scores["no_ai"]
ai_names = [name for name in PROVIDER_LABELS if name != "no_ai"]
passed = [name for name in ai_names if scores[name] > no_ai + margin]
if not passed:
return None
best = max(passed, key=lambda name: scores[name])
return PUBLIC_PROVIDER[best]
def classify_from_scores(
ridge_score: float,
mlp_score: float,
provider_scores: dict[str, float] | None,
*,
ridge_threshold: float = RIDGE_THRESHOLD,
mlp_threshold: float = MLP_THRESHOLD,
provider_margin: float = PROVIDER_MARGIN,
) -> PixelClassification:
"""Pure gate: detector AND provider, no file I/O."""
level = detector_level(
ridge_score,
mlp_score,
ridge_threshold=ridge_threshold,
mlp_threshold=mlp_threshold,
)
label = label_for(level)
provider: PixelProvider | None = None
if label == "ai" and provider_scores is not None:
provider = provider_from_scores(provider_scores, margin=provider_margin)
return PixelClassification(label=label, domain="photo", detector=level, provider=provider)
def classify_pixels(path: Path, *, device: str | None = None) -> PixelClassification:
"""Run Model 1 then, on DEFINITELY, Model 2. Never called by ``identify``."""
if not is_available():
raise RuntimeError(f"pixel classification requires the classify extra. Install: pip install {CLASSIFY_EXTRA}")
runtime = _get_runtime(device)
from PIL import Image
with Image.open(path) as image:
rgb = image.convert("RGB")
clip_vector = _embed(runtime, rgb)
ridge = float(((clip_vector - runtime.ridge_mean) / runtime.ridge_scale) @ runtime.ridge_weights)
mlp = _mlp_score(runtime, clip_vector)
level = detector_level(
ridge,
mlp,
ridge_threshold=runtime.ridge_threshold,
mlp_threshold=runtime.mlp_threshold,
)
provider_scores = None
if level == "definitely":
forensic = _forensic(rgb)
if forensic is not None:
provider_scores = _provider_scores(runtime, forensic)
else:
log.info("124-d features unavailable for %s, provider abstains", path)
return classify_from_scores(
ridge,
mlp,
provider_scores,
ridge_threshold=runtime.ridge_threshold,
mlp_threshold=runtime.mlp_threshold,
provider_margin=runtime.provider_margin,
)
@dataclass
class _Runtime:
device: Any
clip_model: Any
processor: Any
detector: Any
provider_heads: dict[str, Any]
ridge_mean: Any
ridge_scale: Any
ridge_weights: Any
ridge_threshold: float
mlp_threshold: float
provider_margin: float
def _resolve_device(device: str | None) -> Any:
import torch
if device in (None, "auto"):
return torch.device("cuda" if torch.cuda.is_available() else "cpu")
if device == "cpu":
return torch.device("cpu")
if device == "cuda":
if not torch.cuda.is_available():
raise ValueError("cuda was requested but is not available")
return torch.device("cuda")
raise ValueError(f"unsupported classify device {device!r}")
def _weights_dir() -> Path:
override = os.environ.get(WEIGHTS_ENV)
if override:
path = Path(override)
if not path.is_dir():
raise RuntimeError(f"{WEIGHTS_ENV} is not a directory: {path}")
return path
from huggingface_hub import snapshot_download
try:
return Path(
snapshot_download(
repo_id=WEIGHTS_REPO,
revision=WEIGHTS_REVISION,
allow_patterns=[*_WEIGHT_FILES, OPERATING_POINT_FILE],
)
)
except Exception as exc:
raise RuntimeError(
"pixel classification weights are not installed. Set "
f"{WEIGHTS_ENV} to a directory with {', '.join(_WEIGHT_FILES)}, "
f"or install {CLASSIFY_EXTRA} and download {WEIGHTS_REPO}@{WEIGHTS_REVISION}."
) from exc
def _find_weight(folder: Path, name: str) -> Path | None:
direct = folder / name
nested = folder / "run1" / name
if direct.is_file():
return direct
if nested.is_file():
return nested
return None
def _weight_path(folder: Path, name: str) -> Path:
found = _find_weight(folder, name)
if found is None:
raise RuntimeError(f"pixel classification weights are missing ({name}) in {folder}")
return found
def _require_files(folder: Path) -> None:
missing = [name for name in _WEIGHT_FILES if _find_weight(folder, name) is None]
if missing:
raise RuntimeError(
"pixel classification weights are missing "
f"({', '.join(missing)}). Set {WEIGHTS_ENV} to a directory with the "
f"2026-08-31 freeze files, or install {CLASSIFY_EXTRA} and allow the "
f"Hugging Face download from {WEIGHTS_REPO}."
)
def _operating_point(folder: Path) -> tuple[float, float]:
path = folder / OPERATING_POINT_FILE
if not path.is_file():
return MLP_THRESHOLD, PROVIDER_MARGIN
payload = json.loads(path.read_text())
return float(payload["model1"]["mlp_threshold"]), float(payload["model2"]["margin"])
def _build_detector() -> Any:
from torch import nn
return nn.Sequential(
nn.Linear(CLIP_WIDTH, 512),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(512, 128),
nn.ReLU(),
nn.Dropout(0.1),
nn.Linear(128, 1),
)
def _build_provider_head() -> Any:
from torch import nn
return nn.Sequential(
nn.Linear(124, 64),
nn.ReLU(),
nn.Dropout(0.2),
nn.Linear(64, 1),
)
def _load_runtime(device: Any) -> _Runtime:
import numpy as np
import torch
from remove_ai_watermarks._internal.clip_l_ft import load_headed_clip, load_processor
folder = _weights_dir()
_require_files(folder)
probe = np.load(_weight_path(folder, PROBE_FILE))
detector = _build_detector()
detector.load_state_dict(torch.load(_weight_path(folder, DETECTOR_FILE), map_location="cpu", weights_only=True))
detector.to(device).eval()
packed = torch.load(_weight_path(folder, PROVIDER_FILE), map_location="cpu", weights_only=True)
heads = {}
for name in PROVIDER_LABELS:
head = _build_provider_head()
head.load_state_dict(packed[name])
heads[name] = head.to(device).eval()
ridge_threshold = float(probe["thr_oi_1pct"])
mlp_threshold, provider_margin = _operating_point(folder)
log.info("loaded photo-classify freeze from %s", folder)
return _Runtime(
device=device,
clip_model=load_headed_clip(_weight_path(folder, CLIP_FILE), device),
processor=load_processor(),
detector=detector,
provider_heads=heads,
ridge_mean=np.asarray(probe["mean"], dtype=np.float64),
ridge_scale=np.asarray(probe["scale"], dtype=np.float64),
ridge_weights=np.asarray(probe["weights"], dtype=np.float64),
ridge_threshold=ridge_threshold,
mlp_threshold=mlp_threshold,
provider_margin=provider_margin,
)
def _get_runtime(device: str | None) -> _Runtime:
global _runtime
resolved = _resolve_device(device)
with _runtime_lock:
if _runtime is None or _runtime.device != resolved:
_runtime = _load_runtime(resolved)
return _runtime
def _embed(runtime: _Runtime, image: Any) -> Any:
from remove_ai_watermarks._internal.clip_l_ft import embed_image
return embed_image(runtime.clip_model, runtime.processor, image, runtime.device)
def _forensic(image: Any) -> Any:
import numpy as np
from remove_ai_watermarks._internal.forensic_124d import image_features
pixels = np.asarray(image, dtype=np.uint8)
return image_features(pixels)
def _mlp_score(runtime: _Runtime, clip_vector: Any) -> float:
import torch
tensor = torch.from_numpy(clip_vector.astype("float32")).unsqueeze(0).to(runtime.device)
with torch.inference_mode():
return float(runtime.detector(tensor).squeeze().cpu())
def _provider_scores(runtime: _Runtime, forensic: Any) -> dict[str, float]:
import torch
tensor = torch.from_numpy(forensic.astype("float32")).unsqueeze(0).to(runtime.device)
with torch.inference_mode():
return {name: float(head(tensor).squeeze().cpu()) for name, head in runtime.provider_heads.items()}
+39
View File
@@ -1485,6 +1485,45 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
console.print(f" - {c}")
# ── Metadata-free photo classification (opt-in, never a tail of identify) ──
@main.command("classify")
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
@click.option("--json", "as_json", is_flag=True, help="Emit the classification as JSON.")
def cmd_classify(source: Path, as_json: bool) -> None:
"""Classify a photograph as AI or camera-like from pixels.
Runs the frozen photo detector and, only on a DEFINITELY-AI result, the
provider head. This is not provenance. identify never starts this command.
Install: pip install 'remove-ai-watermarks[classify]'
"""
from remove_ai_watermarks import classify as classify_mod
if not classify_mod.is_available():
console.print(
"Error: the pixel-classification dependencies are not installed.\n"
f" Install them with: pip install {classify_mod.CLASSIFY_EXTRA}"
)
raise SystemExit(1)
source = _validate_image(source)
try:
result = classify_mod.classify_pixels(source)
except (OSError, RuntimeError, ValueError) as exc:
raise click.ClickException(str(exc)) from exc
if as_json:
click.echo(json.dumps(result.to_dict(), indent=2))
return
_banner()
console.print("\n Pixel classification (not a provenance verdict)")
console.print(f" Label: {result.label}")
console.print(f" Domain: {result.domain}")
console.print(f" Detector: {result.detector}")
console.print(f" Provider: {result.provider or 'none'}")
console.print(" identify is unchanged. This command does not run cleanup and does not prove a file is clean.")
# ── Combined "all" mode ──
@main.command("all")
@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path))
+206
View File
@@ -0,0 +1,206 @@
"""Pixel classify gate: Model 1 then gated Model 2, no downloads, no identify hook."""
from __future__ import annotations
import json
import sys
from pathlib import Path
from types import SimpleNamespace
import numpy as np
import pytest
from click.testing import CliRunner
from PIL import Image
from remove_ai_watermarks.classify import (
MLP_THRESHOLD,
PROVIDER_MARGIN,
RIDGE_THRESHOLD,
WEIGHTS_REPO,
WEIGHTS_REVISION,
classify_from_scores,
classify_pixels,
detector_level,
label_for,
provider_from_scores,
)
from remove_ai_watermarks.cli import main
def _scores(
*,
openai: float = 0.0,
google: float = 0.0,
tc260: float = 0.0,
meta: float = 0.0,
no_ai: float = 0.0,
) -> dict[str, float]:
return {
"openai": openai,
"google": google,
"tc260": tc260,
"meta_muse_image": meta,
"no_ai": no_ai,
}
def test_hub_snapshot_is_the_freeze_revision() -> None:
assert WEIGHTS_REPO == "wiltodelta/raiw-models"
assert WEIGHTS_REVISION == "c0ac82b6f1ae9fc0b92c467562282e9422da6da6"
def test_shipped_operating_point_matches_the_runtime_defaults() -> None:
payload = json.loads(Path("docs/photo-classify-hf/operating-point.json").read_text())
assert payload["model1"]["mlp_threshold"] == MLP_THRESHOLD
assert payload["model1"]["ridge_threshold"] == RIDGE_THRESHOLD
assert payload["model2"]["margin"] == PROVIDER_MARGIN
assert payload["model2"]["runs_only_after"] == "definitely"
def test_definitely_is_the_and_of_ridge_and_mlp() -> None:
assert detector_level(1.0, 10.0) == "definitely"
assert detector_level(0.0, 10.0) == "possibly"
assert detector_level(1.0, 0.0) == "possibly"
assert detector_level(0.0, 0.0) == "likely_human"
def test_only_definitely_is_ai() -> None:
assert label_for("definitely") == "ai"
assert label_for("possibly") == "unknown"
assert label_for("likely_human") == "human"
def test_provider_requires_margin_over_no_ai() -> None:
assert provider_from_scores(_scores(openai=1.0, no_ai=0.5)) == "openai"
assert provider_from_scores(_scores(openai=0.5, no_ai=0.4)) is None
assert provider_from_scores(_scores(google=2.0, openai=1.5, no_ai=0.0)) == "google"
assert provider_from_scores(_scores(meta=1.0, no_ai=0.0)) == "meta"
assert provider_from_scores(_scores(tc260=1.0, no_ai=0.0)) == "tc260"
def test_definitely_plus_openai_is_ai_openai() -> None:
result = classify_from_scores(1.0, 10.0, _scores(openai=1.0, no_ai=0.0))
assert result.label == "ai"
assert result.domain == "photo"
assert result.detector == "definitely"
assert result.provider == "openai"
assert result.to_dict()["provider"] == "openai"
def test_definitely_without_provider_scores_stays_ai_with_no_provider() -> None:
result = classify_from_scores(1.0, 10.0, None)
assert result.label == "ai"
assert result.provider is None
def test_definitely_plus_no_ai_head_has_no_provider() -> None:
result = classify_from_scores(1.0, 10.0, _scores(no_ai=2.0, openai=1.0))
assert result.label == "ai"
assert result.provider is None
def test_possibly_does_not_run_provider() -> None:
result = classify_from_scores(1.0, 0.0, _scores(openai=9.0, no_ai=0.0))
assert result.label == "unknown"
assert result.detector == "possibly"
assert result.provider is None
def test_likely_human_does_not_run_provider() -> None:
result = classify_from_scores(0.0, 0.0, _scores(openai=9.0, no_ai=0.0))
assert result.label == "human"
assert result.detector == "likely_human"
assert result.provider is None
@pytest.mark.parametrize("caller", ["identify", "has_invisible_target"])
def test_provenance_paths_do_not_import_classify(tmp_path: Path, caller: str) -> None:
path = tmp_path / "plain.png"
Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8)).save(path)
sys.modules.pop("remove_ai_watermarks.classify", None)
from remove_ai_watermarks.identify import has_invisible_target, identify
if caller == "identify":
report = identify(path, check_visible=False, check_invisible=False)
assert report.is_ai_generated is None
else:
assert has_invisible_target(path) is False
loaded = [name for name in sys.modules if name.startswith("remove_ai_watermarks.classify")]
assert loaded == []
def _plain_png(tmp_path: Path) -> Path:
path = tmp_path / "plain.png"
Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8)).save(path)
return path
def test_124d_is_not_extracted_unless_definitely(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
calls: list[int] = []
runtime = SimpleNamespace(
ridge_mean=np.zeros(768),
ridge_scale=np.ones(768),
ridge_weights=np.zeros(768),
ridge_threshold=RIDGE_THRESHOLD,
mlp_threshold=MLP_THRESHOLD,
provider_margin=PROVIDER_MARGIN,
)
monkeypatch.setattr("remove_ai_watermarks.classify.is_available", lambda: True)
monkeypatch.setattr("remove_ai_watermarks.classify._get_runtime", lambda device: runtime)
monkeypatch.setattr("remove_ai_watermarks.classify._embed", lambda *args, **kwargs: np.zeros(768))
monkeypatch.setattr("remove_ai_watermarks.classify._mlp_score", lambda *args, **kwargs: 0.0)
monkeypatch.setattr(
"remove_ai_watermarks.classify._forensic",
lambda image: calls.append(1) or np.zeros(124),
)
result = classify_pixels(_plain_png(tmp_path))
assert result.label == "human"
assert calls == []
def test_weight_files_may_sit_in_run1(tmp_path: Path) -> None:
from remove_ai_watermarks.classify import _find_weight
nested = tmp_path / "run1"
nested.mkdir()
target = nested / "detector.pt"
target.write_bytes(b"x")
assert _find_weight(tmp_path, "detector.pt") == target
assert _find_weight(tmp_path, "clip-l-ft.pt") is None
def test_operating_point_prefers_the_sidecar(tmp_path: Path) -> None:
from remove_ai_watermarks.classify import _operating_point
(tmp_path / "operating-point.json").write_text(
json.dumps({"model1": {"mlp_threshold": 1.25}, "model2": {"margin": 0.5}})
)
assert _operating_point(tmp_path) == (1.25, 0.5)
assert _operating_point(tmp_path / "missing") == (MLP_THRESHOLD, PROVIDER_MARGIN)
def test_missing_extra_raises_the_install_hint(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
import remove_ai_watermarks.classify as classify_mod
monkeypatch.setattr(classify_mod, "is_available", lambda: False)
monkeypatch.setattr(
classify_mod,
"_get_runtime",
lambda device: (_ for _ in ()).throw(AssertionError("must not load weights")),
)
path = tmp_path / "plain.png"
Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8)).save(path)
with pytest.raises(RuntimeError, match=r"remove-ai-watermarks\[classify\]"):
classify_mod.classify_pixels(path)
def test_cli_classify_without_extra_prints_the_install_hint(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
import remove_ai_watermarks.classify as classify_mod
monkeypatch.setattr(classify_mod, "is_available", lambda: False)
path = tmp_path / "plain.png"
Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8)).save(path)
result = CliRunner().invoke(main, ["classify", str(path)])
assert result.exit_code == 1
assert "remove-ai-watermarks[classify]" in result.output
assert "pip install" in result.output
+1 -1
View File
@@ -101,7 +101,7 @@ class TestApiReportsFailedWrite:
class TestDirectoryInputIsRejected:
@pytest.mark.parametrize("cmd", ["identify", "visible", "erase", "metadata", "invisible", "all"])
@pytest.mark.parametrize("cmd", ["identify", "classify", "visible", "erase", "metadata", "invisible", "all"])
def test_directory_as_source_is_a_clean_usage_error(self, tmp_path, cmd):
"""A directory must be refused by argument parsing, never reach the scanners."""
args = [cmd, str(tmp_path)]
+32
View File
@@ -0,0 +1,32 @@
"""124-d freeze extractor: shape and the undersize abstain, no downloads."""
from __future__ import annotations
import numpy as np
from remove_ai_watermarks._internal.forensic_124d import FEATURE_WIDTH, PATCH, image_features
def test_a_full_patch_returns_124_floats() -> None:
pixels = np.random.default_rng(0).integers(0, 256, (PATCH, PATCH, 3), dtype=np.uint8)
vector = image_features(pixels)
assert vector is not None
assert vector.shape == (FEATURE_WIDTH,)
assert vector.dtype == np.float64
def test_a_larger_image_still_returns_124_floats() -> None:
pixels = np.random.default_rng(1).integers(0, 256, (512, 640, 3), dtype=np.uint8)
vector = image_features(pixels)
assert vector is not None
assert vector.shape == (FEATURE_WIDTH,)
def test_an_undersized_image_returns_none() -> None:
pixels = np.zeros((PATCH - 1, PATCH, 3), dtype=np.uint8)
assert image_features(pixels) is None
def test_non_rgb_returns_none() -> None:
pixels = np.zeros((PATCH, PATCH), dtype=np.uint8)
assert image_features(pixels) is None
+6 -1
View File
@@ -71,10 +71,15 @@ def test_trustmark_is_limited_to_its_numpy_compatible_python_range():
def test_extras_use_capability_names_without_legacy_aliases():
extras = set(metadata("remove-ai-watermarks").get_all("Provides-Extra") or [])
assert {"pixels", "heif", "visible", "video", "detect", "diffusion"} <= extras
assert {"pixels", "heif", "visible", "video", "detect", "diffusion", "classify"} <= extras
assert {"gpu", "remove", "detect-pywavelets"}.isdisjoint(extras)
def test_classify_extra_owns_the_photo_heads() -> None:
names = _requirement_names("classify")
assert {"torch", "transformers", "huggingface-hub", "tokenizers"} <= names
def test_production_all_does_not_include_development_tools():
assert {
"pyright",
Generated
+16 -2
View File
@@ -3199,6 +3199,15 @@ all = [
{ name = "transformers" },
{ name = "trustmark", marker = "python_full_version < '3.13'" },
]
classify = [
{ name = "huggingface-hub" },
{ name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" },
{ name = "numpy", version = "2.3.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.13'" },
{ name = "opencv-python-headless" },
{ name = "tokenizers" },
{ name = "torch" },
{ name = "transformers" },
]
detect = [
{ name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" },
{ name = "numpy", version = "2.3.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.13'" },
@@ -3308,6 +3317,7 @@ requires-dist = [
{ name = "click", specifier = ">=8.0.0" },
{ name = "diffsynth", marker = "extra == 'qwen-zimage'", specifier = ">=2.0.17,<3" },
{ name = "diffusers", marker = "extra == 'diffusion'", specifier = ">=0.38.0" },
{ name = "huggingface-hub", marker = "extra == 'classify'", specifier = ">=0.20.0" },
{ name = "huggingface-hub", marker = "extra == 'lama'", specifier = ">=0.20.0" },
{ name = "huggingface-hub", marker = "extra == 'migan'", specifier = ">=0.20.0" },
{ name = "huggingface-hub", marker = "extra == 'text-draft'", specifier = ">=0.20.0" },
@@ -3331,24 +3341,28 @@ requires-dist = [
{ name = "pywavelets", marker = "extra == 'detect'", specifier = ">=1.1.1" },
{ name = "remove-ai-watermarks", extras = ["detect"], marker = "extra == 'dev'" },
{ name = "remove-ai-watermarks", extras = ["diffusion"], marker = "extra == 'qwen-zimage'" },
{ name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'classify'" },
{ name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'detect'" },
{ name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'diffusion'" },
{ name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'visible'" },
{ name = "remove-ai-watermarks", extras = ["qwen-zimage", "lama"], marker = "extra == 'text-restoration'" },
{ name = "remove-ai-watermarks", extras = ["video"], marker = "extra == 'dev'" },
{ name = "remove-ai-watermarks", extras = ["video", "heif", "detect", "trustmark", "text-restoration", "migan"], marker = "extra == 'all'" },
{ name = "remove-ai-watermarks", extras = ["video", "heif", "detect", "trustmark", "text-restoration", "migan", "classify"], marker = "extra == 'all'" },
{ name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'lama'" },
{ name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'migan'" },
{ name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'video'" },
{ name = "ruff", marker = "extra == 'dev'", specifier = ">=0.4.0" },
{ name = "safetensors", marker = "extra == 'diffusion'" },
{ name = "tokenizers", marker = "extra == 'classify'", specifier = ">=0.22,<0.23" },
{ name = "tokenizers", marker = "extra == 'diffusion'", specifier = ">=0.22,<0.23" },
{ name = "torch", marker = "extra == 'classify'", specifier = ">=2.0.0" },
{ name = "torch", marker = "extra == 'diffusion'", specifier = ">=2.0.0" },
{ name = "torchvision", marker = "extra == 'qwen-zimage'", specifier = ">=0.20.0" },
{ name = "transformers", marker = "extra == 'classify'", specifier = ">=5,<6" },
{ name = "transformers", marker = "extra == 'diffusion'", specifier = ">=5,<6" },
{ name = "trustmark", marker = "python_full_version < '3.13' and extra == 'trustmark'", specifier = ">=0.8.0" },
]
provides-extras = ["pixels", "heif", "visible", "video", "detect", "diffusion", "qwen-zimage", "text-restoration", "text-draft", "trustmark", "lama", "migan", "dev", "all"]
provides-extras = ["pixels", "heif", "visible", "video", "detect", "diffusion", "qwen-zimage", "text-restoration", "text-draft", "trustmark", "classify", "lama", "migan", "dev", "all"]
[[package]]
name = "requests"