diff --git a/.claude/rules/development.md b/.claude/rules/development.md index e47532f..4eda3e3 100644 --- a/.claude/rules/development.md +++ b/.claude/rules/development.md @@ -54,6 +54,9 @@ Do not classify an entire module as untestable because its main path downloads a stage's; `sdxl-zimage` shipped doing exactly that and crashed on every image with a face. When one profile inherits another's stage, guard the invariants that stage relies on, not just the code path. +- the photo-classify gate in `test_classify.py`: DEFINITELY is ridge AND MLP, + POSSIBLY does not emit a provider, and `identify` does not import + `remove_ai_watermarks.classify`. - the `InvisibleOptions` defaults, in `test_api.py`. When one signature promises to mirror another, compare them field by field rather than pinning the values you happen to know about, so the next field added on one side and not the other fails at the diff --git a/CLAUDE.md b/CLAUDE.md index db7668e..e14d482 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,7 +39,7 @@ Before a release, read [`docs/release-and-distribution.md`](docs/release-and-dis [`docs/module-internals.md`](docs/module-internals.md) is the canonical per-module map, including design decisions, thresholds, calibration history, incident records, and regression guards. Read the relevant section before changing a subsystem. -Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). +Research and current constraints are routed through [`docs/index.md`](docs/index.md), especially [`docs/known-limitations.md`](docs/known-limitations.md), [`docs/supported-signals.md`](docs/supported-signals.md), [`docs/synthid.md`](docs/synthid.md), and [`docs/watermarking-landscape.md`](docs/watermarking-landscape.md). Pixel photo classification is [`docs/photo-classify.md`](docs/photo-classify.md); it is not `identify`. Classifier research is split between general [`AI-generated image classifiers`](docs/ai-generated-image-classifiers.md) and [`SynthID source classifiers`](docs/synthid-classifiers.md). Other SynthID campaign logs are [`docs/synthid-detector-research.md`](docs/synthid-detector-research.md) and [`docs/synthid-removal-research.md`](docs/synthid-removal-research.md). ## Data safety diff --git a/README.md b/README.md index 54a3ea9..8e007e6 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,7 @@ removal. | Goal | Command | GPU | | --- | --- | --- | | Find provenance signals and watermarks | `identify` | No | +| Classify a photograph from pixels (opt-in, not provenance) | `classify` | No | | Remove known visible AI marks | `visible` | No | | Erase a region you select | `erase` | No | | Strip AI metadata | `metadata` | No | @@ -54,6 +55,7 @@ its linked C2PA manifest; metadata stripping alone removes only the manifest. | Need | Install | | --- | --- | | Metadata inspection and stripping | `remove-ai-watermarks` | +| Photograph AI-versus-camera classification | `remove-ai-watermarks[classify]` | | Visible detection and removal | `remove-ai-watermarks[visible]` | | Visible video processing | `remove-ai-watermarks[video]` | | Video SynthID removal | `remove-ai-watermarks[video,diffusion]` | @@ -80,6 +82,16 @@ Inspect an image: remove-ai-watermarks identify image.png ``` +To classify a photograph from pixels (AI versus camera, optional provider), +install the extra and call `classify`. `identify` never starts it: + +```bash +uv tool install --force "remove-ai-watermarks[classify]" +remove-ai-watermarks classify image.png +``` + +Guide: [photo pixel classification](docs/photo-classify.md). + For visible watermark removal, install the pixel dependencies: ```bash diff --git a/docs/ai-generated-image-classifiers.md b/docs/ai-generated-image-classifiers.md index 5aaec2f..8377997 100644 --- a/docs/ai-generated-image-classifiers.md +++ b/docs/ai-generated-image-classifiers.md @@ -2,7 +2,10 @@ > Research archive for metadata-free `ai_generated` versus > `not_ai_generated` classifiers. These are not SynthID detectors and are not -> shipped product verdicts. Current behavior: [supported signals](supported-signals.md) +> `identify` verdicts. The 2026-08-31 freeze is consumed only by explicit +> `classify` / `classify_pixels`. Current behavior: +> [photo pixel classification](photo-classify.md), +> [supported signals](supported-signals.md), > and [known limitations](known-limitations.md). > > Sister pages: [SynthID source classifiers](synthid-classifiers.md), @@ -18,8 +21,11 @@ origin-locked residual bank are different features for different jobs. ## Frozen result, 2026-08-31 The freeze is **AI versus camera**, not a universal detector, and it is not -wired into `identify`. Detector: CLIP-L-ft ridge AND freeze MLP. -Provider: 124-d focal heads, only after DEFINITELY or POSSIBLY. +wired into `identify`. The library consumes it only through explicit +`classify_pixels` / CLI `classify`. Detector: CLIP-L-ft ridge AND freeze MLP. +Provider: 124-d focal heads, only after DEFINITELY. Library guide: +[photo-classify.md](photo-classify.md). Hub card: +[photo-classify-hf/README.md](photo-classify-hf/README.md). | Cell | Value | | --- | ---: | diff --git a/docs/cli.md b/docs/cli.md index be1796c..db9ebdd 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -14,6 +14,7 @@ defaults. This page focuses on choosing the right command. | Command or signal | Required installation | | --- | --- | | `metadata` and metadata-only `identify` | Default package | +| `classify` | `remove-ai-watermarks[classify]` | | Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) | | Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` | | Adobe TrustMark signals in `identify` on Python 3.11-3.12 | `remove-ai-watermarks[trustmark]` | @@ -65,6 +66,37 @@ remove-ai-watermarks identify image.png --no-visible Despite the historical option name, `--no-visible` skips both visible and open invisible pixel detectors. Metadata inspection still runs. +## Classify a photograph from pixels + +This is not provenance. `identify` never starts it, including after a +no-signal metadata scan. Install the extra first: + +```bash +uv tool install --force "remove-ai-watermarks[classify]" +``` + +Then: + +```bash +remove-ai-watermarks classify image.png +remove-ai-watermarks classify image.png --json +``` + +The command runs the frozen photo detector (CLIP-L-ft ridge AND freeze MLP). +Only a DEFINITELY result is reported as `ai`. On that result only, the 124-d +provider head may return `openai`, `google`, `meta`, or `tc260`. POSSIBLY is +`unknown`. Camera-like photographs are `human`. The call does not run cleanup +and does not set `is_ai_generated`. + +The contract is AI versus camera. Receipts, UI, and digital art are out of +scope. Microsoft is not a pixel class: a DALL-E Bing image scores as `openai`, +an Imagen Designer image as `google`. Weights download on first use from +[`wiltodelta/raiw-models`](https://huggingface.co/wiltodelta/raiw-models), +or from `RAIW_CLASSIFY_WEIGHTS` if that directory already holds the freeze +files. +Guide: [photo pixel classification](photo-classify.md). Hub card: +[photo-classify-hf/README.md](photo-classify-hf/README.md). + ## Remove known visible marks Install `remove-ai-watermarks[visible]` before using `visible` or `erase`. diff --git a/docs/index.md b/docs/index.md index 46df068..ed34af7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -12,6 +12,8 @@ to run the tool. Use the maintainer references only when changing the code. | [Python API](python-api.md) | You want to call the package from Python. | | [Supported signals](supported-signals.md) | You need to know which visible marks, metadata formats, and invisible signals are covered. | | [Known limitations](known-limitations.md) | You need the quality, device, format, or verification boundaries. | +| [Photo pixel classification](photo-classify.md) | You want `classify` / `classify_pixels`: AI versus camera, optional provider. | +| [Photo classifier training](photo-classify-training.md) | You need the retrain pack, public sources, and what is not published. | | [Scope, safety, and legal notes](legal-and-safety.md) | You need the intended use and legal context. | ## Maintainer references @@ -35,7 +37,7 @@ The current behavior is defined by the code, tests, README, and user guides. - [ControlNet removal research](controlnet-removal-pipeline-research.md) - [Qwen improvement research](qwen-improvement-research.md) - [Doubao reverse-alpha research](research-doubao-distillation.md) -- [AI-generated image classifiers](ai-generated-image-classifiers.md) (photo AI-versus-camera freeze, not a product verdict) +- [AI-generated image classifiers](ai-generated-image-classifiers.md) (photo AI-versus-camera freeze consumed by `classify_pixels`, not `identify`) - [Photo classifier Hugging Face card](photo-classify-hf/README.md) (Hub publication text of the 2026-08-31 freeze) - [SynthID source classifiers](synthid-classifiers.md) - [SynthID local detector research](synthid-detector-research.md) diff --git a/docs/installation.md b/docs/installation.md index 10dff15..3fbbd04 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -112,6 +112,7 @@ application actually uses: | `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No | | `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No | | `trustmark` | Adobe TrustMark detection on Python 3.11-3.12 | trustmark | Yes | +| `classify` | Metadata-free photo AI-versus-camera classifier plus gated provider | `pixels`, Torch, Transformers | Yes | | `diffusion` | Torch and Diffusers runtime; video SynthID regeneration | `pixels`, Torch, Diffusers | Yes | | `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch | | `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch | @@ -137,6 +138,7 @@ flowchart LR draft["text-draft"] heif trustmark + classify --> pixels ``` `heif`, `trustmark`, and `text-draft` are independent branches. Combine them diff --git a/docs/known-limitations.md b/docs/known-limitations.md index f1b9cd1..35bc453 100644 --- a/docs/known-limitations.md +++ b/docs/known-limitations.md @@ -4,6 +4,18 @@ This page describes current product limits. Historical measurements and superseded experiments live in the research archive listed in [the documentation index](index.md). +## Pixel classification + +`classify` / `classify_pixels` is AI versus camera, not a universal AI detector. +Receipts, UI screenshots, and digital art are out of contract: a thermal receipt +can score as AI, a photorealistic AI receipt as human. POSSIBLY is reported as +`unknown`. Provider attribution runs only after DEFINITELY and can abstain when +124-d extraction refuses the file. Microsoft is not a pixel class. + +The command is never started by `identify`. A no-signal provenance result stays +unknown. This is not a clean verdict and it does not run cleanup. Full guide: +[photo pixel classification](photo-classify.md). + ## Visible removal ### Fill quality depends on the background diff --git a/docs/module-internals.md b/docs/module-internals.md index 83f771a..12d38eb 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -11,7 +11,8 @@ updated in the same change. ## Architecture -The package has four main paths: +The package has four main paths, plus an opt-in pixel classifier that is not +on the provenance or removal graph: ```mermaid flowchart LR @@ -19,13 +20,18 @@ flowchart LR Input --> Visible[Visible mark removal] Input --> Invisible[Diffusion regeneration] Input --> Metadata[Metadata stripping] + Input --> Classify[classify_pixels] Identify --> Report[ProvenanceReport] Visible --> VisibleOutput[Localized and filled image] Invisible --> InvisibleOutput[Regenerated image] Metadata --> MetadataOutput[Container with AI metadata removed] + Classify --> PixelClassification ``` +`identify` does not call `classify_pixels`. `has_invisible_target` and `all` +do not read it. + The `all` command runs visible removal, optional invisible regeneration, and metadata stripping in that order. @@ -576,6 +582,20 @@ provenance normalizer. Integration code publishes the strict `ProvenanceReport.to_dict()` alongside it rather than letting operational fields or derived results influence detection. +### Photo pixel classifier + +[`classify.py`](../src/remove_ai_watermarks/classify.py) is the 2026-08-31 +photo freeze: CLIP-L-ft ridge AND freeze MLP, then 124-d focal heads only on +DEFINITELY. The public label is `ai` / `human` / `unknown`. POSSIBLY is +`unknown`. Provider is `None` unless the label is `ai`. Tests in +`tests/test_classify.py` pin the gate without downloads and pin that +`identify` does not import this module. + +Weights stay out of git. The Hub snapshot is `wiltodelta/raiw-models`. +`RAIW_CLASSIFY_WEIGHTS` overrides it. The extra is `classify`. User guide: +[photo-classify.md](photo-classify.md). Hub card: +[photo-classify-hf/README.md](photo-classify-hf/README.md). + ### Pixel forensics [`pixel_evidence.py`](../src/remove_ai_watermarks/pixel_evidence.py) measures six diff --git a/docs/photo-classify-training.md b/docs/photo-classify-training.md new file mode 100644 index 0000000..0fa2edb --- /dev/null +++ b/docs/photo-classify-training.md @@ -0,0 +1,86 @@ +# Photo classifier training and data + +How the 2026-08-31 freeze was fit, what a retrain needs, and where those +files come from. Inference is [photo-classify.md](photo-classify.md). The +campaign log is [ai-generated-image-classifiers.md](ai-generated-image-classifiers.md). + +The shipped package does not include the catalog or the embeddings. Fitting +the heads again uses the cache pack. The original pixels still have to be +kept: a missing embedding, a new backbone, or an audit all need the files. + +## Retrain pack (caches) + +CPU retrain of both heads needs sha256-keyed matrices and the catalog of +roles. That step does not open JPEG or PNG files. It is not permission to +drop the pixels. + +| File | Role | +| --- | --- | +| `catalog.json` | 32,690 rows: `sha256`, `roles`, optional `family` / public `source` | +| `embeddings-by-sha256.npz` | CLIP-L-ft 768-d vectors for Model 1 | +| `extra-clip-l.npz` | CLIP-L extras for the 150 Flickr UI hashes | +| `features-124d.npz` | 124-d residual vectors for Model 2 | +| `probe-weights-clip-l-ft.npz` | Ridge mean, scale, weights, `thr_oi_1pct` | +| `meta-split.json` | Meta train 217, hold-out v2 79, hold-out v3 198 | + +Run: + +```bash +uv run python scripts/retrain_photo_classify.py --pack DIR --out DIR +``` + +Seeds are fixed: detector `20260940`, provider `20260956`. Two consecutive +runs must be byte-identical. + +Not in the **published** retrain pack: raw photographs, COCO/Open Images/Kodak +binaries, CORD, Danbooru. Those files stay in their public corpora or in the +private freeze archive. Pixel rebuild of a missing embedding still needs +`clip-l-ft.pt` and the original file. + +## Where the rows come from + +Identity is always sha256 of the file bytes. + +### Public, named corpora + +| Role | Source | +| --- | --- | +| `detector_photo_train`, `detector_photo_coco_hold` | COCO 2017 validation stills | +| `detector_open_images_*`, `detector_photo_dev_oi` | Open Images photographs. Locked fresh never enters train | +| `detector_photo_kodak` | Kodak lossless true color, 24 files | +| `detector_photo_picsum_eval` | Picsum | +| `detector_flux_hf_hold` | Hugging Face FLUX stills, hold-out | +| Flickr UI cell (150) | Openverse `license_type=all-cc`, host `live.staticflickr.com` only | +| Meta hold-out v3 (198) | Meta Muse Image API, eval-only, not in train | + +### In-house labeled photographs, not redistributed + +Provider OpenAI, Google, and TC260 train and test rows, and most detector AI +train and test rows, are labeled photographic generations held privately. +The published pack carries only their sha256, roles, and precomputed vectors. +It does not carry the pixels. + +### Out of the freeze train + +CORD receipts, FUNSD forms, Danbooru, charts, and map tiles were measured as +failure domains. They are not Model 1 train. A future universal router treats +them as other domains, not as extra CLIP-L negatives. + +## What the freeze script does + +Historically `freeze_both_models.py` built the catalog from local paths, then +trained. That campaign code is private and is not in this repository. The +public entry is `scripts/retrain_photo_classify.py`, which skips catalog +construction and fits the same heads from the pack above. + +1. Load CLIP-L-ft vectors by sha256. Fit the 768-512-128-1 MLP on + `detector_ai_train` plus extras versus `detector_photo_train` plus struct. + Threshold is the 1.67% quantile on `detector_photo_dev_oi`. +2. AND that MLP with the frozen ridge in the probe file. DEFINITELY is both. +3. Load 124-d vectors. Fit one-vs-rest focal heads for openai, google, tc260, + meta_muse_image, no_ai. Margin 0.30. +4. Repeat with the same seeds. Compare bytes. + +CLIP-L fine-tune of the last two vision blocks is earlier, seed 20260822, and +is not repeated by the retrain script. The backbone file `clip-l-ft.pt` is an +inference and embedding dependency, not a retrain-pack input. diff --git a/docs/photo-classify.md b/docs/photo-classify.md new file mode 100644 index 0000000..4b61c31 --- /dev/null +++ b/docs/photo-classify.md @@ -0,0 +1,122 @@ +# Photo pixel classification + +`classify` gives a pixel opinion on a **photograph**: camera-like, generated, +or unknown, and optionally which renderer it resembles. It is a separate +command from `identify`. Metadata inspection never starts it. + +This page is the library guide. The Hugging Face model card is the same freeze +written for Hub publication: [photo-classify-hf/README.md](photo-classify-hf/README.md). +Campaign notes and rejected variants live in +[AI-generated image classifiers](ai-generated-image-classifiers.md). + +## When to use it + +Use `identify` for C2PA, IPTC, visible marks, and other provenance. Use +`classify` when those signals are gone and the file is still a photograph. + +Do not use `classify` to prove a file is clean. Do not use it on receipts, UI, +or digital art. Do not treat `provider=openai` as a decoded SynthID payload. + +## Install and run + +```bash +uv tool install --force "remove-ai-watermarks[classify]" +remove-ai-watermarks classify image.png +remove-ai-watermarks classify image.png --json +``` + +```python +from pathlib import Path +from remove_ai_watermarks.classify import classify_pixels + +result = classify_pixels(Path("image.png")) +print(result.label, result.detector, result.provider) +``` + +Missing extra raises `RuntimeError` with +`pip install 'remove-ai-watermarks[classify]'`. `device` is a library +parameter (`None` / `"auto"` / `"cpu"` / `"cuda"`), not a CLI option. + +Weights are not in git. First call downloads +[`wiltodelta/raiw-models`](https://huggingface.co/wiltodelta/raiw-models) +at the freeze revision pinned in `classify.py`, or reads +`RAIW_CLASSIFY_WEIGHTS` if that directory already has `clip-l-ft.pt`, +`probe-weights-clip-l-ft.npz`, `detector.pt`, and `provider.pt`. The training +catalog is not published with the package. + +## What one call returns + +One request runs both heads. Model 2 runs only after Model 1 is DEFINITELY. + +```mermaid +flowchart TD + image[Photograph] --> m1[Model 1 CLIP-L-ft ridge AND MLP] + m1 -->|likely_human| human["label human, provider none"] + m1 -->|possibly| unk["label unknown, provider none"] + m1 -->|definitely| m2[Model 2 124-d focal heads] + m2 -->|openai google meta tc260| named["label ai plus provider"] + m2 -->|no_ai or extract fail| aiOnly["label ai, provider none"] +``` + +| Field | Values | Meaning | +| --- | --- | --- | +| `label` | `ai`, `human`, `unknown` | Public verdict. `ai` only on DEFINITELY | +| `domain` | `photo` | This freeze is photographic only | +| `detector` | `definitely`, `possibly`, `likely_human` | Raw Model 1 gate | +| `provider` | `openai`, `google`, `meta`, `tc260`, or `None` | Model 2, only if `label` is `ai` | + +`human` is camera-like under this contract, not a proof of authorship. +`unknown` is POSSIBLY, or a domain this freeze does not support. `provider=None` +on an `ai` file is often FLUX or another generator that is not one of the four +heads, or a 124-d extract refusal. + +## Approach + +CLIP-L content embeddings separate generated photographs from camera +photographs. They do not separate generated photographs from receipts or UI. +A ridge on fine-tuned CLIP-L (last two vision blocks, 224 letterbox) is Model +1. A small MLP on the same vectors, ANDed with the ridge, is the freeze +DEFINITELY gate. + +Provider attribution uses a different feature: 124-d residual ratios on 256 px +patches. OpenAI versus Gemini is a strong split there. "AI or not" is the +wrong job on that bank, which is why Model 2 is gated. + +Provider names the renderer, not the product UI. Bing Image Creator signed +Microsoft, OpenAI scores `openai`. Designer signed Microsoft, Google LLC +scores `google`. There is no Microsoft pixel class. Meta is Muse Image, not +an Instagram `made_with_ai` tag. + +## Freeze 2026-08-31 + +Two CPU retrains were byte-identical. DEFINITELY is the shipped cut. + +| Head | Cell | Result | +| --- | --- | ---: | +| Detector DEFINITELY | AI-test | 92.4% (n=1,847) | +| Detector DEFINITELY | Open Images fresh | 1.3% (n=3,000) | +| Detector DEFINITELY | Kodak | 0/24 | +| Detector DEFINITELY | FLUX hold | 83.0% (n=300) | +| Provider | OpenAI | 90.8% (345/380 of 381) | +| Provider | Google | 90.9% (339/373 of 377) | +| Provider | TC260 | 78.6% (298/379 of 384) | +| Provider | Meta hold-out v3 | 89.4% (177/198) | +| Provider | Meta hold-out pooled | 88.4% (243/275 listed 277) | +| Provider | meme templates, ungated | 29.1% leak | + +The ungated meme row is why Model 2 never runs on `human` or `unknown`. + +## Limits + +- Receipts, UI, scans, maps, and community art are out of contract. +- Not SynthID, not C2PA, not `is_ai_generated`. +- Images under 256 px cannot yield 124-d features, so provider abstains. +- FLUX is a hold-out at 83% DEF, not a named provider class. + +`identify`, `has_invisible_target`, `all`, and `invisible` do not import this +module. A no-signal provenance result stays unknown until you call `classify` +yourself. + +Training data, the retrain pack (caches only, no images), and +`scripts/retrain_photo_classify.py` are in +[photo-classify-training.md](photo-classify-training.md). diff --git a/docs/python-api.md b/docs/python-api.md index 2c56dd3..5962a6d 100644 --- a/docs/python-api.md +++ b/docs/python-api.md @@ -6,7 +6,7 @@ and pipeline modules are intended for maintainers and specialized workflows. Dependency groups are identical for the CLI and Python API. The default install covers metadata extraction, normalization, verdict logic, and stripping. Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection -uses `detect`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and +uses `detect`; pixel photo classification uses `classify`; invisible image removal uses `qwen-zimage` and an NVIDIA GPU; and visible video processing uses `video`. Video SynthID removal is a separate VAE path that still runs on CPU and combines `video` and `diffusion`. Add `heif` independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See @@ -144,6 +144,33 @@ print(summary.invisible_unavailable) # outputs that still carry the watermark `mode` is `all`, `visible`, `invisible`, or `metadata`. Pass a constructed `InvisibleEngine` as `engine` to load the model once for the whole directory. +## Classify a photograph from pixels + +This is not provenance. `identify` does not call it. Install +`remove-ai-watermarks[classify]` first. + +```python +from pathlib import Path + +from remove_ai_watermarks.classify import classify_pixels + +result = classify_pixels(Path("input.png")) +print(result.label, result.detector, result.provider) +``` + +`label` is `ai` only on a DEFINITELY detector result (ridge AND freeze MLP). +POSSIBLY is `unknown`. Camera-like photographs are `human`. `provider` is +`openai`, `google`, `meta`, or `tc260` only when `label` is `ai` and the 124-d +head beats `no_ai` by the freeze margin. Otherwise it is `None`, including +when 124-d extraction refuses the file. + +`device` is a library parameter: `None` / `"auto"` detect, `"cpu"` or `"cuda"` +pin. It is not a CLI option. + +Missing extra raises `RuntimeError` with the quoted install command +`'remove-ai-watermarks[classify]'`. Guide: +[photo pixel classification](photo-classify.md). + ## Inspect provenance The default installation evaluates file metadata. Add `visible`, `detect`, or diff --git a/docs/release-and-distribution.md b/docs/release-and-distribution.md index e96e033..e69eeb0 100644 --- a/docs/release-and-distribution.md +++ b/docs/release-and-distribution.md @@ -74,8 +74,10 @@ pin. Its callback smoke tests run in that repository's CI on every push; they are the scenario check for the demo surface. The Hugging Face **model** `wiltodelta/raiw-models` is not part of that -release fan-out. It holds the photo-classify freeze weights. Update it only -with `.github/workflows/publish-photo-classify-hf.yml` (`workflow_dispatch`, +release fan-out. It holds the photo-classify freeze weights. The library extra +`classify` downloads that snapshot on first use, or reads +`RAIW_CLASSIFY_WEIGHTS`. Update the Hub repo only with +`.github/workflows/publish-photo-classify-hf.yml` (`workflow_dispatch`, type `publish`). Full mode downloads the four weight files from the GitHub Release tag `photo-classify-freeze-2026-08-31` (or the tag you pass) and uploads them with the card in `docs/photo-classify-hf/`. Card mode updates diff --git a/pyproject.toml b/pyproject.toml index 024520c..91cf5a0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -142,6 +142,15 @@ trustmark = [ # wheels. Keep this feature installable only on its real runtime range. "trustmark>=0.8.0; python_version < '3.13'", ] +# Metadata-free photo classifier (Model 1 then gated Model 2). CPU torch is +# enough. identify never imports this extra. Weights download on first use. +classify = [ + "remove-ai-watermarks[pixels]", + "torch>=2.0.0", + "transformers>=5,<6", + "tokenizers>=0.22,<0.23", + "huggingface-hub>=0.20.0", +] # Universal region eraser backend -- big-LaMa via onnxruntime (Carve/LaMa-ONNX, # Apache-2.0). CPU, no torch. Model (~200 MB) is downloaded on first use and # cached by huggingface_hub; it is never bundled in this repo. The default cv2 @@ -174,7 +183,7 @@ dev = [ ] # ``qwen-zimage`` already pulls ``diffusion``; naming both would suggest diffusion is # independently sufficient for a removal, which it is not. -all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan]"] +all = ["remove-ai-watermarks[video,heif,detect,trustmark,text-restoration,migan,classify]"] [project.scripts] remove-ai-watermarks = "remove_ai_watermarks.cli:main" diff --git a/scripts/README.md b/scripts/README.md index 26764d0..d8abf02 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -43,6 +43,7 @@ explicitly names a tracked canonical result. | `visible_positives.py` | List corpus images carrying a registered visible mark. | | `visible_recall_sample.py` | Build an unbiased blinded sample for recall measurement. | | `visible_sheets.py` | Build blinded contact sheets for relaxation candidates. | +| `retrain_photo_classify.py` | CPU-retrain the 2026-08-31 photo heads from a sha256-keyed cache pack, no images. | | `publish_photo_classify_hf.py` | Upload the photo-classify card and freeze weights to Hugging Face `wiltodelta/raiw-models`. Manual; the Action `publish-photo-classify-hf.yml` is the write-token path. | ## Research and diagnostic prototypes diff --git a/scripts/retrain_photo_classify.py b/scripts/retrain_photo_classify.py new file mode 100644 index 0000000..d6e15f1 --- /dev/null +++ b/scripts/retrain_photo_classify.py @@ -0,0 +1,198 @@ +#!/usr/bin/env python3 +"""Retrain the 2026-08-31 photo heads from a sha256-keyed cache pack. + +No image I/O. Requires catalog.json, embeddings-by-sha256.npz or extra-clip-l.npz +plus embeddings keyed by sha256, features-124d.npz, and probe-weights-clip-l-ft.npz. + + uv run python scripts/retrain_photo_classify.py --pack DIR --out DIR +""" + +from __future__ import annotations + +import argparse +import json +import logging +from pathlib import Path + +import numpy as np +import torch +from torch import nn + +log = logging.getLogger(__name__) + +DETECTOR_SEED = 20260940 +PROVIDER_SEED = 20260956 +EPOCHS = 40 +MLP_WIDTH = 512 +LR = 5e-4 +BATCH = 64 +TARGET_PHOTO_FPR = 0.0167 +FOCAL_GAMMA = 2.0 +FOCAL_EPOCHS = 40 +FOCAL_LR = 1e-3 +FOCAL_BATCH = 64 +LABELS = ("openai", "google", "tc260", "meta_muse_image", "no_ai") + + +def _load_sha_vectors(path: Path) -> dict[str, np.ndarray]: + payload = np.load(path, allow_pickle=True) + if "sha256" in payload.files: + keys = payload["sha256"] + vectors = payload["vectors"] + return {str(key): np.asarray(vector) for key, vector in zip(keys, vectors, strict=True)} + raise SystemExit(f"{path} must contain sha256 and vectors") + + +def _mlp(dim: int) -> nn.Module: + return nn.Sequential( + nn.Linear(dim, MLP_WIDTH), + nn.ReLU(), + nn.Dropout(0.3), + nn.Linear(MLP_WIDTH, MLP_WIDTH // 4), + nn.ReLU(), + nn.Dropout(0.1), + nn.Linear(MLP_WIDTH // 4, 1), + ) + + +def _provider_head() -> nn.Module: + return nn.Sequential( + nn.Linear(124, 64), + nn.ReLU(), + nn.Dropout(0.2), + nn.Linear(64, 1), + ) + + +def _focal_loss(logits: torch.Tensor, targets: torch.Tensor) -> torch.Tensor: + bce = nn.functional.binary_cross_entropy_with_logits(logits, targets, reduction="none") + prob = torch.sigmoid(logits) + pt = torch.where(targets > 0.5, prob, 1.0 - prob) + return ((1.0 - pt) ** FOCAL_GAMMA * bce).mean() + + +def _train_detector(emb: dict[str, np.ndarray], catalog: dict) -> tuple[nn.Module, float]: + torch.manual_seed(DETECTOR_SEED) + roles = {row["sha256"]: set(row["roles"]) for row in catalog["rows"]} + pos, neg = [], [] + for sha, vector in emb.items(): + role = roles.get(sha, set()) + if "detector_ai_train" in role or "detector_ai_extra" in role: + pos.append(vector) + if "detector_photo_train" in role or "detector_struct" in role: + neg.append(vector) + x_pos = torch.from_numpy(np.stack(pos)).float() + x_neg = torch.from_numpy(np.stack(neg)).float() + log.info("detector train pos=%d neg=%d", len(x_pos), len(x_neg)) + model = _mlp(x_pos.shape[1]) + optimizer = torch.optim.AdamW(model.parameters(), lr=LR, weight_decay=1e-3) + loss_fn = nn.BCEWithLogitsLoss() + rng = np.random.default_rng(DETECTOR_SEED) + model.train() + for _epoch in range(EPOCHS): + pp = rng.permutation(len(x_pos)) + pn = rng.permutation(len(x_neg)) + for start in range(0, min(len(pp), len(pn)), BATCH): + features = torch.cat([x_pos[pp[start : start + BATCH]], x_neg[pn[start : start + BATCH]]]) + labels = torch.cat([torch.ones(min(BATCH, len(pp) - start)), torch.zeros(min(BATCH, len(pn) - start))]) + optimizer.zero_grad() + loss_fn(model(features).squeeze(-1), labels).backward() + optimizer.step() + model.eval() + dev = np.stack( + [ + emb[row["sha256"]] + for row in catalog["rows"] + if "detector_photo_dev_oi" in row["roles"] and row["sha256"] in emb + ] + ) + with torch.inference_mode(): + scores = model(torch.from_numpy(dev).float()).squeeze(-1).numpy() + threshold = float(np.quantile(scores, 1.0 - TARGET_PHOTO_FPR)) + return model, threshold + + +def _train_provider(feat: dict[str, np.ndarray], catalog: dict) -> dict[str, nn.Module]: + def stack(role: str) -> np.ndarray: + vecs = [feat[row["sha256"]] for row in catalog["rows"] if role in row["roles"] and row["sha256"] in feat] + return np.stack(vecs) if vecs else np.empty((0, 124), dtype=np.float32) + + extra = stack("provider_openai_extra") + openai = stack("provider_openai") + bank = { + "openai": np.concatenate([openai, extra]) if len(extra) else openai, + "google": stack("provider_google"), + "tc260": stack("provider_tc260"), + "meta_muse_image": stack("provider_meta_train"), + "no_ai": stack("provider_no_ai"), + } + struct = stack("provider_struct") + all_neg = np.concatenate([bank["no_ai"], struct]) if len(struct) else bank["no_ai"] + heads = {} + for i, name in enumerate(LABELS): + if name == "no_ai": + positive = all_neg + negative = np.concatenate([bank[n] for n in LABELS if n != "no_ai"]) + else: + negative = np.concatenate([bank[n] for n in LABELS if n not in (name, "no_ai")] + [all_neg]) + positive = bank[name] + torch.manual_seed(PROVIDER_SEED + i) + rng = np.random.default_rng(PROVIDER_SEED + i) + max_neg = min(len(negative), len(positive) * 2) + neg_idx = rng.permutation(len(negative))[:max_neg] + x_pos = torch.from_numpy(positive.astype(np.float32)) + x_neg = torch.from_numpy(negative[neg_idx].astype(np.float32)) + model = _provider_head() + optimizer = torch.optim.AdamW(model.parameters(), lr=FOCAL_LR, weight_decay=1e-3) + model.train() + for _epoch in range(FOCAL_EPOCHS): + pos_idx = rng.permutation(len(x_pos)) + neg_order = rng.permutation(len(x_neg)) + for start in range(0, min(len(pos_idx), len(neg_order)), FOCAL_BATCH): + features = torch.cat( + [x_pos[pos_idx[start : start + FOCAL_BATCH]], x_neg[neg_order[start : start + FOCAL_BATCH]]] + ) + labels = torch.cat( + [ + torch.ones(min(FOCAL_BATCH, len(pos_idx) - start)), + torch.zeros(min(FOCAL_BATCH, len(neg_order) - start)), + ] + ) + optimizer.zero_grad() + _focal_loss(model(features).squeeze(-1), labels).backward() + optimizer.step() + heads[name] = model.eval() + log.info("provider %s pos=%d", name, len(positive)) + return heads + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--pack", type=Path, required=True) + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args() + logging.basicConfig(level=logging.INFO, format="%(message)s") + pack = args.pack.expanduser().resolve() + catalog = json.loads((pack / "catalog.json").read_text()) + emb: dict[str, np.ndarray] = {} + for name in ("embeddings-by-sha256.npz", "extra-clip-l.npz"): + path = pack / name + if path.is_file(): + emb.update(_load_sha_vectors(path)) + feat = _load_sha_vectors(pack / "features-124d.npz") + probe = np.load(pack / "probe-weights-clip-l-ft.npz") + log.info("catalog=%d embeddings=%d features=%d", catalog["counts"]["files"], len(emb), len(feat)) + args.out.mkdir(parents=True, exist_ok=True) + detector, threshold = _train_detector(emb, catalog) + torch.save(detector.state_dict(), args.out / "detector.pt") + heads = _train_provider(feat, catalog) + torch.save({name: head.state_dict() for name, head in heads.items()}, args.out / "provider.pt") + (args.out / "retrain.json").write_text( + json.dumps({"detector_threshold": threshold, "probe_thr": float(probe["thr_oi_1pct"])}, indent=2) + "\n" + ) + log.info("wrote %s threshold=%.6f", args.out, threshold) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/remove_ai_watermarks/_internal/clip_l_ft.py b/src/remove_ai_watermarks/_internal/clip_l_ft.py new file mode 100644 index 0000000..f3047b5 --- /dev/null +++ b/src/remove_ai_watermarks/_internal/clip_l_ft.py @@ -0,0 +1,86 @@ +# pyright: reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false, reportMissingTypeStubs=false +"""CLIP-L letterbox embedder for the frozen photo detector. + +The freeze fine-tuned the last two vision blocks of +``openai/clip-vit-large-patch14``. Inference letterboxes to 224 with pad +``(123, 117, 104)`` and L2-normalizes ``get_image_features``. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Any + +from PIL import Image +from torch import nn + +if TYPE_CHECKING: + from pathlib import Path + + import torch + from numpy.typing import NDArray + +BACKBONE = "openai/clip-vit-large-patch14" +SIZE = 224 +PAD_RGB = (123, 117, 104) + + +def letterbox(image: Image.Image, size: int = SIZE) -> Image.Image: + """Return a square RGB letterbox matching the freeze preprocessor.""" + image = image.convert("RGB") + width, height = image.size + scale = size / max(width, height) + new_w = max(1, round(width * scale)) + new_h = max(1, round(height * scale)) + resized = image.resize((new_w, new_h), Image.Resampling.BICUBIC) + canvas = Image.new("RGB", (size, size), PAD_RGB) + canvas.paste(resized, ((size - new_w) // 2, (size - new_h) // 2)) + return canvas + + +class HeadedCLIP(nn.Module): + """Checkpoint layout of ``clip-l-ft.pt``: CLIP plus a unused linear head.""" + + def __init__(self, clip: Any) -> None: + super().__init__() + self.clip = clip + self.head = nn.Linear(clip.config.projection_dim, 1) + + def embed(self, pixel_values: Any) -> Any: + outputs = self.clip.get_image_features(pixel_values=pixel_values) + vectors = outputs.pooler_output if hasattr(outputs, "pooler_output") else outputs + return nn.functional.normalize(vectors, dim=-1) + + +def load_headed_clip(checkpoint: Path, device: torch.device) -> HeadedCLIP: + """Build CLIP-L from config and load the freeze state dict.""" + import torch + from transformers import CLIPConfig, CLIPModel + + clip = CLIPModel(CLIPConfig.from_pretrained(BACKBONE)) + model = HeadedCLIP(clip) + state = torch.load(checkpoint, map_location="cpu", weights_only=True) + model.load_state_dict(state, strict=True) + return model.to(device).eval() + + +def load_processor() -> Any: + from transformers import CLIPImageProcessor + + processor = CLIPImageProcessor.from_pretrained(BACKBONE) + processor.do_resize = False + processor.do_center_crop = False + return processor + + +def embed_image(model: HeadedCLIP, processor: Any, image: Image.Image, device: torch.device) -> NDArray[Any]: + """Return a 768-d L2-normalized CLIP-L-ft vector.""" + import numpy as np + import torch + + boxed = letterbox(image) + pixels = processor(images=[boxed], return_tensors="pt")["pixel_values"].to( + device, dtype=next(model.parameters()).dtype + ) + with torch.inference_mode(): + vector = model.embed(pixels).float().cpu().numpy()[0] + return np.asarray(vector, dtype=np.float64) diff --git a/src/remove_ai_watermarks/_internal/forensic_124d.py b/src/remove_ai_watermarks/_internal/forensic_124d.py new file mode 100644 index 0000000..e42a5db --- /dev/null +++ b/src/remove_ai_watermarks/_internal/forensic_124d.py @@ -0,0 +1,144 @@ +"""124-d patch residual features for the frozen provider heads. + +Lifted from the 2026-08-31 freeze extractor. The vector is ratios on +lattice-aligned 256 px patches: FFT band energy, comb contrast, and residual +autocovariance. Images smaller than the patch return None. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +import cv2 +import numpy as np + +if TYPE_CHECKING: + from numpy.typing import NDArray + +PATCH = 256 +MAX_PATCHES = 12 +BLUR_SIGMA = 1.0 +BANDS = 8 +LAGS = (8, 16) +CONTROL_LAGS = (11, 13) +REACH = 3 +FEATURE_WIDTH = 124 + + +def _lag_grid() -> list[tuple[int, int]]: + lags = [(0, dx) for dx in range(1, REACH + 1)] + lags += [(dy, dx) for dy in range(1, REACH + 1) for dx in range(-REACH, REACH + 1)] + return lags + + +LAG_GRID = _lag_grid() + + +def _opponent(pixels: NDArray[np.uint8]) -> NDArray[np.float64]: + values = pixels.astype(np.float64) + red, green, blue = values[:, :, 0], values[:, :, 1], values[:, :, 2] + return np.stack( + [ + 0.299 * red + 0.587 * green + 0.114 * blue, + red - green, + blue - 0.5 * (red + green), + ], + axis=-1, + ) + + +def _residual(planes: NDArray[np.float64]) -> NDArray[np.float64]: + blurred = cv2.GaussianBlur(planes, (0, 0), sigmaX=BLUR_SIGMA, sigmaY=BLUR_SIGMA, borderType=cv2.BORDER_REFLECT_101) + return planes - blurred + + +def _patch_origins(height: int, width: int) -> list[tuple[int, int]]: + if height < PATCH or width < PATCH: + return [] + rows = list(range(0, height - PATCH + 1, PATCH)) + columns = list(range(0, width - PATCH + 1, PATCH)) + origins = [(row, column) for row in rows for column in columns] + if len(origins) <= MAX_PATCHES: + return origins + step = len(origins) / MAX_PATCHES + return [origins[int(index * step)] for index in range(MAX_PATCHES)] + + +def _masks() -> tuple[NDArray[np.bool_], NDArray[np.bool_], NDArray[np.bool_], list[NDArray[np.bool_]]]: + frequency_rows = np.fft.fftfreq(PATCH)[:, None] + frequency_columns = np.fft.fftfreq(PATCH)[None, :] + radius = np.hypot(frequency_rows, frequency_columns) + index_rows = np.arange(PATCH)[:, None] + index_columns = np.arange(PATCH)[None, :] + period16 = (index_rows % (PATCH // 16) == 0) & (index_columns % (PATCH // 16) == 0) + period8 = (index_rows % (PATCH // 8) == 0) & (index_columns % (PATCH // 8) == 0) + comb16_only = period16 & ~period8 + background = ~period16 + edges = np.linspace(0.0, 0.5, BANDS + 1) + bands = [(radius > edges[i]) & (radius <= edges[i + 1]) for i in range(BANDS)] + return comb16_only, period8, background, bands + + +COMB16, COMB8, BACKGROUND, BAND_MASKS = _masks() + + +def _contrast(power: NDArray[np.float64], comb: NDArray[np.bool_]) -> float: + reach = comb & (BAND_MASKS[1] | BAND_MASKS[2] | BAND_MASKS[3] | BAND_MASKS[4]) + floor = BACKGROUND & (BAND_MASKS[1] | BAND_MASKS[2] | BAND_MASKS[3] | BAND_MASKS[4]) + if not reach.any() or not floor.any(): + return 0.0 + background_power = float(power[floor].mean()) + if background_power <= 0.0: + return 0.0 + return float(power[reach].mean() / background_power) + + +def _autocorrelation(patch: NDArray[np.float64], lag: int) -> float: + centred = patch - patch.mean() + energy = float((centred**2).sum()) + if energy <= 0.0: + return 0.0 + rows = float((centred[:-lag, :] * centred[lag:, :]).sum()) + columns = float((centred[:, :-lag] * centred[:, lag:]).sum()) + return (rows + columns) / (2.0 * energy) + + +def image_features(pixels: NDArray[np.uint8]) -> NDArray[np.float64] | None: + """Return the 124-d vector for RGB uint8 pixels, or None when geometry fails.""" + if pixels.ndim != 3 or pixels.shape[2] != 3: + return None + residual = _residual(_opponent(pixels)) + origins = _patch_origins(*pixels.shape[:2]) + if not origins: + return None + per_patch: list[list[float]] = [] + spread_slice: slice | None = None + for row, column in origins: + window = residual[row : row + PATCH, column : column + PATCH] + values: list[float] = [] + for channel in range(3): + plane = window[:, :, channel] + power = np.asarray(np.abs(np.fft.fft2(plane)) ** 2, dtype=np.float64) + total = float(power[1:, 1:].sum()) + if total <= 0.0: + return None + values.extend(float(power[mask].sum() / total) for mask in BAND_MASKS) + values.append(_contrast(power, COMB16)) + values.append(_contrast(power, COMB8)) + if spread_slice is None: + spread_slice = slice(0, len(values)) + values.extend(_autocorrelation(plane, lag) for lag in LAGS) + values.extend(_autocorrelation(plane, lag) for lag in CONTROL_LAGS) + correlation = np.fft.ifft2(power).real + centre = float(correlation[0, 0]) + if centre <= 0.0: + return None + values.extend(float(correlation[dy, dx] / centre) for dy, dx in LAG_GRID) + per_patch.append(values) + stack = np.asarray(per_patch, dtype=np.float64) + if spread_slice is None: + return None + vector = np.concatenate([np.median(stack, axis=0), stack[:, spread_slice].std(axis=0)]) + if vector.shape != (FEATURE_WIDTH,): + return None + return vector diff --git a/src/remove_ai_watermarks/classify.py b/src/remove_ai_watermarks/classify.py new file mode 100644 index 0000000..ef3a460 --- /dev/null +++ b/src/remove_ai_watermarks/classify.py @@ -0,0 +1,363 @@ +# pyright: reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false +"""Metadata-free photo classifier: Model 1 then gated Model 2. + +This is not provenance and is not a cleanup command. ``identify``, +``has_invisible_target``, and ``all`` must not import this module. +Call :func:`classify_pixels` explicitly. +""" + +from __future__ import annotations + +import json +import logging +import os +import threading +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Literal + +log = logging.getLogger(__name__) + +CLASSIFY_EXTRA = "'remove-ai-watermarks[classify]'" +WEIGHTS_ENV = "RAIW_CLASSIFY_WEIGHTS" +WEIGHTS_REPO = "wiltodelta/raiw-models" +WEIGHTS_REVISION = "c0ac82b6f1ae9fc0b92c467562282e9422da6da6" +CLIP_FILE = "clip-l-ft.pt" +PROBE_FILE = "probe-weights-clip-l-ft.npz" +DETECTOR_FILE = "detector.pt" +PROVIDER_FILE = "provider.pt" +OPERATING_POINT_FILE = "operating-point.json" +_WEIGHT_FILES = (CLIP_FILE, PROBE_FILE, DETECTOR_FILE, PROVIDER_FILE) + +# 2026-08-31 freeze operating point. Ridge threshold is also in the probe file; +# the runtime prefers the file when weights load. +MLP_THRESHOLD = 5.9586493237495395 +RIDGE_THRESHOLD = 0.3056212276800537 +PROVIDER_MARGIN = 0.30 +CLIP_WIDTH = 768 +PROVIDER_LABELS = ("openai", "google", "tc260", "meta_muse_image", "no_ai") + +DetectorLevel = Literal["definitely", "possibly", "likely_human"] +PixelLabel = Literal["ai", "human", "unknown"] +PixelDomain = Literal["photo"] +PixelProvider = Literal["openai", "google", "meta", "tc260"] +PUBLIC_PROVIDER: dict[str, PixelProvider] = { + "openai": "openai", + "google": "google", + "tc260": "tc260", + "meta_muse_image": "meta", +} + +_runtime: _Runtime | None = None +_runtime_lock = threading.Lock() + + +@dataclass(frozen=True) +class PixelClassification: + """Combined Model 1 and Model 2 result for one image.""" + + label: PixelLabel + domain: PixelDomain + detector: DetectorLevel + provider: PixelProvider | None + + def to_dict(self) -> dict[str, Any]: + return { + "label": self.label, + "domain": self.domain, + "detector": self.detector, + "provider": self.provider, + } + + +def is_available() -> bool: + """True when the ``classify`` extra can import torch, transformers, numpy, and cv2.""" + from remove_ai_watermarks.optional_deps import module_available + + return module_available("torch", "transformers", "numpy", "cv2") + + +def detector_level( + ridge_score: float, + mlp_score: float, + *, + ridge_threshold: float = RIDGE_THRESHOLD, + mlp_threshold: float = MLP_THRESHOLD, +) -> DetectorLevel: + """Map the freeze AND/OR rule onto definitely / possibly / likely_human.""" + ridge = ridge_score > ridge_threshold + mlp = mlp_score > mlp_threshold + if ridge and mlp: + return "definitely" + if ridge or mlp: + return "possibly" + return "likely_human" + + +def label_for(level: DetectorLevel) -> PixelLabel: + """Public label: only DEFINITELY is ``ai``. POSSIBLY abstains.""" + if level == "definitely": + return "ai" + if level == "likely_human": + return "human" + return "unknown" + + +def provider_from_scores(scores: dict[str, float], *, margin: float = PROVIDER_MARGIN) -> PixelProvider | None: + """Argmax among openai/google/tc260/meta that beat ``no_ai`` by ``margin``.""" + no_ai = scores["no_ai"] + ai_names = [name for name in PROVIDER_LABELS if name != "no_ai"] + passed = [name for name in ai_names if scores[name] > no_ai + margin] + if not passed: + return None + best = max(passed, key=lambda name: scores[name]) + return PUBLIC_PROVIDER[best] + + +def classify_from_scores( + ridge_score: float, + mlp_score: float, + provider_scores: dict[str, float] | None, + *, + ridge_threshold: float = RIDGE_THRESHOLD, + mlp_threshold: float = MLP_THRESHOLD, + provider_margin: float = PROVIDER_MARGIN, +) -> PixelClassification: + """Pure gate: detector AND provider, no file I/O.""" + level = detector_level( + ridge_score, + mlp_score, + ridge_threshold=ridge_threshold, + mlp_threshold=mlp_threshold, + ) + label = label_for(level) + provider: PixelProvider | None = None + if label == "ai" and provider_scores is not None: + provider = provider_from_scores(provider_scores, margin=provider_margin) + return PixelClassification(label=label, domain="photo", detector=level, provider=provider) + + +def classify_pixels(path: Path, *, device: str | None = None) -> PixelClassification: + """Run Model 1 then, on DEFINITELY, Model 2. Never called by ``identify``.""" + if not is_available(): + raise RuntimeError(f"pixel classification requires the classify extra. Install: pip install {CLASSIFY_EXTRA}") + runtime = _get_runtime(device) + from PIL import Image + + with Image.open(path) as image: + rgb = image.convert("RGB") + clip_vector = _embed(runtime, rgb) + ridge = float(((clip_vector - runtime.ridge_mean) / runtime.ridge_scale) @ runtime.ridge_weights) + mlp = _mlp_score(runtime, clip_vector) + level = detector_level( + ridge, + mlp, + ridge_threshold=runtime.ridge_threshold, + mlp_threshold=runtime.mlp_threshold, + ) + provider_scores = None + if level == "definitely": + forensic = _forensic(rgb) + if forensic is not None: + provider_scores = _provider_scores(runtime, forensic) + else: + log.info("124-d features unavailable for %s, provider abstains", path) + return classify_from_scores( + ridge, + mlp, + provider_scores, + ridge_threshold=runtime.ridge_threshold, + mlp_threshold=runtime.mlp_threshold, + provider_margin=runtime.provider_margin, + ) + + +@dataclass +class _Runtime: + device: Any + clip_model: Any + processor: Any + detector: Any + provider_heads: dict[str, Any] + ridge_mean: Any + ridge_scale: Any + ridge_weights: Any + ridge_threshold: float + mlp_threshold: float + provider_margin: float + + +def _resolve_device(device: str | None) -> Any: + import torch + + if device in (None, "auto"): + return torch.device("cuda" if torch.cuda.is_available() else "cpu") + if device == "cpu": + return torch.device("cpu") + if device == "cuda": + if not torch.cuda.is_available(): + raise ValueError("cuda was requested but is not available") + return torch.device("cuda") + raise ValueError(f"unsupported classify device {device!r}") + + +def _weights_dir() -> Path: + override = os.environ.get(WEIGHTS_ENV) + if override: + path = Path(override) + if not path.is_dir(): + raise RuntimeError(f"{WEIGHTS_ENV} is not a directory: {path}") + return path + from huggingface_hub import snapshot_download + + try: + return Path( + snapshot_download( + repo_id=WEIGHTS_REPO, + revision=WEIGHTS_REVISION, + allow_patterns=[*_WEIGHT_FILES, OPERATING_POINT_FILE], + ) + ) + except Exception as exc: + raise RuntimeError( + "pixel classification weights are not installed. Set " + f"{WEIGHTS_ENV} to a directory with {', '.join(_WEIGHT_FILES)}, " + f"or install {CLASSIFY_EXTRA} and download {WEIGHTS_REPO}@{WEIGHTS_REVISION}." + ) from exc + + +def _find_weight(folder: Path, name: str) -> Path | None: + direct = folder / name + nested = folder / "run1" / name + if direct.is_file(): + return direct + if nested.is_file(): + return nested + return None + + +def _weight_path(folder: Path, name: str) -> Path: + found = _find_weight(folder, name) + if found is None: + raise RuntimeError(f"pixel classification weights are missing ({name}) in {folder}") + return found + + +def _require_files(folder: Path) -> None: + missing = [name for name in _WEIGHT_FILES if _find_weight(folder, name) is None] + if missing: + raise RuntimeError( + "pixel classification weights are missing " + f"({', '.join(missing)}). Set {WEIGHTS_ENV} to a directory with the " + f"2026-08-31 freeze files, or install {CLASSIFY_EXTRA} and allow the " + f"Hugging Face download from {WEIGHTS_REPO}." + ) + + +def _operating_point(folder: Path) -> tuple[float, float]: + path = folder / OPERATING_POINT_FILE + if not path.is_file(): + return MLP_THRESHOLD, PROVIDER_MARGIN + payload = json.loads(path.read_text()) + return float(payload["model1"]["mlp_threshold"]), float(payload["model2"]["margin"]) + + +def _build_detector() -> Any: + from torch import nn + + return nn.Sequential( + nn.Linear(CLIP_WIDTH, 512), + nn.ReLU(), + nn.Dropout(0.3), + nn.Linear(512, 128), + nn.ReLU(), + nn.Dropout(0.1), + nn.Linear(128, 1), + ) + + +def _build_provider_head() -> Any: + from torch import nn + + return nn.Sequential( + nn.Linear(124, 64), + nn.ReLU(), + nn.Dropout(0.2), + nn.Linear(64, 1), + ) + + +def _load_runtime(device: Any) -> _Runtime: + import numpy as np + import torch + + from remove_ai_watermarks._internal.clip_l_ft import load_headed_clip, load_processor + + folder = _weights_dir() + _require_files(folder) + probe = np.load(_weight_path(folder, PROBE_FILE)) + detector = _build_detector() + detector.load_state_dict(torch.load(_weight_path(folder, DETECTOR_FILE), map_location="cpu", weights_only=True)) + detector.to(device).eval() + packed = torch.load(_weight_path(folder, PROVIDER_FILE), map_location="cpu", weights_only=True) + heads = {} + for name in PROVIDER_LABELS: + head = _build_provider_head() + head.load_state_dict(packed[name]) + heads[name] = head.to(device).eval() + ridge_threshold = float(probe["thr_oi_1pct"]) + mlp_threshold, provider_margin = _operating_point(folder) + log.info("loaded photo-classify freeze from %s", folder) + return _Runtime( + device=device, + clip_model=load_headed_clip(_weight_path(folder, CLIP_FILE), device), + processor=load_processor(), + detector=detector, + provider_heads=heads, + ridge_mean=np.asarray(probe["mean"], dtype=np.float64), + ridge_scale=np.asarray(probe["scale"], dtype=np.float64), + ridge_weights=np.asarray(probe["weights"], dtype=np.float64), + ridge_threshold=ridge_threshold, + mlp_threshold=mlp_threshold, + provider_margin=provider_margin, + ) + + +def _get_runtime(device: str | None) -> _Runtime: + global _runtime + resolved = _resolve_device(device) + with _runtime_lock: + if _runtime is None or _runtime.device != resolved: + _runtime = _load_runtime(resolved) + return _runtime + + +def _embed(runtime: _Runtime, image: Any) -> Any: + from remove_ai_watermarks._internal.clip_l_ft import embed_image + + return embed_image(runtime.clip_model, runtime.processor, image, runtime.device) + + +def _forensic(image: Any) -> Any: + import numpy as np + + from remove_ai_watermarks._internal.forensic_124d import image_features + + pixels = np.asarray(image, dtype=np.uint8) + return image_features(pixels) + + +def _mlp_score(runtime: _Runtime, clip_vector: Any) -> float: + import torch + + tensor = torch.from_numpy(clip_vector.astype("float32")).unsqueeze(0).to(runtime.device) + with torch.inference_mode(): + return float(runtime.detector(tensor).squeeze().cpu()) + + +def _provider_scores(runtime: _Runtime, forensic: Any) -> dict[str, float]: + import torch + + tensor = torch.from_numpy(forensic.astype("float32")).unsqueeze(0).to(runtime.device) + with torch.inference_mode(): + return {name: float(head(tensor).squeeze().cpu()) for name, head in runtime.provider_heads.items()} diff --git a/src/remove_ai_watermarks/cli.py b/src/remove_ai_watermarks/cli.py index 671ad27..bdb8063 100644 --- a/src/remove_ai_watermarks/cli.py +++ b/src/remove_ai_watermarks/cli.py @@ -1485,6 +1485,45 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo console.print(f" - {c}") +# ── Metadata-free photo classification (opt-in, never a tail of identify) ── +@main.command("classify") +@click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option("--json", "as_json", is_flag=True, help="Emit the classification as JSON.") +def cmd_classify(source: Path, as_json: bool) -> None: + """Classify a photograph as AI or camera-like from pixels. + + Runs the frozen photo detector and, only on a DEFINITELY-AI result, the + provider head. This is not provenance. identify never starts this command. + Install: pip install 'remove-ai-watermarks[classify]' + """ + from remove_ai_watermarks import classify as classify_mod + + if not classify_mod.is_available(): + console.print( + "Error: the pixel-classification dependencies are not installed.\n" + f" Install them with: pip install {classify_mod.CLASSIFY_EXTRA}" + ) + raise SystemExit(1) + + source = _validate_image(source) + try: + result = classify_mod.classify_pixels(source) + except (OSError, RuntimeError, ValueError) as exc: + raise click.ClickException(str(exc)) from exc + + if as_json: + click.echo(json.dumps(result.to_dict(), indent=2)) + return + + _banner() + console.print("\n Pixel classification (not a provenance verdict)") + console.print(f" Label: {result.label}") + console.print(f" Domain: {result.domain}") + console.print(f" Detector: {result.detector}") + console.print(f" Provider: {result.provider or 'none'}") + console.print(" identify is unchanged. This command does not run cleanup and does not prove a file is clean.") + + # ── Combined "all" mode ── @main.command("all") @click.argument("source", type=click.Path(exists=True, dir_okay=False, path_type=Path)) diff --git a/tests/test_classify.py b/tests/test_classify.py new file mode 100644 index 0000000..2ee22b8 --- /dev/null +++ b/tests/test_classify.py @@ -0,0 +1,206 @@ +"""Pixel classify gate: Model 1 then gated Model 2, no downloads, no identify hook.""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path +from types import SimpleNamespace + +import numpy as np +import pytest +from click.testing import CliRunner +from PIL import Image + +from remove_ai_watermarks.classify import ( + MLP_THRESHOLD, + PROVIDER_MARGIN, + RIDGE_THRESHOLD, + WEIGHTS_REPO, + WEIGHTS_REVISION, + classify_from_scores, + classify_pixels, + detector_level, + label_for, + provider_from_scores, +) +from remove_ai_watermarks.cli import main + + +def _scores( + *, + openai: float = 0.0, + google: float = 0.0, + tc260: float = 0.0, + meta: float = 0.0, + no_ai: float = 0.0, +) -> dict[str, float]: + return { + "openai": openai, + "google": google, + "tc260": tc260, + "meta_muse_image": meta, + "no_ai": no_ai, + } + + +def test_hub_snapshot_is_the_freeze_revision() -> None: + assert WEIGHTS_REPO == "wiltodelta/raiw-models" + assert WEIGHTS_REVISION == "c0ac82b6f1ae9fc0b92c467562282e9422da6da6" + + +def test_shipped_operating_point_matches_the_runtime_defaults() -> None: + payload = json.loads(Path("docs/photo-classify-hf/operating-point.json").read_text()) + assert payload["model1"]["mlp_threshold"] == MLP_THRESHOLD + assert payload["model1"]["ridge_threshold"] == RIDGE_THRESHOLD + assert payload["model2"]["margin"] == PROVIDER_MARGIN + assert payload["model2"]["runs_only_after"] == "definitely" + + +def test_definitely_is_the_and_of_ridge_and_mlp() -> None: + assert detector_level(1.0, 10.0) == "definitely" + assert detector_level(0.0, 10.0) == "possibly" + assert detector_level(1.0, 0.0) == "possibly" + assert detector_level(0.0, 0.0) == "likely_human" + + +def test_only_definitely_is_ai() -> None: + assert label_for("definitely") == "ai" + assert label_for("possibly") == "unknown" + assert label_for("likely_human") == "human" + + +def test_provider_requires_margin_over_no_ai() -> None: + assert provider_from_scores(_scores(openai=1.0, no_ai=0.5)) == "openai" + assert provider_from_scores(_scores(openai=0.5, no_ai=0.4)) is None + assert provider_from_scores(_scores(google=2.0, openai=1.5, no_ai=0.0)) == "google" + assert provider_from_scores(_scores(meta=1.0, no_ai=0.0)) == "meta" + assert provider_from_scores(_scores(tc260=1.0, no_ai=0.0)) == "tc260" + + +def test_definitely_plus_openai_is_ai_openai() -> None: + result = classify_from_scores(1.0, 10.0, _scores(openai=1.0, no_ai=0.0)) + assert result.label == "ai" + assert result.domain == "photo" + assert result.detector == "definitely" + assert result.provider == "openai" + assert result.to_dict()["provider"] == "openai" + + +def test_definitely_without_provider_scores_stays_ai_with_no_provider() -> None: + result = classify_from_scores(1.0, 10.0, None) + assert result.label == "ai" + assert result.provider is None + + +def test_definitely_plus_no_ai_head_has_no_provider() -> None: + result = classify_from_scores(1.0, 10.0, _scores(no_ai=2.0, openai=1.0)) + assert result.label == "ai" + assert result.provider is None + + +def test_possibly_does_not_run_provider() -> None: + result = classify_from_scores(1.0, 0.0, _scores(openai=9.0, no_ai=0.0)) + assert result.label == "unknown" + assert result.detector == "possibly" + assert result.provider is None + + +def test_likely_human_does_not_run_provider() -> None: + result = classify_from_scores(0.0, 0.0, _scores(openai=9.0, no_ai=0.0)) + assert result.label == "human" + assert result.detector == "likely_human" + assert result.provider is None + + +@pytest.mark.parametrize("caller", ["identify", "has_invisible_target"]) +def test_provenance_paths_do_not_import_classify(tmp_path: Path, caller: str) -> None: + path = tmp_path / "plain.png" + Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8)).save(path) + sys.modules.pop("remove_ai_watermarks.classify", None) + from remove_ai_watermarks.identify import has_invisible_target, identify + + if caller == "identify": + report = identify(path, check_visible=False, check_invisible=False) + assert report.is_ai_generated is None + else: + assert has_invisible_target(path) is False + loaded = [name for name in sys.modules if name.startswith("remove_ai_watermarks.classify")] + assert loaded == [] + + +def _plain_png(tmp_path: Path) -> Path: + path = tmp_path / "plain.png" + Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8)).save(path) + return path + + +def test_124d_is_not_extracted_unless_definitely(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: + calls: list[int] = [] + runtime = SimpleNamespace( + ridge_mean=np.zeros(768), + ridge_scale=np.ones(768), + ridge_weights=np.zeros(768), + ridge_threshold=RIDGE_THRESHOLD, + mlp_threshold=MLP_THRESHOLD, + provider_margin=PROVIDER_MARGIN, + ) + monkeypatch.setattr("remove_ai_watermarks.classify.is_available", lambda: True) + monkeypatch.setattr("remove_ai_watermarks.classify._get_runtime", lambda device: runtime) + monkeypatch.setattr("remove_ai_watermarks.classify._embed", lambda *args, **kwargs: np.zeros(768)) + monkeypatch.setattr("remove_ai_watermarks.classify._mlp_score", lambda *args, **kwargs: 0.0) + monkeypatch.setattr( + "remove_ai_watermarks.classify._forensic", + lambda image: calls.append(1) or np.zeros(124), + ) + result = classify_pixels(_plain_png(tmp_path)) + assert result.label == "human" + assert calls == [] + + +def test_weight_files_may_sit_in_run1(tmp_path: Path) -> None: + from remove_ai_watermarks.classify import _find_weight + + nested = tmp_path / "run1" + nested.mkdir() + target = nested / "detector.pt" + target.write_bytes(b"x") + assert _find_weight(tmp_path, "detector.pt") == target + assert _find_weight(tmp_path, "clip-l-ft.pt") is None + + +def test_operating_point_prefers_the_sidecar(tmp_path: Path) -> None: + from remove_ai_watermarks.classify import _operating_point + + (tmp_path / "operating-point.json").write_text( + json.dumps({"model1": {"mlp_threshold": 1.25}, "model2": {"margin": 0.5}}) + ) + assert _operating_point(tmp_path) == (1.25, 0.5) + assert _operating_point(tmp_path / "missing") == (MLP_THRESHOLD, PROVIDER_MARGIN) + + +def test_missing_extra_raises_the_install_hint(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: + import remove_ai_watermarks.classify as classify_mod + + monkeypatch.setattr(classify_mod, "is_available", lambda: False) + monkeypatch.setattr( + classify_mod, + "_get_runtime", + lambda device: (_ for _ in ()).throw(AssertionError("must not load weights")), + ) + path = tmp_path / "plain.png" + Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8)).save(path) + with pytest.raises(RuntimeError, match=r"remove-ai-watermarks\[classify\]"): + classify_mod.classify_pixels(path) + + +def test_cli_classify_without_extra_prints_the_install_hint(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: + import remove_ai_watermarks.classify as classify_mod + + monkeypatch.setattr(classify_mod, "is_available", lambda: False) + path = tmp_path / "plain.png" + Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8)).save(path) + result = CliRunner().invoke(main, ["classify", str(path)]) + assert result.exit_code == 1 + assert "remove-ai-watermarks[classify]" in result.output + assert "pip install" in result.output diff --git a/tests/test_cli_robustness.py b/tests/test_cli_robustness.py index 832e45b..fdbac19 100644 --- a/tests/test_cli_robustness.py +++ b/tests/test_cli_robustness.py @@ -101,7 +101,7 @@ class TestApiReportsFailedWrite: class TestDirectoryInputIsRejected: - @pytest.mark.parametrize("cmd", ["identify", "visible", "erase", "metadata", "invisible", "all"]) + @pytest.mark.parametrize("cmd", ["identify", "classify", "visible", "erase", "metadata", "invisible", "all"]) def test_directory_as_source_is_a_clean_usage_error(self, tmp_path, cmd): """A directory must be refused by argument parsing, never reach the scanners.""" args = [cmd, str(tmp_path)] diff --git a/tests/test_forensic_124d.py b/tests/test_forensic_124d.py new file mode 100644 index 0000000..9c7524a --- /dev/null +++ b/tests/test_forensic_124d.py @@ -0,0 +1,32 @@ +"""124-d freeze extractor: shape and the undersize abstain, no downloads.""" + +from __future__ import annotations + +import numpy as np + +from remove_ai_watermarks._internal.forensic_124d import FEATURE_WIDTH, PATCH, image_features + + +def test_a_full_patch_returns_124_floats() -> None: + pixels = np.random.default_rng(0).integers(0, 256, (PATCH, PATCH, 3), dtype=np.uint8) + vector = image_features(pixels) + assert vector is not None + assert vector.shape == (FEATURE_WIDTH,) + assert vector.dtype == np.float64 + + +def test_a_larger_image_still_returns_124_floats() -> None: + pixels = np.random.default_rng(1).integers(0, 256, (512, 640, 3), dtype=np.uint8) + vector = image_features(pixels) + assert vector is not None + assert vector.shape == (FEATURE_WIDTH,) + + +def test_an_undersized_image_returns_none() -> None: + pixels = np.zeros((PATCH - 1, PATCH, 3), dtype=np.uint8) + assert image_features(pixels) is None + + +def test_non_rgb_returns_none() -> None: + pixels = np.zeros((PATCH, PATCH), dtype=np.uint8) + assert image_features(pixels) is None diff --git a/tests/test_packaging.py b/tests/test_packaging.py index 42f087a..5e86c10 100644 --- a/tests/test_packaging.py +++ b/tests/test_packaging.py @@ -71,10 +71,15 @@ def test_trustmark_is_limited_to_its_numpy_compatible_python_range(): def test_extras_use_capability_names_without_legacy_aliases(): extras = set(metadata("remove-ai-watermarks").get_all("Provides-Extra") or []) - assert {"pixels", "heif", "visible", "video", "detect", "diffusion"} <= extras + assert {"pixels", "heif", "visible", "video", "detect", "diffusion", "classify"} <= extras assert {"gpu", "remove", "detect-pywavelets"}.isdisjoint(extras) +def test_classify_extra_owns_the_photo_heads() -> None: + names = _requirement_names("classify") + assert {"torch", "transformers", "huggingface-hub", "tokenizers"} <= names + + def test_production_all_does_not_include_development_tools(): assert { "pyright", diff --git a/uv.lock b/uv.lock index 3fcda2e..8373d5a 100644 --- a/uv.lock +++ b/uv.lock @@ -3199,6 +3199,15 @@ all = [ { name = "transformers" }, { name = "trustmark", marker = "python_full_version < '3.13'" }, ] +classify = [ + { name = "huggingface-hub" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" }, + { name = "numpy", version = "2.3.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.13'" }, + { name = "opencv-python-headless" }, + { name = "tokenizers" }, + { name = "torch" }, + { name = "transformers" }, +] detect = [ { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.13'" }, { name = "numpy", version = "2.3.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.13'" }, @@ -3308,6 +3317,7 @@ requires-dist = [ { name = "click", specifier = ">=8.0.0" }, { name = "diffsynth", marker = "extra == 'qwen-zimage'", specifier = ">=2.0.17,<3" }, { name = "diffusers", marker = "extra == 'diffusion'", specifier = ">=0.38.0" }, + { name = "huggingface-hub", marker = "extra == 'classify'", specifier = ">=0.20.0" }, { name = "huggingface-hub", marker = "extra == 'lama'", specifier = ">=0.20.0" }, { name = "huggingface-hub", marker = "extra == 'migan'", specifier = ">=0.20.0" }, { name = "huggingface-hub", marker = "extra == 'text-draft'", specifier = ">=0.20.0" }, @@ -3331,24 +3341,28 @@ requires-dist = [ { name = "pywavelets", marker = "extra == 'detect'", specifier = ">=1.1.1" }, { name = "remove-ai-watermarks", extras = ["detect"], marker = "extra == 'dev'" }, { name = "remove-ai-watermarks", extras = ["diffusion"], marker = "extra == 'qwen-zimage'" }, + { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'classify'" }, { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'detect'" }, { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'diffusion'" }, { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'visible'" }, { name = "remove-ai-watermarks", extras = ["qwen-zimage", "lama"], marker = "extra == 'text-restoration'" }, { name = "remove-ai-watermarks", extras = ["video"], marker = "extra == 'dev'" }, - { name = "remove-ai-watermarks", extras = ["video", "heif", "detect", "trustmark", "text-restoration", "migan"], marker = "extra == 'all'" }, + { name = "remove-ai-watermarks", extras = ["video", "heif", "detect", "trustmark", "text-restoration", "migan", "classify"], marker = "extra == 'all'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'lama'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'migan'" }, { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'video'" }, { name = "ruff", marker = "extra == 'dev'", specifier = ">=0.4.0" }, { name = "safetensors", marker = "extra == 'diffusion'" }, + { name = "tokenizers", marker = "extra == 'classify'", specifier = ">=0.22,<0.23" }, { name = "tokenizers", marker = "extra == 'diffusion'", specifier = ">=0.22,<0.23" }, + { name = "torch", marker = "extra == 'classify'", specifier = ">=2.0.0" }, { name = "torch", marker = "extra == 'diffusion'", specifier = ">=2.0.0" }, { name = "torchvision", marker = "extra == 'qwen-zimage'", specifier = ">=0.20.0" }, + { name = "transformers", marker = "extra == 'classify'", specifier = ">=5,<6" }, { name = "transformers", marker = "extra == 'diffusion'", specifier = ">=5,<6" }, { name = "trustmark", marker = "python_full_version < '3.13' and extra == 'trustmark'", specifier = ">=0.8.0" }, ] -provides-extras = ["pixels", "heif", "visible", "video", "detect", "diffusion", "qwen-zimage", "text-restoration", "text-draft", "trustmark", "lama", "migan", "dev", "all"] +provides-extras = ["pixels", "heif", "visible", "video", "detect", "diffusion", "qwen-zimage", "text-restoration", "text-draft", "trustmark", "classify", "lama", "migan", "dev", "all"] [[package]] name = "requests"