diff --git a/README.md b/README.md index 99ffea4..1579c86 100644 --- a/README.md +++ b/README.md @@ -32,9 +32,24 @@ Remove AI provenance marks from images you generated yourself: | Run visible, invisible, and metadata removal | `all` | Recommended | | Process a directory | `batch` | Depends on mode | +## Installation modes + +| Need | Install | +| --- | --- | +| Metadata inspection and stripping | `remove-ai-watermarks` | +| Visible detection and removal | `remove-ai-watermarks[visible]` | +| Torch-free DWT-DCT detection | `remove-ai-watermarks[detect]` | +| Diffusion removal | `remove-ai-watermarks[diffusion]` | +| Every production feature | `remove-ai-watermarks[all]` | + +Lower-level and specialized extras include `pixels`, `heif`, `trustmark`, +`migan`, `lama`, `esrgan`, and `qwen-zimage`. The +[installation guide](docs/installation.md#feature-extras) documents their exact +dependency composition and model requirements. + ## Quick start -Install the core CLI: +Install the metadata-focused default CLI: ```bash uv tool install remove-ai-watermarks @@ -46,7 +61,13 @@ Inspect an image: remove-ai-watermarks identify image.png ``` -Remove a known visible mark and AI metadata: +For visible watermark removal, install the pixel dependencies: + +```bash +uv tool install --force "remove-ai-watermarks[visible]" +``` + +Then remove a known visible mark and AI metadata: ```bash remove-ai-watermarks visible image.png -o clean.png @@ -61,7 +82,7 @@ remove-ai-watermarks metadata image.png --remove -o clean.png For invisible watermark removal, install the diffusion dependencies: ```bash -uv tool install --force "remove-ai-watermarks[gpu]" +uv tool install --force "remove-ai-watermarks[diffusion]" remove-ai-watermarks invisible image.png -o clean.png ``` @@ -129,8 +150,9 @@ remove-ai-watermarks erase image.png \ ### Use a learned fill backend -The core install uses OpenCV inpainting when no learned backend is installed. -For more difficult backgrounds: +The `visible` extra uses OpenCV inpainting when no learned backend is installed. +For more difficult backgrounds, the learned-backend extras include the same +pixel dependencies automatically: ```bash uv tool install --force "remove-ai-watermarks[migan]" @@ -197,6 +219,8 @@ See [supported signals](docs/supported-signals.md) and ## Python API +The visible-removal API requires `remove-ai-watermarks[visible]`. + ```python import remove_ai_watermarks as raiw diff --git a/docs/cli.md b/docs/cli.md index da76b8e..a264b31 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -9,15 +9,35 @@ remove-ai-watermarks [OPTIONS] COMMAND [ARGS] Run `remove-ai-watermarks COMMAND --help` for the complete option list and defaults. This page focuses on choosing the right command. +## Command dependency map + +| Command or signal | Required installation | +| --- | --- | +| `metadata` and metadata-only `identify` | Default package | +| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) | +| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` | +| Adobe TrustMark signals in `identify` | `remove-ai-watermarks[trustmark]` | +| `visible` and `erase` with OpenCV | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) | +| `visible` or `erase` with MI-GAN | `remove-ai-watermarks[migan]` | +| `visible` or `erase` with big-LaMa | `remove-ai-watermarks[lama]` | +| `invisible` | `remove-ai-watermarks[diffusion]` | +| `invisible --pipeline qwen-zimage` | `remove-ai-watermarks[qwen-zimage]` | +| HEIC/HEIF/AVIF pixel input | Add `remove-ai-watermarks[heif]` | +| Every production command and backend | `remove-ai-watermarks[all]` | + +`batch` requires the same extra as its selected mode. Extras can be combined in +one installation, for example `remove-ai-watermarks[visible,detect,heif]`. + ## Inspect an image ```bash remove-ai-watermarks identify image.png ``` -`identify` combines supported metadata and pixel signals into one provenance -report. When no signal is found, it reports the origin as unknown. It does not -claim the image is clean. +`identify` always inspects supported metadata. When pixel extras are installed, +it also evaluates supported visible and invisible pixel signals. When no signal +is found, it reports the origin as unknown. It does not claim the image is +clean. Machine readable output: @@ -36,6 +56,8 @@ invisible pixel detectors. Metadata inspection still runs. ## Remove known visible marks +Install `remove-ai-watermarks[visible]` before using `visible` or `erase`. + ```bash remove-ai-watermarks visible image.png -o clean.png ``` @@ -130,7 +152,7 @@ non-ISOBMFF audio and video path. Install the diffusion dependencies first: ```bash -uv tool install --force "remove-ai-watermarks[gpu]" +uv tool install --force "remove-ai-watermarks[diffusion]" ``` Then run: @@ -194,6 +216,12 @@ It is a memory strategy, not a guarantee of better quality. ## Run the full pipeline +The `all` command and the `all` installation extra are separate concepts. The +command runs every applicable stage. Installing `remove-ai-watermarks[all]` +makes every production backend available; a smaller installation such as +`remove-ai-watermarks[visible,diffusion]` can also run the command with fewer +optional backends. + ```bash remove-ai-watermarks all image.png -o clean.png ``` @@ -206,7 +234,7 @@ The command runs: The visible options and diffusion options are also available on `all`. -If diffusion is required but the `gpu` extra is unavailable, `all` still +If diffusion is required but the `diffusion` extra is unavailable, `all` still writes the result of the visible and metadata stages, prints a prominent warning, and exits with code 1. This prevents a partial result from being reported as complete. diff --git a/docs/development.md b/docs/development.md index 49a6591..e951ca3 100644 --- a/docs/development.md +++ b/docs/development.md @@ -6,15 +6,15 @@ Read this reference for environment setup, dependency recovery, CI behavior, and - Use `uv sync --frozen --extra dev` and add only the feature extras needed for the task. - Do not use `uv pip install` for development tools. It can re-resolve `uv.lock` outside the compatible ML dependency set. -- A core-only sync removes GPU packages by design. Package imports remain light through lazy exports; only removal paths should require the heavy stack. -- On an unreliable connection, sync the needed `dev` and `gpu` extras and run the lint, type, and test commands directly instead of downloading every optional learned backend. +- A default-only sync removes every pixel and model package by design. Package imports remain light through lazy exports. +- On an unreliable connection, sync `dev` plus only the required feature extras, such as `diffusion`, and run the checks directly instead of downloading every optional learned backend. - Run `uv` from the repository root or it may create a bare environment without the project dependencies. The optional TrustMark decoder downloads weights into its installed package directory. After pruning that extra, a leftover weights directory can make availability checks see an empty namespace package. If Pyright reports an unknown `TrustMark` import and `find_spec("trustmark")` returns a loader-less spec, remove that regenerable remnant from the active virtual environment and resync. ## CI -`.github/workflows/test.yml` runs Ruff and a cross-platform supported-Python test matrix with core plus development dependencies. GPU and model-running tests skip in that matrix; metadata, identification, visible removal, and the OpenCV eraser remain covered across operating systems. +`.github/workflows/test.yml` runs Ruff and a cross-platform supported-Python test matrix with default plus development dependencies. Diffusion and model-running tests skip in that matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the OpenCV eraser remain covered across operating systems. Keep `uv.lock` compatible with `uv sync --frozen`. Dependency pull-request checks use GitHub's merge result against current `main`; if `main` moves, merge it locally and rerun the full gate because a newer linter can expose stale directives in later code. diff --git a/docs/installation.md b/docs/installation.md index 4f7d2b2..1a21ba9 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -2,15 +2,17 @@ Python 3.10.1 or newer is required. -## Core install +## Default metadata mode -The core package provides: +The default package provides: - provenance inspection; -- visible watermark removal with OpenCV; -- manual region erasing with OpenCV; - AI metadata inspection and removal. +It installs Pillow, piexif, and c2pa-python for reading metadata directly from +files. It does not install NumPy, OpenCV, pillow-heif, Torch, diffusion models, +or invisible-watermark decoders. + Install it as an isolated command with uv: ```bash @@ -29,12 +31,27 @@ You can also install the Homebrew package on macOS or Linux: brew install wiltodelta/tap/remove-ai-watermarks ``` -## Invisible watermark removal +## Visible watermark removal -Diffusion based removal needs the `gpu` extra: +Visible mark detection, OpenCV inpainting, and manual region erasing need the +`visible` extra: ```bash -uv tool install --force "remove-ai-watermarks[gpu]" +uv tool install --force "remove-ai-watermarks[visible]" +``` + +Add `heif` only when the pixel path must decode HEIC, HEIF, or AVIF: + +```bash +uv tool install --force "remove-ai-watermarks[visible,heif]" +``` + +## Invisible watermark removal + +Diffusion based removal needs the `diffusion` extra: + +```bash +uv tool install --force "remove-ai-watermarks[diffusion]" ``` The code supports CUDA, XPU, MPS, and CPU devices. A GPU is recommended because @@ -46,28 +63,73 @@ For the CUDA only Qwen Image plus Z-Image profile: uv tool install --force "remove-ai-watermarks[qwen-zimage]" ``` -The `qwen-zimage` extra includes the normal `gpu` dependencies. +The `qwen-zimage` extra includes the normal `diffusion` dependencies. -## Optional features +## Feature extras -Install only what you need: +Extras are composable. Install only the capabilities and file formats the +application actually uses: -| Extra | Adds | -| --- | --- | -| `migan` | MI-GAN ONNX fill backend | -| `lama` | big-LaMa ONNX fill backend | -| `detect` | Open DWT-DCT watermark decoder used by `identify` | -| `trustmark` | Adobe TrustMark decoder | -| `esrgan` | Real-ESRGAN upscaling before diffusion | -| `qwen-zimage` | CUDA only Qwen Image plus Z-Image pipeline | +| Extra | Capability | Automatically includes | Torch or model download | +| --- | --- | --- | --- | +| `pixels` | Shared BGR array and image-processing runtime | NumPy, headless OpenCV | No | +| `heif` | HEIC, HEIF, and AVIF pixel decoding | pillow-heif | No | +| `visible` | Visible mark detection, OpenCV inpainting, and manual erasing | `pixels` | No | +| `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No | +| `trustmark` | Adobe TrustMark detection | trustmark | Yes | +| `diffusion` | Diffusion-based invisible watermark removal | `pixels`, Torch, Diffusers | Yes | +| `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch | +| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch | +| `esrgan` | Real-ESRGAN upscaling before diffusion | `pixels`, spandrel | Yes | +| `qwen-zimage` | CUDA-only Qwen Image plus Z-Image pipeline | `diffusion`, DiffSynth | Yes | +| `all` | Every production feature | All rows above | Yes | +| `dev` | Tests, linting, typing, and upstream parity checks | `visible`, `detect`, upstream invisible-watermark | Yes, for parity tests | -Example: +Dependency composition: + +```mermaid +flowchart LR + visible --> pixels + detect --> pixels + diffusion --> pixels + migan --> visible + lama --> visible + esrgan --> pixels + qwen["qwen-zimage"] --> diffusion + heif + trustmark +``` + +`heif` and `trustmark` are independent branches. Combine them explicitly with +another feature when required. The `all` bundle contains every production +branch but never includes `dev`. + +Examples: ```bash +# Metadata plus torch-free DWT-DCT detection +uv tool install --force "remove-ai-watermarks[detect]" + +# Visible removal with HEIC/AVIF support and MI-GAN +uv tool install --force "remove-ai-watermarks[migan,heif]" + +# DWT-DCT and TrustMark detection without diffusion removal +uv tool install --force "remove-ai-watermarks[detect,trustmark]" + +# Every production capability +uv tool install --force "remove-ai-watermarks[all]" + +# An arbitrary minimal combination uv tool install --force "remove-ai-watermarks[migan,detect]" ``` -Some optional models download their weights on first use. +`heif` stays independent so applications that only process PNG, JPEG, or WebP +do not install libheif. `detect` uses the in-tree torch-free decoder and does +not install the upstream `invisible-watermark` package. Optional models download +their weights on first use. + +The old `gpu` and `remove` aliases are intentionally not provided. Use +`diffusion` and `visible` respectively. ## Install from the repository @@ -81,7 +143,7 @@ Add the feature groups required for your work: ```bash uv sync --frozen --extra dev -uv sync --frozen --extra dev --extra gpu +uv sync --frozen --extra dev --extra diffusion ``` Run commands from the repository root: diff --git a/docs/known-limitations.md b/docs/known-limitations.md index 72c9e7d..59f26fa 100644 --- a/docs/known-limitations.md +++ b/docs/known-limitations.md @@ -11,7 +11,8 @@ superseded experiments live in the research archive listed in Visible removal changes only the selected mask, but the hidden pixels still have to be reconstructed. -- OpenCV is fast and dependency free. It works well on flat backgrounds but +- OpenCV is fast and requires no model download. It works well on flat + backgrounds but can smear texture or repeated structure. - MI-GAN is a lighter learned backend. It can improve natural texture but may ghost or invent structure. @@ -162,11 +163,13 @@ The metadata path recognizes JPEG XL containers, but the visible and diffusion image paths do not list `.jxl` as a supported pixel format because the package does not include a JPEG XL pixel decoder. -### HEIC, HEIF, and AVIF use a Pillow fallback +### HEIC, HEIF, and AVIF pixel decoding uses an optional Pillow fallback OpenCV does not decode these formats in the project. `image_io.imread` falls -back to Pillow with `pillow-heif`. A corrupt or truncated file may still fail to -decode. +back to Pillow with `pillow-heif` when the `heif` extra is installed alongside +a pixel feature. The +default metadata path scans these containers without that plugin. A corrupt or +truncated file may still fail to decode. ### Some metadata removal requires ffmpeg diff --git a/docs/module-internals.md b/docs/module-internals.md index 9750e29..5d371e5 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -144,6 +144,11 @@ metadata extraction from verdict logic: - `identify` preserves the path-based API and adds the optional registered visible-mark and open invisible-watermark decoders after extraction. +The `detect` extra composes the shared `pixels` runtime with PyWavelets. Its +in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder preserves +the upstream matrix algorithm without installing Torch or non-headless OpenCV. +The upstream MIT notice ships inside the wheel under `licenses/`. + `is_ai_generated` is `True` or `None`; absence of evidence is not reported as a human-made verdict. `ai_source_kind` distinguishes fully generated content from AI-enhanced composites when the source metadata provides that distinction. @@ -379,7 +384,8 @@ Contracts: - `to_bgr` normalizes grayscale and alpha-bearing arrays. - `read_bgr_and_alpha` and `write_bgr_with_alpha` preserve the alpha plane. - `imwrite` returns a success flag; every caller must check it. -- HEIC, HEIF, and AVIF fall back to Pillow plus `pillow-heif`. +- HEIC, HEIF, and AVIF pixel reads fall back to Pillow plus `pillow-heif` from + the independent `heif` extra. Metadata scanning does not require that plugin. - A visible no-op can preserve the original file bytes. Regression coverage: diff --git a/docs/python-api.md b/docs/python-api.md index 5cf04fc..19bed61 100644 --- a/docs/python-api.md +++ b/docs/python-api.md @@ -3,8 +3,17 @@ Use the high level API for normal application integration. Low level detector and pipeline modules are intended for maintainers and specialized workflows. +Dependency groups are identical for the CLI and Python API. The default install +covers metadata extraction, normalization, verdict logic, and stripping. +Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection +uses `detect`; and diffusion removal uses `diffusion`. Add `heif` independently +when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See the complete +[feature-extra matrix](installation.md#feature-extras). + ## Remove visible marks +Install `remove-ai-watermarks[visible]` before using the visible-removal API. + ```python import remove_ai_watermarks as raiw @@ -68,6 +77,9 @@ result, removed = raiw.remove_visible(image, backend="cv2") ## Inspect provenance +The default installation evaluates file metadata. Add `visible`, `detect`, or +`trustmark` to enable the corresponding optional pixel signals. + Get the vendor keys used by visible removal: ```python @@ -172,6 +184,9 @@ as proof that metadata was removed. ## Remove invisible watermarks +Install `remove-ai-watermarks[diffusion]` for the standard pipelines or +`remove-ai-watermarks[qwen-zimage]` for the CUDA-only high-fidelity profile. + ```python from pathlib import Path diff --git a/docs/release-and-distribution.md b/docs/release-and-distribution.md index 1c98000..4265603 100644 --- a/docs/release-and-distribution.md +++ b/docs/release-and-distribution.md @@ -55,8 +55,9 @@ manual Homebrew formula update is the fallback when its automation is blocked. The conda job uses the published artifact rather than a locally built archive as the hash source and commits the resulting recipe change to `main`. Runtime -dependency mapping remains review-controlled: keep it aligned with the core -dependencies in `pyproject.toml`, and document any conda-forge package that is +dependency mapping remains review-controlled: keep it aligned with the default +metadata dependencies in `pyproject.toml`, do not copy optional pixel extras +into the default recipe, and document any conda-forge package that is unavailable and must be omitted. ## Source distribution boundary diff --git a/docs/supported-signals.md b/docs/supported-signals.md index d893e61..7fb5f2b 100644 --- a/docs/supported-signals.md +++ b/docs/supported-signals.md @@ -33,7 +33,7 @@ when you can select the affected area yourself. | Backend | Install | Behavior | | --- | --- | --- | -| `cv2` | Core package | Classical OpenCV inpainting | +| `cv2` | `remove-ai-watermarks[visible]` | Classical OpenCV inpainting | | `migan` | `remove-ai-watermarks[migan]` | MI-GAN through ONNX Runtime | | `lama` | `remove-ai-watermarks[lama]` | big-LaMa through ONNX Runtime | | `auto` | Depends on installed extras | Selects LaMa, then MI-GAN, then OpenCV | @@ -69,6 +69,9 @@ Pixel based image commands discover these extensions: - HEIC and HEIF; - AVIF. +HEIC, HEIF, and AVIF pixel decoding requires the independent `heif` extra in +addition to the selected pixel feature. Metadata scanning does not. + Metadata inspection and removal additionally have container paths for: - JPEG XL metadata; diff --git a/docs/synthid-robust-identity-research.md b/docs/synthid-robust-identity-research.md index b49efa9..3877a63 100644 --- a/docs/synthid-robust-identity-research.md +++ b/docs/synthid-robust-identity-research.md @@ -224,7 +224,7 @@ from the test set + this doc). ## 6. Integration cost (rough) -- New deps: `diffusers` already in the gpu extra; PhotoMaker ships as a `.bin` +- New deps: `diffusers` already in the diffusion extra; PhotoMaker ships as a `.bin` loaded via `pipeline.load_photomaker_adapter(...)`. The OpenCLIP encoder is the same one diffusers already pulls. No new heavy pip dep. - Weight download: PhotoMaker-V1 weights are ~3 GB. Add to the Modal HF volume diff --git a/packaging/conda/recipe.yaml b/packaging/conda/recipe.yaml index f68c5f9..89c61bb 100644 --- a/packaging/conda/recipe.yaml +++ b/packaging/conda/recipe.yaml @@ -25,13 +25,10 @@ requirements: run: - python >=${{ python_min }} - pillow >=10.0.0 - - pillow-heif >=0.13.0 - piexif >=1.1.3 - - numpy >=1.24.0 - - py-opencv >=4.8.0 - click >=8.0.0 - python-dotenv >=1.0.0 - # c2pa-python is a core PyPI dependency but is not packaged on conda-forge. + # c2pa-python is a default PyPI dependency but is not packaged on conda-forge. # The guarded import falls back to the built-in C2PA byte scanner when it is # absent. Add it here once a c2pa-python feedstock exists. @@ -52,11 +49,9 @@ about: homepage: https://github.com/wiltodelta/remove-ai-watermarks summary: Remove visible and invisible AI watermarks from images description: | - Detect and remove registered visible AI-provenance marks and strip - AI-provenance metadata (C2PA, EXIF, IPTC, and PNG text chunks) from images. - The core package covers the identify, metadata, visible, and erase command - surface. Optional pip extras add SynthID diffusion removal and additional - invisible-watermark detectors. + Inspect and strip AI-provenance metadata (C2PA, EXIF, IPTC, and PNG text + chunks) from images. Optional pip extras add visible watermark removal, + SynthID diffusion removal, and additional invisible-watermark detectors. license: Apache-2.0 license_file: LICENSE repository: https://github.com/wiltodelta/remove-ai-watermarks diff --git a/pyproject.toml b/pyproject.toml index 8242281..81e0175 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "remove-ai-watermarks" -version = "0.21.2" +version = "0.22.0" description = "AI watermark remover: strip visible and invisible AI watermarks (Gemini / Nano Banana sparkle, SynthID) and provenance metadata (C2PA, EXIF) from images" readme = "README.md" requires-python = ">=3.10.1" @@ -46,15 +46,7 @@ classifiers = [ ] dependencies = [ "pillow>=10.0.0", - # HEIC/AVIF pixel decode for the removal path (iPhone photos, modern exports): - # OpenCV cannot decode these containers, so image_io.imread falls back to Pillow - # and pillow-heif (bundled libheif, prebuilt wheels) registers the HEIF+AVIF - # openers. The metadata path already handles them via a plugin-free binary scan; - # this closes the same gap for the pixel path so `visible`/`all` work on them. - "pillow-heif>=0.13.0", "piexif>=1.1.3", - "numpy>=1.24.0", - "opencv-python-headless>=4.8.0", "click>=8.0.0", "python-dotenv>=1.0.0", # Official C2PA reader (Content Authenticity Initiative, MIT/Apache-2.0). The @@ -67,7 +59,25 @@ dependencies = [ ] [project.optional-dependencies] -gpu = [ +pixels = [ + "numpy>=1.24.0", + "opencv-python-headless>=4.8.0", +] +# Optional HEIC/AVIF pixel decode. Metadata scanning handles these containers +# without this plugin; combine `heif` with any pixel feature only when needed. +heif = [ + "pillow-heif>=0.13.0", +] +visible = ["remove-ai-watermarks[pixels]"] +# Open DWT-DCT watermarks used by Stable Diffusion / SDXL / FLUX. The in-tree +# decoder avoids the upstream invisible-watermark package's mandatory torch and +# non-headless OpenCV dependencies. +detect = [ + "remove-ai-watermarks[pixels]", + "PyWavelets>=1.1.1", +] +diffusion = [ + "remove-ai-watermarks[pixels]", "torch>=2.0.0", # The default PyPI torch wheel is a CPU/CUDA build. To drive an Intel GPU # (Arc / Data Center) via ``--device xpu`` you need an XPU-enabled torch @@ -75,7 +85,7 @@ gpu = [ # XPU build). Install that build first, then this extra (torch is then # already satisfied and won't be re-pulled): # pip install torch --index-url https://download.pytorch.org/whl/xpu - # pip install 'remove-ai-watermarks[gpu]' + # pip install 'remove-ai-watermarks[diffusion]' # uv users can target the ``pytorch-xpu`` index declared under [tool.uv]: # uv pip install torch --index-url https://download.pytorch.org/whl/xpu "diffusers>=0.38.0", @@ -95,23 +105,13 @@ gpu = [ ] # Full two-stage high-fidelity profile: Qwen-Image-2512 Lightning + DiffSynth # Canny ControlNet for the frame, then SAM-masked Z-Image Turbo face repair. -# CUDA-only and intentionally separate from the normal gpu extra because the +# CUDA-only and intentionally separate from the normal diffusion extra because the # additional model stack and DiffSynth runtime are large. qwen-zimage = [ - "remove-ai-watermarks[gpu]", + "remove-ai-watermarks[diffusion]", "diffsynth>=2.0.17,<3", "torchvision>=0.20.0", ] -# Open invisible-watermark (imwatermark) decoder for detecting the DWT-DCT -# watermarks embedded by Stable Diffusion / SDXL / FLUX. Optional because it -# pulls non-headless opencv AND torch (invisible-watermark declares torch a hard -# dependency, and WatermarkDecoder eagerly imports rivaGan -> torch at import -# time, so the dwtDct-only detect path still needs torch present even though it -# never runs on GPU). So `detect` alone pulls torch -- no need to add `gpu` for -# detection. identify() guards the import and skips the signal when absent. -detect = [ - "invisible-watermark>=0.2.0", -] # Adobe TrustMark decoder -- the open, keyless watermark behind Adobe Durable # Content Credentials (soft-binding alg ``com.adobe.trustmark.P``). Optional # because it pulls torch and downloads model weights on first use. identify() @@ -124,6 +124,7 @@ trustmark = [ # cached by huggingface_hub; it is never bundled in this repo. The default cv2 # eraser backend needs none of this. lama = [ + "remove-ai-watermarks[visible]", "onnxruntime>=1.16.0", "huggingface-hub>=0.20.0", ] @@ -133,6 +134,7 @@ lama = [ # memory-tight learned tier (vs big-LaMa's ~4.7 GB). Select it explicitly when # LaMa, the quality-first `auto` choice, is too large. Same runtime as `lama`. migan = [ + "remove-ai-watermarks[visible]", "onnxruntime>=1.16.0", "huggingface-hub>=0.20.0", ] @@ -146,12 +148,16 @@ migan = [ # weights are fetched with torch.hub (bundled with spandrel's torch), so no extra # download dependency is needed. esrgan = [ + "remove-ai-watermarks[pixels]", "spandrel>=0.3.0", ] dev = [ + "remove-ai-watermarks[visible]", + "remove-ai-watermarks[detect]", "pytest>=8.0.0", "pytest-cov>=4.1.0", "pytest-xdist>=3.5.0", + "packaging>=24.0", "ruff>=0.4.0", "pyright>=1.1.0", "invisible-watermark>=0.2.0", @@ -160,11 +166,11 @@ dev = [ "uv-outdated>=0.1.0; python_version >= '3.12'", "uv-secure>=0.12.0; python_version >= '3.12'", ] -all = ["remove-ai-watermarks[gpu,detect,trustmark,lama,migan,dev]"] +all = ["remove-ai-watermarks[visible,heif,detect,trustmark,diffusion,qwen-zimage,lama,migan,esrgan]"] # PyTorch Intel-GPU (XPU) wheel index. ``explicit = true`` keeps it inert for # the default CPU/CUDA install: uv consults it only when a torch install -# explicitly targets it (see the ``gpu`` extra comment), so it does not alter +# explicitly targets it (see the ``diffusion`` extra comment), so it does not alter # the locked CPU/CUDA resolution. Linux/Windows only -- no macOS XPU build. [[tool.uv.index]] name = "pytorch-xpu" diff --git a/src/remove_ai_watermarks/__init__.py b/src/remove_ai_watermarks/__init__.py index a96a921..a724efe 100644 --- a/src/remove_ai_watermarks/__init__.py +++ b/src/remove_ai_watermarks/__init__.py @@ -25,7 +25,7 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error") _warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*") -__version__ = "0.21.2" +__version__ = "0.22.0" __all__ = ["__version__", "remove_visible", "visible_provenance"] diff --git a/src/remove_ai_watermarks/cli.py b/src/remove_ai_watermarks/cli.py index 3f8676c..a680640 100644 --- a/src/remove_ai_watermarks/cli.py +++ b/src/remove_ai_watermarks/cli.py @@ -183,7 +183,7 @@ _upscaler_option = click.option( "--upscaler", type=click.Choice(["lanczos", "esrgan"]), default="lanczos", - help="How to upscale a small input to the --min-resolution floor: lanczos (default, cv2, no deps) or " + help="How to upscale a small input to the --min-resolution floor: lanczos (default, cv2, no model) or " "esrgan (Real-ESRGAN via the 'esrgan' extra; better detail, slower on CPU). Best for photo/texture " "content -- as a generic GAN with no face/glyph prior it can degrade faces (diffusion mitigates) and " "thin text, so lanczos stays the default. Falls back to lanczos if the extra is absent. Only when upscaling.", @@ -331,7 +331,7 @@ _visible_backend_option = click.option( default="auto", help="Fill backend for visible-mark removal (localize -> fill). auto: best available, " "LaMa > MI-GAN > cv2 (a learned backend needs the 'lama' or 'migan' extra; else cv2, " - "with a warning). cv2: classical inpaint (no deps, smears texture). migan: MI-GAN ONNX " + "with a warning). cv2: classical inpaint (no model download, smears texture). migan: MI-GAN ONNX " "(light, ~1 GB, the memory-tight pick). lama: big-LaMa ONNX (best quality, ~4.7 GB).", ) @@ -800,7 +800,7 @@ def _parse_region(spec: str) -> tuple[int, int, int, int]: "--backend", type=click.Choice(["cv2", "migan", "lama"]), default="cv2", - help="Inpaint backend. cv2: instant, no deps. migan: light ONNX MI-GAN, ~1 GB RAM, " + help="Inpaint backend. cv2: instant, no model download. migan: light ONNX MI-GAN, ~1 GB RAM, " "near-LaMa quality (extra 'migan'). lama: big-LaMa, best quality but ~4.7 GB RAM (extra 'lama').", ) @click.option("--inpaint-method", type=click.Choice(["telea", "ns"]), default="telea", help="cv2 inpaint method.") @@ -941,13 +941,14 @@ def cmd_invisible( """Remove invisible AI watermarks (SynthID, StableSignature, TreeRing). Uses diffusion-based regeneration. Requires GPU for reasonable speed. - Requires the [gpu] extra: pip install 'remove-ai-watermarks[gpu]' + Requires the [diffusion] extra: pip install 'remove-ai-watermarks[diffusion]' """ from remove_ai_watermarks.invisible_engine import is_available as invisible_available if not invisible_available(): console.print( - "Error: GPU dependencies not installed.\n Install them with: pip install 'remove-ai-watermarks[gpu]'" + "Error: Diffusion dependencies not installed.\n" + " Install them with: pip install 'remove-ai-watermarks[diffusion]'" ) raise SystemExit(1) @@ -1298,7 +1299,7 @@ def cmd_all( synthid_skipped = True console.print( " Warning: Skipped - GPU dependencies not installed.\n" - " Install them with: pip install 'remove-ai-watermarks[gpu]'" + " Install them with: pip install 'remove-ai-watermarks[diffusion]'" ) elif _should_skip_invisible_scrub(force, source): # No locally-detectable invisible watermark -> skip the destructive @@ -1404,7 +1405,7 @@ def cmd_all( " visible mark and metadata were stripped.\n" "\n" " Install the extra and rerun to remove it:\n" - " pip install 'remove-ai-watermarks[gpu]'\n" + " pip install 'remove-ai-watermarks[diffusion]'\n" " =====================================================================" ) raise SystemExit(1) @@ -1766,7 +1767,7 @@ def cmd_batch( f"\n WARNING: the invisible (SynthID) watermark was NOT removed on " f"{synthid_skipped_count} image(s) -- the GPU dependencies are not installed, " f"so those outputs still carry the invisible watermark.\n" - f" Install the extra and rerun: pip install 'remove-ai-watermarks[gpu]'" + f" Install the extra and rerun: pip install 'remove-ai-watermarks[diffusion]'" ) # Non-zero exit so a wrapping service detects an incomplete/failed run (batch used diff --git a/src/remove_ai_watermarks/dwt_dct.py b/src/remove_ai_watermarks/dwt_dct.py new file mode 100644 index 0000000..005b4d6 --- /dev/null +++ b/src/remove_ai_watermarks/dwt_dct.py @@ -0,0 +1,95 @@ +"""DWT-DCT decoder compatible with invisible-watermark's ``dwtDct`` path. + +Derived from ShieldMnt/invisible-watermark ``imwatermark/maxDct.py`` (MIT), +trimmed to the matrix path used by Stable Diffusion, SDXL, and FLUX. + +Copyright (c) 2021 ShieldMnt + +The complete upstream license is distributed in +``licenses/invisible-watermark-MIT.txt``. +""" + +# pyright: reportUnknownMemberType=false, reportUnknownArgumentType=false, reportUnknownVariableType=false, reportMissingTypeStubs=false + +from __future__ import annotations + +from typing import TYPE_CHECKING, Any + +import cv2 +import numpy as np +import pywt + +if TYPE_CHECKING: + from numpy.typing import NDArray + +_DEFAULT_SCALES = (0, 36, 36) +_DEFAULT_BLOCK = 4 + + +class _DecodeMaxDct: + """Extract frequency-domain bits using the upstream matrix algorithm.""" + + def __init__( + self, + wm_lengths: tuple[int, ...], + scales: tuple[int, int, int] = _DEFAULT_SCALES, + block: int = _DEFAULT_BLOCK, + ) -> None: + self._wm_lengths = wm_lengths + self._scales = scales + self._block = block + + def decode(self, bgr: NDArray[Any]) -> dict[int, NDArray[Any]]: + row, col, _channels = bgr.shape + yuv = cv2.cvtColor(bgr, cv2.COLOR_BGR2YUV) + + scores_by_length = {wm_len: [[] for _ in range(wm_len)] for wm_len in self._wm_lengths} + for channel in range(2): + if self._scales[channel] <= 0: + continue + ca1, _detail = pywt.dwt2(yuv[: row // 4 * 4, : col // 4 * 4, channel], "haar") + self._decode_frame(ca1, self._scales[channel], scores_by_length) + + return { + wm_len: np.asarray([float(np.asarray(score).mean()) if score else 0.0 for score in scores]) * 255 > 127 + for wm_len, scores in scores_by_length.items() + } + + def _decode_frame( + self, + frame: NDArray[Any], + scale: int, + scores_by_length: dict[int, list[list[int]]], + ) -> None: + row, col = frame.shape + bit_index = 0 + for i in range(row // self._block): + for j in range(col // self._block): + block = frame[ + i * self._block : i * self._block + self._block, + j * self._block : j * self._block + self._block, + ] + inferred = self._infer_bit(block, scale) + for wm_len, scores in scores_by_length.items(): + scores[bit_index % wm_len].append(inferred) + bit_index += 1 + + def _infer_bit(self, block: NDArray[Any], scale: int) -> int: + position = int(np.argmax(np.abs(block.flatten()[1:]))) + 1 + i, j = position // self._block, position % self._block + value = abs(float(block[i][j])) + return int((value % scale) > 0.5 * scale) + + +def decode_dwt_dct(bgr: NDArray[Any], wm_len: int) -> NDArray[Any]: + """Extract ``wm_len`` watermark bits from a BGR image.""" + return decode_dwt_dct_lengths(bgr, (wm_len,))[wm_len] + + +def decode_dwt_dct_lengths(bgr: NDArray[Any], wm_lengths: tuple[int, ...]) -> dict[int, NDArray[Any]]: + """Extract several watermark lengths with one DWT and block scan.""" + if bgr.size == 0 or min(bgr.shape[:2]) * max(bgr.shape[:2]) < 256 * 256: + raise RuntimeError("image too small, should be larger than 256x256") + if not wm_lengths or any(wm_len <= 0 for wm_len in wm_lengths): + raise ValueError("watermark lengths must be positive") + return _DecodeMaxDct(wm_lengths=tuple(dict.fromkeys(wm_lengths))).decode(bgr) diff --git a/src/remove_ai_watermarks/identify.py b/src/remove_ai_watermarks/identify.py index cfb4fdb..8683d90 100644 --- a/src/remove_ai_watermarks/identify.py +++ b/src/remove_ai_watermarks/identify.py @@ -722,8 +722,8 @@ def _visible_text_marks(image_path: Path, *, image: NDArray[Any] | None = None) def _invisible_watermark(image_path: Path) -> str | None: """Open invisible-watermark scheme name (SD/SDXL/FLUX) or None. - Optional: needs the imwatermark decoder (extra ``detect``). Returns None if - it is not installed or no known watermark decodes. + Optional: needs the torch-free DWT-DCT decoder (extra ``detect``). Returns + None if it is not installed or no known watermark decodes. """ from remove_ai_watermarks.invisible_watermark import detect_invisible_watermark @@ -761,6 +761,9 @@ def _collect_visible_signals( image = imread(image_path) except Exception as exc: # cv2 missing - detectors fall back / no-op logger.debug("visible-mark decode unavailable: %s", exc) + return platform + if image is None: + return platform sparkle_conf = _visible_sparkle(image_path, image=image) if sparkle_conf is not None and sparkle_conf >= _SPARKLE_THRESHOLD: @@ -1087,8 +1090,8 @@ def identify( image_path: Path to the image (PNG, JPEG, WebP, or ISOBMFF container). check_visible: Also run the registered visible-mark detectors through cv2. Set False for a metadata-only, dependency-light scan. - check_invisible: Also decode open invisible watermarks (SD/SDXL/FLUX) via - the optional imwatermark library. No-op when it is not installed. + check_invisible: Also decode optional open invisible watermarks + (SD/SDXL/FLUX). No-op when the decoder extra is not installed. File-backed metadata extraction runs first. The extracted evidence is then evaluated independently, followed by the optional pixel-backed visible and diff --git a/src/remove_ai_watermarks/invisible_engine.py b/src/remove_ai_watermarks/invisible_engine.py index de6ac59..0e38d9e 100644 --- a/src/remove_ai_watermarks/invisible_engine.py +++ b/src/remove_ai_watermarks/invisible_engine.py @@ -4,7 +4,7 @@ Wraps the vendored noai-watermark code for removing invisible AI watermarks (SynthID, StableSignature, TreeRing) via diffusion-based regeneration. This module requires the 'gpu' extra dependencies: - uv pip install 'remove-ai-watermarks[gpu]' + uv pip install 'remove-ai-watermarks[diffusion]' """ # cv2/torch boundary: this engine wraps cv2 (resize/imwrite/cvtColor) and the @@ -226,7 +226,7 @@ class InvisibleEngine: input size, so this is a transparent quality boost; it adds time and memory on small inputs. Ignored on a min > max misconfig. upscaler: How to upscale a small input to the ``min_resolution`` floor: - ``"lanczos"`` (default, cv2, no deps) or ``"esrgan"`` (Real-ESRGAN + ``"lanczos"`` (default, cv2, no model download) or ``"esrgan"`` (Real-ESRGAN via the ``esrgan`` extra). Only applies when UPscaling (the floor case); a ``max_resolution`` downscale always uses Lanczos. Falls back to Lanczos if the extra is absent. diff --git a/src/remove_ai_watermarks/invisible_watermark.py b/src/remove_ai_watermarks/invisible_watermark.py index 45cb57a..bf74538 100644 --- a/src/remove_ai_watermarks/invisible_watermark.py +++ b/src/remove_ai_watermarks/invisible_watermark.py @@ -14,21 +14,20 @@ source: The watermark is fragile: it does NOT survive JPEG re-encoding or resizing (verified -- gone after JPEG q90), so detection works only on pristine PNG -originals. Absence is never proof. Requires the optional ``invisible-watermark`` -package (extra: ``detect``); ``detect_invisible_watermark`` returns None when it -is not installed. +originals. Absence is never proof. Requires the optional ``detect`` extra; +``detect_invisible_watermark`` returns None when it is not installed. """ -# imwatermark ships no type stubs (like cv2); its decoder returns are Unknown. -# Relax the untyped-library diagnostics for this thin wrapper module only. +# The optional numeric libraries do not provide complete types for this path. # pyright: reportMissingTypeStubs=false, reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false from __future__ import annotations import logging -from typing import TYPE_CHECKING, cast +from typing import TYPE_CHECKING if TYPE_CHECKING: + from collections.abc import Iterable from pathlib import Path logger = logging.getLogger(__name__) @@ -49,10 +48,10 @@ _MATCH_SD1_FRAC = 0.92 # fraction of the 136 string bits that must match def is_available() -> bool: - """True if the optional imwatermark decoder is installed.""" + """True when all dependencies for the optional DWT-DCT decoder exist.""" from .optional_deps import module_available - return module_available("imwatermark") + return module_available("cv2", "numpy", "pywt") def _bits_match(value: int, ref: int, width: int = 48) -> int: @@ -68,6 +67,20 @@ def _bytes_match_frac(a: bytes, b: bytes) -> float: return 1.0 - diff / (8 * len(b)) +def _bits_to_int(bits: Iterable[object]) -> int: + value = 0 + for bit in bits: + value = (value << 1) | int(bool(bit)) + return value + + +def _bits_to_bytes(bits: Iterable[object], nbytes: int) -> bytes: + import numpy as np + + packed = np.packbits([int(bool(bit)) for bit in bits]) + return bytes(int(value) for value in packed[:nbytes]) + + def detect_invisible_watermark(image_path: Path) -> str | None: """Return the embedding scheme name if a known open watermark is decoded. @@ -78,32 +91,26 @@ def detect_invisible_watermark(image_path: Path) -> str | None: """ if not is_available(): return None - from imwatermark import WatermarkDecoder - from remove_ai_watermarks import image_io + from remove_ai_watermarks.dwt_dct import decode_dwt_dct_lengths img = image_io.imread(image_path) if img is None: return None - # 48-bit fixed-message watermarks (SDXL, FLUX.2). try: - bits = WatermarkDecoder("bits", 48).decode(img, "dwtDct") - value = 0 - for bit in bits: - value = (value << 1) | (1 if bit else 0) - for name, ref in _BITS_48.items(): - if _bits_match(value, ref) >= _MATCH_48: - return name + decoded = decode_dwt_dct_lengths(img, (48, 8 * len(_SD1_STRING))) except Exception as exc: # decode can fail on tiny images - logger.debug("48-bit watermark decode failed for %s: %s", image_path, exc) + logger.debug("watermark decode failed for %s: %s", image_path, exc) + return None - # 136-bit default string watermark (SD 1.x / 2.x). - try: - raw = cast("bytes", WatermarkDecoder("bytes", 8 * len(_SD1_STRING)).decode(img, "dwtDct")) - if _bytes_match_frac(raw, _SD1_STRING) >= _MATCH_SD1_FRAC: - return "Stable Diffusion 1.x / 2.x" - except Exception as exc: - logger.debug("string watermark decode failed for %s: %s", image_path, exc) + value = _bits_to_int(decoded[48]) + for name, ref in _BITS_48.items(): + if _bits_match(value, ref) >= _MATCH_48: + return name + + raw = _bits_to_bytes(decoded[8 * len(_SD1_STRING)], len(_SD1_STRING)) + if _bytes_match_frac(raw, _SD1_STRING) >= _MATCH_SD1_FRAC: + return "Stable Diffusion 1.x / 2.x" return None diff --git a/src/remove_ai_watermarks/licenses/invisible-watermark-MIT.txt b/src/remove_ai_watermarks/licenses/invisible-watermark-MIT.txt new file mode 100644 index 0000000..e09163b --- /dev/null +++ b/src/remove_ai_watermarks/licenses/invisible-watermark-MIT.txt @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 ShieldMnt + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/src/remove_ai_watermarks/noai/__init__.py b/src/remove_ai_watermarks/noai/__init__.py index b8e241e..399ebef 100644 --- a/src/remove_ai_watermarks/noai/__init__.py +++ b/src/remove_ai_watermarks/noai/__init__.py @@ -7,7 +7,7 @@ is exposed **lazily** via PEP 562 ``__getattr__``: importing a light submodule (e.g. ``noai.c2pa`` / ``noai.constants`` from ``identify``) must NOT eagerly pull ``watermark_remover``, which imports torch + diffusers at module top. Keeping this lazy is what lets ``import remove_ai_watermarks.identify`` stay cheap (~36 MB, no -torch) even in a full install where the ``gpu``/``detect`` extras are present -- +torch) even in a full install where the ``diffusion`` extra is present -- otherwise the mere presence of torch in the env inflated identify to ~420 MB and risked OOM on a 512 MB host. """ diff --git a/src/remove_ai_watermarks/noai/c2pa.py b/src/remove_ai_watermarks/noai/c2pa.py index 6005476..81da1de 100644 --- a/src/remove_ai_watermarks/noai/c2pa.py +++ b/src/remove_ai_watermarks/noai/c2pa.py @@ -43,7 +43,7 @@ from remove_ai_watermarks.noai.constants import ( logger = logging.getLogger(__name__) -# Official C2PA reader (c2pa-python, a core dependency). It is the primary, +# Official C2PA reader (c2pa-python, a default dependency). It is the primary, # spec-tracking manifest parser; the hand-rolled caBX/CBOR scanner below stays as # a fallback for synthetic/partial blobs the validator rejects. The import is # guarded so a partially-broken install degrades to the byte-scan rather than diff --git a/src/remove_ai_watermarks/noai/constants.py b/src/remove_ai_watermarks/noai/constants.py index 2431c75..541a77f 100644 --- a/src/remove_ai_watermarks/noai/constants.py +++ b/src/remove_ai_watermarks/noai/constants.py @@ -7,7 +7,7 @@ so adding a new AI tool or metadata key requires updating only this file. from typing import NamedTuple # Supported image formats for the pixel/removal path (CLI input validation + batch -# discovery). PNG/JPEG/WebP decode+encode via cv2; HEIC/HEIF/AVIF via the core +# discovery). PNG/JPEG/WebP decode+encode via cv2; HEIC/HEIF/AVIF via the optional # pillow-heif dep (image_io.imread Pillow fallback + imwrite _pil_write), so batch # now picks them up and the CLI no longer warns on an iPhone HEIC. JPEG-XL is left # out on purpose -- it is metadata/strip-only (no pixel decoder without pillow-jxl). diff --git a/src/remove_ai_watermarks/noai/watermark_remover.py b/src/remove_ai_watermarks/noai/watermark_remover.py index c1ae671..b5c3031 100644 --- a/src/remove_ai_watermarks/noai/watermark_remover.py +++ b/src/remove_ai_watermarks/noai/watermark_remover.py @@ -476,8 +476,9 @@ class WatermarkRemover: """Turn off the diffusers default invisible watermarker on an SDXL pipeline. diffusers embeds an open "Stable Diffusion XL" DWT-DCT invisible watermark on - EVERY SDXL output whenever ``invisible-watermark`` is installed (the ``detect`` - extra). A watermark REMOVER must not re-stamp a detectable AI watermark, or the + EVERY SDXL output whenever ``invisible-watermark`` is installed (kept as a + development parity dependency). A watermark REMOVER must not re-stamp a + detectable AI watermark, or the cleaned output re-reads as AI (``identify`` -> "Open invisible watermark: Stable Diffusion XL"). Shared by both SDXL loaders; the ``ControlNetModel`` sub-model and the Qwen loader never call it (only the pipeline accepts the kwarg). diff --git a/src/remove_ai_watermarks/upscaler.py b/src/remove_ai_watermarks/upscaler.py index 9a386d8..a545f13 100644 --- a/src/remove_ai_watermarks/upscaler.py +++ b/src/remove_ai_watermarks/upscaler.py @@ -4,7 +4,7 @@ Mirrors ``region_eraser``'s optional-backend pattern: ``is_available()`` guards ``spandrel`` import, a lazy singleton (double-checked lock) holds the loaded model, and the weights download on first use (cached by ``torch.hub``) -- they are never bundled. -The DEFAULT upscaler stays Lanczos (cv2, no deps); this is opt-in via the ``esrgan`` +The DEFAULT upscaler stays Lanczos (cv2, no model download); this is opt-in via the ``esrgan`` extra and feeds the ``--upscaler esrgan`` path. ``spandrel`` is a pure model-loader (MIT) with NO basicsr dependency -- it pulls only torch/torchvision/safetensors/numpy/ einops -- so it sidesteps the basicsr / ``torchvision.transforms.functional_tensor`` diff --git a/tests/test_cli.py b/tests/test_cli.py index ed28efa..95ca22d 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -542,7 +542,7 @@ class TestAllCommand: result = runner.invoke(main, ["all", str(sample_png), "-o", str(output)]) assert result.exit_code != 0, result.output assert "NOT removed" in result.output - assert "remove-ai-watermarks[gpu]" in result.output + assert "remove-ai-watermarks[diffusion]" in result.output assert output.exists() # visible + metadata still produced a file def test_all_reports_metadata_that_survived_stripping(self, runner, sample_png, tmp_path): @@ -933,21 +933,21 @@ class TestBatchCommand: class TestGpuHintMarkup: - """The GPU-extra install hint must reach the user with the ``[gpu]`` token + """The diffusion install hint must reach the user with the ``[diffusion]`` token intact (plain output prints it verbatim, with no markup parsing).""" def test_invisible_install_hint_keeps_gpu_extra(self, runner, sample_png): with patch("remove_ai_watermarks.invisible_engine.is_available", return_value=False): result = runner.invoke(main, ["invisible", str(sample_png)]) assert result.exit_code != 0 - assert "remove-ai-watermarks[gpu]" in result.output + assert "remove-ai-watermarks[diffusion]" in result.output def test_all_install_hint_keeps_gpu_extra(self, runner, sample_png): # The `all` pipeline skips the invisible step with a warning that carries - # the same hint; it must keep the [gpu] extra too. + # the same hint; it must keep the [diffusion] extra too. with patch("remove_ai_watermarks.invisible_engine.is_available", return_value=False): result = runner.invoke(main, ["all", str(sample_png)]) - assert "remove-ai-watermarks[gpu]" in result.output + assert "remove-ai-watermarks[diffusion]" in result.output class TestEraseCommand: diff --git a/tests/test_identify.py b/tests/test_identify.py index c683d21..fe6171f 100644 --- a/tests/test_identify.py +++ b/tests/test_identify.py @@ -782,6 +782,15 @@ class TestIdentifyVisibleTextMarks: identify(tmp_clean_png, check_visible=True, check_invisible=False) assert mock_imread.call_count == 1 + def test_missing_pixel_extra_preserves_metadata_verdict(self, tmp_png_with_ai_metadata: Path): + import remove_ai_watermarks.image_io as image_io + + with patch.object(image_io, "imread", side_effect=ModuleNotFoundError("No module named 'cv2'")): + report = identify(tmp_png_with_ai_metadata, check_visible=True, check_invisible=False) + + assert report.is_ai_generated is True + assert report.confidence == "high" + # ── Caveats and serialization ─────────────────────────────────────── @@ -989,7 +998,7 @@ class TestIdentifyC2paDevice: from remove_ai_watermarks.invisible_watermark import is_available as _wm_available # noqa: E402 -@pytest.mark.skipif(not _wm_available(), reason="invisible-watermark not installed") +@pytest.mark.skipif(not _wm_available(), reason="detect extra not installed") class TestIdentifyInvisibleWatermark: def _sdxl_watermarked(self, tmp_path: Path) -> Path: import cv2 diff --git a/tests/test_invisible_engine.py b/tests/test_invisible_engine.py index 94888d2..b44f8ce 100644 --- a/tests/test_invisible_engine.py +++ b/tests/test_invisible_engine.py @@ -18,9 +18,9 @@ class TestIsAvailable: assert isinstance(result, bool) def test_available_reflects_dependencies(self): - """is_available() is True iff torch + diffusers (the gpu extra) import. + """is_available() is True iff torch + diffusers (the diffusion extra) import. - Must not assume the full stack: the core+dev CI env has no diffusers. + Must not assume the full stack: the default+dev CI env has no diffusers. """ import importlib.util @@ -212,7 +212,7 @@ class TestCannyControlImage: def test_edge_map_is_3channel_rgb(self): if not is_available(): - pytest.skip("gpu extra (torch/diffusers) not installed") + pytest.skip("diffusion extra (torch/diffusers) not installed") import numpy as np from remove_ai_watermarks.noai.watermark_remover import WatermarkRemover diff --git a/tests/test_invisible_watermark.py b/tests/test_invisible_watermark.py index 2cf77fb..a516146 100644 --- a/tests/test_invisible_watermark.py +++ b/tests/test_invisible_watermark.py @@ -1,8 +1,7 @@ -"""Tests for open invisible-watermark (imwatermark) detection. +"""Tests for open DWT-DCT watermark detection. -Each known scheme is round-tripped: embed its exact upstream pattern with the -encoder, then assert the detector names it. Skipped entirely if the optional -``invisible-watermark`` package is not installed. +The upstream encoder supplies known watermarks, while the in-tree decoder must +both identify them and match the upstream decoder bit for bit. """ from __future__ import annotations @@ -25,7 +24,7 @@ from remove_ai_watermarks.invisible_watermark import ( is_available, ) -pytestmark = pytest.mark.skipif(not is_available(), reason="invisible-watermark not installed") +pytestmark = pytest.mark.skipif(not is_available(), reason="detect extra not installed") def _base_image() -> np.ndarray: @@ -61,6 +60,20 @@ class TestHelpers: class TestDetect: + def test_in_tree_decoder_matches_upstream(self, tmp_path: Path): + from imwatermark import WatermarkDecoder + + from remove_ai_watermarks.dwt_dct import decode_dwt_dct + from remove_ai_watermarks.image_io import imread + + path = _write_bits_watermark(tmp_path, _BITS_48["Stable Diffusion XL"]) + image = imread(path) + assert image is not None + + upstream = np.asarray(WatermarkDecoder("bits", 48).decode(image, "dwtDct"), dtype=bool) + ours = np.asarray(decode_dwt_dct(image, wm_len=48), dtype=bool) + assert np.array_equal(ours, upstream) + def test_detects_sdxl(self, tmp_path: Path): path = _write_bits_watermark(tmp_path, _BITS_48["Stable Diffusion XL"]) assert detect_invisible_watermark(path) == "Stable Diffusion XL" diff --git a/tests/test_noai.py b/tests/test_noai.py index 25f47cc..f0d707c 100644 --- a/tests/test_noai.py +++ b/tests/test_noai.py @@ -52,8 +52,8 @@ class TestConstants: assert ".jpg" in SUPPORTED_FORMATS def test_supported_formats_include_heic_avif(self): - # HEIC/AVIF are first-class on the pixel path now (read+write via pillow-heif), - # so batch discovers them and the CLI does not warn. + # HEIC/AVIF are first-class when the visible pixel extra is installed + # (read+write via pillow-heif), so batch discovers them without a warning. assert {".heic", ".heif", ".avif"} <= SUPPORTED_FORMATS def test_supported_formats_exclude_jpeg_xl(self): diff --git a/tests/test_packaging.py b/tests/test_packaging.py new file mode 100644 index 0000000..28fd6b8 --- /dev/null +++ b/tests/test_packaging.py @@ -0,0 +1,67 @@ +"""Published dependency boundaries.""" + +from __future__ import annotations + +from importlib.metadata import metadata, requires + +from packaging.requirements import Requirement +from packaging.utils import canonicalize_name + + +def _requirement_names(extra: str | None = None) -> set[str]: + selected_extra = extra or "" + parsed = (Requirement(value) for value in requires("remove-ai-watermarks") or []) + return { + canonicalize_name(requirement.name) + for requirement in parsed + if requirement.marker is None or requirement.marker.evaluate({"extra": selected_extra}) + } + + +def test_default_install_is_metadata_focused(): + default = _requirement_names() + + assert { + "c2pa-python", + "click", + "piexif", + "pillow", + "python-dotenv", + } <= default + assert { + "invisible-watermark", + "numpy", + "opencv-python-headless", + "pillow-heif", + "torch", + "trustmark", + }.isdisjoint(default) + + +def test_pixels_extra_owns_shared_numeric_dependencies(): + assert { + "numpy", + "opencv-python-headless", + } <= _requirement_names("pixels") + + +def test_file_format_and_detector_dependencies_are_independent(): + assert "pillow-heif" in _requirement_names("heif") + assert "pywavelets" in _requirement_names("detect") + + +def test_extras_use_capability_names_without_legacy_aliases(): + extras = set(metadata("remove-ai-watermarks").get_all("Provides-Extra") or []) + + assert {"pixels", "heif", "visible", "detect", "diffusion"} <= extras + assert {"gpu", "remove", "detect-pywavelets"}.isdisjoint(extras) + + +def test_production_all_does_not_include_development_tools(): + assert { + "pyright", + "pytest", + "pytest-cov", + "pytest-xdist", + "ruff", + }.isdisjoint(_requirement_names("all")) diff --git a/tests/test_platform.py b/tests/test_platform.py index 37799ba..2464649 100644 --- a/tests/test_platform.py +++ b/tests/test_platform.py @@ -238,7 +238,7 @@ class TestQwenKwargs: """_build_qwen_kwargs is pure (no torch); guards the Qwen-Image call shape. watermark_remover imports torch under a try/except, so the module (and this pure - helper) imports fine in the core+dev CI env where torch is absent. + helper) imports fine in the default+dev CI env where torch is absent. """ def test_uses_true_cfg_not_guidance_scale(self): @@ -431,7 +431,7 @@ class TestAvailability: def test_watermark_removal_available(self): # Reflects the actual environment: True iff torch + diffusers (the gpu - # extra) are importable. The core+dev CI env has no diffusers, so this + # extra) are importable. The default+dev CI env has no diffusers, so this # must not assume the full stack is present. import importlib.util diff --git a/uv.lock b/uv.lock index 8366acd..5f299e9 100644 --- a/uv.lock +++ b/uv.lock @@ -3187,78 +3187,100 @@ wheels = [ [[package]] name = "remove-ai-watermarks" -version = "0.21.2" +version = "0.22.0" source = { editable = "." } dependencies = [ { name = "c2pa-python" }, { name = "click" }, - { name = "numpy" }, - { name = "opencv-python-headless" }, { name = "piexif" }, { name = "pillow" }, - { name = "pillow-heif" }, { name = "python-dotenv" }, ] [package.optional-dependencies] all = [ { name = "accelerate" }, + { name = "diffsynth" }, { name = "diffusers" }, { name = "huggingface-hub" }, - { name = "invisible-watermark" }, + { name = "numpy" }, { name = "onnxruntime", version = "1.24.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "onnxruntime", version = "1.27.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "pyright" }, - { name = "pytest" }, - { name = "pytest-cov" }, - { name = "pytest-xdist" }, - { name = "ruff" }, + { name = "opencv-python-headless" }, + { name = "pillow-heif" }, + { name = "pywavelets", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "pywavelets", version = "1.9.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, { name = "safetensors" }, + { name = "spandrel" }, { name = "tokenizers" }, { name = "torch" }, + { name = "torchvision" }, { name = "transformers" }, { name = "trustmark" }, - { name = "uv-outdated", marker = "python_full_version >= '3.12'" }, - { name = "uv-secure", marker = "python_full_version >= '3.12'" }, ] detect = [ - { name = "invisible-watermark" }, + { name = "numpy" }, + { name = "opencv-python-headless" }, + { name = "pywavelets", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "pywavelets", version = "1.9.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, ] dev = [ { name = "invisible-watermark" }, + { name = "numpy" }, + { name = "opencv-python-headless" }, + { name = "packaging" }, { name = "pyright" }, { name = "pytest" }, { name = "pytest-cov" }, { name = "pytest-xdist" }, + { name = "pywavelets", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "pywavelets", version = "1.9.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, { name = "ruff" }, { name = "uv-outdated", marker = "python_full_version >= '3.12'" }, { name = "uv-secure", marker = "python_full_version >= '3.12'" }, ] -esrgan = [ - { name = "spandrel" }, -] -gpu = [ +diffusion = [ { name = "accelerate" }, { name = "diffusers" }, + { name = "numpy" }, + { name = "opencv-python-headless" }, { name = "safetensors" }, { name = "tokenizers" }, { name = "torch" }, { name = "transformers" }, ] +esrgan = [ + { name = "numpy" }, + { name = "opencv-python-headless" }, + { name = "spandrel" }, +] +heif = [ + { name = "pillow-heif" }, +] lama = [ { name = "huggingface-hub" }, + { name = "numpy" }, { name = "onnxruntime", version = "1.24.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "onnxruntime", version = "1.27.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "opencv-python-headless" }, ] migan = [ { name = "huggingface-hub" }, + { name = "numpy" }, { name = "onnxruntime", version = "1.24.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "onnxruntime", version = "1.27.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "opencv-python-headless" }, +] +pixels = [ + { name = "numpy" }, + { name = "opencv-python-headless" }, ] qwen-zimage = [ { name = "accelerate" }, { name = "diffsynth" }, { name = "diffusers" }, + { name = "numpy" }, + { name = "opencv-python-headless" }, { name = "safetensors" }, { name = "tokenizers" }, { name = "torch" }, @@ -3268,44 +3290,57 @@ qwen-zimage = [ trustmark = [ { name = "trustmark" }, ] +visible = [ + { name = "numpy" }, + { name = "opencv-python-headless" }, +] [package.metadata] requires-dist = [ - { name = "accelerate", marker = "extra == 'gpu'", specifier = ">=0.25.0" }, + { name = "accelerate", marker = "extra == 'diffusion'", specifier = ">=0.25.0" }, { name = "c2pa-python", specifier = ">=0.35.0" }, { name = "click", specifier = ">=8.0.0" }, { name = "diffsynth", marker = "extra == 'qwen-zimage'", specifier = ">=2.0.17,<3" }, - { name = "diffusers", marker = "extra == 'gpu'", specifier = ">=0.38.0" }, + { name = "diffusers", marker = "extra == 'diffusion'", specifier = ">=0.38.0" }, { name = "huggingface-hub", marker = "extra == 'lama'", specifier = ">=0.20.0" }, { name = "huggingface-hub", marker = "extra == 'migan'", specifier = ">=0.20.0" }, - { name = "invisible-watermark", marker = "extra == 'detect'", specifier = ">=0.2.0" }, { name = "invisible-watermark", marker = "extra == 'dev'", specifier = ">=0.2.0" }, - { name = "numpy", specifier = ">=1.24.0" }, + { name = "numpy", marker = "extra == 'pixels'", specifier = ">=1.24.0" }, { name = "onnxruntime", marker = "extra == 'lama'", specifier = ">=1.16.0" }, { name = "onnxruntime", marker = "extra == 'migan'", specifier = ">=1.16.0" }, - { name = "opencv-python-headless", specifier = ">=4.8.0" }, + { name = "opencv-python-headless", marker = "extra == 'pixels'", specifier = ">=4.8.0" }, + { name = "packaging", marker = "extra == 'dev'", specifier = ">=24.0" }, { name = "piexif", specifier = ">=1.1.3" }, { name = "pillow", specifier = ">=10.0.0" }, - { name = "pillow-heif", specifier = ">=0.13.0" }, + { name = "pillow-heif", marker = "extra == 'heif'", specifier = ">=0.13.0" }, { name = "pyright", marker = "extra == 'dev'", specifier = ">=1.1.0" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.0" }, { name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.1.0" }, { name = "pytest-xdist", marker = "extra == 'dev'", specifier = ">=3.5.0" }, { name = "python-dotenv", specifier = ">=1.0.0" }, - { name = "remove-ai-watermarks", extras = ["gpu"], marker = "extra == 'qwen-zimage'" }, - { name = "remove-ai-watermarks", extras = ["gpu", "detect", "trustmark", "lama", "migan", "dev"], marker = "extra == 'all'" }, + { name = "pywavelets", marker = "extra == 'detect'", specifier = ">=1.1.1" }, + { name = "remove-ai-watermarks", extras = ["detect"], marker = "extra == 'dev'" }, + { name = "remove-ai-watermarks", extras = ["diffusion"], marker = "extra == 'qwen-zimage'" }, + { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'detect'" }, + { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'diffusion'" }, + { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'esrgan'" }, + { name = "remove-ai-watermarks", extras = ["pixels"], marker = "extra == 'visible'" }, + { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'dev'" }, + { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'lama'" }, + { name = "remove-ai-watermarks", extras = ["visible"], marker = "extra == 'migan'" }, + { name = "remove-ai-watermarks", extras = ["visible", "heif", "detect", "trustmark", "diffusion", "qwen-zimage", "lama", "migan", "esrgan"], marker = "extra == 'all'" }, { name = "ruff", marker = "extra == 'dev'", specifier = ">=0.4.0" }, - { name = "safetensors", marker = "extra == 'gpu'" }, + { name = "safetensors", marker = "extra == 'diffusion'" }, { name = "spandrel", marker = "extra == 'esrgan'", specifier = ">=0.3.0" }, - { name = "tokenizers", marker = "extra == 'gpu'", specifier = ">=0.22,<0.23" }, - { name = "torch", marker = "extra == 'gpu'", specifier = ">=2.0.0" }, + { name = "tokenizers", marker = "extra == 'diffusion'", specifier = ">=0.22,<0.23" }, + { name = "torch", marker = "extra == 'diffusion'", specifier = ">=2.0.0" }, { name = "torchvision", marker = "extra == 'qwen-zimage'", specifier = ">=0.20.0" }, - { name = "transformers", marker = "extra == 'gpu'", specifier = ">=5,<6" }, + { name = "transformers", marker = "extra == 'diffusion'", specifier = ">=5,<6" }, { name = "trustmark", marker = "extra == 'trustmark'", specifier = ">=0.8.0" }, { name = "uv-outdated", marker = "python_full_version >= '3.12' and extra == 'dev'", specifier = ">=0.1.0" }, { name = "uv-secure", marker = "python_full_version >= '3.12' and extra == 'dev'", specifier = ">=0.12.0" }, ] -provides-extras = ["gpu", "qwen-zimage", "detect", "trustmark", "lama", "migan", "esrgan", "dev", "all"] +provides-extras = ["pixels", "heif", "visible", "detect", "diffusion", "qwen-zimage", "trustmark", "lama", "migan", "esrgan", "dev", "all"] [[package]] name = "requests"