Merge main into video watermark pipeline

This commit is contained in:
Victor Kuznetsov
2026-07-31 11:13:59 -07:00
42 changed files with 1430 additions and 435 deletions
+44 -6
View File
@@ -9,15 +9,38 @@ remove-ai-watermarks [OPTIONS] COMMAND [ARGS]
Run `remove-ai-watermarks COMMAND --help` for the complete option list and
defaults. This page focuses on choosing the right command.
## Command dependency map
| Command or signal | Required installation |
| --- | --- |
| `metadata` and metadata-only `identify` | Default package |
| Visible signals in `identify` | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
| Open DWT-DCT signals in `identify` | `remove-ai-watermarks[detect]` |
| Adobe TrustMark signals in `identify` | `remove-ai-watermarks[trustmark]` |
| `visible` and `erase` with OpenCV | `remove-ai-watermarks[visible]` (`pixels` is the minimal runtime) |
| `visible` or `erase` with MI-GAN | `remove-ai-watermarks[migan]` |
| `visible` or `erase` with big-LaMa | `remove-ai-watermarks[lama]` |
| `invisible` | `remove-ai-watermarks[diffusion]` |
| `invisible --pipeline qwen-zimage` | `remove-ai-watermarks[qwen-zimage]` |
| `video metadata` and `video identify --no-visible` | Default package |
| `video identify`, `video visible`, and visible/all batch modes | `remove-ai-watermarks[video]` |
| `video invisible` and `video all --invisible` | `remove-ai-watermarks[video,diffusion]` |
| HEIC/HEIF/AVIF pixel input | Add `remove-ai-watermarks[heif]` |
| Every production command and backend | `remove-ai-watermarks[all]` |
`batch` requires the same extra as its selected mode. Extras can be combined in
one installation, for example `remove-ai-watermarks[visible,detect,heif]`.
## Inspect an image
```bash
remove-ai-watermarks identify image.png
```
`identify` combines supported metadata and pixel signals into one provenance
report. When no signal is found, it reports the origin as unknown. It does not
claim the image is clean.
`identify` always inspects supported metadata. When pixel extras are installed,
it also evaluates supported visible and invisible pixel signals. When no signal
is found, it reports the origin as unknown. It does not claim the image is
clean.
Machine readable output:
@@ -36,6 +59,8 @@ invisible pixel detectors. Metadata inspection still runs.
## Remove known visible marks
Install `remove-ai-watermarks[visible]` before using `visible` or `erase`.
```bash
remove-ai-watermarks visible image.png -o clean.png
```
@@ -127,6 +152,13 @@ non-ISOBMFF audio and video path.
## Identify and clean video
Install the video pixel and timestamp runtime for visible identification,
removal, and the complete pipeline:
```bash
uv tool install --force "remove-ai-watermarks[video]"
```
Inspect every locally supported video signal:
```bash
@@ -210,7 +242,7 @@ command.
## Remove video SynthID
```bash
uv tool install --force "remove-ai-watermarks[gpu]"
uv tool install --force "remove-ai-watermarks[video,diffusion]"
remove-ai-watermarks video invisible input.mp4 -o clean.mp4
```
@@ -311,7 +343,7 @@ failed encode does not overwrite an existing result.
Install the diffusion dependencies first:
```bash
uv tool install --force "remove-ai-watermarks[gpu]"
uv tool install --force "remove-ai-watermarks[diffusion]"
```
Then run:
@@ -375,6 +407,12 @@ It is a memory strategy, not a guarantee of better quality.
## Run the full pipeline
The `all` command and the `all` installation extra are separate concepts. The
command runs every applicable stage. Installing `remove-ai-watermarks[all]`
makes every production backend available; a smaller installation such as
`remove-ai-watermarks[visible,diffusion]` can also run the command with fewer
optional backends.
```bash
remove-ai-watermarks all image.png -o clean.png
```
@@ -387,7 +425,7 @@ The command runs:
The visible options and diffusion options are also available on `all`.
If diffusion is required but the `gpu` extra is unavailable, `all` still
If diffusion is required but the `diffusion` extra is unavailable, `all` still
writes the result of the visible and metadata stages, prints a prominent
warning, and exits with code 1. This prevents a partial result from being
reported as complete.
+35
View File
@@ -0,0 +1,35 @@
# Development
Read this reference for environment setup, dependency recovery, CI behavior, and fixture policy. The always-loaded invariants remain in [`.claude/rules/development.md`](../.claude/rules/development.md).
## Local environment
- Use `uv sync --frozen --extra dev` and add only the feature extras needed for the task.
- Do not use `uv pip install` for development tools. It can re-resolve `uv.lock` outside the compatible ML dependency set.
- A default-only sync removes every pixel and model package by design. Package imports remain light through lazy exports.
- On an unreliable connection, sync `dev` plus only the required feature extras, such as `diffusion`, and run the checks directly instead of downloading every optional learned backend.
- Run `uv` from the repository root or it may create a bare environment without the project dependencies.
The optional TrustMark decoder downloads weights into its installed package directory. After pruning that extra, a leftover weights directory can make availability checks see an empty namespace package. If Pyright reports an unknown `TrustMark` import and `find_spec("trustmark")` returns a loader-less spec, remove that regenerable remnant from the active virtual environment and resync.
## CI
`.github/workflows/test.yml` runs Ruff and a cross-platform supported-Python test matrix with default plus development dependencies. Diffusion and model-running tests skip in that matrix; metadata, identification, visible removal, the DWT-DCT decoder, and the OpenCV eraser remain covered across operating systems.
Keep `uv.lock` compatible with `uv sync --frozen`. Dependency pull-request checks use GitHub's merge result against current `main`; if `main` moves, merge it locally and rerun the full gate because a newer linter can expose stale directives in later code.
Release and distribution behavior is canonical in [`release-and-distribution.md`](release-and-distribution.md).
## Fixture and data policy
[`../data/README.md`](../data/README.md) is the source of truth:
- executable provenance fixtures live under `data/fixtures/`;
- minimal controlled detector inputs live under `data/calibration/`;
- canonical provider-oracle originals and their manifests live under `data/synthid/`;
- evaluation-only ground truth lives under `data/evaluations/`;
- runtime detector assets live in the package; unregistered research candidates remain outside the shipped wheel.
Store each binary once. Point tests and manifests at its canonical path. Keep generated and cleaned outputs outside the repository and retain only reproducible public records allowed by the data policy.
Use synthetic byte blobs for unsupported format paths and deterministic generated negatives where a real negative fixture is unnecessary. Detection and removal tests must preserve their format-specific invariants.
+106 -22
View File
@@ -2,15 +2,17 @@
Python 3.10.1 or newer is required.
## Core install
## Default metadata mode
The core package provides:
The default package provides:
- provenance inspection;
- visible watermark removal with OpenCV;
- manual region erasing with OpenCV;
- AI metadata inspection and removal.
It installs Pillow, piexif, and c2pa-python for reading metadata directly from
files. It does not install NumPy, OpenCV, pillow-heif, Torch, diffusion models,
or invisible-watermark decoders.
Install it as an isolated command with uv:
```bash
@@ -29,12 +31,44 @@ You can also install the Homebrew package on macOS or Linux:
brew install wiltodelta/tap/remove-ai-watermarks
```
## Invisible watermark removal
## Visible watermark removal
Diffusion based removal needs the `gpu` extra:
Visible mark detection, OpenCV inpainting, and manual region erasing need the
`visible` extra:
```bash
uv tool install --force "remove-ai-watermarks[gpu]"
uv tool install --force "remove-ai-watermarks[visible]"
```
Add `heif` only when the pixel path must decode HEIC, HEIF, or AVIF:
```bash
uv tool install --force "remove-ai-watermarks[visible,heif]"
```
## Video processing
Video metadata inspection and stripping work with the default package. Stable
visible-mark identification and removal, full video cleaning, and visible/all
batch modes need the `video` extra:
```bash
uv tool install --force "remove-ai-watermarks[video]"
```
The extra includes the visible pixel runtime and PyAV for preserving variable
frame timestamps. Video SynthID regeneration also needs the diffusion stack:
```bash
uv tool install --force "remove-ai-watermarks[video,diffusion]"
```
## Invisible watermark removal
Diffusion based removal needs the `diffusion` extra:
```bash
uv tool install --force "remove-ai-watermarks[diffusion]"
```
The code supports CUDA, XPU, MPS, and CPU devices. A GPU is recommended because
@@ -46,28 +80,78 @@ For the CUDA only Qwen Image plus Z-Image profile:
uv tool install --force "remove-ai-watermarks[qwen-zimage]"
```
The `qwen-zimage` extra includes the normal `gpu` dependencies.
The `qwen-zimage` extra includes the normal `diffusion` dependencies.
## Optional features
## Feature extras
Install only what you need:
Extras are composable. Install only the capabilities and file formats the
application actually uses:
| Extra | Adds |
| --- | --- |
| `migan` | MI-GAN ONNX fill backend |
| `lama` | big-LaMa ONNX fill backend |
| `detect` | Open DWT-DCT watermark decoder used by `identify` |
| `trustmark` | Adobe TrustMark decoder |
| `esrgan` | Real-ESRGAN upscaling before diffusion |
| `qwen-zimage` | CUDA only Qwen Image plus Z-Image pipeline |
| Extra | Capability | Automatically includes | Torch or model download |
| --- | --- | --- | --- |
| `pixels` | Shared BGR array and image-processing runtime | NumPy, headless OpenCV | No |
| `heif` | HEIC, HEIF, and AVIF pixel decoding | pillow-heif | No |
| `visible` | Visible mark detection, OpenCV inpainting, and manual erasing | `pixels` | No |
| `video` | Visible video identification/removal and timestamp preservation | `visible`, PyAV | No |
| `detect` | Open DWT-DCT detection for Stable Diffusion, SDXL, and FLUX | `pixels`, PyWavelets | No |
| `trustmark` | Adobe TrustMark detection | trustmark | Yes |
| `diffusion` | Diffusion-based invisible watermark removal | `pixels`, Torch, Diffusers | Yes |
| `migan` | MI-GAN ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
| `lama` | big-LaMa ONNX fill backend | `visible`, ONNX Runtime | Model download, no Torch |
| `esrgan` | Real-ESRGAN upscaling before diffusion | `pixels`, spandrel | Yes |
| `qwen-zimage` | CUDA-only Qwen Image plus Z-Image pipeline | `diffusion`, DiffSynth | Yes |
| `all` | Every production feature | All rows above | Yes |
| `dev` | Tests, linting, typing, and upstream parity checks | `visible`, `detect`, upstream invisible-watermark | Yes, for parity tests |
Example:
Dependency composition:
```mermaid
flowchart LR
visible --> pixels
video --> visible
detect --> pixels
diffusion --> pixels
migan --> visible
lama --> visible
esrgan --> pixels
qwen["qwen-zimage"] --> diffusion
heif
trustmark
```
`heif` and `trustmark` are independent branches. Combine them explicitly with
another feature when required. The `all` bundle contains every production
branch but never includes `dev`.
Examples:
```bash
# Metadata plus torch-free DWT-DCT detection
uv tool install --force "remove-ai-watermarks[detect]"
# Visible removal with HEIC/AVIF support and MI-GAN
uv tool install --force "remove-ai-watermarks[migan,heif]"
# Visible video removal with preserved timestamps
uv tool install --force "remove-ai-watermarks[video]"
# DWT-DCT and TrustMark detection without diffusion removal
uv tool install --force "remove-ai-watermarks[detect,trustmark]"
# Every production capability
uv tool install --force "remove-ai-watermarks[all]"
# An arbitrary minimal combination
uv tool install --force "remove-ai-watermarks[migan,detect]"
```
Some optional models download their weights on first use.
`heif` stays independent so applications that only process PNG, JPEG, or WebP
do not install libheif. `detect` uses the in-tree torch-free decoder and does
not install the upstream `invisible-watermark` package. Optional models download
their weights on first use.
The old `gpu` and `remove` aliases are intentionally not provided. Use
`diffusion` and `visible` respectively.
## Install from the repository
@@ -81,7 +165,7 @@ Add the feature groups required for your work:
```bash
uv sync --frozen --extra dev
uv sync --frozen --extra dev --extra gpu
uv sync --frozen --extra dev --extra diffusion
```
Run commands from the repository root:
@@ -133,4 +217,4 @@ found. A missing signal does not prove that the image is clean. If you know the
image came from a relevant generator, use `--force`.
If the CLI reports that diffusion dependencies are unavailable, install the
`gpu` extra.
`diffusion` extra. Video SynthID removal needs both `video` and `diffusion`.
+7 -4
View File
@@ -11,7 +11,8 @@ superseded experiments live in the research archive listed in
Visible removal changes only the selected mask, but the hidden pixels still
have to be reconstructed.
- OpenCV is fast and dependency free. It works well on flat backgrounds but
- OpenCV is fast and requires no model download. It works well on flat
backgrounds but
can smear texture or repeated structure.
- MI-GAN is a lighter learned backend. It can improve natural texture but may
ghost or invent structure.
@@ -200,11 +201,13 @@ The metadata path recognizes JPEG XL containers, but the visible and diffusion
image paths do not list `.jxl` as a supported pixel format because the package
does not include a JPEG XL pixel decoder.
### HEIC, HEIF, and AVIF use a Pillow fallback
### HEIC, HEIF, and AVIF pixel decoding uses an optional Pillow fallback
OpenCV does not decode these formats in the project. `image_io.imread` falls
back to Pillow with `pillow-heif`. A corrupt or truncated file may still fail to
decode.
back to Pillow with `pillow-heif` when the `heif` extra is installed alongside
a pixel feature. The
default metadata path scans these containers without that plugin. A corrupt or
truncated file may still fail to decode.
### Some metadata removal requires ffmpeg
+19 -4
View File
@@ -309,9 +309,23 @@ Regression coverage:
### Provenance report
[`identify.py`](../src/remove_ai_watermarks/identify.py) combines metadata,
registered visible marks, and optional open invisible-watermark decoders into a
`ProvenanceReport`.
[`identify.py`](../src/remove_ai_watermarks/identify.py) separates file-backed
metadata extraction from verdict logic:
- `extract_provenance_evidence` reads the supported metadata signals into
`ProvenanceEvidence`.
- `evidence_from_metadata_record` normalizes an externally collected nested
metadata record into the same evidence type without file access. Diagnostic
values under `error` and `kind` are excluded from evidence while nested raw
bytes remain available through encoded binary fields.
- `identify_from_evidence` evaluates that evidence without reopening the source.
- `identify` preserves the path-based API and adds the optional registered
visible-mark and open invisible-watermark decoders after extraction.
The `detect` extra composes the shared `pixels` runtime with PyWavelets. Its
in-tree [`dwt_dct.py`](../src/remove_ai_watermarks/dwt_dct.py) decoder preserves
the upstream matrix algorithm without installing Torch or non-headless OpenCV.
The upstream MIT notice ships inside the wheel under `licenses/`.
`is_ai_generated` is `True` or `None`; absence of evidence is not reported as a
human-made verdict. `ai_source_kind` distinguishes fully generated content from
@@ -548,7 +562,8 @@ Contracts:
- `to_bgr` normalizes grayscale and alpha-bearing arrays.
- `read_bgr_and_alpha` and `write_bgr_with_alpha` preserve the alpha plane.
- `imwrite` returns a success flag; every caller must check it.
- HEIC, HEIF, and AVIF fall back to Pillow plus `pillow-heif`.
- HEIC, HEIF, and AVIF pixel reads fall back to Pillow plus `pillow-heif` from
the independent `heif` extra. Metadata scanning does not require that plugin.
- A visible no-op can preserve the original file bytes.
Regression coverage:
+65 -2
View File
@@ -3,8 +3,18 @@
Use the high level API for normal application integration. Low level detector
and pipeline modules are intended for maintainers and specialized workflows.
Dependency groups are identical for the CLI and Python API. The default install
covers metadata extraction, normalization, verdict logic, and stripping.
Array/pixel APIs use `pixels`; visible removal uses `visible`; DWT-DCT detection
uses `detect`; diffusion removal uses `diffusion`; and visible video processing
uses `video`. Video SynthID removal combines `video` and `diffusion`. Add `heif`
independently when path-based pixel APIs must decode HEIC, HEIF, or AVIF. See
the complete [feature-extra matrix](installation.md#feature-extras).
## Remove visible marks
Install `remove-ai-watermarks[visible]` before using the visible-removal API.
```python
import remove_ai_watermarks as raiw
@@ -68,6 +78,9 @@ result, removed = raiw.remove_visible(image, backend="cv2")
## Inspect provenance
The default installation evaluates file metadata. Add `visible`, `detect`, or
`trustmark` to enable the corresponding optional pixel signals.
Get the vendor keys used by visible removal:
```python
@@ -88,8 +101,8 @@ print(report.platform)
print(report.signals)
```
Use `check_visible=False` and `check_invisible=False` for metadata only
inspection:
Use `check_visible=False` and `check_invisible=False` for metadata-only
inspection through the compatible path-based API:
```python
report = identify(
@@ -99,6 +112,48 @@ report = identify(
)
```
Extraction and detection are also available as separate steps. This is useful
when a file-reading worker collects the metadata once and another component
evaluates the resulting evidence:
```python
from remove_ai_watermarks.identify import (
extract_provenance_evidence,
identify_from_evidence,
)
evidence = extract_provenance_evidence(Path("input.png"))
report = identify_from_evidence(evidence)
```
If metadata was collected by another component, normalize its nested record
without reopening the original file:
```python
from remove_ai_watermarks.identify import (
evidence_from_metadata_record,
identify_from_evidence,
)
record = {
"pil": {"info:parameters": "Steps: 20, Sampler: Euler"},
"exif": {"0th": {"Software": "Stable Diffusion"}},
}
evidence = evidence_from_metadata_record(record, path=Path("input.png"))
report = identify_from_evidence(evidence)
```
The normalizer recursively preserves text and byte values. It also decodes
strings prefixed with `hex:` and fields named `base64` or ending in
`_base64`. Diagnostic values under `error` and `kind` are ignored because they
describe the collector rather than the source file. Pass a C2PA manifest-store
dictionary in `record["c2pa_store"]`, or through the explicit
`c2pa_manifest_store` argument.
`identify_from_evidence` does not reopen the source file. It evaluates metadata
only; registered visible marks and pixel-backed invisible watermarks remain in
the path-based `identify` call.
## Strip metadata
```python
@@ -131,6 +186,8 @@ as proof that metadata was removed.
## Identify and clean video
The high level video API supports MP4, MOV, M4V, WebM, MKV, AVI, and FLV:
metadata-only calls work with the default install, while visible identification,
removal, and the complete pipeline require `remove-ai-watermarks[video]`.
```python
import remove_ai_watermarks as raiw
@@ -212,6 +269,9 @@ removal path.
## Remove video SynthID
Install `remove-ai-watermarks[video,diffusion]` before using the video SynthID
API.
```python
import remove_ai_watermarks as raiw
@@ -320,6 +380,9 @@ to 8-bit SDR.
## Remove invisible watermarks
Install `remove-ai-watermarks[diffusion]` for the standard pipelines or
`remove-ai-watermarks[qwen-zimage]` for the CUDA-only high-fidelity profile.
```python
from pathlib import Path
+6 -5
View File
@@ -55,11 +55,12 @@ manual Homebrew formula update is the fallback when its automation is blocked.
The conda job uses the published artifact rather than a locally built archive
as the hash source and commits the resulting recipe change to `main`. Runtime
dependency mapping remains review-controlled: keep it aligned with the core
dependencies in `pyproject.toml`, and document any conda-forge package that is
unavailable and must be omitted. PyPI's version-split PyAV dependency maps to
`av >=16` in conda: the solver selects the Python-3.10-compatible build or the
current line according to the environment.
dependency mapping remains review-controlled: keep it aligned with the default
metadata dependencies in `pyproject.toml`, do not copy optional pixel extras
into the default recipe, and document any conda-forge package that is
unavailable and must be omitted.
The optional `video` extra carries PyAV with Python-version-specific bounds; it
does not belong in the default metadata-focused conda recipe.
## Source distribution boundary
+4 -1
View File
@@ -54,7 +54,7 @@ independent frame fill.
| Backend | Install | Behavior |
| --- | --- | --- |
| `cv2` | Core package | Classical OpenCV inpainting |
| `cv2` | `remove-ai-watermarks[visible]` | Classical OpenCV inpainting |
| `migan` | `remove-ai-watermarks[migan]` | MI-GAN through ONNX Runtime; practical learned CPU video tier |
| `lama` | `remove-ai-watermarks[lama]` | big-LaMa through ONNX Runtime; offline video quality tier |
| `auto` | Depends on installed extras | Selects LaMa, then MI-GAN, then OpenCV |
@@ -93,6 +93,9 @@ Pixel based image commands discover these extensions:
- HEIC and HEIF;
- AVIF.
HEIC, HEIF, and AVIF pixel decoding requires the independent `heif` extra in
addition to the selected pixel feature. Metadata scanning does not.
Metadata inspection and removal additionally have container paths for:
- JPEG XL metadata;
+1 -1
View File
@@ -224,7 +224,7 @@ from the test set + this doc).
## 6. Integration cost (rough)
- New deps: `diffusers` already in the gpu extra; PhotoMaker ships as a `.bin`
- New deps: `diffusers` already in the diffusion extra; PhotoMaker ships as a `.bin`
loaded via `pipeline.load_photomaker_adapter(...)`. The OpenCLIP encoder is the
same one diffusers already pulls. No new heavy pip dep.
- Weight download: PhotoMaker-V1 weights are ~3 GB. Add to the Modal HF volume
+1 -1
View File
@@ -33,7 +33,7 @@ Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC*
**Stripped on removal too:** `remove_ai_metadata` calls `_scrub_ai_exif` on
JPEG EXIF, which deletes the xAI Signature and UUID Artist pair plus supported
AI generator values while retaining unrelated camera and editor EXIF. The
shared `_is_xai_signature_pair` helper is the single source of truth for the
shared `xai_signature_pair` helper is the single source of truth for the
pair. On the ISOBMFF path, `blank_ai_exif_tokens` provides the corresponding
in-place scrub for supported EXIF values, TC260 AIGC blocks, and the xAI pair.
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`).