Compare commits

..
19 Commits
Author SHA1 Message Date
test-userandClaude Opus 4.7 d24d8a4b14 Extract _target_size helper + regression-test native resolution (v0.5.4)
The native-vs-downscale decision in InvisibleEngine.remove_watermark (the
issue #10/#15 fix: max_resolution=0 must not pre-downscale, since any
downscale both loses quality and lets SynthID survive) had no test. Extract
it into a pure helper invisible_engine._target_size(w, h, max_resolution)
and cover it with tests/test_invisible_engine.py::TestTargetSize so a
re-introduced forced downscale fails CI instead of silently regressing #15.

Also:
- Clamp the short side to >=1 in _target_size: extreme aspect ratios (e.g.
  5000x3 with --max-resolution 1024) truncated it to 0 and crashed
  image.resize(). Pre-existing in the inline math; fixed now that it is a
  named, tested function.
- Consolidate the two duplicated temp-file save blocks into one
  unconditional save (behavior unchanged: the EXIF-transposed image is
  still always persisted before WatermarkRemover reloads it by path), and
  drop the now-redundant `_tmp_path is not None` guard in finally.
- Bump version 0.5.3 -> 0.5.4 (pyproject, __init__, uv.lock); document the
  helper as the regression guard in CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 14:09:33 -07:00
test-userandClaude Opus 4.7 28fe13db8f Document native-res MPS OOM -> CPU-fallback behavior in limitations
Concrete data point from the 2026-05-25 gpt-image SDXL run: native
1254x1254 fp32 OOMs at the UNet step (not just VAE) on a 20 GB MPS
ceiling, and img2img_runner auto-falls back to CPU and completes
(slow, weight-identical, still defeats SynthID). enable_vae_tiling()
alone does not prevent it. Fast Mac workarounds: fp16 on MPS or
--max-resolution; neither is the default.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 13:57:13 -07:00
test-userandClaude Opus 4.7 59d72c5db7 Record verified gpt-image-2 SynthID-cleaned chain in corpus
Add manifest row for the 4ef377bd -> f6dd47a5 chain: a gpt-image-2 sample
(openai.com/verify: SynthID + C2PA detected) cleaned via v0.5.3 `all` at
native 1254x1254 (prod-equivalent SDXL base, strength 0.05, 50 steps).
openai.com/verify reports SynthID NOT detected after the run, re-confirming
that the #10 native-resolution default defeats OpenAI SynthID and resolving
the #15 root cause (older SD-1.5/768px downscale default did not).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 13:55:10 -07:00
test-userandClaude Opus 4.7 e27f24f520 test(samples): commit real Doubao fixture + AIGC real-sample test
data/samples/doubao-1.png is the real #13 sample: carries the China TC260
<TC260:AIGC> XMP label and a visible '豆包AI生成' text mark (bottom-right).
Grounds the AIGC detection on a real file (alongside the synthetic tests)
and serves as the fixture for visible-watermark removal work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 12:37:15 -07:00
test-userandClaude Opus 4.7 1afc1e60ef test(samples): add real Doubao TC260 AIGC reference sample
2048x2048 PNG carrying China's TC260 <TC260:AIGC> label; identify reports
it as a China AIGC-labeled generator (TC260). Reference fixture for manual
re-verification of the TC260 detection path -- the automated tests use
synthetic blobs, so nothing depends on this file being present.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 12:36:28 -07:00
test-userandClaude Opus 4.7 d45f0806a0 chore(release): v0.5.3 — detect China TC260 AIGC label (Doubao)
- feat(identify): detect the China TC260 <TC260:AIGC> XMP label (Doubao
  and other China-served generators); reports platform + ContentProducer.
  Removal already strips it via the existing metadata cleaner.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 12:30:40 -07:00
test-userandClaude Opus 4.7 c7f0d71f90 feat(identify): detect China TC260 AIGC label (Doubao et al.)
China-served generators embed an XMP <TC260:AIGC>{"Label":"1",...} block
(China's mandatory AI-content labeling, TC260 standard). Doubao (ByteDance)
uses it -- verified on the real #13 sample. It's none of C2PA / SynthID /
imwatermark / IPTC, so identify() previously returned unknown.

- metadata: AIGC_MARKERS + aigc_label() (json-decodes the HTML-entity-encoded
  block); has_ai_metadata + get_ai_metadata now surface it.
- identify: new 'aigc' signal -> is_ai True, platform 'China AIGC-labeled
  generator (TC260; e.g. Doubao)', carries the ContentProducer code.
- Container-agnostic raw-byte scan, so it covers the whole China-AIGC ecosystem
  (Jimeng/Kling/Qwen/Ernie share the standard).
- Tests: synthetic TC260 block (metadata + identify). Docs updated.

Addresses #13.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 12:29:51 -07:00
test-userandClaude Opus 4.7 768d997ef0 docs: scope SynthID provenance claims to source-verified facts
Threat model: replace the unverified deployment list (Gemini 3 Pro /
Nano Banana Pro / Imagen 4 / Veo) with the source-verified scope -- SynthID
across Imagen / Veo / Lyria plus Gemini app outputs (>10B items by Dec 2025),
and attribute the 136-bit payload to the paper's SynthID-O variant.

openai-images-2 sample: note the file predates the 19 May 2026 SynthID
rollout across ChatGPT / Codex / API, and that openai.com/verify is now the
public oracle (still no local decoder).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 12:18:13 -07:00
test-userandClaude Opus 4.7 37769453a9 chore(release): v0.5.2 — native-resolution invisible removal (fixes #10)
- fix(invisible): process at native resolution by default; the forced
  downscale-to-1024 -> upscale-back round-trip was the main quality loss
  (#10). Matches the raiw.cc backend (fal fast-sdxl = sdxl-base-1.0).
  New --max-resolution opt-in cap for GPU/MPS memory.
- docs: verified fal checkpoint, native-res, gpt-image-2 SynthID.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 10:00:25 -07:00
test-userandClaude Opus 4.7 f3ecea348a docs: gpt-image-2 carries SynthID (no local decoder; openai.com/verify oracle)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 09:59:16 -07:00
test-userandClaude Opus 4.7 fb42295b3a docs: record verified fal fast-sdxl checkpoint + native-resolution updates
- fal's llms.txt confirms fast-sdxl is stabilityai/stable-diffusion-xl-base-1.0,
  the exact checkpoint the local CLI defaults to -> local == prod weights.
  Recorded in CLAUDE.md and README.
- README How it works + sample README: replace the old downscale->upscale
  description with native-resolution processing (matches the #10 fix);
  document --max-resolution as an opt-in OOM cap.
- README roadmap: idna already bumped (uv-secure clean).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 09:57:03 -07:00
test-userandClaude Opus 4.7 18740969ae fix(invisible): process at native resolution by default
The invisible pipeline force-downscaled inputs >1024px to 1024 before
diffusion, then upscaled the result back -- a lossy round-trip that was
the main cause of the quality loss reported in #10. The hosted raiw.cc
backend (fal fast-sdxl) does no pre-downscale, and at strength ~0.05
SDXL img2img doesn't need it.

Default is now native resolution (max_resolution=0). New --max-resolution
flag (invisible / all / batch) re-introduces an opt-in long-side cap only
to bound GPU/MPS memory on very large inputs.

Addresses #10. End-to-end quality/removal not re-verified locally (no GPU
here); matches raiw-app's proven production config.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 09:50:06 -07:00
test-userandClaude Opus 4.7 93c664f7fb docs: sync README + corpus map with v0.5.x detection coverage
- README Features: add the identify / provenance-detection capability.
- README Supported models: add FLUX, Stability AI, Microsoft/Bing
  (MAI-Image), Meta AI rows; note SD/SDXL/FLUX imwatermark is locally
  detectable; add a detection note pointing at identify.
- corpus README per-platform map: add Stability / Ideogram / Recraft /
  Krea-FLUX rows + an imwatermark column; correct Bing (MAI-Image,
  signs 'Microsoft'); note imwatermark fires only on pristine pipeline
  output, not re-hosts/exports.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 09:32:03 -07:00
test-userandClaude Opus 4.7 20a754ef98 chore(release): v0.5.1 — security + bug fixes
- security: bump idna 3.11 -> 3.16 (GHSA-65pc-fj4g-8rjx)
- fix(ctrlregen): correct module import paths (#11, @neosun100)
- fix(cli): preserve alpha channel through visible/all/batch (#8, @rlorenzo)
- fix(cli): safer re-exec via -m instead of repr(sys.argv) -c string (#9, @eskibars)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 09:21:53 -07:00
test-userandClaude Opus 4.7 e60f183d29 style(ctrlregen): sort imports (follow-up to #11)
#11 left the import block un-sorted (ruff I001); reorder so diffusers
precedes the local ctrlregen import.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 09:21:53 -07:00
Rex Lorenzo d091b9f822 fix(cli): preserve alpha channel in visible-watermark pipeline
`cv2.imread(..., IMREAD_COLOR)` was silently stripping the alpha channel
on RGBA inputs, and `cv2.imwrite` then wrote opaque 3-channel PNGs — so
images with transparent backgrounds came back with an opaque-black (or
white) background and the sparkle area baked in as a solid blob.

Read the source with `IMREAD_UNCHANGED`, keep the alpha plane out of the
detection/inpaint path (those still operate on BGR), and rejoin alpha at
save time. The detected watermark bbox is also zeroed in the alpha plane
so the sparkle region becomes transparent rather than an opaque artifact.

Applies to `visible`, `all`, and `batch` modes. RGB-only inputs and JPEG
outputs are unaffected.
2026-05-25 09:18:39 -07:00
test-userandClaude Opus 4.7 e8d698814a fix(cli): re-exec via -m instead of a repr(sys.argv) -c string
Based on #9 by @eskibars. Replaces the os.execl(..., "-c", repr-string)
restart (used after the CUDA-torch auto-install) with os.execv -m, so we
no longer build an exec string from repr(sys.argv). Forwards sys.argv[1:]
only: under -m Python sets argv[0] to the module path, so passing the full
argv would re-inject the program name as a spurious Click argument.

Verified: python -m remove_ai_watermarks.cli --version works; test_cli green.

Closes #9

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 09:15:55 -07:00
Neo 孫 663c8c64ca fix(ctrlregen): correct module import paths (#11)
The CtrlRegen engine module references a non-existent top-level package
'ctrlregen' in four locations:

  src/remove_ai_watermarks/noai/ctrlregen/engine.py
    L39:  from ctrlregen.pipeline import CustomCtrlRegenPipeline
    L57:  from ctrlregen.color import color_match
    L242: from ctrlregen.tiling import resize_center_crop, run_tiled
    L267: from ctrlregen.tiling import resize_center_crop

These should be absolute imports of the package's own subpackage. As a
result, the top-level try/except sets _HAS_DIFFUSERS=False and
_HAS_COLOR_MATCHER=False even when the [gpu] extra is correctly
installed, and is_ctrlregen_available() always returns False.

Effect on users: invoking the ctrlregen profile crashes with

  ImportError: Failed to auto-install missing dependencies:
  controlnet-aux, color-matcher, safetensors

regardless of whether those packages are installed. The auto-install
fallback also fails in uv-managed venvs (uv does not ship pip in the
venv by default), so the error path is unrecoverable.

Reproduction (before fix):
  uv sync --all-extras
  uv run remove-ai-watermarks invisible <image> --pipeline ctrlregen
  # → ImportError as above

Fix: change the four imports to use the package-qualified path
(matching the absolute-import style used elsewhere in the codebase,
e.g. watermark_remover.py).

Verified post-fix on Linux/CUDA (NVIDIA L40S):
  - is_ctrlregen_available() returns True
  - CtrlRegen pipeline loads, downloads weights, and runs end-to-end
  - Tile-based path (image > 512px) processes 6 tiles cleanly
  - 142 existing pytest tests still pass
2026-05-25 09:11:20 -07:00
test-userandClaude Opus 4.7 b45e2a5731 chore(deps): bump idna 3.11 -> 3.16 (GHSA-65pc-fj4g-8rjx)
Fixes the uv-secure abort that stopped maintain.sh: idna 3.11 had
GHSA-65pc-fj4g-8rjx (fix in 3.15). uv lock --upgrade-package idna pulls
3.16; uv-secure now reports no vulnerabilities. Lock-only change, 266
tests still pass. Updates the stale CLAUDE.md note.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 09:04:24 -07:00
19 changed files with 595 additions and 122 deletions

No files matched your search

+6 -5
View File
@@ -12,9 +12,9 @@ You are a **principal Python engineer** maintaining a CLI tool and library for r
## Test and lint
- `bash maintain.sh` — uv-outdated, uv-secure, ruff check/fix, ruff format, pyright, pytest -n auto
- `maintain.sh` does not currently finish green (pre-existing, not per-change): `uv-secure` aborts on a fixable transitive `idna` vuln, and strict pyright carries debt in `remove_ai_metadata` / `cli.py` (untyped piexif/PIL/click/rich). To gate a change, run `uv run ruff check`, `uv run pyright <changed files>`, `uv run pytest` directly.
- `maintain.sh` may not finish fully green (pre-existing, not per-change): strict pyright carries debt in `remove_ai_metadata` / `cli.py` (untyped piexif/PIL/click/rich). (`uv-secure` is clean since idna was bumped 3.11 -> 3.16, fixing GHSA-65pc-fj4g-8rjx.) To gate a change, run `uv run ruff check`, `uv run pyright <changed files>`, `uv run pytest` directly.
- Run `uv run` from the repo root — from another cwd it falls back to a bare env without numpy/cv2/torch.
- Metadata/C2PA tests assert against real committed fixtures in `data/samples/` (`chatgpt-*.png` = OpenAI C2PA, `firefly-1.png` = Adobe, `not-ai-*` = clean); synthetic byte blobs cover the JPEG/ISOBMFF format paths.
- Metadata/C2PA tests assert against real committed fixtures in `data/samples/` (`chatgpt-*.png` = OpenAI C2PA, `firefly-1.png` = Adobe, `mj-*` = Midjourney IPTC, `doubao-1.png` = ByteDance Doubao with the China TC260 `<TC260:AIGC>` XMP label **and** a visible "豆包AI生成" text mark bottom-right, `not-ai-*` = clean); synthetic byte blobs cover the JPEG/ISOBMFF format paths.
- SynthID reference corpus: `scripts/synthid_corpus.py` ingests labeled images into `data/synthid_corpus/` (`manifest.csv` tracked, `images/` gitignored); see its README for the collection protocol and verification oracles.
## Configuration
@@ -29,7 +29,7 @@ You are a **principal Python engineer** maintaining a CLI tool and library for r
- `metadata.py` — `synthid_source(path)` returns the vendor name(s) if the C2PA manifest implies a SynthID pixel watermark, else None. Format-agnostic: PNG via the caBX parser, JPEG/WebP/AVIF/HEIF/JXL via a binary scan (C2PA marker + SynthID issuer + AI-source marker). `get_ai_metadata` surfaces the verdict, and `metadata --check` prints it as a callout. Both `get_ai_metadata` and `has_ai_metadata` guard the PIL open with `except Exception` (HEIC/unknown formats raise non-OSError) and fall through to the binary scan.
- `identify.py` — `identify(path)` aggregates every locally-readable signal (C2PA issuer→platform, IPTC "Made with AI", embedded SD/ComfyUI params, SynthID proxy, visible Gemini sparkle) into one `ProvenanceReport`. `is_ai_generated` is True or None (never asserted False — stripped metadata is not proof of clean origin). Visible-sparkle is promoted only at confidence ≥ `_SPARKLE_THRESHOLD` (0.5; corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49). The cv2 dependency lives in `gemini_engine.detect_sparkle_confidence`, not here. Add platform mappings to `_ISSUER_PLATFORM`, not inline. For non-PNG containers (JPEG/WebP/AVIF/HEIF/JXL) the caBX parser returns nothing, so issuer (`_issuers_in`) and generator (`_ai_tools_in`, reusing `C2PA_AI_TOOLS`) are recovered by binary-scanning the first MB. EXIF `Software` / `Make` / `Artist` / `ImageDescription` and XMP `CreatorTool` generator tags are read by `metadata.exif_generator` (PIL+piexif for any format PIL opens incl. AVIF, plus a container-agnostic XMP raw-byte scan that also covers HEIF/JXL), matched against `AI_GENERATOR_TOKENS` so ordinary editors (plain "Adobe Photoshop") and real-camera `Make` ("Apple"/"Canon") are not flagged. **Ideogram tags its output with EXIF `Make="Ideogram AI"`** (verified on a real download 2026-05-24) — that's why `Make` is read.
- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU). `detect_sparkle_confidence(path)` is the file-level entry point used by `identify.py`.
- `invisible_watermark.py` — `detect_invisible_watermark(path)` decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the `imwatermark` library. Known fixed patterns (verified against upstream source) live in `_BITS_48` (SDXL 48-bit, FLUX.2 48-bit) and `_SD1_STRING` ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra `detect`); returns None when absent. **Unlike SynthID this is locally detectable**, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs.
- `invisible_watermark.py` — `detect_invisible_watermark(path)` decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the `imwatermark` library. Known fixed patterns (verified against upstream source) live in `_BITS_48` (SDXL 48-bit, FLUX.2 48-bit) and `_SD1_STRING` ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra `detect`); returns None when absent. The `detect` extra pulls **torch** transitively (invisible-watermark declares torch a hard dep, and `WatermarkDecoder` eagerly imports `rivaGan` -> `torch` at import time), so detection needs torch present even though dwtDct runs CPU-only on cv2/numpy/pywavelets — no GPU and no separate `gpu` extra required. **Unlike SynthID this is locally detectable**, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs.
- `face_protector.py` — YOLO detect + soft-blend pattern; mirror this for any "protect region during diffusion" features
## Watermarking landscape (research 2026-05-24)
@@ -38,15 +38,16 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are
- **Locally detectable (open decoder, no key/API):** Stable Diffusion / SDXL / FLUX via `imwatermark` DWT-DCT (now covered by `invisible_watermark.py`). FLUX uses the same library (`black-forest-labs/flux2` `src/flux2/watermark.py`, 48-bit `0b001010101111111010000111100111001111010100101110`); SDXL is the diffusers `WATERMARK_MESSAGE` (`0b101100111110110010010000011110111011000110011110`). Caveat: fragile to re-encoding.
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), and **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`). Still unsampled: Canva (its downloads are re-encoded design *exports* that strip C2PA, so a Canva "positive" is inconclusive — skipped), Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
- **EXIF/XMP generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal).
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). **Doubao** (ByteDance) uses it (verified on the real #13 sample 2026-05-25; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` json-decodes the (HTML-entity-encoded) block; container-agnostic raw-byte scan.
- **No detectable signal on download (correctly reported `unknown`):** **Recraft** (PNG export is a re-encoded design export — strips everything), **Krea hosting FLUX 2** (no imwatermark despite FLUX — the host omits the encoder, same as Stability's hosted SDXL), and Midjourney (embeds nothing). Lesson: the imwatermark detector only fires on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL), not from re-hosts (Krea/Stability) or re-encoded exports (Recraft/Canva).
- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
## Known limitations
- `invisible` pipeline downscales to model-native resolution (1024 px for SDXL) before diffusion. Degrades fine text in infographics. Tracked; fix is tile-based diffusion.
- `invisible` pipeline processes at **native resolution by default** (`max_resolution=0`), matching the hosted raiw.cc backend (fal fast-sdxl, no pre-downscale). The old forced downscale-to-1024 -> upscale-back round-trip was the main quality loss (issue #10) and is gone; at strength ~0.05 SDXL img2img does not need the ~1024 downscale. `--max-resolution N` re-introduces an opt-in long-side cap purely to bound GPU/MPS memory on very large inputs (it reintroduces the lossy round-trip). For huge images that OOM at native, tile-based diffusion is still the proper long-term fix. **Concrete MPS data point (verified 2026-05-25 on a 1254x1254 gpt-image SDXL run, fp32, 20 GB MPS ceiling):** native res OOMs at the *UNet* step (peak ~17 GiB), not only the VAE decode, and the auto-fallback in `img2img_runner` reloads on CPU and finishes (slow, ~13 min) -- the output is still weight-identical and defeats SynthID, so "looks hung/crashed" on Mac is usually this CPU fallback, not a pipeline error. Adding `enable_vae_tiling()` alone does NOT prevent it (the peak is the UNet, not the VAE). The fast Mac workarounds are fp16 on MPS (roughly halves memory) or `--max-resolution` to cap the long side; neither is wired as the default. The native-vs-downscale decision lives in the pure helper `invisible_engine._target_size(w, h, max_resolution)` (returns `None` for native, a clamped target tuple otherwise) so it is unit-tested (`tests/test_invisible_engine.py::TestTargetSize`, the #10/#15 regression guard) without loading the model -- keep that logic in the helper, don't re-inline it.
- Pyright first run is slow (2-3 min) due to ML deps (torch/diffusers/transformers stubs); full-project `uv run pyright` can stall for many minutes — scope it to changed files.
- `ultralytics` monkey-patches `PIL.Image.open` and tries to autoload `pi_heif`. When `pi_heif` is missing, opening files raises `ModuleNotFoundError`, not `UnidentifiedImageError`. Code that opens user-supplied or unknown-format files should `except Exception`, not just `OSError`/`UnidentifiedImageError`.
- Metadata detection for AVIF/HEIF/JPEG-XL relies on a binary scan for `C2PA_UUID` + `IPTC_AI_MARKERS`, plus EXIF `Software` / XMP `CreatorTool` generator tags via `metadata.exif_generator` (validated with synthesized AVIF/JPEG fixtures + an XMP raw-scan fixture). C2PA removal in those containers is implemented via `noai/isobmff.py` (top-level ``uuid`` / ``jumb`` box stripper, no re-encoding). EXIF/XMP boxes inside those containers are read for detection but not yet **scrubbed** on removal.
- **SynthID detection is metadata-only.** There is no reliable *local* detector of the SynthID *pixel* watermark — Google's decoder is proprietary, no public spec or API (only a waitlisted portal). We detect SynthID by its C2PA companion (`synthid_source` / `SYNTHID_C2PA_ISSUERS`), which is reliable while the manifest is intact but says nothing once C2PA is stripped. **Surface-dependent blind spot (verified 2026-05-24):** the same Google model emits different metadata per surface -- the Gemini *app* wraps outputs in Google C2PA, but the *API/playground* (AI Studio, Nano Banana / gemini-2.5-flash-image) emits the SynthID *pixel* watermark (confirmed via the Gemini-app oracle) + the visible sparkle but **no C2PA/IPTC at all**, so `synthid_source` returns None despite SynthID being present. Only the pixel oracle or the visible-sparkle detector catches those. (Meta AI is another surface mismatch: it writes the IPTC `digitalSourceType=trainedAlgorithmicMedia` marker, not C2PA and not SynthID.) Google→SynthID is long-standing; OpenAI→SynthID is confirmed by OpenAI's Help Center (ChatGPT/Codex/API "include both C2PA metadata and SynthID watermarks", updated 2026-05-21) but time-gated (pre-rollout OpenAI images carry C2PA without SynthID), so the OpenAI verdict is hedged "likely". Oracles: Gemini app "Verify with SynthID" (Google), openai.com/verify (OpenAI). The spectral phase-coherence approach from `github.com/aloshdenny/reverse-SynthID` was evaluated (May 2026) and **does not work for real-content detection**: on its own shipped codebook + validation set, watermarked and cleaned images were indistinguishable (conf within noise, cleaned often higher); it only fires on pure-black 1024x1024 reference images at exact resolution (the controlled case it was calibrated on). The README's "90% / conf=0.91" reproduces only in that lab condition. Do not build a production detector on it; if revisited, it is experimental/diagnostic only and needs a per-resolution, per-model reference corpus. A from-scratch gpt-image pilot (2026-05-24) confirmed this independently: 5 independent solid-black gpt-image outputs share a near-identical fixed signature (pairwise residual correlation **0.92**, avg-template retains 97% energy), so the watermark/carrier IS strongly present and consistent on flat content — but the carrier frequencies extracted from it do NOT discriminate real content (carrier-to-random ratio: cleaned 1.86 > watermarked 1.53; a non-gpt-image image scored highest at 3.67). The signature drowns in content texture. Net: a perfectly consistent solid-color signature still yields no real-content pixel detector with magnitude/carrier methods. A corpus discrimination test (2026-05-24, `scripts/synthid_pixel_probe.py`, raw zero-mean residual NCC) independently re-confirms this: at matched resolution, SynthID positives do NOT cluster apart from negatives (within-Gemini 0.07; at 1024 px pos-vs-neg >= pos-vs-pos). The only high correlations were near-duplicate *content* (5 ChatGPT renders of one prompt at ~0.92, while a distinct ChatGPT image scored ~0 against them) — content, not a carrier. The probe is solid-fills-only and EXPERIMENTAL/DIAGNOSTIC; do not use it on real content. **Correction (deeper re-examination 2026-05-25):** the carrier IS real on solid fills — the earlier "no carrier" was a *method* artifact of using spatial / FFT-magnitude NCC, which can't see it. The carrier is a fixed *phase* at specific low frequencies, so the right metric is **per-bin phase coherence**. On 8 white `gemini-2.5-flash-image` fills (generated via the reverse-SynthID trick: identity-edit prompt "Recreate this image exactly as it is" on a synthetic pure-white PNG — this bypasses the recitation block that rejects text prompts for pure colors), phase coherence at the white carriers `(0,±7..±12,±20..±23)` = **0.86** vs **0.31** random; single-image leave-one-out phase-match **+0.83** vs real photos **-0.24**. (Black `2.5-flash` fills clip to std≈0 — SynthID can't push values below 0, so no carrier in black; the repo's dark carriers come from nano-banana-pro.) **But it does not generalize:** (a) carriers are model-version + resolution + color specific — the repo's v4 codebook (built for `gemini-3.1-flash-image-preview` + `nano-banana-pro-preview`) scores ~0.527 on my 2.5-flash white fills, indistinguishable from negatives (~0.50), i.e. carriers shift across model versions and need a per-model codebook; (b) on real content (30 `2.5-flash` images) the carrier collapses — set phase coherence at carriers 0.37 ≈ random 0.42, and the repo's v4 detector gives content 0.518 ≈ negatives 0.504 (no separation; a faint +0.24 single-image lean is likely a brightness confound). Net: the spectral/phase approach is a real *controlled-fill* characterizer, NOT an arbitrary-real-content detector, and is brittle to model version. Metadata proxy + visible sparkle + online oracles remain the ceiling for real content.
- **External AI-vs-real classifier models are out of scope (decided 2026-05-24).** Generic HuggingFace detectors (`Organika/sdxl-detector` Swin Transformer, `umm-maybe/AI-image-detector`, and fine-tunes) exist and report ~0.98 on their *own* SDXL-vs-real validation sets, but they are per-generator and the model cards themselves note degraded accuracy off-distribution; they are untested on gpt-image / Gemini Nano Banana (the metadata-stripped surfaces we care about), and our own light SDXL pass would likely defeat them the same way it defeats SynthID. Detection here stays local + signal-based (metadata + visible sparkle); do not add a bundled classifier dependency.
- **SynthID v2 vs default pipeline:** the SDXL-based default profile (since May 2026) defeats SynthID v2. **Verified end-to-end (May 2026):** local SDXL run on a Gemini 3 Pro output, checked via the Gemini app's "Verify with SynthID" feature, returned "no SynthID watermark detected". Also confirmed against **OpenAI's** SynthID (2026-05-23): a fresh ChatGPT/gpt-image output read "SynthID detected" on openai.com/verify before the local SDXL run and "SynthID not detected" after (corpus regression chain: pos `4ef377bd` -> cleaned `47188e88`). The same configuration is used in raiw-app production (`fal-ai/fast-sdxl` at native ~1024 px, strength 0.05, steps 50). SD-1.5 dreamshaper at 768 px was previously the default and does NOT defeat v2 — verified empirically against the same feature (strength 0.04, 0.10, and elastic warp α∈{5,8} all flagged positive). That SD-1.5 path was removed; only `default` (SDXL) and `ctrlregen` profiles remain.
- **SynthID v2 vs default pipeline:** the SDXL-based default profile (since May 2026) defeats SynthID v2. **Verified end-to-end (May 2026):** local SDXL run on a Gemini 3 Pro output, checked via the Gemini app's "Verify with SynthID" feature, returned "no SynthID watermark detected". Also confirmed against **OpenAI's** SynthID (2026-05-23): a fresh ChatGPT/gpt-image output read "SynthID detected" on openai.com/verify before the local SDXL run and "SynthID not detected" after (corpus regression chain: pos `4ef377bd` -> cleaned `47188e88`). The same configuration is used in raiw-app production (`fal-ai/fast-sdxl/image-to-image`, strength 0.05, steps 50, guidance 7.5, no pre-downscale). fal's own `llms.txt` for `fast-sdxl` names the base checkpoint as `stabilityai/stable-diffusion-xl-base-1.0` (verified 2026-05-25) -- the exact checkpoint the local CLI defaults to (`DEFAULT_MODEL_ID`). So the local `invisible` default is weight-for-weight identical to prod; "fast-sdxl" is fal's optimized serving, not different weights. After the native-resolution fix the local pipeline matches prod on weights + strength + steps + guidance + resolution. SD-1.5 dreamshaper at 768 px was previously the default and does NOT defeat v2 — verified empirically against the same feature (strength 0.04, 0.10, and elastic warp α∈{5,8} all flagged positive). That SD-1.5 path was removed; only `default` (SDXL) and `ctrlregen` profiles remain.
+24 -12
View File
@@ -14,6 +14,7 @@ Strips SynthID, C2PA Content Credentials, EXIF/XMP "Made with AI" labels, and vi
- **Smart Face Protection** — automatic extraction and blending of human faces to prevent AI distortion
- **Batch processing** — process entire directories
- **Detection** — three-stage NCC watermark detection with confidence scoring
- **Provenance detection (`identify`)** — aggregate C2PA issuer, IPTC "Made with AI", embedded SD/ComfyUI params, EXIF/XMP generator tags, the SynthID metadata proxy, the visible sparkle, and the open SD/SDXL/FLUX invisible watermark into one origin-platform + watermark-inventory verdict (`--json` for machine output)
> **Try it online** — don't want to install anything? Use [raiw.cc](https://raiw.cc), a free web service powered by this library.
@@ -27,17 +28,24 @@ Strips SynthID, C2PA Content Credentials, EXIF/XMP "Made with AI" labels, and vi
| AI model | Visible watermark | Invisible watermark | Metadata | Our approach |
| --- | --- | --- | --- | --- |
| **Google Gemini / Nano Banana / Gemini 3 Pro** | ✅ Sparkle logo | ✅ SynthID v1 + v2 (default SDXL pipeline at native ~1024 px) | ✅ C2PA + EXIF | Alpha reversal + diffusion + metadata strip |
| **Google Gemini / Nano Banana / Gemini 3 Pro** | ✅ Sparkle logo | ✅ SynthID v1 + v2 (default SDXL pipeline, native resolution) | ✅ C2PA + EXIF | Alpha reversal + diffusion + metadata strip |
| **OpenAI DALL-E 3 / ChatGPT** | — | — | ✅ C2PA manifest | Metadata strip |
| **OpenAI ChatGPT Images 2.0** (gpt-image-2) | — | ⚠️ imperceptible pixel watermark (no public detector yet) | ✅ C2PA manifest (verified) | Diffusion regeneration + metadata strip |
| **Stable Diffusion (AUTOMATIC1111, ComfyUI)** | — | ✅ DWT / steganographic | ✅ PNG text chunks | Diffusion regeneration + metadata strip |
| **OpenAI ChatGPT Images 2.0** (gpt-image-2) | — | ✅ SynthID + content-specific pixel watermark (since May 2026; no local decoder, openai.com/verify oracle) | ✅ C2PA manifest (verified) | Diffusion regeneration + metadata strip |
| **Stable Diffusion / SDXL (AUTOMATIC1111, ComfyUI)** | — | ✅ DWT-DCT (imwatermark — locally detectable) | ✅ PNG text chunks | Diffusion regeneration + metadata strip |
| **Black Forest Labs FLUX** | — | ✅ DWT-DCT (imwatermark — locally detectable) | ✅ C2PA (FLUX.2 Pro) | Diffusion regeneration + metadata strip |
| **Adobe Firefly** | — | — | ✅ Content Credentials (C2PA) | Metadata strip |
| **Stability AI** (DreamStudio / Stable Image) | — | — | ✅ C2PA ("Stability AI Ltd") | Metadata strip |
| **Microsoft Designer / Bing Image Creator** | — | ✅ SynthID via DALL-E backend (Designer) | ✅ C2PA (Bing runs MAI-Image, signed "Microsoft") | Metadata strip |
| **Midjourney** | — | — | ✅ EXIF + XMP (prompt, model, seed) | Metadata strip |
| **Meta AI** | — | — | ✅ IPTC "Made with AI" (digitalSourceType) | Metadata strip (removes the label) |
| **Doubao** (ByteDance) / China AIGC generators | — | — | ✅ TC260 `<TC260:AIGC>` XMP label (China's mandatory AI labeling) | Metadata strip |
| **StableSignature** (Meta) | — | ✅ In-model watermark | — | Diffusion regeneration |
| **TreeRing** | — | ✅ Latent space watermark | — | Diffusion regeneration |
> Visible watermarks (logo overlays) are currently used only by Google Gemini / Nano Banana. Other services rely on invisible watermarks and/or metadata. Our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain.
> **Detection:** `remove-ai-watermarks identify <image>` reports the origin platform and watermark inventory for all the signals above — C2PA issuer, IPTC "Made with AI", the China TC260 AIGC label, embedded generation params, EXIF/XMP generator tags, the SynthID metadata proxy, the visible sparkle, and (with the `[detect]` extra) the open SD/SDXL/FLUX invisible watermark. The SynthID *pixel* watermark has no local decoder, so it is reported as a metadata proxy only.
## How it works
### Removing the Gemini / Nano Banana sparkle watermark
@@ -65,12 +73,14 @@ Google embeds **SynthID** into every image generated by Gemini / Nano Banana. Ot
The removal pipeline (default profile, SDXL):
```text
image → resize to ~1024px (SDXL native) → encode to latent space (VAE)
image → encode to latent space (VAE) at native resolution
→ add controlled noise (forward diffusion)
→ denoise (reverse diffusion, ~50 steps at strength 0.05)
→ decode back to pixels (VAE) → upscale to original resolution
→ decode back to pixels (VAE)
```
By default the image is processed at its **native resolution** with no pre-downscale, matching the hosted raiw.cc backend (fal `fast-sdxl`, which is `stabilityai/stable-diffusion-xl-base-1.0` — the same checkpoint the CLI defaults to). At strength ~0.05 SDXL img2img does not need the input shrunk, and the old forced downscale-to-1024 then upscale-back round-trip was the main quality loss. Pass `--max-resolution N` to cap the long side only when a very large image runs out of GPU/MPS memory (it reintroduces that lossy round-trip).
SDXL is the default since May 2026: empirically defeats SynthID v2 on Gemini 3 Pro outputs, where the older SD-1.5 pipeline at 768 px did not. The SD-1.5 path was removed once it was verified not to handle v2.
**Face Protection**: before diffusion, YOLO detects people in the image and extracts them. After diffusion, the original faces are blended back with a soft elliptical mask to prevent AI distortion of facial features.
@@ -201,6 +211,8 @@ remove-ai-watermarks visible image.png -o clean.png
# Invisible watermark only (SynthID etc.) — requires GPU
remove-ai-watermarks invisible image.png -o clean.png --humanize 4.0
# Runs at native resolution by default. On a very large image that OOMs the
# GPU/MPS, cap the long side: --max-resolution 2048
# Check / strip AI metadata (C2PA, EXIF, "Made with AI" labels)
# --check also flags SynthID-bearing sources: a C2PA manifest signed by
@@ -277,7 +289,7 @@ Tracked but not yet implemented:
- **Local SynthID *pixel* detector**. Not feasible today: Google's decoder is proprietary, and magnitude/carrier spectral methods do not separate real content (confirmed by three independent evaluations, including a from-scratch gpt-image pilot; see CLAUDE.md). Blocked on either (a) a programmatic generation path (OpenAI / Gemini API) to build a per-(model, resolution) labeled corpus at scale, or (b) a raw watermarked-output dataset. If data arrives, the next approach to try is a learned classifier on diverse content rather than a fixed carrier codebook.
- **Grow the SynthID reference corpus** (`data/synthid_corpus/`) with oracle-labeled samples per model and resolution (Gemini app for Google, openai.com/verify for OpenAI). Prerequisite for any pixel-detector attempt and for an automated removal-regression set.
- **Real non-PNG C2PA fixtures**. SynthID-source detection for JPEG / WebP / AVIF is currently covered only by synthetic byte blobs; replace with real vendor-emitted files to ground the binary-scan path.
- **Maintenance debt**. Bump transitive `idna` to 3.15 (blocks `uv-secure`); clear strict-pyright debt in `remove_ai_metadata` / `cli.py` (untyped piexif / PIL / click / rich) so `maintain.sh` can finish green.
- **Maintenance debt**. Clear strict-pyright debt in `remove_ai_metadata` / `cli.py` (untyped piexif / PIL / click / rich) so `maintain.sh` can finish green. (`uv-secure` is already clean since `idna` was bumped to 3.16.)
- **AVIF / HEIF / JPEG-XL detection limits**. Removal strips top-level C2PA `uuid` and JUMBF `jumb` boxes. EXIF/XMP boxes inside these containers are not yet scrubbed (PNG and JPEG are fully covered).
- **Video pipeline (`noai-video`)**: per-frame inpainting and tracking for Sora 2 dynamic logo, Veo 3.1 badge, Kling, Runway. Separate package, not folded into this repo.
@@ -291,17 +303,17 @@ Watermarking and provenance for AI-generated content is now regulated in several
| Jurisdiction | Instrument | Status (May 2026) | Relevance |
| --- | --- | --- | --- |
| EU | AI Act, Article 50(2) | Marking obligations postponed to **2 December 2026** under the December 2025 omnibus agreement. Code of Practice finalising May/June 2026. | Removing mandated provenance markers with intent to deceive may be sanctioned under national implementations. |
| US (federal) | COPIED Act | Enacted 2025. | Criminalises removal of provenance information with intent to deceive about content origin. The tool itself is lawful; usage may not be. |
| US (state) | CA AB 2655, TX SB 751, similar | In force. | Content-specific (election deepfakes, sexual deepfakes). Not tool-specific. |
| China | Deep Synthesis Regulation, 2025 updates | In force. | Mandatory visible label for AI content. Removal is an administrative offence. |
| UK | Online Safety Act, 2025 transparency extension | In force. | Platform obligations, not user obligations. |
| EU | AI Act, Article 50 | Transparency duties apply from **2 August 2026**. Legacy generative systems (placed on the market before that date) get a grandfathering period to **2 December 2026** for the Article 50(2) marking duty, under the Digital Omnibus (Commission proposal Nov 2025; co-legislator political agreement 7 May 2026). Article 50 guidelines and a marking Code of Practice are being finalised through 2026. | Removing mandated provenance markers with intent to deceive may be sanctioned under national implementations. |
| US (federal) | COPIED Act | **Reintroduced April 2025; not enacted** (pending in the Senate). | If passed, would set NIST provenance standards and prohibit tampering with / removing provenance information. The tool itself is lawful; usage may not be. |
| US (state) | CA AB 2655, TX SB 751, similar | TX SB 751 in force; **CA AB 2655 struck down** by a federal court (Aug 2025, Section 230 / First Amendment). | Content-specific (election deepfakes, sexual deepfakes). Not tool-specific. |
| China | Measures for Labeling AI-Generated Content (+ GB 45438-2025) | In force since **1 September 2025**. | Mandatory explicit (visible) + implicit (metadata) labels for AI content; tampering with or removing labels is prohibited. |
| UK | Online Safety Act 2023 / Ofcom guidance | In force, but **no statutory AI-provenance or watermarking obligation**. | Ofcom encourages watermarking / provenance metadata as voluntary "attribution measures"; platform duties, not user obligations. |
## Threat model
This tool defends already-distributed AI imagery against automatic detection systems (social-platform "Made with AI" labels, third-party classifiers, content-policy filters). It does **not** retroactively anonymise generation.
In particular, **SynthID-Image v2** (Google, deployed October 2025 with Gemini 3 Pro / Nano Banana Pro / Imagen 4 / Veo) embeds a **136-bit payload** ([arxiv 2510.09263](https://arxiv.org/abs/2510.09263)). The payload is believed to encode a user / session identifier. If the original watermarked file ever passed through a system controlled by the prompt originator (a saved Gemini account history, a screenshot uploaded to a Google product, a backup), Google retains the ability to link that original to the generating account. Stripping the watermark from a copy you possess does not erase Google's server-side record.
In particular, **SynthID** (Google DeepMind) is embedded across Google's generative media stack — Imagen (images), Veo (video), Lyria (audio) — and Gemini app image outputs (Nano Banana / Gemini 3 Pro, which we verified positive via the Gemini app's SynthID oracle); Google reported over 10 billion items watermarked by December 2025. It carries a **multi-bit payload** — the research paper's SynthID-O variant encodes 136-bit payloads in 512x512 images ([arxiv 2510.09263](https://arxiv.org/abs/2510.09263)). The payload is believed to encode a user / session identifier. If the original watermarked file ever passed through a system controlled by the prompt originator (a saved Gemini account history, a screenshot uploaded to a Google product, a backup), Google retains the ability to link that original to the generating account. Stripping the watermark from a copy you possess does not erase Google's server-side record.
Use cases where the threat model fits:
- You generated the image yourself, want to publish it as your own work, and accept the consequences if Google ever publishes their detector logs.
Binary file not shown.

After

Width:  |  Height:  |  Size: 5.1 MiB

+2 -2
View File
@@ -26,7 +26,7 @@ The `trainedAlgorithmicMedia` tag is what triggers "Made with AI" labels on Inst
### Invisible pixel-level watermark
OpenAI's system card for Images 2.0 states the model embeds an "imperceptible, robust, and content-specific" pixel-level watermark alongside C2PA. No public detector exists, so bypass cannot be verified empirically for this sample.
OpenAI's system card for Images 2.0 states the model embeds an "imperceptible, robust, and content-specific" pixel-level watermark alongside C2PA. This sample was downloaded 2026-04-22, before OpenAI's 19 May 2026 rollout of Google's SynthID watermark across ChatGPT / Codex / the API, so it likely predates SynthID and carries only the original content-specific watermark. Since that rollout, the openai.com/verify tool (in preview) is the public oracle for both signals; there is still no local decoder, so bypass cannot be verified empirically here without the oracle.
## Reproducing the removal
@@ -38,4 +38,4 @@ remove-ai-watermarks metadata amur-leopard.png --remove -o amur-leopard.clean.pn
remove-ai-watermarks all amur-leopard.png -o amur-leopard.all.png --device mps
```
Note: the diffusion step downscales to 768 px, which degrades fine text on text-heavy infographics like this one. Tracked as a known limitation; see project TODO.
Note: the diffusion step runs at native resolution by default (no pre-downscale), so fine text on a text-heavy infographic like this one is preserved. Pass `--max-resolution N` only if a very large image OOMs the GPU/MPS (that reintroduces a lossy downscale then upscale round-trip).
+20 -13
View File
@@ -118,22 +118,29 @@ Download; Leonardo serves the original JPEG in-chat (download button matches).
What each platform actually embeds, verified by byte-scan (and Gemini-app oracle
where noted). The detector's coverage is complementary: metadata catches C2PA /
IPTC; the visible detector catches the Gemini-family sparkle; the SynthID pixel
itself has no local detector (oracle only).
IPTC; `exif_generator` catches EXIF `Make`/`Software` + XMP `CreatorTool`;
`invisible_watermark.py` (imwatermark) catches the open SD/SDXL/FLUX DWT-DCT
watermark on pristine files; the visible detector catches the Gemini-family
sparkle; the SynthID *pixel* itself has no local detector (oracle only).
| Platform | C2PA issuer | SynthID pixel | IPTC "Made with AI" | Visible sparkle | Corpus label |
|---|---|---|---|---|---|
| Gemini app | Google | yes | - | yes | pos |
| ChatGPT / gpt-image | OpenAI | yes | - | - | pos |
| Microsoft Designer | OpenAI + Microsoft | yes (via OpenAI) | - | - | pos |
| Google AI Studio (Nano Banana) | **none** | yes (oracle-confirmed) | - | yes | pos (metadata blind spot) |
| Meta AI | none | no | **yes** | - | neg (for SynthID) |
| Leonardo.ai | none | no | no | - | neg |
| Grok (xAI) | none (non-adopter) | no | no | - | neg (captured: clean low-res preview) |
| Bing Image Creator (DALL-E) | (expect OpenAI, like Designer) | - | - | - | not captured (UI uncooperative) |
| Platform | C2PA issuer | SynthID pixel | IPTC "Made with AI" | Visible sparkle | imwatermark | Corpus label |
|---|---|---|---|---|---|---|
| Gemini app | Google | yes | - | yes | - | pos |
| ChatGPT / gpt-image | OpenAI | yes | - | - | - | pos |
| Microsoft Designer | OpenAI + Microsoft | yes (via OpenAI) | - | - | - | pos |
| Bing Image Creator | Microsoft (MAI-Image) | no | - | - | - | pos (C2PA "Microsoft", not OpenAI) |
| Google AI Studio (Nano Banana) | **none** | yes (oracle-confirmed) | - | yes | - | pos (metadata blind spot) |
| Stability AI (Brand Studio) | Stability AI Ltd | no | - | - | no | pos (C2PA only) |
| Ideogram | none | no | - | - | no | pos (EXIF `Make="Ideogram AI"` only) |
| Meta AI | none | no | **yes** | - | - | neg (for SynthID) |
| Leonardo.ai | none | no | no | - | no | neg |
| Recraft | none (export strips) | no | no | - | no | neg (re-encoded export, no signal) |
| Krea (FLUX 2 host) | none | no | no | - | no | neg (host omits the imwatermark encoder) |
| Grok (xAI) | none (non-adopter) | no | no | - | no | neg (captured: clean low-res preview) |
Key takeaways:
- The same model differs by *surface*: Gemini app wraps C2PA, AI Studio (API/playground) emits none -- only the pixel + sparkle survive.
- Microsoft Designer's DALL-E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft").
- Microsoft Designer's DALL-E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL-E).
- Meta uses the IPTC `digitalSourceType` marker, not C2PA or SynthID.
- The open imwatermark fires only on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL) -- not from re-hosts (Krea, Stability hosted SDXL) or re-encoded design exports (Recraft, Canva). Ideogram's only signal is the EXIF `Make` tag.
- Bing and Grok web UIs are uncooperative for autonomous capture (no document_idle for screenshots; blob downloads intermittently no-op; low-res in-chat previews). Use their native download button manually if a full-res sample is needed.
+1
View File
@@ -116,3 +116,4 @@ f9bb6b1039ee228221d2eead7daebd46422a36a4a8959a74fa0198e1aa19f9b8,f9bb6b10-a_red_
ca2162624607b8f030c63cb1371d6afdd3f0d7af3d9b17514f7ae64a2e8f378c,ca216262-0001_2_a-red-ceramic-coffee-mug-sits-on-weather_rs0Lt1beXV-Mwjs33vWKVw_F7yI1GFGTayDA1NAUmkraw.jpeg,neg,Ideogram,ideogram-3,1024,1024,jpeg,,,none,2026-05-25T01:36:38Z,EXIF Make='Ideogram AI'; no C2PA/SynthID; caught via exif_generator
8a90f2ddf83ceedd7944531d27248191c79dc321ec76ea1f36df6cff9f6b8916,8a90f2dd-0001_3_a-red-ceramic-coffee-mug-sits-on-weather_Lm96lBwcXsmdeCfCpm52vw_F7yI1GFGTayDA1NAUmkraw.jpeg,neg,Ideogram,ideogram-3,1024,1024,jpeg,,,none,2026-05-25T01:36:38Z,EXIF Make='Ideogram AI'; no C2PA/SynthID; caught via exif_generator
87fca198d5c436b27a16685972802a2ff0a310a941248c9ec8d81560d7f387a1,87fca198-0001_4_a-red-ceramic-coffee-mug-sits-on-weather_pGmlgcH4XqSaq41hc0a9AQ_F7yI1GFGTayDA1NAUmkraw.jpeg,neg,Ideogram,ideogram-3,1024,1024,jpeg,,,none,2026-05-25T01:36:38Z,EXIF Make='Ideogram AI'; no C2PA/SynthID; caught via exif_generator
f6dd47a5ffd319aea21bf10dcf9877097666420b02c2620080bac12b03976e7e,f6dd47a5-4ef377bd-gpt-image-2-cleaned.png,cleaned,"our pipeline (invisible/SDXL, native-res default)",stabilityai/stable-diffusion-xl-base-1.0,1254,1254,png,,,openai-verify,2026-05-25T20:50:38Z,"cleaned from 4ef377bd via v0.5.3 'all' at native 1254x1254 (prod-equivalent); openai.com/verify: SynthID NOT detected. Re-confirms #10 native-res default defeats OpenAI SynthID (closes #15 root cause). Note: native res OOMs on 20GB MPS, auto-fell back to CPU."
1 sha256 filename label source model width height format c2pa_issuer synthid_metadata verified_via added notes
116 ca2162624607b8f030c63cb1371d6afdd3f0d7af3d9b17514f7ae64a2e8f378c ca216262-0001_2_a-red-ceramic-coffee-mug-sits-on-weather_rs0Lt1beXV-Mwjs33vWKVw_F7yI1GFGTayDA1NAUmkraw.jpeg neg Ideogram ideogram-3 1024 1024 jpeg none 2026-05-25T01:36:38Z EXIF Make='Ideogram AI'; no C2PA/SynthID; caught via exif_generator
117 8a90f2ddf83ceedd7944531d27248191c79dc321ec76ea1f36df6cff9f6b8916 8a90f2dd-0001_3_a-red-ceramic-coffee-mug-sits-on-weather_Lm96lBwcXsmdeCfCpm52vw_F7yI1GFGTayDA1NAUmkraw.jpeg neg Ideogram ideogram-3 1024 1024 jpeg none 2026-05-25T01:36:38Z EXIF Make='Ideogram AI'; no C2PA/SynthID; caught via exif_generator
118 87fca198d5c436b27a16685972802a2ff0a310a941248c9ec8d81560d7f387a1 87fca198-0001_4_a-red-ceramic-coffee-mug-sits-on-weather_pGmlgcH4XqSaq41hc0a9AQ_F7yI1GFGTayDA1NAUmkraw.jpeg neg Ideogram ideogram-3 1024 1024 jpeg none 2026-05-25T01:36:38Z EXIF Make='Ideogram AI'; no C2PA/SynthID; caught via exif_generator
119 f6dd47a5ffd319aea21bf10dcf9877097666420b02c2620080bac12b03976e7e f6dd47a5-4ef377bd-gpt-image-2-cleaned.png cleaned our pipeline (invisible/SDXL, native-res default) stabilityai/stable-diffusion-xl-base-1.0 1254 1254 png openai-verify 2026-05-25T20:50:38Z cleaned from 4ef377bd via v0.5.3 'all' at native 1254x1254 (prod-equivalent); openai.com/verify: SynthID NOT detected. Re-confirms #10 native-res default defeats OpenAI SynthID (closes #15 root cause). Note: native res OOMs on 20GB MPS, auto-fell back to CPU.
+6 -3
View File
@@ -1,6 +1,6 @@
[project]
name = "remove-ai-watermarks"
version = "0.5.0"
version = "0.5.4"
description = "Remove visible and invisible AI watermarks from images (Gemini / Nano Banana, ChatGPT, Stable Diffusion)"
readme = "README.md"
requires-python = ">=3.10"
@@ -28,8 +28,11 @@ gpu = [
]
# Open invisible-watermark (imwatermark) decoder for detecting the DWT-DCT
# watermarks embedded by Stable Diffusion / SDXL / FLUX. Optional because it
# pulls non-headless opencv; identify() guards the import and skips the signal
# when absent.
# pulls non-headless opencv AND torch (invisible-watermark declares torch a hard
# dependency, and WatermarkDecoder eagerly imports rivaGan -> torch at import
# time, so the dwtDct-only detect path still needs torch present even though it
# never runs on GPU). So `detect` alone pulls torch -- no need to add `gpu` for
# detection. identify() guards the import and skips the signal when absent.
detect = [
"invisible-watermark>=0.2.0",
]
+1 -1
View File
@@ -1,3 +1,3 @@
"""Remove-AI-Watermarks: Unified tool for removing visible and invisible AI watermarks."""
__version__ = "0.5.0"
__version__ = "0.5.4"
+138 -45
View File
@@ -12,6 +12,7 @@ import json
import logging
import time
from pathlib import Path
from typing import TYPE_CHECKING
import click
from rich.console import Console
@@ -21,6 +22,11 @@ from rich.table import Table
from remove_ai_watermarks import __version__
if TYPE_CHECKING:
import numpy as np
from remove_ai_watermarks.gemini_engine import DetectionResult
console = Console()
SUPPORTED_FORMATS = {".png", ".jpg", ".jpeg", ".webp"}
@@ -56,6 +62,74 @@ def _validate_image(path: Path) -> Path:
return path
_ALPHA_FORMATS = {".png", ".webp"}
def _watermark_region(det: DetectionResult, width: int, height: int) -> tuple[int, int, int, int]:
"""Pick a watermark bbox: detector's region if confident, else the default config slot."""
if det.confidence > 0.15:
return det.region
from remove_ai_watermarks.gemini_engine import get_watermark_config
config = get_watermark_config(width, height)
px, py = config.get_position(width, height)
return (px, py, config.logo_size, config.logo_size)
def _read_bgr_and_alpha(path: Path) -> tuple[np.ndarray | None, np.ndarray | None]:
"""Read an image preserving its alpha channel separately.
Returns ``(bgr, alpha)`` where ``alpha`` is a single-channel ndarray when the
source has transparency, else ``None``. Greyscale inputs are promoted to BGR.
Returns ``(None, None)`` if the image cannot be decoded.
"""
import cv2
image = cv2.imread(str(path), cv2.IMREAD_UNCHANGED)
if image is None:
return None, None
if image.ndim == 2:
return cv2.cvtColor(image, cv2.COLOR_GRAY2BGR), None
if image.shape[2] == 4:
return image[:, :, :3].copy(), image[:, :, 3].copy()
return image, None
def _write_bgr_with_alpha(
path: Path,
bgr: np.ndarray,
alpha: np.ndarray | None,
clear_region: tuple[int, int, int, int] | None = None,
pad: int = 6,
) -> None:
"""Write BGR (with optional alpha) to ``path``.
When ``alpha`` is provided and the output extension supports it, writes a
4-channel image. If ``clear_region`` is given as ``(x, y, w, h)``, alpha is
forced to 0 inside that bbox (expanded by ``pad`` px) so the watermark area
becomes fully transparent in the saved file.
"""
import cv2
import numpy as np
if alpha is None or path.suffix.lower() not in _ALPHA_FORMATS:
cv2.imwrite(str(path), bgr)
return
alpha_out = alpha
if clear_region is not None:
alpha_out = alpha.copy()
x, y, w, h = clear_region
height, width = alpha.shape[:2]
x0, y0 = max(0, x - pad), max(0, y - pad)
x1, y1 = min(width, x + w + pad), min(height, y + h + pad)
if x1 > x0 and y1 > y0:
alpha_out[y0:y1, x0:x1] = 0
bgra = np.dstack([bgr, alpha_out])
cv2.imwrite(str(path), bgra)
# ── Main group ───────────────────────────────────────────────────────
@@ -110,8 +184,6 @@ def cmd_visible(
Uses reverse alpha blending — fast, deterministic, offline.
"""
import cv2
from remove_ai_watermarks.gemini_engine import GeminiEngine
_banner()
@@ -122,8 +194,8 @@ def cmd_visible(
engine = GeminiEngine()
# Load image
image = cv2.imread(str(source), cv2.IMREAD_COLOR)
# Load image (preserving any alpha channel separately)
image, alpha = _read_bgr_and_alpha(source)
if image is None:
console.print(f"[red]Error:[/] Failed to read image: {source}")
raise SystemExit(1)
@@ -151,18 +223,12 @@ def cmd_visible(
# Removal
t0 = time.monotonic()
region: tuple[int, int, int, int] | None = None
with console.status("[cyan]Removing watermark…[/]"):
result = engine.remove_watermark(image)
if inpaint:
if det.confidence > 0.15:
region = det.region
else:
from remove_ai_watermarks.gemini_engine import get_watermark_config
config = get_watermark_config(w, h)
pos = config.get_position(w, h)
region = (pos[0], pos[1], config.logo_size, config.logo_size)
region = _watermark_region(det, w, h)
result = engine.inpaint_residual(
result,
region,
@@ -172,9 +238,9 @@ def cmd_visible(
elapsed = time.monotonic() - t0
# Save
# Save (preserves transparency by clearing alpha in the watermark region)
output.parent.mkdir(parents=True, exist_ok=True)
cv2.imwrite(str(output), result)
_write_bgr_with_alpha(output, result, alpha, clear_region=region)
# Strip metadata
if strip_metadata:
@@ -212,6 +278,12 @@ def cmd_visible(
@click.option(
"--humanize", type=float, default=0.0, help="Analog Humanizer film grain intensity (0 = off, typical: 2.0-6.0)."
)
@click.option(
"--max-resolution",
type=int,
default=0,
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
)
@click.pass_context
def cmd_invisible(
ctx: click.Context,
@@ -224,6 +296,7 @@ def cmd_invisible(
seed: int | None,
hf_token: str | None,
humanize: float,
max_resolution: int,
) -> None:
"""Remove invisible AI watermarks (SynthID, StableSignature, TreeRing).
@@ -270,6 +343,7 @@ def cmd_invisible(
guidance_scale=None,
seed=seed,
humanize=humanize,
max_resolution=max_resolution,
)
elapsed = time.monotonic() - t0
@@ -410,6 +484,12 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
@click.option(
"--humanize", type=float, default=0.0, help="Analog Humanizer film grain intensity (0 = off, typical: 2.0-6.0)."
)
@click.option(
"--max-resolution",
type=int,
default=0,
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
)
@click.pass_context
def cmd_all(
ctx: click.Context,
@@ -425,6 +505,7 @@ def cmd_all(
seed: int | None,
hf_token: str | None,
humanize: float,
max_resolution: int,
) -> None:
"""Remove ALL watermarks: visible + invisible + metadata.
@@ -435,9 +516,7 @@ def cmd_all(
If invisible watermark deps are not installed, skips step 2 with a warning.
"""
import cv2
from remove_ai_watermarks.gemini_engine import GeminiEngine, get_watermark_config
from remove_ai_watermarks.gemini_engine import GeminiEngine
_banner()
source = _validate_image(source)
@@ -461,7 +540,7 @@ def cmd_all(
# ── Step 1: Visible watermark ────────────────────────────────
console.print("\n [bold cyan]① Visible watermark removal[/]")
engine = GeminiEngine()
image = cv2.imread(str(source), cv2.IMREAD_COLOR)
image, alpha = _read_bgr_and_alpha(source)
if image is None:
console.print(f"[red]Error:[/] Failed to read image: {source}")
raise SystemExit(1)
@@ -469,25 +548,21 @@ def cmd_all(
h, w = image.shape[:2]
console.print(f" [dim]Input:[/] {source.name} ({w}x{h})")
region: tuple[int, int, int, int] | None = None
with console.status("[cyan]Removing visible watermark…[/]"):
det = engine.detect_watermark(image)
if det.detected:
result = engine.remove_watermark(image)
if inpaint:
if det.confidence > 0.15:
region = det.region
else:
config = get_watermark_config(w, h)
pos = config.get_position(w, h)
region = (pos[0], pos[1], config.logo_size, config.logo_size)
region = _watermark_region(det, w, h)
result = engine.inpaint_residual(result, region, method=inpaint_method)
console.print(" [green]✓[/] Visible watermark removed")
else:
result = image.copy()
console.print(" [dim]Skipped (no visible watermark detected)[/]")
# Save to temp file for invisible engine input
cv2.imwrite(str(tmp_path), result)
# Save to temp file for invisible engine input (preserve alpha if present)
_write_bgr_with_alpha(tmp_path, result, alpha, clear_region=region)
# ── Step 2: Invisible watermark ──────────────────────────────
console.print("\n [bold cyan]② Invisible watermark removal[/]")
@@ -522,6 +597,7 @@ def cmd_all(
num_inference_steps=steps,
seed=seed,
humanize=humanize,
max_resolution=max_resolution,
)
console.print(" [green]✓[/] Invisible watermark removed")
@@ -536,10 +612,15 @@ def cmd_all(
console.print(f" [yellow]⚠[/] Metadata strip failed: {e}")
# ── Write final result ────────────────────────────────────────
import shutil
# The invisible step (and downstream cv2.IMREAD_COLOR paths) drops alpha,
# so re-attach the original alpha (with the watermark region cleared)
# when writing the final output for transparent formats.
output.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(tmp_path), str(output))
final_bgr, _ = _read_bgr_and_alpha(tmp_path)
if final_bgr is None:
console.print(f"[red]Error:[/] Failed to read intermediate file: {tmp_path}")
raise SystemExit(1)
_write_bgr_with_alpha(output, final_bgr, alpha, clear_region=region)
finally:
# Clean up temp file if it still exists
@@ -568,6 +649,7 @@ def _process_batch_image(
seed: int | None,
hf_token: str | None,
humanize: float,
max_resolution: int = 0,
) -> None:
"""Process a single image for batch mode.
@@ -577,13 +659,11 @@ def _process_batch_image(
Raises:
ValueError: If the image cannot be opened.
"""
if mode in ("visible", "all"):
import cv2
saved_alpha: np.ndarray | None = None
saved_region: tuple[int, int, int, int] | None = None
from remove_ai_watermarks.gemini_engine import (
GeminiEngine,
get_watermark_config,
)
if mode in ("visible", "all"):
from remove_ai_watermarks.gemini_engine import GeminiEngine
if "_vis_engine" not in ctx.obj:
ctx.obj["_vis_engine"] = GeminiEngine()
@@ -591,27 +671,24 @@ def _process_batch_image(
read_path = img_path
if mode == "all" and out_path.exists():
read_path = out_path
image = cv2.imread(str(read_path), cv2.IMREAD_COLOR)
image, alpha = _read_bgr_and_alpha(read_path)
if image is None:
raise ValueError("Failed to read image")
region: tuple[int, int, int, int] | None = None
det = engine.detect_watermark(image)
if det.detected:
result = engine.remove_watermark(image)
if inpaint:
if det.confidence > 0.15:
region = det.region
else:
h, w = image.shape[:2]
config = get_watermark_config(w, h)
pos = config.get_position(w, h)
region = (pos[0], pos[1], config.logo_size, config.logo_size)
h, w = image.shape[:2]
region = _watermark_region(det, w, h)
result = engine.inpaint_residual(result, region)
else:
result = image.copy()
cv2.imwrite(str(out_path), result)
_write_bgr_with_alpha(out_path, result, alpha, clear_region=region)
saved_alpha = alpha
saved_region = region
if mode in ("invisible", "all"):
from remove_ai_watermarks.invisible_engine import (
@@ -635,6 +712,7 @@ def _process_batch_image(
num_inference_steps=steps,
seed=seed,
humanize=humanize,
max_resolution=max_resolution,
)
if mode in ("metadata", "all"):
@@ -642,6 +720,13 @@ def _process_batch_image(
remove_ai_metadata(img_path if mode == "metadata" else out_path, out_path)
# In "all" mode, the invisible step (color-only OpenCV paths) drops alpha,
# so re-attach the cached alpha when the input had transparency.
if mode == "all" and saved_alpha is not None:
final_bgr, _ = _read_bgr_and_alpha(out_path)
if final_bgr is not None:
_write_bgr_with_alpha(out_path, final_bgr, saved_alpha, clear_region=saved_region)
@main.command("batch")
@click.argument("directory", type=click.Path(exists=True, file_okay=False, path_type=Path))
@@ -670,6 +755,12 @@ def _process_batch_image(
@click.option("--device", type=click.Choice(["auto", "cpu", "mps", "cuda"]), default="auto", help="Inference device.")
@click.option("--seed", type=int, default=None, help="Random seed for reproducibility.")
@click.option("--hf-token", type=str, default=None, help="HuggingFace API token.")
@click.option(
"--max-resolution",
type=int,
default=0,
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
)
@click.pass_context
def cmd_batch(
ctx: click.Context,
@@ -684,6 +775,7 @@ def cmd_batch(
hf_token: str | None,
inpaint: bool,
humanize: float,
max_resolution: int,
) -> None:
"""Process all images in a directory."""
_banner()
@@ -733,6 +825,7 @@ def cmd_batch(
seed=seed,
hf_token=hf_token,
humanize=humanize,
max_resolution=max_resolution,
)
processed += 1
+14 -1
View File
@@ -24,8 +24,10 @@ from typing import TYPE_CHECKING
from remove_ai_watermarks.metadata import (
AI_METADATA_KEYS,
AIGC_MARKERS,
C2PA_UUID,
IPTC_AI_MARKERS,
aigc_label,
exif_generator,
get_ai_metadata,
)
@@ -219,6 +221,15 @@ def identify(image_path: Path, *, check_visible: bool = True, check_invisible: b
if platform is None:
platform = "Made-with-AI tag (e.g. Meta AI); platform not specified"
# ── China TC260 AIGC label (Doubao and other China-served gens) ──
aigc = any(m in head for m in AIGC_MARKERS)
if aigc:
producer = (aigc_label(image_path) or {}).get("ContentProducer", "")
signals.append(Signal("aigc", f"TC260 AIGC label{f' (producer {producer})' if producer else ''}", "high"))
watermarks.append("China AIGC label (TC260 standard)")
if platform is None:
platform = "China AIGC-labeled generator (TC260; e.g. Doubao)"
# ── Local diffusion parameters (Stable Diffusion / ComfyUI) ──────
local_keys = sorted(k for k in meta if k.lower() in _LOCAL_GEN_KEYS)
if local_keys:
@@ -247,7 +258,9 @@ def identify(image_path: Path, *, check_visible: bool = True, check_invisible: b
# ── Verdict so far (metadata + embedded watermark) ──────────────
invisible_wm = any(s.name == "invisible_watermark" for s in signals)
exif_gen = any(s.name == "exif_generator" for s in signals)
ai_from_metadata = bool((has_c2pa and (c2pa_is_ai or synthid)) or iptc or local_keys or invisible_wm or exif_gen)
ai_from_metadata = bool(
(has_c2pa and (c2pa_is_ai or synthid)) or iptc or aigc or local_keys or invisible_wm or exif_gen
)
# ── Visible Gemini sparkle (fallback for stripped-metadata case) ─
if check_visible and (conf := _visible_sparkle(image_path)) is not None and conf >= _SPARKLE_THRESHOLD:
+46 -29
View File
@@ -42,6 +42,24 @@ def is_available() -> bool:
return False
def _target_size(width: int, height: int, max_resolution: int) -> tuple[int, int] | None:
"""Compute the downscaled (width, height) for a long-side cap, or None for native.
Returns None when no pre-downscale is needed: ``max_resolution <= 0`` (native
resolution, the default that matches the raiw.cc backend -- see issue #10) or
the long side already fits the cap. Otherwise scales the long side down to
``max_resolution`` preserving aspect ratio (integer-truncated, matching the
PIL ``resize`` call site). Pure function so the native-vs-downscale decision
is unit-testable without loading the diffusion model.
"""
if max_resolution > 0 and max(width, height) > max_resolution:
ratio = max_resolution / max(width, height)
# Clamp the short side to >=1: extreme aspect ratios (e.g. 5000x3 capped
# at 1024) would otherwise truncate it to 0 and crash image.resize().
return (max(1, int(width * ratio)), max(1, int(height * ratio)))
return None
class InvisibleEngine:
"""Remove invisible AI watermarks using diffusion model regeneration.
@@ -107,6 +125,7 @@ class InvisibleEngine:
seed: int | None = None,
humanize: float = 0.0,
protect_faces: bool = True,
max_resolution: int = 0,
) -> Path:
"""Remove invisible watermark from an image.
@@ -119,6 +138,11 @@ class InvisibleEngine:
seed: Random seed for reproducibility.
humanize: Intensity of Analog Humanizer film grain (0 = off).
protect_faces: Boolean to extract and restore faces intact.
max_resolution: Cap the long side (px) before diffusion. 0 (default)
= native resolution, no pre-downscale -- matches the hosted
raiw.cc backend. Set a positive value only to bound GPU/MPS
memory on very large inputs (it reintroduces a lossy
downscale->upscale round-trip).
Returns:
Path to the cleaned image.
@@ -127,43 +151,35 @@ class InvisibleEngine:
from PIL import Image, ImageOps
# SDXL is trained at 1024px and degrades both quality and watermark-removal
# efficacy below that.
max_dimension = 1024
# Process at native resolution by default (max_resolution=0). The hosted
# raiw.cc backend (fal fast-sdxl) does NO pre-downscale either, and at
# strength ~0.05 SDXL img2img does not need the input shrunk to ~1024 --
# the old forced downscale->upscale round-trip was the main quality loss
# (see issue #10). A positive max_resolution caps the long side only to
# bound GPU/MPS memory on very large inputs.
image = Image.open(image_path)
image = ImageOps.exif_transpose(image)
orig_size = image.size # (width, height)
_tmp_path = None
if max(image.width, image.height) > max_dimension:
ratio = max_dimension / max(image.width, image.height)
new_size = (int(image.width * ratio), int(image.height * ratio))
# Optional long-side downscale; native resolution by default (issue #10).
target = _target_size(image.width, image.height, max_resolution)
if target is not None:
if self._progress_callback:
self._progress_callback(
f"Downscaling {image.width}x{image.height} "
f"to {new_size[0]}x{new_size[1]} "
f"(model trained at {max_dimension}px)..."
f"to {target[0]}x{target[1]} "
f"(max-resolution cap {max_resolution}px)..."
)
image = image.resize(new_size, Image.Resampling.LANCZOS)
image = image.resize(target, Image.Resampling.LANCZOS)
# Save to a temp file instead of overwriting the original
_tmp_fd, _tmp_str = tempfile.mkstemp(suffix=image_path.suffix)
_tmp_path = Path(_tmp_str)
image.save(_tmp_path)
import os as _os
_os.close(_tmp_fd)
image_path = _tmp_path
else:
# We must save the transposed image back to a tmp file if it was rotated
# otherwise WatermarkRemover will reload it without EXIF rotation!
_tmp_fd, _tmp_str = tempfile.mkstemp(suffix=image_path.suffix)
_tmp_path = Path(_tmp_str)
image.save(_tmp_path)
import os as _os
_os.close(_tmp_fd)
image_path = _tmp_path
# Always persist to a temp file, even without downscaling: WatermarkRemover
# reloads by path, so the EXIF-transposed pixels must be saved or rotation
# is lost. Cleaned up in the finally block via _tmp_path.
_tmp_fd, _tmp_str = tempfile.mkstemp(suffix=image_path.suffix)
_tmp_path = Path(_tmp_str)
image.save(_tmp_path)
os.close(_tmp_fd)
image_path = _tmp_path
try:
# Optional: Face protection (Phase 1 - Extraction)
@@ -244,7 +260,8 @@ class InvisibleEngine:
return out_path
finally:
if _tmp_path is not None and _tmp_path.exists():
# _tmp_path is always set above (we persist the image unconditionally).
if _tmp_path.exists():
_tmp_path.unlink()
def remove_watermark_batch(
+42
View File
@@ -73,6 +73,17 @@ IPTC_AI_MARKERS: tuple[bytes, ...] = (
b"compositeWithTrainedAlgorithmicMedia",
)
# China's mandatory AI-content labeling (TC260, the national cybersecurity
# standards committee). AI generators serving China embed an XMP block in the
# TC260 namespace -- ``<TC260:AIGC>{"Label":"1",...}``. Doubao (ByteDance) uses
# this; the same standard is mandatory for Jimeng, Kling, Qwen, Ernie, etc.,
# so the marker covers the whole China-AIGC-labeled ecosystem. Container-
# agnostic (XMP is text), so a raw-byte scan catches it in PNG/JPEG/etc.
AIGC_MARKERS: tuple[bytes, ...] = (
b"tc260.org.cn/ns/AIGC",
b"TC260:AIGC",
)
STANDARD_METADATA_KEYS: frozenset[str] = frozenset(
[
"Author",
@@ -139,9 +150,35 @@ def has_ai_metadata(image_path: Path) -> bool:
return True
if C2PA_UUID in data:
return True
if any(marker in data for marker in AIGC_MARKERS):
return True
return any(marker in data for marker in IPTC_AI_MARKERS)
def aigc_label(image_path: Path) -> dict[str, str] | None:
"""Parse a China TC260 ``<TC260:AIGC>`` AI-labeling block, if present.
Returns the decoded JSON (e.g. ``{"Label": "1", "ContentProducer": ...}``)
or None. The block is XMP text (HTML-entity encoded), so it is found by a
container-agnostic raw-byte scan and works for PNG/JPEG/WebP alike.
"""
import html
import json
import re
with open(image_path, "rb") as f:
data = f.read(1024 * 1024)
match = re.search(rb"<TC260:AIGC>(.*?)</TC260:AIGC>", data, re.DOTALL)
if not match:
return None
raw = html.unescape(match.group(1).decode("utf-8", "replace"))
try:
parsed = json.loads(raw)
except ValueError:
return None
return {str(k): str(v) for k, v in parsed.items()} if isinstance(parsed, dict) else None
def synthid_source(image_path: Path) -> str | None:
"""Return the vendor name(s) if the image carries a SynthID pixel watermark.
@@ -286,6 +323,11 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
# fall back to the format-agnostic source check for the SynthID verdict.
if "synthid_watermark" not in result and (vendor := synthid_source(image_path)):
result.setdefault("synthid_watermark", synthid_verdict(vendor))
# China TC260 AI-content label (Doubao and other China-served generators).
if aigc := aigc_label(image_path):
producer = aigc.get("ContentProducer", "")
result["aigc_label"] = f"China AIGC label (TC260){f'; producer {producer}' if producer else ''}"
return result
@@ -36,9 +36,10 @@ _HAS_COLOR_MATCHER = False
_HAS_DIFFUSERS = False
try:
from ctrlregen.pipeline import CustomCtrlRegenPipeline
from diffusers import AutoencoderKL, ControlNetModel, UniPCMultistepScheduler
from remove_ai_watermarks.noai.ctrlregen.pipeline import CustomCtrlRegenPipeline
_HAS_DIFFUSERS = True
except ImportError:
AutoencoderKL = None # type: ignore[assignment,misc]
@@ -54,7 +55,7 @@ except ImportError:
CannyDetector = None # type: ignore[assignment,misc]
try:
from ctrlregen.color import color_match
from remove_ai_watermarks.noai.ctrlregen.color import color_match
_HAS_COLOR_MATCHER = True
except ImportError:
@@ -239,7 +240,7 @@ class CtrlRegenEngine:
needs_tiling = orig_w > TILE_SIZE or orig_h > TILE_SIZE
if needs_tiling:
from ctrlregen.tiling import resize_center_crop, run_tiled
from remove_ai_watermarks.noai.ctrlregen.tiling import resize_center_crop, run_tiled
aligned_w = orig_w // 8 * 8
aligned_h = orig_h // 8 * 8
@@ -264,7 +265,7 @@ class CtrlRegenEngine:
ip_adapter_image=orig_image,
)
else:
from ctrlregen.tiling import resize_center_crop
from remove_ai_watermarks.noai.ctrlregen.tiling import resize_center_crop
proc_image = resize_center_crop(image, PROCESS_SIZE)
self._set_progress(f"Preprocessed {orig_w}x{orig_h}px → {proc_image.size[0]}x{proc_image.size[1]}px")
@@ -176,8 +176,11 @@ def _reinstall_torch_cuda_and_restart() -> None:
return
os.environ[_CUDA_FIX_ENV_KEY] = "1"
restart_code = f"import sys; sys.argv = {sys.argv!r}; from remove_ai_watermarks.cli import main; sys.exit(main())"
os.execl(sys.executable, sys.executable, "-c", restart_code)
# Re-exec via ``-m`` rather than building a ``-c`` string from repr(sys.argv).
# ``-m`` makes Python set argv[0] to the module path, so forward only the
# actual args (sys.argv[1:]); passing the full argv would re-inject the
# program name as a spurious first argument to Click.
os.execv(sys.executable, [sys.executable, "-m", "remove_ai_watermarks.cli", *sys.argv[1:]])
def _ensure_watermark_deps() -> None:
+146
View File
@@ -72,6 +72,27 @@ def _mock_invisible_engine():
return mock_cls, mock_engine
def _mock_invisible_engine_drops_alpha():
"""Mock InvisibleEngine that mimics the real engine's BGR-only output path.
The real diffusion-based engine reads with cv2.IMREAD_COLOR and writes a
3-channel result. This mock simulates that so we can regression-test alpha
preservation across the ``all`` pipeline.
"""
def _mock_remove_watermark(image_path, output_path=None, **kwargs):
out = output_path or image_path.with_stem(image_path.stem + "_clean")
out.parent.mkdir(parents=True, exist_ok=True)
bgr = cv2.imread(str(image_path), cv2.IMREAD_COLOR)
cv2.imwrite(str(out), bgr)
return out
mock_engine = MagicMock()
mock_engine.remove_watermark.side_effect = _mock_remove_watermark
mock_cls = MagicMock(return_value=mock_engine)
return mock_cls, mock_engine
class TestMainGroup:
"""Tests for the top-level CLI group."""
@@ -144,6 +165,73 @@ class TestVisibleCommand:
result = runner.invoke(main, ["visible", "/nonexistent/file.png"])
assert result.exit_code != 0
def test_visible_preserves_rgba_transparency(self, runner, tmp_path):
"""Visible removal on an RGBA PNG must keep the alpha channel,
not silently flatten the image onto an opaque background.
"""
rgba = np.zeros((200, 200, 4), dtype=np.uint8)
rgba[:, :, :3] = 200 # light grey foreground
rgba[50:150, 50:150, 3] = 255 # opaque square in the middle, rest transparent
src = tmp_path / "rgba_in.png"
cv2.imwrite(str(src), rgba)
output = tmp_path / "rgba_out.png"
result = runner.invoke(
main,
["visible", str(src), "-o", str(output), "--no-detect"],
)
assert result.exit_code == 0, result.output
assert output.exists()
out = cv2.imread(str(output), cv2.IMREAD_UNCHANGED)
assert out.ndim == 3, f"output is not 3D: shape={out.shape}"
assert out.shape[2] == 4, f"output is not RGBA: shape={out.shape}"
# The transparent corners must remain transparent.
assert out[0, 0, 3] == 0
assert out[199, 199, 3] == 0
# The opaque centre remains opaque (the watermark region default is bottom-right,
# which doesn't overlap the centre square at 200x200).
assert out[100, 100, 3] == 255
def test_visible_clears_alpha_in_watermark_region(self, runner, tmp_path):
"""When inpainting an RGBA image, the watermark region must be cleared
in the alpha channel so the sparkle area becomes transparent, not opaque-black.
"""
rgba = np.full((200, 200, 4), 255, dtype=np.uint8) # fully opaque white
src = tmp_path / "rgba_full.png"
cv2.imwrite(str(src), rgba)
output = tmp_path / "rgba_cleared.png"
result = runner.invoke(
main,
["visible", str(src), "-o", str(output), "--no-detect"],
)
assert result.exit_code == 0, result.output
out = cv2.imread(str(output), cv2.IMREAD_UNCHANGED)
assert out.shape[2] == 4
# Default sparkle position is in the bottom-right; alpha there must be 0.
from remove_ai_watermarks.gemini_engine import get_watermark_config
cfg = get_watermark_config(200, 200)
px, py = cfg.get_position(200, 200)
size = cfg.logo_size
assert out[py + size // 2, px + size // 2, 3] == 0, "alpha in the watermark region was not cleared"
def test_visible_rgb_input_stays_rgb(self, runner, sample_png, tmp_path):
"""Regression: a plain RGB PNG must NOT gain a spurious alpha channel."""
output = tmp_path / "rgb_out.png"
result = runner.invoke(
main,
["visible", str(sample_png), "-o", str(output), "--no-detect"],
)
assert result.exit_code == 0, result.output
out = cv2.imread(str(output), cv2.IMREAD_UNCHANGED)
assert out.ndim == 3, f"output is not 3D: shape={out.shape}"
assert out.shape[2] == 3, f"RGB input produced non-RGB output: shape={out.shape}"
class TestInvisibleCommand:
"""Tests for the 'invisible' subcommand."""
@@ -210,6 +298,33 @@ class TestAllCommand:
result = runner.invoke(main, ["all", "/nonexistent/file.png"])
assert result.exit_code != 0
def test_all_preserves_rgba_across_invisible_step(self, runner, tmp_path):
"""Regression: ``all`` must keep transparency even when the invisible
step writes a 3-channel result (as the real diffusion engine does).
"""
rgba = np.zeros((200, 200, 4), dtype=np.uint8)
rgba[:, :, :3] = 200
rgba[50:150, 50:150, 3] = 255 # opaque square; corners transparent
src = tmp_path / "rgba_in.png"
cv2.imwrite(str(src), rgba)
output = tmp_path / "rgba_out.png"
mock_cls, _engine = _mock_invisible_engine_drops_alpha()
with (
patch("remove_ai_watermarks.cli.InvisibleEngine", mock_cls, create=True),
patch("remove_ai_watermarks.invisible_engine.InvisibleEngine", mock_cls),
patch("remove_ai_watermarks.cli.invisible_available", return_value=True, create=True),
patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True),
):
result = runner.invoke(main, ["all", str(src), "-o", str(output)])
assert result.exit_code == 0, result.output
out = cv2.imread(str(output), cv2.IMREAD_UNCHANGED)
assert out.ndim == 3, f"output not 3D: shape={out.shape}"
assert out.shape[2] == 4, f"output is not RGBA: shape={out.shape}"
assert out[0, 0, 3] == 0
assert out[100, 100, 3] == 255
class TestMetadataCommand:
"""Tests for the 'metadata' subcommand."""
@@ -350,6 +465,37 @@ class TestBatchCommand:
assert result.exit_code == 0, result.output
assert "3 processed" in result.output
def test_batch_all_mode_preserves_rgba(self, runner, tmp_path):
"""Regression: batch ``all`` must keep transparency across the
alpha-dropping invisible step (mirrors test_all_preserves_rgba_...).
"""
input_dir = tmp_path / "input"
input_dir.mkdir()
rgba = np.zeros((200, 200, 4), dtype=np.uint8)
rgba[:, :, :3] = 200
rgba[50:150, 50:150, 3] = 255
cv2.imwrite(str(input_dir / "rgba.png"), rgba)
output_dir = tmp_path / "output"
mock_cls, _engine = _mock_invisible_engine_drops_alpha()
with (
patch("remove_ai_watermarks.cli.InvisibleEngine", mock_cls, create=True),
patch("remove_ai_watermarks.invisible_engine.InvisibleEngine", mock_cls),
patch("remove_ai_watermarks.cli.invisible_available", return_value=True, create=True),
patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True),
):
result = runner.invoke(
main,
["batch", str(input_dir), "-o", str(output_dir), "--mode", "all"],
)
assert result.exit_code == 0, result.output
out = cv2.imread(str(output_dir / "rgba.png"), cv2.IMREAD_UNCHANGED)
assert out.ndim == 3, f"output not 3D: shape={out.shape}"
assert out.shape[2] == 4, f"output is not RGBA: shape={out.shape}"
assert out[0, 0, 3] == 0
assert out[100, 100, 3] == 255
def test_batch_default_output_dir(self, runner, tmp_path):
input_dir = _make_batch_dir(tmp_path)
result = runner.invoke(
+32
View File
@@ -317,3 +317,35 @@ class TestIdentifyInvisibleWatermark:
def test_check_invisible_false_skips(self, tmp_path: Path):
r = identify(self._sdxl_watermarked(tmp_path), check_visible=False, check_invisible=False)
assert not any(s.name == "invisible_watermark" for s in r.signals)
class TestIdentifyAIGC:
"""China TC260 AIGC label is detected and attributed (e.g. Doubao)."""
def _aigc_png(self, tmp_path: Path) -> Path:
from PIL import Image
p = tmp_path / "doubao.png"
Image.new("RGB", (32, 32)).save(p)
xmp = (
'<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF '
'xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">'
'<rdf:Description xmlns:TC260="http://www.tc260.org.cn/ns/AIGC/1.0/">'
"<TC260:AIGC>{&quot;Label&quot;:&quot;1&quot;,&quot;ContentProducer&quot;:&quot;BYTEDANCE001&quot;}"
"</TC260:AIGC></rdf:Description></rdf:RDF></x:xmpmeta>"
)
with open(p, "ab") as f:
f.write(xmp.encode())
return p
def test_aigc_detected(self, tmp_path: Path):
r = identify(self._aigc_png(tmp_path), check_visible=False)
assert r.is_ai_generated is True
assert r.platform is not None
assert "AIGC" in r.platform or "TC260" in r.platform
assert any("AIGC" in w for w in r.watermarks)
def test_aigc_signal_carries_producer(self, tmp_path: Path):
r = identify(self._aigc_png(tmp_path), check_visible=False)
sig = next(s for s in r.signals if s.name == "aigc")
assert "BYTEDANCE001" in sig.detail
+43 -1
View File
@@ -2,7 +2,7 @@
from __future__ import annotations
from remove_ai_watermarks.invisible_engine import InvisibleEngine, is_available
from remove_ai_watermarks.invisible_engine import InvisibleEngine, _target_size, is_available
class TestIsAvailable:
@@ -26,3 +26,45 @@ class TestInvisibleEngineInit:
def test_ctrlregen_model_id(self):
assert InvisibleEngine.CTRLREGEN_MODEL_ID == "yepengliu/ctrlregen"
class TestTargetSize:
"""Regression guard for the native-resolution decision (issues #10 / #15).
max_resolution=0 must NOT downscale -- the forced downscale->upscale
round-trip was the quality loss in #10, and downscaling at all let SynthID
survive in #15 (the native SDXL pass at strength ~0.05 is what defeats it).
"""
def test_native_default_no_downscale(self):
# The default (0) means native resolution: no resize, regardless of size.
assert _target_size(4096, 4096, 0) is None
assert _target_size(123, 456, 0) is None
def test_negative_cap_treated_as_native(self):
assert _target_size(4096, 4096, -1) is None
def test_cap_below_long_side_downscales(self):
# 2000x1000, cap 1024 -> long side scaled to 1024, aspect preserved.
assert _target_size(2000, 1000, 1024) == (1024, 512)
def test_cap_uses_long_side_for_portrait(self):
# Portrait: height is the long side, so it drives the ratio.
assert _target_size(1000, 2000, 1024) == (512, 1024)
def test_cap_at_or_above_long_side_no_downscale(self):
# Already within the cap (and exactly equal) -> no resize.
assert _target_size(800, 600, 1024) is None
assert _target_size(1024, 768, 1024) is None
def test_integer_truncation_matches_pil_call_site(self):
# 1254x1254 (the gpt-image sample) capped at 1000: int(1254*1000/1254)=1000.
assert _target_size(1254, 1254, 1000) == (1000, 1000)
# Non-divisible ratio truncates toward zero like int() at the call site.
assert _target_size(1000, 333, 500) == (500, 166)
def test_extreme_aspect_ratio_clamps_short_side_to_one(self):
# 5000x3 capped at 1024: int(3 * 1024/5000) = 0 would crash resize();
# the short side must clamp to 1, never 0.
assert _target_size(5000, 3, 1024) == (1024, 1)
assert _target_size(3, 5000, 1024) == (1, 1024)
+60
View File
@@ -390,3 +390,63 @@ class TestExifGenerator:
def test_clean_image_is_none(self, tmp_clean_png: Path):
assert exif_generator(tmp_clean_png) is None
class TestAIGCLabel:
"""China TC260 AIGC labeling (Doubao and other China-served generators)."""
def _aigc_png(self, tmp_path: Path, label: str = "1", producer: str = "TESTPRODUCER001") -> Path:
from remove_ai_watermarks.metadata import aigc_label # noqa: F401 (import-time guard)
p = tmp_path / "doubao.png"
Image.new("RGB", (32, 32)).save(p)
# XMP is HTML-entity encoded in real files; aigc_label must unescape it.
xmp = (
'<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF '
'xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">'
'<rdf:Description rdf:about="" '
'xmlns:TC260="http://www.tc260.org.cn/ns/AIGC/1.0/"><TC260:AIGC>'
f"{{&quot;Label&quot;:&quot;{label}&quot;,&quot;ContentProducer&quot;:&quot;{producer}&quot;}}"
"</TC260:AIGC></rdf:Description></rdf:RDF></x:xmpmeta>"
)
with open(p, "ab") as f:
f.write(xmp.encode())
return p
def test_parses_label_and_producer(self, tmp_path: Path):
from remove_ai_watermarks.metadata import aigc_label
info = aigc_label(self._aigc_png(tmp_path))
assert info is not None
assert info["Label"] == "1"
assert info["ContentProducer"] == "TESTPRODUCER001"
def test_none_when_absent(self, tmp_clean_png):
from remove_ai_watermarks.metadata import aigc_label
assert aigc_label(tmp_clean_png) is None
def test_has_ai_metadata_detects_aigc(self, tmp_path: Path):
assert has_ai_metadata(self._aigc_png(tmp_path))
def test_get_ai_metadata_surfaces_aigc(self, tmp_path: Path):
meta = get_ai_metadata(self._aigc_png(tmp_path))
assert "aigc_label" in meta
assert "TC260" in meta["aigc_label"]
@pytest.mark.skipif(not (SAMPLES_DIR / "doubao-1.png").exists(), reason="doubao sample not present")
class TestAIGCRealSample:
"""Real Doubao (ByteDance) sample carries the China TC260 AIGC XMP label."""
def test_doubao_aigc_label(self):
from remove_ai_watermarks.metadata import aigc_label
info = aigc_label(SAMPLES_DIR / "doubao-1.png")
assert info is not None
assert info["Label"] == "1"
assert info["ContentProducer"] # ByteDance producer code present
def test_doubao_detected_as_ai(self):
assert has_ai_metadata(SAMPLES_DIR / "doubao-1.png")
assert "aigc_label" in get_ai_metadata(SAMPLES_DIR / "doubao-1.png")
Generated
+4 -4
View File
@@ -815,11 +815,11 @@ wheels = [
[[package]]
name = "idna"
version = "3.11"
version = "3.16"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/0703ccc57f3a7233505399edb88de3cbd678da106337b9fcde432b65ed60/idna-3.11.tar.gz", hash = "sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902", size = 194582, upload-time = "2025-10-12T14:55:20.501Z" }
sdist = { url = "https://files.pythonhosted.org/packages/1a/88/bcf9709822fe69d02c2a6a77956c98ce6ea8ca8767a9aadcedc7eb6a2390/idna-3.16.tar.gz", hash = "sha256:d7a6da03db833450fca25d2358ac9ff06cd624577a4aea3a596d5c0f77b8e03d", size = 203770, upload-time = "2026-05-22T00:16:18.781Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/0e/61/66938bbb5fc52dbdf84594873d5b51fb1f7c7794e9c0f5bd885f30bc507b/idna-3.11-py3-none-any.whl", hash = "sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea", size = 71008, upload-time = "2025-10-12T14:55:18.883Z" },
{ url = "https://files.pythonhosted.org/packages/94/16/70255075a9859a0e3adb789b68ceb0e210dec03934245fd98d248226572f/idna-3.16-py3-none-any.whl", hash = "sha256:cc246e3a3f89580c3a951b5ad298ca4638078b2cdd4f115654332b5c26daded5", size = 74165, upload-time = "2026-05-22T00:16:16.698Z" },
]
[[package]]
@@ -2150,7 +2150,7 @@ wheels = [
[[package]]
name = "remove-ai-watermarks"
version = "0.5.0"
version = "0.5.4"
source = { editable = "." }
dependencies = [
{ name = "click" },