Compare commits
No files matched your search
@@ -0,0 +1,2 @@
|
||||
# Enables the "Sponsor" button on the repository page.
|
||||
github: wiltodelta
|
||||
@@ -21,6 +21,10 @@ Thumbs.db
|
||||
# Test results
|
||||
data/results/
|
||||
|
||||
# SynthID corpus reference fills (synthetic black/white calibration tiles,
|
||||
# regenerable; the labeled pos/neg/cleaned images ARE tracked, see README)
|
||||
data/synthid_corpus/refs/
|
||||
|
||||
# Reference materials
|
||||
_refs/
|
||||
|
||||
|
||||
@@ -5,24 +5,52 @@ You are a **principal Python engineer** maintaining a CLI tool and library for r
|
||||
## How to run
|
||||
|
||||
- `uv run remove-ai-watermarks all <image.png> -o <output.png>`
|
||||
- `uv run remove-ai-watermarks identify <image>` — provenance verdict (platform + watermark inventory + confidence); `--json` for machine output, `--no-visible` to skip the cv2 sparkle detector
|
||||
- `uv run remove-ai-watermarks metadata <image.png> --check` — inspect AI metadata (C2PA, EXIF, PNG chunks)
|
||||
- `uv run remove-ai-watermarks metadata <image.png> --remove -o <out.png>` — strip all AI metadata
|
||||
|
||||
## Test and lint
|
||||
|
||||
- `bash maintain.sh` — uv-outdated, uv-secure, ruff check/fix, ruff format, pyright, pytest -n auto
|
||||
- `maintain.sh` may not finish fully green (pre-existing, not per-change): strict pyright carries debt in `remove_ai_metadata` / `cli.py` (untyped piexif/PIL/click/rich). (`uv-secure` is clean since idna was bumped 3.11 -> 3.16, fixing GHSA-65pc-fj4g-8rjx.) To gate a change, run `uv run ruff check`, `uv run pyright <changed files>`, `uv run pytest` directly.
|
||||
- Run `uv run` from the repo root — from another cwd it falls back to a bare env without numpy/cv2/torch.
|
||||
- Metadata/C2PA tests assert against real committed fixtures in `data/samples/` (`chatgpt-*.png` = OpenAI C2PA, `firefly-1.png` = Adobe, `mj-*` = Midjourney IPTC, `doubao-1.png` = ByteDance Doubao with the China TC260 `<TC260:AIGC>` XMP label **and** a visible "豆包AI生成" text mark bottom-right; `grok-1.jpg` = xAI Grok with its EXIF-only `Signature:` blob + UUID `Artist` and no C2PA/SynthID/IPTC); synthetic byte blobs cover the JPEG/ISOBMFF format paths. The "non-AI / clean photo" control is no longer in `data/samples/` -- the `clean_photo` conftest fixture serves a verified-negative image from the corpus `neg/` set (skips if the corpus is absent).
|
||||
- SynthID reference corpus: `scripts/synthid_corpus.py` ingests labeled images into `data/synthid_corpus/`. The labeled `images/` (`pos/` `neg/` `cleaned/`) are **committed** (public repo -- review every image for private content before adding; `manifest.csv` is kept in sync with the files on disk, one row per tracked image); only the synthetic `refs/` calibration fills are gitignored. See its README for the collection protocol and verification oracles.
|
||||
|
||||
## Configuration
|
||||
|
||||
- GPU/ML modules (invisible_engine, ctrlregen, watermark_remover) are optional — guard imports with `is_available()` checks
|
||||
- Tests for ML modules are limited to availability checks (require multi-GB downloads)
|
||||
- Tests for the *model-running* paths are limited to availability checks (multi-GB downloads). But the **pure helpers inside ML-adjacent modules are unit-tested without any download** and must stay that way: `_target_size` (native-vs-downscale, `test_invisible_engine.py`), the MPS->CPU fallback control flow via mocked pipelines (`test_img2img_runner.py`, 100% cover), and the tiling math `tile_positions`/`make_blend_weight`/`resize_center_crop` (`test_tiling.py`; `pytest.importorskip("torch")` since `tiling.py` imports torch at module top). Don't skip these as "ML, needs a model" — only `run_tiled`/`remove_watermark`/the diffusion bodies do.
|
||||
|
||||
## Key modules
|
||||
|
||||
- `noai/c2pa.py` — PNG chunk parser; use `extract_c2pa_chunk(path)` to get raw caBX payload, `has_c2pa_metadata(path)` to detect. Do not reimplement chunk parsing.
|
||||
- `noai/constants.py` — PNG_SIGNATURE, C2PA_CHUNK_TYPE, C2PA_SIGNATURES constants
|
||||
- `noai/c2pa.py` — PNG chunk parser; use `extract_c2pa_chunk(path)` to get raw caBX payload, `has_c2pa_metadata(path)` to detect. Do not reimplement chunk parsing. `extract_c2pa_info(path)` sets `synthid_watermark`/`synthid_vendors` when the manifest is signed by a SynthID-using vendor.
|
||||
- `noai/constants.py` — PNG_SIGNATURE, C2PA_CHUNK_TYPE, C2PA_SIGNATURES, C2PA_ISSUERS, and `SYNTHID_C2PA_ISSUERS` (issuers that pair SynthID with C2PA: Google, OpenAI). Add a new issuer here, not inline.
|
||||
- `metadata.py` — `synthid_source(path)` returns the vendor name(s) if the C2PA manifest implies a SynthID pixel watermark, else None. Format-agnostic: PNG via the caBX parser, JPEG/WebP/AVIF/HEIF/JXL via a binary scan (C2PA marker + SynthID issuer + AI-source marker). `get_ai_metadata` surfaces the verdict, and `metadata --check` prints it as a callout. Both `get_ai_metadata` and `has_ai_metadata` guard the PIL open with `except Exception` (HEIC/unknown formats raise non-OSError) and fall through to the binary scan. `xai_signature(path)` detects xAI/Grok's EXIF-only scheme (`ImageDescription` = `Signature: <base64>` + UUID `Artist`); it feeds `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify`.
|
||||
- `identify.py` — `identify(path)` aggregates every locally-readable signal (C2PA issuer→platform, IPTC "Made with AI", embedded SD/ComfyUI params, SynthID proxy, xAI/Grok EXIF signature via `metadata.xai_signature`, visible Gemini sparkle) into one `ProvenanceReport`. `is_ai_generated` is True or None (never asserted False — stripped metadata is not proof of clean origin). Visible-sparkle is promoted only at confidence ≥ `_SPARKLE_THRESHOLD` (0.5; corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49). The cv2 dependency lives in `gemini_engine.detect_sparkle_confidence`, not here. Add platform mappings to `_ISSUER_PLATFORM`, not inline. For non-PNG containers (JPEG/WebP/AVIF/HEIF/JXL) the caBX parser returns nothing, so issuer (`_issuers_in`) and generator (`_ai_tools_in`, reusing `C2PA_AI_TOOLS`) are recovered by binary-scanning the first MB. EXIF `Software` / `Make` / `Artist` / `ImageDescription` and XMP `CreatorTool` generator tags are read by `metadata.exif_generator` (PIL+piexif for any format PIL opens incl. AVIF, plus a container-agnostic XMP raw-byte scan that also covers HEIF/JXL), matched against `AI_GENERATOR_TOKENS` so ordinary editors (plain "Adobe Photoshop") and real-camera `Make` ("Apple"/"Canon") are not flagged. **Ideogram tags its output with EXIF `Make="Ideogram AI"`** (verified on a real download 2026-05-24) — that's why `Make` is read.
|
||||
- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU). `detect_sparkle_confidence(path)` is the file-level entry point used by `identify.py`.
|
||||
- `invisible_watermark.py` — `detect_invisible_watermark(path)` decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the `imwatermark` library. Known fixed patterns (verified against upstream source) live in `_BITS_48` (SDXL 48-bit, FLUX.2 48-bit) and `_SD1_STRING` ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra `detect`); returns None when absent. The `detect` extra pulls **torch** transitively (invisible-watermark declares torch a hard dep, and `WatermarkDecoder` eagerly imports `rivaGan` -> `torch` at import time), so detection needs torch present even though dwtDct runs CPU-only on cv2/numpy/pywavelets — no GPU and no separate `gpu` extra required. **Unlike SynthID this is locally detectable**, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs.
|
||||
- `face_protector.py` — YOLO detect + soft-blend pattern; mirror this for any "protect region during diffusion" features
|
||||
|
||||
## Watermarking landscape (research 2026-05-24)
|
||||
|
||||
Who embeds what, and whether it is locally detectable (so we know which gaps are fillable). See `identify.py` for what we read.
|
||||
- **Locally detectable (open decoder, no key/API):** Stable Diffusion / SDXL / FLUX via `imwatermark` DWT-DCT (now covered by `invisible_watermark.py`). FLUX uses the same library (`black-forest-labs/flux2` `src/flux2/watermark.py`, 48-bit `0b001010101111111010000111100111001111010100101110`); SDXL is the diffusers `WATERMARK_MESSAGE` (`0b101100111110110010010000011110111011000110011110`). Caveat: fragile to re-encoding.
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), and **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`). Still unsampled: Canva (its downloads are re-encoded design *exports* that strip C2PA, so a Canva "positive" is inconclusive — skipped), Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
- **EXIF/XMP generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal).
|
||||
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).** Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC** — only EXIF `Artist` = a UUID and EXIF `ImageDescription` = `Signature: <base64>` (a crypto signature, unverifiable locally without xAI's public key). This empirically kills the earlier unverified "xAI signs C2PA as xAI" lead — xAI is not even a C2PA member. `exif_generator` misses it (neither field holds an `AI_GENERATOR_TOKENS` token), so a dedicated detector `xai_signature(path)` matches the pair (`ImageDescription ~ ^Signature: [A-Za-z0-9+/=]{64,}` AND UUID `Artist`); wired into `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify` (signal `xai_signature`, platform "xAI (Grok / Aurora)"). **Format confirmed stable across n=3 genuine generations:** exactly three EXIF tags (`Artist`, `ExifOffset`, `ImageDescription`), `Signature:` prefix constant, base64 payload 300-1004 chars. Two capture facts: (a) the `Artist` UUID **equals the public image id** in the asset URL (`https://imagine-public.x.ai/imagine-public/images/<uuid>.jpg`), so it is NOT a private per-user secret — only the `Signature` blob is; (b) the Grok web-UI image is a re-encoded **WebP with no signature** — the EXIF survives only in the *original* JPEG (download button or that public tokenless URL), which is why screenshots / re-encodes are metadata-stripped. Tests use a **synthetic** JPEG fixture (fake UUID + fake `Signature:` blob), never a real Grok image (its repo is public). Not yet *stripped* on removal: `remove_ai_metadata` preserves the EXIF block wholesale for JPEG, so the signature is detected but not scrubbed — a follow-up.
|
||||
- **China TC260 AIGC label (caught by `AIGC_MARKERS` / `metadata.aigc_label`, surfaced by `identify` as the `aigc` signal):** China-served generators embed an XMP `<TC260:AIGC>{"Label":"1","ContentProducer":...}` block — China's mandatory AI-content labeling (TC260 namespace `tc260.org.cn/ns/AIGC`). **Doubao** (ByteDance) uses it (verified on the real #13 sample 2026-05-25; `ContentProducer` `001191110102MACQD9K64010000`, no C2PA/SynthID/imwatermark — the XMP block is the only signal; GitHub attachment upload did NOT strip it). The same standard is mandatory for Jimeng/Kling/Qwen/Ernie etc., so the one marker covers the whole China-AIGC-labeled ecosystem. `aigc_label` json-decodes the (HTML-entity-encoded) block; container-agnostic raw-byte scan.
|
||||
- **No detectable signal on download (correctly reported `unknown`):** **Recraft** (PNG export is a re-encoded design export — strips everything), **Krea hosting FLUX 2** (no imwatermark despite FLUX — the host omits the encoder, same as Stability's hosted SDXL), and Midjourney (embeds nothing). Lesson: the imwatermark detector only fires on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL), not from re-hosts (Krea/Stability) or re-encoded exports (Recraft/Canva).
|
||||
- **Invisible but NOT locally detectable (proprietary, API/oracle only — same wall as SynthID):** Amazon Titan Image Generator + Nova Canvas (Bedrock `DetectGeneratedContent` API), Kakao (new SynthID image adopter, May 2026), NVIDIA Cosmos (SynthID video). No local detector possible; treat like SynthID.
|
||||
- **C2PA 2.4 "Durable Content Credentials" (April 2026; verified against the spec) raise the bar for metadata stripping.** 2.4 defines soft bindings (an invisible watermark or a content fingerprint) plus a server-side manifest repository and a new `c2pa.repository-receipt` assertion. Per the spec: "if a C2PA manifest is removed from an asset, but a copy of that manifest remains in a provenance store elsewhere, the manifest and asset may be matched using available soft bindings." So our local `metadata --remove` deletes the *embedded* manifest, but a fingerprint/watermark soft binding can still re-link the image to its manifest in a repository server-side. Stripping the file is becoming necessary-but-not-sufficient against durable provenance. (Our parsers target the stable embedded-manifest format documented in C2PA 2.1 §11; that format is unchanged in 2.4 -- the new pieces are repository/soft-binding infra, not the on-file box layout, so no parser change is implied.) Spec: https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html
|
||||
- **Regulatory driver (context, not a code change):** AI-content labeling mandates are expanding, which pushes more generators toward exactly the C2PA + watermark signals we read. The full per-jurisdiction table lives in README "## Legal" -- keep it there, not duplicated here. Newly added + primary-source verified 2026-05-26: **EU AI Act Article 50** machine-readable marking applicable **2026-08-02** (verified against the article text); **South Korea AI Framework Act Art. 31(3)** in force since Jan 2026 (verified via Kim & Chang; Enforcement Decree accepts an invisible-watermark label); **California AB 853** (amends the CA AI Transparency Act) latent-disclosure duty operative **2026-08-02**, requiring a disclosure "permanent or extraordinarily difficult to remove" (verified against the leginfo bill text -- this is the exact disclosure our tool strips). When extending the README table, verify each date/article against the statute/bill text before committing, not against search summaries.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- `invisible` pipeline downscales to model-native resolution (1024 px for SDXL) before diffusion. Degrades fine text in infographics. Tracked; fix is tile-based diffusion.
|
||||
- Pyright first run is slow (2-3 min) due to ML deps (torch/diffusers/transformers stubs)
|
||||
- `invisible` pipeline processes at **native resolution by default** (`max_resolution=0`), matching the hosted raiw.cc backend (fal fast-sdxl, no pre-downscale). The old forced downscale-to-1024 -> upscale-back round-trip was the main quality loss (issue #10) and is gone; at strength ~0.05 SDXL img2img does not need the ~1024 downscale. `--max-resolution N` re-introduces an opt-in long-side cap purely to bound GPU/MPS memory on very large inputs (it reintroduces the lossy round-trip). For huge images that OOM at native, tile-based diffusion is still the proper long-term fix. **Concrete MPS data point (verified 2026-05-25 on a 1254x1254 gpt-image SDXL run, fp32, 20 GB MPS ceiling):** native res OOMs at the *UNet* step (peak ~17 GiB), not only the VAE decode, and the auto-fallback in `img2img_runner` reloads on CPU and finishes (slow, ~13 min) -- the output is still weight-identical and defeats SynthID, so "looks hung/crashed" on Mac is usually this CPU fallback, not a pipeline error. Adding `enable_vae_tiling()` alone does NOT prevent it (the peak is the UNet, not the VAE). The fast Mac workarounds are fp16 on MPS (roughly halves memory) or `--max-resolution` to cap the long side; neither is wired as the default. The native-vs-downscale decision lives in the pure helper `invisible_engine._target_size(w, h, max_resolution)` (returns `None` for native, a clamped target tuple otherwise) so it is unit-tested (`tests/test_invisible_engine.py::TestTargetSize`, the #10/#15 regression guard) without loading the model -- keep that logic in the helper, don't re-inline it.
|
||||
- Pyright first run is slow (2-3 min) due to ML deps (torch/diffusers/transformers stubs); full-project `uv run pyright` can stall for many minutes — scope it to changed files.
|
||||
- `ultralytics` monkey-patches `PIL.Image.open` and tries to autoload `pi_heif`. When `pi_heif` is missing, opening files raises `ModuleNotFoundError`, not `UnidentifiedImageError`. Code that opens user-supplied or unknown-format files should `except Exception`, not just `OSError`/`UnidentifiedImageError`.
|
||||
- Metadata detection for AVIF/HEIF/JPEG-XL relies on a binary scan for `C2PA_UUID` + `IPTC_AI_MARKERS`. C2PA removal in those containers is implemented via `noai/isobmff.py` (top-level ``uuid`` / ``jumb`` box stripper, no re-encoding). EXIF/XMP boxes inside those containers are not yet scrubbed.
|
||||
- **SynthID v2 vs default pipeline:** the SDXL-based default profile (since May 2026) defeats SynthID v2 per the deployment in raiw-app (`fal-ai/fast-sdxl` at native ~1024 px, strength 0.05, steps 50). SD-1.5 dreamshaper at 768 px was previously the default and does NOT defeat v2 — verified empirically against Gemini app's "Verify with SynthID" feature (strength 0.04, 0.10, and elastic warp α∈{5,8} all flagged positive). That SD-1.5 path was removed; only `default` (SDXL) and `ctrlregen` profiles remain.
|
||||
- Metadata detection for AVIF/HEIF/JPEG-XL relies on a binary scan for `C2PA_UUID` + `IPTC_AI_MARKERS`, plus EXIF `Software` / XMP `CreatorTool` generator tags via `metadata.exif_generator` (validated with synthesized AVIF/JPEG fixtures + an XMP raw-scan fixture). C2PA removal in those containers is implemented via `noai/isobmff.py` (top-level ``uuid`` / ``jumb`` box stripper, no re-encoding). EXIF/XMP boxes inside those containers are read for detection but not yet **scrubbed** on removal.
|
||||
- **SynthID detection is metadata-only.** There is no reliable *local* detector of the SynthID *pixel* watermark — Google's decoder is proprietary, no public spec or API (only a waitlisted portal). Authoritative confirmation: Google DeepMind's own paper "SynthID-Image: Image watermarking at internet scale" (Gowal et al., arXiv:2510.09263) states the verification service is restricted to "trusted testers" and does not release detector weights or a reproducible algorithm — so a local pixel detector is infeasible by design, not just unbuilt. https://arxiv.org/abs/2510.09263 We detect SynthID by its C2PA companion (`synthid_source` / `SYNTHID_C2PA_ISSUERS`), which is reliable while the manifest is intact but says nothing once C2PA is stripped. **Surface-dependent blind spot (verified 2026-05-24):** the same Google model emits different metadata per surface -- the Gemini *app* wraps outputs in Google C2PA, but the *API/playground* (AI Studio, Nano Banana / gemini-2.5-flash-image) emits the SynthID *pixel* watermark (confirmed via the Gemini-app oracle) + the visible sparkle but **no C2PA/IPTC at all**, so `synthid_source` returns None despite SynthID being present. Only the pixel oracle or the visible-sparkle detector catches those. (Meta AI is another surface mismatch: it writes the IPTC `digitalSourceType=trainedAlgorithmicMedia` marker, not C2PA and not SynthID.) Google→SynthID is long-standing; OpenAI→SynthID is confirmed by OpenAI's Help Center (ChatGPT/Codex/API "include both C2PA metadata and SynthID watermarks", updated 2026-05-21) but time-gated (pre-rollout OpenAI images carry C2PA without SynthID), so the OpenAI verdict is hedged "likely". Oracles: Gemini app "Verify with SynthID" (Google), openai.com/verify (OpenAI). The spectral phase-coherence approach from `github.com/aloshdenny/reverse-SynthID` was evaluated (May 2026) and **does not work for real-content detection**: on its own shipped codebook + validation set, watermarked and cleaned images were indistinguishable (conf within noise, cleaned often higher); it only fires on pure-black 1024x1024 reference images at exact resolution (the controlled case it was calibrated on). The README's "90% / conf=0.91" reproduces only in that lab condition. Do not build a production detector on it; if revisited, it is experimental/diagnostic only and needs a per-resolution, per-model reference corpus. A from-scratch gpt-image pilot (2026-05-24) confirmed this independently: 5 independent solid-black gpt-image outputs share a near-identical fixed signature (pairwise residual correlation **0.92**, avg-template retains 97% energy), so the watermark/carrier IS strongly present and consistent on flat content — but the carrier frequencies extracted from it do NOT discriminate real content (carrier-to-random ratio: cleaned 1.86 > watermarked 1.53; a non-gpt-image image scored highest at 3.67). The signature drowns in content texture. Net: a perfectly consistent solid-color signature still yields no real-content pixel detector with magnitude/carrier methods. A corpus discrimination test (2026-05-24, `scripts/synthid_pixel_probe.py`, raw zero-mean residual NCC) independently re-confirms this: at matched resolution, SynthID positives do NOT cluster apart from negatives (within-Gemini 0.07; at 1024 px pos-vs-neg >= pos-vs-pos). The only high correlations were near-duplicate *content* (5 ChatGPT renders of one prompt at ~0.92, while a distinct ChatGPT image scored ~0 against them) — content, not a carrier. The probe is solid-fills-only and EXPERIMENTAL/DIAGNOSTIC; do not use it on real content. **Correction (deeper re-examination 2026-05-25):** the carrier IS real on solid fills — the earlier "no carrier" was a *method* artifact of using spatial / FFT-magnitude NCC, which can't see it. The carrier is a fixed *phase* at specific low frequencies, so the right metric is **per-bin phase coherence**. On 8 white `gemini-2.5-flash-image` fills (generated via the reverse-SynthID trick: identity-edit prompt "Recreate this image exactly as it is" on a synthetic pure-white PNG — this bypasses the recitation block that rejects text prompts for pure colors), phase coherence at the white carriers `(0,±7..±12,±20..±23)` = **0.86** vs **0.31** random; single-image leave-one-out phase-match **+0.83** vs real photos **-0.24**. (Black `2.5-flash` fills clip to std≈0 — SynthID can't push values below 0, so no carrier in black; the repo's dark carriers come from nano-banana-pro.) **But it does not generalize:** (a) carriers are model-version + resolution + color specific — the repo's v4 codebook (built for `gemini-3.1-flash-image-preview` + `nano-banana-pro-preview`) scores ~0.527 on my 2.5-flash white fills, indistinguishable from negatives (~0.50), i.e. carriers shift across model versions and need a per-model codebook; (b) on real content (30 `2.5-flash` images) the carrier collapses — set phase coherence at carriers 0.37 ≈ random 0.42, and the repo's v4 deteLine truncated
|
||||
- **External AI-vs-real classifier models are out of scope (decided 2026-05-24).** Generic HuggingFace detectors (`Organika/sdxl-detector` Swin Transformer, `umm-maybe/AI-image-detector`, and fine-tunes) exist and report ~0.98 on their *own* SDXL-vs-real validation sets, but they are per-generator and the model cards themselves note degraded accuracy off-distribution; they are untested on gpt-image / Gemini Nano Banana (the metadata-stripped surfaces we care about), and our own light SDXL pass would likely defeat them the same way it defeats SynthID. Detection here stays local + signal-based (metadata + visible sparkle); do not add a bundled classifier dependency.
|
||||
- **SynthID v2 vs default pipeline:** the SDXL-based default profile (since May 2026) defeats SynthID v2. **Verified end-to-end (May 2026):** local SDXL run on a Gemini 3 Pro output, checked via the Gemini app's "Verify with SynthID" feature, returned "no SynthID watermark detected". Also confirmed against **OpenAI's** SynthID (2026-05-23): a fresh ChatGPT/gpt-image output read "SynthID detected" on openai.com/verify before the local SDXL run and "SynthID not detected" after (corpus regression chain: pos `4ef377bd` -> cleaned `47188e88`). The same configuration is used in raiw-app production (`fal-ai/fast-sdxl/image-to-image`, strength 0.05, steps 50, guidance 7.5, no pre-downscale). fal's own `llms.txt` for `fast-sdxl` names the base checkpoint as `stabilityai/stable-diffusion-xl-base-1.0` (verified 2026-05-25) -- the exact checkpoint the local CLI defaults to (`DEFAULT_MODEL_ID`). So the local `invisible` default is weight-for-weight identical to prod; "fast-sdxl" is fal's optimized serving, not different weights. After the native-resolution fix the local pipeline matches prod on weights + strength + steps + guidance + resolution. SD-1.5 dreamshaper at 768 px was previously the default and does NOT defeat v2 — verified empirically against the same feature (strength 0.04, 0.10, and elastic warp α∈{5,8} all flagged positive). That SD-1.5 path was removed; only `default` (SDXL) and `ctrlregen` profiles remain. **Scope of the claim: defeating the SynthID verifier is NOT the same as forensic invisibility.** "Removing the Watermark Is Not Enough: Forensic Stealth in Generative-AI Watermark Removal" (arXiv:2605.09203, 2026-05) shows that six removal attacks across four families (UnMarker, CtrlRegen+, WatermarkAttacker, etc.) all leave forensic traces: independent detectors flag *removal-processed* images vs genuinely-clean ones at **>98% TPR at 1% FPR**. So our SDXL pass makes the oracle read "SynthID not detected," but the output can still be classifiable as "an image that went through a removal pipeline." Do not over-claim "indistinguishable from a real photo." https://arxiv.org/abs/2605.09203
|
||||
@@ -4,18 +4,25 @@ Remove **visible** and **invisible** AI watermarks from images generated by Goog
|
||||
|
||||
Strips SynthID, C2PA Content Credentials, EXIF/XMP "Made with AI" labels, and visible sparkle overlays — all in one command.
|
||||
|
||||
> ## Try it online: [raiw.cc](https://raiw.cc)
|
||||
>
|
||||
> No Python, no GPU, no setup. Visible-watermark and metadata removal are **free**. Invisible-watermark removal (SynthID / SDXL regeneration) normally needs a local GPU and ~2 GB of models. On **[raiw.cc](https://raiw.cc)** it runs on cloud GPUs in one click for a small per-image fee.
|
||||
|
||||
[](https://github.com/sponsors/wiltodelta)
|
||||
|
||||
If this tool saves you time, consider [sponsoring its development](https://github.com/sponsors/wiltodelta).
|
||||
|
||||
## Features
|
||||
|
||||
- **Visible watermark removal** — Gemini / Nano Banana sparkle logo via reverse alpha blending (fast, offline, deterministic)
|
||||
- **Invisible watermark removal** — SynthID, StableSignature, TreeRing via diffusion-based regeneration
|
||||
- **Invisible watermark removal** — SynthID, StableSignature, TreeRing via diffusion-based regeneration (needs a local GPU, or run it with no setup on [raiw.cc](https://raiw.cc))
|
||||
- **AI metadata stripping** — EXIF, PNG text chunks, C2PA provenance manifests (PNG / JPEG / AVIF / HEIF / JPEG-XL), XMP DigitalSourceType
|
||||
- **"Made with AI" label removal** — removes the metadata that triggers AI labels on Instagram, Facebook, X (Twitter)
|
||||
- **Analog Humanizer** — film grain and chromatic aberration to bypass AI image classifiers
|
||||
- **Smart Face Protection** — automatic extraction and blending of human faces to prevent AI distortion
|
||||
- **Batch processing** — process entire directories
|
||||
- **Detection** — three-stage NCC watermark detection with confidence scoring
|
||||
|
||||
> **Try it online** — don't want to install anything? Use [raiw.cc](https://raiw.cc), a free web service powered by this library.
|
||||
- **Provenance detection (`identify`)** — aggregate C2PA issuer, IPTC "Made with AI", embedded SD/ComfyUI params, EXIF/XMP generator tags, the SynthID metadata proxy, the visible sparkle, and the open SD/SDXL/FLUX invisible watermark into one origin-platform + watermark-inventory verdict (`--json` for machine output)
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -27,17 +34,25 @@ Strips SynthID, C2PA Content Credentials, EXIF/XMP "Made with AI" labels, and vi
|
||||
|
||||
| AI model | Visible watermark | Invisible watermark | Metadata | Our approach |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| **Google Gemini / Nano Banana / Gemini 3 Pro** | ✅ Sparkle logo | ✅ SynthID v1 + v2 (default SDXL pipeline at native ~1024 px) | ✅ C2PA + EXIF | Alpha reversal + diffusion + metadata strip |
|
||||
| **Google Gemini / Nano Banana / Gemini 3 Pro** | ✅ Sparkle logo | ✅ SynthID v1 + v2 (default SDXL pipeline, native resolution) | ✅ C2PA + EXIF | Alpha reversal + diffusion + metadata strip |
|
||||
| **OpenAI DALL-E 3 / ChatGPT** | — | — | ✅ C2PA manifest | Metadata strip |
|
||||
| **OpenAI ChatGPT Images 2.0** (gpt-image-2) | — | ⚠️ imperceptible pixel watermark (no public detector yet) | ✅ C2PA manifest (verified) | Diffusion regeneration + metadata strip |
|
||||
| **Stable Diffusion (AUTOMATIC1111, ComfyUI)** | — | ✅ DWT / steganographic | ✅ PNG text chunks | Diffusion regeneration + metadata strip |
|
||||
| **OpenAI ChatGPT Images 2.0** (gpt-image-2) | — | ✅ SynthID + content-specific pixel watermark (since May 2026; no local decoder, openai.com/verify oracle) | ✅ C2PA manifest (verified) | Diffusion regeneration + metadata strip |
|
||||
| **Stable Diffusion / SDXL (AUTOMATIC1111, ComfyUI)** | — | ✅ DWT-DCT (imwatermark — locally detectable) | ✅ PNG text chunks | Diffusion regeneration + metadata strip |
|
||||
| **Black Forest Labs FLUX** | — | ✅ DWT-DCT (imwatermark — locally detectable) | ✅ C2PA (FLUX.2 Pro) | Diffusion regeneration + metadata strip |
|
||||
| **Adobe Firefly** | — | — | ✅ Content Credentials (C2PA) | Metadata strip |
|
||||
| **Stability AI** (DreamStudio / Stable Image) | — | — | ✅ C2PA ("Stability AI Ltd") | Metadata strip |
|
||||
| **Microsoft Designer / Bing Image Creator** | — | ✅ SynthID via DALL-E backend (Designer) | ✅ C2PA (Bing runs MAI-Image, signed "Microsoft") | Metadata strip |
|
||||
| **xAI Grok (Aurora)** | — | — | ✅ EXIF signature scheme (no C2PA): `Signature:` blob + UUID `Artist` | Detected (`identify`); metadata strip |
|
||||
| **Midjourney** | — | — | ✅ EXIF + XMP (prompt, model, seed) | Metadata strip |
|
||||
| **Meta AI** | — | — | ✅ IPTC "Made with AI" (digitalSourceType) | Metadata strip (removes the label) |
|
||||
| **Doubao** (ByteDance) / China AIGC generators | — | — | ✅ TC260 `<TC260:AIGC>` XMP label (China's mandatory AI labeling) | Metadata strip |
|
||||
| **StableSignature** (Meta) | — | ✅ In-model watermark | — | Diffusion regeneration |
|
||||
| **TreeRing** | — | ✅ Latent space watermark | — | Diffusion regeneration |
|
||||
|
||||
> Visible watermarks (logo overlays) are currently used only by Google Gemini / Nano Banana. Other services rely on invisible watermarks and/or metadata. Our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain.
|
||||
|
||||
> **Detection:** `remove-ai-watermarks identify <image>` reports the origin platform and watermark inventory for all the signals above — C2PA issuer, IPTC "Made with AI", the China TC260 AIGC label, embedded generation params, EXIF/XMP generator tags, the SynthID metadata proxy, the visible sparkle, and (with the `[detect]` extra) the open SD/SDXL/FLUX invisible watermark. The SynthID *pixel* watermark has no local decoder, so it is reported as a metadata proxy only.
|
||||
|
||||
## How it works
|
||||
|
||||
### Removing the Gemini / Nano Banana sparkle watermark
|
||||
@@ -65,17 +80,19 @@ Google embeds **SynthID** into every image generated by Gemini / Nano Banana. Ot
|
||||
The removal pipeline (default profile, SDXL):
|
||||
|
||||
```text
|
||||
image → resize to ~1024px (SDXL native) → encode to latent space (VAE)
|
||||
image → encode to latent space (VAE) at native resolution
|
||||
→ add controlled noise (forward diffusion)
|
||||
→ denoise (reverse diffusion, ~50 steps at strength 0.05)
|
||||
→ decode back to pixels (VAE) → upscale to original resolution
|
||||
→ decode back to pixels (VAE)
|
||||
```
|
||||
|
||||
SDXL is the default since May 2026: empirically defeats SynthID v2 on Gemini 3 Pro outputs, where the older SD-1.5 pipeline at 768 px did not. The SD-1.5 path was removed once it was verified not to handle v2.
|
||||
By default the image is processed at its **native resolution** with no pre-downscale, matching the hosted raiw.cc backend (fal `fast-sdxl`, which is `stabilityai/stable-diffusion-xl-base-1.0` — the same checkpoint the CLI defaults to). At strength ~0.05 SDXL img2img does not need the input shrunk, and the old forced downscale-to-1024 then upscale-back round-trip was the main quality loss. Pass `--max-resolution N` to cap the long side only when a very large image runs out of GPU/MPS memory (it reintroduces that lossy round-trip).
|
||||
|
||||
SDXL is the default since May 2026: empirically defeats SynthID v2 on Gemini 3 Pro outputs, where the older SD-1.5 pipeline at 768 px did not. The SD-1.5 path was removed once it was verified not to handle v2. Note the scope: this defeats the SynthID *verifier*, which is not the same as being forensically indistinguishable from a real photo. Recent work ([arXiv:2605.09203](https://arxiv.org/abs/2605.09203)) shows watermark-removal pipelines leave detectable traces, so a separate "this image was processed" classifier can still flag the output.
|
||||
|
||||
**Face Protection**: before diffusion, YOLO detects people in the image and extracts them. After diffusion, the original faces are blended back with a soft elliptical mask to prevent AI distortion of facial features.
|
||||
|
||||
**Analog Humanizer**: optional film grain and chromatic aberration injection that makes the output indistinguishable from a photo of a screen, defeating AI-generated image classifiers.
|
||||
**Analog Humanizer**: optional film grain and chromatic aberration injection that mimics a photo of a screen, raising the bar for AI-generated image classifiers. (It frustrates generic classifiers but does not guarantee forensic invisibility — see the [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) note above.)
|
||||
|
||||
### Stripping C2PA, EXIF, and "Made with AI" metadata
|
||||
|
||||
@@ -135,6 +152,13 @@ After installation the `remove-ai-watermarks` command is available system-wide.
|
||||
> ```bash
|
||||
> pip install -e ".[gpu]" # or: uv pip install -e ".[gpu]"
|
||||
> ```
|
||||
>
|
||||
> To let `identify` decode the open Stable Diffusion / SDXL / FLUX invisible
|
||||
> watermarks, install the `detect` extra (adds the `invisible-watermark` decoder):
|
||||
>
|
||||
> ```bash
|
||||
> pip install -e ".[detect]" # or: uv pip install -e ".[detect]"
|
||||
> ```
|
||||
|
||||
#### Invisible watermark removal
|
||||
|
||||
@@ -180,13 +204,28 @@ remove-ai-watermarks batch ./images/ --mode all
|
||||
#### Individual commands
|
||||
|
||||
```bash
|
||||
# Identify provenance: where an image was made + its watermark inventory.
|
||||
# Aggregates C2PA, IPTC "Made with AI", embedded SD/ComfyUI params, EXIF/XMP
|
||||
# generator tags (incl. inside AVIF/HEIF), the SynthID proxy, the visible Gemini
|
||||
# sparkle, and (with the [detect] extra) the open SD/SDXL/FLUX invisible
|
||||
# watermark into one verdict. Reports "unknown"
|
||||
# (never "clean") when no signal is found, since stripped metadata is not proof
|
||||
# of a clean origin. Add --json for machine-readable output.
|
||||
remove-ai-watermarks identify image.png
|
||||
|
||||
# Visible watermark only (Gemini / Nano Banana sparkle) — fast, offline
|
||||
remove-ai-watermarks visible image.png -o clean.png
|
||||
|
||||
# Invisible watermark only (SynthID etc.) — requires GPU
|
||||
remove-ai-watermarks invisible image.png -o clean.png --humanize 4.0
|
||||
# Runs at native resolution by default. On a very large image that OOMs the
|
||||
# GPU/MPS, cap the long side: --max-resolution 2048
|
||||
|
||||
# Check / strip AI metadata (C2PA, EXIF, "Made with AI" labels)
|
||||
# --check also flags SynthID-bearing sources: a C2PA manifest signed by
|
||||
# Google or OpenAI implies an invisible SynthID watermark in the pixels
|
||||
# (both vendors pair the two). Adobe Firefly / Microsoft sign C2PA without
|
||||
# SynthID, so they are reported as C2PA only.
|
||||
remove-ai-watermarks metadata image.png --check
|
||||
remove-ai-watermarks metadata image.png --remove
|
||||
|
||||
@@ -253,7 +292,11 @@ pip install certifi
|
||||
|
||||
Tracked but not yet implemented:
|
||||
|
||||
- **SynthID-Image v2 automated regression test**. The default SDXL profile defeats v2 per manual checks against the [Gemini app](https://support.google.com/gemini/answer/16722517)'s "Verify with SynthID" feature on a Gemini 3 Pro output (May 2026). An automated end-to-end test would need either programmatic access to the [SynthID Detector portal](https://blog.google/innovation-and-ai/products/google-synthid-ai-content-detector/) (waitlist for media professionals and researchers) or an offline surrogate detector. Open.
|
||||
- **SynthID-Image v2 automated regression test**. The default SDXL profile defeats v2 per manual checks against the [Gemini app](https://support.google.com/gemini/answer/16722517)'s "Verify with SynthID" feature on a Gemini 3 Pro output (May 2026). An automated end-to-end test would need either programmatic access to the [SynthID Detector portal](https://blog.google/innovation-and-ai/products/google-synthid-ai-content-detector/) (waitlist for media professionals and researchers) or an offline surrogate detector. The spectral phase-coherence surrogate from [reverse-SynthID](https://github.com/aloshdenny/reverse-SynthID) was evaluated and does not separate watermarked from cleaned real-content images (it only fires on controlled solid-color references at exact resolution), so it is not a usable oracle. Open.
|
||||
- **Local SynthID *pixel* detector**. Not feasible today: Google's decoder is proprietary, and magnitude/carrier spectral methods do not separate real content (confirmed by three independent evaluations, including a from-scratch gpt-image pilot; see CLAUDE.md). Blocked on either (a) a programmatic generation path (OpenAI / Gemini API) to build a per-(model, resolution) labeled corpus at scale, or (b) a raw watermarked-output dataset. If data arrives, the next approach to try is a learned classifier on diverse content rather than a fixed carrier codebook.
|
||||
- **Grow the SynthID reference corpus** (`data/synthid_corpus/`) with oracle-labeled samples per model and resolution (Gemini app for Google, openai.com/verify for OpenAI). Prerequisite for any pixel-detector attempt and for an automated removal-regression set.
|
||||
- **Real non-PNG C2PA fixtures**. SynthID-source detection for JPEG / WebP / AVIF is currently covered only by synthetic byte blobs; replace with real vendor-emitted files to ground the binary-scan path.
|
||||
- **Maintenance debt**. Clear strict-pyright debt in `remove_ai_metadata` / `cli.py` (untyped piexif / PIL / click / rich) so `maintain.sh` can finish green. (`uv-secure` is already clean since `idna` was bumped to 3.16.)
|
||||
- **AVIF / HEIF / JPEG-XL detection limits**. Removal strips top-level C2PA `uuid` and JUMBF `jumb` boxes. EXIF/XMP boxes inside these containers are not yet scrubbed (PNG and JPEG are fully covered).
|
||||
- **Video pipeline (`noai-video`)**: per-frame inpainting and tracking for Sora 2 dynamic logo, Veo 3.1 badge, Kling, Runway. Separate package, not folded into this repo.
|
||||
|
||||
@@ -267,17 +310,19 @@ Watermarking and provenance for AI-generated content is now regulated in several
|
||||
|
||||
| Jurisdiction | Instrument | Status (May 2026) | Relevance |
|
||||
| --- | --- | --- | --- |
|
||||
| EU | AI Act, Article 50(2) | Marking obligations postponed to **2 December 2026** under the December 2025 omnibus agreement. Code of Practice finalising May/June 2026. | Removing mandated provenance markers with intent to deceive may be sanctioned under national implementations. |
|
||||
| US (federal) | COPIED Act | Enacted 2025. | Criminalises removal of provenance information with intent to deceive about content origin. The tool itself is lawful; usage may not be. |
|
||||
| US (state) | CA AB 2655, TX SB 751, similar | In force. | Content-specific (election deepfakes, sexual deepfakes). Not tool-specific. |
|
||||
| China | Deep Synthesis Regulation, 2025 updates | In force. | Mandatory visible label for AI content. Removal is an administrative offence. |
|
||||
| UK | Online Safety Act, 2025 transparency extension | In force. | Platform obligations, not user obligations. |
|
||||
| EU | AI Act, Article 50 | Transparency duties apply from **2 August 2026**. Legacy generative systems (placed on the market before that date) get a grandfathering period to **2 December 2026** for the Article 50(2) marking duty, under the Digital Omnibus (Commission proposal Nov 2025; co-legislator political agreement 7 May 2026). Article 50 guidelines and a marking Code of Practice are being finalised through 2026. | Removing mandated provenance markers with intent to deceive may be sanctioned under national implementations. |
|
||||
| US (federal) | COPIED Act | **Reintroduced April 2025; not enacted** (pending in the Senate). | If passed, would set NIST provenance standards and prohibit tampering with / removing provenance information. The tool itself is lawful; usage may not be. |
|
||||
| US (state) | CA AB 2655, TX SB 751, similar | TX SB 751 in force; **CA AB 2655 struck down** by a federal court (Aug 2025, Section 230 / First Amendment). | Content-specific (election deepfakes, sexual deepfakes). Not tool-specific. |
|
||||
| US (state) | CA AB 853 (amends the California AI Transparency Act) | Core provider duties operative **2 August 2026** (delayed from 1 January 2026); large platforms 1 January 2027; capture devices 1 January 2028. | Covered providers (1M+ monthly users) must embed a latent disclosure that is "permanent or extraordinarily difficult to remove" and offer a free detection tool. Removing that disclosure is what this tool does. |
|
||||
| South Korea | AI Framework Act (Basic Act on AI), Article 31 | In force since **January 2026** (one-year transition after promulgation). | Art. 31(3): AI output "difficult to distinguish from reality" must be labeled so users "clearly recognize" it; the draft Enforcement Decree accepts a machine-readable (invisible-watermark) label. Artistic/creative works get a presentation exception. |
|
||||
| China | Measures for Labeling AI-Generated Content (+ GB 45438-2025) | In force since **1 September 2025**. | Mandatory explicit (visible) + implicit (metadata) labels for AI content; tampering with or removing labels is prohibited. |
|
||||
| UK | Online Safety Act 2023 / Ofcom guidance | In force, but **no statutory AI-provenance or watermarking obligation**. | Ofcom encourages watermarking / provenance metadata as voluntary "attribution measures"; platform duties, not user obligations. |
|
||||
|
||||
## Threat model
|
||||
|
||||
This tool defends already-distributed AI imagery against automatic detection systems (social-platform "Made with AI" labels, third-party classifiers, content-policy filters). It does **not** retroactively anonymise generation.
|
||||
|
||||
In particular, **SynthID-Image v2** (Google, deployed October 2025 with Gemini 3 Pro / Nano Banana Pro / Imagen 4 / Veo) embeds a **136-bit payload** ([arxiv 2510.09263](https://arxiv.org/abs/2510.09263)). The payload is believed to encode a user / session identifier. If the original watermarked file ever passed through a system controlled by the prompt originator (a saved Gemini account history, a screenshot uploaded to a Google product, a backup), Google retains the ability to link that original to the generating account. Stripping the watermark from a copy you possess does not erase Google's server-side record.
|
||||
In particular, **SynthID** (Google DeepMind) is embedded across Google's generative media stack — Imagen (images), Veo (video), Lyria (audio) — and Gemini app image outputs (Nano Banana / Gemini 3 Pro, which we verified positive via the Gemini app's SynthID oracle); Google reported over 10 billion items watermarked by December 2025. It carries a **multi-bit payload** — the research paper's SynthID-O variant encodes 136-bit payloads in 512x512 images ([arxiv 2510.09263](https://arxiv.org/abs/2510.09263)). The payload is believed to encode a user / session identifier. If the original watermarked file ever passed through a system controlled by the prompt originator (a saved Gemini account history, a screenshot uploaded to a Google product, a backup), Google retains the ability to link that original to the generating account. Stripping the watermark from a copy you possess does not erase Google's server-side record.
|
||||
|
||||
Use cases where the threat model fits:
|
||||
- You generated the image yourself, want to publish it as your own work, and accept the consequences if Google ever publishes their detector logs.
|
||||
|
||||
|
After Width: | Height: | Size: 5.1 MiB |
|
After Width: | Height: | Size: 356 KiB |
|
Before Width: | Height: | Size: 4.2 MiB |
|
Before Width: | Height: | Size: 5.6 MiB |
|
Before Width: | Height: | Size: 553 KiB |
@@ -26,7 +26,7 @@ The `trainedAlgorithmicMedia` tag is what triggers "Made with AI" labels on Inst
|
||||
|
||||
### Invisible pixel-level watermark
|
||||
|
||||
OpenAI's system card for Images 2.0 states the model embeds an "imperceptible, robust, and content-specific" pixel-level watermark alongside C2PA. No public detector exists, so bypass cannot be verified empirically for this sample.
|
||||
OpenAI's system card for Images 2.0 states the model embeds an "imperceptible, robust, and content-specific" pixel-level watermark alongside C2PA. This sample was downloaded 2026-04-22, before OpenAI's 19 May 2026 rollout of Google's SynthID watermark across ChatGPT / Codex / the API, so it likely predates SynthID and carries only the original content-specific watermark. Since that rollout, the openai.com/verify tool (in preview) is the public oracle for both signals; there is still no local decoder, so bypass cannot be verified empirically here without the oracle.
|
||||
|
||||
## Reproducing the removal
|
||||
|
||||
@@ -38,4 +38,4 @@ remove-ai-watermarks metadata amur-leopard.png --remove -o amur-leopard.clean.pn
|
||||
remove-ai-watermarks all amur-leopard.png -o amur-leopard.all.png --device mps
|
||||
```
|
||||
|
||||
Note: the diffusion step downscales to 768 px, which degrades fine text on text-heavy infographics like this one. Tracked as a known limitation; see project TODO.
|
||||
Note: the diffusion step runs at native resolution by default (no pre-downscale), so fine text on a text-heavy infographic like this one is preserved. Pass `--max-resolution N` only if a very large image OOMs the GPU/MPS (that reintroduces a lossy downscale then upscale round-trip).
|
||||
@@ -0,0 +1,152 @@
|
||||
# SynthID reference corpus
|
||||
|
||||
A locally-collected, labeled image corpus for SynthID work. Two downstream uses:
|
||||
|
||||
1. **Per-resolution spectral codebook** for an experimental SynthID detector
|
||||
(carrier frequencies are resolution-dependent, so labels must record the
|
||||
exact native resolution).
|
||||
2. **Removal regression set** — verify that our pipeline turns a SynthID-positive
|
||||
image into a negative one.
|
||||
|
||||
There is no reliable local detector of the SynthID pixel watermark (Google's
|
||||
decoder is proprietary). The ground-truth label therefore comes from an
|
||||
external oracle, recorded per image in `verified_via` (see below).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
data/synthid_corpus/
|
||||
README.md # this protocol (committed)
|
||||
manifest.csv # labels + provenance (committed; one row per tracked image)
|
||||
images/ # the labeled corpus (committed)
|
||||
pos/ # SynthID present
|
||||
neg/ # SynthID absent (incl. reviewed real photos)
|
||||
cleaned/ # our pipeline output from a pos image
|
||||
refs/ # synthetic black/white calibration fills (gitignored, regenerable)
|
||||
```
|
||||
|
||||
The labeled images are committed so the corpus is reproducible and the removal
|
||||
regression set runs in CI. `manifest.csv` is kept in sync with the files on
|
||||
disk (one row per tracked image; dangling rows are pruned when files are
|
||||
removed). Before adding any image, confirm it carries no private or
|
||||
identifiable content you would not publish -- this is a public repo and git
|
||||
history is permanent. The synthetic `refs/` fills stay gitignored (regenerable,
|
||||
not part of the labeled set).
|
||||
|
||||
## Verification levels (`verified_via`)
|
||||
|
||||
Ground-truth quality, strongest first:
|
||||
|
||||
- `gemini-app` — checked via the Gemini app "Verify with SynthID" feature. Gold standard for the pixel watermark (Google models).
|
||||
- `openai-verify` — checked via openai.com/verify (gold standard for OpenAI ChatGPT/Codex/API images).
|
||||
- `synthid-portal` — checked via Google's SynthID Detector portal.
|
||||
- `c2pa-metadata` — issuer-only proxy (Google/OpenAI C2PA manifest present). Weaker: the C2PA can be stripped while the pixel watermark remains.
|
||||
- `third-party` — label asserted by an external dataset, not independently verified.
|
||||
- `none` — unverified.
|
||||
|
||||
Prefer `gemini-app` for any image that will train the codebook or gate a test.
|
||||
|
||||
## What to collect
|
||||
|
||||
For the **codebook** (per target resolution, e.g. 1024x1024, 1024x1536, 1536x2816):
|
||||
|
||||
- 30-50+ SynthID-positive outputs per resolution (more is better; ~150-200 per
|
||||
resolution materially improves carrier discovery).
|
||||
- At each target resolution, also a batch of **pure-black (#000000)** and
|
||||
**pure-white (#FFFFFF)** fills generated by the SynthID model — these isolate
|
||||
the content-independent carrier (the watermark is most of the signal there).
|
||||
|
||||
For the **regression set**:
|
||||
|
||||
- A handful of `pos` images, their `cleaned` counterparts (run through our
|
||||
pipeline), and the cleaned re-verified via `gemini-app` (should read negative).
|
||||
- `neg` controls: non-AI photos and outputs from non-SynthID models (SD,
|
||||
Midjourney, Firefly) verified negative.
|
||||
|
||||
The corpus is committed to a public repo: review every image before adding it
|
||||
and keep out anything private or identifiable you would not publish.
|
||||
|
||||
## Ingesting
|
||||
|
||||
Use `scripts/synthid_corpus.py` — it copies a file in, records its sha256,
|
||||
resolution, format, and C2PA issuer (via our own detector), and appends a row
|
||||
to `manifest.csv`:
|
||||
|
||||
```bash
|
||||
uv run python scripts/synthid_corpus.py ingest path/to/*.png \
|
||||
--label pos --source "Gemini app" --model gemini-3-pro \
|
||||
--verified-via gemini-app --notes "1024x1024 batch"
|
||||
|
||||
uv run python scripts/synthid_corpus.py status # counts by label / resolution / verification
|
||||
```
|
||||
|
||||
## Autonomous collection via Chrome MCP
|
||||
|
||||
Generation can be driven through the browser (the account must be logged in):
|
||||
|
||||
- **Gemini** (`gemini.google.com`): type `Create an image: <prompt>`, wait, hover the
|
||||
result, click the download icon (top-right). Single, reliable click. Outputs
|
||||
carry Google C2PA + SynthID. Occasionally the composer stalls in a
|
||||
"generating" state -> start a New chat to reset.
|
||||
- **ChatGPT** (`chatgpt.com`): the UI download is flaky (the fullscreen viewer
|
||||
races and can grab the previous image; the share-modal path works but is
|
||||
multi-step). Reliable path is an in-page fetch of the rendered image, which
|
||||
preserves the original bytes (C2PA intact, unlike a canvas re-encode):
|
||||
|
||||
```js
|
||||
// run in the ChatGPT tab via the browser MCP javascript tool
|
||||
(async () => {
|
||||
const imgs = [...document.querySelectorAll('img')].filter(i => i.naturalWidth >= 400);
|
||||
const img = imgs[imgs.length - 1]; // newest large image
|
||||
const b = await (await fetch(img.currentSrc || img.src)).blob();
|
||||
const a = document.createElement('a');
|
||||
a.href = URL.createObjectURL(b); a.download = 'dl.png';
|
||||
document.body.appendChild(a); a.click(); a.remove();
|
||||
return 'size=' + b.size; // do NOT return the src (privacy guard blocks query strings)
|
||||
})()
|
||||
```
|
||||
|
||||
Gotcha: confirm the returned `size` differs from the previous image before
|
||||
ingesting -- if the new image has not finished rendering, the script grabs the
|
||||
prior one (the corpus dedups by sha256, but the notes would mislabel it).
|
||||
ChatGPT also shows an A/B "which is better?" picker; click Skip first.
|
||||
|
||||
**Originals, not previews.** Some platforms render a low-res preview in the chat
|
||||
(Grok serves a ~20KB 1024px JPEG/PNG; the in-page `<img>` fetch grabs *that*, not
|
||||
the original). Previews are re-encoded and **strip metadata**, so a "clean"
|
||||
preview is not proof the original is clean. Always pull the original via the
|
||||
platform's native Download / lightbox button and sanity-check the file size (a
|
||||
20KB "1024x1024" is a preview). ChatGPT's in-chat `<img>` *is* the full-res
|
||||
oaiusercontent original, so fetch+blob is fine there; Grok needs the lightbox
|
||||
Download; Leonardo serves the original JPEG in-chat (download button matches).
|
||||
|
||||
## Per-platform watermark map (observed, May 2026)
|
||||
|
||||
What each platform actually embeds, verified by byte-scan (and Gemini-app oracle
|
||||
where noted). The detector's coverage is complementary: metadata catches C2PA /
|
||||
IPTC; `exif_generator` catches EXIF `Make`/`Software` + XMP `CreatorTool`;
|
||||
`invisible_watermark.py` (imwatermark) catches the open SD/SDXL/FLUX DWT-DCT
|
||||
watermark on pristine files; the visible detector catches the Gemini-family
|
||||
sparkle; the SynthID *pixel* itself has no local detector (oracle only).
|
||||
|
||||
| Platform | C2PA issuer | SynthID pixel | IPTC "Made with AI" | Visible sparkle | imwatermark | Corpus label |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Gemini app | Google | yes | - | yes | - | pos |
|
||||
| ChatGPT / gpt-image | OpenAI | yes | - | - | - | pos |
|
||||
| Microsoft Designer | OpenAI + Microsoft | yes (via OpenAI) | - | - | - | pos |
|
||||
| Bing Image Creator | Microsoft (MAI-Image) | no | - | - | - | pos (C2PA "Microsoft", not OpenAI) |
|
||||
| Google AI Studio (Nano Banana) | **none** | yes (oracle-confirmed) | - | yes | - | pos (metadata blind spot) |
|
||||
| Stability AI (Brand Studio) | Stability AI Ltd | no | - | - | no | pos (C2PA only) |
|
||||
| Ideogram | none | no | - | - | no | pos (EXIF `Make="Ideogram AI"` only) |
|
||||
| Meta AI | none | no | **yes** | - | - | neg (for SynthID) |
|
||||
| Leonardo.ai | none | no | no | - | no | neg |
|
||||
| Recraft | none (export strips) | no | no | - | no | neg (re-encoded export, no signal) |
|
||||
| Krea (FLUX 2 host) | none | no | no | - | no | neg (host omits the imwatermark encoder) |
|
||||
| Grok (xAI) | none (non-adopter) | no | no | - | no | neg (captured: clean low-res preview) |
|
||||
|
||||
Key takeaways:
|
||||
- The same model differs by *surface*: Gemini app wraps C2PA, AI Studio (API/playground) emits none -- only the pixel + sparkle survive.
|
||||
- Microsoft Designer's DALL-E backend inherits OpenAI's C2PA+SynthID (issuer "OpenAI, Microsoft"); Bing now runs Microsoft's own **MAI-Image** and signs C2PA as "Microsoft" (not OpenAI/DALL-E).
|
||||
- Meta uses the IPTC `digitalSourceType` marker, not C2PA or SynthID.
|
||||
- The open imwatermark fires only on *pristine* output from a pipeline that runs the encoder (diffusers default, official BFL) -- not from re-hosts (Krea, Stability hosted SDXL) or re-encoded design exports (Recraft, Canva). Ideogram's only signal is the EXIF `Make` tag.
|
||||
- Bing and Grok web UIs are uncooperative for autonomous capture (no document_idle for screenshots; blob downloads intermittently no-op; low-res in-chat previews). Use their native download button manually if a full-res sample is needed.
|
||||
|
After Width: | Height: | Size: 2.7 MiB |
|
After Width: | Height: | Size: 1.8 MiB |
|
After Width: | Height: | Size: 2.7 MiB |
|
After Width: | Height: | Size: 2.2 MiB |
|
After Width: | Height: | Size: 3.1 MiB |
|
After Width: | Height: | Size: 3.0 MiB |
|
After Width: | Height: | Size: 1.9 MiB |
|
After Width: | Height: | Size: 8.3 MiB |
|
After Width: | Height: | Size: 7.8 MiB |
|
After Width: | Height: | Size: 8.6 MiB |
@@ -0,0 +1,26 @@
|
||||
sha256,filename,label,source,model,width,height,format,c2pa_issuer,synthid_metadata,verified_via,added,notes
|
||||
4ef377bde1a1d4eff141972841938643b173f5052992a018b9a21b31ac31731e,"4ef377bd-ChatGPT Image May 23, 2026, 02_43_02 PM.png",pos,ChatGPT,gpt-image,1254,1254,png,OpenAI,yes,openai-verify,2026-05-23T21:48:12Z,fresh post-rollout 2026-05-23; openai.com/verify: SynthID+C2PA detected
|
||||
d09f84c0e4c6d8b336bf4a9a7277314e940dcb5052ae7051e785cbb3bb42d656,d09f84c0-Gemini_Generated_Image_vq7wkwvq7wkwvq7w.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,c2pa-metadata,2026-05-23T21:52:40Z,"user: latest Gemini, SynthID v2"
|
||||
7b650522d42db09568e249c04d683c469fb3e280a2c53fcd1031cb9df27c619a,"7b650522-ChatGPT Image May 24, 2026, 12_19_54 PM.png",pos,ChatGPT,gpt-image,1602,982,png,OpenAI,yes,c2pa-metadata,2026-05-24T19:20:25Z,content: misty pine forest at dawn
|
||||
fb28dba2a82cc101a92fdee5714867b32610d0564f37737fe4bb70782b8ecf32,fb28dba2-Gemini_Generated_Image_dsjlnsdsjlnsdsjl.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,c2pa-metadata,2026-05-24T19:30:25Z,content: elderly fisherman portrait
|
||||
d20d4cc936dbdfe909c52502039a9e84ba93d97b42b24a0acee5b7d6c71930ae,d20d4cc9-Gemini_Generated_Image_ug6kdpug6kdpug6k.png,pos,Gemini app,gemini,2816,1536,png,Google LLC,yes,c2pa-metadata,2026-05-24T19:33:15Z,content: red coffee mug product shot
|
||||
28f323345f6496d936c3f1a72f671ddf59d0f81565c24a63bf3286860f633afe,28f32334-chatgpt_fisherman.png,pos,ChatGPT,gpt-image,1023,1537,png,OpenAI,yes,c2pa-metadata,2026-05-24T19:39:20Z,content: elderly fisherman portrait (fetch+blob dl)
|
||||
88e61a384c2e0b12d97bc66046e4a10542b2987448ba89c4b49e66311e969c84,88e61a38-chatgpt_tokyo.png,pos,ChatGPT,gpt-image,1023,1537,png,OpenAI,yes,c2pa-metadata,2026-05-24T19:42:02Z,content: tokyo street night (fetch+blob dl)
|
||||
1fa5f77f710c11a4cc69fed60195450def734401ed57c2600a84ce191f985440,1fa5f77f-IMG_0786.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:47Z,post photo; no C2PA/SynthID (verified)
|
||||
2ea228ed270cd169e9d38bc3f1a162de7edbf54b6e5ad3c701a3c90010ec7067,2ea228ed-IMG_1791.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:47Z,post photo; no C2PA/SynthID (verified)
|
||||
0bb4c176d83fbcbf4e628de7587d6183b9134c4f3fa85f55b96e94185273c7f6,0bb4c176-IMG_1790.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:47Z,post photo; no C2PA/SynthID (verified)
|
||||
3fc0c4253f86427777904f41355eacbdcc1a29f6897ec694c7fc68b6e7f70846,3fc0c425-IMG_1450.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:48Z,post photo; no C2PA/SynthID (verified)
|
||||
2b148db2b1a314a87647a03828b3d235e7c4c939252f448b572b7658c7bb9723,2b148db2-IMG_1832.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:48Z,post photo; no C2PA/SynthID (verified)
|
||||
2d02fed0b6d60ce1142eadac9837a83896569dddbdad7cc5cfdec018f0506d36,2d02fed0-IMG_1520.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:48Z,post photo; no C2PA/SynthID (verified)
|
||||
8f170c06f843c2bcf4cf6e249dd76365287fdb3a322637ba93c570f50cb19772,8f170c06-IMG_1078.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:48Z,post photo; no C2PA/SynthID (verified)
|
||||
3fc4c8316012abc462df5534d4d995e5af84d35076a2927331107ff994293d9e,3fc4c831-IMG_2566.HEIC,neg,iPhone (me/post_photos),,5712,4284,heic,,,none,2026-05-24T20:58:48Z,post photo; no C2PA/SynthID (verified)
|
||||
5fe59521d579b536340253d2bcaa7c28ddd2485fa964fc27a9b9e5118ad0cdd1,5fe59521-IMG_1496.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:48Z,post photo; no C2PA/SynthID (verified)
|
||||
4ead7918a5aeea7fba78b36af44358fea7ed1db7f46f4bc675152b9d04e68c38,4ead7918-IMG_1300.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:48Z,post photo; no C2PA/SynthID (verified)
|
||||
4c795238b89178cf52a3674e721ed7f4cd5068028385491d12171a3c62545c35,4c795238-IMG_3034.HEIC,neg,iPhone (me/post_photos),,5712,4284,heic,,,none,2026-05-24T20:58:49Z,post photo; no C2PA/SynthID (verified)
|
||||
5abfaccb37c549de67c7f3a751a528a423e24a82c643fdb29094be8debbb206d,5abfaccb-IMG_3018.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:49Z,post photo; no C2PA/SynthID (verified)
|
||||
5fed1923d513c1e9ffcba2f240e617fa9344fb39d35144169570ece8b0bd0f33,5fed1923-IMG_0272.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:49Z,post photo; no C2PA/SynthID (verified)
|
||||
06b04d8fe8e1cd6bee9a973f93bfda37586924cbfec7d372f59d52aa9196160b,06b04d8f-IMG_1474.HEIC,neg,iPhone (me/post_photos),,5712,4284,heic,,,none,2026-05-24T20:58:49Z,post photo; no C2PA/SynthID (verified)
|
||||
8fdb574a94e65e14ac29017cf2d5a2ede18a8c4e3f12e04c64292b0d38570062,8fdb574a-IMG_3557.HEIC,neg,iPhone (me/post_photos),,4032,3024,heic,,,none,2026-05-24T20:58:49Z,post photo; no C2PA/SynthID (verified)
|
||||
c86973424817f62510e2a312b85c52e05adf47ace87a8e717fd442607596f501,c8697342-aistudio_lake.png,pos,Google AI Studio (Nano Banana),gemini-2.5-flash-image,1024,1024,png,,,gemini-app,2026-05-24T21:39:09Z,"API/playground: SynthID pixel CONFIRMED (Gemini-app oracle) + visible sparkle, but NO C2PA/IPTC -> synthid_source blind spot"
|
||||
1f81827c06d67cf6f6c7f5d53ec8f9738183942a6d1d2717b161fea0fdcc540a,1f81827c-Designer.png,pos,Microsoft Designer,dall-e (Designer),1024,1024,png,"OpenAI, Microsoft",yes,c2pa-metadata,2026-05-24T22:18:40Z,C2PA issuer OpenAI+Microsoft; synthid_source=OpenAI (DALL-E surface inherits OpenAI SynthID+C2PA)
|
||||
f6dd47a5ffd319aea21bf10dcf9877097666420b02c2620080bac12b03976e7e,f6dd47a5-4ef377bd-gpt-image-2-cleaned.png,cleaned,"our pipeline (invisible/SDXL, native-res default)",stabilityai/stable-diffusion-xl-base-1.0,1254,1254,png,,,openai-verify,2026-05-25T20:50:38Z,"cleaned from 4ef377bd via v0.5.3 'all' at native 1254x1254 (prod-equivalent); openai.com/verify: SynthID NOT detected. Re-confirms #10 native-res default defeats OpenAI SynthID (closes #15 root cause). Note: native res OOMs on 20GB MPS, auto-fell back to CPU."
|
||||
|
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.4.1"
|
||||
version = "0.5.5"
|
||||
description = "Remove visible and invisible AI watermarks from images (Gemini / Nano Banana, ChatGPT, Stable Diffusion)"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
@@ -26,13 +26,24 @@ gpu = [
|
||||
"ultralytics>=8.0.0",
|
||||
"color-matcher>=0.5.0",
|
||||
]
|
||||
# Open invisible-watermark (imwatermark) decoder for detecting the DWT-DCT
|
||||
# watermarks embedded by Stable Diffusion / SDXL / FLUX. Optional because it
|
||||
# pulls non-headless opencv AND torch (invisible-watermark declares torch a hard
|
||||
# dependency, and WatermarkDecoder eagerly imports rivaGan -> torch at import
|
||||
# time, so the dwtDct-only detect path still needs torch present even though it
|
||||
# never runs on GPU). So `detect` alone pulls torch -- no need to add `gpu` for
|
||||
# detection. identify() guards the import and skips the signal when absent.
|
||||
detect = [
|
||||
"invisible-watermark>=0.2.0",
|
||||
]
|
||||
dev = [
|
||||
"pytest>=8.0.0",
|
||||
"pytest-cov>=4.1.0",
|
||||
"ruff>=0.4.0",
|
||||
"pyright>=1.1.0",
|
||||
"invisible-watermark>=0.2.0",
|
||||
]
|
||||
all = ["remove-ai-watermarks[gpu,dev]"]
|
||||
all = ["remove-ai-watermarks[gpu,detect,dev]"]
|
||||
|
||||
# diffusers 0.38.0 (security fix for GHSA-98h9-4798-4q5v) declares a dependency
|
||||
# on safetensors>=0.8.0rc0 — a pre-release. Allow pre-releases globally so the
|
||||
|
||||
@@ -0,0 +1,240 @@
|
||||
"""Ingest and inspect the local SynthID reference corpus.
|
||||
|
||||
Copies images into ``data/synthid_corpus/images/<label>/`` and records one row
|
||||
per image in ``manifest.csv`` (sha256, resolution, format, C2PA issuer, and the
|
||||
external verification level). Dogfoods the project's own C2PA detector so the
|
||||
recorded metadata matches what the library sees.
|
||||
|
||||
See ``data/synthid_corpus/README.md`` for the collection protocol.
|
||||
|
||||
Usage:
|
||||
uv run python scripts/synthid_corpus.py ingest IMAGES... --label pos \\
|
||||
--source "Gemini app" --model gemini-3-pro --verified-via gemini-app
|
||||
uv run python scripts/synthid_corpus.py status
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import hashlib
|
||||
import logging
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from collections import Counter
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from PIL import Image
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
from remove_ai_watermarks.noai.c2pa import extract_c2pa_info
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
console = Console()
|
||||
|
||||
DEFAULT_ROOT = Path(__file__).resolve().parent.parent / "data" / "synthid_corpus"
|
||||
LABELS = ("pos", "neg", "cleaned")
|
||||
VERIFIED_VIA = ("gemini-app", "openai-verify", "synthid-portal", "c2pa-metadata", "third-party", "none")
|
||||
FIELDNAMES = [
|
||||
"sha256",
|
||||
"filename",
|
||||
"label",
|
||||
"source",
|
||||
"model",
|
||||
"width",
|
||||
"height",
|
||||
"format",
|
||||
"c2pa_issuer",
|
||||
"synthid_metadata",
|
||||
"verified_via",
|
||||
"added",
|
||||
"notes",
|
||||
]
|
||||
|
||||
|
||||
def _manifest_path(root: Path) -> Path:
|
||||
return root / "manifest.csv"
|
||||
|
||||
|
||||
def _read_manifest(root: Path) -> list[dict[str, str]]:
|
||||
path = _manifest_path(root)
|
||||
if not path.exists():
|
||||
return []
|
||||
with open(path, newline="") as f:
|
||||
return list(csv.DictReader(f))
|
||||
|
||||
|
||||
def _sha256(path: Path) -> str:
|
||||
h = hashlib.sha256()
|
||||
with open(path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(1 << 20), b""):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def _dims_via_sips(path: Path) -> tuple[int, int]:
|
||||
"""Read pixel dimensions via macOS ``sips`` (handles HEIC, which PIL can't
|
||||
open without pillow-heif). Returns (0, 0) off macOS or on failure."""
|
||||
sips = shutil.which("sips")
|
||||
if sys.platform != "darwin" or not sips:
|
||||
return 0, 0
|
||||
try:
|
||||
# Fixed command; the only variable is a local file path passed to ingest.
|
||||
out = subprocess.run( # noqa: S603
|
||||
[sips, "-g", "pixelWidth", "-g", "pixelHeight", str(path)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=15,
|
||||
).stdout
|
||||
dims = {"pixelWidth": 0, "pixelHeight": 0}
|
||||
for line in out.splitlines():
|
||||
for key in dims:
|
||||
if f"{key}:" in line:
|
||||
dims[key] = int(line.split(":")[1].strip())
|
||||
return dims["pixelWidth"], dims["pixelHeight"]
|
||||
except Exception: # sips missing/odd output; dims stay unknown
|
||||
return 0, 0
|
||||
|
||||
|
||||
def _probe(path: Path) -> tuple[int, int, str, str, bool]:
|
||||
"""Return (width, height, format, c2pa_issuer, synthid_metadata)."""
|
||||
width = height = 0
|
||||
fmt = path.suffix.lstrip(".").lower()
|
||||
try:
|
||||
with Image.open(path) as img:
|
||||
width, height = img.size
|
||||
fmt = (img.format or fmt).lower()
|
||||
except Exception as exc: # unknown/user formats can raise non-OSError; see CLAUDE.md
|
||||
log.debug("PIL could not open %s: %s", path, exc)
|
||||
|
||||
if width == 0: # e.g. HEIC from iPhone — fall back to macOS sips
|
||||
width, height = _dims_via_sips(path)
|
||||
|
||||
info = extract_c2pa_info(path)
|
||||
issuer = str(info.get("issuer", ""))
|
||||
synthid = "synthid_watermark" in info
|
||||
return width, height, fmt, issuer, synthid
|
||||
|
||||
|
||||
@click.group()
|
||||
def cli() -> None:
|
||||
"""Manage the local SynthID reference corpus."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s")
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True, path_type=Path))
|
||||
@click.option("--label", required=True, type=click.Choice(LABELS), help="SynthID label.")
|
||||
@click.option("--source", default="", help="Where the image came from (free text).")
|
||||
@click.option("--model", default="", help="Generating model, e.g. gemini-3-pro.")
|
||||
@click.option(
|
||||
"--verified-via", "verified_via", default="none", type=click.Choice(VERIFIED_VIA), help="Ground-truth oracle."
|
||||
)
|
||||
@click.option("--notes", default="", help="Free-text notes (e.g. resolution batch).")
|
||||
@click.option("--root", type=click.Path(path_type=Path), default=DEFAULT_ROOT, help="Corpus root.")
|
||||
def ingest(
|
||||
images: tuple[Path, ...],
|
||||
label: str,
|
||||
source: str,
|
||||
model: str,
|
||||
verified_via: str,
|
||||
notes: str,
|
||||
root: Path,
|
||||
) -> None:
|
||||
"""Copy IMAGES into the corpus and append rows to the manifest."""
|
||||
dest_dir = root / "images" / label
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
rows = _read_manifest(root)
|
||||
seen = {r["sha256"] for r in rows}
|
||||
added = 0
|
||||
new_rows: list[dict[str, str]] = []
|
||||
|
||||
for src in images:
|
||||
digest = _sha256(src)
|
||||
if digest in seen:
|
||||
console.print(f" [dim]skip (duplicate sha256): {src.name}[/]")
|
||||
continue
|
||||
seen.add(digest)
|
||||
|
||||
width, height, fmt, issuer, synthid = _probe(src)
|
||||
stored_name = f"{digest[:8]}-{src.name}"
|
||||
shutil.copy2(src, dest_dir / stored_name)
|
||||
|
||||
new_rows.append(
|
||||
{
|
||||
"sha256": digest,
|
||||
"filename": stored_name,
|
||||
"label": label,
|
||||
"source": source,
|
||||
"model": model,
|
||||
"width": str(width),
|
||||
"height": str(height),
|
||||
"format": fmt,
|
||||
"c2pa_issuer": issuer,
|
||||
"synthid_metadata": "yes" if synthid else "",
|
||||
"verified_via": verified_via,
|
||||
"added": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"notes": notes,
|
||||
}
|
||||
)
|
||||
added += 1
|
||||
flag = " [yellow](C2PA-SynthID)[/]" if synthid else ""
|
||||
console.print(f" [green]+[/] {label}/{stored_name} {width}x{height} {fmt}{flag}")
|
||||
|
||||
if new_rows:
|
||||
path = _manifest_path(root)
|
||||
write_header = not path.exists()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(path, "a", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=FIELDNAMES)
|
||||
if write_header:
|
||||
writer.writeheader()
|
||||
writer.writerows(new_rows)
|
||||
|
||||
console.print(f"\n Ingested [bold]{added}[/] image(s); skipped {len(images) - added} duplicate(s).")
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.option("--root", type=click.Path(path_type=Path), default=DEFAULT_ROOT, help="Corpus root.")
|
||||
def status(root: Path) -> None:
|
||||
"""Print corpus counts by label, resolution, and verification level."""
|
||||
rows = _read_manifest(root)
|
||||
if not rows:
|
||||
console.print(" [dim]Corpus is empty.[/]")
|
||||
return
|
||||
|
||||
by_label = Counter(r["label"] for r in rows)
|
||||
by_verified = Counter(r["verified_via"] for r in rows)
|
||||
by_res = Counter(f"{r['label']} {r['width']}x{r['height']}" for r in rows)
|
||||
|
||||
console.print(f"\n [bold]{len(rows)}[/] image(s) in {root}")
|
||||
|
||||
t = Table(title="By label", show_header=True, header_style="bold")
|
||||
t.add_column("Label")
|
||||
t.add_column("Count", justify="right")
|
||||
for k in LABELS:
|
||||
if by_label.get(k):
|
||||
t.add_row(k, str(by_label[k]))
|
||||
console.print(t)
|
||||
|
||||
t = Table(title="By label x resolution", show_header=True, header_style="bold")
|
||||
t.add_column("Label / resolution")
|
||||
t.add_column("Count", justify="right")
|
||||
for k, v in sorted(by_res.items()):
|
||||
t.add_row(k, str(v))
|
||||
console.print(t)
|
||||
|
||||
t = Table(title="By verification", show_header=True, header_style="bold")
|
||||
t.add_column("verified_via")
|
||||
t.add_column("Count", justify="right")
|
||||
for k, v in by_verified.most_common():
|
||||
t.add_row(k, str(v))
|
||||
console.print(t)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
@@ -0,0 +1,149 @@
|
||||
"""SynthID pixel-carrier probe -- EXPERIMENTAL / DIAGNOSTIC ONLY.
|
||||
|
||||
There is no local detector of the SynthID pixel watermark on real content: the
|
||||
carrier drowns in scene texture (see CLAUDE.md, confirmed repeatedly). This
|
||||
probe is meaningful ONLY on **solid-color fills**, where the per-pixel deviation
|
||||
from the image mean is essentially the watermark carrier (almost all the
|
||||
variance). It answers two controlled questions, neither of which is a
|
||||
real-content detector:
|
||||
|
||||
consistency IMAGES...
|
||||
Mean pairwise normalized cross-correlation (NCC) of the carriers across
|
||||
independent solid fills from one model, vs a random baseline. Genuine
|
||||
SynthID positives share a fixed carrier, so they correlate well above
|
||||
random (the pilot saw ~0.92 on gpt-image black fills); clean fills don't.
|
||||
|
||||
removal --pos P... --cleaned C...
|
||||
Build a carrier template from the positive fills, then compare how the
|
||||
positives and the pipeline-cleaned fills correlate to it. If removal
|
||||
worked, the cleaned correlation collapses toward the random baseline --
|
||||
pixel-domain evidence that the pipeline destroys the carrier, not just the
|
||||
C2PA metadata.
|
||||
|
||||
Do NOT run this on real-content images; the numbers are uninformative there.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import click
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
from rich.console import Console
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from numpy.typing import NDArray
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
console = Console()
|
||||
|
||||
|
||||
def load_gray(path: str) -> NDArray[np.float64]:
|
||||
"""Load an image as a float64 grayscale array (mean of RGB channels)."""
|
||||
with Image.open(path) as img:
|
||||
return np.asarray(img.convert("RGB"), dtype=np.float64).mean(axis=2)
|
||||
|
||||
|
||||
def carrier(gray: NDArray[np.float64]) -> NDArray[np.float64]:
|
||||
"""Zero-mean, unit-norm residual of a solid-fill image -- its carrier.
|
||||
|
||||
Returns a flattened unit-norm vector for NCC comparison. A perfectly flat
|
||||
image (std 0, e.g. a synthetic #000000 reference) has no carrier and yields
|
||||
an all-zero vector, which correlates to 0 with everything.
|
||||
"""
|
||||
residual = gray - float(gray.mean())
|
||||
norm = float(np.linalg.norm(residual))
|
||||
if norm == 0.0:
|
||||
return residual.ravel()
|
||||
return (residual / norm).ravel()
|
||||
|
||||
|
||||
def ncc(a: NDArray[np.float64], b: NDArray[np.float64]) -> float:
|
||||
"""Normalized cross-correlation of two carriers (unit-norm zero-mean vectors)."""
|
||||
if a.shape != b.shape or a.size == 0:
|
||||
return 0.0
|
||||
return float(np.dot(a, b))
|
||||
|
||||
|
||||
def mean_pairwise_ncc(carriers: list[NDArray[np.float64]]) -> float:
|
||||
"""Average NCC over all distinct carrier pairs; 0.0 if fewer than two."""
|
||||
scores = [ncc(carriers[i], carriers[j]) for i in range(len(carriers)) for j in range(i + 1, len(carriers))]
|
||||
return float(np.mean(scores)) if scores else 0.0
|
||||
|
||||
|
||||
def template(carriers: list[NDArray[np.float64]]) -> NDArray[np.float64]:
|
||||
"""Average carrier, renormalized to unit norm (the shared-pattern estimate)."""
|
||||
avg = np.mean(carriers, axis=0)
|
||||
norm = float(np.linalg.norm(avg))
|
||||
return avg / norm if norm else avg
|
||||
|
||||
|
||||
def random_baseline(shape: tuple[int, ...], n: int, *, seed: int = 0) -> float:
|
||||
"""Mean pairwise NCC of ``n`` random-noise carriers of ``shape`` (~0)."""
|
||||
rng = np.random.default_rng(seed)
|
||||
noise = [carrier(rng.standard_normal(shape)) for _ in range(max(n, 2))]
|
||||
return mean_pairwise_ncc(noise)
|
||||
|
||||
|
||||
def _load_carriers(paths: tuple[str, ...]) -> list[NDArray[np.float64]]:
|
||||
"""Load carriers for same-shaped images; warn and skip mismatched shapes."""
|
||||
grays = [(p, load_gray(p)) for p in paths]
|
||||
shape = grays[0][1].shape
|
||||
carriers: list[NDArray[np.float64]] = []
|
||||
for p, g in grays:
|
||||
if g.shape != shape:
|
||||
console.print(f" [yellow]skip[/] {p}: shape {g.shape} != {shape}")
|
||||
continue
|
||||
carriers.append(carrier(g))
|
||||
return carriers
|
||||
|
||||
|
||||
@click.group()
|
||||
def cli() -> None:
|
||||
"""SynthID pixel-carrier probe (solid-color fills only)."""
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True))
|
||||
def consistency(images: tuple[str, ...]) -> None:
|
||||
"""Mean pairwise carrier NCC across solid fills, vs the random baseline."""
|
||||
carriers = _load_carriers(images)
|
||||
if len(carriers) < 2:
|
||||
console.print("[red]Need at least two same-shaped images.[/]")
|
||||
raise SystemExit(1)
|
||||
observed = mean_pairwise_ncc(carriers)
|
||||
baseline = random_baseline(carriers[0].shape, len(carriers))
|
||||
console.print(f" carriers: {len(carriers)}")
|
||||
console.print(f" mean pairwise NCC: [bold]{observed:.3f}[/]")
|
||||
console.print(f" random baseline: {baseline:.3f}")
|
||||
verdict = "shared carrier present" if observed > 0.3 else "no shared carrier (within noise)"
|
||||
console.print(f" verdict: [bold]{verdict}[/]")
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.option("--pos", "pos", multiple=True, required=True, type=click.Path(exists=True), help="Positive solid fills.")
|
||||
@click.option(
|
||||
"--cleaned", "cleaned", multiple=True, required=True, type=click.Path(exists=True), help="Pipeline-cleaned fills."
|
||||
)
|
||||
def removal(pos: tuple[str, ...], cleaned: tuple[str, ...]) -> None:
|
||||
"""Does the pipeline drop the carrier correlation toward the random baseline?"""
|
||||
pos_carriers = _load_carriers(pos)
|
||||
cleaned_carriers = _load_carriers(cleaned)
|
||||
if not pos_carriers or not cleaned_carriers:
|
||||
console.print("[red]Need at least one positive and one cleaned fill of matching shape.[/]")
|
||||
raise SystemExit(1)
|
||||
tmpl = template(pos_carriers)
|
||||
pos_corr = float(np.mean([ncc(c, tmpl) for c in pos_carriers]))
|
||||
cleaned_corr = float(np.mean([ncc(c, tmpl) for c in cleaned_carriers]))
|
||||
baseline = random_baseline(tmpl.shape, max(len(cleaned_carriers), 2))
|
||||
console.print(f" positive->template NCC: [bold]{pos_corr:.3f}[/]")
|
||||
console.print(f" cleaned->template NCC: [bold]{cleaned_corr:.3f}[/]")
|
||||
console.print(f" random baseline: {baseline:.3f}")
|
||||
effective = cleaned_corr < pos_corr / 2
|
||||
console.print(f" verdict: [bold]{'carrier attenuated' if effective else 'carrier survives'}[/]")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
@@ -1,3 +1,3 @@
|
||||
"""Remove-AI-Watermarks: Unified tool for removing visible and invisible AI watermarks."""
|
||||
|
||||
__version__ = "0.3.4"
|
||||
__version__ = "0.5.5"
|
||||
@@ -8,9 +8,11 @@ Provides commands for:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import click
|
||||
from rich.console import Console
|
||||
@@ -20,6 +22,11 @@ from rich.table import Table
|
||||
|
||||
from remove_ai_watermarks import __version__
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.gemini_engine import DetectionResult
|
||||
|
||||
console = Console()
|
||||
|
||||
SUPPORTED_FORMATS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
@@ -55,6 +62,74 @@ def _validate_image(path: Path) -> Path:
|
||||
return path
|
||||
|
||||
|
||||
_ALPHA_FORMATS = {".png", ".webp"}
|
||||
|
||||
|
||||
def _watermark_region(det: DetectionResult, width: int, height: int) -> tuple[int, int, int, int]:
|
||||
"""Pick a watermark bbox: detector's region if confident, else the default config slot."""
|
||||
if det.confidence > 0.15:
|
||||
return det.region
|
||||
from remove_ai_watermarks.gemini_engine import get_watermark_config
|
||||
|
||||
config = get_watermark_config(width, height)
|
||||
px, py = config.get_position(width, height)
|
||||
return (px, py, config.logo_size, config.logo_size)
|
||||
|
||||
|
||||
def _read_bgr_and_alpha(path: Path) -> tuple[np.ndarray | None, np.ndarray | None]:
|
||||
"""Read an image preserving its alpha channel separately.
|
||||
|
||||
Returns ``(bgr, alpha)`` where ``alpha`` is a single-channel ndarray when the
|
||||
source has transparency, else ``None``. Greyscale inputs are promoted to BGR.
|
||||
Returns ``(None, None)`` if the image cannot be decoded.
|
||||
"""
|
||||
import cv2
|
||||
|
||||
image = cv2.imread(str(path), cv2.IMREAD_UNCHANGED)
|
||||
if image is None:
|
||||
return None, None
|
||||
if image.ndim == 2:
|
||||
return cv2.cvtColor(image, cv2.COLOR_GRAY2BGR), None
|
||||
if image.shape[2] == 4:
|
||||
return image[:, :, :3].copy(), image[:, :, 3].copy()
|
||||
return image, None
|
||||
|
||||
|
||||
def _write_bgr_with_alpha(
|
||||
path: Path,
|
||||
bgr: np.ndarray,
|
||||
alpha: np.ndarray | None,
|
||||
clear_region: tuple[int, int, int, int] | None = None,
|
||||
pad: int = 6,
|
||||
) -> None:
|
||||
"""Write BGR (with optional alpha) to ``path``.
|
||||
|
||||
When ``alpha`` is provided and the output extension supports it, writes a
|
||||
4-channel image. If ``clear_region`` is given as ``(x, y, w, h)``, alpha is
|
||||
forced to 0 inside that bbox (expanded by ``pad`` px) so the watermark area
|
||||
becomes fully transparent in the saved file.
|
||||
"""
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
if alpha is None or path.suffix.lower() not in _ALPHA_FORMATS:
|
||||
cv2.imwrite(str(path), bgr)
|
||||
return
|
||||
|
||||
alpha_out = alpha
|
||||
if clear_region is not None:
|
||||
alpha_out = alpha.copy()
|
||||
x, y, w, h = clear_region
|
||||
height, width = alpha.shape[:2]
|
||||
x0, y0 = max(0, x - pad), max(0, y - pad)
|
||||
x1, y1 = min(width, x + w + pad), min(height, y + h + pad)
|
||||
if x1 > x0 and y1 > y0:
|
||||
alpha_out[y0:y1, x0:x1] = 0
|
||||
|
||||
bgra = np.dstack([bgr, alpha_out])
|
||||
cv2.imwrite(str(path), bgra)
|
||||
|
||||
|
||||
# ── Main group ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -109,8 +184,6 @@ def cmd_visible(
|
||||
|
||||
Uses reverse alpha blending — fast, deterministic, offline.
|
||||
"""
|
||||
import cv2
|
||||
|
||||
from remove_ai_watermarks.gemini_engine import GeminiEngine
|
||||
|
||||
_banner()
|
||||
@@ -121,8 +194,8 @@ def cmd_visible(
|
||||
|
||||
engine = GeminiEngine()
|
||||
|
||||
# Load image
|
||||
image = cv2.imread(str(source), cv2.IMREAD_COLOR)
|
||||
# Load image (preserving any alpha channel separately)
|
||||
image, alpha = _read_bgr_and_alpha(source)
|
||||
if image is None:
|
||||
console.print(f"[red]Error:[/] Failed to read image: {source}")
|
||||
raise SystemExit(1)
|
||||
@@ -150,18 +223,12 @@ def cmd_visible(
|
||||
|
||||
# Removal
|
||||
t0 = time.monotonic()
|
||||
region: tuple[int, int, int, int] | None = None
|
||||
with console.status("[cyan]Removing watermark…[/]"):
|
||||
result = engine.remove_watermark(image)
|
||||
|
||||
if inpaint:
|
||||
if det.confidence > 0.15:
|
||||
region = det.region
|
||||
else:
|
||||
from remove_ai_watermarks.gemini_engine import get_watermark_config
|
||||
|
||||
config = get_watermark_config(w, h)
|
||||
pos = config.get_position(w, h)
|
||||
region = (pos[0], pos[1], config.logo_size, config.logo_size)
|
||||
region = _watermark_region(det, w, h)
|
||||
result = engine.inpaint_residual(
|
||||
result,
|
||||
region,
|
||||
@@ -171,9 +238,9 @@ def cmd_visible(
|
||||
|
||||
elapsed = time.monotonic() - t0
|
||||
|
||||
# Save
|
||||
# Save (preserves transparency by clearing alpha in the watermark region)
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
cv2.imwrite(str(output), result)
|
||||
_write_bgr_with_alpha(output, result, alpha, clear_region=region)
|
||||
|
||||
# Strip metadata
|
||||
if strip_metadata:
|
||||
@@ -211,6 +278,12 @@ def cmd_visible(
|
||||
@click.option(
|
||||
"--humanize", type=float, default=0.0, help="Analog Humanizer film grain intensity (0 = off, typical: 2.0-6.0)."
|
||||
)
|
||||
@click.option(
|
||||
"--max-resolution",
|
||||
type=int,
|
||||
default=0,
|
||||
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
|
||||
)
|
||||
@click.pass_context
|
||||
def cmd_invisible(
|
||||
ctx: click.Context,
|
||||
@@ -223,6 +296,7 @@ def cmd_invisible(
|
||||
seed: int | None,
|
||||
hf_token: str | None,
|
||||
humanize: float,
|
||||
max_resolution: int,
|
||||
) -> None:
|
||||
"""Remove invisible AI watermarks (SynthID, StableSignature, TreeRing).
|
||||
|
||||
@@ -269,6 +343,7 @@ def cmd_invisible(
|
||||
guidance_scale=None,
|
||||
seed=seed,
|
||||
humanize=humanize,
|
||||
max_resolution=max_resolution,
|
||||
)
|
||||
elapsed = time.monotonic() - t0
|
||||
|
||||
@@ -310,6 +385,8 @@ def cmd_metadata(
|
||||
if has_ai:
|
||||
console.print(f" [yellow]⚠[/] AI metadata detected in {source.name}:")
|
||||
meta = get_ai_metadata(source)
|
||||
if synthid := meta.get("synthid_watermark"):
|
||||
console.print(f" [bold yellow]⚠ SynthID pixel watermark {synthid}[/]")
|
||||
table = Table(show_header=True, header_style="bold")
|
||||
table.add_column("Key", style="cyan")
|
||||
table.add_column("Value")
|
||||
@@ -327,6 +404,59 @@ def cmd_metadata(
|
||||
console.print(f" [green]✓[/] AI metadata stripped → {out}")
|
||||
|
||||
|
||||
# ── Provenance identification ───────────────────────────────────────
|
||||
|
||||
|
||||
@main.command("identify")
|
||||
@click.argument("source", type=click.Path(exists=True, path_type=Path))
|
||||
@click.option(
|
||||
"--no-visible",
|
||||
is_flag=True,
|
||||
help="Skip pixel-domain detectors (visible sparkle + invisible watermark); metadata-only.",
|
||||
)
|
||||
@click.option("--json", "as_json", is_flag=True, help="Emit the report as JSON instead of a table.")
|
||||
@click.pass_context
|
||||
def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bool) -> None:
|
||||
"""Identify where an image was made and what watermarks it carries.
|
||||
|
||||
Aggregates C2PA Content Credentials, IPTC "Made with AI" tags, embedded
|
||||
generation parameters, the SynthID metadata proxy, and the visible Gemini
|
||||
sparkle into a single provenance verdict. Absence of signals is reported as
|
||||
"unknown", never as "clean" (stripped metadata leaves no local proof).
|
||||
"""
|
||||
from dataclasses import asdict
|
||||
|
||||
from remove_ai_watermarks.identify import identify
|
||||
|
||||
source = _validate_image(source)
|
||||
report = identify(source, check_visible=not no_visible, check_invisible=not no_visible)
|
||||
|
||||
if as_json:
|
||||
click.echo(json.dumps(asdict(report), default=str, indent=2))
|
||||
return
|
||||
|
||||
_banner()
|
||||
verdict = {True: "[yellow]AI-generated[/]", False: "[green]not AI[/]", None: "[dim]unknown[/]"}[
|
||||
report.is_ai_generated
|
||||
]
|
||||
console.print(f"\n Verdict: {verdict} [dim](confidence: {report.confidence})[/]")
|
||||
console.print(f" Platform: {report.platform or '[dim]undetermined[/]'}")
|
||||
|
||||
if report.watermarks:
|
||||
table = Table(show_header=True, header_style="bold", title="Watermarks / provenance markers")
|
||||
table.add_column("Marker", style="cyan")
|
||||
for wm in report.watermarks:
|
||||
table.add_row(wm)
|
||||
console.print(table)
|
||||
else:
|
||||
console.print(" [dim]No watermarks or provenance markers found.[/]")
|
||||
|
||||
if report.caveats:
|
||||
console.print("\n [dim]Caveats:[/]")
|
||||
for c in report.caveats:
|
||||
console.print(f" [dim]- {c}[/]")
|
||||
|
||||
|
||||
# ── Combined "all" mode ──────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -354,6 +484,12 @@ def cmd_metadata(
|
||||
@click.option(
|
||||
"--humanize", type=float, default=0.0, help="Analog Humanizer film grain intensity (0 = off, typical: 2.0-6.0)."
|
||||
)
|
||||
@click.option(
|
||||
"--max-resolution",
|
||||
type=int,
|
||||
default=0,
|
||||
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
|
||||
)
|
||||
@click.pass_context
|
||||
def cmd_all(
|
||||
ctx: click.Context,
|
||||
@@ -369,6 +505,7 @@ def cmd_all(
|
||||
seed: int | None,
|
||||
hf_token: str | None,
|
||||
humanize: float,
|
||||
max_resolution: int,
|
||||
) -> None:
|
||||
"""Remove ALL watermarks: visible + invisible + metadata.
|
||||
|
||||
@@ -379,9 +516,7 @@ def cmd_all(
|
||||
|
||||
If invisible watermark deps are not installed, skips step 2 with a warning.
|
||||
"""
|
||||
import cv2
|
||||
|
||||
from remove_ai_watermarks.gemini_engine import GeminiEngine, get_watermark_config
|
||||
from remove_ai_watermarks.gemini_engine import GeminiEngine
|
||||
|
||||
_banner()
|
||||
source = _validate_image(source)
|
||||
@@ -405,7 +540,7 @@ def cmd_all(
|
||||
# ── Step 1: Visible watermark ────────────────────────────────
|
||||
console.print("\n [bold cyan]① Visible watermark removal[/]")
|
||||
engine = GeminiEngine()
|
||||
image = cv2.imread(str(source), cv2.IMREAD_COLOR)
|
||||
image, alpha = _read_bgr_and_alpha(source)
|
||||
if image is None:
|
||||
console.print(f"[red]Error:[/] Failed to read image: {source}")
|
||||
raise SystemExit(1)
|
||||
@@ -413,25 +548,21 @@ def cmd_all(
|
||||
h, w = image.shape[:2]
|
||||
console.print(f" [dim]Input:[/] {source.name} ({w}x{h})")
|
||||
|
||||
region: tuple[int, int, int, int] | None = None
|
||||
with console.status("[cyan]Removing visible watermark…[/]"):
|
||||
det = engine.detect_watermark(image)
|
||||
if det.detected:
|
||||
result = engine.remove_watermark(image)
|
||||
if inpaint:
|
||||
if det.confidence > 0.15:
|
||||
region = det.region
|
||||
else:
|
||||
config = get_watermark_config(w, h)
|
||||
pos = config.get_position(w, h)
|
||||
region = (pos[0], pos[1], config.logo_size, config.logo_size)
|
||||
region = _watermark_region(det, w, h)
|
||||
result = engine.inpaint_residual(result, region, method=inpaint_method)
|
||||
console.print(" [green]✓[/] Visible watermark removed")
|
||||
else:
|
||||
result = image.copy()
|
||||
console.print(" [dim]Skipped (no visible watermark detected)[/]")
|
||||
|
||||
# Save to temp file for invisible engine input
|
||||
cv2.imwrite(str(tmp_path), result)
|
||||
# Save to temp file for invisible engine input (preserve alpha if present)
|
||||
_write_bgr_with_alpha(tmp_path, result, alpha, clear_region=region)
|
||||
|
||||
# ── Step 2: Invisible watermark ──────────────────────────────
|
||||
console.print("\n [bold cyan]② Invisible watermark removal[/]")
|
||||
@@ -466,6 +597,7 @@ def cmd_all(
|
||||
num_inference_steps=steps,
|
||||
seed=seed,
|
||||
humanize=humanize,
|
||||
max_resolution=max_resolution,
|
||||
)
|
||||
console.print(" [green]✓[/] Invisible watermark removed")
|
||||
|
||||
@@ -480,10 +612,15 @@ def cmd_all(
|
||||
console.print(f" [yellow]⚠[/] Metadata strip failed: {e}")
|
||||
|
||||
# ── Write final result ────────────────────────────────────────
|
||||
import shutil
|
||||
|
||||
# The invisible step (and downstream cv2.IMREAD_COLOR paths) drops alpha,
|
||||
# so re-attach the original alpha (with the watermark region cleared)
|
||||
# when writing the final output for transparent formats.
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.move(str(tmp_path), str(output))
|
||||
final_bgr, _ = _read_bgr_and_alpha(tmp_path)
|
||||
if final_bgr is None:
|
||||
console.print(f"[red]Error:[/] Failed to read intermediate file: {tmp_path}")
|
||||
raise SystemExit(1)
|
||||
_write_bgr_with_alpha(output, final_bgr, alpha, clear_region=region)
|
||||
|
||||
finally:
|
||||
# Clean up temp file if it still exists
|
||||
@@ -512,6 +649,7 @@ def _process_batch_image(
|
||||
seed: int | None,
|
||||
hf_token: str | None,
|
||||
humanize: float,
|
||||
max_resolution: int = 0,
|
||||
) -> None:
|
||||
"""Process a single image for batch mode.
|
||||
|
||||
@@ -521,13 +659,11 @@ def _process_batch_image(
|
||||
Raises:
|
||||
ValueError: If the image cannot be opened.
|
||||
"""
|
||||
if mode in ("visible", "all"):
|
||||
import cv2
|
||||
saved_alpha: np.ndarray | None = None
|
||||
saved_region: tuple[int, int, int, int] | None = None
|
||||
|
||||
from remove_ai_watermarks.gemini_engine import (
|
||||
GeminiEngine,
|
||||
get_watermark_config,
|
||||
)
|
||||
if mode in ("visible", "all"):
|
||||
from remove_ai_watermarks.gemini_engine import GeminiEngine
|
||||
|
||||
if "_vis_engine" not in ctx.obj:
|
||||
ctx.obj["_vis_engine"] = GeminiEngine()
|
||||
@@ -535,27 +671,24 @@ def _process_batch_image(
|
||||
read_path = img_path
|
||||
if mode == "all" and out_path.exists():
|
||||
read_path = out_path
|
||||
image = cv2.imread(str(read_path), cv2.IMREAD_COLOR)
|
||||
image, alpha = _read_bgr_and_alpha(read_path)
|
||||
if image is None:
|
||||
raise ValueError("Failed to read image")
|
||||
|
||||
region: tuple[int, int, int, int] | None = None
|
||||
det = engine.detect_watermark(image)
|
||||
if det.detected:
|
||||
result = engine.remove_watermark(image)
|
||||
if inpaint:
|
||||
if det.confidence > 0.15:
|
||||
region = det.region
|
||||
else:
|
||||
h, w = image.shape[:2]
|
||||
config = get_watermark_config(w, h)
|
||||
pos = config.get_position(w, h)
|
||||
region = (pos[0], pos[1], config.logo_size, config.logo_size)
|
||||
|
||||
h, w = image.shape[:2]
|
||||
region = _watermark_region(det, w, h)
|
||||
result = engine.inpaint_residual(result, region)
|
||||
else:
|
||||
result = image.copy()
|
||||
|
||||
cv2.imwrite(str(out_path), result)
|
||||
_write_bgr_with_alpha(out_path, result, alpha, clear_region=region)
|
||||
saved_alpha = alpha
|
||||
saved_region = region
|
||||
|
||||
if mode in ("invisible", "all"):
|
||||
from remove_ai_watermarks.invisible_engine import (
|
||||
@@ -579,6 +712,7 @@ def _process_batch_image(
|
||||
num_inference_steps=steps,
|
||||
seed=seed,
|
||||
humanize=humanize,
|
||||
max_resolution=max_resolution,
|
||||
)
|
||||
|
||||
if mode in ("metadata", "all"):
|
||||
@@ -586,6 +720,13 @@ def _process_batch_image(
|
||||
|
||||
remove_ai_metadata(img_path if mode == "metadata" else out_path, out_path)
|
||||
|
||||
# In "all" mode, the invisible step (color-only OpenCV paths) drops alpha,
|
||||
# so re-attach the cached alpha when the input had transparency.
|
||||
if mode == "all" and saved_alpha is not None:
|
||||
final_bgr, _ = _read_bgr_and_alpha(out_path)
|
||||
if final_bgr is not None:
|
||||
_write_bgr_with_alpha(out_path, final_bgr, saved_alpha, clear_region=saved_region)
|
||||
|
||||
|
||||
@main.command("batch")
|
||||
@click.argument("directory", type=click.Path(exists=True, file_okay=False, path_type=Path))
|
||||
@@ -614,6 +755,12 @@ def _process_batch_image(
|
||||
@click.option("--device", type=click.Choice(["auto", "cpu", "mps", "cuda"]), default="auto", help="Inference device.")
|
||||
@click.option("--seed", type=int, default=None, help="Random seed for reproducibility.")
|
||||
@click.option("--hf-token", type=str, default=None, help="HuggingFace API token.")
|
||||
@click.option(
|
||||
"--max-resolution",
|
||||
type=int,
|
||||
default=0,
|
||||
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
|
||||
)
|
||||
@click.pass_context
|
||||
def cmd_batch(
|
||||
ctx: click.Context,
|
||||
@@ -628,6 +775,7 @@ def cmd_batch(
|
||||
hf_token: str | None,
|
||||
inpaint: bool,
|
||||
humanize: float,
|
||||
max_resolution: int,
|
||||
) -> None:
|
||||
"""Process all images in a directory."""
|
||||
_banner()
|
||||
@@ -677,7 +825,8 @@ def cmd_batch(
|
||||
seed=seed,
|
||||
hf_token=hf_token,
|
||||
humanize=humanize,
|
||||
)
|
||||
max_resolution=max_resolution,
|
||||
)
|
||||
processed += 1
|
||||
|
||||
except Exception as e:
|
||||
|
||||
@@ -546,3 +546,17 @@ class GeminiEngine:
|
||||
result[py1:py2, px1:px2] = blended.astype(np.uint8)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def detect_sparkle_confidence(image_path: Path) -> float | None:
|
||||
"""Visible-sparkle detection confidence for a file, for provenance use.
|
||||
|
||||
Loads the image with cv2 and runs :meth:`GeminiEngine.detect_watermark`.
|
||||
Returns the NCC confidence in [0, 1], or None if the image cannot be read
|
||||
(cv2 returns None for unsupported containers such as HEIC). Kept here so the
|
||||
cv2 dependency stays in this module; callers apply their own threshold.
|
||||
"""
|
||||
img = cv2.imread(str(image_path))
|
||||
if img is None:
|
||||
return None
|
||||
return float(GeminiEngine().detect_watermark(img).confidence)
|
||||
@@ -0,0 +1,308 @@
|
||||
"""Image provenance: identify where an image was made and what watermarks it carries.
|
||||
|
||||
Aggregates every locally-readable signal into a single :class:`ProvenanceReport`:
|
||||
|
||||
- **C2PA Content Credentials** (issuer, claim generator, digital source type) ->
|
||||
the signing platform (OpenAI, Google, Adobe, Microsoft).
|
||||
- **IPTC ``digitalSourceType``** "Made with AI" marker (Meta, X, others).
|
||||
- **PNG text / EXIF generation parameters** (Stable Diffusion, ComfyUI, InvokeAI).
|
||||
- **SynthID metadata proxy** -- a C2PA companion from a SynthID-using vendor
|
||||
(Google / OpenAI) implies the invisible pixel watermark.
|
||||
- **Visible Gemini sparkle** (optional; needs cv2/numpy, no GPU).
|
||||
|
||||
Hard limit: a stripped image (re-encoded, screenshotted, social-media upload)
|
||||
loses all metadata, and the SynthID *pixel* watermark is not locally decodable
|
||||
(proprietary decoder). Absence of signals is therefore reported as ``Unknown``,
|
||||
never as "clean". See CLAUDE.md "SynthID detection is metadata-only".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from remove_ai_watermarks.metadata import (
|
||||
AI_METADATA_KEYS,
|
||||
AIGC_MARKERS,
|
||||
C2PA_UUID,
|
||||
IPTC_AI_MARKERS,
|
||||
aigc_label,
|
||||
exif_generator,
|
||||
get_ai_metadata,
|
||||
xai_signature,
|
||||
)
|
||||
from remove_ai_watermarks.noai.c2pa import extract_c2pa_info
|
||||
from remove_ai_watermarks.noai.constants import C2PA_AI_TOOLS, C2PA_ISSUERS
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# How much of a non-PNG container to binary-scan for the C2PA issuer.
|
||||
_SCAN_BYTES = 1024 * 1024
|
||||
|
||||
# Visible-sparkle confidence above which the signal is trusted as provenance.
|
||||
# Stricter than the removal default (0.25): on the corpus, Gemini-family
|
||||
# sparkles score >= 0.56 while non-sparkle images top out at 0.49, so 0.5
|
||||
# cleanly separates them and avoids false positives when sparkle is the only
|
||||
# signal (e.g. an OpenAI image scored 0.37 -- below threshold, correctly dropped).
|
||||
_SPARKLE_THRESHOLD = 0.5
|
||||
|
||||
# Issuer (C2PA signer) -> human-readable generating platform. Ordered: when a
|
||||
# manifest names several issuers (Microsoft Designer signs as "OpenAI,
|
||||
# Microsoft"), the first match wins so the product, not the backend, is named.
|
||||
_ISSUER_PLATFORM: tuple[tuple[str, str], ...] = (
|
||||
# Microsoft signs both Designer and Bing Image Creator; Bing now runs its
|
||||
# own MAI-Image model (not DALL-E), so the label stays model-neutral.
|
||||
("Microsoft", "Microsoft (Bing Image Creator / Designer)"),
|
||||
("Adobe", "Adobe Firefly"),
|
||||
("OpenAI", "OpenAI (ChatGPT / gpt-image / DALL-E / Sora)"),
|
||||
("Google", "Google (Gemini / Imagen)"),
|
||||
("Stability AI", "Stability AI (Stable Image / DreamStudio)"),
|
||||
)
|
||||
|
||||
# PNG-text / EXIF keys that indicate a local diffusion pipeline (vs. a hosted
|
||||
# platform's C2PA). Subset of AI_METADATA_KEYS; excludes the C2PA/Software keys.
|
||||
_LOCAL_GEN_KEYS = frozenset(
|
||||
AI_METADATA_KEYS & {"parameters", "prompt", "negative_prompt", "workflow", "comfyui", "invokeai_metadata", "dream"}
|
||||
)
|
||||
|
||||
_STRIP_CAVEAT = (
|
||||
"Absence of metadata is not proof the image is clean: C2PA, EXIF, and PNG "
|
||||
"text chunks are stripped by re-encoding, screenshots, or social-media upload."
|
||||
)
|
||||
_SYNTHID_CAVEAT = (
|
||||
"SynthID is a metadata proxy here; the pixel watermark is not locally "
|
||||
"verifiable (proprietary decoder). Confirm via the Gemini app or openai.com/verify."
|
||||
)
|
||||
_OPENAI_CAVEAT = (
|
||||
"OpenAI began pairing SynthID with C2PA around 2026-05; OpenAI images from "
|
||||
"before the rollout carry C2PA without SynthID, so the SynthID verdict is 'likely'."
|
||||
)
|
||||
_IPTC_ONLY_CAVEAT = "The IPTC 'Made with AI' tag flags AI provenance but does not identify the specific platform."
|
||||
_INVISIBLE_WM_CAVEAT = (
|
||||
"The open invisible watermark is fragile: it does not survive JPEG re-encoding "
|
||||
"or resizing, so it confirms origin only on a pristine (un-re-encoded) file."
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Signal:
|
||||
"""A single provenance signal that was found (or affirmatively absent)."""
|
||||
|
||||
name: str
|
||||
detail: str
|
||||
confidence: str # "high" | "medium"
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProvenanceReport:
|
||||
"""Aggregated provenance verdict for one image."""
|
||||
|
||||
path: Path
|
||||
is_ai_generated: bool | None # True / False is never asserted; None = unknown
|
||||
platform: str | None
|
||||
confidence: str # "high" | "medium" | "none"
|
||||
watermarks: list[str] = field(default_factory=list[str])
|
||||
signals: list[Signal] = field(default_factory=list["Signal"])
|
||||
caveats: list[str] = field(default_factory=list[str])
|
||||
|
||||
|
||||
def _issuers_in(data: bytes) -> list[str]:
|
||||
"""C2PA issuer names whose signature byte appears in ``data`` (binary scan)."""
|
||||
return sorted({name for sig, name in C2PA_ISSUERS.items() if sig in data})
|
||||
|
||||
|
||||
def _ai_tools_in(data: bytes) -> list[str]:
|
||||
"""Known C2PA AI-tool / generator names appearing in ``data`` (binary scan).
|
||||
|
||||
PNG has a structured claim_generator; for JPEG/WebP/AVIF/HEIF/JXL the
|
||||
generator lives in a JUMBF/EXIF/XMP blob the PNG parser can't reach, so a
|
||||
byte scan recovers the same attribution (e.g. "Imagen", "DALL-E").
|
||||
"""
|
||||
return sorted({name for sig, name in C2PA_AI_TOOLS.items() if sig in data})
|
||||
|
||||
|
||||
def _attribute_platform(issuers: list[str]) -> str | None:
|
||||
"""Map a set of C2PA issuer names to a human-readable generating platform."""
|
||||
joined = " ".join(issuers)
|
||||
for needle, platform in _ISSUER_PLATFORM:
|
||||
if needle in joined:
|
||||
return platform
|
||||
if issuers: # e.g. Truepic alone -- a signing authority, not a generator
|
||||
return f"C2PA signer: {', '.join(issuers)} (no known AI generator named)"
|
||||
return None
|
||||
|
||||
|
||||
def _visible_sparkle(image_path: Path) -> float | None:
|
||||
"""Visible Gemini-sparkle confidence in [0, 1], or None if unavailable.
|
||||
|
||||
Optional: needs cv2/numpy (no GPU). The cv2 work lives in gemini_engine so
|
||||
this module stays dependency-light; returns None if cv2 or the engine
|
||||
assets are missing, or the image can't be read.
|
||||
"""
|
||||
try:
|
||||
from remove_ai_watermarks.gemini_engine import detect_sparkle_confidence
|
||||
except Exception as exc: # cv2/engine assets missing
|
||||
log.debug("visible-sparkle detector unavailable: %s", exc)
|
||||
return None
|
||||
return detect_sparkle_confidence(image_path)
|
||||
|
||||
|
||||
def _invisible_watermark(image_path: Path) -> str | None:
|
||||
"""Open invisible-watermark scheme name (SD/SDXL/FLUX) or None.
|
||||
|
||||
Optional: needs the imwatermark decoder (extra ``detect``). Returns None if
|
||||
it is not installed or no known watermark decodes.
|
||||
"""
|
||||
from remove_ai_watermarks.invisible_watermark import detect_invisible_watermark
|
||||
|
||||
return detect_invisible_watermark(image_path)
|
||||
|
||||
|
||||
def identify(image_path: Path, *, check_visible: bool = True, check_invisible: bool = True) -> ProvenanceReport:
|
||||
"""Identify an image's origin platform and watermark inventory.
|
||||
|
||||
Args:
|
||||
image_path: Path to the image (PNG, JPEG, WebP, or ISOBMFF container).
|
||||
check_visible: Also run the visible Gemini-sparkle detector (cv2). Set
|
||||
False for a pure-metadata, dependency-light scan.
|
||||
check_invisible: Also decode open invisible watermarks (SD/SDXL/FLUX) via
|
||||
the optional imwatermark library. No-op when it is not installed.
|
||||
|
||||
Returns:
|
||||
A :class:`ProvenanceReport`. ``is_ai_generated`` is True when any AI
|
||||
signal is found and None (unknown) when none is -- it is never asserted
|
||||
False, because stripped metadata leaves no local proof of a clean origin.
|
||||
"""
|
||||
info = extract_c2pa_info(image_path) # PNG-structured; {} for other formats
|
||||
meta = get_ai_metadata(image_path) # PNG text + EXIF + C2PA fields + synthid
|
||||
|
||||
# First MB covers C2PA (PNG caBX, JPEG APP11, AVIF/HEIF/JXL uuid box) and
|
||||
# IPTC markers for the non-PNG path where extract_c2pa_info returns {}.
|
||||
with open(image_path, "rb") as f:
|
||||
head = f.read(_SCAN_BYTES)
|
||||
|
||||
signals: list[Signal] = []
|
||||
watermarks: list[str] = []
|
||||
caveats: list[str] = []
|
||||
|
||||
# ── C2PA Content Credentials ────────────────────────────────────
|
||||
has_c2pa = bool(info) or b"c2pa" in head.lower() or C2PA_UUID in head
|
||||
issuers = [info["issuer"]] if info.get("issuer") else _issuers_in(head)
|
||||
platform = _attribute_platform(issuers) if has_c2pa else None
|
||||
c2pa_is_ai = "trainedAlgorithmicMedia" in info.get("source_type", "") or any(
|
||||
m in head for m in (b"trainedAlgorithmicMedia", b"compositeWithTrainedAlgorithmicMedia")
|
||||
)
|
||||
# Generator: structured for PNG, binary-scanned for other containers.
|
||||
generator = info.get("claim_generator") or (", ".join(tools) if (tools := _ai_tools_in(head)) else None)
|
||||
if has_c2pa:
|
||||
detail = ", ".join(filter(None, [", ".join(issuers), generator, info.get("source_type")]))
|
||||
signals.append(Signal("c2pa", detail or "C2PA manifest present", "high"))
|
||||
watermarks.append(f"C2PA Content Credentials ({', '.join(issuers) or 'unknown signer'})")
|
||||
|
||||
# ── SynthID metadata proxy ──────────────────────────────────────
|
||||
# get_ai_metadata already sets synthid_watermark for both PNG (caBX parser)
|
||||
# and non-PNG (its own synthid_source fallback), so no extra scan is needed.
|
||||
synthid = meta.get("synthid_watermark")
|
||||
if synthid:
|
||||
watermarks.append(f"SynthID pixel watermark ({synthid})")
|
||||
caveats.append(_SYNTHID_CAVEAT)
|
||||
if "OpenAI" in (" ".join(issuers) + synthid):
|
||||
caveats.append(_OPENAI_CAVEAT)
|
||||
|
||||
# ── IPTC "Made with AI" (Meta etc.), only meaningful without C2PA ─
|
||||
iptc = any(m in head for m in IPTC_AI_MARKERS)
|
||||
if iptc and not has_c2pa:
|
||||
signals.append(Signal("iptc", "digitalSourceType (Made with AI)", "high"))
|
||||
watermarks.append("IPTC digitalSourceType (Made with AI)")
|
||||
caveats.append(_IPTC_ONLY_CAVEAT)
|
||||
if platform is None:
|
||||
platform = "Made-with-AI tag (e.g. Meta AI); platform not specified"
|
||||
|
||||
# ── China TC260 AIGC label (Doubao and other China-served gens) ──
|
||||
aigc = any(m in head for m in AIGC_MARKERS)
|
||||
if aigc:
|
||||
producer = (aigc_label(image_path) or {}).get("ContentProducer", "")
|
||||
signals.append(Signal("aigc", f"TC260 AIGC label{f' (producer {producer})' if producer else ''}", "high"))
|
||||
watermarks.append("China AIGC label (TC260 standard)")
|
||||
if platform is None:
|
||||
platform = "China AIGC-labeled generator (TC260; e.g. Doubao)"
|
||||
|
||||
# ── Local diffusion parameters (Stable Diffusion / ComfyUI) ──────
|
||||
local_keys = sorted(k for k in meta if k.lower() in _LOCAL_GEN_KEYS)
|
||||
if local_keys:
|
||||
signals.append(Signal("gen_params", f"embedded keys: {', '.join(local_keys)}", "high"))
|
||||
watermarks.append("Embedded generation parameters (Stable Diffusion / ComfyUI)")
|
||||
if platform is None:
|
||||
platform = "Stable Diffusion / local pipeline (Automatic1111, ComfyUI, InvokeAI)"
|
||||
|
||||
# ── EXIF Software / XMP CreatorTool generator (cross-format) ─────
|
||||
# Catches a generator tag (incl. inside AVIF/HEIF/JXL) when there is no C2PA.
|
||||
if generator_tag := exif_generator(image_path):
|
||||
signals.append(Signal("exif_generator", f"EXIF/XMP generator: {generator_tag}", "high"))
|
||||
watermarks.append(f"Embedded generator tag: {generator_tag}")
|
||||
if platform is None:
|
||||
platform = f"{generator_tag} (EXIF/XMP generator tag)"
|
||||
|
||||
# ── xAI / Grok EXIF signature scheme (no C2PA/SynthID/IPTC) ──────
|
||||
# Grok's only provenance signal: EXIF ImageDescription "Signature: <base64>"
|
||||
# + a UUID Artist. Distinct from exif_generator (which matches generator
|
||||
# tokens); verified stable across 3 generations. See CLAUDE.md.
|
||||
if xai_signature(image_path):
|
||||
signals.append(Signal("xai_signature", "EXIF Signature blob + UUID Artist", "high"))
|
||||
watermarks.append("xAI/Grok EXIF signature")
|
||||
if platform is None:
|
||||
platform = "xAI (Grok / Aurora)"
|
||||
|
||||
# ── Open invisible watermark (SD / SDXL / FLUX, dwtDct) ──────────
|
||||
# Public decoder, no key -- a definitive embedded signal on pristine files.
|
||||
if check_invisible and (scheme := _invisible_watermark(image_path)) is not None:
|
||||
signals.append(Signal("invisible_watermark", scheme, "high"))
|
||||
watermarks.append(f"Open invisible watermark: {scheme}")
|
||||
caveats.append(_INVISIBLE_WM_CAVEAT)
|
||||
if platform is None:
|
||||
platform = f"{scheme} (open DWT-DCT watermark)"
|
||||
|
||||
# ── Verdict so far (metadata + embedded watermark) ──────────────
|
||||
invisible_wm = any(s.name == "invisible_watermark" for s in signals)
|
||||
exif_gen = any(s.name == "exif_generator" for s in signals)
|
||||
xai_sig = any(s.name == "xai_signature" for s in signals)
|
||||
ai_from_metadata = bool(
|
||||
(has_c2pa and (c2pa_is_ai or synthid)) or iptc or aigc or local_keys or invisible_wm or exif_gen or xai_sig
|
||||
)
|
||||
|
||||
# ── Visible Gemini sparkle (fallback for stripped-metadata case) ─
|
||||
if check_visible and (conf := _visible_sparkle(image_path)) is not None and conf >= _SPARKLE_THRESHOLD:
|
||||
signals.append(Signal("visible_sparkle", f"NCC confidence {conf:.2f}", "medium"))
|
||||
watermarks.append(f"Visible Gemini sparkle (confidence {conf:.2f})")
|
||||
if platform is None:
|
||||
platform = "Google Gemini family (visible sparkle detected)"
|
||||
|
||||
visible_only = any(s.name == "visible_sparkle" for s in signals) and not ai_from_metadata
|
||||
|
||||
if ai_from_metadata:
|
||||
is_ai: bool | None = True
|
||||
confidence = "high"
|
||||
elif visible_only:
|
||||
is_ai = True
|
||||
confidence = "medium"
|
||||
else:
|
||||
is_ai = None
|
||||
confidence = "none"
|
||||
|
||||
caveats.append(_STRIP_CAVEAT)
|
||||
# De-duplicate while preserving order.
|
||||
caveats = list(dict.fromkeys(caveats))
|
||||
|
||||
return ProvenanceReport(
|
||||
path=image_path,
|
||||
is_ai_generated=is_ai,
|
||||
platform=platform,
|
||||
confidence=confidence,
|
||||
watermarks=watermarks,
|
||||
signals=signals,
|
||||
caveats=caveats,
|
||||
)
|
||||
@@ -42,6 +42,24 @@ def is_available() -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _target_size(width: int, height: int, max_resolution: int) -> tuple[int, int] | None:
|
||||
"""Compute the downscaled (width, height) for a long-side cap, or None for native.
|
||||
|
||||
Returns None when no pre-downscale is needed: ``max_resolution <= 0`` (native
|
||||
resolution, the default that matches the raiw.cc backend -- see issue #10) or
|
||||
the long side already fits the cap. Otherwise scales the long side down to
|
||||
``max_resolution`` preserving aspect ratio (integer-truncated, matching the
|
||||
PIL ``resize`` call site). Pure function so the native-vs-downscale decision
|
||||
is unit-testable without loading the diffusion model.
|
||||
"""
|
||||
if max_resolution > 0 and max(width, height) > max_resolution:
|
||||
ratio = max_resolution / max(width, height)
|
||||
# Clamp the short side to >=1: extreme aspect ratios (e.g. 5000x3 capped
|
||||
# at 1024) would otherwise truncate it to 0 and crash image.resize().
|
||||
return (max(1, int(width * ratio)), max(1, int(height * ratio)))
|
||||
return None
|
||||
|
||||
|
||||
class InvisibleEngine:
|
||||
"""Remove invisible AI watermarks using diffusion model regeneration.
|
||||
|
||||
@@ -107,6 +125,7 @@ class InvisibleEngine:
|
||||
seed: int | None = None,
|
||||
humanize: float = 0.0,
|
||||
protect_faces: bool = True,
|
||||
max_resolution: int = 0,
|
||||
) -> Path:
|
||||
"""Remove invisible watermark from an image.
|
||||
|
||||
@@ -119,6 +138,11 @@ class InvisibleEngine:
|
||||
seed: Random seed for reproducibility.
|
||||
humanize: Intensity of Analog Humanizer film grain (0 = off).
|
||||
protect_faces: Boolean to extract and restore faces intact.
|
||||
max_resolution: Cap the long side (px) before diffusion. 0 (default)
|
||||
= native resolution, no pre-downscale -- matches the hosted
|
||||
raiw.cc backend. Set a positive value only to bound GPU/MPS
|
||||
memory on very large inputs (it reintroduces a lossy
|
||||
downscale->upscale round-trip).
|
||||
|
||||
Returns:
|
||||
Path to the cleaned image.
|
||||
@@ -127,43 +151,35 @@ class InvisibleEngine:
|
||||
|
||||
from PIL import Image, ImageOps
|
||||
|
||||
# SDXL is trained at 1024px and degrades both quality and watermark-removal
|
||||
# efficacy below that.
|
||||
max_dimension = 1024
|
||||
# Process at native resolution by default (max_resolution=0). The hosted
|
||||
# raiw.cc backend (fal fast-sdxl) does NO pre-downscale either, and at
|
||||
# strength ~0.05 SDXL img2img does not need the input shrunk to ~1024 --
|
||||
# the old forced downscale->upscale round-trip was the main quality loss
|
||||
# (see issue #10). A positive max_resolution caps the long side only to
|
||||
# bound GPU/MPS memory on very large inputs.
|
||||
image = Image.open(image_path)
|
||||
image = ImageOps.exif_transpose(image)
|
||||
orig_size = image.size # (width, height)
|
||||
_tmp_path = None
|
||||
|
||||
if max(image.width, image.height) > max_dimension:
|
||||
ratio = max_dimension / max(image.width, image.height)
|
||||
new_size = (int(image.width * ratio), int(image.height * ratio))
|
||||
# Optional long-side downscale; native resolution by default (issue #10).
|
||||
target = _target_size(image.width, image.height, max_resolution)
|
||||
if target is not None:
|
||||
if self._progress_callback:
|
||||
self._progress_callback(
|
||||
f"Downscaling {image.width}x{image.height} "
|
||||
f"to {new_size[0]}x{new_size[1]} "
|
||||
f"(model trained at {max_dimension}px)..."
|
||||
f"to {target[0]}x{target[1]} "
|
||||
f"(max-resolution cap {max_resolution}px)..."
|
||||
)
|
||||
image = image.resize(new_size, Image.Resampling.LANCZOS)
|
||||
image = image.resize(target, Image.Resampling.LANCZOS)
|
||||
|
||||
# Save to a temp file instead of overwriting the original
|
||||
_tmp_fd, _tmp_str = tempfile.mkstemp(suffix=image_path.suffix)
|
||||
_tmp_path = Path(_tmp_str)
|
||||
image.save(_tmp_path)
|
||||
import os as _os
|
||||
|
||||
_os.close(_tmp_fd)
|
||||
image_path = _tmp_path
|
||||
else:
|
||||
# We must save the transposed image back to a tmp file if it was rotated
|
||||
# otherwise WatermarkRemover will reload it without EXIF rotation!
|
||||
_tmp_fd, _tmp_str = tempfile.mkstemp(suffix=image_path.suffix)
|
||||
_tmp_path = Path(_tmp_str)
|
||||
image.save(_tmp_path)
|
||||
import os as _os
|
||||
|
||||
_os.close(_tmp_fd)
|
||||
image_path = _tmp_path
|
||||
# Always persist to a temp file, even without downscaling: WatermarkRemover
|
||||
# reloads by path, so the EXIF-transposed pixels must be saved or rotation
|
||||
# is lost. Cleaned up in the finally block via _tmp_path.
|
||||
_tmp_fd, _tmp_str = tempfile.mkstemp(suffix=image_path.suffix)
|
||||
_tmp_path = Path(_tmp_str)
|
||||
image.save(_tmp_path)
|
||||
os.close(_tmp_fd)
|
||||
image_path = _tmp_path
|
||||
|
||||
try:
|
||||
# Optional: Face protection (Phase 1 - Extraction)
|
||||
@@ -244,7 +260,8 @@ class InvisibleEngine:
|
||||
|
||||
return out_path
|
||||
finally:
|
||||
if _tmp_path is not None and _tmp_path.exists():
|
||||
# _tmp_path is always set above (we persist the image unconditionally).
|
||||
if _tmp_path.exists():
|
||||
_tmp_path.unlink()
|
||||
|
||||
def remove_watermark_batch(
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
"""Detect open invisible watermarks embedded by the ``invisible-watermark``
|
||||
(imwatermark) library -- used by Stable Diffusion, SDXL, and FLUX.
|
||||
|
||||
Unlike SynthID (proprietary, no local decoder), these are DWT-DCT watermarks
|
||||
with a PUBLIC decoder and no secret key, so a fresh, un-re-encoded output can be
|
||||
identified locally. The known fixed patterns were verified against upstream
|
||||
source:
|
||||
|
||||
- **Stable Diffusion XL** -- diffusers ``StableDiffusionXLWatermarker``
|
||||
``WATERMARK_MESSAGE`` (48-bit).
|
||||
- **FLUX.2** -- ``black-forest-labs/flux2`` ``src/flux2/watermark.py`` (48-bit).
|
||||
- **Stable Diffusion 1.x / 2.x** -- the library's default ``"StableDiffusionV1"``
|
||||
string (136-bit).
|
||||
|
||||
The watermark is fragile: it does NOT survive JPEG re-encoding or resizing
|
||||
(verified -- gone after JPEG q90), so detection works only on pristine PNG
|
||||
originals. Absence is never proof. Requires the optional ``invisible-watermark``
|
||||
package (extra: ``detect``); ``detect_invisible_watermark`` returns None when it
|
||||
is not installed.
|
||||
"""
|
||||
|
||||
# imwatermark ships no type stubs (like cv2); its decoder returns are Unknown.
|
||||
# Relax the untyped-library diagnostics for this thin wrapper module only.
|
||||
# pyright: reportMissingTypeStubs=false, reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, cast
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# Known 48-bit ``bits`` watermarks (dwtDct, no key), name -> message integer.
|
||||
_BITS_48: dict[str, int] = {
|
||||
"Stable Diffusion XL": 0b101100111110110010010000011110111011000110011110,
|
||||
"FLUX.2 (Black Forest Labs)": 0b001010101111111010000111100111001111010100101110,
|
||||
}
|
||||
# The invisible-watermark default string watermark (SD 1.x / 2.x).
|
||||
_SD1_STRING = b"StableDiffusionV1"
|
||||
|
||||
# Decoded bits/bytes never match a 48-bit pattern by chance: random decode lands
|
||||
# near 24/48, an exact embed at 48/48 (measured). 44 (<=4 bit errors) is a safe
|
||||
# floor that tolerates light perturbation without risking a false positive.
|
||||
_MATCH_48 = 44
|
||||
_MATCH_SD1_FRAC = 0.92 # fraction of the 136 string bits that must match
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
"""True if the optional imwatermark decoder is installed."""
|
||||
import importlib.util
|
||||
|
||||
return importlib.util.find_spec("imwatermark") is not None
|
||||
|
||||
|
||||
def _bits_match(value: int, ref: int, width: int = 48) -> int:
|
||||
"""Number of matching bits between two ``width``-bit integers."""
|
||||
return width - bin(value ^ ref).count("1")
|
||||
|
||||
|
||||
def _bytes_match_frac(a: bytes, b: bytes) -> float:
|
||||
"""Fraction of matching bits between two equal-length byte strings."""
|
||||
if len(a) != len(b) or not a:
|
||||
return 0.0
|
||||
diff = sum(bin(x ^ y).count("1") for x, y in zip(a, b, strict=True))
|
||||
return 1.0 - diff / (8 * len(b))
|
||||
|
||||
|
||||
def detect_invisible_watermark(image_path: Path) -> str | None:
|
||||
"""Return the embedding scheme name if a known open watermark is decoded.
|
||||
|
||||
Returns e.g. ``"Stable Diffusion XL"`` / ``"FLUX.2 (Black Forest Labs)"`` /
|
||||
``"Stable Diffusion 1.x / 2.x"``, or None if none matches, the decoder is
|
||||
unavailable, or the image can't be read. Meaningful only on pristine
|
||||
(un-re-encoded) images.
|
||||
"""
|
||||
if not is_available():
|
||||
return None
|
||||
import cv2
|
||||
from imwatermark import WatermarkDecoder
|
||||
|
||||
img = cv2.imread(str(image_path))
|
||||
if img is None:
|
||||
return None
|
||||
|
||||
# 48-bit fixed-message watermarks (SDXL, FLUX.2).
|
||||
try:
|
||||
bits = WatermarkDecoder("bits", 48).decode(img, "dwtDct")
|
||||
value = 0
|
||||
for bit in bits:
|
||||
value = (value << 1) | (1 if bit else 0)
|
||||
for name, ref in _BITS_48.items():
|
||||
if _bits_match(value, ref) >= _MATCH_48:
|
||||
return name
|
||||
except Exception as exc: # decode can fail on tiny images
|
||||
log.debug("48-bit watermark decode failed for %s: %s", image_path, exc)
|
||||
|
||||
# 136-bit default string watermark (SD 1.x / 2.x).
|
||||
try:
|
||||
raw = cast("bytes", WatermarkDecoder("bytes", 8 * len(_SD1_STRING)).decode(img, "dwtDct"))
|
||||
if _bytes_match_frac(raw, _SD1_STRING) >= _MATCH_SD1_FRAC:
|
||||
return "Stable Diffusion 1.x / 2.x"
|
||||
except Exception as exc:
|
||||
log.debug("string watermark decode failed for %s: %s", image_path, exc)
|
||||
|
||||
return None
|
||||
@@ -73,6 +73,17 @@ IPTC_AI_MARKERS: tuple[bytes, ...] = (
|
||||
b"compositeWithTrainedAlgorithmicMedia",
|
||||
)
|
||||
|
||||
# China's mandatory AI-content labeling (TC260, the national cybersecurity
|
||||
# standards committee). AI generators serving China embed an XMP block in the
|
||||
# TC260 namespace -- ``<TC260:AIGC>{"Label":"1",...}``. Doubao (ByteDance) uses
|
||||
# this; the same standard is mandatory for Jimeng, Kling, Qwen, Ernie, etc.,
|
||||
# so the marker covers the whole China-AIGC-labeled ecosystem. Container-
|
||||
# agnostic (XMP is text), so a raw-byte scan catches it in PNG/JPEG/etc.
|
||||
AIGC_MARKERS: tuple[bytes, ...] = (
|
||||
b"tc260.org.cn/ns/AIGC",
|
||||
b"TC260:AIGC",
|
||||
)
|
||||
|
||||
STANDARD_METADATA_KEYS: frozenset[str] = frozenset(
|
||||
[
|
||||
"Author",
|
||||
@@ -139,83 +150,173 @@ def has_ai_metadata(image_path: Path) -> bool:
|
||||
return True
|
||||
if C2PA_UUID in data:
|
||||
return True
|
||||
return any(marker in data for marker in IPTC_AI_MARKERS)
|
||||
if any(marker in data for marker in AIGC_MARKERS):
|
||||
return True
|
||||
if any(marker in data for marker in IPTC_AI_MARKERS):
|
||||
return True
|
||||
# xAI / Grok: no C2PA/IPTC/XMP -- only the EXIF Signature + UUID-Artist pair.
|
||||
return xai_signature(image_path)
|
||||
|
||||
|
||||
def _scan_png_c2pa_chunk(image_path: Path) -> dict[str, str]:
|
||||
"""Extract a human-readable summary of the C2PA manifest in a PNG file.
|
||||
def aigc_label(image_path: Path) -> dict[str, str] | None:
|
||||
"""Parse a China TC260 ``<TC260:AIGC>`` AI-labeling block, if present.
|
||||
|
||||
PIL does not expose the caBX JUMBF box via ``img.info``, so we delegate
|
||||
chunk extraction to the existing ``extract_c2pa_chunk`` helper and pull
|
||||
key fields from the JUMBF payload without a full CBOR parser.
|
||||
Returns the decoded JSON (e.g. ``{"Label": "1", "ContentProducer": ...}``)
|
||||
or None. The block is XMP text (HTML-entity encoded), so it is found by a
|
||||
container-agnostic raw-byte scan and works for PNG/JPEG/WebP alike.
|
||||
"""
|
||||
import html
|
||||
import json
|
||||
import re
|
||||
|
||||
with open(image_path, "rb") as f:
|
||||
data = f.read(1024 * 1024)
|
||||
match = re.search(rb"<TC260:AIGC>(.*?)</TC260:AIGC>", data, re.DOTALL)
|
||||
if not match:
|
||||
return None
|
||||
raw = html.unescape(match.group(1).decode("utf-8", "replace"))
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
except ValueError:
|
||||
return None
|
||||
return {str(k): str(v) for k, v in parsed.items()} if isinstance(parsed, dict) else None
|
||||
|
||||
|
||||
def synthid_source(image_path: Path) -> str | None:
|
||||
"""Return the vendor name(s) if the image carries a SynthID pixel watermark.
|
||||
|
||||
This is a *metadata-based* proxy: Google (Imagen/Gemini) and OpenAI
|
||||
(ChatGPT/DALL-E/gpt-image) embed an invisible SynthID watermark alongside
|
||||
a C2PA manifest, so a C2PA manifest signed by one of them on AI-generated
|
||||
content implies SynthID in the pixels. Adobe Firefly / Microsoft Designer
|
||||
sign C2PA but do not use SynthID, so they return None.
|
||||
|
||||
The verdict is reliable only while the C2PA manifest is intact -- absence
|
||||
is not proof, because C2PA can be stripped while the pixel watermark
|
||||
survives, and the pixel watermark itself is not locally detectable
|
||||
(proprietary decoder).
|
||||
|
||||
Args:
|
||||
image_path: Path to the image (PNG, JPEG, WebP, or ISOBMFF container).
|
||||
|
||||
Returns:
|
||||
Comma-joined vendor name(s) (e.g. ``"OpenAI"``) or None.
|
||||
"""
|
||||
from remove_ai_watermarks.noai.c2pa import extract_c2pa_info, synthid_vendors_in
|
||||
|
||||
# PNG: the caBX chunk parser gives a clean, structured issuer.
|
||||
vendors = extract_c2pa_info(image_path).get("synthid_vendors")
|
||||
if vendors:
|
||||
return ", ".join(vendors)
|
||||
|
||||
# Non-PNG containers (JPEG APP11, WebP, AVIF/HEIF/JXL uuid box) keep the
|
||||
# C2PA manifest where the PNG parser can't reach it. Binary-scan for the
|
||||
# same signal: a C2PA manifest from a SynthID-using issuer on AI content.
|
||||
with open(image_path, "rb") as f:
|
||||
data = f.read(1024 * 1024)
|
||||
has_c2pa = b"c2pa" in data.lower() or C2PA_UUID in data
|
||||
# Matches both "trainedAlgorithmicMedia" and "compositeWithTrainedAlgorithmicMedia".
|
||||
ai_source = b"trainedAlgorithmicMedia" in data or b"TrainedAlgorithmicMedia" in data
|
||||
if not (has_c2pa and ai_source):
|
||||
return None
|
||||
matched = synthid_vendors_in(data)
|
||||
return ", ".join(matched) if matched else None
|
||||
|
||||
|
||||
def exif_generator(image_path: Path) -> str | None:
|
||||
"""Return an AI-generator name from the EXIF ``Software`` / XMP ``CreatorTool``
|
||||
field, if it matches a known generator (see ``AI_GENERATOR_TOKENS``), else None.
|
||||
|
||||
Cross-format: EXIF is read via PIL + piexif for any container PIL can open
|
||||
(JPEG/WebP/AVIF/PNG); an XMP ``CreatorTool`` raw-byte scan additionally covers
|
||||
HEIF/JPEG-XL that PIL can't open without plugins. Only AI tokens match, so
|
||||
ordinary editors (plain "Adobe Photoshop", "GIMP") are not flagged.
|
||||
"""
|
||||
import re
|
||||
|
||||
from remove_ai_watermarks.noai.c2pa import extract_c2pa_chunk
|
||||
from remove_ai_watermarks.noai.constants import AI_GENERATOR_TOKENS
|
||||
|
||||
raw = extract_c2pa_chunk(image_path)
|
||||
if raw is None:
|
||||
return {}
|
||||
candidates: list[str] = []
|
||||
|
||||
# extract_c2pa_chunk returns chunk_header (8 bytes) + data + crc (4 bytes).
|
||||
payload = raw[8:-4]
|
||||
result: dict[str, str] = {"c2pa_manifest": f"C2PA manifest ({len(payload)} bytes)"}
|
||||
# EXIF Software / Artist / ImageDescription (0th IFD) via PIL exif bytes.
|
||||
try:
|
||||
import piexif
|
||||
from PIL import Image
|
||||
|
||||
def _cbor_text_after(key: bytes) -> str | None:
|
||||
"""Return the CBOR text-string immediately following ``key``.
|
||||
with Image.open(image_path) as img:
|
||||
exif_bytes = img.info.get("exif")
|
||||
if exif_bytes:
|
||||
tags = piexif.load(exif_bytes).get("0th", {})
|
||||
# Make catches camera-style tags AI tools reuse (Ideogram writes
|
||||
# Make="Ideogram AI"); real cameras put "Apple"/"Canon" there, which
|
||||
# carry no AI token, so this stays low-false-positive.
|
||||
for tag in (
|
||||
piexif.ImageIFD.Software,
|
||||
piexif.ImageIFD.Make,
|
||||
piexif.ImageIFD.Artist,
|
||||
piexif.ImageIFD.ImageDescription,
|
||||
):
|
||||
value = tags.get(tag)
|
||||
if isinstance(value, bytes):
|
||||
candidates.append(value.decode("latin1", "replace"))
|
||||
except Exception as exc: # unopenable format / malformed EXIF
|
||||
logger.debug("EXIF generator read failed for %s: %s", image_path, exc)
|
||||
|
||||
Handles CBOR major-type 3 length prefixes: direct (0x60-0x77),
|
||||
1-byte (0x78 NN), and 2-byte (0x79 NN NN).
|
||||
"""
|
||||
idx = payload.find(key)
|
||||
if idx < 0:
|
||||
return None
|
||||
p = idx + len(key)
|
||||
if p >= len(payload):
|
||||
return None
|
||||
head = payload[p]
|
||||
if 0x60 <= head <= 0x77:
|
||||
length, start = head - 0x60, p + 1
|
||||
elif head == 0x78 and p + 1 < len(payload):
|
||||
length, start = payload[p + 1], p + 2
|
||||
elif head == 0x79 and p + 2 < len(payload):
|
||||
length, start = (payload[p + 1] << 8) | payload[p + 2], p + 3
|
||||
else:
|
||||
return None
|
||||
raw_str = payload[start : start + length]
|
||||
try:
|
||||
return raw_str.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
return raw_str.decode("latin1", errors="replace")
|
||||
# XMP CreatorTool: text, container-agnostic (covers HEIF/JXL via raw scan).
|
||||
try:
|
||||
with open(image_path, "rb") as f:
|
||||
head = f.read(1024 * 1024)
|
||||
for match in re.finditer(rb"CreatorTool[>\"'=\s]{1,4}([^<\"']{1,80})", head):
|
||||
candidates.append(match.group(1).decode("latin1", "replace"))
|
||||
except Exception as exc:
|
||||
logger.debug("XMP CreatorTool scan failed for %s: %s", image_path, exc)
|
||||
|
||||
if generator := _cbor_text_after(b"name"):
|
||||
result["claim_generator"] = generator
|
||||
for value in candidates:
|
||||
if any(token in value.lower() for token in AI_GENERATOR_TOKENS):
|
||||
return value.strip()
|
||||
return None
|
||||
|
||||
if spec := _cbor_text_after(b"specVersion"):
|
||||
result["c2pa_spec"] = spec
|
||||
|
||||
dst_match = re.search(
|
||||
rb"(http://cv\.iptc\.org/newscodes/digitalsourcetype/[A-Za-z0-9_-]+)",
|
||||
payload,
|
||||
def xai_signature(image_path: Path) -> bool:
|
||||
"""Detect xAI / Grok's EXIF provenance signature scheme.
|
||||
|
||||
Grok image downloads (Aurora model) carry no C2PA, XMP, SynthID, or IPTC --
|
||||
their only provenance signal is a private EXIF pair: ``ImageDescription`` =
|
||||
``"Signature: <base64>"`` together with ``Artist`` = the image UUID. Verified
|
||||
stable across three independent generations (2026-05-26; see CLAUDE.md). The
|
||||
signature is xAI's and is not locally verifiable (no public key); detection
|
||||
keys on this distinctive, low-false-positive shape, not on the signature's
|
||||
validity. It survives only on the *original* JPEG download -- the web-UI
|
||||
image is a re-encoded WebP that drops EXIF.
|
||||
"""
|
||||
import re
|
||||
|
||||
try:
|
||||
import piexif
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(image_path) as img:
|
||||
exif_bytes = img.info.get("exif")
|
||||
if not exif_bytes:
|
||||
return False
|
||||
tags = piexif.load(exif_bytes).get("0th", {})
|
||||
except Exception as exc: # unopenable format / malformed EXIF
|
||||
logger.debug("xAI-signature EXIF read failed for %s: %s", image_path, exc)
|
||||
return False
|
||||
|
||||
def _text(tag: int) -> str:
|
||||
value = tags.get(tag)
|
||||
return value.decode("latin1", "replace").strip() if isinstance(value, bytes) else ""
|
||||
|
||||
description = _text(piexif.ImageIFD.ImageDescription)
|
||||
artist = _text(piexif.ImageIFD.Artist)
|
||||
# A 64+ char base64 blob after "Signature:" is far beyond any incidental
|
||||
# description text, and the UUID Artist makes the pair xAI-specific.
|
||||
has_signature = re.match(r"Signature:\s*[A-Za-z0-9+/=]{64,}", description) is not None
|
||||
is_uuid = (
|
||||
re.fullmatch(r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}", artist, re.IGNORECASE) is not None
|
||||
)
|
||||
if dst_match:
|
||||
result["digital_source_type"] = dst_match.group(1).decode("latin1")
|
||||
|
||||
actions = sorted(
|
||||
{m.decode("latin1") for m in re.findall(rb"c2pa\.(created|converted|edited|opened|placed)", payload)}
|
||||
)
|
||||
if actions:
|
||||
result["c2pa_actions"] = ", ".join(actions)
|
||||
|
||||
# Scan cert DN printable strings for the signer org name.
|
||||
signer_match = re.search(
|
||||
rb"([A-Za-z][A-Za-z0-9 .,&'()\-]{2,48}OpenAI[A-Za-z0-9 .,&'()\-]{0,48})",
|
||||
payload,
|
||||
)
|
||||
if signer_match:
|
||||
result["signer"] = signer_match.group(1).decode("latin1").strip()
|
||||
|
||||
return result
|
||||
return has_signature and is_uuid
|
||||
|
||||
|
||||
def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
@@ -229,19 +330,53 @@ def get_ai_metadata(image_path: Path) -> dict[str, str]:
|
||||
"""
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks.noai.c2pa import extract_c2pa_info, synthid_verdict
|
||||
|
||||
result: dict[str, str] = {}
|
||||
|
||||
with Image.open(image_path) as img:
|
||||
for key, value in img.info.items():
|
||||
if _is_ai_key(key):
|
||||
if isinstance(value, bytes):
|
||||
result[key] = f"<binary {len(value)} bytes>"
|
||||
elif isinstance(value, str) and len(value) > 200:
|
||||
result[key] = value[:200] + "…"
|
||||
else:
|
||||
result[key] = str(value)
|
||||
# PIL may not open AVIF/HEIF/JPEG-XL without optional plugins (and
|
||||
# ultralytics' Image.open patch can raise ModuleNotFoundError); fall through
|
||||
# to the C2PA/binary path on any open failure. See CLAUDE.md.
|
||||
try:
|
||||
with Image.open(image_path) as img:
|
||||
for key, value in img.info.items():
|
||||
if _is_ai_key(key):
|
||||
if isinstance(value, bytes):
|
||||
result[key] = f"<binary {len(value)} bytes>"
|
||||
elif isinstance(value, str) and len(value) > 200:
|
||||
result[key] = value[:200] + "…"
|
||||
else:
|
||||
result[key] = str(value)
|
||||
except Exception as exc:
|
||||
logger.debug("PIL could not open %s for AI-metadata scan: %s", image_path, exc)
|
||||
|
||||
result.update(_scan_png_c2pa_chunk(image_path))
|
||||
# C2PA manifest fields from the single canonical parser (noai/c2pa.py).
|
||||
c2pa = extract_c2pa_info(image_path)
|
||||
for key in (
|
||||
"c2pa_manifest",
|
||||
"claim_generator",
|
||||
"c2pa_spec",
|
||||
"issuer",
|
||||
"source_type",
|
||||
"actions",
|
||||
"synthid_watermark",
|
||||
):
|
||||
if key in c2pa:
|
||||
result.setdefault(key, str(c2pa[key]))
|
||||
|
||||
# Non-PNG containers (JPEG/WebP/AVIF): extract_c2pa_info is PNG-only, so
|
||||
# fall back to the format-agnostic source check for the SynthID verdict.
|
||||
if "synthid_watermark" not in result and (vendor := synthid_source(image_path)):
|
||||
result.setdefault("synthid_watermark", synthid_verdict(vendor))
|
||||
|
||||
# China TC260 AI-content label (Doubao and other China-served generators).
|
||||
if aigc := aigc_label(image_path):
|
||||
producer = aigc.get("ContentProducer", "")
|
||||
result["aigc_label"] = f"China AIGC label (TC260){f'; producer {producer}' if producer else ''}"
|
||||
|
||||
# xAI / Grok EXIF signature scheme (its only provenance signal).
|
||||
if xai_signature(image_path):
|
||||
result.setdefault("xai_signature", "xAI/Grok EXIF signature (Artist UUID + Signature blob)")
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ from remove_ai_watermarks.noai.constants import (
|
||||
C2PA_ISSUERS,
|
||||
C2PA_SIGNATURES,
|
||||
PNG_SIGNATURE,
|
||||
SYNTHID_C2PA_ISSUERS,
|
||||
)
|
||||
|
||||
|
||||
@@ -129,10 +130,56 @@ def extract_c2pa_info(image_path: Path) -> dict[str, Any]:
|
||||
return c2pa_info
|
||||
|
||||
|
||||
def _cbor_text_after(payload: bytes, key: bytes) -> str | None:
|
||||
"""Return the CBOR text-string immediately following ``key`` in ``payload``.
|
||||
|
||||
Handles CBOR major-type 3 length prefixes: direct (0x60-0x77), 1-byte
|
||||
(0x78 NN), and 2-byte (0x79 NN NN). This reads the actual encoded value, so
|
||||
it avoids the byte-grabbing artifacts a loose regex produces (e.g. the
|
||||
leading length byte showing up as ``fGPT-4o``).
|
||||
"""
|
||||
idx = payload.find(key)
|
||||
if idx < 0:
|
||||
return None
|
||||
p = idx + len(key)
|
||||
if p >= len(payload):
|
||||
return None
|
||||
head = payload[p]
|
||||
if 0x60 <= head <= 0x77:
|
||||
length, start = head - 0x60, p + 1
|
||||
elif head == 0x78 and p + 1 < len(payload):
|
||||
length, start = payload[p + 1], p + 2
|
||||
elif head == 0x79 and p + 2 < len(payload):
|
||||
length, start = (payload[p + 1] << 8) | payload[p + 2], p + 3
|
||||
else:
|
||||
return None
|
||||
raw_str = payload[start : start + length]
|
||||
try:
|
||||
return raw_str.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
return raw_str.decode("latin1", errors="replace")
|
||||
|
||||
|
||||
def synthid_verdict(vendors: str) -> str:
|
||||
"""Human-readable SynthID-source verdict, shared by all callers."""
|
||||
return f"likely present ({vendors} embeds SynthID with C2PA)"
|
||||
|
||||
|
||||
def synthid_vendors_in(buffer: bytes) -> list[str]:
|
||||
"""Return SynthID-using C2PA issuer names whose signature appears in ``buffer``.
|
||||
|
||||
Shared by the PNG caBX parser and the format-agnostic binary scan so both
|
||||
apply the same SYNTHID_C2PA_ISSUERS rule against their respective bytes.
|
||||
"""
|
||||
return sorted({name for sig, name in C2PA_ISSUERS.items() if sig in buffer and sig in SYNTHID_C2PA_ISSUERS})
|
||||
|
||||
|
||||
def _parse_c2pa_chunk(chunk_data: bytes, c2pa_info: dict[str, Any]) -> None:
|
||||
"""Parse C2PA chunk data and populate info dictionary."""
|
||||
c2pa_info["c2pa_manifest"] = f"C2PA manifest ({len(chunk_data)} bytes)"
|
||||
|
||||
# Find issuers
|
||||
issuers = []
|
||||
issuers: list[str] = []
|
||||
for sig, name in C2PA_ISSUERS.items():
|
||||
if sig in chunk_data:
|
||||
issuers.append(name)
|
||||
@@ -140,44 +187,25 @@ def _parse_c2pa_chunk(chunk_data: bytes, c2pa_info: dict[str, Any]) -> None:
|
||||
c2pa_info["issuer"] = ", ".join(set(issuers))
|
||||
|
||||
# Find AI tools
|
||||
ai_tools = []
|
||||
ai_tools: list[str] = []
|
||||
for sig, name in C2PA_AI_TOOLS.items():
|
||||
if sig in chunk_data:
|
||||
ai_tools.append(name)
|
||||
if ai_tools:
|
||||
c2pa_info["ai_tool"] = ", ".join(set(ai_tools))
|
||||
|
||||
# Extract software agent (multiple patterns)
|
||||
patterns = [
|
||||
rb"softwareAgent.*?dname([^\x00]+?)(?:q|l|m|n)",
|
||||
rb"software_agent[^\x00]*?([A-Za-z0-9_\-\.]+)",
|
||||
rb"Software[^\x00]*?([A-Za-z0-9_\-\. ]+)",
|
||||
]
|
||||
for pattern in patterns:
|
||||
match = re.search(pattern, chunk_data, re.DOTALL | re.IGNORECASE)
|
||||
if match:
|
||||
agent = match.group(1).decode("utf-8", errors="ignore").strip()
|
||||
if agent and len(agent) < 100:
|
||||
c2pa_info["software_agent"] = agent
|
||||
break
|
||||
|
||||
# Extract claim generator (multiple patterns)
|
||||
claim_patterns = [
|
||||
rb"claim_generator[^\x00]*?([A-Za-z0-9_\-\.\/\:]+)",
|
||||
rb"claimGenerator[^\x00]*?([A-Za-z0-9_\-\.\/\:]+)",
|
||||
rb"dname([^\x00]{3,50})(?:q|l|m|n|i)",
|
||||
]
|
||||
for pattern in claim_patterns:
|
||||
match = re.search(pattern, chunk_data, re.DOTALL | re.IGNORECASE)
|
||||
if match:
|
||||
gen_name = match.group(1).decode("utf-8", errors="ignore").strip()
|
||||
# Filter out common false positives
|
||||
if gen_name and len(gen_name) < 100 and not gen_name.startswith(("\\x", "\\\\x")):
|
||||
c2pa_info["claim_generator"] = gen_name
|
||||
break
|
||||
# Claim generator and spec version: read the CBOR text-string values
|
||||
# directly (regex byte-grabbing produced artifacts like ``fGPT-4o``).
|
||||
# Guard with isprintable(): on some manifests (e.g. Microsoft Designer) the
|
||||
# first ``name`` key precedes a binary field (a hash), not the generator
|
||||
# string, which would otherwise surface as control-char garbage.
|
||||
if (generator := _cbor_text_after(chunk_data, b"name")) and generator.isprintable():
|
||||
c2pa_info["claim_generator"] = generator
|
||||
if (spec := _cbor_text_after(chunk_data, b"specVersion")) and spec.isprintable():
|
||||
c2pa_info["c2pa_spec"] = spec
|
||||
|
||||
# Find actions
|
||||
actions = []
|
||||
actions: list[str] = []
|
||||
for sig, name in C2PA_ACTIONS.items():
|
||||
if sig in chunk_data:
|
||||
actions.append(name)
|
||||
@@ -192,12 +220,23 @@ def _parse_c2pa_chunk(chunk_data: bytes, c2pa_info: dict[str, Any]) -> None:
|
||||
c2pa_info["timestamps"] = [t.decode("utf-8") for t in timestamp_matches[:3]]
|
||||
|
||||
# Find digital source type
|
||||
ai_source = False
|
||||
if b"trainedAlgorithmicMedia" in chunk_data:
|
||||
c2pa_info["source_type"] = "trainedAlgorithmicMedia (AI-generated)"
|
||||
ai_source = True
|
||||
elif b"algorithmicMedia" in chunk_data:
|
||||
c2pa_info["source_type"] = "algorithmicMedia"
|
||||
elif b"compositeWithTrainedAlgorithmicMedia" in chunk_data:
|
||||
c2pa_info["source_type"] = "compositeWithTrainedAlgorithmicMedia (AI-enhanced)"
|
||||
ai_source = True
|
||||
|
||||
# SynthID pixel-watermark proxy: a C2PA manifest from a SynthID-using
|
||||
# vendor (Google/OpenAI) on AI-generated content implies an invisible
|
||||
# SynthID watermark in the pixels (see SYNTHID_C2PA_ISSUERS).
|
||||
synthid_vendors = synthid_vendors_in(chunk_data)
|
||||
if synthid_vendors and ai_source:
|
||||
c2pa_info["synthid_vendors"] = synthid_vendors
|
||||
c2pa_info["synthid_watermark"] = synthid_verdict(", ".join(synthid_vendors))
|
||||
|
||||
|
||||
def extract_c2pa_chunk(image_path: Path) -> bytes | None:
|
||||
|
||||
@@ -85,8 +85,32 @@ C2PA_ISSUERS = {
|
||||
b"Microsoft": "Microsoft",
|
||||
b"OpenAI": "OpenAI",
|
||||
b"Truepic": "Truepic",
|
||||
# Stability AI signs C2PA as "Stability AI" (cert org "Stability AI Ltd").
|
||||
# Verified on a live Brand Studio (DreamStudio successor) output, 2026-05-24.
|
||||
b"Stability AI": "Stability AI",
|
||||
}
|
||||
|
||||
# C2PA issuers whose signed outputs also carry an invisible SynthID pixel
|
||||
# watermark -- a metadata proxy for "SynthID is in the pixels":
|
||||
# - Google (Imagen/Gemini): embeds SynthID, long-standing (DeepMind docs).
|
||||
# - OpenAI (ChatGPT/Codex/API): pairs SynthID with C2PA since ~2026-05-20.
|
||||
# Confirmed by OpenAI's Help Center ("C2PA and SynthID in OpenAI-generated
|
||||
# images", updated 2026-05-21): "Images generated with ChatGPT, Codex, and
|
||||
# our API include both C2PA metadata and SynthID watermarks." OpenAI also
|
||||
# notes a signal may be absent if "the image was created before these
|
||||
# signals were available" -- so OpenAI images from BEFORE the rollout carry
|
||||
# C2PA WITHOUT SynthID (e.g. data/samples/openai-images-2/amur-leopard.png,
|
||||
# C2PA timestamp 2026-04-22). For OpenAI the proxy is therefore "likely",
|
||||
# not certain; the verdict string is hedged accordingly. OpenAI's own oracle
|
||||
# is openai.com/verify (Google's is the Gemini app "Verify with SynthID").
|
||||
# The issuer byte ("OpenAI"/"Google") is verified locally against data/samples;
|
||||
# the SynthID pairing is documented behavior (Google: DeepMind; OpenAI: above).
|
||||
# Adobe Firefly and Microsoft Designer sign C2PA but do NOT use SynthID, so a
|
||||
# C2PA manifest alone is not a SynthID signal -- the issuer is. The pixel
|
||||
# watermark is not locally detectable (proprietary decoder); the C2PA companion
|
||||
# is the proxy, and only while the manifest is intact.
|
||||
SYNTHID_C2PA_ISSUERS = frozenset({b"Google", b"OpenAI"})
|
||||
|
||||
# C2PA known AI tools
|
||||
C2PA_AI_TOOLS = {
|
||||
b"GPT-4o": "GPT-4o",
|
||||
@@ -98,6 +122,32 @@ C2PA_AI_TOOLS = {
|
||||
b"Firefly": "Firefly",
|
||||
}
|
||||
|
||||
# Lowercased substrings that mark an AI generator when found in an EXIF
|
||||
# ``Software`` / XMP ``CreatorTool`` value. Conservative on purpose: plain
|
||||
# editors like "Adobe Photoshop" or "GIMP" must NOT match (no AI token), so only
|
||||
# generator names land here. Add new generators here, not inline.
|
||||
AI_GENERATOR_TOKENS: frozenset[str] = frozenset(
|
||||
{
|
||||
"firefly",
|
||||
"dall-e",
|
||||
"dalle",
|
||||
"midjourney",
|
||||
"stable diffusion",
|
||||
"stable-diffusion",
|
||||
"stablediffusion",
|
||||
"comfyui",
|
||||
"automatic1111",
|
||||
"invokeai",
|
||||
"imagen",
|
||||
"gpt-image",
|
||||
"nightcafe",
|
||||
"ideogram",
|
||||
"leonardo",
|
||||
"flux",
|
||||
"dreamstudio",
|
||||
}
|
||||
)
|
||||
|
||||
# C2PA action types
|
||||
C2PA_ACTIONS = {
|
||||
b"c2pa.created": "created",
|
||||
@@ -106,6 +156,8 @@ C2PA_ACTIONS = {
|
||||
b"c2pa.filtered": "filtered",
|
||||
b"c2pa.cropped": "cropped",
|
||||
b"c2pa.resized": "resized",
|
||||
b"c2pa.opened": "opened",
|
||||
b"c2pa.placed": "placed",
|
||||
}
|
||||
|
||||
# PNG signature
|
||||
|
||||
@@ -36,9 +36,10 @@ _HAS_COLOR_MATCHER = False
|
||||
_HAS_DIFFUSERS = False
|
||||
|
||||
try:
|
||||
from ctrlregen.pipeline import CustomCtrlRegenPipeline
|
||||
from diffusers import AutoencoderKL, ControlNetModel, UniPCMultistepScheduler
|
||||
|
||||
from remove_ai_watermarks.noai.ctrlregen.pipeline import CustomCtrlRegenPipeline
|
||||
|
||||
_HAS_DIFFUSERS = True
|
||||
except ImportError:
|
||||
AutoencoderKL = None # type: ignore[assignment,misc]
|
||||
@@ -54,7 +55,7 @@ except ImportError:
|
||||
CannyDetector = None # type: ignore[assignment,misc]
|
||||
|
||||
try:
|
||||
from ctrlregen.color import color_match
|
||||
from remove_ai_watermarks.noai.ctrlregen.color import color_match
|
||||
|
||||
_HAS_COLOR_MATCHER = True
|
||||
except ImportError:
|
||||
@@ -239,7 +240,7 @@ class CtrlRegenEngine:
|
||||
needs_tiling = orig_w > TILE_SIZE or orig_h > TILE_SIZE
|
||||
|
||||
if needs_tiling:
|
||||
from ctrlregen.tiling import resize_center_crop, run_tiled
|
||||
from remove_ai_watermarks.noai.ctrlregen.tiling import resize_center_crop, run_tiled
|
||||
|
||||
aligned_w = orig_w // 8 * 8
|
||||
aligned_h = orig_h // 8 * 8
|
||||
@@ -264,7 +265,7 @@ class CtrlRegenEngine:
|
||||
ip_adapter_image=orig_image,
|
||||
)
|
||||
else:
|
||||
from ctrlregen.tiling import resize_center_crop
|
||||
from remove_ai_watermarks.noai.ctrlregen.tiling import resize_center_crop
|
||||
|
||||
proc_image = resize_center_crop(image, PROCESS_SIZE)
|
||||
self._set_progress(f"Preprocessed {orig_w}x{orig_h}px → {proc_image.size[0]}x{proc_image.size[1]}px")
|
||||
|
||||
@@ -176,8 +176,11 @@ def _reinstall_torch_cuda_and_restart() -> None:
|
||||
return
|
||||
|
||||
os.environ[_CUDA_FIX_ENV_KEY] = "1"
|
||||
restart_code = f"import sys; sys.argv = {sys.argv!r}; from remove_ai_watermarks.cli import main; sys.exit(main())"
|
||||
os.execl(sys.executable, sys.executable, "-c", restart_code)
|
||||
# Re-exec via ``-m`` rather than building a ``-c`` string from repr(sys.argv).
|
||||
# ``-m`` makes Python set argv[0] to the module path, so forward only the
|
||||
# actual args (sys.argv[1:]); passing the full argv would re-inject the
|
||||
# program name as a spurious first argument to Click.
|
||||
os.execv(sys.executable, [sys.executable, "-m", "remove_ai_watermarks.cli", *sys.argv[1:]])
|
||||
|
||||
|
||||
def _ensure_watermark_deps() -> None:
|
||||
|
||||
@@ -2,10 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
from pathlib import Path
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
@@ -13,6 +10,22 @@ import pytest
|
||||
from PIL import Image
|
||||
from PIL.PngImagePlugin import PngInfo
|
||||
|
||||
CORPUS_NEG_DIR = Path(__file__).resolve().parent.parent / "data" / "synthid_corpus" / "images" / "neg"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clean_photo() -> Path:
|
||||
"""A verified-negative real photo from the corpus neg/ set.
|
||||
|
||||
Used by the "non-AI image" assertions (no SynthID, verdict unknown). These
|
||||
are real photos with no AI provenance, the ground truth for "must not false-
|
||||
positive". Skips if the corpus is not checked out.
|
||||
"""
|
||||
files = sorted(CORPUS_NEG_DIR.glob("*")) if CORPUS_NEG_DIR.exists() else []
|
||||
if not files:
|
||||
pytest.skip("no corpus neg/ images present")
|
||||
return files[0]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tmp_image_path(tmp_path: Path) -> Path:
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import TYPE_CHECKING
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
@@ -71,6 +72,27 @@ def _mock_invisible_engine():
|
||||
return mock_cls, mock_engine
|
||||
|
||||
|
||||
def _mock_invisible_engine_drops_alpha():
|
||||
"""Mock InvisibleEngine that mimics the real engine's BGR-only output path.
|
||||
|
||||
The real diffusion-based engine reads with cv2.IMREAD_COLOR and writes a
|
||||
3-channel result. This mock simulates that so we can regression-test alpha
|
||||
preservation across the ``all`` pipeline.
|
||||
"""
|
||||
|
||||
def _mock_remove_watermark(image_path, output_path=None, **kwargs):
|
||||
out = output_path or image_path.with_stem(image_path.stem + "_clean")
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
bgr = cv2.imread(str(image_path), cv2.IMREAD_COLOR)
|
||||
cv2.imwrite(str(out), bgr)
|
||||
return out
|
||||
|
||||
mock_engine = MagicMock()
|
||||
mock_engine.remove_watermark.side_effect = _mock_remove_watermark
|
||||
mock_cls = MagicMock(return_value=mock_engine)
|
||||
return mock_cls, mock_engine
|
||||
|
||||
|
||||
class TestMainGroup:
|
||||
"""Tests for the top-level CLI group."""
|
||||
|
||||
@@ -143,6 +165,73 @@ class TestVisibleCommand:
|
||||
result = runner.invoke(main, ["visible", "/nonexistent/file.png"])
|
||||
assert result.exit_code != 0
|
||||
|
||||
def test_visible_preserves_rgba_transparency(self, runner, tmp_path):
|
||||
"""Visible removal on an RGBA PNG must keep the alpha channel,
|
||||
not silently flatten the image onto an opaque background.
|
||||
"""
|
||||
rgba = np.zeros((200, 200, 4), dtype=np.uint8)
|
||||
rgba[:, :, :3] = 200 # light grey foreground
|
||||
rgba[50:150, 50:150, 3] = 255 # opaque square in the middle, rest transparent
|
||||
src = tmp_path / "rgba_in.png"
|
||||
cv2.imwrite(str(src), rgba)
|
||||
|
||||
output = tmp_path / "rgba_out.png"
|
||||
result = runner.invoke(
|
||||
main,
|
||||
["visible", str(src), "-o", str(output), "--no-detect"],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert output.exists()
|
||||
|
||||
out = cv2.imread(str(output), cv2.IMREAD_UNCHANGED)
|
||||
assert out.ndim == 3, f"output is not 3D: shape={out.shape}"
|
||||
assert out.shape[2] == 4, f"output is not RGBA: shape={out.shape}"
|
||||
# The transparent corners must remain transparent.
|
||||
assert out[0, 0, 3] == 0
|
||||
assert out[199, 199, 3] == 0
|
||||
# The opaque centre remains opaque (the watermark region default is bottom-right,
|
||||
# which doesn't overlap the centre square at 200x200).
|
||||
assert out[100, 100, 3] == 255
|
||||
|
||||
def test_visible_clears_alpha_in_watermark_region(self, runner, tmp_path):
|
||||
"""When inpainting an RGBA image, the watermark region must be cleared
|
||||
in the alpha channel so the sparkle area becomes transparent, not opaque-black.
|
||||
"""
|
||||
rgba = np.full((200, 200, 4), 255, dtype=np.uint8) # fully opaque white
|
||||
src = tmp_path / "rgba_full.png"
|
||||
cv2.imwrite(str(src), rgba)
|
||||
|
||||
output = tmp_path / "rgba_cleared.png"
|
||||
result = runner.invoke(
|
||||
main,
|
||||
["visible", str(src), "-o", str(output), "--no-detect"],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
out = cv2.imread(str(output), cv2.IMREAD_UNCHANGED)
|
||||
assert out.shape[2] == 4
|
||||
# Default sparkle position is in the bottom-right; alpha there must be 0.
|
||||
from remove_ai_watermarks.gemini_engine import get_watermark_config
|
||||
|
||||
cfg = get_watermark_config(200, 200)
|
||||
px, py = cfg.get_position(200, 200)
|
||||
size = cfg.logo_size
|
||||
assert out[py + size // 2, px + size // 2, 3] == 0, "alpha in the watermark region was not cleared"
|
||||
|
||||
def test_visible_rgb_input_stays_rgb(self, runner, sample_png, tmp_path):
|
||||
"""Regression: a plain RGB PNG must NOT gain a spurious alpha channel."""
|
||||
output = tmp_path / "rgb_out.png"
|
||||
result = runner.invoke(
|
||||
main,
|
||||
["visible", str(sample_png), "-o", str(output), "--no-detect"],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
out = cv2.imread(str(output), cv2.IMREAD_UNCHANGED)
|
||||
assert out.ndim == 3, f"output is not 3D: shape={out.shape}"
|
||||
assert out.shape[2] == 3, f"RGB input produced non-RGB output: shape={out.shape}"
|
||||
|
||||
|
||||
class TestInvisibleCommand:
|
||||
"""Tests for the 'invisible' subcommand."""
|
||||
@@ -209,6 +298,33 @@ class TestAllCommand:
|
||||
result = runner.invoke(main, ["all", "/nonexistent/file.png"])
|
||||
assert result.exit_code != 0
|
||||
|
||||
def test_all_preserves_rgba_across_invisible_step(self, runner, tmp_path):
|
||||
"""Regression: ``all`` must keep transparency even when the invisible
|
||||
step writes a 3-channel result (as the real diffusion engine does).
|
||||
"""
|
||||
rgba = np.zeros((200, 200, 4), dtype=np.uint8)
|
||||
rgba[:, :, :3] = 200
|
||||
rgba[50:150, 50:150, 3] = 255 # opaque square; corners transparent
|
||||
src = tmp_path / "rgba_in.png"
|
||||
cv2.imwrite(str(src), rgba)
|
||||
|
||||
output = tmp_path / "rgba_out.png"
|
||||
mock_cls, _engine = _mock_invisible_engine_drops_alpha()
|
||||
with (
|
||||
patch("remove_ai_watermarks.cli.InvisibleEngine", mock_cls, create=True),
|
||||
patch("remove_ai_watermarks.invisible_engine.InvisibleEngine", mock_cls),
|
||||
patch("remove_ai_watermarks.cli.invisible_available", return_value=True, create=True),
|
||||
patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True),
|
||||
):
|
||||
result = runner.invoke(main, ["all", str(src), "-o", str(output)])
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
out = cv2.imread(str(output), cv2.IMREAD_UNCHANGED)
|
||||
assert out.ndim == 3, f"output not 3D: shape={out.shape}"
|
||||
assert out.shape[2] == 4, f"output is not RGBA: shape={out.shape}"
|
||||
assert out[0, 0, 3] == 0
|
||||
assert out[100, 100, 3] == 255
|
||||
|
||||
|
||||
class TestMetadataCommand:
|
||||
"""Tests for the 'metadata' subcommand."""
|
||||
@@ -243,6 +359,36 @@ class TestMetadataCommand:
|
||||
assert "stripped" in result.output
|
||||
|
||||
|
||||
class TestIdentifyCommand:
|
||||
"""Tests for the 'identify' subcommand."""
|
||||
|
||||
def test_identify_help(self, runner):
|
||||
result = runner.invoke(main, ["identify", "--help"])
|
||||
assert result.exit_code == 0
|
||||
|
||||
def test_identify_clean_png(self, runner, tmp_clean_png):
|
||||
result = runner.invoke(main, ["identify", str(tmp_clean_png), "--no-visible"])
|
||||
assert result.exit_code == 0
|
||||
assert "unknown" in result.output
|
||||
|
||||
def test_identify_ai_png_reports_platform(self, runner, tmp_png_with_ai_metadata):
|
||||
result = runner.invoke(main, ["identify", str(tmp_png_with_ai_metadata), "--no-visible"])
|
||||
assert result.exit_code == 0
|
||||
assert "AI-generated" in result.output
|
||||
assert "Stable Diffusion" in result.output
|
||||
|
||||
def test_identify_json_is_valid(self, runner, tmp_png_with_ai_metadata):
|
||||
result = runner.invoke(main, ["identify", str(tmp_png_with_ai_metadata), "--no-visible", "--json"])
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert payload["is_ai_generated"] is True
|
||||
assert payload["confidence"] == "high"
|
||||
|
||||
def test_identify_nonexistent_file(self, runner):
|
||||
result = runner.invoke(main, ["identify", "/nonexistent/file.png"])
|
||||
assert result.exit_code != 0
|
||||
|
||||
|
||||
class TestBatchCommand:
|
||||
"""Tests for the 'batch' subcommand."""
|
||||
|
||||
@@ -319,6 +465,37 @@ class TestBatchCommand:
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "3 processed" in result.output
|
||||
|
||||
def test_batch_all_mode_preserves_rgba(self, runner, tmp_path):
|
||||
"""Regression: batch ``all`` must keep transparency across the
|
||||
alpha-dropping invisible step (mirrors test_all_preserves_rgba_...).
|
||||
"""
|
||||
input_dir = tmp_path / "input"
|
||||
input_dir.mkdir()
|
||||
rgba = np.zeros((200, 200, 4), dtype=np.uint8)
|
||||
rgba[:, :, :3] = 200
|
||||
rgba[50:150, 50:150, 3] = 255
|
||||
cv2.imwrite(str(input_dir / "rgba.png"), rgba)
|
||||
|
||||
output_dir = tmp_path / "output"
|
||||
mock_cls, _engine = _mock_invisible_engine_drops_alpha()
|
||||
with (
|
||||
patch("remove_ai_watermarks.cli.InvisibleEngine", mock_cls, create=True),
|
||||
patch("remove_ai_watermarks.invisible_engine.InvisibleEngine", mock_cls),
|
||||
patch("remove_ai_watermarks.cli.invisible_available", return_value=True, create=True),
|
||||
patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True),
|
||||
):
|
||||
result = runner.invoke(
|
||||
main,
|
||||
["batch", str(input_dir), "-o", str(output_dir), "--mode", "all"],
|
||||
)
|
||||
assert result.exit_code == 0, result.output
|
||||
|
||||
out = cv2.imread(str(output_dir / "rgba.png"), cv2.IMREAD_UNCHANGED)
|
||||
assert out.ndim == 3, f"output not 3D: shape={out.shape}"
|
||||
assert out.shape[2] == 4, f"output is not RGBA: shape={out.shape}"
|
||||
assert out[0, 0, 3] == 0
|
||||
assert out[100, 100, 3] == 255
|
||||
|
||||
def test_batch_default_output_dir(self, runner, tmp_path):
|
||||
input_dir = _make_batch_dir(tmp_path)
|
||||
result = runner.invoke(
|
||||
|
||||
@@ -12,6 +12,7 @@ from remove_ai_watermarks.gemini_engine import (
|
||||
WatermarkPosition,
|
||||
WatermarkSize,
|
||||
_calculate_alpha_map,
|
||||
detect_sparkle_confidence,
|
||||
get_watermark_config,
|
||||
get_watermark_size,
|
||||
)
|
||||
@@ -174,6 +175,21 @@ class TestDetection:
|
||||
assert isinstance(result.gradient_score, float)
|
||||
|
||||
|
||||
class TestDetectSparkleConfidence:
|
||||
"""File-level entry point used by identify.py."""
|
||||
|
||||
def test_returns_float_in_range_for_real_image(self, tmp_image_path):
|
||||
conf = detect_sparkle_confidence(tmp_image_path)
|
||||
assert conf is not None
|
||||
assert 0.0 <= conf <= 1.0
|
||||
|
||||
def test_returns_none_for_unreadable_file(self, tmp_path):
|
||||
# cv2.imread returns None for a non-image; the helper must not raise.
|
||||
bogus = tmp_path / "not_an_image.png"
|
||||
bogus.write_bytes(b"this is not a PNG")
|
||||
assert detect_sparkle_confidence(bogus) is None
|
||||
|
||||
|
||||
# ── Inpainting ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,378 @@
|
||||
"""Tests for the provenance identifier (identify.py).
|
||||
|
||||
Pure attribution logic is unit-tested directly; end-to-end verdicts assert
|
||||
against the real committed C2PA / IPTC fixtures in data/samples/.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import asdict
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from remove_ai_watermarks.identify import (
|
||||
ProvenanceReport,
|
||||
_ai_tools_in,
|
||||
_attribute_platform,
|
||||
_issuers_in,
|
||||
identify,
|
||||
)
|
||||
|
||||
# Where the lazy import inside identify._visible_sparkle resolves the detector.
|
||||
_SPARKLE_TARGET = "remove_ai_watermarks.gemini_engine.detect_sparkle_confidence"
|
||||
|
||||
SAMPLES_DIR = Path(__file__).resolve().parent.parent / "data" / "samples"
|
||||
|
||||
|
||||
# ── Pure attribution logic (no file IO) ─────────────────────────────
|
||||
|
||||
|
||||
class TestAttributePlatform:
|
||||
def test_openai(self):
|
||||
assert "OpenAI" in (_attribute_platform(["OpenAI"]) or "")
|
||||
|
||||
def test_designer_wins_over_openai_backend(self):
|
||||
# Microsoft Designer signs as "OpenAI, Microsoft"; name the product.
|
||||
platform = _attribute_platform(["OpenAI", "Microsoft"])
|
||||
assert platform
|
||||
assert "Designer" in platform
|
||||
|
||||
def test_adobe(self):
|
||||
assert _attribute_platform(["Adobe"]) == "Adobe Firefly"
|
||||
|
||||
def test_google(self):
|
||||
assert "Google" in (_attribute_platform(["Google LLC"]) or "")
|
||||
|
||||
def test_truepic_is_signer_not_generator(self):
|
||||
platform = _attribute_platform(["Truepic"])
|
||||
assert platform
|
||||
assert "signer" in platform.lower()
|
||||
|
||||
def test_microsoft_label_is_model_neutral(self):
|
||||
# Bing now runs MAI-Image, not DALL-E; the label must not claim DALL-E.
|
||||
platform = _attribute_platform(["Microsoft"])
|
||||
assert platform
|
||||
assert "DALL-E" not in platform
|
||||
|
||||
def test_stability(self):
|
||||
platform = _attribute_platform(["Stability AI"])
|
||||
assert platform
|
||||
assert "Stability AI" in platform
|
||||
|
||||
def test_empty_is_none(self):
|
||||
assert _attribute_platform([]) is None
|
||||
|
||||
|
||||
class TestIssuersIn:
|
||||
def test_finds_openai(self):
|
||||
assert _issuers_in(b"...OpenAI...trainedAlgorithmicMedia") == ["OpenAI"]
|
||||
|
||||
def test_finds_multiple_sorted(self):
|
||||
assert _issuers_in(b"Microsoft and OpenAI") == ["Microsoft", "OpenAI"]
|
||||
|
||||
def test_none_present(self):
|
||||
assert _issuers_in(b"just some bytes") == []
|
||||
|
||||
|
||||
class TestAiToolsIn:
|
||||
def test_finds_generator(self):
|
||||
assert _ai_tools_in(b"...claim_generator Imagen 3...") == ["Imagen"]
|
||||
|
||||
def test_none_present(self):
|
||||
assert _ai_tools_in(b"a regular photo, no tools") == []
|
||||
|
||||
|
||||
class TestIdentifyNonPng:
|
||||
"""Non-PNG containers (JPEG/WebP/AVIF) carry C2PA where the caBX parser can't
|
||||
reach; identify recovers issuer + generator via the binary scan. Synthetic
|
||||
byte blobs mirror tests/test_metadata.py::TestSynthIDSourceNonPng.
|
||||
"""
|
||||
|
||||
def _c2pa_jpeg(self, tmp_path: Path, blob: bytes) -> Path:
|
||||
path = tmp_path / "img.jpg"
|
||||
path.write_bytes(b"\xff\xd8\xff\xe1jumbc2pa" + blob + b"\xff\xd9")
|
||||
return path
|
||||
|
||||
def test_google_imagen_jpeg(self, tmp_path: Path):
|
||||
path = self._c2pa_jpeg(tmp_path, b"Google Imagen ... trainedAlgorithmicMedia")
|
||||
r = identify(path, check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform is not None
|
||||
assert "Google" in r.platform
|
||||
# Generator recovered from the non-PNG blob shows up in the c2pa signal.
|
||||
c2pa_signal = next(s for s in r.signals if s.name == "c2pa")
|
||||
assert "Imagen" in c2pa_signal.detail
|
||||
|
||||
def test_openai_jpeg_has_synthid(self, tmp_path: Path):
|
||||
path = self._c2pa_jpeg(tmp_path, b"OpenAI DALL-E ... trainedAlgorithmicMedia")
|
||||
r = identify(path, check_visible=False)
|
||||
assert any("SynthID" in w for w in r.watermarks)
|
||||
|
||||
def test_stability_ai_issuer_attributed_no_synthid(self, tmp_path: Path):
|
||||
path = self._c2pa_jpeg(tmp_path, b"Stability AI ... trainedAlgorithmicMedia")
|
||||
r = identify(path, check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform is not None
|
||||
assert "Stability AI" in r.platform
|
||||
assert not any("SynthID" in w for w in r.watermarks) # Stability does not use SynthID
|
||||
|
||||
def test_c2pa_without_ai_marker_is_unknown(self, tmp_path: Path):
|
||||
# Adobe signs C2PA on plain Photoshop edits too. Without an AI digital-
|
||||
# source marker, the honest verdict is unknown -- the C2PA watermark is
|
||||
# still listed, but is_ai_generated is not asserted True.
|
||||
path = self._c2pa_jpeg(tmp_path, b"Adobe ... no ai marker here")
|
||||
r = identify(path, check_visible=False)
|
||||
assert r.is_ai_generated is None
|
||||
assert any("C2PA" in w for w in r.watermarks)
|
||||
assert not any("SynthID" in w for w in r.watermarks)
|
||||
|
||||
|
||||
# ── End-to-end verdicts on real fixtures ────────────────────────────
|
||||
|
||||
|
||||
@pytest.mark.skipif(not SAMPLES_DIR.exists(), reason="data/samples not present")
|
||||
class TestIdentifyRealSamples:
|
||||
def test_openai_chatgpt(self):
|
||||
r = identify(SAMPLES_DIR / "chatgpt-1.png", check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.confidence == "high"
|
||||
assert r.platform
|
||||
assert "OpenAI" in r.platform
|
||||
assert any("C2PA" in w for w in r.watermarks)
|
||||
assert any("SynthID" in w for w in r.watermarks)
|
||||
|
||||
def test_adobe_firefly_has_no_synthid(self):
|
||||
r = identify(SAMPLES_DIR / "firefly-1.png", check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform == "Adobe Firefly"
|
||||
assert not any("SynthID" in w for w in r.watermarks)
|
||||
|
||||
def test_iptc_made_with_ai(self):
|
||||
# mj-1.png carries the IPTC digitalSourceType "Made with AI" marker.
|
||||
r = identify(SAMPLES_DIR / "mj-1.png", check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert any("IPTC" in w for w in r.watermarks)
|
||||
|
||||
def test_clean_photo_is_unknown_not_clean(self, clean_photo: Path):
|
||||
r = identify(clean_photo, check_visible=False)
|
||||
assert r.is_ai_generated is None # never asserted False
|
||||
assert r.platform is None
|
||||
assert r.confidence == "none"
|
||||
assert r.watermarks == []
|
||||
|
||||
def test_strip_caveat_always_present(self, clean_photo: Path):
|
||||
r = identify(clean_photo, check_visible=False)
|
||||
assert any("not proof" in c for c in r.caveats)
|
||||
|
||||
def test_returns_report_dataclass(self):
|
||||
assert isinstance(identify(SAMPLES_DIR / "firefly-1.png", check_visible=False), ProvenanceReport)
|
||||
|
||||
|
||||
# ── Local diffusion parameters (Stable Diffusion / ComfyUI) ─────────
|
||||
|
||||
|
||||
class TestIdentifyLocalParams:
|
||||
"""A PNG carrying SD-style generation params is attributed to a local pipeline."""
|
||||
|
||||
def test_sd_params_attributed_to_local_pipeline(self, tmp_png_with_ai_metadata: Path):
|
||||
r = identify(tmp_png_with_ai_metadata, check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.confidence == "high"
|
||||
assert r.platform is not None
|
||||
assert "Stable Diffusion" in r.platform
|
||||
assert any("generation parameters" in w for w in r.watermarks)
|
||||
|
||||
def test_gen_params_signal_lists_keys(self, tmp_png_with_ai_metadata: Path):
|
||||
r = identify(tmp_png_with_ai_metadata, check_visible=False)
|
||||
signal = next(s for s in r.signals if s.name == "gen_params")
|
||||
assert "parameters" in signal.detail
|
||||
assert signal.confidence == "high"
|
||||
|
||||
def test_clean_png_is_unknown(self, tmp_clean_png: Path):
|
||||
r = identify(tmp_clean_png, check_visible=False)
|
||||
assert r.is_ai_generated is None
|
||||
assert r.platform is None
|
||||
assert r.confidence == "none"
|
||||
assert r.signals == []
|
||||
|
||||
|
||||
# ── Visible-sparkle fallback (mocked detector) ──────────────────────
|
||||
|
||||
|
||||
class TestIdentifyVisibleSparkle:
|
||||
"""The visible-sparkle signal gates on the corpus-tuned threshold (0.5)."""
|
||||
|
||||
def test_above_threshold_promotes_to_medium(self, tmp_clean_png: Path):
|
||||
with patch(_SPARKLE_TARGET, return_value=0.7):
|
||||
r = identify(tmp_clean_png, check_visible=True)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.confidence == "medium"
|
||||
assert r.platform is not None
|
||||
assert "Gemini" in r.platform
|
||||
signal = next(s for s in r.signals if s.name == "visible_sparkle")
|
||||
assert signal.confidence == "medium"
|
||||
|
||||
def test_below_threshold_not_promoted(self, tmp_clean_png: Path):
|
||||
with patch(_SPARKLE_TARGET, return_value=0.4):
|
||||
r = identify(tmp_clean_png, check_visible=True)
|
||||
assert r.is_ai_generated is None
|
||||
assert not any(s.name == "visible_sparkle" for s in r.signals)
|
||||
|
||||
def test_detector_unavailable_does_not_crash(self, tmp_clean_png: Path):
|
||||
with patch(_SPARKLE_TARGET, return_value=None):
|
||||
r = identify(tmp_clean_png, check_visible=True)
|
||||
assert r.is_ai_generated is None
|
||||
assert not any(s.name == "visible_sparkle" for s in r.signals)
|
||||
|
||||
def test_check_visible_false_skips_detector(self, tmp_clean_png: Path):
|
||||
# Even a strong detection is ignored when the caller opts out.
|
||||
with patch(_SPARKLE_TARGET, return_value=0.99) as mock_detect:
|
||||
r = identify(tmp_clean_png, check_visible=False)
|
||||
mock_detect.assert_not_called()
|
||||
assert not any(s.name == "visible_sparkle" for s in r.signals)
|
||||
|
||||
def test_metadata_keeps_high_even_with_sparkle(self, tmp_png_with_ai_metadata: Path):
|
||||
# Metadata verdict (high) is not downgraded by an additional sparkle hit.
|
||||
with patch(_SPARKLE_TARGET, return_value=0.7):
|
||||
r = identify(tmp_png_with_ai_metadata, check_visible=True)
|
||||
assert r.confidence == "high"
|
||||
|
||||
|
||||
# ── Caveats and serialization ───────────────────────────────────────
|
||||
|
||||
|
||||
@pytest.mark.skipif(not SAMPLES_DIR.exists(), reason="data/samples not present")
|
||||
class TestIdentifyCaveats:
|
||||
def test_openai_hedge_caveat_present(self):
|
||||
r = identify(SAMPLES_DIR / "chatgpt-1.png", check_visible=False)
|
||||
assert any("before the rollout" in c for c in r.caveats)
|
||||
|
||||
def test_synthid_proxy_caveat_present(self):
|
||||
r = identify(SAMPLES_DIR / "chatgpt-1.png", check_visible=False)
|
||||
assert any("not locally" in c for c in r.caveats)
|
||||
|
||||
def test_caveats_are_deduplicated(self):
|
||||
r = identify(SAMPLES_DIR / "chatgpt-1.png", check_visible=False)
|
||||
assert len(r.caveats) == len(set(r.caveats))
|
||||
|
||||
|
||||
class TestReportSerializable:
|
||||
def test_report_is_json_serializable(self, tmp_png_with_ai_metadata: Path):
|
||||
# The CLI --json path relies on asdict + json.dumps(default=str).
|
||||
report = identify(tmp_png_with_ai_metadata, check_visible=False)
|
||||
dumped = json.dumps(asdict(report), default=str)
|
||||
assert "is_ai_generated" in dumped
|
||||
|
||||
|
||||
class TestIdentifyExifGenerator:
|
||||
"""An AI generator tag in EXIF/XMP (incl. AVIF) drives attribution."""
|
||||
|
||||
def test_avif_firefly_software_attributed(self, tmp_path: Path):
|
||||
import piexif
|
||||
from PIL import Image
|
||||
|
||||
exif = piexif.dump({"0th": {piexif.ImageIFD.Software: b"Adobe Firefly"}, "Exif": {}, "GPS": {}, "1st": {}})
|
||||
path = tmp_path / "firefly.avif"
|
||||
Image.new("RGB", (64, 64), (90, 80, 70)).save(path, exif=exif)
|
||||
r = identify(path, check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform is not None
|
||||
assert "Firefly" in r.platform
|
||||
assert any("generator tag" in w for w in r.watermarks)
|
||||
|
||||
|
||||
class TestIdentifyXaiSignature:
|
||||
"""xAI / Grok's EXIF Signature + UUID-Artist drives an xAI verdict."""
|
||||
|
||||
def test_grok_signature_attributed(self, tmp_path: Path):
|
||||
import piexif
|
||||
from PIL import Image
|
||||
|
||||
exif = piexif.dump(
|
||||
{
|
||||
"0th": {
|
||||
piexif.ImageIFD.ImageDescription: b"Signature: " + b"A" * 120,
|
||||
piexif.ImageIFD.Artist: b"12345678-1234-1234-1234-123456789abc",
|
||||
},
|
||||
"Exif": {},
|
||||
"GPS": {},
|
||||
"1st": {},
|
||||
}
|
||||
)
|
||||
path = tmp_path / "grok.jpg"
|
||||
Image.new("RGB", (64, 64), (70, 80, 90)).save(path, exif=exif)
|
||||
r = identify(path, check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform is not None
|
||||
assert "xAI" in r.platform
|
||||
assert any("xAI/Grok" in w for w in r.watermarks)
|
||||
|
||||
|
||||
# ── Open invisible watermark (SD/SDXL/FLUX) integration ─────────────
|
||||
|
||||
from remove_ai_watermarks.invisible_watermark import is_available as _wm_available # noqa: E402
|
||||
|
||||
|
||||
@pytest.mark.skipif(not _wm_available(), reason="invisible-watermark not installed")
|
||||
class TestIdentifyInvisibleWatermark:
|
||||
def _sdxl_watermarked(self, tmp_path: Path) -> Path:
|
||||
import cv2
|
||||
import numpy as np
|
||||
from imwatermark import WatermarkEncoder
|
||||
|
||||
from remove_ai_watermarks.invisible_watermark import _BITS_48
|
||||
|
||||
bits = [int(b) for b in format(_BITS_48["Stable Diffusion XL"], "048b")]
|
||||
enc = WatermarkEncoder()
|
||||
enc.set_watermark("bits", bits)
|
||||
img = np.random.default_rng(0).integers(0, 255, (512, 512, 3), dtype=np.uint8)
|
||||
path = tmp_path / "sdxl.png"
|
||||
cv2.imwrite(str(path), enc.encode(img, "dwtDct"))
|
||||
return path
|
||||
|
||||
def test_sdxl_watermark_identified(self, tmp_path: Path):
|
||||
r = identify(self._sdxl_watermarked(tmp_path), check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.confidence == "high"
|
||||
assert r.platform is not None
|
||||
assert "Stable Diffusion XL" in r.platform
|
||||
assert any("invisible watermark" in w.lower() for w in r.watermarks)
|
||||
|
||||
def test_check_invisible_false_skips(self, tmp_path: Path):
|
||||
r = identify(self._sdxl_watermarked(tmp_path), check_visible=False, check_invisible=False)
|
||||
assert not any(s.name == "invisible_watermark" for s in r.signals)
|
||||
|
||||
|
||||
class TestIdentifyAIGC:
|
||||
"""China TC260 AIGC label is detected and attributed (e.g. Doubao)."""
|
||||
|
||||
def _aigc_png(self, tmp_path: Path) -> Path:
|
||||
from PIL import Image
|
||||
|
||||
p = tmp_path / "doubao.png"
|
||||
Image.new("RGB", (32, 32)).save(p)
|
||||
xmp = (
|
||||
'<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF '
|
||||
'xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">'
|
||||
'<rdf:Description xmlns:TC260="http://www.tc260.org.cn/ns/AIGC/1.0/">'
|
||||
"<TC260:AIGC>{"Label":"1","ContentProducer":"BYTEDANCE001"}"
|
||||
"</TC260:AIGC></rdf:Description></rdf:RDF></x:xmpmeta>"
|
||||
)
|
||||
with open(p, "ab") as f:
|
||||
f.write(xmp.encode())
|
||||
return p
|
||||
|
||||
def test_aigc_detected(self, tmp_path: Path):
|
||||
r = identify(self._aigc_png(tmp_path), check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform is not None
|
||||
assert "AIGC" in r.platform or "TC260" in r.platform
|
||||
assert any("AIGC" in w for w in r.watermarks)
|
||||
|
||||
def test_aigc_signal_carries_producer(self, tmp_path: Path):
|
||||
r = identify(self._aigc_png(tmp_path), check_visible=False)
|
||||
sig = next(s for s in r.signals if s.name == "aigc")
|
||||
assert "BYTEDANCE001" in sig.detail
|
||||
@@ -0,0 +1,130 @@
|
||||
"""Unit tests for the MPS->CPU fallback orchestration (no GPU/model required).
|
||||
|
||||
``img2img_runner`` has no torch import at module top -- the pipeline is
|
||||
injected as a plain callable -- so the fallback control flow is fully
|
||||
mockable. This guards the exact behavior hit in production on Apple Silicon:
|
||||
a native-resolution SDXL run that OOMs on MPS must transparently retry on CPU,
|
||||
while any non-MPS error must propagate unchanged.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from remove_ai_watermarks.noai import img2img_runner
|
||||
from remove_ai_watermarks.noai.img2img_runner import run_img2img, run_img2img_with_mps_fallback
|
||||
|
||||
_MPS_OOM = "MPS backend out of memory (MPS allocated: 17.21 GiB, max allowed: 20.13 GiB)"
|
||||
|
||||
|
||||
def _result(image: object) -> Mock:
|
||||
"""A stand-in for a diffusers pipeline output object (has .images)."""
|
||||
out = Mock()
|
||||
out.images = [image]
|
||||
return out
|
||||
|
||||
|
||||
class TestMpsFallback:
|
||||
def test_mps_error_reloads_on_cpu_and_retries(self, monkeypatch: pytest.MonkeyPatch):
|
||||
sentinel = object()
|
||||
inner = Mock(side_effect=[RuntimeError(_MPS_OOM), sentinel])
|
||||
monkeypatch.setattr(img2img_runner, "run_img2img", inner)
|
||||
load_pipeline = Mock(return_value="gpu_pipe")
|
||||
reload_on_cpu = Mock(return_value="cpu_pipe")
|
||||
|
||||
img, device = run_img2img_with_mps_fallback(
|
||||
load_pipeline, object(), 0.05, 50, 7.5, "gen", "mps", lambda _m: None, reload_on_cpu=reload_on_cpu
|
||||
)
|
||||
|
||||
assert (img, device) == (sentinel, "cpu")
|
||||
reload_on_cpu.assert_called_once()
|
||||
assert inner.call_count == 2
|
||||
# Retry must use the reloaded CPU pipeline, device "cpu", and drop the
|
||||
# MPS generator (generator=None) so CPU runs deterministically.
|
||||
retry_args = inner.call_args_list[1].args
|
||||
assert retry_args[0] == "cpu_pipe"
|
||||
assert retry_args[5] is None # generator
|
||||
assert retry_args[6] == "cpu" # device
|
||||
|
||||
def test_happy_path_returns_original_device_without_reload(self, monkeypatch: pytest.MonkeyPatch):
|
||||
sentinel = object()
|
||||
monkeypatch.setattr(img2img_runner, "run_img2img", Mock(return_value=sentinel))
|
||||
reload_on_cpu = Mock()
|
||||
|
||||
img, device = run_img2img_with_mps_fallback(
|
||||
Mock(return_value="gpu_pipe"),
|
||||
object(),
|
||||
0.05,
|
||||
50,
|
||||
7.5,
|
||||
"gen",
|
||||
"mps",
|
||||
lambda _m: None,
|
||||
reload_on_cpu=reload_on_cpu,
|
||||
)
|
||||
|
||||
assert (img, device) == (sentinel, "mps")
|
||||
reload_on_cpu.assert_not_called()
|
||||
|
||||
def test_non_mps_runtime_error_propagates(self, monkeypatch: pytest.MonkeyPatch):
|
||||
monkeypatch.setattr(img2img_runner, "run_img2img", Mock(side_effect=RuntimeError("CUDA out of memory")))
|
||||
reload_on_cpu = Mock()
|
||||
|
||||
with pytest.raises(RuntimeError, match="CUDA"):
|
||||
run_img2img_with_mps_fallback(
|
||||
Mock(return_value="gpu_pipe"),
|
||||
object(),
|
||||
0.05,
|
||||
50,
|
||||
7.5,
|
||||
"gen",
|
||||
"mps",
|
||||
lambda _m: None,
|
||||
reload_on_cpu=reload_on_cpu,
|
||||
)
|
||||
reload_on_cpu.assert_not_called()
|
||||
|
||||
def test_mps_error_on_non_mps_device_propagates(self, monkeypatch: pytest.MonkeyPatch):
|
||||
# An "mps"-worded error while running on cpu must NOT trigger the reload.
|
||||
monkeypatch.setattr(img2img_runner, "run_img2img", Mock(side_effect=RuntimeError(_MPS_OOM)))
|
||||
reload_on_cpu = Mock()
|
||||
|
||||
with pytest.raises(RuntimeError, match="MPS backend"):
|
||||
run_img2img_with_mps_fallback(
|
||||
Mock(return_value="cpu_pipe"),
|
||||
object(),
|
||||
0.05,
|
||||
50,
|
||||
7.5,
|
||||
None,
|
||||
"cpu",
|
||||
lambda _m: None,
|
||||
reload_on_cpu=reload_on_cpu,
|
||||
)
|
||||
reload_on_cpu.assert_not_called()
|
||||
|
||||
|
||||
class TestRunImg2Img:
|
||||
def test_returns_first_image_from_pipeline_result(self):
|
||||
sentinel = object()
|
||||
pipeline = Mock(return_value=_result(sentinel))
|
||||
|
||||
out = run_img2img(pipeline, object(), 0.05, 50, 7.5, None, "cpu", lambda _m: None)
|
||||
|
||||
assert out is sentinel
|
||||
|
||||
def test_typeerror_on_callback_retries_without_callback(self):
|
||||
# Older diffusers reject the progress callback kwarg with TypeError;
|
||||
# run_img2img must retry once without it rather than fail.
|
||||
sentinel = object()
|
||||
pipeline = Mock(side_effect=[TypeError("unexpected keyword 'callback'"), _result(sentinel)])
|
||||
|
||||
out = run_img2img(pipeline, object(), 0.05, 50, 7.5, None, "cpu", lambda _m: None)
|
||||
|
||||
assert out is sentinel
|
||||
assert pipeline.call_count == 2
|
||||
# First attempt passes the progress callback; the retry omits it.
|
||||
assert "callback" in pipeline.call_args_list[0].kwargs
|
||||
assert "callback" not in pipeline.call_args_list[1].kwargs
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine, is_available
|
||||
from remove_ai_watermarks.invisible_engine import InvisibleEngine, _target_size, is_available
|
||||
|
||||
|
||||
class TestIsAvailable:
|
||||
@@ -26,3 +26,45 @@ class TestInvisibleEngineInit:
|
||||
|
||||
def test_ctrlregen_model_id(self):
|
||||
assert InvisibleEngine.CTRLREGEN_MODEL_ID == "yepengliu/ctrlregen"
|
||||
|
||||
|
||||
class TestTargetSize:
|
||||
"""Regression guard for the native-resolution decision (issues #10 / #15).
|
||||
|
||||
max_resolution=0 must NOT downscale -- the forced downscale->upscale
|
||||
round-trip was the quality loss in #10, and downscaling at all let SynthID
|
||||
survive in #15 (the native SDXL pass at strength ~0.05 is what defeats it).
|
||||
"""
|
||||
|
||||
def test_native_default_no_downscale(self):
|
||||
# The default (0) means native resolution: no resize, regardless of size.
|
||||
assert _target_size(4096, 4096, 0) is None
|
||||
assert _target_size(123, 456, 0) is None
|
||||
|
||||
def test_negative_cap_treated_as_native(self):
|
||||
assert _target_size(4096, 4096, -1) is None
|
||||
|
||||
def test_cap_below_long_side_downscales(self):
|
||||
# 2000x1000, cap 1024 -> long side scaled to 1024, aspect preserved.
|
||||
assert _target_size(2000, 1000, 1024) == (1024, 512)
|
||||
|
||||
def test_cap_uses_long_side_for_portrait(self):
|
||||
# Portrait: height is the long side, so it drives the ratio.
|
||||
assert _target_size(1000, 2000, 1024) == (512, 1024)
|
||||
|
||||
def test_cap_at_or_above_long_side_no_downscale(self):
|
||||
# Already within the cap (and exactly equal) -> no resize.
|
||||
assert _target_size(800, 600, 1024) is None
|
||||
assert _target_size(1024, 768, 1024) is None
|
||||
|
||||
def test_integer_truncation_matches_pil_call_site(self):
|
||||
# 1254x1254 (the gpt-image sample) capped at 1000: int(1254*1000/1254)=1000.
|
||||
assert _target_size(1254, 1254, 1000) == (1000, 1000)
|
||||
# Non-divisible ratio truncates toward zero like int() at the call site.
|
||||
assert _target_size(1000, 333, 500) == (500, 166)
|
||||
|
||||
def test_extreme_aspect_ratio_clamps_short_side_to_one(self):
|
||||
# 5000x3 capped at 1024: int(3 * 1024/5000) = 0 would crash resize();
|
||||
# the short side must clamp to 1, never 0.
|
||||
assert _target_size(5000, 3, 1024) == (1024, 1)
|
||||
assert _target_size(3, 5000, 1024) == (1, 1024)
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Tests for open invisible-watermark (imwatermark) detection.
|
||||
|
||||
Each known scheme is round-tripped: embed its exact upstream pattern with the
|
||||
encoder, then assert the detector names it. Skipped entirely if the optional
|
||||
``invisible-watermark`` package is not installed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
from remove_ai_watermarks.invisible_watermark import (
|
||||
_BITS_48,
|
||||
_SD1_STRING,
|
||||
_bits_match,
|
||||
_bytes_match_frac,
|
||||
detect_invisible_watermark,
|
||||
is_available,
|
||||
)
|
||||
|
||||
pytestmark = pytest.mark.skipif(not is_available(), reason="invisible-watermark not installed")
|
||||
|
||||
|
||||
def _base_image() -> np.ndarray:
|
||||
# imwatermark needs enough DWT coefficients; use a 512x512 textured image.
|
||||
rng = np.random.default_rng(0)
|
||||
return rng.integers(0, 255, (512, 512, 3), dtype=np.uint8)
|
||||
|
||||
|
||||
def _write_bits_watermark(tmp_path: Path, message: int) -> Path:
|
||||
from imwatermark import WatermarkEncoder
|
||||
|
||||
bits = [int(b) for b in format(message, "048b")]
|
||||
enc = WatermarkEncoder()
|
||||
enc.set_watermark("bits", bits)
|
||||
wm = enc.encode(_base_image(), "dwtDct")
|
||||
path = tmp_path / "wm.png"
|
||||
cv2.imwrite(str(path), wm)
|
||||
return path
|
||||
|
||||
|
||||
class TestHelpers:
|
||||
def test_bits_match_exact(self):
|
||||
assert _bits_match(0b1010, 0b1010, width=4) == 4
|
||||
|
||||
def test_bits_match_one_off(self):
|
||||
assert _bits_match(0b1010, 0b1011, width=4) == 3
|
||||
|
||||
def test_bytes_match_identical(self):
|
||||
assert _bytes_match_frac(_SD1_STRING, _SD1_STRING) == 1.0
|
||||
|
||||
def test_bytes_match_length_mismatch_is_zero(self):
|
||||
assert _bytes_match_frac(b"abc", b"abcd") == 0.0
|
||||
|
||||
|
||||
class TestDetect:
|
||||
def test_detects_sdxl(self, tmp_path: Path):
|
||||
path = _write_bits_watermark(tmp_path, _BITS_48["Stable Diffusion XL"])
|
||||
assert detect_invisible_watermark(path) == "Stable Diffusion XL"
|
||||
|
||||
def test_detects_flux(self, tmp_path: Path):
|
||||
path = _write_bits_watermark(tmp_path, _BITS_48["FLUX.2 (Black Forest Labs)"])
|
||||
assert detect_invisible_watermark(path) == "FLUX.2 (Black Forest Labs)"
|
||||
|
||||
def test_detects_sd1_string(self, tmp_path: Path):
|
||||
from imwatermark import WatermarkEncoder
|
||||
|
||||
enc = WatermarkEncoder()
|
||||
enc.set_watermark("bytes", _SD1_STRING)
|
||||
wm = enc.encode(_base_image(), "dwtDct")
|
||||
path = tmp_path / "sd1.png"
|
||||
cv2.imwrite(str(path), wm)
|
||||
assert detect_invisible_watermark(path) == "Stable Diffusion 1.x / 2.x"
|
||||
|
||||
def test_clean_image_is_none(self, tmp_path: Path):
|
||||
path = tmp_path / "clean.png"
|
||||
cv2.imwrite(str(path), _base_image())
|
||||
assert detect_invisible_watermark(path) is None
|
||||
|
||||
def test_unreadable_file_is_none(self, tmp_path: Path):
|
||||
path = tmp_path / "not_image.png"
|
||||
path.write_bytes(b"not a png")
|
||||
assert detect_invisible_watermark(path) is None
|
||||
@@ -4,16 +4,24 @@ from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import piexif
|
||||
import pytest
|
||||
from PIL import Image
|
||||
from PIL.PngImagePlugin import PngInfo
|
||||
|
||||
from remove_ai_watermarks.metadata import (
|
||||
_is_ai_key,
|
||||
exif_generator,
|
||||
get_ai_metadata,
|
||||
has_ai_metadata,
|
||||
remove_ai_metadata,
|
||||
synthid_source,
|
||||
xai_signature,
|
||||
)
|
||||
|
||||
# Real, committed C2PA sample images used to ground the SynthID-source tests.
|
||||
SAMPLES_DIR = Path(__file__).resolve().parent.parent / "data" / "samples"
|
||||
|
||||
# ── Key detection ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -144,6 +152,116 @@ class TestGetAiMetadata:
|
||||
meta = get_ai_metadata(tmp_clean_png)
|
||||
assert meta == {}
|
||||
|
||||
def test_long_value_is_truncated(self, tmp_path: Path):
|
||||
img = Image.new("RGB", (32, 32))
|
||||
pnginfo = PngInfo()
|
||||
pnginfo.add_text("parameters", "x" * 300)
|
||||
path = tmp_path / "long.png"
|
||||
img.save(path, pnginfo=pnginfo)
|
||||
meta = get_ai_metadata(path)
|
||||
assert meta["parameters"].endswith("…")
|
||||
assert len(meta["parameters"]) <= 205
|
||||
|
||||
def test_unopenable_file_does_not_raise(self, tmp_path: Path):
|
||||
# PIL can't open HEIC without pillow-heif; get_ai_metadata must fall
|
||||
# through to the binary scan, not propagate UnidentifiedImageError.
|
||||
path = tmp_path / "iphone.heic"
|
||||
path.write_bytes(b"\x00\x00\x00\x18ftypheic" + b"\x00" * 64)
|
||||
assert get_ai_metadata(path) == {}
|
||||
|
||||
|
||||
@pytest.mark.skipif(not SAMPLES_DIR.exists(), reason="data/samples not present")
|
||||
class TestGetAiMetadataRealSample:
|
||||
"""get_ai_metadata surfaces the consolidated C2PA fields on real images."""
|
||||
|
||||
def test_openai_sample_fields(self):
|
||||
meta = get_ai_metadata(SAMPLES_DIR / "chatgpt-1.png")
|
||||
assert "claim_generator" in meta
|
||||
assert "OpenAI" in meta["issuer"]
|
||||
assert "OpenAI" in meta["synthid_watermark"]
|
||||
assert "trainedAlgorithmicMedia" in meta["source_type"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"marker",
|
||||
[
|
||||
b"trainedAlgorithmicMedia",
|
||||
b"compositeSynthetic",
|
||||
b"algorithmicMedia",
|
||||
b"compositeWithTrainedAlgorithmicMedia",
|
||||
],
|
||||
)
|
||||
def test_has_ai_metadata_detects_each_iptc_marker(tmp_path: Path, marker: bytes):
|
||||
"""Each IPTC digitalSourceType AI marker in XMP triggers detection."""
|
||||
path = tmp_path / "iptc.jpg"
|
||||
path.write_bytes(b"\xff\xd8\xff\xe1<x:xmpmeta>" + marker + b"</x:xmpmeta>\xff\xd9")
|
||||
assert has_ai_metadata(path)
|
||||
|
||||
|
||||
# ── SynthID-source detection (metadata proxy) ────────────────────────
|
||||
|
||||
|
||||
@pytest.mark.skipif(not SAMPLES_DIR.exists(), reason="data/samples not present")
|
||||
class TestSynthIDSource:
|
||||
"""SynthID detection via the C2PA companion manifest.
|
||||
|
||||
Google (Imagen/Gemini) and OpenAI (ChatGPT/DALL-E/gpt-image) pair an
|
||||
invisible SynthID pixel watermark with a C2PA manifest. Adobe Firefly and
|
||||
Microsoft Designer sign C2PA Content Credentials but do NOT use SynthID,
|
||||
so the discriminating signal is the C2PA *issuer*, not the mere presence
|
||||
of a manifest. These tests run against real, committed sample images.
|
||||
"""
|
||||
|
||||
def test_openai_chatgpt_is_synthid_source(self):
|
||||
assert synthid_source(SAMPLES_DIR / "chatgpt-1.png") == "OpenAI"
|
||||
|
||||
def test_openai_verdict_in_get_ai_metadata(self):
|
||||
meta = get_ai_metadata(SAMPLES_DIR / "chatgpt-1.png")
|
||||
assert "synthid_watermark" in meta
|
||||
assert "OpenAI" in meta["synthid_watermark"]
|
||||
|
||||
def test_adobe_firefly_is_not_synthid_source(self):
|
||||
# Adobe signs C2PA (trainedAlgorithmicMedia) but embeds no SynthID.
|
||||
assert synthid_source(SAMPLES_DIR / "firefly-1.png") is None
|
||||
assert "synthid_watermark" not in get_ai_metadata(SAMPLES_DIR / "firefly-1.png")
|
||||
|
||||
def test_non_ai_image_is_not_synthid_source(self, clean_photo: Path):
|
||||
assert synthid_source(clean_photo) is None
|
||||
|
||||
|
||||
class TestSynthIDSourceNonPng:
|
||||
"""SynthID-source detection must work beyond PNG.
|
||||
|
||||
ChatGPT/Gemini images saved as JPEG/WebP/AVIF carry their C2PA manifest in
|
||||
a non-PNG container (JPEG APP11, ISOBMFF uuid box), so the PNG caBX parser
|
||||
misses them. These use synthetic byte blobs (real fixtures aren't shipped).
|
||||
"""
|
||||
|
||||
def _c2pa_jpeg(self, tmp_path: Path, name: str, issuer: bytes, marker: bytes = b"trainedAlgorithmicMedia") -> Path:
|
||||
path = tmp_path / name
|
||||
# Minimal JPEG shell with an embedded C2PA-ish blob.
|
||||
blob = b"jumbc2pa" + issuer + b"..." + marker
|
||||
path.write_bytes(b"\xff\xd8\xff\xe1" + blob + b"\xff\xd9")
|
||||
return path
|
||||
|
||||
def test_openai_c2pa_in_jpeg(self, tmp_path: Path):
|
||||
path = self._c2pa_jpeg(tmp_path, "chatgpt.jpg", b"OpenAI")
|
||||
assert synthid_source(path) == "OpenAI"
|
||||
|
||||
def test_google_c2pa_in_jpeg(self, tmp_path: Path):
|
||||
path = self._c2pa_jpeg(tmp_path, "gemini.jpg", b"Google")
|
||||
assert synthid_source(path) == "Google LLC"
|
||||
|
||||
def test_adobe_c2pa_in_jpeg_is_none(self, tmp_path: Path):
|
||||
# Adobe signs C2PA but embeds no SynthID.
|
||||
path = self._c2pa_jpeg(tmp_path, "firefly.jpg", b"Adobe")
|
||||
assert synthid_source(path) is None
|
||||
|
||||
def test_openai_without_ai_marker_is_none(self, tmp_path: Path):
|
||||
# Issuer present but no AI digital-source marker -> not a SynthID source.
|
||||
path = self._c2pa_jpeg(tmp_path, "edited.jpg", b"OpenAI", marker=b"")
|
||||
assert synthid_source(path) is None
|
||||
|
||||
|
||||
# ── remove_ai_metadata ──────────────────────────────────────────────
|
||||
|
||||
@@ -217,3 +335,197 @@ class TestRemoveAiMetadata:
|
||||
result = remove_ai_metadata(tmp_clean_png, output)
|
||||
assert isinstance(result, Path)
|
||||
assert result == output
|
||||
|
||||
|
||||
def _img_with_software(tmp_path: Path, fmt: str, software: str) -> Path:
|
||||
"""Write a tiny image carrying an EXIF Software tag."""
|
||||
exif = piexif.dump({"0th": {piexif.ImageIFD.Software: software.encode()}, "Exif": {}, "GPS": {}, "1st": {}})
|
||||
path = tmp_path / f"img.{fmt}"
|
||||
Image.new("RGB", (64, 64), (100, 90, 80)).save(path, exif=exif)
|
||||
return path
|
||||
|
||||
|
||||
class TestExifGenerator:
|
||||
"""exif_generator extracts AI-tool names from EXIF/XMP across formats."""
|
||||
|
||||
def test_avif_software_ai_tool_detected(self, tmp_path: Path):
|
||||
path = _img_with_software(tmp_path, "avif", "Adobe Firefly")
|
||||
assert exif_generator(path) == "Adobe Firefly"
|
||||
|
||||
def test_jpeg_software_ai_tool_detected(self, tmp_path: Path):
|
||||
path = _img_with_software(tmp_path, "jpg", "ComfyUI v1.2")
|
||||
result = exif_generator(path)
|
||||
assert result is not None
|
||||
assert "ComfyUI" in result
|
||||
|
||||
def test_plain_editor_not_flagged(self, tmp_path: Path):
|
||||
# An ordinary editor tag carries no AI token and must not be flagged.
|
||||
path = _img_with_software(tmp_path, "jpg", "Adobe Photoshop 25.0")
|
||||
assert exif_generator(path) is None
|
||||
|
||||
def test_make_tag_ai_tool_detected(self, tmp_path: Path):
|
||||
# Ideogram tags its output with EXIF Make="Ideogram AI" (verified on a
|
||||
# real download), so the Make tag must be read too.
|
||||
exif = piexif.dump({"0th": {piexif.ImageIFD.Make: b"Ideogram AI"}, "Exif": {}, "GPS": {}, "1st": {}})
|
||||
path = tmp_path / "ideogram.jpg"
|
||||
Image.new("RGB", (64, 64)).save(path, exif=exif)
|
||||
assert exif_generator(path) == "Ideogram AI"
|
||||
|
||||
def test_camera_make_not_flagged(self, tmp_path: Path):
|
||||
# A real camera Make ("Apple") carries no AI token -> not flagged.
|
||||
exif = piexif.dump({"0th": {piexif.ImageIFD.Make: b"Apple"}, "Exif": {}, "GPS": {}, "1st": {}})
|
||||
path = tmp_path / "iphone.jpg"
|
||||
Image.new("RGB", (64, 64)).save(path, exif=exif)
|
||||
assert exif_generator(path) is None
|
||||
|
||||
def test_artist_tag_ai_tool_detected(self, tmp_path: Path):
|
||||
# exif_generator also reads the EXIF Artist field for an AI token.
|
||||
exif = piexif.dump({"0th": {piexif.ImageIFD.Artist: b"Midjourney"}, "Exif": {}, "GPS": {}, "1st": {}})
|
||||
path = tmp_path / "artist.jpg"
|
||||
Image.new("RGB", (64, 64)).save(path, exif=exif)
|
||||
assert exif_generator(path) == "Midjourney"
|
||||
|
||||
def test_imagedescription_tag_ai_tool_detected(self, tmp_path: Path):
|
||||
# ...and the EXIF ImageDescription field.
|
||||
exif = piexif.dump(
|
||||
{"0th": {piexif.ImageIFD.ImageDescription: b"Made with Stable Diffusion"}, "Exif": {}, "GPS": {}, "1st": {}}
|
||||
)
|
||||
path = tmp_path / "desc.jpg"
|
||||
Image.new("RGB", (64, 64)).save(path, exif=exif)
|
||||
result = exif_generator(path)
|
||||
assert result is not None
|
||||
assert "Stable Diffusion" in result
|
||||
|
||||
def test_xmp_creatortool_scan_covers_unopenable(self, tmp_path: Path):
|
||||
# PIL can't open this fake HEIF; the raw XMP CreatorTool scan still works.
|
||||
path = tmp_path / "fake.heic"
|
||||
path.write_bytes(
|
||||
b"\x00\x00\x00\x18ftypheic\x00\x00\x00\x00"
|
||||
b"<x:xmpmeta><xmp:CreatorTool>Midjourney v7</xmp:CreatorTool></x:xmpmeta>"
|
||||
)
|
||||
result = exif_generator(path)
|
||||
assert result is not None
|
||||
assert "Midjourney" in result
|
||||
|
||||
def test_clean_image_is_none(self, tmp_clean_png: Path):
|
||||
assert exif_generator(tmp_clean_png) is None
|
||||
|
||||
|
||||
_FAKE_SIG = "A" * 120 # 64+ base64 chars; real Grok payloads are 300-1004
|
||||
_FAKE_UUID = "12345678-1234-1234-1234-123456789abc"
|
||||
|
||||
|
||||
def _grok_jpeg(tmp_path: Path, *, signature: str = _FAKE_SIG, artist: str = _FAKE_UUID) -> Path:
|
||||
"""Write a synthetic Grok-style JPEG: EXIF ImageDescription "Signature: ..."
|
||||
+ a UUID Artist. Synthetic on purpose -- never commit a real Grok image
|
||||
(its Artist UUID + signature are user/session data; this is a public repo)."""
|
||||
exif = piexif.dump(
|
||||
{
|
||||
"0th": {
|
||||
piexif.ImageIFD.ImageDescription: f"Signature: {signature}".encode("latin1"),
|
||||
piexif.ImageIFD.Artist: artist.encode("latin1"),
|
||||
},
|
||||
"Exif": {},
|
||||
"GPS": {},
|
||||
"1st": {},
|
||||
}
|
||||
)
|
||||
path = tmp_path / "grok.jpg"
|
||||
Image.new("RGB", (64, 64), (70, 80, 90)).save(path, exif=exif)
|
||||
return path
|
||||
|
||||
|
||||
class TestXaiSignature:
|
||||
"""xAI / Grok's EXIF Signature + UUID-Artist provenance scheme."""
|
||||
|
||||
def test_signature_plus_uuid_detected(self, tmp_path: Path):
|
||||
assert xai_signature(_grok_jpeg(tmp_path)) is True
|
||||
|
||||
def test_real_grok_sample_detected(self):
|
||||
# Real committed Grok download (data/samples/grok-1.jpg); the EXIF
|
||||
# Signature + UUID-Artist pair is the only AI signal it carries.
|
||||
assert xai_signature(SAMPLES_DIR / "grok-1.jpg") is True
|
||||
|
||||
def test_signature_without_uuid_artist_not_flagged(self, tmp_path: Path):
|
||||
# A "Signature:" blob but a non-UUID Artist is not the Grok pair.
|
||||
assert xai_signature(_grok_jpeg(tmp_path, artist="John Doe")) is False
|
||||
|
||||
def test_bare_uuid_artist_not_flagged(self, tmp_path: Path):
|
||||
# A UUID Artist alone (no Signature blob) must not false-positive.
|
||||
exif = piexif.dump({"0th": {piexif.ImageIFD.Artist: _FAKE_UUID.encode()}, "Exif": {}, "GPS": {}, "1st": {}})
|
||||
path = tmp_path / "uuid_only.jpg"
|
||||
Image.new("RGB", (64, 64)).save(path, exif=exif)
|
||||
assert xai_signature(path) is False
|
||||
|
||||
def test_short_signature_text_not_flagged(self, tmp_path: Path):
|
||||
# Incidental short "Signature: ..." text is below the 64-char base64 bar.
|
||||
assert xai_signature(_grok_jpeg(tmp_path, signature="ok")) is False
|
||||
|
||||
def test_clean_image_is_false(self, tmp_clean_png: Path):
|
||||
assert xai_signature(tmp_clean_png) is False
|
||||
|
||||
def test_surfaced_in_get_ai_metadata(self, tmp_path: Path):
|
||||
assert "xai_signature" in get_ai_metadata(_grok_jpeg(tmp_path))
|
||||
|
||||
def test_has_ai_metadata_true(self, tmp_path: Path):
|
||||
assert has_ai_metadata(_grok_jpeg(tmp_path)) is True
|
||||
|
||||
|
||||
class TestAIGCLabel:
|
||||
"""China TC260 AIGC labeling (Doubao and other China-served generators)."""
|
||||
|
||||
def _aigc_png(self, tmp_path: Path, label: str = "1", producer: str = "TESTPRODUCER001") -> Path:
|
||||
from remove_ai_watermarks.metadata import aigc_label # noqa: F401 (import-time guard)
|
||||
|
||||
p = tmp_path / "doubao.png"
|
||||
Image.new("RGB", (32, 32)).save(p)
|
||||
# XMP is HTML-entity encoded in real files; aigc_label must unescape it.
|
||||
xmp = (
|
||||
'<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF '
|
||||
'xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">'
|
||||
'<rdf:Description rdf:about="" '
|
||||
'xmlns:TC260="http://www.tc260.org.cn/ns/AIGC/1.0/"><TC260:AIGC>'
|
||||
f"{{"Label":"{label}","ContentProducer":"{producer}"}}"
|
||||
"</TC260:AIGC></rdf:Description></rdf:RDF></x:xmpmeta>"
|
||||
)
|
||||
with open(p, "ab") as f:
|
||||
f.write(xmp.encode())
|
||||
return p
|
||||
|
||||
def test_parses_label_and_producer(self, tmp_path: Path):
|
||||
from remove_ai_watermarks.metadata import aigc_label
|
||||
|
||||
info = aigc_label(self._aigc_png(tmp_path))
|
||||
assert info is not None
|
||||
assert info["Label"] == "1"
|
||||
assert info["ContentProducer"] == "TESTPRODUCER001"
|
||||
|
||||
def test_none_when_absent(self, tmp_clean_png):
|
||||
from remove_ai_watermarks.metadata import aigc_label
|
||||
|
||||
assert aigc_label(tmp_clean_png) is None
|
||||
|
||||
def test_has_ai_metadata_detects_aigc(self, tmp_path: Path):
|
||||
assert has_ai_metadata(self._aigc_png(tmp_path))
|
||||
|
||||
def test_get_ai_metadata_surfaces_aigc(self, tmp_path: Path):
|
||||
meta = get_ai_metadata(self._aigc_png(tmp_path))
|
||||
assert "aigc_label" in meta
|
||||
assert "TC260" in meta["aigc_label"]
|
||||
|
||||
|
||||
@pytest.mark.skipif(not (SAMPLES_DIR / "doubao-1.png").exists(), reason="doubao sample not present")
|
||||
class TestAIGCRealSample:
|
||||
"""Real Doubao (ByteDance) sample carries the China TC260 AIGC XMP label."""
|
||||
|
||||
def test_doubao_aigc_label(self):
|
||||
from remove_ai_watermarks.metadata import aigc_label
|
||||
|
||||
info = aigc_label(SAMPLES_DIR / "doubao-1.png")
|
||||
assert info is not None
|
||||
assert info["Label"] == "1"
|
||||
assert info["ContentProducer"] # ByteDance producer code present
|
||||
|
||||
def test_doubao_detected_as_ai(self):
|
||||
assert has_ai_metadata(SAMPLES_DIR / "doubao-1.png")
|
||||
assert "aigc_label" in get_ai_metadata(SAMPLES_DIR / "doubao-1.png")
|
||||
@@ -2,10 +2,19 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import struct
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from remove_ai_watermarks.noai.c2pa import (
|
||||
_cbor_text_after,
|
||||
_parse_c2pa_chunk,
|
||||
extract_c2pa_chunk,
|
||||
extract_c2pa_info,
|
||||
has_c2pa_metadata,
|
||||
inject_c2pa_chunk,
|
||||
synthid_verdict,
|
||||
)
|
||||
from remove_ai_watermarks.noai.cleaner import (
|
||||
has_ai_content,
|
||||
@@ -26,6 +35,10 @@ from remove_ai_watermarks.noai.extractor import (
|
||||
get_ai_metadata_summary,
|
||||
has_ai_metadata,
|
||||
)
|
||||
from remove_ai_watermarks.noai.isobmff import (
|
||||
is_isobmff,
|
||||
strip_c2pa_boxes,
|
||||
)
|
||||
|
||||
# ── Constants ───────────────────────────────────────────────────────
|
||||
|
||||
@@ -128,3 +141,207 @@ class TestC2PA:
|
||||
|
||||
def test_c2pa_returns_false_for_non_png(self, tmp_jpeg_path):
|
||||
assert not has_c2pa_metadata(tmp_jpeg_path)
|
||||
|
||||
|
||||
SAMPLES_DIR = Path(__file__).resolve().parent.parent / "data" / "samples"
|
||||
|
||||
|
||||
@pytest.mark.skipif(not SAMPLES_DIR.exists(), reason="data/samples not present")
|
||||
class TestC2PARealSamples:
|
||||
"""Parser behavior on real committed C2PA images."""
|
||||
|
||||
def test_detects_c2pa_in_openai_png(self):
|
||||
assert has_c2pa_metadata(SAMPLES_DIR / "chatgpt-1.png")
|
||||
|
||||
def test_extract_info_openai_fields(self):
|
||||
info = extract_c2pa_info(SAMPLES_DIR / "chatgpt-1.png")
|
||||
assert info["has_c2pa"] is True
|
||||
assert "OpenAI" in info["issuer"]
|
||||
assert "c2pa_manifest" in info # "C2PA manifest (N bytes)"
|
||||
assert "trainedAlgorithmicMedia" in info["source_type"]
|
||||
# CBOR-clean claim generator, no regex artifacts (e.g. "fGPT-4o").
|
||||
assert info["claim_generator"]
|
||||
assert not info["claim_generator"].startswith("f")
|
||||
assert "synthid_watermark" in info
|
||||
|
||||
def test_extract_info_adobe_has_no_synthid(self):
|
||||
info = extract_c2pa_info(SAMPLES_DIR / "firefly-1.png")
|
||||
assert "Adobe" in info["issuer"]
|
||||
assert "synthid_watermark" not in info
|
||||
|
||||
def test_extract_chunk_returns_bytes(self):
|
||||
chunk = extract_c2pa_chunk(SAMPLES_DIR / "chatgpt-1.png")
|
||||
assert chunk is not None
|
||||
assert chunk[4:8] == b"caBX" # chunk type in the 8-byte header
|
||||
|
||||
def test_inject_round_trip(self, tmp_clean_png, tmp_path):
|
||||
"""Extract a real C2PA chunk, inject into a clean PNG, re-detect."""
|
||||
chunk = extract_c2pa_chunk(SAMPLES_DIR / "chatgpt-1.png")
|
||||
out = tmp_path / "injected.png"
|
||||
inject_c2pa_chunk(tmp_clean_png, out, chunk)
|
||||
assert has_c2pa_metadata(out)
|
||||
assert "OpenAI" in extract_c2pa_info(out)["issuer"]
|
||||
|
||||
|
||||
class TestC2PAInjectValidation:
|
||||
def test_inject_rejects_non_png(self, tmp_path):
|
||||
with pytest.raises(ValueError, match="only supported for PNG"):
|
||||
inject_c2pa_chunk(tmp_path / "in.jpg", tmp_path / "out.png", b"")
|
||||
|
||||
|
||||
# ── CBOR text extraction (parser internals) ─────────────────────────
|
||||
|
||||
|
||||
class TestCborTextAfter:
|
||||
"""_cbor_text_after handles the three CBOR text-string length prefixes."""
|
||||
|
||||
def test_direct_length(self):
|
||||
# major-type 3, direct length (0x60 + len). "abc" -> 0x63.
|
||||
payload = b"name" + bytes([0x63]) + b"abc"
|
||||
assert _cbor_text_after(payload, b"name") == "abc"
|
||||
|
||||
def test_one_byte_length(self):
|
||||
s = b"x" * 30
|
||||
payload = b"name" + bytes([0x78, 30]) + s
|
||||
assert _cbor_text_after(payload, b"name") == "x" * 30
|
||||
|
||||
def test_two_byte_length(self):
|
||||
s = b"y" * 300
|
||||
payload = b"name" + bytes([0x79]) + struct.pack(">H", 300) + s
|
||||
assert _cbor_text_after(payload, b"name") == "y" * 300
|
||||
|
||||
def test_key_not_found_returns_none(self):
|
||||
assert _cbor_text_after(b"nothing here", b"name") is None
|
||||
|
||||
def test_key_at_end_returns_none(self):
|
||||
assert _cbor_text_after(b"prefixname", b"name") is None
|
||||
|
||||
def test_invalid_head_returns_none(self):
|
||||
# 0x00 is not a text-string head.
|
||||
assert _cbor_text_after(b"name" + bytes([0x00]) + b"abc", b"name") is None
|
||||
|
||||
def test_latin1_fallback_on_invalid_utf8(self):
|
||||
payload = b"name" + bytes([0x61]) + b"\xff" # len 1, invalid utf-8
|
||||
assert _cbor_text_after(payload, b"name") is not None
|
||||
|
||||
|
||||
class TestSynthIDVerdict:
|
||||
def test_format(self):
|
||||
assert synthid_verdict("OpenAI") == "likely present (OpenAI embeds SynthID with C2PA)"
|
||||
|
||||
def test_multiple_vendors(self):
|
||||
assert "Google LLC, OpenAI" in synthid_verdict("Google LLC, OpenAI")
|
||||
|
||||
|
||||
class TestParseChunkGuards:
|
||||
"""_parse_c2pa_chunk rejects non-printable claim_generator garbage.
|
||||
|
||||
On some manifests (observed: Microsoft Designer) the first ``name`` key
|
||||
precedes a binary hash field, not the generator string. The clean issuer +
|
||||
SynthID verdict must still come through.
|
||||
"""
|
||||
|
||||
def test_clean_generator_kept(self):
|
||||
# "name" + CBOR text-string (head 0x69 = 0x60+9) "gpt-image"
|
||||
chunk = b"...name" + bytes([0x69]) + b"gpt-image" + b"OpenAI trainedAlgorithmicMedia"
|
||||
info: dict = {}
|
||||
_parse_c2pa_chunk(chunk, info)
|
||||
assert info["claim_generator"] == "gpt-image"
|
||||
assert "OpenAI" in info["issuer"]
|
||||
assert "synthid_watermark" in info # OpenAI + trainedAlgorithmicMedia
|
||||
|
||||
def test_nonprintable_generator_dropped(self):
|
||||
# "name" + CBOR string (head 0x64 = len 4) with a control byte -> garbage
|
||||
chunk = b"...name" + bytes([0x64]) + b"\x81abc" + b"OpenAI trainedAlgorithmicMedia"
|
||||
info: dict = {}
|
||||
_parse_c2pa_chunk(chunk, info)
|
||||
assert "claim_generator" not in info # control-char garbage rejected
|
||||
assert "OpenAI" in info["issuer"] # issuer byte-search still robust
|
||||
|
||||
|
||||
class TestC2PADigitalSourceType:
|
||||
"""The three IPTC digitalSourceType variants drive the AI verdict.
|
||||
|
||||
Only *trained* and *composite-with-trained* mean AI-generated (and so imply
|
||||
a SynthID proxy for a SynthID vendor); plain ``algorithmicMedia`` is
|
||||
procedural (not trained) and must NOT be flagged as AI.
|
||||
"""
|
||||
|
||||
def test_plain_algorithmic_media_not_flagged_ai(self):
|
||||
chunk = b"...name" + bytes([0x69]) + b"some-tool" + b" OpenAI algorithmicMedia"
|
||||
info: dict = {}
|
||||
_parse_c2pa_chunk(chunk, info)
|
||||
assert info["source_type"] == "algorithmicMedia"
|
||||
assert "synthid_watermark" not in info # procedural, not AI-generated
|
||||
|
||||
def test_composite_with_trained_is_ai_and_synthid(self):
|
||||
chunk = b"...name" + bytes([0x69]) + b"some-tool" + b" OpenAI compositeWithTrainedAlgorithmicMedia"
|
||||
info: dict = {}
|
||||
_parse_c2pa_chunk(chunk, info)
|
||||
assert "compositeWithTrainedAlgorithmicMedia" in info["source_type"]
|
||||
assert "synthid_watermark" in info # AI-enhanced + OpenAI issuer
|
||||
|
||||
|
||||
# ── ISOBMFF (AVIF / HEIF / JPEG-XL container stripping) ──────────────
|
||||
|
||||
FTYP = b"\x00\x00\x00\x18ftypavif\x00\x00\x00\x00avifmif1" # 24-byte ftyp box
|
||||
|
||||
|
||||
class TestISOBMFF:
|
||||
def test_is_isobmff_true(self):
|
||||
assert is_isobmff(FTYP)
|
||||
|
||||
def test_is_isobmff_false_for_png(self):
|
||||
assert not is_isobmff(b"\x89PNG\r\n\x1a\n\x00\x00")
|
||||
|
||||
def test_is_isobmff_false_for_short(self):
|
||||
assert not is_isobmff(b"abc")
|
||||
|
||||
def test_strips_jpegxl_jumb_box(self):
|
||||
"""JPEG-XL stores JUMBF in a ``jumb`` box, always stripped."""
|
||||
jumb = struct.pack(">I", 8 + 5) + b"jumb" + b"hello"
|
||||
cleaned, stripped = strip_c2pa_boxes(FTYP + jumb)
|
||||
assert stripped == 1
|
||||
assert cleaned == FTYP
|
||||
|
||||
def test_keeps_non_c2pa_box_with_64bit_size(self):
|
||||
"""size==1 means a 64-bit largesize follows; non-C2PA box is kept."""
|
||||
payload = b"\x00" * 8
|
||||
box = b"\x00\x00\x00\x01" + b"free" + struct.pack(">Q", 16 + len(payload)) + payload
|
||||
cleaned, stripped = strip_c2pa_boxes(FTYP + box)
|
||||
assert stripped == 0
|
||||
assert cleaned == FTYP + box
|
||||
|
||||
def test_malformed_box_does_not_crash(self):
|
||||
# A box claiming size 4 (< 8-byte header) must terminate iteration safely.
|
||||
cleaned, stripped = strip_c2pa_boxes(FTYP + b"\x00\x00\x00\x04XXXX")
|
||||
assert stripped == 0
|
||||
assert cleaned.startswith(FTYP)
|
||||
|
||||
def test_size_zero_box_runs_to_eof(self):
|
||||
# size32==0 means the box extends to EOF; a non-C2PA box round-trips.
|
||||
box = struct.pack(">I", 0) + b"free" + b"\x00\x00\x00\x00"
|
||||
cleaned, stripped = strip_c2pa_boxes(FTYP + box)
|
||||
assert stripped == 0
|
||||
assert cleaned == FTYP + box
|
||||
|
||||
def test_truncated_largesize_terminates_safely(self):
|
||||
# size32==1 promises a 64-bit largesize, but the box ends after 8 bytes;
|
||||
# iteration must stop rather than read the missing largesize past EOF.
|
||||
cleaned, stripped = strip_c2pa_boxes(FTYP + b"\x00\x00\x00\x01uuid")
|
||||
assert stripped == 0
|
||||
assert cleaned == FTYP
|
||||
|
||||
|
||||
class TestC2PAInvalidSignature:
|
||||
"""A .png file that is not actually PNG-signed must read as clean, not crash."""
|
||||
|
||||
def test_has_c2pa_false_for_non_png_bytes(self, tmp_path: Path):
|
||||
fake = tmp_path / "fake.png"
|
||||
fake.write_bytes(b"\xff\xd8\xff\xe0 not a png at all, just garbage bytes")
|
||||
assert has_c2pa_metadata(fake) is False
|
||||
|
||||
def test_extract_chunk_none_for_non_png_bytes(self, tmp_path: Path):
|
||||
fake = tmp_path / "fake.png"
|
||||
fake.write_bytes(b"\xff\xd8\xff\xe0 not a png at all, just garbage bytes")
|
||||
assert extract_c2pa_chunk(fake) is None
|
||||
@@ -87,6 +87,24 @@ class TestModelProfiles:
|
||||
def test_recommended_strength_medium(self):
|
||||
assert get_recommended_strength("unknown_type") == 0.35
|
||||
|
||||
@pytest.mark.parametrize("wm_type", ["stegastamp", "stegasamp", "treering", "ringid"])
|
||||
def test_high_perturbation_watermark_types(self, wm_type):
|
||||
"""Robust spatial watermarks need aggressive (0.7) regeneration."""
|
||||
assert get_recommended_strength(wm_type) == 0.7
|
||||
|
||||
@pytest.mark.parametrize("wm_type", ["stablesignature", "dwtectsvd", "rivagan", "ssl", "hidden"])
|
||||
def test_low_perturbation_watermark_types(self, wm_type):
|
||||
"""Fragile frequency/latent watermarks break at low (0.04) strength."""
|
||||
assert get_recommended_strength(wm_type) == 0.04
|
||||
|
||||
def test_strength_match_is_case_insensitive(self):
|
||||
assert get_recommended_strength("TreeRing") == 0.7
|
||||
assert get_recommended_strength("StableSignature") == 0.04
|
||||
|
||||
def test_strength_matches_substring_in_descriptive_name(self):
|
||||
# e.g. a CLI passing "treering_v2" or "synthid-stegastamp" still maps.
|
||||
assert get_recommended_strength("treering_v2") == 0.7
|
||||
|
||||
|
||||
# ── Format utilities ────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Tests for the SynthID corpus ingestion script."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
# scripts/ is not an installed package; add it to the path for import.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts"))
|
||||
|
||||
import synthid_corpus
|
||||
|
||||
SAMPLES_DIR = Path(__file__).resolve().parent.parent / "data" / "samples"
|
||||
|
||||
|
||||
def _manifest_rows(root: Path) -> list[dict[str, str]]:
|
||||
with open(root / "manifest.csv", newline="") as f:
|
||||
return list(csv.DictReader(f))
|
||||
|
||||
|
||||
@pytest.mark.skipif(not SAMPLES_DIR.exists(), reason="data/samples not present")
|
||||
class TestIngest:
|
||||
def test_ingest_openai_flags_synthid_metadata(self, tmp_path: Path):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
synthid_corpus.cli,
|
||||
["ingest", str(SAMPLES_DIR / "chatgpt-1.png"), "--label", "pos", "--root", str(tmp_path)],
|
||||
)
|
||||
assert result.exit_code == 0, result.output
|
||||
|
||||
rows = _manifest_rows(tmp_path)
|
||||
assert len(rows) == 1
|
||||
row = rows[0]
|
||||
assert row["label"] == "pos"
|
||||
assert row["synthid_metadata"] == "yes"
|
||||
assert int(row["width"]) > 0
|
||||
assert int(row["height"]) > 0
|
||||
# The copied file lands under images/pos/ with a sha-prefixed name.
|
||||
assert (tmp_path / "images" / "pos" / row["filename"]).exists()
|
||||
|
||||
def test_ingest_firefly_not_flagged(self, tmp_path: Path):
|
||||
runner = CliRunner()
|
||||
runner.invoke(
|
||||
synthid_corpus.cli,
|
||||
["ingest", str(SAMPLES_DIR / "firefly-1.png"), "--label", "neg", "--root", str(tmp_path)],
|
||||
)
|
||||
rows = _manifest_rows(tmp_path)
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["synthid_metadata"] == "" # Adobe signs C2PA but not SynthID
|
||||
|
||||
def test_ingest_dedupes_by_sha256(self, tmp_path: Path):
|
||||
runner = CliRunner()
|
||||
args = ["ingest", str(SAMPLES_DIR / "chatgpt-1.png"), "--label", "pos", "--root", str(tmp_path)]
|
||||
runner.invoke(synthid_corpus.cli, args)
|
||||
runner.invoke(synthid_corpus.cli, args) # second time: duplicate
|
||||
assert len(_manifest_rows(tmp_path)) == 1
|
||||
|
||||
def test_status_on_empty_corpus(self, tmp_path: Path):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(synthid_corpus.cli, ["status", "--root", str(tmp_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "empty" in result.output.lower()
|
||||
@@ -0,0 +1,104 @@
|
||||
"""Tests for the SynthID pixel-carrier probe.
|
||||
|
||||
The probe's logic is validated synthetically: a fixed carrier injected into
|
||||
uniform fills must correlate strongly, random noise must not, and simulated
|
||||
removal (dropping the carrier) must collapse the correlation. No real SynthID
|
||||
fills or GPU are needed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
from click.testing import CliRunner
|
||||
from PIL import Image
|
||||
|
||||
# scripts/ is not an installed package; add it to the path for import.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts"))
|
||||
|
||||
import synthid_pixel_probe as probe
|
||||
|
||||
_SHAPE = (64, 64)
|
||||
|
||||
|
||||
def _fixed_carrier(seed: int = 7) -> np.ndarray:
|
||||
"""A reproducible 2-D carrier pattern (the stand-in for the SynthID signal)."""
|
||||
rng = np.random.default_rng(seed)
|
||||
return rng.standard_normal(_SHAPE)
|
||||
|
||||
|
||||
def _fill_with_carrier(base: float, pattern: np.ndarray, noise_seed: int, noise: float = 0.05) -> np.ndarray:
|
||||
"""A solid fill at brightness ``base`` carrying ``pattern`` plus light noise."""
|
||||
rng = np.random.default_rng(noise_seed)
|
||||
return base + pattern + noise * rng.standard_normal(_SHAPE)
|
||||
|
||||
|
||||
class TestCarrier:
|
||||
def test_flat_image_has_zero_carrier(self):
|
||||
# A perfectly uniform fill (std 0, like the synthetic refs) carries nothing.
|
||||
flat = np.zeros(_SHAPE)
|
||||
assert not np.any(probe.carrier(flat))
|
||||
|
||||
def test_carrier_is_unit_norm(self):
|
||||
c = probe.carrier(_fixed_carrier() + 100.0)
|
||||
assert np.isclose(np.linalg.norm(c), 1.0)
|
||||
|
||||
def test_ncc_self_is_one(self):
|
||||
c = probe.carrier(_fixed_carrier())
|
||||
assert np.isclose(probe.ncc(c, c), 1.0)
|
||||
|
||||
def test_ncc_mismatched_shape_is_zero(self):
|
||||
a = probe.carrier(np.random.default_rng(1).standard_normal((8, 8)))
|
||||
b = probe.carrier(np.random.default_rng(2).standard_normal((16, 16)))
|
||||
assert probe.ncc(a, b) == 0.0
|
||||
|
||||
|
||||
class TestConsistency:
|
||||
def test_shared_carrier_correlates_high(self):
|
||||
pattern = _fixed_carrier()
|
||||
carriers = [probe.carrier(_fill_with_carrier(b, pattern, s)) for s, b in enumerate((10, 60, 120, 200))]
|
||||
assert probe.mean_pairwise_ncc(carriers) > 0.8
|
||||
|
||||
def test_random_fills_near_zero(self):
|
||||
rng = np.random.default_rng(0)
|
||||
carriers = [probe.carrier(rng.standard_normal(_SHAPE)) for _ in range(5)]
|
||||
assert abs(probe.mean_pairwise_ncc(carriers)) < 0.2
|
||||
|
||||
def test_random_baseline_near_zero(self):
|
||||
assert abs(probe.random_baseline(_SHAPE, 6)) < 0.2
|
||||
|
||||
|
||||
class TestRemoval:
|
||||
def test_removed_carrier_collapses_correlation(self):
|
||||
pattern = _fixed_carrier()
|
||||
pos = [probe.carrier(_fill_with_carrier(b, pattern, s)) for s, b in enumerate((20, 90, 160))]
|
||||
tmpl = probe.template(pos)
|
||||
# "Cleaned" fills keep the base + noise but lose the shared pattern.
|
||||
rng = np.random.default_rng(99)
|
||||
cleaned = [probe.carrier(b + 0.05 * rng.standard_normal(_SHAPE)) for b in (20, 90, 160)]
|
||||
pos_corr = float(np.mean([probe.ncc(c, tmpl) for c in pos]))
|
||||
cleaned_corr = float(np.mean([probe.ncc(c, tmpl) for c in cleaned]))
|
||||
assert pos_corr > 0.8
|
||||
assert cleaned_corr < 0.2
|
||||
|
||||
|
||||
class TestCli:
|
||||
def _solid_fill_png(self, tmp_path: Path, name: str, base: int, pattern: np.ndarray, seed: int) -> Path:
|
||||
arr = np.clip(_fill_with_carrier(base, pattern, seed), 0, 255).astype(np.uint8)
|
||||
path = tmp_path / name
|
||||
Image.fromarray(np.stack([arr] * 3, axis=2)).save(path)
|
||||
return path
|
||||
|
||||
def test_consistency_command_runs(self, tmp_path: Path):
|
||||
pattern = _fixed_carrier()
|
||||
paths = [str(self._solid_fill_png(tmp_path, f"f{i}.png", b, pattern, i)) for i, b in enumerate((40, 120, 200))]
|
||||
result = CliRunner().invoke(probe.cli, ["consistency", *paths])
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "mean pairwise NCC" in result.output
|
||||
|
||||
def test_consistency_needs_two_images(self, tmp_path: Path):
|
||||
path = str(self._solid_fill_png(tmp_path, "only.png", 100, _fixed_carrier(), 0))
|
||||
result = CliRunner().invoke(probe.cli, ["consistency", path])
|
||||
assert result.exit_code != 0
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Unit tests for the pure tiling helpers (no GPU/model required).
|
||||
|
||||
``tiling.py`` imports torch at module top, so skip cleanly when torch is
|
||||
absent. The helpers themselves are pure numpy/PIL/math -- they decide how a
|
||||
large image is split into overlapping tiles and blended back, so a regression
|
||||
here would seam or crop the CtrlRegen output wrongly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
pytest.importorskip("torch")
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks.noai.ctrlregen.tiling import (
|
||||
make_blend_weight,
|
||||
resize_center_crop,
|
||||
tile_positions,
|
||||
)
|
||||
|
||||
|
||||
class TestTilePositions:
|
||||
def test_image_smaller_than_tile_single_position(self):
|
||||
assert tile_positions(500, 512, 64) == [0]
|
||||
|
||||
def test_image_equal_to_tile_single_position(self):
|
||||
assert tile_positions(512, 512, 64) == [0]
|
||||
|
||||
def test_first_is_zero_last_is_total_minus_tile(self):
|
||||
# The tiles must fully cover the span: first starts at 0, last ends at
|
||||
# the far edge (start == total - tile), or the image's edge is missed.
|
||||
pos = tile_positions(2000, 512, 64)
|
||||
assert pos[0] == 0
|
||||
assert pos[-1] == 2000 - 512
|
||||
|
||||
def test_overlap_positions_are_monotonic_and_exact(self):
|
||||
assert tile_positions(1000, 512, 64) == [0, 244, 488]
|
||||
|
||||
def test_zero_overlap_tiles_are_contiguous(self):
|
||||
# 1024 wide, 512 tile, no overlap -> two tiles butting at 512.
|
||||
assert tile_positions(1024, 512, 0) == [0, 512]
|
||||
|
||||
|
||||
class TestMakeBlendWeight:
|
||||
def test_zero_overlap_is_all_ones(self):
|
||||
w = make_blend_weight(8, 8, 0)
|
||||
assert w.shape == (8, 8)
|
||||
assert w.dtype == np.float64
|
||||
assert np.all(w == 1.0)
|
||||
|
||||
def test_overlap_ramps_corners_to_zero_center_to_one(self):
|
||||
w = make_blend_weight(16, 16, 4)
|
||||
assert w[0, 0] == 0.0 # cosine ramp starts at 0
|
||||
assert w[8, 8] == 1.0 # center is unweighted
|
||||
assert w.max() == 1.0
|
||||
assert w.min() == 0.0
|
||||
|
||||
def test_weight_is_point_symmetric(self):
|
||||
# Symmetric ramps on both edges -> mask equals its 180-degree rotation,
|
||||
# so opposite tile seams blend identically.
|
||||
w = make_blend_weight(16, 16, 4)
|
||||
assert np.allclose(w, w[::-1, ::-1])
|
||||
|
||||
|
||||
class TestResizeCenterCrop:
|
||||
@pytest.mark.parametrize(("width", "height"), [(400, 800), (800, 400), (300, 300), (1000, 1001)])
|
||||
def test_output_is_always_square_of_requested_size(self, width: int, height: int):
|
||||
out = resize_center_crop(Image.new("RGB", (width, height)), 256)
|
||||
assert out.size == (256, 256)
|
||||
|
||||
def test_default_size_is_512(self):
|
||||
out = resize_center_crop(Image.new("RGB", (640, 480)))
|
||||
assert out.size == (512, 512)
|
||||
@@ -815,11 +815,11 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "idna"
|
||||
version = "3.11"
|
||||
version = "3.16"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/0703ccc57f3a7233505399edb88de3cbd678da106337b9fcde432b65ed60/idna-3.11.tar.gz", hash = "sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902", size = 194582, upload-time = "2025-10-12T14:55:20.501Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/1a/88/bcf9709822fe69d02c2a6a77956c98ce6ea8ca8767a9aadcedc7eb6a2390/idna-3.16.tar.gz", hash = "sha256:d7a6da03db833450fca25d2358ac9ff06cd624577a4aea3a596d5c0f77b8e03d", size = 203770, upload-time = "2026-05-22T00:16:18.781Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/0e/61/66938bbb5fc52dbdf84594873d5b51fb1f7c7794e9c0f5bd885f30bc507b/idna-3.11-py3-none-any.whl", hash = "sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea", size = 71008, upload-time = "2025-10-12T14:55:18.883Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/94/16/70255075a9859a0e3adb789b68ceb0e210dec03934245fd98d248226572f/idna-3.16-py3-none-any.whl", hash = "sha256:cc246e3a3f89580c3a951b5ad298ca4638078b2cdd4f115654332b5c26daded5", size = 74165, upload-time = "2026-05-22T00:16:16.698Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -857,6 +857,23 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "invisible-watermark"
|
||||
version = "0.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
|
||||
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
{ name = "opencv-python" },
|
||||
{ name = "pillow" },
|
||||
{ name = "pywavelets", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
|
||||
{ name = "pywavelets", version = "1.9.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
{ name = "torch" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/2b/57/18b5a914f6d7994dd349252873169e946dc824328e9a37fd15ed836deedc/invisible_watermark-0.2.0-py3-none-any.whl", hash = "sha256:644311beed9cfe4a9a5a4a46c740f47800cef184fe2e1297f3f4542e2d992f8b", size = 1633253, upload-time = "2023-07-06T13:56:28.715Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jinja2"
|
||||
version = "3.1.6"
|
||||
@@ -1831,6 +1848,121 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/0b/d7/1959b9648791274998a9c3526f6d0ec8fd2233e4d4acce81bbae76b44b2a/python_dotenv-1.2.2-py3-none-any.whl", hash = "sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a", size = 22101, upload-time = "2026-03-01T16:00:25.09Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pywavelets"
|
||||
version = "1.8.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
resolution-markers = [
|
||||
"python_full_version < '3.11' and sys_platform == 'win32'",
|
||||
"python_full_version < '3.11' and sys_platform != 'win32'",
|
||||
]
|
||||
dependencies = [
|
||||
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/48/45/bfaaab38545a33a9f06c61211fc3bea2e23e8a8e00fedeb8e57feda722ff/pywavelets-1.8.0.tar.gz", hash = "sha256:f3800245754840adc143cbc29534a1b8fc4b8cff6e9d403326bd52b7bb5c35aa", size = 3935274, upload-time = "2024-12-04T19:54:20.593Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/65/7e/c5e398f25c70558ca195dd4144ee004666401f6167084c1e76059d7e68d8/pywavelets-1.8.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:f5c86fcb203c8e61d1f3d4afbfc08d626c64e4e3708207315577264c724632bf", size = 4323291, upload-time = "2024-12-04T19:53:01.836Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d0/d7/2fc8067c3520ce25f7632b0f47b89d1b75653cab804a42700e95126f2679/pywavelets-1.8.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:fafb5fa126277e1690c3d6329287122fc08e4d25a262ce126e3d81b1f5709308", size = 4291864, upload-time = "2024-12-04T19:53:04.659Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2f/17/a868aa26e45c104613d9069f9d8ec0123687cb6945062d274f20a3992436/pywavelets-1.8.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:dec23dfe6d5a3f4312b12456b8c546aa90a11c1138e425a885987505f0658ae0", size = 4447532, upload-time = "2024-12-04T19:53:06.383Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/53/7a/7f5889a57177e2b1182080fc2c52236d1e03a0fad5e0b3d7c5312070c0be/pywavelets-1.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:880a0197e9fa108939af50a95e97c1bf9b7d3e148e0fad92ea60a9ed8c8947c0", size = 4487695, upload-time = "2024-12-04T19:53:08.84Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f9/e6/04d76d93c158919ef0d8e1d40d1d453168305031eca6733fdc844f7cbb07/pywavelets-1.8.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:8bfa833d08b60d0bf53a7939fbbf3d98015dd34efe89cbe4e53ced880d085fc1", size = 4473752, upload-time = "2024-12-04T19:53:10.848Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3b/a7/42ea5bbb6055abd312e45b27d931200fd6eed5414a87ec5d62020a4c651b/pywavelets-1.8.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:e10c3fc7f4a796e94da4bca9871be2186a7bb7a3b3536a0ca9376d84263140f0", size = 4504191, upload-time = "2024-12-04T19:53:13.912Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8c/7e/52df87a9e77adfb12c1b8be79a2053f2eb4c2507dec96ebfd2333b15ff03/pywavelets-1.8.0-cp310-cp310-win32.whl", hash = "sha256:31baf4be6940fde72cc85663154360857ac1b93c251822deaf72bb804da95031", size = 4143794, upload-time = "2024-12-04T19:53:16.296Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/01/e2/06e08230c26049740b2773952fbb12cc7186e5df655a73b1c30ba493e864/pywavelets-1.8.0-cp310-cp310-win_amd64.whl", hash = "sha256:560c39f1ff8cb37f8b8ea4b7b6eb8a14f6926c11f5cf8c09f013a58f895ed5bc", size = 4214262, upload-time = "2024-12-04T19:53:17.998Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6c/8a/9f8e794120b55caa1c4ae8d72696111bc408251615f351a8e54a5d8c4d4e/pywavelets-1.8.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:e8dd5be4faed994581a8a4b3c0169be20567a9346e523f0b57f903c8f6722bce", size = 4324170, upload-time = "2024-12-04T19:53:19.66Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3e/b8/f6246be5c78e9fa73fcbba9ab4cbfe0d4dcb79ea5491f28d673a53466134/pywavelets-1.8.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:8d8abaf7c120b151ef309c9ff57e0a44ba9febf49045056dbc1577526ecec6c8", size = 4294254, upload-time = "2024-12-04T19:53:21.767Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2c/dc/ba1f212e9b43117ed28e0fd092e72e817790427400f88937ea742d260153/pywavelets-1.8.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4b43a4c58707b1e8d941bec7f1d83e67c482278575ff0db3189d5c0dfae23a57", size = 4447178, upload-time = "2024-12-04T19:53:23.525Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/58/10/e59c162a11d2fedb4454abbf7b74a52390aba5edc9605bf829bfa8708dac/pywavelets-1.8.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1c1aad0b97714e3079a2bfe48e4fb8ccd60778d0427e9ee5e0a9ff922e6c61e4", size = 4486799, upload-time = "2024-12-04T19:53:25.238Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/03/ee/90c3d0a0a3bda74e6e097e4c06bff9446ff2a4c90b8617aaf4902c46966b/pywavelets-1.8.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:a0e1db96dcf3ce08156859df8b359e9ff66fa15061a1b90e70e020bf4cd077a0", size = 4486403, upload-time = "2024-12-04T19:53:26.954Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/05/54/58b87f8b636a9f044f3f9814d2ec696cf25f3b33af97c11811f13c364085/pywavelets-1.8.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:e62c8fb52ab0e8ff212fff9acae681a8f12d68b76c36fe24cc48809d5b6825ba", size = 4515011, upload-time = "2024-12-04T19:53:28.832Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/d0/f755cee11ff20668114942d0e777e2b502a8e4665e1fdb2553b587aac637/pywavelets-1.8.0-cp311-cp311-win32.whl", hash = "sha256:bf327528d10de471b04bb725c4e10677fac5a49e13d41bf0d0b3a1f6d7097abf", size = 4139934, upload-time = "2024-12-04T19:53:31.421Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/0b/f4b92d4f00565280ea3e62a8e3dc81a667d67ed7bd59232f2f18d55f9aff/pywavelets-1.8.0-cp311-cp311-win_amd64.whl", hash = "sha256:3814d354dd109e244ffaac3d480d29a5202212fe24570c920268237c8d276f95", size = 4214321, upload-time = "2024-12-04T19:53:33.183Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2d/8b/4870f11559307416470158a5aa6f61e5c2a910f1645a7a836ffae580b7ad/pywavelets-1.8.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3f431c9e2aff1a2240765eff5e804975d0fcc24c82d6f3d4271243f228e5963b", size = 4326187, upload-time = "2024-12-04T19:53:35.19Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c4/35/66835d889fd7fbf3119c7a9bd9d9bd567fc0bb603dfba408e9226db7cb44/pywavelets-1.8.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:e39b0e2314e928cb850ee89b9042733a10ea044176a495a54dc84d2c98407a51", size = 4295428, upload-time = "2024-12-04T19:53:36.962Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/63/1c/42e5130226538c70d4bbbaee00eb1bc06ec3287f7ea43d5fcf85bfc761ce/pywavelets-1.8.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cae701117f5c7244b7c8d48b9e92a0289637cdc02a9c205e8be83361f0c11fae", size = 4421259, upload-time = "2024-12-04T19:53:39.119Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6f/c5/1ce93657432e22a5debc21e8b52ec6980f819ecb7fa727bb86744224d967/pywavelets-1.8.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:649936baee933e80083788e0adc4d8bc2da7cdd8b10464d3b113475be2cc5308", size = 4447650, upload-time = "2024-12-04T19:53:41.589Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b9/d6/b54ef30daca71824f811f9d2322a978b0a58d27674b8e3af6520f67e9ec6/pywavelets-1.8.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8c68e9d072c536bc646e8bdce443bb1826eeb9aa21b2cb2479a43954dea692a3", size = 4448538, upload-time = "2024-12-04T19:53:44.308Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ce/8c/1688b790e55674667ad644262f174405c2c9873cb13e773432e78b1b33e4/pywavelets-1.8.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:63f67fa2ee1610445de64f746fb9c1df31980ad13d896ea2331fc3755f49b3ae", size = 4485228, upload-time = "2024-12-04T19:53:46.778Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c9/9b/69de31c3b663dadd76d1da6bf8af68d8cefff55df8e880fe96a94bb8c9ac/pywavelets-1.8.0-cp312-cp312-win32.whl", hash = "sha256:4b3c2ab669c91e3474fd63294355487b7dd23f0b51d32f811327ddf3546f4f3d", size = 4134850, upload-time = "2024-12-04T19:53:49.101Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1c/88/9e2aa9d5fde08bfc0fb18ffb1b5307c1ed49c24930b4147e5f48571a7251/pywavelets-1.8.0-cp312-cp312-win_amd64.whl", hash = "sha256:810a23a631da596fef7196ddec49b345b1aab13525bb58547eeebe1769edbbc1", size = 4210786, upload-time = "2024-12-04T19:53:51.546Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/94/73/7ff347d77c6bda11330565050c3346c54bc210086380abeb84e402c1c9cd/pywavelets-1.8.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:441ba45c8dff8c6916dbe706958d0d7f91da675695ca0c0d75e483f6f52d0a12", size = 4321474, upload-time = "2024-12-04T19:53:53.369Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b0/70/c58937ebbca1aba3475ca5ee63c7bcebf09f3c93891ae5942eaec7e95707/pywavelets-1.8.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:24bb282bab09349d9d128ed0536fa50fff5c2147891971a69c2c36155dfeeeac", size = 4291502, upload-time = "2024-12-04T19:53:55.396Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/da/55/87b4ad6128b2e85985908e958e856e0b680cdcc03cc490e2cc995164b13a/pywavelets-1.8.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:426ff3799446cb4da1db04c2084e6e58edfe24225596805665fd39c14f53dece", size = 4412669, upload-time = "2024-12-04T19:53:57.393Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/bf/1a/bfca9eab23bd7b27843b0ce95c47796033a7b2c93048315f5fc5d6ac6428/pywavelets-1.8.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aa0607a9c085b8285bc0d04e33d461a6c80f8c325389221ffb1a45141861138e", size = 4454604, upload-time = "2024-12-04T19:53:59.105Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c3/23/9ce38829f57159e812c469c4f9d7b5a16c1ba922c1802985e8c504468206/pywavelets-1.8.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d31c36a39110e8fcc7b1a4a11cfed7d22b610c285d3e7f4fe73ec777aa49fa39", size = 4445507, upload-time = "2024-12-04T19:54:00.78Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e5/d2/e78a976b0600a6ef7a70f4430122d6ad11b3e1cbda3c8b3565661d094678/pywavelets-1.8.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:fa7c68ed1e5bab23b1bafe60ccbcf709b878652d03de59e961baefa5210fcd0a", size = 4479078, upload-time = "2024-12-04T19:54:02.46Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/85/4d/1c4f870010368f3aeb0bdd72929376a1988e4a122e76545bd8c56e549c96/pywavelets-1.8.0-cp313-cp313-win32.whl", hash = "sha256:2c6b359b55d713ef683e9da1529181b865a80d759881ceb9adc1c5742e4da4d8", size = 4133763, upload-time = "2024-12-04T19:54:04.016Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c9/4f/0a709a5732e6cf9297fc87bf545cb879997cde204115f8c0cbc296c5bdd3/pywavelets-1.8.0-cp313-cp313-win_amd64.whl", hash = "sha256:4dbebcfd55ea8a85b7fc8802d411e75337170422abf6e96019d7e46c394e80e5", size = 4209548, upload-time = "2024-12-04T19:54:06.61Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/de/2a/4cac0bba67d3bc0ad697d0680539864db0a6964c7ad953d8d9d887f360b3/pywavelets-1.8.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:2e1c79784bebeafd3715c1bea6621daa2e2e6ed37b687719322e2078fb35bb70", size = 4335183, upload-time = "2024-12-04T19:54:08.349Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/58/d1/3abe4cf34a35b09ad847f0e9a85f340c1988611222926d295fa8710659e7/pywavelets-1.8.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7f489380c95013cc8fb3ef338f6d8c1a907125db453cc4dc739e2cca06fcd8b6", size = 4454723, upload-time = "2024-12-04T19:54:11.007Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/62/f05dd191232ae94e0b48509bb0ee65c9d45abf5e8f3612b09fd309b41384/pywavelets-1.8.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:06786201a91b5e74540f4f3c115c49a29190de2eb424823abbd3a1fd75ea3e28", size = 4472192, upload-time = "2024-12-04T19:54:12.754Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/20/6a/257c95ad1e0fd395cbccd4ffec0d01cc9b51a3bb91e67d8fa10ffebc9c72/pywavelets-1.8.0-cp313-cp313t-win32.whl", hash = "sha256:f2877fb7b58c94211257dcf364b204d6ed259146fc87d5a90bf9d93c97af6226", size = 4183968, upload-time = "2024-12-04T19:54:15.099Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6c/58/7179fd6f87153f2e339171e8cfe9bf901398a89045eefd7a3911bb9b47ad/pywavelets-1.8.0-cp313-cp313t-win_amd64.whl", hash = "sha256:ec5d723c3335ff8aa630fd4b14097077f12cc02893c91cafd60dd7b1730e780f", size = 4265431, upload-time = "2024-12-04T19:54:16.928Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pywavelets"
|
||||
version = "1.9.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
resolution-markers = [
|
||||
"python_full_version >= '3.11' and sys_platform == 'win32'",
|
||||
"python_full_version >= '3.11' and sys_platform != 'win32'",
|
||||
]
|
||||
dependencies = [
|
||||
{ name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/5a/75/50581633d199812205ea8cdd0f6d52f12a624886b74bf1486335b67f01ff/pywavelets-1.9.0.tar.gz", hash = "sha256:148d12203377772bea452a59211d98649c8ee4a05eff019a9021853a36babdc8", size = 3938340, upload-time = "2025-08-04T16:20:04.978Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/bd/8b/ca700d0c174c3a4eec1fbb603f04374d1fed84255c2a9f487cfaa749c865/pywavelets-1.9.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:54662cce4d56f0d6beaa6ebd34b2960f3aa4a43c83c9098a24729e9dc20a4be2", size = 4323640, upload-time = "2025-08-04T16:18:51.683Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b5/f3/0fa57b6407ea9c4452b0bc182141256b9481b479ffbfc9d7fdb73afe193b/pywavelets-1.9.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:0d8ed4b4d1eab9347e8fe0c5b45008ce5a67225ce5b05766b8b1fa923a5f8b34", size = 4294938, upload-time = "2025-08-04T16:18:53.818Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ea/95/a998313c8459a57e488ff2b18e24be9e836aedda3aa3a1673197deeaa59a/pywavelets-1.9.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:862be65481fdfecfd84c6b0ca132ba571c12697a082068921bca5b5e039f1371", size = 4472829, upload-time = "2025-08-04T16:18:55.508Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d8/8c/f316a153f7f89d2753df8a7371d15d0faab87e709fe02715dbc297c79385/pywavelets-1.9.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d76b7fa8fc500b09201d689b4f15bf5887e30ffbe2e1f338eb8470590eb4521a", size = 4524936, upload-time = "2025-08-04T16:18:57.146Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/24/f7/89fdc1caef4b384a341a8e149253e23f36c1702bbb986a26123348624854/pywavelets-1.9.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:aa859d0b686a697c87a47e29319aebe44125f114a4f8c7e444832b921f52de5a", size = 4481475, upload-time = "2025-08-04T16:18:58.725Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/82/53/b733fbfb71853e4a5c430da56e325a763562d65241dd785f0fadb67aed6a/pywavelets-1.9.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:20e97b84a263003e2c7348bcf72beba96edda1a6169f072dc4e4d4ee3a6c7368", size = 4527994, upload-time = "2025-08-04T16:18:59.917Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ed/15/5f6a6e9fdad8341e42642ed622a5f3033da4ea9d426cc3e574ae418b4726/pywavelets-1.9.0-cp311-cp311-win32.whl", hash = "sha256:f8330cdbfa506000e63e79525716df888998a76414c5cd6ecd9a7e371191fb05", size = 4136109, upload-time = "2025-08-04T16:19:01.511Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fd/33/62dbb4aea86ec9d79b283127c42cc896f4d4ff265a9aeb1337a7836dd550/pywavelets-1.9.0-cp311-cp311-win_amd64.whl", hash = "sha256:ed10959a17df294ef55948dcc76367d59ec7b6aad67e38dd4e313d2fe3ad47b2", size = 4228321, upload-time = "2025-08-04T16:19:03.164Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5c/37/3fda13fb2518fdd306528382d6b18c116ceafefff0a7dccd28f1034f4dd2/pywavelets-1.9.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:30baa0788317d3c938560c83fe4fc43817342d06e6c9662a440f73ba3fb25c9b", size = 4320835, upload-time = "2025-08-04T16:19:04.855Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/36/65/a5549325daafc3eae4b52de076798839eaf529a07218f8fb18cccefe76a1/pywavelets-1.9.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:df7436a728339696a7aa955c020ae65c85b0d9d2b5ff5b4cf4551f5d4c50f2c7", size = 4290469, upload-time = "2025-08-04T16:19:06.178Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/05/85/901bb756d37dfa56baa26ef4a3577aecfe9c55f50f51366fede322f8c91d/pywavelets-1.9.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:07b26526db2476974581274c43a9c2447c917418c6bd03c8d305ad2a5cd9fac3", size = 4437717, upload-time = "2025-08-04T16:19:07.514Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0f/34/0f54dd9c288941294898877008bcb5c07012340cc9c5db9cff1bd185d449/pywavelets-1.9.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:573b650805d2f3c981a0e5ae95191c781a722022c37a0f6eba3fa7eae8e0ee17", size = 4483843, upload-time = "2025-08-04T16:19:08.857Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/48/1f/cff6bb4ea64ff508d8cac3fe113c0aa95310a7446d9efa6829027cc2afdf/pywavelets-1.9.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3747ec804492436de6e99a7b6130480e53406d047e87dc7095ab40078a515a23", size = 4442236, upload-time = "2025-08-04T16:19:11.061Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ce/53/a3846eeefe0fb7ca63ae045f038457aa274989a15af793c1b824138caf98/pywavelets-1.9.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5163665686219c3f43fd5bbfef2391e87146813961dad0f86c62d4aed561f547", size = 4488077, upload-time = "2025-08-04T16:19:12.333Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f7/98/44852d2fe94455b72dece2db23562145179d63186a1c971125279a1c381f/pywavelets-1.9.0-cp312-cp312-win32.whl", hash = "sha256:80b8ab99f5326a3e724f71f23ba8b0a5b03e333fa79f66e965ea7bed21d42a2f", size = 4134094, upload-time = "2025-08-04T16:19:13.564Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2c/a7/0d9ee3fe454d606e0f5c8e3aebf99d2ecddbfb681826a29397729538c8f1/pywavelets-1.9.0-cp312-cp312-win_amd64.whl", hash = "sha256:92bfb8a117b8c8d3b72f2757a85395346fcbf37f50598880879ae72bd8e1c4b9", size = 4213900, upload-time = "2025-08-04T16:19:14.939Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/db/a7/dec4e450675d62946ad975f5b4d924437df42d2fae46e91dfddda2de0f5a/pywavelets-1.9.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:74f8455c143818e4b026fc67b27fd82f38e522701b94b8a6d1aaf3a45fcc1a25", size = 4316201, upload-time = "2025-08-04T16:19:16.259Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/aa/0c/b54b86596c0df68027e48c09210e907e628435003e77048384a2dd6767e3/pywavelets-1.9.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c50320fe0a4a23ddd8835b3dc9b53b09ee05c7cc6c56b81d0916f04fc1649070", size = 4286838, upload-time = "2025-08-04T16:19:17.92Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5a/9c/333969c3baad8af2e7999e83addcb7bb1d1fd48e2d812fb27e2e89582cb1/pywavelets-1.9.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d6e059265223ed659e5214ab52a84883c88ddf3decbf08d7ec6abb8e4c5ed7be", size = 4430753, upload-time = "2025-08-04T16:19:19.529Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e5/1b/a24c6ff03b026b826ad7b9267bd63cd34ce026795a0302f8a5403840b8e7/pywavelets-1.9.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ae10ed46c139c7ddb8b1249cfe0989f8ccb610d93f2899507b1b1573a0e424b5", size = 4491315, upload-time = "2025-08-04T16:19:20.717Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d7/c7/e3fbb502fca3469e51ced4f1e1326364c338be91edc5db5a8ddd26b303fa/pywavelets-1.9.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:c8f8b1cc2df012401cb837ee6fa2f59607c7b4fe0ff409d9a4f6906daf40dc86", size = 4437654, upload-time = "2025-08-04T16:19:22.359Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/92/44/c9b25084048d9324881a19b88e0969a4141bcfdc1d218f1b4b680b7af1c1/pywavelets-1.9.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:db43969c7a8fbb17693ecfd14f21616edc3b29f0e47a49b32fa4127c01312a67", size = 4496435, upload-time = "2025-08-04T16:19:23.842Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cd/b6/b27ec18c72b1dee3314e297af39c5f8136d43cc130dd93cb6c178ca820e5/pywavelets-1.9.0-cp313-cp313-win32.whl", hash = "sha256:9e7d60819d87dcd6c68a2d1bc1d37deb1f4d96607799ab6a25633ea484dcda41", size = 4132709, upload-time = "2025-08-04T16:19:25.415Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0a/87/78ef3f9fb36cdb16ee82371d22c3a7c89eeb79ec8c9daef6222060da6c79/pywavelets-1.9.0-cp313-cp313-win_amd64.whl", hash = "sha256:0d70da9d7858c869e24dc254f16a61dc09d8a224cad85a10c393b2eccddeb126", size = 4213377, upload-time = "2025-08-04T16:19:26.875Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/cd/ca0d9db0ff29e3843f6af60c2f5eb588794e05ca8eeb872a595867b1f3f5/pywavelets-1.9.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:4dc85f44c38d76a184a1aa2cb038f802c3740428c9bb877525f4be83a223b134", size = 4354336, upload-time = "2025-08-04T16:19:28.745Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/82/d6/70afefcc1139f37d02018a3b1dba3b8fc87601bb7707d9616b7f7a76e269/pywavelets-1.9.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:7acf6f950c6deaecd210fbff44421f234a8ca81eb6f4da945228e498361afa9d", size = 4335721, upload-time = "2025-08-04T16:19:30.371Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cd/3a/713f731b9ed6df0c36269c8fb62be8bb28eb343b9e26b13d6abda37bce38/pywavelets-1.9.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:144d4fc15c98da56654d0dca2d391b812b8d04127b194a37ad4a497f8e887141", size = 4418702, upload-time = "2025-08-04T16:19:31.743Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/44/e8/f801eb4b5f7a316ba20054948c5d6b27b879c77fab2674942e779974bd86/pywavelets-1.9.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1aa3729585408a979d655736f74b995b511c86b9be1544f95d4a3142f8f4b8b5", size = 4470023, upload-time = "2025-08-04T16:19:32.963Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/cc/44b002cb16f2a392f2082308dd470b3f033fa4925d3efa7c46f790ce895a/pywavelets-1.9.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:e0e24ad6b8eb399c49606dd1fcdcbf9749ad7f6d638be3fe6f59c1f3098821e2", size = 4426498, upload-time = "2025-08-04T16:19:34.151Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/91/fe/2b70276ede7878c5fe8356ca07574db5da63e222ce39a463e84bfad135e8/pywavelets-1.9.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:3830e6657236b53a3aae20c735cccead942bb97c54bbca9e7d07bae01645fe9c", size = 4477528, upload-time = "2025-08-04T16:19:35.932Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e7/ed/d58b540c15e36508cfeded7b0d39493e811b0dce18d9d4e6787fb2e89685/pywavelets-1.9.0-cp313-cp313t-win32.whl", hash = "sha256:81bb65facfbd7b50dec50450516e72cdc51376ecfdd46f2e945bb89d39bfb783", size = 4186493, upload-time = "2025-08-04T16:19:37.198Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/84/b2/12a849650d618a86bbe4d8876c7e20a7afe59a8cad6f49c57eca9af26dfa/pywavelets-1.9.0-cp313-cp313t-win_amd64.whl", hash = "sha256:47d52cf35e2afded8cfe1133663f6f67106a3220b77645476ae660ad34922cb4", size = 4274821, upload-time = "2025-08-04T16:19:38.436Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ba/1f/18c82122547c9eec2232d800b02ada1fbd30ce2136137b5738acca9d653e/pywavelets-1.9.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:53043d2f3f4e55a576f51ac594fe33181e1d096d958e01524db5070eb3825306", size = 4314440, upload-time = "2025-08-04T16:19:39.701Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/eb/e1/1c92ac6b538ef5388caf1a74af61cf6af16ea6d14115bb53357469cb38d6/pywavelets-1.9.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:56bc36b42b1b125fd9cb56e7956b22f8d0f83c1093f49c77fc042135e588c799", size = 4290162, upload-time = "2025-08-04T16:19:41.322Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/96/d3/d856a2cac8069c20144598fa30a43ca40b5df2e633230848a9a942faf04a/pywavelets-1.9.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:08076eb9a182ddc6054ac86868fb71df6267c341635036dc63d20bdbacd9ad7e", size = 4437162, upload-time = "2025-08-04T16:19:42.556Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c9/54/777e0495acd4fb008791e84889be33d6e7fc8af095b441d939390b7d2491/pywavelets-1.9.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4ee1ee7d80f88c64b8ec3b5021dd1e94545cc97f0cd479fb51aa7b10f6def08e", size = 4498169, upload-time = "2025-08-04T16:19:43.791Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/76/68/81b97f4d18491a18fbe17e06e2eee80a591ce445942f7b6f522de07813c5/pywavelets-1.9.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:3226b6f62838a6ccd7782cb7449ee5d8b9d61999506c1d9b03b2baf41b01b6fd", size = 4443318, upload-time = "2025-08-04T16:19:45.368Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/92/74/5147f2f0436f7aa131cb1bc13dba32ef5f3862748ae1c7366b4cde380362/pywavelets-1.9.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:9fb7f4b11d18e2db6dd8deee7b3ce8343d45f195f3f278c2af6e3724b1b93a24", size = 4503294, upload-time = "2025-08-04T16:19:46.632Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3d/d4/af998cc71e869919e0ab45471bd43e91d055ac7bc3ce6f56cc792c9b6bc8/pywavelets-1.9.0-cp314-cp314-win32.whl", hash = "sha256:9902d9fc9812588ab2dce359a1307d8e7f002b53a835640e2c9388fe62a82fd4", size = 4144478, upload-time = "2025-08-04T16:19:47.974Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7d/66/1d071eae5cc3e3ad0e45334462f8ce526a79767ccb759eb851aa5b78a73a/pywavelets-1.9.0-cp314-cp314-win_amd64.whl", hash = "sha256:7e57792bde40e331d6cc65458e5970fd814dba18cfc4e9add9d051e901a7b7c7", size = 4227186, upload-time = "2025-08-04T16:19:49.57Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/bf/1f/da0c03ac99bd9d20409c0acf6417806d4cf333d70621da9f535dd0cf27fa/pywavelets-1.9.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:b47c72fb4b76d665c4c598a5b621b505944e5b761bf03df9d169029aafcb652f", size = 4354391, upload-time = "2025-08-04T16:19:51.221Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/95/b6/de9e225d8cc307fbb4fda88aefa79442775d5e27c58ee4d3c8a8580ceba6/pywavelets-1.9.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:969e369899e7eab546ea5d77074e4125082e6f9dad71966499bf5dee3758be55", size = 4335810, upload-time = "2025-08-04T16:19:52.813Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/33/3b/336761359d07cd44a4233ca854704ff2a9e78d285879ccc82d254b9daa57/pywavelets-1.9.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8aeffd4f35036c1fade972a61454de5709a7a8fc9a7d177eefe3ac34d76962e5", size = 4422220, upload-time = "2025-08-04T16:19:54.068Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/98/61/76ccc7ada127f14f65eda40e37407b344fd3713acfca7a94d7f0f67fe57d/pywavelets-1.9.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f63f400fcd4e7007529bd06a5886009760da35cd7e76bb6adb5a5fbee4ffeb8c", size = 4470156, upload-time = "2025-08-04T16:19:55.379Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e0/de/142ca27ee729cf64113c2560748fcf2bd45b899ff282d6f6f3c0e7f177bb/pywavelets-1.9.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:a63bcb6b5759a7eb187aeb5e8cd316b7adab7de1f4b5a0446c9a6bcebdfc22fb", size = 4430167, upload-time = "2025-08-04T16:19:56.566Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ca/5e/90b39adff710d698c00ba9c3125e2bec99dad7c5f1a3ba37c73a78a6689f/pywavelets-1.9.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9950eb7c8b942e9bfa53d87c7e45a420dcddbd835c4c5f1aca045a3f775c6113", size = 4477378, upload-time = "2025-08-04T16:19:58.162Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f1/1a/89f5f4ebcb9d34d9b7b2ac0a868c8b6d8c78d699a36f54407a060cea0566/pywavelets-1.9.0-cp314-cp314t-win32.whl", hash = "sha256:097f157e07858a1eb370e0d9c1bd11185acdece5cca10756d6c3c7b35b52771a", size = 4209132, upload-time = "2025-08-04T16:20:00.371Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/68/d2/a8065103f5e2e613b916489e6c85af6402a1ec64f346d1429e2d32cb8d03/pywavelets-1.9.0-cp314-cp314t-win_amd64.whl", hash = "sha256:3b6ff6ba4f625d8c955f68c2c39b0a913776d406ab31ee4057f34ad4019fb33b", size = 4306793, upload-time = "2025-08-04T16:20:02.934Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyyaml"
|
||||
version = "6.0.3"
|
||||
@@ -2018,7 +2150,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.4.1"
|
||||
version = "0.5.5"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "click" },
|
||||
@@ -2037,6 +2169,7 @@ all = [
|
||||
{ name = "color-matcher" },
|
||||
{ name = "controlnet-aux" },
|
||||
{ name = "diffusers" },
|
||||
{ name = "invisible-watermark" },
|
||||
{ name = "pyright" },
|
||||
{ name = "pytest" },
|
||||
{ name = "pytest-cov" },
|
||||
@@ -2046,7 +2179,11 @@ all = [
|
||||
{ name = "transformers" },
|
||||
{ name = "ultralytics" },
|
||||
]
|
||||
detect = [
|
||||
{ name = "invisible-watermark" },
|
||||
]
|
||||
dev = [
|
||||
{ name = "invisible-watermark" },
|
||||
{ name = "pyright" },
|
||||
{ name = "pytest" },
|
||||
{ name = "pytest-cov" },
|
||||
@@ -2070,6 +2207,8 @@ requires-dist = [
|
||||
{ name = "color-matcher", marker = "extra == 'gpu'", specifier = ">=0.5.0" },
|
||||
{ name = "controlnet-aux", marker = "extra == 'gpu'", specifier = ">=0.0.9" },
|
||||
{ name = "diffusers", marker = "extra == 'gpu'", specifier = ">=0.38.0" },
|
||||
{ name = "invisible-watermark", marker = "extra == 'detect'", specifier = ">=0.2.0" },
|
||||
{ name = "invisible-watermark", marker = "extra == 'dev'", specifier = ">=0.2.0" },
|
||||
{ name = "numpy", specifier = ">=1.24.0" },
|
||||
{ name = "opencv-python-headless", specifier = ">=4.8.0" },
|
||||
{ name = "piexif", specifier = ">=1.1.3" },
|
||||
@@ -2078,7 +2217,7 @@ requires-dist = [
|
||||
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.0" },
|
||||
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.1.0" },
|
||||
{ name = "python-dotenv", specifier = ">=1.0.0" },
|
||||
{ name = "remove-ai-watermarks", extras = ["gpu", "dev"], marker = "extra == 'all'" },
|
||||
{ name = "remove-ai-watermarks", extras = ["gpu", "detect", "dev"], marker = "extra == 'all'" },
|
||||
{ name = "rich", specifier = ">=13.0.0" },
|
||||
{ name = "ruff", marker = "extra == 'dev'", specifier = ">=0.4.0" },
|
||||
{ name = "safetensors", marker = "extra == 'gpu'" },
|
||||
@@ -2086,7 +2225,7 @@ requires-dist = [
|
||||
{ name = "transformers", marker = "extra == 'gpu'", specifier = ">=4.35.0" },
|
||||
{ name = "ultralytics", marker = "extra == 'gpu'", specifier = ">=8.0.0" },
|
||||
]
|
||||
provides-extras = ["gpu", "dev", "all"]
|
||||
provides-extras = ["gpu", "detect", "dev", "all"]
|
||||
|
||||
[[package]]
|
||||
name = "requests"
|
||||
|
||||