mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-15 14:00:54 +02:00
Compare commits
No files matched your search
@@ -50,3 +50,9 @@ data/samsung_capture/captures/samsung_content_*
|
||||
# (GFPGAN wrote RetinaFace/parsing weights to a CWD ./gfpgan/weights/ working
|
||||
# dir on first use). Runtime artifact, never committed.
|
||||
gfpgan/
|
||||
|
||||
# Qwen ControlNet experiment outputs (throwaway eval; never the committed corpus)
|
||||
scripts/_qwen_exp_out/
|
||||
|
||||
# Local-only working data for analysis (not a committed corpus; never tracked)
|
||||
data/spaces/
|
||||
@@ -18,13 +18,13 @@ Consequences for contributors (do not drift back into the stock niche just becau
|
||||
## How to run
|
||||
|
||||
- `uv run remove-ai-watermarks all <image.png> -o <output.png>` — full pipeline (visible + invisible + metadata). Same diffusion knobs as `invisible` below, plus the visible-pass `--inpaint/--no-inpaint`/`--inpaint-method`. **When the `[gpu]` extra is absent, step 2 (invisible/SynthID) is skipped** — `all` still writes an output (visible mark + metadata stripped) but prints a prominent end-of-run banner ("the invisible (SynthID) watermark was NOT removed") AND exits **non-zero** (1), so a skipped SynthID pass is not mistaken for a clean result (the recurring #14/#47 trap, where the old quiet inline warning was missed). `invisible` already hard-errors without the extra; only `all` continued, hence the loud end-banner. Regression-guarded by `tests/test_cli.py::TestAllCommand::test_all_loud_warning_and_nonzero_exit_when_gpu_missing`. **Test trap:** any `all` test that exercises the full pipeline MUST `patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True)` — CI installs core+dev only (no `[gpu]`), so an unpatched `all` test takes the skip branch and now hits the non-zero exit. This passed locally (gpu present → `is_available()` True) but red-failed every matrix cell on the v0.11.0 commit (`test_all_basic`/`test_all_visible_step_uses_registry` asserted exit 0); both now patch `is_available` True.
|
||||
- `uv run remove-ai-watermarks invisible <image.png> -o <out.png>` — diffusion SynthID removal. **Full knob set** (kept identical across `invisible`/`all`/`batch`): `--strength` (vendor-adaptive default), `--steps`, `--guidance-scale` (CFG, default 7.5), `--pipeline sdxl|controlnet` (default `controlnet`), `--controlnet-scale`, `--model` (HF model id, default SDXL base), `--device`, `--seed`, `--hf-token`, `--max-resolution`/`--min-resolution`, `--upscaler lanczos|esrgan`, `--humanize` (Analog Humanizer grain), `--unsharp` (final sharpen), and `--adaptive-polish/--no-adaptive-polish` (**ON by default**; detail-targeted polish that self-gates to a no-op where there is no deficit). `--auto` is deprecated and now a no-op that only warns (the polish it used to enable is ON by default).
|
||||
- `uv run remove-ai-watermarks invisible <image.png> -o <out.png>` — diffusion SynthID removal. **Full knob set** (kept identical across `invisible`/`all`/`batch`): `--strength` (vendor-adaptive default), `--steps`, `--guidance-scale` (CFG, default 7.5), `--pipeline sdxl|controlnet|qwen` (default `controlnet`; `qwen` is a manual opt-in only — see the qwen note in the module map), `--controlnet-scale`, `--model` (HF model id, default SDXL base), `--device`, `--seed`, `--hf-token`, `--max-resolution`/`--min-resolution`, `--upscaler lanczos|esrgan`, `--humanize` (Analog Humanizer grain), `--unsharp` (final sharpen), `--adaptive-polish/--no-adaptive-polish` (**ON by default**; detail-targeted polish that self-gates to a no-op where there is no deficit), and `--tile/--no-tile` + `--tile-size`/`--tile-overlap` (**OFF by default**; sliding-window tiled diffusion -- the *lossless* alternative to a `--max-resolution` downscale for large inputs that OOM on MPS/GPU. Engages only when the long side exceeds `--tile-size`, default 1024; tiles are feather-blended over `--tile-overlap` px, default 128. Pair with `--max-resolution 0`). `--auto` is deprecated and now a no-op that only warns (the polish it used to enable is ON by default).
|
||||
- `uv run remove-ai-watermarks visible <image.png> -o <out.png>` — known-visible-mark removal, CPU, no GPU. Reverse-alpha based: each mark is removed by inverting its captured alpha map. `--mark auto` (default) picks the strongest detected of the Gemini sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-LEFT, locale-specific — Italian variant calibrated); `--mark gemini` / `--mark doubao` / `--mark jimeng` / `--mark samsung` force one (choices come from the registry). Gemini/Doubao recover pixels exactly with no inpaint at native; **Jimeng and Samsung add an always-on thin residual inpaint over the glyph footprint** (their marks re-rasterize per image, so reverse-alpha alone leaves a faint outline). For arbitrary logos/objects use `erase`. **When `--mark auto` finds no known mark (the common case — ~74% of real uploads carry no registered visible mark), the command does NOT silently re-serve the input as a finished result.** It runs a cheap metadata-only `identify`, prints actionable guidance (if the image carries an invisible/metadata mark, e.g. an OpenAI/Gemini C2PA image, it points to `all`; otherwise it does NOT imply the image is clean -- it warns that an invisible pixel watermark like SynthID cannot be detected once the metadata proxy is gone and routes to both `all` and `erase --region`), writes NO output file, and exits **`EXIT_NO_VISIBLE_MARK` (2)** — distinct from success (0) and a hard error (1) so a wrapping service (raiw.cc) can surface the message instead of treating the unchanged image as done (the production "it didn't work" / score-0 trap). Same handling for an explicit `--mark <name>` that is not detected. Helper `cli._no_visible_mark_exit`; regression-guarded by `tests/test_cli.py::TestVisibleCommand::test_visible_auto_no_mark_exits_two_with_eraser_hint` and `test_visible_auto_no_mark_routes_to_all_when_metadata`. `--no-detect` still forces the gemini fallback and proceeds (exit 0).
|
||||
- `uv run remove-ai-watermarks erase <image.png> --region x,y,w,h -o <out.png>` — universal region eraser (any logo/object, any position). `--backend cv2` (default, no deps) or `--backend lama` (big-LaMa via onnxruntime, extra `lama`); `--region` is repeatable.
|
||||
- `uv run remove-ai-watermarks identify <image>` — provenance verdict (platform + watermark inventory + confidence); `--json` for machine output, `--no-visible` to skip the cv2 sparkle detector
|
||||
- `uv run remove-ai-watermarks metadata <image.png> --check` — inspect AI metadata (C2PA, EXIF, PNG chunks)
|
||||
- `uv run remove-ai-watermarks metadata <image.png> --remove -o <out.png>` — strip all AI metadata
|
||||
- `uv run remove-ai-watermarks batch <directory>` — process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the **full `invisible` knob set above** (`--strength`/`--steps`/`--guidance-scale`/`--pipeline`/`--controlnet-scale`/`--model`/`--device`/`--max-resolution`/`--min-resolution`/`--upscaler`/`--seed`/`--hf-token`/`--humanize`/`--unsharp`/`--adaptive-polish`), plus `--inpaint/--no-inpaint` for the visible pass. `--adaptive-polish` is ON by default; `--auto` is deprecated and a no-op that only warns. One engine cached per pipeline; the polish is resolved once before the loop.
|
||||
- `uv run remove-ai-watermarks batch <directory>` — process every supported image in a directory (output defaults to `<directory>_clean/`, set with `-o`). `--mode visible|invisible|metadata|all` (default `visible`); the invisible/all path reuses the **full `invisible` knob set above** (`--strength`/`--steps`/`--guidance-scale`/`--pipeline`/`--controlnet-scale`/`--model`/`--device`/`--max-resolution`/`--min-resolution`/`--upscaler`/`--seed`/`--hf-token`/`--humanize`/`--unsharp`/`--adaptive-polish`/`--tile`/`--tile-size`/`--tile-overlap`), plus `--inpaint/--no-inpaint` for the visible pass. `--adaptive-polish` is ON by default; `--auto` is deprecated and a no-op that only warns. One engine cached per pipeline; the polish is resolved once before the loop.
|
||||
|
||||
## Test and lint
|
||||
|
||||
@@ -36,7 +36,7 @@ Consequences for contributors (do not drift back into the stock niche just becau
|
||||
- Run `uv run` from the repo root — from another cwd it falls back to a bare env without numpy/cv2/torch.
|
||||
- **Stale `trustmark` remnant in site-packages after an extras change:** the `trustmark` package downloads model weights INTO its own package dir, so when a narrower `uv sync` prunes the package, a `trustmark/models/` directory survives as an empty namespace package. Symptom: pyright `"TrustMark" is unknown import symbol` on `trustmark_detector.py` and `find_spec("trustmark")` returning a loader-less spec (so `is_available()` lies True). Fix: `rm -rf .venv/lib/python3.12/site-packages/trustmark` (regenerable weights cache).
|
||||
- To add a dev tool (pytest/ruff/pyright) into the env, use `uv sync --frozen --extra dev --extra gpu`, **never `uv pip install`** — `uv pip install` re-resolves and rewrites `uv.lock`, which silently bumped `transformers` to a build incompatible with the pinned `diffusers` (`cannot import name 'Qwen3VLForConditionalGeneration'`) and broke every `identify`/metadata import. Recovery: `git checkout uv.lock && uv sync --frozen --extra gpu --extra dev`. The `gpu` extra holds `diffusers`/`transformers`/`torch`, so a bare `uv sync` (no extras) removes them; `noai/__init__` is now **lazy** (PEP 562 `__getattr__`, so importing `identify`/`metadata` no longer pulls `watermark_remover`/torch), so a bare env breaks only when the removal pipeline is actually invoked, not on import. `maintain.sh`'s `uv sync --all-extras` also pulls the heavy `trustmark`/`lama` wheels (pytorch-lightning, onnxruntime) — fine on a good connection, but on flaky DNS sync only `--extra gpu --extra dev` and run the lint/test steps by hand.
|
||||
- Metadata/C2PA tests assert against real committed fixtures in `data/samples/` (`chatgpt-*.png` = OpenAI C2PA, `firefly-1.png` = Adobe, `mj-*` = Midjourney IPTC, `doubao-1.png` = ByteDance Doubao with the China TC260 `<TC260:AIGC>` XMP label **and** a visible "豆包AI生成" text mark bottom-right; `grok-1.jpg` = xAI Grok with its EXIF-only `Signature:` blob + UUID `Artist` and no C2PA/SynthID/IPTC); synthetic byte blobs cover the JPEG/ISOBMFF format paths. The "non-AI / clean photo" control is no longer in `data/samples/` -- the `clean_photo` conftest fixture serves a verified-negative image from the corpus `neg/` set (skips if the corpus is absent).
|
||||
- Metadata/C2PA tests assert against real committed fixtures in `data/samples/` (`chatgpt-*.png` = OpenAI C2PA, `firefly-1.png` = Adobe, `mj-*` = Midjourney IPTC, `doubao-1.png` = ByteDance Doubao with the China TC260 `<TC260:AIGC>` XMP label **and** a visible "豆包AI生成" text mark bottom-right; `grok-1.jpg` = xAI Grok with its EXIF-only `Signature:` blob + UUID `Artist` and no C2PA/SynthID/IPTC; `flux-1.png` / `flux-1.jpg` = real Black Forest Labs FLUX.2 Playground output, signed C2PA (issuer "Black Forest Labs" + `trainedAlgorithmicMedia`) -- `flux-1.jpg` is the first committed **JPEG-with-C2PA** fixture, exercising the c2pa-python non-PNG reader path end to end; whether BFL hosted output also embeds the open DWT-DCT pixel watermark is UNRESOLVED -- our detector returns None on these fox samples, but they are high-texture carriers where even a known-embedded watermark fails the round-trip, see the content-fragility caveat in `docs/watermarking-landscape.md`); synthetic byte blobs cover the remaining JPEG/ISOBMFF format paths. The "non-AI / clean photo" control is no longer in `data/samples/` -- the `clean_photo` conftest fixture serves a verified-negative image from the corpus `neg/` set (skips if the corpus is absent).
|
||||
- SynthID reference corpus: `scripts/synthid_corpus.py` ingests labeled images into `data/synthid_corpus/`. The labeled `images/` (`pos/` `neg/` `cleaned/`) are **committed** (public repo -- review every image for private content before adding; `manifest.csv` is kept in sync with the files on disk, one row per tracked image); only the synthetic `refs/` calibration fills are gitignored. See its README for the collection protocol and verification oracles. **`cleaned/` examples must be produced by a CURRENT shipped removal method** -- the default SDXL img2img pass (optionally `--max-resolution`). Do NOT archive cleaned outputs from methods that are no longer in the pipeline (ctrlregen, the old text/face-protection, IP-Adapter FaceID, CodeFormer) or from the experimental opt-in paths (controlnet, face restore) as corpus examples; a cleaned reference should represent the canonical removal, and a removed method's output is not a reproducible example. Keep those experiment outputs in a local working dir, never in the committed corpus.
|
||||
|
||||
## Configuration
|
||||
@@ -53,15 +53,16 @@ Compact map. The full per-module detail (design decisions, tuned thresholds, cal
|
||||
- `noai/c2pa.py` — C2PA reading. `extract_c2pa_info(path)` uses the official **c2pa-python `Reader`** first (core dep, any container; `read_manifest_store_json` returns the WHOLE store JSON — active + ingredient manifests — so an AI marker on a parent manifest is seen), and falls back to the hand-rolled caBX/CBOR parser (`has_c2pa_metadata` / `extract_c2pa_chunk` / `_extract_c2pa_info_png`) for synthetic/partial blobs the validator rejects or a broken/absent wheel. The registry scan (issuer / source-type / SynthID / soft-binding) is shared by both paths via `_populate_registry_fields`, so the return-dict shape is identical. Do not reimplement chunk parsing; chunk reads are clamped to the remaining file size by design. `extract_c2pa_chunk`/`inject_c2pa_chunk` stay PNG-only (raw caBX bytes, test/extractor use).
|
||||
- `noai/constants.py` — the single `C2PA_AI_VENDORS` registry (+ `C2PA_SOFT_BINDINGS`) from which `C2PA_ISSUERS` / `SYNTHID_C2PA_ISSUERS` / `identify._ISSUER_PLATFORM` are all derived. Add a new vendor as one registry entry; never edit the derived dicts and never add inline.
|
||||
- `metadata.py` — `scan_head(path)` is the shared (memoized) input for every C2PA/AIGC/IPTC byte scan; use it instead of `open().read(1MB)` for any new marker scan. Also home to `synthid_source`, `xai_signature`, `iptc_ai_system`, `aigc_label`, `huggingface_job`, `samsung_genai`, and `remove_ai_metadata` (fail-safe `strip_c2pa_boxes`).
|
||||
- `identify.py` — aggregates every locally-readable signal into one `ProvenanceReport`; `is_ai_generated` is True or None, never asserted False. `import identify` is deliberately light (lazy `noai/__init__`, fits a 512 MB host) — keep heavy imports out. Add capture-camera tokens to `_DEVICE_C2PA_PLATFORM` only when verified against a real C2PA file; editing-app/AI-device signer tokens go to `_SIGNER_C2PA_PLATFORM`; generator/issuer platforms to `C2PA_AI_VENDORS` in `constants.py`. Integrity-clash detection is high-precision by design (only hard generator stamps feed it, source-grouped independence).
|
||||
- `identify.py` — aggregates every locally-readable signal into one `ProvenanceReport`; `is_ai_generated` is True or None, never asserted False. `ProvenanceReport.ai_source_kind` exposes the C2PA digital-source-type split — `"generated"` (trainedAlgorithmicMedia, fully AI) vs `"enhanced"` (compositeWithTrainedAlgorithmicMedia, a real photo with an AI-composited region), else None — so a caller branches full-frame scrub vs region-targeted clean (see `noai/tiling.feather_region_composite` + `WatermarkRemover.remove_watermark(region=...)`). The sparkle provenance threshold is the SHARED `watermark_registry.GEMINI_SPARKLE_TRUST_CONF` (imported, not a private copy) so the provenance "is there a sparkle" verdict and the removal "take the sparkle" decision can never drift. `import identify` is deliberately light (lazy `noai/__init__`, fits a 512 MB host) — keep heavy imports out (the `watermark_registry` constant import stays light: engines are lazy there). Add capture-camera tokens to `_DEVICE_C2PA_PLATFORM` only when verified against a real C2PA file; editing-app/AI-device signer tokens go to `_SIGNER_C2PA_PLATFORM`; generator/issuer platforms to `C2PA_AI_VENDORS` in `constants.py`. Integrity-clash detection is high-precision by design (only hard generator stamps feed it, source-grouped independence).
|
||||
- `watermark_registry.py` — the single catalog of known visible watermarks (gemini / doubao / jimeng / samsung), reverse-alpha based by policy. Add a new visible text mark = one `_text_mark(...)` row + a `TextMarkConfig` with a captured alpha map; do not re-add per-mark `if` branches. `cli._write_bgr_with_alpha` must NOT zero alpha in the watermark bbox (issue #30 white-box regression).
|
||||
- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU): top-K size-weighted fusion candidate selection (`_SELECT_TOPK`), corner-promote, over/under-subtraction guards, false-positive gate, self-verify repair. Detection scores the top-K size-weighted matches by full fusion (spatial+gradient+variance) and keeps the highest — NOT the raw-NCC argmax, which re-admits the tiny-patch FPs the size weight suppresses (the osachub 2026-06-12 sub-0.85 corner-sparkle regression; see `docs/module-internals.md`). Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Gate any removal candidate on a physical brightness check, not the detector alone.
|
||||
- `_text_mark_engine.py` — shared base for the three reverse-alpha text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model).
|
||||
- `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Removal = reverse-alpha (always-align) + thin residual inpaint. A detector-only removal test is insufficient — assert visual residual (the textured-shift tests).
|
||||
- `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Removal = reverse-alpha (always-align) + thin residual inpaint, **with an over-subtraction guard ported from `gemini_engine` (2026-06-20)**: `_reverse_alpha_oversubtracts` predicts the reverse-alpha output PER PIXEL over the glyph body from the INPUT, and when the recovered body lands more than `_OVERSUB_DARK_MARGIN` (25) gray levels below the local ring it abandons the reverse-alpha pixels and inpaints the footprint from the original surroundings (`_inpaint_footprint`) — fixing the dark-pit ghost on dark/mid-tone backgrounds (roadmap P0#8). Predicting per-pixel from the input (not the produced output) keeps a clean full-strength mark byte-identical (no false trip). A detector-only removal test is insufficient — assert visual residual (the textured-shift tests + `tests/test_text_mark_oversubtraction.py`).
|
||||
- `region_eraser.py` — universal region eraser (`erase` CLI): cv2 backend default (no deps), optional big-LaMa via onnxruntime (~3.5-4 GB peak RAM, ~5-6 s/call CPU — does not fit a minimal droplet).
|
||||
- `invisible_watermark.py` — decodes the OPEN DWT-DCT watermarks (SD / SDXL / FLUX) via `imwatermark` (extra `detect`, pulls torch). Fragile: does not survive JPEG re-encode/resize, so it confirms origin only on pristine files.
|
||||
- `invisible_watermark.py` — decodes the OPEN DWT-DCT watermarks (SD / SDXL / FLUX) via `imwatermark` (extra `detect`, pulls torch). Fragile two ways: (1) does not survive JPEG re-encode/resize; (2) **carrier-fragile on a broad class of pristine images** -- a clean encode->decode round-trip recovers 48/48 on chatgpt/firefly/random but FAILS (28-39/48, below the `_MATCH_48`=44 gate) on the FLUX fox, doubao, a flat FLUX generation, AND a clean synthetic flat fill with no watermark. The failure does NOT track texture; it goes with a degenerate **all-ones decode that is a CARRIER ARTIFACT, not a watermark** (synthetic clean image reproduces it). So `detect_invisible_watermark` is **positive-only**: trust a hit; a `None` is inconclusive unless a same-carrier positive-control embed first recovers >=44. Verified 2026-06-19; full caveat in `docs/watermarking-landscape.md`.
|
||||
- `trustmark_detector.py` — Adobe TrustMark open decoder (extra `trustmark`). Do NOT remove the JPEG re-encode false-positive gate — a lone TrustMark hit without it is almost always content noise.
|
||||
- `noai/watermark_remover.py` — `WatermarkRemover` with two diffusion pipelines selected by the explicit `pipeline` ctor arg, never inferred from `model_id`: `sdxl` (plain SDXL img2img) and `controlnet` (SDXL + canny ControlNet, **the DEFAULT since 2026-06-09**). Removal comes from the img2img `strength`; ControlNet only preserves text/face STRUCTURE — SynthID CAN survive controlnet on photoreal content at low strength. No face-restore extra ships, by validated decision (every restore approach looked MORE AI-generated).
|
||||
- `noai/watermark_remover.py` — `WatermarkRemover` with three diffusion pipelines selected by the explicit `pipeline` ctor arg, never inferred from `model_id`: `sdxl` (plain SDXL img2img), `controlnet` (SDXL + canny ControlNet, **the DEFAULT since 2026-06-09**), and `qwen` (Qwen-Image 20B MMDiT img2img, Apache-2.0, CUDA/cloud-class — best **text** preservation (incl. CJK); `_load_qwen_pipeline`/`_run_qwen`, bf16, no MPS fallback; call shape in the pure `_build_qwen_kwargs` using `true_cfg_scale`). Removal comes from the img2img `strength`; ControlNet only preserves text/face STRUCTURE — SynthID CAN survive controlnet on photoreal content at low strength. Qwen CERTIFIED oracle floors (2026-06-20): OpenAI **0.10** (seed-robust, clean on seeds 0-4), Gemini **0.25** (seed 0 verified, pin a seed — Gemini oracle rate-limits volume; higher than the controlnet Gemini floor 0.15). `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically (the old manual `--strength 0.25` workaround is retired). `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via `_qwen_target_size`) — without it the pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (fixed 2026-06-20). **`qwen` is a MANUAL opt-in only — there is NO auto-router.** Measured (`scripts/fidelity_metrics.py`, OCR-CER / ArcFace / LPIPS / Laplacian-var, NOT eyeball): qwen beats controlnet on ONE niche only — **clean body text on a plain background, no faces** (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES (it always has) AND **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes, qwen re-renders and garbles them). So a content `--pipeline auto` router and a faces+text **mixed dual-pass** were prototyped and **DROPPED** (2026-06-20): on the canonical faces+text case controlnet wins every metric incl. text, so mixed loses; and "text→qwen" can't be auto-decided (it is body-vs-display text that matters, undetectable cheaply). qwen stays for callers who KNOW their content is clean-text-heavy and face-free. No face-restore extra ships, by validated decision (every restore approach looked MORE AI-generated). `remove_watermark(region=(x,y,w,h), region_feather=...)` runs the regeneration but feather-composites only the AI box back over the original (via `noai/tiling.feather_region_composite`), preserving the real photo elsewhere — the **AI-enhanced composite** path (`identify` `ai_source_kind == "enhanced"`); the box is supplied by the caller (a C2PA composite manifest carries no reliable machine-readable region, so we do not fabricate one).
|
||||
- `noai/tiling.py` — sliding-window tiled diffusion for large inputs (CLI `--tile`). `WatermarkRemover.remove_watermark` branches to `run_tiled` when `tile` is set AND the long side exceeds `tile_size`, refactoring the single-pass `_generate` into a per-tile `_generate_one` (the ControlNet edge map is rebuilt per tile inside it). Pure helpers `plan_tiles` (uniform-size tiles, last one flush to the edge) and `feather_weights` (strictly-positive separable taper -> partition-of-unity blend) are unit-tested without the model. Also home to `feather_region_composite(base, regenerated, box, *, feather)` — the pure region-targeted compositor for **AI-enhanced composites** (`ai_source_kind == "enhanced"`): blends the regenerated AI box back over the original with a feathered seam, leaving the real photo OUTSIDE the box pixel-exact. It backs `WatermarkRemover.remove_watermark(region=...)` (regenerate ONLY the AI region, not the whole frame); the no-model lossless region path stays `region_eraser.erase`. New tile/region-blend tuning goes in these pure helpers; do not inline blend math into the runner.
|
||||
- `auto_config.py` + the content-detection layer were REMOVED 2026-06-09; `--auto` is a deprecated no-op (controlnet is the default pipeline and the adaptive polish is ON by default and self-gates to a no-op where there is no detail deficit).
|
||||
- `upscaler.py` — optional Real-ESRGAN pre-diffusion super-resolution for small inputs (extra `esrgan`, spandrel only). Manual opt-in; the default `--upscaler` stays `lanczos` and the engine always falls back to Lanczos on absence/error. ESRGAN can degrade faces and thin text.
|
||||
- `image_io.py` — Unicode-safe cv2 IO (issue #17). Every cv2 file read/write in the package routes through `imread`/`imwrite`; do not call `cv2.imread`/`cv2.imwrite` directly. `to_bgr(image)` is the shared channel normalizer — use it instead of inlining `cvtColor` branches.
|
||||
@@ -76,14 +77,14 @@ Who embeds what (C2PA / IPTC / EXIF / TC260 AIGC / xAI signature / open and prop
|
||||
|
||||
Compact list. Full measurements, incident history, and oracle-validation runs live in `docs/known-limitations.md` — **read the relevant section there before changing the diffusion pipelines, strength defaults, resolution handling, or metadata coverage.**
|
||||
|
||||
- `invisible` processes at native resolution for inputs >= 1024px long side and auto-upscales smaller inputs to a 1024px floor (`--min-resolution 0` disables; `--max-resolution N` is an opt-in cap to bound GPU/MPS memory). MPS OOM is memory-tier dependent, not a hard limit: ~24 GB unified memory falls back to CPU (slow but weight-identical output), 32 GB runs native on MPS. The native-vs-cap-vs-floor decision lives in the pure helper `invisible_engine._target_size` — keep the logic there, unit-tested without the model.
|
||||
- `invisible` processes at native resolution for inputs >= 1024px long side and auto-upscales smaller inputs to a 1024px floor (`--min-resolution 0` disables; `--max-resolution N` is an opt-in cap to bound GPU/MPS memory). MPS OOM is memory-tier dependent, not a hard limit: ~24 GB unified memory falls back to CPU (slow but weight-identical output), 32 GB runs native on MPS. The native-vs-cap-vs-floor decision lives in the pure helper `invisible_engine._target_size` — keep the logic there, unit-tested without the model. For large inputs that OOM, `--tile` is the **lossless** alternative to `--max-resolution`: sliding-window diffusion at native resolution, each tile near SDXL's 1024 training size, feather-blended over the overlap (`noai/tiling.py`). It only engages when the long side exceeds `--tile-size`; the geometry (`plan_tiles`) and the blend window (`feather_weights`) are pure and unit-tested (`tests/test_tiling.py`). Caveat: each tile is an independent low-strength regeneration, so at the certified removal strengths (0.20-0.30) tile drift is minimal but not zero; tiling is a memory workaround, not a quality upgrade over a single native pass.
|
||||
- fp16 VAE black-output (issues #29/#41): the fp16-fixed SDXL VAE (`madebyollin/sdxl-vae-fp16-fix`) is swapped in for the default SDXL checkpoint on cuda/xpu fp16, plus a model-agnostic backstop that detects a degenerate (all-black) fp16 output and re-runs once in fp32. cpu/mps run fp32 and never reproduce the bug.
|
||||
- Pyright first run is slow (2-3 min) due to ML deps (torch/diffusers/transformers stubs); full-project `uv run pyright` can stall for many minutes — scope it to changed files.
|
||||
- A third-party PIL plugin autoload (e.g. an HEIF/AVIF plugin) can raise a non-OSError (`ModuleNotFoundError`), not `UnidentifiedImageError`, when opening a file. Code that opens user-supplied or unknown-format files should `except Exception`, not just `OSError`/`UnidentifiedImageError`.
|
||||
- rich was dropped: the CLI + analysis scripts print plain text (`click.echo` / the `scripts/_plain_console.py` shim). `rich` is NOT a dependency — importing it breaks the core+dev CI sync; new scripts must use the shim. No Unicode glyphs / colors / progress bars in CLI output by design.
|
||||
- AVIF/HEIF/JPEG-XL metadata detection is a binary scan; C2PA removal in those containers (and MP4/MOV/M4V) is `noai/isobmff.py`; non-ISOBMFF audio/video (WebM/MP3/WAV/FLAC/OGG) strips losslessly via ffmpeg on PATH. Still NOT built: an `Exif` meta-box *item* (needs `iinf`/`iloc` surgery) and Resemble PerTh audio detection (no presence/confidence flag exists).
|
||||
- AVIF/HEIF/JPEG-XL metadata detection is a binary scan; C2PA removal in those containers (and MP4/MOV/M4V) is `noai/isobmff.py`; non-ISOBMFF audio/video (WebM/MP3/WAV/FLAC/OGG) strips losslessly via ffmpeg on PATH. An AI-generator token in an `Exif` meta-box *item* (bytes in `mdat`/`idat`) is now blanked **in place** by `isobmff.blank_ai_exif_tokens` (same-length space overwrite, piexif-validated so a coincidental II/MM run in pixels is ignored — no `iinf`/`iloc` surgery, mirrors `blank_ai_xmp_packets`); it scrubs the AI-token value only, leaving camera/editor EXIF intact. Still NOT built: Resemble PerTh audio detection (no presence/confidence flag exists).
|
||||
- **SynthID technical reference: `docs/synthid.md`** — primary-source-cited doc covering mechanism (post-hoc encoder/decoder pair, 136-bit payload at 512x512, pixel-space, model weights NOT modified), robustness numbers (arXiv:2510.09263: ~99.98% TPR@0.1%FPR across 30 transforms including JPEG/crop/resize/color/noise), removal attacks and forensic detectability (arXiv:2605.09203: all 6 attacks detectable at >98% TPR@1%FPR), detectability limits (no public decoder, metadata-proxy only), oracle scope, and adoption landscape. Read that doc first before adding notes here.
|
||||
- **SynthID detection is metadata-only.** No local pixel detector is possible by design (Google's decoder is proprietary, trusted-testers only); we read the C2PA companion proxy, which goes quiet once metadata is stripped — a quiet proxy is not proof the pixel watermark is gone. The Gemini app "Verify with SynthID" is the ONLY valid SynthID oracle; `openai.com/verify` is scoped to OpenAI provenance and each vendor's oracle detects only its own content. SynthID survives JPEG re-encode, so GitHub issue attachments remain valid pixel-watermark test subjects. Every spectral/phase detection approach evaluated (reverse-SynthID, our own probes) works only on controlled solid fills, never on real content.
|
||||
- **SynthID detection is metadata-only.** No local pixel detector is possible by design (Google's decoder is proprietary, trusted-testers only); we read the C2PA companion proxy, which goes quiet once metadata is stripped — a quiet proxy is not proof the pixel watermark is gone. Each vendor has its OWN oracle and it detects only that vendor's content: the Gemini app "Verify with SynthID" for Google, `openai.com/verify` for OpenAI. **Validate the OpenAI arm FIRST** — `openai.com/verify` is more accessible (fewer per-check restrictions) and the strongest automation candidate (Playwright / Chrome MCP); the Gemini flow is more manual. Ordering/throughput choice, not a substitution (see `docs/synthid.md`). SynthID survives JPEG re-encode, so GitHub issue attachments remain valid pixel-watermark test subjects. Every spectral/phase detection approach evaluated (reverse-SynthID, our own probes) works only on controlled solid fills, never on real content.
|
||||
- **External AI-vs-real classifier models are out of scope** (decided 2026-05-24): per-generator, degrade off-distribution, and our own light SDXL pass would likely defeat them. Detection stays local + signal-based.
|
||||
- **Default strength is VENDOR-ADAPTIVE, one ladder for BOTH pipelines** (since 2026-06-09): `resolve_strength(strength, vendor)` picks OpenAI **0.20** / Gemini **0.30** / unknown **0.30** when `--strength` is unset; explicit `--strength` always wins. Removal at low strength is content x pipeline dependent, and near-threshold removal is SEED-NON-DETERMINISTIC — pick a strength with margin and oracle-revalidate per content type. Certified controlnet floors (Modal cert 2026-06-04): OpenAI 0.20 (resolution-independent), Gemini 0.30 (only <= 1536px; native large Gemini needs ~0.35+ or a cap).
|
||||
- **`controlnet` is the default pipeline**; `--pipeline sdxl` is the lighter opt-down. Neither pipeline clears all content at low strength (photoreal survives controlnet, flat graphics survive sdxl — the lever is higher strength). A removal-priority caller MUST oracle-validate strength across content types; prod recipe: controlnet + per-vendor floor + FIXED seed. Forensic-stealth caveat (arXiv:2605.09203): defeating the SynthID verifier is NOT forensic invisibility — removal-processed images are flaggable at >98% TPR@1%FPR.
|
||||
@@ -33,7 +33,7 @@ It does **not** target watermarks that protect someone else's paid or copyrighte
|
||||
- **AI metadata stripping** — EXIF, PNG text chunks, C2PA provenance manifests (PNG / JPEG / AVIF / HEIF / JPEG-XL, **MP4 / MOV / M4V / M4A** at the container level, and **WebM / MP3 / WAV / FLAC / OGG** losslessly via ffmpeg), XMP DigitalSourceType
|
||||
- **"Made with AI" label removal** — removes the AI-disclosure metadata that platforms read to apply automatic labels (useful for clearing a false-positive label from a human-edited photograph)
|
||||
- **Analog Humanizer** — optional film grain and chromatic aberration post-processing
|
||||
- **Text and face preservation (default)** — the default pipeline is a canny ControlNet that keeps text and face structure sharp through the removal pass (without copying original pixels, so SynthID is still removed). Use `--pipeline sdxl` for plain SDXL img2img (lighter, no extra model download) on inputs without text or faces. Canny preserves face *structure*, not *identity* (the regenerated face drifts in likeness). The library does not ship a face-restore extra: every approach evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned) regenerated the face via SDXL and made the output look more AI-generated than the cleaned image. The cleaned controlnet output is the least-AI face state achievable without re-introducing SynthID.
|
||||
- **Text and face preservation (default)** — the default pipeline is a canny ControlNet that keeps text and face structure sharp through the removal pass (without copying original pixels, so SynthID is still removed). Use `--pipeline sdxl` for plain SDXL img2img (lighter, no extra model download) on inputs without text or faces. An experimental `--pipeline qwen` runs Qwen-Image (20B, Apache-2.0) img2img, which preserves **text** (including CJK and small text) better than SDXL at equal strength; it is CUDA/cloud-class (does not fit MPS), and its strength floors are not yet certified (pass an explicit `--strength`, especially for Gemini content). Note: measured fidelity (`scripts/fidelity_metrics.py`) shows Qwen wins on text but controlnet preserves **faces** better (Qwen smooths skin more), so Qwen is not a universal upgrade. Canny preserves face *structure*, not *identity* (the regenerated face drifts in likeness). The library does not ship a face-restore extra: every approach evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned) regenerated the face via SDXL and made the output look more AI-generated than the cleaned image. The cleaned controlnet output is the least-AI face state achievable without re-introducing SynthID.
|
||||
- **Batch processing** — process entire directories
|
||||
- **Detection** — three-stage NCC watermark detection with confidence scoring
|
||||
- **Provenance detection (`identify`)** — aggregate C2PA issuer, the C2PA soft-binding forensic-watermark vendor (Adobe TrustMark, Digimarc, Imatag, ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, embedded SD/ComfyUI params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the China TC260 AIGC label (XMP, PNG chunk, EXIF, or JPEG segment), the HuggingFace `hf-job-id` job marker, the SynthID metadata proxy, the C2PA cloud-manifest reference (Adobe Durable Content Credentials, when the embedded manifest is stripped), the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), the open SD/SDXL/FLUX invisible watermark, and (with the `trustmark` extra) the open Adobe TrustMark watermark into one origin-platform + watermark-inventory verdict (`--json` for machine output)
|
||||
@@ -133,7 +133,7 @@ image → encode to latent space (VAE) at native resolution
|
||||
→ decode back to pixels (VAE)
|
||||
```
|
||||
|
||||
- Large inputs run at native resolution (no down-then-up round-trip, which was the main quality loss in issue #10); use `--max-resolution N` only to cap GPU/MPS memory on very large inputs. Small inputs (long side under 1024 px) are auto-upscaled to a 1024 px floor before diffusion, because SDXL distorts on a tiny latent, and the result is restored to the original size (a transparent quality boost). Disable the floor with `--min-resolution 0`. The floor upscale uses Lanczos by default; `--upscaler esrgan` (the `esrgan` extra) runs Real-ESRGAN first for sharper detail and falls back to Lanczos if the extra is absent. ESRGAN is a generic photo/texture GAN with no face/glyph prior, so it is best for photo/texture content -- it can degrade faces (the diffusion pass regenerates them, so the final recovers) and thin text; keep Lanczos for text-heavy inputs.
|
||||
- Large inputs run at native resolution (no down-then-up round-trip, which was the main quality loss in issue #10); use `--max-resolution N` only to cap GPU/MPS memory on very large inputs. For inputs that run out of GPU/MPS memory at native resolution, `--tile` is the lossless alternative to `--max-resolution`: it regenerates the image in overlapping, feather-blended tiles (each near SDXL's 1024 px size) so there is no downscale and no visible seam. It engages only when the long side exceeds `--tile-size` (default 1024; overlap `--tile-overlap`, default 128); pair it with `--max-resolution 0`. Small inputs (long side under 1024 px) are auto-upscaled to a 1024 px floor before diffusion, because SDXL distorts on a tiny latent, and the result is restored to the original size (a transparent quality boost). Disable the floor with `--min-resolution 0`. The floor upscale uses Lanczos by default; `--upscaler esrgan` (the `esrgan` extra) runs Real-ESRGAN first for sharper detail and falls back to Lanczos if the extra is absent. ESRGAN is a generic photo/texture GAN with no face/glyph prior, so it is best for photo/texture content -- it can degrade faces (the diffusion pass regenerates them, so the final recovers) and thin text; keep Lanczos for text-heavy inputs.
|
||||
|
||||
> **Default strength is vendor-adaptive (no flag needed).** The tool reads the C2PA issuer to detect which vendor's SynthID is present and picks the strength accordingly: **OpenAI gpt-image → `0.20`**, **Google Gemini → `0.30`**, **unknown source → `0.30`**. The **same ladder applies to both pipelines** — these are the oracle-certified `controlnet` floors (June 2026 Modal cert, multi-seed). They also cover plain `sdxl`: the two pipelines have opposite hard cases (controlnet leaves SynthID on photoreal, sdxl on flat graphics), but on its own hard case sdxl is the weaker remover, so it needs at least controlnet's strength — using one certified ladder is the safe choice (margin-based for sdxl, not separately certified). The dominant factor is the vendor (Google's SynthID is ~3x more robust). There is no local SynthID detector, so if the oracle still reads SynthID, raise `--strength`; if you care more about preserving fine detail, lower it. (Caveat: Google's `0.30` was validated only at `--max-resolution 1536`; a very large native Gemini image may need ~`0.35`+.)
|
||||
>
|
||||
@@ -354,7 +354,8 @@ remove-ai-watermarks invisible image.png -o clean.png --humanize 4.0 --unsharp 0
|
||||
# Large images run at native resolution; small ones are upscaled to a 1024 floor
|
||||
# first (disable with --min-resolution 0); --upscaler esrgan uses Real-ESRGAN for
|
||||
# that floor upscale (needs the 'esrgan' extra). On a very large image that OOMs the
|
||||
# GPU/MPS, cap the long side: --max-resolution 2048
|
||||
# GPU/MPS, either cap the long side (--max-resolution 2048, lossy) or pass --tile
|
||||
# to regenerate in overlapping feather-blended tiles at native resolution (lossless).
|
||||
# Strength is vendor-adaptive by default (OpenAI 0.20 / Google 0.30, same
|
||||
# for both pipelines); override with --strength. controlnet (text/face
|
||||
# structure preservation) is the default pipeline; --pipeline sdxl opts down
|
||||
@@ -482,7 +483,7 @@ Won't fix:
|
||||
|
||||
- **Visible-mark and metadata removal is lossless.** Reverse-alpha recovers the original pixels under the mark; metadata stripping never touches image data.
|
||||
- **The invisible (SynthID) path is lossy and not guaranteed.** It runs a low-strength SDXL img2img regeneration, so it softens fine detail and is content-dependent. There is no public SynthID decoder, so the tool cannot verify removal locally; confirm with the Gemini app's "Verify with SynthID" oracle and raise `--strength` if it still detects. A vendor can change the scheme at any time, so treat this as an arms race, not a permanent fix.
|
||||
- **Large images are not tiled yet.** The SynthID path runs at the diffusion model's native resolution and caps or upscales larger inputs instead of tiling them, so very large images lose detail. (Nano Banana 2 is natively 1024px; GPT Image 2 supports 4K experimentally.) Tiled diffusion is the planned fix.
|
||||
- **Large images: native by default, opt-in tiling for OOM.** The SynthID path runs at the diffusion model's native resolution; on a memory-constrained GPU/MPS you can either cap the long side with `--max-resolution` (lossy downscale) or pass `--tile` to regenerate in overlapping, feather-blended tiles at native resolution (lossless, no seam). Tiling is a memory workaround, not a quality upgrade over a single native pass: each tile is an independent low-strength regeneration. (Nano Banana 2 is natively 1024px; GPT Image 2 supports 4K experimentally.)
|
||||
- **Out of scope:** defeating trained AI-vs-real classifiers like Hive (see [Threat model](#threat-model)), visible-logo removal from video, and any guarantee that a stripped copy is untraceable server-side.
|
||||
|
||||
## Legal
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
# qwen_in — pipeline-fidelity eval set
|
||||
|
||||
A small, **stable** set of AI-generated images used to compare the diffusion
|
||||
removal pipelines (`controlnet` / `sdxl` / `qwen`) for fidelity with
|
||||
`scripts/fidelity_metrics.py`. Fixing the set in the repo keeps comparisons
|
||||
reproducible across runs and pipelines.
|
||||
|
||||
All four are AI-generated test content (they carry SynthID + C2PA from their
|
||||
generator — verify with `remove-ai-watermarks identify`), same class as the
|
||||
`data/samples/` fixtures. No real-person photos.
|
||||
|
||||
| file | vendor (SynthID) | content | exercises |
|
||||
|---|---|---|---|
|
||||
| `openai_1_original.png` | OpenAI | typography sheet (EN + RU + ZH) | text (multi-script) |
|
||||
| `openai_2_original.png` | OpenAI | Raiw.cc poster | text (EN, small) |
|
||||
| `gemini_1_original.png` | Google | landscape + Chinese sign | text (CJK) |
|
||||
| `gemini_3_original.png` | Google | 3x3 portrait grid | faces (identity / skin texture) |
|
||||
|
||||
## Text ground truth
|
||||
|
||||
`ground_truth.json` (`{basename: text}`) is the **hand-verified** OCR of the
|
||||
text-bearing originals, seeded by `fidelity_metrics.py ocr` and corrected by
|
||||
hand (PaddleOCR mis-reads stylized Cyrillic in particular). It is the reference
|
||||
for the text CER metric — much cleaner than OCR-vs-OCR. Regenerate the seed with:
|
||||
|
||||
uv run scripts/fidelity_metrics.py ocr data/qwen_in/openai_1_original.png \
|
||||
data/qwen_in/openai_2_original.png data/qwen_in/gemini_1_original.png \
|
||||
--langs en,ru,ch --out data/qwen_in/ground_truth.json
|
||||
# then re-verify by hand before trusting it.
|
||||
|
||||
## Compare
|
||||
|
||||
uv run scripts/fidelity_metrics.py compare \
|
||||
--original data/qwen_in/gemini_3_original.png \
|
||||
--variant controlnet=<out>.png --variant qwen=<out>.png --ocr-langs ""
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 8.2 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 9.3 MiB |
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"openai_1_original.png": "This is a longer sample text in English.\nTypography can flow smoothly from large to medium size.\nSmaller lines help demonstrate hierarchy, rhythm, and clarity.\nEven the finest text should remain clean and readable.\n这是一段较长的中文示例文本。\n排版可以从大字号逐渐过渡到中字号。\n更小的文字能够展示层次、节奏与清晰度。\n即使是最小的一行,也应该保持清楚易读。\nЭто более длинный пример текста на русском языке.\nТипографика может плавно переходить от крупного размера к среднему.\nБолее мелкие строки показывают иерархию, ритм и ясность.\nДаже самый маленький текст должен оставаться чистым и читаемым.",
|
||||
"openai_2_original.png": "raiw.cc\nRaiw.cc – The Platform for the Future\nRaiw.cc is a modern project built for those who value quality, innovation, and reliability.\nSecurity & Reliability\nWe use advanced technologies to protect your data and ensure stable service performance.\nInnovation\nWe continuously explore and implement better solutions to help you stay one step ahead.\nCommunity\nRaiw.cc is an active community of like-minded people who share knowledge, experience, and ideas.\nProgress Together\nWhether you're a developer, creator, or dreamer — raiw.cc is your partner in achieving your goals.\nOur mission is to create opportunities and inspire people to reach new heights.\nRaiw.cc\nVisit raiw.cc today\nExplore more possibilities and start your journey to the future!\nGet Started\nSecure\nInnovative\nCommunity\nForward-Thinking\nINNOVATE · CONNECT · CREATE · GROW",
|
||||
"gemini_1_original.png": "每天都是一个新的机会。\n用微笑开始它。\n世界也会向你微笑。"
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 2.3 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 495 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.8 MiB |
@@ -13,7 +13,16 @@ pipelines, strength defaults, or metadata coverage.
|
||||
|
||||
`invisible` pipeline processes at **native resolution for inputs whose long side is >= 1024px**, and **auto-upscales smaller inputs UP to a 1024px floor** (`min_resolution=1024`, the default; `--min-resolution 0` disables) before diffusion -- SDXL img2img distorts badly on a tiny latent (a 381x512 portrait wrecks at native, the #36 follow-up), and the output is restored to the original input size so the floor is a transparent quality boost (it adds time/memory on small inputs). The floor upscale uses Lanczos by default; **`--upscaler esrgan`** (opt-in, the `esrgan` extra) runs Real-ESRGAN first for better detail before the Lanczos resize to the exact target (`upscaler.py` / `InvisibleEngine._esrgan_upscale`, falls back to Lanczos if the extra is absent). `max_resolution=0` (default) means no downscale cap, matching the hosted raiw.cc backend (fal fast-sdxl, no pre-downscale). The old forced downscale-to-1024 -> upscale-back round-trip for LARGE images was the main quality loss (issue #10) and is gone; at strength ~0.05 SDXL img2img does not need a downscale.
|
||||
|
||||
**Final `--unsharp` post-filter (`humanizer.unsharp_mask`, opt-in, default 0):** applied LAST (after the face-restore pass, else it would be smoothed over) to counter the soft/over-smoothed look diffusion + restoration leave (an AI tell); ~0.5-0.8 safe, higher risks halos. Pairs with `--humanize` (grain adds sensor-noise texture, unsharp adds crispness). `--max-resolution N` re-introduces an opt-in long-side cap purely to bound GPU/MPS memory on very large inputs (it reintroduces the lossy round-trip). For huge images that OOM at native, tile-based diffusion is still the proper long-term fix.
|
||||
**Final `--unsharp` post-filter (`humanizer.unsharp_mask`, opt-in, default 0):** applied LAST (after the face-restore pass, else it would be smoothed over) to counter the soft/over-smoothed look diffusion + restoration leave (an AI tell); ~0.5-0.8 safe, higher risks halos. Pairs with `--humanize` (grain adds sensor-noise texture, unsharp adds crispness). `--max-resolution N` re-introduces an opt-in long-side cap purely to bound GPU/MPS memory on very large inputs (it reintroduces the lossy round-trip). For huge images that OOM at native, **`--tile` is the lossless alternative** -- see the tiled-diffusion subsection below.
|
||||
|
||||
### Tiled diffusion for large inputs (`--tile`, issue #10)
|
||||
|
||||
`--tile` (OFF by default; `--tile-size` default 1024, `--tile-overlap` default 128) processes the diffusion pass in overlapping sliding-window tiles instead of one forward pass, so a large image is regenerated at **native resolution** without the OOM and without the lossy `--max-resolution` downscale round-trip. It engages only when the long side exceeds `--tile-size`; a sub-tile image runs a single pass unchanged. `WatermarkRemover.remove_watermark` refactors the single-image `_generate` into a per-tile `_generate_one` (the ControlNet canny edge map is rebuilt per tile, so structure preservation works tile-local) and routes it through `noai.tiling.run_tiled` when tiling is active. The geometry and blend math are pure helpers, unit-tested without the model (`tests/test_tiling.py`):
|
||||
|
||||
- `plan_tiles(w, h, tile_size, overlap)` lays out a row-major grid where every tile is exactly `tile_size` (the last tile on each axis is pulled back flush to the far edge, simply overlapping its predecessor more). Uniform tile size keeps each diffusion pass at SDXL's preferred dimension.
|
||||
- `feather_weights(w, h, overlap)` is a separable linear taper, ~1 in the interior and ramping toward each edge, kept **strictly positive** so the normalised accumulate-and-divide blend (`accum / weight_sum`) is a partition of unity: a region covered by one feathered edge (an image corner) still divides cleanly. Identical (unchanged) tiles therefore reconstruct the input exactly -- the seam-free guarantee, asserted in `test_identity_generate_reconstructs_image`.
|
||||
|
||||
CAVEAT: each tile is an **independent** low-strength regeneration. At the certified removal strengths (0.20-0.30) the per-tile drift is small and the feather blend hides the seams, but tiling is a memory workaround, not a quality upgrade over a single native pass -- a 32 GB MPS box that clears the native UNet peak should prefer no tiling. The MPS->CPU fallback still applies per tile; if the first tile falls back to CPU, the device stays CPU for the rest of the image.
|
||||
|
||||
**Concrete MPS data points (the OOM is memory-tier-dependent, NOT a hard MPS limit):** on a ~24 GB unified-memory machine (verified 2026-05-25, 1254x1254 gpt-image SDXL, fp32) native res OOMs at the *UNet* step (peak ~17 GiB), not only the VAE decode, and the auto-fallback in `img2img_runner` reloads on CPU and finishes (slow, ~13 min) -- the output is still weight-identical and defeats SynthID, so "looks hung/crashed" on Mac is usually this CPU fallback, not a pipeline error. On a **32 GB** unified-memory machine the same default SDXL pass runs entirely on MPS with **no CPU fallback** (verified 2026-05-31, 1122x1402 gpt-image, `all`/default, ~155 s end-to-end), so 32 GB clears the native-res UNet peak that 24 GB could not. Adding `enable_vae_tiling()` alone does NOT prevent the 24 GB OOM (the peak is the UNet, not the VAE). The fast Mac workarounds for memory-constrained machines are fp16 on MPS (roughly halves memory) or `--max-resolution` to cap the long side; neither is wired as the default. The `controlnet` pipeline adds the canny ControlNet weights on top of SDXL, so its peak is a bit higher than the plain `default` pass; the same MPS->CPU fallback covers an OOM. The native-vs-cap-vs-floor decision lives in the pure helper `invisible_engine._target_size(w, h, max_resolution, min_resolution)` (returns `None` for native, a target tuple for a downscale cap OR an upscale floor; cap takes precedence, the floor is skipped on a min>max misconfig) so it is unit-tested (`tests/test_invisible_engine.py::TestTargetSize`, the #10/#15/#36 regression guard) without loading the model -- keep that logic in the helper, don't re-inline it.
|
||||
|
||||
@@ -41,7 +50,7 @@ Metadata detection for AVIF/HEIF/JPEG-XL relies on a binary scan for `C2PA_UUID`
|
||||
|
||||
**Meta-box XMP now handled (`isobmff.blank_ai_xmp_packets`, v0.6.9):** an AI-label XMP packet stored as a meta-box `mime` item (AVIF/HEIF) is blanked in place (overwritten with spaces of the same length, so `iloc` offsets and the coded image stay valid).
|
||||
|
||||
**Still NOT built:** an `Exif` *item* inside the `meta` box (rare -- AI labels are XMP) needs full `iinf`/`iloc` surgery (offset rewrite) with corruption risk -- exiftool (R/W/C for HEIC/AVIF EXIF+XMP, verified on exiftool.org 2026-05-27) would do it but is a non-installed binary dep, so it stays a documented gap.
|
||||
**`Exif` item inside the `meta` box (AVIF/HEIF), now handled in place (2026-06-19):** an AI-generator token in an EXIF item (its TIFF bytes live in `mdat`/`idat`) is blanked by `isobmff.blank_ai_exif_tokens` — it finds EXIF TIFF blocks by their II/MM byte-order header, validates each with **piexif** (a coincidental II/MM run in pixel data won't parse as a TIFF IFD, so it is ignored), and overwrites any `Software`/`Make`/`Artist`/`ImageDescription` value carrying an `AI_GENERATOR_TOKENS` token with spaces of the **same length**. Same-length means every box size and `iloc` offset stays valid and the coded image is untouched — so it avoids the full `iinf`/`iloc` surgery (offset rewrite) that exiftool would need (exiftool is a non-installed binary dep, deliberately not used). It scrubs only the AI-token value; camera/editor EXIF is preserved. Wired into `remove_ai_metadata`'s ISOBMFF path after `blank_ai_xmp_packets`. Limitation: covers the AI-generator-token case (the realistic one); a future xAI-signature-in-meta-box-EXIF (Grok is JPEG-only today) is not separately handled. **Still NOT built:** Resemble PerTh audio detection (no presence/confidence flag exists).
|
||||
|
||||
**Audio watermark DETECTION (Resemble PerTh) was evaluated and NOT built (2026-05-26):** `resemble-perth`'s `PerthImplicitWatermarker.get_watermark()` returns a raw bit-array with **no presence/confidence flag** (clean audio decodes to arbitrary bits too), so reliably distinguishing watermarked-from-clean needs either Resemble's fixed payload or a confidence API -- neither is public, and there's no real Resemble sample to calibrate against. Same wall-class as the SynthID pixel detector: the decode exists, reliable presence-detection does not. (perth's top-level `PerthImplicitWatermarker` is also gated to None unless `librosa` is importable.)
|
||||
|
||||
@@ -122,3 +131,21 @@ See `docs/synthid.md` §5.5 + `docs/controlnet-removal-pipeline-research.md` (ce
|
||||
`controlnet_conditioning_scale` (CLI `--controlnet-scale`, default 1.0) is the structure-preservation knob (higher = closer to the original structure); fp32 on cpu/mps, fp16-fixed VAE on cuda/xpu. The `controlnet` profile is threaded explicitly (`WatermarkRemover(pipeline=...)` / `InvisibleEngine(pipeline=...)`), NOT inferred from `model_id`. This productionizes the `scripts/controlnet_sweep.py` prototype; see `docs/controlnet-removal-pipeline-research.md`.
|
||||
|
||||
**Forensic-stealth caveat still applies** (arXiv:2605.09203): defeating the SynthID verifier is not forensic invisibility -- a "this image went through a removal pipeline" classifier can still flag the output.
|
||||
|
||||
## `qwen` pipeline (experimental, Qwen-Image 20B, certified floors)
|
||||
|
||||
`--pipeline qwen` runs `QwenImageImg2ImgPipeline` on `Qwen/Qwen-Image` (20B MMDiT, Apache-2.0 code AND weights), as an img2img alternative to the SDXL pipelines. Motivation: the controlnet over-regeneration problem above (it plasticizes real photos / loses fine text at the scrub floor). Qwen-Image renders text natively (incl. CJK) and preserves structure markedly better, so at the strength that removes SynthID it damages real content far less.
|
||||
|
||||
The scrub still comes from the img2img `strength` (same lever as SDXL); the call shape lives in the pure `_build_qwen_kwargs` (uses Qwen's `true_cfg_scale`, not SDXL's `guidance_scale` — the CLI `--guidance-scale` maps onto it, and ~4.0 is typical vs the SDXL default 7.5). bf16 on CUDA. It is **CUDA/cloud-class — the 20B does not fit MPS — so `_run_qwen` has NO MPS→CPU fallback** (unlike the SDXL paths). Cost on Modal A100-80GB is ~$0.05-0.10/image vs SDXL.
|
||||
|
||||
**Certified oracle floors (Modal A100-80GB, 2026-06-20):** on native-resolution OpenAI and Gemini cert inputs (`data/qwen_in/`, both controls SynthID-POSITIVE): **OpenAI 0.10** (0.05 and 0.075 still detected; 0.10 clean and SEED-ROBUST — clean on seeds 0-4, so a random seed is safe) and **Gemini 0.25** (0.20 still detected, 0.25 clean on both images; lowered from the 0.30 first measured). Gemini seed-repeat is single-seed (seed 0): the Gemini oracle rate-limits volume, so PIN a seed in production rather than relying on seed-robustness there.
|
||||
|
||||
**Fidelity vs controlnet was MEASURED, not eyeballed (`scripts/fidelity_metrics.py`, text scored against a vision-transcribed ground truth in `data/qwen_in/ground_truth.json` + PaddleOCR on the variants; an initial eyeball read was wrong and overturned by the metrics).** Methodology rule: only compare fidelity at each pipeline's OWN oracle-confirmed scrub floor -- i.e. between outputs where SynthID is actually removed in BOTH (controlnet OpenAI 0.10 / Gemini 0.15; Qwen OpenAI 0.10 / Gemini 0.25). An equal-strength comparison is invalid where it leaves one pipeline un-scrubbed (Qwen at 0.15 does NOT clear Gemini SynthID, so that run was dropped). At those scrub floors:
|
||||
- **Text:** Qwen wins on substantial Latin/mixed-script text -- OCR CER, controlnet vs Qwen: openai_1 (EN+RU+ZH, both 0.10) 0.385 vs **0.241**, openai_2 (EN, both 0.10) 0.341 vs **0.290**. On a SHORT CJK sign (gemini_1, cnet 0.15 / Qwen 0.25) it is a TIE (0.037 vs 0.037 -- both near-perfect; the earlier Qwen 0.000 was at the higher 0.30, not the certified floor).
|
||||
- **Faces:** controlnet wins -- gemini_3, 18 faces (cnet 0.15 / Qwen 0.25): ArcFace identity 0.546 vs 0.382, Laplacian-variance retention 0.62 vs 0.40, face LPIPS 0.09 vs 0.17 (Qwen smooths faces MORE; the gap narrows vs Qwen 0.30 but controlnet still wins clearly).
|
||||
|
||||
**Conclusion: Qwen wins TEXT only for clean body text on a plain background with NO faces; controlnet wins faces AND display/decorative text in a scene. So `qwen` is a MANUAL `--pipeline qwen` opt-in, not a routed lane.** A content `--pipeline auto` router + a faces+text mixed dual-pass were prototyped and DROPPED (2026-06-20): on the canonical faces+text case (the abba poster, faces + display text) controlnet won EVERY metric incl. text (CER 0.114 vs qwen 0.379), so grafting qwen text only hurts; and "text→qwen" is undecidable cheaply (body-vs-display text is what matters). Caveat: `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`, Gemini 0.25), so `--pipeline qwen` gets the 0.25 Gemini floor automatically — the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` now passes an explicit height/width (qwen squished non-square inputs to 1024² without it). Flat-graphic content was not in the sample.
|
||||
|
||||
**Improving Qwen (ship vs improve):** the cited research lives in `docs/qwen-improvement-research.md` -- read it before extending the `qwen` pipeline. Verdict: shippable as an opt-in text lane. **The "add a Qwen-Image ControlNet to fix face smoothing" lead was built, measured, and CLOSED (2026-06-20):** a DiffSynth-Studio Qwen + Apache-2.0 blockwise-canny ControlNet at the Gemini floor 0.25 did NOT restore face skin texture (face Laplacian-variance retention flat 0.40 -> 0.40, 13/16 faces within +-0.02; the SDXL+canny target 0.62 was not approached), because canny carries edges not skin grain and Qwen's higher Gemini floor (0.25 vs SDXL+canny 0.15) forces more smoothing -- and a deep-research sweep confirmed NO permissively-licensed Qwen tile/detail/realism/skin ControlNet exists anywhere (every Qwen conditioning is geometry). So **faces stay on SDXL+controlnet; Qwen is the text lane, not a face fix.** The strongest remaining lead is **Z-Image-Turbo** (6B, Apache-2.0, `ZImageImg2ImgPipeline`, scrub mechanism preserved) -- its own SynthID floor and face/text fidelity are UNMEASURED; that is the next experiment. Non-regenerative high-frequency detail re-injection is NOT safe by assumption (the "clean-output high frequencies do not carry the watermark" claim was refuted) -- it must be oracle-gated. Always validate any improvement at the certified floors with `scripts/fidelity_metrics.py` first.
|
||||
|
||||
**Seed as a quality lever (measured, openai_1 at 0.10, seeds 0-4):** the seed barely moves whole-image fidelity (img LPIPS 0.062-0.065, SSIM 0.855-0.857, PSNR 28.5-28.7 — flat) but does shift TEXT legibility (OCR CER 0.241-0.290, ~17% spread) -- the seed changes WHICH details get regenerated, not the overall level. So a per-image best-of-N-seed selection is a WEAK, text-only lever (pick the lowest-CER seed that still scrubs; fidelity selection needs no oracle). Not worth the N× cost for general use -- pin one decent seed in prod; reserve best-of-N for text-heavy premium cases.
|
||||
@@ -15,7 +15,7 @@ module.
|
||||
|
||||
## `noai/constants.py`
|
||||
|
||||
`noai/constants.py` — PNG_SIGNATURE, C2PA_CHUNK_TYPE, C2PA_SIGNATURES, and `C2PA_AI_VENDORS` — the single `C2paAiVendor` registry of C2PA-signing vendors (issuer byte, resolved org name, the `identify` platform label, and a `synthid` flag), from which `C2PA_ISSUERS`, `SYNTHID_C2PA_ISSUERS` (issuers that pair SynthID with C2PA: Google, OpenAI), and `identify._ISSUER_PLATFORM` are all **derived** — plus `C2PA_SOFT_BINDINGS` (soft-binding `alg` prefix → forensic-watermark vendor: Adobe TrustMark, Digimarc, Imatag, Steg.AI, Microsoft, ...). Add a new C2PA vendor as one `C2PA_AI_VENDORS` entry (never edit the derived dicts), a new soft-binding to `C2PA_SOFT_BINDINGS`; not inline.
|
||||
`noai/constants.py` — PNG_SIGNATURE, C2PA_CHUNK_TYPE, C2PA_SIGNATURES, and `C2PA_AI_VENDORS` — the single `C2paAiVendor` registry of C2PA-signing vendors (issuer byte, resolved org name, the `identify` platform label, and a `synthid` flag), from which `C2PA_ISSUERS`, `SYNTHID_C2PA_ISSUERS` (issuers that pair SynthID with C2PA: Google, OpenAI), and `identify._ISSUER_PLATFORM` are all **derived** — plus `C2PA_SOFT_BINDINGS` (soft-binding `alg` prefix → forensic-watermark vendor: Adobe TrustMark, Digimarc, Imatag, Steg.AI, Microsoft, ...). Add a new C2PA vendor as one `C2PA_AI_VENDORS` entry (never edit the derived dicts), a new soft-binding to `C2PA_SOFT_BINDINGS`; not inline. A vendor that signs under multiple legal names needs one entry PER distinctive issuer byte string: e.g. ByteDance's Volcano Engine is registered both as latin `volcengine` AND the Chinese legal entity `北京火山引擎科技有限公司` (UTF-8; the latin needle misses the Chinese-named certs entirely) — both normalize to the same "ByteDance" needle/platform. ElevenLabs ("Eleven Labs Inc.", pure generative-AI) is registered as a generator. Deliberately EXCLUDED (mined-corpus candidates 2026-06-20, documented in the file): TikTok Inc. (a content-provenance / AI-labeling signer on uploads, not a generator) and PixelBin.io / "Fynd" (an image transform / CDN signer) — registering either as a generator would mis-label human uploads as AI; the `is_ai` verdict keys off the digitalSourceType, which is already honored.
|
||||
|
||||
## `metadata.py`
|
||||
|
||||
@@ -27,7 +27,9 @@ module.
|
||||
|
||||
`identify.py` — the OpenAI rollout caveat is keyed on `_vendor_of(synthid) == "OpenAI"` (not a raw substring over the issuer + verdict blob). `identify(path)` aggregates every locally-readable signal (C2PA issuer→platform, C2PA soft-binding forensic-watermark vendor, **C2PA cloud-manifest reference** via `metadata.c2pa_cloud_manifest` — signal `c2pa_cloud`, **medium**, provenance-only (does NOT set `is_ai`, excluded from `ai_from_metadata` + clash vendors): a C2PA 2.4 Durable-Content-Credentials case where the embedded manifest is stripped but an XMP `dcterms:provenance` pointer to the vendor's cloud manifest store (`_C2PA_MANIFEST_REPOSITORIES`, today `cai-manifests.adobe.com` → "Adobe Content Authenticity") survives, so the credentials stay recoverable server-side; only emitted when no embedded manifest already attributed the file — surfaced on 2 corpus PNGs 2026-06-10 that read fully `unknown` before, IPTC "Made with AI" + IPTC 2025.1 `AISystemUsed`, embedded SD/ComfyUI params, SynthID proxy, xAI/Grok EXIF signature via `metadata.xai_signature`, the China TC260 AIGC label via `metadata.aigc_label`, the HuggingFace `hf-job-id` job marker via `metadata.huggingface_job`, the Samsung Galaxy AI editing marker via `metadata.samsung_genai`, the visible marks — Gemini sparkle plus the ByteDance Doubao 豆包AI生成 / Jimeng 即梦AI / Samsung Galaxy AI "Contenuti generati dall'AI" text marks via the `watermark_registry` — open invisible watermark, Adobe TrustMark via `trustmark_detector`) into one `ProvenanceReport`. `is_ai_generated` is True or None (never asserted False — stripped metadata is not proof of clean origin). The `hf_job`, visible-mark, and Samsung `samsung_genai` signals are **medium** confidence: each lifts an otherwise-Unknown verdict to a tentative AI (`hf_only` / `visible_only` / `samsung_only`, parallel branches; `visible_only` fires on any `visible_*` signal) but is excluded from the high-confidence `ai_from_metadata` set, so none overrides a hard metadata signal.
|
||||
|
||||
**Visible-mark detection** (`check_visible`, signals `visible_sparkle` / `visible_doubao` / `visible_jimeng` / `visible_samsung`): the Gemini sparkle keeps its own file-level path (`_visible_sparkle` → `gemini_engine.detect_sparkle_confidence`, promoted only at confidence ≥ `_SPARKLE_THRESHOLD` 0.5; corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49), while Doubao/Jimeng/Samsung reuse the registry detectors (`_visible_text_marks` → `watermark_registry`, iterating `_VISIBLE_MARK_PLATFORM`), each gated by its own engine NCC threshold via `MarkDetection.detected` (Doubao 0.4, Jimeng 0.45, Samsung 0.4). Doubao/Jimeng are normally also caught by the TC260 AIGC metadata label and Samsung by its C2PA + `genAIType` marker, so the visible path is their stripped-metadata fallback. Visible marks set `platform` only when no harder signal already did, and (like the sparkle) are excluded from integrity-clash vendor claims. The cv2 dependency lives in the engines, not here.
|
||||
**AI-generated vs AI-enhanced** (`ProvenanceReport.ai_source_kind`, roadmap item): the C2PA digital-source-type is split into `"generated"` (trainedAlgorithmicMedia, fully synthetic) vs `"enhanced"` (compositeWithTrainedAlgorithmicMedia, a real photo with an AI-composited region) — the two byte strings are unambiguous (`compositeWithTrainedAlgorithmicMedia` capitalizes the inner "Trained", so a lowercase `trainedAlgorithmicMedia` match is standalone full generation; full generation wins when both appear). `ai_source_kind` is set only when the AI verdict actually came from the C2PA source type (a non-C2PA AI signal — IPTC/AIGC/local gen/xAI — leaves it None). It lets a caller branch a full-frame scrub (`generated`) from a region-targeted clean that preserves the real photo (`enhanced`; see `noai/tiling.feather_region_composite`). The CLI verdict line reads "AI-generated (fully synthetic)" vs "AI-enhanced (real content with an AI-composited region)".
|
||||
|
||||
**Visible-mark detection** (`check_visible`, signals `visible_sparkle` / `visible_doubao` / `visible_jimeng` / `visible_samsung`): the Gemini sparkle keeps its own file-level path (`_visible_sparkle` → `gemini_engine.detect_sparkle_confidence`, promoted only at confidence ≥ `_SPARKLE_THRESHOLD`, which is the SHARED `watermark_registry.GEMINI_SPARKLE_TRUST_CONF` (0.5) — imported, not a private copy, so the provenance detect threshold and the removal `best_auto_mark` / `_gemini_detect` arbitration gate can never drift (the detect-vs-remove desync from roadmap P0#7; regression-guarded by `tests/test_identify.py::TestSparkleDetectRemoveAlignment`, which composites the real demo sparkle at borderline opacities and asserts identify and `best_auto_mark` AGREE on either side of the line). Lowering the gate to recover faint sub-0.5 sparkles was evaluated 2026-06-20 and REJECTED: a real Doubao text mark scores ~0.40-0.42 as a gemini match with a HIGHER core-ring brightness margin than a genuine faint sparkle, so neither confidence nor the brightness gate separates them in the [0.35, 0.5) band — lowering trades a rare miss for false-positive removals on clean images. Corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49), while Doubao/Jimeng/Samsung reuse the registry detectors (`_visible_text_marks` → `watermark_registry`, iterating `_VISIBLE_MARK_PLATFORM`), each gated by its own engine NCC threshold via `MarkDetection.detected` (Doubao 0.4, Jimeng 0.45, Samsung 0.4). Doubao/Jimeng are normally also caught by the TC260 AIGC metadata label and Samsung by its C2PA + `genAIType` marker, so the visible path is their stripped-metadata fallback. Visible marks set `platform` only when no harder signal already did, and (like the sparkle) are excluded from integrity-clash vendor claims. The cv2 dependency lives in the engines, not here.
|
||||
|
||||
**`import identify` is deliberately light** (~26 MB; ~36 MB with cv2 loaded by a visible-mark run, ~106 MB for a full `check_visible` run): it imports the `noai.c2pa`/`noai.constants` submodules, and `noai/__init__` is lazy (see "Test and lint"), so torch/diffusers are NOT pulled at import even in a full `gpu`/`detect` install — fits a 512 MB host. `noai.c2pa` does eagerly import the **c2pa-python** binary (Rust + cryptography, ~+5 MB RSS, no torch) for the primary `Reader` path — light enough to stay on the dependency-light host; a broken/absent wheel degrades to the byte-scan parser (`reader_available()` False). The heavy paths are opt-in: `check_invisible=True` needs the `detect`/`trustmark` extras (each pulls **torch**; TrustMark also **downloads weights**), so on a core-only deploy leave `check_invisible` off (it is a no-op there anyway). Before the lazy `__init__`, the mere presence of torch in the env inflated `import identify` to ~420 MB.
|
||||
|
||||
@@ -103,7 +105,11 @@ The 11 survivors are near-white ill-conditioning (reverse-alpha divides by `1-a`
|
||||
|
||||
`_text_mark_engine.py` — **shared base for the three reverse-alpha text-mark engines (Doubao/Jimeng/Samsung), extracted 2026-06-09** (they were ~90% byte-identical clones). `TextMarkEngine(config: TextMarkConfig)` owns the whole `locate → extract_mask → detect → _fixed/_aligned_alpha_map → _apply_reverse_alpha → remove_watermark_reverse_alpha` pipeline (+ the asset-keyed `load_alpha_template`/`glyph_silhouette`/`template_match_score` caches). Each engine module is now a thin subclass: it supplies only its `TextMarkConfig` (the tuned constants, the bundled asset, and the bounded structural deltas — `corner` br/bl, `margin_floor` 4/2, `morph_open_size` 5/3, `min_gw` 8/16) plus the test-facing module shims (`_alpha_template`/`_glyph_silhouette`/`_template_match_score` + the constants). Behavior is byte-exact vs the old per-engine code (the three engine test suites pass unchanged). Gemini stays a SEPARATE engine (its multi-size fixed-slot sparkle model is genuinely different). Add a new text mark = a new `TextMarkConfig` + a thin subclass + one registry `_text_mark(...)` row. The engine bullets below describe each mark's calibration history; the LOGIC lives here.
|
||||
|
||||
**`_apply_reverse_alpha` runs on the glyph crop only:** `amap` is zero outside the glyph `region` (x, y, w, h), so the blend is a no-op there (`(wm - 0)/(1 - 0) == wm`, and a uint8→float32→uint8 round-trip is exact). It copies the frame through and computes the reverse-alpha math on the `region` crop only — byte-identical to the old full-frame pass (verified: Doubao 130 + Jimeng 22 placements, 0 mismatches) but O(glyph) not O(image). The full-frame pass cost ~275 ms on a 12 MP frame for a glyph that is <0.1% of it, once per candidate placement (fixed + aligned ≈ 2×/removal); the crop drops that to ~2 ms. Mirror of the Gemini `_core_and_bg` crop. `remove_watermark_reverse_alpha` passes the `region` each `_fixed/_aligned_alpha_map` returns.
|
||||
**`_apply_reverse_alpha` runs on the glyph crop only:** the blend is a no-op outside the glyph `region` (x, y, w, h) (`(wm - 0)/(1 - 0) == wm`, and a uint8→float32→uint8 round-trip is exact). It copies the frame through and computes the reverse-alpha math on the `region` crop only — byte-identical to the old full-frame pass (verified: Doubao 130 + Jimeng 22 placements, 0 mismatches) but O(glyph) not O(image). The full-frame pass cost ~275 ms on a 12 MP frame for a glyph that is <0.1% of it, once per candidate placement (fixed + aligned ≈ 2×/removal); the crop drops that to ~2 ms. Mirror of the Gemini `_core_and_bg` crop.
|
||||
|
||||
**Over-subtraction guard (`_reverse_alpha_oversubtracts` → `_inpaint_footprint`, ported from `gemini_engine` 2026-06-20, roadmap P0#8):** on a dark or mid-tone background the captured alpha can over-estimate THIS image's mark opacity, and reverse-alpha leaves a darker-than-background glyph ghost (a "dark pit") instead of recovering the true pixels — the sparkle-only fix (commit 41f6797) left the text marks unhandled. After `remove_watermark_reverse_alpha` selects the winning placement, the guard PREDICTS the reverse-alpha output PER PIXEL over the glyph body from the INPUT (`(obs - a*logo)/(1-a)`, exactly the remover's math) and, when the predicted body lands more than `_OVERSUB_DARK_MARGIN` (25) gray levels below the local background ring, abandons the reverse-alpha output for the footprint and inpaints it from the ORIGINAL surroundings (`_inpaint_footprint`, a wider dilate/radius than the thin residual pass). Predicting per-pixel from the INPUT (not the produced output, which depends on which placement the remover picked) is what keeps a cleanly captured full-strength mark byte-identical — it predicts back to the background everywhere, so the guard never trips on it (verified across Doubao/Jimeng/Samsung on white/mid/dark/midgray backgrounds). A faint mark predicts a body far below the ring and diverts to the inpaint. Regression-guarded by `tests/test_text_mark_oversubtraction.py` (predicate True on faint / False on clean, end-to-end no-dark-pit acceptance, clean-mark byte-identity, textured-background recovery). A flat synthetic background cannot exhibit the residual-inpaint failure (inpaint-from-flat is perfect regardless), so the value shows on textured/real content where the footprint inpaint samples un-darkened original pixels instead of the darkened reverse-alpha halo.
|
||||
|
||||
**`_fixed/_aligned_alpha_map` and `extract_mask` return footprint-sized arrays, not full frames (memory):** the alpha-map helpers return the glyph-sized alpha **block** (`(gh, gw)` float32) plus its placement `(ax, ay, gw, gh)`, and `extract_mask` returns the box-sized glyph mask (`(loc.h, loc.w)` uint8) — both used to allocate a full `(h, w)` array that is read only inside the small glyph/box. A full-frame float32 alpha map is ~48 MB on a 12 MP frame and two were held at once during removal (fixed + aligned ≈ 96 MB of mostly-zeros); the box mask was a ~12 MB uint8 allocation rebuilt per text-mark `detect` on the memory-tight `identify` path. `_apply_reverse_alpha` consumes the block directly; the residual inpaint embeds it into one full-frame uint8 mask only at `cv2.inpaint` time (which needs a full-frame mask). Byte-identical to the old full-frame path — the block equals the old map's `[ay:ay+gh, ax:ax+gw]` slice and the box equals the old mask cropped to `loc.bbox` (regression-guarded by `tests/test_text_mark_memory.py`, which reconstructs the old full-frame path inline and asserts equality, so the proof survives a cv2/asset bump). `remove_watermark_reverse_alpha` tracks the winning `region` alongside `best_amap` to place that mask.
|
||||
|
||||
## `doubao_engine.py`
|
||||
|
||||
@@ -175,10 +181,12 @@ Root cause: bad alpha (under-estimated, max ~0.65) + fixed-no-inpaint + tight bo
|
||||
|
||||
## `noai/watermark_remover.py`
|
||||
|
||||
`noai/watermark_remover.py` — the `WatermarkRemover` class has two diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id` -- both use the same SDXL base, `DEFAULT_MODEL_ID`).
|
||||
`noai/watermark_remover.py` — the `WatermarkRemover` class has three diffusion pipelines, selected by the explicit `pipeline` ctor arg (NOT inferred from `model_id`). `sdxl`/`controlnet` share the SDXL base (`DEFAULT_MODEL_ID`); `qwen` is its own base (`QWEN_MODEL_ID`).
|
||||
|
||||
**`sdxl`** (renamed from `default` 2026-06-09; `default` kept as a back-compat alias via `normalize_profile`) runs plain SDXL img2img (`_run_img2img`); it is the lighter opt-down alternative (no ControlNet weights).
|
||||
|
||||
**`qwen`** (`_run_qwen`, `_load_qwen_pipeline`) runs `QwenImageImg2ImgPipeline` on `Qwen/Qwen-Image` (20B MMDiT, Apache-2.0 code AND weights). The scrub still comes from the img2img `strength`; Qwen's value is **text preservation** (incl. CJK and small text). **Metric-measured nuance (2026-06-19, `scripts/fidelity_metrics.py`, do NOT trust the eyeball here — it misled). Compare ONLY at each pipeline's oracle-confirmed scrub floor (outputs where SynthID is removed in BOTH — an equal-strength compare is invalid where it leaves one un-scrubbed; Qwen at 0.15 does not clear Gemini): Qwen wins TEXT (lower OCR CER across EN/RU/ZH, perfect Chinese) but controlnet wins FACES (higher Laplacian-variance retention and lower LPIPS — Qwen smooths faces MORE; ArcFace identity favors controlnet 0.546 vs 0.331 at the Gemini floors).** So Qwen is the better text-preserving remover, NOT a universal fidelity win — controlnet's canny edge map holds face skin detail better. Specifics: bf16 on CUDA (fp16 risks overflow on the 20B MMDiT — see the dtype branch in `__init__`); loads `QWEN_MODEL_ID` unless `--model` is overridden; the call shape lives in the pure module helper `_build_qwen_kwargs` (unit-tested without torch in `tests/test_platform.py::TestQwenKwargs`), which uses Qwen's `true_cfg_scale` (NOT SDXL's `guidance_scale` — the CLI `--guidance-scale` maps onto it; ~4.0 is typical, the SDXL default 7.5 is high for Qwen) and an explicit `negative_prompt` (`_QWEN_PROMPT`/`_QWEN_NEGATIVE`). It is CUDA/cloud-class (the 20B does not fit MPS), so `_run_qwen` has NO MPS->CPU fallback — an error propagates. `_load_qwen_pipeline` raises a clear ImportError if the installed diffusers lacks `QwenImageImg2ImgPipeline`. **CERTIFIED oracle floors (Modal A100-80GB, 2026-06-20): OpenAI 0.10 (seed-robust — clean on seeds 0-4), Gemini 0.25 (seed 0 verified on 2 images; the Gemini oracle rate-limits volume seed-repeat, so PIN a seed in prod). The Gemini floor (0.25) is HIGHER than the certified controlnet Gemini floor (0.15); `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically -- the old manual `--strength 0.25` workaround is retired. `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via the pure `_qwen_target_size`); WITHOUT it the img2img pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (the abba 2816x1536 case came back 1024x1024, distorting the scene and garbling text — fixed 2026-06-20, tested in `TestQwenKwargs`).** Fidelity vs controlnet was measured at the certified floors (`scripts/fidelity_metrics.py`), NOT eyeballed. **`qwen` is a MANUAL opt-in only — there is NO auto-router (one was prototyped and DROPPED, see below).** It wins ONE niche: clean body text on a plain background, NO faces (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES and **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes; qwen re-renders and garbles them). **`--pipeline auto` + a faces+text mixed dual-pass were built and DROPPED (2026-06-20):** on the canonical faces+text case (abba) controlnet wins EVERY metric incl. text, so grafting qwen text would only hurt; and "text→qwen" is undecidable cheaply (it is body-vs-display text that matters). The router/detector/mixed modules were removed; the geometry fix + the Qwen strength ladder were kept (they make the manual `--pipeline qwen` correct). **Do NOT retry "add a Qwen ControlNet to close the face gap" — it was built, measured, and CLOSED 2026-06-20:** a DiffSynth blockwise-canny Qwen ControlNet did not restore face skin texture (lapvar flat 0.40, canny carries edges not skin grain) and no permissively-licensed Qwen tile/detail/skin ControlNet exists anywhere (all conditioning is geometry). Faces stay on controlnet; the next improvement lead is Z-Image-Turbo (Apache-2.0, unmeasured floor). Full record + the deep-research sweep in `docs/qwen-improvement-research.md`.
|
||||
|
||||
**`controlnet`** (**the DEFAULT pipeline since 2026-06-09** for `invisible`/`all`/`batch` and both engine ctors; `_run_controlnet`, `_load_controlnet_pipeline`) runs `StableDiffusionXLControlNetImg2ImgPipeline` with the SDXL-native canny ControlNet `xinsir/controlnet-canny-sdxl-1.0` (`watermark_profiles.CONTROLNET_CANNY_MODEL`): the control image is `cv2.Canny(gray, 100, 200)` stacked to 3 channels (`_CANNY_LOW`/`_CANNY_HIGH`, prompt `_CONTROLNET_PROMPT` / `_CONTROLNET_NEGATIVE`).
|
||||
|
||||
**Removal comes from the img2img regeneration (`strength`); the ControlNet only PRESERVES text and face STRUCTURE via the edge map.**
|
||||
@@ -189,6 +197,14 @@ At the shared low removal strength the canny edge-conditioning keeps the regener
|
||||
|
||||
**But the reverse also holds: a flat-graphic logo/poster SURVIVED `default` while clearing controlnet** -- removal at the low strength is content×pipeline dependent and neither pipeline is universally safe; the real lever is a higher strength. See the controlnet Known-limitations bullet for the full table + root cause. Canny holds face STRUCTURE but NOT identity (the regenerated face drifts in likeness -- canny carries edges, not identity). The drifted cleaned face is the LEAST-AI state we can reach without re-introducing SynthID; the library does NOT ship a face-restore extra. Every restore approach we evaluated (GFPGAN-on-cleaned, PhotoMaker-V2 txt2img, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps, 2026-06-04 - 2026-06-08 Modal cert sweeps) regenerated the face from an ArcFace embedding via SDXL diffusion -- which makes the output face look MORE AI-generated, not less. Empirical conclusion in `docs/synthid-robust-identity-research-2026-06-08.md` "Empirical follow-up". For production face preservation, ship the cleaned image as-is. `controlnet_conditioning_scale` (ctor arg, default 1.0) is the structure-preservation knob. Same dtype rule as `default` (fp32 on cpu/mps, fp16 only on cuda/xpu; the fp16-fixed SDXL VAE `_SDXL_FP16_VAE_ID` is swapped in on fp16 GPUs -- issue #29) and the same MPS->CPU fallback (reload on cpu/fp32, drop a non-cpu generator, retry once).
|
||||
|
||||
**Tiled diffusion (`tile`/`tile_size`/`tile_overlap` ctor-path args, CLI `--tile`, issue #10):** for large inputs that OOM at native resolution, `remove_watermark` can process the diffusion pass in overlapping sliding-window tiles instead of one forward pass — the lossless alternative to a `--max-resolution` downscale. The single-image generation closure was refactored into `_generate_one(img)` (dispatches controlnet/img2img, generator shared so the seed advances deterministically across tiles), and `_generate()` routes it through `noai.tiling.run_tiled` when `tile` is set AND `max(init_image.size) > tile_size` (a sub-tile image runs one pass unchanged). The ControlNet canny edge map is rebuilt per tile inside `_generate_one`, so structure preservation is tile-local. See `noai/tiling.py` below and the tiled-diffusion subsection in `docs/known-limitations.md` for the geometry, the partition-of-unity blend, and the quality caveat.
|
||||
|
||||
## `noai/tiling.py`
|
||||
|
||||
Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). `plan_tiles(w, h, tile_size, overlap)` returns a row-major grid of uniform-size `Tile` boxes — every tile is exactly `tile_size` (the SDXL training size), with the last tile on each axis pulled back flush to the far edge (`_axis_positions` clamps a pathological `overlap >= tile` to `tile - 1` so the step stays >= 1). `feather_weights(w, h, overlap)` is a separable linear taper (1 in the interior, ramping toward each edge) floored at `_WEIGHT_EPS` so it is **strictly positive everywhere** — that makes the normalised `accum / weight_sum` blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). `run_tiled(generate_tile, image, tile_size, overlap, set_progress)` is the orchestration loop: crop each planned tile, call `generate_tile` (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (`tests/test_tiling.py`: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
|
||||
|
||||
`feather_region_composite(base, regenerated, box, *, feather)` is the pure region-targeted compositor for **AI-enhanced composites** (roadmap P1#8; `identify` `ai_source_kind == "enhanced"`, digitalSourceType `compositeWithTrainedAlgorithmicMedia`). It blends `regenerated` over `base` inside `box = (x, y, w, h)` with a separable linear taper of `feather` px at the box edges (the taper anchors to ~0 at the boundary, so unlike `feather_weights` it is NOT floored — the result equals `base` EXACTLY outside the box), preserving dtype and supporting HxW or HxWxC. It backs `WatermarkRemover.remove_watermark(region=..., region_feather=...)`: the remover regenerates the frame (or tiles), then composites only the AI box back over the original input, so the real photo outside the box stays pixel-exact and only the AI region is scrubbed. The box is caller-supplied (a C2PA composite manifest carries no reliable machine-readable region); the no-model lossless region path remains `region_eraser.erase`. Unit-tested in `tests/test_tiling.py::TestFeatherRegionComposite` (outside-box exactness, interior == regenerated, hard-paste at feather 0, monotonic seam ramp, dtype/grayscale/clamp/empty-box/shape-mismatch).
|
||||
|
||||
## `auto_config.py` (REMOVED 2026-06-09)
|
||||
|
||||
**`auto_config.py` + the content-detection layer were REMOVED 2026-06-09.**
|
||||
@@ -197,6 +213,15 @@ History: `auto_config.plan()` was a content-adaptive planner that detected faces
|
||||
|
||||
**`--auto` is now a DEPRECATED no-op** (`cli._resolve_auto_polish`): controlnet is already the default pipeline AND the adaptive polish is ON by default, so `--auto` has nothing left to do — it only prints a deprecation warning and passes `adaptive_polish` through unchanged (an explicit `--no-adaptive-polish` still wins). (Originally it re-enabled the polish; once the polish default flipped to ON the same day, the parameter-source branch became dead and was dropped.) The **adaptive polish itself lives on** in `humanizer.adaptive_polish` (CLI `--adaptive-polish/--no-adaptive-polish`, **ON by default since 2026-06-09** — it self-gates to a no-op where there is no detail deficit, so default-on is safe; uses the full-res original as the detail reference) — see the `humanizer` test note. `batch` resolves the polish once before the loop (one warning) and caches the invisible engine per pipeline (`ctx.obj["_inv_engines"]`).
|
||||
|
||||
## Content `--pipeline auto` router + faces+text mixed dual-pass — PROTOTYPED and DROPPED (2026-06-20)
|
||||
|
||||
A `--pipeline auto` content router (`pipeline_router.py` + `content_detect.py`: Haar faces + MSER text → route text→qwen / faces→controlnet / both→mixed) and a faces+text **mixed dual-pass** (`mixed_pipeline.py`: scrub the whole frame on BOTH pipelines, then graft the qwen text regions onto the controlnet base via `tiling.feather_region_composite`) were built, run on Modal (the abba poster: faces + display text), measured, and **removed**. Why it failed:
|
||||
- On the canonical faces+text image **controlnet wins EVERY metric, including text** (CER 0.114 vs qwen 0.379; ID 0.64 vs 0.36; lapvar 0.71 vs 0.59) — canny holds the existing letter shapes, qwen re-renders display/decorative text and garbles it. So grafting qwen text onto the controlnet base only HURTS.
|
||||
- qwen beats controlnet on text ONLY for clean body text on a plain background with no faces (openai_1/2) — a niche where there are no faces to route around anyway, so `--pipeline qwen` alone covers it. The faces+clean-body-text intersection is near-empty.
|
||||
- "text→qwen" is not cheaply decidable: it is body-vs-display text that matters, which face/text detectors can't tell apart. MSER also over-fired (47% of the busy poster, incl. faces).
|
||||
|
||||
KEPT from that work (independently valid for the manual `--pipeline qwen`): the qwen **geometry fix** (`_qwen_target_size` + `_build_qwen_kwargs` height/width — qwen squished non-square inputs to 1024² without it) and the **pipeline-aware `resolve_strength`** Qwen ladder (Gemini 0.25). Also kept: the `fidelity_metrics.py` one-to-one face matcher. The throwaway Modal eval scripts were removed after the run (findings recorded here and in `docs/qwen-improvement-research.md`).
|
||||
|
||||
## `upscaler.py`
|
||||
|
||||
`upscaler.py` — optional Real-ESRGAN pre-diffusion super-resolution for small inputs (spandrel boundary, top-of-file pyright pragma). `is_available()` gates on spandrel+torch (via `importlib.util.find_spec`); `upscale(bgr, device=None)` loads a lazily-built spandrel `ImageModelDescriptor` singleton (double-checked lock) and upscales by the model's native factor (x2), with a non-CPU→CPU device fallback mirroring the diffusion engine's MPS→CPU retry. Weights (`RealESRGAN_x2plus.pth`, BSD-3-Clause) download on first use to the `torch.hub` checkpoints cache; never bundled. Used only when UPscaling to the `min_resolution` floor (a `max_resolution` downscale always uses Lanczos). The wiring is `InvisibleEngine._esrgan_upscale(pil, target)` — Real-ESRGAN at native factor, then a Lanczos resize to the exact target, falling back to a plain Lanczos resize if the extra is absent or the model errors (so an optional upscaler can never break removal). The default `--upscaler` is `lanczos` (cv2, no deps).
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
# Qwen-Image improvement research (2026-06-20)
|
||||
|
||||
Cited research behind the decision **"ship the `qwen` pipeline as-is, or improve it
|
||||
first?"** Produced by the multi-source deep-research harness (5 search angles, 22
|
||||
sources fetched, 85 claims extracted, 25 verified by a 3-vote adversarial check, 20
|
||||
confirmed / 5 killed, 104 agent calls). Findings carry their confidence and vote.
|
||||
|
||||
## Context
|
||||
|
||||
The `qwen` pipeline runs base Qwen-Image (20B MMDiT, Apache-2.0) as a low-strength
|
||||
img2img scrub (removal comes from the denoising `strength`). Certified oracle scrub
|
||||
floors: OpenAI 0.10 (seed-robust), Gemini 0.25 (pinned seed). Measured against the
|
||||
SDXL + canny-ControlNet pipeline (`scripts/fidelity_metrics.py`): Qwen preserves
|
||||
**text** markedly better (incl. CJK and Cyrillic, lower OCR CER) but preserves
|
||||
**faces** worse, smoothing skin (Laplacian-variance retention 0.40 vs 0.62, face
|
||||
LPIPS 0.17 vs 0.09, ArcFace identity 0.38 vs 0.55 at the scrub floors). The goal of
|
||||
the research: keep Qwen's text advantage while fixing the face-smoothing, and judge
|
||||
production-readiness.
|
||||
|
||||
## Verdict
|
||||
|
||||
Base Qwen-Image is **shippable now as an opt-in text-content lane** (Apache-2.0 on
|
||||
code and weights, scrub lever confirmed), but it is not a universal upgrade (it loses
|
||||
faces). The strongest verified improvement path is to **add structure conditioning**
|
||||
(a Qwen-Image ControlNet) to the existing base pass, the direct analog of the SDXL +
|
||||
canny conditioning that wins on faces. Separately, **Z-Image / Z-Image-Turbo** (6B,
|
||||
Apache-2.0) is the best-verified lighter alternative to evaluate before committing to
|
||||
the 20B cost. None of the improvements has measured face-fidelity numbers at our
|
||||
scrub floors yet, so each must be validated with `scripts/fidelity_metrics.py` plus
|
||||
the oracle before shipping.
|
||||
|
||||
## Follow-up: ControlNet experiment + deeper research (2026-06-20)
|
||||
|
||||
The verdict's strongest lead -- adding a Qwen-Image ControlNet -- was **built, measured, and
|
||||
CLOSED**.
|
||||
|
||||
**Experiment** (Modal A100-80GB; DiffSynth-Studio `QwenImagePipeline` + the Apache-2.0
|
||||
`DiffSynth-Studio/Qwen-Image-Blockwise-ControlNet-Canny` -- the only framework exposing
|
||||
Qwen-Image + canny ControlNet + img2img `denoising_strength` in ONE call; diffusers ships no
|
||||
`QwenImageControlNetImg2ImgPipeline`, its three Qwen ControlNet pipelines are txt2img only).
|
||||
Measured on `gemini_3` (18 faces) at the Gemini scrub floor 0.25 vs base-Qwen 0.25 with
|
||||
`scripts/fidelity_metrics.py`:
|
||||
- **The actual failure mode (face skin texture) was NOT restored:** Laplacian-variance
|
||||
retention stayed flat (base 0.40 -> qwen+canny 0.40; per-face 13/16 within +-0.02 after a
|
||||
one-to-one face match, sd 0.016 -- not an averaging artifact). The SDXL+canny target 0.62
|
||||
was not approached.
|
||||
- Identity rose modestly and broadly (ArcFace 0.346 -> 0.415, 12/16 faces improved) but the
|
||||
absolute stays ~0.42 ("a different person, slightly closer").
|
||||
- Mechanism (verified, not inferred): canny conditioning was applied fully (scale 1.0, full
|
||||
denoise schedule); the canny edge map is clean facial geometry with BLANK skin (4.83% edge
|
||||
density) -- canny carries edges, not skin grain. Root cause: Qwen's Gemini floor (0.25) is
|
||||
higher than SDXL+canny's (0.15), forcing more denoising -> more smoothing; structure
|
||||
conditioning cannot compensate for that.
|
||||
|
||||
**Deeper research** (deep-research harness, 103 agents, 3-vote adversarial):
|
||||
- **[high, unanimous] No permissively-licensed Qwen-Image tile / detail / realism / skin
|
||||
ControlNet exists anywhere** -- DiffSynth first-party is Canny/Depth/Inpaint only, InstantX
|
||||
Union is canny/soft-edge/depth/pose, the official QwenLM repo ships none. Every Qwen
|
||||
conditioning is GEOMETRY, the same class as the tested canny. **The "add a Qwen ControlNet to
|
||||
fix faces" lead is closed for good.**
|
||||
- **[high, unanimous] Z-Image / Z-Image-Turbo (6B, Apache-2.0 on code AND weights, ~1/3 of
|
||||
Qwen 20B)** ships a documented `ZImageImg2ImgPipeline` with standard strength denoising, so
|
||||
it preserves the scrub mechanism. Its own SynthID scrub floor and face/text fidelity are
|
||||
UNMEASURED -- this is the strongest concrete NEXT experiment.
|
||||
- **[medium] Lowering Qwen's scrub floor has no off-the-shelf SynthID answer:** the "partial
|
||||
img2img ~0.3 breaks robust watermarks" literature tests open schemes
|
||||
(StegaStamp/TrustMark/VINE), NEVER SynthID (proprietary decoder) -- analogy, not proof. No
|
||||
minimal-strength SynthID attack under a named permissive license was found.
|
||||
- **REFUTED [0-3]:** "re-injecting high-frequency detail from a clean diffusion output would
|
||||
not carry the watermark back." So non-regenerative detail transfer is NOT safe by
|
||||
assumption -- the transferred high-frequency band must be gated against the SynthID oracle.
|
||||
|
||||
**Net for the pipeline:** **faces stay on SDXL+controlnet**; there is no Qwen face-fix.
|
||||
The live frontier is Z-Image-Turbo (next experiment) and oracle-gated non-regenerative detail
|
||||
re-injection.
|
||||
|
||||
**Follow-up (2026-06-20) — the content-routed lane / mixed dual-pass was tested and DROPPED.**
|
||||
A `--pipeline auto` router (Haar+MSER → text→qwen / faces→controlnet / both→mixed) and a
|
||||
faces+text mixed dual-pass (scrub the whole frame on both, graft qwen text regions onto the
|
||||
controlnet base) were built and run on Modal (the abba poster: faces + display text). On that
|
||||
canonical faces+text case **controlnet won EVERY metric, including text** (CER 0.114 vs qwen
|
||||
0.379; ID 0.64 vs 0.36) — canny holds existing letter shapes, qwen re-renders display text and
|
||||
garbles it, so grafting qwen text only hurts. Qwen beats controlnet on text ONLY for clean body
|
||||
text on a plain background with no faces (openai_1/2), a niche `--pipeline qwen` alone covers;
|
||||
the faces+clean-body-text intersection is near-empty, and "text→qwen" is undecidable cheaply
|
||||
(body-vs-display text is what matters). So the router + mixed modules were removed and **`qwen`
|
||||
is a manual `--pipeline qwen` opt-in only.** KEPT (independently valid): the qwen geometry fix
|
||||
(it squished non-square inputs to 1024²), the pipeline-aware `resolve_strength` Qwen ladder, and
|
||||
the `fidelity_metrics.py` one-to-one face matcher below.
|
||||
|
||||
**Tooling fix surfaced by this run:** `scripts/fidelity_metrics.py` face matching was changed
|
||||
from per-face nearest-center to a collision-free one-to-one assignment
|
||||
(`assign_faces_one_to_one`, gated by face size), after the 18-face `gemini_3` exposed
|
||||
collisions (the regenerated variants detected 17 faces, so two originals mapped to the same
|
||||
variant face, corrupting the identity metric). lapvar/LPIPS were always anchored to the
|
||||
original bbox and stayed collision-immune. Regression-guarded by
|
||||
`tests/test_fidelity_matching.py`.
|
||||
|
||||
## Findings
|
||||
|
||||
1. **[high, 3-0] A permissively-licensed Qwen-Image ControlNet exists today and is
|
||||
CUDA/diffusers-runnable.** InstantX Qwen-Image-ControlNet-Union supports
|
||||
canny/soft-edge/depth/pose; DiffSynth-Studio maintains blockwise Canny/Depth/Inpaint
|
||||
plus an In-Context-Control-Union; diffusers exposes `QwenImageControlNetPipeline`
|
||||
and `QwenImageMultiControlNetModel` with `controlnet_conditioning_scale` (default
|
||||
1.0) and `control_guidance_start`/`end`. This is the direct analog of the certified
|
||||
SDXL+canny structure conditioning that wins on faces. Caveat: canny/depth preserve
|
||||
geometric structure, not face identity per se, and none is a **tile**-ControlNet
|
||||
(the variant most tied to fine-detail/skin retention in the SDXL world).
|
||||
Sources: InstantX/Qwen-Image-ControlNet-Union, InstantX/Qwen-Image-ControlNet-Inpainting,
|
||||
DiffSynth-Studio Qwen-Image docs, diffusers qwenimage pipeline docs.
|
||||
|
||||
2. **[high, 3-0] The scrub mechanism is preserved, and the license is clean.**
|
||||
`QwenImageImg2ImgPipeline.strength` (default 0.6, range 0-1; DiffSynth names it
|
||||
`denoising_strength`) keeps the partial-regeneration scrub the project relies on,
|
||||
lower values staying closer to the input. Qwen-Image and Qwen-Image-Edit-2509 are
|
||||
Apache-2.0 on both code and weights.
|
||||
|
||||
3. **[medium, mixed 2-1 / 3-0] Qwen-Image-Edit improves identity consistency, but that
|
||||
is not proof it fixes our metric.** The instruction-edit pipeline (2511 better than
|
||||
2509) improves identity/character consistency, but only for identity *through edits*
|
||||
of an input portrait, which is not the same as measured face-skin Laplacian/LPIPS
|
||||
fidelity at a low scrub strength. Architecture: 20B base + Qwen2.5-VL (semantic
|
||||
control) + VAE Encoder (appearance control). Several stronger edit-model face claims
|
||||
were refuted (see below).
|
||||
|
||||
4. **[high, 3-0] Z-Image / Z-Image-Turbo is the best-verified lighter alternative.** A
|
||||
6B model (~1/3 of Qwen-Image's 20B), Apache-2.0 on code and weights, strong bilingual
|
||||
(Chinese + English) native text rendering, with an official diffusers
|
||||
`ZImageImg2ImgPipeline` exposing the same 0-1 denoising-strength scrub lever; Turbo
|
||||
runs at ~8 steps (guidance_scale=0.0) vs ~40. A material cost/footprint reduction vs
|
||||
20B/A100-80GB (but see caveat 4 on the refuted consumer-GPU claim).
|
||||
|
||||
5. **[high, 3-0] EliGen-V2 is NOT relevant** to the face-smoothing problem. It is an
|
||||
entity-level/regional control model (LoRA + regional attention placing entities via
|
||||
text + mask maps, plus entity-level inpainting); it provides no
|
||||
ControlNet/canny/depth/tile structure conditioning or face-skin-detail retention.
|
||||
|
||||
6. **[medium, 2-1] flymy-ai/qwen-image-realism-lora** is Apache-2.0 (code+weights) on
|
||||
base Qwen-Image, so it is permissively usable with the existing base img2img pass,
|
||||
but it is NOT verified to specifically fix the face/skin-smoothing failure mode.
|
||||
|
||||
## Caveats
|
||||
|
||||
1. The research did NOT surface verified evidence for two things specifically asked:
|
||||
(a) a Qwen-Image **tile**-ControlNet (the variant most tied to fine-detail/skin
|
||||
retention; only canny/soft-edge/depth/pose/inpaint were confirmed), and (b) any
|
||||
**non-regenerative detail-restoration** technique (high-frequency residual transfer,
|
||||
guided filtering) that recovers smoothed faces without re-introducing the watermark.
|
||||
Research angle 4 produced zero surviving claims, so it is unanswered.
|
||||
2. No claim provides measured face-fidelity numbers (ArcFace/LPIPS/Laplacian) for ANY
|
||||
recommended intervention at the project's scrub floors. All fidelity evidence is the
|
||||
project's own internal measurement. The improvements are mechanistically sound but
|
||||
unproven for this exact metric, so validate with `scripts/fidelity_metrics.py`
|
||||
before shipping.
|
||||
3. Several vendor model cards are marketing-register primary sources (Qwen blog,
|
||||
Z-Image card). Load-bearing facts (license, params, API levers) are independently
|
||||
corroborated, but comparative quality framings are author glosses.
|
||||
4. Z-Image's "sub-second" figure is H800-specific and author-benchmarked; consumer-GPU
|
||||
third-party benchmarks are still limited (seconds, not sub-second, though within the
|
||||
<16GB envelope).
|
||||
5. Time-sensitivity: Qwen-Image-Edit-2511 and Z-Image are late-2025/2026 releases; the
|
||||
diffusers pipelines cited are on the main/dev branch, so confirm released-version
|
||||
availability before pinning.
|
||||
6. Five claims were refuted (below), clustering on over-strong edit-model face-fidelity
|
||||
and one over-strong Z-Image cost claim.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Does a Qwen-Image **tile-ControlNet** (or equivalent high-resolution detail
|
||||
conditioning) exist under a permissive license?
|
||||
- What **non-regenerative detail-restoration** method recovers smoothed faces WITHOUT
|
||||
re-introducing SynthID? Note: residual transfer from the ORIGINAL risks copying back
|
||||
watermark-carrying high frequencies, so it must be verified against the SynthID oracle.
|
||||
- Does adding Qwen-Image-ControlNet (canny/depth) at the certified floors (OpenAI 0.10,
|
||||
Gemini 0.25) actually raise face Laplacian/LPIPS toward the SDXL+ControlNet numbers
|
||||
(0.62 / 0.09) WITHOUT re-introducing SynthID, or does the structure constraint
|
||||
preserve the watermark the way ControlNet can on photoreal content (the existing
|
||||
"SynthID CAN survive controlnet at low strength" caveat)?
|
||||
- Head-to-head: does Z-Image-Turbo at its scrub floor match Qwen's text advantage
|
||||
(CJK+Cyrillic CER) while not worsening faces, and what are Z-Image's own SynthID
|
||||
scrub floors and seed-robustness (none exist yet)?
|
||||
|
||||
## Refuted claims (do NOT rely on these)
|
||||
|
||||
- [0-3] "Qwen-Image-Edit-2511 specifically targets/mitigates image drift, the same
|
||||
failure mode as face-detail loss in a low-strength scrub." (qwen.ai/blog, 2511)
|
||||
- [0-3] "Qwen-Image-Edit-2509 explicitly improves facial identity preservation and
|
||||
supports portrait styles and pose transformations." (HF Qwen-Image-Edit-2509)
|
||||
- [0-3] "Qwen-Image-Edit-2509 has native built-in ControlNet support (depth/edge/
|
||||
keypoint)." (HF Qwen-Image-Edit-2509)
|
||||
- [1-2] "flymy realism LoRA specifically targets facial and skin detail, the exact
|
||||
failure mode." (HF flymy-ai/qwen-image-realism-lora)
|
||||
- [0-3] "Z-Image-Turbo runs on consumer 16GB-VRAM hardware, far below the A100-80GB of
|
||||
Qwen-Image 20B, materially lowering per-image cost." (HF Tongyi-MAI/Z-Image-Turbo)
|
||||
|
||||
## Sources
|
||||
|
||||
1. https://qwen.ai/blog?id=qwen-image-edit-2511
|
||||
2. https://qwenlm.github.io/blog/qwen-image-edit/
|
||||
3. https://docs.comfy.org/tutorials/image/qwen/qwen-image-edit
|
||||
4. https://github.com/FurkanGozukara/Stable-Diffusion/wiki/Qwen-Image-Edit-2511-Free-and-Open-Source-Crushes-Qwen-Image-Edit-2509-and-Challenges-Nano-Banana-Pro
|
||||
5. https://myaiforce.com/qie-2511/
|
||||
6. https://huggingface.co/Qwen/Qwen-Image-Edit-2509
|
||||
7. https://huggingface.co/InstantX/Qwen-Image-ControlNet-Union
|
||||
8. https://huggingface.co/InstantX/Qwen-Image-ControlNet-Inpainting
|
||||
9. https://huggingface.co/DiffSynth-Studio/Qwen-Image-EliGen-V2
|
||||
10. https://github.com/modelscope/DiffSynth-Studio/blob/main/docs/en/Model_Details/Qwen-Image.md
|
||||
11. https://blog.comfy.org/p/day-1-support-of-qwen-image-instantx
|
||||
12. https://learn.thinkdiffusion.com/how-to-use-qwen-image-with-instantx-union-controlnet-in-comfyui-guide-workflow/
|
||||
13. https://huggingface.co/flymy-ai/qwen-image-realism-lora
|
||||
14. https://huggingface.co/lightx2v/Qwen-Image-Lightning/discussions/4
|
||||
15. https://huggingface.co/docs/diffusers/main/en/api/pipelines/qwenimage
|
||||
16. https://www.diyphotography.net/skin-retouching-technique-frequency-separation/
|
||||
17. https://link.springer.com/content/pdf/10.1007/978-3-642-15549-9_1.pdf
|
||||
18. https://github.com/ShieldMnt/invisible-watermark/wiki/Frequency-Methods
|
||||
19. https://huggingface.co/Tongyi-MAI/Z-Image-Turbo
|
||||
20. https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/z_image/pipeline_z_image_img2img.py
|
||||
21. https://arxiv.org/pdf/2511.22699
|
||||
22. https://github.com/ModelTC/LightX2V-Qwen-Image-Lightning
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
**Conclusion (historical): pure reverse-alpha distilled from content images does NOT work, and the blocker is the WRONG kind of data, not too little of it.**
|
||||
|
||||
The earlier framing ("need ~5-8 PRISTINE same-resolution originals") is obsolete -- `data/spaces/originals/` holds plenty. Curate them with `DoubaoEngine.detect` + an NCC filter against a clean glyph template, keeping only marks at offset ≈ (0,0): that yields e.g. **15 pixel-aligned 2048² marks** (sub-pixel drift, not the ±50 px the old lossy/mixed-res scrapes had), plus 1086x1448 / 1792x2400 clusters. With those, LaMa-clean `O` + weighted-LS (and per-pixel I-on-O regression) for `α` (+ logo colour) was tried end-to-end and **still leaves a persistent ghost outline.**
|
||||
The earlier framing ("need ~5-8 PRISTINE same-resolution originals") is obsolete -- a local corpus of pristine originals holds plenty. Curate them with `DoubaoEngine.detect` + an NCC filter against a clean glyph template, keeping only marks at offset ≈ (0,0): that yields e.g. **15 pixel-aligned 2048² marks** (sub-pixel drift, not the ±50 px the old lossy/mixed-res scrapes had), plus 1086x1448 / 1792x2400 clusters. With those, LaMa-clean `O` + weighted-LS (and per-pixel I-on-O regression) for `α` (+ logo colour) was tried end-to-end and **still leaves a persistent ghost outline.**
|
||||
|
||||
Diagnosed why, empirically (cached stacks, `/tmp/doubao_distill`): (1) the mark is a clean white overlay with **no dark halo** -- over glyph pixels ~54% are brighter than the clean bg, only ~4% darker -- so the white-logo model `I=(1-α)O+α·255` is correct; (2) but content backgrounds are almost never dark *under* the mark (median darkest available bg over glyph pixels = **58/255**; only ~13% of mark pixels are ever observed on a bg < 40), so on bright backgrounds the equation is ill-conditioned and `α` is unidentifiable; (3) LaMa's `O` is a plausible **hallucination**, not the true pre-mark background, which compounds the error, and per-pixel regression on ~15 obs overfits into colour noise.
|
||||
|
||||
|
||||
@@ -181,6 +181,15 @@ pos original plus its minimum-clearing cleaned output (manifest `verified_via` =
|
||||
was oracle-verified but is not committed (third-party content stays out of the
|
||||
public corpus).
|
||||
|
||||
**Oracle validation order: start with OpenAI.** When validating removal across
|
||||
vendors, run the OpenAI arm first. `openai.com/verify` is more accessible than the
|
||||
Gemini app -- fewer per-check restrictions, so it gives the fastest signal and is
|
||||
the strongest candidate for automation (Playwright / Chrome MCP driving
|
||||
`openai.com/verify`); the Gemini "Verify with SynthID" flow is more manual. This is
|
||||
an ordering/throughput choice, not a substitution: each oracle only reads its own
|
||||
vendor's SynthID (`openai.com/verify` is OpenAI-scoped), so Google content still
|
||||
needs the Gemini app.
|
||||
|
||||
| Vendor | Images | Resolution(s) | Pipeline | Removed at |
|
||||
|--------|--------|---------------|----------|------------|
|
||||
| OpenAI (gpt-image) | n=4 (3 archived + 1 external-only) | 1024x1536 .. 1600x1600 | native | **0.05** |
|
||||
|
||||
@@ -5,8 +5,12 @@
|
||||
> no content was changed or summarized.
|
||||
|
||||
Who embeds what, and whether it is locally detectable (so we know which gaps are fillable). See `identify.py` for what we read.
|
||||
- **Locally detectable (open decoder, no key/API):** Stable Diffusion / SDXL / FLUX via `imwatermark` DWT-DCT (now covered by `invisible_watermark.py`). FLUX uses the same library (`black-forest-labs/flux2` `src/flux2/watermark.py`, 48-bit `0b001010101111111010000111100111001111010100101110`); SDXL is the diffusers `WATERMARK_MESSAGE` (`0b101100111110110010010000011110111011000110011110`). Caveat: fragile to re-encoding.
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), and **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`). Still unsampled: Canva (its downloads are re-encoded design *exports* that strip C2PA, so a Canva "positive" is inconclusive — skipped), Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
- **Locally detectable (open decoder, no key/API):** Stable Diffusion / SDXL / FLUX via `imwatermark` DWT-DCT (now covered by `invisible_watermark.py`). FLUX uses the same library (`black-forest-labs/flux2` `src/flux2/watermark.py`, 48-bit `0b001010101111111010000111100111001111010100101110`); SDXL is the diffusers `WATERMARK_MESSAGE` (`0b101100111110110010010000011110111011000110011110`). **Caveat: the `imwatermark` dwtDct decode is carrier-fragile on a broad class of real images, NOT just re-encode-fragile, and it is a POSITIVE-ONLY signal.** A clean encode->decode round-trip (no re-encode at all) recovers 48/48 bits on some carriers (random noise, chatgpt-1.png 48/48, firefly-1.png 45/48) but FAILS on many others — verified 2026-06-19 that a *known-embedded* watermark only round-trips 28-39/48 (below the safe `_MATCH_48` = 44 gate, random baseline ~24) on the FLUX fox sample (28), doubao-1.png (39), a 1024² minimalist-flat FLUX image (28), AND a **clean synthetic bright-flat fill with NO watermark at all (28)**. The failure does NOT track texture (firefly lapvar ~11 passes; the flat FLUX lapvar ~56 fails); it correlates with a degenerate decode where the raw bits read **all-ones (48/48 ones)** — which a clean synthetic image reproduces, so **all-ones is a CARRIER ARTIFACT, NOT a watermark signal** (a double-embed test also showed a pre-existing embed does not corrupt a second embed — no interference). Net: trust a `detect_invisible_watermark` hit, but treat a `None`/no-match as **inconclusive** whenever a positive-control embed on the same carrier does not first recover >=44/48. The 44 gate is a deliberate precision choice (lowering it would admit false positives).
|
||||
|
||||
**Root cause and external confirmation (deep-research 2026-06-19, adversarially verified).** This is the SCHEME's ceiling, not our usage — there is no better decoder to adopt. The imwatermark maintainers state verbatim (both the ShieldMnt and Stability-AI READMEs) that the algorithm "cannot guarantee to decode the original watermarks 100% accurately even though we don't apply any attack." Independent measurement (WMAdapter, arXiv:2406.08337 Table 2) puts dwtDct at only **~0.79 bit accuracy on CLEAN images (~38/48 bits — already below our 44 gate)**, collapsing to ~0.50 (chance) under crop/JPEG. Two code-verified + locally-reproduced mechanisms drive the content-dependent failures: (1) the decoder reads each bit as the **highest-magnitude DCT coefficient per block**, so any content coefficient exceeding the encoded target flips the bit; (2) the default embed is in the **YUV chroma channel, which 8-bit-clamps on white/bright pixels** (a +36 chroma delta survives a white-fill round-trip as only +4, ~89% loss) — this is the mechanism behind the bright-flat / minimalist failures and the all-ones degenerate decode. No maintained fork or detector decodes this scheme reliably: the WAVES benchmark (arXiv:2401.08573) relegates DWT-DCT to supplementary appendix G.5 and targets Stable Signature / Tree-Ring / StegaStamp instead; learned encoder/decoder schemes reach ~0.98-0.99 clean but are a DIFFERENT watermark class (not what SDXL/FLUX stamp). `dwtDctSvd` does not help (SDXL embeds `dwtDct`; dwtDctSvd cannot decode it, and its clean accuracy ~0.72 is lower). **Authoritative conclusion: the open DWT-DCT mark cannot be turned from positive-only into a reliable real-world detector; keep it positive-only and rely on C2PA.** (Refuted along the way: that the library is unmaintained, and that it is robust to JPEG but only fails on geometric attacks — both did not survive verification.)
|
||||
|
||||
Consequence for the FLUX hosted-output question (BFL Playground, FLUX.2 [pro] + FLUX.1 [dev], 2026-06-19): all samples carry the signed C2PA manifest (issuer "Black Forest Labs"); the open DWT-DCT decode returned `None`, but every available FLUX carrier (textured fox AND a minimalist-flat generation) failed the positive control (28/48), so the detector is blind on them and **whether BFL hosted output embeds the open pixel watermark is UNRESOLVED** (an earlier note here wrongly asserted it absent — overstated; a later note blamed "high texture" — also wrong, flat carriers fail too). What IS established: C2PA is the reliable FLUX identifier; the `_BITS_48` pattern is correct (round-trips on chatgpt/firefly/random). Resolving the hosted question needs a hosted FLUX carrier that first passes a >=44/48 positive control, which neither a textured nor a flat prompt produced — low priority (the open mark is only a stripped-metadata fallback).
|
||||
- **C2PA / IPTC (covered by the issuer/marker scan):** OpenAI, Google, Adobe Firefly, Microsoft (Designer + **Bing Image Creator** — collected 2026-05-24; Bing now runs Microsoft's own **MAI-Image** model, signs C2PA as "Microsoft", NOT OpenAI/DALL-E), **Stability AI** (collected from Brand Studio / DreamStudio successor; signs C2PA as "Stability AI Ltd", no SynthID, no imwatermark on its current Stable Image model — issuer added to `C2PA_ISSUERS`), and **Canva** (Magic Media signs C2PA as "Canva" + `trainedAlgorithmicMedia` with a generic `c2pa-rs` claim generator, no SynthID — issuer `b"Canva"` → "Canva (Magic Media)"; found on real production traffic 2026-06-19, which **disproved the earlier assumption** that Canva downloads are re-encoded exports that always strip C2PA). Still unsampled: Getty, Shutterstock. Midjourney embeds NO C2PA and no invisible watermark (our `mj-*` sample carried only the IPTC tag).
|
||||
|
||||
**Samsung Galaxy AI** (Generative Edit / Sketch to Image / Portrait Studio on Galaxy S23 FE / S24 / S25, One UI 7+) signs C2PA as "Samsung Galaxy" with the standard `trainedAlgorithmicMedia` source type AND a proprietary `genAIType` marker; verified on real signed files 2026-05-29 (the standard scan catches the source type; `genAIType` additionally catches a Galaxy S24 file that omits it). It ALSO burns a **visible** localized wordmark into the pixels — a sparkle + "generated with AI" string in the bottom-LEFT corner (issue #37; the Italian "✦ Contenuti generati dall'AI" variant is calibrated) — removed by `samsung_engine.py` / `visible --mark samsung` (reverse-alpha, see the engine bullet); detection feeds `identify` as the medium `visible_samsung` signal. The string is locale-specific, so each locale needs its own captured alpha template.
|
||||
|
||||
@@ -14,7 +18,7 @@ Who embeds what, and whether it is locally detectable (so we know which gaps are
|
||||
|
||||
**Black Forest Labs (FLUX)** API output signs C2PA: `claim_generator_info "Black Forest Labs API"` + a `c2pa.ai_generated_content` assertion + `trainedAlgorithmicMedia` (issuer `b"Black Forest Labs"` added to `C2PA_ISSUERS`, platform "Black Forest Labs (FLUX)").
|
||||
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. All three verified on real signed files 2026-05-29.
|
||||
**ByteDance Volcano Engine (Volcengine)** — the cloud behind Doubao / Jimeng — signs its AI image output with a cert from `certificate_center@volcengine.com` + `trainedAlgorithmicMedia` (issuer `b"volcengine"` → "ByteDance (Volcano Engine)", platform "ByteDance (Doubao / Jimeng / Volcano Engine)"); note this is the C2PA-signed surface, distinct from the XMP/PNG TC260 `AIGC` label Doubao also uses. All three verified on real signed files 2026-05-29. ByteDance's **international brand (BytePlus / Seedream / Seededit)** signs the SAME content as **"Byteplus Pte. Ltd."** — the bare `volcengine` needle missed it, so real BytePlus output was mis-attributed to "Adobe Firefly" (an incidental "Adobe XMP" toolkit string in the file's XMP, picked up by the fallback byte-scan once the clean manifest issuer matched nothing). Added issuer `b"Byteplus"` → org "BytePlus (ByteDance)" (platform resolves to the shared "ByteDance (Doubao / Jimeng / Volcano Engine)" label via the common `ByteDance` needle) so the clean manifest issuer attributes it directly; found on real production traffic 2026-06-19.
|
||||
- **EXIF/XMP generator tag (caught by `exif_generator`):** **Ideogram** writes EXIF `Make="Ideogram AI"` (collected 2026-05-24 — no C2PA, no SynthID, no imwatermark; the Make tag is the only signal).
|
||||
- **xAI / Grok — its own EXIF signature scheme, NOT C2PA (DETECTED by `metadata.xai_signature`, built 2026-05-26).**
|
||||
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "remove-ai-watermarks"
|
||||
version = "0.11.3"
|
||||
version = "0.12.0"
|
||||
description = "AI watermark remover: strip visible and invisible AI watermarks (Gemini / Nano Banana sparkle, SynthID) and provenance metadata (C2PA, EXIF) from images"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
|
||||
@@ -0,0 +1,418 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "numpy",
|
||||
# "opencv-python-headless",
|
||||
# "pillow",
|
||||
# "scikit-image",
|
||||
# "rapidfuzz",
|
||||
# "torch",
|
||||
# "lpips",
|
||||
# "paddleocr",
|
||||
# "paddlepaddle",
|
||||
# "insightface",
|
||||
# "onnxruntime",
|
||||
# ]
|
||||
# ///
|
||||
"""Objective fidelity metrics for comparing watermark-removal outputs.
|
||||
|
||||
Given an ORIGINAL (the reference) and one or more cleaned VARIANTS that have all
|
||||
ALREADY passed the scrub oracle, this scores how much real detail each variant
|
||||
preserved -- so "closer to the original" is the right axis here (between two
|
||||
equally-scrubbed outputs, the one that deviates less from the original wins).
|
||||
|
||||
It is a standalone eval tool, NOT part of the package: PEP 723 inline deps let
|
||||
``uv run`` build a throwaway env so the heavy models (PaddleOCR, insightface,
|
||||
LPIPS) never touch uv.lock or the shipped library. Metrics self-gate: face
|
||||
metrics run only where faces are detected, text metrics only where text is.
|
||||
|
||||
Two subcommands:
|
||||
|
||||
ocr -- OCR images (PaddleOCR PP-OCRv6) into a JSON {basename: text} file.
|
||||
Run this on the ORIGINALS, hand-verify/correct the file, and it
|
||||
becomes the ground truth for ``compare --ground-truth`` -- the clean
|
||||
way to score text, since OCR-vs-OCR is doubly noisy (errors on both
|
||||
images + reading-order differences inflate CER even on identical text).
|
||||
|
||||
compare -- Score each VARIANT against the ORIGINAL across four groups:
|
||||
1. Text -- character error rate (CER) of the variant's OCR vs the
|
||||
verified ground truth (or the original's OCR if no --ground-truth).
|
||||
2. Face identity -- insightface (buffalo_l) ArcFace cosine similarity.
|
||||
3. Face texture -- LPIPS + Laplacian-variance ratio on face crops
|
||||
(catches "plastication": ratio < 1 = smoother than the original).
|
||||
4. Whole image -- LPIPS / SSIM / PSNR vs the original.
|
||||
|
||||
Usage:
|
||||
uv run scripts/fidelity_metrics.py ocr O1.png O2.png --langs en,ru,ch --out gt.json
|
||||
# (edit gt.json by hand to fix any OCR slips, then:)
|
||||
uv run scripts/fidelity_metrics.py compare --original O1.png \
|
||||
--variant controlnet=C.png --variant qwen=Q.png \
|
||||
--ocr-langs en,ru,ch --ground-truth gt.json
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import unicodedata
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import click
|
||||
import cv2
|
||||
import numpy as np
|
||||
from _plain_console import Console, Table
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
# ── helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _load_bgr(path: str) -> np.ndarray:
|
||||
img = cv2.imread(path, cv2.IMREAD_COLOR)
|
||||
if img is None:
|
||||
raise click.ClickException(f"cannot read image: {path}")
|
||||
return img
|
||||
|
||||
|
||||
def _match_size(variant: np.ndarray, ref: np.ndarray) -> np.ndarray:
|
||||
"""Resize a variant to the reference size (outputs differ by a grid-round)."""
|
||||
if variant.shape[:2] != ref.shape[:2]:
|
||||
variant = cv2.resize(variant, (ref.shape[1], ref.shape[0]), interpolation=cv2.INTER_LANCZOS4)
|
||||
return variant
|
||||
|
||||
|
||||
def _norm(text: str) -> str:
|
||||
"""Normalize for CER: NFC + drop ALL whitespace (segmentation-order agnostic)."""
|
||||
return "".join(unicodedata.normalize("NFC", text).split())
|
||||
|
||||
|
||||
# ── text: PaddleOCR (PP-OCRv6) ───────────────────────────────────────
|
||||
|
||||
# Our lang codes -> PaddleOCR lang. The 'ch' model also reads Latin; 'ru' reads
|
||||
# Cyrillic + Latin. Multiple langs in one image -> run each model, union detections.
|
||||
_PADDLE_LANG = {"en": "en", "ru": "ru", "ch": "ch", "ch_sim": "ch", "latin": "latin"}
|
||||
_paddle_cache: dict[str, Any] = {}
|
||||
|
||||
|
||||
def _paddle(lang: str) -> Any:
|
||||
if lang not in _paddle_cache:
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
_paddle_cache[lang] = PaddleOCR(
|
||||
lang=lang,
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
)
|
||||
return _paddle_cache[lang]
|
||||
|
||||
|
||||
def _box_xyxy(box: Any) -> tuple[float, float, float, float]:
|
||||
"""Axis-aligned (x1, y1, x2, y2) of a PaddleOCR rec box ([x1,y1,x2,y2]) or poly (4x2)."""
|
||||
arr = np.asarray(box, dtype=np.float32).reshape(-1)
|
||||
if arr.size == 4:
|
||||
return float(arr[0]), float(arr[1]), float(arr[2]), float(arr[3])
|
||||
pts = arr.reshape(-1, 2)
|
||||
return float(pts[:, 0].min()), float(pts[:, 1].min()), float(pts[:, 0].max()), float(pts[:, 1].max())
|
||||
|
||||
|
||||
def _iou(a: tuple[float, float, float, float], b: tuple[float, float, float, float]) -> float:
|
||||
ix1, iy1 = max(a[0], b[0]), max(a[1], b[1])
|
||||
ix2, iy2 = min(a[2], b[2]), min(a[3], b[3])
|
||||
iw, ih = max(0.0, ix2 - ix1), max(0.0, iy2 - iy1)
|
||||
inter = iw * ih
|
||||
if inter <= 0:
|
||||
return 0.0
|
||||
area_a = (a[2] - a[0]) * (a[3] - a[1])
|
||||
area_b = (b[2] - b[0]) * (b[3] - b[1])
|
||||
return inter / (area_a + area_b - inter + 1e-9)
|
||||
|
||||
|
||||
def _ocr_lines(bgr: np.ndarray, langs: list[str], min_score: float = 0.5) -> list[str]:
|
||||
"""Detected text lines in reading order, unioned across lang models with spatial NMS.
|
||||
|
||||
Several language models over one image re-detect the same lines -- and crucially the
|
||||
WRONG-script models read e.g. Cyrillic as confident Latin gibberish. So instead of a
|
||||
naive union, keep the HIGHEST-score detection per physical location (greedy IoU NMS):
|
||||
the model that actually fits a line wins it (the 'ru' model takes the Cyrillic, 'ch'
|
||||
the CJK, 'en' the Latin), and the cross-script garbage is dropped.
|
||||
"""
|
||||
raw: list[tuple[float, tuple[float, float, float, float], str]] = []
|
||||
for lang in langs:
|
||||
plang = _PADDLE_LANG.get(lang, lang)
|
||||
for page in _paddle(plang).predict(bgr):
|
||||
texts = page.get("rec_texts", [])
|
||||
scores = page.get("rec_scores", [])
|
||||
boxes = page.get("rec_boxes", None)
|
||||
if boxes is None or len(boxes) == 0:
|
||||
boxes = page.get("rec_polys", [])
|
||||
for text, score, box in zip(texts, scores, boxes, strict=False):
|
||||
if score < min_score or not text.strip():
|
||||
continue
|
||||
raw.append((float(score), _box_xyxy(box), text.strip()))
|
||||
|
||||
raw.sort(key=lambda d: d[0], reverse=True)
|
||||
kept: list[tuple[tuple[float, float, float, float], str]] = []
|
||||
for _score, box, text in raw:
|
||||
if any(_iou(box, kbox) > 0.3 for kbox, _ in kept):
|
||||
continue
|
||||
kept.append((box, text))
|
||||
kept.sort(key=lambda d: (round(d[0][1] / 20.0), d[0][0])) # reading order: y then x
|
||||
return [t for _, t in kept]
|
||||
|
||||
|
||||
def _cer(ref: str, hyp: str) -> float:
|
||||
from rapidfuzz.distance import Levenshtein
|
||||
|
||||
return Levenshtein.normalized_distance(_norm(ref), _norm(hyp))
|
||||
|
||||
|
||||
# ── face: detection + ArcFace + texture ──────────────────────────────
|
||||
|
||||
|
||||
@dataclass
|
||||
class FaceStats:
|
||||
n_faces: int = 0
|
||||
identity: list[float] = field(default_factory=list)
|
||||
lpips: list[float] = field(default_factory=list)
|
||||
lapvar_ratio: list[float] = field(default_factory=list)
|
||||
|
||||
|
||||
def _lap_var(bgr: np.ndarray) -> float:
|
||||
gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY)
|
||||
return float(cv2.Laplacian(gray, cv2.CV_64F).var())
|
||||
|
||||
|
||||
def _bbox_center(bbox: Any) -> tuple[float, float]:
|
||||
return (bbox[0] + bbox[2]) / 2, (bbox[1] + bbox[3]) / 2
|
||||
|
||||
|
||||
def _bbox_diag(bbox: Any) -> float:
|
||||
return float(((bbox[2] - bbox[0]) ** 2 + (bbox[3] - bbox[1]) ** 2) ** 0.5)
|
||||
|
||||
|
||||
def assign_faces_one_to_one(
|
||||
ref_centers: list[tuple[float, float]],
|
||||
var_centers: list[tuple[float, float]],
|
||||
ref_diags: list[float],
|
||||
max_frac: float = 0.6,
|
||||
) -> dict[int, int]:
|
||||
"""One-to-one nearest-center face assignment (pure; unit-tested without insightface).
|
||||
|
||||
Per-face nearest matching collides on multi-face images -- two original faces can both
|
||||
pick the SAME variant face (e.g. when regeneration drops a face, so the variant has fewer
|
||||
detections), corrupting the identity metric (the lapvar/LPIPS metrics are immune: they are
|
||||
anchored to the ORIGINAL bbox on both images). This greedy-by-distance assignment is
|
||||
collision-free: it walks candidate pairs nearest-first and never reuses a ref or a variant
|
||||
face. Faces are spatially well-separated, so greedy equals the optimal (Hungarian) result
|
||||
here without the scipy dependency. A pair is dropped when the center distance exceeds
|
||||
``max_frac`` of the original face diagonal (no plausible match -- the face was lost).
|
||||
|
||||
Returns a dict mapping ref-face index -> variant-face index for matched faces only.
|
||||
"""
|
||||
pairs: list[tuple[float, int, int]] = []
|
||||
for i, (rx, ry) in enumerate(ref_centers):
|
||||
for j, (vx, vy) in enumerate(var_centers):
|
||||
pairs.append((((rx - vx) ** 2 + (ry - vy) ** 2) ** 0.5, i, j))
|
||||
pairs.sort()
|
||||
used_ref: set[int] = set()
|
||||
used_var: set[int] = set()
|
||||
matched: dict[int, int] = {}
|
||||
for dist, i, j in pairs:
|
||||
if i in used_ref or j in used_var:
|
||||
continue
|
||||
if dist > max_frac * ref_diags[i]:
|
||||
continue
|
||||
matched[i] = j
|
||||
used_ref.add(i)
|
||||
used_var.add(j)
|
||||
return matched
|
||||
|
||||
|
||||
def _cosine(a: np.ndarray, b: np.ndarray) -> float:
|
||||
return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-9))
|
||||
|
||||
|
||||
def _crop(bgr: np.ndarray, bbox: Any) -> np.ndarray:
|
||||
h, w = bgr.shape[:2]
|
||||
x1, y1, x2, y2 = (int(max(0, bbox[0])), int(max(0, bbox[1])), int(min(w, bbox[2])), int(min(h, bbox[3])))
|
||||
return bgr[y1:y2, x1:x2]
|
||||
|
||||
|
||||
# ── whole image: LPIPS / SSIM / PSNR ─────────────────────────────────
|
||||
|
||||
|
||||
def _lpips_model() -> tuple[Any, Any]:
|
||||
import lpips
|
||||
import torch
|
||||
|
||||
model = lpips.LPIPS(net="alex", verbose=False)
|
||||
model.eval()
|
||||
return model, torch
|
||||
|
||||
|
||||
def _lpips_distance(model_torch: tuple[Any, Any], a_bgr: np.ndarray, b_bgr: np.ndarray) -> float:
|
||||
model, torch = model_torch
|
||||
|
||||
def _t(img: np.ndarray) -> Any:
|
||||
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32) / 127.5 - 1.0
|
||||
return torch.from_numpy(rgb).permute(2, 0, 1).unsqueeze(0)
|
||||
|
||||
with torch.no_grad():
|
||||
return float(model(_t(a_bgr), _t(b_bgr)).item())
|
||||
|
||||
|
||||
def _ssim_psnr(a_bgr: np.ndarray, b_bgr: np.ndarray) -> tuple[float, float]:
|
||||
from skimage.metrics import peak_signal_noise_ratio, structural_similarity
|
||||
|
||||
a = cv2.cvtColor(a_bgr, cv2.COLOR_BGR2GRAY)
|
||||
b = cv2.cvtColor(b_bgr, cv2.COLOR_BGR2GRAY)
|
||||
return float(structural_similarity(a, b)), float(peak_signal_noise_ratio(a, b))
|
||||
|
||||
|
||||
# ── reporting ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _mean(xs: list[float]) -> float | None:
|
||||
return sum(xs) / len(xs) if xs else None
|
||||
|
||||
|
||||
def _fmt(v: float | None, nd: int = 3) -> str:
|
||||
return "-" if v is None else f"{v:.{nd}f}"
|
||||
|
||||
|
||||
# ── CLI ──────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@click.group()
|
||||
def cli() -> None:
|
||||
"""Objective fidelity metrics for watermark-removal outputs."""
|
||||
|
||||
|
||||
@cli.command("ocr")
|
||||
@click.argument("images", nargs=-1, required=True, type=click.Path(exists=True))
|
||||
@click.option("--langs", default="en", help="Comma list of OCR langs (en,ru,ch).")
|
||||
@click.option("--out", type=click.Path(), default=None, help="Write {basename: text} JSON here (for ground truth).")
|
||||
def ocr_cmd(images: tuple[str, ...], langs: str, out: str | None) -> None:
|
||||
"""OCR images into a ground-truth seed -- hand-verify the result before using it."""
|
||||
lang_list = [x.strip() for x in langs.split(",") if x.strip()]
|
||||
result: dict[str, str] = {}
|
||||
for path in images:
|
||||
lines = _ocr_lines(_load_bgr(path), lang_list)
|
||||
text = "\n".join(lines)
|
||||
result[Path(path).name] = text
|
||||
console.print(f"\n=== {Path(path).name} ===")
|
||||
console.print(text or "(no text detected)")
|
||||
if out:
|
||||
Path(out).write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
console.print(f"\n Wrote {out} -- verify/correct it by hand, then pass it to `compare --ground-truth`.")
|
||||
|
||||
|
||||
@cli.command("compare")
|
||||
@click.option("--original", required=True, type=click.Path(exists=True), help="Reference (unprocessed) image.")
|
||||
@click.option(
|
||||
"--variant", "variants", multiple=True, required=True, help="LABEL=PATH of a cleaned output (repeatable)."
|
||||
)
|
||||
@click.option("--ocr-langs", default="en", help="Comma list of OCR langs (en,ru,ch). Empty = skip text.")
|
||||
@click.option("--ground-truth", type=click.Path(exists=True), default=None, help="Verified {basename: text} JSON.")
|
||||
@click.option("--no-faces", is_flag=True, help="Skip face metrics.")
|
||||
def compare(original: str, variants: tuple[str, ...], ocr_langs: str, ground_truth: str | None, no_faces: bool) -> None:
|
||||
"""Score each VARIANT against ORIGINAL across the four fidelity groups."""
|
||||
ref = _load_bgr(original)
|
||||
parsed: list[tuple[str, np.ndarray]] = []
|
||||
for spec in variants:
|
||||
if "=" not in spec:
|
||||
raise click.ClickException(f"--variant must be LABEL=PATH, got {spec!r}")
|
||||
label, path = spec.split("=", 1)
|
||||
parsed.append((label, _match_size(_load_bgr(path), ref)))
|
||||
|
||||
langs = [x.strip() for x in ocr_langs.split(",") if x.strip()]
|
||||
lp = _lpips_model() # AlexNet LPIPS, loaded once and reused for face crops + whole image
|
||||
|
||||
# ── text ──
|
||||
ocr_cer: dict[str, float | None] = {label: None for label, _ in parsed}
|
||||
if langs:
|
||||
ref_text: str | None = None
|
||||
if ground_truth:
|
||||
gt = json.loads(Path(ground_truth).read_text(encoding="utf-8"))
|
||||
ref_text = gt.get(Path(original).name)
|
||||
if ref_text is None:
|
||||
console.print(f" (no ground-truth entry for {Path(original).name}; skipping text)")
|
||||
else:
|
||||
console.print(f" OCR original ({','.join(langs)})...")
|
||||
ref_text = "\n".join(_ocr_lines(ref, langs))
|
||||
if ref_text:
|
||||
console.print(f" OCR variants ({','.join(langs)})...")
|
||||
for label, img in parsed:
|
||||
ocr_cer[label] = _cer(ref_text, "\n".join(_ocr_lines(img, langs)))
|
||||
|
||||
# ── faces ──
|
||||
face_stats: dict[str, FaceStats] = {label: FaceStats() for label, _ in parsed}
|
||||
if not no_faces:
|
||||
console.print(" Faces (insightface buffalo_l)...")
|
||||
from insightface.app import FaceAnalysis
|
||||
|
||||
app = FaceAnalysis(name="buffalo_l", providers=["CPUExecutionProvider"])
|
||||
app.prepare(ctx_id=-1, det_size=(640, 640))
|
||||
ref_faces = app.get(ref)
|
||||
if ref_faces:
|
||||
ref_centers = [_bbox_center(of.bbox) for of in ref_faces]
|
||||
ref_diags = [_bbox_diag(of.bbox) for of in ref_faces]
|
||||
for label, img in parsed:
|
||||
vfaces = app.get(img)
|
||||
st = face_stats[label]
|
||||
# One-to-one assignment for identity (collision-free); lapvar/LPIPS stay
|
||||
# anchored to the original bbox below, so they need no match.
|
||||
matched = assign_faces_one_to_one(ref_centers, [_bbox_center(vf.bbox) for vf in vfaces], ref_diags)
|
||||
for oi, of in enumerate(ref_faces):
|
||||
st.n_faces += 1
|
||||
vf = vfaces[matched[oi]] if oi in matched else None
|
||||
if vf is not None:
|
||||
st.identity.append(_cosine(of.normed_embedding, vf.normed_embedding))
|
||||
oc, vc = _crop(ref, of.bbox), _crop(img, of.bbox)
|
||||
if oc.size == 0 or vc.size == 0:
|
||||
continue
|
||||
vc_r = cv2.resize(vc, (oc.shape[1], oc.shape[0]), interpolation=cv2.INTER_LANCZOS4)
|
||||
st.lpips.append(_lpips_distance(lp, oc, vc_r))
|
||||
ov = _lap_var(oc)
|
||||
st.lapvar_ratio.append(_lap_var(vc_r) / ov if ov > 1e-6 else 0.0)
|
||||
else:
|
||||
console.print(" (no faces detected in the original; skipping face metrics)")
|
||||
|
||||
# ── whole image ──
|
||||
console.print(" Whole-image LPIPS/SSIM/PSNR...")
|
||||
whole: dict[str, tuple[float, float, float]] = {}
|
||||
for label, img in parsed:
|
||||
ssim, psnr = _ssim_psnr(ref, img)
|
||||
whole[label] = (_lpips_distance(lp, ref, img), ssim, psnr)
|
||||
|
||||
# ── report ──
|
||||
table = Table(title=f"Fidelity vs {Path(original).name} (reference)")
|
||||
for col in ("variant", "text CER↓", "faces", "ID cos↑", "face LPIPS↓", "lapvar↑", "img LPIPS↓", "SSIM↑", "PSNR↑"):
|
||||
table.add_column(col)
|
||||
for label, _ in parsed:
|
||||
st = face_stats[label]
|
||||
wl, ws, wp = whole[label]
|
||||
table.add_row(
|
||||
label,
|
||||
_fmt(ocr_cer[label]),
|
||||
str(st.n_faces),
|
||||
_fmt(_mean(st.identity)),
|
||||
_fmt(_mean(st.lpips)),
|
||||
_fmt(_mean(st.lapvar_ratio)),
|
||||
_fmt(wl),
|
||||
_fmt(ws),
|
||||
_fmt(wp, 1),
|
||||
)
|
||||
console.print(table)
|
||||
console.print(
|
||||
" Legend: CER lower=better; ID cos higher=better; face LPIPS lower=better; "
|
||||
"lapvar ratio ~1=detail kept, <1=smoothed/plastic; img LPIPS lower=better; SSIM/PSNR higher=closer."
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
@@ -0,0 +1,128 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# dependencies = [
|
||||
# "diffusers>=0.35.0",
|
||||
# "transformers>=4.51.0",
|
||||
# "torch",
|
||||
# "accelerate",
|
||||
# "pillow",
|
||||
# "click",
|
||||
# ]
|
||||
# ///
|
||||
"""Isolated GPU prototype: does a low-strength Qwen-Image img2img pass scrub the
|
||||
invisible watermark while keeping text/structure legible?
|
||||
|
||||
This is the oracle-gated experiment behind Library roadmap P1#5 (migrate the
|
||||
invisible pipeline onto Qwen-Image-Edit). It is DELIBERATELY standalone:
|
||||
|
||||
* It is NOT imported by the package and NOT in ``uv.lock``. Qwen-Image needs a
|
||||
newer ``diffusers``/``transformers`` (Qwen2.5-VL text encoder) than the SDXL
|
||||
pipeline is pinned to, so wiring it into the locked env would risk the
|
||||
certified SDXL/ControlNet pipeline (the ``cannot import Qwen3VL...`` trap).
|
||||
PEP 723 inline metadata lets ``uv run`` build a throwaway env for it instead.
|
||||
* Qwen-Image is ~20B, so it needs a real GPU (CUDA) -- it will not fit on MPS.
|
||||
|
||||
Run (on a GPU box / Modal), then eyeball the outputs AND submit them to the
|
||||
matching oracle (openai.com/verify for OpenAI, the Gemini app for Google):
|
||||
|
||||
uv run scripts/qwen_scrub_prototype.py INPUT.png -o out/ --strengths 0.1,0.2,0.3,0.4
|
||||
|
||||
What to look for:
|
||||
* SCRUB: the oracle no longer reports the watermark at some strength.
|
||||
* FIDELITY: text stays legible and faces/structure stay faithful at that same
|
||||
strength -- the whole point of trying Qwen over SDXL (which garbles text).
|
||||
The smallest strength that clears the oracle while keeping fidelity is the result
|
||||
to compare against the SDXL/ControlNet floors (OpenAI 0.10 / Google 0.15).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
|
||||
log = logging.getLogger("qwen_proto")
|
||||
|
||||
# A neutral, faithful-regeneration prompt (we want to scrub, not restyle); mirrors
|
||||
# the intent of the SDXL controlnet prompt. Qwen renders text natively, so a light
|
||||
# pass should keep captions legible where SDXL would garble them.
|
||||
_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
|
||||
|
||||
def _pick_device(requested: str) -> tuple[str, object]:
|
||||
import torch
|
||||
|
||||
if requested != "auto":
|
||||
device = requested
|
||||
elif torch.cuda.is_available():
|
||||
device = "cuda"
|
||||
elif getattr(torch.backends, "mps", None) is not None and torch.backends.mps.is_available():
|
||||
device = "mps"
|
||||
else:
|
||||
device = "cpu"
|
||||
# bf16 on CUDA (Qwen's reference dtype); fp32 elsewhere for numerical safety.
|
||||
dtype = torch.bfloat16 if device == "cuda" else torch.float32
|
||||
return device, dtype
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("source", type=click.Path(exists=True, path_type=Path))
|
||||
@click.option("-o", "--output-dir", type=click.Path(path_type=Path), default=Path("qwen_out"))
|
||||
@click.option("--strengths", default="0.1,0.2,0.3,0.4", help="Comma-separated img2img strengths to sweep.")
|
||||
@click.option("--steps", type=int, default=40, help="Inference steps.")
|
||||
@click.option("--cfg", type=float, default=4.0, help="true_cfg_scale (Qwen's CFG; reference default 4.0).")
|
||||
@click.option("--model", default="Qwen/Qwen-Image", help="HF model id (Qwen-Image img2img base).")
|
||||
@click.option("--device", default="auto", type=click.Choice(["auto", "cuda", "mps", "cpu"]))
|
||||
@click.option("--seed", type=int, default=0, help="Reproducible seed.")
|
||||
def main(
|
||||
source: Path,
|
||||
output_dir: Path,
|
||||
strengths: str,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
model: str,
|
||||
device: str,
|
||||
seed: int,
|
||||
) -> None:
|
||||
"""Sweep Qwen-Image img2img strength over SOURCE and save one output per strength."""
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s")
|
||||
import torch
|
||||
from diffusers import QwenImageImg2ImgPipeline
|
||||
from PIL import Image
|
||||
|
||||
dev, dtype = _pick_device(device)
|
||||
log.info("Loading %s on %s (%s)...", model, dev, dtype)
|
||||
pipe = QwenImageImg2ImgPipeline.from_pretrained(model, torch_dtype=dtype)
|
||||
pipe = pipe.to(dev)
|
||||
|
||||
init_image = Image.open(source).convert("RGB")
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
values = [float(s) for s in strengths.split(",") if s.strip()]
|
||||
|
||||
for strength in values:
|
||||
generator = torch.Generator(device="cpu").manual_seed(seed)
|
||||
log.info("Generating strength=%.2f ...", strength)
|
||||
result = pipe(
|
||||
prompt=_PROMPT,
|
||||
negative_prompt=_NEGATIVE,
|
||||
image=init_image,
|
||||
strength=strength,
|
||||
num_inference_steps=steps,
|
||||
true_cfg_scale=cfg,
|
||||
generator=generator,
|
||||
)
|
||||
out_path = output_dir / f"{source.stem}_qwen_s{strength:.2f}.png"
|
||||
result.images[0].save(out_path)
|
||||
log.info(" saved %s", out_path)
|
||||
|
||||
log.info(
|
||||
"\nDone. Eyeball text/face fidelity, then submit each output to the matching oracle "
|
||||
"(openai.com/verify / Gemini app). The smallest strength that clears the oracle while "
|
||||
"keeping fidelity is the number to compare against the SDXL floors (OpenAI 0.10 / Google 0.15)."
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -11,4 +11,4 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
|
||||
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
|
||||
|
||||
|
||||
__version__ = "0.11.3"
|
||||
__version__ = "0.12.0"
|
||||
@@ -37,6 +37,28 @@ if TYPE_CHECKING:
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Reverse-alpha over-subtraction guard (ported from gemini_engine, 2026-06-20).
|
||||
# The reverse-alpha blend ``(wm - a*logo)/(1-a)`` over-subtracts when the captured
|
||||
# alpha over-estimates THIS image's mark opacity: on a dark or mid-tone background
|
||||
# it drives the glyph footprint into a visibly DARKER-than-background ghost (a
|
||||
# "dark pit") instead of recovering the true pixels. The retained-corpus mining
|
||||
# (2026-06-20) showed the sparkle-only fix (commit 41f6797) left this unhandled
|
||||
# for the Doubao/Jimeng text marks. Mirror the sparkle gate: when the recovered
|
||||
# glyph body lands more than this many gray levels below the local background
|
||||
# ring, abandon the reverse-alpha output for the footprint and inpaint it from
|
||||
# the surroundings instead. Calibrated to the same 25-level margin the sparkle
|
||||
# gate uses -- clean text-mark removals recover within ~10 of the ring, the dark
|
||||
# pit lands tens of levels below.
|
||||
_OVERSUB_DARK_MARGIN = 25.0
|
||||
# Glyph-body / background-ring sampling for the guard. The ring is a pad around
|
||||
# the glyph box (excluding the box); the body is the bright-core glyph pixels.
|
||||
_OVERSUB_RING_PAD_FRAC = 0.6 # ring pad as a fraction of the glyph-box height
|
||||
_OVERSUB_BODY_ALPHA_FLOOR = 0.15 # alpha above which a block pixel counts as glyph body
|
||||
# Footprint inpaint when the guard trips: dilate the glyph mask wider than the
|
||||
# thin residual pass so the whole darkened ghost is reconstructed, not just its edge.
|
||||
_OVERSUB_INPAINT_DILATE = 9
|
||||
_OVERSUB_INPAINT_RADIUS = 4
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TextMarkConfig:
|
||||
@@ -184,20 +206,27 @@ class TextMarkEngine:
|
||||
# ── Mask ────────────────────────────────────────────────────────────
|
||||
|
||||
def extract_mask(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any]:
|
||||
"""Build a full-image uint8 mask (255 = watermark glyph) for the box.
|
||||
"""Build a box-sized uint8 mask (255 = watermark glyph) for ``loc``.
|
||||
|
||||
Returns just the glyph mask of the located box (shape ``(loc.h, loc.w)``),
|
||||
not a full-frame array: every caller immediately crops to ``loc.bbox``, so
|
||||
allocating a full ``(h, w)`` mask and embedding the box was O(image) work
|
||||
and memory for an O(box) result -- a wasted full-frame uint8 allocation on
|
||||
each detect (~12 MB on a 12 MP frame, recomputed per text-mark detector on
|
||||
the memory-tight identify path). The box mask is byte-identical to the old
|
||||
full-frame mask cropped to ``loc.bbox``.
|
||||
|
||||
Polarity-aware: the mark is a light, low-saturation gray rendered brighter
|
||||
than the local background (white top-hat), so a white-paper document is left
|
||||
untouched (nothing brighter than its surroundings is masked there).
|
||||
"""
|
||||
c = self.config
|
||||
h, w = image.shape[:2]
|
||||
x, y, bw, bh = loc.bbox
|
||||
# A degenerate ROI (a sliver from an extremely wide/short image) cannot hold
|
||||
# the mark and would feed cv2's GaussianBlur/morphology a ~1-px-tall array,
|
||||
# which can fault native code on some platforms. Skip the cv2 pipeline.
|
||||
if bh < 16 or bw < 16:
|
||||
return np.zeros((h, w), np.uint8)
|
||||
return np.zeros((bh, bw), np.uint8)
|
||||
# Normalize the ROI to 3-channel BGR (grayscale / BGRA would break axis=2).
|
||||
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
|
||||
|
||||
@@ -216,11 +245,7 @@ class TextMarkEngine:
|
||||
glyph = cand.astype(np.uint8) * 255
|
||||
glyph = cv2.morphologyEx(glyph, cv2.MORPH_CLOSE, np.ones((5, 5), np.uint8))
|
||||
k = c.morph_open_size
|
||||
glyph = cv2.morphologyEx(glyph, cv2.MORPH_OPEN, np.ones((k, k), np.uint8))
|
||||
|
||||
mask = np.zeros((h, w), np.uint8)
|
||||
mask[y : y + bh, x : x + bw] = glyph
|
||||
return mask
|
||||
return cv2.morphologyEx(glyph, cv2.MORPH_OPEN, np.ones((k, k), np.uint8))
|
||||
|
||||
# ── Detect ──────────────────────────────────────────────────────────
|
||||
|
||||
@@ -232,9 +257,8 @@ class TextMarkEngine:
|
||||
if image is None or image.size == 0:
|
||||
return det
|
||||
loc = self.locate(image)
|
||||
mask = self.extract_mask(image, loc)
|
||||
x, y, bw, bh = loc.bbox
|
||||
box = mask[y : y + bh, x : x + bw]
|
||||
box = self.extract_mask(image, loc) # box-sized mask (== old full-frame cropped to bbox)
|
||||
_x, _y, bw, bh = loc.bbox
|
||||
coverage = float((box > 0).sum()) / float(max(1, bw * bh))
|
||||
det.region = loc.bbox
|
||||
det.coverage = coverage
|
||||
@@ -254,7 +278,15 @@ class TextMarkEngine:
|
||||
|
||||
def _fixed_alpha_map(self, image: NDArray[Any]) -> tuple[NDArray[Any], tuple[int, int, int, int]] | None:
|
||||
"""Place the template by fixed width-relative geometry (pixel-exact at the
|
||||
captured width)."""
|
||||
captured width).
|
||||
|
||||
Returns the glyph-sized alpha BLOCK (shape ``(gh, gw)``) plus its placement
|
||||
``(ax, ay, gw, gh)``, not a full-frame ``(h, w)`` map. The map is non-zero
|
||||
only inside the glyph box and every consumer reads exactly that box, so a
|
||||
full-frame float32 map was O(image*4 bytes) of mostly zeros -- ~48 MB on a
|
||||
12 MP frame, and two were held at once (fixed + aligned). The block is
|
||||
byte-identical to the old full-frame map's ``[ay:ay+gh, ax:ax+gw]`` slice.
|
||||
"""
|
||||
c = self.config
|
||||
at = self._alpha_template()
|
||||
if at is None:
|
||||
@@ -268,22 +300,23 @@ class TextMarkEngine:
|
||||
else: # bottom-left
|
||||
ax = min(max(0, int(c.alpha_margin_x_frac * w)), max(0, w - gw))
|
||||
ay = max(0, h - int(c.alpha_margin_bottom_frac * w) - gh)
|
||||
amap = np.zeros((h, w), np.float32)
|
||||
amap[ay : ay + gh, ax : ax + gw] = cv2.resize(at, (gw, gh), interpolation=cv2.INTER_LINEAR)
|
||||
return amap, (ax, ay, gw, gh)
|
||||
block = cv2.resize(at, (gw, gh), interpolation=cv2.INTER_LINEAR)
|
||||
return block, (ax, ay, gw, gh)
|
||||
|
||||
def _aligned_alpha_map(self, image: NDArray[Any]) -> tuple[NDArray[Any], tuple[int, int, int, int]] | None:
|
||||
"""Register the captured template to the actual mark via a TM_CCOEFF_NORMED
|
||||
scale + position search. Returns ``(alpha_map, glyph_bbox)`` or None."""
|
||||
scale + position search. Returns the glyph-sized alpha BLOCK and its
|
||||
placement ``(ax, ay, gw, gh)`` (see :meth:`_fixed_alpha_map` for why the
|
||||
block, not a full-frame map), or None."""
|
||||
c = self.config
|
||||
at = self._alpha_template()
|
||||
sil = self._glyph_silhouette()
|
||||
if at is None or sil is None:
|
||||
return None
|
||||
h, w = image.shape[:2]
|
||||
w = image.shape[1]
|
||||
loc = self.locate(image)
|
||||
bx, by, bw, bh = loc.bbox
|
||||
box_mask = self.extract_mask(image, loc)[by : by + bh, bx : bx + bw]
|
||||
box_mask = self.extract_mask(image, loc) # box-sized (== old full-frame cropped to bbox)
|
||||
expected = c.alpha_width_frac * w
|
||||
best: tuple[float, int, int, int, int] | None = None
|
||||
for scale in np.linspace(*c.alpha_align_search):
|
||||
@@ -298,18 +331,17 @@ class TextMarkEngine:
|
||||
return None
|
||||
_, gw, gh, ox, oy = best
|
||||
ax, ay = bx + ox, by + oy
|
||||
amap = np.zeros((h, w), np.float32)
|
||||
amap[ay : ay + gh, ax : ax + gw] = cv2.resize(at, (gw, gh), interpolation=cv2.INTER_LINEAR)
|
||||
return amap, (ax, ay, gw, gh)
|
||||
block = cv2.resize(at, (gw, gh), interpolation=cv2.INTER_LINEAR)
|
||||
return block, (ax, ay, gw, gh)
|
||||
|
||||
def _apply_reverse_alpha(
|
||||
self, image: NDArray[Any], amap: NDArray[Any], region: tuple[int, int, int, int]
|
||||
) -> NDArray[Any]:
|
||||
"""Invert the alpha blend with ``amap``: ``original = (wm - a*logo)/(1-a)``.
|
||||
|
||||
``amap`` is zero everywhere except the glyph ``region`` (x, y, w, h), so the
|
||||
blend is a no-op (``(wm - 0)/(1 - 0) == wm``) outside it. Compute the math on
|
||||
the glyph crop only and copy the rest through unchanged -- byte-identical to a
|
||||
``amap`` is the glyph-sized alpha BLOCK for ``region`` (x, y, w, h); outside
|
||||
it the blend is a no-op (``(wm - 0)/(1 - 0) == wm``). Compute the math on the
|
||||
glyph crop only and copy the rest through unchanged -- byte-identical to a
|
||||
full-frame pass (a uint8 round-trip through float32 is exact), but O(glyph)
|
||||
instead of O(image): a full-frame pass costs ~275 ms on a 12 MP frame for a
|
||||
glyph that is <0.1% of it, and it runs once per candidate placement.
|
||||
@@ -319,12 +351,80 @@ class TextMarkEngine:
|
||||
x2, y2 = x1 + gw, y1 + gh
|
||||
if y1 >= y2 or x1 >= x2:
|
||||
return out
|
||||
a3 = np.clip(amap[y1:y2, x1:x2], 0.0, 1.0)[:, :, None]
|
||||
a3 = np.clip(amap, 0.0, 1.0)[:, :, None]
|
||||
logo = np.array(self.config.alpha_logo_bgr, np.float32)
|
||||
roi = out[y1:y2, x1:x2].astype(np.float32)
|
||||
out[y1:y2, x1:x2] = np.clip((roi - a3 * logo) / np.clip(1.0 - a3, 0.25, 1.0), 0, 255).astype(np.uint8)
|
||||
return out
|
||||
|
||||
def _reverse_alpha_oversubtracts(
|
||||
self, image: NDArray[Any], amap: NDArray[Any], region: tuple[int, int, int, int]
|
||||
) -> bool:
|
||||
"""True when reverse-alpha would darken the glyph footprint into a dark pit.
|
||||
|
||||
Ported from ``gemini_engine._reverse_alpha_oversubtracts`` (2026-06-20):
|
||||
PREDICT the reverse-alpha output at the bright glyph core directly from the
|
||||
INPUT and the captured alpha, ``(core_obs - a*logo)/(1-a)``, and trip when it
|
||||
lands more than ``_OVERSUB_DARK_MARGIN`` gray levels below the local
|
||||
background ring. Predicting from the input (not the produced output) keeps the
|
||||
gate independent of which placement the reverse-alpha picked, so a clean
|
||||
full-strength mark (whose strokes predict back to the background) never trips,
|
||||
while a mark fainter than the capture (over-subtracted into a ghost) does.
|
||||
"""
|
||||
ax, ay, gw, gh = region
|
||||
ih, iw = image.shape[:2]
|
||||
if gw < 4 or gh < 4:
|
||||
return False
|
||||
if float(amap.max()) < 0.2: # too faint a capture to over-subtract meaningfully
|
||||
return False
|
||||
body_box = amap >= _OVERSUB_BODY_ALPHA_FLOOR # glyph strokes
|
||||
if not bool(body_box.any()):
|
||||
return False
|
||||
pad = max(4, int(gh * _OVERSUB_RING_PAD_FRAC))
|
||||
ry1, ry2 = max(0, ay - pad), min(ih, ay + gh + pad)
|
||||
rx1, rx2 = max(0, ax - pad), min(iw, ax + gw + pad)
|
||||
ring = image[ry1:ry2, rx1:rx2].astype(np.float32).mean(axis=2)
|
||||
fy1, fy2, fx1, fx2 = ay - ry1, ay - ry1 + gh, ax - rx1, ax - rx1 + gw
|
||||
ring_mask = np.ones(ring.shape, dtype=bool)
|
||||
ring_mask[fy1:fy2, fx1:fx2] = False
|
||||
if int(ring_mask.sum()) < 10:
|
||||
return False
|
||||
# Predict the reverse-alpha output PER PIXEL over the glyph body -- exactly
|
||||
# the (obs - a*logo)/(1-a) math the remover applies -- so a cleanly captured
|
||||
# mark predicts back to the true background everywhere (no trip), while a mark
|
||||
# fainter than the capture predicts a body far below the local ring. The
|
||||
# per-pixel alpha (not a single peak value) keeps the prediction faithful
|
||||
# across the glyph's anti-aliased alpha gradient.
|
||||
obs = ring[fy1:fy2, fx1:fx2]
|
||||
a = np.clip(amap, 0.0, 0.99)
|
||||
logo = float(np.mean(self.config.alpha_logo_bgr))
|
||||
predicted = (obs - a * logo) / (1.0 - a)
|
||||
predicted_core = float(np.median(predicted[body_box]))
|
||||
bg = float(np.median(ring[ring_mask]))
|
||||
oversub = predicted_core < bg - _OVERSUB_DARK_MARGIN
|
||||
if oversub:
|
||||
logger.debug(
|
||||
"%s reverse-alpha over-subtracts: predicted core=%.1f bg=%.1f (margin %.0f) -> footprint inpaint",
|
||||
self.config.name,
|
||||
predicted_core,
|
||||
bg,
|
||||
_OVERSUB_DARK_MARGIN,
|
||||
)
|
||||
return oversub
|
||||
|
||||
def _inpaint_footprint(
|
||||
self, image: NDArray[Any], amap: NDArray[Any], region: tuple[int, int, int, int]
|
||||
) -> NDArray[Any]:
|
||||
"""Reconstruct the glyph footprint from its surroundings (used when
|
||||
reverse-alpha would over-subtract into a dark pit). Inpaints the ORIGINAL
|
||||
image over a dilated glyph mask, so the result never contains the darkened
|
||||
reverse-alpha pixels."""
|
||||
ax, ay, gw, gh = region
|
||||
mask = np.zeros(image.shape[:2], np.uint8)
|
||||
mask[ay : ay + gh, ax : ax + gw] = (amap > self.config.residual_alpha_floor).astype(np.uint8) * 255
|
||||
mask = cv2.dilate(mask, np.ones((_OVERSUB_INPAINT_DILATE, _OVERSUB_INPAINT_DILATE), np.uint8))
|
||||
return cv2.inpaint(image, mask, _OVERSUB_INPAINT_RADIUS, cv2.INPAINT_NS)
|
||||
|
||||
def remove_watermark_reverse_alpha(self, image: NDArray[Any], *, residual_inpaint: bool = True) -> NDArray[Any]:
|
||||
"""Recover the original pixels by inverting the alpha blend, then clear the
|
||||
residual outline with a thin inpaint over the glyph footprint.
|
||||
@@ -351,16 +451,32 @@ class TextMarkEngine:
|
||||
return image.copy()
|
||||
best_out: NDArray[Any] | None = None
|
||||
best_amap: NDArray[Any] | None = None
|
||||
best_region: tuple[int, int, int, int] | None = None
|
||||
best_residual = float("inf")
|
||||
for amap, region in maps:
|
||||
out = self._apply_reverse_alpha(image, amap, region)
|
||||
residual = self.detect(out).confidence
|
||||
if residual < best_residual:
|
||||
best_residual, best_out, best_amap = residual, out, amap
|
||||
if best_out is None or best_amap is None: # pragma: no cover - maps is non-empty
|
||||
best_residual, best_out, best_amap, best_region = residual, out, amap, region
|
||||
if best_out is None or best_amap is None or best_region is None: # pragma: no cover - maps is non-empty
|
||||
return image.copy()
|
||||
# Over-subtraction guard: on a dark/mid-tone background the captured alpha can
|
||||
# over-estimate the mark's opacity and reverse-alpha leaves a darker-than-
|
||||
# background ghost. When the recovered glyph body sits far below the local
|
||||
# ring, reconstruct the footprint from its surroundings instead of shipping the
|
||||
# dark pit (the thin residual inpaint cannot fix a footprint-wide darkening).
|
||||
if self._reverse_alpha_oversubtracts(image, best_amap, best_region):
|
||||
return self._inpaint_footprint(image, best_amap, best_region)
|
||||
if residual_inpaint:
|
||||
# Embed the glyph-sized alpha block into a full-frame uint8 mask only for
|
||||
# the inpaint (cv2.inpaint needs a mask matching best_out). One uint8
|
||||
# full-frame array, built once, vs the old two full-frame float32 maps;
|
||||
# byte-identical to thresholding the old full-frame float32 map (zero
|
||||
# outside the block, so the dilate/inpaint see the same mask).
|
||||
ax, ay, gw, gh = best_region
|
||||
rm = np.zeros(best_out.shape[:2], np.uint8)
|
||||
rm[ay : ay + gh, ax : ax + gw] = (best_amap > c.residual_alpha_floor).astype(np.uint8) * 255
|
||||
kernel = np.ones((c.residual_dilate, c.residual_dilate), np.uint8)
|
||||
rm = cv2.dilate((best_amap > c.residual_alpha_floor).astype(np.uint8) * 255, kernel)
|
||||
rm = cv2.dilate(rm, kernel)
|
||||
best_out = cv2.inpaint(best_out, rm, c.residual_inpaint_radius, cv2.INPAINT_NS)
|
||||
return best_out
|
||||
@@ -189,6 +189,33 @@ _adaptive_polish_option = click.option(
|
||||
"no-op there. Pass --no-adaptive-polish to disable. Independent of --unsharp/--humanize.",
|
||||
)
|
||||
|
||||
|
||||
# Tiled-diffusion knobs, shared by the diffusion commands (invisible/all/batch).
|
||||
# Tiling is the lossless alternative to --max-resolution for large inputs that OOM
|
||||
# on MPS/GPU: process at native resolution in overlapping, feather-blended tiles.
|
||||
def _tile_options(f: Any) -> Any:
|
||||
"""Apply the --tile / --tile-size / --tile-overlap options to a command."""
|
||||
f = click.option(
|
||||
"--tile-overlap",
|
||||
type=int,
|
||||
default=128,
|
||||
help="Overlap between adjacent tiles in px (feather-blended, no seam). Default 128.",
|
||||
)(f)
|
||||
f = click.option(
|
||||
"--tile-size",
|
||||
type=int,
|
||||
default=1024,
|
||||
help="Tile dimension in px for --tile (SDXL's training size). Default 1024.",
|
||||
)(f)
|
||||
return click.option(
|
||||
"--tile/--no-tile",
|
||||
default=False,
|
||||
help="Process large images in overlapping tiles instead of one forward pass -- the lossless "
|
||||
"alternative to --max-resolution for inputs that OOM on MPS/GPU. Engages only when the long "
|
||||
"side exceeds --tile-size; pair with --max-resolution 0 (default) to keep native resolution. Default off.",
|
||||
)(f)
|
||||
|
||||
|
||||
# HuggingFace model + CFG knobs, shared by the diffusion commands (invisible/all/batch)
|
||||
# so the surface stays identical across them.
|
||||
_model_option = click.option(
|
||||
@@ -226,15 +253,16 @@ def _normalize_pipeline(ctx: click.Context, param: click.Parameter, value: str |
|
||||
return normalized
|
||||
|
||||
|
||||
# ``controlnet`` (the default-SELECTED value) and ``sdxl`` (plain SDXL img2img) are the
|
||||
# two current profiles; ``default`` is an OUTDATED back-compat alias for ``sdxl``
|
||||
# (warned + normalized away by _normalize_pipeline).
|
||||
_PIPELINE_CHOICES = ["sdxl", "controlnet", "default"]
|
||||
# ``controlnet`` (the default-SELECTED value), ``sdxl`` (plain SDXL img2img) and
|
||||
# ``qwen`` (Qwen-Image, CUDA/cloud-class) are the current profiles; ``default`` is an
|
||||
# OUTDATED back-compat alias for ``sdxl`` (warned + normalized away by _normalize_pipeline).
|
||||
_PIPELINE_CHOICES = ["sdxl", "controlnet", "qwen", "default"]
|
||||
_PIPELINE_HELP = (
|
||||
"Pipeline profile. controlnet (DEFAULT) = SDXL + canny ControlNet that preserves "
|
||||
"text/faces via edge conditioning while removing SynthID; sdxl = plain SDXL img2img "
|
||||
"(lighter, no extra model download, but leaves SynthID on flat-graphic content). "
|
||||
"('default' is an OUTDATED alias for 'sdxl' -- use sdxl or controlnet.)"
|
||||
"(lighter, no extra model download, but leaves SynthID on flat-graphic content); "
|
||||
"qwen = Qwen-Image (20B, Apache-2.0) img2img, best text/structure preservation but "
|
||||
"CUDA/cloud-class (does not fit MPS). ('default' is an OUTDATED alias for 'sdxl'.)"
|
||||
)
|
||||
|
||||
# Shared --pipeline / --strength decorators so the three diffusion commands
|
||||
@@ -668,6 +696,7 @@ def cmd_erase(
|
||||
@_guidance_scale_option
|
||||
@_auto_option
|
||||
@_adaptive_polish_option
|
||||
@_tile_options
|
||||
@click.pass_context
|
||||
def cmd_invisible(
|
||||
ctx: click.Context,
|
||||
@@ -689,6 +718,9 @@ def cmd_invisible(
|
||||
guidance_scale: float | None,
|
||||
auto: bool,
|
||||
adaptive_polish: bool,
|
||||
tile: bool,
|
||||
tile_size: int,
|
||||
tile_overlap: int,
|
||||
) -> None:
|
||||
"""Remove invisible AI watermarks (SynthID, StableSignature, TreeRing).
|
||||
|
||||
@@ -730,7 +762,7 @@ def cmd_invisible(
|
||||
vendor = vendor_for_strength(source)
|
||||
console.print(f" Input: {source.name}")
|
||||
console.print(f" Pipeline: {pipeline}")
|
||||
console.print(f" Strength: {resolve_strength(strength, vendor)} Steps: {steps}")
|
||||
console.print(f" Strength: {resolve_strength(strength, vendor, pipeline)} Steps: {steps}")
|
||||
|
||||
t0 = time.monotonic()
|
||||
result_path = engine.remove_watermark(
|
||||
@@ -747,6 +779,9 @@ def cmd_invisible(
|
||||
min_resolution=min_resolution,
|
||||
upscaler=upscaler,
|
||||
vendor=vendor,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
)
|
||||
elapsed = time.monotonic() - t0
|
||||
|
||||
@@ -846,6 +881,13 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
|
||||
|
||||
_banner()
|
||||
verdict = {True: "AI-generated", False: "not AI", None: "unknown"}[report.is_ai_generated]
|
||||
# Sharpen the True verdict when the C2PA source type says the image is a real
|
||||
# photo with an AI-composited region rather than a full AI generation, so the
|
||||
# caller (and the user) can tell "scrub the whole frame" from "scrub the AI region".
|
||||
if report.is_ai_generated and report.ai_source_kind == "enhanced":
|
||||
verdict = "AI-enhanced (real content with an AI-composited region)"
|
||||
elif report.is_ai_generated and report.ai_source_kind == "generated":
|
||||
verdict = "AI-generated (fully synthetic)"
|
||||
console.print(f"\n Verdict: {verdict} (confidence: {report.confidence})")
|
||||
console.print(f" Platform: {report.platform or 'undetermined'}")
|
||||
|
||||
@@ -917,6 +959,7 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
|
||||
@_guidance_scale_option
|
||||
@_auto_option
|
||||
@_adaptive_polish_option
|
||||
@_tile_options
|
||||
@click.pass_context
|
||||
def cmd_all(
|
||||
ctx: click.Context,
|
||||
@@ -940,6 +983,9 @@ def cmd_all(
|
||||
guidance_scale: float | None,
|
||||
auto: bool,
|
||||
adaptive_polish: bool,
|
||||
tile: bool,
|
||||
tile_size: int,
|
||||
tile_overlap: int,
|
||||
) -> None:
|
||||
"""Remove ALL watermarks: visible + invisible + metadata.
|
||||
|
||||
@@ -1029,7 +1075,7 @@ def cmd_all(
|
||||
# already lost its C2PA to the visible-removal pass, so reading it would
|
||||
# always resolve to the unknown-vendor default.
|
||||
vendor = vendor_for_strength(source)
|
||||
console.print(f" Strength: {resolve_strength(strength, vendor)} Steps: {steps}")
|
||||
console.print(f" Strength: {resolve_strength(strength, vendor, pipeline)} Steps: {steps}")
|
||||
inv_engine.remove_watermark(
|
||||
image_path=tmp_path,
|
||||
output_path=tmp_path,
|
||||
@@ -1044,6 +1090,9 @@ def cmd_all(
|
||||
min_resolution=min_resolution,
|
||||
upscaler=upscaler,
|
||||
vendor=vendor,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
)
|
||||
console.print(" Invisible watermark removed")
|
||||
|
||||
@@ -1121,6 +1170,9 @@ def _process_batch_image(
|
||||
model: str | None = None,
|
||||
guidance_scale: float | None = None,
|
||||
adaptive_polish: bool = False,
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
) -> None:
|
||||
"""Process a single image for batch mode.
|
||||
|
||||
@@ -1179,6 +1231,9 @@ def _process_batch_image(
|
||||
max_resolution=max_resolution,
|
||||
min_resolution=min_resolution,
|
||||
upscaler=upscaler,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
# Detect the vendor from the pristine original (`img_path`), not the
|
||||
# visible-processed `out_path` whose C2PA is already gone.
|
||||
vendor=vendor_for_strength(img_path),
|
||||
@@ -1238,6 +1293,7 @@ def _process_batch_image(
|
||||
@_guidance_scale_option
|
||||
@_auto_option
|
||||
@_adaptive_polish_option
|
||||
@_tile_options
|
||||
@click.pass_context
|
||||
def cmd_batch(
|
||||
ctx: click.Context,
|
||||
@@ -1261,6 +1317,9 @@ def cmd_batch(
|
||||
guidance_scale: float | None,
|
||||
auto: bool,
|
||||
adaptive_polish: bool,
|
||||
tile: bool,
|
||||
tile_size: int,
|
||||
tile_overlap: int,
|
||||
) -> None:
|
||||
"""Process all images in a directory."""
|
||||
_banner()
|
||||
@@ -1321,6 +1380,9 @@ def cmd_batch(
|
||||
model=model,
|
||||
guidance_scale=guidance_scale,
|
||||
adaptive_polish=adaptive_polish,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
)
|
||||
processed += 1
|
||||
|
||||
|
||||
@@ -42,6 +42,7 @@ from remove_ai_watermarks.metadata import (
|
||||
)
|
||||
from remove_ai_watermarks.noai.c2pa import cbor_text_after, extract_c2pa_info, soft_binding_vendors_in
|
||||
from remove_ai_watermarks.noai.constants import C2PA_AI_TOOLS, C2PA_AI_VENDORS, C2PA_ISSUERS
|
||||
from remove_ai_watermarks.watermark_registry import GEMINI_SPARKLE_TRUST_CONF
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
@@ -57,11 +58,14 @@ log = logging.getLogger(__name__)
|
||||
_SCAN_BYTES = 1024 * 1024
|
||||
|
||||
# Visible-sparkle confidence above which the signal is trusted as provenance.
|
||||
# Stricter than the removal default (0.25): on the corpus, Gemini-family
|
||||
# sparkles score >= 0.56 while non-sparkle images top out at 0.49, so 0.5
|
||||
# cleanly separates them and avoids false positives when sparkle is the only
|
||||
# signal (e.g. an OpenAI image scored 0.37 -- below threshold, correctly dropped).
|
||||
_SPARKLE_THRESHOLD = 0.5
|
||||
# Shared with the removal arbitration (watermark_registry.GEMINI_SPARKLE_TRUST_CONF)
|
||||
# so the provenance "is there a sparkle" verdict and the removal "take the sparkle"
|
||||
# decision can never drift apart -- the detect-vs-remove desync the retained-corpus
|
||||
# mining surfaced (2026-06-20). On the corpus Gemini-family sparkles score >= 0.56
|
||||
# while non-sparkle images top out at 0.49, so 0.5 cleanly separates them and avoids
|
||||
# false positives when the sparkle is the only signal (e.g. an OpenAI image scored
|
||||
# 0.37 -- below threshold, correctly dropped).
|
||||
_SPARKLE_THRESHOLD = GEMINI_SPARKLE_TRUST_CONF
|
||||
|
||||
# Issuer (C2PA signer) -> human-readable generating platform, derived from the
|
||||
# single C2PA_AI_VENDORS registry. Ordered: when a manifest names several issuers
|
||||
@@ -132,6 +136,14 @@ class ProvenanceReport:
|
||||
is_ai_generated: bool | None # True / False is never asserted; None = unknown
|
||||
platform: str | None
|
||||
confidence: str # "high" | "medium" | "none"
|
||||
# Coarse AI-origin kind from the C2PA digital-source-type, so a caller can
|
||||
# branch on full generation vs an AI-touched real photo:
|
||||
# "generated" -- digitalSourceType trainedAlgorithmicMedia (fully AI).
|
||||
# "enhanced" -- compositeWithTrainedAlgorithmicMedia (real content with an
|
||||
# AI-composited region; scrub the AI region, keep the photo).
|
||||
# None -- no C2PA AI source-type (verdict, if AI, came from another
|
||||
# signal: IPTC, AIGC, local gen params, xAI, ...).
|
||||
ai_source_kind: str | None = None
|
||||
watermarks: list[str] = field(default_factory=list[str])
|
||||
signals: list[Signal] = field(default_factory=list["Signal"])
|
||||
caveats: list[str] = field(default_factory=list[str])
|
||||
@@ -484,9 +496,18 @@ def identify(image_path: Path, *, check_visible: bool = True, check_invisible: b
|
||||
# ── C2PA Content Credentials ────────────────────────────────────
|
||||
has_c2pa = bool(info) or c2pa_marker_in(head)
|
||||
issuers = [info["issuer"]] if info.get("issuer") else _issuers_in(head)
|
||||
c2pa_is_ai = "trainedAlgorithmicMedia" in info.get("source_type", "") or any(
|
||||
m in head for m in (b"trainedAlgorithmicMedia", b"compositeWithTrainedAlgorithmicMedia")
|
||||
)
|
||||
# Full AI generation (trainedAlgorithmicMedia) vs an AI-enhanced real photo
|
||||
# (compositeWithTrainedAlgorithmicMedia). The structured kind is parsed once in
|
||||
# noai.c2pa._populate_registry_fields (covers PNG + any container the c2pa-python
|
||||
# reader handles); fall back to a raw head scan for the non-PNG raw-blob path
|
||||
# where extract_c2pa_info returns {}. Full generation wins when both appear.
|
||||
c2pa_source_kind = info.get("ai_source_kind")
|
||||
if c2pa_source_kind is None:
|
||||
if b"trainedAlgorithmicMedia" in head:
|
||||
c2pa_source_kind = "generated"
|
||||
elif b"compositeWithTrainedAlgorithmicMedia" in head:
|
||||
c2pa_source_kind = "enhanced"
|
||||
c2pa_is_ai = c2pa_source_kind is not None
|
||||
# Generator string (for the signal detail): structured for PNG, CBOR-scanned
|
||||
# for other containers. Best-effort -- some manifests key it as
|
||||
# `claim_generator_info` (Pixel), so this can be None even when a device is
|
||||
@@ -734,6 +755,9 @@ def identify(image_path: Path, *, check_visible: bool = True, check_invisible: b
|
||||
is_ai_generated=is_ai,
|
||||
platform=platform,
|
||||
confidence=confidence,
|
||||
# Only meaningful when the AI verdict actually came from the C2PA source
|
||||
# type; a non-C2PA AI signal (IPTC/AIGC/local gen) leaves it None.
|
||||
ai_source_kind=c2pa_source_kind if (is_ai and has_c2pa) else None,
|
||||
watermarks=watermarks,
|
||||
signals=signals,
|
||||
caveats=caveats,
|
||||
|
||||
@@ -103,8 +103,9 @@ class InvisibleEngine:
|
||||
device: Device for inference (auto/cpu/mps/cuda/xpu). None = auto.
|
||||
pipeline: Pipeline profile. "controlnet" (DEFAULT; SDXL + canny ControlNet
|
||||
that preserves text/face structure via edge conditioning while removing
|
||||
SynthID) or "sdxl" (plain SDXL img2img, lighter but leaves SynthID on
|
||||
flat-graphic content). "default" is a back-compat alias for "sdxl".
|
||||
SynthID), "sdxl" (plain SDXL img2img, lighter but leaves SynthID on
|
||||
flat-graphic content), or "qwen" (Qwen-Image 20B img2img, best text/
|
||||
structure preservation but CUDA/cloud-class). "default" aliases "sdxl".
|
||||
hf_token: HuggingFace API token.
|
||||
progress_callback: Optional callback for progress messages.
|
||||
controlnet_conditioning_scale: ControlNet structure-preservation
|
||||
@@ -170,6 +171,9 @@ class InvisibleEngine:
|
||||
unsharp: float = 0.0,
|
||||
adaptive_polish: bool = False,
|
||||
upscaler: str = "lanczos",
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
) -> Path:
|
||||
"""Remove invisible watermark from an image.
|
||||
|
||||
@@ -205,6 +209,13 @@ class InvisibleEngine:
|
||||
via the ``esrgan`` extra). Only applies when UPscaling (the floor
|
||||
case); a ``max_resolution`` downscale always uses Lanczos. Falls back
|
||||
to Lanczos if the extra is absent.
|
||||
tile: Process the diffusion pass in overlapping tiles instead of one
|
||||
forward pass -- the lossless alternative to ``max_resolution`` for
|
||||
large inputs that OOM on MPS/GPU. Engages only when the long side
|
||||
exceeds ``tile_size``. Pair with ``max_resolution=0`` (the default)
|
||||
so the input keeps its native resolution.
|
||||
tile_size: Tile dimension in px (default 1024).
|
||||
tile_overlap: Overlap between adjacent tiles in px (default 128).
|
||||
|
||||
Returns:
|
||||
Path to the cleaned image.
|
||||
@@ -261,6 +272,9 @@ class InvisibleEngine:
|
||||
guidance_scale=guidance_scale,
|
||||
seed=seed,
|
||||
vendor=vendor,
|
||||
tile=tile,
|
||||
tile_size=tile_size,
|
||||
tile_overlap=tile_overlap,
|
||||
)
|
||||
|
||||
# Post-processing chain: decode the diffusion output ONCE, apply the
|
||||
|
||||
@@ -871,23 +871,32 @@ def remove_ai_metadata(
|
||||
# codestream bit-for-bit. MP4/MOV/M4A are ISOBMFF too, so the same top-level
|
||||
# uuid/jumb box walker applies. Route by suffix OR by an ``ftyp`` content
|
||||
# sniff, so a correctly-shaped container is handled whatever its extension.
|
||||
from remove_ai_watermarks.noai.isobmff import blank_ai_xmp_packets, is_isobmff, strip_c2pa_boxes
|
||||
from remove_ai_watermarks.noai.isobmff import (
|
||||
blank_ai_exif_tokens,
|
||||
blank_ai_xmp_packets,
|
||||
is_isobmff,
|
||||
strip_c2pa_boxes,
|
||||
)
|
||||
|
||||
with open(source_path, "rb") as f:
|
||||
head = f.read(12)
|
||||
if source_path.suffix.lower() in _ISOBMFF_EXTS or is_isobmff(head):
|
||||
data = source_path.read_bytes()
|
||||
# Top-level uuid/jumb boxes (C2PA + AI-label XMP), then AI-label XMP that
|
||||
# lives inside a meta-box ``mime`` item (HEIF/AVIF) -- blanked in place so
|
||||
# box sizes and iloc offsets stay valid and the coded image is untouched.
|
||||
# Top-level uuid/jumb boxes (C2PA + AI-label XMP), then the meta-box items
|
||||
# the top-level stripper can't reach (HEIF/AVIF store them in mdat/idat):
|
||||
# AI-label XMP packets and AI-generator tokens in an Exif item -- both
|
||||
# blanked in place (same length) so box sizes and iloc offsets stay valid
|
||||
# and the coded image is untouched.
|
||||
cleaned, stripped = strip_c2pa_boxes(data)
|
||||
cleaned, blanked = blank_ai_xmp_packets(cleaned)
|
||||
cleaned, exif_blanked = blank_ai_exif_tokens(cleaned)
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_bytes(cleaned)
|
||||
logger.info(
|
||||
"Stripped %d AI-provenance box(es), blanked %d meta-box XMP packet(s) → %s",
|
||||
"Stripped %d AI-provenance box(es), blanked %d meta-box XMP packet(s) + %d EXIF token(s) → %s",
|
||||
stripped,
|
||||
blanked,
|
||||
exif_blanked,
|
||||
output_path,
|
||||
)
|
||||
return output_path
|
||||
|
||||
@@ -363,14 +363,22 @@ def _populate_registry_fields(buf: bytes, c2pa_info: dict[str, Any]) -> bool:
|
||||
# Digital source type (matched anywhere in the store, including ingredient
|
||||
# manifests -- a ChatGPT edit of a Sora generation carries the AI marker on
|
||||
# the parent, not the active manifest).
|
||||
# ``ai_source_kind`` is the structured generated-vs-enhanced split the caller
|
||||
# branches on (full-frame scrub vs region-targeted clean); ``source_type`` is the
|
||||
# human-readable form. The two byte strings are unambiguous:
|
||||
# "compositeWithTrainedAlgorithmicMedia" capitalizes the inner "Trained", so a
|
||||
# lowercase "trainedAlgorithmicMedia" match is standalone full generation, which
|
||||
# wins when both appear (an edit chain).
|
||||
ai_source = False
|
||||
if b"trainedAlgorithmicMedia" in buf:
|
||||
c2pa_info["source_type"] = "trainedAlgorithmicMedia (AI-generated)"
|
||||
c2pa_info["ai_source_kind"] = "generated"
|
||||
ai_source = True
|
||||
elif b"algorithmicMedia" in buf:
|
||||
c2pa_info["source_type"] = "algorithmicMedia"
|
||||
elif b"compositeWithTrainedAlgorithmicMedia" in buf:
|
||||
c2pa_info["source_type"] = "compositeWithTrainedAlgorithmicMedia (AI-enhanced)"
|
||||
c2pa_info["ai_source_kind"] = "enhanced"
|
||||
ai_source = True
|
||||
|
||||
# SynthID pixel-watermark proxy: a C2PA manifest from a SynthID-using
|
||||
|
||||
@@ -122,11 +122,57 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
|
||||
C2paAiVendor(
|
||||
b"volcengine", "ByteDance (Volcano Engine)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"
|
||||
),
|
||||
# Some Volcano Engine certs name the signer with the Chinese legal entity
|
||||
# "北京火山引擎科技有限公司" (Beijing Volcano Engine Technology Co., Ltd.) rather
|
||||
# than the latin "volcengine" -- the latin needle misses it entirely, so real
|
||||
# ByteDance output was un-attributed in production traffic. The issuer is the
|
||||
# UTF-8 of the Chinese name (it appears UTF-8-encoded in the manifest-store
|
||||
# JSON and the raw caBX bytes alike); it normalizes to the same "ByteDance"
|
||||
# needle and platform as the volcengine row, so the two collapse together for
|
||||
# clash detection. Verified against the mined retained corpus, 2026-06-20.
|
||||
C2paAiVendor(
|
||||
"北京火山引擎科技有限公司".encode(),
|
||||
"ByteDance (Volcano Engine)",
|
||||
"ByteDance (Doubao / Jimeng / Volcano Engine)",
|
||||
"ByteDance",
|
||||
),
|
||||
# ByteDance's international brand (BytePlus / Seedream / Seededit) signs its
|
||||
# cert as "Byteplus Pte. Ltd." -- the bare ``volcengine`` needle misses it, so
|
||||
# real BytePlus AI output was mis-attributed (an incidental "Adobe XMP" string
|
||||
# in the file's XMP made it read "Adobe Firefly"). Adding the issuer means the
|
||||
# clean manifest issuer matches "BytePlus (ByteDance)" directly. The platform
|
||||
# string mirrors the volcengine row: both share the "ByteDance" needle, so the
|
||||
# earlier row's label wins anyway -- they normalize together for clash
|
||||
# detection. Verified on real signed files in production traffic, 2026-06-19.
|
||||
C2paAiVendor(b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
|
||||
# Canva Magic Media signs AI-generated images as "Canva" with a generic
|
||||
# c2pa-rs claim generator + trainedAlgorithmicMedia; without this entry the
|
||||
# source read AI but no platform was attributed. Verified on real signed files
|
||||
# in production traffic, 2026-06-19. Canva does not use SynthID.
|
||||
C2paAiVendor(b"Canva", "Canva", "Canva (Magic Media)", "Canva"),
|
||||
# ElevenLabs is a pure generative-AI company (AI voice / audio, and image /
|
||||
# video via its API); it signs output as "Eleven Labs Inc.", so the C2PA
|
||||
# manifest alone marks AI generation. Verified against the mined retained
|
||||
# corpus, 2026-06-20. ElevenLabs does not use SynthID.
|
||||
C2paAiVendor(b"Eleven Labs", "ElevenLabs", "ElevenLabs", "ElevenLabs"),
|
||||
# Truepic is a C2PA signing authority, not an AI generator: no platform label,
|
||||
# never asserts is_ai (the verdict comes from the digital-source-type).
|
||||
C2paAiVendor(b"Truepic", "Truepic", None, None),
|
||||
)
|
||||
|
||||
# Deliberately NOT registered as AI-generation vendors (mined-corpus candidates
|
||||
# evaluated 2026-06-20):
|
||||
# - TikTok Inc.: signs C2PA as a content-provenance / AI-labeling authority on
|
||||
# uploads, not as an image generator. The is_ai verdict keys off the
|
||||
# digitalSourceType (trainedAlgorithmicMedia), which is already honored; a
|
||||
# bare TikTok signer marks distribution provenance, not generation, so adding
|
||||
# it as a generator needle would mis-label human uploads as AI.
|
||||
# - PixelBin.io (issuer "Fynd"): an image transformation / optimization / CDN
|
||||
# service. Its C2PA stamps a transform/upload step, not a generation event.
|
||||
# Both are excluded to avoid false-positive AI attribution; re-evaluate only
|
||||
# against a real signed file whose manifest carries a trainedAlgorithmicMedia
|
||||
# digital-source type produced by the vendor itself.
|
||||
|
||||
# Derived view -- add a vendor to C2PA_AI_VENDORS above, not here.
|
||||
# C2PA issuer signature -> resolved org name, for the manifest byte-scan.
|
||||
C2PA_ISSUERS: dict[bytes, str] = {v.issuer: v.org for v in C2PA_AI_VENDORS}
|
||||
|
||||
@@ -20,7 +20,7 @@ from __future__ import annotations
|
||||
import logging
|
||||
import re
|
||||
import struct
|
||||
from typing import TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
@@ -157,10 +157,11 @@ def strip_c2pa_boxes(data: bytes) -> tuple[bytes, int]:
|
||||
All other boxes (incl. ``mdat`` / codestream) are emitted verbatim, so pixel
|
||||
and audio data is preserved bit-for-bit. Non-ISOBMFF input is returned
|
||||
unchanged. Despite the name this also covers MP4/MOV/M4A video and audio
|
||||
(all ISOBMFF). NOTE: this drops only top-level boxes. An AI-label XMP packet
|
||||
stored as an *item inside the ``meta`` box* (typical for AVIF/HEIF) is handled
|
||||
separately by :func:`blank_ai_xmp_packets`; an ``Exif`` meta-box item is still
|
||||
not removed (would need meta-box surgery) and remains a documented limitation.
|
||||
(all ISOBMFF). NOTE: this drops only top-level boxes. AI metadata stored as an
|
||||
*item inside the ``meta`` box* (typical for AVIF/HEIF) is handled separately and
|
||||
in place (same length, no offset rewrite): AI-label XMP by
|
||||
:func:`blank_ai_xmp_packets`, and AI-generator tokens in an ``Exif`` item by
|
||||
:func:`blank_ai_exif_tokens`.
|
||||
"""
|
||||
if not is_isobmff(data):
|
||||
return data, 0
|
||||
@@ -223,3 +224,64 @@ def blank_ai_xmp_packets(data: bytes) -> tuple[bytes, int]:
|
||||
return packet
|
||||
|
||||
return _XMP_PACKET_RE.sub(_scrub, data), blanked
|
||||
|
||||
|
||||
# EXIF TIFF byte-order headers: little-endian (II 0x2a 0x00) and big-endian
|
||||
# (MM 0x00 0x2a). A HEIF/AVIF ``Exif`` meta-box item stores its TIFF block in
|
||||
# ``mdat`` / ``idat``, so the block (and these headers) appear in the raw bytes.
|
||||
_TIFF_HEADERS: tuple[bytes, ...] = (b"II\x2a\x00", b"MM\x00\x2a")
|
||||
# How far past a TIFF header an EXIF block plausibly extends; bounds the slice we
|
||||
# hand to piexif and search within (EXIF blocks are small kilobyte-scale).
|
||||
_EXIF_WINDOW = 256 * 1024
|
||||
|
||||
|
||||
def blank_ai_exif_tokens(data: bytes) -> tuple[bytes, int]:
|
||||
"""Overwrite (with spaces, in place) any AI-generator token in an EXIF block
|
||||
stored as an ISOBMFF ``meta``-box ``Exif`` item; return ``(data, blanked_count)``.
|
||||
|
||||
HEIF/AVIF can carry EXIF as a ``meta``-box ``Exif`` item whose TIFF bytes live
|
||||
in ``mdat`` / ``idat`` -- out of reach of the top-level box stripper, and (when
|
||||
no pillow-heif plugin is installed) of the PIL EXIF reader too, so an AI
|
||||
``Software`` / ``Make`` / ``Artist`` / ``ImageDescription`` tag there survived
|
||||
``remove_ai_metadata`` (a documented gap). This locates EXIF TIFF blocks by
|
||||
their byte-order header, **validates each with piexif** (so a coincidental
|
||||
II/MM run in pixel data is ignored -- it will not parse as a TIFF IFD), and
|
||||
overwrites any value carrying an ``AI_GENERATOR_TOKENS`` token with spaces of
|
||||
the SAME length. Because the replacement is same-length, every box size and
|
||||
``iloc`` offset stays valid and the coded image is untouched -- only the AI tag
|
||||
content is destroyed; camera/editor EXIF without an AI token is left intact
|
||||
(mirrors ``metadata._scrub_ai_exif`` and ``blank_ai_xmp_packets``).
|
||||
"""
|
||||
import piexif
|
||||
|
||||
from remove_ai_watermarks.noai.constants import AI_GENERATOR_TOKENS
|
||||
|
||||
ai_tags = (
|
||||
piexif.ImageIFD.Software,
|
||||
piexif.ImageIFD.Make,
|
||||
piexif.ImageIFD.Artist,
|
||||
piexif.ImageIFD.ImageDescription,
|
||||
)
|
||||
out = bytearray(data)
|
||||
blanked = 0
|
||||
for header in _TIFF_HEADERS:
|
||||
pos = data.find(header)
|
||||
while pos != -1:
|
||||
window = bytes(out[pos : pos + _EXIF_WINDOW])
|
||||
ifd: dict[int, Any] = {}
|
||||
try:
|
||||
ifd = piexif.load(window).get("0th", {})
|
||||
except Exception:
|
||||
ifd = {}
|
||||
for tag in ai_tags:
|
||||
value = ifd.get(tag)
|
||||
if not isinstance(value, bytes):
|
||||
continue
|
||||
if any(token in value.decode("latin1", "replace").lower() for token in AI_GENERATOR_TOKENS):
|
||||
# Blank the value bytes in place, within this EXIF block only.
|
||||
vpos = out.find(value, pos, pos + _EXIF_WINDOW)
|
||||
if vpos != -1:
|
||||
out[vpos : vpos + len(value)] = b" " * len(value)
|
||||
blanked += 1
|
||||
pos = data.find(header, pos + len(header))
|
||||
return bytes(out), blanked
|
||||
@@ -0,0 +1,195 @@
|
||||
"""Sliding-window tiled diffusion for large images.
|
||||
|
||||
The img2img / ControlNet pipeline denoises the WHOLE image in one forward pass,
|
||||
so it OOMs on MPS/GPU above ~2K (issue #10). Tiling splits the image into
|
||||
overlapping tiles -- each kept near SDXL's ~1024 training size -- regenerates
|
||||
each tile independently, and feather-blends the overlaps. The result is processed
|
||||
at NATIVE resolution with no seam: the lossless alternative to the
|
||||
``--max-resolution`` downscale (which trades quality for a smaller forward pass).
|
||||
|
||||
The geometry (``plan_tiles``) and the blend weighting (``feather_weights``) are
|
||||
pure functions, unit-tested without the diffusion model. ``run_tiled`` is the
|
||||
orchestration loop; it takes a ``generate_tile`` callable (one img2img/ControlNet
|
||||
pass on a single PIL tile) so it stays decoupled from the pipeline internals.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, Any, NamedTuple
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
from numpy.typing import NDArray
|
||||
from PIL import Image as PILImage
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Strictly-positive floor for the accumulated blend weights so a region covered
|
||||
# by a single feathered tile edge (an image corner, no neighbour to blend with)
|
||||
# never divides by zero.
|
||||
_WEIGHT_EPS = 1e-3
|
||||
|
||||
|
||||
class Tile(NamedTuple):
|
||||
"""A tile crop box in the source image: top-left ``(x, y)`` + ``width``/``height``."""
|
||||
|
||||
x: int
|
||||
y: int
|
||||
width: int
|
||||
height: int
|
||||
|
||||
|
||||
def _axis_positions(length: int, tile: int, overlap: int) -> list[int]:
|
||||
"""Tile start offsets along one axis, last tile flush to the far edge.
|
||||
|
||||
Every interior tile is exactly ``tile`` long; the final tile is pulled back
|
||||
to ``length - tile`` so it ends exactly at the edge (it simply overlaps its
|
||||
predecessor a little more). Keeping all tiles the same size is what lets the
|
||||
diffusion pass run at SDXL's preferred dimension on every tile.
|
||||
"""
|
||||
if tile <= 0:
|
||||
raise ValueError(f"tile must be positive, got {tile}")
|
||||
if length <= tile:
|
||||
return [0]
|
||||
# Guarantee forward progress even on a pathological overlap >= tile.
|
||||
overlap = min(max(overlap, 0), tile - 1)
|
||||
step = tile - overlap
|
||||
positions = list(range(0, length - tile + 1, step))
|
||||
last = length - tile
|
||||
if positions[-1] != last:
|
||||
positions.append(last)
|
||||
return positions
|
||||
|
||||
|
||||
def plan_tiles(width: int, height: int, tile_size: int, overlap: int) -> list[Tile]:
|
||||
"""Lay out a grid of overlapping tiles covering ``width`` x ``height``.
|
||||
|
||||
All tiles are ``min(tile_size, width)`` x ``min(tile_size, height)`` (uniform
|
||||
size; the image itself when it fits in one tile). Returned in row-major order.
|
||||
"""
|
||||
xs = _axis_positions(width, tile_size, overlap)
|
||||
ys = _axis_positions(height, tile_size, overlap)
|
||||
tile_w = min(tile_size, width)
|
||||
tile_h = min(tile_size, height)
|
||||
return [Tile(x, y, tile_w, tile_h) for y in ys for x in xs]
|
||||
|
||||
|
||||
def feather_weights(width: int, height: int, overlap: int) -> NDArray[Any]:
|
||||
"""A 2D blend window: ~1 in the interior, ramping down toward each edge.
|
||||
|
||||
Separable linear taper over ``overlap`` pixels from every edge (capped at
|
||||
half the tile so short tiles still taper symmetrically). Strictly positive
|
||||
everywhere, so the normalised blend is well-defined even at an image corner
|
||||
that only one tile covers.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
def ramp(n: int) -> NDArray[Any]:
|
||||
w = np.ones(n, dtype=np.float32)
|
||||
if overlap > 0 and n > 1:
|
||||
ramp_len = min(overlap, max(1, n // 2))
|
||||
taper = (np.arange(ramp_len, dtype=np.float32) + 1.0) / (ramp_len + 1.0)
|
||||
w[:ramp_len] = taper
|
||||
w[n - ramp_len :] = taper[::-1]
|
||||
return w
|
||||
|
||||
weights = np.outer(ramp(height), ramp(width))
|
||||
np.maximum(weights, _WEIGHT_EPS, out=weights)
|
||||
return weights
|
||||
|
||||
|
||||
def feather_region_composite(
|
||||
base: NDArray[Any],
|
||||
regenerated: NDArray[Any],
|
||||
box: tuple[int, int, int, int],
|
||||
*,
|
||||
feather: int = 64,
|
||||
) -> NDArray[Any]:
|
||||
"""Composite ``regenerated`` over ``base`` inside ``box`` only, feathering the seam.
|
||||
|
||||
For AI-ENHANCED composites (digitalSourceType ``compositeWithTrainedAlgorithmicMedia``):
|
||||
the diffusion remover regenerates the whole frame, but only the AI-composited
|
||||
REGION should change -- the rest is a real photo that must be preserved. This
|
||||
blends the regenerated pixels in over ``box = (x, y, w, h)`` with a separable
|
||||
linear taper of ``feather`` px at the box edges, so the result equals ``base``
|
||||
EXACTLY outside the box and ramps smoothly (no hard seam) at the boundary.
|
||||
|
||||
Pure and model-free (unit-tested): ``base`` and ``regenerated`` must be the same
|
||||
shape (H x W, or H x W x C). The output preserves ``base``'s dtype. ``feather`` is
|
||||
clamped to half the box on each axis, so a small region still tapers symmetrically;
|
||||
``feather=0`` is a hard-edged paste.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
if base.shape != regenerated.shape:
|
||||
raise ValueError(f"shape mismatch: base {base.shape} vs regenerated {regenerated.shape}")
|
||||
h, w = base.shape[:2]
|
||||
x, y, bw, bh = box
|
||||
x0, y0 = max(0, x), max(0, y)
|
||||
x1, y1 = min(w, x + bw), min(h, y + bh)
|
||||
out = base.copy()
|
||||
if x1 <= x0 or y1 <= y0:
|
||||
return out # empty / off-image box -> nothing regenerated
|
||||
|
||||
def taper(n: int) -> NDArray[Any]:
|
||||
win = np.ones(n, dtype=np.float32)
|
||||
f = min(max(feather, 0), n // 2)
|
||||
if f > 0:
|
||||
ramp = (np.arange(f, dtype=np.float32) + 1.0) / (f + 1.0) # in (0, 1), 0 at the edge
|
||||
win[:f] = ramp
|
||||
win[n - f :] = ramp[::-1]
|
||||
return win
|
||||
|
||||
rh, rw = y1 - y0, x1 - x0
|
||||
wmap = np.outer(taper(rh), taper(rw)) # ~0 at the box edge, 1 in the interior
|
||||
if base.ndim == 3:
|
||||
wmap = wmap[:, :, None]
|
||||
roi_base = base[y0:y1, x0:x1].astype(np.float32)
|
||||
roi_gen = regenerated[y0:y1, x0:x1].astype(np.float32)
|
||||
blended = roi_base * (1.0 - wmap) + roi_gen * wmap
|
||||
out[y0:y1, x0:x1] = np.clip(blended, 0, 255).astype(base.dtype)
|
||||
return out
|
||||
|
||||
|
||||
def run_tiled(
|
||||
generate_tile: Callable[[PILImage.Image], PILImage.Image],
|
||||
image: PILImage.Image,
|
||||
tile_size: int,
|
||||
overlap: int,
|
||||
set_progress: Callable[[str], None] | None = None,
|
||||
) -> PILImage.Image:
|
||||
"""Tile ``image``, run ``generate_tile`` per tile, and feather-blend the result.
|
||||
|
||||
``generate_tile`` runs one diffusion pass on a single RGB PIL tile and returns
|
||||
the regenerated tile (the ControlNet control image is built per tile inside it,
|
||||
so each tile gets its own edge map). A pass that rounds dimensions to the latent
|
||||
grid is resized back to the exact tile size before blending.
|
||||
"""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
width, height = image.size
|
||||
tiles = plan_tiles(width, height, tile_size, overlap)
|
||||
accum = np.zeros((height, width, 3), dtype=np.float32)
|
||||
weight_sum = np.zeros((height, width, 1), dtype=np.float32)
|
||||
|
||||
# All tiles share one size (plan_tiles is uniform), so the feather window is
|
||||
# loop-invariant -- compute it once.
|
||||
weights = feather_weights(tiles[0].width, tiles[0].height, overlap)[:, :, None]
|
||||
|
||||
total = len(tiles)
|
||||
for index, tile in enumerate(tiles, start=1):
|
||||
if set_progress is not None:
|
||||
set_progress(f"Tiled diffusion: tile {index}/{total} at ({tile.x},{tile.y}) {tile.width}x{tile.height}...")
|
||||
crop = image.crop((tile.x, tile.y, tile.x + tile.width, tile.y + tile.height))
|
||||
result = generate_tile(crop)
|
||||
if result.size != (tile.width, tile.height):
|
||||
result = result.resize((tile.width, tile.height), Image.Resampling.LANCZOS)
|
||||
arr = np.asarray(result.convert("RGB"), dtype=np.float32)
|
||||
accum[tile.y : tile.y + tile.height, tile.x : tile.x + tile.width] += arr * weights
|
||||
weight_sum[tile.y : tile.y + tile.height, tile.x : tile.x + tile.width] += weights
|
||||
|
||||
blended = accum / np.maximum(weight_sum, _WEIGHT_EPS)
|
||||
return Image.fromarray(np.clip(blended, 0, 255).astype(np.uint8))
|
||||
@@ -12,6 +12,20 @@ if TYPE_CHECKING:
|
||||
|
||||
DEFAULT_MODEL_ID = "stabilityai/stable-diffusion-xl-base-1.0"
|
||||
|
||||
# Qwen-Image (20B MMDiT, Apache-2.0 code AND weights) base for the ``qwen`` pipeline:
|
||||
# an img2img alternative to SDXL with native text rendering (incl. CJK). Loaded only
|
||||
# when ``--pipeline qwen`` is selected; CUDA/cloud-class (does not fit MPS). CERTIFIED
|
||||
# oracle floors (2026-06-20): OpenAI **0.10** (seed-robust -- clean on seeds 0-4) and
|
||||
# Google/Gemini **0.25** (seed 0 verified on 2 images; pin a seed in prod, the Gemini
|
||||
# oracle rate-limits volume seed-repeat). The Gemini floor (0.25) is HIGHER than the
|
||||
# certified controlnet Gemini floor (0.15); ``resolve_strength(..., pipeline="qwen")``
|
||||
# now carries this via ``_QWEN_VENDOR_STRENGTH`` (below), so ``--pipeline qwen`` gets the
|
||||
# right floor automatically -- the old manual "pass --strength 0.25 for Gemini on qwen"
|
||||
# workaround is retired.
|
||||
# (Dispatch uses the bare "qwen" literal, matching the sdxl/controlnet sites, so there
|
||||
# is no QWEN_PROFILE constant -- only the model id is referenced from code.)
|
||||
QWEN_MODEL_ID = "Qwen/Qwen-Image"
|
||||
|
||||
# Canonical pipeline-profile names + the back-compat alias. The plain SDXL img2img
|
||||
# profile is ``sdxl``; ``default`` is kept as an accepted alias (it was the profile's
|
||||
# name before ``controlnet`` became the default-selected pipeline, 2026-06-09).
|
||||
@@ -77,6 +91,18 @@ DEFAULT_STRENGTH = UNKNOWN_STRENGTH
|
||||
# Detected-vendor -> default strength. Vendor strings come from `vendor_for_strength`.
|
||||
_VENDOR_STRENGTH = {"openai": OPENAI_STRENGTH, "google": GEMINI_STRENGTH}
|
||||
|
||||
# Qwen has its OWN certified floors (Modal A100-80GB, 2026-06-20), DIFFERENT from the
|
||||
# SDXL ladder above: OpenAI 0.10 (seed-robust), Gemini 0.25 (HIGHER than controlnet's
|
||||
# 0.15 -- the 20B MMDiT perturbs less per denoising step, so it needs more strength to
|
||||
# clear Gemini SynthID). Unknown vendor tracks the higher (Gemini) value, safe-by-default.
|
||||
# `resolve_strength(..., pipeline="qwen")` uses this table so `--pipeline qwen` carries the
|
||||
# right floor automatically -- retiring the old manual "pass --strength 0.25 for Gemini on
|
||||
# qwen" workaround.
|
||||
QWEN_OPENAI_STRENGTH = 0.10
|
||||
QWEN_GEMINI_STRENGTH = 0.25
|
||||
QWEN_UNKNOWN_STRENGTH = 0.25
|
||||
_QWEN_VENDOR_STRENGTH = {"openai": QWEN_OPENAI_STRENGTH, "google": QWEN_GEMINI_STRENGTH}
|
||||
|
||||
|
||||
def strength_default_help() -> str:
|
||||
"""One-line description of the vendor-adaptive default, derived from the constants.
|
||||
@@ -90,20 +116,24 @@ def strength_default_help() -> str:
|
||||
)
|
||||
|
||||
|
||||
def resolve_strength(strength: float | None, vendor: str | None = None) -> float:
|
||||
def resolve_strength(strength: float | None, vendor: str | None = None, pipeline: str | None = None) -> float:
|
||||
"""Resolve the denoising strength, applying the vendor default when unset.
|
||||
|
||||
``None`` means "the user did not pass ``--strength``", which resolves
|
||||
**vendor-adaptively**: ``vendor`` (``"openai"`` / ``"google"`` / None, from
|
||||
``vendor_for_strength``) selects ``OPENAI_STRENGTH`` / ``GEMINI_STRENGTH`` /
|
||||
``UNKNOWN_STRENGTH``. The same ladder applies to both pipelines (see the module
|
||||
comment for why one ladder is correct). An explicit value always wins (including
|
||||
``vendor_for_strength``) selects the per-vendor floor. The ``sdxl`` and ``controlnet``
|
||||
pipelines share ONE ladder (``OPENAI_STRENGTH`` / ``GEMINI_STRENGTH`` /
|
||||
``UNKNOWN_STRENGTH`` -- see the module comment for why); ``qwen`` has its OWN higher
|
||||
ladder (``_QWEN_VENDOR_STRENGTH``, Gemini 0.25 vs controlnet 0.15), selected when
|
||||
``pipeline`` normalizes to ``"qwen"``. An explicit value always wins (including
|
||||
``0.0`` -- the check is ``is None``, not falsiness). Shared by the CLI (for display)
|
||||
and the engine (for execution) so the two never disagree -- both must pass the SAME
|
||||
``vendor``.
|
||||
``vendor`` and ``pipeline``.
|
||||
"""
|
||||
if strength is not None:
|
||||
return strength
|
||||
if pipeline is not None and normalize_profile(pipeline) == "qwen":
|
||||
return _QWEN_VENDOR_STRENGTH.get(vendor or "", QWEN_UNKNOWN_STRENGTH)
|
||||
return _VENDOR_STRENGTH.get(vendor or "", UNKNOWN_STRENGTH)
|
||||
|
||||
|
||||
|
||||
@@ -1,6 +1,16 @@
|
||||
"""Watermark removal using diffusion model regeneration attack.
|
||||
|
||||
Two pipelines:
|
||||
Three pipelines (selected by the explicit ``pipeline`` ctor arg):
|
||||
|
||||
0. ``qwen`` -- Qwen-Image (20B MMDiT, Apache-2.0) img2img. The scrub still comes from
|
||||
the img2img ``strength``; Qwen preserves text (incl. CJK) and structure markedly
|
||||
better than SDXL at the scrub floor, so it over-regenerates real photos far less.
|
||||
CUDA/cloud-class (does not fit MPS). See ``watermark_profiles`` for the certified
|
||||
oracle floors. (Near-threshold scrub is seed-non-deterministic, but the prod path
|
||||
pins ONE fixed seed, so a certified floor reproduces run-to-run -- pinning the seed
|
||||
is the release gate, not sweeping seeds.)
|
||||
|
||||
Two SDXL pipelines:
|
||||
1. ``controlnet`` (DEFAULT) -- SDXL img2img with a canny ControlNet. The watermark
|
||||
REMOVAL still comes from the img2img regeneration (``strength``); the ControlNet
|
||||
only PRESERVES structure (text/faces) by conditioning on the edge map. No original
|
||||
@@ -36,6 +46,7 @@ from remove_ai_watermarks.noai.watermark_profiles import (
|
||||
CONTROLNET_CANNY_MODEL,
|
||||
DEFAULT_MODEL_ID,
|
||||
DEFAULT_STRENGTH,
|
||||
QWEN_MODEL_ID,
|
||||
normalize_profile,
|
||||
resolve_strength,
|
||||
)
|
||||
@@ -308,6 +319,45 @@ _CANNY_HIGH = 200
|
||||
_CONTROLNET_PROMPT = "best quality, high quality, sharp, detailed, photographic"
|
||||
_CONTROLNET_NEGATIVE = "blurry, lowres, deformed, distorted text, garbled text, watermark, jpeg artifacts"
|
||||
|
||||
# Neutral prompts for the Qwen-Image img2img pass (faithful regeneration, not an edit).
|
||||
_QWEN_PROMPT = "high quality, sharp, detailed, faithful to the original"
|
||||
_QWEN_NEGATIVE = "blurry, lowres, distorted text, garbled text, artifacts"
|
||||
|
||||
|
||||
def _qwen_target_size(width: int, height: int) -> tuple[int, int]:
|
||||
"""Floor (width, height) to a multiple of 16 for Qwen's VAE/patchifier (>= 16).
|
||||
|
||||
Pure; unit-tested. Without explicit dims the img2img pipeline defaults to a 1024x1024
|
||||
SQUARE and silently distorts any non-square input.
|
||||
"""
|
||||
return max(16, (width // 16) * 16), max(16, (height // 16) * 16)
|
||||
|
||||
|
||||
def _build_qwen_kwargs(
|
||||
image: Image.Image, strength: float, num_inference_steps: int, true_cfg_scale: float, generator: Any
|
||||
) -> dict[str, Any]:
|
||||
"""Build the QwenImageImg2ImgPipeline call kwargs (pure; unit-tested without torch).
|
||||
|
||||
Qwen-Image uses ``true_cfg_scale`` (not SDXL's ``guidance_scale``) and takes an
|
||||
explicit ``negative_prompt``; the scrub still comes from the img2img ``strength``.
|
||||
Passes an explicit ``height``/``width`` derived from the input (floored to /16): the
|
||||
pipeline otherwise defaults to a 1024x1024 SQUARE, squishing any non-square input
|
||||
(the abba mixed-seam test: a 2816x1536 poster came back 1024x1024, distorting the
|
||||
scene and garbling text). So qwen regenerates at the input's own geometry.
|
||||
"""
|
||||
qw, qh = _qwen_target_size(image.width, image.height)
|
||||
return {
|
||||
"prompt": _QWEN_PROMPT,
|
||||
"negative_prompt": _QWEN_NEGATIVE,
|
||||
"image": image,
|
||||
"strength": strength,
|
||||
"num_inference_steps": num_inference_steps,
|
||||
"true_cfg_scale": true_cfg_scale,
|
||||
"generator": generator,
|
||||
"height": qh,
|
||||
"width": qw,
|
||||
}
|
||||
|
||||
|
||||
class WatermarkRemover:
|
||||
"""Remove watermarks from images using diffusion model regeneration.
|
||||
@@ -348,6 +398,11 @@ class WatermarkRemover:
|
||||
if torch_dtype is None:
|
||||
if self.device == "cpu" or self.device == "mps":
|
||||
self.torch_dtype = torch.float32 # type: ignore
|
||||
elif self.model_profile == "qwen":
|
||||
# Qwen-Image is published in bf16; fp16 risks overflow on the 20B MMDiT.
|
||||
# cuda/xpu-only by construction: the cpu/mps guard above already forced
|
||||
# fp32, and the 20B model does not fit MPS anyway.
|
||||
self.torch_dtype = torch.bfloat16 # type: ignore
|
||||
else:
|
||||
self.torch_dtype = torch.float16 # type: ignore
|
||||
else:
|
||||
@@ -355,6 +410,7 @@ class WatermarkRemover:
|
||||
|
||||
self._pipeline: AutoImg2ImgPipeline | None = None
|
||||
self._controlnet_pipeline: Any = None
|
||||
self._qwen_pipeline: Any = None
|
||||
self._progress_callback = progress_callback
|
||||
self.hf_token: str | None = hf_token or os.environ.get("HF_TOKEN")
|
||||
|
||||
@@ -369,7 +425,9 @@ class WatermarkRemover:
|
||||
|
||||
def preload(self) -> None:
|
||||
"""Eagerly load the pipeline so download progress bars are visible."""
|
||||
if self.model_profile == "controlnet":
|
||||
if self.model_profile == "qwen":
|
||||
self._load_qwen_pipeline()
|
||||
elif self.model_profile == "controlnet":
|
||||
self._load_controlnet_pipeline()
|
||||
else:
|
||||
self._load_pipeline()
|
||||
@@ -420,19 +478,27 @@ class WatermarkRemover:
|
||||
|
||||
return pipeline
|
||||
|
||||
def _base_load_kwargs(self) -> dict[str, Any]:
|
||||
"""The ``from_pretrained`` kwargs shared by all three loaders (dtype + token).
|
||||
|
||||
Each loader adds its own extras (SDXL safety_checker + fp16 VAE, the ControlNet
|
||||
model, etc.). Centralizing the dtype/token pair avoids the drift trap of three
|
||||
copies (a token forgotten on one loader silently breaks gated downloads there).
|
||||
"""
|
||||
load_kwargs: dict[str, Any] = {"torch_dtype": self.torch_dtype}
|
||||
if self.hf_token:
|
||||
load_kwargs["token"] = self.hf_token
|
||||
return load_kwargs
|
||||
|
||||
def _load_pipeline(self) -> AutoImg2ImgPipeline:
|
||||
"""Load the plain SDXL img2img pipeline lazily."""
|
||||
if self._pipeline is None:
|
||||
logger.info("Loading model %s on %s...", self.model_id, self.device)
|
||||
self._set_progress(f"Loading model weights: {self.model_id}")
|
||||
|
||||
load_kwargs: dict[str, Any] = {
|
||||
"torch_dtype": self.torch_dtype,
|
||||
"safety_checker": None,
|
||||
"requires_safety_checker": False,
|
||||
}
|
||||
if self.hf_token:
|
||||
load_kwargs["token"] = self.hf_token
|
||||
load_kwargs = self._base_load_kwargs()
|
||||
load_kwargs["safety_checker"] = None
|
||||
load_kwargs["requires_safety_checker"] = False
|
||||
self._maybe_add_fp16_vae(load_kwargs)
|
||||
|
||||
pipeline = AutoImg2ImgPipeline.from_pretrained(self.model_id, **load_kwargs) # type: ignore
|
||||
@@ -458,9 +524,8 @@ class WatermarkRemover:
|
||||
self._set_progress(f"Loading ControlNet: {CONTROLNET_CANNY_MODEL}")
|
||||
controlnet = ControlNetModel.from_pretrained(CONTROLNET_CANNY_MODEL, torch_dtype=self.torch_dtype)
|
||||
|
||||
load_kwargs: dict[str, Any] = {"controlnet": controlnet, "torch_dtype": self.torch_dtype}
|
||||
if self.hf_token:
|
||||
load_kwargs["token"] = self.hf_token
|
||||
load_kwargs = self._base_load_kwargs()
|
||||
load_kwargs["controlnet"] = controlnet
|
||||
self._maybe_add_fp16_vae(load_kwargs)
|
||||
|
||||
self._set_progress(f"Loading model weights: {self.model_id}")
|
||||
@@ -474,6 +539,37 @@ class WatermarkRemover:
|
||||
|
||||
return self._controlnet_pipeline
|
||||
|
||||
def _load_qwen_pipeline(self) -> Any:
|
||||
"""Load the Qwen-Image img2img pipeline lazily.
|
||||
|
||||
Qwen-Image is its OWN base model (not an SDXL add-on), so it loads
|
||||
``QWEN_MODEL_ID`` unless the caller passed a custom ``--model``. Needs a
|
||||
diffusers build that ships ``QwenImageImg2ImgPipeline``; raises a clear error
|
||||
otherwise. CUDA/cloud-class (the 20B MMDiT does not fit MPS).
|
||||
"""
|
||||
if self._qwen_pipeline is None:
|
||||
try:
|
||||
from diffusers import QwenImageImg2ImgPipeline
|
||||
except ImportError as exc:
|
||||
raise ImportError(
|
||||
"The 'qwen' pipeline needs a diffusers version that ships "
|
||||
"QwenImageImg2ImgPipeline. Upgrade: pip install -U diffusers"
|
||||
) from exc
|
||||
|
||||
# Use the Qwen base unless the user explicitly overrode --model.
|
||||
model = self.model_id if self.model_id != self.DEFAULT_MODEL_ID else QWEN_MODEL_ID
|
||||
logger.info("Loading Qwen-Image (%s) on %s...", model, self.device)
|
||||
self._set_progress(f"Loading model weights: {model}")
|
||||
pipeline = QwenImageImg2ImgPipeline.from_pretrained(model, **self._base_load_kwargs())
|
||||
pipeline = self._move_to_device_and_optimize(pipeline)
|
||||
with contextlib.suppress(Exception):
|
||||
pipeline.set_progress_bar_config(disable=True)
|
||||
|
||||
logger.info("Qwen-Image model loaded successfully")
|
||||
self._qwen_pipeline = pipeline
|
||||
|
||||
return self._qwen_pipeline
|
||||
|
||||
# ── Core removal ─────────────────────────────────────────────────
|
||||
|
||||
def remove_watermark(
|
||||
@@ -485,6 +581,11 @@ class WatermarkRemover:
|
||||
guidance_scale: float | None = None,
|
||||
seed: int | None = None,
|
||||
vendor: str | None = None,
|
||||
tile: bool = False,
|
||||
tile_size: int = 1024,
|
||||
tile_overlap: int = 128,
|
||||
region: tuple[int, int, int, int] | None = None,
|
||||
region_feather: int = 64,
|
||||
) -> Path:
|
||||
"""Remove watermark from an image using regeneration attack.
|
||||
|
||||
@@ -501,6 +602,22 @@ class WatermarkRemover:
|
||||
input with ``watermark_profiles.vendor_for_strength`` before processing
|
||||
strips the metadata; the caller passes it down so display and execution
|
||||
agree.
|
||||
tile: Process the image in overlapping tiles instead of one forward pass.
|
||||
The lossless alternative to a ``--max-resolution`` downscale for large
|
||||
inputs that OOM on MPS/GPU (issue #10). Only engages when the long side
|
||||
exceeds ``tile_size``; smaller images run a single pass unchanged.
|
||||
tile_size: Tile dimension in px (default 1024, SDXL's training size).
|
||||
tile_overlap: Overlap between adjacent tiles in px (default 128), feather-
|
||||
blended so there is no visible seam.
|
||||
region: Restrict the regeneration to the AI-composited box ``(x, y, w, h)``
|
||||
and feather-composite it back over the ORIGINAL pixels everywhere else.
|
||||
For AI-ENHANCED composites (digitalSourceType
|
||||
``compositeWithTrainedAlgorithmicMedia``, surfaced as
|
||||
``identify.ProvenanceReport.ai_source_kind == "enhanced"``): the real
|
||||
photo outside the box is preserved exactly, only the AI region is
|
||||
scrubbed. The box is supplied by the caller (a C2PA composite manifest
|
||||
does not carry a reliable machine-readable region). None -> whole frame.
|
||||
region_feather: Seam taper in px for ``region`` compositing (default 64).
|
||||
|
||||
Returns:
|
||||
Path to the cleaned image.
|
||||
@@ -515,7 +632,7 @@ class WatermarkRemover:
|
||||
if output_path is None:
|
||||
output_path = image_path
|
||||
|
||||
strength = resolve_strength(strength, vendor)
|
||||
strength = resolve_strength(strength, vendor, self.model_profile)
|
||||
|
||||
if not 0.0 <= strength <= 1.0:
|
||||
raise ValueError(f"Strength must be between 0.0 and 1.0, got {strength}")
|
||||
@@ -541,10 +658,21 @@ class WatermarkRemover:
|
||||
|
||||
_total_start = time.monotonic()
|
||||
|
||||
def _generate() -> Image.Image:
|
||||
def _generate_one(img: Image.Image) -> Image.Image:
|
||||
if self.model_profile == "qwen":
|
||||
return self._run_qwen(img, strength, num_inference_steps, guidance_scale, generator)
|
||||
if self.model_profile == "controlnet":
|
||||
return self._run_controlnet(init_image, strength, num_inference_steps, guidance_scale, generator)
|
||||
return self._run_img2img(init_image, strength, num_inference_steps, guidance_scale, generator)
|
||||
return self._run_controlnet(img, strength, num_inference_steps, guidance_scale, generator)
|
||||
return self._run_img2img(img, strength, num_inference_steps, guidance_scale, generator)
|
||||
|
||||
def _generate() -> Image.Image:
|
||||
# Tile only when asked AND the image is larger than one tile; otherwise a
|
||||
# single full-image pass (tiling a sub-tile image is pure overhead).
|
||||
if tile and max(init_image.size) > tile_size:
|
||||
from remove_ai_watermarks.noai.tiling import run_tiled
|
||||
|
||||
return run_tiled(_generate_one, init_image, tile_size, tile_overlap, self._set_progress)
|
||||
return _generate_one(init_image)
|
||||
|
||||
cleaned_image = _generate()
|
||||
|
||||
@@ -561,6 +689,22 @@ class WatermarkRemover:
|
||||
self._controlnet_pipeline = None
|
||||
cleaned_image = _generate()
|
||||
|
||||
# Region-targeted regeneration for AI-enhanced composites: keep the real photo
|
||||
# outside the AI box pixel-exact, blend only the regenerated AI region back in.
|
||||
if region is not None:
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks.noai.tiling import feather_region_composite
|
||||
|
||||
gen = cleaned_image.convert("RGB")
|
||||
if gen.size != init_image.size: # a downscaled/tiled pass can resize
|
||||
gen = gen.resize(init_image.size)
|
||||
cleaned_image = gen
|
||||
base_rgb = np.asarray(init_image) # original RGB, untouched outside the box
|
||||
merged = feather_region_composite(base_rgb, np.asarray(gen), region, feather=region_feather)
|
||||
cleaned_image = Image.fromarray(merged)
|
||||
self._set_progress(f"Region-targeted regeneration: AI box {region}, real photo preserved")
|
||||
|
||||
self._set_progress(f"Regeneration complete · Output: {w}x{h}px {cleaned_image.mode}")
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
@@ -706,6 +850,30 @@ class WatermarkRemover:
|
||||
self._controlnet_pipeline = None
|
||||
return self._load_controlnet_pipeline()
|
||||
|
||||
# ── Qwen runner ──────────────────────────────────────────────────
|
||||
|
||||
def _run_qwen(
|
||||
self,
|
||||
init_image: Image.Image,
|
||||
strength: float,
|
||||
num_inference_steps: int,
|
||||
guidance_scale: float,
|
||||
generator: Any,
|
||||
) -> Image.Image:
|
||||
"""Run the Qwen-Image img2img pass.
|
||||
|
||||
Removal comes from the img2img ``strength`` (same lever as the SDXL paths);
|
||||
Qwen-Image preserves text/structure markedly better at the scrub floor. The
|
||||
CLI ``guidance_scale`` maps to Qwen's ``true_cfg_scale`` (~4.0 is typical;
|
||||
the SDXL default of 7.5 is high for Qwen). No MPS->CPU fallback: the 20B MMDiT
|
||||
is CUDA/cloud-class and does not run on MPS, so an error here propagates.
|
||||
"""
|
||||
pipeline = self._load_qwen_pipeline()
|
||||
self._set_progress(f"Running Qwen-Image img2img (strength={strength}, true_cfg={guidance_scale})...")
|
||||
kwargs = _build_qwen_kwargs(init_image, strength, num_inference_steps, guidance_scale, generator)
|
||||
result = pipeline(**kwargs)
|
||||
return result.images[0]
|
||||
|
||||
# ── Batch ────────────────────────────────────────────────────────
|
||||
|
||||
def remove_watermark_batch(
|
||||
@@ -754,12 +922,17 @@ def remove_watermark(
|
||||
model_id: str | None = None,
|
||||
device: str | None = None,
|
||||
hf_token: str | None = None,
|
||||
region: tuple[int, int, int, int] | None = None,
|
||||
) -> Path:
|
||||
"""Convenience function to remove watermark from an image.
|
||||
|
||||
``strength=None`` lets the profile pick its vendor-adaptive default
|
||||
(0.20 OpenAI / 0.30 Google / 0.30 unknown, from the C2PA SynthID proxy on the
|
||||
input; same ladder for the controlnet and sdxl pipelines). Pass a value to override.
|
||||
|
||||
``region=(x, y, w, h)`` restricts the regeneration to that box and preserves the
|
||||
real photo elsewhere -- for AI-enhanced composites (see
|
||||
``WatermarkRemover.remove_watermark``).
|
||||
"""
|
||||
from remove_ai_watermarks.noai.watermark_profiles import vendor_for_strength
|
||||
|
||||
@@ -769,4 +942,5 @@ def remove_watermark(
|
||||
output_path=output_path,
|
||||
strength=strength,
|
||||
vendor=vendor_for_strength(image_path),
|
||||
region=region,
|
||||
)
|
||||
@@ -90,13 +90,27 @@ class KnownMark:
|
||||
return self._remove(image, inpaint_method, inpaint, inpaint_strength, force)
|
||||
|
||||
|
||||
# Gemini-sparkle confidence above which the registry treats it as a confident
|
||||
# detection for arbitration. Matches identify's corpus-validated sparkle
|
||||
# threshold (0.5): the gemini engine's own detect flag uses a looser internal
|
||||
# threshold and weakly fires (~0.36) on unrelated bottom-right text (e.g. the
|
||||
# Doubao mark), which would otherwise let it hijack `--mark auto`. 0.5 gives 0
|
||||
# false positives on the corpus.
|
||||
_GEMINI_AUTO_MIN_CONF = 0.5
|
||||
# Single source of truth for the Gemini-sparkle "trust this as a real mark"
|
||||
# confidence, shared by BOTH the removal arbitration here (`best_auto_mark` /
|
||||
# `_gemini_detect`) and the provenance detector in `identify` (which imports it
|
||||
# as its sparkle threshold). Defining it once removes the detect-vs-remove
|
||||
# threshold drift the retained-corpus mining surfaced (2026-06-20): identify
|
||||
# would report a sparkle while removal declined it, or vice versa, whenever the
|
||||
# two independently-maintained 0.5 constants fell out of step. Now they cannot.
|
||||
#
|
||||
# Value 0.5 is corpus-validated: the gemini engine's own `detected` flag uses a
|
||||
# looser internal threshold (0.35) and weakly fires (~0.36-0.42) on unrelated
|
||||
# bottom-right text -- a real Doubao mark scores ~0.40-0.42 as a gemini match,
|
||||
# and its core-ring brightness margin is HIGHER than a genuine faint sparkle's,
|
||||
# so neither confidence nor the brightness gate separates them in the [0.35, 0.5)
|
||||
# band. Lowering this gate to recover faint sparkles was evaluated against that
|
||||
# band (2026-06-20) and REJECTED: it cannot be done without re-admitting the
|
||||
# Doubao-text / content false positives, trading a rare miss for false-positive
|
||||
# removals on clean images. The band below the gate is therefore intentionally
|
||||
# left to the higher-strength / metadata paths. 0.5 gives 0 false positives on
|
||||
# the corpus.
|
||||
GEMINI_SPARKLE_TRUST_CONF = 0.5
|
||||
_GEMINI_AUTO_MIN_CONF = GEMINI_SPARKLE_TRUST_CONF
|
||||
|
||||
# ── Engine adapters (lazy singletons; engines are cv2-only, no model load) ──
|
||||
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Regression test for the one-to-one face matcher in ``scripts/fidelity_metrics.py``.
|
||||
|
||||
The shipped per-face nearest matcher collided on multi-face images (two original faces
|
||||
both picking the same variant face when regeneration dropped a face), which inflated/
|
||||
corrupted the identity metric. ``assign_faces_one_to_one`` is the collision-free
|
||||
replacement. The function is pure (centers + diagonals in, index map out), so it is
|
||||
tested here without insightface / the heavy PEP723 env. Caught on the gemini_3 Qwen
|
||||
ControlNet experiment, where the original had 18 faces but the regenerated variants had
|
||||
17, producing two collisions under the old matcher.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent / "scripts"
|
||||
|
||||
|
||||
def _load_assign():
|
||||
# fidelity_metrics is a standalone PEP723 script, not an installed module; load it by
|
||||
# path with scripts/ on sys.path so its `_plain_console` shim import resolves.
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
try:
|
||||
spec = importlib.util.spec_from_file_location("fidelity_metrics", _SCRIPTS / "fidelity_metrics.py")
|
||||
assert spec is not None
|
||||
assert spec.loader is not None
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = mod # @dataclass introspection needs the module registered
|
||||
spec.loader.exec_module(mod)
|
||||
except ImportError as exc: # cv2/click absent in a bare env -> skip, not fail
|
||||
pytest.skip(f"fidelity_metrics import deps missing: {exc}")
|
||||
finally:
|
||||
sys.path.remove(str(_SCRIPTS))
|
||||
return mod.assign_faces_one_to_one
|
||||
|
||||
|
||||
def test_distinct_faces_match_nearest() -> None:
|
||||
assign = _load_assign()
|
||||
ref = [(0.0, 0.0), (100.0, 100.0)]
|
||||
var = [(2.0, 1.0), (98.0, 102.0)]
|
||||
diags = [50.0, 50.0]
|
||||
assert assign(ref, var, diags) == {0: 0, 1: 1}
|
||||
|
||||
|
||||
def test_no_collision_when_variant_drops_a_face() -> None:
|
||||
# Two original faces near the SAME single variant face: the old nearest matcher mapped
|
||||
# BOTH to index 0; one-to-one must give the nearer ref the match and drop the other.
|
||||
assign = _load_assign()
|
||||
ref = [(10.0, 10.0), (14.0, 10.0)] # both close to the lone variant
|
||||
var = [(12.0, 10.0)]
|
||||
diags = [50.0, 50.0]
|
||||
matched = assign(ref, var, diags)
|
||||
assert sorted(matched.values()) == [0] # variant 0 used at most once
|
||||
assert len(matched) == 1
|
||||
|
||||
|
||||
def test_gate_drops_implausibly_far_match() -> None:
|
||||
assign = _load_assign()
|
||||
ref = [(0.0, 0.0)]
|
||||
var = [(1000.0, 1000.0)] # far beyond 0.6 * diag
|
||||
diags = [50.0]
|
||||
assert assign(ref, var, diags) == {}
|
||||
|
||||
|
||||
def test_assignment_is_one_to_one_over_many_faces() -> None:
|
||||
assign = _load_assign()
|
||||
ref = [(float(i * 100), 0.0) for i in range(18)]
|
||||
var = [(float(i * 100) + 3.0, 0.0) for i in range(17)] # one fewer, as in the experiment
|
||||
diags = [50.0] * 18
|
||||
matched = assign(ref, var, diags)
|
||||
assert len(matched) == 17
|
||||
assert len(set(matched.values())) == 17 # every variant used at most once
|
||||
@@ -24,6 +24,7 @@ from remove_ai_watermarks.identify import (
|
||||
_vendor_of,
|
||||
identify,
|
||||
)
|
||||
from remove_ai_watermarks.watermark_registry import GEMINI_SPARKLE_TRUST_CONF
|
||||
|
||||
# Where the lazy import inside identify._visible_sparkle resolves the detector.
|
||||
_SPARKLE_TARGET = "remove_ai_watermarks.gemini_engine.detect_sparkle_confidence"
|
||||
@@ -66,6 +67,19 @@ class TestAttributePlatform:
|
||||
assert platform
|
||||
assert "Stability AI" in platform
|
||||
|
||||
def test_canva(self):
|
||||
platform = _attribute_platform(["Canva"])
|
||||
assert platform
|
||||
assert "Canva" in platform
|
||||
|
||||
def test_byteplus_attributes_to_bytedance(self):
|
||||
# ByteDance's intl brand signs as "Byteplus Pte. Ltd."; the registry maps
|
||||
# it to the ByteDance platform (was mis-read as Adobe via an incidental
|
||||
# "Adobe XMP" file string before the entry existed).
|
||||
platform = _attribute_platform(["BytePlus (ByteDance)"])
|
||||
assert platform
|
||||
assert "ByteDance" in platform
|
||||
|
||||
def test_empty_is_none(self):
|
||||
assert _attribute_platform([]) is None
|
||||
|
||||
@@ -127,6 +141,23 @@ class TestIdentifyNonPng:
|
||||
assert r.is_ai_generated is True
|
||||
assert "ByteDance" in (r.platform or "")
|
||||
|
||||
def test_bytedance_chinese_legal_name_attributed(self, tmp_path: Path):
|
||||
# Some Volcano Engine certs name the signer with the Chinese legal entity
|
||||
# rather than the latin "volcengine"; the latin needle misses it, so the
|
||||
# Chinese-name registry entry is what attributes real ByteDance output.
|
||||
blob = "北京火山引擎科技有限公司".encode() + b" ... trainedAlgorithmicMedia"
|
||||
path = self._c2pa_jpeg(tmp_path, blob)
|
||||
r = identify(path, check_visible=False, check_invisible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert "ByteDance" in (r.platform or "")
|
||||
|
||||
def test_elevenlabs_attributed(self, tmp_path: Path):
|
||||
path = self._c2pa_jpeg(tmp_path, b"Eleven Labs Inc. ... trainedAlgorithmicMedia")
|
||||
r = identify(path, check_visible=False, check_invisible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform == "ElevenLabs"
|
||||
assert not any("SynthID" in w for w in r.watermarks) # ElevenLabs does not use SynthID
|
||||
|
||||
def test_stability_ai_issuer_attributed_no_synthid(self, tmp_path: Path):
|
||||
path = self._c2pa_jpeg(tmp_path, b"Stability AI ... trainedAlgorithmicMedia")
|
||||
r = identify(path, check_visible=False)
|
||||
@@ -135,6 +166,21 @@ class TestIdentifyNonPng:
|
||||
assert "Stability AI" in r.platform
|
||||
assert not any("SynthID" in w for w in r.watermarks) # Stability does not use SynthID
|
||||
|
||||
def test_trained_source_is_generated_kind(self, tmp_path: Path):
|
||||
path = self._c2pa_jpeg(tmp_path, b"OpenAI ... trainedAlgorithmicMedia")
|
||||
r = identify(path, check_visible=False, check_invisible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.ai_source_kind == "generated"
|
||||
|
||||
def test_composite_source_is_enhanced_kind(self, tmp_path: Path):
|
||||
# compositeWithTrainedAlgorithmicMedia: a real photo with an AI-composited
|
||||
# region. Still AI (is_ai True), but the kind must read "enhanced" so a
|
||||
# caller can do region-targeted cleaning instead of a full-frame regen.
|
||||
path = self._c2pa_jpeg(tmp_path, b"Adobe ... compositeWithTrainedAlgorithmicMedia")
|
||||
r = identify(path, check_visible=False, check_invisible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.ai_source_kind == "enhanced"
|
||||
|
||||
def test_c2pa_without_ai_marker_is_unknown(self, tmp_path: Path):
|
||||
# Adobe signs C2PA on plain Photoshop edits too. Without an AI digital-
|
||||
# source marker, the honest verdict is unknown -- the C2PA watermark is
|
||||
@@ -189,6 +235,16 @@ class TestIdentifySamsungGalaxy:
|
||||
assert r.platform == "ASUS Gallery (C2PA signer)"
|
||||
assert any("C2PA" in w for w in r.watermarks)
|
||||
|
||||
def test_galaxy_capture_without_ai_marker_is_not_ai(self, tmp_path: Path):
|
||||
# A genuine Galaxy phone capture carries Samsung Galaxy C2PA provenance but
|
||||
# NO AI source-type / genAIType. It must stay is_ai=None -- the device cert
|
||||
# is authenticity provenance of a real photo, not an AI-generation signal.
|
||||
path = self._jpeg(tmp_path, "s25_capture.jpg", b"Samsung Galaxy Galaxy S25 c2pa-rs no ai marker")
|
||||
r = identify(path, check_visible=False, check_invisible=False)
|
||||
assert r.is_ai_generated is None
|
||||
assert r.platform == "Samsung Galaxy (C2PA)"
|
||||
assert any("C2PA" in w for w in r.watermarks)
|
||||
|
||||
|
||||
# ── End-to-end verdicts on real fixtures ────────────────────────────
|
||||
|
||||
@@ -216,6 +272,19 @@ class TestIdentifyRealSamples:
|
||||
assert r.is_ai_generated is True
|
||||
assert any("IPTC" in w for w in r.watermarks)
|
||||
|
||||
def test_flux_bfl_c2pa_png(self):
|
||||
# flux-1.png: real Black Forest Labs FLUX.2 Playground output (signed C2PA).
|
||||
r = identify(SAMPLES_DIR / "flux-1.png", check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform == "Black Forest Labs (FLUX)"
|
||||
|
||||
def test_flux_bfl_c2pa_jpeg_via_reader(self):
|
||||
# flux-1.jpg: same source as a JPEG -- the real committed JPEG-with-C2PA
|
||||
# fixture that exercises the c2pa-python non-PNG reader path end to end.
|
||||
r = identify(SAMPLES_DIR / "flux-1.jpg", check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.platform == "Black Forest Labs (FLUX)"
|
||||
|
||||
def test_clean_photo_is_unknown_not_clean(self, clean_photo: Path):
|
||||
r = identify(clean_photo, check_visible=False)
|
||||
assert r.is_ai_generated is None # never asserted False
|
||||
@@ -251,6 +320,12 @@ class TestIdentifyLocalParams:
|
||||
assert "parameters" in signal.detail
|
||||
assert signal.confidence == "high"
|
||||
|
||||
def test_local_gen_params_have_no_c2pa_source_kind(self, tmp_png_with_ai_metadata: Path):
|
||||
# AI verdict from local SD params (not C2PA) -> ai_source_kind stays None.
|
||||
r = identify(tmp_png_with_ai_metadata, check_visible=False)
|
||||
assert r.is_ai_generated is True
|
||||
assert r.ai_source_kind is None
|
||||
|
||||
def test_clean_png_is_unknown(self, tmp_clean_png: Path):
|
||||
r = identify(tmp_clean_png, check_visible=False)
|
||||
assert r.is_ai_generated is None
|
||||
@@ -373,6 +448,66 @@ class TestIdentifyVisibleSparkle:
|
||||
assert r.confidence == "high"
|
||||
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
_DEMO_BEFORE = REPO_ROOT / "demo_banana_before.png"
|
||||
_DEMO_AFTER = REPO_ROOT / "demo_banana_after.png"
|
||||
|
||||
|
||||
@pytest.mark.skipif(not (_DEMO_BEFORE.exists() and _DEMO_AFTER.exists()), reason="demo banana pair not present")
|
||||
class TestSparkleDetectRemoveAlignment:
|
||||
"""Detect (identify) and remove (registry.best_auto_mark) must agree on the
|
||||
same image -- the retained-corpus desync where identify reported a sparkle the
|
||||
removal arbitration declined (or vice versa). Both gate on the single shared
|
||||
GEMINI_SPARKLE_TRUST_CONF, so a sparkle just over the line is taken by BOTH
|
||||
and one just under is declined by BOTH. Fixtures composite the real captured
|
||||
sparkle (before-minus-after) back at reduced opacity to land on either side.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _faint_sparkle(tmp_path: Path, opacity: float) -> Path:
|
||||
import numpy as np
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
|
||||
before = image_io.imread(_DEMO_BEFORE).astype("float32")
|
||||
after = image_io.imread(_DEMO_AFTER).astype("float32")
|
||||
faint = np.clip(after + opacity * (before - after), 0, 255).astype("uint8")
|
||||
out = tmp_path / f"sparkle_{int(opacity * 100)}.png"
|
||||
image_io.imwrite(out, faint)
|
||||
return out
|
||||
|
||||
def _detect_remove(self, path: Path) -> tuple[bool, bool, float]:
|
||||
from remove_ai_watermarks import image_io, watermark_registry
|
||||
from remove_ai_watermarks.gemini_engine import detect_sparkle_confidence
|
||||
|
||||
conf = detect_sparkle_confidence(path) or 0.0
|
||||
identify_fires = conf >= GEMINI_SPARKLE_TRUST_CONF
|
||||
best = watermark_registry.best_auto_mark(image_io.imread(path))
|
||||
remove_takes_gemini = best is not None and best.key == "gemini"
|
||||
return identify_fires, remove_takes_gemini, conf
|
||||
|
||||
def test_above_threshold_both_fire(self, tmp_path: Path):
|
||||
path = self._faint_sparkle(tmp_path, 0.7) # ~0.55 conf, just over the line
|
||||
identify_fires, remove_takes, conf = self._detect_remove(path)
|
||||
assert conf >= GEMINI_SPARKLE_TRUST_CONF
|
||||
assert identify_fires, f"identify declined a sparkle above threshold (conf={conf:.3f})"
|
||||
assert remove_takes, f"removal declined a sparkle above threshold (conf={conf:.3f})"
|
||||
|
||||
def test_below_threshold_both_decline(self, tmp_path: Path):
|
||||
path = self._faint_sparkle(tmp_path, 0.5) # ~0.37 conf, just under the line
|
||||
identify_fires, remove_takes, conf = self._detect_remove(path)
|
||||
assert conf < GEMINI_SPARKLE_TRUST_CONF
|
||||
assert not identify_fires, f"identify fired below threshold (conf={conf:.3f})"
|
||||
assert not remove_takes, f"removal fired below threshold (conf={conf:.3f})"
|
||||
|
||||
def test_full_strength_both_fire(self):
|
||||
# The shipped demo sparkle at full strength: unambiguous agreement.
|
||||
identify_fires, remove_takes, conf = self._detect_remove(_DEMO_BEFORE)
|
||||
assert conf >= GEMINI_SPARKLE_TRUST_CONF
|
||||
assert identify_fires
|
||||
assert remove_takes
|
||||
|
||||
|
||||
class TestIdentifyImportIsLight:
|
||||
"""`import identify` must stay torch-free (lazy noai/__init__): the package
|
||||
is deployed on a 512 MB host where eagerly pulling torch/diffusers OOMs."""
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
@@ -125,6 +126,24 @@ class TestHasAiMetadata:
|
||||
|
||||
assert not has_ai_metadata(out)
|
||||
|
||||
def test_remove_ai_metadata_blanks_exif_token_item_in_avif(self, tmp_path: Path):
|
||||
"""End-to-end: ``remove_ai_metadata`` blanks an AI-generator EXIF token
|
||||
stored as a meta-box Exif item (bytes in mdat) without re-encoding."""
|
||||
from remove_ai_watermarks.metadata import remove_ai_metadata
|
||||
|
||||
ftyp = b"\x00\x00\x00\x18ftypavif\x00\x00\x00\x00avifmif1"
|
||||
blob = piexif.dump({"0th": {piexif.ImageIFD.Software: b"Midjourney", piexif.ImageIFD.Make: b"NIKON"}})
|
||||
mdat = struct.pack(">I", 8 + len(blob)) + b"mdat" + blob
|
||||
src = tmp_path / "in.avif"
|
||||
src.write_bytes(ftyp + mdat)
|
||||
|
||||
out = tmp_path / "out.avif"
|
||||
remove_ai_metadata(src, out)
|
||||
cleaned = out.read_bytes()
|
||||
assert len(cleaned) == len(ftyp + mdat) # in place, no re-encode
|
||||
assert b"Midjourney" not in cleaned # AI token gone
|
||||
assert b"NIKON" in cleaned # camera tag preserved
|
||||
|
||||
def test_detects_iptc_trained_algorithmic_media_marker(self, tmp_path: Path):
|
||||
"""Some pipelines embed only the IPTC AI marker in XMP, no C2PA manifest."""
|
||||
path = tmp_path / "fake.jpg"
|
||||
|
||||
@@ -36,6 +36,7 @@ from remove_ai_watermarks.noai.extractor import (
|
||||
has_ai_metadata,
|
||||
)
|
||||
from remove_ai_watermarks.noai.isobmff import (
|
||||
blank_ai_exif_tokens,
|
||||
is_isobmff,
|
||||
strip_c2pa_boxes,
|
||||
)
|
||||
@@ -182,6 +183,14 @@ class TestC2PARealSamples:
|
||||
assert has_c2pa_metadata(out)
|
||||
assert "OpenAI" in extract_c2pa_info(out)["issuer"]
|
||||
|
||||
def test_extract_info_flux_jpeg_via_reader(self):
|
||||
"""Real committed JPEG-with-C2PA fixture: the non-PNG reader path works."""
|
||||
info = extract_c2pa_info(SAMPLES_DIR / "flux-1.jpg")
|
||||
assert info["has_c2pa"] is True
|
||||
assert info["c2pa_manifest"].startswith("C2PA manifest store") # reader, not chunk
|
||||
assert "Black Forest Labs" in info["issuer"]
|
||||
assert "trainedAlgorithmicMedia" in info["source_type"]
|
||||
|
||||
def test_extract_info_uses_reader_store(self):
|
||||
"""The c2pa-python reader path: structured (not heuristic) extraction."""
|
||||
from remove_ai_watermarks.noai import c2pa
|
||||
@@ -358,6 +367,44 @@ class TestISOBMFF:
|
||||
assert stripped == 0
|
||||
assert cleaned == data
|
||||
|
||||
@staticmethod
|
||||
def _avif_with_exif(exif_0th: dict) -> bytes:
|
||||
"""A fake AVIF (ftyp + mdat) whose mdat carries an EXIF TIFF block, as a
|
||||
HEIF/AVIF ``Exif`` meta-box item stores it (bytes in mdat)."""
|
||||
import piexif
|
||||
|
||||
blob = piexif.dump({"0th": exif_0th})
|
||||
mdat = struct.pack(">I", 8 + len(blob)) + b"mdat" + blob
|
||||
return FTYP + mdat
|
||||
|
||||
def test_blank_ai_token_in_exif_item(self):
|
||||
import piexif
|
||||
|
||||
data = self._avif_with_exif({piexif.ImageIFD.Software: b"DALL-E", piexif.ImageIFD.Make: b"Canon"})
|
||||
out, blanked = blank_ai_exif_tokens(data)
|
||||
assert blanked == 1
|
||||
assert len(out) == len(data) # same length -> box sizes / iloc stay valid
|
||||
assert b"DALL-E" not in out # AI token destroyed
|
||||
assert b"Canon" in out # camera tag preserved
|
||||
# The TIFF structure still parses, with the AI value blanked and Make kept.
|
||||
blob = out[out.index(b"Exif\x00\x00") + 6 :]
|
||||
ifd = piexif.load(blob)["0th"]
|
||||
assert ifd[piexif.ImageIFD.Software].strip() == b""
|
||||
assert ifd[piexif.ImageIFD.Make] == b"Canon"
|
||||
|
||||
def test_blank_leaves_clean_exif_untouched(self):
|
||||
import piexif
|
||||
|
||||
data = self._avif_with_exif({piexif.ImageIFD.Software: b"Adobe Photoshop", piexif.ImageIFD.Make: b"NIKON"})
|
||||
out, blanked = blank_ai_exif_tokens(data)
|
||||
assert blanked == 0
|
||||
assert out == data # no AI token -> byte-for-byte unchanged
|
||||
|
||||
def test_blank_no_exif_is_noop(self):
|
||||
out, blanked = blank_ai_exif_tokens(FTYP + b"\x00\x00\x00\x0cmdat" + b"pixels!!")
|
||||
assert blanked == 0
|
||||
assert out == FTYP + b"\x00\x00\x00\x0cmdat" + b"pixels!!"
|
||||
|
||||
|
||||
class TestC2PAInvalidSignature:
|
||||
"""A .png file that is not actually PNG-signed must read as clean, not crash."""
|
||||
|
||||
+78
-3
@@ -115,6 +115,7 @@ class TestModelProfiles:
|
||||
def test_canonical_profiles_unchanged(self):
|
||||
assert normalize_profile("sdxl") == "sdxl"
|
||||
assert normalize_profile("controlnet") == "controlnet"
|
||||
assert normalize_profile("qwen") == "qwen"
|
||||
|
||||
def test_default_alias_resolves_to_sdxl(self):
|
||||
# "default" is the legacy alias for "sdxl" (back-compat for existing scripts).
|
||||
@@ -125,19 +126,93 @@ class TestModelProfiles:
|
||||
assert normalize_profile("CONTROLNET") == "controlnet"
|
||||
|
||||
|
||||
class _StubImage:
|
||||
"""Minimal PIL.Image stand-in: just the ``width``/``height`` the pure helper reads."""
|
||||
|
||||
def __init__(self, width: int, height: int) -> None:
|
||||
self.width = width
|
||||
self.height = height
|
||||
|
||||
|
||||
class TestQwenKwargs:
|
||||
"""_build_qwen_kwargs is pure (no torch); guards the Qwen-Image call shape.
|
||||
|
||||
watermark_remover imports torch under a try/except, so the module (and this pure
|
||||
helper) imports fine in the core+dev CI env where torch is absent.
|
||||
"""
|
||||
|
||||
def test_uses_true_cfg_not_guidance_scale(self):
|
||||
from remove_ai_watermarks.noai.watermark_remover import _build_qwen_kwargs
|
||||
|
||||
gen = object()
|
||||
img = _StubImage(2816, 1536)
|
||||
kwargs = _build_qwen_kwargs(img, strength=0.3, num_inference_steps=40, true_cfg_scale=4.0, generator=gen)
|
||||
# Qwen uses true_cfg_scale, NOT SDXL's guidance_scale.
|
||||
assert kwargs["true_cfg_scale"] == 4.0
|
||||
assert "guidance_scale" not in kwargs
|
||||
# The scrub still comes from strength; image + generator pass through.
|
||||
assert kwargs["strength"] == 0.3
|
||||
assert kwargs["image"] is img
|
||||
assert kwargs["generator"] is gen
|
||||
# Faithful-regeneration prompt + an explicit negative prompt.
|
||||
assert kwargs["prompt"]
|
||||
assert kwargs["negative_prompt"]
|
||||
|
||||
def test_passes_explicit_aspect_preserving_size(self):
|
||||
# Without height/width the pipeline defaults to 1024x1024 and squishes non-square
|
||||
# input (the abba mixed-seam regression). Both already multiples of 16 -> unchanged.
|
||||
from remove_ai_watermarks.noai.watermark_remover import _build_qwen_kwargs
|
||||
|
||||
kwargs = _build_qwen_kwargs(
|
||||
_StubImage(2816, 1536), strength=0.25, num_inference_steps=40, true_cfg_scale=4.0, generator=None
|
||||
)
|
||||
assert kwargs["width"] == 2816
|
||||
assert kwargs["height"] == 1536
|
||||
|
||||
def test_qwen_target_size_floors_to_multiple_of_16(self):
|
||||
from remove_ai_watermarks.noai.watermark_remover import _qwen_target_size
|
||||
|
||||
assert _qwen_target_size(2816, 1536) == (2816, 1536) # already /16
|
||||
assert _qwen_target_size(1122, 1402) == (1120, 1392) # floored
|
||||
assert _qwen_target_size(10, 10) == (16, 16) # min clamp, never 0
|
||||
|
||||
def test_qwen_model_id_is_qwen_image(self):
|
||||
from remove_ai_watermarks.noai.watermark_profiles import QWEN_MODEL_ID
|
||||
|
||||
assert QWEN_MODEL_ID == "Qwen/Qwen-Image"
|
||||
|
||||
|
||||
class TestResolveStrength:
|
||||
"""resolve_strength applies the vendor default only when strength is unset."""
|
||||
|
||||
def test_none_is_vendor_adaptive(self):
|
||||
# No vendor -> unknown default; OpenAI lower, Google == unknown. The SAME ladder
|
||||
# applies to both pipelines (the certified controlnet floors), so there is no
|
||||
# pipeline argument.
|
||||
# No vendor -> unknown default; OpenAI lower, Google == unknown. The sdxl/controlnet
|
||||
# pipelines share this ladder (the certified controlnet floors); qwen has its own
|
||||
# (see test_qwen_pipeline_uses_its_own_higher_ladder).
|
||||
assert resolve_strength(None) == UNKNOWN_STRENGTH
|
||||
assert resolve_strength(None, "openai") == OPENAI_STRENGTH
|
||||
assert resolve_strength(None, "google") == GEMINI_STRENGTH
|
||||
assert resolve_strength(None, None) == UNKNOWN_STRENGTH
|
||||
# An unrecognized vendor string falls through to the unknown default.
|
||||
assert resolve_strength(None, "adobe") == UNKNOWN_STRENGTH
|
||||
# sdxl/controlnet pipelines (and the "default" alias) use the same shared ladder.
|
||||
assert resolve_strength(None, "google", "controlnet") == GEMINI_STRENGTH
|
||||
assert resolve_strength(None, "google", "sdxl") == GEMINI_STRENGTH
|
||||
|
||||
def test_qwen_pipeline_uses_its_own_higher_ladder(self):
|
||||
# Qwen's certified Gemini floor (0.25) is HIGHER than controlnet's (0.15); OpenAI
|
||||
# matches (0.10). Unknown vendor on qwen tracks the higher Gemini value. This retires
|
||||
# the old manual "pass --strength 0.25 for Gemini on qwen" workaround.
|
||||
from remove_ai_watermarks.noai.watermark_profiles import QWEN_GEMINI_STRENGTH, QWEN_OPENAI_STRENGTH
|
||||
|
||||
assert QWEN_GEMINI_STRENGTH == 0.25
|
||||
assert QWEN_OPENAI_STRENGTH == 0.10
|
||||
assert resolve_strength(None, "google", "qwen") == QWEN_GEMINI_STRENGTH
|
||||
assert resolve_strength(None, "openai", "qwen") == QWEN_OPENAI_STRENGTH
|
||||
assert resolve_strength(None, None, "qwen") == QWEN_GEMINI_STRENGTH # unknown -> higher floor
|
||||
assert resolve_strength(None, "google", "qwen") > resolve_strength(None, "google", "controlnet")
|
||||
# An explicit strength still wins on qwen.
|
||||
assert resolve_strength(0.12, "google", "qwen") == 0.12
|
||||
|
||||
def test_ladder_is_the_certified_controlnet_floors(self):
|
||||
# The unified ladder == the oracle-certified controlnet floors. Lowered on the
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
"""Byte-identity guards for the text-mark engine memory optimization.
|
||||
|
||||
The reverse-alpha text-mark engine used to allocate full-frame arrays where only
|
||||
the glyph footprint is ever read:
|
||||
|
||||
* ``extract_mask`` built a full ``(h, w)`` uint8 mask and every caller cropped
|
||||
it to the located box;
|
||||
* ``_fixed_alpha_map`` / ``_aligned_alpha_map`` each built a full ``(h, w)``
|
||||
float32 alpha map that is non-zero only inside the glyph box, and two were
|
||||
held at once during removal.
|
||||
|
||||
Both now return footprint-sized arrays. These tests prove the new footprint-sized
|
||||
path is BYTE-IDENTICAL to the old full-frame path by reconstructing the old
|
||||
behavior inline from the new building blocks (so the proof survives a cv2/asset
|
||||
version bump, unlike a pinned output hash), and lock in the O(footprint) memory
|
||||
characteristic so a regression back to a full-frame allocation fails loudly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
import remove_ai_watermarks.doubao_engine as D
|
||||
import remove_ai_watermarks.jimeng_engine as J
|
||||
import remove_ai_watermarks.samsung_engine as S
|
||||
from remove_ai_watermarks.doubao_engine import DoubaoEngine
|
||||
from remove_ai_watermarks.jimeng_engine import JimengEngine
|
||||
from remove_ai_watermarks.samsung_engine import SamsungEngine
|
||||
|
||||
# (engine factory, engine module) for each reverse-alpha text mark.
|
||||
ENGINES = [
|
||||
pytest.param(DoubaoEngine, D, id="doubao"),
|
||||
pytest.param(JimengEngine, J, id="jimeng"),
|
||||
pytest.param(SamsungEngine, S, id="samsung"),
|
||||
]
|
||||
|
||||
|
||||
def _watermarked(engine, module) -> np.ndarray:
|
||||
"""Composite the engine's real alpha glyph onto a flat mid-gray field at the
|
||||
captured native width (so both placement candidates fire)."""
|
||||
cfg = engine.config
|
||||
nw = module._ALPHA_NATIVE_WIDTH
|
||||
at = module._alpha_template()
|
||||
gw, gh = int(cfg.alpha_width_frac * nw), int(cfg.alpha_height_frac * nw)
|
||||
ax = (nw - int(cfg.alpha_margin_x_frac * nw) - gw) if cfg.corner == "br" else int(cfg.alpha_margin_x_frac * nw)
|
||||
ay = nw - int(cfg.alpha_margin_bottom_frac * nw) - gh
|
||||
amap = np.zeros((nw, nw), np.float32)
|
||||
amap[ay : ay + gh, ax : ax + gw] = cv2.resize(at, (gw, gh))
|
||||
a3 = amap[:, :, None]
|
||||
img = np.full((nw, nw, 3), 100.0, np.float32)
|
||||
return (a3 * np.array(cfg.alpha_logo_bgr, np.float32) + (1 - a3) * img).clip(0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("factory", "module"), ENGINES)
|
||||
class TestExtractMaskFootprint:
|
||||
def test_returns_box_sized_mask(self, factory, module):
|
||||
eng = factory()
|
||||
img = _watermarked(eng, module)
|
||||
loc = eng.locate(img)
|
||||
box = eng.extract_mask(img, loc)
|
||||
assert box.dtype == np.uint8
|
||||
# Shape == loc.bbox, i.e. the old full-frame mask's [y:y+bh, x:x+bw] crop.
|
||||
assert box.shape == (loc.h, loc.w)
|
||||
# Footprint, not full frame: the box is a tiny fraction of the image.
|
||||
assert box.size * 4 < img.shape[0] * img.shape[1]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("factory", "module"), ENGINES)
|
||||
class TestAlphaMapFootprint:
|
||||
def test_maps_are_footprint_sized_blocks(self, factory, module):
|
||||
eng = factory()
|
||||
img = _watermarked(eng, module)
|
||||
for placed in (eng._fixed_alpha_map(img), eng._aligned_alpha_map(img)):
|
||||
assert placed is not None
|
||||
block, (ax, ay, gw, gh) = placed
|
||||
assert block.dtype == np.float32
|
||||
assert block.shape == (gh, gw)
|
||||
# The placement stays fully inside the image (no clipping needed).
|
||||
assert ax >= 0
|
||||
assert ax + gw <= img.shape[1]
|
||||
assert ay >= 0
|
||||
assert ay + gh <= img.shape[0]
|
||||
# O(footprint): far smaller than the frame.
|
||||
assert block.size * 4 < img.shape[0] * img.shape[1]
|
||||
|
||||
def test_apply_reverse_alpha_equals_old_fullframe(self, factory, module):
|
||||
"""``_apply_reverse_alpha`` with the glyph block is byte-identical to the
|
||||
old full-frame path: rebuild the full ``(h, w)`` map, run the old-style
|
||||
full-frame reverse-alpha, and compare to the new block-based output."""
|
||||
eng = factory()
|
||||
img = _watermarked(eng, module)
|
||||
h, w = img.shape[:2]
|
||||
for placed in (eng._fixed_alpha_map(img), eng._aligned_alpha_map(img)):
|
||||
assert placed is not None
|
||||
block, region = placed
|
||||
ax, ay, gw, gh = region
|
||||
|
||||
new_out = eng._apply_reverse_alpha(img, block, region)
|
||||
|
||||
# Old behavior: a full-frame map, indexed by region inside _apply_reverse_alpha.
|
||||
full = np.zeros((h, w), np.float32)
|
||||
full[ay : ay + gh, ax : ax + gw] = block
|
||||
old_out = img.copy()
|
||||
a3 = np.clip(full[ay : ay + gh, ax : ax + gw], 0.0, 1.0)[:, :, None]
|
||||
logo = np.array(eng.config.alpha_logo_bgr, np.float32)
|
||||
roi = old_out[ay : ay + gh, ax : ax + gw].astype(np.float32)
|
||||
old_out[ay : ay + gh, ax : ax + gw] = np.clip(
|
||||
(roi - a3 * logo) / np.clip(1.0 - a3, 0.25, 1.0), 0, 255
|
||||
).astype(np.uint8)
|
||||
|
||||
assert np.array_equal(new_out, old_out)
|
||||
|
||||
def test_residual_mask_equals_old_fullframe(self, factory, module):
|
||||
"""The residual inpaint mask built from the block embedded in a full-frame
|
||||
canvas equals thresholding the old full-frame float32 map (zero outside the
|
||||
block), so the dilate + inpaint see the same mask."""
|
||||
eng = factory()
|
||||
img = _watermarked(eng, module)
|
||||
h, w = img.shape[:2]
|
||||
cfg = eng.config
|
||||
block, (ax, ay, gw, gh) = eng._fixed_alpha_map(img)
|
||||
|
||||
# New: embed the block into a uint8 canvas, then threshold.
|
||||
new_mask = np.zeros((h, w), np.uint8)
|
||||
new_mask[ay : ay + gh, ax : ax + gw] = (block > cfg.residual_alpha_floor).astype(np.uint8) * 255
|
||||
|
||||
# Old: a full-frame float32 map, thresholded everywhere.
|
||||
old_full = np.zeros((h, w), np.float32)
|
||||
old_full[ay : ay + gh, ax : ax + gw] = block
|
||||
old_mask = (old_full > cfg.residual_alpha_floor).astype(np.uint8) * 255
|
||||
|
||||
assert np.array_equal(new_mask, old_mask)
|
||||
@@ -0,0 +1,120 @@
|
||||
"""Reverse-alpha over-subtraction guard for the visible text-mark engines.
|
||||
|
||||
Ported from the Gemini sparkle fix (commit 41f6797) to Doubao/Jimeng/Samsung
|
||||
(retained-corpus mining 2026-06-20, roadmap P0#8): on a dark or mid-tone
|
||||
background the captured alpha can over-estimate THIS image's mark opacity, and
|
||||
reverse-alpha leaves a darker-than-background glyph ghost (a "dark pit") instead
|
||||
of recovering the true pixels. The guard predicts the reverse-alpha output per
|
||||
pixel and, when the glyph body lands far below the local ring, reconstructs the
|
||||
footprint from the original surroundings instead of shipping the pit.
|
||||
|
||||
These assert visual residual (pixel levels vs the local background), not just a
|
||||
detector re-fire -- a dark pit can clear the NCC detector while still looking wrong.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from remove_ai_watermarks import image_io
|
||||
from remove_ai_watermarks._text_mark_engine import _OVERSUB_DARK_MARGIN
|
||||
from remove_ai_watermarks.doubao_engine import DoubaoEngine
|
||||
from remove_ai_watermarks.jimeng_engine import JimengEngine
|
||||
from remove_ai_watermarks.samsung_engine import SamsungEngine
|
||||
|
||||
_ENGINES = [DoubaoEngine, JimengEngine, SamsungEngine]
|
||||
|
||||
|
||||
def _compose(engine, bg: float, opacity_gain: float, w: int = 1024, h: int = 1024):
|
||||
"""Composite the engine's captured mark onto a flat ``bg`` at ``opacity_gain``.
|
||||
|
||||
``opacity_gain < 1`` makes the mark FAINTER than the capture, so reverse-alpha
|
||||
at the full captured alpha over-subtracts into a dark pit -- the case the guard
|
||||
must catch. Returns ``(watermarked_uint8, alpha_block, region)`` where the block
|
||||
and region are exactly what the engine's reverse-alpha receives.
|
||||
"""
|
||||
img = np.full((h, w, 3), float(bg), np.float32)
|
||||
block, (ax, ay, gw, gh) = engine._fixed_alpha_map(img)
|
||||
a = np.clip(block * opacity_gain, 0.0, 0.99)[:, :, None]
|
||||
logo = np.array(engine.config.alpha_logo_bgr, np.float32)
|
||||
img[ay : ay + gh, ax : ax + gw] = img[ay : ay + gh, ax : ax + gw] * (1 - a) + logo * a
|
||||
return np.clip(img, 0, 255).astype(np.uint8), block, (ax, ay, gw, gh)
|
||||
|
||||
|
||||
def _body_vs_ring(out, region, block) -> tuple[float, float]:
|
||||
"""Median luma of the glyph body vs the local background ring in ``out``."""
|
||||
ax, ay, gw, gh = region
|
||||
g = out.astype(np.float32).mean(axis=2)
|
||||
body = block >= 0.15
|
||||
pad = max(4, int(gh * 0.6))
|
||||
ry1, ry2 = max(0, ay - pad), min(g.shape[0], ay + gh + pad)
|
||||
rx1, rx2 = max(0, ax - pad), min(g.shape[1], ax + gw + pad)
|
||||
ring = g[ry1:ry2, rx1:rx2]
|
||||
fy1, fy2, fx1, fx2 = ay - ry1, ay - ry1 + gh, ax - rx1, ax - rx1 + gw
|
||||
ring_mask = np.ones(ring.shape, dtype=bool)
|
||||
ring_mask[fy1:fy2, fx1:fx2] = False
|
||||
core = float(np.median(g[ay : ay + gh, ax : ax + gw][body]))
|
||||
return core, float(np.median(ring[ring_mask]))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("Engine", _ENGINES, ids=lambda e: e.__name__)
|
||||
class TestOversubtractionGuard:
|
||||
@pytest.mark.parametrize(("bg", "gain"), [(120, 0.45), (150, 0.4), (90, 0.5)])
|
||||
def test_guard_trips_on_faint_mark(self, Engine, bg, gain):
|
||||
eng = Engine()
|
||||
wm, block, region = _compose(eng, bg, gain)
|
||||
assert eng._reverse_alpha_oversubtracts(image_io.to_bgr(wm), block, region)
|
||||
|
||||
@pytest.mark.parametrize("bg", [255, 200, 128, 60])
|
||||
def test_guard_skips_clean_full_strength_mark(self, Engine, bg):
|
||||
# A cleanly captured (gain 1.0) mark predicts back to the background, so the
|
||||
# guard must NOT trip -- no regression of the common clean-removal path.
|
||||
eng = Engine()
|
||||
wm, block, region = _compose(eng, bg, 1.0)
|
||||
assert not eng._reverse_alpha_oversubtracts(image_io.to_bgr(wm), block, region)
|
||||
|
||||
@pytest.mark.parametrize(("bg", "gain"), [(120, 0.45), (150, 0.4)])
|
||||
def test_faint_removal_leaves_no_dark_pit(self, Engine, bg, gain):
|
||||
# End-to-end acceptance (roadmap P0#8): after removal the glyph footprint is
|
||||
# not a region more than _OVERSUB_DARK_MARGIN below the local background.
|
||||
eng = Engine()
|
||||
wm, block, region = _compose(eng, bg, gain)
|
||||
out = eng.remove_watermark_reverse_alpha(wm)
|
||||
core, ring_bg = _body_vs_ring(out, region, block)
|
||||
assert core >= ring_bg - _OVERSUB_DARK_MARGIN, f"dark pit: body {core:.0f} vs ring {ring_bg:.0f}"
|
||||
|
||||
def test_clean_mark_removal_unchanged_by_guard(self, Engine, monkeypatch):
|
||||
# On a clean mark the guard must be a no-op: forcing it off yields the same
|
||||
# output (the guard only ever diverts the over-subtraction case).
|
||||
eng = Engine()
|
||||
wm, _block, _region = _compose(eng, 200, 1.0)
|
||||
guarded = eng.remove_watermark_reverse_alpha(wm)
|
||||
monkeypatch.setattr(type(eng), "_reverse_alpha_oversubtracts", lambda self, *a, **k: False)
|
||||
unguarded = eng.remove_watermark_reverse_alpha(wm)
|
||||
assert np.array_equal(guarded, unguarded)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("Engine", _ENGINES, ids=lambda e: e.__name__)
|
||||
def test_guard_recovers_pit_on_textured_background(Engine):
|
||||
"""The guard's footprint inpaint reconstructs from the ORIGINAL surroundings,
|
||||
so a faint mark over-subtracted on a textured background recovers to roughly the
|
||||
local content level rather than a glyph-shaped dark ghost."""
|
||||
eng = Engine()
|
||||
w = h = 1024
|
||||
yy, xx = np.mgrid[0:h, 0:w].astype(np.float32)
|
||||
base = 120 + 35 * np.sin(xx / 80.0) + 25 * np.cos(yy / 60.0)
|
||||
bg_img = np.clip(np.stack([base, base * 0.95, base * 1.05], axis=-1), 0, 255)
|
||||
block, (ax, ay, gw, gh) = eng._fixed_alpha_map(bg_img)
|
||||
a = np.clip(block * 0.45, 0.0, 0.99)[:, :, None]
|
||||
logo = np.array(eng.config.alpha_logo_bgr, np.float32)
|
||||
bg_img[ay : ay + gh, ax : ax + gw] = bg_img[ay : ay + gh, ax : ax + gw] * (1 - a) + logo * a
|
||||
wm = np.clip(bg_img, 0, 255).astype(np.uint8)
|
||||
out = eng.remove_watermark_reverse_alpha(wm).astype(np.float32)
|
||||
# Compare the recovered glyph body to the clean texture under the mark.
|
||||
clean = np.clip(np.stack([base, base * 0.95, base * 1.05], axis=-1), 0, 255)
|
||||
body = block >= 0.15
|
||||
region_out = out[ay : ay + gh, ax : ax + gw].mean(axis=2)
|
||||
region_clean = clean[ay : ay + gh, ax : ax + gw].mean(axis=2)
|
||||
err = float(np.abs(region_out[body] - region_clean[body]).mean())
|
||||
assert err < 25.0, f"glyph body not recovered (mean abs err {err:.1f})"
|
||||
@@ -0,0 +1,210 @@
|
||||
"""Unit tests for the sliding-window tiled-diffusion helpers (no GPU/model).
|
||||
|
||||
``tiling`` is pure numpy/PIL: the geometry (``plan_tiles`` / ``_axis_positions``),
|
||||
the feather window (``feather_weights``), and the blend loop (``run_tiled``) are all
|
||||
exercised here with a plain callable standing in for the diffusion pass, so the
|
||||
seam-free reconstruction and the tile layout are guarded without loading SDXL.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from remove_ai_watermarks.noai.tiling import (
|
||||
Tile,
|
||||
_axis_positions,
|
||||
feather_region_composite,
|
||||
feather_weights,
|
||||
plan_tiles,
|
||||
run_tiled,
|
||||
)
|
||||
|
||||
|
||||
class TestAxisPositions:
|
||||
def test_single_tile_when_length_fits(self):
|
||||
assert _axis_positions(800, 1024, 128) == [0]
|
||||
assert _axis_positions(1024, 1024, 128) == [0]
|
||||
|
||||
def test_two_tiles_last_flush_to_edge(self):
|
||||
# 1500 wide, tile 1024, step 1024-128=896: starts [0], then last = 1500-1024=476.
|
||||
assert _axis_positions(1500, 1024, 128) == [0, 476]
|
||||
|
||||
def test_uniform_step_then_flush(self):
|
||||
# 3000, tile 1024, step 896 -> 0, 896, 1792; last = 1976 appended (2688 would
|
||||
# overrun). The regular range stops at 1792 (next 2688 > 1976), so flush adds 1976.
|
||||
assert _axis_positions(3000, 1024, 128) == [0, 896, 1792, 1976]
|
||||
|
||||
def test_no_duplicate_when_range_already_hits_edge(self):
|
||||
# length-tile divisible by step: the last range entry already equals the edge.
|
||||
# 1024*2-128 = 1920 width, step 896, last = 896 -> range gives [0, 896]; no dup.
|
||||
assert _axis_positions(1920, 1024, 128) == [0, 896]
|
||||
|
||||
def test_overlap_at_least_tile_still_progresses(self):
|
||||
# Pathological overlap >= tile is clamped to tile-1 so step stays >= 1.
|
||||
positions = _axis_positions(2000, 1024, 5000)
|
||||
assert positions[0] == 0
|
||||
assert positions[-1] == 2000 - 1024
|
||||
assert positions == sorted(positions)
|
||||
|
||||
def test_invalid_tile_raises(self):
|
||||
with pytest.raises(ValueError, match="tile must be positive"):
|
||||
_axis_positions(100, 0, 10)
|
||||
|
||||
|
||||
class TestPlanTiles:
|
||||
def test_single_tile_for_small_image(self):
|
||||
tiles = plan_tiles(800, 600, 1024, 128)
|
||||
assert tiles == [Tile(0, 0, 800, 600)]
|
||||
|
||||
def test_grid_is_row_major_and_uniform_size(self):
|
||||
tiles = plan_tiles(1500, 1500, 1024, 128)
|
||||
# 2x2 grid: xs/ys both [0, 476].
|
||||
assert [(t.x, t.y) for t in tiles] == [(0, 0), (476, 0), (0, 476), (476, 476)]
|
||||
# Every tile is exactly tile_size (uniform -> SDXL-friendly).
|
||||
assert all((t.width, t.height) == (1024, 1024) for t in tiles)
|
||||
|
||||
def test_tiles_cover_the_full_canvas(self):
|
||||
width, height = 2600, 1800
|
||||
tiles = plan_tiles(width, height, 1024, 128)
|
||||
covered = np.zeros((height, width), dtype=bool)
|
||||
for t in tiles:
|
||||
covered[t.y : t.y + t.height, t.x : t.x + t.width] = True
|
||||
assert covered.all()
|
||||
|
||||
|
||||
class TestFeatherWeights:
|
||||
def test_shape_matches_tile(self):
|
||||
assert feather_weights(64, 48, 8).shape == (48, 64)
|
||||
|
||||
def test_strictly_positive(self):
|
||||
assert (feather_weights(64, 64, 16) > 0).all()
|
||||
|
||||
def test_interior_higher_than_edge(self):
|
||||
w = feather_weights(64, 64, 16)
|
||||
assert w[32, 32] == pytest.approx(1.0)
|
||||
# The corner sits in the taper, so it is well below the interior.
|
||||
assert w[0, 0] < w[32, 32]
|
||||
|
||||
def test_symmetric(self):
|
||||
w = feather_weights(64, 64, 16)
|
||||
assert np.allclose(w, w[::-1, :])
|
||||
assert np.allclose(w, w[:, ::-1])
|
||||
|
||||
def test_zero_overlap_is_flat(self):
|
||||
# No taper requested -> a flat window of ones.
|
||||
assert np.allclose(feather_weights(32, 32, 0), 1.0)
|
||||
|
||||
|
||||
class TestRunTiled:
|
||||
def test_identity_generate_reconstructs_image(self):
|
||||
# A blend of identical (unchanged) tiles must reproduce the input exactly,
|
||||
# regardless of overlap -- the feather weights are a partition-of-unity once
|
||||
# normalised. This is the seam-free guarantee.
|
||||
rng = np.random.default_rng(0)
|
||||
arr = rng.integers(0, 256, size=(1500, 1300, 3), dtype=np.uint8)
|
||||
image = Image.fromarray(arr)
|
||||
|
||||
out = run_tiled(lambda tile: tile, image, tile_size=512, overlap=64)
|
||||
|
||||
assert out.size == image.size
|
||||
assert np.abs(np.asarray(out, dtype=np.int16) - arr.astype(np.int16)).max() <= 1
|
||||
|
||||
def test_generate_called_once_per_tile(self):
|
||||
calls: list[tuple[int, int]] = []
|
||||
|
||||
def generate(tile: Image.Image) -> Image.Image:
|
||||
calls.append(tile.size)
|
||||
return tile
|
||||
|
||||
image = Image.new("RGB", (1500, 1500), (120, 130, 140))
|
||||
run_tiled(generate, image, tile_size=1024, overlap=128)
|
||||
|
||||
assert len(calls) == len(plan_tiles(1500, 1500, 1024, 128)) == 4
|
||||
|
||||
def test_single_tile_path_for_small_image(self):
|
||||
image = Image.new("RGB", (300, 200), (10, 20, 30))
|
||||
out = run_tiled(lambda tile: tile, image, tile_size=1024, overlap=128)
|
||||
assert out.size == (300, 200)
|
||||
assert np.asarray(out)[0, 0].tolist() == [10, 20, 30]
|
||||
|
||||
def test_mismatched_generate_output_is_resized_back(self):
|
||||
# A pipeline that rounds dims to the latent grid returns a slightly different
|
||||
# size; run_tiled must resize it back so the blend buffers line up.
|
||||
def generate(tile: Image.Image) -> Image.Image:
|
||||
w, h = tile.size
|
||||
return tile.resize((w - w % 8, h - h % 8), Image.Resampling.LANCZOS)
|
||||
|
||||
image = Image.new("RGB", (1500, 1100), (200, 100, 50))
|
||||
out = run_tiled(generate, image, tile_size=1024, overlap=128)
|
||||
assert out.size == (1500, 1100)
|
||||
|
||||
|
||||
class TestFeatherRegionComposite:
|
||||
"""Region-targeted compositing for AI-enhanced composites: only the AI box is
|
||||
regenerated, the real photo outside it stays pixel-exact (roadmap P1#8)."""
|
||||
|
||||
@staticmethod
|
||||
def _frames(h=200, w=300):
|
||||
base = np.full((h, w, 3), 80, np.uint8)
|
||||
regenerated = np.full((h, w, 3), 200, np.uint8)
|
||||
return base, regenerated
|
||||
|
||||
def test_outside_box_is_pixel_exact(self):
|
||||
base, regen = self._frames()
|
||||
out = feather_region_composite(base, regen, (100, 60, 80, 50), feather=8)
|
||||
# Far corners are well outside the box -> identical to base.
|
||||
assert np.array_equal(out[:50, :80], base[:50, :80])
|
||||
assert np.array_equal(out[150:, 220:], base[150:, 220:])
|
||||
|
||||
def test_interior_equals_regenerated(self):
|
||||
base, regen = self._frames()
|
||||
out = feather_region_composite(base, regen, (100, 60, 80, 50), feather=8)
|
||||
# Deep interior of the box (past the feather ramp) is fully regenerated.
|
||||
assert np.array_equal(out[80:90, 130:150], regen[80:90, 130:150])
|
||||
|
||||
def test_hard_paste_when_no_feather(self):
|
||||
base, regen = self._frames()
|
||||
out = feather_region_composite(base, regen, (100, 60, 80, 50), feather=0)
|
||||
assert np.array_equal(out[60:110, 100:180], regen[60:110, 100:180])
|
||||
assert np.array_equal(out[:60], base[:60])
|
||||
|
||||
def test_seam_is_monotonic_ramp(self):
|
||||
base, regen = self._frames()
|
||||
out = feather_region_composite(base, regen, (100, 60, 80, 50), feather=10).astype(np.float32)
|
||||
# Along a horizontal line crossing the left edge, values rise from base(80)
|
||||
# toward regenerated(200) monotonically through the feather band.
|
||||
row = out[85, 100:115, 0]
|
||||
assert row[0] < row[-1]
|
||||
assert np.all(np.diff(row) >= -1e-3)
|
||||
|
||||
def test_dtype_preserved(self):
|
||||
base, regen = self._frames()
|
||||
out = feather_region_composite(base, regen, (50, 50, 40, 40), feather=4)
|
||||
assert out.dtype == base.dtype
|
||||
|
||||
def test_grayscale_2d_supported(self):
|
||||
base = np.full((100, 120), 30, np.uint8)
|
||||
regen = np.full((100, 120), 220, np.uint8)
|
||||
out = feather_region_composite(base, regen, (40, 30, 30, 30), feather=4)
|
||||
assert out.shape == base.shape
|
||||
assert np.array_equal(out[:30], base[:30])
|
||||
|
||||
def test_empty_or_offimage_box_returns_base(self):
|
||||
base, regen = self._frames()
|
||||
assert np.array_equal(feather_region_composite(base, regen, (0, 0, 0, 0)), base)
|
||||
assert np.array_equal(feather_region_composite(base, regen, (500, 500, 40, 40)), base)
|
||||
|
||||
def test_box_clamped_to_image_bounds(self):
|
||||
base, regen = self._frames()
|
||||
# Box overhangs the bottom-right; only the in-image part is composited.
|
||||
out = feather_region_composite(base, regen, (280, 180, 60, 60), feather=0)
|
||||
assert np.array_equal(out[180:, 280:], regen[180:, 280:])
|
||||
assert out.shape == base.shape
|
||||
|
||||
def test_shape_mismatch_raises(self):
|
||||
base, _ = self._frames(200, 300)
|
||||
bad = np.full((100, 100, 3), 200, np.uint8)
|
||||
with pytest.raises(ValueError, match="shape mismatch"):
|
||||
feather_region_composite(base, bad, (10, 10, 20, 20))
|
||||
Reference in new issue
Block a user