From f16216cabc7ccc6a96e135f7607b0fb97f3288fc Mon Sep 17 00:00:00 2001 From: Victor Kuznetsov Date: Sun, 31 May 2026 15:27:14 -0700 Subject: [PATCH] feat(cli): add --no-protect-faces to invisible/all (skip the YOLO face detector) Mirrors --no-protect-text: when the image has no people, skip loading and running the YOLO face detector entirely. The heavy extract+blend already only ran when a face was found, but the detector itself always loaded+inferred to decide; this flag lets callers skip that fixed cost. Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 2 +- README.md | 2 +- src/remove_ai_watermarks/cli.py | 16 ++++++++++++++++ 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index b7e1a64..1a1b1cc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,7 +43,7 @@ You are a **principal Python engineer** maintaining a CLI tool and library for r - `invisible_watermark.py` — `detect_invisible_watermark(path)` decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the `imwatermark` library. Known fixed patterns (verified against upstream source) live in `_BITS_48` (SDXL 48-bit, FLUX.2 48-bit) and `_SD1_STRING` ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra `detect`); returns None when absent. The `detect` extra pulls **torch** transitively (invisible-watermark declares torch a hard dep, and `WatermarkDecoder` eagerly imports `rivaGan` -> `torch` at import time), so detection needs torch present even though dwtDct runs CPU-only on cv2/numpy/pywavelets — no GPU and no separate `gpu` extra required. **Unlike SynthID this is locally detectable**, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs. - `trustmark_detector.py` — `detect_trustmark(path)` decodes the OPEN, keyless **Adobe TrustMark** watermark (the soft binding behind Adobe Durable Content Credentials, `alg` `com.adobe.trustmark.P`) via the optional `trustmark` package (extra `trustmark`; pulls torch, downloads model weights on first use). Mirrors `invisible_watermark.py` (lazy singleton guarded by a double-checked `threading.Lock` so concurrent callers do not double-download the weights, top-of-module pyright pragma, returns None when absent). It detects *provenance*, not AI origin as such (TrustMark also marks human-authored content), so `identify` lists it as a watermark without setting `is_ai_generated`. Other soft-binding vendors (Digimarc/Imatag/Steg.AI/...) have no public decoder — they are only *named* via the `C2PA_SOFT_BINDINGS` scan, not decoded. **False-positive gate (added 2026-05-29):** TrustMark's `wm_present` is a BCH error-correction validity flag that spuriously validates on a content-correlated fraction of un-watermarked images — AI-generated textures trip it far more than camera photos (verified 2026-05-29 on real files: it fires on Gemini/OpenAI/Doubao output that *cannot* carry Adobe's watermark, with a random-bytes decoded secret, while signal-free camera photos did not trip it). A genuine TrustMark is a *durable* soft binding engineered to survive re-encoding, so `detect_trustmark` re-decodes after a mild JPEG round-trip (`_survives_reencode`, `_REENCODE_QUALITY` 95) and requires the same schema both times; every observed false positive collapsed (none survived even q95), so the gate is the durability property the watermark guarantees. The second decode runs only on the rare initial hit, so the cost is negligible. Do NOT remove the gate to "catch more" — a lone TrustMark hit without it is almost always content noise. - `text_protector.py` — text-region protection for the `invisible` SDXL img2img pass (issue #21: CJK/small text deforms at watermark-removal strengths). `is_available()` gates on `cv2.dnn.TextDetectionModel_DB`; `TextProtector.detect_text_boxes(bgr)` runs the **PP-OCRv3 DB** ONNX detector (~2.4 MB, Apache-2.0, opencv_zoo, returns rotated quad polygons) — downloaded+cached to `~/.cache/remove-ai-watermarks` on first use via atomic temp-rename, never bundled, **no torch (cv2.dnn only)**. **Detection is script-agnostic** (DB segments text *regions*, not characters), so Latin / Cyrillic / CJK / Hangul / Arabic / digits all detect identically — language was never the recall lever, **resolution was**. `_detection_input_size(h, w)` (pure, unit-tested) detects at the **native long side capped at `_DET_MAX_LONG_SIDE` (1536), never upscaled**: the old fixed 736 downscaled large canvases so small text fell below the detector and was missed (issue #14, e.g. ~16 px text on a 2048 image). `scripts/text_detection_benchmark.py` measures recall across scripts × sizes × canvas: the cap fix lifts overall hit-rate 0.91 → 1.00 (worst cell 2048/16 px: 0.06 → 1.00) at ~100 ms CPU. Very large canvases with tiny text may still need tiling (documented limit, not built). `build_change_map(boxes, h, w, preserve=0.9, feather=15)` paints a Differential-Diffusion change map. **Polarity (verified empirically):** white(1.0)=PRESERVE original pixels, black(0.0)=MAX change; map is black bg + `preserve` inside text polygons, Gaussian-feathered edges, clipped to [0,1]. `preserve` stays below a hard 1.0 freeze by default so text still scrubs lightly (SynthID survives cropping). **Default text protection is `watermark_remover._run_region_hires`, NOT the differential change map.** Differential Diffusion froze text in latent space (`preserve`<1.0), so the watermark survived *inside* text — violating the "remove SynthID everywhere" requirement; and the SDXL VAE's 8px latent cell softens sub-8px strokes regardless of `preserve` (architectural limit, confirmed by the DD authors — see `docs/text-protection-research.md`). `_run_region_hires` instead: (1) scrubs the whole image (plain img2img), (2) RE-scrubs each detected text block at HIGH resolution and feather-composites it back. `merge_text_regions(boxes,h,w)` groups boxes into local blocks; each crop is upscaled by `_REGION_HIRES_SCALE` 3.0 (applied as an integer factor via int(...), capped so a region stays under `_REGION_MAX_MEGAPIXELS` 1.3 to avoid OOM; skipped if it can't reach 2x — very large text areas then fall back to the global scrub, tiling is the future fix), img2img-scrubbed, downscaled, **phase-correlated back to the original crop to null the ~1-2px round-trip offset** (the shift is applied only on a confident, small correlation -- `response > 0.3` and `|shift| < 4` -- so a spurious large offset on a flat crop no longer garbles the composite; and after a CPU fallback the generator is dropped before the per-region passes to avoid an MPS-vs-CPU generator device mismatch) (a sub-pixel shift garbles the composite even when text is crisp; integer scale alone did NOT fix it because the diffusion pipeline rounds dims to a multiple of 8), then `feather_paste`d. Every pixel is regenerated, so the watermark is removed everywhere AND small text stays crisp (high-res strokes span >1 latent cell). Validated on synthetic 18px multilingual text: text-region SSIM 0.28 (plain) → 0.48 (region-hires), visually garbled → readable across Latin/Cyrillic/CJK, residual shift ~0.5px. Gated to the SDXL `DEFAULT_MODEL_ID` + detector (`_can_protect_text`); no text → plain global scrub (text-free inputs pay only the cheap cv2 detection). CLI off-switch `--no-protect-text` on `invisible`/`all`. `merge_text_regions` + `feather_paste` are pure, unit-tested without a model (`tests/test_text_protector.py`). **MUST still be confirmed by the SynthID oracle** (openai.com/verify / Gemini app) that a region-rescrubbed text zone reads watermark-free before trusting it in prod. The legacy `_run_differential` / `build_change_map` / `_load_differential_pipeline` (community `pipeline_stable_diffusion_xl_differential_img2img`, `custom_revision="0.38.0"`) remain in the file but are no longer the default; the diff pipeline upcasts the VAE to fp32 internally, so do **not** add `upcast_vae()`/`enable_attention_slicing` there (NaN/black on fp16 MPS). `build_change_map` is still unit-tested. -- `face_protector.py` — YOLO detect + soft-blend pattern; mirror this for any "protect region during diffusion" features +- `face_protector.py` — YOLO detect + soft-blend pattern; mirror this for any "protect region during diffusion" features. The expensive extract+blend already runs only when a face is found, but the YOLO detector itself always loads+runs to decide; CLI off-switch `--no-protect-faces` on `invisible`/`all` skips it entirely (mirrors `--no-protect-text`), for images known to have no people. - `humanizer.py` — optional post-process "humanize" effects (cv2/numpy). The chromatic-shift step replicates the border instead of wrapping opposite-edge pixels, so a shifted channel no longer bleeds the far edge into the near one. - `image_io.py` — Unicode-safe cv2 IO (issue #17). `imread(path, flags=None)` / `imwrite(path, img)` wrap `np.fromfile`+`cv2.imdecode` / `cv2.imencode`+`tofile` so non-ASCII paths work on Windows -- bare `cv2.imread`/`cv2.imwrite` use the platform ANSI code-page API there and fail (empty decode + `can't open/read file`) on Chinese/Cyrillic/accented filenames. `imread` keeps `cv2.imread` semantics (defaults to `IMREAD_COLOR`, returns `None` on missing/empty/undecodable). **Every cv2 file read/write in the package routes through here; do not call `cv2.imread`/`cv2.imwrite` directly.** `imwrite` returns `False` on an unwritable path (`OSError` caught) instead of raising, matching `cv2.imwrite` semantics. macOS/Linux already accept UTF-8 paths, so it is behavior-neutral there (the bug only reproduces on Windows). cv2/numpy are imported lazily inside the functions, so the module is cheap to import in a bare env. diff --git a/README.md b/README.md index db50f8a..b864da0 100644 --- a/README.md +++ b/README.md @@ -122,7 +122,7 @@ SDXL is the default since May 2026: empirically defeats SynthID v2 on Gemini 3 P > **Oracle vs `identify` can disagree, and that is expected.** An online verifier reads the actual SynthID *pixel* watermark and detects only its own vendor's content — [openai.com/research/verify](https://openai.com/research/verify/) states "OpenAI generation signals will only be detected if the image was generated with our tools". Our `identify` cannot decode the pixel watermark (no vendor ships a local decoder), so it infers SynthID from the **C2PA metadata** instead. So after the SDXL pass the oracle can read "no SynthID" (pixel watermark gone) while `identify` still reports SynthID from a surviving C2PA manifest. They measure different signals. Run `metadata --remove` (or `all`) to also strip the manifest; note that a quiet metadata proxy is not proof the pixel watermark itself is gone. -**Face Protection**: before diffusion, YOLO detects people in the image and extracts them. After diffusion, the original faces are blended back with a soft elliptical mask to prevent AI distortion of facial features. +**Face Protection**: before diffusion, YOLO detects people in the image and extracts them. After diffusion, the original faces are blended back with a soft elliptical mask to prevent AI distortion of facial features. Pass `--no-protect-faces` to skip the detector entirely (useful when the image has no people). **Analog Humanizer**: optional film grain and chromatic aberration injection that mimics a photo of a screen, raising the bar for AI-generated image classifiers. (It frustrates generic classifiers but does not guarantee forensic invisibility — see the [arXiv:2605.09203](https://arxiv.org/abs/2605.09203) note above.) diff --git a/src/remove_ai_watermarks/cli.py b/src/remove_ai_watermarks/cli.py index 0aadaa6..ea1f05e 100644 --- a/src/remove_ai_watermarks/cli.py +++ b/src/remove_ai_watermarks/cli.py @@ -465,6 +465,12 @@ def cmd_erase( default=False, help="Disable automatic text protection (text/CJK is preserved by default on the SDXL pipeline).", ) +@click.option( + "--no-protect-faces", + is_flag=True, + default=False, + help="Disable face protection (skips the YOLO face detector; use when the image has no people).", +) @click.pass_context def cmd_invisible( ctx: click.Context, @@ -479,6 +485,7 @@ def cmd_invisible( humanize: float, max_resolution: int, no_protect_text: bool, + no_protect_faces: bool, ) -> None: """Remove invisible AI watermarks (SynthID, StableSignature, TreeRing). @@ -525,6 +532,7 @@ def cmd_invisible( seed=seed, humanize=humanize, protect_text=not no_protect_text, + protect_faces=not no_protect_faces, max_resolution=max_resolution, ) elapsed = time.monotonic() - t0 @@ -705,6 +713,12 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo default=False, help="Disable automatic text protection (text/CJK is preserved by default on the SDXL pipeline).", ) +@click.option( + "--no-protect-faces", + is_flag=True, + default=False, + help="Disable face protection (skips the YOLO face detector; use when the image has no people).", +) @click.pass_context def cmd_all( ctx: click.Context, @@ -722,6 +736,7 @@ def cmd_all( humanize: float, max_resolution: int, no_protect_text: bool, + no_protect_faces: bool, ) -> None: """Remove ALL watermarks: visible + invisible + metadata. @@ -813,6 +828,7 @@ def cmd_all( seed=seed, humanize=humanize, protect_text=not no_protect_text, + protect_faces=not no_protect_faces, max_resolution=max_resolution, ) console.print(" Invisible watermark removed")