style: drop dead Candidate fields, simplify resolve_backend, US spelling

- Candidate carries only the fields the arbiter reads (key, label,
  detected_strict, detected_relaxed, features); location/region/confidence were
  vestigial from the removed best_auto_mark max-by-confidence path.
- resolve_backend returns preferred_inpaint_backend() directly (typed Literal)
  instead of an identity ternary.
- colour/normalise/behaviour -> US spelling across code comments and docs.

No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Victor Kuznetsov
2026-07-09 16:18:13 +03:00
parent 178fed69a7
commit 9756189eaf
10 changed files with 27 additions and 34 deletions
+1 -1
View File
@@ -60,7 +60,7 @@ Compact map. The full per-module detail (design decisions, tuned thresholds, cal
- `gemini_engine.py` — visible Gemini-sparkle detector + localizer (cv2/numpy, no GPU): top-K size-weighted fusion candidate selection (`_SELECT_TOPK`), corner-promote, false-positive gate (the provenance prior relaxes the gate + lowers the trust threshold when a Google/Gemini C2PA issuer confirms the vendor). Detection scores the top-K size-weighted matches by full fusion (spatial+gradient+variance) and keeps the highest — NOT the raw-NCC argmax, which re-admits the tiny-patch FPs the size weight suppresses (the osachub 2026-06-12 sub-0.85 corner-sparkle regression; see `docs/module-internals.md`). Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Removal is localize -> fill: `footprint_mask` returns the sparkle footprint (the captured alpha thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin), and the shared `watermark_registry.fill` inpaints it. The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery.
- `_text_mark_engine.py` — shared base for the three text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. Detection still matches the glyph silhouette (NCC, keys on glyph shape). The removal mask is TEMPLATE-FREE: it is the bounding box of the top-hat glyph blob (`extract_mask`), filled solid + dilated, so the shared fill inpaints the whole wordmark rectangle. This drops the fixed alpha-template placement, so a re-rendered or differently-placed mark is still masked; the captured alpha maps are now used only for the detection silhouette, not for removal. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model).
- `pill_engine.py` — the CAPTURE-LESS Jimeng-basic "AI生成" pill (top-left, issue #54). No alpha map: `detect` is edge-NCC of a synthetic font-rendered silhouette (`assets/jimeng_pill.png`, regenerate via `scripts/render_pill_silhouette.py`; committed, data-safe -- corpus stays out of the repo) in the top-left ROI, calibrated on 61 local real positives to threshold 0.22; `footprint_mask` is a generous FIXED top-left geometry box (NOT the NCC match position -- the synthetic silhouette localizes only approximately, the corner is negative space, so a geometry box fills cleanly while a match box leaves outline residue). `footprint_texture`/`footprint_is_flat` (median-Sobel over that box, `_FLAT_TEXTURE_MAX`) back the metadata-only safe-fill gate. Removal is the shared localize -> fill (MI-GAN/cv2). Detector precision is weak (~7% raw false-fire), so it is registry-gated in `remove_auto_marks` via `_keep_pill`: never on Doubao; the bottom-right wordmark removes it unrestricted (~94% precise, survives metadata-STRIPPED uploads); TC260-metadata-only removes it ONLY on a flat footprint (its textured false fires -- ceilings/walls -- are what the fill smears). Do NOT loosen those gates.
- `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Detection matches the glyph silhouette (NCC); removal localizes the glyph blob to a solid dilated box (`extract_mask`) and hands it to the shared fill. Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no colour shift, no dark pit); clean images with no vendor signature had 0% false removal. **Samsung detection is calibrated only for the Italian "Contenuti generati dall'AI" string** (a pre-existing limit, unchanged by the localize -> fill refactor but now surfaced because detection gates removal): non-Italian Samsung locales are not detected, and thus not removed, even though the fill mask itself is locale-independent; other locales need their own captured detection template.
- `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Detection matches the glyph silhouette (NCC); removal localizes the glyph blob to a solid dilated box (`extract_mask`) and hands it to the shared fill. Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. **Samsung detection is calibrated only for the Italian "Contenuti generati dall'AI" string** (a pre-existing limit, unchanged by the localize -> fill refactor but now surfaced because detection gates removal): non-Italian Samsung locales are not detected, and thus not removed, even though the fill mask itself is locale-independent; other locales need their own captured detection template.
- `region_eraser.py` — universal region eraser (`erase` CLI) and the shared fill backend behind `watermark_registry.fill` for the visible localize -> fill removal. Three backends: `cv2` (default, no deps, the floor), `migan` (MI-GAN ONNX, extra `migan`, MIT, ~28 MB / ~0.95 GB peak / ~0.19 s — the droplet-friendly tier, **the preferred default fill** when the extra is installed), `lama` (big-LaMa ONNX, extra `lama`, ~200 MB / ~4.7 GB peak — best quality, does not fit a minimal droplet, explicit opt-in only). **MI-GAN mask polarity is INVERTED** (0=hole/255=known) vs this package's 255-erase convention; `erase_migan` inverts before feeding the model (feeding 255=hole regenerates the whole frame into stripes — corpus-validated). Both ONNX models download on first use, never bundled. The `erase` command keeps its own `--backend`/`--inpaint-method` (unchanged).
- `invisible_watermark.py` — decodes the OPEN DWT-DCT watermarks (SD / SDXL / FLUX) via `imwatermark` (extra `detect`, pulls torch). Fragile two ways: (1) does not survive JPEG re-encode/resize; (2) **carrier-fragile on a broad class of pristine images** -- a clean encode->decode round-trip recovers 48/48 on chatgpt/firefly/random but FAILS (28-39/48, below the `_MATCH_48`=44 gate) on the FLUX fox, doubao, a flat FLUX generation, AND a clean synthetic flat fill with no watermark. The failure does NOT track texture; it goes with a degenerate **all-ones decode that is a CARRIER ARTIFACT, not a watermark** (synthetic clean image reproduces it). So `detect_invisible_watermark` is **positive-only**: trust a hit; a `None` is inconclusive unless a same-carrier positive-control embed first recovers >=44. Verified 2026-06-19; full caveat in `docs/watermarking-landscape.md`.
- `trustmark_detector.py` — Adobe TrustMark open decoder (extra `trustmark`). Do NOT remove the JPEG re-encode false-positive gate — a lone TrustMark hit without it is almost always content noise.
+6 -6
View File
@@ -28,7 +28,7 @@ It does **not** target watermarks that protect someone else's paid or copyrighte
## Features
- **Visible watermark removal** — a registry of known marks in their usual places: the Gemini / Nano Banana sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-left, locale-specific). Each mark is **localized to a footprint mask, then filled**: the engine finds the mark, builds a binary mask over its footprint, and one shared, swappable fill inpaints that region. Choose the fill with `--backend`: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN, light, the preferred default when the `migan` extra is installed), or `lama` (big-LaMa, best quality, heavier, explicit opt-in); the default `auto` uses MI-GAN if installed, else cv2. The localizer is cheap CPU (cv2/numpy), so it runs anywhere; the heavier MI-GAN/LaMa fill is opt-in. Detection keys on each mark's own shape (NCC against a captured silhouette; the alpha captures rebuilt by `scripts/visible_alpha_solve.py` are used to detect and to shape the mask, not for pixel recovery). The visual detector needs no metadata, but a borderline (faint or moved) mark is only trusted with corroboration: `--sensitivity` (default `auto`) relaxes a mark's gate when local metadata confirms the vendor or a same-product sibling mark is found; `--sensitivity assume-ai` relaxes every mark on the assertion that the image is AI, recovering the moved or re-rendered marks the conservative gate skips on a metadata-stripped screenshot (`strict` never relaxes). `visible --mark auto` finds and removes every detected mark in one pass. Fast, offline, no GPU. (For arbitrary logos/objects, see `erase`.)
- **Universal region eraser (`erase`)** — remove any logo / watermark / object inside boxes you specify, regardless of position or colour. Default cv2 inpainting (CPU, instant); optional big-LaMa via onnxruntime (`lama` extra) for higher quality
- **Universal region eraser (`erase`)** — remove any logo / watermark / object inside boxes you specify, regardless of position or color. Default cv2 inpainting (CPU, instant); optional big-LaMa via onnxruntime (`lama` extra) for higher quality
- **Invisible watermark removal** — SynthID, StableSignature, TreeRing via diffusion-based regeneration (needs a local GPU, or run it with no setup on [raiw.cc](https://raiw.cc))
- **AI metadata stripping** — EXIF, PNG text chunks, C2PA provenance manifests (PNG / JPEG / AVIF / HEIF / JPEG-XL, **MP4 / MOV / M4V / M4A** at the container level, and **WebM / MP3 / WAV / FLAC / OGG** losslessly via ffmpeg), XMP DigitalSourceType
- **"Made with AI" label removal** — removes the AI-disclosure metadata that platforms read to apply automatic labels (useful for clearing a false-positive label from a human-edited photograph)
@@ -66,7 +66,7 @@ It does **not** target watermarks that protect someone else's paid or copyrighte
| **StableSignature** (Meta) | — | ✅ In-model watermark | — | Diffusion regeneration |
| **TreeRing** | — | ✅ Latent space watermark | — | Diffusion regeneration |
> Visible overlays are used by Google Gemini / Nano Banana (sparkle logo), by ByteDance's Doubao ("豆包AI生成" corner text) and Jimeng / Dreamina ("★ 即梦AI" wordmark), and by Samsung Galaxy AI ("✦ Contenuti generati dall'AI" strip, bottom-left, locale-specific). All are removed by localizing the mark to a footprint mask and inpainting it with one shared fill (cv2 by default, MI-GAN or big-LaMa via `--backend`); the localizer is CPU-cheap and the heavier fills are opt-in. Other services rely on invisible watermarks and/or metadata; our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain. For a visible mark from any other source (any position, any colour), use the universal `erase --region` command.
> Visible overlays are used by Google Gemini / Nano Banana (sparkle logo), by ByteDance's Doubao ("豆包AI生成" corner text) and Jimeng / Dreamina ("★ 即梦AI" wordmark), and by Samsung Galaxy AI ("✦ Contenuti generati dall'AI" strip, bottom-left, locale-specific). All are removed by localizing the mark to a footprint mask and inpainting it with one shared fill (cv2 by default, MI-GAN or big-LaMa via `--backend`); the localizer is CPU-cheap and the heavier fills are opt-in. Other services rely on invisible watermarks and/or metadata; our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain. For a visible mark from any other source (any position, any color), use the universal `erase --region` command.
> **Detection:** `remove-ai-watermarks identify <image>` reports the origin platform and watermark inventory for all the signals above — C2PA issuer, the C2PA soft-binding forensic-watermark vendor (TrustMark / Digimarc / Imatag / ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, the China TC260 AIGC label (XMP, PNG chunk, EXIF, or JPEG segment), the HuggingFace `hf-job-id` job marker, embedded generation params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the SynthID metadata proxy, the C2PA cloud-manifest reference (Adobe Durable Content Credentials, when the embedded manifest is stripped), the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), and (with the `[detect]` / `[trustmark]` extras) the open SD/SDXL/FLUX and Adobe TrustMark invisible watermarks. SynthID and the proprietary soft-binding watermarks (Digimarc etc.) have no local decoder, so they are reported by metadata proxy / vendor name only.
@@ -86,13 +86,13 @@ A multi-scale NCC (Normalized Cross-Correlation) detector finds the sparkle posi
### Removing the Doubao "豆包AI生成" text watermark
Doubao (ByteDance) stamps every output with a light, semi-transparent "豆包AI生成" text strip in the bottom-right corner — the visible AIGC label mandated by China's TC260 standard. Detection matches the glyph silhouette against the corner (normalized correlation), so it keys on the "豆包AI生成" shape, not on textured corners. Removal is **localize then fill**: the light glyph blob is localized to a solid, dilated footprint mask (template-free, so a re-rendered or differently-placed mark is still masked) and the shared fill inpaints it. On corpus images the filled region blends into its surroundings within a few LAB levels, with no colour shift and no dark pit. Pick the fill with `--backend` (cv2 default, MI-GAN or big-LaMa opt-in).
Doubao (ByteDance) stamps every output with a light, semi-transparent "豆包AI生成" text strip in the bottom-right corner — the visible AIGC label mandated by China's TC260 standard. Detection matches the glyph silhouette against the corner (normalized correlation), so it keys on the "豆包AI生成" shape, not on textured corners. Removal is **localize then fill**: the light glyph blob is localized to a solid, dilated footprint mask (template-free, so a re-rendered or differently-placed mark is still masked) and the shared fill inpaints it. On corpus images the filled region blends into its surroundings within a few LAB levels, with no color shift and no dark pit. Pick the fill with `--backend` (cv2 default, MI-GAN or big-LaMa opt-in).
**Speed**: fast on CPU with the default cv2 fill, no GPU needed.
### Removing the Jimeng "★ 即梦AI" wordmark
Jimeng / Dreamina (即梦AI, also ByteDance, distinct from Doubao) stamps a "★ 即梦AI" wordmark — a four-point sparkle followed by the 即梦AI characters — in the bottom-right corner. Detection keys on the wordmark's glyph shape (NCC against a captured silhouette). `visible --mark auto` detects and removes it (or force it with `--mark jimeng`). Removal is **localize then fill**: the glyph blob is localized to a solid, dilated footprint mask and the shared fill inpaints it, so a re-rendered or differently-placed mark is still cleared. On corpus images the filled region blends into its surroundings within a few LAB levels, no colour shift. The two ByteDance marks do not confuse `auto`: detection keys on each mark's own glyph shape (the Jimeng detector scores far below its threshold on a Doubao strip, and vice versa).
Jimeng / Dreamina (即梦AI, also ByteDance, distinct from Doubao) stamps a "★ 即梦AI" wordmark — a four-point sparkle followed by the 即梦AI characters — in the bottom-right corner. Detection keys on the wordmark's glyph shape (NCC against a captured silhouette). `visible --mark auto` detects and removes it (or force it with `--mark jimeng`). Removal is **localize then fill**: the glyph blob is localized to a solid, dilated footprint mask and the shared fill inpaints it, so a re-rendered or differently-placed mark is still cleared. On corpus images the filled region blends into its surroundings within a few LAB levels, no color shift. The two ByteDance marks do not confuse `auto`: detection keys on each mark's own glyph shape (the Jimeng detector scores far below its threshold on a Doubao strip, and vice versa).
```bash
remove-ai-watermarks visible jimeng.png -o clean.png # --mark auto picks Jimeng
@@ -110,7 +110,7 @@ remove-ai-watermarks visible samsung.jpg --mark samsung -o clean.jpg
### Universal region eraser
For any visible mark the dedicated engines do not cover — a logo anywhere, any colour — `erase --region x,y,w,h` inpaints the box you specify. The default `cv2` backend is instant and dependency-free; the optional `lama` backend (big-LaMa via onnxruntime, `lama` extra, ~200 MB model downloaded on first use) gives much cleaner fills on textured regions at the cost of ~3-4 GB RAM per call.
For any visible mark the dedicated engines do not cover — a logo anywhere, any color — `erase --region x,y,w,h` inpaints the box you specify. The default `cv2` backend is instant and dependency-free; the optional `lama` backend (big-LaMa via onnxruntime, `lama` extra, ~200 MB model downloaded on first use) gives much cleaner fills on textured regions at the cost of ~3-4 GB RAM per call.
### Removing SynthID and other invisible watermarks
@@ -500,7 +500,7 @@ Won't fix:
## Limitations
- **Visible-mark removal is localized inpainting; metadata removal is lossless.** Each visible mark is localized to a footprint mask and filled by inpainting (cv2, MI-GAN, or big-LaMa), so only the small masked region is reconstructed and it blends into its surroundings within a few LAB levels; a slightly-off localization just fills a small region near-losslessly rather than leaving a colour-shifted smear. Metadata stripping never touches image data.
- **Visible-mark removal is localized inpainting; metadata removal is lossless.** Each visible mark is localized to a footprint mask and filled by inpainting (cv2, MI-GAN, or big-LaMa), so only the small masked region is reconstructed and it blends into its surroundings within a few LAB levels; a slightly-off localization just fills a small region near-losslessly rather than leaving a color-shifted smear. Metadata stripping never touches image data.
- **The invisible (SynthID) path is lossy and not guaranteed.** It runs a low-strength SDXL img2img regeneration, so it softens fine detail and is content-dependent. There is no public SynthID decoder, so the tool cannot verify removal locally; confirm with the Gemini app's "Verify with SynthID" oracle and raise `--strength` if it still detects. A vendor can change the scheme at any time, so treat this as an arms race, not a permanent fix.
- **Large images: native by default, opt-in tiling for OOM.** The SynthID path runs at the diffusion model's native resolution; on a memory-constrained GPU/MPS you can either cap the long side with `--max-resolution` (lossy downscale) or pass `--tile` to regenerate in overlapping, feather-blended tiles at native resolution (lossless, no seam). Tiling is a memory workaround, not a quality upgrade over a single native pass: each tile is an independent low-strength regeneration. (Nano Banana 2 is natively 1024px; GPT Image 2 supports 4K experimentally.)
- **Out of scope:** defeating trained AI-vs-real classifiers like Hive (see [Threat model](#threat-model)), visible-logo removal from video, and any guarantee that a stripped copy is untraceable server-side.
+1 -1
View File
@@ -20,7 +20,7 @@ pipelines, strength defaults, or metadata coverage.
`--tile` (OFF by default; `--tile-size` default 1024, `--tile-overlap` default 128) processes the diffusion pass in overlapping sliding-window tiles instead of one forward pass, so a large image is regenerated at **native resolution** without the OOM and without the lossy `--max-resolution` downscale round-trip. It engages only when the long side exceeds `--tile-size`; a sub-tile image runs a single pass unchanged. `WatermarkRemover.remove_watermark` refactors the single-image `_generate` into a per-tile `_generate_one` (the ControlNet canny edge map is rebuilt per tile, so structure preservation works tile-local) and routes it through `noai.tiling.run_tiled` when tiling is active. The geometry and blend math are pure helpers, unit-tested without the model (`tests/test_tiling.py`):
- `plan_tiles(w, h, tile_size, overlap)` lays out a row-major grid where every tile is exactly `tile_size` (the last tile on each axis is pulled back flush to the far edge, simply overlapping its predecessor more). Uniform tile size keeps each diffusion pass at SDXL's preferred dimension.
- `feather_weights(w, h, overlap)` is a separable linear taper, ~1 in the interior and ramping toward each edge, kept **strictly positive** so the normalised accumulate-and-divide blend (`accum / weight_sum`) is a partition of unity: a region covered by one feathered edge (an image corner) still divides cleanly. Identical (unchanged) tiles therefore reconstruct the input exactly -- the seam-free guarantee, asserted in `test_identity_generate_reconstructs_image`.
- `feather_weights(w, h, overlap)` is a separable linear taper, ~1 in the interior and ramping toward each edge, kept **strictly positive** so the normalized accumulate-and-divide blend (`accum / weight_sum`) is a partition of unity: a region covered by one feathered edge (an image corner) still divides cleanly. Identical (unchanged) tiles therefore reconstruct the input exactly -- the seam-free guarantee, asserted in `test_identity_generate_reconstructs_image`.
CAVEAT: each tile is an **independent** low-strength regeneration. At the certified removal strengths (0.20-0.30) the per-tile drift is small and the feather blend hides the seams, but tiling is a memory workaround, not a quality upgrade over a single native pass -- a 32 GB MPS box that clears the native UNet peak should prefer no tiling. The MPS->CPU fallback still applies per tile; if the first tile falls back to CPU, the device stays CPU for the rest of the image.
+8 -8
View File
@@ -61,7 +61,7 @@ module.
`watermark_registry.py`**single catalog of known visible watermarks**, the unified "find known marks in their usual places, recognize, remove" entry.
**Localize -> fill by policy (replaced reverse-alpha):** each mark is localized to a binary full-frame footprint mask (a `Localization`), and one shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). This replaced the old reverse-alpha removal (invert a captured alpha map, `original = (wm - a*logo)/(1-a)`, plus a thin residual inpaint) for ALL marks — gemini, doubao, jimeng, samsung, and jimeng_pill. **Why it changed:** reverse-alpha depended on a fixed captured alpha map at a fixed position, so it broke whenever a vendor moved or re-rendered its mark; and it was not colour-lossless even with the right map (it amplifies 8-bit quantization and JPEG-chroma error by `1/(1-a)`), which showed up as "the colour just changed, not removed" reports. Localize -> fill has a benign failure mode: a slightly-off localization just inpaints a small region near-losslessly instead of leaving a colour-shifted smear. The captured alpha maps are still used to DETECT the marks and to shape the mask (gemini's footprint), but NOT for pixel recovery. Fill backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the preferred default when the `migan` extra is installed), `lama` (big-LaMa ONNX, best quality, heavier, explicit opt-in); `auto` = MI-GAN if installed, else cv2. Each `KnownMark` ties a key to {usual `location`, `in_auto` flag, a `_detect` callable → uniform `MarkDetection`, a `_mask` callable → full-frame footprint mask}; `KnownMark.remove(image, *, backend="auto", provenance=False, force=False)`. Entries today: `gemini` (bottom-right sparkle), `doubao` (bottom-right "豆包AI生成"), `jimeng` (bottom-right "★ 即梦AI"), `samsung` (bottom-**LEFT** "✦ Contenuti generati dall'AI", Samsung Galaxy AI, Italian locale), and the capture-less `jimeng_pill` (top-left "AI生成"). `detect_marks(image, *, provenance=frozenset())` scans all (strict, for the identify verdict); `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` removes every detected mark in one pass. **Sensitivity (`auto`/`strict`/`assume_ai`)** decides how hard a borderline mark is trusted: the visual detectors are pixel-based (no metadata needed) and the recall gain comes from relaxing the false-positive gate, not from metadata. `resolve_relax` turns the policy + evidence into the per-mark relaxation boolean — `strict` never relaxes, `assume_ai` always relaxes (the caller asserts AI, e.g. a metadata-stripped screenshot; ~46% -> ~92% Gemini recall, at the cost of a small near-lossless fill on some clean corners), `auto` relaxes only on same-product evidence (metadata provenance for that vendor, or a confidently strict-detected sibling of the same `_PRODUCT_OF` — Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax). **Perception / decision / action are separated three ways** (the removal path only; `identify` keeps calling `KnownMark.detect` directly, so its verdict is untouched): `_build_candidates(image)` is PERCEPTION — it runs each detector at both trust levels and packages raw verdicts + the pill's flatness feature into `Candidate`s, no policy; `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` is the pure DECISION arbiter — all keep/drop policy (`resolve_relax` cross-mark corroboration + the pill gate) in one image-free, unit-testable function (`tests/test_watermark_registry.py::TestArbiter`); then `remove_auto_marks` does the ACTION, localizing -> filling each winner. The Gemini FP gate deliberately stays inside `gemini_engine` (not the arbiter) because `identify` reads that same gated confidence — pulling it out would drift the provenance verdict. Behavior is byte-identical to the pre-arbiter two-pass (corpus-verified: strict/auto 46%, assume_ai 92% Gemini recall unchanged).
**Localize -> fill by policy (replaced reverse-alpha):** each mark is localized to a binary full-frame footprint mask (a `Localization`), and one shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). This replaced the old reverse-alpha removal (invert a captured alpha map, `original = (wm - a*logo)/(1-a)`, plus a thin residual inpaint) for ALL marks — gemini, doubao, jimeng, samsung, and jimeng_pill. **Why it changed:** reverse-alpha depended on a fixed captured alpha map at a fixed position, so it broke whenever a vendor moved or re-rendered its mark; and it was not color-lossless even with the right map (it amplifies 8-bit quantization and JPEG-chroma error by `1/(1-a)`), which showed up as "the color just changed, not removed" reports. Localize -> fill has a benign failure mode: a slightly-off localization just inpaints a small region near-losslessly instead of leaving a color-shifted smear. The captured alpha maps are still used to DETECT the marks and to shape the mask (gemini's footprint), but NOT for pixel recovery. Fill backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the preferred default when the `migan` extra is installed), `lama` (big-LaMa ONNX, best quality, heavier, explicit opt-in); `auto` = MI-GAN if installed, else cv2. Each `KnownMark` ties a key to {usual `location`, `in_auto` flag, a `_detect` callable → uniform `MarkDetection`, a `_mask` callable → full-frame footprint mask}; `KnownMark.remove(image, *, backend="auto", provenance=False, force=False)`. Entries today: `gemini` (bottom-right sparkle), `doubao` (bottom-right "豆包AI生成"), `jimeng` (bottom-right "★ 即梦AI"), `samsung` (bottom-**LEFT** "✦ Contenuti generati dall'AI", Samsung Galaxy AI, Italian locale), and the capture-less `jimeng_pill` (top-left "AI生成"). `detect_marks(image, *, provenance=frozenset())` scans all (strict, for the identify verdict); `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` removes every detected mark in one pass. **Sensitivity (`auto`/`strict`/`assume_ai`)** decides how hard a borderline mark is trusted: the visual detectors are pixel-based (no metadata needed) and the recall gain comes from relaxing the false-positive gate, not from metadata. `resolve_relax` turns the policy + evidence into the per-mark relaxation boolean — `strict` never relaxes, `assume_ai` always relaxes (the caller asserts AI, e.g. a metadata-stripped screenshot; ~46% -> ~92% Gemini recall, at the cost of a small near-lossless fill on some clean corners), `auto` relaxes only on same-product evidence (metadata provenance for that vendor, or a confidently strict-detected sibling of the same `_PRODUCT_OF` — Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax). **Perception / decision / action are separated three ways** (the removal path only; `identify` keeps calling `KnownMark.detect` directly, so its verdict is untouched): `_build_candidates(image)` is PERCEPTION — it runs each detector at both trust levels and packages raw verdicts + the pill's flatness feature into `Candidate`s, no policy; `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` is the pure DECISION arbiter — all keep/drop policy (`resolve_relax` cross-mark corroboration + the pill gate) in one image-free, unit-testable function (`tests/test_watermark_registry.py::TestArbiter`); then `remove_auto_marks` does the ACTION, localizing -> filling each winner. The Gemini FP gate deliberately stays inside `gemini_engine` (not the arbiter) because `identify` reads that same gated confidence — pulling it out would drift the provenance verdict. Behavior is byte-identical to the pre-arbiter two-pass (corpus-verified: strict/auto 46%, assume_ai 92% Gemini recall unchanged).
**Provenance prior:** when local metadata already confirms the vendor, the mark's detection trust gate is relaxed (a confirmed vendor means the mark is present with high prior, so a mark the conservative detector would demote as a content false positive is trusted). `detect_marks` / `remove_auto_marks` take a `provenance` frozenset and `KnownMark.remove` a `provenance` flag. Mapping: a Google/Gemini C2PA issuer relaxes gemini (skips its false-positive gate and lowers the trust threshold from 0.5 to 0.35); a China-AIGC (TC260) label relaxes doubao/jimeng; `samsung_genai` relaxes samsung. Corpus finding: on Google-C2PA images, Gemini sparkle recall rose from ~46% (plain detector) to ~90% with the provenance prior (recovering marks the vendor moved or re-rendered). The localizer is cheap CPU (cv2/numpy), so a memory-tight caller runs it anywhere; the heavy MI-GAN/LaMa fill is opt-in and chosen by the caller.
@@ -85,7 +85,7 @@ module.
The cost (mislabel ~8-33% of non-Gemini content as Gemini) outweighs the benefit -- the visible sparkle is a medium-confidence stripped-metadata fallback, and intact Gemini is caught by C2PA in `identify` regardless. Remaining square misses are an accepted known limitation; a real fix would need a sparkle-specific discriminator (template match on a background-subtracted image, or a hard fixed-margin position prior), which is open research, not a threshold tweak.
**Removal is localize -> fill** (`footprint_mask``watermark_registry.fill`): `footprint_mask` returns the sparkle footprint = the captured alpha (computed `alpha = max(R,G,B)/255` from the bundled sparkle-on-black captures `assets/gemini_bg_{96,48}.png`, capture max ~130 for the ~51%-opaque overlay) thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin. That binary mask is inpainted by the shared fill (cv2 / MI-GAN / big-LaMa). The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. This replaced the old reverse-alpha removal path; because the fill only reconstructs the masked footprint from its surroundings (rather than dividing by `1-a`), the whole reverse-alpha removal tail — the over-subtraction guard (`_reverse_alpha_oversubtracts`, the dark-background black-pit fix), the under-subtraction alpha-gain estimate (`_estimate_alpha_gain`), and the self-verify repair — is GONE, along with the near-white `1/(1-a)` ill-conditioning and the "colour changed, not removed" failure mode those guards patched around. A slightly-off localization now just fills a small region near-losslessly instead of leaving a colour-shifted smear.
**Removal is localize -> fill** (`footprint_mask``watermark_registry.fill`): `footprint_mask` returns the sparkle footprint = the captured alpha (computed `alpha = max(R,G,B)/255` from the bundled sparkle-on-black captures `assets/gemini_bg_{96,48}.png`, capture max ~130 for the ~51%-opaque overlay) thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin. That binary mask is inpainted by the shared fill (cv2 / MI-GAN / big-LaMa). The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. This replaced the old reverse-alpha removal path; because the fill only reconstructs the masked footprint from its surroundings (rather than dividing by `1-a`), the whole reverse-alpha removal tail — the over-subtraction guard (`_reverse_alpha_oversubtracts`, the dark-background black-pit fix), the under-subtraction alpha-gain estimate (`_estimate_alpha_gain`), and the self-verify repair — is GONE, along with the near-white `1/(1-a)` ill-conditioning and the "color changed, not removed" failure mode those guards patched around. A slightly-off localization now just fills a small region near-losslessly instead of leaving a color-shifted smear.
**False-positive gate (added 2026-06-03):** `detect_watermark`'s shape-only NCC (`spatial*0.5 + gradient*0.3 + var*0.2`) fires on ornate/flat content (text strips, banners, hatching) that coincidentally matches the diamond shape — a real Gemini sparkle is a bright WHITE overlay, so its core sits above the local background, but the NCC is contrast-invariant and cannot see that. The fusion now **demotes** (caps confidence to 0.30) any low-confidence (`< _SPARKLE_FP_CONF` 0.65) match that shows NEITHER real-sparkle signature: a bright core (`_core_ring_margin >= _SPARKLE_FP_MARGIN` 5) OR a crisp star silhouette (`gradient_score >= _SPARKLE_FP_GRAD` 0.55). I.e. demote when `low_margin OR low_grad`. Real sparkles escape via high confidence (white-bg sparkles score ≥0.79 despite a low margin — the NCC shape match is strong), high margin (dark/mid backgrounds, incl. the #36 faint-corner case, lift well clear), OR high gradient (a real sparkle is grad ~0.971.0). **The gradient condition (added 2026-06-26) closes the bright-background FP class** the margin check alone missed: a snow+sky photo and a white-background product render both scored ~0.51 at `identify`, because a bright background gives the match a HIGH core-ring margin (it genuinely IS brighter than its surroundings), so the brightness gate read it as a real overlay — but a smooth luminance blob that shape-NCC-matches the rough diamond has low gradient fidelity (the two FPs measured grad 0.105 and 0.463 vs ≥0.8 for real sparkles), so the gradient floor demotes them. The OR is **strictly a superset** of the old margin-only demotion (it only ADDS demotions on bright backgrounds, where a real sparkle keeps grad ~0.97), so it cannot regress a dark/mid sparkle (kept by margin) or a white-bg one (kept by confidence ≥ 0.65). The gate is **monotonic** (only ever removes detections, never adds), so it cannot regress the verified-negative corpus (already 0 FPs); the 2026-06-26 corpus re-sweep flipped only OpenAI/ChatGPT content (no Gemini sparkle exists there) and already-`cleaned/` outputs, all sub-0.5 (below the `identify` threshold), so no provenance verdict changed. The original gate demoted 16/495 flagged sparkles on the validation corpus (13 carried no AI metadata = content FPs; the 3 AI-meta were visually FPs / a near-invisible white-on-white sparkle whose AI verdict is held by metadata anyway). `_core_ring_margin` uses the `_core_and_bg` helper (core 75th-pct brightness vs background-ring median). This gate is detection-side and unchanged by the localize -> fill refactor; the provenance prior skips it when a Google/Gemini C2PA issuer confirms the vendor. Regression-guarded by `test_gemini_engine.py::TestSparkleFalsePositiveGate` (incl. `test_bright_background_low_gradient_match_demoted`).
@@ -97,15 +97,15 @@ The cost (mislabel ~8-33% of non-Gemini content as Gemini) outweighs the benefit
`_text_mark_engine.py`**shared base for the three text-mark engines (Doubao/Jimeng/Samsung), extracted 2026-06-09** (they were ~90% byte-identical clones). `TextMarkEngine(config: TextMarkConfig)` owns the `locate → extract_mask → detect` detection pipeline plus the removal that localizes the glyph blob to a footprint mask and hands it to the shared `watermark_registry.fill` (+ the asset-keyed `load_alpha_template`/`glyph_silhouette`/`template_match_score` caches). Detection still matches the glyph silhouette (NCC against the captured template); the removal MASK is TEMPLATE-FREE — it is the bounding box of the top-hat glyph blob from `extract_mask`, filled solid + dilated, so a re-rendered or differently-placed mark is still masked. This dropped the fixed alpha-template placement; the captured alpha maps are now used only for the detection silhouette, not for removal. Each engine module is a thin subclass supplying only its `TextMarkConfig` (the tuned constants, the bundled asset, and the bounded structural deltas — `corner` br/bl, `margin_floor` 4/2, `morph_open_size` 5/3, `min_gw` 8/16) plus the test-facing module shims (`_alpha_template`/`_glyph_silhouette`/`_template_match_score` + the constants). Gemini stays a SEPARATE engine (its multi-size fixed-slot sparkle model is genuinely different). Add a new text mark = a new `TextMarkConfig` + a thin subclass + one registry `_text_mark(...)` row. The engine bullets below describe each mark's calibration history; the LOGIC lives here. **Small-image detection guard (`_MIN_DETECT_SHORT_SIDE` 200, added 2026-06-26):** `detect` returns not-detected when the image short side is below 200px. Below that the glyph template degrades to the `min_gw` floor (~8px) and `TM_CCOEFF_NORMED` on a few pixels is noise, so an unrelated small geometric shape can spuriously correlate with the CJK silhouette — a 48×48 app-icon chevron scored Doubao 0.41 / Jimeng 0.47 (both above their thresholds), a pure small-size artifact (the same icon upscaled collapses to ~0.060.10 NCC at ≥256px). A real AI-generation label is stamped on a full-resolution render (the captured samples are 10862048px wide, the smallest positive test image is 1086px), so the floor sits far below any genuine mark while killing the icon/thumbnail band (≤96px); `identify` falls back to "unknown" (the safe default) and removal, gated on detection, is suppressed too. Regression-guarded by `test_{doubao,jimeng,samsung}_engine.py::TestDetect::test_small_image_guarded_from_false_positive`.
**Removal is localize -> fill.** The engine localizes the glyph blob (`extract_mask` over the located box) into a solid, dilated footprint mask and hands it to the shared `watermark_registry.fill` (cv2 / MI-GAN / big-LaMa). The template-free mask (bounding box of the glyph blob, not the fixed alpha template) means a re-rendered or moved mark is still covered, and the fill reconstructs the box from its surroundings. On corpus images doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no colour shift, no dark pit); clean images with no vendor signature had 0% false removal.
**Removal is localize -> fill.** The engine localizes the glyph blob (`extract_mask` over the located box) into a solid, dilated footprint mask and hands it to the shared `watermark_registry.fill` (cv2 / MI-GAN / big-LaMa). The template-free mask (bounding box of the glyph blob, not the fixed alpha template) means a re-rendered or moved mark is still covered, and the fill reconstructs the box from its surroundings. On corpus images doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal.
**The reverse-alpha removal machinery is retired.** The old per-glyph reverse-alpha blend (`_apply_reverse_alpha`), the fixed/aligned alpha-map helpers, the over-subtraction guard (`_reverse_alpha_oversubtracts``_inpaint_footprint`, the dark-pit fix on dark/mid-tone backgrounds), and the always-align placement search are all gone — the fill reconstructs the footprint from its surroundings rather than inverting the captured alpha, so the dark-pit and colour-shift failure modes those guards patched around no longer arise. `extract_mask` still returns a box-sized (`(loc.h, loc.w)`) mask rather than a full frame, which keeps the memory-tight `identify` detect path cheap.
**The reverse-alpha removal machinery is retired.** The old per-glyph reverse-alpha blend (`_apply_reverse_alpha`), the fixed/aligned alpha-map helpers, the over-subtraction guard (`_reverse_alpha_oversubtracts``_inpaint_footprint`, the dark-pit fix on dark/mid-tone backgrounds), and the always-align placement search are all gone — the fill reconstructs the footprint from its surroundings rather than inverting the captured alpha, so the dark-pit and color-shift failure modes those guards patched around no longer arise. `extract_mask` still returns a box-sized (`(loc.h, loc.w)`) mask rather than a full frame, which keeps the memory-tight `identify` detect path cheap.
## `doubao_engine.py`
`doubao_engine.py`**a thin `_text_mark_engine.TextMarkEngine` subclass (config only) since 2026-06-09.** visible Doubao "豆包AI生成" detector + localizer (cv2/numpy, no GPU). `DoubaoEngine.locate` anchors a bottom-right box by **geometry** (mark scales with image WIDTH), `extract_mask` pulls the light, low-chroma glyphs (the detection candidate) using a per-pixel channel-spread proxy `sat = roi.max(axis=2) - roi.min(axis=2)` (no HSV conversion). `detect` is **shape-consistent**: it matches the bundled glyph silhouette (`assets/doubao_alpha.png`) against the candidate via zero-mean normalized correlation (`_template_match_score`, cv2 `TM_CCOEFF_NORMED`), gated at `DETECT_NCC_THRESHOLD` 0.4 over a small `DETECT_MIN_COVERAGE` floor. Keying on glyph SHAPE (not coverage heuristics) fixed #23 (corpus FP 7/1243).
**Removal is localize -> fill:** the glyph blob is localized to a solid, dilated footprint mask (`extract_mask` over the located box) and the shared `watermark_registry.fill` inpaints it. On corpus images this removes at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no colour shift, no dark pit).
**Removal is localize -> fill:** the glyph blob is localized to a solid, dilated footprint mask (`extract_mask` over the located box) and the shared `watermark_registry.fill` inpaints it. On corpus images this removes at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit).
**The detection template (`assets/doubao_alpha.png`) is rebuilt by `scripts/visible_alpha_solve.py`** (the careful gray-self solve: cubic background fit, mean over channels, full halo, unblurred), same recipe as Jimeng — the captures are committed in `data/doubao_capture/captures/`. It is used only as the detection silhouette, not for pixel recovery.
@@ -117,7 +117,7 @@ The cost (mislabel ~8-33% of non-Gemini content as Gemini) outweighs the benefit
**The detection template (`assets/jimeng_alpha.png`) is rebuilt by `scripts/visible_alpha_solve.py` from the GRAY capture** (`data/jimeng_capture/captures/`, the solid captures committed): `a = (I - B)/(255 - B)`, B a per-capture **cubic** background fit over the non-glyph pixels, **averaged over channels, full halo extent (down to a~0.02), unblurred**. Gray (bg ~132) is the deliberate choice over black: it is the best proxy for real content (the mark sits on bright photo areas, not on black). The captured template is used only as the detection silhouette, not for pixel recovery. Solver geometry at `_ALPHA_NATIVE_WIDTH` 2048: `_ALPHA_WIDTH_FRAC` 0.202, `_ALPHA_HEIGHT_FRAC` 0.058, margins ~0.029.
**Removal is localize -> fill:** the glyph blob is localized to a solid, dilated footprint mask and the shared `watermark_registry.fill` inpaints it; the `WM_*` locate box is generous so a re-rasterized, corner-ward-shifted mark stays inside the localized box (the same widen that fixed Doubao). On corpus images this removes at ~100% with clean footprints (blends within a few LAB levels, no colour shift). The registry gates removal on `detect`.
**Removal is localize -> fill:** the glyph blob is localized to a solid, dilated footprint mask and the shared `watermark_registry.fill` inpaints it; the `WM_*` locate box is generous so a re-rasterized, corner-ward-shifted mark stays inside the localized box (the same widen that fixed Doubao). On corpus images this removes at ~100% with clean footprints (blends within a few LAB levels, no color shift). The registry gates removal on `detect`.
**No committed real sample** (only the solid calibration captures are committed) — `tests/test_jimeng_engine.py` synthesizes a mark from the bundled template, and `test_recovers_shifted_mark_on_texture` guards the localize-on-shift path that the Doubao defect exposed. Jimeng images are independently caught by the China TC260 AIGC label in `metadata`/`identify`, so this engine is the visible-mark *removal* path, not a new `identify` signal.
@@ -177,7 +177,7 @@ At the shared low removal strength the canny edge-conditioning keeps the regener
## `noai/tiling.py`
Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). `plan_tiles(w, h, tile_size, overlap)` returns a row-major grid of uniform-size `Tile` boxes — every tile is exactly `tile_size` (the SDXL training size), with the last tile on each axis pulled back flush to the far edge (`_axis_positions` clamps a pathological `overlap >= tile` to `tile - 1` so the step stays >= 1). `feather_weights(w, h, overlap)` is a separable linear taper (1 in the interior, ramping toward each edge) floored at `_WEIGHT_EPS` so it is **strictly positive everywhere** — that makes the normalised `accum / weight_sum` blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). `run_tiled(generate_tile, image, tile_size, overlap, set_progress)` is the orchestration loop: crop each planned tile, call `generate_tile` (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (`tests/test_tiling.py`: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). `plan_tiles(w, h, tile_size, overlap)` returns a row-major grid of uniform-size `Tile` boxes — every tile is exactly `tile_size` (the SDXL training size), with the last tile on each axis pulled back flush to the far edge (`_axis_positions` clamps a pathological `overlap >= tile` to `tile - 1` so the step stays >= 1). `feather_weights(w, h, overlap)` is a separable linear taper (1 in the interior, ramping toward each edge) floored at `_WEIGHT_EPS` so it is **strictly positive everywhere** — that makes the normalized `accum / weight_sum` blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). `run_tiled(generate_tile, image, tile_size, overlap, set_progress)` is the orchestration loop: crop each planned tile, call `generate_tile` (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (`tests/test_tiling.py`: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
`feather_region_composite(base, regenerated, box, *, feather)` is the pure region-targeted compositor for **AI-enhanced composites** (roadmap P1#8; `identify` `ai_source_kind == "enhanced"`, digitalSourceType `compositeWithTrainedAlgorithmicMedia`). It blends `regenerated` over `base` inside `box = (x, y, w, h)` with a separable linear taper of `feather` px at the box edges (the taper anchors to ~0 at the boundary, so unlike `feather_weights` it is NOT floored — the result equals `base` EXACTLY outside the box), preserving dtype and supporting HxW or HxWxC. It backs `WatermarkRemover.remove_watermark(region=..., region_feather=...)`: the remover regenerates the frame (or tiles), then composites only the AI box back over the original input, so the real photo outside the box stays pixel-exact and only the AI region is scrubbed. The box is caller-supplied (a C2PA composite manifest carries no reliable machine-readable region); the no-model lossless region path remains `region_eraser.erase`. Unit-tested in `tests/test_tiling.py::TestFeatherRegionComposite` (outside-box exactness, interior == regenerated, hard-paste at feather 0, monotonic seam ramp, dtype/grayscale/clamp/empty-box/shape-mismatch).
@@ -228,7 +228,7 @@ Diffusion SynthID removal. The `--tile/--no-tile` knob is the *lossless* alterna
### `visible`
Known-visible-mark removal by **localize -> fill**: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. `--backend auto|cv2|migan|lama` (default `auto`) picks the fill: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, the preferred default when the `migan` extra is installed), `lama` (big-LaMa ONNX, best quality, heavier, explicit opt-in); `auto` = MI-GAN if installed, else cv2 (big-LaMa is NOT auto-selected). `--sensitivity auto|strict|assume-ai` (default `auto`) controls how hard a borderline mark is trusted (see the registry section: the visual detectors are metadata-independent; `auto` relaxes a mark only on same-product evidence, `assume-ai` relaxes every mark on the caller's AI assertion — the only path to high recall on a metadata-stripped screenshot). `--backend` and `--sensitivity` are shared across `visible`/`all`/`batch`. Detection keys on each mark's own shape, and under `auto` the trust gate is relaxed when local metadata confirms the vendor (a Google/Gemini C2PA issuer relaxes gemini, a China-AIGC label relaxes doubao/jimeng, `samsung_genai` relaxes samsung), so a moved or re-rendered mark is still caught. `--mark auto` (default) removes EVERY detected mark in one pass (`registry.remove_auto_marks`, not the single strongest -- a Jimeng-basic image carries both the top-left pill and the bottom-right wordmark) from: the Gemini sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-LEFT, Italian-locale detection), and the capture-less Jimeng "AI生成" pill (top-left, `pill_engine`). The pill's weak edge-NCC detector is gated in `remove_auto_marks` via `_keep_pill` (32k real-upload corpus validation 2026-07): never on Doubao, and two confirmation arms since metadata confirms the platform, not pill presence. (1) The bottom-right wordmark fired — ~94% precise and survives metadata-STRIPPED uploads (screenshots / re-saves) — removes the pill unrestricted. (2) TC260 metadata confirms Jimeng (`"jimeng" in provenance`, from `cli._visible_provenance`) OR the caller asserts AI (`sensitivity == "assume_ai"`), no wordmark — ~27% precise, its false fires are textured ceilings/walls that the fill visibly SMEARS — removes the pill ONLY when the top-left footprint is flat enough for an invisible fill (`pill_engine.footprint_is_flat`, median-Sobel ≤ `_FLAT_TEXTURE_MAX`; the flatness guard holds even under `assume_ai`). No confirmation → never removed. `--mark gemini|doubao|jimeng|samsung|jimeng_pill` forces one (choices come from the registry). Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no colour shift, no dark pit); clean images with no vendor signature had 0% false removal. For arbitrary logos/objects use `erase`. **When `--mark auto` finds no known mark (the common case — ~74% of real uploads carry no registered visible mark), the command does NOT silently re-serve the input as a finished result.** It runs a cheap metadata-only `identify`, prints actionable guidance (if the image carries an invisible/metadata mark, e.g. an OpenAI/Gemini C2PA image, it points to `all`; otherwise it does NOT imply the image is clean -- it warns that an invisible pixel watermark like SynthID cannot be detected once the metadata proxy is gone and routes to both `all` and `erase --region`), writes NO output file, and exits **`EXIT_NO_VISIBLE_MARK` (2)** — distinct from success (0) and a hard error (1) so a wrapping service (raiw.cc) can surface the message instead of treating the unchanged image as done (the production "it didn't work" / score-0 trap). Same handling for an explicit `--mark <name>` that is not detected. Helper `cli._no_visible_mark_exit`; regression-guarded by `tests/test_cli.py::TestVisibleCommand::test_visible_auto_no_mark_exits_two_with_eraser_hint` and `test_visible_auto_no_mark_routes_to_all_when_metadata`. `--no-detect` still forces the gemini fallback and proceeds (exit 0).
Known-visible-mark removal by **localize -> fill**: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. `--backend auto|cv2|migan|lama` (default `auto`) picks the fill: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, the preferred default when the `migan` extra is installed), `lama` (big-LaMa ONNX, best quality, heavier, explicit opt-in); `auto` = MI-GAN if installed, else cv2 (big-LaMa is NOT auto-selected). `--sensitivity auto|strict|assume-ai` (default `auto`) controls how hard a borderline mark is trusted (see the registry section: the visual detectors are metadata-independent; `auto` relaxes a mark only on same-product evidence, `assume-ai` relaxes every mark on the caller's AI assertion — the only path to high recall on a metadata-stripped screenshot). `--backend` and `--sensitivity` are shared across `visible`/`all`/`batch`. Detection keys on each mark's own shape, and under `auto` the trust gate is relaxed when local metadata confirms the vendor (a Google/Gemini C2PA issuer relaxes gemini, a China-AIGC label relaxes doubao/jimeng, `samsung_genai` relaxes samsung), so a moved or re-rendered mark is still caught. `--mark auto` (default) removes EVERY detected mark in one pass (`registry.remove_auto_marks`, not the single strongest -- a Jimeng-basic image carries both the top-left pill and the bottom-right wordmark) from: the Gemini sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-LEFT, Italian-locale detection), and the capture-less Jimeng "AI生成" pill (top-left, `pill_engine`). The pill's weak edge-NCC detector is gated in `remove_auto_marks` via `_keep_pill` (32k real-upload corpus validation 2026-07): never on Doubao, and two confirmation arms since metadata confirms the platform, not pill presence. (1) The bottom-right wordmark fired — ~94% precise and survives metadata-STRIPPED uploads (screenshots / re-saves) — removes the pill unrestricted. (2) TC260 metadata confirms Jimeng (`"jimeng" in provenance`, from `cli._visible_provenance`) OR the caller asserts AI (`sensitivity == "assume_ai"`), no wordmark — ~27% precise, its false fires are textured ceilings/walls that the fill visibly SMEARS — removes the pill ONLY when the top-left footprint is flat enough for an invisible fill (`pill_engine.footprint_is_flat`, median-Sobel ≤ `_FLAT_TEXTURE_MAX`; the flatness guard holds even under `assume_ai`). No confirmation → never removed. `--mark gemini|doubao|jimeng|samsung|jimeng_pill` forces one (choices come from the registry). Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. For arbitrary logos/objects use `erase`. **When `--mark auto` finds no known mark (the common case — ~74% of real uploads carry no registered visible mark), the command does NOT silently re-serve the input as a finished result.** It runs a cheap metadata-only `identify`, prints actionable guidance (if the image carries an invisible/metadata mark, e.g. an OpenAI/Gemini C2PA image, it points to `all`; otherwise it does NOT imply the image is clean -- it warns that an invisible pixel watermark like SynthID cannot be detected once the metadata proxy is gone and routes to both `all` and `erase --region`), writes NO output file, and exits **`EXIT_NO_VISIBLE_MARK` (2)** — distinct from success (0) and a hard error (1) so a wrapping service (raiw.cc) can surface the message instead of treating the unchanged image as done (the production "it didn't work" / score-0 trap). Same handling for an explicit `--mark <name>` that is not detected. Helper `cli._no_visible_mark_exit`; regression-guarded by `tests/test_cli.py::TestVisibleCommand::test_visible_auto_no_mark_exits_two_with_eraser_hint` and `test_visible_auto_no_mark_routes_to_all_when_metadata`. `--no-detect` still forces the gemini fallback and proceeds (exit 0).
### `batch`
+2 -2
View File
@@ -10,9 +10,9 @@
**Conclusion (historical): pure reverse-alpha distilled from content images does NOT work, and the blocker is the WRONG kind of data, not too little of it.**
The earlier framing ("need ~5-8 PRISTINE same-resolution originals") is obsolete -- a local corpus of pristine originals holds plenty. Curate them with `DoubaoEngine.detect` + an NCC filter against a clean glyph template, keeping only marks at offset ≈ (0,0): that yields e.g. **15 pixel-aligned 2048² marks** (sub-pixel drift, not the ±50 px the old lossy/mixed-res scrapes had), plus 1086x1448 / 1792x2400 clusters. With those, LaMa-clean `O` + weighted-LS (and per-pixel I-on-O regression) for `α` (+ logo colour) was tried end-to-end and **still leaves a persistent ghost outline.**
The earlier framing ("need ~5-8 PRISTINE same-resolution originals") is obsolete -- a local corpus of pristine originals holds plenty. Curate them with `DoubaoEngine.detect` + an NCC filter against a clean glyph template, keeping only marks at offset ≈ (0,0): that yields e.g. **15 pixel-aligned 2048² marks** (sub-pixel drift, not the ±50 px the old lossy/mixed-res scrapes had), plus 1086x1448 / 1792x2400 clusters. With those, LaMa-clean `O` + weighted-LS (and per-pixel I-on-O regression) for `α` (+ logo color) was tried end-to-end and **still leaves a persistent ghost outline.**
Diagnosed why, empirically (cached stacks, `/tmp/doubao_distill`): (1) the mark is a clean white overlay with **no dark halo** -- over glyph pixels ~54% are brighter than the clean bg, only ~4% darker -- so the white-logo model `I=(1-α)O+α·255` is correct; (2) but content backgrounds are almost never dark *under* the mark (median darkest available bg over glyph pixels = **58/255**; only ~13% of mark pixels are ever observed on a bg < 40), so on bright backgrounds the equation is ill-conditioned and `α` is unidentifiable; (3) LaMa's `O` is a plausible **hallucination**, not the true pre-mark background, which compounds the error, and per-pixel regression on ~15 obs overfits into colour noise.
Diagnosed why, empirically (cached stacks, `/tmp/doubao_distill`): (1) the mark is a clean white overlay with **no dark halo** -- over glyph pixels ~54% are brighter than the clean bg, only ~4% darker -- so the white-logo model `I=(1-α)O+α·255` is correct; (2) but content backgrounds are almost never dark *under* the mark (median darkest available bg over glyph pixels = **58/255**; only ~13% of mark pixels are ever observed on a bg < 40), so on bright backgrounds the equation is ill-conditioned and `α` is unidentifiable; (3) LaMa's `O` is a plausible **hallucination**, not the true pre-mark background, which compounds the error, and per-pixel regression on ~15 obs overfits into color noise.
**Why Gemini's engine is clean (verified in GeminiWatermarkTool `src/core/watermark_engine.cpp`): its alpha map is the watermark stamped on a PURE-BLACK background**, where `watermarked = α·255 + (1-α)·0 = α·255`, so `alpha = capture/255` exactly -- no estimation. (`gemini_bg_*.png` is literally the sparkle in grey on black.) So the real Doubao unlock is the same controlled capture, **not more content images**. Black/white/gray seeds exist (`data/doubao_capture/seeds/seed_*_1x1_2048x2048.png`); a capture run (feed a black seed through doubao.com edit mode, download the *original*) was requested from the #13 reporter 2026-05-29. With ~2-3 black captures we get `α = capture/255` for free, Gemini-quality.
+1 -1
View File
@@ -699,7 +699,7 @@ def cmd_erase(
"""Erase arbitrary region(s) from an image via inpainting.
Universal and position-agnostic: removes any logo / watermark / object inside
the boxes you pass, regardless of colour or location. Runs on CPU. Use this
the boxes you pass, regardless of color or location. Runs on CPU. Use this
for marks the dedicated ``visible`` engines (Gemini, Doubao) do not cover.
"""
from remove_ai_watermarks.region_eraser import erase
+1 -1
View File
@@ -81,7 +81,7 @@ def feather_weights(width: int, height: int, overlap: int) -> NDArray[Any]:
Separable linear taper over ``overlap`` pixels from every edge (capped at
half the tile so short tiles still taper symmetrically). Strictly positive
everywhere, so the normalised blend is well-defined even at an image corner
everywhere, so the normalized blend is well-defined even at an image corner
that only one tile covers.
"""
import numpy as np
+2 -2
View File
@@ -2,7 +2,7 @@
Position- and content-agnostic. You supply the rectangle(s); the eraser inpaints
whatever is inside, so it removes any visible logo / watermark / object regardless
of colour, style, or location. Localisation is the user's responsibility (pass the
of color, style, or location. Localization is the user's responsibility (pass the
box); restoration runs on CPU. This is the universal fallback for marks the
deterministic per-generator engines (Gemini sparkle, Doubao) do not cover.
@@ -151,7 +151,7 @@ def erase_lama(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]:
crop_mask = mask[cy0:cy1, cx0:cx1]
ch, cw = crop.shape[:2]
# Resize crop + mask to the model size, normalise to [0,1] RGB CHW.
# Resize crop + mask to the model size, normalize to [0,1] RGB CHW.
crop_rs = cv2.resize(crop, (size, size), interpolation=cv2.INTER_AREA)
mask_rs = cv2.resize(crop_mask, (size, size), interpolation=cv2.INTER_NEAREST)
img_in = cv2.cvtColor(crop_rs, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
+4 -11
View File
@@ -11,7 +11,7 @@ that mask to ONE shared, swappable fill backend (``region_eraser``: cv2 Telea/NS
MI-GAN, or big-LaMa). No mark carries a reverse-alpha step any more: the old
``original = (wm - a*logo)/(1-a)`` recovery depended on a fixed captured alpha map
at a fixed position, broke whenever a vendor re-rendered or moved its mark, and was
not colour-lossless even with the right map (it amplifies quantization/JPEG-chroma
not color-lossless even with the right map (it amplifies quantization/JPEG-chroma
error by ``1/(1-a)`` -- the "the color just changed, not removed" reports). The
localizer stays cheap (cv2/numpy, CPU) so a memory-tight caller can run it on a
small worker; the heavy fill (MI-GAN / LaMa) is opt-in and chosen by the caller.
@@ -123,11 +123,8 @@ class Candidate:
key: str
label: str
location: str
region: Region
detected_strict: bool
detected_relaxed: bool
confidence: float
features: dict[str, float] # generic; both construction sites always supply it (empty when none)
@@ -279,7 +276,7 @@ def inpaint_model_available() -> bool:
return region_eraser.migan_available() or region_eraser.lama_available()
def preferred_inpaint_backend() -> str:
def preferred_inpaint_backend() -> Literal["migan", "cv2"]:
"""Backend used by the ``auto`` fill: MI-GAN (light, droplet-friendly, the
default) when its ONNX runtime is available, else cv2. big-LaMa is NOT auto-
selected -- it is a heavier explicit opt-in via ``--backend lama`` (both models
@@ -293,7 +290,7 @@ def preferred_inpaint_backend() -> str:
def resolve_backend(backend: Backend) -> Literal["cv2", "migan", "lama"]:
"""Resolve ``auto`` to the preferred installed backend; pass the rest through."""
if backend == "auto":
return "migan" if preferred_inpaint_backend() == "migan" else "cv2"
return preferred_inpaint_backend()
return backend
@@ -494,11 +491,7 @@ def _build_candidates(image: NDArray[Any]) -> list[Candidate]:
strict = m.detect(image, provenance=False)
relaxed = m.detect(image, provenance=True)
feats = m.features(image) if (strict.detected or relaxed.detected) else {}
cands.append(
Candidate(
m.key, m.label, m.location, strict.region, strict.detected, relaxed.detected, strict.confidence, feats
)
)
cands.append(Candidate(m.key, m.label, strict.detected, relaxed.detected, feats))
return cands
+1 -1
View File
@@ -176,7 +176,7 @@ class TestArbiter:
@staticmethod
def _c(key, *, strict=False, relaxed=False, flat=False):
feats = {"footprint_flat": 1.0} if flat else {}
return reg.Candidate(key, f"L:{key}", "loc", (0, 0, 1, 1), strict, relaxed, 0.6, feats)
return reg.Candidate(key, f"L:{key}", strict, relaxed, feats)
def _keys(self, cands, ctx):
return {d.candidate.key for d in reg.decide(cands, ctx)}