diff --git a/CLAUDE.md b/CLAUDE.md index 3eb8c02..09d79c4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -61,7 +61,7 @@ Compact map. The full per-module detail (design decisions, tuned thresholds, cal - `_text_mark_engine.py` — shared base for the three text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. Detection still matches the glyph silhouette (NCC, keys on glyph shape). The removal mask is TEMPLATE-FREE: it is the bounding box of the top-hat glyph blob (`extract_mask`), filled solid + dilated, so the shared fill inpaints the whole wordmark rectangle. This drops the fixed alpha-template placement, so a re-rendered or differently-placed mark is still masked; the captured alpha maps are now used only for the detection silhouette, not for removal. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model). - `pill_engine.py` — the CAPTURE-LESS Jimeng-basic "AI生成" pill (top-left, issue #54). No alpha map: `detect` is edge-NCC of a synthetic font-rendered silhouette (`assets/jimeng_pill.png`, regenerate via `scripts/render_pill_silhouette.py`; committed, data-safe -- corpus stays out of the repo) in the top-left ROI, calibrated on 61 local real positives to threshold 0.22; `footprint_mask` is a generous FIXED top-left geometry box (NOT the NCC match position -- the synthetic silhouette localizes only approximately, the corner is negative space, so a geometry box fills cleanly while a match box leaves outline residue). `footprint_texture`/`footprint_is_flat` (median-Sobel over that box, `_FLAT_TEXTURE_MAX`) back the metadata-only safe-fill gate. Removal is the shared localize -> fill (MI-GAN/cv2). Detector precision is weak (~7% raw false-fire), so it is registry-gated in `remove_auto_marks` via `_keep_pill`: never on Doubao; the bottom-right wordmark removes it unrestricted (~94% precise, survives metadata-STRIPPED uploads); TC260-metadata-only removes it ONLY on a flat footprint (its textured false fires -- ceilings/walls -- are what the fill smears). Do NOT loosen those gates. - `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Detection matches the glyph silhouette (NCC); removal localizes the glyph blob to a solid dilated box (`extract_mask`) and hands it to the shared fill. Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. **Samsung detection is calibrated only for the Italian "Contenuti generati dall'AI" string** (a pre-existing limit, unchanged by the localize -> fill refactor but now surfaced because detection gates removal): non-Italian Samsung locales are not detected, and thus not removed, even though the fill mask itself is locale-independent; other locales need their own captured detection template. -- `region_eraser.py` — universal region eraser (`erase` CLI) and the shared fill backend behind `watermark_registry.fill` for the visible localize -> fill removal. Three backends: `cv2` (default, no deps, the floor), `migan` (MI-GAN ONNX, extra `migan`, MIT, ~28 MB / ~0.95 GB peak / ~0.19 s — the droplet-friendly tier, **the preferred default fill** when the extra is installed), `lama` (big-LaMa ONNX, extra `lama`, ~200 MB / ~4.7 GB peak — best quality, does not fit a minimal droplet, explicit opt-in only). **MI-GAN mask polarity is INVERTED** (0=hole/255=known) vs this package's 255-erase convention; `erase_migan` inverts before feeding the model (feeding 255=hole regenerates the whole frame into stripes — corpus-validated). Both ONNX models download on first use, never bundled. The `erase` command keeps its own `--backend`/`--inpaint-method` (unchanged). +- `region_eraser.py` — universal region eraser (`erase` CLI) and the shared fill backend behind `watermark_registry.fill` for the visible localize -> fill removal. Three backends: `cv2` (default, no deps, the floor), `migan` (MI-GAN ONNX, extra `migan`, MIT, ~28 MB / ~0.19 s — the droplet-friendly tier, **the preferred default fill** when the extra is installed), `lama` (big-LaMa ONNX, extra `lama`, ~200 MB / ~4.7 GB peak — best quality, does not fit a minimal droplet, explicit opt-in only). Both `migan` and `lama` **crop a padded region around the mask** before inference and paste only masked pixels back, so peak RAM is bounded by the MARK size, not the image (`migan` ~0.6-0.9 GB regardless of upload size — feeding the whole frame scaled it to ~2.4 GB at 25 MP; `migan` feeds the crop at native resolution, `lama` resizes to its fixed 512²). **MI-GAN mask polarity is INVERTED** (0=hole/255=known) vs this package's 255-erase convention; `erase_migan` inverts before feeding the model (feeding 255=hole regenerates the whole frame into stripes — corpus-validated). Both ONNX models download on first use, never bundled. The `erase` command keeps its own `--backend`/`--inpaint-method` (unchanged). - `invisible_watermark.py` — decodes the OPEN DWT-DCT watermarks (SD / SDXL / FLUX) via `imwatermark` (extra `detect`, pulls torch). Fragile two ways: (1) does not survive JPEG re-encode/resize; (2) **carrier-fragile on a broad class of pristine images** -- a clean encode->decode round-trip recovers 48/48 on chatgpt/firefly/random but FAILS (28-39/48, below the `_MATCH_48`=44 gate) on the FLUX fox, doubao, a flat FLUX generation, AND a clean synthetic flat fill with no watermark. The failure does NOT track texture; it goes with a degenerate **all-ones decode that is a CARRIER ARTIFACT, not a watermark** (synthetic clean image reproduces it). So `detect_invisible_watermark` is **positive-only**: trust a hit; a `None` is inconclusive unless a same-carrier positive-control embed first recovers >=44. Verified 2026-06-19; full caveat in `docs/watermarking-landscape.md`. - `trustmark_detector.py` — Adobe TrustMark open decoder (extra `trustmark`). Do NOT remove the JPEG re-encode false-positive gate — a lone TrustMark hit without it is almost always content noise. - `noai/watermark_remover.py` — `WatermarkRemover` with three diffusion pipelines selected by the explicit `pipeline` ctor arg, never inferred from `model_id`: `sdxl` (plain SDXL img2img), `controlnet` (SDXL + canny ControlNet, **the DEFAULT since 2026-06-09**), and `qwen` (Qwen-Image 20B MMDiT img2img, Apache-2.0, CUDA/cloud-class — best **text** preservation (incl. CJK); `_load_qwen_pipeline`/`_run_qwen`, bf16, no MPS fallback; call shape in the pure `_build_qwen_kwargs` using `true_cfg_scale`). Removal comes from the img2img `strength`; ControlNet only preserves text/face STRUCTURE — SynthID CAN survive controlnet on photoreal content at low strength. Qwen CERTIFIED oracle floors (2026-06-20): OpenAI **0.10** (seed-robust, clean on seeds 0-4), Gemini **0.25** (seed 0 verified, pin a seed — Gemini oracle rate-limits volume; higher than the controlnet Gemini floor 0.15). `resolve_strength(..., pipeline="qwen")` carries the Qwen ladder (`_QWEN_VENDOR_STRENGTH`), so `--pipeline qwen` gets the 0.25 Gemini floor automatically (the old manual `--strength 0.25` workaround is retired). `_build_qwen_kwargs` passes an explicit `height`/`width` from the input (floored to /16 via `_qwen_target_size`) — without it the pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (fixed 2026-06-20). **`qwen` is a MANUAL opt-in only — there is NO auto-router.** Measured (`scripts/fidelity_metrics.py`, OCR-CER / ArcFace / LPIPS / Laplacian-var, NOT eyeball): qwen beats controlnet on ONE niche only — **clean body text on a plain background, no faces** (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES (it always has) AND **display/decorative text in a scene** (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes, qwen re-renders and garbles them). So a content `--pipeline auto` router and a faces+text **mixed dual-pass** were prototyped and **DROPPED** (2026-06-20): on the canonical faces+text case controlnet wins every metric incl. text, so mixed loses; and "text→qwen" can't be auto-decided (it is body-vs-display text that matters, undetectable cheaply). qwen stays for callers who KNOW their content is clean-text-heavy and face-free. No face-restore extra ships, by validated decision (every restore approach looked MORE AI-generated). `remove_watermark(region=(x,y,w,h), region_feather=...)` runs the regeneration but feather-composites only the AI box back over the original (via `noai/tiling.feather_region_composite`), preserving the real photo elsewhere — the **AI-enhanced composite** path (`identify` `ai_source_kind == "enhanced"`); the box is supplied by the caller (a C2PA composite manifest carries no reliable machine-readable region, so we do not fabricate one). diff --git a/docs/module-internals.md b/docs/module-internals.md index cc211c0..491f404 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -139,7 +139,7 @@ The cost (mislabel ~8-33% of non-Gemini content as Gemini) outweighs the benefit `region_eraser.py` — universal region eraser (`erase` CLI) AND the shared fill backend behind `watermark_registry.fill` for the visible localize -> fill removal. `erase(image, boxes=|mask=, backend=)` accepts grayscale (2D) and RGBA (4-channel) inputs on **all** backends (each splits off any alpha plane and re-attaches it unchanged, and promotes grayscale to BGR): `boxes_to_mask` → one of three backends. - `cv2` (default, no deps): `cv2.inpaint`. -- `migan` (extra `migan`, `andraniksargsyan/migan` ONNX, MIT, ~28 MB): `erase_migan`. The MI-GAN ONNX crops around the mask bbox and re-composites internally, so the FULL image is fed at native resolution; only masked pixels are pasted back. **Mask polarity is INVERTED** vs this package's 255-erase convention — the shipped ONNX wants 0=hole / 255=known, so `erase_migan` feeds `(mask<=127)*255`; feeding 255=hole regenerates the whole frame into stripes (corpus-validated 2026-07, cost hours to find). ~0.95 GB peak / ~0.19 s. This is the **preferred default fill** for the visible localize -> fill path. +- `migan` (extra `migan`, `andraniksargsyan/migan` ONNX, MIT, ~28 MB): `erase_migan`. Like `erase_lama`, it crops a padded region around the mask (`pad = max(256, 2*bbox)`), feeds only that crop to the ONNX model, and pastes only masked pixels back — but since MI-GAN accepts arbitrary dims (unlike LaMa's fixed 512² square) the crop is fed at NATIVE resolution (no resize). This **bounds the ONNX working set by the mark size, not the image**: feeding the whole frame made peak RAM scale with the upload (~0.6 GB at 4 MP up to ~2.4 GB at 25 MP, measured 2026-07); cropping holds it roughly constant (~0.6-0.9 GB), so a memory-tight host (a 1-2 GB web worker) can run MI-GAN on a 25 MP upload. The crop does not degrade the fill — a small mark only needs local context, and on real marks the cropped fill is on par with / sometimes cleaner than the full-frame fill (a tighter view gives the GAN less room to hallucinate large background structure; verified by eye on real Gemini/Doubao marks + a ground-truth reconstruction sweep). **Mask polarity is INVERTED** vs this package's 255-erase convention — the shipped ONNX wants 0=hole / 255=known, so `erase_migan` feeds `(crop_mask<=127)*255`; feeding 255=hole regenerates the whole frame into stripes (corpus-validated 2026-07, cost hours to find). ~0.19 s. This is the **preferred default fill** for the visible localize -> fill path. - `lama` (extra `lama`, `Carve/LaMa-ONNX` Apache-2.0, ~200 MB): `erase_lama` crops a padded region around the mask, runs at LaMa's fixed 512² input, pastes only masked pixels back. Best quality but ~4.7 GB peak — explicit opt-in only, NOT auto-selected. Lazy `_get_{lama,migan}_session` singletons; `{lama,migan}_available()` guard the optional imports (both == onnxruntime present). Note both extras install the same onnxruntime, so the two `*_available()` checks are identical — the fill's `auto` backend therefore resolves to MI-GAN whenever onnxruntime is present, else cv2, and big-LaMa is reachable only by an explicit `lama` backend (`--backend lama` on `erase`, or the shared fill's `backend="lama"`). diff --git a/src/remove_ai_watermarks/region_eraser.py b/src/remove_ai_watermarks/region_eraser.py index 34be0be..bba7aee 100644 --- a/src/remove_ai_watermarks/region_eraser.py +++ b/src/remove_ai_watermarks/region_eraser.py @@ -10,9 +10,12 @@ Backends: - ``cv2`` (default): ``cv2.inpaint`` (Telea / Navier-Stokes). Instant, no extra dependencies, lower quality on large or textured regions. - ``migan`` (optional, extra ``migan``): MI-GAN via onnxruntime - (``andraniksargsyan/migan``, MIT). CPU, ~28 MB model, ~700-950 MB peak RAM, - ~0.19 s/call -- the droplet-friendly tier: near-big-LaMa quality on small - marks at ~5x less RAM and ~8x faster. Model downloaded on first use. + (``andraniksargsyan/migan``, MIT). CPU, ~28 MB model, ~0.19 s/call -- the + droplet-friendly tier: near-big-LaMa quality on small marks. Model downloaded + on first use. Like ``lama`` it crops a padded region around the mask before + inference (at native resolution -- MI-GAN takes arbitrary dims), so peak RAM is + bounded by the mark size (~0.6-0.9 GB) rather than scaling with the image, which + is what lets a memory-tight host run it on a large upload. - ``lama`` (optional, extra ``lama``): big-LaMa via onnxruntime (``Carve/LaMa-ONNX``, Apache-2.0). CPU, resolution-robust, best quality on texture but ~200 MB model and ~4.7 GB peak RAM (too heavy for a small host). @@ -69,6 +72,26 @@ def boxes_to_mask( return mask +def _padded_crop_box( + mask: NDArray[Any], h: int, w: int, *, pad_frac: float, pad_min: int +) -> tuple[int, int, int, int] | None: + """Bounding box of the set mask pixels, padded and clamped to the image. + + Returns ``(x0, y0, x1, y1)`` or ``None`` when the mask is empty. Both learned + backends crop to this box so the ONNX working set is bounded by the mark size + rather than the whole image; ``pad_frac``/``pad_min`` tune how much surrounding + context the inpainter sees (LaMa then resizes the crop to its fixed square, + MI-GAN feeds it at native resolution). + """ + ys, xs = np.where(mask > 0) + if len(xs) == 0: + return None + pad = max(pad_min, int(pad_frac * max(xs.max() - xs.min() + 1, ys.max() - ys.min() + 1))) + x0, y0 = max(0, int(xs.min()) - pad), max(0, int(ys.min()) - pad) + x1, y1 = min(w, int(xs.max()) + 1 + pad), min(h, int(ys.max()) + 1 + pad) + return x0, y0, x1, y1 + + def erase_cv2( image_bgr: NDArray[Any], mask: NDArray[Any], @@ -139,14 +162,10 @@ def erase_lama(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]: size = next((d for d in reversed(dims) if isinstance(d, int) and d > 1), 512) h, w = image_bgr.shape[:2] - ys, xs = np.where(mask > 0) - if len(xs) == 0: + box = _padded_crop_box(mask, h, w, pad_frac=0.4, pad_min=16) + if box is None: return image_bgr.copy() - - # Padded crop around the mask (context for the inpainter). - pad = max(16, int(0.4 * max(xs.max() - xs.min() + 1, ys.max() - ys.min() + 1))) - cx0, cy0 = max(0, int(xs.min()) - pad), max(0, int(ys.min()) - pad) - cx1, cy1 = min(w, int(xs.max()) + 1 + pad), min(h, int(ys.max()) + 1 + pad) + cx0, cy0, cx1, cy1 = box crop = image_bgr[cy0:cy1, cx0:cx1] crop_mask = mask[cy0:cy1, cx0:cx1] ch, cw = crop.shape[:2] @@ -201,16 +220,25 @@ def _get_migan_session() -> object: def erase_migan(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]: """Inpaint ``mask`` (255 = erase) with MI-GAN via onnxruntime (CPU). - The MI-GAN ONNX pipeline crops around the mask bbox internally and re-composites, - so the full image is fed at native resolution. Only the masked pixels are pasted - back, so untouched areas stay pixel-exact. + Like ``erase_lama``, we crop a padded region around the mask, feed only that crop + to the ONNX model, and paste only the masked pixels back -- so untouched areas stay + pixel-exact and the ONNX working set is bounded by the MARK size, not the whole + image. MI-GAN accepts arbitrary spatial dims, so (unlike LaMa's fixed 512 square) + the crop is fed at NATIVE resolution -- no resize, so the mark is seen at full + scale. Feeding the whole frame instead made peak RAM scale with the image + (~0.6 GB at 4 MP up to ~2.4 GB at 25 MP, measured 2026-07); cropping holds the + inpaint working set roughly constant, so a memory-tight host (e.g. a 1-2 GB web + worker) can run MI-GAN on a 25 MP upload. Cropping does not degrade the fill: + a small mark only needs local context, and on real marks the cropped fill is on + par with -- sometimes cleaner than -- the full-frame fill (a tighter view gives + the GAN less room to hallucinate large background structure). Mask polarity: the shipped ``andraniksargsyan/migan`` ONNX expects 0 = hole (inpaint) / 255 = known (keep) -- the INVERSE of this package's 255-erase convention -- so the mask is inverted before feeding the model (corpus-validated 2026-07; feeding 255=hole regenerates the whole frame into stripes). - Like ``erase_lama``, accepts 1-channel (grayscale) and 4-channel (BGRA) input. + Accepts 1-channel (grayscale) and 4-channel (BGRA) input. """ if image_bgr.ndim == 2: bgr = erase_migan(cv2.cvtColor(image_bgr, cv2.COLOR_GRAY2BGR), mask) @@ -219,26 +247,40 @@ def erase_migan(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]: bgr = erase_migan(np.ascontiguousarray(image_bgr[:, :, :3]), mask) return np.dstack([bgr, image_bgr[:, :, 3]]) + h, w = image_bgr.shape[:2] + # 2x the mark size (min 256 px) gives ample local context while a small corner + # mask keeps the crop small regardless of the full image resolution; the crop is + # clamped to the image, so a mark already spanning the frame degrades to the + # whole image (the old behavior) rather than erroring. + box = _padded_crop_box(mask, h, w, pad_frac=2.0, pad_min=256) + if box is None: + return image_bgr.copy() + cx0, cy0, cx1, cy1 = box + crop = np.ascontiguousarray(image_bgr[cy0:cy1, cx0:cx1]) + crop_mask = mask[cy0:cy1, cx0:cx1] + ch, cw = crop.shape[:2] + session = _get_migan_session() inp = session.get_inputs() # type: ignore[attr-defined] img_name, mask_name = inp[0].name, inp[1].name - h, w = image_bgr.shape[:2] - rgb = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB) - img_in = np.transpose(rgb, (2, 0, 1))[None].astype(np.uint8) # (1,3,H,W) + rgb = cv2.cvtColor(crop, cv2.COLOR_BGR2RGB) + img_in = np.transpose(rgb, (2, 0, 1))[None].astype(np.uint8) # (1,3,ch,cw) # invert to MI-GAN polarity: 255 where KNOWN (keep), 0 where hole (erase) - known = (mask <= 127).astype(np.uint8) * 255 - mask_in = known[None, None] # (1,1,H,W) + known = (crop_mask <= 127).astype(np.uint8) * 255 + mask_in = known[None, None] # (1,1,ch,cw) out = session.run(None, {img_name: img_in, mask_name: mask_in})[0] # type: ignore[attr-defined] - res = np.transpose(np.asarray(out)[0], (1, 2, 0)).astype(np.uint8) # (H',W',3) RGB - if res.shape[:2] != (h, w): - res = cv2.resize(res, (w, h), interpolation=cv2.INTER_LINEAR) + res = np.transpose(np.asarray(out)[0], (1, 2, 0)).astype(np.uint8) # (ch',cw',3) RGB + if res.shape[:2] != (ch, cw): + res = cv2.resize(res, (cw, ch), interpolation=cv2.INTER_LINEAR) out_bgr = cv2.cvtColor(res, cv2.COLOR_RGB2BGR) result = image_bgr.copy() - hole = mask > 127 - result[hole] = out_bgr[hole] + region = result[cy0:cy1, cx0:cx1] + hole = crop_mask > 127 + region[hole] = out_bgr[hole] + result[cy0:cy1, cx0:cx1] = region return result diff --git a/tests/test_region_eraser.py b/tests/test_region_eraser.py index 9b3c8e6..57aeefd 100644 --- a/tests/test_region_eraser.py +++ b/tests/test_region_eraser.py @@ -171,6 +171,7 @@ class TestMiganWrapper: def run(self, _outputs, feeds): self.outer.captured["mask"] = feeds["mask"] + self.outer.captured["image_shape"] = feeds["image"].shape img = feeds["image"] # (1,3,H,W) RGB red = np.zeros_like(img) red[:, 0] = 255 # pure red in RGB @@ -191,6 +192,21 @@ class TestMiganWrapper: assert m[50, 50] == 0 assert m[10, 10] == 255 + @pytest.mark.usefixtures("_fake_migan") + def test_crops_around_mask_so_onnx_input_is_bounded(self): + # Large frame, small corner mark: the tensor fed to MI-GAN is the padded + # CROP (pad = max(256, 2*bbox)), not the full image -- this is what holds the + # ONNX working set roughly constant on big uploads instead of scaling with + # the image (the memory fix). Untouched pixels stay exact; the mark is filled. + img = np.full((2000, 3000, 3), 120, np.uint8) + out = erase(img, boxes=[(2900, 1900, 60, 60)], backend="migan", dilate=0) + assert out.shape == img.shape + _, _, fh, fw = self.captured["image_shape"] + assert fh < 700 # crop height, not the 2000px frame + assert fw < 700 # crop width, not the 3000px frame + assert tuple(int(v) for v in out[1930, 2930]) == (0, 0, 255) # mark -> red fill + assert np.array_equal(out[:100, :100], img[:100, :100]) # far corner untouched + @pytest.mark.usefixtures("_fake_migan") def test_grayscale_2d_does_not_raise(self): gray = np.full((100, 100), 120, np.uint8)