mirror of
https://github.com/wiltodelta/remove-ai-watermarks.git
synced 2026-10-19 23:47:30 +02:00
fix(gemini): rescue small corner sparkle buried by the size weight (#36)
detect_watermark's size-weighted global NCC search lets a larger, mediocre match (e.g. a bright collar in a portrait) outrank a small, near-perfect sparkle in the bottom-right corner, so a faint sparkle on a busy background scored below threshold and the image read as clean -- the regression from widening the search window 256px->512px between v0.7.2 and v0.8.8. Add _corner_promote: a bottom-right-corner raw-NCC pass that overrides the global pick when the corner holds a match with raw NCC >= 0.85 that beats it. It only ever replaces a lower-fidelity pick (cannot weaken an existing detection) and keeps the wider window for variant margins. The corner side is relative-clamped (0.20 of the short side, [96, 384]) so it stays a true corner at every scale: a fixed 256px covers ~70% of a small portrait, where a real photo raw-matches the star at ~0.81; relative tightening drops that to ~0.69. The 0.85 gate sits between the worst real-photo corner match (~0.78) and a genuine faint sparkle (~0.93): zero false positives across native + downscaled negatives, headshot rescued from below-threshold to 0.71. Factor the shared multi-scale matchTemplate loop into _scan_scales. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Victor KuznetsovandClaude Opus 4.8 committed
1 parent
07c96bed53
commit
175609b60a
3 files changed
+160
-16
No files matched your search
@@ -36,7 +36,7 @@ You are a **principal Python engineer** maintaining a CLI tool and library for r
|
||||
- `metadata.py` — `scan_head(path, size=1MB)` is the shared input for every C2PA/AIGC/IPTC byte scan: first `size` bytes plus the payloads of any provenance metadata found beyond that window — for ISOBMFF, the late provenance boxes from `isobmff.scan_c2pa_region` (catches a manifest after a large `mdat`); for **PNG**, the late `tEXt`/`iTXt`/`zTXt`/`eXIf`/`iCCP` chunks from `_png_late_metadata` (catches an XMP/EXIF packet appended after a large `IDAT`, e.g. a TC260 AIGC label at ~2.7 MB). Behavior-neutral (`f.read(size)`) for non-ISOBMFF inputs and for any file that fits within `size`. Use it instead of `open().read(1MB)` for any new marker scan. `synthid_source(path)` returns the vendor name(s) if the C2PA manifest implies a SynthID pixel watermark, else None. Format-agnostic: PNG via the caBX parser, JPEG/WebP/AVIF/HEIF/JXL via a binary scan (C2PA marker + SynthID issuer + AI-source marker). `get_ai_metadata` surfaces the verdict, and `metadata --check` prints it as a callout. Both `get_ai_metadata` and `has_ai_metadata` guard the PIL open with `except Exception` (HEIC/unknown formats raise non-OSError) and fall through to the binary scan. `xai_signature(path)` detects xAI/Grok's EXIF-only scheme (`ImageDescription` = `Signature: <base64>` + UUID `Artist`); it feeds `has_ai_metadata`, `get_ai_metadata` (key `xai_signature`), and `identify`. `iptc_ai_system(path)` detects the IPTC Photo Metadata 2025.1 AI-disclosure XMP properties (`IPTC_AI_FIELD_MARKERS` = `AISystemUsed`/`AISystemVersionUsed`/`AIPromptInformation`/`AIPromptWriterName`) and returns the `AISystemUsed` generator name (or `"fields present"`). `remove_ai_metadata` routes **ISOBMFF video** (`.mp4`/`.mov`/`.m4v`) through the same `isobmff.strip_c2pa_boxes` as AVIF/HEIF (MP4 is ISOBMFF), and `_scrub_ai_exif` removes the xAI signature + AI-generator EXIF tags on JPEG output. `strip_c2pa_boxes` is **fail-safe** on a malformed box: it returns the original bytes unchanged with a logged warning instead of truncating the tail to EOF (detection-only `scan_c2pa_region` still stops at a malformed box). `_png_late_metadata` clamps each late-chunk read to the remaining file size (`safe_length = min(length, remaining)`) so a malformed `length` cannot drive a multi-GB allocation.
|
||||
- `identify.py` — the OpenAI rollout caveat is keyed on `_vendor_of(synthid) == "OpenAI"` (not a raw substring over the issuer + verdict blob). `identify(path)` aggregates every locally-readable signal (C2PA issuer→platform, C2PA soft-binding forensic-watermark vendor, IPTC "Made with AI" + IPTC 2025.1 `AISystemUsed`, embedded SD/ComfyUI params, SynthID proxy, xAI/Grok EXIF signature via `metadata.xai_signature`, the China TC260 AIGC label via `metadata.aigc_label`, the HuggingFace `hf-job-id` job marker via `metadata.huggingface_job`, the Samsung Galaxy AI editing marker via `metadata.samsung_genai`, the visible marks — Gemini sparkle plus the ByteDance Doubao 豆包AI生成 / Jimeng 即梦AI text marks via the `watermark_registry` — open invisible watermark, Adobe TrustMark via `trustmark_detector`) into one `ProvenanceReport`. `is_ai_generated` is True or None (never asserted False — stripped metadata is not proof of clean origin). The `hf_job`, visible-mark, and Samsung `samsung_genai` signals are **medium** confidence: each lifts an otherwise-Unknown verdict to a tentative AI (`hf_only` / `visible_only` / `samsung_only`, parallel branches; `visible_only` fires on any `visible_*` signal) but is excluded from the high-confidence `ai_from_metadata` set, so none overrides a hard metadata signal. **Visible-mark detection** (`check_visible`, signals `visible_sparkle` / `visible_doubao` / `visible_jimeng`): the Gemini sparkle keeps its own file-level path (`_visible_sparkle` → `gemini_engine.detect_sparkle_confidence`, promoted only at confidence ≥ `_SPARKLE_THRESHOLD` 0.5; corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49), while Doubao/Jimeng reuse the registry detectors (`_visible_text_marks` → `watermark_registry`), each gated by its own engine NCC threshold via `MarkDetection.detected` (Doubao 0.4, Jimeng 0.45). Doubao/Jimeng are normally also caught by the TC260 AIGC metadata label, so the visible path is their stripped-metadata fallback. Visible marks set `platform` only when no harder signal already did, and (like the sparkle) are excluded from integrity-clash vendor claims. The cv2 dependency lives in the engines, not here. **`import identify` is deliberately light** (~21 MB; ~36 MB with cv2 loaded by a visible-mark run, ~106 MB for a full `check_visible` run): it imports only the pure `noai.c2pa`/`noai.constants` submodules, and `noai/__init__` is lazy (see "Test and lint"), so torch/diffusers are NOT pulled at import even in a full `gpu`/`detect` install — fits a 512 MB host. The heavy paths are opt-in: `check_invisible=True` needs the `detect`/`trustmark` extras (each pulls **torch**; TrustMark also **downloads weights**), so on a core-only deploy leave `check_invisible` off (it is a no-op there anyway). Before the lazy `__init__`, the mere presence of torch in the env inflated `import identify` to ~420 MB. **C2PA platform attribution is device-token-first, issuer-scan fallback** (`_device_platform` scans manifest bytes for `_DEVICE_C2PA_PLATFORM` tokens, then `_attribute_platform`/`_ISSUER_PLATFORM`). **Why, verified on real signed files 2026-05-26:** the old issuer-only byte-scan matched ANY issuer substring anywhere, so multi-entity manifests mis-attributed -- Leica→"Truepic" (a signing authority in the trust chain), Nikon→"Adobe Firefly" (XMP-toolkit "Adobe" + the sample's "Adobe_MAX" name), Pixel→"Google (Gemini)" ("Google LLC" cert org), Truepic→"Google". A distinctive device token wins instead. **Token distinctiveness is load-bearing:** bare `b"Truepic"` mis-fires (it appears in unrelated trust chains -- it mis-attributed the OpenAI `chatgpt-1.png` fixture), so the token is the specific `b"Truepic_Lens"` from the Lens SDK claim generator; likewise `b"Pixel Camera"` (cert CN) not bare `b"Pixel"`. `_DEVICE_C2PA_PLATFORM` lists ONLY tokens **verified against a real C2PA file**: Leica (`lc_c2pa`/`Leica Camera`), Nikon (`NIKON`), Pixel (`Pixel Camera` -- from a real Pixel 10 Pro file attached to c2pa-rs issue #1609/#1554), Sony (`sony.sig`/`sony.cert` -- Sony's own C2PA assertion namespace, verified on a real Sony PXW-Z300 file; NOT bare "Sony" which is a common EXIF Make), Truepic (`Truepic_Lens`). Canon/Bria have **no public direct-download C2PA sample** (checked exhaustively: GitHub issue/PR attachments, contentcredentials gallery, HF datasets -- all upload-to-verify or token-gated; Canon's only public file was a self-signed hobbyist CR3, not factory), so they stay unmapped until a real file is captured (same fixture discipline as Grok/Doubao). The Sony sample is video (MP4) -- our ISOBMFF C2PA path detects it; Sony Alpha stills likely share the `sony.*` namespace but are not separately verified. **Samsung Galaxy + ASUS Gallery live in a separate `_SIGNER_C2PA_PLATFORM` (scanned after `_device_platform`, before the issuer fallback), NOT in `_DEVICE_C2PA_PLATFORM`** — verified on real signed files 2026-05-29. Reason: a Galaxy phone stamps BOLine truncated
|
||||
- `watermark_registry.py` — **single catalog of known visible watermarks**, the unified "find known marks in their usual places, recognize, remove" entry. **Reverse-alpha based by policy**: a mark is listed only once a real alpha map has been captured for it, and removal inverts that map (`original = (wm - a*logo)/(1-a)`) — Gemini recovers cleanly with no inpaint (its sparkle alpha comes from a pure-black capture, so it is near-exact), while **Doubao and Jimeng both add an always-on THIN residual inpaint** over the glyph footprint (their text marks re-rasterize + jitter a few px per image, so a single capture cannot pixel-cancel them; the inpaint blends into the reverse-alpha-recovered pixels). Arbitrary-region inpainting still lives in `region_eraser`/`erase`. Each `KnownMark` ties a key to {usual `location`, `in_auto` flag, `recovery` (="reverse-alpha"), a `detect` adapter → uniform `MarkDetection`, a `remove` adapter}. Entries today: `gemini` (bottom-right sparkle), `doubao` (bottom-right "豆包AI生成"), and `jimeng` (bottom-right "★ 即梦AI"). `detect_marks` scans all; `best_auto_mark` picks the highest-confidence detection. **Cross-engine confidences aren't directly comparable**, so the gemini adapter applies the corpus-validated 0.5 sparkle threshold (`_GEMINI_AUTO_MIN_CONF`) for its `detected` flag — otherwise the gemini engine's loose internal threshold weakly fires (~0.36) on the Doubao text and hijacks `auto`. The shape-keyed Doubao/Jimeng NCC detectors don't cross-fire (jimeng scores ~0.22 on the Doubao strip, well under its 0.45 threshold), so `auto` picks the right one on a Doubao vs Jimeng image. `cli.cmd_visible` is registry-driven: `--mark auto` → `best_auto_mark`, `--mark <key>` → that mark; `--mark` choices come from `mark_keys()`. `_doubao_remove`/`_jimeng_remove` apply reverse-alpha only when the mark is detected AND `reverse_alpha_available`; outside that, removal is **skipped** (not inpainted). Add a new visible mark = one `KnownMark` entry + its engine (with a captured alpha map); do not re-add per-mark `if` branches in the CLI. **Alpha-on-save policy (issue #30):** `cli._write_bgr_with_alpha` rejoins the input's alpha plane **unchanged** — it must NOT zero alpha in the watermark bbox. Reverse-alpha (and `erase` inpaint) recover real pixels there, so zeroing alpha punched a transparent hole that renders as a solid **white box** on any non-transparent viewer (Gemini app exports are opaque RGBA, so every user hit it; regression-guarded by `test_visible_keeps_alpha_opaque_in_watermark_region`). The registry `remove()` still returns its region (used for `inpaint_residual` positioning), but the CLI no longer uses it to clear alpha.
|
||||
- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU). `detect_sparkle_confidence(path)` is the file-level entry point used by `identify.py`. The public entry points normalize a grayscale (2D) or RGBA (4-channel) input to BGR up front so a non-BGR image does not crash the cv2 pipeline. **Removal is reverse-alpha with an over-subtraction guard** (`remove_watermark` → `_reverse_alpha_blend`, else `_inpaint_footprint`): the sparkle alpha is computed (`alpha = max(R,G,B)/255`) from the bundled sparkle-on-black captures `assets/gemini_bg_{96,48}.png` (the capture max is ~130, NOT 255 — the sparkle is a ~51%-opaque white overlay, so `alpha` maxes at ~0.51, which is CORRECT for the capture, not under-exposed). The alpha is near-exact only when the real mark's effective opacity matches the capture, which holds on bright/flat backgrounds — re-verified clean on `demo_banana_before.png` 2026-05-31. **Issue #30 (dark-background black pit):** on a dark/textured background (e.g. grass, ~73) the real sparkle's effective opacity is LOWER than the captured 0.51, so the fixed-alpha reverse blend OVER-subtracts (`watermarked - a*logo` goes negative) and drives the footprint to black — the white sparkle becomes a black diamond. `remove_watermark` now detects this via `_reverse_alpha_oversubtracts` (fraction of footprint pixels with `alpha >= _FOOTPRINT_ALPHA` 0.1 whose numerator < 0 exceeds `_OVERSUB_FOOTPRINT_FRAC` 0.05) and **inpaints the footprint** (`_inpaint_footprint`, cv2 NS over the dilated alpha mask) from the surrounding pixels instead. **Behavior-neutral on the working case:** a bright background over-subtracts at ~0% so reverse-alpha is used and the output is byte-identical to before (verified: demo_banana 0.0 frac vs issue-#30 grass 0.61 frac; regression-guarded by `test_gemini_engine.py::TestOverSubtractionGuard`, which composites the sparkle at a reduced effective alpha to reproduce the mismatch). The registry's optional `inpaint_residual` (edge cleanup) is a no-op on a clean reverse-alpha removal; an earlier "Gemini smears" read was a misjudged soft-fur original, not an artifact. **The bg assets are now rebuilt from OUR OWN controlled captures** (`data/gemini_capture/captures/`, committed) by `scripts/visible_alpha_solve.py gemini`, which locates the 96px sparkle on the black capture and crops it to the two logo sizes; our capture matched the previously third-party-sourced `gemini_bg_96.png` to **NCC 0.9998**, validating the asset and making it reproducible. Gemini's multi-size fixed-slot model is genuinely different from the Doubao/Jimeng text-strip engines (so it stays a separate engine, not part of the shared-base refactor).
|
||||
- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU). `detect_sparkle_confidence(path)` is the file-level entry point used by `identify.py`. The public entry points normalize a grayscale (2D) or RGBA (4-channel) input to BGR up front so a non-BGR image does not crash the cv2 pipeline. **Detection localization (issue #36):** `detect_watermark`'s global multi-scale NCC search applies a size weight (`(scale/96)**0.5`) that suppresses tiny-patch false positives but can let a larger, mediocre match (e.g. a bright collar in a portrait) outrank a small, near-perfect sparkle in the corner — so a faint sparkle on a busy background scored below threshold and read as clean (the regression osachub reported from widening the search window 256px->512px between v0.7.2 and v0.8.8). `_corner_promote` adds a bottom-right-corner raw-NCC pass on top of the global search: a match with raw NCC >= `_CORNER_PROMOTE_NCC` 0.85 that beats the global pick overrides it (it only ever replaces a lower-fidelity pick, so it cannot weaken an existing detection), rescuing the buried sparkle without reverting the wider window. The corner side is **relative-clamped** (`_CORNER_PROMOTE_FRAC` 0.20 of the short side, clamped to `[_CORNER_PROMOTE_MIN` 96, `_CORNER_PROMOTE_MAX` 384`]`): a fixed 256px is a true corner on a large image but covers ~70% of a small portrait, where a real photo raw-matches the star at ~0.81 (relative tightening drops that worst case to ~0.69, while the upper clamp stops the corner ballooning on huge images where a real photo reached ~0.83 at 512px). The 0.85 gate sits midway between the worst real-photo corner match (~0.78 across native + downscaled negatives) and a genuine faint sparkle (~0.93), so promotion adds true detections with zero corpus false positives (Gemini's sparkle sits ~60-160px from the corner at fixed margins, covered by the [96, 384] band at every measured size). Regression-guarded by `test_gemini_engine.py::TestCornerPromotion`. **Removal is reverse-alpha with an over-subtraction guard** (`remove_watermark` → `_reverse_alpha_blend`, else `_inpaint_footprint`): the sparkle alpha is computed (`alpha = max(R,G,B)/255`) from the bundled sparkle-on-black captures `assets/gemini_bg_{96,48}.png` (the capture max is ~130, NOT 255 — the sparkle is a ~51%-opaque white overlay, so `alpha` maxes at ~0.51, which is CORRECT for the capture, not under-exposed). The alpha is near-exact only when the real mark's effective opacity matches the capture, which holds on bright/flat backgrounds — re-verified clean on `demo_banana_before.png` 2026-05-31. **Issue #30 (dark-background black pit):** on a dark/textured background (e.g. grass, ~73) the real sparkle's effective opacity is LOWER than the captured 0.51, so the fixed-alpha reverse blend OVER-subtracts (`watermarked - a*logo` goes negative) and drives the footprint to black — the white sparkle becomes a black diamond. `remove_watermark` now detects this via `_reverse_alpha_oversubtracts` (fraction of footprint pixels with `alpha >= _FOOTPRINT_ALPHA` 0.1 whose numerator < 0 exceeds `_OVERSUB_FOOTPRINT_FRAC` 0.05) and **inpaints the footprint** (`_inpaint_footprint`, cv2 NS over the dilated alpha mask) from the surrounding pixels instead. **Behavior-neutral on the working case:** a bright background over-subtracts at ~0% so reverse-alpha is used and the output is byte-identical to before (verified: demo_banana 0.0 frac vs issue-#30 grass 0.61 frac; regression-guarded by `test_gemini_engine.py::TestOverSubtractionGuard`, which composites the sparkle at a reduced effective alpha to reproduce the mismatch). The registry's optional `inpaint_residual` (edge cleanup) is a no-op on a clean reverse-alpha removal; an earlier "Gemini smears" read was a misjudged soft-fur original, not an artifact. **The bg assets are now rebuilt from OUR OWN controlled captures** (`data/gemini_capture/captures/`, committed) by `scripts/visible_alpha_solve.py gemini`, which locates the 96px sparkle on the black capture and crops it to the two logo sizes; our capture matched the previously third-party-sourced `gemini_bg_96.png` to **NCC 0.9998**, validating the asset and making it reproducible. Gemini's multi-size fixed-slot model is genuinely different from the Doubao/Jimeng text-strip engines (so it stays a separate engine, not part of the shared-base refactor).
|
||||
- `doubao_engine.py` — visible Doubao "豆包AI生成" remover/detector (cv2/numpy, no GPU). `DoubaoEngine.locate` anchors a bottom-right box by **geometry** (mark scales with image WIDTH), `extract_mask` pulls the light, low-chroma glyphs (the detection candidate) using a per-pixel channel-spread proxy `sat = roi.max(axis=2) - roi.min(axis=2)` (no HSV conversion). `detect` is **shape-consistent**: it matches the bundled alpha glyph silhouette (`assets/doubao_alpha.png`) against the candidate via zero-mean normalized correlation (`_template_match_score`, cv2 `TM_CCOEFF_NORMED`), gated at `DETECT_NCC_THRESHOLD` 0.4 over a small `DETECT_MIN_COVERAGE` floor. Keying on glyph SHAPE (not coverage heuristics) fixed #23 (corpus FP 7/1243). **Removal = reverse-alpha + thin residual inpaint** (`remove_watermark_reverse_alpha`): `original = (wm - a*logo)/(1-a)` from the bundled alpha map + `_ALPHA_LOGO_BGR` (pure white) + `_ALPHA_*_FRAC` geometry, then a deliberately THIN inpaint (`_RESIDUAL_*`, `INPAINT_NS`) over the glyph footprint clears leftover edges without smearing. **Alpha is rebuilt by `scripts/visible_alpha_solve.py` (the careful gray-self solve: cubic background fit, mean over channels, full halo, unblurred), same recipe as Jimeng** — the captures are committed in `data/doubao_capture/captures/`. **Removal aligns ALWAYS** (no `_ALPHA_NATIVE_BAND` fast-path): it tries fixed geometry AND `_aligned_alpha_map`'s `TM_CCOEFF_NORMED` scale+position search and keeps the lower-residual one — the mark is re-rasterized and a few px off per image, so fixed geometry alone leaves a visible outline even at 2048. **The locate box (`WM_*`) is generous (0.22 wide, margins 0.004) and reaches close to the corner** — a tight box (the old 0.185 / margin 0.012) let a corner-ward shift fall OUTSIDE the alignment search, so the align missed and a readable outline survived; regression-guarded by `test_recovers_shifted_mark_on_texture` (composes the alpha shifted on a known texture; old box ~29 vs new ~1 mean residual). **Issue #13 follow-up defect (found 2026-05-31): the SHIPPED Doubao removal left a clearly READABLE "豆包AI生成" outline on the real `doubao-1.png` sample, while `detect` returned conf 0.0 (it is fooled by a thin outline) so `test_reverse_alpha_removes_mark` passed and the old "56/56 clean" claim was detector-measured, not visual.** Root cause: bad alpha (under-estimated, max ~0.65) + fixed-no-inpaint + tight box; the careful rebuild + always-align + thin inpaint + wide box takes it from a readable outline to faint texture-level traces (parity with Jimeng — a single capture cannot pixel-cancel a per-image re-rasterized mark). **Lesson: a detector-only removal test is insufficient; assert visual residual (the textured-shift test).** **`extract_mask` guards a degenerate ROI (`bh < 16 or bw < 16` -> empty mask, skips cv2):** the always-align removal scores each placement with a residual `detect(out)`, and on an extremely wide/short image (e.g. 2048x1, `test_wide_short_does_not_raise`) that fed cv2's GaussianBlur a ~1-px-tall ROI and **faulted natively on Windows py3.12 (access violation, non-deterministic — one CI cell went red while a re-run passed)**; the old at-native path never ran `detect` on degenerate sizes. Real images always clear the guard (the `WM_*` box floors are `max(16, …)` height / `max(40, …)` width), so it only short-circuits slivers. `reverse_alpha_available` is just "asset present"; the registry gates removal on `detect`. The shipped third-party `_refs/zhengsuanfa_doubao_alpha_120x20.png` is NOT a usable alpha (verified 2026-05-29). Arbitrary-region inpainting is `region_eraser`/`erase`.
|
||||
- `jimeng_engine.py` — visible Jimeng / Dreamina "★ 即梦AI" remover/detector (cv2/numpy, no GPU), built 2026-05-30 from issue #13's solid captures (@powersee). Mirrors `doubao_engine`: `locate` anchors a bottom-right box by **geometry** (scales with WIDTH), `extract_mask` pulls the light low-chroma glyphs (white top-hat + grayish + min-luma), `detect` matches the bundled "即梦AI" glyph silhouette (`assets/jimeng_alpha.png`) via `TM_CCOEFF_NORMED` over a coverage floor. Threshold `DETECT_NCC_THRESHOLD` **0.45** cleanly separates real Jimeng marks (>=0.81) from the Doubao strip (0.21) and other AI output (0.0), so the two ByteDance marks don't cross-fire in `--mark auto`. **Logo is pure white (255,255,255)** (`_ALPHA_LOGO_BGR`; the white capture + an L-pair-solve confirm ~254.6); compositing is **sRGB, not linear** (a linear-light solve tripled the cross-residual). **Alpha rebuilt by `scripts/visible_alpha_solve.py` from the GRAY capture** (`data/jimeng_capture/captures/`, the solid captures now committed): `a = (I - B)/(255 - B)`, B a per-capture **cubic** background fit over the non-glyph pixels, **averaged over channels, full halo extent (down to a~0.02), unblurred**. Gray (bg ~132) is the deliberate choice over black: it is the best proxy for real content (the mark sits on bright photo areas, not on black), and the careful build drops the gray self-residual to ~1.3. **The mask quality, not the method, was the earlier limit** — a max-channel / quadratic-bg / blurred / halo-truncated build (and a black-dominated LS) left a visible outline (lesson from issue #13: when reverse-alpha leaves a ghost, suspect the captured alpha map before adding heuristics or switching method). Geometry emitted by the solver at `_ALPHA_NATIVE_WIDTH` 2048: `_ALPHA_WIDTH_FRAC` 0.202, `_ALPHA_HEIGHT_FRAC` 0.058, margins ~0.029. **Removal = reverse-alpha + a deliberately THIN residual inpaint** (`remove_watermark_reverse_alpha`, `_RESIDUAL_DILATE` 5 over the `_RESIDUAL_ALPHA_FLOOR` 0.05 footprint, `_RESIDUAL_INPAINT_RADIUS` 2, `INPAINT_NS`): a single 2048 alpha cannot pixel-cancel the mark re-rasterized at another resolution (alpha maps from independent captures correlate 0.998, not 1.0; off-native reverse-alpha alone only halves the mark), so a tight inpaint clears the residual edges WITHOUT the texture/edge smear a wide full-footprint pass caused. **Placement ALWAYS tries fixed geometry AND `_aligned_alpha_map`'s NCC scale+position search, keeping the lower-residual** — the mark re-rasterizes + jitters a few px per image even at the captured width, so fixed geometry alone misses (there is no `_ALPHA_NATIVE_BAND` fast-path; the scale search `_ALPHA_ALIGN_SEARCH` is fine-stepped, and the `WM_*` locate box is generous so a corner-ward shift stays inside the search — the same widen that fixed Doubao). Verified clean on the solid captures (native 2048; faint self-residual ~1.3 visible only on a dead-flat field, hidden by real texture) and a real 1440-wide Jimeng download (off-native, table edge preserved). `reverse_alpha_available` is just "asset present"; the registry gates on `detect`. **No committed real sample** (the real content download stays gitignored; only the solid calibration captures are committed) — `tests/test_jimeng_engine.py` synthesizes a mark from the bundled alpha asset, and `test_recovers_shifted_mark_on_texture` guards the align-on-shift path that the Doubao defect exposed. Jimeng images are independently caught by the China TC260 AIGC label in `metadata`/`identify`, so this engine is the visible-mark *removal* path, not a new `identify` signal.
|
||||
- `region_eraser.py` — universal region eraser (`erase` CLI). `erase(image, boxes=|mask=, backend=)` normalizes grayscale (2D) and RGBA (4-channel) inputs up front (`erase_cv2` splits off any alpha plane and re-attaches it on the result): `boxes_to_mask` → `cv2.inpaint` (`cv2` backend, default, no deps) or big-LaMa via onnxruntime (`lama` backend, extra `lama`, `Carve/LaMa-ONNX` Apache-2.0 model downloaded on first use, never bundled). `erase_lama` crops a padded region around the mask, runs LaMa at its fixed 512² input, pastes only masked pixels back (untouched areas stay pixel-exact). Lazy `_get_lama_session` singleton; `lama_available()` guards the optional import. **LaMa-ONNX costs ~3.5-4 GB peak RAM and ~5-6 s/call on CPU** (FFC working set, not arena — `enable_cpu_mem_arena=False` does not help), so it does NOT fit a minimal droplet; the cv2 backend (tens of MB, ~30 ms) does. LaMa quality at low RAM = serverless/GPU, mirroring how raiw.cc offloads SDXL to fal.
|
||||
|
||||
@@ -29,6 +29,8 @@ import cv2
|
||||
import numpy as np
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
from numpy.typing import NDArray
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -138,6 +140,34 @@ class GeminiEngine:
|
||||
# gate separates them with a wide margin.
|
||||
_OVERSUB_FOOTPRINT_FRAC = 0.05
|
||||
|
||||
# Corner promotion (issue #36): the size weight that suppresses tiny-patch
|
||||
# false positives also buries a small, near-perfect sparkle when a larger,
|
||||
# mediocre match sits elsewhere (e.g. a bright collar in a portrait). A small
|
||||
# faint sparkle on a busy background therefore loses the global argmax and the
|
||||
# image reads as clean -- the regression osachub reported when the search
|
||||
# window widened 256px -> 512px (v0.7.2's tighter window still found it).
|
||||
# Remedy: if the bottom-right corner holds a very-high-fidelity raw-NCC match,
|
||||
# trust it regardless of size, without reverting the wider window (which is
|
||||
# needed for variant margins). The threshold sits midway between the worst
|
||||
# real-photo corner match (~0.78 across native + downscaled real photos) and a
|
||||
# genuine faint sparkle (~0.93), so it adds true detections without adding
|
||||
# false ones; it only ever overrides a lower-fidelity global pick, so it cannot
|
||||
# weaken an existing detection.
|
||||
_CORNER_PROMOTE_NCC = 0.85
|
||||
# Bottom-right corner side for the promotion search, as a fraction of the
|
||||
# image's short side, clamped to an absolute pixel band. Relative so the corner
|
||||
# stays a true corner at every scale: a fixed 256 px is a genuine corner on a
|
||||
# large image but covers ~70% of a small portrait, where a busy real photo can
|
||||
# then raw-match the star template at ~0.81 (only 0.04 below the promote gate).
|
||||
# Scaling the side down on small images drops that worst case to ~0.69, while
|
||||
# the upper clamp stops it ballooning on huge images (more corner area = more
|
||||
# random texture to false-match -- a real photo reached ~0.83 at 512 px). The
|
||||
# Gemini sparkle sits ~60-160 px from the corner (fixed margins, not
|
||||
# proportional), and the [96, 384] band covers that at every measured size.
|
||||
_CORNER_PROMOTE_FRAC = 0.20
|
||||
_CORNER_PROMOTE_MIN = 96
|
||||
_CORNER_PROMOTE_MAX = 384
|
||||
|
||||
def __init__(self, logo_value: float = 255.0) -> None:
|
||||
"""Initialize the engine with embedded alpha maps.
|
||||
|
||||
@@ -183,6 +213,24 @@ class GeminiEngine:
|
||||
|
||||
# ── Detection ────────────────────────────────────────────────────
|
||||
|
||||
def _scan_scales(self, gray: NDArray[Any]) -> Iterator[tuple[int, float, tuple[int, int]]]:
|
||||
"""Yield ``(scale, max_ncc, max_loc)`` for the alpha template matched at each scale.
|
||||
|
||||
Shared multi-scale ``TM_CCOEFF_NORMED`` primitive over a normalized [0, 1]
|
||||
grayscale region, used by both the size-weighted global search in
|
||||
``detect_watermark`` and the raw-NCC corner pass in ``_corner_promote`` --
|
||||
each applies its own scoring/argmax to the yielded values. The 96x96
|
||||
``_alpha_large`` is the high-quality source downscaled per scale; the range
|
||||
covers aggressively downscaled to slightly upscaled logos.
|
||||
"""
|
||||
for scale in range(16, 120, 2):
|
||||
if scale > gray.shape[0] or scale > gray.shape[1]:
|
||||
continue
|
||||
tmpl = cv2.resize(self._alpha_large, (scale, scale), interpolation=cv2.INTER_AREA)
|
||||
match_res = cv2.matchTemplate(gray, tmpl, cv2.TM_CCOEFF_NORMED)
|
||||
_, max_val, _, max_loc = cv2.minMaxLoc(match_res)
|
||||
yield scale, float(max_val), max_loc
|
||||
|
||||
def detect_watermark(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
@@ -198,9 +246,6 @@ class GeminiEngine:
|
||||
base_size = force_size or get_watermark_size(w, h)
|
||||
result.size = base_size
|
||||
|
||||
# Use large alpha template (96x96) as the high-quality source for downscaling
|
||||
source_alpha = self._alpha_large
|
||||
|
||||
# Dynamically search bottom-right corner. 512 covers up to 512px from the
|
||||
# corner -- enough for known Gemini margin variations (standard: 64+96=160px;
|
||||
# observed variants up to ~300px). 256 was too tight and caused misses.
|
||||
@@ -216,25 +261,15 @@ class GeminiEngine:
|
||||
|
||||
gray_sr_f = gray_sr.astype(np.float32) / 255.0
|
||||
|
||||
# Phase 1 & 2: Multi-scale spatial NCC search
|
||||
# Phase 1 & 2: multi-scale spatial NCC search, size-weighted argmax.
|
||||
best_scale = 0
|
||||
best_score = -1.0
|
||||
best_raw_ncc = -1.0
|
||||
best_loc = (0, 0)
|
||||
|
||||
# Search scales from 16 to 120 (covering aggressively downscaled or slightly upscaled logos)
|
||||
for scale in range(16, 120, 2):
|
||||
if scale > search_region.shape[0] or scale > search_region.shape[1]:
|
||||
continue
|
||||
|
||||
tmpl = cv2.resize(source_alpha, (scale, scale), interpolation=cv2.INTER_AREA)
|
||||
match_res = cv2.matchTemplate(gray_sr_f, tmpl, cv2.TM_CCOEFF_NORMED)
|
||||
_, max_val, _, max_loc = cv2.minMaxLoc(match_res)
|
||||
|
||||
for scale, max_val, max_loc in self._scan_scales(gray_sr_f):
|
||||
# Size-adjusted score to overcome NCC bias toward tiny patches (mimics C++ weight)
|
||||
weight = min(1.0, (scale / 96.0) ** 0.5)
|
||||
adj_val = max_val * weight
|
||||
|
||||
if adj_val > best_score:
|
||||
best_score = adj_val
|
||||
best_scale = scale
|
||||
@@ -244,6 +279,14 @@ class GeminiEngine:
|
||||
# Exact dynamic location & size
|
||||
pos_x = sx1 + best_loc[0]
|
||||
pos_y = sy1 + best_loc[1]
|
||||
|
||||
# Corner promotion: a near-perfect but small sparkle in the bottom-right
|
||||
# corner is otherwise outranked by a larger, mediocre size-weighted match
|
||||
# (see _CORNER_PROMOTE_NCC). Override the global pick with it when present.
|
||||
promoted = self._corner_promote(image, best_raw_ncc)
|
||||
if promoted is not None:
|
||||
best_scale, pos_x, pos_y, best_raw_ncc = promoted
|
||||
|
||||
result.region = (pos_x, pos_y, best_scale, best_scale)
|
||||
result.spatial_score = float(best_raw_ncc)
|
||||
|
||||
@@ -317,6 +360,42 @@ class GeminiEngine:
|
||||
|
||||
return result
|
||||
|
||||
def _corner_promote(
|
||||
self,
|
||||
image: NDArray[Any],
|
||||
current_raw_ncc: float,
|
||||
) -> tuple[int, int, int, float] | None:
|
||||
"""Search the bottom-right corner for a very-high-fidelity sparkle match.
|
||||
|
||||
Returns ``(scale, x, y, raw_ncc)`` when the corner holds a match with raw
|
||||
NCC >= ``_CORNER_PROMOTE_NCC`` that beats the global pick's ``current_raw_ncc``,
|
||||
else None. Used to rescue a small sparkle that the size weight buried under
|
||||
a larger, lower-fidelity match elsewhere. See ``_CORNER_PROMOTE_NCC`` and
|
||||
``_CORNER_PROMOTE_FRAC`` for the corner sizing.
|
||||
"""
|
||||
h, w = image.shape[:2]
|
||||
side = max(
|
||||
self._CORNER_PROMOTE_MIN, min(self._CORNER_PROMOTE_MAX, round(min(w, h) * self._CORNER_PROMOTE_FRAC))
|
||||
)
|
||||
cs = int(min(min(w, h), side))
|
||||
cx1, cy1 = max(0, w - cs), max(0, h - cs)
|
||||
corner = image[cy1:h, cx1:w]
|
||||
gray = cv2.cvtColor(corner, cv2.COLOR_BGR2GRAY) if corner.ndim == 3 and corner.shape[2] >= 3 else corner
|
||||
gray = gray.astype(np.float32) / 255.0
|
||||
|
||||
best_raw = -1.0
|
||||
best_scale = 0
|
||||
best_loc = (0, 0)
|
||||
for scale, max_val, max_loc in self._scan_scales(gray):
|
||||
if max_val > best_raw:
|
||||
best_raw = max_val
|
||||
best_scale = scale
|
||||
best_loc = max_loc
|
||||
|
||||
if best_raw >= self._CORNER_PROMOTE_NCC and best_raw > current_raw_ncc:
|
||||
return best_scale, cx1 + best_loc[0], cy1 + best_loc[1], float(best_raw)
|
||||
return None
|
||||
|
||||
# ── Removal ──────────────────────────────────────────────────────
|
||||
|
||||
def remove_watermark(
|
||||
|
||||
@@ -284,3 +284,68 @@ class TestOverSubtractionGuard:
|
||||
dark, dpos = self._composite_sparkle(bg_value=60)
|
||||
dalpha = self.engine.get_interpolated_alpha(dpos[2])
|
||||
assert self.engine._reverse_alpha_oversubtracts(dark, dalpha, (dpos[0], dpos[1])) is True
|
||||
|
||||
|
||||
class TestCornerPromotion:
|
||||
"""Issue #36: a small sparkle in the corner must not be lost to a larger decoy.
|
||||
|
||||
The size weight that suppresses tiny-patch false positives also lets a larger,
|
||||
mediocre match elsewhere outrank a small, near-perfect sparkle in the corner --
|
||||
so a faint sparkle on a busy background (e.g. a portrait whose bright collar
|
||||
out-scores it) reads as clean. The corner-promotion override rescues it.
|
||||
"""
|
||||
|
||||
_W, _H = 400, 520
|
||||
_CORNER = (_W - 40 - 20, _H - 40 - 20, 20) # bottom-right small sparkle (x, y, scale)
|
||||
_DECOY = (15, 210, 92) # large decoy: inside the search window, left of the corner
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _setup_engine(self):
|
||||
self.engine = GeminiEngine()
|
||||
|
||||
def _paste(self, img: np.ndarray, scale: int, x: int, y: int, alpha_scale: float) -> None:
|
||||
tmpl = cv2.resize(self.engine._alpha_large, (scale, scale), interpolation=cv2.INTER_AREA)
|
||||
a = (tmpl * alpha_scale)[:, :, None]
|
||||
roi = img[y : y + scale, x : x + scale]
|
||||
img[y : y + scale, x : x + scale] = a * 255.0 + (1.0 - a) * roi
|
||||
|
||||
def _scene(self, bg_value: int = 40) -> np.ndarray:
|
||||
"""Dark scene with a large decoy on the left and a small sparkle in the corner.
|
||||
|
||||
Without the corner-promotion fix the global, size-weighted search locks onto
|
||||
the larger decoy; with it the small corner sparkle wins.
|
||||
"""
|
||||
img = np.full((self._H, self._W, 3), bg_value, dtype=np.float32)
|
||||
self._paste(img, self._DECOY[2], self._DECOY[0], self._DECOY[1], 0.55)
|
||||
self._paste(img, self._CORNER[2], self._CORNER[0], self._CORNER[1], 0.55)
|
||||
return np.clip(img, 0, 255).astype(np.uint8)
|
||||
|
||||
def _in_bottom_right(self, region: tuple[int, int, int, int]) -> bool:
|
||||
x, y = region[0], region[1]
|
||||
return x >= self._W * 0.6 and y >= self._H * 0.6
|
||||
|
||||
def test_small_corner_sparkle_is_detected_and_localized(self):
|
||||
det = self.engine.detect_watermark(self._scene())
|
||||
assert det.detected
|
||||
# Must localize to the planted corner sparkle, not the larger left-side decoy.
|
||||
assert self._in_bottom_right(det.region), f"localized to decoy, not corner: {det.region}"
|
||||
assert abs(det.region[0] - self._CORNER[0]) < 16
|
||||
assert abs(det.region[1] - self._CORNER[1]) < 16
|
||||
|
||||
def test_promotion_is_what_rescues_it(self, monkeypatch):
|
||||
"""Guard the mechanism: disabling the override mislocalizes to the decoy.
|
||||
|
||||
Proves the scene genuinely needs the override (so the localization test above
|
||||
is not a fluke): with the gate set unreachable the larger decoy wins.
|
||||
"""
|
||||
scene = self._scene()
|
||||
assert self._in_bottom_right(self.engine.detect_watermark(scene).region)
|
||||
monkeypatch.setattr(GeminiEngine, "_CORNER_PROMOTE_NCC", 2.0)
|
||||
assert not self._in_bottom_right(self.engine.detect_watermark(scene).region), (
|
||||
"decoy expected to win without the override"
|
||||
)
|
||||
|
||||
def test_no_promotion_on_clean_flat_image(self):
|
||||
"""A flat image with no sparkle yields no corner match to promote."""
|
||||
flat = np.full((self._H, self._W, 3), 40, dtype=np.uint8)
|
||||
assert self.engine._corner_promote(flat, -1.0) is None
|
||||
Reference in new issue
Block a user