diff --git a/CLAUDE.md b/CLAUDE.md index b392bac..6e66a7e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,7 +42,7 @@ Compact map. The full per-module detail (design decisions, tuned thresholds, cal - `metadata.py` — `scan_head(path)` is the shared (memoized) input for every C2PA/AIGC/IPTC byte scan; use it instead of `open().read(1MB)` for any new marker scan. Also home to `synthid_source`, `xai_signature`, `iptc_ai_system`, `aigc_label`, `huggingface_job`, `samsung_genai`, and `remove_ai_metadata` (fail-safe `strip_c2pa_boxes`). - `identify.py` — aggregates every locally-readable signal into one `ProvenanceReport`; `is_ai_generated` is True or None, never asserted False. `import identify` is deliberately light (lazy `noai/__init__`, fits a 512 MB host) — keep heavy imports out. Add capture-camera tokens to `_DEVICE_C2PA_PLATFORM` only when verified against a real C2PA file; editing-app/AI-device signer tokens go to `_SIGNER_C2PA_PLATFORM`; generator/issuer platforms to `C2PA_AI_VENDORS` in `constants.py`. Integrity-clash detection is high-precision by design (only hard generator stamps feed it, source-grouped independence). - `watermark_registry.py` — the single catalog of known visible watermarks (gemini / doubao / jimeng / samsung), reverse-alpha based by policy. Add a new visible text mark = one `_text_mark(...)` row + a `TextMarkConfig` with a captured alpha map; do not re-add per-mark `if` branches. `cli._write_bgr_with_alpha` must NOT zero alpha in the watermark bbox (issue #30 white-box regression). -- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU): corner-promote, over/under-subtraction guards, false-positive gate, self-verify repair. Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Gate any removal candidate on a physical brightness check, not the detector alone. +- `gemini_engine.py` — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU): top-K size-weighted fusion candidate selection (`_SELECT_TOPK`), corner-promote, over/under-subtraction guards, false-positive gate, self-verify repair. Detection scores the top-K size-weighted matches by full fusion (spatial+gradient+variance) and keeps the highest — NOT the raw-NCC argmax, which re-admits the tiny-patch FPs the size weight suppresses (the osachub 2026-06-12 sub-0.85 corner-sparkle regression; see `docs/module-internals.md`). Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Gate any removal candidate on a physical brightness check, not the detector alone. - `_text_mark_engine.py` — shared base for the three reverse-alpha text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model). - `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Removal = reverse-alpha (always-align) + thin residual inpaint. A detector-only removal test is insufficient — assert visual residual (the textured-shift tests). - `region_eraser.py` — universal region eraser (`erase` CLI): cv2 backend default (no deps), optional big-LaMa via onnxruntime (~3.5-4 GB peak RAM, ~5-6 s/call CPU — does not fit a minimal droplet). diff --git a/docs/module-internals.md b/docs/module-internals.md index f86e3e9..0fe3b2a 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -71,6 +71,8 @@ module. **Detection localization (issue #36):** `detect_watermark`'s global multi-scale NCC search applies a size weight (`(scale/96)**0.5`) that suppresses tiny-patch false positives but can let a larger, mediocre match (e.g. a bright collar in a portrait) outrank a small, near-perfect sparkle in the corner — so a faint sparkle on a busy background scored below threshold and read as clean (the regression osachub reported from widening the search window 256px->512px between v0.7.2 and v0.8.8). `_corner_promote` adds a bottom-right-corner raw-NCC pass on top of the global search: a match with raw NCC >= `_CORNER_PROMOTE_NCC` 0.85 that beats the global pick overrides it (it only ever replaces a lower-fidelity pick, so it cannot weaken an existing detection), rescuing the buried sparkle without reverting the wider window. The corner side is **relative-clamped** (`_CORNER_PROMOTE_FRAC` 0.20 of the short side, clamped to `[_CORNER_PROMOTE_MIN` 96, `_CORNER_PROMOTE_MAX` 384`]`): a fixed 256px is a true corner on a large image but covers ~70% of a small portrait, where a real photo raw-matches the star at ~0.81 (relative tightening drops that worst case to ~0.69, while the upper clamp stops the corner ballooning on huge images where a real photo reached ~0.83 at 512px). The 0.85 gate sits midway between the worst real-photo corner match (~0.78 across native + downscaled negatives) and a genuine faint sparkle (~0.93), so promotion adds true detections with zero corpus false positives (Gemini's sparkle sits ~60-160px from the corner at fixed margins, covered by the [96, 384] band at every measured size). Regression-guarded by `test_gemini_engine.py::TestCornerPromotion`. +**Top-K fusion selection (osachub follow-up 2026-06-12):** `_corner_promote`'s 0.85 raw-NCC gate still missed a class the 256->512 widening exposed — a genuine MID-scale sparkle whose raw NCC sits *below* 0.85 but is buried by a LARGER, low-fidelity decoy that wins the size weight. The reporter's image (a scale-48 sparkle on light bedding) measured spatial 0.775 / grad 0.960 / fusion 0.676 at the true sparkle, but the size-weighted argmax instead locked onto a decoy at spatial 0.628 / grad 0.036 (fusion 0.325) — so `identify` read `unknown` on v0.8-0.11 where v0.7.2 (256px window) had caught it at 0.676. Fix: `detect_watermark` now keeps the **top-`_SELECT_TOPK` (3)** size-weighted candidates (NMS-deduped by location) plus the corner-promote candidate, scores EACH by the full fusion (spatial+gradient+variance) via the extracted `_grad_var_scores` helper, and selects the highest — the gradient term (the discriminator a contrast-invariant spatial NCC lacks) lifts the true sparkle over the decoy. Critically, selection ranks by the SIZE-WEIGHTED score, NOT raw NCC: a raw-NCC argmax (tried first) re-admitted the exact tiny-patch (scale 16-18) false positives the size weight exists to suppress — it flagged 14/65 doubao + 4/11 jimeng visible-corpus images (non-Gemini content) as Gemini sparkles. Top-K keeps tiny-patch suppression intact: a coincidental 16px match never ranks in the size-weighted top-K, so widening selection added **zero** flips on the doubao/jimeng corpora and left the 495-image Gemini set unchanged (479 detected, both before and after) while recovering the reporter's image. Regression-guarded by `test_gemini_engine.py::TestCornerPromotion::test_low_gradient_decoy_loses_to_high_gradient_corner_sparkle` (mirrors the real spatial/grad signature via a monkeypatched scan) and `test_size_weighted_search_alone_traps_on_the_decoy`. + **Square-image residual misses are NOT fixable by lowering the detector threshold (measured + REJECTED 2026-06-11):** osachub (#36 follow-up) reported the corner-promote still misses Gemini sparkles on Google **square (1:1)** outputs. Reproduced on the spaces corpus: of 330 square Google-C2PA images, 140 score below the identify 0.5 threshold, and visual review confirmed a real class -- faint white sparkles on dark/textured/colored backgrounds (raw NCC 0.46-0.73, below the 0.85 promote gate) landing at fusion conf 0.41-0.47. A margin-gated promote (promote when raw NCC >= 0.50 AND `_core_ring_margin` >= 40) rescued 32/33 confirmed misses at an apparent 0 FP, but that 0 was a **measurement artifact** -- the negative set was the margin<40 misses, which a margin>=40 gate excludes by construction. On an honest 518-image non-Google pool the same gate fired on **~174 (≈33%)**, visually content (screenshots, Chinese "AI生成" Doubao/Jimeng text marks, logos, bright textures), not sparkles. Adding an achromatic-core constraint (`chroma <= 15`) did not separate them either (kept 15/33 POS, 41 NEG still firing). Root cause is the documented contrast-invariant-NCC wall: a faint sparkle on a busy background is indistinguishable from a bright/ornate content corner at the (shape-NCC, brightness-margin, core-chroma) feature level. **Conclusion: keep the 0.85 corner gate; do NOT add a margin/chroma-gated lower promote.** diff --git a/src/remove_ai_watermarks/gemini_engine.py b/src/remove_ai_watermarks/gemini_engine.py index 834838f..c6f3833 100644 --- a/src/remove_ai_watermarks/gemini_engine.py +++ b/src/remove_ai_watermarks/gemini_engine.py @@ -218,6 +218,16 @@ class GeminiEngine: _CORNER_PROMOTE_MIN = 96 _CORNER_PROMOTE_MAX = 384 + # Number of top size-weighted spatial candidates scored by full fusion before one + # is selected. The single size-weighted argmax can bury a genuine mid-size sparkle + # under a LARGER, lower-fidelity shape match (the 256->512 search-widening + # regression: a real corner sparkle at raw ~0.77 lost to a decoy at raw ~0.63). + # Scoring the top-K by gradient-bearing fusion rescues it. Top-K (NOT the raw-NCC + # argmax) keeps the tiny-patch suppression intact: a coincidental 16 px match never + # ranks in the size-weighted top-K, so widening selection cannot add a false + # positive on non-Gemini content (verified on the doubao/jimeng visible corpora). + _SELECT_TOPK = 3 + def __init__(self, logo_value: float = 255.0) -> None: """Initialize the engine with embedded alpha maps. @@ -316,91 +326,65 @@ class GeminiEngine: gray_sr_f = gray_sr.astype(np.float32) / 255.0 - # Phase 1 & 2: multi-scale spatial NCC search, size-weighted argmax. - best_scale = 0 - best_score = -1.0 - best_raw_ncc = -1.0 - best_loc = (0, 0) + # Phase 1 & 2: multi-scale spatial NCC search. The size weight (mimicking the + # C++ vendor weight) overcomes the NCC bias toward tiny patches, but its single + # argmax can bury a genuine mid-size sparkle under a LARGER, lower-fidelity + # shape match (the 256->512 search-widening regression). So score the top-K + # size-weighted candidates by the FULL fusion and keep the highest -- the + # gradient term separates a true white sparkle from a shape-only decoy. See + # _SELECT_TOPK for why top-K (not the raw-NCC argmax) preserves tiny-patch + # suppression and so cannot add a false positive on non-Gemini content. + scored: list[tuple[float, int, int, int, float]] = [] # (adj, scale, raw, x, y) for scale, max_val, max_loc in self._scan_scales(gray_sr_f): - # Size-adjusted score to overcome NCC bias toward tiny patches (mimics C++ weight) - weight = min(1.0, (scale / 96.0) ** 0.5) - adj_val = max_val * weight - if adj_val > best_score: - best_score = adj_val - best_scale = scale - best_loc = max_loc - best_raw_ncc = max_val + adj_val = max_val * min(1.0, (scale / 96.0) ** 0.5) + scored.append((adj_val, scale, max_val, sx1 + max_loc[0], sy1 + max_loc[1])) + scored.sort(reverse=True) - # Exact dynamic location & size - pos_x = sx1 + best_loc[0] - pos_y = sy1 + best_loc[1] + # Top-K candidates at distinct locations (NMS: drop a lower-ranked match that + # overlaps an already-kept one -- the same sparkle matches at adjacent scales). + candidates: list[tuple[int, int, int, float]] = [] + for _adj, scale, raw, x, y in scored: + if any( + abs(x - px) < 0.5 * max(scale, ps) and abs(y - py) < 0.5 * max(scale, ps) + for ps, px, py, _ in candidates + ): + continue + candidates.append((scale, x, y, raw)) + if len(candidates) >= self._SELECT_TOPK: + break - # Corner promotion: a near-perfect but small sparkle in the bottom-right - # corner is otherwise outranked by a larger, mediocre size-weighted match - # (see _CORNER_PROMOTE_NCC). Override the global pick with it when present. - promoted = self._corner_promote(image, best_raw_ncc) + # Corner promotion: a near-perfect small bottom-right sparkle the size weight + # buries even below the top-K (see _CORNER_PROMOTE_NCC) -- add it as a candidate. + promoted = self._corner_promote(image, candidates[0][3] if candidates else -1.0) if promoted is not None: - best_scale, pos_x, pos_y, best_raw_ncc = promoted + candidates.append(promoted) + + # Select the candidate with the highest full-fusion confidence (pre-FP-gate). + best_scale, pos_x, pos_y, best_raw_ncc = candidates[0] + grad_score, var_score, best_fused = 0.0, 0.0, -1.0 + for c_scale, c_x, c_y, c_raw in candidates: + if c_raw < 0.25: + c_grad, c_var, c_fused = 0.0, 0.0, max(0.0, c_raw * 0.5) + else: + c_grad, c_var = self._grad_var_scores(image, c_scale, c_x, c_y) + c_fused = c_raw * 0.50 + c_grad * 0.30 + c_var * 0.20 + if c_fused > best_fused: + best_fused = c_fused + best_scale, pos_x, pos_y = c_scale, c_x, c_y + best_raw_ncc, grad_score, var_score = c_raw, c_grad, c_var result.region = (pos_x, pos_y, best_scale, best_scale) result.spatial_score = float(best_raw_ncc) - - # Generate exact alpha map for matched size - alpha_region = self.get_interpolated_alpha(best_scale) - - # Extract exactly the matched region for Gradient & Variance analysis - x1 = pos_x - y1 = pos_y - x2 = min(w, x1 + best_scale) - y2 = min(h, y1 + best_scale) - - region = image[y1:y2, x1:x2] - if len(region.shape) == 3 and region.shape[2] >= 3: - gray_region = cv2.cvtColor(region, cv2.COLOR_BGR2GRAY) - else: - gray_region = region.copy() - - gray_f = gray_region.astype(np.float32) / 255.0 - - # Adjust alpha_region if clipped by image boundary (rare, but possible) - ay1, ax1 = 0, 0 - alpha_region = alpha_region[ay1 : ay1 + (y2 - y1), ax1 : ax1 + (x2 - x1)] + result.gradient_score = float(grad_score) + result.variance_score = float(var_score) if result.spatial_score < 0.25: result.confidence = float(max(0.0, result.spatial_score * 0.5)) return result - # ── Stage 2: Gradient NCC ──────────────────────────────────── - img_gx = cv2.Sobel(gray_f, cv2.CV_32F, 1, 0, ksize=3) - img_gy = cv2.Sobel(gray_f, cv2.CV_32F, 0, 1, ksize=3) - img_gmag = cv2.magnitude(img_gx, img_gy) - - alpha_gx = cv2.Sobel(alpha_region, cv2.CV_32F, 1, 0, ksize=3) - alpha_gy = cv2.Sobel(alpha_region, cv2.CV_32F, 0, 1, ksize=3) - alpha_gmag = cv2.magnitude(alpha_gx, alpha_gy) - - grad_match = cv2.matchTemplate(img_gmag, alpha_gmag, cv2.TM_CCOEFF_NORMED) - _, grad_score, _, _ = cv2.minMaxLoc(grad_match) - result.gradient_score = float(grad_score) - - # ── Stage 3: Variance Analysis ─────────────────────────────── - var_score = 0.0 - ref_h = min(y1, best_scale) - - if ref_h > 8: - ref_region = image[y1 - ref_h : y1, x1:x2] - gray_ref = cv2.cvtColor(ref_region, cv2.COLOR_BGR2GRAY) if len(ref_region.shape) == 3 else ref_region - - _, s_wm = cv2.meanStdDev(gray_region) - _, s_ref = cv2.meanStdDev(gray_ref) - - if s_ref[0][0] > 5.0: - var_score = max(0.0, min(1.0, 1.0 - (s_wm[0][0] / s_ref[0][0]))) - - result.variance_score = float(var_score) - # ── Fusion ─────────────────────────────────────────────────── - confidence = result.spatial_score * 0.50 + result.gradient_score * 0.30 + var_score * 0.20 + # best_fused is the selected candidate's spatial*0.5 + grad*0.3 + var*0.2. + confidence = best_fused # False-positive gate: a low-confidence shape match whose core is NOT brighter # than its surroundings is a content false positive, not a white sparkle overlay. @@ -429,6 +413,49 @@ class GeminiEngine: return result + def _grad_var_scores( + self, + image: NDArray[Any], + scale: int, + pos_x: int, + pos_y: int, + ) -> tuple[float, float]: + """Return ``(gradient_score, variance_score)`` for a candidate sparkle. + + Factored out of ``detect_watermark`` so each top-K candidate can be scored by + the full fusion before one is selected. The gradient NCC correlates + Sobel-magnitude maps (shape fidelity, contrast-robust); the variance score + rewards a flat overlay region against the row band above it. + """ + h, w = image.shape[:2] + x1, y1 = pos_x, pos_y + x2, y2 = min(w, x1 + scale), min(h, y1 + scale) + region = image[y1:y2, x1:x2] + gray_region = cv2.cvtColor(region, cv2.COLOR_BGR2GRAY) if region.ndim == 3 and region.shape[2] >= 3 else region + gray_f = gray_region.astype(np.float32) / 255.0 + alpha_region = self.get_interpolated_alpha(scale)[: y2 - y1, : x2 - x1] + + # ── Gradient NCC ── + img_gmag = cv2.magnitude( + cv2.Sobel(gray_f, cv2.CV_32F, 1, 0, ksize=3), cv2.Sobel(gray_f, cv2.CV_32F, 0, 1, ksize=3) + ) + alpha_gmag = cv2.magnitude( + cv2.Sobel(alpha_region, cv2.CV_32F, 1, 0, ksize=3), cv2.Sobel(alpha_region, cv2.CV_32F, 0, 1, ksize=3) + ) + _, grad_score, _, _ = cv2.minMaxLoc(cv2.matchTemplate(img_gmag, alpha_gmag, cv2.TM_CCOEFF_NORMED)) + + # ── Variance ── + var_score = 0.0 + ref_h = min(y1, scale) + if ref_h > 8: + ref_region = image[y1 - ref_h : y1, x1:x2] + gray_ref = cv2.cvtColor(ref_region, cv2.COLOR_BGR2GRAY) if ref_region.ndim == 3 else ref_region + _, s_wm = cv2.meanStdDev(gray_region) + _, s_ref = cv2.meanStdDev(gray_ref) + if s_ref[0][0] > 5.0: + var_score = max(0.0, min(1.0, 1.0 - (s_wm[0][0] / s_ref[0][0]))) + return float(grad_score), float(var_score) + def _corner_promote( self, image: NDArray[Any], diff --git a/tests/test_gemini_engine.py b/tests/test_gemini_engine.py index b5359dc..ce746eb 100644 --- a/tests/test_gemini_engine.py +++ b/tests/test_gemini_engine.py @@ -539,20 +539,72 @@ class TestCornerPromotion: assert abs(det.region[0] - self._CORNER[0]) < 16 assert abs(det.region[1] - self._CORNER[1]) < 16 - def test_promotion_is_what_rescues_it(self, monkeypatch): - """Guard the mechanism: disabling the override mislocalizes to the decoy. + def test_size_weighted_search_alone_traps_on_the_decoy(self): + """Guard the mechanism: the size-weighted argmax ALONE mislocalizes to the decoy. - Proves the scene genuinely needs the override (so the localization test above - is not a fluke): with the gate set unreachable the larger decoy wins. + Proves the scene genuinely needs the dual-candidate fusion selection (so the + localization test above is not a fluke): replicating the old size-weighted-only + pick lands on the larger left-side decoy, while ``detect_watermark`` -- which + scores all top-K candidates by full fusion -- lands on the corner sparkle. """ scene = self._scene() + h, w = scene.shape[:2] + ss = min(min(w, h), 512) + sx1, sy1 = max(0, w - ss), max(0, h - ss) + gray = cv2.cvtColor(scene[sy1:h, sx1:w], cv2.COLOR_BGR2GRAY).astype(np.float32) / 255.0 + best = (-1.0, (0, 0)) + for scale, max_val, max_loc in self.engine._scan_scales(gray): + adj = max_val * min(1.0, (scale / 96.0) ** 0.5) + if adj > best[0]: + best = (adj, max_loc) + weighted_region = (sx1 + best[1][0], sy1 + best[1][1]) + assert not self._in_bottom_right((*weighted_region, 0, 0)), "size-weighted argmax expected to hit the decoy" + # The full detector, scoring both candidates by fusion, rescues the corner. assert self._in_bottom_right(self.engine.detect_watermark(scene).region) - monkeypatch.setattr(GeminiEngine, "_CORNER_PROMOTE_NCC", 2.0) - assert not self._in_bottom_right(self.engine.detect_watermark(scene).region), ( - "decoy expected to win without the override" - ) def test_no_promotion_on_clean_flat_image(self): """A flat image with no sparkle yields no corner match to promote.""" flat = np.full((self._H, self._W, 3), 40, dtype=np.uint8) assert self.engine._corner_promote(flat, -1.0) is None + + def test_low_gradient_decoy_loses_to_high_gradient_corner_sparkle(self, monkeypatch): + """Regression (reporter 2026-06-12, v0.7.2 detected / v0.8-0.11 missed): the + 256->512 search widening let a large, low-gradient size-weighted match outrank + a smaller, high-gradient corner sparkle whose raw NCC (~0.77) sits BELOW the + 0.85 corner-promote gate -- so a real Gemini sparkle on light bedding read as + clean. The detector now scores all top-K candidates by full fusion and keeps + the highest; the gradient term (~0.04 on flat content vs ~1.0 on a true + sparkle) rescues the corner. Mirrors the real signature: + decoy spatial 0.63 / grad 0.04 vs corner spatial 0.78 / grad 0.96. + """ + eng = self.engine + size = 500 + img = np.full((size, size, 3), 225, dtype=np.float32) + # Low-gradient smooth bright blob = a content false match the size weight favors. + blob = np.zeros((size, size), np.float32) + cv2.circle(blob, (89, 89), 60, 1.0, -1) + blob = cv2.GaussianBlur(blob, (0, 0), 30) + img += (blob * 25)[:, :, None] + # A real, high-gradient sparkle in the bottom-right corner. + cx, cy, cs = 430, 430, 48 + tmpl = cv2.resize(eng._alpha_large, (cs, cs), interpolation=cv2.INTER_AREA) + a = (tmpl * 0.6)[:, :, None] + img[cy : cy + cs, cx : cx + cs] = a * 255.0 + (1.0 - a) * img[cy : cy + cs, cx : cx + cs] + img = np.clip(img, 0, 255).astype(np.uint8) + dx, dy, ds = 40, 40, 98 + + # The two locations carry the real-signature gradient split. + assert eng._grad_var_scores(img, ds, dx, dy)[0] < 0.1 # decoy: shape-only, ~0.04 + assert eng._grad_var_scores(img, cs, cx, cy)[0] > 0.8 # corner sparkle: ~1.0 + + def fake_scan(_self, _gray): + yield ds, 0.66, (dx, dy) # large, low-fidelity: size-weighted adj 0.66 -> winner + yield cs, 0.80, (cx, cy) # small sparkle: raw 0.80 but adj 0.57 -> size-weight loses + + monkeypatch.setattr(GeminiEngine, "_scan_scales", fake_scan) + det = eng.detect_watermark(img) + # Full-fusion selection keeps the high-gradient corner sparkle, not the decoy. + assert det.detected + assert det.confidence >= 0.5 + assert abs(det.region[0] - cx) < 4, f"localized to decoy: {det.region}" + assert abs(det.region[1] - cy) < 4, f"localized to decoy: {det.region}"