diff --git a/CLAUDE.md b/CLAUDE.md index 4b989f7..7061d1c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -57,7 +57,7 @@ Compact map. The full per-module detail (design decisions, tuned thresholds, cal - `metadata.py` — `scan_head(path)` is the shared (memoized) input for every C2PA/AIGC/IPTC byte scan; use it instead of `open().read(1MB)` for any new marker scan. Also home to `synthid_source`, `xai_signature`, `iptc_ai_system`, `aigc_label`, `huggingface_job`, `samsung_genai`, and `remove_ai_metadata` (fail-safe `strip_c2pa_boxes`). **`remove_ai_metadata` is the SINGLE metadata stripper** (the legacy PIL-re-encoding `noai/cleaner` was deleted; the diffusion core and the public `noai.remove_ai_metadata` re-export now point here). It strips **losslessly** per container: ISOBMFF (HEIC/AVIF/MP4) blanks tokens / strips boxes in place; **JPEG uses `_strip_jpeg_metadata_lossless`** — a marker-segment walk that drops the AI-bearing APP segments (C2PA APP11, AI XMP APP1, IPTC APP13) and scrubs AI EXIF tags via piexif, copying the entropy-coded scan verbatim so **the pixels are bit-identical** (no DCT re-encode). This is what lets a `--strip-metadata` on a q100 removal output NOT crush it back to q75. PNG/WebP re-saves are already pixel-lossless. Regression: `tests/test_metadata.py::TestHasAiMetadata::test_jpeg_metadata_strip_is_pixel_lossless` (real grok/flux fixtures). `exif_generator` matches a VALUE against `AI_GENERATOR_TOKENS` across EXIF `Software`/`Make`/`Artist`/`ImageDescription`, XMP `CreatorTool`, AND PNG `tEXt` chunks (`Software`/`Source`/`Title`/`Description` — NovelAI stamps there, not EXIF). **Detection and removal must stay in parity:** a generator that stamps an AI-shaped VALUE under a non-AI KEY (NovelAI's `Title`/`Source`) is dropped on removal by `_is_ai_value` (value-token match, mirrors `exif_generator`), NOT by `_is_ai_key` alone — else the cleaned file still reads as that generator. Add a new no-C2PA generator = one `AI_GENERATOR_TOKENS` entry (use a distinctive token, e.g. `reve.com` not bare `reve`); detection and removal then both follow. Regression: `tests/test_metadata.py::TestExifGenerator::{test_novelai_png_text_chunk_detected,test_novelai_removal_parity}`. - `identify.py` — aggregates every locally-readable signal into one `ProvenanceReport`; `is_ai_generated` is True or None, never asserted False. `ProvenanceReport.ai_source_kind` exposes the C2PA digital-source-type split — `"generated"` (trainedAlgorithmicMedia, fully AI) vs `"enhanced"` (compositeWithTrainedAlgorithmicMedia, a real photo with an AI-composited region), else None — so a caller branches full-frame scrub vs region-targeted clean (see `noai/tiling.feather_region_composite` + `WatermarkRemover.remove_watermark(region=...)`). The sparkle provenance threshold is the SHARED `watermark_registry.GEMINI_SPARKLE_TRUST_CONF` (imported, not a private copy) so the provenance "is there a sparkle" verdict and the removal "take the sparkle" decision can never drift. `import identify` is deliberately light (lazy `noai/__init__`, fits a 512 MB host) — keep heavy imports out (the `watermark_registry` constant import stays light: engines are lazy there). Add capture-camera tokens to `_DEVICE_C2PA_PLATFORM` only when verified against a real C2PA file; editing-app/AI-device signer tokens go to `_SIGNER_C2PA_PLATFORM`; generator/issuer platforms to `C2PA_AI_VENDORS` in `constants.py`. Integrity-clash detection is high-precision by design (only hard generator stamps feed it, source-grouped independence). - `watermark_registry.py` — the single catalog of known visible watermarks (gemini / doubao / jimeng / samsung / jimeng_pill). **Removal is LOCALIZE -> FILL for every mark:** each mark is localized to a binary full-frame footprint mask (a `Localization`), then ONE shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). Reverse-alpha (the old `original = (wm - a*logo)/(1-a)` inversion of a captured alpha map + thin residual inpaint) is GONE for ALL marks; why it was dropped is recorded in `docs/module-internals.md`. Backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. The captured alpha maps (`scripts/visible_alpha_solve.py`) are still used to DETECT the marks and to shape the mask, but NOT for pixel recovery. **`--mark auto` removes EVERY detected mark in one pass** via `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` (marks coexist -- a Jimeng-basic image has the top-left pill AND the bottom-right wordmark; a single-strongest pick would leave one). **Three orthogonal axes:** `backend` (the fill), `sensitivity` (how hard to trust a borderline mark: `auto`/`strict`/`assume_ai`, see the `Sensitivity` literal), and `provenance` (vendor keys metadata confirms -- the evidence that drives `auto`). **Perception / decision / action are separated:** `_build_candidates(image)` runs every detector at BOTH trust levels (strict + relaxed) and packages raw verdicts + features into `Candidate`s (no policy); the pure arbiter `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` makes every keep/drop call (per-mark `resolve_relax` + the pill gate) with no image/IO, so it is unit-testable in isolation; then each winner is localized -> filled. Do NOT put policy back into the engines (the one exception, the Gemini FP gate, stays in `gemini_engine` because `identify` shares that confidence). `detect_marks(..., provenance=frozenset())` stays strict (identify verdict, precision over recall); `KnownMark.remove/detect/localize(..., provenance: bool)` take the already-resolved boolean. **How auto/assume decide (this is metadata-INDEPENDENT for recall):** the visual detectors are pixel-based and need no metadata; the recall gain comes from RELAXING the false-positive gate, not from metadata. `strict` never relaxes (clean images untouched); `auto` relaxes a mark only on same-product evidence -- metadata provenance for that vendor OR a confidently detected sibling mark of the SAME product (`_PRODUCT_OF`; Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax); `assume_ai` relaxes every mark (the caller asserts AI -- a metadata-stripped screenshot uploaded to a remover). Corpus finding: Gemini sparkle removal on Google-C2PA images is ~46% under `strict`/metadata-free `auto` and ~92% under `assume_ai` (recovering marks the vendor moved or re-rendered); the library CANNOT infer AI from a stripped image, so only the caller's `assume_ai` reaches the high recall there. A wrong relaxation just fills a small corner near-losslessly (the localize -> fill benign failure mode), which is what makes `assume_ai` acceptable. Metadata provenance mapping (feeds `auto`, read by `cli._visible_provenance`): Google/Gemini C2PA issuer -> gemini; China-AIGC (TC260) label -> doubao/jimeng; `samsung_genai` -> samsung. **The `jimeng_pill` is CAPTURE-LESS** (`pill_engine.py`): the top-left "AI生成" label has no captured alpha map, so it is detect-by-synthetic-silhouette; its footprint is a fixed top-left geometry box. Its weak edge-NCC detector (~7% raw false-fire) is gated in `remove_auto_marks` via **`_keep_pill`** (32k real-upload corpus validation 2026-07): the pill never rides on a **Doubao** detection, and has confirmation arms because metadata/intent confirms the platform, not pill presence. **(1) Bottom-right "★ 即梦AI" wordmark fired** — ~94% precise and survives **metadata-STRIPPED uploads** (screenshots / re-saves, ~61% of pills carry a detectable wordmark): remove **unrestricted**. **(2) TC260 metadata confirms Jimeng** (`"jimeng" in provenance`, no wordmark) **OR the caller asserts AI** (`sensitivity == "assume_ai"`) — the metadata-only arm is only ~27% precise and its false fires are **textured ceilings/walls that the fill visibly SMEARS**, so remove **only when the top-left footprint is flat enough for an invisible fill** (`pill_engine.footprint_is_flat`, median-Sobel texture ≤ `_FLAT_TEXTURE_MAX`) — the flatness guard holds even under `assume_ai`. This keeps real flat-scene pills (incl. metadata-only ones the wordmark misses) plus harmless flat false fires, and leaves the damaging textured false fires untouched. Do NOT drop the wordmark arm or loosen the flatness guard. `cli._write_bgr_with_alpha` must NOT zero alpha in the watermark bbox (issue #30 white-box regression). **The localizer is cheap CPU (cv2/numpy), so a memory-tight caller runs it anywhere; the heavy MI-GAN/LaMa fill is opt-in and chosen by the caller** (a small worker can use cv2; a GPU/model worker can use MI-GAN/LaMa). Adding a new mark still needs a real detection template: a new alpha-solved silhouette from `scripts/visible_alpha_solve.py` (solid black + gray (+white) captures of the mark produced by the actual app/device at native resolution, committed under `data/_capture/captures/`) for a captured mark, or a synthetic font-rendered silhouette for a capture-less one like the pill. Do NOT synthesize a captured mark's detection template by font-rendering the wordmark (the user rejected synthetic reconstruction as below the quality bar, 2026-06-22), and do NOT derive one from user uploads (data-safety: no corpus-derived committed assets). So if no flat capture exists for a mark, the work is parked — do not propose synthetic, do not derive from user uploads. (Meta AI, more Samsung locales, and any Grok visible mark are all parked on this; Grok additionally needs confirming it even HAS a visible mark — its known signal is EXIF-only `xai_signature`.) -- `gemini_engine.py` — visible Gemini-sparkle detector + localizer (cv2/numpy, no GPU): top-K size-weighted fusion candidate selection (`_SELECT_TOPK`), corner-promote, false-positive gate (the provenance prior relaxes the gate + lowers the trust threshold when a Google/Gemini C2PA issuer confirms the vendor). Detection scores the top-K size-weighted matches by full fusion (spatial+gradient+variance) and keeps the highest — NOT the raw-NCC argmax, which re-admits the tiny-patch FPs the size weight suppresses (the osachub 2026-06-12 sub-0.85 corner-sparkle regression; see `docs/module-internals.md`). Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Removal is localize -> fill: `footprint_mask` returns the sparkle footprint (the captured alpha thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin), and the shared `watermark_registry.fill` inpaints it. The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. +- `gemini_engine.py` — visible Gemini-sparkle detector + localizer (cv2/numpy, no GPU): top-K size-weighted fusion candidate selection (`_SELECT_TOPK`), corner-promote, false-positive gate (the provenance prior relaxes the gate + lowers the trust threshold when a Google/Gemini C2PA issuer confirms the vendor). **White-core rescue:** the FP gate demotes a low-gradient match (soft edges), but a real FAINT sparkle also has soft edges -- so the gate keeps a low-grad match that is a strong (conf ≥ `_SPARKLE_KEEP_CONF` 0.52), bright (margin), near-WHITE-core sparkle (`_core_saturation` ≤ `_SPARKLE_WHITE_SAT` 0.20): a real sparkle core is white, a clean bright corner that shape-matches (sky/sun) is colored. This recovers ~14/20 metadata-stripped faint sparkles under the DEFAULT strict/auto (no flag, no metadata) at ~1.25% clean false-fire (baseline 0.55%); the ~0.51-scoring bright-bg FPs stay demoted (below 0.52). A learned classifier on the SAME features was measured WORSE than the tuned gate (2026-07 tier-1: MLP 86.7% recall vs 90.8% at equal FP), so the heuristic stays; a patch-CNN with richer features is the only lever left (roadmapped P2, low expected value -- the wall is fundamental). Detection scores the top-K size-weighted matches by full fusion (spatial+gradient+variance) and keeps the highest — NOT the raw-NCC argmax, which re-admits the tiny-patch FPs the size weight suppresses (the osachub 2026-06-12 sub-0.85 corner-sparkle regression; see `docs/module-internals.md`). Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Removal is localize -> fill: `footprint_mask` returns the sparkle footprint (the captured alpha thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin), and the shared `watermark_registry.fill` inpaints it. The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. - `_text_mark_engine.py` — shared base for the three text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. Detection still matches the glyph silhouette (NCC, keys on glyph shape). The removal mask is TEMPLATE-FREE: it is the bounding box of the top-hat glyph blob (`extract_mask`), filled solid + dilated, so the shared fill inpaints the whole wordmark rectangle. This drops the fixed alpha-template placement, so a re-rendered or differently-placed mark is still masked; the captured alpha maps are now used only for the detection silhouette, not for removal. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model). - `pill_engine.py` — the CAPTURE-LESS Jimeng-basic "AI生成" pill (top-left, issue #54). No alpha map: `detect` is edge-NCC of a synthetic font-rendered silhouette (`assets/jimeng_pill.png`, regenerate via `scripts/render_pill_silhouette.py`; committed, data-safe -- corpus stays out of the repo) in the top-left ROI, calibrated on 61 local real positives to threshold 0.22; `footprint_mask` is a generous FIXED top-left geometry box (NOT the NCC match position -- the synthetic silhouette localizes only approximately, the corner is negative space, so a geometry box fills cleanly while a match box leaves outline residue). `footprint_texture`/`footprint_is_flat` (median-Sobel over that box, `_FLAT_TEXTURE_MAX`) back the metadata-only safe-fill gate. Removal is the shared localize -> fill (MI-GAN/cv2). Detector precision is weak (~7% raw false-fire), so it is registry-gated in `remove_auto_marks` via `_keep_pill`: never on Doubao; the bottom-right wordmark removes it unrestricted (~94% precise, survives metadata-STRIPPED uploads); TC260-metadata-only removes it ONLY on a flat footprint (its textured false fires -- ceilings/walls -- are what the fill smears). Do NOT loosen those gates. - `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Detection matches the glyph silhouette (NCC); removal localizes the glyph blob to a solid dilated box (`extract_mask`) and hands it to the shared fill. Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. **Samsung detection is calibrated only for the Italian "Contenuti generati dall'AI" string** (a pre-existing limit, unchanged by the localize -> fill refactor but now surfaced because detection gates removal): non-Italian Samsung locales are not detected, and thus not removed, even though the fill mask itself is locale-independent; other locales need their own captured detection template. @@ -81,7 +81,7 @@ Who embeds what (C2PA / IPTC / EXIF / TC260 AIGC / xAI signature / open and prop Compact list. Full measurements, incident history, and oracle-validation runs live in `docs/known-limitations.md` — **read the relevant section there before changing the diffusion pipelines, strength defaults, resolution handling, or metadata coverage.** -- **Visible-mark fill quality is background/backend-dependent.** The fill only touches the mark footprint (no outside-box damage) and whether the mark is removed is fill-independent — cv2/MI-GAN/LaMa all strip the shape; only the recovered region's *quality* differs. Flat backgrounds: all clean (cv2 often crispest). Textured/regular-structured (fabric, grid): cv2 smears, MI-GAN can ghost/hallucinate, LaMa best. The old reverse-alpha recovered true pixels so it was sometimes cleaner on structure, but localize -> fill trades that for robustness (moved/re-rendered marks, no per-mark capture); `auto` = LaMa > MI-GAN > cv2 with a one-time cv2-fallback warning. Head-to-head vs v0.12.1 on the full visible set: doubao/jimeng identical (100%/100%), gemini strict coverage a few points lower (all recovered via `assume-ai`), clearance ~98% both. Detail in `docs/known-limitations.md`. +- **Visible-mark fill quality is background/backend-dependent.** The fill only touches the mark footprint (no outside-box damage) and whether the mark is removed is fill-independent — cv2/MI-GAN/LaMa all strip the shape; only the recovered region's *quality* differs. Flat backgrounds: all clean (cv2 often crispest). Textured/regular-structured (fabric, grid): cv2 smears, MI-GAN can ghost/hallucinate, LaMa best. The old reverse-alpha recovered true pixels so it was sometimes cleaner on structure, but localize -> fill trades that for robustness (moved/re-rendered marks, no per-mark capture); `auto` = LaMa > MI-GAN > cv2 with a one-time cv2-fallback warning. Head-to-head vs v0.12.1 on the full visible set: doubao/jimeng identical (100%/100%), gemini strict coverage a few points lower (the metadata-stripped faint ones now mostly recovered by the default white-core rescue in the gemini FP gate; the residual via `assume-ai`), clearance ~98% both. Detail in `docs/known-limitations.md`. - `invisible` processes at native resolution for inputs >= 1024px long side and auto-upscales smaller inputs to a 1024px floor (`--min-resolution 0` disables; `--max-resolution N` is an opt-in cap to bound GPU/MPS memory). MPS OOM is memory-tier dependent, not a hard limit: ~24 GB unified memory falls back to CPU (slow but weight-identical output), 32 GB runs native on MPS. The native-vs-cap-vs-floor decision lives in the pure helper `invisible_engine._target_size` — keep the logic there, unit-tested without the model. For large inputs that OOM, `--tile` is the **lossless** alternative to `--max-resolution`: sliding-window diffusion at native resolution, each tile near SDXL's 1024 training size, feather-blended over the overlap (`noai/tiling.py`). It only engages when the long side exceeds `--tile-size`; the geometry (`plan_tiles`) and the blend window (`feather_weights`) are pure and unit-tested (`tests/test_tiling.py`). Caveat: each tile is an independent low-strength regeneration, so at the certified removal strengths (0.20-0.30) tile drift is minimal but not zero; tiling is a memory workaround, not a quality upgrade over a single native pass. - fp16 VAE black-output (issues #29/#41): the fp16-fixed SDXL VAE (`madebyollin/sdxl-vae-fp16-fix`) is swapped in for the default SDXL checkpoint on cuda/xpu fp16, plus a model-agnostic backstop that detects a degenerate (all-black) fp16 output and re-runs once in fp32. cpu/mps run fp32 and never reproduce the bug. - Pyright first run is slow (2-3 min) due to ML deps (torch/diffusers/transformers stubs); full-project `uv run pyright` can stall for many minutes — scope it to changed files. diff --git a/docs/module-internals.md b/docs/module-internals.md index 1a1b2bf..f444f43 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -63,7 +63,7 @@ module. **Localize -> fill by policy (replaced reverse-alpha):** each mark is localized to a binary full-frame footprint mask (a `Localization`), and one shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). This replaced the old reverse-alpha removal (invert a captured alpha map, `original = (wm - a*logo)/(1-a)`, plus a thin residual inpaint) for ALL marks — gemini, doubao, jimeng, samsung, and jimeng_pill. **Why it changed:** reverse-alpha depended on a fixed captured alpha map at a fixed position, so it broke whenever a vendor moved or re-rendered its mark; and it was not color-lossless even with the right map (it amplifies 8-bit quantization and JPEG-chroma error by `1/(1-a)`), which showed up as "the color just changed, not removed" reports. Localize -> fill has a benign failure mode: a slightly-off localization just inpaints a small region near-losslessly instead of leaving a color-shifted smear. The captured alpha maps are still used to DETECT the marks and to shape the mask (gemini's footprint), but NOT for pixel recovery. Fill backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. Each `KnownMark` ties a key to {usual `location`, `in_auto` flag, a `_detect` callable → uniform `MarkDetection`, a `_mask` callable → full-frame footprint mask}; `KnownMark.remove(image, *, backend="auto", provenance=False, force=False)`. Entries today: `gemini` (bottom-right sparkle), `doubao` (bottom-right "豆包AI生成"), `jimeng` (bottom-right "★ 即梦AI"), `samsung` (bottom-**LEFT** "✦ Contenuti generati dall'AI", Samsung Galaxy AI, Italian locale), and the capture-less `jimeng_pill` (top-left "AI生成"). `detect_marks(image, *, provenance=frozenset())` scans all (strict, for the identify verdict); `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` removes every detected mark in one pass. **Sensitivity (`auto`/`strict`/`assume_ai`)** decides how hard a borderline mark is trusted: the visual detectors are pixel-based (no metadata needed) and the recall gain comes from relaxing the false-positive gate, not from metadata. `resolve_relax` turns the policy + evidence into the per-mark relaxation boolean — `strict` never relaxes, `assume_ai` always relaxes (the caller asserts AI, e.g. a metadata-stripped screenshot; ~46% -> ~92% Gemini recall, at the cost of a small near-lossless fill on some clean corners), `auto` relaxes only on same-product evidence (metadata provenance for that vendor, or a confidently strict-detected sibling of the same `_PRODUCT_OF` — Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax). **Perception / decision / action are separated three ways** (the removal path only; `identify` keeps calling `KnownMark.detect` directly, so its verdict is untouched): `_build_candidates(image)` is PERCEPTION — it runs each detector at both trust levels and packages raw verdicts + the pill's flatness feature into `Candidate`s, no policy; `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` is the pure DECISION arbiter — all keep/drop policy (`resolve_relax` cross-mark corroboration + the pill gate) in one image-free, unit-testable function (`tests/test_watermark_registry.py::TestArbiter`); then `remove_auto_marks` does the ACTION, localizing -> filling each winner. The Gemini FP gate deliberately stays inside `gemini_engine` (not the arbiter) because `identify` reads that same gated confidence — pulling it out would drift the provenance verdict. Behavior is byte-identical to the pre-arbiter two-pass (corpus-verified: strict/auto 46%, assume_ai 92% Gemini recall unchanged). -**Head-to-head validation (v0.12.1 reverse-alpha vs the current localize -> fill):** run over the full labelled visible-mark set, with the cv2 / MI-GAN / LaMa fills each compared against the old reverse-alpha. **doubao and jimeng are identical** across every backend -- 100% coverage and 100% clearance either way. **gemini** strict coverage is a few points below reverse-alpha's (the deliberate false-positive tightening), but every strict-missed mark is recovered under `assume_ai`, clearance is equal (~98% both), and neither version touches pixels outside the mark box (outside-box PSNR ~99). **Clearance is fill-independent** -- cv2, MI-GAN and LaMa all strip the mark's shape equally, so the re-detect metric does not separate them; the difference is purely the *visual fill quality* on the recovered region, and it is background-dependent. reverse-alpha recovered textured and especially regular/structured backgrounds (a lattice, a grid) more cleanly than any inpaint; **LaMa closes most of that gap** (the best learned backend), **MI-GAN can ghost or hallucinate structure**, and **cv2 smears** (the last-resort floor). This is why `auto` resolves `LaMa > MI-GAN > cv2` (`preferred_inpaint_backend`) and warns once on the cv2 fallback; on flat backgrounds every backend is clean. +**Head-to-head validation (v0.12.1 reverse-alpha vs the current localize -> fill):** run over the full labelled visible-mark set, with the cv2 / MI-GAN / LaMa fills each compared against the old reverse-alpha. **doubao and jimeng are identical** across every backend -- 100% coverage and 100% clearance either way. **gemini** strict coverage is a few points below reverse-alpha's (the deliberate false-positive tightening), but the metadata-stripped faint ones are now mostly recovered by the DEFAULT white-core rescue in the FP gate (`gemini_engine`: a bright near-WHITE core distinguishes a real faint sparkle from a colored bright corner -- ~14/20 recovered at ~1.25% clean false-fire; a learned classifier on the same features measured worse, 2026-07 tier-1), the residual under `assume_ai`; clearance is equal (~98% both), and neither version touches pixels outside the mark box (outside-box PSNR ~99). **Clearance is fill-independent** -- cv2, MI-GAN and LaMa all strip the mark's shape equally, so the re-detect metric does not separate them; the difference is purely the *visual fill quality* on the recovered region, and it is background-dependent. reverse-alpha recovered textured and especially regular/structured backgrounds (a lattice, a grid) more cleanly than any inpaint; **LaMa closes most of that gap** (the best learned backend), **MI-GAN can ghost or hallucinate structure**, and **cv2 smears** (the last-resort floor). This is why `auto` resolves `LaMa > MI-GAN > cv2` (`preferred_inpaint_backend`) and warns once on the cv2 fallback; on flat backgrounds every backend is clean. **Provenance prior:** when local metadata already confirms the vendor, the mark's detection trust gate is relaxed (a confirmed vendor means the mark is present with high prior, so a mark the conservative detector would demote as a content false positive is trusted). `detect_marks` / `remove_auto_marks` take a `provenance` frozenset and `KnownMark.remove` a `provenance` flag. Mapping: a Google/Gemini C2PA issuer relaxes gemini (skips its false-positive gate and lowers the trust threshold from 0.5 to 0.35); a China-AIGC (TC260) label relaxes doubao/jimeng; `samsung_genai` relaxes samsung. Corpus finding: on Google-C2PA images, Gemini sparkle recall rose from ~46% (plain detector) to ~90% with the provenance prior (recovering marks the vendor moved or re-rendered). The localizer is cheap CPU (cv2/numpy), so a memory-tight caller runs it anywhere; the heavy MI-GAN/LaMa fill is opt-in and chosen by the caller. diff --git a/src/remove_ai_watermarks/gemini_engine.py b/src/remove_ai_watermarks/gemini_engine.py index 29a642b..427db72 100644 --- a/src/remove_ai_watermarks/gemini_engine.py +++ b/src/remove_ai_watermarks/gemini_engine.py @@ -167,6 +167,21 @@ class GeminiEngine: # confidence >= 0.65, above the gate). _SPARKLE_FP_GRAD = 0.55 + # White-core rescue for the gate above. A real but FAINT sparkle -- a soft white + # star on a bright/textured background -- has a high core-ring margin but low + # gradient fidelity, the SAME signature the grad gate uses to demote the smooth + # colored-corner FP, so faint real sparkles get demoted with it. The separator the + # grad gate discards is the CORE COLOR: a real Gemini sparkle core is near-WHITE + # (low saturation), while a clean bright corner that shape-matches (sky, sun, a warm + # light) is COLORED. So do NOT demote a low-grad match that already clears the trust + # confidence (_SPARKLE_KEEP_CONF -- the registry's 0.5 sparkle gate plus a small + # margin so the ~0.51 bright-background FPs the grad gate was added for stay demoted) + # AND has a bright (margin) near-neutral core (_core_saturation <= _SPARKLE_WHITE_SAT). + # Corpus-measured on metadata-stripped faint sparkles: recovers ~14/20 the low-grad + # demotion would drop, at ~0.8% clean false-fire vs the ~0.55% baseline. + _SPARKLE_KEEP_CONF = 0.52 + _SPARKLE_WHITE_SAT = 0.20 + # Corner promotion (issue #36): the size weight that suppresses tiny-patch # false positives also buries a small, near-perfect sparkle when a larger, # mediocre match sits elsewhere (e.g. a bright collar in a portrait). A small @@ -381,18 +396,26 @@ class GeminiEngine: # FP (low margin) AND the bright-background smooth-blob FP (high margin but low # gradient), which the margin check alone misses. See _SPARKLE_FP_GRAD. if confidence < self._SPARKLE_FP_CONF and not trust_provenance: - margin = self._core_ring_margin(image, self.get_interpolated_alpha(best_scale), (pos_x, pos_y)) + alpha = self.get_interpolated_alpha(best_scale) + pos = (pos_x, pos_y) + margin = self._core_ring_margin(image, alpha, pos) low_margin = margin is not None and margin < self._SPARKLE_FP_MARGIN low_grad = grad_score < self._SPARKLE_FP_GRAD if low_margin or low_grad: - logger.debug( - "Sparkle FP gate: conf=%.3f, core-ring margin=%s, grad=%.3f < %.2f; demoting.", - confidence, - f"{margin:.1f}" if margin is not None else "n/a", - grad_score, - self._SPARKLE_FP_GRAD, - ) - confidence = min(confidence, 0.30) + # White-core rescue: a real faint sparkle clears the trust confidence, + # has a bright core (not low_margin), and a near-WHITE core -- unlike the + # colored-corner FP the low-grad demotion targets. See _SPARKLE_WHITE_SAT. + core_sat = self._core_saturation(image, alpha, pos) + white_core = not low_margin and core_sat is not None and core_sat <= self._SPARKLE_WHITE_SAT + if not (confidence >= self._SPARKLE_KEEP_CONF and white_core): + logger.debug( + "Sparkle FP gate: conf=%.3f, margin=%s, grad=%.3f, core_sat=%s; demoting.", + confidence, + f"{margin:.1f}" if margin is not None else "n/a", + grad_score, + f"{core_sat:.2f}" if core_sat is not None else "n/a", + ) + confidence = min(confidence, 0.30) result.confidence = float(max(0.0, min(1.0, confidence))) result.detected = result.confidence >= 0.35 @@ -628,6 +651,35 @@ class GeminiEngine: cb = self._core_and_bg(image, alpha_map, position) return None if cb is None else cb[0] - cb[1] + def _core_saturation( + self, + image: NDArray[Any], + alpha_map: NDArray[Any], + position: tuple[int, int], + ) -> float | None: + """Median color saturation of the sparkle core (0 = white/neutral, higher = + colored). A real Gemini sparkle is a white star, so its core is near-neutral; + a clean bright corner that shape-matches (sky, sun, a warm light) is colored, + so a high core saturation flags the false positive the brightness/gradient + gates miss. Samples the same high-alpha core pixels as :meth:`_core_and_bg`. + None when the footprint cannot be placed or the core is empty. + """ + placed = self._footprint_indices(alpha_map, position, image.shape) + if placed is None: + return None + alpha_roi, (y1, y2, x1, x2) = placed + a_cap = float(alpha_roi.max()) + if a_cap < 0.2: + return None + core = alpha_roi >= a_cap * self._CORE_ALPHA_FRAC + box = image[y1:y2, x1:x2] + if box.shape[:2] != core.shape or not bool(core.any()): + return None + px = box[core].astype(np.float32) # (N, 3) BGR core pixels + hi = px.max(axis=1) + lo = px.min(axis=1) + return float(np.median((hi - lo) / (hi + 1.0))) + def detect_sparkle_confidence(image_path: Path, *, image: NDArray[Any] | None = None) -> float | None: """Visible-sparkle detection confidence for a file, for provenance use. diff --git a/tests/test_gemini_engine.py b/tests/test_gemini_engine.py index a50ac02..52b8442 100644 --- a/tests/test_gemini_engine.py +++ b/tests/test_gemini_engine.py @@ -292,14 +292,15 @@ class TestSparkleFalsePositiveGate: assert det.confidence < 0.5 assert not det.detected - def test_bright_background_low_gradient_match_demoted(self): - """Bright-background content FP (2026-06-26 landing-page reports: a snow+sky - photo and a white product render scored ~0.51). A bright background gives the - match a HIGH core-ring margin, so the margin gate cannot demote it -- but the - smooth blob lacks the crisp star edges of a real sparkle, so its GRADIENT NCC - is low. The gradient gate (``_SPARKLE_FP_GRAD``) demotes it. Reproduces the - regime with a heavily-blurred bright sparkle: high margin, low gradient, and a - pre-gate fusion above the 0.5 promote bar (so it WOULD have been detected). + def test_bright_white_blurred_sparkle_rescued(self): + """A strong, bright, WHITE blurred sparkle is now KEPT (white-core rescue). A + faint / re-compressed real sparkle has soft edges (low gradient) like a smooth + blob, but its core is bright AND near-WHITE and its fused confidence clears the + trust gate, so the rescue (``_SPARKLE_WHITE_SAT`` / ``_SPARKLE_KEEP_CONF``) keeps + it -- recovering the metadata-stripped faint sparkles the grad gate used to drop. + The weaker (~0.51) bright-background FPs the grad gate was added for still get + demoted, because they fall below ``_SPARKLE_KEEP_CONF`` (0.52); the colored-blob + case below is demoted on core saturation. """ size = 1400 config = get_watermark_config(size, size) @@ -311,16 +312,28 @@ class TestSparkleFalsePositiveGate: img[y : y + ah, x : x + aw] = a * 255.0 + (1.0 - a) * img[y : y + ah, x : x + aw] img = cv2.GaussianBlur(np.clip(img, 0, 255).astype(np.uint8), (39, 39), 0) det = self.engine.detect_watermark(img) - # Pre-gate fusion clears the 0.5 promote bar (would have been detected)... - pre = det.spatial_score * 0.5 + det.gradient_score * 0.3 + det.variance_score * 0.2 - assert pre > 0.5 - # ...the OLD margin gate cannot catch it (bright background -> high margin)... - margin = self.engine._core_ring_margin(img, self.engine.get_interpolated_alpha(det.region[2]), det.region[:2]) - assert margin is not None - assert margin >= self.engine._SPARKLE_FP_MARGIN - # ...but the gradient is low (smooth blob, not a crisp star)... + assert det.gradient_score < self.engine._SPARKLE_FP_GRAD # soft edges, low gradient + assert det.confidence >= self.engine._SPARKLE_KEEP_CONF # strong white-core match, kept + assert det.detected + + def test_bright_colored_blob_still_demoted(self): + """The white-core rescue must NOT re-admit a COLORED bright-corner FP (sky, sun, + a warm light): its core saturation is high, so the rescue does not apply and the + gradient gate still demotes it. Same blurred construction as the white sparkle + above, but tinted -- proving the discriminator is core color, not just brightness. + """ + size = 1400 + config = get_watermark_config(size, size) + x, y = config.get_position(size, size) + alpha = self.engine.get_alpha_map(WatermarkSize.LARGE) + ah, aw = alpha.shape[:2] + img = np.full((size, size, 3), 110, dtype=np.float32) + a = np.clip(alpha * 1.6, 0.0, 1.0)[:, :, None] + color = np.array([40.0, 140.0, 235.0]) # BGR warm/orange -> saturated (non-white) core + img[y : y + ah, x : x + aw] = a * color + (1.0 - a) * img[y : y + ah, x : x + aw] + img = cv2.GaussianBlur(np.clip(img, 0, 255).astype(np.uint8), (39, 39), 0) + det = self.engine.detect_watermark(img) assert det.gradient_score < self.engine._SPARKLE_FP_GRAD - # ...so the gradient gate demotes it below the detection bar. assert det.confidence < 0.5 assert not det.detected