fix(identify): kill three visible-detector false positives

Bright-background photos/renders and a tiny app icon were flagged as
AI-generated by the visible detectors. Two failure modes:

- Gemini sparkle on a bright background (snow+sky photo, white product
  render) scored ~0.51. The FP gate only demoted on a low core-ring
  brightness margin, which a bright background makes high. Add a gradient
  floor (_SPARKLE_FP_GRAD 0.55): a real sparkle is a crisp star (grad
  ~0.97-1.0), a smooth luminance blob that NCC-matches the diamond is not
  (the two FPs measured grad 0.105 / 0.463). The OR is a strict superset
  of the old margin-only demotion, so it cannot regress dark/mid (kept by
  margin) or white-bg (kept by confidence) real sparkles.

- A 48x48 geometric icon matched the Doubao/Jimeng CJK silhouette at
  0.41/0.47 NCC. Purely a small-size artifact (the same icon at >=256px
  collapses to ~0.06-0.10). Guard text-mark detection below a 200px short
  side (_MIN_DETECT_SHORT_SIDE); real marks ship on full-resolution
  renders (smallest captured sample 1086px).

Corpus re-sweep flips only OpenAI content and already-cleaned outputs,
all sub-0.5, so no provenance verdict changes. Add synthetic regression
fixtures for both modes; docs/module-internals.md updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Victor Kuznetsov
2026-06-26 09:44:24 -07:00
co-authored by Claude Opus 4.8
parent f269b75ded
commit 0f54c6b54d
7 changed files with 125 additions and 8 deletions
+2 -2
View File
@@ -91,7 +91,7 @@ The cost (mislabel ~8-33% of non-Gemini content as Gemini) outweighs the benefit
**Under-subtraction (the symmetric case, fixed 2026-06-03):** some real Gemini sparkles are rendered MORE opaque than the captured ~0.51, so the fixed-alpha reverse blend UNDER-subtracts and leaves a bright sparkle residual the detector still fires on (measured on the spaces corpus: a visible-removal audit through the registry path left a detectable sparkle on a meaningful fraction of marks, all under-removals, NOT a background-brightness class — failures and successes had the same input confidence and the same background-luma distribution; the discriminator was the removal delta itself). `remove_watermark` now estimates a per-image alpha gain (`_estimate_alpha_gain`: effective sparkle opacity at the bright core vs the local background ring, `a_eff/a_cap`, clamped `[1.0, _ALPHA_GAIN_MAX` 1.94`]`) and scales the alpha to match before the over-sub/blend branch. The gain cleanly separates on the corpus (under-removed marks ~1.47, cleanly-removed ~1.00), and a deadband (`_ALPHA_GAIN_DEADBAND` 1.05) keeps a matching sparkle **byte-identical** to the pre-fix output, so the fix is purely additive (0 regressions on the audit set; the over-sub guard still runs on the scaled alpha as the safety net for an over-shooting estimate). Regression-guarded by `test_gemini_engine.py::TestUnderSubtractionGain` (composites a more-opaque-than-capture sparkle; **asserts on footprint pixels, NOT the detector** — the detector's NCC is degenerate on a flat synthetic background, so a re-detect conf is meaningless there; the real corpus removal drops the detector from ~0.80 to ~0.27).
**False-positive gate (added 2026-06-03):** `detect_watermark`'s shape-only NCC (`spatial*0.5 + gradient*0.3 + var*0.2`) fires on ornate/flat content (text strips, banners, hatching) that coincidentally matches the diamond shape — a real Gemini sparkle is a bright WHITE overlay, so its core sits above the local background, but the NCC is contrast-invariant and cannot see that. The fusion now **demotes** (caps confidence to 0.30) any match that is BOTH low-confidence (`< _SPARKLE_FP_CONF` 0.65) AND has a low core-ring brightness margin (`_core_ring_margin < _SPARKLE_FP_MARGIN` 5). Real sparkles escape via EITHER high confidence (white-bg sparkles score ≥0.79 despite a low margin — the NCC shape match is strong) OR high margin (dark/mid backgrounds, incl. the #36 faint-corner case, lift well clear), so BOTH must fail to demote. The gate is **monotonic** (only ever removes detections, never adds), so it cannot regress the verified-negative corpus (already 0 FPs). On the spaces corpus it demoted 16/495 flagged sparkles (13 carried no AI metadata = content FPs; the 3 AI-meta were visually FPs / a near-invisible white-on-white sparkle whose AI verdict is held by metadata anyway), and dropped the removal-audit failures 20→15 (post-removal flat footprints the NCC re-fired on). `_core_ring_margin` and `_estimate_alpha_gain` share the `_core_and_bg` helper (core 75th-pct brightness vs background-ring median). Regression-guarded by `test_gemini_engine.py::TestSparkleFalsePositiveGate`.
**False-positive gate (added 2026-06-03):** `detect_watermark`'s shape-only NCC (`spatial*0.5 + gradient*0.3 + var*0.2`) fires on ornate/flat content (text strips, banners, hatching) that coincidentally matches the diamond shape — a real Gemini sparkle is a bright WHITE overlay, so its core sits above the local background, but the NCC is contrast-invariant and cannot see that. The fusion now **demotes** (caps confidence to 0.30) any low-confidence (`< _SPARKLE_FP_CONF` 0.65) match that shows NEITHER real-sparkle signature: a bright core (`_core_ring_margin >= _SPARKLE_FP_MARGIN` 5) OR a crisp star silhouette (`gradient_score >= _SPARKLE_FP_GRAD` 0.55). I.e. demote when `low_margin OR low_grad`. Real sparkles escape via high confidence (white-bg sparkles score ≥0.79 despite a low margin — the NCC shape match is strong), high margin (dark/mid backgrounds, incl. the #36 faint-corner case, lift well clear), OR high gradient (a real sparkle is grad ~0.971.0). **The gradient condition (added 2026-06-26) closes the bright-background FP class** the margin check alone missed: a snow+sky photo and a white-background product render both scored ~0.51 at `identify`, because a bright background gives the match a HIGH core-ring margin (it genuinely IS brighter than its surroundings), so the brightness gate read it as a real overlay — but a smooth luminance blob that shape-NCC-matches the rough diamond has low gradient fidelity (the two FPs measured grad 0.105 and 0.463 vs ≥0.8 for real sparkles), so the gradient floor demotes them. The OR is **strictly a superset** of the old margin-only demotion (it only ADDS demotions on bright backgrounds, where a real sparkle keeps grad ~0.97), so it cannot regress a dark/mid sparkle (kept by margin) or a white-bg one (kept by confidence ≥ 0.65). The gate is **monotonic** (only ever removes detections, never adds), so it cannot regress the verified-negative corpus (already 0 FPs); the 2026-06-26 corpus re-sweep flipped only OpenAI/ChatGPT content (no Gemini sparkle exists there) and already-`cleaned/` outputs, all sub-0.5 (below the `identify` threshold), so no provenance verdict changed. On the spaces corpus the original gate demoted 16/495 flagged sparkles (13 carried no AI metadata = content FPs; the 3 AI-meta were visually FPs / a near-invisible white-on-white sparkle whose AI verdict is held by metadata anyway), and dropped the removal-audit failures 20→15 (post-removal flat footprints the NCC re-fired on). `_core_ring_margin` and `_estimate_alpha_gain` share the `_core_and_bg` helper (core 75th-pct brightness vs background-ring median). Regression-guarded by `test_gemini_engine.py::TestSparkleFalsePositiveGate` (incl. `test_bright_background_low_gradient_match_demoted`).
**Self-verify repair (added 2026-06-04):** the gain estimate corrects most under-subtractions, but a tail of strong sparkles still survived reverse-alpha (position jitter, or a gain the `[1.0, 1.94]` clamp could not fully reach). After the reverse blend, `remove_watermark` re-detects via `_verify_and_repair`; when a sparkle at or above `_VERIFY_FALLBACK_CONF` 0.5 (the registry's real fail line) remains, it inpaints the footprint and **keeps that only when it lowers the re-detect confidence** — purely additive (the common clean removal re-detects below 0.5 and is returned untouched, so it can never regress). On the spaces corpus this rescued **4 of the 15 remaining gemini removal-audit failures** (15→11, doubao/jimeng still 0), verified through the registry/CLI path. Costs one extra `detect_watermark` per removal (two when the fallback fires). Regression-guarded by `test_gemini_engine.py::TestVerifyAndRepair` (stubs `detect_watermark` to drive the keep-best control flow, since the NCC is degenerate on flat synthetics).
@@ -105,7 +105,7 @@ The 11 survivors are near-white ill-conditioning (reverse-alpha divides by `1-a`
## `_text_mark_engine.py`
`_text_mark_engine.py`**shared base for the three reverse-alpha text-mark engines (Doubao/Jimeng/Samsung), extracted 2026-06-09** (they were ~90% byte-identical clones). `TextMarkEngine(config: TextMarkConfig)` owns the whole `locate → extract_mask → detect → _fixed/_aligned_alpha_map → _apply_reverse_alpha → remove_watermark_reverse_alpha` pipeline (+ the asset-keyed `load_alpha_template`/`glyph_silhouette`/`template_match_score` caches). Each engine module is now a thin subclass: it supplies only its `TextMarkConfig` (the tuned constants, the bundled asset, and the bounded structural deltas — `corner` br/bl, `margin_floor` 4/2, `morph_open_size` 5/3, `min_gw` 8/16) plus the test-facing module shims (`_alpha_template`/`_glyph_silhouette`/`_template_match_score` + the constants). Behavior is byte-exact vs the old per-engine code (the three engine test suites pass unchanged). Gemini stays a SEPARATE engine (its multi-size fixed-slot sparkle model is genuinely different). Add a new text mark = a new `TextMarkConfig` + a thin subclass + one registry `_text_mark(...)` row. The engine bullets below describe each mark's calibration history; the LOGIC lives here.
`_text_mark_engine.py`**shared base for the three reverse-alpha text-mark engines (Doubao/Jimeng/Samsung), extracted 2026-06-09** (they were ~90% byte-identical clones). `TextMarkEngine(config: TextMarkConfig)` owns the whole `locate → extract_mask → detect → _fixed/_aligned_alpha_map → _apply_reverse_alpha → remove_watermark_reverse_alpha` pipeline (+ the asset-keyed `load_alpha_template`/`glyph_silhouette`/`template_match_score` caches). Each engine module is now a thin subclass: it supplies only its `TextMarkConfig` (the tuned constants, the bundled asset, and the bounded structural deltas — `corner` br/bl, `margin_floor` 4/2, `morph_open_size` 5/3, `min_gw` 8/16) plus the test-facing module shims (`_alpha_template`/`_glyph_silhouette`/`_template_match_score` + the constants). Behavior is byte-exact vs the old per-engine code (the three engine test suites pass unchanged). Gemini stays a SEPARATE engine (its multi-size fixed-slot sparkle model is genuinely different). Add a new text mark = a new `TextMarkConfig` + a thin subclass + one registry `_text_mark(...)` row. The engine bullets below describe each mark's calibration history; the LOGIC lives here. **Small-image detection guard (`_MIN_DETECT_SHORT_SIDE` 200, added 2026-06-26):** `detect` returns not-detected when the image short side is below 200px. Below that the glyph template degrades to the `min_gw` floor (~8px) and `TM_CCOEFF_NORMED` on a few pixels is noise, so an unrelated small geometric shape can spuriously correlate with the CJK silhouette — a 48×48 app-icon chevron scored Doubao 0.41 / Jimeng 0.47 (both above their thresholds), a pure small-size artifact (the same icon upscaled collapses to ~0.060.10 NCC at ≥256px). A real AI-generation label is stamped on a full-resolution render (the captured samples are 10862048px wide, the smallest positive test image is 1086px), so the floor sits far below any genuine mark while killing the icon/thumbnail band (≤96px); `identify` falls back to "unknown" (the safe default) and removal, gated on detection, is suppressed too. Regression-guarded by `test_{doubao,jimeng,samsung}_engine.py::TestDetect::test_small_image_guarded_from_false_positive`.
**`_apply_reverse_alpha` runs on the glyph crop only:** the blend is a no-op outside the glyph `region` (x, y, w, h) (`(wm - 0)/(1 - 0) == wm`, and a uint8→float32→uint8 round-trip is exact). It copies the frame through and computes the reverse-alpha math on the `region` crop only — byte-identical to the old full-frame pass (verified: Doubao 130 + Jimeng 22 placements, 0 mismatches) but O(glyph) not O(image). The full-frame pass cost ~275 ms on a 12 MP frame for a glyph that is <0.1% of it, once per candidate placement (fixed + aligned ≈ 2×/removal); the crop drops that to ~2 ms. Mirror of the Gemini `_core_and_bg` crop.