Restructure documentation, validate metadata removal, consolidate assets

This commit is contained in:
Victor Kuznetsov
2026-07-25 21:08:04 -07:00
parent 214c9bb3e7
commit 03cd00f132
172 changed files with 2356 additions and 2863 deletions
+26 -44
View File
@@ -1,22 +1,13 @@
"""Shared base for the visible text-mark detectors/localizers (localize -> fill).
"""Shared base for visible text-mark detectors and localizers.
The Doubao "豆包AI生成", Jimeng "★ 即梦AI", and Samsung "✦ Contenuti generati
dall'AI" marks are the SAME algorithm: anchor a bottom-corner box by geometry
relative to the image's SHORT side, extract the light low-saturation glyph candidate (white top-hat), detect
by matching the bundled alpha-glyph silhouette via ``TM_CCOEFF_NORMED``, and build a
removal MASK from the glyph blob's bounding box (:meth:`footprint_mask`) for the
shared fill (region_eraser). The mask is template-FREE -- the top-hat glyph bbox, not
a fixed alpha-template placement -- so a re-rendered or differently-placed mark (e.g.
a non-Italian Samsung string) is still masked. The old reverse-alpha pixel recovery
(``original = (wm - a*logo)/(1-a)``) is gone.
Each mark supplies a :class:`TextMarkConfig` with its silhouette, expected area,
scale ladder, detector frontend, and calibrated gates. The shared engine locates the
candidate, scores the silhouette with normalized correlation, and builds a removal
footprint for the common fill backend. Individual engines may override detection or
footprint behavior when their measured variant requires it.
They differ ONLY in a bounded set of tuned values captured by :class:`TextMarkConfig`:
the constants, the bundled silhouette asset, the corner (Doubao/Jimeng bottom-right,
Samsung bottom-left), and a few structural knobs. Each engine module is a thin
:class:`TextMarkEngine` subclass plus the test-facing module constants/helpers.
Gemini stays a SEPARATE engine (``gemini_engine``): its multi-size sparkle model is
genuinely different, not a tuned variant of this one.
The removal path never performs reverse-alpha pixel recovery. Gemini and the Jimeng
pill remain separate engines because their geometry and gating differ from text marks.
"""
# cv2/numpy boundary: third-party libs ship no usable element types; relax the
@@ -61,7 +52,7 @@ _MIN_DETECT_SHORT_SIDE = 200
# This used to be ONE shared 0.7 for every text mark. Measured 2026-07-18 on the
# `auto` path (the default -- no flag, driven by TC260 metadata), it turned out to
# mean two completely different things per mark. Blind hand-label of the ADDITIONS
# (accepted with provenance, rejected without) over 4417 unique TC260 carriers,
# (accepted with provenance, rejected without) over a labelled TC260 evaluation set,
# two-sided control (labeller sensitivity 100%/96%, specificity 100%/100%):
#
# mark band precision 95% CI n
@@ -182,9 +173,9 @@ class TextMarkDetection:
coverage: float = 0.0 # fraction of the box occupied by glyph pixels
# Alpha / silhouette templates, cached per asset name (the originals cached per
# module global; this keys by asset so the three engines share the loader without
# re-reading). Only SUCCESSFUL loads are cached, so a missing asset is retried.
# Alpha / silhouette templates, cached per asset name. This shared cache lets every
# text-mark engine reuse the loader without re-reading an asset. Only SUCCESSFUL loads
# are cached, so a missing asset is retried.
_alpha_cache: dict[str, NDArray[Any]] = {}
_silhouette_cache: dict[str, NDArray[Any]] = {}
@@ -289,21 +280,18 @@ class TextMarkEngine:
Two marks sharing a corner and a script (Doubao "豆包AI生成" and Jimeng
"★ 即梦AI", both bottom-right, both near-white CJK) survive binarization into
very similar blobs, so an absolute gate cannot separate them -- and under the
provenance relaxation it stopped trying: 33 of jimeng's 68 false additions
were Doubao marks (corpus-measured 2026-07-18).
provenance relaxation it stopped trying because many Jimeng false additions
were actually Doubao marks.
Measured separability on hand-labelled examples, scoring BOTH templates
against the same glyph blob (n=40 jimeng / 75 doubao / 20 other-vendor labels
/ 89 clean):
against the same glyph blob:
feature separability (0.5 = useless, 1.0 = perfect)
absolute ncc_jimeng 0.96
ncc_jimeng MINUS ncc_doubao 0.99
At a 0.10 margin: real Jimeng wordmarks pass 100%, Doubao strips 8%, other
vendors' AI labels (千问/百度/星绘/抖音) 55%, no-mark corners 12%. Because the
real marks pass at 100%, this costs NO recall -- it is a pure precision gain,
unlike raising the threshold, which trades recall away.
A 0.10 margin separated Jimeng wordmarks from the rival and clean examples
without reducing recall, unlike raising the absolute threshold.
Marks with no same-corner rival declare `rivals=()` and are unaffected.
"""
@@ -327,8 +315,7 @@ class TextMarkEngine:
thin translucent overlay shatters into specks under the threshold, and no
template can match a blob that is not there.
Measured 2026-07-18 on hand-verified corpus positives (40 doubao, 14 千问, 60
verified-clean), scoring each mark with its own template:
Calibration on hand-verified examples, scoring each mark with its own template:
front-end doubao clean neg AUC doubao/neg
binary 0.723 ~0.12 --
@@ -484,19 +471,15 @@ class TextMarkEngine:
were all calibrated on PORTRAIT captures, where width and short side coincide,
so the basis was never exercised until landscape inputs were measured.
Corpus-measured 2026-07-18 (2572 unique TC260 carriers; the harness is
`scripts/visible_eval.py`). Before any fix, doubao detection by aspect ratio:
portrait 60%, square 41%, **landscape 0% (0 of 435)** -- the width-scaled box
is inflated by the aspect ratio on a wide image and the glyph never lands in
it. Re-running the previously-undetected set with a short-side basis recovered
**56% of landscape** images (12% square, 4% portrait).
Detector calibration showed that a width-scaled box is inflated by the aspect
ratio on a wide image and can miss the glyph entirely. A short-side basis
recovered the affected Doubao landscape cases.
But the same switch took JIMENG's labelled landscape positives from 13/13 to
0/13: its wordmark tracks the WIDTH. Both marks are ByteDance and share a
The same switch broke Jimeng landscape positives because its wordmark tracks
the WIDTH. Both marks are ByteDance and share a
corner, and they still scale differently -- so this is a per-mark measurement,
not a house rule to generalize. Samsung keeps ``width`` because there is no
corpus evidence either way (1 addition corpus-wide) and an unmeasured change
is not an improvement.
not a house rule to generalize. Samsung keeps ``width`` because it has not been
calibrated for a different basis, and an unmeasured change is not an improvement.
China's GB 45438-2025 clause 5.2(e) mandates glyph height >= 5% of "the
shortest side" for CN marks, which is why a short-side basis is the natural
@@ -708,8 +691,7 @@ class TextMarkEngine:
elif self.config.detect_frontend == "tophat" and self.detect(image).detected:
# A mark found only by the CONTINUOUS front-end has no binary glyph blob to
# bound, so the mask came back empty and removal was a silent no-op while
# `identify` still reported the mark (corpus-measured 2026-07-20: 57 of 60
# sampled still-detected Doubao marks were untouched, ~8% of its detections).
# `identify` still reported the mark while removal left it untouched.
# Use the DETECTOR'S OWN best-match box: the correlation already located the
# mark at a position and scale, and thresholding the response was a strictly
# worse proxy for that. An earlier fix thresholded the max-normalized uint8
+6 -5
View File
@@ -128,9 +128,11 @@ def remove_visible(
strip_metadata: bool = True,
write_noop: bool = True,
) -> tuple[NDArray[Any], list[str]]:
"""Remove every detected known visible AI mark (Gemini sparkle, the registered
vendor text marks including Tencent Yuanbao, and the Jimeng pill) via
localize -> fill, returning ``(result_bgr, [labels removed])``.
"""Remove every detected known visible AI mark through localize then fill.
The registry currently covers the Gemini sparkle; Doubao, Jimeng, Qwen, Kling,
Yuanbao, Samsung, RunningHub, Baidu, and LibLibAI text marks; and the Jimeng
pill. Returns ``(result_bgr, [labels removed])``.
``source`` is a file path OR a BGR ndarray. For a PATH, metadata provenance is read
automatically (so ``sensitivity="auto"`` recovers a moved/faint mark whenever the
@@ -141,8 +143,7 @@ def remove_visible(
known was found (e.g. route to the diffusion ``all`` path or ``erase``).
``sensitivity`` (``auto``/``strict``) and ``backend``
(``auto``/``cv2``/``migan``/``lama``) are the same knobs as the CLI. Pass
corner if the guess is wrong).
(``auto``/``cv2``/``migan``/``lama``) are the same controls as the CLI.
``strip_metadata`` (default True, matching the CLI ``visible --strip-metadata``)
also strips AI provenance metadata (C2PA/EXIF/XMP/IPTC) from the written output via
Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.1 KiB

+6 -31
View File
@@ -16,23 +16,9 @@ Removal is the shared **localize -> fill** (:meth:`footprint_mask` ->
synthetic silhouette from ``scripts/render_vendor_silhouettes.py``, never cut
from an upload).
Measured on the vendor cohort (16 TC260 carriers whose producer USCC
91110000802100433B names Baidu, harvested 2026-07-22 by
``scripts/vendor_cohort_harvest.py``), NOT inherited from Doubao:
* The 百度 text run is 0.090 of the SHORT side wide (measured on 720/768/
1024-px frames), with its right edge ~0.099 of short off the right edge
(the pill tag sits between the text and the corner), bottom margin
~0.006; the locate box below covers the whole mark (text + tag).
* Gate 0.43 (tophat front-end): on 278 hand-labelled clean frames
(cohort-contamination-guarded) the max is 0.352 / p99 0.314, and the
visibly-marked cohort frames score 0.386-0.65. Picked over the clean-arm
0.37 after a full-corpus check on the 741-frame blind-labelled eval set
surfaced 13 cross-fires at 0.38-0.43 (12 Qwen marks + one 抖音 mark) --
see DETECT_NCC_THRESHOLD below. At 0.43 the cohort keeps 7 detections
(0.61-0.65) and the whole 741 set fires only on the true Baidu frame.
* STRICT ONLY (``provenance_ncc_factor`` 1.0): the cohort is small (16) and
the sub-gate band is unmeasured, so no provenance relaxation exists.
The detector uses a synthetic silhouette, short-side geometry, a strict
confidence gate, and a Qwen rival margin. The footprint covers both the text
run and its adjacent pill tag.
"""
# The module-level _alpha_template / _glyph_silhouette / _template_match_score below
# are thin test-facing shims (imported by tests/), so pyright's src-only pass sees them
@@ -67,20 +53,9 @@ LOGO_MIN_LUMA = 150
TOPHAT_DELTA = 12
DETECT_MIN_COVERAGE = 0.04 # unused by the tophat front-end (kept for config parity)
# Calibrated 2026-07-22 on the vendor cohort vs 278 hand-labelled clean frames
# (clean p99 0.314 / max 0.352), THEN raised 0.37 -> 0.43 after a full-corpus
# check: on the 741-frame blind-labelled eval set the 0.37 gate fired 14 times
# outside the cohort, and only ONE was the vendor -- 12 were 千问AI生成 (Qwen)
# marks (the 百/千 first glyphs are near-identical after binarization) and one
# was a 抖音 AI创作 mark at 0.425. The Qwen fires are handled by the rival
# margin (Qwen scores 0.58-0.76 there, beating Baidu by 0.17-0.35), but the
# 抖音 one named no registered rival, so the gate moved above it. Cost: the
# cohort's low trio at 0.386 (3 genuine marks) -- precision over recall on a
# small cohort. Raised again 0.43 -> 0.48 after the full-corpus sweep
# (2026-07-22): outside-cohort true Baidu carriers score 0.50-0.66 while the
# false fires (大众点评 UI, a math blackboard, an 80s banner, a checkerboard)
# top out at 0.47. Remaining cohort detections: 7 at 0.61-0.65, plus the 6
# metadata-stripped true carriers the cohort cannot see.
# Calibrated against vendor, rival-mark, and clean compatibility examples.
# The Qwen rival margin handles visually similar marks; the threshold rejects
# remaining unrelated bottom-right text.
DETECT_NCC_THRESHOLD = 0.48
# Detection-silhouette geometry (fraction of the short side): the 百度 text run
+38 -35
View File
@@ -193,9 +193,9 @@ _auto_option = click.option(
"--auto",
is_flag=True,
default=False,
help="DEPRECATED: controlnet is already the default pipeline, so --auto now only "
"enables --adaptive-polish (the content detectors were removed). Use "
"--adaptive-polish instead.",
help="DEPRECATED: controlnet and adaptive polish are already the defaults, so "
"--auto only emits a warning and changes nothing. Use --no-adaptive-polish "
"to disable polishing.",
)
_adaptive_polish_option = click.option(
@@ -210,8 +210,8 @@ _adaptive_polish_option = click.option(
# Tiled-diffusion knobs, shared by the diffusion commands (invisible/all/batch).
# Tiling is the lossless alternative to --max-resolution for large inputs that OOM
# on MPS/GPU: process at native resolution in overlapping, feather-blended tiles.
# Tiling avoids an explicit resolution cap for large inputs that OOM on MPS/GPU:
# it regenerates overlapping tiles at the input's native dimensions.
def _tile_options(f: Any) -> Any:
"""Apply the --tile / --tile-size / --tile-overlap options to a command."""
f = click.option(
@@ -229,9 +229,9 @@ def _tile_options(f: Any) -> Any:
return click.option(
"--tile/--no-tile",
default=False,
help="Process large images in overlapping tiles instead of one forward pass -- the lossless "
"alternative to --max-resolution for inputs that OOM on MPS/GPU. Engages only when the long "
"side exceeds --tile-size; pair with --max-resolution 0 (default) to keep native resolution. Default off.",
help="Process large images in overlapping tiles instead of one forward pass. This keeps "
"the input's native dimensions instead of applying --max-resolution, but still regenerates "
"every tile. Engages only when the long side exceeds --tile-size. Default off.",
)(f)
@@ -417,8 +417,8 @@ def _remove_visible_auto(
"""Remove every auto-detected visible mark via the registry (localize -> fill).
Routes the ``all``/``batch`` visible step through the same registry path the
standalone ``visible`` command uses, so EVERY registered mark is handled (the
Gemini sparkle and all registered vendor text marks), not just the sparkle.
standalone ``visible`` command uses, so every registered mark is handled rather
than only the Gemini sparkle.
Returns ``(result, label-or-None)``; when no ``in_auto`` mark fires the image is
returned unchanged with ``None``. ``backend`` selects the shared fill; ``sensitivity``
controls how hard a borderline mark is trusted (auto reads metadata provenance)."""
@@ -474,10 +474,9 @@ def _write_output_or_exit(output: Path, bgr: NDArray[Any], alpha: NDArray[Any] |
def _no_visible_mark_exit(source: Path) -> NoReturn:
"""Explain why no visible watermark was removed, then exit non-zero.
The visible registry handles only known visual marks (the Gemini sparkle and
the registered vendor text marks). Most real uploads carry no such mark
-- frequently an invisible/metadata watermark instead (e.g. an OpenAI or
Gemini image whose only signal is C2PA + SynthID). Returning the input
The visible registry handles only known visual marks. Most images carry no
registered mark and may instead have an invisible or metadata watermark.
Returning the input
unchanged with exit 0 reads as success to a caller and re-serves the
watermarked image -- the recurring "it didn't work" report. Instead, run a
cheap metadata-only :func:`identify`, tell the user what the image actually
@@ -533,9 +532,9 @@ def _no_visible_mark_exit(source: Path) -> NoReturn:
# Same value as EXIT_NO_VISIBLE_MARK (2): a distinct-from-success / distinct-from-
# error code that tells a wrapping service (raiw.cc) "the diffusion scrub was skipped
# because no invisible watermark was locally detectable", so it can surface the
# message instead of charging for and serving an unchanged image as done.
# error code that tells a wrapping service "the diffusion scrub was skipped because
# no invisible watermark was locally detectable", so it can surface the message
# instead of treating an unchanged image as a completed removal.
EXIT_NO_INVISIBLE_SIGNAL = 2
@@ -639,7 +638,7 @@ def _run_visible_auto(
console.print(f" Input: {source.name} ({w}x{h})")
if not removed:
# write_noop=False means nothing was written, so a pre-existing output is intact.
console.print(" No registered visible mark detected.")
console.print(f" No known visible mark detected. Checked: {', '.join(watermark_registry.mark_keys())}.")
_no_visible_mark_exit(source)
console.print(f" Removed: {', '.join(removed)}")
size_kb = output.stat().st_size / 1024
@@ -737,11 +736,11 @@ def cmd_visible(
) -> None:
"""Remove a known visible AI watermark from an image.
Finds a known mark in its usual place (Gemini sparkle / Doubao-Jimeng-Qwen-Samsung
text) via the watermark registry and removes it by LOCALIZING the mark to a mask
and filling that mask with the chosen ``--backend`` (auto: best available, LaMa >
MI-GAN > cv2). ``--mark auto`` removes every detected mark in one
pass. For arbitrary logos/objects, use ``erase``.
Finds registered marks in their expected areas and removes them by localizing
each mark to a mask, then filling that mask with the selected ``--backend``.
``--mark auto`` removes every detected registry entry in one pass. Run
``--help`` to see the current mark keys. For arbitrary logos and objects, use
``erase``.
"""
_banner()
source = _validate_image(source)
@@ -822,7 +821,7 @@ def cmd_erase(
Universal and position-agnostic: removes any logo / watermark / object inside
the boxes you pass, regardless of color or location. Runs on CPU. Use this
for marks the dedicated ``visible`` engines (Gemini, Doubao) do not cover.
for marks the dedicated ``visible`` registry does not cover.
"""
from remove_ai_watermarks.region_eraser import erase
@@ -899,7 +898,7 @@ def cmd_erase(
"--max-resolution",
type=int,
default=0,
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
help="Cap long side (px) before diffusion; 0 = native and preserves the most detail. Raise only on GPU/MPS OOM.",
)
@_controlnet_scale_option
@_min_resolution_option
@@ -1096,10 +1095,10 @@ def cmd_metadata(
def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bool) -> None:
"""Identify where an image was made and what watermarks it carries.
Aggregates C2PA Content Credentials, IPTC "Made with AI" tags, embedded
generation parameters, the SynthID metadata proxy, and the visible Gemini
sparkle into a single provenance verdict. Absence of signals is reported as
"unknown", never as "clean" (stripped metadata leaves no local proof).
Aggregates supported C2PA, IPTC, EXIF, XMP, generator, visible-mark, and
optional invisible-watermark signals into one provenance verdict. Absence of
signals is reported as "unknown", never as "clean" because stripped metadata
leaves no local proof.
"""
from dataclasses import asdict
@@ -1189,7 +1188,7 @@ def cmd_identify(ctx: click.Context, source: Path, no_visible: bool, as_json: bo
"--max-resolution",
type=int,
default=0,
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
help="Cap long side (px) before diffusion; 0 = native and preserves the most detail. Raise only on GPU/MPS OOM.",
)
@_controlnet_scale_option
@_min_resolution_option
@@ -1361,12 +1360,16 @@ def cmd_all(
# ── Step 3: Metadata ──
console.print("\n 3) AI metadata stripping")
try:
from remove_ai_watermarks.metadata import remove_ai_metadata
from remove_ai_watermarks.metadata import strip_and_verify
remove_ai_metadata(tmp_path, tmp_path)
console.print(" AI metadata stripped")
_, leftover = strip_and_verify(tmp_path, tmp_path)
except Exception as e:
console.print(f" Warning: Metadata strip failed: {e}")
console.print(f" Error: metadata strip failed: {e}")
raise SystemExit(1) from e
if leftover:
console.print(f" Error: metadata stripping was incomplete; {', '.join(sorted(leftover))} survived")
raise SystemExit(1)
console.print(" AI metadata stripped")
# ── Write final result ──
# The invisible step (and downstream cv2.IMREAD_COLOR paths) drops alpha,
@@ -1629,7 +1632,7 @@ def _process_batch_image(
"--max-resolution",
type=int,
default=0,
help="Cap long side (px) before diffusion; 0 = native (best quality, like raiw.cc). Raise only on GPU/MPS OOM.",
help="Cap long side (px) before diffusion; 0 = native and preserves the most detail. Raise only on GPU/MPS OOM.",
)
@_min_resolution_option
@_unsharp_option
+2 -2
View File
@@ -6,8 +6,8 @@ label mandated by China's TC260 standard, a near-white semi-transparent overlay.
Detection matches the bundled glyph silhouette against the corner candidate; removal
is the shared **localize -> fill** (the glyph-bbox :meth:`footprint_mask` feeds
``region_eraser``), NOT reverse-alpha. This is one of the registered text-mark engines that
share :class:`remove_ai_watermarks._text_mark_engine.TextMarkEngine`; this module
``region_eraser``), NOT reverse-alpha. This module shares
:class:`remove_ai_watermarks._text_mark_engine.TextMarkEngine` and
supplies only Doubao's tuned :class:`TextMarkConfig` (bottom-right corner,
``assets/doubao_alpha.png`` -- the detection silhouette, rebuilt by
``scripts/visible_alpha_solve.py``). Arbitrary-region inpainting still lives in
+5 -5
View File
@@ -150,9 +150,9 @@ class GeminiEngine:
# or flat content (text, banners, hatching) can score >0.5 without that lift.
# Demote a detection that is BOTH low-confidence AND low core-ring brightness
# margin -- the joint signature of a content false positive (verified on the
# spaces corpus: of 16 demoted, 13 carried no AI metadata and the 3 AI-meta ones
# were visually FPs / a near-invisible white-on-white sparkle whose AI verdict is
# held by metadata anyway). Real sparkles escape via EITHER high confidence
# detector calibration: demoted examples were visual false positives or a
# near-invisible white-on-white sparkle whose AI verdict is held by metadata
# anyway). Real sparkles escape via EITHER high confidence
# (white-bg sparkles score >=0.79 despite a low margin) OR high margin (dark/mid
# backgrounds, incl. the #36 faint-corner case, lift well clear), so both must
# fail to demote.
@@ -184,8 +184,8 @@ class GeminiEngine:
# confidence (_SPARKLE_KEEP_CONF -- the registry's 0.5 sparkle gate plus a small
# margin so the ~0.51 bright-background FPs the grad gate was added for stay demoted)
# AND has a bright (margin) near-neutral core (_core_saturation <= _SPARKLE_WHITE_SAT).
# Corpus-measured on metadata-stripped faint sparkles: recovers ~14/20 the low-grad
# demotion would drop, at ~0.8% clean false-fire vs the ~0.55% baseline.
# Calibrated on metadata-stripped faint sparkles to recover low-gradient marks
# without materially increasing clean false fires.
_SPARKLE_KEEP_CONF = 0.52
_SPARKLE_WHITE_SAT = 0.20
+1 -1
View File
@@ -89,7 +89,7 @@ def unsharp_mask(image: NDArray, amount: float = 0.5, sigma: float = 1.0) -> NDA
# ── Adaptive polish (target the input's detail level; spare text) ──────────────
# A capped unsharp scaled to the sharpness deficit, then edge-masked grain to close
# the rest -- tunable constants. Validated 2026-06-03 on the spaces corpus: a soft
# the rest -- tunable constants. Compatibility testing showed that a soft
# gemini_3 face/photo (lap-var 84 vs the 592 of its original) is pulled up to ~327
# with full polish, while a sharp openai_1 text card (1175 vs 1644) gets near-zero
# (the deficit is tiny) so text is left alone -- the polish self-limits on text.
+9 -11
View File
@@ -8,8 +8,8 @@ Aggregates every locally-readable signal into a single :class:`ProvenanceReport`
- **PNG text / EXIF generation parameters** (Stable Diffusion, ComfyUI, InvokeAI).
- **SynthID metadata proxy** -- a C2PA companion from a SynthID-using vendor
(Google / OpenAI) implies the invisible pixel watermark.
- **Visible marks** (optional; needs cv2/numpy, no GPU): the Gemini sparkle and
the ByteDance Doubao 豆包AI生成 / Jimeng 即梦AI text marks.
- **Registered visible marks** (optional; needs cv2/numpy, no GPU) through the
shared watermark registry.
Hard limit: a stripped image (re-encoded, screenshotted, social-media upload)
loses all metadata, and the SynthID *pixel* watermark is not locally decodable
@@ -65,9 +65,8 @@ _SCAN_BYTES = 1024 * 1024
# Visible-sparkle confidence above which the signal is trusted as provenance.
# Shared with the removal arbitration (watermark_registry.GEMINI_SPARKLE_TRUST_CONF)
# so the provenance "is there a sparkle" verdict and the removal "take the sparkle"
# decision can never drift apart -- the detect-vs-remove desync the retained-corpus
# mining surfaced (2026-06-20). On the corpus Gemini-family sparkles score >= 0.56
# while non-sparkle images top out at 0.49, so 0.5 cleanly separates them and avoids
# decision can never drift apart. Calibration showed that 0.5 separates Gemini-family
# sparkles from non-sparkle images and avoids
# false positives when the sparkle is the only signal (e.g. an OpenAI image scored
# 0.37 -- below threshold, correctly dropped).
_SPARKLE_THRESHOLD = GEMINI_SPARKLE_TRUST_CONF
@@ -463,7 +462,7 @@ def _visible_text_marks(image_path: Path, *, image: NDArray[Any] | None = None)
"""Detected visible text marks (registry ``MarkDetection`` list).
The Gemini sparkle keeps its own ``_visible_sparkle`` path (file-level
confidence); these two text marks reuse the registry detectors, which apply
confidence); the text marks reuse the registry detectors, which apply
each engine's calibrated NCC threshold via ``MarkDetection.detected``.
Optional: needs cv2/numpy; returns ``[]`` if the engines/assets are missing
or the image can't be read. ``image`` is a pre-decoded BGR array shared
@@ -555,9 +554,8 @@ def identify(image_path: Path, *, check_visible: bool = True, check_invisible: b
Args:
image_path: Path to the image (PNG, JPEG, WebP, or ISOBMFF container).
check_visible: Also run the visible-mark detectors (cv2) -- the Gemini
sparkle and vendor text marks from the registry. Set
False for a pure-metadata, dependency-light scan.
check_visible: Also run the registered visible-mark detectors through cv2.
Set False for a pure-metadata, dependency-light scan.
check_invisible: Also decode open invisible watermarks (SD/SDXL/FLUX) via
the optional imwatermark library. No-op when it is not installed.
@@ -679,8 +677,8 @@ def identify(image_path: Path, *, check_visible: bool = True, check_invisible: b
if platform is None:
# Apple Photos Clean Up (Apple Intelligence object removal) marks
# the edit with photoshop:Credit / IPTC "Apple Photos Clean Up"
# next to compositeWithTrainedAlgorithmicMedia -- corpus-measured
# 2026-07-23 (35 files); it was detected but never attributed.
# next to compositeWithTrainedAlgorithmicMedia. It was detected but
# previously never attributed.
platform = (
"Apple Photos (Clean Up AI edit)"
if b"Apple Photos Clean Up" in head
+3 -4
View File
@@ -226,10 +226,9 @@ class InvisibleEngine:
case); a ``max_resolution`` downscale always uses Lanczos. Falls back
to Lanczos if the extra is absent.
tile: Process the diffusion pass in overlapping tiles instead of one
forward pass -- the lossless alternative to ``max_resolution`` for
large inputs that OOM on MPS/GPU. Engages only when the long side
exceeds ``tile_size``. Pair with ``max_resolution=0`` (the default)
so the input keeps its native resolution.
forward pass. This retains the input's native dimensions instead
of applying ``max_resolution``, but each tile is still regenerated.
Engages only when the long side exceeds ``tile_size``.
tile_size: Tile dimension in px (default 1024).
tile_overlap: Overlap between adjacent tiles in px (default 128).
+6 -7
View File
@@ -6,13 +6,13 @@ class as the Doubao text strip.
Detection matches the bundled glyph silhouette against the corner; removal is the
shared **localize -> fill** (the glyph-bbox :meth:`footprint_mask` feeds
``region_eraser``), NOT reverse-alpha. This is one of the registered text-mark engines that
share :class:`remove_ai_watermarks._text_mark_engine.TextMarkEngine`; this module
``region_eraser``), NOT reverse-alpha. This module shares
:class:`remove_ai_watermarks._text_mark_engine.TextMarkEngine` and
supplies only Jimeng's tuned :class:`TextMarkConfig` (bottom-right corner,
``assets/jimeng_alpha.png`` -- the detection silhouette, rebuilt by
``scripts/visible_alpha_solve.py`` from the gray capture). Jimeng images are also caught
by the China TC260 AIGC metadata label, so this is the visible-mark *removal* path, not
a new ``identify`` signal.
by the China TC260 AIGC metadata label. The visual detector also feeds
``identify`` when metadata is absent.
"""
# The module-level _alpha_template / _glyph_silhouette / _template_match_score below
# are thin test-facing shims (imported by tests/), so pyright's src-only pass sees them
@@ -44,9 +44,8 @@ TOPHAT_DELTA = 12
# (>=0.81) from the Doubao strip (0.21), so the two ByteDance marks do not cross-fire.
#
# That separation holds at the STRICT threshold ONLY. Relaxed under provenance it
# collapses: at the old shared 0.7 factor (gate 0.315) the arm ran at 17% precision
# and 33 of its 68 false additions were Doubao marks (corpus-measured 2026-07-18 --
# full table at _DEFAULT_PROVENANCE_NCC_FACTOR). That was patched with a tighter 0.85
# collapses: at the old shared 0.7 factor, many false additions were Doubao marks.
# That was patched with a tighter 0.85
# factor at the time; the competitive rival margin replaced it -- see below.
DETECT_MIN_COVERAGE = 0.02
DETECT_NCC_THRESHOLD = 0.45
+6 -25
View File
@@ -11,19 +11,9 @@ This module supplies only LibLibAI's tuned :class:`TextMarkConfig`
(``assets/liblib_alpha.png`` from ``scripts/render_vendor_silhouettes.py``,
never cut from an upload).
Measured on the vendor cohort (15 TC260 carriers, harvested 2026-07-22 by
``scripts/vendor_cohort_harvest.py``), NOT inherited from Doubao:
* The wordmark is ~0.10 of the frame WIDTH wide, centered horizontally, its
baseline ~0.94-0.95 of the height; consistent across 768..2240-px frames.
* The silhouette font is Arial, NOT the STHeiti the CJK marks use: the real
wordmark is a grotesque, and measured across 7 candidate fonts Arial lifts
the cohort positives from 0.31-0.47 to 0.42-0.73 while the full-corpus
false arm (latin UI text) drops to max 0.398. A 200x200 icon false-fired
at 0.444, so a per-mark size floor (``_MIN_SHORT_SIDE``) backs the gate.
* Gate 0.42 (tophat front-end): false arm max 0.398, cohort 0.43-0.59.
* STRICT ONLY (``provenance_ncc_factor`` 1.0): small cohort, the relaxed band
is unmeasured.
The detector uses an Arial-class synthetic silhouette, width-based geometry, a
strict confidence gate, and a minimum image size. The footprint includes both
the logo and wordmark.
"""
# The module-level _alpha_template / _glyph_silhouette / _template_match_score below
# are thin test-facing shims (imported by tests/), so pyright's src-only pass sees them
@@ -57,13 +47,8 @@ LOGO_MIN_LUMA = 150
TOPHAT_DELTA = 12
DETECT_MIN_COVERAGE = 0.04 # unused by the tophat front-end (kept for config parity)
# Calibrated 2026-07-22 on the vendor cohort vs 286 hand-labelled clean frames
# (clean p99 0.315 / max 0.367) and re-measured after the font fix: the wordmark
# is set in an Arial-class grotesque, and the Arial silhouette lifts the cohort
# positives to 0.43-0.59 while the full-corpus false arm (latin UI text bands,
# website screenshots) drops to max 0.398 -- generic latin text matches the
# wrong font less, which is exactly where the discrimination comes from. Gate
# 0.42 keeps all 8 marked cohort frames with a 0.022 margin over the false arm.
# Calibrated against vendor and clean compatibility examples. The Arial-class
# silhouette separates the wordmark from generic Latin UI text.
DETECT_NCC_THRESHOLD = 0.42
# Detection-silhouette geometry (fraction of the frame width): the wordmark,
@@ -120,11 +105,7 @@ def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
class LibLibEngine(TextMarkEngine):
"""Detect/localize the visible LibLibAI wordmark (bottom-center; localize -> fill)."""
# Per-mark size floor: the wordmark template is 0.10 of the frame width, so
# below ~480px short side it degrades under ~48px -- the one full-corpus
# false fire with the final Arial template was a 200x200 icon (0.444, above
# the gate, on a 20px template; measured 2026-07-22). The smallest true
# carrier in the cohort is 768px.
# Per-mark size floor prevents small generic icons from matching the wordmark.
_MIN_SHORT_SIDE = 480
def __init__(self) -> None:
+9 -9
View File
@@ -153,8 +153,8 @@ _TC260_FIELDS: frozenset[str] = frozenset(
"PropagateID",
"ReservedCode1",
"ReservedCode2",
# Service-provider schema (Tencent Cloud's AIGC variant, mined from the
# retained corpus 2026-07): the same ``{"AIGC":{...}}`` wrapper but keyed
# Service-provider schema (Tencent Cloud's AIGC variant): the same
# ``{"AIGC":{...}}`` wrapper but keyed
# ``ServiceProvider`` / ``ServiceUser`` (+ generic ``Time`` / ``ContentId``,
# not gated on), embedded in EXIF ``ImageDescription``.
"ServiceProvider",
@@ -1062,8 +1062,8 @@ _EXT_TO_PIL_FORMAT = {".jpg": "JPEG", ".jpeg": "JPEG", ".webp": "WEBP", ".png":
def _sniff_image_format(head: bytes) -> str | None:
"""Actual raster format from a file's leading magic bytes (>= 12 bytes), as a PIL
format name ("JPEG"/"PNG"/"WEBP"), or None when unrecognized. The file EXTENSION is
unreliable: ~2% of real uploads carry a mismatched one (a PNG served as ``.jpg`` is
common). Choosing the save format by extension re-encodes a lossless PNG/WebP into a
unreliable because an input may carry a mismatched one (for example, PNG content
served as ``.jpg``). Choosing the save format by extension re-encodes a lossless PNG/WebP into a
real JPEG, silently degrading the pixels -- so the strip routes on content instead.
ISOBMFF/GIF are handled before this point or fall through to PNG; only the
lossy-vs-lossless distinction that matters here is resolved."""
@@ -1087,9 +1087,9 @@ def strip_and_verify(
:func:`remove_ai_metadata` is deliberately fail-safe: a file PIL cannot decode is
copied through UNCHANGED rather than crashing a caller, and the path it returns is
indistinguishable from a real strip. Any caller that reports an outcome to a user
therefore cannot tell a no-op from a success -- corpus-observed on real Samsung
Galaxy S22 C2PA PNGs, where `metadata --remove` printed "stripped" and exited 0 while
the output still read as AI (2026-07-19 parity audit).
therefore cannot tell a no-op from a success. A Samsung C2PA compatibility case
exposed this when `metadata --remove` reported success while the output still read
as AI.
Returns ``(output_path, surviving_markers)``; an empty mapping means a real strip.
"""
@@ -1164,8 +1164,8 @@ def remove_ai_metadata(
if source_path.suffix.lower() in _FFMPEG_STRIP_EXTS:
return _strip_with_ffmpeg(source_path, output_path)
# Route on the ACTUAL content format, not the extension (which lies on ~2% of real
# uploads -- a PNG served as .jpg, etc.). Trusting the extension would push a
# Route on the actual content format, not the extension. PNG content may be served
# as .jpg, and trusting the extension would push a
# lossless PNG/WebP through the lossy JPEG re-encode below just because its name
# ends .jpg, breaking the "work with originals" invariant.
true_fmt = _sniff_image_format(head) # reuse the 12 bytes already read above
+19 -24
View File
@@ -135,12 +135,11 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
),
# Some Volcano Engine certs name the signer with the Chinese legal entity
# "北京火山引擎科技有限公司" (Beijing Volcano Engine Technology Co., Ltd.) rather
# than the latin "volcengine" -- the latin needle misses it entirely, so real
# ByteDance output was un-attributed in production traffic. The issuer is the
# than the latin "volcengine" -- the latin needle misses it entirely. The issuer is the
# UTF-8 of the Chinese name (it appears UTF-8-encoded in the manifest-store
# JSON and the raw caBX bytes alike); it normalizes to the same "ByteDance"
# needle and platform as the volcengine row, so the two collapse together for
# clash detection. Verified against the mined retained corpus, 2026-06-20.
# clash detection. Verified against compatible signed samples.
C2paAiVendor(
"北京火山引擎科技有限公司".encode(),
"ByteDance (Volcano Engine)",
@@ -154,7 +153,7 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
# clean manifest issuer matches "BytePlus (ByteDance)" directly. The platform
# string mirrors the volcengine row: both share the "ByteDance" needle, so the
# earlier row's label wins anyway -- they normalize together for clash
# detection. Verified on real signed files in production traffic, 2026-06-19.
# detection. Verified on compatible signed samples.
C2paAiVendor(b"Byteplus", "BytePlus (ByteDance)", "ByteDance (Doubao / Jimeng / Volcano Engine)", "ByteDance"),
# Dreamina (ByteDance's international Jimeng brand) signs C2PA as "Bytedance
# Pte. Ltd." with a "Dreamina/x.y" claim generator and, unlike the Volcano
@@ -164,8 +163,8 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
# active manifest is often a plain c2pa-tool transcode). ``asserts_ai`` lets the
# issuer alone flag AI without trainedAlgorithmicMedia; "Dreamina" is a
# distinctive brand string, so it does not risk the incidental-mention problem
# the common-word issuers have. Verified on real signed files in the retained
# corpus, 2026-07. Normalizes to the same "ByteDance" needle/platform as the
# the common-word issuers have. Verified on compatible signed samples.
# Normalizes to the same "ByteDance" needle/platform as the
# volcengine row (they collapse together for clash detection).
C2paAiVendor(
b"Dreamina",
@@ -176,18 +175,17 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
),
# Canva Magic Media signs AI-generated images as "Canva" with a generic
# c2pa-rs claim generator + trainedAlgorithmicMedia; without this entry the
# source read AI but no platform was attributed. Verified on real signed files
# in production traffic, 2026-06-19. Canva does not use SynthID.
# source read AI but no platform was attributed. Verified on compatible signed
# samples. Canva does not use SynthID.
C2paAiVendor(b"Canva", "Canva", "Canva (Magic Media)", "Canva"),
# ElevenLabs is a pure generative-AI company (AI voice / audio, and image /
# video via its API); it signs output as "Eleven Labs Inc.", so the C2PA
# manifest alone marks AI generation. Verified against the mined retained
# corpus, 2026-06-20. ElevenLabs does not use SynthID.
# manifest alone marks AI generation. Verified on compatible signed samples.
# ElevenLabs does not use SynthID.
C2paAiVendor(b"Eleven Labs", "ElevenLabs", "ElevenLabs", "ElevenLabs"),
# fal.ai (generative inference platform, issuer "fal - Features & Labels
# Inc." / common name "fal.ai", claim generators like "fal-ai/seedvr",
# "fal-ai/gpt-image-2"). Corpus-measured 2026-07-23 on the retained
# uploads (17 files): the files carry trainedAlgorithmicMedia, so the
# "fal-ai/gpt-image-2"). The files carry trainedAlgorithmicMedia, so the
# verdict already fired, but the platform stayed unattributed. fal.ai is
# a pure generative platform, so ``asserts_ai`` also covers its output
# that omits the source-type.
@@ -195,7 +193,7 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
# Bria AI (bria.ai, generative platform) signs as "Bria Artificial
# Intelligence" with a "Bria Ai" claim generator and source type
# ``empty`` (NOT trainedAlgorithmicMedia), so a real signed file was
# completely missed by identify -- corpus-found 2026-07-23. A pure-AI
# completely missed by identify. A pure-AI
# vendor with distinctive strings, so ``asserts_ai`` is safe here.
C2paAiVendor(b"Bria", "Bria Artificial Intelligence", "Bria AI", "Bria", asserts_ai=True),
# Truepic is a C2PA signing authority, not an AI generator: no platform label,
@@ -203,8 +201,7 @@ C2PA_AI_VENDORS: tuple[C2paAiVendor, ...] = (
C2paAiVendor(b"Truepic", "Truepic", None, None),
)
# Deliberately NOT registered as AI-generation vendors (mined-corpus candidates
# evaluated 2026-06-20):
# Deliberately NOT registered as AI-generation vendors:
# - TikTok Inc.: signs C2PA as a content-provenance / AI-labeling authority on
# uploads, not as an image generator. The is_ai verdict keys off the
# digitalSourceType (trainedAlgorithmicMedia), which is already honored; a
@@ -234,12 +231,11 @@ C2PA_IDENTITY_AI_ORGS: frozenset[str] = frozenset(v.org for v in C2PA_AI_VENDORS
# images", updated 2026-05-21): "Images generated with ChatGPT, Codex, and
# our API include both C2PA metadata and SynthID watermarks." OpenAI also
# notes a signal may be absent if "the image was created before these
# signals were available" -- so OpenAI images from BEFORE the rollout carry
# C2PA WITHOUT SynthID (e.g. data/samples/openai-images-2/amur-leopard.png,
# C2PA timestamp 2026-04-22). For OpenAI the proxy is therefore "likely",
# signals were available" -- so OpenAI images from before the rollout can
# carry C2PA without SynthID. For OpenAI the proxy is therefore "likely",
# not certain; the verdict string is hedged accordingly. OpenAI's own oracle
# is openai.com/verify (Google's is the Gemini app "Verify with SynthID").
# The issuer byte ("OpenAI"/"Google") is verified locally against data/samples;
# The issuer byte ("OpenAI"/"Google") is verified locally against data/fixtures/provenance;
# the SynthID pairing is documented behavior (Google: DeepMind; OpenAI: above).
# Adobe Firefly and Microsoft Designer sign C2PA but do NOT use SynthID, so a
# C2PA manifest alone is not a SynthID signal -- the issuer is. The pixel
@@ -311,8 +307,7 @@ AI_GENERATOR_TOKENS: frozenset[str] = frozenset(
"leonardo",
"flux",
"dreamstudio",
# Mined from the retained corpus 2026-06-22 (no C2PA -- a plain EXIF/PNG
# generator stamp was the only signal and we read none of them):
# Generator stamps without C2PA:
# - NovelAI (anime SD): PNG tEXt Software="NovelAI", Source="NovelAI
# Diffusion V4.5 <hash>", Title="NovelAI generated image".
# - Reve Image (reve.com): EXIF Software / XMP CreatorTool = "reve.com"
@@ -321,11 +316,11 @@ AI_GENERATOR_TOKENS: frozenset[str] = frozenset(
"novelai",
"reve.com",
"aphrodite ai",
# Corpus-mined 2026-07-23:
# Additional verified markers:
# - Apple Photos Clean Up (Apple Intelligence object removal): XMP
# photoshop:Credit / IPTC credit value; composite source-type
# covered detection, this token covers removal parity (35 files).
# - fal-ai: generative-platform generator string (17 files).
# covered detection, this token covers removal parity.
# - fal-ai: generative-platform generator string.
"apple photos clean up",
"fal-ai",
}
+3 -3
View File
@@ -3,9 +3,9 @@
The img2img / ControlNet pipeline denoises the WHOLE image in one forward pass,
so it OOMs on MPS/GPU above ~2K (issue #10). Tiling splits the image into
overlapping tiles -- each kept near SDXL's ~1024 training size -- regenerates
each tile independently, and feather-blends the overlaps. The result is processed
at NATIVE resolution with no seam: the lossless alternative to the
``--max-resolution`` downscale (which trades quality for a smaller forward pass).
each tile independently, and feather-blends the overlaps. The result retains the
input's native dimensions without an explicit ``--max-resolution`` downscale, but
it is not pixel-lossless because every tile is regenerated.
The geometry (``plan_tiles``) and the blend weighting (``feather_weights``) are
pure functions, unit-tested without the diffusion model. ``run_tiled`` is the
@@ -75,7 +75,7 @@ CONTROLNET_CANNY_MODEL = "xinsir/controlnet-canny-sdxl-1.0"
# ladder" below.
#
# Data basis (see docs/synthid.md sections 2.2 / 5.5): ORACLE-CERTIFIED controlnet floors.
# A 2026-06-14 re-test on the deployed Modal worker (the production controlnet pipeline)
# Oracle re-testing
# LOWERED the ladder back to OpenAI 0.10 / Google 0.15: each output verified on its own
# oracle (openai.com/verify for OpenAI, the Google Gemini app for Google), all clean ->
# - OpenAI 0.10: 2 photoreal images (1402 / 1448 px), SynthID not found on either.
@@ -98,9 +98,8 @@ CONTROLNET_CANNY_MODEL = "xinsir/controlnet-canny-sdxl-1.0"
# case (flat fills) `sdxl` is the WEAKER remover -- plain img2img at low strength barely
# perturbs a flat region -- so it needs AT LEAST as much strength as controlnet, not
# less. Hence the certified controlnet floor is the right floor for `sdxl` too. The
# higher strength costs little quality where it matters: `controlnet` is now the default
# pipeline, so `sdxl` is reached only for structure-less inputs (via `--auto`) or an
# explicit `--pipeline sdxl`, where over-regeneration has no faces/text to damage. NOTE:
# higher strength costs little quality where it matters. `controlnet` is now the default
# pipeline and `sdxl` is reached only through an explicit `--pipeline sdxl`. NOTE:
# this is a MARGIN argument for `sdxl`, not a fresh certification -- there is no local
# SynthID detector, so if an oracle still reads SynthID on a flat `sdxl` output, raise
# `--strength`.
@@ -85,8 +85,8 @@ def is_watermark_removal_available() -> bool:
# Drop-in fp16-safe replacement for the SDXL VAE. The stock SDXL VAE overflows
# to NaN in fp16 and decodes to an all-black image (issue #29: the raiw.cc black
# result on a CUDA fp16 backend). This community VAE is numerically rescaled to
# to NaN in fp16 and decodes to an all-black image (issue #29, reproduced on a
# CUDA fp16 backend). This community VAE is numerically rescaled to
# stay in fp16 range. SDXL-architecture only.
_SDXL_FP16_VAE_ID = "madebyollin/sdxl-vae-fp16-fix"
@@ -692,9 +692,10 @@ class WatermarkRemover:
strips the metadata; the caller passes it down so display and execution
agree.
tile: Process the image in overlapping tiles instead of one forward pass.
The lossless alternative to a ``--max-resolution`` downscale for large
inputs that OOM on MPS/GPU (issue #10). Only engages when the long side
exceeds ``tile_size``; smaller images run a single pass unchanged.
This keeps the input's native dimensions instead of applying a
``--max-resolution`` downscale, but every tile is still regenerated.
Only engages when the long side exceeds ``tile_size``; smaller images
run a single pass unchanged.
tile_size: Tile dimension in px (default 1024).
tile_overlap: Overlap between adjacent tiles in px (default 128), feather-
blended so there is no visible seam.
+7 -8
View File
@@ -7,14 +7,14 @@ silhouette; like every mark it is then removed by the shared localize -> fill:
* Detect: edge-NCC of a font-rendered SILHOUETTE (``assets/jimeng_pill.png``,
synthetic, data-safe -- see ``scripts/render_pill_silhouette.py``) against the
top-left ROI, at the pill's known width fraction. Corpus-calibrated threshold
(61 real positives + jimeng negatives): ``_DETECT_THRESHOLD`` 0.22.
top-left ROI, at the pill's known width fraction. The calibrated
``_DETECT_THRESHOLD`` is 0.22.
* Remove: place the pill footprint at the matched location and inpaint it
(MI-GAN / cv2 via the registry). Quality comes from the inpaint backend, so the
silhouette need not be pixel-accurate -- which is why a synthetic render is
sufficient and no corpus-derived asset is committed.
sufficient and no source-derived asset is committed.
Geometry measured on 51 real examples (8 resolutions, all 3:4): width ~0.161*W,
Geometry uses width ~0.161*W,
height ~0.091*W, top-left, margins ~0.02-0.05.
"""
@@ -42,7 +42,7 @@ _ASSET = Path(__file__).parent / "assets" / "jimeng_pill.png"
_WIDTH_FRAC = 0.161
_ROI_W_FRAC = 0.34 # search window width (of W)
_ROI_H_FRAC = 0.14 # search window height (of H)
_DETECT_THRESHOLD = 0.22 # edge-NCC gate, corpus-calibrated
_DETECT_THRESHOLD = 0.22 # calibrated edge-NCC gate
# Inpaint mask GEOMETRY (fractions of W unless noted): a generous fixed top-left box
# covering the pill (measured ~0.167*W wide, ~0.09*W tall, margin ~0.02-0.05) plus
# margin. The mask uses stable geometry, NOT the NCC match position -- the synthetic
@@ -59,9 +59,8 @@ _MASK_W, _MASK_H = 0.205, 0.115 # width of W, height of W
# corners (sky / wall / solid) where inpaint is invisible. So the metadata-only arm
# removes the pill only when the footprint background is flat enough for a safe,
# invisible inpaint. Threshold = median Sobel magnitude over the footprint box at a
# normalized width; corpus-validated on 32k real uploads 2026-07 (real pills median
# ~3.2, textured-ceiling false fires median ~8+). The reliable bottom-right wordmark
# arm is NOT texture-gated -- a wordmark-confirmed pill is removed regardless.
# normalized width. The reliable bottom-right wordmark arm is NOT texture-gated:
# a wordmark-confirmed pill is removed regardless.
_FLAT_TEXTURE_MAX = 6.0
_silhouette: NDArray[Any] | None = None
+3 -3
View File
@@ -4,7 +4,7 @@ Position- and content-agnostic. You supply the rectangle(s); the eraser inpaints
whatever is inside, so it removes any visible logo / watermark / object regardless
of color, style, or location. Localization is the user's responsibility (pass the
box); restoration runs on CPU. This is the universal fallback for marks the
deterministic per-generator engines (Gemini sparkle, Doubao) do not cover.
registered visible detectors do not cover.
Backends:
- ``cv2`` (default): ``cv2.inpaint`` (Telea / Navier-Stokes). Instant, no extra
@@ -235,8 +235,8 @@ def erase_migan(image_bgr: NDArray[Any], mask: NDArray[Any]) -> NDArray[Any]:
Mask polarity: the shipped ``andraniksargsyan/migan`` ONNX expects 0 = hole
(inpaint) / 255 = known (keep) -- the INVERSE of this package's 255-erase
convention -- so the mask is inverted before feeding the model (corpus-validated
2026-07; feeding 255=hole regenerates the whole frame into stripes).
convention -- so the mask is inverted before feeding the model. Feeding 255=hole
regenerates the whole frame into stripes.
Accepts 1-channel (grayscale) and 4-channel (BGRA) input.
"""
@@ -135,15 +135,8 @@ class RunningHubEngine(TextMarkEngine):
def __init__(self) -> None:
super().__init__(_CONFIG)
# Anchor window for the match position, as a fraction of the FRAME: the true
# mark hugs the corner (measured x 0.008-0.014, y 0.005-0.007 of the frame on
# every cohort positive), while the full-corpus false fires (hair, shelves,
# window frames, CJK banners -- 37 of 42009 outside-cohort frames at the 0.34
# gate, 2026-07-22 sweep) match off-anchor at x 0.013-0.150 / y 0.009-0.045.
# No NCC gate separates them (false max 0.384 vs two positives at 0.381), but
# position does: every false fire sits outside this window, every positive
# inside. Contrast-dependent raw-gray NCC keys on "some text-like structure
# anywhere in the box"; the anchor is what makes it about THIS mark.
# Anchor window for the match position, as a fraction of the frame. The true
# mark hugs the corner, while common false matches sit farther from the anchor.
_ANCHOR_MAX_X = 0.025
_ANCHOR_MAX_Y = 0.015
+2 -2
View File
@@ -9,8 +9,8 @@ Doubao/Jimeng marks but bottom-left.
Detection matches the bundled glyph silhouette against the corner; removal is the
shared **localize -> fill** (the glyph-bbox :meth:`footprint_mask` feeds
``region_eraser``), NOT reverse-alpha. This is one of the registered text-mark engines that
share :class:`remove_ai_watermarks._text_mark_engine.TextMarkEngine`; this module
``region_eraser``), NOT reverse-alpha. This module shares
:class:`remove_ai_watermarks._text_mark_engine.TextMarkEngine` and
supplies only Samsung's tuned :class:`TextMarkConfig` (bottom-LEFT corner, a lower glyph
luma since the mark is faint, ``assets/samsung_alpha.png`` -- the detection silhouette,
solved from the flat captures by ``scripts/visible_alpha_solve.py``). Samsung Galaxy AI
+16 -18
View File
@@ -104,8 +104,8 @@ _PRODUCT_OF: dict[str, str] = {
# trust, which bypasses the sibling's false-positive gate outright -- so a detector
# that false-fires often must not be able to hand that bypass to anyone.
#
# The pill qualifies on its own documented numbers: ~7% raw false-fire (5.5% measured
# on 578 vendor negatives, 2026-07-18). Letting it corroborate produced a closed loop
# The pill detector has a meaningful raw false-fire rate. Letting it corroborate
# produced a closed loop
# on the DEFAULT auto path, no user flag involved:
# pill false-fires on a clean non-ByteDance image
# -> _PRODUCT_OF maps it to "jimeng", so jimeng resolves to `confirmed`
@@ -113,10 +113,9 @@ _PRODUCT_OF: dict[str, str] = {
# -> _keep_pill now sees "jimeng" in keys and takes the WORDMARK arm, which
# removes the pill unrestricted -- skipping the flatness guard that exists
# precisely to stop the fill smearing a textured corner.
# Measured on the corpus: 3 of 578 negatives ran the full loop, one of them with
# footprint_flat=0 (the exact case the guard was written to block). Cutting the pill
# out of corroboration removed all 3 and cost NOTHING on 4417 TC260 carriers
# (jimeng fires 398 -> 398), so this is a defect fix, not a recall trade.
# Calibration reproduced the full loop, including a textured footprint. Cutting the
# pill out of corroboration removed the loop without reducing Jimeng detections, so
# this is a defect fix, not a recall trade.
#
# `_keep_pill` already encodes the same distrust for the pill's own ACTION; this
# closes the gap that its TESTIMONY was never gated.
@@ -293,11 +292,11 @@ class KnownMark:
# confidence, shared by BOTH the removal arbitration here (`_gemini_detect`) and
# the provenance detector in `identify` (which imports it as its sparkle threshold).
# Defining it once removes the detect-vs-remove
# threshold drift the retained-corpus mining surfaced (2026-06-20): identify
# threshold drift found during compatibility testing: identify
# would report a sparkle while removal declined it, or vice versa, whenever the
# two independently-maintained 0.5 constants fell out of step. Now they cannot.
#
# Value 0.5 is corpus-validated: the gemini engine's own `detected` flag uses a
# Value 0.5 is calibrated: the Gemini engine's own `detected` flag uses a
# looser internal threshold (0.35) and weakly fires (~0.36-0.42) on unrelated
# bottom-right text -- a real Doubao mark scores ~0.40-0.42 as a gemini match,
# and its core-ring brightness margin is HIGHER than a genuine faint sparkle's,
@@ -451,7 +450,8 @@ def resolve_backend(backend: Backend) -> Literal["cv2", "migan", "lama"]:
def fill(image: NDArray[Any], mask: NDArray[Any], *, backend: Backend = "auto") -> NDArray[Any]:
"""The ONE shared, mark-agnostic removal: erase ``mask`` (255 = remove) via the
chosen inpaint backend. Delegates to :func:`region_eraser.erase`; ``auto``
resolves to MI-GAN when installed else cv2 (see :func:`resolve_backend`)."""
resolves in quality order, LaMa then MI-GAN then cv2 (see
:func:`resolve_backend`)."""
from remove_ai_watermarks import region_eraser
return region_eraser.erase(image, mask=mask, backend=resolve_backend(backend))
@@ -479,8 +479,8 @@ def _gemini_mask(
return _engine("gemini").footprint_mask(image, force=force, region=region)
# The registered text-mark engines share the TextMarkEngine interface, so one
# parameterized adapter pair drives all of them -- a new
# The text-mark engines share the TextMarkEngine interface, so one parameterized
# adapter pair drives all of them -- a new
# text mark is one `_text_mark(...)` row below, not another copy-paste of these
# bodies. Detection matches the glyph silhouette; the mask is the template-free
# glyph-bbox footprint (see TextMarkEngine.footprint_mask).
@@ -505,8 +505,7 @@ def _text_mark_mask(key: str) -> Callable[..., NDArray[Any] | None]:
def _text_mark(key: str, label: str, location: str) -> KnownMark:
"""A text-mark registry row (Doubao/Jimeng/Samsung): glyph-silhouette detect +
template-free glyph-bbox mask."""
"""Build a text-mark registry row from its shared detector and mask adapters."""
return KnownMark(key, label, location, True, _text_mark_detect(key, label, location), _text_mark_mask(key))
@@ -611,12 +610,11 @@ def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bo
"""Whether to auto-remove the capture-less 'AI生成' pill given the fired marks.
Pure decision (the flatness feature is precomputed at perception time and passed
in). The pill detector is weak (~7% raw false-fire) and metadata/intent confirms
the platform, not pill presence, so a naive confirmation-OR gate over-fires: on a
32k real-upload corpus (2026-07) the metadata-only arm was only ~27% precise and
its false fires were textured ceilings/walls that the fill visibly SMEARS. Arms:
in). The pill detector is weak and metadata/intent confirms the platform, not pill
presence, so a naive confirmation-OR gate over-fires on textured ceilings and walls
that the fill visibly smears. Arms:
* bottom-right "★ 即梦AI" wordmark fired -> ~94% precise, and it survives
metadata-STRIPPED uploads: remove the pill unrestricted;
metadata-STRIPPED images: remove the pill unrestricted;
* TC260 metadata confirms Jimeng (``"jimeng" in provenance``, no wordmark) -> remove ONLY when the
top-left footprint is flat enough for an invisible fill (``footprint_flat``),
so real flat-scene pills (and harmless flat false fires) are cleaned while the