Compare commits

...
2 Commits
Author SHA1 Message Date
Victor Kuznetsov 80cdaff2d5 chore(release): v0.18.0 2026-07-22 13:17:00 -07:00
Victor Kuznetsov f1a5eecf98 Register RunningHub, Baidu, and LibLibAI visible marks; park Qingyan and MiniMax (measured)
New engines, each calibrated on its TC260 USCC cohort and validated by a
full-corpus sweep (42009 files):
- runninghub: top-left corner (new corner="tl"), faint mid-gray text via
  the new raw-grayscale "gray" detection front-end, anchor-position gate
- baidu: text-run-only template (pill is a bright-blob magnet), load-bearing
  Doubao+Qwen rival margins, corner-extended footprint for the white tag
- liblib: bottom-center (new corner="bc"), Arial silhouette (font is the
  discriminative lever against latin UI text), logo-extended footprint

Qingyan parked (no clean-arm separation at any render/box), MiniMax/Hailuo
parked (1 visible frame, the xinghui rule); silhouettes kept as starting
points.
2026-07-22 13:03:03 -07:00
25 changed files with 1236 additions and 26 deletions

No files matched your search

+6 -3
View File
@@ -21,7 +21,7 @@ Per-command exit-code semantics (the no-signal / GPU-missing skip branches), tes
- `uv run remove-ai-watermarks all <image.png> -o <output.png>` — full pipeline (visible + invisible + metadata). Same diffusion knobs as `invisible`, plus the visible-pass `--backend auto|cv2|migan|lama` (default `auto`) and `--sensitivity auto|strict` (default `auto`) for the localize -> fill visible removal (see the `visible` bullet). Skips step 2 (invisible/SynthID) when the `[gpu]` extra is absent or no invisible signal is detectable; see the module doc for the distinct exit codes.
- `uv run remove-ai-watermarks invisible <image.png> -o <out.png>` — diffusion SynthID removal. **Full knob set** (kept identical across `invisible`/`all`/`batch`): `--strength` (vendor-adaptive default), `--steps` (**interacts with `--strength`**: diffusers derives its timesteps as `int(steps * strength)`, so a low `--steps` used to crash inside torch with `cannot reshape tensor of 0 elements` -- at the default 0.15 that was every value below 7. `noai/watermark_profiles.viable_steps` now raises the count to the minimum that denoises and logs the adjustment; keep the guard where it is, above `_generate_one`, so all three pipelines and the tiled path inherit it), `--guidance-scale` (CFG, default 7.5), `--pipeline sdxl|controlnet|qwen` (default `controlnet`; `qwen` is a manual opt-in only — see the qwen note in the module map), `--controlnet-scale`, `--model` (HF model id, default SDXL base), `--device`, `--seed`, `--hf-token`, `--max-resolution`/`--min-resolution`, `--upscaler lanczos|esrgan`, `--humanize` (Analog Humanizer grain), `--unsharp` (final sharpen), `--adaptive-polish/--no-adaptive-polish` (**ON by default**), `--tile/--no-tile` + `--tile-size`/`--tile-overlap` (**OFF by default**), `--force/--no-force` (default skip = ON, runs the scrub even with no detected signal). `--auto` is deprecated and a no-op that only warns. Skips the diffusion when no invisible signal is detectable (the no-signal gate); see the module doc.
- `uv run remove-ai-watermarks visible <image.png> -o <out.png>` — known-visible-mark removal by **localize -> fill**: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. `--backend auto|cv2|migan|lama` (default `auto`) picks the fill: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. `--mark auto` (default) removes EVERY detected mark in one pass (a Jimeng-basic image carries the top-left "AI生成" pill AND the bottom-right "★ 即梦AI" wordmark) from: Gemini sparkle, Doubao "豆包AI生成", Jimeng "★ 即梦AI", Qwen "千问AI生成", Samsung Galaxy AI "✦ Contenuti generati dall'AI", and the capture-less Jimeng "AI生成" pill (top-left, metadata-gated); `--mark gemini|doubao|jimeng|qwen|samsung|jimeng_pill` forces one. `--sensitivity auto|strict` (default `auto`) sets how hard a borderline mark is trusted: `auto` relaxes a mark's gate only on same-product evidence (metadata provenance for that vendor, or a confidently detected sibling mark of the same product — clean images stay untouched); `strict` never relaxes. Metadata provenance is read automatically and feeds `auto`. (`assume-ai` was REMOVED in 0.16 — see the registry bullet; a user who can SEE a missed mark should point at it with `erase --region`, or name it with `--mark <text-mark> --no-detect`.) For arbitrary logos/objects use `erase`. When no known mark is detected the command writes no output and exits with the no-visible-mark code instead of re-serving the input; `--no-detect` forces the gemini fallback and proceeds. See the module doc for the routing/exit detail. `--backend` and `--sensitivity` are shared across `visible`/`all`/`batch`.
- `uv run remove-ai-watermarks visible <image.png> -o <out.png>` — known-visible-mark removal by **localize -> fill**: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. `--backend auto|cv2|migan|lama` (default `auto`) picks the fill: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. `--mark auto` (default) removes EVERY detected mark in one pass (a Jimeng-basic image carries the top-left "AI生成" pill AND the bottom-right "★ 即梦AI" wordmark) from: Gemini sparkle, Doubao "豆包AI生成", Jimeng "★ 即梦AI", Qwen "千问AI生成", Samsung Galaxy AI "✦ Contenuti generati dall'AI", Baidu "百度 AI生成", LibLibAI wordmark (bottom-center), RunningHub "RunningHub AI生成" (top-left), and the capture-less Jimeng "AI生成" pill (top-left, metadata-gated); `--mark gemini|doubao|jimeng|qwen|kling|samsung|baidu|liblib|runninghub|jimeng_pill` forces one. `--sensitivity auto|strict` (default `auto`) sets how hard a borderline mark is trusted: `auto` relaxes a mark's gate only on same-product evidence (metadata provenance for that vendor, or a confidently detected sibling mark of the same product — clean images stay untouched); `strict` never relaxes. Metadata provenance is read automatically and feeds `auto`. (`assume-ai` was REMOVED in 0.16 — see the registry bullet; a user who can SEE a missed mark should point at it with `erase --region`, or name it with `--mark <text-mark> --no-detect`.) For arbitrary logos/objects use `erase`. When no known mark is detected the command writes no output and exits with the no-visible-mark code instead of re-serving the input; `--no-detect` forces the gemini fallback and proceeds. See the module doc for the routing/exit detail. `--backend` and `--sensitivity` are shared across `visible`/`all`/`batch`.
- `uv run remove-ai-watermarks erase <image.png> --region x,y,w,h -o <out.png>` — universal region eraser (any logo/object, any position). `--backend cv2` (default, no deps), `--backend migan` (MI-GAN via onnxruntime, extra `migan`; ~28 MB, ~1 GB RAM, near-LaMa), or `--backend lama` (big-LaMa, extra `lama`; best quality but ~4.7 GB RAM); `--region` is repeatable.
- `uv run remove-ai-watermarks identify <image>` — provenance verdict (platform + watermark inventory + confidence); `--json` for machine output, `--no-visible` to skip the cv2 sparkle detector
- `uv run remove-ai-watermarks metadata <image.png> --check` — inspect AI metadata (C2PA, EXIF, PNG chunks)
@@ -56,13 +56,16 @@ Compact map. The full per-module detail (design decisions, tuned thresholds, cal
- `noai/constants.py` — the single `C2PA_AI_VENDORS` registry (+ `C2PA_SOFT_BINDINGS`) from which `C2PA_ISSUERS` / `SYNTHID_C2PA_ISSUERS` / `C2PA_IDENTITY_AI_ORGS` / `identify._ISSUER_PLATFORM` are all derived. Add a new vendor as one registry entry; never edit the derived dicts and never add inline. A vendor's `asserts_ai=True` flag means its mere presence asserts AI generation even without a `trainedAlgorithmicMedia` digital-source-type (a pure-generator brand with a distinctive issuer/generator string, e.g. **Dreamina** — ByteDance's international Jimeng brand, signed as "Bytedance Pte. Ltd." with a "Dreamina/x.y" claim generator and no source-type); NEVER set it for common-word issuers (Adobe/Google/OpenAI/Microsoft) that appear incidentally in unrelated bytes — those stay source-type-gated in `identify._attribute_platform`.
- `metadata.py` — `scan_head(path)` is the shared (memoized) input for every C2PA/AIGC/IPTC byte scan; use it instead of `open().read(1MB)` for any new marker scan. Also home to `synthid_source`, `xai_signature`, `iptc_ai_system`, `aigc_label`, `huggingface_job`, `samsung_genai`, and `remove_ai_metadata` (fail-safe `strip_c2pa_boxes`). **A caller that REPORTS an outcome must use `strip_and_verify`, not `remove_ai_metadata` directly** -- the stripper is deliberately fail-safe (a file PIL cannot decode is copied through UNCHANGED rather than crashing), so its return value cannot distinguish a no-op from a real strip. `metadata --remove` and `batch --mode metadata|all` both re-scan the OUTPUT through it and fail loudly; corpus-observed on real Samsung Galaxy S22 C2PA PNGs, where the command printed "stripped" and exited 0 while the output still read as AI (2026-07-19). **`remove_ai_metadata` is the SINGLE metadata stripper** (the legacy PIL-re-encoding `noai/cleaner` was deleted; the diffusion core and the public `noai.remove_ai_metadata` re-export now point here). It strips **losslessly** per container: ISOBMFF (HEIC/AVIF/MP4) blanks tokens / strips boxes in place; **JPEG uses `_strip_jpeg_metadata_lossless`** — a marker-segment walk that drops the AI-bearing APP segments (C2PA APP11; XMP APP1 carrying C2PA, a China-AIGC token, OR an IPTC `digitalSourceType` / 2025.1 AI-disclosure marker; IPTC-IIM APP13) and scrubs AI EXIF tags via piexif, copying the entropy-coded scan verbatim so **the pixels are bit-identical** (no DCT re-encode). **Detection<->removal parity across every marker placement is load-bearing** — anything a scanner flags, the strip must reach, or a re-served file still reads as AI: (a) the APP1-XMP branch of `_jpeg_app_carries_ai` checks the IPTC marker sets too, not only C2PA/AIGC (the Instagram/MidJourney/Meta "Made with AI" `digitalSourceType` lives in XMP, not the APP13 IIM record); (b) a bare `AIGC{...}` / `{"AIGC":{...}}` block in ANY JPEG APP segment — the specific C2PA(APP11)/XMP(APP1)/IPTC(APP13) checks FALL THROUGH to a generic `_is_aigc_exif_value` drop, so a bare AIGC in APP11 (the common real-corpus placement, NOT a C2PA manifest) is caught, not swallowed by the C2PA-only 0xEB branch — plus the same AIGC block in a STANDARD **PNG text chunk** value (e.g. `Description`, which `_is_ai_key` keeps) is dropped on the value; (c) the China TC260 `{"AIGC":{...}}` block in EXIF `UserComment`/`ImageDescription` is scrubbed by `_scrub_ai_exif` (Doubao producer + Tencent service-provider schemas); (d) the Samsung Galaxy AI `PhotoEditor_Re_Edit_Data` trailer past the JPEG EOI is truncated by `_strip_samsung_trailer` (and `samsung_genai` reads the file tail so a multi-MB photo's trailer past the 512 KB quick-scan window is still DETECTED). Pixels stay bit-identical throughout, so a `--strip-metadata` on a q100 removal output does NOT crush it back to q75; PNG/WebP re-saves are pixel-lossless (WebP written at cv2 lossless mode, quality 101 — quality 1-100 is lossy). **The PIL-fallback save format is chosen by the source's CONTENT, not its file extension** (`_sniff_image_format`, and the JPEG-lossless gate is content-gated too): ~2% of real uploads are misnamed (a PNG served as `.jpg` is the common one), and routing on the extension re-encoded a lossless PNG/WebP into a real JPEG — a silent degradation that broke "work with originals" (corpus-measured ~0.9% of files). A **misnamed** lossless source (source-extension format != content) is preserved in its true format; a **correctly-named** source still honors a deliberate output-extension conversion (e.g. `source.png -> output.jpg`). Not yet handled: a 16-bit PNG is downconverted to 8-bit on the PIL re-save (rare; would need a byte-level PNG chunk stripper). Regression: `tests/test_metadata.py::TestHasAiMetadata::test_strip_preserves_lossless_content_with_mismatched_extension`. **`remove_ai_metadata` is fail-safe on an undecodable image:** a truncated/corrupt file (PIL raises `OSError` decoding it; ~0.2% of real uploads) is copied through UNCHANGED rather than crashing a direct library caller (a web worker would 500 on a partial upload), mirroring `strip_c2pa_boxes` — we cannot strip what we cannot parse, but we never raise. Regression: `tests/test_metadata.py::TestHasAiMetadata::test_remove_ai_metadata_failsafe_on_truncated_png`. Regression: `tests/test_metadata.py::TestHasAiMetadata::{test_jpeg_metadata_strip_is_pixel_lossless, test_jpeg_strip_removes_iptc_marker_in_xmp}`, `TestSamsungGenai::{test_remove_strips_post_eoi_trailer, test_detects_trailer_past_scan_window}`, the AIGC-EXIF/bare-APP removal tests, and `tests/test_noai.py::TestISOBMFF::{test_blank_aigc_block_in_exif, test_blank_xai_signature_pair_in_exif}`. `exif_generator` matches a VALUE against `AI_GENERATOR_TOKENS` across EXIF `Software`/`Make`/`Artist`/`ImageDescription`, XMP `CreatorTool`, AND PNG `tEXt` chunks (`Software`/`Source`/`Title`/`Description` — NovelAI Line truncated
- `identify.py` — aggregates every locally-readable signal into one `ProvenanceReport`; `is_ai_generated` is True or None, never asserted False. `ProvenanceReport.ai_source_kind` exposes the C2PA digital-source-type split — `"generated"` (trainedAlgorithmicMedia, fully AI) vs `"enhanced"` (compositeWithTrainedAlgorithmicMedia, a real photo with an AI-composited region), else None — so a caller branches full-frame scrub vs region-targeted clean (see `noai/tiling.feather_region_composite` + `WatermarkRemover.remove_watermark(region=...)`). The sparkle provenance threshold is the SHARED `watermark_registry.GEMINI_SPARKLE_TRUST_CONF` (imported, not a private copy) so the provenance "is there a sparkle" verdict and the removal "take the sparkle" decision can never drift. `import identify` is deliberately light (lazy `noai/__init__`, fits a 512 MB host) — keep heavy imports out (the `watermark_registry` constant import stays light: engines are lazy there). Add capture-camera tokens to `_DEVICE_C2PA_PLATFORM` only when verified against a real C2PA file; editing-app/AI-device signer tokens go to `_SIGNER_C2PA_PLATFORM`; generator/issuer platforms to `C2PA_AI_VENDORS` in `constants.py`. The IPTC `digitalSourceType` **`algorithmicMedia`** (bare) is PROCEDURAL (an algorithm not trained on sampled data), NOT AI/ML generation, so it is deliberately absent from `IPTC_AI_MARKERS` — flagging it made `identify` assert AI + `has_invisible_target` True, scrubbing clean procedural content (it is a distinct token from `trainedAlgorithmicMedia`, so real "Made with AI" labels are unaffected; regression `test_metadata.py::...test_bare_algorithmic_media_not_flagged_ai`). Integrity-clash detection is high-precision by design (only hard generator stamps feed it, source-grouped independence). `_vendor_of` normalizes ByteDance/Canva/ElevenLabs/Black Forest Labs (as well as OpenAI/Google/... ) so their C2PA claims participate in the clash check; the generic **China TC260 AIGC label names no specific vendor**, so when a TC260-applying vendor (ByteDance, `_TC260_VENDORS`) is co-attributed the label is attributed to it (a legit Doubao image carrying its own TC260 label must NOT clash), while a NON-TC260 vendor next to a TC260 label still clashes as a laundering tell. Corpus-validated: adding the vendors introduced 0 new clashes on 5000 carriers.
- `watermark_registry.py` — the single catalog of known visible watermarks (gemini / doubao / jimeng / qwen / kling / samsung / jimeng_pill). **Removal is LOCALIZE -> FILL for every mark:** each mark is localized to a binary full-frame footprint mask (a `Localization`), then ONE shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). Reverse-alpha (the old `original = (wm - a*logo)/(1-a)` inversion of a captured alpha map + thin residual inpaint) is GONE for ALL marks; why it was dropped is recorded in `docs/module-internals.md`. Backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. The captured alpha maps (`scripts/visible_alpha_solve.py`) are still used to DETECT the marks and to shape the mask, but NOT for pixel recovery. **`--mark auto` removes EVERY detected mark in one pass** via `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` (marks coexist -- a Jimeng-basic image has the top-left pill AND the bottom-right wordmark; a single-strongest pick would leave one). **Three orthogonal axes:** `backend` (the fill), `sensitivity` (how hard to trust a borderline mark: `auto`/`strict`, see the `Sensitivity` literal), and `provenance` (vendor keys metadata confirms -- the evidence that drives `auto`). **Perception / decision / action are separated:** `_build_candidates(image)` runs every detector at BOTH trust levels (strict + relaxed) and packages raw verdicts + features into `Candidate`s (no policy); the pure arbiter `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` makes every keep/drop call (per-mark `resolve_trust` + the assumed-trust floor + the pill gate) with no image/IO, so it is unit-testable in isolation; then each winner is localized -> filled. Do NOT put policy back into the engines (the one exception, the Gemini FP gate, stays in `gemini_engine` because `identify` shares that confidence). `detect_marks(..., provenance=frozenset())` stays strict (identify verdict, precision over recall); `KnownMark.remove/detect/localize(..., provenance: bool)` take the already-resolved boolean. **How `auto` decides (this is metadata-INDEPENDENT for recall):** the visual detectors are pixel-based and need no metadata; the recall gain comes from RELAXING the false-positive gate, not from metadata. `strict` never relaxes (clean images untouched); `auto` relaxes a mark only on same-product evidence -- metadata provenance for that vendor OR a confidently detected sibling mark of the SAME product (`_PRODUCT_OF`; Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax). **`resolve_trust` resolves TWO levels:** `confirmed` bypasses the engine's false-positive gate, and only `confirmed` has evidence naming THAT vendor, which is exactly what the bypass is contracted to require (`GeminiEngine.detect_watermark`'s `trust_provenance` docstring: "external metadata already proves this is a Google generation"). **Historical, kept as the reason the third level is gone:** a removed `assumed` level let `assume_ai` bypass the gate on the bare assertion an image is AI. On its first form that left only the raw 0.35 detector threshold and it fired on **59.8% of 256 genuine camera captures**, filling a phantom sparkle on ~6 of every 10 CLEAN photos; a confidence floor made it tolerable, and the mode was removed outright in 0.16. Corpus-measured 2026-07-16 before removal (400 Google-C2PA positives with metadata hidden; 256 camera-capture negatives): recall strict 55.0% / auto 55.2% / assume_ai 62.8%, false fire 0.0% / 0.0% / 2.3% -- the extra recall was never free. A wrong relaxation only fills a small corner near-losslessly (the localize -> fill benign failure mode), which is what made a SMALL false-fire rate arguable; it was never a licence for a 60% one. Metadata provenance mapping (feeds `auto`, read by `cli._visible_provenance`): Google/Gemini C2PA issuer -> gemini; China-AIGC (TC260) label -> doubao/jimeng; `samsung_genai` -> samsung. **The `jimeng_pill` is CAPTURE-LESS** (`pill_engine.py`): the top-left "AI生成" label has no captured alpha map, so it is detect-by-synthetic-silhouette; its footprint is a fixed top-left geometry box. Its weak edge-NCC detector (~7% raw false-fire) is gated in `remove_auto_marks` via **`_keep_pill`** (32k real-upload corpus validation 2026-07): the pill never rides on a **Doubao** or **Qwen** detection (either bottom-right mark names its own product, and neither is Jimeng-basic), and has confirmation arms because metadata/intent confirms the platform, not pill presence. **(1) Bottom-right "★ 即梦AI" wordmark fired** — ~94% precise and survives **metadata-STRIPPED uploads** (screenshots / re-saves, ~61% of pillsLine truncated
- `watermark_registry.py` — the single catalog of known visible watermarks (gemini / doubao / jimeng / qwen / kling / samsung / runninghub / baidu / liblib / jimeng_pill). **Removal is LOCALIZE -> FILL for every mark:** each mark is localized to a binary full-frame footprint mask (a `Localization`), then ONE shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). Reverse-alpha (the old `original = (wm - a*logo)/(1-a)` inversion of a captured alpha map + thin residual inpaint) is GONE for ALL marks; why it was dropped is recorded in `docs/module-internals.md`. Backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. The captured alpha maps (`scripts/visible_alpha_solve.py`) are still used to DETECT the marks and to shape the mask, but NOT for pixel recovery. **`--mark auto` removes EVERY detected mark in one pass** via `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` (marks coexist -- a Jimeng-basic image has the top-left pill AND the bottom-right wordmark; a single-strongest pick would leave one). **Three orthogonal axes:** `backend` (the fill), `sensitivity` (how hard to trust a borderline mark: `auto`/`strict`, see the `Sensitivity` literal), and `provenance` (vendor keys metadata confirms -- the evidence that drives `auto`). **Perception / decision / action are separated:** `_build_candidates(image)` runs every detector at BOTH trust levels (strict + relaxed) and packages raw verdicts + features into `Candidate`s (no policy); the pure arbiter `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` makes every keep/drop call (per-mark `resolve_trust` + the assumed-trust floor + the pill gate) with no image/IO, so it is unit-testable in isolation; then each winner is localized -> filled. Do NOT put policy back into the engines (the one exception, the Gemini FP gate, stays in `gemini_engine` because `identify` shares that confidence). `detect_marks(..., provenance=frozenset())` stays strict (identify verdict, precision over recall); `KnownMark.remove/detect/localize(..., provenance: bool)` take the already-resolved boolean. **How `auto` decides (this is metadata-INDEPENDENT for recall):** the visual detectors are pixel-based and need no metadata; the recall gain comes from RELAXING the false-positive gate, not from metadata. `strict` never relaxes (clean images untouched); `auto` relaxes a mark only on same-product evidence -- metadata provenance for that vendor OR a confidently detected sibling mark of the SAME product (`_PRODUCT_OF`; Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax). **`resolve_trust` resolves TWO levels:** `confirmed` bypasses the engine's false-positive gate, and only `confirmed` has evidence naming THAT vendor, which is exactly what the bypass is contracted to require (`GeminiEngine.detect_watermark`'s `trust_provenance` docstring: "external metadata already proves this is a Google generation"). **Historical, kept as the reason the third level is gone:** a removed `assumed` level let `assume_ai` bypass the gate on the bare assertion an image is AI. On its first form that left only the raw 0.35 detector threshold and it fired on **59.8% of 256 genuine camera captures**, filling a phantom sparkle on ~6 of every 10 CLEAN photos; a confidence floor made it tolerable, and the mode was removed outright in 0.16. Corpus-measured 2026-07-16 before removal (400 Google-C2PA positives with metadata hidden; 256 camera-capture negatives): recall strict 55.0% / auto 55.2% / assume_ai 62.8%, false fire 0.0% / 0.0% / 2.3% -- the extra recall was never free. A wrong relaxation only fills a small corner near-losslessly (the localize -> fill benign failure mode), which is what made a SMALL false-fire rate arguable; it was never a licence for a 60% one. Metadata provenance mapping (feeds `auto`, read by `cli._visible_provenance`): Google/Gemini C2PA issuer -> gemini; China-AIGC (TC260) label -> doubao/jimeng; `samsung_genai` -> samsung. **The `jimeng_pill` is CAPTURE-LESS** (`pill_engine.py`): the top-left "AI生成" label has no captured alpha map, so it is detect-by-synthetic-silhouette; its footprint is a fixed top-left geometry box. Its weak edge-NCC detector (~7% raw false-fire) is gated in `remove_auto_marks` via **`_keep_pill`** (32k real-upload corpus validation 2026-07): the pill never rides on a **Doubao**, **Qwen**, **Kling**, **RunningHub** or **Baidu** detection (each of those marks names its own product, and none is Jimeng-basic), and has confirmation arms because metadata/intent confirms the platform, not pill presence. **(1) Bottom-right "★ 即梦AI" wordmark fired** — ~94% precise and survives **metadaLine truncated
- `gemini_engine.py` — visible Gemini-sparkle detector + localizer (cv2/numpy, no GPU): top-K size-weighted fusion candidate selection (`_SELECT_TOPK`), corner-promote, false-positive gate (the provenance prior relaxes the gate + lowers the trust threshold when a Google/Gemini C2PA issuer confirms the vendor). **White-core rescue:** the FP gate demotes a low-gradient match (soft edges), but a real FAINT sparkle also has soft edges -- so the gate keeps a low-grad match that is a strong (conf ≥ `_SPARKLE_KEEP_CONF` 0.52), bright (margin), near-WHITE-core sparkle (`_core_saturation` ≤ `_SPARKLE_WHITE_SAT` 0.20): a real sparkle core is white, a clean bright corner that shape-matches (sky/sun) is colored. This recovers ~14/20 metadata-stripped faint sparkles under the DEFAULT strict/auto (no flag, no metadata) at ~1.25% clean false-fire (baseline 0.55%); the ~0.51-scoring bright-bg FPs stay demoted (below 0.52). A learned classifier on the SAME features was measured WORSE than the tuned gate (2026-07 tier-1: MLP 86.7% recall vs 90.8% at equal FP), so the heuristic stays; a patch-CNN with richer features is the only lever left (roadmapped P2, low expected value -- the wall is fundamental). Detection scores the top-K size-weighted matches by full fusion (spatial+gradient+variance) and keeps the highest — NOT the raw-NCC argmax, which re-admits the tiny-patch FPs the size weight suppresses (the osachub 2026-06-12 sub-0.85 corner-sparkle regression; see `docs/module-internals.md`). Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Removal is localize -> fill: `footprint_mask` returns the sparkle footprint (the captured alpha thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin), and the shared `watermark_registry.fill` inpaints it. The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. `detect_sparkle_confidence` reuses a process-wide `_shared_engine()` singleton (lru_cache) — the engine holds only constant assets (captures, alpha maps, a precomputed 16..118 template ladder) and takes the image as an arg, so do NOT reconstruct `GeminiEngine()` per call: that reloaded assets + recomputed alpha maps + rebuilt the template cache on every one of ~34k `identify` calls (−24% on the sparkle path once made a singleton, output byte-identical). `detect_watermark`/`footprint_mask` guard `image.size == 0` before `to_bgr`, and return an empty (detected=False) result when no template scale fits (short side < 16 px), rather than dereferencing an empty candidate list.
- `_text_mark_engine.py` — shared base for the text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. Detection still matches the glyph silhouette (NCC, keys on glyph shape). The removal mask is TEMPLATE-FREE: it is the bounding box of the top-hat glyph blob (`extract_mask`), filled solid + dilated, so the shared fill inpaints the whole wordmark rectangle. This drops the fixed alpha-template placement, so a re-rendered or differently-placed mark is still masked; the captured alpha maps are now used only for the detection silhouette, not for removal. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model). The detection scale ladder is per-mark (`TextMarkConfig.ladder`, default `(0.8, 1.0, 1.25)` -- added 2026-07-21 for qwen's two size modes; the shared default is unchanged for every other mark, and densifying the SHARED ladder was measured and rejected, see `docs/verification-plan.md` B2).
- `_text_mark_engine.py` — shared base for the text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. Detection still matches the glyph silhouette (NCC, keys on glyph shape). The removal mask is TEMPLATE-FREE: it is the bounding box of the top-hat glyph blob (`extract_mask`), filled solid + dilated, so the shared fill inpaints the whole wordmark rectangle. This drops the fixed alpha-template placement, so a re-rendered or differently-placed mark is still masked; the captured alpha maps are now used only for the detection silhouette, not for removal. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model). The corner anchor is `corner` = `br`/`bl`/`tl`/`bc` (tl added 2026-07-22 for runninghub, bc for liblib's centered wordmark); `detect_frontend` is `binary`/`tophat`/`gray` (`gray` = raw-grayscale NCC for the faint mid-gray runninghub mark, added 2026-07-22; contrast-DEPENDENT, so its gates never port). The detection scale ladder is per-mark (`TextMarkConfig.ladder`, default `(0.8, 1.0, 1.25)` -- added 2026-07-21 for qwen's two size modes; the shared default is unchanged for every other mark, and densifying the SHARED ladder was measured and rejected, see `docs/verification-plan.md` B2).
- `pill_engine.py` — the CAPTURE-LESS Jimeng-basic "AI生成" pill (top-left, issue #54). No alpha map: `detect` is edge-NCC of a synthetic font-rendered silhouette (`assets/jimeng_pill.png`, regenerate via `scripts/render_pill_silhouette.py`; committed, data-safe -- corpus stays out of the repo) in the top-left ROI, calibrated on 61 local real positives to threshold 0.22; `footprint_mask` is a generous FIXED top-left geometry box (NOT the NCC match position -- the synthetic silhouette localizes only approximately, the corner is negative space, so a geometry box fills cleanly while a match box leaves outline residue). `footprint_texture`/`footprint_is_flat` (median-Sobel over that box, `_FLAT_TEXTURE_MAX`) back the metadata-only safe-fill gate. Removal is the shared localize -> fill (MI-GAN/cv2). Detector precision is weak (~7% raw false-fire), so it is registry-gated in `remove_auto_marks` via `_keep_pill`: never on Doubao; the bottom-right wordmark removes it unrestricted (~94% precise, survives metadata-STRIPPED uploads); TC260-metadata-only removes it ONLY on a flat footprint (its textured false fires -- ceilings/walls -- are what the fill smears). Do NOT loosen those gates.
- `doubao_engine.py` / `jimeng_engine.py` / `samsung_engine.py` — thin `TextMarkEngine` subclasses: Doubao "豆包AI生成" (bottom-right), Jimeng "★ 即梦AI" (bottom-right), Samsung Galaxy AI "✦ Contenuti generati dall'AI" (bottom-LEFT, locale-specific — Italian variant calibrated). Detection matches the glyph silhouette (NCC); removal localizes the glyph blob to a solid dilated box (`extract_mask`) and hands it to the shared fill. Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. **Samsung detection is calibrated only for the Italian "Contenuti generati dall'AI" string** (a pre-existing limit, unchanged by the localize -> fill refactor but now surfaced because detection gates removal): non-Italian Samsung locales are not detected, and thus not removed, even though the fill mask itself is locale-independent; other locales need their own detection silhouette (the locale string font-rendered + calibrated on real positives), NOT an app capture.
- `qwen_engine.py` — the Qwen (Alibaba Tongyi Qianwen) "千问AI生成" text mark (bottom-right), registered 2026-07-21 off the 117-frame TC260-producer cohort (`scripts/vendor_cohort_harvest.py` + `scripts/vendor_mark_calibrate.py`; the full calibration record is in `docs/module-internals.md`). Same `TextMarkEngine` subclass shape, but EVERY tuned number is measured on the cohort, not inherited from doubao: `scale_basis=short`, a **per-mark 2-rung `ladder=(0.78, 1.27)`** (the mark sits in two size modes ~0.124/~0.203 of the short side, ratio 1.64 -- wider than the shared ladder's 1.5625 span, so the best single fraction covers only 74.5% and the small mode falls into the comb's collapse zone; `TextMarkConfig.ladder` was added for this, default `(0.8, 1.0, 1.25)` unchanged for every other mark), a fitted locate box (the real mark sits ~0.025 of short off the right edge and doubao's 0.004-margin box clipped the first glyph, 0.73 -> 0.26 NCC), `alpha_height_frac` from the measured 0.260 aspect (not the silhouette's, not doubao's), gate **0.45** (clean p99 0.301; 83 of ~96 eyeballed visible marks fire = 86% recall of visible marks). **STRICT ONLY** (`provenance_ncc_factor` 1.0, no provenance mapping): the sub-gate band is dominated by non-Qwen banners (夸克 strip 0.274, 造点 mark 0.253), so a relaxed arm would be mostly false fills. **No rival margin** (0 cross-fires on 400 doubao / 298 jimeng / 286 clean frames at the gate; a 0.10 margin would cost ~10% of genuine qwen detections). A confident qwen detection suppresses the jimeng pill like doubao's does. Parity: 83/83 real marks detector-clean after cv2 fill. **Calibration trap for the next vendor:** the 2026-07-18 `present: []` clean labels are in the vocabulary of the REGISTERED marks only -- qwen-cohort frames visibly carrying 千问AI生成 sat in the "clean" arm and were its whole top tail (p99 0.69); `vendor_mark_calibrate.load_sets` now excludes every frame in ANY TC260 cohort.
- `kling_engine.py` — the Kling (Kuaishou 可灵) "可灵AI 3.0" text mark (bottom-right), registered 2026-07-21 off the 30-frame TC260-producer cohort, same playbook as qwen. UNIMODAL at 0.12 of the short side, so it keeps the shared 3-rung ladder and gets a fitted locate box (margins ~0.03/0.023); gate **0.35** (clean p99 0.304 / max 0.320), STRICT ONLY (the faint "Omni"-suffix / latin "KlingAI" / version-less variants score 0.17-0.25, inside the clean arm's top tail -- unreachable, capping recall at ~47% of visible marks, precision 9/9), no rival margin (crossfire 1/400 doubao below gate, 0 jimeng, 0 clean). Parity 9/9. Suppresses the jimeng pill like doubao/qwen. **元宝 (Tencent Yuanbao) was attempted the same day and PARKED as a measured negative** -- its two-line italic-slanted block does not separate from clean corners on either front-end at any render/box setting (full record in `docs/verification-plan.md`); the fitted render recipe stays in `render_vendor_silhouettes.py` `MARK_OPTS` as the starting point for a future structural/learned lever. The **cat-logo** cohort was also probed and parked: 19 frames but only 2 unique carriers, so nothing to calibrate recall against; the `draw_catlogo` silhouette already separates (mark 0.50 vs clean max 0.333) and registration is a gate pick (~0.42) once more unique carriers arrive.
- `runninghub_engine.py` — the RunningHub "RunningHub AI生成" mark (**top-left**, the first `corner="tl"`), registered 2026-07-22 off the 73-frame TC260 cohort. Faint mid-gray text that the top-hat suppresses to clean-arm levels, so it introduced the **third detection front-end `gray`** (raw-grayscale silhouette NCC, contrast-DEPENDENT, one `_gray_best` method serving detection and mask): positives 0.38-0.54 vs clean max 0.295, gate **0.34**, STRICT ONLY. Two measured traps: the tight ladder `(0.95, 1.0, 1.05)` sits exactly on the measured 0.32-of-width (the comb is razor-sharp: 0.537 on-size vs 0.223 at +5.6%), and an **anchor gate** in `detect` (match must sit at x<=0.025 / y<=0.015 of the frame) kills the 37/42009 full-corpus false fires (hair, shelves, CJK banners) that no NCC threshold separates from the 0.381 positives. Footprint is always the detector's match box (the binary blob under-segments the faint head glyphs and left "Runni" unremoved on the first try).
- `baidu_engine.py` — the Baidu "百度 AI生成" mark (bottom-right), registered 2026-07-22 off the 16-frame cohort (USCC names Baidu). Detection keys on the 百度 **text run ONLY** (a text+pill template was a measured bright-blob magnet with no separation); the white "AI生成" tag is still removed because the custom `footprint_mask` extends the match box right to the corner (the tag's flat white interior gives no top-hat response, so a blob bbox left it as a ghost). Gate settled at **0.48** after three measured steps (0.37 clean-arm; the 741-eval set fired on 12 Qwen marks + one 抖音 mark, so `rivals=("doubao_alpha.png","qwen_alpha.png")` with load-bearing margins; the full-corpus sweep put true outside-cohort carriers at 0.50-0.66 vs the false arm max 0.47). Cohort 7/16, all true, plus 6 metadata-stripped true carriers the cohort cannot see. STRICT ONLY.
- `liblib_engine.py` — the LibLibAI (哩布哩布AI) "LibLibAI" wordmark (**bottom-center**, the first `corner="bc"`), registered 2026-07-22 off the 15-frame cohort. The discriminative lever is the silhouette **font**: STHeiti scored the cohort at the false arm's level (latin UI text bands); measured across 7 fonts, **Arial** lifts the cohort to 0.42-0.73 and drops the false arm to max 0.398. Gate **0.42**, STRICT ONLY, plus a per-mark size floor (`_MIN_SHORT_SIDE=480`: the one remaining false fire was a 200x200 icon on a 20px template). Custom footprint = match box extended left ~1.3 glyph heights for the triangle logo (the blob bbox bled into background structure and never owned the logo). **Zhipu Qingyan (清言·AI生成) and MiniMax/Hailuo were attempted the same day and PARKED** as measured negatives (qingyan: cohort 0.34-0.39 vs clean max 0.34-0.37, no separation at any render/box; minimax: 1 of 6 frames visibly marked, the xinghui rule) -- record in `docs/verification-plan.md` ("The 2026-07-22 vendor round").
- `region_eraser.py` — universal region eraser (`erase` CLI) and the shared fill backend behind `watermark_registry.fill` for the visible localize -> fill removal. Three backends: `cv2` (default, no deps, the floor), `migan` (MI-GAN ONNX, extra `migan`, MIT, ~28 MB / ~0.19 s — the droplet-friendly tier, **the preferred default fill** when the extra is installed), `lama` (big-LaMa ONNX, extra `lama`, ~200 MB / ~4.7 GB peak — best quality, does not fit a minimal droplet, explicit opt-in only). Both `migan` and `lama` **crop a padded region around the mask** before inference and paste only masked pixels back, so peak RAM is bounded by the MARK size, not the image (`migan` ~0.6-0.9 GB regardless of upload size — feeding the whole frame scaled it to ~2.4 GB at 25 MP; `migan` feeds the crop at native resolution, `lama` resizes to its fixed 512²). **Measured end to end 2026-07-20** (`scripts/resource_ceilings.py`, fresh process per cell, 1 MP → 25 MP): `migan` 603 → 775 MB and `lama` 4679 → 4779 MB, both **flat in input size** — the crop-around-the-mask design holds and both documented figures reproduce. **`cv2` is the only backend that GROWS with the input** (74 → 440 MB, 5.9x) because it inpaints the full frame rather than a crop; still the cheapest tier, but size it for the largest upload accepted. Cold wall time 0.02-0.12 s (cv2) / ~0.6 s (migan) / ~3.8 s (lama), model load included. (The harness's own no-op check originally allocated a full-frame temp before reading peak RSS and inflated these by up to 17% at 25 MP; it now compares only the mask box. The conclusion survived re-measurement, the digits moved.) **MI-GAN mask polarity is INVERTED** (0=hole/255=known) vs this package's 255-erase convention; `erase_migan` inverts before feeding the model (feeding 255=hole regenerates the whole frame into stripes — corpus-validated). Both ONNX models download on first use, never bundled. The `erase` command keeps its own `--backend`/`--inpaint-method` (unchanged).
- `invisible_watermark.py` — decodes the OPEN DWT-DCT watermarks (SD / SDXL / FLUX) via `imwatermark` (extra `detect`, pulls torch). Fragile two ways: (1) does not survive JPEG re-encode/resize; (2) **carrier-fragile on a broad class of pristine images** -- a clean encode->decode round-trip recovers 48/48 on chatgpt/firefly/random but FAILS (28-39/48, below the `_MATCH_48`=44 gate) on the FLUX fox, doubao, a flat FLUX generation, AND a clean synthetic flat fill with no watermark. The failure does NOT track texture; it goes with a degenerate **all-ones decode that is a CARRIER ARTIFACT, not a watermark** (synthetic clean image reproduces it). So `detect_invisible_watermark` is **positive-only**: trust a hit; a `None` is inconclusive unless a same-carrier positive-control embed first recovers >=44. Verified 2026-06-19; full caveat in `docs/watermarking-landscape.md`.
- `trustmark_detector.py` — Adobe TrustMark open decoder (extra `trustmark`). Do NOT remove the JPEG re-encode false-positive gate — a lone TrustMark hit without it is almost always content noise.
+13 -7
View File
@@ -21,13 +21,13 @@ If this tool saves you time, consider [sponsoring its development](https://githu
## Scope
This tool removes **AI-provenance watermarks** that a platform stamps onto content **you generated yourself** — SynthID, the Gemini / Nano Banana sparkle, the Doubao / Jimeng / Qwen / Samsung visible AI labels, the Chinese TC260 "由…AI生成" label, and C2PA / IPTC / EXIF "Made with AI" metadata. The point is your autonomy over your own output.
This tool removes **AI-provenance watermarks** that a platform stamps onto content **you generated yourself** — SynthID, the Gemini / Nano Banana sparkle, the Doubao / Jimeng / Qwen / Kling / Baidu / LibLibAI / RunningHub / Samsung visible AI labels, the Chinese TC260 "由…AI生成" label, and C2PA / IPTC / EXIF "Made with AI" metadata. The point is your autonomy over your own output.
It does **not** target watermarks that protect someone else's paid or copyrighted content — stock-agency overlays (Shutterstock, Getty, iStock, Adobe Stock), classifieds-site marks, or any tiled "preview" watermark whose job is to gate a purchase. Removing those is out of scope by design. `erase` is a generic, user-driven region tool for your own objects, not an automatic stock-watermark remover.
## Features
- **Visible watermark removal** — a registry of known marks in their usual places: the Gemini / Nano Banana sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Qwen "千问AI生成" text strip, and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-left, locale-specific). Each mark is **localized to a footprint mask, then filled**: the engine finds the mark, builds a binary mask over its footprint, and one shared, swappable fill inpaints that region. Choose the fill with `--backend`: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN, light, the memory-tight pick where LaMa will not fit), or `lama` (big-LaMa, best quality, heavier, auto-preferred when a learned backend is available); the default `auto` uses LaMa > MI-GAN > cv2, best available. The localizer is cheap CPU (cv2/numpy), so it runs anywhere; the heavier MI-GAN/LaMa fill is opt-in. Detection keys on each mark's own shape (NCC against a captured silhouette; the alpha captures rebuilt by `scripts/visible_alpha_solve.py` are used to detect and to shape the mask, not for pixel recovery). The visual detector needs no metadata, but a borderline (faint or moved) mark is only trusted with corroboration: `--sensitivity` (default `auto`) relaxes a mark's gate when local metadata confirms the vendor or a same-product sibling mark is found; `strict` never relaxes. There is deliberately no "assume this is AI" mode: asserting that an image is AI says nothing about *which* vendor made it or *where* the mark is, which is exactly what a gate bypass needs. If you can SEE a mark the detector missed, point at it with `erase --region x,y,w,h`, or force a known text mark by name with `--mark <name> --no-detect` -- both act on what you actually see instead of relaxing every detector at once. `visible --mark auto` finds and removes every detected mark in one pass. Fast, offline, no GPU. (For arbitrary logos/objects, see `erase`.)
- **Visible watermark removal** — a registry of known marks in their usual places: the Gemini / Nano Banana sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Qwen "千问AI生成" and Kling "可灵AI 3.0" text strips, the Baidu "百度 AI生成" text + tag, the LibLibAI wordmark (bottom-center), the RunningHub "RunningHub AI生成" text (top-left), and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-left, locale-specific). Each mark is **localized to a footprint mask, then filled**: the engine finds the mark, builds a binary mask over its footprint, and one shared, swappable fill inpaints that region. Choose the fill with `--backend`: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN, light, the memory-tight pick where LaMa will not fit), or `lama` (big-LaMa, best quality, heavier, auto-preferred when a learned backend is available); the default `auto` uses LaMa > MI-GAN > cv2, best available. The localizer is cheap CPU (cv2/numpy), so it runs anywhere; the heavier MI-GAN/LaMa fill is opt-in. Detection keys on each mark's own shape (NCC against a captured silhouette; the alpha captures rebuilt by `scripts/visible_alpha_solve.py` are used to detect and to shape the mask, not for pixel recovery). The visual detector needs no metadata, but a borderline (faint or moved) mark is only trusted with corroboration: `--sensitivity` (default `auto`) relaxes a mark's gate when local metadata confirms the vendor or a same-product sibling mark is found; `strict` never relaxes. There is deliberately no "assume this is AI" mode: asserting that an image is AI says nothing about *which* vendor made it or *where* the mark is, which is exactly what a gate bypass needs. If you can SEE a mark the detector missed, point at it with `erase --region x,y,w,h`, or force a known text mark by name with `--mark <name> --no-detect` -- both act on what you actually see instead of relaxing every detector at once. `visible --mark auto` finds and removes every detected mark in one pass. Fast, offline, no GPU. (For arbitrary logos/objects, see `erase`.)
- **Universal region eraser (`erase`)** — remove any logo / watermark / object inside boxes you specify, regardless of position or color. Default cv2 inpainting (CPU, instant); optional big-LaMa via onnxruntime (`lama` extra) for higher quality
- **Invisible watermark removal** — SynthID, StableSignature, TreeRing via diffusion-based regeneration (needs a local GPU, or run it with no setup on [raiw.cc](https://raiw.cc))
- **AI metadata stripping** — EXIF, PNG text chunks, C2PA provenance manifests (PNG / JPEG / AVIF / HEIF / JPEG-XL, **MP4 / MOV / M4V / M4A** at the container level, and **WebM / MP3 / WAV / FLAC / OGG** losslessly via ffmpeg), XMP DigitalSourceType
@@ -36,7 +36,7 @@ It does **not** target watermarks that protect someone else's paid or copyrighte
- **Text and face preservation (default)** — the default pipeline is a canny ControlNet that keeps text and face structure sharp through the removal pass (without copying original pixels, so SynthID is still removed). Use `--pipeline sdxl` for plain SDXL img2img (lighter, no extra model download) on inputs without text or faces. An experimental `--pipeline qwen` runs Qwen-Image (20B, Apache-2.0) img2img, which preserves **text** (including CJK and small text) better than SDXL at equal strength; it is CUDA/cloud-class (does not fit MPS), and its strength floors are not yet certified (pass an explicit `--strength`, especially for Gemini content). Note: measured fidelity (`scripts/fidelity_metrics.py`) shows Qwen wins on text but controlnet preserves **faces** better (Qwen smooths skin more), so Qwen is not a universal upgrade. Canny preserves face *structure*, not *identity* (the regenerated face drifts in likeness). The library does not ship a face-restore extra: every approach evaluated (GFPGAN-on-cleaned, PhotoMaker-V2, InstantID txt2img, InstantID img2img-on-cleaned) regenerated the face via SDXL and made the output look more AI-generated than the cleaned image. The cleaned controlnet output is the least-AI face state achievable without re-introducing SynthID.
- **Batch processing** — process entire directories
- **Detection** — three-stage NCC watermark detection with confidence scoring
- **Provenance detection (`identify`)** — aggregate C2PA issuer, the C2PA soft-binding forensic-watermark vendor (Adobe TrustMark, Digimarc, Imatag, ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, embedded SD/ComfyUI params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the China TC260 AIGC label (XMP, PNG chunk, EXIF, or JPEG segment), the HuggingFace `hf-job-id` job marker, the SynthID metadata proxy, the C2PA cloud-manifest reference (Adobe Durable Content Credentials, when the embedded manifest is stripped), the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Qwen "千问AI生成" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), the open SD/SDXL/FLUX invisible watermark, and (with the `trustmark` extra) the open Adobe TrustMark watermark into one origin-platform + watermark-inventory verdict (`--json` for machine output)
- **Provenance detection (`identify`)** — aggregate C2PA issuer, the C2PA soft-binding forensic-watermark vendor (Adobe TrustMark, Digimarc, Imatag, ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, embedded SD/ComfyUI params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the China TC260 AIGC label (XMP, PNG chunk, EXIF, or JPEG segment), the HuggingFace `hf-job-id` job marker, the SynthID metadata proxy, the C2PA cloud-manifest reference (Adobe Durable Content Credentials, when the embedded manifest is stripped), the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Qwen "千问AI生成" / Kling "可灵AI 3.0" / Baidu "百度 AI生成" / LibLibAI / RunningHub "RunningHub AI生成" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), the open SD/SDXL/FLUX invisible watermark, and (with the `trustmark` extra) the open Adobe TrustMark watermark into one origin-platform + watermark-inventory verdict (`--json` for machine output)
## Examples
@@ -62,14 +62,18 @@ It does **not** target watermarks that protect someone else's paid or copyrighte
| **Doubao** (ByteDance) / China AIGC generators | ✅ "豆包AI生成" text strip (bottom-right) | — | ✅ TC260 AIGC label (`<TC260:AIGC>` XMP, `AIGC` PNG chunk, or EXIF JSON) **+ C2PA** signed by ByteDance Volcano Engine (`volcengine`) | Localize glyph footprint + fill + metadata strip |
| **Jimeng / Dreamina** (即梦AI, ByteDance) | ✅ "★ 即梦AI" wordmark (bottom-right) | — | ✅ TC260 AIGC label + C2PA (Volcano Engine) | Localize glyph footprint + fill + metadata strip |
| **Qwen / Tongyi Qianwen** (千问, Alibaba) | ✅ "千问AI生成" text strip (bottom-right) | — | ✅ TC260 AIGC label | Localize glyph footprint + fill + metadata strip |
| **Kling** (可灵, Kuaishou) | ✅ "可灵AI 3.0" text strip (bottom-right) | — | ✅ TC260 AIGC label | Localize glyph footprint + fill + metadata strip |
| **Baidu** | ✅ "百度 AI生成" text + tag (bottom-right) | — | ✅ TC260 AIGC label | Localize glyph footprint + fill + metadata strip |
| **LibLibAI** (哩布哩布AI) | ✅ "LibLibAI" wordmark (bottom-center) | — | ✅ TC260 AIGC label | Localize glyph footprint + fill + metadata strip |
| **RunningHub** | ✅ "RunningHub AI生成" text (top-left) | — | ✅ TC260 AIGC label | Localize glyph footprint + fill + metadata strip |
| **Samsung Galaxy AI** (Generative Edit, Sketch to Image, ...) | ✅ "✦ Contenuti generati dall'AI" strip (bottom-left, Italian-locale detection) | — | ✅ C2PA (signer "Samsung Galaxy") + `trainedAlgorithmicMedia` / proprietary `genAIType` marker | Localize glyph footprint + fill + metadata strip |
| **Black Forest Labs** (FLUX API) | — | — | ✅ C2PA (`Black Forest Labs API` + `c2pa.ai_generated_content` + `trainedAlgorithmicMedia`) | Metadata strip |
| **StableSignature** (Meta) | — | ✅ In-model watermark | — | Diffusion regeneration |
| **TreeRing** | — | ✅ Latent space watermark | — | Diffusion regeneration |
> Visible overlays are used by Google Gemini / Nano Banana (sparkle logo), by ByteDance's Doubao ("豆包AI生成" corner text) and Jimeng / Dreamina ("★ 即梦AI" wordmark), by Alibaba's Qwen / Tongyi Qianwen ("千问AI生成" corner text), and by Samsung Galaxy AI ("✦ Contenuti generati dall'AI" strip, bottom-left, locale-specific). All are removed by localizing the mark to a footprint mask and inpainting it with one shared fill (cv2 by default, MI-GAN or big-LaMa via `--backend`); the localizer is CPU-cheap and the heavier fills are opt-in. Other services rely on invisible watermarks and/or metadata; our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain. For a visible mark from any other source (any position, any color), use the universal `erase --region` command.
> Visible overlays are used by Google Gemini / Nano Banana (sparkle logo), by ByteDance's Doubao ("豆包AI生成" corner text) and Jimeng / Dreamina ("★ 即梦AI" wordmark), by Alibaba's Qwen / Tongyi Qianwen ("千问AI生成" corner text), by Kuaishou's Kling ("可灵AI 3.0" corner text), by Baidu ("百度 AI生成" text + tag), by LibLibAI ("LibLibAI" wordmark, bottom-center), by RunningHub ("RunningHub AI生成", top-left), and by Samsung Galaxy AI ("✦ Contenuti generati dall'AI" strip, bottom-left, locale-specific). All are removed by localizing the mark to a footprint mask and inpainting it with one shared fill (cv2 by default, MI-GAN or big-LaMa via `--backend`); the localizer is CPU-cheap and the heavier fills are opt-in. Other services rely on invisible watermarks and/or metadata; our diffusion-based regeneration works against any invisible watermark in pixel or frequency domain. For a visible mark from any other source (any position, any color), use the universal `erase --region` command.
> **Detection:** `remove-ai-watermarks identify <image>` reports the origin platform and watermark inventory for all the signals above — C2PA issuer, the C2PA soft-binding forensic-watermark vendor (TrustMark / Digimarc / Imatag / ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, the China TC260 AIGC label (XMP, PNG chunk, EXIF, or JPEG segment), the HuggingFace `hf-job-id` job marker, embedded generation params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the SynthID metadata proxy, the C2PA cloud-manifest reference (Adobe Durable Content Credentials, when the embedded manifest is stripped), the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Qwen "千问AI生成" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), and (with the `[detect]` / `[trustmark]` extras) the open SD/SDXL/FLUX and Adobe TrustMark invisible watermarks. SynthID and the proprietary soft-binding watermarks (Digimarc etc.) have no local decoder, so they are reported by metadata proxy / vendor name only.
> **Detection:** `remove-ai-watermarks identify <image>` reports the origin platform and watermark inventory for all the signals above — C2PA issuer, the C2PA soft-binding forensic-watermark vendor (TrustMark / Digimarc / Imatag / ...), IPTC "Made with AI" plus the IPTC 2025.1 `AISystemUsed` field, the China TC260 AIGC label (XMP, PNG chunk, EXIF, or JPEG segment), the HuggingFace `hf-job-id` job marker, embedded generation params, EXIF/XMP generator tags, the xAI/Grok EXIF signature, the SynthID metadata proxy, the C2PA cloud-manifest reference (Adobe Durable Content Credentials, when the embedded manifest is stripped), the visible marks (Gemini sparkle plus the Doubao "豆包AI生成" / Jimeng "即梦AI" / Qwen "千问AI生成" / Kling "可灵AI 3.0" / Baidu "百度 AI生成" / LibLibAI / RunningHub "RunningHub AI生成" / Samsung Galaxy AI "Contenuti generati dall'AI" text marks), and (with the `[detect]` / `[trustmark]` extras) the open SD/SDXL/FLUX and Adobe TrustMark invisible watermarks. SynthID and the proprietary soft-binding watermarks (Digimarc etc.) have no local decoder, so they are reported by metadata proxy / vendor name only.
## How it works
@@ -332,8 +336,10 @@ remove-ai-watermarks identify image.png
# Visible watermark only — fast, offline, CPU. --mark auto (default) removes every
# detected known mark (Gemini sparkle / Doubao "豆包AI生成" / Jimeng "即梦AI" /
# Samsung Galaxy AI "Contenuti generati dall'AI"); force one with
# --mark gemini / doubao / jimeng / qwen / samsung. Removal localizes each mark to a
# Qwen "千问AI生成" / Kling "可灵AI 3.0" / Baidu "百度 AI生成" / LibLibAI /
# RunningHub "RunningHub AI生成" / Samsung Galaxy AI "Contenuti generati dall'AI");
# force one with --mark gemini / doubao / jimeng / qwen / kling / baidu / liblib /
# runninghub / samsung. Removal localizes each mark to a
# footprint mask and inpaints it with a shared fill; --backend auto|cv2|migan|lama
# (default auto) picks the fill (auto = LaMa > MI-GAN > cv2, best available).
# --sensitivity auto|strict (default auto) sets how hard a borderline mark is
+26
View File
@@ -271,6 +271,32 @@ The cost (mislabel ~8-33% of non-Gemini content as Gemini) outweighs the benefit
**The clean-arm contamination trap (load-bearing for any future calibration):** the 2026-07-18 `present: []` labels are in the vocabulary of the REGISTERED marks only, so 146 of the 432 "clean" frames sit in a TC260 cohort -- including Qwen-cohort frames visibly carrying 千问AI生成. They made up the clean arm's entire top tail (clean p99 0.37 -> 0.69). `vendor_mark_calibrate.load_sets` now excludes every frame in ANY TC260 cohort from the clean arm; a gate read off the unguarded arm is meaningless.
## `runninghub_engine.py`
`runninghub_engine.py` — **thin `TextMarkEngine` subclass, registered 2026-07-22.** RunningHub (hosted ComfyUI platform, USCC 91340100MAEB4N8H76, 73-frame cohort) "RunningHub AI生成" detector + localizer, **top-left** (the first `corner="tl"` mark), faint mid-gray latin+CJK text. Feeds `identify` as `visible_runninghub`.
* **`gray` front-end (the third one, added for this mark):** the mark's faint gray is suppressed by the white top-hat to clean-arm levels (positives 0.16-0.23 vs clean p99 0.31), while raw-grayscale silhouette NCC separates (positives 0.38-0.54 vs clean p99 0.264 / max 0.304 on 283 guarded clean frames). It is contrast-DEPENDENT, unlike tophat -- one method `_gray_best` serves both detection and the mask, same one-method parity contract as `_tophat_best`.
* **Tight ladder (0.95, 1.0, 1.05) exactly on the measured 0.32-of-width:** the NCC comb is razor-sharp in size (0.537 on-size, 0.223 at +5.6% off -- Qwen's comb behaviour, measured again here), so the shared 3 rungs (nearest rung 5.6% off) collapsed the match to 0.22 and the first calibration showed no separation at all.
* **Anchor gate in `detect`:** the full-corpus sweep (`data/spaces/_sweep_new_marks.py`, 42009 files) fired on 37 outside-cohort frames at 0.34-0.38 (hair tops, shelves, window frames, CJK banners) -- no NCC threshold separates them from the 0.381 positives. Every positive sits at the measured corner (x 0.008-0.014, y 0.005-0.007 of the frame) and every false fire off it (x 0.013-0.150, y 0.009-0.045), so detection additionally requires the match box inside x<=0.025 / y<=0.015 of the frame. 0/37 false, 4/4 positives kept.
* **Footprint is always the detector's match box** (never the binary blob): the blob under-segments the faint head glyphs and left "Runni" unremoved (caught visually on the first removal). Gate 0.34, STRICT ONLY. Regression: `tests/test_runninghub_engine.py`.
## `baidu_engine.py`
`baidu_engine.py` — **thin `TextMarkEngine` subclass with a custom `footprint_mask`, registered 2026-07-22.** Baidu (USCC 91110000802100433B, 16-frame cohort) "百度 AI生成" detector + localizer, bottom-right: a white bold 百度 text run + a separate white rounded tag with dark "AI生成". Feeds `identify` as `visible_baidu`.
* **Detection keys on the 百度 text run ONLY.** A two-component (text+pill) template was measured and rejected: the solid white pill is a bright-blob magnet and both front-ends scored the clean arm at cohort levels (tophat clean p95 0.445 / gray clean p95 0.487 vs cohort ~0.5). The text-only silhouette separates (cohort 0.39-0.65 vs clean max 0.352). The white tag is still removed: the footprint extends to the corner (below).
* **Two load-bearing rival margins** (`rivals=("doubao_alpha.png","qwen_alpha.png")`): 百度 vs 豆包 share their second glyph, and 百度 vs 千问 are near-identical after binarization -- at the 0.37 gate the template fired on 45.8% of 400 Doubao-marked frames and, on the 741-frame blind-labelled eval set, on 12 Qwen-marked frames at 0.38-0.43. Doubao's template beats it by ~0.56 on Doubao marks, Qwen's by 0.17-0.35 on Qwen marks; the 0.10 margin suppresses all crossfire at zero genuine-Baidu cost (cohort fire+m == fire).
* **Gate history, each step measured:** 0.37 from the clean arm (max 0.352) -> 0.43 after the eval-set crossfires (the one 抖音 AI创作 fire at 0.425 named no registered rival) -> **0.48** after the full-corpus sweep put outside-cohort TRUE carriers at 0.50-0.66 vs the false arm max 0.47 (大众点评 UI, a math blackboard, an 80s banner). Cohort keeps 7/16 (all true); the sweep also found 6 metadata-STRIPPED true Baidu carriers the TC260 cohort cannot see -- the direct evidence that registration pays beyond the cohort.
* **Custom `footprint_mask`:** the tag's flat white interior gives no top-hat response (a top-hat answers edges, not flats), so the base blob bbox ended at the text run and the fill left the tag as a ghost. The mask is the detector's match box extended RIGHT to the corner. STRICT ONLY. Regression: `tests/test_baidu_engine.py`.
## `liblib_engine.py`
`liblib_engine.py` — **thin `TextMarkEngine` subclass with a custom `footprint_mask`, registered 2026-07-22.** LibLibAI (哩布哩布AI, USCC 91110105MACJ6K1C8A, 15-frame cohort) triangle logo + "LibLibAI" wordmark detector + localizer, **bottom-CENTER** (the first `corner="bc"` mark; the locate box is horizontally centered). Feeds `identify` as `visible_liblib`.
* **The discriminative lever is the silhouette FONT.** With the CJK house font (STHeiti) the cohort scored 0.31-0.47 against a full-corpus false arm (latin UI text bands, website screenshots) at 0.50 -- no separation at any gate. Measured across 7 candidate fonts, **Arial** lifts the cohort to 0.42-0.73 and DROPS the false arm to max 0.398: generic latin text matches the wrong font less, which is where the discrimination comes from. Gate 0.42 keeps all 8 marked cohort frames (0.43-0.59).
* **Per-mark size floor (`_MIN_SHORT_SIDE=480`):** the one false fire with the final template was a 200x200 icon (0.444, on a 20px template). The shared `_MIN_DETECT_SHORT_SIDE` (200) is a crash guard, not a discrimination floor; the template needs ~48px to discriminate.
* **Custom `footprint_mask`:** the base blob bbox was wrong in both directions -- it bled UP into background structure (ate a shirt's real print on the 768x1024 cohort frame) and never owned the triangle logo. The mask is the detector's match box extended LEFT by ~1.3 glyph heights (the logo is ~1.0x the glyph height, gap ~0.3x, measured on the cohort zoom). STRICT ONLY. Regression: `tests/test_liblib_engine.py`.
## `region_eraser.py`
`region_eraser.py` — universal region eraser (`erase` CLI) AND the shared fill backend behind `watermark_registry.fill` for the visible localize -> fill removal. `erase(image, boxes=|mask=, backend=)` accepts grayscale (2D) and RGBA (4-channel) inputs on **all** backends (each splits off any alpha plane and re-attaches it unchanged, and promotes grayscale to BGR): `boxes_to_mask` → one of three backends.
+66 -2
View File
@@ -742,8 +742,10 @@ priority order:
against the contamination-guarded clean arm, crossfire against doubao/jimeng. 可灵
additionally stamps a second mark bottom-LEFT, which no current text-mark config
expresses (the pill is top-left; a bottom-left CJK mark needs a `corner="bl"` CJK
config -- samsung is `bl` but Latin-script and width-based). 星绘/百度 are NOT in the
corpus in labelable quantity -- verified, do not hunt them again.
config -- samsung is `bl` but Latin-script and width-based). 星绘 is NOT in the
corpus in labelable quantity -- verified, do not hunt it again. (百度 WAS found
later via the USCC cohort harvest and is registered since 2026-07-22 -- see
"The 2026-07-22 vendor round" below.)
**STATUS 2026-07-21 (same day): 可灵 REGISTERED, 元宝 measured and PARKED.**
* **可灵 (`kling_engine.py`)** -- "可灵AI 3.0" bottom-right, strict-only, gate
@@ -794,6 +796,68 @@ to confirm the whole surface still works after a change is
`uv run python scripts/real_examples_e2e.py` (~2 min, real corpus examples through the real
CLI) plus `uv run python scripts/robustness_suite.py` (~3 min, adversarial inputs).
### The 2026-07-22 vendor round -- 3 REGISTERED (runninghub / baidu / liblib), 2 parked
A fresh metadata-mining pass over the whole corpus (`data/spaces/_mine_signals.py`)
found NO new metadata signals (the channel is saturated), so the round worked the
visible-mark cohorts (`vendor_cohort_harvest.py`, 4606 TC260 carriers / 46 entities).
Registered, each by the qwen playbook (synthetic silhouette -> measured geometry ->
clean-arm gate -> crossfire -> full-corpus sweep):
* **RunningHub (`runninghub_engine.py`)** -- "RunningHub AI生成" TOP-LEFT (a new
`corner="tl"`), faint mid-gray text. The white top-hat suppresses it to clean-arm
levels (positives 0.16-0.23 vs clean p99 0.31), so it introduced the third
detection front-end, **`gray`** (raw-grayscale silhouette NCC, contrast-DEPENDENT):
positives 0.38-0.54 vs clean max 0.295 -> gate 0.34, strict-only. The NCC comb is
razor-sharp in size (0.537 on-size, 0.223 at +5.6%), so the ladder is a tight
(0.95, 1.0, 1.05) exactly on the measured 0.32-of-width. Two measured traps with
their fixes: (1) the binary blob under-segments the faint head glyphs, so the
blob-bbox footprint left "Runni" unremoved -- the gray front-end's footprint is
always the detector's own match box; (2) the full-corpus sweep surfaced 37/42009
outside-cohort false fires at 0.34-0.38 (hair, shelves, CJK banners) with no NCC
separation from the 0.381 positives -- the **anchor gate** (the match must sit at
the measured corner, x<=0.025/y<=0.015 of the frame) rejects all 37 at zero
positive cost.
* **Baidu (`baidu_engine.py`)** -- "百度" white bold text + a white rounded tag
"AI生成", bottom-right. Detection keys on the 百度 TEXT RUN ONLY: a
text+pill template was a measured bright-blob magnet (no separation on either
front-end). Gate history, each step measured: 0.37 from the clean arm (max 0.352);
the 741-frame eval set then fired 14x outside the cohort and 13 were NOT the
vendor (12x 千问 -- 百/千 are near-identical after binarization -- plus one 抖音
AI创作 at 0.425), so `rivals=("doubao_alpha.png","qwen_alpha.png")` (both margins
load-bearing, zero genuine cost) and the gate moved 0.37 -> 0.43; the full-corpus
sweep then put outside-cohort true carriers at 0.50-0.66 vs the false arm max
0.47, so the gate settled at **0.48**. Cohort: 7/16 fire (all true). The
footprint is custom: the tag's flat white interior gives no top-hat response, so
a blob bbox leaves the tag as a ghost -- the mask is the match box extended right
to the corner.
* **LibLibAI (`liblib_engine.py`)** -- triangle logo + "LibLibAI" wordmark
**bottom-CENTER** (a new `corner="bc"`). The discriminative lever was the FONT:
STHeiti scored the cohort 0.31-0.47 against a false arm (latin UI text bands) at
0.50; measured across 7 fonts, **Arial** lifts the cohort to 0.42-0.73 and DROPS
the false arm to max 0.398 (generic latin text matches the wrong font less). Gate
0.42, strict-only; a per-mark size floor (`_MIN_SHORT_SIDE=480`) backs it (the
one remaining false fire was a 200x200 icon on a 20px template). Custom
footprint: match box extended left by ~1.3 glyph heights for the triangle logo
(the blob bbox both bled into background structure -- ate a shirt's real print --
and did not own the logo).
Parked, both as measured negatives with the silhouette kept in
`render_vendor_silhouettes.py` as the starting point:
* **Zhipu Qingyan (清言·AI生成)** -- 7-frame cohort, white semi-transparent text +
swirl logo. On both front-ends the cohort scores 0.34-0.39 vs clean max
0.34-0.37 -- no separation at any render/box setting (text-only and
logo-composite templates, two CJK fonts). Same wall class as 元宝.
* **MiniMax / Hailuo AI** -- only 1 of 6 cohort frames carries a visible mark
(Hailuo is a video product; the mark is a video-frame stamp). The xinghui rule:
nothing registered off a single frame.
The full-corpus sweep harness is `data/spaces/_sweep_new_marks.py` (read-only,
gitignored); its artifact `_new_marks_sweep.jsonl` records every fire. The sweep
also proved the outside-cohort value of registration: 6 metadata-STRIPPED true
Baidu carriers the TC260 cohort cannot see are now detected and cleaned.
### The 千问 harvest (2026-07-21) -- RESOLVED, registered the same day
**The unlock: the TC260 label is not anonymous.** Its `ContentProducer` field carries the
+1 -1
View File
@@ -70,7 +70,7 @@ Two consequences we can exploit: (1) the 5% floor is a **scale prior** -- a comp
**星绘 is ByteDance (VERIFIED (a): Baidu Baike + App Store listing, now branded 豆包旗下, team folded into Doubao April 2025).** So `星绘AI生成` is very likely the Doubao house style -- same typeface, same corner, possibly the same top-left `AI生成` pill. Starting from the Doubao `TextMarkConfig` and swapping the two lead glyphs is the cheap path. String/position themselves are (c) inferred.
**Baidu: could not establish.** No primary or credible secondary source names the exact string or position; it could be `百度AI生成`, `文心一格AI生成`, or product-specific. Harvest the glyphs from corpus positives, not the web.
**Baidu: RESOLVED 2026-07-22, registered (`baidu_engine.py`).** The mark is a white bold "百度" text run + a separate white rounded tag with dark "AI生成", bottom-right -- settled by the TC260 USCC cohort harvest (16 frames, USCC 91110000802100433B), not by web research. Detection keys on the text run only; details in `docs/module-internals.md`.
**Meta `Imagined with AI` (string VERIFIED (a) from Meta's own newsroom; POSITION NOT VERIFIED).** Sources conflict (bottom-left vs bottom-right) and one claims newer Meta models dropped the visible mark for invisible watermarking; none survived a fetch. Do NOT encode a corner without a corpus sample. Meta also embeds IPTC + invisible watermarks, which `identify` already reads. Source: `https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/`.
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "remove-ai-watermarks"
version = "0.17.0"
version = "0.18.0"
description = "AI watermark remover: strip visible and invisible AI watermarks (Gemini / Nano Banana sparkle, SynthID) and provenance metadata (C2PA, EXIF) from images"
readme = "README.md"
requires-python = ">=3.10"
+40 -2
View File
@@ -127,6 +127,33 @@ MARKS = {
# max on a diverse clean arm, so registration is a gate pick (0.42) the moment
# more unique carriers arrive.
"catlogo_alpha.png": "CATLOGO", # sentinel: drawn by draw_catlogo(), not font-rendered
# RunningHub (ComfyUI platform, USCC 91340100MAEB4N8H76, 73-frame cohort
# 2026-07-22): white one-line "RunningHub AI生成" text mark.
"runninghub_alpha.png": "RunningHub AI生成",
# LibLibAI / 哩布哩布AI (USCC 91110105MACJ6K1C8A, 15-frame cohort): white
# "LibLibAI" wordmark with a triangle logo (logo not rendered, logos vary).
"liblib_alpha.png": "LibLibAI",
# Zhipu Qingyan (USCC 91110108MA01KP2T5U, 7-frame cohort): white bold
# "清言·AI生成" with a circular logo (logo not rendered). PARKED 2026-07-22
# as a measured negative: on both front-ends the cohort scores 0.34-0.39
# against a clean-arm max of 0.34-0.37 -- no separation at any render/box
# setting (text-only and logo-composite templates both plateau ~0.34 raw;
# the white semi-transparent text on variable backgrounds is the wall).
# Silhouette stays as the starting point for a structural/learned lever.
"qingyan_alpha.png": "清言·AI生成",
# MiniMax / Hailuo (6-frame cohort): "MINIMAX" + "Hailuo AI" latin wordmarks.
# PARKED 2026-07-22: only 1 of the 6 cohort frames carries a visible mark --
# nothing to calibrate recall against (the xinghui rule). Registration is a
# gate pick once more unique carriers arrive.
"hailuo_alpha.png": "Hailuo AI",
# Baidu (USCC 91110000802100433B, 16-frame cohort): white bold "百度" text
# + a separate white rounded tag with dark "AI生成", bottom-right. Detection
# keys on the 百度 text run ONLY: a two-component template (text+pill) scored
# at clean-arm levels (pill = bright-blob magnet, clean p95 0.45-0.55 vs cohort
# ~0.5, no separation on either front-end, 2026-07-22); the text-only silhouette
# separates (cohort 0.39-0.65 vs clean max 0.352). The white tag is removed
# with the mark because the fill blob covers both bright components.
"baidu_alpha.png": "百度",
}
# Per-mark post-processing for the multi-line / slanted stamps (see render()).
@@ -136,6 +163,15 @@ MARK_OPTS: dict[str, dict[str, Any]] = {
# tight gap + stroke dilation + shear -0.75 reaches 0.65-0.70 on the same frames,
# at/above the real-vs-real ceiling (~0.6).
"yuanbao_alpha.png": {"gap_frac": 0.05, "dilate": 2, "shear": -0.75},
# Qingyan's real stamp is a heavier weight than STHeiti Medium -- Hiragino
# Sans GB W6 matches the measured stroke (2026-07-22; with Medium the
# silhouette aspect came out 0.19 vs the real 0.28 and NCC plateaued ~0.3).
"qingyan_alpha.png": {"font": "/System/Library/Fonts/Hiragino Sans GB.ttc", "font_index": 2},
# LibLibAI's wordmark is set in an Arial-class grotesque, not STHeiti:
# measured 2026-07-22 across 7 candidate fonts, Arial lifts the cohort
# positives from 0.31-0.47 to 0.42-0.73 while the full-corpus false-fire arm
# DROPS to max 0.398 (generic latin UI text matches the wrong font less).
"liblib_alpha.png": {"font": "/System/Library/Fonts/Supplemental/Arial.ttf"},
}
@@ -151,16 +187,18 @@ def render(text: str, width: int = 335, opts: dict[str, Any] | None = None) -> n
gap_frac = float(opts.get("gap_frac", 0.15))
dilate = int(opts.get("dilate", 0))
shear_k = float(opts.get("shear", 0.0))
font_path = str(opts.get("font", _FONT))
font_index = int(opts.get("font_index", 0))
probe = Image.new("L", (10, 10))
d0 = ImageDraw.Draw(probe)
lines = text.split("\n")
size = 8
while size < 200: # grow until the LONGEST line fills the target width
f = ImageFont.truetype(_FONT, size)
f = ImageFont.truetype(font_path, size, index=font_index)
if max(d0.textbbox((0, 0), ln, font=f)[2] for ln in lines) >= width * 0.98:
break
size += 1
font = ImageFont.truetype(_FONT, size)
font = ImageFont.truetype(font_path, size, index=font_index)
boxes = [d0.textbbox((0, 0), ln, font=font) for ln in lines]
line_h = max(bb[3] - bb[1] for bb in boxes)
gap = max(1, int(line_h * gap_frac))
+8
View File
@@ -538,6 +538,12 @@ def main() -> None:
ap.add_argument("--name", default="", help="label for output files (defaults to the asset stem)")
ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2))
ap.add_argument("--scale-basis", choices=("short", "width"), default="short")
ap.add_argument(
"--corner",
choices=("br", "bl", "tl"),
default=None,
help="override the inherited br corner (e.g. tl for RunningHub)",
)
ap.add_argument("--sheets", action="store_true")
ap.add_argument(
"--fit-geometry",
@@ -573,6 +579,8 @@ def main() -> None:
):
if arg is not None:
overrides[field] = arg
if a.corner is not None:
overrides["corner"] = a.corner
if ladder is not None:
overrides["ladder"] = ladder
+1 -1
View File
@@ -25,7 +25,7 @@ _os.environ.setdefault("TRANSFORMERS_VERBOSITY", "error")
_warnings.filterwarnings("ignore", message=r".*ImageProcessorFast.*")
__version__ = "0.17.0"
__version__ = "0.18.0"
__all__ = ["__version__", "remove_visible", "visible_provenance"]
+70 -5
View File
@@ -97,7 +97,9 @@ class TextMarkConfig:
name: str # short label for log lines (e.g. "Doubao")
asset_name: str # bundled alpha PNG under assets/ (e.g. "doubao_alpha.png")
corner: Literal["br", "bl"] # bottom-right (Doubao/Jimeng) or bottom-left (Samsung)
corner: Literal[
"br", "bl", "tl", "bc"
] # bottom-right (Doubao/Jimeng), bottom-left (Samsung), top-left (RunningHub), bottom-center (LibLibAI)
margin_floor: int # min margin in px for locate (4 for br marks, 2 for Samsung)
# locate geometry (fraction of scale_base -- see scale_base())
width_frac: float
@@ -125,7 +127,12 @@ class TextMarkConfig:
# correlates a binary silhouette against it; "tophat" correlates the CONTINUOUS
# top-hat response against a soft template and never binarizes. See
# TextMarkEngine.tophat_response for the measurement that motivated the split.
detect_frontend: Literal["binary", "tophat"] = "binary"
# "gray" correlates the silhouette against the raw GRAYSCALE of the locate box:
# for a faint mid-gray mark (RunningHub) the top-hat's background-subtraction and
# max-normalization suppress the response to clean-arm levels (positives 0.16-0.23
# vs clean p99 0.31), while raw gray NCC separates (positives 0.38-0.54 vs clean
# p99 0.264 / max 0.304, measured 2026-07-22). Contrast-DEPENDENT, unlike tophat.
detect_frontend: Literal["binary", "tophat", "gray"] = "binary"
# Gaussian sigma applied to the template in the "tophat" front-end (0 = none).
template_blur: float = 0.0
# Which image dimension the mark's size and margins scale with. VENDOR-SPECIFIC,
@@ -391,6 +398,42 @@ class TextMarkEngine:
"""The detection score alone -- the box the removal mask needs is discarded here."""
return self._tophat_best(image, loc)[0]
def _gray_best(self, image: NDArray[Any], loc: TextMarkLocation) -> tuple[float, tuple[int, int, int, int] | None]:
"""Best TM_CCOEFF_NORMED of the silhouette against the raw GRAYSCALE ROI, and
the ROI-local box (x0, y0, x1, y1) of that best match.
Mirrors :meth:`_tophat_best` (same ladder sweep, same one-method contract so
detection and the removal mask can never drift), but skips the top-hat
entirely: the RunningHub mark is a faint mid-gray text the top-hat's
background subtraction suppresses to clean-arm levels, while raw gray NCC
separates (see ``TextMarkConfig.detect_frontend``). Contrast-DEPENDENT by
construction, so the gate must be picked against the clean arm, which is
what ``scripts/vendor_mark_calibrate.py`` does.
"""
c = self.config
x, y, bw, bh = loc.bbox
if bh < 16 or bw < 16:
return (0.0, None)
roi = cv2.cvtColor(image_io.to_bgr(image[y : y + bh, x : x + bw]), cv2.COLOR_BGR2GRAY)
sil = self._glyph_silhouette()
if sil is None:
return (0.0, None)
base = self.scale_base(image)
best_score = 0.0
best_box: tuple[int, int, int, int] | None = None
for scale in c.ladder:
gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
gh = max(4, int(c.alpha_height_frac * base * scale))
if gw >= roi.shape[1] or gh >= roi.shape[0]:
continue
tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA)
result = cv2.matchTemplate(roi, tmpl, cv2.TM_CCOEFF_NORMED)
_, score, _, top_left = cv2.minMaxLoc(result)
if score > best_score:
tx, ty = int(top_left[0]), int(top_left[1])
best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1)
return (best_score, best_box)
def scale_base(self, image: NDArray[Any]) -> int:
"""The image dimension this mark's geometry scales with.
@@ -433,8 +476,14 @@ class TextMarkEngine:
wm_h = max(16, int(base * c.height_frac))
margin_x = max(c.margin_floor, int(base * c.margin_x_frac))
margin_b = max(c.margin_floor, int(base * c.margin_bottom_frac))
x = max(0, w - margin_x - wm_w) if c.corner == "br" else min(margin_x, max(0, w - wm_w))
y = max(0, h - margin_b - wm_h)
if c.corner == "br":
x = max(0, w - margin_x - wm_w)
elif c.corner == "bc": # bottom-center: horizontally centered, margin_x unused
x = max(0, (w - wm_w) // 2)
else:
x = min(margin_x, max(0, w - wm_w))
# "tl" anchors at the top instead: margin_bottom_frac is then the TOP margin.
y = min(margin_b, max(0, h - wm_h)) if c.corner == "tl" else max(0, h - margin_b - wm_h)
wm_w = min(wm_w, w - x)
wm_h = min(wm_h, h - y)
return TextMarkLocation(x=x, y=y, w=wm_w, h=wm_h, is_fallback=True)
@@ -526,6 +575,15 @@ class TextMarkEngine:
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
logger.debug("%s detect (tophat): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
return det
if c.detect_frontend == "gray":
# Same no-coverage-gate reasoning as tophat: the gray front-end never
# binarizes, so a blob-area heuristic does not apply to it either.
score = self._gray_best(image, loc)[0]
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
det.confidence = score
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
logger.debug("%s detect (gray): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
return det
if coverage >= c.detect_min_coverage:
score = self._template_match_score(box, self.scale_base(image))
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
@@ -578,7 +636,14 @@ class TextMarkEngine:
glyph = self.extract_mask(image, loc) # box-sized, 255 = glyph
ys, xs = np.where(glyph > 0)
box: tuple[int, int, int, int] | None = None
if xs.size >= self._MIN_GLYPH_PIXELS:
if self.config.detect_frontend == "gray" and self.detect(image).detected:
# The gray front-end exists for marks the top-hat under-segments, so the
# binary blob is NOT authoritative here: trusting it first bounded the
# fill by a PARTIAL blob (the faint head glyphs dropped out) and left the
# leftmost "Runni" of "RunningHub AI生成" unremoved (2026-07-22). Use the
# detector's own best-match box, same as the tophat faint path below.
_, box = self._gray_best(image, loc)
elif xs.size >= self._MIN_GLYPH_PIXELS:
box = (int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max()))
elif self.config.detect_frontend == "tophat" and self.detect(image).detected:
# A mark found only by the CONTINUOUS front-end has no binary glyph blob to
Binary file not shown.

After

Width:  |  Height:  |  Size: 3.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 KiB

+203
View File
@@ -0,0 +1,203 @@
"""Baidu visible watermark detector/localizer.
Baidu stamps its generations with a white bold "百度" text run plus a separate
white rounded tag carrying dark "AI生成", bottom-right -- the China TC260
explicit AIGC label. Detection keys on the **百度 text run only**: a
two-component template (text + pill tag) was measured and REJECTED -- the solid
white pill is a bright-blob magnet and both front-ends scored the clean arm at
cohort levels (tophat clean p95 0.445 / gray clean p95 0.487 vs cohort ~0.5,
2026-07-22). The text-only silhouette separates cleanly (below). The white tag
is still removed with the mark: the fill blob covers both bright components in
the corner box.
Removal is the shared **localize -> fill** (:meth:`footprint_mask` ->
``region_eraser``). This module supplies only Baidu's tuned
:class:`TextMarkConfig` (``assets/baidu_alpha.png`` -- a font-rendered
synthetic silhouette from ``scripts/render_vendor_silhouettes.py``, never cut
from an upload).
Measured on the vendor cohort (16 TC260 carriers whose producer USCC
91110000802100433B names Baidu, harvested 2026-07-22 by
``scripts/vendor_cohort_harvest.py``), NOT inherited from Doubao:
* The 百度 text run is 0.090 of the SHORT side wide (measured on 720/768/
1024-px frames), with its right edge ~0.099 of short off the right edge
(the pill tag sits between the text and the corner), bottom margin
~0.006; the locate box below covers the whole mark (text + tag).
* Gate 0.43 (tophat front-end): on 278 hand-labelled clean frames
(cohort-contamination-guarded) the max is 0.352 / p99 0.314, and the
visibly-marked cohort frames score 0.386-0.65. Picked over the clean-arm
0.37 after a full-corpus check on the 741-frame blind-labelled eval set
surfaced 13 cross-fires at 0.38-0.43 (12 Qwen marks + one 抖音 mark) --
see DETECT_NCC_THRESHOLD below. At 0.43 the cohort keeps 7 detections
(0.61-0.65) and the whole 741 set fires only on the true Baidu frame.
* STRICT ONLY (``provenance_ncc_factor`` 1.0): the cohort is small (16) and
the sub-gate band is unmeasured, so no provenance relaxation exists.
"""
# The module-level _alpha_template / _glyph_silhouette / _template_match_score below
# are thin test-facing shims (imported by tests/), so pyright's src-only pass sees them
# as unused; the use is cross-module.
# pyright: reportUnusedFunction=false
from __future__ import annotations
from typing import TYPE_CHECKING, Any
from remove_ai_watermarks import _text_mark_engine
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
if TYPE_CHECKING:
from pathlib import Path
from numpy.typing import NDArray
# Locate geometry as a fraction of the image SHORT side (measured basis). The
# box covers the text run AND the pill tag to its right (tag right edge ~0.002
# off the frame edge, text run left edge ~0.19 off).
WM_WIDTH_FRAC = 0.25
WM_HEIGHT_FRAC = 0.07
MARGIN_RIGHT_FRAC = 0.002
MARGIN_BOTTOM_FRAC = 0.002
# Glyph appearance: white bold text on a usually-darker background (white
# top-hat), same overlay class as Doubao -- inherited, harmless because the
# tophat front-end turns these gates into weights.
MAX_SATURATION = 55
LOGO_MIN_LUMA = 150
TOPHAT_DELTA = 12
DETECT_MIN_COVERAGE = 0.04 # unused by the tophat front-end (kept for config parity)
# Calibrated 2026-07-22 on the vendor cohort vs 278 hand-labelled clean frames
# (clean p99 0.314 / max 0.352), THEN raised 0.37 -> 0.43 after a full-corpus
# check: on the 741-frame blind-labelled eval set the 0.37 gate fired 14 times
# outside the cohort, and only ONE was the vendor -- 12 were 千问AI生成 (Qwen)
# marks (the 百/千 first glyphs are near-identical after binarization) and one
# was a 抖音 AI创作 mark at 0.425. The Qwen fires are handled by the rival
# margin (Qwen scores 0.58-0.76 there, beating Baidu by 0.17-0.35), but the
# 抖音 one named no registered rival, so the gate moved above it. Cost: the
# cohort's low trio at 0.386 (3 genuine marks) -- precision over recall on a
# small cohort. Raised again 0.43 -> 0.48 after the full-corpus sweep
# (2026-07-22): outside-cohort true Baidu carriers score 0.50-0.66 while the
# false fires (大众点评 UI, a math blackboard, an 80s banner, a checkerboard)
# top out at 0.47. Remaining cohort detections: 7 at 0.61-0.65, plus the 6
# metadata-stripped true carriers the cohort cannot see.
DETECT_NCC_THRESHOLD = 0.48
# Detection-silhouette geometry (fraction of the short side): the 百度 text run
# only, measured 0.090 wide with aspect 0.51.
_ALPHA_WIDTH_FRAC = 0.090
_ALPHA_HEIGHT_FRAC = 0.046
# Tight ladder: the NCC comb is sharp in size (see runninghub_engine), so the
# nominal sits exactly on the measured 0.090 with +-5% rungs.
_LADDER = (0.95, 1.0, 1.05)
_CONFIG = TextMarkConfig(
name="Baidu",
asset_name="baidu_alpha.png",
corner="br",
margin_floor=4,
width_frac=WM_WIDTH_FRAC,
height_frac=WM_HEIGHT_FRAC,
margin_x_frac=MARGIN_RIGHT_FRAC,
margin_bottom_frac=MARGIN_BOTTOM_FRAC,
max_saturation=MAX_SATURATION,
logo_min_luma=LOGO_MIN_LUMA,
tophat_delta=TOPHAT_DELTA,
morph_open_size=5,
detect_min_coverage=DETECT_MIN_COVERAGE,
detect_ncc_threshold=DETECT_NCC_THRESHOLD,
detect_frontend="tophat",
scale_basis="short",
ladder=_LADDER,
alpha_width_frac=_ALPHA_WIDTH_FRAC,
alpha_height_frac=_ALPHA_HEIGHT_FRAC,
min_gw=8,
# Load-bearing rival margins (crossfire measured 2026-07-22): the 百度 and
# 豆包 silhouettes share their second glyph and a similar first, and 百度 vs
# 千问 are near-identical after binarization -- at the 0.37 gate this
# template fires on 45.8% of 400 Doubao-marked frames AND on Qwen-marked
# frames at 0.38-0.43. Doubao's template beats it by ~0.56 on Doubao marks,
# Qwen's by 0.17-0.35 on Qwen marks, so the 0.10 margin suppresses all of
# that crossfire at zero genuine-Baidu cost (cohort fire+m == fire).
rivals=("doubao_alpha.png", "qwen_alpha.png"),
# STRICT ONLY: small cohort, the relaxed band is unmeasured.
provenance_ncc_factor=1.0,
)
BaiduDetection = TextMarkDetection
def _alpha_template() -> NDArray[Any] | None:
"""The bundled Baidu alpha template (float [0,1]), or None."""
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
def _glyph_silhouette() -> NDArray[Any] | None:
"""Binary "百度" silhouette (255 = glyph) from the alpha map, or None."""
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
"""TM_CCOEFF_NORMED of the Baidu glyph silhouette against ``box_mask``."""
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
class BaiduEngine(TextMarkEngine):
"""Detect/localize the visible Baidu "百度 AI生成" mark (bottom-right; localize -> fill)."""
def __init__(self) -> None:
super().__init__(_CONFIG)
def footprint_mask(
self, image: NDArray[Any] | None, *, force: bool = False, dilate: int | None = None
) -> NDArray[Any] | None:
"""Full-frame mask of the WHOLE mark (text run + the pill tag to its right).
The base class's blob-bbox footprint UNDERCOVERS this mark: the white tag's
flat interior gives no top-hat response (a top-hat answers edges, not flats),
so the blob ends at the text run and the fill leaves the tag's right half as
a ghost (measured 2026-07-22 on the 768x1024 cohort frame: blob bbox x
632..746 vs the tag ending ~758). The layout is measured and fixed -- the
text run is at the left of the locate box, the tag runs to the corner -- so
the footprint is the detector's match box extended RIGHT to the corner.
"""
if image is None or image.size == 0:
return None
from remove_ai_watermarks import image_io, region_eraser
image = image_io.to_bgr(image)
h, w = image.shape[:2]
if h < 32 or w < 64:
return None
loc = self.locate(image)
bx, by, bw, bh = loc.bbox
if force:
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
else:
if not self.detect(image).detected:
return None
_, box = self._tophat_best(image, loc)
if box is None:
return None
gx0, gy0, _gx1, gy1 = box
pad = max(4, int(0.15 * bh))
rx1 = max(0, bx + gx0 - pad)
ry1 = max(0, by + gy0 - pad)
rx2 = min(w, bx + bw) # the tag runs to the corner end of the box
ry2 = min(h, by + gy1 + 1 + pad)
if rx1 >= rx2 or ry1 >= ry2:
return None
d = dilate if dilate is not None else max(3, int(0.02 * bw))
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
def load_image_bgr(path: str | Path) -> NDArray[Any]:
"""Read an image as BGR ndarray (helper for scripts/tests)."""
from remove_ai_watermarks import image_io
img = image_io.imread(path)
if img is None:
raise FileNotFoundError(f"Failed to read image: {path}")
return img
+3
View File
@@ -449,6 +449,9 @@ _VISIBLE_MARK_PLATFORM = {
"qwen": "Alibaba Tongyi Qianwen (visible 千问AI生成 mark detected)",
"kling": "Kuaishou Kling (visible 可灵AI 3.0 mark detected)",
"samsung": "Samsung Galaxy AI (visible 'Contenuti generati dall'AI' mark detected)",
"runninghub": "RunningHub (visible RunningHub AI生成 mark detected)",
"baidu": "Baidu (visible 百度 AI生成 mark detected)",
"liblib": "LibLibAI (visible LibLibAI mark detected)",
}
+189
View File
@@ -0,0 +1,189 @@
"""LibLibAI visible watermark detector/localizer.
LibLibAI (哩布哩布AI, USCC 91110105MACJ6K1C8A) stamps its generations with a
white triangle logo + "LibLibAI" latin wordmark at **bottom-center** (not a
corner -- the locate box is horizontally centered). Detection matches the
bundled font-rendered "LibLibAI" silhouette (the triangle logo is NOT rendered
-- logos vary, the wordmark discriminates); removal is the shared **localize ->
fill** (the glyph blob covers logo + wordmark, both bright).
This module supplies only LibLibAI's tuned :class:`TextMarkConfig`
(``assets/liblib_alpha.png`` from ``scripts/render_vendor_silhouettes.py``,
never cut from an upload).
Measured on the vendor cohort (15 TC260 carriers, harvested 2026-07-22 by
``scripts/vendor_cohort_harvest.py``), NOT inherited from Doubao:
* The wordmark is ~0.10 of the frame WIDTH wide, centered horizontally, its
baseline ~0.94-0.95 of the height; consistent across 768..2240-px frames.
* The silhouette font is Arial, NOT the STHeiti the CJK marks use: the real
wordmark is a grotesque, and measured across 7 candidate fonts Arial lifts
the cohort positives from 0.31-0.47 to 0.42-0.73 while the full-corpus
false arm (latin UI text) drops to max 0.398. A 200x200 icon false-fired
at 0.444, so a per-mark size floor (``_MIN_SHORT_SIDE``) backs the gate.
* Gate 0.42 (tophat front-end): false arm max 0.398, cohort 0.43-0.59.
* STRICT ONLY (``provenance_ncc_factor`` 1.0): small cohort, the relaxed band
is unmeasured.
"""
# The module-level _alpha_template / _glyph_silhouette / _template_match_score below
# are thin test-facing shims (imported by tests/), so pyright's src-only pass sees them
# as unused; the use is cross-module.
# pyright: reportUnusedFunction=false
from __future__ import annotations
from typing import TYPE_CHECKING, Any
from remove_ai_watermarks import _text_mark_engine
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
if TYPE_CHECKING:
from pathlib import Path
from numpy.typing import NDArray
# Locate geometry as a fraction of the image WIDTH (measured basis). The box is
# horizontally centered (corner="bc") and covers the logo + wordmark with NCC
# slack around the measured 0.10 width.
WM_WIDTH_FRAC = 0.20
WM_HEIGHT_FRAC = 0.09
MARGIN_BOTTOM_FRAC = 0.02
# Glyph appearance: white wordmark on a usually-darker background (white
# top-hat), same overlay class as Doubao -- inherited, harmless because the
# tophat front-end turns these gates into weights.
MAX_SATURATION = 55
LOGO_MIN_LUMA = 150
TOPHAT_DELTA = 12
DETECT_MIN_COVERAGE = 0.04 # unused by the tophat front-end (kept for config parity)
# Calibrated 2026-07-22 on the vendor cohort vs 286 hand-labelled clean frames
# (clean p99 0.315 / max 0.367) and re-measured after the font fix: the wordmark
# is set in an Arial-class grotesque, and the Arial silhouette lifts the cohort
# positives to 0.43-0.59 while the full-corpus false arm (latin UI text bands,
# website screenshots) drops to max 0.398 -- generic latin text matches the
# wrong font less, which is exactly where the discrimination comes from. Gate
# 0.42 keeps all 8 marked cohort frames with a 0.022 margin over the false arm.
DETECT_NCC_THRESHOLD = 0.42
# Detection-silhouette geometry (fraction of the frame width): the wordmark,
# measured 0.10 wide with aspect 0.26.
_ALPHA_WIDTH_FRAC = 0.10
_ALPHA_HEIGHT_FRAC = 0.026
# Tight ladder: the NCC comb is sharp in size (see runninghub_engine).
_LADDER = (0.9, 1.0, 1.1)
_CONFIG = TextMarkConfig(
name="LibLibAI",
asset_name="liblib_alpha.png",
corner="bc",
margin_floor=4,
width_frac=WM_WIDTH_FRAC,
height_frac=WM_HEIGHT_FRAC,
margin_x_frac=0.0, # unused for corner="bc" (horizontally centered)
margin_bottom_frac=MARGIN_BOTTOM_FRAC,
max_saturation=MAX_SATURATION,
logo_min_luma=LOGO_MIN_LUMA,
tophat_delta=TOPHAT_DELTA,
morph_open_size=5,
detect_min_coverage=DETECT_MIN_COVERAGE,
detect_ncc_threshold=DETECT_NCC_THRESHOLD,
detect_frontend="tophat",
scale_basis="width",
ladder=_LADDER,
alpha_width_frac=_ALPHA_WIDTH_FRAC,
alpha_height_frac=_ALPHA_HEIGHT_FRAC,
min_gw=8,
# STRICT ONLY: small cohort, the relaxed band is unmeasured.
provenance_ncc_factor=1.0,
)
LibLibDetection = TextMarkDetection
def _alpha_template() -> NDArray[Any] | None:
"""The bundled LibLibAI alpha template (float [0,1]), or None."""
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
def _glyph_silhouette() -> NDArray[Any] | None:
"""Binary "LibLibAI" silhouette (255 = glyph) from the alpha map, or None."""
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
"""TM_CCOEFF_NORMED of the LibLibAI glyph silhouette against ``box_mask``."""
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
class LibLibEngine(TextMarkEngine):
"""Detect/localize the visible LibLibAI wordmark (bottom-center; localize -> fill)."""
# Per-mark size floor: the wordmark template is 0.10 of the frame width, so
# below ~480px short side it degrades under ~48px -- the one full-corpus
# false fire with the final Arial template was a 200x200 icon (0.444, above
# the gate, on a 20px template; measured 2026-07-22). The smallest true
# carrier in the cohort is 768px.
_MIN_SHORT_SIDE = 480
def __init__(self) -> None:
super().__init__(_CONFIG)
def detect(self, image: NDArray[Any] | None, *, provenance: bool = False) -> TextMarkDetection:
if image is None or not image.size or min(image.shape[:2]) < self._MIN_SHORT_SIDE:
return TextMarkDetection()
return super().detect(image, provenance=provenance)
def footprint_mask(
self, image: NDArray[Any] | None, *, force: bool = False, dilate: int | None = None
) -> NDArray[Any] | None:
"""Full-frame mask of the logo + wordmark, bounded by the detector's match box.
The base class's blob-bbox footprint is wrong in both directions here: the
blob bleeds UP into bright background structure (on the 768x1024 cohort
frame it reached y 931 and the fill ate the shirt's own print) and it does
not own the triangle logo anyway. The match box bounds the wordmark exactly
(that is what the NCC localized); the logo sits its own height to the LEFT
of the text (measured on the cohort zoom: logo ~1.0x the glyph height, gap
~0.3x), so the footprint is the match box extended left by ~1.3 heights.
"""
if image is None or image.size == 0:
return None
from remove_ai_watermarks import image_io, region_eraser
image = image_io.to_bgr(image)
h, w = image.shape[:2]
if h < 32 or w < 64:
return None
loc = self.locate(image)
bx, by, bw, bh = loc.bbox
if force:
rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh)
else:
if not self.detect(image).detected:
return None
_, box = self._tophat_best(image, loc)
if box is None:
return None
gx0, gy0, gx1, gy1 = box
gh = gy1 - gy0 + 1
pad = max(3, int(0.25 * gh))
rx1 = max(0, bx + gx0 - int(1.3 * gh)) # the triangle logo, left of the text
ry1 = max(0, by + gy0 - pad)
rx2 = min(w, bx + gx1 + 1 + pad)
ry2 = min(h, by + gy1 + 1 + pad)
if rx1 >= rx2 or ry1 >= ry2:
return None
d = dilate if dilate is not None else max(3, int(0.02 * bw))
return region_eraser.boxes_to_mask((h, w), [(rx1, ry1, rx2 - rx1, ry2 - ry1)], dilate=d)
def load_image_bgr(path: str | Path) -> NDArray[Any]:
"""Read an image as BGR ndarray (helper for scripts/tests)."""
from remove_ai_watermarks import image_io
img = image_io.imread(path)
if img is None:
raise FileNotFoundError(f"Failed to read image: {path}")
return img
@@ -0,0 +1,180 @@
"""RunningHub visible watermark detector/localizer.
RunningHub (a hosted ComfyUI platform, USCC 91340100MAEB4N8H76) stamps its
generations with a faint light-gray "RunningHub AI生成" text mark in the
**top-left** corner -- the China TC260 explicit AIGC label, but placed top-left
(unlike the GB 45438-2025 house style bottom-right of Doubao/Qwen/Kling) and
rendered in a mid-gray that the white top-hat front-end suppresses to clean-arm
levels.
Detection therefore uses the ``gray`` front-end (raw-grayscale silhouette NCC,
see ``TextMarkConfig.detect_frontend``); removal is the shared **localize ->
fill** (the detector's best-match box feeds :meth:`footprint_mask` ->
``region_eraser``). This module supplies only RunningHub's tuned
:class:`TextMarkConfig` (``assets/runninghub_alpha.png`` -- a font-rendered
synthetic silhouette from ``scripts/render_vendor_silhouettes.py``, never cut
from an upload).
EVERY tuned number below was measured on the vendor cohort (73 TC260 carriers
whose producer USCC names the entity, harvested 2026-07-22 by
``scripts/vendor_cohort_harvest.py``), NOT inherited from Doubao:
* Only ~4 of the 73 cohort frames carry a visible mark (the rest are
metadata-only TC260 carriers -- the platform labels frames it does not
stamp), so recall of visible marks is 4/4 but the cohort fire rate is not
a recall estimate. Positions/geometry are consistent across the positives.
* The mark's width is ~0.32 of the frame WIDTH (0.319 measured on 832/1080/
1536-wide frames) at ~0.008/0.006 x/y margins; the locate box below covers
it with NCC slack.
* ``alpha_height_frac`` comes from the silhouette aspect (0.128) at the
measured width (0.27 * 1.25 rung ~= 0.3375 >= 0.32), per the standing rule
that it is measured, not inherited.
* STRICT ONLY (``provenance_ncc_factor`` 1.0): raw gray NCC is
contrast-DEPENDENT and the sub-gate band of a corner-anchored gray match is
unmeasured beyond the clean arm, so no provenance relaxation exists.
* Gate 0.34: on 283 hand-labelled clean frames (cohort-contamination-guarded)
corner-anchored gray NCC p99 is 0.264 / max 0.304, while the 4 positives
score 0.38-0.54. 0.34 sits above the clean max with a small margin; the
positives are few, so the margin is deliberately thin on the recall side.
"""
# The module-level _alpha_template / _glyph_silhouette / _template_match_score below
# are thin test-facing shims (imported by tests/), so pyright's src-only pass sees them
# as unused; the use is cross-module.
# pyright: reportUnusedFunction=false
from __future__ import annotations
import logging
from typing import TYPE_CHECKING, Any
from remove_ai_watermarks import _text_mark_engine
from remove_ai_watermarks._text_mark_engine import TextMarkConfig, TextMarkDetection, TextMarkEngine
logger = logging.getLogger(__name__)
if TYPE_CHECKING:
from pathlib import Path
from numpy.typing import NDArray
# Locate geometry as a fraction of the image WIDTH (the measured basis: every
# positive is portrait, where width == short side). The mark hugs the top-left
# corner (~0.008 of width off the left edge, ~0.006 of height off the top).
WM_WIDTH_FRAC = 0.45
WM_HEIGHT_FRAC = 0.10
MARGIN_LEFT_FRAC = 0.002
MARGIN_TOP_FRAC = 0.002
# Glyph appearance fields are unused by the gray front-end (it never binarizes)
# and kept only for config parity with the other text marks.
MAX_SATURATION = 55
LOGO_MIN_LUMA = 150
TOPHAT_DELTA = 12
DETECT_MIN_COVERAGE = 0.04 # unused by the gray front-end (kept for config parity)
# Calibrated 2026-07-22 on the vendor cohort vs 283 hand-labelled clean frames:
# corner-anchored gray NCC, clean p99 0.264 / max 0.304; positives 0.38-0.54.
DETECT_NCC_THRESHOLD = 0.34
# Detection-silhouette geometry (fraction of the image width), measured on the
# positives: mark width is ~0.320 of width on all three frame sizes (266px at 832,
# 345px at 1080, 491px at 1536), and the NCC is razor-sharp in size (0.537 on-size,
# 0.223 at +5.6% -- the same comb behaviour Qwen measured), so the nominal sits
# exactly on the measured size with a TIGHT ladder around it, not the shared 3 rungs
# (whose nearest rung landed 5.6% off and collapsed the match to 0.22).
_ALPHA_WIDTH_FRAC = 0.32
_ALPHA_HEIGHT_FRAC = 0.04
_LADDER = (0.95, 1.0, 1.05)
_CONFIG = TextMarkConfig(
name="RunningHub",
asset_name="runninghub_alpha.png",
corner="tl",
margin_floor=4,
width_frac=WM_WIDTH_FRAC,
height_frac=WM_HEIGHT_FRAC,
margin_x_frac=MARGIN_LEFT_FRAC,
margin_bottom_frac=MARGIN_TOP_FRAC, # top margin for corner="tl"
max_saturation=MAX_SATURATION,
logo_min_luma=LOGO_MIN_LUMA,
tophat_delta=TOPHAT_DELTA,
morph_open_size=5,
detect_min_coverage=DETECT_MIN_COVERAGE,
detect_ncc_threshold=DETECT_NCC_THRESHOLD,
detect_frontend="gray",
scale_basis="width", # measured: mark width tracks the frame width (0.32)
ladder=_LADDER,
alpha_width_frac=_ALPHA_WIDTH_FRAC,
alpha_height_frac=_ALPHA_HEIGHT_FRAC,
min_gw=8,
# STRICT ONLY: contrast-dependent gray NCC; the relaxed band is unmeasured.
provenance_ncc_factor=1.0,
)
RunningHubDetection = TextMarkDetection
def _alpha_template() -> NDArray[Any] | None:
"""The bundled RunningHub alpha template (float [0,1]), or None."""
return _text_mark_engine.load_alpha_template(_CONFIG.asset_name)
def _glyph_silhouette() -> NDArray[Any] | None:
"""Binary "RunningHub AI生成" silhouette (255 = glyph) from the alpha map, or None."""
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
"""TM_CCOEFF_NORMED of the RunningHub glyph silhouette against ``box_mask``."""
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
class RunningHubEngine(TextMarkEngine):
"""Detect/localize the visible RunningHub "RunningHub AI生成" mark (top-left; localize -> fill)."""
def __init__(self) -> None:
super().__init__(_CONFIG)
# Anchor window for the match position, as a fraction of the FRAME: the true
# mark hugs the corner (measured x 0.008-0.014, y 0.005-0.007 of the frame on
# every cohort positive), while the full-corpus false fires (hair, shelves,
# window frames, CJK banners -- 37 of 42009 outside-cohort frames at the 0.34
# gate, 2026-07-22 sweep) match off-anchor at x 0.013-0.150 / y 0.009-0.045.
# No NCC gate separates them (false max 0.384 vs two positives at 0.381), but
# position does: every false fire sits outside this window, every positive
# inside. Contrast-dependent raw-gray NCC keys on "some text-like structure
# anywhere in the box"; the anchor is what makes it about THIS mark.
_ANCHOR_MAX_X = 0.025
_ANCHOR_MAX_Y = 0.015
def detect(self, image: NDArray[Any], *, provenance: bool = False) -> TextMarkDetection:
det = super().detect(image, provenance=provenance)
if not det.detected:
return det
loc = self.locate(image)
_, box = self._gray_best(image, loc)
if box is None:
det.detected = False
return det
h, w = image.shape[:2]
ax = (loc.x + box[0]) / w
ay = (loc.y + box[1]) / h
if ax > self._ANCHOR_MAX_X or ay > self._ANCHOR_MAX_Y:
logger.debug(
"RunningHub detect: score %.3f but match off-anchor (x=%.3f y=%.3f); demoting.",
det.confidence,
ax,
ay,
)
det.detected = False
return det
def load_image_bgr(path: str | Path) -> NDArray[Any]:
"""Read an image as BGR ndarray (helper for scripts/tests)."""
from remove_ai_watermarks import image_io
img = image_io.imread(path)
if img is None:
raise FileNotFoundError(f"Failed to read image: {path}")
return img
+22 -1
View File
@@ -24,6 +24,9 @@ Entries:
- ``kling`` -- Kuaishou Kling "可灵AI 3.0" text strip, bottom-right.
- ``samsung`` -- Samsung Galaxy AI "Contenuti generati dall'AI" strip, bottom-left.
- ``jimeng_pill`` -- Jimeng-basic "AI生成" pill, top-left (capture-less).
- ``runninghub`` -- RunningHub "RunningHub AI生成" text, top-left (gray front-end).
- ``baidu`` -- Baidu "百度 AI生成" text + white tag, bottom-right.
- ``liblib`` -- LibLibAI "LibLibAI" wordmark, bottom-center.
"""
from __future__ import annotations
@@ -88,6 +91,9 @@ _PRODUCT_OF: dict[str, str] = {
"qwen": "qwen",
"kling": "kling",
"samsung": "samsung",
"runninghub": "runninghub",
"baidu": "baidu",
"liblib": "liblib",
}
@@ -373,6 +379,18 @@ def _engine(key: str) -> Any:
from remove_ai_watermarks.pill_engine import PillEngine
_engines[key] = PillEngine()
elif key == "runninghub":
from remove_ai_watermarks.runninghub_engine import RunningHubEngine
_engines[key] = RunningHubEngine()
elif key == "baidu":
from remove_ai_watermarks.baidu_engine import BaiduEngine
_engines[key] = BaiduEngine()
elif key == "liblib":
from remove_ai_watermarks.liblib_engine import LibLibEngine
_engines[key] = LibLibEngine()
else: # pragma: no cover - guarded by the registry keys
raise KeyError(key)
return _engines[key]
@@ -517,6 +535,9 @@ _REGISTRY: tuple[KnownMark, ...] = (
_text_mark("qwen", "Qwen 千问AI生成 text", "bottom-right"),
_text_mark("kling", "Kling 可灵AI 3.0 text", "bottom-right"),
_text_mark("samsung", "Samsung Galaxy AI text", "bottom-left"),
_text_mark("runninghub", "RunningHub AI生成 text", "top-left"),
_text_mark("baidu", "Baidu 百度 AI生成 text", "bottom-right"),
_text_mark("liblib", "LibLibAI wordmark", "bottom-center"),
KnownMark("jimeng_pill", "Jimeng AI生成 pill", "top-left", True, _pill_detect, _pill_mask, _pill_features),
)
@@ -597,7 +618,7 @@ def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bo
Doubao detection; a Qwen image likewise (another vendor's bottom-right mark naming
its own product), so a confident Qwen detection suppresses the pill the same way.
No confirmation at all -> never remove (blocks false fires on non-Jimeng content)."""
if "doubao" in keys or "qwen" in keys or "kling" in keys:
if "doubao" in keys or "qwen" in keys or "kling" in keys or "runninghub" in keys or "baidu" in keys:
return False
if "jimeng" in keys:
return True
+133
View File
@@ -0,0 +1,133 @@
"""Tests for the Baidu ("百度 AI生成") visible-watermark engine.
Every tuned constant in ``baidu_engine`` was measured on the 16-frame vendor
cohort (2026-07-22); these tests pin the load-bearing ones: detection keys on
the 百度 text run ONLY (the text+pill template was a measured bright-blob
magnet), the load-bearing Doubao rival margin, the strict-only gate, and the
corner-extended footprint (the tag's flat white interior gives no top-hat
response, so a blob-bbox mask leaves the tag as a ghost).
"""
from __future__ import annotations
import cv2
import numpy as np
from remove_ai_watermarks import watermark_registry as registry
from remove_ai_watermarks.baidu_engine import (
_ALPHA_HEIGHT_FRAC,
_ALPHA_WIDTH_FRAC,
BaiduEngine,
_alpha_template,
)
_TEXT_FRAC = 0.090 # measured 百度 text-run width, fraction of the short side
_TEXT_RIGHT = 0.099 # measured right margin of the text run (the tag is right of it)
_TAG_FRAC = 0.075 # the white tag's width, approx (text-right to corner)
def _compose(w: int, h: int, bg: float = 100.0):
"""Composite the 百度 text run + a solid white tag at the measured layout."""
img = np.full((h, w, 3), bg, np.float32)
at = _alpha_template()
short = min(w, h)
gw = int(_TEXT_FRAC * short)
gh = max(4, int(_TEXT_FRAC * (_ALPHA_HEIGHT_FRAC / _ALPHA_WIDTH_FRAC) * short))
margin_b = int(0.006 * short)
ax = w - int(_TEXT_RIGHT * short) - gw
ay = h - margin_b - gh
amap = np.zeros((h, w), np.float32)
amap[ay : ay + gh, ax : ax + gw] = cv2.resize(at, (gw, gh))
# the white rounded tag between the text and the corner
tx0 = w - int(0.015 * short) - int(_TAG_FRAC * short)
amap[ay - gh // 8 : ay + gh + gh // 8, tx0 : w - int(0.015 * short)] = 1.0
a3 = amap[:, :, None]
wm = (a3 * 255.0 + (1 - a3) * img).clip(0, 255).astype(np.uint8)
return wm, (ax, ay, gw, gh, tx0)
class TestLocate:
def test_box_anchored_bottom_right(self):
eng = BaiduEngine()
img = np.zeros((2048, 2048, 3), np.uint8)
loc = eng.locate(img)
assert 2048 - (loc.x + loc.w) < 40
assert 2048 - (loc.y + loc.h) < 40
def test_box_scales_with_short_side(self):
eng = BaiduEngine()
landscape = eng.locate(np.zeros((640, 1280, 3), np.uint8))
wider = eng.locate(np.zeros((640, 2560, 3), np.uint8))
assert wider.w == landscape.w
class TestConfig:
def test_tophat_frontend(self):
assert BaiduEngine().config.detect_frontend == "tophat"
def test_doubao_rival_margin(self):
# 百度 vs 豆包 share a glyph and a corner: the candidate fires on 45.8% of
# Doubao-marked frames at the gate, and the 0.10 margin suppresses ALL of
# it at zero genuine-detection cost (crossfire, 2026-07-22).
assert "doubao_alpha.png" in BaiduEngine().config.rivals
def test_strict_only_no_provenance_relaxation(self):
assert BaiduEngine().config.provenance_ncc_factor == 1.0
def test_gate_above_clean_arm_max(self):
# Clean arm scored p99 0.314 / max 0.352 on 278 hand-labelled frames;
# the 741-frame eval set surfaced cross-fires up to 0.426 (a 抖音
# AI创作 mark no rival can suppress), and the full-corpus sweep put the
# outside-cohort false arm at 0.47 max vs true carriers at 0.50-0.66,
# so the gate sits at 0.48.
assert BaiduEngine().config.detect_ncc_threshold >= 0.48
def test_qwen_is_a_rival(self):
# 百度 vs 千问 are near-identical after binarization: 12 of 14 full-corpus
# cross-fires at the 0.37 gate were Qwen marks (Qwen's template beats
# Baidu's there by 0.17-0.35, so the margin suppresses them).
assert "qwen_alpha.png" in BaiduEngine().config.rivals
def test_registry_row(self):
mark = registry.get_mark("baidu")
assert mark.location == "bottom-right"
assert mark.in_auto
class TestDetectAndMask:
def test_detects_composed_mark(self):
eng = BaiduEngine()
wm, _ = _compose(1024, 1024)
det = eng.detect(wm)
assert det.detected, f"composed mark missed (conf={det.confidence:.3f})"
def test_clean_frame_stays_quiet(self):
eng = BaiduEngine()
img = np.full((1024, 1024, 3), 100, np.uint8)
assert not eng.detect(img).detected
def test_mask_extends_to_the_corner_tag(self):
"""Regression (2026-07-22): the tag's flat white interior gives no top-hat
response, so a blob-bbox mask ended at the text run and the fill left the
tag as a ghost. The footprint must extend right to the corner."""
eng = BaiduEngine()
wm, (ax, _ay, gw, _gh, tx0) = _compose(1024, 1024)
mask = eng.footprint_mask(wm)
assert mask is not None
_ys, xs = np.where(mask > 0)
assert xs.min() <= ax + int(0.1 * gw) # covers the text run's left edge
assert xs.max() >= tx0 + 10 # covers the white tag right of the text
def test_no_mask_on_clean_frame(self):
eng = BaiduEngine()
img = np.full((1024, 1024, 3), 100, np.uint8)
assert eng.footprint_mask(img) is None
class TestPillInteraction:
def test_confident_baidu_detection_suppresses_the_jimeng_pill(self):
# A Baidu image is TC260 too but is not Jimeng-basic: like Doubao/Qwen/
# Kling, a confident Baidu detection must veto the pill (``_keep_pill``).
from remove_ai_watermarks.watermark_registry import _keep_pill
assert not _keep_pill({"baidu"}, provenance=frozenset({"jimeng"}), footprint_flat=1.0)
+114
View File
@@ -0,0 +1,114 @@
"""Tests for the LibLibAI ("LibLibAI" wordmark) visible-watermark engine.
Every tuned constant in ``liblib_engine`` was measured on the 15-frame vendor
cohort (2026-07-22); these tests pin the load-bearing ones: the bottom-CENTER
anchor, the strict-only gate, and the match-box footprint (the blob bbox both
bled into background structure and did not own the triangle logo).
"""
from __future__ import annotations
import cv2
import numpy as np
import pytest
from remove_ai_watermarks import watermark_registry as registry
from remove_ai_watermarks.liblib_engine import (
_ALPHA_HEIGHT_FRAC,
_ALPHA_WIDTH_FRAC,
LibLibEngine,
_alpha_template,
)
_MARK_FRAC = 0.10 # measured wordmark width, fraction of the frame WIDTH
def _compose(w: int, h: int, bg: float = 100.0):
"""Composite a triangle logo + the LibLibAI wordmark, bottom-center."""
img = np.full((h, w, 3), bg, np.float32)
at = _alpha_template()
gw = int(_MARK_FRAC * w)
gh = max(4, int(_MARK_FRAC * (_ALPHA_HEIGHT_FRAC / _ALPHA_WIDTH_FRAC) * w))
ax = (w - gw) // 2
ay = int(0.94 * h) - gh
amap = np.zeros((h, w), np.float32)
amap[ay : ay + gh, ax : ax + gw] = cv2.resize(at, (gw, gh))
# the triangle logo, its own height to the LEFT of the wordmark
lx1 = ax - int(0.3 * gh)
lx0 = lx1 - gh
cv2.fillPoly(amap, [np.array([(lx0, ay + gh), (lx1, ay + gh), (lx1, ay)])], 1.0)
a3 = amap[:, :, None]
wm = (a3 * 255.0 + (1 - a3) * img).clip(0, 255).astype(np.uint8)
return wm, (ax, ay, gw, gh, lx0)
class TestLocate:
def test_box_horizontally_centered(self):
eng = LibLibEngine()
img = np.zeros((2048, 1536, 3), np.uint8)
loc = eng.locate(img)
assert (1536 - loc.w) // 2 == pytest.approx(loc.x, abs=2) # corner="bc"
assert 2048 - (loc.y + loc.h) > 0 # bottom-anchored
def test_box_scales_with_width(self):
eng = LibLibEngine()
narrow = eng.locate(np.zeros((2048, 1024, 3), np.uint8))
wide = eng.locate(np.zeros((2048, 2048, 3), np.uint8))
assert wide.w == pytest.approx(narrow.w * 2, rel=0.05)
class TestConfig:
def test_tophat_frontend(self):
assert LibLibEngine().config.detect_frontend == "tophat"
def test_strict_only_no_provenance_relaxation(self):
assert LibLibEngine().config.provenance_ncc_factor == 1.0
def test_gate_above_clean_arm_max(self):
# With the Arial silhouette the full-corpus false arm (latin UI text)
# tops out at 0.398 while the cohort sits at 0.43-0.59; gate 0.42.
assert LibLibEngine().config.detect_ncc_threshold >= 0.42
def test_small_image_size_floor(self):
# The one full-corpus false fire with the final template was a 200x200
# icon on a 20px template; the engine refuses small images outright.
eng = LibLibEngine()
assert not eng.detect(np.full((200, 200, 3), 100, np.uint8)).detected
wm, _ = _compose(200, 200)
assert not eng.detect(wm).detected # even a composed mark under the floor
def test_registry_row(self):
mark = registry.get_mark("liblib")
assert mark.location == "bottom-center"
assert mark.in_auto
class TestDetectAndMask:
def test_detects_composed_mark(self):
eng = LibLibEngine()
wm, _ = _compose(1792, 2400)
det = eng.detect(wm)
assert det.detected, f"composed mark missed (conf={det.confidence:.3f})"
def test_clean_frame_stays_quiet(self):
eng = LibLibEngine()
img = np.full((2400, 1792, 3), 100, np.uint8)
assert not eng.detect(img).detected
def test_mask_covers_logo_and_wordmark(self):
"""The footprint must cover the triangle logo LEFT of the wordmark while
staying bounded by the match box vertically (the blob bbox bled into
background structure and ate real content, 2026-07-22)."""
eng = LibLibEngine()
wm, (ax, ay, gw, gh, lx0) = _compose(1792, 2400)
mask = eng.footprint_mask(wm)
assert mask is not None
ys, xs = np.where(mask > 0)
assert xs.min() <= lx0 + gh // 2 # covers the logo
assert xs.max() >= ax + gw - int(0.05 * gw) # covers the wordmark's right edge
assert ys.min() >= ay - gh # does not bleed far above the mark
def test_no_mask_on_clean_frame(self):
eng = LibLibEngine()
img = np.full((2400, 1792, 3), 100, np.uint8)
assert eng.footprint_mask(img) is None
+135
View File
@@ -0,0 +1,135 @@
"""Tests for the RunningHub ("RunningHub AI生成") visible-watermark engine.
Every tuned constant in ``runninghub_engine`` was measured on the 73-frame
vendor cohort (2026-07-22, ``scripts/vendor_cohort_harvest.py`` +
``scripts/vendor_mark_calibrate.py``); these tests pin the load-bearing ones:
the top-left corner, the gray front-end, the exact-size tight ladder, the
strict-only gate, and the mask/coverage parity regression (the partial-blob
"Runni" miss).
"""
from __future__ import annotations
import cv2
import numpy as np
import pytest
from remove_ai_watermarks import watermark_registry as registry
from remove_ai_watermarks.runninghub_engine import (
_ALPHA_HEIGHT_FRAC,
_ALPHA_WIDTH_FRAC,
RunningHubEngine,
_alpha_template,
)
_MARK_FRAC = 0.32 # measured mark width, fraction of the frame WIDTH
def _compose(w: int, h: int, mode: float = _MARK_FRAC, bg: float = 100.0):
"""Composite the RunningHub silhouette at the measured size, top-left."""
img = np.full((h, w, 3), bg, np.float32)
at = _alpha_template()
gw = int(mode * w)
gh = max(4, int(mode * (_ALPHA_HEIGHT_FRAC / _ALPHA_WIDTH_FRAC) * w))
ax, ay = int(0.008 * w), int(0.006 * h)
amap = np.zeros((h, w), np.float32)
amap[ay : ay + gh, ax : ax + gw] = cv2.resize(at, (gw, gh))
a3 = amap[:, :, None]
wm = (a3 * 255.0 + (1 - a3) * img).clip(0, 255).astype(np.uint8)
return wm, (ax, ay, gw, gh)
class TestLocate:
def test_box_anchored_top_left(self):
eng = RunningHubEngine()
img = np.zeros((2048, 1536, 3), np.uint8)
loc = eng.locate(img)
assert loc.x < 40 # hugs the left edge
assert loc.y < 40 # hugs the top edge (corner="tl")
def test_box_scales_with_width(self):
# scale_basis="width" (measured: mark width is 0.32 of the frame width).
eng = RunningHubEngine()
narrow = eng.locate(np.zeros((2048, 1024, 3), np.uint8))
wide = eng.locate(np.zeros((2048, 2048, 3), np.uint8))
assert wide.w == pytest.approx(narrow.w * 2, rel=0.05)
class TestConfig:
def test_gray_frontend(self):
# The mark is a faint mid-gray the top-hat suppresses to clean-arm levels;
# the raw-grayscale front-end is what separates (measured 2026-07-22).
assert RunningHubEngine().config.detect_frontend == "gray"
def test_tight_ladder(self):
# The NCC comb is razor-sharp in size (0.537 on-size, 0.223 at +5.6%), so
# the nominal sits exactly on the measured 0.32 with +-5% rungs.
assert RunningHubEngine().config.ladder == (0.95, 1.0, 1.05)
assert RunningHubEngine().config.alpha_width_frac == pytest.approx(0.32)
def test_strict_only_no_provenance_relaxation(self):
assert RunningHubEngine().config.provenance_ncc_factor == 1.0
def test_gate_above_clean_arm_max(self):
# Clean arm scored p99 0.273 / max 0.295 on 286 hand-labelled frames.
assert RunningHubEngine().config.detect_ncc_threshold > 0.295
def test_registry_row(self):
mark = registry.get_mark("runninghub")
assert mark.location == "top-left"
assert mark.in_auto
class TestDetectAndMask:
def test_detects_composed_mark(self):
eng = RunningHubEngine()
wm, _ = _compose(1080, 1620)
det = eng.detect(wm)
assert det.detected, f"composed mark missed (conf={det.confidence:.3f})"
def test_clean_frame_stays_quiet(self):
eng = RunningHubEngine()
img = np.full((1620, 1080, 3), 100, np.uint8)
assert not eng.detect(img).detected
def test_mask_covers_the_whole_mark(self):
"""Regression (2026-07-22): the binary blob under-segments the faint head
glyphs, so a blob-bbox mask left "Runni" unremoved. The gray front-end's
mask must come from the detector's own match box and cover the mark."""
eng = RunningHubEngine()
wm, (ax, ay, gw, gh) = _compose(1080, 1620)
mask = eng.footprint_mask(wm)
assert mask is not None
ys, xs = np.where(mask > 0)
assert xs.min() <= ax + int(0.05 * gw) # covers the LEFT edge of the mark
assert xs.max() >= ax + gw - int(0.05 * gw)
assert ys.min() <= ay + gh // 2 <= ys.max()
def test_no_mask_on_clean_frame(self):
eng = RunningHubEngine()
img = np.full((1620, 1080, 3), 100, np.uint8)
assert eng.footprint_mask(img) is None
def test_anchor_window_rejects_off_corner_match(self):
"""The raw-gray front-end false-fires on text-like structure ANYWHERE in
the box (37/42009 outside-cohort frames in the 2026-07-22 sweep); the
anchor window is what keeps it about THIS mark. A composed mark placed
off the measured corner anchor must NOT be detected."""
eng = RunningHubEngine()
wm, _ = _compose(1080, 1620)
det = eng.detect(wm)
assert det.detected # on-anchor control
# the same mark shifted right/down, off the anchor window
shifted = np.full((1620, 1080, 3), 100, np.uint8)
region = wm[10:60, 12:360]
shifted[100 : 100 + region.shape[0], 200 : 200 + region.shape[1]] = region
assert not eng.detect(shifted).detected
class TestPillInteraction:
def test_confident_runninghub_detection_suppresses_the_jimeng_pill(self):
# A RunningHub frame names its own product; its detection must veto the
# Jimeng pill the same way Doubao/Qwen/Kling do (``_keep_pill``).
from remove_ai_watermarks.watermark_registry import _keep_pill
assert not _keep_pill({"runninghub"}, provenance=frozenset({"jimeng"}), footprint_flat=1.0)
+24 -2
View File
@@ -14,7 +14,18 @@ DOUBAO_SAMPLE = Path(__file__).resolve().parents[1] / "data" / "samples" / "doub
class TestCatalog:
def test_keys(self):
assert reg.mark_keys() == ["gemini", "doubao", "jimeng", "qwen", "kling", "samsung", "jimeng_pill"]
assert reg.mark_keys() == [
"gemini",
"doubao",
"jimeng",
"qwen",
"kling",
"samsung",
"runninghub",
"baidu",
"liblib",
"jimeng_pill",
]
def test_all_in_auto(self):
assert all(m.in_auto for m in reg.known_marks())
@@ -43,7 +54,18 @@ class TestScan:
def test_detect_marks_scans_all(self):
img = np.zeros((256, 256, 3), np.uint8)
keys = {d.key for d in reg.detect_marks(img)}
assert keys == {"gemini", "doubao", "jimeng", "qwen", "kling", "samsung", "jimeng_pill"}
assert keys == {
"gemini",
"doubao",
"jimeng",
"qwen",
"kling",
"samsung",
"runninghub",
"baidu",
"liblib",
"jimeng_pill",
}
def test_blank_image_no_auto_mark(self):
dets = reg.detect_marks(np.zeros((256, 256, 3), np.uint8), include_explicit=False)
Generated
+1 -1
View File
@@ -2759,7 +2759,7 @@ wheels = [
[[package]]
name = "remove-ai-watermarks"
version = "0.17.0"
version = "0.18.0"
source = { editable = "." }
dependencies = [
{ name = "c2pa-python" },