Remove assume_ai, add tophat front-end and rival margin, fix two CLI defects

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Victor Kuznetsov
2026-07-20 08:14:50 -07:00
co-authored by Claude Opus 4.8
parent a8f3536d3e
commit cfefd9d819
26 changed files with 1715 additions and 257 deletions
+6 -6
View File
File diff suppressed because one or more lines are too long
+6 -6
View File
File diff suppressed because one or more lines are too long
+8 -8
View File
@@ -27,7 +27,7 @@ It does **not** target watermarks that protect someone else's paid or copyrighte
## Features
- **Visible watermark removal** — a registry of known marks in their usual places: the Gemini / Nano Banana sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-left, locale-specific). Each mark is **localized to a footprint mask, then filled**: the engine finds the mark, builds a binary mask over its footprint, and one shared, swappable fill inpaints that region. Choose the fill with `--backend`: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN, light, the memory-tight pick where LaMa will not fit), or `lama` (big-LaMa, best quality, heavier, auto-preferred when a learned backend is available); the default `auto` uses LaMa > MI-GAN > cv2, best available. The localizer is cheap CPU (cv2/numpy), so it runs anywhere; the heavier MI-GAN/LaMa fill is opt-in. Detection keys on each mark's own shape (NCC against a captured silhouette; the alpha captures rebuilt by `scripts/visible_alpha_solve.py` are used to detect and to shape the mask, not for pixel recovery). The visual detector needs no metadata, but a borderline (faint or moved) mark is only trusted with corroboration: `--sensitivity` (default `auto`) relaxes a mark's gate when local metadata confirms the vendor or a same-product sibling mark is found; `--sensitivity assume-ai` relaxes every mark on the assertion that the image is AI, recovering the moved or re-rendered marks the conservative gate skips on a metadata-stripped screenshot (`strict` never relaxes). Because asserting "this is AI" says nothing about *which* vendor made it, a mark relaxed on that assertion alone still has to clear a confidence floor, so a clean photo is left untouched. `visible --mark auto` finds and removes every detected mark in one pass. Fast, offline, no GPU. (For arbitrary logos/objects, see `erase`.)
- **Visible watermark removal** — a registry of known marks in their usual places: the Gemini / Nano Banana sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, and the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-left, locale-specific). Each mark is **localized to a footprint mask, then filled**: the engine finds the mark, builds a binary mask over its footprint, and one shared, swappable fill inpaints that region. Choose the fill with `--backend`: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN, light, the memory-tight pick where LaMa will not fit), or `lama` (big-LaMa, best quality, heavier, auto-preferred when a learned backend is available); the default `auto` uses LaMa > MI-GAN > cv2, best available. The localizer is cheap CPU (cv2/numpy), so it runs anywhere; the heavier MI-GAN/LaMa fill is opt-in. Detection keys on each mark's own shape (NCC against a captured silhouette; the alpha captures rebuilt by `scripts/visible_alpha_solve.py` are used to detect and to shape the mask, not for pixel recovery). The visual detector needs no metadata, but a borderline (faint or moved) mark is only trusted with corroboration: `--sensitivity` (default `auto`) relaxes a mark's gate when local metadata confirms the vendor or a same-product sibling mark is found; `strict` never relaxes. There is deliberately no "assume this is AI" mode: asserting that an image is AI says nothing about *which* vendor made it or *where* the mark is, which is exactly what a gate bypass needs. If you can SEE a mark the detector missed, point at it with `erase --region x,y,w,h`, or force a known text mark by name with `--mark <name> --no-detect` -- both act on what you actually see instead of relaxing every detector at once. `visible --mark auto` finds and removes every detected mark in one pass. Fast, offline, no GPU. (For arbitrary logos/objects, see `erase`.)
- **Universal region eraser (`erase`)** — remove any logo / watermark / object inside boxes you specify, regardless of position or color. Default cv2 inpainting (CPU, instant); optional big-LaMa via onnxruntime (`lama` extra) for higher quality
- **Invisible watermark removal** — SynthID, StableSignature, TreeRing via diffusion-based regeneration (needs a local GPU, or run it with no setup on [raiw.cc](https://raiw.cc))
- **AI metadata stripping** — EXIF, PNG text chunks, C2PA provenance manifests (PNG / JPEG / AVIF / HEIF / JPEG-XL, **MP4 / MOV / M4V / M4A** at the container level, and **WebM / MP3 / WAV / FLAC / OGG** losslessly via ffmpeg), XMP DigitalSourceType
@@ -335,17 +335,17 @@ remove-ai-watermarks identify image.png
# --mark gemini / doubao / jimeng / samsung. Removal localizes each mark to a
# footprint mask and inpaints it with a shared fill; --backend auto|cv2|migan|lama
# (default auto) picks the fill (auto = LaMa > MI-GAN > cv2, best available).
# --sensitivity auto|strict|assume-ai (default auto) sets how hard a borderline
# mark is trusted; use assume-ai on a metadata-stripped screenshot to recover a
# moved or faint mark the conservative gate would skip.
# --sensitivity auto|strict (default auto) sets how hard a borderline mark is
# trusted; there is no "assume this is AI" mode -- to act on a mark you can see
# but the detector missed, point at it with `erase --region` (below).
# If no known visible mark is found, it writes no output and exits 2 (not 0),
# pointing you to `all` (for an invisible/metadata mark) or `erase` (for an
# arbitrary logo) instead of handing back the unchanged image.
remove-ai-watermarks visible image.png -o clean.png
# Metadata-stripped screenshot: assert it is AI to relax detection and recover
# a moved/faint mark (a wrong guess just fills a small corner near-losslessly).
remove-ai-watermarks visible screenshot.png --sensitivity assume-ai -o clean.png
# Metadata-stripped screenshot where you can SEE the mark but detection missed it:
# point at it. This executes what you see instead of relaxing every detector at once.
remove-ai-watermarks erase screenshot.png --region 812,1180,190,44 -o clean.png
# Erase arbitrary region(s) — universal, any logo/watermark/object, any position.
# Default cv2 inpainting (CPU). --backend lama uses big-LaMa (extra 'lama').
@@ -409,7 +409,7 @@ clean, removed = raiw.remove_visible(cv2.imread("in.png"), backend="cv2")
# Metadata-stripped screenshot you know is AI-generated: relax detection to recover a
# moved or faint mark (a wrong guess just fills a small corner near-losslessly).
raiw.remove_visible("screenshot.png", "clean.png", sensitivity="assume_ai")
raiw.remove_visible("screenshot.png", "clean.png", sensitivity="strict")
# What the file's metadata confirms (drives the default `auto` sensitivity):
raiw.visible_provenance("in.png") # e.g. frozenset({'gemini'})
+114 -9
View File
@@ -65,16 +65,121 @@ module.
**Head-to-head validation (v0.12.1 reverse-alpha vs the current localize -> fill):** run over the full labelled visible-mark set, with the cv2 / MI-GAN / LaMa fills each compared against the old reverse-alpha. **doubao and jimeng are identical** across every backend -- 100% coverage and 100% clearance either way. **gemini** strict coverage is a few points below reverse-alpha's (the deliberate false-positive tightening), but the metadata-stripped faint ones are now mostly recovered by the DEFAULT white-core rescue in the FP gate (`gemini_engine`: a bright near-WHITE core distinguishes a real faint sparkle from a colored bright corner -- ~14/20 recovered at ~1.25% clean false-fire; a learned classifier on the same features measured worse, 2026-07 tier-1), the residual under `assume_ai`; clearance is equal (~98% both), and neither version touches pixels outside the mark box (outside-box PSNR ~99). **Clearance is fill-independent** -- cv2, MI-GAN and LaMa all strip the mark's shape equally, so the re-detect metric does not separate them; the difference is purely the *visual fill quality* on the recovered region, and it is background-dependent. reverse-alpha recovered textured and especially regular/structured backgrounds (a lattice, a grid) more cleanly than any inpaint; **LaMa closes most of that gap** (the best learned backend), **MI-GAN can ghost or hallucinate structure**, and **cv2 smears** (the last-resort floor). This is why `auto` resolves `LaMa > MI-GAN > cv2` (`preferred_inpaint_backend`) and warns once on the cv2 fallback; on flat backgrounds every backend is clean.
**Assumed-trust confidence floor (`_ASSUMED_CONF_FLOOR` / `assumed_floor_ok`, 2026-07-16):** relaxing a mark bypasses the engine's false-positive gate ENTIRELY, leaving only the bare detector threshold (gemini: fused confidence >= 0.35). That is defensible when metadata names the vendor (`confirmed`) and indefensible on a bare "assume this is AI" (`assumed`), because the assertion carries no vendor information. Measured on 256 genuine camera captures (Make/Model/exposure/aperture present, no AI token -- a Gemini sparkle cannot be there) vs 697 Google-C2PA positives with the metadata used only as a label:
**`assume_ai` removed (2026-07-19).** It relaxed EVERY mark's false-positive gate on the caller's bare assertion that an image is AI. That assertion says nothing about WHICH vendor or WHERE the mark is, which is exactly what a gate bypass is contracted to require -- before it carried a confidence floor it filled a phantom sparkle on 59.8% of genuine camera photos, and even with the floor it was a statistical gamble rather than an instruction. It also had no place in the product's model: detector finds a mark -> remove it; detector finds nothing -> leave the image alone; the USER sees a mark and says so -> act on that.
| bypassed threshold | recall | false fire on clean photos |
|---|---|---|
| 0.35 (the bare gate) | 82.6% | **59.8%** |
| 0.45 | 66.6% | 12.5% |
| **0.50 (chosen)** | 59.4% | **0.0%** |
| strict gate | 56.4% | 0.0% |
Removing it collapsed the trust ladder from three levels to two (`strict` / `confirmed`) and took `_ASSUMED_CONF_FLOOR`, `assumed_floor_ok` and the `assumed` level with it. `_keep_pill` lost its `sensitivity` parameter (its assume-arm is gone; the metadata arm and its flatness guard are unchanged). Verified on the 240-image unbiased recall sample: doubao 92%/99%, gemini 96%/80%, jimeng 71%/71%, pill 50%/60% -- identical before and after, so nothing on the default path moved.
**The replacement advice is PER MARK, because the forced paths are not equally reliable** (measured 2026-07-19):
| path | reliability |
|---|---|
| `erase --region x,y,w,h` | sound by construction -- the user supplies the coordinates |
| `--mark <text-mark> --no-detect` | reasonable: the forced mask is the real glyph blob, non-empty on 13/13 missed doubao marks |
| `--mark gemini --no-detect` | **NOT recommended** -- falls back to a fixed default sparkle slot, which covered the true sparkle on only **31% of 97** sparkles the strict gate missed (median offset 63px up-and-left). The other 69% fill a clean corner AND report a removal that did not happen. |
`cli._no_visible_mark_exit` therefore recommends `erase --region` first and a named text mark second, and never suggests forcing gemini. It previously recommended `--sensitivity assume-ai`, i.e. the product's own hint contradicted its model.
**Migration is LOUD, not silent.** `Sensitivity` is a `Literal` and unenforced at runtime, so a 0.15 caller passing `sensitivity="assume_ai"` would otherwise get `auto` behaviour in silence -- a quiet semantic change on exactly the release where they need telling. `validate_sensitivity` (called by `api.remove_visible` and by `Context.__post_init__`) raises a `ValueError` naming the replacement. Regression: `tests/test_watermark_registry.py::TestNoBlanketRelaxation`.
**Continuous top-hat detection front-end (`detect_frontend` / `tophat_response`, 2026-07-18).** `extract_mask` thresholds the white top-hat into a 0/255 glyph blob and correlates a binary silhouette against it. That is fine for a mark stamped bold and opaque, and destructive for a faint one: a thin translucent overlay shatters into specks under the threshold, and no template can match a blob that is not there (千问 measured 0.170 mean NCC, **0%** over its gate, against doubao's 0.723 / 82% -- same pipeline, each with its own template). The `tophat` front-end never binarizes: the saturation and absolute-luma gates become WEIGHTS instead of hard cuts, so a faint stroke contributes in proportion to its strength, and the response is max-normalized, which makes the score contrast-invariant.
Doubao is switched to it; jimeng and samsung stay `binary` until measured, because a front-end change must be measured per mark before it ships. Corpus effect on the 240-image unbiased recall sample:
| mark | recall before | recall after | precision before | precision after |
|---|---|---|---|---|
| doubao | 89% | **92%** | 99% | **99%** |
| jimeng / gemini / pill | unchanged | unchanged | unchanged | unchanged |
**The gate is FRONT-END SPECIFIC and must be re-calibrated, not ported.** The continuous response scores higher overall (mean 0.809 vs 0.723 on the same 90 positives), so the binary-era 0.40 left the provenance-relaxed gate (x0.7) far too low: at 0.40 the arm ran 96% recall / 91% precision (8 false fires), at 0.50 it runs 92% / 99% (1 false fire). 0.50 was chosen because it beats the binary front-end on recall at IDENTICAL precision -- a front-end that only trades one for the other would not have been worth shipping. A first pass at 0.40 also silently depressed the PILL (recall 50% -> 33%), because `_keep_pill` suppresses the pill whenever doubao fires; a coupling worth remembering when tuning any bottom-right mark.
**What this does NOT solve: vendor ATTRIBUTION for the shared-suffix marks.** With the continuous front-end 千问 becomes separable from clean corners (AUC 0.92) but NOT from Doubao (**AUC 0.41-0.59, i.e. a coin flip**), because "千问AI生成" and "豆包AI生成" share the `AI生成` tail -- three of five glyphs, same face, same corner. So the front-end removes the *detection* blocker and exposes an *attribution* one. Since removal is identical for either (localize the glyph blob -> fill), the natural next design is a GENERIC "CJK AI-generation text mark" detector covering 千问/百度/星绘/小云雀/TRAE and any future GB 45438-2025-compliant vendor in one template, with per-vendor attribution treated as optional metadata rather than a detection requirement -- the standard mandates that every compliant string contain (人工智能|AI) and (生成|合成), so the shared tail is guaranteed. That needs its own precision-labelling round before it ships.
**Why 千问 / 星绘 are NOT registered (measured 2026-07-18).** Adding a text mark is documented as "a `TextMarkConfig` + a thin subclass + one registry row", and that is true only when the mark is stamped like Doubao's. It does not hold for a FAINT mark, and 千问 is the counter-example. Measured on 14 hand-verified corpus positives, same pipeline, each mark scored with its OWN template:
| mark | n | mean NCC | median | above the 0.40 gate |
|---|---|---|---|---|
| doubao | 40 | 0.723 | 0.835 | **82%** |
| qwen | 14 | 0.170 | 0.179 | **0%** |
Three candidate explanations were ruled out in order, each by measurement:
1. **Not the synthetic render.** A template cut from an ACTUAL Qwen mark scores the same as the font-rendered one (real-vs-real 0.307 vs synthetic 0.308), and real masks do not match EACH OTHER.
2. **Not the morphology kernel.** `MORPH_OPEN`/`MORPH_CLOSE` use fixed 5px kernels regardless of mark size (~9% of a 57px-tall box, ~2.7% of a 188px one). Scaling them with the box height gained +0.014 mean and moved nothing across the gate.
3. **Not the appearance thresholds.** Sweeping `tophat_delta` / `logo_min_luma` / kernel size peaked at mean 0.35 with 4/14 over the gate.
The blocker is SEGMENTATION on a faint mark. Doubao is stamped bold and opaque so the white top-hat returns a clean glyph blob; the Qwen mark is a thin translucent overlay that shatters into specks, and no template can match a blob that is not there. **So the registry's cheap-to-add promise is conditional on mark contrast, and that condition should be checked before promising a new mark.** Adding 千问 needs a detection front-end that does not binarize the glyph first -- grayscale/edge correlation on the raw top-hat, or a learned patch classifier -- not a new silhouette. 星绘 additionally has only ONE confirmed corpus example, so even a working front-end could not calibrate its threshold yet. The synthetic renderer and the full evidence chain are kept in `scripts/render_vendor_silhouettes.py`; researched vendor specs are in `docs/watermarking-landscape.md`.
**RECALL, measured at last (unbiased random sample, 2026-07-18).** Every earlier round sampled where detectors FIRED, so recall was structurally unmeasurable. This round draws 240 images at RANDOM within each provenance class (160 TC260, 80 Google-C2PA) and labels them EXHAUSTIVELY -- both corners shown at native scale, so a missed mark is visible as a miss rather than absent from the data. Build it with `scripts/visible_recall_sample.py`; labels live in the gitignored research dir.
| mark | present | recall | 95% CI | precision | 95% CI |
|---|---|---|---|---|---|
| doubao | 90 | **89%** | 81-94% | **99%** | 93-100% |
| gemini | 46 | **96%** | 85-99% | **80%** | 68-88% |
| jimeng | 14 | 71% | 45-88% | 71% | 45-88% |
| jimeng_pill | 6 | 50% | 19-81% | 60% | 23-88% |
Effect of the `scale_basis` fix on the same sample (strict verdicts recorded before it): **doubao recall 71% -> 89%**, gemini 91% -> 96%, jimeng 64% -> 71%. Part of the doubao gain is the provenance relaxation rather than the basis alone, since the after-numbers run the full `auto` path.
**Three corrections this forced to earlier numbers:**
* **Gemini precision is 80% on an unbiased sample, not the 41% the addition-sampled harness reports.** The 41% is precision restricted to relaxation ADDITIONS, which are by construction the marginal cases; production sees mostly strict fires, which are near-perfect. Quote 80% for the product and 41% only when discussing the relaxation arm.
* Doubao is in excellent shape (89/99) and is no longer the problem it looked like before the basis fix.
* Landscape is improved but NOT solved: doubao recall by aspect is portrait 92% / square 92% / **landscape 79%**, so a residual geometry gap remains beyond the basis.
**Where the remaining loss actually is:** jimeng and the pill, both at small n with intervals too wide to tune against (a 14-positive and a 6-positive sample), plus **uncovered vendors at 6% of all sampled images** (千问/百度/星绘/抖音-class marks that no registered detector can ever fire on). Adding those vendors is now a larger win than any further tuning of the covered four, and `docs/watermarking-landscape.md` carries their researched specs.
**Per-mark geometry scaling (`scale_basis` / `scale_base`, 2026-07-18) -- the largest single recall defect found so far.** Every tuned fraction in `TextMarkConfig` was calibrated on PORTRAIT captures, where the width and the short side coincide, so the scaling basis was never exercised until landscape inputs were measured. Corpus-measured on 2572 unique TC260 carriers, BEFORE the fix:
| aspect ratio | detected | missed | miss rate |
|---|---|---|---|
| tall portrait <0.70 | 323 | 212 | 40% |
| portrait 0.70-0.95 | 607 | 533 | 47% |
| square ~1.0 | 190 | 272 | 59% |
| landscape 1.15-1.6 | 0 | 143 | **100%** |
| wide >1.6 | 0 | 292 | **100%** |
**Not one landscape image in the corpus ever produced a detection** -- 435 of them, zero. A width-scaled box is inflated by the aspect ratio on a wide image, so the glyph never lands inside it and the blob never gets scored: of the 1452 no-detection TC260 images the median doubao NCC was **0.057**, with 49% at ~zero. This is a LOCALIZATION failure, not a threshold one -- only 2.7% of those images sat in the band a threshold change could reach, which is why a day of threshold tuning could never have found it. Re-running the previously-undetected set with a short-side basis recovers **56% of landscape** (12% square, 4% portrait; 20% overall).
**The basis is PER MARK because the vendors genuinely differ.** The same switch took jimeng's labelled landscape positives from 13/13 to **0/13**: the Jimeng wordmark tracks the WIDTH while the Doubao strip tracks the short side, even though both are ByteDance and share a corner. So doubao is `short`, jimeng is `width`, and samsung stays `width` because there is no corpus evidence either way (1 addition corpus-wide) and an unmeasured change is not an improvement. China's GB 45438-2025 clause 5.2(e) mandates glyph height >= 5% of "the shortest side", which is why short-side is the natural prior -- but jimeng's measured behaviour overrides the prior. Regression: `tests/test_text_mark_engine.py::TestScaleBasis`.
**How this was missed for so long:** precision was measured repeatedly and recall never was. The eval harness now reports a `missed` column for exactly this reason, and it is what caught the jimeng regression the short-side switch introduced.
**Competitive detection among same-corner marks (`rivals` / `_rival_margin_ok`, 2026-07-18).** Detection was purely ABSOLUTE -- every engine scored its own template against its own threshold, so nothing ever asked the discriminative question "does this blob match the NEIGHBOUR's mark better than mine?". Doubao "豆包AI生成" and Jimeng "★ 即梦AI" both sit bottom-right in near-white CJK and survive the top-hat binarization as very similar blobs, so no absolute gate can separate them. Measured on hand-labelled examples, scoring BOTH templates against the SAME glyph blob (n=40 jimeng / 75 doubao / 20 other-vendor labels / 89 clean):
| feature | separability (0.5 = useless, 1.0 = perfect) |
|---|---|
| absolute `ncc_jimeng` | 0.96 |
| `ncc_jimeng` MINUS `ncc_doubao` | **0.99** |
At a 0.10 margin: real Jimeng wordmarks pass **100%**, Doubao strips 8%, other vendors' AI labels (千问/百度/星绘/抖音) 55%, no-mark corners 12%. Corpus effect (`scripts/visible_eval.py`, 741 labelled images): **jimeng precision 38% -> 63%, genuine detections unchanged at 40, false fires 65 -> 23.** Because real marks pass at 100% this is a pure precision gain, unlike raising a threshold -- so the earlier 0.85 relaxation patch was REVERTED to 0.70 and the recall it had sacrificed came back. **The gate is deliberately asymmetric:** doubao declares no rival, because the symmetric gate cost it 7 genuine detections to prevent 5 false ones (1.4:1 against) while jimeng gained 25pp for free -- doubao's absolute detector is already 86% precise and has nothing to buy. A rival's config is looked up lazily by asset name (`_rival_config`) so its template is scored at ITS own geometry; scoring it at the host mark's geometry would compare a correctly-sized template against a mis-sized one and hand the margin a free win. Regression: `tests/test_text_mark_engine.py::TestRivalMargin`.
**Evaluation harness (`scripts/visible_eval.py` + `scripts/visible_groundtruth.py`, 2026-07-18).** Run before AND after any detector change; `--save NAME` snapshots, `--vs NAME` diffs. Ground truth is 741 blind-labelled corpus images (779 cells across two rounds, two-sided control each). Three properties of the harness are load-bearing and were each added after the naive version produced a wrong number:
* **Adjudication scope.** A crop centred on one mark only lets the labeller rule on marks visible IN THAT CROP. Scoring jimeng against a pill-round image (top-left crop) books real bottom-right detections as false fires -- ~61% of pills carry a wordmark. Each image records which marks its crop could rule on; bottom-right marks co-adjudicate each other.
* **Provenance must come from METADATA, never from the labels.** A relaxation arm only fires when provenance names the vendor, so label-derived provenance hands the detector the answer: it scored gemini at 99% instead of the true 41%.
* **Recall is NOT reported.** The labelled set was sampled where detectors fired, so images every detector missed are absent by construction; a recall computed here would divide by a denominator that excludes exactly the failures recall exists to expose. The `missed` column catches a change LOSING marks it used to find, nothing more. True recall needs a random corpus sample labelled exhaustively -- not yet done.
Baseline at the time of writing (sensitivity `auto`, provenance from metadata): gemini 41% (321 fires), doubao 86% (77), jimeng 63% (63), jimeng_pill 64% (83), samsung unmeasurable (1 addition corpus-wide).
**Per-mark provenance NCC relaxation + the corroboration gate (2026-07-18).** Two defects, both on the DEFAULT `auto` path (no flag, driven by TC260 metadata), found by blind hand-labelling the ADDITIONS (accepted with provenance, rejected without) over 4417 unique TC260 carriers. Two-sided control: labeller sensitivity 100% (doubao) / 96% (jimeng), specificity 100% / 100% — the controls are what make the low numbers trustworthy, and the "clean" stratum is structural (another vendor's C2PA image, where a ByteDance mark cannot exist) rather than detector-defined, so it is not circular.
(1) **One shared `_PROVENANCE_NCC_FACTOR = 0.7` meant two different things per mark:**
| mark | band | precision | 95% CI | n |
|---|---|---|---|---|
| doubao | whole arm | 76% | 61-87% | 42 |
| | [0.280,0.340) | 58% | 36-77% | 19 |
| | [0.340,0.400) | 91% | 73-98% | 23 |
| jimeng | whole arm | 17% | 10-27% | 82 |
| | [0.315,0.383) | 12% | 6-22% | 68 |
| | [0.383,0.450) | 43% | 21-67% | 14 |
The factor is now a per-mark `TextMarkConfig.provenance_ncc_factor`. Doubao stays 0.70 — both bands return more true marks than false fills, so tightening would cost 11 genuine recoveries to prevent 8. Jimeng moves to 0.85 (gate 0.3825), dropping the 12% band: 8 genuine recoveries, 60 false fills (7.5:1), arm precision 17% → 43%. **Why jimeng fails is a detector problem, not a threshold one:** of its 68 false additions, 33 were DOUBAO marks and 17 were other vendors' AI labels (千问 / 百度 / 星绘 / 抖音) — relaxed, the silhouette keys on "some text in the bottom-right corner", not on "★ 即梦AI". Damage was scored separately because doubao and jimeng share a corner: 45 of the 68 fill a corner nothing else would touch, the other 23 are harmless (doubao fires strictly there and fills the same box anyway). A better silhouette, not a lower factor, is the real fix.
(2) **A weak detector must not corroborate a sibling (`_CANNOT_CORROBORATE`).** `resolve_trust` grants `confirmed` on a strict-detected sibling of the same `_PRODUCT_OF`, and `confirmed` bypasses the sibling's FP gate outright. The pill (~7% documented raw false-fire; 5.5% on 578 vendor negatives) maps to product "jimeng", so it could hand that bypass to the wordmark — a closed loop: pill false-fires on clean non-ByteDance content → jimeng relaxes 0.45 → 0.3825 and false-fires → `_keep_pill` sees "jimeng" in keys and takes the WORDMARK arm, removing the pill **unrestricted**, skipping the flatness guard written to stop exactly that smear. 3 of 578 negatives ran the full loop, one with `footprint_flat=0`. The fix costs nothing: negatives 3 → 0, TC260 carriers unchanged (jimeng 398 → 398, pill 117 → 117). `_keep_pill` already encoded this distrust for the pill's ACTION; the gap was that its TESTIMONY was ungated.
**Pill arms, re-measured on the same corpus** (149 blind-labelled TC260-arm fires, 35 wordmark, 33 unconfirmed): wordmark **94%** (CI 81-98%, confirming the original claim), TC260-metadata-only **21%** raw (CI 16-29%, consistent with the original ~27%) — **29%** (CI 20-40%) among the flat footprints the guard PASSES vs 14% among those it blocks. The guard works directionally but weakly: the shipped arm still runs at ~2.4 false fills per genuine one. Whether an arm that inaccurate belongs on the default path is a product call, not a tuning one.
**Samsung's relaxation is UNMEASURED and not measurable here:** the corpus holds 14 `samsung_genai` carriers and 3 visible Samsung detections total. Any precision estimate would carry a Wilson interval spanning most of [0,1]. Likely structural — detection is calibrated to the Italian locale string only.
0.35 sat on a cliff: +26pp recall over strict bought by filling a corner on ~6 of every 10 CLEAN photos, and `api.remove_visible(sensitivity="assume_ai")` reproduced it on 8/15 of the committed verified-clean negatives. The floor is applied in the arbiter and is **monotonic over strict** -- a mark the strict gate accepted is never dropped by it, so `assume_ai` only ever adds recall. End-to-end after the fix (400 Google-C2PA positives with metadata hidden from the detector, 256 camera negatives, through the public `api.remove_visible`): recall strict 55.0% / auto 55.2% / assume_ai 62.8%; false fire 0.0% / 0.0% / 2.3%. The residual 2.3% contains **no gemini at all** (doubao 2, jimeng_pill 2, jimeng 1, samsung 1 of 256) -- the text marks' own relaxed gates (<1% each) plus the pill's flat-footprint arm, all pre-existing and benign. The superseded "~46% -> ~92%" figure measured recall only, on Google-C2PA files where the answer was always Google; false fire on non-Google content was never measured. Marks other than gemini carry no floor because their bypassed false-fire is already under 1%. Regression: `tests/test_watermark_registry.py::TestArbiter::{test_assume_ai_drops_sparkle_below_the_assumed_floor, test_assume_ai_keeps_sparkle_confirmed_by_metadata_below_the_floor, test_assume_ai_is_monotonic_over_strict}`.
**Provenance prior:** when local metadata already confirms the vendor, the mark's detection trust gate is relaxed (a confirmed vendor means the mark is present with high prior, so a mark the conservative detector would demote as a content false positive is trusted). `detect_marks` / `remove_auto_marks` take a `provenance` frozenset and `KnownMark.remove` a `provenance` flag. Mapping: a Google/Gemini C2PA issuer relaxes gemini (skips its false-positive gate and lowers the trust threshold from 0.5 to 0.35); a China-AIGC (TC260) label relaxes doubao/jimeng; `samsung_genai` relaxes samsung. Corpus finding: on Google-C2PA images, Gemini sparkle recall rose from ~46% (plain detector) to ~90% with the provenance prior (recovering marks the vendor moved or re-rendered). That gain is why the bypass exists, and it is conditional on the metadata actually naming the vendor — a caller merely ASSUMING the image is AI does not get it unconditionally (see the assumed-trust confidence floor above). The localizer is cheap CPU (cv2/numpy), so a memory-tight caller runs it anywhere; the heavy MI-GAN/LaMa fill is opt-in and chosen by the caller.
@@ -241,7 +346,7 @@ Diffusion SynthID removal. The `--tile/--no-tile` knob is the *lossless* alterna
### `visible`
Known-visible-mark removal by **localize -> fill**: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. `--backend auto|cv2|migan|lama` (default `auto`) picks the fill: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available (LaMa is auto-preferred when a learned backend is present; a memory-tight deploy pins migan). `--sensitivity auto|strict|assume-ai` (default `auto`) controls how hard a borderline mark is trusted (see the registry section: the visual detectors are metadata-independent; `auto` relaxes a mark only on same-product evidence, `assume-ai` relaxes every mark on the caller's AI assertion, subject to the assumed-trust confidence floor where the vendor is unconfirmed — the only path to higher recall on a metadata-stripped screenshot). `--backend` and `--sensitivity` are shared across `visible`/`all`/`batch`. Detection keys on each mark's own shape, and under `auto` the trust gate is relaxed when local metadata confirms the vendor (a Google/Gemini C2PA issuer relaxes gemini, a China-AIGC label relaxes doubao/jimeng, `samsung_genai` relaxes samsung), so a moved or re-rendered mark is still caught. `--mark auto` (default) removes EVERY detected mark in one pass (`registry.remove_auto_marks`, not the single strongest -- a Jimeng-basic image carries both the top-left pill and the bottom-right wordmark) from: the Gemini sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-LEFT, Italian-locale detection), and the capture-less Jimeng "AI生成" pill (top-left, `pill_engine`). The pill's weak edge-NCC detector is gated in `remove_auto_marks` via `_keep_pill` (32k real-upload corpus validation 2026-07): never on Doubao, and two confirmation arms since metadata confirms the platform, not pill presence. (1) The bottom-right wordmark fired — ~94% precise and survives metadata-STRIPPED uploads (screenshots / re-saves) — removes the pill unrestricted. (2) TC260 metadata confirms Jimeng (`"jimeng" in provenance`, from `cli._visible_provenance`) OR the caller asserts AI (`sensitivity == "assume_ai"`), no wordmark — ~27% precise, its false fires are textured ceilings/walls that the fill visibly SMEARS — removes the pill ONLY when the top-left footprint is flat enough for an invisible fill (`pill_engine.footprint_is_flat`, median-Sobel ≤ `_FLAT_TEXTURE_MAX`; the flatness guard holds even under `assume_ai`). No confirmation → never removed. `--mark gemini|doubao|jimeng|samsung|jimeng_pill` forces one (choices come from the registry). Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. For arbitrary logos/objects use `erase`. **When `--mark auto` finds no known mark (the common case — ~74% of real uploads carry no registered visible mark), the command does NOT silently re-serve the input as a finished result.** It runs a cheap metadata-only `identify`, prints actionable guidance (if the image carries an invisible/metadata mark, e.g. an OpenAI/Gemini C2PA image, it points to `all`; otherwise it does NOT imply the image is clean -- it warns that an invisible pixel watermark like SynthID cannot be detected once the metadata proxy is gone and routes to both `all` and `erase --region`), writes NO output file, and exits **`EXIT_NO_VISIBLE_MARK` (2)** — distinct from success (0) and a hard error (1) so a wrapping service (raiw.cc) can surface the message instead of treating the unchanged image as done (the production "it didn't work" / score-0 trap). Same handling for an explicit `--mark <name>` that is not detected. Helper `cli._no_visible_mark_exit`; regression-guarded by `tests/test_cli.py::TestVisibleCommand::test_visible_auto_no_mark_exits_two_with_eraser_hint` and `test_visible_auto_no_mark_routes_to_all_when_metadata`. `--no-detect` still forces the gemini fallback and proceeds (exit 0).
Known-visible-mark removal by **localize -> fill**: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. `--backend auto|cv2|migan|lama` (default `auto`) picks the fill: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available (LaMa is auto-preferred when a learned backend is present; a memory-tight deploy pins migan). `--sensitivity auto|strict|assume-ai` (default `auto`) controls how hard a borderline mark is trusted (see the registry section: the visual detectors are metadata-independent; `auto` relaxes a mark only on same-product evidence, `assume-ai` relaxes every mark on the caller's AI assertion, subject to the assumed-trust confidence floor where the vendor is unconfirmed — the only path to higher recall on a metadata-stripped screenshot). `--backend` and `--sensitivity` are shared across `visible`/`all`/`batch`. Detection keys on each mark's own shape, and under `auto` the trust gate is relaxed when local metadata confirms the vendor (a Google/Gemini C2PA issuer relaxes gemini, a China-AIGC label relaxes doubao/jimeng, `samsung_genai` relaxes samsung), so a moved or re-rendered mark is still caught. `--mark auto` (default) removes EVERY detected mark in one pass (`registry.remove_auto_marks`, not the single strongest -- a Jimeng-basic image carries both the top-left pill and the bottom-right wordmark) from: the Gemini sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-LEFT, Italian-locale detection), and the capture-less Jimeng "AI生成" pill (top-left, `pill_engine`). The pill's weak edge-NCC detector is gated in `remove_auto_marks` via `_keep_pill` (32k real-upload corpus validation 2026-07): never on Doubao, and two confirmation arms since metadata confirms the platform, not pill presence. (1) The bottom-right wordmark fired — ~94% precise and survives metadata-STRIPPED uploads (screenshots / re-saves) — removes the pill unrestricted. (2) TC260 metadata confirms Jimeng (`"jimeng" in provenance`, from `cli._visible_provenance`) OR the caller asserts AI (`sensitivity == "assume_ai"`), no wordmark — **re-measured 2026-07-18 on 149 blind-labelled pill fires: 21% precise raw (CI 16-29%), 29% (CI 20-40%) among the flat footprints the guard actually PASSES, 14% among those it blocks** its false fires are textured ceilings/walls that the fill visibly SMEARS — removes the pill ONLY when the top-left footprint is flat enough for an invisible fill (`pill_engine.footprint_is_flat`, median-Sobel ≤ `_FLAT_TEXTURE_MAX`; the flatness guard holds even under `assume_ai`). No confirmation → never removed. `--mark gemini|doubao|jimeng|samsung|jimeng_pill` forces one (choices come from the registry). Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. For arbitrary logos/objects use `erase`. **When `--mark auto` finds no known mark (the common case — ~74% of real uploads carry no registered visible mark), the command does NOT silently re-serve the input as a finished result.** It runs a cheap metadata-only `identify`, prints actionable guidance (if the image carries an invisible/metadata mark, e.g. an OpenAI/Gemini C2PA image, it points to `all`; otherwise it does NOT imply the image is clean -- it warns that an invisible pixel watermark like SynthID cannot be detected once the metadata proxy is gone and routes to both `all` and `erase --region`), writes NO output file, and exits **`EXIT_NO_VISIBLE_MARK` (2)** — distinct from success (0) and a hard error (1) so a wrapping service (raiw.cc) can surface the message instead of treating the unchanged image as done (the production "it didn't work" / score-0 trap). Same handling for an explicit `--mark <name>` that is not detected. Helper `cli._no_visible_mark_exit`; regression-guarded by `tests/test_cli.py::TestVisibleCommand::test_visible_auto_no_mark_exits_two_with_eraser_hint` and `test_visible_auto_no_mark_routes_to_all_when_metadata`. `--no-detect` still forces the gemini fallback and proceeds (exit 0).
### `batch`
+27
View File
@@ -52,3 +52,30 @@ Grok JPEG downloads (Aurora model) carry **no C2PA, no XMP, no SynthID, no IPTC*
**Visible-mark landscape beyond the registry.** Meta stamps a visible "Imagined with AI" mark (bottom-LEFT, a small symbol) on its OWN Meta AI / "Imagine" output; for third-party images it relies on C2PA / IPTC, not a visible mark. Samsung Galaxy AI additionally uses a **four-star icon** variant in a corner alongside the localized text wordmark `samsung_engine` calibrates (only the Italian text variant is covered) -- the icon is a distinct, uncovered variant. Every source agrees visible + metadata marks are trivially removable (crop / screenshot, ~2 s), which is the tool's premise.
**Regulatory driver -- China GB 45438-2025 is the strongest VISIBLE-mark mandate.** The CAC / TC260 "Measures for Labeling AI-Generated Synthesized Content" (issued March 2025, **effective 2025-09-01**, technical standard **GB 45438-2025**, building on the TC260 Aug-2023 practice guide) MANDATE a **visible** label for AI images -- a visible textual mark whose height must be **>= 5% of the image's shortest side** -- plus the metadata (implicit) label. So every major Chinese platform now ships visible "AI生成"-style text marks (we cover Doubao / Jimeng; expect more CJK-text marks under this driver). By contrast EU AI Act Article 50 mandates only the MACHINE-READABLE mark (enforceable 2026-08-02, grace to 2026-12-02); a visible label is proposed and modality-specific (visible for images) but is NOT a hard "fixed icon" mandate -- a claim that Art 50 requires a clearly-visible fixed icon for images was refuted in verification. Primary-source dates verified against the article/standard text, not search summaries.
## Uncovered visible marks: implementation specs (deep-research 2026-07-18)
Triggered by a corpus finding: **50% of TC260-labelled uploads (1452 of 2896 unique) produced no detection at all**, and hand-inspection showed the bulk are MISSED Doubao marks (a localization defect, now fixed -- see `scale_basis` in `docs/module-internals.md`) plus a minority of genuinely uncovered vendors. Verification status is labelled per claim; treat (b)/(c) as leads, not ground truth.
**GB 45438-2025 clause 5.2, the binding constraint for every Chinese mark (VERIFIED (a) -- full standard text extracted from the TC260-hosted PDF).** Verbatim requirements for an image's explicit label:
- 应采用文字提示 (must be a TEXT prompt);
- must contain BOTH an AI element (`人工智能` or `AI`) AND a generation element (`生成` and/or `合成`);
- 应位于图片的边或角 -- **edge OR corner**, so bottom-right is NOT mandated (Annex C.2's bottom-right figure is a non-normative example);
- 字型应清晰可辨 (legible typeface -- no family named);
- **文字高度不应低于画面最短边长度的 5%** -- glyph HEIGHT >= 5% of the frame's SHORTEST side, with a note defining the shortest side for non-rectangular images.
Two consequences we can exploit: (1) the 5% floor is a **scale prior** -- a compliant CN mark's glyphs are large (>= 51 px on a 1024² image), so a CN silhouette ladder can be anchored at ~5-10% of the short side instead of swept broadly, which should cut false fires; (2) every compliant string shares the tail `AI生成` / `AI合成`, so a shared suffix silhouette plus a per-vendor prefix may beat five independent templates. NOT established: whether 文字高度 means cap height, em box, or rendered bounding box (a ~1.3x spread in template scale). Sources: `https://www.tc260.org.cn/upload/2025-03-15/1742009439794081593.pdf`, parent CAC measure `https://www.cac.gov.cn/2025-03/14/c_1743654684782215.htm` (the CAC text itself specifies no size or corner).
**Alibaba Qwen -- two surfaces that differ (API tier VERIFIED (a)).** Model Studio docs state verbatim that the API adds a `Qwen-Image` watermark 在图像右下角 and 默认值为 false -- so API output is **unwatermarked by default**, and when enabled the mark is a LATIN wordmark, not CJK, and not GB-compliant in wording. The consumer app's `千问AI生成` (bottom-right) is (b) secondary only -- no Alibaba primary page states it. So Qwen needs TWO templates, and its absence is never evidence of a clean image. Source: `https://help.aliyun.com/zh/model-studio/qwen-image-api`.
**星绘 is ByteDance (VERIFIED (a): Baidu Baike + App Store listing, now branded 豆包旗下, team folded into Doubao April 2025).** So `星绘AI生成` is very likely the Doubao house style -- same typeface, same corner, possibly the same top-left `AI生成` pill. Starting from the Doubao `TextMarkConfig` and swapping the two lead glyphs is the cheap path. String/position themselves are (c) inferred.
**Baidu: could not establish.** No primary or credible secondary source names the exact string or position; it could be `百度AI生成`, `文心一格AI生成`, or product-specific. Harvest the glyphs from corpus positives, not the web.
**Meta `Imagined with AI` (string VERIFIED (a) from Meta's own newsroom; POSITION NOT VERIFIED).** Sources conflict (bottom-left vs bottom-right) and one claims newer Meta models dropped the visible mark for invisible watermarking; none survived a fetch. Do NOT encode a corner without a corpus sample. Meta also embeds IPTC + invisible watermarks, which `identify` already reads. Source: `https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/`.
**Samsung English/other locales: still not established.** Samsung's own support page says only that "A Galaxy AI watermark will appear on AI-generated images" -- no string, no corner. Every community thread carrying the exact English string returned HTTP 403 to WebFetch, so the search paraphrase (bottom-left) is deliberately NOT recorded as fact. Feature-tier detail (b): the mark is applied by Generative Edit / sketch-to-image but reportedly NOT by Object Eraser, so Samsung absence is feature-dependent. The four-star icon variant: nothing found.
**The one document that would settle ByteDance placement is BLOCKED.** Douyin's 《抖音关于人工智能生成内容标识的水印与元数据规范》 aims to give AI tools a unified watermark style and position, which would cover Doubao / Jimeng / 星绘 at once. Both mirrors return HTTP 403 to WebFetch; a secondary report (b, unconfirmed) says the watermark is `AI生成` + tool name + company name placed **top-left** -- which would explain the Jimeng pill's top-left position but contradicts the GB annex's bottom-right example. Worth one retry through Chrome MCP with a real browser session.
**No vendor publishes typeface, colour, opacity, plate, or margin for ANY of these marks.** The only font-adjacent requirement anywhere is GB's "legible typeface". So each synthetic silhouette's font must be calibrated against corpus positives exactly as the Jimeng pill was; candidate CJK families by platform convention (inference): HarmonyOS Sans / Source Han Sans / Noto Sans CJK SC for Android-origin apps, PingFang SC for iOS-origin.
+104
View File
@@ -0,0 +1,104 @@
"""Render SYNTHETIC detection silhouettes for the CJK vendor text marks (data-safe).
Adding a mark needs only a DETECTION silhouette, and it must be font-rendered rather
than derived from user uploads: the corpus is real user content and may never reach a
tracked asset (see the repo CLAUDE.md data-safety rule). Seeing real samples to learn
the glyphs, weight and layout is fine; the committed template stays synthetic.
Covered here:
qwen "千问AI生成" -- Alibaba Tongyi Qianwen, bottom-right, 3-lobed logo + text
xinghui "星绘AI生成" -- ByteDance 星绘, bottom-right, 4-point sparkle + text
The leading LOGO is deliberately NOT rendered. It is the part that varies most between
releases and is hardest to reproduce synthetically, while the CJK run is stable and is
what actually discriminates one vendor from another (the shared `AI生成` tail is exactly
what does NOT discriminate -- see the rival-margin mechanism in _text_mark_engine).
Regenerate with: uv run python scripts/render_vendor_silhouettes.py
STATUS 2026-07-18: these two marks are NOT registered, and this script is kept as the
method + the record of why. Measured on 14 hand-verified 千问 positives from the corpus,
the current detect architecture (top-hat glyph blob -> binary TM_CCOEFF_NORMED) cannot
see this mark AT ALL:
same pipeline, each mark scored with its OWN template, on real positives
doubao n=40 mean NCC 0.723 median 0.835 >= 0.40 gate: 82%
qwen n=14 mean NCC 0.170 median 0.179 >= 0.40 gate: 0%
Three checks ruled out the obvious explanations, in order:
1. NOT the synthetic render. A template cut from an ACTUAL Qwen mark scores the same
as the font-rendered one (real-vs-real 0.307 vs synthetic 0.308) -- and real masks
do not even match EACH OTHER.
2. NOT the morphology kernel size. Scaling MORPH_OPEN/CLOSE with the box height (they
are fixed 5px, ~9% of a 57px-tall box) gained only +0.014 mean and moved nothing
across the gate.
3. NOT the appearance thresholds. Sweeping tophat_delta / logo_min_luma / kernel
reached at best mean 0.35 with 4/14 over the gate.
The blocker is SEGMENTATION on a faint mark: Doubao is stamped bold and opaque, so the
white top-hat returns a clean glyph blob; the Qwen mark is a thin translucent overlay
that shatters into specks, and no template can match a blob that is not there. Adding
it therefore needs a detection front-end that does not depend on binarizing the glyph
(grayscale/edge correlation on the raw top-hat, or a learned patch classifier) -- not a
new silhouette. Shipping it on the current front-end would mean a detector that finds
almost nothing and, at any threshold low enough to fire, fires on arbitrary corner text.
星绘 additionally has only ONE confirmed example in the corpus, so even a working
front-end could not have its threshold calibrated yet.
"""
from __future__ import annotations
import sys
from pathlib import Path
import numpy as np
from PIL import Image, ImageDraw, ImageFont
_ASSETS = Path(__file__).resolve().parents[1] / "src" / "remove_ai_watermarks" / "assets"
# STHeiti Medium approximates the semibold CJK sans these marks are set in; the exact
# family is unpublished for every vendor (GB 45438-2025 only requires a legible face).
_FONT = "/System/Library/Fonts/STHeiti Medium.ttc"
MARKS = {
"qwen_alpha.png": "千问AI生成",
"xinghui_alpha.png": "星绘AI生成",
}
def render(text: str, width: int = 335) -> np.ndarray:
"""Binary glyph silhouette (255 = glyph), sized to the doubao asset's convention.
Matching doubao's 335px asset width keeps the `alpha_*_frac` numbers transferable,
since these marks are the same house style and scale.
"""
probe = Image.new("L", (10, 10))
d0 = ImageDraw.Draw(probe)
size = 8
while size < 200: # grow until the run fills the target width
f = ImageFont.truetype(_FONT, size)
if d0.textbbox((0, 0), text, font=f)[2] >= width * 0.98:
break
size += 1
font = ImageFont.truetype(_FONT, size)
bb = d0.textbbox((0, 0), text, font=font)
w, h = bb[2] - bb[0], bb[3] - bb[1]
pad = max(2, int(h * 0.12))
im = Image.new("L", (w + 2 * pad, h + 2 * pad), 0)
ImageDraw.Draw(im).text((pad - bb[0], pad - bb[1]), text, font=font, fill=255)
return np.array(im)
def main() -> None:
try:
for name, text in MARKS.items():
sil = render(text)
Image.fromarray(sil).save(_ASSETS / name)
print(f"wrote {_ASSETS / name} ({sil.shape[1]}x{sil.shape[0]}) text={text!r}")
except OSError as e:
print(f"Font not found ({e}); install a CJK font or edit _FONT.", file=sys.stderr)
raise SystemExit(1) from e
if __name__ == "__main__":
main()
+164
View File
@@ -0,0 +1,164 @@
"""Benchmark harness for the visible-mark detectors.
Run this before AND after any detector change. It re-runs perception over the
hand-labelled ground truth and reports, per mark, how often a fire is correct --
with Wilson intervals, so a change inside the noise is visible as such.
uv run python scripts/visible_eval.py # score current code
uv run python scripts/visible_eval.py --save baseline # snapshot for comparison
uv run python scripts/visible_eval.py --vs baseline # diff against a snapshot
WHAT THIS SET CAN AND CANNOT MEASURE -- read before quoting a number:
* PRECISION: sound. Every labelled crop is centred on the region a detector
pointed at, so "the detector fired mark K here, was K actually there" is
exactly the question the labels answer.
* RECALL: NOT measurable here, and the harness refuses to print it. The labelled
images were SAMPLED WHERE DETECTORS FIRED (relaxation additions plus controls),
so images carrying a mark that every detector missed are absent by construction.
Computing recall on this set would divide by a denominator that excludes exactly
the failures recall is meant to expose, and would report a flattering number.
Recall needs a RANDOM corpus sample laballed exhaustively -- a separate round.
* `other_ai_label` (千问 / 百度 / 星绘 / 抖音) counts as a FALSE fire for any
registered mark, because it is a different vendor's label. It is tracked
separately in the confusion output since it is the dominant jimeng failure.
"""
from __future__ import annotations
import argparse
import json
import math
import sys
from collections import Counter
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from remove_ai_watermarks import watermark_registry as wr
from remove_ai_watermarks.image_io import imread
GT = Path("data/spaces/_research_20260718_textmark_relaxation/groundtruth.jsonl")
SNAP = Path("data/spaces/_research_20260718_textmark_relaxation/snapshots")
MARKS = ("gemini", "doubao", "jimeng", "samsung", "jimeng_pill")
def wilson(k: int, n: int) -> tuple[float, float]:
if n == 0:
return (0.0, 0.0)
z, p = 1.96, k / n
d = 1 + z * z / n
c = p + z * z / (2 * n)
s = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n))
return ((c - s) / d, (c + s) / d)
def provenance_for(rec: dict) -> frozenset[str]:
"""The provenance production would have: read from the file's own METADATA.
Never derive this from the labels. A relaxation arm only fires when provenance
names the vendor, so label-derived provenance silently tells the detector the
answer and every arm scores near-perfect (observed: gemini 99% instead of 34%).
"""
return frozenset(rec.get("provenance", []))
def score(sensitivity: str = "auto") -> dict:
recs = [json.loads(line) for line in GT.open()]
per: dict[str, Counter] = {m: Counter() for m in MARKS}
confusion: dict[str, Counter] = {m: Counter() for m in MARKS}
missing = 0
for rec in recs:
img = imread(rec["path"])
if img is None:
missing += 1
continue
cands = wr._build_candidates(img)
ctx = wr.Context(sensitivity=sensitivity, provenance=provenance_for(rec))
fired = {d.candidate.key for d in wr.decide(cands, ctx)}
for m in MARKS:
# Only score a mark on images whose shown crop could rule on it. Scoring
# outside that scope books real detections as false fires -- see the
# adjudication note in visible_groundtruth.py.
if m not in rec.get("adjudicated", []):
continue
if m not in fired:
per[m]["fn"] += 1 if m in rec["present"] else 0
continue
if m in rec["present"]:
per[m]["tp"] += 1
else:
per[m]["fp"] += 1
for s in rec["seen"]:
confusion[m][s] += 1
scope = {m: sum(1 for r in recs if m in r.get("adjudicated", [])) for m in MARKS}
return {
"per": {m: dict(c) for m, c in per.items()},
"scope": scope,
"confusion": {m: dict(c) for m, c in confusion.items()},
"missing": missing,
"n": len(recs),
"sensitivity": sensitivity,
}
def report(res: dict, prev: dict | None = None) -> None:
print(f"\nground truth: {res['n']} images ({res['missing']} unreadable) sensitivity={res['sensitivity']}")
print("=" * 78)
print(
f"{'mark':12s} {'scope':>6s} {'fires':>6s} {'correct':>8s} "
f"{'precision':>11s} {'95% CI':>13s} {'missed':>7s} delta"
)
print("-" * 78)
for m in MARKS:
c = res["per"][m]
tp, fp, fn = c.get("tp", 0), c.get("fp", 0), c.get("fn", 0)
n = tp + fp
scope = res["scope"].get(m, 0)
if n == 0:
print(f"{m:12s} {scope:6d} {0:6d} {'-':>8s} {'-':>11s} {'-':>13s} {fn:7d}")
continue
lo, hi = wilson(tp, n)
delta = ""
if prev:
pc = prev["per"][m]
pn = pc.get("tp", 0) + pc.get("fp", 0)
if pn:
d = tp / n - pc.get("tp", 0) / pn
delta = f"{d:+.1%} (fires {pn}->{n})"
print(f"{m:12s} {scope:6d} {n:6d} {tp:8d} {tp / n:10.0%} {lo:5.0%}-{hi:<6.0%} {fn:7d} {delta}")
print("\nwhat the FALSE fires actually were:")
for m in MARKS:
conf = {k: v for k, v in res["confusion"][m].items() if k != m}
if conf:
print(f" {m:12s} {dict(sorted(conf.items(), key=lambda kv: -kv[1]))}")
print("\n'scope' = images whose crop could rule on that mark; 'missed' = labelled marks it did not fire on.")
print("NOTE: 'missed' is NOT recall -- this set was sampled where detectors fired, so images")
print(" every detector missed are absent by construction. Use it only to catch a change")
print(" LOSING marks it used to find; an unbiased random sample is needed for true recall.")
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("--save", metavar="NAME")
ap.add_argument("--vs", metavar="NAME")
ap.add_argument("--sensitivity", default="auto")
a = ap.parse_args()
res = score(a.sensitivity)
prev = None
if a.vs:
f = SNAP / f"{a.vs}.json"
if f.exists():
prev = json.loads(f.read_text())
else:
print(f"(no snapshot {f}; showing absolute numbers)")
report(res, prev)
if a.save:
SNAP.mkdir(parents=True, exist_ok=True)
(SNAP / f"{a.save}.json").write_text(json.dumps(res, indent=1))
print(f"\nsaved snapshot -> {SNAP / f'{a.save}.json'}")
if __name__ == "__main__":
main()
+134
View File
@@ -0,0 +1,134 @@
"""Consolidate the hand-labelled contact-sheet rounds into ONE ground-truth file.
Ground truth is `uid -> the set of visible marks actually present`, hand-labelled
blind against contact sheets with a two-sided control in every round. Rounds so far:
2026-07-18 text-mark/pill round : 423 cells (doubao / jimeng / jimeng_pill arms)
2026-07-18 gemini round : 356 cells (gemini relaxation additions)
DATA SAFETY: the corpus is real user uploads. This script reads the gitignored
corpus and writes a gitignored ground-truth file. Neither the images nor this
output may be committed; only the harness is. See the repo CLAUDE.md.
The labels record what the LABELLER SAW in the crop, one of:
doubao | jimeng | pill | sparkle | other_ai_label | none | uncertain
`other_ai_label` is a real visible AI label from a vendor we do NOT have a mark for
(千问 / 百度 / 星绘 / 抖音); it is NOT a positive for any registered mark, but it is
also not "clean" -- it is exactly what the relaxed jimeng detector confuses.
`uncertain` rows are EXCLUDED from scoring rather than coerced, so a labeller's
honest doubt never becomes a fabricated data point.
"""
from __future__ import annotations
import csv
import json
import sys
from pathlib import Path
SEEN_TO_MARK = {
"doubao": "doubao",
"jimeng": "jimeng",
"pill": "jimeng_pill",
"sparkle": "gemini",
}
# Which marks a crop centred on `key` lets the labeller rule on (same corner = visible
# in the same crop). Doubao and Jimeng share the bottom-right corner.
_ADJUDICATES = {
"doubao": ("doubao", "jimeng"),
"jimeng": ("doubao", "jimeng"),
"jimeng_pill": ("jimeng_pill",),
"gemini": ("gemini",),
"samsung": ("samsung",),
}
ROUNDS = [
("textmark", "labels.csv", "manifest.csv"),
("gemini", "gemini_labels.csv", "gemini_manifest.csv"),
]
def metadata_provenance(path: str) -> list[str]:
"""The vendor keys LOCAL METADATA confirms -- what cli._visible_provenance reads.
Must come from the file's own metadata, never from the labels: deriving it from
the ground truth would hand the detector the answer it is being scored on (a
relaxation only fires when provenance names the vendor, so label-derived
provenance makes every arm look near-perfect). Read from the corpus `identify`
sidecar, which is the same signal production computes.
"""
p = Path(path)
sidecar = Path(str(p.parent).replace("/originals/", "/identify/")) / (p.name.split("_src")[0] + ".json")
if not sidecar.exists():
return []
try:
with sidecar.open() as fh:
wm = " | ".join(json.load(fh).get("watermarks", []))
except Exception:
return []
keys: list[str] = []
if "China AIGC label" in wm:
keys += ["doubao", "jimeng"]
if "C2PA Content Credentials (Google LLC" in wm:
keys.append("gemini")
if "Samsung Galaxy AI" in wm:
keys.append("samsung")
return keys
def main() -> None:
root = Path(sys.argv[1] if len(sys.argv) > 1 else "data/spaces/_research_20260718_textmark_relaxation")
out = root / "groundtruth.jsonl"
rows: dict[str, dict] = {}
stats: dict[str, int] = {}
for round_name, labels_file, manifest_file in ROUNDS:
with (root / labels_file).open() as fh:
labels = {int(r["idx"]): r["seen"] for r in csv.DictReader(fh)}
with (root / manifest_file).open() as fh:
manifest_rows = list(csv.DictReader(fh))
for m in manifest_rows:
seen = labels.get(int(m["idx"]))
if seen is None:
continue
stats[seen] = stats.get(seen, 0) + 1
if seen == "uncertain":
continue # excluded by design -- never coerce a doubt into a label
rec = rows.setdefault(
m["uid"],
{"uid": m["uid"], "path": m["path"], "present": [], "seen": [], "rounds": [], "adjudicated": []},
)
# ADJUDICATION SCOPE -- load-bearing. A crop centred on one mark only lets
# the labeller rule on marks visible IN THAT CROP. A pill crop (top-left)
# says nothing about a bottom-right wordmark, so scoring jimeng against a
# pill-round image would book real detections as false fires (~61% of pills
# carry a wordmark). Bottom-right marks co-adjudicate each other: one crop
# of that corner shows whichever of Doubao/Jimeng is there.
for k in _ADJUDICATES.get(m["key"], (m["key"],)):
if k not in rec["adjudicated"]:
rec["adjudicated"].append(k)
mark = SEEN_TO_MARK.get(seen)
if mark and mark not in rec["present"]:
rec["present"].append(mark)
if seen not in rec["seen"]:
rec["seen"].append(seen)
if round_name not in rec["rounds"]:
rec["rounds"].append(round_name)
for r in rows.values():
r["provenance"] = metadata_provenance(r["path"])
with out.open("w") as fh:
for r in rows.values():
fh.write(json.dumps(r) + "\n")
print(f"wrote {out} images={len(rows)}")
print("label distribution across all rounds:", dict(sorted(stats.items(), key=lambda kv: -kv[1])))
n_pos = sum(1 for r in rows.values() if r["present"])
print(f"images with at least one registered mark: {n_pos}; clean-of-registered-marks: {len(rows) - n_pos}")
adj: dict[str, int] = {}
for r in rows.values():
for k in r["adjudicated"]:
adj[k] = adj.get(k, 0) + 1
print("images each mark can be SCORED on (adjudication scope):", dict(sorted(adj.items())))
if __name__ == "__main__":
main()
+126
View File
@@ -0,0 +1,126 @@
"""Build an UNBIASED random sample for measuring visible-mark RECALL.
Every earlier labelling round sampled where detectors FIRED, so images that every
detector missed were absent by construction and recall was unmeasurable. This round
samples at random within a provenance class and shows the labeller the corners where
a mark can physically be, so a MISSED mark is visible as such.
Design decisions that matter:
* SAMPLING FRAME is per provenance class, not the whole corpus. Recall is only
meaningful against a denominator where the mark CAN occur: TC260 carriers for the
ByteDance marks and the pill, Google-C2PA for the sparkle. Reporting one blended
recall over all uploads would mostly measure how often each vendor appears.
* NATIVE RESOLUTION crops, never a downscaled whole image: a 220px preview destroys a
faint mark (measured in an earlier round), which would inflate the miss count with
the labeller's own blindness rather than the detector's.
* BOTH corners per image (top-left pill, bottom-right wordmark/strip/sparkle), so one
pass adjudicates every registered mark instead of one mark per crop.
* The detector's verdict is NOT shown and is not in the sheet order -- the manifest
holds it and must not be opened until labelling ends.
"""
from __future__ import annotations
import csv
import json
import random
import sys
from pathlib import Path
from typing import TYPE_CHECKING, Any
import cv2
import numpy as np
if TYPE_CHECKING:
from numpy.typing import NDArray
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from remove_ai_watermarks.image_io import imread
CELL_W = 300
COLS, ROWS = 4, 3
PER = COLS * ROWS
def corner_strip(img: NDArray[Any]) -> NDArray[Any] | None:
"""Top-left and bottom-right corners stacked, at (near) native scale.
Crop size is a fraction of the SHORT side because that is what marks scale with
(China's GB 45438-2025 sizes the mandated label off the shortest side).
"""
h, w = img.shape[:2]
short = min(h, w)
cw, ch = int(short * 0.46), int(short * 0.17)
cw, ch = min(cw, w), min(ch, h)
tl = img[0:ch, 0:cw]
br = img[h - ch : h, w - cw : w]
strip = np.vstack([tl, np.full((6, cw, 3), 90, np.uint8), br])
s = min(1.0, CELL_W / strip.shape[1]) # never UPSCALE past native
if s < 1.0:
strip = cv2.resize(strip, (CELL_W, max(1, int(strip.shape[0] * s))), interpolation=cv2.INTER_AREA)
return strip
def main() -> None:
scan = Path(sys.argv[1])
out = Path(sys.argv[2])
n_tc260 = int(sys.argv[3]) if len(sys.argv) > 3 else 160
n_google = int(sys.argv[4]) if len(sys.argv) > 4 else 80
out.mkdir(parents=True, exist_ok=True)
recs = [json.loads(line) for line in scan.open() if '"marks"' in line]
seen: dict[tuple, dict] = {}
for r in recs: # exact-duplicate uploads share the whole NCC vector
seen.setdefault(tuple(r.get("shape", ())) + tuple(sorted((k, m["conf"]) for k, m in r["marks"].items())), r)
uniq = list(seen.values())
tc = [r for r in uniq if r["cls"] == "tc260"]
goog = [r for r in uniq if r["cls"] == "neg" and "Google" in r.get("platform", "")]
rng = random.Random(2026) # noqa: S311 -- sampling, not cryptography
rng.shuffle(tc)
rng.shuffle(goog)
picked = [("tc260", r) for r in tc[:n_tc260]] + [("google", r) for r in goog[:n_google]]
rng.shuffle(picked)
manifest, cells = [], []
for cls, r in picked:
img = imread(r["path"])
if img is None:
continue
strip = corner_strip(img)
if strip is None:
continue
cells.append(strip)
manifest.append(
{
"idx": len(cells) - 1,
"uid": r["uid"],
"cls": cls,
"path": r["path"],
"fired": "|".join(sorted(k for k, m in r["marks"].items() if m["strict"])),
**{f"ncc_{k}": m["conf"] for k, m in r["marks"].items()},
}
)
cell_h = max(c.shape[0] for c in cells)
for si in range(0, len(cells), PER):
chunk = cells[si : si + PER]
sheet = np.full((ROWS * (cell_h + 26), COLS * (CELL_W + 8), 3), 40, np.uint8)
for i, c in enumerate(chunk):
rr, cc = divmod(i, COLS)
y0, x0 = rr * (cell_h + 26) + 22, cc * (CELL_W + 8) + 4
sheet[y0 : y0 + c.shape[0], x0 : x0 + c.shape[1]] = c
cv2.putText(sheet, f"{si + i:03d}", (x0, y0 - 6), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1)
cv2.imwrite(str(out / f"r{si // PER:02d}.png"), sheet)
with (out / "MANIFEST_DO_NOT_OPEN.csv").open("w", newline="") as fh:
w = csv.DictWriter(fh, fieldnames=list(manifest[0]))
w.writeheader()
w.writerows(manifest)
print(f"cells={len(cells)} sheets={(len(cells) + PER - 1) // PER} -> {out}")
print("each cell = TOP-LEFT corner above, BOTTOM-RIGHT corner below, near native scale")
if __name__ == "__main__":
main()
+101
View File
@@ -0,0 +1,101 @@
"""Build BLIND contact sheets for hand-labelling relaxation additions.
Crops are centered on the DETECTED REGION (not the corner), padded by ~0.9x the mark
size, and resized to 240px with INTER_NEAREST -- a downscaled preview destroys a faint
mark, so nothing here may smooth. The manifest is written to a separate file that must
NOT be read until labelling is finished.
Each sheet mixes three strata in shuffled order:
add - the relaxation additions whose precision we are measuring
pos - strict-consistent detections (a mark is really there): labeller sensitivity
clean - verified-clean negatives (no mark can be there): labeller specificity
The two control strata are what make a low measured precision trustworthy.
"""
import csv
import json
import random
import sys
from pathlib import Path
from typing import Any
import cv2
import numpy as np
from numpy.typing import NDArray
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from remove_ai_watermarks import watermark_registry as wr
from remove_ai_watermarks.image_io import imread
CELL = 240
COLS, ROWS = 6, 3
PER = COLS * ROWS
def region_for(path: str, key: str) -> tuple[int, int, int, int] | None:
"""Re-detect to recover the mark bbox (Candidate carries no region)."""
img = imread(path)
if img is None:
return None
mark = next(m for m in wr._REGISTRY if m.key == key)
d = mark.detect(img, provenance=True)
return d.region if d.region else None
def crop(path: str, region: tuple[int, int, int, int] | None, pad_factor: float = 0.9) -> NDArray[Any] | None:
img = imread(path)
if img is None:
return None
h, w = img.shape[:2]
if region:
x, y, rw, rh = region
else:
return None
px, py = int(rw * pad_factor), int(rh * pad_factor)
x0, y0 = max(0, x - px), max(0, y - py)
x1, y1 = min(w, x + rw + px), min(h, y + rh + py)
c = img[y0:y1, x0:x1]
if c.size == 0:
return None
s = CELL / max(c.shape[0], c.shape[1])
return cv2.resize(c, (max(1, int(c.shape[1] * s)), max(1, int(c.shape[0] * s))), interpolation=cv2.INTER_NEAREST)
def main() -> None:
with open(sys.argv[1]) as fh:
items = json.load(fh) # [{uid,path,key,stratum,conf}]
outdir = Path(sys.argv[2])
outdir.mkdir(parents=True, exist_ok=True)
random.Random(1234).shuffle(items) # noqa: S311 -- sheet ordering, not cryptography
manifest = []
cells = []
for it in items:
reg = region_for(it["path"], it["key"])
c = crop(it["path"], reg)
if c is None:
continue
cells.append((it, c))
for si in range(0, len(cells), PER):
chunk = cells[si : si + PER]
sheet = np.full((ROWS * (CELL + 26), COLS * (CELL + 8), 3), 40, np.uint8)
for i, (it, c) in enumerate(chunk):
r, col = divmod(i, COLS)
y0 = r * (CELL + 26) + 22
x0 = col * (CELL + 8) + 4
sheet[y0 : y0 + c.shape[0], x0 : x0 + c.shape[1]] = c
label = f"{si + i:04d}" # index ONLY -- no stratum, no confidence
cv2.putText(sheet, label, (x0, y0 - 6), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1)
manifest.append({"idx": si + i, **it})
cv2.imwrite(str(outdir / f"sheet_{si // PER:03d}.png"), sheet)
with open(outdir / "MANIFEST_DO_NOT_OPEN.csv", "w", newline="") as fh:
w = csv.DictWriter(fh, fieldnames=list(manifest[0]))
w.writeheader()
w.writerows(manifest)
print(f"sheets={(len(cells) + PER - 1) // PER} cells={len(cells)} -> {outdir}")
if __name__ == "__main__":
main()
+248 -23
View File
@@ -1,8 +1,8 @@
"""Shared base for the visible text-mark detectors/localizers (localize -> fill).
The Doubao "豆包AI生成", Jimeng "★ 即梦AI", and Samsung "✦ Contenuti generati
dall'AI" marks are the SAME algorithm: anchor a bottom-corner box by width-relative
geometry, extract the light low-saturation glyph candidate (white top-hat), detect
dall'AI" marks are the SAME algorithm: anchor a bottom-corner box by geometry
relative to the image's SHORT side, extract the light low-saturation glyph candidate (white top-hat), detect
by matching the bundled alpha-glyph silhouette via ``TM_CCOEFF_NORMED``, and build a
removal MASK from the glyph blob's bounding box (:meth:`footprint_mask`) for the
shared fill (region_eraser). The mask is template-FREE -- the top-hat glyph bbox, not
@@ -54,10 +54,41 @@ _MIN_DETECT_SHORT_SIDE = 200
# Provenance-confirmed NCC relaxation. When external metadata already confirms the
# vendor (so the mark is present with high prior), a faint or slightly re-rendered
# glyph that scores just below the standard NCC gate is still trusted. 0.7 recovers
# the near-threshold marks without dropping so low that an unrelated corner texture
# on a (provenance-confirmed) image would match -- the coverage gate still applies.
_PROVENANCE_NCC_FACTOR = 0.7
# glyph that scores just below the standard NCC gate is still trusted. The relaxed
# gate is ``detect_ncc_threshold * provenance_ncc_factor``; the coverage gate still
# applies on top.
#
# This used to be ONE shared 0.7 for every text mark. Measured 2026-07-18 on the
# `auto` path (the default -- no flag, driven by TC260 metadata), it turned out to
# mean two completely different things per mark. Blind hand-label of the ADDITIONS
# (accepted with provenance, rejected without) over 4417 unique TC260 carriers,
# two-sided control (labeller sensitivity 100%/96%, specificity 100%/100%):
#
# mark band precision 95% CI n
# doubao whole arm 76% 61-87% 42
# [0.280,0.340) 58% 36-77% 19
# [0.340,0.400) 91% 73-98% 23
# jimeng whole arm 17% 10-27% 82
# [0.315,0.383) 12% 6-22% 68
# [0.383,0.450) 43% 21-67% 14
#
# Doubao stays at 0.70: both its bands return more true marks than false fills, so
# tightening would cost 11 genuine recoveries to prevent 8 false ones.
#
# Jimeng moves to 0.85. Its relaxed detector does not key on the "★ 即梦AI" wordmark
# any more -- it keys on "some text in the bottom-right corner": of 68 false
# additions, 33 were DOUBAO marks and 17 were other vendors' AI labels (千问, 百度,
# 星绘, 抖音). 45 of those 68 fill a corner nothing else would touch (the other 23
# are harmless -- doubao fires strictly there and fills the same box anyway). At
# 0.85 the [0.315,0.383) band is dropped: 8 genuine recoveries lost, 60 false fills
# prevented (7.5:1). A false fill is the worse error -- it destroys pixels AND makes
# the caller report a removal that did not happen, while a miss leaves the image
# untouched.
#
# NOTE: 0.85 is a patch on a detector problem, not a fix. Jimeng's silhouette is not
# discriminative against Doubao's (same corner, same script, both ByteDance), and no
# threshold repairs that -- it needs a better detection silhouette.
_DEFAULT_PROVENANCE_NCC_FACTOR = 0.7
@dataclass(frozen=True)
@@ -68,7 +99,7 @@ class TextMarkConfig:
asset_name: str # bundled alpha PNG under assets/ (e.g. "doubao_alpha.png")
corner: Literal["br", "bl"] # bottom-right (Doubao/Jimeng) or bottom-left (Samsung)
margin_floor: int # min margin in px for locate (4 for br marks, 2 for Samsung)
# locate geometry (fraction of image WIDTH)
# locate geometry (fraction of scale_base -- see scale_base())
width_frac: float
height_frac: float
margin_x_frac: float # right margin (br) or left margin (bl)
@@ -81,12 +112,31 @@ class TextMarkConfig:
# detection
detect_min_coverage: float
detect_ncc_threshold: float
# alpha-map glyph geometry (fraction of WIDTH) emitted by
# alpha-map glyph geometry (fraction of scale_base) emitted by
# scripts/visible_alpha_solve.py, sizing the detection silhouette for
# template_match_score
alpha_width_frac: float
alpha_height_frac: float
min_gw: int # minimum glyph width for the template match (8 br, 16 Samsung)
# Asset names of RIVAL marks that occupy the same corner and can therefore be
# scored against the same glyph blob. Detection becomes COMPETITIVE: this mark's
# template must beat every rival's by `rival_margin`. See _rival_margin_ok.
# Detection front-end. "binary" thresholds the top-hat into a glyph blob and
# correlates a binary silhouette against it; "tophat" correlates the CONTINUOUS
# top-hat response against a soft template and never binarizes. See
# TextMarkEngine.tophat_response for the measurement that motivated the split.
detect_frontend: Literal["binary", "tophat"] = "binary"
# Gaussian sigma applied to the template in the "tophat" front-end (0 = none).
template_blur: float = 0.0
# Which image dimension the mark's size and margins scale with. VENDOR-SPECIFIC,
# measured, not assumed -- see TextMarkEngine.scale_base. "short" = min(h, w), "width" = w.
scale_basis: Literal["short", "width"] = "width"
rivals: tuple[str, ...] = ()
rival_margin: float = 0.10
# Multiplier applied to detect_ncc_threshold when provenance confirms the vendor.
# Per-mark, NOT shared: see _DEFAULT_PROVENANCE_NCC_FACTOR for the measured
# precision that forced the split. Last field so it can carry a default.
provenance_ncc_factor: float = _DEFAULT_PROVENANCE_NCC_FACTOR
@dataclass
@@ -146,18 +196,49 @@ def glyph_silhouette(asset_name: str) -> NDArray[Any] | None:
return _silhouette_cache[asset_name]
def template_match_score(box_mask: NDArray[Any], image_width: int, config: TextMarkConfig) -> float:
_RIVAL_MODULES = {
"doubao_alpha.png": "remove_ai_watermarks.doubao_engine",
"jimeng_alpha.png": "remove_ai_watermarks.jimeng_engine",
"samsung_alpha.png": "remove_ai_watermarks.samsung_engine",
}
def _rival_config(asset_name: str, fallback: TextMarkConfig) -> TextMarkConfig:
"""The rival mark's own config, for scoring its template on a shared blob.
Looked up LAZILY by asset name: a rival's template geometry
(``alpha_*_frac`` / ``min_gw``) is its own, and scoring it with this mark's
geometry would compare a correctly-sized template against a mis-sized one and
hand the margin a free win. Lazy because the engine modules import this one.
"""
mod_path = _RIVAL_MODULES.get(asset_name)
if mod_path is None:
return fallback
from importlib import import_module
try:
return import_module(mod_path)._CONFIG
except Exception: # a missing/renamed engine must not break detection
logger.debug("rival config %s unavailable; skipping its margin check.", asset_name)
return fallback
def template_match_score(box_mask: NDArray[Any], scale_base: int, config: TextMarkConfig) -> float:
"""Zero-mean normalized correlation of the alpha-template glyph silhouette
(scaled to the mark's expected size) against the candidate ``box_mask``.
``TM_CCOEFF_NORMED`` keys on glyph SHAPE, not coverage, so a dense textured
corner does not score highly -- only the actual glyph shape does.
``scale_base`` is the mark's own scaling dimension (:meth:`TextMarkEngine.scale_base`),
not always the width: sizing the template on the wrong basis stretches it by the
aspect ratio on landscape inputs and the correlation collapses.
"""
sil = glyph_silhouette(config.asset_name)
if sil is None or box_mask.size == 0:
return 0.0
gw = min(box_mask.shape[1] - 1, max(config.min_gw, int(config.alpha_width_frac * image_width)))
gh = min(box_mask.shape[0] - 1, max(4, int(config.alpha_height_frac * image_width)))
gw = min(box_mask.shape[1] - 1, max(config.min_gw, int(config.alpha_width_frac * scale_base)))
gh = min(box_mask.shape[0] - 1, max(4, int(config.alpha_height_frac * scale_base)))
if gw < config.min_gw or gh < 4:
return 0.0
template = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_NEAREST)
@@ -178,19 +259,153 @@ class TextMarkEngine:
def _glyph_silhouette(self) -> NDArray[Any] | None:
return glyph_silhouette(self.config.asset_name)
def _template_match_score(self, box_mask: NDArray[Any], image_width: int) -> float:
return template_match_score(box_mask, image_width, self.config)
def _template_match_score(self, box_mask: NDArray[Any], scale_base: int) -> float:
return template_match_score(box_mask, scale_base, self.config)
def _rival_margin_ok(self, score: float, box_mask: NDArray[Any], scale_base: int) -> bool:
"""Whether this mark's template beats every same-corner RIVAL's on the SAME blob.
Detection was purely ABSOLUTE -- each engine scored its own template and
compared against its own threshold, so nothing ever asked the discriminative
question "does this blob look more like the neighbour's mark than like mine?".
Two marks sharing a corner and a script (Doubao "豆包AI生成" and Jimeng
"★ 即梦AI", both bottom-right, both near-white CJK) survive binarization into
very similar blobs, so an absolute gate cannot separate them -- and under the
provenance relaxation it stopped trying: 33 of jimeng's 68 false additions
were Doubao marks (corpus-measured 2026-07-18).
Measured separability on hand-labelled examples, scoring BOTH templates
against the same glyph blob (n=40 jimeng / 75 doubao / 20 other-vendor labels
/ 89 clean):
feature separability (0.5 = useless, 1.0 = perfect)
absolute ncc_jimeng 0.96
ncc_jimeng MINUS ncc_doubao 0.99
At a 0.10 margin: real Jimeng wordmarks pass 100%, Doubao strips 8%, other
vendors' AI labels (千问/百度/星绘/抖音) 55%, no-mark corners 12%. Because the
real marks pass at 100%, this costs NO recall -- it is a pure precision gain,
unlike raising the threshold, which trades recall away.
Marks with no same-corner rival declare `rivals=()` and are unaffected.
"""
c = self.config
if not c.rivals:
return True
for rival_asset in c.rivals:
rival = _rival_config(rival_asset, c)
if score - template_match_score(box_mask, scale_base, rival) < c.rival_margin:
logger.debug("%s detect: loses the %s rival margin; rejecting.", c.name, rival_asset)
return False
return True
# ── Locate ──────────────────────────────────────────────────────────
def tophat_response(self, image: NDArray[Any], loc: TextMarkLocation) -> NDArray[Any] | None:
"""The CONTINUOUS white top-hat in the located box -- the glyph signal, unbinarized.
:meth:`extract_mask` thresholds this same response into a 0/255 glyph blob. That
is fine for a mark stamped bold and opaque, and destructive for a faint one: a
thin translucent overlay shatters into specks under the threshold, and no
template can match a blob that is not there.
Measured 2026-07-18 on hand-verified corpus positives (40 doubao, 14 千问, 60
verified-clean), scoring each mark with its own template:
front-end doubao clean neg AUC doubao/neg
binary 0.723 ~0.12 --
tophat 0.781 0.122 1.00
The gates that were hard cuts in the binary path (saturation, absolute luma)
become WEIGHTS here, so a faint stroke contributes in proportion to its strength
instead of being dropped at a threshold. The response is max-normalized, which
makes the score contrast-invariant -- the point of the exercise.
Kept per-mark (``detect_frontend``) rather than switched globally, because a
front-end change must be measured per mark before it ships.
"""
c = self.config
x, y, bw, bh = loc.bbox
if bh < 16 or bw < 16:
return None
roi = image_io.to_bgr(image[y : y + bh, x : x + bw]).astype(np.float32)
luma = roi.mean(axis=2)
sat = roi.max(axis=2) - roi.min(axis=2)
sigma = max(4.0, bh * 0.4)
tophat = luma - cv2.GaussianBlur(luma, (0, 0), sigmaX=sigma, sigmaY=sigma)
resp = np.clip(tophat, 0, None) * (sat < c.max_saturation)
peak = float(resp.max())
if peak <= 1e-6:
return None
return (resp / peak * 255).astype(np.uint8)
def _tophat_score(self, image: NDArray[Any], loc: TextMarkLocation) -> float:
"""TM_CCOEFF_NORMED of a soft template against the continuous response.
Sweeps a small scale band: the nominal glyph size is derived from the mark's
geometry, but a vendor re-rasterization shifts it by a few percent and the
continuous response is sharp enough that an exact-size template would miss.
"""
c = self.config
resp = self.tophat_response(image, loc)
sil = self._glyph_silhouette()
if resp is None or sil is None:
return 0.0
base = self.scale_base(image)
best = 0.0
for scale in (0.8, 1.0, 1.25):
gw = max(c.min_gw, int(c.alpha_width_frac * base * scale))
gh = max(4, int(c.alpha_height_frac * base * scale))
if gw >= resp.shape[1] or gh >= resp.shape[0]:
continue
tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32)
if c.template_blur > 0:
tmpl = cv2.GaussianBlur(tmpl, (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur)
best = max(best, float(cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED).max()))
return best
def scale_base(self, image: NDArray[Any]) -> int:
"""The image dimension this mark's geometry scales with.
Per-mark, and MEASURED -- a single shared basis is wrong. The tuned fractions
were all calibrated on PORTRAIT captures, where width and short side coincide,
so the basis was never exercised until landscape inputs were measured.
Corpus-measured 2026-07-18 (2572 unique TC260 carriers; the harness is
`scripts/visible_eval.py`). Before any fix, doubao detection by aspect ratio:
portrait 60%, square 41%, **landscape 0% (0 of 435)** -- the width-scaled box
is inflated by the aspect ratio on a wide image and the glyph never lands in
it. Re-running the previously-undetected set with a short-side basis recovered
**56% of landscape** images (12% square, 4% portrait).
But the same switch took JIMENG's labelled landscape positives from 13/13 to
0/13: its wordmark tracks the WIDTH. Both marks are ByteDance and share a
corner, and they still scale differently -- so this is a per-mark measurement,
not a house rule to generalize. Samsung keeps ``width`` because there is no
corpus evidence either way (1 addition corpus-wide) and an unmeasured change
is not an improvement.
China's GB 45438-2025 clause 5.2(e) mandates glyph height >= 5% of "the
shortest side" for CN marks, which is why a short-side basis is the natural
prior -- but Jimeng's measured behaviour overrides the prior, and measurement
wins over the standard's wording.
"""
return min(image.shape[:2]) if self.config.scale_basis == "short" else image.shape[1]
def locate(self, image: NDArray[Any]) -> TextMarkLocation:
"""Anchor the watermark box in the configured bottom corner by geometry."""
"""Anchor the watermark box in the configured corner, scaled by ``scale_basis``.
Every fraction is taken against ``scale_base(image)`` -- see
:data:`TextMarkConfig.scale_basis`, which is per-mark because the vendors
genuinely differ.
"""
c = self.config
h, w = image.shape[:2]
wm_w = max(40, int(w * c.width_frac))
wm_h = max(16, int(w * c.height_frac))
margin_x = max(c.margin_floor, int(w * c.margin_x_frac))
margin_b = max(c.margin_floor, int(w * c.margin_bottom_frac))
base = self.scale_base(image)
wm_w = max(40, int(base * c.width_frac))
wm_h = max(16, int(base * c.height_frac))
margin_x = max(c.margin_floor, int(base * c.margin_x_frac))
margin_b = max(c.margin_floor, int(base * c.margin_bottom_frac))
x = max(0, w - margin_x - wm_w) if c.corner == "br" else min(margin_x, max(0, w - wm_w))
y = max(0, h - margin_b - wm_h)
wm_w = min(wm_w, w - x)
@@ -251,7 +466,8 @@ class TextMarkEngine:
(China-AIGC / byteimg for Doubao/Jimeng, ``samsung_genai`` for Samsung); the
NCC gate exists to keep a corner texture on an UNRELATED image from matching
the glyph silhouette, so when provenance confirms the vendor it is relaxed by
``_PROVENANCE_NCC_FACTOR`` to recover a faint or slightly re-rendered mark.
the mark's own ``provenance_ncc_factor`` to recover a faint or slightly
re-rendered mark (per-mark, not shared -- see _DEFAULT_PROVENANCE_NCC_FACTOR).
"""
c = self.config
det = TextMarkDetection()
@@ -274,11 +490,20 @@ class TextMarkEngine:
coverage = float((box > 0).sum()) / float(max(1, bw * bh))
det.region = loc.bbox
det.coverage = coverage
if coverage >= c.detect_min_coverage:
score = self._template_match_score(box, image.shape[1])
threshold = c.detect_ncc_threshold * (_PROVENANCE_NCC_FACTOR if provenance else 1.0)
if c.detect_frontend == "tophat":
# The continuous front-end does not depend on the binarized blob, so the
# coverage gate (a blob-area heuristic) does not apply to it.
score = self._tophat_score(image, loc)
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
det.confidence = score
det.detected = score >= threshold
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
logger.debug("%s detect (tophat): ncc=%.2f thr=%.2f detected=%s", c.name, score, threshold, det.detected)
return det
if coverage >= c.detect_min_coverage:
score = self._template_match_score(box, self.scale_base(image))
threshold = c.detect_ncc_threshold * (c.provenance_ncc_factor if provenance else 1.0)
det.confidence = score
det.detected = score >= threshold and self._rival_margin_ok(score, box, self.scale_base(image))
logger.debug(
"%s detect: coverage=%.3f ncc=%.2f thr=%.2f detected=%s",
c.name,
+5 -4
View File
@@ -8,7 +8,7 @@ up metadata provenance, or preserve the alpha channel by hand:
import remove_ai_watermarks as raiw
raiw.remove_visible("in.png", "out.png") # path -> file, provenance auto
result, removed = raiw.remove_visible(bgr_array) # array -> array
raiw.remove_visible("shot.png", "out.png", sensitivity="assume_ai")
raiw.remove_visible("shot.png", "out.png", sensitivity="strict")
raiw.visible_provenance("in.png") # -> frozenset({"gemini"})
Imports stay lazy (inside the functions), so ``import remove_ai_watermarks`` is cheap.
@@ -134,10 +134,8 @@ def remove_visible(
array is always returned as well, so an empty ``removed`` list tells a caller nothing
known was found (e.g. route to the diffusion ``all`` path or ``erase``).
``sensitivity`` (``auto``/``strict``/``assume_ai``) and ``backend``
``sensitivity`` (``auto``/``strict``) and ``backend``
(``auto``/``cv2``/``migan``/``lama``) are the same knobs as the CLI. Pass
``sensitivity="assume_ai"`` for a metadata-stripped screenshot the caller knows is
AI-generated (best recall, at the cost of a small near-lossless fill on a clean
corner if the guess is wrong).
``strip_metadata`` (default True, matching the CLI ``visible --strip-metadata``)
@@ -152,6 +150,9 @@ def remove_visible(
"""
from remove_ai_watermarks import watermark_registry
# Reject a removed sensitivity loudly; `Sensitivity` is a Literal and not enforced
# at runtime, so a 0.15 caller would otherwise get `auto` behaviour in silence.
watermark_registry.validate_sensitivity(sensitivity)
loaded = _load_visible_input(source)
result, removed = watermark_registry.remove_auto_marks(
loaded.bgr,
+56 -33
View File
@@ -307,14 +307,13 @@ _visible_backend_option = click.option(
_visible_sensitivity_option = click.option(
"--sensitivity",
"sensitivity",
type=click.Choice(["auto", "strict", "assume-ai"]),
type=click.Choice(["auto", "strict"]),
default="auto",
help="How hard to trust a borderline mark. auto: relax a mark only when metadata "
"or a same-product sibling mark corroborates it (safe; clean images untouched). "
"strict: high-precision visual gate only, never relaxed. assume-ai: treat the "
"image as AI and relax every mark, keeping a confidence floor where the vendor is "
"unconfirmed (best recall on metadata-stripped screenshots; a clean image is still "
"left untouched).",
"strict: high-precision visual gate only, never relaxed. To act on a mark YOU can "
"see but the detector missed, use 'erase --region' or '--mark <name> --no-detect' "
"rather than a blanket relaxation.",
)
@@ -396,13 +395,12 @@ def _remove_visible_auto(
def _parse_sensitivity(value: str) -> watermark_registry.Sensitivity:
"""Map the CLI ``--sensitivity`` choice (kebab ``assume-ai``) to the registry
literal (``assume_ai``); ``auto``/``strict`` pass through unchanged."""
if value == "assume-ai":
return "assume_ai"
if value == "strict":
return "strict"
return "auto"
"""Map the CLI ``--sensitivity`` choice to the registry literal.
A pass-through since ``assume-ai`` was removed (2026-07-19); kept as the single
conversion point so a future kebab-cased choice has an obvious home.
"""
return "strict" if value == "strict" else "auto"
# Exit code for the standalone ``visible`` command when no visible mark was
@@ -412,7 +410,7 @@ def _parse_sensitivity(value: str) -> watermark_registry.Sensitivity:
EXIT_NO_VISIBLE_MARK = 2
def _no_visible_mark_exit(source: Path, *, sensitivity: str = "auto") -> NoReturn:
def _no_visible_mark_exit(source: Path) -> NoReturn:
"""Explain why no visible watermark was removed, then exit non-zero.
The visible registry handles only known visual marks (the Gemini sparkle and
@@ -423,10 +421,25 @@ def _no_visible_mark_exit(source: Path, *, sensitivity: str = "auto") -> NoRetur
watermarked image -- the recurring "it didn't work" report. Instead, run a
cheap metadata-only :func:`identify`, tell the user what the image actually
carries and which command removes it, and exit
:data:`EXIT_NO_VISIBLE_MARK`. When the conservative detector found nothing and
the user has NOT already asked for the aggressive pass, point them at
``--sensitivity assume-ai`` (a faint or moved visible mark may be sitting just
below the default gate).
:data:`EXIT_NO_VISIBLE_MARK`.
When the user can SEE a mark the detector missed, the honest next step is one that
executes their instruction rather than guessing harder. This used to recommend
``--sensitivity assume-ai``, which did the opposite -- it relaxed every mark's gate
on a blanket assumption -- and that mode is gone (2026-07-19).
The advice is per-mark, because the forced paths are not equally reliable
(measured 2026-07-19):
* ``erase --region`` is always sound: the user supplies the coordinates, so there
is nothing to guess. This is the primary recommendation.
* ``--mark <text-mark> --no-detect`` is reasonable for the TEXT marks: the forced
mask is built from the actual glyph blob, non-empty on 13/13 real marks the
detector missed.
* ``--mark gemini --no-detect`` is NOT recommended and is deliberately not
suggested here: with no detection it falls back to a fixed default sparkle slot,
which covered the real sparkle on only **31% of 97** genuine sparkles the strict
gate missed (median offset 63px up-and-left). The other 69% fill a clean corner
AND report a removal that did not happen -- the worst outcome the tool has.
"""
from remove_ai_watermarks.identify import identify
@@ -448,12 +461,13 @@ def _no_visible_mark_exit(source: Path, *, sensitivity: str = "auto") -> NoRetur
" If instead there is a logo or object to remove, target it with the region eraser:\n"
f" remove-ai-watermarks erase {source.name} --region x,y,w,h"
)
if sensitivity != "assume_ai":
console.print(
" If you know this image is AI-generated, retry the visible pass with\n"
f" remove-ai-watermarks visible {source.name} --sensitivity assume-ai\n"
" which relaxes detection to catch a faint or moved mark the default gate skips."
)
console.print(
" If you can SEE a mark here that was not detected, point at it directly --\n"
" that removes what you actually see instead of guessing:\n"
f" remove-ai-watermarks erase {source.name} --region x,y,w,h\n"
" For a known CJK text mark you can also force it by name:\n"
f" remove-ai-watermarks visible {source.name} --mark doubao --no-detect"
)
raise SystemExit(EXIT_NO_VISIBLE_MARK)
@@ -562,7 +576,7 @@ def _run_visible_auto(
if not removed:
# write_noop=False means nothing was written, so a pre-existing output is intact.
console.print(" No known visible mark detected (gemini / doubao / jimeng / jimeng-pill / samsung).")
_no_visible_mark_exit(source, sensitivity=sensitivity)
_no_visible_mark_exit(source)
console.print(f" Removed: {', '.join(removed)}")
size_kb = output.stat().st_size / 1024
console.print(f" Saved: {output} ({size_kb:.0f} KB, {elapsed:.2f}s)")
@@ -592,7 +606,7 @@ def _run_visible_explicit(
target = "gemini" if mark == "auto" else mark # --no-detect auto: gemini fallback
chosen = watermark_registry.get_mark(target)
# A single explicit mark has no sibling corroboration. Keep its trust resolution
# aligned with the registry arbiter, including the assumption-only floor.
# aligned with the registry arbiter.
trust = watermark_registry.resolve_trust(
chosen.key,
sensitivity=sensitivity,
@@ -601,12 +615,9 @@ def _run_visible_explicit(
)
relax = trust != "strict"
detection = chosen.detect(image, provenance=relax)
if trust == "assumed" and not watermark_registry.assumed_floor_ok(chosen.key, detection.confidence):
relax = False
detection = chosen.detect(image, provenance=False)
if detect and not detection.detected:
console.print(f" {chosen.label} not detected (conf {detection.confidence:.2f}). Use --no-detect to force.")
_no_visible_mark_exit(source, sensitivity=sensitivity)
_no_visible_mark_exit(source)
if detection.detected:
console.print(f" {chosen.label} detected ({chosen.location}, conf {detection.confidence:.2f})")
@@ -954,7 +965,7 @@ def cmd_metadata(
from WebM/MP3/WAV/FLAC/OGG. The coded image, audio, and video data are left
untouched.
"""
from remove_ai_watermarks.metadata import get_ai_metadata, has_ai_metadata, remove_ai_metadata
from remove_ai_watermarks.metadata import get_ai_metadata, has_ai_metadata, strip_and_verify
# No _validate_image() here: unlike the image-only commands, metadata also
# accepts video/audio containers, so the image-format warning would misfire.
@@ -982,10 +993,16 @@ def cmd_metadata(
# Remove
try:
out = remove_ai_metadata(source, output, keep_standard=keep_standard)
out, leftover = strip_and_verify(source, output, keep_standard=keep_standard)
except (OSError, ValueError) as e: # unreadable / truncated / non-image (PIL raises OSError subclasses)
console.print(f" Error: cannot process {source.name}: {e}")
raise SystemExit(1) from e
if leftover:
console.print(f" FAILED: {len(leftover)} AI metadata marker(s) survived in {out}")
console.print(f" still present: {', '.join(sorted(leftover))}")
console.print(" the file could not be decoded, so it was copied through unchanged")
raise SystemExit(1)
console.print(f" AI metadata stripped -> {out}")
@@ -1448,9 +1465,15 @@ def _process_batch_image(
synthid_skipped = _run_batch_invisible(ctx, img_path, out_path, mode, options)
if mode in ("metadata", "all"):
from remove_ai_watermarks.metadata import remove_ai_metadata
from remove_ai_watermarks.metadata import strip_and_verify
remove_ai_metadata(img_path if mode == "metadata" else out_path, out_path)
# Same verification the single-image command does: the fail-safe copy-through
# would otherwise leave an AI-reading output and still exit 0, contradicting the
# batch contract that a failed image must make the run exit non-zero.
_, leftover = strip_and_verify(img_path if mode == "metadata" else out_path, out_path)
if leftover:
msg = f"AI metadata survived the strip ({', '.join(sorted(leftover))}); file could not be decoded"
raise RuntimeError(msg)
# In "all" mode, the invisible step (color-only OpenCV paths) drops alpha,
# so re-attach the cached alpha when the input had transparency.
+23 -3
View File
@@ -47,7 +47,21 @@ TOPHAT_DELTA = 12 # glyph must exceed the local background by this many levels
# Shape-consistent detection: match the bundled alpha glyph silhouette against the
# corner candidate via TM_CCOEFF_NORMED (keys on glyph SHAPE, not coverage; #23).
DETECT_MIN_COVERAGE = 0.04
DETECT_NCC_THRESHOLD = 0.4
# NOTE: this gate is FRONT-END SPECIFIC. The continuous top-hat front-end scores higher
# overall than the binary one (mean 0.809 vs 0.723 on the same 90 positives), so the
# binary-era 0.40 left the provenance-relaxed gate (x0.7) far too low and admitted false
# fires. Calibrated on the 240-image unbiased recall sample, full auto path:
#
# gate relaxed recall precision true false
# 0.40 0.280 96% 91% 86 8
# 0.45 0.315 94% 93% 85 6
# 0.50 0.350 92% 99% 83 1 <- chosen
# 0.60 0.420 87% 99% 78 1
#
# 0.50 beats the binary front-end on recall (92% vs 89%) at identical precision (99%),
# which is the only reason the front-end switch is worth it. Do not port this number to
# a binary-front-end mark; re-calibrate per front-end.
DETECT_NCC_THRESHOLD = 0.50
# Detection-silhouette geometry, emitted by scripts/visible_alpha_solve.py at the
# captured width. Sizes the glyph silhouette for the TM_CCOEFF_NORMED detection match
@@ -71,6 +85,12 @@ _CONFIG = TextMarkConfig(
morph_open_size=5,
detect_min_coverage=DETECT_MIN_COVERAGE,
detect_ncc_threshold=DETECT_NCC_THRESHOLD,
detect_frontend="tophat",
scale_basis="short", # measured: recovers 56% of landscape misses (see scale_base)
# No rival margin: measured 2026-07-18, the symmetric gate cost Doubao 7 genuine
# detections to prevent 5 false ones (1.4:1 against). Doubao's absolute detector
# is already 86% precise, so it has nothing to buy; Jimeng's is 38% and gains 25pp
# for free. The confusion is asymmetric, so the remedy is too.
alpha_width_frac=_ALPHA_WIDTH_FRAC,
alpha_height_frac=_ALPHA_HEIGHT_FRAC,
min_gw=8,
@@ -90,9 +110,9 @@ def _glyph_silhouette() -> NDArray[Any] | None:
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
def _template_match_score(box_mask: NDArray[Any], image_width: int) -> float:
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
"""TM_CCOEFF_NORMED of the Doubao glyph silhouette against ``box_mask``."""
return _text_mark_engine.template_match_score(box_mask, image_width, _CONFIG)
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
class DoubaoEngine(TextMarkEngine):
+16 -2
View File
@@ -42,9 +42,21 @@ TOPHAT_DELTA = 12
# Shape-consistent detection. Threshold 0.45 cleanly separates real Jimeng marks
# (>=0.81) from the Doubao strip (0.21), so the two ByteDance marks do not cross-fire.
#
# That separation holds at the STRICT threshold ONLY. Relaxed under provenance it
# collapses: at the old shared 0.7 factor (gate 0.315) the arm ran at 17% precision
# and 33 of its 68 false additions were Doubao marks (corpus-measured 2026-07-18 --
# full table at _DEFAULT_PROVENANCE_NCC_FACTOR). That was patched with a tighter 0.85
# factor at the time; the competitive rival margin replaced it -- see below.
DETECT_MIN_COVERAGE = 0.02
DETECT_NCC_THRESHOLD = 0.45
# Back to the 0.70 default. The 0.85 patch (2026-07-18) existed only to blunt the
# Doubao cross-fire by sacrificing recall; the competitive RIVAL MARGIN below
# discriminates directly and passes real wordmarks at 100%, so the recall the patch
# gave up is no longer the price of precision. See _rival_margin_ok.
PROVENANCE_NCC_FACTOR = 0.7
# Detection-silhouette geometry, emitted by scripts/visible_alpha_solve.py from the
# gray capture at the captured width (sizes the silhouette for the detection match;
# removal is the template-free glyph-bbox footprint mask).
@@ -67,6 +79,8 @@ _CONFIG = TextMarkConfig(
morph_open_size=5,
detect_min_coverage=DETECT_MIN_COVERAGE,
detect_ncc_threshold=DETECT_NCC_THRESHOLD,
provenance_ncc_factor=PROVENANCE_NCC_FACTOR,
rivals=("doubao_alpha.png",),
alpha_width_frac=_ALPHA_WIDTH_FRAC,
alpha_height_frac=_ALPHA_HEIGHT_FRAC,
min_gw=8,
@@ -85,9 +99,9 @@ def _glyph_silhouette() -> NDArray[Any] | None:
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
def _template_match_score(box_mask: NDArray[Any], image_width: int) -> float:
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
"""TM_CCOEFF_NORMED of the Jimeng glyph silhouette against ``box_mask``."""
return _text_mark_engine.template_match_score(box_mask, image_width, _CONFIG)
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
class JimengEngine(TextMarkEngine):
+21
View File
@@ -1076,6 +1076,27 @@ def _sniff_image_format(head: bytes) -> str | None:
return None
def strip_and_verify(
source_path: Path,
output_path: Path | None = None,
*,
keep_standard: bool = True,
) -> tuple[Path, dict[str, str]]:
"""Strip AI metadata, then RE-SCAN the output and report what survived.
:func:`remove_ai_metadata` is deliberately fail-safe: a file PIL cannot decode is
copied through UNCHANGED rather than crashing a caller, and the path it returns is
indistinguishable from a real strip. Any caller that reports an outcome to a user
therefore cannot tell a no-op from a success -- corpus-observed on real Samsung
Galaxy S22 C2PA PNGs, where `metadata --remove` printed "stripped" and exited 0 while
the output still read as AI (2026-07-19 parity audit).
Returns ``(output_path, surviving_markers)``; an empty mapping means a real strip.
"""
out = remove_ai_metadata(source_path, output_path, keep_standard=keep_standard)
return out, get_ai_metadata(out)
def remove_ai_metadata(
source_path: Path,
output_path: Path | None = None,
@@ -5,6 +5,7 @@ Pure configuration and lookup functions with no ML dependencies.
from __future__ import annotations
import math
from typing import TYPE_CHECKING, Literal
if TYPE_CHECKING:
@@ -137,6 +138,30 @@ def resolve_strength(strength: float | None, vendor: str | None = None, pipeline
return _VENDOR_STRENGTH.get(vendor or "", UNKNOWN_STRENGTH)
def viable_steps(num_inference_steps: int, strength: float) -> int:
"""The smallest step count >= ``num_inference_steps`` that actually denoises.
diffusers derives its img2img timesteps as ``int(steps * strength)``. When that
rounds to ZERO the pipeline builds an empty latent and dies deep inside attention
with ``cannot reshape tensor of 0 elements into shape [0, -1, 1, 512]`` -- an opaque
torch error for what is really "these two options cannot work together".
The combination is reachable with entirely valid CLI arguments: at the default
strength 0.15 every ``--steps`` below 7 crashed, and nothing told the user that
``--steps`` and ``--strength`` interact (found by the release smoke matrix,
2026-07-19). Raising the count to the minimum that denoises keeps the caller's intent
-- they asked for "few steps", not "zero" -- and the engine logs the adjustment.
A non-positive ``strength`` cannot denoise at any step count; return the caller's
value unchanged rather than dividing by zero.
"""
if strength <= 0:
return num_inference_steps
if int(num_inference_steps * strength) >= 1:
return num_inference_steps
return math.ceil(1 / strength)
def vendor_for_strength(image_path: Path) -> Literal["openai", "google"] | None:
"""Detect the SynthID vendor for strength selection: ``"openai"`` / ``"google"`` / None.
@@ -49,6 +49,7 @@ from remove_ai_watermarks.noai.watermark_profiles import (
QWEN_MODEL_ID,
normalize_profile,
resolve_strength,
viable_steps,
)
logger = logging.getLogger(__name__)
@@ -690,6 +691,20 @@ class WatermarkRemover:
self._set_progress(f"Setting reproducible seed: {seed}")
generator = _make_seed_generator(self.device, seed)
# A step count whose product with strength rounds to zero kills the pipeline
# inside attention with an opaque reshape error, so raise it to the minimum that
# denoises. Must be applied to the value HANDED TO THE PIPELINE -- the old
# max(1, ...) below only clamped the number in the log line.
adjusted = viable_steps(num_inference_steps, strength)
if adjusted != num_inference_steps:
logger.warning(
"steps=%s at strength=%s denoises 0 steps and would crash; using steps=%s (1 effective)",
num_inference_steps,
strength,
adjusted,
)
num_inference_steps = adjusted
effective_steps = max(1, int(num_inference_steps * strength))
self._set_progress(
f"Config: strength={strength}, steps={num_inference_steps} "
+2 -2
View File
@@ -92,9 +92,9 @@ def _glyph_silhouette() -> NDArray[Any] | None:
return _text_mark_engine.glyph_silhouette(_CONFIG.asset_name)
def _template_match_score(box_mask: NDArray[Any], image_width: int) -> float:
def _template_match_score(box_mask: NDArray[Any], scale_base: int) -> float:
"""TM_CCOEFF_NORMED of the Samsung glyph silhouette against ``box_mask``."""
return _text_mark_engine.template_match_score(box_mask, image_width, _CONFIG)
return _text_mark_engine.template_match_score(box_mask, scale_base, _CONFIG)
class SamsungEngine(TextMarkEngine):
+127 -86
View File
@@ -52,21 +52,26 @@ Backend = Literal["auto", "cv2", "migan", "lama"]
# evidence the mark is there -- metadata provenance for that vendor, or a confidently
# detected sibling mark of the same product (see ``resolve_trust``). No evidence ->
# stays strict. Safe: it only escalates where the mark is corroborated.
# * ``assume_ai``: relax every mark's gate regardless of evidence -- the caller asserts
# the image is AI and wants the mark gone (e.g. a metadata-stripped screenshot uploaded
# to a watermark remover). Recovers the faint/moved marks the strict gate demotes. The
# library CANNOT infer this from a stripped image -- only the caller's out-of-band
# context (the user uploaded to remove a mark) justifies it. An assertion that the
# image is AI is NOT evidence of WHICH vendor made it, so a mark relaxed on assumption
# alone must still clear ``_ASSUMED_CONF_FLOOR``; see that constant for why.
Sensitivity = Literal["auto", "strict", "assume_ai"]
#
# REMOVED 2026-07-19: ``assume_ai`` relaxed every mark's gate on the caller's bare
# assertion that the image is AI. It was a statistical gamble, not an instruction:
# "this image is AI" says nothing about WHICH vendor or WHERE, which is exactly what a
# gate bypass needs, so it took a confidence floor to be tolerable at all (before that
# floor it filled a phantom sparkle on 59.8% of genuine camera photos). It also had no
# place in the product's own model -- detector finds a mark, remove it; detector finds
# nothing, leave the image alone; the USER sees a mark and says so, act on that. A user
# who can see the mark is better served by pointing at it (``erase --region``) or naming
# it (``--mark X --no-detect`` for a text mark), both of which execute an instruction
# instead of guessing. Removing it also collapsed the trust ladder from three levels to
# two. See docs/module-internals.md for the measurements.
Sensitivity = Literal["auto", "strict"]
# The trust level a mark's detection gate is resolved to (see ``resolve_trust``). The
# split between ``assumed`` and ``confirmed`` is load-bearing: both bypass the engine's
# false-positive gate, but only ``confirmed`` has evidence naming THIS vendor, which is
# exactly what that bypass is documented to require (see GeminiEngine.detect_watermark's
# ``trust_provenance`` contract). ``assumed`` therefore carries a confidence floor.
Trust = Literal["strict", "assumed", "confirmed"]
# The trust level a mark's detection gate is resolved to (see ``resolve_trust``).
# ``confirmed`` bypasses the engine's false-positive gate, and that bypass is documented
# to require evidence naming THIS vendor (see GeminiEngine.detect_watermark's
# ``trust_provenance`` contract) -- so it is only ever reached from same-product evidence.
# A third ``assumed`` level existed for ``assume_ai`` and went with it (2026-07-19).
Trust = Literal["strict", "confirmed"]
# Product family per mark, for the ``auto`` cross-mark corroboration: a confidently
# detected mark relaxes only OTHER marks of the SAME product (different corners, one
@@ -82,6 +87,32 @@ _PRODUCT_OF: dict[str, str] = {
}
# Marks whose own detection is too weak to serve as EVIDENCE for a sibling of the
# same product, even though they share one. Sibling corroboration grants ``confirmed``
# trust, which bypasses the sibling's false-positive gate outright -- so a detector
# that false-fires often must not be able to hand that bypass to anyone.
#
# The pill qualifies on its own documented numbers: ~7% raw false-fire (5.5% measured
# on 578 vendor negatives, 2026-07-18). Letting it corroborate produced a closed loop
# on the DEFAULT auto path, no user flag involved:
# pill false-fires on a clean non-ByteDance image
# -> _PRODUCT_OF maps it to "jimeng", so jimeng resolves to `confirmed`
# -> jimeng's NCC gate drops 0.45 -> 0.3825 and it false-fires too
# -> _keep_pill now sees "jimeng" in keys and takes the WORDMARK arm, which
# removes the pill unrestricted -- skipping the flatness guard that exists
# precisely to stop the fill smearing a textured corner.
# Measured on the corpus: 3 of 578 negatives ran the full loop, one of them with
# footprint_flat=0 (the exact case the guard was written to block). Cutting the pill
# out of corroboration removed all 3 and cost NOTHING on 4417 TC260 carriers
# (jimeng fires 398 -> 398), so this is a defect fix, not a recall trade.
#
# `_keep_pill` already encodes the same distrust for the pill's own ACTION; this
# closes the gap that its TESTIMONY was never gated.
# Regression: tests/test_watermark_registry.py::TestArbiter::
# test_weak_pill_detection_does_not_confirm_the_jimeng_wordmark
_CANNOT_CORROBORATE: frozenset[str] = frozenset({"jimeng_pill"})
@dataclass(frozen=True)
class MarkDetection:
"""Uniform detection result for a known mark (across heterogeneous engines)."""
@@ -109,6 +140,32 @@ class Localization:
mask: NDArray[Any] | None
_REMOVED_SENSITIVITIES = {
"assume_ai": (
"sensitivity='assume_ai' was removed in 0.16: it relaxed EVERY mark's detection "
"gate on the bare assertion that an image is AI, which says nothing about which "
"vendor made it or where the mark is. If you can see a mark the detector missed, "
"act on what you see: erase(image, region=(x, y, w, h)), or the CLI "
"`--mark <name> --no-detect` for a known text mark. Use sensitivity='auto' for "
"the default evidence-driven behaviour."
)
}
def validate_sensitivity(value: str) -> Sensitivity:
"""Reject a removed sensitivity LOUDLY instead of silently falling back to ``auto``.
``Sensitivity`` is a ``Literal``, which is not enforced at runtime, so a caller
upgrading from 0.15 would pass ``"assume_ai"`` and quietly get ``auto`` behaviour --
a silent semantic change on the one release where they most need to be told.
"""
if value in _REMOVED_SENSITIVITIES:
raise ValueError(_REMOVED_SENSITIVITIES[value])
if value not in ("auto", "strict"):
raise ValueError(f"unknown sensitivity {value!r}; expected 'auto' or 'strict'")
return value # type: ignore[return-value]
@dataclass(frozen=True)
class Context:
"""The evidence + policy the removal arbiter decides against (perception is
@@ -120,6 +177,9 @@ class Context:
sensitivity: Sensitivity = "auto"
provenance: frozenset[str] = frozenset()
def __post_init__(self) -> None:
validate_sensitivity(self.sensitivity)
@dataclass(frozen=True)
class Candidate:
@@ -127,9 +187,8 @@ class Candidate:
Carries the mark's verdict at BOTH trust levels (``detected_strict`` = the
conservative gate, ``detected_relaxed`` = the gate the engine relaxes to under
provenance/assume), so the arbiter can pick per mark without re-running detection.
``relaxed_confidence`` is the gate-bypassed detection's confidence, which the arbiter
needs to apply :func:`assumed_floor_ok` when a mark is relaxed on assumption alone.
provenance), so the arbiter can pick per mark without re-running detection.
``features`` is a generic bag of physical measurements a mark's gate may need (the
mark owns which it reports via ``KnownMark._features``); e.g. the pill supplies
``footprint_flat`` (0/1). Empty for marks whose gate needs no extra evidence."""
@@ -138,7 +197,6 @@ class Candidate:
label: str
detected_strict: bool
detected_relaxed: bool
relaxed_confidence: float
features: dict[str, float] # generic; both construction sites always supply it (empty when none)
@@ -240,16 +298,41 @@ GEMINI_SPARKLE_TRUST_CONF = 0.5
_GEMINI_AUTO_MIN_CONF = GEMINI_SPARKLE_TRUST_CONF
# Provenance-confirmed Gemini trust gate. When external metadata already proves the
# image is a Google generation (C2PA issuer "Google"/"Gemini"), the [0.35, 0.5)
# band that the no-provenance gate leaves out is no longer ambiguous with Doubao
# text: a Doubao image carries ByteDance provenance, not Google, so it never reaches
# this relaxed gate. The vendor moving/re-rendering the sparkle (bigger, lighter,
# shifted north-west) drops a real sparkle into this band, and the fixed-slot
# detector demotes it -- provenance is exactly the extra evidence that lets us trust
# it. Set to the engine's own internal `detected` floor (0.35); combined with the
# engine's FP-gate being skipped under provenance (see gemini_engine), this recovers
# the moved-mark misses without touching the no-provenance precision.
_GEMINI_PROVENANCE_MIN_CONF = 0.35
# image is a Google generation (C2PA issuer "Google"/"Gemini"), the [gate, 0.5) band
# that the no-provenance gate leaves out is no longer ambiguous with Doubao text: a
# Doubao image carries ByteDance provenance, not Google, so it never reaches this
# relaxed gate. The vendor moving/re-rendering the sparkle (bigger, lighter, shifted
# north-west) drops a real sparkle into this band, and the fixed-slot detector demotes
# it -- provenance is exactly the extra evidence that lets us trust it.
#
# The gate was originally the engine's own `detected` floor (0.35). Raised to 0.42
# on 2026-07-18 after measuring what this arm actually admits, because the Doubao
# argument above -- while correct -- is not the binding constraint. Google C2PA is
# carried by Imagen, API generations and NotebookLM exports, none of which stamp a
# visible sparkle at all, so the relaxed gate spends most of its budget on images
# that never had a mark rather than on moved ones.
#
# Measured blind on 954 unique Google-metadata uploads (detector never saw the
# metadata), hand-labelled against a two-sided control (labeller sensitivity ~88%,
# specificity 100%). "Additions" = accepted with provenance but not without:
#
# band precision 95% CI population
# 0.35-0.42 13% 5-30% 120
# 0.42-0.46 35% 19-54% 47
# 0.46-0.50 27% 14-46% 44
# 0.50-0.54 40% 20-64% 15
#
# Precision is flat above 0.42 and collapses below it, and that bottom band alone is
# half the arm's volume -- so this is a step, not a gradient, and 0.42 is where it
# sits. Raising the gate here drops ~16 genuine recoveries to prevent ~104 false
# fills (6.5:1), cutting false fills from 18.7% to 7.8% of Google-metadata uploads.
# A false fill is the worse error: it destroys pixels AND makes the caller report a
# removal that did not happen, while a miss leaves the image untouched.
#
# NOTE: even at 0.42 this arm runs at ~33% precision (two false fills per genuine
# recovery). Whether an arm that inaccurate should exist at all is a product call,
# not a tuning one -- do not read this constant as "now correct".
_GEMINI_PROVENANCE_MIN_CONF = 0.42
# ── Engine adapters (lazy singletons; engines are cv2-only, no model load) ──
@@ -411,7 +494,7 @@ def _pill_mask(
def _pill_features(image: NDArray[Any]) -> dict[str, float]:
"""The pill's own gate feature: top-left footprint flatness (1.0 = flat enough for
an invisible fill), read by the metadata/assume arm of :func:`_keep_pill`."""
an invisible fill), read by the metadata arm of :func:`_keep_pill`."""
return {"footprint_flat": float(_engine("jimeng_pill").footprint_is_flat(image))}
@@ -457,36 +540,6 @@ def detect_marks(
return [m.detect(image, provenance=m.key in provenance) for m in _REGISTRY if include_explicit or m.in_auto]
# Minimum gate-bypassed confidence a mark must reach when it is relaxed on ASSUMPTION
# (``assume_ai``) rather than on evidence naming its vendor. Relaxing bypasses the
# engine's false-positive gate entirely, which is justified by vendor CONFIRMATION; an
# assumption that the image is AI says nothing about WHICH vendor, so the bypassed
# detector needs its own floor or it fires on ordinary content.
#
# Corpus-measured 2026-07-16 (256 genuine camera captures -- Make/Model/exposure/aperture
# present and no AI token, so a Gemini sparkle cannot be there -- vs 697 Google-C2PA
# positives, metadata used only as the label, never fed to the detector):
#
# bypassed threshold recall false-fire on clean photos
# 0.35 82.6% 59.8% <- the bare detector gate
# 0.45 66.6% 12.5%
# 0.50 59.4% 0.0% <- chosen
# strict gate 56.4% 0.0%
#
# So 0.35 sat on a cliff: it bought +26pp recall over strict by filling a corner on ~6
# of every 10 CLEAN photos. At 0.50 the flag is honest -- it still beats strict, for free.
# Marks absent from this dict relax identically at both levels; their bypassed false-fire
# on the same negatives is under 1% (doubao 0.8%, jimeng 0.4%, samsung 0.4%).
_ASSUMED_CONF_FLOOR: dict[str, float] = {"gemini": 0.50}
def assumed_floor_ok(key: str, confidence: float) -> bool:
"""Whether an ``assumed``-trust detection of ``key`` at ``confidence`` is trustworthy
enough to act on (see :data:`_ASSUMED_CONF_FLOOR`). Marks with no floor always pass."""
floor = _ASSUMED_CONF_FLOOR.get(key)
return floor is None or confidence >= floor
def resolve_trust(
key: str,
*,
@@ -500,19 +553,19 @@ def resolve_trust(
level (which the engines consume as ``provenance = level != "strict"``). ``strict``
never relaxes. A mark is ``confirmed`` only on same-product evidence -- the vendor
confirmed by metadata (``key in provenance``) or a confidently strict-detected
sibling of the same product (``_PRODUCT_OF``). Without that evidence, ``assume_ai``
yields ``assumed`` (relaxed, but subject to :func:`assumed_floor_ok`) and ``auto``
stays ``strict``."""
sibling of the same product (``_PRODUCT_OF``, minus the marks too weak to vouch,
:data:`_CANNOT_CORROBORATE`). Without that evidence a mark stays ``strict``: there is
no path that relaxes a gate on anything less than same-product evidence."""
if sensitivity == "strict":
return "strict"
product = _PRODUCT_OF[key]
confirmed = key in provenance or any(_PRODUCT_OF[k] == product for k in strict_keys if k != key)
if confirmed:
return "confirmed"
return "assumed" if sensitivity == "assume_ai" else "strict"
confirmed = key in provenance or any(
_PRODUCT_OF[k] == product for k in strict_keys if k != key and k not in _CANNOT_CORROBORATE
)
return "confirmed" if confirmed else "strict"
def _keep_pill(keys: set[str], *, provenance: frozenset[str], sensitivity: Sensitivity, footprint_flat: bool) -> bool:
def _keep_pill(keys: set[str], *, provenance: frozenset[str], footprint_flat: bool) -> bool:
"""Whether to auto-remove the capture-less 'AI生成' pill given the fired marks.
Pure decision (the flatness feature is precomputed at perception time and passed
@@ -522,11 +575,10 @@ def _keep_pill(keys: set[str], *, provenance: frozenset[str], sensitivity: Sensi
its false fires were textured ceilings/walls that the fill visibly SMEARS. Arms:
* bottom-right "★ 即梦AI" wordmark fired -> ~94% precise, and it survives
metadata-STRIPPED uploads: remove the pill unrestricted;
* TC260 metadata confirms Jimeng (``"jimeng" in provenance``, no wordmark) OR the
caller asserts AI (``sensitivity == "assume_ai"``) -> remove ONLY when the
* TC260 metadata confirms Jimeng (``"jimeng" in provenance``, no wordmark) -> remove ONLY when the
top-left footprint is flat enough for an invisible fill (``footprint_flat``),
so real flat-scene pills (and harmless flat false fires) are cleaned while the
damaging textured false fires are left untouched even under assume_ai.
damaging textured false fires are left untouched.
A Doubao image is TC260 too but is not Jimeng-basic, so the pill never rides on a
Doubao detection. No confirmation at all -> never remove (blocks false fires on
non-Jimeng content)."""
@@ -534,7 +586,7 @@ def _keep_pill(keys: set[str], *, provenance: frozenset[str], sensitivity: Sensi
return False
if "jimeng" in keys:
return True
if "jimeng" in provenance or sensitivity == "assume_ai":
if "jimeng" in provenance:
return footprint_flat
return False
@@ -557,7 +609,7 @@ def _build_candidates(image: NDArray[Any]) -> list[Candidate]:
strict = m.detect(image, provenance=False)
relaxed = m.detect(image, provenance=True)
feats = m.features(image) if (strict.detected or relaxed.detected) else {}
cands.append(Candidate(m.key, m.label, strict.detected, relaxed.detected, relaxed.confidence, feats))
cands.append(Candidate(m.key, m.label, strict.detected, relaxed.detected, feats))
return cands
@@ -566,9 +618,8 @@ def decide(candidates: list[Candidate], context: Context) -> list[Decision]:
ordered list of marks to remove (and the trust level each was accepted at).
All policy lives here, in one place: per-mark trust resolution (:func:`resolve_trust`,
which needs the strict-detected siblings for ``auto`` cross-mark corroboration), the
assumed-trust confidence floor (:func:`assumed_floor_ok`) and the capture-less pill
gate (:func:`_keep_pill`). No image, no I/O -- so it is unit-testable in isolation and
which needs the strict-detected siblings for ``auto`` cross-mark corroboration) and
the capture-less pill gate (:func:`_keep_pill`). No image, no I/O -- so it is unit-testable in isolation and
the same decision drives every caller."""
strict_keys = {c.key for c in candidates if c.detected_strict}
fired: list[Decision] = []
@@ -578,23 +629,13 @@ def decide(candidates: list[Candidate], context: Context) -> list[Decision]:
)
relax = trust != "strict"
ok = c.detected_relaxed if relax else c.detected_strict
if trust == "assumed" and not assumed_floor_ok(c.key, c.relaxed_confidence):
# Relaxed on assumption alone and too weak to trust: fall back to the strict
# verdict rather than dropping the mark, so assume_ai is monotonic -- it only
# ever ADDS recall over strict, never removes less than strict would.
ok, relax = c.detected_strict, False
if ok:
fired.append(Decision(c, relax))
keys = {d.candidate.key for d in fired}
if "jimeng_pill" in keys:
pill = next(d for d in fired if d.candidate.key == "jimeng_pill")
flat = bool(pill.candidate.features.get("footprint_flat", 0.0))
if not _keep_pill(
keys,
provenance=context.provenance,
sensitivity=context.sensitivity,
footprint_flat=flat,
):
if not _keep_pill(keys, provenance=context.provenance, footprint_flat=flat):
fired = [d for d in fired if d.candidate.key != "jimeng_pill"]
return fired
+1 -1
View File
@@ -49,7 +49,7 @@ class TestRemoveVisibleArray:
def test_array_accepts_knobs(self):
arr = np.zeros((256, 256, 3), np.uint8)
result, removed = raiw.remove_visible(arr, sensitivity="assume_ai", backend="cv2")
result, removed = raiw.remove_visible(arr, sensitivity="strict", backend="cv2")
assert removed == []
assert result.shape == arr.shape
+25
View File
@@ -573,6 +573,31 @@ class TestMetadataCommand:
assert result.exit_code == 0
assert "stripped" in result.output
def test_metadata_remove_reports_failure_when_the_strip_was_a_no_op(self, runner, tmp_path):
"""A file PIL cannot decode is copied through UNCHANGED by the fail-safe.
That is correct (never crash a worker on a partial upload) but the command used
to print "AI metadata stripped ->" and exit 0 for it, so a caller could not tell
a real strip from a no-op and the output still read as AI. Found on real Samsung
Galaxy S22 C2PA PNGs during the corpus parity audit, 2026-07-19.
"""
# PNG signature + a C2PA (caBX) chunk, then garbage: the byte scanner sees the
# marker, PIL cannot decode it.
src = tmp_path / "undecodable.png"
payload = b"c2pa" + b"\x00" * 32
chunk = len(payload).to_bytes(4, "big") + b"caBX" + payload + b"\x00\x00\x00\x00"
src.write_bytes(b"\x89PNG\r\n\x1a\n" + chunk + b"NOTAPNG" * 8)
out = tmp_path / "cleaned.png"
result = runner.invoke(main, ["metadata", str(src), "--remove", "-o", str(out)])
from remove_ai_watermarks.metadata import get_ai_metadata
if not get_ai_metadata(src):
pytest.skip("fixture does not register as an AI-metadata carrier")
assert result.exit_code != 0, "a no-op strip must not report success"
assert "stripped ->" not in result.output
def test_metadata_remove_in_place(self, runner, tmp_png_with_ai_metadata):
"""With ``-o`` omitted, the strip overwrites the source in place (default
output_path=None). Previously every test passed an explicit ``-o``."""
+7 -13
View File
@@ -107,10 +107,10 @@ class TestPillRegistry:
class TestPillGate:
"""Pill removal is gated (``_keep_pill``): the reliable bottom-right wordmark
removes it unrestricted, the metadata (``"jimeng"`` provenance) / assume_ai arm
removes it ONLY on a flat footprint (safe fill), Doubao/no-confirmation never
remove it. Fakes each mark's detect so no image content is needed; cv2 backend so
nothing downloads. Frame flatness matters, so tests pass a flat or textured frame."""
removes it unrestricted, the metadata arm (``"jimeng"`` provenance) removes it ONLY
on a flat footprint (safe fill), Doubao/no-confirmation never remove it. Fakes each
mark's detect so no image content is needed; cv2 backend so nothing downloads. Frame
flatness matters, so tests pass a flat or textured frame."""
@staticmethod
def _fakes(monkeypatch: pytest.MonkeyPatch, keys: set[str]) -> None:
@@ -149,16 +149,10 @@ class TestPillGate:
_, removed = registry.remove_auto_marks(_textured_frame())
assert "Jimeng AI生成 pill" in removed
def test_pill_kept_via_assume_ai_on_flat_footprint(self, monkeypatch: pytest.MonkeyPatch) -> None:
# assume_ai (no metadata) removes the pill on a flat footprint (safe fill)...
def test_pill_dropped_on_textured_footprint(self, monkeypatch: pytest.MonkeyPatch) -> None:
# The metadata arm keeps the flatness guard: textured false fires visibly smear.
self._fakes(monkeypatch, {"jimeng_pill"})
_, removed = registry.remove_auto_marks(np.full((400, 300, 3), 150, np.uint8), sensitivity="assume_ai")
assert "Jimeng AI生成 pill" in removed
def test_pill_dropped_via_assume_ai_on_textured_footprint(self, monkeypatch: pytest.MonkeyPatch) -> None:
# ...but even assume_ai keeps the flatness guard (textured false fires smear).
self._fakes(monkeypatch, {"jimeng_pill"})
_, removed = registry.remove_auto_marks(_textured_frame(), sensitivity="assume_ai")
_, removed = registry.remove_auto_marks(_textured_frame(), provenance=frozenset({"jimeng"}))
assert "Jimeng AI生成 pill" not in removed
def test_pill_dropped_without_metadata_or_wordmark(self, monkeypatch: pytest.MonkeyPatch) -> None:
+191
View File
@@ -0,0 +1,191 @@
"""Policy-level tests for the shared text-mark engine config.
These assert TUNING that was set by corpus measurement, not algorithm behaviour --
they exist so a future edit cannot silently revert a calibrated constant back to a
value that was measured to be wrong. The measurements themselves live in
`docs/module-internals.md` and in the comment at
`_text_mark_engine._DEFAULT_PROVENANCE_NCC_FACTOR`.
"""
class TestRivalMargin:
"""Detection among same-corner marks is COMPETITIVE, not just absolute.
Doubao "豆包AI生成" and Jimeng "★ 即梦AI" both sit bottom-right in near-white CJK
and survive binarization as similar blobs, so an absolute NCC gate cannot tell
them apart -- 33 of jimeng's 68 false additions were Doubao marks. Measured
separability scoring both templates on the SAME blob (n=40 jimeng / 75 doubao):
absolute ncc_jimeng 0.96, ncc_jimeng MINUS ncc_doubao 0.99. Corpus effect of the
margin gate: jimeng precision 38% -> 63% with genuine detections unchanged at 40
(false fires 65 -> 23).
"""
def test_jimeng_competes_against_doubao(self):
from remove_ai_watermarks import jimeng_engine
assert "doubao_alpha.png" in jimeng_engine._CONFIG.rivals
def test_doubao_has_no_rival_margin(self):
"""Asymmetric by measurement, not oversight: the symmetric gate cost Doubao 7
genuine detections to prevent 5 false ones (1.4:1 against), while Jimeng gained
25pp for free. Doubao's absolute detector is already 86% precise."""
from remove_ai_watermarks import doubao_engine
assert doubao_engine._CONFIG.rivals == ()
def test_a_doubao_shaped_blob_loses_the_jimeng_margin(self):
"""The decisive case: a blob matching Doubao BETTER than Jimeng must not be
booked as a Jimeng wordmark, however high its absolute Jimeng score."""
import numpy as np
from remove_ai_watermarks import jimeng_engine
from remove_ai_watermarks._text_mark_engine import glyph_silhouette
eng = jimeng_engine.JimengEngine()
doubao_blob = glyph_silhouette("doubao_alpha.png")
assert doubao_blob is not None
canvas = np.zeros((doubao_blob.shape[0] + 20, doubao_blob.shape[1] + 20), np.uint8)
canvas[10 : 10 + doubao_blob.shape[0], 10 : 10 + doubao_blob.shape[1]] = doubao_blob
width = int(doubao_blob.shape[1] / jimeng_engine._CONFIG.alpha_width_frac)
jimeng_score = eng._template_match_score(canvas, width)
assert not eng._rival_margin_ok(jimeng_score, canvas, width)
class TestPerMarkProvenanceRelaxation:
"""The provenance NCC relaxation is PER MARK, not one shared multiplier.
Corpus-measured 2026-07-18 on the default `auto` path (4417 unique TC260
carriers, blind hand-label, two-sided control): the single shared 0.7 ran at
76% precision on doubao but 17% on jimeng, because jimeng's relaxed silhouette
keys on "text in the bottom-right corner" rather than the wordmark -- 33 of its
68 false additions were DOUBAO marks. Full table at
`_text_mark_engine._DEFAULT_PROVENANCE_NCC_FACTOR`.
"""
class TestScaleBasis:
"""Mark geometry scales with a PER-MARK image dimension, measured not assumed.
Every tuned fraction was calibrated on PORTRAIT captures, where width and short
side coincide, so the basis was never exercised until landscape inputs were
measured. Corpus-measured 2026-07-18 (2572 unique TC260 carriers): doubao
detection was portrait 60% / square 41% / **landscape 0% of 435** -- a width-scaled
box is inflated by the aspect ratio on a wide image and the glyph never lands in
it. A short-side basis recovered 56% of the previously-undetected landscape set.
The same switch broke JIMENG (labelled landscape positives 13/13 -> 0/13), whose
wordmark tracks the width -- hence per-mark, not a house rule.
"""
def test_doubao_scales_with_the_short_side(self):
from remove_ai_watermarks import doubao_engine
assert doubao_engine._CONFIG.scale_basis == "short"
def test_jimeng_scales_with_width(self):
"""Measured, not an oversight: the short-side basis took jimeng's labelled
landscape positives from 13/13 to 0/13."""
from remove_ai_watermarks import jimeng_engine
assert jimeng_engine._CONFIG.scale_basis == "width"
def test_samsung_keeps_width_because_it_is_unmeasured(self):
"""1 addition corpus-wide, so there is no evidence either way; an unmeasured
change is not an improvement."""
from remove_ai_watermarks import samsung_engine
assert samsung_engine._CONFIG.scale_basis == "width"
def test_basis_only_differs_on_non_square_images(self):
"""The basis is a no-op wherever width IS the short side, which is why the bug
survived: every calibration capture was portrait."""
import numpy as np
from remove_ai_watermarks import doubao_engine, jimeng_engine
portrait = np.zeros((1600, 900, 3), np.uint8)
landscape = np.zeros((900, 1600, 3), np.uint8)
d, j = doubao_engine.DoubaoEngine(), jimeng_engine.JimengEngine()
assert d.scale_base(portrait) == j.scale_base(portrait) == 900
assert d.scale_base(landscape) == 900
assert j.scale_base(landscape) == 1600
def test_landscape_box_stays_inside_the_frame(self):
"""The concrete failure: a width-scaled box on a wide image overshoots the
mark's real footprint. The short-side box must be proportionally smaller."""
import numpy as np
from remove_ai_watermarks import doubao_engine
landscape = np.zeros((900, 2400, 3), np.uint8)
loc = doubao_engine.DoubaoEngine().locate(landscape)
assert loc.w < int(2400 * doubao_engine._CONFIG.width_frac)
assert loc.x + loc.w <= 2400
assert loc.y + loc.h <= 900
class TestTophatFrontend:
"""Detection can correlate the CONTINUOUS top-hat instead of a binarized blob.
`extract_mask` thresholds the top-hat into a 0/255 glyph blob, which is fine for a
mark stamped bold and opaque and destructive for a faint one -- a thin translucent
overlay shatters into specks and no template can match a blob that is not there
(measured: 千问 scored 0.170 mean vs doubao's 0.723 through the binary path, 0% over
the gate). The `tophat` front-end never binarizes: the saturation/luma gates become
weights, and the response is max-normalized so the score is contrast-invariant.
Corpus effect on the 240-image unbiased recall sample: doubao recall 89% -> 92% at
an unchanged 99% precision.
"""
def test_doubao_uses_the_continuous_frontend(self):
from remove_ai_watermarks import doubao_engine
assert doubao_engine._CONFIG.detect_frontend == "tophat"
def test_other_marks_stay_binary_until_measured(self):
"""A front-end switch must be measured per mark before it ships; jimeng and
samsung have no such measurement yet."""
from remove_ai_watermarks import jimeng_engine, samsung_engine
assert jimeng_engine._CONFIG.detect_frontend == "binary"
assert samsung_engine._CONFIG.detect_frontend == "binary"
def test_response_is_contrast_invariant(self):
"""The whole point: a faint mark and a bold one produce the same response, so a
single threshold works for both. Binarizing is what loses the faint one."""
import numpy as np
from remove_ai_watermarks import doubao_engine
eng = doubao_engine.DoubaoEngine()
h, w = 400, 900
out = []
for amplitude in (12, 90): # a barely-there overlay and a bold one
img = np.full((h, w, 3), 100, np.uint8)
loc = eng.locate(img)
x, y, bw, bh = loc.bbox
img[y + bh // 3 : y + 2 * bh // 3, x + bw // 4 : x + 3 * bw // 4] = 100 + amplitude
resp = eng.tophat_response(img, loc)
assert resp is not None
out.append(resp)
# max-normalized, so the two responses agree despite a 7.5x contrast difference
assert abs(int(out[0].max()) - int(out[1].max())) <= 1
def test_flat_input_yields_no_response(self):
"""A blank corner has no top-hat at all; the engine must return None rather
than divide by a zero peak."""
import numpy as np
from remove_ai_watermarks import doubao_engine
eng = doubao_engine.DoubaoEngine()
img = np.full((400, 900, 3), 128, np.uint8)
assert eng.tophat_response(img, eng.locate(img)) is None
def test_threshold_is_frontend_specific(self):
"""The continuous front-end scores higher overall (0.809 vs 0.723 mean on the
same positives), so it needs its own gate; the binary-era 0.40 left the
provenance-relaxed gate low enough to admit 8 false fires where 0.50 admits 1."""
from remove_ai_watermarks import doubao_engine
assert doubao_engine._CONFIG.detect_ncc_threshold == 0.50
+48
View File
@@ -0,0 +1,48 @@
"""Pure tests for the strength/steps profile helpers (no model, no torch needed)."""
from __future__ import annotations
import pytest
from remove_ai_watermarks.noai.watermark_profiles import resolve_strength, viable_steps
class TestViableSteps:
"""Guards the crash found by the release smoke matrix on 2026-07-19.
diffusers derives its img2img timesteps as ``int(steps * strength)``. When that
rounds to zero the pipeline builds an empty tensor and dies deep inside attention
with "cannot reshape tensor of 0 elements into shape [0, -1, 1, 512]". At the
default strength 0.15 that was every ``--steps`` below 7, reachable with entirely
valid CLI arguments and no special flags.
"""
@pytest.mark.parametrize(
("steps", "strength"),
[(1, 0.15), (2, 0.15), (5, 0.15), (6, 0.15), (5, 0.10), (9, 0.10), (1, 0.5)],
)
def test_never_returns_a_count_that_denoises_zero_steps(self, steps: int, strength: float):
assert int(viable_steps(steps, strength) * strength) >= 1
@pytest.mark.parametrize(
("steps", "strength"),
[(50, 0.15), (20, 0.15), (7, 0.15), (10, 0.10), (2, 0.5), (50, 1.0)],
)
def test_leaves_a_workable_count_untouched(self, steps: int, strength: float):
assert viable_steps(steps, strength) == steps
def test_raises_only_to_the_minimum_needed(self):
# strength 0.15 needs 7 (int(7*0.15)==1); it must not jump to some larger default.
assert viable_steps(5, 0.15) == 7
assert viable_steps(1, 0.10) == 10
def test_the_vendor_defaults_all_have_a_reachable_floor(self):
for vendor in (None, "openai", "google"):
strength = resolve_strength(None, vendor)
assert int(viable_steps(1, strength) * strength) >= 1
@pytest.mark.parametrize("strength", [0.0, -0.1])
def test_a_non_positive_strength_cannot_loop_or_divide_by_zero(self, strength: float):
# No denoising is possible at all here; return the caller's value rather than
# dividing by zero or spinning.
assert viable_steps(20, strength) == 20
+115 -61
View File
@@ -113,24 +113,51 @@ class TestFill:
class TestProvenanceGate:
"""The Gemini trust gate relaxes from 0.5 to 0.35 when provenance confirms Google;
tested deterministically by stubbing the engine's raw detection confidence."""
"""The Gemini trust gate relaxes from GEMINI_SPARKLE_TRUST_CONF to
_GEMINI_PROVENANCE_MIN_CONF when provenance confirms Google; tested
deterministically by stubbing the engine's raw detection confidence."""
def _stub(self, monkeypatch: pytest.MonkeyPatch, conf: float) -> None:
from remove_ai_watermarks.gemini_engine import DetectionResult
# `detected` mirrors the engine's own internal floor (0.35), which is
# independent of the registry gate under test here.
def fake_detect(image, force_size=None, *, trust_provenance=False):
return DetectionResult(detected=conf >= 0.35, confidence=conf, region=(10, 10, 48, 48))
monkeypatch.setattr(reg._engine("gemini"), "detect_watermark", fake_detect)
def test_midband_conf_needs_provenance(self, monkeypatch: pytest.MonkeyPatch):
# conf 0.42 sits in [0.35, 0.5): demoted without provenance, trusted with it.
self._stub(monkeypatch, 0.42)
# Comfortably inside the relaxed band: demoted without provenance, trusted with it.
conf = (reg._GEMINI_PROVENANCE_MIN_CONF + reg.GEMINI_SPARKLE_TRUST_CONF) / 2
self._stub(monkeypatch, conf)
img = np.zeros((256, 256, 3), np.uint8)
assert reg.get_mark("gemini").detect(img).detected is False
assert reg.get_mark("gemini").detect(img, provenance=True).detected is True
def test_below_provenance_gate_rejected_even_with_provenance(self, monkeypatch: pytest.MonkeyPatch):
"""Provenance relaxes the gate, it does not remove it.
Guards the 2026-07-18 raise of _GEMINI_PROVENANCE_MIN_CONF (0.35 -> 0.42).
The engine still reports `detected` down at its own 0.35 floor, so without
the registry gate this confidence would be accepted and inpainted. Measured
precision just below the gate was 13% (n=30, 95% CI 5-30%) on real
Google-metadata uploads -- i.e. ~7 of 8 accepts there destroy pixels on an
image that never carried a sparkle, and report a removal that did not happen.
"""
# 0.38 is inside the measured 13%-precision band and above the engine's own
# 0.35 floor, so the engine reports `detected` and only the registry gate can
# reject it. Hardcoded on purpose: if the gate is ever lowered back under this
# value, this test must fail on the BEHAVIOUR below, not on its own arithmetic.
self._stub(monkeypatch, 0.38)
img = np.zeros((256, 256, 3), np.uint8)
assert reg.get_mark("gemini").detect(img).detected is False
assert reg.get_mark("gemini").detect(img, provenance=True).detected is False
def test_provenance_gate_stays_below_the_strict_gate(self):
"""The relaxed gate must actually relax, and must not collapse onto the floor."""
assert 0.35 < reg._GEMINI_PROVENANCE_MIN_CONF < reg.GEMINI_SPARKLE_TRUST_CONF
def test_high_conf_detected_either_way(self, monkeypatch: pytest.MonkeyPatch):
self._stub(monkeypatch, 0.72)
img = np.zeros((256, 256, 3), np.uint8)
@@ -175,19 +202,6 @@ class TestSensitivity:
== "strict"
)
def test_assume_ai_without_evidence_is_assumed_not_confirmed(self):
# asserting the image is AI says nothing about WHICH vendor made it, so the mark
# is relaxed on assumption only -- it must not inherit the confirmed-vendor bypass
assert (
reg.resolve_trust("gemini", sensitivity="assume_ai", provenance=frozenset(), strict_keys=set()) == "assumed"
)
def test_assume_ai_with_metadata_is_confirmed(self):
assert (
reg.resolve_trust("gemini", sensitivity="assume_ai", provenance=frozenset({"gemini"}), strict_keys=set())
== "confirmed"
)
def test_auto_relaxes_on_own_metadata(self):
assert (
reg.resolve_trust("gemini", sensitivity="auto", provenance=frozenset({"gemini"}), strict_keys=set())
@@ -210,34 +224,74 @@ class TestSensitivity:
reg.resolve_trust("doubao", sensitivity="auto", provenance=frozenset(), strict_keys={"jimeng"}) == "strict"
)
def test_assumed_floor_rejects_weak_sparkle_but_passes_strong(self):
# the gate-bypassed sparkle detector fires on ~60% of ordinary photos at its bare
# 0.35 threshold; only a match well clear of that floor is trustworthy on assumption
assert reg.assumed_floor_ok("gemini", 0.35) is False
assert reg.assumed_floor_ok("gemini", 0.50) is True
def test_assumed_floor_default_passes_for_unfloored_marks(self):
# text marks relax cleanly (<1% bypassed false-fire), so they carry no floor
assert reg.assumed_floor_ok("doubao", 0.36) is True
def test_remove_auto_marks_accepts_all_sensitivities(self):
blank = np.zeros((256, 256, 3), np.uint8)
for s in ("auto", "strict", "assume_ai"):
for s in ("auto", "strict"):
_, removed = reg.remove_auto_marks(blank, sensitivity=s, backend="cv2")
assert removed == []
class TestNoBlanketRelaxation:
"""There is NO path that relaxes a mark's gate without same-product evidence.
``assume_ai`` was that path and was removed 2026-07-19: it bypassed every mark's
false-positive gate on the caller's bare assertion that the image is AI, which says
nothing about WHICH vendor or WHERE -- exactly what the bypass is contracted to
require. Before it carried a confidence floor it filled a phantom sparkle on 59.8%
of genuine camera photos. A user who can SEE a mark is served by `erase --region`
(they supply the coordinates) or `--mark <name> --no-detect` for a text mark.
"""
def test_sensitivity_has_exactly_two_levels(self):
import typing
assert set(typing.get_args(reg.Sensitivity)) == {"auto", "strict"}
def test_trust_ladder_has_no_assumed_level(self):
import typing
assert set(typing.get_args(reg.Trust)) == {"strict", "confirmed"}
def test_no_sensitivity_relaxes_without_same_product_evidence(self):
for sens in ("auto", "strict"):
assert reg.resolve_trust("gemini", sensitivity=sens, provenance=frozenset(), strict_keys=set()) == "strict"
def test_the_assumed_floor_helper_is_gone(self):
"""It existed only to make the blanket relaxation tolerable."""
assert not hasattr(reg, "assumed_floor_ok")
assert not hasattr(reg, "_ASSUMED_CONF_FLOOR")
def test_the_removed_value_raises_instead_of_silently_meaning_auto(self):
"""`Sensitivity` is a Literal and unenforced at runtime, so a 0.15 caller passing
the removed value would quietly get `auto` -- a silent semantic change on exactly
the release where they need to be told. The error names the replacement."""
import pytest
with pytest.raises(ValueError, match="erase"):
reg.validate_sensitivity("assume_ai")
with pytest.raises(ValueError, match="unknown sensitivity"):
reg.validate_sensitivity("aggressive")
assert reg.validate_sensitivity("auto") == "auto"
assert reg.validate_sensitivity("strict") == "strict"
def test_context_rejects_it_too(self):
"""The arbiter's own entry point validates, so a direct `decide()` caller cannot
smuggle the removed mode past the public API."""
import pytest
with pytest.raises(ValueError, match=r"removed in 0\.16"):
reg.Context(sensitivity="assume_ai")
class TestArbiter:
"""``decide`` is the PURE removal arbiter: (candidates, context) -> ordered
winners, no image / no I/O. Tested in isolation by handing it fabricated
Candidates -- this is the payoff of separating decision from perception."""
@staticmethod
def _c(key, *, strict=False, relaxed=False, flat=False, relaxed_conf=1.0):
# relaxed_conf defaults high so a test that does not care about the assumed-trust
# confidence floor exercises the trust logic, not the floor.
def _c(key, *, strict=False, relaxed=False, flat=False):
feats = {"footprint_flat": 1.0} if flat else {}
return reg.Candidate(key, f"L:{key}", strict, relaxed, relaxed_conf, feats)
return reg.Candidate(key, f"L:{key}", strict, relaxed, feats)
def _keys(self, cands, ctx):
return {d.candidate.key for d in reg.decide(cands, ctx)}
@@ -249,35 +303,6 @@ class TestArbiter:
# relaxed-only detection must NOT fire under strict
assert self._keys([self._c("gemini", relaxed=True)], reg.Context(sensitivity="strict")) == set()
def test_assume_ai_uses_relaxed(self):
fired = reg.decide([self._c("gemini", relaxed=True)], reg.Context(sensitivity="assume_ai"))
assert [d.candidate.key for d in fired] == ["gemini"]
assert fired[0].relax is True
def test_assume_ai_drops_sparkle_below_the_assumed_floor(self):
# REGRESSION (2026-07-16): assume_ai passed trust_provenance=True to the engine,
# bypassing the sparkle false-positive gate on the mere ASSERTION that the image is
# AI -- but that flag is contracted to mean "metadata proved this vendor". The bare
# bypassed gate (conf 0.35) fired on 59.8% of 256 genuine camera captures, so
# `--sensitivity assume-ai` filled a phantom sparkle on ~6 of every 10 clean photos.
weak = self._c("gemini", relaxed=True, relaxed_conf=0.40)
assert reg.decide([weak], reg.Context(sensitivity="assume_ai")) == []
def test_assume_ai_keeps_sparkle_confirmed_by_metadata_below_the_floor(self):
# the floor exists because the vendor is UNKNOWN; once metadata names Google the
# bypass is contract-legal again, so a weak match is still trusted
weak = self._c("gemini", relaxed=True, relaxed_conf=0.40)
ctx = reg.Context(sensitivity="assume_ai", provenance=frozenset({"gemini"}))
assert [d.candidate.key for d in reg.decide([weak], ctx)] == ["gemini"]
def test_assume_ai_is_monotonic_over_strict(self):
# a mark the STRICT gate accepted must never be dropped by the assumed floor:
# assume_ai only ever adds recall
weak_but_strict = self._c("gemini", strict=True, relaxed=True, relaxed_conf=0.40)
fired = reg.decide([weak_but_strict], reg.Context(sensitivity="assume_ai"))
assert [d.candidate.key for d in fired] == ["gemini"]
assert fired[0].relax is False # accepted on the strict verdict, so mask at strict
def test_auto_relaxes_on_provenance(self):
c = [self._c("gemini", relaxed=True)]
assert self._keys(c, reg.Context(provenance=frozenset({"gemini"}))) == {"gemini"}
@@ -309,6 +334,35 @@ class TestArbiter:
]
assert "jimeng_pill" in self._keys(cands, reg.Context())
def test_weak_pill_detection_does_not_confirm_the_jimeng_wordmark(self):
"""The pill is too false-fire-prone (~7%) to grant a sibling `confirmed` trust.
Corpus-measured defect (2026-07-18): a pill false fire on clean non-ByteDance
content confirmed jimeng, relaxing its NCC gate 0.45 -> 0.3825; jimeng then
false-fired, and _keep_pill's wordmark arm removed the pill UNRESTRICTED,
skipping the flatness guard. Closed loop on the default `auto` path.
"""
# Only the pill is strictly detected. jimeng scores in the band that is
# reachable ONLY via the relaxed gate.
cands = [
self._c("jimeng_pill", strict=True, relaxed=True, flat=False),
self._c("jimeng", strict=False, relaxed=True),
]
fired = {d.candidate.key for d in reg.decide(cands, reg.Context("auto", frozenset()))}
assert "jimeng" not in fired, "a weak pill hit must not relax the jimeng wordmark"
# and with jimeng gone, the pill loses the wordmark arm too -- no unrestricted
# removal of a textured footprint on content nothing confirmed.
assert "jimeng_pill" not in fired
def test_real_jimeng_wordmark_still_corroborates_the_pill(self):
"""The fix removes only the pill's TESTIMONY, not the wordmark's."""
cands = [
self._c("jimeng", strict=True, relaxed=True),
self._c("jimeng_pill", strict=True, relaxed=True, flat=False),
]
fired = {d.candidate.key for d in reg.decide(cands, reg.Context("auto", frozenset()))}
assert fired == {"jimeng", "jimeng_pill"}
class TestProvenanceMaskThreading:
"""Regression for the provenance-relaxed Gemini no-op (#1) and the false 'removed'