diff --git a/CLAUDE.md b/CLAUDE.md index c7c8c3f..fea91ca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -56,7 +56,7 @@ Compact map. The full per-module detail (design decisions, tuned thresholds, cal - `noai/constants.py` — the single `C2PA_AI_VENDORS` registry (+ `C2PA_SOFT_BINDINGS`) from which `C2PA_ISSUERS` / `SYNTHID_C2PA_ISSUERS` / `C2PA_IDENTITY_AI_ORGS` / `identify._ISSUER_PLATFORM` are all derived. Add a new vendor as one registry entry; never edit the derived dicts and never add inline. A vendor's `asserts_ai=True` flag means its mere presence asserts AI generation even without a `trainedAlgorithmicMedia` digital-source-type (a pure-generator brand with a distinctive issuer/generator string, e.g. **Dreamina** — ByteDance's international Jimeng brand, signed as "Bytedance Pte. Ltd." with a "Dreamina/x.y" claim generator and no source-type); NEVER set it for common-word issuers (Adobe/Google/OpenAI/Microsoft) that appear incidentally in unrelated bytes — those stay source-type-gated in `identify._attribute_platform`. - `metadata.py` — `scan_head(path)` is the shared (memoized) input for every C2PA/AIGC/IPTC byte scan; use it instead of `open().read(1MB)` for any new marker scan. Also home to `synthid_source`, `xai_signature`, `iptc_ai_system`, `aigc_label`, `huggingface_job`, `samsung_genai`, and `remove_ai_metadata` (fail-safe `strip_c2pa_boxes`). **A caller that REPORTS an outcome must use `strip_and_verify`, not `remove_ai_metadata` directly** -- the stripper is deliberately fail-safe (a file PIL cannot decode is copied through UNCHANGED rather than crashing), so its return value cannot distinguish a no-op from a real strip. `metadata --remove` and `batch --mode metadata|all` both re-scan the OUTPUT through it and fail loudly; corpus-observed on real Samsung Galaxy S22 C2PA PNGs, where the command printed "stripped" and exited 0 while the output still read as AI (2026-07-19). **`remove_ai_metadata` is the SINGLE metadata stripper** (the legacy PIL-re-encoding `noai/cleaner` was deleted; the diffusion core and the public `noai.remove_ai_metadata` re-export now point here). It strips **losslessly** per container: ISOBMFF (HEIC/AVIF/MP4) blanks tokens / strips boxes in place; **JPEG uses `_strip_jpeg_metadata_lossless`** — a marker-segment walk that drops the AI-bearing APP segments (C2PA APP11; XMP APP1 carrying C2PA, a China-AIGC token, OR an IPTC `digitalSourceType` / 2025.1 AI-disclosure marker; IPTC-IIM APP13) and scrubs AI EXIF tags via piexif, copying the entropy-coded scan verbatim so **the pixels are bit-identical** (no DCT re-encode). **Detection<->removal parity across every marker placement is load-bearing** — anything a scanner flags, the strip must reach, or a re-served file still reads as AI: (a) the APP1-XMP branch of `_jpeg_app_carries_ai` checks the IPTC marker sets too, not only C2PA/AIGC (the Instagram/MidJourney/Meta "Made with AI" `digitalSourceType` lives in XMP, not the APP13 IIM record); (b) a bare `AIGC{...}` / `{"AIGC":{...}}` block in ANY JPEG APP segment — the specific C2PA(APP11)/XMP(APP1)/IPTC(APP13) checks FALL THROUGH to a generic `_is_aigc_exif_value` drop, so a bare AIGC in APP11 (the common real-corpus placement, NOT a C2PA manifest) is caught, not swallowed by the C2PA-only 0xEB branch — plus the same AIGC block in a STANDARD **PNG text chunk** value (e.g. `Description`, which `_is_ai_key` keeps) is dropped on the value; (c) the China TC260 `{"AIGC":{...}}` block in EXIF `UserComment`/`ImageDescription` is scrubbed by `_scrub_ai_exif` (Doubao producer + Tencent service-provider schemas); (d) the Samsung Galaxy AI `PhotoEditor_Re_Edit_Data` trailer past the JPEG EOI is truncated by `_strip_samsung_trailer` (and `samsung_genai` reads the file tail so a multi-MB photo's trailer past the 512 KB quick-scan window is still DETECTED). Pixels stay bit-identical throughout, so a `--strip-metadata` on a q100 removal output does NOT crush it back to q75; PNG/WebP re-saves are pixel-lossless (WebP written at cv2 lossless mode, quality 101 — quality 1-100 is lossy). **The PIL-fallback save format is chosen by the source's CONTENT, not its file extension** (`_sniff_image_format`, and the JPEG-lossless gate is content-gated too): ~2% of real uploads are misnamed (a PNG served as `.jpg` is the common one), and routing on the extension re-encoded a lossless PNG/WebP into a real JPEG — a silent degradation that broke "work with originals" (corpus-measured ~0.9% of files). A **misnamed** lossless source (source-extension format != content) is preserved in its true format; a **correctly-named** source still honors a deliberate output-extension conversion (e.g. `source.png -> output.jpg`). Not yet handled: a 16-bit PNG is downconverted to 8-bit on the PIL re-save (rare; would need a byte-level PNG chunk stripper). Regression: `tests/test_metadata.py::TestHasAiMetadata::test_strip_preserves_lossless_content_with_mismatched_extension`. **`remove_ai_metadata` is fail-safe on an undecodable image:** a truncated/corrupt file (PIL raises `OSError` decoding it; ~0.2% of real uploads) is copied through UNCHANGED rather than crashing a direct library caller (a web worker would 500 on a partial upload), mirroring `strip_c2pa_boxes` — we cannot strip what we cannot parse, but we never raise. Regression: `tests/test_metadata.py::TestHasAiMetadata::test_remove_ai_metadata_failsafe_on_truncated_png`. Regression: `tests/test_metadata.py::TestHasAiMetadata::{test_jpeg_metadata_strip_is_pixel_lossless, test_jpeg_strip_removes_iptc_marker_in_xmp}`, `TestSamsungGenai::{test_remove_strips_post_eoi_trailer, test_detects_trailer_past_scan_window}`, the AIGC-EXIF/bare-APP removal tests, and `tests/test_noai.py::TestISOBMFF::{test_blank_aigc_block_in_exif, test_blank_xai_signature_pair_in_exif}`. `exif_generator` matches a VALUE against `AI_GENERATOR_TOKENS` across EXIF `Software`/`Make`/`Artist`/`ImageDescription`, XMP `CreatorTool`, AND PNG `tEXt` chunks (`Software`/`Source`/`Title`/`Description` — NovelAI stamps there, not EXIF). **Detection and removal must stay in parity:** a generator that stamps an AI-shaped VALUE under a non-AI KEY (NovelAI's `Title`/`Source`) is dropped on removal by `_is_ai_value` (value-token match, mirrors `exif_generator`), NOT by `_is_ai_key` alone — else the cleaned file still reads as that generator. Add a new no-C2PA generator = one `AI_GENERATOR_TOKENS` entry (use a distinctive token, e.g. `reve.com` not bare `reve`); detection and removal then both follow. Regression: `tests/test_metadata.py::TestExifGenerator::{test_novelai_png_text_chunk_detected,test_novelai_removal_parity}`. - `identify.py` — aggregates every locally-readable signal into one `ProvenanceReport`; `is_ai_generated` is True or None, never asserted False. `ProvenanceReport.ai_source_kind` exposes the C2PA digital-source-type split — `"generated"` (trainedAlgorithmicMedia, fully AI) vs `"enhanced"` (compositeWithTrainedAlgorithmicMedia, a real photo with an AI-composited region), else None — so a caller branches full-frame scrub vs region-targeted clean (see `noai/tiling.feather_region_composite` + `WatermarkRemover.remove_watermark(region=...)`). The sparkle provenance threshold is the SHARED `watermark_registry.GEMINI_SPARKLE_TRUST_CONF` (imported, not a private copy) so the provenance "is there a sparkle" verdict and the removal "take the sparkle" decision can never drift. `import identify` is deliberately light (lazy `noai/__init__`, fits a 512 MB host) — keep heavy imports out (the `watermark_registry` constant import stays light: engines are lazy there). Add capture-camera tokens to `_DEVICE_C2PA_PLATFORM` only when verified against a real C2PA file; editing-app/AI-device signer tokens go to `_SIGNER_C2PA_PLATFORM`; generator/issuer platforms to `C2PA_AI_VENDORS` in `constants.py`. The IPTC `digitalSourceType` **`algorithmicMedia`** (bare) is PROCEDURAL (an algorithm not trained on sampled data), NOT AI/ML generation, so it is deliberately absent from `IPTC_AI_MARKERS` — flagging it made `identify` assert AI + `has_invisible_target` True, scrubbing clean procedural content (it is a distinct token from `trainedAlgorithmicMedia`, so real "Made with AI" labels are unaffected; regression `test_metadata.py::...test_bare_algorithmic_media_not_flagged_ai`). Integrity-clash detection is high-precision by design (only hard generator stamps feed it, source-grouped independence). `_vendor_of` normalizes ByteDance/Canva/ElevenLabs/Black Forest Labs (as well as OpenAI/Google/... ) so their C2PA claims participate in the clash check; the generic **China TC260 AIGC label names no specific vendor**, so when a TC260-applying vendor (ByteDance, `_TC260_VENDORS`) is co-attributed the label is attributed to it (a legit Doubao image carrying its own TC260 label must NOT clash), while a NON-TC260 vendor next to a TC260 label still clashes as a laundering tell. Corpus-validated: adding the vendors introduced 0 new clashes on 5000 carriers. -- `watermark_registry.py` — the single catalog of known visible watermarks (gemini / doubao / jimeng / samsung / jimeng_pill). **Removal is LOCALIZE -> FILL for every mark:** each mark is localized to a binary full-frame footprint mask (a `Localization`), then ONE shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). Reverse-alpha (the old `original = (wm - a*logo)/(1-a)` inversion of a captured alpha map + thin residual inpaint) is GONE for ALL marks; why it was dropped is recorded in `docs/module-internals.md`. Backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. The captured alpha maps (`scripts/visible_alpha_solve.py`) are still used to DETECT the marks and to shape the mask, but NOT for pixel recovery. **`--mark auto` removes EVERY detected mark in one pass** via `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` (marks coexist -- a Jimeng-basic image has the top-left pill AND the bottom-right wordmark; a single-strongest pick would leave one). **Three orthogonal axes:** `backend` (the fill), `sensitivity` (how hard to trust a borderline mark: `auto`/`strict`, see the `Sensitivity` literal), and `provenance` (vendor keys metadata confirms -- the evidence that drives `auto`). **Perception / decision / action are separated:** `_build_candidates(image)` runs every detector at BOTH trust levels (strict + relaxed) and packages raw verdicts + features into `Candidate`s (no policy); the pure arbiter `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` makes every keep/drop call (per-mark `resolve_trust` + the assumed-trust floor + the pill gate) with no image/IO, so it is unit-testable in isolation; then each winner is localized -> filled. Do NOT put policy back into the engines (the one exception, the Gemini FP gate, stays in `gemini_engine` because `identify` shares that confidence). `detect_marks(..., provenance=frozenset())` stays strict (identify verdict, precision over recall); `KnownMark.remove/detect/localize(..., provenance: bool)` take the already-resolved boolean. **How `auto` decides (this is metadata-INDEPENDENT for recall):** the visual detectors are pixel-based and need no metadata; the recall gain comes from RELAXING the false-positive gate, not from metadata. `strict` never relaxes (clean images untouched); `auto` relaxes a mark only on same-product evidence -- metadata provenance for that vendor OR a confidently detected sibling mark of the SAME product (`_PRODUCT_OF`; Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax). **`resolve_trust` resolves TWO levels:** `confirmed` bypasses the engine's false-positive gate, and only `confirmed` has evidence naming THAT vendor, which is exactly what the bypass is contracted to require (`GeminiEngine.detect_watermark`'s `trust_provenance` docstring: "external metadata already proves this is a Google generation"). **Historical, kept as the reason the third level is gone:** a removed `assumed` level let `assume_ai` bypass the gate on the bare assertion an image is AI. On its first form that left only the raw 0.35 detector threshold and it fired on **59.8% of 256 genuine camera captures**, filling a phantom sparkle on ~6 of every 10 CLEAN photos; a confidence floor made it tolerable, and the mode was removed outright in 0.16. Corpus-measured 2026-07-16 before removal (400 Google-C2PA positives with metadata hidden; 256 camera-capture negatives): recall strict 55.0% / auto 55.2% / assume_ai 62.8%, false fire 0.0% / 0.0% / 2.3% -- the extra recall was never free. A wrong relaxation only fills a small corner near-losslessly (the localize -> fill benign failure mode), which is what made a SMALL false-fire rate arguable; it was never a licence for a 60% one. Metadata provenance mapping (feeds `auto`, read by `cli._visible_provenance`): Google/Gemini C2PA issuer -> gemini; China-AIGC (TC260) label -> doubao/jimeng; `samsung_genai` -> samsung. **The `jimeng_pill` is CAPTURE-LESS** (`pill_engine.py`): the top-left "AI生成" label has no captured alpha map, so it is detect-by-synthetic-silhouette; its footprint is a fixed top-left geometry box. Its weak edge-NCC detector (~7% raw false-fire) is gated in `remove_auto_marks` via **`_keep_pill`** (32k real-upload corpus validation 2026-07): the pill never rides on a **Doubao** detection, and has confirmation arms because metadata/intent confirms the platform, not pill presence. **(1) Bottom-right "★ 即梦AI" wordmark fired** — ~94% precise and survives **metadata-STRIPPED uploads** (screenshots / re-saves, ~61% of pills carry a detectable wordmark): remove **unrestricted**. **(2) TC260 metadata confirms Jimeng** (`"jimeng" in provenance`, no wordmark) — the metadata-only arm is only **21% precise raw (CI 16-29%), 29% (CI 20-40%) among the flat footprints the guard PASSES** (re-measured 2026-07-18, 149 blind-labelled fires) and its false fires are **textured ceilings/walls that the fill visibly SMEARS**, so remove **only when the top-left footprint is flat enough for an invisible fill** (`pill_engine.footprint_is_flat`, median-Sobel texture ≤ `_FLAT_TEXTURE_MAX`) — the flatness guard always holds. This keeps real flat-scene pills (incl. metadata-only ones the wordmark misses) plus harmless flat false fires, and leaves the damaging textured false fires untouched. Do NOT drop the wordmark arm or loosen the flatness guard. **`assume_ai` was REMOVED (2026-07-19); `--sensitivity` is now `auto`/`strict` only.** It relaxed every mark's FP gate on the bare assertion an image is AI -- which names no vendor and no location, exactly what the bypass requires -- and had no place in the model (detector finds -> remove; finds nothing -> leave alone; user SEES a mark -> act on that). It took `_ASSUMED_CONF_FLOOR` / `assumed_floor_ok` / the `assumed` trust level with it, collapsing the ladder to `strict`/`confirmed`, and `_keep_pill` lost its `sensitivity` arg. Recall/precision on the unbiased sample are unchanged, so nothing on the default path moved. **Replacement advice is per mark:** `erase --region` is sound by construction; `--mark --no-detect` is reasonable (forced mask = the real glyph blob, non-empty 13/13); **`--mark gemini --no-detect` is NOT** -- it falls back to a fixed slot that covered the true sparkle on only **31% of 97** missed sparkles, so 69% fill a clean corner AND report a removal that did not happen. `cli._no_visible_mark_exit` follows that order and no longer suggests the removed mode. Migration raises loudly (`validate_sensitivity`, called from `api.remove_visible` and `Context.__post_init__`) because a `Literal` is unenforced at runtime and would silently downgrade a 0.15 caller to `auto`. **Detection can skip binarization** (`TextMarkConfig.detect_frontend`): `binary` thresholds the top-hat into a glyph blob (the original path), `tophat` correlates the CONTINUOUS top-hat against a soft template, turning the saturation/luma gates into weights and max-normalizing so the score is contrast-invariant. **Doubao uses `tophat`** (recall 89% -> 92% at an unchanged 99% precision on the unbiased sample); jimeng/samsung stay `binary` until measured per mark. **Detection and the REMOVAL MASK must use the SAME front-end.** `tophat` detection does not binarize, but `extract_mask` (which bounds the fill) still does, so a mark faint enough to be found only by the continuous response produced an EMPTY mask: `localize` returned None, `remove()` was a silent no-op, and `identify` reported `visible_doubao` while `visible` said "no visible mark" on the same file. Corpus-measured 2026-07-20: 57 of 60 sampled still-detected Doubao marks were untouched, ~8% of all its detections. `footprint_mask` now falls back to thresholding the CONTINUOUS response at `_FAINT_GLYPH_LEVEL` when the binary blob is empty -- **gated on an actual detection**, because the response is max-normalized and would otherwise scale NOISE up to 1.0 and mask a random patch of a clean corner. Verified on the corpus: 60/60 previously-untouched cases now clear. Regression: `tests/test_text_mark_faint_mask.py`. **Any future front-end change must move BOTH paths or re-check this.** **The gate is front-end specific -- re-calibrate, never port it**: the continuous response scores higher (mean 0.809 vs 0.723), so the binary-era 0.40 ran 96%/91% (8 false fires) while 0.50 runs 92%/99% (1). A first pass at 0.40 also silently halved the PILL's recall, since `_keep_pill` suppresses it whenever doubao fires. The front-end fixes DETECTION of faint marks but not ATTRIBUTION: 千问 becomes separable from clean corners (AUC 0.92) yet not from Doubao (**AUC ~0.5**) because they share the `AI生成` tail. Next design is a GENERIC CJK-AI-mark detector (GB 45438-2025 guarantees the shared tail) with vendor attribution as optional metadata. **Adding a new text mark is only cheap when the mark is stamped BOLD** -- 千问/星绘 are deliberately NOT registered. Measured on 14 corpus positives with the same pipeline and each mark's own template: doubao mean NCC **0.723** (82% over the 0.40 gate) vs qwen **0.170** (**0%**). Ruled out by measurement, in order: the synthetic render (a template cut from a REAL Qwen mark scores 0.307 vs the rendered 0.308, and real masks do not match each other), the fixed 5px morphology kernels (+0.014 mean when scaled with box height), and the appearance thresholds (best sweep: mean 0.35, 4/14 over the gate). The blocker is SEGMENTATION on a faint mark: the white top-hat shatters a thin translucent overlay into specks, and no template matches a blob that is not there. Adding it needs a front-end that does not binarize the glyph first (grayscale/edge correlation on the raw top-hat, or a learned patch classifier). **Check a candidate mark's contrast before promising it is a config-plus-silhouette job.** Evidence chain in `scripts/render_vendor_silhouettes.py`. **RECALL is measured on an unbiased random sample** (`scripts/visible_recall_sample.py`; 240 images drawn at random per provenance class and labelled exhaustively, 2026-07-18): doubao **89% recall / 99% precision**, gemini **96% / 80%**, jimeng 71%/71% (n=14), jimeng_pill 50%/60% (n=6). The `scale_basis` fix moved doubao recall **71% -> 89%** on this same sample. **Gemini's real precision is 80%, NOT the 41% `visible_eval.py` reports** -- that harness scores an addition-sampled set, so it measures the relaxation arm's marginal cases, not what production sees; quote 80% for the product. Landscape is improved but unsolved (doubao by aspect: portrait 92% / square 92% / landscape 79%). The largest remaining gap is not tuning but COVERAGE: **6% of sampled images carry an uncovered vendor's mark** (千问/百度/星绘/抖音-class) that no registered detector can fire on -- researched specs are in `docs/watermarking-landscape.md`. **Mark geometry scales with a PER-MARK dimension** (`TextMarkConfig.scale_basis` / `TextMarkEngine.scale_base`): doubao `short` (= min(h,w)), jimeng and samsung `width`. All the tuned fractions were calibrated on PORTRAIT captures where width == short side, so the basis went unexercised until landscape inputs were measured -- and **doubao detected 0 of 435 landscape TC260 images, a 100% miss rate**. It is a LOCALIZATION failure, not a threshold one (median doubao NCC on the 1452 no-detection images was 0.057, only 2.7% in a threshold-reachable band), so no amount of gate tuning could reach it. Short-side geometry recovers **56% of the previously-undetected landscape set**. The basis is per-mark because the SAME switch took jimeng's landscape positives from 13/13 to 0/13 -- its wordmark tracks the width even though both marks are ByteDance and share a corner; samsung stays `width` because it is unmeasured (1 addition corpus-wide). GB 45438-2025 5.2(e) mandates glyph height >= 5% of the shortest side, which is why short-side is the prior -- but measurement overrides the standard's wording. This was invisible for months because **precision was measured repeatedly and recall never was**; the harness now reports a `missed` column, which is what caught the jimeng regression the fix introduced. **Detection among same-corner marks is COMPETITIVE** (`TextMarkConfig.rivals` / `_rival_margin_ok`): a mark's template must beat every same-corner rival's on the SAME glyph blob by `rival_margin` (0.10). Absolute-only scoring could never separate Doubao and Jimeng (both bottom-right near-white CJK, near-identical after binarization) -- measured separability: absolute `ncc_jimeng` 0.96, `ncc_jimeng` MINUS `ncc_doubao` **0.99**. Corpus effect: **jimeng precision 38% -> 63% with genuine detections unchanged (false fires 65 -> 23)**, so it is a pure precision gain and the earlier 0.85 threshold patch was reverted to 0.70. **Asymmetric by measurement:** doubao declares NO rival -- the symmetric gate cost it 7 genuine detections to prevent 5 false (1.4:1 against) while jimeng gained 25pp for free. **Benchmark any detector change with `uv run python scripts/visible_eval.py --vs `** (741 blind-labelled corpus images; `scripts/visible_groundtruth.py` builds the set, `scripts/visible_sheets.py` makes new labelling rounds). Three harness rules are load-bearing: score a mark only within its crop's **adjudication scope**, take **provenance from metadata not from labels** (label-derived provenance scored gemini at 99% vs the true 41%), and **never report recall** from this set -- it was sampled where detectors fired, so an unbiased random sample is still needed. **The provenance NCC relaxation is PER MARK (`TextMarkConfig.provenance_ncc_factor`), not one shared multiplier** — measured 2026-07-18 on the default `auto` path over 4417 unique TC260 carriers (blind hand-label, two-sided control, labeller sensitivity 100%/96% and specificity 100%/100%), the old shared 0.7 ran at **76% precision on doubao but 17% on jimeng**. Doubao stays 0.70 (both its bands return more true marks than false fills). Jimeng moves to **0.85**: its relaxed silhouette keys on "text in the bottom-right corner" rather than the wordmark — of 68 false additions **33 were DOUBAO marks** and 17 were other vendors' AI labels, and 45 of the 68 filled a corner nothing else would touch; 0.85 costs 8 genuine recoveries to prevent 60 false fills (7.5:1), lifting the arm to 43%. That is a patch on a DETECTOR problem — jimeng's silhouette is not discriminative against doubao's, and no threshold fixes that. **A weak mark must not CORROBORATE a sibling** (`_CANNOT_CORROBORATE`): sibling corroboration grants `confirmed` trust, which bypasses the sibling's FP gate, so the pill (~7% raw false-fire) handing that bypass to jimeng created a closed loop on the DEFAULT path — pill false-fires → jimeng relaxes and false-fires → `_keep_pill`'s wordmark arm then removes the pill UNRESTRICTED, skipping the flatness guard (3/578 negatives ran the full loop, one with `footprint_flat=0`). Cutting the pill out of corroboration removed all 3 and cost NOTHING on the TC260 carriers (jimeng 398 → 398). `_keep_pill` already distrusted the pill's ACTION; this closes the gap that its TESTIMONY was ungated. `cli._write_bgr_with_alpha` must NOT zero alpha in the watermark bbox (issue #30 white-box regression). **The localizer is cheap CPU (cv2/numpy), so a memory-tight caller runs it anywhere; the heavy MI-GAN/LaMa fill is opt-in and chosen by the caller** (a small worker can use cv2; a GPU/model worker can use MI-GAN/LaMa). Adding a new mark needs only a DETECTION silhouette (removal is template-free — the glyph-blob bbox is filled, no capture involved). Produce that silhouette SYNTHETICALLY: font-render the mark's glyphs (the pill's `scripts/render_pill_silhouette.py` is the pattern; commit the rendered PNG under `assets/`) and calibrate the NCC threshold on real positives. The old solid/gray/white app-capture workflow (`scripts/visible_alpha_solve.py`) is RETIRED with reverse-alpha — existing marks still carry their captured silhouettes, but a NEW mark does NOT require captures. (The 2026-06-22 "synthetic reconstruction below the quality bar" objection was about reverse-alpha PIXEL recovery, which is gone; it does not apply to a synthetic detection silhouette.) Data-safety still binds the committed asset: the silhouette must be font-rendered synthetic, never derived from user uploads — seeing a real sample to learn the glyphs / font / position / locale is fine, but the committed template stays synthetic. So nothing is parked for lack of a capture: Meta AI and more Samsung locales just need the glyphs + font + locale + calibration positives; any Grok visible mark additionally needs confirming it even HAS one (its known signal is EXIF-only `xai_signature`). +- `watermark_registry.py` — the single catalog of known visible watermarks (gemini / doubao / jimeng / samsung / jimeng_pill). **Removal is LOCALIZE -> FILL for every mark:** each mark is localized to a binary full-frame footprint mask (a `Localization`), then ONE shared, swappable fill inpaints that mask via `fill(image, mask, backend=...)` (delegates to `region_eraser.erase`). Reverse-alpha (the old `original = (wm - a*logo)/(1-a)` inversion of a captured alpha map + thin residual inpaint) is GONE for ALL marks; why it was dropped is recorded in `docs/module-internals.md`. Backends: `cv2` (classical inpaint, no deps, the floor), `migan` (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), `lama` (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); `auto` = LaMa > MI-GAN > cv2, best available. The captured alpha maps (`scripts/visible_alpha_solve.py`) are still used to DETECT the marks and to shape the mask, but NOT for pixel recovery. **`--mark auto` removes EVERY detected mark in one pass** via `remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto")` (marks coexist -- a Jimeng-basic image has the top-left pill AND the bottom-right wordmark; a single-strongest pick would leave one). **Three orthogonal axes:** `backend` (the fill), `sensitivity` (how hard to trust a borderline mark: `auto`/`strict`, see the `Sensitivity` literal), and `provenance` (vendor keys metadata confirms -- the evidence that drives `auto`). **Perception / decision / action are separated:** `_build_candidates(image)` runs every detector at BOTH trust levels (strict + relaxed) and packages raw verdicts + features into `Candidate`s (no policy); the pure arbiter `decide(candidates, Context(sensitivity, provenance)) -> [Decision]` makes every keep/drop call (per-mark `resolve_trust` + the assumed-trust floor + the pill gate) with no image/IO, so it is unit-testable in isolation; then each winner is localized -> filled. Do NOT put policy back into the engines (the one exception, the Gemini FP gate, stays in `gemini_engine` because `identify` shares that confidence). `detect_marks(..., provenance=frozenset())` stays strict (identify verdict, precision over recall); `KnownMark.remove/detect/localize(..., provenance: bool)` take the already-resolved boolean. **How `auto` decides (this is metadata-INDEPENDENT for recall):** the visual detectors are pixel-based and need no metadata; the recall gain comes from RELAXING the false-positive gate, not from metadata. `strict` never relaxes (clean images untouched); `auto` relaxes a mark only on same-product evidence -- metadata provenance for that vendor OR a confidently detected sibling mark of the SAME product (`_PRODUCT_OF`; Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax). **`resolve_trust` resolves TWO levels:** `confirmed` bypasses the engine's false-positive gate, and only `confirmed` has evidence naming THAT vendor, which is exactly what the bypass is contracted to require (`GeminiEngine.detect_watermark`'s `trust_provenance` docstring: "external metadata already proves this is a Google generation"). **Historical, kept as the reason the third level is gone:** a removed `assumed` level let `assume_ai` bypass the gate on the bare assertion an image is AI. On its first form that left only the raw 0.35 detector threshold and it fired on **59.8% of 256 genuine camera captures**, filling a phantom sparkle on ~6 of every 10 CLEAN photos; a confidence floor made it tolerable, and the mode was removed outright in 0.16. Corpus-measured 2026-07-16 before removal (400 Google-C2PA positives with metadata hidden; 256 camera-capture negatives): recall strict 55.0% / auto 55.2% / assume_ai 62.8%, false fire 0.0% / 0.0% / 2.3% -- the extra recall was never free. A wrong relaxation only fills a small corner near-losslessly (the localize -> fill benign failure mode), which is what made a SMALL false-fire rate arguable; it was never a licence for a 60% one. Metadata provenance mapping (feeds `auto`, read by `cli._visible_provenance`): Google/Gemini C2PA issuer -> gemini; China-AIGC (TC260) label -> doubao/jimeng; `samsung_genai` -> samsung. **The `jimeng_pill` is CAPTURE-LESS** (`pill_engine.py`): the top-left "AI生成" label has no captured alpha map, so it is detect-by-synthetic-silhouette; its footprint is a fixed top-left geometry box. Its weak edge-NCC detector (~7% raw false-fire) is gated in `remove_auto_marks` via **`_keep_pill`** (32k real-upload corpus validation 2026-07): the pill never rides on a **Doubao** detection, and has confirmation arms because metadata/intent confirms the platform, not pill presence. **(1) Bottom-right "★ 即梦AI" wordmark fired** — ~94% precise and survives **metadata-STRIPPED uploads** (screenshots / re-saves, ~61% of pills carry a detectable wordmark): remove **unrestricted**. **(2) TC260 metadata confirms Jimeng** (`"jimeng" in provenance`, no wordmark) — the metadata-only arm is only **21% precise raw (CI 16-29%), 29% (CI 20-40%) among the flat footprints the guard PASSES** (re-measured 2026-07-18, 149 blind-labelled fires) and its false fires are **textured ceilings/walls that the fill visibly SMEARS**, so remove **only when the top-left footprint is flat enough for an invisible fill** (`pill_engine.footprint_is_flat`, median-Sobel texture ≤ `_FLAT_TEXTURE_MAX`) — the flatness guard always holds. This keeps real flat-scene pills (incl. metadata-only ones the wordmark misses) plus harmless flat false fires, and leaves the damaging textured false fires untouched. Do NOT drop the wordmark arm or loosen the flatness guard. **`assume_ai` was REMOVED (2026-07-19); `--sensitivity` is now `auto`/`strict` only.** It relaxed every mark's FP gate on the bare assertion an image is AI -- which names no vendor and no location, exactly what the bypass requires -- and had no place in the model (detector finds -> remove; finds nothing -> leave alone; user SEES a mark -> act on that). It took `_ASSUMED_CONF_FLOOR` / `assumed_floor_ok` / the `assumed` trust level with it, collapsing the ladder to `strict`/`confirmed`, and `_keep_pill` lost its `sensitivity` arg. Recall/precision on the unbiased sample are unchanged, so nothing on the default path moved. **Replacement advice is per mark:** `erase --region` is sound by construction; `--mark --no-detect` is reasonable (forced mask = the real glyph blob, non-empty 13/13); **`--mark gemini --no-detect` is NOT** -- it falls back to a fixed slot that covered the true sparkle on only **31% of 97** missed sparkles, so 69% fill a clean corner AND report a removal that did not happen. `cli._no_visible_mark_exit` follows that order and no longer suggests the removed mode. Migration raises loudly (`validate_sensitivity`, called from `api.remove_visible` and `Context.__post_init__`) because a `Literal` is unenforced at runtime and would silently downgrade a 0.15 caller to `auto`. **Detection can skip binarization** (`TextMarkConfig.detect_frontend`): `binary` thresholds the top-hat into a glyph blob (the original path), `tophat` correlates the CONTINUOUS top-hat against a soft template, turning the saturation/luma gates into weights and max-normalizing so the score is contrast-invariant. **Doubao uses `tophat`** (recall 89% -> 92% at an unchanged 99% precision on the unbiased sample); jimeng/samsung stay `binary` until measured per mark. **Detection and the REMOVAL MASK must use the SAME front-end.** `tophat` detection does not binarize, but `extract_mask` (which bounds the fill) still does, so a mark faint enough to be found only by the continuous response produced an EMPTY mask: `localize` returned None, `remove()` was a silent no-op, and `identify` reported `visible_doubao` while `visible` said "no visible mark" on the same file. Corpus-measured 2026-07-20: 57 of 60 sampled still-detected Doubao marks were untouched, ~8% of all its detections. When the binary blob is empty on a `tophat` mark the detector fired on, `footprint_mask` now bounds the fill with **the detector's OWN best-match box** (`_tophat_best`, the shared method whose score gates detection and whose box bounds the mask -- one method, so the two can never drift). **Gated on an actual detection** (on a clean corner the box would be spurious). The FIRST version of this fix (2026-07-19) instead thresholded the max-normalized uint8 response at `_FAINT_GLYPH_LEVEL = 0.5`, which selects every non-zero pixel (NOT "half the peak" as its comment claimed) and filled ~120% of the corner box on textured frames; corpus-measured 2026-07-20 over 14 real faint-path frames, the match box fills 58.7% and the whole-corner threshold 120.9%, both 100% detector-clean. So the threshold LOOKED fixed on parity (a mask that fills everything is trivially detector-clean) and was only caught by measuring fill AREA. The `0.5` constant is gone. Regression: `tests/test_text_mark_faint_mask.py` (its fixture MUST carry texture -- on a flat frame the response is non-zero only on the glyph, so every threshold gives the same box and the test cannot see a wrong one; mutating the old constant to 99.0 left the flat test green). **Any future front-end change must move BOTH paths or re-check this.** **The gate is front-end specific -- re-calibrate, never port it**: the continuous response scores higher (mean 0.809 vs 0.723), so the binary-era 0.40 ran 96%/91% (8 false fires) while 0.50 runs 92%/99% (1). A first pass at 0.40 also silently halved the PILL's recall, since `_keep_pill` suppresses it whenever doubao fires. The front-end fixes DETECTION of faint marks but not ATTRIBUTION: 千问 becomes separable from clean corners (AUC 0.92) yet not from Doubao (**AUC ~0.5**) because they share the `AI生成` tail. Next design is a GENERIC CJK-AI-mark detector (GB 45438-2025 guarantees the shared tail) with vendor attribution as optional metadata. **Adding a new text mark is only cheap when the mark is stamped BOLD** -- 千问/星绘 are deliberately NOT registered. Measured on 14 corpus positives with the same pipeline and each mark's own template: doubao mean NCC **0.723** (82% over the 0.40 gate) vs qwen **0.170** (**0%**). Ruled out by measurement, in order: the synthetic render (a template cut from a REAL Qwen mark scores 0.307 vs the rendered 0.308, and real masks do not match each other), the fixed 5px morphology kernels (+0.014 mean when scaled with box height), and the appearance thresholds (best sweep: mean 0.35, 4/14 over the gate). The blocker is SEGMENTATION on a faint mark: the white top-hat shatters a thin translucent overlay into specks, and no template matches a blob that is not there. Adding it needs a front-end that does not binarize the glyph first (grayscale/edge correlation on the raw top-hat, or a learned patch classifier). **Check a candidate mark's contrast before promising it is a config-plus-silhouette job.** Evidence chain in `scripts/render_vendor_silhouettes.py`. **RECALL is measured on an unbiased random sample** (`scripts/visible_recall_sample.py`; 240 images drawn at random per provenance class and labelled exhaustively, 2026-07-18): doubao **89% recall / 99% precision**, gemini **96% / 80%**, jimeng 71%/71% (n=14), jimeng_pill 50%/60% (n=6). The `scale_basis` fix moved doubao recall **71% -> 89%** on this same sample. **Gemini's real precision is 80%, NOT the 41% `visible_eval.py` reports** -- that harness scores an addition-sampled set, so it measures the relaxation arm's marginal cases, not what production sees; quote 80% for the product. Landscape is improved but unsolved (doubao by aspect: portrait 92% / square 92% / landscape 79%). The largest remaining gap is not tuning but COVERAGE: **6% of sampled images carry an uncovered vendor's mark** (千问/百度/星绘/抖音-class) that no registered detector can fire on -- researched specs are in `docs/watermarking-landscape.md`. **Mark geometry scales with a PER-MARK dimension** (`TextMarkConfig.scale_basis` / `TextMarkEngine.scale_base`): doubao `short` (= min(h,w)), jimeng and samsung `width`. All the tuned fractions were calibrated on PORTRAIT captures where width == short side, so the basis went unexercised until landscape inputs were measured -- and **doubao detected 0 of 435 landscape TC260 images, a 100% miss rate**. It is a LOCALIZATION failure, not a threshold one (median doubao NCC on the 1452 no-detection images was 0.057, only 2.7% in a threshold-reachable band), so no amount of gate tuning could reach it. Short-side geometry recovers **56% of the previously-undetected landscape set**. The basis is per-mark because the SAME switch took jimeng's landscape positives from 13/13 to 0/13 -- its wordmark tracks the width even though both marks are ByteDance and share a corner; samsung stays `width` because it is unmeasured (1 addition corpus-wide). GB 45438-2025 5.2(e) mandates glyph height >= 5% of the shortest side, which is why short-side is the prior -- but measurement overrides the standard's wording. This was invisible for months because **precision was measured repeatedly and recall never was**; the harness now reports a `missed` column, which is what caught the jimeng regression the fix introduced. **Detection among same-corner marks is COMPETITIVE** (`TextMarkConfig.rivals` / `_rival_margin_ok`): a mark's template must beat every same-corner rival's on the SAME glyph blob by `rival_margin` (0.10). Absolute-only scoring could never separate Doubao and Jimeng (both bottom-right near-white CJK, near-identical after binarization) -- measured separability: absolute `ncc_jimeng` 0.96, `ncc_jimeng` MINUS `ncc_doubao` **0.99**. Corpus effect: **jimeng precision 38% -> 63% with genuine detections unchanged (false fires 65 -> 23)**, so it is a pure precision gain and the earlier 0.85 threshold patch was reverted to 0.70. **Asymmetric by measurement:** doubao declares NO rival -- the symmetric gate cost it 7 genuine detections to prevent 5 false (1.4:1 against) while jimeng gained 25pp for free. **Benchmark any detector change with `uv run python scripts/visible_eval.py --vs `** (741 blind-labelled corpus images; `scripts/visible_groundtruth.py` builds the set, `scripts/visible_sheets.py` makes new labelling rounds). Three harness rules are load-bearing: score a mark only within its crop's **adjudication scope**, take **provenance from metadata not from labels** (label-derived provenance scored gemini at 99% vs the true 41%), and **never report recall** from this set -- it was sampled where detectors fired, so an unbiased random sample is still needed. **The provenance NCC relaxation is PER MARK (`TextMarkConfig.provenance_ncc_factor`), not one shared multiplier** — measured 2026-07-18 on the default `auto` path over 4417 unique TC260 carriers (blind hand-label, two-sided control, labeller sensitivity 100%/96% and specificity 100%/100%), the old shared 0.7 ran at **76% precision on doubao but 17% on jimeng**. Doubao stays 0.70 (both its bands return more true marks than false fills). Jimeng moves to **0.85**: its relaxed silhouette keys on "text in the bottom-right corner" rather than the wordmark — of 68 false additions **33 were DOUBAO marks** and 17 were other vendors' AI labels, and 45 of the 68 filled a corner nothing else would touch; 0.85 costs 8 genuine recoveries to prevent 60 false fills (7.5:1), lifting the arm to 43%. That is a patch on a DETECTOR problem — jimeng's silhouette is not discriminative against doubao's, and no threshold fixes that. **A weak mark must not CORROBORATE a sibling** (`_CANNOT_CORROBORATE`): sibling corroboration grants `confirmed` trust, which bypasses the sibling's FP gate, so the pill (~7% raw false-fire) handing that bypass to jimeng created a closed loop on the DEFAULT path — pill false-fires → jimeng relaxes and false-fires → `_keep_pill`'s wordmark arm then removes the pill UNRESTRICTED, skipping the flatness guard (3/578 negatives ran the full loop, one with `footprint_flat=0`). Cutting the pill out of corroboration removed all 3 and cost NOTHING on the TC260 carriers (jimeng 398 → 398). `_keep_pill` already distrusted the pill's ACTION; this closes the gap that its TESTIMONY was ungated. `cli._write_bgr_with_alpha` must NOT zero alpha in the watermark bbox (issue #30 white-box regression). **The localizer is cheap CPU (cv2/numpy), so a memory-tight caller runs it anywhere; the heavy MI-GAN/LaMa fill is opt-in and chosen by the caller** (a small worker can use cv2; a GPU/model worker can use MI-GAN/LaMa). Adding a new mark needs only a DETECTION silhouette (removal is template-free — the glyph-blob bbox is filled, no capture involved). Produce that silhouette SYNTHETICALLY: font-render the mark's glyphs (the pill's `scripts/render_pill_silhouette.py` is the pattern; commit the rendered PNG under `assets/`) and calibrate the NCC threshold on real positives. The old solid/gray/white app-capture workflow (`scripts/visible_alpha_solve.py`) is RETIRED with reverse-alpha — existing marks still carry their captured silhouettes, but a NEW mark does NOT require captures. (The 2026-06-22 "synthetic reconstruction below the quality bar" objection was about reverse-alpha PIXEL recovery, which is gone; it does not apply to a synthetic detection silhouette.) Data-safety still binds the committed asset: the silhouette must be font-rendered synthetic, never derived from user uploads — seeing a real sample to learn the glyphs / font / position / locale is fine, but the committed template stays synthetic. So nothing is parked for lack of a capture: Meta AI and more Samsung locales just need the glyphs + font + locale + calibration positives; any Grok visible mark additionally needs confirming it even HAS one (its known signal is EXIF-only `xai_signature`). - `gemini_engine.py` — visible Gemini-sparkle detector + localizer (cv2/numpy, no GPU): top-K size-weighted fusion candidate selection (`_SELECT_TOPK`), corner-promote, false-positive gate (the provenance prior relaxes the gate + lowers the trust threshold when a Google/Gemini C2PA issuer confirms the vendor). **White-core rescue:** the FP gate demotes a low-gradient match (soft edges), but a real FAINT sparkle also has soft edges -- so the gate keeps a low-grad match that is a strong (conf ≥ `_SPARKLE_KEEP_CONF` 0.52), bright (margin), near-WHITE-core sparkle (`_core_saturation` ≤ `_SPARKLE_WHITE_SAT` 0.20): a real sparkle core is white, a clean bright corner that shape-matches (sky/sun) is colored. This recovers ~14/20 metadata-stripped faint sparkles under the DEFAULT strict/auto (no flag, no metadata) at ~1.25% clean false-fire (baseline 0.55%); the ~0.51-scoring bright-bg FPs stay demoted (below 0.52). A learned classifier on the SAME features was measured WORSE than the tuned gate (2026-07 tier-1: MLP 86.7% recall vs 90.8% at equal FP), so the heuristic stays; a patch-CNN with richer features is the only lever left (roadmapped P2, low expected value -- the wall is fundamental). Detection scores the top-K size-weighted matches by full fusion (spatial+gradient+variance) and keeps the highest — NOT the raw-NCC argmax, which re-admits the tiny-patch FPs the size weight suppresses (the osachub 2026-06-12 sub-0.85 corner-sparkle regression; see `docs/module-internals.md`). Keep the 0.85 corner-promote NCC gate; a margin/chroma-gated lower promote was measured and REJECTED 2026-06-11 (~33% FP on non-Google content). Removal is localize -> fill: `footprint_mask` returns the sparkle footprint (the captured alpha thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin), and the shared `watermark_registry.fill` inpaints it. The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. `detect_sparkle_confidence` reuses a process-wide `_shared_engine()` singleton (lru_cache) — the engine holds only constant assets (captures, alpha maps, a precomputed 16..118 template ladder) and takes the image as an arg, so do NOT reconstruct `GeminiEngine()` per call: that reloaded assets + recomputed alpha maps + rebuilt the template cache on every one of ~34k `identify` calls (−24% on the sparkle path once made a singleton, output byte-identical). `detect_watermark`/`footprint_mask` guard `image.size == 0` before `to_bgr`, and return an empty (detected=False) result when no template scale fits (short side < 16 px), rather than dereferencing an empty candidate list. - `_text_mark_engine.py` — shared base for the three text-mark engines (extracted 2026-06-09); the per-engine modules are config-only subclasses. Detection still matches the glyph silhouette (NCC, keys on glyph shape). The removal mask is TEMPLATE-FREE: it is the bounding box of the top-hat glyph blob (`extract_mask`), filled solid + dilated, so the shared fill inpaints the whole wordmark rectangle. This drops the fixed alpha-template placement, so a re-rendered or differently-placed mark is still masked; the captured alpha maps are now used only for the detection silhouette, not for removal. New text mark = a `TextMarkConfig` + a thin subclass + one registry row. Gemini stays a separate engine (different model). - `pill_engine.py` — the CAPTURE-LESS Jimeng-basic "AI生成" pill (top-left, issue #54). No alpha map: `detect` is edge-NCC of a synthetic font-rendered silhouette (`assets/jimeng_pill.png`, regenerate via `scripts/render_pill_silhouette.py`; committed, data-safe -- corpus stays out of the repo) in the top-left ROI, calibrated on 61 local real positives to threshold 0.22; `footprint_mask` is a generous FIXED top-left geometry box (NOT the NCC match position -- the synthetic silhouette localizes only approximately, the corner is negative space, so a geometry box fills cleanly while a match box leaves outline residue). `footprint_texture`/`footprint_is_flat` (median-Sobel over that box, `_FLAT_TEXTURE_MAX`) back the metadata-only safe-fill gate. Removal is the shared localize -> fill (MI-GAN/cv2). Detector precision is weak (~7% raw false-fire), so it is registry-gated in `remove_auto_marks` via `_keep_pill`: never on Doubao; the bottom-right wordmark removes it unrestricted (~94% precise, survives metadata-STRIPPED uploads); TC260-metadata-only removes it ONLY on a flat footprint (its textured false fires -- ceilings/walls -- are what the fill smears). Do NOT loosen those gates. diff --git a/docs/module-internals.md b/docs/module-internals.md index a66e4be..b7a3075 100644 --- a/docs/module-internals.md +++ b/docs/module-internals.md @@ -92,6 +92,8 @@ Doubao is switched to it; jimeng and samsung stay `binary` until measured, becau **The gate is FRONT-END SPECIFIC and must be re-calibrated, not ported.** The continuous response scores higher overall (mean 0.809 vs 0.723 on the same 90 positives), so the binary-era 0.40 left the provenance-relaxed gate (x0.7) far too low: at 0.40 the arm ran 96% recall / 91% precision (8 false fires), at 0.50 it runs 92% / 99% (1 false fire). 0.50 was chosen because it beats the binary front-end on recall at IDENTICAL precision -- a front-end that only trades one for the other would not have been worth shipping. A first pass at 0.40 also silently depressed the PILL (recall 50% -> 33%), because `_keep_pill` suppresses the pill whenever doubao fires; a coupling worth remembering when tuning any bottom-right mark. +**The removal MASK must ride the same front-end, and how it does so was fixed twice.** `tophat` detection does not binarize, but `extract_mask` (which bounds the fill) still does, so a mark faint enough to be found only by the continuous response produced an EMPTY binary blob: `localize` returned `mask=None`, `remove()` was a silent no-op, and `identify` reported `visible_doubao` while `visible` said "no visible mark" on the same file (corpus-measured 2026-07-20: 57 of 60 sampled still-detected Doubao marks untouched, ~8% of its detections). The FIRST fallback (2026-07-19) thresholded the continuous response and took the bounding box of everything above the level -- but the level was `0.5` compared against the max-normalized **uint8 0..255** response, so it selected every non-zero pixel and filled ~120% of the corner box on textured frames (a padded whole-ROI box). It passed parity (a mask that fills everything is trivially detector-clean) and its regression test (a FLAT fixture, where the response is non-zero only on the glyph, so every threshold yields the same box). The SECOND fallback (2026-07-20) uses the detector's OWN best-match box instead: `_tophat_score` was split into **`_tophat_best(image, loc) -> (score, box)`**, the single method whose score gates detection and whose argmax box bounds the mask -- so the two cannot drift by construction, which is how the mismatch arose in the first place. Measured over 14 real faint-path frames (cv2 fill, detector re-run after): the match box fills a **58.7%**-median corner box vs the threshold's **120.9%**, both 100% detector-clean. The largest-connected-component alternative was tighter (10.5%) but removed the mark on only 21% of frames, so it does not cover it and was rejected. Regression: `tests/test_text_mark_faint_mask.py`, whose fixture now carries texture (the flatness of the old one is exactly why it could not see the threshold bug -- mutating the constant to 99.0 left it green). **Any future front-end change must move both the detection and the mask path, or re-check this.** + **What this does NOT solve: vendor ATTRIBUTION for the shared-suffix marks.** With the continuous front-end 千问 becomes separable from clean corners (AUC 0.92) but NOT from Doubao (**AUC 0.41-0.59, i.e. a coin flip**), because "千问AI生成" and "豆包AI生成" share the `AI生成` tail -- three of five glyphs, same face, same corner. So the front-end removes the *detection* blocker and exposes an *attribution* one. Since removal is identical for either (localize the glyph blob -> fill), the natural next design is a GENERIC "CJK AI-generation text mark" detector covering 千问/百度/星绘/小云雀/TRAE and any future GB 45438-2025-compliant vendor in one template, with per-vendor attribution treated as optional metadata rather than a detection requirement -- the standard mandates that every compliant string contain (人工智能|AI) and (生成|合成), so the shared tail is guaranteed. That needs its own precision-labelling round before it ships. **Why 千问 / 星绘 are NOT registered (measured 2026-07-18).** Adding a text mark is documented as "a `TextMarkConfig` + a thin subclass + one registry row", and that is true only when the mark is stamped like Doubao's. It does not hold for a FAINT mark, and 千问 is the counter-example. Measured on 14 hand-verified corpus positives, same pipeline, each mark scored with its OWN template: diff --git a/docs/verification-plan.md b/docs/verification-plan.md index 84cebef..55f35ff 100644 --- a/docs/verification-plan.md +++ b/docs/verification-plan.md @@ -358,12 +358,66 @@ optimizing PSNR, and a faint mark is still a mark. What it says is that the fill cost, it is now measurable, and for faint marks it exceeds the thing it removes -- which makes "how faint is too faint" a product decision that can finally be made on evidence. -### B2. Detector response curves +### B2. Detector response curves -- RUN 2026-07-20 -Stamp marks across a controlled grid -- size, contrast, background texture, aspect, -JPEG quality -- and measure detection rate per cell. Produces a recall **curve** instead of -a point estimate, and directly tests the `scale_basis` geometry that hid a 100% landscape -miss for months. Cheap, repeatable, no human in the loop. +`scripts/detector_response.py`. Stamp marks across a controlled grid (size x contrast, +crossed with real corpus backgrounds and frame aspects) and measure two things per cell: +`detected`, and whether the same call path then yields a non-empty removal mask +(`maskable`). The second column exists because the gap between them is a silent no-op -- +`identify` reports a mark that `visible` skips -- and any detection-only harness scores +that bug as a success. + +**Read the nominal cell, never the aggregate.** The grid deliberately visits sizes and +opacities no engine was calibrated for, so its overall rate is an adversarial score, not +recall. Production recall still comes from the unbiased corpus sample. + +What it found: + +- **The size response is a COMB, not a curve.** `_tophat_score` sweeps exactly three rungs + (0.8, 1.0, 1.25). Doubao scores ~0.99 at each rung and collapses to 0.37-0.48 between + them, against a 0.50 gate -- so a mark ~10% off a rung is missed at FULL contrast. The + `binary` front-end has no ladder at all: jimeng holds one lobe over 0.90-1.20, samsung + only 0.95-1.05, i.e. samsung requires a mark at essentially its exact nominal size. +- **Contrast is nearly irrelevant** on the tophat front-end -- the response is + max-normalized, so a mark at 15 luma levels of contrast still scores 0.984. +- **No detected-but-unmaskable cells** at any grid point, so the front-end/mask parity fix + holds across the whole operating range, not just where the corpus happened to look. + +And the follow-up that stopped a bad change: dead zones only cost recall if real marks land +in them. `scripts/ladder_headroom.py` measured that on the corpus (positives = frames whose +metadata names the vendor, negatives = frames with no metadata signal, deliberately NOT +filtered on the detector's own verdict, which would make its false-fire rate 0 by +construction). A 13-rung dense ladder recovers **28 of 368 misses (7.6%)** while false fire +goes **2.52% -> 3.05%**. So the comb is real and mostly unvisited: the fractions were +calibrated on real captures and real marks cluster at the rungs. **Do not densify the +ladder** on this evidence. + +The residual looked like a clean lead and, on a full 923-positive / 3546-negative run, +turned out not to be. A targeted `plus_one` ladder (one extra rung at ~1.116) recovers 36 +of the 39 marks the 13-rung dense ladder recovers -- so the win really is that one rung, +not density. But **all 36 recoveries AND all 18 of its added false fires are LANDSCAPE at +that same rung** (2.0:1 overall, 1.7:1 even gated to landscape). The rung is not vendor +signal; it is a size shift that helps and hurts equally. + +And the obvious "fix it at the source" -- bump the landscape width fraction so that +subpopulation lands on the nominal rung -- is **falsified by the same data**: the 55 +currently-DETECTED landscape positives already peak at scale 1.0, not 1.116. So there are +two size clusters of landscape doubao marks, one at the calibrated nominal and one ~11% +larger, and moving the nominal would drop the cluster that works to catch the one that does +not. The larger cluster is genuinely a different size and is inseparable from landscape +false fire at the NCC gate -- the same detector-discrimination wall as vendor attribution, +not a geometry constant anyone forgot to set. **Do not add the rung, and do not move the +landscape fraction.** The per-rung scores are stored in `_ladder_headroom_doubao.jsonl` so +this verdict can be re-derived without re-running the sweep. + +Why the misses are not a tuning problem: on the sampled misses the dense-ladder score is +bimodal -- detected marks sit at median 0.938, misses at 0.155, and **the band 0.31-0.52 is +empty**. There is no near-gate cluster, so no threshold recovers them. Eyeballing 26 miss +corners explains it: ~6 are jimeng (TC260 names no vendor, so a "doubao positive" is often +a jimeng frame), ~14 carry no visible mark in that corner at all, 2 belong to **uncovered +vendors** (`千问AI生成`, `百度 AI生成`), and only ~4 are genuine doubao failures -- on +saturated or structurally busy corners, with heterogeneous causes (the saturation gate +explains exactly one of five tested). ### B3. Invisible round-trip, positive-control gated @@ -536,8 +590,55 @@ would take, so none of it has to be rediscovered. |---|---|---|---| | 1 | 16-bit PNGs are downconverted to 8-bit by a metadata strip | 42 of 27,018 corpus PNGs (0.16%); one went 9.2 MB -> 2.5 MB | a byte-level PNG chunk stripper, so the PIL re-save is skipped entirely | | 2 | Exit code 2 means three different things (no visible mark / no invisible signal / Click usage error) | any wrapper must parse stderr to tell them apart | split the codes; **breaking for existing wrappers**, so it needs a deliberate call | -| 3a | the full visible-parity sweep has NOT been re-run since the doubao front-end fix | doubao parity should move 91.8% -> ~99%, unconfirmed | re-run `visible_removal_audit.py --paths-file data/spaces/_visible_positives.txt --backend cv2` (~2 h) | +| 3a | ~~the full visible-parity sweep has NOT been re-run since the doubao front-end fix~~ **DONE 2026-07-20** | doubao parity moved **91.8% -> 99.3%** (2562/2580) as predicted; gemini/jimeng/samsung unchanged, `_visible_parity_cv2_v2.csv` | closed | | 3 | `visible_removal_audit.py` measures the UNGATED per-mark path | reports the pill at 32% where the product runs at 100% precision | teach it the product path (`remove_auto_marks`) for gated marks, or at minimum say so loudly in its docstring | +| 4 | the faint-mask fallback fills the WHOLE corner box, not the glyph | 12 of 12 real faint-path frames covered 100% of the corner ROI (120.9% median with padding); the path fires on ~8% of doubao detections | fixed 2026-07-20 -- see below | + +**Defect 4 in full, because it is instructive.** The fallback I added on 2026-07-19 reads +`np.where(resp >= _FAINT_GLYPH_LEVEL)` with the constant at `0.5`, and its comment says +"thresholded relative to its own peak". But `tophat_response` returns **uint8 0..255**, so +`>= 0.5` selects every pixel with value >= 1 -- the entire non-zero response, not half the +peak. Measured against alternatives on 14 real frames (cv2 fill, detector re-run after): + +| mask | detector clean after | median filled area (% of corner box) | +|---|---|---| +| `thr0.5` (as shipped) | 100% | 120.9% | +| `thr190` | 100% | 68.5% | +| largest connected component at 190 | **21%** | 10.5% | +| **the detector's own best-match box** | 100% | **58.7%** | + +The fix is the last row: the correlation already located the mark at a position and scale, +so thresholding its response was always a weaker proxy for information we had. The +connected-component variant is rejected outright -- it is the tightest but removes the mark +on only 21% of frames, i.e. it does not cover it. + +Two things about how this was found are worth keeping: + +- **Parity could not see it.** Parity asks whether the detector is clean after removal, and + a mask that fills everything passes trivially. The defect was in the COST, and nothing + measured cost on that path. A green parity run is not evidence about mask size. +- **The regression test could not see it either, by construction.** Its fixture is a FLAT + frame, where the top-hat response is non-zero only on the glyph, so every threshold gives + the same bounding box. Mutating the constant to an absurd 99.0 left it green. The fixture + now carries texture, which is the condition under which sizing matters and what real + corner backgrounds look like -- and it reproduces the corpus number exactly (127% of the + corner box, against 120.9% measured). + +### Latent, pre-existing, not fixed this pass + +The detect-fires / mask-empty silent no-op that fix 4 closed on the `tophat` front-end has a +**narrower cousin on the `binary` front-end** (jimeng/samsung), surfaced by the /simplify +altitude review. Binary detection gates on `coverage >= detect_min_coverage` (a FRACTION) +while the mask gates on `xs.size >= _MIN_GLYPH_PIXELS = 20` (an absolute COUNT), both on the +same blob. For samsung (`detect_min_coverage = 0.01`) they disagree in a small-image band +(width ~200-294 px): detection can fire at 10-19 glyph px while the 20-px mask floor returns +None -> the identical observable. It is NOT introduced by this work (the `else: return None` +fall-through predates it), it sits well below real mark sizes (captured positives are +1086-2048 px wide), and fixing it means changing binary detection's gate to match the mask's +-- which needs its own per-mark measurement. So the "cannot drift by construction" claim is +scoped to `tophat` (where score and box are one computation); the binary path is coupled but +by a threshold pair that can still disagree at the edges. Fix only alongside a binary-mark +detector change, never on its own. ### Dependency alert @@ -549,10 +650,31 @@ patched torch version exists -- do not re-triage it", which is now stale. Either (it is transitive from the optional `gpu` extra) or re-dismiss on the remaining grounds (the codebase never calls `torch.jit`, grep-verified) and correct that note. +### Where detection work should go next + +Measured this session, in the order the evidence supports: + +1. **Not the ladder, not the threshold, not the landscape rung.** All three were measured + to completion and all three are dead ends: the dense ladder buys 7.6% of misses for a + 21% relative rise in false fire; the score band below the gate is empty so no threshold + recovers the misses; and the one targeted rung that helps (1.116, landscape) adds false + fire at 1.7:1 because the recoveries and the false fires are the same landscape size + shift. Moving the landscape width fraction is also out -- detected landscape marks + already sit at the nominal, so it would break more than it fixes. Do not spend here. +2. **Coverage of uncovered vendors is the largest lever** and is blocked on EVIDENCE, not + architecture. `千问` and `百度` marks sit in the same corner we already scan, and the + front-end that `render_vendor_silhouettes.py` said was missing now exists. But this + session found exactly one confirmed positive per vendor, and the 14 千问 positives that + note quotes are not reachable from any current script. Nothing may be registered off a + single frame. Harvest 30+ per vendor with `scripts/cjk_tail_probe.py`, then calibrate. +3. **A generic shared-tail template is not a shortcut.** `AI生成` is guaranteed across + compliant vendors by GB 45438-2025, so one template covering all of them is the obvious + idea -- and measured on the tophat front-end it separates a bold 千问 positive from clean + corners by only 0.407 vs a clean p99 of 0.298. A 4-glyph run is simply less specific + than a 6-glyph one. Treat it as a harvesting aid, not a detector. + ### Verification tiers not run -- **B2 detector response curves** -- recall as a function of size, contrast and background - texture on stamped marks. No labelling needed. - **B4 resource ceilings** -- peak RSS and wall time per backend x input size to 25 MP. - **E robustness** -- truncated, corrupt, absurd dimensions, decompression bombs, unicode and RTL filenames, read-only output dirs, concurrent runs on one file. @@ -560,11 +682,11 @@ patched torch version exists -- do not re-triage it", which is now stale. Either ### Recommended next step -**B2, before any detector work.** Jimeng recall rests on n=14 and the pill's on n=6; -improving what six samples measure means not knowing whether it improved. B2 is also the -instrument that catches the geometry class of bug that has now surfaced twice -- the -`scale_basis` landscape miss, and the detect/mask front-end mismatch that made ~8% of -Doubao detections unremovable. +**Harvest labelled positives for the uncovered vendors.** Everything cheaper has now been +measured and found empty, and every remaining question -- can 千问 be registered, what gate, +does the landscape rung generalize -- is blocked on the same missing thing: labelled +examples. Jimeng recall still rests on n=14 and the pill's on n=6, so those are equally +unimprovable-because-unmeasurable. ## Standing gap diff --git a/scripts/cjk_tail_probe.py b/scripts/cjk_tail_probe.py new file mode 100644 index 0000000..f969796 --- /dev/null +++ b/scripts/cjk_tail_probe.py @@ -0,0 +1,272 @@ +"""Can one GENERIC template cover the CJK AI labels no per-vendor detector fires on? + +THE OPPORTUNITY + Corpus inspection of doubao-provenance misses turned up `千问AI生成` (Alibaba Qwen) and + `百度 AI生成` (Baidu) sitting in the same bottom-right corner as the marks we do cover, + bold and plainly legible, with no detector able to fire on either. `docs/...landscape` + puts uncovered vendors at ~6% of sampled images -- larger than any tuning gain left in + the covered ones (a dense scale ladder recovers 1 miss in 26; see ladder_headroom.py). + +WHY A SHARED TEMPLATE IS EVEN PLAUSIBLE + China's GB 45438-2025 mandates the label, and every compliant vendor ends it with the + same run: `AI生成`. The vendor PREFIX varies (豆包 / 千问 / 百度 / 星绘) and is what makes + per-vendor silhouettes expensive; the TAIL is guaranteed. So the tail is the part worth + detecting, and vendor attribution -- which the shared tail cannot provide anyway + (measured 千问-vs-doubao AUC ~0.5) -- moves to metadata, where it is reliable. + +WHY IT IS WORTH RETRYING NOW + `render_vendor_silhouettes.py` records 千问 as ruled out, and its reasoning was sound: + the then-current front-end binarized the glyph, and a thin translucent overlay shatters + into specks that no template can match. It closes by naming what would be needed -- + "a detection front-end that does not depend on binarizing the glyph (grayscale/edge + correlation on the raw top-hat)". That front-end now EXISTS: `detect_frontend="tophat"` + was built for doubao and correlates a soft template against the continuous response. + The blocker was removed by unrelated work, so the ruling deserves re-measurement rather + than inheritance. + +THE MEASUREMENT + Three arms, one scorer (the tail template on the tophat response, swept over a dense + size ladder because an unregistered vendor's glyph size is genuinely unknown -- the one + place a dense ladder earns its cost): + + covered a registered mark already fires -- a sanity arm; the tail is inside those + marks too, so a tail template that cannot score THESE is simply broken + uncovered TC260 provenance but no detector fires -- the target population + clean no metadata signal at all -- the precision arm + + The decision rests on whether `uncovered` carries a high-scoring subpopulation that + `clean` does not. It does NOT rest on the raw rate: TC260 provenance does not imply a + visible mark, so a modest fire rate on that arm is expected even if the detector is + perfect. Any threshold this suggests must then be confirmed by eyeballing the crops it + selects -- the script writes a contact sheet for exactly that. + +DATA SAFETY + Corpus images are user uploads: read-only, local analysis, gitignored output. The + template is font-rendered synthetic, never cut from a user upload. + + uv run python scripts/cjk_tail_probe.py --n 6000 +""" + +from __future__ import annotations + +import argparse +import glob +import json +import os +import random +import sys +from concurrent.futures import ProcessPoolExecutor, as_completed +from pathlib import Path +from typing import Any + +import cv2 +import numpy as np + +sys.path.insert(0, str(Path(__file__).parent.parent)) +sys.path.insert(0, str(Path(__file__).parent)) + +REPO = Path(__file__).resolve().parents[1] +CORPUS = REPO / "data" / "spaces" / "originals" +OUT = REPO / "data" / "spaces" / "_cjk_tail_probe.jsonl" +# Cached under the gitignored data dir, not in scripts/: this is a probe artifact, not a +# product asset. If the tail mark is ever registered, `render_vendor_silhouettes.py` is +# what writes the committed silhouette into src/.../assets/. +TAIL_PNG = REPO / "data" / "spaces" / "_cjk_tail_silhouette.png" + +# The tail is a fraction of a full vendor mark's width (`豆包AI生成` is ~5 CJK widths, +# `AI生成` ~3), and the prefix length differs per vendor, so the size is genuinely +# unknown here -- unlike a registered mark, whose fraction is calibrated. Hence a wide, +# dense ladder: this is the case a dense ladder is actually for. +TAIL_SCALES = tuple(round(0.30 * (1.08**i), 4) for i in range(18)) # ~0.30 .. ~1.15 + + +def build_tail_silhouette() -> np.ndarray: + """Font-rendered `AI生成` silhouette, cached next to this script (data-safe).""" + if TAIL_PNG.exists(): + img = cv2.imread(str(TAIL_PNG), cv2.IMREAD_GRAYSCALE) + if img is not None: + return img + from render_vendor_silhouettes import render + + sil = render("AI生成", width=200) + cv2.imwrite(str(TAIL_PNG), sil) + return sil + + +def tail_score(engine: Any, image: np.ndarray, sil: np.ndarray) -> tuple[float, float]: + """Best tail correlation over the size ladder; returns (score, winning_scale).""" + loc = engine.locate(image) + resp = engine.tophat_response(image, loc) + if resp is None: + return (0.0, 0.0) + c = engine.config + base = engine.scale_base(image) + ar = sil.shape[0] / max(1, sil.shape[1]) + best, best_s = 0.0, 0.0 + for s in TAIL_SCALES: + gw = max(12, int(c.alpha_width_frac * base * s)) + gh = max(6, int(gw * ar)) + if gw >= resp.shape[1] or gh >= resp.shape[0]: + continue + t = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA) + v = float(cv2.matchTemplate(resp, t, cv2.TM_CCOEFF_NORMED).max()) + if v > best: + best, best_s = v, s + return (best, best_s) + + +def _one(path_str: str) -> dict[str, Any] | None: + from remove_ai_watermarks.api import visible_provenance + from remove_ai_watermarks.doubao_engine import DoubaoEngine + from remove_ai_watermarks.identify import identify + from remove_ai_watermarks.image_io import imread + from remove_ai_watermarks.watermark_registry import detect_marks + + path = Path(path_str) + try: + prov = visible_provenance(path) + except Exception: + return None + tc260 = bool({"doubao", "jimeng"} & prov) + + if not tc260: + try: + if identify(path, check_visible=False).signals: + return None # some other provenance: neither target nor clean control + except Exception: + return None + + img = imread(path_str) + if img is None or min(img.shape[:2]) < 200: + return None + try: + fired = [d.key for d in detect_marks(img) if d.detected] + except Exception: + return None + + if fired: + arm = "covered" + elif tc260: + arm = "uncovered" + else: + arm = "clean" + + score, scale = tail_score(DoubaoEngine(), img, build_tail_silhouette()) + return { + "src": path.name, + "arm": arm, + "fired": fired, + "tail_score": round(score, 4), + "tail_scale": scale, + } + + +def main() -> None: + ap = argparse.ArgumentParser() + ap.add_argument("--n", type=int, default=6000) + ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2)) + ap.add_argument("--out", type=Path, default=OUT) + ap.add_argument("--report-only", action="store_true") + ap.add_argument( + "--sheet-at", type=float, default=0.0, help="write a contact sheet of `uncovered` crops scoring >= this" + ) + a = ap.parse_args() + + if a.report_only: + rows = [json.loads(x) for x in a.out.read_text(encoding="utf-8").splitlines() if x.strip()] + report(rows) + if a.sheet_at: + contact_sheet(rows, a.sheet_at) + return + + build_tail_silhouette() + pool = glob.glob(str(CORPUS / "*" / "*")) + random.Random(19).shuffle(pool) # noqa: S311 -- deterministic sampling, not crypto + pool = pool[: a.n] + print(f"scanning {len(pool)} corpus files workers={a.workers}") + print(f"tail ladder {TAIL_SCALES[0]} .. {TAIL_SCALES[-1]} ({len(TAIL_SCALES)} rungs)\n", flush=True) + + rows: list[dict[str, Any]] = [] + a.out.parent.mkdir(parents=True, exist_ok=True) + with open(a.out, "w", encoding="utf-8") as fh, ProcessPoolExecutor(max_workers=a.workers) as ex: + futures = [ex.submit(_one, p) for p in pool] + for i, fut in enumerate(as_completed(futures), 1): + try: + rec = fut.result() + except Exception: # noqa: S112 -- one bad file must not kill the sweep + continue + if rec is None: + continue + fh.write(json.dumps(rec) + "\n") + rows.append(rec) + if i % 500 == 0: + fh.flush() + print(f" {i}/{len(pool)} usable={len(rows)}", flush=True) + report(rows) + if a.sheet_at: + contact_sheet(rows, a.sheet_at) + + +def report(rows: list[dict[str, Any]]) -> None: + arms = {k: [r for r in rows if r["arm"] == k] for k in ("covered", "uncovered", "clean")} + print(f"\n{'=' * 80}\nGENERIC CJK TAIL PROBE ({', '.join(f'{k}={len(v)}' for k, v in arms.items())})\n{'=' * 80}") + print("\nScore distribution of the shared `AI生成` tail on the tophat response\n") + print(f"{'arm':11s} {'n':>6s} {'p50':>7s} {'p75':>7s} {'p90':>7s} {'p95':>7s} {'p99':>7s} {'max':>7s}") + for k, v in arms.items(): + if not v: + continue + s = np.array([r["tail_score"] for r in v]) + qs = [np.percentile(s, q) for q in (50, 75, 90, 95, 99)] + print(f"{k:11s} {len(v):6d} " + " ".join(f"{q:7.3f}" for q in qs) + f" {s.max():7.3f}") + + if arms["clean"] and arms["uncovered"]: + clean = np.array([r["tail_score"] for r in arms["clean"]]) + unc = np.array([r["tail_score"] for r in arms["uncovered"]]) + print("\n\nOPERATING POINTS -- threshold set on the CLEAN arm, yield read on `uncovered`") + print("`clean` fires are the cost; `uncovered` fires are candidate recoveries that") + print("still need eyeballing, since TC260 provenance does not imply a visible mark.\n") + print(f"{'threshold':>10s} {'clean fire':>12s} {'uncovered fire':>16s} {'candidates':>12s}") + for q in (95, 97.5, 99, 99.5, 99.9): + t = float(np.percentile(clean, q)) + cf = float((clean >= t).mean()) + uf = float((unc >= t).mean()) + print(f"{t:10.3f} {100 * cf:11.2f}% {100 * uf:15.1f}% {int((unc >= t).sum()):12d}") + + cov = arms["covered"] + if cov: + s = np.array([r["tail_score"] for r in cov]) + print(f"\n\nSANITY: on frames where a registered mark fires, the tail scores median {np.median(s):.3f}.") + print("A tail template that cannot score these is broken, whatever it does elsewhere.") + + +def contact_sheet(rows: list[dict[str, Any]], thresh: float, limit: int = 30) -> None: + """Crop the corner of the top-scoring `uncovered` frames so the fires can be judged.""" + from remove_ai_watermarks.doubao_engine import DoubaoEngine + from remove_ai_watermarks.image_io import imread + + eng = DoubaoEngine() + picks = sorted( + (r for r in rows if r["arm"] == "uncovered" and r["tail_score"] >= thresh), + key=lambda r: -r["tail_score"], + )[:limit] + tiles = [] + for r in picks: + hits = glob.glob(str(CORPUS / "*" / r["src"])) + if not hits: + continue + img = imread(hits[0]) + if img is None: + continue + loc = eng.locate(img) + crop = img[loc.y : loc.y + loc.h, loc.x : loc.x + loc.w] + if crop.size: + tiles.append(cv2.resize(crop, (320, 96), interpolation=cv2.INTER_AREA)) + if tiles: + dest = REPO / "data" / "spaces" / "_cjk_tail_sheet.png" + cv2.imwrite(str(dest), np.vstack(tiles)) + print(f"\ncontact sheet ({len(tiles)} crops, score >= {thresh:.3f}) -> {dest}") + print("scores: " + ", ".join(f"{r['tail_score']:.2f}" for r in picks[: len(tiles)])) + + +if __name__ == "__main__": + main() diff --git a/scripts/detector_response.py b/scripts/detector_response.py new file mode 100644 index 0000000..928c82a --- /dev/null +++ b/scripts/detector_response.py @@ -0,0 +1,319 @@ +"""Tier B2: detector response curves -- recall vs mark size, contrast, background and aspect. + +WHY THIS EXISTS + Every recall number the project quotes rests on found positives: doubao 89% on n=240, + jimeng 71% on n=14, the pill 50% on n=6. Those tell you how the detectors do on the + marks a corpus happened to contain, and nothing about WHERE they fall over. Two bugs + have now shipped in exactly that blind spot: + + * `scale_basis` -- doubao detected 0 of 435 LANDSCAPE marks, a 100% miss, because + every fraction was calibrated on portrait captures where width == short side. + * the front-end mismatch -- detection moved to the continuous `tophat` response while + the removal mask still came from the BINARIZED blob, so ~8% of doubao detections + produced an empty mask and removal was a silent no-op. + + Both are geometry/plumbing failures at the edge of the operating range, and both were + invisible to a corpus sweep because the corpus is concentrated in the middle of it. + This harness constructs the edges instead of waiting for them to turn up. + +THE AGGREGATE HERE IS NOT RECALL + The grid deliberately visits sizes and opacities the engines were never calibrated for, + so a mark counted as missed at size=0.6 is not evidence of a weak detector -- it is the + point of the sweep. Quote the NOMINAL cell (size=1.0, alpha=1.0) as the reference, and + read the per-axis tables for shape. Production recall comes from an unbiased corpus + sample (`scripts/visible_recall_sample.py`), never from here. + +WHAT IT MEASURES -- TWO NUMBERS, NOT ONE + `detected` the detector fires. + `maskable` the same call path then yields a NON-EMPTY removal mask. + + The second is the one that has no natural home anywhere else. A mark that is detected + but not maskable is reported by `identify` and silently skipped by `visible` -- the user + sees a mark the tool says it removed. Anything that measures only detection scores that + bug as a success. They are reported side by side and their GAP is the headline. + +THE CONSTRUCTION + Take a verified-clean corpus image, stamp a real mark onto it with the forward model the + reverse-alpha work established (`stamped = (1-a)*bg + a*white`), and sweep two knobs the + engine's own geometry normally pins: + + size multiplies the glyph box the engine would use (a re-rendered or rescaled mark) + alpha multiplies the captured opacity (bold stamp -> faint translucent overlay) + + Background texture and frame aspect are not swept -- they come from the corpus and are + recorded, so the grid is crossed with real backgrounds rather than synthetic flats. + +WHY EVERY SOURCE ALSO RUNS UNSTAMPED + A recall curve with no false-fire baseline is uninterpretable in the same way a fill + PSNR with no damage baseline is: "detected on 70% of faint marks" is a triumph or a + scandal depending on whether the detector also fires on 70% of the clean frames those + marks were stamped onto. The control runs the identical call path on the identical + image with no stamp. + +READING `mask_hit` + Fraction of the stamped glyph box the removal mask covers -- real localization accuracy + for the text marks and gemini, because the mask has to be placed from a detection. + For `jimeng_pill` it is CONSTRUCTED and means nothing: the pill's footprint is a fixed + top-left geometry box, so it covers the stamp by definition. Reported, and flagged. + +DATA SAFETY + Corpus images are user uploads: read-only, local analysis, gitignored output under + data/spaces/. Records source filenames and measurements, never image content. + + uv run python scripts/detector_response.py --n 12 # trial, measures throughput + uv run python scripts/detector_response.py --n 150 # the real run, resumable +""" + +from __future__ import annotations + +import argparse +import collections +import json +import math +import os +import sys +from concurrent.futures import ProcessPoolExecutor, as_completed +from pathlib import Path +from typing import Any + +import numpy as np + +sys.path.insert(0, str(Path(__file__).parent.parent)) +sys.path.insert(0, str(Path(__file__).parent)) + +from fill_quality import SLOT_STAMPABLE, STAMPABLE, clean_sources, stamp_any, texture_of + +REPO = Path(__file__).resolve().parents[1] +OUT = REPO / "data" / "spaces" / "_detector_response.jsonl" + +# 1.0 = the geometry/opacity the engine's own constants assume. The sweep reaches below +# it (a mark rendered smaller, or a faint translucent overlay -- the class the tophat +# front-end was introduced for) and above it (a larger re-render, or an upscaled upload). +SIZES = (0.6, 0.8, 1.0, 1.3) +ALPHAS = (0.3, 0.5, 0.75, 1.0) +MARKS = (*STAMPABLE, *SLOT_STAMPABLE) + + +def aspect_of(image: np.ndarray) -> str: + h, w = image.shape[:2] + r = w / max(1, h) + return "portrait" if r < 0.9 else ("landscape" if r > 1.1 else "square") + + +def _probe(image: np.ndarray, mark_key: str, box: tuple[int, int, int, int] | None) -> dict[str, Any]: + """Run the product detect -> localize path once and score it. + + ``box`` is the stamped glyph box, or None for the unstamped control. + """ + from remove_ai_watermarks.watermark_registry import get_mark + + mark = get_mark(mark_key) + loc = mark.localize(image, force=False) + mask_px = 0 if loc.mask is None else int(np.count_nonzero(loc.mask)) + hit = None + if box is not None and loc.mask is not None: + x, y, w, h = box + inside = loc.mask[y : y + h, x : x + w] + hit = round(float(np.count_nonzero(inside)) / max(1, w * h), 3) + return { + "detected": bool(loc.detected), + "confidence": round(float(loc.confidence), 4), + "maskable": mask_px > 0, + "mask_px": mask_px, + "mask_hit": hit, + } + + +def _one_source(path_str: str) -> list[dict[str, Any]]: + """Every (mark, size, alpha) cell for one clean background, plus its controls.""" + from remove_ai_watermarks.image_io import imread + + base = imread(path_str) + if base is None: + return [] + name = Path(path_str).name + asp = aspect_of(base) + h, w = base.shape[:2] + rows: list[dict[str, Any]] = [] + + for key in MARKS: + common = {"src": name, "mark": key, "aspect": asp, "px": w * h} + # Control first: the same call path on the same frame with nothing stamped. + rows.append({**common, "cell": "control", "size": None, "alpha": None, **_probe(base, key, None)}) + for size in SIZES: + for alpha in ALPHAS: + st = stamp_any(base, key, size_mult=size, alpha_mult=alpha) + if st is None: + continue + stamped, box = st + # Texture of the region the mark sits on, from the CLEAN frame -- the + # same median-Sobel proxy the pill's flatness gate uses. + x, y, gw, gh = box + rows.append( + { + **common, + "cell": "stamped", + "size": size, + "alpha": alpha, + "texture": round(texture_of(base[y : y + gh, x : x + gw]), 2), + **_probe(stamped, key, box), + } + ) + return rows + + +def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]: + if n == 0: + return (0.0, 0.0) + p, d = k / n, 1 + z * z / n + c = (p + z * z / (2 * n)) / d + hw = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d + return (100 * (c - hw), 100 * (c + hw)) + + +def main() -> None: + ap = argparse.ArgumentParser() + ap.add_argument("--n", type=int, default=150, help="verified-clean source images") + ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2)) + ap.add_argument("--out", type=Path, default=OUT) + ap.add_argument("--restart", action="store_true") + ap.add_argument("--report-only", action="store_true", help="re-read --out and print the report") + a = ap.parse_args() + + if a.report_only: + rows = [json.loads(line) for line in a.out.read_text(encoding="utf-8").splitlines() if line.strip()] + report(rows) + return + + if a.restart and a.out.exists(): + a.out.unlink() + done: set[str] = set() + if a.out.exists(): + with open(a.out, encoding="utf-8") as fh: + for line in fh: + try: + done.add(json.loads(line)["src"]) + except Exception: # noqa: S112 -- tolerate a torn last line + continue + + print(f"selecting {a.n} verified-clean sources (already measured: {len(done)})...", flush=True) + sources = [p for p in clean_sources(a.n) if p.name not in done] + print(f"to do {len(sources)} workers {a.workers} grid {len(MARKS)}x{len(SIZES)}x{len(ALPHAS)}\n", flush=True) + + rows: list[dict[str, Any]] = [] + a.out.parent.mkdir(parents=True, exist_ok=True) + with open(a.out, "a", encoding="utf-8") as fh, ProcessPoolExecutor(max_workers=a.workers) as ex: + futures = {ex.submit(_one_source, str(p)): p for p in sources} + for i, fut in enumerate(as_completed(futures), 1): + try: + got = fut.result() + except Exception as e: # a crashed source must not kill the sweep + print(f" source failed: {type(e).__name__}: {e}", flush=True) + continue + for r in got: + fh.write(json.dumps(r) + "\n") + rows += got + if i % 10 == 0: + fh.flush() + print(f" {i}/{len(sources)}", flush=True) + + if done: # merge the resumed rows so the report covers the whole file + rows = [json.loads(line) for line in a.out.read_text(encoding="utf-8").splitlines() if line.strip()] + report(rows) + + +def _rate(rs: list[dict[str, Any]], field: str) -> tuple[float, float, float, int]: + k, n = sum(bool(r[field]) for r in rs), len(rs) + lo, hi = wilson(k, n) + return (100 * k / n if n else 0.0, lo, hi, n) + + +def report(rows: list[dict[str, Any]]) -> None: + stamped = [r for r in rows if r["cell"] == "stamped"] + control = [r for r in rows if r["cell"] == "control"] + if not stamped: + print("no measurements") + return + srcs = len({r["src"] for r in rows}) + print(f"\n{'=' * 84}\nDETECTOR RESPONSE {len(stamped)} stamped cells over {srcs} clean backgrounds\n{'=' * 84}") + + # Lead with the nominal cell. The grid deliberately includes sizes and opacities the + # engines were never calibrated for, so an aggregate over the whole grid is an + # adversarial score, NOT production recall -- quoting it as recall would understate + # the detectors badly. The nominal cell is the honest reference point: the mark at the + # exact geometry and opacity the engine assumes, on a real clean background. + print("\nNOMINAL CELL (size=1.0, alpha=1.0) -- a mark exactly as the engine expects it.") + print("Read every other number against THIS, not as production recall: the grid is") + print("adversarial by design and its aggregate is not a recall figure.\n") + print(f"{'mark':13s} {'n':>5s} {'detected':>10s} {'maskable':>10s}") + for key in MARKS: + rs = [r for r in stamped if r["mark"] == key and r["size"] == 1.0 and r["alpha"] == 1.0] + if rs: + print(f"{key:13s} {len(rs):5d} {_rate(rs, 'detected')[0]:9.1f}% {_rate(rs, 'maskable')[0]:9.1f}%") + + print("\n\nFALSE FIRE on the unstamped controls (the baseline every recall below is read against)\n") + print(f"{'mark':13s} {'n':>5s} {'fires':>8s} {'95% CI':>16s}") + for key in MARKS: + rs = [r for r in control if r["mark"] == key] + if rs: + pct, lo, hi, n = _rate(rs, "detected") + print(f"{key:13s} {n:5d} {pct:7.1f}% {f'{lo:.1f}-{hi:.1f}':>16s}") + + print("\n\nDETECTED vs MASKABLE by mark. The GAP is the silent-no-op rate:") + print("a mark counted here as detected-but-not-maskable is reported by `identify` and") + print("skipped by `visible`, so the user is told it was removed when it was not.\n") + print(f"{'mark':13s} {'n':>6s} {'detected':>10s} {'maskable':>10s} {'gap':>7s} {'mask_hit':>9s}") + for key in MARKS: + rs = [r for r in stamped if r["mark"] == key] + if not rs: + continue + det = [r for r in rs if r["detected"]] + d, m = _rate(rs, "detected")[0], _rate(rs, "maskable")[0] + gap = (100 * sum(1 for r in det if not r["maskable"]) / len(det)) if det else 0.0 + hits = [r["mask_hit"] for r in det if r["mask_hit"] is not None] + note = " (constructed)" if key == "jimeng_pill" else "" + hv = f"{float(np.median(hits)):.2f}" if hits else "-" + print(f"{key:13s} {len(rs):6d} {d:9.1f}% {m:9.1f}% {gap:6.1f}% {hv:>9s}{note}") + + for axis, label in (("alpha", "CONTRAST (alpha multiplier)"), ("size", "SIZE (glyph box multiplier)")): + print(f"\n\nRECALL BY {label} -- 1.0 is the geometry/opacity the engine assumes\n") + vals = sorted({r[axis] for r in stamped}) + print(f"{'mark':13s}" + "".join(f"{v:>12}" for v in vals)) + for key in MARKS: + cells = [] + for v in vals: + rs = [r for r in stamped if r["mark"] == key and r[axis] == v] + cells.append(f"{_rate(rs, 'detected')[0]:10.0f}%" if rs else f"{'-':>11s}") + print(f"{key:13s}" + "".join(f"{c:>12}" for c in cells)) + + print("\n\nRECALL BY FRAME ASPECT -- the axis the `scale_basis` bug lived on\n") + asps = ["portrait", "square", "landscape"] + print(f"{'mark':13s}" + "".join(f"{x:>14s}" for x in asps)) + for key in MARKS: + cells = [] + for asp in asps: + rs = [r for r in stamped if r["mark"] == key and r["aspect"] == asp] + cells.append(f"{_rate(rs, 'detected')[0]:.0f}% (n={len(rs)})" if rs else "-") + print(f"{key:13s}" + "".join(f"{c:>14s}" for c in cells)) + + tex = sorted(r["texture"] for r in stamped if "texture" in r) + if tex: + t1, t2 = tex[len(tex) // 3], tex[2 * len(tex) // 3] + print(f"\n\nRECALL BY BACKGROUND TEXTURE (terciles at {t1:.1f} / {t2:.1f} median-Sobel)\n") + print(f"{'mark':13s}" + "".join(f"{x:>14s}" for x in ("flat", "mid", "textured"))) + for key in MARKS: + cells = [] + for lo, hi in ((-1, t1), (t1, t2), (t2, 1e9)): + rs = [r for r in stamped if r["mark"] == key and lo < r.get("texture", -2) <= hi] + cells.append(f"{_rate(rs, 'detected')[0]:.0f}% (n={len(rs)})" if rs else "-") + print(f"{key:13s}" + "".join(f"{c:>14s}" for c in cells)) + + worst = collections.Counter((r["mark"], r["size"], r["alpha"]) for r in stamped if not r["detected"]).most_common(8) + if worst: + print("\n\nWORST CELLS (most misses)\n") + for (mark, size, alpha), n in worst: + print(f" {mark:13s} size={size} alpha={alpha} {n} misses") + + +if __name__ == "__main__": + main() diff --git a/scripts/fill_quality.py b/scripts/fill_quality.py index 689dace..1e6c7b0 100644 --- a/scripts/fill_quality.py +++ b/scripts/fill_quality.py @@ -89,8 +89,31 @@ def engine_for(mark_key: str) -> Any: return cls() -def stamp(image: np.ndarray, mark_key: str) -> tuple[np.ndarray, tuple[int, int, int, int]] | None: - """Composite a mark's alpha glyph into its canonical corner. Returns (stamped, bbox).""" +def composite(image: np.ndarray, alpha: np.ndarray, x: int, y: int) -> np.ndarray: + """The single forward model every stamp uses: ``stamped = (1-a)*bg + a*white``. + + ``alpha`` is float and clipped to [0,1] here, so a caller may pre-multiply it by an + opacity factor without worrying about overflow. Shared by ``stamp``/``stamp_slot`` + here and by ``scripts/detector_response.py`` (which imports it) so the model is + written ONCE. + """ + gh, gw = alpha.shape[:2] + out = image.copy() + roi = out[y : y + gh, x : x + gw].astype(np.float32) + a3 = np.clip(alpha, 0.0, 1.0)[..., None] + out[y : y + gh, x : x + gw] = np.clip(roi * (1 - a3) + 255.0 * a3, 0, 255).astype(np.uint8) + return out + + +def stamp( + image: np.ndarray, mark_key: str, *, size_mult: float = 1.0, alpha_mult: float = 1.0 +) -> tuple[np.ndarray, tuple[int, int, int, int]] | None: + """Composite a mark's alpha glyph into its canonical corner. Returns (stamped, bbox). + + ``size_mult`` scales the glyph box off the engine's nominal geometry and ``alpha_mult`` + its opacity -- both default to 1.0 (the geometry the engine assumes), and + `scripts/detector_response.py` sweeps them to build response curves. + """ from remove_ai_watermarks._text_mark_engine import load_alpha_template engine = engine_for(mark_key) @@ -101,19 +124,15 @@ def stamp(image: np.ndarray, mark_key: str) -> tuple[np.ndarray, tuple[int, int, loc = engine.locate(image) base = engine.scale_base(image) - gw = max(cfg.min_gw, int(cfg.alpha_width_frac * base)) - gh = max(4, int(cfg.alpha_height_frac * base)) + gw = max(cfg.min_gw, int(cfg.alpha_width_frac * base * size_mult)) + gh = max(4, int(cfg.alpha_height_frac * base * size_mult)) if gw < 8 or gh < 4 or gw > loc.w or gh > loc.h: return None - a = cv2.resize(alpha, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32) + a = cv2.resize(alpha, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32) * alpha_mult x = loc.x + (loc.w - gw) // 2 y = loc.y + (loc.h - gh) // 2 - out = image.copy() - roi = out[y : y + gh, x : x + gw].astype(np.float32) - a3 = a[..., None] - out[y : y + gh, x : x + gw] = np.clip(roi * (1 - a3) + 255.0 * a3, 0, 255).astype(np.uint8) - return out, (x, y, gw, gh) + return composite(image, a, x, y), (x, y, gw, gh) def _slot_alpha(mark_key: str) -> np.ndarray | None: @@ -134,12 +153,15 @@ def _slot_alpha(mark_key: str) -> np.ndarray | None: return None -def stamp_slot(image: np.ndarray, mark_key: str) -> tuple[np.ndarray, tuple[int, int, int, int]] | None: +def stamp_slot( + image: np.ndarray, mark_key: str, *, size_mult: float = 1.0, alpha_mult: float = 1.0 +) -> tuple[np.ndarray, tuple[int, int, int, int]] | None: """Stamp a mark into its OWN default footprint slot (the `--no-detect` geometry). Used for gemini and the pill, which have no bundled alpha asset with corner fractions. The mark is fitted into the middle of its slot so the fill has to - recover the same kind of region it would in production. + recover the same kind of region it would in production. ``size_mult``/``alpha_mult`` + match ``stamp`` (both default to 1.0). """ from remove_ai_watermarks.watermark_registry import get_mark @@ -154,21 +176,21 @@ def stamp_slot(image: np.ndarray, mark_key: str) -> tuple[np.ndarray, tuple[int, return None y0, y1, x0, x1 = int(ys.min()), int(ys.max()), int(xs.min()), int(xs.max()) bw, bh = x1 - x0 + 1, y1 - y0 + 1 - gw, gh = max(8, int(bw * 0.7)), max(8, int(bh * 0.7)) - if gw < 8 or gh < 8 or gw > image.shape[1] or gh > image.shape[0]: - return None - a = cv2.resize(alpha, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32) - a = np.clip(a, 0.0, 1.0) + gw, gh = max(8, int(bw * 0.7 * size_mult)), max(8, int(bh * 0.7 * size_mult)) + a = cv2.resize(alpha, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32) * alpha_mult x, y = x0 + (bw - gw) // 2, y0 + (bh - gh) // 2 - out = image.copy() - roi = out[y : y + gh, x : x + gw].astype(np.float32) - a3 = a[..., None] - out[y : y + gh, x : x + gw] = np.clip(roi * (1 - a3) + 255.0 * a3, 0, 255).astype(np.uint8) - return out, (x, y, gw, gh) + # A large size_mult can grow the glyph past its slot and push x/y negative; guard so + # composite never writes out of bounds (numpy would wrap a negative index silently). + if x < 0 or y < 0 or x + gw > image.shape[1] or y + gh > image.shape[0]: + return None + return composite(image, a, x, y), (x, y, gw, gh) -def stamp_any(image: np.ndarray, mark_key: str) -> tuple[np.ndarray, tuple[int, int, int, int]] | None: - return stamp(image, mark_key) if mark_key in STAMPABLE else stamp_slot(image, mark_key) +def stamp_any( + image: np.ndarray, mark_key: str, *, size_mult: float = 1.0, alpha_mult: float = 1.0 +) -> tuple[np.ndarray, tuple[int, int, int, int]] | None: + fn = stamp if mark_key in STAMPABLE else stamp_slot + return fn(image, mark_key, size_mult=size_mult, alpha_mult=alpha_mult) def clean_sources(n: int, seed: int = 11) -> list[Path]: diff --git a/scripts/ladder_headroom.py b/scripts/ladder_headroom.py new file mode 100644 index 0000000..d99e722 --- /dev/null +++ b/scripts/ladder_headroom.py @@ -0,0 +1,305 @@ +"""How much recall is the coarse scale ladder costing, and what would a denser one cost? + +THE FINDING THIS MEASURES + `_tophat_score` sweeps three rungs -- (0.8, 1.0, 1.25) -- and the `binary` front-end + sweeps none at all. Measured on stamped marks over controlled backgrounds + (`scripts/detector_response.py`), the response is a COMB: doubao scores 0.99 exactly at + each rung and collapses to 0.37-0.48 between them, against a 0.50 gate. So a mark whose + rendered size lands mid-gap is missed at FULL contrast. Jimeng (binary, one nominal + size) holds a single 0.90-1.20 lobe; samsung only 0.95-1.05. + +WHY A SYNTHETIC SWEEP IS NOT ENOUGH TO JUSTIFY A FIX + Dead zones only cost recall if real marks land in them. The fractions were calibrated on + real captures, so it is entirely possible that real marks cluster at ratio 1.0 and the + gaps are never visited. That question is not answerable by stamping -- it needs the + corpus. + +THE MEASUREMENT + Positives are images carrying INDEPENDENT vendor provenance (TC260 / C2PA metadata + naming the vendor), which is evidence that does not come from the pixel detector we are + grading -- the same discriminator that settled the pill gate. For every such image the + detector currently MISSES, rescore at a dense ladder and ask whether it would now cross + the gate, and at which scale. + + Negatives are images with NO metadata signal at all. That is deliberately NOT "and no + mark was detected": defining the set by the detector's own verdict would make its + false-fire rate 0 by construction and the comparison vacuous. The cost is that a + metadata-stripped screenshot of a marked image sits in the negative set and its correct + detection is counted as a false fire -- but that impurity is identical under both + ladders, so the DELTA between them, which is what the fix is judged on, stays sound. + + Two more impurities to keep in view when reading the positive arm. Provenance says the + vendor produced the file, not that a visible mark is on it. And the TC260 label names no + specific vendor, so `visible_provenance` maps it to BOTH doubao and jimeng -- a "doubao + positive" may carry a jimeng mark or none. Both inflate the miss count, so the recovery + percentage is an upper bound on what a denser ladder buys. + +DATA SAFETY + Corpus images are user uploads: read-only, local analysis, gitignored output. + + uv run python scripts/ladder_headroom.py --mark doubao --n 4000 +""" + +from __future__ import annotations + +import argparse +import collections +import glob +import json +import math +import os +import random +import sys +from concurrent.futures import ProcessPoolExecutor, as_completed +from pathlib import Path +from typing import Any + +import cv2 +import numpy as np + +sys.path.insert(0, str(Path(__file__).parent.parent)) + +REPO = Path(__file__).resolve().parents[1] +CORPUS = REPO / "data" / "spaces" / "originals" +OUT = REPO / "data" / "spaces" / "_ladder_headroom.jsonl" + +# The rungs the product ships today, and the dense ladder under evaluation. The dense one +# is geometric with a ~6% step, chosen from the measured half-width of a rung's lobe +# (doubao holds >=0.5 for about +/-6% around each peak), so no gap is left uncovered. +SHIPPED = (0.8, 1.0, 1.25) +LADDERS: dict[str, tuple[float, ...]] = { + # A geometric ~6% step across the whole plausible range. Measures the CEILING of what + # any ladder change can buy, and the worst case of what it costs in false fires. + "dense": tuple(round(0.70 * (1.06**i), 4) for i in range(13)), # 0.70 .. ~1.49 + # The targeted alternative. On the full run, 26 of 28 recoveries came from ONE rung + # (~1.116) and 22 of 28 were LANDSCAPE frames -- that is a geometry gap, not a density + # gap, so the honest comparison is one extra rung against thirteen. + "plus_one": (0.8, 1.0, 1.1157, 1.25), + "shipped": SHIPPED, +} +DENSE = LADDERS["dense"] +# Score every rung any candidate ladder might use and store them all, so a new candidate +# is evaluated by re-reading the JSONL instead of re-running the hour-long sweep. +PROBE_SCALES: tuple[float, ...] = tuple(sorted({s for rungs in LADDERS.values() for s in rungs})) + + +def wilson(k: int, n: int, z: float = 1.96) -> tuple[float, float]: + if n == 0: + return (0.0, 0.0) + p, d = k / n, 1 + z * z / n + c = (p + z * z / (2 * n)) / d + hw = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d + return (100 * (c - hw), 100 * (c + hw)) + + +def score_at_scales(engine: Any, image: np.ndarray, scales: tuple[float, ...]) -> dict[float, float]: + """`_tophat_score`'s inner loop, opened up so the ladder is a parameter. + + Deliberately reaches into the engine (`tophat_response`, `_glyph_silhouette`): a + measurement script may, product code may not. Kept a faithful copy of the shipped + scoring -- if it drifts, the numbers below stop describing the product. + """ + c = engine.config + loc = engine.locate(image) + resp = engine.tophat_response(image, loc) + sil = engine._glyph_silhouette() + if resp is None or sil is None: + return {} + base = engine.scale_base(image) + out: dict[float, float] = {} + for scale in scales: + gw = max(c.min_gw, int(c.alpha_width_frac * base * scale)) + gh = max(4, int(c.alpha_height_frac * base * scale)) + if gw >= resp.shape[1] or gh >= resp.shape[0]: + continue + tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32) + if c.template_blur > 0: + tmpl = cv2.GaussianBlur(tmpl, (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur) + out[scale] = float(cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED).max()) + return out + + +def _one(args: tuple[str, str]) -> dict[str, Any] | None: + path_str, mark_key = args + from remove_ai_watermarks.api import visible_provenance + from remove_ai_watermarks.identify import identify + from remove_ai_watermarks.image_io import imread + from remove_ai_watermarks.watermark_registry import get_mark + + path = Path(path_str) + try: + prov = visible_provenance(path) + except Exception: + return None + positive = mark_key in prov + + if not positive: + # A negative is a frame with NO metadata signal at all -- evidence independent of + # the detector being graded. Do NOT also require that no mark was detected: that + # defines the negative set by the very verdict under test, and "false fire today" + # would then be 0 by construction on every run. + try: + if identify(path, check_visible=False).signals: + return None + except Exception: + return None + + img = imread(path_str) + if img is None or min(img.shape[:2]) < 200: + return None + + mark = get_mark(mark_key) + det = mark.detect(img) + + import importlib + + mod = importlib.import_module(f"remove_ai_watermarks.{mark_key}_engine") + cls = next( + o for n, o in vars(mod).items() if isinstance(o, type) and n.endswith("Engine") and n != "TextMarkEngine" + ) + engine = cls() + scores = score_at_scales(engine, img, PROBE_SCALES) + if not scores: + return None + gate = engine.config.detect_ncc_threshold + dense = {s: v for s, v in scores.items() if s in LADDERS["dense"]} + best_scale = max(dense, key=lambda s: dense[s]) if dense else 0.0 + h, w = img.shape[:2] + return { + "src": path.name, + "arm": "positive" if positive else "negative", + "detected_now": bool(det.detected), + "conf_now": round(float(det.confidence), 4), + "gate": gate, + # Every rung, so a new candidate ladder is evaluated by re-reading this file. + "scores": {str(s): round(v, 4) for s, v in scores.items()}, + "dense_best": round(dense[best_scale], 4) if dense else 0.0, + "dense_best_scale": best_scale, + "dense_crosses": bool(dense) and dense[best_scale] >= gate, + "aspect": "portrait" if w / h < 0.9 else ("landscape" if w / h > 1.1 else "square"), + } + + +def main() -> None: + ap = argparse.ArgumentParser() + # doubao only, and that is not laziness. `score_at_scales` reproduces the TOPHAT + # front-end, which is the only one with a ladder to widen; scoring it for jimeng or + # samsung would measure a code path their detectors never run. Doubao is also the one + # mark that declares NO rival, so crossing its NCC gate really is the whole verdict -- + # for a mark with rivals, `dense_crosses` would ignore the competitive margin and read + # optimistically. Widening the binary front-end is a separate experiment. + ap.add_argument("--mark", default="doubao", choices=["doubao"]) + ap.add_argument("--n", type=int, default=4000, help="corpus files to scan") + ap.add_argument("--workers", type=int, default=max(1, (os.cpu_count() or 4) - 2)) + ap.add_argument("--out", type=Path, default=OUT) + ap.add_argument("--report-only", action="store_true") + a = ap.parse_args() + + out = a.out.with_name(f"{a.out.stem}_{a.mark}{a.out.suffix}") + if a.report_only: + report([json.loads(x) for x in out.read_text(encoding="utf-8").splitlines() if x.strip()], a.mark) + return + + pool = glob.glob(str(CORPUS / "*" / "*")) + random.Random(19).shuffle(pool) # noqa: S311 -- deterministic sampling, not crypto + pool = pool[: a.n] + print(f"mark={a.mark} scanning {len(pool)} corpus files workers={a.workers}") + print(f"shipped ladder {SHIPPED}\ndense ladder {DENSE}\n", flush=True) + + rows: list[dict[str, Any]] = [] + out.parent.mkdir(parents=True, exist_ok=True) + with open(out, "w", encoding="utf-8") as fh, ProcessPoolExecutor(max_workers=a.workers) as ex: + futures = [ex.submit(_one, (p, a.mark)) for p in pool] + for i, fut in enumerate(as_completed(futures), 1): + try: + rec = fut.result() + except Exception: # noqa: S112 -- one bad file must not kill the sweep + continue + if rec is None: + continue + fh.write(json.dumps(rec) + "\n") + rows.append(rec) + if i % 500 == 0: + fh.flush() + pos = sum(1 for r in rows if r["arm"] == "positive") + print(f" {i}/{len(pool)} usable={len(rows)} (pos={pos})", flush=True) + report(rows, a.mark) + + +def report(rows: list[dict[str, Any]], mark: str) -> None: + pos = [r for r in rows if r["arm"] == "positive"] + neg = [r for r in rows if r["arm"] == "negative"] + print(f"\n{'=' * 82}\nLADDER HEADROOM mark={mark} positives={len(pos)} verified-clean negatives={len(neg)}") + print(f"{'=' * 82}") + if not pos and not neg: + print("no usable rows") + return + + if pos: + miss = [r for r in pos if not r["detected_now"]] + rec = [r for r in miss if r["dense_crosses"]] + now = len(pos) - len(miss) + lo, hi = wilson(len(rec), len(miss)) if miss else (0.0, 0.0) + print("\nPOSITIVE ARM (images whose METADATA names this vendor -- an upper bound,") + print("provenance says the vendor produced the file, not that a mark is visible on it)\n") + print(f" detected today {now:5d} / {len(pos)} ({100 * now / len(pos):.1f}%)") + print(f" of the {len(miss)} misses, the dense ladder crosses the gate on {len(rec)}") + if miss: + print(f" ({100 * len(rec) / len(miss):.1f}%, 95% CI {lo:.1f}-{hi:.1f})") + after = now + len(rec) + print(f" detection would go {100 * now / len(pos):.1f}% -> {100 * after / len(pos):.1f}%") + if rec: + by_scale = collections.Counter(r["dense_best_scale"] for r in rec) + print("\n which rung recovers them (a rung near a SHIPPED one recovering many means") + print(" the gain is the finer STEP, not the wider RANGE):") + for s, n in sorted(by_scale.items()): + near = " <- shipped" if any(abs(s - x) < 0.02 for x in SHIPPED) else "" + print(f" scale {s:<6} {n:4d}{near}") + by_asp = collections.Counter(r["aspect"] for r in rec) + print(f"\n by aspect: {dict(by_asp)}") + + if neg: + fires_now = sum(1 for r in neg if r["detected_now"]) + fires_dense = sum(1 for r in neg if r["dense_crosses"]) + lo0, hi0 = wilson(fires_now, len(neg)) + lo1, hi1 = wilson(fires_dense, len(neg)) + print("\n\nNEGATIVE ARM (no metadata signal; NOT filtered on the detector's own verdict)\n") + p0, p1 = 100 * fires_now / len(neg), 100 * fires_dense / len(neg) + print(f" false fire today {fires_now:4d} / {len(neg)} {p0:.2f}% (CI {lo0:.2f}-{hi0:.2f})") + print(f" false fire, dense ladder {fires_dense:4d} / {len(neg)} {p1:.2f}% (CI {lo1:.2f}-{hi1:.2f})") + print("\n This is the price. A denser ladder gives a spurious blob more chances to") + print(" match at SOME scale, so the gain above is only real if this line barely moves.") + + ladder_table(pos, neg) + + +def _crosses(row: dict[str, Any], rungs: tuple[float, ...]) -> bool: + """Would this frame cross its gate on the given ladder, from the stored per-rung scores.""" + sc = row.get("scores") or {} + gate = row["gate"] + return any(v >= gate for k, v in sc.items() if float(k) in rungs) + + +def ladder_table(pos: list[dict[str, Any]], neg: list[dict[str, Any]]) -> None: + """Every candidate ladder side by side: what it recovers against what it costs.""" + if not (pos and neg) or "scores" not in pos[0]: + return # older run without per-rung scores + miss = [r for r in pos if not r["detected_now"]] + base_fire = sum(1 for r in neg if r["detected_now"]) + print("\n\nLADDER CANDIDATES -- recovered marks against the false fires they cost\n") + print("A candidate is only worth shipping if the recovered column beats the added-fires") + print("column by enough to survive the base rates: clean frames vastly outnumber marked") + print("ones in real traffic, so a small percentage on the negative arm is a large count.\n") + hdr = f"{'ladder':10s} {'rungs':>6s} {'recovered':>10s} {'of misses':>10s}" + print(f"{hdr} {'false fire':>11s} {'added':>7s} {'ratio':>7s}") + for name, rungs in LADDERS.items(): + rec = sum(1 for r in miss if _crosses(r, rungs)) + fire = sum(1 for r in neg if _crosses(r, rungs) or r["detected_now"]) + added = fire - base_fire + ratio = f"{rec / added:.1f}:1" if added > 0 else ("inf" if rec else "-") + pct = 100 * rec / len(miss) if miss else 0.0 + print(f"{name:10s} {len(rungs):6d} {rec:10d} {pct:9.1f}% {100 * fire / len(neg):10.2f}% {added:7d} {ratio:>7s}") + + +if __name__ == "__main__": + main() diff --git a/scripts/render_vendor_silhouettes.py b/scripts/render_vendor_silhouettes.py index 1dd8eed..7b6e1d7 100644 --- a/scripts/render_vendor_silhouettes.py +++ b/scripts/render_vendor_silhouettes.py @@ -45,6 +45,35 @@ almost nothing and, at any threshold low enough to fire, fires on arbitrary corn 星绘 additionally has only ONE confirmed example in the corpus, so even a working front-end could not have its threshold calibrated yet. + +UPDATE 2026-07-20: the named blocker is GONE, and the retry is still inconclusive. +`detect_frontend="tophat"` (built later, for doubao) is exactly the "grayscale correlation +on the raw top-hat" this note asked for, so the 2026-07-18 ruling rests on a premise that +no longer holds and must not simply be inherited. Two things were measured against it, and +neither settles the question: + + * A GENERIC template of the shared `AI生成` tail -- attractive because GB 45438-2025 + guarantees that run across vendors, so one template would cover 千问 / 百度 / 星绘 and + anything compliant that ships next. Measured on the tophat front-end at the shipped + 3-rung ladder: a bold 千问 positive scores 0.407 against clean corners at p99 0.298 / + max 0.321. It separates on that one frame, but only by a hair, and a 4-glyph template + is inherently less specific than a 6-glyph one -- the shorter the run, the more + arbitrary corner structure correlates with it. + * The FULL 千问 template on the same front-end scores 0.248 against a clean max of 0.537, + i.e. no separation at all -- WORSE than the generic tail, which is the opposite of + what the specificity argument predicts and is itself a reason to distrust n=1. + +The blocker is now EVIDENCE, not architecture: this session found exactly one 千问 and one +百度 positive (both by eyeballing doubao-provenance misses), and the 14 positives quoted +above were not preserved anywhere the current scripts can reach. Nothing should be +registered off a single frame. What it takes: harvest 30+ confirmed positives per vendor +-- `scripts/cjk_tail_probe.py` exists for exactly this, scoring TC260-provenance frames +that no detector fires on and writing a contact sheet of the top scorers to label -- then +calibrate a gate against the clean arm. Two traps worth knowing before repeating this +measurement: score with `alpha_height_frac`, not the silhouette's own aspect ratio (the +latter inflated the clean p99 from 0.30 to 0.58 and made every comparison meaningless), +and keep the ladder at the shipped 3 rungs, since a wide sweep hands clean corners many +extra chances to match. """ from __future__ import annotations diff --git a/src/remove_ai_watermarks/_text_mark_engine.py b/src/remove_ai_watermarks/_text_mark_engine.py index b7f3b50..bb628d7 100644 --- a/src/remove_ai_watermarks/_text_mark_engine.py +++ b/src/remove_ai_watermarks/_text_mark_engine.py @@ -90,13 +90,6 @@ _MIN_DETECT_SHORT_SIDE = 200 # threshold repairs that -- it needs a better detection silhouette. _DEFAULT_PROVENANCE_NCC_FACTOR = 0.7 -# Level (fraction of the response's own peak) at which the CONTINUOUS top-hat is -# turned into a glyph blob, used only when the binarized path found nothing on a -# mark the detector did fire on. Half the peak keeps the stroke cores and drops the -# halo; the enclosing rectangle is what gets filled anyway, so this only has to be -# good enough to BOUND the mark, not to segment it. -_FAINT_GLYPH_LEVEL = 0.5 - @dataclass(frozen=True) class TextMarkConfig: @@ -346,20 +339,30 @@ class TextMarkEngine: return None return (resp / peak * 255).astype(np.uint8) - def _tophat_score(self, image: NDArray[Any], loc: TextMarkLocation) -> float: - """TM_CCOEFF_NORMED of a soft template against the continuous response. + def _tophat_best( + self, image: NDArray[Any], loc: TextMarkLocation + ) -> tuple[float, tuple[int, int, int, int] | None]: + """Best TM_CCOEFF_NORMED of a soft template against the continuous response, and + the ROI-local box (x0, y0, x1, y1) where that best match sits. Sweeps a small scale band: the nominal glyph size is derived from the mark's geometry, but a vendor re-rasterization shifts it by a few percent and the continuous response is sharp enough that an exact-size template would miss. + + Detection and the removal mask BOTH read this one method -- the score gates + detection, the box bounds the fill. Sharing it is deliberate: the standing rule is + that detection and the mask use the same front-end, and the way that rule was last + broken was a drift between two separate implementations. One method makes the drift + impossible instead of merely discouraged. """ c = self.config resp = self.tophat_response(image, loc) sil = self._glyph_silhouette() if resp is None or sil is None: - return 0.0 + return (0.0, None) base = self.scale_base(image) - best = 0.0 + best_score = 0.0 + best_box: tuple[int, int, int, int] | None = None for scale in (0.8, 1.0, 1.25): gw = max(c.min_gw, int(c.alpha_width_frac * base * scale)) gh = max(4, int(c.alpha_height_frac * base * scale)) @@ -368,8 +371,16 @@ class TextMarkEngine: tmpl = cv2.resize(sil, (gw, gh), interpolation=cv2.INTER_AREA).astype(np.float32) if c.template_blur > 0: tmpl = cv2.GaussianBlur(tmpl, (0, 0), sigmaX=c.template_blur, sigmaY=c.template_blur) - best = max(best, float(cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED).max())) - return best + result = cv2.matchTemplate(resp, tmpl.astype(np.uint8), cv2.TM_CCOEFF_NORMED) + _, score, _, top_left = cv2.minMaxLoc(result) + if score > best_score: + tx, ty = int(top_left[0]), int(top_left[1]) + best_score, best_box = float(score), (tx, ty, tx + gw - 1, ty + gh - 1) + return (best_score, best_box) + + def _tophat_score(self, image: NDArray[Any], loc: TextMarkLocation) -> float: + """The detection score alone -- the box the removal mask needs is discarded here.""" + return self._tophat_best(image, loc)[0] def scale_base(self, image: NDArray[Any]) -> int: """The image dimension this mark's geometry scales with. @@ -557,25 +568,29 @@ class TextMarkEngine: bx, by, bw, bh = loc.bbox glyph = self.extract_mask(image, loc) # box-sized, 255 = glyph ys, xs = np.where(glyph > 0) - faint = xs.size < self._MIN_GLYPH_PIXELS and self.config.detect_frontend == "tophat" - # A mark found only by the CONTINUOUS front-end has no binary glyph blob to bound, - # so the mask came back empty and removal was a silent no-op while `identify` still - # reported the mark (corpus-measured 2026-07-20: 57 of 60 sampled still-detected - # Doubao marks were untouched, ~8% of its detections). Fall back to the same - # response the DETECTOR scored, thresholded relative to its own peak. Gated on an - # actual detection: the response is max-normalized, so on a clean corner it would - # normalize NOISE up to 1.0 and mask a random patch -- the detector's verdict is - # what separates signal from noise here. - if faint and self.detect(image).detected: - resp = self.tophat_response(image, loc) - if resp is not None: - ys, xs = np.where(resp >= _FAINT_GLYPH_LEVEL) + box: tuple[int, int, int, int] | None = None if xs.size >= self._MIN_GLYPH_PIXELS: + box = (int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())) + elif self.config.detect_frontend == "tophat" and self.detect(image).detected: + # A mark found only by the CONTINUOUS front-end has no binary glyph blob to + # bound, so the mask came back empty and removal was a silent no-op while + # `identify` still reported the mark (corpus-measured 2026-07-20: 57 of 60 + # sampled still-detected Doubao marks were untouched, ~8% of its detections). + # Use the DETECTOR'S OWN best-match box: the correlation already located the + # mark at a position and scale, and thresholding the response was a strictly + # worse proxy for that. An earlier fix thresholded the max-normalized uint8 + # response at 0.5 -- which selects every non-zero pixel, not "half the peak" as + # its comment claimed -- and filled ~120% of the corner box on textured frames + # (measured: whole corner vs 58.7% for the match box, both detector-clean). + # Gated on an actual detection: on a clean corner the box would be spurious. + _, box = self._tophat_best(image, loc) + if box is not None: + gx0, gy0, gx1, gy1 = box pad = max(4, int(0.10 * bh)) - rx1 = max(0, bx + int(xs.min()) - pad) - rx2 = min(w, bx + int(xs.max()) + 1 + pad) - ry1 = max(0, by + int(ys.min()) - pad) - ry2 = min(h, by + int(ys.max()) + 1 + pad) + rx1 = max(0, bx + gx0 - pad) + rx2 = min(w, bx + gx1 + 1 + pad) + ry1 = max(0, by + gy0 - pad) + ry2 = min(h, by + gy1 + 1 + pad) elif force: rx1, ry1, rx2, ry2 = bx, by, min(w, bx + bw), min(h, by + bh) else: diff --git a/tests/test_text_mark_faint_mask.py b/tests/test_text_mark_faint_mask.py index f098d25..1807a6a 100644 --- a/tests/test_text_mark_faint_mask.py +++ b/tests/test_text_mark_faint_mask.py @@ -19,13 +19,20 @@ import pytest from remove_ai_watermarks.doubao_engine import DoubaoEngine -def _faint_mark_image(w: int = 900, h: int = 1200, alpha: float = 0.06) -> np.ndarray: - """A mid-gray frame carrying the REAL Doubao glyph shape at very low opacity. +def _faint_mark_image(w: int = 900, h: int = 1200, alpha: float = 0.06, textured: bool = False) -> np.ndarray: + """A frame carrying the REAL Doubao glyph shape at very low opacity. The shape has to be genuine or the NCC detector will not fire and the test would be exercising nothing; the low ``alpha`` is what keeps the binarizing path from finding a blob. Composited with the same forward model the marks use: ``stamped = (1-a)*bg + a*white``. + + ``textured`` adds fine luminance noise to the background. It is not decoration: on a + FLAT frame the top-hat response is non-zero only on the glyph, so every threshold + yields the same bounding box and a test built on a flat fixture cannot see a wrong + threshold at all -- mutating the constant to an absurd value left the flat tests green. + Texture puts response outside the glyph, which is the condition under which the mask's + sizing actually matters, and is what real corner backgrounds look like. """ from remove_ai_watermarks._text_mark_engine import load_alpha_template @@ -33,6 +40,11 @@ def _faint_mark_image(w: int = 900, h: int = 1200, alpha: float = 0.06) -> np.nd if tmpl is None: pytest.skip("doubao alpha asset unavailable") img = np.full((h, w, 3), 120, np.uint8) + if textured: + rng = np.random.default_rng(7) + noise = rng.normal(0, 14, (h, w, 1)).repeat(3, axis=2) + img = np.clip(img.astype(np.float32) + noise, 0, 255).astype(np.uint8) + img = cv2.GaussianBlur(img, (0, 0), sigmaX=1.2) eng = DoubaoEngine() loc = eng.locate(img) base = eng.scale_base(img) @@ -82,3 +94,48 @@ class TestFaintMarkIsMaskable: mask = eng.footprint_mask(img, force=False) assert mask is not None assert int((mask > 0).sum()) > 0 + + +class TestFaintMaskStaysTight: + """The faint fallback must cover the mark WITHOUT filling the whole corner. + + Corpus-found 2026-07-20: the first version of the fallback thresholded the continuous + response at 0.5, but `tophat_response` returns uint8 0..255 -- so it selected every + non-zero pixel, not "half the peak" as its comment claimed. On 12 of 12 real frames + taking this path the resulting box covered 100% of the corner ROI, so removal inpainted + the entire corner instead of the glyph. Parity could not see it: parity asks whether + the detector is clean afterwards, and a mask that fills everything passes trivially. + The cost, not the outcome, was the defect. + """ + + def test_mask_does_not_swallow_the_whole_corner_on_a_textured_frame(self): + eng = DoubaoEngine() + img = _faint_mark_image(alpha=0.10, textured=True) + # Asserted, not skipped: a skip here would silently stop guarding the moment the + # detector changed, which is exactly when this needs to be guarding. + assert eng.detect(img).detected, "fixture must reach the faint path to test it" + mask = eng.footprint_mask(img, force=False) + assert mask is not None, "a detected faint mark must still produce a mask" + area = int((mask > 0).sum()) + loc = eng.locate(img) + roi = loc.w * loc.h + # The mark's own glyph box is ~40% of the corner ROI and the mask pads it, so a + # correct mask lands near 60%. The pre-fix behaviour measured 120.9% (the whole + # ROI plus padding), which this bound excludes. + assert area < 0.85 * roi, f"mask covers {100 * area / roi:.0f}% of the corner box" + + def test_mask_still_covers_the_stamped_glyph(self): + """Tightness is only a virtue if the mark is still inside. Guards the other way.""" + eng = DoubaoEngine() + img = _faint_mark_image(alpha=0.10, textured=True) + assert eng.detect(img).detected, "fixture must reach the faint path to test it" + mask = eng.footprint_mask(img, force=False) + assert mask is not None + loc = eng.locate(img) + base = eng.scale_base(img) + gw = max(eng.config.min_gw, int(eng.config.alpha_width_frac * base)) + gh = max(4, int(eng.config.alpha_height_frac * base)) + x = loc.x + (loc.w - gw) // 2 + y = loc.y + (loc.h - gh) // 2 + covered = int(np.count_nonzero(mask[y : y + gh, x : x + gw])) / max(1, gw * gh) + assert covered > 0.6, f"mask covers only {100 * covered:.0f}% of the stamped glyph"