135 KiB
Module internals
Relocated verbatim from
CLAUDE.mdon 2026-06-11 to keep the always-loaded context small. Long single-line entries were reformatted into paragraphs; no content was changed or summarized.
Full per-module detail: design decisions, tuned thresholds, calibration
history, incident records, and the regression-guard map. The compact module
list lives in CLAUDE.md; read the relevant section here before changing a
module.
noai/c2pa.py
noai/c2pa.py — C2PA reading, official c2pa-python Reader first, hand-rolled parser as fallback (migrated 2026-06-18; the official lib is a core dep, MIT/Apache, spec-tracking). read_manifest_store_json(path) runs Reader.try_create with a default Context (NO trust enforcement — we report what is in the file, we do not gate on cert trust) and returns the whole manifest-store JSON (every manifest plus ingredient manifests); it is memoized per (path, mtime) (lru_cache(maxsize=8)) because one identify/get_ai_metadata call invokes the structured parser ~3x on the same file. extract_c2pa_info(path) builds its dict from that store JSON (_info_from_store_json: structured claim_generator from the active manifest's claim_generator / claim_generator_info[].name, timestamp from signature_info.time) and falls back to the legacy caBX parser (_extract_c2pa_info_png) when the reader is unavailable (broken/absent wheel, reader_available() False) or finds no parseable manifest (synthetic/partial test blobs, the inject round-trip's re-stitched chunk). Both paths share _populate_registry_fields(buf, info) — the issuer / AI-tool / action / source-type / SynthID / soft-binding registry byte-scan applied to the store JSON (reader path) or the raw caBX bytes (fallback) — so the return-dict shape is identical and the registry stays the single source of truth. Whole-store scanning is load-bearing: a ChatGPT edit of a Sora generation keeps trainedAlgorithmicMedia + issuer "OpenAI" on the parent/ingredient manifest, not the active "opened" one (the active manifest's signature_info.issuer is "OpenAI", common_name "Truepic Lens CLI in Sora", so the issuer field now reads "OpenAI, Truepic" — first-match-wins platform attribution still resolves OpenAI). extract_c2pa_info now also serves non-PNG containers (JPEG/AVIF/MP4) structurally via the reader; the consumers (identify, synthid_source, get_ai_metadata) already merge info OR byte-scan, so this strictly upgrades the non-PNG path with no double-counting. synthid_watermark/synthid_vendors is set when the manifest is signed by a SynthID-using vendor on AI content; soft_binding/soft_binding_vendors when a c2pa.soft-binding alg names a forensic-watermark vendor (soft_binding_vendors_in(buffer) is the shared byte-scan, used by both paths and the non-PNG binary path). extract_c2pa_chunk / inject_c2pa_chunk / has_c2pa_metadata stay the PNG caBX byte tools (raw-chunk extraction for extractor.py, test injection, fallback detection). PNG/caBX chunk reads are clamped to the remaining file size (safe_length = min(length, remaining); skipped chunks use seek) so a malformed huge length cannot drive a multi-GB allocation (shared safety discipline matching isobmff.scan_c2pa_region). Regression-guarded by tests/test_noai.py::TestC2PARealSamples::{test_extract_info_uses_reader_store,test_fallback_to_png_parser_when_reader_unavailable}.
noai/constants.py
noai/constants.py — PNG_SIGNATURE, C2PA_CHUNK_TYPE, C2PA_SIGNATURES, and C2PA_AI_VENDORS — the single C2paAiVendor registry of C2PA-signing vendors (issuer byte, resolved org name, the identify platform label, and a synthid flag), from which C2PA_ISSUERS, SYNTHID_C2PA_ISSUERS (issuers that pair SynthID with C2PA: Google, OpenAI), and identify._ISSUER_PLATFORM are all derived — plus C2PA_SOFT_BINDINGS (soft-binding alg prefix → forensic-watermark vendor: Adobe TrustMark, Digimarc, Imatag, Steg.AI, Microsoft, ...). Add a new C2PA vendor as one C2PA_AI_VENDORS entry (never edit the derived dicts), a new soft-binding to C2PA_SOFT_BINDINGS; not inline. A vendor that signs under multiple legal names needs one entry PER distinctive issuer byte string: e.g. ByteDance's Volcano Engine is registered both as latin volcengine AND the Chinese legal entity 北京火山引擎科技有限公司 (UTF-8; the latin needle misses the Chinese-named certs entirely) — both normalize to the same "ByteDance" needle/platform. ElevenLabs ("Eleven Labs Inc.", pure generative-AI) is registered as a generator. A vendor may also set asserts_ai=True — its presence asserts AI generation even without a trainedAlgorithmicMedia digital-source-type; the derived C2PA_IDENTITY_AI_ORGS frozenset feeds identify, which lifts the AI verdict for such an issuer. Set it ONLY for a pure-generator brand with a distinctive issuer/generator string: Dreamina (ByteDance's international Jimeng brand, signed as "Bytedance Pte. Ltd." with a Dreamina/x.y claim generator and NO source-type — the caBX / store-JSON byte-scan sees the Dreamina token across active + ingredient manifests, where the active one is often a plain c2pa-tool transcode; verified on the retained corpus 2026-07; normalizes to the shared "ByteDance" needle/platform). Do NOT set asserts_ai on common-word issuers (Adobe/Google/OpenAI/Microsoft) — they appear incidentally in unrelated XMP/trust-chain bytes, so they must stay source-type-gated. Deliberately EXCLUDED (mined-corpus candidates 2026-06-20, documented in the file): TikTok Inc. (a content-provenance / AI-labeling signer on uploads, not a generator) and PixelBin.io / "Fynd" (an image transform / CDN signer) — registering either as a generator would mis-label human uploads as AI; the is_ai verdict keys off the digitalSourceType, which is already honored.
metadata.py
metadata.py — scan_head(path, size=1MB) is the shared input for every C2PA/AIGC/IPTC byte scan: first size bytes plus the payloads of any provenance metadata found beyond that window — for ISOBMFF, the late provenance boxes from isobmff.scan_c2pa_region (catches a manifest after a large mdat); for PNG, the late tEXt/iTXt/zTXt/eXIf/iCCP chunks from _png_late_metadata (catches an XMP/EXIF packet appended after a large IDAT, e.g. a TC260 AIGC label at ~2.7 MB). Behavior-neutral (f.read(size)) for non-ISOBMFF inputs and for any file that fits within size. Use it instead of open().read(1MB) for any new marker scan.
Memoized per (path, size, mtime) (added 2026-06-09, _scan_head_cached lru_cache, maxsize=8): one identify/get_ai_metadata call fans out to ~8 byte-scan detectors that each re-read the same file head, so the cache turns those into a single read; the mtime key invalidates on change, a stat failure falls back to an uncached read. synthid_source(path) returns the vendor name(s) if the C2PA manifest implies a SynthID pixel watermark, else None. Format-agnostic: PNG via the caBX parser, JPEG/WebP/AVIF/HEIF/JXL via a binary scan (C2PA marker + SynthID issuer + AI-source marker). get_ai_metadata surfaces the verdict, and metadata --check prints it as a callout. Both get_ai_metadata and has_ai_metadata guard the PIL open with except Exception (HEIC/unknown formats raise non-OSError) and fall through to the binary scan. xai_signature(path) detects xAI/Grok's EXIF-only scheme (ImageDescription = Signature: <base64> + UUID Artist); it feeds has_ai_metadata, get_ai_metadata (key xai_signature), and identify. iptc_ai_system(path) detects the IPTC Photo Metadata 2025.1 AI-disclosure XMP properties (IPTC_AI_FIELD_MARKERS = AISystemUsed/AISystemVersionUsed/AIPromptInformation/AIPromptWriterName) and returns the AISystemUsed generator name (or "fields present"). remove_ai_metadata routes ISOBMFF video (.mp4/.mov/.m4v) through the same isobmff.strip_c2pa_boxes as AVIF/HEIF (MP4 is ISOBMFF), and _scrub_ai_exif removes the xAI signature + AI-generator EXIF tags on JPEG output. strip_c2pa_boxes is fail-safe on a malformed box: it returns the original bytes unchanged with a logged warning instead of truncating the tail to EOF (detection-only scan_c2pa_region still stops at a malformed box). _png_late_metadata clamps each late-chunk read to the remaining file size (safe_length = min(length, remaining)) so a malformed length cannot drive a multi-GB allocation, AND advances the cursor by safe_length (not the raw length) so an inflated length cannot jump past EOF and abort the scan, silently skipping a genuine AI-label chunk after it.
identify.py
identify.py — the OpenAI rollout caveat is keyed on _vendor_of(synthid) == "OpenAI" (not a raw substring over the issuer + verdict blob). identify(path) aggregates every locally-readable signal (C2PA issuer→platform, C2PA soft-binding forensic-watermark vendor, C2PA cloud-manifest reference via metadata.c2pa_cloud_manifest — signal c2pa_cloud, medium, provenance-only (does NOT set is_ai, excluded from ai_from_metadata + clash vendors): a C2PA 2.4 Durable-Content-Credentials case where the embedded manifest is stripped but an XMP dcterms:provenance pointer to the vendor's cloud manifest store (_C2PA_MANIFEST_REPOSITORIES, today cai-manifests.adobe.com → "Adobe Content Authenticity") survives, so the credentials stay recoverable server-side; only emitted when no embedded manifest already attributed the file — surfaced on 2 corpus PNGs 2026-06-10 that read fully unknown before, IPTC "Made with AI" + IPTC 2025.1 AISystemUsed, embedded SD/ComfyUI params, SynthID proxy, xAI/Grok EXIF signature via metadata.xai_signature, the China TC260 AIGC label via metadata.aigc_label, the HuggingFace hf-job-id job marker via metadata.huggingface_job, the Samsung Galaxy AI editing marker via metadata.samsung_genai, the visible marks — Gemini sparkle plus the ByteDance Doubao 豆包AI生成 / Jimeng 即梦AI / Samsung Galaxy AI "Contenuti generati dall'AI" text marks via the watermark_registry — open invisible watermark, Adobe TrustMark via trustmark_detector) into one ProvenanceReport. is_ai_generated is True or None (never asserted False — stripped metadata is not proof of clean origin). The hf_job, visible-mark, and Samsung samsung_genai signals are medium confidence: each lifts an otherwise-Unknown verdict to a tentative AI (hf_only / visible_only / samsung_only, parallel branches; visible_only fires on any visible_* signal) but is excluded from the high-confidence ai_from_metadata set, so none overrides a hard metadata signal.
AI-generated vs AI-enhanced (ProvenanceReport.ai_source_kind, roadmap item): the C2PA digital-source-type is split into "generated" (trainedAlgorithmicMedia, fully synthetic) vs "enhanced" (compositeWithTrainedAlgorithmicMedia, a real photo with an AI-composited region) — the two byte strings are unambiguous (compositeWithTrainedAlgorithmicMedia capitalizes the inner "Trained", so a lowercase trainedAlgorithmicMedia match is standalone full generation; full generation wins when both appear). ai_source_kind is set only when the AI verdict actually came from the C2PA source type (a non-C2PA AI signal — IPTC/AIGC/local gen/xAI — leaves it None). It lets a caller branch a full-frame scrub (generated) from a region-targeted clean that preserves the real photo (enhanced; see noai/tiling.feather_region_composite). The CLI verdict line reads "AI-generated (fully synthetic)" vs "AI-enhanced (real content with an AI-composited region)".
Visible-mark detection (check_visible, signals visible_sparkle / visible_doubao / visible_jimeng / visible_qwen / visible_samsung): the Gemini sparkle keeps its own file-level path (_visible_sparkle → gemini_engine.detect_sparkle_confidence, promoted only at confidence ≥ _SPARKLE_THRESHOLD, which is the SHARED watermark_registry.GEMINI_SPARKLE_TRUST_CONF (0.5) — imported, not a private copy, so the provenance detect threshold and the removal detect_marks / _gemini_detect arbitration gate can never drift (the detect-vs-remove desync from roadmap P0#7; regression-guarded by tests/test_identify.py::TestSparkleDetectRemoveAlignment, which composites the real demo sparkle at borderline opacities and asserts identify and detect_marks AGREE on either side of the line). Lowering the gate to recover faint sub-0.5 sparkles was evaluated 2026-06-20 and REJECTED: a real Doubao text mark scores ~0.40-0.42 as a gemini match with a HIGHER core-ring brightness margin than a genuine faint sparkle, so neither confidence nor the brightness gate separates them in the [0.35, 0.5) band — lowering trades a rare miss for false-positive removals on clean images. Corpus-tuned to separate Gemini sparkles ≥0.56 from non-sparkle ≤0.49), while Doubao/Jimeng/Qwen/Samsung reuse the registry detectors (_visible_text_marks → watermark_registry, iterating _VISIBLE_MARK_PLATFORM), each gated by its own engine NCC threshold via MarkDetection.detected (Doubao 0.50, Jimeng 0.45, Qwen 0.45, Samsung 0.40). Doubao/Jimeng are normally also caught by the TC260 AIGC metadata label and Samsung by its C2PA + genAIType marker, so the visible path is their stripped-metadata fallback. Visible marks set platform only when no harder signal already did, and (like the sparkle) are excluded from integrity-clash vendor claims. The cv2 dependency lives in the engines, not here.
import identify is deliberately light (~26 MB; ~36 MB with cv2 loaded by a visible-mark run, ~106 MB for a full check_visible run): it imports the noai.c2pa/noai.constants submodules, and noai/__init__ is lazy (see "Test and lint"), so torch/diffusers are NOT pulled at import even in a full gpu/detect install — fits a 512 MB host. noai.c2pa does eagerly import the c2pa-python binary (Rust + cryptography, ~+5 MB RSS, no torch) for the primary Reader path — light enough to stay on the dependency-light host; a broken/absent wheel degrades to the byte-scan parser (reader_available() False). The heavy paths are opt-in: check_invisible=True needs the detect/trustmark extras (each pulls torch; TrustMark also downloads weights), so on a core-only deploy leave check_invisible off (it is a no-op there anyway). Before the lazy __init__, the mere presence of torch in the env inflated import identify to ~420 MB.
C2PA platform attribution is device-token-first, issuer-scan fallback (_device_platform scans manifest bytes for _DEVICE_C2PA_PLATFORM tokens, then _attribute_platform/_ISSUER_PLATFORM).
Why, verified on real signed files 2026-05-26: the old issuer-only byte-scan matched ANY issuer substring anywhere, so multi-entity manifests mis-attributed -- Leica→"Truepic" (a signing authority in the trust chain), Nikon→"Adobe Firefly" (XMP-toolkit "Adobe" + the sample's "Adobe_MAX" name), Pixel→"Google (Gemini)" ("Google LLC" cert org), Truepic→"Google". A distinctive device token wins instead.
Token distinctiveness is load-bearing: bare b"Truepic" mis-fires (it appears in unrelated trust chains -- it mis-attributed the OpenAI chatgpt-1.png fixture), so the token is the specific b"Truepic_Lens" from the Lens SDK claim generator; likewise b"Pixel Camera" (cert CN) not bare b"Pixel". _DEVICE_C2PA_PLATFORM lists ONLY tokens verified against a real C2PA file: Leica (lc_c2pa/Leica Camera), Nikon (NIKON), Pixel (Pixel Camera -- from a real Pixel 10 Pro file attached to c2pa-rs issue #1609/#1554), Sony (sony.sig/sony.cert -- Sony's own C2PA assertion namespace, verified on a real Sony PXW-Z300 file; NOT bare "Sony" which is a common EXIF Make), Truepic (Truepic_Lens). Canon/Bria have no public direct-download C2PA sample (checked exhaustively: GitHub issue/PR attachments, contentcredentials gallery, HF datasets -- all upload-to-verify or token-gated; Canon's only public file was a self-signed hobbyist CR3, not factory), so they stay unmapped until a real file is captured (same fixture discipline as Grok/Doubao). The Sony sample is video (MP4) -- our ISOBMFF C2PA path detects it; Sony Alpha stills likely share the sony.* namespace but are not separately verified.
Samsung Galaxy + ASUS Gallery live in a separate _SIGNER_C2PA_PLATFORM (scanned after _device_platform, before the issuer fallback), NOT in _DEVICE_C2PA_PLATFORM — verified on real signed files 2026-05-29. Reason: a Galaxy phone stamps BOTH its device cert AND a trainedAlgorithmicMedia/genAIType AI marker on a Generative-Edit image, so treating it as a "genuine camera capture" would false-fire integrity-clash rule 2 on every Galaxy AI edit. The signer tokens (b"Samsung Galaxy" cert org — distinct from the EXIF SM-xxxx model string on ordinary Samsung photos; b"com.asus.gallery" claim generator) only resolve the platform label; the AI verdict still comes from the source-type / genAIType. ASUS Gallery is a C2PA-signed edit with no AI marker, so it attributes the platform without asserting is_ai.
Samsung's genAIType (in the proprietary PhotoEditor_Re_Edit_Data JSON) is an undocumented Galaxy-AI editing marker (metadata.samsung_genai, gated on the PhotoEditor_Re_Edit_Data container; non-zero value = AI tool used, values {1,5} observed; Galaxy AI appends it as a trailer AFTER the JPEG EOI, so samsung_genai reads the file TAIL when the 512 KB quick-scan head misses it — else a multi-MB photo's trailer past the window went undetected while removal, which reads the whole file, would still strip it; removal truncates the post-EOI Samsung trailer via metadata._strip_samsung_trailer, pixels bit-identical): medium-confidence because the field has no public spec (verified 2026-05-29: absent from C2PA spec + Samsung docs), but it co-occurred with trainedAlgorithmicMedia in 3/3 verified files that record a source-type and was the SOLE AI marker on a Galaxy S24 file that omits the source type. Camera C2PA marks capture authenticity, not AI (Pixel carries computationalCapture, not trainedAlgorithmicMedia), so these never set is_ai -- that stays driven by digital-source-type. c2pa.cbor_text_after (now public) is best-effort for the generator detail string only and can be None when the manifest keys it claim_generator_info (Pixel).
Issuer→generator mapping is is_ai-gated (_attribute_platform(issuers, is_ai=c2pa_is_ai)): a specific AI-generator platform is named only when the digital-source-type is trainedAlgorithmicMedia; on a non-AI source an issuer substring is treated as incidental (an "Adobe XMP" toolkit string in an unmapped Canon/Sony capture would otherwise mislabel it "Adobe Firefly"), so it degrades to the neutral "C2PA signer: X" label. The one exception is an identity-AI issuer (c2pa_is_ai = c2pa_source_kind is not None or c2pa_identity_ai, where c2pa_identity_ai is any resolved issuer org in C2PA_IDENTITY_AI_ORGS): a vendor flagged asserts_ai (today only Dreamina) sets c2pa_is_ai True on its own, so its platform resolves even though the manifest carries no trainedAlgorithmicMedia. This is safe precisely because the flag is restricted to distinctive brand strings, not the incidental-mention-prone common words. Real Firefly/OpenAI/Google output carries the AI source-type, so it is unaffected (verified: chatgpt-1.png→OpenAI, firefly-1.png→Adobe Firefly still attribute). _attribute_platform defaults is_ai=True so the mapping stays unit-testable in isolation. Add capture-camera tokens to _DEVICE_C2PA_PLATFORM, editing-app/AI-device signer tokens to _SIGNER_C2PA_PLATFORM, generator/issuer platforms to the C2PA_AI_VENDORS registry in constants.py (which derives _ISSUER_PLATFORM), not inline. For non-PNG containers (JPEG/WebP/AVIF/HEIF/JXL) the caBX parser returns nothing, so issuer (_issuers_in) and generator (_ai_tools_in, reusing C2PA_AI_TOOLS) are recovered by binary-scanning the first MB. EXIF Software / Make / Artist / ImageDescription, XMP CreatorTool, and PNG tEXt chunks (Software/Source/Title/Description — NovelAI stamps its generator there, not EXIF) are read by metadata.exif_generator (PIL+piexif for any format PIL opens incl. AVIF, plus a container-agnostic XMP raw-byte scan that also covers HEIF/JXL), matched against AI_GENERATOR_TOKENS so ordinary editors (plain "Adobe Photoshop") and real-camera Make ("Apple"/"Canon") are not flagged. Tokens mined from the retained corpus 2026-06-22: novelai, reve.com (full token, not bare reve), aphrodite ai — all no-C2PA generator stamps that previously read as no-signal (and under the P0#5 no-signal skip would have skipped the scrub).
Ideogram tags its output with EXIF Make="Ideogram AI" (verified on a real download 2026-05-24) — that's why Make is read.
Integrity-clash detection (_integrity_clashes, surfaced as ProvenanceReport.integrity_clashes, printed in red by identify and serialized to --json): contradictions between independent generator stamps are a laundering/spoofing tell. Two rules: (1) two or more distinct AI-origin vendors named by independent signals (e.g. C2PA OpenAI + EXIF Make="Ideogram AI"), and (2) a camera-capture C2PA device (_DEVICE_C2PA_PLATFORM) coexisting with an AI-generation marker from a source INDEPENDENT of the camera's own manifest.
Rule 2's independence gate (added 2026-06-11): a device that both captures and runs on-device generative AI (Google Pixel Magic Editor / Pixel Studio) records the capture AND the AI edit in ONE C2PA manifest — so the AI vendor is named only from that same manifest (c2pa issuer + synthid proxy, both c2pa_manifest source) — a legitimate edit chain, NOT a clash. Rule 2 therefore fires only when some ai_vendor_claims family has a source != "c2pa_manifest" (EXIF/XMP generator, IPTC, TC260 AIGC, a second manifest naming AI on a camera capture — the real laundering tell). This killed a false-positive class on the corpus: 2 real Pixel generative-edit PNGs (computationalCapture + trainedAlgorithmicMedia + "Applied imperceptible SynthID watermark" in one Google manifest) read as camera-vs-AI clashes before the gate. Pure cameras (Leica/Sony/Nikon/Truepic) that do NOT generate AI still clash on any within-manifest AI marker only if it is independent — they never legitimately carry one, so the gate is behavior-neutral for them while fixing Pixel (regression-guarded by test_identify.py::TestIntegrityClashesHelper::{test_pixel_generative_edit_same_manifest_no_clash,test_camera_plus_independent_ai_marker_still_clashes} + TestIntegrityClashEndToEnd::test_pixel_generative_edit_no_clash).
Independence is source-grouped (_CLASH_SOURCE, added 2026-06-02): the C2PA issuer attribution (c2pa) and the SynthID proxy (synthid) are NOT independent — the proxy is inferred from the same manifest — so they share one source and two vendors named within a single manifest do not clash. This killed a false-positive class found on the spaces corpus: legitimate multi-actor manifests where a product wraps another vendor's engine (Microsoft Designer on OpenAI → OpenAI, Microsoft; Microsoft on Google → Microsoft, Google LLC, Google C2PA Core Generator Library) or an edit chain re-signs (Adobe over a Gemini original → Adobe c2pa + Google synthid) — 19 such files across the 2026-06-01/02 batches read as clashes before the fix. Rule 1 still fires when a manifest vendor disagrees with a genuinely independent stamp (EXIF/XMP generator, IPTC AISystemUsed, AIGC, xAI); each non-c2pa/synthid family is its own source (test_identify.py::TestIntegrityClashes::{test_multi_actor_manifest_no_clash,test_manifest_vendor_vs_independent_signal_clashes}). Vendor normalization is _vendor_of over _AI_VENDOR_TOKENS (so a C2PA "Google (Gemini)" issuer and a SynthID-Google proxy agree, while different vendors clash). _AI_VENDOR_TOKENS covers ByteDance (all brands: bytedance/doubao/jimeng/dreamina/volcengine), Canva, ElevenLabs, and Black Forest Labs in addition to the OpenAI/Google/Adobe/... set — without them a transplanted ByteDance/Canva/BFL C2PA manifest next to an independent conflicting stamp was silently missed. The generic China AIGC (TC260) label names no SPECIFIC vendor (any Chinese generator applies it), so it cannot vendor-conflict in the spoofing sense: when a Chinese TC260-applying vendor (_TC260_VENDORS, today {ByteDance}) is co-attributed, Rule 1 attributes the label to that vendor (a legit Doubao image carries BOTH a ByteDance C2PA manifest and its own TC260 label and must not clash); against a NON-TC260 vendor (OpenAI etc.) the label stays generic and still clashes as a laundering tell (test_bytedance_c2pa_plus_own_aigc_no_clash, test_foreign_vendor_plus_aigc_still_clashes, test_bytedance_c2pa_plus_foreign_generator_clashes). Corpus-validated: 0 new clashes on 5000 ByteDance/AIGC/Canva/FLUX carriers.
High-precision by design: only hard generator stamps feed it (C2PA-issuer when source is AI, SynthID, EXIF/XMP generator, IPTC AISystemUsed, xAI, AIGC); the fuzzy visible sparkle and the open invisible watermark are excluded (both are low-precision/positive-only signals; the open watermark was also historically a by-product of our own SDXL removal pass, until watermark_remover was fixed to load the SDXL pipelines with add_watermarker=False — it stays excluded as a fuzzy signal regardless). The c2pa vendor is classified from the issuer attribution / generator, NOT the resolved platform (a camera label like "Google Pixel" would mis-normalize to "Google"). All real single-origin fixtures (chatgpt/firefly/doubao/grok/mj) verified to produce zero clashes (false-positive guard in test_identify.py::TestRealSamplesHaveNoClash).
ai_from_metadata field + has_invisible_target helper (P0#5, 2026-06-22): the high-confidence union (everything that sets confidence == "high": C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — the medium-confidence hf_only/visible_only/samsung_only are excluded) is now surfaced as the public ProvenanceReport.ai_from_metadata boolean, so callers gate on intent rather than on the confidence string. has_invisible_target(path) wraps identify(path, check_visible=False, check_invisible=True) and returns that field — it is the decision gate for the diffusion scrub (the CLI invisible/all/batch no-signal skip, cli._no_invisible_signal_exit): a visible-only or no-signal image has it False, so regeneration (which would only degrade a clean image) does not run. It fails SAFE — any detector exception returns True so the removal still runs (leaving a watermark on a paid removal is worse than over-regenerating). It does NOT prove a pixel SynthID is absent (SynthID is detectable only via its metadata proxy, gone once stripped), so a False means "no locally-detectable target", never "clean". Guarded by test_identify.py::{TestIdentifyRealSamples::test_has_invisible_target_*,TestHasInvisibleTargetFailSafe}.
watermark_registry.py
watermark_registry.py — single catalog of known visible watermarks, the unified "find known marks in their usual places, recognize, remove" entry.
Localize -> fill by policy (replaced reverse-alpha): each mark is localized to a binary full-frame footprint mask (a Localization), and one shared, swappable fill inpaints that mask via fill(image, mask, backend=...) (delegates to region_eraser.erase). This replaced the old reverse-alpha removal (invert a captured alpha map, original = (wm - a*logo)/(1-a), plus a thin residual inpaint) for ALL marks — gemini, doubao, jimeng, samsung, and jimeng_pill. Why it changed: reverse-alpha depended on a fixed captured alpha map at a fixed position, so it broke whenever a vendor moved or re-rendered its mark; and it was not color-lossless even with the right map (it amplifies 8-bit quantization and JPEG-chroma error by 1/(1-a)), which showed up as "the color just changed, not removed" reports. Localize -> fill has a benign failure mode: a slightly-off localization just inpaints a small region near-losslessly instead of leaving a color-shifted smear. The captured alpha maps are still used to DETECT the marks and to shape the mask (gemini's footprint), but NOT for pixel recovery. Fill backends: cv2 (classical inpaint, no deps, the floor), migan (MI-GAN ONNX, light, the memory-tight pick where LaMa will not fit), lama (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); auto = LaMa > MI-GAN > cv2, best available. Each KnownMark ties a key to {usual location, in_auto flag, a _detect callable → uniform MarkDetection, a _mask callable → full-frame footprint mask}; KnownMark.remove(image, *, backend="auto", provenance=False, force=False). Entries today: gemini (bottom-right sparkle), doubao (bottom-right "豆包AI生成"), jimeng (bottom-right "★ 即梦AI"), qwen (bottom-right "千问AI生成", Alibaba Tongyi Qianwen), samsung (bottom-LEFT "✦ Contenuti generati dall'AI", Samsung Galaxy AI, Italian locale), and the capture-less jimeng_pill (top-left "AI生成"). detect_marks(image, *, provenance=frozenset()) scans all (strict, for the identify verdict); remove_auto_marks(image, *, sensitivity="auto", provenance=frozenset(), backend="auto") removes every detected mark in one pass. Sensitivity (auto/strict/assume_ai) decides how hard a borderline mark is trusted: the visual detectors are pixel-based (no metadata needed) and the recall gain comes from relaxing the false-positive gate, not from metadata. resolve_trust turns the policy + evidence into the per-mark trust level the engines consume as provenance = level != "strict" — strict never relaxes; auto relaxes only on same-product evidence (metadata provenance for that vendor, or a confidently strict-detected sibling of the same _PRODUCT_OF — Doubao and Jimeng are both bottom-right ByteDance but distinct products, so they do NOT cross-relax); assume_ai relaxes every mark (the caller asserts AI, e.g. a metadata-stripped screenshot). Three levels, not two: strict / assumed / confirmed. Relaxing bypasses the engine's false-positive gate outright, and that bypass is contracted to mean the vendor is CONFIRMED (GeminiEngine.detect_watermark's trust_provenance: "external metadata already proves this is a Google generation"). An assume_ai caller asserts the image is AI, which says nothing about WHICH vendor, so a mark relaxed on assumption alone must also clear _ASSUMED_CONF_FLOOR (assumed_floor_ok; gemini 0.50) — see "Assumed-trust confidence floor" below. Perception / decision / action are separated three ways (the removal path only; identify keeps calling KnownMark.detect directly, so its verdict is untouched): _build_candidates(image) is PERCEPTION — it runs each detector at both trust levels and packages raw verdicts + the pill's flatness feature into Candidates, no policy; decide(candidates, Context(sensitivity, provenance)) -> [Decision] is the pure DECISION arbiter — all keep/drop policy (resolve_trust cross-mark corroboration + the assumed-trust floor + the pill gate) in one image-free, unit-testable function (tests/test_watermark_registry.py::TestArbiter); then remove_auto_marks does the ACTION, localizing -> filling each winner. The Gemini FP gate deliberately stays inside gemini_engine (not the arbiter) because identify reads that same gated confidence — pulling it out would drift the provenance verdict. Behavior was byte-identical to the pre-arbiter two-pass when the arbiter landed; assume_ai has since gained the assumed-trust confidence floor (see below), which deliberately changes its verdict on weak gate-bypassed matches.
Head-to-head validation (v0.12.1 reverse-alpha vs the current localize -> fill): run over the full labelled visible-mark set, with the cv2 / MI-GAN / LaMa fills each compared against the old reverse-alpha. doubao and jimeng are identical across every backend -- 100% coverage and 100% clearance either way. gemini strict coverage is a few points below reverse-alpha's (the deliberate false-positive tightening), but the metadata-stripped faint ones are now mostly recovered by the DEFAULT white-core rescue in the FP gate (gemini_engine: a bright near-WHITE core distinguishes a real faint sparkle from a colored bright corner -- ~14/20 recovered at ~1.25% clean false-fire; a learned classifier on the same features measured worse, 2026-07 tier-1), the residual under assume_ai; clearance is equal (~98% both), and neither version touches pixels outside the mark box (outside-box PSNR ~99). Clearance is fill-independent -- cv2, MI-GAN and LaMa all strip the mark's shape equally, so the re-detect metric does not separate them; the difference is purely the visual fill quality on the recovered region, and it is background-dependent. reverse-alpha recovered textured and especially regular/structured backgrounds (a lattice, a grid) more cleanly than any inpaint; LaMa closes most of that gap (the best learned backend), MI-GAN can ghost or hallucinate structure, and cv2 smears (the last-resort floor). This is why auto resolves LaMa > MI-GAN > cv2 (preferred_inpaint_backend) and warns once on the cv2 fallback; on flat backgrounds every backend is clean.
assume_ai removed (2026-07-19). It relaxed EVERY mark's false-positive gate on the caller's bare assertion that an image is AI. That assertion says nothing about WHICH vendor or WHERE the mark is, which is exactly what a gate bypass is contracted to require -- before it carried a confidence floor it filled a phantom sparkle on 59.8% of genuine camera photos, and even with the floor it was a statistical gamble rather than an instruction. It also had no place in the product's model: detector finds a mark -> remove it; detector finds nothing -> leave the image alone; the USER sees a mark and says so -> act on that.
Removing it collapsed the trust ladder from three levels to two (strict / confirmed) and took _ASSUMED_CONF_FLOOR, assumed_floor_ok and the assumed level with it. _keep_pill lost its sensitivity parameter (its assume-arm is gone; the metadata arm and its flatness guard are unchanged). Verified on the 240-image unbiased recall sample: doubao 92%/99%, gemini 96%/80%, jimeng 71%/71%, pill 50%/60% -- identical before and after, so nothing on the default path moved.
The replacement advice is PER MARK, because the forced paths are not equally reliable (measured 2026-07-19):
| path | reliability |
|---|---|
erase --region x,y,w,h |
sound by construction -- the user supplies the coordinates |
--mark <text-mark> --no-detect |
reasonable: the forced mask is the real glyph blob, non-empty on 13/13 missed doubao marks |
--mark gemini --no-detect |
NOT recommended -- falls back to a fixed default sparkle slot, which covered the true sparkle on only 31% of 97 sparkles the strict gate missed (median offset 63px up-and-left). The other 69% fill a clean corner AND report a removal that did not happen. |
cli._no_visible_mark_exit therefore recommends erase --region first and a named text mark second, and never suggests forcing gemini. It previously recommended --sensitivity assume-ai, i.e. the product's own hint contradicted its model.
Migration is LOUD, not silent. Sensitivity is a Literal and unenforced at runtime, so a 0.15 caller passing sensitivity="assume_ai" would otherwise get auto behaviour in silence -- a quiet semantic change on exactly the release where they need telling. validate_sensitivity (called by api.remove_visible and by Context.__post_init__) raises a ValueError naming the replacement. Regression: tests/test_watermark_registry.py::TestNoBlanketRelaxation.
Continuous top-hat detection front-end (detect_frontend / tophat_response, 2026-07-18). extract_mask thresholds the white top-hat into a 0/255 glyph blob and correlates a binary silhouette against it. That is fine for a mark stamped bold and opaque, and destructive for a faint one: a thin translucent overlay shatters into specks under the threshold, and no template can match a blob that is not there (千问 measured 0.170 mean NCC, 0% over its gate, against doubao's 0.723 / 82% -- same pipeline, each with its own template). The tophat front-end never binarizes: the saturation and absolute-luma gates become WEIGHTS instead of hard cuts, so a faint stroke contributes in proportion to its strength, and the response is max-normalized, which makes the score contrast-invariant.
Doubao is switched to it; jimeng and samsung stay binary until measured, because a front-end change must be measured per mark before it ships. Corpus effect on the 240-image unbiased recall sample:
| mark | recall before | recall after | precision before | precision after |
|---|---|---|---|---|
| doubao | 89% | 92% | 99% | 99% |
| jimeng / gemini / pill | unchanged | unchanged | unchanged | unchanged |
The gate is FRONT-END SPECIFIC and must be re-calibrated, not ported. The continuous response scores higher overall (mean 0.809 vs 0.723 on the same 90 positives), so the binary-era 0.40 left the provenance-relaxed gate (x0.7) far too low: at 0.40 the arm ran 96% recall / 91% precision (8 false fires), at 0.50 it runs 92% / 99% (1 false fire). 0.50 was chosen because it beats the binary front-end on recall at IDENTICAL precision -- a front-end that only trades one for the other would not have been worth shipping. A first pass at 0.40 also silently depressed the PILL (recall 50% -> 33%), because _keep_pill suppresses the pill whenever doubao fires; a coupling worth remembering when tuning any bottom-right mark.
The removal MASK must ride the same front-end, and how it does so was fixed twice. tophat detection does not binarize, but extract_mask (which bounds the fill) still does, so a mark faint enough to be found only by the continuous response produced an EMPTY binary blob: localize returned mask=None, remove() was a silent no-op, and identify reported visible_doubao while visible said "no visible mark" on the same file (corpus-measured 2026-07-20: 57 of 60 sampled still-detected Doubao marks untouched, ~8% of its detections). The FIRST fallback (2026-07-19) thresholded the continuous response and took the bounding box of everything above the level -- but the level was 0.5 compared against the max-normalized uint8 0..255 response, so it selected every non-zero pixel and filled ~120% of the corner box on textured frames (a padded whole-ROI box). It passed parity (a mask that fills everything is trivially detector-clean) and its regression test (a FLAT fixture, where the response is non-zero only on the glyph, so every threshold yields the same box). The SECOND fallback (2026-07-20) uses the detector's OWN best-match box instead: _tophat_score was split into _tophat_best(image, loc) -> (score, box), the single method whose score gates detection and whose argmax box bounds the mask -- so the two cannot drift by construction, which is how the mismatch arose in the first place. Measured over 14 real faint-path frames (cv2 fill, detector re-run after): the match box fills a 58.7%-median corner box vs the threshold's 120.9%, both 100% detector-clean. The largest-connected-component alternative was tighter (10.5%) but removed the mark on only 21% of frames, so it does not cover it and was rejected. Regression: tests/test_text_mark_faint_mask.py, whose fixture now carries texture (the flatness of the old one is exactly why it could not see the threshold bug -- mutating the constant to 99.0 left it green). Any future front-end change must move both the detection and the mask path, or re-check this.
Vendor ATTRIBUTION for the shared-suffix marks -- solved for 千问 by exact sizing, not by a generic template. The 2026-07-18 worry was that "千问AI生成" and "豆包AI生成" share the AI生成 tail (four of six glyph cells), so their templates would cross-fire. That held while Qwen was scored at the WRONG SIZE (AUC 0.41-0.59, a coin flip). Re-measured 2026-07-21 at the fitted geometry on real pools (scripts/vendor_mark_calibrate.py --crossfire): Qwen's template scores p50 0.224 / p90 0.242 on 400 Doubao-marked frames against a 0.45 gate -- 0 cross-fires, and 0 on 298 Jimeng-marked frames and 286 clean frames -- because an exact-size 6-glyph template is specific enough that the 2-glyph prefix mismatch dominates the shared tail. Qwen was therefore registered WITHOUT a rival margin (a 0.10 margin would have suppressed ~10% of genuine Qwen detections, whose margin p10 sits at 0.00). A GENERIC shared-tail template remains a harvesting aid, not a detector (measured: 0.407 on a bold positive vs clean p99 0.298).
千问 is registered since 2026-07-21; 星绘 is NOT (still one confirmed example). The 2026-07-18 measurement below is why 千问 was originally held out -- it is kept because it records the failure CLASS that the registration then had to solve. Measured back then on 14 hand-verified corpus positives, same pipeline, each mark scored with its OWN template:
| mark | n | mean NCC | median | above the 0.40 gate |
|---|---|---|---|---|
| doubao | 40 | 0.723 | 0.835 | 82% |
| qwen | 14 | 0.170 | 0.179 | 0% |
Three candidate explanations were ruled out in order, each by measurement:
- Not the synthetic render. A template cut from an ACTUAL Qwen mark scores the same as the font-rendered one (real-vs-real 0.307 vs synthetic 0.308), and real masks do not match EACH OTHER.
- Not the morphology kernel.
MORPH_OPEN/MORPH_CLOSEuse fixed 5px kernels regardless of mark size (~9% of a 57px-tall box, ~2.7% of a 188px one). Scaling them with the box height gained +0.014 mean and moved nothing across the gate. - Not the appearance thresholds. Sweeping
tophat_delta/logo_min_luma/ kernel size peaked at mean 0.35 with 4/14 over the gate.
The blocker was named as SEGMENTATION on a faint mark -- and the tophat front-end (built for Doubao the same week) removed exactly that blocker, yet 千问 still did not register, because the real residual was never segmentation alone: it was mis-sized geometry (two size modes the shared 3-rung ladder cannot straddle, plus a locate box that clipped the first glyph). The 2026-07-21 cohort harvest (117 labelled frames) and the full calibration chain are in the qwen_engine.py section below and docs/verification-plan.md. 星绘 remains where it was: ONE confirmed corpus example, so nothing to calibrate a gate against -- do not register it off a single frame. The synthetic renderer and the evidence chain are kept in scripts/render_vendor_silhouettes.py; researched vendor specs are in docs/watermarking-landscape.md.
RECALL, measured at last (unbiased random sample, 2026-07-18). Every earlier round sampled where detectors FIRED, so recall was structurally unmeasurable. This round draws 240 images at RANDOM within each provenance class (160 TC260, 80 Google-C2PA) and labels them EXHAUSTIVELY -- both corners shown at native scale, so a missed mark is visible as a miss rather than absent from the data. Build it with scripts/visible_recall_sample.py; labels live in the gitignored research dir.
| mark | present | recall | 95% CI | precision | 95% CI |
|---|---|---|---|---|---|
| doubao | 90 | 89% | 81-94% | 99% | 93-100% |
| gemini | 46 | 96% | 85-99% | 80% | 68-88% |
| jimeng | 14 | 71% | 45-88% | 71% | 45-88% |
| jimeng_pill | 6 | 50% | 19-81% | 60% | 23-88% |
Effect of the scale_basis fix on the same sample (strict verdicts recorded before it): doubao recall 71% -> 89%, gemini 91% -> 96%, jimeng 64% -> 71%. Part of the doubao gain is the provenance relaxation rather than the basis alone, since the after-numbers run the full auto path.
Three corrections this forced to earlier numbers:
- Gemini precision is 80% on an unbiased sample, not the 41% the addition-sampled harness reports. The 41% is precision restricted to relaxation ADDITIONS, which are by construction the marginal cases; production sees mostly strict fires, which are near-perfect. Quote 80% for the product and 41% only when discussing the relaxation arm.
- Doubao is in excellent shape (89/99) and is no longer the problem it looked like before the basis fix.
- Landscape is improved but NOT solved: doubao recall by aspect is portrait 92% / square 92% / landscape 79%, so a residual geometry gap remains beyond the basis.
Where the remaining loss actually is: jimeng and the pill, both at small n with intervals too wide to tune against (a 14-positive and a 6-positive sample), plus uncovered vendors at 6% of all sampled images (千问/百度/星绘/抖音-class marks that no registered detector can ever fire on). Adding those vendors is now a larger win than any further tuning of the covered four, and docs/watermarking-landscape.md carries their researched specs.
Per-mark geometry scaling (scale_basis / scale_base, 2026-07-18) -- the largest single recall defect found so far. Every tuned fraction in TextMarkConfig was calibrated on PORTRAIT captures, where the width and the short side coincide, so the scaling basis was never exercised until landscape inputs were measured. Corpus-measured on 2572 unique TC260 carriers, BEFORE the fix:
| aspect ratio | detected | missed | miss rate |
|---|---|---|---|
| tall portrait <0.70 | 323 | 212 | 40% |
| portrait 0.70-0.95 | 607 | 533 | 47% |
| square ~1.0 | 190 | 272 | 59% |
| landscape 1.15-1.6 | 0 | 143 | 100% |
| wide >1.6 | 0 | 292 | 100% |
Not one landscape image in the corpus ever produced a detection -- 435 of them, zero. A width-scaled box is inflated by the aspect ratio on a wide image, so the glyph never lands inside it and the blob never gets scored: of the 1452 no-detection TC260 images the median doubao NCC was 0.057, with 49% at ~zero. This is a LOCALIZATION failure, not a threshold one -- only 2.7% of those images sat in the band a threshold change could reach, which is why a day of threshold tuning could never have found it. Re-running the previously-undetected set with a short-side basis recovers 56% of landscape (12% square, 4% portrait; 20% overall).
The basis is PER MARK because the vendors genuinely differ. The same switch took jimeng's labelled landscape positives from 13/13 to 0/13: the Jimeng wordmark tracks the WIDTH while the Doubao strip tracks the short side, even though both are ByteDance and share a corner. So doubao is short, jimeng is width, and samsung stays width because there is no corpus evidence either way (1 addition corpus-wide) and an unmeasured change is not an improvement. China's GB 45438-2025 clause 5.2(e) mandates glyph height >= 5% of "the shortest side", which is why short-side is the natural prior -- but jimeng's measured behaviour overrides the prior. Regression: tests/test_text_mark_engine.py::TestScaleBasis.
How this was missed for so long: precision was measured repeatedly and recall never was. The eval harness now reports a missed column for exactly this reason, and it is what caught the jimeng regression the short-side switch introduced.
Competitive detection among same-corner marks (rivals / _rival_margin_ok, 2026-07-18). Detection was purely ABSOLUTE -- every engine scored its own template against its own threshold, so nothing ever asked the discriminative question "does this blob match the NEIGHBOUR's mark better than mine?". Doubao "豆包AI生成" and Jimeng "★ 即梦AI" both sit bottom-right in near-white CJK and survive the top-hat binarization as very similar blobs, so no absolute gate can separate them. Measured on hand-labelled examples, scoring BOTH templates against the SAME glyph blob (n=40 jimeng / 75 doubao / 20 other-vendor labels / 89 clean):
| feature | separability (0.5 = useless, 1.0 = perfect) |
|---|---|
absolute ncc_jimeng |
0.96 |
ncc_jimeng MINUS ncc_doubao |
0.99 |
At a 0.10 margin: real Jimeng wordmarks pass 100%, Doubao strips 8%, other vendors' AI labels (千问/百度/星绘/抖音) 55%, no-mark corners 12%. Corpus effect (scripts/visible_eval.py, 741 labelled images): jimeng precision 38% -> 63%, genuine detections unchanged at 40, false fires 65 -> 23. Because real marks pass at 100% this is a pure precision gain, unlike raising a threshold -- so the earlier 0.85 relaxation patch was REVERTED to 0.70 and the recall it had sacrificed came back. The gate is deliberately asymmetric: doubao declares no rival, because the symmetric gate cost it 7 genuine detections to prevent 5 false ones (1.4:1 against) while jimeng gained 25pp for free -- doubao's absolute detector is already 86% precise and has nothing to buy. A rival's config is looked up lazily by asset name (_rival_config) so its template is scored at ITS own geometry; scoring it at the host mark's geometry would compare a correctly-sized template against a mis-sized one and hand the margin a free win. Regression: tests/test_text_mark_engine.py::TestRivalMargin.
Evaluation harness (scripts/visible_eval.py + scripts/visible_groundtruth.py, 2026-07-18). Run before AND after any detector change; --save NAME snapshots, --vs NAME diffs. Ground truth is 741 blind-labelled corpus images (779 cells across two rounds, two-sided control each). Three properties of the harness are load-bearing and were each added after the naive version produced a wrong number:
- Adjudication scope. A crop centred on one mark only lets the labeller rule on marks visible IN THAT CROP. Scoring jimeng against a pill-round image (top-left crop) books real bottom-right detections as false fires -- ~61% of pills carry a wordmark. Each image records which marks its crop could rule on; bottom-right marks co-adjudicate each other.
- Provenance must come from METADATA, never from the labels. A relaxation arm only fires when provenance names the vendor, so label-derived provenance hands the detector the answer: it scored gemini at 99% instead of the true 41%.
- Recall is NOT reported. The labelled set was sampled where detectors fired, so images every detector missed are absent by construction; a recall computed here would divide by a denominator that excludes exactly the failures recall exists to expose. The
missedcolumn catches a change LOSING marks it used to find, nothing more. True recall needs a random corpus sample labelled exhaustively -- not yet done.
Baseline at the time of writing (sensitivity auto, provenance from metadata): gemini 41% (321 fires), doubao 86% (77), jimeng 63% (63), jimeng_pill 64% (83), samsung unmeasurable (1 addition corpus-wide).
Per-mark provenance NCC relaxation + the corroboration gate (2026-07-18). Two defects, both on the DEFAULT auto path (no flag, driven by TC260 metadata), found by blind hand-labelling the ADDITIONS (accepted with provenance, rejected without) over 4417 unique TC260 carriers. Two-sided control: labeller sensitivity 100% (doubao) / 96% (jimeng), specificity 100% / 100% — the controls are what make the low numbers trustworthy, and the "clean" stratum is structural (another vendor's C2PA image, where a ByteDance mark cannot exist) rather than detector-defined, so it is not circular.
(1) One shared _PROVENANCE_NCC_FACTOR = 0.7 meant two different things per mark:
| mark | band | precision | 95% CI | n |
|---|---|---|---|---|
| doubao | whole arm | 76% | 61-87% | 42 |
| [0.280,0.340) | 58% | 36-77% | 19 | |
| [0.340,0.400) | 91% | 73-98% | 23 | |
| jimeng | whole arm | 17% | 10-27% | 82 |
| [0.315,0.383) | 12% | 6-22% | 68 | |
| [0.383,0.450) | 43% | 21-67% | 14 |
The factor is now a per-mark TextMarkConfig.provenance_ncc_factor. Doubao stays 0.70 — both bands return more true marks than false fills, so tightening would cost 11 genuine recoveries to prevent 8. Jimeng moves to 0.85 (gate 0.3825), dropping the 12% band: −8 genuine recoveries, −60 false fills (7.5:1), arm precision 17% → 43%. Why jimeng fails is a detector problem, not a threshold one: of its 68 false additions, 33 were DOUBAO marks and 17 were other vendors' AI labels (千问 / 百度 / 星绘 / 抖音) — relaxed, the silhouette keys on "some text in the bottom-right corner", not on "★ 即梦AI". Damage was scored separately because doubao and jimeng share a corner: 45 of the 68 fill a corner nothing else would touch, the other 23 are harmless (doubao fires strictly there and fills the same box anyway). A better silhouette, not a lower factor, is the real fix.
(2) A weak detector must not corroborate a sibling (_CANNOT_CORROBORATE). resolve_trust grants confirmed on a strict-detected sibling of the same _PRODUCT_OF, and confirmed bypasses the sibling's FP gate outright. The pill (~7% documented raw false-fire; 5.5% on 578 vendor negatives) maps to product "jimeng", so it could hand that bypass to the wordmark — a closed loop: pill false-fires on clean non-ByteDance content → jimeng relaxes 0.45 → 0.3825 and false-fires → _keep_pill sees "jimeng" in keys and takes the WORDMARK arm, removing the pill unrestricted, skipping the flatness guard written to stop exactly that smear. 3 of 578 negatives ran the full loop, one with footprint_flat=0. The fix costs nothing: negatives 3 → 0, TC260 carriers unchanged (jimeng 398 → 398, pill 117 → 117). _keep_pill already encoded this distrust for the pill's ACTION; the gap was that its TESTIMONY was ungated.
Pill arms, re-measured on the same corpus (149 blind-labelled TC260-arm fires, 35 wordmark, 33 unconfirmed): wordmark 94% (CI 81-98%, confirming the original claim), TC260-metadata-only 21% raw (CI 16-29%, consistent with the original ~27%) — 29% (CI 20-40%) among the flat footprints the guard PASSES vs 14% among those it blocks. The guard works directionally but weakly: the shipped arm still runs at ~2.4 false fills per genuine one. Whether an arm that inaccurate belongs on the default path is a product call, not a tuning one.
Samsung's relaxation is UNMEASURED and not measurable here: the corpus holds 14 samsung_genai carriers and 3 visible Samsung detections total. Any precision estimate would carry a Wilson interval spanning most of [0,1]. Likely structural — detection is calibrated to the Italian locale string only.
Provenance prior: when local metadata already confirms the vendor, the mark's detection trust gate is relaxed (a confirmed vendor means the mark is present with high prior, so a mark the conservative detector would demote as a content false positive is trusted). detect_marks / remove_auto_marks take a provenance frozenset and KnownMark.remove a provenance flag. Mapping: a Google/Gemini C2PA issuer relaxes gemini (skips its false-positive gate and lowers the trust threshold from 0.5 to 0.35); a China-AIGC (TC260) label relaxes doubao/jimeng; samsung_genai relaxes samsung. Corpus finding: on Google-C2PA images, Gemini sparkle recall rose from ~46% (plain detector) to ~90% with the provenance prior (recovering marks the vendor moved or re-rendered). That gain is why the bypass exists, and it is conditional on the metadata actually naming the vendor — a caller merely ASSUMING the image is AI does not get it unconditionally (see the assumed-trust confidence floor above). The localizer is cheap CPU (cv2/numpy), so a memory-tight caller runs it anywhere; the heavy MI-GAN/LaMa fill is opt-in and chosen by the caller.
Cross-engine confidences aren't directly comparable, so the gemini adapter applies the corpus-validated 0.5 sparkle threshold (_GEMINI_AUTO_MIN_CONF) for its detected flag (lowered to 0.35 under the Google/Gemini provenance prior) — otherwise the gemini engine's loose internal threshold weakly fires (~0.36) on the Doubao text and hijacks auto. The shape-keyed Doubao/Jimeng/Samsung NCC detectors don't cross-fire (jimeng scores ~0.22 on the Doubao strip, well under its 0.45 threshold; Samsung is bottom-left so it shares no corner with the others, and scored 0.0 on Doubao/Jimeng captures and they 0.0 on a real Samsung photo), so auto picks the right one. cli.cmd_visible is registry-driven: --mark auto → remove_auto_marks (removes every detected mark), --mark <key> → that mark; --mark choices come from mark_keys().
cli._remove_visible_auto is the shared visible-removal helper used by cmd_all/cmd_batch too (they no longer hardcode GeminiEngine), so all/batch remove Doubao/Jimeng/Qwen/Samsung text marks, not just the Gemini sparkle (regression-guarded by test_all_visible_step_uses_registry). The three text-mark adapters were consolidated 2026-06-09: a single _text_mark(key, label, location) builds the registry row from one parameterized _text_mark_detect/_text_mark_remove pair (the remove adapter localizes the glyph footprint and hands it to the shared fill only when detected/forced, else skipped); the gemini adapters stay bespoke. Add a new visible mark = one _text_mark(...) row + its TextMarkConfig (with a captured alpha map for the detection silhouette); do not re-add per-mark if branches or copy-paste adapters.
Alpha-on-save policy (issue #30): image_io.write_bgr_with_alpha (it lives in image_io, not cli — moved so the CLI and the library api share ONE implementation) rejoins the input's alpha plane unchanged — it must NOT zero alpha in the watermark bbox. The fill reconstructs real pixels there, so zeroing alpha punched a transparent hole that renders as a solid white box on any non-transparent viewer (Gemini app exports are opaque RGBA, so every user hit it; regression-guarded by test_visible_keeps_alpha_opaque_in_watermark_region). The registry remove() still returns its region, but the CLI no longer uses it to clear alpha. It returns imwrite's success flag and callers must check it (2026-07-20): imwrite is contractually non-raising, so that bool is the only signal the file was not created. The wrapper previously returned None and swallowed it, so every CLI write site ran output.stat() to report the size and a read-only destination died with a bare FileNotFoundError traceback pointing at the stat instead of the write. The CLI now writes through the shared cli._write_output_or_exit. Regression: tests/test_cli_robustness.py::TestFailedWriteIsReported.
gemini_engine.py
gemini_engine.py — visible Gemini-sparkle remover/detector (cv2/numpy, no GPU). detect_sparkle_confidence(path) is the file-level entry point used by identify.py. The public entry points normalize a grayscale (2D) or RGBA (4-channel) input to BGR up front so a non-BGR image does not crash the cv2 pipeline.
Detection localization (issue #36): detect_watermark's global multi-scale NCC search applies a size weight ((scale/96)**0.5) that suppresses tiny-patch false positives but can let a larger, mediocre match (e.g. a bright collar in a portrait) outrank a small, near-perfect sparkle in the corner — so a faint sparkle on a busy background scored below threshold and read as clean (the regression osachub reported from widening the search window 256px->512px between v0.7.2 and v0.8.8). _corner_promote adds a bottom-right-corner raw-NCC pass on top of the global search: a match with raw NCC >= _CORNER_PROMOTE_NCC 0.85 that beats the global pick overrides it (it only ever replaces a lower-fidelity pick, so it cannot weaken an existing detection), rescuing the buried sparkle without reverting the wider window. The corner side is relative-clamped (_CORNER_PROMOTE_FRAC 0.20 of the short side, clamped to [_CORNER_PROMOTE_MIN 96, _CORNER_PROMOTE_MAX 384]): a fixed 256px is a true corner on a large image but covers ~70% of a small portrait, where a real photo raw-matches the star at ~0.81 (relative tightening drops that worst case to ~0.69, while the upper clamp stops the corner ballooning on huge images where a real photo reached ~0.83 at 512px). The 0.85 gate sits midway between the worst real-photo corner match (~0.78 across native + downscaled negatives) and a genuine faint sparkle (~0.93), so promotion adds true detections with zero corpus false positives (Gemini's sparkle sits ~60-160px from the corner at fixed margins, covered by the [96, 384] band at every measured size). Regression-guarded by test_gemini_engine.py::TestCornerPromotion.
Top-K fusion selection (osachub follow-up 2026-06-12): _corner_promote's 0.85 raw-NCC gate still missed a class the 256->512 widening exposed — a genuine MID-scale sparkle whose raw NCC sits below 0.85 but is buried by a LARGER, low-fidelity decoy that wins the size weight. The reporter's image (a scale-48 sparkle on light bedding) measured spatial 0.775 / grad 0.960 / fusion 0.676 at the true sparkle, but the size-weighted argmax instead locked onto a decoy at spatial 0.628 / grad 0.036 (fusion 0.325) — so identify read unknown on v0.8-0.11 where v0.7.2 (256px window) had caught it at 0.676. Fix: detect_watermark now keeps the top-_SELECT_TOPK (3) size-weighted candidates (NMS-deduped by location) plus the corner-promote candidate, scores EACH by the full fusion (spatial+gradient+variance) via the extracted _grad_var_scores helper, and selects the highest — the gradient term (the discriminator a contrast-invariant spatial NCC lacks) lifts the true sparkle over the decoy. Critically, selection ranks by the SIZE-WEIGHTED score, NOT raw NCC: a raw-NCC argmax (tried first) re-admitted the exact tiny-patch (scale 16-18) false positives the size weight exists to suppress — it flagged 14/65 doubao + 4/11 jimeng visible-corpus images (non-Gemini content) as Gemini sparkles. Top-K keeps tiny-patch suppression intact: a coincidental 16px match never ranks in the size-weighted top-K, so widening selection added zero flips on the doubao/jimeng corpora and left the 495-image Gemini set unchanged (479 detected, both before and after) while recovering the reporter's image. Regression-guarded by test_gemini_engine.py::TestCornerPromotion::test_low_gradient_decoy_loses_to_high_gradient_corner_sparkle (mirrors the real spatial/grad signature via a monkeypatched scan) and test_size_weighted_search_alone_traps_on_the_decoy.
Square-image residual misses are NOT fixable by lowering the detector threshold (measured + REJECTED 2026-06-11): osachub (#36 follow-up) reported the corner-promote still misses Gemini sparkles on Google square (1:1) outputs. Reproduced on the spaces corpus: of 330 square Google-C2PA images, 140 score below the identify 0.5 threshold, and visual review confirmed a real class -- faint white sparkles on dark/textured/colored backgrounds (raw NCC 0.46-0.73, below the 0.85 promote gate) landing at fusion conf 0.41-0.47. A margin-gated promote (promote when raw NCC >= 0.50 AND _core_ring_margin >= 40) rescued 32/33 confirmed misses at an apparent 0 FP, but that 0 was a measurement artifact -- the negative set was the margin<40 misses, which a margin>=40 gate excludes by construction. On an honest 518-image non-Google pool the same gate fired on ~174 (≈33%), visually content (screenshots, Chinese "AI生成" Doubao/Jimeng text marks, logos, bright textures), not sparkles. Adding an achromatic-core constraint (chroma <= 15) did not separate them either (kept 15/33 POS, 41 NEG still firing). Root cause is the documented contrast-invariant-NCC wall: a faint sparkle on a busy background is indistinguishable from a bright/ornate content corner at the (shape-NCC, brightness-margin, core-chroma) feature level.
Conclusion: keep the 0.85 corner gate; do NOT add a margin/chroma-gated lower promote.
The cost (mislabel ~8-33% of non-Gemini content as Gemini) outweighs the benefit -- the visible sparkle is a medium-confidence stripped-metadata fallback, and intact Gemini is caught by C2PA in identify regardless. Remaining square misses are an accepted known limitation; a real fix would need a sparkle-specific discriminator (template match on a background-subtracted image, or a hard fixed-margin position prior), which is open research, not a threshold tweak.
Removal is localize -> fill (footprint_mask → watermark_registry.fill): footprint_mask returns the sparkle footprint = the captured alpha (computed alpha = max(R,G,B)/255 from the bundled sparkle-on-black captures assets/gemini_bg_{96,48}.png, capture max ~130 for the ~51%-opaque overlay) thresholded LOW so the faint halo is included, then dilated by a sparkle-relative margin. That binary mask is inpainted by the shared fill (cv2 / MI-GAN / big-LaMa). The captured alpha maps are used only to detect and to shape the mask, not for pixel recovery. This replaced the old reverse-alpha removal path; because the fill only reconstructs the masked footprint from its surroundings (rather than dividing by 1-a), the whole reverse-alpha removal tail — the over-subtraction guard (_reverse_alpha_oversubtracts, the dark-background black-pit fix), the under-subtraction alpha-gain estimate (_estimate_alpha_gain), and the self-verify repair — is GONE, along with the near-white 1/(1-a) ill-conditioning and the "color changed, not removed" failure mode those guards patched around. A slightly-off localization now just fills a small region near-losslessly instead of leaving a color-shifted smear.
False-positive gate (added 2026-06-03): detect_watermark's shape-only NCC (spatial*0.5 + gradient*0.3 + var*0.2) fires on ornate/flat content (text strips, banners, hatching) that coincidentally matches the diamond shape — a real Gemini sparkle is a bright WHITE overlay, so its core sits above the local background, but the NCC is contrast-invariant and cannot see that. The fusion now demotes (caps confidence to 0.30) any low-confidence (< _SPARKLE_FP_CONF 0.65) match that shows NEITHER real-sparkle signature: a bright core (_core_ring_margin >= _SPARKLE_FP_MARGIN 5) OR a crisp star silhouette (gradient_score >= _SPARKLE_FP_GRAD 0.55). I.e. demote when low_margin OR low_grad. Real sparkles escape via high confidence (white-bg sparkles score ≥0.79 despite a low margin — the NCC shape match is strong), high margin (dark/mid backgrounds, incl. the #36 faint-corner case, lift well clear), OR high gradient (a real sparkle is grad ~0.97–1.0). The gradient condition (added 2026-06-26) closes the bright-background FP class the margin check alone missed: a snow+sky photo and a white-background product render both scored ~0.51 at identify, because a bright background gives the match a HIGH core-ring margin (it genuinely IS brighter than its surroundings), so the brightness gate read it as a real overlay — but a smooth luminance blob that shape-NCC-matches the rough diamond has low gradient fidelity (the two FPs measured grad 0.105 and 0.463 vs ≥0.8 for real sparkles), so the gradient floor demotes them. The OR is strictly a superset of the old margin-only demotion (it only ADDS demotions on bright backgrounds, where a real sparkle keeps grad ~0.97), so it cannot regress a dark/mid sparkle (kept by margin) or a white-bg one (kept by confidence ≥ 0.65). The gate is monotonic (only ever removes detections, never adds), so it cannot regress the verified-negative corpus (already 0 FPs); the 2026-06-26 corpus re-sweep flipped only OpenAI/ChatGPT content (no Gemini sparkle exists there) and already-cleaned/ outputs, all sub-0.5 (below the identify threshold), so no provenance verdict changed. The original gate demoted 16/495 flagged sparkles on the validation corpus (13 carried no AI metadata = content FPs; the 3 AI-meta were visually FPs / a near-invisible white-on-white sparkle whose AI verdict is held by metadata anyway). _core_ring_margin uses the _core_and_bg helper (core 75th-pct brightness vs background-ring median). This gate is detection-side and unchanged by the localize -> fill refactor; the provenance prior skips it when a Google/Gemini C2PA issuer confirms the vendor. Regression-guarded by test_gemini_engine.py::TestSparkleFalsePositiveGate (incl. test_bright_background_low_gradient_match_demoted).
The reverse-alpha removal tail is retired. The self-verify repair (_verify_and_repair), the offset+scale alignment search, and the near-white 1/(1-a) ill-conditioning survivors were all artifacts of solving the sparkle by inverting the alpha map. Under localize -> fill the footprint is reconstructed from its surroundings by the shared fill, so those failure classes and their guards no longer exist. The lesson from that era still holds and generalizes: a re-detect-confidence audit metric is gameable by reshaping the residual, so judge a visible removal by physical inspection of the footprint, not the detector alone.
The bg assets are rebuilt from OUR OWN controlled captures (data/gemini_capture/captures/, committed) by scripts/visible_alpha_solve.py gemini, which locates the 96px sparkle on the black capture and crops it to the two logo sizes; our capture matched the previously third-party-sourced gemini_bg_96.png to NCC 0.9998, validating the asset and making it reproducible. Gemini's multi-size fixed-slot model is genuinely different from the Doubao/Jimeng text-strip engines (so it stays a separate engine, not part of the shared-base refactor).
_text_mark_engine.py
_text_mark_engine.py — shared base for the three text-mark engines (Doubao/Jimeng/Samsung), extracted 2026-06-09 (they were ~90% byte-identical clones). TextMarkEngine(config: TextMarkConfig) owns the locate → extract_mask → detect detection pipeline plus the removal that localizes the glyph blob to a footprint mask and hands it to the shared watermark_registry.fill (+ the asset-keyed load_alpha_template/glyph_silhouette/template_match_score caches). Detection still matches the glyph silhouette (NCC against the captured template); the removal MASK is TEMPLATE-FREE — it is the bounding box of the top-hat glyph blob from extract_mask, filled solid + dilated, so a re-rendered or differently-placed mark is still masked. This dropped the fixed alpha-template placement; the captured alpha maps are now used only for the detection silhouette, not for removal. Each engine module is a thin subclass supplying only its TextMarkConfig (the tuned constants, the bundled asset, and the bounded structural deltas — corner br/bl, margin_floor 4/2, morph_open_size 5/3, min_gw 8/16, and since 2026-07-21 ladder — the scale rungs _tophat_best sweeps, per-mark because 千问's two size modes do not fit the shared 3-rung comb (default (0.8, 1.0, 1.25), unchanged for every other mark; densifying the SHARED ladder was measured and rejected -- see the verification plan's B2). plus the test-facing module shims (_alpha_template/_glyph_silhouette/_template_match_score + the constants). Gemini stays a SEPARATE engine (its multi-size fixed-slot sparkle model is genuinely different). Add a new text mark = a new TextMarkConfig + a thin subclass + one registry _text_mark(...) row. The engine bullets below describe each mark's calibration history; the LOGIC lives here. Small-image detection guard (_MIN_DETECT_SHORT_SIDE 200, added 2026-06-26): detect returns not-detected when the image short side is below 200px. Below that the glyph template degrades to the min_gw floor (~8px) and TM_CCOEFF_NORMED on a few pixels is noise, so an unrelated small geometric shape can spuriously correlate with the CJK silhouette — a 48×48 app-icon chevron scored Doubao 0.41 / Jimeng 0.47 (both above their thresholds), a pure small-size artifact (the same icon upscaled collapses to ~0.06–0.10 NCC at ≥256px). A real AI-generation label is stamped on a full-resolution render (the captured samples are 1086–2048px wide, the smallest positive test image is 1086px), so the floor sits far below any genuine mark while killing the icon/thumbnail band (≤96px); identify falls back to "unknown" (the safe default) and removal, gated on detection, is suppressed too. Regression-guarded by test_{doubao,jimeng,samsung}_engine.py::TestDetect::test_small_image_guarded_from_false_positive.
Removal is localize -> fill. The engine localizes the glyph blob (extract_mask over the located box) into a solid, dilated footprint mask and hands it to the shared watermark_registry.fill (cv2 / MI-GAN / big-LaMa). The template-free mask (bounding box of the glyph blob, not the fixed alpha template) means a re-rendered or moved mark is still covered, and the fill reconstructs the box from its surroundings. On corpus images doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal.
The reverse-alpha removal machinery is retired. The old per-glyph reverse-alpha blend (_apply_reverse_alpha), the fixed/aligned alpha-map helpers, the over-subtraction guard (_reverse_alpha_oversubtracts → _inpaint_footprint, the dark-pit fix on dark/mid-tone backgrounds), and the always-align placement search are all gone — the fill reconstructs the footprint from its surroundings rather than inverting the captured alpha, so the dark-pit and color-shift failure modes those guards patched around no longer arise. extract_mask still returns a box-sized ((loc.h, loc.w)) mask rather than a full frame, which keeps the memory-tight identify detect path cheap.
doubao_engine.py
doubao_engine.py — a thin _text_mark_engine.TextMarkEngine subclass (config only) since 2026-06-09. visible Doubao "豆包AI生成" detector + localizer (cv2/numpy, no GPU). DoubaoEngine.locate anchors a bottom-right box by geometry (mark scales with image WIDTH), extract_mask pulls the light, low-chroma glyphs (the detection candidate) using a per-pixel channel-spread proxy sat = roi.max(axis=2) - roi.min(axis=2) (no HSV conversion). detect is shape-consistent: it matches the bundled glyph silhouette (assets/doubao_alpha.png) against the candidate via zero-mean normalized correlation (_template_match_score, cv2 TM_CCOEFF_NORMED), gated at DETECT_NCC_THRESHOLD 0.4 over a small DETECT_MIN_COVERAGE floor. Keying on glyph SHAPE (not coverage heuristics) fixed #23 (corpus FP 7/1243).
Removal is localize -> fill: the glyph blob is localized to a solid, dilated footprint mask (extract_mask over the located box) and the shared watermark_registry.fill inpaints it. On corpus images this removes at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit).
The detection template (assets/doubao_alpha.png) is rebuilt by scripts/visible_alpha_solve.py (the careful gray-self solve: cubic background fit, mean over channels, full halo, unblurred), same recipe as Jimeng — the captures are committed in data/doubao_capture/captures/. It is used only as the detection silhouette, not for pixel recovery.
The locate box (WM_*) is generous (0.22 wide, margins 0.004) and reaches close to the corner so a re-rasterized, corner-ward-shifted mark still falls inside the localized box; regression-guarded by test_recovers_shifted_mark_on_texture (composes the mark shifted on a known texture). extract_mask guards a degenerate ROI (bh < 16 or bw < 16 -> empty mask, skips cv2) — an extremely wide/short image (e.g. 2048x1, test_wide_short_does_not_raise) once fed cv2's GaussianBlur a ~1-px-tall ROI and faulted natively on Windows py3.12; real images always clear the guard (the WM_* box floors are max(16, …) height / max(40, …) width), so it only short-circuits slivers. The registry gates removal on detect. The shipped third-party _refs/zhengsuanfa_doubao_alpha_120x20.png is NOT a usable template (verified 2026-05-29). Arbitrary-region inpainting is region_eraser/erase. Lesson from the reverse-alpha era (still holds): a detector-only removal test is insufficient; assert visual residual (the textured-shift test).
jimeng_engine.py
jimeng_engine.py — a thin TextMarkEngine subclass (config only) since 2026-06-09. visible Jimeng / Dreamina "★ 即梦AI" detector + localizer (cv2/numpy, no GPU), built 2026-05-30 from issue #13's solid captures (@powersee). Shares the base with doubao_engine: locate anchors a bottom-right box by geometry (scales with WIDTH), extract_mask pulls the light low-chroma glyphs (white top-hat + grayish + min-luma), detect matches the bundled "即梦AI" glyph silhouette (assets/jimeng_alpha.png) via TM_CCOEFF_NORMED over a coverage floor. Threshold DETECT_NCC_THRESHOLD 0.45 cleanly separates real Jimeng marks (>=0.81) from the Doubao strip (0.21) and other AI output (0.0), so the two ByteDance marks don't cross-fire in --mark auto.
The detection template (assets/jimeng_alpha.png) is rebuilt by scripts/visible_alpha_solve.py from the GRAY capture (data/jimeng_capture/captures/, the solid captures committed): a = (I - B)/(255 - B), B a per-capture cubic background fit over the non-glyph pixels, averaged over channels, full halo extent (down to a~0.02), unblurred. Gray (bg ~132) is the deliberate choice over black: it is the best proxy for real content (the mark sits on bright photo areas, not on black). The captured template is used only as the detection silhouette, not for pixel recovery. Solver geometry at _ALPHA_NATIVE_WIDTH 2048: _ALPHA_WIDTH_FRAC 0.202, _ALPHA_HEIGHT_FRAC 0.058, margins ~0.029.
Removal is localize -> fill: the glyph blob is localized to a solid, dilated footprint mask and the shared watermark_registry.fill inpaints it; the WM_* locate box is generous so a re-rasterized, corner-ward-shifted mark stays inside the localized box (the same widen that fixed Doubao). On corpus images this removes at ~100% with clean footprints (blends within a few LAB levels, no color shift). The registry gates removal on detect.
No committed real sample (only the solid calibration captures are committed) — tests/test_jimeng_engine.py synthesizes a mark from the bundled template, and test_recovers_shifted_mark_on_texture guards the localize-on-shift path that the Doubao defect exposed. Jimeng images are independently caught by the China TC260 AIGC label in metadata/identify, so this engine is the visible-mark removal path, not a new identify signal.
samsung_engine.py
samsung_engine.py — a thin TextMarkEngine subclass (config only) since 2026-06-09. visible Samsung Galaxy AI "✦ Contenuti generati dall'AI" detector + localizer (cv2/numpy, no GPU), built 2026-06-05 from issue #37's flat captures (@f-liva). Shares the base but anchored bottom-LEFT (Doubao/Jimeng are bottom-right): locate anchors a bottom-left box by geometry (scales with WIDTH), extract_mask pulls the light low-chroma glyphs (white top-hat + grayish + min-luma — LOGO_MIN_LUMA is lowered to 110 because the mark is faint, peak alpha ~0.38, so on a mid/dark background its glyph luma is lower than Jimeng's), detect matches the bundled glyph silhouette (assets/samsung_alpha.png) via TM_CCOEFF_NORMED over a coverage floor. Threshold DETECT_NCC_THRESHOLD 0.40 (real marks ~0.79 on a real photo, ~0.57/0.71 on the black/gray captures; 0.0 on Doubao/Jimeng captures, and Doubao/Jimeng score 0.0 on a real Samsung photo — no cross-fire, also because the corner differs).
The detection template (assets/samsung_alpha.png) is solved by scripts/visible_alpha_solve.py samsung from the GRAY capture (data/samsung_capture/captures/, the flat black/gray/white captures committed; the solver gained a corner="bl" mode + left-margin logging for this), same careful recipe as Jimeng (cubic background, mean-channel, full halo, unblurred). Geometry emitted at _ALPHA_NATIVE_WIDTH 1086 (the flat-edit capture width): _ALPHA_WIDTH_FRAC 0.3195, _ALPHA_HEIGHT_FRAC 0.0378, _ALPHA_MARGIN_LEFT_FRAC 0.0110, _ALPHA_MARGIN_BOTTOM_FRAC 0.0064. Used only as the detection silhouette, not for pixel recovery.
Removal is localize -> fill: the glyph blob is localized to a solid, dilated footprint mask and the shared watermark_registry.fill inpaints it. Verified on a real 2958-wide @f-liva photo: re-detect 0.79→0.00, no readable text or outline on the recovered wooden table — checked visually, not just by the detector. The registry gates removal on detect.
Detection is locale-specific (the string differs per language); this build detects only the Italian "Contenuti generati dall'AI" variant, so non-Italian Samsung locales are not detected — and, because detection gates removal, not removed — even though the fill mask itself is locale-independent. Other locales need their own detection silhouette — the locale string font-rendered and calibrated on real positives (the pill's scripts/render_pill_silhouette.py pattern), NOT an app capture (the solid/gray/white capture workflow retired with reverse-alpha). This is a pre-existing limit, unchanged by the localize -> fill refactor.
No committed real sample (only the flat calibration captures are committed) — tests/test_samsung_engine.py synthesizes a mark from the bundled template (bottom-left geometry), with test_recovers_shifted_mark_on_texture guarding the localize-on-shift path. Samsung Galaxy AI edits are independently caught by C2PA + the genAIType marker in metadata/identify, so this engine is the visible-mark removal path; it also feeds identify as the medium-confidence visible_samsung signal via the registry (the stripped-metadata fallback).
qwen_engine.py
qwen_engine.py — a thin TextMarkEngine subclass (config only), registered 2026-07-21. visible Qwen (Alibaba Tongyi Qianwen) "千问AI生成" detector + localizer (cv2/numpy, no GPU), bottom-right, the same GB 45438-2025 6-glyph house style as Doubao (2-glyph vendor prefix + mandated AI生成 tail; the vendor's tri-lobe logo precedes the text and is deliberately NOT in the silhouette -- logos vary between releases, the CJK run is what discriminates). The detection silhouette assets/qwen_alpha.png is font-rendered synthetic (scripts/render_vendor_silhouettes.py), never cut from an upload. Feeds identify as the medium-confidence visible_qwen signal via the registry.
Why this registration took two attempts: the 2026-07-18 attempt died at n=1 positives; the unlock was the TC260 label's ContentProducer field, whose USCC names the signing entity and partitions carriers into per-vendor cohorts from metadata alone (scripts/vendor_cohort_harvest.py -- 117 labelled Qwen frames, owing nothing to any pixel detector). Every constant was then MEASURED on that cohort against 286 hand-labelled clean frames (scripts/vendor_mark_calibrate.py), not inherited from Doubao:
- Basis
short(frac_short CV 0.189 vs width 0.273). - Per-mark 2-rung
ladder=(0.78, 1.27)againstalpha_width_frac0.160: the mark sits in TWO size modes (~0.124 and ~0.203 of the short side, ratio 1.64, both clusters tight) -- wider than the shared 3-rung ladder's 1.5625 span, so the best single fraction covers only 74.5% and the small mode lands in the comb's collapse zone (real marks at the exact rung score ~0.94 vs ~0.67 on the shared ladder). A 4-rung variant scored strictly worse (its big-mode rung sits 4.6% off the mode). The shared default is untouched for every other mark. - Fitted locate box (
width_frac0.231,height_frac0.074, margins ~0.021): the real mark sits ~0.025 of the short side off the right edge, and Doubao's box (0.004 margin) clipped the first glyph -- an exact-size template collapsed 0.73 -> 0.26 on a real frame. alpha_height_frac0.0416 from the aspect fit (p50 aspect 0.260 at the winning width) -- not the silhouette's own aspect (0.2219) and not Doubao's ratio.- Gate 0.45: clean p99 0.301 / max 0.316 (286 frames), and every cohort frame >= 0.45 carries a visible mark (83 of ~96 eyeballed visible marks fire = 86% recall of visible marks; the misses are white-on-near-white contrast losses). 0.45 was picked over 0.32 for margin against unseen clean content at zero measured recall cost.
- STRICT ONLY (
provenance_ncc_factor1.0): the score band just below the gate is dominated by non-Qwen banners on same-cohort frames (a 夸克 anti-forgery strip at 0.274, a 造点 mark at 0.253), so a provenance-relaxed arm would be mostly false fills. There is no provenance mapping for qwen and no relaxed arm. - No rival margin: 0 cross-fires on 400 Doubao-marked / 298 Jimeng-marked / 286 clean frames at the gate, while a 0.10 margin would have cost ~10% of genuine Qwen detections. A confident Qwen detection suppresses the Jimeng pill exactly like Doubao's does (
_keep_pill-- a Qwen frame is TC260 too but is not Jimeng-basic).
Parity: detect -> cv2 fill -> re-detect is clean on 83/83 real cohort marks, no empty masks (the tophat faint-mask fallback rides the same _tophat_best); the e2e suite drives one live cohort positive through the real CLI (scripts/real_examples_e2e.py, qwen bucket = symlinks under the gitignored _visible_datasets/). Regression: tests/test_qwen_engine.py (pins the ladder, the strict-only factor, the box anchor, and both size modes at a score floor that discriminates the shared-ladder and Doubao-margin mutations).
The clean-arm contamination trap (load-bearing for any future calibration): the 2026-07-18 present: [] labels are in the vocabulary of the REGISTERED marks only, so 146 of the 432 "clean" frames sit in a TC260 cohort -- including Qwen-cohort frames visibly carrying 千问AI生成. They made up the clean arm's entire top tail (clean p99 0.37 -> 0.69). vendor_mark_calibrate.load_sets now excludes every frame in ANY TC260 cohort from the clean arm; a gate read off the unguarded arm is meaningless.
runninghub_engine.py
runninghub_engine.py — thin TextMarkEngine subclass, registered 2026-07-22. RunningHub (hosted ComfyUI platform, USCC 91340100MAEB4N8H76, 73-frame cohort) "RunningHub AI生成" detector + localizer, top-left (the first corner="tl" mark), faint mid-gray latin+CJK text. Feeds identify as visible_runninghub.
grayfront-end (the third one, added for this mark): the mark's faint gray is suppressed by the white top-hat to clean-arm levels (positives 0.16-0.23 vs clean p99 0.31), while raw-grayscale silhouette NCC separates (positives 0.38-0.54 vs clean p99 0.264 / max 0.304 on 283 guarded clean frames). It is contrast-DEPENDENT, unlike tophat -- one method_gray_bestserves both detection and the mask, same one-method parity contract as_tophat_best.- Tight ladder (0.95, 1.0, 1.05) exactly on the measured 0.32-of-width: the NCC comb is razor-sharp in size (0.537 on-size, 0.223 at +5.6% off -- Qwen's comb behaviour, measured again here), so the shared 3 rungs (nearest rung 5.6% off) collapsed the match to 0.22 and the first calibration showed no separation at all.
- Anchor gate in
detect: the full-corpus sweep (data/spaces/_sweep_new_marks.py, 42009 files) fired on 37 outside-cohort frames at 0.34-0.38 (hair tops, shelves, window frames, CJK banners) -- no NCC threshold separates them from the 0.381 positives. Every positive sits at the measured corner (x 0.008-0.014, y 0.005-0.007 of the frame) and every false fire off it (x 0.013-0.150, y 0.009-0.045), so detection additionally requires the match box inside x<=0.025 / y<=0.015 of the frame. 0/37 false, 4/4 positives kept. - Footprint is always the detector's match box (never the binary blob): the blob under-segments the faint head glyphs and left "Runni" unremoved (caught visually on the first removal). Gate 0.34, STRICT ONLY. Regression:
tests/test_runninghub_engine.py.
baidu_engine.py
baidu_engine.py — thin TextMarkEngine subclass with a custom footprint_mask, registered 2026-07-22. Baidu (USCC 91110000802100433B, 16-frame cohort) "百度 AI生成" detector + localizer, bottom-right: a white bold 百度 text run + a separate white rounded tag with dark "AI生成". Feeds identify as visible_baidu.
- Detection keys on the 百度 text run ONLY. A two-component (text+pill) template was measured and rejected: the solid white pill is a bright-blob magnet and both front-ends scored the clean arm at cohort levels (tophat clean p95 0.445 / gray clean p95 0.487 vs cohort ~0.5). The text-only silhouette separates (cohort 0.39-0.65 vs clean max 0.352). The white tag is still removed: the footprint extends to the corner (below).
- Two load-bearing rival margins (
rivals=("doubao_alpha.png","qwen_alpha.png")): 百度 vs 豆包 share their second glyph, and 百度 vs 千问 are near-identical after binarization -- at the 0.37 gate the template fired on 45.8% of 400 Doubao-marked frames and, on the 741-frame blind-labelled eval set, on 12 Qwen-marked frames at 0.38-0.43. Doubao's template beats it by ~0.56 on Doubao marks, Qwen's by 0.17-0.35 on Qwen marks; the 0.10 margin suppresses all crossfire at zero genuine-Baidu cost (cohort fire+m == fire). - Gate history, each step measured: 0.37 from the clean arm (max 0.352) -> 0.43 after the eval-set crossfires (the one 抖音 AI创作 fire at 0.425 named no registered rival) -> 0.48 after the full-corpus sweep put outside-cohort TRUE carriers at 0.50-0.66 vs the false arm max 0.47 (大众点评 UI, a math blackboard, an 80s banner). Cohort keeps 7/16 (all true); the sweep also found 6 metadata-STRIPPED true Baidu carriers the TC260 cohort cannot see -- the direct evidence that registration pays beyond the cohort.
- Custom
footprint_mask: the tag's flat white interior gives no top-hat response (a top-hat answers edges, not flats), so the base blob bbox ended at the text run and the fill left the tag as a ghost. The mask is the detector's match box extended RIGHT to the corner. STRICT ONLY. Regression:tests/test_baidu_engine.py.
liblib_engine.py
liblib_engine.py — thin TextMarkEngine subclass with a custom footprint_mask, registered 2026-07-22. LibLibAI (哩布哩布AI, USCC 91110105MACJ6K1C8A, 15-frame cohort) triangle logo + "LibLibAI" wordmark detector + localizer, bottom-CENTER (the first corner="bc" mark; the locate box is horizontally centered). Feeds identify as visible_liblib.
- The discriminative lever is the silhouette FONT. With the CJK house font (STHeiti) the cohort scored 0.31-0.47 against a full-corpus false arm (latin UI text bands, website screenshots) at 0.50 -- no separation at any gate. Measured across 7 candidate fonts, Arial lifts the cohort to 0.42-0.73 and DROPS the false arm to max 0.398: generic latin text matches the wrong font less, which is where the discrimination comes from. Gate 0.42 keeps all 8 marked cohort frames (0.43-0.59).
- Per-mark size floor (
_MIN_SHORT_SIDE=480): the one false fire with the final template was a 200x200 icon (0.444, on a 20px template). The shared_MIN_DETECT_SHORT_SIDE(200) is a crash guard, not a discrimination floor; the template needs ~48px to discriminate. - Custom
footprint_mask: the base blob bbox was wrong in both directions -- it bled UP into background structure (ate a shirt's real print on the 768x1024 cohort frame) and never owned the triangle logo. The mask is the detector's match box extended LEFT by ~1.3 glyph heights (the logo is ~1.0x the glyph height, gap ~0.3x, measured on the cohort zoom). STRICT ONLY. Regression:tests/test_liblib_engine.py.
region_eraser.py
region_eraser.py — universal region eraser (erase CLI) AND the shared fill backend behind watermark_registry.fill for the visible localize -> fill removal. erase(image, boxes=|mask=, backend=) accepts grayscale (2D) and RGBA (4-channel) inputs on all backends (each splits off any alpha plane and re-attaches it unchanged, and promotes grayscale to BGR): boxes_to_mask → one of three backends.
cv2(default, no deps):cv2.inpaint.migan(extramigan,andraniksargsyan/miganONNX, MIT, ~28 MB):erase_migan. Likeerase_lama, it crops a padded region around the mask (pad = max(256, 2*bbox)), feeds only that crop to the ONNX model, and pastes only masked pixels back — but since MI-GAN accepts arbitrary dims (unlike LaMa's fixed 512² square) the crop is fed at NATIVE resolution (no resize). This bounds the ONNX working set by the mark size, not the image: feeding the whole frame made peak RAM scale with the upload (~0.6 GB at 4 MP up to ~2.4 GB at 25 MP, measured 2026-07); cropping holds it roughly constant (~0.6-0.9 GB), so a memory-tight host (a 1-2 GB web worker) can run MI-GAN on a 25 MP upload. The crop does not degrade the fill — a small mark only needs local context, and on real marks the cropped fill is on par with / sometimes cleaner than the full-frame fill (a tighter view gives the GAN less room to hallucinate large background structure; verified by eye on real Gemini/Doubao marks + a ground-truth reconstruction sweep). Mask polarity is INVERTED vs this package's 255-erase convention — the shipped ONNX wants 0=hole / 255=known, soerase_miganfeeds(crop_mask<=127)*255; feeding 255=hole regenerates the whole frame into stripes (corpus-validated 2026-07, cost hours to find). ~0.19 s. This is the preferred default fill for the visible localize -> fill path.lama(extralama,Carve/LaMa-ONNXApache-2.0, ~200 MB):erase_lamacrops a padded region around the mask, runs at LaMa's fixed 512² input, pastes only masked pixels back. Best quality but ~4.7 GB peak — explicit opt-in only, NOT auto-selected. Lazy_get_{lama,migan}_sessionsingletons;{lama,migan}_available()guard the optional imports (both == onnxruntime present). Note both extras install the same onnxruntime, so the two*_available()checks are identical — the fill'sautobackend therefore resolves to MI-GAN whenever onnxruntime is present, else cv2, and big-LaMa is reachable only by an explicitlamabackend (--backend lamaonerase, or the shared fill'sbackend="lama").
LaMa-ONNX costs ~3.5-4 GB peak RAM and ~5-6 s/call on CPU (FFC working set, not arena — enable_cpu_mem_arena=False does not help), so it does NOT fit a minimal droplet; the cv2 backend (tens of MB, ~30 ms) does. LaMa quality at low RAM = serverless/GPU, mirroring how raiw.cc offloads SDXL to fal.
invisible_watermark.py
invisible_watermark.py — detect_invisible_watermark(path) decodes the OPEN DWT-DCT watermarks (public decoder, no key) embedded by Stable Diffusion / SDXL / FLUX via the imwatermark library. Known fixed patterns (verified against upstream source) live in _BITS_48 (SDXL 48-bit, FLUX.2 48-bit) and _SD1_STRING ("StableDiffusionV1", SD 1.x/2.x). Optional dep (extra detect); returns None when absent. The detect extra pulls torch transitively (invisible-watermark declares torch a hard dep, and WatermarkDecoder eagerly imports rivaGan -> torch at import time), so detection needs torch present even though dwtDct runs CPU-only on cv2/numpy/pywavelets — no GPU and no separate gpu extra required.
Unlike SynthID this is locally detectable, but the watermark is fragile (does not survive JPEG re-encode/resize — verified gone after JPEG q90), so it confirms origin only on pristine files. Add new known patterns here. The file carries a top-of-module pyright pragma because imwatermark/cv2 ship no type stubs.
trustmark_detector.py
trustmark_detector.py — detect_trustmark(path) decodes the OPEN, keyless Adobe TrustMark watermark (the soft binding behind Adobe Durable Content Credentials, alg com.adobe.trustmark.P) via the optional trustmark package (extra trustmark; pulls torch, downloads model weights on first use). Mirrors invisible_watermark.py (lazy singleton guarded by a double-checked threading.Lock so concurrent callers do not double-download the weights, top-of-module pyright pragma, returns None when absent). It detects provenance, not AI origin as such (TrustMark also marks human-authored content), so identify lists it as a watermark without setting is_ai_generated. Other soft-binding vendors (Digimarc/Imatag/Steg.AI/...) have no public decoder — they are only named via the C2PA_SOFT_BINDINGS scan, not decoded.
False-positive gate (added 2026-05-29): TrustMark's wm_present is a BCH error-correction validity flag that spuriously validates on a content-correlated fraction of un-watermarked images — AI-generated textures trip it far more than camera photos (verified 2026-05-29 on real files: it fires on Gemini/OpenAI/Doubao output that cannot carry Adobe's watermark, with a random-bytes decoded secret, while signal-free camera photos did not trip it). A genuine TrustMark is a durable soft binding engineered to survive re-encoding, so detect_trustmark re-decodes after a mild JPEG round-trip (_survives_reencode, _REENCODE_QUALITY 95) and requires the same schema both times; every observed false positive collapsed (none survived even q95), so the gate is the durability property the watermark guarantees. The second decode runs only on the rare initial hit, so the cost is negligible. Do NOT remove the gate to "catch more" — a lone TrustMark hit without it is almost always content noise.
noai/watermark_remover.py
noai/watermark_remover.py — the WatermarkRemover class has four diffusion pipelines, selected by the explicit pipeline ctor arg (NOT inferred from model_id). sdxl/controlnet share the SDXL base (DEFAULT_MODEL_ID); qwen is its own base (QWEN_MODEL_ID); qwen-zimage delegates to the fixed two-stage stack in noai/qwen_zimage_pipeline.py.
sdxl (renamed from default 2026-06-09; default kept as a back-compat alias via normalize_profile) runs plain SDXL img2img (_run_img2img); it is the lighter opt-down alternative (no ControlNet weights).
qwen (_run_qwen, _load_qwen_pipeline) runs QwenImageImg2ImgPipeline on Qwen/Qwen-Image (20B MMDiT, Apache-2.0 code AND weights). The scrub still comes from the img2img strength; Qwen's value is text preservation (incl. CJK and small text). Metric-measured nuance (2026-06-19, scripts/fidelity_metrics.py, do NOT trust the eyeball here — it misled). Compare ONLY at each pipeline's oracle-confirmed scrub floor (outputs where SynthID is removed in BOTH — an equal-strength compare is invalid where it leaves one un-scrubbed; Qwen at 0.15 does not clear Gemini): Qwen wins TEXT (lower OCR CER across EN/RU/ZH, perfect Chinese) but controlnet wins FACES (higher Laplacian-variance retention and lower LPIPS — Qwen smooths faces MORE; ArcFace identity favors controlnet 0.546 vs 0.331 at the Gemini floors). So Qwen is the better text-preserving remover, NOT a universal fidelity win — controlnet's canny edge map holds face skin detail better. Specifics: bf16 on CUDA (fp16 risks overflow on the 20B MMDiT — see the dtype branch in __init__); loads QWEN_MODEL_ID unless --model is overridden; the call shape lives in the pure module helper _build_qwen_kwargs (unit-tested without torch in tests/test_platform.py::TestQwenKwargs), which uses Qwen's true_cfg_scale (NOT SDXL's guidance_scale — the CLI --guidance-scale maps onto it; ~4.0 is typical, the SDXL default 7.5 is high for Qwen) and an explicit negative_prompt (_QWEN_PROMPT/_QWEN_NEGATIVE). It is CUDA/cloud-class (the 20B does not fit MPS), so _run_qwen has NO MPS->CPU fallback — an error propagates. _load_qwen_pipeline raises a clear ImportError if the installed diffusers lacks QwenImageImg2ImgPipeline. CERTIFIED oracle floors (Modal A100-80GB, 2026-06-20): OpenAI 0.10 (seed-robust — clean on seeds 0-4), Gemini 0.25 (seed 0 verified on 2 images; the Gemini oracle rate-limits volume seed-repeat, so PIN a seed in prod). The Gemini floor (0.25) is HIGHER than the certified controlnet Gemini floor (0.15); resolve_strength(..., pipeline="qwen") carries the Qwen ladder (_QWEN_VENDOR_STRENGTH), so --pipeline qwen gets the 0.25 Gemini floor automatically -- the old manual --strength 0.25 workaround is retired. _build_qwen_kwargs passes an explicit height/width from the input (floored to /16 via the pure _qwen_target_size); WITHOUT it the img2img pipeline defaults to a 1024x1024 SQUARE and silently squishes non-square inputs (the abba 2816x1536 case came back 1024x1024, distorting the scene and garbling text — fixed 2026-06-20, tested in TestQwenKwargs). Fidelity vs controlnet was measured at the certified floors (scripts/fidelity_metrics.py), NOT eyeballed. qwen is a MANUAL opt-in only — there is NO auto-router (one was prototyped and DROPPED, see below). It wins ONE niche: clean body text on a plain background, NO faces (openai_1/2 CER 0.241 vs 0.385). controlnet wins FACES and display/decorative text in a scene (abba poster: controlnet CER 0.114 vs qwen 0.379 — canny holds letter shapes, qwen re-renders and garbles them). --pipeline auto + a faces+text mixed dual-pass were built and DROPPED (2026-06-20): on the canonical faces+text case controlnet wins EVERY metric incl. text, so grafting qwen text would only hurt; and "text→qwen" is undecidable cheaply (it is body-vs-display text that matters). The router/detector/mixed modules were removed; the geometry fix + the Qwen strength ladder were kept (they make the manual --pipeline qwen correct). Do NOT retry "add a Qwen ControlNet to close the face gap" — it was built, measured, and CLOSED 2026-06-20: a DiffSynth blockwise-canny Qwen ControlNet did not restore face skin texture (lapvar flat 0.40, canny carries edges not skin grain) and no permissively-licensed Qwen tile/detail/skin ControlNet exists anywhere (all conditioning is geometry). The Z-Image face-crop lead is now implemented as the separate qwen-zimage profile and has direct face metrics on two official upstream examples plus one crowded fixture. Its exact current six-output candidate is negative in the corresponding provider oracles, while broad seeded removal and text behavior remain unmeasured. Full record + the deep-research sweep in docs/qwen-improvement-research.md.
noai/qwen_zimage_pipeline.py
qwen-zimage is the recommended high-quality, manual CUDA profile ported from cebeuq/Synthid-Bypass v2. controlnet remains the default for compatibility and cost; callers that prioritize output fidelity, especially face identity, should select qwen-zimage. The full-frame stage uses DiffSynth QwenImagePipeline with Qwen/Qwen-Image-2512, lightx2v/Qwen-Image-2512-Lightning at four steps, and DiffSynth-Studio/Qwen-Image-Blockwise-ControlNet-Canny. The Lightning scheduler uses exponential_shift_mu=log(3), the DiffSynth equivalent of the source graph's AuraFlow shift 3. Its default denoise is the source custom node's exact megapixel formula at adaptive level 6; an explicit --strength overrides that global value. Its profile seed defaults to 0, matching the oracle-negative release candidate; an explicit seed still wins. Other profiles keep their existing random default.
The face stage detects boxes on the original input with OpenCV YuNet and follows the active Impact Pack SAM path from the source graph: each box supplies both the center-1 positive point and the box prompt; proposals at predicted IoU >= 0.93 are unioned, or the highest-IoU proposal is used when none passes; the result is intersected with the detector box. The inactive MediaPipe node in the workflow has no downstream link. YuNet uses its own calibrated score threshold, 0.5: copying the upstream YOLO threshold of 0.2 admitted background/decorative false positives and duplicate boxes, which multiplies the serial face-stage cost. The 0.5 gate retained all visible faces in the public and upstream comparison fixtures while reducing the crowded group from 36 boxes to 18 and the poster from 30 to 10. Crops expand by the source graph's factor 2.5 and run Tongyi-MAI/Z-Image-Turbo for eight steps. Every face uses the denoise derived from the largest face's area ratio, matching the source graph's largest_face mode, then pastes through the clipped SAM mask with feather 10. If SAM fails, a box-derived ellipse mask is used rather than aborting the global removal.
The active graph was traced from upstream commit 3007d0351596ae0a78b7074dae7ad179710b1e48 and its linked Impact Pack implementation, not inferred from the README or node names. YuNet is one intentional substitution: the active reference path uses an Ultralytics YOLO face detector, while this package avoids adding its AGPL runtime. The first-use model download targets GitHub's media endpoint rather than the repository's 131-byte Git LFS pointer and verifies the published 232589-byte model by SHA-256 before caching it. The other runtime differences are full safetensors rather than quantized GGUF models, DiffSynth's first-order Qwen Lightning and Z-Image FlowMatch samplers rather than the graph's DPM++ 2M / SGM Uniform and res_2s / bong_tangent pairs, and the absence of the detailer's 20 px latent noise-mask feather. This port regenerates the expanded crop and composites only the feathered SAM mask; generated pixels outside the face mask are discarded. The architecture and active decision path match the graph, but the runtime is not bit-identical to ComfyUI.
The implementation has its own qwen-zimage optional dependency group because DiffSynth, torchvision, and the additional model downloads are large. --model is rejected for this fixed profile. --tile runs only the global Qwen stage through noai.tiling.run_tiled; the global denoise is still derived from the full-frame megapixel count and the same seed is reused for each deterministic tile. After feather blending, YuNet, SAM, and Z-Image run once against the full original/global result, so faces are neither duplicated nor dropped at tile boundaries. The SDXL minimum-resolution floor is disabled, and CLI adaptive polish defaults off for this profile, so the two-stage result is not followed by a repository-specific post-process. DiffSynth requires the PIL input, Canny control, and explicit dimensions to agree on the same /16 latent grid: _resize_to_target aligns global and face pixels before build_global_kwargs / build_face_kwargs, and the global output is restored to the exact original size. Passing floored dimensions with unaligned pixels caused a real VAE/noise-grid shape mismatch on the official example 12 input; the call-shape assertions were observed failing before the fix. An explicit --adaptive-polish still opts in. Pure helpers cover both adaptive denoise formulas, /16 dimensions, call shapes, Canny generation, masked compositing, and the qwen-zimage tiling seam; an integration test guards dispatch from WatermarkRemover. A real 4096x3072, 20-tile H100 smoke completed through this exact branch on 2026-07-25 with dimensions preserved and no visible or 99th-percentile gradient outlier at a tile boundary; the measured runtime, memory, and fidelity figures are in docs/known-limitations.md. The exact seed-0 non-tiled release candidate is oracle-verified; tiled outputs still require their own provider-oracle check.
DiffSynth normally offloads the Z-Image text encoder, DiT, and VAE to CPU after every face call. That placement dominated crowded-scene latency even though the eight diffusion steps themselves were fast. resolve_face_model_residency keeps the full face stack on CUDA when total VRAM is at least 64 GiB; smaller cards preserve the original offload path. Callers can explicitly override the decision through QwenZImagePipeline.keep_face_models_on_device. The implementation intentionally loads the stack with the normal CPU-managed config, rewrites the managed modules' offload/onload/preparing placement to their CUDA computation device, and moves them once. Loading the same models directly into CUDA cut face inference further but increased setup from 32.282 to 254.632 seconds, making a cold single request more expensive; that variant was rejected. The shipped fast-load residency changes only model placement, not weights, dtypes, prompts, seeds, schedules, masks, or compositing. On the 18-face H100 fixture it produced a pixel-identical output versus offload while reducing face regeneration from 181.764 to 38.272 seconds and total inference from 262.072 to 133.543 seconds. Setup rose only from 32.282 to 43.960 seconds, so cold setup plus inference fell from 294.354 to 177.503 seconds. Peak CUDA allocation rose from 24.364 to 43.477 GiB.
Both stage prompts are constants, but DiffSynth 2.0.18 exposes their embeddings only through internal PipelineUnits and re-runs the corresponding text encoder on every call. _cache_static_prompt_embeddings wraps the exact prompt unit selected by its output signature and memoizes its returned tensors by prompt text. It bypasses the cache whenever edit_image participates, so image-conditioned embeddings cannot be reused accidentally. With CFG 1.0, the unit runner already shares the positive result with the negative branch. On a warm H100 sequence, a preceding no-face request populated the Qwen prompt cache; the 18-face case then fell from 133.543 to 78.474 seconds. The global stage fell from 89.720 to 43.334 seconds, and the face stage from 38.272 to 30.592 seconds as the fixed Z-Image prompt was encoded only once. Both the no-face and 18-face cached outputs were pixel-identical to their no-cache references, and peak allocation remained 43.477 GiB. The Qwen saving applies from the second request in a container; the Z-Image saving applies within the first multi-face request after its first face.
Validation status is deliberately narrower than the existing qwen certification. The user reported the upstream workflow as Gemini-oracle negative. The current port has API, unit, dispatch, GPU integration runs, and a direct comparison with two official upstream before/after pairs. On 2026-07-25 the user checked all six current outputs in the provider-separated full-clean-final-candidate-2026-07-25-by-oracle bundle with the corresponding provider oracles and confirmed that none retained SynthID or the provider generation signal. The checked bytes used the complete visible -> qwen-zimage -> metadata route, the calibrated YuNet 0.5 gate, and the shipped prompt-cache/model-residency optimizations. This supersedes the earlier first-port batch check as the release-candidate result. It certifies the exact seed-0 outputs, not every seed, resolution, or content class. Broad text certification remains open, so the profile stays an experimental manual opt-in even though it is the recommended quality mode.
Direct port measurement (2026-07-24, seed 0): on official upstream examples 10 and 12, local qwen-zimage retained ArcFace identity at 0.950 and 0.947 versus 0.701 and 0.548 for the current polished ControlNet output. Face LPIPS was 0.045 and 0.015 versus ControlNet's 0.105 and 0.061. The published upstream outputs retained identity at 0.976 on both and face LPIPS at 0.172 and 0.014; upstream example 10 is strongly penalized by its published downscale, while example 12 was compared at the same published dimensions. Local whole-image LPIPS / SSIM were 0.167 / 0.765 and 0.085 / 0.896, better than the published upstream 0.259 / 0.627 and 0.111 / 0.777. ControlNet still preserved more texture, but the faces drifted. The earlier crowded gemini_3 run showed the same identity direction, 0.795 versus 0.587/0.588, while smoothing skin and changing the full frame more. The final July 25 oracle check supplies the removal verdict for the exact current candidate bytes only.
Two integration failures from the first Modal passes are regression-guarded. SAM model pixels must be cast to the model's bfloat16 while geometric prompts remain float32; casting everything either fails or changes prompt semantics. SAM pred_masks and iou_scores must then be converted through float32 before NumPy because NumPy rejects bfloat16. A third visually severe failure came from accepting an unconstrained SAM mask: it split faces with hard seams. The center point plus box prompt and the final detector-box intersection are both load-bearing.
controlnet (the DEFAULT pipeline since 2026-06-09 for invisible/all/batch and both engine ctors; _run_controlnet, _load_controlnet_pipeline) runs StableDiffusionXLControlNetImg2ImgPipeline with the SDXL-native canny ControlNet xinsir/controlnet-canny-sdxl-1.0 (watermark_profiles.CONTROLNET_CANNY_MODEL): the control image is cv2.Canny(gray, 100, 200) stacked to 3 channels (_CANNY_LOW/_CANNY_HIGH, prompt _CONTROLNET_PROMPT / _CONTROLNET_NEGATIVE).
Removal comes from the img2img regeneration (strength); the ControlNet only PRESERVES text and face STRUCTURE via the edge map.
No original pixels are copied or frozen, BUT validation 2026-06-04 disproved the old "so SynthID does not survive" claim: SynthID CAN survive controlnet on photoreal/high-detail content.
At the shared low removal strength the canny edge-conditioning keeps the regeneration so close to the original that the pixel perturbation that destroys SynthID does not happen (oracle-confirmed: an OpenAI bracelet photo + a 9-face grid read SynthID-detected after controlnet at strength 0.10/0.15, but SynthID-not-detected after the default pipeline at the SAME strength + resolution -- only the pipeline differed).
But the reverse also holds: a flat-graphic logo/poster SURVIVED default while clearing controlnet -- removal at the low strength is content×pipeline dependent and neither pipeline is universally safe; the real lever is a higher strength. See the controlnet Known-limitations bullet for the full table + root cause. Canny holds face STRUCTURE but NOT identity (the regenerated face drifts in likeness -- canny carries edges, not identity). The drifted cleaned face is the LEAST-AI state we can reach without re-introducing SynthID; the library does NOT ship a face-restore extra. Every restore approach we evaluated (GFPGAN-on-cleaned, PhotoMaker-V2 txt2img, InstantID txt2img, InstantID img2img-on-cleaned at three parameter sweeps, 2026-06-04 - 2026-06-08 Modal cert sweeps) regenerated the face from an ArcFace embedding via SDXL diffusion -- which makes the output face look MORE AI-generated, not less. Empirical conclusion in docs/synthid-robust-identity-research-2026-06-08.md "Empirical follow-up". For production face preservation, ship the cleaned image as-is. controlnet_conditioning_scale (ctor arg, default 1.0) is the structure-preservation knob. Same dtype rule as default (fp32 on cpu/mps, fp16 only on cuda/xpu; the fp16-fixed SDXL VAE _SDXL_FP16_VAE_ID is swapped in on fp16 GPUs -- issue #29) and the same MPS->CPU fallback (reload on cpu/fp32, drop a non-cpu generator, retry once).
Tiled diffusion (tile/tile_size/tile_overlap ctor-path args, CLI --tile, issue #10): for large inputs that OOM at native resolution, remove_watermark can process the diffusion pass in overlapping sliding-window tiles instead of one forward pass — the lossless alternative to a --max-resolution downscale. For SDXL, ControlNet, and base Qwen, the single-image generation closure was refactored into _generate_one(img) (dispatches controlnet/img2img, generator shared so the seed advances deterministically across tiles), and _generate() routes it through noai.tiling.run_tiled when tile is set AND max(init_image.size) > tile_size (a sub-tile image runs one pass unchanged). The ControlNet canny edge map is rebuilt per tile inside _generate_one, so structure preservation is tile-local. qwen-zimage takes a different route: _generate() dispatches once to QwenZImagePipeline.run, that runtime tiles only _run_global, then performs one full-frame face stage after blending. See noai/tiling.py below and the tiled-diffusion subsection in docs/known-limitations.md for the geometry, the partition-of-unity blend, and the quality caveat.
noai/tiling.py
Pure sliding-window tiling for the diffusion path (no torch import; numpy/PIL only). plan_tiles(w, h, tile_size, overlap) returns a row-major grid of uniform-size Tile boxes — every tile is exactly tile_size, with the last tile on each axis pulled back flush to the far edge (_axis_positions clamps a pathological overlap >= tile to tile - 1 so the step stays >= 1). feather_weights(w, h, overlap) is a separable linear taper (1 in the interior, ramping toward each edge) floored at _WEIGHT_EPS so it is strictly positive everywhere — that makes the normalized accum / weight_sum blend a partition of unity, so identical/unchanged tiles reconstruct the input exactly (the seam-free guarantee). run_tiled(generate_tile, image, tile_size, overlap, set_progress) is the orchestration loop: crop each planned tile, call generate_tile (one diffusion pass on a single PIL tile — injected, so this stays decoupled from the pipeline), resize a latent-grid-rounded result back to the exact tile size, and feather-accumulate. All three are unit-tested without the model (tests/test_tiling.py: axis math, grid coverage, taper shape/symmetry/positivity, identity reconstruction, per-tile call count, and the resize-back path). New blend tuning belongs in these pure helpers, not inlined into the runner.
feather_region_composite(base, regenerated, box, *, feather) is the pure region-targeted compositor for AI-enhanced composites (roadmap P1#8; identify ai_source_kind == "enhanced", digitalSourceType compositeWithTrainedAlgorithmicMedia). It blends regenerated over base inside box = (x, y, w, h) with a separable linear taper of feather px at the box edges (the taper anchors to ~0 at the boundary, so unlike feather_weights it is NOT floored — the result equals base EXACTLY outside the box), preserving dtype and supporting HxW or HxWxC. It backs WatermarkRemover.remove_watermark(region=..., region_feather=...): the remover regenerates the frame (or tiles), then composites only the AI box back over the original input, so the real photo outside the box stays pixel-exact and only the AI region is scrubbed. The box is caller-supplied (a C2PA composite manifest carries no reliable machine-readable region); the no-model lossless region path remains region_eraser.erase. Unit-tested in tests/test_tiling.py::TestFeatherRegionComposite (outside-box exactness, interior == regenerated, hard-paste at feather 0, monotonic seam ramp, dtype/grayscale/clamp/empty-box/shape-mismatch).
auto_config.py (REMOVED 2026-06-09)
auto_config.py + the content-detection layer were REMOVED 2026-06-09.
History: auto_config.plan() was a content-adaptive planner that detected faces/text/edges (bundled OpenCV YuNet + PP-OCRv3 DBNet models) to route the pipeline and toggle the adaptive polish. Once controlnet became the default-and-only auto pipeline (it no longer downgrades a structure-less image to sdxl) and the adaptive polish was confirmed to self-gate by detail level (humanizer.adaptive_polish no-ops when the cleaned image already meets the input's Laplacian variance, so it does real work only on over-smoothed photo/face texture and ~nothing on text/flat), the detection no longer changed any behavior — it only annotated a reason string. So the whole layer was deleted: auto_config.py, tests/test_auto_config.py, and the two detection assets (assets/face_detection_yunet_2023mar.onnx, assets/text_detection_ppocrv3_2023may.onnx, ~2.6 MB).
--auto is now a DEPRECATED no-op (cli._resolve_auto_polish): controlnet is already the default pipeline AND the adaptive polish is ON by default, so --auto has nothing left to do — it only prints a deprecation warning and passes adaptive_polish through unchanged (an explicit --no-adaptive-polish still wins). (Originally it re-enabled the polish; once the polish default flipped to ON the same day, the parameter-source branch became dead and was dropped.) The adaptive polish itself lives on in humanizer.adaptive_polish (CLI --adaptive-polish/--no-adaptive-polish, ON by default since 2026-06-09 for the original profiles — it self-gates to a no-op where there is no detail deficit; qwen-zimage defaults it off to preserve its upstream-matching output, and an explicit flag overrides either default) — see the humanizer test note. batch resolves the polish once before the loop (one warning) and caches the invisible engine per pipeline (ctx.obj["_inv_engines"]).
Content --pipeline auto router + faces+text mixed dual-pass — PROTOTYPED and DROPPED (2026-06-20)
A --pipeline auto content router (pipeline_router.py + content_detect.py: Haar faces + MSER text → route text→qwen / faces→controlnet / both→mixed) and a faces+text mixed dual-pass (mixed_pipeline.py: scrub the whole frame on BOTH pipelines, then graft the qwen text regions onto the controlnet base via tiling.feather_region_composite) were built, run on Modal (the abba poster: faces + display text), measured, and removed. Why it failed:
- On the canonical faces+text image controlnet wins EVERY metric, including text (CER 0.114 vs qwen 0.379; ID 0.64 vs 0.36; lapvar 0.71 vs 0.59) — canny holds the existing letter shapes, qwen re-renders display/decorative text and garbles it. So grafting qwen text onto the controlnet base only HURTS.
- qwen beats controlnet on text ONLY for clean body text on a plain background with no faces (openai_1/2) — a niche where there are no faces to route around anyway, so
--pipeline qwenalone covers it. The faces+clean-body-text intersection is near-empty. - "text→qwen" is not cheaply decidable: it is body-vs-display text that matters, which face/text detectors can't tell apart. MSER also over-fired (47% of the busy poster, incl. faces).
KEPT from that work (independently valid for the manual --pipeline qwen): the qwen geometry fix (_qwen_target_size + _build_qwen_kwargs height/width — qwen squished non-square inputs to 1024² without it) and the pipeline-aware resolve_strength Qwen ladder (Gemini 0.25). Also kept: the fidelity_metrics.py one-to-one face matcher. The throwaway Modal eval scripts were removed after the run (findings recorded here and in docs/qwen-improvement-research.md).
upscaler.py
upscaler.py — optional Real-ESRGAN pre-diffusion super-resolution for small inputs (spandrel boundary, top-of-file pyright pragma). is_available() gates on spandrel+torch (via importlib.util.find_spec); upscale(bgr, device=None) loads a lazily-built spandrel ImageModelDescriptor singleton (double-checked lock) and upscales by the model's native factor (x2), with a non-CPU→CPU device fallback mirroring the diffusion engine's MPS→CPU retry. Weights (RealESRGAN_x2plus.pth, BSD-3-Clause) download on first use to the torch.hub checkpoints cache; never bundled. Used only when UPscaling to the min_resolution floor (a max_resolution downscale always uses Lanczos). The wiring is InvisibleEngine._esrgan_upscale(pil, target) — Real-ESRGAN at native factor, then a Lanczos resize to the exact target, falling back to a plain Lanczos resize if the extra is absent or the model errors (so an optional upscaler can never break removal). The default --upscaler is lanczos (cv2, no deps).
ESRGAN is a generic photo/texture GAN with no face/glyph prior, so it best fits photo/texture content and can degrade faces (glassy/asymmetric eyes -- the diffusion pass regenerates faces so the full-pipeline final recovers) and thin/small text (the GAN invents wrong strokes, and low-strength diffusion will not fix it). Verified 2026-06-04: isolated upscale lap-var ~5x Lanczos on faces+textures but glassy eyes; end-to-end invisible final lap-var 1634 vs Lanczos 663 with natural faces (diffusion cleaned the artifact). Kept a manual opt-in knob (the auto plan never selects it) with lanczos the default; not content-gated by design (use Lanczos for text-heavy inputs). spandrel is MIT and pulls no basicsr. Unit-tested without the model: tests/test_upscaler.py (availability guard + the not-installed RuntimeError) and tests/test_invisible_engine.py::TestEsrganUpscale (the three _esrgan_upscale branches via a monkeypatched upscaler).
image_io.py
image_io.py — Unicode-safe cv2 IO (issue #17). imread(path, flags=None) / imwrite(path, img) wrap np.fromfile+cv2.imdecode / cv2.imencode+tofile so non-ASCII paths work on Windows -- bare cv2.imread/cv2.imwrite use the platform ANSI code-page API there and fail (empty decode + can't open/read file) on Chinese/Cyrillic/accented filenames. imread keeps cv2.imread semantics (defaults to IMREAD_COLOR, returns None on missing/empty/undecodable).
Every cv2 file read/write in the package routes through here; do not call cv2.imread/cv2.imwrite directly.
imwrite returns False on an unwritable path (OSError caught) instead of raising, matching cv2.imwrite semantics. macOS/Linux already accept UTF-8 paths, so it is behavior-neutral there (the bug only reproduces on Windows).
to_bgr(image) (added 2026-06-09) is the shared channel normalizer: promotes 2D grayscale / (h,w,1) / 4-channel BGRA to 3-channel BGR (a 3-channel input is returned unchanged, no copy). Use it instead of inlining the cvtColor(GRAY2BGR/BGRA2BGR) branch — the gemini engine and the TextMarkEngine base both route through it so a grayscale/BGRA input (a real Gemini-app export is opaque RGBA) does not crash the axis=2 channel reductions. cv2/numpy are imported lazily inside the functions, so the module is cheap to import in a bare env.
CLI commands (cli.py)
Full per-command behavior for the skip/exit branches summarized in CLAUDE.md's "How to run". The CLI distinguishes three exit codes: success (0), hard error (1), and a "nothing to do" code (2, EXIT_NO_VISIBLE_MARK / EXIT_NO_INVISIBLE_SIGNAL) so a wrapping service (raiw.cc) can surface guidance instead of treating an unchanged image as done (the production "it didn't work" / score-0 trap).
Every single-image command's source argument declares dir_okay=False (2026-07-20). click.Path(exists=True) accepts a directory unless told otherwise, so identify <dir> sailed past argument parsing and raised IsADirectoryError out of metadata.scan_head's open() — a traceback, not a usage error. Refusing it at the argument layer is the right place: every command gets it, and none needs its own check. (batch's directory argument was already correct with file_okay=False.) Found by the Tier E adversarial sweep; regression: tests/test_cli_robustness.py::TestDirectoryInputIsRejected.
all
Full pipeline (visible + invisible + metadata). Same diffusion knobs as invisible, plus the visible-pass --backend auto|cv2|migan|lama (default auto) that picks the fill for the localize -> fill visible removal. When the [gpu] extra is absent, step 2 (invisible/SynthID) is skipped — all still writes an output (visible mark + metadata stripped) but prints a prominent end-of-run banner ("the invisible (SynthID) watermark was NOT removed") AND exits non-zero (1), so a skipped SynthID pass is not mistaken for a clean result (the recurring #14/#47 trap, where the old quiet inline warning was missed). invisible already hard-errors without the extra; only all continued, hence the loud end-banner. Regression-guarded by tests/test_cli.py::TestAllCommand::test_all_loud_warning_and_nonzero_exit_when_gpu_missing. No-signal skip (P0#5): step 2 also runs the same has_invisible_target gate (see invisible below) — when no invisible watermark is detectable and --force is not set, step 2 is skipped and the pixels are left intact, but unlike the GPU-missing skip this is a SUCCESS (exit 0): the visible pass + metadata strip still ran and a file is written (the message says so without claiming the image is clean). Distinct exit semantics by design: GPU-missing = couldn't do the work (non-zero); no-signal = nothing to do (zero). Regression-guarded by test_all_skips_invisible_on_no_signal_but_succeeds. Test trap: any all test that exercises the full pipeline MUST patch("remove_ai_watermarks.invisible_engine.is_available", return_value=True) — CI installs core+dev only (no [gpu]), so an unpatched all test takes the skip branch and now hits the non-zero exit. This passed locally (gpu present → is_available() True) but red-failed every matrix cell on the v0.11.0 commit (test_all_basic/test_all_visible_step_uses_registry asserted exit 0); both now patch is_available True.
invisible
Diffusion SynthID removal. The --tile/--no-tile knob is the lossless alternative to a --max-resolution downscale for large inputs that OOM on MPS/GPU: it engages only when the long side exceeds --tile-size (default 1024); tiles are feather-blended over --tile-overlap px (default 128); pair with --max-resolution 0. --adaptive-polish is a detail-targeted polish that self-gates to a no-op where there is no deficit; it defaults off only on qwen-zimage. --auto is deprecated and now a no-op that only warns. No-signal skip (P0#5, roadmap): before the diffusion runs, the command checks identify.has_invisible_target(source) (the ProvenanceReport.ai_from_metadata union: C2PA AI-issuer / SynthID proxy, IPTC, AIGC, local gen params, EXIF/xAI, open DWT-DCT / TrustMark — visible marks do NOT count, they are a separate pass). When nothing is locally detectable it does NOT regenerate (that would only degrade a clean image — the dominant paid score-0 cause on no-watermark uploads): it writes NO output, prints guidance that does NOT claim the image is clean (a pixel SynthID is undetectable once its metadata proxy is gone), and exits EXIT_NO_INVISIBLE_SIGNAL (2) — same value/role as the visible EXIT_NO_VISIBLE_MARK. --force/--no-force (default skip = ON) runs the scrub regardless. The check fails SAFE (a detector exception → run, since leaving a watermark on a paid removal is worse than over-regenerating). Helpers cli._no_invisible_signal_exit + identify.has_invisible_target; regression-guarded by tests/test_cli.py::TestInvisibleCommand::{test_invisible_no_signal_skips_and_exits_two,test_invisible_force_runs_scrub_on_no_signal,test_invisible_runs_without_force_when_signal_present} and tests/test_identify.py::TestHasInvisibleTargetFailSafe. Test trap: any invisible/all/batch test that exercises the diffusion path on a signal-LESS fixture (e.g. the synthetic sample_png) MUST pass --force, or the new gate skips step 2 (so mock_engine.remove_watermark is never called / invisible exits 2).
visible
Known-visible-mark removal by localize -> fill: each detected mark is localized to a binary full-frame footprint mask, then one shared, swappable fill inpaints that mask. --backend auto|cv2|migan|lama (default auto) picks the fill: cv2 (classical inpaint, no deps, the floor), migan (MI-GAN ONNX, the memory-tight pick where LaMa will not fit), lama (big-LaMa ONNX, best quality, heavier, auto-preferred when a learned backend is available); auto = LaMa > MI-GAN > cv2, best available (LaMa is auto-preferred when a learned backend is present; a memory-tight deploy pins migan). --sensitivity auto|strict|assume-ai (default auto) controls how hard a borderline mark is trusted (see the registry section: the visual detectors are metadata-independent; auto relaxes a mark only on same-product evidence, assume-ai relaxes every mark on the caller's AI assertion, subject to the assumed-trust confidence floor where the vendor is unconfirmed — the only path to higher recall on a metadata-stripped screenshot). --backend and --sensitivity are shared across visible/all/batch. Detection keys on each mark's own shape, and under auto the trust gate is relaxed when local metadata confirms the vendor (a Google/Gemini C2PA issuer relaxes gemini, a China-AIGC label relaxes doubao/jimeng, samsung_genai relaxes samsung), so a moved or re-rendered mark is still caught. --mark auto (default) removes EVERY detected mark in one pass (registry.remove_auto_marks, not the single strongest -- a Jimeng-basic image carries both the top-left pill and the bottom-right wordmark) from: the Gemini sparkle, the Doubao "豆包AI生成" text strip, the Jimeng "★ 即梦AI" wordmark, the Qwen "千问AI生成" text strip, the Samsung Galaxy AI "✦ Contenuti generati dall'AI" strip (bottom-LEFT, Italian-locale detection), and the capture-less Jimeng "AI生成" pill (top-left, pill_engine). The pill's weak edge-NCC detector is gated in remove_auto_marks via _keep_pill (32k real-upload corpus validation 2026-07): never on Doubao or Qwen, and two confirmation arms since metadata confirms the platform, not pill presence. (1) The bottom-right wordmark fired — ~94% precise and survives metadata-STRIPPED uploads (screenshots / re-saves) — removes the pill unrestricted. (2) TC260 metadata confirms Jimeng ("jimeng" in provenance, from cli._visible_provenance) OR the caller asserts AI (sensitivity == "assume_ai"), no wordmark — re-measured 2026-07-18 on 149 blind-labelled pill fires: 21% precise raw (CI 16-29%), 29% (CI 20-40%) among the flat footprints the guard actually PASSES, 14% among those it blocks — its false fires are textured ceilings/walls that the fill visibly SMEARS — removes the pill ONLY when the top-left footprint is flat enough for an invisible fill (pill_engine.footprint_is_flat, median-Sobel ≤ _FLAT_TEXTURE_MAX; the flatness guard holds even under assume_ai). No confirmation → never removed. --mark gemini|doubao|jimeng|qwen|samsung|jimeng_pill forces one (choices come from the registry). Corpus validation: doubao and jimeng localize + remove at ~100% with clean footprints (the filled region blends into its surroundings within a few LAB levels, no color shift, no dark pit); clean images with no vendor signature had 0% false removal. For arbitrary logos/objects use erase. When --mark auto finds no known mark (the common case — ~74% of real uploads carry no registered visible mark), the command does NOT silently re-serve the input as a finished result. It runs a cheap metadata-only identify, prints actionable guidance (if the image carries an invisible/metadata mark, e.g. an OpenAI/Gemini C2PA image, it points to all; otherwise it does NOT imply the image is clean -- it warns that an invisible pixel watermark like SynthID cannot be detected once the metadata proxy is gone and routes to both all and erase --region), writes NO output file, and exits EXIT_NO_VISIBLE_MARK (2) — distinct from success (0) and a hard error (1) so a wrapping service (raiw.cc) can surface the message instead of treating the unchanged image as done (the production "it didn't work" / score-0 trap). Same handling for an explicit --mark <name> that is not detected. Helper cli._no_visible_mark_exit; regression-guarded by tests/test_cli.py::TestVisibleCommand::test_visible_auto_no_mark_exits_two_with_eraser_hint and test_visible_auto_no_mark_routes_to_all_when_metadata. --no-detect still forces the gemini fallback and proceeds (exit 0).
batch
Process every supported image in a directory (output defaults to <directory>_clean/, set with -o). --mode visible|invisible|metadata|all (default visible); the invisible/all path reuses the full invisible knob set (--strength/--steps/--guidance-scale/--pipeline/--controlnet-scale/--model/--device/--max-resolution/--min-resolution/--upscaler/--seed/--hf-token/--humanize/--unsharp/--adaptive-polish/--tile/--tile-size/--tile-overlap/--force), plus --backend for the visible localize -> fill pass. --adaptive-polish is ON by default except on qwen-zimage; --auto is deprecated and a no-op that only warns. No-signal skip (P0#5): in invisible/all mode each image runs the same has_invisible_target gate — a signal-less image is skipped (no diffusion); in invisible mode the input is copied through to the output dir so it stays complete, in all mode the visible-removed result is kept and metadata is still stripped. --force scrubs every image regardless. One engine cached per pipeline; the polish is resolved once before the loop. Exit code (batch used to always exit 0, hiding failures): cmd_batch raises SystemExit(1) when any image errored, OR when a --mode invisible/all image carried an invisible signal but the GPU extra was absent so its SynthID scrub was skipped — mirroring single all, it emits a loud "the invisible watermark was NOT removed on N image(s)" warning and (invisible mode) copies the input through so the output dir stays complete, rather than silently dropping the signal-bearing files that most needed processing. _process_batch_image returns that skipped-scrub flag; the loop tallies it. Regression-guarded by tests/test_cli.py::TestBatchCommand::{test_batch_errors_exit_nonzero, test_batch_invisible_gpu_missing_writes_output_and_exits_nonzero}.